Control of uncertain systems--modelling, approximation, and design: a workshop on the occasion of Keith Glover's 60th birthday

Lecture Notes in Control and Information Sciences Editors: M. Thoma · M. Morari 329 B. A. Francis M. C. Smith J. C. W...

Author: Bruce A. Francis | Malcolm C. Smith | Jan C. Willems

250 downloads 924 Views 10MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Lecture Notes in Control and Information Sciences Editors: M. Thoma · M. Morari

329

B. A. Francis M. C. Smith J. C. Willems (Eds.)

Control of Uncertain Systems: Modelling, Approximation, and Design A Workshop on the Occasion of Keith Glover's 60th Birthday With 109 Figures

Series Advisory Board

F. Allg¨ower · P. Fleming · P. Kokotovic · A.B. Kurzhanski · H. Kwakernaak · A. Rantzer · J.N. Tsitsiklis

Editors Prof. Bruce A. Francis

Prof. Malcolm C. Smith

University of Toronto Department of Electrical and Computer Engineering King’s College Road 10 M5S 3G4 Toronto Canada

Cambridge University Department of Engineering Trumpington Street CB2 1PZ Cambridge United Kingdom

Prof. Jan C. Willems Katholieke Universiteit Leuven Departement Elektrotechniek (ESAT) SISTA/COSIC Division Kasteelpark Arenberg 10 3001 Leuven-Heverlee Belgium ISSN 0170-8643 ISBN-10 3-540-31754-6 Springer Berlin Heidelberg New York ISBN-13 978-3-540-31754-8 Springer Berlin Heidelberg New York Library of Congress Control Number: 2006920069 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, speciﬁcally the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microﬁlm or in other ways, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under German Copyright Law. Springer is a part of Springer Science+Business Media springer.com © Springer-Verlag Berlin Heidelberg 2006 Printed in Germany The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a speciﬁc statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Data conversion by editors. Final processing by PTP-Berlin Protago-TEX-Production GmbH, Germany Cover-Design: design & production GmbH, Heidelberg Printed on acid-free paper 89/3141/Yu - 5 4 3 2 1 0

This Festschrift contains a collection of articles by friends, co-authors, colleagues, and former Ph.D. students of Keith Glover. They are a tribute to his seminal work, which has been a source of inspiration to all. The editors are grateful to all authors for their eﬀorts in writing their articles and for meeting the time schedule. They also thank the referees for reviewing the draft manuscripts, and the publisher Springer Verlag for accepting this book as a volume in the Lecture Notes in Control and Information Sciences and for making this book available in time for the workshop Control of Uncertain Systems: Modelling, Approximation, and Design. This workshop to celebrate Keith Glover’s sixtieth birthday takes place on April 21 and 22, 2006, at St John’s and Gonville and Caius Colleges of the University of Cambridge. The generous ﬁnancial support from the Engineering and Physical Sciences Research Council (EPSRC, UK) and from the MathWorks is gratefully acknowledged. Last but not least, the editors would like to thank Malcolm Smith from the University of Cambridge, the General Chair of the workshop, for organizing the event surrounding Keith Glover’s sixtieth birthday, and for taking care of all the practical details.

November 2005

Bruce Francis and Jan Willems

To Keith Glover on the occasion of his sixtieth birthday

CV — Keith Glover Keith Glover was born on April 23, 1946, in Bromley, Kent, UK. He studied electrical engineering at Imperial College in London and graduated in June, 1967. He worked as an engineer in the communications division for the Marconi Company from 1967 until 1969. From 1969 until 1973 he was a Kennedy Scholar at the Massachusetts Institute of Technology, where he received his Ph.D. degree in 1973 with a dissertation entitled Structural Aspects of System Identiﬁcation. After completing his Ph.D., he was an assistant professor with the Department of Electrical Engineering of the University of Southern California. In 1976, he joined the Department of Engineering of the University of Cambridge, where he has been since, with a one year leave of absence in 1983–84 as a Visiting Fellow at the Australian National University in Canberra. At Cambridge, Keith Glover rose through the ranks: University Lecturer from 1976 to 1987, Reader in Control Engineering in 1987, and Professor of Engineering since 1989. He has been Head of the Information Engineering Division from 1993 until 2002, and since 2002, he is Head of the Department of Engineering. Keith is a fellow of Sidney Sussex College. His scientiﬁc work spans a wide variety of topics, with as main themes: system identiﬁcation, model reduction and approximation, robust controller synthesis, and control of aircrafts and engines. He is the author and co-author of two very highly cited articles, both among the most inﬂuential papers in the ﬁeld of systems and control in the 20-th century. His 1984 article in the International Journal on Control on Hankel norm and balanced model reduction marked the beginning of a systematic study on model simpliﬁcation of dynamical systems. The four-author paper (by J. C. Doyle, K. Glover, P. P. Khargonekar, and B. A. Francis) in the 1989 IEEE Transactions on Automatic Control revealed the role of the Riccati equation in H∞ -control. These and related results were incorporated in a successful software package, the µ-Analysis and Synthesis Toolbox (for use with Matlab), distributed by The Mathworks Inc. and written by G. J. Balas, J. C. Doyle, K. Glover, A. Packard, and R. Smith. This work has been honored by many awards and several best paper prizes, among these the IEEE W. R. G. Baker Award for the most outstanding paper reporting original work in the Transactions, Journals and Magazines of the Societies or in the Proceedings of the IEEE in 1991, for the four-author paper mentioned earlier. Keith Glover was elected a Fellow of the Royal Society (1993), of the IEEE (1993), of the Institute of Measurement and Control (1999), and of the Royal Academy of Engineering (2000). In 1998 he was awarded the Sir Harold Hartley Medal by the Institute of Measurement and Control. He was the recipient of the 2001 IEEE Control Systems Award. Keith Glover married Liz (n´ee Jean Elizabeth Priestley) on New Year’s Day 1970. They live in Cambridge, UK, and have two children, Sarah and William.

VIII

Curriculum Vitae

Research students supervised 1. M. de Glas, A Theory of Fuzzy Systems, 1981. 2. M. J. Saxon, Digital Control Strategies for an Industrial Batch Process, 1982. 3. G. O. Corrˆea, Identiﬁable Structures in Linear System Estimation, 1983. 4. P. L. Mills, The Design of Systolic Arrays for Finite Numerical Functions, 1984. 5. G. A. Bundell, Robust Decentralised Adaptive Control in the Manipulator Control Problem, 1985. 6. J. Lam, Model Reduction of Delay Systems, 1988. 7. D. C. McFarlane, Robust Controller Design using Normalized Co-prime Factor Plant Descriptions, 1988. 8. D. Mustafa, Minimum Entropy H∞ Control, 1989. 9. P. A. Iglesias, Robust and Adaptive Control for Discrete-Time Systems, 1991. 10. R. A. Hyde, The Application of Robust Control to VSTOL Aircraft, 1991. 11. J. A. Sefton, A Geometrical Approach to Feedback Stability, 1991. 12. F. S. T. van Diggelen, Hadamard Weighting in Robust Control, 1992. 13. G. Vinnicombe, Measuring Robustness of Feedback Systems, 1992. 14. G. E. Dullerud, Control of Uncertain Sampled-Data Systems, 1994. 15. R. A. Davis, Model Validation for Robust Control, 1995. 16. P. J. Goddard, Performance-preserving Controller Approximation, 1995. 17. S. Lall, Robust Control Synthesis in the Time Domain, 1995. 18. Y. C. Chu, Control of Systems with Repeated Scalar Nonlinearities, 1996. 19. M. Brandstetter, Robust Air-Fuel Ratio Control for Combustion Engines, 1996. 20. M. W. Cantoni, Linear Periodic Systems: Robustness Analysis and Sampleddata Control, 1998. 21. E. Moynihan, Robust Control of Diesel Internal Combustion Engines, 1998. 22. G. Papageorgiou, Robust Control System Design: H∞ Loop Shaping and Aerospace Applications, 1998. 23. R. G. Ford, Robust Automotive Idle Speed Control in a Novel Framework, 2000. 24. S. J. Cornelius, Modelling and Control of Automotive Catalysts, 2001. 25. K. M. Halsey, Nested Feedback Systems: Analysis and Design within an H∞ -Loopshaping Framework, 2002. 26. A. U. Genc, Linear Parameter-Varying Modelling and Robust Control of Variable Cam Timing Engines, 2002. 27. M. Jung, Mean-Value Modelling and Robust Control of the Airpath of a Turbocharged Diesel Engine, 2003. 28. C. Papageorgiou, Robustness Analysis of Nonlinear Dynamic Inversion Control Laws for Flight Control Application, 2003.

Curriculum Vitae

IX

Publications Books 1. K. Zhou, J. C. Doyle and K. Glover, Robust and Optimal Control, Prentice Hall, 1996. 2. D. C. McFarlane and K. Glover, Robust Controller Design using Normalised Coprime Factor Plant Descriptions, Springer Verlag, Lecture Notes in Control and Information Sciences, volume 138, 1989. 3. D. Mustafa and K. Glover, Minimum Entropy H∞ Control, Springer Verlag, Lecture Notes in Control and Information Sciences, volume 146, 1990.

Articles in scientiﬁc journals and edited books 1. P. R. Kleindorfer and K. Glover, Linear convex stochastic optimal control with applications in production planning, IEEE Transactions on Automatic Control, volume AC-18, pages 56–59, 1973. 2. K. Glover and J. C. Willems, On the stability of numerical integration routines for ordinary diﬀerential equations, Journal of the Institute of Mathematics and its Applications, volume 11, pages 171–180, 1973. 3. K. Glover and J. C. Willems, Parametrization of linear dynamical systems: Canonical forms and identiﬁability, IEEE Transactions on Automatic Control, volume AC-19, pages 640–645, 1974. 4. K. Glover and L. M. Silverman, Characterization of structural controllability, IEEE Transactions on Automatic Control, volume AC-21, pages 534– 537, 1976. 5. M. S. Grewal and K. Glover, Identiﬁability of linear and nonlinear dynamical systems, IEEE Transactions on Automatic Control, volume AC-21, pages 833–837, 1976. 6. E. Emre, L. M. Silverman and K. Glover, Generalized dynamic covers for linear systems with applications to deterministic identiﬁcation and realization problems, IEEE Transactions on Automatic Control, volume AC-22, pages 26–35, 1977. 7. B. A. Francis and K. Glover, Bounded peaking in the optimal linear regulator with cheap control, IEEE Transactions on Automatic Control, volume AC-23, pages 608–617, 1978. 8. K. Glover, New techniques for approximate realization, Proceedings of the IEE, volume 126, pages 596–604, 1979. 9. L. Ljung and K. Glover, Frequency domain versus time domain methods in system identiﬁcation - A brief discussion, Automatica, volume 17, pages 71–88, 1981. 10. K. Glover, All optimal Hankel-norm approximations of linear multivariable systems and their L∞ -error bounds, International Journal of Control, volume 6, pages 1115–1193, 1984. 11. G. Correa and K. Glover, Pseudo-canonical forms, identiﬁable parametrizations and simple parameter estimation for linear multivariable systems: Input-output models, Automatica, volume 20, pages 429–442, 1984.

X

Curriculum Vitae

12. G. Correa and K. Glover, Pseudo-canonical forms, identiﬁable parametrizations and simple parameter estimation for linear multivariable systems: Parameter estimation, Automatica, volume 20, pages 443–452, 1984. 13. R. F. Curtain and K. Glover, Controller design for distributed systems based on Hankel-norm approximation, IEEE Transactions on Automatic Control, volume AC-31, pages 173–176, 1986. 14. G. O. Correa and K. Glover, On the choice of parametrization for identiﬁcation, IEEE Transactions on Automatic Control, volume AC-31, pages 8– 15, 1986. 15. K. Glover, Robust stabilization of linear multivariable systems: Relations to approximation, International Journal of Control, volume 43, pages 741– 766, 1986. 16. R. F. Curtain and K. Glover, Robust stabilization of inﬁnite dimensional systems by ﬁnite dimensional controllers, Systems & Control Letters, volume 7, pages 41–47, 1986. 17. J. B. Moore, L. Xia and K. Glover, On improving control-loop robustness of model-matching controllers, Systems & Control Letters, volume 7, pages 83–88, 1986. 18. Y. S. Hung and K. Glover, Optimal Hankel-norm approximation of stable systems with ﬁrst-order stable weighting functions, Systems & Control Letters, volume 7, pages 165–172, 1986. 19. K. Glover and J. R. Partington, Bounds on achievable accuracy in model reduction, in Modelling Robustness and Sensitivity Reduction in Control Systems, edited by R.F. Curtain, NATO ASI Series F, pages 95–118, 1987. 20. K. Glover, Identiﬁcation: Frequency domain methods, in Systems and Control Encyclopedia, edited by M.G. Singh, Pergamon Press, pages 2264– 2270, 1987. 21. J. R. Partington, K. Glover, H. J. Zwart and R. F. Curtain, L∞ approximation and nuclearity of delay systems, Systems & Control Letters, volume 10, pages 59–65, 1988. 22. H. J. Zwart, R. F. Curtain, J. R. Partington and K. Glover, Partial fraction expansions for delay systems, Systems & Control Letters, volume 10, pages 235–243, 1988. 23. K. Glover, R. F. Curtain and J. R. Partington, Realisation and approximation of linear inﬁnite dimensional systems with error bounds, SIAM Journal of Control, volume 26, pages 863–898, 1988. 24. K. Glover and J. C. Doyle, State-space formulae for all stabilizing controllers that satisfy an H∞ -norm bound and relations to risk sensitivity, Systems & Control Letters, volume 11, pages 167–172, 1988. 25. J. C. Doyle, K. Glover, P. P. Khargonekar and B. A. Francis, State-space solutions to standard H2 and H∞ control problems, IEEE Transactions on Automatic Control, volume AC-34, pages 831–847, 1989. 26. K. Glover and D. C. McFarlane, Robust stabilization of normalized coprime factor plant descriptions with H∞ -bounded uncertainty, IEEE Transactions on Automatic Control, volume AC-34, pages 821–830, 1989.

Curriculum Vitae

XI

27. K. Glover and D. Mustafa, Derivation of the maximum entropy H∞ controller and a state-space formula for its entropy, International Journal of Control, volume 50, pages 899–916, 1989. 28. D. J. Clements and K. Glover, Spectral factorization via Hermitian pencils, Linear Algebra and its Applications, volume 122/123/124, pages 797– 846, 1989. 29. K. Glover and J. C. Doyle, A state space approach to H∞ optimal control, in Three Decades of Mathematical Systems Theory: A Collection of Surveys at the Occasion of the 50th Birthday of Jan C. Willems, edited by H. Nijmeijer and J. M. Schumacher, Springer-Verlag Lecture Notes in Control and Information Sciences volume 135, pages 179–218, 1989. 30. K. Glover, A tutorial on Hankel-norm approximation, in From Data to Model, by J. C. Willems, Springer-Verlag, pages 26–48, 1989. 31. D. J. N. Limebeer, G. Halikias and K. Glover, State-space algorithm for the computation of super-optimal matrix interpolating functions, International Journal of Control, volume 50, pages 2431–2466, 1989. 32. J. B. Moore, K. Glover and A. Telford, All stabilizing controllers as frequency shaped state estimate feedback, IEEE Transactions on Automatic Control, volume AC-35, pages 203–208, 1990. 33. D. C. McFarlane, K. Glover and M. Vidyasagar, Reduced order controller design using coprime factor model reduction, IEEE Transactions on Automatic Control, volume AC-35, pages 369–373, 1990. 34. P. A. Iglesias, D. Mustafa and K. Glover, Discrete time H∞ controllers satisfying a minimum entropy criterion, Systems & Control Letters, volume 14, pages 275–286, 1990. 35. J. A. Sefton and K. Glover, Pole/zero cancellations in general H∞ problem with reference to a two block design, Systems & Control Letters, volume 14, pages 295–306, 1990. 36. J. R. Partington and K. Glover, Robust stabilization of delay systems by approximation of coprime factors, Systems & Control Letters, volume 14, pages 325–332, 1990. 37. K. Glover, J. Lam and J. R. Partington, Rational approximation of a class of inﬁnite-dimensional systems I: Singular values of Hankel operators, Mathematics of Control, Systems and Signals, volume 3, pages 325–344, 1990. 38. M. Green, K. Glover, D. J. N. Limebeer and J. C. Doyle, A J-spectral factorization approach to H∞ control, SIAM Journal on Control and Optimimization, volume 28, pages 1350–1371, 1990. 39. K. Glover, J. Lam and J. R. Partington, Rational approximation of a class of inﬁnite-dimensional systems II: Optimal convergence rates of L∞ approximants, Mathematics of Control, Systems and Signals, volume 4, pages 233–246, 1991. 40. K. Glover, J. Lam and J. R. Partington, Rational approximation of a class of inﬁnite-dimensional systems: The L2 case, in Progress in Approximation

XII

41. 42.

43. 44. 45.

46. 47. 48. 49. 50. 51. 52. 53. 54.

Curriculum Vitae

Theory, edited by P. Nevai and A. Pinkus, Academic Press, pages 405–440, 1991. D. Mustafa, K. Glover and D. J. N. Limebeer, Solutions to the H∞ general distance problem which minimize an entropy integral, Automatica, volume 27, pages 193–199, 1991. K. Glover, D. J. N. Limebeer, J. C. Doyle, E. M. Kasenally, and M. G. Safonov, A characterization of all solutions to the four block general distance problem, SIAM Journal on Control and Optimization, volume 29, pages 283–324, 1991. D. Mustafa and K. Glover, Model reduction by H∞ -balanced truncation, IEEE Transactions on Automatic Control, volume 36, pages 668–682, 1991. P. A. Iglesias and K. Glover, State-space approach to discrete-time H∞ control, Int. J. Control, volume 54, pages 1031–1073, 1991. K. Glover, D. J. N. Limebeer and Y. S. Hung, A structured approximation problem with applications to frequency weighted model reduction, IEEE Transactions on Automatic Control, volume 37, pages 447–465, April 1992. D. C. McFarlane and K. Glover, A loop shaping design procedure using H∞ synthesis, IEEE Transactions on Automatic Control, volume 37, pages 759–769, 1992. G. D. Halikias, D. J. N. Limebeer and K. Glover, A state-space algorithm for the superoptimal Hankel-norm approximation problem, SIAM Journal on Control and Optimization, volume 31, pages 960–982, 1993. R. A. Hyde and K. Glover, The application of scheduled H∞ controllers to a VSTOL aircraft, IEEE Transactions on Automatic Control, volume 38, pages 1021–1039,1993. G. E. Dullerud and K. Glover, Robust stabilization of sampled-data systems to structured LTI perturbations, IEEE Transactions on Automatic Control, volume 38, pages 1497–1508, 1993. F. S. T. van Diggelen and K. Glover, State space solutions to Hadamard weighted H∞ and H2 control problems, International Journal of Control, volume 59, pages 357–394, 1994. F. S. T. van Diggelen and K. Glover, A Hadamard weighted loop shaping design procedure for robust decoupling, Automatica, volume 30, pages 831–845, 1994. K. Zhou, K. Glover, B. Bodenheimer and J. C. Doyle, Mixed H2 and H∞ performance objectives I: Robust performance analysis, IEEE Transactions on Automatic Control, volume 39, pages 1564–1574, 1994. J. C. Doyle, K. Zhou, K. Glover and B. Bodenheimer, Mixed H2 and H∞ performance objectives II: Optimal control, IEEE Transactions on Automatic Control, volume 39, pages 1575–1587, 1994. G. E. Dullerud and K. Glover, Analysis of structured LTI uncertainty in sampled-data systems, Automatica, volume 31, pages 99–113, 1995.

Curriculum Vitae

XIII

55. R. A. Hyde, K. Glover and G. T. Shanks, VSTOL ﬁrst ﬂight of an Hinﬁnity control law, Computing and Control Engineering Journal, volume 6, pages 11–16, 1995. 56. G. E. Dullerud and K. Glover, Robust performance of periodic systems, IEEE Transactions on Automatic Control, volume 41, pages 1146– 1159,1996. 57. C. L. Beck, J. C. Doyle and K. Glover, Model-reduction of multidimensional and uncertain systems IEEE Transactions on Automatic Control, volume 41, pages 1466–1477, 1996. 58. M. W. Cantoni and K. Glover, H∞ sampled-data synthesis and related numerical issues, Automatica, volume 33, pages 2233–2241, 1997. 59. P. J. Goddard and K. Glover, Controller approximation: Approaches for preserving H∞ performance, IEEE Transactions on Automatic Control, volume 43, pages 858–871, 1998. 60. Y-C. Chu and K. Glover, Bounds of the induced norm and model reduction errors for systems with repeated scalar nonlinearities, IEEE Transactions on Automatic Control, volume 44, pages 471–483, 1999. 61. Y-C. Chu and K. Glover, Stabilization and performance synthesis for systems with repeated scalar nonlinearities, IEEE Transactions on Automatic Control, volume 44, pages 484–496, 1999. 62. M. W. Cantoni and K. Glover, Gap-metric robustness analysis of linear periodically time-varying feedback systems, SIAM Journal on Control and Optimization, volume 38, pages 803–822, 2000. 63. M. W. Cantoni and K. Glover, Existence of right and left representations of the graph for linear periodically time-varying systems, SIAM Journal on Control and Optimization, volume 38, pages 786–802, 2000. 64. K. Glover and M. G. Safonov, George Zames — 7 January 1934–10 August 1997, International Journal on Robust and Nonlinear Control, volume 10, pages 849–850, 2000. 65. S. J. Cornelius, N. Collings and K. Glover, The role of oxygen storage in NO conversion in automotive catalysts, Topics in Catalysis, volume 16, pages 57–62, 2001. 66. R. G. Ford and K. Glover, Spark ignition engine idle speed control using a novel framework and enabling control of the tradeoﬀ between fuel consumption and load rejection capability, Vehicle System Dynamics, volume 36 , pages 225–251, 2001. 67. G. Papageorgiou and K. Glover, Two-degree-of-freedom control of an actively controlled wind-tunnel model, Journal of Guidance, Control and Dynamics, volume 25, pages 510–516, 2002. 68. M. Jung, R. G. Ford, K. Glover, N. Collings, and M. Watts, Parameterization and transient validation of a variable geometry turbocharger for mean-value modeling at low and medium speed-load points, Journal on Engines, pages 2480–2493, 2002.

XIV

Curriculum Vitae

69. Y-C. Chu, K. Glover and A. P. Dowling, Control of combustion instabilities via H∞ -loop shaping, µ-analysis and integral quadratic constraints, Automatica volume 39, pages 219–231, 2003. 70. M. Jung, K. Glover and U. Christen, Comparison of uncertainty parameterisations for H-inﬁnity robust control of turbocharged diesel engines, Control Engineering Practice, volume 13, pages 15–25, 2005. 71. C. Papageorgiou and K. Glover, Robustness analysis of nonlinear ﬂight controllers, Journal of Guidance, Control and Dynamics, volume 28, pages 639–648, 2005. 72. K. M. Halsey and K. Glover, Analysis and synthesis of nested feedback systems, IEEE Transactions on Automatic Control, volume 50, pages 984– 996, 2005. 73. K. M. Halsey and K. Glover, Analysis and synthesis on a generalized stability region, IEEE Transactions on Automatic Control, volume 50, pages 997–1009, 2005.

Selected conference papers 1. K. Glover and D. J. N. Limebeer, Robust Multivariable Control System Design Using Optimal Reduced Order Plant Model, Proceedings of the American Control Conference, San Francisco, pages 644–649, 1983. 2. K. Glover, Multiplicative Approximation of Linear Multivariable Systems with L∞ Error Bounds, Proceedings of the American Control Conference, Seattle, pages 1705–1709, 1986. 3. K. Glover, Minimum Entropy and Risk Sensitive Control : the Continuous Time Case, Proceedings of the 28-th IEEE Conference on Decision and Control, Tampa, pages 388–391, Dec. 1989. 4. K. Glover, State space formulae for the multiplicative approximation of multivariable transfer functions, Proceedings of the International Symposium on Networks and Systems, MTNS-91, Kobe, Japan, 1991. Published as Recent Advances in Mathematical Theory of Systems, Control, Networks and Signal Processing I, Mita Press, Tokyo, pages 467–472, 1992. 5. R. A. Davis and K. Glover, An Application of recent model validation techniques to ﬂight test data,Proceedings of the Third European Control Conference ECC95, Rome, 1995. 6. G. Papageorgiou and K. Glover, A Systematic Procedure for Designing Non-Diagonal Weights to Facilitate H∞ Loop Shaping, Proceedings of the 36-th IEEE Conference on Decision and Control, San Diego, pages 2127– 2132, 1997. 7. K. Glover, G. Vinnicombe and G. Papageorgiou, Guaranteed multi-loop stability margins and the gap metric, Proceedings of the 39-th IEEE Conference on Decision and Control, Sydney, pages 4084-4085, 2000.

Contents

LMI-Based Gain Scheduled Controller Synthesis for a Class of Linear Parameter Varying Systems Brian D. O. Anderson, Alexander Lanzon, Jan Bendtsen . . . . . . . . . . . . .

1

Control of High-Speed Underwater Vehicles Gary J. Balas, J´ ozsef Bokor, B´ alint Vanek, Roger E.A. Arndt . . . . . . . . 25 Using Feedback to Improve System Identiﬁcation Roger W. Brockett . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 On the Gap Metric for Finite-Dimensional Linear Time-Varying Systems in Continuous Time Michael Cantoni . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Robustly Stabilizing Controllers with Internal Loop Ruth F. Curtain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Model Reduction of Strongly Stable Nonstationary LPV Systems Mazen Farhood and Geir E. Dullerud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Getting Mobile Autonomous Robots to Rendezvous Zhiyun Lin, Bruce Francis, Manfredi Maggiore . . . . . . . . . . . . . . . . . . . . . . . 119 Experiment Design for Robust Control: Why Do More Work than Is Needed? M. Gevers, X. Bombois, G. Scorletti, P. Van den Hof, R. Hildebrand . . . 139 Past, Present and Future of Automotive Control Lino Guzzella and Christopher Onder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

XVI

Contents

Review of Multivariable Control Applied to the VAAC Harrier and Remaining Open Questions Rick A. Hyde . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Regulating Cell Shape During Cytokinesis Janet C. Eﬄer, Pablo A. Iglesias, Douglas N. Robinson . . . . . . . . . . . . . . . 203 Intrinsic Uncertainty in Gene Regulation Network Hidenori Kimura . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 An Approximate Dynamic Programming Approach to Decentralized Control of Stochastic Systems Randy Cogill, Michael Rotkowitz, Benjamin Van Roy, Sanjay Lall . . . . . . 243 An H∞ Loop-Shaping Approach to Steering Control for High-Performance Motorcycles Simos Evangelou, David J. N. Limebeer, Robin S. Sharp, Malcolm C. Smith . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Frequency Domain Versus Time Domain Methods in System Identiﬁcation –Revisited Lennart Ljung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Optimal, Worst Case Filter Design via Convex Optimization Kunpeng Sun, Andy Packard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 Distance Measures, Robust Stability Conditions and Robust Performance Guarantees for Uncertain Feedback Systems George Papageorgiou, Alexander Lanzon . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 Stochastic Modelling Over a Finite Alphabet and Algorithms for Finding Genes from Genomes M. Vidyasagar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 Feedback Networks Glenn Vinnicombe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371 Thoughts on System Identiﬁcation Jan C. Willems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 Sample Reuse Techniques for Probabilistic Robust Control Xinjia Chen, Jorge L. Aravena and Kemin Zhou . . . . . . . . . . . . . . . . . . . . . 417

LMI-Based Gain Scheduled Controller Synthesis for a Class of Linear Parameter Varying Systems Brian D.O. Anderson1 , Alexander Lanzon1 , and Jan Bendtsen2 1

2

Research School of Information Sciences and Engineering, The Australian National University Canberra ACT 0200, Australia National ICT Australia Ltd., Locked Bag 8001, Canberra ACT 2601, Australia [email protected], [email protected] Department of Control Engineering, Aalborg University Fredrik Bajers Vej 7C, 9220 Aalborg East, Denmark [email protected]

This paper presents a novel method for constructing controllers for a class of single-input multiple-output (SIMO) linear parameter varying (LPV) systems. This class of systems encompasses many physical systems, in particular systems where individual components vary with time, and is therefore of signiﬁcant practical relevance to control designers. The control design presented in this paper has the properties that the system matrix of the closed loop is multi-aﬃne in the various scalar parameters, and that the resulting controller ensures a certain degree of stability for the closed loop even when the parameters are varying, with the degree of stability related directly to a bound on the average rate of allowable parameter variations. Thus, if knowledge of the parameter variations is available, the conservativeness of the design can be kept at a minimum. The construction of the controller is formulated as a standard linear time-invariant (LTI) design combined with a set of linear matrix inequalities, which can be solved eﬃciently with software tools. The design procedure is illustrated by a numerical example.

1 Introduction Many nonlinear systems can be formulated within the framework of linear parameter varying (LPV) systems and, as a consequence, LPV systems have received much attention in the control research community within the last two decades.

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 1–23, 2006. © Springer-Verlag Berlin Heidelberg 2006

2

B.D.O. Anderson, A. Lanzon, and J. Bendtsen MΛ (s)

NΛ (s) Λ

u

M(s)−1

N(s)

y

Fig. 1. Perturbed plant represented using coprime factorizations.

In general, LPV systems are systems that can be described by linear transfer functions or state space realizations, but where certain parameters in the description are non-constant. Obviously, such a description is quite general; it captures, for instance, many nonlinear systems that can be modeled using linearized dynamics, but where the parameters vary as the operating point changes. Many examples of such systems can be found in the literature; [1] describes a method for formulating LPV models of propagation phenomena and shows two examples involving temperature control for a plate cooled by convection and mass ﬂow rate regulation in a pneumatic network. [2] contains an example of gain scheduled H∞ control design for the pitch axis of a (simpliﬁed) LPV model of a missile. [3] shows how to write the Van der Pol equation in LPV form; and so forth. Due to the close relation to LTI plants, it is natural to want to use gain scheduling techniques to control LPV plants. Care must be exercised, however, when designing gain scheduled controllers that do not explicitly take into account the rate of parameter variations for LPV plants, since a gain scheduled closed-loop system can become unstable unless the scheduling happens “suﬃciently slowly” [4]. It is thus desirable to take knowledge of the rate of parameter variations into account in the design, since otherwise one may be forced to introduce conservativeness by allowing arbitrarily fast variations. [5] provides a general survey of gain scheduling techniques and also lists references to various applications. 1.1 Systems of Interest In this paper, we are interested in LPV plants where the parameter dependence can be written using a matrix fractional description, and where the parameter dependence terms are aﬃne or in fact multi-aﬃne in the parameters. Speciﬁcally, we look at SIMO plants that permit a description of the form P (λ, s) = (N (s) + NΛ (s)Λ)(M (s) + MΛ (s)Λ)−1

(1)

where N (s), NΛ (s), M (s) and MΛ (s) are appropriately chosen stable transfer function matrices and N (s), M (s) are coprime factors over RH∞ of P (0, s), and Λ is a vector multi-linear in some scalar λi , thus

LMI-Based Gain Scheduling

3

Λ = [λ1 , λ2 , . . . , λr , λ1 λ2 , . . . , λr−1 λr , . . . , λ1 λ2 . . . λr ]T ¯ i ≤ λi ≤ λ ¯i , i = 1, . . . , r}. The with λ = [λ1 , . . . , λr ]T ∈ λbox = {−λ term multi-aﬃne allows products of diﬀerent λi , but no powers. The vector λ describes the variation from a speciﬁc ’central’ or ’nominal’ value of the LPV system. This conﬁguration is illustrated in Figure 1, from which it is clear that with λ = 0, the system is described by the ’nominal’ plant P (0, s) = N (s)M (s)−1 . Reference [6] pointed out that many physical linear systems with variations in individual physical parameters could be modelled in this way; essentially, the only restriction is that the varying components must not allow crosscouplings of energy storage. Examples of systems with multi-aﬃne dependence on parameters thus include electric circuits where individual components such as resistors, capacitors and inductors vary, corresponding mechanical systems (masses, friction coeﬃcients, inertias etc.), thermo-dynamical systems (ﬂuid storage, valves etc.), and so forth3 . Such systems often arise as a selection of linearizations of a nonlinear plant in various operating points, but may also appear from applications of system identiﬁcation or from ﬁrst-principles modeling (see e.g. [7]). Note also that parameter variations in physical systems may not be symmetrical about a central value of zero in the way implied by the deﬁnition of λbox above; however, it is always possible, through a simple variable substitution, to rewrite the system such that the parameter variation becomes symmetrical. We are going to embed the problem of designing controllers to stabilise the plant set of (1) within the problem of designing controllers to stabilise a larger set of plants. Denote the 2r corners of λbox by λcor . Thus λ ∈ λcor if and ¯ i for each i. Call the set of such λ values λcori , i ∈ {1, . . . , 2r }. only if λi = ±λ Each corner value of λ gives rise to a plant P (λcori , s). It is not hard to see, and indeed argued in [8], that for a generic value of Λ generated by a generic λ ∈ λbox , there exists a set of nonnegative weights δi summing to 1 such that 2r

δi [N (s) + NΛ (s)Λcori ]

N (s) + NΛ (s)Λ =

(2)

i=1

Indeed, we review this below, where it is also pointed out that the same set of δi yield 2r

δi [M (s) + MΛ (s)Λcori ]

M (s) + MΛ (s)Λ =

(3)

i=1 3

Note that sometimes the entry corresponding to an individual λi may correspond to the inverse of a physical component value rather than the component value itself; this is of course not relevant in terms of stability etc., since e.g., the behavior of an electric circuit may be equally well described by a resistor’s conductance as by its resistance.

4

B.D.O. Anderson, A. Lanzon, and J. Bendtsen

[Here, Λcori denotes the value of Λ corresponding to λcori ]. So we cover all plants of the set (1) by considering all plants of the form

P¯ (δ, s) = {

2r

2r

δi [N (s) + NΛ (s)Λcori ]}{ i=1

δi [M (s) + MΛ (s)Λcori ]}−1

i=1

δi ≥ 0

δi = 1 (4)

Call the set (1) Sλ and call the set just deﬁned Sδ . There holds Sλ ⊆ Sδ . We will actually explain how to design controllers for the set Sδ . The construction of the δi from a particular λ can be made independent of NΛ (s), MΛ (s) etc, as indicated in the following argument, referred to in [9], which is a minor development of [8]. Let g(λ1 , λ2 , . . . , λr ) be any multi-aﬃne function of the λi , with −1 ≤ λi ≤ 1.Then successive use of the multi-aﬃne property yields: g(λ1 , λ2 , . . . , λr ) = (1/2)(1+λ1 )g(1, λ2 , . . . , λr )+(1/2)(1−λ1 )g(−1, λ2 , . . . , λr ) (5) = (1/2)(1 + λ1 )[(1/2(1 + λ2 )g(1, 1, λ3 , . . . , λr ) + (1/2)(1 − λ2 )g(1, −1, λ3 , . . . , λr )] + (1/2)(1 − λ1 )[(1/2(1 + λ2 )g(−1, 1, λ3 , . . . , λr ) + (1/2)(1 − λ2 )g(−1, −1, λ3, . . . , λr )]

(1/2r )[1 + (−1)i1 λ1 ][1 + (−1)i2 λ2 ]

= ij ∈1,2,j=1,...,r

. . . [1 + (−1)ir λr ]g((−1)i1 , (−1)i2 , . . . , (−1)ir ) (6) The δi are evidently the quantities (1/2r )[1 + (−1)i1 λ1 ][1 + (−1)i2 λ2 ] . . . [1 + (−1)ir λr ]. We shall describe this decomposition as the standard convex representation. Now, in order to treat the controller synthesis problem and furthermore be able to handle time-varying parameter values without losing stability guarantees, we will appeal to two diﬀerent (though somewhat related) approaches for dealing with LPV systems. The ﬁrst approach relates directly to aﬃne parameter dependencies, but does not consider time variation. This idea was developed in [9], which considered strict positive realness of families of transfer function matrices and [10], which provided a convex parameterization of all ﬁxed controllers that simultaneously stabilize a parameter-dependent system for any permissible ﬁxed values of the parameters. The second idea was

LMI-Based Gain Scheduling

5

developed in [11] and relates the rate of parameter variation for time varying systems to degree of stability. We will brieﬂy review the latter ideas in the following section. 1.2 Robust Stability Results for LPV Systems Reference [10] addressed the synthesis of robust controllers for linear timeinvariant systems. The uncertain real parameters were assumed to appear linearly in the closed loop characteristic polynomial, and the parameters were assumed to lie in a bounded set, i.e., a description that matched the setup in (1). The paper proceeded to give a convex parameterization of all controllers that stabilize P (λ, s) for ﬁxed frozen values of λ. The stability proof, which was directly inspired by [9], relied on showing the equivalence between the stability of a certain transfer function involving the parameters in an aﬃne way and the existence of a frequency-dependent (stable, positive-real) multiplier π such that the inverse of the transfer function multiplied by π has positive real part for all frequencies. If a π can be found, it is then possible to compute a controller that is independent of λ and that stabilizes P for any value of λ. However, no stability guarantees were given if the parameters are allowed to vary. In the method presented here, the controller is allowed to depend on the time varying parameters; in fact, some of the parameters in the controller will be tracking the parameters in the plant directly. This oﬀers the prospect of better performance, or ability to deal with a larger uncertainty set. A signiﬁcant contribution of this paper is to give a theoretical justiﬁcation for being able to tolerate time varying parameters in a setup somewhat similar to the one given in [10]. To this end, we will appeal to the second approach to dealing with systems that are aﬃne in the parameters mentioned in the previous subsection. Recall that an autonomous linear time varying (LTV) system χ(t) ˙ = A(λ(t))χ(t)

(7)

where χ ∈ Rn is the state vector and λ ∈ λbox ⊂ Rr is a parameter vector describing the variation in the system parameters at time t, is said to be exponentially asymptotically stable with degree of stability γ > 0 if there exist real scalars c, α > 0 such that for all χ(t0 ) and t ≥ t0 , we have χ(t) eγ(t−t0 ) ≤ c χ(t0 ) e−α(t−t0 ) Reference [11] dealt with systems of this form for which the existence can be demonstrated of a quadratic Lyapunov function χT P (λ)χ with P (λ) = P T (λ) positive deﬁnite, multi-aﬃne in λ, and such that A(λ)T P (λ) + P (λ)A(λ) < −2σP (λ)

(8)

6

B.D.O. Anderson, A. Lanzon, and J. Bendtsen

[Existence of such P (λ) is demonstrated in [11] for the case where A(λ) = A + h(λ)g T with h(λ) ∈ Rn aﬃne in the elements of λ; we will however not appeal to this part of reference [11]]. Equation (8) ensures that for any frozen value of λ, the eigenvalues of A(λ) lie to the left of Re[s] = −σ. Propositions 5.1 and 5.2 of reference [11] proceed to quantify the stability of the system (7) in terms of the permissible rate of change of parameter variation. Theorem 1. For λ ∈ λbox , let A(λ) be a set of n × n matrices such that there exists a positive deﬁnite symmetric P (λ), multi-aﬃne in λ, satisfying (8) for some σ > 0. Then the time-varying system (7) is exponentially asymptotically stable with degree of stability γ ∈ [0, σ) if there exist T > 0 and 1i , 2i > 0, i = 1, . . . , r such that for all i = 1, . . . , r and all t > 0 we have ¯i + λi (t) ∈ [−λ and sup t≥0

t+T t

¯ −

1i , λi

2i ]

(9)

¯

r

|

)+λi d ln λλ¯ ii (τ −λi (τ )

i=1

dτ

|dτ < 4(σ − γ)

(10)

Proof. See [11]. This theorem implies that if it is possible to synthesize controllers for an LPV plant such that the closed loop is σ-Hurwitz for all (frozen) values of λ, and an associated quadratic Lyapunov function can be found with multi-aﬃne dependence on λ, then we will have stability guarantees for the time-varying closed-loop system provided known bounds on the average rate of variation of λ are observed. It is important to note here that (10) is a bound on the average rate of parameter variation, and that abrupt parameter variations such as steps are allowed, as long as they occur suﬃciently infrequently. 1.3 Contribution of This Paper The contribution of this paper is thus to provide a method for constructing controllers for such systems with the property that the A-matrix of the closed loop has the crucial property set out in Theorem 1. The construction of these controllers is formulated by means of a set of linear matrix inequalities, which can be solved eﬃciently with readily available software tools. If these matrix inequalities are feasible, the resulting controller ensures a certain degree of stability for the closed loop with any allowed constant parameter vector, and we can then quantify the rate at which the parameter variations are allowed to occur if certain extra conditions are satisﬁed, and build this knowledge into the controller. This provides a potential for less conservative design than other LFT-based methods in the sense that the resulting gain scheduled controller does not have to take unreasonably fast parameter variations into account.

LMI-Based Gain Scheduling

7

1.4 Notation Note that throughout the paper, 0 and I indicate zero and identity elep×m ments of appropriate vector spaces (of compatible dimension). RH∞ denotes the space of all proper, real rational stable transfer matrices mapping m-dimensional input signals to p-dimensional output signals. A system P (s) p×m is σ-Hurwitz if P (s + σ) ∈ RH∞ . We will also use the notion of a Laguerre basis (see e.g., [12]). If a transfer function matrix P (s) with m inputs and p outputs can be written as P (s) = P˘ Bτk (s)

(11)

where P˘ = P˘0 P˘1 · · · P˘k Bτk (jω) = Im

2 τ −s 2 τ +s

Im

∈ Rp×(k+1)m , 2 τ −s 2 τ +s

2

Im · · ·

2 τ 2 τ

−s +s

k

T

Im

,

p×m for some τ ∈ R+ and k ∈ Z+ , then P (s) obviously belongs to RH∞ . Finally, Co{·} denotes the convex hull of (·).

1.5 Outline of Paper The outline of the rest of this paper is as follows. In the following section we provide an overview of the systems we are considering and discuss how to handle the parameter dependencies in a matrix fractional description of the plant and controller. Then, in Section 3, we formulate a ﬁnite set of linear matrix inequalities, which, if feasible, provides a controller that not only stabilizes the plant under consideration for all possible parameter values, but also preserves the appropriate parameter dependency for the closed loop characteristic polynomial allowing application of Theorem 1. In Section 4 we provide an explicit implementation of our proposed design procedure. Finally, we present a numerical example illustrating the method in Section 5 and make some closing remarks in Section 6.

2 LPV Framework As stated in the introduction, we will employ a matrix fractional description of the plant with parameter variations and demonstrate the existence of a set of synthesis LMIs which, if feasible, will provide a controller that stabilizes the LPV system for all permissible values of λ, and which allows λ to vary with time. As will be shown, the controller constructed in this way depends on λ and the closed-loop characteristic polynomial of the controller-plant interconnection will have multi-aﬃne dependency on λ.

8

B.D.O. Anderson, A. Lanzon, and J. Bendtsen

2.1 Plant and Controller Fractional Descriptions Let P be a SIMO plant of the form (1), i.e., mapping a single input signal to p ≥ 1 output signals; we assume the parameter dependence allows us to ﬁnd 1×1 p×l p×1 , M (s) ∈ RH∞ , NΛ (s) ∈ RH∞ stable transfer functions N (s) ∈ RH∞ 1×l and MΛ (s) ∈ RH∞ such that P (λ, s) = (N (s) + NΛ (s)Λ)(M (s) + MΛ (s)Λ)−1 where

(12) T

Λ = [λ1 , λ2 , . . . , λr , λ1 λ2 , . . . , λr−1 λr , . . . , λ1 λ2 . . . λr ] ¯ i ≤ λi ≤ λ ¯ i , i = 1, . . . , r}. One can check with λ = [λ1 , . . . , λr ]T ∈ λbox = {−λ r that l = 2 − 1. This conﬁguration is illustrated in Figure 1, from which it is clear that with λ = 0 we have the nominal LTI plant P (0, s) = N (s)M (s)−1 . Speciﬁcally, N (s) and M (s) are chosen to be right coprime factorizations of P (0, s), while NΛ (s) and MΛ (s) are chosen such that N (s) + NΛ (s)Λ and M (s) + MΛ (s)Λ represent a right coprime factorization of P (λ, s) over RH∞ for almost all λ ∈ λbox . There is a nontrivial assumption buried here. Note though that a continuity argument shows that the coprimeness existing at λ = 0 persists for suﬃciently small values of λ. So the assumption is harmless in a suitably small region. Also, in case p > 1, one could argue generically that the zeros of the scalar denominator are never likely to be simultaneously zeros of all entries of the vector numerator in (12), even if as λ varies, these zeros may move backwards and forwards along the real axis in the complex plane. We shall further assume that NΛ and MΛ are strictly proper; if the form of P (λ, s) arises as described in Section 1.1 or reference [6], this will occur. Using this matrix fractional description, we deﬁne the following transfer function G(λ, s) =

N (s) + NΛ (s)Λ M (s) + MΛ (s)Λ

(p+1)×1 ∈ RH∞ .

(13)

We now postulate a controller of basically the same structure as the plant, i.e., ˜ (s) + ΛT U ˜Λ (s)) C(λ, s) = (V˜ (s) + ΛT V˜Λ (s))−1 (U (14) 1×p 1×1 l×p l×1 ˜ (s) ∈ RH , V˜ (s) ∈ RH , U ˜Λ (s) ∈ RH ˜ where U ∞ ∞ ∞ and VΛ (s) ∈ RH∞ ˜ (s), V˜ (s) are left coprime factors over RH∞ of the are chosen such that U ˜ (s) + ΛT U ˜Λ (s) and V˜ (s) + ΛT V˜Λ (s) represent nominal controller C(0, s) and U a left coprime factorization of C(λ, s) over RH∞ for almost all λ ∈ λbox . In ˜ (s) should be designed (using any standard method) fact, C(0, s) = V˜ (s)−1 U to stabilize the nominal plant P (0, s) = N (s)M (s)−1 and indeed achieve other speciﬁed performance objectives. C(0, s) might for instance be designed to provide certain stability margins etc. Analogously with G, we introduce the transfer function ˜ ˜Λ (s) V˜ (s) + ΛT V˜Λ (s) ˜ (s) − ΛT U K(λ, s) = −U corresponding to the controller C(λ, s).

1×(p+1) ∈ RH∞

(15)

LMI-Based Gain Scheduling

+$

*

9

,

( "!& )#

+%

'"!& )# Fig. 2. Closed loop of plant and controller.

2.2 Internal Stability and Multi-aﬃne Parameter Dependence The controller is placed in closed loop with the plant as depicted in Figure 2. The internal stability of the closed loop can then be determined by examining the collection of transfer functions P (λ, s) (I − C(λ, s)P (λ, s))−1 −C(λ, s) I I ˜ −1 K ˜ = G(KG)

H(P, C) =

(16)

Inserting the plant and controller factorizations in this expression yields H(P, C) = G

˜ − ΛT U ˜Λ V˜ + ΛT V˜Λ −U

N + NΛ Λ M + MΛ Λ

−1

˜ K

˜ N ) + ΛT (V˜Λ M − U ˜Λ N ) + (V˜ MΛ − U ˜ NΛ )Λ = G (V˜ M − U ˜Λ NΛ )Λ + ΛT (V˜Λ MΛ − U

−1

˜ K

(17)

where the dependencies on s have been suppressed in the notation for the sake of brevity. Note that the inverted term in the above expression is a scalar function of the entries of Λ and thus of λ; this term (without the inverse) plays the role of the characteristic polynomial of the closed loop transfer matrix of the ˜ are stable controlled system including the λ-dependence. Since both G and K transfer functions, the zeros of this “characteristic polynomial” determine the stability of the closed loop, as any unstable poles must occur here. In order to yield the desired multi-aﬃne dependence on λ of the closed-loop characteristic polynomial, this scalar denominator must be aﬃne in Λ (cf. [6]), which is ˜Λ (s)NΛ (s) to be skew-symmetric. Since equivalent to enforcing V˜Λ (s)MΛ (s)−U NΛ (s) and MΛ (s) are given from the plant description, it is clear that we must ˜Λ (s) and V˜Λ (s) to achieve this. choose U Towards this end, let F =

NΛ (p+1)×l ∈ RH∞ MΛ

(18)

10

B.D.O. Anderson, A. Lanzon, and J. Bendtsen

and suppose the normal rank of F is r. When r < p + 1, F has a nontrivial left kernel of dimension q = p + 1 − r. We pick a basis for this kernel F ⊥ ∈ q×(p+1) RH∞ that has full row normal rank and satisﬁes F ⊥ (s)F (s) = 0 ∀s ∈ C ⊥ (such an F always exists when r < p+1 and this is easily seen via a McMillan decomposition of F ). We shall further require that F ⊥ is strictly proper. This is a mild restriction, but appears necessary for the results to follow. The case r < p + 1 is guaranteed to arise when l < p + 1, a situation which corresponds to having at least as many outputs as scalar parameters. The case may arise even if l >= p+ 1, but it is not generic. If p = 1, i.e. we are working with SISO systems; then r = 1 when there is one scalar parameter only. When r = p + 1, there is of course no nontrivial left kernel for F ; for consistency of notation q×(p+1) . Then a simple choice however, we let q = 1 and set F ⊥ = 0 ∈ RH∞ ⊥ T ˜ ˜ (19) −UΛ (s) VΛ (s) = α(s)F (s) + F (s)β(s) (p+1)×(p+1)

l×q with an arbitrary α ∈ RH∞ and an arbitrary β ∈ RH∞ that T satisﬁes β = −β (i.e. is skew-symmetric) does the job, since V˜Λ (s)MΛ (s) − ˜Λ (s)NΛ (s) is clearly skew-symmetric and hence U ˜Λ (s)NΛ (s))Λ = 0 ∀s ∈ C. ΛT (V˜Λ (s)MΛ (s) − U

Moreover, some lines of linear algebra shows that (19) includes all possible choices which ensure the quadratic form is zero. Notice for future reference ˜Λ (s) and V˜Λ (s) are necessarily strictly proper. that U For notational convenience, we deﬁne ˜ (s)NΛ (s) Φ(s) = V˜ (s)MΛ (s) − U (20) Inserting this expression in equation (17), we arrive at ˜ (s)N (s)] + Φ(s)Λ H(P, C) = G(λ, s) [V˜ (s)M (s) − U + ΛT [α(s)F ⊥ (s) + F T (s)β(s)]G(0, s)

−1

˜ K(λ, s).

Note that there is no loss of generality in forcing the nominal controller co˜ to satisfy the Bezout identity V˜ (s)M (s) − U ˜ (s)N (s) = prime factors V˜ and U ˜ 1, as can be seen from the following argument. Consider V (s)M (s)−U˜(s)N (s) = Z(s) = 1, where Z(s), Z(s)−1 ∈ RH∞ since the nominal controller is assumed to stabilize the nominal plant. Then it will be possible to extract a common ˜ −1 , which cancels the common factor Z(s) factor Z(s)−1 on the right of (KG) ˜ that can be extracted on the left of K(λ, s).

3 Controller Synthesis We are now ready to state our main results. We ﬁrst present a controller synthesis result, and then show that the corresponding controller-plant interconnection has an appropriate multi-aﬃne parameter dependence that allows us to invoke Theorem 1 in order to deal with time variations.

LMI-Based Gain Scheduling

11

3.1 Parameter-Dependent Controller Construction The synthesis result presented in the following theorem relates the existence of a parameter-dependent controller to the feasibility of a set of linear matrix inequalities. Theorem 2. Consider the plant P (λ, s) given in (12) where T

Λ = [λ1 , λ2 , . . . , λr , λ1 λ2 , . . . , λr−1 λr , . . . , λ1 λ2 . . . λr ] , l = 2r − 1, ¯i ≤ and the parameter vector λ belongs to the polytope λbox = {λ ∈ Rr : −λ ¯i , i = 1, . . . , r}, and assume that the fractional description in (12) λi ≤ λ is coprime for all λ ∈ λbox , with NΛ (s) and MΛ (s) strictly proper. Assume furthermore that a nominal LTI controller C(0, s) that stabilizes P (0, s) has ˜ (s) be a left coprime factorisation over been found and let C(0, s) = V˜ −1 (s)U ˜ , V˜ satisfy the Bezout identity V˜ (s)M (s) − U ˜ (s)N (s) = 1. RH∞ such that U ˜Λ (s) Then there exists a controller C(λ, s) of the form (14), where U ˜ and VΛ (s) are chosen in accordance with (18), (19) and the associated remarks, that stabilizes P (λ, s) for all (frozen) λ ∈ Λ if there exist integers ˘ ∈ R1×(N1 +1) , Γ˘ ∈ N1 , N2 , N3 ∈ Z+ , constants τ1 , τ2 , τ3 ∈ R+ and matrices Π l×q(N2 +1) ˘ T (p+1)×(p+1) ˘ R , Ξi = −Ξi ∈ R for i ∈ {0, 1, . . . , N3 } that satisfy ˘ N1 (jω) + ΛT [Γ˘ B N2 (jω)F ⊥ (jω) Re [1 + Φ(jω)Λ]ΠB τ2 τ1 ˘N3 BτN3 (jω)]G(0, jω) > 0 ˘0 Ξ˘1 . . . Ξ + F T (jω) Ξ 3

(21)

for all ω ∈ R ∪ {∞} and all Λ = Λcori , where Λcori denotes the value assumed by Λ when λ ∈ λcori , i.e. λ assumes a value at a corner of λbox , and where G, F, F ⊥ , Φ are all deﬁned in Section 2. Proof. We shall ﬁrst establish that (21) is a necessary and suﬃcient condition for the stability of a closed loop formed from one of the set of plants Sδ , and an associated controller. The set Sλ of plants (12) is a subset of Sδ , and thus establishing the claim for Sδ will yield the result for Sλ . To establish the claim in relation to Sδ , we shall link a series of equivalent statements. Consider the plant set introduced in Section 1 deﬁned by

P¯ (δ, s) = {

2r

2r

δi [N (s) + NΛ (s)Λcori ]}{ i=1

δi [M (s) + MΛ (s)Λcori ]}−1

i=1

δi ≥ 0 Consider also a corresponding set of controllers:

δi = 1

(22)

12

B.D.O. Anderson, A. Lanzon, and J. Bendtsen

¯ s) = { C(δ,

2r

δi [V˜ (s) +

ΛTcori V˜Λ (s)]}−1 {

2r

˜ (s) + ΛT V˜Λ (s)]} δ i [U cori

i=1

i=1

δi ≥ 0

δi = 1

(23)

˜Λ (s) V˜Λ (s)] is chosen as described in the Theorem hypothesis. where [−U We assert now the equivalence of the following statements. ¯ s) of the form (23), where U ˜Λ (s) and V˜Λ (s) 1. there exists a controller C(δ, are chosen in accordance with (18), (19) and the associated remarks, that stabilizes P¯ (δ, s) in (22) for all (frozen) δi with δi ≥ 0 and δi = 1. (p+1)×(p+1) l×q 2. ∃α ∈ RH∞ and β ∈ RH∞ that satisﬁes β = −β T such that ¯ = G(λ, s) 1 + Φ(s) H(P¯ , C)

2r

δi Λcori

+

i=1 2r

δi ΛTcori

[α(s)F ⊥ (s) + F T (s)β(s)]G(0, s)

−1

˜ K(λ, s) ∈ RH∞

i=1

∀δi ≥ 0,

δi = 1,

˜ F, F ⊥ , Φ are all deﬁned in Section 2. This is merely a rewhere q, G, K, statement of (a) above and the algebra for massaging H(P, C) into the form given here was discussed in Section 2. (p+1)×(p+1) l×q 3. ∃α ∈ RH∞ and β ∈ RH∞ that satisﬁes β = −β T such that 2r

2r

δi Λcori +

1+Φ(s) i=1

δi ΛTcori [α(s)F ⊥ (s)+F T (s)β(s)]G(0, s)

−1

i=1

∈ RH∞

∀δi ≥ 0,

δi = 1.

˜ Suﬃciency is obvious since G(λ, s), K(λ, s) ∈ RH∞ . Necessity follows via the following argument: Since coprimeness of the fractional descriptions4 ˜ of P (λ, s) and C(λ, s) means that G(λ, s) and K(λ, s) have full column ¯ and full row rank respectively at all s ∈ C+ , then there exist left and right ˜ stable inverses for G(λ, s) and K(λ, s) respectively. (p+1)×(p+1) 1×1 l×q 4. ∃π ∈ RH∞ , α ∈ RH∞ and β ∈ RH∞ that satisﬁes β = −β T such that 4

Notice that if the fractional description of the plant P (λ, s) (resp. the controller ˆ ∈ Λ, then G(λ, s) (resp. K(λ, ˜ C(λ, s)) is not coprime for some λ s)) will drop ¯ + and for λ, ˆ and the interconnection of plant P (λ, s) in rank for some s0 ∈ C and controller C(λ, s) will not be internally stable. Thus the presumption that the controller C(λ, s) stabilises P (λ, s) automatically requires coprimeness of the fractional description for all λ of interest.

LMI-Based Gain Scheduling

13

2r

Re

δi Λcori

1 + Φ(jω) i=1 2r

+

δi ΛTcori

[α(jω)F ⊥ (jω) + F T (jω)β(jω)]G(0, jω) π(jω)

>0

i=1

∀ω ∈ R ∪ {∞}, λ ∈ Λ. This equivalence is via [9]. (p+1)×(p+1) 1×1 l×q 5. ∃π ∈ RH∞ , α ∈ RH∞ and β ∈ RH∞ that satisﬁes β = −β T such that Re

1 + Φ(jω)Λcori + ΛTcori [α(jω)F ⊥ (jω) + F T (jω)β(jω)]G(0, jω) π(jω)

>0

∀ω ∈ R ∪ {∞}, i = 1, . . . , 2r , Necessity is obvious since if condition (d) holds for all δi ≥ 0, δi = 1, then it is also satisﬁed for the corner points. On the other hand, suﬃciency follows by taking a positive linear combination of the inequalities of condition (e). (p+1)×(p+1) 1×1 l×q 6. ∃π ∈ RH∞ , γ ∈ RH∞ and ξ ∈ RH∞ that satisﬁes ξ = −ξ T such that Re [1 + Φ(jω)Λcori ]π(jω) + ΛTcori [γ(jω)F ⊥ (jω) + F T (jω)ξ(jω)]G(0, jω)

>0

∀ω ∈ R ∪ {∞}, i = 1, . . . , 2r . Necessity is easy via a relabling of γ = απ and ξ = βπ. Suﬃciency follows via the following argument: Summing up the 2r inequalities above yields 2r Re{π(jω)} > 0

∀ω ∈ R ∪ {∞}

since all the other terms disappear due to the symmetry about zero of the polytope considered. Consequently, π is stable and strictly positive 1×1 real, which implies it must be a unit in RH∞ . Hence, given any γ ∈ (p+1)×(p+1) l×q RH∞ and any ξ ∈ RH∞ that satisﬁes ξ = −ξ T , it is possible (p+1)×(p+1) l×q to construct α = γ/π ∈ RH∞ and β = ξ/π ∈ RH∞ that T satisﬁes β = −β . 7. there exist integers N1 , N2 , N3 ∈ Z+ , constants τ1 , τ2 , τ3 ∈ R+ and ma˘ ∈ R1×(N1 +1) , Γ˘ ∈ Rl×q(N2 +1) , Ξ ˘i = −Ξ ˘ T ∈ R(p+1)×(p+1) for trices Π i i ∈ {0, 1, . . . , N3 } that satisfy

14

B.D.O. Anderson, A. Lanzon, and J. Bendtsen

˘ τN1 (jω) + ΛT [Γ˘ BτN2 (jω)F ⊥ (jω) Re [1 + Φ(jω)Λ]ΠB 2 1 ˘0 Ξ ˘1 . . . Ξ˘N3 BτN3 (jω)]G(0, jω) > 0 + F T (jω) Ξ 3 for all ω ∈ R ∪ {∞} and all Λ = Λcori . Here, BτNi i (s) (i ∈ {1, 2, 3}) denotes Laguerre basis matrices as deﬁned in (11). The index m in (11) takes the values 1, q and p + 1 respectively. 1×1 ˘ τN1 (s) ∈ RH∞ Suﬃciency is trivially easy via a labelling π = ΠB ,γ = 1 (p+1)×(p+1) N2 N3 l×q ˘ ˘ ˘ ˘ Γ Bτ2 (s) ∈ RH∞ and ξ = Ξ0 Ξ1 . . . ΞN3 Bτ3 (s) ∈ RH∞ ˘i is skewand noting that the constructed ξ also satisﬁes ξ = −ξ T since Ξ symmetric for all i ∈ {0, 1, . . . , N3 }. Necessity on the other hand follows via the following argument: Since Laguerre parametrisations provide a uniform approximation of objects in RH∞ and furthermore since the inequalities in (f) are strict inequalities, it follows that there always exists suﬃciently large Ni and suﬃciently small τi (i ∈ {1, 2, 3}) to ensure that the truncation error does not alter the sign of the inequalities. At this point, we have a necessary and suﬃcient condition for the controllers (23) to stabilize the corresponding plant in the set Sδ , see (22). Now recall from Section 1 that for any Λ generated by a generic λ ∈ λbox , there exists a set of nonnegative weights δi summing to 1 such that 2r

δi [N (s) + NΛ (s)Λcori ]

N (s) + NΛ (s)Λ =

(24)

i=1

This is the standard convex representation and there are three similar equations for M (s) + MΛ (s)Λ and the two fractional components of the controller. Recall also that the injective mapping from Λ to the δi is independent of NΛ (s) etc. It follows that the set of plants P (λ, s) and controllers C(λ, s), λ ∈ λbox , are respectively subsets of the set of plants P¯ (δ, s) and ¯ s). Hence condition (g) is a suﬃcient condition guaranteeing controllers C(δ, the C(λ, s) stabilize the corresponding P (λ, s). 3.2 Incorporating a Degree of Stability in the Frozen Parameter Design As pointed out in the Introduction, Theorem 1 relates a certain degree of stability of a collection of frozen parameter systems to the degree of stability of a time-variable parameter system given a bound on the average rate of change of the parameters. In order to allow a reasonable amount of timevariation in the parameter λ, we have to ensure that the closed loop with frozen parameter values has a degree of stability of σ and that this property is expressible with a particular Lyapunov function. We shall deal with the Lyapunov function in the next subsection, and the degree of stability issue here. The results of Subsection 3.1 only ensure the closed-loop is stable with

LMI-Based Gain Scheduling

15

no extra degree of stability, i.e., that the closed-loop poles are only guaranteed to be in the left half of the complex plane for all λ ∈ λbox . Fortunately, this diﬃculty is straightforward to address. We will ﬁrst shift the given plant by an amount σ > 0 to the right in the complex plane, work with this σ-shifted version of the plant, design a controller for this shifted plant such that the plant/controller interconnection is stable for all permissible values of λ, and then shift the controller back to obtain the necessary degree of stability σ for the closed-loop interconnection. In more detail, assume the original given plant that we wish to control is Pˆ (λ, s) ∈ Rp×1 and that the shifted plant, denoted by P (λ, s), is deﬁned by P (λ, s) := Pˆ (λ, s − σ) ˆ (s − σ) + N ˆΛ (s − σ)Λ)(M ˆ (s − σ) + M ˆ Λ (s − σ)Λ)−1 (25) = (N p×1 ˆ p×l ˆ 1×1 ˆ (s − σ) ∈ RH∞ where N , NΛ (s − σ) ∈ RH∞ , M (s − σ) ∈ RH∞ and 1×l ˆ MΛ (s − σ) ∈ RH∞ and σ > 0 is a real constant. Of course, (25) must be a coprime factorization for all λ of interest, which is a stronger condition than ˆ (s) + N ˆΛ (s)Λ)(M ˆ (s) + M ˆ Λ (s)Λ)−1 being a coprime realization of requiring (N ˆ P (λ, s). Design a controller C(λ, s) for the shifted plant P (λ, s) according to the results presented in the previous two subsections. This controller C(λ, s) ˆ s) that will should then be shifted back to get the correct controller C(λ, actually be implemented in closed loop with the original given plant Pˆ (λ, s). ˆ s) is given by This controller C(λ,

ˆ s) := C(λ, s + σ). C(λ, Note that this σ-shift does not ruin the multi-aﬃne parameter dependency appearing in the fractional descriptions of the plant and controller, nor in the closed-loop quantity required to have no zeros in the closed right half plane. 3.3 Securing a Quadratic Lyapunov Function with Multi-aﬃne Parameter Dependence Theorem 1 is the key result from which one can conclude retention of stability even in the face of parameter variations. In this subsection we explain how the Lyapunov equation (8) can be satisﬁed with a multi-aﬃne P (λ). The argument is akin to that in [11]. With notation as arising in the proof of Theorem 2, deﬁne for i = 1, 2, . . . , 2r strictly positive real functions Zi (s) by Zi (jω) = 1 + Φ(jω)Λcori + ΛTcori α(jω)F ⊥ (jω) + F T (jω)β(jω) G(0, jω) π(jω)

(26)

where, without loss of generality, we assume π(s) has been normalised to satisfy π(∞) = 1. Suppose that each Zi (jω) has a minimal state-variable

16

B.D.O. Anderson, A. Lanzon, and J. Bendtsen

realization 1 + cTi (sI − A)−1 b. [Recall that Φ(s) and α(s)F ⊥ (s) + F T (s)β(s) are strictly proper; this ensures Z(∞) = 1]. We are implicitly ruling out the possibility of any Λcori being non-generic in the sense of giving rise to a pole-zero cancellation, or nonminimality for some i of the state-variable realization. As recalled in [11] by the Kalman-Yakubovic Lemma, there exist positive deﬁnite symmetric Pi and Qi with −Pi A − AT Pi − Qi Pi b − ci (Pi b − ci )T 2

>0

(27)

Now we know that for any λ ∈ λbox , the transfer function Z(λ, jω) = [1 + Φ(jω)Λ + ΛT [α(jω)F ⊥ (jω) + F T (jω)β(jω)]G(0, jω)]π(jω) (28) is expressible as a convex combination of the Zi (jω), and accordingly is also strictly positive real. Moreover, the standard convex representation in Section 1 implies the existence of a unique multi-aﬃne P (λ), Q(λ) and c(λ) (expressible with the same weights as a convex linear combination of the respective corner values as for Z(λ, jω)) for which Z(λ, jω) = 1 + cT (λ)(jωI − A)−1 b

(29)

and −P (λ)A − AT P (λ) − Q(λ) P (λ)b − c(λ) >0 (P (λ)b − c(λ))T 2

(30)

Of course, P (λ), Q(λ) and c(λ) assume the values Pi , Qi and ci at the corners of λbox . Now when the shifting procedure of subsection 3.3 is used, it actually results that Zi (jω − σ) and Z(λ, jω − σ) are strictly positive real, which means that −P (λ)A − AT P (λ) − Q(λ) − 2σP (λ) P (λ)b − c(λ) >0 (P (λ)b − c(λ))T 2

(31)

Performing a congruence transformation I c(λ) 0 I on inequality (31) yields inequality (32) below −P (λ)[A − bcT (λ)] − [A − bcT (λ)]T P (λ) − Q(λ) − 2σP (λ) P (λ)b + c(λ) >0 (P (λ)b + c(λ))T 2 (32) which is associated with X −1 (s) = 1 − cT (sI − A + bcT )−1 b being strictly positive real, due to X(s) = 1 + cT (sI − A)−1 b being strictly positive real.

LMI-Based Gain Scheduling

17

From inequality (32), it is immediate that [A − bcT (λ)]T P (λ) + P (λ)[A − bcT (λ)] < −2σP (λ)

(33)

Now the frozen closed-loop characteristic polynomial as derived in the proof of Theorem 2 is the numerator of 1 + Φ(s)Λ + ΛT [α(s)F ⊥ (s) + F T (s)β(s)]G(0, s)

(34)

which is a factor of the numerator of [1 + Φ(s)Λ + ΛT [α(s)F ⊥ (s) + F T (s)β(s)]G(0, s)]π(s) = 1 + cT (λ)(sI − A)−1 b (35) Thus the closed-loop characteristic polynomial is a factor of the characteristic polynomial of A − bcT (λ). Equation (33) above, Theorem 1 and the multi-aﬃne character of P (λ) then assure stability for time-varying λ, as summarized in the following theorem. Theorem 3. Consider a plant Pˆ (λ, s) and the associated σ-shifted plant P (λ, s) as given in (25). Assume that a controller of the form (14) that satisﬁes the suppositions of Theorem 2 has been synthesised for the σ-shifted plant ˆ s) := C(λ, s+σ) as the controller that is actually P (λ, s). Further, deﬁne C(λ, implemented with the original plant Pˆ (λ, s). Then the closed-loop interconnecˆ s) is exponentially asymptotically stable with degree tion of Pˆ (λ, s) and C(λ, of stability γ ∈ [0, σ] for all time-varying λ(t) ∈ λbox that satisfy conditions (9) and (10) of Theorem 1.

4 Design Procedure We will now provide an explicit procedure that can be followed in order to construct the LPV controller proposed above. We will assume that a description of the plant Pˆ (λ, s) with multi-aﬃne parameter dependence of the form (1) is available. If the plant is described in state space, it can be rewritten in transfer matrix form by following the procedure in [6]. 1. Rescale, if necessary, the polytope λbox so that it is symmetric around λ = 0 and obtain P (λ, s) by shifting Pˆ (λ, s) by σ > 0 as detailed in (25). 2. Design a nominal controller C(0, s) for the nominal shifted plant P (0, s) ˜ (s) and V˜ (s) satisfying a Bezout identity and calculate coprime factors U with the nominal plant coprime factors N (s) and M (s). 3. Deﬁne G(λ, s) as in (13), F (s) as in (18) and construct F ⊥ (s) as described immediately after (18) ensuring F ⊥ (s) is strictly proper. Deﬁne Φ(s) as in (20) and BτNi i (s) for i ∈ {1, 2, 3} as in (11), with m = 1, q, (p + 1) respectively. 4. Set up the 2r linear matrix inequalities (21) corresponding to every vertex in λbox and specify an appropriate ω-grid in order to obtain a ﬁnitedimensional problem (a set of LMIs). Attempt to solve (as is standard,

18

B.D.O. Anderson, A. Lanzon, and J. Bendtsen

˘ ∈ R1×(N1 +1) , Γ˘ ∈ Rl×q(N2 +1) , Ξ ˘i = see [13]) these LMIs for matrices Π (p+1)×(p+1) T ˘ ∈ R for i ∈ {0, 1, . . . , N }: start with small integers −Ξ 3 i N1 , N2 , N3 and large τ1 , τ2 , τ3 > 0, yielding a crude Laguerre parametrisation, and if infeasible, gradually increase N1 , N2 , N3 and decrease τ1 , τ2 , τ3 > 0 until the LMIs can be solved. ˘ ∈ R1×(N1 +1) , Γ˘ ∈ Rl×q(N2 +1) , Ξ ˘i = −Ξ ˘ T ∈ R(p+1)×(p+1) 5. Once matrices Π i ˘ τN1 (s) ∈ for i ∈ {0, 1, . . . , N3 } have been found, construct π = ΠB 1 1×1 N2 l×q ˘ ˘ ˘ ˘ RH∞ , γ = Γ Bτ2 (s) ∈ RH∞ and ξ = Ξ0 Ξ1 . . . ΞN3 BτN33 (s) ∈ (p+1)×(p+1)

RH∞ . (p+1)×(p+1) l×q 6. Deﬁne α = γ/π ∈ RH∞ and β = ξ/π ∈ RH∞ , and model reduce if necessary (one should check that the reduced quantities still satisfy the LMIs). 7. Compute ˜Λ (s) V˜Λ (s) = α(s)F ⊥ (s) + F T (s)β(s) −U as described in (19) and construct C(λ, s) as given in (14). ˆ s) = C(λ, s+σ). 8. Shift the controller by σ to obtain the ﬁnal controller C(λ, ˆ s) and Pˆ (λ, s) will then have degree of The closed loop involving C(λ, stability σ > 0 for all (frozen) λ ∈ λbox , and will also be guaranteed to be exponentially asymptotically stable with degree of stability γ ∈ [0, σ) for all time-varying λ(t) ∈ λbox that satisfy conditions (9) and (10) in Theorem 1. ¯ i , without really For a given (ﬁxed) σ > 0, we can attempt to maximize λ worrying about the possible isolated points where we might lose coprimeness, as the existence of a solution to the LMIs is still suﬃcient for H(P, C) to ¯i at a time (or all together, be stable. Maximization over each separate λ ¯ ¯ ¯ assuming λ1 = λ2 = · · · = λl ) is a generalized eigenvalue (i.e., quasi-convex) problem, which can be easily solved using commercially available software. ¯i and optimize Consequently, one strategy is to ﬁrst set the same value for all λ this value until the largest hyper-cube of feasible parameters is found. Then, ¯ i is ﬁxed for some i and the optimization is repeated for a hyper-cube of λ lower dimension, and so forth. By this strategy, eventually, a large polytope symmetric around λ = 0 for which the LMIs are satisﬁed will be found, allowing for correspondingly large parameter variations. ¯ a bisection algorithm may be To ﬁnd a good trade-oﬀ between σ and λ, run for diﬀerent values of σ, where the quasi-convex optimization strategy outlined above can be carried out for each chosen value of σ. One has to employ a bisection algorithm (or similar technique) to optimise σ because σ enters the expressions for Φ(s), F (s), F ⊥ (s), G(0, s), . . . in a highly nonlinear fashion.

LMI-Based Gain Scheduling

19

Perturbed system, nominal control

4

1

2

y (−) , y (−.−)

2 0 −2 −4 −6 −8

0

5

10

15

20

25

30

0

5

10

15 Time

20

25

30

2

u (−)

0 −2 −4 −6 −8

ˆ s); Fig. 3. Simulation of Pˆ (λ, s) interconnected with the nominal controller C(0, the parameters are ﬁxed to λ = [−1, −0.98]T . The system is disturbed by a step in w2 (see Figure 2) to w2 = −1 at time t = 0 followed by another step back to w2 = 0 at time t = 5.

5 Numerical Example Here, we will illustrate the procedure outlined in Section 3 with a numerical example. Consider the unshifted LPV plant Pˆ (λ, s) given by Pˆ (λ, s) =

n1 (s)+λ1 ν11 (s) m(s)+λ1 λ2 µ(s) n2 (s)+λ2 ν22 (s) m(s)+λ1 λ2 µ(s)

where the nominal part is given as Pˆ (0, s) =

n1 (s) m(s) n2 (s) m(s)

=

1 s4 +2.32s3 +7.48s2 +9.16s+5 s s4 +2.32s3 +7.48s2 +9.16s+5

and the system variations given by ν1 (s) = 0.13s + 0.0494, ν2 (s) = 0.156 and µ(s) = 0.195s + 0.2041. The parameters λ = [λ1 , λ2 ]T are allowed to vary within the square λbox = [−1 ; 1]2 . The plant is shifted by σ = 0.38 and an observer-based state space compensator C(0, s) is designed to place the closed-loop poles in s = −1, s = −2, ..., s = −8. Figure 3 shows a closed-loop simulation with λ = [−1, −0.98]T ; as can be seen, the nominal controller is able to stabilize the system for this ﬁxed value of the parameters. The nominal controller also stabilises the system for the four corners of λbox and indeed throughout λbox .

20

B.D.O. Anderson, A. Lanzon, and J. Bendtsen LPV system, nominal control

5

−10

1

2

y (−) , y (−.−)

0 −5

−15 −20

0

5

10

15

20

25

30

0

5

10

15 Time

20

25

30

5

u (−)

0 −5 −10 −15 −20

ˆ s) and Fig. 4. Simulation of Pˆ (λ, s) interconnected with the nominal controller C(0, time varying parameters. The same disturbance as in Figure 3 is used. The system becomes unstable due to the time variation.

However, when λ is allowed to vary as   sin 5t λ = λ(t) =  cos 5t  sin 5t cos 5t the nominal controller is not able to stabilize the system, even in case of a vanishing disturbance. This is shown in Figure 4. Notice that the frequency of variation is substantial in comparison with the degree of stability of the frozen-parameter nominal closed-loop system. The plant and controller are now factorized as   1 d(s)

N (s) s−0.38   = , d(s) M (s) 4 3 2 s +0.8s +5.702s +4.261s+2.493  0.13s d(s)

NΛ (s)  = 0 MΛ (s) 0

d(s)

0

0.156 d(s)

0

0 0

0.195s+0.13 d(s)

  

where d(s) = s4 + 10s3 + 35s2 + 50s + 24, and 3 4 2 5 4 ˜ (s) = −6046s − 7.679 × 10 s − 2.248 × 10 s + 6.125 × 10 U 4 3 2 s + 26s + 251s + 1066s + 1680 3 −4280s − 2.867 × 104 s2 + 3.394 × 104 s + 1.888 × 105 s4 + 26s3 + 251s2 + 1066s + 1680

LMI-Based Gain Scheduling

21

LPV system, LPV control

2

0 −1

1

2

y (−) , y (−.−)

1

−2 −3

0

5

10

15

20

25

30

0

5

10

15 Time

20

25

30

2

u (−)

1 0 −1 −2 −3

ˆ Fig. 5. Simulation of Pˆ (λ, s) interconnected with the LPV controller C(λ, s). The same disturbance and parameter variation as in Figure 4 is used; the closed-loop system remains stable.

s4 + 35.2s3 + 512.1s2 + 3921s + 1.196 × 104 V˜ (s) = , s4 + 26s3 + 251s2 + 1066s + 1680 respectively. NΛ (s) In our case, F (s) = is diagonal and obviously has full normal MΛ (s) ⊥ rank. Hence, the annihilator F (s) for F (s) must be zero, and α(s) is not needed. We deﬁne a grid over the jω axis (in this case, equidistant in the interval from ω = 10−2 rad/s to ω = 105 rad/s, along with inﬁnity) and set up the synthesis LMIs (21) using Matlab’s LMILAB toolbox. In this example, the LMIs were found to be feasible for N1 = 1,N3 = 0 and τ1 = 0.05. Note that N2 does not enter the picture because α(s) is absent, and τ3 is not needed because N3 = 0. We were able to compute the parameter-dependent factors in the controller as   50.72s2 +2029s 0 ρ(s)  −60.86s−2435 ˜Λ =  0 U   ρ(s)  V˜Λ =

30.15s2 +1226s+804 181.1s2 +7367s+4830 ρ(s) ρ(s)



20.1s2 +804s ρ(s)   144.9s+5797   ρ(s)

0

with ρ(s) = s5 + 50.27s4 + 437.7s3 + 1459s2 + 2037s + 966.4. This controller is then shifted back (by σ = −0.38) and implemented in closed loop with Pˆ (λ, s). Figure 5 shows a simulation of the closed loop with the LPV controller, in which it is seen that the controller is now able to stabilize the system.

22

B.D.O. Anderson, A. Lanzon, and J. Bendtsen

6 Discussion In this paper we presented a novel procedure for design of controllers for LPV systems whose transfer functions display multi-aﬃne parameter dependence. This class of systems encompasses many physical systems, and is therefore of signiﬁcant practical relevance to control designers. The controller structure was chosen to reﬂect the structure of the plant, and the terms associated with parameter dependencies in the controller were chosen to remove terms that are not multi-aﬃne in the parameters. The idea was to obtain a multiaﬃne dependence for the parameters of the closed loop A-matrix, and thus allow the designer to obtain guarantees on the extent of time variation in the parameters that can be allowed. The problem of ﬁnding the controller parameter dependencies was cast as a ﬁnite-dimensional LMI problem. We outlined the procedure in a constructive manner for a particular choice of controllers and illustrated the feasibility of the method with a numerical example. With r parameters, we actually have 2r diﬀerent frequency domain conditions which have to be fulﬁlled. As it stands, the method is limited to apply to single-input (but possibly multi-output) plants. This is due to the fact that some of the manipulations of ˜ −1 rely on this quantity being scalar. This is a weakness of the current (KG) design and should be alleviated in future research if possible. Finally, it should be noted that all the results presented deal with stability, not performance, i.e., we are not able to quantify loss of performance compared to the nominal designs, either due to time variations or due to the constrained way of handling the parameter dependence in the controller. This would be interesting to deal with in the future.

Acknowledgements This work has been supported in part by ARC Discovery-Projects Grant DP0342683 and National ICT Australia Ltd. National ICT Australia Ltd. is funded through the Australian Government’s Backing Australia’s Ability initiative, in part through the Australian Research Council. Large parts of this research were performed during a month’s visit by the third author at the Australian National University and National ICT Australia Ltd.

References 1. M. Mattei, “An LPV approach to the robust control of a class of quasi-linear processes,” Journal of Process Control, vol. 14, pp. 651–660, 2004. 2. P. Apkarian, J. M. Biannic, and P. Gahinet, “Self-scheduled H∞ control of missile via linear matrix inequalities,” AIAA Journal on Guidance, Control and Dynamics, vol. 18, pp. 532–538, 1995.

LMI-Based Gain Scheduling

23

3. F. Bruzelius, S. Petterson, and C. Breitholtz, “Linear parameter-varying descriptions of nonlinear systems,” in Proc. of the 2004 American Control Conference, 2004. 4. J. S. Shamma and M. Athans, “Analysis of gain scheduled control for nonlinear plants,” IEEE Transactions on Automatic Control, vol. 35, pp. 898–907, 1990. 5. W. Rugh and J. S. Shamma, “Research on gain scheduling,” Automatica, vol. 36, pp. 1401–1425, 2000. 6. S. Dasgupta and B. D. O. Anderson, “Physically based parameterizations for designing adaptive algorithms,” Automatica, vol. 23, pp. 469–477, 1987. 7. D. J. Leith, A. Tsourdos, B. A. White, and W. E. Leithead, “Application of velocity-based gain-scheduling to lateral autopilot design for an agile missile,” Control Engineering Practice, vol. 9, pp. 1079–1093, 2001. 8. L. Zadeh and Desoer, Linear Systems Theory. Krieger Publishing Company, 1979. 9. B. D. O. Anderson, S. Dasgupta, P. Khargonekar, F. J. Kraus, and M. Mansour, “Robust strict positive realness: Characterization and construction,” IEEE Transactions on Circuits and Systems, vol. 37, pp. 869–876, 1990. 10. A. Rantzer and A. Megretski, “A convex parameterization of robustly stabilizing controllers,” IEEE Transactions on Automatic Control, vol. 39, pp. 1802–1808, 1994. 11. S. Dasgupta, B. D. O. Anderson, G. Chockalingam, and M. Fu, “Lyapunov functions for uncertain systems with applications to the stability of time varying systems,” IEEE Transactions on Circuits and Systems, vol. 41, pp. 93–106, 1994. 12. A. Lanzon, “Pointwise in frequency performance weight optimization in musynthesis,” International Journal of Robust and Nonlinear Control, vol. 15, pp. 171–199, 2005. 13. P. Gahinet, A. Nemirovski, A. J. Laub, and M. Chilali, LMI control toolbox, The MathWorks, Inc., 1995, for use with Matlab.

Control of High-Speed Underwater Vehicles Gary J. Balas1 , J´ozsef Bokor2 , B´alint Vanek1 , and Roger E.A. Arndt3 1 2 3

Aerospace Engineering and Mechanics, 110 Union St. SE, University of Minnesota, Minneapolis, MN 55455 [email protected], [email protected] MTA SZTAKI, Hungarian Academy of Science, Budapest XI Kende u. 13-17, H-1518 Budapest, POB 63, Hungary [email protected] Civil Engineering and St. Anthony Falls Laboratory, 2 Third Ave. SE, University of Minnesota, Minneapolis, MN 55414 [email protected]

I (GB) was fortunate that early in my career I had the opportunity to work with Keith Glover. Keith had just begun his obsession with golf. A simple game where a ball stands still and the player advances it from diﬀerent parts of the landscape into a hole. During a golf outing, Keith vigorously struck a ball oﬀ the fairway. It drove through a water hazard and miraculously ended up a few feet oﬀ the green. I was amazed by the image I saw as the ball moved through the water. Due to its velocity, a cavitation bubble was generated behind it. I had no inkling that my golf outing with Keith would lead many years later to an interest in the control of supercavitating vehicles.

1 Introduction An object traveling at very high speeds generates cavitation bubbles on the corner of sharp contours due to pressure gradients that result in ﬂow separation. This is often the case if a propeller spins fast enough, the surrounding liquid is vaporized due to the decrease in pressure generating cavitation bubbles [1]. As pressure surrounding the liquid decreases, the bubbles collapse. B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 25–44, 2006. © Springer-Verlag Berlin Heidelberg 2006

26

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt

Cavitation bubbles contain signiﬁcant energy that can lead to pitting on the blades and reduced eﬃciency. Minimization or reduction of cavitation is the focus of signiﬁcant engineering eﬀort. Alternatively there is a signiﬁcant beneﬁt in some circumstances to increased cavitation or supercavitation. Supercavitation can be used by vehicles to dramatically reduce drag, enabling vehicles able to maintain high high speed ﬂight under water. Supercavitation results in a single, stable bubble that can envelope an entire vehicle. Hence only the front tip of the vehicle, upon which is often mounted a cavitator, is in contact with the ﬂuid. The cavitator is designed to create a low pressure cavity around the vehicle when traveling through the ﬂuid at high speeds. The cavity may be ventilated with gas to increase its size and reduce the operating speed needed to generate a cavity. Drag on a supercavitating vehicle is dramatically reduced due to the vehicle ﬂying in a gas rather than a ﬂuid. The Russian military has exploited the beneﬁts of supercavitation with the development of the Shkval torpedo which is claimed to reach speeds of 100 m/s [2]. This is roughly a factor of 5 faster than the fastest traditional underwater vehicles. A picture of Shkval vehicle is shown in Figure 1.

Fig. 1. Shkval vehicle

Steering and stabilization within the cavity area are signiﬁcant technical challenges facing high speed supercavitating vehicles (HSSVs) [3]. The stability of the vehicle is a function of the cavitator size, shape and the size and immersion of the control surfaces. The cavity generated by the cavitator surrounds the vehicle though they may interact at the vehicle trailing edge and at the points where the control surfaces impinge on the cavity wall. The low buoyancy acting on the body requires the vehicle weight to be supported by the cavitator and control surfaces, if present. Body interaction with the cavity walls which leads to high restoring forces known as planing. Planing can be used as a support force to improved maneuverability and eliminate the ﬁn actuators, there by reducing cost and complexity. Planing though can lead to limit cycle oscillations and increases drag if not controlled. Hence, there are many conﬁgurations and tradeoﬀs associated with control surface selection [5].

Control of High-Speed Underwater Vehicles

27

Even if the body is stable inside the cavity, the vehicle may not be stable when in contact with the cavity. Nonlinear interaction of the control surfaces and the body with the cavity wall is very important when calculating the ﬁn and planing forces acting on the vehicle. Note that the distance between the body and the cavity wall is on the order of 4-10 cm. The cavity wall exerts a large restoring force over the short period of time. The nature of this instability forces the vehicle back into the cavity, often resulting in limit cycle behavior [3]. Cavity-vehicle interaction also exhibits strong memory eﬀects and cavity shape is a function of the history of the vehicle motion. Suppression of limit cycles oscillations and disturbance attenuation require high bandwidth actuators and high sample rate real-time control algorithms. This paper investigates the control challenges associated with supercavitating vehicles using a low order, longitudinal axis vehicle model. The ﬁrst part of the paper presents the simpliﬁed pitch-plane model of a HSSV vehicle. This model draws heavily on the benchmark HSSV model proposed by Dzielski and Kurdila [3]. That model is extended to include delay dependent interaction of the vehicle and cavity and a more reﬁned planing model. The second part of the paper describes the control approach applied to the supercavitating vehicle model. The plant model is described as a switched, bimodal system whose switching surface, the cavity boundary, is delay dependent [9]. Controllability of delay dependent, bimodal systems is discussed and a systematic control design strategy based on dynamic inversion is proposed. A controller is synthesized to track pitch angle, angular rate, vertical position and vertical speed for the HSSV vehicle model using the proposed approach. Simulations of the closed-loop vehicle are performed and analyzed in the third section of the paper. Challenges facing the control designer are highlighted with respect to actuator and sensor requirements, modeling issues, robustness and performance. The paper concludes with a summary of the results and directions for future study.

2 Modeling of Supercavitating Vehicles Several mathematical descriptions of high speed supercavitation vehicles (HSSVs) are available in the literature [3, 6, 8]. A brief synopsis of these models follows. •

•

A high ﬁdelity, nonlinear six degrees-of-freedom (DOF) mathematical model was proposed in [8]. Lookup tables are used to calculate the forces generated by control surfaces as a function of immersion depth. The centerline of the cavity bubble is a function of the time history of the path of the vehicle nose aﬀected by buoyancy. A two DOF longitudinal axis model for control design purposes was proposed in [3]. The simpliﬁed model is linear within the cavity and nonlinear when the vehicle is in contact with the cavity (planing). The planing con-

28

•

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt

dition is deﬁned as a function of vertical velocity and no time history dependency of the cavity is included. A one DOF longitudinal axis model was proposed to study the switching behavior of the supercavitating vehicle in [6]. The model contains only longitudinal translational motion and its interaction with the cavity. The shape of the cavity boundary is described as a function of the time delayed position of the vehicle nose.

This paper extends the two DOF longitudinal axis model developed in [3] to include the time dependent behavior of the cavity and redeﬁnes the planing condition to be a function of the vehicle vertical position, pitch angle and time history of the cavity. The only characteristic of the longitudinal axis, nonlinear six DOF model not included in the extended model is the nonlinear control surface force and moments dependence on the surface immersion depth and angle-of-attack. 2.1 Two Degree of Freedom Model with Delay A natural way to simplify the complex behavior of the 6-DOF model is to restrict the motion of the vehicle to the longitudinal plane. Consider the two degree of freedom vehicle shown in Figure 2.

Fig. 2. Schematic diagram of two degree of freedom model

The geometry of the vehicle is intended to approximate the geometry of the actual vehicle. The body consist of a conical and a cylindrical section, where the cylindrical section is twice as long as the front conical section. The reference coordinate system is placed into the cavitator’s center, the x axis is the horizontal axis (perpendicular to z) in direction of the cavitator. θ is the angular rotation of the x-axis. The z axis is oriented towards the center of Earth. The body length is denoted by L and its radius is R. The body has uniform density ρb , from which the mass and inertia are calculated. The reference point is at the cavitator, hence the inertia matrix is computed at

Control of High-Speed Underwater Vehicles

29

that point. If the full vehicle body is in the cavity, hydrodynamic forces will only act on the cavitator and ﬁns. The cavitator drag coeﬃcient is modeled as Cx = Cx0 (1 + σ) where σ is the cavitation number and Cx0 = 0.82 [4]. The resulting lift on the cavitator is approximately: Fcav =

1 πρRn2 V 2 Cx αc = Cl αc 2

(1)

where Rn is the cavitator radius, ρ water density, V vehicle velocity, δc cavitator deﬂection and αc = Vw + δc is the cavitator angle of attack. The force acting on the ﬁns is modeled as Ff in = −nCl αf

(2)

where n denotes the relative eﬀectiveness of the ﬁns to the cavitator, δf the + δf is the ﬁns angle of attack. All ﬁn deﬂection, q pitch rate and αf = w+qL V angles are assumed to be small, hence small angle approximations apply to calculate the forces. The dynamic equations of motion, after dividing through by (πρmR2 L), are       1−n −n 7 17L w ˙ w 36    = CV  mL m     9 (3) 133 2 17L 11 2 −n −n q˙ q 36 60 R + 405 L m mL  +V 

0 0

 

7 9  w  17L q 36



+ CV

 

−n 1 2  mL mL  δf  −n 0 δc m

+ Fgrav + Fplane (t, τ )

2

where C = 21 Cx RRn . The time delay between the ﬂuid impinging on the cavity and the aft end of the vehicle is δ(t) = VL . The time dependence of the states and controls is suppressed in the paper unless it is needed for clarity. The force and moments due to planing, Fplane [7], and gravity, Fgrav , are     7 1 R 1 + h )2 ( )αp , Fgrav =  9  g, (4) Fplane = −V 2  mL  (1 − 1 17L h +R 1 + 2h m

36

h is the normalized immersion depth corresponding to the distance the body penetrates the water and αp is the immersion angle corresponding to the angle the body surface makes with the cavity. R is the normalized median distance Rc −R between the transom and the cavity. The parameter values used in the R numerical calculations are given in Table 1. In the original two degree-of-freedom model proposed in [3], the interaction of the vehicle and cavity was not a function of the cavitator position. Hence it was not delay dependent. Vertical velocity, w(t), was used to predict the occurrence of planing assuming zero pitch angle and steady state

30

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt Table 1. System parameters for simulation model [3] Parameter

Description

Value and Units

g m n Rn R L V σ Cx0

Gravitational acceleration Density ratio, ρρm Fin eﬀectiveness Cavitator radius Vehicle radius Length Velocity Cavitation number Lift coeﬃcient

9.81 sm2 2 0.5 0.0191m 0.0508m 1.80m 75 m s 0.03 0.82

horizontal speeds. For short delay times, this is a valid assumption. In the extended model, the planing condition is a function of the cavity boundary and a delayed function of the cavitator position. This requires an extension of the vehicle state space description by two additional states: dz = w − V θ. z(t) vertical position of vehicle at the nose, dt dθ • θ(t) vehicle pitch angle, = q. dt The centerline of the cavity is determined by the path of the nose through the ﬂuid, see Figure 2. It is assumed that the diameter of the cavity at any given location depends only on the length between its centerline position and the actual position of the nose. The resulting equations of motion are     z˙ z  θ˙  θ    = A   + B δe + Fgrav + Fplane (t, τ ) (5) w˙  w  δc q q˙

•

The matrices A and B can be derived from equations (3, 2.1) and the deﬁnitions of the added states z and θ. The vehicle motion as stated above is a nonlinear, switched delayed-state dependent system. Contact with the gas ﬁlled cavity bubble is responsible for the nonlinear eﬀects. Three situations are possible during the longitudinal motion: (i) the entire vehicle is inside the cavity no forces are generated by planing, (ii) the transom (aft body) is immersed in the bottom of the cavity (+ direction), or (iii) the planing occurs as the transom hits the top of the cavity. In both situations the resulting planing force acts in the opposite direction of the immersion, providing a large enough force to bounce the body back into the cavity, leading to oscillatory motion like a fast boat bouncing on the top of water. The planing conditions and immersion angle are given by the following equations:

Control of High-Speed Underwater Vehicles

31

 1  R [z(t) + θL + R − z(t − τ ) − Rc ] if z(t − τ ) + Rc < z(t) + θL + R h = 0 (inside cavity) 1 [z(t − τ ) − R − z(t) − θL + R] if z(t − τ ) − Rc > z(t) + θL + R c R (6)  

θ−

w(t − τ ) + R˙ c V

bottom contact

(7) inside cavity ˙c w(t − τ ) − R θ − top contact V It can be seen by the geometric relationships how the two diﬀerent planing conditions are stated. The position of the vehicle transom is a function of the vehicle nose position, rotation and vehicle radius at the transom. The cavity’s boundary is located at Rc distance from centerline position of the cavity τ time before (z(t − τ )), the immersion is the diﬀerence between the two. The immersion angle is a function of three diﬀerent variables: The nominal cavity shape is due to a nonconstant cavity radius through the length of the body (R˙ c ), the shape of the cavity distorted by the past vertical velocity of the vehicle (w(t − τ )), and the vehicle rotation during boundary penetration, (θ). All these expressions use small angle approximations around straight level ﬂight, assuming also that the horizontal speed of the vehicle (V ) is constant and the surrounding environment including the water temperature, pressure do not change. To simplify the nonlinear equations of the planing forces, two numerical constants are deﬁned [7]: αp =

κ1 =

0

L 1.92 ( − 3)−1 − 1, Rn σ

κ2 = (1 − (1 −

4.5σ 40 1 )κ 17 ) 2 1+σ 1

(8)

Using constants κ1 and κ2 , the relationship for the cavity radius at the transom is: 1+σ 1 Rc = Rn (0.82 (9) ) 2 κ2 σ The expression for the contraction rate of the cavity (R˙ c ) is: 23

4.5σ 1 + σ 1 (1 − 1+σ κ117 ) 20 )2 V R˙ c = − (0.82 17 σ κ2 ( 1.92 σ − 3)

(10)

The equations for the cavity shape are valid if the following condition is satisﬁed: 1.92 − 3). (11) L > Rn ( σ The state space equations describing the system in equation (3) represent a bimodal, switched system. There are some speciﬁc characteristics of this model that we will use later. These are (i) in the ﬁrst mode the system dynamics is

32

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt

linear (inside cavity) and in the second mode it is nonlinear (planing) input aﬃne, i.e. the control inputs eﬀect the dynamics linearly in both modes, (ii) the switching condition does not depend on the control inputs, (iii) the switching hyperplane depends on the delayed state variable z¯(t − τ ). To develop a control strategy for this system, the following considerations will be applied. Properties (i) and (ii) allow for feedback linearization in both modes. This will be performed in a coordinate system that reﬂect the most the geometric structure of the problem. It will be shown that this design step results in linear dynamics in both modes ensuring continuous dynamics on the switching hyper surface. Since the latter depends on delayed state variables, controllability has to be analyzed and a controller has to be designed that ensures stability and tracking performance. This is investigated in the following section.

3 Theoretical Background of the Controller Design Our approach relies on the assumption that the delay in the equations of motion can be eliminated by applying a suitable feedback. Then the controllability analysis and the controller design can be performed for bimodal LTI systems. Since the concept of relative degree plays a central role in this approach, its deﬁnition for nonlinear, time delay and LTI systems will be given ﬁrst. 3.1 Feedback Linearization Given a nonlinear input aﬃne system: m

gi (x)ui ,

x˙ = f (x) +

x ∈ X,

u∈U

(12)

i=1

yj = hj (x),

yj ∈ Y,

j = 1, . . . , p,

(13)

has a vector relative degree r = [r1 , . . . , rp ], ri ≥ 0, ∀i, if at a point x0 (i) Lgj Lkf hi (x) = 0, . . . j = 1, . . . , m, k < ri−1 , (ii) The matrix  Lg1 Lrf1 −1 h1 (x), . . . , Lg1 Lrf1 −1 h1 (x)   .. =  ., rp −1 rp −1 Lgm Lf hp (x), . . . , Lgm Lf hp (x) 

AIA

has rank p at x0 .

(14)

Control of High-Speed Underwater Vehicles

33

For linear time invariant (LTI) systems given by (A, B, C), we have that Lgj Lfri −1 hi (x) = ci Ari −1 bj and if p = m then the vector relative degree is deﬁned if rankALT I = n where n is the state dimension. The concept of relative degree can be extended to time delay systems, too. Usually this is deﬁned for a discrete time equivalent of the continuous time systems by introducing the discrete time shift operator Δ as Δxt = xt−τ with τ denoting the given time delay. The time delay system is given now by (A(Δ), B(Δ), C(Δ)), i.e. the matrices depend on the delay operator. This implies that the coeﬃcients are elements of the polynomial ring R[Δ]. The relative degree is deﬁned similarly to the LTI case as follows. Deﬁnition 1 Given the single input - single output linear time delay system (A(Δ), b(Δ), c(Δ)). It has relative degree r > 0 if cAk b = 0, k = 0, . . . , r − 1 and cAr b = 0. It has pure relative degree r if in addition cAr b is an invertible element of R[Δ]. This deﬁnition has an obvious extension to the multivariable case. It requires the matrix AT D is invertible over Rp×p [Δ]. To perform the analysis and design a controller, new state variables for equation 3 are chosen as:     z(t) x1 (t) x2 (t) −V θ(t) + w(t)     (15)  x3 (t) =  θ(t) q(t) x4 (t) The matrix used for this coordinate transformation is:  T    c1 1 0 00 cT1 A 0 −V 1 0    Tc =   cT2  = 0 1 0 0 cT2 A 0 0 01

(16)

The state space equations in the new coordinate system are: x ¯˙ = where

Ac x¯(t) + Bc u(t) + F¯g Ac x¯(t) + F¯p (t, x, δ) + Bc u(t) + F¯g

if c¯T (δ)¯ x(t) ≤ 0, T if c¯ (δ)¯ x(t) ≥ 0,

(17)



   0 1 0 0 0 −α110 −α111 −α120 −α121   T   Bc = c1 AB  Ac =   0   0  0 0 1 −α210 −α211 −α220 −α221 , cT2 AB

(18)

and F¯grav = Tc Fgrav + C1 where C1 is a constant associated with the shift in the origin of the coordinate system. Similarly F¯plane = Tc Fplane .

34

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt

In our situation the inputs enter linearly in the state equations in both modes. In addition, it is assumed that all states can be measured. This allows us to select two outputs deﬁned as y1 = x1 and y2 = x3 , such that there exists pure vector relative degree in both modes, and in addition, these are identical, i.e. by deﬁning: 1000 Cc = (19) 0010 The relative degree for the modes are: r12 = 2,

r21 = 2,

r12 + r21 = n = 4

r12

r22

r12

= 2,

= 2,

+

r22

=n=4

Mode 1

(20)

Mode 2

(21)

The consequence of this property is that one can apply state feedback in both modes such that this will eliminate time delay in Mode 1 and nonlinearity (exact feedback linearization) in Mode 2. This feedback is given by: M1−1 (y˙ 13 (t) + Fα x(t) + Fg + vI (t)) M1−1 (y˙ 13 (t) + Fα x(t) + Fg − Fp (x, δ) + vII (t))

if cT (δ)x(t) ≤ 0, if cT (δ)x(t) ≥ 0, (22) where M1 = (Cc Ac Bc )−1 , y13 = [y1 , y3 ]T , and the feedback gain Fα is deﬁned by the controllability invariants αijk , i = 1, 2, j = 1, 2, k = 0, 1 of the linear part A, B of the system. uf lc =

Applying this control with identical external signals vI = vII = v, the feedback linearized closed loop has the following form in both modes: x˙ F L = AF L xF L + BF L v, rwhere AF L

 0 0 = 0 0

10 00 00 00

 0 0 , 1 0

BF L

  00 1 0  = 0 0 , 01

(23)

(24)

and the switching condition is given by the sign of ys = c(δ)xc . 3.2 Controllability Analysis of the Bimodal System The controllability of the linearized, bimodal dynamics needs to be analyzed and a tracking controller designed. Results on controllability of single input single LTI systems with single switching surface and relative degrees r = r1 = r2 = 1 has been published by [12].

Control of High-Speed Underwater Vehicles

35

In their approach the problem is reduced to analyzing the dynamics of the system on the switching surface. This is given by the zero dynamics derived with respect to the “switching output” ys . It was shown that the zero dynamics have to be controllable when using positive ys in one mode (negative ys in the second mode, respectively). It was assumed that the system is both left and right invertible and the dynamics is continuous on the switching surface, i.e. A1 x + b1 u = A2 x + b2 u. Since the relative degree r = 1, under the above assumptions, the zero dynamics can be written as: η(t) ˙ = Hη +

g1 ys (t) g2 ys (t)

if ys (t) ≤ 0, if ys (t) ≥ 0,

(25)

with η ∈ Rn−1 . It can be proved that the problem (25) is equivalent to the following (sign constrained) switching problem with systems (H, g1 ) and (H, −g2 ) with a nonnegative input u, see [19]. Then the same results can be obtained using the following reasoning: in the unconstrained input case to compute the reachability set the following Lie-algebra of the vector spaces Hη + g1 u and Hη − g2 u, i.e., the Lie-algebra generated by Hη + [g1 − g2 ][u u]T , needs to be deﬁned. Denote this set by R(H, [g1 , −g2 ]). Thus a necessary condition of controllability is that R(H, [g1 , −g2 ]) = Rn , i.e., the pair (H, [g1 , −g2 ]) has to be controllable. This is a Kalman - like rank condition. Since one can use only sign constrained inputs, this imposes an additional condition on H. A suﬃcient condition is that if H has an even number of eigenvalues with zero real parts, then the zero dynamics is controllable with nonnegative inputs. More results on controllability with nonnegative inputs can be found in [11, 13]. This result is extended for our application as follows. Consider the MISO system with B ∈ Rn×m and ys = Cx. Also consider the case, when there is a direction p ∈ ImB such that the system is left and right invertible cor¯ one has the responding to the direction p. Using the notation B = [ p B], system: ¯u x˙ = Ax + pup + B ¯,

ys = Cx.

(26)

Let us denote by V ∗ the largest (A, p) - invariant subspace in C = kerC and by W∗ the smallest (C, A) invariant subspace over Imp. It follows that system has the following decomposition induced by a choice of basis in V ∗ and W∗ : ξ˙ = A11 ξ + γv 1 ¯21 u ¯ + v) up = (−A12 η − B γ ¯22 u¯ + Gys , η˙ = A22 η + B

(27) (28) (29)

36

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt

Since r = 1, ξ = ys , equation (29) describes the dynamics of the system on C. Rewrite the equation of this zero dynamics as η˙ = P η + Q¯ u + Rys .

(30)

assuming that Q is monic. Proposition 1 If the pair (P, Q) is controllable, then η is controllable “without” using ys , e.g. by applying u¯ = Q# (−Rys + w). If the pair (P, Q) is not controllable, then the conditions of controllability with unconstrained u ¯ but nonnegative ys is the following. 1. The pair (P, [Q R]) has to be controllable. 2. Consider the decomposition induced by the reachability subspace R(P, Q), η˙ 1 = P11 η1 + P12 η2 + Q1 u ¯ + R1 ys

(31)

η˙ 2 =

(32)

P22 η2 + R2 ys ,

where R2 = 0. Then P22 has no real eigenvalues. Remark 1 The ﬁrst condition is a Kalman-rank condition. The second one can be given in some alternative forms using e.g. results from [11, 13]. For the high speed supercavitating vehicle model, this result has to be applied to a time delay system. The following approach is taken. Since only one delay time is present in the switching condition, it is possible to discretize the system with extended state space by including the delayed state variable. Since feedback linearization has been already applied, it is possible to use a backward diﬀerence scheme deﬁned for LTI systems that preserves the geometry needed to analyze the zero dynamics. The resulting discrete time state equations are: x(t + 1) = Ad x(t) + Bd v(t), where

 1 0  Ad =  0 0 1

T 1 0 0 0

0 0 0 0 1T 0 1 0 0

 0 0  0 , 0 0



0

0

ys = Cd x(t)

(33)



β21 T β22 T    0  Bd =   0 , β41 T β42 T  0 0

Cd = [1, 0, v, 0, −1]

(34)

where T denoted the sample time. The next step is to ﬁnd the relative degrees by selecting one of the inputs, say v1 ﬁrst. They are identically r = 2 for both modes since the feedback linearization and state transform resulted in the same linear canonic form in both modes.

Control of High-Speed Underwater Vehicles

37

To obtain the zero dynamics one has to construct a state transform matrix Tcd from the row vectors spanning the orthogonal complement of V ∗ and imBd1 where imBd1 is the ﬁrst column of Bd . It can be shown that V ∗⊥ = span{cs , cs Ad } and that the remaining three rows of Tcd is selected from ImB1⊥ resulting in the transform:   cs   cs A     1 0 0 0 0 (35) Tcd =   0 0 1 0 0 0 β41 T 0 −β21 T 0 Using this state transform [ξ T (t), η T (t)]T = Tcd x(t) and that V ∗ is (Ad , Bd1 ) invariant, the following decomposition is obtained: ξ(t + 1) =

0 a12 0 0 ξ(t) + v (t) + v (t) b21 1 e22 2 0 a22

ys = 1 0 ξ(t)

switching condition

η(t + 1) = P η(t) + Rξ(t) + Qv2 (t), where

  p11 p12 p13 P =  0 p22 p23  , 0 0 p33

  0 r12 R = 0 r22  , 0 0

(36) (37) (38)

 0 Q =  0 . q31 

(39)

The zero dynamics are described by the last equation. (The same approach can be repeated when selecting the second column of Bd .) Using Proposition 1, it can be seen that due to their special structure, the (P, Q), pair is controllable. This implies that the dynamic inversion controller with switching and pole placement for tracking error stability can be applied to control the bimodal system.

4 Multivariable Pole Placement With feedback linearization the system behaves the same regardless of the interior switching state, hence a single outer loop controller can guarantee stability and appropriate tracking properties for the complex system. A variety of linear design approaches can be used for that purpose [15–18], but the focus of this paper is on the feedback linearization controller. Hence a simple poleplacement controller is synthesized as the outer-loop tracking controller. The inner loop dynamics after feedback linearization is given by x˙ F L = AF L xF L + BF L v,

y = Cs xF L ,

(40)

38

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt

where using the structure of Cs one can see that the ﬁrst and the third state variables are measured. The system (Cc , AF L , BF L )has relative degrees for r1 = 2, r2 = 2 and the dynamic inversion controller designed for tracking can be written as: ref (41) v = M1−1 (x˙ ref c24 − Fppl (xF L − xc )), where xref is the reference state vector transformed to the canonic state coorc dinates, xref c24 is a 2-dimensional vector containing the 2nd and the 4th elements of xref and the gain matrix designed for pole allocation has the structure: c Fppl = as

−α ¯ 110 −α ¯ 111 0 0 0 0 −α ¯ 220 −α ¯ 221

(42)

Deﬁning the tracking error as etr = xF L −xref c , its dynamics can be written e˙ tr = Acl etr , where Acl = AF L − BF L Fppl , and   0 1 0 0  −α ¯ 110 −α ¯ 111 0 0  . Acl =   0 0 0 1  0 0 −α ¯ 220 −α ¯ 221

(43)

This shows that the coeﬃcients of the tracking error characteristic equation (thus the poles, too) are arbitrarily assigned by the choice of the feedback gain Fppl . The closed-loop and the tracking error system are stable for a given set of α ¯ coeﬃcients. The structure of the complete control-loop system is shown in Figures 3 and 4 where the inner-loop is responsible to feedback linearize the system and the outer-loop is responsible for reference tracking. As stated before the plant is switched and nonlinear, hence diﬀerent controllers are used in the three diﬀerent modes, which are selected by a state dependent switching based on the planing model and measurements. As noise enters the cavity wall, which is the switching surface also these selections are not always perfect, hence the outer-loop must be robust enough to handle that “noise.” After feedback linearization the reference tracking part needs only to be designed for the linear model. It is possible to track position, velocity, angle and angle rate commands with diﬀerent weights if they are selected consistently. The pole placement controller is synthesized in the canonic coordinates. The special structure of the feedback linearized system results in independent control position and angle tracking.

Control of High-Speed Underwater Vehicles

v

✲

✲ Bc ✻

✲

❄

vehicle

x0

Fgrav

✲

✲

❄

✲

▼ ✻ Ac

✛

39

✲ ys

Cs

x ¯(t − τ ) ✻ ✲ δ

x ¯(t)

✛

Fplane (Bimodal) ✛ Feedback Linearizing Controller

✛ ✛ ✛

(switching) Fig. 3. Inner loop: Switching feedback linearizing controller Feedback linearized system xref ✲ −1 Tc

✲ ✻

✲ Fppl vp✲ BF L ✲

xF L

x˙ F ✲ L

✲

✻ AF L ✛

Fig. 4. Outer loop: Inversion controller for tracking

5 Control of a Supercavitating Vehicle Model Simulations are performed in Matlab/Simulink environment and all parameter dependencies are analyzed with respect to the reference setup. The maneuver is an obstacle avoidance maneuver: the horizontal speed is constant 75m/s while the vehicle moves up 17.5m and returns to continue its straight path as seen in Figure 5. It is assumed that pressure and cavitation number remain constant during the 4 second maneuver. In addition to the original problem, an additional noise component is added to the cavity wall leading to uncertainty in the cavity wall position uncertain - the controller switching surface. The cavity wall is not in Rc distance from the cavity centerline, it ranges between Rc ± 0.1(Rc − R) 1 . Recall the baseline control problem assumes a with a ﬁlter Gn = 600s+1

40

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt

z (m)

20

Basic AMP 15 AMP 20

10 0 −10 0

1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

w (m/s)

10 5 0 −5 −10 0

Fig. 5. Simulated trajectory of high speed supercavtating vehicle

unity actuator model. Figure 7 shows the eﬀect of including a 30 Hz, ﬁrst order actuator model. The performance speciﬁcations are to track θ reference commands and reduce limit cycle oscillations. The reference tracking properties received the higher priority as compared with the damping the oscillatory behavior. The following controller gains were selected: −α ¯ 110 = −40000; −α ¯ 111 = −400; −α ¯ 220 = −90000; −α ¯ 221 = −600

(44)

With which the resulting eigenvalues are −300; −300; −200; −200. The resulting contribution from the tracking part of the controller with these high gains is negligible comparing with the inversion based part’s contribution to compensate against the eﬀect of planing. Actuator bandwidth has a signiﬁcant impact on closed-loop performance. The time response of the closed-loop system with the addition of a 30 Hz ﬁrst order actuator is shown in Figure 7. Increased oscillations of the vehicle in the cavity are clearly evident after cavity strikes during the maneuver. The baseline response in Figure 6 has signiﬁcantly less cavity wall strikes. All other simulation results presented have unity as the actuator model, i.e. no actuator. The value of the delay time also plays an important role in the controller performance. This is due to the feedback controller directly inverting the delay. To see this, a 5 ms reduction in the delay time is introduced into the simulation. The vehicle tracks the reference signal well though oscillations due to cavity wall strikes last longer and the maximum planing depth increases when the delay is not known precisely, see Figure 8. The cavity wall is often not a clear glass boundary between the vapor and the ﬂuid. Rather there may be a turbulent boundary associated with that interface. The eﬀect of a non-perfect cavity boundary is modeled as the nominal boundary with the addition of a “noisy” ﬂuid. The boundary is modeled as Rc ± 0.1(Rc − R). In the simulation the maximum planing depth remains the same but the actuator deﬂection is more radical, see Figure 9.

Control of High-Speed Underwater Vehicles 4

No. act (rad)

0.1

fin

0

δ

q (rad/sec)

0.2

−0.1 −0.2 0

1

2

3

4

5

6

h

4 2 0 0

0 −2 1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

0.2 δcav (rad)

prime

(m/m)

6

No act.

2

−4 0

7

41

1

2

3 4 time (s)

5

6

0.1 0 −0.1 −0.2 0

7

Fig. 6. Trajectory tracking: No actuator 4

Basic (rad)

0.2

fin

0

δ

q (rad/sec)

0.4

−0.2 −0.4 0

1

2

3

4

5

6

h

1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

0.4 δcav (rad)

prime

(m/m)

6 4 2 0 0

0 −2 −4 0

7

Basic

2

1

2

3 4 time (s)

5

6

0.2 0 −0.2 −0.4 0

7

Fig. 7. Trajectory tracking: First order 30 Hz actuator model 4

25ms delay (rad)

0.2

δ

fin

0 −0.2

h

prime

(m/m)

−0.4 0

1

2

3

4

5

6

2 0 −2 −4 0

7

4

0.4

3

0.2

δcav (rad)

q (rad/sec)

0.4

2 1 0 0

1

2

3 4 time (s)

5

6

7

25ms delay 1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

0 −0.2 −0.4 0

Fig. 8. Trajectory tracking: Uncertain time delay

42

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt 4 (rad)

q (rad/sec)

0.5

δ

fin

0

−0.5 0

Slow noise 1

2

3

4

5

6

Slow noise 1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

0.5

6

δcav (rad)

(m/m) prime

h

0 −2 −4 0

7

8

4 2 0 0

2

1

2

3 4 time (s)

5

6

0 −0.5 −1 0

7

Fig. 9. Trajectory tracking: Cavity noise

4

0.2

2

(rad)

0.4

δ

fin

0 −0.2 −0.4 0

1

2

3

4

5

h

prime

(m/m)

8

6

4 2 0 0

1

2

3 4 time (s)

5

6

1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

0.4

Basic AMP 15 AMP 20

6

0 −2 −4 0

7

δcav (rad)

q (rad/sec)

Increasing the noise magnitude results in larger actuator deﬂection spikes and has longer settling times. A larger magnitude noise of size Rc ± 0.5(Rc − R) cavity radius and a 1 slow noise case when the noise ﬁlter is Gn = 60s+1 , Figure 10. As expected the maximum planing depth increases as the maneuver become more radical but it is interesting to note that the control signals are not following the same trend, they ﬁn deﬂection remain basically the same, while the cavitator deﬂection has small contribution from the increased planing, possibly because the planing angle is diﬀerent in the diﬀerent cases.

7

0.2 0 −0.2 −0.4 0

Fig. 10. Trajectory tracking: 15, 18, 20m amplitude maneuvers

6 Summary and Future Research This paper describes the control challenges associated with supercavitating vehicles. The feedback control problem features a delayed switched system

Control of High-Speed Underwater Vehicles

43

with signiﬁcant nonlinearities aﬀecting the design. There are a number of open questions to be resolved before such a vehicle can be successfully controlled. These include: • • •

Desired maneuvering requirements and performance metrics Actuation methods, sensing requirements and accuracy needed to meet performance requirements. Robustness of feedback controller to modeling errors and poor physical models

The performance and robustness requirements facing the design of a feedback controller will signiﬁcantly shape the design of future supercavitating vehicles.

Acknowledgments This work was funded by the Oﬃce of Naval Research, award number N000140110229, Dr. Kam Ng Program Manager. The ﬁrst author would like to thank Professor Keith Glover for many stimulating conversations on the design of feedback control algorithms and model reduction for engineering systems. I am a better control engineer because of the lessons he taught me.

References 1. Sturgeon, V (2001), “Racing through water: Supercavitation,” Caltech Undergraduate Research Journal vol. 1, no. 2, 16–21 2. Ashley S (2001) “Warp drive underwater,” Scientiﬁc American, 42–51. 3. Dzielski J, Kurdila, A (2003), “A Benchmark control problem for supercavitating vehicles and an initial investigation of solutions,” J. Vibration and Control 9:791–804 4. Logvinovich GV (1972), “Hydrodynamics of free-boundary ﬂows,” translated from the Russian (NASA-TT-F-658), US Department of Commerce, Washington D.C. 5. Arndt REA, Balas GJ, Wosnik M (2005), “Control of cavitating ﬂows: A perspective,” accepted for publication, Japan Society of Mechanical Engineers International Journal, Japan 6. Kirschner IN, Rosenthal BJ, Uhlman JS (2003), “Simpliﬁed dynamical system analysis of supercavitating high-speed bodies,” Fifth International Symposium on Cavitation, Osaka Japan 7. Paryshev EV, (2002), “The Plane problem of immersion of an expanding cylinder through a cylindrical free surface of variable radius,” International Summer Scientiﬁc School on High-Speed Hydrodynamics, Cheboksary, Russia 8. Kirschner IN, Kring DC, Stokes AW, Uhlman JS (2002), “Control strategies for supercavitating vehicles,” J Vibration and Control 8:219–242 9. Shao Y, Mesbahi M, Balas GJ (2003), “Planing, switching and supercavitating ﬂight control,” AIAA Guidance, Navigation and Control Conference, AIAA2003-5724

44

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt

10. Balas G, Bokor J, Szabo Z (2003), “Invariant Subspaces for LPV Systems and Their Applications,” IEEE Trans Automat Contr, 48:2065–2068 11. Brammer RF (1972), “Controllability in linear autonomous systems with positive controllers,” SIAM J. Control, 10:329–353 12. C ¸ amlibel MK, Heemels WPMH, Schumacher JM (2004), On the controllability of bimodal piecewise linear systems. In: Alur R, Pappas GJ (eds.) Hybrid Systems: Computationand Control, LNCS 2993, Springer, Berlin, 250–264 13. Saperstone SH, Yorke JA (1971), “Controllability of linear oscillatory systems using positive controls,” SIAM J. Control, 9:253–262 14. Balas G, Bokor J, Szab´ o (2004), “Tracking of continuous LPV systems using dynamic inversion,” Conference on Decision and Control 3:2929–2933 15. Doyle JC, Glover K, Khargonekar P, Francis B (1989), “State-space solutions to standard H2 and H∞ control problems,” IEEE Trans Auto Control 34:831–847 16. McFarlane DC, Glover K (1989), Robust Controller Design using Normalised Coprime Factor Plant Descriptions, Springer-Verlag Lecture Notes in Control and Information Sciences, V138 17. McFarlane DC, Glover K (1992), “A Loop Shaping Design Procedure using H∞ Synthesis,” IEEE Trans Auto Control 37:759–769 18. Balas GJ, Packard AK, Doyle JC, Glover K, Smith R (1993) µ-Analysis and Synthesis Toolbox, MUSYN Inc. and The MathWorks, Natick MA 19. G. Balas , Z. Szab´ o and J. Bokor, Controllability of bimodal LTI systems. Submitted to the 2006 American Control Conference, Minneapolis, MN 2005.

Using Feedback to Improve System Identiﬁcation Roger W. Brockett Division of Engineering and Applied Sciences Harvard University

1 Introduction In his Doctoral thesis and subsequent publications (e.g., [1]) Keith Glover explored the parameterization of multivariable time invariant linear systems investigating identiﬁability and suitable standard forms for use in system identiﬁcation. Over the years the idea of identiﬁability has come to play an important role in the literature. In this paper we describe some new results centering around a quantitative measure of identiﬁability deﬁned in terms of a suitable Gramian. The motivation comes from a desire to provide a quantitative evaluation of the so-called “two dimensional method” widely used in nuclear magnetic resonance (NMR) spectroscopy and we provide this in terms of a Cramer-Rao bound. In the ﬁnal section we expand the scope of these ideas, providing a more general system theoretic development which discusses a new role for feedback in system identiﬁcation. An important part of the methodology to be discussed involves merging the results of a number of independent experiments in order to reduce the identiﬁcation error. The use of a sequence of independent experiments is not a new concept in system identiﬁcation in that process control engineers have often used sine wave testing to determine the frequency response of a linear system. This requires a sequence of inputs of the form ui (t) = sin ωi t, in each case waiting for the system to come to steady state, and then recording the amplitude and phase of the output signal. In the implementation of two dimensional NMR there is again a sequence of independent experiments but now what is recorded is a set of transient responses. The elements of the set are distinguished by the diﬀerence between the time at which the transient is initiated and the time at which an impulsive feedback action is applied. These responses are stitched together to form a function of two variables which is then used as the basis for identiﬁcation. Further explanation is given below.

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 45–65, 2006. © Springer-Verlag Berlin Heidelberg 2006

46

R.W. Brockett

2 Moore-Penrose and Cramer-Rao Let † denote conjugate transpose. It is a familiar fact that if A is an n by m complex matrix of rank m then the choice of x that minimizes (Ax−b)† (Ax−b) is x = (A† A)−1 A† b. Thus (A† A)−1 A† is an explicit expression for the MoorePenrose inverse applicable when columns of A are linearly independent. Of course if it happens that b is in the range space of A then this choice of x leads to zero error and minimizes ||x|| relative to any other choice that results in zero error. Suppose f is a random variable taking on values in a complex inner product space F and that f has a ﬁnite mean and variance. If A is a linear operator deﬁned on a second inner product space, A : X → F it often happens that it is desirable to ﬁnd an estimator such that the expected value of Ax(f ) agrees with the expected value of f and the variance of x(f ) is minimal. If † −1 1 f has a Gaussian density, ρ(f ) = (1/N )e− 2 (f −m) Σ (f −m) then, again, the Moore-Penrose inverse can play a role. More speciﬁcally, if it happens that Ef lies in the range space of A and if A† A is invertible, then x(f ) = (A† A)−1 A† f is the minimum variance unbiased estimate. It results in a random variable x with variance (A† A)−1 A† ΣA(A† A)−1 , where Σ is the variance of f . If the equations are scaled so that Σ = κ2 I then the variance of x is just κ2 (A† A)−1 . This result holds even when A : Cn → H with H being an inﬁnite dimensional Hilbert space, provided that the operator A† A is well deﬁned and invertible. For example, if we want an unbiased estimate of b and can observe y(k) = ak b + κn(k) for k = 0, 1, ..., with n(k) being zero mean, unity variance Gaussians, then the minimum variance, unbiased estimate of b is ∞

ˆb =

a2k

−1 ∞

k=0

ak y(k)

k=0

provided that |a| < 1 so that the sums are deﬁned. We prefer to formulate our results in continuous time so as to avoid certain technicalities. The usual interpretation of what is meant by a continuoustime measurement with additive white noise involves a limiting process which begins with discrete-time measurements with an additive Gaussian random variable, independent from sample to sample. By repeatedly doubling the sampling frequency while doubling the variance of the noise one can obtain a meaningful limit. The precise description of this limit, however, depends on an initial scaling. Adopting the usual interpretation of continuous-time measurements, as in y(t) ˙ = eλt b + κν(t) ˙ the previous analysis carries over to continuous time as ˆb =

∞ t=0

e2λt dt

−1

∞ t=0

eat y(t)dt

provided λ < 0. Using the above formulae with a = eλ and λ < 0, we see that in the two cases we have error variances

Using Feedback to Improve System Identiﬁcation

σcontinuous =

47

κ2 −κ2 ; σdiscrete = 2λ 1 − e2λ

which agree in the limit as λ goes to zero, but not otherwise. With some limitations, this idea can be applied to nonlinear problems in which the goal is to ﬁnd x(f ) such that Eφ(x) = Ef . For example, if φ is diﬀerentiable and some means for ﬁnding an approximate solution is available then rewrite φ(x) as φ(x) = φ(x0 + δx) = f and make a Taylor series approximation ∂φ δx ≈ f φ(x0 ) + ∂x Letting A denote the Jacobian we are then led to δx = (A† A)−1 A† (f − φ(x0 )) ; A =

∂φ ∂x

x0

This can be used iteratively as in x(k + 1) = x(k) − h(k) A† A)−1 A† (f − φ(x(k)) with h(k) either constant or becoming smaller with increasing k as it would in stochastic approximation. Again, this applies in the case where the range space is an inﬁnite dimensional Hilbert space provided that the operator A† A is well deﬁned and invertible. For example, if λ0 is real and if we observe y(t) ˙ = eλt b + ν(t) ˙ but now regard both b and λ as numbers to be estimated, we may linearize about (b0 , λ0 ) and work with [eλ0 t , teλ0 t b0 ]

δb = y(t) ˙ − eλ0 t b0 δλ

In this case the Moore-Penrose inverse involves the operator A† A =

∞ 0

eλ0 t teλ0 t b0

[eλ0 t , teλ0 t b0 ]

dt =

−1/2λ0 b0 /4λ20 b0 /4λ20 −b20 /8λ30

If the probability density of f is known and if it has ﬁnite variance then the Cramer-Rao inequality provides a lower bound on the variance of an unbiased estimate. For example, if, as we assumed above, f is Gaussian with density ρ(f ) =

1 −12 (f −φ(x))† (f −φ(x)) e 2κ N

and compute the Fisher information matrix M =E

∂ ln ρ(f ) ∂x

∂ ln ρ(f ) ∂x

T

48

R.W. Brockett

we have M −1 as a lower bound on the variance associated with any unbiased estimate. In the above setting M coincides with κ−2 A† A evaluated at the true value of (λ, b). The idea behind the Cramer-Rao bound is that there is a “gain factor” that describes how much a change in the parameter to be estimated changes the probability distribution of the quantities being observed. This circle of ideas provides a way to make precise the intuitive idea that the quality of any estimate is limited both by the variance of the noise and by the magnitude of the change in the probability distribution caused by a change in the quantity being estimated.

3 Moore-Penrose and Smoothing We can also consider the linear problem discussed above as an inﬁnite time smoothing problem. It is a textbook fact that for the Gauss-Markov estimation problem (all parameters are real here) x˙ = Ax + B w˙ ; y˙ = Cx + ν˙ with A, C observable, the variance of the error satisﬁes Σ˙ = AΣ + ΣAT + BB T − ΣC T CΣ In the special situation in which B is zero, a steady state solution would need to satisfy 0 = AΣ + ΣAT − ΣC T CΣ Of course Σ = 0 is one possible solution and it is easy to see that only when A has unstable eigenvalues will there be solutions with Σ both nonzero and nonnegative deﬁnite. On the other hand, when considering smoothing problems the matter is diﬀerent. If all the eigenvalues of A have negative real parts then in view of the fact that the solution of the smoothing problem runs backward in time, the positive deﬁnite solution of 0 = −AΣ − ΣAT − ΣC T CΣ given by −1

(Σ∞ )

=

∞ 0

T

eA t C T CeAt dt

is of interest. More speciﬁcally, suppose we want to estimate x(0), given y on [0, ∞) but no prior information on x(0). Rearranging the B = 0 version of the variance equation, we see that A − Σ∞ C T C = −Σ∞ AT (Σ∞ )−1 Thus, on the basis of the observation of y on [t, ∞), the best estimate of x(t) is given by the solution of

Using Feedback to Improve System Identiﬁcation

49

d x ˆ = ΣAT Σ −1 x ˆ + ΣC T y dt By the variation of constants formula, x ˆ(t) =

∞ t

Σe−A

T

or, equivalently, x ˆ(t) = ΣeAt

(t−σ)

∞ t

Σ −1 ΣC T y(σ)dσ

e−A

T

σ

C T y(σ)dσ

Setting t = 0 we see that the solution of this inﬁnite time smoothing problem coincides with the Moore-Penrose solution of CeAt x(0) = y(t) ; t ∈ [0, ∞), i.e., x ˆ(0) = Σ

∞

0

eA

T

σ

C T y(σ)dσ

4 Stable Systems; Partial Fraction Coordinates Our main new results are given in sections 5-7; this section is largely devoted to additional background material needed for comparison. The basic problem, ﬁtting a function of time with a sum of exponentials, has roots going back to work of Prony in 1795. (See [2] for references and more recent developments.) Our treatment of these ideas, based on continuous observation and taking full advantage of state-space notation, seems eﬃcient and, in any case, is essential for our later work. Suppose we are given y(tk ) + κn(tk ) with the n(tk ) being zero mean Gaussian random variables of unit variance and wish to ﬁnd (possibly complex) vectors b = [b1 , b2 , ...bn ]T and λ = [λ1 , λ2 , ..., λn ]T such that n

bi eλi tk ; k = 1, 2, 3, ...

y(tk ) = i=1

As suggested by the Cramer-Rao bound, a key role is played by the linear term in the Taylor series expansion n i=1

(bi + δi )e(λi +γi )tk =

n i=1

bi eλi tk +

n i=1

δi eλi tk +

n

bi γi tk eλi tk + ...

i=1

It is convenient to adopt a vector-matrix notation. Let Λ and b be deﬁned as     b1 λ1 0 ... 0  b2   0 λ2 ... 0     Λ=  ... ... ... ...  ; b =  ...  0 0 ... λn bn

50

R.W. Brockett

and let η denote a row vector whose entries are all ones. Then, in a notation in which the derivative is expressed as a row vector, we have ∂ηeΛt b ∂ηeΛt b ∂b ∂Λ

= ηeΛt tbT eΛt

Observe that if the real parts of λi and λj are negative then for all nonnegative integers k ∞

(−1)k+1 k! ¯ tk eλi t eλj t dt = ¯ (λi + λj )k+1

0

so that

∞ 0

¯

tk eΛt eΛt dt = Wk+1 (Λ)

with Wk being the Hermitean matrix    Wk (Λ) = (−1)k (k − 1)!  

1 1 ¯ 1 +λ1 )k (λ ¯ 1 +λ2 )k (λ 1 1 ¯ 2 +λ1 )k (λ ¯ 2 +λ2 )k (λ

...

...

1 1 ¯ n +λ1 )k (λ ¯ n +λ2 )k (λ

... ... ... ...

1 ¯ 1 +λn )k (λ 1 ¯ 2 +λn )k (λ

...

1 ¯ n +λn )k (λ

    

Given two matrices of the same size we can form a product using element by element multiplication; we write Q ∗ H to denote the matrix whose ij th entry is qij hij . This is variously called the Schur product or the Hadamard product. Note that A ∗ B = B ∗ A, that (A + B) ∗ C = A ∗ C + B ∗ C and that A ∗ (B ∗ C) = (A ∗ B) ∗ C but that mixtures of ordinary matrix product and Schur product, such as (AB) ∗ C, can not be simpliﬁed except in special cases. Our uses of the Schur product are mostly related to the identity ∞ 0

¯

tk eΛt QeΛt dt = Wk+1 (Λ) ∗ Q

valid for diagonal matrices Λ whose diagonal entries have negative real parts. As is well known, if Q and H are Hermitian and nonnegative deﬁnite their Schur product is also nonnegative deﬁnite. However, it is possible for the Schur product to be positive deﬁnite even if neither factors are even positive semideﬁnite. Lemma 1: Let η be the row vector [1, 1, ..., 1], let Λ be a diagonal matrix whose diagonal entries have negative real parts, and let b be a column vector with complex entries. Considered as a mapping from (b, λ) to L2 [0, ∞), the Jacobian of ψ(b, λ)(t) = ηeΛt b is

J=

∂ψ ∂ψ ∂b ∂Λ

= ηeΛt tbT eΛt

The Moore-Penrose inverse of J maps L2 [0, ∞) to C2n and is given by

Using Feedback to Improve System Identiﬁcation ∞

J # (·) = F −1 where

0

51

¯

eΛt η T (·)dt ¯ teΛt¯b

W1 W2 ∗ (η T bT ) ¯ W2 ∗ (bη) W3 ∗ (¯bbT )

F =

The matrix F is positive deﬁnite if and only if the λi are distinct and all the components of b are nonzero. Remark 1: The matrix F provides a measure of the linear independence of the eﬀects of changing the parameter vectors b and λ and as such might be thought of as an identiﬁability Gramian, akin to the controllability and observability Gramians of linear control theory. Proof: The derivatives evaluated at (b, Λ) can be identiﬁed with the vectors having L2 [0, ∞) components ∂ψ = eλ1 t eλ2 t ... eλn t ∂b

∂ψ = b1 teλ1 t b2 teλ2 t ... bn teλn t ∂Λ

;

Form the 2n-dimensional vector J(t) =

∂ψ ∂ψ ∂b ∂λ

and observe that if we regard J as deﬁning a map from C2n to L2 [0, ∞) then the adjoint is the integral operator ∞

J ∗ (·) =

0

J † (t)(·)dt

Thus the role of A† A in the Moore-Penrose inverse is played by the matrix F =

∞ 0

∞ 0

¯

eΛt η T ¯ teΛt¯b

ηeΛt tbT eΛt

dt

identiﬁed in the lemma. The evaluation of F requires only the properties of Wk established above, together with the observation that W1 ∗ (η T η) = W1 By construction, F is Hermitean and nonnegative deﬁnite. It will be positive deﬁnite unless there is a linear combination of the functions {eλ1 t , eλ2 t , ..., eλn t , b1 teλ1 t , b2 teλ2 t , ..., bn teλn t } that vanishes. It is a familiar fact from the theory of ordinary linear diﬀerential equations that these functions are linearly dependent if and only if λi = λj for some i = j, assuming that the bi are all nonzero. As we have seen, the Fisher information matrix determines a lower bound on the error variance associated with identiﬁcation. If this matrix is poorly conditioned in the sense that the ratio of its largest to its smallest eigenvalue is large, then some aspects of the system will be much less well characterized

52

R.W. Brockett

than others. For example, if the Gramian associated with ψ(b, λ)(t) = b1 eλ1 t + b2 eλ2 t is evaluated at ψ(t) = e−t + e−2t , then F is the four-by-four matrix 1 1 1 1  2

3 4 9 1 1 1 4 9 16 1 2 2 4 9 8 27 1 1 2 2 9 16 27 64

1 3 F = 1

  

The eigenvalues of this matrix range from about 5 × 10−5 to about .88. The smallest eigenvalue has an associated eigenvector v T ≈ [.60, −.58, −.18, −.52]. If we observe y(t) ˙ = b1 e−λ1 t + b2 e−λ2 t + κw˙ on [0, ∞) the rms error in determining b1 − b2 − .5λ1 − λ2 is about 244κ for the given values of the parameters. Clearly, even in such an unexceptional situation, reasonable identiﬁcation of b and λ requires a very small value of κ. Marginalizing the probability distribution for the error by eliminating the dependence on b increases the size of the smallest eigenvalue, reducing the rms error associated with its eignevector to about 33κ. This is still a signiﬁcant ampliﬁcation of the observational noise level. This ill conditioning is consistent with, and even to be expected, given the well documented diﬃculties with Prony’s problem. As was shown above, the Moore-Penrose inverse of the Jacobian of the C2n to L2 [0, ∞) map y(b, λ)(t) = ηeΛt b involves the integral ¯ ∞ eΛt η T ξ= y(t)dt ¯ teΛt¯b 0 Let yˆ denote the Laplace transform of y. yˆ(s) =

∞ 0

e−st y(t)dt

If y is in L2 [0, ∞) then yˆ is holomorphic in the half-plane Res > 0 and yˆ (s) =

∂ yˆ (s) = − ∂s

∞ 0

e−st ty(t)dt

This leads to an alternative expression for ξ which has both conceptual and typographic advantages, namely ξ= where

and similarly for yˆ .

¯ yˆ(−λ) ¯ −ˆ y (−λ) ∗ ¯b

 ¯1 ) yˆ(−λ ¯2 )   yˆ(−λ  yˆ(−λ) =   ...  ¯n ) yˆ(−λ 

Using Feedback to Improve System Identiﬁcation

53

Theorem 1: Let b, Λ and Wi be as above and suppose that y is an element of L2 [0, ∞). Then if the choice (b0 , Λ0 ) minimizes the integral ∞

q= we have

0

|y(t) − ηeΛt b|2 dt

¯ ; W2 (Λ0 )b0 = −ˆ ¯ W1 (Λ0 )b0 = yˆ(−λ) y (−λ)

Proof: Taking the partial derivative of q with respect to b and interpreting the derivative as a column vector yields 1 ∂q = 2 ∂b

∞ 0

Likewise,

1 ∂q = 2 ∂λ

Note that

∞ 0

so that

∞ 0

th

0

¯

eΛt η T y(t)dt

¯

−teΛt¯b(y(t) − ηeΛt b)dt

¯

−teΛt¯bbT eΛt dt = W2 ∗ (¯bbT )

0 = W2 ∗ (¯bbT ) η −

If we indicate the ij equation is

∞

¯

−eΛt η T (y(t) − ηeΛt b)dt = W1 b −

∞ 0

¯

teΛt¯by(t)dt

element of W2 as mij then in component notation this n

mij ¯bi bj =

j=1

∞ 0

¯

teλi t¯bi y(t)dt

Assuming that all the components of b are nonzero we can divide the ith of these equations by ¯bi to get W2 b =

∞ 0

¯

teΛt η T y(t)dt

Rewriting this in terms of the Laplace transform gives the ﬁnal statement in the theorem. Two comments about the theorem are in order. First of all, for some choices of y ∈ L2 [0, ∞) there is no value of (b, Λ) that minimizes the integral. For example, if y = te−t and b is two dimensional there is no minimizing value of (b, Λ), just an inﬁnimizing sequence. Secondly, if a minimum exists and W1 is invertible we can rearrange matters to get an equation for Λ that is independent of b, W2 W1−1

∞ 0

∞

¯

eΛt η T y(t)dt =

0

¯

teΛt η T y(t)dt

54

R.W. Brockett

The last equation in the theorem identiﬁes n complex equations for the n unknown elements in Λ. If there is one solution with bi = 0 and λi = λj there are n! distinct solutions with bi = 0 and λi = λj because the ordering of the λi is arbitrary. However, once Λ has been determined the equations for b are linear and uniquely solvable. Consider two special cases. Example 1: Suppose we observe a complex function of time y(t) = b0 eλ0 t + n(t) and want to estimate the complex numbers b0 and λ0 . Writing λ = σ + iω and b = α + iβ the theorem requires that we solve 2σ =

∞ λt ¯ 0 e y(t)dt ∞ λt te ¯ y(t)dt 0

−

=

¯ yˆ(−λ) ¯ yˆ (−λ)

¯ y (−λ) ¯ must vanish. Because This implies that the imaginary part of yˆ(−λ)/ˆ λ0 t the Laplace transform of b0 e is b0 /(s − λ0 ) the derivative is −b0 /(s − λ0 )2 and we see that when the noise is absent the equation for λ takes the form 2σ = σ − iω + σ0 + iω0 Having found λ, b is given by ¯ b = 2σ yˆ(−λ) When n is small relative to the size of b0 eλ0 t its eﬀect on the solution will be small. It is insightful to reinterpret the equation for σ in the following way. Recall that ln(ˆ y (s)) = ln |ˆ y(s)| + i arg yˆ(s) Rewriting the above equation for σ as ∂ ln(ˆ y (s)) 1 = σ ∂s we see that the Imaginary part of the derivative of ln(ˆ y (s) vanishes at the optimal value of σ+iω. The Cauchy-Riemann conditions applied to ln(ˆ y (s)) assert that we may express the imaginary part of the derivative in the alternative ways Re∂ ln(ˆ y (σ + iω)) Im∂ ln(ˆ y (σ + iω)) =− ∂ω ∂σ Thus, with respect to changes in ω, the logarithm of the magnitude is stationary at (b0 , Λ0 ), and in this case a maximum. ¯ Example 2: Suppose we observe y(t) = b0 eλ0 t + ¯b0 eλ0 t + w˙ and want to estimate b and λ. Let b = α + iβ and let λ = σ + iω. In this case W1−1 =

−1 −1 2σ (2σ−2iω) −1 −1 (2σ+2iω) 2σ

−1

=

2σ 2 (σ 2 + ω 2 ) ω2

1 −1 σ (σ+iω) −1 1 (σ−iω) σ

−1

Using Feedback to Improve System Identiﬁcation

W2 = −

55

1 1 4σ2 (2σ+2iω)2 1 1 (2σ−2iω)2 4σ2

Although the general solution of these equations is messy, there are important situations for which these equations can be easily interpreted. Suppose that the two eigenvalues lie near the imaginary axis, having negative real parts but with imaginary parts much bigger than the real parts. If the imaginary parts ¯ j ) is small in comparison with both 1/(λi + λ ¯i) are well separated then 1/(λi + λ ¯ and 1/(λj + λj ) and the matrices W1 and W2 are nearly diagonal. In this case 1 I and so λ is approximately the arg the matrix W2 W1−1 is approximately 2σ max of yˆ(s) Remark 2: In lemma one no allowance is made for the fact that one may know that y is real and hence the eigenvalues and the residues must occur in complex conjugate pairs. The eﬀect of restricting consideration to the real case may be analyised as follows. The general Hermitean form, expressed as φ(u + iv) = (u + iv)† (Q + iΩ)(u + iv) with both Q = QT and Ω = −Ω T real matrices, can be rewritten as a real form φ(u + iv) =

[uT , v T ]

Q Ω ΩT Q

u v

The eigenvalues of this matrix coincide with the eigenvalues of Q + iΩ, but now occur with multiplicity two. Remark 3: Lemma 1 applies directly to the partial fraction representation of a transfer function. If the transfer function is described in terms of the coeﬃcients of the numerator and denominator polynomials then there is a mapping (b, λ) → (q, p) deﬁned by n i=1

qn−1 sn−1 + qn−2 sn−2 + ... + q0 bi = s + λi sn + pn−1 sn−1 + ... + p0

that relates the 2n parameters b1 , b2 , ..., bn , λ1 .λ2 , ..., λn to the 2n parameters q0 , q1 , ..., qn−1 , p0 , , p1 , ..., pn−1 . The Jacobian of this map relates small changes in b and λ to small changes in (q, p). This Jacobian can be computed as follows. Let φ1 , φ2 , ..., φn denote the elementary symmetric functions of {λ1 , λ2 , ..., λn }, i.e., φk is the coeﬃcient of sk−1 in the polynomial p(s) = (s+λi ), so that φ1 = p0 = λ1 λ2 · · · λn , φ2 = p1 = λ1 λ2 +...+λn−1 λn ... and φn = pn−1 = λ1 + λ2 + ... + λn . Let S be the matrix whose ij th entry is ∂pi sij = ∂λj A short calculation then shows that S dq = dp 0

∂S ∂λi bi

S

db dλ

56

R.W. Brockett

The determinant of the Jacobian is nonzero unless the eigenvalues are repeated. Note that if the noise is suﬃciently small so that linearization applies the variance in (q, p) coordinates are related to the variance expressed in partial fraction coordinates by a congruence transformation Σq,p = JΣb,Λ J ∗ with J being this Jacobian.

5 A Two Variable Identiﬁability Gramian Against this background, we now turn to the main point of the paper, which is to evaluate the error reduction that is possible by the appropriate use of feedback. In NMR spectroscopy, the values of λi are often close to each other and for some important applications, such as those involving protein structure determination, the information sought relates directly to the amount by which they are separated. In this setting the accurate determination of both the absolute and relative values of nearby eigenvalues is of central importance. However, the matrix F of the previous section becomes singular when eigenvalues approach each other and hence the Cramer-Rao bound approaches inﬁnity. The diﬃculty of resolving the diﬀerence between nearby eigenvalues became apparent early on in the development of NMR and an important technique for improving the situation was proposed by Jenner in 1971. Subsequently this idea was reﬁned and experimentally developed by Ernst and his colleagues, work for which Ernst received the 1991 Nobel prize in physics. (See reference [3] for a detailed account.) The experimental realization involves recording data (eﬀectively impulse responses) from a sequence of experiments, the elements in the sequence being distinguished by the precise time at which an impulsive feedback is applied. The purpose of the feedback is to couple the characteristic responses of diﬀerent eigenmodes, in eﬀect modulating one set of frequencies with a diﬀerent set that are more easily identiﬁed. The study of the optimal way to apply feedback to achieve such modulation has been undertaken by Khaneja and his collaborators, see e.g., [4-5]. This work uses and extends ideas from Lie Group controllability and trajectory optimization. At present the application of these ideas in NMR spectroscopy represents one of the most important uses of control in the quantum domain. Postponing for now some details, this approach leads to a modiﬁed problem of the Prony type, now involving ﬁtting a function of two variables y(t, τ ) with an exponential approximation. The exponents used in the expansion must be shared so the problem takes the form n,n

y(t, τ ) ≈

bij eλi t eλj τ

i=1,j=1

with special constraints on the bij which has the eﬀect of limiting the number of independent variables to 2n, as above. This will become clear in the development below.

Using Feedback to Improve System Identiﬁcation

57

The measure of the improvement in the accuracy is reﬂected in the Fisher information matrix which, as we will see, is expressible using Schur products. As remarked above, if Q and H are Hermitian and nonnegative deﬁnite their Schur product is also nonnegative deﬁnite. This is easily shown by expanding one of the factors, say H, as a sum of the form bi b†i and observing that for † T an arbitrary vector x, x (Q ∗ b1 b1 †)x = (x ∗ b1 ) Q(x ∗ b1 ) ≥ 0 or by observing that bb† ∗ Q is actually congruent to to Q. More explicitly, bb† ∗ Q = BQB † where   b1 0 ... 0  0 b2 ... 0   B=  ... ... ... ...  0 0 ... bn In fact, it is possible to be more speciﬁc about the lower bound. If Q and H are n by n Hermitean matrices and if Q − αI ≥ 0 and H − βI ≥ 0 then the Schur product satisﬁes (Q − αI) ∗ (H − βI) ≥ 0 However (Q − αI) ∗ (H − βI) = Q ∗ H − βdiag(Q) − αdiag(H) + αβI ≥ 0 which leads to the more speciﬁc bound Q ∗ H ≥ βdiag(Q) + αdiag(H) − αβI ≥ 0 In connection with the ﬁnal inequality, note that Q ≥ αI implies diagQ ≥ αI, etc. We need the following two variable version of Lemma 1. Lemma 2: Let η, b and Λ be as in Lemma 1 and let T be a nonsingular matrix. Considered as a mapping from (b, λ) to L2 [0, ∞)×[0, ∞), the Jacobian of ψ(b, Λ)(t, τ ) = ηeΛt T eΛτ b is J(t, τ ) =

∂ψ ∂ψ ∂b ∂Λ

= ηeΛt T eΛτ , tbT eΛτ T T eΛt + τ ηeΛτ T eΛt ∗ bT

Assuming that the λi are distinct and all components of b are nonzero, the Moore-Penrose inverse of this Jacobian maps L2 [0, ∞) × [0, ∞) to C2n and is given by J # (·) = F −1 where

∞ 0

∞ 0

¯

¯

eΛτ T † eΛt η T (·)dt dτ ¯ ¯ τ eΛτ T † eΛt η T ∗ ¯b

¯ ¯ teΛt T¯eΛτ ¯b +

58

R.W. Brockett ∞

∞

F =

0

0

J † (t, τ )J(t, τ )dt dτ

Proof: The arithmetic is an elaboration of that required for Lemma 1. The calculation of the derivative with respect to b is as above. The calculation of the derivative with respect to Λ now yields two terms because Λ enters twice, in ηeΛt T eΛτ b. Writing these as ηeΛt β(τ ) on one hand and as α(t)eΛτ b on the other, facilitates the computation. The derivative that generates the term with the factor t could be written as bT eΛτ T T eΛt t ∗ η T but we can drop the ∗η T because it has no eﬀect. We decompose the evaluation of F as suggested by F =

∞ 0

∞ 0

J † (t, τ )J(t, τ )dt dτ =

W11 W12 † W22 W12

The one-one block is given by W11 =

∞

∞ 0

0

†

†

eΛ τ T † eΛ t η T ηeΛt T eΛτ dt dτ = (T † W1 T ) ∗ W1

(Again, the matrix η T η is the matrix of all ones so that W11 ∗ (η T η) = W11 .) The one-two term ∞

∞

W12 =

0

0

†

†

eΛ τ T † eΛ t η T tbT eΛτ T T eΛt + bT ∗ τ ηeΛt T eΛτ dt dτ

can not be expressed entirely in terms of the Wi because of noncommutivity. It can, however, be simpliﬁed to W12 =

∞ 0

∞ 0

†

†

teΛ τ T † eΛ t η T bT eΛτ T † eΛt dt dτ + W2 ∗ T † W1 T ∗ (η T bT )

Finally, W22 is almost completely refractory and is left as ∞ 0

∞ 0

teΛt T eΛτ ¯b + τ eΛt T T eΛτ η T ∗ ¯b

tbT eΛτ T T eΛt + τ ηeΛt T eΛτ ∗ bT dtdτ

The above formulae simplify in an important way if T ΛT −1 commutes with Λ. Because we assume that Λ is diagonal with distinct eigenvalues this implies that T ΛT −1 is diagonal and thus T ΛT −1 is simply a reordering of the original Λ. Theorem 2: Let η, b, Λ and Wi be as in Lemma 1. Assume further that T is orthogonal, that T −1 ΛT commutes with Λ, and that ηT = η. Considered as a mapping from (b, Λ) to L2 [0, ∞) × [0, ∞), the Jacobian of ψ(b, Λ)(t, τ ) = ηeΛt T eΛτ b is

Using Feedback to Improve System Identiﬁcation

J=

∂ψ ∂ψ ∂b ∂Λ

59

= ηeΛt T eΛτ , b† eΛτ T T eΛt (It + T τ )

Assuming that the λi are distinct and all components of b are nonzero, the Moore-Penrose inverse of this Jacobian maps L2 [0, ∞) × [0, ∞) to C2n and is given by J # (·) = F −1

0

¯

∞

∞ 0

te

¯ Λτ

¯

eΛτ T † eΛt η T (·)dt dτ ¯ ¯ T e b + τ eΛτ T T eΛt η T ∗ ¯b ¯ ¯ † Λτ

where F = with

W11 W12 W21 W22

W11 = W1 ∗ T T W1 T † W12 = W21 = (W2 ∗ (T W1 T T ) + W1 ∗ (T W2 T )) ∗ (η T b)

and

W22 = W3 ∗ T W1 T T + 2W2 ∗ T W2 T T + T W1 T T W3

Proof: This is an obvious specialization of lemma 2 and so we omit the proof. The generation of the term with factor τ is facilitated by using the identities ηeΛt T eΛτ b = ηeΛt T eΛτ T −1 T b = ηT eΛτ T −1 eΛt T b = ηeΛτ T −1 eΛt T b What one sees from the theorem is that the eﬀect of T is to alter the identiﬁability Gramian, opening up the possibility of increasing the smallest eigenvalue. In section 7 we consider and resolve the question of the best choice of T . Example: To illustrate how these ideas can be applied in an NMR setting we consider a somewhat idealized example which, however, captures many of the essential features of 2D NMR. The system has nine state variables and four resonant frequencies; no simpler model captures all the signiﬁcant points. The system√is described by x˙ = (A+ uB1 + vB2 )x+ b with output y = cx. The vector b is 2e1 where e1 is the standard basis vector and (A + uB1 + vB2 ) is a nine-by-nine matrix   −1 u 0 u 0 0 0 0 0  −u −σ f 0 0 v 0 0 0     0 −f −σ 0 0 0 0 0 0     −u 0 0 −σ f 0 0 v 0     A + uB1 + vB2 =   0 0 0 −f −σ 0 0 0 0   0 −v 0 0 0 −σ g 0 0     0 0 0 0 0 −g −σ 0 0     0 0 0 −v 0 0 0 −σ g  0 0 0 0 0 0 0 −g −σ

60

R.W. Brockett

and y = x6 + x8 . The identiﬁcation procedure can be described using a sequence of 5 vectors representing diﬀerent stages in the evolution of the state x. It involves the repetition of the following steps. √ 1. While √ in the steady state value of xss = 2e1 apply an impulse u = (π/ 2)δ(t). This has the eﬀect of reducing x1 to zero, changing x2 and x4 to minus one, and leaving the remaining components of x unchanged. 2. Let time evolve for τ units and then apply an impulse v = (π/2)δ(τ ). This has the eﬀect of exchanging the values of x2 and x6 and exchanging the values of x4 and x8 while leaving all other components of x unchanged. 3. As time continues to evolve, record y. These steps are summarized by the following sequence. √        0 0 0 2  0  −1   −e−στ cos f τ         −στ   −στ 0   0  0   e   e  sin f τ sin f τ      −στ     0  −1   −e    cos f τ 0      −στ   −στ   0 →  0  →  e → e  sin f τ sin f τ          0  0     −e−σt cos f τ  0          0  0      0        −σt 0   0  0     −e 0 cos f τ  0 0 0 0 followed by



 0  e−στ sin f τ · e−σt sin f t   −στ  −σt  e   −στ sin f τ · e −σt cos f t   e   −στ sin f τ · e−σt sin f t   sin f τ · e cos f t  → e   −e−στ sin f τ · e−σt cos gt   −στ  −σt  e   −στ sin f τ · e −σtsin gt   −e sin f τ · e cos g t  −στ −σt e sin f τ · e sin g t

with the output following the last step being y(t) = −e−στ sin f τ · e−σt cos gt − e−στ sin f τ · e−σt cos g t We can use the development given here to quantitatively evaluate how much mode coupling helps. We focus on the portion of W11 that involves the two modes that are close together. In the case of two complex eigenvalues σ + iω and σ + iν with δ = ω − ν, the corresponding W11 part of the Gramian contains as a sub matrix ˜ 11 = −1 W 2σ

1

−1 1+iδ/2σ

−1 1−iδ/2σ

1

Using Feedback to Improve System Identiﬁcation

61

˜ 11 is approxFor δ small in comparison with |σ|, the smallest eigenvalue of W imately smin = δ 2 /16σ 3 Thus in this range the smallest eigenvalue of W11 is proportional to δ 2 . By replacing W11 by W11 ∗ T W11 T −1 we eﬀectively replace δ by the diﬀerence between two other eigenvalues thus increasing δ and increasing the smallest eigenvalue in the Fisher information matrix. In a situation of the type explored in the example, this means replacing |f − f |, a small diﬀerence, with |g − g | which would be much larger. The resulting error variance is then reduced by a factor of |f − f |2 /|g − g |2 .

6 2D Transforms and the Resonant Case In order to express the equations for b and Λ in terms of the Laplace transform we now require a two-dimensional transform. Let yˆ be deﬁned as yˆ(s1 , s2 ) = Deﬁning Y (λ) as

∞ 0

∞ 0

y(t, τ )e−s1 t e−s2 τ y(t, τ )dtdτ



 yˆ(λ1 , λ1 ) yˆ(λ1 , λ2 ) ... yˆ(λ1 , λn )  yˆ(λ2 , λ1 ) yˆ(λ2 , λ2 ) ... yˆ(λ2 , λn )   Y (λ) =    ... ... ... ... yˆ(λn , λ1 ) yˆ(λn , λ2 ) ... yˆ(λn , λn )

we observe that ∞ 0

∞ 0

† † ¯ eΛ τ T † eΛ t y(t, τ )dtdτ = T ∗ Y (−λ)

Deﬁning

∂ yˆ(s1 , s2 ) ∂si gives rise to matrices Yi (Λ), by analogy with the deﬁnition of Y . Using these we can express yˆi (s1 , s2 ) =

ξ=

∞ 0

∞ 0

† † † † teΛ t T † eΛ τ ¯b + τ eΛ τ T T eΛ t η T ∗ ¯b y(t, τ )dt dτ

Thus the transform nicely packages the information needed to compute the Moore-Penrose inverse. Part of the value of this approach is comes from the fact that in the system identiﬁcation problem we do not know in advance which values of λ will be relevant. In computing the transform we evaluate all the functionals that might be needed, postponing the need to select the particular ones that ultimately matter. It may also be noted that in NMR applications the eigenvalues of interest are lightly damped. In this case the transform can be approximated using methods involving the fast Fourier transform.

62

R.W. Brockett

7 A Matrix Inequality The example of the previous section illustrates a speciﬁc situation in which the identiﬁability Gramian can be improved by means of the coupling distinct modes through a short feedback pulse. In this section we establish a limit on the improvement that can be achieved in this way. In 1953 Alfred Horn proved a remarkable result, providing a converse for a much older result of Issac Schur on the possible values of the diagonal of a Hermitean matrix having a given set of eigenvalues. Schur had provided a short proof that the diagonals dii and the eigenvalues µi are related by a set of inequalities which imply that if we think of the diagonal of the matrix as a point in Rn then they lie within the convex hull of the n! points having coordinates (µπ(1) , µπ(2) , ..., µπ(n) ) where π is an arbitrary permutation of the indices. Schur’s inequalities include, as a special case, the fact that the smallest eigenvalue of a symmetric matrix is upper bounded by the smallest diagonal. Horn established that every point in this convex set is achievable as the diagonal of a Hermitean matrix with the given eigenvalues. The proof of this is more challenging. Later, Bertram Kostant showed that this circle of ideas could be divorced from matrix theory and recast in terms of Lie algebras [7]. In this more general setting the Schur-Horn polytope has come to play a role in a variety of elegant theories involving topics as diverse as completely integrable systems and neural networks. The Schur-Horn polytope associated with a symmetric matrix Q has a well deﬁned center, namely the vector all of whose components are trQ/n. This point qualiﬁes as a center in that it is the midpoint of the intersection of the Schur-Horn polytope with any pencil of diagonal matrices passing through it. If K is a convex set in Rn and T is a linear transformation mapping Rn into itself, then the image of K under T is again convex. We denote this by T (K). Theorem 3: Let Q and H be n by n Hermitean matrices. Let Q have diagonal entries {qii } = {di } with d1 ≥ d2 ≥ ... ≥ dn and let H have eigenvalues µ1 ≥ µ2 ≥ ... ≥ µn . Let K be the Schur-Horn polytope associated with H. The set of possible diagonals of the matrix Q ∗ U † HU as U ranges over the unitary matrices coincides with the set diag(Q)(K) where diag(Q) is the linear transformation   q11 0 ... 0  0 q22 ... 0   Diag(Q) =   ... ... ... ...  0 0 ... qnn Moreover, if Q and H are nonnegative deﬁnite then max min λi (Q ∗ U † HU ) ≤ min(di µn−i+1 ) U

i

i

and there exists U such that equality is achieved. Such a U can be chosen from the set of U that diagonalize H while putting the eigenvalues of U † HU in reverse order by size as compared with the diagonals of Q.

Using Feedback to Improve System Identiﬁcation

63

Proof: The smallest eigenvalue of a Hermitean matrix is less than or equal to the smallest diagonal entry of the matrix. In fact, by the Schur-Horn theorem, the set of possible diagonals of U † HU , considered as the coordinates of points in Rn are bounded by, and ﬁll out, the convex hull of the n! points having coordinates (µπ(1) , µπ(2) , ..., µπ(n) ). Thus the set of points S = {(q11 h11 , q22 h22 , ..., qnn hnn )|(h11 , h22 , ..., hnn ) = (diagU† HU) ; U† U = I}) is convex and we can assert that the smallest eigenvalue of U † HU is less than or equal to the smallest component of any element of this set. Of course the product Q ∗ U † HU is diagonal if U † HU is. Thus we have the inequality of the Theorem. To show that equality can be achieved, we observe that for any U that diagonalizes H we have Q ∗ U † HU ≥ I min qii λi i

Because one can achieve an arbitrary permutation of the diagonals of a matrix by selection of a unitary matrix U and acting on the matrix by U † HU , we can rewrite the right-hand side as Q ∗ U † HU ≥ I(min qii λπ(i) ) i

which gives the result claimed. Remark 4: In the case of two-by-two symmetric matrices there is only one nontrivial permutation and it acts to interchange both the rows and the columns. In this situation it is possible to show that the Schur product of W and Π T W Π represents an improvement over the Schur product of W and W in the sense that λmin

ab ab ∗ bc bc

≤ λmin

ab cb ∗ bc ba

To see this, note that the we are asserting λmin

a2 b 2 b 2 c2

≤ λmin

ac b2 b2 ac

These matrices have the same determinant (product of the eigenvalues) and, by the geometric-arithemetic mean inequality, the matrix on the left has a smaller trace (sum of the eigenvalues). Thus the eigenvalues of W ∗ Π T W Π lie between those of W ∗ W .

8 Beyond NMR Spectroscopy The NMR problem is a linear system identiﬁcation problem but the emphasis is on the eigenvalue location rather than the accurate determination of a

64

R.W. Brockett

transfer function. Furthermore, physical principles imply that the eigenvalues of interest are lightly damped, giving rise to lengthy transient responses. As we have seen, in the NMR setting these techniques are highly eﬀective and thus it is interesting to explore their wider use, mindful of the fact that the type of additional information to be gained in this way may be more closely related to eigenvalue locations rather than direct improvements in the accuracy of the impulse response. An essential element is the need to generate one or more functions of the form y(t, τ ) = ηeΛt T eΛτ b with T being such that certain modes are made more observable than they would be based on ηeΛt alone. This requires a input channel to generate eΛt b and a mode coupler to generate T . To explore this idea in a general linear system theoretic context consider a multivariable system of the form x˙ = Ax + Bu ; y = Cx and consider the possibility of applying a time varying feedback K(t) to get x˙ = (A − BK(t)C)x(t) + Bu ; y = Cx Fact: Through repeated experiments on x˙ = (A−BK(t)C)x(t)+Bu ; y = Cx we can generate an arbitrarily good approximation to any function of the form y(t, τ ) = CeAt T eAτ B ; 0 ≤ τ < ∞ ; τ + d ≤ t < ∞ where T is any matrix that is reachable from the identity in d time units following solutions of the matrix equation ˙ X(t) = (A + BK(t)C)X(t) ; X(0) = I Explanation: The function y(t, τ ) can be generated by applying an impulse u = δ(t) at time zero, letting time evolve for τ seconds, applying the time varying gain K over the interval [τ, τ + d] and then letting the system evolve without additional input. It is necessary to ignore the output on the interval between τ and τ + d. If the values of T so reachable contains matrices that mix modes eﬀectively one can expect to be able to use two dimensional methods to enhance identiﬁa˙ bility. The characterization of the reachable set for X(t) = (A+ BK(t)C)X(t) with X(0) = I has been studied for many years and naturally begins with a determination of the Lie algebra generated by matrices of the form A+BKi C. In our paper [9] we showed that even in the case where k is a scalar but (A, b, c) controllable and observable, the Lie algebra generated in this way is of one of four types and generically the Lie algebra is all of gl(n). This means that typically the use of time varying feedback will allow us to steer the state of x˙ = Ax + k(t)bcx + bu to an open set in Rn . Thus by linearly combining

Using Feedback to Improve System Identiﬁcation

65

various functions of the form CeAt T eAτ B, including the obvious possibility CeAt eAτ B, we can achieve any T we like. Of course ﬁnding a K which generates a desired value of T may be diﬃcult if the system is completely unknown. These techniques are most likely to be useful if the system is approximately or partially known based on some preliminary experiments.

Acknowledgment. This work was supported in part by the US Army Research Office under Boston University prime GC169369 NGD, the National Science Foundation grant number ECS 0424342, DARPA under Stanford University Prime PY-1606.

References 1. Keith Glover and Jan C. Willems, “Parametrizations of Linear Dynamical Systems: Canonical Forms and Identiﬁability,” IEEE Trans. Automatic Control, volAC-19, No. 6, 1974 pp. 640-646. 2. M.R. Osborn and G. K. Smyth, “A modiﬁed Prony algorithm for Exponential Function Fitting,” J. of Applied Probability,, 1986, pp. 419-430. 3. Richard Ernst, et al. Principles of Nuclear Magnetic Resonance in One and Two Dimensions, Oxford University Press, Oxford, England, 1987. 4. Navin Khaneja et al. “Boundary of Quantum Evolution under Decoherence,” PNAS, vol 100, (2003) pp 13162-13166. 5. Navin Khaneja et al. “Time Optimal Control of Spin Systems,” Physical Review A, vol. 63 (2001). 6. R. Horn, “Doubly Stochastic Matrices and the Diagonal of a Rotation Matrix,” American Journal of Mathematics, Vol. 76,1953, pp. 620–630. 7. Bertram Kostant, “On Convexity, The Weyl Group and the Iwasawa Decomposition,” Ann. Sci. Ecole Norm. Sup., Vol. 6, (1973) pp. 413–455. ¨ 8. I. Schur, “Uber eine Klasse von Mittelbiddungen mit Anwendungen die Determinanten Theorie,” Sitzungsberichte der Berliner Mathematische Gesellschaft, 22, pages 9–20, 1923. 9. Roger W. Brockett, “Linear Feedback Systems and the Groups of Lie and Galois,” Linear Algebra and Its Applications, Vol. 50 (1983) pp. 45-60.

On the Gap Metric for Finite-Dimensional Linear Time-Varying Systems in Continuous Time Michael Cantoni The University of Melbourne, Department of Electrical and Electronic Engineering, Parkville, VIC 3010, Australia – [email protected]

Keywords: non-autonomous diﬀerential equations, gap metric, Riccati equations, (J-)spectral factorisation, robust control, approximation

Notation R, R+ , Rn Rp×m AT L2 B πt Lloc 2 L H

G∗ ΠG G⊥

The real line, non-negative reals and n-dimensional Euclidean space R × · · · × R, respectively. The space of (p × m)-dimensional matrices with entries in R. The matrix transpose of A ∈ Rp×m . The Hilbert space of square integrable functions over R+ , with inner product ·, · and norm · ; The Banach algebra of linear operators P : L2 → L2 with induced norm P := sup0=u∈L2 P u / u . The truncating projection on functions over R+ such that (πt f )(τ ) := f (τ ) if τ ≤ t, and 0 otherwise. The extended space of functions over R+ which satisfy πt f ∈ L 2 ∀ t ∈ R+ ; The space of causal (i.e. non-anticipatory) linear operators loc P : Lloc 2 → L2 satisfying πt P πt = πt P ∀ t ∈ R+ . The subspace of all P ∈ L with ﬁnite-gain P := supu∈Lloc π P u / π u . For each P ∈ H, the t t 2 , πt u=0, t∈R+ operator P |L2 ∈ B, where P |L2 denotes the restriction of P : Lloc → Lloc to L2 . When P ∈ H, the notational dis2 2 tinction between P and P |L2 is often suppressed and P is identiﬁed with the corresponding operator in B. The Hilbert adjoint of G ∈ B – y, Gu = G∗ y, u ∀ u, y ∈ L2 . The orthogonal projection onto the closed subspace G ⊂ L2 ; G ⊥ := {g ∈ L2 | g , g = 0 for all g ∈ G} = (I − ΠG1 )L2 .

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 67–78, 2006. © Springer-Verlag Berlin Heidelberg 2006

68

M. Cantoni

1 Introduction It is well-known that the gap metric, and its variants, provide a natural framework for studying the robustness of feedback interconnections [1–4]. In fact, for arbitrary linear systems, it is known that the gap metric induces the weakest topology in which both feedback stability and closed-loop performance are both robust properties [5]. In this article, we study the gap metric for the class of ﬁnite-dimensional, linear time-varying systems, in continuous-time. In particular, it is shown that the gap between two systems is equal to the norm of an operator composed of normalised left and right representations of the system graphs. The development relies on tools familiar from the time-invariant setting. This is in contrast to the abstract results which underpin the discretetime generalisations of the gap framework to the time-varying setting [6, 7]. Using ideas from [8, 9], an alternative characterisation of the gap is also established in terms of a standard linear fractional synthesis problem from robust control theory. This characterisation appears to be useful for approximation in the gap, and might be extendable to other classes of system. The article develops along the following line. Section 2 serves to introduce the class of ﬁnite-dimensional systems and to establish the required notions of stability, coprime factors and so-called graph symbols. In Section 3, the gap metric is deﬁned and shown to be equal to the norm of an operator composed of normalised left and right representations of the system graphs. The norm of this operator is then linked to the solution of a diﬀerential Riccati equation. This section also includes a discussion of the time-invariant case. The alternative characterisation of the gap is provided in Section 4, and some concluding remarks are made in Section 5.

2 Finite-Dimensional Systems The class of linear time-varying systems considered here are those generated by a ﬁnite-dimensional realisation ΣP = (A, B, C, D), of the form x(t) ˙ A(t) B(t) = y(t) C(t) D(t)

x(t) , u(t)

(1)

with initial condition x(0) = 0, where A : R+ → Rn×n , B : R+ → Rn×m , C : R+ → Rp×n and D : R+ → Rp×m are bounded and continuous functions and B and C are not identically zero. Indeed, the realisation (1) generates an operator P : Lloc → Lloc := u → y where, using the so-called variation of 2 2 parameters/constants formula [10], y(t) =

t 0

C(t)X(t)X(s)−1 B(s)u(s) ds + D(t)u(t),

(2)

with X : R+ → Rn×n taken to be a fundamental matrix for the homogeneous part of (1), x(t) ˙ = A(t)x(t) – e.g. the solution to the matrix diﬀerential

On the Gap Metric for FDLTV Systems in Continuous Time

69

˙ equation X(t) = A(t)X(t) with initial condition X(0) = I. In particular, P ∈ L (i.e. it is causal) and P is said to admit a ﬁnite-dimensional realisation. If D−1 (t) := D(t)−1 is bounded on R+ , then P −1 ∈ L with a realisation ΣP −1 = (A−BD−1 C, BD−1 , −D−1 C, D−1 ). In this case, P and its realisation ΣP , are said to be bi-causal. Given realisations ΣP1 = (A1 , B1 , C1 , D1 ) and ΣP1 = (A1 , B1 , C1 , D1 ) of two systems P1 ∈ L and P2 ∈ L, respectively, it follows directly that A1 0 B , 1 , C1 C2 , D1 + D2 B2 0 A2

ΣP1 +P2 :=

is a realisation of P1 + P2 ∈ L, and A2 B2 C1 B D , 2 1 , C2 D2 C1 , D2 D1 0 A1 B1

ΣP2 P1 :=

is a realisation of P2 P1 ∈ L. 2.1 Exponential Stability, Stabilisability and Detectability Let ΣP = (A, B, C, D) be a ﬁnite-dimensional realisation of a system P ∈ L. The homogeneous part, x(t) ˙ = A(t)x(t),

(3)

is said to be exponentially stable if there exist real constants γ > 0 and λ > 0 such that X(t)X(s)−1 ≤ γe−λ(t−s) for all 0 ≤ s ≤ t, where X(t) is any fundamental matrix for (3) [11]. In this case, it follows that the operator P := u → y, deﬁned by (2), is ﬁnite-gain stable and hence, that P ∈ H. Furthermore, note that Ξ := X −T is a fundamental matrix for the homogenous equation ˙ = −AT (t)ξ(t). ξ(t) This equation is said to be anti-stable, since Ξ(t)Ξ(s)−1 ≤ γe−λ(t−s) for 0 ≤ t ≤ s. Now, for ν ∈ Lloc 2 the function ω(t) = DT (t)ν(t) +

∞ t

B T (t)Ξ(t)Ξ −1 (τ )C T (τ )ν(τ ) dτ,

t ∈ R+ ,

(4)

is a continuous solution to the system of equations ˙ −AT (t) −C T (t) ξ(t) = B T (t) DT (t) ω(t)

ξ(t) , ν(t)

(5)

without boundary conditions [11, Chapter 3]. From this it can be seen that loc the corresponding operator P : Lloc 2 → L2 := ν → ω is anti-causal in the sense πt P πt = P πt .

70

M. Cantoni

Moreover, since ν, P u = =

∞ 0 ∞ 0

t

ν T (t) ∞

= P ∗ ν, u

t

0

C(t)X(t)X −1 (s)B(s)u(s) ds + D(t)u(t)

B T (τ )Ξ(τ )Ξ −1 (t)C T (t)ν(t) dt + DT (τ )ν(τ )

dt T

u(τ ) dτ

for any ν, u ∈ L2 , we have P |L2 = P ∗ (i.e. the Hilbert adjoint of P .) Given a realisation ΣP = (A, B, C, D), the pair (B, A) is said to stabilisable if there exists a bounded F : R+ → Rm×n such that x ˜˙ (t) = (A + BF )(t)˜ x(t) is exponentially stable. Similarly, the pair (A, C) is said to be detectable if there exists a bounded H : R+ → Rm×n such that x˜˙ (t) = (A + HC)(t)˜ x(t) is exponentially stable. Importantly, when a realisation is both stabilisable and detectable, then exponential (i.e. internal) stability is equivalent to ﬁnite-gain (i.e. input-output) stability of the causal operator P deﬁned by (2) [12, 13]. 2.2 Coprime Factors and Graph Symbols A system P ∈ L is said to admit a stable left-coprime factorisation if there ˜, M ˜ , U, V ∈ H, with M ˜ −1 , V −1 ∈ L, such that P = M ˜ −1 N ˜ and exist N ˜U +M ˜ V = I. Similarly, a P ∈ L admits a stable right-coprime factorisation N ˜ , V˜ ∈ H, with M −1 , V˜ −1 ∈ L, such that P = N M −1 if there exist N, M, U ˜ N = I. A left-coprime (resp. right-coprime) factorisation is said and V˜ M + U ˜M ˜∗+N ˜N ˜ ∗ = I (resp. M ∗ M + N ∗ N = I) on L2 . Note to be normalised if M that the superscript ∗ here denotes the Hilbert adjoint of the corresponding operator in B. Remark 1. The normalisation condition speciﬁed above is diﬀerent from the one usually employed in the time-invariant case [2, 14, 15], where A(·) ≡ A ∈ Rn×n , B(·) ≡ B ∈ Rn×m , C(·) ≡ C ∈ Rp×n and D(·) ≡ D ∈ Rp×m . In this case, normalisation is usually imposed in terms of the natural extensions of the factors to the doubly-inﬁnite time axis R. < > The L2 -graph of a system P ∈ L is deﬁned to be G :=

P dom(P ) ⊂ L2 × L2 , I

where dom(P ) := {u ∈ L2 | P u ∈ L2 }. Now suppose P admits stable left˜ and P = N M −1 , respectively. ˜ −1 N and right-coprime factorisations P = M Then with G :=

M ˜ := −N ˜ M ˜ ∈ H, ∈ H and G N

(6)

On the Gap Metric for FDLTV Systems in Continuous Time

71

˜ are, respectively, left- and right-invertible in H and it follows that G and G ˜ = 0, which implies GG ˜ = 0. In view of this, G and hence, G = GL2 and GG ˜ are called right and left graph symbols, respectively. Furthermore, since G G is left-invertible, G is closed. The right and left graph symbols are said to be normalised if G∗ G = I ˜G ˜ ∗ = I. That is, when G is an isometry and G ˜ is a co-isometry. In this and G case, G∗ ˜ G

˜∗ = I 0 . GG 0I

(7)

Correspondingly, G∗ ˜ L2 = L2 G

G∗ ˜ = G ⊥ ∩ G = {0}, and ker( ˜ ) = ker(G∗ ) ∩ ker(G) G

where ker(A) := {w ∈ L2 | Aw = 0}. Thus, ˜∗ GG

G∗ ˜ = I. G

(8)

Proposition 1. (c.f. Corollary 4.2 in [13]) Suppose that P ∈ L admits a ﬁnite-dimensional realisation. Then the following are equivalent: 1. P is feedback stabilisable, in that there exists a ﬁnite-dimensional C ∈ L such that −1 I C ∈ H; P I 2. P admits a stabilisable and detectable realisation; 3. P admits (normalised) right- and left-coprime factorisations and hence (normalised) right and left graph symbols, all with ﬁnite-dimensional realisations. Suppose the ﬁnite-dimensional realisation ΣP = (A, B, C, D) of P is stabilisable and detectable, and that D ≡ 0, for convenience. Then by construction, given any bounded F : R+ → Rm×n and H : R+ → Rn×p , such that x ˜˙ (t) = (A + BF )(t)˜ x(t) and x ˜˙ (t) = (A + HC)(t)˜ x(t) are exponentially stable, the realisations ΣM := (A + BF, B, F, I), ΣN := (A + BF, B, C, 0), ΣV˜ := (A + HC, −B, F, I), ΣU˜ := (A + HC, H, F, 0), ΣM˜ := (A + HC, H, C, I), ΣN˜ := (A + HC, B, C, 0), ΣV := (A + BF, −H, C, I), ΣU := (A + BF, H, F, 0), ˜, M ˜ , N, ˜ V, U ∈ H with zero initial conditions, generate operators M, N, V˜ , U −1 −1 ˜ ˜ such that P = N M = M N and ˜ V˜ U ˜ ˜ −N M

M −U I0 = . N V 0I

72

M. Cantoni

˜ are right- and left-coprime factorisations of P . ˜ −1 N As such, N M −1 and M Furthermore, by Lemmas 3.3 and 3.4 in [13], the so-called control diﬀerential Riccati equation ˙ −X(t) = AT (t)X(t) + X(t)A(t) − X(t)BB T (t)X(t) + C T C(t),

(9)

admits a bounded symmetric diﬀerentiable solution X(t) ≥ 0, such that x˜˙ = (A−BB T X)(t)˜ x is exponentially stable, and the so-called ﬁltering diﬀerential Riccati equation with initial condition ˙ Z(t) = A(t)Z(t) + Z(t)AT (t) − Z(t)C T C(t)Z(t) + BB T (t), Z(0) = 0, (10) admits a bounded symmetric diﬀerentiable solution Z(t) ≥ 0, such that x ˜˙ = T (A − ZC C)(t)˜ x(t) is exponentially stable. Indeed, by Theorem 4.1 in [13], ˜,N ˜ , and the with F = −B T X and H = −ZC T , the coprime factors M, N, M corresponding graph symbols, are normalised. Remark 2. Note that (10) has an initial condition, and as such, even in the time-invariant case, the normalised (in the sense deﬁned above) left-coprime factors and left graph symbols are time-varying. This is avoided in the usual time-invariant analysis by imposing normalisation in terms of the natural extensions to the doubly-inﬁnite time axis R. On the other hand, the normalised (as deﬁned here) right-coprime factors and hence, right graph symbol, can be taken to be time-invariant, because (9) has no boundary conditions. < >

3 The Gap Metric The gap, or aperture, between two closed subspaces G1 , G2 ⊂ L2 , is deﬁned to be [16, Chapter 4] →

gap(G1 , G2 ) := ΠG1 − ΠG2 = max gap(Gi , Gj ), i,j=1,2

(11)

where →

gap(Gi , Gj ) := (I − ΠGj )ΠGi is called the directed gap from Gi to Gj . Now suppose these subspaces are ˜ i Gi = 0, where Gi and G ˜i graphs and, for i ∈ {1, 2}, that Gi = Gi L2 and G are normalised right and left graph symbols for Pi ∈ L. Then the following important identities can be established for i, j ∈ {1, 2}: ΠGi = Gi G∗i ; (I − ˜∗G ˜ ΠGi ) = G i i; → ˜ ∗j G ˜ j Gi G∗i = G ˜ j Gi = G∗i G ˜ ∗j gap(Gi , Gj ) = G

=

1−

inf

0=q∈L2

G∗i Gj q q 2

2

→ ∗ ˜ ˜ i Gj = G ˜ ∗G = G i i Gj Gj =gap(Gj , Gi ).

(12)

On the Gap Metric for FDLTV Systems in Continuous Time

73

˜ ∗ is an isometry and G∗ The second and sixth equalities in (12) hold because G i i is a co-isometry, and the fourth and ﬁfth equalities hold by Lemma 1 (in the ˜ ∗ is a co-isometry appendix) and equations (7) and (8), by which G∗i Gj G j

G∗

and G˜i Gj is an isometry. Correspondingly, when the normalised right and i left graph symbols exist, we may conclude that the gap between P1 and P2 is → ˜ 2 G1 = G ˜ 1 G2 . (13) δ(P1 , P2 ) := gap(G1 , G2 ) = max gap (Gi , Gj ) = G i,j=1,2

Remark 3. Proposition 1 establishes the existence of the required graph symbols when P1 , P2 ∈ L are ﬁnite-dimensional and feedback stabilisable. Correspondingly, for such systems, the directed gaps (deﬁned in the obvious way) are equal, and hence equal to the gap. < > Remark 4. Observe the similarity between the relationships established above and the standard characterisation of the ν-gap metric for time-invariant systems [3, 17].1 < > Suppose that P1 , P2 ∈ L admit stabilisable and detectable realisations ΣP1 := (A1 , B1 , C1 , D1 ) and ΣP2 := (A2 , B2 , C2 , D2 ), respectively. Furthermore, assume that D1 = D2 ≡ 0, for convenience. Now let Aδ :=

−B2 A2 + H2 C2 H2 C1 − B2 F1 , Bδ := B1 0 A1 + F1 B1

and Cδ := C2 0 , (14)

where F := −B T X, H := −ZC T , X ≥ 0 is a stabilising solution of (9) and Z ≥ 0 is a stabilising solution of (10). Then Σδ = (Aδ , Bδ , Cδ , 0) is an ˜ 2 G1 . exponentially stable (and so stabilisable and detectable) realisation of G Proposition 2. For P1 , P2 as described immediately above, the following are equivalent: 1. δ(P1 , P2 ) < β; ˜ 2 G1 < β; 2. G 3. The diﬀerential Riccati equation ˙ + AT (t)S(t) + S(t)Aδ (t) + C T Cδ (t) + 1 S(t)Bδ B T (t)S(t) = 0, S(t) δ δ δ β2 with Aδ , Bδ , Cδ as in (14), has a symmetric diﬀerentiable solution S ≥ 0. Proof. The result follows by (13), the fact that (Aδ , Bδ , Cδ , 0) is a realisation ˜ 2 G1 , and direct application of [18, Theorem 5.4] – also see [19]. of G 1

˜ 2 term in the usual ν-gap formula [3, 17] is not normalised in the sense The G used here, so that it can be time-invariant. This goes some way to explaining the additional winding number condition in the deﬁnition of the ν-gap.

74

M. Cantoni

Remark 5. Given realisations for the normalised graph symbols, it follows by Proposition 2 that the gap can be computed to any desired accuracy by performing a bisection search, where at each step a diﬀerential Riccati equation, without boundary conditions, is solved. < > Remark 6. In the time-invariant case – i.e. Ai (·) ≡ Ai ∈ Rni ×ni , Bi (·) ≡ Bi ∈ Rni ×mi , Ci (·) ≡ Ci ∈ Rpi ×ni – the approach outlined above would involve: 1. Solving the control algebraic Riccati equation, to get the isometric right graph symbol, which is time-invariant; 2. Solving the ﬁltering diﬀerential Riccati equation, with a boundary condition, to get the co-isometric left graph symbol, which is time-varying; 3. Conducting a bisection search, where at each step a diﬀerential Riccati equation, without boundary conditions, is solved according to Proposition 2. However, as is shown in [15], it is possible to exploit the shift-invariance to obtain the following expression for the directed gap from P1 to P2 : inf

Q∈Q

G1 − G2 Q ,

(15)

where Q is the subspace of shift-invariant operators in H. Computing the value of this expression involves solving one control algebraic Riccati equation to get G1 (G2 does not have to be normalised, and as such, it can be constructed by solving an algebraic Lyapunov equation), and then a bisection search, where at each step we either: 1. Solve one algebraic Riccati equation to obtain a spectral factor and two algebraic Lyapunov equations to compute a Hankel norm [2]; or 2. Solve one algebraic Riccati equation to determine the existence of a solution to a “full-information” H∞ synthesis problem [13, 17]. Furthermore, in view of Rem. 3, conducting this bisection search for only one of the directed gaps yields the gap. < >

4 A Characterisation of the Gap for Approximation In the time-invariant case, the well-known directed gap formula (15) can be used to show the equivalence of operator balls deﬁned in terms of the gap metric and operator balls constructed by perturbing the normalised graph symbols [2]. This is suggestive of the fact that approximation in the gap is closely tied to Hankel norm approximation of the graph symbol, as is conﬁrmed in [20]. On the other hand, it not immediately clear how (14) can be used to establish this type of equivalence here. As such, an alternative characterisation of the gap is presented below, for linear time-varying systems. This

On the Gap Metric for FDLTV Systems in Continuous Time

75

characterisation would appear to be useful from the perspective of approximation. In particular, it corresponds to a standard synthesis problem from robust control, which can be solved by established methods [19]. The result is in fact similar to results established in [8, 9]. These papers employ a factorisation consisting of three factors, which implicitly underpin the framework for H2/∞ control problems in the celebrated paper by Doyle, Glover, Khargonekar and Francis [21]. The approach taken here is slightly diﬀerent. In particular, we use a J-spectral factorisation of the kind employed in [22–24], for linear time-invariant problems. A full development of the result is being prepared for publication elsewhere. Proposition 3. Suppose that P1 , P2 ∈ L are both ﬁnite-dimennsional and ˜ 1 denote a normalised left graph symbol of P1 . feedback stabilisable, and let G ¯ Furthermore, deﬁne β := supC∈L b(P1 , C), where b(P1 , C) := 1/ ΠK

G1

,

G1 and K are the L2 -graphs of P1 and C ∈ L, respectively, and ΠK G1 := (ΠG1⊥ |K )−1 ΠG1⊥ is the parallel-projection onto K along G1 . Then, for 0 < β < ¯ the following J-spectral factorisation exists β, ˜∗ βI G

I 0 0 −I

βI R11 R12 = ˜ R G 21 R22

∗

I 0 0 −I

R11 R12 =: R∗ JR, R21 R22

−1 0 with R, R−1 ∈ H and R21 ∈ L, where J is the signature matrix ( I0 −I ). Furthermore, the following are equivalent:

1. δ(P1 , P2 ) < β; 2. R[ uy22 ], JR[ uy22 ] > 0 for all [ uy22 ] ∈ G2 , where G2 is the L2 -graph of P2 ; 3. The linear fractional operator F(R, P2 ) := (R11 P2 + R12 )(R21 P2 + R22 )−1 ∈ H and F(R, P2 ) < 1. ˜ the required J-spectral factor can be Remark 7. Given a realisation of G, obtained as described in [18, Corollary 3.26]. This involves solving a diﬀerential Riccati equation, without boundary conditions. < > Remark 8. Proposition 3 establishes the equivalence required for Conjecture 1 in [8] to hold. This conjecture is used to described the diﬀerence between the so-called ν-gap [3, 17] and gap metrics for time-invariant systems. In a completely time-varying setting, there would appear to be no diﬀerence. < >

5 Concluding Remarks The linear fractional characterisation of the gap in Proposition 3 is appealing from the perspective of approximation when designing feedback control systems. Indeed, the analogous result in [9], which only involves the solutions of

76

M. Cantoni

algebraic Riccati equations, is used therein to obtain a procedure for sampleddata approximation of LTI controllers. A similar framework for model order reduction of time-invariant systems is also proposed in [25]. Much of the machinery used above can be extended, to varying degrees, in order to accommodate inﬁnite-dimensional [26], and non-linear [27] systems. It would be interesting to explore the possibility of obtaining similar characterisations of gap-like metrics for these classes of system. Finally, on a more general note, it would be interesting to more closely investigate links between gap-metric results of the type established here within an input-output framework, and ideas about approximation from the behavioural approach [28–30].

Acknowledgements The author is grateful for the ﬁnancial support of The University of Melbourne, through their Melbourne Research Grant Scheme (MRGS), and several discussions with Dr. C.-Y. Kao. Finally, the author would like to acknowledge the guiding inﬂuence of Prof. Keith Glover over the years.

A Technical Lemma Lemma 1. Let H1 , H2 and H3 be Hilbert spaces. If the linear operator [ X Y ] : H1 → H2 × H3 is an isometry, then Y

2

= 1−

inf

0=u∈H1

Xu 2 . u 2

˜ Y˜ : H1 × H2 → H3 is a co-isometry, then Similarly, if the linear operator X Y˜

2

= 1−

inf

0=u∈H1

˜ 2 Xu . u 2

Proof. If [ X Y ] is an isometry, then for any 0 = u ∈ H1 it follows that 1=

X [X Y u, Y u Xu, Xu Y ] u, [ Y ] u = + , u, u u, u u, u

where ·, · denotes the appropriate inner product. As such, the ﬁrst result follows by the deﬁnition of the induced norm. The co-isometry counterpart follows in a similar manner. Remark 9. A similar result, and a slight extension of it, lies at the heart of the ν-gap metric framework of Vinnicombe [17]. < >

On the Gap Metric for FDLTV Systems in Continuous Time

77

References 1. G. Zames and A. K. El-Sakkary, “Unstable systems and feedback: The gap metric,” in Proceedings of the Allerton Conference, 1980, pp. 380–385. 2. T. T. Georgiou and M. C. Smith, “Optimal robustness in the gap metric,” IEEE Transactions on Automatic Control, vol. 35, pp. 673–686, 1990. 3. G. Vinnicombe, “Frequency domain uncertainty and the graph topology,” IEEE Transactions on Automatic Control, vol. 38, pp. 1371–1383, 1993. 4. T. T. Georgiou and M. C. Smith, “Robustness analysis of nonlinear feedback systems: An input-output approach,” IEEE Transactions on Automatic Control, vol. 42, pp. 1200–1221, 1997. 5. M. Cantoni and G. Vinnicombe, “Linear feedback systems and the graph topology,” IEEE Transactions on Automatic Control, vol. 47, pp. 710–719, 2002. 6. W. N. Dale and M. C. Smith, “Stabilizability and existence of system representations for discrete-time time-varying systems,” SIAM Journal of Control and Optimization, vol. 31, pp. 1538–1557, 1993. 7. A. Feintuch, Robust Control Theory in Hilbert Space, ser. Applied Mathematical Sciences. New York: Springer, 1998. 8. G. Vinnicombe, “On closed-loop objectives and guaranteed robustness properties,” in Proceedings of the 35th IEEE Conference on Decision and Control, Kobe, 1996, pp. 1869–1874. 9. M. Cantoni and G. Vinnicombe, “Controller discretization: A gap metric framework for analysis and synthesis,” IEEE Transactions on Automatic Control, vol. 49, pp. 2033–2039, 2004. 10. W. Boyce and R. DiPrima, Elementary Diﬀerential Equations and Boundary Value Problems, 4th ed. John Wiley & Sons, 1986. 11. W. Coppel, Dichotomies in Stability Theory, ser. Lecture Notes in Mathematics. New York: Springer-Verlag, 1978. 12. B. Anderson and J. Moore, Optimal Control – Linear Quadratic Methods. London: Prentic-Hall, 1989. 13. R. Ravi, A. M. Pascoal, and P. P. Khargonekar, “Normalized coprime factorizations for linear time-varying systems,” Systems and Control Letters, vol. 18, pp. 455–465, 1992. 14. M. Vidyasagar, Control System Synthesis : A Factorization Approach. Cambridge (Mass.): The MIT Press, 1985. 15. T. T. Georgiou, “On the computation of the gap metric,” Systems and Control Letters, vol. 11, pp. 253–257, 1988. 16. T. Kato, Perturbation Theory for Linear Operators. New York: Springer-Verlag, 1966. 17. G. Vinnicombe, Uncertainty and Feedback - H ∞ loop-shaping and the ν-gap metric. London: Imperial College Press, 2001. 18. V. Ionescu and R. Stefan, “Generalized time-varying Riccati theory: A Popov operator based approach,” Integral Equations and Operator Theory, vol. 48, pp. 159–212, 2004. 19. R. Ravi, K. Nagpal, and P. Khargonekar, “H∞ control of linear time-varying systems - A state-space approach,” SIAM Journal on Control and Optimization, vol. 29, pp. 1394–1413, 1991. 20. T. T. Georgiou and M. C. Smith, “Upper and lower bounds for approximation in the gap metric,” IEEE Transactions on Automatic Control, vol. 38, pp. 946–951, 1993.

78

M. Cantoni

21. J. Doyle, K. Glover, P. Khargonekar, and B. Francis, “State-space solutions to the standard H2 and H∞ control problems,” IEEE Transactions on Automatic Control, vol. 34, pp. 831–847, 1989. 22. J. Ball, J. Helton, and M. Verma, “A factorization principle for stabilization of linear control systems,” International Journal of Robust and Nonlinear Control, vol. 1, pp. 229–294, 1991. 23. M. Green, K. Glover, D. Limebeer, and J. Doyle, “A J-spectral factorization approach to H∞ control,” SIAM Journal on Control and Optimization, vol. 28, pp. 1350–1371, 1990. 24. H. Kimura, Chain Scattering Approach to H ∞ Control. Boston: Birkhauser, 1997. 25. M. Cantoni, “On model reduction in the ν-gap metric,” in Proceedings of the 40th IEEE Conference on Decision and Control, Orlando, 2001, pp. 3665–3670. 26. M. Weiss, “Riccati equations in Hilbert spaces: A Popov function approach,” Ph.D. dissertation, Univeristy of Groningen, The Netherlands, 1994. 27. J. Ball and A. van der Schaft, “J-inner-outer factorization, J-spectral factorization, and robust control for nonlinear systems,” IEEE Transactions on Automatic Control, vol. 41, pp. 379–392, 1996. 28. J. W. Polderman and J. C. Willems, Introduction to Mathematical Systems Theory: A Behavioral Approach. New York: Springer Verlag, 1998. 29. B. Roorda and S. Weiland, “Optimal angle reduction: A behavioral approach to linear system approximation,” Linear Algebra and its Applications, vol. 337, pp. 189–235, 2001. 30. A. Sasane, “Distance between behaviours,” International Journal of Control, vol. 76, pp. 1214–1223, 2003.

Robustly Stabilizing Controllers with Internal Loop Ruth F. Curtain Mathematics Institute, University of Groningen, P.O. Box 800, 9700 AV Groningen, the Netherlands [email protected]

1 Introduction The problem of robust stabilization with respect to left coprime factor perturbations was ﬁrst solved explicity for the rational case in Glover and McFarlane [14]. The irrational case was treated in Georgiou and Smith [12], where it was shown that a solution exists provided that the transfer function has a normalized doubly coprime factorization. Our new contribution is to solve this classic problem for the following very general class of transfer functions G(s) ∈ L(U, Y ), where U, and Y are separable Hilbert spaces and G has the following realization: • G(s) = G(0) + sC(sI − A)−1 A−1 B; • A is a closed, densely deﬁned operator on a separable Hilbert space Z such that ρ(A) contains 0 and a right half-plane and there exist M > 0 and n ∈ Z such that (sI − A)−1 ≤ M |s|n on some right half-plane; • C ∈ L(D(A), Y ); • B ∗ ∈ L(D(A∗ ), U ). This class and even more general ones were introduced in Opmeer [22]. They include all delay systems and most p.d.e. systems, including those that are not well-posed ones (see Staﬀans [24]). The extension to inﬁnite-dimensional U and Y is entirely new and it supersedes all state-space classes previously studied in the literature. The frequency domain approach in [12] contains an existence result assuming that the transfer function has a normalized doubly coprime factorization, but no suﬃcient conditions for the existence of such factorizations are given and no formulas for the controllers are derived. Our state-space approach is in the style of [14] which uses an explicit normalized doubly coprime factorization of the transfer function G in terms of the system parameters A, B, C, D and results from the Nehari problem to obtain an explicit parametrization of all robust controllers. Previous results that developed this approach for inﬁnite-dimensional linear systems include Curtain and Zwart [3, Chapter 9.4], Curtain [2] and Oostveen [20]. The class of systems we consider is far larger than any of these B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 79–98, 2006. © Springer-Verlag Berlin Heidelberg 2006

80

R.F. Curtain

classes. This generalization was made possible by the recent results in Curtain and Opmeer [7] on the existence of normalized doubly coprime factorizations and in Curtain and Opmeer [6] on the Nehari problem. A new feature is that we use controllers with internal loop as introduced in Weiss and Curtain [26] and Curtain, Weiss and Weiss [9]. In Section 2 we generalize the results on controllers with internal loop from [9] to allow for transfer functions that are not well-posed. In Section 3 we formulate the problem and reduce it to a Nehari problem for a normalized coprime factorization. The strategy is to ﬁrst solve the problem for a special class of state linear systems, Σ(A, B, C, D), where A generates a C0 semigroup, B and C are bounded and G(s) = D + C(sI − A)−1 B for some D ∈ L(U, Y ) (see Curtain and Zwart [3]). The relevant results for state linear systems on the Nehari problem from [6] are summarized in Section 4 and those on normalized doubly coprime factorizations from [7] in Section 5. The solution to the robustness problem for state linear systems contained in Section 6 is a nice generalization of the results in [19]: If the system and its dual both satisfy a ﬁnite-cost condition, then we obtain the expected parametrization of inﬁnitely many robustly stabilizing conventional controllers. Unlike the rational case, we are not sure that all solutions have this form (c.f. Georgiou and Smith [13]). An interesting corollary is the interpretation of the strange looking B´ezout factors [X; Y] found in [7] as those that produce the robustly stabilizing controller K = YX−1 . In Section 7 we extend these results to the very general class of irrational transfer functions speciﬁed above using the reciprocal approach as in Curtain and Opmeer, [4], [5], [6] and [7]. If the system and its dual both satisfy a ﬁnite-cost condition and 0 ∈ ρ(A), then we can solve the robust stabilization problem by solving one for its reciprocal system that is a state linear system. We obtain an explicit parametrization of inﬁnitely many robustly stabilizing controllers with internal loop for the original system. If either both U and Y are ﬁnite-dimensional or the component of ρ(A) that contains a right halfplane also contains the origin, then the controllers with internal loop can be realized as conventional controllers. We remark that, as in Curtain [4], the condition 0 ∈ ρ(A) can be replaced by a condition iω ∈ ρ(A) for some real ω. It is also possible to remove these conditions altogether by using the connection with discrete-time systems via a Cayley transform as in [21] and [22]. The price one pays is much more complicated formulas. We apply the theory to a class of dissipative systems with colocated actuators and sensors that includes the well-posed ones studied in Curtain and Weiss [8].

2 Controllers with Internal Loop In order to formulate our problem we need to deﬁne the class of transfer functions, normalized coprime factorizations and especially our concept of a stabilizing controller with internal loop.

Robustly Stabilizing Controllers with Internal Loop

81

The class of stable transfer functions we consider are elements from H∞ (L(Z)), where Z is either the Hilbert space U or Y or elements from H∞ (L(U, Y )), or H∞ (L(Y, U )). For simplicity of notation we denote any of these classes by HH∞ . The class of unstable transfer functions we consider are quotients of the form G = NM−1 , where N ∈ H∞ (L(U, Y )) and M ∈ H∞ (L(U )) has an ˜ −1 N ˜, inverse on some nonempty subset S of C0+ = {s : Res > 0} or G = M ˜ ˜ where N ∈ H∞ (L(Y, U )) and M ∈ H∞ (L(Y )) has an inverse on S. For simplicity of notation we denote any of these classes by HH∞ /H∞ (S). This deﬁnes a very general algebra of transfer functions and the factorization approach to controller synthesis developed in Vidyasagar [25] can be applied. The case with ﬁnite-dimensional U, Y and the inverses are deﬁned almost everywhere on C0+ is treated in Georgiou and Smith [12]. The well-posed case where the inverses are deﬁned and bounded on some right half-plane is treated in Curtain, Weiss and Weiss [9] and we extend this approach here to allow for systems that are not necessarily well-posed. Our deﬁnition of coprimeness is standard, but we use a Matlab notation for row blocks [W, T ] = [W T ] and column blocks [W ; T ] = [W ∗ T ∗ ]∗ . Deﬁnition 1 The transfer function G ∈ HH∞ /H∞ (S) has a right-coprime ˜ Y] ˜ ∈ factorization over S if there exist [N; M] ∈ H∞ (L(U, Y ⊕ U )) and [X, H∞ (L(U ⊕ Y, U )) such that M has an inverse on S, G(s) = N(s)M(s)−1 on S and for all s ∈ C+ 0 there holds ˜ ˜ X(s)M(s) − Y(s)N(s) = I. ˜ M] ˜ ∈ H∞ (L(U ⊕ It has a left-coprime factorization over S if there exist [N; ˜ has an inverse on S, Y, Y )) and [X; Y] ∈ H∞ (L(Y, Y ⊕ U )) such that M −1 ˜ ˜ G(s) = M(s) N(s) on S and for all s ∈ C+ 0 there holds ˜ ˜ M(s)X(s) − N(s)Y(s) = I. G has a doubly coprime factorization if there exist a right-coprime factoriza˜ −1 N ˜ over S tion G = NM−1 over S and a left-coprime factorization G = M ˜ ˜ and for certain [X, Y] ∈ H∞ (L(U ⊕ Y, U )) and [X; Y] ∈ H∞ (L(Y, Y ⊕ U )) on C+ 0 there holds ˜ −Y ˜ MY X = ˜ ˜ N X −N M

−1

.

(1)

It is normalized if, in addition, for almost all ω ∈ R ∗ ∗ ˜ ˜ ˜ ˜ M(iω)∗ M(iω) + N(iω)∗ N(iω) = I = M(iω) M(iω) + N(iω) N(iω) . (2)

First we give the natural generalization of the conventional concept of stabilization for transfer functions in our algebra class. Deﬁnition 2 Assume that G and C are in HH∞ /H∞ (S). We say that C is an admissible feedback transfer function for G if I − CG (or equivalently

82

R.F. Curtain

I − GC) has an inverse in HH∞ /H∞ (S). An admissible feedback transfer function C for G stabilizes G if I −C −G I

−1

∈ H∞ (L(U× Y )).

While it is straightforward to generalize the factorization theory in [25] for this class, it has fundamental drawbacks that have been pointed out in [26] and [9]. They proposed a new concept of a controller with internal loop for well-posed systems (the transfer function is deﬁned and bounded on some right half-plane S). In fact their theory easily extends to non well-posed systems. We begin with the formal deﬁnition. K11 K12 K21 K22 be transfer functions in HH∞ /H∞ (S) with values in L(U, Y ) and in L(Y × G 0 . We say that K is an adR, U × R), respectively, and denote P = 0 K missible feedback transfer function with internal loop for G if I − F P has a well-deﬁned inverse on S, where   0I0 F = I 0 0 , F ∈ L(Y × U× R, U× Y × R). (3) 00I Deﬁnition 3 Let U , Y and R be Hilbert spaces. Let G and K =

In particular, we say that K stabilizes G with internal loop if (I − F P)−1 ∈ H∞ (L(U× Y × R)). The intuitive interpretation of Deﬁnition 3 is the following: G represents the system to be stabilized and K is the transfer function of the controller in Figure 1 from [yk ; ζi ] to [uk ; ζo ], when all the connections are open. The connection from ζo to ζi is the so-called internal loop. The admissibility part of the deﬁnition means that it is possible to close all the connections in Figure 1, and obtain well-deﬁned transfer functions in HH∞ /H∞ (S) from the three external inputs (v, r and ξ) to all the other signals. The stabilization part of the deﬁnition means that in fact all these closed-loop transfer functions are stable. If I − K22 has an inverse in HH∞ , then the above stability concept coincides with the conventional one in Deﬁnition 2 with the controller K11 + K12 (I − K22 )−1 K21 . In [9] it was shown that with the concept of stabilization with internal loop one obtains a nice generalization of the factorization theory in [25], including a more elegant Youla-Biongiorno factorization. The proofs there were for wellposed systems G, but they work for our more general algebra of transfer functions. In [9] they introduce canonical controllers with the form K=

0 I , K21 K22

with K21 , K22 ∈ H∞ ,

(4)

Robustly Stabilizing Controllers with Internal Loop

v

up

+ +

83

yp

G

uk

yk

r

+ +

K

+

ζo

ζi

+

ξ

Fig. 1. The system G connected to a controller K with internal loop

and dual canonical controllers with the form K=

0 K12 , I K22

with K12 , K22 ∈ H∞ .

(5)

We summarize the fundamental properties of the canonical controllers. Theorem 4 Let G be in HH∞ /H∞ (S). a. The canonical controller K stabilizes G with internal loop iﬀ D0 := I − K22 − K21 G is invertible on S and D0−1 , GD0−1 ∈ HH∞ . b. If G has a right-coprime factorization G = NM, then K stabilizes G with internal loop iﬀ D := M − K22 M − K12 N is invertible over HH∞ . c. G has a right-coprime factorization if and only if G is stabilizable with internal loop by a canonical controller. d. If G has a doubly coprime factorization as in Deﬁnition 1, then all canonical controllers which stabilize G with internal loop are given by K=

0 ˜ ˜ ˜ E(Y + J M)

I ˜ ˜ ˜ , I − E(X + J N)

˜ ∈ HH∞ and E ˜ is invertible over HH∞ . where J, E We call J appearing in Theorem 4 and in the following dual theorem the Youla parameter of K. Theorem 5 Let G be in HH∞ /H∞ (S). ˜ 0 := a. The dual canonical controller K stabilizes G with internal loop iﬀ D ˜ −1 G ∈ ˜ −1 , D I − K22 − GK12 is invertible on some right half-plane and D 0 0 HH∞ .

84

R.F. Curtain

˜ then K stabilizes G ˜ −1 N, b. If G has a left-coprime factorization G = M ˜ := M ˜ − MK ˜ 22 − NK ˜ 12 is invertible over HH∞ . with internal loop iﬀ D c. G has a left-coprime factorization if and only if G is stabilizable with internal loop by a dual canonical controller. d. If G has a doubly coprime factorization as Deﬁnition 1, then all dual canonical controllers which stabilize G with internal loop are given by K=

0 I

(Y + MJ)E , I − (X + NJ)E

where J, E ∈ HH∞ and E is invertible over HH∞ . All stabilizing controllers with internal loop are equivalent to a (dual) canonical controller. To explain what is meant by ”equivalent” let us analyze the connection in Figure 1 in more detail. We use the auxiliary inputs v and r from Figure 1, so that up = uk + v and yk = yp + r, while assuming that ξ = 0. If K stabilizes G with internal loop, then we can deﬁne the transfer function TG,K ∈ H∞ by [ˆ up ; yˆk ] = TG,K [ˆ v ; rˆ]. We call TG,K the compensation operator of the pair (G, K) and if K1 and K2 are two stabilizing controllers with internal loop for G, we call them equivalent if their compensation operators (G, K1 ) and (G, K2 ) are equal. Theorem 6. Suppose that G ∈ HH∞ /H∞ (S) and K is a stabilizing controller with internal loop for G. Then there exists a unique J ∈ HH∞ such that K is equivalent to the canonical controller Kl =

0 I ˜ + JM ˜ I − (X ˜ + J N) ˜ , Y

and is also equivalent to the dual canonical controller Kr =

0 Y + MJ . I I − (X + NJ)

The formulae for the parameterization of all stabilizing (dual) canonical controllers in Theorems 4 and 5 are reminiscent of the conventional Youla parameterization. Consequently, we expect a strong relationship between stabilization with internal loop and the usual concept of stabilization. Lemma 1. Let G and C be in HH∞ /H∞ (S). Assume that C has a rightcoprime factorization C = K12 (I − K22 )−1 over S. Then G is stabilized by C iﬀ G is stabilized with internal loop by the dual canonical controller 0 K12 . K= I K22 Of course the above proposition has an obvious dual version.

Robustly Stabilizing Controllers with Internal Loop

85

3 Formulation of the Problem We aim to design controllers with internal loop that stabilize transfer functions in HH∞ /H∞ (S) robustly with respect to certain perturbations. We now deﬁne the type of perturbations we consider. ˜ M] ˜ Deﬁnition 7 Let G ∈ HH∞ /H∞ (S) have a left-coprime factorization [N, over S. ε-left-coprime factor perturbations of G are transfer functions ˜ + ΔM )−1 (N ˜ + ΔN ), GΔ = (M ˜ + ΔM is invertible over S, Δ = [ΔN , ΔM ] ∈ HH∞ and Δ where M

∞

< ε.

Our robust stabilization problem for G is the following. Given ε > 0 ﬁnd controllers with internal loop K ∈ HH∞ /H∞ (S) that stabilize all ε-left-coprime factor perturbations of G with internal loop. Moreover, ﬁnd the maximum robustness margin εmax . In Glover and McFarlane [14] it was shown that with this particular class of perturbations one can obtain explicit solutions for the rational case. The following lemma is a generalization of a key result in [14]. As explained in Section 2, we can without loss of generality consider dual canonical controllers. Theorem 8 Suppose that G ∈ HH∞ /H∞ (S) has a normalized doubly coprime factorization over S and K ∈ HH∞ is a dual canonical controller given by (5). Let 0 < ε < 1. If K stabilizes G with internal loop and K12 ˜ −1 ∞ ≤ 1 , (6) D I − K22 ε ˜ = M(I ˜ − K22 )− NK ˜ 12 , then it stabilizes all ε-admissible left-coprime where D factor perturbations of G with internal loop. Proof. From Theorem 5 K stabilizes GΔ with internal loop if and only if ˜ Δ = (M ˜ + ΔM )(I − K22 ) − (N ˜ + ΔN )K12 is invertible over HH∞ . Now D ˜Δ = D

I − −ΔM ΔN

I − K22 ˜ −1 ˜ D D K12

I − K22 ˜ −1 D is in HH∞ K12 and it has norm strictly less than one. So I − Q is invertible over HH∞ and ˜ Δ = (I − Q)D. ˜ so is D So if (6) holds, then the term Q = −ΔM ΔN

In the following theorem we show how the solution to (6) is related to a Nehari problem. Since we only use it in Section 5 for S = SRHP , some right half-plane, it suﬃces to prove a version for G ∈ HH∞ /H∞ (SRHP ) and for controllers with right-coprime factorizations. We use the notation G† (s) = G(−¯ s)∗ Theorem 9 Suppose that G has a normalized doubly coprime factorization over SRHP and let 0 < ε < 1. The following statements are equivalent.

86

• •

R.F. Curtain

K with a right-coprime factorization over SRHP stabilizes G and (6) holds; there exist W, V ∈ MH∞ such that V is invertible on SRHP and ˜† W −N + ˜ V M†

•

∞≤

1 − ε2 ;

(7)

K has a right-coprime factorization K = WV−1 on some right half-plane, where W, V satisfy (7).

Proof. The proof follows the same arguments as in [20, Theorem 7.2.3], except for one step. First note that all the terms in (1) have boundary functions on iR and we denote their values by M(iω), N(iω), etc., for almost all ω ∈ R. All arguments in this proof are made for a ﬁxed ω and apply almost everywhere M(iω)∗ N(iω)∗ is co-isometric: on R. It is readily veriﬁed that W(iω) = ˜ ˜ M(iω) −N(iω) W(iω)W(iω)∗ = I. In [20, Theorem 7.2.3] it is a square matrix, and so trivially we have W(iω)∗ W(iω) = I. It is this isometric property that we need to prove for inﬁnite-dimensional U, Y . We show that W is one-to-one on L2 , i.e., W[u; y] = 0 =⇒ [u; y] = 0, where for brevity we omit the argument ˜ = NM ˜ ˜ M]. ˜ (iω). From (1) we have MN and so Ran[M; N] ⊂ Ker[−N, Let [x; y] ∈ the closure of Ran[M; N]. Then there exist a sequence un ∈ U such that [x; y] = lim→∞ [Mun ; Nun ]. Using (2) we obtain M∗ Mun + N∗ Nun = un and since the left-hand side converges to M∗ x + N∗ y, un has a limit u. Thus [x; y] = [Mu; Nu] and Ran[M; N] is closed, yielding the decomposition [U ; Y ]=Ran[M; N]⊕ Ran[M; N]⊥ = Ran[M; N]⊕ Ker[N∗ , M∗ ]. We now show ˜ M] ˜ by a contradiction argument. that Ran[M; N]=Ker[−N, ˜ ˜ but not in Ran[M; N]. Then, from the above Suppose that x ∈ Ker[−N, M], ˜ M]∩ ˜ ˜ − Nx ˜ = decomposition of [U ; Y ], [x; y] ∈ Ker[−N, Ker[M∗ , N∗ ], i.e., My ∗ ∗ 0 = M x + N y. Using (1) gives ˜ −Y ˜ MY X [x; y] = [x; y] ˜ ˜ N X −N M ˜ − and premultiplying both sides by [M∗ , N∗ ] and using (2) shows that Xx ˜ Yy = 0. Substituting this back into the previous equation shows that [x; y] = 0. So Ker[M∗, N∗ ] ∩ Ran [N; M] = {0} almost everywhere on iR and ˜ ˜ Ker W(iω) = Ker [N(iω)∗ , M(iω)∗ ] ∩ Ker [−N(iω), M(iω)] = {0}. The rest of the proof is as in [20, Theorem 7.2.3].

4 The Suboptimal Nehari Problem In this section we review known results on the Nehari problem from Curtain and Opmeer [6]. It is well-known (see Page [23]) that for G ∈ L∞ (iR; L(U, Y ))

Robustly Stabilizing Controllers with Internal Loop

inf

F(−s)∈H∞ (L(U,Y ))

G+F

∞=

HG ,

87

(8)

where HG is the Hankel operator with symbol G. We note that in the following L∞ denotes the class of weakly measurable essentially bounded functions and that all H∞ functions have boundary functions in this class. Deﬁnition 10 For G ∈ L∞ (iR; L(U, Y )) we deﬁne the Hankel operator with symbol G as the operator HG : H2 (U ) → H2 (Y ) given by HG f = Π(ΛG f− ) for f ∈ H2 (U ), where ΛG is the multiplication map on L2 (iR; U ) induced by G, Π is the orthogonal projection from L2 (iR; U ) onto H2 (U ) and f− (s) := f (−s). From Theorem 9 we see that to obtain formulas for our robustly stabilizing controllers we need an explicit solution to the suboptimal Nehari problem, which is to ﬁnd for any σ > HG a F(−s) ∈ H∞ (L(U, Y )) such that G + F ∞ ≤ σ. We only need solutions for the special class of system-stable state linear systems Σ(A, B, C, D), which we deﬁne below. Deﬁnition 11 Let A be the generator of a strongly continuous semigroup T (·) on a separable Hilbert space Z, B ∈ L(U, Z), C ∈ L(Z, Y ), D ∈ L(U, Y ) with U, Y separable Hilbert spaces. We call Σ(A, B, C, D) a state linear system. It is system-stable if • • •

it is input stable: there exists a constant β > 0 such that for all u with compact support in L2 (0, ∞; U ) ∞ ∞ 2 ≤ β 0 u(t) 2 dt; 0 T (t)Bu(t) dt it is output stable: there exists a constant γ > 0 such that for all z ∈ Z ∞ CT (t)z 2 dt ≤ γ z 2; 0 it is input-output stable: the transfer function G = D + C(sI − A)−1 B ∈ H∞ (L(U, Y )).

That these concepts of input and output stability are appropriate ones is clear from the following known results from [15] and [16]. Lemma 2 The state linear system Σ(A, B, C, D) is output stable if and only if the following observability Lyapunov equation has a self-adjoint nonnegative solution L ∈ L(Z) : A∗ Lz + LAz = −C ∗ Cz for all z ∈ D(A). The observability Gramian LC is the smallest bounded nonnegative solution. The state linear system Σ(A, B, C, D) is input stable if and only if the following controllability Lyapunov equation has a self-adjoint nonnegative solution L ∈ L(Z) : ALz + LA∗ z = −BB ∗ z for all z ∈ D(A∗ ). The controllability Gramian LB is the smallest bounded nonnegative solution.

88

R.F. Curtain

These gramians are the key to ﬁnding the Hankel norm of a system. Lemma 3 ( [20, Lemma 7.1.5], [6, Theorem 4.4]) Suppose that Σ(A, B, C, D) is a system-stable state linear system with impulse response h(t) = CT (t)B and transfer function G. For u ∈ L2loc ([0, ∞); L(U, Y )) of compact support we deﬁne the time domain Hankel operator associated with h by ∞ Γh u(t) = 0 h(t + s)u(s) ds. 1. Γh is bounded from L2 (0, ∞; U ) to L2 (0, ∞; Y ); 1 2. HG = Γh = r 2 (LB LC ), where r denotes the spectral radius and LB , LC are the controllability and observability gramians, respectively, of Σ(A, B, C, D). 3. If σ > Γh , then Nσ := (I − σ −2 LB LC )−1 ∈ L(Z). Moreover, W = + Nσ LB is self-adjoint and nonnegative and ρ(A)∩C+ 0 ⊂ ρ(AW )∩C0 , where −2 ∗ AW = A − σ W C C. First we quote the result concerning the existence of at least one solution to the suboptimal Nehari problem from [7, Theorem 6.5]. Theorem 12 Suppose that Σ(A, B, C, D) is a system-stable state linear system with transfer function G and controllability and observability grami1 ans LB , LC , respectively. If σ > HG = r 2 (LB LC ) and Nσ := (I − −2 −1 σ LB LC ) , then Fc (−s) = Z(s) ∈ H∞ (L(U, Y )) satisﬁes G+ Fc ∞ ≤ σ, where Z(s) = −D − σ −2 CW (sI − A∗ + σ −2 C ∗ CW )−1 LC B. The following theorem that parametrizes inﬁnitely many solutions to the suboptimal Nehari problem follows easily from the above and from [6, Theorem 7.1]. Theorem 13 Suppose that Σ(A, B, C, D) is a system-stable state linear system with transfer function G and controllability and observability grami1 ans LB , LC , respectively. If σ > HG = r 2 (LB LC ) and Nσ := (I − −2 −1 σ LB LC ) , then solutions F(−s) ∈ H∞ (L(U, Y )) to the suboptimal Nehari problem are given by F(−s) = R1 (s)R2 (s)−1 , where [R1 (s); R2 (s)] = V(¯ s)∗ [Q(−s); IU ], and V is the transfer function of the state linear system Σ(A, σ −2 [−LB C ∗ − BD∗ , B], [C; σ −1 B ∗ LC ]Nσ , Dσ ), where IY 0 and Q(−s) ∈ H∞ (L(U, Y )) satisﬁes Q ∞ < 1. Dσ = −σ −1 D∗ σ −1 IU The above represents a nice generalization of the ﬁnite-dimensional solutions. The diﬀerences are that that we are not sure that all solutions have such a parametrization and that parameters Q with norm one are excluded. Under either of the additional assumptions that U, Y are ﬁnite-dimensional or σ(A)∩ iR has measure zero parameters Q with norm one are allowed ( [6, Theorem 7.1]).

Robustly Stabilizing Controllers with Internal Loop

89

5 Normalized Coprime Factorizations We consider the special class of state linear systems Σ(A, B, C, D). As discovered by Meyer and Franklin [17], there is a strong connection between normalized coprime factorizations and the linear quadratic regulator problem or, more precisely, the optimal control problem minu∈L2 (0,∞;U)

∞ 0

y(t)

2

+

u(t)

2

dt,

where z(t) ˙ = Az(t) + Bu(t); y(t) = Cz(t) + Du(t). It is well-known (see Curtain and Zwart [3, Chapter 6]) that if Σ(A, B, C, D) satisﬁes the ﬁnite-cost condition, i.e., for all initial states z0 ∈ Z there exists an input u ∈ L2 (R+ ; U ) such that the output y ∈ L2 (R+ ; Y ), then for each z0 ∈ Z there exists a unique uopt ∈ L2 (R+ ; U ) for which the minimum is attained and there exists a bounded nonnegative operator Qopt such that the minimal cost is given by Qopt z0 , z0 . The optimal input is a state feedback: uopt (t) = F opt z opt (t), where F opt := −S −1 (D∗ C + B ∗ Qopt ), S := I + D∗ D. Qopt is the smallest bounded nonnegative solution to the control algebraic Riccati equation on D(A) A∗ Q + QA + C ∗ C = (QB + C ∗ D)S −1 (B ∗ Q + D∗ C),

(9)

where F := −S −1 (D∗ C + B ∗ Q) and S := I + D∗ D. Let AQ = A + BF . If the dual system Σ(A∗ , C ∗ , B ∗ , D∗ ) satisﬁes the ﬁnite-cost condition, then then there exists a smallest bounded nonnegative solution P opt to the ﬁlter Riccati equation on D(A∗ ) AP + P A∗ + BB ∗ = (P C ∗ + BD∗ )R−1 (CP + DB ∗ ),

(10)

where R := I + DD∗ and L := −(P C ∗ + BD∗ )R−1 . Let AP = A + LC. In [6] the following extra properties were shown to be useful. Lemma 4 If both Σ(A, B, C, D) and its dual satisfy the ﬁnite-cost condition, then the left-factor system Σ(AP , R−1/2 [B + LD, L], C, R−1/2 [D, I]), and the right-factor system Σ(AQ , BS −1/2 , [C + DF ; F ], [D; I]S −1/2 ) are both systemstable. The closed-loop generators AQ and AP are related by (I + P Q)AQ z = AP (I + P Q)z for z ∈ D(A),

(11)

+ and so ρ(AQ ) = ρ(AP ). Moreover, ρ(A) ∩ C+ 0 ⊂ ρ(AQ ) ∩ C0 and ρ(A) ∩ iR ⊂ ρ(AQ ) ∩ iR.

Henceforth we choose the smallest solutions to the Riccati equations and drop the suﬃx “opt”. In this case, P and Q are the controllability, respectively, observability gramians of the left- and right-factor systems. The observability gramian of the left-factor system is Q(I + P Q)−1 and the controllability

90

R.F. Curtain

gramian of the right-factor system is P (I + P Q)−1 (see [7, Lemma 4.9]). So from Lemma 3 we see that these factor systems have the same Hankel norm r1/2 (P Q(I +P Q)−1 ) whisch is strictly less than 1. So by choosing the smallest bounded nonnegative solutions, the transfer functions are normalized which implies that they are coprime. We summarize these new suﬃcient conditions for Σ(A, B, C, D) to have normalized left- and right-coprime factorizations over H∞ from [7]. The candidates for the coprime factors agree with the known ﬁnite-dimensional formulas, but the B´ezout factors are surprizingly diﬀerent. Theorem 14 If both Σ(A, B, C, D) and its dual satisfy the ﬁnite-cost condi˜ , M] ˜ of the left-factor system Σ(AP , [B + tion, then the transfer function [N −1/2 −1/2 LD, L], R C, R [D, I]), and the transfer function [N; M] of the rightfactor system Σ(AQ , BS −1/2 , [C + DF ; F ], [D; I]S −1/2 ) form a normalized doubly coprime factorization of G, provided that we choose P and Q to be the smallest nonnegative solutions to their respective Riccati equations. ˜ , M] ˜ and of [N; M] equal r1/2 (P Q(I + P Q)−1 ) < 1. The Hankel norm of [N A Bezout pair corresponding to the left-coprime factor is the transfer function of the system-stable state linear system ˜ = (I + P Q − Σ (A˜F , P C ∗ R−1/2 , [C + DF˜ ; F˜ ], [I, −D∗ ]R−1/2 ), where W −2 −1 ˜ −1 −2 ∗ ∗ ˜ ˜ σ P Q) , F = −S (σ B QW + D C), AF = A + BF˜ and σ is such that r1/2 (P Q(I + P Q)−1 ) < σ < 1. A Bezout pair corresponding to the rightcoprime factor is the transfer function of the system-stable state linear system ˜ L ˜ ], S −1/2 B ∗ Q, S −1/2 [I, −D∗ ]), where L ˜ = −(σ −2 W ˜ P C∗ + Σ(A˜L , [B + LD, ∗ −1 ˜ ˜ BD )R and AL = A + LC . If σ = 1, the formulas for the B´ezout factors reduce to the same formulas as in the ﬁnite-dimensional case. However, in general we can only show that these σ = 1 B´ezout factors are in H2 (modulo a constant) and not necessarily in H∞ . The B´ezout factors X, Y are obtained by solving the Nehari problem ˜ , M] ˜ + [F1 , F2 ] ≤ σ, where r1/2 (P Q(I + P Q)−1 ) < σ < 1 and they are [− N given by ˜ − NW ˜ )−1 , Y = W(MV ˜ − NW ˜ )−1 , X = V(MV ∗ ∗ where [W(s), V(s)] = [F1 (−s¯) , F2 (−s¯) ].

6 Robustly Stabilizing Controllers for State Linear Systems In this section we generalize the results in [20] to allow for a larger class of state linear systems that are not necessarily strongly stabilizing and U, Y are allowed to be inﬁnite-dimensional. To obtain formulas for our robustly stabilizing controllers we need an explicit solution to the suboptimal Nehari problem (7). Applying the Page result (8) and Theorems 8, 9 and 14 to our robust stabilization problem, we see that

Robustly Stabilizing Controllers with Internal Loop

91

we can only hope to solve (7) for ε < εmax , the maximum robustness margin, which is given by ε2max = 1−

H[−N, ˜ M] ˜

2

= 1 − r(P Q(I + P Q)−1 ).

(12)

The solution to our robust stabilization problem for state linear systems now follows readily from the known results summarized in Sections 2-4. Theorem 15 If the state linear system Σ(A, B, C, D) and its dual satisfy the ﬁnite-cost condition, then the robust stabilization problem is solvable for all ε < εmax , where εmax = (1 + r(P Q))−1/2 ,

(13)

and P, Q are the smallest bounded nonnegative solutions to the Riccati equations (10) and (9), respectively. Robustly stabilizing controllers with a robustness margin of ε, 0 < ε < εmax , are given by the following right-coprime factorizations over SRHP C(s) = [φ11 (s)J(s) + φ12 (s)][φ21 (s)J(s) + φ22 (s)]−1 .

(14)

φ is the transfer function of the state linear system Σ(A + BF, T, σ −2 [F ; C + DF ], Dφ ), where F = −S −1 (D∗ C + B ∗ Q), T = (I + P Q − σ −2 P Q)−1 [ε−1 BS −1/2 , −σ −1 P C ∗ R−1/2 ], 1 ε−1 S −1/2 σ −1 D∗ R−1/2 Dφ = −1 , σ = (1 − ε2 ) 2 , −1/2 −1 −1/2 ε DS σ R and J ∈ H∞ (U ) satisﬁes J

∞

< 1.

Proof. From Theorem 14 we see that a suitable choice for a doubly normalized coprime factorization is given by the transfer functions of the left- and rightfactor systems where P, Q are chosen to be the smallest bounded nonnegative solutions to the Riccati equations (10) and (9), respectively. From Theorem 9 we see that the robust stabilization problem has a solution if H[−N, < 1 and this also holds by Theorem 14. As explained above, ˜ M] ˜ the maximum robustness margin satisﬁes (12). Note that since both P and Q are nonnegative, I + P Q is boundedly invertible. We show that r(P Q(I + P Q)−1 ) =

r(P Q) . 1 + r(P Q)

For λ = 1 it is readily veriﬁed that (λI − P Q) has a bounded inverse if and λ only if 1+λ I − P Q(I + P Q)−1 has. So λ < r(P Q) =⇒

λ < r(P Q(I + P Q)−1 ), and 1+λ

92

R.F. Curtain

r(P Q) ≤ r(P Q(I + P Q)−1 ). 1 + r(P Q) Similarly,

µ < r(P Q(I + P Q)−1 ) =⇒

µ < r(P Q), 1−µ

which implies the reverse inequality and (13) holds. From Theorem 9 the parametrization of the robustly stabilizing controllers with a robustness margin less than εmax proceeds by ﬁnding a parametrization of W, V that solve (7). This is done by applying Theorem 13 and ﬁnally C = WV−1 on SRHP . These formulas can be simpliﬁed as in Curtain and Zwart [3, Theorem 9.4.15]. Finally we show that the central controller (J = 0) corresponds to the controller obtained using the B´ezout factors from Theorem 14. Lemma 5 Under the assumptions of Theorem 15, the central controller Cc obtained by substituting J = 0 in (14) coincides with the controller −1 Cb = √ YX , where [X; Y] are the B´ezout factors from Theorem 14 with σ = 1 − ε2 . ˆ −1 W ˜ P C∗, Proof. Cc (s) = φ12 (s)φ22 (s)−1 = −D∗ − σ −2 B ∗ Q(sI − A) −2 −1 −2 ˜ ∗ −1 ˜ ˆ where W = (I + P Q − σ P Q) and A = AQ − σ W P C R (C − DB ∗ Q). Cb (s) = Y(s)X(s)−1 = −D∗ + F˜ (sI − A˜F )−1 P C ∗ (I + (C + DF˜ )(sI − A˜F )−1 P C ∗ )−1 ˜ −1 P C ∗ ) = −D∗ + F˜ (sI − A˜F )−1 P C ∗ (I − (C + DF˜ )(sI − A) ˜ (sI − A) ˜ −1 P C ∗ , = −D∗ − σ −2 B ∗ QW ˜ . To prove the equality of the where A˜ = AP − σ −2 (B − P C ∗ D)S −1 B ∗ QW −1 ˆ ˜ ˜ −1 . This is equivalent to two controllers it suﬃces to prove that W A = A˜W proving (I + P Q − σ −1 P Q)AQ − σ −2 P C ∗ R−1 C = AP (I + P Q − σ −1 P Q) − σ −2 BS −1 B ∗ Q, which is equivalent to proving (I + P Q)AQ − σ −2 (I + P Q)AQ + σ −2 AQ − σ −2 P C ∗ R−1 C = AP (I + P Q) − σ −1 AP (I + P Q) + σ −2 AP − σ −2 BS −1 B ∗ Q. Using (11) we see that this is indeed true. The above gives a nice interpretation of the strange formulas for the B´ezout factors in Theorem 14: they are the √ ones that lead to a robustly stabilizing controller with robustness margin 1 − σ 2 . Note that with σ = 1 the controller C = YX−1 reduces to a symmetric LQG-controller.

Robustly Stabilizing Controllers with Internal Loop

93

7 Integrated Nodes and Reciprocals First we brieﬂy review the deﬁnition of an operator node (see Staﬀans [24, Section 4.7]) and introduce the concept of an integrated node (this is a specialization of the concept of a distributional resolvent linear system introduced in Opmeer [22]). U , Y , Z are separable Hilbert spaces. An operator node is speciﬁed by three operators A, B, C and a characteristic function G. These are assumed to satisfy: • A is a closed densely deﬁned operator on Z with nonempty resolvent set. • C ∈ L(D(A), Y ) is bounded where D(A) is equipped with the graph norm. • B ∗ ∈ L(D(A∗ ), U ) is bounded where D(A∗ ) is equipped with the graph norm. • G : ρ(A) → L(U, Y ) and for α, s ∈ ρ(A) satisﬁes G(s) − G(α) = (α − s)C(sI − A)−1 (αI − A)−1 B. The dual of an operator node is speciﬁed by the operators A∗ , C ∗ , B ∗ , G(¯ α)∗ . Deﬁnition 16 An integrated node is an operator node for which ρ(A) contains a right half-plane and there exist M > 0 and n ∈ Z such that (sI − A)−1 ≤ M |s|n on this right half-plane. We recall that the Laplace transform can be deﬁned for certain Banach space valued distributions and that the image of the set of Laplace transformable distributions is exactly the set of functions deﬁned on some right half-plane that are analytic and polynomially bounded. This allows us to deﬁne the state and output of an integrated node as Laplace transformable distributions (Z-valued and Y -valued, respectively). Deﬁnition 17 For an initial state z0 ∈ Z and a Laplace transformable distribution u the state and output of an integrated node are deﬁned through their Laplace transforms by zˆ(s) = (sI − A)−1 z0 + (sI − A)−1 B u ˆ(s), yˆ(s) = C(sI − A)−1 z0 + G(s)ˆ u(s), with s in some right half-plane that is contained in ρ(A). G is the transfer function and it equals the characteristic function G on any right half-plane contained in ρ(A), but they can diﬀer outside this region (see [6]). The ﬁnite cost condition for integrated nodes is deﬁned as for state linear systems: for every z0 ∈ Z there exists a u ∈ L2 (0, ∞; U ) such that the corresponding output y ∈ L2 (0, ∞; Y ). The concept of a reciprocal system was introduced in [4] for well-posed linear system and it easily extends to integrated nodes with 0 ∈ ρ(A).

94

R.F. Curtain

Deﬁnition 18 Suppose that the integrated node Σ with generating operators A, B, C and characteristic function G is such that 0 ∈ ρ(A). Its reciprocal system is the state linear system Σ(A−1 , A−1 B, −CA−1 , G(0)). The linear quadratic regulator problem for a system node can be solved via its reciprocal system. The following result was shown in [21] for well-posed linear systems and for the very general class of distributional resolvent systems in Opmeer [22]. Theorem 19 An integrated node Σ with 0 ∈ ρ(A) satisﬁes the ﬁnite cost condition if and only if its reciprocal system does. Moreover, if the ﬁnite cost condition is satisﬁed, then there exists unique optimal controls for Σ and for its reciprocal system and the optimal costs are equal. So we obtain the optimal cost operator for an integrated node as the smallest bounded nonnegative solution to the following reciprocal control Riccati equation A−∗ Q + QA−1 + A−∗ C ∗ CA−1 =

(15)

(QA−1 B − A−∗ C ∗ Dr )Sr−1 (B ∗ A−∗ Q − Dr∗ CA−1 ), where Dr = G(0) and Sr = I + Dr∗ Dr . The closed-loop main generating operator is given by ArQ = A−1 + Sr−1 (Dr∗ CA−1 − B ∗ A−∗ Q). An analogous remark applies to the dual system with the reciprocal ﬁlter Riccati equation A−1 P + P A−∗ + A−1 BB ∗ A−∗ = −∗

(P A

∗

−1

C −A

BDr∗ )Rr−1 (CA−∗ P

(16) ∗

−∗

− Dr B A

),

where Rr = I + Dr Dr∗ . The closed-loop main generating operator is given by ArP = A−1 + (−P A−∗ C ∗ + A−1 BDr∗ )Rr−1 CA−1 . This connection provides a normalized doubly coprime factorization of the transfer function of the integrated node. The following result is proven in Curtain and Opmeer [7, Theorem 8.9]. Lemma 6 Suppose that Σ is an integrated node with generating operators A, B, C and characteristic function G such that 0 ∈ ρ(A) and Σ and its dual satisfy the ﬁnite cost condition. Denote the transfer functions of the left- and ˜ r, M ˜ r] right-factor systems of its reciprocal system given in Theorem 14 by [N r r and [N ; M ], respectively. Then the transfer function of the integrated node has a left (right)-coprime factorization given on some right half-plane by ˜ r ( 1 )−1 N ˜ r ( 1 ) = Nr ( 1 )Mr ( 1 )−1 . G(s) = M s s s s

(17)

So an idea would be to ﬁnd the conventional robustly stabilizing controllers Cr for the reciprocal transfer function Gr to show that C(s) = Cr ( 1s ) stabilizes G. However, in general, C need not be well-deﬁned and so we design controllers with internal loop.

Robustly Stabilizing Controllers with Internal Loop

95

Theorem 20 Suppose that the integrated node Σ with generating operators A, B, C and characteristic function G is such that 0 ∈ ρ(A). If Σ and its dual satisfy the ﬁnite-cost condition, then the robust stabilization problem is solvable for all ε < εmax = (1 + r(P Q))−1/2 , where P, Q are the smallest bounded nonnegative solutions to the Riccati equations (16) and (15), respectively. Robustly stabilizing dual canonical controllers with internal loop and a robustness margin of ε (0 < ε < εmax ) are given by (5), where 1 1 1 1 K12 (s) = φr11 ( )J(s) + φr12 ( ), K22 (s) = I − φr22 ( ) − φr21 ( )J(s). s s s s φr is deﬁned in Theorem 5 applied to the reciprocal system and J ∈ H∞ (U ) satisﬁes J ∞ < 1. Proof. From Lemma 6 we know that the doubly coprime factorizations of G and its reciprocal transfer function Gr are in a one to one correspondence through (17) on some right half-plane. So we solve the robust stabilization problem for the reciprocal system with transfer function Gr using the results from Section 5. Denote the controllers deﬁned by the formulas (14) applied to the reciprocal system by Cr . From Lemma 1 this is equivalent to the dual canonical controller with open loop with Kr12 (s) = φr11 ( 1s )J( 1s ) + φr12 ( 1s ), Kr22 (s) = I − φr22 ( 1s ) − φr21 ( 1s )J( 1s ). Note that F(s) ∈ HH∞ if and only if F( 1s ) ∈ HH∞ . So from Theorem 4 and Theorem 6 it follows that K(s) = Kr ( 1s ) stabilizes G with internal loop if and only if Kr stabilizes Gr with internal loop. Moreover, since the reciprocal system has the same P and Q operators, the robustness margins are identical. We also give conditions under which stabilizing controllers with internal loop can be realized as conventional controlllers. Corollary 1 Suppose that the integrated node Σ satisﬁes the assumptions in Theorem 20 and in addition either both U and Y are ﬁnite-dimensional or the component of ρ(A) containing a right half-plane also contains the origin. Then conventional robustly stabilizing controllers with a robustness margin of ε (0 < ε < εmax ) are given by C(s) =

1 1 φr11 ( )J(s) + φr12 ( ) s s

1 1 φr22 ( ) + φr21 ( )J(s) s s

−1

,

φr is deﬁned in Theorem 5 applied to the reciprocal system and J ∈ H∞ (U ) satisﬁes J ∞ < 1. Proof. We seek a controller C ∈ HH∞ /H∞ (S) for some set S in the right half-plane. From Lemma 1 we also need to show that C(s) = Cr ( 1s ) has a right-coprime factorization over S. As in the proof of Theorem 20 we ﬁnd the robustly stabilizing controllers Cr for the reciprocal system Gr and consider

96

R.F. Curtain

the candidate C(s) = Cr ( 1s ). The coprimeness property follows from that of the reciprocal controller Cr (see Theorem 15). Now C is well-deﬁned for 1/s in some right half-plane, or equivalently for s in the disc D(µ, µ) = {|z − µ| < µ} for some positive µ. Under the assumption that the component of ρ(A) containing a right half-plane also contains the origin, the coprime factorization (17) of G from Lemma 6 extends to this component and this has a nonzero intersection SA with D(µ, µ). So both G and C are in HH∞ /H∞ (SA ) and C is a conventional stabilizing controller. In the case that U and Y are ﬁnite-dimensional we note that square matrix+ valued holomorphic functions on C+ 0 are invertible almost everywhere on C0 and so we choose S to be the intersection of this region with the right halfplane where G has its normalized doubly coprime factorization. We remark that our proof for the case of U, Y ﬁnite-dimensional coincides with the class considered in [12], [13]. There are, however ﬂaws in this approach. The controller is only deﬁned almost everywhere on some right halfplane and so it does not have a good dynamical interpretation. The controller with internal loop is a more satisfactory approach. The condition that the component of ρ(A) containing a right half-plane should also contain the origin is satisﬁed by generators of contraction semigroups. In particular, dissipative systems with colocated actuators and sensors have contraction semigroups; the bounded input and output case has been studied in [19]) and the well-posed case in Curtain and Weiss [8]. In these papers the question of when the systems can be strongly or exponentially stabilized by the controller C = −kI for k > 0 is posed. As can be seen in the examples in [8], it is not trivial to verify the well-posedness assumptions. The advantage of our approach is that this is not necessary. However, our approach only looks at the weaker property of input-output stabilization, and we allow a more general class of controllers than static output feedback. Nonetheless, there are suﬃcient commonalities to make it an interesting comparison. Example 1 Consider the integrated node Σ with generating operators A, B, B ∗ , where A is skew adjoint and dissipative, i.e., D(A∗ ) = D(A) and A = −A∗ , Y = U and C = B ∗ and A generates a contraction semigroup. If 0 ∈ ρ(A), we obtain the reciprocal system Σ(A−1 , A−1 B, B ∗ A−∗ , G(0)). This reciprocal system falls into the class studied in [20, Chapter 2.2] and it is known that it and its dual satisfy the ﬁnite cost condition. So we obtain a whole class of systems that satisfy Theorem 20 and can be robustly stabilized. In principle we have explicit formulas for the robustly stabilizing controllers, but in general they do depend on the solutions of the reciprocal Riccati equations and the formulas are complicated. So this result is essentially an existence result. If, however, we consider the special case that G(0) + G(0)∗ = 0, then P = I = Q are solutions and they will be unique provided that the closed-loop

Robustly Stabilizing Controllers with Internal Loop

97

operators are strongly stable. Suﬃcient conditions for the latter to hold are that Σ is well-posed and exactly controllable (or equivalently exactly observable) for then the closed-loop Riccati semigroups will be exponentially stable (see Mikkola [18]). A physical example that satisﬁes all the above assumptions is the Rayleigh beam model Example 6.3 in [8] if we choose one sensor and one actuator. Under the above assumptions, we obtain the central controller 1 ∗ −∗ ˜ −1 A−1 B, Kc (s) = G(0) − 1−2ε (1/sI − A) 2B A −1 B(I + G(0))−1 B ∗ A−∗ and ε < √12 , the maximum where A˜ = A−1 − 2−2ε 1−2ε2 A robustness margin. Finally, it is interesting to note that if we choose J = σε I, then we obtain the robustly stabilizing controller C = −I as in [8]. 2

References 1. R.F. Curtain, G. Weiss, and M. Weiss, ”Coprime factorization for regular linear systems”, Automatica, vol. 32, pp. 1519–1531, 1996. 2. R.F. Curtain, ”Robust Stabilizability of Normalized Coprime Factors; the Inﬁnite-Dimensional Case”, Int. J. Control, vol. 51, pp. 1173–1190, 1990. 3. R.F. Curtain and H.J. Zwart, An Introduction to Inﬁnite-Dimensional Linear Systems Theory. Springer-Verlag, New York, 1995. 4. R.F. Curtain, ”Regular linear systems and their reciprocals: applications to Riccati equations”, Systems Control Lett., vol. 49, pp. 81–89, 2003. 5. R.F. Curtain, ”Riccati equations for stable well-posed linear systems; the generic case”, SIAM J. Contr. and Optim., vol. 42, pp. 1681–1702, 2003. 6. R.F. Curtain and M.R. Opmeer, ”The suboptimal Nehari problem for stable inﬁnite-dimensional linear systems”, SIAM J. Contr. and Optim., to appear. 7. R.F. Curtain and M.R. Opmeer, ”Normalized doubly coprime factorizations for well-posed linear systems”, MCSS, to appear. 8. R.F. Curtain and G. Weiss. Exponential stabilization of well-posed systems by colocated feedback. SIAM J. Contr. and Optim., to appear. 9. R.F. Curtain, G. Weiss and M. Weiss, ”Stabilization of irrational transfer functions with internal loop”, p.179–208 in Systems, Approximation, Singular integral operators and related topics, Int. workshop on operator theory and applications, IWOTA 2000. Eds., A.A. Borichev and N.K. Nikolski, Vol. 129, in series Advances in operator Theory and Applications, Birkh¨ auser. 10. B.A. Francis, A Course in H ∞ Control Theory. Springer-Verlag, LNCIS, vol. 88, Berlin, 1987. 11. T.T. Georgiou and M.C. Smith, ”Graphs, causality and stabilizability: linear, shift-invariant systems on L2 [0, ∞)”, Math. for Control, Signals and Systems, vol. 6, pp. 195–223, 1993. 12. T.T. Georgiou and M.C. Smith, ”Optimal robustness in the gap metric”, IEEE Trans. Autom. Control, vol. 35, pp. 673–686, 1990. 13. T.T. Georgiou and M.C. Smith, ”Robust stabilization: Controller design for distributed plants”, IEEE Trans. Autom. Control, vol. 37, pp. 1133–1143, 1992. 14. K. Glover and D. McFarlane, ”Robust stabilization of normalized coprime factor plant descriptions with H∞ -bounded uncertainty”, IEEE Trans. Autom. Control, vol. 34, pp. 821–830, 1989.

98

R.F. Curtain

15. P. Grabowski, ”On the spectral-Lyapunov approach to parametric optimization of distributed parameter systems”, IMA Journal of Mathematical Control and Information, vol. 7, pp. 317–338, 1991. 16. S. Hansen and G. Weiss, ”New results on the operator Carleson measure criterion”, IMA Journal of Mathematical Control and Information, vol. 14, pp. 3–32, 1997. 17. D.G. Meyer and G.F. Franklin, ”A connection between normalized coprime factorizations and linear quadratic regulator theory”, IEEE Trans. Autom. Control, vol. 28, pp. 227–228, 1987. 18. K. Mikkola, Inﬁnite-Dimensional Linear Systems, Optimal Control and Riccati Equations, PhD thesis, October 2002, Helsinki University of Technology. Available at http://www.math.hut.ﬁ/reports 19. J.C. Oostveen and R.F. Curtain, ”Robustly stabilizing controllers for disipative inﬁnite-dimensional systems with collocated actuators and sensors”, Automatica, vol. 36, pp. 337–348, 2000. 20. J.C. Oostveen, Strongly Stabilizable Distributed Parameter Systems, SIAM, Philadelphia, 2000. 21. Mark R. Opmeer and Ruth F. Curtain, ”New Riccati equations for well-posed linear systems”, Systems Control Lett., vol. 52, pp. 339–347, 2004. 22. Mark R. Opmeer, ”Inﬁnite-dimensional linear systems: a distributional approach”, Proc. London Math. Soc. (to appear). 23. L.B. Page, ”Bounded and Compact vectorial Hankel operators”, Trans. Amer. Math. Soc., vol. 150, pp. 529–539, 1970. 24. O.J. Staﬀans, Well-Posed Linear Systems, volume 103, Encyclopedia of Mathematics and its Applications, Cambridge University Press, Cambridge, UK, 2005. 25. M. Vidyasagar, Control Systems Synthesis: A Factorization Approach, M.I.T. Press, 1985. 26. G. Weiss and R.F. Curtain, ”Dynamic stabilization of regular linear systems”, IEEE Trans. Aut. Control, vol. 42, pp. 1–18, 1997.

Model Reduction of Strongly Stable Nonstationary LPV Systems Mazen Farhood and Geir E. Dullerud Department of Mechanical Engineering, University of Illinois, Urbana, IL 61801, USA; [email protected], [email protected]

1 Introduction This paper deals with the model reduction of nonstationary linear parametervarying (NLPV) systems. Our interest in LPV models is motivated by the desire to control nonlinear systems along prespeciﬁed trajectories. LPV models arise naturally in such scenarios as a method to capture the possible nonlinear dynamics, while maintaining a model that is amenable to control synthesis. Frequently, when pursuing such an LPV formulation, one ends up with models of relatively large dimension. Accordingly, ﬁnding control syntheses for such models, which usually involves solving a number of linear operator inequalities as discussed in [5], requires substantial computation. For this reason, developing a theory that provides systematic methods of approximating such models is beneﬁcial. In the paper, we utilize the theory of generalized gramians to deﬁne the notion of balanced realizations for NLPV systems. We also examine the balanced truncation method in detail and derive error bounds for such a reduction process. The contributions of the paper are as follows: • •

•

generalization of the balanced truncation model reduction procedure to the class of NLPV systems; several results on the worst-case balanced truncation error. These results when restricted to the purely time-varying case (i.e., no parameters) provide the least conservative error bounds currently available in the literature; operator theoretic machinery is developed in the context of standard robust control tools for working with NLPV models.

Our paper deploys a combination of recent work on NLPV models in [5] and new work on model reduction using balanced truncation for standard LTV systems in [13, 18]. The basic approach is motivated by the work in [1] on the generalization of balanced truncation to stationary multidimensional systems, and that in [12] on discrete time model reduction of standard LTI B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 99–118, 2006. © Springer-Verlag Berlin Heidelberg 2006

100

M. Farhood and G.E. Dullerud

systems. The basic approach behind balanced truncation originates in [16], and the by-now famous error bounds associated with this method in the LTI case were ﬁrst demonstrated in [3, 11]. The NLPV models used here are the natural generalization of LPV models, ﬁrst introduced in [15, 17], to the case of nonstationary systems. The paper is organized as follows: we begin with a section which establishes notation and collects some needed deﬁnitions; in Section 3 we introduce NLPV models and discuss two forms of stability; Section 4 provides the balanced truncation procedure and proves the main results of the paper; we conclude with a summary statement.

2 Preliminaries The set of real n × m matrices and that of real symmetric n × n matrices are denoted by Rn×m and Sn respectively. The maximum singular value of a matrix X is denoted by σ ¯ (X). Given two Hilbert spaces E and F , we denote the space of bounded linear operators mapping E to F by L(E, F ), and shorten this to L(E) when E equals F . If X is in L(E, F ), we denote the E to F induced norm of X by X E→F ; when the spaces involved are obvious, we write simply X . The adjoint of X is written X ∗ . When an operator X ∈ L(E) is self-adjoint, we use X 0 to mean it is negative deﬁnite; that is there exists a number α > 0 such that, for all nonzero x ∈ E, the inequality x, Xx < −α x 2 holds, where ·, · denotes the inner product and · denotes the corresponding norm on E. We use E⊕F to denote the Hilbert space direct sum of E and F . If Si is a sequence of operators, then diag(Si ) denotes their block-diagonal augmentation. The main Hilbert space of interest in this paper is formed from an inﬁnite sequence of Euclidean spaces (Rn(0) , Rn(1) , Rn(2) , . . .), and is denoted by 2 (Rn(0) , Rn(1) , . . .), or just 2 (Rn ) for short. It is deﬁned as the subn(k) consisting of elements x = space of the Hilbert space direct sum ⊕∞ k=0 R ∞ (x(0), x(1), x(2), . . .), with x(k) ∈ Rn(k) , so that x 2 = k=0 x(k)∗ x(k) < ∞. The inner product of x, y in 2 (Rn ) is hence deﬁned as the sum x, y 2 = ∞ ∗ k=0 x(k) y(k). If the spatial dimensions n(k) are either evident or irrelevant to the discussion, then the notation 2 (Rn ) is abbreviated to 2 . Also, we will n(k) use (Rn ) to denote ⊕∞ . k=0 R A key operator used in the paper is the unilateral shift Z, deﬁned as follows: Z:

2

(Rn(1) , Rn(2) , . . .) → Z

2 (R

n(0)

, Rn(1) , Rn(2) , . . .)

(a(1), a(2), . . .) −→ (0, a(1), a(2), . . .). Clearly this deﬁnition is extendable to , and in the sequel, we will not distinguish between these mappings. Given a time-varying dimension n(k), we deﬁne the notation I n2 := diag(In(0) , In(1) , In(2) , . . .), where In(k) is an n(k)× n(k) identity matrix.

Model Reduction of Strongly Stable Nonstationary LPV Systems

101

Following the notation and approach in [2], we make the following deﬁnitions. First, we say a bounded linear operator Q mapping 2 (Rm(0) , Rm(1) , . . .) to 2 (Rn(0) , Rn(1) , . . .) is block-diagonal if there exists a sequence of matrices Q(k) in Rn(k)×m(k) such that, for all w, z, if z = Qw, then z(k) = Q(k)w(k). Then Q has the representation diag(Q(0), Q(1), Q(2), . . .). A diagonal operator is a block-diagonal operator where each of the matrix blocks is diagonal. Suppose F , G, R and S are block-diagonal operators, and let A be a partitioned operator of the form A=

F G . RS

Then we deﬁne the following notation: A := diag

F (1) G(1) F (0) G(0) ,... , R(1) S(1) R(0) S(0)

,

which we call the diagonal realization of A. Clearly for any given operator A of this particular structure, A is simply A with the rows and columns permuted appropriately so that A

k

=

F (k) G(k) . R(k) S(k)

From this deﬁnition, it is easy to see that A + B = A + B and AC = A C hold for appropriately dimensioned operators, and similarly that A βI holds if and only if A βI, where β is a scalar. Namely, the • operation is a homomorphism from partitioned operators with blockdiagonal entries to block-diagonal operators.

3 NLPV Systems We now brieﬂy review NLPV models. The reader is referred to [5] for an indepth treatment of the theory. To start, the NLPV models of this paper are of the form x(k + 1) = A(δ(k), k)x(k) + B(δ(k), k)w(k) z(k) = C(δ(k), k)x(k) + D(δ(k), k)w(k), where A(·, ·), B(·, ·), C(·, ·), and D(·, ·) are matrix-valued functions that are known a priori. The variable k is time, and δ(k) := (δ1 (k), . . . , δd (k) ) is a vector of real scalar parameters. In this paper, we are concerned only with the subclass of NLPV models satisfying the condition that the dependence of the matrix functions A, B, C, and D on the parameters δi is rational and given in terms of a feedback coupling. Such models are commonly referred to as LFT systems and are basically the straightforward generalization of the

102

M. Farhood and G.E. Dullerud

✲

α(k) z(k) ✛

Δ(k)

G

β(k) ✛ ✛ w(k)

Fig. 1. The interconnection of G with Δ(k)

LPV systems ﬁrst introduced in [15, 17]. We now introduce a model of the said subclass. Let G be a linear time-varying discrete-time system deﬁned by the following state space equation:      x(k + 1) Ass (k) Asp (k) Bs (k) x(k)  α(k)  = Aps (k) App (k) Bp (k)  β(k)  , x(0) = 0, (1) z(k) Cs (k) Cp (k) D(k) w(k) for w ∈ 2 . The vector-valued signals x(k), α(k), β(k), z(k), and w(k) are real and have time-varying dimensions, with the constraint that dim(β(k)) = dim(α(k)). We denote the dimensions of these signals by n0 (k), n(k), n(k), nz (k), and nw (k) respectively. We assume that all the state space matrices are uniformly bounded functions of time. For any scalar sequences δ1 (k), . . . , δd (k) d and associated dimensions n1 (k), . . . , nd (k) satisfying i=1 ni (k) = n(k), we deﬁne the diagonal matrix Δ(k) as Δ(k) := diag(δ1 (k)In1 (k) , . . . , δd (k)Ind (k) ) ∈ Rn(k)×n(k) . Also, we also constrain σ ¯ (Δ(k)) ≤ 1 for all k ≥ 0. We will be concerned with the feedback arrangement in Figure 1, where G and Δ(k) are connected in feedback. This system can be expressed formally by x(k + 1) x(k) = H(k) , z(k) w(k)

(2)

where H(k) is given by H(k) =

A (k) Ass (k) Bs (k) A (k) . (3) + sp Δ(k) (I − App (k)Δ(k))−1 ps Bp (k) Cs (k) D(k) Cp (k)

We will refer to the mapping w → z in (2) as the system Gδ . Hence, Gδ is a linear time-varying system with arbitrary rational state-space parameter dependence formulated in an LFT framework, where the time-varying parameters δi act on the system G through the linear fractional feedback channels

Model Reduction of Strongly Stable Nonstationary LPV Systems

103

(α, β). We assume App (k) such that Sk = I − App (k)Δ(k) is invertible for all k ≥ 0 so that the LFT in (3) is well-deﬁned at each time k. This well-posedness condition guarantees that there are unique solutions in to (1). Using the previously deﬁned notation, clearly the matrix sequences Ass (k), Bs (k), Cs (k), and D(k) from (1) deﬁne block-diagonal operators. The blocks of the matrix Δ(k) naturally partition α(k) and β(k) into d separate vectorvalued channels, conformably with which we partition the following state space matrices such that Asp (k) = A1sp (k) A2sp (k) · · · Adsp (k)  11  App (k) · · · A1d pp (k)  ..  .. App (k) =  ... . . 

dd Ad1 pp (k) · · · App (k)  1    1 Bp (k) Aps (k) Bp2 (k) A2ps (k)     Bp (k) =  .  Aps (k) =  .   ..   .. 

(4)

Bpd (k)

Adps (k)

Cp (k) = Cp1 (k) Cp2 (k) · · · Cpd (k) , ni (k)×nj (k) where Aisp (k) ∈ Rn0 (k+1)×ni (k) , Aij , Aips (k) ∈ Rni (k)×n0 (k) , pp (k) ∈ R Bpi (k) ∈ Rni (k)×nw (k) , and Cpi (k) ∈ Rnz (k)×ni (k) . The matrix sequence of each of the elements of the state space matrices in (4) deﬁnes a block-diagonal operator; and so we construct from the sequence of each of these state space matrices a partitioned operator, each of whose elements is block-diagonal and deﬁned in the obvious way. For instance, the matrix sequences A1sp (k), . . . , Adsp (k) deﬁne block-diagonal operators that compose the partitioned operator Asp . With Z being the shift, we rewrite our system equations as      x x ZAss ZAsp ZBs α =  Aps App Bp   β  , (5) Cs Cp D z w

x x x = diag(I n20 , Δ1 , . . . , Δd ) = Δ , β α α where x ∈ (Rn0 ), w ∈ 2 (Rnw ), z ∈ (α1 , . . . , αd ), βi , αi ∈ (Rni ), and

(6)

(Rnz ), β = (β1 , . . . , βd ), α =

Δi = diag(δi (0)Ini (0) , δi (1)Ini (1) , δi (2)Ini (2) , . . .). Before concluding this section, we make some convenient deﬁnitions that will be used extensively in the sequel. To start, we deﬁne Z˜ := diag(Z, I 2 ) and Bs Ass Asp (7) , C := Cs Cp . , B := A := Bp Aps App

104

M. Farhood and G.E. Dullerud

Also, we deﬁne

(n0 ,...,nd ) 2

Δ := {Δ ∈ L(

:=

2 (R

(n0 ,...,nd ) ) 2

n0

)⊕

2 (R

n1

) ⊕ ···⊕

2 (R

nd

) and

: Δ is partitioned as in (6) and Δ ≤ 1}.

Note that the operator Z˜ = diag(Z, I 2 ) has a conformable partitioning to that of Δ = diag(Δs , Δp ), where Δs = I n20 and Δp = diag(Δ1 , Δ2 , . . . , Δd ). 3.1 Stability of NLPV Models This section tackles the various concepts of stability that are essential to our work. To begin we deﬁne a basic notion of stability. Deﬁnition 21 An NLPV model is for all Δ ∈ Δ.

2 -stable

˜ has a bounded inverse if I −ΔZA

(n ,...,n )

d Thus, there exists a unique (x, β) ∈ 2 0 satisfying (5) and (6) if the model is 2 -stable. In such a scenario, equations (5) and (6) can be rewritten in the form z = Gδ w, where

˜ −1 ΔZB ˜ + D ∈ L( 2 (Rnw ), Gδ = C(I − ΔZA)

2 (R

nz

)).

(8)

(n ,...,n )

d At this point, we deﬁne T as the set of operators T ∈ L( 2 0 ) that have bounded inverses and are of the form T = diag(T0 , T1 , . . . , Td ), where each Ti ∈ L( 2 (Rni )) is block-diagonal so that Ti k = Ti (k) ∈ Rni (k)×ni (k) . Observe that T is a commutant of Δ. Moreover, we deﬁne the subset X of T by X = {X 0 : X ∈ T }.

Deﬁnition 22 An NLPV model is strongly 2 -stable if there exists P ∈ X satisfying AP A∗ − Z˜ ∗ P Z˜ 0. (9) The following lemma asserts that strongly a subset of the 2 -stable ones.

2 -stable

NLPV models constitute

Lemma 7 A strongly 2 -stable NLPV model is also converse is not true in general.

2 -stable;

however, the

The proof parallels the standard case and is hence omitted. ˜ −1 is bounded for Remark 2 We know that, under 2 -stability, (I − ΔZA) all Δ ∈ Δ. But, this boundedness is not necessarily uniform. On the other hand, strong 2 -stability guarantees that the aforementioned norm is uniformly bounded; this is clearly shown by the following norm condition which is easily derived: ˜ −1 ≤ (I − ΔZA)

1

1

P−2 · P 2 1 1 ˜ 2 1 − P − 2 ZAP

< ∞ for all Δ ∈ Δ,

where P is any solution in X to inequality (9).

Model Reduction of Strongly Stable Nonstationary LPV Systems

105

One of the key features of strongly 2 -stable NLPV models is that they can always be represented by an equivalent balanced realization, as we will show next. But ﬁrst, we need to deﬁne the balanced realizations of an NLPV model. Deﬁnition 23 An NLPV system realization is balanced if there exists a diagonal operator Σ ∈ X satisfying AΣA∗ − Z˜ ∗ Σ Z˜ + BB ∗

0,

(10)

˜ − Σ + C ∗C A∗ Z˜ ∗ Σ ZA

0.

(11)

Lemma 8 An NLPV model can be equivalently represented by a balanced realization if and only if it is strongly 2 -stable. Proof . Consider a strongly 2 -stable NLPV model (A, B, C, D; Δ). This is equivalent to the existence of P ∈ X satisfying (9), which in turn is equivalent to the existence of operators X, Y ∈ X solving the generalized Lyapunov inequalities AXA∗ − Z˜ ∗ X Z˜ + BB ∗

0,

˜ − Y + C ∗C A∗ Z˜ ∗ Y ZA

0.

Clearly, these conditions are themselves equivalent. Now we deﬁne the operator T ∈ T by 1 1 T := Σ 2 U ∗ X − 2 , where unitary operator U ∈ T and positive deﬁnite diagonal operator Σ are 1 1 obtained by performing a singular value decomposition on X 2 Y X 2 , namely 1 1 2 ∗ U Σ U = X 2 Y X 2 . Then, the following holds: T XT ∗ = (T ∗ )−1 Y T −1 = Σ. −1 ˜ ˜ The equivalent realization (Z˜ ∗ T Z)AT , (Z˜ ∗ T Z)B, CT −1 , D; Δ as a result is obviously balanced.

4 Model Reduction of Strongly

2 -Stable

NLPV Systems

This section focuses on the balanced truncation model reduction of strongly 2 -stable NLPV systems. It is divided into three subsections: the ﬁrst presents a precise formulation of the balanced truncation problem; the second gives upper bounds on the error induced in such a reduction process; and the last deals with eventually periodic NLPV systems and delivers guaranteed ﬁnite error bounds for the balanced truncation of such systems.

106

M. Farhood and G.E. Dullerud

4.1 Balanced Truncation Consider the balanced NLPV realization (A, B, C, D; Δ) with generalized diagonal gramian Σ ∈ X satisfying both of the generalized Lyapunov inequalities (10) and (11). Recall that Σ is of the form     Σi (0) Σ0     Σi (1) Σ1     , , where eachΣ = Σ=   i .. Σi (2)      . .. . Σ d

and Σi (k) is a diagonal positive deﬁnite matrix in Sni (k) . We assume without loss of generality that, in each block Σi (k), the diagonal entries are ordered with the largest ﬁrst. Now given the integers ri (k) such that 0 ≤ ri (k) ≤ ni (k) for all k ≥ 0, we partition each of the matrices Σi (k) into two sub-blocks Γi (k) ∈ Sri (k) and Ωi (k) ∈ Sni (k)−ri (k) so that Σi =

Γi 0 , 0 Ωi

(12)

where Γi and Ωi are block-diagonal operators. Note that, since ri (k) is allowed to be equal to zero or ni (k) at any time k, it is possible to have one of the matrices Ωi (k) or Γi (k) with zero dimension; this corresponds to the case where either zero states or all states are truncated at a particular k. Allowing for matrices with no entries, although a slight abuse of notation, will be very helpful in the manipulations of the sequel. We deﬁne the operators Γ and Ω to have a similar structure to that of Σ, namely Γ = diag(Γ0 , Γ1 , . . . , Γd ) and Ω = diag(Ω0 , Ω1 , . . . , Ωd ). The singular values corresponding to the states and parameters that will be truncated are in Ω. At this point, we want to partition A, B and C conformably with the partitioning of Σ. Recall from Section 3 that A, B and C have the following forms:     Ass A1sp · · · Adsp Bs 1d  Bp1  A1ps A11 · · · A pp pp     1 d A= . .  , B =  ..  , C = Cs Cp · · · Cp , .. . .  .   .. . ..  . dd Adps Ad1 pp · · · App

Bpd

where each of the elements of these partitioned system operators is blockdiagonal. Note further that     Ass (k) A1sp (k) · · · Adsp (k) Bs (k) 1d A1ps (k) A11  Bp1 (k) pp (k) · · · App (k)    A k= . B , =   ..  , . . .. k .. ..   ..  .  . (13) dd Bpd (k) Adps (k) Ad1 pp (k) · · · App (k) C

k

= Cs (k) Cp1 (k) · · · Cpd (k) .

Model Reduction of Strongly Stable Nonstationary LPV Systems

107

Let us now focus on the matrices Ass (k), Bs (k), Cs (k) and partition them in accordance with the partitionings of Σ0 (k) = diag(Γ0 (k), Ω0 (k)) and Σ0 (k + 1) = diag(Γ0 (k + 1), Ω0 (k + 1)) so that Ass (k) =

ˆs (k) Aˆss (k) Ass12 (k) B , Bs (k) = , Cs (k) = Cˆs (k) Cs2 (k) , Ass21 (k) Ass22 (k) Bs2 (k)

ˆs (k) ∈ Rr0 (k+1)×nw (k) , and Cˆs (k) ∈ Rnz (k)×r0 (k) . where Aˆss (k) ∈ Rr0 (k+1)×r0 (k) , B Hence, we have Ass =

ˆs Aˆss Ass12 B , Bs = , Cs = Cˆs Cs2 , Ass21 Ass22 Bs2

where each of the elements is block-diagonal. Similarly, the system matrices in (13) are partitioned compatibly of the associated Σi (k) so that  Aˆ1sp A1sp12 Aˆdsp Adsp12 Aˆss Ass12 ···  A 1 1 Adsp21 Adsp22 Asp21 Asp22  ss21 Ass22  ˆ1 1d 11 11 1 ˆ Aˆ1d App App12  Aps Aps12 pp App12 ···  1 11 11 1 1d 1d App21 App22 App21 App22 A A A=  ps21 . ps22 . .. .  .. .. ..  .  d1 ˆdd Add  Aˆdps Adps12 Aˆd1 A A pp pp12 pp pp12 ··· d1 dd Adps21 Adps22 Ad1 Add pp21 App22 pp21 App22 C=

Cˆs Cs2

Cˆp1 Cp12 · · ·

the other elements of with the partitioning 



         , B =           

Cˆpd Cpd2

ˆs B Bs2 ˆp1 B Bp12 .. . ˆpd B Bpd2

      ,    

.

Then a state space realization for the balanced truncation Gδ,r of the system Gδ is (Ar , Br , Cr , Dr ; Δr ) where

Ar Br Cr Dr

 ˆ Ass  Aˆ1  ps  . . =  .  ˆd  Aps Cˆs

Aˆ1sp Aˆ11 pp .. . Aˆd1 pp Cˆp1

Aˆdsp Aˆ1d pp .. . · · · Aˆdd pp ˆ · · · Cpd

··· ··· .. .

ˆs B ˆ Bp1 .. . ˆpd B

    ,   

D

and Δr = diag(I r20 , Δˆ1 , . . . , Δˆd ), with Δˆi = diag(δi (0)Iri (0) , δi (1)Iri (1) , . . .). Notice that Δr is constructed from the same parameters δi as those in Δ. Lemma 9 Suppose (A, B, C, D; Δ) is a balanced realization of Gδ . Then the corresponding balanced truncation Gδ,r is also strongly 2 -stable and balanced. Proof . To start, there exists a unique permutation Q such that Q∗ ΣQ = diag(Γ, Ω) formally; then we have

108

M. Farhood and G.E. Dullerud

˜ Ar A¯12 ˜2 A, ¯ Q∗ ZB ˜ = Z = Z ¯ ¯ Z˜ Z˜ A21 A22 ¯ CQ = Cr C¯2 = C,

˜ Q∗ ZAQ =

Z˜

Br ˜ ¯ ¯2 = Z2 B, B

¯2 ), where (Ar , Br , Cr , Dr ; Δr ) is a realization of and Δ¯ = Q∗ ΔQ = diag(Δr , Δ the truncation Gδ,r , and the rest of the operators are deﬁned in the obvious way. As the generalized gramian Σ satisﬁes both of inequalities (10) and (11), then focusing on (10), and with the aforesaid permutation in mind, the following ensues: Ar A¯12 A¯21 A¯22

Γ

Ω

This clearly gives

B A∗r A¯∗21 Z˜ ∗ Γ Z˜ + ¯r − ∗ ∗ B2 A¯12 A¯22 Z˜ ∗ Ω Z˜ Ar Γ A∗r − Z˜ ∗ Γ Z˜ + Br Br∗

¯∗ Br∗ B 2

0.

0.

Similarly, starting with (11), we can show that ˜ r − Γ + C ∗ Cr A∗r Z˜ ∗ Γ ZA r

0.

Thus directly from the deﬁnitions of strong stability and a balanced system we have the desired conclusion. 4.2 Error Bounds This subsection gives upper bounds on the error induced in the balanced truncation model reduction process. We start with the following result. Lemma 10 An NLPV model Gδ is strongly 2 -stable and satisﬁes the condition Gδ < 1, for all Δ ∈ Δ, if there exists a positive deﬁnite operator V in the commutant of Δ such that   −V 0 A∗ C∗  0 −I B∗ D∗    (14)  A B −Z˜ ∗ V −1 Z˜ 0  0. C D 0 −I This is a generalization of the suﬃciency part of the Kalman-YakubovichPopov (KYP) Lemma. Its proof is routine and so we do not include it here. Note that the above inequality is necessary and suﬃcient in the purely timevarying case as proved in [19]; however, in our case, it is in general only suﬃcient. We will ﬁnd the following notation convenient: ˜ −1 ΔZB ˜ + D, Gδ = Δ G = C(I − ΔZA) where

G=

˜ ZB ˜ ZA C D

and Δ ∈ Δ.

(15)

Model Reduction of Strongly Stable Nonstationary LPV Systems

109

Theorem 24 Suppose that (A, B, C, D; Δ) is a balanced realization for the NLPV system Gδ , and that the diagonal generalized gramian Σ ∈ X , satisfying both of inequalities (10) and (11), is partitioned as in (12). If Ωi = I 2 for all i = 0, 1, . . . , d, then, for all Δ ∈ Δ, the balanced truncation Gδ,r of Gδ satisﬁes the following inequality: Gδ − Gδ,r < 2. Proof . As Gδ and Gδ,r are both strongly 2 -stable, then so is 12 (Gδ − Gδ,r ). One realization of the system 12 (Gδ − Gδ,r ) is given in linear fractional form by  ˜ ˜ r ZAr 0 √12 ZB   1 Δr ¯,  0 Z˜2 A¯ √12 Z˜2 B (Gδ − Gδ,r ) = ¯   Δ 2 0 − √12 Cr √12 C¯ ¯ B, ¯ C, ¯ and Δ¯ are as deﬁned in the proof of Lemma 9, and Z˜m = where A, m diag({Ji }i=1 ) where Ji = Z˜ for all i. In the sequel, we will construct a positive ¯ and satisﬁes inequality deﬁnite operator V that commutes with diag(Δr , Δ) 1 (14) for this 2 (Gδ − Gδ,r ) realization. Then, invoking Lemma 10 completes the proof. Given that the diagonal operator Σ ∈ X satisﬁes inequalities (10) and (11), then direct applications of the Schur complement formula, along with some permutations, guarantee the validity of the following condition: −R1 K∗ ¯ K −Za∗ R2 Z¯a where Z¯a = diag(Z˜2 , I, Z˜2 ),  −1 Γ 0 0  0 Ω −1 0  0 I q2i Ri =   0  0 0 0 0 0 0

0 0 0 Γ 0

 0 0  0 , 0 Ω

0,



0  0  K =  0  Ar A¯21

0 0 0 A¯12 A¯22

0 0 0 Br ¯2 B

Ar A¯21 Cr 0 0

 A¯12 A¯22   C¯2  , 0  0

and q1 = nw , q2 = nz . Deﬁne the invertible operators L and S by     −I 0 0 I 0 I I 0 0 0  I  0 0 I 0 I  0 0 I 0   √ 1  1    . L = √  0 I √0 0 I  , S = √  0 0 0 2I 0   2 0 0 2  −I I 0 0 0   2I 0 0  0 −I 0 0 I 0 0 I 0 −I Multiplying the preceding condition on the left by diag(S ∗ , L) and on the right by diag(S, L∗ ) gives the following equivalent inequality:

110

M. Farhood and G.E. Dullerud

−S ∗ R1 S S ∗ K ∗ L∗ ¯ LKS −Zb∗ LR2 L∗ Z¯b

0,

(16)

˜ Performing the multiplications in this inequality where Z¯b = diag(Z˜3 , I, Z). leads to  1 −1  + Γ ) 12 (Γ −1 − Γ ) 0 0 0 2 (Γ  1 (Γ −1 − Γ ) 1 (Γ −1 + Γ )  0 0 0 2  2 , 1 1 −1 −1 S ∗ R1 S =  0 0 (Ω + Ω) 0 (Ω − Ω)   2 2   0 0 0 I 0 1 1 −1 −1 0 0 − Ω) 0 2 (Ω + Ω) 2 (Ω 1

2 (Γ  1 (Γ 2

LR2 L∗ =        LKS =    

−1

+ Γ)

− Γ −1 ) 0 0 0

Ar 0 0 − √12 Cr A¯21

1 −1 ) 2 (Γ − Γ 1 −1 + Γ) 2 (Γ

0 0 0

0

0

0 + Ω) 0 1 −1 ) 2 (Ω − Ω

0 0 I 0

1 −1 2 (Ω

0 Ar A¯21 √1 Cr 2

0 A¯12 A¯22 ¯2 √1 C

√1 Br 2 √1 Br 2 ¯ √1 B 2 2

0

A¯12 0 0 − √1 C¯2

0

0

¯ √1 B 2 2

A¯22

2

2

0



 0  −1  , − Ω )  0 1 −1 + Ω) 2 (Ω 1 2 (Ω

    =   

M N12 . N21 A¯22

Note that, in the preceding expressions, some of the operators might contain at certain time-instants matrices of zero dimensions. In such scenarios, the rows and columns of which the said matrices are elements would not be present, and the corresponding operator inequalities remain valid. Deﬁne the operator V as   1 −1 + Γ ) 21 (Γ −1 − Γ ) 0 2 (Γ . 0 V =  21 (Γ −1 − Γ ) 21 (Γ −1 + Γ ) 1 −1 0 0 (Ω + Ω) 2 Note that, since S ∗ R1 S 0, then V 0. Also, V clearly commutes with the ¯ Recall that, by assumption, Ω = I; hence, 1 (Ω −1 + operator diag(Δr , Δ). 2 Ω) = I and 21 (Ω −1 − Ω) = 0. With this in mind, it is not diﬃcult to see that inequality (16) implies that   V ∗ − M n   I 2w   0. ∗ −1 ˜ ˜   Z V Z3 M − 3 nz I2 Then, invoking Lemma 10, we get

1 2

(Gδ − Gδ,r ) < 1.

Model Reduction of Strongly Stable Nonstationary LPV Systems

111

Theorem 25 Given a balanced NLPV model Gδ , then, for all Δ ∈ Δ, its balanced truncation Gδ,r satisﬁes the error bound d

ωi,j ,

Gδ − Gδ,r < 2 i=0

j

where ωi,j are the distinct diagonal entries of the block-diagonal operator Ωi . The proof follows from scaling, Lemma 9 and repeated application of the previous theorem. Note that this error bound might involve an inﬁnite summation which in general may not converge to a ﬁnite number. In the following, we improve on this result and derive tighter bounds. We will ﬁrst consider balanced systems where the singular values corresponding to the states and parameters to be truncated are monotonic in time. Before doing this it will be convenient to establish the following terminology. Deﬁnition 26 Given a scalar sequence αk deﬁned on a subset W of the nonnegative integers, we deﬁne the following hold rule which extends the domain of αk to all k ≥ 0: let kmin = min{k ≥ 0 : k ∈ W} and then set αk =

αkmin , αq , where q := max{q ≤ k : q ∈ W},

if 0 ≤ k ≤ kmin ; if kmin < k.

We now have the following result. Theorem 27 (monotonic case) Suppose that (A, B, C, D; Δ) is a balanced realization for the NLPV system Gδ , and that the diagonal generalized gramian Σ ∈ X , satisfying both of inequalities (10) and (11), is partitioned as in (12). Let si (k) = ni (k) − ri (k) and deﬁne the set Fi , for i = 0, 1, . . . , d, by Fi = {k ≥ 0 : si (k) > 0}. Also suppose that for each i = 0, 1, . . . , d the scalar sequence ωi,k satisﬁes Ωi (k) = ωi,k Isi (k) for all k ∈ Fi . If for each i = 0, 1, . . . , d the sequence ωi,k is monotonic on Fk , then for all Δ ∈ Δ the balanced truncation Gδ,r of Gδ satisﬁes the following inequality: d

Gδ − Gδ,r < 2

sup ωi,k .

i=0 k∈Fi

Proof . It is suﬃcient to prove the theorem for the case where only one parameter or state block is being truncated (i.e., si = 0 for all i except for one, say j ∈ {0, 1, 2, . . . , d}), since the general case then follows simply by the standard use of the telescoping series and triangle inequality. Also, we assume without loss of generality that ωj,k ≤ 1 for all k ∈ Fj ; this can always be achieved by scaling inequalities (10) and (11).

112

M. Farhood and G.E. Dullerud

To begin, we extend the domain of deﬁnition of ωj,k to all k ≥ 0 using the hold rule deﬁned in Deﬁnition 26; note that the extended sequence is still monotonic. We now split the remainder of our proof into two separate cases, one where this sequence is nondecreasing and the other where it is nonincreasing. Case ωj,k nondecreasing: In this case, we have ωj,k ≤ ωj,k+1 for all k ≥ 0. We deﬁne the state space transformation T ∈ T as T

k

− 21

= (ωj,k )

I.

(17)

Note that, since Σ 0, then T is indeed bounded. This gives the following balanced realization for Gδ : ¯ B, ¯ C, ¯ D; Δ := A,

Z˜ ∗ T Z˜ AT −1 , Z˜ ∗ T Z˜ B, CT −1 , D; Δ .

(18)

¯ δ to refer to the system Gδ when the For convenient reference, we will use G realization in use is (18). Our goal now is to show that this new realization is balanced. To this end, given the state transformation T , we use (10) and (11) to arrive at −1 A¯∗ Z˜ ∗ (T ∗ ) ΣT −1

¯ A¯∗ − Z˜ ∗ Σ ¯ Z˜ + B ¯B ¯∗ A¯Σ −1 Z˜ A¯ − (T ∗ ) ΣT −1 + C¯ ∗ C¯

0, 0,

(19)

¯ = T ΣT ∗. Because of the special structure of T and the fact that where Σ ωj,k ≤ ωj,k+1 ≤ 1, it is not diﬃcult to see that C¯ ∗ C¯ = (T ∗ )−1 C ∗ CT −1

T ∗ C ∗ CT,

¯ 2. (T ∗ )2 A¯∗ Z˜ ∗ (T ∗ )−1 ΣT −1 Z˜ AT

¯ Z˜ A¯ = A¯∗ Z˜ ∗ (T ΣT ∗) Z˜ A¯ A¯∗ Z˜ ∗ Σ

Then, pre- and post-multiplying inequality (19) by (T ∗ )2 and T 2 respectively and then using the above inequalities give ¯ Z˜ A¯ − Σ ¯ + C¯ ∗ C¯ A¯∗ Z˜ ∗ Σ

0.

¯ is a diagonal gramian satisfying the generalized Lyapunov inequalHence, Σ ¯ δ . Notice that, by the deﬁnition of T , we ities for the system realization G have ¯j (k) = Tj (k)Σj (k)Tj∗ (k) = (ωj,k )−1 Γj (k) Σ =

(ωj,k )

−1

Γj (k)

I

=

Γ¯j (k)

Ωj (k)

¯j (k) . Ω

¯j = I 2 , and so, by invoking Theorem 24, we deduce that the balanced Thus, Ω ¯ δ,r of the system G ¯ δ satisﬁes the norm condition truncation G

Model Reduction of Strongly Stable Nonstationary LPV Systems

113

¯δ − G ¯ δ,r < 2 G for all Δ ∈ Δ. Now, it is not diﬃcult to see that, because of the special ¯δ − G ¯ δ,r are in structure of T , the error system realizations Gδ − Gδ,r and G fact equivalent, and as a result, we have ¯δ − G ¯ δ,r < 2. Gδ − Gδ,r = G Case ωj,k nonincreasing: A similar argument applies where here the state transformation T ∈ T is 1 deﬁned as T k = (ωj,k ) 2 I. We now consider the more general case, where singular values need not be monotonic in time. But ﬁrst, we require the following deﬁnition from [18]. Deﬁnition 28 Given a vector v = (v1 , v2 , . . . , vs ) for some integer s ≥ 1, suppose that v1 cannot be considered as a local maximum and vs cannot be considered as a local minimum. Then vector v has m local maxima vmax,i and m local minima vmin,i for some integer m ≥ 0, and the max-min ratio of v, denoted Sv , is deﬁned as m

Sv = v1

vmax,i , v i=1 min,i

Sv = v1 ,

m>0

m = 0.

Theorem 29 (nonmonotonic case) Given a balanced realization (A, B, C, D; Δ) for the NLPV system Gδ , suppose that a diagonal operator Σ ∈ X satisﬁes both of inequalities (10) and (11) and is partitioned as in (12), where, for all i = 0, 1, . . . , d and k ∈ Fi = {k ≥ 0 : si (k) > 0} we have Ωi (k) = ωi,k Isi (k) , with si (k) := ni (k) − ri (k). Deﬁne the vector ω ˆ i to consist of the elements ωi,k for k ∈ Fi . If for each i = 1, . . . , d we have dim(ˆ ωi ) < ∞, then for all Δ ∈ Δ the balanced truncation Gδ,r of Gδ satisﬁes the following inequality: d

Sωˆ i .

Gδ − Gδ,r < 2 i=0

The proof below uses the same idea as that of Theorem 27. Basically, we need to deﬁne some state space transformation that results in a balanced realization ¯ solving the generalized Lyafor the system Gδ where the diagonal gramian Σ ¯j = I 2 . Then, invoking punov inequalities for this realization is such that Ω Theorem 24 completes the proof. The choice of this state space transformation used is inspired by that of the monotonic case. Proof . As with the proof of Theorem 27 it is suﬃcient to prove the result for the case where the only Ωi that has non-zero dimension is Ωj for some ﬁxed j in {0, 1, . . . , d}; without loss of generality we assume that ωj,k ≤ 1 for all k.

114

M. Farhood and G.E. Dullerud

To keep the notation simple, we suppress the subscript j in ωj and ω ˆj . The vector ω ˆ is of the form ˆ min,1 , . . . , ω ˆ max,1 , . . . . . . , ω ˆ max,m , . . . , ωks ), ω ˆ = (ωk1 , ωk2 , . . . , ω corresponding to values of the sequence ωk evaluated at the ordered time points (k1 , k2 , . . . , kmin,1 , . . . , kmax,1 , . . . , kmax,m , . . . , ks ). The denoted local minima and maxima of the vector ω ˆ are as deﬁned in Deﬁnition 28. We now use the hold rule of Deﬁnition 26 to extend the sequence ωk to all k ≥ 0; the maxima and minima of ωk are illustrated below.

ω ˆ max,m

ω ˆ max,1

ω ˆ max,2

ωk1 ω ˆ min,2

ω ˆ min,1

ωks

We deﬁne the state space transformation T ∈ T as  1 for k = 0, 1, . . . , k1 − 1, ωk21 I, 1

ωk2 I,

T

k

=

− 21 ω ω I, min,1 k  1 −1 ωmin,1 ωmax,1 ωk2 I, 1



.. .

for k = k1 , k1 + 1, . . . , kmin,1 , for k = kmin,1 + 1, . . . , kmax,1 , for k = kmax,1 + 1, . . . , kmin,2 , .. .

ρωk2 I,

for k = kmax,m + 1, . . . , ks ,

ρω I,

for k = ks + 1, ks + 2, . . .,

1 2 ks

1

m

−1 . Also, deﬁne P, Q ∈ T such that P k = ωk2 I where ρ = i=1 ωmin,i ωmax,i −1 and Q = T P . It is not diﬃcult to see that the constituent scalars of operator T deﬁne a nonincreasing sequence, and so do those of operator Q and those ¯ B, ¯ C, ¯ D; Δ = of operator QP 2 . Then, given the equivalent realization A, ∗ −1 ∗ −1 Z˜ T Z˜ AT , Z˜ T Z˜ B, CT , D; Δ of the system Gδ , which we de-

¯ δ , and because of the special structure of T and note for ease of reference by G the assumption that ωj,k ≤ 1, the following ensues: ¯ A¯∗ − Z˜ ∗ Σ ¯ Z˜ + P −2 Q−1 B ¯B ¯ ∗ (Q∗ )−1 (P ∗ )−2 A¯Σ ¯ ¯ Z˜ A¯ − Σ ¯ + Q∗ C¯ ∗ CQ A¯∗ Z˜ ∗ Σ

0, 0,

Model Reduction of Strongly Stable Nonstationary LPV Systems

115

¯ = (P ∗ )−1 ΣP −1 . Notice that P −2 Q−1 where Σ ωk−1 I and Q ρI. Thus, 1 ¯ the diagonal operator Σ satisﬁes the generalized Lyapunov inequalities (10) ¯ ρC, ¯ D; Δ). As Ω ¯j = I 2 , then, invoking ¯ ω −1 B, and (11) for the realization (A, k1 Theorem 24, we get ¯δ − G ¯ δ,r < 2. ωk−1 ρ G 1 Finally, the special structure of operator T and the fact that Sω = ωk−1 ρ 1 lead to ¯δ − G ¯ δ,r < 2Sω . Gδ − Gδ,r = G

−1

We remark that Theorem 25 generalizes the LTV result in [14] to the NLPV framework. Also, Theorems 27 and 29 are mainly generalizations of their LTV counterparts in [18], with the important exception that the truncations in the theorems need not be restricted connected intervals. To illustrate how to apply these results, we consider the following hypothetical example. Suppose we are to truncate the states corresponding to the sequence Ω0 (k) = ω0,k Is0 (k) for k ∈ [1, 9], where 9

{ω0,k }k=1 = {1, 0.75, 2, 1.25, 3, 1.75, 4, 2.25, 5}. Then the corresponding error bound obtained from Theorem 25 is 2 × (1 + 0.75 + 2 + 1.25 + 3 + 1.75 + 4 + 2.25 + 5) = 42. This is exactly the same bound that the main result of [14] would give assuming a standard LTV system. If we are to apply Theorem 29 to truncate the states in one step, then we obtain the error bound 2×1×

2 3 4 5 × × × ≈ 65. 0.75 1.25 1.75 2.24

This bound is quite conservative and can be signiﬁcantly improved if we truncate the states in three steps and accordingly divide the sequence ω0,k into the following: {1, 0.75, 2, 1.25}, {3, 1.75, 4, 2.25}, and {5}. Then, applying Theorem 29 recursively, we obtain the improved error bound 2× 1×

4 2 +3× +5 0.75 1.75

≈ 29.

This can also be obtained from the results of [18] if the system in question is a standard LTV system. But, in our case, we can actually further improve on the last bound by dividing the sequence ω0,k into the two monotonic sequences {1, 2, 3, 4, 5} and {0.75, 1.25, 1.75, 2.25} and then applying Theorem 27 twice to get the error bound 2 × (5 + 2.25) = 14.5.

116

M. Farhood and G.E. Dullerud

4.3 Eventually Periodic LPV Systems This subsection focuses on the balanced truncation of eventually periodic LPV systems. These systems are aperiodic for an initial amount of time, and then become periodic afterwards. One scenario in which they originate is when parametrizing nonlinear systems about eventually periodic trajectories. Such trajectories can be arbitrary for a ﬁnite amount of time, but then settle down into a periodic orbit; a special case of this occurs when a system transitions between two operating points. In addition to that, eventually periodic systems naturally arise when considering problems involving plants with uncertain initial states. Note that both ﬁnite horizon and periodic systems are subclasses of eventually periodic systems. We refer the reader to [6–8] for some useful results on eventually periodic models. We now give a precise deﬁnition of an eventually periodic operator. Deﬁnition 30 A block-diagonal mapping P on odic if, for some integers h ≥ 0, q ≥ 1, we have

2

is (h, q)-eventually peri-

Z q ((Z ∗ )h P Z h ) = ((Z ∗ )h P Z h )Z q , that is P is q-periodic after an initial transient behavior up to time h. Moreover, a partitioned operator, whose elements are block-diagonal, is (h, q)eventually periodic if each of its block-diagonal elements is (h, q)-eventually periodic. Theorem 31 Suppose that state space operators A, B, and C are (h, q)eventually periodic. Then solutions X, Y ∈ X satisfying Lyapunov inequalities (10) and (11) exist if and only if (h, q)-eventually periodic solutions Xeper , Yeper ∈ X exist. The outline of the proof is as follows: ﬁrst, employ a similar averaging technique to that used in [2] to show that the periodic part of any of the generalized Lyapunov inequalities admits a q-periodic solution if feasible, then, having established that, the above result follows from scaling. Thus, if the system is strongly 2 -stable and (h, q)-eventually periodic, then we can construct an (h, q)-eventually periodic balanced realization with an (h, q)-eventually periodic diagonal gramian Σ ∈ X satisfying Lyapunov inequalities (10) and (11). Theorem 32 Suppose that system Gδ is an (h, q)-eventually periodic system with a balanced realization (A, B, C, D; Δ). Then the following hold: (i) There exists an (h, q)-eventually periodic diagonal operator Σ ∈ X , partitioned as in (12), satisfying both of the generalized Lyapunov inequalities (10) and (11); (ii) The balanced truncation Gδ,r of Gδ is balanced and satisﬁes the ﬁnite error bound

Model Reduction of Strongly Stable Nonstationary LPV Systems

117

d

ωi,j < ∞,

G − Gr < Ef h + 2 i=0

j

where the scalar parameters ωi,j are the distinct diagonal entries of the matrix diag(Ωi (h), . . . , Ωi (h + q − 1)), and Ef h is the ﬁnite upper bound on the error induced in the balanced truncation of the ﬁnite horizon part of Gδ and is derived by applying Theorem 29.

5 Conclusions In this paper we have introduced balanced truncation model reduction for NLPV systems, and derived explicit error bounds for this procedure. Even when restricted to purely time-varying systems the results obtained provide the least conservative bounds currently available in the literature. Although there has been considerable recent achievement in the literature on model reducing nonstationary systems, which are all directly motivated by the original LTI results in [3, 11], the authors conjecture that signiﬁcantly better bounds may be obtainable.

References 1. C.L. Beck, J.C. Doyle and K. Glover, “Model reduction of multidimensional and uncertain systems,” IEEE Transactions on Automatic Control, 41:1466–1477, 1996. 2. G.E. Dullerud and S.G. Lall, “A new approach to analysis and synthesis of time-varying systems,” IEEE Transactions on Automatic Control, 44:1486– 1497, 1999. 3. D. Enns, “Model reduction with balanced realizations: An error bound and a frequency weighted generalization”, Proc. IEEE Conference on Decision and Control, 1984. 4. M. Farhood, C.L. Beck and G.E. Dullerud, “Model reduction of nonstationary LPV systems,” Proc. IEEE Conference on Decision and Control, 2002. 5. M. Farhood and G.E. Dullerud, “Control of nonstationary LPV systems,” submitted to Automatica. 6. M. Farhood and G.E. Dullerud, “LMI tools for eventually periodic systems,” Sys. and Cont. Letters, 47:417–432, 2002. 7. M. Farhood and G.E. Dullerud, “On the 2 -induced control for eventually periodic systems,” Proceedings of the American Control Conference, 2004. 8. M. Farhood and G.E. Dullerud, “Duality and eventually periodic systems,” to appear in International Journal of Robust and Nonlinear Control. 9. M. Farhood, C.L. Beck, and G.E. Dullerud, “On the model reduction of nonstationary LPV systems,” Proceedings of the American Control Conference, 2003. 10. P. Gahinet and P. Apkarian, “A linear matrix inequality approach to H∞ control,” International Journal of Robust and Nonlinear Control, 4:421–448, 1991.

118

M. Farhood and G.E. Dullerud

11. K. Glover, “All optimal hankel-norm approximations of linear multivariable systems and their L∞ -error bounds,” International Journal of Control, 39:1115– 1193, 1984. 12. D. Hinrichsen and A.J. Pritchard, “An improved error estimate for reducedorder models of discrete-time systems,” IEEE Transactions on Automatic Control, 35:317–320, 1990. 13. S. Lall and C. Beck, “Error bounds for balanced model reduction of linear timevarying systems,” submitted to IEEE Transactions on Automatic Control. 14. S. Lall and C. Beck, “Model reduction of complex systems in the linear-fractional framework,” Proc. IEEE Conference on Decision and Control, 1999. 15. W.M. Lu, K. Zhou, and J.C. Doyle, “Stabilization of uncertain linear systems: an LFT approach,” IEEE Transactions on Automatic Control, 41:50–65, 1996. 16. B.C. Moore, “Principal component analysis in linear systems: controllability, observability, and model reduction, ” IEEE Transactions on Automatic Control, 26:17–34, 1981. 17. A. Packard, “Gain scheduling via linear fractional transformations,” Systems and Control Letters, 22:79–92, 1994. 18. H. Sandberg and A. Rantzer, “Balanced truncation of linear time-varying systems,” IEEE Transactions on Automatic Control, 49:217–229, 2004. 19. V.A. Yacubovich, “A frequency theorem for the case in which the state and control spaces are Hilbert spaces with an application to some problems of synthesis of optimal controls,” Sibirskii Mat. Zh., 15:639–668, 1975, (English translation in Siberian Mathematics Journal).

Getting Mobile Autonomous Robots to Rendezvous Zhiyun Lin, Bruce Francis, and Manfredi Maggiore Department of Electrical and Computer Engineering University of Toronto 10 King’s College Road Toronto, Ontario Canada M5S 3G4 [email protected], [email protected], [email protected]

1 Introduction Imagine n mobile robots, i.e., rovers, moving in the plane without human supervision. In addition to a motor drive, each robot has onboard a computer and a camera with which it can see the positions of some others relative to itself. The robots are assumed not to possess a common coordinate system— they don’t have GPS receivers, and there are no landmarks in view of all. Also, they are unable to communicate with each other. So it’s problematic if they can meet at a common location by distributed control strategies alone. This is called the rendezvous problem.1 Why take rendezvous to be the task? There are undoubtedly real situations where rendezvous is a goal: The robots should gather for servicing or recharging. In any event, rendezvous is the most basic formation task. It also arises in the notion of consensus [6]: A group of autonomous and distributed automata should come to agree on a piece of information. The robot rendezvous problem lies within a more general subject— distributed control of autonomous agents—where the goal is for a collective intelligence to emerge from relatively simple agents. One interesting aspect of that subject involves the structure of the couplings among the agents; that structure can be modeled by a graph. Returning to robots, let us create a directed graph G with n nodes labeled 1, . . . , n, one for each robot, and an arc from i to j if robot i can see robot j. 1

The Oxford English Dictionary deﬁnition of to rendezvous is “to assemble at a place previously appointed; also generally, to assemble, come together, meet.” Without a supervisor or global coordinates known to all, the robots are incapable of assembling at a place previously appointed. The best they can do is meet at some not pre-speciﬁed place.

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 119–137, 2006. © Springer-Verlag Berlin Heidelberg 2006

120

Z. Lin, B. Francis, and M. Maggiore

This visibility graph will be state-dependent, and consequently time-varying, if robots may come into and go out of view of each other. Our primary interest is in the connectivity properties of G that permit the existence of distributed control so that the robots rendezvous. Since our interest is in G, we’ll simplify the kinematics of the robots to the extreme and model them as points moving around continuously in the complex plane. Suppose the robots are numbered 1, . . . , n and the position of robot i is denoted zi (t). Suppose the velocity of robot i is directly controllable, so the kinematic equations are z˙i = ui ,

i = 1, . . . , n.

Let Ni denote those robots within view of robot i—its neighbours—and let yi denote the vector composed of the relative positions zj − zi , as j ranges over Ni . Thus yi is the allowed input to the controller on robot i. Example 1. Consider four robots where each can always see the next modulo 4: N1 = {2}, N2 = {3}, N3 = {4}, N4 = {1}. Thus G is the strongly connected graph in Figure 1 (there’s no implication that the robots are standing to form a square) and the sensed variables are

1

2

4

3

Fig. 1. Visibility graph: cyclic structure.

y1 = z 2 − z 1 , y2 = z 3 − z 2 , y3 = z 4 − z 3 , y4 = z 1 − z 4 .

(1)

The block diagram has the structure shown in Figure 2. Notice that the robots are coupled only through the visibility network, deﬁned by (1). Strong connectivity of G is more than suﬃcient for solvability of the rendezvous problem. A natural distributed control strategy to get the robots to rendezvous is ui = yi . Then the overall state vector z = (z1 , z2 , z3 , z4 ) satisﬁes z˙ = M z, where M is the cyclic matrix   −1 1 0 0  0 −1 1 0   M =  0 0 −1 1  . 1 0 0 −1 The following facts can easily be established:

Getting Mobile Autonomous Robots to Rendezvous

121

u1 z1 y1 visibility network Fig. 2. Block diagram.

1. The eigenvalues of M lie on a circle centred on the negative real axis; the circle is tangent to the imaginary axis at the origin; there is one eigenvalue at the origin; the vector (1, 1, 1, 1) is an eigenvector corresponding to the zero eigenvalue. From these it follows that the centroid of the robots is stationary, and all robots converge asymptotically to the centroid. 2. As the formation of robots evolves, if you zoom in so that the diameter of the formation is constant, you will see that the formation converges to an ellipse [17]. One could argue that this cyclic pursuit is not truly a distributed control strategy, in that it relies on a numbering of the robots. On the other hand, it is easy to engineer the robots to be distinguishable if there aren’t too many, by colour, for example. Finally, the control law ui = yi assumes instantaneous, noiseless sensor measurements. A more realistic model would allow time delay for image capture, errors in resolution, sampled data, and non-synchroneity in processors from robot to robot. Formation evolution under cyclic pursuit is studied in [13].

2 Fixed Visibility Graph In this section we continue with a ﬁxed G; that is, a robot’s set of visible neighbours is assumed to be ﬁxed over time. Obviously, a special situation since it requires the cameras to have unlimited visibility in terms of distance. We ﬁnd a necessary and suﬃcient connectivity condition for solvability of the rendezvous problem. Let mi denote the number of elements in Ni , so dim yi = mi ; robot i can see mi other robots. The permissible decentralized controllers are ui = Fi yi , where Fi is a real matrix of size 1 × mi ; that is, ui is permitted to be a linear combination of the relative positions zj − zi , j a neighbour of i. In the special case that robot i sees no other, then ui = 0.

122

Z. Lin, B. Francis, and M. Maggiore

The rendezvous problem is to ﬁnd, if they exist, permissible controllers so that, for every set of initial positions, zi (0), there exists a (rendezvous) point zr such that every zi (t) converges to zr . As we saw with cyclic pursuit, we have to allow zr to depend on the initial positions for the goal to be feasible. For example, if all the robots are initially placed at a point zi (0) = w, they’ll stay there forever. So it would have to be that zr = w. Example 2. Let there be four robots with G as Figure 3. The graph isn’t

1

2

3

4

Fig. 3. Visibility graph of four robots.

strongly connected, but, for example, node 2 is reachable from all others. We say node 2 is globally reachable. (So are nodes 3 and 4.) We have y1 = z 2 − z 1 , y2 = z 3 − z 2 , y3 =

z2 − z3 , y4 = z 3 − z 4 . z4 − z3

As before, let us take F1 = 1, F2 = 1, F3 = 1 1 , F4 = 1. Then the overall state equation is z˙ = M z, where   −1 1 0 0  0 −1 1 0   M =  0 1 −2 1  . 0 0 1 −1 The eigenvalues of M are {−3, −1, −1, 0} and (1, 1, 1, 1) is again an eigenvector for the eigenvalue 0. It follows that the robots rendezvous. Bringing in algebraic graph theory, introduce the adjacency matrix   0100 0 0 1 0  A= 0 1 0 1, 0010 the out-degree matrix (diagonal matrix of row-sums of A)   1000 0 1 0 0  D= 0 0 2 0, 0001

Getting Mobile Autonomous Robots to Rendezvous

123

and the Laplacian L = D − A. Then M = −L. The key property of L (and hence M ) is that 0 is a simple eigenvalue. And this is because G has a globally reachable node.

Here’s the main result for a ﬁxed G: Theorem 1. [14] The rendezvous problem is solvable iﬀ G has a globally reachable node. When the problem is solvable, one solution is ui =

(zj − zi ),

(2)

j∈Ni

or ui = 0 if robot i has no neighbours. A related result in discrete time (the controllers (2) solve the rendezvous problem iﬀ G has a globally reachable node) is in [21]. The proof of Theorem 1 uses classical algebraic graph theory [2]: For any digraph G, the eigenvalues of L are in Re λ ≥ 0; 0 is an eigenvalue of L, but there are no others on the imaginary axis; if G is strongly connected, then 0 is a simple eigenvalue. It follows that every trajectory of z˙ = −Lz converges to a point on ker L, the subspace of stationary points. If 0 is a simple eigenvalue, that is, dim ker L = 1, then, since (1, . . . , 1) is a spanning vector for ker L, the robots rendezvous. A trivial example where 0 isn’t simple is two blind robots: L is the 2 × 2 zero matrix. Thus the key to the proof of Theorem 1 is the following result derived independently in [21], [14], and [20]. Lemma 1. G has a globally reachable node iﬀ 0 is a simple eigenvalue of L. If G is symmetric, in the sense that if i sees j then j sees i (the adjacency matrix is symmetric), then we may as well consider G to be undirected. Then existence of a globally reachable node is equivalent to connectivity.

3 Limited Visibility Cameras In a seminal paper [8], Jadbabaie et al. study a system of “boids” moving at constant speed in the plane. Each boid has a heading angle, θi , evolving in discrete time. Also, at time t boid i can see a set Ni (t) of neighbours, and this leads to a time-varying graph G(t). Jadbabaie et al. assume if boid i can see boid j, then j can see i. Thus, G(t) is undirected. The paper studies a single control strategy, namely, at time t + 1 boid i changes its heading to the average heading at time t of itself and its neighbours. The paper proves that all the heading angles converge to a common value (similar to rendezvous) provided G(t) has a connectedness property over time, namely, there exists T > 0 such that the union graph t0 ≤t≤t0 +T G(t) is connected for all t0 .

124

Z. Lin, B. Francis, and M. Maggiore

Here, the union of graphs with the same node set is obtained by taking the union of the edges. Unfortunately, the condition on G(t) is not checkable—it would require an inﬁnite time simulation. The proof in [8] uses a theorem of Wolfowitz on ergodicity [27]. In the theorems in [8] the vector θ(0) of initial heading angles is ﬁxed. This allows the visibility graph to be a function of t alone, and not both t and θ(0). In actuality, for a sensible model of limited visibility, the graph is state dependent, not time dependent. To see this, let us return to our robots. Suppose robot i has an omnidirectional camera of range Ri . The set of visible neighbours of robot i is Ni (z) = {j : j ∈ {1, . . . , n}, |zj − zi | ≤ Ri }. Then there’s an edge in the visibility graph from i to j iﬀ j ∈ Ni (z), and therefore the visibility graph is a function of z , G(z). If the control strategy is given and if the state z(t) evolves uniquely from z(0), the visibility graph is a function of time, G(z(t)). Interesting insight into the rendezvous problem is presented by Moreau in [20]. The framework is nonlinear, discrete-time, and the rendezvous analysis is based on convexity instead of Wolfowitz’s theorem. The idea is, roughly speaking, that if every robot moves toward the interior of the convex hull of the set of neighbour robots at each step, rendezvous will result. Now we turn to the circumcentre control law, ﬁrst proposed for robot formations by Ando et al. [1]. Example 3. Consider six robots, with omnidirectional cameras of identical ranges, positioned at t = 0 as in Figure 4. The discs show the ﬁelds of view for robots 1 and 2. Since each camera has the same range, the visibility graph is undirected—Figure 5. Thus the neighbour sets at t = 0 are

3 2

4

3 5

6

1 Fig. 4. Six robots; the ﬁelds of view of robots 1 and 2 are shown.

2 1

4

6 5

Fig. 5. Visibility graph.

Getting Mobile Autonomous Robots to Rendezvous

3

3 2 1

4

125

2

6 5

1

Fig. 6. Direction of u1 .

4

5

6

Fig. 7. Direction of u2 .

N1 = {2}, N2 = {1, 4}, N3 = {4}, N4 = {2, 3, 5}, N5 = {4, 6}, N6 = {5}. The circumcentre control law is deﬁned as follows: Robot 1 has one neighbour, robot 2. Let Z1 = {z1 , z2 } and let c1 denote the circumcentre of Z1 —the centre of the smallest circle containing Z1 . Then set u1 = c1 − z1 ; see Figure 6. (In the picture, the little arrow is u1 translated from the origin to z1 .) So robot 1 moves towards the centre at t = 0: z˙1 = c1 − z1 . Actually, in this case where 1 sees only 2, clearly c1 = (z1 + z2 )/2, so at t = 0 z˙1 =

1 (z2 − z1 ). 2

Similarly, let c2 denote the circumcentre of the set {z2 } ∪ {zj : j ∈ N2 } and deﬁne u2 = c2 − z2 ; see Figure 7. And so on. These control laws can actually be implemented using onboard cameras, that is, relative positions, by translation. For example, for robot 2, the relative positions {z1 − z2 , z4 − z2 } are sensed. Let Z2 denote the set of points {0, z1 − z2 , z4 − z2 } (the translate of {z2 , z1 , z4 } by −z2 ), and let c2 denote the circumcentre of Z2 . Then deﬁne u2 = c2 . Let’s look at u1 again. The set Z1 equals {z1 , z2 } and so the circumcentre c1 of Z1 is a function of z; write c1 (z ). It turns out that c1 (z) is continuous in z, but not Lipschitz continuous—see below. In this way, the robots’ motions are governed by the coupled equations z˙1 = u1 (z) = c1 (z) − z1 .. . z˙6 = u6 (z) = c6 (z) − z6 , or in aggregate form z˙ = u(z), where the vector ﬁeld u(z) is only continuous, not Lipschitz. Thus uniqueness of a solution is not guaranteed. In what follows, a statement about a solution should be interpreted as applying to all solutions if indeed there is more than one.

126

Z. Lin, B. Francis, and M. Maggiore

The fact that the circumcentre control law isn’t a Lipschitz continuous function causes diﬃculty in its use, as we’ll see later. So it’s perhaps of interest to see a proof. Lemma 2. [3] The circumcentre control law isn’t Lipschitz continuous. Proof Construct three points {p1 , p2 , p3 } and their circumcentre c, and three perturbed points {p1 , p2 , p3 } and their circumcentre c , as shown in Figure 8. Deﬁne the vectors p = (p1 , p2 , p3 ), We’ll show that the ratio

p = (p1 , p2 , p3 ).

|c − c | p−p

is not bounded by a constant. This proves the mapping p → c isn’t Lipschitz. Let the radii of the circles be 1 and deﬁne x = |c − c |, y = |p2 − p2 |. Since p1 didn’t move and |p2 − p2 | = |p3 − p3 |, √ (3) p − p = 2y. Look at Figure 9. Deﬁne the angle θ. Then we have the lengths p1 q = cos θ, qc = sin θ. Thus qc = 1 − cos θ, so by Pythagoras on the small triangle qcc

p1

p1

θ

p2 p3 p2

c

q c

c

c

x 1

1

p3

Fig. 8. Three points, their displacements, and their circumcentres.

p2

1

y

p2

Fig. 9. Figure for analysis.

Getting Mobile Autonomous Robots to Rendezvous

127

x2 = (1 − cos θ)2 + sin2 θ = 2(1 − cos θ), and therefore qc = x2 /2. By Pythagoras again on the triangle qcc , the length x2 of qc equals x 1 − . Finally, apply Pythagoras to triangle qp2 c : 4 (y + 1)2 =

1+

2

x2 2

Thus we have x= so from (3)

+ x2 1 −

x2 4

= 2x2 + 1.

1 2 y + y, 2 2 1+ . y

1 x |c − c | =√ = p−p 2 2y

However, the great thing about the circumcentre law is that it preserves connectivity of the visibility graph. Thus we will only have to assume that the visibility graph is connected at t = 0. In fact, under the circumcentre control law, no links are dropped (though the distances between some neighbours may increase), so if G(z(0)) is connected, then G(z(t)) is connected for all t > 0. Of course, new links may form: As the robots rendezvous, eventually the graph becomes complete. Lemma 3. Under the circumcentre control law, over time no links are dropped in the visibility graph. Outline of Proof [16] For this proof it’s more convenient to view a robot position zi as a vector in R2 instead of a complex number. Let t ≥ 0 be arbitrary. Let Vij (z(t)) denote the distance-squared between two neighbour robots i and j, and let V (z(t)) denote the maximum distancesquared between any two neighbours: V (z(t)) = max i

max

j∈Ni (z(t))

Vij (z(t)).

Let I(z(t)) denote the set of pairs of indices where the maximum is attained; that is, (i, j) ∈ I(z(t)) iﬀ robots i and j are neighbours of maximum distance apart among all neighbours. Thus V (z(t)) =

max

(i,j)∈I(z(t))

Vij (z(t)).

We would like to show that d/dtV (z(t)) ≤ 0. Unfortunately, V (z(t)) is not diﬀerentiable. We need some non-smooth analysis—the upper Dini derivative:

128

Z. Lin, B. Francis, and M. Maggiore

D+ V (z(t)) = lim supτ →0+

V (z(t + τ )) − V (z(t)) . τ

Then, it is a fact that D+ V (z(t)) =

max

(i,j)∈I(z(t))

d Vij (z(t)). dt

(4)

In this way we get D+ V (z(t)) = = = = ≤

max

d zi (t) − zj (t) 2 dt 2 zi (t) − zj (t), z˙i (t) − z˙j (t)

max

2 zi (t) − zj (t), ui (t) − uj (t)

max

2{ zi (t) − zj (t), ui (t) + 2 zj (t) − zi (t), uj (t) }

max

2 zi (t) − zj (t), ui (t)

max

(i,j)∈I(z(t)) (i,j)∈I(z(t)) (i,j)∈I(z(t)) (i,j)∈I(z(t)) (i,j)∈I(z(t))

+

max

(i,j)∈I(z(t))

To conclude that we’ll show that

D+ V (z(t)) ≤ 0,

(5)

max

zi (t) − zj (t), ui (t) ≤ 0

(6)

max

zj (t) − zi (t), uj (t) ≤ 0.

(7)

(i,j)∈I(z(t))

and

2 zj (t) − zi (t), uj (t) .

(i,j)∈I(z(t))

To illustrate the argument, suppose (1, j) ∈ I(z(t)) for some j, that is, the maximum separation between robot neighbours occurs for robot 1 (and perhaps others). Suppose the neighbours of robot 1 are robots 2, 3, 4, 5; Figure 10 shows the circumcentre control vector u1 (translated to z1 ). The ﬁgure shows three neighbour robots—2, 3, and 4—on the smallest encompassing circle. Now in Figure 11 construct the line as shown through z1 perpendicular to u1 , and using this line as diameter, draw a second circle. In the shaded crescent there must be a neighbour of robot 1, for otherwise the encompassing circle in Figure 10 would be smaller (in fact it would be the unshaded circle). Consider the robot in the shaded crescent that is maximum distance from robot 1; in the ﬁgure it is robot 3. The angle between the vectors u1 and z1 − z3 is greater than π/2. Therefore z1 (t) − z3 (t), u1 (t) ≤ 0 and so

max

(1,j)∈I(z(t))

z1 (t) − zj (t), u1 (t) ≤ 0.

This proves (6), and (7) follows from this.

Getting Mobile Autonomous Robots to Rendezvous

z4

z4

z1 u1 z5

129

z1 u1

z3

z2 Fig. 10. Circumcentre law.

z5

z3

z2 Fig. 11. Construction for proof.

Finally, from (4) and (5), if two neighbours i and j are maximum distance apart (among all neighbours), then d/dtVij (z(t)) ≤ 0 and so the distance between them is non-increasing. Here’s the main result that the circumcentre control law solves the rendezvous problem: Theorem 2. [15] Suppose z(0) is such that G(z(0)) is connected. Under the circumcentre control law, the robots rendezvous. The proof [12] uses LaSalle’s theorem. Here we want to discuss the ideas without the details. Ideas for a Proof We’re given that G(z(0)) is connected. By Lemma 3, G(z(t)) is connected for all t > 0. Now G(z(t)) is either ﬁxed or it’s not. Suppose not. Then at some time a new link appears (no link is dropped). After this, G(z(t)) is either ﬁxed or it’s not. Suppose not. Then at some time, another new link appears. Since there are only ﬁnitely many nodes, this process must stop. Thus we may assume without loss of generality that G(z(t)) is ﬁxed and connected for all t ≥ 0. (We don’t assume the graph is complete, but it must actually be so, since the robots rendezvous.) Bring in Example 3 for illustrative purposes. Figure 12 shows the constellation at t = 0, its convex hull co{z(0)}, and the instantaneous velocities ui (z(0)) of the robots at the vertices. Even though the vector ﬁelds ui (z(0)) point into co{z(0)}, we can’t conclude that zi (t) ∈ co{z(0)} because we don’t have Lipschitz continuity. So it’s problematic to prove even that a solution z(t) is bounded. Let a be an arbitrary point in the plane and deﬁne the function V a (z) to be the distance squared from a to the farthest zi . (Again, we take the plane R2 .) Assume for simplicity that the farthest-away robot doesn’t

130

Z. Lin, B. Francis, and M. Maggiore

3 2 1

u3

z2

4

6 5

Fig. 12. The convex hull and the initial velocities of the vertex robots.

z3

z4

z1

z6 z5 z3 − a a

Fig. 13. The point a.

change, that it’s always robot 3; see Figure 13. Then V a (z) is diﬀerentiable and d d a V (z(t)) = z3 (t) − a 2 dt dt = 2 u3 (z(t)), z3 (t) − a . From the vector orientations in Figure 13, u3 (z(t)), z3 (t) − a ≤ 0. Thus V a (z(t)) is nonincreasing, and this kind of argument shows that z(t) is deﬁned for all t > 0 and is bounded. Now invoke LaSalle’s theorem. The solution converges to the largest invariant manifold M in {z : V˙ a (z) = 0}. To see what this manifold is, let z(0) ∈ M. Continuing with the assumption that the farthest-away zi (0) from a is z3 (0), we have that u3 (z(0)) and z3 (0) − a are orthogonal. Looking at Figure 13 we conclude that u3 (z(0)) = 0; for if u3 (z(0)) = 0 then z4 (0), the only neighbour of z3 (0), is farther from a than is z3 (0). Since u3 (z(0)) = 0, then z3 and z4 must be collocated at t = 0. If z2 (0) and z5 (0), the neighbours of z4 (0), are not also collocated with z4 (0), then z4 (t) will move away from z3 (0), which is impossible since M is invariant. Using this kind of argument, one can prove that for z ∈ M, all zi are equal. A rigorous proof is considerably more complicated since V a (z) is not actually diﬀerentiable. In [26] Tanner et al. study a group of autonomous mobile continuous-time boids. They propose distributed control laws, involving potential functions, that achieve convergence to a common heading while avoiding collisions. Persistent connectivity of the visibility graph is assumed rather than guaranteed.

Getting Mobile Autonomous Robots to Rendezvous

131

4 Related Work Here we look at the rendezvous problem, and extensions, in a few other contexts. Discrete-Event Robots Versions of the robot rendezvous problem have been studied extensively in computer science (where it is usually called the gathering problem). Let us look at [7] as an interesting example. Each robot is viewed as a point in the plane. The robots have limited visibility: Each can see only the other robots within a ﬁxed radius. Moreover, the robots are modeled as asynchronous discrete-event systems having four possible states: Wait, that is, not moving and idle; Look, during which the robot senses the relative positions of the other robots within its ﬁeld of view; Compute, during which it computes its next move; and Move, during which it moves at some pre-determined speed to its computed destination. There are soft timing assumptions, such as, a robot can be in Wait for only a ﬁnite period of time. The robots have local coordinate frames and these are assumed to have a common orientation, e.g, they may each have a compass; see Figure 14. The paper proposes the following control law, in the form of four if-then rules: 1. If in the Look state a robot sees a another robot to its left or vertically above, then it does not move. 2. If a robot sees robots only below on its vertical axis, then it moves down toward the nearest robot. 3. If a robot sees robots only to its right, then it moves horizontally toward the vertical axis of the nearest robot.

Fig. 14. Common orientation.

132

Z. Lin, B. Francis, and M. Maggiore

4. If a robot sees robots both below on its vertical axis and on its right, then it computes a certain destination point and performs a diagonal move down and to the right. It is proved that, assuming the initial visibility graph is connected, the robots rendezvous after a ﬁnite number of events. For example, starting as in Figure 14, the lower-right robot will not move, and the other three will become collocated with it. The proof is quite complicated, because, although each robot goes through a sequence of event cycles Wait-Look-Compute-Move, the robots are entirely unsynchronized, and so a robot may start to move before another has ﬁnished moving. Discrete-Time Robots Let us look in more detail at the model of Jadbabaie et al. [8] mentioned already. For simplicity, suppose there are only two boids, each a neighbour of the other. They move at unit speed with heading angles θ1 , θ2 with respect to the global frame. The model in [8] is 1 [θ1 (k) + θ2 (k)] 2 1 θ2 (k + 1) = u2 (k) = [θ1 (k) + θ2 (k)]. 2

θ1 (k + 1) = u1 (k) =

The heading angles converge in just one time step. It may not be obvious that these steering laws are feasible by onboard sensors, but they are. Rewrite the equations as 1 [θ2 (k) − θ1 (k)] 2 1 θ2 (k + 1) − θ2 (k) = [θ1 (k) − θ2 (k)]. 2

θ1 (k + 1) − θ1 (k) =

Thus the heading angles can be updated from the measured relative heading angles. Cort´es et al. [5] and Lin et al. [11] take the discrete-time robot model zi (k + 1) = zi (k) + ui (k),

(8)

that is, the position update zi (k + 1) − zi (k) is directly controllable. If only local onboard cameras are available, then again ui must be a function of the relative positions zj − zi , j ∈ Ni . The circumcentre law is applied in [5] to solve the rendezvous problem. Lin et al. propose properties for a distributed control law and prove rendezvous under these properties—with the additional assumption of graph connectivity, of course. The circumcentre law satisﬁes their properties, as does the following: Suppose robot 1 has two visible neighbours, 2 and 3. Let D1 , D2 , D3 denote the closed disks of radius R, the camera

Getting Mobile Autonomous Robots to Rendezvous

133

range, centred at z1 , z2 , z3 , respectively. Then robot 1 moves toward the centroid of D1 ∩ D2 ∩ D3 . Theorem 2 is the continuous time equivalent of these results. From our point of view of onboard sensors and distributed control, there’s an interesting point about discrete-time models. Consider, again for simplicity, just two robots, each the neighbour of the other. Suppose the robots head for each other according to the equations z˙1 = u1 = z2 − z1 z˙2 = u2 = z1 − z2 . Now suppose the onboard controllers are digital: The sensed signals z2 − z1 , z1 − z2 are sampled via a periodic sampler S with sampling period T , then converted from discrete time to continuous time via a zero-order hold H. See Figure 15. Then the model at the sampling instants is z1 [(k + 1)T ] = z1 (kT ) + T [z2 (kT ) − z1 (kT )]

(9)

z2 [(k + 1)T ] = z2 (kT ) + T [z1 (kT ) − z2 (kT )].

(10)

Note that it requires the two digital controllers in Figure 15 to be synchronized somehow, by a centralized clock and communication system. So the system isn’t really distributed. Lack of synchroneity would lead to jitter, which could alternatively be modeled. (It’s interesting to note that system (9), (10) is unstable for large enough T .) Is the continuous-time setup in Section 3 asynchronous, that is, are the onboard clocks of the robots independent? No, because t = 0 is the common starting time for all robots. However, one can show that if the initial times for the robots are independent, the robots use the circumcentre law, and the visibility graph is connected before all the initial times, then rendezvous occurs. Wheeled Robots The simplest wheeled robot is a unicycle. With respect to a global coordinate system, it has coordinates (x, y ) and a heading angle θ measured positively

−

S z2

H u2 H

u1

z1 S

−

Fig. 15. Two robots with digital control.

134

Z. Lin, B. Francis, and M. Maggiore

from the horizontal axis. The model is x˙ = v cos(θ) y˙ = v sin(θ) θ˙ = ω, where v is the forward speed input and ω the turning rate input. The rendezvous problem is much more challenging for unicycles. Rendezvous means that unicycles converge to a common point in the plane—there is no requirement of convergence to a common heading. The ﬁrst solution of the rendezvous problem for unicycles, proving global convergence, is [14]. There it is assumed that the visibility graph is ﬁxed, and it is proved that, remarkably, the rendezvous problem is solvable iﬀ the visibility graph has a globally reachable node—the same condition as for point robots! The controllers used for the proof are due to Yamaguchi and Burdick [28]. Suppose there are n unicycles and the forward speeds are all equal and constant. Then the control inputs are the turning rates only. Since the unicycles must keep moving, rendezvous is not possible, but certain moving formations are feasible. An equilibrium formation is one where all the unicycles appear stationary to each other. Justh and Krishnaprasad [10] proved that there are only two equilibrium formations: circular (all the unicycles move around a ﬁxed circle) and rectilinear (all the unicycles move in straight lines in the same direction). Cyclic pursuit of unicycles is treated in [18], [19]. In particular, local controllers are designed to achieve stable circular formations. Coupled Oscillators There are fascinating examples in nature of synchronized oscillations [25]. One example is where thousands of ﬁreﬂies ﬂash in unison after a transient period. A seminal study to understand synchronization was done by Kuramoto in 1975; the work is reviewed in [24]. Kuramoto’s model can be depicted as in Figure 16. There are n radial lines rotating, say, counterclockwise. Line i has phase θi and is rotating at a nominal angular speed of ωi . But the lines are coupled according to the equations θ˙i = ωi + k

sin(θj − θi ), j=i

where k > 0 is the coupling coeﬃcient. Synchronization means that all the rotating lines are merged into one. Notice in the Kuramoto model that the coupling is all-to-all—each line “sees” all the others. For identical local frequencies, i.e., ωi = ω, the transformation xi = θi − ωt yields the model x˙ i = k sin(xj − xi ). j=i

Getting Mobile Autonomous Robots to Rendezvous

135

θ2 θ1

Fig. 16. Two Kuramoto oscillators.

Then synchronization means that x1 (t), . . . , xn (t) all converge to a common value. This problem is studied, for example, in [24], [23], [16], [9].

5 Conclusions Much recent research on the rendezvous problem has resulted in non-checkable conditions on a time-varying visibility graph. What is needed instead is a control strategy that guarantees suﬃcient connectivity that rendezvous will result. The circumcentre law is one such strategy for point robots. Distributed control of mobile robots falls under the general heading of decentralized control. That subject, though quite old, has only a few main results. Consider the stabilization problem, for example: Stabilize a largescale system using only local control. There is a solution [4], but it’s not distributed; it says to make the overall system controllable from one local input, and then stabilize from there. By contrast, the solutions reviewed in this paper are distributed. Endnote by B.F. It is a great pleasure to contribute to this Festschrift on the occasion of Keith’s 60th birthday. My friendship and professional relationship with Keith began in 1976 at the University of Cambridge. Keith was a new Lecturer and I a Postdoctoral Fellow. We collaborated on an optimal control problem (“Bounded peaking in the optimal linear regulator with cheap control,” IEEE Trans. Aut. Control. 1978). Then in the 1980s we were both active in H∞ control theory and collaborated on the paper “State-space solutions to standard H2 and H∞ control problems,” IEEE Trans. Auto. Control, 1989. I have the fondest memories of both collaborations.

136

Z. Lin, B. Francis, and M. Maggiore

References 1. H. Ando, Y. Oasa, I. Suzuki, and M. Yamashita, “Distributed memoryless point convergence algorithm for mobile robots with limited visibility,” IEEE Trans. on Robotics and Automation, vol. 15, pp. 818–828, 1999. 2. A. Berman and R. J. Plemmons, Nonnegative Matrices in the Mathematical Sciences. Philadelphia: SIAM, 1994. 3. S. Bespamyatnikh, B. Bhattacharya, D. Kirkpatrick, and M. Segal, “Mobile facility location,” Fourth Int. ACM Workshop on Discrete Algorithms and Methods for Mobile Computing and Communication, pp. 46-53, 2000. 4. J. P. Corfmat and A. S. Morse, “Decentralized control of linear multivariable systems,” Automatica, pp. 479-495, 1976. 5. J. Cort´es, S. Martinez, and F. Bullo, “Robust rendezvous for mobile autonomous agents via proximity graphs in d dimensions,” to appear IEEE Trans. on Automatic Control. 6. M. H. DeGroot, “Reach a consensus,” J. of the American Statistical Association, vol. 69, pp. 118-121, 1974. 7. P. Flocchini, G. Prencipe, N. Santoro, and P. Widmeyer, “Gathering of asynchronous robots with limited visibility,” Proc. Int. Symp. Theoretical Aspects of Computer Science, Lecture Notes in Computer Science, vol. 2010, pp. 247-258, 2001. 8. A. Jadbabaie, J. Lin, and A. S. Morse, “Coordination of groups of mobile autonomous agents using nearest neighbor rules,” IEEE Trans. on Automatic Control, vol. 48, pp. 988-1001, 2003. 9. A. Jadbabaie, N. Motee, and M. Barahona, “On the stability of the Kuramoto model of coupled nonlinear oscillators,” Proc. American Control Conference, pp. 4296-4301, 2004. 10. E. Justh and P.. S. Krishnaprasad, “Steering laws and continuum models for planar formations,” Proc. IEEE Conf. on Decision and Control, pp. 3609-3615, 2003. 11. J. Lin, A. S. Morse, and B. D. O. Anderson, “The multi-agent rendezvous problem - part 1: the synchronous case,” submitted for publication, 2005. 12. Z. Lin, Coupled Dynamic Systems: From Structure Towards Stability, PhD thesis, in progress, University of Toronto, 2005. 13. Z. Lin, M. Broucke, and B. Francis, “Local control strategies for groups of mobile autonomous agents,” IEEE Trans. on Automatic Control, vol. 49, pp. 622-629, 2004. 14. Z. Lin, B. Francis, and M. Maggiore, “Necessary and suﬃcient graphical conditions for formation control of unicycles,” IEEE Trans. on Automatic Control, vol. 50, pp. 121-127, 2005. 15. Z. Lin, B. Francis, and M. Maggiore, “On the state agreement problem for multiple nonlinear dynamical systems,” Proc. IFAC World Congress, Prague, 2005. 16. Z. Lin, B. Francis, and M. Maggiore, “State agreement for coupled nonlinear systems with time-varying interaction,” submitted for publication, 2005. 17. J. Marshall, Coordinated Autonomy: Pursuit Formations of Multivehicle Systems, PhD thesis, University of Toronto, 2005. 18. J. A. Marshall, M. E. Broucke, and B. A. Francis, “Formations of vehicles in cyclic pursuit,” IEEE Trans. on Automatic Control, vol. 49, pp 1963-1974, 2004.

Getting Mobile Autonomous Robots to Rendezvous

137

19. J. A. Marshall, M. E. Broucke, and B. A. Francis, “Pursuit formations of unicycles,” Automatica, to appear. 20. L. Moreau, “Stability of multiagent systems with time-dependent communication links,” IEEE Trans. on Automatic Control, vol. 50, pp. 169-182, 2005. 21. W. Ren, R. W. Beard, and T. W. McLain, “Coordination variables and consensus building in multiple vehicle systems,” in Cooperative Control: A Post-Workshop Volume 2003 Block Island Workshop on Cooperative Control, Springer-Verlag, pp. 171-188, 2004. 22. W. Ren, R. W. Beard, and E. Atkins, “A survey of consensus problems in multiagent coordination,” Proc. American Control Conf., pp. 1859-1865, 2005. 23. R. Sepulchre, D. Paley, and N. Leonard, “Collective motion and oscillator synchronization,” in Cooperative Control: A Post-Workshop Volume 2003 Block Island Workshop on Cooperative Control, Springer-Verlag, pp. 189-205, 2004. 24. S. H. Strogatz, “From Kuramoto to Crawford: exploring the onset of synchronization in populations of coupled oscillators,” Physica D, vol. 143, pp. 1-20, 2000. 25. S. H. Strogatz, Sync: How Order Emerges from Chaos in the Universe, Nature, and Daily Life, Hyperion, 2003. 26. H. Tanner, A. Jadbabaie, and G. J. Pappas, “Stable ﬂocking of mobile agents, part II: dynamic topology,” Proc. IEEE Conf. on Decision and Control, pp. 2016-2021, 2003. 27. J. Wolfowitz, “Products of indecomposable, aperiodic, stochastic matrices,” Proc. Amer. Mathematical Soc., vol. 15, pp. 733-736, 1963. 28. H. Yamaguchi and J. Burdick, “Time-varying feedback control for nonholonomic mobile robots forming group formations,” Proc. IEEE Conf. on Decision and Control, pp. 4156-4163, 1998.

Experiment Design for Robust Control: Why Do More Work Than Is Needed? M. Gevers1 , X. Bombois2 , G. Scorletti3 , P. Van den Hof2 and R. Hildebrand4 1 2 3 4

CESAME, Universit´e Catholique de Louvain, Belgium [email protected] Delft Center for Systems and Control, Delft University of Technology, The Netherlands GREYC Equipe Auto, Caen, France IMAG, Universit´e J. Fourier, Grenoble, France

1 Introduction Experiment design for open-loop identiﬁcation Optimal input design for system identiﬁcation was an active area of research in the 1970’s, with diﬀerent quality measures of the identiﬁed model being used for this optimal design [1–3]. The questions at that time addressed open-loop identiﬁcation and the objective functions that were minimized were various measures of the parameter covariance matrix Pθ , where θ is the parameter vector of the model structure. Let the “true system” be given by: v(t)

G0 (z)

S : y(t) = G(z, θ0 ) u(t) + H(z, θ0 )e(t)

(1)

for some unknown parameter vector θ0 ∈ Rk , where e(t) is white noise of variance σe2 , while G(z, θ0 ) and H(z, θ0 ) are stable discrete-time transfer functions, with H(z, θ0 ) a monic and minimum-phase transfer function5 . In the optimal input design literature, it is assumed that this system is identiﬁed with a model structure M = {G(z, θ), H(z, θ)}, θ ∈ Rk , that is able to represent the true system; we shall call such structure a “full order model structure”. When Prediction Error identiﬁcation is used with a full order model structure, the estimated parameter vector θˆN is known to converge, under mild assumptions, to a Gaussian distribution: N →∞

(θˆN − θ0 ) −→ N (0, Pθ ),

(2)

where the asymptotic parameter covariance matrix Pθ can be estimated from the data. Important examples of optimal design criteria developed in the 5

By monic is meant that H(z) = 1 +

P∞

k=1

hk z −k .

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 139–162, 2006. © Springer-Verlag Berlin Heidelberg 2006

140

M. Gevers et al.

1970’s are D-optimal design which minimizes det(Pθ ), E-optimal design which minimizes λmax (Pθ ), and L-optimal design which minimizes tr(W Pθ ), where W is a nonnegative weighting matrix. In open-loop identiﬁcation, the dependence of the covariance matrix on the input spectrum is made apparent by the following expression: Pθ−1 = + Here, Fu (z, θ0 ) =

N 1 σe2 2π 1 N 2π

π −π π −π

ΛG (z,θ0 ) H(z,θ0 ) ,

Fu (ejω , θ0 )Fu (ejω , θ0 )∗ Φu (ω)dω

Fe (ejω , θ0 )Fe (ejω , θ0 )∗ dω

Fe (z, θ0 ) =

ΛH (z,θ0 ) H(z,θ0 ) ,

ΛG (z, θ) =

(3) ∂G(z,θ) ∂θ

and

ΛH (z, θ) = ∂H(z,θ) . The formula shows that the data length N and the input ∂θ spectrum Φu (ω) appear linearly in the expression of the information matrix Pθ−1 , and that, for a given data length N , the input spectrum is the only

design quantity that can shape the parameter covariance matrix. Zarrop used Tchebycheﬀ system theory to parametrize the input spectrum in terms of its so-called “trigonometric moments” with respect to the system [2]. These π Φu (ω) 1 trigonometric moments are deﬁned as mk = 2πσ 2 −π |H(ejω ,θ0 )|2 cos(kω)dω. e

The information matrix Mθ Pθ−1 can then be expressed as a ﬁnite linear combination of these moments, m0 , m1 , . . . , mn , which express the eﬀect of the ﬁltered input spectrum Φu (ω) on that information matrix. The number n depends on the degree of H(z, θ). These moments have to obey some positivity constraint in order for them to be generated by a genuine spectral density function. The optimal input design problem can then be reformulated as a constrained optimization problem in terms of these trigonometric moments. Once the optimal moment sequence has been obtained, it is an easy step to compute a corresponding quasistationary input signal u(t) that will match this optimal moment sequence. A solution can always be obtained using multisines. We refer the reader to [2–5] for background material on Tchebycheﬀ systems, trigonometric moments, and their use in input design problems. One should also note that another way of obtaining a ﬁnite parametrization of the information matrix is to restrict the admissible input signals to those that generate a ﬁnite dimensional parametrization of the spectrum Φu (ω), or to approximate the input spectrum by a ﬁnite dimensional parametrization: see e.g. [6]. The classical experiment design results of the 1970’s were limited to openloop identiﬁcation with full order model structures, and they were based on parameter covariance formulas. In the mid-eighties, Ljung and collaborators produced bias and variance formulas [7, 8] directly for the transfer function estimates, rather than for the parameter estimates which only serve as auxiliary variables in the representation of these transfer functions. The asymptotic variance formulas were derived under the assumption that the model order n tends to inﬁnity in some appropriate way when the data length N tends

Experiment Design for Robust Control

141

to inﬁnity. Thus, for the variance of the vector of transfer function estimates G(z, θˆN ), H(z, θˆN ), the following approximation was obtained in [7] under an assumption of high model order: Cov

G(ejω , θˆN ) H(ejω , θˆN )

n Φu (ω) Φue (ω) ∼ = Φv (ω) Φeu (ω) σ 2 N

−1

(4)

where n is the model order, N is the number of data, Φu (ω) is the input spectrum, Φv (ω) is the output disturbance spectrum, and Φue (ω) is the crossspectrum between u and e: see (1). When identiﬁcation is performed in open loop, we have Φue (ω) = 0. These variance formulas for the transfer function estimates paved the way for the formulation of goal-oriented experiment design problems. In [7] a number of open-loop optimal input design problems were formulated and solved. Experiment design for closed-loop identiﬁcation The ﬁrst closed-loop optimal design problems, including control-oriented problems, were formulated in [9]. In particular, it was shown in that paper that, when the model is identiﬁed for the purpose of designing a minimum variance controller, the optimal design is to perform the experiment in closed loop with the minimum variance controller in the loop. These results were later extended to other control design objectives in [10, 11]. The optimal design criterion used in all these contributions was the average performance degradation, namely the mean squared error between the output of the optimal loop (i.e. the loop that would be obtained if the optimal controller, dependent on the unknown true system, were applied to the system), and the output of the achieved loop (i.e. the loop in which the controller obtained from the estimated model is applied to the true system). The results were all based on the transfer function variance formulas of [7], derived under the assumption that the model order tends to inﬁnity, and it was observed in recent years that the use of these formulas for ﬁnite order models can sometimes lead to erroneous conclusions. This observation triggered a revival of interest in optimal design formulations based on variance expressions for ﬁnite order models. Experiment design for robust control Robust stability and robust performance criteria are typically expressed as constraints on frequency weighted expressions of the variance of the transfer function error, rather than as L2 performance criteria. For example, a robust stability constraint is typically formulated as V ar G(ejω , θˆN ) ≤ W −1 (ejω ) ∀ω

(5)

where W (ejω ) is a frequency weighting function that takes account of closedloop properties (e.g. robust stability condition). In order to formulate optimal input design problems in terms of control-oriented quality measures on G(ejω , θˆN ) such as in (5), using the ﬁnite model order formula (3) rather than

142

M. Gevers et al.

the asymptotic (in model order) variance formulas, several approaches can be taken. One commonly used approach to go from parameter covariance to transfer function covariance is to use the following ﬁrst order Taylor series approximation: σ 2 ∂G∗ (ejω , θ0 ) ∂G(ejω , θ0 ) V ar G(ejω , θˆN ) ≈ e Pθ (6) N ∂θ ∂θ This approach was adopted in [12] where it is shown that several useful H∞ design criteria can be reformulated as weighted trace optimal input design problems subject to LMI constraints. A sensible open-loop optimal input design problem can then be formulated as follows: min tr[W (θ0 )Pθ ] subject to

Φu (ω)

π −π

Φu (ω)dω ≤ α,

and Φu (ω) ≥ 0 ∀ω, (7)

where α is some positive constant. This is still a diﬃcult, inﬁnite dimensional optimization problem. However, by the use of Schur complement, the problem can be reformulated as a convex optimization problem under Linear Matrix Inequality (LMI) constraints. The numerical solution of such problems became possible in the nineties with the advent of interior point optimization methods [13, 14]. The problem becomes ﬁnite dimensional if the input spectrum Φu (ω) can be ﬁnitely parametrized. There are various ways of doing this, as noted earlier. Another approach to optimal input design for robust control is based on the use of the ellipsoidal uncertainty set Uθ centred on θˆN : Uθ = {θ|(θ − θˆN )T Pθ−1 (θ − θˆN ) < χ2 }.

(8)

It follows from the property (2) that the true parameter vector θ0 ∈ Rd belongs to Uθ with probability α(d, χ2 ) = P r(χ2 (d) ≤ χ2 ), where χ2 (d) denotes the χ2 distribution with d degrees of freedom. The results in [15, 16], which connect robust stability and robust performance measures directly to the ellipsoidal uncertainty region Uθ , allow one to formulate experiment design problems for robust control in terms of the minimization of some appropriate function of Uθ (or of Pθ ) without the need for the intermediate step of transfer function variance estimation, which typically requires both a Taylor series approximation and/or a conservative step of overbounding of the uncertainty set. The ﬁrst open-loop optimal input design problem for robust control based on the direct use of the uncertainty ellipsoid Uθ was formulated in [5]. The robust stability measure minimized in that paper, with respect to the input spectrum Φu (ω), was the worst-case ν-gap δW C (G(z, θˆN ), D) between the identiﬁed model G(z, θˆN ) and all models in the Prediction Error uncertainty set D {G(z, θ)|θ ∈ Uθ }: δW C (G(z, θˆN ), D) = sup δν (G(z, θˆN ), G(z, θ)) θ∈Uθ

(9)

Experiment Design for Robust Control

143

where the ν-gap is deﬁned in [17]. One of the merits of the worst-case ν-gap is that it is directly related to the size of the set of its stabilizing controllers: the smaller the worst-case ν-gap of the uncertainty set D, the larger is the set of controllers that stabilize all models in D. The optimal input design problem solved in [5] was min δW C (G(z, θˆN ), D) subject to Φu

π −π

Φu (ω)dω ≤ α, and Φu (ω) ≥ 0 ∀ω.

(10) The solution proposed in [5] is based on Tchebycheﬀ system theory (see above); the optimal solution can always be obtained as a multisine. Why do more work than is needed ? The traditional approach to optimal input design, as exempliﬁed by the problem formulations (7) or (10), has been to optimize some measure of the resulting uncertainty, subject to a constraint on the input signal power. However, in an identiﬁcation for robust control setting, one should not spend more eﬀort on the identiﬁcation than what is needed for the design of a robust controller, under the constraint that this controller must achieve stability and a prespeciﬁed level of performance with all models in the uncertainty set. This idea has led to the recent concept of “least costly identiﬁcation for control”, which was ﬁrst proposed in [18]. Instead of minimizing some measure of the uncertainty set, the objective is to deliver an uncertainty set that is just within the bounds required by the robust control speciﬁcations, and to do so at the smallest possible cost. In [19] open-loop identiﬁcation is considered and the cost is then deﬁned as the total input signal power. The idea of least costly (or minimum energy) identiﬁcation experiment for control has been further developed in an open-loop framework in [20]. Here we present the formulation of this optimal experiment design objective in a closed-loop disturbance rejection setup, where no reference excitation is applied in normal operation6 . The identiﬁcation cost is then deﬁned as the additional penalty that occurs in the control performance cost when an excitation signal is added for the purposes of doing the identiﬁcation. The full technical details can be found in [21]. We ﬁrst recall some basic results and tools about Prediction Error identiﬁcation for closed-loop systems.

2 Prediction Error Identiﬁcation Aspects We consider the identiﬁcation of the closed-loop system (1) using the full order model structure M = {G(z, θ), H(z, θ)}, θ ∈ Rk . In addition, we 6

This setup is very reminiscent of M. Gevers’ ﬁrst practical experience with experiment design in a closed-loop disturbance rejection framework on the lakes of Canberra, Australia in 1985, when he and Keith Glover had jointly bought their ﬁrst windsurfer and were identifying its dynamics.

144

M. Gevers et al.

assume throughout the paper that the model structure is globally identiﬁable at θ0 [22]. This means that θ0 is the only value of θ for which G(z, θ0 ) and H(z, θ0 ) represent the true system. Conditions for global identiﬁability of the commonly used model structures (ARX, ARMAX, BJ, etc) can be found in Theorem 4.2 of [22]. Note that they do not strictly require that all polynomial orders be known exactly. This true system is assumed to be operated in closed loop with a controller Cid : C : u(t) = r(t) − Cid (z)y(t) (11) to be replaced by a better controller: see Fig. 1. In normal operation, the external excitation signal r(t) is assumed to be zero. The objective is to perform a closed-loop identiﬁcation experiment in order to estimate a model that must lead to a new robust controller with better performance. This is typically achieved by applying an external excitation signal r, even though we shall see that this is not always necessary. The closed-loop system can be written as: yr (t)

y(t) = Sid v(t) + G0 Sid r(t)

(12)

ur (t)

u(t) = −Cid Sid v(t) + Sid r(t) with Sid = 1/(1+CidG0 ). Under normal operation, only the ﬁrst terms (driven by v) appear in the input and output signals. During an identiﬁcation experiment with an external excitation signal r, the controlled output and input of the closed-loop system contain added contributions yr and ur due to the excitation signal r(t); these are perturbations with respect to the normal operating conditions.

r(t) +

u(t)

v(t) + G0

+

y(t)

− Cid Fig. 1. Block-diagram of the closed-loop system

ˆ ˆ Consider now that a model G(z) = G(z, θˆN ), H(z) = H(z, θˆN ) of the true system is identiﬁed using a direct Prediction Error identiﬁcation method on the basis of N input-output data collected on the actual closed loop system with the controller Cid in the loop. If an external excitation signal r(t) is

Experiment Design for Robust Control

145

applied during this identiﬁcation experiment, we assume that it is a sample taken from a quasi-stationary signal (see [22]) for which a power spectrum Φr (ω) exists. The parameter vector estimate θˆN is deﬁned by: 1 Δ θˆN = arg min θ N

N

2

(t, θ)

(13)

t=1

Δ

where (t, θ) = H(z, θ)−1 (y(t) − G(z, θ)u(t)). Note that (t, θ) depends on the chosen signal r(t) via (12). We introduce the following cost function: ¯ 2 (t, θ) V¯ (θ) = E

1 N →∞ N

N

lim

E 2 (t, θ).

(14)

t=1

As shown in [22], if the identiﬁcation experiment is “informative enough”, then θˆN tends w.p.1 to a minimum of the cost function V¯ (θ); by our standing assumption that the system is in the model set and that the model structure is globally identiﬁable at θ0 , this cost function has θ0 as its unique minimum. The easiest and most common way to make a closed-loop experiment informative enough is to choose a reference signal that is persistently exciting of suﬃcient order (in closed-loop identiﬁcation, the required order of the excitation is related to the degree of the sensitivity function Sid ); however, a closed-loop experiment can also be informative enough even without any external excitation, provided the controller Cid is suﬃciently complex, i.e. provided its degree is suﬃcient: see [21, 23]. In this paper, we shall assume throughout that the experimental conditions are informative enough. Thus we make the following Standing Assumption. Assumption 2.1 With reference to the closed-loop identiﬁcation experiment presented above (see (12)-(13)), assume that the system is in the model set, that the model structure is globally identiﬁable at θ0 , and that the experimental conditions are informative enough so that the true parameter vector θ0 is the only global minimum of the cost function V¯ (θ) of (14). We observe that this assumption is restrictive only in its requirements on the model structure. The requirements on the experimental conditions can always be satisﬁed. Assumption 2.1 ensures that the uncertainty region constructed around θˆN contains θ0 (and not another minimum of V¯ (θ)) and that this uncertainty region is not inﬁnitely large. When Assumption 2.1 holds, we have the following result. Lemma 2.1 Consider the closed-loop identiﬁcation experiment described above (see (12)-(13)) together with Assumption 2.1. Then the identiﬁed parameter vector θˆN is asymptotically normally distributed around the true parameter vector θ0 , i.e. θˆN −θ0 converges in distribution to N (0, Pθ ), and the covariance matrix Pθ has the following expression [22]:

146

M. Gevers et al.

Pθ = with ψ(t, θ) = − ∂

σe2 ¯ E ψ(t, θ0 )ψ(t, θ0 )T N

−1

(15)

(t,θ) ∂θ .

The covariance matrix Pθ depends on the experimental conditions (i.e. the data length N and the spectrum Φr (ω) used during the identiﬁcation), as evidenced by the following expression of Pθ−1 , which is easily deduced from (12) and (15): Pr−1 (Φr (ω),θ0 ,σe2 )

Pθ−1 = N

1 1 σe2 2π

π −π

Fr (ejω , θ0 )Fr (ejω , θ0 )∗ Φr (ω)dω Pv−1 (θ0 )

+N

1 2π

π −π

Fe (ejω , θ0 )Fe (ejω , θ0 )∗ dω

G (z,θ0 ) , Fe (z, θ0 ) = Here, Fr (z, θ0 ) = Sid ΛH(z,θ 0)

∂G(z,θ) ∂θ

∂H(z,θ) . ∂θ

ΛH (z,θ0 ) H(z,θ0 )

(16)

− Cid Sid ΛG (z, θ0 ),

and ΛH (z, θ) = Note that Pθ−1 is made up of ΛG (z, θ) = a part depending on Φr (ω) and a part which does not depend on Φr (ω). Both parts are linear in N and both depend on the controller Cid . Note also that, in the vector Fr , the entries corresponding to the parameters that are only present in H(z, θ) are identically zero. The vector Fe has no identically zero entries; Fe is made up of the sum of two components: one component pertains to the parameters in H(z, θ) and one pertains to the parameters in G(z, θ). Lemma 2.2 ( [22]) Consider the closed-loop identiﬁcation experiment described above (see (12)-(13)) and assume that Assumption 2.1 holds. Then Pθ−1 is strictly positive deﬁnite: Pθ−1 0. Using Lemmas 2.1 and 2.2, it is possible to deﬁne an uncertainty region D(θˆN , Pθ ) around the identiﬁed model which contains the unknown true system G(z, θ0 ) at any desired probability level β [15, 24]: ZN (z)θ |θ ∈ U = {θ|(θ − θˆN )T Pθ−1 (θ − θˆN ) < χ} 1 + ZD (z)θ (17) where χ is a real constant dependent on the chosen probability level β and ZN , ZD are row vectors containing powers of z −1 and zeros. The size of the uncertainty region D(θˆN , Pθ ) is a function of the covariance matrix Pθ and thus, by (16), a function of the design parameters N and Φr (ω) used during the identiﬁcation experiment. D(θˆN , Pθ ) = G(z, θ) =

Experiment Design for Robust Control

147

3 Control Design Objectives and Control Design Method As stated before, our aim is to replace the present controller Cid in the loop of ˆ Fig. 1 by a new controller C(z) = C(G(z, θˆN )) that has better performance. We adopt the following control performance measure for a stable closed-loop system [C G]: ¯ G, C, Wl , Wr ) J(G, C, Wl , Wr ) = sup J(ω, ω

(18)

where ¯ G, C, Wl , Wr ) = σmax (Wl (ejω )F (G(ejω ), C(ejω ))Wr (ejω )) J(ω, Δ

F (G, C) =

(19)

G GC 1+GC 1+GC C 1 1+GC 1+GC

Here σmax (A) denotes the largest singular value of A, and Wl (z), Wr (z) are given diagonal ﬁlters which reﬂect the performance speciﬁcations that we want to achieve with the true system. The performance measure (18) is quite general: J(G, C, Wl , Wr ) ≤ 1 ensures that the four entries of Wl (z)F (G, C)Wr (z) have an H∞ norm smaller than one. Simpler H∞ criteria can be chosen as special cases. A controller C will be deemed satisfactory for the system G0 if [C G0 ] is stable and if J(G0 , C, Wl , Wr ) ≤ 1. As mentioned in the introduction, we want to design the new controller ˆ = G(z, θˆN ) of G0 . For this Cˆ = C(G(z, θˆN )) using an identiﬁed model G purpose, we use a pre-selected nominal control design method. Assumption 3.1 We have pre-selected a ﬁxed nominal control design method which maps the identiﬁed model G(z, θˆN ) to one controller C(G(z, θˆN )) which stabilizes G(z, θˆN ) and achieves with this model a nominal performance level J(G(z, θˆN ), C(G(z, θˆN )), Wl (z), Wr (z)) ≤ γ < 1,

(20)

where γ is a ﬁxed scalar, strictly smaller than 1. One possible control design choice that satisﬁes Assumption 3.1 is to choose for C(G(z, θˆN )) the central controller of the four-block H∞ control design method with performance objective (20). If Assumption 3.1 holds, then the controller Cˆ = C(G(z, θˆN )) designed ˆ = G(z, θˆN ) will achieve J(G, ˆ C, ˆ Wl , Wr ) ≤ γ < 1. from an identiﬁed model G ˆ When this controller C is applied to the true system G0 , the achieved performance will generically be poorer than the designed performance. However, by choosing the design criterion (20) with γ < 1, we ensure that there is a whole set of systems G(z) around G(z, θˆN ) that are also stabilized by Cˆ and that ˆ Wl , Wr ) ≤ 1. In the sequel, we will denote by Dadm (θˆN ) the achieve J(G, C, largest set of systems G(z) having these properties.

148

M. Gevers et al.

4 Demands on the Identiﬁcation Experiment Our objective is to determine the experimental conditions (N and Φr (ω)) of the identiﬁcation experiment on the loop [Cid G0 ] in such a way that the ˆ identiﬁed through this experiment, delivers a controller Cˆ which model G, ˆ Wl , Wr ) ≤ 1. Since G0 is stabilizes the unknown G0 and achieves J(G0 , C, unknown but lies (with probability β) in the uncertainty region D(θˆN , Pθ ) ˆ this performance constraint will be replaced by the identiﬁed along with G, following checkable constraint7 . Constraint 4.1 The experimental conditions (N and Φr (ω)) of the identiﬁcation experiment on the loop [Cid G0 ] (see Section 2) must be such that the ˆ = G(z, θˆN ) and the identiﬁed uncertainty region D(θˆN , Pθ ) identiﬁed model G ˆ Wl , Wr ) ≤ 1 for all G(z) ∈ D(θˆN , Pθ ), where have the property that J(G, C, ˆ ˆ using the control design method presented C is the controller designed from G in Assumption 3.1. ˆ Wl , Wr ) ≤ 1 for all G(z) in the set Dadm (θˆN ) deﬁned in the Since J(G, C, last paragraph of Section 3, Constraint 4.1 imposes that D(θˆN , Pθ ) is a subset of this set Dadm (θˆN ). We now discuss the requirements on the identiﬁcation design that will guarantee this. Remember that Dadm (θˆN ) is the largest set of systems G(z) around ˆ Wl , Wr ) ≤ 1. By Assumption 2.1 and Lemma 2.2, G(z, θˆN ) for which J(G, C, ˆ ˆ D(θN , Pθ ) ⊆ Dadm (θN ) is always achievable if N and/or Φr (ω) are chosen large enough: see (16). Moreover, we have the following trade-oﬀs: the larger N is chosen, the smaller Φr (ω) can be while still verifying Constraint 4.1; conversely, the larger Φr (ω) is chosen, the smaller N can be while still verifying this constraint. The sets Dadm (θˆN ) and D(θˆN , Pθ ) are both a function of the parameter vector θˆN that we want to identify. Moreover, besides being a function of Φr (ω) and N , Pθ is also a function of the unknown quantities θ0 and σe2 . Consequently, whatever the method we use to determine experimental conditions Φr (ω) and N satisfying Constraint 4.1, it will need to be based on some 2 initial estimates θo,est , θˆN,est and σe,est of those unknown quantities. In this context, we will state that Constraint 4.1 is satisﬁed for given N and Φr (ω) when D(θˆN,est , Pθ,est ) ⊆ Dadm (θˆN,est ) where Pθ,est is computed using (16) 2 with θ0 and σe2 replaced by θo,est and σe,est . Note that, in the methods we present in the sequel, we will not determine the set Dadm explicitly, but implicitly (see Theorems 7.1 and 7.2). 7

ˆ Wl , Wr ) ≤ 1 with all G(z) ∈ D(θˆN , Pθ ), then, under mild assumptions, If J(G, C, ˆ C(z) = C(G(z, θˆN )) also stabilizes all G ∈ D(θˆN , Pθ ).

Experiment Design for Robust Control

149

5 The Cost of a Closed-Loop Identiﬁcation Experiment It is clear that many possible choices of experimental conditions allow one to fulﬁll Constraint 4.1. Among those, we seek to determine an identiﬁcation experiment with the smallest possible cost. In this section, we give a precise deﬁnition of the cost of an identiﬁcation experiment in the context where the closed-loop system operates with an acting controller Cid and with a disturbance rejection performance objective. As mentioned in Section 2, in normal operation the signals u(t) and y(t) are given by: y(t) = Sid v(t), u(t) = −Cid Sid v(t). (21) By applying an external signal r(t) to the loop during the identiﬁcation, we introduce additional disturbances yr (t) and ur (t) on top of the normal operation signals: see (12). Those disturbances represent the cost of the identiﬁcation experiment, since they entail a performance degradation. The ideal closedloop identiﬁcation experiment would be one in which the normal operation signals u(t) and y(t) are used for a certain length N without any external excitation, i.e. with r(t) = 0. We show in Section 6 that such costless identiﬁcation experiment can, in certain circumstances, lead to fulﬁllment of the Constraint 4.1. We also show how to compute the minimum number Nmin of measurements that are necessary to reach this objective. In the cases where Constraint 4.1 can not be achieved with r(t) = 0, the application of a nonzero external signal r(t) for a certain amount of time is unavoidable, but we show how N and Φr (ω) can be chosen in order to achieve Constraint 4.1 with minimal cost. This cost can be a function of either the experiment time N , the power of the perturbations yr and ur , or a combination of both. In the sequel, we consider three diﬀerent situations which are representative of practical situations, and determine for each of them how we can optimally choose the experimental conditions. Situation 1. The cost of the identiﬁcation is mainly determined by the duration N of the identiﬁcation experiment. Based on the trade-oﬀ discussed in Section 4 between excitation power and duration of the experiment, the power spectrum Φr (ω) of the to-be-applied signal r(t) is in this case chosen at each frequency as large as the constraints on the actuators allow. For such ﬁxed Φr (ω), the optimal experiment time can subsequently be determined via an optimization problem yielding the smallest identiﬁcation time Nmin satisfying Constraint 4.1. Situation 2. Situation 2 is the converse situation: the cost of the identiﬁcation is mainly determined by the power of the perturbations yr (t) and ur (t) due to the excitation signal r(t). Based on the same trade-oﬀ, the experiment time N is in this case chosen as large as is allowed. For such ﬁxed N , the optimal power spectrum Φr (ω) can then be determined via an optimization problem

150

M. Gevers et al.

whose objective is to minimize the following cost function Jr representing the total disturbance power, subject to satisfaction of the Constraint 4.1: Jr = αy =

1 2π

1 2π π

−π

π −π

Φyr (ω) dω

+ αu

1 2π

π −π

Φur (ω) dω

αy |G0 (ejω )Sid (ejω )|2 + αu |Sid (ejω )|2 Φr (ω) dω

(22)

where αy and αu are scalars chosen by the designer to reﬂect the relative importance of the costs due to each of the perturbation signals, and where Φyr (ω) and Φur (ω) are the power spectra of these disturbance signals yr (t) and ur (t). Situation 3. Situation 3 is the situation where N and Φr (ω) are both important in the cost of the identiﬁcation. In this situation, we can determine the optimal spectrum Φr (ω) such as in Situation 2 for diﬀerent values of the length N . Since, for increasing values of N , the optimal cost function Jr decreases, such approach allows one to ﬁnd the “optimal” combination for the duration of the identiﬁcation experiment and the induced disturbance on the input and output signals.

6 Identiﬁcation Experiments Without External Excitation In this section we examine the situation where Constraint 4.1 can be achieved using an identiﬁcation experiment on the closed loop [Cid G0 ] without any external excitation signal r(t), i.e. using only the excitation due to the noise v(t) (see (12) with r = 0). We have the following result. Theorem 6.1 Consider a closed-loop identiﬁcation experiment as presented in Section 2 with r(t) = 0 and assume that Assumption 2.1 holds. Then, Constraint 4.1 can always be veriﬁed by using for the identiﬁcation a set of input-output data (21) of suﬃcient length N . Proof. When r = 0, expression (16) of Pθ−1 becomes: Pθ−1 = N Pv−1 (θ0 ).

(23)

By Assumption 2.1 and Lemma 2.2, we have Pv−1 (θ0 ) 0. Therefore, with N suﬃciently large, Pθ−1 = N Pv−1 (θ0 ) can be made such that D(θˆN , Pθ ) ⊆ Dadm (θˆN ) for any set Dadm (θˆN ) around G(z, θˆN ), which implies that Constraint 4.1 holds. Theorem 6.1 shows that, if Assumption 2.1 holds with r = 0, the identiﬁcation leading to a new and satisfactory controller Cˆ for G0 can be achieved without applying any external excitation: we just need to measure the input

Experiment Design for Robust Control

151

and output signal in normal operation for a suﬃcient amount of time. For this theorem to be of practical use, we need to examine under what conditions Assumption 2.1 holds when r = 0, and which smallest data length Nmin is required for the robustness Constraint 4.1 to be satisﬁed when r = 0. The computation of Nmin for the case r = 0 is a special case of the computation of Nmin for a given reference excitation spectrum Φr (ω); it will be presented in the next section. As for the ﬁrst question, a necessary and suﬃcient condition for Assumption 2.1 to hold is that (t, θ) = (t, θ0 ) =⇒ θ = θ0 [23]. In the case where r = 0, this condition specializes to the following result. Lemma 6.1 ( [23]) Consider the closed-loop identiﬁcation conﬁguration of Section 2 with r = 0. Then, Assumption 2.1 holds if and only if, for any θ, H −1 (z, θ)(1 + Cid G(z, θ)) = H −1 (z, θ0 )(1 + Cid G(z, θ0 )) =⇒ θ = θ0 .

(24)

A necessary condition for this identiﬁability condition to hold, in the case considered here of a linear time-invariant regulator Cid , is that the regulator be suﬃciently complex (i.e. of suﬃciently high order): see Section 13.4 of [22]. One can make this statement more precise by considering speciﬁc model structures. This has been done in [23] (see Complement C10.1) for the case of an ARMAX or ARX model structure, and in [21] for the case where an OE or BJ model structure is used. The conditions are essentially degree constraints on the structure of the controller Cid .

7 Least Costly Identiﬁcation Experiments for Control An identiﬁcation experiment without external excitation may be impossible for two reasons: i) the initial controller Cid , which is often not chosen by the user, is of lower complexity than required, or ii) the data length required to satisfy Constraint 4.1 (see Theorem 6.1) is unrealistic (e.g. Nmin corresponds to one year of data). We now address the problem of computing the least costly identiﬁcation experiment for control, as has been deﬁned by the problems presented at the end of Section 4. Those problems involve the computation under Constraint 4.1 of either the smallest data length for a given Φr (ω) (Situation 1) or of the power spectrum Φr (ω) minimizing Jr for a given N (Situations 2 and 3). 7.1 Shortest Identiﬁcation Experiment for Control with Fixed Φr(ω) The ﬁrst experiment design problem can be formulated as follows. Experiment Design Problem 1. Consider the closed-loop identiﬁcation experiment of Section 2. Consider also that the power spectrum Φr (ω) of the

152

M. Gevers et al.

excitation signal r(t) is given. Determine then the smallest length N of an excitation signal r(t) with power spectrum Φr (ω) that must be applied to [Cid G0 ] in order to fulﬁll Constraint 4.1. We show that this problem can be expressed as an LMI-based optimization problem [14]. For this purpose, we ﬁrst express the robust performance ¯ G, C, ˆ Wl , Wr ) ≤ 1 ∀G ∈ D(θˆN , Pθ ) at one particular frequency constraint J(ω, ˆ Wl , Wr ) ≤ 1 ω as an LMI, linear in Pθ−1 . Note that, according to (18), J(G, C, ˆ ¯ ˆ ∀G ∈ D(θN , Pθ ) ⇐⇒ J(ω, G, C, Wl , Wr ) ≤ 1 ∀ω and ∀G ∈ D(θˆN , Pθ ). Proposition 2 Consider the controller Cˆ = C(G(z, θˆN )) designed from the model G(z, θˆN ) using the control design method presented in Assumption 3.1. Consider also the set D(θˆN , Pθ ) deﬁned in (17). Then Cˆ achieves ¯ G, C, ˆ Wl , Wr ) ≤ 1 with all G in D(θˆN , Pθ ) if and only if ∃ τ (ω) > 0 J(ω, (τ (ω) ∈ R) and a skew-symmetric matrix L(ω) ∈ R(k+1)×(k+1) (i.e. L(ω) = −L(ω)T ) such that √ with j = −1 and

τ (ω)E(ω, θˆN ) − R(θˆN ) + j L(ω) ≤ 0

R(θˆN ) =

„

Ik T −θˆN

«

Pθ−1

„

Ik T −θˆN

«T

„ +

0 0 0 −χ

(25) «

« I4 0 Ω(ejω ) 0 −1 „ ««« „ 1 10 0„ ˆ ZN 0 C W I ⊗ W 0 2 r l A@ ZD 1A Ω(z) = @ 1 ˆ N 1 ZD + CZ 0 1 E(ω, θˆN ) = Ω ∗ (ejω )

„

The symbol ⊗ denotes the Kronecker product. Proof. See [21]. The previous proposition shows that Constraint 4.1 can be replaced by LMI’s at each frequency, linear in Pθ−1 . Expression (16) shows that Pθ−1 is linear in the decision variable N of Experiment Design Problem 1. By combining these two facts, it is easy to see that Experiment Design Problem 1 would be solvable exactly if Pθ−1 was not a function of θ0 and σe2 , and if condition (25) was not a function of the to-be-identiﬁed θˆN . This diﬃculty is inherent to all experiment design problems [22] and is generally circumvented 2 by using a-priori estimates for those quantities: θo,est , σe,est and θˆN,est . The problem can then be solved using the LMI optimization problem of Theorem 7.1 below. Note that θˆN,est will often be chosen equal to θo,est . Theorem 7.1 Consider the approximations θ0 ≈ θo,est , θˆN ≈ θˆN,est and 2 2 σe2 ≈ σe,est and the shorthand notations: Pr−1 (Φr (ω)) = Pr−1 (Φr (ω), θo,est , σe,est ),

Experiment Design for Robust Control

153

Pv−1 = Pv−1 (θo,est ) (see (16)) and E(ω) = E(ω, θˆN,est ) (see (25)). Then, the minimum duration N which solves Experiment Design Problem 1 is the solution (rounded up to the nearest integer) of the following LMI optimization problem: min N under the constraint that ∃ a frequency function τ (ω) valued in R and a frequency-dependent skew-symmetric matrix L(ω) valued in R(k+1)×(k+1) such that τ (ω)E(ω) − −

Ik T ˆ −θN,est 0 0 0 −χ

N (Pr−1 (Φr (ω)) + Pv−1 ) + j L(ω) ≤ 0

Ik T ˆ −θN,est

∀ω

T

(26)

Proof. Direct consequence of Proposition 2 and the expression of Pθ−1 in (16).

Comment 1. Condition (26) must be considered at every frequency. This is impossible in practice. The optimal N can nevertheless be approximated by using a ﬁnite frequency grid. An exact but more cumbersome solution consists of using the Kalman-Yakubovitch-Popov (KYP) lemma [25]; see [21] for details. Comment 2. As stated in Section 6, the minimal data length for an identiﬁcation experiment without external excitation can be determined via Theorem 7.1 as well, by setting Φr (ω) = 0 in (26). 7.2 Least costly identiﬁcation experiment with ﬁxed data length We now examine the second situation presented at the end of Section 4, in which the data length N is ﬁxed and one seeks to satisfy Constraint 4.1 with a power spetrum Φr (ω) that minimizes the identiﬁcation cost Jr deﬁned in (22). We restrict our search to signals whose power spectrum Φr (ω) can be written as [26]: m

Rr (i)cos(iω) ≥ 0

Φr (ω) = Rr (0) + 2

∀ω

(27)

i=1

where m is a positive integer selected by the user. The parameters Rr (i) (i = 0...m) can be interpreted as the auto-correlation sequence of a signal that has been generated by a white noise passing through an FIR ﬁlter of length m + 1. If we select Rr (i) = σ 2 NNc −i and m = Nc , then Φr (ω) represents the c power spectrum of a Random Binary Signal with clock period Nc and maximal

154

M. Gevers et al.

amplitude σ. Such a parametrization can be very useful if the amplitude of the time domain signal is constrained. An important property of the parametrization (27) is that Pθ−1 and Jr (see (16) and (22)) are aﬃne functions of the design variables Rr (i) (i = 0...m), as we show in the following two propositions. Note that other parametrizations of Φr (ω) have the same property and could therefore also be considered here: m e.g. Φr (ω) = i=1 Rr (i)δ(ω − ωi ) corresponding to a multisine signal r(t) m [27], or Φr (ω) = i=0 Rr (i) Bi (ejω ) + Bi∗ (ejω ) where Bi (ejω ) are preselected basis functions [28]. Proposition 3 Consider the expression (16) of Pθ−1 and let Φr (ω) be parametrized ˜ k (θ0 ) be the sequence of Markov parameters of Fr F ∗ i.e. by (27). Let M r ∞ Fr (ejω , θ0 )Fr (ejω , θ0 )∗ = k=−∞ M˜k (θ0 )e−jkω with Fr (z, θ0 ) as deﬁned in (16). Then, Pθ−1 ∈ Rk×k can be written as: Pθ−1

¯ (θ0 ) + =M

m

Mi (θ0 , σe2 ) Rr (i)

i=0

¯ (θ0 ) = N P −1 (θ0 ), M0 (θ0 , σ 2 ) = where M v e N T ˜ ˜ 2 (Mi (θ0 ) + M (θ0 )) for i = 1...m. σe

N σe2

˜ 0 (θ0 ), and Mi (θ0 , σ 2 ) = M e

i

Proof. Direct consequence of Result 5.6 of [26] applied to the closed-loop expression for Pθ−1 as given in (16). Proposition 4 Consider the cost function Jr deﬁned in (22) and let Φr (ω) be parametrized by (27). Then Jr can also be written as: m

[αy ci (θ0 ) + αu di (θ0 )]Rr (i),

Jr = [αy c0 (θ0 ) + αu d0 (θ0 )]Rr (0) + 2 i=1

∗ where the coeﬃcients ci (θ0 ) and di (θ0 ) are the Markov parameters of G0 G∗0 Sid Sid ∗ and Sid Sid respectively, i.e.

G0 (ejω )G0 (ejω )∗ Sid (ejω )Sid (ejω )∗ =

∞

ck (θ0 )e−jkω

k=−∞

and Sid (ejω )Sid (ejω )∗ =

∞ k=−∞

dk (θ0 )e−jkω .

Proof. Direct consequence of Result 5.4 of [26] applied to yr (t) = G0 Sid r(t) and ur (t) = Sid r(t). With the parametrization (27) for Φr (ω), the experiment design problem corresponding to Situation 2 can then be formulated as follows.

Experiment Design for Robust Control

155

Experiment Design Problem 2: Consider the closed-loop identiﬁcation experiment of Section 2 with a ﬁxed number N of data. Determine the parameters Rr (i) (i = 0...m) of the spectrum Φr (ω) in (27) which minimize Jr , subject to satisfaction of the Constraint 4.1. The experiment design problem described above would be solvable exactly, using the results of Propositions 3, 4 and 2, if the parametrizations of Pθ−1 and Jr with respect to the design variables Rr (i) were not functions of the unknown θ0 and σe2 , and if condition (25) was not a function of the to-beidentiﬁed θˆN . This diﬃculty is again circumvented by using a-priori estimates 2 and θˆN,est . The solution is then obtained by for those quantities: θo,est , σe,est solving the LMI optimization problem described in Theorem 7.2 below. Theorem 7.2 Consider Experiment Design Problem 2. Consider also the ap2 proximations θ0 ≈ θo,est , θˆN ≈ θˆN,est and σe2 ≈ σe,est and the shorthand no2 ¯ ¯ tations: ci = ci (θo,est ), di = di (θo,est ), M = M (θo,est ), Mi = Mi (θo,est , σe,est ) ˆ and E(ω) = E(ω, θN,est ). Then the auto-correlation sequence Rr (i) (i = 0...m) which solves Experiment Design Problem 2 is the solution of the following LMI optimization problem: min

Rr (i)(i=0...m)

[αy c0 + αu d0 ]Rr (0) + 2

m X

[αy ci + αu di ]Rr (i)

i=1

under the constraint that there exists a symmetric matrix Q of appropriate dimension, a frequency function τ (ω) valued in R and a frequency-dependent skew-symmetric matrix L(ω) valued in R(k+1)×(k+1) such that τ (ω)E(ejω ) − −

m

Ik −θˆT

¯ + M

0 0 0 −χ

+ j L(ω) ≤ 0

N,est

Mi Rr (i) i=0

Ik −θˆT

N,est

∀ω

Q − AT QA C T − AT QB T C − B QA D + DT − B T QB

and that

T

(28) ≥0

(29)

with the following deﬁnitions of A, B, C, D: 0 0 B = 1 0 ... 0 Im−1 0 C = Rr (1) Rr (2) ... Rr (m) D = Rr2(0) A=

The optimal spectrum Φr (ω) can thereafter be computed using (27). Proof. The existence of a symmetric matrix Q such that (29) holds is a necm essary and suﬃcient condition for Rr (0) + 2 i=1 Rr (i)cos(iω) to be positive at each ω and thus for (27) to represent a spectrum, as shown in [26] via the Positive Real Lemma. Consequently, the result in this theorem is a direct consequence of Propositions 3, 4 and 2.

156

M. Gevers et al.

Comment. Condition (28) must be considered at every frequency, which is impossible in practice. The optimal Φr (ω) can nevertheless be approximated by using a ﬁnite frequency grid. An exact but more cumbersome solution consists of using the Kalman-Yakubovitch-Popov (KYP) lemma [25], as shown in [21]. 7.3 Dealing with the Unknown Quantities θ0 , θˆN and σe2 Theorems 7.1 and 7.2 provide solutions to the optimal identiﬁcation experiment design problems deﬁned above. However, these solutions require that an approximation of θ0 , θˆN and σe2 be used. If those approximations are not accurate, this could lead to poor results. Here we present a procedure for dealing with those unknown variables. Let us ﬁrst recall the problem setup. One wants to replace the existing controller Cid , which operates on the true system, by a new controller via a new identiﬁcation of the true system. In order to design the experimental conditions of this identiﬁcation experiment, one needs reliable estimates of the unknown quantities θ0 , θˆN and σe2 . It is very often the case that estimates for θ0 and σe2 are already available, because the initial controller Cid has typically been computed from an initial identiﬁed model. This initial identiﬁcation typically delivers estimates not only of θ0 and σe2 , but also of uncertainty regions for those quantities. Moreover, it is also possible to deduce from this initial identiﬁcation a (truncated) Gaussian probability density function which deﬁnes the likelihood of each element of these uncertainty regions. If this is not possible, the density functions are then chosen uniform over the uncertainty regions. The estimate, the uncertainty region, and the probability density function of the to-be-identiﬁed θˆN are typically chosen equal to those of θ0 . To summarize, from the initial identiﬁcation, one can assume that q0 = T T ˆT lies in a set Q and that the likelihood of the event q = q0 is given θ0 θN σe2 by a probability function p(q). Based on this information, one can robustify the procedure that consists in adopting a unique and possibly poor estimate of q0 for the design of the experimental conditions by adopting instead an approach based on randomized algorithms (see e.g. [29, 30]). We brieﬂy describe such approach. In the case of Experiment Design Problem 1, one wants to determine the smallest duration N for which the Constraint 4.1 is veriﬁed, whatever the value of q0 . For this speciﬁc problem, since one assumes that q0 lies in Q, ˆ of sup this is equivalent to computing an estimate8 N q∈Q Nq where Nq is the solution obtained by Theorem 7.1 with the approximation q. Considering Nq as a function of q, this can be done [30] with accuracy and conﬁdence9 δ by generating n ≥ ln(δ −1 )/ln((1 − )−1 ) estimates qj of q0 according to the 8 9

The exact computation is NP-hard. ˆ ) ≤ ) ≥ 1 − δ. This means that P r(P r(Nq > N

Experiment Design for Robust Control

157

probability density function p(q), and by determining Nqj for each of these ˆ of supq∈Q Nq is then given estimates qj using Theorem 7.1. The estimate N by supqj (j=1...n) Nqj . In the case of Experiment Design Problem 2, the approach above can not be considered since one determines the parameters Rr (i) (i = 0...m) of Φr (ω) rather than Φr (ω) itself. The so-called scenario-approach can then be considered (see e.g. [29, Chapter 12]). This approach is also based on a randomized algorithm which uses the probability density function p(q). The main diﬀerence between the two approaches is that, for Experiment Design Problem 1, Theorem 7.1 is applied a ﬁxed number of times for diﬀerent estimates of q0 that are randomly generated with p(q); while for Experiment Design Problem 2, the optimization problem is solved only once but with several robust performance constraints (28), each of them evaluated at a diﬀerent estimate of q0 randomly generated with p(q).10

8 Simulation Results In order to illustrate our results, we consider the following ARX system [31] as the true system: y(t) =

1 z −3 B0 (z) u(t) + e(t) A0 (z) A0 (z)

(30)

with B0 (z) = 0.10276 + 0.18123z −1, A0 (z) = 1 − 1.99185z −1 + 2.20265z −2 − 1.84083z −3 + 0.89413z −4, and e(t) a realization of a white noise signal of variance σe2 = 0.5. The control performance criterion J(G, C, Wl , Wr ) focuses on the sensitivity function. It is deﬁned as in (18) with the ﬁlters: Wl (z) = diag (0, W (z)) Wr (z) = diag (0, 1) W (z) =

0.5165−0.4632z −1 1−0.999455z −1 .

The true system initially operates in closed loop with a controller Cid which has been designed using an initial estimate of the true system θo,est = (−1.9755, 2.1965, −1.8495, 0.8881, 0.0817, 0.172)T and the 4-block H∞ control design method of [32] that satisﬁes Assumption 3.1: Cid =

10

0.036249(z + 0.9244)(z 2 − 1.951z + 1.101) (z − 0.9995)(z 2 − 1.002z + 0.3641) (z 2 − 0.5109z + 0.8248)(z 2 − 0.1828z + 0.9416) × (z 2 − 1.279z + 0.835)(z 2 − 0.1746z + 0.9229)

(31)

For each of these robust performance constraints (28), the frequency functions τ (ω) and L(ω) are diﬀerent.

158

M. Gevers et al. 1

10

0

10

−1

10

−2

10

−3

10

−2

10

−1

ω

10

0

10

Fig. 2. Identiﬁcation without external excitation, with N = 4901: ˛ ˛ ˛ ˛ −1 −1 1 ˆ ˆ supG∈D(θˆN ,Pθ ) ˛ 1+C(e (solid), |(1 + C G) | (dashdot) and |W | (dotted) ˆ jω )G(ejω ) ˛

Identiﬁcation without external excitation. The complexity of the initial controller Cid is suﬃcient for Assumption 2.1 to hold with r(t) = 0. Thus, Constraint 4.1 can be veriﬁed with an identiﬁcation experiment that uses noise excitation only (see 21) provided the experiment is of suﬃcient length. The minimal length required can then be determined using Theorem 7.1 with Φr (ω) = 0 ∀ω. This theorem is applied here using the approximations θ0 ≈ θo,est and θˆN ≈ θo,est (an estimate of σe2 is not necessary since Pv−1 (θ0 ) is not a function of σe2 ). This delivers a minimal length Nmin = 4901. In order to verify the validity of this result, we have measured 4901 samples of the signals y(t) and u(t) obtained in normal operation on the loop [Cid G0 ] ˆ = G(z, θˆN ) along with its uncertainty region and we have identiﬁed a model G ˆ ˆ D(θN , Pθ ). From G, we have then designed a controller Cˆ using the method ˆ Wl , Wr ) ≤ 1 with of [32] and we have veriﬁed whether Cˆ achieves J(G, C, 1 jω −1 for all G all G in D(θˆN , Pθ ), or equivalently 1+C(e ˆ jω )G(ejω ) ≤ |W (e )| ˆ in D(θN , Pθ ). This is indeed the case as can be seen in Figure 2. Moreover, 1 jω −1 we also observe in Figure 2 that supG∈D(θˆN ,Pθ ) 1+C(e ˆ jω )G(ejω ) = |W (e )| in the low frequencies. Consequently, N = 4901 is indeed the smallest N for which Constraint 4.1 holds with Φr (ω) = 0 ∀ω.

Sensitivity to the initial estimates. In our example, the initial estimate θo,est chosen to approximate the unknown quantities θ0 and θˆN has delivered accurate results, as shown by Figure 2. This may not always be the case. Thus, it is safer to compute the minimal data length N using the method proposed in Section 7.3. We illustrate the application of this method to our example. In order to generate multiple estimates of θ0 (which are then used to approximate both the true θ0 and θˆN ), we have used the information provided by the initial

Experiment Design for Robust Control

159

identiﬁcation which had delivered θo,est . This was an open-loop identiﬁcation with Φu (ω) = 1 ∀ω and N = 500.11 Using the covariance matrix of θo,est , we have randomly generated 46 parameter vectors θi (i = 1...46) around θo,est ; 46 samples correspond to a conﬁdence of 80%. For each of these estimates, we have applied Theorem 7.1 and we have thus obtained 46 diﬀerent lengths Nθi . A more robust choice of the length N is then (see Section 7.3): maxθi Nθi = 5897 The standard deviation of these 46 Nθi was 383. 25

20

Jr

15

10

5

0

0

500

1000

1500

2000

2500 N

3000

3500

4000

4500

5000

Fig. 3. Cost of the least costly experiment that satisﬁes Constraint 4.1 (circles) and of the white noise experiment that satisﬁes the same constraint (crosses), for diﬀerent values of the data length.

Least costly identiﬁcation with external excitation. For the same example, we have also studied the eﬀect of applying least costly excitation signals r for a range of data lengths that were too short to lead to identiﬁability using only the noise excitation. Thus, we have computed the optimal signal spectra Φr (ω) resulting from Theorem 7.2 for data lenghts N ranging from 500 to 4,500 by steps of 500. For each of these data lenghts, we have then compared the identiﬁcation cost Jr resulting from the application of the optimal excitation signal (when m = 10 in (27)) with the cost that would result by applying a white noise reference excitation (i.e. m = 0 in (27)) with a variance that is suﬃcient to satisfy the robust performance Constraint 4.1. The comparison between the cost of the least costly experiment and the cost of a corresponding identiﬁcation experiment with white noise excitation is shown in Figure 3. As can be seen from this ﬁgure, the use of an optimally designed excitation signal r reduces the identiﬁcation cost by a factor of 2 to 3 whatever the data length. Similar comparisons leading to similar conclusions can be found for the case of open-loop identiﬁcation in [6]. 11

This initial identiﬁcation was too cheap to verify Constraint 4.1.

160

M. Gevers et al.

9 Conclusions We have presented a new paradigm for optimal experiment design in an identiﬁcation for robust control context, where the objective is to design an identiﬁcation experiment at the smallest possible cost. The identiﬁcation cost must be as small as possible while still delivering a model uncertainty set that just meets the robust control performance constraints. The cost of the identiﬁcation experiment is expressed either as the experiment time, or in terms of a measure of the deterioration of the closed-loop performance, during the identiﬁcation experiment, with respect to the closed-loop performance under normal (non perturbed) operation. This paradigm was initially proposed in [18], and subsequently elaborated upon and extended in a number of other papers [19, 21, 28]. The underlying theme of this new paradigm is “Why do more (identiﬁcation) work than is needed?” One might wonder whether it pays to apply the heavy mathematical machinery required to compute optimal input designs, given that the optimal solution necessarily depends on the unknown system, which means that a preliminary model estimate must be obtained ﬁrst before an approximately optimal input signal can be computed. This is sometimes referred to as adaptive (or iterative) optimal input design. In [33] the beneﬁts of optimal input design for control have been demonstrated for two benchmark problems. In both cases, signiﬁcant savings are obtained by the application of a two-step identiﬁcation procedure, where the second step uses an optimally designed input signal computed from a preliminary model estimate. From a practical point of view, the cost of identiﬁcation is an issue of major importance. It is often estimated that 75% of the cost associated to an advanced control project goes into model development. Even though the deﬁnition of the cost used in the present work on “least costly identiﬁcation for control” does by no means cover all the practical costs of modelling, the disruption caused to normal operation and the time required to arrive at a satisfactory model are considered to be very signiﬁcant elements of this total modelling cost. These two costs are incorporated in the “least costly” criterion developed in the present paper. Acknowledgements This paper presents research results of the Belgian Programme on Interuniversity Attraction Poles, initiated by the Belgian Federal Science Policy Oﬃce. The scientiﬁc responsibility rests with its authors.

References 1. R. Mehra, “Optimal input signals for parameter estimation in dynamic systems - survey and new results,” IEEE Trans. on Automatic Control, vol. AC-19, no. 6, pp. 753–768, December 1974.

Experiment Design for Robust Control

161

2. M. Zarrop, Optimal experiment design for dynamic system identiﬁcation. Berlin, New York: Lecture Notes in Control and Information Sciences, Vol. 21, Springer Verlag, 1979. 3. G. Goodwin and R. Payne, Dynamic System Identiﬁcation: Experiment Design and Data Analysis. New York: Academic Press, 1977. 4. S. Karlin and W. Studden, Tchebycheﬀ systems with applications in analysis and statistics. New York: Pure Appl. Math. 15, Wiley-Interscience, 1966. 5. R. Hildebrand and M. Gevers, “Identiﬁcation for control: optimal input design with respect to a worst-case ν-gap cost function,” SIAM Journal on Control and Optimization, vol. 41, no. 5, pp. 1586–1608, March 2003. 6. H. Jansson, “Experiment design with applications in identiﬁcation for control,” Ph.D. dissertation, Royal Institute of Technology (KTH), December 2004, tRITA-S3-REG-0404. 7. L. Ljung, “Asymptotic variance expressions for identiﬁed black-box transfer function models,” IEEE Trans. Automatic Control, vol. AC-30, pp. 834–844, 1985. 8. B. Wahlberg and L. Ljung, “Design variables for bias distribution in transfer function estimation,” IEEE Trans. Automatic Control, vol. AC-31, pp. 134–144, 1986. 9. M. Gevers and L. Ljung, “Optimal experiment designs with respect to the intended model application,” Automatica, vol. 22, pp. 543–554, 1986. 10. H. Hjalmarsson, M. Gevers, and F. D. Bruyne, “For model-based control design, closed-loop identiﬁcation gives better performance,” Automatica, vol. 32, pp. 1659–1673, 1996. 11. U. Forssell and L. Ljung, “Some results on optimal experiment design,” Automatica, vol. 36, pp. 749–756, 2000. 12. H. Jansson and H. Hjalmarsson, “A framework for mixed H∞ and H2 input design,” in Proc. MTNS, Leuven, Belgium, 2004. 13. Y. Nesterov and A. Nemirovskii, Interior Point polynomial algorithms in convex programming. Philadelphia: SIAM Studies in Applied Mathematics, 1994, vol. 13. 14. S. Boyd, L. E. Ghaoui, E. Feron, and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory. Philadelphia: SIAM Studies in Applied Mathematics, 1994. 15. X. Bombois, M. Gevers, G. Scorletti, and B. Anderson, “Robustness analysis tools for an uncertainty set obtained by prediction error identiﬁcation,” Automatica, vol. 37, no. 10, pp. 1629–1636, October 2001. 16. M. Gevers, X. Bombois, B. Codrons, G. Scorletti, and B. Anderson, “Model validation for control and controller validation in a prediction error identiﬁcation framework - Part I: theory,” Automatica, vol. 39, no. 3, pp. 403–415, March 2003. 17. G. Vinnicombe, “Frequency domain uncertainty and the graph topology,” IEEE Trans Automatic Control, vol. AC-38, pp. 1371–1383, 1993. 18. X. Bombois, G. Scorletti, P. Van den Hof, and M. Gevers, “Least costly identiﬁcation experiment for control: a solution based on a high-order model approximation,” in CD-ROM Proc. American Control Conference, Boston, MA, USA, July 2004, pp. 2818–2823. 19. X. Bombois, G. Scorletti, M. Gevers, R. Hildebrand, and P. Van den Hof, “Cheapest open-loop identiﬁcation for control,” in CD-ROM Proc. 33rd IEEE Conf on Decision and Control, The Bahamas, December 2004, pp. 382–387.

162

M. Gevers et al.

20. H. Jansson and H. Hjalmarsson, “A general framework for mixed H∞ and H2 input design,” submitted for publication to IEEE Trans. Auto. Control, 2004. 21. X. Bombois, G. Scorletti, Gevers, P. Van den Hof, and R. Hildebrand, “Least costly identiﬁcation experiment for control,” submittted to Automatica, 2005. 22. L. Ljung, System Identiﬁcation: Theory for the User, 2nd Edition. Englewood Cliﬀs, NJ: Prentice-Hall, 1999. 23. T. S¨ oderstr¨ om and P. Stoica, System Identiﬁcation. Hemel Hempstead, Hertfordshire: Prentice-Hall International, 1989. 24. X. Bombois, “Connecting prediction error identiﬁcation and robust control analysis: a new framework,” Ph.D. thesis, CESAME, Universit´e Catholique de Louvain, Tech. Rep., 2000. 25. V. Popov, Hyperstability of Control Systems. New-York: Springer-Verlag, 1973. 26. K. Lindqvist, On experiment design in identiﬁcation of smooth linear systems. Stockholm, Sweden: PhD Thesis, Royal Institute of Technology, 2001. 27. E. V. den Eijnde and J. Schoukens, “On the design of optimal excitation signals,” in Proc. 9th IFAC/IFORS Symp. on Identiﬁcation and System Parameter Estimation, Budapest, Hungary, 1991, pp. 827–832. 28. H. Jansson and H. Hjalmarsson, “Mixed H∞ and H2 input design for identiﬁcation,” in CD-ROM Proc. 43rd IEEE Conf on Decision and Control, The Bahamas, 2004, pp. 388–393. 29. R. Tempo, G. Calaﬁore, and F. Dabbene, Randomized algorithms for analysis and control of uncertain systems. New York: Springer Verlag, 2004. 30. R. Tempo, E. Bai, and F. Dabbene, “Probabilistic robustness analysis: explicit bound for the minimum number of samples,” Systems and Control Letters, vol. 30, pp. 237–242, 1997. 31. I. Landau, D. Rey, A. Karimi, A. Voda, and A. Franco, “A ﬂexible transmission system as a benchmark for robust digital control,” European Journal of Control, vol. 1, no. 2, pp. 77–96, 1995. 32. G. Ferreres and V. Fromion, “H∞ control for a ﬂexible transmission system,” in CD-ROM Proc. European Control Conference, Brussels, Belgium, 1997. 33. M. Barenthin, H. Jansson, and H. Hjalmarsson, “Applications of mixed H2 and H∞ input design in identiﬁcation,” in 16th IFAC World Congress on Automatic Control, paper 03882, Prague, July 2005.

Past, Present and Future of Automotive Control Lino Guzzella and Christopher Onder1 Swiss Federal Institute of Technology (ETH), Zurich, Switzerland [email protected], [email protected]

This paper contains a survey of the application of control systems theory to automotive systems. For this application ﬁeld, the most useful modeling approaches and control design methods are described. Some open problems are mentioned and possible trends for future developments are listed. A case study shows what role control systems may play for maximizing the eﬃciency of future vehicle propulsion systems.

1 Objectives of Automotive Control The design of feedforward and feedback control systems for automotive applications (here automotive control for short) is one of the most important application areas of classical and advanced control systems theory. The key aspect that radically diﬀerentiates the automotive ﬁeld from other areas where control systems play a vital role is the cost issue. While in aerospace and defense applications this argument comes much later in the priority list, in the automotive industry the designer is always faced with very stringent cost limitations. The request to include a new sensor that adds 0.1% to the total manufacturing cost can be the reason to stop that development project. On the other hand, a project that aims at developing an improved control algorithm that meets the speciﬁcations with a reduced number of sensors or actuators will easily receive substantial funding from management. The reasons for this are, of course, the ﬁerce competition in the automotive industry and the economies of scale. The value of electronic hardware and computer software built into automobiles by all OEMs is estimated to be of the order of 100 billion Euros per year. This ﬁgure is very likely to increase in the future. In particular the upcoming very demanding pollution emission limits and the increasing concerns regarding greenhouse eﬀects and fossil fuel depletion will cause an increased demand for embedded control systems. This will lead to the situation where

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 163–182, 2006. © Springer-Verlag Berlin Heidelberg 2006

164

L. Guzzella and C. Onder

approximately one fourth of the manufacturing cost of a passenger car will be induced by mechatronic components and the corresponding software. Figure 1 shows the typical complexity of a modern gasoline engine system. Control systems theory (feedforward and feedback control, system identiﬁcation, monitoring and diagnosis, etc.) may be a hidden technology (Astrom), but in this problem area (as in many others) it is an enabling technology as well without which a systematic and cost-eﬃcient problem solution is not possible. ECU

ET

MA

TA

VE

PM

CP

FP

IC

λ1

AK

CCV

CCV

AK CP IC MA SE FP

CC

TWC

SA TE

Tank

knock sensor camshaft sensor ignition command air mass-flow sensor engine speed sensor fuel pressure control

DP

λ2 TWC

SE

PM ET TA TE CC λ1,2

manifold pressure sensor electronic throttle intake air temperature sensor cooling water temperature sensor active carbon canister air/fuel ratio sensors

VE SA TWC ECU CCV DP

EGR valve secondary air valve 3-way catalyst controller CC control valves driver pedal

Fig. 1. Schematic representation of the sensors and actuators present in modern gasoline engines.

In automotive applications there are three main areas where control systems play a vital role: 1. pollution abatement, 2. active safety, and 3. fuel consumption. Looking back, the pollution abatement problem, in particular the feedforward and feedback control of the air/fuel ratio necessary to achieve the desired conversion rates of three-way catalytic converters, was the entry door by which embedded control systems were accepted by the traditionally rather conservative automotive industry. This process, which started in the seventies, was inexorably accelerated by more stringent pollution legislation and the development of aﬀordable active safety systems (automatic braking, vehicle stability enhancement, collision detection and appropriate passenger protection systems, etc.). The fact that by the year 2000 almost 50% of all new cars sold in Europe were equipped with electronic stability systems (ESP, ASC, etc.) may be taken as a token for that development.

Past, Present and Future of Automotive Control

165

The pollutant emissions of modern gasoline engines have been reduced to almost negligible levels. It is clear that there is no real need to further reduce these emissions, in particular beyond the planned EURO V legislation. This is not true for Diesel engines1 and for novel combustion methods that are discussed at the moment. Without sophisticated control systems these approaches will not work properly. Huge progress will be made in active safety systems as well. Radar and video sensors will be used to observe the traﬃc situation in which a vehicle is operated. Appropriate data interpretation algorithms will provide information to safety systems that can indirectly (driver information) or directly (active steering and braking) react to impending dangers. Finally, the fuel economy of road vehicles must be substantially improved as well. Automotive control systems will be necessary to cope with the increasing complexity of such fuel-eﬃcient power trains. Downsized and supercharged IC engines, automated gear boxes, hybrid-electric and fuel cell power trains, navigation and communication systems, path planning and inter-vehicle communication and coordination are a few of the key words that describe the plethora of opportunities that arise in this area.

2 Engine System Models 2.1 Model Classes Because of their reciprocating behavior, internal combustion engines are inherently discrete-event systems. The single events (ignition, injection, valve closing, etc.) are triggered by the crank angle which is the independent variable of this process. However, on very short and on very long time scales, several eﬀects are time-based such that a very detailed description should use a hybrid systems approach. Examples of the former class of eﬀects are the combustion delays, the ﬂame propagation, and the torque peaks. The latter class includes manifold dynamics, turbocharger speed, and the engine’s thermal behavior. In the models proposed so far the single-combustion events were mostly disregarded since a mean-value description proved to be suﬃcient in most cases (see below). In future, novel combustion processes such as the controlled auto ignition (or homogeneous charge compression ignition) approach will require a very precise cycle-by-cycle process control. In particular the rate of recirculated (hot) exhaust gas must be precisely controlled in such systems. Novel actuators and sensors and appropriate control algorithms will be needed for that. 1

Particularly in heavy-duty and oﬀ-road applications, Diesel engines are unlikely to be replaced by any alternative prime mover. Due to their superior fuel economy, Diesel engines have recently gained a big market share in Europe in the passenger car segment as well.

166

L. Guzzella and C. Onder

Several authors have proposed discrete-event models of IC engines (see for instance [10] and [11]). A common simpliﬁcation is obtained by assuming constant engine speeds. In this case, the engine dynamics may be described using a discrete-time approach. It was shown in [18] that this simpliﬁcation is usually well justiﬁed and that the time constants of all other relevant dynamic eﬀects are much smaller than those of the engine-speed dynamics. In the mean-value paradigm, the engine is assumed to be working as a continuous-ﬂow machine such as a gas turbine. This approximation is equivalent to the assumption that the engine has inﬁnitely many inﬁnitely small cylinders, with the total displacement remaining constant. 2.2 Some Important Examples In gasoline engines one of the most salient dynamic eﬀects is caused by the wall-wetting phenomenon in the intake part (see Fig. 2). The dynamics of this eﬀect can be described by 1 1 d mf (t) = − mf (t) + β u(t), y(t) = mf (t) + (1 − β) u(t), dt τ τ

(1)

where u(t) is the amount of fuel to be injected (in a mean-value setting this is a mass ﬂow) and y(t) is the amount of fuel entering the cylinder. The correct modeling of this eﬀect and the identiﬁcation of the parameters τ and β is still one of the most important problems in automotive control [2]. In particular during cold starts or after fuel-cut-oﬀ events, nonlinear eﬀects cause substantial deviations from the description (1) [1], [18].

intake

injector exhaust wall-film mass mf (t)

Fig. 2. Wall wetting in port-injected gasoline engines; the variable mf (t) represents the total mass of fuel stored on the walls of the intake ports.

For eﬃciency reasons, in all modern Diesel engines the fuel is directly injected into their combustion chamber. Since this process is very fast, the main dynamic eﬀects in such engines are not caused by the fuel path. On

Past, Present and Future of Automotive Control

167

the other hand, since all modern Diesel engines are turbocharged2 this device renders this type of engine systems hard to control.3 up

p2

ui injection

ωtc

p1

ϑ1

compressor (fluid- and thermodynamics) .

mc

Tc

.

ϑc

mα

intake receiver p2

p3

ϑ2 gas exchange

.

ωe

.

ϑ2

mϕ

mα

p3

combustion and torque generation

ϑe

.

.

.

mt

mα+mϕ outlet receiver

Tl

Te

engine inertia

ωe

uvnt

ϑ3

p3

p4

ωtc

turbine (fluid- and thermodynamics) .

mt

Tt

Tc

inertia turbocharger

ωtc

Fig. 3. Causality diagram of a turbocharged Diesel engine (exhaust gas recirculation not shown).

Figure 3 shows the causality diagram of such a turbocharged Diesel engine. Obviously, the turbocharger introduces a feedback loop from the exhaust to the intake side of the engine. The mechanical inertia of the turbocharger is part of this loop. Not surprisingly, this causes substantial delays and requires advanced automatic control methods for a correct operation. 2 3

Without turbocharging, the power density would be too low, whereas using larger naturally aspirated engines would lead to higher costs and a reduced fuel economy. The situation is exacerbated by the additional exhaust gas recirculation (EGR) loop that is required to lower the N Ox emissions.

168

L. Guzzella and C. Onder

Both nonlinear and linear mean-value models have been proposed [3] to represent such engine systems. The behavior of this system is diﬃcult to model at very low turbocharger speeds [4] where large parameter variations and uncertainties must be taken into consideration. Unfortunately, this situation is often encountered in average driving conditions. Accordingly, the design of controllers for turbocharged engine systems must be very robust [5].

3 Control Systems 3.1 Introduction

amplifier, relays, etc.

event controller (TPU)

D/A converter, digital output

microcontroller

input signals from engine

A/D converter, digital input

From the beginning, internal combustion engines had to be designed to be homeostatic systems. In steady-state conditions they nominally operate on periodic orbits. Unavoidable disturbances (combustion ﬂuctuations caused by turbulence eﬀects, chaotic piston/wall slapping, load torque disturbances, etc.) are rejected by feedback loops that in older engines were realized using hardware components. In gasoline engines the main stabilizing eﬀect is the feedback loop from the crank angle to the ignition event. Traditionally, this loop was realized using mechanical ignition distributors; in modern engines two magnetic pickups sense the crank-angle position while solid-state semiconductor ampliﬁers actuate the ignition coil. Similarly, in Diesel engines the main stabilizing eﬀect is the feedback loop from the crank angle to the injection event. Traditionally, this loop was realized using in-line pumps that were mechanically connected to the crank shaft; in modern engines the same sensors as in gasoline engines are used to control the electromagnetically actuated fuel injectors. In this sense “burn-by-wire” systems have become the standard approach.

command signals to engine

crank-angle pulses

Fig. 4. Hardware structure of an engine control unit including time-based (slow) and event-based (crank-angle triggered) control channels.

Due to the engine’s reciprocating behavior, all engine control systems contain what is known as a time processing unit (TPU). These circuits (often on

Past, Present and Future of Automotive Control

169

the same chip as the microcontroller itself) interface the several asynchronous processes present in engine control loops. The adaptation of, say, the ignition angle to varying engine speeds is a time-based and relatively slow process. However, the ignition itself has to be triggered in each engine cycle at precisely the desired crank-angle location. This task is accomplished by the TPU. In addition to the basic loops mentioned above, all modern IC engines contain many higher-level control systems. The main emphasis is on feedforward systems while feedback, if used at all, is usually rather simple (SISO PI loops). The feedforward control systems often use tabulated values for the main system input commands (injection, ignition, EGR rate, etc.) with engine speed and load (torque) as main scheduling variables. These maps (see the example of a spark-advance map shown in Fig. 5) are calibrated oﬀ line. In these calibrations the manufacturers minimize the fuel consumption and optimize the comfort while ensuring that the pollutant legislation are satisﬁed. Some maps are partially adapted during operation, but in general the factory settings are kept constant.

spark advance [° ca]

40

20

0 100

6000

80 4000

60 relative engine torque [%]

40

2000

20 0

engine speed [rpm]

0

Fig. 5. Spark advance as a function of engine speed and load (torque).

These map-based control loops produce a static controller behavior. In other loops dynamic feedforward control systems are used as well. For instance the wall-wetting dynamics (1) are (partially) compensated for by choosing the injection command as follows u(t) = ∗

1 1 ∗ mf,d (t) − m ˆ f (t) , 1−β τ

(2)

where mf,d (t) is the desired mean-value fuel mass ﬂow entering the cylinder and m ˆ f (t) the estimated mass of fuel in the wall ﬁlm. Of course the parameters

170

L. Guzzella and C. Onder

τ and β depend on the engine speed, load, temperature, etc. These dependencies must be experimentally identiﬁed by the manufacturer and are stored in appropriate tables for later use. This process is very cumbersome and its complexity increases with each additional interacting loop. As mentioned above, feedback loops are realized using rather direct approaches. For instance the feedforward control loop for the mass of fuel to be injected described by Eq. (2) must be complemented with a feedback loop to attain the required precision of the air/fuel ratio (see below). However, the bandwidths of these loops are rather modest and feedback is only used to reject very low frequency disturbances and, of course, modeling errors in the feedforward path. Simple PI controllers are mainly used whose parameters are scheduled according to the main engine operating conditions (speed, load, temperature, etc.). Several authors have proposed and tested advanced design methods such as H2 or H∞ -optimal designs [6]. These approaches showed clear improvements over the traditional solutions. However, because the speciﬁcations for the feedback loops are relatively easy to meet, so far few of these advanced approaches have been applied in practice. This situation will change when the speciﬁcations become tighter and when the increasing plant complexity enforces MIMO control systems to be used. 3.2 Some Important Examples All IC engines include several feedback control loops. In addition to the basic loops mentioned in the last section, several additional loops may be identiﬁed. Almost all gasoline engines have an air/fuel ratio control loop, which is necessary for the correct operation of the catalytic converter [14]. Figure 6 shows the conversion eﬃciency of such a device. To achieve a simultaneous reduction of all three of the main pollutant species N Ox (oxides of nitrogen), HC (hydrocarbons), and CO (carbon monoxide), the air/fuel ratio has to be kept on the average very close to its stoichiometric value with an error of less than 0.1%. Fortunately, the catalyst is able to smooth out any fast transient deviations [13] such that the air/fuel ratio feedback loop can be chosen to have a rather small bandwidth. Moreover, periodic excitations around that value may improve the system behavior [12]. Another feedback control system that is present in most IC engines is the idle-speed control loop. Despite unobservable disturbances, such systems permit the engine’s idling speed to be kept at low values and, thus, fuel savings [7]. Due to the large nonlinearities and uncertainties always present in engine systems, these control loops must be very robust [9]. Moreover, the transfer from regular to idling operation (and vice versa) requires sophisticated transition strategies [8]. As mentioned above, novel approaches to combustion hold the promise of combining excellent fuel economy with very low engine-out pollutant emissions. The key idea is to very rapidly burn highly diluted, homogeneous mix-

Past, Present and Future of Automotive Control

171

conversion efficiency % 80

HC

60 40

CO

20 NOx 0.9

1

1.1

λ

Fig. 6. Conversion eﬃciency of a warmed-up three-way catalytic converter (100% complete removal of harmful species . . .) as a function of normalized air/fuel ratio λ (λ = 1 stands for stoichiometric conditions).

tures. The fast combustion yields excellent thermodynamic eﬃciencies and the high dilution reduces peak temperatures and hence the formation of N Ox . In addition, since the mixture is premixed (homogenous), almost no particulate matter is formed during the combustion. Unfortunately, the reaction kinetics of such combustion processes are very diﬃcult to control. Moreover, it is impossible to use this approach in the full range of engine operation. Therefore, in automotive systems, these controlled auto-ignition operation modes must be smoothly connected with other combustion modes. Fully controllable valves will be a corner stone of such systems. Only such actuators will allow for a precise management of the amount of recirculated exhaust gases that will play a very important role in controlling the reaction kinetics. The modeling and control of these devices is a challenging design problem [15] and [16]. 3.3 Future Trends and Open Problems The most important developments in automotive control are expected to happen in the following areas: • • • •

hybrid control systems (discrete and continuous dynamics), automated and model-assisted control system calibration and veriﬁcation, adaptive and “learning” systems, and model-based optimal feedforward control.

The ﬁrst development will be needed in order to realize novel combustion systems that must be controlled on a cycle-by-cycle basis while time-based processes have a big inﬂuence as well.4 The challenge here will be the un4

Auto-ignition combustion engines lack the stabilizing eﬀect of a crank-angle-based feedback provided in traditional engines by the ignition (gasoline) or injection (Diesel) events. For this reason, the kinetics of the auto-ignition combustion must be carefully controlled by imposing the gas temperature and composition and the ﬂow conditions inside the cylinder.

172

L. Guzzella and C. Onder

derstanding of the basic dynamic phenomena and the formulation of feedback algorithms that are able to react within one engine cycle to reject disturbances. As mentioned above, the number of interacting control loops is inexorably increasing. The design, the experimental calibration, and the veriﬁcation of these systems on a purely experimental basis is very time consuming, better solutions would be very welcome. Key points here are software reusability and guaranteed performance conditions that must be satisﬁed under all circumstances. Because of the ever increasing system complexity, the third development is an absolute must. Adaptive feedforward and feedback control could help to cope with relaxed manufacturing tolerances and thus could lower the system price. These algorithms must be guaranteed to work properly and to be very robust. Some must be able to quickly adapt to new situations, while others will be allowed to require long adaptation intervals (for instance, the continuous improvement of feedforward controls). Finally, the potential oﬀered by the increasing computational power of modern microprocessors must be tapped for the optimization of the engine and power train management. Advanced vehicles will include powerful communication and navigation devices. Model-based control systems, capable of optimizing the fuel consumption while taking into consideration the planned trip and the actual traﬃc situation are feasible within the next few years.

4 Case Study – The PAC Car II Project 4.1 Introduction The vehicle analyzed in this case study, called “PAC Car II,” was designed for the Shell Eco-marathon events, which are competitions that reward the lowest fuel consumption. The PAC Car II vehicle is equipped with a fuel cell electric power train. To compare its hydrogen consumption to cars which use liquid hydrocarbons, the lower heating value of those fuels is taken as the reference. Besides the optimization of each component of the vehicle and its power train, the driving strategy had to be carefully chosen in order to achieve the best possible fuel economy. This optimization is the topic of the analysis presented below. First a short description of the main features of the vehicle is given. Then, starting from the formulation of the optimal control problem (OCP) various aspects of the optimization of the driving strategy are discussed in this case study. With such an optimized driving strategy PAC Car II won the Nogaro (F) Shell Eco-marathon race in May 2005. A fuel consumption of 7 g of hydrogen per 100 km was recorded in that race (using the lower heating values of hydrogen and gasoline, this corresponds to an equivalent mileage of 3836 km per liter gasoline). In the Shell Eco-marathon in Ladoux (F), one month later the same vehicle set a new world record for fuel eﬃcient ground vehicles by

Past, Present and Future of Automotive Control

173

achieving an equivalent mileage of 5385 km per liter gasoline (5 g of hydrogen per 100 km). 4.2 Description of PAC Car II The PAC Car I and II vehicles were developed in the years 2000 to 2005 by a team of mechanical engineering graduate and undergraduate students at ETH Zurich. Figure 7 shows a picture of PAC Car II taken during the Nogaro 2005 event.

Fig. 7. The PAC Car II vehicle at the 2005 Shell Eco-marathon in Nogaro (F).

The vehicle was designed and manufactured according to the rules deﬁned by the Shell Eco-marathon organizers. The main parameters are listed in Table 1. The vehicle is driven by two optimized direct current motors. The electric energy is produced by a fuel cell system fed by compressed hydrogen. The high-level control of the powertrain, i.e., the control of the fuel cell system, the monitoring of all important data in the vehicle, and the realization of the driving strategy are performed by an on-board computer system. Higheﬃciency DC/DC converters that distribute the electric power to the highperformance electric motors are connected to the control system via an optical link. The total force Ftot which acts on a road vehicle is the sum of the propulsion force u, the aerodynamic resistance Fa , the rolling resistance Fr , and the gravitational force Fg Ftot = u − Fa − Fr − Fg ρ = u − Af cd v 2 − cr m g cos(γ) − m g sin(γ), 2

(3) (4)

174

L. Guzzella and C. Onder Table 1. Main parameters of the PAC Car II vehicle frontal area Af aerodynamic friction coeﬃcient cd mass of the vehicle m rolling friction coeﬃcient cr

0.254 0.075 29 0.001

m2 kg -

where the variable γ stands for the inclination of the track at the actual position x (g = 9.81 m/s2 is the earth gravitational acceleration). Since only small inclinations must be considered, this force may be approximated by ρ Ftot ≈ u − Af cd v 2 − m g cr − m g γ. (5) 2 More detailed models are known [19], but such descriptions are not needed for the purposes discussed in the following sections. 4.3 Optimal Control Problem The problem of driving on a speciﬁc track while minimizing the fuel consumption can be formulated as an optimal control problem [17]. In a ﬁrst step, the force at the wheel is minimized. Using Newton’s law and (5), the following equations deﬁne the dynamics of the OCP x˙ = v v˙ = c1 u − c2 v 2 − c3 − c4 γ(x)

(6) (7)

As mentioned above, the travelled distance is symbolized by x, the vehicle speed by v, and the slope of the track by γ(x). The coeﬃcients ci immediately follow from (5) dividing all constant terms by the vehicle mass m. Braking the vehicle always results in a loss of energy and thus must be avoided under all circumstances. Accordingly, the force at the wheels has to be between zero and a maximum value u ∈ [0, umax ]. The objective function, which has to be minimized, is the total work at the wheel over the demanded time tb J(u) =

tb

0

u(t) v(t) dt,

v > 0 ∀t.

(8)

The Hamiltonian associated to this OCP can be written as H = u v + λ1 v + + λ2 (c1 u − c2 v 2 − c3 − c4 γ(x)).

(9)

Since the problem is time-invariant, the Hamiltonian has to be constant. The diﬀerential equations that deﬁne the dynamics of the co-state vector are ∂γ λ˙1 = λ2 c4 ∂x

(10)

λ˙2 = −u − λ1 + 2 λ2 c2 v.

(11)

Past, Present and Future of Automotive Control

175

According to the minimum principle, the Hamiltonian has a global minimum for the optimal control input H(x◦ (t), v ◦ (t), u◦ (t), t, λ◦ (t)) ≤ H(x◦ (t), v ◦ (t), u(t), t, λ◦ (t)),

(12)

where ◦ indicates the optimal trajectory. Taking into account the constraints for the input, the following control strategy can be deﬁned  0 if v ◦ + λ◦2 c1 > 0  u◦ = using =? if v ◦ + λ◦2 c1 = 0 (13)  u max

if v ◦ + λ◦2 c1 < 0

Below, the variable h(t) = v ◦ + λ◦2 c2 will be referred to as the switching function. When h(t) ≡ 0 for a ﬁnite interval of time, the optimal control input cannot be derived directly from the switching function and the system is on a singular arc. In this case the derivatives of h(t) with respect to time have to be used to ﬁnd the input u(t). The ﬁrst derivative of h(t) is given by ˙ h(t) = v˙ + c1 λ˙ 2

(14)

= 2 λ2 c1 c2 v − c1 λ1 − c2 v 2 − c3 − c4 γ(x). Inserting h = 0 results in ˙ h(t) = −3 c2 v 2 − c1 λ1 − c3 − c4 γ(x).

(15)

Unfortunately, the input u does not explicitly appear in this expression. Therefore, an additional derivation is necessary ¨ = −6 c2 v (c1 u − c2 v 2 − c3 − c4 γ(x)). h(t)

(16)

With this equation the optimal control input using on the singular arc can be found 1 (c2 v 2 + c3 + c4 γ(x)). (17) using = c1 Using (7) it is easy to show that using results in v = constant. This result is somewhat surprising because the slope of the track is not constant. 4.4 Optimization of the Driving Strategy The discussion of the OCP showed that only three types of control signals need to be considered, viz. full load (boost), keeping the speed constant (sustain) and no power (coast). Using these three possibilities, some subproblems are now to be discussed in order to ﬁnd a solution to the full problem.

176

L. Guzzella and C. Onder

Small Elevations, Full Cycle Since the aerodynamic resistance is quadratic in the velocity it is obvious that as long as the downhill forces do not exceed the sum of rolling and aerodynamic frictions, the smallest constant speed which exactly satisﬁes the average-speed boundary condition will be the solution for the inﬁnite-horizon OCP. For a ﬁnite horizon it might be preferable to sustain a slightly higher speed and to recover some of the kinetic energy by adding a coasting phase at the end. This problem is analyzed ﬁrst because it can be solved without being forced to solve a two-point boundary-value problem; a simple parameter optimization suﬃces to ﬁnd the optimal driving strategy. 8

velocity [m/s]

6

4

2

0

0

50

100

150

200

2800

2850

2900

2950

3000

time [s]

Fig. 8. Speed traces for the two diﬀerent strategies to solve the “small elevations” problem.

Figure 8 shows the traces of the velocities for the inﬁnite-horizon strategy and the optimized ﬁnite-horizon strategy for a speciﬁc average-speed situation. Figure 9 illustrates the demanded energy at the wheel for varying sustain speeds. The smallest sustain speed corresponds to the inﬁnite-horizon strategy. Thus, in this speciﬁc case, reducing the end speed to approximately half of the sustain speed results in a 2.8% reduction of the demanded energy. 4

energy at wheel [J]

4.2

x 10

4.15 4.1 4.05 4 3.95 6.7

6.8

6.9

7

7.1 7.2 vhold [m/s]

7.3

7.4

7.5

Fig. 9. Integrated energy at the wheel for varying sustain speeds.

Past, Present and Future of Automotive Control

177

Large Elevations, Inﬁnite Horizon For a small and aerodynamically optimized vehicle even small elevations can result in an acceleration when driving downhill. Thus the force at the wheels can be set to zero and the control strategy has to be changed. Here, the inﬁnite-horizon problem for a speciﬁc race track is discussed. The race track chosen is the motor speedway in Rockingham (UK). Its elevation proﬁle is illustrated in Fig. 10. An average minimum speed of vavg,des = 6.7 m/s is required by the race regulations. The length of one lap on the track is slap = 2244 m. velocity [m/s] , traction force on wheel [N] , elevation [m]

14 12 10

velocity

8

powertrain force

6 4

elevation

t1

2

t2 tend

0

0

500

1000 1500 distance [m]

2000

2500

Fig. 10. Elevation proﬁle, optimal vehicle speed, and force at the wheel calculated for the Rockingham race track.

Knowing the only possible form of the optimal control signals, the following driving strategy is proposed: • • •

starting with an initial velocity v0 , a coasting phase is chosen ﬁrst since the slope of the trace at the start line is known to be negative, at t1 the control is switched on to keep a constant speed vsust , and at t2 the motors are shut oﬀ again until tend is reached.

The unknown variables v0 , t1 , t2 , and tend are chosen such that the energy at the wheel is minimized and that the boundary conditions: • • •

v(tend ) = v0 , s(tend ) = slap , and vavg,dem = s(tend )/tend

all are satisﬁed.

178

L. Guzzella and C. Onder

The solution of a numerical optimization problem with four unknown variables and three constraints poses quite some problems to standard numerical solvers. Therefore another method is proposed: • • • • • •

choose v0 as the independent variable of the optimization problem, use vsust to satisfy the boundary conditions, integrate the diﬀerential equations forward starting with t = 0, s = 0, and v = v0 until vsust is reached with decreasing velocity; this yields the two uknown variables t1 and s(t1 ), integrate the system equations backwards starting with t = slap /(vavg,dem ), s = slap , and v = v0 until vsust is reached with increasing velocity (with decreasing time) resulting in t2 and s(t2 ), vary vsust until vsust = (s(t2 ) − s(t1 ))/(t2 − t1 ), and vary v0 in order to minimize energy at wheel.

Energy at wheel [J]

5640 5630 5620 5610 5600 5590 2.7

2.8

2.9

3

3.1 3.2 v0 [m/s]

3.3

3.4

3.5

3.6

Fig. 11. Energy at the wheel for varying initial speeds.

The driving strategy for the Rockingham race track obtained with this approach is illustrated in Fig. 10. The sensitivity of the optimum to varying initial speeds is shown in Fig. 11, whereas Fig. 12 shows the dependency of the sustain speed for varying initial speeds. The sustain speed strongly depends on the initial speed, whereas the energy at the wheel varies only by approximately 1% in the range investigated. Thus it is important to choose the correct combination of initial and sustain speeds in order to meet the average-speed constraint. However, the proposed control strategy is robust with respect to variations in these parameters, i.e., the energy demand will not substantially increase even if the chosen initial and sustain speeds are suboptimal. So far the behavior of the subsystem providing the propulsion energy has not been included in the optimization. All results obtained so far remain valid if this subsystems has a constant eﬃciency when transforming chemical energy stored in the hydrogen to mechanical energy that is absorbed by the traction wheel. Fortunately, this condition is (approximately) satisﬁed for the fuel cell propulsion system of PAC Car II, as the eﬃciency map of Fig. 13 shows. The

Past, Present and Future of Automotive Control

179

vsustain [m/s]

6.5 6 5.5 5 4.5 2.7

2.8

2.9

3

3.1 3.2 v0 [m/s]

3.3

3.4

3.5

3.6

Fig. 12. Sustain speed for varying initial speeds.

gear ratio between motors and traction wheel must be chosen such that the operation limits are not violated and such that the propulsion system operates most of the time in the best-eﬃciency region (approximately at an eﬃciency of 0.45 in this example). The limits of operation are deﬁned by the motor characteristics (current and power limits) and the polarization curve of the fuel cell. These limits are shown in Fig. 13 as well.

c)

0.35

0

5

0

0.4

5

0.4

0.2 0.1

a)

0.35

0.1

0.45

5

0.4

0.4

0 0 .1 0.25 .15 0.3 0.2

0.2

0.3 0.25

motor torque [Nm]

b)

0.3 0. 5 0.253 0.2 0. 1 0.05 2000

0.4

0.45

0. 0.3 35 0.2 5 0.2 0.15 0.15 0.1 0.05 4000 motor speed [rpm]

0.45 0.4 0.35 0.3 0.25 0.1 0.05 0.2 6000

Fig. 13. Tank-to-wheel eﬃciency map of the PAC Car II propulsion system.

Since PAC Car II is built for best aerodynamic performance, at narrow curves the vehicle may roll over if the speed is too high. Therefore, for a race track with curves with small radii, the speed in the turns has to be limited as well. The track of Nogaro (F), whose main characteristics are shown in Fig. 14, belongs to this class. If this restriction is taken into account, the space of possible solutions to the OCP is signiﬁcantly reduced.

180

L. Guzzella and C. Onder

NOGARO MOTOR CIRCUIT

15 14 13

16

17 18

12

TOP

21

BOTTOM

20

19

8

11

9 10

7 6 22

5 27 4 3 2

26

28 29 30

25

31

1

24

23

32

33 34 35

Fig. 14. Layout and elevetaion information of the race track in Nogaro (F).

In Fig. 14 positive elevations are indicated with a square, negative ones with a circle. The two critical curves are in the segments 16 and 23. In these curves the speed must not be higher than 10m/s. Thus, the following strategy is chosen: • • • • •

drive with constant speed until the vehicle reaches that point from which, by shutting oﬀ the motor, the vehicle coasts to the appropriate speed at the boundary between segments 10 and 11, continue coasting until the sustain speed is again reached in the next ascent, since in this phase the grade is so steep that the speed cannot be held constant, use a short boost (maximum torque) phase here, after that, a coasting phase follows, such that the vehicle does not violate the top speed limitation at the trasition point from segment 22 to 23, drive with constant speed to the end of the lap.

The elevation, the speed of the vehicle, the traction force, and the external force to the wheel are indicated in Fig. 15. If the the torque that the motors are supposed to produce is very low, a bad eﬃciency of the powertrain results (see Fig. 13). In such situations, the motors are shut oﬀ until the vehicle speed drops to 2% below the demanded value. Then, a short high-torque phase is commanded until the demanded speed is again reached. This duty-cycle mode is continued as long as necessary.

Past, Present and Future of Automotive Control

181

velocity [m/s] , traction force on wheel [N] , ext. forces [N] , elevation [m] 15 velocity 10 powertrain force

5

0 external force −5

0

500

1000

elevation 1500 2000 distance [m]

2500

3000

3500

Fig. 15. Driving strategy for Nogaro (F).

Outlook The strategies presented in this case study utilize the insight about the form of the optimal control signals provided by the minimum principle. With this information and using heuristic arguments, the OCP is simpliﬁed to a nonlinear programming problem, which then can be solved easily. Another method to ﬁnd optimal control strategies is to use a dynamicprogramming approach. This method is particularly well suited to optimization problems in which the elevation and the speed proﬁle are known a priori. This information can be estimated using modern communication and navigation systems, once the driver informs the system about the desired destination. Combining this information with the available mathematical models of the vehicle and the propulsion system and utilizing fast numerical optimization algorithms will permit to ﬁnd feedback control algorithms that are able to adapt to varying traﬃc situations.

References 1. Davison D E, Cornelius S J, Collings N, Glover K (2000) Observations of Transstoichiometric AFR spikes in UEGO Sensors. SAE International Fall Fuels and Lubricants Meeting and Exposition 2. Genc U, Ford F, Glover K, Collings N (2002) Experimental Investigation of Changing Fuel Path Dynamics in Twin-Independent Variable Camshaft Timing Engines. SAE Powertrain and Fluid Systems Conference

182

L. Guzzella and C. Onder

3. Jung M, Glover K (2003) Control-Oriented Linear Parameter Varying Modelling of a Turbocharged Diesel Engine. Proceedings of the IEEE Conference on Control Applications 4. Jung M, Ford R, Glover K, Collings N, Christen U, Watts M (2002) Parameterization and Transient Validation of a Variable Geometry Turbocharger for Mean-Value Modeling at Low and Medium Speed-Load Points. SAE Powertrain and Fluid Systems Conference 5. Jung M, Glover K, Christen U (2005) Comparison of Uncertainty Parameterisations for H-inﬁnity Robust Control of Turbocharged Diesel Engines. Control Engineering Practice, 13:15–25 6. Jung M (2003) Mean-Value Modelling and Robust Control of the Airpath of a Turbocharged Diesel Engine. University of Cambridge PhD Thesis 7. Ford R, Glover K (2000) Spark Ignition Engine Idle Speed Control Utilising a Novel Framework and Facilitating Control of the Tradeoﬀ Between Fuel Consumption and Disturbance Rejection Capability. AVEC International Symposium on Advanced Vehicle Control 8. Ford R, Glover K (2000) An application of coprime factor based anti-windup and bumpless transfer control to the spark ignition engine idle speed control problem. IEEE Conference on Decision and Control 9. Ford R (2000) Robust Automotive Idle Speed Control in a Novel Framework. University of Cambridge PhD Thesis 10. Christen U, Vantine K J, Glover K, Collings N (2001) Event-based MeanValue Modelling of DI Diesel Engines for Controller Design. SAE International Congress, Detroit 11. Vantine K J, Christen U, Glover K, Collings N (2001) Analysis of an EventBased Diesel Engine Model for Control Purposes. 3rd IFAC Workshop on Advances in Automotive Control 12. Davison D E, Cornelius S J, Glover K (2000) A Control Scheme to Produce a Catalyst Feed-gas with Arbitrary Periodic Air-to-fuel Ratio. ASME International Congress and Exposition Symposium on Advanced Automotive Technologies 13. Cornelius, S J, Collings N, Glover K (2000) The Role of Oxygen Storage in NO Conversion in Automotive Catalysts. CAPoC5, Brussels 14. Brandstetter M (1996) Robust Air-Fuel Ratio Control for Combustion Engines. University of Cambridge PhD Thesis 15. Genc U, Glover K, Ford R (2001) Nonlinear Control of Hydraulic Actuators in Variable Cam Timing Engines. International Workshop on Modeling, Emissions and Control in Automotive Engines, MECA’01, Salerno 16. Genc U (2003) Linear Parameter-Varying Modelling and Robust Control of Variable Cam Timing Engines. University of Cambridge PhD Thesis 17. Bryson E, Ho YC (1975) Applied Optimal Control. Taylor & Francis, New York, NY 18. Guzzella L, Onder C (2004) Introduction to Modeling and Control of IC Engine Systems. Springer, Berlin Heidelberg New York 19. Guzzella L, Sciarretta A (2005) Vehicle Propulsion Systems — Introduction to Modeling and Optimization. Springer, Berlin Heidelberg New York

Review of Multivariable Control Applied to the VAAC Harrier and Remaining Open Questions Rick A. Hyde Department of Mechanical Engineering, University of Bristol, University Walk, Bristol BS8 1TR, U.K. [email protected]

1 Introduction In the early nineties, a multivariable ﬂight control law, designated FCL005, was developed for the Defence Evaluation and Research Agency (DERA), now QinetiQ, Vectored thrust Aircraft Advanced ﬂight Control (VAAC) research Harrier XW175, and has been previously reported [1–3]. It was developed within the Cambridge University Engineering Department (CUED) Control Group, and much of its success depended on parallel theoretical developments within the group. The design itself used H∞ loop-shaping [4], with increased understanding of how to select weights being deduced from the application. Gain scheduling was initially done using switching between controllers, but in piloted simulation the switching points were evident, even when using bumpless transfer techniques. At around this time it was shown that the H∞ loop-shaping controller could be written in observer form (see for example [5]), providing an improved scheduling method. When it came to implementation of the control law on the aircraft, the lack of processing power necessitated discrete time implementation at a relatively slow rate compared to the bandwidth. To support this, the discrete time solution for the optimal H∞ controller in observer form developed in [6] was used. Later research within the CUED Control Group looked at model (in)validation [7] using the Harrier as a vehicle for testing its practicality. The H∞ loop-shaping controller developed for the VAAC Harrier is but one of a number of control laws studied and sponsored, at that time, by DERA [2] under the MOD Applied Research Programme. The primary VAAC programme aim was to look at requirements for the Future Carrier-Borne Aircraft (FCBA) that would ultimately replace the Sea Harrier. A key research objective was to reduce pilot workload, one way of achieving this being to move from a three-inceptor (throttle, nozzle, stick) to a two-inceptor (for example, airspeed and ﬂight path demands) control system. Balancing powered and aerodynamic lift (the third degree of freedom) in semi-wingborne ﬂight

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 183–202, 2006. © Springer-Verlag Berlin Heidelberg 2006

184

R.A. Hyde

Fig. 1. The VAAC Harrier - Crown Copyright, reproduced with permission

presents a challenging design problem, particularly as there is strong cross coupling between vectored thrust and aerodynamic forces. Given the successful implementation and testing of FCL005 and many other applications of H∞ including the Lynx rotorcraft [8] and the Bell 205 helicopter [9], it might reasonably be concluded that multivariable control design is now a mature technology. This paper explores this based on experience of the VAAC Harrier multivariable control law. Section 2 sets the scene by providing a short history of the development of FCL005 and highlighting challenges faced during its design and implementation. Open issues are then highlighted and developed in Sections 4, 5 and 6. In addition to this, Section 3 presents previously unpublished ﬂight test results and applies an analysis method proposed and tested in [7] that provides a measure, in the paradigm of normalized coprime factor uncertainty, of whether the model and its assumed uncertainty can explain the ﬂight test data. Conclusions are then drawn in Section 7.

2 Review of Control Law 005 2.1 Control Law Development History Table 1 shows the development of the ﬂight control law from its inception in 1988 through to latest test ﬂights in 2001. Changes made to the control law were mainly to improve the transition between the wing-borne and powered lift regimes, design in other ﬂight regimes being relatively straightforward. The main challenge at transition speeds is the large pitching moment induced as the nozzles swing down to increase powered lift. On approach, these large pitching moments cause rapid changes in angle-of-attack and hence aerodynamic lift thereby adversely aﬀecting the ﬂight path control. A second challenge is to manage the ratio of powered-lift to aerodynamic lift in a way that is both eﬃcient and gives good pilot handling qualities. The most mature and successful solution has previously been outlined in [10]. The control law has always exhibited good robustness with no signs of the onset of instabilities at any speed. This may, in part, be due to the ﬁdelity of the aircraft model used, but could as equally well be testament to

Review of Multivariable Control Applied to the VAAC Harrier

185

Table 1. Development history of FCL005 Period

Activities

1988 – 1991

Application of loop-shaping & selection of weighting functions. Scheduling of controller gains by switching and interpolation based on observer form. Piloted simulation on DERA Large Motion Simulator. Re-design for implementation on DERA VAAC Harrier. Improved implementation based on hybrid measurements of form [stabilization variable + λ× hold variable]. Direct discrete time design using discrete solution of H∞ controller. Successful ﬁrst ﬂight trial. Review of additional challenges posed by multivariable controllers when preparing a clearance case. Fidelity of model reviewed against ﬂight data making use of the method of [7] and conventional methods. Identiﬁcation of nozzle-tailplane interference eﬀects using non-linear identiﬁcation tools. Re-design of controller at transition speeds to support further ﬂight tests and understanding of model uncertainty.

1992 – 1993

1995 – 1996 1999 – 2001

the appropriateness of the coprime factor uncertainty measure to capture diﬀerences between aircraft and model. In order to try and gauge the level of stability achieved, post-ﬂight stability analysis was carried out by Davis and Glover [7, 11]. Only limited testing was possible at this time due to aircraft availability, but the feasibility of the method was demonstrated. Here, in 1999, the approach was applied again to the transition regime, and then in 2000 the idea of calculating bounds on achieved stability in real-time during ﬂight explored. This work has not been previously reported, and is presented in Section 3. The control law has never been extensively reﬁned and tested in order to optimize handling qualities, the main interest in FCL005 for the VAAC programme being to understand the merits and practicalities of using H∞ feedback controllers. It can be reasonably assumed that if the feedback controller achieves a robust and well decoupled closed-loop, then pilot command conditioning can be implemented in much the same way as for the other control laws studied. 2.2 Open Questions The VAAC Harrier has provided an ideal test bed on which to evaluate the practicalities of the H∞ loop-shaping method. Issues addressed and answered include [3] selecting inputs and outputs, scaling, weights selection (based largely on the earlier work of McFarlane and Glover [4, 12], gain scheduling and anti-windup. In parallel with this, there have been many other similar

186

R.A. Hyde

studies using loop-shaping for airframe control, most notably the work of Walker and Postlethwaite [8] on the Westland Lynx helicopter model and in piloted ﬂight simulation. The selection of weights has also been separately studied, with an investigation of the use of non-diagonal weights [13, 14]. The net result is that, for multivariable systems, the control designer has access to a very comprehensive toolbox. However, there remain some unanswered questions that are now described. What stability requirement should the controller be designed to? For aircraft, Nichols exclusion regions [15] are often used to deﬁne uncertainty on the primary surfaces to which the closed-loop must remain stable. It will be shown in Section 4.3 that generalisation of this approach to multivariable systems and controllers can be problematic, even if the use of H∞ based uncertainty does provide some answers. How signiﬁcant is the choice of feedback architecture and how should it be chosen? For the earlier FCL005 designs [3], the H∞ controllers were three-input threeoutput and with a range of open-loop cross-over frequencies at each of the three primary actuators. With experience, it was found that much better closed-loop dynamics could be achieved with an inner-loop pitch rate controller plus outer-loop 3-input 3-output multivariable controller [10]. The implications of architecture choice are considered in more detail in Sections 5.1 and 5.2. Is it sensible to invert plant dynamics in the weights, and if so, under what conditions? FCL005 only uses diagonal weights on the assumption made at the time that inverting system dynamics can result in poor stability margins. Subsequently, in [14], it was shown that if non-diagonal weights are used, their condition number should be small. In [16] results from designing Non-linear Dynamic Inversion (NDI) and H∞ loop-shaping autopilots for a highly cross-coupled missile suggest (but do not prove) that better robustness but similar performance is achieved when not directly cancelling plant cross-terms. The implications of non-diagonal weights in the context of H∞ loop-shaping is considered further in Section 5.3. Is the H∞ controller unnecessarily complex? The H∞ controllers ﬂown on the VAAC Harrier have 21 states following model reduction. This looks high compared to a conventional controller consisting of single-input single-output feedback loops, where the number of states might be typically two or three per feedback path. This is compounded when it is considered how many state-space matrix entries must be gain scheduled with ﬂight condition. This is explored further in Section 6.

Review of Multivariable Control Applied to the VAAC Harrier

187

3 Estimation of Achieved Stability Margins 3.1 Method Conventionally stability margins are estimated from ﬂight data by performing stick doublets, and then estimating the loop gain and phase by means of an FFT of the response. An alternative is proposed in [11] that can handle multi-input multi-output systems, and is based on normalized coprime factor uncertainty as is used for the FCL005 design. Consider the block diagram ˜ and M ˜ represent the normalised left coprime shown in Figure 2. Here N factorisation of the linearised plant model G for a given ﬂight condition i.e. ˜ . The blocks Δ ˜ and Δ ˜ represent the uncertainty on the nominal ˜ −1 N G=M N M coprime factors. In other words, the assumption is that the true plant is given by: ˜ + Δ ˜ )−1 (N ˜ + Δ ˜) Gtrue = (M M N where

ΔN˜ ΔM˜

∞

≤ γ.

-

Δ N~

u

~ N

Δ M~

~ M −1

y

n

Fig. 2. Left coprime factor model of plant uncertainty

The analysis approach proposed in [11] introduces an additional noise or exogenous disturbance input n shown in Figure 2 and poses the question: given an assumed maximum amplitude for each element of the vector n, how large does γ need to be to explain the measured output y in response to the measured input u? The linear model can be either treated as Linear Time Invariant (LTI) or Time Varying (TV). These two cases impose diﬀerent restrictions on the class of ΔN˜ and ΔM˜ , and result in a diﬀerent value of γ required to accommodate the ﬂight test data. The TV case allows a larger class of uncertainty, and hence results in a smaller value of γ. The notation introduced in [11] for the two values of γ are γmin (T I) and γmin (T V ) respectively. A straightforward way of calculating γmin (T V ) is given in Lemma 5.2 of [7], and is as follows. Let plant input and output sampled data be denoted T T by sampled sequences u0 u1 . . . . . . and y0 y1 . . . . . . . Then deﬁne w and b as:

188

R.A. Hyde T

w = u 0 y0 u 1 y1 . . . . . .     u0 y0  u1   y1      .  .  b = TM˜   ..  − TN˜  ..  = TM˜ y¯ − TN˜ u¯     .. .. . . where TN˜ and TM˜ are the Toeplitz matrices mapping discrete-time system ˜ and M ˜ . Then the model-set is validated if and only inputs to outputs for N if there exists a real vector n ∈ Rny , with n 2 ≤ δ, satisfying πk (b − n)

2

≤ γ π2k w

2

where • 2 denotes the standard Euclidean norm and πk denotes the k-step truncation operator: πk : u0 u1 . . . → u0 u1 . . . uk−1 0 . . . . The minimum value of γ required to validate the model set can calculated by minimising the left-hand side of this inequality. Given an inﬁnity norm bound of the form n ∞ < δ, the smallest value of γ is achieved by setting:   bi , if |bi | < δ ni = δ, if bi ≥ δ  −δ, if bi ≤ −δ Calculation of γmin (T I) involves ﬁnding the minimum of a linear function subject to Linear Matrix Inequality (LMI) constraints. This quickly becomes impractical for relatively short lengths of ﬂight data. An upper bound on γmin (T I) is proposed in [7], denoted γub (T I). Its calculation involves solving a positive deﬁnite quadratic programming problem. Even this is computationally intensive, and it was found that for the three-input three-output system considered here that at most 15 seconds of ﬂight data could be handled when sampling at 25Hz. 3.2 Application In order to gain conﬁdence that the aircraft dynamic response can be explained by the model and assumed uncertainty set, it is important to exercise all dynamics and signal paths through the system. To achieve this, swept sine waves were applied during ﬂight to each of the shaped plant’s inputs (d1 of dimension 3) and outputs (d2 of dimension 3) in turn and u and y recorded. Each sweep took 15 seconds, giving a total data capture time of 90 seconds. All data was taken at approximately 130 knots airspeed and 2000 feet. At this ﬂight condition good modelling and/or stability margins are crucial as the aircraft is partly jet-borne, and powerplant-airframe interactions are very signiﬁcant. The overall amplitude of the disturbances was set with the aid of the nonlinear aircraft model. Too low a disturbance will be insigniﬁcant compared to

Review of Multivariable Control Applied to the VAAC Harrier

189

background noise and turbulence, and too large a disturbance risks plant nonlinearity driving up the value of γ required to explain results. Following [7], each sweep has the form: (

3.8 ωmax − ωmin 4 t + 0.2) × sin(ωmin t + t ) 15 13, 500

where 0 < t < 15, ωmin = 0.2 rad/s and ωmax = 15 rad/s. The eﬀect of the gain term is to increase the amplitude of the disturbance at higher frequencies in order to improve signal-to-noise ratio. This term was designed by trial and error on the non-linear model. The gain at lower frequencies was inﬂuenced mainly by the aircraft incidence limit and engine max rpm limit. 3.3 Results Results presented here are for VAAC ﬂight test F1593 on 12th October 1999. The data was taken under good meteorological conditions resulting in a reasonably good internally-generated disturbance to external disturbance ratio. Ultimately this should result in a tighter bound on the stability margin calculation than if the tests were carried out in turbulent air. To determine bounds on the exogenous turbulence and noise (n), the measurements of u and y for the 10 seconds prior to the start of the swept sine −3

10

x 10

n1 n2

8

n

3

Noise (normalized by [N M])

6 4 2 0 −2 −4 −6 −8 0

1

2

3

4

5 6 Time (seconds)

7

Fig. 3. Equivalent noise levels

8

9

10

190

R.A. Hyde

˜ M ˜ . This gives an equivalent noise vector waves were passed as inputs to N deﬁned by: u n = TN˜ TM˜ y Figure 3 shows a plot of n. From this ﬁgure the L1 noise bound is taken to T T be −0.007, −0.010, −0.005 < n < 0.007, 0.010, 0.005 . The variation on this signal is at a relatively low frequency, suggesting that its primary source is low-level turbulence. Figures 4 and 5 show part of the ﬂight data collected. Constant oﬀsets are removed from the raw data using the MATLAB detrend.m command. In total six frequency sweeps are applied, each taking 15 seconds. Given the sample time of 0.02 seconds this gives a total of 4500 points. The calculated value of γmin (T V ) for this data set is 0.0664. What the result precisely says is, that given the assumed turbulence/noise values, the bound on the time-varying coprime factor uncertainty required to explain the diﬀerences between the model and the aircraft is γmin (T V )=0.0664. Of course, in practice, the timeinvariant uncertainty required to explain the data could be higher than this. Also, during the test, higher levels of turbulence than measured during the pre-test ten seconds of ﬂight will act to give a misleadingly high value of γmin (T V ). The value of γmin (T V ) should be compared with the bound on the smallest normalized coprime factor uncertainty [4] that will destabilize the nominal shaped plant and controller K. Loop shapes were chosen in order to ensure that was at least 0.4. The value of γmin (T V ) is small in comparison to this, suggesting more than adequate robustness of the design. Additional insight can be obtained from calculating γmin (T V ) for a subset of the data. Calculating γmin (T V ) for the ﬁrst 15 seconds of data only gives γmin (T V )=0.013, perhaps indicating that the modelling in pitch is better than in other loops given that the ﬁrst sweep is on the tailplane demand. 3.4 Using Pilot Commands The use of swept sine wave generators integrated into the control law gives control over the frequency content and the paths excited. This gives more conﬁdence in the results than if an unspeciﬁc excitation signal, such as general pilot-invoked manoeuvring, is used. However, pilot manoeuvring could be used to monitor aircraft stability margins in-service as part of a fault-detection system. This idea was explored by calculating γmin (T V ) for speciﬁc manoeuvres. Figure 6 shows a ﬂight path change from 0 to +5 degrees at around 130 knots. For this set of ﬂight data, the calculated value of γmin (T V ) is 0.0837, which again is low compared to the linear-time invariant nominal robustness value of ε =0.40. The diﬃculty with applying this method in general is that the analysis, as presented, must start from steady initial conditions. Without steady initial

Review of Multivariable Control Applied to the VAAC Harrier 0.04 0.03 0.02

Longitudinal thrust Normal thrust Tailplane

Umeasured

0.01 0 −0.01 −0.02 −0.03 −0.04 −0.05 0

100

200

300 400 sample number

500

600

700

Fig. 4. Flight F1593 plant input data (ﬁrst 15 seconds only)

0.04 Forward motion 0.03

Normal motion Pitching motion

Ymeasured

0.02

0.01

0

−0.01

−0.02

−0.03 0

100

200

300 400 sample number

500

600

700

Fig. 5. Flight F1593 plant output data (ﬁrst 15 seconds only)

191

192

R.A. Hyde

conditions, the transients due to the initial state will look like an external disturbance. In order to address this, predicted initial condition transients derived from the nominal plant model and controller can be subtracted from the aircraft dynamic response. The initial plant state x0G and controller state x0K are calculated by simulating the following two equations for a few seconds of ﬂight prior to the start of the analysis: x0G = [ TNG I − TMG ] x0K = [ TN I − TM ]

um ym

ym um

Here [N M ] is a normalized left coprime factorization of the controller K∞ , [NG MG ] is a normalized left coprime factorization of the nominal shaped plant model W2 GW1 and u¯m and y¯m are the sequences of measured aircraft inputs and outputs. With this architecture γmin (T V ) can be calculated in real-time during ﬂight, giving the pilot or fault monitoring system a continuous readout of how the aircraft is behaving in comparison to a set of linearized models. To date this has been tested only during piloted simulation.

6 5 4

Response (see key)

3 2 1 0 −1 Flight path hold flag

−2

Flight path (degrees) −3 −4 1460

Stick position (inches) 1461

1462

1463

1464 1465 1466 Time (seconds)

1467

1468

1469

Fig. 6. Flight path change from 0 to +5 degrees

1470

Review of Multivariable Control Applied to the VAAC Harrier

193

4 Retrospective Stability Analysis 4.1 Measuring Stability This section aims to review the stability properties of FCL005 in the light of recent theoretical developments. But before doing this, it is worth exploring how the manufacturers assess the stability of military aircraft. One of the challenges for the early work on FCL005 was to relate the chosen target normalised coprime factor robustness level of > 0.4 to conventional measures such as gain and phase margins or Nichols exclusion regions. Only with this would the method ﬁnd wider application. The dashed line in Figure 7 shows a Nichols exclusion region as used by the manufacturers [15]. It deﬁnes a range of gain and phase oﬀsets which, when applied to each of the actuators simultaneously, should not result in instability. This region is made larger for the case where it is applied only to one actuator at a time. Note that this multi-loop stability test is only applied at the actuators and not at the plant outputs. The original design target of > 0.4 was deﬁned based on experience of a number of design examples, and then later supported by the bounds on gain and phase margin derived in [17], namely that gain margin > (1 + ) / (1 − ) and phase margin > 2 sin−1 . A direct relationship between and equivalent Nichols exclusion regions was derived much later in [18], but the underlying approach of approximating the exclusion region as an ellipse in the 10

(1+δ)/(1−δ), |δ|<0.2 (1+δ)/(1−δ), |δ|<0.4 Conventional offset

8 6

Magnitude (dB)

4 2 0 −2 −4 −6 −8 −10 −60

−40

−20

0

Phase (degrees)

20

40

Fig. 7. Example Nichols exclusion region boundaries

60

194

R.A. Hyde

context of a µ-analysis test was developed in a collaborative project between Cambridge University Engineering Department, Cambridge Control Ltd and British Aerospace Military Aircraft Division [19] in 1990. The main result in [18] is that achieving a normalised coprime factor robustness value of is equivalent to showing that the closed-loop will remain stable if uncertainty is applied to all inputs and outputs of the weighted plant W2 GW1 , each uncertainty being bounded by the region in the Nichols plane deﬁned by: r=

1+δ 1−δ

where δ is a complex number satisfying |δ| < ε/2. Figure 8 shows two example regions deﬁned by |δ| < 0.2 and |δ| < 0.4. The FCL005 designs are now reviewed in terms of these boundaries. 4.2 Review of FCL005 Stability The FCL005 controller architecture will be deﬁned in more detail in Section 5. In this section only the outer three-input three-output compensator is considered, the airframe plus inner pitch rate loop being considered to be the plant. Results presented here are for the controller designed on a model linearization at 137 knots airspeed (hereafter the 137 knot controller). At this speed, the aircraft is partly wing-borne and partly jet-borne and there is very signiﬁcant cross-coupling. The loop-shapes were chosen so as to give a relatively large nominal stability margin, namely = 0.4. For a SISO system, the bounds in [17] would guarantee a phase margin of least 47 degrees and a gain margin of at least 7.4dB. The more recent result of [18] gives a better understanding of what = 0.4 means as follows: this closed-loop is guaranteed to remain stable for simultaneous uncertainties applied to all sensors and actuators that are of the form r = (1 + δ) / (1 − δ) where |δ| < 0.2 i.e. the smaller elliptical region deﬁned in Figure 7. If either the controller or the shaped plant had no cross-coupling, then it could also be deduced that the system would remain stable if all of the uncertainty were placed at the inputs i.e. an uncertainty would be applied to each of the three actuators with the form r = (1 + δ) / (1 − δ) where |δ| < 0.4. As both the plant and controller do have cross-coupling, robustness to uncertainties applied to just the inputs or just the outputs may diﬀer from this. Table 3 shows the results for this controller, the norm on the δ having been determined by setting up an equivalent µ-analysis problem. Notice that robustness to uncertainties at the inputs is greater than that at the outputs. Hence if stability analysis were based just on a test at the inputs, the results could be optimistic. The lower robustness level at the plant outputs is also reﬂected by the phase margin for the third loop (which corresponds to the pitching motion). The coprime factor robustness, in this case, may be a more reliable measure if a single measure of stability is required. It may also more realistic to deﬁne uncertainty at both the inputs and the outputs rather

Review of Multivariable Control Applied to the VAAC Harrier

195

Table 2. Stability results for the 137 knot controller Uncertainty location/type

Limiting value for stability

r = (1 + δ) / (1 − δ) applied to all three actuators and all three sensors r = (1 + δ) / (1 − δ) applied to all three actuators r = (1 + δ) / (1 − δ) applied to all three sensors Gain margin (dB) at the three actuators Phase margins (degrees) at the three actuators Gain margin (dB) at the three sensors Phase margins (degrees) at the three sensors

|δ| < 0.2 |δ| < 0.44 |δ| < 0.41 [18.9 14.3 [71.5 70.5 [20.4 17.2 [72.4 84.2

10.7] 74.5] 10.6] 59.0]

than just the inputs, the reasons for which are explored further in the next section. 4.3 Open Questions on Stability Requirements Analysis of FCL005 shows that there is still an open question about how to assess the stability of multivariable systems. In the case of previous military airframe analysis, the practice of analysing stability only at the actuators was suﬃcient because of relatively low cross-coupling in the airframe and/or controller. In this case, testing at a given actuator (say total pitching moment demand) is equivalent to testing at the plant measurement (pitch rate). Figure 8 shows what can happen if there is large cross-coupling. Here a two-input two-output shaped plant W2 GW1 is denoted by its input-output pairs g11 , g12 , g21 , g22 , and the controller is assumed diagonal. As the controller K is diagonal, applying uncertainty at the plant inputs as shown is equivalent to applying them at the outputs. However, if the plant

g11

g12

g 21

g 22

Δ1

d e

Δ2

k11 k 22

Fig. 8. Uncertainty deﬁned at both plant inputs

196

R.A. Hyde

cross-terms g12 and g21 are large, then the uncertainty as deﬁned eﬀectively gets applied twice as the path from d to e is k11 g12 Δ2 k22 g21 Δ1 . Hence mandating the multi-loop exclusion region of the size shown in Figure 7 might unnecessarily limit design freedom. If the controller also has cross-terms, then robustness is no longer the same at the inputs and outputs. In this context, the normalized coprime factor robustness measure looks like a good compromise in that uncertainties are deﬁned at all actuators and sensors (with the proviso that diagonal weights are used for the loop-shaping exercise). In practice, further reﬁnement of where to apply the uncertainties comes down to how the uncertainty arises in the ﬁrst place. For example, errors in the lift coeﬃcient for a control surface is best reﬂected as an input uncertainty, whereas uncertainty on the main body lift coeﬃcient might be better represented as an uncertainty on vertical acceleration (a plant output). In conclusion, the normalized coprime factor uncertainty measure is a good candidate for assessing stability of cross-coupled plants and controllers, it being more reliable than single-loop analysis or simultaneous uncertainties applied to inputs only. It remains for the airframe manufacturers to adopt this kind of measure, or to reﬁne it if more information is available as to the expected relative levels of uncertainty at inputs (control surfaces) and outputs (body lift and moment terms).

5 Controller Architecture 5.1 FCL005 Feedback Architecture Figure 10 shows the feedback architecture used for the 137 knot controller. Note that how the pilot commands enter the loop is not shown, this being quite complex. The symbols used for plant outputs are as follows: α is angle of attack (degrees), q is pitch rate (degrees/s), θ is pitch attitude (degrees), and x, ˙ z, ˙ x ¨, z¨ are body axes velocities and accelerations. Plant inputs are demand fraction of maximum thrust component along the fuselage (Tx ), demanded fraction of maximum thrust normal to the fuselage (Tz ) and tailplane angle demand in degrees (η). Note that achieved resolved thrusts lag behind the demanded due to nozzle actuation and engine dynamics. The plant input weight is given by the product of a scaling matrix and a set of integrators:     3 2/s  3/s W1 =  3  ×  (1) 10 3/s Plant outputs are all of the form stabilization variable + λ× hold variable. If the hold variable in each case is an approximate integral of the stabilization variable, then the result is that loop shape around cross-over is determined by the stabilization variable, and the low-frequency loop shape is determined

Review of Multivariable Control Applied to the VAAC Harrier

K∞

197

W2

TZ

W1 (s)

Wq

ηd

..

.

..

.

x + λx x

TX

z + λz z G

α q + λθθ

Fig. 9. Feedback architecture for transition speeds

by the hold variable. In the case of the pitch loop, this stabilizes the airframe with the higher integrity gyro measurement, and gives a long term attitude hold. The outputs are scaled by the matrix W2 deﬁned by:   0.2 0.2  W2 =  (2) 1 The inner-loop feedback controller Wq is purely a proportional term, and is designed to have an open-loop cross-over of 5 radians/s. The outer loop cross-over frequencies are 1, 1.5 and 2.5 for forward acceleration, vertical acceleration and angle-of-attack respectively. Earlier versions of the control law did not have an inner feedback loop, the belief being, that for robustness reasons, the design was better carried out in a single multivariable step. However, these earlier designs were unable to achieve the necessary level of performance, and exhibited an undesirable time response attribute referred to at the time as slow modes. A step on one channel (for example pitch or angle of attack) would result in small but signiﬁcant long-term cross coupling into other channels e.g. normal acceleration. The origin of these slow modes is due to mixing slow and fast loops with a single multivariable controller. Low bandwidth feedback paths via controller and plant oﬀ-diagonal terms can result and exhibit themselves as the slow modes in the time response. This leaves the question of how closing an inner loop might aﬀect robustness, and this is now explored. 5.2 Impact of Inner Loop on Uncertainty To explore the eﬀect of the inner pitch rate feedback loop, ten aerodynamic and propulsion system parametric uncertanities of +-20% were deﬁned on the

198

R.A. Hyde

nominal state-space plant model. All possible combinations of the 10 uncertainities, when taking their extreme values, were used to build up the Nichols plots shown in Figure 10. In this ﬁgure, the nine subplots corresponding to the ordered inputs [TX , TZ , η] and outputs [¨ x + λx x, ˙ z¨ + λz z, ˙ and α]. It can be seen that the eﬀect of closing the pitch rate inner loop is to collapse the size of the uncerainty set, particularly that associated with the thrust inputs i.e. columns one and two. In addition, the control of angle-of-attack is made easier through changes of the phase of the [3,3] element.

Fig. 10. Eﬀect of closing the inner feedback loop

Figure 10 shows that assuming equal amounts of uncertainty on all inputs and outputs could result in a sub-optimal design, there being more uncertainty associated with the [2,1], [2,2], [3,1] and [3,2] elements than the others. However, the inner loop reduces this potential conservatism signiﬁcantly. 5.3 Non-diagonal Weights Earlier versions of FCL005 made use of a non-diagonal W1 as this was found to help align singular values and thereby achieve similar time responses in diﬀerent loops. However, it was thought at the time that non-diagonal weights might be poor for robustness, and this was later analysed by Papageorgiou [13] who showed that the weight should have low condition number. The Nichols-based interpretation of robustness to normalized coprime factor uncertainty in [18] also sheds some light on this. If the weights are diagonal, then the uncertainties deﬁned on the shaped plant are equivalent to

Review of Multivariable Control Applied to the VAAC Harrier

199

deﬁning uncertainties on the unshaped plant. The issue of whether uncertainties should be applied to the shaped or unshaped plant was debated during early application of H∞ loop-shaping. The equivalent Nichols test suggests that: • • •

There is equivalence if diagonal weights are used. Results depend on how the plant is to be used i.e. the speciﬁc loop-shapes speciﬁed by the weighting functions. Using non-diagonal weights skews the uncertainty set, even if the condition number of the weights is one.

This type if insight is also relevant to the widely studied method of non-linear dynamic inversion (NDI). NDI eﬀectively acts like a non-diagonal controller, inverting plant cross terms, which can be bad for robustness. 5.4 Summary Once an architecture is ﬁxed, it’s analysis in terms of stability and performance can be eﬃciently tested using Nichols or H∞ norm based tests. What is missing is tools and supporting theory to support the architecture selection in the ﬁrst place. This comment is relevant to the optimality or not of NDI design in that the designer has little inﬂuence over the controller architecture which by deﬁnition inverts the nominal model. In practice, one of the most import aspects of controller design for multivariable systems, is primarily down to the designer’s experience or intuition.

6 Complexity 6.1 Implementation of FCL005 The transition controller K∞ has 21 states. This high order is a common criticism of H∞ controllers, the complexity being signiﬁcantly higher than for conventional controllers. However, when it is considered that these states are shared by nine input-output pairs equating to just over two states each, the complexity does not look so high, provided that the cross-terms are beneﬁcial to stability and/or performance. A measure of complexity that is more relevant is not the number of states, but the number of ﬂoating point operations required every time step. Table 3 summarizes the number of parameters and ﬂoating point operations for the transition controller. If A + HC is calculated a priori, then the total number of ﬂoating operations reduces slightly. Note that the table does not include the additional computation required when interpolating state-space entries between design points.

200

R.A. Hyde Table 3. K∞ compensator metadata Matrix

Dimensions Parameters

Additions

Multiplications

A B C F H Totals

21x21 21x3 3x21 3x21 21x3

410 42 60 60 42 614

21 63 63 63 63 273

441 63 63 63 63 693

This complexity should be compared with a conventional controller consisting of three SISO terms, one for each feedback loop. It is easy to show that an nth order transfer function is deﬁned by 2n + 1 parameters, and requires 3n − 1 additions and 3n multiplications. Table 4 summarizes additions and multiplications for 1st , 2nd and 3rd order transfer functions. So, for example, three 3rd order SISO compensators requires only 21 parameters, 21 additions and 27 multiplications. Table 4. Metadata for SISO transfer functions Order

Parameters

Additions

Multiplications

1 2 3

3 5 7

1 4 7

3 6 9

6.2 Implementation in SISO Form The analysis in Section 6.1 tends to suggest that it may be more eﬃcient to implement a multivariable controller as a set of SISO transfer functions rather than as a single state-space system. To investigate this possibility, each of the nine elements of the 137 knot H∞ controller design was model matched to a low order transfer function using least squares minimisation to match the gain and phase. The orders required for a good match are (row-wise): [3, 2, 2, 5, 3, 3, 6, 6, 2]. The achieved normalized coprime factor robustness with this approximated controller is 0.39 compared to 0.40 for the original 21 state multi-input multi-output (MIMO) state-space controller i.e. there is no signiﬁcant loss in robustness. This SISO implementation requires 73 parameters, 78 additions and 96 multiplications, signiﬁcantly less than for the MIMO implementation. In conclusion, minimum complexity implementation remains an open problem. From an implementation point of view, methods such as non-linear dynamic inversion look more appealing, but lack the stability guarantees of H∞ -

Review of Multivariable Control Applied to the VAAC Harrier

201

based methods. Using the normalized coprime factor robustness measure to steer an NDI design (inversion of the controls, selection of closed-loop bandwidth) is an attractive compromise, and partly explored in [20].

7 Conclusions The H∞ design method has proved itself to be a robust and easy-to-use design method for multivariable systems. The VAAC Harrier provided an ideal test bed for the method, resulting in insights in how best to choose loop shapes and also how to implement the controller. However, some questions on how best to apply the method still remain open. First and foremost is the question of controller architecture, and this extends to other methods, and in particular NDI. Despite several studies on successive loop-closing and the impact of non-diagonal weights, there is still further merit in developing a more rigorous framework within which to select an optimal feedback architecture. The question of controller complexity is also still open, with no current closed-form methods of designing optimal controllers to speciﬁed maximum level of complexity. The question of what robustness measure should be used for a multivariable system may be largely solved from the control theoretician’s point of view, but this needs to be accepted by aircraft manufacturers. Furthermore, it has been shown for the VAAC Harrier that the industry approach of testing simultaneous uncertainties just at the actuators may give optimistic results. The implications of having strong cross-coupling in both the plant and controller need to be more widely understood. The methods applied to look at diﬀerences between the design models and the ﬂight test data may have signiﬁcant potential for in-ﬂight performance monitoring, possibly as part of a fault detection system. It could also be used as a tool for expanding the envelope of developmental aircraft. In the case of the VAAC Harrier, the results give some conﬁdence that the uncertainty measure used was appropriate to capture diﬀerences between the design model and the aircraft. Acknowledgements This work would not have been possible without the support and guidance of Professor Keith Glover at Cambridge University Engineering Department, both in the earlier work when he was my PhD supervisor, and in the more recent ﬂight testing reported here where he provided advice on the re-application of his model validation work with Rob Davis. The later parts of the work described in this paper were funded by the UK Defence Evaluation and Research Agency as part of the VAAC programme, and carried out when I was working at The MathWorks.

202

R.A. Hyde

References 1. R. A. Hyde and K. Glover, “The application of scheduled H∞ controllers to vstol aircraft,” IEEE Transactions on Automatic Control, vol. 38, no. 7, 1993. 2. G. Shanks, C. Fielding, S. J. Andrews, and R. A. Hyde, “Flight control and handling research with the vaac harrier,” International Journal of Control, vol. 59, no. 1, 1994. 3. R. A. Hyde, H∞ Aerospace Control - A VSTOL Flight Application, ser. Advances in Industrial Control Series. Springer-Verlag, 1995. 4. K. Glover and D. McFarlane, “Robust stabilization of normalized coprime factor plant descriptions with H∞ -bounded uncertainty,” IEEE Transactions of Automatic Control, vol. 34, pp. 821–830, 1989. 5. J. Sefton and K. Glover, “Pole/zero cancellations in the general H∞ problem with reference to a two block design,” Systems and Control Letters, vol. 14, p. 295:306, 1990. 6. P. Iglesias, “Robust adaptive control for discrete-time systems,” Ph.D. dissertation, University of Cambridge, 1991. 7. R. C. Davis, “Model validation for robust control,” Ph.D. dissertation, University of Cambridge, January 1995. 8. D. J. Walker and I. Postlethwaite, “Advanced helicopter ﬂight control using two-degree-of-freedom H∞ optimization,” AIAA Journal of Guidance, Control and Dynamics, vol. 19, no. 2, pp. 461–468, 1996. 9. I. Postlethwaite, E. Prempain, E. Turkoglu, and M. C. Turner, “Various H∞ controllers for the Bell 205 helicopter: design and ﬂight test,” in International Federation on Automatic Control World Congress, Barcelona, 2002. 10. R. A. Hyde, “The VAAC harrier design study,” in AIAA Guidance Navigation and Control Conference, Portland, USA, August 1999. 11. R. Davis and K. Glover, “An application of recent model validation techniques to ﬂight test data,” in European Control Conference, 1995. 12. D. McFarlane, “Robust controller design using normalized coprime factor plant descriptions,” phd thesis, University of Cambridge, 1988. 13. G. Papageorgiou, “Robust control system design : H-inﬁnity loop shaping and aerospace applications.” Ph.D. dissertation, University of Cambridge, 1998. 14. G. Papageorgiou and K. Glover, “A systematic procedure for designing nondiagonal weights to facilitate h-inﬁnity loop shaping.” in Proceedings of CDC’97, December 1997. 15. U. Korte, “The industrial process for clearance of ﬂight control laws of ﬁghter aircraft,” GARTEUR, Tech. Rep. TP-119-6, FM(AG11), January 2000. 16. R. A. Hyde and J. R. Cleminson, “Control and maneuverability of a square cross-section missile,” to appear in AIAA Guidance, Dynamics, Control Jnl. 17. G. Vinnicombe, “Measuring the robustness of feedback systems,” Ph.D. dissertation, Churchill College, University of Cambridge, December 1992. 18. K. Glover, G. Vinnicombe, and G. Papageorgiou, “Guaranteed multi-loop stability margins and the gap metric,” in IEEE 39th Conference on Decision and Control, Sydney, December 2000. 19. P. P. Azlin and K. Glover, “µ-analysis technical report,” Cambridge Control Limited, Tech. Rep. P164-TR-401 Version 1, 1990. 20. G. Papageorgiou and R. A. Hyde, “Analysing the stability of ndi-based ﬂight controllers with lpv methods,” in AIAA Guidance Navigation and Control Conference, Montreal, August 2001.

Regulating Cell Shape During Cytokinesis Janet C. Eﬄer1,2 , Pablo A. Iglesias1 , and Douglas N. Robinson2 1 2

Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, {jeffler1,pi}@jhu.edu Department of Cell Biology, The Johns Hopkins School of Medicine, Baltimore, MD 21205, [email protected]

Summary. During cytokinesis, the last step in cell division, cells must rearrange their shape so as to produce two daughter cells of equal size. To this end, the temporal and spatial distribution of a wide variety of proteins must be coordinated. In this paper we review some of the basic steps in the process. Moreover, we argue that a key step in the process is the feedback regulation of motor proteins. In doing so, we show that the study of cellular shape change can beneﬁt from a systems-level approach.

1 Introduction The ability of a cell to change shape is crucial for the proper function of many cellular processes. For example, cells of the immune system migrate in response to pathogen infections by crawling, which involves cycles of extensions and contractions that deform their entire cell surface [1]. Similarly, during the last stage of cell division, known as cytokinesis, cells rearrange themselves so as to produce two identical daughter cells; see Fig. 1. The ability to change cellular shape requires that the distribution of different classes of molecules be regulated both spatially and temporally. How these events are coordinated is understood only in broad terms, but appears to be a tightly regulated process. In this review we outline some of the features of this regulation, focusing on cytokinesis. We argue that cytokinesis is controlled by a mechanical feedback path between the cell shape and the localization of proteins that provide most of the actuation. Understanding the principles of cytokinesis not only sheds light on how cytokinesis is regulated but also has broad implications for cell shape changes in other biological processes such as nuclear organization, gene expression, protein synthesis and cytoskeletal organization [2–4]. Cytokinesis is also essential for human health since the human body regulates ongoing basal levels of cell proliferation, which are required to replenish continuously the blood streams, skin, hair, absorptive epithelia of the gut and numerous other processes. This process is extremely important to humans since at any moment B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 203–224, 2006. © Springer-Verlag Berlin Heidelberg 2006

204

J.C. Eﬄer, P.A. Iglesias, and D.N. Robinson

0

100

162

326

434

10 µm

214

Fig. 1. Time series of a dividing Dictyostelium cell. The cytokinesis process can be separated into three phases; see Section 4.4. The numbers represent seconds after the ﬁrst frame. During the ﬁrst phase the cell elongates into a cylinder (0– 162 s) and the furrow begins to ingress. During Phase 2 the cleavage furrow (white arrow at 162 s) continues to ingress (162–326 s). During Phase 3, the bridge (white arrow at 326 s) elongates leading to eventual daughter cell separation (434 s).

∼ 108 –109 cytokinesis events are underway among the ∼ 1013 eukaryotic cells of the human body.

2 Biological Background Cytokinesis is the mechanical separation of a single mother cell into two daughter cells [5]. During this process, the mother cell rounds up and then elongates. A contractile ring forms at the middle of the cell and constricts until a thin cylindrical bridge connecting the two daughter cells is formed. This cylindrical bridge continues to thin until it severs, resulting in two separate daughter cells. 2.1 Stages of Mitosis Cytokinesis generally begins late in mitosis, the process of nuclear division, and continues until cellular division is achieved; see Fig. 2. Prior to mitosis, chromosomes that contain the cell’s genetic information have been duplicated but are physically connected and contained within a single nucleus. For each

Regulating Cell Shape During Cytokinesis

205

cleavage furrow

Anaphase Telophase Phase 1

Phase 2

bridge dwelling

Cytokinesis Phase 3 mitotic spindle

Metaphase

chromosomes nuclear envelope

Interphase Prometaphase

Prophase

Fig. 2. Steps in the cell cycle. Cells spend most of their life in interphase, where DNA replication takes place. This is followed by nuclear division, known as mitosis. Mitosis consists of ﬁve sequential stages including prophase, prometaphase, metaphase, anaphase, and telophase. During prophase, chromosomes condense and the mitotic spindle starts to form. The mitotic spindle is composed of an array of microtubules, which are rigid hollow rods composed of tubulin subunits. Prometaphase consists of dissolution of the nuclear envelope, the barrier that separates the nucleus from the interior of the cell. As a result of nuclear envelope breakdown, spindle microtubules are able to enter the nuclear region and bind to kinetochores attached to centromeres, which constitute regions where replicated chromosomes are joined. In some organisms, such as Dictyostelium, the nuclear envelope does not break down, but instead, becomes porous. During metaphase, chromosomes become aligned at the metaphase plate, which is an imaginary plane positioned at a right angle between the spindle poles. In anaphase, the chromosomes start to separate due to forces generated by the mitotic spindle. Finally, during telophase, chromosomes reach opposite ends of the cell, the nuclear envelope reforms and chromatin decondenses. Cytokinesis begins prior to anaphase and completes following mitosis. The mitotic spindle disassembles during cytokinesis, after the completion of mitosis. The phases of cytokinesis are explained in Section 4.4.

daughter cell to receive an equivalent set of genetic material, the replicated chromosomes must be physically separated and encapsulated within separate nuclei positioned at opposite ends of the cell prior to the completion of cytokinesis.

206

J.C. Eﬄer, P.A. Iglesias, and D.N. Robinson

3 Key Molecular Players in Cytokinesis Across diﬀerent species, a couple hundred proteins have been identiﬁed as playing a variety of roles during cytokinesis. Here, we will focus only on central players that are known to aﬀect the mechanical aspects of cytokinesis directly [6] with a particular emphasis on the components found in the social amoeba Dictyostelium discoideum. In most organisms, the cytoskeleton is composed of three types of ﬁlaments: actin, microtubules, and intermediate ﬁlaments. Some organisms such as Dictyostelium do not contain cytoplasmic intermediate ﬁlaments. Both actin and microtubules polymerize from individual subunits (monomers) into ﬁlament form. Once in ﬁlament form, these molecules play crucial roles in cytokinesis. The actin ﬁlament network contributes structure and support for the cell and is the major cytoskeletal system that drives cytokinesis [6, 7]. However, the microtubule network, which is primarily responsible for chromosomal separation, delivers cytokinesis regulatory cues to the actin network, and bidirectional communication exists between the two polymer networks [8, 9]. Here, we will primarily discuss the actin-based network of the cell. 3.1 Actin Actin is a globular protein that uses energy from ATP hydrolysis to assemble into semiﬂexible rod-like ﬁlaments [7]. Actin ﬁlaments have the structure of a two-start right handed helix that spans 36 nm for every 13 actin monomers; thus, every new actin monomer extends the ﬁlament a distance of 2.7 nm [10]. By drawing upon the energy from ATP hydrolysis at a given concentration of actin monomers, actin polymerization can generate pN-scale forces while lengthening the actin ﬁlament. Since the γ-phosphate of a single molecule of ATP stores ∼ 100 pN·nm of energy and given the 1–2 pN of force that has been measured for actin polymerization, the eﬃciency of the actin-based force generation is of the order of 3–5% [11]. However, at cellular concentrations of monomeric actin and with associated proteins that may increase the ﬂexural rigidity of the polymer, it is quite possible for the actin machinery to generate signiﬁcantly greater forces during polymerization in vivo. Actin dynamics. Actin ﬁlaments are polymerized from a pool of monomers following a reaction mechanism whose parameters depend on the nucleotide state of the actin monomers. Thus, a number of regulatory mechanisms, and therefore actin-associated proteins, have evolved to govern and regulate the assembly and higher order organization of actin ﬁlaments [12]. Regulatory activities include actin ﬁlament nucleation (Arp2/3, formin), nucleotide exchange (proﬁlin), monomer sequestration (proﬁlin, β-thymosin), actin ﬁlament capping (capping protein), actin ﬁlament stabilization (tropomyosin, dynacortin), actin ﬁlament severing (coﬁlin) and actin ﬁlament crosslinking

Regulating Cell Shape During Cytokinesis

207

(dynacortin, cortexillin, ﬁmbrin). In living cells, many of these proteins have been implicated in cytokinesis in one or more biological systems, ranging from amoebas and yeasts to humans. Actin polymer dynamics are also critically important for furrow ingression [13–18]. Depending on the organism, new actin ﬁlaments can be assembled in the furrow cortex or new ﬁlaments are recruited by ﬂow along the plasma membrane. Coﬁlin, an actin severing protein, is required for actin ﬁlament turnover in the cleavage furrow [18]. In mammalian cells, myosin-II activity also appears to be required for normal removal of actin from the furrow cortex [19]. Myosin-II mechanochemistry may pull actin ﬁlaments from bundles exposing sites for coﬁlin to sever the actin. Thus, actin dynamics are controlled at a variety of levels. 3.2 Myosin-II Central to cytokinesis, myosin-II is responsible for much of the net forces of cytokinesis [5, 20, 21]. Filaments of myosin-II bind to an actin substrate, also in ﬁlament form. Myosins are actin-activated motors that convert the chemical energy stored in the γ-phosphate of ATP into mechanical work [22]. This energy conversion is carried out by each myosin’s motor domain, which undergoes a conformational change to generate force (∼ 3 pN for myosin-II) while taking a step (∼ 10 nm for myosin-II), to translocate its associated actin ﬁlament [23]. Thus, myosin-II is approximately 30% eﬃcient in its mechanochemical transduction. Myosin-II monomers, which actually consist of two heavy chains each with two light chains bound, assemble into bipolar thick ﬁlaments consisting of 10–400 monomers [24–26]. Myosin-II is typically considered a low duty ratio enzyme where each motor domain spends a fraction of its ATPase cycle in the force-generating state. However, the speciﬁc duty ratio can range from 1–80% [27–30]. Thus, as few as one or as many as tens of heads can be in the force-generating state at any point in time. Myosin-II regulation. Signiﬁcantly for cytokinesis in Dictyostelium, thick ﬁlament assembly is highly regulated. Phosphorylation promotes disassembly of the thick ﬁlaments while dephosphorylation promotes assembly [31]. This thick ﬁlament assembly regulation controls the amounts of myosin-II sent to the cleavage furrow cortex [21, 32]. Mutant myosin-II ﬁlaments that are nonphosphorylatable over-accumulate in the cleavage furrow cortex relative to wild type levels and at least one myosin heavy chain kinase, an enzyme that catalyzes phosphorylation reactions, is recruited to the cleavage furrow late in cytokinesis, presumably to drive disassembly of the myosin-II in preparation for ﬁnal bridge scission [33].

208

J.C. Eﬄer, P.A. Iglesias, and D.N. Robinson

3.3 Actin Crosslinking Proteins Actin crosslinking has an integral role in regulating cytokinesis dynamics. Actin crosslinkers organize the actin and increase the mechanical resistance of the network by linking individual ﬁlaments to each other in the network. Pure actin networks generate some mechanical resistance owing to entanglements, salt-bridging, and the fact that the actin polymers are semiﬂexible, having a persistence length of ∼ 15 nm. However, in in vitro systems, the presence of crosslinkers at varying stoichiometries with actin can vary the mechanical resistance of the actin network by greater than three orders-of-magnitude [7, 34–39]. In vivo, we have only detected a little over one decade of mechanical variation, ranging from latrunculin-treatment (which prevents polymerization of actin monomers) to overexpression of actin crosslinking proteins [40]. Thus, the allowable range for cytoskeletal resistance compatible with viability may be much narrower than is achievable in vitro. Several actin crosslinking proteins have been implicated in cytokinesis either by localization to the cleavage furrow cortex or implicated using genetics. In Dictyostelium, the cortexillins are actin crosslinking proteins that are specifically recruited to the cleavage furrow cortex [41–44]. Cortexillins are required to maintain normal cleavage furrow morphology under both adhesive and nonadhesive conditions, indicating that they may play a more fundamental role in cytokinesis. Dynacortin is an actin crosslinker distributed around the cell cortex and excluded from the cleavage furrow; this was the ﬁrst indication that both global and equatorial cortices have genetically separable mechanical roles during cytokinesis [45, 46]. Dynacortin is a dimeric actin crosslinker, identiﬁed in Dictyostelium. Both cortexillin and dynacortin are required for wild type levels of cortical viscoelasticity during interphase [40]. 3.4 Rho-Superfamily Members Rho-superfamily proteins act as molecular switches to regulate cytoskeletal components. In mammals, two classes of this superfamily, Rac and Rho, have essential roles during cytokinesis and are both localized to the plasma membrane [47–49]. Rho is speciﬁcally activated in the cleavage furrow cortex by proteins localized to the interzonal microtubules of the central spindle. Inhibition of Rho leads to a failure to assemble a contractile ring properly. Rac appears to act globally and inhibition of Rac leads to furrowing of the global cortex [50]. In Dictyostelium, no clear Rho homologs can be found in the genome, rather there are several Rac-subfamily genes that have varying degrees of similarity to Rac and Rho. Instead, the Rho function appears to be provided by Rac1 and inhibition of Rac1 leads to failure to properly recruit the cortexillins to the cleavage furrow [51]. The RacE gene product is globally distributed during cytokinesis. Deletion of RacE leads to a severe defect (∼ 20% of wild type

Regulating Cell Shape During Cytokinesis

a. Maxwell

b. Voigt

γ

k

γ

γ

k1

k2

k x

c. Kelvin

209

x

x

Fig. 3. Traditional mechanical models for cell deformation. The corresponding transfer functions between force and position are given in Table 1.

levels) in cortical mechanics and cytokinesis is defective under nonadherent conditions [52, 53]. The RacE mutant has normal ﬁlamentous actin levels but is defective in recruiting dynacortin to the cell cortex [45, 53]. Thus, the cortical mechanical defect of the RacE mutant is most likely due to a reduction in the actin crosslinkers. However, dynacortin is unlikely to be the only protein that is regulated by RacE. Further, removal of RacE and dynacortin from cells has synergistic eﬀects on the dynamics of cleavage furrow ingression [20].

4 Mechanical Nature of the Cell To study cell shape changes requires that we have a description of the underlying mechanical properties of the cell. Cellular shape is primarily dictated by the forces acting on the cortex and cytoplasm, which is usually viewed as a viscoelastic material; that is, it exhibits properties of both viscous liquids and elastic solids [1, 7]. This behavior can be observed in the time-dependent deformation response to mechanical stresses. Mechanical models of material behavior are traditionally based on combinations of idealized linear spring and dashpot elements [7]; see Fig. 3. Determining the response of a system to induced stresses is a relatively straightforward application of linear system theory. Maxwell, Voigt and Kelvin bodies diﬀer as to their composition. For example, the deformation induced by an applied force on a Maxwell body (Fig. 3a) is described by the diﬀerential equation: x˙ =

1 ˙ 1 f+ f k γ

where we have used the fact that the force on the spring and dashpot are equal, and that the total deformation is the sum of the individual terms. In the Laplace domain, the ratio G(s) :=

ks F (s) = X(s) s + k/γ

210

J.C. Eﬄer, P.A. Iglesias, and D.N. Robinson

is known in the ﬁeld as the viscoelastic impedance or, when s = jω, as the complex viscoelastic modulus. Note that it is common to express and plot G(jω) as a function of frequency in log-log form, as in Bode plots, but this is frequently done in Cartesian form G(jω) = G (jω) + jG (jω) rather than in polar form. In this case G (jω) and G (jω) are known as the storage and loss moduli respectively. One common means of studying the mechanical properties of cells is by applying a step change in the force and observing the ensuing cellular deformation, known as the creep compliance: Γ (t) = x(t)

f (t)=“unit step”

= L−1

1 . sG(s)

(1)

Alternatively, one may apply a deformation and measure the relaxation modulus; that is, the force that must be applied to maintain a unit-length deformation. This equals3 Υ (t) = f (t)

x(t)=“unit step”

= L−1

1 G(s) . s

Based on time series analysis of the creep or relaxation responses, it is possible to obtain estimates for the viscoelastic properties of the cell. Note that, because of the presence of a pole and a zero in G(s) for the Kelvin body, both responses exhibit exponential decays. In contrast, the Maxwell and Voigt bodies only exhibit an exponential decay in either the creep (Voigt) or relaxation (Maxwell) responses. For more complicated responses, time-domain analysis is diﬃcult to perform. Several other techniques are currently used to measure the frequencydependent rheological properties of cells. The mechanical properties of biological polymers can be studied by mechanically stimulating in vitro samples at diﬀerent frequencies by external harmonic excitation [35, 54]. One mechanism for doing so is the cone-plate rheometer, where actin gels are placed between a cone and plate. The plate is rotated at a given frequency thereby inducing a rotation of the cone that can be measured to obtain the complex viscoelastic modulus, assuming that x(t) = G(jω)ejωt . For live cells, more passive methods are also employed [55–58]. One technique, referred to as microrheology, is based on placing small beads at the surface of the cell and tracking their mean-squared-displacement. Assuming 3

Note that the usual notation for the relaxation modulus is G(t). We will eschew this notation to avoid the obvious confusion with G(s).

Regulating Cell Shape During Cytokinesis

211

Table 1. Mechanical properties for diﬀerent viscoelastic models of cells. We have used the notation k1 k2 = k1 k2 /(k1 + k2 ). Type

F (s)/X(s)

Spring k Dashpot γs Maxwell Voigt Kelvin

ks s + k/γ

Creep Function (x(t)) 1/k t/γ

1/k + t/γ ” 1“ k + γs 1 − e−kt/γ k „ „ « 1 k1 k1 s + (k1 k2 )/γ + 1− e−(k1 (k1 + k2 ) s + k1 /γ k1 + k2 γ γ

« k2 )t/γ

that motion is due to Brownian motion, the mean-square displacement of a particle diﬀusing in a three-dimensional medium equals x2 (t) = 6Dt where D is the diﬀusion coeﬃcient [59]. Using the Einstein relation (Dγ = kB T ) leads to 6kB T x2 (t) = t γ where γ is the coeﬃcient of drag. For a spherical particle ﬂowing in a viscous medium the drag coeﬃcient is given by Stokes theorem: γ = 6πηr, where r is the particle’s radius and η the viscosity of the medium. For a viscoelastic medium we replace η with G(s) to obtain: L[ x2 (t) ] =

kB T . πrsG(s)

This is known as the Generalized Stokes-Einstein Relation (GSER). Note that, from (1) we ﬁnd a link between the macro- and microrheological measurements: kB T Γ (t). x2 (t) = πr 4.1 Viscoelastic Properties of Cross-Linked Actin Networks Crosslinking proteins form dynamic associations, meaning that the kinetic rates for association and dissociation determine the mechanical properties of the cortex. For cross-linked polymer networks, the viscoelastic modulus is given by as G(s) = (2) s+k where k is the dissociation constant and a is the stiﬀness if the network were permanently crosslinked [38].

212

J.C. Eﬄer, P.A. Iglesias, and D.N. Robinson

In our experiments, the observed frequency dependent viscoelastic modulus agrees well with this model for a wide-variety of genetic mutants for frequencies between 1–1000 rad/s (our unpublished data). At higher frequencies, however, the viscoelastic modulus appears to increase as |G(jω)| ≈ ω 3/2 which is consistent with some models of polymer dynamics [60]. Interestingly, several mutants we have considered do not follow the Maxwell model of (2), but instead have frequency-dependent viscoelastic moduli that do not ﬁt a lumped-parameter model. System identiﬁcation techniques, including those developed by Keith Glover [61], will be invaluable in generating mechanical models of the mechanical properties of these cells. Quantitative analysis and analytical modeling of furrow dynamics of wild type and several genetically modiﬁed strains indicate that the viscoelastic properties of the equatorial and global cortices have distinct and signiﬁcant roles in controlling cytokinesis dynamics as well as distinct mechanical properties [20]. 4.2 Viscoelastic Model of the Cell Three general classes of models for cellular mechanics have been proposed; see [6] for a review. We focus on the cortical shell-liquid core model. This model assumes that the cell is composed of a cortical shell composed of membrane, actin cytoskeleton and associated cross-linking proteins and can be modeled using a Maxwell element as in (2). Inside the shell is a liquid core composed of viscoelastic cytoplasm where the viscosity is force-dependent. Furthermore, we assume that the cell is much larger than its individual components, so that a continuum assumption is valid. Using this model and level-set methods we have begun to simulate cytokinesis (article in preparation). 4.3 The Yoneda-Dan Model of Cytokinesis Yoneda and Dan suggested a model for describing the balance of forces at work during cytokinesis [21, 62]. In their model, the minimal contractile force required for stabilizing each of these intermediate shapes is proportional to the global steady state stiﬀness of the cell and is dependent on the extent of furrow ingression; see Fig. 4. Assuming that cellular volume is conserved during cytokinesis, we can compute a formula for the radius of the cleavage furrow (rf ) as a function of the angle θ. Let r be the radius of the mother cell just before commencing cytokinesis and r (θ) be the radius of the two daughter cells as a function of the angle θ. The total volume of the dividing cell is 4π [r (θ)]3 + 2π[r (θ)]3 cos θ(1 − 3

1 3

cos2 θ)

and this must equal 4πr3 /3. This means that the ratio of cell radii satisﬁes:

Regulating Cell Shape During Cytokinesis

a

213

Sc

2Sc cosθ θ r'(θ)

r'(θ)

θ

Relative force (F/Sc r)

b Sc

F rf (θ)

0.8 0.6 0.4 0.2 0

time

0

π/4

π/2

Angle (θ) Fig. 4. Yoneda and Dan model of force distribution during cytokinesis. a) The force required to ingress the furrow, F , must counterbalance the pull from the stretch modulus (Sc ). The radius of the two daughter cells equals r ; that of the furrow rf . b) The force distribution as a function of the angle θ.

r (θ) = 1+ r

1 2

cos θ(3 − cos2 θ)

−1/3

Using the Yoneda-Dan equation for computing the force needed to divide the cell yields F = 2rf Sc cos θ = 2Sc r (θ) sin θ cos θ sin 2θ = Sc r 1 1 + 2 cos θ(3 − cos2 θ)

1/3

Here Sc represents the stretch modulus (also referred to as surface tension), which is the viscoelasticity in the plane of the cell cortex. A plot of the function F/Sc r, as a function of θ shows that the required force to stabilize the shape is biphasic; see Fig. 4b. Using the Yoneda and Dan model, the amounts of myosin-II recruited to the cleavage furrow cortex can be predicted by considering biophysical properties of myosin-II [21, 63]. Measured amounts of myosin-II sent to the cleavage furrow cortex agree closely with the amount predicted, suggesting that cells regulate myosin-II localization to accomplish cytokinesis [21]. However, given the observed geometry, this model should only accurately apply when the curvature of the furrow is continuous with the emerging daughter cells.

214

J.C. Eﬄer, P.A. Iglesias, and D.N. Robinson

4.4 Stages in Cytokinesis: Diﬀerent Phases In Dictyostelium, cytokinesis proceeds via a highly stereotypical geometry, which can be subdivided into three phases [6, 20]. Each phase is regulated by diﬀerent subsets of molecules. During Phase 1, the mother cell rounds up, elongates into a cylinder and the furrow begins to ingress while maintaining continuous curvature with the two daughter cells. In Fig. 1, this corresponds to the ﬁrst 100 s. This stage appears to be dependent upon myosin-II, which is recruited to the cleavage furrow cortex during this stage, since Dictyostelium cells that lack myosin-II grown under nonadhesive conditions undergo cell rounding, but fail to elongate, constrict and divide [64]. During Phase 2, the cleavage furrow continues to ingress and forms an intracellular bridge with a cylindrical shape connecting the two daughter cells [20]. The diameter of the cylindrical bridge decreases approximately 50fold while lengthening only 2–4 fold, pushing cytoplasm out of the bridge. Genetic perturbations have revealed that this process is regulated in part by actin crosslinking proteins, which impart mechanical resilience to the cytoskeleton. Speciﬁcally, genetic inhibition of dynacortin and RacE reveal a substantial increase in the rate of cylindrical furrow thinning. During this stage, myosin-II reaches a maximum concentration at the cleavage furrow and then begins to decrease. The Yoneda and Dan model can explain Phase 1. However, during Phase 2 the cylindrical bridge has discontinuous curvature with the two daughter cells so that a diﬀerent scheme is required to account for the mechanics of the cell. To address the issue of how furrow thinning is governed, a cylinder thinning model has been developed for Dictyostelium cytokinesis [20]. In this model, four principal mechanical parameters (active radial stress, stretch modulus, viscosity and an axial compressive stress) are proposed to govern cytokinesis. By combining genetics, live-cell imaging, and biochemistry with modeling, each model parameter was able to be given a molecular identity. From the analysis, each genetic strain was shown to have a unique furrow-thinning trajectory. Wild type furrows thin following an exponential decay, while many of the mutant strains follow nonlinear, non-exponential dynamics. The wild type exponential trajectory is surprising since viscoelastic ﬂuids are predicted to thin with a linear trajectory, suggesting that a braking mechanism slows the wild type furrows and converts the trajectory into an exponential decay. RacE and dynacortin provide the brake that slows cytokinesis and myosin-II acts as an accelerator [20]. Thus, it may be possible that the cell can regulate cytokinesis dynamics in part by modulating RacE and dynacortin activity, thereby allowing it to respond to environmentally induced stress. Phase 3 is the bridge dwelling phase. It is genetically and quantitatively separable from Phases 1 and 2 [20]. During this phase the bridge does not thin appreciably and appears to be regulated by cytoskeletal disassembly.

Regulating Cell Shape During Cytokinesis

215

This series of shape changes has important implications for the mechanics of cytokinesis. First, Phase 1 can be thought of as a movement away from an equilibrium shape while Phase 2 is a return to an equilibrium shape. Thus, a mechanical transition from an active process requiring force generation from myosin-II to a passive-like process is suggested.

5 Controlling Mechanics of Cytokinesis Cytokinesis mechanics are regulated by the complex interplay between multiple concurrent processes including myosin-II mechanochemistry and localization, actin dynamics, actin crosslinking proteins, and molecular switches of the Rho-superfamily. How do these various processes ﬁt together spatially and temporally to regulate this complex process? Microtubules provide cues which initiate cytokinesis. Interzonal microtubules positioned at the midpoint of the cell act as tracks through which the components of the contractile rings are recruited. Rac1 recruits the cortexillins to the cleavage furrow, while RacE recruits dynacortin to the global cortex and globally inhibits contraction. During Phase 1, the cell rounds up, elongates and furrow ingression begins. At a molecular level, myosin-II and cortexillin-I are localized to the furrow, while dynacortin is excluded from the furrow and enriched in the polar cortex. New actin ﬁlaments are also formed at the furrow region, and some studies suggest that actin polymerization at the two polar cortices may provide forces which help drive elongation. During Phase 2, thinning of the cleavage furrow to a narrow cylindrical bridge, myosin-II reaches its peak accumulation at the furrow and then decreases. Myosin-II generates contractile forces to drive ingression, but may also play a role in removal of actin from the furrow cortex [19]. Myosin-II mechanochemistry may pull actin ﬁlaments from bundles, exposing sites for coﬁlin, which is recruited to the furrow cortex, to sever the actin. As the furrow thins, myosin-II heavy chain kinase is recruited to the furrow late in Phase 2, driving disassembly of myosin-II ﬁlaments [33]. Cytokinesis concludes with disassembly of the thin bridge connecting the two daughter cells.

6 Role of Mechanical Feedback Understanding how cells perform complex processes under a diverse array of conditions with high ﬁdelity is an important question in cell biology. Feedback control systems are well appreciated in numerous biosynthetic and metabolic pathways. For example, the well studied glucose homeostasis pathway maintains a constant level of glucose in the bloodstream by employing negative feedback. The mechanochemical feedback system of muscle contraction has also historically been well studied [65–67]. Muscle contraction requires the

216

J.C. Eﬄer, P.A. Iglesias, and D.N. Robinson

actomyosin system of the sarcomere (a single contractile unit of muscle) to generate force. The amount of force that is generated changes in response to the applied load. The myosin-II mechanoenzyme responds to applied loads by a strain-induced alteration of its mechanochemical cycle — namely, decreasing the ADP-release rate, which increases the amount of time the motor remains bound to the actin ﬁlament. This decrease in a single kinetic step increases the number of motor heads in the force-generating state, allowing each sarcomere to generate greater tension. Thus, in muscle, mechanical feedback acts through a direct modiﬁcation of the mechanochemical coupling of the force-generating enzyme. Evidence in support of nonmuscle cells having mechanical feedback mechanisms also exists. A number of studies have demonstrated that applied stresses change the biochemical makeup of the cortical cytoskeleton and lead to activation of regulatory kinases [68–70]. Cells also respond with cytoskeletal alterations to environmental stresses such as osmotic shock. In Dictyostelium, osmotic stress leads to a dramatic recruitment of myosin-II to the cortex and myosin-II mutant cells are signiﬁcantly more sensitive to osmotic pressure [71]. 6.1 Evidence for Mechanical Feedback in Cytokinesis Dividing cells may also have feedback mechanisms that allow them to successfully complete cytokinesis in diverse mechanical environments. By sensing when cytokinesis is not progressing properly and responding by making mechanical adjustments to correct the process, cells may utilize force-feedback mechanisms to control the amount of force that is exerted by the cleavage furrow cortex, possibly by regulating myosin-II accumulation. Supporting evidence comes from studies in which Dictyostelium cytokinesis has been compared between three growth conditions: on surfaces alone, on surfaces with a sheet of agar overlaying the cells, and in suspension culture. In Dictyostelium, myosin-II is not required when the cells are propagated on surfaces, which allows for traction or adhesive forces to assist in cytokinesis; however, myosin-II is essential for non-adherent cytokinesis [64, 72] as well as under agar overlay [73]. By compressing the cell, the sheet of agar may increase the tension on the cytoskeleton, increasing the resistance so that additional force is required for furrow contractility. Consistent with this hypothesis, more myosin-II appears to accumulate in the cleavage furrow cortex of wild type cells sandwiched under a sheet of agar than when cells are grown on surfaces [73]. This suggests that the cell responds to mechanical perturbation by accumulating more myosin-II in the cleavage furrow, enabling successful completion of cytokinesis. Phase 1, in which the cell transforms from a spherical to a elongated cylindrical shape, utilizes the accumulation of myosin-II at the cleavage furrow to drive this shape change. In Phase 2, the concentration of myosin-II reaches a peak and then decreases, with myosin-II phosphorylation thought to play a role in myosin-II

Regulating Cell Shape During Cytokinesis

external perturbations other CR proteins spindle signals

forces

217

genetic manipulations

cytoskeleton mechanics

cell shape

myosin-II localization mechanosensor

Fig. 5. Feedback model for regulation during cytokinesis. This block diagram captures our view of the mechanical regulation taking place during cytokinesis. Protein localization signals from the mitotic spindle polarize the cell. Together they help generate the forces that act on the cell mechanics and generate cell shape. We conjecture that a mechanosensor feeds back signals that correlate either to cell mechanics (such as cortical tension) or cell shape to inﬂuence the spatial localization of myosin-II (dotted lines) or other contractile ring (CR) proteins.

disassembly. Intriguingly, mutant myosin-IIs resistant to this regulation accumulate to higher levels in the cleavage furrow cortex suggesting not only a regulatory role for phosphorylation, but also illustrating that myosin accumulation in the furrow is a highly dynamic process [21]. Maintaining a myosin concentration under conditions of high turnover likely incorporates a sensory mechanism capable of detecting shape along with feedback mechanisms regulating progression through various mechanical stages. Since cells can complete cytokinesis with high eﬃciency in suspension culture or on surfaces (Dictyostelium) or in tissues or blood streams (mammalian cells) where they often experience compressive forces, mechanical feedback mechanisms may prove to be the essential principle that allows for successful cytokinesis. 6.2 Evaluating Models of Mechanical Feedback To elucidate the complex regulation of cytokinesis, two methods of perturbing the system are available; see Fig. 5. The transfer function of the cell can be modiﬁed by genetic perturbation. The cortical stretch modulus can be reduced by genetically removing the genes that regulate or encode for actin cross-linking proteins. For example, a genetic strain with a compromised cortical stretch modulus may require less force (since there is less cortical resistance) for cleavage furrow ingression. The second method of perturbing the system is by applying an external mechanical force. The cell’s ability to reject this mechanical disturbance can be measured by changes in localization and

218

J.C. Eﬄer, P.A. Iglesias, and D.N. Robinson

concentration of proteins. For instance, force may be applied to impede cytokinesis mechanically, which requires an increase in contractile force at the cleavage furrow to drive ingression. Combining these two methods, it is possible to probe for the existence of mechanical sensors that regulate the amount of force produced based on the mechanical conditions the cell experiences.

7 Conclusions Cytokinesis involves a complex, dynamic interplay between biochemistry and cell mechanics. Its understanding has potential to shed light on a number of processes dependent on cell shape and may provide insight into numerous genetic disorders and tumor cell biology, as well as identify new therapeutic targets. Cytokinetic failure results in binucleate cells, which increases the number of single sets of chromosomes. Accompanying this increase is the ampliﬁcation of the centrosomes, which lead to multipolar spindles [74]. Subsequent errors in chromosomal segregation can also lead to aneuploidy, the state of having an incorrect number of chromosomes or chromosome sets, which is associated with many types of tumors and is correlated with tumor type and disease progression [75–77]. Cytokinesis also provides a source of novel potential drug targets that may be useful for the treatment of hyperproliferative diseases such as cancer. Taxol, a leading anti-cancer agent, targets mitosis by stabilizing microtubules, thereby inhibiting cell proliferation. However, because taxol inhibits a widerange of microtubule functions, many microtubule-dependent processes are aﬀected, leading to a number of side-eﬀects, such as peripheral neuropathies. Investigators hope that unwanted side-eﬀects can be avoided by speciﬁcally targeting dividing cells. Thus, chemical approaches are underway in a number of venues to identify novel mitosis and cytokinesis inhibitors; see for example [78]. Inextricably linked to cell mechanics, cytokinesis is a slowly evolving intrinsic cell shape change that depends on the cell’s biochemistry to manipulate its fundamental physical properties. The regulation of cell shape during cytokinesis is a complex biological process, and its complete elucidation requires knowledge from a variety of ﬁelds including mechanics, biophysics, genetics and cell biology. Moreover, because of the tight regulatory feedback paths that exist, the study of cytokinesis can greatly beneﬁt by the application of control theory as other systems have, such as bacterial [79] and eukaryotic chemotaxis [80, 81], heat-shock regulation [82], cell polarity [83] and cell cycle regulation [84]. We believe that this presents a new frontier for the ﬁeld of control engineering, a fact that has not gone unnoticed by Keith Glover who, as head of the Cambridge Engineering Department, selected “engineering for life sciences” as one the central

Regulating Cell Shape During Cytokinesis

219

research themes for the department. To Keith we dedicate this paper on his 60th birthday. Acknowledgements This work was supported in part by a Dean’s Fellowship from the Whiting School of Engineering (JCE), an NSF Biocomplexity grant DMS-083500 (PAI), NIH grants R01-GM71920 (PAI) and R01-GM66817 (DNR), and a Beckman Young Investigator Fellowship (DNR).

References 1. D. Bray, Cell Movements: From Molecules to Motility, 2nd ed. New York: Garland, 2001. 2. C. S. Chen, M. Mrksich, S. Huang, G. Whitesides, and D. E. Ingber, “Geometric control of cell life and death,” Science, vol. 276, no. 5317, pp. 1425–1428, 1997. 3. J. L. Tan, J. Tien, D. M. Pirone, D. S. Gray, K. Bhadriraju, and C. S. Chen, “Cells lying on a bed of microneedles: An approach to isolate mechanical force,” Proc Natl Acad Sci USA, vol. 100, no. 4, pp. 1484–1489, Feb 2003. 4. L. Vergani, M. Grattarola, and C. Nicolini, “Modiﬁcations of chromatin structure and gene expression following induced alterations of cellular shape,” Int J Biochem Cell Biol, vol. 36, no. 8, pp. 1447–1461, Aug 2004. 5. R. Rappaport, Cytokinesis in Animal Cells., ser. Developmental and Cell Biology Series. Cambridge: Cambridge University Press, 1996. 6. E. M. Reichl, J. C. Eﬄer, and D. N. Robinson, “The stress and strain of cytokinesis,” Trends Cell Biol, vol. 15, no. 4, pp. 200–206, Apr 2005. 7. J. Howard, Mechanics of Motor Proteins and the Cytoskeleton. Sunderland, MA: Sinauer Associates, 2001. 8. M. Glotzer, “The molecular requirements for cytokinesis,” Science, vol. 307, no. 5716, pp. 1735–1739, Mar 2005. 9. D. N. Robinson and J. A. Spudich, “Towards a molecular understanding of cytokinesis,” Trends Cell Biol, vol. 10, no. 6, pp. 228–237, Jun 2000. 10. K. C. Holmes, D. Popp, W. Gebhard, and W. Kabsch, “Atomic model of the actin ﬁlament,” Nature, vol. 6, no. 347, pp. 44–49, Sep 1990. 11. D. R. Kovar and T. D. Pollard, “Insertional assembly of actin ﬁlament barbed ends in association with formins produces piconewton forces,” Proc Natl Acad Sci USA, vol. 101, no. 41, pp. 14 725–14 730, Oct 2004. 12. T. D. Pollard and G. G. Borisy, “Cellular motility driven by assembly and disassembly of actin ﬁlaments,” Cell, vol. 112, no. 4, pp. 459–65, Feb 2003. 13. R. J. Pelham and F. Chang, “Actin dynamics in the contractile ring during cytokinesis in ﬁssion yeast,” Nature, vol. 419, no. 6902, pp. 82–86, Sep 2002. 14. K. C. Y. Wong, V. M. D’souza, N. I. Naqvi, F. Motegi, I. Mabuchi, and M. K. Balasubramanian, “Importance of a myosin II-containing progenitor for actomyosin ring assembly in ﬁssion yeast.” Curr Biol, vol. 12, no. 9, pp. 724–729, Apr 2002. 15. L. G. Cao and Y. L. Wang, “Mechanisms of the formation of contractile ring in dividing cultured animal cells. I. Recruitment of preexisting actin ﬁlaments into the cleavage furrow.” J. Cell Biol., vol. 110, no. 4, pp. 1089–1095, Apr 1990.

220

J.C. Eﬄer, P.A. Iglesias, and D.N. Robinson

16. S. Yumura and Y. Fukui, “Spatiotemporal dynamics of actin concentration during cytokinesis and locomotion in Dictyostelium.” J. Cell Sci., vol. Pt 6, pp. 2097–2108, Aug 1998. 17. L. G. Cao and Y. L. Wang, “Mechanism of the formation of contractile ring in dividing cultured animal cells. II. Cortical movement of microinjected actin ﬁlaments.” J. Cell Biol., vol. 111, no. 5, pp. 1905–1911, Nov 1990. 18. K. C. Gunsalus, S. Bonaccorsi, E. Williams, F. Verni, M. Gatti, and M. L. Goldberg, “Mutations in twinstar, a Drosophila gene encoding a coﬁlin/ADF homologue, results in defects in centrosome migration and cytokinesis,” J Cell Biol, vol. 131, no. 5, pp. 1243–1259, Dec 1995. 19. M. Guha, M. Zhou, and Y. L. Wang, “Cortical actin turnover during cytokinesis requires myosin II,” Curr Biol, vol. 15, no. 8, pp. 732–736, Apr 2005. 20. W. Zhang and D. N. Robinson, “Balance of actively generated contractile and resistive forces controls cytokinesis dynamics,” Proc Natl Acad Sci USA, 2005. 21. D. N. Robinson, G. Cavet, H. M. Warrick, and J. A. Spudich, “Quantitation of the distribution and ﬂux of myosin-II during cytokinesis,” BMC Cell Biology, vol. 3, no. 1, p. 4, Feb 2002. 22. J. Spudich, “The myosin swinging cross-bridge model,” Nat Rev Mol Cell Biol, vol. 2, no. 5, pp. 387–92, 2001. 23. J. T. Finer, R. M. Simmons, and J. A. Spudich, “Single myosin molecule mechanics: Piconewton forces and nanometre steps,” Nature, vol. 368, no. 6467, pp. 113–119, Mar 1994. 24. R. Niederman and T. D. Pollard, “Human platelet myosin II. In vitro assembly and structure of myosin ﬁlaments,” J Cell Biol, vol. 67, no. 1, pp. 72–92, Oct 1975. 25. R. K. Mahajan and J. D. Pardee, “Assembly mechanism of Dictyostelium myosin II: Regulation by K+ , Mg2+ , and actin ﬁlaments,” Biochemistry, vol. 35, no. 48, pp. 15 504–15 514, Dec 1996. 26. F. A. Pepe and B. Drucker, “The myosin ﬁlament VI. myosin content,” J Mol Biol, vol. 130, no. 4, pp. 379–393, Jun 1979. 27. J. Howard, “Molecular motors: Structural adaptations to cellular functions,” Nature, vol. 389, no. 6651, pp. 561–567, Oct 1997. 28. S. S. Rosenfeld, J. Xing, L.-Q. Chen, and H. L. Sweeney, “Myosin IIB is unconventionally conventional,” J Biol Chem, vol. 278, no. 30, pp. 27 449–27 455, Jul 2003. 29. M. Kovacs, F. Wang, A. Hu, Y. Zhang, and J. R. Sellers, “Functional divergence of human cytoplasmic myosin II: Kinetic characterization of the non-muscle IIA isoform,” J Biol Chem, vol. 278, no. 40, pp. 38 132–38 140, Oct 2003. 30. F. Wang, M. Kovacs, A. Hu, J. Limouze, E. V. Harvey, and J. R. Sellers, “Kinetic mechanism of non-muscle myosin IIB: functional adaptations for tension generation and maintenance,” J Biol Chem, vol. 278, no. 30, pp. 27 439–27 448, Jul 2003. 31. T. T. Egelhoﬀ, R. J. Lee, and J. A. Spudich, “Dictyostelium myosin heavy chain phosphorylation sites regulate myosin ﬁlament assembly and localization in vivo,” Cell, vol. 75, no. 2, pp. 363–371, Oct 1993. 32. J. H. Sabry, S. L. Moores, S. Ryan, J.-H. Zang, and J. A. Spudich, “Myosin heavy chain phosphorylation sites regulate myosin localization during cytokinesis in live cells,” Mol Biol Cell, vol. 8, no. 12, pp. 2647–2657, Dec 1997.

Regulating Cell Shape During Cytokinesis

221

33. W. Liang, L. S. Licate, H. M. Warrick, J. A. Spudich, and T. T. Egelhoﬀ, “Differential localization in cells of myosin II heavy chain kinases during cytokinesis and polarized migration,” BMC Cell Biology, vol. 3, no. 1, Jul 2002. 34. M. L. Gardel, J. H. Shin, F. C. MacKintosh, L. Mahadevan, P. Matsudaira, and D. A. Weitz, “Elastic behavior of cross-linked and bundled actin networks,” Science, vol. 304, no. 5675, pp. 1301–1305, May 2004. 35. C. Storm, J. J. Pastore, F. C. MacKintosh, T. C. Lubensky, and P. A. Janmey, “Nonlinear elasticity in biological gels,” Nature, vol. 435, no. 7039, pp. 191–194, May 2005. 36. J. Xu, D. Wirtz, and T. D. Pollard, “Dynamic cross-linking by α-actinin determines the mechanical properties of actin ﬁlament networks,” J Biol Chem, vol. 273, no. 16, pp. 9570–9576, May 1998. 37. M. Sato, W. H. Schwarz, and T. D. Pollard, “Dependence of the mechanical properties of actin/α-actinin gels on deformation rate,” Nature, vol. 325, no. 6107, pp. 828–830, 1987. 38. D. H. Wachsstock, W. H. Schwarz, and T. D. Pollard, “Aﬃnity of α-actinin determines the structure and mechanical properties of actin ﬁlament gels,” Biophys. J., vol. 65, no. 1, pp. 205–214, Jul 1993. 39. ——, “Cross-linker dynamics determine the mechanical properties of actin gels.” Biophys J, vol. 66, pp. 801–809, 1994. 40. K. D. Girard, C. Chaney, M. Delannoy, S. C. Kuo, and D. N. Robinson, “Dynacortin contributes to cortical viscoelasticity and helps deﬁne the shape changes of cytokinesis,” EMBO J, vol. 23, no. 7, pp. 1536–1546, Apr 2004. 41. I. Weber, G. Gerisch, C. Heizer, J. Murphy, K. Badelt, A. Stock, J.-M. Schwartz, and J. Faix, “Cytokinesis mediated through the recruitment of cortexillins into the cleavage furrow,” EMBO J, vol. 18, no. 3, pp. 586–594, Feb 1999. 42. I. Weber, R. Neujahr, A. Du, J. K¨ ohler, J. Faix, and G. Gerisch, “Two-step positioning of a cleavage furrow by cortexillin and myosin II,” Curr Biol, vol. 10, no. 9, pp. 501–506, May 2000. 43. J. Faix, I. Weber, U. Mintert, J. K¨ ohler, F. Lottspeich, and G. Marriott, “Recruitment of cortexillin into the cleavage furrow is controlled by Rac1 and IQGAP-related proteins,” EMBO J, vol. 20, no. 14, pp. 3705–3715, Jul 2001. 44. J. Faix, M. Steinmetz, H. Boves, R. A. Kammerer, F. Lottspeich, U. Mintert, J. Murphy, A. Stock, U. Aebi, and G. Gerisch, “Cortexillins, major determinants of cell shape and size, are actin-bundling proteins with a parallel coiled-coil tail,” Cell, vol. 86, no. 4, pp. 631–642, Aug 1996. 45. D. N. Robinson and J. A. Spudich, “Dynacortin, a genetic link between equatorial contractility and global shape control discovered by library complementation of a Dictyostelium discoideum cytokinesis mutant,” J Cell Biol, vol. 150, no. 4, pp. 823–838, Aug 2000. 46. D. N. Robinson, S. S. Ocon, R. S. Rock, and J. A. Spudich, “Dynacortin is a novel actin bundling protein that localizes to dynamic actin structures,” J Biol Chem, vol. 277, no. 11, pp. 9088–9095, Mar 2002. 47. W. G. Somers and R. Saint, “A RhoGEF and Rho family GTPase-activating protein complex links the contractile ring to cortical microtubules at the onset of cytokinesis,” Dev Cell, vol. 4, no. 1, pp. 29–39, Jan 2003. 48. M. Mishima and M. Glotzer, “Cytokinesis: A logical GAP,” Curr Biol, vol. 13, no. 15, pp. 589–591, Aug 2003.

222

J.C. Eﬄer, P.A. Iglesias, and D.N. Robinson

49. S. N. Prokopenko, A. Brumby, L. O’Keefe, L. Prior, Y. He, R. Saint, and H. J. Bellen, “A putative exchange factor for Rho1 GTPase is required for initiation of cytokinesis in Drosophila.” Genes Dev, vol. 13, no. 17, pp. 2301–2314, Sep 1999. 50. C. B. O’Connell, S. P. Wheatley, S. Ahmed, and Y. L. Wang, “The small GTPbinding protein Rho regulates cortical activities in cultured cells during division,” J Cell Biol, vol. 144, no. 2, pp. 305–313, Jan 1999. 51. J. Faix, C. Clougherty, A. Konzok, U. Mintert, J. Murphy, R. Albrecht, B. M¨ uhlbauer, and J. Kuhlmann, “The IQGAP-related protein DGAP1 interacts with Rac and is involved in the modulation of the F-actin cytoskeleton and control of cell motility,” J Cell Sci, vol. 111, pp. 3059–3071, Oct 1998. 52. D. A. Larochelle, K. K. Vithalani, and A. DeLozanne, “Role of Dictyostelium RacE in cytokinesis: Mutational analysis and localization studies by the use of green ﬂuorescent protein,” Mol Biol Cell, vol. 8, no. 5, pp. 935–944, May 1997. 53. N. Gerald, J. Dai, H. P. Ting-Beall, and A. DeLozanne, “A role for Dictyostelium RacE in cortical tension and cleavage furrow progression,” J Cell Biol, vol. 141, no. 2, pp. 483–492, Apr 1998. 54. J. C. Crocker, M. T. Valentine, E. R. Weeks, T. Gisler, P. D. Kaplan, A. G. Yodh, and D. Weitz, “Two-point microrheology of inhomogeneous soft materials,” Phys Rev Lett, vol. 85, no. 4, pp. 888–891, Jul 2000. 55. P. A. Janmey and D. A. Weitz, “Dealing with mechanics: Mechanisms of force transduction in cells,” Trends Biochem Sci, vol. 29, no. 7, pp. 364–370, Jul 2004. 56. Y. P. Tseng, T. P. Kole, and D. Wirtz, “Micromechanical mapping of live cells by multiple-particle tracking microrheology,” Biophys J, vol. 83, no. 6, pp. 3162– 3176, Dec 2002. 57. S. Yamada, D. Wirtz, and S. C. Kuo, “Mechanics of living cells measured by laser trackig microrheology,” Biophys J, vol. 78, no. 4, pp. 1736–1747, Apr 2000. 58. A. W. Lau, B. D. Hoﬀman, A. Davies, J. C. Crocker, and T. C. Lubensky, “Microrheology, stress ﬂuctuations, and active behavior of living cells,” Phys Rev Lett, vol. 91, no. 19, Nov 2003. 59. H. C. Berg, Random Walks in Biology. Princeton: Princeton UP, 1993. 60. D. C. Morse, “Viscoelasticity of tightly entangled solutions of semiﬂexible polymers,” Phys Rev E, vol. E58, pp. 1237–1240, Aug 1998. 61. L. Ljung and K. Glover, “Frequency domain versus time domain methods in system identiﬁcation,” Automatica, vol. 17, no. 1, pp. 71–86, 1981. 62. M. Yoneda and K. Dan, “Tension at the surface of the dividing sea-urchin egg,” J Exp Biol, vol. 57, no. 3, pp. 575–587, Dec 1972. 63. D. N. Robinson and J. A. Spudich, “Mechanics and regulation of cytokinesis,” Curr Opin Cell Biol, vol. 16, no. 2, pp. 182–188, Apr 2004. 64. J. H. Zang, G. Cavet, J. H. Sabry, P. Wagner, S. L. Moores, and J. A. Spudich, “On the role of myosin-II in cytokinesis: Division of Dictyostelium cells under adhesive and nonadhesive conditions,” Mol Biol Cell, vol. 8, no. 12, pp. 2617– 2629, Dec 1997. 65. J. D. Jontes, “Theories of muscle contraction,” J Struct Biol, vol. 115, no. 2, pp. 119–143, Sep 1995. 66. R. M. Simmons and A. F. Huxley, “Proposed mechanism of force generation in striated muscle,” Nature, vol. 233, no. 5321, pp. 533–538, Oct 1971. 67. A. Mehta, “Myosin learns to walk,” J Cell Sci, vol. 114, pp. 1981–1998, 2001. 68. M. Tamada, M. P. Sheetz, and Y. Sawada, “Activation of signaling cascade by cytoskeleton stretch,” Dev Cell, vol. 7, no. 5, pp. 709–718, Nov 2004.

Regulating Cell Shape During Cytokinesis

223

69. X. Ren, W. B. Kiosses, and M. A. Schwartz, “Regulation of the small GTPbinding protein Rho by cell adhesion and the cytoskeleton,” EMBO J, vol. 18, no. 3, pp. 578–585, Mar 1999. 70. D. Riveline, E. Zamir, N. Q. Balaban, U. S. Schwarz, T. Ishizaki, S. Narumiya, Z. Kam, B. Geiger, and A. D. Bershadsky, “Focal contacts as mechanosensors: Externally applied local mechanical force induces growth of focal contacts by an mDia1-dependent and ROCK-independent mechanism,” J Cell Biol, vol. 153, no. 6, pp. 1175–1186, Jun 2001. 71. H. Kuwayama, M. Ecke, G. Gerisch, and P. J. M. Van Haastert, “Protection against osmotic stress by cGMP-mediated myosin phosphorylation,” Science, vol. 271, no. 5246, pp. 207–209, Jan 1996. 72. A. DeLozanne and J. A. Spudich, “Disruption of the Dictyostelium myosin heavy chain gene by homologous recombination,” Science, vol. 236, no. 4805, pp. 1086– 1091, May 1987. 73. T. Q. P. Uyeda, C. Kitayama, and S. Yumura, “Myosin ii-independent cytokinesis in dictyostelium: Its mechanism and implications,” Cell Struct Funct, vol. 25, no. 1, pp. 1–10, Feb 2000. 74. N. J. Quintyne, J. E. Reing, D. R. Hoﬀelder, S. M. Gollin, and W. S. Saunders, “Spindle multipolarity is prevented by centrosomal clustering,” Science, vol. 307, no. 5706, pp. 127–129, Jan 2005. 75. P. Dey, “Aneuploidy and malignancy: An unsolved equation,” J Clin Pathol, vol. 57, no. 12, pp. 1245–1249, Dec 2004. 76. R. Giet, C. Petretti, and C. Prigent, “Aurora kinases, aneuploidy and cancer, a coincidence or real link?” Trends Cell Biol, vol. 15, no. 5, pp. 241–250, May 2005. 77. W. S. Saunders, M. Shuster, X. Huang, B. Gharaiben, A. K. Enyenihi, I. Petersen, and S. M. Gollon, “Chromosomal instability and cytoskeletal defects in oral cancer cells,” Proc Natl Acad Sci USA, vol. 97, no. 1, pp. 303–308, Jan 2000. 78. A. Cheung, J. Dantzig, S. Hollingworth, S. M. Baylor, Y. E. Goldman, T. J. Mitchison, and A. F. Straight, “A small-molecule inhibitor of skeletal muscle myosin II,” Nat Cell Biol, vol. 4, no. 1, pp. 83–88, Jan 2002. 79. T. M. Yi, Y. Huang, M. I. Simon, and J. Doyle, “Robust perfect adaptation in bacterial chemotaxis through integral feedback control,” Proc Natl Acad Sci USA, vol. 97, no. 9, pp. 4649–4653, Apr 2000. 80. L. Ma, C. Janetopoulos, L. Yang, P. N. Devreotes, and P. A. Iglesias, “Two complementary, local excitation, global inhibition mechanisms acting in parallel can explain the chemoattractant-induced regulation of PI(3,4,5)P3 response in Dictyostelium cells,” Biophys J, vol. 87, no. 6, pp. 3764–3774, Dec 2004. 81. P. A. Iglesias, “Feedback control in intracellular signaling pathways: Regulating chemotaxis in Dictyostelium discoideum,” Eur J Control, vol. 9, no. 2–3, pp. 227–236, 2003. 82. H. El-Samad, H. Kurata, J. C. Doyle, C. A. Gross, and M. Khammash, “Surviving heat shock: Control strategies for robustness and performance,” Proc Natl Acad Sci USA, vol. 102, no. 8, pp. 2736–2741, Feb 2005. 83. K. Amonlirdviman, N. A. Khare, D. R. P. Tree, W.-S. Chen, J. D. Axelrod, and C. J. Tomlin, “Mathematical modeling of planar cell polarity to understand domineering nonautonomy,” Science, vol. 307, no. 5708, pp. 423–426, Jan 2005.

224

J.C. Eﬄer, P.A. Iglesias, and D.N. Robinson

84. J. R. Pomerening, E. D. Sontag, and J. E. Ferrell Jr., “Building a cell cycle oscillator: Hysteresis and bistability in the activation of Cdc2,” Nat Cell Biol, vol. 5, no. 4, pp. 346–351, Apr 2003.

Intrinsic Uncertainty in Gene Regulation Network Hidenori Kimura Bio-Mimetic Control Research Center, RIKEN 2271-130, Anagahora, Shimoshidami, Moriyama-ku, Nagoya, 463-0003, Japan [email protected]

1 Introduction Regulation of gene expression is the fundamental biological process through which a variety of cell functions are organized and executed ( [9] [12]). Due to recent advance of molecular biology, a number of gene regulatory networks have been identiﬁed which are capable of executing important tasks such as signal transduction, switching, homeostasis, rhythm generation, developmental pattern formation and so on. They are surprizingly complex and highly sophisticated. Even a simple network of metabolic pathway in Escherichia coli is complex enough to be comparable with the most advanced man-made control system in industry. Theoretical research to understand the underlying principle of gene regulation mechanism has been quite active in the last few decades, and successfully enhanced our understanding of the network operations through the notions of positive and negative feedbacks, robustness, time-delay, stability, bifurcation and so on. However, nonlinearity and complexity which represent fundamental characteristic features of life phenomena have been insurmountable obstacles for getting in-depth inderslanding of gene regulating networks beyond modelling/simulation paradigm. Recently, an interesting new approach in this area emerges which creates a simple artiﬁcial gene regulation network in vivo and make extensive analysis both theoretically and experimentarily. This is the engineering approach that enables us to focus on the core features of the network and to make extensive theoretical analysis beyond modelling/simulation paradigm. The ﬁrst research along this line was made by Gardner et al. [5]. They constructed an artiﬁcial network of coupled negative feedback that behaves like a biological switch in Escherichia coli. They showed that their network was bistable and demonstrated its perfect switching behaviors both theoretically and experimentally. Later, the engineered gene regulatory networks have been constructed and analyzed by several researchers that have contributed to promote our understanding of the fundamental mechanisms of gene regulatory network (e.g., [1] [2] [4]). A good review in this area is found in [18]. B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 225–242, 2006. © Springer-Verlag Berlin Heidelberg 2006

226

H. Kimura

Recently, it became clear that the chemical dynamics are subject to substantial ﬂuctuations, mainly due to the smallness of the numbers of molecules involved in the reactions. Considerable experimental evidence that indicates the presence of signiﬁcant stochasticity in transcriptional regulation has been reported [3], [11] and [14]. These molecular ﬂuctuations are sometimes obstacles for executing cell’s normal functions, but they are thought to be useful to generate population heterogeneity and phenotypic variation. Theoretical investigations are also actively undertaken based on various theoretical tools on stochastic process [19], [20] and [22] and also on novel simulation algorithms specially deviced for chemical reactions [6], [7] and [13]. The prupose of this paper is to address a fundamental gap between the deterministic and stochastic frameworks which is insurmountable for some cases. These two approaches are diﬀerent, not only in terms of the ways of looking at the system, but also in terms of predictions of behaviors they predict theoretically. These gaps manifest themselves particularly in the long-run behaviors. In deterministic discription, some gene regulatory networks are multi-stable, i.e., the system converges to diﬀerent states depending on the initial states. In some sense, a multi stable system can memorize the initial state in terms of the stable equilibrium states to which it converges. In stochastic description, all systems are gmono-stable,” in the sense that during the time evolution, the probability distribution converges to a unique equilibrium distribution irrespective of its initial distribution, provided that the system is irreducible (Irreducibility always holds in common biochemical processes). Hence, all the biological functions that take advantage of the multi-stability of the network do not work in long-run. A typical example is the biological toggle switch whose main function is to memorize the state of the system before outer excitation comes. Stochastic theory, however, shows that the state of the switch not only ﬂuctuates randomly but also it tends to an identical probability distribution. Thus, the switch does not work eventually. In this paper, we show that this is actually the case. The state distribution is shown to converge a unique stationary distribution which has the multiple peaks located at the stable equilibrium states. We shall discuss the biological meaning of this fact.

2 Master Equation for Chemical Reactions Consider a system of chemical reactions composed of M reactions which involve K chemical species. Denote by x = (x1 , x2 , · · · , xK )T the molecular population vector whose i-th component xi represents the number of the molecules of the i-th specy. Associated with the system is the stoichiometric matrix A whose (i, j)- component aij represents the molecular number of the j-th specy created through the i-th chemical reaction. Let wj (x) be the propensity function of the j-th reaction at the state x. It represents the

Intrinsic Uncertainty in Gene Regulation Network

227

rate of the j-th reaction. It can be interpreted, with some modiﬁcation, as the probability that the j-th reaction occurs in the unit time. Write the propensity T vector as w(x) = [w1 (x), w2 (x), · · · , wM (x)] . As an example, consider a dimetrization/dissociation reaction 2S In this case, A=

−2 2 , 1 −1

k1 k2

S2 .

w(x) =

k1 x1 (x1 − 1) . k2 x2

The time evolution of the population vector x is subject to dx = Aw(x), dt

(1)

which represents to the mass-action law representing the time behavior of the concentrations of species rather than the molecular populations. The time behavior of x is completely determined if we specify the initial condition x(0). The deterministic view represented by the diﬀerential equation (1) is far from reality at least in intracellular processes, if we consider large ﬂuctuations associated with the biochemical process taking place inside cells. Fluctuations are originated roughly in two ways. The ﬁrst comes from outside. Extracellular signals or some other biochemical interactions with intracellular environment may induce some noise. The second comes from the very nature of the biochemical process itself, mainly due to the smallness of the volume and the molecular numbers involved. The former is called extrinsic ﬂuctuation and the latter intrinsic. We are mainly concerned with the intrinsic ﬂuctuations, rather than extrinsic ones. Several methods have been used to deal with the stochasticity of the chemical reactions. The most orthodox one is to use the master equation which describes the time evolution of the probability P (x, t) of the population vector x at time t [10] based on its Markovian property. Let aj denote the j-th column of the stoichiometric matrix A. Note that the i-th element aij of aj represents the number of created molecules of the i-th specy in the j-th reaction. Then, by considering the probability ﬂow, we have the following relations: M

wj (x − aj )P (x − aj , t)Δt

P (x, t + Δt) = j=1

M

wj (x)Δt P (x, t).

+ 1− j=1

Taking the limit Δt in the above relation yields

228

H. Kimura

dP (x, t) = dt

M

M

wj (x)P (x, t)

wj (x − aj )P (x − aj , t) −

(2)

j=1

j=1

The diﬀerential equation (2) is called master equation. Master equation appears also in other ﬁelds (e.g., quantum mechanics). To distinguish (2) from other master equations, it is called chemical master equation (CME). The relation between the mass action dynamics (1) and its stochastic version (2) is not direct. The average xi (t) of xi (t) deﬁned as xi (t) =

··· x1

x2

xi P (x, t)

(3)

xK

does not follow the mass action dynamics (1), except the case where w(x) is linear. However, if the deterministic equation (1) has the unique attractor, the solution of (1) is a good approximation of the average and the stochasticity is represented as the ﬂuctuation taking place arround the solution of (1) [20]. However, if the deterministic equation (1) has multiple attractors, the situation is far from simple. In that case, it is natural to think that the probability distribution has the multiple peaks. In that case, the average (3) loses its connection to the solution of the deterministic version (1), and the stochasticity will bring a new feature. We are specially interested in the long-run behaviors of the solution of the master equation (2) and its relation to the deterministic equation (1). There are a few cases where the master equation (1) is solvable analytically [16], or approximations of its solution have analytic representations [17]. For general cases, however, it is diﬃcult to obtain or approximate the solution of (2) analytically. To solve (1) numerically has been considered to be intractable, because the equation (2) is inﬁnite dimensional. Gillespie introduced a stochastic algorithm of computing time evolution of the population vector x(t) based on Monte-Carlo method [7]. Gillespie’s algorithm has been the only method of computing the stochastic behavior of chemical reactions, though it is also time-consuming. The population vector takes the combination of K natural numbers, i.e., x(t) ∈ N K , where N denotes the set of natural numbers. If we ﬁx a rule to map N K to N , we can number all the possible state vector in one-dimensional order x0 , x1 , x2 , · · · . An example of such a rule for the case K = 2 is as follows: x0 = (0, 0), x1 = (1, 0), x2 = (0, 1) x3 = (1, 1), x4 = (2, 0), x5 = (0, 2) x6 = (2, 1), x7 = (1, 2), x8 = (2, 2) ········· According to each enumeration rule, we can assign the probability pi = P (xi ), i = 0, 1, · · · , and we denote by p = (p0 , p1 , · · · , pn , · · · )T the prob-

Intrinsic Uncertainty in Gene Regulation Network

229

ability vector. Then, the master equation (1) is represented as a linear timeinvariant diﬀerential equation dp = Lp, dt

(4)

where L is an inﬁnite matrix whose elements are computed from the original master equation (2). The matrix L is stochastic in the sense that all the elements of each column of L add to zero and all the elements except the diagonal elements are non-negative. Thus, 1T L = 0 lij ≥ 0,

(5) i = j,

(6)

where 1 denotes the inﬁnite dimensional vector whose elements are all unity, i.e., T (7) 1 = 1 1 ··· 1 ··· . The equation (4) is simple but inﬁnite dimensional. If we can truncate p(t) to a ﬁnite dimensional vector, we can solve the ﬁnite-dimensional truncated equation quite easily. Actually, from the nature of the problem, the probability of large population number will decrease in usual cases. In that case, we can a priori assume that every specy has the upper bound of its population beyond which the probability is zero. More concretely, we assume that there are integers N1 , N2 , · · · , NK such that p(x) = 0, if xi > Ni for some i. (8) It is called ﬁnite probability assumption. The ﬁnite probability assumption is justiﬁed if the deterministic version (1) is stable. In that case, the solution subject to the assumption (8) will give a good approximation to the solution of the original master equation (2). Due to the assumption (8), we can eliminate all the components of the population vector corresponding xi > Ni for some i and retain only the components corresponding to xi ≤ Ni for each i. Then, we obtain the truncated population vector xf of dimension N = N1 N2 · · · NK . Writing pf = P (xf ), the truncated master equation is now represented as dpf = L f pf , dt

(9)

where Lf is a part of the original L in (4) corresponding to pf in p. It is important to notice that Lf in (9) is not a stochastic matrix in the sense that it does not satisfy (5), because some of the oﬀ-diagonal elements of L is excluded in Lf so that 1Tf Lf ≤ 0, (10) which implies that each component of 1Tf Lf is non-negative. Here, 1f is a truncated version of 1, i.e., 1f = 1 1 · · · 1

T

.

230

H. Kimura

In order to make Lf stochastic, we must assume that if any specy reaches at the upper bound, then no reaction occurs which increases the specy. This is equivalent to assume wj (x) = 0, if xi = Ni and aji > 0 for some i.

(11)

Under this additional constraint, it is shown that Lf becomes stochastic, i.e., 1Tf Lf = 0.

(12)

3 Toggle Switch Now, we discuss a gene regulatory system which acts as a biological switch depicted in Figure 1. This system was constructed artiﬁcially in a cell of Esherichia coli as a synthetic addressable memory unit of the living organism [5]. It consists of two proteins which are called repressors 1 and 2 in Figure 1, respectively. They have their promotors (the areas of the gene that control the start of the transcription) adjacent to each other. Each repressor inhibits the transcription (and thus the expression) of the other. Thus, this regulatory system forms bilateral inhibition network (double negative feedback loops) which is common in biological networks.

promotor 2

repressor 1 promotor 1

repressor 2

Fig. 1. Biological Switch

The system is composed of the following four reactions: α

(13)

δ

(14)

φ −→ R2

β

(15)

R2 −→ φ2

(16)

φ −→ R1 R1 −→ φ

where Ri denotes the repressor i (i = 1, 2), and φ denotes void. The relation (13) represents the synthesis of the repressor 1 with rate α, while (14) represents its degradation with rate δ. Equations (15) and (16) have the same meaning for the repressor 2. The synthesis rates α and β depend crucially on the rate of transcriptions, and as we assumed in Figure1, they depend on the

Intrinsic Uncertainty in Gene Regulation Network

231

amount of the other repressor already synthesized. Let x and y be the numbers of molecules of the repressors 1 and 2, respectively. Since synthesis of the repressor 1 is inhibited by the repressor 2, synthesis rate α is a decreasing function of y. In enzymology, this function is of the form α(y) =

α , 1 + yρ

(17)

where α denotes the maximum synthesis rate and ρ the cooperatively index. Likewise, β is of the form β β(x) = . (18) 1 + xµ The functions (17) and (18) play the role of propensity functions in the stochastic analysis of the system. Now, the equation (1) is represented in this system as α − δx 1 + yρ β − y y˙ = 1 + xµ

x˙ =

(19) (20)

The system (19)(20) was fully analysed in [5], and it was shown that for some parameter values, the system (19)(20) has two stable equilibrium points, each of which synthesis of one repressor dominates the other. They showed that the outer stimuli can transform the state from one equilibrium point to the outer, thus the system (19)(20) can act as a toggle switch which can memorize the state before the stimulus comes.

4 CME for Toggle Switch Now, we deal with stochastic version of the system. We ﬁrst assume that during the inﬁnitesimal time interval, only one molecular is synthesized or degradated of each repressor. In other words, only the transitions x −→ x ± 1 y −→ y ± 1 occur in the inﬁnitesimal time interval. Under this assumption, we can write the master equation for the switch as dP (x, y; t) = α(y)(x − 1, y; t) + β(x)P (x, y − 1; t) dt + δ · (x + 1)P (x + 1, y; t) + · (y + 1)P (x, y + 1; t) − α(y) + β(x) + δx + y P (x, y; t)

(21)

232

H. Kimura

Now we rewrite the master equation in a more tractable form based on the probability matrix   .. P (0, 0) P (0, 1) · · · P (0, m) · · · .      P (1, 0) P (1, 1) · · · P (1, m) · · · ...     .  .. P (t) =  (22) ······ ,   .  P (n, 0) P (n, 1) · · · P (n, m) · · · ..   .. .. .. . . . where P (n, m) = P rob. x(t) = n, y(t) = m . Introduce the following inﬁnite matrices U, V, U1 , V1 and Σ1 as follows:   −α(i − 1) − (i − 1) , i = j (U )ij = (i − 1) , (23) i=j+1  0 otherwise   −β(i − 1) − (i − 1)δ, i = j (V )ij = (i − 1)δ, (24) j =i+1  0 otherwise U1 = diag [α(0), α(1), α(2), · · · ]

(25)

V1 = diag [β(0), β(1), β(2), · · · ]

(26)

To help the visual conception, we write down the explicit forms of U and V as   −α(0) 0 0 ···  −α(1) − 0 · · ·   U = 0 (27) 2 −α(2) − 2 · · ·   .. .. .. . . .   −β(0) δ 0 ···  0 −β(1) − δ 2δ · · ·   . (28) V = 0 0 −β(2) − 2δ · · ·   .. .. .. . . . It is not diﬃcult to see that the master equation (21) can be represented as dP (t) = P (t)U + V P (t) + Σ1 P (t)U1 + V1 P (t)Σ1T , dt where Σ1 is the operator of column shift, i.e.,

(29)

Intrinsic Uncertainty in Gene Regulation Network



0 0  1 0  Σ1 =  0 1  .. .. . . It is clear that

(U + U1 )1 = 0,

233



··· ···  . ··· 

(30)

···

1T (V + V1 ) = 0,

(31)

where 1 is the vector deﬁned in (7).

5 Truncation and Reduction to Standard Form Now, we consider the truncation of the master equation under the assumption P (x, y, t) = 0, if either x > N or y > N,

(32)

where N is chosen suﬃciently large to be beyond the upper bound of population numbers of the repressors. Since all the solutions of the deterministic equation (7) is assured to be bounded, the assumption (32) is considered to hold. It is important to note that the assumption (32) implies that the transitions x(t) = N −→ x(t + Δt) = N + 1 y(t) = N −→ y(t + Δt) = N + 1 cannot occur. In order to meet this condition, the master equation (21) should be rewritten as dP (x, y; t) = α(y)P (x − 1, y; t) + β(x)P (x, y − 1; t) dt + δ · (x + 1)P (x + 1, y; t) + · (y + 1)P (x, y + 1; t) − α1 (x, y) + β1 (x, y) + δx + y P (x, y; t),

(33)

where α1 (x, y) =

α(y) if 0 ≤ x ≤ N − 1 0 if x ≥ N

(34)

β1 (x, y) =

β(x) if 0 ≤ y ≤ N − 1 0 if y ≥ N.

(35)

This modiﬁcation corresponds to the assumption (11). Under this truncation, the equation (29) is rewritten as dP N (t) =P N (t)U N + V N P N (t) + Σ1N P N (t)U1N + V1N P N (t)(Σ1N )T dt + eN eTN P N (t)U1N + V1N P N (t)eN eTN , (36)

234

H. Kimura

where the superscript N denotes the truncations of matrices (27)(28)(25)(26) and eN is the (N + 1)- dimensional vector whose elements are all zero except the last one, i.e., T eN = 0 0 · · · 0 1 . The last two terms in the right-hand side of (36) corresponds to the modiﬁcation (34)(35). We can further rewrite (36) as (“t”is omitted) dP N = P N U N + V N P N + Σ N P N U1N + V1N P N (Σ N )T , dt where



Σ N = Σ1N + eN eTN

0 0 1 0   0 1 = 0 0   ··· 0 0

··· ··· ··· ··· ··· ···

0 0 0 0 ··· 1

 0 0   0 . 0   1

(37)

(38)

The relations (31) hold in the truncated version: (U N + U1N )1N = 0,

(1N )T (V N + V1N ) = 0.

(39)

Now, we rewrite (37) in the form (9). To do so, we introduce the Kronecker product ⊗ between matrices A and B as 

a11 B  a21 B A⊗B = 

a12 B a22 B ··· am1 B am2 B

 · · · a1n B · · · a2n B  .  ··· · · · amn B

(40)

The size of the matrix A ⊗ B is mr × np if A is m × n and B r × p. Now, we introduce the vectorization Pˆ of a matrix P = (pij ) as Pˆ = p11 p21 · · · pn1 p12 · · · pn2 · · · pnm

T

(41)

Then, it is easily seen that AP = (I ⊗ A)Pˆ

(42)

P B = (B T ⊗ I)Pˆ

(43)

Let us write the vectorization of the probability matrix P N as pN := Pˆ N Then, repeated use of (42) and (43) in the vectorization of (37) yields

(44)

Intrinsic Uncertainty in Gene Regulation Network

dpN = ((U N )T ⊗ I)pN + I ⊗ V N pN dt T + U1N ⊗ (Σ N ) pN + Σ N ⊗ V1N pN

235

(45)

N

= LN p , where LN = (U N )T ⊗ I + I ⊗ V N + (U1N )T ⊗ Σ N + Σ N ⊗ V1N .

(46)

Here, we used an obvious identity (A ⊗ B)(C ⊗ D) = AC ⊗ BD subject to the size consistency. Let 1N be the N-dimensional vector with all elements being 1, i.e., 1N = T 1 1 · · · 1 . Then, from (31) and the relation Σ N 1N = 1N , we have (1N )T ⊗ (1N )T LN = 0

(47)

Here, we used the identity f1 ⊗ g + f2 ⊗ g = (f1 + f2 ) ⊗ g. The implies that LN deﬁned by (46) is a stochastic matrix. Now, we can write down an explicit form of LN , which can be seen an array of N × N cells, each of which is an N × N matrix. Let Lij denote the (i, j) cell, i.e.,   L11 L12 · · · L1N  L21 L22 · · · L2N   , Lij ∈ RN ×N . LN =  (48)   ··· ··· LN 1 LN 2 · · · LN N Then, Lij is calculated to be  (1) uii I + ui Σ N + V N ,  uji I Lij = V1N  0,

i=j j =i+1 i=j+1 otherwise.

(49)

(1)

where uij denotes the (i, j)-element of U N and ui the i-th diagonal element of U1N . Thus, LN is a block tridiagonal matrix. From the structure of Σ N and V N , the diagonal blocks Lii are tridiagonal with only the diagonal elements being negative.

6 Irreducibility The time evolution of the truncated probability distribution pN follows the linear diﬀerential equation (45), i.e., dpN = L N pN , dt

(50)

236

H. Kimura

where LN is given by (48) and (49). We are interested in the convergence of pN (t) to a stationary distribution pN ∞ as t goes to inﬁnity. According to the stochastic time-evolution theory [10], the diﬀerential equation (50) has the unique stationary solution pN ∞ satisfying L N pN ∞ = 0,

(1N )T pN ∞ = 1

and any solution of (50) tends to pN ∞ as t goes to inﬁnity, irrespective of initial distribution pN (0), provided that LN is irreducible, i.e., LN cannot be represented in a form LN =

A B , 0 C

A, C; square matrices

(51)

through permutations of rows and columns. Now, we shall prove that LN given by (48)(49) is really irreducible. The following lemma is crucial for our purpose. Lemma 1. The matrix LN given in the block form (48) is irreducible, if each diagonal block Lii is irreducible and each super-diagonal and sub-diagonal blocks Li i+1 and Li+1 i are non-zero. Proof . From the deﬁnition of a reducible matrix (51), LN is reducible if there exists an index set I such that lik = 0,

¯ i ∈ I, k ∈ I.

(52)

¯ Lii If there is a diagonal block Lii such that its columns are split in I and I, itself is obviously reducible. Hence, each column of Lii belongs to either I or I¯ for any i. Assume, contrary to the assumption, that LN is reducible. Then, there exists a index set I such that Lik = 0,

i ∈ I, k ∈ I¯

It is obvious that there exists a boundary of I and I¯ such that i ∈ I and ¯ This obviously implies that either Li i+1 = 0 or i + 1 ∈ I¯ or i + 1 ∈ I, i ∈ I. Li+1 i = 0 holds, which contradicts the assumption. In view of Lemma 1, we can establish the irreducibility of LN , if we show that Lii given in (49) are irreducible. From (49) and the forms of Σ N and V N , Lii is tri-diagonal given by  uii , k=j  (1) k =j+1 ui (Lii )kj = (k − 1)δ j = k + 1  0 otherwise

Intrinsic Uncertainty in Gene Regulation Network

237

The assertion follows immediately, if we can show that any tri-diagonal matrix is irreducible, provided that all of its super-diagonal and sub-diagonal elements are non-zero. The assertion is veriﬁed using the same argument as the proof of Lemma 1 with Ljk being replaced by (Lii )jk . Thus, we have establised the following result: Theorem 1. The truncated CME (36) has the unique stationary probability distribution p∞ (x, y), to which every solution of (21) converges irrespective of the initial probability distribution p(x, y; 0).

7 Simulation Results We have performed numerical simulations of (21). Figures 2-4 shows the simulation results. The system parameters are selected such that the deterministic dynamics (19)(20) has two stable equillibrium points. Actually, without loss of generality, parameters are chosen symmetrically, i.e., α = β, δ = , ρ = µ, in (19)(20). The upper bound N of molecular numbers are chosen to be N = 200, 400, 600. We used Euler method to solve the equation (50). Figure 2 shows a simulation result for the case where the stable equillibrium points are (x, y) = (40, 7) and (x, y) = (7, 40). The initial state is chosen to be x(0) = 60, y(0) = 30. Initially, the peak probability gradually moves to (40, 7) which is the attractor for the initial condition. However, as time proceeds, another peak gradually grows which corresponds to the other stable equillibrium point. Finally, approximately at 500,000 sec from the initial time, the probability distribution forms two peaks of almost equal height. These peaks are generated almost at the points (x, y) = (7, 40) and (x, y) = (40, 7) as was expected. Figure 3 demonstrates another simulation result for the same system with a diﬀerent initial condition. In this case, the peak probability distribution approaches to the other equillibrium point (7, 40), then another peak gradually grows and ﬁnally the probability distribution tends to the same probability distribution as in Figure 2. Figure 4 illustrates another simulation result where the initial condition is at (5, 10) which has almost equal distances to the two stable equillibrium points. In this case, the transient behavior is diﬀerent from those shown in Figures 2 and 3, but eventually, the probability distribution tends to the same one as in Figures 2 and 3. These results are perfectly consistent with the theoretical results we have shown. In order to check the validity of the ﬁnite probability assumption we have performed simulation choosing N = 600. The result is totally identical to the case N = 200. From this observation, we can conclude that the ﬁnite probability assumption is justiﬁed at least in this case.

238

H. Kimura

25sec Start at (60, 30) 250sec The fluctuation grows 2500sec The peak is

formed around as stable equilibrium

6000sec (70, 4) Stable state is kept

15000sec Gradual flow out the probability

Equillibrium Points are (40, 7) and (7, 40) Initial State (60, 30)

50000sec Gradual increase 100000sec The formation 30000sec No changes after the beginning of probability of the probability at another of small peak at another EP EP flow out

250000sec Gradual growth of peak at another EP

500000sec The stationary distribution

Fig. 2. Snapshots of probability distributions for the initial condition (60,30)

Intrinsic Uncertainty in Gene Regulation Network

100sec (50, 75) starting at

2500sec

15000sec

100000sec

500sec

100sec

5000sec

7500sec

30000sec

200000sec

239

50000sec

500000sec

The final distribution does not depends on the initial state! Fig. 3. Snapshots of probability distributions for the initial condition (60,70)

240

H. Kimura

50sec

2000sec

300sec

4000sec

1000sec

6000sec sudden Growth of Fluctuation

Initial State (5, 10)

10000sec

15000sec

100000sec

250000sec

30000sec

Fig. 4. Snapshots of probability distributions for the initial condition (5,10)

8 Conclusion A gap between the deterministic and the stochastic frameworks has been demonstrated. It is reduced to the fact that the master equation which is a representative of the stochastic framework is almost always monostable, while the deterministic dynamics may not be monostable. In that case, the two frameworks might lead to diﬀerent conclusions which are contradictory to each other in the limit of the time evolution. Since bio-chemical processes taking place inside cells are essentially stochastic, this gap may create serious problems.

Intrinsic Uncertainty in Gene Regulation Network

241

We take the genetic toggle switch as a typical example of such cases. In deterministic description, its dynamics is bistable, i.e., it has two stable equillibrium points. In stochastic formulation, however, dynamics is mono-stable, i.e., all the probabilistic states tend to a unique stationary distribution, irrespective of the initial distribution. We have demonstrated the unique convergence of the master equation theoretically and numerically by showing that the dynamics corresponding to the genetic toggle switch is irreducible through the conversion of a two dimensional stochastic array to one dimensional distribution. Through this result, we have shown that the genetic toggle switch does not work eventually. This suggests that the long-run breakdown of biological function due to stochasticity is a reason for the ﬁniteness of the life. Since our method is universal, it is applicable to many other problems. It is an interesting problem to see how the life manages to solve the intrinsic uncertainty to properly execute the normal functions. Finally, we have actually validated theoretical results experimentally for the engineered toggle switch formed by Gardners et. al. [5]. The results are discussed in a diﬀerent paper [21].

References 1. A. Becskei, B. Ser´ aphin and L. Serrano, Positive feedback in eukaryotic gene network: cell diﬀerentiation by graded to lineary response conversion,” The EMBO J., vol. 20, pp. 2528-2535, 2001. 2. A. Becskei and L. Serrano, Engineering stability in gene networks by autoregulation,” Nature, vol. 405, pp. 590-593, 2000. 3. W.J. Blake, M. Kaern, C.R. Cantor and J.J. Collins, Noise in enkaryotic gene expression,” Nature, vol. 422, pp. 633-637, 2003. 4. M.B. Elowitz and S. Leibler, A synthetic oscillatory network of transcriptional regulators,” Nature, vol. 403, pp. 335-338, 2000. 5. T.S. Gardner, C.R. Cantor and J.J. Collins, Construction of a genetic toggle switch in Esherichia coli,” Nature, vol. 403, pp. 339-342, 2000. 6. D.T. Gillespie, Approximate accelerated stochastic simulation of chemically reacting systems,” J. of Chem. Phys., vol. 115, pp. 1716-1733, 2001. 7. D.T. Gillespie, Exact stochastic simulation of coupled chemical reactions,” J. of Chem. Phys., vol. 81, pp. 2341-2361, 1977. 8. B. Houchmandzadeh et al., Establishment of developmental precision and proportions in the early Drosophila embryo,” Nature, vol. 415, pp. 798-802, 2002. 9. F. Jacob and J. Monod, Genetic and regulatory mechanisms in the synthesis of proteins,” J. of Molecular Biology, vol. 3, pp. 318-356, 1961. 10. N.G. Van Kampen, Stochastic Processes in Physics and Chemistry, Elsevier, 1992. 11. T.B. Keller and T.C. Elston, Stochasticity in transcriptional regulation: Origin, consequences, and mathematical representations, ” Biophysics J., vol. 81, pp. 3116-3136, 2001.

242

H. Kimura

12. J. Monod and F. Jacob, General conclusions: Teleonomic mechanisms in cellular metabolism, growth and diﬀerentiation,” Cold Spring Harbor Symp. Quant. Biology, vol. 26, pp. 389-401, 1962. 13. N. Le Nov´ere and T.S. Shimizu, STOCHSIM: modelling of stochastic biomolecular processes,” Bioinformatics, vol. 17, pp. 575-576, 2001. 14. E.M. Ozubdak et al., Regulation of noise in the expression of a single gene,” Nature Genet., vol. 31, pp. 69-73, 2002. 15. C.V. Rao, D.M. Wolf and A.P. Arkin, Control, exploitation and tolerance of intracellular noise,” Nature, vol. 420, pp. 231-237, 2002. 16. J. Ross, K.L.C. Hunt and P.M. Hunt, Thermodynamics far from equilibrium: Reactions with multiple stationary states,” J. Chem. Phys., vol. 88, pp. 27192728, 1988. 17. J. Ross, K.L.C. Hunt and P.M. Hunt, Thermodynamic and stochastic theory for nonequilibrium systems with multiple reactive intermediates: The concept and role of excess work,” J. Chem. Phys., vol. 96, pp. 618-629, 1992. 18. P. Smolen, D.A. Boxter and J.H. Byrne, Modern transcriptional control in gene networks—methods, recent results, and future directions,” Bull. Mathematical Biology, vol. 62, pp. 247-292, 2000. 19. Y. Tao, Intrinsic and external noise in an auto-regulatory genetic network,” J. Theoretical Biology, vol. 229, pp. 147-156, 2004. 20. R. Tomioka, H. Kimura, T.J. Kobayashi and K. Aihara, Multivariable analysis of noise in genetic regulatory networks,” J. Theor. Biology, vol. 229, pp. 501521, 2004. 21. H. Tozaki, R. Yamamoto, H. Okano, K. Aihara and H. Kimura, A long-run eﬀect of noise in genetic toggle switch,” submitted for publication. 22. Q. Yang, P.A. Lindahl and J.J. Morgan, Dynamic responses of protein homeostatic regulatory mechanisms to perturbations from steady state,” J. Theor. Biology, vol. 222, pp. 407-423, 2003.

An Approximate Dynamic Programming Approach to Decentralized Control of Stochastic Systems Randy Cogill1 , Michael Rotkowitz2 , Benjamin Van Roy3 , and Sanjay Lall4 1 2 3

4

Department of Electrical Engineering, Stanford University. [email protected] Department of Aeronautics and Astronautics, Stanford University. [email protected] Departments of Management Science and Engineering and Electrical Engineering, Stanford University. [email protected] Department of Aeronautics and Astronautics, Stanford University. [email protected]

Summary. We consider the problem of computing decentralized control policies for stochastic systems with ﬁnite state and action spaces. Synthesis of optimal decentralized policies for such problems is known to be NP-hard [1]. Here we focus on methods for eﬃciently computing meaningful suboptimal decentralized control policies. The algorithms we present here are based on approximation of optimal Qfunctions. We show that the performance loss associated with choosing decentralized policies with respect to an approximate Q-function is related to the approximation error.

1 Introduction The study of decentralized control is motivated by the fact that, in many practical control applications, control decisions must be made using incomplete information about the system state. In particular, it is common that multiple interacting decision makers must make control decisions at each time period, and each decision must be made using a diﬀerent limited observation of the overall system state. The complicating factor in such problems is that the impact of individual decisions is system-wide. We consider the problem of determining decentralized control policies for stochastic systems. Decentralized control problems have been well studied over the past several decades [2], and it is generally recognized that decentralized problems are often considerably more complex than their centralized counterparts. For example, it has been shown in [1] that computing optimal

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 243–256, 2006. © Springer-Verlag Berlin Heidelberg 2006

244

R. Cogill et al.

policies for a simple single stage control problem is NP-hard, whereas the corresponding centralized problem is trivial. This single-stage problem arises as a special case of the problems considered here. With this in mind, our focus is on eﬃcient computation of meaningful suboptimal decentralized control policies. In stochastic control problems, the relevant information required for decision making can be captured by a Q-function. When a Q-function has special structure, we can easily compute a decentralized policy which is greedy with respect to this function. The approach taken here can be interpreted as approximating the optimal Q-function for a problem by one with this special structure. We show that the performance loss associated with choosing policies with respect to an approximate Q-function is related to the approximation error. A related approach for computing suboptimal policies for specially structured multi-agent control problems can be found in [3]. The problems considered there have transition probabilities modeled as a dynamic Bayesian network and structured single-stage costs. The authors show that by using specially structured approximations to the cost-to-go function, the required computations can be simpliﬁed. The approach discussed here resembles a method commonly used to synthesize decentralized control policies for linear time-invariant systems [4]. Semideﬁnite programming conditions can be formulated for these problems which produce stabilizing decentralized control laws when they are satisﬁed. These conditions involve a simultaneous search for a stabilizing control law and a Lyapunov function proving stability. The Lyapunov functions used are restricted to have special structure, where the structure is chosen to facilitate the search for a decentralized control law.

2 Dynamic Programming Background Here we consider discrete-time stochastic control problems. The systems considered here have a ﬁnite state space X , and a ﬁnite set U of actions available at each time step. Taking action u ∈ U when in state x ∈ X incurs a cost g(x, u). After taking action u in state x, the system state in the next time period is y ∈ X with probability p(y |x, u). The goal is to ﬁnd a rule for choosing actions which minimizes some measure of the overall cost incurred. A rule for choosing actions is commonly referred to as a policy. A static state-feedback policy µ : X → U is a rule which chooses an action based on the current system state. Here we consider the problem of choosing a policy to minimize the expected total discounted cost : J µ (x) = E

∞ t=0

αt g(xt , µ(xt )) x0 = x ,

An Approximate Dynamic Programming Approach

245

where 0 ≤ α < 1. For any policy µ, we can compute the cost-to-go J µ by solving the linear equations J µ (x) = g(x, µ(x)) + α

p(y|x, µ(x))J µ (y).

(1)

y∈X

We can write these equations in matrix-vector notation as J µ = gµ + αPµT J µ . Here, gµ is the cost vector evaluated for the policy µ, Pµ is the state transition matrix associated with the policy µ, and PµT is its transpose. We can further simplify this equation using the shorthand J µ = Tµ J µ , where the operator Tµ maps a vector J to the vector gµ + αPµT J. This notation will be convenient in later sections. For the expected total discounted cost criterion, there is a single policy which minimizes the cost-to-go for all initial states. Also, the minimum costto-go J ∗ is unique and satisﬁes Bellman’s equation   J ∗ (x) = min g(x, u) + α u∈U

p(y|x, u)J ∗ (y) .

(2)

y∈X

We can simplify notation by writing this equation as J ∗ = T J ∗ , where the operator T maps a vector J to the vector T J deﬁned as   (T J)(x) = min g(x, u) + α u∈U

p(y|x, u)J(y) . y∈X

It is often useful to work directly with the expression in the right hand side of Bellman’s equation. This is often referred to as a Q-function, and the optimal Q-function Q∗ is given by Q∗ (x, u) = g(x, u) + α

p(y|x, u)J ∗ (y). y∈X

The optimal policy µ∗ is obtained from Q∗ by taking µ∗ (x) = argminu Q∗ (x, u). Like cost-to-go functions, we can compute the Q-function associated with a particular policy µ by solving the linear equations Qµ (x, u) = g(x, u) + α

p(y|x, u)Qµ (y, µ(y)). y∈X

We can introduce an operator Fµ and write the equations above simply as Qµ = Fµ Qµ . Also, Bellman’s equation can be expressed directly in terms of Q functions as Q∗ (x, u) = g(x, u) + α

p(y|x, u) min Q∗ (y, w) , y∈X

w

(3)

246

R. Cogill et al.

with Q∗ being the unique solution. As before, we can express this equation in the notation Q∗ = F Q∗ , where the operator F maps a vector Q into a vector F Q satisfying p(y|x, u) min Q(y, w)

(F Q)(x, u) = g(x, u) + α y∈X

w

Q-functions will play an important role in the upcoming material. We will conclude this section by mentioning that the operators Tµ , T , Fµ , and F have some important properties [5]. The following properties, presented only in terms of F for convenience, hold for Tµ , T , Fµ , and F : • •

Contractiveness: For any Q1 and Q2 , F Q1 − F Q2 ∞ ≤ α Q1 − Q2 ∞ . As a result of the contraction mapping theorem, F has a unique ﬁxed point Q∗ which is obtained as the limit of the sequence {F n Q} for any Q. Monotonicity: For any Q1 and Q2 , if Q1 ≤ Q2 , then F Q1 ≤ F Q2 .

These properties will be used in the proofs of some upcoming theorems.

3 Computing Decentralized Policies The problem of computing decentralized policies is considerably more complex than the corresponding problem of computing centralized policies. For example, we can consider the simple special case where α = 0. In this case, the optimal centralized policy is simply µ∗ (x) = argminu g(x, u). On the other hand, we can consider a simple decentralized variant of this problem where the state is described by two state variables x1 and x2 and each action is described by two decision variables u1 and u2 . The desired decentralized policy µ chooses u1 based only on x1 and chooses u2 based only on x2 . Unlike the centralized problem, there may not be one decentralized policy which achieves lower cost than all other decentralized policies at all states. Therefore, the problem of minimizing expected cost may not be well deﬁned until we specify an initial probability distribution on the states. Once an initial distribution is speciﬁed, the resulting optimization problem is NP-hard [1]. This example highlights the importance of eﬃcient algorithms for computing meaningful suboptimal policies for decentralized control problems. In the systems we consider, the action taken at each time period can be described by a collection of decision variables as u = (u1 , . . . , uk ). The action space for this system is U = U1 × · · · × Uk . Likewise, the state of the system is described by a collection of state variables and the corresponding state space is X = X1 ×· · ·×Xp . In general, the policies produced by dynamic programming determine each action ui based on the entire state x = (x1 , . . . , xp ). However, we are interested in computing policies where each decision is only based on a subset of the state variables. A decentralized policy is a policy in which each action ui depends only on some speciﬁed subset of the state variables x1 , . . . , xp .

An Approximate Dynamic Programming Approach

247

The required dependencies of actions on state variables is typically referred to as the information structure of the desired policy. We will use the following notation to describe the information structure of a particular policy. The set of variables which may be used to decide action i is denoted as Ii = {j | ui depends on xj }. In particular, Ii gives the indices of the state variables that action i depends on. The Cartesian product of the spaces of the variables used to decide action i is denoted as Xj . Xi =

×

j∈Ii

Using this notation, a decentralized policy is a set of functions µi : Xi → Ui for i = 1, . . . , k. Recall that the optimal centralized policy is µ∗ (x) = argminu Q∗ (x, u), where Q∗ is obtained as the unique solution to Bellman’s equation. Consider a k policy µ(x) = argminu Q(x, u), where Q is a function of the form Q = i=1 Qi and each Qi : Xi × Ui → R. Since ui is the only decision variable appearing in the argument of Qi , choosing u to be argminu Q(x, u) is equivalent to choosing each ui to be argminui Qi (x, ui ). Also, since the only state variables appearing in the argument of Qi are those in Xi , the choice of ui only depends on the values of the variables in Xi . Therefore, the policy µ is decentralized since each decision ui is determined only in terms of the state variables in Xi . This suggests an algorithm, in which an appropriately structured Q is chosen to approximate Q∗ , and a decentralized policy is obtained from Q. Such an algorithm will be presented in the next section. Theorem 1 justiﬁes this approach by showing that the suboptimality of the performance achieved by µ is related to the approximation error between Q and Q∗ . Before presenting Theorem 1, we will introduce some notation. Let Qµ denote the function Q evaluated for the policy µ. That is, Qµ (x) = Q(x, µ(x)). We must be careful to make the distinction between Qµ : X → R, the function Q evaluated for the policy µ, and Qµ : X × U → R, the Q-function associated with following the policy µ from time t = 1 onward. Also, let · 1,ν denote the weighted 1-norm, deﬁned as J

1,ν

=

ν(x)|J(x)|, x

where the weights ν are positive. For the upcoming theorem, we will restrict ourselves, without loss of generality, to weights which satisfy x ν(x) = 1. Theorem 1. Suppose Q satisﬁes Q ≤ F Q. If µ is the policy deﬁned as µ(x) = argminu Q(x, u), then the following bound holds Jµ − J∗

1,ν

≤

1 Q∗ − Qµ 1−α µ

1,ω

,

248

R. Cogill et al.

where ν is an arbitrary probability distribution on X and ω is the probability distribution deﬁned as ω = (1 − α)(I − αPµ )−1 ν. Proof. First we will show that ω is a probability distribution. Let e denote the vector with 1 for each entry. Since since eT ν = 1 and eT Pµt = eT for all t, eT ω = (1 − α)eT (I − αPµ )−1 ν ∞

= (1 − α) t=0 ∞

= (1 − α)

αt eT Pµt ν αt eT ν

t=0

=1

Also, since Pµ ≥ 0 and ν ≥ 0 componentwise, ω ≥ 0. Recall that we require Q ≤ F Q. By monotonicity of F , this implies Q ≤ F n Q for all n. Since F n Q → Q∗ for any Q, this implies Q ≤ Q∗ . Also, the policy µ is greedy with respect to Q, so Qµ ≤ J ∗ ≤ J µ . Therefore, Jµ − J∗

1,ν

≤ J µ − Qµ 1,ν = ν T ((I − αPµT )−1 gµ − Qµ ) = ν T (I − αPµT )−1 (gµ − (I − αPµT )Qµ ) 1 ω T ((gµ + αPµT Qµ ) − Qµ ) = 1−α

Since µ is greedy with respect to Q, we have (F Q)µ = gµ +αPµT Qµ . Therefore, the inequality above can be expressed as Jµ − J∗

1,ν

≤

1 (F Q)µ − Qµ 1−α

1,ω

Since Qµ ≤ (F Q)µ ≤ Q∗µ , we have the result Jµ − J∗

1,ν

≤

1 Q∗ − Qµ 1−α µ

1,ω

A theorem similar to Theorem 1, based only on cost-to-go functions, can be found in [6].

4 The Linear Programming Approach It is well known that the solution to Bellman’s equation J ∗ can be obtained by solving a linear program [7]. We can determine Q∗ (x, u) = g(x, u) +

An Approximate Dynamic Programming Approach

249

α y∈X p(y|x, u)J ∗ (y) once J ∗ is known. In the previous section, we outlined a method for computing a decentralized policy based on an approximation of Q∗ . In this section, we present an algorithm for computing such an approximation. This algorithm is based on an extension of the linear programming approach to solving Bellman’s equation. The standard linear programming formulation is extended to explicitly include Q. The need for including Q will become clear shortly. Theorem 2. The solutions to the equations (2) and (3) can be obtained by solving the linear program maximize:

x∈X

u∈U

Q(x, u)

subject to: Q(x, u) ≤ g(x, u) + α

y∈X

p(y|x, u)J(y)

J(x) ≤ Q(x, u)

for all x ∈ X , u ∈ U for all x ∈ X , u ∈ U

Proof. Q∗ satisﬁes Q∗ (x, u) = g(x, u) + α

p(y|x, u) min Q∗ (y, w) . y∈X

w

Also, J ∗ (x) = minu Q∗ (x, u) ≤ Q∗ (x, u) for all x, u. Therefore, Q∗ and J ∗ are feasible for this LP. Also, since J(x) ≤ Q(x, u) for any feasible J, any feasible Q satisﬁes Q(x, u) ≤ g(x, u) + α

p(y|x, u)J(y) y∈X

≤ g(x, u) + α

p(y|x, u) min Q(y, w) , y∈X

w

or more succinctly Q ≤ F Q. By monotonicity of F , this implies Q ≤ F n Q for all n. Since F n Q → Q∗ for any Q, any feasible Q satisﬁes Q ≤ Q∗ . Therefore Q∗ is the optimal feasible solution. Note that the usual LP approach to solving Bellman’s equation only involves J, and therefore contains fewer variables and constraints. The explicit inclusion of Q in the linear program is required because we will require Q to have special structure in order to obtain a decentralized policy. In general, requiring J to have some special structure is not suﬃcient to guarantee that Q will have the required structure. Using this linear program, we have the following algorithm for computing a decentralized policy.

250

R. Cogill et al.

Algorithm 1. k

1. For any information structure, let Q = i=1 Qi and J = Qi : Xi × Ui → R and Ji : Xi → R. 2. For arbitrary positive values ω(x), solve the linear program maximize:

x∈X

u∈U

k i=1

Ji , where

ω(x)Q(x, u)

subject to: Q(x, u) ≤ g(x, u)+α

y∈X

p(y|x, u)J(y) for all x ∈ X , u ∈ U

J(x) ≤ Q(x, u)

for all x ∈ X , u ∈ U,

3. Let µ(x) = argminu Q(x, u). This policy is decentralized with the desired information structure. It is not immediately clear how the optimization problem solved in Algorithm 1 relates to the cost-to-go achieved by µ. We will show that this optimization problem is equivalent to minimizing a weighted norm of the error between Q∗ and Q subject to some constraints. From Theorem 1, this error is directly related to the performance achieved by µ. Theorem 3. The linear program in Algorithm 1 is equivalent to minimize: Q∗ − Q

1,Ω

subject to: Q ≤ F Q, where Ω(x, u) = ω(x) for all x ∈ X and u ∈ U. Proof. From the proof of Theorem 2, Q is feasible if and only if Q ≤ F Q, which implies Q ≤ Q∗ . Therefore, for any feasible Q Q − Q∗

1,Ω

ω(x)(Q∗ (x, u) − Q(x, u))

= x∈X u∈U

=−

ω(x)Q∗ (x, u).

ω(x)Q(x, u) + x∈X u∈U

x∈X u∈U

Since x∈X u∈U ω(x)Q∗ (x, u) is constant, minimizing this quantity is equivalent to maximizing x∈X u∈U ω(x)Q(x, u), the objective in the original LP. From the previous theorem, we see that Algorithm 2 seeks to minimize the approximation error Q − Q∗ 1,Ω subject to the constraint Q ≤ F Q. For any policy µ ˆ and any probability distribution ω, the bound Qµb − Q∗µb 1,ω ≤ Q − Q∗

1,Ω

holds. This, together with Theorem 1, assures us that Algorithm

An Approximate Dynamic Programming Approach

251

1 attempts to produce a decentralized policy µ with performance close to that of an optimal centralized policy. When no structure is imposed on Q (i.e., the centralized case), the weights ω in the objective function are unimportant since we obtain Q∗ for any positive weights. However, when Q has structure, the weights chosen will aﬀect the quality of the policy obtained. A good choice of weights may be guided by the interpretation of these weights in Theorem 1. In particular, these weights impose a tradeoﬀ in the quality of the approximation across the states. Ideally, we would like to closely approximate states that are visited most often under any policy. The role of these weights are discussed in greater detail in [6]. For highly decentralized information structures, the number of variables in the linear program increases linearly with the number of state variables. Therefore, the number of variables appearing in this linear program is often signiﬁcantly less than the number of variables required for computation Q∗ . However, the number of constraints appearing in the linear program is still on the order of the number of state-action pairs. The application of Algorithm 1 to large-scale problems may still be prohibitive due to the large number of constraints. Similar problems are encountered in the application of the approximate linear programming methods of [6] to large problems, however constraint sampling has been shown to be an eﬀective method for overcoming this problem [8]. We believe that the analysis and application of constraint sampling as in [8] will carry over to the problems here, although we have not explored the details at this point.

5 Example Here we will apply the algorithm developed in Section 4 to an example problem. The problem considered is the following. Two robots are placed in opposite corners of a region, and calls for services to be performed by the robots originate from various points within the region. This is depicted in Figure 1. Calls for service are sent to both robots. Each robot, if idle, must make a decision of whether or not to service the current location. Each robot must make this decision without knowledge of the other robot’s status (idle or busy), resulting in a decentralized decision making problem. For this problem, it is not clear which strategy the robots should follow. If both robots, when idle, respond to all calls for service, then we will often have both robots responding to calls when only one is required to service the location. On the other hand, if we partition the set of locations and assign each location to a single robot, then locations will often go unserviced when their assigned robot is busy at another location. We will see that both of these strategies are typically outperformed by strategies which assign a single robot to some locations, and both robots to others. In our model, events occur at discrete time periods t = 0, 1, . . .. The system state has three components x = (x1 , x2 , x3 ), where the compo-

252

R. Cogill et al.

Fig. 1. Robots are stationed in the lower left and upper right corners, and respond to calls for service at the locations shown by circles.

nents are described as follows. At time t, robot i is in state xi (t) ∈ Xi = {robot i idle, robot i busy}, depending on whether or not it is idle or currently serving a location. For a problem with N locations, the call for service at time t is x3 (t) ∈ X3 = {location 1, . . . , location N, no service requested}. We make the restriction that only one call for service may be active at a time. At time t, if robot i is idle and there is an active call for service, robot i chooses an action ui (t) ∈ Ui = {respond, don’t respond}. The overall state space for this system is X = X1 × X2 × X3 . The overall action space is U = U1 × U2 . At each time period, calls for service are independent and identically distributed. Therefore, if there is no response to a call, it does not persist and is replaced by a new call (or no call) in the following time period. Let p3 (y3 ) give the probability of calls for service in each location. If a robot is busy servicing a location in any time period, it remains busy in the following period with probability 0.9. Let pi (yi |xi , ui ) give the probability that robot i is busy or idle in the following time period given its current state and action. The dynamics of this system are described by the transition probability function p(y|x, u) = p1 (y1 |x1 , u1 )p2 (y2 |x2 , u2 )p3 (y3 ). The control objective is expressed through costs that are incurred in each time period. Costs incurred for servicing a location are based on the distance required to reach the location and the time required to complete servicing. Upon responding to a location, a cost of 2(distance to location) is incurred by each robot which responds (i.e., the cost of getting there and back). For our example, the region considered is the unit square, and the distance used is the Euclidean distance from the the robot’s corner to the call location. For each time period that a robot is busy, it incurs an additional cost of 0.25 units. These servicing costs for each robot are given by a function gi (xi , ui ).

An Approximate Dynamic Programming Approach

253

Fig. 2. The decentralized policy computed in Example 1. Lighter shaded circles are served only by robot 1, darker shaded circles are served only by robot 2, and circles with both shades are served by both robots. 7380 computed respond closest respond all

7370

7360

7350

7340

7330

7320

7310

7300

0

5

10

15

20

25

30

35

40

45

Fig. 3. The total discounted cost as a function of state for the policies used in Example 1.

Additionally, if neither robot responds to a call, a large penalty is incurred. In this case, we use a penalty of 8 units. These penalties are given by a function g3 (x1 , x2 , u1 , u2 ). The overall per-period costs for this problem are given by g(x, u) = g1 (x1 , u1 ) + g2 (x2 , u2 ) + g3 (x1 , x2 , u1 , u2 ). Our goal is to determine control policies µ1 : X1 × X3 → U1 and µ2 : X2 × X3 → U2 which make the total discounted cost as small as possible (with α = 0.999).

254

R. Cogill et al.

Fig. 4. The decentralized policy computed in Example 2. Lighter shaded circles are served only by robot 1, darker shaded circles are served only by robot 2, and circles with both shades are served by both robots. 7250 computed respond closest respond all

7240 7230 7220 7210 7200 7190 7180 7170 7160 7150

0

5

10

15

20

25

30

35

40

45

Fig. 5. The total discounted cost as a function of state for the policies used in Example 2.

We applied Algorithm 1 to several instances of this problem with N = 10. The results of the ﬁrst example are shown in Figures 2 and 3. In this example, a decentralized policy is computed and compared against two heuristics, ‘respond closest’ and ‘respond all’. In the ‘respond closest’ policy, robots only respond to the locations closest to them. In the ‘respond all’ policy, each robot responds to every location. The results of the second example are shown in Figures 4 and 5. It is interesting to note the diﬀerence between the perfor-

An Approximate Dynamic Programming Approach

255

mance of the heuristic policies in the two examples. In particular, relative performance of the two heuristic policies depends on the particular problem instance. The computed policies outperform both heuristics in both examples.

6 Conclusions Here we considered the problem of designing decentralized control policies for stochastic systems. An algorithm based on linear programming was presented. This algorithm obtains a decentralized policy from a function with special structure that approximates the optimal centralized Q-function. The performance loss associated with the resulting decentralized policy was shown to be related to the approximation error.

References 1. J. Tsitsiklis and M. Athans, “On the complexity of decentralized decision making and detection problems,” IEEE Trans. Automatic Control, vol. 30, no. 5, pp. 440–446, 1985. 2. N. Sandell, P. Varayia, M. Athans, and M. Safonov, “Survey of decentralized control methods for large-scale systems,” IEEE Trans. Automatic Control, vol. 26, pp. 108–128, 1978. 3. C. Guestrin, D. Koller, and R. Parr, “Multiagent planning with factored MDPs,” Advances in Neural Information Processing Systems, vol. 14, pp. 1523–1530, 2001. 4. J. Bernussou, P. Peres, and J. Geromel, “Robust decentralized regulation: a linear programming approach,” Proc. IFAC Symposium on Large Scale Systems, vol. 1, pp. 133–136, 1989. 5. D. Bertsekas and J. Tsitsiklis, Neuro-dynamic Programming. Athena Scientiﬁc, 1996. 6. D. de Farias and B. V. Roy, “The linear programming approach to approximate dynamic programming,” Operations Research, vol. 51, no. 6, pp. 850–865, 2003. 7. A. Manne, “Linear programming and sequential decisions,” Management Science, vol. 6, no. 3, pp. 259–267, 1960. 8. D. de Farias and B. V. Roy, “On constraint sampling in the linear programming approach to approximate dynamic programming,” to appear in Mathematics of Operations Research, submitted 2001. 9. M. Aicardi, F. Davoli, and R. Minciardi, “Decentralized optimal control of Markov chains with a common past information set,” IEEE Trans. Automatic Control, vol. 32, no. 11, pp. 1028–1031, 1987. 10. R. Cogill, M. Rotkowitz, B. V. Roy, and S. Lall, “An approximate dynamic programming approach to decentralized control of stochastic systems,” Proceedings of the 2004 Allerton Conference on Communication, Control, and Computing, pp. 1040–1049, 2004. 11. Y. Ho, “Team decision theory and information structures,” Proceedings of the IEEE, vol. 68, no. 6, pp. 644–654, 1980.

256

R. Cogill et al.

12. K. Hsu and S. Marcus, “Decentralized control of ﬁnite state Markov processes,” IEEE Trans. Automatic Control, vol. 27, no. 2, pp. 426–431, 1982. 13. C. Papadimitriou and J. Tsitsiklis, “Intractable problems in control theory,” SIAM Journal on Control and Optimization, vol. 24, no. 4, pp. 639–654, 1986. 14. ——, “The complexity of Markov decision processes,” Mathematics of operations research, vol. 12, no. 3, pp. 441–450, 1987. 15. M. Puterman, Markov decision processes. John Wiley and Sons, New York, 1994. 16. R. Srikant and T. Ba¸sar, “Optimal solutions in weakly coupled multiple decision maker Markov chains with nonclassical information,” Proceedings of the IEEE Conference on Decision and Control, pp. 168–173, 1989.

An H∞ Loop-Shaping Approach to Steering Control for High-Performance Motorcycles Simos Evangelou1 , David J. N. Limebeer1 , Robin S. Sharp1 , and Malcolm C. Smith2 1

2

Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK. {s.evangelou; d.limebeer; robin.sharp}@imperial.ac.uk Department of Engineering, University of Cambridge, Cambridge CB2 1PZ, UK. [email protected]

Summary. A ﬁxed-parameter active steering compensation scheme that is designed to improve the dynamic behaviour of high-performance motorcycles is introduced. The design methodology is based on the Glover-McFarlane H∞ loop-shaping procedure. The steering compensator so designed, is seen as a possible replacement for a conventional steering damper, or as an alternative to the more recently introduced passive mechanical compensation networks. In comparison with these networks, active compensation has several potential advantages including: (i) the positive-reality of the compensator is no longer a requirement; (ii) it is no longer necessary for the device to be low-order; (iii) in a software implementation, it is easy to adjust the compensator parameters and (iv) an adaptive, or parameter varying version of this scheme is a routine extension. The study makes use of computer simulations that exploit a state-of-the-art motorcycle model whose parameter set is based on a Suzuki GSX-R1000 sports machine. The results extend further the signiﬁcant improvements achieved in the dynamic properties of the primary oscillatory modes (‘wobble’ and ‘weave’) obtained previously by replacing the conventional steering damper with passive mechanical steering compensation schemes.

1 Introduction The lightly damped modes associated with high-performance motorcycles have a limiting eﬀect on their performance. In extreme cases one or more of these modes may even be unstable. These lightly damped modes can also lead to an undesirable response to road undulations [1]. This citation refers to several accidents in which resonant forcing type phenomena played a role in their initiation. Early theoretical work on bicycle and motorcycle dynamics was conﬁned to the study of small oscillations around a straight-running trim condition [2–5]. In later work these studies, and the models on which they were based, were extended to include the analysis of small perturbations

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 257–275, 2006. © Springer-Verlag Berlin Heidelberg 2006

258

S. Evangelou et al.

from steady-state cornering conditions [6–11]. In addition to this theoretical work, motorcycle oscillations have been widely studied via measurement programs [12–27]. The main lateral oscillations in two-wheeled vehicles are ‘wobble’ and ‘weave’. In straight running, the weave mode is well damped at moderate speeds, but becomes less so as the machine’s forward speed increases. The natural frequency rises from zero at very low speed to somewhere in the range 2-4 Hz, depending on the mass and size of the machine—the lower frequencies corresponding to heavier motorcycles. The only properly documented wobble oscillations involve moderate speeds, although there are many anecdotal accounts of wobble at high speeds [28]. Theoretical results indicate that the torsional stiﬀness of the motorcycle frame at the steering head determines whether a machine will be prone to wobbling at medium speeds (compliant frame), or at high speeds (stiﬀ frame) [29, 30]. The frequency of the wobble mode is insensitive to speed variations, and is governed primarily by the mechanical trail, the front tire cornering stiﬀness and the front frame steer inertia. The wobble mode’s frequency is normally in the range 6-9 Hz. Stiﬀ framed machines, being prone to wobbling at high speed, often depend on a steering damper for satisfactory wobble-mode damping. Normally, however, a steering damper will destabilize the high-speed weave mode. In cornering, the lateral modes and the in-plane modes associated with tire deﬂections and suspension motions become coupled, as was ﬁrst shown in any detail by Koenen [6]. The motorcycle becomes prone to resonant forcing via regular road undulations when the displacement forcing they produce is tuned to the lightly damped modal frequencies of the machine. Moderate roll angles are likely to represent the worst case conditions [1]. Mechanical steering compensation networks were introduced in [31] as a possible replacement for conventional steering dampers. These networks consist of springs, dampers and inerters, a component recently introduced in [32]. The inerter made it possible to obtain a complete analogy between passive electrical and mechanical components. The force-current mechanical-electrical analogy has the following correspondences: spring↔inductor, damper↔resistor and inerter↔capacitor. The use of the inerter instead of the mass element allows electrical circuit synthesis theory to be applied to mechanical networks with equal facility. These ideas were used in conjunction with classical BodeNyquist ideas to design mechanical compensation schemes that can improve substantially the stability and performance properties of motorcycles [31]. Motivated by this success, a new question arises: “what additional improvement is possible if active control schemes are entertained?”. The primary aim of this research is to answer this question by introducing an active steering control scheme that makes use of H∞ synthesis theory. We begin our study with a review of the essential ingredients of the particular H∞ control synthesis problem that will be used in this study. In Section 3 the background to the motorcycle model is described. Some of the important characteristics of the reference motorcycle-rider system are described in Sec-

An H∞ Loop-Shaping Approach

259

tion 4. The controller design is given in Section 5.1 and the performance of the base motorcycle ﬁtted with the designed controller is evaluated in Section 5.2. Conclusions are drawn in Section 6.

2 Normalized Coprime Factor H∞ Synthesis The active control system design approach used in this paper is based on the robust stabilization of the perturbed normalized coprime factors of a linearized plant model. This method incorporates a loop shaping design procedure to obtain the requisite performance and robust stability features. The key step is the solution of an H∞ optimization problem that guarantees the robust stability of the closed-loop. The theoretical basis of this approach was developed in [33–35]. Robustness optimization for normalized coprime factor perturbations is closely related to robustness optimization in the gap metric [36–38]. 2.1 The Optimization Problem The normalized coprime factor modeling error is used to represent uncertainty in the system model. As compared with the usual additive or multiplicative uncertainty representations, the nominal and perturbed models need not have the same number of right-half-plane poles [39, 40]. In the following analysis, boldface letters are used to represent transfer functions, while normal letters and symbols represent signals. The deﬁnitions of H∞ spaces and the norm · ∞ are standard [39, 40]. The small gain theorem and coprime factor representations will also be used [39, 40]. For a given plant with transfer function G, a normalized left coprime factorization of G is given by G = M −1 N in which M , N ∈ H∞ are coprime and satisfy M M ∼ + N N ∼ = I;

(1)

the symbol ∼ denotes the conjugate transpose. The left coprime factor perturbed plant is given by GΔ = (M − ΔM )−1 (N + ΔN ) where ΔM , ΔN ∈ H∞ are unknown modeling perturbations. Assuming that δ = GΔ Ts ,

(2)

with δ and Ts representing the motorcycle’s steering angle and steering torque respectively, one obtains M δ = N Ts + φ (3)

260

S. Evangelou et al.

with φ = ΔN ΔM

Ts . δ

(4)

The inﬂuence of external disturbances is introduced by modifying (3) so that:  M δ = N Ndr Ndf

 Ts  dr  + φ df

(5)

in which dr and df are road-induced vertical displacement forcing signals applied to the rear and front wheels respectively of the vehicle. The transfer functions Gdr = M −1 Ndr and Gdf = M −1 Ndf map dr and df to the steering angle δ. In the context of the problem being studied here, the nominal plant transfer function G is obtained by linearizing a nonlinear motorcycle dynamic model. A feedback loop is closed using the controller K such that Ts = Kδ. The resulting situation is illustrated diagrammatically in Figure 1. The aim is to design a controller that attenuates road displacement forcing disturbances and stabilizes the perturbed closed-loop system. With the loop broken at φ, direct calculation shows that   φ Ts K −1 −1  (I − GK) = M Gdr Gdf dr  . I δ df Now consider the problem of ﬁnding K to minimize γ such that K (I − GK)−1 M −1 Gdr Gdf I

< γ.

(6)

∞

Assuming (6) holds, (1) implies that K (I − GK)−1 I G I

<γ

(7)

∞

and hence closed-loop stability follows from the small gain theorem whenever ΔN ΔM ∞ ≤ γ −1 [33–35]. Furthermore, if (6) holds, the closed-loop dr transfer function from θ0 δ will also satisfy · ∞ < γ. These two condf siderations motivate the minimisation of (a suitably weighted version of) (6). This will be described in the next section. 2.2 Loop Shaping Design Procedure The design procedure is divided into three stages [34, 35, 39]. In the ﬁrst, the plant transfer function G is shaped with a dynamic weighting function W

An H∞ Loop-Shaping Approach

dr

✲

Ndr

df

✲

Ndf

Ts ✲ N ✲

ΔN

❄ ✲ Σ ✲ M −1

G

261

δ

✲

✻

φ

✲ Σ✛

ΔM ✛

K ✛ Fig. 1. Closed-loop system for the normalized coprime factor model error representation.

so that the singular values of GW have the desired frequency dependence. Two other frequency shaping functions Wdr and Wdf are chosen to shape the disturbance transfer functions Gdr and Gdf . In the controller synthesis stage of the design process, G, Gdr and Gdf are replaced with their frequency weighted versions GW , Gdr Wdr and Gdf Wdf . The optimal stability margin γopt and optimal controller K∞ are computed by minimising the following cost function: K∞ (I − GW K∞ )−1 I GW Gdr Wdr Gdf Wdf I

.

(8)

∞

If the resulting value of γopt is too high, or the resulting closed-loop system does not have the desired characteristics, the loop shape W and disturbance weights Wdr and Wdf are modiﬁed and the procedure repeated until a satisfactory compromise is found. In the ﬁnal step, W is absorbed into the controller by replacing K∞ with K = W K∞ .

3 Motorcycle Model The motorcycle dynamic model used in this study has been described in detail elsewhere [7, 10, 11]. In essence, the main frame is allowed six degrees of freedom; a steering freedom is associated with the rotation of the front frame relative to the main frame; the gyroscopic inﬂuences of the spinning road wheels

262

S. Evangelou et al.

are included; front and rear suspension systems are included; the main frame has a twist freedom in the region of the steering head; aerodynamic eﬀects are represented as forces applied at the center of pressure and the rider’s upper body is allowed to roll relative to the main frame around a longitudinal axis. The forces and moments associated with the tires are modeled using ‘magic formulas’ whose parameters have been optimized to ﬁt measured rig data [41– 44]. The simulation model is written in LISP and makes use of the multibody modeling code AUTOSIMT M [45]. The model ﬁle can be obtained from the website http://www.imperial.ac.uk/controlandpower/motorcycles. The AUTOSIM [45] model can be conﬁgured as either ‘linear’ or ‘nonlinear’. In the latter case, a C++ simulation program is generated. If this code is executed, the describing equations of motion are solved using numerical integration. The simulation program can be used to ﬁnd equilibrium trim states, which typically involve a ﬁxed target lean angle and a ﬁxed target speed. In the linear conﬁguration, the system is symbolically linearized for small perturbations about a general trim condition. A MATLAB ‘M’ ﬁle that generates a linear model in state-space form is generated automatically. The parameters used in the model derive from laboratory experiments conducted on a contemporary commercially available sports machine [11], the Suzuki GSX-R1000. The eﬀects of a steering damper, or a more general steering controller can be incorporated via the diﬀerential equations that describe them. For the particular purposes of the study presented here, the steering compensation system is separated from the rest of the model in the feedback structure shown in Figure 1. The steering compensator appears as K and has the steering angle δ as its input (in [31] the steering velocity was used as the input) and steering torque Ts as output. As has been explained elsewhere [1, 7, 10], speed and steering torque controllers are used in the nonlinear model to establish trim states. The reader is reminded that these controllers are not included in any of the linearized models, or in any of the results presented here.

4 Characteristics of the Standard Machine 4.1 Standard Characteristics The important oscillatory modes associated with ‘wobble’ and ‘weave’ are illustrated in the root-locus diagrams of Figures 2 and 3. In both cases the machine’s forward speed is the swept parameter. Figure 2 corresponds to the straight-running machine with the steering damper removed. It can be seen from this diagram that the wobble-mode frequency varies between 47 and 57 rad/s, while the weave mode’s resonant frequency varies between 10 and 28 rad/s. It is clear that the damping of the wobble mode decreases with increased speed and the mode becomes unstable at approximately 25 m/s. With further speed increases, up to approximately 62 m/s, the real part of the wobble mode eigenvalue increases.

An H∞ Loop-Shaping Approach

263

60

wobble

50

Imaginary Axis

40

30

weave

20

10

0 −18

−16

−14

−12

−10

−8

−6

−4

Real Axis

−2

0

2

4

Fig. 2. Root-locus plot for the straight-running motorcycle with speed the varied parameter. No steering damper is ﬁtted. The speed is increased from 5 m/s ( ) to 85 m/s ( ). 60

50

Imaginary Axis

40

30

20

10

0 −18

−16

−14

−12

−10

−8

Real Axis

−6

−4

−2

0

2

Fig. 3. Root-locus plots for: Straight running (×), 15 deg (◦), 30 deg (+) and 45 deg (♦) of roll angle with speed the varied parameter. The nominal steering damper is ﬁtted. The speed is increased from 7 m/s ( ) to 75 m/s ( ).

264

S. Evangelou et al.

In Figure 3 the nominal steering damper is reﬁtted in the model and speed sweeps are carried out for four values of lean angle. By comparing Figures 2 and 3, it can be seen that the steering damper stabilizes and then increases the damping of the wobble mode, while reducing that of the weave mode. Figure 3 also shows that increased values of roll angle tend to increase the high-speed weave-mode damping. Since the coupling between the in-plane and out-of-plane dynamics increases with roll angle, one expects the weave mode’s vulnerability to road displacement forcing to maximize at an intermediate value (of approximately 15 deg [1]). It can also be seen from Figure 3 that for roll angles of up to 30 deg, the high-speed wobble-mode damping increases with roll angle. Further increases in roll destabilize this mode. At low speeds, the wobble-mode damping decreases monotonically with roll angle, and the vulnerability of this mode is worst at low speed and high roll angles. The open-loop linearized motorcycle model can also be used to generate the Nyquist diagrams shown in Figure 4. Frequency response plots for straight running at ﬁve diﬀerent values of forward speed are considered. In the case of a steering damper as the compensator in the feedback loop of Figure 1, K(s) = Ks, in which K is the steering damping. It follows from the Nyquist criterion [46] that closed-loop stability requires N anticlockwise encirclements of the −1/K point, where N is the number of unstable poles of the open-loop system and K is the steering damper coeﬃcient. At low speeds, both the wobble and weave modes are stable (see Figure 2) and so no encirclements of the −1/K point are required for the 15 m/s case as shown in Figure 4. At 35 m/s the

0.3

35 m/s

15 m/s

Imaginary Axis

0.2

55 m/s

0.1

75 m/s 0

85 m/s −0.1

−0.2

−0.3 −0.5

−0.4

−0.3

−0.2

−0.1

Real Axis

0

0.1

0.2

0.3

Fig. 4. Straight-running Nyquist diagrams for the open-loop motorcycle model for ˙ ﬁve diﬀerent values of forward speed. The input is Ts and the returned signal is δ.

An H∞ Loop-Shaping Approach

265

wobble mode of the nominal motorcycle, without its steering damper, becomes unstable as the corresponding complex conjugate pair of poles crosses the imaginary axis into the right-half plane. Under these operating conditions two anticlockwise encirclements of the −1/K point are therefore required. For higher speeds, two anticlockwise encirclements of the −1/K point are also required, but the range of damper values which will achieve this reduces. Indeed, if the steering damping value is too low, the wobble mode becomes unstable and if it is set too high, the weave mode becomes unstable. When the speed increases from 75 m/s to 85 m/s, the interval on the negative-real axis associated with system-stabilizing damper parameter values moves to the left. This shift will move the weave-mode crossing point towards the −1/K point associated with the nominal damper parameter value, and the wobble-mode crossing away from it. As a result, this speed increase will lead to a reduction in the weave-mode damping and an increase in that of the wobble-mode. 4.2 Characteristics of Machines Compensated by Mechanical Networks In this section we review brieﬂy results from [31] that are relevant to the study carried out here. In [31] the idea of replacing a steering damper with a passive mechanical network (comprising springs, dampers and inerters) was introduced. Various network conﬁgurations were studied and of these the best performing was a series resonant ﬁlter (denoted as N3), which consists of a spring, a damper and an inerter in series. The admittance function, Y (s), (ratio of Laplace-transformed torque and velocity) for this network takes the form: ks Y (s) = 2 s + sk/c + k/b where k, c, b are the spring, damping and inertance parameters respectively. This network can be represented in the form of the control scheme of Fig. 1 by writing K(s) = sY (s). A frequency domain optimization for this network gave the optimal parameters as k = 594.08 Nm/rad, c = 13.716 Nm/(rad/s) and b = 0.24252 kgm2. Figure 5 shows the locus of the motorcycle’s important modes for a wide range of speeds and roll angles in the case that the optimized N3 network is ﬁtted. It is clear that, when compared with Figure 3 where a damper is ﬁtted, this network achieves greatly improved damping ratios for each of the machine’s lightly damped modes. The networks used in [31] were at most second-order rational functions and were special cases of the positive-real bi-quadratic function Y (s) =

a2 s2 + a1 s + a0 d2 s2 + d1 s + d0

(9)

where the parameters a0 , a1 , a2 , d0 , d1 , d2 obey certain positive-reality constraints [47, 48]. The general bi-quadratic positive real compensator (given the

266

S. Evangelou et al. 70

60

Imaginary Axis

50

40

30

20

10

0 −18

−16

−14

−12

−10

−8

Real Axis

−6

−4

−2

0

2

Fig. 5. Root-locus plots for: Straight running (×), 15 deg (◦), 30 deg (+) and 45 deg (♦) of roll angle, with the speed varied from 7 m/s ( ) to 75 m/s ( ). The frequency-domain optimized network N3 [31] is ﬁtted.

name N8) was also tested, although no attempt was made to ﬁnd a realization for it, and it was found to perform marginally better than N3. Optimized N8 parameters were found to be a2 = 13.944, a1 = 85.172, a0 = 11.167, d2 = 1.0, d1 = 13.535 and d0 = 941.83. Figure 6 shows the locus of the motorcycle’s important modes in the case that the optimized N8 network is ﬁtted to the machine. It is again clear that this network increases the damping of the lightly damped modes of the machine when compared with the nominal machine performance illustrated in Figure 3. As a further indication of performance, road-forcing responses were calculated over the machine’s operating regime [31]. Figure 7 reproduces the results for network N3, where the H∞ -norm of the transfer function mapping the combined rear-front road-input3 to the steer angle is found as the trim state ranges over the motorcycle’s cornering regime (under straight-running conditions the road-forcing response goes to zero). 3

Since the road forcing signal of the rear wheel is a delayed version of that ‘seen’ by the front wheel, dr and df can be generated from a single disturbance d via df = d and dr = e−sτ d. The time delay τ is given by l/v in which l is the vehicle’s wheel base and v its speed.

An H∞ Loop-Shaping Approach

267

70

60

Imaginary Axis

50

40

30

20

10

0 −18

−16

−14

−12

−10

−8

Real Axis

−6

−4

−2

0

2

Fig. 6. Root-locus plots for: Straight running (×), 15 deg (◦), 30 deg (+) and 45 deg (♦) of roll angle, with the speed varied from 7 m/s ( ) to 75 m/s ( ). The frequencydomain optimized network N8 [31] is ﬁtted.

30 25

dB

20 15 10 5 0 75 60

40 40

45

30

Speed (m/s)

20

20 7

3

10

Roll Angle (deg)

Fig. 7. Inﬁnity norm of the closed-loop transfer function that maps the combined road forcing input to the steer angle. The speed is varied from 7 m/s to 75 m/s, and the roll angle is varied from 3 to 45 degs. The optimized N3 network is ﬁtted.

268

S. Evangelou et al.

5 Results 5.1 Controller Design The H∞ loop shaping design follows the procedure given in Section 2. The plant G comes from linearizing the equations of motion around a trim state of 75 m/s and a lean angle of 3 deg. The trim condition selected corresponds to an equilibrium state in which the weave mode is particularly lightly damped and the machine is vulnerable to road displacement forcing. The plant frequency response is shaped using a weighting function W that is a cascade of sY (s) and W , where Y (s) is given by (9), and W =

1.5(s + 1) . 1 s + 1)(s + 12) ( 31

(10)

The optimized N8 network parameters are given in Section 4.2. The extra ‘s’ which multiplies Y (s) is included because steering angle is used in Fig. 1 as the controller input, whereas steering angular velocity is used in [31]. The use of N8 as part of the loop shape is motivated by its excellent global performance as a passive steering compensator [31]. The further shaping of N8 via W is motivated by the need to limit the bandwidth of the controller at high frequencies, and to reduce the control activity at low frequencies. The low low-frequency control activity objective is motivated by the need to prevent the steering compensator from interfering with low-frequency rider steering controls. The ﬁnal controller K is found by post-multiplying K∞ by W . As a result, the band-pass properties of the loop shape will be inherited by K. The frequency response of network N8 and of the loop shaping weighting function are shown in Figure 8. For convenience, the ‘s’ in the numerator of W is not included, so eﬀectively the two weighting functions are referred to a compensator input of δ˙ rather than δ. A further design issue is the choice of road disturbance input weights Wdr and Wdf . If these weights are set to zero a pure normalized coprime factor robust stability optimization problem results, but the resulting controller performs no better than N3 or N8. Consequently the inﬂuences of the road-forcing disturbance inputs were introduced explicitly into the optimization problem. To do this, disturbance weights were introduced and it turned out from trials that non-dynamic road-forcing weighting functions were adequate. These were chosen to be Wdr = 1.3 and Wdf = 2.6. The optimization problem is set up in Matlab [49] and the function hinfsyn is used to solve the H∞ problem. The resulting closed-loop gain is γ = 3.3302, so perturbations to the coprime factors of less than 0.3 can be tolerated. The resulting H∞ normalized coprime factor controller has 31 states. Some iteration in the choice of the weights was necessary in order to arrive at the ﬁnal design. The frequency response of the controller is shown in Figure 9 together with the N8 network and the loop shaping weighting function, all referred to a compensator input of δ˙ rather than δ. It is clear from

An H∞ Loop-Shaping Approach

269

Magnitude (dB)

30 25 20 15 10 5 0 −5

Phase (deg)

−10 90 45 0 −45 −90 −2 10

−1

0

10

1

10

2

10

10

Frequency (rad/sec)

3

10

Fig. 8. The solid line represents the frequency response of the frequency optimised N8 network. The dashed line shows W /s where W is the weighting function used in the H∞ controller design.

Magnitude (dB)

35 30 25 20 15 10 5 0 −5

Phase (deg)

−10 900 720 540 360 180 0 −180 −1 10

0

10

1

10

Frequency (rad/sec)

2

10

3

10

Fig. 9. The solid line represents the frequency response of the frequency-domain optimized N8 network; the dashed line shows the weighting function used in the H∞ controller design and the dash-dot line shows the frequency response of the normalized coprime factor H∞ controller.

270

S. Evangelou et al.

this ﬁgure that the controller inherits the low- and high-frequency characteristics of the loop shape. In particular, for frequencies lower than approximately 2 rad/s, the response magnitude is lower than that associated with N8. For frequencies less than 1 rad/s, the magnitude is approximately constant and the phase shift approximately zero—as with a weak damper. 5.2 Controller Performance Results that test the performance of the H∞ controller in stabilising the weave and wobble modes, and also in decreasing the vulnerability of these modes to road displacement forcing are now presented. Figure 10 shows a root-locus for a range of speeds and lean angles. As compared with the nominal steering damper and networks N3 and N8 (see Figures 3, 5, 6 respectively), it is clear that the active compensator improves substantially the damping of the wobble mode. The weave mode damping is also improved for high-speed low-lean-angle conditions. Set against this, the active compensator appears to produce a small degradation in the weave mode damping under high roll angle conditions. As we will see in later results, this does not appear to have a negative impact on the weave-mode-related road-forcing responses.

70

Imaginary Axis

60

50

40

30

20

10

0 −18

−16

−14

−12

−10

−8

Real Axis

−6

−4

−2

0

2

Fig. 10. Root-locus plots for: Straight running (×), 15 deg (◦), 30 deg (+) and 45 deg (♦) of roll angle with the speed varied from 7 m/s ( ) to 75 m/s ( ). The H∞ controller is used as the compensator.

Figure 11 shows the road-forcing response as the trim state ranges over the motorcycle’s roll angle and speed operating envelope. It is evident that Figures 7 and 11 have the same broad characteristics and that the H∞ controller reduces the peak magnitude of the response by approximately 4 dB.

An H∞ Loop-Shaping Approach

271

25

dB

20 15 10 5 0 75 60

40 40

45

30

Speed (m/s)

20

20 7

3

10

Roll Angle (deg)

Fig. 11. Gain of the closed-loop road forcing transfer function for 7 to 75 m/s, and for 3 to 45 deg roll angle. The H∞ active compensator is ﬁtted.

It was argued earlier that the active control scheme reduces the damping of the weave mode at high lean angles. In order to establish the road-forcing characteristics of the machine operating under these conditions, the frequency responses shown in Figure 12 are examined. This ﬁgure corresponds to a highlean-angle and moderate-speed condition in which the damping of the weave mode is at a minimum (see Figure 10). The wobble-mode peak is reduced slightly as compared with the N8 network response and it is reduced by approximately 5 dB as compared with the N3 network response. From these results we conclude that the performance of the active controller is adequate under high roll angle conditions.

6 Conclusions The research in this paper was motivated by a previous study that established the beneﬁcial inﬂuence of passive mechanical networks on the steering behaviour of high-powered motorcycles. Such networks were proposed for use in place of the conventional steering damper [31], and showed the potential to remove the compromise faced when trying to stabilize simultaneously the wobble and weave modes with a steering damper. In this paper active steering compensators are investigated as a possible alternative to passive mechanical networks. The active steering compensator has a number of potential advantages over the passive networks presented in [31]. These are: (i) the compensator need no longer be positive real (passive); (ii) the compensator complexity is a less pressing issue; (iii) in a software implementation it

272

S. Evangelou et al. 25

Magnitude (dB)

20

15

10

5

0

−5 1 10

2

Frequency (rad/sec)

10

Fig. 12. Bode magnitude plot for the closed-loop road displacement forcing response (0 dB = 1 rad/m). The machine is operating at a forward speed of 47 m/s and a roll angle of 45 deg. The solid line represents the machine ﬁtted with the frequencydomain optimized N3 network, the dashed line corresponds to the machine ﬁtted with the frequency-domain optimized N8 network and the dash-dot line represents the machine compensated by the H∞ controller.

is easy to adjust the compensator parameters; (iv) there is the possibility to implement parameter-adaptive control schemes. The results of Section 4 make use of an advanced motorcycle simulation model [7, 10, 11], which is thought likely to be representative of highperformance production motorcycles generally. The nominal model shows that the machine is vulnerable to high-speed weave oscillations, particularly under low-roll-angle conditions. The low-speed wobble mode, particularly at high roll angles is similarly vulnerable. The previous study [31] showed that there is only a small usable range of steering damper parameter values, and illustrated why conventional motorcycles are dogged by an apparently intractable wobble-weave-mode damping compromise. It also demonstrated that passive mechanical networks can be used to break this circle of compromise and improve the dynamic behaviour of both wobble and weave modes together. The results presented in Section 5 show that further improvements are achievable using ﬁxed parameter active controllers. The performance is improved in respect of the wobble and weave mode damping factors and the reduction of the resonant peaks in the road forcing response. An important feature of the loop shaping methodology is its ability to ensure that the compensator has bandpass characteristics that cover the wobble and weave range of frequencies. This allows the controller to be designed so that the very low band of frequencies used by the rider for path tracking and balancing is not

An H∞ Loop-Shaping Approach

273

impacted signiﬁcantly, and that the steering actuator (torque motor) is not subjected to high frequency drive signals. The practical implementation of active steering compensation schemes is the subject of on-going work. In broad terms, such a system is likely to be based on a torque motor, power ampliﬁer and digital signal processing chip set. The rating of the torque motor and its associated gearing and drive electronics are likely to be particularly important.

References 1. D. J. N. Limebeer, R. S. Sharp, and S. Evangelou, “Motorcycle steering oscillations due to road proﬁling,” Transactions of the ASME, Journal of Applied Mechanics, vol. 69, no. 6, pp. 724–739, 2002. 2. D. J. N. Limebeer and R. S. Sharp, “Bicycles, motorcycles and models,” IEEE Transactions Control Systems Magazine, 2006. 3. R. S. Sharp, “The stability and control of motorcycles,” Proc. I MECH E Part C Jour. Mech. Eng. Sci., vol. 13, no. 5, pp. 316–329, 1971. 4. ——, “Vibrational modes of motorcycles and their design parameter sensitivities,” in Vehicle NVH and Reﬁnement, Proc. Int. Conf. Birmingham: Mech. Eng. Publications, London, 3–5 May 1994, pp. 107–121. 5. D. J. N. Limebeer, R. S. Sharp, and S. Evangelou, “The stability of motorcycles under acceleration and braking,” Proc. I MECH E Part C Jour. Mech. Eng. Sci., vol. 215, no. 9, pp. 1095–1109, 2001. 6. C. Koenen, “The dynamic behaviour of motorcycles when running straight ahead and when cornering,” Ph.D. dissertation, Delft University of Technology, 1983. 7. R. S. Sharp and D. J. N. Limebeer, “A motorcycle model for stability and control analysis,” Multibody System Dynamics, vol. 6, no. 2, pp. 123–142, 2001. 8. V. Cossalter and R. Lot, “A motorcycle multi-body model for real time simulations based on the natural coordinates approach,” Vehicle System Dynamics, vol. 37, no. 6, pp. 423–447, 2002. 9. R. S. Sharp, “Stability, control and steering responses of motorcycles,” Vehicle System Dynamics, vol. 35, no. 4–5, pp. 291–318, 2001. 10. R. S. Sharp, S. Evangelou, and D. J. N. Limebeer, “Improved modelling of motorcycle dynamics,” in ECCOMAS Thematic Conference on Advances in Computational Multibody Dynamics, J. A. C. Ambr´ osio, Ed., Lisbon, July 1–4 2003, MB2003-029 (CD-ROM). 11. ——, “Advances in the modelling of motorcycle dynamics,” Multibody System Dynamics, vol. 12, no. 3, pp. 251–283, 2004. 12. E. D¨ ohring, “Steering wobble in single-track vehicles,” ATZ, vol. 58, no. 10, pp. 282–286, 1956. 13. D. J. Eaton, “Man-machine dynamics in the stabilization of single-track vehicles,” Ph.D. dissertation, University of Michigan, 1973. 14. Y. Watanabe and K. Yoshida, “Motorcycle handling performance for obstacle avoidance,” in 2nd Int. Congress on Automotive Safety, San Fransisco, 1973. 15. G. Jennings, “A study of motorcycle suspension damping characteristics,” SAE 740628, 1974.

274

S. Evangelou et al.

16. G. E. Roe and T. E. Thorpe, “A solution of the low-speed wheel ﬂutter instability in motorcycles,” Proc. I MECH E Part C Jour. Mech. Eng. Sci., vol. 18, no. 2, pp. 57–65, 1976. 17. M. K. Verma, “Theoretical and experimental investigation of motorcycle dynamics,” Ph.D. dissertation, University of Michigan, 1978. 18. J. S. McKibben, “Motorcycle dynamics – fact, ﬁction and folklore,” SAE 780309, 1978. 19. R. S. Rice, “Rider skill inﬂuences on motorcycle maneuvring,” SAE 780312, 1978. 20. A. Aoki, “Experimental study on motorcycle steering performance,” SAE 790265, 1979. 21. D. H. Weir and J. W. Zellner, “Experimental investigation of the transient behaviour of motorcycles,” SAE 790266, 1979. 22. B. Thomson and H. Rathgeber, “Automated systems used for rapid and ﬂexible generation of system models exempliﬁed by a veriﬁed passenger car and a motorcycle model,” in The Dynamics of Vehicles on Roads and on Railway Tracks, J. K. Hedrick, Ed. Swets and Zeitlinger, Lisse, 1984, pp. 645–654. 23. B. Bayer, “Flattern und pendeln bei kraftr¨ adern,” Automobil Industrie, vol. 2, pp. 193–197, 1988. 24. T. Takahashi, T. Yamada, and T. Nakamura, “Experimental and theoretical study of the inﬂuence of tires on straight-running motorcycle weave response,” SAE 840248, 1984. 25. W. M. Otto, “Eﬀect of motorcycle accessories on stability,” in Proc. International Motorcycle Safety Conference. Washington DC: Motorcycle Safety Foundation, Linthicum MD 21090, May 1980, pp. 1560–1581. 26. M. Sugizaki and A. Hasegawa, “Experimental analysis of transient response of motorcycle rider systems,” SAE 881783, 1988. 27. F. Biral, D. Bortoluzzi, V. Cossalter, and M. Da Lio, “Experimental study of motorcycle transfer functions for evaluating handling,” Vehicle System Dynamics, vol. 39, no. 1, pp. 1–26, 2003. 28. T. Foale. (2002) Motorcycle handling and chassis design – the art and science. Available: http://www.tonyfoale.com 29. R. S. Sharp and C. J. Alstead, “The inﬂuence of structural ﬂexibilities on the straight running stability of motorcycles,” Vehicle System Dynamics, vol. 9, no. 6, pp. 327–357, 1980. 30. P. T. J. Spierings, “The eﬀects of lateral front fork ﬂexibility on the vibrational modes of straight-running single-track vehicles,” Vehicle System Dynamics, vol. 10, no. 1, pp. 21–35, 1981. 31. S. Evangelou, D. J. N. Limebeer, R. S. Sharp, and M. C. Smith, “Mechanical steering compensators for high-performance motorcycles,” Cambridge Engineering Department, CUED/F-INFENG/TR 535, Tech. Rep., 2005. 32. M. C. Smith, “Synthesis of mechanical networks: The inerter,” IEEE Transactions on Automatic Control, vol. 47, no. 10, pp. 1648–1662, October 2002. 33. K. Glover and D. C. McFarlane, “Robust stabilization of normalized coprime factor plant descriptions with H∞ -bounded uncertainty,” IEEE Transactions on Automatic Control, vol. 34, no. 8, pp. 821–830, 1989. 34. D. C. McFarlane and K. Glover, Lecture Notes in Control and Information Sciences. Springer-Verlag, 1990, vol. 138, ch. Robust controller design using normailzed coprime factor plant descriptions.

An H∞ Loop-Shaping Approach

275

35. ——, “A loop shaping design procedure using H∞ synthesis,” IEEE Transactions on Automatic Control, vol. 37, no. 6, pp. 759–769, 1992. 36. A. K. El-Sakkary, “The gap metric: Robustness of stabilization of feedback systems,” IEEE Transactions on Automatic Control, vol. 30, no. 3, pp. 240– 247, 1985. 37. T. T. Georgiou and M. C. Smith, “Optimal robustness in the gap metric,” IEEE Transactions on Automatic Control, vol. 35, no. 6, pp. 673–686, 1990. 38. G. Vinnicombe, “Frequency domain uncertainty and the graph topology,” IEEE Transactions on Automatic Control, vol. 38, no. 9, pp. 1371–1383, 1993. 39. M. Green and D. J. N. Limebeer, Linear Robust Control. Englewood Cliﬀs, New Jersey 07632: Prentice Hall, 1995. 40. K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control. Prentice Hall, 1995. 41. E. J. H. de Vries and H. B. Pacejka, “The eﬀect of tyre modeling on the stability analysis of a motorcycle,” in Proc. AVEC’98. Nagoya: SAE of Japan, 1998, pp. 355–360. 42. ——, “Motorcycle tyre measurements and models,” in Proc. 15th IAVSD Symposium on the Dynamics of Vehicles on Roads and on Tracks, L. Palkovics, Ed., Budapest Hungary, August 25–29 1997, suppl. Vehicle System Dynamics, 28, 1998, 280–298. 43. H. B. Pacejka, Tyre and Vehicle Dynamics. Oxford: Butterworth Heinemann, 2002. 44. Y. Tezuka, H. Ishii, and S. Kiyota, “Application of the magic formula tire model to motorcycle maneuverability analysis,” JSAE Review, vol. 22, pp. 305–310, 2001. 45. Anon., Autosim 2.5+ Reference Manual, Mechanical Simulation Corporation, 709 West Huron, Ann Arbor MI, 1998, http://www.carsim.com. 46. R. C. Dorf and R. H. Bishop, Modern Control Systems. Upper Saddle River, NJ: Prentice Hall, 2001. 47. M. E. V. Valkenburg, Introduction to modern network synthesis. Wiley, 1960. 48. R. M. Foster and E. L. Ladenheim, “A class of biquadratic impedances,” IEEE Transactions on Circuit Theory, vol. 10, pp. 262–265, 1963. 49. The Mathworks Inc., MATLAB 6 Reference Manual, 2000, http://www.mathworks.com.

Frequency Domain Versus Time Domain Methods in System Identiﬁcation – Revisited Lennart Ljung Dept of Electrical Engineering, Link¨ oping University, SE-582 45 Link¨ oping, Sweden, [email protected]

1 A Personal Introduction For some reason I now forget, Keith Glover and I were asked to prepare a “discussion paper” on frequency and time domain methods in system identiﬁcation by the organizers of the IFAC symposium on System Identiﬁcation in Darmstadt 1979, [8]. I visited Keith in Cambridge in April 1979 to prepare our paper. This was my ﬁrst “scientiﬁc” visit to Cambridge, and I had some very productive and pleasant days with Keith there. After the symposium we were asked to produce an Automatica version of the discussion paper, and this was eventually published as [9]. Although I have met Keith and had interesting and enjoyable discussions with him many, many times since then, these two papers remain our only joint publications. While Keith has moved on to many more exciting problems, I have continued to stay close to system identiﬁcation. Quite recently, as I extended the System Identiﬁcation toolbox [6] to deal with frequency domain data, I had occasion to revisit the issues we dealt with in our joint paper [9]. This paper has probably been all but forgotten by both of us. It is not a central publication of either of us and has been cited only 25 times over the years according the the ISI Web of Knowledge. Anyway, rereading it today shows that it has not aged that much, and several of the issues are still relevant. I think it is a witness to Keith’s ability and ambition to do a professional job in all situations. This is the background why I found it appropriate to celebrate Keith’s birthday and illustrious career by going back to our joint work, a quarter of a century ago. There has been a useful development of several of the problems and questions dealt with in [9]. Questions of Maximum Likelihood formulations, asymptotic analysis, choice and design of noise ﬁlters and numerical algorithms, for example have been discussed in, among many references, [4], [3], [19], [15], [12], [13], [20], [10],and [16].

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 277–291, 2006. © Springer-Verlag Berlin Heidelberg 2006

278

L. Ljung

In this contribution I will focus on three aspects that relate to using frequency domain data for identiﬁcation: The role of periodic input data, the role of the input intersample behavior and packaging a transparent time/frequency domain tool for identiﬁcation. Section 2 describes the direct use of frequency domain data for identiﬁcation, while the apparent importance of periodic inputs is illustrated in Section 3. The fact that also non-periodic data can be handled in a correct way by proper use of initial conditions is explained in Section 4. The issues of the intersample behavior of the input and the role of bandlimited data are discussed in Sections 5 and 6. Finally, the software aspects are dealt with in Section 7.

2 Data in the Frequency Domain Our paper [9] primarily deals with frequency domain methods such as spectral analysis, estimating frequency functions and spectra and using Parseval’s relationship to gain insight into time domain expressions. At the same time, in the instrumentation and measurement community, frequency domain data were being used as primary measurements for model building. This involved periodic, band-limited inputs in well controlled experiments, to construct transfer functions and reﬁned frequency function estimates. Frequency analyzers were often used to generate, collect and compress data in such experiments, which could be seen as direct extensions of frequency analysis (sine-wave testing). Vibration and modal analysis were and are common applications of this type. With band-limited, periodic data, the Fourier transforms (or Fourier coefﬁcients) of the input and output signals U (iω) and Y (iω) can readily be determined from sampled data. In case non-band-limited noise aﬀects the output, this should ﬁrst be anti-alias ﬁltered, in order not to confuse the transform below the Nyquist frequency. If the linear system that relates the input and output has the frequency function G(iω), the relationship is simply Y (iω) = G(iω)U (iω) + V (iω)

(1)

where V (iω) is the Fourier transform of the additive noise (contributions below the Nyquist frequency). For good signal-to-noise ratios the quotient Y (iω) ˆˆ G(iω) = U (iω)

(2)

(“the empirical transfer function estimate”, ETFE) will be a good estimate of the frequency function and is typically delivered as the output of frequency analyzers. If the frequency function is parameterized as G(iω, θ), a natural, least squares estimate of θ will be θˆN = arg min θ

N

k=1

|Y (iωk ) − G(iωk , θ)U (iωk )|2

where the summation is over the DFT-frequency grid.

(3)

Frequency Domain Versus Time Domain Methods

279

A weighted version is obtained by θˆN = arg min θ

N

ck |Y (iωk ) − G(iωk , θ)U (iωk )|2

(4)

k=1

where varying reliability and relevance of the measurements at diﬀerent frequencies can be taken care of by proper choices of the weights ck .. All this is very basic, and can be elaborated into a maximum likelihood framework, e.g. [18], [15], [5], Section 7.7.

3 Periodic Inputs For (1) to hold exactly, the input and output must be periodic, otherwise transient eﬀects from circular convolution will corrupt the relationship. This could be a non-trivial problem, as the following simple example shows: Consider the discrete time system G(q) =

1 + 2q −1 + 3q −2 1 − q −1 + 0.9q −2

(q denotes the shift operator). If this system is simulated noise-free with a white input over 50 samples, a time domain ﬁt to the data will give us the true system, exactly. By Fourier transforming the data and applying (3) we get instead the estimate 1.617 + 2.056q −1 + 3.217q −2 ˆ G(q) = 1 − 0.96q −1 + 0.83q −2 Since the data is noise free, the discrepancy is entirely due to non-periodic convolution eﬀects in (1). Now, there are several good reasons to use periodic inputs in system identiﬁcation, whenever possible. Perhaps the most important one is that after the transient has died out, the diﬀerences in the output over diﬀerent periods must be due to the noise inﬂuence. This means that simple estimates of the noise level will be possible. Such knowledge is invaluable for evaluating model quality. Still, it may not be possible to apply periodic inputs in experiments that are not fully controlled. Then one would think that frequency domain methods like (3) can not be applied. In fact, it seems like this was a major reason why methods based on frequency-domain data were rejected – or at least not used – in the control oriented system identiﬁcation community until recently. However, it eventually was found, as shown in the next section, that the question of periodic or not periodic is just a reﬂection of unknown initial conditions, which is equally relevant for time-domain methods.

280

L. Ljung

4 Transients and Initial States Let us go back to the basic relationship (1) and focus on discrete time. The arguments below are applicable to multi-input-multi-output system, even though the notation suggests a SISO system. Let us consider the noise-free part of the response yu (t) = G(q)u(t)

(5)

We assume only a ﬁnite number of samples of inputs and outputs be known: y(1), y(2), . . . , y(N ) u(1), u(2), . . . , u(N ) The inputs prior to t = 1 are thus not known. Let yˇu (t) denote what would be the output corresponding to a particular assumption about u(t), t = −∞, . . . , −2, −1, 0. Two typical cases would be yˇu (t) = yu0 (t) is the output of (5) if u(t) = 0, t ≤ 0

yˇu (t) = yup (t) is the output of (5) if u(t) is periodic with period N from t = −∞ to t = N Let now (5) be realized in state space form: x(t + 1) = Ax(t) + Bu(t) y(t) = Cx(t)

(6) (7)

Whatever assumption about prior values of u(t) we have made, it would have left us in a certain state x(0) = x ˇ at time 0. (For example, all prior u:s being zero would give x(0) = 0.) Let the actual, typically unknown, initial state be x(0) = x∗ . Then yu (t) = yˇu (t) + y˜u (t) y˜u (t) = C(qI − A)−1 A(x∗ − x ˇ)δ(t) δ(t) =

0 1

if t = 0 if t = 0

The term y˜u (t) is thus the response from the initial conditions. Alternatively it can be seen as the impulse response from an additional input, which is an impulse: x(t + 1) = Ax(t) + Bu(t) + Rδ(t) x(0) = xˇ (the assumed input behavior prior to t = 0) y(t) = Cx(t) R = A(x∗ − x ˇ)

Frequency Domain Versus Time Domain Methods

281

The consequence is that any (possibly erroneous) guess of input behavior prior to time t = 0 can always be made up for by adding an extra input which is an impulse at time 0. The dynamics from this input has the same poles as the system but unknown zeros. Note that one extra input is suﬃcient, even if there are several regular inputs. Of course, the whole process can be described without reference to a state space realization (6). The model to be considered in input-output form would be y(t) = G(q)u(t) + Gi (q)δ(t) =

R(q) B(q) u(t) + δ(t) F (q) F (q)

(8)

where the second step corresponds to a rational transfer function parameterization of the model. Note that the denominators are the same, and that the order of the R-polynomial is one less than the order of F (q). Also, if the input u is of dimension r, there would be r diﬀerent numerator polynomials Bk (q), but still just one R(q)-polynomial. The typical two cases for assumed prior behavior of the inputs are 1. In the time domain: Assume that all prior values of u(t) are zero. This will give the simple predictor yˆ(t) = G(q)u(t) with all values of u prior to t = 1 being zero. 2. In the frequency domain: Assume that all prior values of u are obtained by periodic continuation of u backwards in time. This will make the Fourier transformed relation YN (eiωTs ) = G(eiωTs )UN (eiωTs )

(9)

exact for the u-inﬂuence at the DFT-grid-points. Here YN and UN are the Discrete Fourier Transforms of the output and input (deﬁned by (12) below). Now, for general data sets, these two assumptions are not correct, but the point is that an extra input signal which is an impulse will make them correct, if this input is passed through a system with the same poles as the model, and the zeros are adjusted to data (to match the assumption.) This extra input can be neglected, only if we know that the input is periodic in the frequency domain case, or past values are zero in the time domain. For long data records, it may be of less importance, since the eﬀects of this impulse response may decay quickly compared to the data length. This way to compensate for non-periodic frequency domain data was described in [17]. See also [19] for an instructive discussion. Note that the DFT of the impulse δ(t) is just a sequence of 1’s. To actually achieve this treatment of the non-periodicity is thus quite simple in practice: Assume in the model structure that there is one more input and append to the input DFT UN one column of 1’s. The analysis has also brought out the kinship between time- and frequency methods: The non-periodicity is for frequency domain data exactly the same

282

L. Ljung

problem as non-zero initial state for time-domain data. One should note that the estimated R-vector (or R-polynomial) is speciﬁc for the estimation data. In particular, one must be careful when using an estimated initial condition for another data sequence. This is the same problem for time and frequency domain data. Example, [7] The system y(t) =

q −1 + 0.5q −2 1 − 1.5q −1 + 0.99q −2

was simulated noise-free over 150 data points with a white noise input. Samples 101 to 150 were selected for identiﬁcation. Models were ﬁt to these data both in the time and frequency domains and both with and without adding an extra input being an impulse. This gave the following estimates: •

Time domain and frequency domain with extra input: q −1 + 0.5q −2 1 − 1.5q −1 + 0.99q −2 The “initial state” or numerator polynomial for the

y(t) =

extra impulse input were Ri (q) = −14.06q −1 − 1.079q −2 and Ri (q) = −18.99 − 1.63q −1 respectively. •

Time domain without extra input: y(t) =

•

1.32q −1 − 0.26q −2 1 − 1.55q −1 + 0.94q −2

Frequency domain without extra input y(t) =

1.04q −1 + 0.68q −2 1 − 1.47q −1 + 0.87q −2

This illustrates clearly that erroneous assumptions about past inputs (they are neither zero nor periodic in this case) can give bad results both in timeand frequency domains even for noise-free data. However, they can be handled with proper use of initial conditions (an extra input impulse) in the same way in the time and the frequency domains. (The four models could be reproduced in [6] by the code m0 = idpoly(1,[0 1 0.5],1,1,[1 -1.5 0.99]); u = randn(150,1): y = sim(m0,u);

Frequency Domain Versus Time Domain Methods

283

z = iddata(y,u); ze = z(101:150); udel = [1;zeros(49,1)]; zed = ze; zed.u = [ze.u udel]; m1 = oe(zed,[2 2 2 2 1 1],’ini’,’z’); m2 = oe(fft(zed),[2 2 2 2 1 0],’ini’,’z’); m3 = oe(ze,[2 2 1],’ini’,’z’); m4 = oe(fft(ze),[2 2 1],’ini’,’z’); (The pair ’ini’,’z’ is there to inhibit the default estimation of initial states.) Of course, the addition of the extra impulse input and the estimation of it associated zeros can easily be automated. This is done in [6] by m1 = oe(ze,[2 2 1],’ini’,’e’); %’e’ for ’estimate’ m2 = oe(fft(ze),[2 2 1],’ini’,’e’);

5 Continuous-Time Data with Band-Limited Input In the time-domain there is no real possibility to deal with continuous-time data directly. A great feature of frequency domain data is that it can handle continuous time measurements. Let U (iω) and Y (iω) be the (continuous time) Fourier transforms of the input and the output: U (iω) = Y (iω) =

∞ 0

0

∞

u(t)e−iωt dt y(t)e−iωt dt

(10)

Then a ﬁnite dimensional representation of the data is obtained by collecting these transforms at a ﬁnite number of frequencies: Z N = {U (iω1 ), Y (iω1 ), . . . , U (iωN ), Y (iωN )}

(11)

Such data could be used for directly ﬁtting continuous time models as in (3). Clearly, in practice we neither have inﬁnite nor continuous time data records. Still, the Fourier transform (10) could be evaluated under some circumstances. To get correct values for (10)-(11) in practice, it is required that the whole continuous signal u(t), −∞ < t < ∞ can be reconstructed from the sampled, observed data {u(Ts ), . . . , u(N Ts )} (Ts being the sampling interval). This is the case if u is periodic and band-limited. (In the periodic case, the Fourier transform values U (iωk ) will have to be interpreted as the Fourier coeﬃcients corresponding to the fundamental frequency and its harmonics, resulting from applying (10) to one period of data only.) That is to say, that the results of the simple DFT transformation

284

L. Ljung

UN (eiωTs ) = Ts

N −1

u(kTs )e−iωkTs

(12)

k=0

evaluated on the “DFT-grid” ωk = 2πk/(N Ts ), k = 0, 1, . . . , N − 1

(13)

can be interpreted as the Fourier coeﬃcients of the inﬁnite signal obtained by making u(t) periodic with period N Ts , and making it time continuous by trigonometric interpolation (no power above the Nyquist frequency). If the relation between input and output is y(t) = G(p)u(t) + v(t) the output y need not be periodic and band-limited even if u is that, due to the inﬂuence of the noise v. However, the component of y that originates from u (i.e. Gu) is periodic and band-limited. Since the estimate of G only should depend on this component, frequencies in y higher than the highest frequency in u should be removed. Then the output could be averaged over the periods to get accurate values of the relevant components of YN (iωk ). Another, more pragmatic way is when the sampling interval T is small compared to the interesting dynamics of the system and compared to the rate of change in u. Then the input could be considered to be band-limited in practice, and the values from (12) (after suitable normalization) will be good estimates of UN (iω).

6 Other Intersample Behaviors The problem to compute the continuous-time Fourier transform from equidistantly sampled data is, as we saw, trivial if the signal is band-limited. In general, the continuous-time transform can always be computed from sampled data if the intersample behavior of the signal is known. Typical intersample behaviors of the input are • •

Zero-order hold (zoh): The input is piecewise constant between samples First-order hold (foh): The input is piecewise linear between samples

In the zoh case with sampling interval Ts we have u(t) = u(kTs ) for kTs ≤ t < (k + 1)Ts so assuming a periodic input with period T = N Ts we have

Frequency Domain Versus Time Domain Methods

Uc (iω) =

T 0 N −1

=

N −1

u(t)e−iωt dt =

k=0

u(kTs )e−iωkTs

k=0

(k+1)Ts kTs −iωTs

1−e iω

285

u(kTs )e−iωt dt

= UN (eiωTs )HTzs (iω)

(14)

1 − e−iωTs HTzs (iω) = Ts iω Similarly, for a ﬁrst order hold input we have Uc (iω) = UN (eiωTs )HTfs (iω) HTfs (iω)

=e

iωTs

1 − e−iωTs Ts iω

2

(15)

A very important aspect of band-limited input is that also the modeldependent part of the output is band-limited, so that also the continuous-time Fourier transform of this part can be readily computed. This is not the case for zoh and foh inputs: If we have a continuous time system y(t) = Gc (p)u(t) which is sampled with a zero-order hold input, the sampled signals obey y(kTs ) = Gd (q)u(kTs ) where Gd is computed from Gc in a well-deﬁned and well-known manner. This gives the relationships between the Fourier transforms (UN is the DFT (12) and Uc is deﬁned by the ﬁrst equality in (14), and analogously for Y ): YN (eiωTs ) = Gd (eiωTs )UN (eiωTs ) Yc (iω) = Gc (iω)Uc (iω) = Gc (iω)HTzs (iω)UN (eiωTs ) from which we ﬁnd the continuous time transform for the output: Yc (iω) = FTs (iω)YN (eiωTs ) FTs (iω) =

Gc (iω) H z (iω) Gd (eiωTs ) Ts

(16)

where H z was deﬁned in (14). Note that the function FTs (iω) depends on the system Gc . For example, if Gc (s) = 1/s we obtain, after simple calculations, that FTs (iω) = HTfs (iω), which naturally enough shows that a zoh input becomes a foh output after having been integrated. Anyway, the result (16) cannot be applied in an identiﬁcation situation, where Gc is unknown. This shows the unique advantage of band-limited input for estimation continuous time models in the frequency domain.

286

L. Ljung

Even though (16) cannot be applied directly for identiﬁcation, we can reason as follows, see [2]: If the signals are reasonably fast sampled, it is only the high frequency behavior of F that matters, that is only the roll oﬀ of the frequency functions Gc and Gd is important. That means that if we know that the system Gc has a pole excess of then we can approximate Gc by integrations: Gc (s) =

b s

(17)

when computing FTs in (16). The system approximation (17) has been used in several related contexts, like approximating zeros of sampled systems, [22], [21]. Sampling (17) gives expressions involving the Euler-Frobenius polynomials, so these will play a role in (16). All this means that from sampled input-output data we can easily ﬁnd the continuous-time Fourier transforms if the input is band-limited. If the intersample behavior of the input is known (e.g. zoh or foh), its continuoustime transform can also be constructed, while the reconstruction of transform of the output would require knowledge of the system. However, various approximations can be formed, whose accuracies improve as the sampling gets faster. It such cases it could be an advantage from a numerical point of view to use a δ-parameterization of the transfer function, see [14]. Also, some of the expressions (14)-(16) become more transparent when expressed in their δ-parameterized version.

7 Software Aspects We have described how frequency domain data can be used for identiﬁcation, also for non periodic data, and the role of continuous time data in the frequency domain. This gives a ground for a package for identifying linear systems that truly integrates time- and frequency data. In this section we shall describe how this is done in the System Identiﬁcation Toolbox (SITB) [6]. The goal of the syntax is to handle time domain data, frequency domain inputoutput data, and frequency response data in entirely analogous fashions both for estimating and validating models. 7.1 Input-Output Fourier Transform Data The iddata object contains input – output data. In the time domain case the deﬁnition of the object from data vectors or matrices is straightforward: dat = iddata(y,u,Ts) where Ts is the sampling interval. If the data instead is deﬁned as continuous time Fourier transform data over arbitrary frequencies as in (11) it can be collected into the data object as

Frequency Domain Versus Time Domain Methods

287

dat = iddata(Y,U,’Frequencies’,W,’Ts’,0); Note that the sampling interval, Ts , (’Ts’) is still relevant, since it has information of how the signal Fourier transforms Y and U have been computed from time domain data. Discrete time Fourier Transforms conceptually have the frequency argument eiωTs . Note however, that frequency domain data, unlike time domain data allow continuous time signals (Ts = 0). Compare the discussion in Section 5. With frequency domain data objects, several Matlab commands are naturally extended to be directly applied to the objects: DF = fft(dat) dat = ifft(DF) da = abs(dat) df = phase(dat) etc. 7.2 Frequency Response Data Frequency response data for a system given as G(iωk ) for a frequency vector ωk can be obtained, e.g. as the ETFE (2). Such data can be stored in the idfrd object in the SITB. It corresponds to the frequency response data object frd in the Control Systems Toolbox. The object is formed by dat = idfrd(G,W,Ts,’cov’,C) G contains the response data G(iωk ), and W the frequencies ωk . Ts is the sampling interval Ts (Ts = 0 denotes continuous time). It may be that a measure of reliability can be associated with the diﬀerent values of the frequency response. If the number Ck corresponds to an estimate of the covariance of the value G(iωk ), this information can be stored in the idfrd object by deﬁning the vector C = [C1 , C2 , . . . CN ] as indicated above. If such an uncertainty measure is not known, it can be omitted. It is often the case that experimental frequency domain response data are stored in this way as frequency functions rather than as inputs and outputs, separately. This also could be more economical, e.g. by using logarithmically spaced frequencies. There is also data acquisition equipment that collect and deliver the result of the experiment as frequency response data. 7.3 Estimation and Validation The point now is that whatever the format of dat, estimation and validation of models follow the same syntax: m1 = oe(dat,[2 2 1]) m2 = n4sid(dat,3) m3 = pem(dat)

288

L. Ljung

compare(dat,m1,m2,m3) resid(dat,m1) etc. The prediction error approaches (oe, pem etc) implement variants of the minimization problem (3), while the subspace estimation command n4sid is described in [12] for frequency domain data. (See also [11].) Arbitrary weighings ck in the frequency domain ﬁts, as in (4), can be obtained by m

= oe(dat,[2 2 1],’focus’,c)

By default, initial states are always estimated, as described in Section 4. This estimation can be inhibited by m = oe(dat,[2 2 1],’InitialState’,’zero’) If the frequency domain data is denoted as continuous time, a continuous time model is estimated directly by ﬁtting its frequency function to data as in (3). Compare also [1]. All the toolbox commands for simulation and validation handle time/frequency domain data in a transparent manner. The only restriction is that, for the moment, estimation of noise models and time-series models is not supported for frequency domain data. The reason is the more complex criteria of ﬁt that must be used for continuous-time and non-equidistantly spaced frequency data. See, e.g. page 230 in [5]. 7.4 Some Further Features Frequency domain data oﬀer useful potentials also for other problems: •

A focus ﬁlter can be implemented as speciﬁc frequencies for which the ﬁt should be made. For example, m = oe(dat,[2 2 1],’focus’,[0.2 1]) will concentrate the ﬁt to the pass band from 0.2 to 1 rad/s. This idea of preﬁltering is of course also available in the time domain, but it is particularly easy to implement in the frequency domain: Just cut out the relevant part of the frequency domain data vectors. The desired frequency bands may not necessarily be known a priori, but could be selected from a preliminary model, like using frequencies that correspond to the Nyquist curve being in the third quadrant, or being close to the critical point −1. Example: Build a ﬁfth order black-box model of the data, and compute its frequency response. Then adjust a simple ﬁrst order process model with a delay G(s) =

K e−Td s 1 + T1 s

(P1D)

to this frequency response, restricted to the third quadrant:

Frequency Domain Versus Time Domain Methods

289

m = n4sid(data,5); f = idfrd(m); ph = phase(squeeze(f.resp)); fs = fselect(f,find(ph>-pi & ph<-pi/2)); mp = pem(fs,’p1d’); •

If the inter-sample input behavior is band-limited, moving to the frequency domain will be the easiest way to handle the sampling. The FFT (discrete Fourier transform) of the input will then be equal to the Fourier transform of the underlying continuous time input signal. (Cf Section 5.) The FFT of the output will similarly correctly describe the continuous time Fourier transform of that part of the output that originates from the input, and we can directly ﬁt a continuous time model – in this case of the kind G(s) =

b s2 + f 1 s + f 2

dat = iddata(y,u,0.5); datf = fft(dat); datf.Ts = 0; mp = oe(datf,[1 2]) This will work also if the input is not band-limited, but sampled fast compared to dominating time constants. Then it will be a good idea to focus the ﬁt to lower frequencies: mp = oe(datf,[1 2],’foc’,[0 1]) 7.5 GUI Support The graphical user interface (GUI) has been extended to be transparent wrt the data domain. Frequency domain iddata and frequency response data as frd or idfrd objects can be imported into the GUI in the same way as time domain data. See Figure 1. The icons for the diﬀerent types of data sets are marked by diﬀerent background colors. The data preprocessing menus allow the transformation between the various representations. Also the use of data objects of diﬀerent types for estimation and validation is entirely transparent. For example, if an idfrd object is chosen as validation data, the Model output view shows the frequency responses of the models, together with the data.

8 Conclusions A fair amount of development of the understanding of frequency domain methods in system identiﬁcation has taken place in the 25 years since [9] was written. Among the most important ones are the practical insights into how to handle non-periodic data, and the potential to use frequency domain data for estimating continuous-time models.

290

L. Ljung

Fig. 1. The GUI

References 1. H. Garnier and P. C. Young, Time-domain approaches to continuous-time model identiﬁcation of dynamical systems from sampled data. In Proc. American Control Conference, Boston, MA, July 2004. 2. J. Gillberg and L. Ljung, Frequency domain identiﬁcation of continuous-time output error models from sampled data. In Proc. 16th IFAC World Congress, Prague, Czech Republic, July 2005. 3. I. Kollar, Frequency Domain System Identiﬁcation Toolbox for use with MATLAB. The MathWorks Inc., Natick, MA, USA, 1994. 4. L. Ljung, Building models from frequency domain data. In K. J. strm, G. C. Goodwin, and P. R. Kumar, editors, Adaptive Control, Filtering and Signal Processing, volume 74 of The IMA volumes in Mathematics and its Applications, pages 229–240. Springer-Verlag, New York, 1995. 5. L. Ljung, System Identiﬁcation - Theory for the User. Prentice-Hall, Upper Saddle River, N.J., 2nd edition, 1999. 6. L. Ljung, System Identiﬁcation Toolbox for use with Matlab. Version 6. The MathWorks, Inc, Natick, MA, 6th edition, 2003.

Frequency Domain Versus Time Domain Methods

291

7. L. Ljung, State of the art in linear system identiﬁcation: Time and frequency domain methods. In Proc. American Control Conference, Boston, MA, July 2004. 8. L. Ljung and K. Glover, Frequency domain versus time domain methods in system identiﬁcation - a brief discussion,. In R. Isermann, editor, Proc. Fifth IFAC Symposium on Identiﬁcation and System Parameter Estimation,, pages 1309–1322, Darmstadt, Germany, September 1979. Discussion Paper. 9. L. Ljung and K. Glover, Frequency domain versus time domain methods in system identiﬁcation. Automatica, 17(1):71 – 86, 1981. 10. T. McKelvey, Frequency domain identiﬁcation. In R. Smith and D. Seborg, editors, Proc 12th IFAC Symposium on System Identiﬁcation, Santa Barbara, CA, USA, June 2000. Plenary Paper. 11. T. McKelvey, Subspace methods for frequency domain data. In Proc. American Control Conference, Boston, MA, July 2004. 12. T. McKelvey, H. Ak¸cay, and L. Ljung, Subspace-based multivariable system identiﬁcation from frequency response data. IEEE Trans. on Automatic Control, 41(7):960–979, Jul 1996. 13. T. McKelvey and L. Ljung, Frequency domain maximum likelihood identiﬁcation. In Proc. 10th IFAC Symposium on System Identiﬁcation, Fokuoka, Japan, 1997. 14. R. H. Middleton and G. C. Goodwin, Digital Control and Estimation: A Uniﬁed Approach. Prentice-Hall, 1990. 15. R. Pintelon and J. Schoukens, System Identiﬁcation – A Frequency Domain Approach. IEEE Press, New York, 2001. 16. R. Pintelon and J. Schoukens, Box-Jenkins identiﬁcation revisited - Parts I and II. Automatica, 2005. To appear. 17. R. Pintelon, J. Schoukens, and G. Vandersteen, Frequency domain system identiﬁcation using arbitrary signals. IEEE Trans. Automatic Control, 42:17171– 1720, 1997. 18. J. Schoukens and R. Pintelon, Identiﬁcation of Linear Systems: A Practical Guideline to Accurate Modeling. Pergamon Press, London (U.K.), 1991. 19. J. Schoukens, R. Pintelon, and Y. Rolain, Time domain identiﬁcation, frequency domain identiﬁcation. equivalences! diﬀerences? In Proc. American Control Conference, Boston, MA, July 2004. 20. J. Schoukens, R. Pintelon, G. Vandersteen, and P. Guillaume, Frequency-domain system identiﬁcation using non-parametric noise models estimated from a small number of data sets. Automatica, 33(6):1073–1087, 1997. 21. S. R. Weller, W. Moran, B. Ninness, and A. D. Pollinton, Sampling zeros and the Euler-Frobenius polynomials. IEEE Transactions on Automatic Control, 46(2):340–343, 2001. 22. K. J. ˚ Astr¨ om, P. Hagander, and J. Sternby, Zeros of sampled systems. Automatica, 20(1):31–38, Jan 1984.

Optimal, Worst Case Filter Design via Convex Optimization Kunpeng Sun1 and Andy Packard2 1 2

Lehman Brothers, 745 7th Ave., New York, NY, 10019 [email protected] Mechanical Engineering, UC Berkeley, Berkeley CA 94720 [email protected]

1 Introduction The problem of model based signal estimation is fundamental to control theory and signal processing, and several approaches have been developed in last decades, for instance, the Kalman ﬁlter (H2 optimal ﬁltering) [1], H∞ optimal ﬁltering [15] and set-membership approach [4]. The performance of these optimal ﬁlters degrades in the presence of model/plant mismatch. Robust ﬁlter techniques have been studied to relieve this situation, and numerous papers on this subject have appeared, namely, [9, 24] and references therein, and the text by I.R. Petersen and A.V. Savkin [21] is a comprehensive collection of Riccati based (H2 , H∞ and set-membership) approaches. Most of these results are characterized by ﬁrst upper-bounding the performance objective, then selecting ﬁlter parameters to minimize the upper bound. There is little quantitative analysis on the conservativeness introduced by the use of upper bounds. Usually, these bounds guarantee performance not just over all ﬁxed values of the uncertainty, but over time-varying uncertainty as well. Hence, if the actual uncertainty model is time-invariant, these design methods may be conservative. A recent paper by Geromel et. al ( [11]) considers systems with time-invariant uncertainty, and the H2 bound they optimize partially exploits the time-invariance of the uncertainty. In our work we exploit that the robust ﬁlter design problem (with model uncertainty and noise) is convex in the ﬁlter as an operator. This fact seems largely unnoticed in the literature. Its implication is that the optimization can be carried out directly, rather than minimizing an upper bound of the objective function. The diﬃculty is that the optimization problem is inﬁnite dimensional, in both variables and constraints. We show that ﬁnite dimensional approximations can be used to obtain suboptimal solutions with any degree of accuracy. A design algorithm is proposed, made up of successive ﬁnite dimensional (in the ﬁlter degree) approximations. Converging bounds for approximation errors are calculated for each ﬁnite dimensional problem, via B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 293–315, 2006. © Springer-Verlag Berlin Heidelberg 2006

294

K. Sun and A. Packard

worst case analysis and convex optimization. Paganini and Giusto [19] also noticed the convexity of robust synthesis, but their work was on preﬁlters only. Our work not only studies more general cases, but also validates the use of ﬁnite dimensional approximation and gives an algorithm to design a suboptimal ﬁlter for any given precision. Some ideas in this paper follow from the work of Boyd [5], Ghulchak and Rantzer [12] and Dahleh [8]. This paper is organized as follows: in Section 2 a robust ﬁltering problem for systems with structured, time invariant uncertainty is formulated. We consider uncertain discrete-time systems, but the results can easily be transformed to continuous-time via the bilinear transform. It is shown in Section 3 that for any given > 0, there exists a ﬁnite dimensional relaxation that results in an -suboptimal solution for the original inﬁnite dimensional problem. In Section 4, we give an algorithm to design a suboptimal ﬁlter using convex optimization. An example is given in Section 6, and conclusions are drawn in Section 7. The notation is standard. The space RHm×n is all proper, real-rational ∞ functions with poles outside the unit disk. For A ∈ Cm×n , σ ¯ (A) and σ(A) denote the maximum and minimum singular value of matrix A. In a normed space (X, · ), for x ∈ X and r > 0, deﬁne B(x, r) := {y ∈ X : y − x < r}. B11 B12 Suppose B and T are linear operators, with B partitioned as B = . B21 B22 If T is dimensioned correctly and I − B11 T is invertible, deﬁne Fu (B, T ) := −1 B22 + B21 T (I − B11 T ) B12 .

2 Problem Formulation and Preliminaries We pose a generalized robust “control” problem that does not, in fact, involve feedback around the designed controller. The absence of feedback simply means that genuine feedback control problems are not addressed by this work, but problems such as robust input design/shaping and robust ﬁltering/estimation are covered. The lack of feedback also means that the resulting minimax optimization is convex (though inﬁnite dimensional). The focus of this paper is a design algorithm for such problems, as well as the application to robust H∞ linear estimation. The system considered is shown in Figure 1. All signals are multivariable, though for clarity we will not be overly speciﬁc in their dimensions. Nothing is assumed to be scalar, so care in manipulating the noncommuting multivariable operators will be maintained. For ease of expression, we denote the dimensions of y and u as ny and nu , respectively. The system consists of a known, stable system G, in feedback with an unknown matrix Δ and a to-be-designed linear controller F . The unknown matrix Δ takes values in

Optimal, Worst Case Filter Design via Convex Optimization

295

✲ Δ p e ✛

G y

✛ q ✛ ✛

d

u

✲ F Fig. 1. General Robust Design Problem r IrN , δ1c Ik1 , . . . , δSc IkS , Δ1 , . . . , ΔF ] : δir ∈ R, Δ := {diag[δ1r Ir1 , . . . , δN

δic ∈ C, Δi ∈ Cli ×li

(1)

with σ ¯ (Δ) ≤ 1 as an additional constraint. We denote {Δ ∈ Δ : σ ¯ (Δ) ≤ 1} by BΔ . There is an obvious 3-by-3 block partition of G into individual relations between its 3 sets of inputs and 3 sets of outputs. With G already assumed stable, it follows that each Gij ∈ RH∞ (of appropriate dimension). We make two main assumptions: Assumption 1 For all Δ ∈ BΔ , (I − G11 Δ)−1 is stable, i.e., in RH∞ . Assumption 2 For all Δ ∈ BΔ , G33 + G31 Δ (I − G11 Δ) cally zero.

−1

G13 is identi-

The ﬁrst assumption is standard for most robust ﬁlter formulations. The second assumption is the “no feedback around F ” assumption, forcing the problem we consider to be an open-loop problem, which accounts for the persistent use of the word “ﬁlter” as opposed to “controller.” Consequently, the closedloop map from d to e, denoted Td→e (G, Δ, F ), is Td→e (G, Δ, F ) = RΔ + UΔ F VΔ , where

−1

RΔ := G22 + G21 Δ (I − G11 Δ) G12 −1 UΔ := G23 + G21 Δ (I − G11 Δ) G13 −1 VΔ := G32 + G31 Δ (I − G11 Δ) G12 .

In most cases we use R (and V , U ) for simplicity, however to emphasize the dependence on Δ or even frequency, we use RΔ or RΔ,θ (and similar for U and V ). Assumption 1 implies that there exist positive constants r¯, u ¯ and v¯ such that for all Δ ∈ BΔ and θ ∈ [0 2π], σ ¯ (RΔ,θ ) ≤ r¯, σ ¯ (UΔ,θ ) ≤ u ¯ and σ ¯ (VΔ,θ ) ≤ v¯. We also make the modest assumption that at some value of Δ ∈ BΔ , U and V are bounded below for all θ ∈ [0 2π], namely Assumption 3 There exists Δ ∈ BΔ and u > 0 and v > 0 such that for all θ ∈ [0 2π], σ(UΔ,θ ) ≥ u and σ(VΔ,θ ) ≥ v.

296

K. Sun and A. Packard

An example of a relevant problem without feedback, consider the robust estimation problem depicted in Figure 2.

d✲

y✲ PΔ

zˆ

F

z

−✲ ✲❄ W +

e ✲

Fig. 2. Uncertain Plant and Filter

Here, an uncertain plant PΔ is driven by a generalized disturbance d. Variable y is measured and processed with the intent to estimating z. The estimation error zˆ − z is weighted by a (frequency-dependent) W , giving the error signal which deﬁnes the performance objective. A goal of estimation would be to minimize (by choice of F ) the worst-case (over all Δ ∈ BΔ ) H∞ gain from d → e. In general, the robust design problem is to minimize, by choice of F , the worst-case (over BΔ ) gain of Td→e (G, Δ, F ). In terms of RΔ , UΔ and VΔ , this is inf max RΔ + UΔ F VΔ ∞ (2) F ∈RH∞ Δ∈BΔ

Let Λopt denote the inﬁmum. Given > 0, the objective is to ﬁnd an F ∈ RH∞ whose cost is within of Λopt . Such an F is called an -suboptimal solution. Deﬁnition 1 ( -suboptimal solution). For > 0, a given F ∈ RH∞ is an -suboptimal solution for (2) if maxΔ∈BΔ RΔ + UΔ F VΔ ∞ ≤ Λopt + . The uncertainty model in this paper consists of uncertain real parameters (which are intuitive, and natural to include in an uncertain model) and uncertain complex matrices. Theorems, proofs and algorithms are presented for that set. Nevertheless, some discussion as to the relevance of constant, complexvalued uncertainty is in order. In the case of H∞ performance bounds, uncertain complex matrix uncertainty also addresses more realistic unmodeled dynamics uncertainty. In fact, when considering robustness of stability, and robustness of H∞ performance, norm-bounded, complex matrix uncertainty is mathematically equivalent to linear, norm bounded, time-invariant realrational dynamic uncertainty. Roughly, an uncertain system is robustly stable to constant, complex-valued, norm-bounded (by 1, using singular value) uncertain parameters, if and only if it is robustly stable to linear, time-invariant, real-rational, dynamic uncertainty with H∞ norm less than 1. Similar statements hold true for robust and worst-case performance as well. Detailed theoretical statements can be found in [28] and references therein.

Optimal, Worst Case Filter Design via Convex Optimization

A important observation is that max RΔ + UΔ F VΔ Δ∈BΔ

297

∞

is convex on F ∈ RH∞ . Indeed, F appears aﬃnely within the norm, so for any ﬁxed uncertainty, the objective is a convex function of F . Furthermore, the pointwise supremum of a family of convex functions is convex ( [7]). Therefore, the robust ﬁltering problem in (2) is a convex optimization problem. Though convex, the optimization problem is inﬁnite dimensional: sets BΔ and [0, 2π] are inﬁnite; RH∞ is inﬁnite dimensional. Practically, we can only obtain suboptimal solutions using ﬁnite and ﬁnite dimensional approximations. Let FΔ denote a ﬁnite subset of BΔ , FΘ a ﬁnite subset of [0 2π] and Φ := {φ1 , φ2 , . . . , φK } a ﬁnite, linear independent set in RH∞ . Deﬁne a ﬁnite dimensional subspace QΦ K

QΦ :=

F : F (z) =

Qk φk (z), Qk ∈ Rnu ×ny

.

(3)

k=1

For ρ > 0, let Bρ QΦ := {F ∈ QΦ : F ∞ ≤ ρ}, and Qρ,Φ := {F ∈ QΦ : σ ¯ (Qk ) ≤ ρ, k = 1, . . . , K}. A ﬁnite dimensional approximation of problem (2), parameterized by ρ, Λρ,Φ,FΔ ,FΘ :=

inf

max max σ ¯ (RΔ,θ + UΔ,θ F (θ)VΔ,θ ) .

F ∈Qρ,Φ Δ∈FΔ θ∈FΘ

(4)

Since Qρ,Φ is compact, and FΔ and FΘ are ﬁnite, the inﬁmum is achieved. ∗ Denote a minimizer by Fρ,Φ,F . In Section 4.2, we show that the minΔ FΘ imization can be carried out with semideﬁnite programming. For this ﬁnite dimensional relaxation to be theoretically useful, we need to show that for any > 0, there exists ρ and ﬁnite sets FΔ , FΘ and Φ such that the associated ∗ Fρ,Φ,F is -suboptimal for (2). Δ FΘ For the remainder of the paper, we always take φk (z) := z k−1 , and denote ∗ Φ by its cardinality K, using QK , Bρ QK , Qρ,K , Λρ,K,FΔ ,FΘ and Fρ,K,F Δ ,FΘ ∗ (or even more compactly, Λρ,K,F and Fρ,K,F ) in place of the general notation above. Lemma 1. Bρ QK ⊂ Qρ,K ⊂ B√Kρ QK . The proof is given in Appendix 8.1. A continuity result will be used in Section 3. Lemma 2. The mapping σ ¯ R(·,·) : BΔ × [0, 2π] → C is uniformly continuous on BΔ × [0, 2π], with norm σ ¯ (Δ) + |θ|. Moreover, for ﬁxed ρ > 0 and K, the family of mappings {¯ σ (U(·,·) F (·)V(·,·) ) : F ∈ Qρ,K } is uniformly equicontinuous. Proof . To show RΔ,θ is uniformly continuous on BΔ × [0, 2π], it is enough to show that RΔ,θ is continuous on this set, since BΔ × [0, 2π] is compact and σ ¯ (·) is a continuous mapping. Recall that RΔ,θ = G22 (θ) +

298

K. Sun and A. Packard

G21 (θ)Δ (I − G11 (θ)Δ)−1 G12 (θ). By Assumption 1, each individual term is continuous, so R is continuous. Similar arguments show that UΔ,θ and VΔ,θ are uniformly continuous. For given K, we know the family of functions {ej(k−1)θ , k = 1, 2, . . . , K} is equicontinuous. By deﬁnition, for any F ∈ Qρ,K , each Qk is uniformly bounded, so the family of functions Qρ,K is also uniformly equicontinuous. So we conclude that the family of functions {UΔ,θ F (θ)VΔ,θ : F ∈ Qρ,K } is uniformly equicontinuous. The following simple lemma gives a procedure for generating ﬁnite covers for a compact set in a metric space with a given radius. The proof is omitted. Lemma 3. In a metric space X, given any compact set A ⊂ X and δ > 0. 0 0 does not ⊂ A that contains a ﬁnite number of points. If FA Pick any set FA satisfy A ⊂ ∪x∈FA B(x, δ), then ﬁnd any y ∈ A that y ∈ ∪x∈FA B(x, δ), and k+1 k update FA := FA ∪ {y}. This procedure stops in ﬁnite steps and generates a ﬁnite cover of A with balls of radius δ. Finally, we make assumptions concerning our algorithmic performance needs of uncertain systems gain analysis. Uncertain systems are represented in linear fractional form, as in Figure 3. We require worst-case gain algorithms with the following convergence property: given S ∈ RH∞ , an uncertain set Δ and > 0, there exist algorithms L and U (which return real numbers) such that L (S, Δ) ≤ max Fu (S, Δ) ≤ U (S, Δ) Δ∈BΔ

and U (S, Δ) − L (S, Δ) < . Algorithm L must also produce a Δ ∈ BΔ and θ ∈ [0, 2π] such that σ ¯ Fu (S(ejθ ), Δ ) ≥ L (S, Δ).

✲ Δ p e ✛

S

✛ q ✛

d

Fig. 3. General Linear Fractional Form

In Appendix II, we outline (tersely and implicitly) a conceptual generalized branch & bound algorithm based on sequences of ﬁnite coverings of BΔ with decreasing radii, to compute such bounds. Actual algorithms, combining upper bounds based on LMIs, lower bounds using problem-speciﬁc search methods, and practical branch & bound schemes are the subject of other work and not the focus here. Refer to [2], [16], [17] and [31] for more details on this subject.

Optimal, Worst Case Filter Design via Convex Optimization

299

3 -Suboptimal Filters via Finite Dimensional Relaxation In this section, we show that the ﬁnite dimensional approximations of equation (4) can be used to get suboptimal solutions of (2). First, modify the original problem in (2) to only consider ﬁlters in QK , ΛK := inf

max RΔ + UΔ F VΔ

F ∈QK Δ∈BΔ

(5)

∞

An -suboptimal solution to (5) is any F ∈ QK which achieves a cost within of the inﬁmum, (just as Deﬁnition 1 deﬁnes suboptimal solutions to (2)). Theorem 1. The inﬁmum for ΛK is achieved. 2¯ r , then using the lower bounds on U Proof . If F ∈ QK satisﬁes F ∞ > uv and V and the upper bound on R (assumptions 3 and 1), we have

max

Δ∈BΔ

RΔ + UΔ F VΔ

∞

> max RΔ Δ∈BΔ

∞

and hence such an F is “worse” than simply taking F = 0. Consequently, for 2¯ r , all ρ > uv ΛK =

inf

max RΔ + UΔ F VΔ

F ∈Bρ QK Δ∈BΔ

∞

(6)

Therefore, with Bρ QK compact, it follows that the inﬁmum is achieved. ∗ Denote any such minimizer by FK ∈ Bρ QK . Next we show that, given ∗ > 0, there exists K0 < ∞, such that when K > K0 , FK is an -suboptimal solution for (2). To do this, recall a classic problem of uniformly approximating functions in H∞ by functions in QK .

Theorem 2. (from [22, 25]) Suppose r > 1 and η > 0 are given. Let U := {z : |z| < r} ⊂ C denote an open disk. For any X that is analytic on U , and supz∈U σ ¯ (X(z)) ≤ η, then inf F ∈QK X − F ∞ ≤ rηK . Using this result, we can obtain an error bound for the robust ﬁltering by using QK rather than RH∞ : Theorem 3. For every K, Λopt ≤ ΛK . Moreover ΛK → Λopt as K → ∞. Proof . Λopt ≤ ΛK is trivial because RH∞ ⊃ QK . For any > 0, let F 0 ∈ RH∞ be an 2 -suboptimal solution for (2). Since F 0 ∈ RH∞ , it has only ﬁnitely many poles outside the unit disk D := {z : |z| < 1}. Hence there exists r > 1 and η > 0 such that F 0 is analytic on rD, and bounded by η. Choose K0 so that rKη 0 < 4¯uv¯ . Then for any K ≥ K0 , by Theorem 2, there exists FK ∈ QK , such that F 0 − FK ∞ < 4¯uv¯ . So, for any Δ ∈ BΔ , we have

300

K. Sun and A. Packard

RΔ + UΔ FK VΔ

∞

= RΔ + UΔ F 0 VΔ + UΔ FK − F 0 VΔ ≤ RΔ + UΔ F 0 VΔ ≤ Λopt + ≤ Λopt +

2

∞

+ UΔ FK − F 0 VΔ ∞

+u ¯v¯ FK − F

0

∞

∞

3 . 4

This is true for all Δ ∈ BΔ , so for all K ≥ K0 ΛK ≤ max RΔ + UΔ FK VΔ Δ∈BΔ

Since

∞

< Λopt + .

is arbitrary, the result follows.

Next, we show that given and K, we can ﬁnd ρ > 0 and ﬁnite sets ∗ FΔ ⊂ BΔ and FΘ ⊂ [0, 2π], such that Fρ,K,F is an -suboptimal solution for (5). Theorem 4. For any and FΘ such that max

Δ∈BΔ

> 0 and K > 0, there exists ρ > 0 and ﬁnite sets FΔ ∗ RΔ + UΔ Fρ,K,F VΔ

∞

< Λρ,K,F + .

Remark 1. Additional inequalities (which always hold) are useful to record. Λρ,K,F ≤ ΛK ≤ max

Δ∈BΔ

∗ RΔ + UΔ Fρ,K,F VΔ

∞

< Λρ,K,F + ≤ ΛK +

Theorem 4 ∗ (to show Λρ,K,F ≤ ΛK , notice that FK ∈ Bρ QK , and by Lemma 1, Qρ,K ⊃ Bρ QK , furthermore, fewer constraints are used in obtaining Λρ,K,F ). 2¯ r Proof . Choose ρ > uv . By Lemma 2, there exists δ1 > 0, such that if (Δ1 , θ1 ) − (Δ2 , θ2 ) < δ1 , then σ ¯ (RΔ1 ,θ1 − RΔ2 ,θ2 ) < /2. Similarly, there exists δ2 > 0, such that when (Δ1 , θ1 ) − (Δ2 , θ2 ) < δ2 , then σ ¯ (UΔ1 ,θ1 F (θ1 )VΔ1 ,θ1 − UΔ2 ,θ2 F (θ2 )VΔ2 ,θ2 ) < /2 for all F ∈ Qρ,K . Take M δ := min(δ1 , δ2 ). Consider any ﬁnite sets FΔ = {Δi }i=1 ⊂ BΔ , and N δ FΘ = {θj }j=1 ⊂ [0, 2π], such that. 2 -balls around these ﬁnite sets cover BΔ and [0 2π],

BΔ ⊂

B(Δi , δ/2), 1≤i≤M

[0, 2π] ⊂

B(θj , δ/2). 1≤j≤N

Clearly, for any F ∈ Qρ,K , max max σ ¯ (RΔ,θ + UΔ,θ F (θ)VΔ,θ ) ≤ max RΔ + UΔ F VΔ

Δ∈FΔ θ∈FΘ

Δ∈BΔ

∞

.

Then, for every Δ ∈ BΔ and every θ ∈ [0 2π], there exist Δi ∈ FΔ , θj ∈ FΘ such that Δ ∈ B(Δi , δ2 ) and θ ∈ B(θj , 2δ ). It follows that for all F ∈ Qρ,K ,

Optimal, Worst Case Filter Design via Convex Optimization

301

σ ¯ RΔ,θ + UΔ,θ F (θ)VΔ,θ − RΔi ,θj − UΔi ,θj F (θj )VΔi ,θj =σ ¯ RΔ,θ − RΔi ,θj + UΔ,θ F (θ)VΔ,θ − UΔi ,θj F (θj )VΔi ,θj ≤σ ¯ RΔ,θ − RΔi ,θj + σ ¯ UΔ,θ F (θ)VΔ,θ − UΔi ,θj F (θj )VΔi ,θj < . (7) Summarizing (and rearranging) – for every Δ ∈ BΔ and θ ∈ [0 2π], there exist Δi ∈ FΔ , θj ∈ FΘ such that for all F ∈ Qρ,K σ ¯ [RΔ,θ + UΔ,θ F (θ)VΔ,θ ] < σ ¯ RΔi ,θj + UΔi ,θj F (θj )VΔi ,θj + . Maximizing over each set yields that for all F ∈ Qρ,K max RΔ + UΔ F VΔ

Δ∈BΔ

∞

< max max σ ¯ [RΔ,θ + UΔ,θ F (θ)VΔ,θ ] + . Δ∈FΔ θ∈FΘ

(8)

∗ Use F = Fρ,K,F to obtain the result.

To conclude the section, combine Theorem 3 and 4, along with the proof of Theorem 4, yielding Theorem 5. For any > 0, there exists ρ > 0 and K0 < ∞, and for any K > K0 , there exists δ > 0, such that for any ﬁnite sets FΔ ⊂ BΔ and FΘ ⊂ [0, 2π] satisfying ∪Δ∈FΔ B(Δ, δ) ⊃ BΔ and ∪θ∈FΘ B(θ, δ) ⊃ [0, 2π], it follows that ∗ max R + U Fρ,K,F V ∞ ≤ Λopt + . Δ∈BΔ

Based on , this gives a criterion for choosing ﬁlter order K and ﬁnite sets ∗ is -suboptimal. In practice though, this criterion FΔ and FΘ such that Fρ,K,F is conservative (regarding δ and K0 ) and not constructive. In Section 4, we propose a successive ﬁnite dimensional approximation to carry out the design.

4 Design Algorithms and Some Error Bounds In this section, a successive design procedure is proposed, and sets FΔ and FΘ will be reﬁned in each iteration. 4.1 Fixed Order Filter Design In this section, we propose a robust ﬁlter design algorithm for a given ﬁlter order K. For any given > 0, the algorithm yields an -suboptimal ﬁlter to the problem in equation (5). The basic idea is to start with FΔ and FΘ having ∗ a few elements, and design Fρ,K,F via the solution to a ﬁnite dimensional convex optimization problem. Then check the error bounds. If they are close to zero, stop. Otherwise, add some points to FΔ and FΘ , and design the ﬁlter again. This process is repeated until the desired tolerance is reached.

302

K. Sun and A. Packard

Algorithm 1 (Fixed order robust ﬁlter design algorithm) Initialization: Filter order K is given, pick > 0 as the desired 2¯ r suboptimal tolerance; set ρ > uv ; pick ﬁnite sets FΔ ⊂ BΔ and FΘ ⊂ [0, 2π]; ∗ ; Step 1: Solve problem (4) with current FΔ and FΘ , yielding Fρ,K,F ∗ Step 2: Apply worst case gain analysis on RΔ +UΔ Fρ,K,F VΔ with a tolerance ∗ VΔ , Δ , and deﬁne Γ := of 0 := 4 . Let γK,F := U 0 RΔ + UΔ Fρ,K,F γK,F − Λρ,K,F . If Γ ≤ , exit; otherwise, add the points (from L 0 ) Δ 0 and θ 0 to FΔ and FΘ , and repeat Step 1; The explicit formulation of Step 1 as a ﬁnite-dimensional SDP, and convergence of this algorithm are addressed in the following subsections. This algorithm can be thought of as a generalization of the cutting plane method, see [23] and [20]. Its convergence is guaranteed, but there are no bounds on the rate-of-convergence. 4.2 Casting the Finite-Dimensional Problem (Equation 4) as an SDP The ﬁnite dimensional problem of equation (4) is convex, and can be formulated as a semideﬁnite program, [6], using standard linear algebra to convert σ ¯ constraints into matrix deﬁniteness constraints. The ﬁnite dimensional space QK is deﬁned in (3). Let QK := {Q1 , . . . , QK } ∈ RK×nu ×ny denote the “vector” of coeﬃcients. Enumerate the elements of FΔ as {Δ1 , . . . , ΔM } and the elements of FΘ as {θ1 , . . . , θN }. Problem (4) can be written as an SDP, min

t,QK ∈R

t

s.t. Gmn (t, QK ) QTk

ρI Qk ρI

(9) 0 ∀ m = 1, 2, . . . , M and n = 1, 2, . . . , N 0

∀ k = 1, 2, . . . , K.

The matrix Gmn (t, QK ) is given by Gmn (t, QK ) :=

tI Pmn (QK )∗ K Pmn (Q ) tI

where Pmn is Pmn (QK ) := RΔm ,θn + UΔm ,θn

K

Qk φk (θn ) VΔm ,θn . k=1

∗ . Solving (9) yields t∗ = Λρ,K,FΔ ,FΘ , along with the associated Fρ,K,F Δ ,FΘ

Remark 2. This conversion of an aﬃne σ ¯ (·) minimization into an SDP is common in robust control. The SDP in equation (9) is readily solved ( [29]).

Optimal, Worst Case Filter Design via Convex Optimization

303

4.3 Bounds via Worst Case Analysis We next verify that the stopping criteria of Step 1 and 2 will eventually be met. Lemma 4. For any iterations.

> 0, Algorithm 1 will terminate in a ﬁnite number of

Proof . We need to show Γ ≤ 2 in ﬁnite iterations as steps 1-2 of Algorithm 1 are repeated. Notice that (dropping arguments for clarity) Γ = (γK,F − L 0 )+ (L 0 − ΛK,F ) . The ﬁrst term U

0

∗ VΔ , Δ − L RΔ + UΔ FK,F

0

∗ VΔ , Δ RΔ + UΔ FK,F

<

0

=

4

follows from worst case gain calculation. Hence, we will show L

0

∗ RΔ + UΔ FK,F VΔ , Δ − ΛK,F <

4

in ﬁnite iterations when repeating Steps 1-2. It is enough to show L

0

∗ RΔ + UΔ FK,F VΔ , Δ − ΛK,F <

4

in ﬁnite iterations, since ΛK,F ≤ γK,F < L

0

∗ RΔ + UΔ FK,F VΔ , Δ +

4

is always true. By Lemma 2, for 8 , there exists δ, such that if (Δ1 , θ1 ) − (Δ2 , θ2 ) < δ, then σ ¯ (RΔ1 ,θ1 − RΔ2 ,θ2 ) < 8 and σ ¯ (UΔ1 ,θ1 F (θ1 )VΔ1 ,θ1 − UΔ2 ,θ2 F (θ2 )VΔ2 ,θ2 ) < for all F ∈ Qρ,K . If (Δ 0 , θ 0 ) generated by L Δ

0

∈

B(Δ, δ/2),

θ

0

0

satisﬁes B(θ, δ/2)

∈

Δ∈FΔ

8

θ∈FΘ

then L

0

∗ RΔ + UΔ FK,F VΔ , Δ − ΛK,F

≤σ ¯ RΔ 0 ,θ

0

∗ + UΔ 0 ,θ 0 FK,F (θ 0 )VΔ 0 ,θ

0

− ΛK,F ≤

4

,

∗ and the iteration terminates. Conversely, if L 0 RΔ + UΔ FK,F VΔ , Δ − ΛK,F > /4, then either Δ 0 ∈ Δ∈FΔ B(Δ, δ/2), or θ 0 ∈ θ∈FΘ B(θ, δ/2). Without loss of generality, suppose the ﬁrst case, then Δ∈FΔ B(Δ, δ/2) does not cover BΔ , and Δ 0 ∈ BΔ . Because BΔ is compact, by Lemma 3, Steps 1 and 2 will generate a ﬁnite cover of BΔ . Similarly, we can get another ﬁnite cover of [0, 2π]. So we know this procedure guarantees ∗ L 0 RΔ + UΔ FK,F VΔ , Δ − ΛK,F < 4 in ﬁnite iterations, and the proof is complete.

304

K. Sun and A. Packard

Remark 3. The stopping criterion is guaranteed to eventually be satisﬁed, using (in the proof) a continuity argument and the algorithmic procedure which generates ﬁnite covers of BΔ and [0, 2π] for any given radius. This is a merely a suﬃcient condition, and for the stopping criterion to be met in practice, the actual number of elements in FΔ and FΘ may be quite small. 4.4 A Lower Bound The simplest lower bound on the achievable performance is obtained by relaxing the analyticity constraints on F , and simply allowing F to vary arbitrarily with θ. On the ﬁnite sets, this is ηF := max

min

max σ ¯ (RΔ,θ + UΔ,θ QVΔ,θ )

θ∈FΘ Q∈Cnu ×ny Δ∈FΔ

This is easily cast as an SDP as well. We have the chain of inequalities: ηF ≤ Λopt ∗ ≤ max RΔ + UΔ Fρ,K,F VΔ Δ∈BΔ

≤U

0

∞

∗ RΔ + UΔ Fρ,K,F VΔ , Δ = γK,F

∗ Let Ψ := γK,F −ηF . If Ψ < , then Fρ,K,F is -suboptimal for (2), and therefore Ψ serves as a potential stopping criteria as the ﬁlter order (K) is increased. However, since ηF is based on relaxing the analytic constraint on F , there is no guarantee that Ψ → 0 as K → ∞. It would be desirable to extend the strong duality results of Ghulchak and Rantzer [13] but we have been unable to do so.

5 Suboptimal Filters The ﬁlter design algorithm in Section 4 relies on converging worst case gain upper and lower bound algorithms U (·) and L (·). For given problem instances, obtaining arbitrarily tight bounds may require large computational eﬀorts. In this section, we show that if only suboptimal ﬁlters are required, then less ineﬃcient, non-converging worst-case gain algorithms can be used in the design. Roughly speaking, for a given order K, the ﬁlter will be only as suboptimal as the gap in the worst-case analysis. Suppose Usub (·) and Lsub (·) are worst case gain upper and lower bound algorithms, respectively, satisfying Usub (·) − Lsub (·) ≤ τ on all problem instances, with τ some ﬁxed number. Let ∗ VΔ , Δ − ΛK,F Γsub := Lsub RΔ + UΔ FK,F ∗ Ψsub := Lsub RΔ + UΔ FK,F VΔ , Δ − η.

Optimal, Worst Case Filter Design via Convex Optimization

305

Lemma 5. Given > 0, and worst case gain upper and lower bound algorithms Usub and Lsub , where Lsub also yields a perturbation that achieves the lower bound. If we use Γsub to substitute Γ in Algorithm 1, then for any given K, the algorithm yields a (τ + )-suboptimal solution of K’th order ﬁlter. Proof . Notice that Γsub can be made arbitrarily small by repeating Steps 1-2 of Algorithm 1. The arguments are the same as in the proof of Lemma 4. So we have max

Δ∈BΔ

∗ RΔ + UΔ FK,F VΔ

∞

∗ VΔ , Δ − Γsub − ΛK,F ≤ Usub RΔ + UΔ FK,F

≤τ+ in ﬁnite iterations. This completes the proof.

6 Example The example here is a linear system in state space form with real parameter uncertainties. To transform this back to the form of (1–4) is an easy exercise. Software tools used to solve the optimization problem (in LMIs) are Yalmip (parser) [14] and SeDuMi (solver) [26] running in Matlab. Consider discretetime system: x(k + 1) =

0.7 0.5 + 0.5δ 0 x(k) + d(k) −0.5 0.6 1

y(k) = 1 0.4 x(k) + 0.2 d(k) z(k) = x(k) where δ ∈ [−1, 1]. To illustrate the eﬀects of FIR order, we specify the length of the FIR ﬁlter in this example, and only Step 1)-3) of Algorithm 1 are used. We use stop tolerance Γ ≤ 0.02, and start with FΔ 0 = {0}. When K = 2, the ﬁnal uncertainty set is FΔ = {0, 1, −1} and the worst case performance is 2.724. The two taps are Q1=

0.3477 0.6368 . , Q2 = −0.8640 0.9025

More details are given in the following table: FΔ {0} {0, 1} {0, 1, −1}

Λ2,F ≤ minQ2 maxΔ∈BΔ ||Td→e ∞ ≤ γ2,F Λ2,F γ2,F 0.91 1.72 2.72

8.00 2.84 2.72

306

K. Sun and A. Packard

When K = 5: FΔ

Λ5,F ≤ minQ5 maxΔ∈BΔ ||Td→e ∞ ≤ γ5,F Λ5,F γ5,F

{0} {0, 1} {0, 1, −1}

0.83 1.44 1.73

7.55 3.07 1.73

When K = 25: Λ25,F ≤ minQ25 maxΔ∈BΔ ||Td→e ∞ ≤ γ25,F Λ25,F γ25,F

FΔ {0} {0, 1} {0, 1, −1} {0, 1, −1, −0.50} {0, 1, −1, −0.50, 0.44}

0.83 1.32 1.60 1.61 1.61

7.54 2.91 1.71 1.66 1.62

Table 1 summarizes the performance of FIR ﬁlters with diﬀerent orders. The ﬁrst column shows the lower bound η of Λ, which is calculated with corresponding FΔ and a uniform frequency grid with 50 points. When the ﬁlter order K = 25, Φ = γ − η = 0.11. Table 1. Robust FIR Filters with Diﬀerent Order Upper η FΔ 1.51 {0, 1, −1} 1.51 {0, 1, −1} 1.51 {0, 1, −1, −0.5, 0.44} Lower

K 2 5 25

γK,F 2.72 1.73 1.62

Table 2 compares nominal and worst case performance of the 25th order FIR ﬁlter in this paper with other ﬁlter design techniques. The other methods are: design for the nominal model (δ = 0) miminizing the H∞ norm from d → e, [3]; formulate the worst-case gain minimization as a mixed-µ synthesis, and use the (D, G) − K iteration of [30]; design using LMI techniques from Appendix 8.2, which are a “translation” of an H2 result for robust ﬁltering with time-invariant uncertainty, [11] into an H∞ result (additional discussion is given in the appendix); and, design using LMI techniques, [10], which minimize a bound on the l2 → l2 gain from d to e, in the presence of time-varying uncertainties. Obviously, and as expected, the 25th order FIR ﬁlter outperforms all other ﬁlters in Table 2 in terms of worst case performance.

Optimal, Worst Case Filter Design via Convex Optimization

307

Table 2. Comparison of Various Filters FIR25 Nominal (δ = 0) µ syn. [30] App. 8.2 Robust [10] Nominal 1.61 0.833 1.89 3.10 5.60 Worst case 1.62 7.88 2.91 4.02 6.70

Fig. 4 compares the performance of diﬀerent ﬁlters further. It shows the H∞ norm from disturbance to estimation error as a function of δ. The lowest curve shows the performance of optimal point-wise H∞ ﬁlters, which are designed at each ﬁxed value of δ ∈ [−1, 1]. This curve is a lower bound for all ﬁlters. Over the speciﬁed range, −1 ≤ δ ≤ 1, the robust FIR ﬁlter proposed in this paper achieves the best worst case performance. 8 7 6

Robust filter, SingleLyap

H norm

5 Nominal

∞

4 Robust filter, PD−Lyap

3

µ synthesis 2 FIR 25 1 0 −1

Point−wise −0.5

0

Uncertain Parameter δ

0.5

1

Fig. 4. Comparison of Various Filters

7 Conclusion In this paper, we show that the robust linear ﬁlter design can be cast as a convex optimization problem. The use of ﬁnite dimensional approximations is justiﬁed and an algorithm is proposed to carry out the design. In the limit, the result approaches to the optimal solution. An example shows the eﬀectiveness

308

K. Sun and A. Packard

of the proposed algorithm. Incorporating other constraints (H2 , time-domain) will also lead to convex problems, though convergent worst-case analysis for these types of speciﬁcations remains to be developed. Deriving a convergent lower bound, using perhaps the tools and ideas in Ghulchak and Rantzer, [13], remains the dominant open theoretical (and practical) problem.

Acknowledgments A preliminary version of this work appeared in [27]. This research was supported by NASA Langley, NASA Cooperative Agreement No. NCC-1-337 through a subcontract from the University of Minnesota. Dr. Celeste M. Belcastro was our technical monitor. The authors would also like to thank DARPAs Software Enabled Control Program under USAF contract #F3361599-C-1497 for providing support for this project. We are also grateful to professor de Souza for pointing out reference [11].

8 Appendix 8.1 Proof of Lemma 1 Proof . Let F ∈ Bρ QK , then F K

∞

Qk ej(k−1)θ

≤ ρ, hence we have ∗

(·)

ρ2 I

∀ θ ∈ [0, 2π]

k=1 K

2π 0 K

Qk ej(k−1)θ

∗

(·) dθ

2πρ2 I

k=1 K

k=1 l=1

2π 0

Qk QTl ej(k−l)θ dθ

2πρ2 I.

Notice that 2π 0

ej(k−l)θ dθ =

2π k = l 0 k=l

T So we have K ρ2 I, hence Qk QTk ρ2 I for k = 1, . . . , K, i.e., k=1 Qk Qk F ∈ Qρ,K . This proves Bρ QK ⊂ Qρ,K . To show Qρ,K ⊂ B√Kρ QK , notice

T that if Qk QTk ρ2 I for k = 1, . . . , K, then K Kρ2 I, hence any k=1 Qk Qk √ F ∈ Qρ,K satisﬁes F ∈ B Kρ QK . This completes the proof.

Optimal, Worst Case Filter Design via Convex Optimization

309

8.2 Robust H∞ Filtering with Parameter Dependent Lyapunov functions For uncertain systems lying in a polytope, LMI formulations for robust ﬁltering with a single Lyapunov function are well known [10]. Recently, Geromel formulated robust H2 ﬁltering problem as LMIs using parameter dependent Lyapunov functions, to exploit the time-invariant uncertainty. This appendix generalizes the method provided in [11] to solve H∞ robust ﬁlter problems. All the derivations and results are similar to those in [11], so the presentation is deliberately terse. Note that this method still optimize an upper bound of the true objective. 8.3 Problem Setup and Preliminaries Consider the linear time-invariant discrete-time system x(k + 1) = Ax(k) + Bw(k) y(k) = Cx(k) + Dw(k) z(k) = Lx(k), where x ∈ Rn , w ∈ Rm , y ∈ Rr and z ∈ Rl . Matrices A, B, C, D and L are of appropriate dimensions. Assume that L is known and the time-invariant AB parameters gathered in matrix M := are unknown but belong to the CD given convex polyhedron N

N

M :=

ξi = 1, ξi ≥ 0 .

ξi Mi ,

M (ξ) = i=1

i=1

The robust H∞ ﬁltering problem is to design the following full order LTI ﬁlter F : x ˆ(k + 1) = Af xˆ(k) + Bf y(k) zˆ(k) = Cf xˆ(k)

(10) (11)

The estimation error is e := z−ˆ z. The state-space matrices for the augumented system are A¯ =

A 0 B ¯ = 0. ¯= , C¯ = L −Cf , D ,B Bf C Af Bf D

We use TM (z) to denote the transfer function from w to estimation error e. The H∞ ﬁltering problem is then inf F supM∈M TM ∞ . The goal is to formulate this problem as LMIs by using parameter dependent Lyapunov function. We need some preliminary results to proceed.

310

K. Sun and A. Packard

Lemma 6 (Geromel, [10]). Given M = only if there exists P

0, such  P  P AT   BT 0

that AP P 0 CP

AB , then TM CD

 B 0 0 P CT   I DT  D γI

2 ∞

< γ if and

0.

The suﬃcient part of Lemma 6 can be extended to the case that M ∈ M: Lemma 7. For γ > 0, TM 2∞ < γ for all M ∈ M, if there exists Pi 0, i = 1, . . . , N , and G, such that   Pi Ai G Bi 0  GT ATi G + GT − Pi 0 GT CiT    0. (12)  BT 0 I DiT  i 0 Ci G Di γI N

N

Proof . Let P (ξ) := i=1 ξi Pi , where i=1 ξi = 1 and ξi ≥ 0. Since P (ξ) 0, we have (P (ξ) − G)T P (ξ)−1 (P (ξ) − G) 0, and hence GT P (ξ)−1 G G+ T G − P (ξ). So when (12) is true, we have   P (ξ) A(ξ)G B(ξ) 0  GT A(ξ)T GT P (ξ)−1 G 0 GT C(ξ)T     B(ξ)T 0 I D(ξ)T  0 C(ξ)G D(ξ) γI   P (ξ) A(ξ)G B(ξ) 0  GT A(ξ)T G + GT − P (ξ) 0 GT C(ξ)T    0  B(ξ)T 0 I D(ξ)T  0

C(ξ)G

D(ξ)

γI

Multiplying above inequality by T := diag(I, P (ξ)G−T , I, I) from left and T T from right, we get   P (ξ) A(ξ)P (ξ) B(ξ) 0  P (ξ)A(ξ)T P (ξ) 0 P (ξ)C(ξ)T    0.  B(ξ)T 0 I D(ξ)T  0 C(ξ)P (ξ) D(ξ) γI By Lemma 6, we conclude that TM

2 ∞

< γ for all M ∈ M.

8.4 Filtering Result In this section, the goal is to transform the robust H∞ ﬁltering problem into LMIs. To this end, some change of variables are necessary:

Optimal, Worst Case Filter Design via Convex Optimization

G :=

311

Y ? Z −1 ? . , G−1 := V ? U ?

Notice that we can always calculate blocks “?” in order to have GG−1 = I. Also introduce the following: Af Bf Cf 0

VT 0 := 0 I

−1

QF H 0

UZ 0 0 I

−1

.

Let R := V T U Z, and we are ready for the main result: Theorem 6. For γ > 0, the estimation error transfer function TM 2∞ < γ for all M ∈ M, if the following LMI is satisﬁed   Pi Ji Z T Ai Z T Ai Z T Bi 0  ∗∗ S Y T A + F C + Q  Y T Ai + F Ci Y T Bi + F D i 0 i i i    ∗∗ ∗ T T T T T  Z + Z − Pi Z + Y + R − Ji 0 L −H     0,  ∗∗ ∗  ∗ Y + Y T − Si 0 LT    ∗∗ ∗  ∗ ∗ I 0 ∗∗ ∗ ∗ ∗ ∗ γI where matrices Q ∈ Rn×n , H ∈ Rl×n , F ∈ Rn×r , R, Z, Y ∈ Rn×n and Pi = PiT , Si = SiT , Ji ∈ Rn×n for i = 1, . . . , N . And the ﬁlter parameters are given by Af = QR−1 , Bf = F, Cf = HR−1 . The proof of Theorem 6 can be done by mimicing that of Theorem 5.1 in [11]. 8.5 Worst-Case Analysis Consider an uncertain linear system as shown in Figure 3. With S ﬁxed, a state-space model of the uncertain relation between d and e is of the form 

    xk+1 A B1 B2 xk  pk  =  C1 D11 D12   qk  ek C2 D21 D22 dk

qk = Δpk

where the entries A, B1 , . . . , D22 depend on the state-space models of S. We assume that this uncertain system is well-posed (unique solutions to xk+1 , pk , ek , qk for any xk , dk ) and exponentially stable for all Δ ∈ BΔ . Deﬁne a constant matrix H by the partition implied below   A B1 B2 H11 H12   :=  C1 D11 D12  H21 H22 C2 D21 D22

312

K. Sun and A. Packard

and Δe := {diag [δc I, Δ] : δc ∈ C, Δ ∈ Δ}. Then well-posedness and stability are equivalent to (see [18]) det (I − H11 Δe ) = 0, ∀Δe ∈ BΔe (note: this is equivalent to the spectral radius of A, ρ(A) being less than 1, and I − −1 D11 + C1 ejθ I − A B1 Δ being nonsingular for all 0 ≤ θ ≤ 2π, Δ ∈ BΔ ; or, equivalently, ρ(A) < 1 and I − D11 + C1 (zI − A) for all Δ ∈ BΔ ). Furthermore, max

Δ∈BΔ

Fu (G, Δ)

∞

−1

B1 Δ

−1

∈ RH∞

= max σ ¯ (Fu (H, Δe )) Δe ∈BΔe

So, the worst case H∞ gain of an uncertain dynamic system is the worst case gain of an uncertain constant matrix. Since Δe is also of the general form for Δ in equation (1), we simply focus on a general problem of the form: given a set Δ as in (1), and a complex matrix M , with I − M11 Δ nonsingular for all Δ ∈ BΔ , develop computable, converging lower and upper bounds for the ¯ (Fu (M, Δ)). Note that since BΔ is compact, it worst-case gain, maxΔ∈BΔ σ follows that (I − M11 Δ)−1 and Fu (M, Δ) are bounded on Δ ∈ BΔ . 8.6 Branch and Bound Algorithm We outline the necessary continuity arguments which ensure that “branch & bound”-like algorithms will succeed in tightly estimating the worst-case gain. For c ∈ BΔ and r > 0, deﬁne Qc,r := {x : x = c + rΔ, Δ ∈ BΔ } to denote a ball in Δ (possibly spilling out of BΔ ) with center c and radius r > 0. With M ﬁxed, deﬁne Lc,r := σ ¯ (Fu (M, c)) and Uc,r :=

r + inf α 2 α>0 1

1

subject to σ ¯

1

r 2 M11 (I − cM11 )−1 r 2 r 2 (I − M11 c)−1 M12 1 1 −1 21 r α M21 (I − cM11 ) α Fu (M, c)

< 1. (13)

Since (I − M11 c)−1 is bounded on c ∈ BΔ , it follows that for small enough r, Uc,r is ﬁnite for all c ∈ BΔ , and moreover Lc,r ≤

max

Δ∈Qc,r ∩BΔ

σ ¯ [Fu (M, Δ)] ≤ max σ ¯ [Fu (M, Δ)] ≤ Uc,r . Δ∈Qc,r

To see this, note that if α > 0 is feasible in the deﬁnition of Uc,r , then for all Δ ∈ BΔ it follows (eg., [18]) that σ ¯ [Fu (M, c + rΔ)] < α. The bound on Lc,r is trivial. In a branch & bound approach, we cover BΔ with a ﬁnite union of these balls, BΔ ⊂ ∪N i=1 Qci ,ri , and deﬁne bounds associated with the particular covering as L := max Lci ,ri ≤ max σ ¯ [Fu (M, Δ)] ≤ max Uci ,ri =: U 1≤i≤N

Δ∈BΔ

1≤i≤N

Optimal, Worst Case Filter Design via Convex Optimization

313

(the element of BΔ that achieves L is simply the ci value associated with the maximization). If it is true that Uc,r − Lc,r → 0 uniformly on c ∈ BΔ , as r → 0, then one guarantees ( [2]) that L and U will converge as the covering becomes ﬁner (ie., as the covering radii shrink). A proof verifying the uniform convergence of Uc,r − Lc,r in our situation is given below. Theorem 7. Given > 0, there exists δ > 0, such that if r < δ, c ∈ BΔ , then Uc,r − Lc,r < . Proof . Since σ ¯ (Fu (M, c)) is bounded on c ∈ BΔ , given any > 0, there exist γ < 1 such that for all c ∈ BΔ , σ ¯ (Fu (M, c)) ≤ σ ¯ (Fu (M, c)) +

2

γ = Lc,r +

2

γ.

(14)

It is always true that 1

1

1

r 2 M11 (I − cM11 )−1 r 2 r 2 (I − M11 c)−1 M12 1 1 −1 12 r Lc,r + M21 (I − cM11 ) Lc,r + Fu (M, c)

σ ¯

2

2

1 2

−1

≤σ ¯ r M11 (I − cM11 ) σ ¯

r

1 2

1 2

+σ ¯ r (I − M11 c)−1 M12 +

1 1 M21 (I − cM11 )−1 r 2 Lc,r + 2

+σ ¯

1 Fu (M, c) . (15) Lc,r + 2

Since σ ¯ ((I − M11 c)−1 ) and σ ¯ ((I − cM11 )−1 ) are bounded on c ∈ BΔ , and 2 1 Lc,r + 2 is bounded on c ∈ BΔ by , there exists δ > 0 (and without loss of generality, δ < ), such that if r ≤ δ, 1

1

1

¯ r 2 (I − M11 c)−1 M12 σ ¯ r 2 M11 (I − cM11 )−1 r 2 + σ +¯ σ

1 1 M21 (I − cM11 )−1 r 2 Lc,r + 2

< 1 − γ.

Substituting above and (14) into (15), we have 1

1

σ ¯

1

r 2 M11 (I − cM11 )−1 r 2 r 2 (I − M11 c)−1 M12 1 1 −1 12 r Lc,r + M21 (I − cM11 ) Lc,r + Fu (M, c) 2

< (1 − γ) + γ = 1.

2

So Lc,r + 2 ≥ Uc,r − 2r , hence Uc,r −Lc,r ≤

r 2+2

< and the proof is complete.

References 1. B. Anderson and J. B. Moore. Optimal Filtering. Prentice–Hall, 1979. 2. V. Balakrishnan, S. Boyd, and S. Balemi. Branch and bound algorithm for computing the minimum stability degree of parameter-dependent linear systems. Technical report, ISL, Stanford Univ, 1991.

314

K. Sun and A. Packard

3. G. J. Balas, J. C. Doyle, K. Glover, A. K. Packard, and R. Smith. µ-Analysis and Synthesis Toolbox: For Use with MATLAB. The MathWorks, Inc., 3 edition, 2001. 4. D. P. Bertsekas and I. B. Rhodes. Recursive state estimation for a setmembership description of uncertainty. IEEE Trans. Auto. Contr., AC16(2):117–128, April 1971. 5. S. Boyd, C. Barratt, and S. Norman. Linear controller design: limites of performance via convex optimization. Proceedings of the IEEE, 78(3):529–574, March 1990. 6. S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix Inequalities in System and Control Theory. SIAM, 1994. 7. S. Boyd and L. Vandenberghe. Convex Optimization. 2002. Available via: http://www.stanford.edu/~boyd. 8. M. Dahleh and I. Diaz-Bobillo. Control of Uncertain Systems: A Linear Programming Approach. Prentice Hall, NJ, 1995. 9. M. Fu, C.E. de Souza, and Zhi-Quan Luo. Finite-horizon robust Kalman ﬁlter design. IEEE Trans. Signal Proc., 49(9):2103–12, 2001. 10. J. C. Geromel, J. Bernussou, G. Garcia, and M. C. De Oliveira. H2 and H∞ robust ﬁltering for discrete-time linear systems. SIAM J. Contr. Optim., 38(5):1353–1368, 2000. 11. J. C. Geromel, M. C. De Oliveira, and J. Bernussou. Robust ﬁltering of discretetime linear systems with parameter dependent lyapunov functions. SIAM J. Contr. Optim., 41(3):700–711, 2002. 12. A. Ghulchak and A. Rantzer. Robust control under parametric uncertainty via primal-dual convex analysis. IEEE Trans. Auto. Contr., 47(4):632–636, April 2002. 13. A. Ghulchak and A. Rantzer. Robust control under parametric uncertainty via primal-dual convex analysis. SIAM J. Control Optim., 41(1):253–277, January 2002. 14. J. L¨ ofberg. Yalmip: Yet another LMI parser, 2003. Available via: http://www.control.isy.liu.se/~johanl/yalmip.html. 15. K. M. Nagpal and P. P. Khargonekar. Filtering and smoothing in an H∞ setting. IEEE Trans. Auto. Contr., 36(2):152–166, February 1991. 16. M.P. Newlin and M.P. Young. Mixed mu problems and branch and bound techniques. Int. J. Rob. Nonlinear Contr., 7(2):145–64, 1997. 17. A. Packard, G. Balas, R. Liu, and J. Shin. Results on worst-case performance assessment. In Proc. Amer. Conf. Contr., pages 2425–2427, Chicago, Il, 2000. 18. A. Packard and J. C. Doyle. The complex structured singular value. Automatica, 29:71–109, 1993. 19. F. Paganini and A. Giusto. Robust synthesis of dynamic preﬁlters. In Proc. Amer. Conf. Contr., pages 1314–1318, June 1997. 20. P. Parrilo. On the numerical solution of LMIs derived from the KYP lemma. In Proc. Conf. Decision Contr., pages 2334–2338, Phoenix, Ar, 1999. 21. I. R. Petersen and A. V. Savkin. Robust Kalman Filtering for Signals and Systems with Large Uncertainties. Birkhauser, Boston, 1999. 22. A. Pinkus. n-Widths in Approximation Theory (Ergebnisse der Mathematik und ihrer Grenzgebiete). Berlin: Springer-Verlag, 1985. 23. E. Polak. Optimization: Algorithms and Consistent Approximations. Springer, 1997.

Optimal, Worst Case Filter Design via Convex Optimization

315

24. A. H. Sayed. A framework for state-space estimation with uncertain models. IEEE Trans. Auto. Contr., 46(7):998–1013, July 2001. 25. C. W. Scherer. Multiobjective H2 /H∞ control. IEEE Trans. Auto. Contr., 40(6):1054–1062, June 1995. 26. J. F. Sturm. Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optimization Methods and Software, 11–12:625–653, 1999. Version 1.05 available from http://fewcal.kub.nl/sturm. 27. K. Sun and A. K. Packard. Optimal worst-case H∞ ﬁltering via convex optimization. In Proc. Conf. Decision Contr., pages 1380–1385, 2003. 28. A. Tits and M. Fan. On the small-µ theorem. Automatica, 31(8):1199–1201, 1995. 29. L. Vandenberghe and S. Boyd. Semideﬁnite programming. SIAM Review, 38(1):49–95, March 1996. 30. P.M. Young. Controller design with real parametric uncertainty. Int. Jour. Contr., 65(3):469–509, 1996. 31. P.M. Young, M.P. Newlin, and J.C. Doyle. Computing bounds for the mixed mu problem. Int. J. Rob. Nonlinear Contr., 5(6):573–590, 1995.

Distance Measures, Robust Stability Conditions and Robust Performance Guarantees for Uncertain Feedback Systems George Papageorgiou1 and Alexander Lanzon2 1 2

Honeywell Laboratories, 3660 Technology Drive, Minneapolis, MN 55418, USA. Research School of Information Sciences and Engineering, The Australian National University, Canberra ACT 0200, Australia and National ICT Australia Ltd., Locked Bag 8001, Canberra ACT 2601, Australia.

Summary. Given a nominal plant, a perturbed plant, an uncertainty structure and performance weights, we use robust model validation ideas to deﬁne and compute a measure of the distance between the nominal and perturbed plants. We also deﬁne a stability margin for a feedback system that is related to robust stability and nominal performance, and derive conditions for the stability and bounds for the performance degradation of the perturbed feedback system in terms of the distance measure. These robust stability and robust performance results give the distance measure a feedback interpretation. The simplicity and power of our procedure for computing the distance between two systems is illustrated using a normalized coprime factor uncertainty model to derive results that have already been published in the literature using diﬀerent techniques. All systems considered in this paper are linear timeinvariant.

Keywords: Robust stability, robust performance, distance measures, model validation, feedback systems, gap metric, ν-gap matric, H∞ control, stability margin

1 Introduction When asked to design a controller for a plant, a control engineer will typically identify a number of plant models that approximate the true plant dynamics, and design a controller that achieves the desired robustness and performance objectives with all the plant models. Let us consider an example. The current standard for designing ﬂight controllers is to divide the ﬂight envelope of an aircraft into a number of ﬂight conditions, design a linear time-invariant (LTI) controller for each ﬂight condition and schedule the gains of the LTI controllers to give a full envelope ﬂight controller. The LTI controllers are typically not scheduled on weight and B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 317–344, 2006. © Springer-Verlag Berlin Heidelberg 2006

318

G. Papageorgiou and A. Lanzon

center-of-gravity location, at least for civil aircraft. At each ﬂight condition, the engineer is given a number of aircraft models that correspond to diﬀerent combinations of weight and center-of-gravity location, and is asked to design a LTI controller for all combinations. A control engineer can handle more than one plant models in at least two diﬀerent ways. The ﬁrst way is current practice in industry: 1. Use engineering experience to choose a nominal plant model; 2. Design a controller that achieves the desired robustness and performance objectives with the nominal plant model; 3. Check robustness and performance with all other plant models; and 4. Iterate if necessary. This approach relies on the engineer’s experience for the choices of a nominal plant model and a measure of robustness. An alternative approach that can be more systematic, is to embed all the plant models into a family of plants and then design a controller for the family. A family of plants is characterized by a nominal plant model, an uncertainty structure and one or more weighting functions that specify the desired performance. It must be emphasized that even though our deﬁnition of a family of plants has been taken from H∞ control, the controller for the family does not have to be designed using H∞ synthesis. In this paper, given a nominal plant model, an uncertainty structure, weighting functions and a perturbed plant model (all systems are assumed to be LTI), we develop a generic procedure for computing the size of the smallest family that contains both the nominal and perturbed plants. We deﬁne the distance between the nominal and perturbed plants to be the size of the smallest family. We believe that such a distance measure is of practical interest since it can help an engineer choose a suitable family of plants (nominal plant, uncertainty structure and weighting functions). Given a number of plant models, the idea is to choose the smallest family that contains all models, the presumption being that it will be easier for a controller to achieve the desired closedloop objectives with a small family. Indeed, every distance measure comes with a dual quantity, the so-called stability margin, that captures essential information about the robustness and performance of a feedback system. It is shown in this paper, that the smaller the distance between two plants, the larger the guaranteed residual stability margin and the smaller the guaranteed diﬀerence in closed-loop performance when both plants are in feedback with the same controller. The high-level ideas outlined above are not new. Vidyasagar [1], ElSakkary [2], Georgiou and Smith [3], Qiu and Davison [4], and Vinnicombe [5, 6] have all deﬁned and worked on similar ideas but speciﬁcally for normalized

Distance Measures, RS Conditions and RP Guarantees

319

coprime factor or 4-block uncertainty structures. 3 What is new is the generic procedure for computing the distance between two plants and the robust stability and robust performance guarantees which can handle many diﬀerent uncertainty structures including normalized coprime factor uncertainty that was studied by Georgiou and Smith, and Vinnicombe. In fact, in this paper we show that our procedure captures the relevant results of Vinnicombe. Our approach has been inspired by robust model validation theory (see Davis [8], Poolla et al. [9], Chen and Gu [10], and Newlin and Smith [11]). The objective of robust model validation is, given a family of plants and data from the real plant, to verify that there exists a plant model in the family that interpolates the data. If there doesn’t, then we have invalidated the family and any controller designed with the family is not guaranteed to work. In our case, we do not have real plant data but a perturbed plant model; that is, inﬁnite data. Given a nominal plant, an uncertainty structure, weighting functions and a perturbed plant, the ﬁrst step of our generic procedure is to solve a socalled consistency equation. For example, in the case of input multiplicative uncertainty, we would solve (if a solution exists) PΔ = P (I − W Δ) for all perturbations Δ ∈ RL∞ that satisfy the equation, where P is the given nominal plant, PΔ is the given perturbed plant and W is a given weighting function (the uncertainty structure is implied by the structure of the equation). Solving the consistency equation can be thought of as a model validation problem. Once we have parameterized all admissible Δ, the second step is to compute the L∞ -norm of the smallest perturbation which satisﬁes consistency and this is our distance measure. Given a controller that stabilizes the nominal plant, we also determine conditions for stability and bound the degradation in performance of the perturbed plant in feedback with the same controller, all in terms of our distance measure. This step is based on a powerful lemma by Vinnicombe which essentially generalizes the small gain theorem [6, p. 45, Lemma 1.22]. In fact, much of our work has been inﬂuenced by the ”Cambridge view of feedback control”. In summary, we have used robust model validation ideas to develop a generic procedure for computing the distance between two LTI systems. The procedure works with diﬀerent uncertainty structures and the deﬁned distance measure has a feedback interpretation because we have also derived robust stability and robust performance guarantees in terms of this distance measure. Compared to other work reported in the literature, although the philosophical 3

The works of Georgiou and Smith, and Vinnicombe have been used to design a number of experimental controllers, the most notable one being for a Harrier aircraft that was successfully ﬂight tested (see Hyde [7]).

320

G. Papageorgiou and A. Lanzon

idea is not new, we derive our distance measures in a diﬀerent way and our procedure can systematically handle diﬀerent uncertainty structures. What follows is an outline of the paper. In Section 2, we deﬁne our distance measure, the stability margin, show how to compute the distance measure via a systematic procedure and derive robust stability and robust performance results all generic. In Section 3, we apply our tools to a speciﬁc uncertainty structure that has been published in the literature. In Section 4, we indicate that our procedure works for many uncertainty structures and in Section 5 we conclude the paper. 1.1 Notation Let R denote the set of proper real-rational transfer functions. 4 Also, let P ∗ (s) denote the adjoint of P (s) ∈ R deﬁned by P ∗ (s) = P (−s)T . Let RL∞ denote the space of proper real-rational functions bounded on jR including ∞ and RH∞ denote the space of proper real-rational functions bounded and analytic in the open right half complex plane. Denote also the space of functions that are units in RH∞ by GH∞ (i.e. f ∈ GH∞ ⇔ f, f −1 ∈ RH∞ ). Let Fl (·, ·) (resp. Fu (·, ·)) denote a lower (resp. upper) linear fractional transformation. Also, let · · denote the Redheﬀer star product of two transfer functions with respect to some partition. For a scalar p(s) ∈ R, its winding number wno p(s) is deﬁned as the number of encirclements of the origin made by p(s) as s follows the standard Nyquist D-contour, indented into the right half plane around any imaginary axis poles or zeros of p(s). The ordered pair {N, M }, with M, N ∈ RH∞ , is a right-coprime factorization (rcf ) of P ∈ R if M is invertible in R, P = N M −1 and N and M are right-coprime. Furthermore, the ordered pair {N, M } is a normalized rcf of P if {N, M } is an rcf of P and M ∗ M + N ∗ N = I. Likewise, the ordered pair ˜ M ˜ }, with M ˜,N ˜ ∈ RH∞ , is a left-coprime factorization (lcf ) of P ∈ R if {N, ˜ ˜ −1 N ˜ and N ˜ and M ˜ are left-coprime. FurtherM is invertible in R, P = M ˜ ˜ ˜ M ˜ } is an lcf more, the ordered pair {N, M } is a normalized lcf of P if {N, ∗ ∗ ˜ ˜ ˜ ˜ ˜ ˜ and M M + N N = I. Let {N, M } be a right and {N, M } be a left coprime ˜ , V˜ } be a left factorization of a plant P . Also, let {U, V } be a right and {U coprime factorization of a controller C. Deﬁne G :=

N ˜ := −U ˜ := −M ˜ V˜ , ˜ N ˜ , K := V , K ,G U M

˜ will be referred to as the graph symbols of P and K and K ˜ where G and G will be referred to as the inverse graph symbols of C. 5 4 5

P ∈ R implies that σ(P (∞)) < ∞. Also, P ∈ R is invertible in R if, and only if, P is square and det P (∞) = 0. For normalized graph symbols,

Distance Measures, RS Conditions and RP Guarantees

321

2 A Systematic Procedure for Robust Stability and Robust Performance Analysis 2.1 Objective Given a nominal plant P , an uncertainty structure with weights, and a controller C, derive conditions for the stability of the feedback interconnection of a perturbed plant PΔ with C, and bounds for the degradation in performance when P is replaced by PΔ that depend only on the given data. 2.2 Deﬁnitions Let [P, C] denote the standard feedback interconnection illustrated in Fig. 1. From Fig. 1 and after some algebra, P y (I − CP )−1 −C I = I u

v1 . v2

(1)

Deﬁnition 1. Given a plant P ∈ R and a controller C ∈ R. [P, C] is said to be well-posed if the transfer function in (1) belongs to R, that is, if (I − CP )−1 ∈ R. Also, [P, C] is said to be internally stable if, in addition, the transfer function in (1) belongs to RH∞ . Let H, C denote the standard H∞ synthesis interconnection depicted in Fig. 2. From Fig. 2 and after some algebra,         z H11 0 H12 w H12 y  = H21 0 H22  + H22  C(I − H22 C)−1 H21 −I H22  v1  . (2) u 0 0 I v2 I "

# " # G∗ h ˜ ∗ i h ˜ ∗ i G∗ GG = GG ˜ ˜ = I. G G

y

P

✛

u

✛ v2 ✻

v1 ✲ ❄ ✲ C Fig. 1. The standard feedback interconnection.

322

G. Papageorgiou and A. Lanzon z ✛ y v1 ✲ ❄ -

" # H11 H12 ✛ H21 H22 ✛ u

w

✛ v2 ✻

✲ C

Fig. 2. The standard H ∞ synthesis interconnection.

Deﬁnition 2. Given a generalized plant H ∈ R and a controller C ∈ R. H, C is said to be well-posed if the transfer function in (2) belongs to R, that is, if (I − H22 C)−1 ∈ R. Also, H, C is said to be internally stable if, in addition, the transfer function in (2) belongs to RH∞ . Finally, H is said to be stabilizable if there exists a C such that H, C is internally stable. 6 It will be shown in the sequel, that if [H22 , C] is stable, then the larger the size of Fl (H, C) −1 ∞ , the larger the size of the set of plants that C is −1 guaranteed to stabilize. Therefore, it can be argued that Fl (H, C) ∞ is a measure of the robust stability of a feedback system. Also, the nominal performance of a feedback system, that is the performance with no plant −1 uncertainty, can be related to H and Fl (H, C) ∞ (by performance it is typically meant reference tracking, disturbance rejection and noise rejection). This is because typically the size of any closed-loop transfer function can be bounded above in terms of the size of any weighting functions absorbed in H −1 −1 and Fl (H, C) ∞ . 7 Therefore, Fl (H, C) ∞ is a very important quantity that summarizes the robustness and performance of a feedback system. Deﬁnition 3. Given a plant P ∈ Rp×q , a generalized plant H ∈ R with H22 = P , and a controller C ∈ Rq×p . Deﬁne the stability margin bH (P, C) of the feedback interconnection H, C by: bH (P, C) :=

Fl (H, C) 0,

−1 ∞ ,

if Fl (H, C) ∈ RL∞ and [P, C] is internally stable otherwise.

The choice of generalized plant H and the required size of the stability margin are application speciﬁc. Now, we deﬁne a distance measure between a nominal plant and a perturbed plant. # A B2 is the state-space realization of H22 inherited from a stabilizable and C2 D22 detectable state-space realization of H, then this deﬁnition of ”H is stabilizable” is equivalent to (A, B2 ) being stabilizable and (C2 , A) being detectable [12]. For an example of how nominal performance can be related to H and Fl (H, C) −1 ∞ , see [13, p. 493, Theorem 18.11]. "

6

7

If

Distance Measures, RS Conditions and RP Guarantees

323

Deﬁnition 4. Given a plant P ∈ Rp×q , a generalized plant H ∈ R with H22 = P , and a perturbed plant PΔ ∈ Rp×q . Let the set of all admissible uncertainties that satisfy consistency of equations be given by: Δ = Δ ∈ RL∞ : (I − H11 Δ)−1 ∈ R, PΔ = Fu (H, Δ) . Deﬁne the distance measure dH (P, PΔ ) between plants P and PΔ for the uncertainty structure Fu (H, Δ) by: 8 dH (P, PΔ ) :=

minΔ∈Δ Δ ∞,

∞

,

if Δ = ∅ otherwise.

Note that dH (P, P ) = 0 since P = Fu (H, 0). It will be shown in the sequel, that the smaller the size of dH (P, PΔ ), the smaller the worst-case degradation of the stability margin due to a plant perturbation. Therefore, we will say that dH (P, PΔ ) can be interpreted as the distance between two plants from a feedback perspective. 9 Deﬁnition 5. Given a plant P ∈ Rp×q , a generalized plant H ∈ R with H22 = P , and a perturbed plant PΔ ∈ Rp×q . Deﬁne Δmin :=

{Δ ∈ Δ : Δ ∅,

∞

= dH (P, PΔ )},

if Δ = ∅ otherwise.

The set Δmin is the set of the smallest-sized Δ ∈ RL∞ that satisfy the consistency equation PΔ = Fu (H, Δ). Note that Δmin ⊂ Δ and that Δ = ∅ if, and only if, Δmin = ∅. 2.3 Robust Stability The following theorem gives a necessary and suﬃcient condition for the stability of a perturbed feedback system given a bound on the distance between the nominal and perturbed plants. Theorem 1 (Robust Stability). Given a plant P ∈ Rp×q , a stabilizable generalized plant H ∈ R with H22 = P , a perturbed plant PΔ ∈ Rp×q and a controller C ∈ Rq×p such that dH (P, PΔ ) < bH (P, C), then [PΔ , C] is internally stable ⇔ ∃Δ ∈ Δmin : η(PΔ ) = η(P ) + wno det(I − H11 Δ), (3) 8

9

We are using min instead of inf in the deﬁnition of dH (P, PΔ ) as we know that the minimizer belongs to the set Δ for our use in Section 3. In general, one should use inf and adjust a few technical issues that arise. We have not yet explored whether dH (·, ·) is a metric on Rp×q or some subset of R for any uncertainty structure H, though it can be shown that in some cases it is a metric.

324

G. Papageorgiou and A. Lanzon

where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ . Proof . If dH (P, PΔ ) < bH (P, C), then dH (P, PΔ ) < ∞ and bH (P, C) > 0, and therefore by deﬁnition, Δ = ∅ and [P, C] is internally stable. Δ = ∅ implies that Δmin = ∅. dH (P, PΔ ) < bH (P, C) ⇒

Δ

⇒

Δ Fl (H, C)

∞

< Fl (H, C) ∞

< 1,

−1 ∞

,

∀Δ ∈ Δmin

∀Δ ∈ Δmin .

Therefore, for all Δ ∈ Δmin • • • •

Δ ∈ RL∞ ; Δ Fl (H, C) ∞ < 1; (I − H11 Δ)−1 ∈ R; and PΔ = Fu (H, Δ).

The proof follows from [6, p. 45, Lemma 1.22]. (⇒) Since H is stabilizable and [P, C] is internally stable, from Lemma 1.22 in [6] [PΔ , C] is internally stable ⇒ η(PΔ ) = η(P ) + wno det(I − H11 Δ),

∀Δ ∈ Δmin,

where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ . (⇐) Also from Lemma 1.22 in [6], [PΔ , C] is internally stable ⇐ ∃Δ ∈ Δmin : η(PΔ ) = η(P ) + wno det(I − H11 Δ), where the winding number is evaluated on the aforementioned contour. The robust stability theorem also holds if, instead of supposing that dH (P, PΔ ) < bH (P, C), we suppose that Fl (H, C) ∈ RL∞ , that [P, C] is internally stable and that for all Δ ∈ Δmin σ(Δ(jω)) <

1 , σ(Fl (H, C))(jω)

∀ω.

If bH (P, C) > (≥ 0) and H is stabilizable, then [PΔ , C] is internally stable for all PΔ that belong to the set P = PΔ = Fu (H, Δ) : dH (P, PΔ ) ≤ , Δ ∈ RL∞ , (I − H11 Δ)−1 ∈ R, η(PΔ ) = η(P ) + wno det(I − H11 Δ) .

Distance Measures, RS Conditions and RP Guarantees

325

Therefore, a controller C that achieves a stability margin greater than with a nominal plant P , is guaranteed to stabilize all the plants in the set P. The robust stability theorem will be of practical use if, given a nominal plant P , a generalized plant H and a perturbed plant PΔ , we can readily compute dH (P, PΔ ) and rewrite the winding number condition in terms of only P , H and PΔ , that is, independent of the perturbation Δ ∈ Δmin . What follows is a step-by-step procedure for computing dH (P, PΔ ) and rewriting the winding number condition. Given P ∈ Rp×q , H ∈ R with H22 = P , and PΔ ∈ Rp×q : 1. 2. 3. 4.

Simplify PΔ = Fu (H, Δ) assuming that (I − H11 Δ) is invertible in R; Parameterize the set Δ; Compute dH (P, PΔ ); and Parameterize the set Δmin and rewrite the winding number condition independent of Δ ∈ Δmin .

Our experience from working with the generalized plants described in the next section suggests, that steps 1 and 3 are fairly straightforward but steps 2 and 4 can be much harder. 10 In Section 3, we will apply our procedure for a particular choice H. 2.4 Generalized Plants Considered Given a plant P ∈ R and a controller C ∈ R. Consider the feedback interconnection illustrated in Fig. 3 that will be referred to as the left 4-block interconnection. From Fig. 3,      w1 I −P P z1  z2  =  0 0 I   w2  . y I −P P u In order to pin down the technical machinery, in the remainder of this paper, we will only consider generalized plants that can be derived from Fig. 3. Let 10

There is evidence that dH (P, PΔ ) can also be computed if Δ is a structured uncertainty set. This will be reported in a subsequent paper. z 1 w1

✻ y

w2 z 2

❄ ✛

P

✛- ❄ ✛

✻ u

✲ C Fig. 3. The left 4-block interconnection.

326

G. Papageorgiou and A. Lanzon

z1 , z2

z = Sz

w1 = Sw w, w2

where Sw , Sz ∈ R select and possibly ﬁlter the inputs and outputs of interest, 

H=

Sz

 I −P P  0 0 I  Sw I I I −P P

and

w = Δz,

where Δ ∈ R. If

(4)

I − Sz

I −P Sw Δ 0 0

−1

∈ R, then PΔ ∈ R and

PΔ = Fu (H, Δ) ⇔

PΔ = P + I −P Sw Δ I − Sz

I −P Sw Δ 0 0

⇔

PΔ − P = I −P Sw Δ I − Sz

I 0

PΔ − P =

I 0

⇔ ⇔

I − I −P Sw Δ Sz

I − I −P Sw Δ Sz

I 0

⇔

PΔ − P = I −P Sw ΔSz

⇔

PΔ − P = I −P Sw Δ Sz

−1

Sz

P I

−1

Sz

P I

I −P Sw ΔSz

P I

I −P Sw Δ −1

(PΔ − P ) = I −P Sw ΔSz

P I

P PΔ − P + I 0 PΔ . I

Also, if (I − P C)−1 ∈ R, then Fl (H, C) ∈ R and

(5)

Distance Measures, RS Conditions and RP Guarantees

327

Fl (H, C) = Sz

P I −P C(I − P C)−1 I −P Sw Sw + Sz I 0 0

= Sz

I 0

I −P Sw + Sz

P C(I − P C)−1 I −P Sw I

= Sz

P I C(I − P C)−1 + I 0

= Sz

I − PC PC + 0 C

= Sz

I −P Sw

(I − P C)−1 I −P Sw

I (I − P C)−1 I −P Sw . C

(6)

˜, M ˜ } is a normalized left coprime Table 1. Uncertainty structures considered. {N factorization of P and R ∈ GH ∞ . Uncertainty structure

"Sw# I 0 " # I 0 " # 0 I " # 0 I " # I 0 " # 0 I

Output inverse multiplicative Additive Inverse additive Input multiplicative Output inverse multiplicative and additive Inverse additive and input multiplicative

Sz h i I 0 h i 0I h i I 0 h i 0I I I

Output inverse multiplicative and inverse additive

I

h i I 0

Additive and input multiplicative

I

h i 0I

Left 4-block Left coprime

"

I # ˜ (RM)−1 0

I I

328

G. Papageorgiou and A. Lanzon

Table 1 lists the uncertainty structures that will be considered. The theory and tools developed in this paper readily apply to any uncertainty structure that can be derived from the left and right 4-block interconnections. 11 2.5 Robust Performance The robust stability theorem gives a necessary and suﬃcient condition for the stability of a perturbed feedback system. In addition to checking whether [PΔ , C] is stable, an engineer typically wishes to know the residual stability margin bHΔ (PΔ , C), where HΔ is the generalized plant that corresponds to the perturbed plant PΔ . We will now derive an upper and a lower bound for the stability margin of a perturbed feedback system, and an upper bound for Fl (HΔ , C) − Fl (H, C) ∞ which is a measure of the diﬀerence in performance between the nominal and perturbed feedback systems. To derive the bounds, we will need the following lemma. Lemma 1. Given a nominal plant P ∈ Rp×q , a generalized plant 

H=

Sz

 I −P P  0 0 I  Sw , I I I −P P

where Sw , Sz ∈ R, a controller C ∈ Rq×p and a perturbation Δ ∈ R such that • • •

(I − P C)−1 ∈ R; (I − Fl (H, C) Δ)−1 ∈ R; and (I − H11 Δ)−1 ∈ R.

Let PΔ = Fu (H, Δ), SwΔ = Sw (I − kΔSz Sw )−1 ∈ R for a given k ∈ {0, 1}, 

HΔ =

Sz

 I −PΔ PΔ SwΔ 0 0 I  I I I −PΔ PΔ

and S = (1 − k)Sz Sw , then Fl (HΔ , C) = Fl (H, C) + Fl (H, C) Δ(Fl (HΔ , C) − S).

3 0 P P 7 6 The generalized plant of the right 4-block interconnection is equal to 4 0 I I 5 −I P P " # " # y z1 . Hence, it is a dual problem and = and can be derived from Fig. 1 with z2 u can be treated easily using similar techniques. 2

11

Distance Measures, RS Conditions and RP Guarantees

329

Proof . First consider the case k = 0. We will only prove the case when Sw = I and Sz = I (that is, left 4-block uncertainty) for simplicity, as the proof in the general case is identical (with Sw and Sz carried on either side to obtain the required result).  I −PΔ HΔ =  0 0 I −PΔ  0 I = Fu  0 I  

I 0   = Fu  I  0  I

     I I00 PΔ I  = 0 0 I  + 0 PΔ 0 −I I PΔ I I00     0 0 −I 0 −I I I I 0 I 0 0  , Fu (H, Δ) = Fu H   0 0 0  0 0 I I 0 0 I I 0  −P 0 −P 0 −P

0 0 I 0 I

−P −I −P 0 −P

  I 0 ,Δ I  0

 P I   P , Δ .  I P 

J

By expanding

I −H

det I − H

Δ 0 0 C

Δ 0 0 C

in terms of its Schur complements,

= det(I − H11 Δ) det(I − PΔ C) = det(I − P C) det(I − Fl (H, C) Δ).

Since (I −P C), (I −Fl (H, C) Δ) and (I −H11 Δ) are invertible in R, it follows that det(I − PΔ C)(∞) = 0 and thus (I − PΔ C) is also invertible in R.

330

G. Papageorgiou and A. Lanzon

Fl (HΔ , C) = Fl (Fu (J, Δ) , C) = Fu (Fl (J, C) , Δ)      0 −I I = Fu Fl  0 0 I  H, C  , Δ I I 0      0 −I I = Fl Fu  0 0 I  , Δ , Fl (H, C) I I 0 = Fl

0 I , Fl (H, C) I −Δ Δ

= Fl (H, C) (I − ΔFl (H, C))−1 (I − Δ) = (I − Fl (H, C) Δ)−1 Fl (H, C) (I − Δ). Therefore, Fl (HΔ , C) ∈ R and ⇔

Fl (HΔ , C) = (I − Fl (H, C) Δ)−1 Fl (H, C) (I − Δ) (I − Fl (H, C) Δ)Fl (HΔ , C) = Fl (H, C) (I − Δ)

⇔

Fl (HΔ , C) = Fl (H, C) + Fl (H, C) Δ(Fl (HΔ , C) − I).

This concludes the proof for the ﬁrst case. Now consider the case k = 1. From the ﬁrst case,    I −PΔ PΔ Sz F l   0 0 I  , C  Sw = Fl (H, C) + I −PΔ PΔ      I −PΔ PΔ Fl (H, C) Δ Sz Fl  0 0 I  , C  Sw − Sz Sw  I −PΔ PΔ which can be rearranged as    I −PΔ PΔ Sz F l   0 0 I  , C  Sw = Fl (H, C) (I − ΔSz Sw )+ I −PΔ PΔ    I −PΔ PΔ Fl (H, C) ΔSz Fl  0 0 I  , C  Sw . I −PΔ PΔ The result follows by multiplying both sides of the above equation by (I − ΔSz Sw )−1 and noting that in this case    I −PΔ PΔ Fl (HΔ , C) = Sz Fl  0 0 I  , C  Sw (I − ΔSz Sw )−1 . I −PΔ PΔ

Distance Measures, RS Conditions and RP Guarantees

331

The following theorem bounds the stability margin of a perturbed feedback system and the performance diﬀerence between the nominal and perturbed feedback systems. Theorem 2. [Robust Performance] Given a nominal plant P ∈ Rp×q , a stabilizable generalized plant   I −P P Sz  0 0 I  Sw H= , I I I −P P where Sw , Sz ∈ R, a perturbed plant PΔ ∈ Rp×q and a controller C ∈ Rq×p such that dH (P, PΔ ) < bH (P, C). Assume furthermore that there exists a Δ ∈ Δmin that satisﬁes η(PΔ ) = η(P ) + wno det(I − H11 Δ), where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ . Let SwΔ = Sw (I−kΔSz Sw )−1 ∈ R for a given k ∈ {0, 1},   I −PΔ PΔ Sz SwΔ 0 0 HΔ = I  I I I −PΔ PΔ and S = (1 − k)Sz Sw . Then the following results hold when S ∈ RL∞ : 1. Fl (HΔ , C) ∈ RL∞ and [PΔ , C] is internally stable 2. bHΔ (PΔ , C) − bH (P, C) ≤ Fl (HΔ , C) − S 3. Fl (HΔ , C) − Fl (H, C)

∞

≤

∞

bHΔ (PΔ , C) dH (P, PΔ )

Fl (HΔ , C) − S ∞ dH (P, PΔ ) . bH (P, C)

Proof . We will only consider the case k = 0, as the proof for the other case is identical. If dH (P, PΔ ) < bH (P, C), then dH (P, PΔ ) < ∞ and bH (P, C) > 0, and therefore by deﬁnition, Δ = ∅, Fl (H, C) ∈ RL∞ and [P, C] is internally stable. Δ = ∅ implies that Δmin = ∅. dH (P, PΔ ) < bH (P, C) ⇒

Δ Fl (H, C)

∞

< 1,

∀Δ ∈ Δmin

⇔

0 < 1 − σ(Δ Fl (H, C))(jω),

∀ω, ∀Δ ∈ Δmin

⇒

0 < σ(I − Δ Fl (H, C))(jω),

∀ω, ∀Δ ∈ Δmin

⇔

det(I − Δ Fl (H, C))(jω) = 0,

∀ω, ∀Δ ∈ Δmin .

332

G. Papageorgiou and A. Lanzon

Thus, (I − Δ Fl (H, C))−1 ∈ RL∞ for all Δ ∈ Δmin. Since internal stability of [P, C] implies that (I − P C)−1 ∈ RH∞ and Δ ∈ Δmin implies that (I − H11 Δ)−1 ∈ R, from Lemma 1 and for all Δ ∈ Δmin , (7)

⇔

Fl (HΔ , C) = Fl (H, C) + Fl (H, C) Δ(Fl (HΔ , C) − Sz Sw ) (I − Fl (H, C) Δ)Fl (HΔ , C) = Fl (H, C) (I − ΔSz Sw )

⇔

Fl (HΔ , C) = Fl (H, C) (I − ΔFl (H, C))−1 (I − Δ Sz Sw ).

(8)

Since Δ, (I − Δ Fl (H, C))−1 ∈ RL∞ for all Δ ∈ Δmin and bH (P, C) > 0 implies that Fl (H, C) ∈ RL∞ , if Sz Sw ∈ RL∞ , then from (8), Fl (HΔ , C) ∈ RL∞ for all Δ ∈ Δmin. From Theorem 1 and because by supposition there exists a Δ ∈ Δmin that satisﬁes the winding number condition, [PΔ , C] is internally stable. This concludes the proof of part (a). Inequality (b) follows from the properties of the L∞ -norm and (7). Fl (HΔ , C) Fl (H, C) ⇔

Fl (H, C)

−1 ∞

∞

≤ Fl (H, C) Δ

∞

∞

∞

+

Fl (HΔ , C) − Sz Sw

≤ Fl (HΔ , C)

−1 ∞

∞

+ −1

Fl (HΔ , C) − Sz Sw ∞ Δ ∞ Fl (HΔ , C) ∞ (dividing both sides by Fl (H, C) ∞ Fl (HΔ , C) ⇔

− Fl (HΔ , C) − Sz Sw Fl (HΔ , C)

−1 ∞

−

∞ Fl −1 Fl (H, C) ∞ . ∞

Δ

(HΔ , C)

−1 ∞

∞)

≤

Similarly, Fl (HΔ , C)

−1 ∞

− Fl (H, C)

−1 ∞

≤

Fl (HΔ , C) − Sz Sw

∞

Δ

∞

Fl (HΔ , C)

−1 ∞ ,

∞

Δ

∞

Fl (HΔ , C)

−1 ∞ .

and therefore, Fl (HΔ , C)

−1 ∞

− Fl (H, C)

−1 ∞

≤

Fl (HΔ , C) − Sz Sw

Since bH (P, C) > 0, Fl (H, C) ∞ = 1/bH (P, C) and since Δ ∈ Δmin , Δ ∞ = dH (P, PΔ ). The inequality follows by noting that since Fl (HΔ , C) ∈ RL∞ and [PΔ , C] is internally stable, Fl (HΔ , C) ∞ = 1/bHΔ (PΔ , C). We will now prove inequality (c). Again from the properties of the L∞ norm and (7), Fl (HΔ , C) − Fl (H, C) The result then follows.

∞

≤ Fl (H, C)

∞

Δ

∞

Fl (HΔ , C) − Sz Sw

∞

.

Distance Measures, RS Conditions and RP Guarantees

333

Note that it is straightforward to derive analogues of both inequalities that are valid at each frequency. If Fl (HΔ , C) − S ∞ bHΔ (PΔ , C) ≤ 1, as is the case for additive, inverse additive, left 4-block and coprime factor uncertainties, we can state the following corollary which is a more concise version of the robust performance theorem. Corollary 1. For any nominal plant P ∈ Rp×q with a stabilizable and detectable state-space realization, perturbed plant PΔ ∈ Rp×q and controller C ∈ Rq×p . Let •

• • •

W1 , Sz = 0 W2 and SwΔ = Sw , where W1 , W2 ∈ RL∞ (that is, 0 additive uncertainty); or 0 , Sz = W2 0 and SwΔ = Sw , where W1 , W2 ∈ RL∞ (that is, Sw = W1 inverse additive uncertainty); or Sw = I, Sz = I and SwΔ = Sw (that is, left 4-block uncertainty); or ˜ )−1 (RM , Sz = I and SwΔ = Sw (I − Δ Sz Sw )−1 (that is, coprime Sw = 0 factor uncertainty),   I −P P Sz  0 0 I  Sw H= I I I −P P Sw =

and



HΔ = If

Sz

 I −PΔ PΔ SwΔ 0 0 . I  I I I −PΔ PΔ

∃Δ ∈ Δmin : η(PΔ ) = η(P ) + wno det(I − H11 Δ),

where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ , then bHΔ (PΔ , C) ≥ bH (P, C) − dH (P, PΔ ).

(9)

Furthermore, when bH (P, C) > dH (P, PΔ ), Fl (HΔ , C) − Fl (H, C)

∞

≤

dH (P, PΔ ) . bH (P, C)bHΔ (PΔ , C)

(10)

Proof . First suppose that bH (P, C) ≤ dH (P, PΔ ). Then the right-hand-side of inequality (9) is ≤ 0. But from Deﬁnition 3, bHΔ (PΔ , C) ≥ 0 and hence (9) is automatically true.

334

G. Papageorgiou and A. Lanzon

Now suppose that bH (P, C) > dH (P, PΔ ). The proof follows from Theorem 2, noting that if P has a stabilizable and detectable state-space realization, then H is stabilizable, and that in the case of coprime factor uncertainty −1

SwΔ = Sw (I − ΔSz Sw )

˜ )−1 (RM = 0

˜ )−1 (RM I −Δ 0

−1

belongs to R for all Δ ∈ Δmin because by deﬁnition (I − H11 Δ)−1 =

I−

˜ )−1 (RM Δ 0

−1

belongs to R for all Δ ∈ Δmin . Inequalities (9) and (10) immediately follow from conditions (b) and (c) of Theorem 2 on noting that for each of the cases considered Fl (HΔ , C) − S ∞ bHΔ (PΔ , C) ≤ 1.

3 Left 4-Block Uncertainty Characterization with a Left 4-Block Performance Measure In this section, we illustrate the novel generic procedure described in Section 2 on a left 4-block uncertainty characterization with a left 4-block performance measure and derive results speciﬁc for this case. The results for this particular (and only this particular) case have also been derived via other technical machinery in [5, 6]. Consequently, in this section, we are illustrating that the generic procedure of Section 2 reproduces existing results in a simple systematic manner. We shall indicate in Section 4 that new results, generated by our novel procedure of Section 2 on diﬀerent uncertainty structures and performance measures, are also possible. Towards this end, consider Figure 3 again and note that for this particular case Sw = I and Sz = I, as also seen in Table 1. 3.1 Deﬁne the Robust Stability Margin b(P, C) Plugging in Sw = I and Sz = I in equation (4) gives   I −P P H = 0 0 I  I −P P and hence Fl (H, C) =

I (I − P C)−1 I −P . C

Consequently, straight from Deﬁnition 3, we see that the stability margin b(P, C) for left 4-block uncertainty characterizations is given by:

Distance Measures, RS Conditions and RP Guarantees

 b(P, C) :=

I (I − P C)−1 I −P C

 

335

−1

if [P, C] is internally stable, ∞

0

otherwise.

(11) In this speciﬁc case, Fl (H, C) ∈ RL∞ is dropped in (11) since [P, C] is internally stable automatically guarantees this. 3.2 When Is the Given Data Consistent and What Are All Solutions? Given a nominal plant P and a perturbed plant PΔ , we would now like to derive necessary and suﬃcient conditions for there to exist a Δ ∈ RL∞ satisfying PΔ = Fu (H, Δ) . That is, we would like to know when the given P , PΔ data is consistent with the uncertainty characterization considered, expressed via generalized plant H. Furthermore, provided these necessary and suﬃcient conditions are satisﬁed, we would like to parameterize all solutions Δ that ﬁt the given P , PΔ data. For this speciﬁc case, plugging in Sw = I and Sz = I in equation (5) gives PΔ = Fu (H, Δ) ⇔

PΔ − P = I −P Δ

PΔ I

which can then be equivalently rearranged into PΔ I

⇔

P − PΔ = −I P Δ

⇔ ⇔

˜ ˜ (P − PΔ )MΔ = GΔG M Δ ˜ Δ = GΔG ˜ GG Δ

⇔

˜ Δ=G ˜ G ˜∗ G GG

⇔

˜ Δ= I0 GG

⇔

˜∗ G Δ= G

(12)

˜ G ˜ ∗ GΔ Δ G Δ G∗

˜ G ˜ ∗ GΔ Δ G Δ G∗ ˜Δ ˜ Δ G Q1 GG

Q2 Q3

G∗Δ

˜Δ G GΔ G∗Δ

0 I (13)

where Qi ∈ RL∞ are arbitrary. Consequently, for this speciﬁc case, the given P , PΔ data is always consistent with the uncertainty characterization considered, as there always exists

336

G. Papageorgiou and A. Lanzon

at least one solution for Δ (assuming that (I − H11 Δ)−1 ∈ R). Also, given any P , PΔ pair, there in fact always exist multiple solutions for Δ ∈ RL∞ , and the non-uniqueness in the solution Δ is parameterized above by the arbitrary objects Qi ∈ RL∞ . 3.3 Deﬁne the Solution Set Δ Since PΔ = Fu (H, Δ) is well-posed when det(I − H11 Δ)(∞) = 0, we need to express det(I − H11 Δ)(∞) independently of Δ. To this end, observe that: det(I − H11 Δ) = det I −

I −P Δ 0 0

I −I P Δ 0 ˜ ˜ ˜ −1 M ˜ G ˜ ∗ G Q1 GGΔ GΔ G I+ G∗Δ Q2 Q3 0 ˜Δ ˜ Δ G ˜ −1 Q1 GG M I+ I0 G∗Δ Q2 Q3 0 ˜Δ M ˜ −1 G ˜ Δ I+ Q1 GG ∗ 0 GΔ ˜Δ ˜ −1 Q1 G M I+ (GM −1 MΔ − GΔ ) ∗ GΔ 0 ˜Δ ˜ −1 G ˜ ∗ + M Q1 GM −1 MΔ . G Δ G∗Δ 0

= det I + = det = det = det = det = det

(14)

(15)

Consequently, det(I − H11 Δ)(∞) = 0

⇔

det

˜ −1 ˜ ∗ + M Q1 G Δ 0

G

(∞) = 0.

We are now in a position to deﬁne the solution set Δ, which contains all feasible solutions Δ satisfying PΔ = Fu (H, Δ), for left 4-block uncertainty characterizations. It is clear from Deﬁnition 4 and the preceeding derivations that set Δ reduces to Δ=

˜∗ G Δ= G

˜ Δ Q1 GG Q2 Q3

˜Δ G G∗Δ det

: Qi ∈ RL∞ , ˜ −1 ˜ ∗ + M Q1 G Δ 0

G

(∞) = 0

in this speciﬁc case. Note also that Δmin is never empty in this speciﬁc case ˜ Δ )(∞) < 1. for σ(GG

Distance Measures, RS Conditions and RP Guarantees

337

3.4 Deﬁne the Distance Measure d(P, PΔ ) ˜ ˜ ∗ G and GΔ are all-pass, and Δmin = ∅, it easily follows straight Since G G∗Δ from Deﬁnition 4 that the distance measure d(P, PΔ ) for left 4-block uncertainty characterizations is given by: d(P, PΔ ) = min Δ Δ∈Δ

= =

∞

˜ Δ Q1 GG Q2 Q3

min

˜∗ G G

min

˜ Δ Q1 GG Q2 Q3

Qi ∈RL∞

Qi ∈RL∞

˜ Δ = GG

∞

˜Δ G G∗Δ

∞

∞

.

(16)

Note that the minimizer belongs to the set Δ because we have assumed that ˜ Δ )(∞) < 1. σ(GG 3.5 Write the Winding Number Condition Independent of Δ Now, we need to rewrite winding number condition (3) independently of Δ. This will give us a necessary and suﬃcient condition for robust stability as presented in Theorem 1 that is calculable from the given data as it is independent of Δ. Towards this end, note that for left 4-block uncertainty characterizations, winding number condition (3) reduces to η(PΔ ) − η(P ) = wno det

˜Δ G G∗Δ

˜ −1 ˜ ∗ + M Q1 G Δ 0

GM −1 MΔ

(17)

via equation (15). Therefore, ˜Δ ˜ −1 G ˜ ∗ + M Q1 G G Δ G∗Δ 0 ˜ ΔM ˜ −1 Q1 G ˜ΔG I −M = wno det . ∗ ˜ −1 N M Q1 G∗ G

0 = wno det

Δ

Δ

(18)

Before we proceed in massaging this winding number condition further, note that [14] proved that in 4-block uncertainty characterizations (as the case considered here), I ≥ 1. (I − P C)−1 I −P C ∞ Hence b(P, C) ≤ 1. Consequently, in this speciﬁc case, d(P, PΔ ) < 1 whenever the supposition d(P, PΔ ) < b(P, C) (see Theorem 1) is enforced. Then, since

338

G. Papageorgiou and A. Lanzon

˜ Δ d(P, PΔ ) = GG

∞

< 1 and σ G∗Δ G

2

˜ = 1 − σ GG Δ

2

on the jω-axis [6,

(G∗Δ G)−1

∈ RL∞ . Exploiting this fact, we can use p. 121], it follows that Schur complements to rewrite equation (18) as 0 = wno det

˜ −1 Q1 0 ˜ ΔM ˜ −1 Q1 − G ˜ Δ G(G∗ G)−1 N ∗ M I −M Δ Δ 0 G∗Δ G

which is equivalent to wno det(G∗Δ G)

∗ ˜ −1 ˜Δ + G ˜ Δ G(G∗Δ G)−1 NΔ )M Q1 ] = − wno det[I − (M ∗ ˜ ˜ −1 Q1 ] = − wno det[I − MΔ (I + (PΔ − P )M (G G)−1 N ∗ )M Δ

Δ

∗ ˜ −1 ∗ ˜ Δ (I + (PΔ − P )(I + PΔ )M Q1 ] P )−1 PΔ = − wno det[I − M ∗ ∗ −1 −1 ˜ Δ (I + P P )(I + P P ) M ˜ Q1 ] = − wno det[I − M Δ Δ Δ −∗ ∗ −1 −1 ˜ (I + P PΔ ) M ˜ Q1 ] = − wno det[I − M Δ

˜G ˜ ∗ )−1 Q1 ] = − wno det[I − (G Δ ∗ −1 ˜ ˜ ∗ ˜ ˜ = − wno det[(GGΔ ) (GGΔ − Q1 )]

˜G ˜ ∗ ] − wno det[G ˜G ˜ ∗ − Q1 ]. = wno det[G Δ Δ

(19)

Hence, equation (19) is a formulation of the desired winding number condition written independently of Δ so that it can be computed as a necessary and suﬃcient condition for robust stability as presented in Theorem 1. However, for this to be checked, one needs to check the condition over all ˜ Δ Q1 GG ˜ Δ = GG Qi ∈ RL∞ that yield a Δ ∈ Δmin (i.e. satisfy Q2 Q3 ∞ ∞ ˜ −1 Q1 M ˜∗ + and det G G (∞) = 0). Δ 0 Non-uniqueness in Δmin Provides No Extra Freedom The following lemma states that winding number condition (19) can be simpliﬁed further and can be made also independent of Q1 whenever the supposition d(P, PΔ ) < b(P, C) (see Theorem 1) is enforced.12 This makes computation of the winding number test considerably easier. ˜ GΔ , G ˜ Δ deﬁned in Section 1.1 Lemma 2. Given normalized graph symbols G, ˜ and GGΔ ∞ < 1. Then ˜G ˜ ∗Δ ] = wno det[G ˜G ˜ ∗Δ − Q1 ] wno det[G for all Qi ∈ RL∞ that satisfy 12

˜ Δ The supposition d(P, PΔ ) < b(P, C) implies GG

∞

< 1 via [14].

Distance Measures, RS Conditions and RP Guarantees

det

˜ −1 ˜ ∗ + M Q1 G Δ 0

G

˜ Δ Q1 GG Q2 Q3

(∞) = 0 and

˜ Δ = GG ∞

339

∞

.

Proof . Since ˜ Δ Q1 GG

∞

≤

˜ Δ Q1 GG Q2 Q3

˜ Δ = GG ∞

∞

,

it follows that

⇒

˜ Δ 2 I − GG ˜ Δ G∗Δ G ˜ ∗ ∀ω ∈ R ∪ {∞} Q1 Q∗1 ≤ GG ∞ ∗ ˜∗ ˜ Δ 2 I − GG ˜ ∀ω ∈ R ∪ {∞} σ(Q1 )2 ≤ λ GG Δ GΔ G ∞ (since AA∗ ≤ B ⇒ σ(A)2 ≤ λ(B))

⇔

˜ Δ σ(Q1 )2 ≤ GG

2 ∞

˜ − σ GG Δ

2

∀ω ∈ R ∪ {∞}

2

⇔

˜ Δ ∀ω ∈ R ∪ {∞} σ(Q1 )2 < 1 − σ GG ˜ Δ (since GG < 1 was assumed) ∞ ˜G ˜ ∗Δ σ(Q1 ) < σ G ∀ω ∈ R ∪ {∞}

⇒

2 ˜ ˜G ˜ ∗ 2 = 1 − σ GG via [6, p. 121]) (since σ G Δ Δ ∗ ˜G ˜ Δ − Q1 ) = wno det(G ˜G ˜ ∗Δ ) wno det(G

⇒

(via [6, p. 16]). Consequently, whenever the supposition d(P, PΔ ) < b(P, C) (see Theorem 1) is enforced12 , it follows that equation (19) reduces to wno det(G∗Δ G) = 0.

(20)

That is, Q1 in equation (19) does not yield any extra freedom on top of Vinnicombe’s winding number test [6, Deﬁnition 3.1, p. 119]. Hence, equation (20) is the desired winding number condition written independently of Δ so that it can be computed as a necessary and suﬃcient condition for robust stability as presented in Theorem 1. It is also easily computable. 3.6 State Robust Stability and Robust Performance Theorems In this subsection, we rewrite the generic Robust Stability Theorem 1 and the generic Robust Performance Theorem 2 for this speciﬁc design case. That is, we reduce Robust Stability Theorem 1 and Robust Performance Theorem 2 speciﬁcally for left 4-block uncertainty characterizations with left 4-block performance measures. Theorem 3 (Robust Stability - Four block). Given a plant P ∈ Rp×q , a perturbed plant PΔ ∈ Rp×q and a controller C ∈ Rq×p . Deﬁne normalized

340

G. Papageorgiou and A. Lanzon

˜ GΔ , G ˜ Δ as in Section 1.1, a stability margin b(P, C) as graph symbols G, G, in (11) and a distance measure d(P, PΔ ) as in (16). Furthermore, suppose d(P, PΔ ) < b(P, C). Then [PΔ , C] is internally stable ⇔ wno det(G∗Δ G) = 0, where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ . Proof . Trivial reduction of Theorem 1. Note that the supposition that “H is stabilizable” is automatically fulﬁlled in this speciﬁc design case, and hence does not need to be independently enforced, since H has the following special form (i.e. P repeated in a special way):   I −P P H = 0 0 I . I −P P Similarly, we now reduce Robust Performance Theorem 2 speciﬁcally for left 4-block uncertainty characterizations with left 4-block performance measures. Theorem 4 (Robust Performance - Four block). Given the suppositions of Theorem 3 and furthermore assuming d(P, PΔ ) < b(P, C) and wno det(G∗Δ G) = 0, where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ . Then |b(PΔ , C) − b(P, C)| ≤ d(P, PΔ ) (21) and Fl (HΔ , C) − Fl (H, C) where



 I −P P H = 0 0 I  I −P P

∞

≤

d(P, PΔ ) , b(P, C)b(PΔ , C)

(22)

 I −PΔ PΔ HΔ =  0 0 I . I −PΔ PΔ 

and

Proof . Trivial reduction of Theorem 2 on noting that Fl (HΔ , C) − Sz Sw

∞

= Fl (HΔ , C) − I = Fl (HΔ , C) 1 = . b(PΔ , C)

∞

∞

The second equality follows from [13, Lemma 18.6, p. 484] since Fl (HΔ , C) is idempotent in this speciﬁc case. The third equality follows from deﬁnition (11) of b(PΔ , C) on noting that [PΔ , C] is internally stable via Theorem 3.

Distance Measures, RS Conditions and RP Guarantees

341

A tighter robust performance result exploiting case-speciﬁc geometry In this speciﬁc case (i.e. left 4-block uncertainty characterization with a left 4-block performance measure), it is possible to derive a slightly tighter result by exploiting case-speciﬁc geometry. ˜ denote a normalized right (resp. left) inverse To do this, let K (resp. K) ˜ GΔ , G ˜ Δ be deﬁned as in Section graph symbol for controller C and let G, G, 1.1. Then, since Sw = I and Sz = I, it is easy to see that I (I − P C)−1 I −P C ˜ −1 G ˜ = K(GK)

Fl (H, C) =

via (6)

˜ −1 K ˜ = I − G(KG) and similarly ˜ Δ K)−1 G ˜Δ Fl (HΔ , C) = K(G ˜ Δ )−1 K. ˜ = I − GΔ (KG Consequently, using the result of Lemma 1 for this speciﬁc case (i.e. with Sw = I and Sz = I), we get

⇔

Fl (HΔ , C) = Fl (H, C) + Fl (H, C) Δ(Fl (HΔ , C) − I) ˜ Δ K)−1 G ˜ Δ = K(GK) ˜ −1 G ˜ K(G

⇒

−1 ˜ ˜ −1 GΔG ˜ ˜ − K(GK) K Δ (KGΔ ) ˜ Δ K)−1 G ˜ ∗ = (GK) ˜ −1 G ˜ ΔK ˜K ˜ ∗ − (GK) ˜ −1 GG ˜ Δ (KG ˜ Δ )−1 (G

˜ ∗ , substitute (pre-multiply by K ∗ , postmultiply by K ˜ Δ = GΔG ˜ GG Δ via (12)) ⇒

⇔

⇔

˜ ˜∗ ˜ Δ) σ(GG ˜ ∗ ≤ σ(GK ) + ˜ Δ K)−1 G ˜ΔK σ (G ˜ ˜ ˜ Δ) σ(GK) σ(GK) σ(KG (via singular value inequalities) ˜ Δ K)2 1 − σ(G ≤ ˜ Δ K) σ(G

˜ 2 ˜ Δ) 1 − σ(GK) σ(GG + ˜ ˜ ˜ Δ) σ(GK) σ(GK) σ(KG ˜Δ K K ˜∗ (via [6, Lemma 2.2(ii), p. 71] using ZΔ = G ∗ ˜ KK ˜ ∗ satisfying ZZ ∗ = I) satisfying ZΔ ZΔ = I and Z = G 1 − α2 − α 1 − β 2 ≤ γ ˜ Δ K) = σ(KG ˜ Δ ), β = σ(GK), ˜ ˜ Δ )) γ = σ(GG (α = σ(G

β

342

us

G. Papageorgiou and A. Lanzon

⇔

sin(βˆ − α ˆ ) ≤ sin γˆ (ˆ α = arcsin α, βˆ = arcsin β, γˆ = arcsin γ such that ˆ γˆ ∈ [0, π] since α, β, γ ∈ [0, 1] — see [6]) α, ˆ β,

⇔

arcsin β − arcsin α ≤ arcsin γ.

It can be similarly shown that “arcsin α − arcsin β ≤ arcsin γ”, thus giving |arcsin α − arcsin β| ≤ arcsin γ.

Then since ˜ Δ K) = α = σ(G

1 1 ˜ ˜ Δ ), = , β = σ(GK) , γ = σ(GG σ(Fl (HΔ , C)) σ(Fl (H, C))

it follows that |arcsin b(PΔ , C) − arcsin b(P, C)| ≤ arcsin d(P, PΔ )

(23)

on noting that [P, C] and [PΔ , C] are internally stable in Theorem 2.

4 Other Structures The systematic procedure proposed in Section 2 was used in Section 3 on the speciﬁc problem of left 4-block uncertainty characterizations with left 4block performance measures to derive, in a diﬀerent way, results that already existed in the literature [5, 6]. In doing so, we have shown that the proposed systematic procedure captures completely existing results in the literature. This systematic procedure of Section 2 can also be used on several other uncertainty structures and several other performance measures to derive brand new robust stability and robust performance results. It allows us to deﬁne stability margins and distance measures for each speciﬁc uncertainty structure and performance measure considered. This then allows us to give a necessary and suﬃcient winding number condition for stability of a perturbed system, and residual performance guarantees, when a nominal system is replaced by a perturbed system for each speciﬁc uncertainty structure and performance measure considered. We have derived such results (using the outline systematic procedure) for a number of diﬀerent uncertainty structures and performance measures, including to mention a few: Additive, Inverse Additive, Multiplicative, Inverse Multiplicative, Coprime Factor and 4-block uncertainty characterizations with Additive, Inverse Additive, Multiplicative, Inverse Multiplicative, Coprime Factor and 4-block performance measures, and various mixes of the above as also outlined in Table 1. These results are not given here due to space constraints and will be published shortly elsewhere.

Distance Measures, RS Conditions and RP Guarantees

343

5 Conclusions Given a nominal plant model, an uncertainty structure, weights and a perturbed plant model we have deﬁned a measure of distance between the nominal and perturbed plant models that is essentially a measure of diﬀerence between the two plants from a feedback perspective. We have shown how to compute the distance measure using model validation ideas. We have also derived a necessary and suﬃcient condition for the stability of the perturbed feedback system given a bound on the distance between the two plants. Furthermore, we have derived upper and lower bounds for the residual stability margin and an upper bound for a measure of the diﬀerence in performance. These bounds justify our claim that the distance measure is a measure of diﬀerence from a feedback perspective. All the above results are generic and can be applied to a large number of uncertainty structures. To show that our procedure works we have applied it to left 4-block uncertainty with left 4-block performance, a case that has already been studied by Vinnicombe using diﬀerent techniques and we systematically reproduce his results. We have also applied our procedure to other uncertainty structures not discussed in this paper.

Acknowledgements This work has been supported in part by ARC Discovery-Projects Grant DP0342683 and National ICT Australia Ltd. National ICT Australia Ltd. is funded through the Australian Government’s Backing Australia’s Ability initiative, in part through the Australian Research Council. This work has also been supported in part by Honeywell Inc. Internal Research and Development funds. Parts of this research were performed during a month’s visit by the ﬁrst author at the Australian National University and National ICT Australia Ltd.

References 1. M. Vidyasagar, “The graph metric for unstable plants and robustness estimates for feedback stability,” IEEE Transactions on Automatic Control, vol. 29, no. 5, pp. 403–418, May 1984. 2. A. K. El-Sakkary, “The gap metric: Robustness of stabilization of feedback systems,” IEEE Transactions on Automatic Control, vol. 30, no. 3, pp. 240– 247, Mar. 1985. 3. T. T. Georgiou and M. C. Smith, “Optimal robustness in the gap metric,” IEEE Transactions on Automatic Control, vol. 35, no. 6, pp. 673–686, June 1990.

344

G. Papageorgiou and A. Lanzon

4. L. Qiu and E. J. Davison, “Feedback stability under simultaneous gap metric uncertainties in plant and controller,” Systems and Control Letters, vol. 18, no. 1, pp. 9–22, Jan. 1992. 5. G. Vinnicombe, “Frequency domain uncertainty and the graph topology,” IEEE Transactions on Automatic Control, vol. 38, no. 9, pp. 1371–1383, Sept. 1993. 6. ——, Uncertainty and Feedback: H-inﬁnity loop-shaping and the nu-gap metric. Imperial College Press, 2001. 7. R. A. Hyde, “H-inﬁnity aerospace control design — a VSTOL ﬂight application,” in Advances in Industrial Control Series. Springer-Verlag, 1995. 8. R. A. Davis, “Model validation for robust control,” Ph.D. dissertation, University of Cambridge, UK, 1995. 9. K. Poolla, P. Khargonekar, A. Tikku, J. Krause, and K. Nagpal, “A timedomain approach to model validation,” IEEE Transactions on Automatic Control, vol. 39, no. 5, pp. 951–959, May 1994. 10. J. Chen and G. Gu, Control-Oriented System Identiﬁcation: An H-inﬁnity Approach. John Wiley & Sons, Inc., 2000. 11. M. P. Newlin and R. S. Smith, “A generalization of the structured singular value and its application to model validation,” IEEE Transactions on Automatic Control, vol. 43, no. 7, pp. 901–907, July 1998. 12. M. Green and D. J. N. Limebeer, Linear Robust Control. Prentice-Hall, Inc., 1995. 13. K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control. PrenticeHall, Inc., 1996. 14. K. Glover and D. McFarlane, “Robust stabilization of normalized coprime factor plant descriptions with H-inﬁnity-bounded uncertainty,” IEEE Transactions on Automatic Control, vol. 34, no. 8, pp. 821–830, Aug. 1989.

Stochastic Modelling over a Finite Alphabet and Algorithms for Finding Genes from Genomes M. Vidyasagar Tata Consultancy Services, Software Units Layout, Madhapur, Hyderabad 500081, INDIA [email protected] Summary. In this paper, we study the problem of constructing models for a stationary stochastic process {Yt } assuming values in a ﬁnite set M := {1, . . . , m}. It is assumed that only a ﬁnite length sample path of the process is known, and not the full statistics of the process. Two kinds of problems are studied, namely: modelling for prediction, and modelling for classiﬁcation. For the prediction problem, in a companion paper it is shown that a well-known approach of modelling the given process as a multi-step Markov process is in fact the only solution satisfying certain nonnegativity constraints. In the present paper, accuracy and conﬁdence bounds are derived for the parameters of this multi-step Markov model. So far as the author is aware, such bounds have not been published previously. For the classiﬁcation problem, it is assumed that two distinct sets of sample paths of two separate stochastic processes are available – call them {u1 , . . . , ur } and {v1 , . . . , vs }. The objective here is to develop not one but two models, called C and N C respectively, such that the strings ui have much larger likelihoods with the model C than with the model N C, and the opposite is true for the strings vj . Then a new string w is classiﬁed into the set C or N C according as its likelihood is larger from the model C or the model N C. For the classiﬁcation problem, we develop a new algorithm called the 4M (Mixed Memory Markov Model) algorithm, which is an improvement over variable length Markov models. We then apply the 4M algorithm to the problem of ﬁnding genes from the genome. The performance of the 4M algorithm is compared against that of the popular Glimmer algorithm. In most of the test cases studied, the 4M algorithm correctly classiﬁes both coding as well as non-coding regions more than 90% of the time. Moreover, the accuracy of the 4M algorithm compares well with that of Glimmer. At the same time, the 4M algorithm is amenable to statistical analysis.

1 Introduction 1.1 Stochastic Modelling Suppose m is a positive integer, and deﬁne M := {1, . . . , m}. Suppose {Yt } is a stationary stochastic process assuming values in M. The problem studied

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 345–369, 2006. © Springer-Verlag Berlin Heidelberg 2006

346

M. Vidyasagar

in this paper is that of constructing a model for this stochastic process. By itself this is a very old and broad problem, so it is necessary to make the scope of the problem more precise. In case the complete statistics of the process are known, and if the aim is to reproduce the complete statistics, then the problem is called one of ‘complete realization.’ The complete realization problem in the context of ﬁnding a hidden Markov model (HMM) is studied in a companion paper [22], and is not discussed here. In [22], attention is restricted to stochastic processes that have an additional property called alpha-mixing (a very weak kind of long-term independence); conditions are derived for such a process to have a hidden Markov model, that are ‘almost’ necessary as well as suﬃcient. In the present paper, attention is focused here on the so-called ‘partial realization’ problem, which comes in two ﬂavours. In the ﬁrst ﬂavour, which may be called ‘exact’ partial realization, one is given the statistics of the process up to some order k, and the objective is to match these statistics exactly. In the second ﬂavour, which may be called ‘inexact’ partial realization, one is given just a sample path of the process, from which one has to infer the statistics of the process, and then construct a model. Let us be more precise in our problem formulation, beginning with the problem of exact partial realization. Suppose an integer k is speciﬁed, and the frequencies of all strings u ∈ Mk (i.e., frequencies of all k-tuples) are speciﬁed. The exact partial realization problem is to construct a stochastic process whose frequencies of all k-tuples exactly match the speciﬁed values. Hereafter, the frequency of occurence of a string is denoted by fu . Thus if u is a string of length k, say u = u1 . . . uk , then fu := Pr{(Yt−k+1 , . . . , Yt ) = u1 . . . uk }. A well known solution to the exact partial realization problem is to model the process as a multi-step Markov model of order k − 1. In other words, it is assumed that if u ∈ M∗ is a string of length larger than k, then Pr{ut |u1 . . . ut−1 } = Pr{ut |ut−k+1 . . . ut−1 }. In short, it is assumed that the probability of ut is not aﬀected by values of ui when i ≤ t − k. Moreover, the transition probability is computed as Pr{ut |ut−k+1 . . . ut−1 } ==

fut−k+1 ...ut−1 ut . fut−k+1 ...ut−1

Since the process is stationary, the above probability is independent of t. The above model, though it is often called a (k − 1)-step Markov model, is also a Markov model over a state space Mk−1 . Suppose u, v ∈ Mk−1 are two states. Then a transition from v to u is possible if and only if the last k − 2 symbols of v (read from left to right) are the same as the ﬁrst k − 2 symbols of u; in other words, it must be the case that v = v1 v2 , u = v2 w for some v2 ∈ Mk−2 , v1 , w ∈ M.

Stochastic Modelling over a Finite Alphabet

347

In this case the probability of transition from the state v to the state u is given by fvw av,u = . (1) fv It is clear that, though the state transition matrix has dimension mk−1 ×mk−1 , every row contains at most m nonzero entries. Such a (k − 1)-step Markov model perfectly reproduces the k-tuple frequencies fu for all u ∈ Mk . In a companion paper [23], it is shown that the multi-step Markov model is the only solution to the problem that satisﬁes certain natural nonnegativity conditions. The multi-step Markov model has some clear advantages over a conventional hidden Markov model (HMM) of the kind studied in [16] when we are dealing with extremely long sequences of the kind encountered in biological problems. To highlight this point, we use the deﬁnition of a HMM used by Anderson [1]. This model can be shown to be equivalent to the conventional HMM; see for example [22]. Given the process {Yt }, we deﬁne a HMM to be another process {Xt } over a ﬁnite alphabet N := {1, . . . , n} such that the joint process {(Yt , Xt )} is Markov, and in addition, Pr{(Yt+1 , Xt+1 )|(Yt , Xt )} = Pr{(Yt+1 , Xt+1 )|Xt }. In other words, not only is the joint process Markov, but both Yt+1 and Xt+1 are random functions of Xt alone and don’t depend on Yt . This condition guarantees that {Xt } is a Markov process by itself. Deﬁne, for every u ∈ M, i, j ∈ N , (u) mij := Pr{Yt+1 = u&Xt+1 = j|Xt = i}, (2) (u)

and deﬁne M (u) to be the n × n matrix whose ij-th element is mij . Let π denote the stationary distribution of the Markov chain {Xt }, and let en denote the column vector of n one’s. Then for every string u = u1 . . . ul , we have fu = πM (u1 ) . . . M (ul ) en . (3) Moreover, the state transition matrix of the Markov chain {Xt } equals A := (u) , and we have πA = π and Aen = en . u∈M M The above formula (3) for computing the likelihood of a string u works quite well when l is a small number, such as 8 or 10. In problems of speech processing, these are typical values of l. However, in biological problems, l is typically 150 or 200. In such a case, the value of the likelihood function is well below the machine zero. In problems of genomics of the kind studied here, a rule of thumb is that the likelihood of a string u of length l over the foursymbol nucleotide alphabet {A, C, G, T } is going to be of the order of 4−l . So if l = 150, then the likelihood is going to be of the order of 2−300 ≈ 10−90 . No machine can handle such small numbers directly. In such a case, the natural remedy is to replace the likelihood by the log-likelihood, which will still be a reasonable number. However, the ‘sum of products’ formula (3) does not lend itself easily to taking the logarithm of both sides. If instead one uses a

348

M. Vidyasagar

multi-step Markov model, then the computation of the log-likelihood is quite straight-forward. Given a long string u of length l, we can write l

Pr{u} =

Pr{Yi = ui |(Yi−k+1 , . . . , Yi−1 ) = ui−k+1 . . . ui−1 } i=l−k+1

=

fui−k+1 ...ui−1 ui . fui−k+1 ...ui−1

As a result, l

log fui−k+1 ...ui−1 ui − log fui−k+1 ...ui−1 .

log Pr{u} = i=l−k+1

Now, in the above summation, all numbers are of reasonable size. Thus it can be said that the traditional HMM is well-suited for problems of speech processing (for example) but not for problems in genomics. Now we come to the problem of inexact partial realization. A problem arises when the probabilities in the equation above are not speciﬁed, but need to be inferred from one or more sample paths u1 , . . . , us . In this case, one can still apply the formula (1), but with the actual (but unknown) frequencies fvu and fv replaced by their empirical estimates. Each of these gives an unbiased estimate of the corresponding probability. But this alone is not enough. It is of interest to derive accuracy and conﬁdence bounds, not just on the estimates of the statistics themselves, but on the parameters of the stochastic model. By adding the assumption the stochastic process under study is α-mixing, we succeed in deriving accuracy and conﬁdence bounds both on the estimates of the statistics, and on the estimated parameters of the multi-step Markov model. So far as the author is aware, such bounds have not been published previously. There are several problems in science and engineering that ﬁt into this framework of constructing a stochastic model based on a set of sample paths. Examples include speech recognition (see e.g., [9]), data compression (e.g., the Lempel-Ziv algorithm [26, 27] and its variants), and picture compression (e.g., the GIF ﬁle format, which is based on an improvement of the LZ algorithm due to Welch [25]). Note that the Lempel-Ziv-Welch algorithm, commonly known as LZW, is normally used for lossless data compression. However, as shown in [12] for example, the algorithm can be reinterpreted in terms of making predictions (which is another way of saying stochastic modelling). 1.2 Algorithms for Finding Genes The motivation of the current author for studying this problem is ﬁnding genes from the genome. The relevant aspects of biology are explained in Section 4, but in essence the problem can be stated as follows (for prokaryote genomes):

Stochastic Modelling over a Finite Alphabet

349

One is given several examples of “coding regions” and several examples of “noncoding regions,” where each region is a fairly long string (150 to 200 long) over the four-symbol nucleotide alphabet {A, C, G, T }. Then a new stretch of the genome, call it w, is given, and the objective is to classify it as either a coding region (gene) or noncoding region (non-gene). This is accomplished by constructing two diﬀerent stochastic models, call them C and N C, for coding regions and for non-coding regions respectively. Then the likelihoods pC (w) and pN C (w) of the new string w being generated by the models C and N C are computed. If pC (w) pN C (w), then w is classiﬁed as a coding region. Similarly, if pC (w) pN C (w), then w is classiﬁed as a non-coding region. If the two likelihoods are of comparable magnitude, then the test is inconclusive. For eukaryotic genomes, there is an additional complication of detecting so-called splice sites, or exon-intron boundaries. But for the purposes of stochastic modelling, the above description still suﬃces. It turns out that the classiﬁcation problem is quite diﬀerent from the stochastic modelling problem, when it comes to implementation. In some ways, classiﬁcation is actually a simpler problem than modelling. Speciﬁcally, in the modelling problem, if a particular string u does not appear as a substring of any of the sample paths, one normally does not take the obvious step of setting the probability of u to zero. This is because, if the ‘true’ probability of u is nonzero but the probability generated by the stochastic model is exactly zero, then the so-called ‘loss function’ is inﬁnite. So, even if a particular string u does not appear anywhere in the sample paths, one normally uses some ad hoc techniques to assign some kind of small, but nonzero, probability to u. See [3], p. 391 et seq. for a discussion of this issue. In contrast, in the classiﬁcation problem one wishes only to determine which model is more (or less likely) to have produced a particular sequence. In this problem, one not only welcomes zero empirical frequencies, but one can also set small empirical frequencies to zero. This distinction is brought in subsequent discussion. In this way, the size of the stochastic model can be made much smaller in the case of classiﬁcation.

2 Parameter Estimation for Multi-step Markov Chain: Accuracy and Conﬁdence Bounds 2.1 Formula for Estimating the Transition Matrix of a Markov Chain There is a vast literature on estimating the mean of a function on the basis of randomy generated samples. The classical Monte Carlo simulation is the starting point of this literature. Over time the literature has expanded to encompass the problem of estimating not just one mean, but inﬁnitely many means at once – the so-called ‘uniform convergence of empirical means’ problem. See [19–21] for detailed discussions of this problem.

350

M. Vidyasagar

However, not much attention seems to have been paid to the problem of estimating the transition matrix of a Markov chain from a sample path. We present below an approach that can be used for this purpose. In this section, we present some accuracy and conﬁdence bounds for a natural method of estimating the transition matrix. To state the problem precisely, suppose {Xt } is a Markov chain over the ﬁnite alphabet N := {1, . . . , n}, state transition matrix A, and stationary distribution π. Suppose n is known but both A and π are unknown. However, we have available a ﬁnite length sample path {x1 , . . . , xl } of this Markov chain. The objective is to form estimates Aˆ and π ˆ for A and π respectively. An obvious way to do this is as follows: Recall that aij = Pr{Xt+1 = j|Xt = i} =

Pr{Xt Xt+1 = ij} . Pr{Xt = i}

Thus we ﬁx a state i, and observe how many times the state i occurs during the stretch X1 , Xl−1 ; call this number µi . Next, observe how many times the pair ij occurs during the stretch (X1 , X2 ), . . . , (Xl−1 , Xl ); call this number νij . With this notation, the estimated value of the element aij of the state transition matrix is given by νij . (4) a ˆij := µi Thus a ˆij equals the fraction of times that the state i is followed by the state j. It is shown in [14] that a ˆij is the maximum likelihood estimate for aij . But there is a small diﬃculty with the above nice-sounding result. Ultimately, when we wish to determine how far the current estimate of the Markov chain is from the ‘true’ Markov chain, we need not only an estimate Aˆ for A, but also an estimate π ˆ for π. Let Aˆ denote the maximum likelihood estimate for the transition matrix, as deﬁned in (4). If we deﬁne π ˆ to be the stationary ˆ then in general π distribution of A, ˆ is not the maximum likelihood estimate of π. Neither is the vector (1/(l − 1))[µ1 . . . µn ]. It is possible to write down necessary conditions satisﬁed by the simultaneous maximum likelihood estimates of both π and A. However, in general these equations are diﬃcult to solve in closed form. There is one special case, however, in which the problem is tractable. Suppose the sample path is a cycle, that is, X1 = Xl . Then in essence we can ‘wrap around’ the sample path and deﬁne the variables as follows: νij now equals the number of occurences of the pair ij in the sequence (X1 , X2 ), . . . , (Xl−1 , X1 ), since Xl = X − 1. Finally, we deﬁne a ˆij :=

νij µi , π ˆi := . µi l

(5)

ˆ In the case of cycles, In this particular case, it can be shown that π ˆ =π ˆ A. therefore, it is easy to obtain a maximum likelihood estimate of the state transition matrix and also obtain, readily, its stationary distribution.

Stochastic Modelling over a Finite Alphabet

351

What if X1 = Xl ? In this case, we use the longest cycle within the sample path. In other words, given a sample path X1 , . . . , Xl , we throw away bits of the sample path at either end until we get a cycle. Then we apply the formula (5) to the cycle. It is not diﬃcult to see that, if the underlying Markov chain is irreducible, then the length of the longest cycle in any sample path approaches inﬁnity almost surely. Hence, by throwing away a little bit of the sample path, we get a maximum likelihood estimate of A as well as its stationary distribution. This allows us to obtain quantitative estimates of how far the ˆ π estimated chain (A, ˆ is from the ‘true’ chain (A, π). The next step is to derive accuracy and conﬁdence bounds for the estimate in (5). This is done via a series of steps. 2.2 Bernstein’s Inequality for I.I.D. and Alpha-Mixing Samples In this section we present some standard inequalities for binary-valued processes when the underlying samples are independent (which they certainly are not, if the samples come from a Markov chain). Then we show how these inequalities can be extended to the case of α-mixing samples (which includes the case where the samples are from a Markov chain). Suppose {Zt } is an i.i.d. process of random variables taking values in [0, 1] and having variance σ 2 . Let {zt } be a realization of this process, and deﬁne 1 ˆ E(Z) := l

l

zi . i=1

ˆ Then E(Z) is called the empirical mean of Z, and converges almost surely to the true mean E(Z) as l → ∞. However, what is of interest to us now is the rate at which this convergence takes place. This is provided by a whole ˆ series of so-called ‘ﬁnite time’ estimates, which tell us just how far E(Z) is from E(Z), and with what conﬁdence. Of the many such inequalities, we use Bernstein’s inequality, which states that ˆ Pr{E(Z) − E(Z) > } ≤ exp{−l 2/[2(σ 2 + /3)]} ∀ > 0,

(6)

ˆ Pr{E(Z) − E(Z) < − } ≤ exp{−l 2/[2(σ 2 + /3)]} ∀ > 0.

(7)

See [7, 15] for a proof of this inequality. If Z is a binary-valued process and Pr{Zt = 1} = p, then we have σ 2 = p(1 − p). Let us substitute = γp and σ 2 = p(1 − p) into (6) and (7). This gives the exponent on the right side as −lγ 2 p2 −lγ 2p2 −3lγ 2p ≤ = , 2[p(1 − p) + γp/3] 8p/3 8 after noting that 1 − p ≤ 1, γ ≤ 1. Hence, if Z is a binary-valued process and Pr{Zt = 1} = p, it follows that

352

M. Vidyasagar

ˆ Pr{E(Z) − E(Z) > γp} ≤ exp(−3lγ 2 p/8),

(8)

ˆ Pr{E(Z) − E(Z) < −γp} ≤ exp(−3lγ 2 p/8).

(9)

Note that the so-called Chernoﬀ bounds are slightly tighter than the above. One Chernoﬀ bound states that ˆ Pr{E(Z) − E(Z) > γp} ≤ exp(−lγ 2 p/3), which results from (8) if the 8 in the denominator is replaced by 9. The other Chernoﬀ bound states that ˆ Pr{E(Z) − E(Z) < −γp} ≤ exp(−lγ 2 p/2), which results from (9) if the 3 in the numerator is replaced by 4. However, for present purposes, the Bernstein inequality is good enough, as it is quite close to the Chernoﬀ bound, and can be readily extended to the case of dependent samples. It is possible to extend Bernstein’s inequality to α-mixing processes; this is done in [24]. Recall the deﬁnition of α-mixing. (See [4, 5] for further details.) 0 Given a stationary stochastic process {Zt }, we deﬁne Σ−∞ to be the σ-algebra generated by the ‘past’ random variables Zt , t ≤ 0, and Σs∞ to be the σ-algebra generated by the ‘future’ random variables Zt , t ≥ s. Then α(s) :=

sup

0 A∈Σ−∞ ,B∈Σs∞

|P (A ∩ B) − P (A)P (B)|.

(10)

Hence the number α(s) measures the extent to which events after time t are independent of events before time 0. Note that α(s) ∈ [0, 1] for all s and that α(s) ≥ α(s + 1). The process {Zt } is said to be α-mixing if α(s) → 0 as s → ∞. Suppose {Zt } is a stationary, α-mixing, zero mean process assuming values in an interval [−F, F ], and we form its empirical mean based on l samples. It is desired to know how close the empirical mean is to the true mean, and with what conﬁdence. In general, if {Zt } is a stationary, α-mixing, and bounded process, but not necessarily zero mean, we can apply the theory below with F := max{ess. sup(Zt − E(Zt )), ess. inf(E(Zt ) − Zt )}. We now quote without proof the relevant result from the paper [24]. Given l samples, choose any integer s ≤ l, and deﬁne r := l/s , the integer part of l/s. Let ˆ qu (l, ) = Pr{E(Z) − E(Z) > }, ˆ ql (l, ) = Pr{E(Z) − E(Z) < − }. Then qu (l, ), ql (l, ) ≤ exp

−r 2 + 4α(s)r 2 2(σ + F/3)

1+ F σ2

r

.

(11)

Stochastic Modelling over a Finite Alphabet

353

Note that (11) reduces to the classical Bernstein inequality if the samples are i.i.d. In this case, we take r = l, s = 1 and note that α(1) = 0 for an i.i.d. process. Now suppose {Zt } is a stationary binary-valued, α-mixing stochastic process. In this case we can take F = 1. Let p denote Pr{Zt = 1}, and pˆ denote its empirical estimate based on l samples. We can use the same trick as earlier to estimate the likelihood that pˆ deviates from p by a percentage of γ. This gives Pr{ˆ p > (1 + γ)p}, Pr{ˆ p < (1 − γ)p} ≤ µ (12) where µ := exp(−3rγ 2 p/8) + 4α(s)r

1 + γp p(1 − p)

r

(13)

Finally, suppose Zt is not binary-valued, but assumes one of m values, and is also α-mixing. Let M denote the set of values assumed by Zt . For each ﬁxed u ∈ M, the process Yu,t = 1 if Zt = u and Yu,t = 0 otherwise is binaryvalued. Moreover, the mean value of Yu,t is precisely Pr{Zt = u}. Hence the preceding theory applies also to processes that assume a ﬁnite number of values. For each u ∈ M, let pu denote Pr{Zt = u}, and let pˆu denote its empirical estimate based on l samples. Then (12) holds with p and pˆ replaced by pu and pˆu respectively. 2.3 Accuracy and Conﬁdence Estimates for Markov Chains Now we are ﬁnally in a position to derive accuracy and conﬁdence bounds for the estimated state transition matrix of a Markov chain. Lemma 1. A Markov chain is α-mixing if and only if its state transition matrix A satisﬁes the ‘strong Perron property,’ that is, A has only a simple eigenvalue at one, and all other eigenvalues of A have magnitude strictly less than one. The above result is well known and in any case is easy to prove. The hypotheses on A ensure that there is a unique, strictly positive stationary distribution π. Moreover, Al → en π as l → ∞. Note that the i-th row of the matrix Al is the probability distribution of the Markov chain after l steps, if the chain is started oﬀ in the initial state i. Thus an α-mixing Markov chain has the property that, irrespective of the starting state, the l-step transition probability distribution approaches the stationary distribution π. Note that α-mixing is a strictly stronger property than ergodicity. In the case of a ﬁnite state Markov chain, the corresponding stochastic process is

354

M. Vidyasagar

ergodic if and only if the state transition matrix is irreducible. In such a case, the ‘ergodic average’ l 1 Al → en π as l → ∞. l i=1 However, the power Al can simply oscillate around without converging to anything in particular. A simple example of this behaviour is the two-state chain with 01 A= . 10 For a Markov chain, it is easy to derive a closed form formula for its α-mixing coeﬃcient. Lemma 2. Let A and π denote respectively the transition matrix and the stationary distribution of a Markov chain. Then   α(l) = max πi  i∈N

n

|(Al )ij − πj | .

(14)

j=1

The formula (14) can be interpreted as follows: The mixing coeﬃcient α(l) equals the maximum value, over the state i, of the 1 -norm of the diﬀerence between the l-step transition probability (Al )i· and the stationary distribution π, multiplied by πi (which is the probability that the initial state is i). Note that the above formula is valid for all Markov chains, α-mixing or otherwise. Now we are in a position to obtain accuracy and conﬁdence bounds. Recall from (4) that a ˆij is the ratio of two quantities, namely fˆij and π ˆi . If the Markov chain is α-mixing, meaning that the state sequence {Xt } is an αmixing process, we can use (12) and (13) to estimate how far π ˆi is from the true value πi . To estimate how far the ‘doublet frequency’ fˆij is from the true frequency fij , we need the next result. Lemma 3. Suppose a process {Yt } is α-mixing. Deﬁne the process of k-tuples {Yt } by Zt := (Yt , Yt−1 , . . . , Yt−k+1 ). Then the process {Zt } is also α-mixing. Moreover, and we have αZ (s) = αY (s − k + 1).

(15)

Proof: It is obvious that 0 0 ∞ Σ−∞ (Z) = Σ−∞ (Y), Σs∞ (Z) = Σs−k+1 (Y).

Equation (15) now follows from the deﬁnition. Hence if the Markov chain {Xt } is α-mixing, so is the sequence of doublets {Xt Xt+1 }. Now we are able to put all the results together.

Stochastic Modelling over a Finite Alphabet

355

Theorem 1. Suppose the Markov chain {Xt } is α-mixing and has the stationary distribution π. Given a cyclic sample path of length l, choose an integer s ≤ l, and deﬁne r := l/s . Deﬁne ci := exp(−3rγ 2 πi /8) + 4α(s)

1 + γπi πi (1 − πi )

dij := exp(−3rγ 2 fij /8) + 4α(s − 1) Then

r

,

1 + γfij fij (1 − fij

(16) r

.

(17)

Pr{ˆ πi > (1 + γ)πi }, Pr{ˆ πi < (1 − γ)πi } ≤ ci , Pr{fˆij > (1 + γ)fij }, Pr{fˆij < (1 − γ)fij } ≤ dij ,

(18) (19)

1+γ 1−γ Pr{ˆ aij > aij }, Pr{ˆ aij < aij } ≤ ci + dij . 1−γ 1+γ

(20)

2.4 Conﬁdence and Accuracy Bounds for Partial Realizations The preceding results extend in a straight-forward manner to the partial realization discussed in (1), for (k − 1)-step Markov chains. The only diﬀerence is that, in estimating the entry aij of a Markov chain, we form the ratio fˆij /πi , whereas in (1) we form the ratio fˆvu /fˆv for v ∈ Mk−1 , u ∈ M. Theorem 2. Let {Yt } be an α-mixing stationary stochastic process assuming values over the ﬁnite alphabet M. For a ﬁxed integer k, let fu denote the frequency of the string u ∈ Mk , and let fˆu denote its empirical estimate based on a sample path of length l. Choose any integer s ≤ l, and deﬁne r := l/s . Deﬁne r 2 1 + γfu , (21) c(k, u) := e−3lγ fu /8 + 4α(s − k + 1)r fu (1 − fu ) c(k − 1, v) := e−3lγ

2

fv /8

+ 4α(s − k + 2)r

1 + γfv fv (1 − fv )

r

.

(22)

Then we have the following relationships: First, for each u ∈ Mk , we have Pr{fˆu > (1 + γ)fu }, Pr{fˆu < (1 − γ)fu } ≤ c(k, u).

(23)

Next, for each v ∈ Mk−1 , we have Pr{fˆv > (1 + γ)fv }, Pr{fˆv < (1 − γ)fv } ≤ c(k − 1, v). Finally, for all u ∈ M

k−1

(24)

, v ∈ M, we have

Pr{

1 + γ fuv fˆuv } ≤ c(k, uv) + c(k − 1, u), > ˆ 1 − γ fu fu

(25)

Pr{

fˆuv 1 − γ fuv } ≤ c(k, uv) + c(k − 1, u). < ˆ 1 + γ fu fu

(26)

356

M. Vidyasagar

3 The 4M Algorithm and Its Analysis 3.1 The 4M Algorithm In this section, we introduce a new algorithm for stochastic modelling called the 4M (Mixed Memory Markov Model) algorithm. The idea of the 4M algorithm is to begin with a multi-step Markov model and to reduce the size of the state space further by using a criterion for determining whether some states are ‘Markovian.’ Recall the essence of the multi-step Markov model. Given frequencies of all k-tuples u ∈ Mk , one deﬁnes a Markov chain with mk−1 states, whereby a transition from a state v to a state u is possible if and only if the last k − 2 symbols of v are the same as the ﬁrst k − 2 symbols of u, i.e., if v = v1 v2 and u = v2 u3 for some v2 ∈ Mk−2 , v1 , u3 ∈ M. In this case, the transition probability is deﬁned as Pr{Yt+1 = u|Yt = v} =

fvu3 . fv

(27)

Note that the size of the state space is mk−1 . In the case of gene ﬁnding (discussed subsequently), we have m = 4 and usually k ≥ 6. Since 46 = 4, 096, the resulting Markov chain is rather large. To reduce the size of the state space, we begin with the following simple observation. Let us switch notation and consider a Markov chain {Xt } evolving over an alphabet N := {1, . . . , n}. Let i, j, k ∈ N and consider the frequency of the string ijk. Clearly for any process (Markov or not), we have fijk = Pr{k|ij} · Pr{ij}. Note that in the above formula we simplify notation by writing Pr{k|ij} := Pr{Yt+1 = k|Yt−1 = i&Yt = j} and so on. Now, because of the Markovian nature of the process, we have Pr{k|ij} = Pr{k|j}, ∀i ∈ N . Hence, if we examine the n × n matrix Fj := [fijk , i, k ∈ N ], it will have rank one. This is because, with j ﬁxed, it looks like  f1j   Fj =  ...  · [Pr{1|j} . . . Pr{n|j}]. 

fnj

Stochastic Modelling over a Finite Alphabet

357

The above reasoning can be applied to an individual state j. Speciﬁcally, suppose we are examining k-tuple frequencies, and suppose v ∈ Ml for some l < k−1. We deﬁne a state v ∈ Ml to be a Markovian state if the associated mk−l−1 × m matrix Fv := [fuvw , u ∈ Mk−l−1 , w ∈ M]

(28)

has rank one. If this is indeed so, then we can conclude that Pr{w|uv} = Pr{w|v} ∀u ∈ Mk−l−1 .

(29)

If the above equation holds, then we can ‘collapse’ all mk−l−1 states of the form uv for all u ∈ Mk−l−1 into a single state v. Recall that when we compute the likelihood of a string w of length s, we write s

Pr{w} =

Pr{wi+1 |wi−k+1 . . . wi } · Pr{w1 . . . wk−1 }. i=k−1

If a substring of the string wi−k+1 . . . wi is Markovian, say wi−l+1 . . . wi , then we can do the substitution Pr{wi+1 |wi−k+1 . . . wi } = Pr{wi+1 |wi−l+1 . . . wi } in the above formula. This is the reason for calling the algorithm ‘mixed memory Markov model’ since diﬀerent (k − 1)-tuples uv have memories of diﬀerent lengths. 3.2 Bounds on the Tail Probability for Markov Chains The preceding theory is applicable provided we use true probabilities in the various computations. However, in setting up the multi-step Markov model, we are using empirically observed frequencies and not true probabilities. Thus it is desirable to analyze the consequences of applying such simpliﬁcations to observed frequencies as opposed to true probabilities. This is done in two diﬀerent ways. In the current subsection, we assume that a state v is Markovian, and that we are given a sample path of ﬁnite length. In this case, the true but unknown matrix of probabilities Fv := [fuvw , u ∈ Mk−l−1 , w ∈ M] is approximated by the matrix of observed frequencies Fˆv := [fˆuvw , u ∈ Mk−l−1 , w ∈ M]. If the string v is a Markovian state, the matrix Fv will have rank one. However, the matrix Fˆv of empirical frequencies will not exactly have rank one, but will

358

M. Vidyasagar

in some sense be ‘close’ to a rank one matrix. The objective of this subsection is to make this statement quantitatively precise using (25) and (26). Note that arbitrarily small perturbations of a rank one matrix (or indeed, any rank-deﬁcient matrix) can result in a full-rank matrix. Hence the emphasis here is on quantifying the notion of a given matrix being close to a rank one matrix, as opposed to the rank itself being close to one. (An entirely diﬀerent approach based on divergence between Markov chains is adopted in the next subsection.) For this purpose, we use the distance measure |fˆuvw − fuvw |.

ρ(Fˆv , Fv ) := u∈Mk−l−1 ,w∈M

Clearly ρ is just the 1 -norm of the vector obtained by stacking all the columns of the diﬀerence Fˆv − Fv on top of each other. Moreover, ρ(Fˆv , Fv ) measures how close the matrix Fˆv is to the rank one matrix Fv . In the sequel, we derive estimates of the ‘tail probability’ that ρ(Fˆv , Fv ) exceeds a given small number. Suppose we form the matrix Fˆv of empirical frequencies. Then, as stated in (25) and (26), we have with conﬁdence 1 − 2c(k, uvw) that 1+γ 1−γ fuvw ≤ fˆuvw ≤ fuvw . 1+γ 1−γ In turn this implies that, with conﬁdence 1 − 2c(k, uv), we can say |fˆuvw − fuvw | ≤

2γ fuvw . 1−γ

(30)

Now suppose (28) is satisﬁed. Then it is easy to see that fuvw = fv . u∈Mk−l−1 ,w∈M

Deﬁne

cv :=

c(k, uvw). u∈Mk−l−1 ,w∈M

Then it readily follows from (30) that Pr{ρ(Fˆv , Fv ) >

2γ fv } ≤ 2cv . 1−γ

(31)

This bound establishes that, if we examine a sample path of an α-mixing Markov chain, and if we compute the matrix Fˆv of empirical frequencies, then ‘tail’ probability of the matrix Fˆv being ‘close’ to a rank one matrix is given by (31).

Stochastic Modelling over a Finite Alphabet

359

3.3 Kullback-Leibler Divergence Rates Between Markov Models While the preceding analysis is mathematically rigorous, it is diﬃcult to implement in practice. To simplify the implementation of the 4M algorithm, we deﬁne a state v ∈ Ml to be a Markovian state if σ2 (Fv ) ≤ σ1 (Fv ),

(32)

where σ1 , σ2 denote respectively the largest and the second largest singular value of a matrix, and is an adjustable parameter. In principle the distance measure ρ(Fˆv , Fv ) can be related to the above criterion, but in practice the estimates will be hopelessly conservative. Instead, in this subsection we derive estimates of how ‘close’ the reduced order mixed memory Markov model is to the original full order (k − 1)-step Markov model. For this purpose, we need a measure of disparity between Markov chains. Such a measure is provided by the Kullback-Leibler (K-L) divergence rate. Suppose p, q are probability distributions on a ﬁnite set {1, . . . , n}. Then the Kullback-Leibler (K-L) divergence H(p q) is deﬁned by n

H(p

pi log(pi /qi ).

q) := i=1

In order for the K-L divergence to be ﬁnite, it is necessary that pi = 0 whenever qi = 0. If this is the case, then we write p q or q p, read as ‘p is dominated by q’ or ‘q dominates p.’ The K-L divergence has a very nice intuitive interpretation. Suppose we observe a sample path {x1 , . . . , xl } of an i.i.d. process, and we know ahead of time that the underlying probability distribution is either p or q. To discriminate between the two competing hypotheses, we use the log-likelihood ratio. The likelihoods of the sample path arising if the underlying distribution is p or q are, respectively n

Lp = i=1

pνi i , Lq =

n i=1

qiνi ,

where νi denotes the number of occurences of i in the sample path. Therefore the log likelihood ratio normalized for the length of the sample is n

(νi /l) log(pi /qi ).

(1/l) log(Lp /Lq) = i=1

Now, suppose the ‘truth’ is that the underlying probability distribution is p. Then the expected value of each νi equals pi l, and the expected value of the normalized log likelihood ratio is precisely H(p q). The preceding argument is ﬁne for i.i.d. processes. Suppose now that we observe a sample path of a stochastic process, and the two competing hypotheses are that the joint law of the process is p or q. Note that now both p

360

M. Vidyasagar

and q are probability measures on the inﬁnite cartesian product of the state space {1, . . . , n}. In theory, we could compute the K-L divergence between the two laws by replacing the ratio pi /qi by the Radon-Nikodym derivative and the summation by an integral. However, except in very trivial cases, the divergence will be inﬁnite and will therefore be a meaningless number. To get around this diﬃculty, we use the K-L divergence rate, deﬁned as follows: For each integer l, let p( l) and q(l) denote the marginal measures of p and q respectively on {1, . . . , n}l . Then the K-L divergence rate is deﬁned as R(p

q) := lim

l→∞

1 H(p(l) l

q(l) ),

assuming that the limit exists. This notion was apparently ﬁrst introduced in [10]. The K-L divergence rate has the same intuitive interpretation as the original K-L divergence. Suppose we observe a sample path of a stochastic process, and we wish to discriminate between two alternate hypotheses, namely: the underlying law is p (the truth), and the underlying law is q. Then the normalized log likelihood ratio asymptotically looks like R(p q). In [17], an explicit formula is given for the K-L divergence rate between two Markov chains over the same state space. Suppose A, B are two stochastic matrices of dimension n × n, and let φ, θ denote their (possibly nonunique) stationary distributions. Suppose both Markov chains are started oﬀ in the initial distribution φ. Let p denote the (stationary) law of the process with the transition matrix A, and let q denote the (possibly nonstationary) law of the process with the transition matrix B. Let ai , bi denote the i-th rows of the matrices A and B respectively, and note that each is a probability distribution. Then we have the following result: Theorem 3. Suppose bij = 0 implies that aij = 0 for all i, j. Let φ∗ denote the ergodic limit l 1 φ∗ := lim Ai . l→∞ l i=1 (Note that φ∗ = φ if A is irreducible.) Then n

R(p

q) =

φ∗i H(ai

bi ).

(33)

i=1

Since a multi-step Markov model is just a one-step Markov model over a larger state space, the above formula readily generalizes to multi-step Markov models. This theorem can be readily applied to derive a simple formula for the K-L divergence rate between the original (k − 1)-step Markov model and the 4M reduced order model.

Stochastic Modelling over a Finite Alphabet

361

Theorem 4. Suppose we are given the frequencies of all k-tuples, and we choose various tuples as ‘Markovian states.’ Let v1 , . . . , vs denote the Markovian states and let l(vi ) < k denote the length of the Markovian state vi . Then the K-L divergence rate between the original (k − 1)-step Markov model and the 4M reduced Markov model is given by s

fuvi log

K= i=1 u∈Mk−1−l(vi ) w∈M

fuvi w fvi · . fuvi fvi w

(34)

Note that, in applying the 4M algorithm, what we are doing is to approximate the ratio fuvi w /fuvi by the ratio fvi w /fvi for each string vi that is deemed to be a Markovian state. Hence the quantity inside the logarithm in (34) should be quite close to one, and its logarithm should be close to zero. The proof of the theorem is fairly easy and is left to the reader. Note that the full (k − 1)-st order Markov model has exactly m nonzero entries in each row labelled by u, and these entries are fuw /fu as w varies over M. One can think of the reduced order model obtained by the 4M algorithm has containing the same mk rows, except that if v is a Markovian state, then the entry in all rows of the form uv are changed from fuvw /fuv to fvw /fv . The vector fu , u ∈ Mk−1 is a stationary distribution of both stochastic matrices. Now the formula (34 readily follows from (33).

4 Application to Gene Finding Algorithms In this section, we present a concrete application of the 4M algorithm by applying it to the problem of ﬁnding genes in prokaryote genomes. 4.1 Introduction to Genomics Let us begin with an introduction to the relevant aspects of biology. All living things are made up of DNA, or deoxyribonucleic acid. In humans and most ‘higher’ (i.e., highly evolved) organisms, DNA is shaped like a double helix, as discovered by Watson and Crick more than ﬁfty years ago. In some ‘lower’ forms of life such a viruses, the DNA is shaped like a circle. In both cases, DNA consists of a sequence of nucleic acids, or nucleotides. There are just four nucleotides, namely: Adenine (A), Cytosine (C), Guanine (G), and Thymine (T). Thus one can think of DNA as just a very long string of symbols coming from the four symbol alphabet N := {A, C, G, T }. In the case of organisms with a double helix structure to the DNA (such as humans), each strand of the helix has a starting point (called the 5’ end) and an end point (called the 3’ end). Each strand has a deﬁnite spatial orientation – it cannot be ‘read backwards.’ The deﬁnite spatial orientation permits us to treat the spatial sequence as a temporal sequence and then to apply methods of time series analysis. The two strands run in opposite directions; thus the

362

M. Vidyasagar

5’ end of one side corresponds to the 3’ end in the other side, and vice versa. Considerations of chemical bonding ensure that if one strand of the DNA has an A in a particular location, in the corresponding location in the other strand must be a T , and vice versa. Similarly, C corresponds to G and vice versa. Thus, if one knows the sequence of nucleotides that make up one strand of the DNA, one would be able to deduce unambiguously what the other strand is. One would have to replace A by T , C by G, and vice versa, and then run the sequence backwards. This is a process known as reverse complementation. The listing of one strand, which is just a very long string over the four symbol nucleotide alphabet, is called the ‘genome’ of an organism. The genome is the ‘lowest’ level description of an organism. 4.2 The Gene Finding Problem The next level of complexity is given by genes. The process of life requires that DNA is transcribed to RNA which in turn produces several proteins. For the present purposes, it is enough to think of RNA as another string over the alphabet of RNA nucleotide symbols {A, C, G, U }. Thus, so far as we are concerned, the symbol T in the genome has been replaced by U . Triplets of RNA nucleotide symbols are referred to as “codons.” Clearly there are 43 = 64 codons. Out of these 64 codons, three are stop codons (UAA, UAG, UGA), while the remaining 61 codons produce 20 amino acids. A table giving the correspondence between the 61 codons and the 20 amino acid symbols, as well as sketches of the shapes of the 20 amino acids, can be found in [2], pp. 137138. It is therefore apparent that there is a redundancy in the map between the 64 codons and the 20 amino acid symbols plus the STOP symbol. There are several theories to explain the rationale and the nature of this redundancy. However, to date no universally accepted explanation has been advanced for the logic behind this map. This question does not concern us in the present paper. In so-called lower level organisms called prokaryotes, the entire genome consists of just genes and non-coding regions (non-genes). Each gene is a continuous, unbroken stretch of codons, each of which generates the corresponding amino acid. The sequence of amino acids forms the protein coded by the gene. Thus it is clear that the length of each gene is an exact multiple of three. The regions between genes are the so-called non-coding regions, and their length need not be a multiple of three. In eukaryotes, the genes themselves need not be contiguous, but are divided up into ‘introns’ that separate ‘exons.’ The exons come together at the time of reproduction to code for proteins. Suppose now that a new organism, a prokaryote, has its genome sequenced. We would like to know which parts of the genome correspond to genes, and if possible, what their function is. This can be done either through computation or by experiment. Computational algorithms can only ‘predict’ that a stretch of DNA is a gene, but this prediction can be veriﬁed only through experiment. However, experimentation is both costly and time-consuming, so it is highly

Stochastic Modelling over a Finite Alphabet

363

desirable to have computational algorithms that can predict which parts of the genome correspond to genes with a fair degree of accuracy. In making such predictions, obviously there is a trade-oﬀ between false positives and false negatives. A false positive is the situation where a string is predicted to be a gene, but is proven not be a gene by subsequent experimentation, while a false negative is the converse. From the standpoint of biology, it is much more preferable to have a false positive than a false negative. One does not mind performing a few unnecessary experiments, but one would deﬁnitely not like to ‘miss’ a gene. The problem of gene ﬁnding can be put into the previous formalism as follows: Suppose we have r gene sequences, denoted by u1 , . . . , ur , and s nongene sequences, denoted by v1 , . . . , vs , each of which is a string over the four-symbol nucleotide alphabet. The premise is that u1 , . . . , ur are all sample paths of one stochastic process, while v1 , . . . , vs are all sample paths of another stochastic process. The aim therefore is to construct two models, denoted by C (for ‘coding’ model) and N C (for ‘non-coding’ model) respectively, such that the model C produces the gene sample paths with high probability and the non-gene sample paths with low probability, while the model N C has the opposite behaviour. Then, given a new sequence w, we compute the likelihood of the sequence w from each of the two models. If PrC (w) PrN C (w), we declare that w is a gene, whereas if PrC (w) PrN C (w), we declare that w is a non-coding region. The use of multi-step Markov models is quite popular in the biology literature, because they can exactly match the observed frequencies of k-tuples, as shown in Section 1. Some references that apply this approach to problems of gene ﬁnding are [6, 11, 13, 18]. In biology, it is normal to take k to be anywhere from zero to eight. (Note that a zeroth order model presumes that the sample path is that of an i.i.d. process.) In Glimmer [18], which is among the more popular algorithms, the model is assumed to be a weighted average of k-step Markov models for k ranging from zero to eight. There is just one more point to be noted. Both gene sequences and (to a lesser extent) non-coding regions display three periodicity. This means that neither of the Markov chains C, N C is stationary. Instead, the quantity Pr{ut |ut−1 , ut−2 , . . . , ut−k }. depends on t with a periodicity of three. Moreover, if a particular stretch of genome on the ‘forward strand’ of DNA is a gene, it does not follow that the reverse complemented sequence is also a gene – often it is not. Thus the model C for the coding regions is actually a collection of six models, corresponding to t = 0, 1, 2 mod 3, and on the forward and reverse strands. Similar remarks apply to the non-coding regions. But it should be mentioned that perhaps the eﬀects of three periodicity and forward-backward strands are in general less pronounced in the non-coding regions than they are in the coding regions. Moreover, the eﬀect of forward strand versus reverse strand is not that signiﬁcant even for coding regions.

364

M. Vidyasagar

The disadvantage of a method such as Glimmer based on multi-step markov models is that it leads to Markov models with huge state spaces, of up to 48 = 65, 536 states. Instead, by using the 4M algorithm described in the previous section, it is possible to achieve a reduction in the size of the state space by as much as 50%. 4.3 Computational Results The 4M algorithm was applied to several prokaryote genomes. (To apply the algorithm to eukaryote genomes, it is necessary to modify it slightly to detect so-called splice sites, or intron-exon boundaries. This is not diﬃcult and we will take up such examples next.) The ﬁrst table shows the results when the 4M algorithm is applied to various bacteria, and demonstrates its eﬃcacy in predicting both coding regions as well as non-coding regions accurately. Throughout, we use k = 6, which means that the original model was a ﬁth-order Markov model that perfectly reproduced all sextuplet frequencies. In applying the algorithm, the parameter in (32) taken as 0.1. In each organism, the coding regions have been divided arbitrarily into two classes, and a model C is developed based on half of the coding regions. Similarly, the noncoding regions have been divided arbitrarily into two classes, and a model N C is developed based on half of the noncoding regions. Then the remaining halves of the coding and noncoding regions are classiﬁed using the two models. Thus the ﬁrst line of the table means that, for the bacterium B. subtilis, we have divided the 4,106 known or putative genes arbitrarily into two sets of 2,053 strings each. One of the two sets is used to train the model C. Similarly the regions known not to be genes are divided into two classes and used to train the model N C. Then the other set of 2,053 genes (known and putative) are classiﬁed using the two models C and N C. Out of these 2,053 genes, 2,011 were classiﬁed correctly. Thus the ﬁrst line of the table means that when 2,053 strings (that are known to be coding regions) are classiﬁed according to the two models, 98.4% are correctly classiﬁed as coding regions. Similarly, out of the 2,286 strings that are known to be noncoding regions, 2,202 are correctly classiﬁed as noncoding regions.

Stochastic Modelling over a Finite Alphabet

Organism

365

No of No of % of No of No of % of genes in genes genes non non non test set prepregenes in genes genes dicted dicted test set preprecorcordicted dicted rectly rectly corcorrectly rectly 2053 2011 97.9 2286 2202 96.3 py- 745 741 99.4 620 600 96.7

B.subtilis Helicobacter loriJ99 Archaeoglobus 1209 fulgidus Ecoli K12 2127 Haemophilus in- 828 ﬂuenzae Mycoplasma geni- 241 talium Thermotoga mar- 929 itima Treponema pal- 518 lidum Borrelia burgdor- 425 feri

1190

98.4

788

729

92.5

2025 824

95.2 99.5

2298 944

2225 920

96.8 97.4

240

99.5

118

85

72.03

922

99.2

414

308

74.3

514

99.2

432

170

39.3

424

99.7

282

135

47.8

The next table shows the results of ‘whole genome analysis.’ In this study, we begin with so-called ORF (Open Reading Frames), which are putative genes. Using some training set, we classify the ORF’s as to being genes or non-genes. The column marked ‘Correct Starts’ is not as signiﬁcant as the two columns marked ‘Matches,’ which correspond to the accuracy of predictions. The table also shows a comparison between the prediction accuracies of the 4M algorithm versus Glimmer. It can be seen that the results are quite comparable. One column of the table shows the genes that are ‘missed.’ These ‘missed’ genes are further classiﬁed into those that are known to be genes (which is a bad thing for an algorithm to miss), and those that are only hypothesized genes (in which case it is not so bad to miss them, since they may not be genes after all). It can be seen that most of the ‘missed’ genes are only hypothesized and not true genes. The last column shows the number of ‘additional’ genes that are predicted by the 4M algorithm and not by Glimmer. This last column is of great interest, as the true payoﬀ for any algorithm comes from ‘predicting’ new genes that are subsequently conﬁrmed by experimentation. There is one very interesting organism, namely Mycoplasma genatalium. For this organism, both the 4M algorithm as well as Glimmer perform quite

366

M. Vidyasagar

poorly. Given how well both algorithms perform for other organisms, we believe that the results say more about the organism than about the algorithms. Organism

No. of Correct Matches Matches Missed Add’l Genes Starts (4M) (Glimmer) Real Genes Genes B. subtilis 4106 2382 3874 3989 30, 202 117 (58.01%) (94.34%) (97.15%) (5.65%) Helicobacter 1491 1166 1470 1468 2, 19 61 pylori J99 (78.20%) (98.59%) (98.45) (1.4%) Archeoglobus 2420 2093 2348 2356 3, 69 451 fulgidus (86.48%) (97.02%) (97.35%) (2.97%) E. Coli K12 4254 3358 3986 4082 123, 145 81 (78.93%) (93.70%) (95.95%) (6.29%) Haemophilus 1657 1464 1639 1638 2, 16 103 inﬂuenzae (88.35%) (98.91%) (98.85%) (1.08%) Treponema 1036 687 1016 979 1, 19 13 pallidum (66.31%) (98.06%) (94.49%) (1.93%) Mycoplasma 484 129 (26) 318 337 94, 72 418 genitalium (26.65) (65.70%) (69.62%) (34.29%) Thermotoga 1858 1214 1833 1816 2, 23 29 maritima (65.33%) (98.65%) (97.73%) (1.34%) Borrelia 851 660 835 823 1, 15 19 burgdorferi (77.5%) (98.11%) (96.70%) (01.88%) The next table shows the reduction in the dimension of the state space that results from using the 4M algorithm. Recall that we are perfectly matching sextuplet frequencies, which means that the original model is a 5th order Markov process, and therefore has 45 = 1, 024 states. In this table, we have ignored the eﬀect of the forward versus reverse strand. Hence there six models (t = 0, 1, 2 mod3 for both coding and non-coding regions), with 1,024 states each. Organism C0 C1 C2 N0 N1 N2 Archaeoglobus fulgidus 418 493 514 1006 997 985 B.subtilis 148 118 196 700 640 691 E. coli 385 283 283 781 865 892 Helicobacter pylori J99 550 475 631 931 928 913 Haemophilus inﬂuenzae 478 364 418 886 898 889 Mycoplasma Genitalium 769 547 655 985 973 994 Treponema pallidum 598 544 592 1021 1024 1015 Thermotoga maritima 328 388 316 1021 1018 1012 Borrelia burgdorferi 436 392 526 904 886 922

Stochastic Modelling over a Finite Alphabet

367

It is clear that there is signiﬁcant reduction in the size of the state space in the coding regions, but not so in the non-coding regions. This is because there is much less structure in non-coding regions. The most dramatic reduction is in the case of B. subtilis, where the state space size reduces by nearly 90%. Finally, to round out the results, we vary the parameter in (32). We use a value of = 0.1 to begin with, and then adjust it up or down. The purpose is to show that the value of is not signiﬁcant by itself. Rather, the signiﬁcant quantity is the value of the K-L divergence rate between the full (k − 1)-th order Markov model and the reduced state space model generated by the 4M algorithm. Two organisms were analyzed, namely B. subtilis and E. coli k 12. In each case, the table shows the K-L divergence rate between the ‘true’ model for frames C0, C1 and C2, and the reduced order model, as a function of . It also shows the percentage of genes classiﬁed correctly (in all three frames). The performance appears to start falling oﬀ when the K-L divergence rate between the true model and the 4M model is above 0.01. Based on this rule of thumb, we can in fact choose diﬀerent values of the threshold in diﬀerent frames, so as to ensure that the K-L divergence is not more than 0.01. However, this has not been done as yet. Results for B. subtilis: C0

C1

C2

%Genes Predicted Correctly 0.1 0.012 0.009 0.011 97.90 0.2 0.029 0.027 0.032 97.00 0.25 0.029 0.031 0.115 92.20 0.3 0.029 0.031 0.115 92.20 0.4 0.029 0.031 0.115 92.20 0.5 0.029 0.031 0.115 92.20 Results for E. coli k 12: C0

C1

C2

%Genes Predicted Correctly 0.1 0.009 0.034 0.050 95.2 0.2 0.074 0.077 0.082 93.9 0.25 0.081 0.083 0.089 93.7 0.3 0.081 0.083 0.151 91.0 0.4 0.081 0.083 0.179 90.1 0.5 0.081 0.083 0.179 90.1

368

M. Vidyasagar

5 Conclusions The problem of ﬁtting a set of sample paths with a multi-step Markov chain model arises in a variety of applications, such as data compression, speech recognition, and ﬁnding genes from a genome. In this paper, we have presented, perhaps for the ﬁrst time, accuracy and conﬁdence bounds for the parameters of a multi-step Markov chain estimated from ﬁnite length sample paths. A new algorithm, called the 4M (Mixed memory Markov model) algorithm is presented, for constructing multi-step Markov models with reduced state space. This method is better suited for problems of classiﬁcation than for problems of prediction. The 4M algorithm is then applied to the problem of ﬁnding genes in several prokaryote genomes, and gives excellent results. With slight modiﬁcations, the algorithm can also be applied to eukaryotes, and this work is under way.

References 1. B. D. O. Anderson, “The realization problem for hidden Markov models,” Mathematics of Control, Signals, and Systems, 12(1), 80-120, 1999. 2. P. Baldi and S. Br¨ unak, Bioinformatics: A Machine Learning Approach, MIT Press, Cambridge, MA, 2001. 3. R. Begleiter, R. El-Yaniv and G. Yona, “On prediction using variable order Markov models,” Journal of Artiﬁcial Intelligence Research, 22, 385-421, 2004. 4. P. Billingsley, Probability and Measure, Wiley, New York, 1986. 5. P. Billingsley, Probability and Measure, (Third Edition), Wiley, New York, 1995. 6. A. L. Delcher, D. Harmon, S. Kasif, O. White and S. L. Salzberg, “Improved microbial gene identiﬁcation with GLIMMER,” Nucleic Acids Research, 27(23), 4636-4641, 1999. 7. L. Devroye, L. Gyorﬁ and G. Lugosi, A Probabilistic Theory of Pattern Recognition, Springer, 1996. 8. W. J. Ewens and G. R. Grant, Statistical Methods in Bioinformatics, SpringerVerlag, New York, 2001. 9. F. Jelinek, Statistical Methods for Speech Recognition, MIT Press, Cambridge, MA, 1997. 10. B-H. Juang and L. R. Rabiner, “A probabilistic distance measure for hidden Markov models,” AT&T Tech. J., 64(2), 391-408, February 1985. 11. A. Krogh, I. S. Mian and D. Haussler, “A hidden Markov model that ﬁnds genes in E. coli DNA,” Nucleic Acids Research, 22(22), 4768-4778, 1994. 12. G. Langdon, “A note on the Ziv-Lempel model for compressing individual data sequences,” IEEE Transactions on Information Theory, 29, 284-287, 1983. 13. W. H. Majoros and S. L. Salzberg, “An empirical analysis of training protocols for probabilistic gene ﬁnders,” BMC Bioinformatics, available at http://www.biomedcentral.com/1471-2105/5/206, 21 December 2004. 14. J. R. Norris, Markov Chains, Cambridge University Press, Cambridge, UK, 1998. 15. D. Pollard, Convergence of Stochastic Processes, Springer-Verlag, 1984.

Stochastic Modelling over a Finite Alphabet

369

16. L. W. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proc. IEEE, 77(2), 257-285, February 1989. 17. Z. Rached, F. Alalaji and L. L. Campbell, “The Kullback-Leibler divergence rate between Markov sources,” IEEE Trans. Info. Thy., 50(5), 917-921, May 2004. 18. S. L. Salzberg, A. L. Delcher, S. Kasif and O. White, “Microbial gene identiﬁcation using interpolated Markov models,” Nucleic Acids Research, 26(2), 544-548, 1998. 19. V. N. Vapnik, Statistical Learning Theory, John Wiley, New York, 1998. 20. M. Vidyasagar, A Theory of Learning and Generalization, Springer-Verlag, London, 1997. 21. M. Vidyasagar, Learning and Generalization with Applications to Neural Networks, Springer-Verlag, London, 2003. 22. M. Vidyasagar, “The realization problem for hidden Markov models: The complete realization problem,” submitted for publication. 23. M. Vidyasagar, “The realization problem for hidden Markov models: The partial realization problem,” submitted for publication. 24. M. Vidyasagar, “Convergence of empirical means with alpha-mixing input sequences, and an application to PAC learning,” submitted for publication. 25. T. A. Welch, “A technique for high performance data compression, IEEE Computer, 17(6), 8-19, 1984. 26. J. Ziv and A. Lempel, “A universal algorithm for sequential data compression,” IEEE Transactions on Information Theory, 23(3), 337-343, 1977. 27. J. Ziv and A. Lempel, “Compression of sequences via variable rate encoding,” IEEE Transactions on Information Theory, 24(5), 530-536, 1978.

Feedback Networks Glenn Vinnicombe1 Department of Engineering, University of Cambridge [email protected]

1 Introduction This paper presents a survey of some recent results which are primarily concerned with the action of relatively simple controllers working in complex networks. Feedback system design is all about getting the right amount of gain into the right places. In the design of a controller for a given linear multivariable system (with maybe an uncertain frequency response, but with a known topology of inputs and outputs), this means getting the right amount of gain into the right frequency ranges and in the right directions. H∞ theory, to which Keith Glover has of course contributed a great deal, tells us a lot about managing these tradeoﬀs (the H∞ loop shaping design method in particular [1]). Here though, we are not concerned with complex multivariable systems, but rather simple scalar systems which are perfectly well controlled by simple controllers – or rather they would be if only they weren’t also connected to other systems who are in turn connected with more systems in a complex web of feedback interactions. A further complication in the applications we have in mind is that we rarely have information about the whole network, we only know the local interactions. A natural question again is “how do we get the right amount of gain into the right places”. The sorts of networks we are thinking of are communication networks, economic networks, power distribution networks, coordinated networks of vehicles and biological (gene regulatory networks in particular). In fact, nature appears to use far more feedback regulation in the cell than we typically dare use in our technologies (an exception is the Internet). It is well known to engineers that feedback can’t have a uniformly good eﬀect; if it makes some things better it will always make other things worse. Managing those tradeoﬀs in large regulatory networks is extremely diﬃcult. So diﬃcult, in fact, that engineers typically don’t try. It’s a mystery how nature manages it. This paper falls into two parts. First, we’ll look at some tradeoﬀs in simple feedback loops from a biological perspective, taking as an example the regulation of concentration of a single molecular species. This will involve a fresh B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 371–387, 2006. © Springer-Verlag Berlin Heidelberg 2006

372

G. Vinnicombe

slant on some classical results. Fluctuations in the numbers of molecules are inevitable, we’ll see that delays provide a lower bound on the variance reduction possible using feedback. Furthermore, feedback in biochemical pathways almost always has to proceed via intermediate molecules. We’ll see that limits on the metabolic burden imposed by these pathways in turn place limits on their information carrying capacity. This in turns limits the potential beneﬁts of feedback. In the second part of the paper, we’ll look at one way in which it is possible to manage feedback gain in large networks. If the network is suﬃciently symmetric then it is possible to predict properties of the network from local feedback interactions. Indeed, if everyone satisﬁes a convexiﬁed Nyquist-like criterion with their neighbours then the network will be stable.

2 Regulation of Molecule Numbers Consider a molecular species being produced at a constant average rate c with each molecule degrading at average rate α, so the total degradation rate is αn. This is a good model of constitutive (i.e. uncontrolled) gene expression, with 1st order degradation. In this case n would be the number of molecules of a particular mRNA. (no feedback)

n

c

n+1

n

αn

n−1

That is, we have a jump Markov process, where the probability of an mRNA being produced in time dt is c dt and the probability of one being degraded is αn dt. The number of molecules will ﬂuctuate around the steady state values c = αn0 ⇒ n0 =

c α

In fact, for a linear system such as this, the ﬁrst and second order statistics are identical to those of solutions to the SDE n˙ = c − αn + 2cη, where η is a unit variance Gaussian white noise. This is in spite of the fact that in the Markov process n is an integer, whereas here it is real, and also the true statistics are Poissonian as opposed to Gaussian. Without feedback, the equilibrium concentration is tied to the degradation rate. Consider a change in α, α → α + δα , n0 → n0 + δn c = (α + δα )(n0 + δn ) ≈ αn0 + n0 δα + αδn ⇒

δα δn ≈− n0 α

Feedback Networks

373

The relative change in the equilibrium concentration, or expected number, of the species is equal to the relative change in the degradation rate. A 10% change in α leads to a 10% change in n, roughly. We can improve this with feedback, of course, by making the rate of production of our species depend on its concentration. (with feedback)

n n

f (n) αn

n+1 n−1

In steady state: f (n0 ) = αn0 . If we perturb: α → α + δα and n0 → n0 + δn then f + f δn ≈ αn0 + n0 δα + αδn δn 1 δα ⇒ =− f n0 1− α α Sensitivity

If the derivative of the function f is negative, the relative change is reduced by a factor equal to the sensitivity function. −f /α here corresponds to the total feedback gain. Graphically all that’s happening is that by steepening the birth rate, we reduce the movement of the intersection of the birth and death curves as the death rate changes. f

n Of course, it is always true that, for small perturbations relative error in = S × relative error in closed loop open loop where the sensitivity function S is given by S=

1 1+L

and L is the amount of feedback (loop gain). In this example, the sensitivity can be made arbitrarily small, by making f large and negative. However, in practice it is rarely possible for the species n to directly control its own birth rate; the control must proceed via intermediate species. In addition, there will always be some delay in the feedback. Both of these factors limit the potential beneﬁts of feedback. These limitations are well understood in the deterministic case, here we discuss their extension to the stochastic case, where the underlying processes are jump Markov processes.

374

G. Vinnicombe

2.1 Feedback Channels In practice, a species can’t usually control its birth rate directly. Because of the biochemistry, there will need to be a second molecular species, with the birth rate of the ﬁrst species depending on the concentration of the second, and the birth rate of the second species depending on the concentration of the ﬁrst. An example might be that the ﬁrst species is an mRNA, and the second a DNA binding protein it codes for. So the rate of transcription depends on the protein concentration and the rate of translation on the the mRNA concentration. System:

Feedback:

n n m m

f (m) αn

g(n) βm

n+1 n−1 m+1 m−1

We will give details of the control mechanism used in the example (i.e. the particular f , g , β) later, but the result is shown in Figure 1. Note how the feedback has clearly reduced the variance, by a factor of 6 in fact, taking out the low frequency ﬂuctuations. However, it is fairly clear from these plots

130 120

open loop

N

110 100 90 80 70 0

1000

2000

time

3000

4000

5000

3000

4000

5000

130 120

closed loop

N

110 100 90 80 70 0

1000

2000

time

Fig. 1.

Feedback Networks

375

alone that the closed loop response has a larger high frequency component. This is conﬁrmed by looking at the power spectral densities (Figure 2). The system with feedback has less energy at low frequencies and more at high frequencies. The ratio of these can be deﬁned as the sensitivity function again (here frequency dependent – we were just looking at steady state, or DC, values before). Again, it’s the ratio of closed loop to open loop properties (with the square root because these are power spectra). S(f ) =

Pcl (f ) Pol (f )

Probably the most fundamental trade oﬀ in control is the celebrated Bode tradeoﬀ; that the integral of log sensitivity is nonnegative. The area of log sensitivity reduction can be no greater than the area of log sensitivity increase.

6

10

open loop 4

Pxx

10

2

10

closed loop 0

10

0

0.02

0.04 0.06 freq (Hz)

0.08

0.1

0.08

0.1

Fig. 2. 1

log|S|

0 −1 −2 −3 0

0.02

0.04

freq (Hz)

0.06

Fig. 3. S(f )

376

G. Vinnicombe

What you gain somewhere, you have to lose somewhere else. For this deﬁnition of the sensitivity function, it has been shown by Martins & Dahleh [2]1 that ∞ 0

log S(f ) df ≥ 0

Of course, this becomes an equality if the uncontrolled system is stable, and the feedback is linear and deterministic ([Bode 1945]) Fundamental as it is, the Bode integral doesn’t in itself necessarily limit the beneﬁts of feedback. As in this example, the area of sensitivity increase can be shifted to a frequency range where it is less important. Are there limits to the amount of log sensitivity that can be shifted? It turns out that, yes, further constraints. are imposed by: • •

Feedback capacity (metabolic load) Delays

2.2 Limitations Due to Feedback Capacity One of the main results of [2] is actually that is that the total amount of negative log sensitivity is bounded by the capacity, in the sense of Shannon, of the feedback path. −

∞

0

[log S(f )]− df ≤ C

(1)

where C is Shannon capacity of the feedback path and [x]− =

x 0

if x < 0 otherwise

so the LHS of (1) is the total area of log sensitivity decrease. So, there is a limit to the total beneﬁt of feedback even if we are prepared to tolerate an arbitrarily large area of sensitivity increase somewhere. This result was derived in the context of control over bitrate limited communication channels, but is still applicable here. The channel, a molecular species, is perhaps a bit unusual though. Recall that our feedback channel is the jump Markov process m m

g(n) βm

m+1 m−1

(2)

It is clear that the deaths carry no information (they serve only to limit the maximum number of the species m); only the births carry information. So the capacity is actually the same as that of the photon counting channel, 1

The results of [2] are actually given in discrete time, but are easily extended to continuous time

Feedback Networks

377

i.e. the maximum amount of information about a light intensity that can be encoded in counts from a photon detector. The capacity is 1/e log(2) (about 0.5) times the maximum birth rate, which is the maximum synthesis rate of m. This synthesis rate corresponds to the metabolic load imposed by the presence of the feedback channel. So, in a very real sense, we can claim that the beneﬁts of feedback have to be traded oﬀ against metabolic burden, beneﬁts of feedback 0.5× maximum metabolic burden. The example in the simulations above was, in fact, the strongly nonlinear interaction: g(n) = f (m) =

1 0

if n < 100 otherwise

10 if m > 0 0 otherwise

α = .01 β=5 Note that the m species, the regulating one, is produced at an average rate of 1 per unit time if the n species is below its target and not at all otherwise. So, C = 1/e log(2). This choice was motivated by the fact that in order to attain capacity in the photon counting channel it is necessary to modulate the intensity to either its maximum or minimum level and not use intermediate levels. If the channel is used in an approximately analogue manner, with a linear dependence of birth rate on the concentration of n (so that m is approximately proportional to n) then the capacity is substantially less than this. Incidentally, the example works just as well with n = 10, which would be more realistic for the gene expression example discussed. Whether nature uses nonlinear schemes to maximize the information carrying capability of molecular channels is not clear (but surely likely). 2.3 Limitations Due to Delay There will always be a delay in the feedback, as molecules take a ﬁnite time to make. In the case of DNA, RNA and proteins these times can be appreciable. In [3], where we consider a plasmid replication example in detail, the total delay in the loop is over 40 seconds. These delays can limit performance. Consider the example n n

f (n(t−T ))

n+1

αn

n−1

with f (n0 ) = αn0 , so the equilibrium is at < n >= n0 . We have ignored the need for a second species, but made the birth rate an arbitrary function of the

378

G. Vinnicombe

concentration T seconds ago, i.e. we have assumed that the molecule takes T seconds to make. In open loop, i.e. f (n0 ) = 0, the variance of n is the same as its mean, i.e. σn2 = n0 . Figure 4 shows the reduction in variance against the feedback gain f (n0 ), the slope of f (relative to the open loop value). As one would expect, if the gain is too small then the feedback is overly sluggish, taking a long time to reach equilibrium. If too large, the response is oscillatory. In between it’s just right, but still there are some ﬂuctuations. Clearly, to get the beneﬁts of feedback we need to carefully manage the gain. It’s worth asking the question though of whether we can reduce the ﬂuctuations further with alternative feedback strategies. 2

1

1.8 1.6

0.9

1.4

2

1.2

0.8

1.8

2

0.7

1.4

0.8

1.6

0.6

1.4

1.2

0.6

1

0.4

1.2

0.2

1

0.8

0.5

0.6

0

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0.8 0.6

0.4

0.4

0.2 0

1

1.8

1.6

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

0.4 0.2

2

0.3

0

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0.2 0.1 0

0

10

20

30

k

40

50

60

70

Fig. 4. Variation with k = −f (n0 )

For large equilibrium numbers this system behaves (at least in terms of statistics up to second order) as approximately n(t) ˙ = f n( t − T ) − αn + 2αn0 η. delay of T

Standard optimal control tools can be used to show that the minimum obtainable variance is (1 − e−2αT )n0 . So, the answer is that the ﬂuctuations cannot be reduced, or at least not by much. Figure 5 shows the maximum possible reduction in variance as a function of αT , the product of the degradation rate and the time delay, if a constant controller is used and if the optimal controller is used. So the product αT which is the probability of any given molecule dying in the time it takes to make one, has to be small for there to be any possibility of signiﬁcantly reducing variance by feedback.

Feedback Networks no feedback

1

best 1st order

= 1 − e−2αT

0.6

optimal

2 n

σ /

0.8

379

0.4

0.2

0 0

0.5

1 αT

1.5

2

Fig. 5.

3 Feedback Networks So, what about networks of large numbers of interacting systems, with lots of feedback loops as in Figure 6. This isn’t a graph, the nodes represent dynamical systems, and they might all be diﬀerent. For this analysis, we’re not just interested in what’s connected to what, but also in the details of the interactions.

Fig. 6.

380

G. Vinnicombe

Examples of complex systems that can be modelled like this are economic networks, where the nodes might represent buyers, sellers, market makers etc. Power networks, where we have generators, consumers and transmission lines, communication networks (e.g. Internet) and vehicle formations (ﬂocking phenomena/platooning etc). In order to obtain the beneﬁts of feedback we need somehow to get the right amount of gain into all the right places, not too little and not too much. So how much is right, we clearly can’t treat each loop in isolation. In general, apart from suggesting a massive centralized optimization problem, control theory has little to oﬀer. As engineers we tend to avoid making massively horizontally distributed feedback systems of this kind. We put up barriers and create hierarchies. We actively break feedback loops to avoid getting into trouble. However, in the case that any nonlinearities are weak, and connections are predominantly symmetric as in Figure 6, there is a way to do it. The results involve relating local and global properties. If we have ways of predicting global properties from local ones then we’ll know how to design the local interactions. The key questions are: • •

How can we predict global properties from local interactions? How can we design the individual systems (agents) and their local interactions (protocols) to ensure desirable global properties?

Note: If the agents are stable, we can always get a stable network by making the feedback gains small. What we really want to know how large we can get away with making them, in order to get good performance everywhere. I’m not suggesting for a moment that biology works like this – my aim is to simply point out that there is an issue here. Note, though, that this doesn’t appear to be as much of a problem for developmental networks. It was recently shown [4], that, in the case of drosophila segmentation, once the topology of the interactions was right, many diﬀerent combinations of gains led to the right result. In regulatory networks this cannot be the case – the choice of gain is critical. Some of the most powerful tools in control systems theory are ones that allow properties of an interconnected system to be inferred from properties of its subsystems and the structure of their interactions. One of the very best is the Nyquist stability criterion. η +

Σ

u

g¯(jω)

− k Fig. 7.

y

Feedback Networks

381

In order to determine the stability of a simple feedback system as in Figure 7 we just have to look at whether the the locus of the frequency response k¯ g(jω) encircles the point −1 in the complex plane. Can similar principles be applied to more complex interactions of the sort we are describing? Figure 8 shows a small part of such a network. Each node represents a system or ﬁlter with a given frequency response. In the time domain, there would be a delay diﬀerential equation representing each node. The input to each node is the average of the outputs of its neighbours. g2

g1

g7

g5

g4

g3

g6

Fig. 8.

The interconnection is described as follows: •

Each vertex: stable system/ﬁlter with frequency response g¯i (jω). y¯i (jω) = g¯i (jω)¯ ui (jω)

•

Input: average of the signals from its neighbours. ui (t) =

1 Ni

yk (τ ) i,k connected

The main result of [5] is that, if each system satisﬁes a convexiﬁed Nyquistlike condition with its neighbours then the entire feedback network is guaranteed to be stable. To be more precise, ﬁrst deﬁne the S-hull of a set S(·) as the convex hull of the square root of the set all squared. That is, for X ⊂ C let 2 √ , X S(X) = Co √ where X := {y : y 2 ∈ X}. The S-hull of a set, S(·), is slightly larger than the convex hull of the union of set with 0, as shown in Figure 9. For each agent, we then take the S-hull of the products of its own and its neighbours dynamics, these are the loop gains. Finally we check whether −1 is in the convex hull of these sets. Theorem 1. [5] An interconnection of the type depicted in Figure 8 is stable if −1 ∈ / Co ∪i S −¯ gi (jω)¯ gk (jω) : k connected to i ∀ω

382

G. Vinnicombe 1

0.5

0

−0.5

−1

0

0.5

1

1.5

2

2.5

3

Fig. 9.

A key point is that, apart from the ﬁnal convexiﬁcation, the result is entirely local, and a line to separate these S-hulls from −1 can always be agreed on in advance. The main conclusion is that, provided the interconnections are symmetric it is suﬃcient to just look at each feedback loop separately – and say that as long as each agent behaves well with its neighbours then the network as a whole will be behave predictably. 3.1 Modules and Protocols One way in which this theorem can be used to build stable complex networks is by restricting the agents/modules to parameterized classes and insisting on interconnection protocols. Imagine you have a box of diﬀerent modules or

Module: Σ

fT

e.g.

N dy kg X = ui (t − T ) dt N T i=1

Protocol: e.g. f/b interconnection allowed if kg kr ≤ 1, τ < 2T Module: Σ

gτ

e.g. τ

Fig. 10.

N dy kr X +y = ui (t) dt N i=1

Feedback Networks

383

building blocks, each representing diﬀerent delay diﬀerential equations, as in Figure 10. Say the green ones are delayed integrators with a delay T and the red ones are lags with time constant τ . Each green has a diﬀerent T and each red a diﬀerent τ . Imagine that the T ’s and τ ’s are written on the side, and say they have a dial on them to set their gain. The protocol is that we’re allowed to connect a red and a green brick together if the product of their gains is no greater than 1 and the red’s τ is no greater than twice the greens T . Then we carry on adding on adding more red and green bricks according to the same rules, as many as we like. We have been representing this interconnection as in Figure 11. fT

gτ Fig. 11.

The claim is that we’ll always end up with a stable network, and in this case the certiﬁcate in Figure 12 proves it. Moreover, the condition is nonconservative for just two blocks. Note that this one picture provides a stability certiﬁcate for all possible interconnections satisfying the protocol. 0.5

0

−0.5

−1

−1.5

−2

−2.5

−3 −3

−2.5

−2

−1.5

−1

−0.5

⇒ closed-loop stability. Fig. 12.

0

0.5

384

G. Vinnicombe

3.2 Example: Vehicle Platooning As another example, consider the platooning of heterogeneous vehicles. That is, each vehicle might has a diﬀerent mass, delay, rate of response and a diﬀerent controller designed to track a ﬁxed reference. x

g¯(jω)

e

Σ

+ xref −

Producing a stable platoon, with ﬁxed spacings between the vehicles, is actually a diﬃcult problem even if all the vehicles are the same. The ﬁrst thing one might try is to design the control system for each car to minimize the error between its position and that of the vehicle in front. However, that doesn’t work [6]. It’s because of another fundamental limitation, related to the Bode integral. Under reasonable physical assumptions, a weighted integral of the log of the ratio of x to the reference must be strictly positive, so any disturbances get ampliﬁed as you go back along the chain. This is known as lack of string stability and is unavoidable with this architecture, since [7] ∞ 0

V5

V4

log

Pxx (ω) dω > 0. Pxref xref (ω) ω 2

V3

V2

V1

The standard solution [6] is to communicate leader information to all vehicles in the platoon.

V5

V4

V3

V2

V1

However, an alternative solution, which has the beneﬁt of being decentralized, is to look backwards as well as forwards. We wouldn’t normally like this solution from an engineering point of view - there’s too much feedback, how can we possibly guarantee it would work for all combinations? For example, it is easy to come up with two cars (with their associated control systems) like this which follow a ﬁxed reference perfectly well, but go unstable when tracking each other. If the cars just look forward, then it requires a string of cars before the disturbances at the back get excessive; with the bidirectional scheme just two vehicles is enough for instability. Nevertheless, it is relatively straightforward to design a set of control laws and interconnection protocols

Feedback Networks

385

in such a way that Theorem 1 will guarantee we do always end up with stable platoons. V5

V4

V3

V2

V1

3.3 Internet Congestion Control The platooning example of the previous section probably still involves too much feedback to be practicable. The most successful application of Theorem 1 (or rather the application from which the theory evolved) is to Internet congestion control. The problem is that TCP, the protocol which controls data ﬂow, doesn’t work well over long distance uncongested high speed links. It’s not suﬃciently aggressive at grabbing bandwidth (in other words, the gain is too low in that part of the network). The theory tells us the correct way of scaling the gains. It is important to emphasize again though that stability is not the main aim here, we can always get that with small gains; what we want are good estimates as to how large we can make the gains. In this game the players are the • •

Participating Dynamics : – Users (generating packet ﬂow) – Routers (sending congestion signals) Protocol : Everybody does TCP.

The theory has suggested a modiﬁcation to TCP for high speed networks [8], which is incorporated in the IETF Draft [9]. A full implementation of [8] has been tested over a transatlantic link and worked well [10]. Finally, a question that must be addressed is “Why is linear analysis interesting?” What’s the point of all this linear analysis, when the real world is nonlinear and stochastic. Most mildly nonlinear systems of this kind, though, are pretty well behaved. Instead of going wildly unstable they tend to break into a limit cycles. Figure 13, from [11] shows the performance of a simple congestion control system through a single queue, with delay. It illustrates the fact that the critical point for linear stability (1 on the x-axis) is often a good operating point. The diﬃculty is in ﬁnding this operating point simultaneously in all parts of the network (with delays varying over several orders of magnitude).

4 Conclusions The design of feedback systems involves carefully managing tradeoﬀs, i.e. getting the gain right. Managing these tradeoﬀs in large regulatory networks

386

G. Vinnicombe Coefficient of variation VS normalized gain and number of packets in flight. Queue threshold=5

0.7

λ×T=150 λ×T=750 λ×T=3750 fluid limit

25

0.6

20

µ(pckts/s)

15

0.5

10 5

25

25 20

0.4

−5

20

0

500

time(s)

1000

15

1500

µ(pckts/s)

std(µ)/mean(µ)

0

5

0.3

0

15 µ(pckts/s)

10

−5

10 5

0

500

time(s)

1000

1500

0.2

0 −5

0

500

time(s)

1000

1500

0.1

0

0

0.2

0.4

0.6

0.8

k/kc

1

1.2

1.4

1.6

1.8

Fig. 13.

(getting the right gain in the right places) is diﬃcult. We have described some new tradeoﬀs apparent in biochemical regulatory systems, and we have described one way in which tradeoﬀs can be managed in designed technological feedback networks. It is an open question how nature manages these tradeoﬀs.

Acknowledgements To Keith, for starting me on this road and encouraging me along the way. A ﬁner, and fairer, adviser, division head and now Head of Department would be diﬃcult to imagine.

References 1. D.C. McFarlane and K. Glover, “A loop shaping design procedure using H∞ synthesis,” IEEE Trans. on Automatic Control, vol. 37, no. 6, pp. 759–769, 1992. 2. N. C. Martins and M. A. Dahleh, “Feedback control in the presence of noisy channels: ”Bode-like” fundamental limitations of performance,” preprint, 2004. 3. I. Lestas, G. Vinnicombe and T. Bishop, “Tradeoﬀs in gene regulatory networks: the plasmid replication control problem,” in Proceedings of IEEE Conference on Decision and Control and European Control Conference, December 2005, Accepted for publication.

Feedback Networks

387

4. G. von Dassow, E. Meir, E. M. Munro and G. M. Odell, ,“The segment polarity network is a robust developmental module” Nature, vol. 406, pp. 188–192, 2000. 5. I. Lestas and G. Vinnicombe, “Scalable decentralized robust stability certiﬁcates for networks of interconnected heterogeneous dynamical systems,” Tech. Rep., Cambridge University, Engineering Department, November 2004, CUED/FINFENG/TR.501. Accepted for publication in IEEE Transactions on Automatic control. 6. D. Swaroop and J.K. Hedrick, “String stability of interconnected systems,” IEEE transactions on automatic control, vol. 41, no. 3, pp. 349–357, 1996. 7. I. M. Horowitz, Synthesis of feedback systems, Academic Press, New York, 1963. 8. G. Vinnicombe, “On the stability of networks operating TCP-like congestion control,” in IFAC, 2002. 9. S. Floyd, “HighSpeedTCP,” RFC 3649, 2002. 10. T. Kelly, “Improving performance in high speed wide area networks,” First International Workshop on Protocols for Fast Long-Distance Networks, 2003. 11. I. Lestas and G. Vinnicombe, “How good are deterministic models for analyzing congestion control in delayed stochastic networks?,” in IEEE Conference on Decision and Control, December 2004.

Thoughts on System Identiﬁcation Jan C. Willems SISTA-ESAT, K.U. Leuven, B-3001 Leuven, Belgium [email protected]

1 Introduction It is a pleasure to dedicate this Festschrift article to Keith Glover on the occasion of his 60-th birthday. Keith came to MIT in 1969, armed with a degree from Imperial College, a couple of years of industrial experience, and a Kennedy scholarship, ready for some serious research. I, at that time a young assistant professor at MIT, had the good fortune to become Keith’s M.Sc. and Ph.D. supervisor, who he could choose freely because his scholarship made him, so to speak, ‘self-supporting’. In his M.Sc. thesis Keith applied circle-criterion based stability conditions to numerical integration routines [9]. His Ph.D. dissertation [10] was about System Identiﬁcation (SYSID), more speciﬁcally, about Identiﬁability. The main results of his dissertation were published in [11]. It was one of the ﬁrst articles that dealt with the parametrization of linear systems, an intricate subject that was studied very deeply later on. This research reached its apogee in Hannan and Deistler’s book [15] where the relevance for SYSID algorithms of parametrizations and canonical forms of multivariable linear systems is pursued in great depth. Keith’s dissertation showed a penetrating understanding of linear systems. It was a forebode of his later work in this area, culminating in his classic papers on model reduction [12] and on the double Riccati equation solution of the H∞ feedback control problem [7], both among the most cited in the ﬁeld of systems and control. A Festschrift is a welcome occasion to write an article with a personal and historical ﬂavor. Because of the occasion, I chose the subject of Keith Glover’s Ph.D. dissertation, SYSID. My interest in this area remained originally limited to the implications of the structure of linear systems. This situation changed with the Automatica papers [35]. These contain, in addition to the ﬁrst comprehensive exposition of the behavioral approach to systems theory, a number of new ideas and subspace type-algorithms for SYSID. The aim of the present article is to explain in a somewhat informal style my own B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 389–416, 2006. © Springer-Verlag Berlin Heidelberg 2006

390

J.C. Willems

personal point of view on SYSID. Among other things, I will describe in some detail the many representations of linear time-invariant systems, leading up to some exact deterministic SYSID algorithms based on the notion of the most powerful unfalsiﬁed model. I will then explain the idea behind subspace algorithms, where the state trajectory is constructed directly from the observations and a system model in state form is deduced from there. Subsequently, I will discuss the role of latent variables in SYSID. This leads in a natural way to stochastic models. I will ﬁnish with some remarks on the rationale, or lack of it, of viewing SYSID in a stochastic framework. I view SYSID in terms of the following picture from classical statistics: ID : D ∈ D → M ∈ M.

(ID)

Here D is the data, D the data class, M the model class, ID the identiﬁcation procedure, and M = ID(D) the model chosen from M on the basis of the data D through the identiﬁcation procedure ID. Our interest is in the case that the data consists of a time-series (throughout, time-series means vector time-series) w ˜T

=

w ˜ (1) , w ˜ (2) , . . . , w ˜ (T)

w ˜ (t) ∈ Rw

(w ˜T )

T

i.e. D ∈ (Rw ) . However, while our primary aim is at ﬁnite time-series T < ∞, we will occasionally consider T → ∞ and even T = ∞. So, we may as well take T for the data class D = ∪T∈N ∪ {∞} (Rw ) . The model class M consists of linear time-invariant, deterministic or stochastic, dynamical systems. The intent of this essay is to explain some of the issues involved in deﬁning and choosing M, and setting up an identiﬁcation procedure (ID). I prefer to deal with SYSID by going from the simple to the complex: Exact deterministic SYSID Approximate deterministic SYSID

Exact stochastic SYSID

Approximate stochastic SYSID In view of space limitations, I will not discuss the approximate stochastic case.

2 Deterministic Models Accordingly, I will start with deterministic systems. The ﬁrst question which we need to confront is: What are we after? What does an identiﬁcation procedure aim at? When we accept a mathematical model as a description of a phenomenon, what do we really do? Mind you, I am not asking a philosophical question. I am not inviting an erudite discourse on the convoluted relation between a real world manifestation of a physical or an economic phenomenon and a mathematical description of it. The question is meant to be a mathematical one.

Thoughts on System Identiﬁcation

391

The answer to this question is simple, evident, but startlingly enlightening (in German, a ‘Aha Erlebnis’ ). A model is simply a subset, B, of a universe, U, of a priori possibilities. This subset B ⊆ U is called the behavior of the model. Thus, before the phenomenon was captured in a model, all outcomes from U were in principle possible. But after we accept B as the model, we declare that only outcomes from B are possible. For example, the ideal gas law states that the temperature T , pressure P , volume V , and quantity (number of moles) N of an ideal gas satisfy P V /N T = R, with R a universal constant. So, before Boyle, Charles, and Avogadro got into the act, T, P, V, and N may have seemed unrelated, yielding U = R+4 . The ideal gas law restricts the possibilities to B = {(T, P, V, N ) ∈ R+4 | P V /N T = R}. In most applications, a model is given in terms of a set of equations (think of the basic laws of physics, e.g. Maxwell’s equations). The set of solutions to these equations is then the behavior. In the context of modeling, equations simply serve as a representation of their solution set. This view comes in very useful, since it gives a clear and unambiguous answer to the question when two sets of equations represent the same model. Often, it is trivial that a transformation of equations (as changing the order in which Maxwell’s equations are written down) does not change a model, but for some transformations this may be much more diﬃcult to establish (think of expressing Maxwell’s equations in terms of potential functions). In the case of dynamical systems, the phenomenon which is being modelled produces functions that map the set of time instances relevant to the model to the signal space. This is the space in which these functions take on their values. For the sake of concreteness, and because of its relevance to SYSID, I will assume in this article that the set of relevant time instances is N (the theory is analogous for Z, R, and R+ ). This choice means that we are studying discrete time systems and that we postulate that the model is valid for all time (of course, this does not mean that observations need to extend over all of N). I also assume throughout that the signal space is a ﬁnite dimensional real vector space, typically Rw . Following our idea of a model, the behavior for the dynamical systems which we consider is therefore a collection B of functions mapping the time set N into the signal space Rw . A dynamical model can hence be identiﬁed with N its behavior B ⊆ (Rw ) . The behavior is hence a family of maps from N to w R . Of course, also for dynamical systems the behavior B is usually speciﬁed as the set of solutions of equations, for the case at hand typically diﬀerence equations. As dynamical models, diﬀerence equations thus merely serve as a representation of their solution set. Note that this immediately leads to a notion of equivalence and to canonical forms for diﬀerence equations. These are particularly relevant in the context of dynamical systems, because of the multitude of, usually over-parameterized, representations of the behavior of a dynamical system.

392

J.C. Willems

3 Deterministic Linear Dynamical Systems SYSID usually employs as the model class dynamical systems that are (i) linear, (ii) time-invariant, and (iii) that satisfy a third property, related to the ﬁnite dimensionality of the underlying state space, or to the rationality of a transfer function. It is, however, clearer and advantageous to approach this situation in a more intrinsic way, by imposing this third property directly on the behavior, and not on a representation of it. In this section, I discuss this model class in considerable detail. N A behavior B ⊆ (Rw ) is said to be linear if w ∈ B, w ∈ B, and α ∈ R imply w + w ∈ B and αw ∈ B, and time-invariant if σB ⊆ B. The shift, σ, plays a central role in SYSID. It is deﬁned by (σf ) (t) := f (t + 1), and often called the backwards shift, since it shifts the time function f backwards. The third property that enters into the speciﬁcation of the model class is completeness. B is called complete if it has the following property: [[ w : N → Rw belongs to B ]] ⇔ [[ w|[1,t] ∈ B|[1,t] for all t ∈ N ]]. In words: B is complete if we can decide that w : N → Rw is ‘legal’ (i.e. belongs to B) by verifying that everyone of its ‘preﬁxes’ w (1) , w (2) , . . . , w (t) is ‘legal’ (i.e. belongs to B|[1,t] ). So, roughly speaking, B is complete iﬀ the laws of B do not involve what happens at ∞. Requirements as w ∈ 2 (N, Rw ), w has compact support, or limt→∞ w (t) exists, risk at obstructing completeness. However, often crucial information about a complete B can be obtained by considering its intersection with 2 (N, Rw ), or its compact support elements, etc. Recall the following standard notation. R [ξ] denotes the polynomials with real coeﬃcients in the indeterminate ξ, R(ξ) the real rational functions, and Rn1 ×n2 [ξ] the polynomial matrices with real n1 × n2 matrices as coeﬃcients. When the number of rows is irrelevant and the number of columns is n, the notation R•×n [ξ] is used. So, in eﬀect, R•×n [ξ] = ∪k∈N Rk×n [ξ]. A similar notation is used for polynomial vectors, or when the number of rows and/or columns is irrelevant. The degree of P ∈ R•×• [ξ] equals the largest degree of its entries, and is denoted by degree (P ). Given a time-series w : N → Rw and a polynomial matrix R ∈ Rv×w [ξ], say R (ξ) = R0 + R1 ξ + · · · + RL ξ L , we can form the new v-dimensional time-series R (σ) w = R0 w + R1 σw + · · · + RL σ L w. Hence R (σ) : (Rw )N → (Rv )N , with R (σ) w : t ∈ N → R0 w (t)+R1 w (t + 1)+ · · · + RL w (t + L) ∈ Rv . The combination of linearity, time-invariance, and completeness can be expressed in very many equivalent ways. In particular, the following are equivalent: 1. B ⊆ (Rw )N is linear, time-invariant, and complete;

Thoughts on System Identiﬁcation

393

2. B is a linear, shift invariant (:⇔ σB ⊆ B), closed subset of (Rw )N , with ‘closed’ understood in the topology of pointwise convergence; 3. ∃ R ∈ R•×w [ξ] such that B consists of the solutions w : N → Rw of R (σ) w = 0.

(Ker)

N

The set of behaviors B ⊆ (Rw ) that satisfy the equivalent conditions 1. to 3. is denoted by Lw , or, when the number of variables is unspeciﬁed, by L• . Thus, in eﬀect, L• = ∪w∈N Lw . Since B = ker (R (σ)) in (Ker), we call (Ker) a kernel representation of the behavior B. We will meet other representations later. But ﬁrst we introduce a characterization that is mathematically more abstract, but very pertinent in the context of SYSID. It identiﬁes a behavior B ∈ L• with an R [ξ]-module. Consider B ∈ Lw . The polynomial vector n ∈ R1×w [ξ] is called an annihilator (or a consequence) of B if n (σ) B = 0, i.e. if n (σ) w = 0 for all w ∈ B. Denote by NB the set of annihilators of B. Observe that NB is an R [ξ]-module. Indeed, n ∈ NB , n ∈ NB , and α ∈ R [ξ] imply n + n ∈ NB and αn ∈ NB . Hence the map B → NB associates with each B ∈ Lw a submodule of R1×w [ξ]. It turns out that this map is actually a bijection, i.e. to each submodule of R1×w [ξ], there corresponds exactly one element of Lw . It is easy to see what the inverse map is. Let K be a submodule of R1×w [ξ]. Submodules of R1×w [ξ] have nice properties. In particular, they are ﬁnitely generated, meaning that there exist elements (‘generators’) g1 , g2 , . . . , gg ∈ K such that K consists precisely of the linear combinations α1 g1 + α2 g2 + · · · + αg gg where the αk ’s range over R [ξ]. Now consider the system (Ker) with R = col(g1 , g2 , . . . , gg ) and prove that Nker(col(g1 , g2 ,..., gg )(σ)) = K (⊇ is obvious, ⊆ requires a bit of analysis). In terms of (Ker), we obtain the characterization [[ ker R (σ) = B ]]

⇔

[[NB =

R ]]

where R denotes the R [ξ]-module generated by the rows of R. The observation that there is a bijective correspondence between Lw and the R[ξ]-submodules of R1×w [ξ] is not altogether trivial. For instance, the surjectivity of the map B = ker R (σ) ∈ Lw → NB = R onto the R[ξ]-submodules of R1×w [ξ] depends on the solution concept used in (Ker). If we would have considered only solutions with compact support, or that are square integrable, this bijective correspondence is lost. Equations, in particular diﬀerence or diﬀerential equations, all by themselves, without a clear solution concept, i.e. without a deﬁnition of the corresponding behavior, are an inadequate speciﬁcation of a mathematical model. Expressed otherwise: the theory of (linear) systems is not the domain of pure algebra. Analysis

394

J.C. Willems

enters through the solution concept involved in the diﬀerence or diﬀerential equations, and guides the subsequent algebraic structure. The characterization of B in terms of its module of annihilators is very useful in the context of deterministic SYSID. It shows precisely what we are looking for in order to identify a system in the model class L• : (a set of generators of) the submodule NB . Behaviors in L• admit many other representations. The following two are exceedingly familiar to system theorists. In fact, 4. [[B ∈ Lw ]] ⇔ [[ ∃ integers m, p ∈ Z+ , with m + p = w, polynomial matrices P ∈ Rp×p [ξ] , Q ∈ Rp×m [ξ] , with det (P ) = 0, and a permutation matrix Π ∈ Rw×w such that B consists of all w : N → Rw for which there exist u : N → Rm and y : N → Rp such that P (σ) y = Q (σ) u

(I/O)

u ]]. The matrix of rational functions G = P −1 Q ∈ y p×m (R (ξ)) is called the transfer function of (I/O). Actually, for a given B ∈ Lw , it is always possible to choose Π such that G is proper. If we would allow a basis change in Rw , i.e. allow any non-singular matrix for Π (instead of only a permutation matrix), then we could always take G to be strictly proper. 5. [[B ∈ Lw ]] ⇔ [[ ∃ integers m, p, n ∈ Z+ with m + p = w, matrices A ∈ Rn×n , B ∈ Rn×m , C ∈ Rp×n , D ∈ Rp×m , and a permutation matrix Π ∈ Rw×w such that B consists of all w : N → Rw for which there exist u : N → Rm , x : N → Rn , and y : N → Rp such that and w = Π

σx = Ax + Bu, y = Cx + Du

(S)

u ]]. If we would allow a basis change in Rw , i.e. allow any y non-singular matrix for Π, then we could always take D = 0.

and w = Π

(I/O) is called an input/output (i/o) and (S) an input/state/output (i/s/o) representation of the corresponding behavior B ∈ Lw . Why, if any element B ∈ L• indeed admits a representation (I/O) or (S), should one not use one of these familiar representations ab initio? There are many good reasons for not doing so. To begin with, and most importantly, ﬁrst principles models aim at describing a behavior, but are seldom in the form (I/O) or (S). Consequently, one must have a theory that supersedes (I/O) or (S) in order to have a clear idea what transformations are allowed in bringing a ﬁrst principles model into the form (I/O) or (S). Secondly, as a rule, physical systems are simply not endowed with a signal ﬂow direction. Adding a signal ﬂow direction is often a ﬁgment of one’s imagination, and when something is not real, it will turn out to be cumbersome sooner or later. A third reason, very

Thoughts on System Identiﬁcation

395

much related to the second, is that the input/output framework is totally inappropriate for dealing with all but the most special system interconnections. We are surrounded by interconnected systems, but only very sparingly can these be viewed as input-to-output connections. Fourthly, the structure implied by (I/O) or (S) often needlessly complicates matters, mathematically and conceptually. A good theory of systems takes the behavior as the basic notion and the reference point for concepts and deﬁnitions, and switches back and forth between a wide variety of convenient representations. (I/O) or (S) have useful properties, but for many purposes other representations may be more convenient. For example, a kernel representation (Ker) is very relevant in SYSID. It suggests that we should look for (approximate) annihilators. On the other hand, when it comes to constructing trajectories, (S) is very convenient. It shows how trajectories are parameterized and generated : by the initial state x(1) ∈ Rn and the input u : N → Rm . Our next representation involves rational functions and is a bit more •×w ‘tricky’. Let G ∈ (R(ξ)) and consider the system of ‘diﬀerence equations’ G(σ)w = 0.

(G)

What is meant by the behavior of (G) ? Since G is a matrix of rational functions, it is not evident how to deﬁne solutions. This may be done in terms of co-prime factorizations, as follows. G can be factored G = P −1 Q with P ∈ R•×• [ξ] square, det(P ) = 0, Q ∈ R•×w [ξ] and (P, Q) left co-prime (meaning that F = [P Q] is left prime, i.e. [[(U, F ∈ R•×• [ξ]) ∧ (F = U F )]] ⇒ [[U is square and unimodular ]], equivalently ∃ H ∈ R•×• [ξ] such that F H = I). We deﬁne the behavior of (G) as that of Q(σ)w = 0, i.e. as ker (Q (σ)) Hence (G) deﬁnes a behavior ∈ Lw . It is easy to see that this deﬁnition is independent of which co-prime factorization is taken. There are other reasonable ways of approaching the problem of deﬁning the behavior of (G), but they all turn out to be equivalent to the deﬁnition given. Note that, in a trivial way, since (Ker) is a special case of (G), every element of Lw admits a representation (G). Certain integer ‘invariants’ (meaning maps from L• to Z+ ) associated with systems in L• are important in SYSID. One is the lag, denoted by L(B), deﬁned as the smallest L ∈ Z+ such that [[w|[t, t+L] ∈ B|[1,t+1] for all t ∈ N ]] ⇒ [[w ∈ B]]. Equivalently, the smallest degree over the polynomial matrices R such that B = ker (R(σ)). A second integer invariant that is important is the input cardinality, denoted by m (B), deﬁned as m, the number of input variables in any (I/O) representation of B. It turns out that m is an invariant (while the input/output partition, i.e. the permutation matrix Π in (I/O), is not). The number of output variables, p, yields the output cardinality p (B). A

396

J.C. Willems

third important integer invariant is the state cardinality, n (B), deﬁned as the smallest number n of state variables over all i/s/o representations (S) of B. The three integer invariants m (B), n (B), and L (B) can be nicely captured in one single formula, involving the growth as a function of t of the dimension of the subspace B|[1,t] . Indeed, there holds dim(B|[1,t] ) ≤ m (B) t + n (B)

with equality iﬀ t ≥ L (B) .

State models (S) are an example of the more general, but very useful, class of latent variable models. Such models involve, in addition to the manifest variables (denoted by w in (L)), the variables which the model aims at, also auxiliary, latent variables (denoted by in (L)). For the case at hand this leads to behaviors Bfull ∈ Lw+l described by R(σ)w = M (σ) ,

(L)

with R ∈ R•×w [ξ] and M ∈ R•×l [ξ]. Although the notion of observability applies more generally, we use it here for latent variable models only. We call Bfull ∈ Lw+l observable if [[ (w,

1)

∈ Bfull and (w,

2)

∈ Bfull ]] ⇒ [[

1

=

2

]].

(L) deﬁnes an observable latent variable system iﬀ M (λ) has full row rank for all λ ∈ C. For state systems (with x the latent variable), this corresponds to the usual observability of the pair (A, C). An important result, the elimination theorem, states that L• is closed under projection. Hence Bfull ∈ Lw+l implies that the manifest behavior B = {w : N → Rw | ∃

: N → Rl such that (L) holds}

belongs to Lw , and therefore admits a kernel representation (Ker) of its own. So, in a trivial sense, (L) is yet another representation of Lw . Latent variable representations (also unobservable ones) are very useful in all kinds of applications. This, notwithstanding the elimination theorem. They are the end result of modeling interconnected systems by tearing and zooming, with the interconnection variables viewed as latent variables. Many physical models (for example, in mechanics) express basic laws using latent variables. In the context of SYSID, the aim of most classical algorithms is in fact to arrive at a model (L), often unobservable, and with usually interpreted as an unobserved (stochastic) input – I will return to this later. But, of course, we are all most familiar with state models, states being the latent variables par excellence. In the next section we will see how latent variables can be used to express controllability.

4 Controllability As in many areas of system theory, controllability often enters in SYSID as a regularizing assumption. In the behavioral theory, an appealing notion of

Thoughts on System Identiﬁcation

397

controllability has been put forward. It expresses what is needed intuitively, it applies to any dynamical system, regardless of its representation, it has the classical state transfer deﬁnition as a special case, and it is readily generalized, for instance to distributed systems. It is somewhat strange that this deﬁnition has not been generally adopted. Adapted to the case at hand, it reads as follows. The time-invariant behavior B ⊆ (R• )N is said to be controllable if for any w1 ∈ B, w2 ∈ B, and t1 ∈ N, there exists a t2 ∈ N and a w ∈ B such that w (t) = w1 (t) for 1 ≤ t ≤ t1 , and w (t) = w2 (t − t1 − t2 ) for t > t1 + t2 . For B ∈ L• , one can take without loss of generality w1 = 0 in the above deﬁnition. The property just deﬁned is hence akin to what is sometimes called reachability, but here I use the term controllability as synonymous. Denote the controllable elements of L• by L•controllable and of Lw by Lwcontrollable . (Ker) deﬁnes a controllable system iﬀ R(λ) has the same rank for each λ ∈ C. There is a very nice representation result that characterizes controllability: it is equivalent to the existence of an image representation. More precisely, B ∈ L•controllable iﬀ there exists M ∈ R•×• [ξ] such that B equals the manifest behavior of the latent variable system w = M (σ) .

(Im)

In other words, iﬀ B = im (M (σ)). So, images, contrary to kernels, are always controllable. This image representation of a controllable system can always be taken to be observable. For B ∈ L• , we deﬁne its controllable part, denoted by Bcontrollable , as Bcontrollable := {w ∈ B | ∀t ∈ N, ∃t ∈ Z+ , and w ∈ B such that w (t) = 0 for 1 ≤ t ≤ t and w (t) = w(t − t − t ) for t > t + t }. Equivalently, Bcontrollable is the largest controllable subsystem contained in B. It turns out that two systems of the form (I/O) (with the same input/output partition) have the same transfer function iﬀ they have the same controllable part. Consider B ∈ Lw . The vector of rational functions n ∈ R1×w (ξ) is called a rational annihilator of B if n (σ) B = 0 (note that, since we gave a meaning to (G), this is well deﬁned). Denote by NBrational the set of rational annihilators of B. Observe that NBrational is a R(ξ)-subspace of R1×w (ξ). The map B → NBrational is not a bijection from Lw to the R(ξ)-subspaces of R1×w (ξ). Indeed, [[ NBrational = NBrational ]] ⇔ [[ Bcontrollable = Bcontrollable ]]. In fact, there exists a bijective correspondence between Lwcontrollable and the R(ξ)-subspaces of R1×w (ξ). Summarizing, R [ξ]-submodules of R1×w [ξ] stand in bijective correspondence with Lw , with each submodule corresponding to the set of polynomial annihilators, while R(ξ)-subspaces of R1×w (ξ) stand in bijective correspondence with Lwcontrollable , with each subspace corresponding to the set of rational annihilators.

398

J.C. Willems

Controllability enters in a subtle way whenever a system is identiﬁed with its transfer function. Indeed, it is easy to prove that the system described by w2 = G(σ)w1 ,

w=

w1 , w2

(T)

a special case of (G), is automatically controllable. This again shows the limitation of identifying a system with its transfer function. Two input/output systems (I/O) with the same transfer function are the same iﬀ they are both controllable. In the end, transfer function thinking can deal with non-controllable systems only in contorted ways. A property related to controllability is stabilizability. The behavior B ⊆ (R• )N is said to be stabilizable if for any w ∈ B and t ∈ N, there exists a w ∈ B such that w (t ) = w (t ) for 1 ≤ t ≤ t, and w (t) → 0 for t → ∞. (Ker) deﬁnes a stabilizable system iﬀ R(λ) has the same rank for each λ ∈ C with Real(λ) ≥ 0. An important system theoretic result (leading up to the Youla-Kuvcera parametrization of stabilizing controllers) states that •×w B ∈ Lw is stabilizable iﬀ it allows a representation (G) with G ∈ (R(ξ)) left prime over the ring RH∞ (:= {f ∈ R(ξ) | f is proper and has no poles in the closed right half of the complex plane }). B ∈ Lw is controllable iﬀ it allows a representation w = G(σ) with G ∈ (R(ξ))w×• right prime over the ring RH∞ . Autonomous systems are on the other extreme of controllable ones. B ⊆ (R• )N is said to be autonomous if for every w ∈ B, there exists a t ∈ N such that w|[1,t] uniquely speciﬁes w|[t+1,∞) , i.e. such that w ∈ B and w|[1,t] = w |[1,t] imply w = w. It can be shown that B ∈ L• is autonomous iﬀ it is ﬁnite dimensional. Autonomous systems and, more generally, uncontrollable systems are of utmost importance in systems theory (including SYSID), in spite of much system theory folklore claiming the contrary. Controllability as a systems property is much more restrictive than is generally appreciated.

5 The MPUM The SYSID algorithms which I will discuss associate with an observed timeseries w ˜T an element of L• , i.e. a complete linear time-invariant system. The ﬁrst model class that comes to mind is Lw , i.e. models described by R (σ) w = 0.

(Ker)

However, for reasons which I will go into more detail later, important ﬂexibility is gained by aiming at a latent variable model R(σ)w = M (σ) , w+l

(L) w

with full behavior Bfull ∈ L and manifest behavior B ∈ L . In this model we assume that the observed data is generated by a system that contains unobserved latent variables.

Thoughts on System Identiﬁcation

399

Assume that the model (L) is deduced from the observed data w ˜T =

w ˜ (1) , w ˜ (2) , . . . , w ˜ (T) .

How should we assess this choice ? There are several conﬂicting quantitative measures which we wish to keep small, e.g. the complexity, the misﬁt, and the latency. The complexity roughly measures how many adjustable parameters the model has. The dimension of the subspaces dim Bfull|[1,t] for t ∈ N, or, more simply, the triple (m (Bfull ) , n (Bfull) , L (Bfull )) is a good measure of the complexity. The misﬁt measures the extent to which the model fails to explain the observations. The minimum of ||w˜T − w|[1,T] || 2 ([1,T],Rw ) over w ∈ B is a good measure of the misﬁt. The latency measures the extent to which the model needs latent variables to explain the observations. The minimum of || |[1,T] || 2 (N,Rl ) over such that (w, ) ∈ Bfull and w|[1,T] = w ˜T is a good measure of the latency. Another prevalent way for assessing an identiﬁcation procedure is consistency. This means that if the data w ˜T is actually generated by an element of the model class, then the identiﬁcation procedure should return the model that generated the data. Consistency is primarily used in a stochastic setting for inﬁnite observed time series (T = ∞), but one can apply it for deterministic SYSID and T ﬁnite as well. A related consideration for assessing an identiﬁcation algorithm is the way the estimate based on w ˜T behaves as T → ∞, if it is considered to be a truncation of an inﬁnite time series. In the remainder of this section, we discuss how we can associate a deterministic model (Ker) ‘exactly’ (with zero misﬁt) to an observed time-series. The idea is to obtain the least complex model that explains the data exactly. The concept which implements this idea is the Most Powerful Unfalsiﬁed Model (MPUM). This notion was introduced in [35, Part II] for inﬁnite observed time-series (T = ∞ in w ˜T ). Here I adapt it to the case of ﬁnite time-series. N The behavior B ⊆ (Rw ) is unfalsiﬁed by w ˜T if w ˜ (1) , w ˜ (2) , . . . , w ˜ (T) ∈ B|[1,T] . N

N

B ⊆ (Rw ) is more powerful than B ⊆ (Rw ) if B ⊂ B . Hence, following Karl Popper (1902–1994), the more a model forbids, the better it is. B ∈ Lw is the MPUM in Lw for w ˜T if it is unfalsiﬁed by w ˜T and more powerful than any other behavior in Lw that is also unfalsiﬁed by w ˜T . It is easy to prove that for an inﬁnite observed time-series (T = ∞ in w ˜T ), this MPUM always exists. In fact, it is equal to the closure (in the topology of pointwise convergence) of span{σ t w ˜T , t ∈ N}. For T ﬁnite, the MPUM in Lw may not exist, but it is not particularly useful anyway: when it exists, its behavior will be ﬁnite dimensional (corresponding to an autonomous behavior). However, it is desirable, also in the case T < ∞, to obtain an MPUM that can also recover a behavior with some of the variables free inputs. This can be accomplished

400

J.C. Willems

by looking for the MPUM in the class of systems with a restricted lag. This can be viewed as limiting the complexity of the model. Deﬁne LwL := {B ∈ Lw | L (B) ≤ L}. ˜T if it is unfalsiﬁed by w ˜T and more powerful B ∈ Lw is the MPUM in LLw for w than any other behavior in LwL that is also unfalsiﬁed by w ˜T . Denote the MPUM in LwL for w ˜T by BL,w˜T . Questions that arise are: When does BL,w˜T exist ? How can it be computed from the data w ˜T ? If w ˜T is generated by an element B ∈ Lw , when will BL,w˜ T = B ? Conditions for existence of this MPUM are readily deduced from the Hankel matrix formed by the data. We ﬁrst introduce our notation for Hankel matrices (meaning block Hankel matrices). Exclusively Hankel matrices formed by vectors are needed. Let f : [1, t] → Rn . Deﬁne the Hankel matrix formed by f with depth Δ1 and width Δ2 , Δ1 + Δ2 − 1 ≤ t, by    H(Δ1 , Δ2 ) (f ) :=  

f (1) f (2) .. . f (Δ1 )

f (2) f (3) .. .

··· ··· .. .

f (Δ1 + 1) · · ·

f (Δ2 ) f (Δ2 + 1) .. .

    

f (Δ1 + Δ2 − 1)

The left kernel of a matrix M ∈ Rn1 ×n2 , denoted by leftker (M ), consists of the row vectors n ∈ R1×n1 that are annihilated by postmultiplication by M : nM = 0. In the case of a Hankel matrix, it is often useful to view the elements in the left kernel as vector polynomials, as follows. Assume that n = n0 n1 · · · nΔ1 −2 nΔ1 −1 ∈ leftker H(Δ1 ,Δ2 ) (f ) with the nk ’s ∈ R1×n . Associate with n the polynomial vector n0 + n1 ξ + · · ·+ nΔ1 −2 ξ Δ1 −2 + nΔ1 −1 ξ Δ1 −1 ∈ R1×n [ξ], and denote it, with slight abuse of notation, also as n (ξ). Note that it may appear as if degree (n) = Δ1 − 1, but since some of the coeﬃcients of n may be zero, the actual degree may be lower. Call leftker H(Δ1 , Δ2 ) (f ) module-like if [[ [ n0 n1 · · · nΔ1 −3 nΔ1 −2 0 ] ∈ leftker H(Δ1 , Δ2 ) ]] ⇒ [[ [ 0 n0 n1 · · · nΔ1 −3 nΔ1 −2 ] ∈ leftker H(Δ1 , Δ2 ) (f ) (f ) ]]. Equivalently, if [[ n(ξ) ∈ leftker H(Δ1 , Δ2 ) (f )

∧ degree(n) < Δ1 − 1 ]] ⇒ [[ ξn(ξ) ∈ leftker H(Δ1 , Δ2 ) (f ) ]].

This last implication shows where the terminology ‘module-like’ refers to. These notions are very useful for computing BL,w˜T (which may not exist – take w ˜T = (1, 1, 2, 2, 2) and L = 2). Observe that if B ∈ Lw is unfalsiﬁed by w ˜T , then

Thoughts on System Identiﬁcation

401

[[ (n ∈ NB ) ∧ (degree(n) ≤ L) ]] ⇒ [[ n ∈ leftker H(L+1,T−L) (w ˜T ) ]]. It is logical to aim at the module generated by leftker H(L+1,T−L) (w ˜T ) as the annihilators of the MPUM BL,w˜T . However, for arbitrary w ˜T , this module may contain falsiﬁed elements (consider, e.g. w ˜T = (0, 0, . . . , 0, 0, 1)). The module-like property ensures that all elements in the module generated by leftker H(L+1,T−L) (w ˜T ) are unfalsiﬁed. This leads to the following results. (i) A suﬃcient condition for the existence of the MPUM BL,w˜T is that leftker H(L+1,T−L) (w ˜T ) is module-like. (ii) leftker H(L+1,T−L) (w ˜T ) is module-like iﬀ rank H(L,T−L) (w ˜T ) = rank H(L,T−L+1) (w ˜T ) . (iii) If leftker H(L+1,T−L) (w ˜T ) is module-like, then BL,w˜T = ker (N (σ)) , with N ∈ R•×w [ξ] such that the R [ξ]-module generated by its rows is equal to the R [ξ]-module generated by leftker H(L+1,T−L) (w ˜T ) . ˜T recovers the system that Under mild conditions, the MPUM in LwL for w generated the data. In other words, the procedure is consistent. The crucial condition here is that the input component in the observations must be persistently exciting. f : [1, T] → R• is said to be persistently exciting of order P if the rows of the Hankel matrix H(P,T−P+1) (f ) are linearly independent. u = We have the following ‘consistency’ result. Consider B ∈ Lw . Let y Π −1 w be an input/output partition of B as in (I/O). Denote the corresponding observed input component of w ˜T by u ˜T . It can be shown that BL,w˜T = B if the following conditions are satisﬁed: (i) (ii) (iii) (iv)

w ˜T ∈ B|[1,T], L(B) ≤ L, B is controllable, u ˜T is persistently exciting of order L + n(B) + 1.

Moreover, under these conditions, the left kernel of H(L+1,T−L) (w ˜T ) is modulelike, and therefore B = ker (N (σ)) , where N ∈ R•×w [ξ] is any polynomial matrix such that the R [ξ]-module generated by its rows equals the R [ξ]module generated by leftker H(L+1,T−L) (w ˜T ) . In a typical application of this result, assume that an inﬁnite time-series w ˜: N → Rw is observed, generated by an unknown, but controllable, system B ∈ Lw , for which upper bounds L for the lag L(B) and n for the state dimension n(B) are known (since L(B) ≤ n(B), it suﬃces to have the upper bound n). Then, if the observed input component u ˜ of w ˜ is persistently exciting of order ˜T is L + n + 1, the MPUM BL,w˜T will be equal to B for all T such that u persistently exciting of order L + n + 1. So, this exact deterministic SYSID algorithm yields consistency in ﬁnite time.

402

J.C. Willems

We now turn to the computation of BL,w˜T . One possibility suggested by the above results is to compute a basis for the whole left kernel of H(L+1,T−L) (w ˜T ). This is feasible (also approximately, using SVD-like algorithms), but it is not necessary nor eﬃcient to compute the whole left kernel, especially when the data is generated by a system in LwL for which the lags in the diﬀerence equations (Ker) vary widely, or when L is only a rough upper bound for L(B). It suﬃces to compute vector polynomials n1 , n2 , . . . , ng ∈ R1×w [ξ] such that the R [ξ]-module spanned by n1 , n2 , . . . , ng equals the R [ξ]-module spanned by the left kernel of H(L+1,T−L) (w ˜T ). The problem hence comes down to obtain a set of polynomial vectors in the left kernel of H(L+1,T−L) (w ˜T ) that generate this submodule. This is algorithmically much simpler that computing the whole left kernel, which may have a dimension that is much larger than the dimension (the number of generators) of this submodule. This submodule may be computed with an algorithm that is recursive in the observation horizon T or in the lag L. An algorithm that is recursive in T has been obtained in [20, 36]. This algorithm is based on a recursive computation of the MPUM for inﬁnite time-series, and is a generalization of the Berlekamp-Massey decoding algorithm. We have recently also obtained an algorithm that is recursive in L. The details will be reported elsewhere.

6 Subspace ID The importance and usefulness of state models are beyond dispute. By explicitly displaying the memory of a system, state models bring the dynamical properties of a system to the surface, clearly into view. In fact, since the 1960’s, input/state/output models are, up to a fault, used as the basic framework for studying open dynamical systems. Since the 1980’s, it has become apparent that state models are also very well suited for model reduction. The highlight of this development is Keith Glover’s classic paper [12] on balanced realizations and AAK type model reduction algorithms. The one area in systems theory where state models play a somewhat secondary role is SYSID. In earlier sections, I have dealt extensively with the multitude of representations of elements of L• . SYSID algorithms typically aim at one of these representations, usually a kernel representation (Ker), an input/output version (I/O), or a latent variable representation (L). One major exception to this are the subspace identiﬁcation algorithms. These algorithms pass from the data w ˜T to an input/state/output representation (S). In this section, I deal with these algorithms. Contrary to many authors, I do not consider the classical realization algorithms as part of SYSID. In realization theory, one has a model at the very outset, a model that, for example, gives the output as a convolution of the input with the impulse response matrix, and the problem is to ﬁnd an equivalent state representation. Finding the parameters of a state representation from the impulse response is a representation problem and does not have much to do with SYSID (although one may

Thoughts on System Identiﬁcation

403

get many good ideas for SYSID from these algorithms). Much more SYSID oriented are algorithms that pass from w ˜T or (˜ uT , y˜T ) if an input/output partition is known, to a state representation of BL,w˜ T . This is the problem that I will discuss now. In this section we consider SYSID algorithms that aim at i/s/o models (S). As always in state representations, we meet the issue of non-uniqueness, both of the state representation parameters, and of the state trajectory corresponding to a given w-trajectory. It is easy to show that every B ∈ Lw admits an observable representation (S) (but not necessarily a controllable one!). Moreover, for a ﬁxed input/output partition Π, observable representations (S) of the same B ∈ Lw are unique up to a basis choice in the state space. If we allow also Π (i.e. the input/output partition) to vary, things become a bit more complicated, but we need not deal with this here. The discussion in this section is a bit informal: we assume, where needed, that w ˜T is such that the MPUM in LwL exists, that L and T are suﬃciently large, controllability, observability, persistency of excitation, etc., and also that the input/output partition in BL,w˜T is known. Assume that from the observations w ˜T , we have somehow identiﬁed a model B ∈ Lw such that w ˜T ∈ B|[1,T] (for example, assume that we have identiﬁed the MPUM BL,w˜T ). We could then compute a state representation (S) for B, and obtain a corresponding state trajectory x ˜T =

(˜ x (1) , x ˜ (2) , . . . , x ˜ (T))

x ˜ (t) ∈ Rn .

uT , y˜T ) are related This lays out the path to go from w ˜T to x ˜T . x ˜T and w ˜T ∼ = (˜ by the equations x ˜(2) x ˜(3) · · · x ˜(T) y˜(1) y˜(2) · · · y˜(T − 1)

=

AB CD

x ˜(1) x˜(2) · · · x ˜(T − 1) . u ˜(1) u˜(2) · · · u ˜(T − 1)

(♣)

However, if we could somehow obtain ﬁrst the state trajectory x ˜T , directly from w ˜T , without using the underlying model, we could subsequently solve the linear system of equations (♣) for the unknown parameter matrices (A, B, C, D). This would yield a SYSID algorithm that identiﬁes a state representation for BL,w˜T . The advantage – and a major one indeed – of dealing with the equations (♣) is that, from a numerical linear algebra point of view, they lend themselves very well to approximations. Indeed, one can ﬁrst use SVD-like algorithms to reduce the state dimension by rank reduction of the matrix x ˜(1) x˜(2) · · · x ˜(T − 1) x ˜(T) and subsequently solve the equations (♣) approximately for (A, B, C, D) in a least squares (LS) sense. These numerical features are very well explained and eﬀectively exploited in [30]. The following question emerges: How can we pass directly from w ˜T to the corresponding state trajectory x˜T of a state representation of the MPUM BL,w˜T ?

404

J.C. Willems

One algorithm which achieves this is based on partitioning the Hankel matrix of the data into the ‘past’ and the ‘future’, as follows   w(1) ˜ w(2) ˜ ··· w(T ˜ − 2L − 1)   .. .. .. ..   . . . .  ← past P   w(L ˜ + 2) ··· w(T ˜ − L − 1)    ˜ + 1) w(L    −−−−− −−−−− −−− −−−−−−−      w(L ˜ + 2) w(L ˜ + 3) ··· w(T ˜ − L)  ← future F  .. .. .. ..     . . . . w(2L ˜ + 2) w(2L ˜ + 3)

···

w(T) ˜

i.e. 

 H(L+1, T−L−1) (w ˜T ) H(2L+2, T−2L−1) (w ˜T ) =  − − − − − − − − − − −  H(L+1, T−L−1) (σ L+1 w ˜T )

← past P ← future F

Consider now the span of the rows of the ‘past’ matrix P and the span of the rows of the ‘future’ matrix F . It turns out that the intersection of these spans is equal to the state space of BL,w˜T , and that the linear combinations of the rows of P (or F ) contained in this intersection give the ‘present’ state trajectory! More precisely, assume that the span of the rows of the matrix X equals the intersection of the span of the rows of P with the span of the rows of F . Then the columns of X are the ‘present’ state trajectory. Note that we may as well assume that the rows of X are linearly independent. The fact that X is unique up to pre-multiplication by a nonsingular matrix corresponds to the freedom of the choice of the basis of the state space of the underlying system. In other words, X = (˜ x (L + 2) , x ˜ (L + 3) , . . . , x ˜ (T − L − 1) , x˜ (T − L)) . This fact, ﬁrst noticed in [35, sections 15–17], and generalized in many directions by De Moor and co-workers (see e.g. [30]), allows to identify the system parameter matrices (A, B, C, D) of BL,w˜T in input/state/output form by solving equations as (♣). This intersection result is reminiscent of the following observation, usually attributed to Akaike [1]. Consider a zero mean stationary ergodic gaussian vector process z, z(t) ∈ Rz , t ∈ Z, with a rational spectral density. Let z˜ : Z → Rz be a realization of it. Now form the Hankel matrix

Thoughts on System Identiﬁcation



.. .. .. .. .. .. .. . . . . . .  .  ··· z˜ (−2) z˜ (1) z˜ (0) · · · z˜ (t − 1) · · ·   ··· z˜ (−1) z˜ (0) z˜ (1) · · · z˜ (t) ···   − − − − − −− − − −− − − − − − − − − −− − − −   ··· z˜ (0) z˜ (1) z˜ (2) · · · z˜ (t + 1) · · ·   ··· z ˜ (1) z˜ (2) z˜ (3) · · · z˜ (t + 2) · · ·  .. .. .. .. .. .. .. . . . . . . .

405

            

← past

← future

Then the orthogonal projection (in the inner product induced by the second order averages) of the span of the rows of the future onto the span of the rows of the past is ﬁnite dimensional, and the orthogonal projection of the rows of the future onto the span of the rows of the past yields a realization of the corresponding ‘past-induced’ Markovian trajectory x ˜ associated with z˜. But, how do we compute the common linear combinations spanned by the 1 rows of M1 and M2 in the partitioned matrix M M2 ? This can be done by M1 computing a basis for the left kernel of M2 . Indeed, if [n1 n2 ] is in the left kernel, then n1 M1 = −n2 M2 yields one of these linear combinations. By letting [n1 n2 ] range over a basis of the left kernel, all common linear combinations are obtained. This can be applied to

P −− F

and allows to comP

pute the desired state trajectory by computing the left kernel of −− . The F disadvantage of this algorithm is that it does not make use of the Hankel structure of the matrices P and F (a common drawback of subspace algorithms). Equivalently, it makes no use of the module structure of the left kernel. For L large (say, much larger than L(B)), the algorithms that compute the whole left kernel risk at being very ineﬃcient. But, there is more. As we have shown P in the previous section, the left kernel of −− can in fact be deduced from F the left kernel of P alone by simple module operations, and therefore there is no need to consider the separation into ‘past’ and ‘future’. These considerations lead to the following algorithm for constructing the state trajectory from the observed data, assuming persistency of excitation, controllability, L suﬃciently large, etc. Let N ∈ R•×w [ξ] be a polynomial matrix such that the R [ξ]-module generated by its rows equals the R [ξ]module generated by leftker H(L+1,T−L) (w ˜T ) , and assume degree(N ) ≤ L. In the previous section, we have explained how to view elements in this left kernel as vector polynomials. The construction of the state trajectory involves the ‘shift-and-cut’ operator π on R [ξ], deﬁned by π : p0 + p1 ξ + p2 ξ 2 + · · · + pn ξ n → p1 + p2 ξ + + · · · + pn ξ n−1 . π can be extended in an obvious way to act on polynomial vectors and matrices. Now deﬁne XN as XN := col(πN, π 2 N, . . . , π degree(N )−1 N ),

406

J.C. Willems

and view XN as a matrix by reversing the process by which we viewed elements in the left kernel of H(L+1,T−L) (w ˜T ) as polynomial vectors. Then XN H(L+1,T−L) (w ˜T ) = [˜ x (1) x˜ (2) · · · x˜ (T − L − 1) x˜ (T − L)] . This yields a very eﬀective construction of a state trajectory. This algorithm is actually an implementation of the ‘cut-and-shift’ algorithm for constructing a state representation starting from a kernel representation developed in [27]. There exist many related subspace identiﬁcation type algorithms, for example based on oblique projections and various LS methods [30], [34], [22]. We have already mentioned that subspace ID is very well suited, pragmatically, through rank reduction of the state trajectory X , followed by LS methods for solving (♣), for identifying a model that ﬁts the observed data w ˜T approximately. Of course, the most satisfying way theoretically of approaching SYSID through approximate modeling is by deﬁning a misﬁt criterion, say misﬁt(w ˜T , B) = minimum ||w ˜T − w|[1,T] || w∈B

2 ([1,T],R

w)

.

and then minimizing the misﬁt misﬁt(w˜T , B) over a model class, for example the elements of B ∈ Lw with a limited complexity, with complexity deﬁned as something like (m(B), n(B), L(B)). This is the approach taken in [16], [28], and [22]. This last reference shows how to approach this problem using STLS (structured total least squares) algorithms.

7 Latency Minimization The usual model class considered in SYSID lead to equations of the form R (σ) w = M (σ) ε

(E)

with R and M real polynomial matrices of suitable sizes. These equations involve, in addition to the time-series w : N → Rw , also the time-series ε : N → Re . In (E), w is the time-series which is observed on the interval [1, T] through w ˜T , and ε consists of unobserved latent variables, which serve to help to explain the data. In the behavioral language, this means that we start with a latent variable representation and aim at explaining the observed manifest variables w with the help of the unobserved latent variables ε. It is often useful to consider a more structured version of (E), for example, P (σ) y = Q (σ) u + M (σ) ε,

w=

u , y

(E )

with det(P ) = 0, ensuring that both u and ε are inputs, and y is the output. The question then basically is to identify the behavior Bfull , or the polynomial matrices (R, M ) or (P, Q, M ), from observations (of a ﬁnite sample) of w, or u and y, entangled by an unobserved latent variable trajectory ε.

Thoughts on System Identiﬁcation

407

The performance, from a SYSID point of view, of the model (E) with behavior Bfull ∈ Lw+e can be assessed by the latency, say latency(w˜T , Bfull) :=

minimum

[[(w,ε)∈Bfull ]] ∧ [[w|[1,T] =w ˜ T ]]

||ε||

2 (N,R

e)

.

Subsequently minimizing the latency over the model class (a family of Bfull’s) leads to a SYSID algorithm. Note that because the models (E) are usually unobservable (in the sense that ε cannot be deduced from w, R, and M ), it is in general not possible to reformulate the latency criterion in terms of the w variables alone. The following questions arise. Given the behavior Bfull ∈ Lw+e how do we compute the latency in terms of the polynomial matrices (R, M )? Given a model class M ⊆ Lw+e , how do we minimize the latency over M? The computation of the latency is a deterministic Kalman ﬁltering problem. The minimization over the model class usually leads to a non-convex minimization problem. It turns out, in fact, that from a numerical (but not from a conceptual) point of view, latency minimization is precisely what is done in prediction error minimization methods (PEM) [21] (strongly related to maximum likelihood (ML)). The diﬀerence is ‘merely’ the interpretation of the latent variables ε. In latency minimization, ε is introduced in order to explain the observed data w ˜T (but no physical reality is attached to it), while in PEM and ML, ε is interpreted as a stochastic ‘disturbance’ input which, together with the input component in the w variable in (E) (see (E )) and the initial conditions, produces the data. Under suitable stochastic assumptions, as ε independent of the input component u of (E ), one obtains precisely the same estimate for (R, M ) by using latency minimization as by using (PEM) or (ML). In latency minimization, one wants to keep ε small, while explaining the observations, in PEM one minimizes the a posteriori prediction error in a stochastic sense, and in ML one maximizes the likelihood of the observations, given the model. Assume now that (R, M ) in (E), equivalently (P, Q, M ) in (E ), have been identiﬁed from the observations w ˜T , how should one proceed? Which model does this identiﬁcation procedure entail? If these estimates have been obtained by latency minimization, then it is natural to take R(σ)w = 0 i.e. P (σ)y = Q(σ)u as the model to be employed for prediction, control, or what have you. This is diﬀerent, of course, from taking the manifest behavior of (E). Setting ε = 0 in (E) yields a much smaller behavior than eliminating ε from (E). However, if the estimates have been obtained by (PEM) or (ML), then it is natural to stick to (E), and take into consideration the identiﬁed stochastic characteristics of the variables ε in (E) or (E ). This stochastic model can then be used for predicting future outputs from future inputs, for control, etc. Does (E) provide additional ﬂexibility compared to (Ker)? Is the set of w trajectories that can be modelled this way larger than what can be achieved without the ε’s? This may appear so at ﬁrst sight, but it is not. Indeed, by the elimination theorem the ‘manifest’ behavior of (E), i.e.

408

J.C. Willems

B = {w :→ Rw | ∃ ε → Re such that (E) holds} in fact also belongs to Lw . In other words, there exists R ∈ R•×w [ξ] such that B = ker (R (σ)), yielding the kernel representation R (σ) w = 0. What is then the rationale for introducing ε and using the model class (E)? We can intuitively interpret the use of latency minimization, followed by setting the latent variables ε = 0 and using R(σ)w = 0 as the identiﬁed model, as a way of minimizing some sort of misﬁt. But it is a misﬁt that involves unobservable latent variables, with ‘unobservable’ interpreted in the technical sense for latent variable systems explained earlier. The advantage of introducing latent variables above straight misﬁt minimization can be seen by considering the case w = 1. B ∈ L1 implies that either B = RN , not useful as a model, or that B is an autonomous system, and therefore ﬁnite dimensional, consisting of trajectories that are a ﬁnite sum of polynomial exponentials. If we assume in addition that B is stable, i.e. [[ w ∈ B ]] ⇒ [[ w(t) → 0 for t → ∞ ]], then we see that an autonomous behavior B cannot adequately capture the dynamic features of a persistent trajectory w ˜T . The assumption of the presence of unobservable latent inputs ε in (E) indeed oﬀers better modeling possibilities for data ﬁtting. Of course, one can also use a combination of the straight misﬁt minimization and latency minimization. It is an open question if a model obtained by latency minimization and setting ε = 0, i.e. R(σ)w = 0, or P (σ)y = Q(σ)u, equivalently, obtained by (PEM) and keeping the deterministic part, or a model obtained by misﬁt minimization, will do signiﬁcantly better than a model (in a corresponding model class) obtained from, say, a heuristic subspace ID type algorithm obtained by reduction of the state trajectory X followed by a LS solution (♣), as explained earlier. Extensive testing on industrial data [23] suggests that models obtained by latency minimization or (PEM) give a marginally better ﬁt in predicting future outputs from future inputs as compared to subspace ID, but at the expense of a much larger computational eﬀort. Systematic methods to ﬁt a linear dynamical system to the data will lead to a reasonable model. It makes sense to expect that a linear approximation found during the learning stage will prevail during the validation stage. But there is no reason to expect that statistical features which happen to be present in the data during the learning stage to prevail during the validation stage. Which interpretation, latency minimization, or the stochastic interpretation of (PEM) or (ML) should one prefer? My own preference lies squarely with the latency minimization. The main argument is that, in my opinion, deterministic approximation articulates in a much more satisfying way the essence of the SYSID problem. It seems to me that the lack of ﬁt between the observed data and the chosen model will in most practical situations (both in engineering and in economics) be due to the fact that the data has been produced by a nonlinear, high order, time varying system, inﬂuenced by spurious inputs and other unknown interactions with the environment. Minimization of the latency or the misﬁt pointedly articulates the ﬁtting problem. Stochas-

Thoughts on System Identiﬁcation

409

tic methods assume that the lack of ﬁt is due to some unobserved input with stochastic regularity. It is unclear how to justify such an assumption, let alone the statistical independence of ε and the driving input component of u, etc. Furthermore, it is awkward how to deal with the additive noise. Obviously, in many applications this noise term cannot be justiﬁed when the system is at rest, it is incompatible with physical properties as dissipativeness, it often assumes an inﬁnite supply of energy, etc. In typical applications it is simply artiﬁcial to insist on a stochastic interpretation, even as an approximation. It is even hard to imagine how such stochastic regularity could come about in reality. The whole issue is obviously a very complex one, that can not be discussed without examining the interpretation of probability.

8 Interpretations of Probability There is scarcely any ﬁeld of science for which the foundations and the interpretation have stirred as much controversy and debate as probability. It shares this doubtful honor with the rules of inference (tertium non datur?) and with quantum mechanics, but even there the interpretation question is to a large extent tied up with the meaning of probability. It is ironic that both quantum mechanics and probability, perhaps the most successful scientiﬁc developments of the twentieth century, appear to have such shaky foundations. The foundations and interpretation of probability have been discussed by some of the ﬁnest scientiﬁc minds in history. Jakob Bernoulli, Borel, Carnap, de Finetti, Jeﬀreys, Fr´echet, Keynes, Kolmogorov, Laplace, von Mises, Poincar´e, Popper, Ramsey, Venn, are only a few of the many scientists who have written books on this topic (see [8, 14, 32] for the history and the status of this subject). In this section, I describe my own impressions about this, brieﬂy, and perhaps too superﬁcially. The subject area is quite subtle, unsettled, and very much la mer a ` boire. Four main views have emerged, in combination with a seemingly uncountable number of intermediate nuances. The ﬁrst two interpretations are considered objective, empirical, physical. The ﬁrst one is the relative frequency point of view, formalized by von Mises [31]. Popper [25, 26] devised a second view, in which probability is interpreted as ‘propensity’. The third and fourth main interpretations are epistemological and lead to probability as degree of belief. They often go under the name ‘Bayesian’, but this certainly does not nail down the viewpoint uniquely [13]. The third interpretation, called the ‘logical’ theory, deduces probability from relations among basic propositions. Keynes [19] is usually considered the person who introduced this approach, later put in a more deﬁnitive form by Carnap [3]. The fourth interpretation, championed by De Finetti [5], is radically subjective: probability articulates one’s own personal degree of belief. Nothing more, nothing less. Between these interpretations, we ﬁnd many subtle nuances leading, in the words of Galavotti [8], to “a whole array of seemingly irreconcilable perspectives”.

410

J.C. Willems

The relative frequency interpretation assumes that we have an ensemble of objects, and that each individual object has certain characteristics, which differ from object to object. This characteristic then becomes a random variable. For example, if a characteristic x is real-valued, then its distribution function F (x) is equal to the relative frequency of objects for which the characteristic is less than or equal to x. If we apply this to a well-deﬁned existing ﬁnite ensemble and a well-deﬁned set of characteristics, there seems no apparent interpretational diﬃculty. However, for applications in which the ensemble involves events that are still to come, as is unavoidable in time-series analysis (and hence in stochastic SYSID), or that are only potentially realized (ﬂips of a coin, throws of a die), or are too numerous to be realizable (bridge hands), or when the ensemble is inﬁnite, the frequency interpretation poses severe interpretation problems. Popper formulated the propensity interpretation in order to accommodate for the fact that quantum mechanics asks for a physical interpretation of a single event probability. In this interpretation, certain physical objects, even if prepared in identical ways and operating in identical environments, may react diﬀerently, with outcomes that have ﬁxed probabilities. This probability will then be the relative frequency obtained from repeated identical trials. Since, in Feynman’s words, nobody understands quantum mechanics, I must (reluctantly) accept that this view may be appropriate, perhaps unavoidable, for accommodating the present day orthodox view of quantum mechanics. But I ﬁnd it hard to accept propensity as a description of our day to day (seemingly) random experiences. However, some physicists appear to suggest that the propensity interpretation may even apply to mundane things like coin tossing, and there are many publications concerning the transition from ‘determinism’ to ‘randomness’ of a wobbling coin. In fact, recently experiments have been conducted which show [6] that a coin, repeatedly ﬂipped by means of a carefully constructed mechanical device, lands practically each time on the same side. These experimenters summarize their results by stating – ironically, I hope – ”We conclude that coin tossing is ‘physics’, not ‘random’”, as if there was a reason to expect otherwise. These experiments have been avidly reported in the popular press, whose main concern seems to be that the toss at the beginning of a football match may not be fair. The logical theory is based on rules of inductive logic, leading to relations between propositions, for example that certain hypotheses make other hypotheses more probable. Or that certain basic ‘symmetric’ events (as any pair of cards being on top of a well-shuﬄed deck) demand equal probabilities. This is useable for games of chance, with discrete outcomes, where probability quickly reduces to combinatorics. However, it is hard to apply this principle to continuous random variables, as the many discussions surrounding the principles of indiﬀerence or insuﬃcient reason demonstrate. The personal belief interpretation is in a sense the clearest of them all. By stating a probability, an expert quantiﬁes his or her degree of belief in the occurrence of an event. The diﬃculty is to describe how these degrees

Thoughts on System Identiﬁcation

411

of belief are arrived at, certainly when it concerns complex phenomena, in turn inﬂuenced by many uncertainties, leading to a seemingly inﬁnite regress. Think, for example, of the problem of trying to arrive, in a systematic way, at the probability as a degree of belief of the Dutch soccer team winning the coming World Cup. It is clear what probability as degree of belief means, but it is diﬃcult to obtain it in a scientiﬁc manner, let alone to falsify. And, of course, it leads to the question why I should care about your degree of belief. There are frequentists who maintain that the frequency interpretation is the only viable one. Propensity states a law about a limiting frequency which is yet to be realized, belief expresses the relative frequency of what would happen if a situation presented itself over and over again. For example, when we say that the probability of the Dutch team winning the World Cup is 21 , a frequentist would say that we mean that if this tournament were repeated indeﬁnitely, under identical circumstances, the team would be victorious in 50% of the repeats. However, it is hard to take the latter earnestly, in part because, in my opinion, identical circumstances lead to identical outcomes. I am more sympathetic with Carnap’s admission of two concepts of probability [2], relative frequency on the one hand, and belief on the other. And I am also sympathetic with complaints that it is simply confusing and bad mathematical etiquette to use the same term ‘probability’ for distinct ideas as ‘frequency’ and ‘belief’. An important point that is unfortunately all too often absent or insuﬃciently emphasized in discussions about mathematical models is the issue of open versus closed systems. In open systems, we aim for a law that describes the relation between system variables, but does not go to the point of stating what will actually happen. The interaction with the (unmodeled) environment is explicitly part of the model, but what the environment will be, is not speciﬁed. The paradigmatic example of an open system is an input/output system, which explains the output in terms of the input, but leaves the input as imposed by the environment, unexplained and unspeciﬁed. The paradigmatic example of a closed system is an autonomous system. Systems and control theory, signal processing, and computing science seem to be the only areas which clearly address open systems. Mathematicians and physicists, for example, unfortunately usually end up viewing dynamical systems as closed systems. The very deﬁnition of a random variable or a random process suggests a closed system, and the inﬂuence of the environment is not explicitly taken into account in the mathematical set-up. Compare this, for example, with our notion of a (deterministic) behavior, which clearly accounts for the interaction with the environment. The literature on coin tossing or dice throwing, often viewed as the paradigmatic examples of randomness, suﬀers from a lack of understanding of the fundamental diﬀerence between open and closed systems. All too often, it is suggested that the randomness in a coin toss or a dice throw comes from the coin or the die itself. For example, in the textbook by Cram´er [4, page 154],

412

J.C. Willems

it is stated “The numbers pr [the probability that the die shows outcome r] should in fact be regarded as physical constants of the particular die that we are using [· · · ] experience shows that for a well-made die the frequency of any event r in a long series of throws usually approaches 1/6 [· · · ]”. Obviously, in dice throwing, the source of randomness, if present, comes not from the die, but from the actions of the dice thrower (including the way the die is prepared for a throw). In other words, we are dealing with an open, not a closed system. So, it is absurd to insist that the probability of the outcomes is a property of the die, well-made or not. When adherents of the propensity interpretation state (see Hacking [14, page 14]) “The propensity to give heads is as much a property of the coin as its mass, and the stable long run frequency found on repeated trials is an objective fact of nature independent of anyone’s knowledge of it, or evidence for it”, they are stating something about the environment of the coin. And this something is either untrue or trivially true. If the coin is experimented with in a ﬁxed environment, i.e. if it is tossed repeatedly in exactly the same way, the coin will each time fall on the same side, in accordance with the laws of mechanics and as the experimenters in [6] have demonstrated. Whether it shows always heads or always tails, depends on the experimental set-up – the environment – and on the physical properties of the coin. Some ﬁxed environments will give a run with all heads, some will give a run with all tails. For some experimental set-ups, it is trivial to predict which run will happen, for others, it may be diﬃcult – perhaps impossible, given the present state of knowledge, sensor technology, and computer power – to predict what will happen before the toss, but that we will see a ﬁxed run is certain. However, if the coin is experimented with in an environment that changes from trial to trial, then the coin may sometimes fall on the one side and sometimes on the other, each time in accordance with the laws of mechanics. The relative frequency (including the question of whether the limit exists) depends of how the experimenter changes the environment from trial to trial. There need not be a stable long run frequency found on repeated trials, by properly selecting the trials, any relative frequency can be found, and so it is not an objective fact of nature. If we see a stable long run frequency for the coin, it is only because we choose elements in the diﬀerent trials according to some stable long run frequency, or better, according to some kind of randomized law that generates a stable long run frequency. The fact that needs to be explained, thou, is why and when a little bit of randomness introduced in the trials generates a robust stable long run frequency in the outcomes. It looks reasonable to assume that untrained, uncoordinated individual who enthusiastically throws a well-made die will produce a stable long frequency, but this is more a matter of belief than an objective fact of nature. Or, worse yet, a truism in the sense it is by deﬁnition what is meant by untrained, uncoordinated, enthusiastic throws. The many experiments that have been carried out to prove that coin tossing is random at best serve to prove that the coin transmits, and perhaps ampliﬁes or regularizes, randomness that is external to the coin. So, one may

Thoughts on System Identiﬁcation

413

as well use a random number generator to drive a mechanism that tosses the coin, avoid the conclusion that coin tossing is ‘random’, not ‘physics’, but instead opt for the punchline that a random number generator generates random numbers. It is also instructive to deconstruct Buﬀon’s needle or Bertrand’s ‘paradox’ as originating by trying to explain the outcomes of variables of an open system using the limited language of closed systems. Of course, we get diﬀerent probabilities if we throw a needle using diﬀerent mechanisms. There is nothing paradoxical in this whatsoever. This also applies to stochastic processes. As such, I ﬁnd ARMAX systems (with the exogenous variables left unexplained) a much more convincing model class (e.g. for econometric modeling) than ARMA systems.

9 Stochastics in SYSID In engineering education in general, and in systems and control in particular, the issue of the interpretation of probability is invariably glossed over completely. At best, there is some passing reference to the relative frequency interpretation. The mathematical structure and algorithms invariably take the central place in textbooks. The saving grace for this nonchalance is the fact that the mathematical measure theoretic framework appears to be useable by and large regardless of the interpretation. The extent to which the interpretation of probability is relevant in the use of probabilistic models and methods depends, in my opinion, to a large measure on the ﬁeld of application. In descriptive sciences as physics and economics, it is indispensable to articulate the interpretation used. In mathematics, the interpretation usually does not really matter at all. Deep and useful theorems have been derived about the mathematical theory of probability without worrying about the interpretation. In engineering (and prescriptive aspects of economics) one can, it seems to me, take the following intermediate position. An algorithm-based engineering device, say in signal processing, communication, or control, comes with a set of ‘certiﬁcates’, i.e. statements that guarantee that the device or the algorithm will work well under certain speciﬁed circumstances. These circumstances need not be the ones under which the device will operate in actual practice. They may not even be circumstances which can happen in the real world. These certiﬁcates are merely quality speciﬁcations. Examples of such performance guarantees may be that an error correcting code corrects an encoded message that is received with on the average not more than a certain percentage of errors, or that a ﬁlter generates the conditional expectation of an unobserved signal from an observed one under certain prescribed stochastic assumptions, or that a controller ensures robust stability if the plant is in a certain neighborhood of a nominal one, etc. Which interpretation of probability prevails in SYSID? Of course, in this case the aim is to model reality, and one cannot avoid the interpretation is-

414

J.C. Willems

sue all together. But one can view SYSID as a design of algorithms, and as such, it is reasonable to test a proposed algorithm under certain ‘idealized’ assumptions, i.e. for a ‘test case’ in order to obtain a certiﬁcate, in the spirit of what has been explained in the previous paragraph. For example, one could assume that (u, y, ε) in (E ) is a stationary ergodic gaussian process with u and ε independent and satisfying (E ) for some (P, Q, M ) from a model class, and to examine how the algorithm that chooses an (P, Q, M ) on the basis of w ˜T performs for T → ∞. This, of course, points to a relative frequency interpretation. As a ‘testbed’ this has appealing features: it is well-founded mathematically and algorithmically, and the probabilistic structure will have an ‘averaging eﬀect’ on the data, avoiding excessive inﬂuence of unusual features. As a testbed, there is nothing wrong with this, to the contrary. As long as one avoids the claim that it yields an (approximate) description of the reality that has produced the data. However, it is just as reasonable to test the algorithms for signals (w, ε) for which the correlations like lim

T→∞

1 T

T t =1

σ t w(t + t) σ t ε(t + t)

w(t ) ε(t )

exist for certain t ∈ N. Or demand good SYSID features for certain classes of signals (w, ε) ∈ 2 (N, Rw+e ), thereby avoiding stochastic complexities. The relation of probability with the real world is a serious problem that should be addressed in SYSID, and it is a pity that attempts to put this on the systems theory agenda [17] have fallen on deaf ears. The development of probability as limiting frequency owes much to John Venn, best known to us through the ‘Venn diagrams’ for testing syllogisms. John Venn (1834–1923) was president of Gonville and Caius college, fortuitously the same college which will host our symposium dinner. Venn wrote the ﬁrst book [33] devoted to the frequency interpretation of probability. He complains that ”probability has been very much abandoned to mathematicians, who as mathematicians have generally been unwilling to treat it thoroughly. In the work of mathematicians we would search in vain for anything like a critical discussion of the fundamental principles upon which its rules rest, the class of enquiries to which it is most properly applicable, or the relation it bears to logic and the general rules of inductive evidence”. Little seems to have changed in this respect since these words were written, except that the unwillingness to which John Venn refers has now been extended beyond mathematics. It certainly includes SYSID. None of my favorite books on the subject [15, 18, 21, 29, 30] discusses this issue at all, except [21, page 22], where it is merely stated that ”information about past disturbances could, however, be important for making qualiﬁed guesses about future values. It is therefore natural to employ a probabilistic framework to describe future disturbances”. Surely, an important area of system theory with an ambition to be widely applicable in engineering practice deserves a more balanced justiﬁcation for the mathematical modeling framework in which it chooses to operate.

Thoughts on System Identiﬁcation

415

Acknowledgments This research is supported by the Belgian Federal Government under the DWTC program Interuniversity Attraction Poles, Phase V, 2002–2006, Dynamical Systems and Control: Computation, Identiﬁcation and Modelling, by the KUL Concerted Research Action GOA AMBioRICS, and by several grants the Flemish Fund for Scientiﬁc Research: ICCoS, ANMMM, MLDM. I would also like to thank Ivan Markovsky (K.U. Leuven), Paolo Rapisarda (Un. of Southampton), and Bart De Moor (K.U. Leuven) for their collaboration on many aspects of the problems discussed in this paper, and Gert de Cooman (Ghent Un.) and Manfred Deistler (T.U. Vienna) for remarks concerning the latter sections of this article.

References 1. H. Akaike, Markovian representation of stochastic processes by canonical variables, SIAM Journal on Control, volume 13, pages 162–173, 1975. 2. R. Carnap, The two concepts of probability, Philosophy and Phenomenological Research, volume V, pages 513–532, 1945. 3. R. Carnap, Logical Foundations of Probability , Chicago University Press, 1950. 4. H. Cram´er, Mathematical Methods of Statistics, Princeton University Press, 1946. 5. B. De Finetti, Theory of Probability, Volume 1 and 2, Wiley, 1974. 6. P. Diaconis, S. Holmes and R. Montgomery, Dynamical bias in the coin toss, manuscript, 2004. 7. J. C. Doyle, K. Glover, P. P. Khargonekar, and B. A. Francis, State-space solutions to standard H2 and H∞ control problems, IEEE Transcations on Automatic Control, volume 43, pages 831–847, 1989. 8. M. C. Galavotti, Philosophical Introduction to Probability, CSLI Publications, 2005. 9. K. Glover and J. C. Willems, On the stability of numerical integration routines for ordinary diﬀerential equations, Journal of the IMA, volume 11, pages 171– 180, 1973. 10. K. Glover, Structural Aspects of System Identiﬁcation, Ph.D. dissertation, M.I.T., 1973. 11. K. Glover and J. C. Willems, Parametrizations of linear dynamical systems: Canonical forms and identiﬁability, IEEE Transcations on Automatic Control, volume 19, pages 640–646, 1974. 12. K. Glover, All optimal Hankel-norm approximations of linear multivariable systems and their L∞ error bounds, International Journal of Control,volume 39, pages 1115–1193, 1984. 13. I. J. Good, 46656 varieties of Bayesians, American Statistician, volume 25, pages 62–63, 1971. Reprinted in Good Thinking. The Foundations of Probability and its Applications, University of Minnesota Press, pages 20–21, 1983. 14. I. Hacking, The Emergence of Probability, Cambridge University Press, 1975. 15. E. J. Hannan and M. Deistler, The Statistical Theory of Linear Systems, Academic Press, 1979. 16. C. Heij, Deterministic Identiﬁcation of Dynamical Systems, Lecture Notes in Control and Information Sciences, volume 127, Springer Verlag, 1989.

416

J.C. Willems

17. R. E. Kalman, Randomness reexamined, Modeling, Identiﬁcation and Control, volume 15, pages 141–151, 1994. 18. T. Katayama, Subspace Methods for System Identiﬁcation, Springer Verlag, 2005. 19. J. M. Keynes, A Treatise on Probability, MacMillan, 1963 (originally published in 1921). 20. M. Kuijper, An algorithm for constructing a minimal partial realization in the multivariable case, Systems & Control Letters, volume 31, pages 225–233, 1997. 21. L. Ljung, System Identiﬁcation: Theory for the User, Prentice-Hall, 1999. 22. I. Markovsky, J. C. Willems, S. Van Huﬀel, and B. L. M. De Moor, Exact and Approximate Modeling of Linear Systems: A Behavioral Approach, SIAM Publications, to appear. 23. I. Markovsky, J. C. Willems, and B. L. M. De Moor, Comparison of identiﬁcation algorithms on the database for system identiﬁcation DAISY, Report 05–227, Department of Electrical Engineering, K.U. Leuven, 2005. 24. J. W. Polderman and J. C. Willems, Introduction to Mathematical Systems Theory: A Behavioral Approach, Springer Verlag, 1998. 25. K. R. Popper, The propensity interpretation of probability, British Journal for the Philosophy of Science, volume X, pages 25–42, 1959. 26. K. J. Popper, A World of Propensities, Thoemmes, 1990. 27. P. Rapisarda and J. C. Willems, State maps for linear systems, SIAM Journal on Control and Optimization, 35, pages 1053–1091, 1997. 28. B. Roorda, Algorithms for global least squares modelling of ﬁnite multivariable time series, Automatica, volume 31, pages 391–404, 1995. 29. T. S¨ oderstr¨ om and P. Stoica, System Identiﬁcation, Prentice Hall, 1989. 30. P. Van Overschee and B. L. M. De Moor, Subspace Identiﬁcation for Linear Systems: Theory, Implementation, Applications, Kluwer Academic Press, 1996. 31. R. von Mises, Warscheinlichkeit, Statistik und Warheit, Springer, 1928. 32. J. von Plato, Creating Modern Probability, Cambridge University Press, 1994. 33. J. Venn, The Logic of Chance, MacMillan, 1866. 34. M. Verhaegen, Identiﬁcation of the deterministic part of MIMO state space models given in innovations form from input-output data, Automatica, volume 30, pages 61–74, 1994. 35. J. C. Willems, From time series to linear system — Part I. Finite dimensional linear time invariant systems, Part II. Exact modelling, Part III. Approximate modelling, Automatica, volume 22, pages 561–580 and 675–694, 1986, volume 23, pages 87–115, 1987. 36. J. C. Willems, Fitting data sequences to linear systems, Systems and Control in the Twenty-First Century (Edited by C. I. Byrnes, B. N. Datta, D. S. Gilliam, and C. F. Martin), Birkh¨ auser, pages 405–416, 1996.

Sample Reuse Techniques for Probabilistic Robust Control Xinjia Chen, Jorge L. Aravena and Kemin Zhou Department of Electrical and Computer Engineering Louisiana State University Baton Rouge, LA 70803 {chan, aravena, kemin}@ece.lsu.edu

Sample reuse techniques have signiﬁcantly reduced the numerical complexity of probabilistic robustness analysis. Existing results show that for a nested collection of hyper-spheres the complexity of the problem of performing N equivalent i.i.d. (identical and independent) experiments for each sphere is absolutely bounded, independent of the number of spheres and depending only on the initial and ﬁnal radii. In this chapter we elevate sample reuse to a new level of generality and establish that the numerical complexity of performing N equivalent i.i.d. experiments for a chain of sets is absolutely bounded if the sets are nested. Each set does not even have to be connected, as long as the nested property holds. Thus, for example, the result permits the integration of deterministic and probabilistic analysis to eliminate regions from an uncertainty set and reduce even further the complexity of some problems. With a more general view, the result enables the analysis of complex decision problems mixing real-valued and discrete-valued random variables.

1 Introduction The results presented in this chapter evolved from our previous work in probabilistic robustness analysis. For completeness we give a brief overview of the problem originally considered and show how it is embedded in our present, more general, formulation. Probabilistic robust control methods have been proposed with the goal of overcoming the NP hard complexity and the conservatism associated with the deterministic worst-case framework of robust control (see, [1]– [35] and the references therein). At the heart of the probabilistic control paradigm is the idea of sacriﬁcing the extreme instances of uncertainty. This is in sharp B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 417–429, 2006. © Springer-Verlag Berlin Heidelberg 2006

418

X. Chen, J.L. Aravena, and K. Zhou

contrast to the deterministic robust control which approaches the issue of uncertainty with a “worst case” philosophy. Due to the obvious possibility of violation of robustness requirements associated with the probabilistic method, it has been the common contention that applying the probabilistic method for control design may be more dangerous than using the deterministic worstcase approach. Interestingly, it has been demonstrated (Chen, Aravena and Zhou, [11]) that it is not uncommon for a probabilistic controller (which guarantees only most instances of the uncertainty bounding set assumed in the design) to be signiﬁcantly less risky than a deterministic worst-case controller. The reasons are the “uncertainty in modeling uncertainties” and the fact that the worst-case design cannot, in some instances, be “all encompassing.” Although this philosophy is proposed in the context of robust design, a direct consequence on robustness analysis is that it is not necessary to evaluate the system robustness in a deterministic worst-case framework. This is because a system certiﬁed to be robust in a deterministic worst-case framework is not necessarily less risky than a system with a probability that the robustness requirement is not always satisﬁed. While the worst-case control theory uses the deterministic robustness margin to evaluate the system robustness, probabilistic control theory introduced the robustness function as a tool to measure the robustness properties of a control system subject to uncertainties. Such function is deﬁned as P(r) = vol({X ∈ Br | P is guaranteed for X })/vol(Br ) where vol(.) is the Lebesgue measure, P denotes the robustness requirement, and Br denotes the uncertainty bounding set with radius r. This function describes quantitatively the relationship between the proportion of systems guaranteeing the robustness requirement and the radius of uncertainty set. Such a function has been proposed by a number of researchers. For example, Barmish and Lagoa [3] have constructed a curve of robustness margin ampliﬁcation versus risk in a probabilistic setting. In a similar spirit, Calaﬁore, Dabbene and Tempo [5] have constructed a probability degradation function in the context of real and complex parametric uncertainty. The so-called robustness function can serve as a guide for control engineers in evaluating the robustness of a control system once a controller design is completed. In addition to overcome the issues of conservatism and NP complexity of the worst-case robustness analysis, the probabilistic robustness analysis based on the robustness function has the following advantages. First, the robustness function can address problems which are intractable by deterministic worst-case methods. For many real world control problems, robust performance is more appropriately captured by multiple objectives such as stability, transient response (speciﬁed, for example, in terms of overshoot, rise time and settling time), disturbance rejection measured by H∞ or

Sample Reuse Techniques for Probabilistic Robust Control

419

H2 norm, etc. Thus, for a more insightful analysis of the robust performance of uncertain systems, the robustness requirement is usually multi-objective. The complexity of such robustness requirement can easily make the robustness problems intractable by the deterministic worst-case methods. For example, existing methods fail to solve robustness analysis problems when the robustness requirement is a combination of H∞ norm bound and stability. However, the robustness curve can still be constructed and provides suﬃcient insights on the robustness of the system. Second, the probability that the robustness requirement is guaranteed can be inferred from the robustness function, while the deterministic margin has no relationship to such probability. Based on the assumption that the density function of uncertainty is radially symmetric and non-increasing with respect to the norm of uncertainty, it has been shown in [2] that the probability that P is guaranteed is no less than inf ρ∈(0,r] P(ρ) when the uncertainty is contained in a bounding set with radius r. The underlying assumption is in agreement with conventional modeling and manufacturing practices that consider uncertainty as unstructured, with all directions equally likely, and make small perturbations more likely than large perturbations. It was discovered in [2] that the robustness function is not monotonically decreasing. Hence, the lower bound of the probability depends on P(ρ) for all ρ ∈ (0, r]. At the ﬁrst glance, it may seem diﬃcult or infeasible to estimate inf ρ∈(0,r] P(ρ) since the estimation of P(ρ) for every ρ relies on the Monte Carlo simulation. For such probabilistic method to overcome the NP hard of worst-case methods, it is necessary to show that the complexity for estimating inf ρ∈(0,r] P(ρ) for a given r is polynomial in terms of computer running time and memory space. Recently, sample reuse techniques have been developed in [10, 12] and it is demonstrated that the complexity in terms of space and time is surprisingly low and is linear in the uncertainty dimension and the logarithm of the relative width of the range of uncertainty radius. Third, using the robustness function for the evaluation of the system robustness allows the designer to make more accurate statements than using just the robustness margins. Here, by robustness margins, we mean both the deterministic robustness margin and its risk-adjusted version – the probabilistic robustness margin, deﬁned as ρε = sup{r | P(r) ≥ 1 − ε}. For virtually all practical systems, the deterministic robustness margin can be viewed as a special case of the probabilistic robustness margin ρε with ε = 0. This property should not be confused with the numerical accuracy in evaluating margins nor with the issue of conservatism. The fundamental reason is the lack of information that can be available from the robustness margins. It has been demonstrated in [12] that both the deterministic and probabilistic robustness margins have inherent limitations. In other words, using ρε as a measure of robustness can be misleading. Figure 1 shows the conceptual robustness functions for two controllers. From the ﬁgure it is apparent that the robustness

420

X. Chen, J.L. Aravena, and K. Zhou

margin with ε ∈ [0, 0.005], ρA ε , for controller A is much larger than the corresponding value, ρB , for controller B. Then, based on the comparison of ρε , ε control system A is certainly more robust and should be recommended for safety purpose. However, if the coverage probability of the uncertainty set BρA is low and the robustness curve (i.e, the graphical representation of the ε robustness function) of control system A rolls oﬀ rapidly beyond ρA ε , then the robustness of system A may be poor. On the other hand, if the robustness curve of control system B maintains a high level for a wide range of uncertainty radius, then control system B may be actually more robust than system A.

1

Controller A Controller B

0.95

Proportion

0.9

0.85

0.8

0.75

0.7 45

50

55 60 Uncertainty Radius

65

70

Fig. 1. Comparison of Controller Alternatives

In general, the evaluation of the robustness function requires extensive Monte Carlo simulations. In applications, many iterations of robust design and analysis may be needed in the development of a satisfactory control system, it is therefore crucial to improve the eﬃciency of estimating the robustness function. Complexity has been reduced by considering models for the uncertainties that depend on a single “uncertainty radius.” In this case, the formal evaluation of the robustness function requires N , i.i.d. uncertain parameter

Sample Reuse Techniques for Probabilistic Robust Control

421

selections for each of a sequence r1 < r2 < · · · < rm of uncertainty radii, which is still a daunting task. The sample reuse principle allows carrying the evaluation to any degree of accuracy and with absolute bounds in complexity (see, [10, 12]). The use of uncertainty bounding sets with a given radius can still be viewed as a limitation since one may have to include situations that never arise in practice. This is the limitation addressed in this work. Moreover, we cast to result as a general problem in decision-making under uncertainties. We show that the sample reuse principle can be applied with equal eﬀectiveness in a much more general scenario. We shall be concerned with an arbitrary sequence of nested sets B1 ⊂ B2 ⊂ · · · ⊂ Bm where we need to perform N experiments for elements uniformly and independently drawn from each set. For each element it is necessary to verify if a certain statement P is true or not. The idea of the sample reuse principle is to start experiments from the largest set and if it also belongs to smaller subset the experimental result is saved for later use in the smaller set. The experimental result that can be saved includes not only the samples from the sets but also the outcome of the evaluation of the statement P. We note that this formulation enables the eﬃcient use of Monte Carlo simulations for the evaluation of multi-dimensional distributions and the combination of continuous and discrete variables.

2 Absolute Bound of Complexity Consider a sequence of nested sets B1 ⊂ B2 ⊂ · · · ⊂ Bm . If one needs to perform N experiments from each set, a conventional approach would require a total of N m experiments. However, due to sample reuse, the actual number of experiments for set Bi is a random number ni , which is usually less than N . Our main result, which depends only on the nested property, shows that this strategy saves a signiﬁcant amount of experimental or computational eﬀort. Theorem 1. Let Vmin and Vmax be constants such that 0 < Vmin ≤ Vmax < ∞. For an arbitrary sequence of nested sets Bi , i = 1, · · · , m such that B1 ⊂ B2 ⊂ · · · ⊂ Bm and Vmin ≤ vol(B1 ) ≤ vol(Bm ) ≤ Vmax , the expected total number of experiments, n, to obtain N experiments for each set is absolutely bounded, independent of the number, m, of sets in the chain and given by E [n] <

1 + ln

Vmax Vmin

N

where E[.] denotes the expectation of a random variable.

422

X. Chen, J.L. Aravena, and K. Zhou

Remark 1. The fact that the result is independent of the number of sets in the nested chain may appear surprising but it is a direct consequence of the power of the sample reuse principle. Loosely speaking, the more sets are there in the chain, the more chances that an experiment can be reused. In fact this characteristic makes the result especially powerful when the demands for accuracy, indicated by a large number of sets, is high. As a special case of Theorem 1, we have the following result, reported by Chen, Zhou, Aravena [12] and presented here now as a corollary to our main result. Corollary 1. Let rmin and rmax be constants such that 0 < rmin ≤ rmax < ∞. Let Br denote the uncertainty bounding sets with radius r. Suppose that vol(Br ) = rd vol(B1 ) for any radius r. Then, for any sequence of radius r1 < r2 < · · · < rm such that rmin ≤ r1 < rm ≤ rmax , E [n] <

1 + d ln

rmax rmin

N.

2.1 Observations about the Result In the result presented here, the only requirement for the uncertainty sets is that they must be nested. This is in sharp contrast to the existing model of uncertainty wherein we deﬁne an uncertainty “radius” and larger uncertainty sets are simply ampliﬁed versions of the smaller sets, deﬁning a chain of sets of essentially the same shape. Such limitation is completely eliminated now. Another signiﬁcant feature of the new result is that the uncertainty sets can have “holes” in them; i.e., one can easily eliminate situations, or values, that cannot physically take place. In a later section we examine this option in more detail and show the advantage provided by the general result. In fact, as long as the sets are nested, the sets don’t even have to be connected. This permits modeling of situations that were not feasible, for example, combination of discrete and continuous-valued random variables. Finally, the power of the result lies in the eﬃcient use of experiments. The property that is being tested is not germane to the result. In this sense, we have provided a tool for decision making in complex environments.

3 Proof of Main Theorem This section provides a formal proof of our main result. First we establish some preliminary results that will be needed in the proof.

Sample Reuse Techniques for Probabilistic Robust Control

423

Lemma 1. For i = 2, . . . , m, m

E [ni−1 ] = N − j=i

vi−1 vj

E [nj ]

where vj = vol(Bj ), j = 1, · · · , m. Proof. Let m ≥ j ≥ i ≥ 2. Let q 1 , q 2 , . . . , q nj be the samples generated from Bj . For = 1, . . . , nj , deﬁne random variable Xj,i−1 such that 1 0

def

Xj,i−1 =

if q fall in Bi−1 , otherwise.

Based on the principle of sample reuse, we have nk

m

nm = N,

Xk,j ,

nj = N −

j = 1, · · · , m − 1,

k=j+1 =1

which implies that the value of nj depends only on the samples generated from sets Bk , j + 1 ≤ k ≤ m. Hence, event {nj = n} is independent of event {Xj,i−1 = 1}. It follows that Pr Xj,i−1 = 1, nj = n = Pr Xj,i−1 = 1 Pr {nj = n} where Pr{.} denotes the probability of an event. Recall that q is a random variable with uniform distribution over Bj , we have Pr Xj,i−1 = 1 =

vi−1 , vj

= 1, · · · , N.

By the principle of sample reuse, m

nj

N = ni−1 +

Xj,i−1 . j=i =1

Thus for i = 2, . . . , m, nj

m

E

E [ni−1 ] = N − j=i m

Xj,i−1 =1

N

n

=N−

Pr Xj,i−1 = 1, nj = n j=i n=1 =1

424

X. Chen, J.L. Aravena, and K. Zhou m

N

n

=N−

Pr Xj,i−1 = 1 Pr{nj = n} j=i n=1 =1 m

N

vi−1 vj

n

=N− j=i n=1 m

N

vi−1 vj

=N− j=i m

n Pr{nj = n} n=1

vi−1 vj

=N− j=i

Pr{nj = n}

E [nj ] .

✷ This result gives the expected number of experiments for a set, Bi−1 , in terms of the expected values for all the sets that contain it. The recursion can be solved as follows: Since all the experiments must belong to the set Bm we have E[nm ] = N , now for i < m we can write m

E [ni ] = N − j=i+1

m

vi vj

E [nj ]

vi vj

=⇒ j=i+1

E [nj ] = N − E [ni ]

and m

E [ni−1 ] = N − j=i

=N− =N−

vi−1 vj

E [nj ] m

vi−1 vi

E [ni ] −

vi−1 vi

E [ni ] −

vi−1 vj

j=i+1

vi−1 vi

E [nj ]

m j=i+1

vi vj

E [nj ] .

Therefore, E [ni−1 ] = N − =N−

vi−1 vi vi−1 vi

E [ni ] −

vi−1 vi

[N − E [ni ]]

N.

Thus we have established Lemma 2. Under the sample reuse principle, for an arbitrary sequence of nested sets Bi , i = 1, · · · , m such that B1 ⊂ B2 ⊂ · · · ⊂ Bm and 0 < vol(B1 ) ≤ vol(Bm ) < ∞, the expected total number of experiments, E [ni ], to obtain N experiments for the set Bi is vi N ; i = 1, 2, . . . , m − 1. E(ni ) = N − vi+1

Sample Reuse Techniques for Probabilistic Robust Control

425

Remark 2. We note that if we use the convention vm+1 = ∞ then the previous expression can be made valid for i = m. Once more one can see the power of the sample reuse principle. If any two sets in the chain are “very similar,” then most of the experiments for the larger set can be reused. Now we establish a basic inequality that will be used to prove the main result. Lemma 3. For any x > 1,

1 + ln x > 1. x

Proof. Let f (x) =

1 + ln x. x

Then f (1) = 1 and d f (x) x−1 = > 0, dx x2

∀x > 1.

It follows that f (x) > 1, ∀x > 1.

✷

Using the previous result now we can prove Lemma 4. For an arbitrary sequence of numbers 0 < r1 < r2 < · · · < rm , m−1

m− i=1

ri

< 1 + ln

ri+1

rm r1

.

Proof. Observing that rm = r1 we have ln

rm r1

m−1

ri+1 , ri

i=1 m−1

=

ln i=1

ri+1 ri

.

Therefore, m−1 i=1

Since

ri+1 ri

ri ri+1

+ ln

rm r1

m−1

= i=1

1

ri+1 ri

+ ln

ri+1 ri

.

> 1, i = 1, · · · , m − 1, it follows from Lemma 3 that 1

ri+1 ri

+ ln

ri+1 ri

> 1,

i = 1, · · · , m − 1.

426

X. Chen, J.L. Aravena, and K. Zhou

Hence,

m−1 i=1

ri + ln ri+1

rm r1

> m − 1.

The lemma is thus proved.

✷

Now we are in the position to prove Theorem 1. By Lemma 2, we have m

E[n] = E

ni i=1 m−1 N −N

=N+ i=1

m−1

= Nm − N i=1

vi vi+1

vi . vi+1

Therefore, by Lemma 4, E

n =m− N

m−1 i=1

vi < 1 + ln vi+1

vm v1

≤

1 + ln

Vmax Vmin

and thus the proof of Theorem 1 is completed.

4 Combination with Deterministic Methods In this section we demonstrate the ﬂexibility allowed by the general nested conditions by examining a situation that could not be properly handled with existing tools. Especially, we consider uncertainty sets where, for example by deterministic analysis, one can establish subsets that are not feasible; i.e., the uncertainty set has “holes” in it. There exist rich results for computing the exact or conservative bounds of the robustness margins, e.g., structure singular value µ theory or Kharitonov type methods. Let Sr be a hyper-sphere with radius r. Suppose the robustness requirement is satisﬁed for the nominal system. By the deterministic approach, in some situations, it may be possible to determine r0 such that the robustness requirement is satisﬁed for Sr0 . Then, to estimate P(r) =

vol({q ∈ Sr | P is guaranteed for q}) vol(Sr )

for r1 < r2 < · · · < rm with r1 > r0 , we can apply the sample reuse techniques over a nested chain of “donut” sets D1 ⊂ D2 ⊂ · · · ⊂ Dm with

Sample Reuse Techniques for Probabilistic Robust Control

D i = Sr i \ Sr 0 ,

427

i = 1, · · · , m

where “\” denotes the operation of set minus. Instead of directly estimate P(ri ), we can estimate ℘i =

vol({q ∈ Di | P is guaranteed for q}) vol(Di )

and obtain P(ri ) =

℘i vol(Di ) + vol(Sr0 ) , vol(Sri )

i = 1, · · · , m.

Let ℘i be the estimate of ℘i . It can be shown that E and E

℘i vol(Di ) + vol(Sr0 ) = P(ri ) vol(Sri )

℘i vol(Di ) + vol(Sr0 ) − P(ri ) vol(Sri )

where λi =

vol(Di ) , vol(Sri )

2

=

(1 − ℘i )℘i λ2i N

i = 1, · · · , m.

If we obtain an estimate P(ri ) of P(ri ) without applying any deterministic technique, then E[P(ri ) − P(ri )]2 =

(1 − ℘i )λi [1 − (1 − ℘i )λi ] . N

It can be shown that, the ratio of variance of the two estimate is E

℘ bi vol(Di )+vol(Sr0 ) vol(Sri )

− P(ri )

E[P(ri ) − P(ri

)]2

2

=

℘i λi < 1. 1 − (1 − ℘i )λi

This implies that, for the same sample size N , the estimation can be more accurate when combining the deterministic results and the probabilistic techniques. Since the accuracy is exchangeable with the computational eﬀort, we can conclude that the computational eﬀort can be reduced by blending the power of deterministic methods and randomized algorithms with the sample reuse mechanism.

5 Conclusions Sample reuse has made possible the evaluation of robustness functions with, essentially, arbitrary accuracy and bounded complexity. In this work we have

428

X. Chen, J.L. Aravena, and K. Zhou

expanded the power of the sample reuse concept and shown that it can be applied to the evaluation of complex decision problem with the only requirement that the uncertainty sets be nested. We have demonstrated the power of the generalization by integrating deterministic analysis and randomized algorithms and showing that one can develop even more eﬃcient computational approaches for the evaluation of robustness functions.

References 1. Bai EW, Tempo R, Fu M (1998), “Worst-case properties of the uniform distribution and randomized algorithms for robustness analysis,” Mathematics of Control, Signals and Systems 11:183–196 2. Barmish BR, Lagoa CM, Tempo R (1997), “Radially truncated uniform distributions for probabilistic robustness of control systems,” Proceedings of American Control Conference 853–857 3. Barmish BR, Lagoa CM (1997), “The uniform distribution: a rigorous justiﬁcation for its use in robustness analysis,” Mathematics of Control, Signals and Systems 10:203–222 4. Barmish BR, Shcherbakov PS (2002), “On avoiding vertexization of robustness problems: The approximate feasibility concept,” IEEE Trans on Auto Control 42:819–824 5. Calaﬁore G, Dabbene F, Tempo R (2000), “Randomized algorithms for probabilistic robustness with real and complex structured uncertainty,” IEEE Trans on Auto Control 45:2218–2235 6. Calaﬁore G, Campi MC (2005), “Uncertain convex programs: randomized solutions and conﬁdence levels,” Mathematical Programming 102:25–46 7. Calaﬁore G, Dabbene F (2002), “A probabilistic framework for problems with real structured uncertainty in systems and control,” Automatica 38:1265–1276 8. Chen X, Zhou K (1998), “Order statistics and probabilistic robust control,” System and Control Letters 35:175–182 9. Chen X, Zhou K (2000), “Constrained robustness analysis and synthesis by randomized algorithms,” IEEE Trans on Auto Control 45:1180–1186 10. Chen X, Zhou K, Aravena J (2004), “Fast construction of robustness degradation function,” SIAM Journal on Control and Optimization 42:1960–1971 11. Chen X, Aravena J, Zhou K (2005), “Risk analysis in robust control — Making the case for probabilistic robust control, ” Proceedings of American Control Conference 1533–1538 12. Chen X, Zhou K, Aravena J (2005), “Probabilistic robustness analysis — risks, complexity and algorithms,” submitted for publication. 13. Fujisaki Y, Dabbene F, Tempo R (2003), “Probabilistic robust design of LPV control systems,” Automatica 39:1323–1337 14. Fujisaki Y, Kozawa Y (2003), “Probabilistic Rrobust controller design: probable near minimax value and randomized algorithms,” Proceedings of IEEE Conference on Decision and Control 1938–1943 15. Hokayem PF, Abdallah CT (2003), “Quasi-Monte Carlo methods in robust control design,” Proceedings of IEEE Conference on Decision and Control 2435– 2440

Sample Reuse Techniques for Probabilistic Robust Control

429

16. Kanev S, Schutter BD, Verhaegen M (2003), “An ellipsoid algorithm for probabilistic robust controller design,” Systems and Control Letters 49:365–375 17. Kanev S, Verhaegen M (2003), “Robust output-feedback integral MPC: A probabilistic approach,” Proceedings of IEEE Conference on Decision and Control 1914–1919 18. Khargonekar PP, Tikku A (1996), “Randomized algorithms for robust control analysis and synthesis have polynomial complexity,” Proceedings of IEEE Conference on Decision and Control 3470–3475 19. Koltchinskii V, Abdallah CT, Ariola M, Dorato P, Panchenko D (2000), “Improved sample complexity estimates for statistical learning control of uncertain systems,” IEEE Trans on Auto Control 46:2383–2388 20. Lagoa CM (2000), “Probabilistic enhancement of classic robustness margins: A class of none symmetric distributions,” Proceedings of American Control Conference 3802–3806 21. Lagoa CM, Li X, Sznaier M, “On the design of robust controllers for arbitrary uncertainty structures,” to appear in IEEE Trans on Auto Control 22. Lagoa CM, Li X, Mazzaro MC, Sznaier M (2003), “Sampling random transfer functions,” Proceedings of IEEE Conference on Decision and Control 2429–2434 23. Marrison C, Stengel RF (1997), “Robust control system design using random search and genetic algorithms,” IEEE Trans on Auto Control 42:835–839 24. Oishi Y, Kimura H (2001), “Randomized algorithms to solve parameterdependent linear matrix inequalities and their computational complexity,” Proceedings of IEEE Conference on Decision and Control 2025–2030 25. Oishi Y (2003), “Probabilistic design of a robust state-feedback controller based on parameter-dependent Lyapunov functions,” Proceedings of IEEE Conference on Decision and Control 1920–1925 26. Polyak BT, Tempo R (2001), “Probabilistic robust design with linear quadratic regulators,” Systems and Control Letters 43:343–353 27. Polyak BT, Shcherbakov PS (2000), “Random spherical uncertainty in estimation and robustness,” IEEE Trans on Auto Control 45:2145–2150 28. Ray LR, Stengel RF (1993), “A Monte Carlo approach to the analysis of control systems robustness,” Automatica 3:229–236 29. Ross SR, Barmish BR (2001), “Distributionally robust gain analysis for systems containing complexity,” Proceedings of IEEE Conference on Decision and Control 5020–5025 30. Stengel RF, Ray LR (1991), “Stochastic robustness of linear time-invariant systems,” IEEE Trans on Auto Control 36:82–87 31. Tempo R, Calaﬁore G, Dabbene F (2004) Randomized Algorithms for Analysis and Control of Uncertain Systems. Springer-Verlag, New York 32. Tempo R, Bai EW, Dabbene F (1997), “Probabilistic robustness analysis: explicit bounds for the minimum number of samples,” Systems and Control Letters 30:237–242 33. Vidyasagar M, Blondel VD (2001), “Probabilistic solutions to NP-hard matrix problems,” Automatica 37:1597–1405 34. Vidyasagar M (2001), “Randomized algorithms for robust controller synthesis using statistical learning theory,” Automatica 37:1515–1528 35. Wang Q, Stengel RF (2002), “Robust control of nonlinear systems with parametric uncertainty,” Automatica 38:1591–1599

Lecture Notes in Control and Information Sciences Edited by M. Thoma and M. Morari Further volumes of this series can be found on our homepage: springer.com

Vol. 328: Lor a, A.; Lamnabhi-Lagarrigue, F.; Panteley, E. (Eds.) Advanced Topics in Control Systems Theory 305 p. 2006 [1-84628-313-2] Vol. 327: Fournier, J.-D.; Grimm, J.; Leblond, J.; Partington, J.R. (Eds.) Harmonic Analysis and Rational Approximation 301 p. 2006 [3-540-30922-5] Vol. 326: Wang, H.-S.; Yung, C.-F.; Chang, F.-R. H∞ Control for Nonlinear Descriptor Systems 164 p. 2006 [1-84628-289-6] Vol. 325: Amato, F. Robust Control of Linear Systems Subject to Uncertain Time-Varying Parameters 180 p. 2006 [3-540-23950-2] Vol. 324: Christoˇdes, P.; El-Farra, N. Control of Nonlinear and Hybrid Process Systems 446 p. 2005 [3-540-28456-7] Vol. 323: Bandyopadhyay, B.; Janardhanan, S. Discrete-time Sliding Mode Control 147 p. 2005 [3-540-28140-1] Vol. 322: Meurer, T.; Graichen, K.; Gilles, E.D. (Eds.) Control and Observer Design for Nonlinear Finite and Inˇnite Dimensional Systems 422 p. 2005 [3-540-27938-5] Vol. 321: Dayawansa, W.P.; Lindquist, A.; Zhou, Y. (Eds.) New Directions and Applications in Control Theory 400 p. 2005 [3-540-23953-7] Vol. 320: Steffen, T. Control Reconˇguration of Dynamical Systems 290 p. 2005 [3-540-25730-6] Vol. 319: Hofbaur, M.W. Hybrid Estimation of Complex Systems 148 p. 2005 [3-540-25727-6] Vol. 318: Gershon, E.; Shaked, U.; Yaesh, I. H∞ Control and Estimation of State-muliplicative Linear Systems 256 p. 2005 [1-85233-997-7] Vol. 317: Ma, C.; Wonham, M. Nonblocking Supervisory Control of State Tree Structures 208 p. 2005 [3-540-25069-7]

Vol. 316: Patel, R.V.; Shadpey, F. Control of Redundant Robot Manipulators 224 p. 2005 [3-540-25071-9] Vol. 315: Herbordt, W. Sound Capture for Human/Machine Interfaces: Practical Aspects of Microphone Array Signal Processing 286 p. 2005 [3-540-23954-5] Vol. 314: Gil', M.I. Explicit Stability Conditions for Continuous Systems 193 p. 2005 [3-540-23984-7] Vol. 313: Li, Z.; Soh, Y.; Wen, C. Switched and Impulsive Systems 277 p. 2005 [3-540-23952-9] Vol. 312: Henrion, D.; Garulli, A. (Eds.) Positive Polynomials in Control 313 p. 2005 [3-540-23948-0] Vol. 311: Lamnabhi-Lagarrigue, F.; Lor a, A.; Panteley, E. (Eds.) Advanced Topics in Control Systems Theory 294 p. 2005 [1-85233-923-3] Vol. 310: Janczak, A. Identiˇcation of Nonlinear Systems Using Neural Networks and Polynomial Models 197 p. 2005 [3-540-23185-4] Vol. 309: Kumar, V.; Leonard, N.; Morse, A.S. (Eds.) Cooperative Control 301 p. 2005 [3-540-22861-6] Vol. 308: Tarbouriech, S.; Abdallah, C.T.; Chiasson, J. (Eds.) Advances in Communication Control Networks 358 p. 2005 [3-540-22819-5]

Vol. 307: Kwon, S.J.; Chung, W.K. Perturbation Compensator based Robust Tracking Control and State Estimation of Mechanical Systems 158 p. 2004 [3-540-22077-1] Vol. 306: Bien, Z.Z.; Stefanov, D. (Eds.) Advances in Rehabilitation 472 p. 2004 [3-540-21986-2] Vol. 305: Nebylov, A. Ensuring Control Accuracy 256 p. 2004 [3-540-21876-9] Vol. 304: Margaris, N.I. Theory of the Non-linear Analog Phase Locked Loop 303 p. 2004 [3-540-21339-2]

Vol. 303: Mahmoud, M.S. Resilient Control of Uncertain Dynamical Systems 278 p. 2004 [3-540-21351-1]

Vol. 287: Mahmoud, M.M.; Jiang, J.; Zhang, Y. Active Fault Tolerant Control Systems 239 p. 2003 [3-540-00318-5]

Vol. 302: Filatov, N.M.; Unbehauen, H. Adaptive Dual Control: Theory and Applications 237 p. 2004 [3-540-21373-2]

Vol. 286: Rantzer, A. and Byrnes C.I. (Eds) Directions in Mathematical Systems Theory and Optimization 399 p. 2003 [3-540-00065-8]

Vol. 301: de Queiroz, M.; Malisoff, M.; Wolenski, P. (Eds.) Optimal Control, Stabilization and Nonsmooth Analysis 373 p. 2004 [3-540-21330-9] Vol. 300: Nakamura, M.; Goto, S.; Kyura, N.; Zhang, T. Mechatronic Servo System Control Problems in Industries and their Theoretical Solutions 212 p. 2004 [3-540-21096-2] Vol. 299: Tarn, T.-J.; Chen, S.-B.; Zhou, C. (Eds.) Robotic Welding, Intelligence and Automation 214 p. 2004 [3-540-20804-6] Vol. 298: Choi, Y.; Chung, W.K. PID Trajectory Tracking Control for Mechanical Systems 127 p. 2004 [3-540-20567-5] Vol. 297: Damm, T. Rational Matrix Equations in Stochastic Control 219 p. 2004 [3-540-20516-0] Vol. 296: Matsuo, T.; Hasegawa, Y. Realization Theory of Discrete-Time Dynamical Systems 235 p. 2003 [3-540-40675-1] Vol. 295: Kang, W.; Xiao, M.; Borges, C. (Eds) New Trends in Nonlinear Dynamics and Control, and their Applications 365 p. 2003 [3-540-10474-0] Vol. 294: Benvenuti, L.; De Santis, A.; Farina, L. (Eds) Positive Systems: Theory and Applications (POSTA 2003) 414 p. 2003 [3-540-40342-6] Vol. 293: Chen, G. and Hill, D.J. Bifurcation Control 320 p. 2003 [3-540-40341-8] Vol. 292: Chen, G. and Yu, X. Chaos Control 380 p. 2003 [3-540-40405-8] Vol. 291: Xu, J.-X. and Tan, Y. Linear and Nonlinear Iterative Learning Control 189 p. 2003 [3-540-40173-3] Vol. 290: Borrelli, F. Constrained Optimal Control of Linear and Hybrid Systems 237 p. 2003 [3-540-00257-X] Vol. 289: Giarre, L. and Bamieh, B. Multidisciplinary Research in Control 237 p. 2003 [3-540-00917-5] Vol. 288: Taware, A. and Tao, G. Control of Sandwich Nonlinear Systems 393 p. 2003 [3-540-44115-8]

Vol. 285: Wang, Q.-G. Decoupling Control 373 p. 2003 [3-540-44128-X] Vol. 284: Johansson, M. Piecewise Linear Control Systems 216 p. 2003 [3-540-44124-7] Vol. 283: Fielding, Ch. et al. (Eds) Advanced Techniques for Clearance of Flight Control Laws 480 p. 2003 [3-540-44054-2] Vol. 282: Schroder, J. Modelling, State Observation and Diagnosis of Quantised Systems 368 p. 2003 [3-540-44075-5] Vol. 281: Zinober A.; Owens D. (Eds) Nonlinear and Adaptive Control 416 p. 2002 [3-540-43240-X] Vol. 280: Pasik-Duncan, B. (Ed) Stochastic Theory and Control 564 p. 2002 [3-540-43777-0] Vol. 279: Engell, S.; Frehse, G.; Schnieder, E. (Eds) Modelling, Analysis, and Design of Hybrid Systems 516 p. 2002 [3-540-43812-2] Vol. 278: Chunling D. and Lihua X. (Eds) H∞ Control and Filtering of Two-dimensional Systems 161 p. 2002 [3-540-43329-5] Vol. 277: Sasane, A. Hankel Norm Approximation for Inˇnite-Dimensional Systems 150 p. 2002 [3-540-43327-9] Vol. 276: Bubnicki, Z. Uncertain Logics, Variables and Systems 142 p. 2002 [3-540-43235-3] Vol. 275: Ishii, H.; Francis, B.A. Limited Data Rate in Control Systems with Networks 171 p. 2002 [3-540-43237-X] Vol. 274: Yu, X.; Xu, J.-X. (Eds) Variable Structure Systems: Towards the 21st Century 420 p. 2002 [3-540-42965-4] Vol. 273: Colonius, F.; Grune, L. (Eds) Dynamics, Bifurcations, and Control 312 p. 2002 [3-540-42560-9]

Multiscale, Nonlinear and Adaptive Approximation: Dedicated to Wolfgang Dahmen on the Occasion of his 60th Birthday

Nonlinear analysis : stability, approximation, and inequalities ; In honor of Themistocles M. Rassias on the occasion of his 60th birthday

Frontiers of computational fluid dynamics 2006: [papers dedicated to Professor David A. Caughey on the occasion of his 60th birthday]

Scaling and disordered systems: international workshop and collection of articles honoring Professor Antonio Coniglio on the occasion of his 60th birthday

Control of uncertain systems--modelling, approximation, and design: a workshop on the occasion of Keith Glover's 60th birthday

Multiscale, Nonlinear and Adaptive Approximation: Dedicated to Wolfgang Dahmen on the Occasion of his 60th Birthday

Perspectives in Analysis, Geometry, and Topology: On the Occasion of the 60th Birthday of Oleg Viro

Perspectives in Analysis, Geometry, and Topology: On the Occasion of the 60th Birthday of Oleg Viro

Nonlinear analysis : stability, approximation, and inequalities ; In honor of Themistocles M. Rassias on the occasion of his 60th birthday

A. A. Lyapunov (on his 60th birthday)

Groups of Diffeomorphisms: In Honor of Shigeyuki Morita on the Occasion of His 60th Birthday

Fluctuating Paths and Fields: Festschrift Dedicated to Hagen Kleinert on the Occasion of His 60th Birthday

Computation, Cooperation, and Life: Essays Dedicated to Gheorghe Păun on the Occasion of His 60th Birthday

Topics in Discrete Mathematics: Dedicated to Jarik Nesetril on the Occasion of his 60th birthday

Topics in discrete mathematics: dedicated to Jarik Nešetřil on the occasion of his 60th birthday

Frontiers of computational fluid dynamics 2006: [papers dedicated to Professor David A. Caughey on the occasion of his 60th birthday]

The Design of Approximation Algorithms

The Design of Approximation Algorithms

The design of approximation algorithms

The design of approximation algorithms

The Design of Approximation Algorithms

The design of approximation algorithms

The Design of Approximation Algorithms

The Design of Approximation Algorithms

The Design of Approximation Algorithms

Scaling and disordered systems: international workshop and collection of articles honoring Professor Antonio Coniglio on the occasion of his 60th birthday

Multiscale, nonlinear and adaptive approximation: To Wolfgang Dahmen on 60th birthday

Models and Algorithms for Global Optimization: Essays Dedicated to Antanas Zilinskas on the Occasion of His 60th Birthday

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Robust Adaptive Control of Uncertain Nonlinear Systems

The Composition of Matter: Symposium Honouring Johannes Geiss on the Occasion of His 80th Birthday

Control of human behavior, mental processes, and consciousness: essays in honor of the 60th birthday of August Flammer

Control of uncertain systems--modelling, approximation, and design: a workshop on the occasion of Keith Glover's 60th birthday

Multiscale, Nonlinear and Adaptive Approximation: Dedicated to Wolfgang Dahmen on the Occasion of his 60th Birthday

Perspectives in Analysis, Geometry, and Topology: On the Occasion of the 60th Birthday of Oleg Viro

Perspectives in Analysis, Geometry, and Topology: On the Occasion of the 60th Birthday of Oleg Viro

Nonlinear analysis : stability, approximation, and inequalities ; In honor of Themistocles M. Rassias on the occasion of his 60th birthday

A. A. Lyapunov (on his 60th birthday)

Groups of Diffeomorphisms: In Honor of Shigeyuki Morita on the Occasion of His 60th Birthday

Fluctuating Paths and Fields: Festschrift Dedicated to Hagen Kleinert on the Occasion of His 60th Birthday

Computation, Cooperation, and Life: Essays Dedicated to Gheorghe Păun on the Occasion of His 60th Birthday

Topics in Discrete Mathematics: Dedicated to Jarik Nesetril on the Occasion of his 60th birthday

Topics in discrete mathematics: dedicated to Jarik Nešetřil on the occasion of his 60th birthday

Frontiers of computational fluid dynamics 2006: [papers dedicated to Professor David A. Caughey on the occasion of his 60th birthday]

The Design of Approximation Algorithms

The Design of Approximation Algorithms

The design of approximation algorithms

The design of approximation algorithms

The Design of Approximation Algorithms

The design of approximation algorithms

The Design of Approximation Algorithms

The Design of Approximation Algorithms

The Design of Approximation Algorithms

Scaling and disordered systems: international workshop and collection of articles honoring Professor Antonio Coniglio on the occasion of his 60th birthday

Multiscale, nonlinear and adaptive approximation: To Wolfgang Dahmen on 60th birthday

Models and Algorithms for Global Optimization: Essays Dedicated to Antanas Zilinskas on the Occasion of His 60th Birthday

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Robust Adaptive Control of Uncertain Nonlinear Systems

The Composition of Matter: Symposium Honouring Johannes Geiss on the Occasion of His 80th Birthday

Control of human behavior, mental processes, and consciousness: essays in honor of the 60th birthday of August Flammer

Recommend Documents