Lecture Notes in Control and Information Sciences Editors: M. Thoma, M. Morari
374
Biao Huang, Ramesh Kadali
Dynami...
72 downloads
1326 Views
4MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
Lecture Notes in Control and Information Sciences Editors: M. Thoma, M. Morari
374
Biao Huang, Ramesh Kadali
Dynamic Modeling, Predictive Control and Performance Monitoring A Data-driven Subspace Approach
ABC
Series Advisory Board F. Allgöwer, P. Fleming, P. Kokotovic, A.B. Kurzhanski, H. Kwakernaak, A. Rantzer, J.N. Tsitsiklis
Authors Prof. Biao Huang University of Alberta Dept. Chemical & Materials Engineering Edmonton AB T6G 2G6 Canada
Dr. Ramesh Kadali Suncor Energy Inc. Fort McMurray AB T9H 3E3 Canada
ISBN 978-1-84800-232-6
e-ISBN 978-1-84800-233-3
DOI 10.1007/978-1-84800-233-3 Lecture Notes in Control and Information Sciences
ISSN 0170-8643
British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library
Library of Congress Control Number: 2008923061 c
Springer-Verlag London Limited 2008
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms of licences issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers. The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant laws and regulations and therefore free for general use. The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made. Typeset & Cover Design: Scientific Publishing Services Pvt.Ltd., Chennai, India. Printed on acid-free paper 987654321 springer.com
To Yali and Linda - BH To baby Rohan - RK
Preface
Aim of the Book The aim of this book is: 1) to provide an introduction to conventional system identification, model predictive control, and control performance monitoring, and 2) to present a novel subspace framework for closed-loop identification, datadriven predictive control and control performance monitoring. Dynamic modeling, control and monitoring are three central themes in systems and control. Under traditional design frameworks, dynamic models are the prerequisite of control and monitoring. However, models are only vehicles towards achieving these design objectives. Once the design of a controller or a monitor is completed, the model often ceases to exist. The use of models serves well for the design purpose as most traditional designs are model based; it also introduces unavoidable modeling error and complexity in building the model. If a model is identified from data, it is obvious that information contained in the model is no more than that within the original data. Can a controller or monitor be designed directly from input-output data bypassing the modeling step? This book aims to present novel subspace methods to address these questions. In addition, as necessary background material, this book also provides an introduction to the conventional system identification methods for both open-loop and closed-loop processes, conventional model predictive control design, conventional control loop performance assessment techniques, and state-of-the-art model predictive control performance monitoring algorithms. Thus, readers who are interested in conventional approaches to system identification, model predictive control, and control loop performance assessment will also find the book a useful tutorial-style reference.
Novel Data-driven Subspace Approach The subspace approach to process identification has been a great success over the last two decades. The interest in subspace methods continues to grow, towards, for example, the data driven subspace approach to process control and control
VIII
Preface
performance monitoring. On the other hand, subspace identification methods have also been used in industrial model predictive control design software packages. Preliminary studies have shown that the subspace based controller design approach can simplify design procedure and may achieve more desirable properties than the conventional design approaches. Similarly, the subspace based approach has the potential to simplify the multivariable controller performance assessment procedure. These observations motivated the authors to develop a unified framework for predictive control and control performance monitoring, driven directly by input-output data via the subspace approach. The other motivating factor for the development of the data-driven approach is the explosive growth in the amount of process data that is available for analysis. In almost all process environments, ranging from petroleum refining to pulp and paper processing, easy data access through distributed control systems (DCS) is now the rule rather than the exception. In fact, in most process industries data are collected and simply archived. Estimates indicate that most chemical plants require over 100 gigabytes of storage space to archive a year’s worth of data. The data-warehousing problem is a manifestation of the exponential increase in information flow as a result of the recent advances in networks and computers. In summary, we live in an information age in which the process industry world is awash with data. Properly archived, experimentally designed, or routinely operated data can be a tremendous source of information. The question is how to extract useful information from these data and then put them to good use? One of the main objectives of this research monograph is to suggest how data can be used to design predictive control, obtain non-invasive or lessinvasive measures of the performance of the control loops in the plant, or even develop models if they are needed.
Prerequisites Some of the material presented in the book has been published in the archival literature over the last several years by the authors. This book attempts to consolidate the previous results with many new ones in one place. In this respect, the book is likely to be of use for researchers as a monograph and as a place to look for basic information presented in a tutorial style as well as advanced material developed most recently. However, there are also results here that will appeal to industrial practitioners. Portions of the book will also be of use in a graduate level course in process control, control performance monitoring, and system identification. Introductory courses in control and linear algebra are considered the most appropriate prerequisites for the book. It would be of an advantage if readers are acquainted with some knowledge in system identification, but it is not a vital prerequisite for understanding the material as the book also contains a tutorial chapter on conventional system identification.
Preface
IX
Outline of the Book This book is organized as follows: Chapter 1 gives an overall introduction and motivation of the work presented in the book. Chapter 2 is devoted to an introduction of the classical system identification method, namely the prediction error method, and open-loop versus closed-loop identification methods, in a tutorial style. Chapter 3 gives an overview of open-loop subspace identification methods. This chapter provides the background material required for understanding most of the remaining chapters. It also provides a place for readers to find an overview of most open-loop subspace identification algorithms in the literature. Chapter 4 provides an overview of closed-loop subspace identification methods. It also introduces a novel closed-loop subspace identification approach through an orthogonal projection. As a by-product of this development, it answers the question of why the presence of feedback control results in a bias error in some existing subspace identification methods. A solution is provided to eliminate the bias. Chapter 5 introduces a practical subspace identification method for directly obtaining the process dynamic matrix and the noise model from closed-loop data, which is useful for industrial model predictive control design. Chapter 6 provides an introduction to conventional model predictive control design. A simplified step-by-step dynamic matrix control design procedure is illustrated. Chapter 7 describes a method for designing predictive controllers directly from input-output data via the subspace approach. Chapter 8 gives a tutorial on conventional univariate as well as multivariate control loop performance assessment methods and introduces representative algorithms behind this important technology. Chapter 9 presents an overview on most existing model predictive control performance monitoring algorithms. It serves as an introduction to the related research directions and motivations for the remaining chapters. Chapter 10 provides a novel approach for multivariate feedback control performance assessment directly from input-output data using subspace methods, without relying on a model or interactor matrix. The method consolidates the traditional three-step procedure, model identification, close-loop time series analysis, and extraction of the minimum variance control (MVC) benchmark into a one-step solution, leading to an algorithm of considerable compactness. Chapter 11 explores alternative solutions for multivariate control performance assessment problems. It introduces a prediction error approach and does not need a priori knowledge about the process model or time delays/interactor matrices for control performance assessment. Two data-driven subspace algorithms are presented. Chapter 12 describes a method for LQG-benchmark based controller performance analysis following the data-driven subspace approach. The LQG benchmark is useful for performance monitoring of model predictive control systems.
X
Preface
Suggested Sequence of Reading This book aims to provide a relatively complete and coherent view of subspace methods for identification, predictive control, and performance monitoring, each started from a tutorial introduction or overview of the subject matter, followed by in-depth discussions. Thus each of the three subjects may be read independently with some necessary background on subspace methods. For readers who are interested in general subspace methods for all three subjects as well as future research directions, complete reading of the book is recommended. For readers who are only interested in a specific subject, selected chapters and reading sequences are recommended as follows: • Open-loop and closed-loop subspace identification: Chapter 2 =⇒ Chapter 3 =⇒ Chapter 4 • Subspace identification of dynamic matrices for model predictive control: Chapter 2 =⇒ Chapter 3 =⇒ Chapter 5 • Model predictive control tutorial and advanced subspace approach for datadriven predictive control: Chapter 6 =⇒ Chapter 3 =⇒ Chapter 7 • Conventional control performance monitoring and MPC performance monitoring: Chapter 8 =⇒ Chapter 9 • Subspace approach to control performance monitoring: Chapter 3 =⇒ Chapter 8 =⇒ Chapter 10 =⇒ Chapter 11 =⇒ Chapter 12
Acknowledgements The material presented in this book has been the outcome of several years of research efforts by the authors. We started the research in this direction in 2000. We began preparing this book in 2004 when the first author was on sabbatical leave in the University of Duisburg in Germany. After returning to the University of Alberta the writing has continued over three and half years to the current version. Many chapters of the book have gone through several rounds of revisions by discussions with a number of colleagues, graduate students, and postdoctoral fellows. It is a pleasure to be able to thank many people who have contributed so generously in the conception and creation of the material in this text.
Preface
XI
We would like specifically to thank our colleagues and collaborators, Prof. Anthony Rossiter (Sheffield University), Prof. Joe Qin (University of Texas at Austin), Prof. Nina Thornhill (Imperial College London), Prof. Sirish Shah (University of Alberta), Prof. Steven Ding (University of Duisburg), and Prof. Tongwen Chen (University of Alberta), who have inspired many discussions in this or related topics over the past years; Chair of Department of Chemical and Materials Engineering at the University of Alberta, Prof. Fraser Forbes, for his supports of the work; former and current graduate students and postdoctoral fellows, Dr. Fangwei Xu, Mr. Sien Xu, Dr. Xiaorui Wang, Dr. Salim Ahmed, Dr. Shrikant Bhat, Mr. Nima Danesh, Mr. Yu Zhao for their participation in the discussions; our computing support staffs, Bob Barton and Jack Gibeau; and other supporting stuffs of the Department of Chemical and Materials Engineering at the University of Alberta. The supports from the Natural Sciences Engineering Research Council of Canada (NSERC) and Alexander von Humboldt foundation for this and related research work are gratefully acknowledged. Last but not least, we would also like to acknowledge Mr. Oliver Jackson (Springer) for his editorial comments and detailed examination of the book.
Contents
Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . XIX 1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 An Overview of This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Main Features of This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Organization of This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 1 4 4
Part I Dynamic Modeling through Subspace Identification 2
System Identification: Conventional Approach . . . . . . . . . . . . . 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Discrete-time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Finite Difference Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Exact Discretization for Linear Systems . . . . . . . . . . . . . . . 2.2.3 Backshift Operator and Discrete-time Transfer Functions 2.3 An Example of System Identification: ARX Modeling . . . . . . . . . 2.4 Persistent Excitation in Input Signal . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Model Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Prediction Error Model (PEM) . . . . . . . . . . . . . . . . . . . . . . . 2.5.2 AutoRegressive with Exogenous Input Model (ARX) . . . . 2.5.3 AutoRegressive Moving Average with Exogenous Input Model (ARMAX) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.4 Box-Jenkins Model (BJ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.5 Output Error Model (OE) . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.6 MISO (Multi-input and Single-output) Prediction Error Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.7 State Space Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Prediction Error Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.2 Optimal Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.3 Prediction Error Method . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9 9 9 10 10 11 12 13 15 15 15 16 17 17 18 18 19 19 21 24
XIV
Contents
2.7 Closed-loop Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.1 Identifiability without External Excitations . . . . . . . . . . . . 2.7.2 Direct Closed-loop Identification . . . . . . . . . . . . . . . . . . . . . 2.7.3 Indirect Closed-loop Identification . . . . . . . . . . . . . . . . . . . . 2.7.4 Joint Input-output Closed-loop Identification . . . . . . . . . . 2.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25 26 27 28 29 29
3
Open-loop Subspace Identification . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Subspace Matrices Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 State Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Notations and Subspace Equations . . . . . . . . . . . . . . . . . . . 3.3 Open-loop Subspace Identification Methods . . . . . . . . . . . . . . . . . . 3.4 Regression Analysis Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Projection Approach and N4SID . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Projections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Non-steady-state Kalman Filters . . . . . . . . . . . . . . . . . . . . . 3.5.3 Projection Approach for Subspace Identification . . . . . . . . 3.6 QR Factorization and MOESP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Statistical Approach and CVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.1 CVA Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.2 Determination of System Order . . . . . . . . . . . . . . . . . . . . . . 3.8 Instrument-variable Methods and EIV Subspace Identification . . 3.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31 31 31 31 33 40 40 43 43 44 46 48 49 49 51 51 53
4
Closed-loop Subspace Identification . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Review of Closed-loop Subspace Identification Methods . . . . . . . 4.2.1 N4SID Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Joint Input-Output Approach . . . . . . . . . . . . . . . . . . . . . . . . 4.2.3 ARX Prediction Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.4 An Innovation Estimation Approach . . . . . . . . . . . . . . . . . . 4.3 An Orthogonal Projection Approach . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 A Solution through Orthogonal Projection . . . . . . . . . . . . . 4.3.2 The Problem of Biased Estimation and the Solution . . . . 4.3.3 Model Extraction through Kalman Filter State Sequence 4.3.4 Extension to Error-in-variable (EIV) Systems . . . . . . . . . . 4.3.5 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55 55 57 57 59 60 61 63 63 67 69 71 72 78
5
Identification of Dynamic Matrix and Noise Model Using Closed-loop Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Estimation of Process Dynamic Matrix and Noise Model . . . . . . 5.2.1 Estimation of Dynamic Matrix of the Process . . . . . . . . . . 5.2.2 Estimation of the Noise Model . . . . . . . . . . . . . . . . . . . . . . .
79 79 81 84 85
Contents
5.3 Some Guidelines for the Practical Implementation of the Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Extension to the Case of Measured Disturbance Variables . . . . . 5.5 Closed-loop Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.1 Univariate System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.2 Multivariate System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6 Identification of the Dynamic Matrix: Pilot-scale Experimental Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
XV
86 87 89 89 90 94 96
Part II Predictive Control 6
Model Predictive Control: Conventional Approach . . . . . . . . . . 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Understanding MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Fundamentals of MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.1 Process and Disturbance Models . . . . . . . . . . . . . . . . . . . . . 6.3.2 Predictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.3 Free and Forced Response . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.4 Objective Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.5 Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.6 Control Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Dynamic Matrix Control (DMC) . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.1 The Prediction Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.2 Unconstrained DMC Design . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.3 Penalizing the Control Action . . . . . . . . . . . . . . . . . . . . . . . . 6.4.4 Handling Disturbances in DMC . . . . . . . . . . . . . . . . . . . . . . 6.4.5 Multivariate Dynamic Matrix Control . . . . . . . . . . . . . . . . . 6.4.6 Hard Constrained DMC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.7 Economic Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Generalized Predictive Control (GPC) . . . . . . . . . . . . . . . . . . . . . . 6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
101 101 102 103 103 105 106 107 107 108 108 109 111 111 112 113 115 116 117 118
7
Data-driven Subspace Approach to Predictive Control . . . . . . 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Predictive Controller Design from Subspace Matrices . . . . . . . . . . 7.2.1 Inclusion of Integral Action . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.2 Inclusion of Feedforward Control . . . . . . . . . . . . . . . . . . . . . 7.2.3 Constraint Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Tuning the Noise Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Experiment on a Pilot-scale Process . . . . . . . . . . . . . . . . . . . . . . . . 7.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
121 121 122 125 127 128 130 132 138 141
XVI
Contents
Part III Control Performance Monitoring 8
9
Control Loop Performance Assessment: Conventional Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 SISO Feedback Control Performance Assessment . . . . . . . . . . . . . 8.3 MIMO Feedback Control Performance Assessment . . . . . . . . . . . . 8.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
145 145 146 150 155
State-of-the-art MPC Performance Monitoring . . . . . . . . . . . . . 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 MPC Performance Monitoring: Model-based Approach . . . . . . . . 9.2.1 Minimum-variance Control Benchmark . . . . . . . . . . . . . . . . 9.2.2 LQG/MPC Benchmark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.3 Model-based Simulation Approach . . . . . . . . . . . . . . . . . . . . 9.2.4 Designed/Historical vs Achieved . . . . . . . . . . . . . . . . . . . . . . 9.2.5 Historical Covariance Benchmark . . . . . . . . . . . . . . . . . . . . . 9.2.6 MPC Performance Monitoring through Model Validation 9.3 MPC Performance Monitoring: Model-free Approach . . . . . . . . . . 9.3.1 Impulse-Response Curvature . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.2 Prediction-error Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.3 Markov Chain Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 MPC Economic Performance Assessment and Tuning . . . . . . . . . 9.5 Probabilistic Inference for Diagnosis of MPC Performance . . . . . 9.5.1 Bayesian Network for Diagnosis . . . . . . . . . . . . . . . . . . . . . . 9.5.2 Decision Making in Performance Diagnosis . . . . . . . . . . . . 9.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
157 157 158 158 159 160 161 161 162 165 165 166 166 167 171 171 173 175
10 Subspace Approach to MIMO Feedback Control Performance Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2 Subspace Matrices and Their Estimation . . . . . . . . . . . . . . . . . . . . 10.2.1 Revisit of Important Subspace Matrices . . . . . . . . . . . . . . . 10.2.2 Estimation of Subspace Matrices from Open-loop Data . . 10.3 Estimation of MVC-benchmark from Input/Output Data . . . . . . 10.3.1 Closed-loop Subspace Expression of Process Response under Feedback Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.2 Estimation of MVC-benchmark Directly from Input/Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4 Simulations and Application Example . . . . . . . . . . . . . . . . . . . . . . . 10.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
177 177 179 179 180 181 181 183 190 193
Contents
11 Prediction Error Approach to Feedback Control Performance Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Prediction Error Approach to Feedback Control Performance Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Subspace Algorithm for Multi-step Optimal Prediction Errors . . 11.3.1 Preliminary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.2 Calculation of Multi-step Optimal Prediction Errors . . . . 11.3.3 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Performance Assessment with LQG-benchmark from Closed-loop Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Obtaining LQG-benchmark from Feedback Closed-loop Data . . . 12.3 Obtaining LQG-benchmark with Measured Disturbances . . . . . . 12.4 Controller Performance Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.1 Case 1: Feedback Controller Acting on the Process with Unmeasured Disturbances . . . . . . . . . . . . . . . . . . . . . . 12.4.2 Case 2: Feedforward Plus Feedback Controller Acting on the Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.3 Case 3: Feedback Controller Acting on the Process with Measured Disturbances . . . . . . . . . . . . . . . . . . . . . . . . . 12.5 Summary of the Subspace Approach to the Calculation of LQG-benchmark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.6 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.7 Application on a Pilot-scale Process . . . . . . . . . . . . . . . . . . . . . . . . 12.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
XVII
195 195 196 201 201 202 206 211
213 213 214 217 219 219 220 221 222 223 224 227
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
Notation
[A, B, C, D] [Acl , Bcl , Ccl , Dcl ] [Ac , Bc , Cc , Dc ] [Bm , Dm ] A(z −1 ) B(z −1 ) C(z −1 ) D(z −1 ) ∆u ˆ y yˆt r y∗ ˜ p (z −1 ) G at d E[∗] Ef Ei Ep et Fi fi Gm i Gcl (z −1 )
Dynamic state space system matrices of the process Dynamic state space system matrices of the closed-loop system Dynamic state space system matrices of the controller Dynamic state space system matrices corresponding to the measured variables Polynomials in the backshift operator, z −1 Incremental control moves vector over the control horizon Predicted output trajectory vector over the prediction horizon Predicted value of system output(s) at sampling instant t Setpoint (reference) trajectory vector over the prediction horizon Predicted free response trajectory vector over the prediction horizon Delay-free transfer function matrix of Gp Integrated white noise Process time delay for the univariate process or order of the interactor matrix for multivariate process Expectation operator Future data Hankel matrix for et Polynomial obtained in Diophantine expansion Past data Hankel matrix for et White noise (innovations) sequences Markov parameter (or impulse response) matrix at ith sample Impulse response coefficient at ith sample Multivariate step response coefficient matrix corresponding to the measured disturbance input at ith sample Transfer function representation of the closed-loop system
XX
Notation
Gc (z −1 ) Gsc Gi gi Gl (z −1 ) Gp (z −1 ) Gsp h cl HN
c HN
d HN
m HN
s HN
I J j Jmvc K ′ K
Transfer function representation of the controller State space representation of the controller Multivariate step response coefficient matrix corresponding to the deterministic input at ith sample Univariate step response coefficient at ith sample Transfer function representation of the stochastic part of the system or disturbance model Transfer function representation of the deterministic part of the system or process model State space representation of the process Dimension of measured disturbance(s) Lower triangular Toeplitz matrix of closed-loop system defined as Dcl 0 ... 0 Ccl Bcl Dcl ... 0 = ... ... ... ... −2 −3 Ccl AN Bcl Ccl AN Bcl ... Dcl cl cl Lower triangular Toeplitz matrix of controller defined as Dc 0 ... 0 Cc Bc Dc ... 0 = ... ... ... ... −2 −3 Cc AN Bc Cc AN Bc ... Dc c c Lower triangular deterministic Toeplitz matrix defined as 0 0 ... 0 CB 0 ... 0 = ... ... ... ... CAN −2 B CAN −3 B ... 0 Lower triangular Toeplitz matrix corresponding to the measured disturbances defined as Dm 0 ... 0 m CB m D ... 0 = ... ... ... ... CAN −2 B m CAN −3 B m ... Dm Lower matrix defined as triangular stochastic Toeplitz I 0 ... 0 CK I ... 0 = ... ... ... ... CAN −2 K CAN −3 K ... I Identity matrix Optimization objective function Number of block-columns in the block-Hankel matrices Minimum variance control objective function Kalman filter gain matrix = K − K∗
Notation
K∗ Klqg l LCL ue LCL ur LCL u LCL ye LCL yr LCL y Le
Lm Lu
Lw Lbw m Mf Mp mt N n N1 , N2 Nu Q R R1 , R2 Rf Rp rt Sf SN st U, S, V U1 , U2
XXI
Modified Kalman filter gain matrix LQG state feedback gain Dimension of system input(s) Closed-loop subspace matrix from Ef −→ Uf Closed-loop subspace matrix from Rf −→ Uf Closed-loop subspace matrix from WpCL −→ Uf Closed-loop subspace matrix from Ef −→ Yf Closed-loop subspace matrix from Rf −→ Yf Closed-loop subspace matrix from WpCL −→ Yf Subspace matrix containing the noise model Markov parameters; Le is shorthand representation of His where i is typically selected as N Subspace matrix containing the measured disturbances Markov parameters Subspace matrix containing the process Markov parameters; Lu is shorthand representation of Hid where i is typically selected as N Subspace matrix corresponding to past inputs and outputs (or state) Subspace matrix corresponding to the past inputs and outputs (or state) in the presence of measured disturbances Dimension of system output(s) Future data Hankel matrix for mt Past data Hankel matrix for mt Measured disturbance(s) at sampling instant t Number of block-rows in the block-Hankel matrices Dimension of system state(s) Parameters in the prediction horizon Control horizon Non-negative definite weighting matrix Non-negative definite weighting matrix Non-negative definite weighting matrices in control objective function Future data Hankel matrix for setpoint rt Past data Hankel matrix for setpoint rt System setpoint(s) at sampling instant t Future data Hankel matrix for st Dynamic matrix (with N block rows and N block columns) of step-response coefficients Output(s) measurement noise at sampling instant t Matrices from singular value decomposition Left matrices obtained in singular value decomposition
XXII
Notation
Uf
Uf∗ Up
Up∗ ut u∗t V1 , V2 Vf Vp vt W, W1 , W2 Wf Wf∗ Wpr ∗ Wpr
Wp Wpb WpCL wt Xfc Xpc xct
Future data Hankel matrixfor ut defined as uN uN +1 ... uN +j−1 uN +1 uN +2 ... uN +j ... ... ... ... u2N −1 u2N ... u2N +j−2 Future data Hankel matrix for measured inputs, u∗t , for EIV systems; see also Uf Past data Hankel matrix for ut defined as u0 u1 ... uj−1 u1 u2 ... uj ... ... ... ... uN −1 uN ... uN +j−2 Past data Hankel matrix for measured inputs, u∗t , for EIV systems; see also Up System input(s) at sampling instant t Measured system input(s) at sampling instant t for EIV systems Right matrices obtained in singular value decomposition Future data Hankel matrix for vt ; see also Uf Past data Hankel matrix for vt ; see also Up Input measurement noise Non-negative definite weighting matrices Yf U f∗ Yf U∗ f Rf Wp Rf W∗ f Yp U p Yp Up M p Yp Up Rp Process noise Future state matrix of the controller defined as xcN xcN +1 ... xcN +j−1 Past state matrix of the controller defined as xc0 x21 ... c xj−1 Controller state(s) at sampling instant t
Notation XXIII
Xf Xfb Xfcl Xp xt xst yc,t yf,t Yf Yf∗ Yp Yp∗ yt yt∗ ytd yts D(z) F , Fi Gi , Γi (η)f b , (η)f f &f b (E)f b , (E)f f &f b (Iη )f b , (Iη )f f &f b (IE )f b , (IE )f f &f b Γ¯N ∆ ∆cN ∆dN ∆sN η
Future state matrix defined as xN ... xN +j−1 Future state matrix, when the system has measured disturbance variables, defined as xbN ... xbN +j−1 Future closed-loop state matrix Past state matrix defined as x0 ... xj−1 System states(s) at sampling instant t Stochastic component of system states(s) at sampling instant t Forced response of the process output Free response of the process output Future data Hankel matrix for yt Future data Hankel matrix for measured outputs, yt∗ , for EIV systems; see also Uf Past data Hankel matrix for yt Past data Hankel matrix for measured outputs, yt∗ , for EIV systems; see also Up System output(s) at sampling instant t Measured system output(s) at sampling instant t Deterministic component of the system output(s) at sampling instant t Stochastic component of system output(s) at sampling instant t Interactor matrix Polynomials obtained in Diophantine expansion Polynomials obtained in Diophantine expansion LQG-benchmark based controller performance indices with respect to process output variance LQG-benchmark based controller performance indices with respect to process input variance LQG-benchmark based performance improvement indices with respect to process output variance LQG-benchmark based performance improvement indices with respect to process input variance ΓN with its first m rows removed Differencing operator (1 − z −1 ) Reversed extended controllability matrix of {Ac , Bc } −1 −2 Bc AN Bc ... Bc = AN c c Reversed extended controllability matrix of {A, B} = AN −1 B AN −2 B ... B Reversed extended controllability matrix of {A, K} = AN −1 K AN −2 K ... K Performance index
XXIV Notation
ΓNb ΓNcl ΓNc ΓN γN λ ωi , λi , γi , ψi Σe ΓN (t + j | t)
(t | t − j) A/B C t1 | t2
< ∗ ⊥ † T
Extended observability matrix of the expanded system with inputs and measured disturbances Extended observability matrix of the closed-loop system Extended observability matrix of the controller Extended observability matrix of the process, defined as T = C T (CA)T ... (CAN −1 )T Markov parameters used in noise model tuning Weighting on the control effort in control objective functions Parameters for calculating input and output variances for LQG benchmarking Covariance of innovation (white noise) sequence et ΓN with its last m rows removed j step ahead prediction from time instant t. For example, yˆ(t + 2 | t) is a two-step ahead prediction from time instant t; for system identification, the prediction is based on past inputs and outputs, while for predictive control, the prediction is based on past outputs, past inputs, and future inputs. j step ahead prediction from time instant t−j. For example, yˆ(t | t − 2) is a two-step ahead prediction from time instant t − 2; see (t + j | t) for additional explanations Oblique projection = [A/B ⊥ ][C/B ⊥ ]† Subscript; the first column of subspace matrix/vector starts from time t1 and ends by time t2 . If x is a vector, for example, then xt1 xt1 +1 xt1 |t2 = ... xt2 X < Y implies that X − Y is semi-positive definite Superscript; measured value of the corresponding variable Superscript; orthogonal complement Superscript; Moore-Penrose pseudo-inverse Superscript; transpose transformation
1 Introduction
1.1 An Overview of This Book A typical industrial plant can have controllers ranging from proportionalintegral-derivative controllers (PI/PID) to advanced model predictive controllers (MPC), such as dynamic matrix control (DMC) [1, 2], quadratic dynamic matrix control (QDMC) [3, 4], robust multivariable predictive control technology (RMPCT) (see review in [5, 6]), generalized predictive control (GPC) [7, 8], etc. With the goals of optimal performance, energy conservation and cost effectiveness of process operations in industry, the design of optimal controllers and controller performance assessment1 have received great attention in both industry and academia. Typically a ‘model’ or some sort of mathematical representation of the process and the control objective are required not only for designing suitable controllers but also for analyzing controller performance. For predictive controllers, which use a model of the system to make predictions, model identification forms the critical part of controller design. Identification aims at finding a mathematical model from the measurement record of inputs and outputs of a system [9, 10, 11]. Parametric model identification, such as of a transfer function or a state space model, involves obtaining reduced-order models of a pre-specified structure for a system that could be of a very high order and complexity. Nonparametric modeling approaches, such as impulse/step response modeling and frequency-domain based modeling, can also be found in the literature for controller design and performance analysis. Interestingly, nonparametric-model based controller design and analysis tools have been used in industry quite successfully. As a convention, identification of parametric or nonparametric models for the process is typically used as a first step in MPC design, and the models are then converted to prediction matrices as the second step before the calculation of control actions is performed. Data-driven approaches to obtain the prediction matrices used in the controller design directly from 1
In the literature, control performance assessment has also been called control performance monitoring. This book will not attempt to distinguish between these two terms.
B. Huang et al.: Dyn. Model. Predict. Ctrl. & Perform. Monitor., LNCIS 374, pp. 1–5, 2008. c Springer-Verlag London Limited 2008 springerlink.com
2
1 Introduction
process data, and avoid the intermediate explicit model identification procedure, has become an area of active research in recent years [12, 13, 14, 15, 16, 17, 18]. Traditionally, prediction error methods are used to identify parametric models. Subspace identification methods, with their computational advantages, have emerged as a powerful alternative to prediction error methods. They estimate state space models directly from the input-output data and eliminate certain constraints of prediction error methods such as a priori structure selection and non-linear optimization. In subspace identification methods, certain matrices, which capture the correlations between the process inputs and outputs in the form of subspace matrices, are estimated as a first step, by projections of the process input-output data. Lower-order state space system matrices are then obtained from these intermediate subspace matrices. It has been found, however, that the intermediate matrices contain the prediction information and can be used to derive predictors directly for the system. There have been attempts to design predictive controllers directly from these matrices. This book is motivated by the idea of designing predictive controllers directly from the intermediate subspace matrices, and extends the data-driven subspace approach to other important process control areas such as closed-loop identification and controller performance assessment. As a comparison, the conventional control design approach based on system identification, such as subspace identification, consists of 1) calculating subspace matrices through data projections, 2) extracting state space models from subspace matrices, and 3) deriving prediction matrices from the state space models for calculating control actions. The data-driven approach of this book, however, consists of 1) calculating subspace matrices through data projections, and 2) deriving prediction matrices directly from the subspace matrices. Thus, the intermediate step to obtain state space models is avoided. The research on control loop performance monitoring and diagnostics has been and remains one of the most active research areas in the process control community [19, 20, 21, 22, 23, 24]. Despite numerous developments, it remains a challenging problem to obtain a minimum variance control benchmark from routine operating data for a multivariable process, since the solution relies on the interactor matrix (or inverse time delay matrix). Knowing the interactor matrix is tantamount to having a complete knowledge of process models that are often either not available or not accurate enough for a meaningful calculation of the benchmark. There is growing research interest in reducing the complexity of the a priori knowledge requirement, such as Ko and Edgar (2001) [25], Kadali and Huang (2003) [26], and McNabb and Qin (2003) [27]. Although these attempts have reduced the complexity of the a priori knowledge requirement to some extent, they all require certain information that is computationally simpler but fundamentally equivalent to the interactor matrices; for example, the openloop process Markov parameter matrices, the lower triangular Toeplitz matrix, or the multivariate time delay (MTD) matrix. That is, they all require a priori knowledge that is beyond the time delays between each pair of the inputs
1.1 An Overview of This Book
3
and outputs. Thus, derivation of a method for multivariate control performance monitoring directly from input-output data is of great interest. By exploring the subspace approach to control performance monitoring, a solution is presented that depends on input-output data only. In addition to the challenge facing minimum variance control benchmarking, many other challenges also exist, for example, model predictive control performance monitoring; thus other alternative performance monitoring approaches are also introduced, built upon the same data-driven subspace framework. Just as model identification is a critical pre-step for the design of optimal predictive controllers, performance monitoring is an important post-step to ensure optimal performance of the designed controller in operation. As process modifications/changes take place over time the models used in the controller may need to be revisited and re-identified in order to sustain controller performance if the model-based control strategy is used. Open-loop model identification may not be always possible when the process is in operation. Closed-loop identification is a more feasible solution under such a scenario. Hence, closed-loop identification, predictive controller design and controller performance assessment are closely related in industrial control applications. Although this book focuses on a data-driven approach to predictive control design and control performance assessment, model identification, particularly closed-loop subspace model identification, from input-output data is another important ingredient of this book. Conventional prediction error approaches for closed-loop identification have been well addressed [28, 29, 11, 30, 10]. Subspace closed-loop identification remains mostly an outstanding problem, and has received much attention by a number of researchers [31, 13, 14, 30, 32, 33, 34, 35, 36]. It has been found that conventional open-loop subspace identification algorithms often yield biased estimation when applied to closed-loop data [30]. Through an overview of the existing subspace identification methods, this book introduces a novel approach that is based on orthogonal projection of input-output data [36] to address a bias problem. Why do we call the approach introduced in this book ‘data-driven’ ? This question may be answered merely by browsing the topics to be discussed, including: closed-loop estimation of the dynamic matrix and noise model directly from input-output data; predictive controller design without explicit models; controller performance analysis using the linear quadratic Gaussian (LQG) controller without an explicit model; estimation of multivariate minimum variance control (MVC) benchmark without calculating the interactor matrix or models; prediction-error approach to performance monitoring directly from subspace matrices. These topics can be considered ‘data driven’ in the sense that no explicit models are used. Certain intermediate subspace matrices obtained directly from the process data, which are calculated as a first step in subspace identification, are used in the closed-loop identification, controller design and performance assessment. Therefore, the primary difference between the data-driven approach and the conventional approach is whether an explicit model is used.
4
1 Introduction
1.2 Main Features of This Book This book presents both theoretical work and applications in the three important areas of advanced control: system identification, predictive control, and performance assessment, using a data-driven subspace approach. Pertinent features include: 1. A tutorial-style overview of conventional system identification, open-loop and closed-loop subspace identification, predictive control design, and control loop performance assessment. 2. A novel data-driven framework for synthesis of modeling, predictive control, and control performance assessment. 3. Development of a closed-loop subspace identification approach via orthogonal projection. 4. Analysis of bias error in closed-loop subspace identification. 5. Identification of dynamic matrices and noise models from closed-loop data. 6. Derivation of a predictive control law directly from process input-output data using subspace matrices, including all the practical features, such as, • integrator in the controller law, • feedforward control, and • noise model tuning required for practical implementation of the datadriven subspace predictive controller. 7. Derivation of algorithms for calculation of the LQG-benchmark from the subspace matrices estimated using closed-loop data. 8. Theoretical proof and derivation of algorithms for obtaining the multivariate minimum variance control benchmark that can be calculated directly from subspace matrices. 9. Derivation of alternative multivariate control performance assessment methods using the data-driven subspace approach.
1.3 Organization of This Book As reflected in the title, this book is concerned with three subject areas, namely dynamic modeling, predictive control, and control performance monitoring, which are linked through the subspace approach. Consequently, the book can be naturally divided into three parts with a common theme. To appreciate the subspace approach as it is applied to the three subject areas, conventional approaches need to be understood. Fundamental knowledge of conventional system identification, model predictive control, and control performance monitoring is essential for understanding the advanced material presented in this book. Thus, each part of the book starts with a tutorial introduction to the subject area. Part I starts in Chapter 2, which provides an overview of dynamic modeling through the conventional system identification methods. Chapter 3 serves as a starting point to introduce the data-driven subspace approach. The key subspace matrix equations that form the core of data-driven approaches can be
1.3 Organization of This Book
5
found in this chapter. Chapter 4 provides an introduction to conventional closedloop subspace identification, followed by a novel approach to solve a closed-loop subspace identification problem. Part I ends with Chapter 5, which introduces a data-driven closed-loop identification method for dynamic matrix estimation, a critical matrix for conventional model predictive control. Part II starts with a tutorial-oriented introduction to conventional model predictive control in Chapter 6, followed by a novel data-driven approach to predictive control design in Chapter 7. In Part III, conventional control performance monitoring is introduced in Chapter 8, followed by an overview of state-of-the-art model predictive control performance monitoring techniques in Chapter 9. The other three chapters of Part III are devoted to novel data-driven subspace methods for control performance monitoring. Some of the material presented in this book has been published in archival journals by the authors, and is included in this book after necessary modifications or updates (some modifications are major ones) to ensure accuracy, relevance, completeness and coherence. This portion of material includes: • Section 4.3 of Chapter 4, reprinted from Journal of Process Control, vol 15, Huang, B., S.X. Ding, J. Qin, “Closed-loop Subspace Identification: an Orc thogonal Projection Approach”, 53-66, 2005 Elsevier Ltd., with permission from Elsevier. • Chapter 5, reprinted with permission from Ind. Eng. Chem. Res., vol 41, Kadali, R., B. Huang, “Estimation of the Dynamic Matrix and Noise Model c for Model Predictive Control Using Closed-Loop Data”, 842-852, 2002 American Chemical Society. • Chapter 7, reprinted from Control Engineering Practice, vol 11, Kadali, R., B. Huang, A. Rossitor, “A Data Driven Subspace Approach to Predictive c Controller Design”, 261-278, 2003 Elsevier Ltd., with permission from Elsevier. • Chapter 11, reprinted from Journal of Process Control, vol 16, Huang, B., S.X. Ding, and N. Thornhill, “Alternative Solutions to Multivariate Feedback c Control Performance Assessment Problems”, 457-471, 2006 Elsevier Ltd., with permission from Elsevier. • Chapter 12, reprinted from ISA Transactions, vol 41, R. Kadali, and B. Huang, “Controller Performance Analysis with LQG Benchmark Obtained under c Closed Loop Conditions”, 521-537, 2002, 2002 ISA. All Rights Reserved. Reprinted with permission.
2 System Identification: Conventional Approach
2.1 Introduction A necessary prerequisite for conventional model-based control is a model of the process. Such certainty-equivalence, model-based control schemes rely on an offline estimated model of the process, i.e., the process is “probed” or excited by a carefully designed input signal under open-loop conditions and the inputoutput data are used to generate a suitable model of the process. Almost always, reduced-complexity models are generated to capture the most dominant dynamics of the process. Such batch or off-line identification methods represent a major effort and may require anywhere from several hours to several weeks of open-loop tests [22]. In contrast to this, the objective in closed-loop identification is to use closedloop operating data with external excitations to develop a dynamic model of the process. Practically, it is an appealing idea. In this mode, process identification can begin with the process in its natural closed-loop state. In some cases, the plant has to run under closed-loop conditions for safety reasons. In other cases, if a linearized dynamic model around a nominal operating point is desired, this may be achieved better by closed-loop identification. Classical open-loop and closed-loop identification theory has been well developed and presented in the celebrated textbooks by Ljung (1999) [11] and S¨ oderstr¨ om and Stoica (1989) [10]. The most common identification methods are the prediction error method [11] and the instrument variable method [10]. As a prerequisite for understanding the remainder of this book, this chapter is devoted to a tutorial-style introduction to the most representative classical system identification method, namely the prediction error method, following the approach of [10, 11]. The closed-loop identification problem will also be reviewed. For in-depth discussion of these and related subjects, readers are referred to [10, 11].
2.2 Discrete-time Systems The data used for system identification is discrete in general while the system itself is continuous. In this section, following an approach taken by [37], we B. Huang et al.: Dyn. Model. Predict. Ctrl. & Perform. Monitor., LNCIS 374, pp. 9–29, 2008. c Springer-Verlag London Limited 2008 springerlink.com
10
2 System Identification: Conventional Approach
will review the conversion between continuous- and discrete-time systems. For detailed derivations, readers are referred to [37] and other textbooks on digital control or sampled-data control systems. 2.2.1
Finite Difference Models
The simplest approach to convert continuous-time models to discrete-time models is through the finite difference technique. A differential equation, dy(t) = f (y(t), u(t)) dt
(2.1)
can be numerically integrated by introducing a finite difference approximation for the derivative. For example, the first-order, backward difference approximation to the derivative is dy(t) yt − yt−1 ≈ (2.2) dt Ts where Ts is the integration interval or sampling interval that is specified by the user, and yt is sampled quantity of y(t). Substituting Equation 2.2 in Equation 2.1 and evaluating function f (y(t), u(t)) at the previous sample of y and x (i.e., yt−1 and un−1 ) gives yt − yt−1 ≈ f (yt−1 , ut−1 ) Ts
(2.3)
yt ≈ yt−1 + Ts f (yt−1 , ut−1 )
(2.4)
or Equation 2.4 is a first-order difference equation that can be used to predict the value of y at time instant t based on information at the previous time sample t−1, namely, yt−1 and f (yt−1 , ut−1 ). This type of expression is called a recurrence relation. It can be used to numerically integrate Equation 2.1 by calculating yt for t = 0, 1, 2, · · · starting from known initial conditions, y0 and the sequence of inputs. In general, the resulting numerical solution, {yt , t = 1, 2, 3, · · · } becomes more accurate and approaches the correct solution y(t) as Ts decreases. However, for extremely small values of Ts , computer roundoff errors can be a significant source of error. Thus, the finite difference method is only an approximation of continuous-time systems by discrete-time ones. 2.2.2
Exact Discretization for Linear Systems
For a system described by a linear differential equation, an alternative discretetime model can be derived based on the analytical solution for a piecewise constant input. This approach yields an exact discrete-time model if the input variable is actually constant between samplings. Thus, this analytical approach eliminates the discretization error inherent in finite difference approximations for the important practical situation where the digital computer output (process input) is held constant between sampling instants. This is indeed the case if the
2.2 Discrete-time Systems
11
digital to analog device acts as a zero-order hold. We shall illustrate the exact discretization procedure through an example. Consider a first-order differential equation: 1 k dy(t) = − y(t) + u(t) dt τ τ
(2.5)
where u(t) is a piecewise continuous signal. For 0 < t ≤ Ts , u(t) is constant, such that u(t) = u(0). Taking the Laplace transform of Equation 2.5 gives 1 k u(0) sY (s) − y(0) = − Y (s) + τ τ s Solving for Y (s), Y (s) =
1 1 ku(0) [ + y(0)] s + 1/τ τ s
and taking inverse Laplace transform gives y(t) = ku(0)(1 − e−t/τ ) + y(0)e−t/τ
(2.6)
Equation 2.6 is valid for all values of 0 < t ≤ Ts . Thus, after one sampling period, at t = Ts , we have y(Ts ) = ku(0)(1 − e−Ts /τ ) + y(0)e−Ts /τ
(2.7)
We can generalize the analysis by considering the time interval, (t − 1)Ts to tTs . For an initial condition y[(t − 1)Ts ] and a constant input, u(t) = u[(t − 1)Ts ] between (t − 1)Ts and tTs , the analytical solution to Equation 2.5 is y(tTs ) = ku[(t − 1)Ts ](1 − e−Ts /τ ) + y[(t − 1)Ts ]e−Ts /τ
(2.8)
Note that the exponential terms are the same as in Equation 2.7. Finally, Equation 2.8 can be written more compactly as yt = e−Ts /τ yt−1 + k(1 − e−Ts /τ )ut−1 The result of exact discretization discussed can be generalized to higher order systems by following a state space representation of systems. 2.2.3
Backshift Operator and Discrete-time Transfer Functions
The backshift operator z −1 is an operator which moves a signal one step back, i.e., z −1 yt = yt−1 . Similarly, z −2 yt = z −1 yt−1 = yt−2 and zyt = yt+1 . It is convenient to use the backshift operator to represent a difference equation. For example, a difference equation yt = 1.5yt−1 − 0.5yt−2 + 0.5ut−1 can be represented as
(2.9)
12
2 System Identification: Conventional Approach
yt = 1.5z −1 yt − 0.5z −2yt + 0.5z −1ut This can be further written as yt 0.5z −1 = ut 1 − 1.5z −1 + 0.5z −2
(2.10)
This equation is also called a discrete transfer function. Time-delays can be easily represented by the backshift operator. For example, e−5s represents 5 units of time-delay for a continuous-time system. If the sampling period is 1 unit, then this delay can be represented as z −5 for the discrete-time system.
2.3 An Example of System Identification: ARX Modeling System identification is the field of modeling dynamics systems from experimental data, often through discrete-time transfer functions or difference equations. For example, if a dynamics system can be represented by a difference equation: yt + a1 yt−1 + . . . + ana yt−na = b1 ut−1 + b2 ut−2 + . . . + bnb ut−nb
(2.11)
then the task of system identification is to estimate parameters {a1 , · · · , ana } and {b1 , · · · , bnb } using plant input and output data {u1 , u2 , · · · , uN } and {y1 , y2 , · · · , yN }. The advantage of linear difference equation models is that they are particularly convenient to estimate the unknown parameters using linear regression. Consider Equation 2.11, corrupted by noise: yt = −a1 yt−1 − a2 yt−2 − · · · − ana yt−na + +b1 ut−1 + b2 ut−2 + · · · + bnb ut−nb + et
(2.12)
where et is white noise. This is also regarded as an AutoRegressive with eXogenous input (ARX) model. The transfer function form of the ARX model is yt =
b1 z −1 + b2 z −2 + · · · + bnb z −nb ut 1 + a1 z −1 + a2 z −2 + · · · + ana z −na 1 + et 1 + a1 z −1 + a2 z −2 + · · · + ana z −na
For example, a second order ARX model can be written as yt = −a1 yt−1 − a2 yt−2 + b1 ut−1 + b2 ut−2 + et
(2.13)
Suppose that after having excited the system with the input sequence: {u1 , u2 , · · · , uN −1 , uN }, the process response data {y1 , y2 , · · · , yN −1 , yN } are measured. Starting from n = 3 in Equation 2.13, we can write y3 = −a1 y2 − a2 y1 + b1 u2 + b2 u1 + e3 y4 = −a1 y3 − a2 y2 + b1 u3 + b2 u2 + e4 ··· = ··· yN −1 = −a1 yN −2 − a2 yN −3 + b1 uN −2 + b2 uN −3 + eN −1 yN = −a1 yN −1 − a2 yN −2 + b1 uN −1 + b2 uN −2 + eN
2.4 Persistent Excitation in Input Signal
13
Then the parameters {a1 , a2 , b1 , b2 } are unknown in these equations. We need at least 4 equations for these four unknown parameters. Because of the equation a1 , a ˆ2 , ˆb1 , ˆb2 } will not be the exactly error {e3 , e4 , · · · , eN , eN +1 }, the solution {ˆ same as {a1 , a2 , b1 , b2 }, i.e., there exists estimation error. We have to find an estimation method that can minimize the error. One of the estimation methods is known as least squares estimation as stated below. Writing the algebraic equations in a vector-matrix form, we obtain ⎞ ⎛ ⎞ ⎞ ⎛ ⎛ ⎛ ⎞ −y2 y3 e3 −y1 u2 u1 ⎟ ⎜ ⎟ a ⎟ ⎜ ⎜ ⎟ ⎜ ⎟⎜ 1⎟ ⎜ ⎟ ⎜ ⎟ ⎜ −y3 ⎟ ⎜ ⎟ ⎜ e4 ⎟ ⎜ y4 −y2 u3 u2 ⎟ ⎜ ⎟⎜a ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ 2⎟ ⎜ ⎟ ⎜ + ⎟ ⎜··· ⎟ = ⎜··· ⎜ ··· ··· ··· ⎟ ⎜··· ⎟ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ b1 ⎟ ⎜ ⎟ ⎜ ⎜ yN −1 ⎟ ⎜ −yN −2 −yN −3 uN −2 uN −3 ⎟ ⎝ ⎠ ⎜ eN −1 ⎟ ⎠ ⎝ ⎠ b2 ⎠ ⎝ ⎝ yN −yN −1 −yN −2 uN −1 uN −2 eN This can be written as a compact form: Y = Xθ + The well-known least squares solution is then given by θˆ = (X T X)−1 X T Y where θˆ is the estimate of θ and defined as ⎛ ⎞ a ˆ ⎜ 1⎟ ⎜ ⎟ ˆ2 ⎟ ⎜a ⎟ θˆ = ⎜ ⎜ˆ ⎟ ⎜ b1 ⎟ ⎝ ⎠ ˆb2
2.4 Persistent Excitation in Input Signal Up to this stage, we have only discussed estimation of the unknown parameters using the least squares method. We understand that noise et can affect the quality of the estimation. However, in addition to the noise, the input signal, ut , can also affect the accuracy of the estimate. Take a second-order ARX model as an example again: yt = −a1 yt−1 − a2 yt−2 + b1 ut−1 + b2 ut−2 + et Suppose that there is no noise, i.e., et = 0. We therefore expect an exact solution of the 4 unknown parameters by using only 4 equations. However, if the input,
14
2 System Identification: Conventional Approach
ut , has a correlation with ut−1 , say, ut = cut−1 , i.e., u2 = cu1 , u3 = c2 u1 , u4 = c3 u1 , u5 = c4 u1 , then beginning from t = 3 y3 = −a1 y2 − a2 y1 + b1 cu1 + b2 u1 y4 = −a1 y3 − a2 y2 + b1 c2 u1 + b2 cu1 y5 = −a1 y4 − a2 y3 + b1 c3 u1 + b2 c2 u1 y6 = −a1 y5 − a2 y4 + b1 c4 u1 + b2 c3 u1 .. . i.e.,
⎛
y3
⎞
⎛
−y2
⎜ ⎟ ⎜ ⎜ ⎟ ⎜ ⎜ y4 ⎟ ⎜ −y3 ⎜ ⎟ ⎜ ⎜ ⎟ ⎜ ⎜··· ⎟ = ⎜··· ⎜ ⎟ ⎜ ⎜ ⎟ ⎜ ⎜ yN −1 ⎟ ⎜ −yN −2 ⎝ ⎠ ⎝ yN −yN −1
−y1
cu1
−y2 ··· −yN −3 −yN −2
u1
⎞
⎛ ⎞ ⎟ a 1 ⎟ ⎟ ⎟⎜ c2 u 1 cu1 ⎟ ⎟⎜ ⎜ a 2 ⎟⎜ ⎟ = Xθ ⎟⎜ ⎟ ··· ··· ⎟⎜b ⎟ 1 ⎟ ⎟ cN −3 u1 cN −4 u1 ⎟ ⎝ ⎠ ⎠ b2 cN −2 u1 cN −3 u1
It is clear that the last two columns of X matrix are linearly dependent; thus it is not possible to distinguish b1 and b2 in this experiment. Even if there is more data, X T X will not be invertible and the least squares solution does not exist. When the correlation is not perfectly linear, a strong correlation will, however, also inflate (X T X)−1 , and thus inflates the covariance of the estimation since ˆ = V ar(et )(X T X)−1 [10]. In this case, we say that the input sequence is Cov(θ) not persistent exciting. In other words, the estimation error could be infinitely large due to ill-designed input signals. A signal u(t) is said to be persistently exciting (pe) of order n if [10]: 1. the following limit exists: N 1 ut+τ uTt , N →∞ N t=1
ru (τ ) = lim and 2. the matrix
⎛
ru (0)
ru (1) · · · ru (n − 1)
⎜ ⎜ ⎜ ru (−1) ru (0) Ru (n) = ⎜ .. .. ⎜ ⎜ . . ⎝ ru (1 − n) · · ·
is positive definite.
⎞
⎟ ⎟ · · · ru (n − 2) ⎟ ⎟ .. .. ⎟ ⎟ . . ⎠ ···
ru (0)
In general, a necessary condition for consistent estimation of an nth-order linear system is that the input signal is persistently exciting of order 2n [10].
2.5 Model Structures
15
2.5 Model Structures A general model structure is given by [11]: yt = Gp (z −1 ; θ)ut + Gl (z −1 ; θ)et where Gp (z −1 ; θ) is the process/plant transfer function, and Gl (z −1 ; θ) is the disturbance transfer function. We assume: −1 −1 ; θ) and G−1 ; θ)Gp (z −1 ; θ) are asymptotically stable. • G−1 l (z l (z • Gp (0; θ) = 0, Gl (0; θ) = I
However, Gl (z −1 ; θ) is not restricted to be asymptotically stable. Models with unstable Gl (z −1 ; θ) can be useful for describing drift in the data. Some of the most commonly-seen model structures are discussed below. 2.5.1
Prediction Error Model (PEM)
The most general model in the class of commonly used model structures is the prediction error model, illustrated in Figure 2.1 and described by the following equation: B(z −1 ) C(z −1 ) ut + et A(z −1 )yt = −1 F (z ) D(z −1 ) where A(z −1 ) = 1 + a1 z −1 + . . . + ana z −na B(z −1 ) = b1 z −1 + . . . + bnb z −nb C(z −1 ) = 1 + c1 z −1 + . . . + cnc z −nc D(z −1 ) = 1 + d1 z −1 + . . . + dnd z −nd F (z −1 ) = 1 + f1 z −1 + . . . + fnf z −nf The parameter to be estimated is θ = [a1 , . . . , ana , b1 , . . . , bnb , c1 , . . . , cnc , d1 , . . . , dnd , f1 , . . . , fnf ]T 2.5.2
AutoRegressive with Exogenous Input Model (ARX)
By letting C(z −1 ) = D(z −1 ) = F (z −1 ) = 1 in the PEM model, we get an ARX model that has been discussed in the previous section as A(z −1 )yt = B(z −1 )ut + et Alternatively, as illustrated in Figure 2.2, an ARX model can be written as yt =
B(z −1 ) 1 ut + et A(z −1 ) A(z −1 )
Thus, an ARX model is very restrictive comparing to the prediction error model. However, a high order ARX model may be used to approximate most of other model structures [10]. Since an ARX can be estimated easily, this makes an ARX structure attractive as an initial solution to model identification in practice.
16
2 System Identification: Conventional Approach
et
C ( z −1 ) A( z −1 ) D( z −1 )
ut
yt
B ( z −1 ) A( z −1 ) F ( z −1 )
Fig. 2.1. PEM model structure
et
1 A( z −1 )
ut B ( z −1 ) A( z −1 )
yt
Fig. 2.2. ARX model structure
2.5.3
AutoRegressive Moving Average with Exogenous Input Model (ARMAX)
The ARX model is very restrictive. A slight relaxation in the numerator of the disturbance model yields an Autoregressive Moving Average with eXogenous input (ARMAX) model, shown in Figure 2.3, and is given by the following equation A(z −1 )yt = B(z −1 )ut + C(z −1 )et The transfer function form of the ARMAX model is yt =
B(z −1 ) C(z −1 ) u et + t A(z −1 ) A(z −1 )
Thus, an ARMAX model assumes that the process and disturbance models have a common denominator. An ARMAX model is useful in the design of Kalman filter or generalized predictive control (GPC).
2.5 Model Structures
17
et
C ( z −1 ) A( z −1 )
ut
yt
B ( z −1 ) A( z −1 )
Fig. 2.3. ARMAX model structure
2.5.4
Box-Jenkins Model (BJ)
The denominators of both process and disturbance models in ARMAX structure are the same. Relaxing this restrictiveness in the ARMAX model yields the BoxJenkins model, shown in Figure 2.4, and is given in the following equation: yt =
B(z −1 ) C(z −1 ) u et + t F (z −1 ) D(z −1 )
which is a fairly general model in practice where disturbance can have a completely different model from the process.
et
C ( z −1 ) D( z −1 )
ut
yt
B ( z −1 ) F ( z −1 )
Fig. 2.4. BJ model structure
2.5.5
Output Error Model (OE)
If the estimation of the disturbance model is not of interest, a special case of PEM is the OE model, illustrated in Figure 2.5, and given by yt =
B(z −1 ) u t + et F (z −1 )
18
2 System Identification: Conventional Approach
Although the OE model uses et (white noise) as its disturbance, the OE model can be used for other disturbances as well. Because there are fewer parameters to be estimated in the OE model, it is often a good option of model structures in practice.
et ut B ( z −1 ) F ( z −1 )
yt
Fig. 2.5. OE model structure
2.5.6
MISO (Multi-input and Single-output) Prediction Error Model
An l-input and m-output multivariate model may be decomposed into m multiinput and single-output (MISO) models as A(z −1 )yi,t =
2.5.7
B1 (z −1 ) Bl (z −1 ) u ul,t + . . . + 1,t F1 (z −1 ) Fl (z −1 ) C(z −1 ) et i = 1, . . . , m + D(z −1 )
State Space Model
A state space model is typically identified through an innovation or Kalman predictor form: xt+1 = A(θ)xt + B(θ)ut + K(θ)et yt = C(θ)xt + et where A, B, C are system matrices, D matrix is commonly omitted owing to zero-order-hold in the sampling (thus one sample delay results), et is called the innovation sequence, and K is the Kalman predictor gain. This state space model can also be transferred to the general form of the model yt = Gp (z −1 ; θ)ut + Gl (z −1 ; θ)et where Gp (z −1 ; θ) = C(θ)[zI − A(θ)]−1 B(θ) Gl (z −1 ; θ) = I + C(θ)[zI − A(θ)]−1 K(θ) Thus state space models and transfer function models (or difference equation models) are transferable to each other.
2.6 Prediction Error Method
19
2.6 Prediction Error Method 2.6.1
Motivation
The ARX model has the following form: yt = −a1 yt−1 − a2 yt−2 − · · · − ana yt−na + +b1 ut−1 + b2 ut−2 + · · · + bnb ut−nb + et Its estimation is given by θˆ = (X T X)−1 X T Y Let the real model be yt = −a10 yt−1 − a20 yt−2 − · · · − ana0 yt−na + +b10 ut−1 + b20 ut−2 + · · · + bnb0 ut−nb + et Then estimation of the ARX model by applying least squares method is consisN →∞ tent, i.e., θˆ → θ0 . This is verified below. Write ⎛ ⎞ −yt−1 ⎜. ⎟ ⎜. ⎟ ⎜. ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ −yt−na ⎟ ⎜ ⎟ φt = ⎜ ⎟ ⎜ ut−1 ⎟ ⎜ ⎟ ⎜. ⎟ ⎜ .. ⎟ ⎝ ⎠ ut−nb Then the ARX model becomes
yt = φTt θ + et and the real system model is yt = φTt θ0 + et
(2.14)
Furthermore, in the least squares estimation, θˆ = (X T X)−1 X T Y , we have ⎛
φT1
⎞
⎟ ⎜ ⎜ T ⎟ ⎜ φ2 ⎟ ⎟ X=⎜ ⎜ .. ⎟ ⎜. ⎟ ⎠ ⎝ φTN
20
2 System Identification: Conventional Approach
Thus N N φt φTt ]−1 [ φt yt ] θˆ = [ t=1
t=1
N N 1 1 φt φTt ]−1 [ φt yt ] =[ N t=1 N t=1
Replacing yt by Equation 2.14 gives N N 1 1 θˆ = [ φt φTt ]−1 [ φt (φTt θ0 + et )] N t=1 N t=1
= θ0 + [
N N 1 1 φt φTt ]−1 [ φt et ] N t=1 N t=1
N provided [ N1 t=1 φt φTt ]−1 exists, i.e., the input is persistent exciting of sufficient order. As N → ∞, N 1 φt φTt → E[φt φTt ] = Σφ N t=1 N 1 φt et → E[φt et ] N t=1
Thus
θˆ = θ0 + Σφ−1 E[φt et ]
Now whether θˆ is consistent depends on whether E[φt et ] = 0. Since ⎛
⎛ ⎞ ⎞ −yt−1 −E[yt−1 et ] ⎜. ⎜. ⎟ ⎟ ⎜. ⎜. ⎟ ⎟ ⎜. ⎜. ⎟ ⎟ ⎜ ⎜ ⎟ ⎟ ⎜ ⎜ ⎟ ⎟ ⎜ −yt−na ⎟ ⎜ −E[yt−na et ] ⎟ ⎜ ⎜ ⎟ ⎟=0 E[φt et ] = E ⎜ ⎟ et = ⎜ ⎟ ⎜ ut−1 ⎟ ⎜ E[ut−1 et ] ⎟ ⎜ ⎜ ⎟ ⎟ ⎜. ⎜. ⎟ ⎟ ⎜ .. ⎜ .. ⎟ ⎟ ⎝ ⎝ ⎠ ⎠ E[ut−nb et ] ut−nb i.e., white noise is independent of past inputs and outputs, θˆ by least squares is a consistent estimate of θ0 . On the other hand, if et in the ARX model is replaced by vt that is not white noise, then the consistency will be no longer guaranteed. That is to say, all model structures other than ARX may not have the property of the consistency if the (ordinary) least squares method is applied. This procedure of verifying the property of consistency of the ARX model by
2.6 Prediction Error Method
21
applying least squares does give us a hint on how to look for consistency of other model structures. Re-write the ARX model as yt = −a1 yt−1 − a2 yt−2 − · · · − ana yt−na + +b1 ut−1 + b2 ut−2 + · · · + bnb ut−nb + et
= yˆ(t|t − 1) + et where yˆ(t|t − 1) = −a1 yt−1 − · · · − ana yt−na + b1 ut−1 + · · · + bnb ut−nb is one-step ahead prediction of yt at time t−1, based on all information available at time t − 1, namely yt−1 , . . . , yt−na and ut−1 , . . . , ut−nb . The prediction error is yt − yˆ(t|t − 1) = et which is white noise. For other models, can we look for a one-step predictor so that the prediction error is white noise? Does the white-noise prediction error imply certain consistency of identification methods based on the corresponding predictor? The question and answer which we look for are whether such a predictor exists and whether the consistency conclusion can be extended to other model structures. 2.6.2
Optimal Prediction
Prediction error is defined by ε(t, θ) = yt − yˆ(t|t − 1) where yˆ(t|t − 1) now denotes a prediction of yt given all data up to and including time t − 1 (i.e., yt−1 , ut−1 , yt−2 , ut−2 , . . .). Consider the general model structure yt = Gp (z −1 ; θ)ut + Gl (z −1 ; θ)et with the assumption that Gp (0; θ) = 0. A general linear one-step ahead predictor is described as [10]: yˆ(t|t − 1) = L1 (z −1 ; θ)yt + L2 (z −1 ; θ)ut
(2.15)
which is a function of past data if the filters L1 (z −1 ; θ) and L2 (z −1 ; θ) are constrained by L1 (0; θ) = 0
(2.16)
L2 (0; θ) = 0
(2.17)
22
2 System Identification: Conventional Approach
Thus, the prediction error can be further written as ε(t, θ) = Gp (z −1 ; θ)ut + Gl (z −1 ; θ)et − L1 (z −1 ; θ)yt − L2 (z −1 ; θ)ut = Gp (z −1 ; θ)ut + (Gl (z −1 ; θ) − I)et + et − L1 (z −1 ; θ)yt − L2 (z −1 ; θ)ut = (Gp (z −1 ; θ) − L2 (z −1 ; θ))ut + (Gl (z −1 ; θ) − I − L1 (z −1 ; θ))yt + et
= Ψu (z −1 ; θ)ut + Ψy (z −1 ; θ)yt + et Given the conditions Gp (0; θ) = 0, Gl (0; θ) = I, L1 (0; θ) = 0, and L2 (0; θ) = 0, it can be verified that Ψu (0; θ) = 0 Ψy (0; θ) = 0 Namely, both Ψu (z −1 ; θ) and Ψy (z −1 ; θ) have at least one sample time delay. Thus, by expanding transfer functions into impulse response functions, we have Ψu (z −1 ; θ)ut = ψu1 ut−1 + ψu2 ut−2 + . . . Ψy (z −1 ; θ)yt = ψy1 yt−1 + ψy2 yt−2 + . . . Being a future white-noise disturbance relative to Ψu (z −1 ; θ)ut and Ψy (z −1 ; θ)yt , et is independent of both Ψu (z −1 ; θ)ut and Ψy (z −1 ; θ)yt . As a result Cov(ε(t, θ)) = Cov[Ψu (z −1 ; θ)ut + Ψy (z −1 ; θ)yt ] + Cov[et ] Cov(et ) or as a norm expression trace[Cov(ε(t, θ))] ≥ trace[Cov(et )] Therefore, the minimum is given by Cov(et ), i.e., the covariance of white noise et , which is Σe . Consequently, an optimal one-step predictor should give this lower bound as its prediction error. Let’s start from a simple example to demonstrate how the optimal prediction can be derived. Consider the following ARMAX model: yt =
bz −1 1 + cz −1 u + et t 1 + az −1 1 + az −1
The white noise term et can be derived from this equation as et =
1 + az −1 bz −1 (yt − ut ) −1 1 + cz 1 + az −1
(2.18)
The following derivation yields optimal one-step prediction: bz −1 1 + cz −1 ut + et −1 1 + az 1 + az −1 bz −1 (c − a)z −1 = u + (1 + )et t 1 + az −1 1 + az −1 bz −1 (c − a)z −1 = u + et + et t 1 + az −1 1 + az −1
yt =
(2.19)
2.6 Prediction Error Method
23
Replacing et in the second last term of Equation 2.19 by Equation 2.18 yields bz −1 (c − a)z −1 1 + az −1 bz −1 u + (y − u t ) + et t t 1 + az −1 1 + az −1 1 + cz −1 1 + az −1 bz −1 (c − a)z −1 bz −1 = ut + (yt − u t ) + et −1 −1 1 + az 1 + cz 1 + az −1 bz −1 (c − a)z −1 = u + yt + et t 1 + cz −1 1 + cz −1 Let the predictor be yt =
(c − a)z −1 bz −1 u + yt (2.20) t 1 + cz −1 1 + cz −1 which obviously satisfies Equations 2.15-2.17 as a linear one-step ahead predictor. With this prediction, the prediction error is given as yˆ(t|t − 1) =
ε(t, θ) = yt − yˆ(t|t − 1; θ) = et which is white noise; thus the one-step predictor derived in Equation 2.20 is the optimal one-step ahead predictor. For the general model yt = Gp (z −1 ; θ)ut + Gl (z −1 ; θ)et
(2.21)
the optimal prediction and the prediction error can be derived similarly as −1 −1 yˆ(t|t − 1) = G−1 ; θ)Gp (z −1 ; θ)ut + [I − G−1 ; θ)]yt l (z l (z −1 −1 −1 ε(t, θ) = Gl (z ; θ)[yt − Gp (z ; θ)ut ]
(2.22) (2.23)
Combining Equation 2.21 with Equation 2.23, it can be verified that ε(t, θ) = et for the predictor described by Equation 2.22; thus it is an optimal one-step ahead predictor. For a state space model xt+1 = A(θ)xt + B(θ)ut + wt yt = C(θ)xt + et where wt and et are mutually uncorrelated white noise sequences with zero means and covariance matrices R1 (θ) and R2 (θ), the optimal predictor is the Kalman predictor: x ˆ(t + 1|t) = A(θ)ˆ x(t|t − 1) + B(θ)ut + K(θ)[yt − C(θ)ˆ x(t|t − 1)] yˆ(t|t − 1) = C(θ)ˆ x(t|t − 1) where the gain is given by K(θ) = A(θ)P (θ)C T (θ)[C(θ)P (θ)C T (θ) + R2 (θ)]−1 and where P (θ) is the solution of the following algebraic Riccati equation: P (θ) = A(θ)P (θ)AT (θ) + R1 (θ) − K(θ)C(θ)P (θ)AT (θ) Thus, the Kalman filter (innovation) form of the state space models is commonly used in system identification.
24
2 System Identification: Conventional Approach
2.6.3
Prediction Error Method
Estimation of parameters θˆ through the prediction error method is to minimize the prediction error ε(1, θ), ε(2, θ), . . . , ε(N, θ). To define a prediction error one has to make the following choices: • choice of a model structure, • choice of a predictor, i.e., choice of L1 (z −1 ; θ) and L2 (z −1 ; θ), and • choice of a criterion, i.e., measure of the prediction error. In the stochastic framework, L1 (z −1 ; θ) and L2 (z −1 ; θ) are typically chosen to obtain an optimal predictor, as discussed above. The criterion to measure the prediction error can be chosen in many different ways. For illustration, let’s choose the trace of the sample covariance matrix of the prediction errors as the measure:
JN (θ) = trace[ΣN (θ)] = trace[
N 1 ε(t, θ)εT (t, θ)] N t=1
Let the real system be described by yt = Gp (z −1 ; θ0 )ut + Gl (z −1 ; θ0 )et
(2.24)
Substituting this in Equation 2.23 gives −1 ε(t, θ) = G−1 ; θ)[yt − Gp (z −1 ; θ)ut ] l (z −1 −1 ; θ)[Gp (z −1 ; θ0 ) − Gp (z −1 ; θ)]ut + G−1 ; θ)Gl (z −1 ; θ0 )et = G−1 l (z l (z
= Φu (z −1 ; θ, θ0 )ut + Φe (z −1 ; θ, θ0 )et
(2.25)
= Φu (z −1 ; θ, θ0 )ut + (Φe (z −1 ; θ, θ0 ) − I)et + et where −1 Φu (z −1 ; θ, θ0 ) = G−1 ; θ)[Gp (z −1 ; θ0 ) − Gp (z −1 ; θ)] l (z −1 ; θ)Gl (z −1 ; θ0 ) Φe (z −1 ; θ, θ0 ) = G−1 l (z
(2.26) (2.27)
Given Gp (0; θ) = 0 and Gl (0; θ) = I, which also implies Gp (0; θ0 ) = 0 and Gl (0; θ0 ) = I , it can be verified that Φu (0; θ, θ0 ) = 0 Φe (0; θ, θ0 ) − I = 0 Namely, both Φu (0; θ, θ0 ) and (Φe (0; θ, θ0 ) − I) have at least one sample time delay. Consequently, by assuming that ut and et are independent, Cov[ε(t, θ)] = Cov[Φu (z −1 ; θ, θ0 )ut ] + Cov[(Φe (z −1 ; θ, θ0 ) − I)et ] +Cov[et ] Cov[et ] or as a norm expression
2.7 Closed-loop Identification
25
trace[Cov(ε(t, θ))] ≥ trace[Cov(et )] The minimum is, once again, the covariance of the white noise Σe . This minimum is achieved when Φu (z −1 ; θ, θ0 ) = 0
(2.28)
Φe (z −1 ; θ, θ0 ) = I
(2.29)
Comparing these results with Equations 2.26-2.27, achieving the minimu implies Gp (z −1 ; θ) = Gp (z −1 ; θ0 ) Gl (z −1 ; θ) = Gl (z −1 ; θ0 )
(2.30) (2.31)
i.e., consistency is achieved. The procedure of the prediction error method can be summarized below: 1. Select a model structure yt = Gp (z −1 ; θ)ut + Gl (z −1 ; θ)et and determine the corresponding optimal predictor: −1 −1 ; θ)Gp (z −1 ; θ)ut + [I − G−1 ; θ)]yt yˆ(t|t − 1) = G−1 l (z l (z
2. According to the experiment data u1 , u2 , . . . , uN and y1 , y2 , . . . , yN , obtain the predictions yˆ(1|0), yˆ(2|1), . . . , yˆ(N |N − 1), all being functions of θ. The corresponding prediction errors are given by ε(1, θ), ε(2, θ), . . . , ε(N, θ). 3. Write the sample covariance of the prediction error: N 1 ε(t, θ)εT (t, θ) ΣN (θ) = N t=1
ˆ This least squares Minimize ΣN (θ) through its norm (trace), JN (θ), to find θ. problem is generally nonlinear and numerical solutions are typically necessary. As N → ∞, ΣN (θ) → Cov[ε(t, θ)]; if the above optimization procedure yields white noise as the prediction error, then the estimated parameters converge to the true parameters. Thus, the prediction error method has the property of N →∞ consistency for the general model structure, namely θˆ → θ0 .
2.7 Closed-loop Identification Open-loop identification problems have been discussed in the previous sections. What is the difference between open-loop and closed-loop identification? What
26
2 System Identification: Conventional Approach
is the challenge in closed-loop identification? From Figure 2.6 (by adding appropriate parameters), one can observe that two equations exist in the closed-loop system: yt = Gp (z −1 , θ0 )ut + Gl (z −1 , θ0 )et ut = −Gc (z −1 )yt + Gc (z −1 )rt These equations can be written as yt = Gp (z −1 , θ0 )ut + Gl (z −1 , θ0 )et
(2.32)
−1 yt = −G−1 )ut + rt c (z
(2.33)
The input and output data should satisfy both these equations. It is clear that et is the disturbance to Equation 2.32 while rt plays a similar role as a disturbance to Equation 2.33. An identification procedure is to find a model that fits the input and output data ut and yt the best; say fit the following model: yt = Gp (z −1 , θ)ut
(2.34)
Depending on the relative “size” of et and rt , disturbances to the two equations respectively, an identification result may end up with Equation 2.32 (i.e., −1 Gp (z −1 , θ) = Gp (z −1 , θ0 )), Equation 2.33 (i.e., Gp (z −1 , θ) = −G−1 )), or c (z somewhere in between. We shall discuss closed-loop identifiability under several different conditions.
et Gl
rt -
Gc
ut
Gp
yt
Fig. 2.6. Closed-loop system
2.7.1
Identifiability without External Excitations
If rt = 0, i.e., there is no external excitation, Equation 2.34 fits Equation 2.33 −1 perfectly. Any identification algorithm will naturally pick up −G−1 ) as the c (z solution; namely, an inverse of the controller transfer function is identified, and closed-loop identifiability is lost. If Gc (z −1 ) is nonlinear, such as a nonlinear control law, or there is constraint on the control actions and the constraint is active, then it is possible for an identification algorithm to pick up Equation 2.32, i.e., Gp (z −1 , θ0 ) may be identified even without external excitation rt . Since the model to be fitted by data is con−1 ) in Equation 2.33 is nonlinear, strained to a linear model structure but G−1 c (z −1 in this case the linear model Gp (z , θ) of Equation 2.34 may fit Gp (z −1 , θ0 ) of −1 Equation 2.32 better than −G−1 ) of Equation 2.33. c (z
2.7 Closed-loop Identification
2.7.2
27
Direct Closed-loop Identification
If there is at least one sample time delay in Gp (z −1 , θ) and rt is persistent excitation of sufficient order, then as N → ∞, both Gp (z −1 , θ) and Gl (z −1 , θ) can converge to their true values by applying the prediction error method directly to input and output data ut , yt . This is known as the direct method for closed-loop identification [10, 11]. The proof of this result is illustrated through a single-input and single-output system below. Let the real process and disturbance model be Gp (z −1 , θ0 ) and Gl (z −1 , θ0 ), respectively. By replacing Gp and Gl with Gp (z −1 , θ0 ) and Gl (z −1 , θ0 ) in Figure 2.6, in the case of a single-input and single-output system, the real closed-loop input response can be derived as ut =
Gc (z −1 ) −Gl (z −1 , θ0 )Gc (z −1 ) rt + et −1 −1 1 + Gc (z )Gp (z , θ0 ) 1 + Gc (z −1 )Gp (z −1 , θ0 )
(2.35)
Substituting the real system described in Equation 2.24 into the prediction error described in Equation 2.23, gives 1 [yt − Gp (z −1 ; θ)ut ] Gl (z −1 ; θ) Gl (z −1 ; θ0 ) 1 −1 −1 [G et (z ; θ ) − G (z ; θ)]u + = p 0 p t Gl (z −1 ; θ) Gl (z −1 ; θ)
ε(t, θ) =
This is so since we are applying the direct identification method that is no different from open-loop identification in the methodology. Using Equation 2.35 in the equation above, we have ε(t, θ) =
Gc (z −1 ) rt Gl (z −1 ; θ) 1 + Gc (z −1 )Gp (z −1 , θ0 ) −Gl (z −1 , θ0 )Gc (z −1 ) Gl (z −1 ; θ0 ) e et + ] + (2.36) t 1 + Gc (z −1 )Gp (z −1 , θ0 ) Gl (z −1 ; θ) 1
[Gp (z −1 ; θ0 ) − Gp (z −1 ; θ)][
Define sensitivity functions: 1
S(z −1 , θ) =
1+
Gc (z −1 )Gp (z −1 , θ) 1
S(z −1 , θ0 ) =
1+
Gc (z −1 )Gp (z −1 , θ0 )
Then Equation 2.36 can be simplified to ε(t, θ) =
1 [Gp (z −1 ; θ0 ) − Gp (z −1 ; θ)]S(z −1 , θ0 )Gc (z −1 )rt Gl (z −1 ; θ) S(z −1 , θ0 ) Gl (z −1 ; θ0 ) et + (2.37) S(z −1 , θ) Gl (z −1 ; θ)
= Φr (z −1 ; θ, θ0 )rt + Φe (z −1 ; θ, θ0 )et
(2.38)
28
2 System Identification: Conventional Approach
where [Gp (z −1 ; θ0 ) − Gp (z −1 ; θ)]S(z −1 , θ0 )Gc (z −1 ) Gl (z −1 ; θ) −1 −1 S(z , θ0 ) Gl (z ; θ0 ) Φe (z −1 ; θ, θ0 ) = S(z −1 , θ) Gl (z −1 ; θ)
Φr (z −1 ; θ, θ0 ) =
(2.39) (2.40)
Since external excitation rt is independent of white noise et , Equation 2.38 has the same form as Equation 2.25. Following the same proof procedure as the proof of Equations 2.28-2.29, the following equations are obtained when Cov[ε(t, θ)] is minimized: Φr (z −1 ; θ, θ0 ) = 0 Φe (z −1 ; θ, θ0 ) = 1 Thus it can be concluded that, under the closed-loop condition and by applying the direct identification method, when Cov[ε(t, θ)] achieves its minimum Cov[et ], the following equalities hold: Gp (z −1 ; θ) = Gp (z −1 ; θ0 ) S(z −1 , θ) = S(z −1 , θ0 ) Gl (z −1 ; θ) = Gl (z −1 ; θ0 ) i.e., consistency is achieved. 2.7.3
Indirect Closed-loop Identification
The closed-loop identification problem can also be solved through an indirect approach; namely identify the closed-loop transfer function first and then extract Gp (z −1 , θ) from the closed-loop transfer function. The procedure is illustrated below. In the case of a single-input and single-output system, the closed-loop response expression (model) according to Figure 2.6 can be written as yt =
Gc (z −1 )Gp (z −1 , θ) Gl (z −1 , θ) + r et t 1 + Gc (z −1 )Gp (z −1 , θ) 1 + Gc (z −1 )Gp (z −1 , θ)
= M (z −1 , θ)rt + N (z −1 , θ)et This is an open-loop identification problem since rt and et are uncorrelated. Thus M (z −1 , θ) and N (z −1 , θ) can be identified first and then Gp (z −1 , θ) and Gl (z −1 , θ) can be recovered from M (z −1 , θ) and N (z −1 , θ), respectively. For example, in the case of single-input and single-output system, Gp (z −1 , θ) can be calculated as 1 Gp (z −1 , θ) = G (z−1 ) c −1 ) M(z −1 ,θ) − Gc (z The limitation of this approach is that the controller Gc (z −1 ) must be known and the estimated Gp (z −1 , θ) may be of high order. Further model reduction is necessary.
2.8 Summary
2.7.4
29
Joint Input-output Closed-loop Identification
Gc (z −1 ) can also be estimated from data together with Gp (z −1 , θ) and Gl (z −1 ,θ). This approach is known as the joint input-output approach for closed-loop identification. It is illustrated below. In the case of a single-input and single-output system, the closed-loop input and output responses expression (model) can be written as yt =
Gc (z −1 )Gp (z −1 , θ) Gl (z −1 , θ) r et + t 1 + Gc (z −1 )Gp (z −1 , θ) 1 + Gc (z −1 )Gp (z −1 , θ)
= M (z −1 , θ)rt + N (z −1 , θ)et Gc (z −1 ) −Gl (z −1 , θ)Gc (z −1 ) ut = + r et t 1 + Gc (z −1 )Gp (z −1 , θ) 1 + Gc (z −1 )Gp (z −1 , θ)
= P (z −1 , θ)rt + Q(z −1 , θ)et They can be written further as a vector equation: ⎞ ⎞ ⎛ ⎛ ⎞ ⎛ N (z −1 , θ) yt M (z −1 , θ) ⎠ rt + ⎝ ⎠ et ⎝ ⎠=⎝ P (z −1 , θ) Q(z −1 , θ) ut
Since rt and et are uncorrelated, M (z −1 , θ), N (z −1 , θ), P (z −1 , θ), Q(z −1 , θ) can be estimated as an open-loop identification problem, from which Gp (z −1 , θ), Gc (z −1 ), Gl (z −1 , θ) can be extracted. For example Gp (z −1 , θ) can be calculated as M (z −1 , θ) Gp (z −1 , θ) = P (z −1 , θ)
2.8 Summary The most representative classical system identification approach, the prediction error method, has been reviewed in a tutorial form in this chapter. The challenges in closed-loop identification have been discussed. The material presented in this chapter is useful for understanding the chapters which follow, particularly those in the remainder of Part I.
3 Open-loop Subspace Identification
3.1 Introduction Conventionally, a system is modeled by a transfer function, which is a fractional representation of two polynomials with real coefficients, identified using an optimization scheme for a nonlinear least-squares fit to the data, as discussed in Chapter 2. Subspace identification methods offer an alternative identification of a model for the systems and are based on computational tools such as QRfactorization and SVD, which makes them intrinsically robust from a numerical point of view. Subspace identification methods are also non-iterative procedures (avoiding local minima and convergence problems) and may also be converted into an adaptive version of model identification [9]. Subspace identification methods are intrinsically suitable for multivariate systems identification compared to the prediction error methods. This chapter gives an overview of subspace identification methods. A more detailed presentation for specific methods can be found in references such as the books on subspace identification [38, 39], the special issues of the journals Automatica [40, 41] and Signal Processing [42], along with references therein. More algorithms have been added to the literature recently, such as [43, 35, 44, 45, 46, 47]. A variant of subspace identification methods is presented in [48, 49]. Although the principal goal of the methods described in this chapter is to identify the state space system matrices { A, B, C, D }, certain subspace matrices are first calculated as an intermediate step. A significant portion of this book, however, uses only these intermediate matrices for identification, control and monitoring purposes.
3.2 Subspace Matrices Description 3.2.1
State Space Models
There are several forms of state space representation of systems. The innovation form is the one most commonly used in system identification, given by B. Huang et al.: Dyn. Model. Predict. Ctrl. & Perform. Monitor., LNCIS 374, pp. 31–53, 2008. c Springer-Verlag London Limited 2008 springerlink.com
32
3 Open-loop Subspace Identification
xt+1 = Axt + But + Ket yt = Cxt + Dut + et
(3.1) (3.2)
where xt ∈ Rn , ut ∈ Rl , yt ∈ Rm , and et ∈ Rm is white noise (innovations) sequence with covariance Σe . Other forms of state space models have also been used in subspace identification literature, such as
where
⎛
E[⎝
wp sp
xt+1 = Axt + But + wt
(3.3)
yt = Cxt + Dut + st
(3.4)
⎞⎛ ⎠⎝
wp sp
⎞T
⎛
⎠ ]=⎝
Qw
S ws
(S ws )T Rs
⎞
⎠ δpq
where δpq is the Kronecker delta. Another variant of the state space models is represented by xt+1 = Axt + But + Fw wt yt = Cxt + Dut + Hwt + st
(3.5) (3.6)
In Equations 3.5 and 3.6, if we denote wt = Fw wt vt = Hwt + st then this straightforward transformation shows that the model represented by Equations 3.5 and 3.6 is equivalent to that represented by Equations 3.3 and 3.4. It has also been shown [50, 51] that the model represented by Equations 3.3 and 3.4 is also transferrable to that represented by Equations 3.1 and 3.2. This is elaborated below. Equations 3.3 and 3.4 consist of two additive subsystems, one deterministic subsystem driven by deterministic input ut and the other stochastic subsystem driven by noise wt and st . Extracting the stochastic subsystem, we get xst+1 = Axst + wt yts
=
Cxst
+ st
Define Λ = E[(yts )(yts )T ] Σ s = E[(xst )(xst )T ]
(3.7) (3.8)
3.2 Subspace Matrices Description
33
It has been shown [50, 51] that the model represented by Equations 3.3 and 3.4 can be transferred to the innovation form represented by Equations 3.1 and 3.2 with E[et eTt ] = Σe = Λ − CΣ s C T K = (Gs − AΣ s C T )(Λ − CΣ s C T )−1 where Σ s = AΣ s AT + (Gs − AΣ s C T )(Λ − CΣ s C T )−1 (Gs − AΣ s C T )T Gs = AΣ s C T + S ws Λ = CΣ s C T + Rs Owing to the equivalence of different representations of the state space model, following the common practice, we shall focus on the innovation form represented by Equations 3.1 and 3.2 in the sequel unless specified otherwise. 3.2.2
Notations and Subspace Equations
From Equation 3.1, we can derive, for t = N , xN +1 = AxN + BuN + KeN
(3.9)
xN +2 = AxN +1 + BuN +1 + KeN +1
(3.10)
For t = N + 1, Substituting Equation 3.9 in Equation 3.10 yields xN +2 = A(AxN + BuN + KeN ) + BuN +1 + KeN +1 u e N N = A2 xN + AB B + AK K uN +1 eN +1 Continue this procedure for t = N +2, t = N +3, until t = 2N −1. For t = 2N −2 ⎞ ⎛ uN ⎟ ⎜ ⎜ uN +1 ⎟ N −1 N −2 N −3 ⎟ ⎜ x2N −1 = A xN + A BA B · · · B ⎜ .. ⎟ ⎠ ⎝. u2N −2 ⎞ ⎛ eN ⎟ ⎜ ⎜ eN +1 ⎟ N −2 N −3 ⎟ ⎜ (3.11) + A KA K · · · K ⎜ .. ⎟ ⎠ ⎝. e2N −2
For t = 2N − 1
34
3 Open-loop Subspace Identification
x2N
⎛
uN ⎜ ⎜ uN +1 = AN xN + AN −1 B AN −2 B · · · B ⎜ ⎜ .. ⎝. ⎛
u2N −1 ⎞
eN ⎟ ⎜ ⎜ eN +1 ⎟ N −1 N −2 ⎟ ⎜ + A KA K · · · K ⎜ .. ⎟ ⎠ ⎝. e2N −1
⎞ ⎟ ⎟ ⎟ ⎟ ⎠ (3.12)
From Equations 3.2, we can derive, for t = N , yN = CxN + DuN + eN
(3.13)
yN +1 = CxN +1 + DuN +1 + eN +1
(3.14)
For t = N + 1,
Substituting Equation 3.9 in Equation 3.14 yields yN +1 = C(AxN + BuN + KeN ) + DuN +1 + eN +1 uN eN = CAxN + CB D + CK I uN +1 eN +1 Similarly for t = N + 2, using Equations 3.2 and 3.10, we can derive
yN +2
⎞ ⎛ ⎞ uN eN ⎟ ⎜ ⎟ ⎜ = CA2 xN + CAB CB D ⎝ uN +1 ⎠ + CAK CK I ⎝ eN +1 ⎠ uN +2 eN +2
⎛
Continue this procedure until t = 2N − 1. For t = 2N − 1, using Equations 3.2 and 3.11, we can derive
y2N −1
⎛
uN ⎜ ⎜ u N +1 = CAN −1 xN + CAN −2 B CAN −3 B · · · D ⎜ ⎜ .. ⎝.
u2N −1
⎛
⎞
eN ⎟ ⎜ ⎜ eN +1 ⎟ ⎟ N −2 N −3 ⎜ + CA K CA K · · · I ⎜ .. ⎟ ⎠ ⎝. e2N −1
⎞ ⎟ ⎟ ⎟ ⎟ ⎠
3.2 Subspace Matrices Description
35
Assembling the results, for t = N, N + 1, . . . , 2N − 1, yields a matrix equation: ⎛ ⎞ ⎛ ⎞ C yN ⎜ ⎟ ⎜ ⎟ ⎜ yN +1 ⎟ ⎜ CA ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ yN +2 ⎟ = ⎜ CA2 ⎟ xN + ⎜ ⎟ ⎜ ⎟ ⎝ ··· ⎠ ⎝ ··· ⎠ y2N −1 CAN −1 ⎛ ⎞ ⎞⎛ uN D 0 0 ··· 0 ⎜ ⎟ ⎟⎜ ⎜ CB D 0 · · · 0 ⎟ ⎜ uN +1 ⎟ ⎜ ⎟ ⎟⎜ + ⎜ CAB CB D · · · 0 ⎟ ⎜ uN +2 ⎟ ⎜ ⎟ ⎟⎜ ⎝ ··· ··· ··· ··· ···⎠⎝ ··· ⎠ N −2 N −3 N −4 CA B CA B CA B ··· D u2N −1 ⎛ ⎞ ⎞⎛ I 0 0 ··· 0 eN ⎜ ⎟ ⎟⎜ ⎜ CK I 0 · · · 0 ⎟ ⎜ eN +1 ⎟ ⎜ ⎟ ⎟⎜ + ⎜ CAK (3.15) CK I · · · 0 ⎟ ⎜ eN +2 ⎟ ⎜ ⎟ ⎟⎜ ⎝ ··· ··· ··· ··· ···⎠⎝ ··· ⎠ e2N −1 CAN −2 K CAN −3 K CAN −4 K · · · I Adding the time subscripts of the variables with 1, Equation 3.15 changes to ⎞ ⎛ ⎞ ⎛ C yN +1 ⎟ ⎜ ⎟ ⎜ ⎜ yN +2 ⎟ ⎜ CA ⎟ ⎟ ⎜ ⎟ ⎜ 2 ⎜ yN +3 ⎟ = ⎜ CA ⎟ xN +1 + ⎟ ⎜ ⎟ ⎜ ⎝ ··· ⎠ ⎝ ··· ⎠ y2N CAN −1 ⎛ ⎞ ⎞⎛ uN +1 D 0 0 ··· 0 ⎜ ⎟ ⎟⎜ ⎜ CB D 0 · · · 0 ⎟ ⎜ uN +2 ⎟ ⎜ ⎟ ⎟⎜ + ⎜ CAB CB D · · · 0 ⎟ ⎜ uN +3 ⎟ ⎜ ⎟ ⎟⎜ ⎝ ··· ··· ··· ··· ···⎠⎝ ··· ⎠ CAN −2 B CAN −3 B CAN −4 B · · · D u2N ⎛ ⎞ ⎞⎛ eN +1 I 0 0 ··· 0 ⎜ ⎟ ⎟⎜ ⎜ CK I 0 · · · 0 ⎟ ⎜ eN +2 ⎟ ⎜ ⎟ ⎟⎜ + ⎜ CAK CK I · · · 0 ⎟ ⎜ eN +3 ⎟ ⎜ ⎟ ⎟⎜ ⎝ ··· ··· ··· ··· ···⎠⎝ ··· ⎠ CAN −2 K CAN −3 K CAN −4 K · · · I e2N
Continue this procedure by adding the time subscripts of the variables with 2, 3, until j − 1. Assembling the resultant j matrix equations, column by column, gives
36
3 Open-loop Subspace Identification
⎞ ⎛ ⎞ C · · · yN +j−1 ⎜ yN +1 · · · yN +j ⎟ ⎜ CA ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ yN +2 ⎟ = ⎜ CA2 ⎟ x x · · · y N +j+1 · · · x N N +1 N +j−1 ⎜ ⎟ ⎜ ⎟ ⎝ ··· ··· ··· ⎠ ⎝ ··· ⎠ · · · y2N +j−2 y2N −1 CAN −1 ⎞ ⎛ ⎞⎛ uN uN +1 · · · uN +j−1 D 0 0 ··· 0 ⎟ ⎜ ⎟⎜ ⎜ CB D 0 · · · 0 ⎟ ⎜ uN +1 uN +2 · · · uN +j ⎟ ⎟ ⎜ ⎟⎜ + ⎜ CAB CB D · · · 0 ⎟ ⎜ uN +2 uN +3 · · · uN +j+1 ⎟ ⎟ ⎜ ⎟⎜ ⎝ ··· ··· ··· ··· ···⎠⎝ ··· ··· ··· ··· ⎠ CAN −2 B CAN −3 B CAN −4 B · · · D u2N −1 u2N · · · u2N +j−2 ⎛ ⎞ ⎞⎛ I 0 0 ··· 0 eN eN +1 · · · eN +j−1 ⎜ ⎟ ⎟⎜ ⎜ CK I 0 · · · 0 ⎟ ⎜ eN +1 eN +2 · · · eN +j ⎟ ⎜ ⎟ ⎟⎜ + ⎜ CAK CK I · · · 0 ⎟ ⎜ eN +2 eN +3 · · · eN +j+1 ⎟ ⎜ ⎟ ⎟⎜ ⎝ ··· ··· ··· ··· ···⎠⎝ ··· ··· ··· ··· ⎠ CAN −2 K CAN −3 K CAN −4 K · · · I e2N −1 e2N · · · e2N +j−2 (3.16) ⎛
yN
yN +1 yN +2 yN +3 ··· y2N
This completes the derivation of the subspace matrix equation for the output expressions. Next, we shall consider subspace state equation. Adding the time subscripts of the variables in Equation 3.12 with −N gets ⎞ ⎛ u0 ⎟ ⎜ u1 ⎟ ⎜ xN = AN x0 + AN −1 B AN −2 B · · · B ⎜ .. ⎟ ⎠ ⎝. uN −1 ⎞ ⎛ e0 ⎟ ⎜ e1 ⎟ ⎜ (3.17) + AN −1 K AN −2 K · · · K ⎜ .. ⎟ ⎠ ⎝. eN −1
Continuing adding time subscripts of the variables in Equation 3.12 with −(N − 1), −(N − 2), · · · , until −(N − j + 1). By adding time subscripts of the variables in Equation 3.12 with −(N − j + 1), we get ⎞ ⎛ uj−1 ⎟ ⎜ ⎟ ⎜ uj N ⎟ ⎜ N −1 N −2 xN +j−1 = A xj−1 + A BA B · · · B ⎜ .. ⎟ ⎠ ⎝. ⎛
uN +j−2 ⎞
ej−1 ⎟ ⎜ ⎟ ⎜ ej ⎟ N −1 N −2 ⎜ + A KA K · · · K ⎜ .. ⎟ ⎠ ⎝. eN +j−2
(3.18)
3.2 Subspace Matrices Description
By assembling all the state equations just derived, we get N xN xN +1 · · · xN +j−1 = A x0 x1 · · · xj−1 ⎞ ⎛ u0 u1 · · · uj−1 ⎟ ⎜ ⎜ u u2 · · · uj ⎟ + AN −1 B AN −2 B · · · B ⎜ 1 ⎟ ⎝ ··· ··· ··· ··· ⎠ uN −1 uN · · · uN +j−2 ⎞ ⎛ e0 e1 · · · ej−1 ⎟ ⎜ ⎜ e e2 · · · ej ⎟ + AN −1 K AN −2 K · · · K ⎜ 1 ⎟ ⎝ ··· ··· ··· ··· ⎠ eN −1 eN · · · eN +j−2
37
(3.19)
The above derivations lead to the following subspace matrix equations: d s Yf = ΓN Xf + HN U f + HN Ef d s Yp = ΓN Xp + HN Up + HN Ep
(3.20) (3.21)
Xf = AN Xp + ΔdN Up + ΔsN Ep
(3.22)
Equation 3.20 is a short-hand version of Equation 3.16, with detailed notations to be defined shortly. Equation 3.21 can be derived in a same way as that of Equation 3.16. Equation 3.22 is a short-hand version of Equation 3.19. Equations 3.20-3.22 are fundamental subspace equations in subspace literature, where subscript p stands for the “past” and f for the “future”. The past and future input block-Hankel matrices are defined as ⎞ ⎛ u0 u1 · · · uj−1 ⎟ ⎜ ⎜ u u2 · · · uj ⎟ (3.23) Up = U0|N −1 = ⎜ 1 ⎟ ⎝ ··· ··· ··· ··· ⎠ uN −1 uN · · · uN +j−2 ⎞ ⎛ uN uN +1 · · · uN +j−1 ⎟ ⎜ ⎜ u u · · · uN +j ⎟ Uf = UN |2N −1 = ⎜ N +1 N +2 (3.24) ⎟ ⎝ ··· ··· ··· ··· ⎠ u2N −1 u2N · · · u2N +j−2
where Up , Uf ∈ RlN ×j ; the subscript t1 |t2 indicates that the first column of the subspace matrix/vector starts from time t1 and ends by time t2 . The notation of ‘past’ and ‘future’ can be understood from the first column of the block-Hankel matrices. The data in the first column of the ‘future’ block-Hankel matrix follows data in the first column of the ‘past’ block-Hankel matrix in the time sequence. This relation holds for all columns in fact. Note that, in Equations 3.23 and 3.24, the row dimension of Uf could be different from that of Up , which would provide an extra freedom to tune the identification algorithm. For simplicity of presentation, we assume that past and future input block-Hankel matrices have the same dimension.
38
3 Open-loop Subspace Identification
The past/future output and innovation block-Hankel matrices Yp , Yf ∈ RmN×j , Ep , Ef ∈ RmN ×j , respectively, are defined conformably with Up , Uf as
Yp =
Yf =
Ep =
Ef =
⎛
⎞ y0 y1 · · · yj−1 ⎜ ⎟ yj ⎟ ⎜ y1 y2 · · · Y0|N −1 = ⎜ ⎟ ⎝ ··· ··· ··· ··· ⎠ yN −1 yN · · · yN +j−2 ⎞ ⎛ yN yN +1 · · · yN +j−1 ⎟ ⎜ ⎜ yN +1 yN +2 · · · yN +j ⎟ YN |2N −1 = ⎜ ⎟ ⎝ ··· ··· ··· ··· ⎠ y2N −1 y2N · · · y2N +j−2 ⎞ ⎛ e0 e1 · · · ej−1 ⎟ ⎜ ej ⎟ ⎜ e1 e2 · · · E0|N −1 = ⎜ ⎟ ⎝ ··· ··· ··· ··· ⎠ eN −1 eN · · · eN +j−2 ⎞ ⎛ eN eN +1 · · · eN +j−1 ⎟ ⎜ e · · · eN +j ⎟ ⎜e EN |2N −1 = ⎜ N +1 N +2 ⎟ ⎝ ··· ··· ··· ··· ⎠ e2N −1 e2N · · · e2N +j−2
(3.25)
(3.26)
(3.27)
(3.28)
where j should be chosen ‘sufficiently large’ (so that the data Hankel matrices contain sufficient ‘sample size’ to identify the system), and typically j max(mN, lN ) (‘very rectangular’ block Hankel matrices), as this reduces noise sensitivity [9]. Thus j plays a similar role as the number of observations in regression analysis. N is closely related to the ‘order’ of the system to be identified; thus N cannot be too small, but it cannot be too large, as a large N causes an ‘over-parameterization’ problem. Since the input and output are of l and m dimensions, respectively, each element in the above data Hankel matrices is a column vector of inputs and outputs, i.e., ⎛
⎞ ui1 ⎜ ⎟ . ⎟ ui = ⎜ ⎝ .. ⎠ uil
⎛
⎞ yi1 ⎜ ⎟ . ⎟ yi = ⎜ ⎝ .. ⎠ yim
The state matrices are defined as Xp = X0 = x0 x1 · · · xj−1 Xf = XN = xN xN +1 · · · xN +j−1
where Xp , Xf ∈ Rn×j .
(3.29) (3.30)
3.2 Subspace Matrices Description
39
The extended observability matrix ΓN is given as ⎞ ⎛ C ⎜ CA ⎟ ⎟ ⎜ ⎟ ⎜ ΓN = ⎜ CA2 ⎟ ⎟ ⎜ ⎝ ··· ⎠ CAN −1
where ΓN ∈ RmN ×n . The reversed extended controllability matrices ΔdN and ΔsN are given as follows: ΔdN = AN −1 B AN −2 B · · · AB B ΔsN = AN −1 K AN −2 K · · · AK K
Namely, they are the reversed extended controllability matrices of {A, B} and {A, K} respectively. d s and HN are given by The lower triangular block-Toeplitz matrices HN ⎛ ⎞ D 0 0 ··· 0 ⎜ ⎟ ⎜ CB D 0 ··· 0 ⎟ ⎜ ⎟ d (3.31) HN = ⎜ CAB CB D ··· 0 ⎟ ⎜ ⎟ ⎝ ··· ··· ··· ··· ···⎠ CAN −2 B CAN −3 B CAN −4 B · · · D ⎛ ⎞ I 0 0 ··· 0 ⎜ ⎟ ⎜ CK I 0 ··· 0 ⎟ ⎜ ⎟ s (3.32) HN = ⎜ CAK CK I ··· 0 ⎟ ⎜ ⎟ ⎝ ··· ··· ··· ··· ···⎠ CAN −2 K CAN −3 K CAN −4 K · · · I d s where HN ∈ RmN ×lN , HN ∈ RmN ×mN . By substituting equation (3.21) in equation (3.22) we can write ⎡
Yp
⎤
⎥ ⎢ ⎥ ⎢ d s Xf = AN ΓN† ΔdN − AN ΓN† HN ΔsN − AN ΓN† HN ⎢ Up ⎥ ⎦ ⎣ Ep
(3.33)
where † represents the Moore-Penrose pseudo-inverse. In subspace identification literature, the following short-hand notation is often used: ⎛ ⎞ Yp Wp = ⎝ ⎠ Up where Wp ∈ R(mN +lN )×j .
40
3 Open-loop Subspace Identification
3.3 Open-loop Subspace Identification Methods In principle, subspace identification methods can be described as the estimation of a covariance model from observed data followed by stochastic realization [52]. In the open-loop subspace state space identification methods, the sequence of the future states, Xf , and the extended observability matrix, ΓN , are estimated using Equations 3.20-3.22, and the estimation requires thatthe pair {A, C} is completely observable. Furthermore, the pair {A, B KΣe1/2 } is required to be
controllable. All modes must also be sufficiently excited (persistent excitation). Note that even though the deterministic subsystem can have unstable modes, the excitation ut has to be chosen in such a way that the deterministic state and output are bounded at all times. Additionally, the deterministic and stochastic subsystem may have common or completely decoupled input-output dynamics. If the pair {A, C} is observable, then the rank of ΓN equals the state order n. Subspace identification involves estimating a basis for the states of the system from the data Hankel matrices. It must be remembered that the states identified using these techniques do not have any physical meaning. The different subspace identification techniques available in the literature also differ in the manner in which the basis of the state space is estimated. The choices for a basis differ in a transformational matrix T that transforms a model {A, B, C, D } into an equivalent model {T −1 AT, T −1B, CT, D } [53]. The numerical tools used in the estimation of this basis range from SVD (used in [9] and N4SID [38]), QR-decomposition (used in MOESP [54, 55, 56]), canonical variables analysis (used in CVA [57, 58, 59, 60, 61, 62, 63]), etc. Some subspace identification methods also differ on how the disturbances are characterized. In the following sections, we will give an overview of several commonly-seen subspace identification methods.
3.4 Regression Analysis Approach The following derivations [64] are based on Equations 3.1 and 3.2 by shifting the time subscripts: xt = Axt−1 + But−1 + Ket−1
(3.34)
yt−1 = Cxt−1 + Dut−1 + et−1
(3.35)
Combining Equations 3.35 and 3.34 yields xt = (A − KC)xt−1 + Kyt−1 + (B − KD)ut−1
(3.36)
Recursively substituting Equation 3.36 results in xt = (A − KC)N xt−N + (A − KC)N −1 (Kyt−N + (B − KD)ut−N ) + + . . . + (A − KC)(Kyt−2 + (B − KD)ut−2 ) + +Kyt−1 + (B − KD)ut−1
(3.37)
Collecting equations corresponding to t = N, N + 1, . . . , N + j − 1 according to Equation 3.37, the following form of a subspace matrix equation can be obtained:
3.4 Regression Analysis Approach
Xf = Φy Yp + Φu Up + Φx Xp
41
(3.38)
where we note that Φx = (A − KC)N Since K is the Kalman filter gain, Φx represents error dynamics of the Kalman filter; thus as N → ∞, Φx → 0 owing to the stability of a Kalman filter. As a result, for large N , Equation 3.38 converges to Xf = Φy Yp + Φu Up
(3.39)
This result will also be reiterated in the CVA method (to be discussed) for subspace identification. Using Wp , Equation 3.39 can be written as (3.40) Xf = Φy Φu Wp = Lp Wp Substituting Equation 3.40 in 3.20, one can write
Yf = Lw Wp + Lu Uf + Le Ef
(3.41)
where • Lw : subspace matrix corresponding to the past inputs and outputs, • Lu : subspace matrix corresponding to the deterministic inputs, and Lu = d , HN s • Le : subspace matrix corresponding to the stochastic inputs, and Le = HN . In view of Equations 3.20 and 3.41, we can see that Lw = ΓN Lp
(3.42)
Subspace matrices can be identified from data Hankel matrices using regression techniques. Several regression techniques, including the use of weighting matrices, have been suggested in the literature for numerical advantages and special cases of data collection. The simplest regression method is the least squares method. The conditions of the process data for using regressions techniques, as evident from Equation 3.41, are as follows: • The deterministic input ut is uncorrelated with et . • ut is persistently exciting. • The number of measurements is sufficiently large. With the above conditions satisfied, the open-loop identification of the subspace matrices using the least squares solution involves finding the prediction of the future outputs Yf using a linear predictor: Yˆf = Lw Wp + Lu Uf
(3.43)
where the parameters to be determined contain system model information. An alternative derivation of Equation 3.43 will be given in Chapter 10 as well.
42
3 Open-loop Subspace Identification
The prediction Yˆf can be found by solving a least squares problem: Wp min ||Yf − Lw Lu ||2F Lw ,Lu Uf where the subscript F stands for Frobenius norm. The solution is given by [38]:
Lw Lu
= Yf
Wp Uf
†
= Yf
WpT
UfT
Wp ( Uf
−1 Wp Uf )
(3.44)
This algorithm1 can also be implemented in a numerically robust way with a QR-decomposition [65, 66, 38, 48, 54] or using PLS [49]. Following is one of the algorithms: Perform QR decomposition: ⎛ ⎞ ⎛ ⎞⎛ ⎞ Wp Q1 R11 0 0 ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎝ Uf ⎠ = ⎝ R21 R22 0 ⎠ ⎝ Q2 ⎠ Yf Q3 R31 R32 R33 Let
L= Then L can be solved as L=
Lw Lu
R31 R32
R11 0 R21 R22
†
The subspace matrices Lw , Lu and Le will be useful for predictive control and monitoring applications in several chapters of this book. For conventional subspace identification, however, the system matrices A, B, C, D need to be retrieved from results of the regression approach. To retrieve system matrices, the following SVD needs to be performed first: V1T Σ1 L w = U1 U2 (3.45) Σ2 V2T Theoretically, for a finite dimension of state space models, there exists Σ2 = 0 and the dimension of Σ1 determines the dimension of the state space matrix A. In practice, however, Σ2 does not need to be zero owing to the noise in the data. In most subspace literature, it is suggested that the dimension of Σ2 be 1
Strictly speaking, Lw Lu in Equation 3.44 is an estimate and should have been written with a notation different from the original. However, following the common practice in subspace literature without causing confusion, we do not make notational difference for this as well as other subspace matrices between the estimate and the original.
3.5 Projection Approach and N4SID
43
determined through the inspection of the number of significant singular values. A statistical procedure will be presented shortly through Equation 3.77. According to [51] 1/2
ΓN = U1 Σ1 ˆ f = Σ 1/2 V T Wp X 1 1
(3.46) (3.47)
ˆ f consists of the predicted state sequence as a function of past input where X and output data. Equation 3.46 can be understood from Equation 3.42, where we see that the column space of ΓN is the same as that of Lw . Thus the matrix 1/2 U1 Σ1 , spanned by column space of Lw , is an estimate of ΓN . Equation 3.47 can be explained from Equations 3.40, 3.42, 3.45, and 3.46. With the state sequence available, the system matrices may be extracted using several methods. It can be shown that [67, 51] XN +1 = A B XN + K EN |N (3.48) YN |N CD UN |N I where the convention of the subscripts used in Equation 3.48 follows that defined in Equations 3.23-3.30; XN and XN +1 can be constructed by the predicted ˆ f . Another simple linear regression analysis of Equation 3.48 state sequences X will yield an estimation of system matrices A, B, C, D, K. Computational details can be found in [67, 51]. However, because of the non-uniqueness owing to the similarity transformation of state space models, care has to be taken to ensure both XN +1 and XN are extracted from the same basis. This issue has been discussed in [67, 51] and will be further discussed when deriving Equation 4.51. To avoid this basis selection problem, alternatively one may adopt the CVA procedure [57, 58, 59, 60, 61, 62, 68] which will be addressed via Equation 3.76.
3.5 Projection Approach and N4SID 3.5.1
Projections
The following two projections are important for deriving the projection approach. Orthogonal projection: The orthogonal projection of the row space of A on to the row space of B is denoted by A/B and can be calculated through A/B = AB † B where B † is the pseudo inverse of B. In MATLAB , the pseudo inverse can be calculated by using the function pinv or B T (BB T )−1 . Oblique projection: When a vector projects on to two non-orthogonal vectors, the projection is called oblique projection. The oblique projection of the row space of A ∈ Rp×j along the row space of B ∈ Rq×j on the row space of
44
3 Open-loop Subspace Identification
C ∈ Rr×j is defined as A/B C and can be calculated via (following MATLAB matrix notation) A/B C = [A/B ⊥ ][C/B ⊥ ]† = D(:, 1 : r)C where
⎛
D = A⎝
C B
(3.49)
⎞† ⎠
where D(:, 1 : r) denotes the extraction of the matrix elements from the 1st column to the rth column, and B ⊥ is the orthogonal complement of B. The oblique projection, A/B C, can also be visualized through Figure 3.1, where vector A is projected into B and C.
B
A
A /B C
C
Fig. 3.1. Illustration of oblique projection
Two important properties of the oblique projection are often used and they are A/A C = 0
(3.50)
A/B A = A
(3.51)
These two properties are straightforward results by the definition of the oblique projection or by visualizing the corresponding projection through Figure 3.1. 3.5.2
Non-steady-state Kalman Filters
With the above definitions, a non-steady-state Kalman filter can be derived from Equations 3.21-3.22, according to the following procedure:
3.5 Projection Approach and N4SID
45
First, from Equation 3.21, one can obtain −1
s ) Ep = (HN
−1
s Yp − (HN )
−1
s ΓN Xp − (HN )
d HN Up
(3.52)
Substituting Equation 3.52 in Equation 3.22 yields s −1 Xf = (AN − ΔsN (HN ) ΓN )Xp
s −1 s −1 d + ΔsN (HN ) ΔdN − ΔsN (HN ) HN Wp
(3.53)
s −1 Xf /R Wp = (AN − ΔsN (HN ) ΓN )Xp /R Wp
−1 s s −1 d + ΔsN (HN ) ΔdN − ΔsN (HN ) HN Wp /R Wp
(3.54)
It is easy to see that Xf is the non-steady state solution of Kalman filter state with Xp as its initial value since it is derived from Equations 3.21-3.22, which are the solutions of the innovation form of the state space model, Equations 3.1 and 3.2. The innovation form of the state space equations is the Kalman filter [69, 11]. In subspace literature, one often performs an oblique projection of Equation 3.53 on to Wp through R where R is any non-zero constant matrix of appropriate dimension
Clearly, Wp /R Wp = Wp , by the property of the oblique projection, Equation 3.51. Therefore, Equation 3.54 can be simplified to −1
s ) ΓN )Xp /R Wp Xf /R Wp = (AN − ΔsN (HN
s −1 s −1 d + ΔsN (HN ) ΔdN − ΔsN (HN ) HN Wp
(3.55)
s −1 ) ΓN )Xp /Wp Xf /Wp = (AN − ΔsN (HN
−1 s s −1 d + ΔsN (HN ) ΔdN − ΔsN (HN ) HN Wp
(3.56)
ˆ f = Xf /Wp X
(3.57)
ˆ p = Xp /Wp X
(3.58)
Comparing Equation 3.55 with Equation 3.53, one can see that both serve as the solutions of the innovation state space equations 3.1 and 3.2. In Equation 3.53, Xf is the Kalman filter state with Xp as its initial condition. In ˆ f is also a Kalman filter state2 but with the initial Equation 3.55, Xf /R Wp = X ˆ condition Xp /R Wp = Xp . Following the same line, if we now make an orthogonal projection of Equation 3.53 onto Wp , we should have
and we can equally treat Xf /Wp as another Kalman filter state solution but with Xp /Wp as its initial condition, i.e.,
with the initial state as 2
Since the Kalman filter state is a function of past input and output data Wp (i.e., a ˆ f to reflect this fact. prediction), we have used the notation X
46
3 Open-loop Subspace Identification
3.5.3
Projection Approach for Subspace Identification
Let’s now revisit Equation 3.20 d s Yf = ΓN Xf + HN U f + HN Ef
(3.59)
The essential system information is contained in the extended observability matrix ΓN or in the state Xf . That is, the first term on the right hand side of Equation 3.59 deserves special attention. To calculate, for example, ΓN from Equation 3.59, one has to remove the terms containing Uf and Ef . If Ef is independent of past input Up , past output Yp (or equivalently their combination Wp ), and future input Uf , then one can easily achieve the above objective by performing an oblique projection of Equation 3.59 along the row space Uf onto the row space of Wp , i.e., d s Yf /Uf Wp = ΓN Xf /Uf Wp + HN Uf /Uf Wp + HN Ef /Uf Wp
(3.60)
It is easy to see why the last two terms of Equation 3.60 are zero: Uf /Uf Wp = 0 is by the property of the oblique projection according to Equation 3.50; Ef /Uf Wp = 0 is based on the assumption that future white-noise disturbance is independent of past inputs/outputs and future inputs. This assumption holds under the open-loop condition. Thus Equation 3.60 can be simplified to Yf /Uf Wp = ΓN Xf /Uf Wp
(3.61)
This result indicates that the column space of ΓN is the same as the column space of Yf /Uf Wp , which can be calculated by the SVD decomposition of Yf /Uf Wp . Similarly, the row space of Yf /Uf Wp is the same as the row space of Xf /Uf Wp , which is the Kalman filter state solution with Xp /Uf Wp as its initial condition, as has been discussed before. Therefore, the Kalman state sequences can also be calculated from the SVD decomposition of Yf /Uf Wp . Subsequently, the system state space matrices can be recovered either from the extended observability matrix or from the Kalman state sequences. This is a solution to the open-loop subspace system identification problem through projection. It has been shown in Equation 3.40 as well as in [51] that the state matrix Xf can be written as linear combination of Wp ; thus by performing oblique projection of Equation 3.41, we get Yf /Uf Wp = Lw Wp
(3.62)
This result has related the projection approach to the regression approach. Thus by performing SVD of Lw , as shown in Equation 3.45, ΓN and Xf can be estimated according to Equations 3.46 and 3.47. The system matrices may be estimated according to Equation 3.48. This procedure is known as N4SID (Numerical subspace state space identification). Alternatively, following the approach of MOESP (MIMO output error state space model identification) [54], system matrices may also be extracted from subspace matrices as discussed in the following.
3.5 Projection Approach and N4SID
47
It is noted that Γ¯N = Γ N A
(3.63)
where Γ¯N is ΓN by removing its first m rows, and Γ N is ΓN by removing its last m rows. Thus A = Γ † Γ¯N N
C = ΓN (1 : m, :) namely, the system matrix C can be simply extracted from the first m rows of ΓN . It can be further shown [70] that d (ΓN⊥ )T Yf Uf† = (ΓN⊥ )T HN
(3.64)
where ΓN⊥ is the orthogonal complement of ΓN with full rank. This equation can be rewritten, with term-by-term correspondence to Equation 3.64, as [51]
M1 M2 . . . MN
⎛
⎞ ··· 0 ⎜ ··· 0 ⎟ ⎟ ⎜ ⎜ ⎟ ··· 0 ⎟ = L1 L2 . . . LN ⎜ ⎜ ⎟ ⎝ ··· ··· ··· ··· ···⎠ CAN −2 B CAN −3 B CAN −4 B · · · D D CB CAB
0 D CB
0 0 D
By algebraic rearrangement, an explicit equation results [51]: ⎛
⎞ L L · · · L L 1 2 N −1 N M ⎛ ⎞⎛ ⎞ ⎜ 1⎟ ⎜ L2 L3 · · · LN 0 ⎟ ⎟ I ⎜ M2 ⎟ ⎜ 0 D N ⎟ ⎜ . ⎟=⎜ ⎠⎝ ⎠ 0 ⎟⎝ L3 L4 · · · 0 ⎜ . ⎟ ⎜ ⎟ ⎝ . ⎠ ⎜ 0 ΓN (1 : end − m, :) B ⎝··· ··· ··· ··· ··· ⎠ MN 0 LN 0 0 · · · ⎛
⎞
which is a set of linear equations for solving D and B. One may ask, at this stage, whether the calculation of the column space of Yf /Uf Wp alone is sufficient for the estimation of ΓN . The answer is that the exact value of ΓN is immaterial owing to the similarity transformation of the state space model. That is to say, the state space matrices extracted from the column space of ΓN are all equivalent up to the similarity transformation. However, there may exist an optimal solution. For an in-depth discussion, readers are referred to Overschee and De Moor(1996)[67] and Favoreel (1999)[51]. Other related references include [34, 58, 60, 65, 66, 38, 48, 49]. The above procedure can also be explained as a linear regression based on the multi-step ahead prediction error method with certain rank constraints [71].
48
3 Open-loop Subspace Identification
Rather than performing singular value decomposition on Lw , if the following weighted SVD is performed: ⎛ ⎞ ⎞⎛ Σ1 V1T ⎠ ⎠⎝ (3.65) W1 Lw W2 = U1 U2 ⎝ 0 V2T then according to [51], ΓN = W1−1 U1 Σ1 ˆ f = Σ 1/2 V T W −1 Wp X 1 2 1 1/2
(3.66) (3.67)
This is known as a unified approach to various subspace algorithms [51]. The MOESP algorithm, which will be discussed in detail in the next section, corresponds to the following weights: W1 = I and W2 = I −Uf⊥ Uf . This interpretation of subspace identification unifies various subspace identification algorithms including N4SD and CVA [51].
3.6 QR Factorization and MOESP MOESP [54] considers the following state space model:
xt+1 = Axt + But + Fw wt
(3.68)
yt = Cxt + Dut + Hwt + st
(3.69)
where the process noise wt and measurement noise st are zero-mean white noise, both independent of the input ut . The approach taken by MOESP is through QR factorization and then a procedure similar to statistical correlation analysis, by extensive use of the property of white noise. Owing to the use of QR factorization for solving the problem, the MOESP is expected to be quite robust numerically. Let the input ut be persistent excitation of sufficient order. Perform the following QR factorization: ⎛ ⎞ ⎛ ⎞⎛ ⎞ Q1 Uf L11 0 0 0 ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ Up ⎟ ⎜ L21 L22 0 0 ⎟ ⎜ Q2 ⎟ (3.70) ⎜ ⎟=⎜ ⎟⎜ ⎟ ⎝ Yp ⎠ ⎝ L31 L32 L33 0 ⎠ ⎝ Q3 ⎠ Yf Q4 L41 L42 L43 L44 and SVD:
L42 L43
=
⎛
U1 U2 ⎝
Σ1 0 0 0
⎞⎛ ⎠⎝
V1T V2T
⎞ ⎠
(3.71)
3.7 Statistical Approach and CVA
49
Then the column space of U1 is the same as that of ΓN . Consequently, if T is a non-singular n × n matrix, then ΓN = U1 T 1/2
As an example, taking T = Σ1 , then 1
ΓN = U1 Σ12 Hence the system order is obtained using the SVD. From the estimated ΓN , system matrices A and C are derived. In addition, the following identity also holds from the QR factorization and SVD procedure: (U1⊥ )T
L31 L32 L41
L21 L22 L11
†
d = (U1⊥ )T HN
(3.72)
d can be solved from Equation 3.72. Thus HN d With the estimation of ΓN and HN , the system matrices A, B, C, D can be extracted following the same procedure as discussed in the previous section.
3.7 Statistical Approach and CVA 3.7.1
CVA Approach
CVA (canonical variate analysis) considers the same model structure as that of MOESP, shown in Equations 3.68 and 3.69. In this method canonical variables are used to provide an ordered basis of the state space, ordered in terms of the predictive ability of the states [57, 58, 59, 60, 61, 62, 63]. The canonical correlations between ‘the past, Wp ’ and ‘the future outputs, Yf , conditional on the future inputs, Uf ’ are used as the basis of the state space. If n is the true and finite state order and there is sufficient information to determine reliably the order n, then the first n canonical variables give an optimal selection of system states in terms of maximizing the likelihood function [58]. The CVA [60, 58, 63] algorithm uses Akaike information criteria (AIC) to determine the state order. Following is the detailed algorithm. Define data column vectors over a horizon of N for the output, input, and their combination, respectively, as ⎞ ⎛ y ⎞ ⎞ ⎛ ⎛ ⎟ ⎜ t−N ⎜ .. ⎟ yt ut ⎜ . ⎟ ⎟ ⎟ ⎜ ⎜ ⎟ ⎜ .. .. ⎟ ⎟ ⎜ ⎜ yt−1 ⎟ ⎟ , uf = ⎜ ⎟ , wp = ⎜ . . (3.73) yf = ⎜ ⎟ ⎜ ⎟ ⎟ ⎜ ⎜ ⎜ ut−N ⎟ ⎝ yt+N −2 ⎠ ⎝ ut+N −2 ⎠ ⎜ . ⎟ ⎜ . ⎟ yt+N −1 ut+N −1 ⎝ . ⎠ ut−1
50
3 Open-loop Subspace Identification
By eliminating the effect of future input uf from future output yf , u y˜f = yf − Σyf uf Σu−1 f uf f where y˜f denotes yf after eliminating the effect of uf , Σyf uf = Cov(yf , uf ), and Σuf uf = Cov(uf , uf ). Σyf uf can be estimated from matrices Yf and Uf , and Σuf uf can be estimated from matrix Uf . It is argued [57, 58, 59, 60, 61, 62, 68] that a fundamental property of a linear, time invariant, and strict sense controlled Markov process with the state order n is the existence of a n-dimensional state xt , which can be written as a linear function of the past input-output wp and expressed as xt = Jn wp
(3.74)
where Jn denotes n rows of a matrix J to be determined. Note that this property has also been shown in Equation 3.40. With state xt , the optimal linear prediction y˜f is given by ˆf = Σy˜f xt Σx−1x xt y˜ t
t
Thus the CVA problem boils down to the optimal solution of the state or alternatively the matrix Jn , to minimize the prediction error. The prediction error is measured by ˜f )T Λ† (˜ yf − yˆ˜f ) (3.75) E(˜ yf − yˆ where Λ† is an arbitrary quadratic weighting that can be singular. For an optimal solution, Λ = Σy f y f which, according to [60], results in a near maximum likelihood system identification procedure. The state estimation problem is the following optimization problem: min E(˜ yf − yˆ ˜f )T Λ† (˜ yf − yˆ ˜f ) xt
yf − Σy˜f xt Σx−1 x )T Λ† (˜ yf − Σy˜f xt Σx−1 x) = min E(˜ t xt t t xt t xt
The solution is given by the generalized singular value solution: JΣwp wp J T = I LΛLT = I JΣwp y˜f LT = diag(r1 , r2 , . . . , rn , 0, . . . , 0) where r1 ≥ r2 ≥ . . . ≥ rn > 0 are generalized singular values. The estimation of Jn is given by the first n rows of J. In practice, those theoretically “zero” singular values may not be zero. Thus a rank determination procedure needs to be performed and will be discussed shortly.
3.8 Instrument-variable Methods and EIV Subspace Identification
51
Once the state is estimated according to Equation 3.74, the state space model can be calculated as follows [68]: ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ xt+1 xt x x AB ⎠ = Cov(⎝ ⎠ , ⎝ ⎠)Cov −1 (⎝ t ⎠ , ⎝ t ⎠) ⎝ (3.76) CD yt ut ut ut
Note that Van Overschee and De Moor [66, 51] have classified the CVA method as a special case under a unifying framework and the difference between different subspace algorithms can be expressed in terms of the weighting matrices involved in the SVD. They have argued that, in Equation 3.65, if W1 = [(Yf /Uf⊥ )(Yf /Uf⊥ )T ]−1/2 and W2 = I − Uf⊥ Uf , then the CVA algorithm is obtained. 3.7.2
Determination of System Order
The AIC criterion is given by [60, 72] AICN (n) = j(m(1 + ln2π) + ln|Σn |) + 2δn Mn
(3.77)
where j
Σn = and
1 T t j t=1 t
t = yt − yˆt represents the residual, where yˆt is the one-step prediction based on the identified model. Mn is the total number of unknown parameters in the model. For the CVA model given by Equations 3.68 and 3.69, Mn = (2n + l)m + lm + m(m + 1)/2 The correcting factor, δn , accounting for the limited number of samples, is given by j δn = j − ( Mmn + m+1 2 ) The model order n or the number of significant singular values is the one that minimizes AICN (n).
3.8 Instrument-variable Methods and EIV Subspace Identification Several subspace identification methods can also be unified under the instrument variable framework [73]. With an appropriate selection of the instrument variables, error-in-variable (EIV) problem can also be dealt with.
52
3 Open-loop Subspace Identification
Consider the following EIV state space model, originated from Equations 3.3 and 3.4 without measurement noise: xt+1 = Axt + But + γt yt = Cxt + Dut In the EIV case, both yt and ut cannot be measured directly. The measured input and output are given by u∗t = ut + vt yt∗ = yt + st where γt , vt , st are white noise. With this EIV system, a subspace matrix equation can be derived as γ d v Uf∗ + HN Vf + HN Γf + Sf Yf∗ = ΓN Xf + HN
(3.78)
where Vf , Γf , Sf are data Hankel matrices corresponding to the white noise vt , γt , st , respectively. If the following instrument variable is selected: ⎞ ⎛ Up∗ ⎠ Π=⎝ Yp∗
then it is clear that Π is independent of Vf , Γf , Sf owing to the fact that past input and output are independent of future white noise. Right multiplying Equation 3.78 by 1j Π T yields 1 ∗ T 1 1 d ∗ T 1 v 1 γ 1 Y Π = ΓN Xf Π T + HN U f Π + HN Vf Π T + HN Γf Π T + Sf Π T j f j j j j j
As j → ∞, the following result is obtained: 1 ∗ T 1 1 d ∗ T Y Π = ΓN Xf Π T + HN Uf Π j f j j
(3.79)
To obtain the column space of ΓN , multiply Equation 3.79 by the orthogonal complement of Uf∗ Π T . This gives 1 ∗ T ∗ T ⊥ 1 Y Π (Uf Π ) = ΓN Xf Π T (Uf∗ Π T )⊥ j f j Thus ΓN can be estimated through SVD of 1j Yf∗ Π T (Uf∗ Π T )⊥ . With availability of ΓN , following the same procedure as discussed in previous sections, the system matrices can be retrieved. Detailed derivations of the instrument variable method for subspace identification can be found in [73]. It is claimed that the above procedure can be applied to closed-loop data, provided that the controller is causal, there is at least one sample time delay in the controller, and the closed-loop system is asymptotically stable. The limitation, however, lies in the assumption that there is at least one sample time delay in the controller.
3.9 Summary
53
Remark 3.1. If there is no measurement noise on the input, i.e., vt = 0, then the following instrument variable is also suggested [73]: ⎞ ⎛ Up ⎟ ⎜ ⎟ ⎜ Π = ⎜ Yp∗ ⎟ ⎠ ⎝ Uf
In an earlier paper by Chou and Verhaegen [74], a similar approach for EIV subspace identification based on QR factorization was proposed and the results are provided below. Perform the following QR factorization: ⎛ ⎞ ⎛ ⎞⎛ ⎞ Uf Up∗T Uf Yp∗T Q1 R11 0 ⎝ ⎠=⎝ ⎠⎝ ⎠ Yf∗ Up∗T Yf∗ Yp∗T Q2 R21 R22 Then as j → ∞,
1 1 d √ R21 = √ {ΓN Xf Up∗T Yp∗T QT1 + HN R11 } j j 1 1 √ R22 = √ ΓN Xf Up∗T Yp∗T QT2 j j
(3.80) (3.81)
According to Equation 3.81, the column space of ΓN is the same as that of √1 R22 ; thus SVD of √1 R22 will provide an estimate of ΓN following the N N same SVD procedure as discussed previously in this chapter. Combining Equad can be calculated. Thus all A, B, C, D matrices can be tions 3.80 and 3.81, HN d retrieved from ΓN and HN .
3.9 Summary An overview of a variety of open-loop subspace identification methods has been presented in this chapter. Many new subspace identification methods are being added to the literature, indicating a consistent interest in this direction. Many well-known model predictive control systems have also taken a subspace approach as options in building dynamic models, indicating the practicality of subspace identification. Nevertheless, open-loop subspace identification has been considered a matured subject. Many problems and current research interest lie in closed-loop subspace identification, which will be discussed in the next chapter.
4 Closed-loop Subspace Identification∗
4.1 Introduction The problem of closed-loop identification has been investigated for over 30 years. Important issues such as identifiability under closed-loop conditions have received attention by many researchers [75, 76, 77, 10]. A number of identification strategies have been developed [11, 10]. Closed-loop identification refers to the identification of process models using the data sampled under feedback control. Correlation between the disturbances entering the process and the input offers a fundamental limitation [28, 29, 11, 30, 10] for utilizing the standard open-loop identification methods with closed-loop data. Several closed-loop parametric model identification methods have been suggested in the literature which require either certain assumptions about the model structure or knowledge of the controller model. The closed-loop identification methods found in the literature are broadly classified into direct, indirect and joint input/output identification methods [29]. See Chapter 2 and references in [78, 29, 79] for a review of the features and limitations of different classical closed-loop identification methods. Closed-loop identification has attracted increasing interest over the last two decades owing to the work of integrated identification and control. The key idea in the integrated identification and control strategy (as opposed to a ‘disjoint’ or separate identification and control) is to identify and control with the objective of optimizing a control performance criterion. This topic has received attention under such headings as: control-relevant identification, iterative identification and control, etc. Readers are referred to [80, 81, 82] for detailed discussions on these topics. After the establishment of the classical prediction error methods (PEM) for system identification, the subspace identification method is a relatively new approach used for the state space model identification. In this approach, certain subspace matrices of the process are, at the first step, calculated through data projection [65, 66, 67]. These matrices include, for example, an extended observability matrix, a lower triangular block-Toeplitz matrix, and/or state sequences ∗
Section 4.3 is reprinted from Journal of Process Control, vol 15, Huang, B., S.X. Ding, J. Qin, “Closed-loop Subspace Identification: an Orthogonal Projection c 2005 Elsevier Ltd., with permission from Elsevier. Approach”, 53-66,
B. Huang et al.: Dyn. Model. Predict. Ctrl. & Perform. Monitor., LNCIS 374, pp. 55–78, 2008. c Springer-Verlag London Limited 2008 springerlink.com
56
4 Closed-loop Subspace Identification
as discussed in Chapter 3. These subspace matrices are directly calculated from the input/output data Hankel matrices without any iteration compared to the iterative or nonlinear optimization schemes used in the prediction error methods. At the second step, the system state space matrices are recovered from either the extended observability matrix and the lower triangular block-Toeplitz matrix or directly from the state sequences. Identification of the subspace matrices from closed-loop data has also received attention from a number of researchers [31, 13, 14, 30, 32, 33]. It is found that the conventional open-loop subspace identification algorithm yields a biased estimate when applied to closed-loop data [30]. Van Overschee and De Moor(1997) [33] proposed an N4SID based method for closed-loop subspace identification which requires knowledge of the first i impulse response coefficients of the controller, where i is the maximum order of the state space model to be identified. Through analysis of closed-loop subspace methods, Ljung and McKelvey(1996) [30] presented a method for the identification of subspace matrices from closedloop data using estimated predictors according to ARX models. MOESP and CVA approaches are also proposed for the identification of state space models using closed-loop data [57, 58, 59, 60, 61, 62, 63, 83]. Another class of subspace system identification is called the instrument variable methods [34, 84, 72], discussed in Chapter 3. In the class of instrument variable methods, the effect of disturbances is eliminated by appropriate selection of instrument variables that are independent of the disturbances. Under the framework of MOESP, Chou and Verhaegen (1997) [34] developed an instrument variable subspace identification algorithm as discussed in Chapter 3. It is claimed that the algorithm works for closed-loop systems provided there is at least one sample time delay in the controller, which can be restrictive in practice. Aiming at solving the open-loop error-in-variable (EIV) identification problem, Wang and Qin (2002) [72] developed an instrument variable subspace identification method via principal component analysis, but it may also deliver a bias in closed-loop identification. Using the subspace EIV model structure of [72], in this chapter we will develop a novel closed-loop subspace identification algorithm through an orthogonal subspace projection, and then use either the extended observability matrix/lower block-Toeplitz matrix or the Kalman filter states resulting from the projection, to extract system models. The models are obtained using projections of subspace matrices, as is the case in most other subspace identification algorithms, and has certain additional properties compared to the instrument variable methods. This chapter also provides an analysis of a bias problem in subspace closedloop identification and subsequently discusses a solution to this problem. The remainder of this chapter is organized as follows. An overview of the existing closed-loop subspace identification methods is presented in Section 4.2. The main results, a novel subspace closed-loop identification approach and a solution to eliminate the bias under closed-loop conditions, are discussed in Section 4.3. Simulation results are presented in Section 4.3.5, followed by summary in Section 4.4.
4.2 Review of Closed-loop Subspace Identification Methods
57
4.2 Review of Closed-loop Subspace Identification Methods Figure 4.1 shows a general representation of process under feedback control, referred to as closed-loop system. Closed-loop data cannot be used with the open-loop subspace methods for model identification because of the correlations between ut and et . Several approaches have been used to circumvent this problem in the literature. The various closed-loop subspace identification approaches will be reviewed in this section.
et
wt Gl
εt
rt -
uε ,t Gc1
ut
Gp
yt
Gc 2 Fig. 4.1. Closed-loop system for subspace identification
4.2.1
N4SID Approach
The approach proposed in [33] makes use of only the process input-output data to identify the model Gp . In this framework, Gc1 = 1; Gc2 = Gc ; wt = 0 and rt does not have to be measured, and the knowledge of the first few impulse responses (Markov parameters) of the controller need to be known. The controller equations can be written as xct+1 = Ac xct + Bc yt ut = rt − Cc xct − Dc yt
(4.1) (4.2)
where { Ac , Bc , Cc , Dc } are the controller’s system matrices, and xct is the controller state. The controller cannot be unstable. Following the notations of data Hankel matrices as defined in Chapter 3, certain pseudo data matrices are constructed from the input and output data Hankel matrices using the knowledge of the controller Markov parameters:
c Y0|2N −1 Np|q = U0|2N −1 + H2N
c Mp|q = Up|q + Hp−q+1 Yp|q
(4.3) (4.4)
58
4 Closed-loop Subspace Identification
c where 0 ≤ p ≤ q ≤ 2N − 1, H2N is the lower triangular block-Toeplitz matrix for the controller. Np|q and Mp|q can be shown to be uncorrelated with the disturbances [33]. On the assumptions that:
1. 2. 3. 4.
rt is uncorrelated with the disturbances; the matrix N0|2N −1 has a full row rank 2mN ; j −→ ∞; the closed-loop problem is well posed, i.e., (I + DDc ) is invertible;
then it is shown [33] that ⎛
U0|N −1
⎞
⎟ ⎜ ⎟ ⎜ Z = YN |2N −1 / ⎜ Y0|N −1 ⎟ ⎠ ⎝ MN |2N −1 ⎛ ⎞ N0|2N −1 ⎠ = YN |2N −1 / ⎝ Y0|N −1
ˆ f + H d MN |2N −1 ] = TN [ΓN X N
where
d c −1 HN ) TN = (I + HN
Therefore, to obtain the state estimation apply the following oblique projection and SVD: ⎛ ⎞ U0|N −1 ⎠ O = YN |2N −1 /MN |2N −1 ⎝ (4.5) Y0|N −1 ˆf = TN ΓN X =
(4.6)
⎛
⎞⎛
⎞
Σ1 0 VT ⎠⎝ 1 ⎠ U1 U2 ⎝ 0 0 V2T
(4.7)
where state order = rank(O). Then it can be seen that 1/2
TN ΓN = U1 Σ1 ˆ f = Σ 1/2 V1T X 1 States sequences can therefore be extracted. A state space model for the process is subsequently identified from the estimated states. However, it is pointed by [33] that the state sequence estimated in this way will give a slightly biased result for extracting system matrices; a better algorithm has also been proposed, which is obtained through a more complex procedure. In [33] there exist two algorithms of calculating state space matrices according to the estimated states and another algorithm according to ΓN . The limitation of this method is the requirement of the knowledge of the controller. In practice, accurate knowledge of the impulse response (IR) coefficients/Markov parameters of the controller may not always be available.
4.2 Review of Closed-loop Subspace Identification Methods
4.2.2
59
Joint Input-Output Approach
The overall strategy of these methods is similar to the joint input-output identification strategy familiar in conventional closed-loop system identification. These methods do not require the knowledge of the controller. Apart from setpoint excitation, these approaches [58, 85, 83, 86] need an additional external excitation, wt , added to the controller output, uε,t , to make the process input independent of noise, as shown in Figure 4.1, where Gc1 = Gc and Gc2 = 1; rt is the setpoint that is a white noise sequence; et is the white noise (disturbance) added to the process output, yt , through a noise model, Gl . Then the measurable input T T vector is wtT rtT and measurable output vector is uTε,t εTt uTt ytT , where uε,t = Gc εt = ut − wt and εt = rt − yt . Using this information a global state space model is first identified using the MOESP algorithm. The global state space model is denoted as:
wt (4.8) xt+1 = Axt + B1 B2 rt ⎛ ⎞ ⎛ ⎞ ⎞ ⎛ C1 uε,t D11 D12 ⎜ ⎟ ⎜ ⎟ ⎟ ⎜ ⎜ εt ⎟ ⎜ C2 ⎟ ⎜ D D ⎟ wt (4.9) ⎜ ⎟ = ⎜ ⎟ xt + ⎜ 21 22 ⎟ ⎝ ut ⎠ ⎝ C3 ⎠ ⎝ D31 D32 ⎠ rt yt C4 D41 D42 xt has an order n + nc , where n is the order of the process, Gp , and nc is the order of the controller, Gc . The state space representation for the transfer functions Hw→y and Hw→u is given by the system matrices [A, B1 , C4 , D41 ] and [A, B1 , C3 , D31 ] respectively. Using the rules for concatenating and inverting state space models, the process model in the state space form, Gsp , and the controller model in the state space form, Gsc , are obtained as: Gsp = [A, B1 , C4 , D41 ][A, B1 , C3 , D31 ]−1
and
−1 −1 −1 −1 = [A, B1 , C4 , D41 ][A − B1 D31 C3 , B1 D31 , −D31 C3 , D31 ] ⎞ ⎛ ⎞ ⎞ ⎛⎛ −1 −1 B1 D31 A −B1 D31 C3 −1 ⎠ ⎠,⎝ ⎠ , C4 −D41 D−1 C3 , D41 D31 = ⎝⎝ 31 −1 −1 0 A − B1 D31 D3 B1 D31
Gsc = [A, B2 , C3 , D32 ][A, B2 , C2 , D22 ]−1 ⎞ ⎛ ⎞ ⎞ ⎛⎛ −1 −1 B2 D22 A −B2 D22 C2 −1 ⎠ ⎠,⎝ ⎠ , C3 −D32 D−1 C2 , D32 D22 = ⎝⎝ 22 −1 −1 0 A − B2 D22 D2 B2 D22
The overall deterministic state space model is first identified. The order selection in this step needs the specification of the sum of process model order and
60
4 Closed-loop Subspace Identification
controller model order [83]. The individual plant and controller model orders are determined/selected in the subsequent model reduction step. The approach of [86] based on CVA uses practically the same approach as MOESP except that the CVA algorithm is used to identify the state space matrices of the global closed-loop system, with AIC used in the selection of the state order. The state space representation is then converted to the transfer function representation, followed by the concatenation and inversion of the closed-loop transfer functions to obtain the open-loop process transfer function. 4.2.3
ARX Prediction Approach
Ljung and McKelvey (1996) [30] provide an insight into the problem of using the conventional subspace methods for closed-loop identification, and then demonstrate a feasible approach for closed-loop subspace identification. Consider an impulse response model for the process with input ut and noise et yt =
∞
Fku ut−k +
k=0
∞
Fke et−k
(4.10)
k=0
of which a predictor without considering future control actions can be extracted as yˆ(t + j|t) =
∞
Fku ut+j−k +
k=j
∞
Fke et+j−k
(4.11)
k=j
Define a prediction vector: ⎞ yˆ(t + 1|t) ⎟ ⎜ .. ⎟ ⎜ Yr (t) = ⎜ ⎟ . ⎠ ⎝ yˆ(t + r|t) ⎛
Then it can be shown [30] that the system order n is the rank of Yr (t) where r ≥ n. The Kalman filter state can be reconstructed by picking up a basis from Yr (t), namely xt = LYr (t) The prediction yˆ(t + j|t) can be solved through the following equation that is derived directly from Equation 4.10 by replacing time subscript t with t + j: yt+j =
∞
Fku ut+j−k
k=0
+
∞
Fke et+j−k
(4.12)
k=0
Equation 4.12 can be transferred to: yt+j =
j−1
Fku ut+j−k +
k=0
=
j−1
k=0
Fku ut+j−k +
∞
u Fk+j ut−k +
k=0 ∞
k=0
u Fk+j ut−k +
j−1
Fke et+j−k +
k=0 j−1
k=0
Fke et+j−k +
∞
e Fj+k et−k
k=0 ∞
k=0
e F˜j,k yt−k
(4.13)
4.2 Review of Closed-loop Subspace Identification Methods
61
The last term of Equation 4.13 is obtained by replacing es with ys for s ≤ t since there is one to one correspondence between the innovation es and the output ys [30]. By truncating infinite series into finite ones, Equation 4.13 is approximated by yt+j =
j−1
k=0
Fku ut+j−k +
nb
k=0
u Fk+j ut−k +
na
e F˜j,k yt−k + t+j
(4.14)
k=0
j−1 where t+j = k=0 Fke et+j−k . By removing the first term and the last term on the right hand side of Equation 4.14, we get an approximate prediction without considering the control actions and noise after time t. Thus the prediction defined by Equation 4.11 can be approximated by the middle two terms on the right hand side of Equation 4.14 by noting the correspondence between the innovation et and the output yt . As a result, the key to reconstructing the state is to estimate Equation 4.14. It is shown by [30] that the estimation of Equation 4.14 is consistent if and only if et+j , . . . , et+1 and ut+j , . . . , ut+1 are independent. This condition will not hold if there is feedback control. However, if there is one sample time delay in the system and j = 1 (one step prediction), according to [30] estimation of Equation 4.14 will be consistent despite of the feedback control. Note that, with j = 1, Equation 4.14 becomes an ARX model. Consequently, to circumvent the inconsistency one can recursively use one-step prediction to derive two-step prediction, then three-step, until r-step prediction. In summary, the various steps of this approach are: 1. find yˆ(t + j|t) for j = 1, ..., r by recursively using the one-step prediction of the ARX model; 2. form Yr (t) for t = 1, ..., N and estimate its rank n; 3. select L to obtain a well conditioned basis and x(t) = LYr (t), for t = 1, ..., N ; 4. find the matrices {A, B, C, D, K} by using the estimated states. Note that the states are identified from the estimated predictors and not directly from the data, unlike the previously illustrated approaches. The authors of [30] state that this method is only a ‘feasible’ method rather than the ‘best way’ of identifying systems operating in closed-loop. 4.2.4
An Innovation Estimation Approach
According to Equation 3.42, Equation 3.41 can be written further as the following form: Yf = ΓN Lp Wp + Lu Uf + Le Ef (4.15) There are N block rows in Equation 3.41. Qin and his coworkers [35, 44, 45] proposed a method that partitions Equation 3.41 into N block rows and used the estimated innovations from previous rows to further estimate model parameters of the next row sequentially.
62
4 Closed-loop Subspace Identification
The ith block row equation can be written as Yf i = ΓN i Lp Wp + Lui Uf i + L− ei Ef,i−1 + Ei
(4.16)
where Yf i is the ith block row of Yf , ΓN i the ith block row of ΓN , Lui the ith block row of Lu by excluding the last j − i block columns (they are actually zeros), L− ei the ith block row of Le by excluding the last j − i + 1 block columns, Uf i the first i block rows of Uf , Ef,i−1 the first i − 1 block rows of Ef , and Ei the ith block row of Ef . The above structure is due to the lower triangular nature of Lu and Le . For i = 1, Equation 4.16 can be written as Yf 1 = ΓN 1 Lp Wp + Lu1 Uf 1 + E1
(4.17)
E1 is uncorrelated with Wp since it is future white noise disturbance relative to Wp . Consider that there is a zero-order-hold in the system, i.e, D = 0; thus Lu1 = D = 0. Equation 4.17 is simplified to Yf 1 = ΓN 1 Lp Wp + E1 Hence ΓN 1 Lp can be estimated as ΓN 1 Lp = Yf 1 Wp† The innovation E1 can then be estimated as E1 = Yf 1 − ΓN 1 Lp Wp For i = 2, . . . , N , Yf i = ΓN i Lp Wp + Lui Uf i + L− ei Ef,i−1 + Ei where
⎛
Ef,i−1
E1
(4.18)
⎞
⎟ ⎜ ⎟ ⎜ ⎜ E2 ⎟ ⎟ =⎜ ⎟ ⎜ .. ⎟ ⎜. ⎠ ⎝ Ei−1
has been estimated from the previous steps. If there is at least one sample time delay in the controller, then Ei is uncorrelated with Uf i . It follows from Equation 4.18 that ⎛ ⎞ †
ΓN i Lp Lui
W ⎜ p ⎟ ⎜ ⎟ − = Y ⎜ ⎟ Lei U fi ⎝ fi ⎠ Ef,i−1
4.3 An Orthogonal Projection Approach
63
Assembling all ΓN i Lp for i = 1, 2, . . . , N yields ⎛
ΓN 1 Lp
⎞
⎜ ⎟ ⎜ ⎟ ⎜ ΓN 2 Lp ⎟ ⎟ ΓN Lp = ⎜ ⎜ .. ⎟ ⎜. ⎟ ⎝ ⎠ ΓN N Lp which is an estimate of Lw . The remaining steps to extract system matrices are the same as that given in Equations 3.45, 3.46, and 3.47.
4.3 An Orthogonal Projection Approach In this section, a closed-loop subspace identification procedure based on orthogonal projection [36] is presented. By detailed analysis of feedback effect on subspace projection, a source that introduces bias error to closed-loop subspace identification is identified, and a procedure is proposed for elimination of the bias error. The analysis presented in this section is informative for understanding the limitations of closed-loop subspace identification and for the path to the solutions. Thus this section is elaborated in detail. 4.3.1
A Solution through Orthogonal Projection
Open-loop subspace identification has been discussed in Chapter 3. One of the solutions, N4SID, is obtained through the oblique projection of the following equation: d s U f + HN Ef (4.19) Yf = ΓN Xf + HN Several terms on the right hand side of the equation become zero after the oblique projection. However, the situation becomes more complex for closed-loop identification where the future disturbance Ef is no longer independent of the future input Uf due to the feedback. The implication of this dependency is that the oblique projection of Ef along Uf on to Wp is no longer zero, although the orthogonal projection of Ef onto Wp is zero. This phenomenon can be observed from Figure 4.2, where the orthogonal projection of A to C is zero but the oblique projection of A to C via B is not zero. To circumvent this problem, by adopting the EIV structure of [72], we move the term related to Uf into the left hand side of Equation 4.19 as it would be a problematic term if it remained on the right hand side of the equation. This yields a new equation, very similar to the total least squares equation [87, 88] in the sense that both input and output variables are on the same side of the equation. ⎞ ⎛ Yf s d ⎠ = ΓN Xf + HN ⎝ Ef (4.20) I −HN Uf
64
4 Closed-loop Subspace Identification
B
A
A /B C
C
Fig. 4.2. Illustration of oblique projection under the closed-loop condition
Using a short-hand notation: ⎛
Wf = ⎝
Yf Uf
⎞ ⎠
Equation 4.20 can be simplified to s d Wf = ΓN Xf + HN Ef I −HN
(4.21)
Performing an orthogonal projection of Equation 4.21 on to the row space of Wp yields d W /W = Γ X /W + H s E /W (4.22) I −HN f p N f p p N f The last term of Equation 4.22 is an orthogonal projection of the future disturbance (white noise) on to the row space of past input and output matrix Wp , which is zero. Therefore, Equation 4.22 can be simplified to d ˆf Wf /Wp = ΓN Xf /Wp = ΓN X (4.23) I −HN The second equality is obtained from Equation 3.57.
Remark 4.1. Equation 4.23 is a natural result through the projection as is often done in subspace system identification literature. The orthogonal projection of Equation 4.21 on to the row space of Wp results in Equation 4.23, which includes a multiplication term between the extended observability matrix ΓN and Kalman ˆ f . Wang and Qin (2002) [72], on the other hand, used an instrument state X variable method to arrive at an equation as d W WT = Γ X WT (4.24) I −HN f N f p p
4.3 An Orthogonal Projection Approach
65
where the instrument variable is the past input and output Wp . This equation is derived by multiplying Equation 4.21 by WpT and noticing the independency between Wp and Ef . In [89], an alternative method through a column weighting is proposed aiming at reducing the variance error and providing a consistent estimation. The projection method may also be mathematically regarded as an instrument variable method where the instrument is WpT (Wp WpT )−1 Wp (since the orthogonal projection Wf /Wp = Wf WpT (Wp WpT )−1 Wp ). However, using the projection results in a multiplication term between the extended observability matrix and the Kalman filtered state, which provides some additional features and performance, as will be seen shortly. We shall call the projection method the subspace orthogonal projection identification method, abbreviated SOPIM, while the subspace identification method via PCA of Wang and Qin (2002) [72] is abbreviated as SIMPCA, in this chapter. Now multiplying both sides of Equation 4.23 by the orthogonal column space of ΓN , denoted by ΓN⊥ , yields d W /W = 0 (ΓN⊥ )T I −HN (4.25) f p
Denoting Z = Wf /Wp , the problem is transferred to finding the orthogonal column space of Z, which should equal the column space of T d . (ΓN⊥ )T I −HN Perform SVD decomposition of Z as ⎞ ⎛ ⎞⎛ Σ1 V1T ⎠ ⎠⎝ (4.26) Z = U1 U2 ⎝ 0 V2T where, in practice, Z is not singular and one has to determine its rank by checking its singular values. In theory, its rank should be lN + n [72] assuming that the external perturbation is persistent excitation. The rank determination is equivalent to the determination of system orders (see Section 3.7.2). With Equation 4.26, one can find the orthogonal column space of Z, which is U2 . Therefore T d = U2 M (4.27) (ΓN⊥ )T I −HN
where M ∈ R(mN −n)×(mN −n) is any full-rank constant matrix. Partitioning ⎛ ⎞ P1 U2 M = ⎝ ⎠ P2 then Equation 4.27 can be written as ⎛ ⎞ ⎛ ⎞ (ΓN⊥ ) P ⎝ ⎠= ⎝ 1⎠ d T ⊥ −(HN ) ΓN P2
(4.28)
66
4 Closed-loop Subspace Identification
Therefore,
and
ΓN = P1⊥
(4.29)
d −P2T = (ΓN⊥ )T HN
(4.30)
d , and then to extract the system The remaining problem is to solve for ΓN and HN d . The right hand side of Equation 4.30 has matrices A, B, C, D from ΓN and HN the same format as that of Equation 3.64. The procedure thus discussed in Section 3.5.3 for extracting A, B, C, D can be applied. d depend only Remark 4.2. As can be seen from the above derivation, ΓN and HN on the last few left singular vectors of Z. The left singular vectors of Z can also be calculated from eigen decomposition of ZZ T as ⎛ ⎞ ⎞⎛ Σ2 U1T 1 T ⎠ ⎠⎝ (4.31) ZZ = U1 U2 ⎝ 0 U2T
ZZ T has a dimension of (mN + lN ) × (mN + lN ) while Z has a dimension of (mN + lN ) × j. As typically j >> (mN + lN ), the decomposition of ZZ T is much simpler than that of Z. Therefore, we recommend calculating U2 from the eigen decomposition of ZZ T . U2 is the eigenvectors that correspond to the minor eigenvalues of ZZ T (zero eigenvalues if there is no disturbance). However, according to [90], squaring the singular values in the eigen decomposition method may lead to poor accuracy in the smallest singular values, which is the tradeoff of using the eigen decomposition approach. Remark 4.3. Subspace identification algorithms draw a lot of their strength from the use of QR decomposition for the projections, where in general Q is never calculated. Such a QR decomposition is known to speed up the projection by up to several orders of magnitude. The approach of using QR decomposition for solving this problem is presented below. Perform a QR decomposition ⎞ ⎛ ⎞⎛ ⎞ ⎛ R11 0 QT1 Wp ⎠=⎝ ⎠⎝ ⎠ ⎝ R21 R22 Wf QT2 Then the orthogonal projection can be calculated via the following equation: −1 Z = Wf /Wp = R21 R11 Wp
(4.32)
Can SOPIM and SIMPCA work under closed-loop conditions? Up to now, it appears that there should be no question but that both work under closed-loop conditions as the input Uf , the correlation of which with the future disturbance Ef is the key problem in closed-loop identification, is not involved in the projection. Both Equations 4.23 and 4.24 appear to be able to determine the process model uniquely irrespective of open or closed-loop. However, simulation results indicate that this is not the case, although both work for open-loop systems.
4.3 An Orthogonal Projection Approach
4.3.2
67
The Problem of Biased Estimation and the Solution
The problem must have resulted from the controller since both algorithms work fine under the open-loop condition. To diagnose the problem, let’s consider that the controller is described by the following state space model: xct+1 = Ac xct + Bc (rt − yt )
(4.33)
ut = Cc xct + Dc (rt − yt )
(4.34)
where r is the setpoint excitation. Using subspace notations, we should have the controller expressed as c Uf = ΓNc Xfc + HN (Rf − Yf ) c c c Up = ΓN Xp + HN (Rp − Yp )
(4.35) (4.36)
where, following the similar notations of Section 3.2.2, Rp and Rf are data Hankel matrices of the setpoint, Xpc and Xfc are the state matrices of the controller, c ΓNc is the extended observability matrix, and HN is the lower triangular blockToeplitz matrix. Equation 4.35 can be re-arranged to give c c c c (4.37) HN I Wf = ΓN Xf + HN Rf Combining Equations 4.21 and 4.37 gives ⎞ ⎛
d s I −HN Γ X H E N f f N ⎠ Wf = ⎝ + c c ΓNc Xfc HN Rf HN I
(4.38)
Pre-multiplying Equation 4.38 by
ΓN⊥
ΓNc⊥
T
to remove the state variables yields
ΓN⊥
ΓNc⊥
T
d I −HN c HN I
Wf =
s (ΓN⊥ )T HN
c (ΓNc⊥ )T HN
Ef Rf
(4.39)
In order to remove the effect of noise, post-multiply Equation 4.39 by an instrument variable W T ⎞ ⎛ ⎞T ⎛ ⎞⎛ ⎞ ⎛ d ⊥ T s ΓN⊥ I −HN E (Γ ) H N ⎠ Wf W T = ⎝ N ⎝ ⎠ ⎝ ⎠⎝ f ⎠WT c⊥ c c⊥ T c ΓN HN I (ΓN ) HN Rf (4.40)
68
4 Closed-loop Subspace Identification
where WT =
⎧ ⎨WT
for SIMPCA p ⎩ W T (W W T )−1 W for SOPIM p p p p
(4.41)
For the sake of rigor in the following derivation, multiply Equation 4.40 by and this results in ⎛ ⎞T ⎛ ⎞ d I −HN 1 ⎝ ΓN⊥ ⎠ ⎝ ⎠ Wf W T c j ΓNc⊥ HN I ⎛ ⎞⎛ ⎞ s Ef 1 ⎝ (ΓN⊥ )T HN ⎠⎝ ⎠WT = c j (ΓNc⊥ )T HN Rf
1 j
(4.42)
Let’s first consider SIMPCA where W = Wp . Since the rows of Ef are independent of the rows of Wp , 1j Ef WpT → 0 as j → ∞ and the first block row on the d , right hand side of Equation 4.42 will be zero. Consequently (ΓN⊥ )T I −HN which contains the essential information of the process model, must fall in the subspace orthogonal to the column space of Wf WpT and, therefore, process models can be recovered from the SVD decomposition of Wf WpT , as discussed before. However, if the rows of Rf are also independent of the rows of Wp , e.g., in the case of white noise, then 1j Rf WpT → 0 as j → ∞, and the second block row on the c right hand side of Equation 4.42 will also be zero. This means (ΓNc⊥ )T HN I , which contains the essential information of the controller model, is also part of the subspace orthogonal to the column space of Wf WpT . As a consequence, the subspace orthogonal to the column space of Wf WpT , which is used to identify the process model, contains both the process model relation and the controller relation. By merely selecting this subspace to extract a model, it is likely that the controller relation is also selected in addition to the process model relation, and the SVD decomposition is not able to tell which model, i.e., a process, controller or mixed model, has been selected. In this case the process identifiability is lost. In order to guarantee that the left null space of Wf W T contains only the process model subspace, the instrument variable W must satisfy the following conditions: 1) 1j Ef W T = 0 2) 1j Rf W T = 0 as j → ∞. A good choice for such an instrument in the case of SIMPCA should be ⎞ ⎛ Rf ⎠ (4.43) W = Wpr = ⎝ Wp
4.3 An Orthogonal Projection Approach
69
i.e., the instrument variable should include Rf , since multiplication of Rf by its transpose guarantees the product to be non-zero. The same applies to SOPIM. In this case, rather than projecting to Wp , we should project Equation 4.21 on to Wpr . This yields d W /W ˆ (4.44) I −HN f pr = ΓN Xf /Wpr = ΓN Xf This new projection guarantees Rf /Wpr = 0 in the projection of Equation 4.37 on to Wpr , and therefore eliminates the overlap between the process model subspace (Equation 4.44) and controller subspace (by projecting Equation 4.37 on to Wpr ), which is c c c c (4.45) HN I Wf /Wpr = ΓN Xf /Wpr + HN Rf /Wpr
where the last term of the equation is guaranteed to be non-zero. With the modification, all the computation procedures discussed in the last section are valid for closed-loop identification after replacing Wp by Wpr , and then both SOPIM and SIMPCA can be truly applied to closed-loop data. We shall call the modified algorithms closed-loop SOPIM (abbreviated as CSOPIM) and closed-loop SIMPCA (abbreviated as CSIMPCA), respectively. To summarize, for open-loop identification, the instrument variable could be chosen according to Equation 4.41, while for closed-loop identification, the instrument variable must be chosen according to Equation 4.46 ⎧ ⎨WT for CSIMPCA pr (4.46) WT = ⎩ W T (W W T )−1 W for CSOPIM pr
pr
pr
pr
Of course, the closed-loop algorithms are more general and also applicable to open-loop identification.
Remark 4.4. From the above discussion, one can see that the undesired effect of the feedback control on the identifiability of open-loop instrument and/or projection subspace methods may also be alleviated if the setpoint rt “stays away” from whiteness, i.e., if they are (strongly) autocorrelated or colored. But this “non-whiteness” is ambiguous and there is no a priori indication on how Rf /Wp or Rf WpT will be different from zero even if rt is not white noise. In addition, the effect of whiteness or non-whiteness on the identifiability also depends on the controller in the feedback and the disturbances that are affecting the process. For example, if the output cannot follow the setpoint closely due to large disturbances, then Wp can have a little correlation with Rf even if the setpoint is not white, resulting 1j Rf WpT → 0. Consequently, Wp may not be suitable to be an instrument in this case even though the external excitation is non-white. 4.3.3
Model Extraction through Kalman Filter State Sequence
In this subsection, we shall develop an alternative method for both open-loop and closed-loop identification through the estimated Kalman filter state sequences.
70
4 Closed-loop Subspace Identification
Combine Equations 4.23 and 4.44 as d ˆf Z = ΓN X I −HN
where
⎧ ⎨ W /W for SOPIM f p Z= ⎩ W /W for CSOPIM f pr
(4.47)
(4.48)
d Z should It is easy to see from Equation 4.47 that the row space of I −HN ˆ span the row space of Xf . Therefore, up to similarity transformation, the Kalman state can be calculated, for example, from ˆ f = Γ † I −H d Z X (4.49) N N With the availability of the state sequence, the system matrices may be extracted using several methods [67]. It has been discussed in Chapter 3 that
XN +1 AB XN K (4.50) = + EN |N YN |N CD UN |N I ˆ f ). To calXN can be directly calculated from Equation 4.47 (i.e., XN ← X culate XN +1 , one can simply change data Hankel matrices according to Rf = RN +1|2N −1 Up = U0|N Yp = Y0|N Uf = UN +1|2N −1 Yf = YN +1|2N −1 where, again, the subscripts follow the notations of Equations 3.23-3.30. Z is changed accordingly. Then XN +1 can be calculated as d XN +1 = Γ −1 Z (4.51) I −H N N
where Γ N is obtained by eliminating the last m rows of ΓN , and H dN is obtained d by eliminating the last m rows and the last l columns of HN . With the availability of XN , XN +1 , all parameter matrices, A, B, C, D, K can be estimated by performing least squares on Equation 4.50. We shall call this method the subspace orthogonal projection identification method via the state estimation for model extraction, abbreviated SOPIM-S. However, SOPIM-S may not be applicable to closed-loop data even though the state has been estimated. The main problem is Equation 4.50. Depending on the feedback control law and the strength of the external excitation, UN |N can be highly correlated with XN . A bias in the estimation of the parameters
4.3 An Orthogonal Projection Approach
71
from Equation 4.50 will therefore be expected. The remedy to this problem is through the following three-step procedure: Step 1: First estimate A from ΓN through the following equation: Γ¯N = Γ N A
(4.52)
where Γ¯N is ΓN by removing its first m rows, and Γ N is ΓN by removing its last m rows. Step 2: Assuming that there is at least one sample time delay in the process, i.e., D = 0, then C can be estimated from the following equation through regression analysis: (4.53) YN |N = CXN + EN |N The residual EN |N can be calculated from this equation. Step 3: B and K can be estimated from the following equation: ⎞ ⎛ U N |N ⎠ XN +1 − AXN = B K ⎝ EN |N
(4.54)
We shall call this modified method the closed-loop subspace orthogonal projection identification method via the state estimation for model extraction, abbreviated CSOPIM-S. Model estimation algorithms based on Kalman filter state, SOPIM-S and CSOPIM-S, appear to be more cumbersome than the algorithms that directly d matrices, such as SOPIM and CSOPIM. Howextract models from ΓN and HN ever, SOPIM-S and CSOPIM-S also calculate disturbance model parameters (or Kalman gain matrix K) directly, while SOPIM and CSOPIM need additional steps to extract K. 4.3.4
Extension to Error-in-variable (EIV) Systems
The derived closed-loop identification algorithm can also be directly applied to closed-loop identification of error in variable (EIV) systems with the following input-output measurements: u∗t = ut + vt yt∗ = yt + st
(4.55) (4.56)
where ut and yt are given by the state space equations (3.1) - (3.2); u∗t and yt∗ are measured input and output respectively; vt and st are white noise. Let the future data Hankel matrices of vt and st be written as Vf and Sf , and the future data Hankel matrices of yt∗ and u∗t be written as Yf∗ and Uf∗ . Define short-hand notations ⎛ ⎞ Vf Df = ⎝ ⎠ Sf
72
4 Closed-loop Subspace Identification
⎛
Wf = ⎝
and
⎛
Wf∗ = ⎝
Yf Uf Yf∗ Uf∗
⎞ ⎠
⎞ ⎠
Using these short-hand notations, Equations 4.55 and 4.56 can be written as Wf∗ = Wf + Df
(4.57)
Then for the EIV systems, it follows from Equations 4.21 and 4.57 that d (W ∗ − D ) = Γ X + H s E (4.58) I −HN f N f f N f or
Now define
s d d D Wf∗ = ΓN Xf + HN Ef + I −HN I −HN f ⎛
Rf
(4.59)
⎞
⎟ ⎜ ⎟ ⎜ ∗ Wpr = ⎜ Yf∗ ⎟ ⎠ ⎝ Uf∗
∗ Projecting Equation 4.59 orthogonally on to Wpr yields
d W ∗ /W ∗ = Γ X /W ∗ + H s E /W ∗ + d D /W ∗ (4.60) I −H I −HN N f f f f pr pr N pr pr N
∗ Following the same argument made before, Ef /Wpr = 0. Since vt and st are ∗ white noise, Df /Wpr = 0. Therefore,
∗ ∗ d ˆf Wf∗ /Wpr = ΓN Xf /Wpr = ΓN X I −HN
(4.61)
ˆ f = Xf /W ∗ . Solving Equation 4.61 through the singular value decomwhere X pr position procedure or the Kalman filter state estimation method as we did for d the regular systems yields HN , ΓN and the Kalman filter state sequence for the EIV systems, and the system matrices can be extracted subsequently. 4.3.5
Simulation
In this section, we will use a benchmark problem to evaluate the orthogonal projection approach and compare it with other existing subspace identification algorithms. We will apply the following representative subspace algorithms in the literature: the closed-loop algorithm by Van Overschee and De Moor (1996,1997)
4.3 An Orthogonal Projection Approach
73
[32, 33], the closed-loop algorithm by Verhaegen (1993)[83], the closed-loop algorithm by Ljung and McKelvey(1996) [30], two classical subspace algorithms, N4SID [67] and MOESP [91], and two versions of CVA, i.e., MATLAB N4SID with CVA weighting and CVA according to [60]. To comply with the standard practice in subspace identification literature [32, 33], we will perform MonteCarlo simulations, and the averaged Bode magnitude plot from the Monte-Carlo simulations will be used to represent bias error, while the scatter plot of estimated poles will be used to represent the variance error of the estimation. The system to be considered was first presented by Verhaegen (1993) [83], and was used again as a benchmark problem by Van Overschee and De Moor (1996,1997) [32, 33] for the comparison of closed-loop subspace identification algorithms. The block diagram of the original system is shown in Figure 4.3. The model, expressed in the innovation state space form [32], is given by Equations 3.1 and 3.2 with the following numerical values: ⎛
4.40 1 ⎜ ⎜ −8.09 0 ⎜ A = ⎜ 7.83 0 ⎜ ⎝ −4.00 0 0.86 0
0 1 0 0 0
0 0 1 0 0
⎞ ⎛ ⎞ ⎛ ⎛ ⎞ ⎞ 2.3 0.00098 1 0 ⎟ ⎜ ⎟ ⎜ ⎜ ⎟ ⎟ ⎜ −6.64 ⎟ ⎜ 0.01299 ⎟ ⎜0⎟ 0⎟ ⎟ ⎜ ⎟ T ⎜ ⎟ ⎜ ⎟ 0 ⎟ , B = ⎜ 0.01859 ⎟ , C = ⎜ 0 ⎟ , K = ⎜ 7.515 ⎟ ⎟ ⎜ ⎟ ⎜ ⎜ ⎟ ⎟ ⎝ −4.0146 ⎠ ⎝ 0.0033 ⎠ ⎝0⎠ 1⎠ 0.86336 −0.00002 0 0
The state space model of the feedback control has the following values: ⎛
⎞ ⎛ ⎞ ⎛ ⎞ 2.65 −3.11 1.75 −0.39 1 −0.4135 ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ 1 ⎜0⎟ ⎜ 0.8629 ⎟ 0 0 0 ⎟ Ac = ⎜ ⎟ , Bc = ⎜ ⎟ , CcT = ⎜ ⎟ ⎝ 0 ⎝0⎠ ⎝ −0.7625 ⎠ 1 0 0 ⎠ 0 0 1 0 0 0.2521 with Dc = 0.61. The simulation conditions are exactly the same as those used by Van Overschee and De Moor (1996) [32]: et is a Gaussian white noise sequence with variance 1/9; the reference signal rt is a Gaussian white noise sequence with variance 1. Each simulation generates 1200 data points. We generate 100 simulated data sets, each with the same reference input rt but with a different noise sequence et . Van Overschee and De Moor (1996,1997) [32, 33] used this example to compare several closed-loop identification algorithms, including their proposed one (denoted here as Algorithm Van Overschee and De Moor)1 , the closed-loop algorithm by Verhaegen (1993) [83] (denoted here as Algorithm Verhaegen), the closed-loop algorithm by Ljung and McKelvey(1996) [30] (denoted here as Algorithm Ljung and McKelvey), and the classical N4SID algorithm. The outcome 1
In [32, 33], three algorithms are proposed. Algorithm 1, the best one among the three, is used in this chapter for the comparison.
74
4 Closed-loop Subspace Identification
w s y
u
r
Gp
-
Gc Fig. 4.3. Block diagram of the process example used by van Overschee and De Moor
CSOPIM Imag Part
dB
100
0
0.5 0 −0.5
−1
10
0
10
1
10
Van Overschee−De Moor Imag Part
dB
−100 −2 10 100
0
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5 0 −0.5
−1
10
0
10
1
10
Verhaegen Imag Part
dB
−100 −2 10 100
0
0.5 0 −0.5
−1
10
0
10
1
10
Ljung−McKelvey Imag Part
dB
−100 −2 10 100
0
0.5 0 −0.5
−100 −2 10
−1 10 rad/sec
0
10
1
10
Fig. 4.4. Closed-loop Monte-Carlo simulations. The left column is Bode magnitude plots; the dotted lines are the true values and the solid lines are the estimated values averaged from 100 runs. The right column is the scatter plots of the eigenvalues of the estimated A matrix.
of the comparison by Van Overschee and De Moor (1996,1997) [32, 33] was that algorithm 1 of Van Overschee and De Moor and algorithm Verhaegen gave an unbiased estimate under closed-loop conditions, while all other algorithms gave a certain bias. Although algorithm Verhaegen gave an unbiased estimate, the variance of the estimated poles was significantly larger than other algorithms due to the fact that the high-order closed-loop model has to be fitted first in
4.3 An Orthogonal Projection Approach
CSOPIM, meas. variance=0.2 Imag Part
dB
100
0
75
0.5 0 −0.5
−1
10
0
10
1
10
Van Overschee−De Moor, meas. variance=0.2
Imag Part
dB
−100 −2 10 100
0
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5 0 −0.5
−1
10
0
10
1
10
CSOPIM, meas. variance=0.5 Imag Part
dB
−100 −2 10 100
0
0.5 0 −0.5
−1
10
0
10
1
10
Van Overschee−De Moor, meas. variance=0.5
Imag Part
dB
−100 −2 10 100
0
0.5 0 −0.5
−100 −2 10
−1
10
0
rad/sec 10
1
10
Fig. 4.5. Closed-loop Monte-Carlo simulations for the EIV system. The left column is Bode magnitude plots; the dotted lines are the true values and the solid lines are the estimated values averaged from 100 runs. The right column is the scatter plots of the eigenvalues of the estimated A matrix.
the algorithm. Therefore, the conclusion was that algorithm 1 of Van Overschee and De Moor is superior to other algorithms in terms of performance. However, the algorithms proposed in Van Overschee and De Moor (1996,1997) [32, 33] required precise information of at least first i Markov parameters of the controller model, while others did not. In this simulation, we will reproduce the results of Van Overschee and De Moor (1996,1997) [32, 33] and compare them with the most representative algorithm of this chapter, CSOPIM. The simulation results are shown in Figure 4.4. From this figure, we can see that the algorithm CSOPIM has almost identical performance as that of algorithm Van Overschee and De Moor in both bias and variance aspects. However, the algorithm CSOPIM has the advantage over algorithm Van Overschee and De Moor in the sense that it does not need any knowledge about the controller model while algorithm Van Overschee and De Moor does. To see the further advantage of the algorithm CSOPIM, we consider an EIV case by adding white noises to the measurements of both ut and yt . We perform two Monte-Carlo simulations for the EIV case, one with measurement noise variance 0.2 and the other 0.5. The comparison results are shown in Figure 4.5. From this
4 Closed-loop Subspace Identification
dB
100
N4SID
Imag Part
76
0
0.5 0 −0.5
−1
0
10
10
1
10
MOESP Imag Part
dB
−100 −2 10 100
0
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5 0 −0.5
−1
0
10
10
1
10
CVA−MATLAB Imag Part
dB
−100 −2 10 100
0
0.5 0 −0.5
−1
0
10
10
1
10
CVA Imag Part
dB
−100 −2 10 100
0
0.5 0 −0.5
−100 −2 10
−1
10 rad/sec
0
10
1
10
Fig. 4.6. Closed-loop Monte-Carlo simulations. The left column is Bode magnitude plots; the dotted lines are the true values and the solid lines are the estimated values averaged from 100 runs. The right column is the scatter plots of the eigenvalues of the estimated A matrix.
figure, one can see that the algorithm CSOPIM performs better than algorithm Van Overschee and De Moor in the presence of measurement noises. To appreciate closed-loop subspace identification algorithms, we have also applied classical subspace algorithms, N4SID, MOESP, and two CVAs (CVA according to [60] and MATLAB N4SID with CVA weighting) to closed-loop data. The results are shown in Figure 4.6. From these results, one can see that N4SID, MOESP, and MATLAB based CVA deliver essentially the same performance and all are biased in the presence of feedback control. The CVA programmed according to [60] gives some improved performance compared to the MATLAB N4SID with CVA weighting but the bias is also observed in this example. Since the algorithm CSOPIM is a closed-loop identification algorithm, it must be applicable to open-loop identification. A natural question is how it performs in open-loop identification. Thus, we perform open-loop Monte-Carlo simulations using the open-loop plant without feedback control. The results together with the comparisons with other classical subspace identification algorithms are shown in Figure 4.7. From this figure, one can see that all classical algorithms
4.3 An Orthogonal Projection Approach
CSOPIM Imag Part
dB
100
0
77
0.5 0 −0.5
−1
0
10
10
1
10
N4SID Imag Part
dB
−100 −2 10 100
0
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5
Real Part 1
1.5
0.5 0 −0.5
−1
10
0
10
1
10
MOESP Imag Part
dB
−100 −2 10 100
0
0.5 0 −0.5
−1
10
0
10
1
10
CVA Imag Part
dB
−100 −2 10 100
0
0.5 0 −0.5
−100 −2 10
−1
10 rad/sec
0
10
1
10
Fig. 4.7. Open-loop Monte-Carlo simulations. The left column is Bode magnitude plots; the dotted lines are the true values and the solid lines are the estimated values averaged from 100 runs. The right column is the scatter plots of the eigenvalues of the estimated A matrix.
deliver similar performance for open-loop identification in this example, and the algorithm CSOPIM delivers an essentially identical performance to that of the classical algorithms in both bias and variance errors. Finally, we verify the claim that one of the instrument-variable based subspace identification methods, SIMPCA, can deliver a bias estimate for closed-loop identification at least under the condition that the external excitation is white noise, but the bias can be reduced if the external excitation is autocorrelated, and the CSIMPCA can effectively eliminate the bias. The closed-loop simulation results for SIMPCA are presented in Figure 4.8. The results presented in the left column of Figure 4.8 correspond to the white-noise external excitation with variance of 1. The results presented in the right column correspond to the autocorrelated external excitation r, where r is filtered white noise; the filter is first-order with a pole 0.9 and the filtered excitation signal has variance 1. The results demonstrate that 1) autocorrelation of the external excitation can indeed reduce the bias of SIMPCA, and 2) the CSIMPCA indeed performs significantly better than SIMPCA.
78
4 Closed-loop Subspace Identification 5
5
r
0
−5
0
0
200
400
100
600
800
−5
1000 1200
dB
dB
−50
100
400
600
800
1000 1200
SIMPCA
0
−50 −1
0
10
10
1
10
−100 −2 10
−1
0
10
100
CSIMPCA
10
1
10
CSIMPCA
50 dB
50 0
−50 −100 −2 10
200
50
0
−100 −2 10
0
100
SIMPCA
50
dB
r
0
−50 −1
10
0
rad/sec 10
1
10
−100 −2 10
−1
10
0
rad/sec 10
1
10
Fig. 4.8. Closed-loop Monte-Carlo simulations
4.4 Summary In this chapter, several closed-loop subspace identification algorithms have been reviewed. Then by adopting the EIV model structure, a subspace orthogonal projection identification method (SOPIM) was elaborated. However, it yields a bias in closed-loop identification under certain conditions. Through analysis of the bias error of SOPIM under closed-loop conditions, it was discovered that other existing instrument subspace methods in the literature may also suffer from the same bias error and therefore, these methods may not be suitable for closed-loop identification. Motivated by this finding, a solution was derived to eliminate bias error for SOPIM as well as for an existing instrument subspace identification algorithm, SIMPCA, for the sake of closed-loop identification. As a result two novel subspace closed-loop identification algorithms CSOPIM and CSIMPCA, named after SOPIM and SIMPCA, respectively, are developed. In addition, the orthogonal projection method discussed in this chapter provides both the extended observability matrix and the Kalman filter state sequence. Therefore, system models may also be recovered from the estimated Kalman state sequence, and the closedloop subspace orthogonal projection identification method via state estimation (CSOPIM-S) is also suggested. Simulations based on a benchmark problem compared the performance of the orthogonal projection algorithms with a number of existing subspace identification algorithms and verified the feasibility and closedloop applicability of the orthogonal projection algorithms.
5 Identification of Dynamic Matrix and Noise Model Using Closed-loop Data∗
5.1 Introduction Model predictive controllers (MPC) have found many successful applications in process industries for about three decades. One of the key aspects of MPC is the prediction of the future process response and minimization of the output deviation from the setpoint by manipulating the inputs. A model for the process is required to make these predictions based on past data. Hence an MPC design starts with first identifying a nominal model for the process. One of the industrially successful predictive control schemes is the dynamic matrix control or DMC, which explicitly uses a lower triangular matrix called ‘dynamic matrix’ containing the step response coefficients corresponding to the deterministic input(s) to the process [1, 2]. Many other MPC formulations also use the dynamic matrix in one way or the other [92, 5, 93]. For constructing the dynamic matrix, in the case of DMC, a step response model for the process is first obtained from the open-loop data. The step response coefficients are then arranged in a specific lower triangular form in the dynamic matrix, as will be discussed in detail in Chapter 6. However, for safety reasons and other practical limitations, open-loop operation of the process may not always be possible, or in some cases there may be a hidden feedback in the system. Estimation of the dynamic matrix from closed-loop data is desirable in such cases. It has been shown [94] that if the model is used for model-based control design then the favorable experimental conditions are actually under closed-loop conditions. Even though subspace identification is used as a vehicle, the goal of the datadriven closed-loop identification method discussed in this chapter is not the estimation of state space matrices {A, B, C, D and K} but the estimation of the dynamic matrices of the process and noise models directly from input-output data. Figure 5.1 illustrates the differences between the data-driven closed-loop subspace approach to be considered here and other existing closed-loop subspace identification methods as discussed in Chapter 4, for getting a dynamic matrix. ∗
This chapter (with revisions) is reprinted with permission from Ind. Eng. Chem. Res., vol 41, Kadali, R., B. Huang, “ Estimation of the Dynamic Matrix and Noise c Model for Model Predictive Control Using Closed-Loop Data”, 842-852, 2002 American Chemical Society. Sections 5.2 and 5.4 have major revisions.
B. Huang et al.: Dyn. Model. Predict. Ctrl. & Perform. Monitor., LNCIS 374, pp. 79–97, 2008. c Springer-Verlag London Limited 2008 springerlink.com
80
5 Identification of Dynamic Matrix and Noise Model Using Closed-loop Data
The method considered in this chapter may be considered as a nonparametric approach for closed-loop identification. Nonparametric model identification methods, although known to give less bias error due to less model structure and order limitations, could result in higher variance (due to the larger number of parameters) compared to parametric model identification methods. This is a tradeoff between bias error and variance error in process identification. All physical processes are typically of high-order and nonlinear in nature, and it is not always possible to represent them by a single linear parametric model. Consequently, bias error is inevitable in practice. On the other hand, it is known that the variance error can be reduced with the increased sample size [11]. Therefore, depending on the application, for example, depending on the data sample size, one can choose between the parametric or nonparametric identification method. Closed-loop data with sufficient excitation
Subspace matrices Conventional approach
Data-driven approach
System A, B, C, D, K
Dynamic matrix Fig. 5.1. Comparing the conventional closed-loop subspace state space identification methods and the proposed data-driven closed-loop subspace approach, for estimating dynamic matrices
However, nonparametric model identification methods do have some practical advantages. Consider the case when we want to identify a process model for designing and implementing a model predictive controller. Even if we identify a parametric model for the process (using closed- or open-loop process data), the parametric model often has to be transferred to a nonparametric (impulse or step response) model form first and then a dynamic matrix for designing the MPC. The concern is that this additional intermediate step may introduce an additional error and inconvenience. When it comes to industrial implementation, nonparametric-model based MPCs have shown considerable success. The idea explored in this chapter is to identify a nonparametric-like model directly and avoid choosing a ‘model structure’ which is unknown and a prerequisite needed
5.2 Estimation of Process Dynamic Matrix and Noise Model
81
for parametric model identification methods. In addition, instead of identifying a traditional nonparametric model such as a step response model, the proposed approach identifies the dynamic matrix directly. The remainder of the chapter is organized as follows. Section 5.2 gives a description of the estimation of the process dynamic matrix and noise model from closed-loop data. Remarks on the different steps of the closed-loop identification along with some guidelines for the practical implementation of the algorithm are provided in Section 5.3. The closed-loop identification method is extended to the case of measured disturbances in Section 5.4, and explained using MATLAB simulations in Section 5.5 and implementation on a pilot scale process in Section 5.6. Summary from the above work is given in Section 5.7.
5.2 Estimation of Process Dynamic Matrix and Noise Model Consider the case when the system described by (3.1)-(3.2) is operating under closed-loop with a linear time-invariant feedback-only controller Gc , expressed in a transfer function form as ut = Gc (rt − yt )
(5.1)
where rt (m × 1) is the setpoint for the process output at the sampling instant t and (rt −yt ) is the output deviation from the setpoint. Assume that the controller does not cancel any plant dynamics, and the controller is expressed in state space representation as xct+1 = Ac xct + Bc (rt − yt )
(5.2)
+ Dc (rt − yt )
(5.3)
ut =
Cc xct
where {Ac , Bc , Cc and Dc } are the dynamic state space system matrices of the controller. By recursively using the above state space equations, as discussed in Chapter 3, we can write the input/output subspace equations for the controller as c (Rp − Yp ) Up = ΓNc Xpc + HN
(5.4)
− Yp ) − Yf )
(5.5) (5.6)
Xfc
= Uf =
c c AN c Xp + ΔN (Rp c ΓNc Xfc + HN (Rf
where
ΓNc
c HN
⎞ Cc ⎟ ⎜ ⎜ Cc Ac ⎟ =⎜ ⎟ ⎠ ⎝ ... N −1 Cc Ac ⎛ 0 Dc ⎜ C B D ⎜ c c c =⎜ ⎝ ... ... −2 −3 Bc Cc AN Bc Cc AN c c ⎛
⎞ ... 0 ... 0 ⎟ ⎟ ⎟ ... ... ⎠ ... Dc
82
5 Identification of Dynamic Matrix and Noise Model Using Closed-loop Data
−1 −2 AN Bc AN Bc ... Bc c c Xpc = xc0 xc1 ... xcj−1 Xfc = xcN xcN +1 ... xcN +j−1
ΔcN =
The matrices Rp and Rf are past and future block-Hankel matrices corresponding to rt . Following the same approach as the derivation of Equation 3.41 from Equations 3.1 and 3.2, we can derive the following equation from Equations 5.2 and 5.3 (5.7) Uf = Lcw Wpc + Lcr Rf − Lcy Yf where Lcw is a coefficient matrix corresponding to past inputs, outputs and setpoints; ⎞ ⎛ Yp ⎟ ⎜ ⎟ ⎜ Wpc = ⎜ Up ⎟ ⎠ ⎝ Rp c and Lcr = Lcy = HN . Substituting Equation 5.7 in Equation 3.41 yields
Yf = Lw Wp + Lu (Lcw Wpc + Lcr Rf − Lcy Yf ) + Le Ef
(5.8)
Solving Yf from Equation 5.8 gives us Yf = (I + Lu Lcy )−1 (Lw Wp + Lu Lcw Wpc ) + (I + Lu Lcy )−1 Lu Lcr Rf +(I + Lu Lcy )−1 Le Ef CL CL = LCL + LCL y Wp yr Rf + Lye Ef
(5.9)
where CL LCL = (I + Lu Lcy )−1 (Lw Wp + Lu Lcw Wpc ) y Wp c −1 Lu Lcr = (I + Lu Lcy )−1 Lu Lcy LCL yr = (I + Lu Ly )
LCL ye
= (I +
Lu Lcy )−1 Le
(5.10) (5.11)
and WpCL = Wpc . Similarly, substituting Equation 3.41 in Equation 5.7 yields CL CL Uf = LCL + LCL u Wp ur Rf + Lue Ef
(5.12)
where CL LCL = (I + Lcy Lu )−1 (Lcw Wpc − Lcy Lw Wp ) u Wp c −1 c Lr = (I + Lcy Lu )−1 Lcy LCL ur = (I + Ly Lu )
(5.13)
Lcy Lu )−1 Lcy Le
(5.14)
LCL ue
= −(I +
5.2 Estimation of Process Dynamic Matrix and Noise Model
83
With the above results, the estimation of the closed-loop subspace matrices using closed-loop data is essentially an open-loop identification problem. Putting Equations 5.9 and 5.12 together, we get ⎞ ⎛ ⎞ ⎞ ⎞ ⎛ ⎛ ⎛ CL CL LCL L L Yf ⎠ = ⎝ y ⎠ WpCL + ⎝ yr ⎠ Rf + ⎝ ye ⎠ Ef ⎝ (5.15) Uf LCL LCL LCL u ur ue From the above equation, and since the setpoint Rf can be chosen to be uncorreCL CL CL lated with Ef , the closed-loop subspace matrices {LCL u , Lur , Ly , Lyr } can be obtained as a solution of a least squares estimation problem. They are estimated by the orthogonal projection of the row space of {Uf and Yf } into the row space spanned by {WpCL and Rf }:
LCL LCL u ur
⎛
= Uf ⎝ = Uf
LCL LCL y yr
= Yf ⎝ = Yf
Rf
⎞† ⎠
(WpCL )T RfT
⎛
WpCL
WpCL Rf
⎞† ⎠
(WpCL )T RfT
⎛
WpCL
⎛
WpCL
(⎝
(⎝
Rf
Rf
⎞
⎠ (W CL )T RT )−1 (5.16) p f
⎞
⎠ (W CL )T RT )−1 (5.17) p f
This projection can be implemented in a numerically robust way with a QRdecomposition. With the estimation of the subspace matrices, the predictions ˆf , Yˆf can then be written as U ⎛ ⎞ ⎛ ⎞ ⎞ ⎛ CL Yˆf LCL L y yr ⎝ ⎠=⎝ ⎠ WpCL + ⎝ ⎠ Rf CL ˆf U LCL L u ur
The first row of Yˆf represents one-step ahead predictions for the output. Therefore the white noise disturbance sequence entering the process can be estimated as T ef = eN eN +1 ... eN +j−1 = Yf (1 : m, :) − Yˆf (1 : m, :)
where (1 : m, :) represents a vector containing the elements from rows 1 to m and all the columns of the matrix. Let us define ˆf = LCL Ξ f = Uf − U ue Ef
84
5 Identification of Dynamic Matrix and Noise Model Using Closed-loop Data
The block Hankel matrix, Ef , for the noise can be constructed using the estimated noise, ef . Therefore, LCL ue is estimated as = Ξf /Ef = Ξf Ef† LCL ue 5.2.1
Estimation of Dynamic Matrix of the Process
Typically if there is no delay in the controller and the system has the same number of inputs and outputs, Lcy or equivalently Lcr is invertible; thus LCL ur = (I + Lcy Lu )−1 Lcy is invertible. In the following derivation, we assume that this is true. Otherwise, the inverse has to be replaced by pseudo inverse. The well-known matrix inversion lemma is given below: [A + BCD]−1 = A−1 − A−1 B[C −1 + DA−1 B]−1 DA−1 Using this lemma, we can write (I + Lu Lcy )−1 = I − Lu (I + Lcy Lu )−1 Lcy
(5.18)
CL −1 can be written as Now, LCL yr (Lur ) CL −1 LCL = (I + Lu Lcy )−1 Lu Lcy [(I + Lcy Lu )−1 Lcy ]−1 yr (Lur )
= (I + Lu Lcy )−1 Lu Lcy [(Lcy )−1 + Lu ] Applying Equation 5.18 to the equation above gives CL −1 = [I − Lu (I + Lcy Lu )−1 Lcy ]Lu Lcy [(Lcy )−1 + Lu ] LCL yr (Lur )
= [Lu Lcy − Lu (I + Lcy Lu )−1 Lcy Lu Lcy ][(Lcy )−1 + Lu ] = Lu [Lcy − (I + Lcy Lu )−1 Lcy Lu Lcy ][(Lcy )−1 + Lu ] = Lu {Lcy − [(Lcy )−1 + Lu ]−1 Lu Lcy }[(Lcy )−1 + Lu ] = Lu {Lcy − Lcy [I + Lu Lcy ]−1 Lu Lcy }[(Lcy )−1 + Lu ] Applying the matrix inverse lemma again, we get {Lcy − Lcy [I + Lu Lcy ]−1 Lu Lcy } = [(Lcy )−1 + Lu ]−1 Thus CL −1 = Lu [(Lcy )−1 + Lu ]−1 [(Lcy )−1 + Lu ] LCL yr (Lur ) d = L u = HN
(5.19)
d , HN
which contains the Markov parameters corresponding to the deHence terministic input, can be identified according to the above derivation as d CL −1 = LCL HN yr (Lur ) ⎛ F 0 ... ⎜ 0 ⎜ ⎜ F1 F0 ... =⎜ ⎜ ⎜ ... ... ... ⎝ FN −1 ... ...
0
⎞
⎟ ⎟ 0⎟ ⎟ ⎟ 0⎟ ⎠ 0
(5.20)
5.2 Estimation of Process Dynamic Matrix and Noise Model
85
where Fi represents the i-th Markov parameter (or impulse response coefficient) with the deterministic input. The dynamic matrix containing the system step response coefficients, SN , can be obtained as ⎞ ⎛ ⎞⎛ ⎛ ⎞ G0 0 ... 0 I 0 ... 0 F0 0 ... 0 ⎟ ⎜ ⎟⎜ ⎜ ⎟ ⎟ ⎜ ⎟⎜ ⎜ ⎟ ⎜ G1 G0 ... 0 ⎟ ⎜ F1 F0 ... 0 ⎟ ⎜ I I ... 0 ⎟ ⎟ ⎟ ⎜ ⎜ ⎜ ⎟ SN = ⎜ ⎟=⎜ ⎟⎜ ⎟ ⎜ ... ... ... 0 ⎟ ⎜ ... ... ... 0 ⎟ ⎜ ... ... ... 0 ⎟ ⎠ ⎝ ⎠⎝ ⎝ ⎠ GN −1 ... ... 0 FN −1 ... ... 0 I I ... I ⎛ ⎞ I 0 ... 0 ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ I I ... 0 d ⎜ ⎟ (5.21) = HN ⎜ ⎟ ⎜ ... ... ... 0 ⎟ ⎝ ⎠ I I ... I where Gi represents the ith step response coefficient with the deterministic input.
5.2.2
Estimation of the Noise Model
The noise model can be estimated from the residuals of the input data. Using CL the definitions for LCL ur and Lue , the noise dynamic matrix is obtained as s −1 CL = Le = −(LCL Lue HN ur )
(5.22)
s HN contains the impulse response coefficients corresponding to the stochastic input.
s HN
⎛
I ⎜ ⎜ L1 =⎜ ⎝ ... LN −1
⎞ 0 ... 0 ⎟ I ... 0 ⎟ ⎟ ... ... 0 ⎠ ... ... 0
where Li represents the ith impulse response coefficient with the stochastic input. s Thus the first column of HN represents the noise model H(z −1 ) in the impulse response form: H(z −1 ) = I + L1 z −1 + L2 z −2 + ... + Li z −i + ... + LN −1 z −N +1 Alogrithm 5.1. The following are the implementation procedures for the datadriven closed-loop subspace identification method discussed above: 1. Construction of the data Hankel matrices {Up , Uf , Yp , Yf , Rp , Rf } using closed-loop data. By linear regression the deterministic closed-loop subspace matrices are identified.
86
5 Identification of Dynamic Matrix and Noise Model Using Closed-loop Data
Remark 5.2. The guidelines presented in Section 5.3 can be used in the selection of the number of rows and columns. By adding a persistent exciting signal, which is uncorrelated with the process noise in the setpoint, we ensure identifiability of the closed-loop subspace matrices. This step is an open-loop identification problem with the setpoint(s) change being the deterministic external inputs and the closed-loop subspace matrices as the model to be identified. 2. Estimation of the vector of noise data from the ‘output data Hankel matrix’, and ‘residual data Hankel matrix’ corresponding to ‘input data Hankel matrix’. Estimation of the stochastic closed-loop subspace matrices. Remark 5.3. The first row of the residual matrix (Yf −Yˆf ) represents one-step ahead prediction errors and an estimate of the noise entering the process. The ˆf ) is the residual data Hankel matrix corresponding to matrix, Ξf = (Uf − U the input. The noise data Hankel matrix, Ef , is constructed using the vector of estimated process noise. By linear regression, the stochastic closed-loop subspace matrix is estimated. 3. Retrieving the open-loop deterministic subspace matrix from the closed-loop subspace matrices. Remark 5.4. Closed-loop subspace matrices are just the open-loop subspace matrices weighted by the subspace matrix corresponding to the sensitivity function. The analogies between the process/noise transfer functions and the open-loop deterministic/stochastic subspace matrices are obvious. The method presented in this chapter is parallel to the approach used in ‘joint input/output closed-loop identification method’, which is well known in the transfer function domain. However with the ‘joint input/output closed-loop identification method’, inverting the transfer function (or transfer function matrices for the multivariate systems) can cause problems: the resultant transfer function (matrix) may be improper or of high order. No such problems are encountered in the subspace-matrices based closed-loop approach since we are dealing with algebraic matrices instead of transfer functions. 4. Retrieving the open-loop stochastic subspace matrix from the closed-loop subspace matrices.
5.3 Some Guidelines for the Practical Implementation of the Algorithm Building the data Hankel matrices is the first step in all the subspace based identification methods. If we want to identify a state space model for the system using the subspace identification methods, then as discussed in Chapter 3 the number of rows, N , is chosen to be larger than the order of the state space model to be identified [38]. The number of columns, j, should tend to infinity. Since we have finite data in real situations we can only choose a finite j, the maximum
5.4 Extension to the Case of Measured Disturbance Variables
87
number of columns that can be constructed with the available data. As far as the closed-loop identification method presented in this chapter is concerned, we are identifying only the subspace matrices and not the state space system matrices. Following are some guidelines that can help in deciding the number of rows and columns of the data Hankel matrices: 1. To obtain the complete process dynamics, the number of rows, N , should be chosen such that the last impulse response coefficient (last element of the d ) approaches zero. first block-column of HN 2. The choice of the number of columns depends on the excitation signal used for identification. The richer the excitation signal the fewer the number of columns required. The number of columns is chosen in such a way that the corresponding impulse response coefficients along the columns of the subspace matrices are very close. The higher the number of columns taken in the data Hankel matrices the closer will the corresponding coefficients in the columns be. 3. Numerical tools like QR decomposition can be used to avoid numerical problems associated with the inversion of large matrices, specially in step 1. 4. Studies on derivation of statistical properties for subspace based identification methods are an area of active research and have been considered in [95, 64, 38, 54] and the references therein, where it has been shown that under open-loop conditions subspace identification can yield a consistent estimation of the parameters. As the closed-loop method discussed in this chapter is equivalent to an open-loop subspace identification problem, the same conclusion may be applied.
5.4 Extension to the Case of Measured Disturbance Variables The closed-loop subspace-based identification method explained in the previous section can be extended to the case where some measured disturbances are available for feedforward control. Consider the case when measurements of some of the disturbance variables are available and we want to identify the subspace matrices corresponding to these variables. Let mt (h × 1) represent the vector of the measured disturbance variables. Assume that the measured disturbance variables are uncorrelated with the setpoint changes. Consider a feedback-only controller described in Equations 5.1-5.6 acting on a process represented by
⎛
ut
xt+1 = Axt + B Bm ⎝
mt
yt = Cxt + D Dm ⎝
mt
⎛
ut
⎞
⎠ + Ket
(5.23)
⎞
⎠ + et
(5.24)
88
5 Identification of Dynamic Matrix and Noise Model Using Closed-loop Data
The matrix input-output Equations 3.20 and 3.41 are modified to include measured disturbances as d m s U f + HN M f + HN Ef Yf = ΓNb Xfb + HN
= Lbw Wpb + Lu Uf + Lm Mf + Le Ef
(5.25) (5.26)
where the superscript b in ΓNb and Xfb is used to distinguish them from ΓN and Xf defined in subspace Equations 3.20 and 3.41 (i.e., the case without considering measurement disturbances) and ⎞ ⎛ Dm 0 ... 0 ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ CB D ... 0 m m m ⎟ ⎜ (5.27) L m = HN = ⎜ ⎟ ⎜ ... ... ... ... ⎟ ⎠ ⎝ CAN −2 Bm CAN −3 Bm ... Dm ⎞ ⎛ Yp ⎟ ⎜ ⎟ ⎜ Wpb = ⎜ Up ⎟ (5.28) ⎠ ⎝ Mp Mp and Mf are the ‘past’ and ‘future’ data Hankel matrices of mt defined in the same way as those corresponding to ut defined in Equations 3.23 and 3.24. Define ⎞ ⎛ Yp ⎟ ⎜ ⎟ ⎜ ⎜ U p ⎟ ⎟ WpCLb = ⎜ ⎟ ⎜ ⎜ Mp ⎟ ⎠ ⎝ Rp
Similar to Equation 5.15, we can derive ⎛ ⎞ ⎛ ⎞ ⎞ ⎞ ⎛ ⎛ CL CL Yf LCLb L L ⎝ ⎠ = ⎝ y ⎠ WpCLb + ⎝ yr ⎠ Rf + ⎝ ym ⎠ Mf Uf LCLb LCL LCL u ur um ⎞ ⎛ LCL ye ⎠ +⎝ Ef LCL ue
(5.29)
where, in addition to the subspace matrices given in Equations 5.10, 5.11, 5.13, and 5.14, we have c −1 c c d −1 c m LCL Ly Lm = −(I + HN HN ) H N HN um = −(I + Ly Lu )
LCL ym
= (I +
Lu Lcy )−1 Lm
= (I +
d c −1 m HN HN ) HN
(5.30) (5.31)
5.5 Closed-loop Simulations
89
The closed-loop subspace matrices are identified by data projections as shown m containing the Markov parameters (impulse rein Section 5.2. The matrix HN sponse coefficient matrices) corresponding to the measured disturbance variables is obtained from the closed-loop subspace matrices as m −1 CL HN = Lm = −(LCL Lum ur )
5.5 Closed-loop Simulations Certain comparative simulations were carried out in MATLAB for two cases, univariate and multivariate systems, among the data-driven subspace approach presented in this chapter, MOESP [83], and CVA [86, 85], all based on the closed-loop version. The purpose of this exercise is to check the validity of the data-driven approach of closed-loop identification for both univariate and multivariate systems, which does not require controller knowledge for closed-loop identification, and also to see how it performs compared to the existing subspace identification methods, namely the MOESP and CVA approaches. It has been shown by Van Overschee and De Moor [66] that the difference among the three subspace identification algorithms, N4SID, MOESP and CVA, is the difference in the way the weighting matrices are used in the subspace identification algorithm. In fact, MATLAB offers the ‘N4SID’ command wherein the user can specify MOESP or CVA and the respective weighting matrices will be used, so that it is equivalent to using MOESP/CVA subspace identification algorithms. Hence in the simulations presented in this section, the MOESP/CVA weighting matrices are used in the ‘N4SID’ algorithm in MATLAB . 5.5.1
Univariate System
Consider the following system [38]: ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ 0.6 0.6 0 1.6161 −1.1472 ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ xt+1 = ⎜ −0.6 0.6 0 ⎟ xt + ⎜ −0.3481 ⎟ ut + ⎜ −1.5204 ⎟ et ⎝ ⎠ ⎝ ⎠ ⎝ ⎠ 0 0 0.7 2.6319 −3.1993 yt = −0.4373 −0.5046 0.0936 xt + (−0.7759)ut + et
A PID controller, 0.1 + 0.08 s + 0.08s, is tuned online for the above system for good setpoint tracking and disturbance rejection performance. We assume that the controller knowledge is unknown for the closed-loop identification. Closedloop input/output/setpoint data are obtained by exciting the system using a designed RBS signal of magnitude 1 for the system setpoint and random white noise of standard deviation 0.1 for the disturbance in MATLAB and Simulink . The closed-loop data is plotted in Figure 5.2. Using the closed-loop subspace identification method presented in Section 5.2, with the number of rows (N ) = 30 and the number of columns (j) = 2000
90
5 Identification of Dynamic Matrix and Noise Model Using Closed-loop Data
Closed−loop data 2
R and Y
1
0
−1
−2
0
500
1000
1500
0
500
1000
1500
2000
2500
3000
3500
2000
2500
3000
3500
1.5 1
U
0.5 0 −0.5 −1 −1.5
Samples
Fig. 5.2. Univariate system: Closed-loop data d s in the data Hankel matrices, the subspace matrices HN and HN are identid fied. Due to the presence of noise, the upper non-diagonal elements in HN and s HN will not be exactly zero but very small numbers (they approach zero as j −→ ∞). The true impulse response coefficients of the system can be calculated from the state space system matrices provided above. The identified impulse response coefficients are plotted against the true impulse response coefficients in Figure 5.3. It is illustrated that the identified impulse response coefficients match well with the true ones. Joint input-output closed-loop MOESP/CVA identification methods are used to identify the deterministic part of the system using the same set of closedloop data. Impulse response coefficients identified through MOESP and CVA are plotted against the true impulse response coefficients in Figure 5.4. We can see that impulse response coefficients identified from both MOESP and CVA methods match reasonably well with the true ones for this univariate system. Therefore, all three methods yielded similar results in this case.
5.5.2
Multivariate System
Consider the following system taken from the MATLAB /MPC toolbox manual: ⎞ ⎛ ⎞ ⎛ ⎛ ⎞⎛ ⎞ 12.8e−s −18.9e−3s 3.8e−8s y1 (s) u (s) 1 ⎠ = ⎝ 16.7s+1 21.0s+1 ⎠ ⎝ ⎠ + ⎝ 14.9s+1 ⎠ e(s) ⎝ 6.6e−7s −19.4e−3s 4.9e−3s y2 (s) u (s) 2 10.9s+1 14.4s+1 13.2s+1
5.5 Closed-loop Simulations
91
Comparing the actual IR−coefficeints with those from Lu and Le 1
Process
0.5
0
−0.5
−1
0
5
10
15
20
25
30
0
5
10
15 Lag
20
25
30
1
Noise
0.5
0
−0.5
−1
Fig. 5.3. Univariate system: comparing the true (solid) IR-coefficients with those obtained in the subspace matrices (dotted) True(−) and MOESP identified(..)
Deterministic response
1 0.5 0 −0.5 −1
0
5
10
15 Time
20
25
30
20
25
30
True(−) and CVA identified(..)
Deterministic response
1 0.5 0 −0.5 −1
0
5
10
15 Time
Fig. 5.4. Univariate system: comparing the true (solid) IR-coefficients with those identified by MOESP/CVA approaches
92
5 Identification of Dynamic Matrix and Noise Model Using Closed-loop Data
Closed−loop data
Closed−loop data
0.5
0.5 R2(−) y2(..)
1
R1(−) y1(..)
1
0
−0.5
0
−0.5
−1
−1 0
500
1000 1500 Time
2000
2500
0
500
1000 1500 Time
2000
2500
0
500
1000 1500 Time
2000
2500
1.5
2
1 0.5 U2
U1
1 0
0
−0.5
−1
−1 −2
−1.5 0
500
1000 1500 Time
2000
2500
Fig. 5.5. Multivariate system: closed-loop data
True(−) and identified(..)
True(−) and identified(..)
1.5
1 G12
G11
1 0.5
0.5 0
0
20
40
60
80
0
100
0
20
40
Lags 0
G22
G21
100
60
80
100
60
80
100
0
−1
−1 −1.5 −2
−1.5 0
20
40
60
80
100
−2.5
0
20
40
Lags
Lags 0.6
0.4 0.3
H2
H1
80
−0.5
−0.5
0.2
0.4 0.2
0.1 0
60 Lags
0
20
40
60 Lags
80
100
0
0
20
40 Lags
Fig. 5.6. Multivariate system: comparing the true (solid) IR-coefficients with those obtained in the subspace matrices (dotted)
5.5 Closed-loop Simulations
1.5
93
0
−0.5 G12
G11
1
−1
0.5 −1.5 0
0
20
40
60
80
100
0
20
40
Lags
60
80
100
60
80
100
Lags
0 1 −0.5
0.6
G22
G21
0.8
−1.5
0.4
−2
0.2 0
−1
0
20
40
60 Lags
80
100
−2.5
0
20
40 Lags
Fig. 5.7. Multivariate system: comparing the true (solid) IR-coefficients with those identified by MOESP approach
where {y1 (s), y2 (s)}, {u1 (s), u2 (s)} and w(s) represent the system outputs, inputs and random noise disturbance respectively. A state-space based MPC controller is designed in MATLAB for the system. A sampling period of T = 2 time units is used in the simulations. Closed-loop input/output data is obtained by exciting the system using a designed RBS signal of magnitude 1 for the setpoint (rt ) and random white noise (et ) of standard deviation 0.1 for the disturbance in MATLAB and Simulink . We again assume that the controller is not known for the closed-loop identification. The closed-loop data is plotted in Figure 5.5. The data-driven closed-loop subspace identification algorithm is used with rows(N ) = 50 and columns(j) d and = 2500 in the data Hankel matrices to identify the subspace matrices HN s HN . The identified impulse response coefficients are plotted against the true impulse response coefficients in Figure 5.6. It can be seen from the plot that the identified impulse response coefficients match closely with the true ones. Next, the MOESP approach is used to identify the deterministic part of the system using the same closed-loop data. The impulse response coefficients identified using the MOESP approach are compared with the true coefficients in Figure 5.7 and the result is not quite comparable with that obtained using the data-driven closed-loop subspace method. It should be noted that a better match between the identified and true coefficients using MOESP could be achieved only with a very high order model. The order of the end model could probably be reduced using a standard model reduction method but with a compromise
94
5 Identification of Dynamic Matrix and Noise Model Using Closed-loop Data
between bias error and complexity. The closed-loop CVA approach [86] involves the inversion of a transfer function matrix which may not always be possible in MATLAB due to the fact that the resultant matrix could contain improper transfer functions.
5.6 Identification of the Dynamic Matrix: Pilot-scale Experimental Evaluation The data-driven closed-loop subspace method for the estimation of the dynamic matrix is tested on a pilot-scale process. The process considered is shown in Figure 5.8.
LT
FC
Input (flow rate)
Controlled variable (level)
LT
Outlet valve (kept constant)
water drain Fig. 5.8. Experimental setup
The input (u) is the inlet water flow rate and the process variable to be controlled (y) is the level of water in the tank. The tank outlet flow valve is kept at a constant position. The head (pressure) of the water in the inlet pipe can be considered as (an unmeasured) disturbance. The tank level is controlled by a PID controller, 2.5 + 0.05 s + 1s. An RBS signal of series of setpoint changes to the level is designed in MATLAB . Closed-loop data of the process input, setpoint and output is collected and plotted in Figure 5.9.
5.6 Identification of the Dynamic Matrix: Pilot-scale Experimental Evaluation
95
Closed−loop data 10
R
9.5 9 8.5 8 200
400
600
800
1000
1200
1400
1600
1800
200
400
600
800
1000
1200
1400
1600
1800
200
400
600
800
1000 Samples
1200
1400
1600
1800
16
U
14 12 10 8
11
Y
10 9 8 7
Fig. 5.9. Pilot scale process: closed-loop system data
Plot of columns(1,20,40 and 60) in subspace matrices 0.1
From ‘Lu‘
0.08 0.06 0.04 0.02 0 0
20
40
60
80
100
120
140
160
180
200
0
20
40
60
80
100 Lags
120
140
160
180
200
1.5
From ‘Le‘
1
0.5
0
Fig. 5.10. Pilot scale process: IR-coefficients from the columns of the identified subspace matrices
96
5 Identification of Dynamic Matrix and Noise Model Using Closed-loop Data
Comparing the IR−models 0.1
From ‘Lu‘
0.08 0.06 0.04 0.02 0 0
20
40
60
80
100
120
140
160
180
200
0
20
40
60
80
100 Lags
120
140
160
180
200
1.5
From ‘Le‘
1
0.5
0
Fig. 5.11. Pilot scale process: comparing the IR-coefficients from subspace matrices identified using the open-loop data (dotted line) and that from the closed-loop data (solid line)
Data Hankel matrices of dimensions with rows(N ) = 200 and columns(j) = 1500 are constructed using the closed-loop data and the subspace matrices d s HN and HN are identified. The columns of the subspace matrices are plotted in Figure 5.10. The figure illustrates that the corresponding impulse response coefd s and HN do match after time shift (time shift ficients in different columns of HN is necessary due to the lower triangular structure of block-Toeplitz matrices). d is checked The accuracy of the impulse response coefficients in the matrix HN by doing an open-loop identification. Open-loop data is collected by exciting the process with an input RBS signal of magnitude 1. The impulse response coefficients identified using the open-loop subspace identification method are plotted together with the coefficients identified suing closed-loop data in Figure 5.11. We can see that there is some mismatch in the impulse response models in the subspace matrices identified using closed-loop data and those identified using open-loop data. The mismatch may be due to the different operating regions between closed-loop and open-loop identifications and some effect of feedback control.
5.7 Summary This chapter provides a data-driven subspace method for identification of process dynamic matrices and the noise model from closed-loop data. Closed-loop subspace matrices are first obtained with persistent setpoint excitations of the
5.7 Summary
97
closed-loop systems. Open-loop subspace matrices are then retrieved from the closed-loop subspace matrices. The process dynamic matrix is obtained from the deterministic subspace matrix and the noise model in the impulse response form is obtained from the stochastic subspace matrix. The method has been extended to the case of measured disturbances under feedback-only control. Results from simulations and from an evaluation on a pilot scale plant are provided to illustrate the proposed data-driven closed-loop subspace identification method.
6 Model Predictive Control: Conventional Approach
6.1 Introduction Predictive controllers have been widely used in process industries for about three decades [92, 5, 93, 6]. Several forms of predictive controllers like IDCOM [96], DMC [1, 2, 5], QDMC [3, 4], RMPCT (reviewed by [5, 6]), GPC [7, 8], etc. have been successfully implemented in process industries through the years. The term predictive control does not designate a specific control strategy but a wide range of control algorithms which make explicit use of a (predictive) process model in a cost function minimization to obtain the control signal [92, 97]. There are many good references for model predictive control in the literature, such as [92, 98, 99, 100, 6, 101, 102]. In this chapter, following MPC introduction in [92, 98, 99, 100], lecture notes in [103, 104, 105], and other references stated above, a tutorial on model predictive control is provided. More technical details can be looked for in the above references as well as many other publications in model predictive control over the last 30 years. Model predictive control is an appropriately descriptive name for a class of model based control schemes that utilize a process model for two central tasks [99]: 1) explicit prediction of future process behavior, and 2) computation of appropriate corrective control action required to drive the predicted output as close as possible to the desired target values. The overall objectives of an MPC may be summarized as [6, 100]: • Prevent violations of input and output constraints. • Drive some output variables to their optimal setpoints, while maintaining other outputs within specified ranges. • Prevent excessive movement of the input variables. • Control as many process variables as possible when a sensor or actuator is not available. The ideas appearing to a greater or lesser degree in all predictive controls are basically [92]: • dependence of the control law on predicted behavior, • explicit use of models to predict the process output at future time instants, B. Huang et al.: Dyn. Model. Predict. Ctrl. & Perform. Monitor., LNCIS 374, pp. 101–119, 2008. c Springer-Verlag London Limited 2008 springerlink.com
102
6 Model Predictive Control: Conventional Approach
• calculation of control sequence minimizing an objective function, and • a receding horizon strategy, i.e., updating of input and shifting of the horizon towards the future at each time instant. A number of MPC algorithms are available commercially. Following are some representative MPC technologies [92, 6]: • DMC Plus (Dynamic Matrix Control) and Aspen Target: ASPEN Tech, • ADMC (Adaptive DMC): CTC, • IDCOM (Identification and Command), HIECON (Hierarchial Constraint Control), PFC (Predictive Functional Control): Adersa, • RMPCT (Robust Multivariable Predictive Control Technology): Honeywell, • SMCA (Setpoint Multivariable Control Architecture), IDCOM-M (Multivariable): Setpoint Inc, • APCS (Adaptive Predictive Control System): SCAP Europa, • SMOC (Shell Multivariable Optimising Controller): Shell, • Connoisseur (Control and Identification package): Invensys, • MVC (Multivariate Control): Continental Controls Inc., • NOVA-NLC (NOVA Nonlinear Controller): DOT Products, • Process Perfecter: Pavilion Technologies. Some of these technologies are nonlinear MPC while most are linear MPC.
6.2 Understanding MPC Predictive control is intuitive and used in our daily activities like walking, driving, studying and so on. Think about the course of studying in a school. Basically one has to do a set of things: • Predict: When one sets a target for a “desired” grade, one has to plan and work towards the target. It may be too early to consider the final target at beginning of a term. Instead, one should think a few days or a few weeks ahead and predict what performance may be achieved over this shorter time window. The target within the shorter time period can be, for example, certain “desired” grades in the assignment, quiz, etc. • Plan: Compare the predicted performance with the shorter time target. If a difference is to be expected, for example, lower than the target, then additional efforts should be considered, subject to constraints of course, such as there are only 24 hours a day. • Act: If it is expected that the additional efforts likely make one meet the target, then the additional efforts will be put into action. Although a set of the additional efforts, for today, tomorrow, and so on, has been planned days or weeks ahead, only the effort planned for today can actually be materialized today. In the next day, the procedure of prediction and planning is repeated, and a new set of efforts is determined. Thus the next day’s action will be taken according to the new planning. This process proceeds continuously until the end of the term
6.3 Fundamentals of MPC
103
Other well-known daily-life analogies in MPC literature include crossing a road and playing chess. In chess, a good player predicts the game a few steps ahead based on the moves of the opponent, and plans a few future moves. However, only one move can actually be applied each time. Based on the following-up move of the opponent, a new set of predictions has to be made and a new set of future moves is determined as a result. This procedure is repeated throughout the game. Analogously, the methodology of all the controllers belonging to the MPC family can be characterized by the following steps [92]: 1. The future outputs for a predetermined horizon N2 , called the prediction horizon, are predicted at each instant t using the process model. These predicted outputs yˆ(t + i|t) for i = 1 . . . N2 depend on the known values up to instant t (past inputs and outputs) and on the future control signals ut+i , i = 0, 1 . . . N2 − 1, which are those to be sent to the process and to be calculated. 2. The set of future control signals is calculated by optimizing a determined criterion in order to keep the process as close as possible to the reference trajectory. The criterion usually takes the form of a quadratic function of the errors between the predicted output signal and the reference trajectory. The control effort is included in the objective function in most cases. An explicit solution can be obtained if the criterion is quadratic, the model is linear and there are no hard constraints; otherwise an iterative optimization method has to be used. Some assumptions about the structure of the future control law are also made in some cases, such as that it will remain constant after a given instant. 3. The control signal ut , i.e., the control action calculated for the current time, is sent to the process whilst the calculated future control actions are discarded because in the next sampling instant yt+1 is already known; step 1 is repeated with the new value and all the sequences are brought up to date. Thus ut+1 is calculated at time instant t + 1 (which in principle will be different from ut+1 calculated at time instant t because of the new information available) using the receding horizon concept.
6.3 Fundamentals of MPC All the MPC algorithms possess common elements and different options can be chosen for each one of these elements, giving rise to different algorithms. These elements are [92]: • prediction model, • objective function, and • algorithms for obtaining the control law. 6.3.1
Process and Disturbance Models
The cornerstone of the MPC is the model. The model generally consists of two parts: process model and disturbance model. While the process model typically
104
6 Model Predictive Control: Conventional Approach
represents input-output relations of a physical plant, the disturbance model is often used to represent the disturbance or simply used to approximate modelplant mismatch. There are different forms of models used in MPC. Some common model structures are illustrated below. Impulse Response Model yt =
∞
fi ut−i
i=1
where, fi is the sampled output when the process is excited by a unit impulse, also known as an impulse response coefficient. Often this sum is truncated and only Ns values are considered, where Ns is the settling time of the process; namely, starting from instant Ns + 1, impulse responses are approximately zero. Thus Ns yt = fi ut−i = (f1 z −1 + f2 z −2 + · · · + fNs z −Ns )ut i=1
Advantages • intuitive and clearly reflects the influence of each manipulated variable; • no a priori information about the process is needed for identification; • allows complex dynamics such as non-minimum phase or delays to be described with ease. Disadvantages • only stable processes without integrators can be represented; • a large number of parameters is necessary.
Step Response Model yt =
∞
gi Δut−i
i=1
where, gi ’s are the sampled output values for the unit step input and Δut = ut − ut−1 . For a stable system, gi will be constant after settling time Ns . As an impulse response coefficient can be considered as the difference between two consecutive step response coefficients, the following relations hold: fi = gi − gi−1 gi =
i
fj
j=1
Step response models have the same advantages and disadvantages of the impulse response models. Transfer Function Model yt =
B(z −1 ) ut A(z −1 )
6.3 Fundamentals of MPC
105
Advantages • valid for unstable processes; • needs fewer parameters. Disadvantages • the structure of the process is fundamental for identification, especially the order of A and B. • some processes may not be described sufficiently by a parametric model with a limited number of parameters. State Space Model xt+1 = Axt + But yt = Cxt + Dut Advantages • multivariate processes can be presented in a straightforward manner. • a large collection of modern control theory and analysis methods can be applied. Disadvantages • the calculations may be complicated with the additional necessity of including an observer if the states are not accessible. • some processes may not be described sufficiently by a parametric model with a limited number of parameters. Time Series Model for the Disturbance A model widely used is the AutoRegressive and Integrated Moving Average (ARIMA) in which the disturbance, i.e., the difference between the measured output and the one calculated by the model, is given by νt =
C(z −1 ) et ΔD(z −1 )
where, Δ = 1 − z −1 , νt is the disturbance, and D and C are often chosen as 1 in practical MPC. 6.3.2
Predictions
Models are generally the vehicles towards control design. In MPC, however, models are not used directly for the control design. Instead, predictors are designed first according to the models, and then the control law is designed according to the predictions. Predictions have been discussed extensively in previous several chapters, where the future outputs are predicted by past inputs and outputs. However, to design an MPC is to determine future inputs in order to drive the process to a desired target. Thus, the future inputs are critical components of the predictors. Furthermore, in MPC the process output does not just follow the setpoint at one specific point, but a trajectory of the setpoint. Thus prediction is not simply one-step ahead but multiple steps ahead.
106
6 Model Predictive Control: Conventional Approach
As an illustration, let’s consider the following simple ARX model: yt = −ayt−1 + bΔut−1 + et
(6.1)
This model is slightly different from the conventional ARX model by replacing ut with Δut . The one-step prediction of the ARX model can be derived straightforwardly by writing Equation 6.1 as yt+1 = −ayt + bΔut + et+1
(6.2)
and then ignoring the future white noise et+1 , following the procedure shown in Chapter 2. This gives us yˆ(t + 1|t) = −ayt + bΔut
(6.3)
Similarly, the two-step prediction can be derived by first writing Equation 6.1 as yt+2 = −ayt+1 + bΔut+1 + et+2
(6.4)
Using Equation 6.2, Equation 6.4 can be written as yt+2 = −a(−ayt + bΔut + et+1 ) + bΔut+1 + et+2 = a2 yt + bΔut+1 − abΔut + et+2 − aet+1 By ignoring future white noises et+1 , et+2 , the two-step ahead prediction can be derived as (6.5) yˆ(t + 2|t) = a2 yt + bΔut+1 − abΔut Combining Equations 6.3 and 6.5, both one- and two-step predictions based on the past input, past output, and future input are obtained: ⎞ ⎞ ⎛ ⎛ ⎞ ⎛ ⎞⎛ yˆ(t + 1|t) b 0 Δut −a ⎠ yt + ⎝ ⎠ ⎝ ⎠=⎝ ⎠⎝ (6.6) Δut+1 yˆ(t + 2|t) −ab b a2 Predictions over additional steps can be derived similarly. Dealing with other model structures such as ARIMAX, BJ, PEM, etc. is, however, more complicated but the idea remains the same. 6.3.3
Free and Forced Response
Predictions, such as Equation 6.6, consist of two terms. The second term (on the right hand side of the equation) depends on future inputs1 . The first term depends only on past outputs (as well as past inputs in general). This fact has been expressed in MPC literature as yˆ(t + j|t) = yf,t+j + yc,t+j
(6.7)
where the two terms on the right hand side are called free response and forced (controlled) response, respectively. The free response, yf,t+j , corresponds to the prediction of the output if the manipulated variable is kept constant at its present value over the prediction horizon. On the other hand, the forced response, yc,t+j , corresponds to the prediction of the output due to the future control actions. 1
At time t, ut is to be determined and therefore, ut is also classified as a future control action.
6.3 Fundamentals of MPC
6.3.4
107
Objective Function
The various MPC algorithms propose different cost functions for obtaining the control law. The general aim is that [92]: • the future output should follow a determined reference signal over the considered horizon; • the control effort necessary for doing so should be considered in the objective function. The general expression for such an objective function is J =
N2
T
[rt+j − yˆ(t + j|t)] Qj [rt+j − yˆ(t + j|t)]
j=N1
+
Nu
[Δut+j−1 ]T Rj [Δut+j−1 ]
(6.8)
j=1
where Qj and Rj are the weighting matrices. Other parameters are discussed below. Prediction Horizon and Control Horizon Prediction starts at N1 , and N2 is the maximum prediction horizon, Nu is the control horizon. N2 − N1 + 1 determines a window of the predictions in which it is desirable for the predicted output to follow the setpoint. Thus, taking a large value of N1 implies that it is not important if there are errors in the first few instants up to N1 . Taking a large value of N2 − N1 + 1, however, implies that output errors are concerned over a long time horizon. Reference Trajectory Reference trajectory is also called setpoint, which is a sequence of future “desired” process outputs. The desired output is not necessarily to be the same as the actual output due to performance limitations of control systems, such as hard constraints on the actuator, time delay of the process, model-plant mismatch etc. However, it is desirable for the actual process output to follow up the desired setpoint eventually. Considering performance limitations, most objective functions use a reference trajectory which does not necessarily have to coincide with the real reference. It is normally a smooth approximation from the current value of the output yt towards the known reference. 6.3.5
Constraints
Constraints are a reality and exist almost everywhere. For instance, no valve can move beyond its operating limits, and most of them cannot allow rapid changes of large magnitude (i.e., large slew rate) or, at least it is not desirable to do so. Many process outputs are also subject to constraints for economic or safety reasons. In chemical reactors, for example, higher temperature may be
108
6 Model Predictive Control: Conventional Approach
desirable for certain reactions but the reactor material has limited temperature tolerance. Too high temperature will cause safety problem. Therefore, in almost all practical model predictive controls, constraints in the amplitude, in the slew rate of the control signal, and in the output are considered: umin ≤ δumin ≤ ymin ≤
ut
≤ umax
ut − ut−1 yˆ(t + j|t)
≤ δumax ≤ ymax
∀t ∀t N1 ≤ j ≤ N2 , ∀t
Note that, although ideally the output constraints should be direct to the actual output, due to the unpredictable disturbances, the actual output can not be directly constrained through the controller. Thus only the predicted output can actually be constrained and, as a result, good predictions are critical in MPC. 6.3.6
Control Law
Control actions, Δut+i , are calculated by minimizing the objective function. To do this the prediction yˆ(t + i|t), such as that in Equation 6.6, are calculated and are substituted in Equation 6.8. By taking derivatives of J with respect to Δut , Δut+1 , . . . , Δut+Nu −1 , and then equating the derivatives to zero, an analytical solution can be obtained . This is a typical least squares problem. However, if there are hard constraints on ut , Δut , or yˆ(t+j|t), analytical solutions are not possible, and numerical optimization is necessary. All computation must be completed within the sampling interval. When numerical optimization is needed, obtaining the solution is not trivial because there will be Nu decision variables in the optimization. The control horizon is used to impose a structure on the control law. Under this concept it is considered that after a certain time window Nu , ut becomes constant or equivalently Δut = 0, i.e., Δut+j−1 = 0
j > Nu
To summarize, the design of a model predictive control involves specification of prediction models, objective functions, and optimization to obtain control laws. To illustrate, a design procedure for dynamic matrix control is elaborated next.
6.4 Dynamic Matrix Control (DMC) DMC was initially developed by Cutler and Ramaker of Shell Oil Co. at the end of the seventies. This algorithm has been widely accepted in industry especially in the petrochemical sector. It was mentioned earlier that any MPC algorithm has three basic elements, namely the prediction model, the objective function and the solution procedure to calculate the control action. In this section we will discuss these elements for a simplified DMC algorithm.
6.4 Dynamic Matrix Control (DMC)
6.4.1
109
The Prediction Model
The process model used in the DMC algorithm is the step response model of the process. Consider an infinite step response model yt =
∞
gk Δut−k
k=1
This can be written as yt =
Ns
gk Δut−k +
∞
gk Δut−k
(6.9)
k=Ns +1
k=1
where the second term of the above equation represents the response due to control actions that were taken from infinite past up to and including time t − Ns − 1. Ns is the settling time in terms of samples. Denote the second term as Zt ; i.e., write Equation 6.9 as yt =
Ns
gk Δut−k + Zt
(6.10)
k=1
In this way the infinite step response model appears to have a finite step response model form, based on which predictions can be made: yˆ(t + p|t) =
Ns
gk Δut+p−k + Zt+p
(6.11)
k=1
As has been discussed, under the MPC design framework, the predicted response is divided into two parts: the free response term and the forced response term. To achieve this, we separate Equation 6.11 into two terms: one term containing future control actions (Δut , Δut+1 , . . .) and the other term containing past control actions (Δut−1 , Δut−2 , . . .): yˆ(t + p|t) =
p
gk Δut+p−k +
k=1
Ns
gk Δut+p−k + Zt+p
k=p+1
We can rearrange this equation and then write it in terms of free and forced response as p ∗ yˆ(t + p|t) = gk Δut+p−k + yt+p (6.12) k=1
where, the term
∗ yt+p =
Ns
gk Δut+p−k + Zt+p
(6.13)
k=p+1
represents the free response, i.e., the response that would be expected if no ∗ future actions are taken. Thus yf,t+p = yt+p . On the other hand the first term
110
6 Model Predictive Control: Conventional Approach
on the right hand side of Equation 6.12 is the forced (controlled) response, i.e., the response due to the future control moves starting from t. Thus this term is yc,t+p under the MPC framework. The objective function in DMC, as in MPC, is to minimize differences between the setpoint and the predicted output over some prediction horizon, say over the next 1 to N2 sample intervals (i.e., choose N1 = 1). The set of predicted output values over this prediction horizon is: ∗ yˆ(t + 1|t) = yt+1 + g1 Δut ∗ yˆ(t + 2|t) = yt+2 + g2 Δut + g1 Δut+1 ∗ yˆ(t + 3|t) = yt+3 + g3 Δut + g2 Δut+1 + g1 Δut+2 .. . ∗ yˆ(t + Nu |t) = yt+N + gNu Δut + gNu −1 Δut+1 + · · · u · · · + g1 Δut+Nu −1 .. . ∗ yˆ(t + N2 |t) = yt+N + gN2 Δut + gN2 −1 Δut+1 + · · · 2 · · · + g1 Δut+N2 −1
Note that Δut+Nu = Δut+Nu +1 = · · · = Δut+N2 −1 = 0 owing to the structural constraint on the future control moves, specified by the control horizon Nu . Define: ⎤ ⎤ ⎤ ⎡ ⎡ ⎡ ∗ yˆ(t + 1|t) yt+1 Δut ⎥ ⎥ ⎢ ⎢ ∗ ⎥ ⎢ ⎥ ⎥ ˆ(t + 2|t) ⎥ ∗ ⎢ yt+2 ⎢ y ⎢ Δut+1 ⎥ ⎥ ⎥ ⎢ ⎢ ⎢ ˆ=⎢ y .. .. ⎥ ⎥ , y = ⎢ .. ⎥ , Δu = ⎢ . ⎦ ⎦ ⎣ . ⎣ . ⎦ ⎣ ∗ yt+N 2
yˆ(t + N2 |t)
Δut+Nu −1
We can express the predicted output using a vector/matrix notation as ˆ = y∗ + GΔu y
(6.14)
where, G is defined as ⎡
g1 g2 .. .
0 g1 .. .
0 0 .. .
⎢ ⎢ ⎢ ⎢ ⎢ G=⎢ ⎢ gNu gNu −1 gNu −2 ⎢ . .. .. ⎢ . ⎣ . . . gN2 gN2 −1 gN2 −2
and is called the dynamic matrix.
··· ··· .. .
0 0 .. .
··· .. .
g1 .. .
· · · gN2 −Nu +1
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
6.4 Dynamic Matrix Control (DMC)
6.4.2
111
Unconstrained DMC Design
Let the setpoint trajectory over the prediction horizon be: ⎡
rt+1
⎤
⎥ ⎢ ⎥ ⎢ r ⎥ ⎢ t+2 ⎥ r=⎢ ⎢ .. ⎥ ⎢ . ⎥ ⎦ ⎣ rt+N2 Then a set of control actions can be solved through the following optimization problem: minimize Δu
ˆ )T Q(r − y ˆ) J(Δu) = (r − y
subject to: ˆ = y∗ + GΔu y ˆ in the objective function yields where Q is a weighting matrix. Substituting y the modified objective function: minimize
J(Δu) = (r − y∗ − GΔu)T Q(r − y∗ − GΔu)
Δu
The optimum value of Δu is obtained by taking derivative of J, i.e., ∂J = −2GT QT (r − y∗ − GΔu) = 0 ∂Δu This gives the optimal sequence of control moves:
−1 T T Δu = GT QT G G Q (r − y∗ ) 6.4.3
Penalizing the Control Action
A more common problem formulation for DMC is to add a control-action related term in the objective function to penalize possible “aggressive” control actions: minimize Δu
ˆ )T Q(r − y ˆ ) + ΔuT RΔu J(Δu) = (r − y
subject to: ˆ = y∗ + GΔu y
112
6 Model Predictive Control: Conventional Approach
Following the same approach as the derivation of the unconstrained controller gives us: ∂J = −2GT QT (r − y∗ − GΔu) + 2RT Δu = 0 ∂Δu which can be simplified to yield: −GT QT (r − y∗ ) + (GT QT G + RT )Δu = 0 Thus, the controller becomes:
−1 T T Δu = GT QT G + R G Q (r − y∗ ) 6.4.4
(6.15)
Handling Disturbances in DMC
Up to now, we have not included a disturbance term in the model. In DMC it is assumed that the disturbance exists but remains constant along the entire prediction horizon at the same value as that at time instant t, i.e., the disturbance is assumed to be a step-type disturbance. Alternatively speaking, we can consider that the disturbance model is specified as 1−z1 −1 . With the consideration of the disturbance, the free response expressed by Equation 6.13 has now an additional term of this disturbance νt+p , i.e., ∗ = Zt+p + yt+p
Ns
gk Δut+p−k + νt+p
k=p+1
Now, for the step-type disturbance, as assumed in DMC, we can calculate the disturbance as νt+p = νt = yt − yˆ(t|t − 1) where, the one step prediction yˆ(t|t − 1) is given by yˆ(t|t − 1) =
∞
gk Δut−k
k=1
Therefore, νt+p = νt = yt −
∞
gk Δut−k
k=1
Since Zt =
∞
gk Δut−k
∞
gk Δut+p−k
k=Ns +1
Zt+p =
i=Ns +1
6.4 Dynamic Matrix Control (DMC)
113
Consequently, ∞
∗ yt+p =
gk Δut+p−k +
i=Ns +1
+yt −
Ns
gk Δut+p−k
k=p+1
∞
gk Δut−k
k=1 Ns
= yt +
gk Δut+p−k
k=Ns +1
k=p+1 ∞
−
∞
gk Δut+p−k +
gk Δut−k
k=1
= yt +
∞
gk Δut+p−k −
k=p+1
∞
gk Δut−k
k=1
By change of the index variable with j = k − p, i.e., k = j + p in the second term on the right hand side of the equation above, we get ∗ yt+p
= yt +
∞
gj+p Δut−j −
j=1
∞
gk Δut−k
(6.16)
k=1
By changing the index variable from j to k in the second term on the right hand side of Equation 6.16 again, we get ∗ yt+p = yt −
∞
(gk − gk+p ) Δut−k
k=1
Steady-state is reached in Ns sample intervals after a step change in the input, which implies gNs +1 = gNs +2 = · · · = gNs +∞ Therefore, we can write the general expression for the DMC free response as ∗ yt+p = yt −
Ns
(gk − gk+p ) Δut−k
(6.17)
k=1
6.4.5
Multivariate Dynamic Matrix Control
Recall that for a SISO system the DMC control problem was stated as: minimize Δu
ˆ ) + ΔuT RΔu ˆ )T Q(r − y J(Δu) = (r − y
subject to: ˆ = y∗ + Sm,l Δu y where the dynamic matrix G has been replaced by Sm,l for the sake of generalization. The subscripts of Sm,l denote that the system has l inputs and m outputs.
114
6 Model Predictive Control: Conventional Approach
Consider a multivariate system with m outputs and l inputs. A set of output variables (yi ) is controlled via a set of manipulated variables (uj ). Then the relationship between the ith output and all inputs (i.e., a MISO system) can be expressed using a finite step response model:
yi,t =
Ns l
gij,k Δuj,t−k + Zi,t ,
i = 1, . . . , m
j=1 k=1
where gij,k is the kth step response coefficient from the jth input to the ith output; yi,t is the ith output at time t, Δuj,t−k is the control move at time t − k in the jth input, and Zi,t is similarly defined as in Equation 6.10. With the finite step response model, following the derivation of Equation 6.12, the ith output prediction yˆi (t + p|t) can be derived as:
yˆi (t + p|t) =
p l
∗ gij,k Δuj,t+p−k + yi,t+p
j=1 k=1
∗ where yi,t+p is the free response of the ith output at time instant t + p. Recall that we have defined the vectors for SISO process and the same notation can be carried to MIMO process. For the ith output and the jth input, let
⎡
yˆi (t + 1|t)
⎤
⎡
∗ yi,t+1
⎤
⎥ ⎥ ⎢ ⎢ ⎥ ⎥ ⎢ ⎢ ∗ ⎥ ⎢ yˆi (t + 2|t) ⎥ ∗ ⎢ yi,t+2 ⎥ ⎥,y = ⎢ ˆi = ⎢ y .. .. ⎥ ⎥ i ⎢ ⎢ ⎥ ⎥ ⎢ ⎢ . . ⎦ ⎦ ⎣ ⎣ ∗ yi,t+N2 yˆi (t + N2 |t) ⎤ ⎤ ⎡ ⎡ ri,t+1 Δuj,t+1 ⎥ ⎥ ⎢ ⎢ ⎥ ⎥ ⎢ ⎢ ⎢ ri,t+2 ⎥ ⎢ Δuj,t+2 ⎥ ⎥ ⎥ ⎢ ⎢ ri = ⎢ .. ⎥ , Δuj = ⎢ .. ⎥ ⎥ ⎥ ⎢ ⎢ . ⎦ . ⎦ ⎣ ⎣ ri,t+N2
Δuj,t+Nu −1
Define vectors with all inputs and outputs: ⎡
⎢ ⎢ ⎢ ˆ=⎢ y ⎢ ⎢ ⎣
ˆ1 y
⎤
⎡
y1∗
⎤
⎡
r1
⎤
⎡
Δu1
⎤
⎢ ⎢ ⎥ ⎥ ⎥ ⎥ ⎢ ⎢ ⎢ ⎥ ⎥ ⎥ ⎥ ⎢ ⎢ r2 ⎥ ⎢ Δu2 ⎥ ⎥ ∗ ⎢ y2∗ ⎥ ⎥,y = ⎢ ⎥,r = ⎢ ⎥ , Δu = ⎢ ⎥ ⎢ .. ⎥ ⎢ .. ⎥ ⎥ ⎢ .. ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎥ ⎢ . ⎥ ⎣ ⎣ ⎦ ⎦ ⎦ ⎦ ⎣ ∗ ˆm y ym rm Δul ˆ2 y .. .
6.4 Dynamic Matrix Control (DMC)
115
The multivariable dynamic matrix Sm,l is now defined as: ⎡
Sm,l
⎢ ⎢ ⎢ =⎢ ⎢ ⎢ ⎣
G11 G12 · · · G1l G21 G22 · · · G2l .. .. .. .. . . . . Gm1 Gm2 · · · Gml
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
Hence, the multivariable dynamic matrix consists of l × m sub-matrices which contain the step response coefficients relating the individual input-output pairs. The sub-matrix for relating the i-th output yi to the j-th input uj is given by: ⎡
gij,1
0
0
⎢ ⎢ g gij,1 0 ⎢ ij,2 ⎢ . . .. ⎢ . .. ⎢ . . Gij = ⎢ ⎢ ⎢ gij,Nu gij,Nu −1 gij,Nu −2 ⎢ ⎢ . .. .. ⎢ .. . . ⎣ gij,N2 gij,N2 −1 gij,N2 −2
···
0
··· .. .
0 .. .
··· .. .
gij,1 .. .
· · · gij,N2 −Nu +1
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
With these definitions, we can write the multivariate output prediction equation: ˆ = y∗ + Sm,l Δu y The multivariate DMC problem can now be stated as: minimize Δu
ˆ )T QT (r − y ˆ ) + ΔuT RΔu J(Δu) = (r − y
subject to: ˆ = y∗ + Sm,l Δu y Following the same approach as the derivation of the univariate DMC solution, the multivariate DMC problem has the solution:
T −1 T Δu = Sm,l QT Sm,l + R Sm,l QT (r − y∗ ) 6.4.6
(6.18)
Hard Constrained DMC
Up to this point we have assumed that any control action we calculate can be implemented (e.g., Δu is unbounded). Also, we have similarly assumed that yˆ(t + j|t) is not restricted. This is rarely the case.
116
6 Model Predictive Control: Conventional Approach
A natural formulation for handling hard constraints is through numerical optimization: minimize Δu
ˆ ) + ΔuT RΔu ˆ )T Q(r − y J(Δu) = (r − y
subject to: ˆ = y∗ + Sm,l Δu y uL ≤ u ≤ uH δuL ≤ Δu ≤ δuH yL ≤ y ˆ ≤ yH There is no analytical solution to this optimization problem. It must be solved numerically at each control interval. 6.4.7
Economic Optimization
In practice, there are often more output variables (controlled variables) than input variables (manipulated variables). It is also possible, however, to have more manipulated variables than controlled variables. This situation could occur when, for example, some controlled variables are constrained variables rather than setpoint tracking variables. As long as these variables are inside constraints, there is no need to “control” them; thus some manipulated variables are released and this may produce “additional” manipulated variables. When extra degree of freedom is available, a steady-state “economic” optimization problem can be used to specify the economic compromises among the process variables. This economic optimization is perhaps the most distinguished feature of the model predictive control technology compared to other modern control technologies. When there are excess input variables, the economic optimization problem is often specified in a linear programming (LP) form: minimize uss
P (uss ) = cTu uss
subject to: rss = Km,l uss + b g(rss , uss ) ≥ 0 uL ≤ uss ≤ uH where Km,l is the gain matrix of the system, rss is steady state setpoint target, and b is a constant vector. The solution to the steady-state economic optimization problem provides steady-state targets uss to which the input variables (u)
6.5 Generalized Predictive Control (GPC)
117
should be driven. The steady-state targets are then introduced into the DMC objective function (for dynamic-layer optimization). These steady state and dynamic optimization problems must be solved online and can be quite computationally expensive, but it is often justified on key process units. In practice, the steady-state economic optimization problem can be solved less frequently than the dynamic one, while the dynamic optimization problem must be solved within a sampling interval. The same economic optimization objective can also apply to the output variables. A typical industrial MPC does economic optimization for both input and output variables, as will be discussed further in Chapter 9.
6.5 Generalized Predictive Control (GPC) GPC and DMC follow the same principles and are two special cases of MPC; thus the MPC framework just introduced applies to both. The main difference between DMC and GPC is: 1) GPC uses parametric process models to derive the predictor; thus a Diophantine equation needs to be solved, while DMC directly uses step response models for predictions. 2) GPC uses a more general disturbance model than the step-type disturbance model of DMC. With the background information just introduced, in this section an MPC solution that handles a more general disturbance model is introduced through a brief overview of GPC. DMC design procedure has been illustrated in the previous section. GPC follows the same principle but with different details. Generalized predictive control or GPC design [106, 7, 8] starts by first identifying an ARIMAX model for the process, expressed as A(z −1 )yt = B(z −1 )ut−1 +
C(z −1 ) et
(6.19)
A, B and C are polynomials in the backshift operator, z −1 , with A and C being monic. = (1 − z −1 ) is the differencing operator. The role of the is to ensure integral action in the controller by including an internal disturbance model of typical load perturbations arising in the process industry [106]. For the sake of simplicity, consider a univariate process as an illustration. A quadratic cost function to be minimized, as a univariate version of Equation 6.8, is J =
N2
i=N1
(rt+i − yˆ(t + i|t))2 +
Nu
λ(ut+i−1 )2
(6.20)
i=1
with λ being the weighting on the control effort. N1 is usually chosen as 1 or the process time delay d. We choose N1 = 1 here for simplicity of presentation. For a discussion on the selection of values for N1 , N2 , Nu and λ, specifically for GPC, readers are referred to [7] and [8].
118
6 Model Predictive Control: Conventional Approach
Using the Diophantine equations [7, 8] Fi C(z −1 ) = Ei + z −i −1 A(z ) A(z −1 )
(6.21)
Ei B = Gi C + z −i Γi
(6.22)
and Equation 6.19, by ignoring the future disturbance term Ei et+i , we obtain the p-step ahead output prediction equation: Fp Γp yt + ut−1 + Gp ut+p−1 C C = Fp ytf + Γp uft−1 + Gp ut+p−1
yˆ(t + p|t) =
∗ = yt+p + Gp ut+p−1
(6.23)
where uft = C −1 ut ytf = C −1 yt and
∗ yt+p = Fp ytf + Γp uft−1
(6.24)
is the free response of the process. Thus the prediction given by Equation 6.23 is the counterpart of DMC prediction given by Equation 6.12, and Equation 6.24 is the counterpart of Equation 6.17 but with a more general noise model used here. Predict yˆ(t+p|t), for p = 1, . . . , N2 , using Equation 6.23. Then the predictions in the vector form can also be formed as ˆ = y∗ + GΔu y
(6.25)
where G is the dynamic matrix containing the step response coefficients of B A derived from Gp of Equation 6.23; y∗ is derived from Equation 6.24. Similar to the derivation of the DMC law of Equation 6.15, the GPC law becomes −1 T
G (r − y∗ ) (6.26) Δu = GT G + λI Other practical issues, such as hard constraints, are dealt with in the same way as that of DMC.
6.6 Summary MPC is the most applied advanced control strategy in process industries. DMC is the first version of MPC and its control philosophy is still being used in modern MPC. Since the 1980s, model predictive control has been studied extensively in both academia and industry. Notable developments have been observed in a variety of models used, higher dimensions of processes considered, robustness
6.6 Summary
119
and stability considerations, etc. Modeling is considered as the most challenging and time consuming work in MPC design. In the next chapter, we shall consider a predictive control design approach that does not need an explicit model but possesses many common MPC features such as tuning of disturbance models, constraint handling etc.
7 Data-driven Subspace Approach to Predictive Control∗
7.1 Introduction A dynamic model of the process is the basic requirement for the design of predictive controllers, and the model is first identified using plant input and output data. Using the process model, predictor matrices are constructed (for example, the dynamic matrix constructed using step response coefficients in DMC [1, 2]), as discussed in Chapter 6. The predictor matrices are used to obtain multistep ahead predictions of the process output(s) and used in the controller design. However these predictor matrices can be directly obtained from the input/output data using subspace matrices, which eliminates the intermediate step of parametric process model identification, providing a means for designing a predictive controller in the generalized predictive controller (GPC) framework (e.g. [16]) or the model predictive control framework, without first identifying a parametric model. Under the subspace framework, since no traditional parametric model is required for the MPC type of controller design, the subspace approach is also referred to as the “model-free approach”, and this term has been adopted in the literature (for example see [14, 15]). The idea explored in this chapter is to obtain the controller matrices used in the predictive controllers directly from process input/output data without the intermediate parametric model identification step. Hence this approach has also been considered as a data-driven approach. Moreover, subspace identification methods involve minimizing the summation of multi-step ahead prediction errors, making the subspace matrices based design approach appealing for predictive control synthesis. The predictive controller based on subspace matrices, discussed in this chapter, uses the same cost-function as MPC and hence an important question is how one obtains the predictions utilized in the cost function. As will be seen, the subspace matrices to be used contain similar information as that of the model used by GPC; thus the analogy to GPC will be the focus of this chapter. One of the key aspects in GPC is the assumption of an ARIMAX model for the process [106, 7]. This requires pre-specification of the order and structure of the model to ∗
This chapter (with revisions) is reprinted from Control Engineering Practice, vol 11, Kadali, R., B. Huang, A. Rossitor, “A Data Driven Subspace Approach to Predictive c 2003 Elsevier Ltd., with permission from Elsevier. Controller Design”, 261-278, Section 7.2 has major revisions.
B. Huang et al.: Dyn. Model. Predict. Ctrl. & Perform. Monitor., LNCIS 374, pp. 121–141, 2008. c Springer-Verlag London Limited 2008 springerlink.com
122
7 Data-driven Subspace Approach to Predictive Control
be identified for controller design. Typically one uses reduced-complexity models which frequently introduce bias errors. Moreover, the model is usually identified in a nonlinear, iterative manner, and in general Diophantine equations need to be solved to obtain the prediction matrices. On the other hand, the predictive controller designed using subspace matrices makes no pre-assumptions about the structure and order of the process model (thus alleviating bias errors). Moreover the prediction matrices are obtained through a single matrix algebraic calculation. In summary, the subspace approach to predictive control will have the key features of GPC [7, 8] such as: (1) long-range prediction over a finite horizon; (2) inclusion of weighting on outputs and control moves in the cost-function; (3) choice of a prediction horizon and a control horizon, after which projected control moves are taken to be zero; and 4) use of a more general noise model. It combines these features with the added advantages of: (1) no pre-assumptions about model order or structure; (2) prediction matrices obtained without iteration; and (3) eliminating the step of solving the Diophantine equations. We also note that extension to the multivariate systems is straightforward with the subspace framework. Although the idea of designing predictive controllers using subspace matrices, such as model-free LQG and subspace predictive controller [12, 13, 14], or using the state space model identified through subspace approach [107, 108, 109], has been around for many years, designing a predictive controller directly from subspace matrices with all the features of the traditional predictive controller has not been investigated. The equivalence of finite horizon LQG to GPC is well known [106]; however, there are several other important issues that need to be addressed in the subspace predictive control framework and they form the main content of this chapter. These issues are: (1) derivation of a predictive control law in the GPC framework (with systematic inclusion of integral actions); (2) extension of the predictive control law to include feedforward control to compensate for measured disturbances; (3) inclusion of a constraint handling facility; and (4) tuning of the noise model. The chapter is arranged as follows. Subspace approach to the predictive controller design with enhanced features is explained in Section 7.2. Inclusion of a noise model for tuning is discussed in Section 7.3. Results from the simulations and actual implementation on a pilot-scale plant using the data-driven subspace predictive control scheme are presented in Section 7.4 and Section 7.5 respectively. The summary is presented in Section 7.6.
7.2 Predictive Controller Design from Subspace Matrices The GPC objective function, Equation 6.20, can also be written as: J = (r − y ˆ)T (r − y ˆ) + uT (λI)u
(7.1)
where the future outputs are over the prediction horizon, t + 1 to t + N2 , and the future incremental inputs are over the control horizon, t to t + Nu − 1. All notations of Equation 7.1 have been defined in Chapter 6. An alternative version
7.2 Predictive Controller Design from Subspace Matrices
123
of objective function without differencing the input in subspace literature is given by ˆ) + uT (λI)u (7.2) J = (r − y ˆ)T (r − y Following the subspace notations of Chapter 3, data column vectors over a horizon for the output and input, respectively, are similarly defined as ⎛ ⎞ yt+1 ⎜ ⎟ .. ⎜ ⎟ ⎜ ⎟ . (7.3) yf = yt+1|t+N2 = ⎜ ⎟ ⎝ yt+N2 −1 ⎠ yt+N2
uf = ut+1|t+N2
⎞ ut+1 ⎟ ⎜ .. ⎟ ⎜ ⎟ ⎜ . =⎜ ⎟ ⎝ ut+N2 −1 ⎠ ut+N2 ⎛
(7.4)
⎞ yt−N ⎜ . ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎟ ⎜ ⎜ y ⎟ =⎜ t ⎟ ⎜ ut−N ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎝ . ⎠ ut
(7.5)
yf = Lw wp + Lu uf + Le ef
(7.6)
⎛
wp = wt−N |t
Analogous to the matrix subspace Equation 3.41, a vector-form subspace equation can be written as
Similarly, analogous to Equation 3.43, the prediction version (by omitting future white-noise disturbances) is given by yˆf = Lw wp + Lu uf
(7.7)
yˆt+1|t+N2 = Lw wt−N |t + Lu ut+1|t+N2
(7.8)
or This is a vector form of predictive subspace representation for the plant model 3.1-3.2. Extract ut from Lw wt−N |t as Lw wt−N |t =
Lw,1 Lw,2
wt−N |t−1 ut
= Lw,1 wt−N |t−1 + Lw,2 ut
(7.9)
124
7 Data-driven Subspace Approach to Predictive Control
Substituting Equation 7.9 in Equation 7.8 yields yˆt+1|t+N2 = Lw,1 wt−N |t−1 + Lw,2 ut + Lu ut+1|t+N2
ut = Lw,1 wt−N |t−1 + Lw,2 Lu ut+1|t+N2 = Lw,1 wt−N |t−1 + Luu ut|t+N2
where Luu =
Lw,2 Lu
(7.10)
Given ut+Nu = ut+Nu +1 = · · · = ut+N2 = 0, Equation 7.10 can be written as yˆt+1|t+N2 = Lw,1 wt−N |t−1 + L∗uu ut|t+Nu −1
(7.11)
where L∗uu is Luu by removing the last N2 − Nu + 1 block columns. For simplicity of presentation, we write Equation 7.11 in the same form as Equation 7.7: yˆf = Lw wp + Lu uf
(7.12)
Here f stands for “future” but with time span from t + 1 to t + N2 for yˆf and from t to t + Nu − 1 for uf ; p in wp stands for “past” but with time span of y backwards from t and time span of u backwards from t − 1. The predictor equation in Equation 7.12 is used in the minimization of the objective function, Equation 7.2, noticing y ˆ = yˆf , u = uf . For consistency of notation, denote r by rf . Equation 7.2 can then be written as J = (rf − yˆf )T (rf − yˆf ) + uTf (λI)uf
(7.13)
To derive the ‘subspace predictive control’ law as in [12], substituting Equation 7.12 in Equation 7.13 and then taking derivative operation computes the optimal future control moves as uf = (λI + LTu Lu )−1 LTu (rf − Lw wp )
(7.14)
For finite {N2 , Nu }, the above control law is called SPC or subspace predictive controller in [12]. As {N2 , Nu } −→ ∞, the above control law becomes an LQG-controller as presented in [13, 14]. However, for implementation on real processes the controller should have an integrator, since the objective function, Equation 7.2, does not admit zero static error in the case of non-zero constant reference unless the open-loop process contains an integrator [106]. Hence we need to use the MPC objective function, with incremental inputs uf or Δu, shown in Equation 7.1. Only one [107, 108, 109] among the several subspacematrices based predictive controller design approaches presented in the literature included an integrator in the predictor equation. In this method, to get an integrator in the predictor, Equation 7.12 is multiplied on both sides with a difference operator, = 1 − z −1 , and then rearranged to get a predictor equation with incremental inputs and outputs. A slightly different subspace identification
7.2 Predictive Controller Design from Subspace Matrices
125
method called DSR [48, 49] is used in this approach. The approach in [17] uses an integrated noise model directly. Since the subspace model in Equation 3.41 is input-output based, it is logical to use a similar technique to that adopted in conventional MPC/GPC [7, 8]. This approach is equivalent to the original MPC/GPC design since the innovations form of the state space representation in Equations 3.1-3.2 combined with integrated noise assumption is equivalent to the ARIMAX representation. 7.2.1
Inclusion of Integral Action
To include integral action in the subspace-matrices based predictive controller, we can rewrite the process representation in Equations 3.1-3.2 as xt+1 = Axt + But + Kat
(7.15)
yt = Cxt + Dut + at
(7.16)
where the noise input at is an integrated white noise, which is common in process industry. Therefore, at+1 = at + et et at =
(7.17) (7.18)
Note that the system considered in Equations 7.15-7.16 is equivalent to an ARIMAX representation, as in Equation 6.19, which is considered in the GPC design. Substituting Equation 7.18 in Equations 7.15-7.16, we obtain ξt+1 = Aξt + But + Ket yt = Cξt + Dut + et
(7.19) (7.20)
where the state ξt = xt − xt−1 . Following the same arguments as the derivation of Equation 7.6, a subspace space version of Equations 7.19-7.20 can be derived as Δyt+1|t+N2 = Lw Δwt−N |t + Lu Δut+1|t+N2 + Le et+1|t+N2 where ⎛
⎛ ⎞ ⎞ Δyt+1 Δut+1 ⎜ ⎜ ⎟ ⎟ .. .. ⎜ ⎜ ⎟ ⎟ ⎜ ⎜ ⎟ ⎟ . . Δyt+1|t+N2 = ⎜ ⎟ , Δut+1|t+N2 = ⎜ ⎟ ⎝ Δyt+N2 −1 ⎠ ⎝ Δut+N2 −1 ⎠ Δyt+N2 Δut+N2 ⎛ ⎞ Δyt−N ⎜ ⎟ .. ⎜ ⎟ . ⎜ ⎟ ⎜ Δy ⎟ t ⎟ Δwt−N |t = ⎜ ⎜ ⎟ ⎜ Δut−N ⎟ .. ⎜ ⎟ ⎝ ⎠ . Δut
(7.21)
126
7 Data-driven Subspace Approach to Predictive Control
By omitting future white-noise disturbances in Equation 7.21, we get the predicted version of the subspace equation: Δˆ yt+1|t+N2 = Lw Δwt−N |t + Lu Δut+1|t+N2
(7.22)
Following the same procedure as the derivation of Equation 7.12, the following prediction equation is obtained Δˆ yf = Lw Δwp + Lu Δuf
(7.23)
where, as usual, f stands for “future” with time span from t + 1 to t + N2 (for Δˆ yf ) and from t to t + Nu − 1 (for Δuf ); p in wp stands for “past” with time span backwards from t (for Δy) and backwards from t − 1 (for Δu). Equation 7.23 can be written as yˆf =
1 1 Lw Δwp + Lu Δuf Δ Δ
Thus, it can be shown that yˆf = yt + ΠLw Δwp + ΠLu Δuf
= yt + LwI Δwp + LuI Δuf
= F + LuI Δuf where
(7.24) (7.25)
⎛
⎞ yt ⎜ ⎟ ⎜ yt ⎟ ⎟ yt = ⎜ ⎜ .. ⎟ ⎝. ⎠ yt
LwI = ΠLw LuI = ΠLu and
⎛
I 0 ... ⎜ ⎜ I I ... Π=⎜ ⎝ ... ... ... I I ...
⎞ 0 ⎟ 0⎟ ⎟ ... ⎠ I
and F is the free response of the process output.
F = yt + LwI Δwp
(7.26)
Note that the matrices LwI and LuI are related in a simple manner to Lw and Lu . Even though LwI and LuI can be alternatively directly identified from the differenced data (i.e., by applying the operator 1 − z −1 to the data), differencing
7.2 Predictive Controller Design from Subspace Matrices
127
data in system identification is not generally recommended. Hence, a simple strategy is to identify Lw and Lu and use these matrices to form LwI and LuI . ˆ = yˆf , the objective function in Equation 7.1 With Equation 7.25, noting y can be expanded as J = (rf − F − LuI uf )T (rf − F − LuI uf ) + uTf (λI)uf Differentiating J with respect to uf and equating it to zero gives the control law uf = (LTuI LuI + λI)−1 LTuI (rf − F )
(7.27)
Only first block row of uf is implemented and the calculation is repeated at each time instant. Note that the above control law derived has a guaranteed integral control action obtained directly from the subspace matrices, without any intermediate parametric model identification step. 7.2.2
Inclusion of Feedforward Control
Measured disturbances are those process input variables which cannot be manipulated for controlling the process outputs. If some of the process disturbances are measurable, then, as shown in Section 5.4, the state space representation of the process in Equations 3.1-3.2 is modified to Equations 5.23-5.24. Further modifying Equations 5.23-5.24 by introducing the differencing operator in a similar way as the derivation of Equations 7.19-7.20 yields ⎛ ⎞ Δut ⎠ + Ket (7.28) ξt+1 = Aξt + B Bm ⎝ Δmt ⎛ ⎞ Δut ⎠ + et Δyt = Cξt + D Dm ⎝ (7.29) Δmt where ξt = xt − xt−1 . Now, analogous to Equation 5.26, subspace representation of Equations 7.287.29 is given by ΔYf = Lbw ΔWpb + Lu ΔUf + Lm ΔMf + Le Ef Omitting the future white-noise term gives us the prediction: ΔYˆf = Lbw ΔWpb + Lu ΔUf + Lm ΔMf where
⎛
ΔYp
⎞
⎟ ⎜ ⎟ ⎜ ΔWpb = ⎜ ΔUp ⎟ ⎠ ⎝ ΔMp
(7.30)
128
7 Data-driven Subspace Approach to Predictive Control
An vector form of Equation 7.30 can be extracted as Δˆ yf = Lbw Δwpb + Lu Δuf + Lm Δmf For predictive control we have the values of measured disturbance only up to the current sampling instant, t, (and do not have the knowledge of the future values of the measured disturbance) i.e., mi for i = t, t − 1, t − 2, ... are known but mt+i for i = 1, 2, ..., N2 are not available. Therefore by ignoring future measurement noises, we may write the prediction expression for Δyf simply as Δˆ yf = Lbw Δwpb + Lu Δuf
(7.31)
Omitting future Δmf can be justified if Δmt is white noise and this is indeed likely to be the case after the differencing of mt . However, if there is some a priori knowledge about Δmt such as its correlation structure, prediction of Δmf is also possible. A procedure similar to that of Equations 7.7-7.12 needs to be performed so that Equation 7.31 can be used for the predictive control design. Again, analogous to the derivation of Equation 7.25, the following equation is obtained: yˆf = yt + LbwI Δwpb + LuI Δuf b
= F + LuI Δuf
(7.32) (7.33)
where LbwI = ΠLbw , LuI = ΠLu , and F b = yt + LbwI Δwpb is the free response. Thus the feedback plus feedforward control law becomes uf = (LTuI LuI + λI)−1 LTuI (rf − F b ) 7.2.3
(7.34)
Constraint Handling
Constraints arise due to physical limitations, quality specifications, safety concerns and limiting the wear of the equipment. One of the main features of MPC, its prediction capability, is useful in anticipating constraint violations and correcting them in an appropriate way [92]. The explicit handling of constraints may allow the process to operate closer to optimal operating conditions [92]. For the constrained case, the computations are more involved. QP-formulation of the constraints is well known and widely available in the literature. Typical process constraints have been discussed in Section 6.3.5. The constraints for the case of predictive controller can be transferred to umin ≤ ut+i ≤ umax i = 0, 1, 2, ..., Nu − 1 δumin ≤ ut+i ≤ δumax i = 0, 1, 2, ..., Nu − 1 ymin ≤ yˆt+i ≤ ymax
i = 1, 2, ..., N2
7.2 Predictive Controller Design from Subspace Matrices
129
Define L1 = U1 =
L2 =
U2 =
L3 =
U3 =
δumin ... ... δumin
T
δumax ... ... δumax ⎞ ⎛ umin − ut−1 ⎟ ⎜ ⎟ ⎜ ... ⎟ ⎜ ⎟ ⎜ ⎠ ⎝ ... umin − ut−1 ⎛ ⎞ umax − ut−1 ⎜ ⎟ ⎜ ⎟ ... ⎜ ⎟ ⎜ ⎟ ⎝ ⎠ ... umax − ut−1 ⎞ ⎛ ymin ⎟ ⎜ ⎜ ... ⎟ ⎟−F ⎜ ⎟ ⎜ ⎝ ... ⎠ ymin ⎛ ⎞ ymax ⎜ ⎟ ⎜ ... ⎟ ⎜ ⎟−F ⎜ ⎟ ⎝ ... ⎠ ymax
T
and ⎛
1 0 ... ⎜ 1 1 ... ⎜ R=⎜ ⎝ ... ... ... 1 1 1
⎞ 0 0⎟ ⎟ ⎟ ... ⎠ 1
where F is the free response vector. The constraints can be rewritten as: L1 ≤ uf ≤ U1 L2 ≤ Ruf ≤ U2 L3 ≤ LuI uf ≤ U3 These constraints can then be combined to the form of a single matrix inequality: Auf ≤ B
(7.35)
130
7 Data-driven Subspace Approach to Predictive Control
with
⎞ −I ⎟ ⎜ ⎜ −R ⎟ ⎟ ⎜ ⎜ −LuI ⎟ A=⎜ ⎟ ⎜ I ⎟ ⎟ ⎜ ⎝ R ⎠ LuI ⎛
⎛
⎞ −L1 ⎜ ⎟ ⎜ −L2 ⎟ ⎜ ⎟ ⎜ −L3 ⎟ B=⎜ ⎟ ⎜ U1 ⎟ ⎜ ⎟ ⎝ U2 ⎠ U3
With the above formulation of process constraints, the optimization problem for a predictive controller takes the form of a standard quadratic programming (QP) formulation and the optimization is done numerically. The quadratic program solved at every instant is min J = min[(rf − F )T (rf − F ) + uTf (LTuI LuI )uf uf
uf
−2(rf − F )T LuI uf + uTf (λI)uf ] = min[uTf (LTuI LuI + λI)uf − 2(rf − F )T LuI uf ] uf
= min[uTf Puf + C T uf ] uf
s.t.
Auf ≤ B
(7.36) (7.37)
where P = LTuI LuI + λI C=
−2LTuI (rf
− F)
(7.38) (7.39)
The optimization of the above QP formulation is carried out by means of the standard commercially available optimization QP code at each sampling instant and then the value of ut is sent to the process. Though computationally more involved than the unconstrained algorithms, the flexible constraints-handling capabilities of predictive controllers are very attractive for practical applications, since the economic operating point of a typical process unit often lies at the intersection of constraints [110, 92]. For more discussion on other types of constraints, for example soft constraints or specifications on the process response characteristics, readers are referred to [92, 97, 5, 6] and the references therein.
7.3 Tuning the Noise Model The disturbance dynamics of industrial processes may change with time frequently. Tuning of the noise model is a key feature of predictive controller formulations like general MPC or GPC [106]. It is necessary to incorporate such a feature in the predictive controller derived from subspace matrices. For this reason we need to separate the state space model of the system into two parts, a
7.3 Tuning the Noise Model
131
deterministic part and a stochastic part, which are similar to the process model and noise model in an equivalent input-output transfer function framework as yt = ytd + yts = [C(zI − A)−1 B + D]ut + [C(zI − A)−1 K + I]et
(7.40)
It can be observed that both deterministic and stochastic parts have the same poles. Hence an equivalent representation for the above equation in the discrete transfer function domain would be an ARMAX model, which is a special case of the general linear model: yt = Gp (z −1 )ut + Gl (z −1 )et where the transfer functions Gp (z −1 ) and Gl (z −1 ) have the same denominators for the ARMAX model. Detailed definition of various model structures can be found in Chapter 2. The noise (or disturbance) model in Equation 7.40 can be written as [C(zI − A)−1 K + I] = I + CKz −1 + CAKz −2 + ...
(7.41)
Thus an assumption that the noise model Gl (z −1 ) = 1 would be equivalent to assuming that the Kalman gain matrix, K, equals zero. Therefore if the user desires to change the stochastic part of the identified innovation model, without changing the deterministic model and hence without changing the system matrices C and A, the only way to do it is by changing the Kalman gain matrix, K. Suppose that a new Kalman gain matrix is represented as K ∗ , which is a matrix for a multiple-output system, and a vector for a single-output system. K ∗ can be expressed as K∗ = K + K
(7.42)
We can then write the new noise model as [C(zI − A)−1 K ∗ + I] = I + CK ∗ z −1 + CAK ∗ z −2 + ... = I + C(K + K )z −1 + CA(K + K )z −2 + ...
= [C(zI − A)−1 K + I] + [CK z −1 + CAK z −2 + ...]
(7.43)
The stochastic part of the output with the new Kalman gain matrix is (yts )∗ = [C(zI − A)−1 K ∗ + I]et
= [C(zI − A)−1 K + I]et + [CK z −1 + CAK z −2 + ...]et
= yts + [CK z −1 + CAK z −2 + ...]et ⎞ et−1 ⎟ ⎜ e ⎜ t−2 ⎟ ≈ yts + CK CAK ... CAN −1 K ⎜ ⎟ ⎝ ... ⎠ et−N ⎛
= yts + γN ep
(7.44)
132
7 Data-driven Subspace Approach to Predictive Control
where
⎞ et−1 ⎟ ⎜ ⎜ et−2 ⎟ ⎟ ep = ⎜ ⎜ .. ⎟ ⎝ . ⎠ ⎛
et−n
and γN =
CK CAK ... CAN −1 K
can be calculated from the elements of the estimated ΓN matrix and a user specified gain matrix K . Estimation of ΓN has been discussed in Chapter 3. However N here should be chosen sufficiently large so that CAN −1 → 0 to justify the truncation used in deriving Equation 7.44. Since knowledge of the state space system matrices A and C is not required, the new stochastic model can be incorporated in a model-free manner. Now the prediction with the “tuned” noise model can be written as (ˆ yt )∗ = yˆt + γN ep
(7.45)
where γN can be considered as a vector of impulse response coefficients (Markov parameters for the multivariate systems) of the tuned component of the new noise model. Noise model tuning is often used as a tool to compensate for the process-model mismatch resulting from changes of the process from time to time or simply as tuning parameters. ep contains the past prediction errors and can be estimated from the data as one step ahead prediction errors. In essence adding the term γN ep is equivalent to filtering the past prediction errors. Hence K is used as a tuning parameter and is chosen in such a way that it minimizes the prediction errors. Thus incorporating a new noise model simply involves the addition of a new term in the calculation of the free response of the process. Hence the free response calculation, for example Equation 7.26, is modified as F = yt + LwI Δwp + Υ ep
(7.46)
where Υ is a left-upper triangular matrix constructed from the elements of γN . ⎛ ⎞ γN (1) γN (2) ... γN (N − 1) γN (N ) ⎜ ⎟ γN (N ) 0 ⎟ ⎜ γ (2) γN (3) ... Υ =⎜ N ⎟ ⎝ ... ... ... ... ... ⎠ γN (N2 ) ... γN (N ) ... 0
where γN (i) is the ith block column in γN .
7.4 Simulations The data-driven subspace predictive control design method is tested in simulations. The system example is taken from MATLAB /MPC toolbox working example.
7.4 Simulations
⎛ ⎝
y1 (s) y2 (s)
⎞
⎛
⎠=⎝
12.8e−s −18.9e−3s 16.7s+1 21.0s+1 6.6e−7s −19.4e−3s 10.9s+1 14.4s+1
⎛
+⎝
⎞⎛ ⎠⎝
3.8e−8s 14.9s+1 4.9e−3s 13.2s+1
⎞
u1 (s) u2 (s)
133
⎞ ⎠
⎛
⎠ w(s) + ⎝
e1 (s) e2 (s)
⎞
(7.47)
⎠
Open-loop input/output data is obtained by exciting the open-loop system using a designed RBS signal of magnitude 1 for the inputs, ut , and random numbers of standard deviation 0.1 for the white noise sequences, et , in MATLAB and Simulink . A random-walk signal is designed for the measured disturbance wt by passing a white-noise signal of standard deviation 0.1 through an integrator. Sampling interval is taken as 2 units of time. Using subspace identification, with N = 50 (row blocks) and j = 2000 (column blocks) in the data Hankel matrices, the subspace matrices Lw , Lu and Lw are identified. The simulation data and the models from subspace matrices are plotted in Figures 7.1-7.2. As can be seen from Figure 7.2, the impulse response models from the identified subspace matrices match well with the true impulse response models. Data from simulations
U1
1 0 −1 0
1000
2000
0
1000
2000
0
1000
2000
0
1000
2000
U2
1 0 −1
W
5 0 −5
Y1
50 0 −50
Y2
50 0 −50
0
1000 2−time units samples
2000
Fig. 7.1. Inputs, measured disturbance and outputs data from simulations
In Figure 7.3 the simulation results with subspace based predictive controller without an integrator (SPC in [12]) is compared with the predictive controller with integral actions. As illustrated, the controller with integrator gives no offset
134
7 Data-driven Subspace Approach to Predictive Control
1.5
0 −0.5 G12
G11
1
−1
0.5 −1.5 0
0
20
40
60
80
100
0
20
40
Lags
60
80
100
60
80
100
60
80
100
Lags 0
1 G22
G21
−0.5 0.5
−1 −1.5 −2
0
0
20
40
60
80
−2.5
100
0
20
40
Lags
Lags 0.6
0.3
G23
G13
0.4
0.2
0.2
0.1 0
0.4
0
20
40
60
80
0
100
0
20
Lags
40 Lags
Fig. 7.2. Comparison of the process and noise models from subspace matrices with the true models
1 Y2
Y1
1
0.5
0
0.5
0 0
100
200
300
400
0
100
200
300
400
0
100
200
300
400
0
100
200 300 Samples
400
0.1
0.2 U2
U1
0.1 0 −0.1
0 −0.1
−0.2 0
100
200
300
−0.2
400
0.2
0.05 0 Δ U2
Δ U1
0.1 0 −0.1
−0.05 −0.1
−0.2 0
100
200 300 Samples
400
Fig. 7.3. Predictive controller without and with the integrator. wt = 0; et =0
7.4 Simulations
1 0.5
0.5
Y2
Y1
1
135
0 −0.5
0 0
100
200
300
−1
400
0
100
200
300
400
0
100
200
300
400
0
100
200 300 Samples
400
0.1 0.2
0 U2
U1
0 −0.2
−0.2 −0.3
0
100
200
300
−0.4
400
0.2
0.05
0.1
0 Δ U2
Δ U1
−0.4
−0.1
0
−0.05
−0.1
−0.1
−0.2 0
100
200 300 Samples
400
Fig. 7.4. Predictive controller without and with the feedforward control. et =0
1 Y2
Y1
1 0.5 λ=2 0 100
200
300
400 0.2
0.2
0.1 U2
0.4
0 −0.2 −0.4
λ=2
λ=25
0 0
U1
0.5
λ=25
0
100
200
300
400
0
100
200
300
400
0
100
200 300 Samples
400
0 −0.1 −0.2
0
100
200
300
400
0.5
Δ U2
Δ U1
0.1
0
0 −0.1 −0.2
−0.5
0
100
200 300 Samples
400
Fig. 7.5. Variation of input weighting, λ. wt = 0; et =0
7 Data-driven Subspace Approach to Predictive Control
1 Y2
Y1
1
0.5
0.5 m=10
m=1 m=10
m=1
0 0
100
200
300
0 400
0
100
200
300
400
0
100
200
300
400
0
100
200 300 Samples
400
0.1
0.2
0
U2
U1
0.1 0 −0.1
−0.1
−0.2 100
200
300
−0.2
400
0.2
0.1
0
0
Δ U2
Δ U1
0
−0.2 −0.4
−0.1
0
100
200 300 Samples
−0.2
400
Fig. 7.6. Variation of control horizon, Nu . wt = 0; et =0
1 Y2
Y1
1 0.5 n=3
0.5
n=3
n=20
n=20
0
0 0
100
200
300
400
0
100
200
300
400
0
100
200
300
400
0
100
200 300 Samples
400
0.2 0.2
0.1 U2
U1
0.1 0 −0.1
0 −0.1
−0.2 0
100
200
300
−0.2
400
0.1 0.2 0.05 Δ U2
0.1 Δ U1
136
0
0 −0.05
−0.1 −0.1 −0.2 0
100
200 300 Samples
400
Fig. 7.7. Variation of prediction horizon, N2 . wt = 0; et =0
7.4 Simulations
1
Y2
Y1
1
0.5
0
0.5
0 0
100
200
300
400
0
100
200
300
400
0
100
200
300
400
0.1 0.2 0.05 U2
U1
0.1 0
0 −0.05
−0.1 −0.1 −0.2 0
100
200
300
400
0.02
0.01 Δ U2
Δ U1
0
|Δ U|≤ 0.006
|Δ U|≤ 0.012
|Δ U|≤ 0.016
|Δ U|≤ 0.02
0.01
0
−0.01 −0.01
−0.02 0
100
200 300 Samples
400
0
100
200 300 Samples
400
Fig. 7.8. Constrained predictive controller. wt = 0; et =0
1 Y2
Y1
1 0.5 0
0 0
100
200
300
400
0.4
0.2
0.2
0.1 U2
U1
0.5
0 −0.2 −0.4
0
100
200
300
400
0
100
200
300
400
0
100
200 300 Samples
400
0 −0.1 −0.2
0
100
200
300
400
0.5
Δ U2
Δ U1
0.1
0
0 −0.1 −0.2
−0.5
0
100
200 300 Samples
400
Fig. 7.9. Noise model tuning for model mismatch. wt = 0; et =0
137
138
7 Data-driven Subspace Approach to Predictive Control
for non-zero setpoints. In Figure 7.4 the predictive controller performance is compared for the cases without and with feedforward control. Better controller performance has been achieved with feedforward control. For a range of values for λ, N2 , Nu and constraints on the input moves, u, a subspace-matrices based predictive controller is implemented on the above process in MATLAB and Simulink . The closed-loop system responses for different sets of tuning parameters are illustrated in Figures 7.5-7.8. In Figure 7.5 it can be seen that as the weighting, λ, on the input increases the controller response becomes less aggressive. For a given prediction horizon, as the control horizon, Nu , increases, the controller gives more aggressive tracking performance as shown in Figure 7.6. For a given control horizon, as the prediction horizon, N2 , increases, the controller gives better setpoint tracking performance shown in Figure 7.7. Figure 7.8 shows the setpoint tracking under different constraints on the incremental control moves, u. It can be seen that smaller the magnitude of the maximum allowed control moves, more sluggish is the controller response to setpoint changes. Noise Model Tuning To illustrate the tuning of the noise model with the subspace approach, consider that the process model changes with time; in other words there is a mismatch between the true process model and the identified process model used in the controller design. The model-plant mismatch transfers to additional unknown “disturbances”. Consider the case when the process model from Equation 7.47 changes to ⎛ ⎝
y1 (s) y2 (s)
⎞
⎛
⎠=⎝
12.8e−s −18.0e−3s 16.7s+1 21.0s+1 8e−7s 10.9s+1
−18e−3s 14.4s+1
⎛
+⎝
⎞⎛ ⎠⎝
3.8e−8s 14.9s+1 4.9e−3s 13.2s+1
u1 (s) u2 (s)
⎞
⎞ ⎠
⎛
⎠ w(s) + ⎝
e1 (s) e2 (s)
⎞ ⎠
Figure 7.9 illustrates the controller response without and with the on-line noise model tuning feature.
7.5 Experiment on a Pilot-scale Process The data-driven subspace predictive controller is tested on a multivariate pilotscale system. The system considered, shown in Figure 7.10, is a three tank system with two inlet water flows. The levels of tank-1 and tank-2 are the two controlled variables (CVs). The setpoints (SPs) for the flow rates through the valves-A & B are the two manipulated variables (MVs). The flow rates through the valves-A & B are controlled through local PID controllers on each valve. The setpoints for the flow rates come from a higher-level advanced controller application. The local
7.5 Experiment on a Pilot-scale Process
139
PID controllers, which are univariate, are at faster sampling rate (1 second). The higher level controller, which is multivariate and does computations to minimize an optimization objective function, sends controller outputs every 6 seconds. The system is configured so as to emulate a typical multivariate system in the industry. Valve-A
Water flow Local CASCADE-B
A/D and D/A converter
SP
FC
FI
SP
FI
LI
Valve-B
FC
Local CASCADE-A
LI
TANK-1
TANK-2
TANK-3
Valve-C Drain
Drain
Fig. 7.10. Experimental setup
Tank-3 and valve-C are used primarily to introduce interactions between the variables in the system. As can be seen in Figure 7.10 a change in the level in tank-1 affects the level in tank-2 via tank-3 level. The degree of interaction can be manipulated by changing the valve-C position. If the valve-C is completely closed then the level in tank-2 is independent of the level in tank-1 (zero interactions). By opening the valve-C interactions are introduced in the tank-2 level. Valve-C is maintained at a fixed open position throughout the exercise. Note that the level in tank-1 is independent of the levels in tanks-2 & 3. The step response models for the system, which are formed from the impulse response coefficients in the subspace matrices, are plotted in Figure 7.11. The interactions between the variables are clear from the step response plots. Open-loop step-test data for the system is collected by sending two uncorrelated PRBS (pseudo random binary signals) for the SPs of the flow rates through valves-A & B. Subspace matrices are identified using the open-loop data. A multivariate subspace-matrices based predictive controller is then designed for the system. The controller parameters (weighting matrices, prediction horizon, control horizon and noise model) are tuned for a smooth control response. The closed-loop response for the unconstrained and constrained (|Δu| ≤ 0.5) cases are plotted in Figure 7.12 and Figure 7.13 respectively.
140
7 Data-driven Subspace Approach to Predictive Control
0.02 0 Tank−1 level and U2
4
−0.02
3
−0.04
2
−0.06
1
−0.08
0
−0.1
−1
−0.12
0
50
100
150
200
1.4
1
1.2
0
50
100
150
200
0
50
100
150
200
2
1.2
Tank−2 level and U
Tank−2 level and U1
Tank−1 level and U
1
5
0.8 0.6 0.4 0.2 0
−0.2
1 0.8 0.6 0.4 0.2
0
50
100
150
0
200
Fig. 7.11. Step response coefficients from the subspace matrices identified using the open-loop data 3 Tank−2 level
Tank−1 level
3 2 1 0 200
400
600
800
0 200
400
600
800
1000
200
400
600
800
1000
200
400
600
800
1000
18
U2 (SP for Valve−B)
18
1
−1
1000
16
16
14
14
12
12
1
U (SP for Valve−A)
−1
2
10
10
200
400
600
800
1000
1
0.5 Δ U2
ΔU
1
0.5 0
0
−0.5 −1
−0.5 200
400
600
800
1000
Fig. 7.12. Subspace based predictive controller on the pilot scale process
7.6 Summary
3 Tank−2 level
Tank−1 level
3 2 1 0 200
400
600
800
0 200
400
600
800
1000
200
400
600
800
1000
200
400
600
800
1000
18 16
16
14
14
12
12
10
10
200
400
600
800
1000
Δ U2
0.5
1
0.5
ΔU
1
U2 (SP for Valve−B)
18
2
−1
1000
1
U (SP for Valve−A)
−1
141
0
−0.5
0
−0.5 200
400
600
800
1000
Fig. 7.13. Subspace based predictive controller for the constrained case on the pilot scale process
7.6 Summary In this chapter, the design of the predictive controller, in the MPC/GPC framework, using the subspace matrices calculated through the subspace projection method is addressed. Important issues in practical implementation of the predictive controllers such as integral actions, constraint handling and feedforward control are discussed. It has been shown that the noise model can be independently specified by the user through the addition of a new term to the predictor equation in the model-free manner, which is shown to be equivalent to changing the Kalman filter gain matrix. The data-driven subspace predictive controller is tested on multivariate systems in simulations and on a pilot scale process.
8 Control Loop Performance Assessment: Conventional Approach
8.1 Introduction Established in the later 1980s by the work of Harris [19], the research on control loop performance monitoring has been and remains one of the most active research areas in process control community. A number of new approaches have been developed since then [22, 21, 20]. It is estimated that several hundreds of papers have been published on this or related topics [111]. On the practical side, Eastman Kodak recently reported regular loop monitoring on over 14000 PID loops [111], and some commercial control performance assessment software including multivariate performance assessment has also come available. Harris (1989) [19] developed a performance assessment technique using only routine operating data for univariate control loops. Assuming that the control objective is to reduce process variance, minimum variance control is used naturally as the benchmark standard against which current control loop performance is assessed. It has been shown [19, 22] that for a system with time delay d, a portion of the output variance is feedback control invariant and can be estimated from routine operating data. This portion of the output variance equals the variance achieved under the minimum variance control; thus the method for the estimation of the minimum variance from routine operating data is established. To separate this feedback-control invariant term, one needs to model the closed-loop output data yt by a moving average (MA) model such as yt = f0 et + f1 et−1 + · · · + fd−1 et−(d−1) +fd et−d + fd+1 et−(d+1) + · · · ymv
where et is a white noise sequence and ymv is the portion representing the minimum variance control output, independent of feedback control [22]. This portion of the output can be estimated by time series analysis of routine closed-loop operating data, and its variance is used subsequently as a benchmark measure of theoretically achievable absolute lower bound of output variance to assess control loop performance. Using minimum variance control as the benchmark does not mean that one has to implement such a controller on the actual process. B. Huang et al.: Dyn. Model. Predict. Ctrl. & Perform. Monitor., LNCIS 374, pp. 145–155, 2008. c Springer-Verlag London Limited 2008 springerlink.com
146
8 Control Loop Performance Assessment: Conventional Approach
This benchmark control may or may not be achievable in practice depending on several physical constraints [22]. However, as a benchmark, it provides useful information such as how good the current controller performance is compared to the minimum variance controller and how much potential there is to improve controller performance further. If the controller indicates a good performance measure relative to minimum variance control, further tuning or re-designing of the control algorithm is neither necessary nor helpful. In this case, if further reduction of process variation is desired, implementation of other strategies such as feedforward control or re-engineering of the process itself may be necessary. On the other hand, if the controller indicates a poor performance measure, further analysis, such as model analysis, robustness analysis, constraint analysis, etc., may be necessary, since the poor performance may be due to soft or hard constraints, such as poorly damped zeros or hard constraints on control actions. As an introduction to feedback control performance assessment, problems and solutions for both SISO processes and MIMO processes are reviewed in a tutorial form and key MATLAB programs are illustrated through examples. Based on the authors’ experience in the applications of control performance assessment algorithms, we believe that the procedure introduced in this chapter is intuitive and easy to implement. Other advanced algorithms may also be found from [112, 22, 25, 27]. For the detailed theory behind this chapter, readers are referred to [22]. The remainder of this chapter is organized as follows: In Section 8.2 the feedback-control invariant term is derived and extraction of the minimum variance from time series analysis of routine operating data for SISO process is presented. The treatment of MIMO process is discussed in Section 8.3, followed by summary in Section 8.4.
8.2 SISO Feedback Control Performance Assessment Consider a SISO process under feedback control shown in Figure 8.1, where Gp is ˜ p , where d is the time-delay, the plant transfer function. Furthermore, Gp = z −d G ˜ p is the delay-free plant transfer function. Gl is the disturbance transfer and G function, et is a white noise sequence with zero mean, and Gc is the controller transfer function. It follows from Figure 8.1 that yt =
Gl Gl et = e −d ˜ p Gc t 1 + Gp Gc 1+z G
(8.1)
Using the Diophantine identity: Gl = f0 + f1 z −1 + · · · + fd−1 z −d+1 +Rz −d F
where fi (for i = 1, · · · , d − 1) are impulse response coefficients of Gl , and R is the remaining rational, proper transfer function, Equation 8.1 can be written as
8.2 SISO Feedback Control Performance Assessment
147
et
GL ut -
yt
Gp
Gc
Fig. 8.1. Schematic diagram of SISO process under feedback control
F + z −d R e ˜ p Gc t 1 + z −d G ˜ p Gc R − FG = [F + z −d ]et ˜ p Gc 1 + z −d G
yt =
= F et + Let−d
(8.2)
˜ G R−F G
where L = 1+z−d G˜p Gc is a proper transfer function. Since F et = f0 et + · · · + p c fd−1 et−d+1 , which is independent of white noise occurring before t − d + 1, the two terms on the right hand side of Equation 8.2 are independent, and as a result, V ar(yt ) = V ar(F et ) + V ar(Let−d ) Accordingly, V ar(yt ) ≥ V ar(F et ) The equality holds when L = 0, i.e., ˜ p Gc = 0 R − FG This yields the minimum variance control law: Gc =
R ˜pF G
Since F is independent of the controller transfer function Gc , the term F et , which is the process output under minimum variance control, is feedback-control invariant. Therefore, if a stable process output yt is modeled by an infinite movingaverage model, then its first d terms constitute an estimate of the minimum variance term F et . The estimation procedure is elaborated below. Consider that a set of representative closed-loop routine operating data is available, y1 , y2 , . . . , yN . Using time series analysis, a model in the difference equation form can be built: yt + a ˆ1 yt−1 + · · · + a ˆp yt−p = et + cˆ1 et−1 + · · · + cˆγ et−γ
(8.3)
148
8 Control Loop Performance Assessment: Conventional Approach
With backshift operator, z −1 , this difference equation can be written as a discretetime transfer function yt 1 + cˆ1 z −1 + . . . + cˆγ z −γ = et 1+a ˆ1 z −1 + . . . + a ˆp z −p
(8.4)
Equation 8.4 is an estimate of Equation 8.1. Owing to the feedback-invariance property, after long division of Equation 8.4: yt = fˆ0 + fˆ1 z −1 + . . . + fˆd−1 z −d+1 + . . . et the first d terms of this long division result, fˆ0 , fˆ1 , . . . , fˆd−1 , are the estimates of f0 , f1 , . . . , fd−1 respectively. Thus minimum variance can be calculated as 2 2 = V ar[(fˆ0 + fˆ1 z −1 + . . . + fˆd−1 z −d+1 )et ] = [fˆ02 + fˆ12 + . . . + fˆd−1 ]σe2 σ ˆmv
where σe2 can be estimated as the variance of the residuals after fitting Equation 8.3. Example 8.1. Consider following SISO process, as used by Desborough and Harris(1992) [113], with time delay d = 2: yt = ut−2 +
1 − 0.2z −1 et 1 − z −1
(8.5)
For a simple integral controller Δut = −Kyt , it can be derived that the closed-loop response is given by yt = et + 0.8et−1 +
0.8(1 − K/0.8 − Kz −1) et−2 1 − z −1 + Kz −2
(8.6)
where σe2 = 1. Note that the first two terms are independent of K and represent the process output under minimum variance control. Thus, the theoretical (exact) minimum variance can be calculated as 2 = V ar(et + 0.8et−1 ) = (1 + 0.82 )σe2 = 1.64 σmv
(8.7)
Note that this minimum variance is independent of the controller parameter K. Let K = 0.5 for example. A set of data (8000 data points) can be simulated from the closed-loop process, and a time series model is identified as 1.001 − 0.1988z −1 + 0.01122z −2 yt = et 1 − 0.9859z −1 + 0.5024z −2
(8.8)
with a variance of the fitting residuals as 1.0011. Using a long division or using the MATLAB function impulse, an infinite impulse response model or moving average model is obtained as yt = 1 + 0.7873z −1 + 0.2850z −2 + . . . et
8.2 SISO Feedback Control Performance Assessment
149
Thus the minimum variance can be calculated as 2 = (1 + 0.78732)σe2 σ ˆmv
where σe2 can be estimated from the variance of the residuals, which is 1.0011. Therefore, 2 = 1.6216 (8.9) σ ˆmv The actual variance of data can be calculated as σy2 = 1.8363. Although this actual variance can be calculated directly from data, it is recommended to use the estimated time series model (Equation 8.8), variance of the residuals σe2 , and MATLAB function covar to estimate the variance of data, σy2 . As a result, the performance index is η=
1.6216 = 0.8831 1.8363
A new set of data corresponding to K = 0.9 is simulated again. The variance is calculated as σy2 = 6.3046. A different time series model estimated from the closed-loop data is yt 1.001 − 0.2165z −1 + 0.00228z −2 = et 1 − 1.003z −1 + 0.9038z −2 with variance of the residuals as 1.0010. The impulse response model can be calculated as yt = 1 + 0.7868z −1 − 0.1168z −2 + . . . et and the minimum variance can be estimated as 2 σ ˆmv = 1.6207
(8.10)
The performance index is calculated as η=
1.6207 = 0.2571 6.3046
Comparing Equations 8.9 and 8.10, one can see that the minimum variance is the same despite different control parameters used and thus it is indeed feedbackcontrol invariant. The following is an algorithm for SISO feedback control performance assessment using MATLAB : Algorithm 8.2. SISO feedback control performance assessment: 1. Sample representative routine operating data from the closed-loop system to be evaluated, when there is no setpoint change. Subtract the mean from the data.
150
8 Control Loop Performance Assessment: Conventional Approach
2. Perform time series analysis of this set of data using MATLAB functions such as n4sid and refining it by pem to identify a time series model. Make sure to find a stable model as the closed-loop system is stable. Several iterations of modeling procedures may be necessary for an appropriate model. ˆ cl . Let’s call this estimated time series model G 3. Obtain time delay, d, according to a priori process knowledge. ˆ cl to an impulse response model using MATLAB functions such 4. Expand G as impulse so that ˆ cl = fˆ0 + fˆ1 q −1 + . . . + fˆd−1 q −d−1 + . . . G 5. Obtain the variance of the noise (residuals), σe2 , directly from the model object after applying the MATLAB function n4sid or pem. 6. Calculate the minimum variance as 2 2 = (fˆ02 + fˆ12 + . . . + fˆd−1 )σe2 σ ˆmv
7. Estimate the actual variance σy2 using MATLAB functions such as covar ˆ cl and σe2 as input arguments. with G 8. The performance index can be calculated as η=
2 σ ˆmv σy2
8.3 MIMO Feedback Control Performance Assessment Huang et al. (1997) [114] developed a filtering and correlation analysis algorithm (FCOR) for MIMO feedback control performance assessment. Harris et al. (1996) [112] developed a parallel approach based on a spectral interactor and spectral factorization, which can also extract the minimum variance term from routine operating data. Since then Ko and Edgar (2001) [25] have developed an algorithm that avoided the direct use of the interactor matrix by imbedding calculation of the interactor matrix into algebraic manipulation of Markov parameters of the process model. McNabb and Qin (2003) [27] developed an alternative approach for minimum-variance benchmarking based on state space models estimated from subspace identification. Huang et al. (2005) [115] proposed an estimation of bounds on the minimum variance without knowing the interactor matrix, but the order of the interactor matrix is required. In this section conventional MIMO feedback control performance assessment is introduced in a tutorial form. A MIMO process can be modeled as yt = Gp ut + Gl et ,
(8.11)
where Gp and Gl are proper, rational transfer function matrices for the plant and noise, respectively; yt is an output vector and ut an input vector. et represents a white noise vector with a zero mean and a covariance matrix of Σe .
8.3 MIMO Feedback Control Performance Assessment
151
Furthermore, if Gp is a proper, full-rank transfer function matrix, a unitary interactor matrix D can be factored out such that DGp = G˜p , where G˜p is the delay-free transfer function matrix of Gp . Therefore, Equation 8.11 can be expressed as (8.12) yt = Gp ut + Gl et = D−1 G˜p ut + Gl et . Pre-multiplying both sides of Equation 8.12 by z −d D, where d is the order of the interactor matrix D and is defined as the maximum power of z in D, gives z −d Dyt = z −d G˜p ut + z −d DGl et .
(8.13)
Let y˜t = z −d Dyt and G˜l = z −d DGl , Equation 8.13 becomes y˜t = z −d G˜p ut + G˜l et .
(8.14)
Huang and Shah (1999) [22] showed that since D is a unitary interactor matrix, the minimum variance control law that minimizes the objective function of the ytT y˜t ), also minimizes the objective interactor-filtered variable y˜t , namely J1 = E(˜ function of the original variable yt , J2 = E(ytT yt ), and J1 = J2 , which means that E(˜ ytT y˜t ) = E(ytT yt ). It is further shown that if one writes G˜l as G˜l = F0 + F1 z −1 + . . . + Fd−1 z −d+1 + z −d R = F + z −d R
(8.15)
then the minimum variance can be calculated as minE(˜ ytT y˜t ) = trace[Cov(F et )] ˆ cl is estimated from time series modeling Furthermore, if a transfer function G of output data yt (after mean-centered), by multiplying it with z −d D and after ˆ cl can be written as expanding it into a Markov parameter form, z −d DG ˆ cl = F + z −d L z −d DG i.e., its first d terms that are represented by F are the same as expanded from G˜l shown in Equation 8.15. This property is also known as the feedback-control invariance property [112, 114]. Therefore, the minimum variance term F et can be estimated from routine operating data yt . The following is an algorithm for MIMO feedback control performance assessment using MATLAB : Algorithm 8.3. MIMO feedback control performance assessment: 1. Sample representative routine operating data from closed-loop control system to be evaluated, when there is no setpoint change. Subtract the mean from the data. 2. Perform time series analysis of this set of data using MATLAB functions such as n4sid and refining the resulting model by pem to identify a multivariate time series model. Make sure to find a stable model. Several iterations of modeling procedures may be necessary for an appropriate model. Let’s call ˆ cl . this estimated time series model G
152
8 Control Loop Performance Assessment: Conventional Approach
3. Obtain the interactor matrix, D, according to a priori process knowledge, using the algorithm discussed in Huang and Shah (1999) [22] based on the open-loop model Gp , or based on identification from closed-loop experiment data [22]. ˆ cl by the interactor matrix q −d D and then expand the product to 4. Multiply G a Markov parameter form using MATLAB functions such as impulse to obtain ˆ cl = F0 + F1 q −1 + . . . + Fd−1 q −d−1 + . . . q −d DG 5. Extract the covariance matrix of the noise (residuals), Σe , from the model object obtained after applying the MATLAB function n4sid or pem. 6. Calculate the minimum variance matrix for the interactor-filtered output y˜t as T ˜ mv = F0 Σe F0T + F1 Σe F1T + . . . + Fd−1 Σe Fd−1 Σ 7. Estimate the actual variance Σy using MATLAB functions such as covar ˆ cl and Σe as the input arguments. with G y T y˜), the multivariate performance index can be calcu8. Since E(y T y) = E(˜ lated as ˜ mv ) trace(Σ η= trace(Σy ) If only the multivariate performance index is of interest, the computation is complete at this point. However, if one is also interested in estimating performance indices for each individual output, the following additional steps are needed. 9. Write the interactor matrix in the form D = D0 q d + D1 q d−1 + . . . + Dd−1 q Perform the following matrix calculation to obtain matrices E0 , E1 , · · · , Ed−1 : [E0 , E1 , · · · , Ed−1 ] = ⎛
F0
⎜ ⎜F ⎜ 1 ⎜. ⎜ T [D0T , D1T , · · · , Dd−1 ] ⎜ .. ⎜ ⎜. ⎜ .. ⎝ Fd−1
F1
· · · Fd−1
F2 .. .
···
Fd−1
⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠
10. Then the minimum variance matrix of the original output yt can be calculated as T Σmv = E0 Σe E0T + E1 Σe E1T + . . . + Ed−1 Σe Ed−1
8.3 MIMO Feedback Control Performance Assessment
153
11. Finally, the performance indices for each individual output can be calculated as η1:m = [diag(Σmv )]. ∗ [diag(Σy )]−1 where .∗ is an element-by-element multiplication operator following MATLAB notations. Example 8.4. Consider a 2 × 2 multivariable process with open-loop transfer function matrix Gp and disturbance transfer function matrix Gl given by1 ⎞ ⎛ Gp = ⎝ ⎛
Gl = ⎝
z −1 4z −2 1−0.4z −1 1−0.1z −1 0.3z −1 z −2 1−0.1z −1 1−0.8z −1
1 −0.6z −1 1−0.5z −1 1−0.5z −1 0.5z −1 1.0 1−0.5z −1 1−0.5z −1
⎠
⎞ ⎠
The white noise, et , is two-dimensional normal-distributed with Σe = I. The output performance is measured by J = E[ytT yt ]. The controller is two multiloop minimum variance controllers designed according to the two single loops without considering their interaction: ⎞ ⎛ 0.5−0.20z −1 0 −1 1−0.5z ⎠ Gc = ⎝ 0.25−0.200z −1 0 −1 −1 (1−0.5z )(1+0.5z ) A unitary interactor matrix D can be factored out from Gp as ⎛ ⎞ −0.9578z −0.2873z ⎠ D=⎝ −0.2873z 2 0.9578z 2
Then z −d DGl where d = 2 is given by ⎛ z −d DGl = ⎝
−1.1015z −1 1−0.5z −1
−0.2873z −1 +0.5747z −2 1−0.5z −1
−0.2873+0.4789z −1 1−0.5z −1
0.9578+0.1724z −1 1−0.5z −1
⎞ ⎠
This can be further expanded as ⎛ ⎞ ⎛ ⎞ 0 0 −1.1015 −0.2873 ⎠+⎝ ⎠ z −1 + . . . z −d DGl = ⎝ −0.2873 0.9578 0.3352 0.6513 1
(8.16)
(8.17)
The off-diagonal elements of Gl need to have at least one sample delay if using System Identification toolbox for computation, since a standard assumption of the toolbox is Gl (z −1 = 0) = I.
154
8 Control Loop Performance Assessment: Conventional Approach
Thus, the feedback controller-invariant term is ⎞ ⎛ −1.1015z −1 −0.2873z −1 ⎠ et F et = ⎝ −0.2873 + 0.3352z −1 0.9578 + 0.6513z −1
The theoretical variance matrix under minimum variance control can be calculated as ⎛ ⎞ 1.2958 −0.5563 ⎝ ⎠ −0.5563 1.5365
This will be the theoretical (exact) minimum variance to assess performance of the feedback controller. Now a set (5000 samples) of closed-loop data is simulated from the system stated above. By applying n4sid followed by pem to the mean-centered data, a ˆ cl is estimated. The covariance of the residuals multivariate time series model G is extracted from the model object as ⎛ ⎞ 1.0149 0.0073 ⎠ Σe = ⎝ 0.0073 1.0133 ˆ cl (where d = 2), can be Using the impulse function of MATLAB , z −dDG expanded into the Markov parameter form: ⎛ ⎞ ⎛ ⎞ 0 0 −0.9651 −0.2894 ˆ cl = ⎝ ⎠+⎝ ⎠ z −1 + . . . z −d DG −0.2843 0.9648 0.3358 0.6751
Comparing this to Equation 8.17, one can see indeed that the first d terms (d = 2 in this example) are feedback-control invariant. The following results are obtained according to Algorithm 8.3: ⎛ ⎞ 1.0341 −0.5322 ˜mv = ⎝ ⎠ Σ −0.5322 1.6001 ⎛ ⎞ 1.0745 −0.1462 ⎠ Σmv = ⎝ −0.1462 1.5596 ⎞ ⎛ 3.1688 −0.2488 ⎠ ΣY = ⎝ −0.2488 1.5241
With these intermediate results, performance indices can be calculated as ˜mv ) trace(Σ trace(Σmv ) = = 0.5613 ΣY ΣY ⎞ ⎛ ⎛ 0.3391 η1 ⎠ ⎝ ⎠ = [diag(Σmv )]. ∗ [diag(ΣY )]−1 = ⎝ 1.0233 η2 η= ⎞
8.4 Summary
155
8.4 Summary A tutorial for control loop performance assessment has been presented in this chapter. It has demonstrated that there exists a term that is independent of feedback control, and this term is the minimum variance of the feedback control loop. Two algorithms for SISO and MIMO feedback control performance assessment were presented respectively, followed by illustrative examples with a step-bystep guide for calculating the minimum variance. The techniques presented in this chapter are limited to minimum variance control benchmarking. However, it constitutes a foundation for control performance assessment. Other advanced and recently-developed methods will be discussed in the next few chapters.
9 State-of-the-art MPC Performance Monitoring
9.1 Introduction Among many algorithms for control performance monitoring, minimumvariance-control based performance monitoring, as discussed in Chapter 8, remains the most popular one owing to its non-intrusiveness. While the univariate performance monitoring has found widespread applications, the multivariate counterpart has found limited applications due to the complexity of the algorithms and demand of more a priori information. Recently several simplified multivariate performance monitoring algorithms have been developed and the perspective of practical applications appears to be promising. Just like PID that is the most accepted univariate control algorithm and needs special attention for its control performance, MPC is the most accepted multivariate control and its performance monitoring technology is in great demand. Some special features of MPC include constraint handling and economic optimization, which require special treatment when monitoring its control performance. It is often debated whether minimum variance control is a suitable benchmark for MPC. The main argument lies in the fact that minimum variance control has a different control structure or objective from MPC; for example, it does not handle hard constraints. However, minimum variance control yields an absolute lower bound of process variance. This lower bound of variance is independent of which control law is used. If output variance of a control system is near minimum variance, no control tuning will help the improvement of variance no matter what type of control algorithms has been or will be used and no matter whether the control algorithm is the constrained one or not. How the variance reduction is transferred to MPC’s benefit potential and how MPC’s performance can be assessed through the variance are the research topics of current interest, and answers to these problems are the key to make minimum variance benchmarking useful for MPC monitoring. In addition to variance monitoring, there are many algorithms and case studies for MPC performance monitoring that have been reported in the literature. Broadly speaking, they can be classified into two groups, namely model-based B. Huang et al.: Dyn. Model. Predict. Ctrl. & Perform. Monitor., LNCIS 374, pp. 157–175, 2008. c Springer-Verlag London Limited 2008 springerlink.com
158
9 State-of-the-art MPC Performance Monitoring
and model-free approaches. Among the model-based approaches, there are minimum variance control benchmarking, generalized minimum variance control benchmarking, LQG (linear quadratic Gaussian)/MPC tradeoff curve approach, design objective versus achieved objective approach, historical-data benchmarking, on-line model validation, and model-based simulation approach. The common feature of this group of algorithms is the need of process models or interactor matrices. The model-free group includes impulse response curvature, prediction error curvature, Markov chain, and statistical data analysis. This group of algorithms does not assess control performance against a benchmark control and is nonparametric in general. No models or interactor matrices are needed; thus they are attractive for practical applications. Most recently, economic performance assessment of MPC has also received attention [116, 117, 118]. A recent work by Ordys et al. (2007) [119] provides a broad context to control performance monitoring other than merely evaluating the controller performance. What is discussed in [119] includes business performance benchmarking, economic auditing of control systems, plantwide disturbance diagnosis, oscillation detection, and valve stiction monitoring. As such, the research on control performance monitoring has gone beyond the traditional scope. Another interesting way of considering MPC performance monitoring is through on-line model validation. Since MPC is an optimizing algorithm based on the models, unless the MPC objective function is not well formulated, MPC will be optimal by nature if the models are correct. Therefore, if the MPC objective function has been designed well, the model that is being used by MPC needs more attention. Following these arguments, monitoring of MPC performance boils down to two basic problems: evaluation and validation. Evaluation is for the MPC objective function and validation is for the model used by MPC through the real data. Parameters that affect controller objective functions are often called tuning parameters. Factors that may affect model quality include process nonlinearity, instrument malfunctions, time varying process and/or disturbance dynamics, inadequate system identification, etc. As pointed out by Jelali (2006) [120], a fundamental question related to the performance assessment of MPC is whether detected poor control performance is due to bad controller tuning or inaccurate modeling. Noticeably, the model validation problem has been iterated in Kozub (1997) [121], Kesavan and Lee (1997) [122], and Dumont et al. (2002) [123].
9.2 MPC Performance Monitoring: Model-based Approach 9.2.1
Minimum-variance Control Benchmark
Minimum variance control continues to be the most frequently used benchmark for feedback control performance assessment owing to its theoretical elegance, non-intrusiveness and ability to provide the absolute lower bound of process variance. It originated from Harris’s work [19] for univariate control loop
9.2 MPC Performance Monitoring: Model-based Approach
159
performance assessment. It has since been extended to multivariate control systems by [112, 114, 25, 27]. A tutorial on this topic has been provided in Chapter 8. The drawback of the minimum variance benchmark, however, is the requirement of the interactor matrix or its equivalence that is conceptually difficult and computationally challenging. In addition, minimum variance control is an aggressive control and is rarely implemented in practice; thus its control objective may not be directly compatible with that of MPC. 9.2.2
LQG/MPC Benchmark
Practical constraints on control systems such as valve constraints imply that performance of minimum variance control is most likely not achievable in practice and the best performance of the practical controllers is often lower than that of minimum variance control. Thus, using minimum variance as a benchmark to evaluate a practical control system such as MPC tends to underestimate the performance. In general, tighter quality specifications result in smaller variation in the process output but typically require more control effort. It may therefore be interesting to know how far the control performance is from the “best” achievable performance for any pre-specified control effort; i.e., in mathematical form the resolution of the following problem may be of interest [22]: Given ||ut ||2 ≤ α, what is min{||rt − yt ||2 }? The solution is given by a tradeoff curve as shown in Figure 9.1. This curve can be obtained from solving the LQG problem [124, 125], where the LQG objective function is defined by J(λ) = E[(rt − yt )2 ] + λE[u2t ] By varying λ, various optimal solutions of E[(rt − yt )2 ] and E[u2t ] can be calculated. Thus a curve with the optimal E[u2t ] as the abscissa and E[(rt − yt )2 ] as the ordinate is formed from these solutions. A