Differential Equations Classical to Controlled
This is Volume 162 in MATHEMATICS IN SCIENCE AND ENGINEERING A Series of Monographs and Textbooks Edited by RICHARD BELLMAN, University of Southern California The complete listing of books in this series is available from the Publisher upon request.
Differential Equations Classical to Controlled Dahlard L. Lukes Department of Applied Mathematics and Computer Science University of Virginia Charlottesville, Virginia
1982
ACADEMIC PRESS A subsidiary of Harcourt Brace Jovanovich, Publishers
New York London Paris San Diego San Francisco
Sao Paulo Sydney
Tokyo
Toronto
©
COPYRIGHT 1982, BY ACADEMIC PRESS, INC. ALL RIGHTS RESERVED. NO PART OF THIS PUBLICATION MAY BE REPRODUCED OR TRANSMITTED IN ANY FORM OR BY ANY MEANS, ELECTRONIC OR MECHANICAL, INCLUDING PHOTOCOPY, RECORDING, OR ANY INFORMATION STORAGE AND RETRIEVAL SYSTEM, WITHOUT PERMISSION IN WRITING FROM THE PUBLISHER.
•
ACADEMIC PRESS, INC.
111 Fifth Avenue, New York, New York 10003
United Kingdom Edition published by ACADEMIC PRESS, INC. {LONDON) LTD. 24/28 Oval Road, London NWI
7DX
Library of Congress Cataloging in Publication Data lukes, Dahlard l. Differential eq~ations. (Mathematics in science and engineering) Bibl iography: p. includes index. I. Differential equations, Linear. 2. Differential equations, Nonlinear. 3. Control theory. I. Title. I i. Series.
QA372.L84 515.3'5 ISBN O-12-459980-X
82-6797 AACR2
PRINTED IN THE UNITED STATES OF AMERICA
82 83 84 85
987654321
To my parents, Lawrence and Josephine
This page intentionally left blank
Contents
xi
Preface
Chapter 1 Introduction 1.1. 1.2. 1.3. 1.4.
Chapter 2
Origin and Evolution Sources of First-Order Equations Classical Questions Control Questions
1
4
12 15
Matrix Algebra-The Natural Language of Linear Systems 2.1. 2.2. 2.3. 2.4. 2.5. 2.6. 2.7. 2.8.
Terminology Addition, Multiplication, and Scalar Multiplication The n-Vector Spaces ?}in Systems of Linear Algebraic Equations Matrix Inversion Large-Scale Matrix Computations Using Computers Determinant Functions Eigenvalues, Eigenspaces, Characteristic and Minimal Polynomials 2.9. The Jordan Form of a Matrix
19
20 23
25 29
31 35 39
43
Chapter 3 Linear Constant-Coeffident Equations: Computation 3.1. The Initial-Value Problem 3.2. The ABC Algorithm 3.3. A Vector Method for Computing the Matrix C
47 48
54
viii
Contents 3.4. Explicit Formulas for C(c) and eIl(t) 3.5. Additional Comments about Computing and Changing Variables
Chapter 4
59 60
Linear Constant-Coefficient Equations: Theory 4.1. Preliminary Remarks 4.2. Some Elementary Matrix Analysis 4.3. Existence, Uniqueness, and Solution Representations Using etA. 4.4. The Structure of etA 4.5. The Theoretical Foundation of the ABC Algorithm for Computing etA. 4.6. Explicit Formulas for C(c) and etA.
64 65 72 77 78
83
Chapter 5 Linear Constant-Coefficient Equations: Qualitative Behavior 5.1. Stability and Periodic Oscillation 5.2. The Lyapunov Approach to Asymptotic Stability
92 95
Chapter 6 General Linear Equations with Time-Varying Coefficients 6.1. 6.2. 6.3. 6.4. 6.5. 6.6. 6.7. 6.8.
Widening the Scope Preliminaries The Fundamental Matrix Series Defining ell Existence and Uniqueness of Solutions to Vector Differential Equations and Their Duals Variation-of-Parameters Formulas for Vector and Matrix Differential Equations Estimates of Solution Norms Fundamental Matrices of Equations with Time-Varying Coefficients Continuous Time, Discrete State Markov Processes
106
107 108 112 114 115 128 132
Chapter 7 Commutative Linear Differential Equations 7.1. Introductory Comments 7.2. The Definition and an Example of Commutative Equations
138 139
ix
Contents 7.3. Computation of Closed-Form Solutions to Linear Commutative Equations 7.4. Some Sufficient Conditions for Commutivity 7.5. Matrix Functions and Operations Preserving Commutivity 7.6. The Special Case n = 2 7.7. A Criterion for Exponential Decay of Solutions
140 145 148 157 160
Chapter 8 Periodic Linear Equations 8.1. Periodic Homogeneous Equations 8.2. Qualitative Behavior 8.3. Nonhomogeneous Equations 8.4. Periodic Commutative Equations
Chapter 9
162 167
172
175
Local Existence and Uniqueness Theory of Nonlinear Equations 9.1. Complications That Can Occur 9.2. Local Existence and Uniqueness
Chapter 10 Global Solutions
10.1. Introduction 10.2. Maximal Solutions and Extensions of Local Solutions 10.3. The Behavior of Solutions on Maximal Intervals
181 182
190 190
192
Chapter11 The General Solutio~Dependence of Solutions on Parameters 11.1. Mathematical Preliminaries 11.2. A Space of Curves 11.3. The General Solution (D, 'f) to the Cauchy Problem (D, f) and Continuous Dependence on Parameters 11.4. Differential Dependence of the General Solution (D, 'f) on Parameters and Related Variational Equations
200
202
205
212
Chapter12 Limit Properties of Solutions 12.1. 12.2. 12.3. 12.4. 12.5.
Limit Sets and Invariance Stability of Nonlinear Equations Partial Stability Local Behavior and Linearization about Rest Points Planar Systems
223
231
236
243
250
Contents
x
Chapter 13 Applications in Control Theory 13.1. 13.2. 13.3. 13.4. 13.5. 13.6. 13.7. 13.8. 13.9.
The Controlled and Uncontrolled Equations Transient and Steady-State Parts of the Solution The Frequency Response Function The ABC Approach to Computing Frequency Response Matrices Controllability Altering System Dynamics by Linear Feedback Decoupling and Recoupling-Noninteractive Design Controllability of Nonlinear Equations The Domain C of Null Controllability
261 262 263 266 269 275 284 307 313
References
317
Index
319
Preface
This book deals with the classical theory of differential equations, showing some of the impact that computers and control theory are having on the subject. It is based on notes that I developed in recent years while teaching an advanced-undergraduate beginning-graduate course aimed at applied mathematics and engineering students. These notes succeeded better than the numerous textbooks that I tried in dealing with the somewhat heterogeneous background of my students. All had studied calculus and beginning differential equations and had some familiarity with matrix manipulations. Additional courses in advanced calculus or analysis proved helpful but were not crucial. The book provides a basis for a onesemester course for such students. To complete the text, say in a secondsemester course, the students should first have acquired some knowledge of analysis, linear algebra, and topology. The historical development of differential equations and control theory is sketched in Chapter 1, in which motivation and an overview of the subjects are also provided. Chapter 2 contains a brief review of matrix theory and notions of linear algebra that are drawn on throughout much of the book. It can be skipped and used only as a reference if the instructor wishes to require a course in matrix theory as a prerequisite. On the basis of a heuristic matrix approach, which I have dubbed the ABC method, I introduce in Chapter 3 the computational aspects of linear constant-coefficient systems. The theoretical underpinning and the move to progressively more general equations are found in succeeding chapters, in contrast to the usual approach of treating nonlinear equations first. The loss in efficiency seemed to be outweighed by the pedagogical advantage gained, wherein the applied science students first see the theory building upon real analysis at a less abstract level and only after they are well into xi
xii
Preface
the course. At the same time the more mathematically inclined students get a better picture of where the loss of theoretical detail occurs during the transition to more general equations and more abstract concepts. In recent years, great progress has been made in "the use of computers to solve large systems of linear algebraic equations and to compute eigenvalues. The ABC approach developed in this book reflects my efforts to reduce the problem of generating the analytical (closed-form) solutions to large systems of linear .differential equations to the first problem and hence to exploit this progress . The applications of the ABC formula made here demonstrate its theoretical as well as computational value. The study of the variable coefficient systems that I call commutative equations, which appears in Chapters 7 and 8, grew out of this renewed interest in computing. The theorem concerning linearization of nonlinear equations about a hyperbolic rest point, first studied by Poincare in the context of analytic systems and later settled by Hartman, is given an elementary proof in Chapter 12in order to add further significance to linear equations. In Chapters 9-12, I cover many of the standard topics dealing with nonlinear equations. Dependence of solutions on parameters and associated variational equations are given particular attention. Although Chapter 13 is intended to provide only an introduction to control theory, the material covered is of fundamental importance to anyone with limited opportunities for studying modern control theory. In many respects the feedback theory presented in this chapter goes far beyond the old-fashioned root locus and transfer function methods still being taught in many courses: I believe it is necessary to devote special attention to the computational aspect of the decoupling problem since the abstract theory of the past ten years does not seem to have percolated down into the engineering texts, in spite of the recognized importance of the problem in applications. It should be noted that the sections of Chapter 13 on control theory (1-7) could be covered after finishing Chapters 1-5, and in general the book can be used in several ways. For example, a one-semester introductory level systems-oriented course might cover Chapters 1-6 and 8, and Sections 1-7 of Chapter 13. If students have had matrix theory and some complex function theory, then Chapter 2 could be skimmed over and Chapter 7 inserted into the course coverage. If more emphasis were to be placed on nonlinear equations, using Chapters 1-6,9, and 10 would leave some time to pick and choose from the topics in Chapters 11 and 12. If the text were to be used for a two-semester sequence, the chapters would be covered in the normal order, with the option of skipping Chapter 7. Numerous examples and exercise sections have been inserted at appropriate points throughout the book.
Preface
xiii
I wish both to thank Ruth Nissley for so graciously contributing her energy and professional skill in typing the final manuscript and to express my gratitude to Carolyn Duprey and Faye O'Neil for their help in preparing the earlier drafts.
This page intentionally left blank
Chapter
1
Introduction
This chapter begins with a historical sketch of events that shaped the development of the subject. Its simple examples illustrate the well-established applicability of differential equations. They also provide motivation for the choices of the model equations studied in later chapters and indicate some of the new directions that control ideas have given to the classical theory. 1.1
Origin and Evolution
Since its invention by Isaac Newton circa 1666, the theory of ordinary differential equations has occupied a central position in the development of mathematics. One reason for this is its widespread applicability in the sciences. Another is its natural connectivity with other areas of mathematics. Newton conceived the idea of a gravitational field and concluded that the rate of change of the momentum of a free particle of mass in such a field must equal the gravitational force exerted upon it by that field. His equality is an example of what is now called a differential equation. The necessity for defining the meaning of the rate of change of a non uniformly changing variable prompted his discovery of differential and integral calculus. With these new concepts he was able to explain the tides and place the motion of the moon and planets upon a firm mathematical foundation. The phenomenal success of extensions of his ideas into engineering, physics, mathematics, and other branches of science now has a rich history. The profundity of Newton's original discovery is reflected in the fact that the continually expanding notions of derivative, integral and differential equation still underlie much of modern mathematical research. Following Newton, a number of great mathematicians, including Cauchy, Peano, and Riemann, contributed to what is now referred to as the classical 1
2
1.
Introduction
foundations of differential equations-a major topic of this text. The theory provides an elegant and powerful treatment of existence and uniqueness problems as well as other questions. In effect, it established the logical legitimacy of differential equations as models for scien.tifi.c phenomena.The scientist's concern is thus reduced to selecting the appropriate equations. Control. theory, a relatively modern development, attempts to go a step further. Control phenomena, in contradistinction to control theory, have their primordial origins in nature. The biologist Richard Dawkins regards the gene as a fundamental control element in his view of Darwin's theory of natural selection [6]. He envisions a gene as arising from a special configuration of organic molecules that somehow appeared in the primordial soup with the stability property of attracting sequences of elementary molecules from the environment and then splitting into two copies of the original after a threshold of complexity had been reached-a kind of primitive cell division. Another biologist, Lyle Watson, entertains the idea that these seed molecules might have originated from beyond the earth, possibly from the tails of comets through whose debris the earth passed [26]. At any rate, this replicator soon populated and in the process altered its environment by depleting the supply of elementary molecules. The variability in the replication produced a family of competing replica tor types. Some complexes of replicator molecules stumbled upon adaptive control processes that allowed them to survive the competition and the changing environment. Those discovering reproductive behavior and possessing other strong stability characteristics persisted through" successive generations. Thus, nature advanced toward creating a biological invariant called life. After some 4 billion years, this scenario culminates in the fundamental problem of biology; to determine how the information stored in the DNA of the cell's genes is used in controlling the cellular development of the surviving organisms. Dawkins regards living organisms as elaborate, highly successful survival structures built about their genes. Proceeding from this view, he goes on to offer new explanations for altruism and other unusual forms of behavior exhibited by some species. Watson suggests that man may have reached such a complex state of development that, in some profound manner, the genes' control is linked with and perhaps is superseded by the subconscious mind. Some scientists speculate that man's behavior, institutions, perception of himself, and possibly physical reality itself evolve according to some higher levels of Darwinian natural selection. Whatever the origin and evolution of living things, the present world abounds with organisms, each operating sophisticated control mechanisms. The human body's system for maintaining its temperature within the narrow
1.1 Origin and Evolution
3
range necessary for survival is a good example. Another is its immunological system for identifying and destroying invading bacteria and other foreign substances. Watson discusses the possibility that unstable cell growth is a common occurrence in physiology and that cancer develops when the identification process fails or when for some other reason the stabilizing immunological feedback system malfunctions. Examples of such automatic control systems can easily be multiplied. Civilization has long been concerned with control at various levels of consciousness. Childrearing, education, religion, and politics all involve a high degree of psychological control. The manifold institutions of government-including the maverick education-are highly evolved control devices intended to help manage (but unfortunately often dictating) man's affairs. Thus it appears that control phenomena have pervaded all aspects of life since its beginning and undoubtedly will continue to do so. Only in relatively recent times have engineers finally injected the element of control into their equations. The manner in which this came about is rather interesting. In 1769 James Watt, a Scottish engineer, invented the governor for steam engines. This early example of a feedback control system was a device for maintaining a specific speed of the flywheel of an engine operating under variable load. Before that- time, the applications of the steam engine were quite limited. The Watt governor dramatically transformed the steam engine into a feasible mechanism for delivering steady power and consequently was an important factor in the occurrence of the industrial revolution. Control phenomena attracted very little mathematical interest before 1900, with the exception of J. C. Maxwell's 1868 study of the steady-state error which occurred in applications of the Watt governor [17]. A great period of-industrialization took place from about 1900 up to World War II. It witnessed the development of large-scale power generation and transmission, aeronautics, the chemical industry, communications, and electronic engineering. This activity gave rise to the design of many pneumatic, hydraulic, electromechanical, and electrical regulators. The period, sometimes referred, to as the era of handbook engineering, was narrowly directed toward a few specific applications, relied heavily upon low-order linear models in Laplace transform form, and generally employed rather limited areas of mathematics. This status began to change rapidly during World War II, stimulated by such problems as fire control (which involves the rapid and accurate positioning of guns). The great advances in electronics, followed by the emergence of analog and digital computers, greatly expanded the horizons of control. The upgrading of the mathematical training of engineers was another
4
1.
Introduction
important factor involved in the transformation of control engineering into a science. These engineers were better prepared to define problems mathematically and to develop new concepts of control. This is tum allowed mathematicians to bring to bear upon the problems the power of abstract mathematics. Control theory is now regarded as one of the major areas of mathematics. Increasing numbers of scientists and laymen are recognizing the controltheoretic aspect of many of the serious problems of mankind that arise from the expansion and growing complexity of the world's evolving systems with their seemingly inextricable coupling. A major difficulty is that the actual dynamics and coupling of these systems are often poorly understood and sometimes ignored for economic, political, or other reasons. One important example of this is the problem of controlling insect damage to crops [1 J. It is estimated that in 1977 at least 250 million pounds of pesticides were dumped on California alone in an effort to combat insect damage to farm crops. (The complexity of the problem is indicated by the fact that in spite of these chemicals there remain 70 major insect pests that plague corn.) With Darwin's natural selection operating as predicted, the intended victims often develop resistant varieties, and these survivors are even more difficult to control. Moreover, the chemicals often create harmful imbalances in the ecosystem that trigger other infestations, contaminate drinking water, and generate many other problems. Again, the examples can be multiplied. The critical nature of these problems sorely awaits applications of control science outside classical areas of engineering. During the past 15 years, a great deal has been learned about the mathematical control of finite-dimensional systems, although the theory is far from complete. Currently there is considerable interest in the control theory of systems with time-delay and in systems modeled by partial differential equations (see [20J). Many of these problems can be treated by the theory of functional analysis as ordinary differential equations in infinite-dimensional (function) spaces. In this book, we treat primarily the classical theory of differential equations at the more modest finite-dimensional level where, ordinarily, more detailed results are available. This provides a firm foundation for more advanced study of the control aspects of such equations for those wishing to pursue the subject. 1.2 Sources of First-Order Equations
Although only limited space can be devoted to discussing practical applications in this book, some feeling for simple kinds of problem situations
5
1.2 Sources of First-Order Equations
-
rl rl
Tank
u(t)1 - -
I
rz
Tank 3
Tank 2
f---
I--
rl
Fig. 1.2.1 A heat-flow system with input.
which can be modeled by differential equations is desirable. The following examples serve that purpose.
Example 1.2.1 Flow and Mixing. Suppose that water is pumped between three thermally insulated tanks through connecting pipes at rates '1 = 1, '2 = 2 gallons per minute, configured as in Fig. 1.2.1. Initially each tank contains 100 gallons of water at specified temperatures. The temperatures of the water in the tanks change with time due to the pumping between tanks. Tank 1 has a heat exchanger which allows heat to be injected or extracted directly at a rate of u(t) British thermal units (Btu) per minute. Thus u(t) is regarded as an input to the system and the output of interest is the temperatures of the water in the tanks. Since the temperature of a gallon of water is proportional to the heat that it contains, it is sufficient to work with xlt), the amount of heat in the ith tank at time t (i = 1, 2, 3). By taking into account the heat flow both into and out of each tank and using the fact that the heat flow rate through a pipe equals the water flow rate times the heat per unit volume of water, it is easy to see that the appropriate system of differential equations satisfied by the x;(t) (assuming the heat loss and the contents of the pipes to be negligible) is
2[x100 + u(t), 2(t)] dX2(t) = l[x (t) ]_l[x100 100 ' dt dX3(t) = l[x (t)] 1[x2(t)] _ 2[x 3(t)] 100 + 100 100 . dt
dx 1(t) . dt
= 2[x 3(t)] _ 100 1
1(t)]
(1.2.1)
1
Letting x(t) denote a column-vector variable with scalar coordinate variables X2(t), X3(t), system (1.2.1) can be rewritten as a single vector-matrix
Xl (t),
1. Introduction
6
differential equation
x=
Ax
+
f(t), where
- lo A=
f(t) =
[
1
0
50
1
TOO
-TOO
1 100
1
TOO
o1 ] , --10
U(t)] [~ .
(1.2.2)
(1.2.3)
The same model would apply to other mixing problems such as the flow of pollutants in a water supply system, with the tanks being lakes or reservoirs. Example 1.2.2 Mechanics. Consider the forced spring-mass system with friction indicated in Fig. 1.2.2. The mass m moves in response to the spring force - k 2 x generated by displacement of the mass x units from equilibrium, friction force - k 1 x, and external driving force u(t). The k 1 and k 2 are nonnegative constants of proportionality associated with the assumed standard models of springs and friction. Application of Newton's law of motion leads directly to the second-order scalar equation
(1.2.4) By introducing variables Xl = x and X 2 = dx.ldt, Eq. (1.2.4) can be transformed into the equivalent first-order system (1.2.5) This system can likewise be written in the vector-matrix form x = Ax
u(t)
Fig. 1.2.2
A forced spring-mass system with friction.
+ f(t),
1.2
7
Sources of First-Order Equations
where (1.2.6) (1.2.7) Example 1.2.3 Electrical Circuits. Differential equations are often used for modeling electrical circuits. Recall that the voltage drop across an idealized resistor, inductor, or capacitor is vR = Ri, vL = L dildt, or uc = (1/C) i dt, respectively, where R, L, and C are physical constants and i(t)is the current passing through the circuit, Thus application of Kirchhoff s law, which says that the total voltage drop around a simple closed circuit must be zero, shows that the current i ( t ) in the circuit of Fig. 1.2.3, containing a timevarying voltage source e(t), must satisfy the-equation di Ldt
+ Ri +
AJ:
i(z)d z = e(t)
(1.2.8)
(assuming the charge on the capacitor to be zero at t = 0). Introducing the variables x 1 = i, x 2 = di/dt, and u = (1/L) deldt, the differentiated form of (1.2.8) transforms into the first-order system (1.2.9)
which once more has the vector-matrix form f
=
0
Ax
+ f (t),this time with (1.2.10) (1.2.11)
More complicated forms of Examples 1.2.1-1.2.3 can easily be obtained by R
C
I
Fig. 1.2.3 A-imple circuit.
8
1. Introduction
considering more tanks, springs and masses, or circuit elements connected in countless complex patterns. All the resultant systems of differential equations as well as the equations arising from many other areas of application are subsumed by the general equation of the form
x=
Ax
+ f(t),
(1.2.12)
in which x is a column-vector of n real variables, A is an n x n real matrix, and f(t) is a prescribed column-vector of n real-valued functions of t. In Chapters 3- 5, we develop the classical theory of linear differential equations with constant-coefficient matrix as given by (1.2.12). Obviously a substantial amount of matrix algebra is involved in such a treatment and this is anticipated by the review of matrix algebra that is presented in Chapter 2. The classical theory of (1.2.12) presented will be found to be in a rather satisfactory state of completion. It provides, amongst other results, an effective method for computing all solutions to (1.2.12) in a closed form. A moment of reflection makes it clear that only slight changes in the physical problems underlying Examples 1.2.1-1.2.3 produce systems of equations requiring a model more general than (1.2.12). For example, suppose that the value of the flow rate rl of the system presented in Example 1.2.1 were changed from r 1 = 1 to r 1 = 2. It is easy to check that this would require replacement of the real matrix (1.2.2) in the model equation by the matrix function of t, . [- 2/(50 - t) 1/(50 - r) 1/(50 - t)
0
-lo
1/(50 + t) ] 0 . 1/(50 + t)
to -
(1.2.13)
Clearly, something as simple as an error in a pumping rate or valve setting could turn the original system into one with time-dependent coefficients. One can as well envision the situation for which in Example 1.2.2 heat generated from the mass sliding along the surface or from other sources causes the' coefficient of friction k 1 to change with time. Likewise the resistor in Example 1.2.3 might heat up with the operation of the circuit and thereby result in a time-dependent value of resistance R. Again, these time-dependent effects would appear in the matrix coefficient of (1.2.12) and thus it would seem appropriate to expand the theory to cover the more general equation
x=
A(t)x
+ f(t),
(1.2.14)
in which the coefficient A(t) is an n x n real matrix-valued function of t. This indeed is on the program. It turns out that important questions concerning the behavior of many nonlinear differential equations as well can be studied
9
1.2 Sources of First-Order Equations
through associated (variational) linear time-dependent equations. This provides additional cause for interest in equations of type (1.2.14). Differential equations of the form (1.2.12) and, more generally, (1.2.14) are called linear first-order equations. These equations are sufficiently general to cover many important applications. However there are problems where they are not appropriate. For example, in the study of large-amplitude oscillations of the system in Example 1.2.2, some springs might be better modeled with the spring force - k2 x replaced by - k2 x 3 • The resultant differential equations replacing (1.2.5) would. then be (1.2.15) which is no longer of the linear type (1.2.14). Similarly, if the smoothness of the surface upon which the mass slides were not uniform, then the model of the friction force might require alteration; e.g., - k 1 Xmight be replaced by -k 1x2 X. This too would destroy the linearity. The same kinds of remarks apply to Example 1.2.3. In fact the famous Van der Pol equation .2 d2i dt 2 + (I
-
di.
1) dt
+1=
u(t)
(1.2.16)
arose from attempts to account for self-excited electrical oscillations in circuits by assuming the circuit resistance dependent upon the current. The first-order form of this equation, of course, is again not linear. Examples of such nonlinearities led researchers to the study of equations of the type
d2x dt2
dx
+ f(x) dt + g(x) = u(t)
(1.2.17)
called Lienard equations and, more generally, 2x
d dt2
+f
(dX) dx x, dt dt + g(x) = u(t),
(1.2.18)
the Levinson-Smith equation, in which the coefficient function f is allowed to depend on the derivative dxldt as well as on x. Discussion of these and other nonlinear equations will reappear in later chapters. For some applied problems a linear or slightly altered (perturbed) linear model is altogether inappropriate; e.g., long-term motion of bodies in gravitational fields (Newton's equations were nonlinear) or population models. In the final design stage of engineering it is frequently necessary to include numerous nonlinear terms in the simulations in order to fix the hardware specifications accurately.
10
1.
Introduction
Fortunately a substantial part of the classical theory applies to equations of the very general form dx (1.2.19) dt = f(t, x), with only rather mild assumptions imposed on f; for example, that f be a real n vector-valued function satisfying some continuity or differentiability condition. The precise details and consequences of the nonlinearities will emerge in later chapters. EXERCISES
1. Rewrite the following equations in first-order form, identifying the matrix coefficient and forcing term of the form (1.2.14) for those cases in which the system is linear. (a) x + f(x) X + g(x) = u(t), (b) x + f(x, x) x + g(x) = u(t), (c) x + (sin t) x + 2x = u(t), (d) {m 1 ~1 + k 3 ~1 + (k 1 + k 2 ) X l - k 2 X2 = ul(t),
m2 X 2 + k 3 X 2 - k 2X l + k2X2 = U2(t). 2. Find a scalar differential equation whose first-order form is x = Ax f(t) in which
+
y const.
Verify that the matrix function A(t) given by (1.2.13) is correct. Show that if x(t) is a solution to the homogeneous vector-matrix equation x = Ax, where A is an n x n constant matrix, then J& x(t) dt = 0 implies that x(T) = x(O). (Although its proof requires a result not yet derived, the stronger conclusion that x(t + T) = x(t) for all t is valid, i.e., x(t) is periodic . of period T.) 5. A small drop of oil is placed at the rest point of the mass in Fig. 1.2.2. Which of the following alterations of (1.2.4) best reflects that phenomenon? (a) m x + 2k l X + k2 x = u(t), X2 (b) m x + k l (1 - e- ) x + k 2 x = u(t), (c) m x - k l X + k 2 x = u(t), (d) m x + k 1 X + k 2 x 3 = u(t). 6. Derive the differential equations model for the forced spring-mass system indicated in Fig. 1.2.4, letting Xl and X2 denote the displacement of 3. 4.
11
1.2 Sources of First-Order Equations
k3
Fig. 1.2.4 A double forced spring-mass system with friction.
the respective masses from equilibrium and using linear modeling as in Example 1.2.2. 7. Three tanks are connected by pipes through which salt brines flow with xi(t)denoting the number of pounds of salt in tank i ( i = 1,2,3) at time t . The variables u1 and u2 are pure salt flow rates. Determine the configuration of pipes, flow rates, and volume of water initially in each tank that would be modeled by the equations
What is the significance of the negative coefficient of u2 in the equation for i2? 8. Derive the differential equation model for the currents i l ( t ) and i z ( t ) passing through the respective resistors in the electrical circuit indicated in Fig. 1.2.5. 9. The populations x1and x2 of two predator-prey species of fish occupying a lake are modeled by the nonlinear differential equations i 1
=
~ 1 x 1
klXlxz,
i 2
=~
2 x 2 k2XIX2,
in which p l , p z , k l , k2 are positive constants. Discuss the possible interpretation of the terms in the equations in context of reproduction and losses due to predation. Think of situations in which alternative models of birth and predation might be appropriate.
Fig. 1.2.5 A two-stage electrical network.
12
1. Introduction
1.3 Classical Questions
A major part of the classical theory is concerned with the pair of equations
dx
dt = f(t, x),
X(t) = ~,
(1.3.1)
called the Cauchy or initial-value problem. Roughly speaking, the function f and parameters r, f are presumed given, with a solution x( .) being any continuous function defined on an open interval that contains t and satisfying (1.3.1) on that interval. In the text, we deal with the existence and uniqueness of solutions to (1.3.1) and study their dependence on the parameters r and ~. The general results obtained are particularly significant since, with the exception of the special equations of type (1.2.12), it is usually impossible to represent solutions to such problems in terms of elementary functions. In Chapter 3 we present a method (the ABC method) for constructing an explicit representation of the solution to the initial-value problem associated with (1.2.12) using only the operations of finding roots of a polynomial (i.e., the eigenvalues of A), solving systems of linear algebraic equations, and performing elementary integration. Even for this simplest class of equations, an independent existence proof is needed to substantiate critical steps in the algebraic construction. In applications of differential equations, any information about the qualitative behavior of the solutions is usually extremely valuable. For equations of type (1.2.12), a rather complete picture is presented in Chapter 3. The solution is shown to be the sum of a forced solution and the unforced solution. In particular, it turns out that for a fixed matrix A, a coordinate function Xi(t) of the unforced solution x(t) to (1.2.12) can exhibit one of the forms indicated in Fig. 1.3.1 (neglecting transients), and computations on A will predict which can be expected. In applications in which xi(t) denotes the difference (error) between the actual value and the ideal value of some variable in an operating system, forms (a) or (c), having decreasing amplitude, are usually the most desirable, with the periodic form (e)sometimes tolerable but unstable forms (b) and (d) assuredly disastrous. The theory also provides estimates of the rate of attenuation of (a) and (c) as well as information concerning the frequencies of oscillation that might occur. Another property of the solutions that is of importance in applications is the extent of the coupling between the coordinate variables of the solution; i.e., the extent to which a change in initial values of some of the xi(t) willaffect the others. (For example, a well-designed automatic control system for an airplane would not allow one
13
1.3 Classical Questions
I
(el
Fig. 1.3.1 Qualitative response forms of unforced solutions.
orienting variable to respond to a disturbance in another.) Here again the theory is helpful. The computational and' qualitative theory of the time-varying equation (1.2.14) is less complete, as might be expected due to the increased generality of the equation. Although it turns out that a certain finite dimensionality of the solutions persists, the variety of possible qualitative behavior is so rich that it is difficult to select any one set of properties upon which to develop the theory and it appears impossible to derive a comprehensive notion of closedform solution that approaches the relative simplicity of that of constantcoefficient equations. Specifications arising from applications provide some insight into which properties might be most important, but presently that source likewise appears overwhelmed by the possibilities. It is clear that any results which will predict periodicity in the solutions or detect the stability property that x ( t ) 0 as t 00 are of utmost interest. Some rather detailed results concerning the structure and qualitative behavior of the solutions are provided in this book for the special cases in which A@)is periodic or satisfies a certain commutivity property. In applications it is often necessary to resort --+
--f
14
1. Introduction
to numerical integration or computer simulation of the equations to obtain an adequate understanding of the behavior of the solutions. The broader generalization from linear time-varying equations to nonlinear equations (1.3.1) creates much greater difficulties. Even for autonomous equations for which f in (1.3.1) is a function of the variable x alone, the finite dimensionality disappears and new complications arise. One of these is what we call the immortality problem; that is, the phenomenon in which even though f in (1.3.1) is defined for all values of its argument, the solutions to the initial-value problem can fail to exist for t outside a finite interval containing r. An example would be the nonlinear equation (1.3.1) associated with the second-order nonlinear equation ii
+ uu
- u 3 = 0,
(1.3.2)
which has a solution u(t) = 1/(t - 1) that cannot be extended beyond the interval - 00 < t < 1. In spite of the incomplete state of the theory, later chapters of this book do contain some often useful results dealing with the immortality problem and other aspects of qualitative behavior. These results are largely based upon perturbation techniques, differential inequalitites, and linear variational equations. In passing, it should be mentioned that modern mathematicians have made serious attempts to deal with the question of what form the qualitative theory should take when the equations are generalized beyond the linear constantcoefficient type modeled by (1.2.12). One idea considered by Smale [24] and others is to define some choice of equivalence relation on, roughly speaking, all differential equations on some region of (t, x)-space (more generally, on a manifold) which in some sense preserves the qualitative behavior of the solutions. Next some choice of a subset of differential equations is made, satisfying two conditions: The subset must be sufficiently rich to provide approximations to all differential equations, on one hand, and, on the other hand, if a differential equation is approximated sufficiently accurately by another from the prescribed set, then both will be in the same equivalence class. Ideally, all the choices would be made so that the equivalence classes could be distinguished by numerical and algebraic invariants. This would be a reasonable compromise with the unrealistic goal of trying to in some sense solve and completely understand the behavior of the solutions to all differential equations. Thus far, such a program has not proved to be of much practical value in studying specific types of nonlinear equations. To a considerable degree the technical difficulties encountered in the study of nonlinear equations have turned the attention of many engineers and mathematicians toward new questions about linear equations, namely, toward the control theory of linear systems. Needless to say, nonlinear equations reappear as an obstacle in control theory.
15
1.4 Control Questions
I
--J----..
s
u - -........
... x
Fig. 1.4.1 An input-output system.
1.4 Control Questions
A natural setting for the application of control theory is any situation involving one or more input-output systems, as depicted in Fig. 1.4.1, involving input or control variable u and output or state variable x. The intuitive notion of an input-output system is quite general and ordinarily applies to any phenomenon, real or imagined, which conforms to a causeeffect relationship. Examples would include problems modeled by difference, algebraic, ordinary differential, partial differential, or functional-differential equations. There is no need for a formal definition here, since the example to be discussed shortly will serve the purpose adequately. Control theory deals with the problem of connecting compatible inputoutput systems to create new input-output systems satisfying some criterion of performance or otherwise directed toward some goal. As discussed in earlier sections, nature seems to be highly successful in this process. To illustrate the idea, consider two input-output systems Sl and S2 of compatible types, connected as indicated in Fig. 1.4.2. The resultant system S3 with input variable U3 and output X 3 is described as the consequence of wrapping a feedback system S2 about Sl' An example of why and how this might happen will now be discussed. The control-theoretic nature of Examples 1.2.1-1.2.3 becomes apparent under closer examination of the term f(t) in their differential equation models. In Example 1.2.1,
J
U(t)l
[ ~
f(t) =
= u(t)
[1]
(1.4.1)
~ .
The differential equations (1.2.1) can be written as
d [Xl]
dt
X2 X3
=
[-lo °:~o lo0-L][Xl] + 1~0 -
100100
X2
-50
X3
[1]
u(t) 0 . 0
(1.4.2)
Here the scalar control or input function u(t) provides an opportunity to influence the vector output or state vector variable X as it evolves with time. Observe that the value of the control function u(t) at time t directly affects the
16
1-------,
"'1 C
1"'·1
5,
5,
L
1. Introduction
hr" J
Fig. 1.4.2 Connecting input-output systems using feedback.
Xl
rate of change of (t), whereas the numbers in the coefficient matrix A direct the manner in which this control influence on Xl (t) indirectly feeds into the other state variables. Control theory attempts to illuminate the consequences of the manner in which the control variables enter the equations and their capabilities for altering the uncontrolled dynamics of the system. It tends to focus upon a limited number of mathematical problems abstracted from the study of applied problems. Undoubtedly other important concepts and fundamental problems will emerge as the field of applications expands. It will be sufficient, in this introductory chapter, to mention a few which occur in the continued discussion of Example 1.2.1. One fruitful concept is that of system controllability. Roughly speaking, the idea is to assess those output states which can be achieved by applying all allowed inputs. For Example 1.2.1, a typical controllability question would ask whether, over a prescribed finite interval of operation to ::s;; t ::s;; t l and for arbitrarily prescribed initial and final temperature values of water in each tank, there exists some appropriate control input function u(t) which will drive the initial temperatures at time to to their final prescribed values at time t i - It turns out that if the only constraint upon u(t) is that it be continuous, then it is in fact always possible, whatever the values of to < t 1 and prescribed temperatures, although this is not at all obvious. Similar questions could be asked concerning Examples 1.2.2 and 1.2.3, whose respective control differential equations are
[Xl] [0 1 ][Xl] + u()t[OJi ' -k /m -kdm d [Xl] [0 1 J[XlJ +ut()[OJLl' dt = -1/LC -R/L
d dt
X2
X2
=
2
X2
X2
(1.4.3) (1.4.4)
Again the answer is in the affirmative, and thus the mass in Fig. 1.2.2 can be driven from any state of position and velocity to any other such state over any
17
1.4 Control Questions
finite time interval by appropriate choice of input function u(t). From the similarity between (1.4.3) and (1.4.4) it is clear that the result translates into a corresponding valid statement about the driven circuit of Fig. 1.2.3. Other examples complicated by more state variables and multiple control inputs are easily imagined. Obviously the controllability can fail for those systems involving subsystems that are totally disconnected from inputs and controllable state variables, and in more subtle ways whose discussion is deferred to later chapters. The concept of controllability has strong mathematical appeal and obvious utility in mapping out an upper limit of the extent to which a system might be controlled. Interest in the phenomenon of an even more compelling sort has been stimulated through discovery of its remarkable implications concerning other important aspects of control. Stabilizability is one such notion, and automatic regulation is another. The general meaning of these terms is most easily conveyed by further discussion of Example 1.2.1. Intuition suggests that if the input u(t) into the system of Fig. 1.2.1 eventually became and remained zero, then the heat in the system would tend to a uniform distribution due to the circulating flow. Hence the water in each tank would approach the same limiting temperature
Too = _1_ [x? 300e
+ x~ + x~ + roo u(a) daJ Jo
(1.4.5)
as t - 00, in which e is the specific heat of water in the compatible units of calories per gallon per degree. Suppose that the intended purpose of the system were to drive the temperature of the water in the tanks to some prescribed value Too' If the initial temperatures were measured, the values of x? (i = 1, 2, 3) would be known, and then upon assuming some particular form of u(t) (such as constant for an appropriate initial interval and then forever zero), Eq. (1.4.5) could be employed to solve for the appropriate u(t). Each choice of Too and each "run" of the system would require recalculation of u(t) and an operator to turn the heat-exchanger on and then off. This raises the question of whether it might be possible to design a (regulator) device that would sense the tank temperatures and operate upon these along with a prescribed temperature command to drive the temperatures toward the prescribed value automatically. A simple form of regulator that might be considered would be of the linear type (1.4.6) in which the k, (i
= 1,2,3) are parameters to be determined and
Uc
is a
18
1. Introduction
constant input command. Thus the control problem is transformed into the study of the system with parameters kl> k 2 , k 3 , ue
~ [Xl] _ [- (k~ dt
X2 X3
-
100 1
TOO
lo)
3] [Xl] ~k12 fa -k 0 100 1
TOO
X2
-
1 50
X3
+
Ue
[1]0
(1.4.7)
0
resulting from substitution of (1.4.6) into (1.4.2). This equation is a concrete example of an input-output system with input Ue and output X = (Xl' X2' X3) obtained by wrapping a feedback system around the system of Example 1.2.1 in order to solve a control problem. It fits into the abstract diagram of Fig. 1.4.2 with U3 = Ue , 8 1 the input-output system of Fig. 1.2.1,and 8 2 the affine operator defined by Eq. (1.4.6). The stabilization problem is that of determining which values of the parameters k, (i = 1,2,3) in (l.4.7) will ensure that each x;(t) approaches a limit as t --+ 00 for every initial value X O of x. One elegant result of control theory states that the possibility for stabilization is ensured by the system's controllability. Hence the search for the appropriate values of the k;s is not futile. This is but one of many interesting relationships between various concepts of control theory. The question of how the limiting values ofthe X; depend on the kis and Ue is also of vital interest. For example, if the regulator were intended to drive all tank temperatures to the same prescribed value automatically, then it would be desirable to arrange for the limiting temperature to be some scalar multiple of U e if possible. The qualitative behavior of the responses of the feedback system to the initial X O and Ue , as well as certain quantitative properties such as the rate of convergence, also would likely bear upon the final choice of the values of the k;s in completion of the design. It is hoped that this discussion of the examples has given the reader at least a glimpse of the manner in which control problems can lead to interesting questions about differential equations. It should also be evident that a firm foundation in the classical theory of differential equations is a logical prerequisite for the pursuit of such questions. Hence, in Chapter 2 we shall begin the study of the classical theory by reviewing some basic matrix algebra.
Chapter
2
Matrix Algebra-The Natural Language of Linear Systems
The invention of vectors and matrices arose from the need to manage large numbers of variables and equations efficiently. Although eventually they must be pursued to a somewhat higher level of abstraction, the basic ideas are quite visible in the problem of solving systems of linear algebraic equations. That problem is taken as the starting point for a rapid review of the standard theory discussed in the context of finite-dimensional real and complex number spaces. This chapter paves the way for the elementary treatment oflinear systems of differential equations with constant coefficients that appears in Chapters 3 and 4. It also provides a convenient intermediate point from which to complete the transition to the abstract axiomatic view of vector spaces as its need begins to materialize at the end of Chapter 4. 2.1 Terminology
An m x n matrix A over afield!F is defined as any function with domain {(i,j)ll ~ i ~ m, 1 ~j ~ n} and range in a field fli. In this book, the fields
employed in the discussions of differential equations are all subfields of the complex number field re, i.e., subsets of re closed under the operations a - b and ab - 1, b :1= 0 on its elements a, b. The most frequently occurring examples are fli = re or the field of real numbers !F = fJl. Elements of fli are called scalars.
The standard notational convention denotes the value of A at (i,j) by A;j' called the (i,j)th element of A. When the elements of A are arranged in the 19
20
2. Matrix Algebra-The Natural Language of Linear Systems
rectangular array
(2.1.1) A m2· · ·
the index (variable) i is interpreted as a row position and j as the column position. Conversely, every such array offield elements determines a matrix A over IF. If m = n, the A is called square. The positions in A occupied by Au (i = 1, 2, ... , n) are referred to as the main diagonal. A square matrix is called a diagonal matrix if all its elements off the main diagonal are zero, and a diagonal matrix A with all A ii = c is called a scalar matrix, which is identified with the scalar c in IF. An upper-triangular matrix is one whose elements Aij satisfy Aij = 0 for i > j, and a lower-triangular matrix is one satisfying that equality for i < j. 2.2
Addition, Multiplication, and Scalar Multiplication
Problems of working with large numbers of scalar variables and numerical data motivated development of matrix algebra as an extension of the one-dimensional scalar case. The following are the standard operations which arose:
Definition Let A and B be m x n matrices over a field IF. The matrix sum + B is the m x n matrix defined by
A
(A
+
B)ij = Aij
+
Bij'
(2.2.1)
It is a simple matter to verify the following consequences of definition (2.2.1): A A
+ (B
+B = + C) =
+ A; (A + B) + C B
(2.2.2) (2.2.3)
for all m x n matrices A, B, C over IF; A+Z=A
(2.2.4)
has a unique solution for each m x n A, namely, Zij = 0 for all i.j; Z is called the m x n zero matrix and is denoted by 0; and A+N=O
(2.2.5)
has a unique solution N for each m x n matrix A, namely, N i j = - A ij for all i,j; N is called the additive inverse of A. The standard notation for N is - A.
21
2.1 Addition, Multiplication, and Scalar Multiplication
Definition A second operation on an m x n matrix A over f/i called scalar multiplication by c in f/i is defined by
for all
(CA)ij = cA ij
(2.2.6)
i,j.
As a simple consequence of the definitions it follows that (cIC2)A
=
(2.2.7)
CI(c2A),
+ c2)A = cl(A + B) =
cIA
(CI
cIA
+ C2 A, + c .B,
(2.2.8) (2.2.9) (2.2.10)
lA = A
for all m x n matrices A, B over f/i and scalar
Cl' C2
in f/i.
Definition Let A and B be m x rand r x n matrices, respectively, over f/i. The matrix product AB is the m x n matrix defined by (AB)ij
=
r
I
k=1
(2.2.11)
AikB kj
= 1,2, .. 0' m;j = 1,2,. 0" n). Note the compatibility requirement, namely, that A must have the same number of columns as B has rows.
(i
The appropriateness of the definitions of matrix addition and multiplication for producing extensions of familiar algebraic operations on scalar variables to matrices is supported by the following consequent properties: (2.2.12)
(AB)C = A(BC),
+ B)C = C(A + B) =
(A
AC CA
+ BC, + CB
(2.2.13) (2.2.14)
for all matrices over f/i of appropriately compatible sizes. The manner in which algebraic properties of f/i extend to matrices via definitions (2.2.1) and (2.2.11) is illuminated by the following calculation: [(A
+ B)C] ij =
I
(A
k=l
+ B)ikCkj
r
=
I
k=l
(A ik
+ Bik)Ckj
r
=
I
k=l
= (AC
AikCkj
+
+ BC)ij'
I
k=1
=
I
k=1
BikC kj
(AikCkj
=
(AC)ij
+ BikCkj) + (BC)ij (2.2.15)
22
2. Matrix Algebra- The Natural language of linear Systems
Since (2.2.15) holds for all i, j, obviously the corresponding elements of (A + B)C and AC + BC are the same elements of:F and (2.2.13) is thereby proved. There is a matrix generalization of the multiplicative identity 1 in :F. That is, (2.2.16)
IA = A = AI
has a unique n x n matrix solution I over:F holding for all n x n A over :F; namely, (I)ij =. bijwhere bij takes on the value 1 in :F when i = j and the value 0 in :F when i '" j. I is called the n x n (multiplicative) identity matrix over:F and is denoted by In when its size needs to be emphasized. For n x n matrices A over:F and c in:F, cl; is a scalar matrix and (2.2.17) Thus it is natural to identify each c in:F with the matrix cl.; Scalar matrices commute, under multiplication, with all square matrices of corresponding size and, more generally, for all scalars c in :F, (2.2.18)
(cA)B = A(cB)
for all matrices A, B over :F of compatible size. Thus the definitions of matrix addition, multiplication, and scalar multiplication provide extensions of many of the formal rules of algebra in :F to matrices over :F. A word of warning: Even with square matrices, not all the formal manipulations and inferences of elementary algebra carry over to matrices. For example,
(2.2.19)
AB", BA
for some square matrices A and B; and AB = 0
(2.2.20)
does not imply that at least one of A, B is the zero matrix. Hence while a factorization such as (2.2.21) is valid for all n x n matrices A over gj, the factorization A2
-
B 2 = (A - B)(A
+ B)
(2.2.22)
fails for matrices satisfying (2.2.19), and it would be erroneous to conclude
23
2.3 The n-Vector Spaces 9'"
from the equation (2.2.23) that A = ±2In • EXERCISES
1.. Prove the following matrix identities directly from the definitions of matrix sum, product, and scalar product: (a) A + B == B + A, (c) A(BC) = (AB)C, (e) (- c)A = - (cA),
(g) cO
(b) (d) (f)
= O.
A + (B + C) = (A + B) C(A + B) = CA + CB,
+ C,
OA = 0,
for matrices A, B, C and scalars c in fF. 2.. A nonstandard product A * B of an r x n matrix A with an m x r matrix B, both over a field fF, is defined by (A
* B)ij =
r
L AkjB ik· k=l
(a) Do the identities corresponding to (c) and (d) of Exercise 1 hold for the nonstandard product? (b) Solve the equations X * A = A and A * X = A for the unknown n x n matrix X with A likewise n x n. (c) Find matrices A, B for which A * B i= AB. (d) Find an example 'for which A * B i= B * A. 3. Find an example of a square matrix A for which A 4 i= 0 but AS = O. 4. Evaluate (A - 2B)2 - A 2 + 4AB - 4B 2 in which A =
101]
[o -1
1 2, 1 3
B=
[~
-1
~ ~].
1 0
2.3 The n-Vector Spaces 17-
The n x 1 matrices over a field fF are called n-uectors over fF, with the collection of all such n-vectors being denoted by fFn. As matrices, the elements of fFn can be added and multiplied by elements of fF to again produce n-vectors in fFn. These two operations thus conform to Eqs. (2.2.1)(2.2.10). fFn together with the two operations is called the n-oector space
24
2. Matrix Algebra-The Natural Language of linear Systems
over :IF. Often n-vectors are called column-vectors or simply vectors. The superfluous column index 1 is dropped, and thus the equations defining addition and scalar multiplication of n-vectors are written as
Xl] [Yl] [Xl +-Yl] X2 Y2 X + Y2 ·· + .. = 2 .. , . . [· X" Yn X" + Y" C
Xl] X2
.. •
[
Xn
=
[CXl] CX2 .
..
,
(2.3.1)
(2.3.2)
CX"
respectively, in which Xi' Yi' and c are all elements of:IF (i = 1,2, ... , n). If v1 , V 2' ... , Vm are n-vectors over :IF and c l' C2, ... , Cm are scalars in :IF, then the vector ClV l + C2V2 + .. , + CmV mis unambiguously defined in:IF" (since matrix addition was found to be associative) and is called a (finite) linear combination ofvl' V 2, ••• , V m with respective coefficients Cl' c 2 , ••• , Cm • A subspace of :IF" is any nonempty subset of :IF" containing all linear combinations of vectors taken from the subset. (The subset is said to be closed under linear combinations.) A subset!/' of distinct vectors in .'F" is called linearly independent if the only linear combinations of distinct elements taken from!/' that yield the zero vector are those with all coefficientszero. If!/' is not linearly independent, then it is called linearly. dependent. If !/' is a subset of :IF", then the collection of all linear combinations of vectors taken from!/' is called the span of !/' and is denoted by span(!/'). Note that if !/' is not empty, then span(!/') is a subspace of :IF". Let!/' be a subspace of :lFn • A subset of!/' is called a basis for!/' if it is linearly independent and spans !/'. A basis for!/' is called an ordered basis if its elements have been placed in a one-to-one correspondence with a collection of consecutive positive integers, including the integer 1. Important facts concerning the above concepts in :IF" are summarized by the following exercises. EXERCISES
Prove the following statements: 1. A collection {Vl' V2" .., vm} of distinct vectors in:IF" is linearly dependent ifand only ifat least one of the vectors in the collection is a linear combination of the others.
25
2.4 Systems of linear Algebraic Equations
2. If g is a subset of distinct vectors in a subspace of IF n , then it is a basis for the subspace if and only if each vector in the subspace is a linear combination of distinct vectors in g with the choice of nonzero coefficients being unique. 3. The single zero n-vector over IF constitutes a subspace of IF n called the trivial subspace of IF n. It has no basis and is said to have dimension zero. Each nontrivial subspace of IF n has a basis; the basis can be ordered; and any two ordered bases for the subspace have the same (finite) number of elements, called the dimension of the subspace. 4. IFn is a subspace of itself having the consecutive columns of the identity matrix In = (en e2' ... , en) as an ordered basis called the standard ordered basis for IF n.
2.4
Systems of Linear AlgebraicEquations
A system of m linear algebraic equations all xl + al2x2 + a21 xl + a22x2 +
+ alnXn = bl , + a2nXn = b2,
(2.4.1)
with the aij, b, (i = 1,2, ... , m; j = 1,2, ... , n) prescribed elements of a field IF, can be written as the single matrix equation Ax
= b.
(2.4.2)
The m x n matrix A with elements Au = aij is called the coefficient matrix;
(2.4.3)
is regarded as an n-vector variable in IFn; and the m-vector b is defined as
(2.4.4)
Equations (2.4.1) and (2.4.2) are called homogeneous if b = O. It is possible to write down examples of (2.4.2) that have no solution, precisely one solution, or an infinite number of solutions. Generally, counting
26
2.
Matrix Algebra-The
Natural Language of Linear Systems
equations and unknowns provides no information about which is the case. Fortunately, there exists a systematic method of computing that generates any and all solutions and detects the case for which there is no solution. The technique involves performance of elementary row operations on the augmented matrix (Alb),obtained by adjoining b as an (n -t 1)st column to A . The three allowed operations are (I) addition of any scalar multiple of one row to any other row, (11) multiplication of any row by any nonzero scalar, and (111) interchanging any two rows. (It is an easy exercise to show that a type I11 operation can be accomplished by an appropriate sequence of elementary row operations of types I and 11.) The method is based on the fact that the solutions to the system (2.4.2) are preserved (invariant) under the three elementary row operations. Hence an appropriate finite sequence of elementary row operations is applied to (Alb), transforming it into its so-called row-reduced echelon form.A matrix is in its unique row-reduced echelon form if the following conditions are met: 1. All the zero rows (if there are any) occupy the bottom rows. 2. The leftmost nonzero term in each nonzero row is a one, called a leading one. 3. Each column containing a leading one has all its other elements zero. 4. The column occupied by a leading one is to the right of the column occupied by the leading one in the row (if there is one) directly above it.
Example 2.4.1 The solution to the system with row-reduced echelon form of its augmented matrix 0
1 0 - 1 (2.4.5)
0 0 0 is x2 = x4
+ 3,
0
x3 = -2x4 - 1,
x1 and x4 arbitrary.
(2.4.6)
That is, the solutions are all vectors of the form
[[
:[2:
ll
a,b arbitrary].
1]
(2.4.7)
2.4
27
Systems of Linear Algebraic Equations
Example 2.4.2 The unique solution to the system having row-reduced echelon form of its augmented matrix (2.4.8)
[:x ;],
(2.4.9)
0 0 0 0
It is easy to show that the row space of A is invariant under elementary row operations on A . Hence the nonzero rows of the row-reduced echelon form of A are a basis for the row space of A and the number of such rows is the rank of A . In particular, row reduction provides a method for testing for the It . is also true linear independence of a finite collection of m-vectors in 9'" but not obvious that rank ( A T )= rank ( A ) .
CRITERIA FOR THE SOLVABILITY O F SYSTEMS
The following results are quite transparent when (2.4.2) is looked at in the form for which its augmented matrix is in row-reduced echelon form. With A an m x n matrix, the homogeneous equation AX = 0
(2.4.11)
has a nontrivial (i.e., nonzero) solution if and only if rank(A) < n. A sufficient
28
2. Matrix Algebra-The Natural Language of Linear Systems
condition for (2.4.1 I) to have a nontrivial solution is that m < n, i.e.,that there be fewer equations than unknowns. In general, (2.4.2) will have at least one solution if and only if rank(Alb) = rank(A).
EXERCISES
1. Find all solutions to the following systems by row reducing the associated augmented matrices: (a)
(b)
(c)
3Xl + 2X2 + 3X3 + 12x4 = 9, -Xl + X2 - 3X3 - 3X4 = -5, Xl + X2 + 3X4 = 2, Xl + X2 + X3 + 5X4 = 3. Xl + X2 + 4X3 = 3, 3Xl + 2x 2 + IOx3 = 4, 2Xl + X2 + 6X3 = 2. Xl - X2 - X3 = - 2, Xl + 2x 2 + X3 = -3, Xl + 2x 2 + 2X3 = 1, Xl + 4X2 + 2x 3 = -5, Xl + 3x 2 + X3 = -6.
2. Compute the ranks of the following matrices. (a)
(c)
[1 2-1]
1 1 2, 231
[ 12]u: -1 0 1 1 1
(b)
[l
2 0 1 2 1 3
(d)
[~
3 2 1 2
~l -~l
234 3. Determine all the subsets of rows which are bases for the row space for the matrix (c) of Exercise 2. 4. Prove that the transpose operation has the following properties: (a) (b) (c)
(AT)T = A, (AB)T = B TAT, (A + B)T = AT
+ BT.
29
2.5 Matrix Inversion
2.5 Matrix Inversion
In the scalar case for which in (2.4.2) m = n = 1, an elementary method of solving (2.4.2) is to multiply that equation by the solution L to LA
= In
(2.5.1)
(at least in the case for which the coefficient of x in (2.4.2) is not zero.) Observe that if (2.5.1) has a solution L, the same technique solves (2.4.2) for general n. By writing out (2.5.1) as n column equations, it is apparent that (2.5.1) is equivalent to a system of type (2.4.2) but of a special form and larger size. Equation (2.5.1) plays such a prominent role in matrix theory that its solution is given the status of a definition.
Definition If A is an n x n matrix over a field:#' for which (2.5.1) has a matrix solution Lover :#', then A is called invertible or nonsingular and L, ordinarily denoted by A-I, is called the (multiplicative) inverse of A. Remark 2.5.1 It is possible to prove that if L is a solution to (2.5.1), then it is also a solution to AL
= In
(2.5.2)
and conversely, if a matrix L satifies (2.5.2), then it satisfies (2.5.1) as well. Remark 2.5.2 As a corollary of Remark 2.5.1, it is an easy exercise to show that if A has an inverse, then the inverse is unique and is itself invertible with (A- 1 ) - 1 = A. COMPUTING A-'
The problem of inverting a matrix is one of solving a system of linear algebraic equations. To see that this is indeed true, let A be an n x n invertible matrix and denote the columns of A-I by Cl' C 2, ... , Cn and those of In by e 1 , e2' ... , en' Equation (2.5.2) can then be written as (2.5.3) which is the same as (2.5.4) Thus the problem of computing A-I = (c., C2, problem of solving the system of vector equations Ac; = e;
(i
... ,
= 1,2, ... , n).
cn) is precisely the
(2.5.5)
30
Matrix Algebra-The
2.
Natural Language of Linear Systems
But all these equations have the same coefficient matrix A. Hence by row reducing (Ale,, e2, . . ., en),all equations (2.5.5) are solved simultaneously, and clearly the row-reduced form of the matrix (Ale,, e2, . . ., en)will be U,IA
-
(2.5.6)
).
Hence for any n x n matrix A, either row reduction of (AII,) will produce a matrix with the first n columns constituting I,, in which case A is invertible and A-' can be read off the reduced form (2.5.6), or else the row-reduced form will have a string of n zeros at the left end of the bottom row, indicating that A is not invertible. This method for computing A-' is called the Gauss-Jordan method.
Example 2.5.1 Consider the matrix A=
[ ' :]. -1 -1 -1 -2
(2.5.7)
-2
To study its invertibility, row reduce the associated augmented matrix
[ [
1 2 3 -1-1 2 -1 -2 -2
1 2 0 0 1 0 0 0 1
1 0 0 0 1 0 0 0 1-
I
(2.5.8)
- 2 0 - 3 - 4 1 - 5 1 0 1
The conclusion drawn is that A is invertible and the inverse is read off to be
(2.5.9)
Example 2.5.2 Consider the matrix A=
['
-1 -1 2 5
:],
11
(2.5.10)
31
2.6 Large-Scale Matrix Computations Using Computers
whose augmented matrix row reduces as follows:
H
2 3 -1 2 5 11
1 0 0 1 0 0
~]-[~ -[~
2 3 1 5 1 5 2 3 1 5 0 0
1 0 1 1 -2 0 1 1
0 1 -3 -1
~]
(2.5.11 )
~]
and the calculation can be terminated, since the three zeros in the bottom row indicate that A is not invertible.
2.6 Large-Scale MatrixComputations Using Computers
In recent years the capability for doing numerical matrix computations on digital computers has progressed to the point where large-scale problems can now be solved quickly and accurately. This success is attributable to the numerical analyst's careful attention to the accuracy and efficiency of the algorithms developed for doing the computing and the programming effort expended in implementing those algorithms. Any scientist contemplating extensive numerical work is advised to consult the references to learn about the capabilities and limitations of available packaged routines such as ElSPACK and UNPACK (see [5, 18]). Much of Chapter 3 concerns derivation of appropriate forms of the equations for treating large-scale systems of differential equations. For a discussion of the large-scale systems of linear algebraic equations to which they are reduced, the reader should consult the cited references. Only one aspect of the latter problem is mentioned here. Programs for solving large systems of linear algebraic equations ordinarily do not reduce the augmented matrix to the echelon form described earlier in this chapter. The more widely used method of Gauss elimination row reduces the matrix to an upper-triangular form and then completes the computation by back substitution, starting with the bottom nonzero row, rather than proceeding with back elimination to reach the echelon form. To compare the efficiency of algorithms, suppose that each division and each multiplication-addition is counted as one operation. For an n x n nonsingular system, it is easy to show that Gauss elimination takes the order
32
2,
Matrix Algebra-The Natural Language of Linear Systems
of !-n3 operations to reach the upper-triangular form and the order of another in2 operations to complete the back substitutions-a total still of the order tn3 • (The back substitution takes fewer operations, since it is done to a triangular matrix.) Clearly the multiplication of two n x n matrices done in the standard way takes n 3 multiplications-the same order of operations required to invert an n x n matrix using the Gauss-Jordan method described in Section 2.5. Thus Gauss elimination compares favorably to the method of solution based on inverting the coefficient matrix. Suppose that the task were to solve the system of equations Ax;
= b,
(2.6.1)
= 1,2, ... , m) in which A is an n x n invertible matrix. A novice might make the mistake of applying Gauss elimination to reduce each matrix (Alb;), with a total operation count of order !-mn 3 • If all the data b, were available at the start, a more efficient strategy would be to apply the Gauss elimination to the augmented matrix (Alb 1 , b2 , ••• , bm ) , which could be done with the order of j-n3 + mn 2 operations-a worthwhile improvement if n were large and m large but relatively small compared with n. If m were much larger than n, then the latter method again would be more efficient. Of course, in an application in which some of the initial XiS had to be computed before the later bjs arrived, the reduction of (Alb 1 , b2 , ••• , bm ) just one time would not be possible. However, there is another efficient solution to the problem that the uninitiated most likely would overlook! Suppose that (2.6.1) were to be solved for m relatively large compared with n. Obviously the only reason for row reducing A in (2.6.1) is to generate the row operations that need to be applied to each b.. This information is available after solving (2.6.1) for i = 1. The problem reduces to the question of how best to record those operations so that they can be repeated on the other biS in whatever order and whenever they arise. A solution to this problem is well-known in numerical analysis. It is called triangular (i
factorization.
Triangular factorization is particularly well adapted to problems of type (2.6.1) in which the coefficient matrix A is common to all the equations. The idea is to compute a lower-triangular matrix L and an upper-triangular matrix U such that A = LU. Then each equation (2.6.1) splits into the pair LYi = b.,
(2.6.2)
Ux, =Yi
(2.6.3)
(i = 1,2, ..., m). The solution Xi to (2.6.3)-and hence to (2.6.I)-is achieved by first solving (2.6.2) to get Yi and then substituting Yi into (2.6.3) to get Xi'
33
2.6 Large-Scale Matrix Computations Using Computers
The advantage gained by the pair is that each takes but i n 2 operations, since L and U are triangular. The computation of L and U is accomplished once and for all in i n 3 operations. Thus the total count for LU factorization is about i n 3 + mn'. Notice the improvement in efficiency over the first approach, which took the order of 3mn3 operations. An advantage of the LU approach over the second method is that one equation in (2.6.1) can be solved at a time with no penalty in computation. The matrix U is the one obtained by doing Gauss elimination in reducing A to upper-triangular form. L is simply a bookkeeping matrix that records the multipliers involved in the triangularization of A and requires no operations. For a more detailed discussion of LU factorization and related topics see [ 5 ] .
Example 2.6.1 This example carries out the solution of (2.6.1)by the method utilizing L U factorization for A=
[ J 2 1 3 - 2 - 10 1
bl =
[-;I9 [;I. b,
=
" ' 3 " ["3
(2.6.4)
By Gauss elimination A is reduced to upper-triangular form to get 1 3 -2 0 1 - 0 1 4 + 0 1 4 = u , 4 -1 2 0 - 3 -4 0 0 8
(2.6.5)
and the multipliers used are recorded in the corresponding positions below the diagonal of the identity matrix to get 1
L=
.]
0 0 1 2 -3 1
[-'
(2.6.6)
Note that LU = A as intended. The solution to (2.6.1) for i = 1 is accomplished by doing the forward and backward substitutions that solve (2.6.2) and (2.6.3):
[-:
0 0 1 0 2 -3 1
[
2 1 3 0 1 4 0 0 8
(2.6.7)
(2.6.8)
34
2. Matrix Algebra-The Natural Language of LinearSystems
The reader is invited to repeat the substitutions following replacement of the augmented column in (2.6.7) by b 2 to get X2 = (-i, -1, !)T. The problem of efficiency is just one of those that must be dealt with in computerized numerical linear algebra. A. more extensive discussion of it and other problems such as roundoff errors can be found in the cited references. EXERCISES
1. Determine which of the following matrices are invertible and compute the inverse in those cases where it exists.
(a)
H
(b)
~],
-1
(c)
2.
[~
(d)
[~
1
3
5
o 1 o -1 o 0
fl
Prove the following statements: (a) If A and B are invertible n x n matrices, then AB is invertible and
(AB)-l = B-IA- I. (b) If A is invertible, then so is A-I and (A-l)-l = A. (c) If A is invertible, then so is AT and (AT)-l = (A-I)T. (d) rank(AA T ) = rank(A) for every m x n matrix A. (e) An n x n matrix A is invertible if and only if rank(A) = n. 3. Compute all the matrix solutions X to the matrix equation AX = B,
where
1 - 1 1
2]
A= 3 02 -1, [2 1 1 3 (Hint:
4 0 4]
B= 2 6 7 .
[ 409
minimize the calculations by row reducing the appropriate matrix.)
35
2.7 Determinant Functions
2.7
Determinant Functions
In the following discussion A is an n x n matrix variable over a field !F written out in terms of its columns A = (al> a2'" ., an)'
Definition A determinant function det(A) is defined to be an fF-valued function (of the columns) of A, det(A) = dia«, az, ... , an), with the properties (1) d(el' ez,· .. , en) = 1, where In = (el' ez,"" en);
d(a 1, az,"" an) = -d(al' az,.··, an), where the matrix (ai, a z, , an) arises from the interchange of any two adjacent columns of (ai' az, , an); (3) dia«, az,···, an)la,=ktbt +k2 b2 = k 1d(al> az,···, an)!a,=bt + k zd(al' az,···, an)la,=b2 for every kj in IF, bj in IFn (j = 1,2; i = 1,2, ... , n). (2)
A function satisfying (2) is called alternating and one satisfying (3) is said to be n-linear. Hence a determinant function is an n-linear, alternating IFvalued function of an n x n matrix variable A over fF with value 1 at the identity matrix. Case n = 1. For n = 1 the function det(a) = a is clearly a determinant function and there is no other, since if d(a) is a determinant function, by (1)-(3), d(a) = d(a'l Case n
+ 0,1) = ad(l) + Od(l) =
a.
= 2. For n = 2 and A
11 12] det[a a a21 a22
==
[aau11 a22 a12 ] ,
the standard function,
= a11 aZ2 - a12a21
can easily be checked to be a determinant function, and there is but one, since if dia-, a2) is a determinant function, properties (1)-(3) can be applied to compute d[:::
:::]
= d[ a11 =
a11d[~
[~]
+
: ~]
a2{~J[: :] +
a21d[~
: ~] (continues)
36
2. Matrix Algebra-The Natural Language of Linear Systems
=
+
a2Id[ ~1
a12
[~J
all
{a12d[~
~J
+ a22d[~
+ a21
{a12d[~
~J
= alla22d[~
~J
= alla22-
+ a22
[~J] .~J}
+ a22d[~
~J}
+ a2IaI2d[~
a2Ia12d[~
~J
~J =
alla22 - a21 a12'
(2.7.1)
Case n General. Expansion by Minors Formulas. Let A be an n x n matrix and let AOJj) denote the(n - 1) x (n -l)matrixobtainedbystriking out the ith row and jth column of A. Let D be a determinant function on (n - 1) x (n - 1) matrices. Then the formulas n
det(A) =
I
( - l r j AijDA(iU)
(2.7.2)
i= I
can easily be shown to define a determinant function on n x n matrices for each fixedj (j = 1,2, ... , n). In this way, by induction, a determinant function can be shown to exist for each n = 1,2, .... With some work it can be shown that for each n there is but one determinant function, which is then simply called the determinant of order n. Hence in (2.7.2) D can be replaced by det and the resultant formula is one of the expansion by minors formulas. Thus the right-hand side of (2.7.2) is actually independent of j. This and other consequent results concerning determinants can now be summarized. DERIVED PROPERTIES OF DETERMINANTS
det(a l, a2""'~)
=
0 if a, = aj
for some
det(al' a2' ... , an)la, =kb = k det(al' a2' ... , an)1 a, =b for each k in IF, b in IFn (i
i "# j.
(2.7.3) (2.7.4)
= 1,2, ... , n).
det(AR) = det(A) deteR). det(A- I } = [det(A)]-1
(2.7.5) (2.7.6)
37
2.7 Determinant Functions
for nonsingular A. det(AT ) = det(A).
(2.7.7)
n
L (-l)i+iAijdetAUIj)
det(A) =
(j = 1,2, .. . ,n).
(2.7.8)
(i = 1,2, ... , n).
(2.7.9)
i= 1 n
det(A) =
L (-l)i+iAijdetA(ilj)
i> 1
(2.7.10) in which the sum is over all permutations a of {I, 2, ... , n} and sgn(a) takes on the value 1 or -1 as a is even or odd, respectively. Formula (2.7.10) is sometimes taken as the definition of the determinant function and is logically equivalent to that employed in Section 2.7. Frequent use will be made of the following standard result concerning determinants. Theorem 2.7.1 For A an n x n matrix over afield iF, the followinq statements are equivalent:
det(A) = O. Ax
=0
has a nontrivial solution A
is not invertible.
(2.7.11)
x in
(2.7.12) (2.7.13)
COMPUTING DETERMINANTS
Formula (2.7.10) shows that the determinant of a matrix variable is a very special polynomial in the elements of the matrix. Note that if the matrix is triangular, then the value of its determinant is just the product of the diagonal elements. Determinant functions are used primarily in theoretical work. In numerical work their use should be avoided when possible, since the number of arithmetical operations involved in their evaluation tends to grow rapidly with the size of the matrix. When the determinant of a sizable matrix must be computed, it is generally advisable to base the calculation on triangularization of the matrix using elementary operations on rows and columns. It is apparent that every square matrix can be triangularized by row operations. The calculations often utilize (2.7.4) and (2.7.7) as well as the invariance of the value of a determinant under type I row operations (which follows readily from the defining properties of the determinant function). Another useful operation is the interchange of two rows or columns, which simply alters the
38
2. Matrix Algebra-The
Natural Language of Linear Systems
sign of the value of the determinant. The latter is a consequence of the alternating property. For notational convenience det(A) is occasionally denoted by [ A [ . -
Example 2.7.1 0 1 3-1 2 1 3-1
1-2 2 2 3 1 2 - - _1 0 1 2 1 3 3 2 2 2 1 4 3 - 1 1
4 3 2 4
1-2 2 4 - - _1 0 1 2 3 2 0 5 -1 - 6 O 5 - 5 -8
1 0
0 0 -15
1 0 0 0 1 1 0 1 0 1 0 0 = - 2 0 2 0 0-11 1 0 0 -15 7 0 1 - _1 0 2 0 0
0 0 1 0 0 6 2 0 0
0
0 1 0 0 6
0 0 0 2
0
-23 0 0 1 7
0 0 = 31. 7 1
(2.7.14)
EXERCISES
1. Derive the formula det(A) = a l l % ~ a 3 3+ a13a2lU32+ a12a23a31 - a11a23a32
- a12a21a33 - 413u22u31
for a 3 x 3 matrix: (a) directly from the defining properties (1)-(3), (b) directly from formula (2.7.10), (c) directly from (2.7.9) and the standard formula for a 2 x 2 matrix. 2. Prove that an interchange of any two rows of a matrix results in the change in sign of the value of the determinant. 3. Evaluate the determinant of each of the square matrices appearing in Exercise 1, Section 2.6.
39
2.8 Eigenvalues, Eigenspaces, Characteristic and Minimal Polynomials
4. Show that (2.7.5) implies (2.7.6). 5. Apply the defining properties (1)-(3) of a determinant to prove (2.7.3) and (2.7.4). 6. Deduce (2.7.7) from (2.7.10). 7. Prove Theorem 2.7.1. 8. Verify that (2.7.2) defines a determinant function of an n x n matrix.
2.8
Eigenvalues, Eigenspaces, Characteristic and Minimal Polynomials
Definition Let A be any n x n matrix over a field iF and v be an n-vector in iF n • The scalar A in iF is called an eigenvalue (or characteristic value) of A if (2.8.1)
Av = AV n
for some v :I: O. If Ais an eigenvalue of A, then any v in iF satisfying (2.8.1) is called an eigenvector (or characteristic vector) associated with A. The collection "11/;. of eigenvectors associated with an eigenvalue Ais clearly a subspace of iF n• Note that if 1 is an eigenvalue of A, then the problem of computing "11/;. is the problem of finding all solutions v in iF n to the system of linear algebraic equations
(2.8.2) (Un - A)v = O. A polynomial p(x) = Co + CtX + ... + cnxn of degree n is said to be over the field iF if its coefficients are elements of iF. Such a polynomial is called monic if c; = 1.
Definition The characteristic polynomial p..i x) of an n x n matrix A over a field iF is the monic polynomial defined as PA(X) = det(xl n
-
A).
(2.8.3)
In view of (2.8.2) and Theorem 2.7.1, it is apparent that the eigenvalues of A are precisely the-roots of PA in iF. By the fundamental theorem of algebra, for iF the complex field, A will have n eigenvalues, possibly with repetitions. A real matrix may be regarded as over the real field or over the complex field. If A is an n x n matrix with real elements, it might not have any eigenvalues when regarded as a matrix over the real field, e.g., A = [_? aJ. The main point to be made is that the problem of computing the eigenvalues and eigenvectors of a matrix over a field is a problem of computing the roots of a polynomial and solving the associated systems of linear algebraic equations.
40
2. Matrix Algebra-The Natural Language of Linear Systems
This is not to say that eigenvalues should be computed by numerically computing with PA(X). Numerical analysts have a variety of algorithms which work more directly with A, each with its virtues and disadvantages. For f(x) = Co + c ix + .. , + cmxm a polynomial over a field ff define f(A) = col + ciA + '" + cmAm called a matrix polynomial. If f(A) = 0 then f is said to annihilate A. These notions appear in the following theorem. Theorem 2.8.1 Cayley-Hamilton Let A be any n x n matrix over afield ff and PA be its characteristic polynomial. Then PA(A) = 0, i.e.; PA annihilates A.
Definition The minimal polynomial qA of an n x n matrix A over a field ff is defined to be the monic polynomial over ff annihilating A and ofleast degree. It can be shown that qA divides PA and that both polynomials have the same distinct roots in ff. Thus qA can be computed by finding PA which is then factored over ff into the product of powers of distinct prime polynomials. The minimal polynomial is then obtained by the finite process of deleting one prime factor at a time and testing to see whether the resultant polynomial annihilates A. The proof of these facts concerning PA, qA, and Theorem 2.8.1 can be found in [11].
A demonstration of the calculations involved in obtaining the characteristic polynomial, eigenvalues, and associated eigenvectors is given for Example 2.8.1
A =
2 2 -2] -4 3 1, [ 413
(2.8.4)
where A is regarded as a matrix over the real field ~. First it is necessary to compute the characteristic polynomial PA(X) = Ix13
=
-
AI =
(x - 4)
x-2 2 -2 -1 4 x-3 -1 x-3 -4
x-2
-2
4
x-3
o
1
= (x - 4)[(x - 2)2 + 16].
=
2 x-2 -2 -1 4 x-3 0 x-4 x -4
2 -1 = (x - 4)
1
x - 2
-4
4
x-2
0
o
2 -1 1
(2.8.5)
From PA it is apparent that A has but the single eigenvalue A = 4 in ~. The associated eigenvectors are obtained by row reducing the matrix of (2.8.2)
41
2.8 Eigenvalues, Eigenspaces, Characteristic and Minimal Polynomials
[
2 -2
-44 - 11
3-t
2
1 - 1
-;
0
1
5 - 50 0.
3 (2.8.7)
which is noted to be one-dimensional.
Example 2.8.2 Consider the same problem as in Example 2.8.1 but with A now regarded as a matrix over the complex field V. The characteristic polynomial computes exactly as in (2.8.5) but now factors into a product of linear polynomials over V, PA(x)
= (X -
4 ) [ ( ~- 2)’
= (x - 4)[x - (2
+ 161
+ 4i)] [x - (2 - 4i)l.
(2.8.8)
Hence the eigenvalues of A are I = 4, 2 f 4i. From the calculations of Example 2.8.1 it is possible to immediately write (2.8.9)
4i 4 [-4
-2 -1+4i -1
’
2 -1 -1+4i
0
1
0
(2.8.10)
42
2. Matrix Algebra-The Natural Language of Linear Systems
From (2.8.10) it follows directly that (2.8.11) which also has dimension 1. Similar calculations show (2.8.12) Remark 2.8.1 If A is a real n x n matrix with a complex eigenvalue A. = a + ico, the associated eigenvectors can be computed by real arithmetic. Let v = x + iy. Equation (2.8.1) splits into the pair of equations over fJl, (rxI - A)x
=
(2.8.13)
wy,
(2.8.14)
(rxI - A)y = - wx,
which can be written as [(IXI - A)Z
+ wZI]x = 0,
(2.8.15) (2.8.16)
Y = (Ilw)(rxI - A)x.
The former is a homogeneous linear system over fJl that can be solved for x, and then y can be computed by substitution into (2.8.16), doing only real arithmetic.
EXERCISES
1. Find the characteristic and minimal polynomials of the following matrices:
. (a)
(c)
[~ [~
21]
-1
o
1 2 0
1, 3
(b)
[~,
1 0 kz k3
~l
n H ~J (d)
2 1 0 0
0 0 2 1 . 0 2
43
2.9 The Jordan Form of a Matrix
2. Compute the eigenvalues and associated eigenspaces of the matrices in Exercise 1, first regarding the matrices as over 9 and then over W. 3. Let A be a real n x n matrix having eigenvalue u + iw with associated eigenvector x + iy where x and y are in P.Show that u - i o is likewise an eigenvalue of A with associated eigenvector x - iy.
2.9
The Jordan Form of a Matrix
Let A be an n x n matrix over a field 9 in which the characteristic polynomial pA “splits.” That is, assume that PA(X)
= (x - n,)ml(x- J-2)m2..
‘
(x - n
~)~‘
(2.9.1)
with the Ai in 9distinct. The positive integer mi is called the multiplicity of Li. As remarked earlier, the minimal polynomial qA will have the same linear factors but with possibly lower multiplicities qA(X) = (x - n’)l’(x - n2)12 ’. ( x - &)r., 1 5 ri 5 mi
(i = 1 , 2 , .. .,s).
(2.9.2)
Theorem 2.9.1 Jordan Form. There exists a nonsingular n x n matrix P over 9 such that PAP-’ has the block diagonal form J
=
diag[J,, J z , . . ., Js]
(2.9.3)
in which block Jiassociated with ,Ii is mi x mi. Each Ji is itselfa block diagonal matrix whose subblocks are of the form 1 J-i
0
1
. .. .
(2.9.4)
with all diagonal positions occupied by li,ones along the superdiagonal, and all other elements zero. ‘The uppermost subblock of Ji of type (2.9.4) has size ri x ri where ri is the multiplicity of Ai in the minimal polynomial qA of A. IThe sizes o f t h e subblocks (2.9.4) of Ji form a nonincreasing sequence down the diagonal of Ji. IThe number of subblocks of Ji equals the dimension of the eigenspace W n ,associated with ,Ii.
44
2. Matrix Algebra-The Natural language of Linear Systems
Remark 2.9.1 If the multiplicity of A.i in qA is one, then dim "I(/' Ai = m., and if this is true for each i = I, 2, ... , S, then PAP - 1 is diagonal. This then gives a necessary and sufficient condition for the diagonalizability of A by a similarity transformation over :F. A sufficient condition for diagonalizability is that PA have no repeated roots. . Remark 2.9.2 The characteristic polynomial PA always splits over :F = 'IJ. (This is a consequence of the fundamental theorem of algebra.) Sometimes P can be taken over fJl or some other subfield of ce. For example, if the elements of A are all real and PA has only real roots, then P can be taken to have real elements. Other examples of subfields of'IJ are [J,ta, the rational numbers, the subfield obtained by adjoining to [J,ta, etc.
±.J2
Remark 2.9.3 The Jordan form of a matrix A as defined by Theorem 2.9.1 is unique up to the order of the numbering of the distinct eigenvalues AI' A2' ... , As of A. Any such ordering can be selected. Generally, the characteristic and minimal polynomials of a matrix do not carry enough information about the matrix to determine its Jordan form. In those cases where they do or where J is somehow known, P can be determined by solving the system of linear algebraic equations JP - PA = 0
(2.9.5)
for a nonsingular P. Generally, such a P is not unique. Example 2.9.1 Suppose that A is a real matrix with pix) = (x - 2)2(X - 1) and qA(X) = (x - 2)2(X - 1). In this case the Jordan form of A can be written as
J =
Moreover, dim
"1(/'2 =
dim
"1(/'1 =
2 1I 0] . [~--=-l~ o 0 I 1
(2.9.6)
1.
Example 2.9.2 Consider a matrix A whose characteristic and minimal polynomials are PA(X) = (x - i)3(X + i?(x + 2)2 and qA(X) = (x - i)2(X + i)2(X + 2), respectively. Here again, Theorem 2.9.1 determines
4.5
2.9 The lordan Form of a Matrix
J , which would be
(2.9.7)
EXERCISES
1. Consider the matrix
A=
I
1 -2 0 0 -1
0 0 3 -1 0 2 0 1 1-1
1 1 0 2 0
0 1 1 0 0
over the rational field. Compute (a) the characteristic polynomial pa, (b) the minimal polynomial qa , (c) the Jordan form J , (d) dim WA,. 2. Suppose that all that is known of a matrix A over the field W is its characteristic polynomial and minimal polynomial
respectively. What can be said concerning the Jordan form of A? What can be said about the dimensions of the eigenspaces? If A itself were given, how could it be used in solving the problem? 3. Consider the matrix
'1
A = [ ' k2 k l
46
2. Matrix Algebra-The Natural language of linear Systems
in which k 2 = - (a 2 + ( 2 ), k1 = -20(, with a and ca real and t» #- O. (a) Determine J, the Jordan form of A. (b) Determine all nonsingular 2 x 2 matrices P for which PAP- 1 = J. 4. Suppose that A is a matrix with pix) = (x - Ar, qA(X) = (x - A)3, and dim 11/;. = 3. Is this enough information to determine the Jordan form of A ? 5. If A and Bare n x n matrices over a field /#', then they are called similar if A = PBP- 1 for some n x n nonsingular matrix P over /#'. Prove that if A and B are similar, then (a) det(A) = det(B), (b)
PA = PB'
(c) tr(A) = tr(B), where the trace of a matrix A is defined by tr(A) = I:?= 1 Au. 6. Prove that if A is an n x n matrix over ~, then (a) tr(A) = I:i = 1 m;Ai' (b) det(A) = ni= 1 ).i', where Al' ... , As are the distinct eigenvalues of A and m, is the multiplicity of Ai in the characteristic polynomial PA • 7. Prove that each n x n matrix A over ~ is similar to its transpose AT.
Chapter
3
Linear Constant-Coefficient Equations: Computation
In this chapter, we present several versions of a step-by-step procedure (the ABC algorithm) for computing the closed-form solution to the general,
linear, first-order differential equation with constant-coefficient matrix. In solving the homogeneous equation, all forms of the computation require determination of the eigenvalues of the coefficient matrix (more precisely, the roots of the characteristic polynomial with their multiplicities). One version involves the solution of a finite system of recurrent linear algebraic vector equations. Another is based on solution of a single matrix equation with nonsingular coefficient. Both require a certain amount of matrix-vector multiplication and addition. All operations are adaptable to standard computer routines using either real or complex arithmetic. The solution to the nonhomogeneous problem is accomplished by an additional integration. Although heuristic derivations of the methods are given, their rigorous justification is deferred to Chapter 4.
3.1
The Initial-Value Problem
The procedure to be discussed deals with the initial-value problem in $'",
x = Ax + f(t), x(O) = x",
(3.1.1) (3.1.2)
in which A is an n x n matrix over $' and f: fYt -- $'" is assumed continuous with Xo in $'" prescribed. In a step-by-step manner it constructs a closed-form 47
48
3.
Linear Constant-Coefficient Equations: Computation
solution to (3.1.1) and (3.1.2) in terms of elementary functions and an integral involving f. The procedure includes the option of avoiding complex arithmetic when F = fJl. The following discussion treats the real case first and requires only real arithmetic. The problem in which A is complex or A is real but the facility for doing complex arithmetic is available is covered by Remark 3.2.2.
3.2
The
ABC Algorithm
Step 1. Compute the eigenvalues {/l'l' A2' ... , An} of A (some possibly repeated and/or complex) regarded as a matrix over re. This is equivalent to finding all the real and complex solutions to the equation (3.2.1)
in which PA is the characteristic polynomial of A. This is done analytically or numerically. More often than not, the eigenvalues would be found by using a computer. A number of programmed algorithms are available, several of which work with A directly rather than with (3.2.1). A numerical analysis or text should be consulted about which is most appropriate and about difficulties that can occur (see [5]). When A is real, complex eigenvalues will occur in conjugate pairs. Step 2.
Apply the real correspondence rule
(3.2.2)
which assigns to the collection of eigenvalues a collection of n distinct realvalued functions arranged as an n-vector function according to the scheme 2
m-l
' ).t t Al t Al t Al rea 1I\. - - e , 1! e '2! e , ... , (m _ I)! e , complex A =
Ct.
± uo -- e" cos tot, e" sin cot; I t e~ Ism ' cot . -t e~ cos cot I! ' 1! '
tm - l tm - l ~I t ~/' t (m _ I)! e cos w '(m _ I)! e sm w .
(3.2.3)
3.2
49
The ABC Algorithm
In (3.2.3) m is the multiplicity of the eigenvalue A. The algorithm adopts the convention that in writing down y(t), (3.2.3) is applied to each distinct real eigenvalue and then to each distinct complex conjugate pair. Otherwise the order is immaterial.
Step 3.
Determine the real n x n matrix B such that
By(t)
= y(t).
(3.2.4)
This can be done by inspection, due to the fact that the derivative of each Yi(t) assigned by the correspondence rule is a linear combination of the coordinate functions composing y(t).
Step 4. With A given and B computed in step 3, solve the linear homogeneous matrix equation (3.2.5) AC - CB = 0 to obtain its general n x n matrix solution C. Equation (3.2.5) can be regarded as a system of n2linear algebraic equations in n2 unknowns, and the standard technique of solution discussed in Chapter 2 applies. However, B is a sparse matrix (i.e., it has many zero elements) with special structure. More practical techniques for solving (3.2.5) which utilize the special form of B are presented later in this chapter. Whatever approach is followed, the general solution C to (3.2.5) will have elements that are linear combinations of n arbitrary constants C 1, C2' ••• , Crt which can be arranged as the n-vector (3.2.6)
Hence, by inspection it is possible to read off a unique n x n matrix function Y(t) such that Y(t)C = Cy(t). (3.2.7) (The matrix function Y(t) is an example of what will later be called a fundamental matrix function for (3.1.1).)
Step 5. Using the matrix function Y(t) obtained in step 4, evaluate Y(O) and invert to obtain the matrix function
(t) = Y (t)Y (0)- 1. The solution to the homogeneous form of (3.1.1), (3.1.2), oX = Ax,
(3.2.8) (3.2.9)
is now x(t) = (t)xo.
50
3.
Linear Constant-Coefficient Equations: Computation
Step 6. The matrix function (t) likewise provides a solution (later shown to be unique) to the nonhomogeneous problem (3.1.1), (3.1.2),
x(t) = (t)XO
+ f~
(t -:- a)f(a) da,
(3.2.10)
called the oariation-of-parameters formula for the solution. Example 3.2.1 The six steps of the ABC algorithm are demonstrated by applying the procedure to solve the simple example problem
:t[;:] = [-~ [;:](0) [;~J
~J[;J
+
[e~J
(3.2.11) (3.2.12)
=
In this example, (3.2.13) Step 1.
The characteristic polynomial is computed as
PA(X)=IXI2-AI=I~
-~I=X2+1,
whose roots are the eigenvalues {A'l' ),2} Step 2.
(3.2.14)
= {± i}.
Application of the real correspondence rule provides
yet) = [c~ssm rtJ.
(3.2.15)
Step 3. The equation for B is
B[cosm. stJt = [-sintJ, cos t
(3.2.16)
whose solution is read off to be B
=
[°1 -olJ.
(3.2.17)
Step 4. With A given in (3.2.13) and B as found in (3.2.17), Eq. (3.2.5), written out in terms of the elements of C, appears in the form
51
3.2 The ABC Algorithm
The resultant scalar equations reduce to the pair (3.2.19) It follows directly that the general solution is
(3.2.20) involving two arbitrary constants now computed, Eq. (3.2.7) reads Y(t)[cIJ Cl
=
Cl
and
[C I Cl
Cl
as anticipated. With y(t) and C
ClJ[C~S
-CI
(3.2.21)
tJ'
sm t
from which is read off Y(t) = [ Step 5. Setting t
c~s t sin tJ. -sm t cos t
(3.2.22)
= 0 and inverting give Y(O)-l =
[~ ~rl
=
[~ ~J
(3.2.23)
The matrices of (3.2.22) and (3.2.23) produce
sintJ. cos t
(3.2.24)
Step 6. Substitution of the computed
[=~~:G
=
[-:~: +
:~: J[=n
ft [ cos(t Jo -sin(t -
0) u)
U)J[e:"0 Jda,
sin(t cos(t - u)
(3.2.25)
which integrates to give the sought closed-form solution XI(t) = (x~ - i) cos t + (x~ + o I 0 Xl(t) = (Xl + ~) cos t - (Xl -
t) sin t + ie- t, 1
~)
1
sin t - "Ie-t.
(3.2.26)
Remark 3.2.1 The rather simple idea leading to the equations of the ABC algorithm is easily explained. Consider first the homogeneous problem (3.2.8), (3.2.9). The basic strategy is to attempt a solution of the form x(t) = Cy(t) with the matrix C to be determined. Substitution of this form into (3.2.8) imposes the condition that Cy(t) = ACy(t), but because of (3.2.4), the equation reduces to (CB - AC)y(t) = O. The linear independence of the
52
3.
Linear Constant-Coefficient Equations: Computation
functions composing y(t) allows y to be canceled, and this accounts for the appearance of the resultant matrix equation (3.2.5) for the unknown C. With the general solution to (3.2.8) now forced into the form x(t) = Cy(t), Eq. (3.2.7) (which in the present heuristic derivation is taken on faith) further transforms the representation into the form x(i) = Y(t)c. Since c is arbitrary, it can be selected so as to realize the initial condition x(O) = x", and the result is the equation x(t) = cI>(t)XO, in which cI>(t) = Y(t) y- 1 (0). It turns out that the existence of the inverse that appears follows from the linear independence of.the elements of y(t). Once the solution x(t) = cI>(t)XO to (3.2.8), (3.2.9) has been obtained, the solution (3.2.10) to the nonhomogeneous problem (3.1.1), (3.1.2) can be discovered using the standard uariation-of-parameters technique; i.e., attempt a solution of the form x(t) = cI>(t)u(t) with the vector function u(t) to be determined. Since (t) = AcI>(t), substitution of the proposed form into (3.1.1) leads to the requirement on u(t), cI>(t)u(t) = f(t), which can be written as u(t) = cI>-l(t)f(t) and then integrated, providing the candidate
u(t)
= XO +
L
cI>-l(u)f(u) da.
(3.2.27)
But it turns out that cI>(t)cI>-l(U) = cI>(t - e), and (3.2.27) then leads to (3.2.10), which can be verified to be the sought solution.
Remark 3.2.2 If A is a real n x n matrix, then the procedure described in steps 1-6 involves doing only real arithmetic on the real matrices A, B, and C. Some computers can do complex arithmetic, although generally it is rather awkward. An alternative procedure is available to those willing to do complex arithmetic. It replaces (3.2.3) by the complex correspondence rule, 1
2
AttAttAt
('1-1
At
JI.-e 'TIe '2!e ""'(m_l)!e ,
(3.2.28)
applied to each distinct eigenvalue whether it be real or complex. The remaining steps are the same, although they require working with complex numbers, since the resulting complex exponential functions will produce B and C with complex elements. If A is real, then of course the matrix cI>(t) will be real. If A itself is complex, then (3.2.28) can likewise be applied and the ensuing complex arithmetic performed.
Remark 3.2.3 Undetermined Coefficients. It has been stated that once the homogeneous equation has been solved, the solution to the nonhomogeneous equation is attainable by the additional integration appearing in the variation-of-parameters formula (3.2.10). In the special case for which f(t)
S3
3.2 The ABC Algorithm
is itself a solution to a constant-coefficient homogeneous linear system, the integration can be avoided by embedding the problem in a higher-dimensional homogeneous problem; i.e., suppose that in the equation
x=
Ax
+
(3.2.29)
f(t)
the coordinate functions /;(t) are linear combinations of functions of the type arising from one of the correspondence rules. Then f(t) = Fy f(t),
(3.2.30)
= Bfy/t)
(3.2.31)
YJ(t)
for appropriate matrices F and Br- Hence the problem of solving the initialvalue problem (3.1.1), (3.1.2) for f(t) of the special type under discussion is reduced to solving the homogeneous problem
z=Az, z(O)
(3.2.32)
=z",
(3.2.33)
where
z=[~J A=[~
(3.2.34)
~J
(3.2.35)
EXERCISES
1.
Compute the matrix function w(t) for the matrix A =
[_~ -~J
by applying the complex correspondence rule (3.2.28) to get y(t) and doing the complex arithmetic required to solve (3.2.5). 2. Solve the initial-value problem
x = 2x + 3t,
X(O)
=4
without integrating by embedding it as a homogeneous problem in the method of Remark 3.2.3. 3. What is the dimension n of fJl" in which the system Xl = -Xl + 2X2 + et sin 2t + t",
x2 = Xl + X2 + te 2tcos2t could be embedded as a homogeneous system?
~3
by
54
3.
Linear Constant-Coefficient Equations: Computation
4. If variation of parameters were selected as the technique for finding the closed-form solution of the problem in the previous exercise, approximately how maJ?y integrations by parts would be involved? 3.3 A VectorMethodfor Computing the Matrix C
In the ABC algorithm, A is given and B is just an intermediate matrix used to obtain the matrix C as the general solution to the equation AC - CB
= O.
(3.3.1)
For even moderately large n, it is not realistic to solve (3.3.1) by regarding it as a system of n2 scalar equations in n2 scalar unknowns. By exploiting the special block diagonal structure of B, this section derives a vector method for computing C directly from A. The resultant equations produce a vast decrease in the computational complexity and allow passage directly from step 1 to Eq. (3.2.7) of step 4. The discussion again first assumes A to be real and the calculations are set up for doing real arithmetic. Treatment of the problem with A complex or by employment of complex arithmetic is covered by Remark 3.3.2. By writing out the matrix variable C in (3.3.1) in terms of column variables and computing the form of B, it is easily shown that each distinct real eigenvalue A. of multiplicity m contributes m successive columns Um , Um-I"'" UI of C obtained by successively solving (k
= 1,2,
00
.,
m)
(3.3.2)
with Uo = O. Similarly, each distinct complex conjugate pair of eigenvalues A. = oc ± iw of multiplicity m contributes m successive ordered pairs of columns (vm , um ), (Vm-l, um-d,· .. , (VI' ud of C obtained by successively solving [(ocI - A)2 + w 2I ]uk = -2(ocI - A)Uk-I - Uk- 2 (k = 1,2,oo.,m) (3.3.3) with
Uo
= U-l :;:: 0 and then computing
Vk
= w-1[(ocI - A)uk + Uk-l]
(k = 1,2,oo.,m).
(3.3.4)
Remark 3.3.1 Note that the first equation (k = 1) in (3.3.2) is that for the real eigenvectors U 1 associated with the real eigenvalue A. of A, and the first equations in (3.3.3) and (3.3.4) are those for the complex eigenvectors U 1 + iVl associated with the complex eigenvalue oc + ico. In solving (3.3.2) and (3.3.3), the equations at each stage k must be solved to obtain the general solution Uk' It would be erroneous to conclude that for A. a real eigenvalue with associated eigenspace 11';. of dimension d there will
3.3 A Vector Method for Computing the Matrix C
55
be d arbitrary constants in the part u1 of the solution to (3.3.2). Certainly when (3.3.2) is solved with k = 1, the resultant solution will contain d tentative arbitrary constants, but as the computation proceeds, some of the constants will be forced to be zero to maintain the consistency of the equations at successive stages. In other words, the solution u1 to (3.3.2) is a subspace of the eigenspace WL,and u1 results from successive extraction of subspaces, starting with W A as , the calculation proceeds to stage m. This phenomenon is illustrated by Example 3.3.1.
Remark 3.3.2 If complex arithmetic is to be employed by choice, as discussed in Remark 3.2.2, or because A is complex, then the computation of C associated with (3.3.1) can be done by using (3.3.2) alone. The literature sometimes calls the vectors that are generated generalized eigenuectors.
[i 3.
Example 3.3.1 Let A be the matrix A=
(3.3.5)
The eigenvalues obviously are 1, 1, 1. The general solution (eigenspace Wl) of (3.3.2) with k = 1 obtained by the row reduction (3.3.6)
is read off as (3.3.7)
with pl, Bz arbitrary real numbers. The solution of (3.3.2) with k = 2 is accomplished by the reduction (3.3.8)
with Bz forced to be zero to maintain consistency. Thus u1 is cut down to the one-dimensional subspace of Wl , (3.3.9)
56
3.
Linear Constant-Coefficient Equations: Computation
and the resultant solution, read off (3.3.Q is u2
(3.3.10)
=
/I1, /I3, p4 arbitrary. The equation (3.3.2) with k
=3
is solved as (3.3.11)
with consistency forcing /Il = /I4 = 0 and the resultant solution being (3.3.12)
/I3, &, /I6 arbitrary. After the /Is have been renamed as cs, the general solution to (3.3.1), for A given by (3.3.5), is thereby computed to be
c=
[:6' 3 c2
0
0 .
(3.3.13)
The remaining steps in the ABC algorithm can now be computed:
=
[
d td 0 0 d 0 0 e'
c1
= Y(t)c,
@(t)= y(t)Y-'(O) =
Example 3.3.2
(3.3.14)
c3
(3.3.15)
Consider 2 = Ax in which (3.3.16)
3.3
57
A Vector Method for Computing the Matrix C
The characteristic polynomial computes as pA(x) = (x - 1)’(x2 + 2x + 2), from which it is apparent that the eigenvalues are {Al,,l2,A3,,l4} = {1,1, -1 lL- i}. The row reduction (11- All4) + (El M ) with 1= 1 produces the matrices
.]
1 0 0-2
0 0 1 - 1 ’ 0 0 0 0
2-i M = [ : -1
-1
:].
(3.3.17)
2 0
(3.3.18)
1 1
The general solution u1 to (3.3.2) for k = 1 is read off E as (3.3.19)
and substitution of (3.3.19) into the row-reduced form of (3.3.2) with k = 2, (3.3.20)
provides (3.3.21)
Since the pair of eigenvalues -1 & i has multiplicity one, (3.3.3) is solved directly by the row reduction
58
3.
linear Constant-Coefficient Equations: Computation
with the solution being (3.3.23)
Application of (3.3.4) results in
V1
=
(J)
-1
+
tC4 C3J 1 --Z C4 + C 3
«(XI - A)u 1 = [
2
-
C3 -
4C3
C4
(3.3.24)
.
+ C4
Since the cs are arbitrary constants, C4 can be replaced by 2c4 throughout for the convenience of eliminating fractions. With all columns computed, the solution C is 2C
2 -
5C1
-C 1
C= [
C2 -
C1
C2
1
2
C3
+
0C
C3 -
C1
-2C3 -
C1
4C3
C4 C4
-C3
J
+ 2C4
3C3
2C4
2C3
+ 2C4
2C4
.
(3.3.25)
The ABC algorithm can now be continued by jumping to Eq. (3.2.7) of step 4 with C as computed in (3.3.25) and
y(t) =
By inspection,
Y(t)
=
~:)e'
[
(2t (t _ 1)e' tet
2;' e'
e'
e' ] te' [ e cos t «: sin t -r
t
(3.3.26)
•
.c:«: e-'(2:~~_~o~tcost)]. 2e-'(sin t - cos t) 4e-' cos t
(3.3.27)
cos t 2e-t(cos t + sin t) - 2e
!
The precise manner in which the general solution x(t) = Y(t)c depends on t is vividly displayed by (3.3.27). A routine calculation yields
Y-
1(O)
=
[=; =~ ; 50 7
"5O-so 23
12
30 -"50 -"50
50 22
so 1.QJ
8 50 2 -50
.
(3.3.28)
59
3.4 Explicit Formulas for C(c) and «II(t)
The matrix «1>(t) = Y(t) y- 1(O), needed for the solution of the initial-value problem and for the nonhomogeneous form of the differential equation, can be obtained from (3.3.27) and (3.3.28) by multiplication.
3.4 Explicit Formulas for C(c) and Cl»W)
Let y(t) and B be defined as in Section 3.2. Let x = Ax, x(O) = c, and x(t) = C(c)y(t). Denote y(O) by yO. Since y = By, it follows formally from these equations that
.
(3.4.1) for all n-vectors c. The claim that the inverse in (3.4.1) exists and the presumption that (3.4.1) provides the general solution to (3.2.5) as well as the consequent formula (3.4.2) in which (3.4.3) (i = 1,2, ... , n), In = (el' e2"'" en), will be rigorously established in Chapter
4. In particular, it follows from (3.4.1) that the solution x(t) to (3.2.8) and (3.2.9) could be obtained as
(3.4.4)
x(t) = C(XO)y(t),
where the matrix
C(XO)
is computed by solving the system of equations
(yO, By", . . ., Bn-lyO)TCT(xO)
=
(XO, Ax o, ... , A n-
1xO)T.
(3.4.5)
Since the coefficient matrix in (3.4.5) is nonsingular, that equation could be solved most efficiently by doing a triangular factorization of the coefficient matrix and applying it to solve (3.4.5) as a system of vector equations. It should be noted that on the order of 2n 3 multiplications would be required to compute the two 'data matrices in (3.4.5), about ~n3 would be required to do the factorization, and about n 3 more would be required to do all the back substitutions. Thus this version of the ABC algorithm has an operation count of the order of n3 , which is better by at least one order of magnitude than are most other methods! It is important to note that if all that were required were the calculation of x(t) for a few values of XO with n large, then the calculation (based on (3.4.5)) just described would be much more efficient than computing «1>(t). This is apparent from (3.4.3), which would require on the order of n4 operations.
60
3.
Linear Constant-Coefficient Equations: Computation
EXERCISES
1.
Apply (3.4.4) and (3.4.5) to solve the initial-value problem Xl = X2 + X3' X2 X3
= Xl + X3' = Xl + X2
with Xl(O) = 1, X2(0) = -1, xiO) = 2. 2. Apply (3.4.4) and (3.4.5) to solve the initial-value problem
3.5 Additional Comments about Computing and Changing Variables
Earlier in this chapter it was shown that once the homogeneous problem (f = 0) has been solved, that is, the fundamental matrix (t) is computed, then the solution to the nonhomogeneous problem is obtained by an integration. Thus attention is returned to the homogeneous problem X
=
Ax,
(3.5.1) (3.5.2)
The change of variable z = r :». involving any nonsingular matrix P, transforms the equations into i =
Az,
z(O) = z",
(3.5.3) (3.5.4)
in which A = r:' AP and Zo = Px". Thus the change of variables induces a similarity transformation on the coefficient matrix. The rigorous analysis of (3.5.1) and (3.5.2) presented in the next chapter will utilize such a transformation. The approach will be to select P in a manner rendering the behavior of (3.5.3) transparent. This will be adequate since most dynamical behavior of interest is invariant under such a trans-
3.5 Additional Comments about Computing and Changing Variables
61
formation. Frequently, there is considerable freedom anyway in the choice of variables selected to model a physical system. One instance of this occurs when an engineer or scientist decides which system of physical units to use. To give some indication of the role of coordinate transformations in the subject, consider the special class of matrices A in (3.5.1) that have a full linearly independent system of eigenvectors WI' W2,"" Wn, corresponding to respective eigenvalues AI, A2, ... , An (not necessarily distinct or real). Recall that these are precisely those matrices for which each distinct eigenvalue has multiplicity one in the minimal polynomial qA' By the independence, it follows that the matrix (WI, W2, ..., Wn) is nonsingular. Thus suppose that the change of variable made in (3.5.1) and (3.5.2) uses P = (WI' W 2, .•• , wn ) :
A = P-1AP = P-I(AwI,Aw2,oo.,Awn) = P-1(AIWI,A2W2, ... ,AnWn)
= P- 1pD(AI,A2,oo.,An) (3.5.5)
in which D(AI' A2,"" An) is the diagonal matrix with AI' A2"'" An along the main diagonal. From the calculation (3.5.5), it is now clear that a solution to (3.5.3) and (3.5.4) is (3.5.6) and hence a solution to (3.5.1) and (3.5.2) is computed to be x(t) = (WI> W2,"" wn)D(e"'I, e"2 1, ... , e"nl)zO -- (e"t1w 1e"2l ' . w2'···, e"n1wn )ZO
(3.5.7) Moreover, it is apparent that the fundamental matrix (t) for (3.5.1), as discussed in Section 3.2, could be taken to be (3.5.8) Observe that if A is real, then although some of the AS and ws may be complex, (t) will remain real. Computation of (t), using (3.5.8) for those matrices to which it applies, is sometimes called the method of eigenvectors.
There is a notable case in which (t) can be computed with minimal effort. Consider the nth-order scalar constant coefficient equation
Remark 3.5.1
(3.5.9)
62
Linear Constant-Coefficient Equations: Computation
3.
The coefficient matrix of the first-order form of (3.5.9) is . 0
0
0 . ... 1 0 ...
1 0
0
0 0
(3.5.10)
A=
0 -a,
. ... -anFl . . . . . 0
1
0 -a2
0 1 -a,
For such a matrix, it will be shown in Chapter 4 that @ ( t ) = Y(t)Y-'(O),where Y ( t )is the matrix function whose first-row elements are those assigned to A by the appropriate correspondence rule (written in any order) and whose successive rows are generated by differentiation of the row immediately above it.
Remark 3.5.2 The solution to the initial-value problem in which the value xo of x is prescribed at t o , i= A x X(t0) =
+ f(t),
(3.5.11) (3.5.12)
xo,
is given by the modification of (3.2.10)
+
x ( t ) = @(t- t O ) x o
@(t - o)f(o)do.
(3.5.13)
EXERCISES
1. Verify that (3.5.13) satisfies (3.5.11) and (3.5.12). 2. Show that Eqs. (3.5.11) and (3.5.12) in complex n-space V" with A a matrix over 3 ' can be converted into a problem in g2"with a real 2n x 2n coefficient matrix. This points out that equations in complex n-space could be treated by the ABC approach, using real arithmetic. 3. Solve the initial-value problem (3.1.1), (3.1.2) in which
using the method of Section 3.3. 4. Solve the problem of Exercise 3, using the method of Section 3.4.
3.5 Additional Comments about Computing and Changing Variables
5.
63
Solve the initial-value problem (3.5.1), (3.5.2) in which 1
~J,
o
-8 -5
(a) by the ABC algorithm, employing Eq. (3.3.2), (b) by the method employing Eqs. (3.4.4) and (3.4.5), (c) by the method of Remark 3.5.1. Does the method of eigenvectors, employing (3.5.7), also provide the solution? 6. Compute the closed-form solution to (3.5.1), (3.5.2) in which
A
~
U_~ !]
(a) by the ABC algorithm employing Eqs. (3.3.2), (3.3.3), and (3.3.4), (b) by the method of Section 3.4, (c) by the method of eigenvectors (3.5.7). 7. Find the solution to x = Ax, x(1) = XO with A as in Exercise 6. 8. Solve the initial-value problem eb = AlP, lP(O) = 14 in which
A
=
0 1 1-1] [-~ -~ -~ ~. -4 -4 -8
9.
Solve the matrix differential equation problem eb = AlP + lPA with
A=[_~ and
0
~J
Chapter
4
Linear Constant-Coefficient Equations: Theory
The heuristic reasoning which led to the equations purportedly computing the closed-form solution as discussed in Chapter 3 will now be given a rigorous foundation. As is the case with many existence theorems, a solution to the initial-value problem will be shown to exist by means of a limit process, namely, through the exponential function of a matrix. This solution is proved to be unique. The requisite matrix solution C to the equation AC - CB = 0 is constructed from the solution to the initial-value problem, and a representation of the solution of the type computed by the ABC algorithm is thereby shown to exist. . The Jordan form of a matrix applied to the exponential representation of the solution illuminates the choice of elementary functions assigned by the correspondence rules and at the same time reveals precisely those characteristics of the coefficient matrix which govern the qualitative aspects of solution behavior of primary interest in applications. 4.1 Preliminary Remarks
A rigorous and thorough treatment of Eqs. (3.1.1) and (3.1.2) of Chapter 3 will now be developed. It is the homogeneous equation that requires the most attention since the nonhomogeneous problem involves only an additional integration. 64
65
4.2 Some Elementary Matrix Analysis
Assuming that the construction of the closed-form solution presented there is correct, one is struck by the fact that such a limited class of elementary functions is involved. A clue as to why this is the case is already present in the one-dimensional problem in which in beginning calculus the solution to the scalar equation x = ax is found to be the exponential function ce", usually defined as a power series. Hence, the notions of convergence of series, their differentiability, etc., arise. It will soon become apparent that one ofthe main reasons for working with differential equations in first-order form (at least for linear equations) is that the higher-dimensional problem can be treated along somewhat similar lines. (The analogy actually extends well beyond finite-dimensional problems.) The first step will be to extend some familiar notions of real analysis to matrices.
4.2 Some Elementary Matrix Analysis In this chapter, :F denotes any subfield of the complex numbers and, as before, :F" is the collection of n-vectors over :F with the standard algebraic operations. When there is a need to be more specific, :F is replaced by ~ when considering the complex numbers and by 9t when dealing with real numbers. Recall the basic properties of the absolute-value function on :F:
o s Ix I
with equality if and only if
lexl = lei I~I, [x - yl ~ Ixl + Iyl
x = 0,
(4.2.1) (4.2.2) (4.2.3)
for all e, x, and y in :F. A real-valued function with these properties can be extended to higher dimensions. Definition
For x in :F" with coordinates
Ixl =
max 1 ~i~n
Xi'
lXii,
define (4.2.4)
called the norm of x. Remark 4.2.1 Note that when n = 1, the norm of x and its absolute value agree. Moreover, the norm function defined by (4.2.4) continues to satisfy (4.2.1)--(4.2.3), where in (4.2.2) e denotes any element in :F while x is in :F". (The student should verify these claims.) To a considerable extent, the notion can be extended one step further to m x n matrices.
66
4. linear Constant-Coefficient Equations: Theory
Definition
For A an m x n matrix over
IAI
= max
1"1,, 1
§',
define
IAxl,
(4.2.5)
in which x denotes a variable in §'n. IAI as defined by (4.2.5) is called the operator norm of A relative to the norm defined on §'n.
Proposition 4.1
The operator norm of an m x n matrix A over §' is a realvalued function and satisfies the conditions
o ~ IAI if and
with equality holding
(4.2.6)
if A = O.
only
leAl
=
IcllAI
(4.2.7)
for every c in §'.
IA - BI s IAI + IBI
(4.2.8)
for all m x n matrices A and B over §'.
IAxl
~
IA/lxl
(4.2.9)
for all x in §'n.
IABI ~ IAIIBI
(4.2.10)
for all matrices A, B over §' of compatible size.
JAI = IIn l = Proof.
For all x in
§'n
n
L IAul·
max l~i~mj=l
l
(4.2.11)
(n=1,2, ...).
satisfying
Ixl
~ 1,
n
n
n
j;l
j;l
j;l
I L Auxjl ~ L IAul ~ L IAi.jl (i
=
(4.2.12)
(4.2.13)
1,2, ... ,m), where i* is chosen so that
max l~i~m
Defining y* in
§'n
n
n
j=l
j=l
L IAul = L IAiJ·
(4.2.14)
by
I
yj =
sgn(A i• j
(4.2.15)
)
(j = 1,2, ... , n), note that y*/ ~ 1, and furthermore n
n
n
L1 IA jI = L1 A,.jyj = Ii>L1 Ai.jYjl·
j=
i•
j;
(4.2.16)
67
4.2 Some Elementary Matrix Analysis
Thus it follows from (4.2.13) and (4.2.16) that n
max l~i~m
IL
n
j=l
Ai';xjl::; max l~i~m
IL
(4.2.17)
Aijyjl
j=l
for all xin fFn with Ixl ::; 1. From (4.2.17) it is apparent that the left-hand side achieves a maximum over x and moreover from (4.2.13) on one hand and (4.2.16) on the other it follows from (4.2.17) that ,
n
n
L
max max I Aijxj! Ixl,;;l l,;;i,;;m j=l
=
L IAi.jl, j=l
(4.2.18)
which, according to (4.2.2), (4.2.5), and (4.2.14), is the required inequality n
L IAijl·
IAI = max
(4.2.19)
j=l
l~i~m
Properties (4.2.6) and (4.2.7) are obvious consequences of (4.2.19). Since (4.2.20) for numbers n
L IAij j=l
n
Bijl ::;
n
L IAijl + j=l L IBijl, j=l
(4.2.21)
and thus by formula (4.2.19), etc., IA -
BI =
n
max l~i~m
s
L
j=l
IAij - Bij!::; max l~i~m
n
max l~i~mj=l
L IAijl +
n
n
j=l
j=l
[L IAijl + L IBijl]
n
max
L
IBijl = !AI
l~i~mj=l
+ IB!,
(4.2.22)
proving (4.2.8). To prove (4.2.9), apply (4.2.2) and (4.2.5) to get
IAx'l =
IxllA
-,xII::; Ixl lyl,;;l max IAyl x
=
IxllAj,
(4.2.23)
which is valid for x#-O in fFn. (Note that the maximum in (4.2.23) was shown to exist in the previous part of the proof.) Thus (4.2.9) follows from (4.2.23) for x#-O and also holds (with equality) for x = O. This finishes the proof of (4.2.9). For
Ixl s
1, by applying (4.2.9) twice,
I(AB)x/
= /A(Bx)
I::; IAllBxl s IAllBllxl ::; IAIIBI,
(4.2.24)
68
4.
which maximized over
Linear Constant-Coefficient Equations: Theory
Ixl :5: 1 yields IABI :5: IAIIBI,
(4.2.25)
concluding the proof of (4.2.10). Equality (4.2.11) was already proved and (4.2.12) is a direct consequence of (4.2.11). This' concludes the proof of Proposition 4.1. The main reason for introducing the absolute-value function on /F is for convenience in talking about convergence and continuity. It is assumed that the reader is familiar with these ideas in the case of real or complex numbers; i.e., in dimension one. Norms are introduced in order to expand these notions to higher dimensions, where it soon becomes clear that formally there is little difference from the one-dimensional case.
Definition For A k (k = 1; 2, ...), a sequence of m x n matrices over /F, the sequence is said to converge to an m x n matrix Aoo as k - 00 if (4.2.26) as k - 00. The matrix Aoo is denoted by lim k _ oo Ak> and one writes A k as k- 00.
Aoo
Remark 4.2.2 Recall that a sequence of numbers ak is called Cauchy if for each real e > 0 there exists a real K. such that k ~ K. and p ~ K. implies that lak - api < s, The Cauchy criterion (theorem) says that a sequence of numbers is convergent if and only if it is Cauchy. The result is important because it can be used to prove a sequence convergent without the difficult task of finding the value of the limit. It is easily seen from formula (4.2.11) that a matrix sequence A k - Aoo as k - 00 if and only if (Ak)ij - (A oo )ij as k - 00 for each i, j. That is, convergence of a matrix sequence is equivalent to elementwise convergence. Thus it is clear that the Cauchy criterion is also valid for matrix sequences.
Definition the type
If A k is a sequence of m x n matrices arising as a matrix sum of
Ak =
k
L e,
(4.2.27)
i=O
and is convergent, then its limit is denoted by L~
0
B;.
The text follows the standard abuse of notation wherein the same symbol is used to denote the series, whether or not it converges. Series can be used to define matrix functions in much the same way as they are used to define scalar functions. The exponential function plays a particularly dominant role in this chapter.
4.2
69
Some Elementary Matrix Analysis
Definition For A any n x n matrix over of A is defined by the series
§,
the exponential function e A
(4.2.28)
The series is convergent for all A. This can be seen by computing the difference between two terms in the sequence of sums to get
Ai PAil IAli IAli "--"-<" L...., L...·f L... -<" L... - - 0 I k
k
i;O l.
i;O l.
00
i;p+l
'f
-
l.
i;p+l
as
'f
l.
P':"
00,
(4.2.29)
since the series for eX converges for all x. Thus convergence of the series (4.2.28) follows by the Cauchy criterion. The next lemma proves a more general result. Definition
An m x n matrix series over
§,
(4.2.30)
is called absolutely convergent if 00
L I(Bk)ijl k;O
(4.2.31)
is convergent for each i = 1, 2, ... , m; j = 1, 2, ... , n. Lemma 4.2.1 Let P and Q be matrices over § of size p x m and n x q, respectively. If the series (4.2.30) is absolutely convergent, then PSQ is likewise absolutely convergent. Proof. i andj.
Let
IX = max!Pij!, fJ =
maxlQijl with the maxima taken over
Since the series (4:2.30) is assumed to be absolutely convergent, for each real e > 0 there exists an integer K. such that 00
L I(Bk)ijl k;.
< e
(4.2.33)
for K. ::0:; s (i = 1,2, ... , m; j = 1,2, ... , n). From (4.2.32) and (4.2.33) it follows that 00
00
L I(PBkQ)ij I ::0:; IXfJ L L L J(Bk)'!'21
k=s
S2
SI
k==s
::0:; IXfJ
LL(. = 52
SI
IXfJmn(..
(4.2.34)
70
4.
Linear Constant-Coefficient Equations: Theory
Since e > 0 is arbitrary, the required absolute convergence follows directly from (4.2.34).
Theorem 4.2.2 Suppose that a power series <Xl
fez)
=
L
akzk
(4.2.35)
k=O
over ((j has a radius of convergence p. Let A be an n x n matrix over ((j. A sufficient condition for the absolute convergence of the matrix series f(A) is that either AP = 0 for some positive integer p (in which case A is called nilpotent) or IAI < p for each eigenvalue Aof A. Proof. In case A is nilpotent, the series f(A) is a finite sum and hence is obviously absolutely convergent. In the other case to be considered, p > O. Let A = PJP- 1 where J is the Jordan form of A. If f(J) is absolutely convergent, then by Lemma 4.2.1 it follows that f(A) will be absolutely convergent, since Pf(J)P- 1 = f(PJP- 1 ) = f(A). Thus without loss of generality, assume that A is in Jordan form. Furthermore, since power series preserve the block diagonal structure, it is sufficient to assume that A is an elementary Jordan matrix
(4.2.36) in which Z is the n x n nilpotent matrix with all zero elements but for a superdiagonal of ones as in (2.9.4). By the binomial theorem, which applies since AIn and Z commute, (4.2.37)
(4.2.38) Since a scalar power series can be differentiated termwise within its circle of convergence, f(S)(z)
= s!
f ak (k)s ~-s
(4.2.39)
k=s
for Izi < p. Consequently the last term and hence the first term in Eqs. (4.2.38) have a limit as K -+ 00 if IAl < P and, moreover, the limiting equation
71
4.2 Some Elementary Matrix Analysis
shows that (4.2.40) The absolute nature of the convergence is a consequence of that of power series and of the position of the ones in ZS. This concludes the proof of Theorem 4.2.2. Since the radius of convergence for the series defining eZ is p = 00, the convergence of the series for eA for every n x n A also followsas a direct corollary to Theorem 4.2.2. Remark 4.2.3
EXERCISES
1. Compute
(a)
[~-~
3 -2
(c)
1
AI, the norm of A, for the following matrices:
-~],
(b)
2
[_~ ~ _~ ~l, 0 0 -2
[1 2 -2 3J,
(d)
nl
dJ
2. Applying the Cauchy criterion, show that for any n x n matrix over with IAI < 1, . (a) Lo A k is convergent, (b) In - (In - A)L~ A k - 0 as K - 00, (c) LO A k = (In - A)-1. Determine whether or not (c) is valid for
A =
[~ ~J
and compute IA I. ·Is your conclusion consistent with Theorem 4.2.2? 3. Consider the power series fez)
=
1
+ 2z + 24 z2 + ... + 2n2zn + ....
(a) Determine the radius of convergence of the power series. (b) Compute f(A) for this power series where the matrix is
A=[~ ~ ~J. 000
C(j
72
4.
linear Constant-Coefficient Equations: Theory
(c) Are your results for (a) and (b) consistent with Theorem 4.2.2? 4. In the following statements, A denotes any n x n matrix over the complex field except where otherwise indicated. Determine which are true. Justify your conclusions. (Hint: Apply Theorem 4.2.2 where appropriate.) (a) leAl::::;; e lAI. (b) (eAr = e'A (r = 0, ± 1, ±2, ...). (c) (d)
(eA)-1 = e- A. A) det(e ¥erlAer2A = e(r1+r2)A, all real r l and r2' eAeB = ~+B where A = ~], B = [~
°
(e) (f) [g gl 5. The Euclidean norm on fJi" is defined by the formula )xl e = ~ all x in fJi". Find the smallest positive numbers PI and P2 such that
for
for all x in fJi". Note that a sequence in fJi" is convergent relative to one norm if and only if it is convergent relative to the other. The operator norm of a real n x n matrix A induced by the Euclidean norm is defined by the formula IAle = maxlAxl e, where the max is over x in fJi" with [xIe ::::;; 1. Can you find a formula explicitly computing IA/ e from the elements of A? Show that there exist positive numbers rl and r2 such that for all real n x n matrices A. Can you compute values of r l and r2?
4.3 Existence, Uniqueness, and Solution Representations Using etA.
If M(t) is anm x n matrix-valued function defined on an interval (ex, /3), then the derivative dM(t)jdt at a point t in (ex, /3) is defined to be the limit dM(t) = lim M(t dt h-O
+ h) h
- M(t)
(4.3.1)
when it exists. Note that differentiation can be performed elementwise and that this is consistent with the elementwise differentiation of n-vector functions appearing in earlier chapters. An integral of a matrix-valued function can likewise be defined as a matrix limit and integration can be done elementwise. Many of the standard formulas of elementary calculus continue to hold for matrix-valued functions such as
4.3 Existence, Uniqueness, and Solution Representations Using
73
etA
They will be applied without further mention since they are all easy exercises for the reader. By replacing A in (4.2.28) by tA, the matrix-valued function of t, (4.3.2) is defined for all t in ( - 00, (0). The next theorem shows that this matrix function can be differentiated by using the same formula that applies in the scalar case. For any n x n matrix A over C(j, the function etA has a derivative satisfying the equation
Theorem 4.3.1
(4.3.3) for all t in (- 00, (0). Proof.
~[e(t+h)A h
For all real t and h =I 0, _ etA] _ Ae tA = !etA[e hA _ I _ hA] = ~etA h h =
~ et~hA)2
~
I (hA)k k! 2
(hA1: - 2
(4.3.4)
Restricting Ihl < 1, it follows that 1 Ih[e
~
IhlletAleIA'_O
(4.3.5)
as h - 0, and the proof is finished. Remark 4.3.1 It follows as a corollary to Theorem 4.3.1 that differentiated any number of times with k
d _etA = AketA
r
drk
e'" can be (4.3.6)
'
and if A is invertible, then
etA dt = A- 1[ebA
-
eaAJ.
(4.3.7)
Note that Theorem 4.2.2 can be applied to obtain an alternative proof of Theorem 4.3.1 as well as (4.3.6) and (4.3.7).
74
4.
Linear Constant-Coefficient Equations: Theory
Corollary 4.3.2 Let A be an n x n matrix over .fF and f: (ct, P) _.fF n be
continuous. For each to in (e, P) and initial-value problem
x=
X
O
Ax
in .fFn there exists a solution to the
+
(4.3.8)
f(t),
x(to) = xo,
(4.3.9)
given by the formula x(t)
=
e(t-,to)AxO
+
rt e(/-lT)A f(a) da,
ct < t <
Jlo
p.
(4.3.10)
Proof. Certainly (4.3.9) is satisfied, and (4.3.8) follows directly from differentiation and application of Theorem 4.3.1. The continuity ensures the existence and differentiability of the integral in (4.3.10).
Corollary 4.3.3
There exists a solution to the initial-value problem
x=
(4.3.11)
Ax,
(4.3.12) given by - 00
< t<
(4.3.13)
00.
Remark 4.3.2 The next task will be to prove that the solutions to the problems in Corollaries 4.3.2 and 4.3.3 are unique. Although it has not yet been proved, upon comparing (4.3.10) with (3.2.10) one might suspect that the
Lemma 4.3.4 Gronwall's Inequality. every continuous solution to x(t) ,;$; ct +
p
f:
x(a) da,
o ;$; t <
T
p~
0,
(4.3.14)
;$; 00,
satisfies the inequality
o ;$; t <
(4.3.15)
T.
Proof. There is nothing to prove if p = 0, so assume that plication of (4.3.14) by e-(Jt gives
~[e-(Jt
I
x(a) daJ
;$;
cte-(Jt,
0
;$;
t < T.
p>
O. Multi-
(4.3.16)
75
4.3 Existence, Uniqueness, and Solution Representations Using e'A
Integration of (4.3.16) shows that
I
e- Pt
~
x(oJ da
~(e-Pt
-
-
(4.3.17)
1),
and applying (4.3.14) to eliminate the integral in (4.3.17) gives x(t) -
p
Q(
< _ ~(1
-
_ elit)
p
,
os t <
T,
(4.3.18)
from which (4.3.15) follows directly to conclude the proof. Theorem 4.3.5
The solutions given in Corollaries 4.3.2 and 4.3.3 are unique.
Proof. It is sufficient to consider Corollary 4.3.2. Let z(t) = x(t) - y(t) where x(t) and y(t) are both solutions to (4.3.8)and (4.3.9). It is easy to check that z(t) satisfies :i
=
(4.3.19)
Az,
z(to) = 0,
Q(
< t < p.
(4.3.20)
By integration of (4.3.19) and (4.3.20) it follows that z(t + to) =
for 0 ~ t < T
= P-
I
Az(u + to) da
(4.3.21)
to, and hence that
/z(t +. to)1
~ IAI S: jz(u
+ to)1 do,
(4.3.22)
Application of Lemma 4.3.4 to (4.3.22) shows that z(t + to) = 0 for o ~ t < p - to, which is to say that z(t) = 0 for to ~ t < p. A similar argument shows that z(t) = 0 for Q( < t ~ to, and the proof of Theorem 4.3.5 is thereby completed.
Corollary 4.3.6 For A and <1>0 any n x n matrices over~, to the initial-value problem
d> =
A,
(t o) = <1>0
Corollary 4.3.7 i.e., for which AB
the unique solution
(4.3.23) (4.3.24)
For A and B any two n x n matrices over <{j which commute; = BA,
(4.3.25)
76
4.
Linear Constant-Coefficient Equations: Theory
EXERCISES
1. Find three proofs of the uniqueness concluded in Corollary 4.3.6. 2. Prove Corollary 4.3.7 by considering the initial-value problem = A, <1>(0) = eB • 3. Prove that the unique solution to the matrix equations = A
+ B,
(4.3.26)
<1>(0) = <1>0 (4.3.27) is (t) = etA°etB. 4. Compute etA for the following matrices, using the series defining the exponential of a matrix: (a)
(c)
A =
A =
[_~
~J
(b) A =
[
00 0I 0]I ,
000
[-~ -~ ~]. o
0-2
5. Compute etA for the following matrices by solving the appropriate initial value problems.
6.
(a)
A =
[_~
~J
(c)
A =
r~ ~ ~l,
L1
0
oJ
(b)
A =
(d)
A =
[-1 1J
-1 -1 '
[~o -1~ 0~J
.
Consider the second-order, initial-value problem in X
{)In,
+ A x = 0, 2
x(O)
=
x(O)
= VO
(4.3.28)
0,
(4.3.29)
where
VO = Ac,
cE
{)In.
(4.3.30)
(a) Define sin A as a power series. (b) For which n x n matrices A does the series converge? (c) Prove that x(t) = [sin(tA)]c is a solution to (4.3.28)-(4.3.30). (d) Prove that the solution is unique (two ways). 7. Compute the elements in the matrix function !(A) = cos(itA), where A =
[=~ -~ ~]
002
(Suggestion:
cos cot = -!<e iwt
+ e- iwt ). )
and
4.4
4.4
77
The Structure of elA
The Structure of e'"
This section takes a deeper look into the problem of determining, to the extent possible, the manner in which the elements in the matrix function etA depend on the matrix A. Consider an n x n matrix A over a subfield 9of W in which the characteristic polynomial pa splits. By Theorem 2.9.1 A
=
(4.4.1)
PJP-'
in which J is the Jordan form of A and P is a nonsingular matrix over 9, From the series definition of the exponential of a matrix it is easy to see that
(4.4.2)
etA = Pe'Jp-'.
In view of (4.4.2) and the fact that J has a block diagonal form J = diag[J,, J 2 , . .., J , ] ,
(4.4.3)
it follows that erJ = diag[eZJ', e l J 2 , .. ., e l J s ] .
(4.4.4)
Each Jk is itself block diagonal with blocks of the type LZ + Z as indicated in (2.9.4). Hence computation of elJk reduces to computing the elements of the matrix
(4.4.5)
e'(kkk'+Z)= etkkelz
Since Z is a nilpotent matrix with all zero elements but with the superdiagonal terms ones, it folIows readily that
I - t - -t2 t3 l ! 2 ! 3! t t2 1 I ! 2!
0 r-1
...
+k
... (4.4.6)
0 0
1
... 0
0
in which r is determined by the size of Z which is r x r. It follows from (4.4.5) and (4.4.6) that written out in terms of its columns, er(kkr+z)= et l Y e l ,t e ,
f - 1
+ e 2 , .. ., (r - l)!el + (r - 2) ! e2 + ... + en), (4.4.7) ~
tr-2
~
78
4.
linear Constant-Coefficient Equations: Theory
in which the e, are the columns of the identity matrix I., Some important observations are summarized in the following remarks. Remark 4.4.1 The elements of etA consist of linear combinations, with coefficients in ff, of functions of the type
tk _etA)
k!
(k=O,l, ... ,rj-l;j= 1,2, ... ,s),
(4.4.8)
in which the Aj are the distinct eigenvalues of A and r j is the multiplicity of Aj in the minimal polynomial of A. Remark 4.4.2 For those eigenvalues Aj = IY.j + ico, which are complex, the functions in (4.4.8) can be split into real and imaginary parts. Hence if A is real, then although Pin (4.4.1) might contain complex numbers, the elements of etA must be real and consequently will consist of linear combinations, with real coefficients, of the functions
(4.4.9)
= 0, 1, ... , rj -
(k
4.5
1; j
=
1, 2, ... , s).
The Theoretical Foundation of the ABC Algorithm for Computing etA
In Section 4.3, we showed that the problem of computing the unique closed-form solution to the problem in PAn,
x=
Ax,
(4.5.1) (4.5.2)
reduced to that of computing etA, and the solution of the nonhomogeneous form of(4.5.1)could then be gotten by an additional integration. The solution of (4.5.1) and (4.5.2) was shown to be x(t) = etAxO.
(4.5.3)
The rigorous details needed to substantiate the algebraic procedure presented in Chapter 3 for computing cI>(t) = etA can now be given. Recall the procedure: Step 1. Compute the eigenvalues of A (possibly repeated and complex). Step 2. Determine the real n-vector function y(t) by applying the corre-
79
4.5 Theoretical Foundation of the ABC Algorithm for Computing erA
spondence rule to the eigenvalues according to the scheme
I ' ...1 t At t At "'t. rear x -e e 'De '2!e '''''(m_1),e, 2
complex A = oc
t'"-1
± ico --+- e" cos rot, e" sin rot; It -t e~ cos rot I! ' I!
tm -
e~
Ism ' rot .
'
tm -
1
1
... ; (m _ 1), e" cos rot,(m _ 1), e" sin rot ; in which m is the multiplicity of A in the characteristic polynomial. Step 3. Determine the real n x n matrix B such that By(t) = y(t). Step 4. Obtain the general n x n matrix solution C to the equation AC - CB = 0 in terms of an n-vector of arbitrary constants c. Step 5. By inspection, determine the n x n matrix function Y(t) such that Y(t)c = Cy(t). Step 6. Invert Y(O) to compute (t) = Y(t)y- 1(0) = e": The appropriate lemmas to be used in the argument will now be stated and proved. Definition A finite collection of $'-valued functions cl>1(t), cl>it), ... , cl>m(t), defined on ( - ex), oo), is called linearly independent over $I' if (4.5.4) for all t and fixed c, in $I' implies Cl
Lemma 4.5.1
=
C2
= ... =
Cm
= O.
(4.5.5)
The collection offunctions
f'e"' j l
(4.5.6)
(k = 0, 1, ... , rj :- 1; j = 1, 2, ... , s), with the Aj distinct elements of re, is linearly independent over re. Proof. Note that the equations resulting from setting linear combinations of the functions (4.5.6) equal to zero are precisely the equations of the type pl(t)e... ,1
+
pit)e... ,1
+ '" + p.(t)e...•1 = 0, t over re, and this class
(4.5.7)
in which the Pi are polynomials in of equations is invariant under differentiation. In fact, since exponential functions never take on the value zero, and since the Ai are distinct, by multiplying by e" . . ,/,
80
4.
it can be assumed that A1 any equation of the type
=
Linear Constant-Coefficient Equations: Theory
O. Thus the problem reduces to showing that
Po(t) + P1(t)eA1t +
0
••
°
+ pit)eAnt =
(4.5.8)
for all t, with Pj polynomials in t over C(j and the Aj # 0 and distinct, implies that all Pj = O. The proof is carried out by induction on n. Clearly the result is true for n = O. Thus assume that (4.5.8) implies that all Pi = 0 whatever the Aj as long as they are nonzero and distinct. The objective is to prove the corresponding statement for poet) + P1(t)eA1t + ... + pit)e Ant + Pn+1(t)e An+ 1' = O. (4.5.9) Let no be the degree of Po in (4.5.9) and then differentiate (4.5.9) no + 1 times to get the equations p~l(t)
(k
+
p~(t)eAlt
+ ... +
p~+1eAn+lt
=
°
(4.5.10)
= 0, 1, ... , no + 1). Note that (4.5.11)
where o
Pj (j = 1,2, ... , On
=
(4.5.12)
Pj
+ 1). But in particular, since pg'0+ 1)
P'i°+ 1(t) + Pzo+ 1(t)el'2t + ... + p~'?t.+/(t)elln+l1
=
0, = 0,
(4.5.13)
in which the J1.j = Aj - A1 are nonzero and distinct. Hence by the induction hypothesis (4.5.8), (4.5.14) (j = 1,2, ... , n
+ 1), and thus from (4.5.11), (pjoPl + AjpjD = 0
(4.5.15)
°
= 1,2, ... , n 0+ 1). But the unique solutions to the linear first-order differential equations (4.5.15) are exponential functions, and since Aj # and P~;o is a polynomial, it follows that p'r = O. By repeating the same argument, it follows by induction on k that
(j
P' =
0
(4.5.16)
°
= 1,2, ... ,n + 1;k = no, no - 1, ... ,0),andhenceby(4.5.12),allpj = in (4.5.10). This completes the induction, and the proof of Lemma 4.5.1 is complete.
(j
4.5 Theoretical Foundation of the ABC Algorithm for Computing etA
Corollary 4.5.2
81
The collection offunctions
(4.5.17) (k = 0, 1, ... , rj - 1; j = 1, 2, ... , s) with !Xj + ico, distinct is linearly independent relative to the real coefficient field 91..
Proof:
The result follows from Lemma 4.5.1 and the fact that 1. 2
t
cos Wot = -(e""j! + e-,roj ) J
0
'
. wot = -1 (e'"! t-O t e-,roj ) J 2i .
SIn
0
(4.5.18) (4.5.19)
The details are left to the reader. Remark 4.5.1
pendent over
Obviously, the collection (4.5.17) remains linearly indeC(J.
Remark 4.5.2 (Those readers who have studied matrix theory in the context of linear algebra can skip this remark.) The basic idea of a linear combination of objects has appeared in no less than three contexts up to this point; namely, linear combinations of n-vectors over $', m x n matrices over $', and $'-valued functions have been defined. Mathematicians have abstracted this common structure into an axiomatically defined algebraic system called a vector space over a field. The three mentioned examples and many others fall into this general framework. Important maps between the prototype vector spaces such as multiplication of n-vectors by a fixed m x n matrix or time differentiation of functions are abstracted by the notion of linear transformation of one vector space into another. Linear algebra develops the important concepts and results pertaining to vector spaces and linear transformations between vector spaces. This has the advantage of efficiency in that definitions of bases, linear independence, etc., are made once and for all, and when a scientist or mathematician encounters a system bearing the structure, he can apply the abstract theory and concepts to it without doing much additional work. Essentially, all matrix theory fits neatly within the framework, and the part of the standard theory that is nothing but convention becomes more apparent. Thus the uninitiated should now be well-motivated to take a few hours to work through an elementary text such as [12] and, it is hoped, include in his program of studies a course based at the level of, say, [11J. The material is important for understanding some of the details presented in the remainder of this section, and it has an even more important bearing on certain parts of the remaining chapters.
82
4. linear Constant-Coefficient Equations: Theory
Lemma 4.5.2
Consider the equation Cy(t) = etAv
(4.5.20)
for all t in f:Jl, in which y( t) is the n-oector function determined from the eigenvalues of A by the correspondence rule (3.2.2). For each v in f:Jln (4.5.20) has a unique n x n matrix solution C and the equation thereby determines a nonsingular linear transformation of f:Jln onto an n-dimensional linear subspace of the real n x n matrices. Proof. The existence of at least one solution C for each v in f:Jln follows directly from Remark 4.4.2. 1f(4.5.20)had two solutions C 1 and C2 for some fixed v in f:Jln, then it would follow that
(4.5.21) for all t, contradicting the linear independence proved in Corollary 4.5.2. Thus (4.5.20) defines a mapping, and it is linear; for if both C1y(t) = etAvb
(4.5.22)
C 2y(t) = etAv2
(4.5.23) (4.5.24)
for all t in f:Jl. The map is nonsingular, for clearly in (4.5.20) C = 0 implies that v = O. Since f:Jln has dimension n, the nonsingularity implies that the map has rank n, and this completes the proof. Proof of the ABC Algorithm.
A real n x n matrix C satisfies (4.5.20) for
some v in f:Jln if and only if AC - CB
= O.
(4.5.25)
Equation (4.5.25) is a consequence of the fact that y = By, which is seen by differentiating (4.5.20) and appealing to the linear independence of the coordinate functions of y(t). To show the reverse implication, multiply (4.5.25) by y(t) to get d
d/Cy) - A(Cy) = O.
(4.5.26)
But this equation has the unique solution Cy(t) = etAv
(4.5.27)
where v = Cy(O). This proves that (4.5.25) has the same n-dimensional linear space of solutions as does (4.5.20). Thus if (bb b 2, ... , bn ) is any real non-
83
4.6 Explicit Formulas for C(c) and e'A
singular n x n matrix (i.e., the b, constitute a basis for 9l n ), then the general solution C to (4.5.25) can be written as a linear combination C = C1Ci
for arbitrary real scalar Eq. (4.5.20)
Ci
+ C2C2 + ... + c.C;
(4.5.28)
in which the Ci are a basis of matrices defined by (4.5.29)
(i = 1,2, . 00' n). That is,
Cy(t) = CietAbl + C2etAb2 + ... + cnetAbn = etA(b 1 , b 2, ..• , bn)c = Y(t)c,
(4.5.30)
in which (4.5.31)
and c is the n-vector over 9l with cT = (c., C2"'" cn)' Hence all of the steps in the ABC algorithm are substantiated, showing that it computes (t) = Y(t)y- i(O) = e tA(bi, b2, . 00' bn)(b1 , b 2, ... , bn)-i = etA (4.5.32) as intended. EXERCISES
1. Show that for any n x n matrix A over
rc,
dette") = etr(A).
(4.5.33)
2. Let Ai' A 2, and <1>0 be any n x n matrices over ~. Consider (t), the unique matrix solution to the initial-value problem
=
A 1 <1>
<1>(0) = <1>0.
+
A 2 ,
(4.5.34) (4.5.35)
(a) Compute det (t). (b) Show that det (t) = 0 if and only if det <1>0 = O. (c) Show that if det <1>0 :F 0, then det (t) - 0 as t - 00 if and only if tr(A 1
4.6
+
A2 ) <
o.
Explicit Formulas for C(c) and etA
For a given matrix A, once the decision has been made as to whether to use correspondence rule (3.2.2) or (3.2.28), the associated yet) and Bare automatically determined.
84
4.
Linear Constant-Coefficient Equations: Theory
Assuming that the choice has been made, the dependence of the general solution C(c) of
AC - CB
=
0,
(4.6.1)
defined by the equation (4.6.2) on the n-vector c, and of etA on y(t), is rather explicitly shown in the following theorem. Theorem 4.6.1 C(c) = (c, Ac, ... , An-1c)(yO, Byo, ... , B n- 1yO)- \
in which yO
(4.6.3)
= y(O), and = (A1y(t), A 2y(t), ... , Any(t»,
(4.6.4)
= (ei, Aei"'" An-1ei)(Yo, Byo, ... , Bn-1yO)-1,
(4.6.5)
etA in which Ai
(el,e2, ... ,en) = In(i = 1,2, ... ,n). Proof. First the matrix inverse appearing in (4.6.3) and (4.6.5) is shown to exist. Select any n-vector ~ and consider (4.6.6) z(t) = (y(t), By(t), ... , Bn- 1y(t»~. It is apparent that z(t) is the unique solution to the initial-value problem z';" Bz,
(4.6.7) (4.6.8)
Thus if z(O) = 0, then z(t) = 0 for all t in f7t, and moreover, by computing successive dot products of(4.6.6) with the columns of In' it would follow that ~ = O. This proves that the matrix appearing in (4.6.8) is nonsingular. From successive differentiation of (4.6.2) at t = 0, the matrix equation
CCVo, By", ... , Bn-1yO) = (c, Ac, ... , An-1c)
(4.6.9)
is obtained, and thus (4.6.3) follows from (4.6.9). Substitution of (4.6.3) into (4.6.2) and the definition (4.6.5) show that
etAc = (c, Ac, ... , An-1c)(yO, Byo, ... , Bn-1yO)-ly(t) n
=
L i= 1
c;(ei,Aei, ... , An-1e;)(yO, Byo, ... , Bn-1yO)-ly(t)
n
=
L
i= 1
ciAiy(t) = (A1y(t), A 2y(t),.··, Any(t»c
(4.6.10)
4.6
85
Explicit Formulas for Ck) and erA
for all n-vectors c, implying (4.6.4), and Theorem 4.6.1 is established.
Remark 4.6.1 An elementary calculation shows that formula (4.6.4) is invariant under the transformation y(t) -+ My(t), B --t MBM-' for any nonsingular n x n matrix M over the complex field. It follows directly from this observation that the correspondence rule (3.2.2) applying to real matrices A could be replaced by {Al, A 2 , . . ., A,} --+ My(t) for any real n x n nonsingular matrix M and that the correspondence rule (3.2.28) applying to complex matrices A could be replaced by {Al, A 2 , . . ., A,} + My(t) for M any n x n nonsingular complex matrix, with the consequence that all the derived results would remain valid. The two original correspondence rules have the virtue that their associated B matrices are particularly simple. Recall that for any monic polynomial p ( x ) = x"
+ clx"-I + c2x,-2 + ... + c,
(4.6.11)
over a field F, the associated matrix
(4.6.12)
is called the companion matrix of p .
Remark 4.6.2 By computing the matrix B arising from each of the two correspondence rules (3.2.2) and (3.2.28),it is easy to check that the matrices A and B have the same characteristic polynomials. By Remark 4.6.1 it follows that this relation would be preserved as well for all other correspondence rules of the type discussed. Corollary 4.6.2
For any real or complex n x n matrix A etA = (z,<(t), z 2 t ( t ) ,* *
*,
zne
(4.6.13)
in which
zi = (ei,A e i , .. ., A"-'ei)
(4.6.14)
( i = 1, 2,. . ., n), I, = ( e l , e 2 , . . ., e,), and
<(t) = efAcel,
(4.6.15)
where A, denotes the companion matrix of the characteristic polynomial p A
86
4.
linear Constant-Coefficient Equations: Theory
of A. Moreover, e(t) can be computed as the unique solution to (yO, Byo,. 00' Bn-lyO)e(t) = y(t). Proof.
(4.6.16)
In view of Theorem 4.6.1, the problem reduces to showing that (yO, Byo, ... , Bn-ly0)etAcel = y(t)
(4.6.17)
for all t in rJf. Note that the proposed equality (4.6.17) does hold at t = O. Since y = By, it would be sufficient to show that the left-hand side of (4.6.17) satisfies the same differential equation. Thus the problem further reduces to showing that (yO, Byo, ... , Bn-ly0)Ac
= B(yO, Byo, ... , Bn-lyO).
(4.6.18)
But this is a valid equality since (4.6.19) and - cn! -
Cn _ I B
-
00' -
CI B
n-
I
= Bn
since in Remark 4.6.2 the matrices A and B were noted to have the same characteristic polynomial (4.6.20) which by the Cayley-Hamilton theorem annihilates B. This concludes the proof of Corollary 4.6.2. Remark 4.6.3 The representation of etA given by (4.6.13)-(4.6.15) is interesting because of its invariance relative to correspondence rules. Moreover, if A were perturbed so as to maintain the characteristic polynomial PA invariant, then the perturbation of etA would be fully reflected in the Z, matrices. The significance of(4.6.16)is that it provides a means for computing and hence etA. Corollary 4.6.2 establishes the rather remarkable fact that the general solution to x = Ax, which is an n-dimensional vector space, can be constructed out of the solution to the single initial-value problem in n-space: = Ace, e(O) = e l·
e
e
Theorem 4.6.3
= (pl(A)c, pz(A)c, etA = (A1y(t), A 2y(t),
C(c)
, piA)c),
(4.6.21)
, Any(t))
(4.6.22)
for all n-vectors c and all t in rJf in which A j = (pl(A)ej, p2(A)ej"'" piA)e),
(4.6.23)
4.6 Explicit Formulas for GC) and
87
etA
where the coefficients of the polynomials Pj(x) =
PI j + P2r + ... + Pnr n- I
(4.6.24)
are the elements of(yO, Byo, ... , Bn-IyO)-1 (j = 1,2, ... , n). In the situation in which y(t), yO, and B result from the complex correspondence rule (3.2.28), alternative formulas for the polynomials are
n (x -
m,-t
pix) =
L
s
rJ.jk(X - Ad
i=2
k~j-1
rJ.jk
=
1 (j _ 1)!(k _j
ak - j + 1
+ 1)' aA~-j+1
(4.6.25)
A;)m.,
I\ (AI s
A;)-ml
(4.6.26)
(k =j - l,j, .. . ,ml - 1; j = 1,2, ... ,m1)' The succeeding m, polynomials are given by (4.6.25) and (4.6.26) by the interchange mi +-+ m" Al +-+ A" etc. (r = 2, 3, ... , s). Proof. Equations (4.6.21) and (4.6.22) are just restatements of (4.6.3) and (4.6.4). Since B = diag(Bt> B 2, ... , B s ), and accordingly
y
° --
[:~J
.
(4.6.27)
,
es
where in (4.6.27) e, denotes the first column of the identity matrix 1m " el
s,«,
B2e2 (y,° B y,0 ... , tr:' y 0) = e2 .. . . . . [
e,
s»,
B~-1ell B~-le2 ..
.
B:-
1
...
.
(4.6.28)
e.
The matrix in (4.6.28) was already proved invertible, and the inverse has unique polynomials associated with its columns via (4.6.24). Thus the polynomials associated with the first m1 columns of the inverse are precisely those satisfying [piBdet]k = bi pj(B;)e i = 0
(k = 1,2, (i = 2,3,
, md, , s; j = 1,2, ... , md.
(4.6.29) (4.6.30)
The proof reduces to showing that these polynomials are the ones given by (4.6.25) and (4.6.26). (The remaining columns of the inverse matrix can be dealt with by repetition of the same argument upon performance of the interchanges mentioned in the statement of Theorem 4.6.3.)
88
4.
Linear Constant-Coefficient Equations: Theory
Since each Bi is an mi x mimatrix of the elementary Jordan type
Bi =
0
*..
1 Ri 0 1
-..
0
-
9
-0
1 li 0 . . . 0 1 Ai
0
A
it follows (see Theorem 4.2.2) that (4.6.29) and (4.6.30) are equivalent to the equations (4.6.32) (4.6.33) i = 2, 3 , . . ., s; j = 1 , 2 , . . .)ml).Note that the poly(k = 1, 2 , . . .,mi; nomials of (4.6.25) and (4.6.26)are of degree n - 1 or less since (ml- 1) + m 2 + m 3 + . . . + m, = n - 1. Since the polynomials p j of degree n - 1 or less satisfying (4.6.29) and (4.6.30),and hence (4.6.32) and (4.6.33))exist and are unique (since the matrix in (4.6.28)has a unique inverse), it is sufficient to show that the polynomials of (4.6.25)and (4.6.26)satisfy (4.6.32)and (4.6.33). From formula (4.6.25) it is clear that those polynomials satisfy
(4.6.34)
(4.6.35) .-a
Thus (4.6.32)holds for k = 1. To perform a proof by induction, assume that (4.6.32) is valid for 1, 2, . . .) k c m l . The task is to show that it is likewise valid for k + 1. Rewriting (4.6.25) as mi-
1
1
ajXx -
r=j-l
n,)r
n (x S
=
pj(x)
(4.6.36)
-
i=2
and differentiating that equation k times relative to x produce the equation ml - 1
r=j-1
r(r - 1).. . (r - k k
=
1 C,kpjyx)
r=O
+ l)aj,(x - Aly-k
"n (x i=2
I' ( k -0
-
li)-m'
(4.6.37)
4.6
Explicit Formulas for GC) and
89
e'A
in which C~ = k !/r !(k - r)! is the binomial coefficient. The resultant equation, obtained by setting x = Al in (4.6.37) and invoking the induction hypothesis, is s
bl+lk!(ljk
=
pJkl(Al)
IT (AI
- ~i)-m,.
(4.6.38)
i=2
Hj t= k + 1, then certainly (4.6.38) implies that pjkl(Ad = O. For j = k formula (4.6.26) reduces to
+ 1,
(4.6.39) which when substituted into (4.6.38) gives p)kl(Ad = 1 in this case. Thus the inductive proof of (4.6.32) for polynomials (4.6.25) is achieved. The fact that those polynomials satisfy (4.6.33) is obvious because of the product term in (4.6.25). The proof of Theorem 4.6.3 is now complete. Since the minimal polynomial of a matrix A divides the characteristic polynomial, it is clear upon distribution of the multiplication in (4.6.25) that piA) = 0 for j ~ rl + 1, where rl denotes the multiplicity of the eigenvalue Al in the minimal polynomial of A. A similar remark applies to the polynomials associated with the remaining distinct eigenvalues. Each matrix A j in (4.6.22) is simply the matrix whose columns are thejth columns of p;(A) (i = 1, 2, ... , n). The net effect is that the active terms of yet) in (4.6.22) associated with a multiple eigenvalue Ak = (lk + iWk are limited to those containing maximal factors t", P ::;; r k - 1, where rk is the multiplicity of Ak (k = 1, 2, ... , s) in the minimal polynomial. Remark 4.6.4
Remark 4.6.5 The polynomials (4.6.25) and (4.6.26) provide generating formulas for the elements of the inverse matrix (4.6.28); i.e.,
pJil(O)
Po = -i!-
(4.6.40)
t. = 1,2, ... , md. The remaining elements of the inverse can be computed in similar fashion from the polynomials obtained via the specified interchanges.
(i = 1,2, ... , ml;
Example 4.6.1
The matrix A =
2 2 -2] 0 3 1 [ o 1 3
(4.6.41)
has characteristic polynomial PA(X) = (x - 2)2(X - 4), and thus the
90
4.
Linear Constant-Coefficient Equations: Theory
eigenvalues are Al = 2, A2 = 4, with multiplicities m, = 2, m2 respectively. Direct substitution into (4.6.25) and (4.6.26) provides
(4.6.42)
Pl(X) = - lx(x - 4),
= - t(x - 2)(x - 4):
P2(X)
The interchange ml remaining
+-+ m2,
Al
+-+
= 1,
(4.6.43)
A2 in (4.6.25) and (4.6.26) produces the
pix) = l(x - 2)2.
(4.6.44)
Substitution of A as given by (4.6.41) into (4.6.42)-(4.6.44) leads to p,(A)
~ [~
p,(A)
~
p,(A)
~ [~
j -tJ'
[H -~l
(46.45)
(4.6.46)
i ~,
(4.6.47)
from which can he read off Al
=
[~ ~ ~l, o
A2 = [
~J
0
t ~ j],
-2
A3 =
(4.6.48)
0
? -20
[!
-2
0
(4.6.49)
2
~]2
!
.
(4.6.50)
Substitution of (4.6.48)-(4.6.50) into (4.6.22) finishes the computation: 2t 2t e 2te2t _ 2te ~ 4t) 2t 4t etA = 0 t(e + e t(e _ e 2t) . (4.6.51) [ 4t) 2t 4t 2t) o !(e _ e !(e + e
4.6 Explicit Formulas for C(e) and
91
erA
EXERCISES
1.
In the representation elA
=
Z2~(t),
... , Zn~(t))
(Zn(t), Z!~(t),
... , Z:~(t))
(Z1~(t),
of Corollary 4.6.2, show that elA T
=
in which (i = 1, 2, ... , n).
2. As in Exercise 1, discuss the action induced upon the remaining representation formulas appearing in Corollary 4.6.2 and Theorems 4.6.1 and 4.6.3 under the transformation A __ AT. 3. Study the question dealt with in Exercises 1 and 2, but for the transformation A -- M AM -1 for M fixed. Can you say anything about A -- M A with M fixed?
Chapter
5
Linear Constant-Coefficient Equations:
Qualitative Behavior
In many applications of differential equations the dependent variables denote errors or displacement of the state of a system relative to some reference or equilibrium state. It is important in such situations that the errors converge to zero appropriately fast with increasing time. Sometimes it is extremely useful to know whether a given differential equation has one or more periodic solutions. The results in this chapter provide answers to such questions based upon somewhat less computational effort than is involved in finding closed-form solutions. 5.1 Stability and Periodic Oscillation
This section concentrates on the qualitative behavior of the solution to the Cauchy problem in £Jt",
x=
Ax,
(5.1.1) (5.1.2)
in which A is an n x n matrix over £Jt. All results can be extended to complex space in a rather obvious fashion. Definition Equation (5.1.1) is called stable if for each XO in £Jt" the solution x(t) to (5.1.1) and (5.1.2) has !x(t)! bounded on 0 :::; t < 00. If for some solution Ix(tk)l-+ 00 as tk -+ 00 for some sequence tk> then (5.1.1) is said to be unstable. 92
93
5.1 Stability and Periodic Oscillation
Definition Equation (5.1.1) is called asymptotically stable if each solution x(t) to (5.1.1) has x(t) - 0 as t - 00. The characteristic polynomial PA and minimal polynomial qA of A contain a great deal of information about the qualitative properties of the solutions to (5.1.1). Such results are summarized in the following theorem. Theorem 5.1.1
Equation (5.1.1) is asymptotically stable (Xj
< 0
(j = 1,2, ... , s) for all distinct eigenvalues satisfying (5.1.3) is called a stability matrix). Instability occurs if
if and
only
if (5.1.3)
Aj
=
(Xj
+
uo, of A. (A matrix
(5.1.4) for at least one j in {1, 2, ... , s}, No conclusion can be drawn if (Xj :-::;; 0 (j = 1,2, ... , s) without further information: If (Xj :-::;; 0 (j = 1,2, ... , s), then stability occurs if and only !f those Aj = (Xj + ico, with (Xj = 0 have multiplicity one in the minimal polynomial qA; i.e., dim "If/;.} equals the multiplicity of Aj in PA' A sufficient condition for stability is that (Xj :-::;; 0 (j = 1,2, ... , s) and that each Aj = (Xj + ico, with (Xj = 0 have multiplicity one in the characteristic polynomial pA of A. Theorem 5.1.2 Equation (5.1.1) will have a nonconstant (oscillating) periodic solution offrequency W if and only if at least one of the distinct eigenvalues A j = (Xj + ico, (j = 1,2, ... , s) of A has (Xj = 0 and Wj = w.
Both Theorems 5.1.1 and 5.1.2 are easily verified consequences of the analysis of the structure of etA that was made in Section 4.4, using the Jordan form of a matrix. The next theorem is useful in estimating the rate of decay of the solutions to an asymptotically stable linear system and provides an upper bound on the rate of growth of an unstable system. Theorem 5.1.3
For A an n x n matrix over
~
n
letAI:-::;;
L IpiA) IIYk(t) I
(5.1.5)
k= 1
for 0 :-::;; t < 00 in which the polynomials Pk(X) are those defined in Theorem 4.6.3 and the Yit) are the coordinate functions of yet).
94
5.
Proof.
linear Constant-Coefficient Equations: Qualitative Behavior
Definition (4.6.23) and elementary estimation give n
I t>
n
1
n
I I kI
I(AjY)il =
j= 1
= 1
n
I
n
s I I
j= 1 k = 1
n
=
(Pk(A»ijyit)
I(Pk(A»ijIIYit)1
n
I I I(piA»ij IIYit) I
k = 1 j= 1
n
I
=
k= 1
O::;t
IpiA)IIYk(t)l,
(5.1.6)
Maximizing (5.1.6) over i produces the inequality n
I(A1y(t), A 2y(t),· .. , Any(t»I::;
I
k=l
Ipk(A)IIYit)l,
(5.1.7)
which, according to (4.6.22), concludes the proof.
Example 5.1.1
Let A be the matrix
- 1 A =
1
o -1 0
[
0
0 0-1
000
which clearly has eigenvalues A.l
~l, -2J
(5.1.8)
= -1, A.2 = - 2 of respective multiplicities
ml = 3, m2 = 1. Application of the formulas of Theorem 4.6.3 provides
the polynomials
= (x 2 + X + l)(x + 2), P2(X) = -x(x + l)(x + 2),
Pl(X)
P3(X) = (x pix)
=
+ 1)2(x + 2),
-(x
+
1)3.
(5.1.9) (5.1.10) (5.1.11) (5.1.12)
Computation of the corresponding matrix polynomials of A can be checked to produce matrices with Ipl(A)1 = IpiA)1 = IpiA) I = 1 and P3(A) = O. The estimate (5.1.5) is then (5.1.13)
95
5.2 The Lyapunov Approach to Asymptotic Stability
for all t 2 0, whereas computation of eta gives e-'
le'*l
=
0 0 0
te-' e-' 0 0
0 0 0 e-"
0 0 e-' O
= .e-[
+ te-'.
(5.1.14)
Remark 5.1.1 At the cost of accuracy in the estimate, evaluation of the matrix polynomials occurring in (5.1.5) can be avoided by dominating their . in Example 5.1.1, IAl = 2, norms by the estimated polynomials in ] A [ Thus and from (5.1.9)-(51.12) it follows immediately that Ip,(A)I I28, Ip2(A)I I 24, Ip,(A)I I18, Ip,(A)/ I 27, producing the less accurate estimate
lefAl I28e-'
+ 24te-' + 9t2e-' + 27e-".
(5.1.15)
Remark 5.1.2 It follows from the analysis of era made in Section 4.4 that if A is a stability matrix, then
o It
[eta/Ice-",
< co,
(5.1.16)
for a any positive number satisfying a
<
-ak
(5.1.17)
for each eigenvalue3.k = t l k + iokof A. The c is a positive number depending on the choice of M and, of course, A. The approach taken there falls short of providing a numerical value of c since the Jordan theorem is an existence theorem alone. Theorem 5.1.3 likewise provides the estimate (5.1.16), but it has the advantage of providing easily computable values for c and is not restricted to stability matrices. Since the information about the possible presence of asymptotic stability is present in the characteristic polynomial pa ,the problem of merely detecting asymptotic stability is a question about the coefficients of pa. The RouthHurwicz criterion effectively deals with the question (see Exercises, pp. 103105), as do other methods. In the next section we discuss an alternative method originated by Lyapunov. In conjunction with the Cetaev estimates, it leads to alternative estimates of the rate of convergence of asymptotically stable linear equations and has the virtue of being extendable to certain nonlinear equations. 5.2
The Lyapunov Approach to Asymptotic Stability
The Lyapunov approach is best introduced by means of a few simple examples in 9'. The first is XI =
x2,
i2 = -XI.
(5.2.1)
96
5.
Linear Constant-Coefficient Equations: Qualitative Behavior
Note that the eigenvalues of the associated matrix A are A. = ± i. By dividing one equation in (5.2.1) by the other and separating variables, it follows that along the solution curves, (5.2.2) where the constant r is determined by the initial values of Xt and X2' Thus the solution curves of (5.2.1) determine a clockwise motion about the origin in ~2 along circles of radius determined by the initial values of Xt and X 2, as indicated in Fig. 5.2.1. The time derivative of the real-valued function on [J/2, V = xi + x~, composed with the solution (Xt(t), xzCt)) of (5.2.1), is d dXl(t) dt v(x t (t), x 2(t)) = 2x t(t) ---;{t
dxzCt)
+ 2x zCt)---;{t for all
t. (5.2.3)
Equation (5.2.3) reflects the fact that the motion generated by (5.2.1) follows the level curves of v. The function vex1, X2) provides a measure of the distance of (x t, X2) from the origin. Suppose that (5.2.1) is altered by addition of a term to give (5.2.4) Although the separation-of-variables approach no longer works, insight into the nature of the motion generated in ~2 by (5.2.4) is attained by again differentiating the composition v(Xt(t), xzCt)) to get :t V(Xt(t), X2(t)) ,,; 2Xt(t)xzCt) + 2X2(t) [ -Xt(t) - x 2(t)] = - 2x~(t)
::::;; 0,
for all
t.
(5.2.5)
Inequality (5.2.5) indicates that the solution curves associated with (5.2.4) are transverse to the level curves of v = xi + x~ at all points where X2 #- O. Thus the motion conforms to that indicated in Fig. 5.2.2. Notice that the
--+--+-+-__1--+--- XI
Fig. 5.2.1 Periodic motion of(5.2.1).
97
5.2 The Lyapunov Approach to Asymptotic Stability
--+-+---1--+--1-- XI
Fig.5.2.2 The asymptotically stable motion of (5.2.4).
conclusion drawn from (5.2.5) was attained without solving the differential equations (5.2.4). The basic Lyapunov approach avoids solving the differential equations but instead selects an appropriate function v which, when differentiated along the solutions of (5.1.1), produces an inequality illuminating the stability-theoretic aspect of the motion. Before abstracting the approach for treating the general equation in (jpn, consider another system in (jpz, (5.2.6) Differentiating v = xi the inequality
+
x~
along the solution curves of (5.2.6) produces
d dt V(Xl(t), xz(t)) = 2x lxz
+ 2xz( -Xl + Xz) =
z 2xz ~ 0,
(5.2.7)
which suggests that the solution curves are of an unstable character, as indicated in Fig. 5.2.3. The final example, (5.2.8)
--+---1r--+--+--+-- XI
Fig.5.2.3 The unstable motion of(5.2.6).
98
5.
Linear Constant-Coefficient Equations: Qualitative Behavior
illustrates an important point leading to the appropriate generalization. Differentiation of v = xi + x~ along the solutions of (5.2.8) results in d dt V(XI(t), xit))
= 2XIX2 + 2xi -2x I - X2) (5.2.9)
= -2XIX2 - 2xt
which provides little information. The problem lies with the choice of v, for an alternative choice v = xi + ix~ gives :t v(xl(t), ~2(t)
= 2XIX2
+ X2( - 2XI - X2)
=
-x~ ~
O.
(5.2.10)
The level curves of the latter, v = xi + ixt are ellipses centered at the origin. Inequality (5.2.10) indicates that the motion is transverse to the level curves, inward, and again indicates the likely possibility of asymptotic stability. The approach and basic phenomena appearing in the above twodimensional examples are abstracted in the Lyapunov treatment of (5.1.1) that follows. Definition A real-valued function f with domain &in of the type n
f(x) =
n
L L sijxiXj,
(5.2.11)
j= I i= I
in which the Xi are the coordinate variables of x in &in and the Sij are fixed in &i, is called a real quadraticform. The Sijcan be regarded as elements of an n x n symmetric matrix S, and then in terms of the dot product on &in (5.2.11) can be written as . f(x)
= x·Sx.
(5.2.12)
Example 5.2.1
f(x)
= Xl2
- 2XIX2 + 3X22
= [Xl] X2 . [ -11
-31][XX21].
(5.2.13)
Observe what happens to S under a linear change of variables x = My, with M any real n x n matrix and y in &in: g(y)
=
f(My)
=
My' SMy
=
y' (MTSM)y.
(5.2.14)
Note that MTSM is symmetric and g is a quadratic form in y. The induced transformation on symmetric matrices S - MTSM is called a congruence transformation. (Recall that S - M - I SM is called a similarity transformation.) Definition A real nonsingular matrix N is called orthogonal if NT = N- I .
99
5.2 The Lyapunov Approach to Asymptotic Stability
Theorem 5.2.1 Let S be any real, symmetric n x n matrix. There exists a real orthogonal matrix N such that (5.2.15)
The A.i are real and are the eigenvalues of S. N can be taken to be of the form (5.2.16)
in which the columns c 1 , C2,"" c" constitute an orthonormal and hence linearly independent collection of eigenvectors of S. Remark 5.2.1 Since Theorem 5.2.1 is a standard result of linear algebra, the proof is omitted. It is the basis for diagonalization of quadratic forms; i.e., if N is applied to make the change of variables x = Ny in (5.2.12), the resulting quadratic form (5.2.14) in y becomes
g(y)
" = L A.iyf.
(5.2.17)
i= 1
Corollary 5.2.2
If S is a real symmetric n x n matrix, then
alx/ 2
~
x· Sx
.Blxl 2
~
Ixl
in which the norm is the Euclidean norm = ~ the smallest and the largest eigenvalues of S.
(5.2.18) and a,
.B are, respectively,
Definition If S is a real symmetric n x n matrix such that x . Sx > 0 for all nonzero x in rJl", then S is called positive-definite. If x . Sx ~ 0 for all nonzero x in rJl", then S is called positive-semidefinite. The notation S > 0 or S ~ 0 is adopted to signify that S is positive-definite or positive-semidefinite, respectively. Definition
Let A be a real n x n matrix. The linear matrix equation
ATp
+ PA =
-In,
(5.2.19)
in which P is regarded as a variable matrix, is called Lyapunov's linear equation. Theorem 5.2.3 If A is a stability matrix, then Lyapunov's equation (5.2.19) has a unique matrix solution P with P > O. Conversely, if(5.2.19) has a solution P with P > 0, then it has no other such matrix solution and A is a stability matrix.
Proof.
Suppose that A is a stability matrix. According to Remark 5.1.2,
o~
t <
00,
(5.2.20)
for some positive numbers c and a. Since A and AT have the same eigenvalues,
100
5.
Linear Constant-Coefficient Equations: Qualitative Behavior
an inequality (5.2.20) with A replaced by AT (with possibly a different value of c) also holds. It follows that the integral
:1
P =
(5.2.21)
etATetAdt
is convergent. This matrix is clearly symmetric. By integration by parts and (5.2.20), elementary computation gives
=
-I,
-
ATP,
(5.2.22)
which shows that (5.2.21) defines a symmetric solution to (5.2.19). To show that it is positive-definite, select arbitrary y in W nand employ definition (5.2.21) to compute
(5.2.23) Since letayl2 is a continuous function of t, equality y * Py = 0 in (5.2.23) can hold if and only if
(5.2.24)
etAy = 0,
for all 0 It < co,and hence precisely when y = 0. Thus P > 0. Suppose Pl were another solution to (5.2.19) with Pl > 0. Consider the real-valued functions on W", u(x) = X - P Xand ul(x) = x * Plx. Along every solution x(t) of (5.1.1) and (5.1.2); d dt
- u(x(t)) = i ( t ) . Px(t)
+ X(t). P q t )
= A x ( t ) *P x ( t ) = X(t).(ATP
+ x(t)-PAx(t)
+ PA)x(t) = - IX(t)12,
(5.2.25)
where the last equality is due to (5.2.19). A similar calculation yields d dt
- u&(t))
=
- lx(t)12,
05t<
(5.2.26)
0O
Subtraction of (5.2.26) from (5.2.25) and integration lead to u(x(t)) - u,(x(t)) = u ( x 0 ) - u1(x0),
0 It <
00.
(5.2.27)
But since x(t) + 0 as t -+ co,it follows from the last equation that u(xo) = ul(xo) for all xo in 9". This can happen only if P = PI. The proof of the
converse part of Theorem 5.2.3 will follow as a corollary to the next theorem.
101
5.2 The lyapunov Approach to Asymptotic Stability
Theorem 5.2.4 Cetaev Estimates. Let A be a real n x n matrix and P be a solution to Lyapunov's equation (5.2.19)for which P > O. Denote the smallest and the largest eigenvalues of P by ex and p, respectively. Then F/iie- t/2a ::;; letAI ::;; ~e-t/2(J for 0::;; t < 00. (5.2.28)
(The norm is the operator norm induced by the Euclidean norm on
~n.)
Proof. In terms of P, an assumed positive-definite solution of (5.2.19), define vex) = x· Px. According to Corollary 5.2.2, alx(tW ::;; v(x(t)) ::;; Plx(tW
(5.2.29)
along each solution x(t) to (5.1.1) and (5.1.2). As in (5.2.25),
dv~(t)
o ::;;
= -lx(tW,
t <
00.
(5.2.30)
From (5.2.29) and (5.2.30) it follows that dv(x(t)) dt
v(x(t))
s --p-,
(5.2.31)
which can be multiplied by et/(J and integrated to get 0 ::;; t <
v(x(t)) ::;; e-t/(Jv(XO),
00.
(5.2.32)
Application of both inequalities of (5.2.29) to (5.2.32) shows that exlx(tW ::;; e-t/(JPlxoI2,
(5.2.33)
which is divided by a and the square root taken, with the result that Ix(t)1 ::;; ~ e- t/2(J IxoI, 0 ::;; t < 00. (5.2.34)
By taking the sup of (5.2.34) over Ixol = 1 and recalling that x(t) = etAxO, the second inequality of (5.2.28) is established. The remaining inequality can be derived in a similar fashion, and Theorem 5.2.4is considered proved. Conclusion of the Proof of Theorem 5.2.3. Let A be a real n x n matrix for which (5.2.19) has a solution P > O. Thus inequality (5.2.34) holds for each XO in ~n. If A were not a stability matrix, then according to Theorem 5.1.1 the condition x(t) - 0 as t - 00 would fail for some solution to (5.1.1). Thus a contradiction is reached and the proof of Theorem 5.2.3 is complete. Remark 5.2.2 In the literature, Lyapunov's linear equation (5.2.19) sometimes appears in the form ATp
+ PA
=
-Q,
(5.2.35)
where Q > 0 is given. However, upon multiplication of (5.2.35) on the left
102
5.
Linear Constant-Coefficient Equations: Qualitative Behavior
and right by Q-1 /2, where QI /2 is a positive-definite square root of Q, and replacement of the variable P by QI /2PQI /2 the equation becomes ATp
+
PA = -In'
(5.2.36)
in which A = Q1/2 AQ -1/ 2. That is, under a change of variable y = Q1/2 X, Eq. (5.2.35) is transformed into the form (5.2.19), and hence there is no loss in taking Q = In. Remark 5.2.3 Since both Theorem 5.1.3 and Theorem 5.2.4 provide estimates of IetA I, their relative merits in applications deserve some discussion. Suppose that the matrix A in (5.1.1) is prescribed (e.g., arises from some engineering design process). If all that is required is verification that A is a stability matrix and that Ix(t) within some rate specification as t - 00 for each solution x(t) of (5.1.1) and (5.1.2), then the most appropriate procedure would be to solve (5.2.19) as a system of linear algebraic equations to obtain P and then compute its eigenvalues. (Actually only the largest and smallest eigenvalues are needed.) If A is indeed a stability matrix, then according to Theorem 5.2.3 the solution will automatically be symmetric and unique. Thus to cut down the dimension of the system of algebraic equations to be solved, the variable P in (5.2.19) should be written in symmetric form. If it turns out that the smallest eigenvalue of P is positive, the asymptotic stability of (5.1.1) is ensured. The inequalities in (5.2.28) provide estimates of the rates of convergence in terms of the smallest and largest eigenvalues of P. The advantage of working with P is that its symmetry implies that its eigenvalues are all real, meaning that P is generally more easily treated using numerical methods. If all that is required is confirmation that A is a stability matrix, then it is enough to check that P is positive-definite. The theory of matrices provides a standard determinant test of a symmetric real n x n matrix P for positivedefiniteness:
1- °
(5.2.37)
(k = 0,1, ... , n - 1), where Pk is the symmetric matrix obtained from P by striking off the 'last k rows and k columns of P. The appeal of this test diminishes for very large n, since even the most efficient computation of determinants involves considerable arithmetic. Of course, (5.1.5) and (5.2.28) are only estimates, which can be quite conservative. The estimate given by (5.1.5) would generally be expected to be the less conservative of the two, and it makes the dependence of the estimate upon the eigenvalues and elements of A more explicit. Moreover (5.1.5) is more comprehensive in that it holds whether or not A is a stability matrix. In short, there is a trade-off between the estimates, involving the amount
5.2
103
The Lyapunov Approach to Asymptotic Stability
of information given on the one hand, and the computing required on the other. Conceivably the situation might arise in which failure of the system specification to meet the estimates would force closed-form computation of etA and explicit calculation of 1 etA1.
Example 5.2.2 Consider the matrix A of Example 5.1.1. From its block diagonal form it is apparent that P can be taken to be of the form P = diag[P,, P2, P 3 ] in which
Substitution of this form of P into Lyapunov's equation (5.2.19) leads to the 1 1 solution x1 = x4 = z, x3 = x 5 = a, x2 = 1. The eigenvalues of the resultant matrix P are $ - a f i , $, +, $ + S f i . For the matrix under consideration, the second' inequality of (5.2.28) is thereby
letAl I 2.62e-0.38',
0I
t
<
(5.2.38)
00.
Although the values of the estimates (5.1.13) and (5.2.38) cannot be compared directly at t = 0 since the norms involved are not the same (only equivalent), the more conservative nature of the decay rate of (5.2.38) is obvious.
a,
1
D, = a,,
Dz'= 11'
1
a2
.. .,
Dk
=
0 0
... 0
a3 a5 ... a 2 a4 a1 a3 1 a2 0 a1
...
...
0
0
a2k- 1
a2k-2
9
ak
104
5.
Linear Constant-Coefficient Equations: Qualitative Behavior
(a) find the symmetric matrix S such that j(x) = x - Sx, (b) find an orthogonal matrix N such that NTSN is diagonal, (c) let x = Ny and compute the quadratic form g(y) = j(Ny). 3. Prove Corollary 5.2.2. 4. Compute letAI as well as the estimates' of the norm appearing in Theorems 5.1.3 and 5.2.4 for the matrix
A 5.
[_~ -~J
=
Compute letAI and its estimate, given in Theorem 5.1.3, for the matrices
(b)
A
=
[
~ ~ ~].
-1 6.
0 0
Prove that each matrix of the type A = P- 1(K - tIn),
(5.2.39)
where P is a real n x n positive-definite matrix and K is an n x n real skew-symmetric matrix (i.e., K T = - K), is a real stability matrix. Show that each real n x n stability matrix A can be factored into the form indicated for unique choice of P > 0 and skew-symmetric K; namely, P is the solution to Lyapunov's equation
ATp
+ PA =
-In'
P = fo'" etATetA dt
(5.2.40) (5.2.41)
and K is the solution to
ATK
+ KA
= _(A -; AT).
K = fo'" e
tAT
(A -; AT)e tA dt.
(5.2.42) (5.2.43)
Note that P and K can be computed by solving a system of linear algebraic equations, i.e., Lyapunov's equation, since K = PA + tIn. 7. Evaluate the integral
105
S.2 The Lyapunov Approach to Asymptotic Stability
in which
- 1 A = 8 [ -1
-~ ~],
M
o -3
=
[
~ -~ ~].
-1 -3
0
8. In the unique factorization of stability matrices (5.2.39), show that the transformation A - A -1 on stability matrices induces the transformation P-(K - tIn)Tp- 1(K - tIn), K - -K, on the symmetric and skew-symmetric parts, respectively. 9. Gershgorin's Theorem. By working with the defining equation of an eigenvector, prove that all eigenvalues of an n x n matrix A over the complex field lie in the union of disks in the complex plane n
U{z:lz -
i= 1
aiil
:s;;
rd,
(5.2.44)
in which the radii are computed as
r,
n
=
L laijl
(i
=
1,2, ... ,n).
(5.2.45)
j= 1.Ni
10. Gershgorin's theorem provides an easily computable localization of the eigenvalues of a matrix which sometimes provides useful information. (a) Which of the following are stability matrices?
1 73] .[-3 1-1] -3 2 1] [1-1 2 1 0-2 [ 3 1-1] r~ ~ _~], 2 1 OJ [ [o l~ 5 -1
0,
2 -4
1,
1 -4 O. -1 1-2
(b) Which of the following are positive-definite matrices?
1 -2 -1 , -1 -1 2
1
-2
4
3 -1 . -1 2
11. Three tanks, each containing v gallons of water, are connected in a loop by pipes through which the contents of each tank is pumped to the next, with all pumping rates equal. Initially arbitrary amounts of salt are instantaneously dissolved in each tank. Derive an appropriate differential equations model of the system and apply it to prove mathematically that the total amount of salt in the system will tend to a uniform distribution amongst the tanks as t - 00.
Chapter
6
General Linear Equations with Time-Varying Coefficients
The class of linear equations treated in previous chapters is expanded to include those with time-varying coefficients. A unique solution to the initialvalue problem is constructed by means of a matrix series (which is no longer a matrix power series) along lines somewhat similar to those taken in the constant-coefficient case. The notion of a fundamental matrix persists, reflecting the fact that the solutions to the homogeneous equation continue to possess a finite-dimensional vector space structure. The major properties of fundamental matrices are developed and applied to obtain representations of the solutions to vector and matrix differential equations and their duals. Estimates of the growth (or decay) rates of the solutions are derived. 6.1 Widening the Scope
A great deal more attention could be spent in pursuing detailed analyses of specific equations of the constant-coefficient type studied in Chapters 3-5. Those equations playa limited but important role in applications, and the results presented are quite useful in such investigations. However, there are compelling reasons for taking an opposite path leading to a more comprehensive theory. Examples in Chapter 1 indicated that only slight changes in problems modeled by linear constant-coefficient equations can produce equations with time-varying coefficients. An even more important source of such equations arises in the theory of nonlinear systems. In Chapter 11, we show that the 106
6.2
107
Preliminaries
derivatives of solutions to nonlinear equations relative to initial and other parameters satisfy linear equations with time-varying coefficients. These linear variational equations are often useful in studying the nonlinear equations from which they originate. Thus there is good reason to study a more general linear model. The question concerning the appropriate form of the theory for such an extension deserves some thought. One approach would be to attempt to follow the program that was completed for the constant-coefficient equation, namely, to develop (1) an existence and uniqueness theory, (2) a computational method for representing the solutions in some closed form that illuminates the underlying structure of the solutions, (3) an efficient method for assessing important aspects of the qualitative behavior of the solutions. We shall find that this admittedly optimistic approach does achieve some level of success. 6.2 Preliminaries
Attention is focused on the equations in J x :Fn,
x= x(to)
=
A(t)x
+ f(t),
(6.2.1) (6.2.2)
x",
where J is a nonempty open subinterval of ~, to E J, :F is either the real or the complex field, and A(' ) and f( .) are matrix-valued functions defined on J of respective sizes n x nand n x 1, generally over :F. Recall that a matrix-valued function, say A('), is locally integrable on J if each of its elements is measurable (relative to the Borel subsets of J) and the norm IAOI is (Lebesgue) integrable on each compact subinterval of J; i.e., lIA(t)1 dt <
00
(6.2.3)
for each closed and bounded subinterval I c J. The text always assumes that both A( .) and f( .) are locally integrable on J. This condition is automatically fulfilled if A(' ) and f(·) are continuous or only piecewise continuous. In this more restrictive setting, the integral is then just the Riemann integral of elementary calculus. The added generality allows modeling systems in which extensive switching can occur without requiring more than the standard theorems of real analysis in effecting the proofs. A solution x(-) to (6.2.1) is defined to be any absolutely continuous map from J into :F n satisfying (6.2.1) on all of J but for possibly a subset of mea-
108
6,
General linear Equations with Time-Varying Coefficients
sure zero; i.e., almost everywhere (a.e.). Of course if A(') and f(·) are continuous on J, then solutions will actually satisfy (6.2.1) at all t E J. Just as for equations having constant coefficients, the variation-of-parameters approach makes it possible to pass from a solution of the homogeneous equation to one for the nonhomogeneous equation by a single integration, and hence it is sufficient to begin by concentrating on the homogeneous equation. 6.3
The Fundamental Matrix Series Defining.
Some experimental integration by parts, etc., leads to the appropriate generalization of the exponential matrix series lI>(t, ..)
=
L [L~(In)](t, CXJ
(6.3.1)
r),
k=O
in which the matrix function A( .) is defined on an open interval J c fA, t and .. are in J, and L~ is the k-fold composition of the linear integral operator defined by [LA(M)](t,
t)
=
f
M(t, a)A(a) da,
(6.3.2)
In (6.3.1) L~ denotes the identity operator, and thus L~(In) = In. It is observed that for A(') locally integrable the terms of the series (6.3.1) are continuous n x n matrix-valued functions defined on J x J. To keep the notation simple, L~(In) will be denoted simply by which, remember, is an operator product, not a matrix product. Thus (6.3.1) is written as
u,
ex>
lI>(t, ..) =
L
L~t,
r ).
(6.3.3)
k=O
Lemma 6.3.1 If A(') is locally integrable on J, then
IL~t, ")1 s
:! If
IA(a)1 dar.
IlI>(t, ")1 ~ k~O IL~t, ")1 s exp (IS)A(a) 1do I) all (t, ..) E J x J (k = 1, 2, ...). lI> is continuous on J x J
(6.3.4) (6.3.5)
for and its series (6.3.3) is absolutely and uniformly convergent on compact subsets of J x J. Proof.
To minimize the number of absolute value signs, the proof is
109
6.3 The Fundamental Matrix Series Defining
written out for r < t. The first step is to prove (6.3.4), ILk(t, t)1
s
:! [f
jA(u)! do
l
k E {1, 2, ...},
(6.3.6)
by induction. Certainly (6.3.6) holds for k = 1 by the standard inequality, taking the norm under the integral. Now assume that (6.3.6) holds for fixed k E {1, 2, ...}. By the induction hypothesis (6.3.6) and the definition of L, etc., IV+ l(t, t)1
~ s
If f [f I
L't, u)A(u) dul :::;;
k\
f
IL't, u)IIA(u)1 do
I JIA(U)I do
A(ro) dco
1
II [II [f J+ 0
= - (k
+ 1)!
t
= (k :
1)!
!A(u)j do
ou
Jk+l do
"IA(ro)1 do:
(6.3.7)
1,
which completes the proof of (6.3.6) by induction. Inequalities (6.3.5) follow directly from (6.3.4) as do the absolute convergence and uniform convergence on compact subsets. The continuity of is a consequence of the uniform convergence and continuity of the L'·,·). This concludes the proof of Lemma 6.3.1.
Theorem 6.3.2 Let A( .) be a locally integrable n x n matrix-valued function defined on an open interval J c f!It. Then the matrix function defined by (6.3.1) is the unique matrix solution to the initial-value problem on J x J,
o
ot (t, r) = A(t)(t, r),
a.e.
t E J,
(6.3.9)
r E J.
for each
(6.3.8)
Moreover is the unique matrix solution to the initial-value problem
0<1>
a:r(t, r) $(t, t)
= -(t, t)A(t),
= In
for each
a.e. t E J.
r
E
J,
(6.3.10) (6.3.11)
As well as being absolutely continuous in each variable, is continuous on J x J. $(t, r) is invertible for each (r, r) E J x J and $-l(t, r) = (t, t).
(6.3.12)
110
6.
General Linear Equations with Time-Varying Coefficients
If A(') is continuous on J, then
k~O
f u«,
(6.3.13)
a)A(o) da.
From (6.3.4) of Lemma 6.3.1 it follows readily that (6.3.14) for , ~ t, and consequently Fubini's theorem applies, allowing the order of summation and integration in (6.3.13) to be reversed. The latter equation can then be written as
J,t k~O e«. a)A(a) do, 00
(6.3.15)
and similar reasoning shows that (6.3.15) is valid for all (t, r) E J x J. By (6.3.4) the series in (6.3.15) is locally integrable for each fixed t E J, and hence (6.3.15) can be differentiated relative to " giving
a -a
L 00
= -
k=O
Lk(t, ,)A(,) = -
a.e.
rEJ
(6.3.16)
for each t E J. This proves (6.3.10). It is asserted that each Lk(t, r) is absolutely continuous in t and that
a I:k+l (r, r)
at
k{
= A(t)r, \t, r),
a.e.
tEJ
(6.3.17)
for each r E J(k = 0, 1, ...). The proof of this assertion is left as an exercise for the reader. To complete theproof of (6.3.8), apply (6.3.17) to rewrite the sum
(6.3.18)
111
6.3 The Fundamental Matrix Series Defining «I>
From (6.3.4) it is clear that !A(W)
:t~
Lk(w,
r)1 ~
W
for r
IA(w)1 exp(i IA(a)1 da)
~
co. (6.3.19)
The series defining l1> was proved convergent. Moreover, the inequality (6.3.19) allows the Lebesgue dominated convergence theorem to be invoked to take the limit as N -+ (f) under the integral in (6.3.18), with the consequent equality l1>(t, r)
= In +
f
A(w)l1>(w, r) dco,
(6.3.20)
and moreover, because of (6.3.19), Eq. (6.3.20) can be differentiated to conclude that
o
ot l1>(t, r)
= A(t)l1>(t, r),
a.e. t e J
(6.3.21)
for each r ~ t in J. The case in which r > t is treated with the obvious minor change. The uniqueness claims are easy consequences of Gronwall's inequality (see Exercise 1 at the end of this section). Equation (6.3.12) can be proved as follows: Consider '¥(t) defined as '¥(t) = l1>(t, r)l1>(r, r),
t E J,
(6.3.22)
with arbitrary r E J fixed. Applying (6.3.8) and (6.3.10), Eq. (6.3.22) can be differentiated to get d
dt'¥(t) = A(t)'¥(t) - '¥(t)A(t),
a.e. t E J.
(6.3.23)
Thus '11(.) is a solution to (6.3.23) satisfying the initial condition '¥( r) = In as is clear from (6.3.9) and (6.3.22). But the matrix function '¥(t) = In, t E J, is another solution to the same initial-value problem. Application of Gronwall's inequality again shows that the solution must be unique, and the conclusion that 'P(t) = In, t E J, is inevitable. This finishes the proof of (6.3.12). If A( .) happens to be continuous, then by inspection of the proofjust given it is evident that (6.3.8) and (6.3.10) hold everywhere in J x J and moreover that l1>(t, r] and its first-order partial derivatives are continuous in J x J. It follows that l1> is thus once continuously differentiable on J x J. This concludes the proof of Theorem 6.3.2. Notation. Occasionally the matrix function l1> will be denoted by l1>A when its dependence upon A(' ) needs to be emphasized.
112
6.
General Linear Equations with Time-Varying Coefficients
Remark 6.3.1 From Theorem 6.3.2 and results derived in Chapter 4 it is apparent that A is just e(l- t)A in the case in which A( .) = A is independent of tEJ. EXERCISES
1. Prove the following form of Gronwall's theorem: Each continuous solution x( .) to the inequality x(t)
s
lX(t)
+
f
also satisfies the inequality x(t)
s
lX(t)
+
I
p(U)IX(U)
p(u)x(u) da,
exp(f
pew)
tE
[a, b),
dW) do,
t E [a, b),
where IX(') is continuous and 13(') is nonnegative and locally integrable on
[a, b).
2. Apply Gronwall's theorem to prove the uniqueness claims concerning Eqs. (6.3.8) and (6.3.9), (6.3.10) and (6.3.l.1), and (6.3.23) that were made in the proof of Theorem 6.3.2. 3. Prove the assertion concerning (6.3.17) that was made in the proof of Theorem 6.3.2.
6.4 Existence and Uniqueness of Solutions to Vector Differential Equations 'and Their Duals The results of Theorem 6.3.2 can be applied to vector differential equations. As often occurs in mathematics, it is natural to discuss some of the results in terms of the dual space of ~n rather than in terms of ~n itself. The appropriate definitions are recalled. Definition An tional on ~n if
~-valued
function I, defined on .
~n
is called a linear func(6.4.1)
for all Cl' Cz in ~ and all Vl, Vz in ~n. The collection of all linear functionals on ~n is denoted by ~". and can be turned into a vector space relative to the addition and scalar multiplication defined by
(11
+ Iz)(v) =
11(v)
+ Iz(v),
(c/ 1)(v) = c/1(v)
(6.4.2) (6.4.3)
113
6.4 Existence and Uniqueness of Solutions and Dual Systems
for all 11 ,12 e $'"*, v e $'", and c e $'. The resultant vector space, still denoted by $'"*, is called the dual space of $'".
Remark 6.4.1 $'"* has dimension n and its elements x* can be identified with I x n (row) vectors x over $' if x*(v) is defined to be the matrix product xv, for all v e $'". Remark 6.4.2 Any n x n matrix function A( .) defined on an interval J can have associated with it the differential equation in J x $'", x
= A(t)x,
(6.4.4)
as well as a differential equation in J x $'"*, (6.4.5)
x* = x*A(t).
Some interesting relationships between the solutions of the two equations appear in Theorem 6.4.1.
Theorem 6.4.1 Let A(·) be a locally integrable n x n matrix-valuedfunction (over $') defined on an open interval J c fit. Then there exists a unique solution x A to the initial-value problem on J x J x $'", x
= A(t)x,
x(t)/t=t =
(6.4.6)
a.e. te J,
e
(6.4.7)
namely,
(6.4.8) There also exists a unique solution x~ JxJx$'"* x* x*(t)lt=t
to the initial-value problem on
= -x*A(t), a.e. t e J, = e* for each (r, e*)eJ x
(6.4.9) $'"*,
(6.4.10)
namely,
(6.4.11) Moreover, the solutions to the two equations satisfy the invariance relation
e
x~(t,
r, e*)xA(t, r, e) = e*e
(6.4.12)
for all (t, r) e J x J, e $'", e* e $'"*. If A(·) is continuous on J, then x A and x~ are once continuously differentiable relative to their argument (t, r) e J x J, and Eqs. (6.4.6) and (6.4.9) hold for all (t, r) e J x J. Proof.
All statements are rather obvious consequences of Theorem 6.3.2.
114
6.
General linear Equations with Time-Varying Coefficients
Remark 6.4.3 Equation (6.4.9), transposed so as to appear as a differential equation in :Fn , is called the adjoint of (6.4.6) and appears frequently in control theory and other areas of mathematics involving differential equations or optimization. Remark 6.4.4 If A(') is assumed to be k-times continuously differentiable on J, then since A satisfies (6.3.8) and (6.3.10), it is clear that A will be (k + Il-times continuously differentiable relative to (t, T) E J x J (k = 0, 1, 2, ...). Thus the solutions (6.4.8) and (6.4.11) will likewise be (k + Il-times continuously differentiable on J x J.
6.5 Variation-of-Parameters Formulas for Vector and Matrix Differential Equations
Theorem 6.5.1 Let A(') and f(·) be locally integrable matrix-valued functions (over :F) of sizes n x nand n x 1, respectively, both defined on an open interval J c flt. Then there exists a unique solution x A to the initial-value problem on J x J x :Fn
x=
A(t)x
+
f(t),
a.e.
for each
x(t)lt=< = ~
(6.5.1)
tEJ, (T,
x J
~)EJ
(6.5.2)
given by the variation-of-parameters formula
xA(t, T,
~)
=
A(~' r)~ +
f
4> A(t, (1)f(a) da,
(6.5.3)
Proof. A proof can be based upon application of Theorem 6.4.1. The details as well as the proof of the following theorem are left to the reader.
Theorem 6.5.2 Let Al ('), A 2 ( ' ) , and F(') be locally integrable n x n matrixvalued functions on an open interval J with arbitrary r E J and n x n matrix Z fixed. Then there exists a unique solution to the initial-value problem
'P =
A1(t)'P - 'P A 2(t)
+ F(t),
a.e.
t EJ,
(6.5.5)
'Plt=< = Z, namely,
'P(t, r, Z)
Example 6.5.1
= 4>Al(t, r)Z4>A2(T, t) +
(6.5.4)
f
4>A/t, a)F(a)A,(a, t) da,
(6.5.6)
An important example of (6.5.4) is the equation a.e. t
E
J,
(6.5.7)
6.6
115
Estimates of Solution Norms
arising from A 1(t) = -AT(t), A 2(t) = A(t), and F(t) = In. Here A(') is assumed to be locally integrable on an interval J. In the special case in which A(') is independent of t, the (steady-state) equation, gotten by replacing P in (6.5.7) by zero, is Lyapunov's algebraic equation (5.2.19). Thus it should be not altogether surprising to find that in the next section (6.5.7) plays an important role in the problem of estimating rates of growth or decay of norms of solutions to differential equations associated with A(·). The solution to (6.5.7), as represented by the variation-of-parameters formula (6.5.6), is pet, r) =
t)Po
f
t)
(6.5.8)
in which Po = plt=<.
6.6
Estimates of Solution Norms
The problem of estimating the rate of growth or decay of solutions, studied earlier in the context of constant-coefficient equations, continues to be of interest, but understandably it is more complicated for equations with timevarying coefficients. Again the (real) n x n matrix-valued function A(') is given on an open interval J for which it is assumed to be locally integrable. The principal equations are
x=
A(t)x,
a.e. t E J,
xlt=t = ~,
(6.6.1)
(6.6.2)
(r, ~) E J x !!An. The primary goal is to obtain estimates of the solution Ix(tW and its integral. Throughout this section the norm employed on !!An is the Euclidean norm and the induced operator norm IMj = maxlMxl, max over Ixl = [x] = 1, is applied to a matrix M. The (r, ~) appearing in (6.6.2) should be regarded as fixed although arbitrary. As already announced, the problem will be dealt with by the device of studying (6.5.7). In explaining the connection, first note that in (6.5.8), if Po is symmetric, then so is pet, r). Moreover if Po > 0 (positive-definite), then likewise pet, r) > 0 for r ~ t and t appropriately near r. Consequently there is no difficulty in fulfilling the assertion that
.vx:x
o< for all nonzero y E !!An, r
a(t)IYI2 ~ y' P(t)y ~ {3(t)lyI2 ~
(6.6.3)
t and t near r for at least some of the solution
116
6.
General linear Equations with Time-Varying Coefficients
curves pet) to (6.5.7). The scalar functions 0(') and {3(-) are assumed to be continuous and could be taken to be, respectively, the smallest and largest eigenvalues of pet), but they are not necessarily so chosen. The problem of selecting the appropriate solution curve P(·) and computing 0(') and {3(') from A(' ) will be discussed later. The next theorem provides the first justification for interest in inequality (6.6.3).
Theorem 6.6.1 Suppose that P(·) is a symmetric solution to (6.5.7) satisfying (6.6.3) for all t in a subinterval ofJ with t :S t. On that subinterval O(t)
[1 -
exp( -
f ~:»)
] \e1
2
:S
s for all
eE
~n.
f
IX(CTW dCT
{3(t) [1 - exp( -
f ct~:»)
]
Jel
2
(6.6.4)
e
Proof. Fix arbitrary E ~n. Since the solution to (6.6.1), (6.6.2) has the representation x(t) = <1JA(t, t), and pet) has a representation given by (6.5.8), by substitution of the latter into (6.6.3) along with y = x(t), it follows that O(t)lx(tW :S
f
e· p(t)e - e· <1J~(CT,
which is equivalent to O(t)lx(tW
+-
f
IX(CTW dCT:S
t)<1JA(U, 7:) do
,·P(t)'.
e,
(6.6.5)
(6.6.6)
Upon division of (6.6.6) by x(t), application of the rightmost inequality of (6.6.3), and multiplication by expm dCT/ct(CT)], it is easy to see that
for t :S t. Integration 'of (6.6.7) from r to t and division by the exponential term lead directly to the second inequality of (6.6.4). Similar operations, starting with the second inequality of(6.6.3),provide the remaining inequality of (6.6.4) to complete the proof of Theorem 6.6.1.
Remark 6.6.1 One immediate consequence of Theorem 6.6.1 is that the second inequality in (6.6.8)
117
6.6 Estimates of Solution Norms
will hold if iX(t) > 0 on
['t,
00),
and if moreover
roo
J.
du f3(u)
=
00,
(6.6.9)
then the first inequality of (6.6.8) is likewise valid.
Remark 6.6.2 The proof of Theorem 6.6.1 relied on the assumption that iX(t) remains positive over the range of t being considered. (In applications, iX(t) is usually obtained, as a lower approximation to the smallest eigenvalue of P(t).) Examples demonstrate that the smallest eigenvalue of P(t) can become zero as t is increased from r to some T < 00. Theorem 6.6.1, for the given P('), will then not apply outside the interval ['t, T] since (6.6.3) must necessarily fail. One remedy for extending the interval determined by some approximating iX(·) would be to find a better solution P(·) to (6.5.7). Another would be to find a better lower approximation than iX(·). An effective procedure for accomplishing these objectives is provided in the next section.
DETERMINATION OF 01(') AND P(·)
To convert Theorem 6.6.1 into a result having practical as well as theoretical interest, two obstacles must be surmounted. The first is to single out solutions P(·) to (6.5.7) which are positive-definite over the interval of concern. The other is to find an effective procedure for approximating the extreme eigenvalues of such' P(·) to within any prescribed accuracy. The former is handled by restricting attention to any finite interval of the type r ~ t < b < 00, (r, b) E J x J, and using P(t)
=
i
b
cI>T(U, t)cI>(u, t) do
(6.6.10)
on the interval. The latter can then be approached through the series expansion appearing in the next theorem.
Theorem 6.6.2 Let A(') be locally integrable on J and consider any compact interval ['t, b] c: J. The integral (6.6.10) provides a symmetric positive-definite matrix solution to (6.5.7) on the interval ['t, b) and P(t)
=
r
Ao(t)
+ At(t) + ... + Ak(t)
+
cI>T(U, t)[AT(u)Ak(u)
+ Ak(u)A(u)]cI>(u, t) da,
(6.6.11)
118
6.
in which
General Linear Equations with Time-Varying Coefficients
r
(6.6.12)
Ao(t) = (b - t)I n , A k+ 1 i
::;;
=
+ Ak(a)A(a)] do,
(6.6.13)
[1'J~(~)]k
(6.6.14)
Ak(t),
(6.6.15)
[AT(a)Ak(o)
t ::;; b (k = 0, 1,2, ...). Moreover,
IAit) I ::;; (b -
t)
in which
In particular, co
pet) =
L
k=O
with the series uniformly convergent on
[i, b]
and
pet) ::;; (b - t)eqA(t)I n •
(6.6.16)
Proof. Routine calculations show that (6.6.10) is a matrix solution to (6.5.7) that is symmetric and positive-definite at each t E [i, b). Formula (6.6.11) with k finite follows by repeated integration by parts applied to (6.6.10). The estimate (6.6.14) is easily established using induction on (6.6.13). Equation (6.6.15), the uniformity of the convergence, and (6.6.16) are elementary consequences of(6.6.14) applied to (6.6.11). The reader is invited to check the details, and Theorem 6.6.2 is thereby proved.
Lemma 6.6.3 Let S(· ) be a real n x n symmetric matrix-valued function defined on an open interval J. The minimum and maximum eigenvalues of Sea) at each point a E J define real-valued functions on J which are continuous at each point in J at whicn S( .) is continuous. Proof. Since Sea) is symmetric at each a E J, its eigenvalues are real, and thus the smallest eigenvalue ac(a) of Sea) defines a function ac(') on J. Let t E J be a point at which S(·) is continuous, and in order to reach a contradiction suppose that ac(') were not continuous there. Then there would exist a sequence t k -- t as k -- 00 with t k E J such that ac(tk) -f+ ac(t) as k -- 00. Thus by extracting an appropriate subsequence, it can be assumed that either
(k = 1,2,
)
(6.6.17)
= 1,2,
)
(6.6.18)
or (k
6.6
119
Estimates of Solution Norms
for some number p. From the symmetry of S(·) it follows that there exist unit vectors ~k' ~* in [JItn such that Ct(tk) = ~k' S(tkRk and Ct(t) = ~*. S(t)~*. By extracting a further subsequence, it can be assumed that ~k ~ ~oo as k-r CXJ for some ~ 00 E [JItn that necessarily will have I~ 00 I = 1.Thus inequalities (6.6.17) and (6.6.18) could be written respectively as (k
= 1,2, ...)
(6.6.19)
(k
= 1,2, ...).
(6.6.20)
and By the continuity of S(·) at t, it follows upon taking the limit as k ~ (6.6.19) that
00
in
(6.6.21) over the collection of which is a contradiction since ~ * minimizes ~. S(t)~ unit vectors in [JItn. In the complementary case, (6.6.20) obtains and can be rewritten as - ~*'S(tk)~*
~k'S(tk)~k
+ ~*'[S(tk)
- S(t)]~*:2: (k
P> 0
= 1,2, ... ). (6.6.22)
By the continuity of S( .) at t, etc., I~*
.[S(t k) -
s(t)]~*1
~ IS(tk) - S(t)II~*12
= IS(tk) - S(t)1 ~O
(6.6.23)
as k ~ 00. However, for all sufficiently large k, inequalities (6.6.22) and (6.6.23) imply (6.6.24) over the collection which is again a contradiction since ~k minimizes ~. S(tk)~ of ~ E [JItn of unit length. This concludes the proof that the continuity of S(.) at a point t E J implies the continuity of Ct( •) at t. The proof of the remaining conclusion of Lemma 6.6.3 regarding the largest eigenvalue of S(· ) uses the same ideas and similar arguments and is therefore omitted. Lemma 6.6.4 Let Sk(-) and Soo(') be symmetric real n x n matrix-valued functions, continuous on a compact interval [a, b] (k = 1, 2, ...). Denote the minimum and maximum eigenvalues of these matrices by Cti'), CtooC) and Pi'), Poo('), respectively. If Sk(-) ~ Soo(-) on [a, b] as k ~ 00, then Ctk(') ~ Ct oo ( ' ) and Pk(') ~ Poo(') uniformly on [a, b] as k ~ 00.
Proof. To prove that Ctk(') ~ Ct oo ( ' ) uniformly, it is sufficient to prove only pointwise convergence since by Lemma 6.6.3 the Ctk(') and Ct oo ( ' ) are continuous real-valued functions on a compact interval. To effect a proof by contradiction, assume that, at some point t E [a, b], Ctit) fr Ctoo(t) as k ~ 00.
120
6.
General Linear Equations with Time-Varying Coefficients
By extracting an appropriate subsequence, there is no loss in assuming that either
= 1,2,
)
(6.6.25)
(k = 1,2,
)
(6.6.26)
(k
or for some number p > O. As in the proof of Lemma 6.6.3, inequality (6.6.25) can be rewritten as (k = 1,2, ...)
(6.6.27)
for appropriate unit vectors ~k' ~oo in fJtn. By assuming a further extraction of an appropriate subsequence, it can be assumed that ek -+ e* as k -+ 00 for some ~* E rJin of unit length. The same elementary manipulations as those that appeared in the proof of Lemma 6.6.3, along with the hypothesis that Sk( .) -+ S 00 ( • ) on [a, b] as k -+ 00, show that (6.6.27) leads to the contradiction eoo .Soo(t)e 00 >
e*· Soo(t)~*
(6.6.28)
and (6.6.26) leads to the contradiction of the fact that ~k minimizes ~. Sk(t)~ over the collection of unit vectors in fJtn (k = 1,2, ...). In any case, the appropriate contradiction is achieved, thus completing the proof of the statement that if Sk(') -+ Soo(') on [a, b] as k -+ 00, then OCk(') -+ oc oo(') uniformly on [a, b] as k -+ 00. The conclusion concerning Pk) and Poo(') can be argued in a similar manner and is thus left to the reader. Lemma 6.6.4is now established. The next theorem is a continuation of the results obtained in Theorem 6.6.2 and uses the same notation. It provides a means for approximating the extreme eigenvalues of P(·) without requiring the computation of cI>A' and in a form (6.6.3) needed for application of Theorem 6.6.1. The following string of definitions is involved:
I -t; a.e. a E ['t, b]}, = inf{s I AT(a)Ak(a) + Ak(a)A(a) ~ si; a.e. a E ['t, b]},
"Ik = sup{y AT(a)Ak(o) + Ak(a)A(a) ~ ~k
k
Ak(t)
= max{A I L
i=O
A;(t) ~ AIn},
k
f.Lk(t)
= min{f.L I L
i=O
(6.6.29)
A;(t) ~ f.LI n },
OCk(t) = Ak(t)/(l - Yk),
Pk(t) = f.Lk(t)/(l - ~k)'
ocoo(t) = max]« I P(t) ~ ocln }, Poo(t)
= min{PI P(t)
~ PIn},
r ~ t ~ b.
6.6
121
Estimates of Solution Norms
Remark 6.6.3
If
lim inf Yk
~
0,
k ....oo
then Yk -- 0 as k -- 00. To see this, for arbitrary e > 0 consider Yk = 1, 2, ...). For this choice of Y in the definition of Yb
= Yk - e
(k
AT(IT)Ak(IT)
+ Ak(IT)A(IT)
~ (Yk - £)In ,
(6.6.30)
a.e. IT E ["t, b], which implies that lim sup k-C(l
~. (b cIlT(IT, t)[ AT(IT) Ak(IT) + Ak(IT)A(IT)]cIl(lT, t) dIT ~ Jt
~ [lim sup Yk - £]~'P(t)~ k-+ co
(6.6.31)
.
for each nonzero ~ E f1tn, t E ["t, b). From (6.6.14) of Theorem 6.6.2 and the fact that pet) > 0, inequality (6.6.31) reduces to lim sup Yk ::s; e.
(6.6.32)
k ....oo
But
£
> 0 is arbitrary, and this is consistent with (6.6.32) only if lim sup Yk k ....oo
= O.
(6.6.33)
°
This concludes the demonstration that Yk -- as k -- 00. A similar argument shows that bk - - 0 as k -- 00 if lim sup, .... 00 bk ::s; O. The functions Ak(-), Jlk('), IXk('), Pk('), 1X 00 ( - ) ' and Poo(-) are continuous on ["t, b]-a consequence of Lemma 6.6.3. Both IX 00 ( . ) and Poo(') are strictly positive on ["t, b) since pet) > 0 on that intervaL They are the extreme eigenvalues of P(·).
Remark 6.6.4
Theorem 6.6.5 Let A(') be locally integrable on J and consider any compact interval ["t, b] c J. Assume that
lim infYk ~ 0,
(6.6.34)
k ....oo
lim sup k ....co
s, s
O.
(6.6.35)
Then for each e > 0 there exists a K. such that
o < IXk(t)In s 1X00(t)In s pet) s Poo(t)In ::s; Pit)In for
"t ::s; t ::s;
(6.6.36)
b - e, all k ~ K •. Moreover, assumptions (6.6.34) and (6.6.35)
122
6.
General Linear Equations with Time-Varying Coefficients
imply that
both uniformly on [""t", bJ as k -
(6.6.37)
and
IXk(-) - 1X 00(-) 00.
Proof. Lemmas 6.6.3 and 6.6.4and Theorem 6.6.2imply that Ak(' )-1X 00(') and Pi") - 1300(-) uniformly on [""t", bJ as k - 00. As discussed in Remark 6.6.3, assumptions (6.6.34) and (6.6.35) ensure that Yk - 0 and £>k - 0 as k - 00. The uniform convergences (6.6.37) thus follow from the equations defining lXi') and Pk(')' On applying the inequality defining Yk' it follows from (6.6.11) of Theorem 6.6.2 that for every f. > 0,
k
=
L
Ai(t)
i=O
+ (Yk
- f.)P(t),
(6.6.38)
r ~ t ~ b,
for all k sufficiently large that 1 - Yk > O. Since gives
f.
> 0 was arbitrary, (6.6.38)
""t" ~
t
s
b,
(6.6.39)
for all k large enough that 1 - Yk > O. In similar fashion it is easy to see that ""t" ~
t
s
b,
(6.6.40)
for all k so large that 1 - £>k > O. In summary, IXk(t)I" ~ P(t) ~ f3k(t) I" ,
r ~ t ~ b,
(6.6.41)
for all appropriately large k such that 1 - Yk > 0 and 1 - £>k > O. In view of Remark 6.6.4 and the uniform convergence (6.6.37), it follows immediately that for each f. > 0 there exists a K. such that (6.6.42) on
""t" ~ t ~
b -
f.
for all k
~
K •. This completes the proof of Theorem 6.6.5.
6.6
123
Estimates of Solution Norms
Remark 6.6.5 Theorem 6.6.5 and the definitions leading up to it were developed to provide computationally feasible approximations Q(i.), Pk(') to the extreme eigenvalues 01 00 ( -}, Poo(-) of P(·) needed in (6.6.3) for the application of Theorem 6.6.1 in computing upper and lower estimates of the integral norm of the solutions to the initial-value problem (6.6.1) and (6.6.2). The precise manner in which the theory accomplishes this goal deserves some reiteration and further comment. The extreme eigenvalues 01 00 ( ' ) , Poo(-) would provide the best possible choice of 01(') and P(·) in (6.6.3) were it not for the fact that their calculation would require the computation of A or some equivalent to get P(· ). Theorem 6.6.5 advocates the less accurate Q(k(' ), Pk(') on a subinterval T :::; t :::; b - e as appropriate choices of Q( .) and P(·). The computations of the Q(k(') and Pk( .) work more directly with A(') itself, and this of course vastly enhances the applicability of Theorem 6.6.1. The uniform convergence concluded by Theorem 6.6.5 ensures that any desired level of approximation can be achieved (at least in principle) by increasing k at the cost of the additional computation. Remark 6.6.6 In some applications the computational effort can be reduced by sacrificing accuracy. That is, in some cases, rather than compute 'Yk> Ok' Ak(-), and J.lk(· ) for some fixed (perhaps small) value of k, it might be easier to find numbers Yk' bk, E < b and continuous lk), f1.k(·) such that
AT(o)Ak(a)
+ Ak(a)A(a) ~
AT(u)Ak(u)
+ Ak.(u)A(u) :::; bkl n,
°< l
a.e.
Yk1n,
with 1 - Yk > 0, 1 -
I
i=O
(6.6.43)
EJ,
(6.6.44)
lJJ,
(6.6.45)
a.e. a E [T,
k
k(t)I n :::;
a E [T, EJ,
all
Ai(t):::; f1.k(t)I n,
t E [T,
bk > 0. Then application of(6.6.11) leads to for
t E [ r, lJ),
(6.6.46)
where iik(t) =
l k(t}/(l
- Yk),
(6.6.47)
Pit)
f1.it)/(1 - J'k)'
(6.6.48)
=
Thus (6.6.46) provides an estimate of the extreme values of P(·) of the type needed for application of Theorem 6.6.1. The acceptability of the estimates must be judged in the context of the application. If they are adequate, the question of arbitrary high degree of approximation is irrelevant.
124
6.
General Linear Equations with Time-Varying Coefficients
Remark 6.6.7 Hypotheses (6.6.34) and (6.6.35) are rather mild assumptions. For example, they are met if I AT(t) 1 + IA(t)1 is bound on [T, b] since, according to Theorem 6.6.2, A k ( • ) as k - 00 uniformly on [T, b].
1 1- °
The following corollary follows readily from Theorem 6.6.1 and Remark 6.6.6. Corollary 6.6.6 Let A(') be an n x n integrable real matrix-valued function on a compact interval [a, bJ. Assume that for some numbers y and b < 1, a.e.
t E [a, b
J.
(6.6.49)
Then x(t, ~), the unique absolutely continuous solution to the initial-value problem in [a, b] x PAn:
x = A(t)x, x(a, ~)
=
a.e.
t E [ a, b],
(6.6.50) (6.6.51)
~,
satisfies the inequalities
(6.6.53) Proof.
T
The conclusion follows directly from computing, using k =
= a in Theorem 6.6.1 and Remark 6.6.6.
Example 6.6.1
°
and
In the scalar equation
v + a(t)v =
a.e.
0,
t e [0,1],
(6.6.54)
[0, 1],
(6.6.55)
assume that a(' ) is measurable and satisfies
°s
aCt)
s
1,
tE
with strict inequality holding on a subset of [0, 1] of positive measure. The claim is that for each solution v(' ) to (6.6.54) on [0, 1], the norm (6.6.56)
125
6.6 Estimates of Solution Norms
satisfies the inequality
~(O)
[1 ~ f~
~
exp ( -
]
f~ ~:»)
Ix(O)J2
Ix(a)J2 da
13(0)
[1 -
f~ ~~:»)
exp ( -
]I
tE
x(O)J2,
[0, 1],
(6.6.57)
in which ~
1 - t - f/ (1 - a)[1 - a(a)] da 1 + 2 fA (1 - a)[1 - a(a)] do '
(t) -
(6.6.58)
~-::--;;-;----,------:-;>,=--------:--:-=i~
pet) = 1 - t + f: (1 - a)[1 - a(a)] da 1 - 2 fA (1 - a)[1 - a(a)] da
(6.6.59)
To verify this claim, write out (6.6.54) in first-order form to obtain the matrix coefficient A(t)
=
[-~(t)
~J.
(6.6.60)
With formulas (6.6.12) and (6.6.13), compute (6.6.61)
Ao(t) = (1 - t)12'
J1 (1 -
A 1(t) =
a)[1 - a(a)]
da[~
~J,
(6.6.62)
and hence A T(t)A 1(t)
+ A 1(t)A(t)
=
2
5,1 (1 -
a)[1 - a(a)] do [ -
~(t)
~J.
(6.6.63)
From (6.6.63) it is apparent that ")1112
s
A T(t)A 1(t)
")11
=
-2
+ A 1(t)A(t) ~
(6.6.64)
(;112'
where
(;1 =
fa1 (1 -
a)[1-a(a)] da,
(6.6.65)
2 fa1 (1 - a)[1 - a(a)] da
because of (6.6.55). That restriction on a( .) also ensures that 1 -
(6.6.66) ")I 1
> 0 and
126
6.
General Linear Equations with Time-Varying Coefficients
1 - <>1 > O. The eigenvalues of 1
L
Ai(t) = Ao(t)
i=O
=
+ A 1(t)
[1
Jtr (l -
11
1- t
(l - 0')[1 - a(O')] dO'l
0')[1 - a(O')] do
~~
1- t
compute to be Al(t)
=
1- t
fll(t) = 1 - t
-11 + 11
(l - 0')[1 - a(O')] da,
(6.6.68)
(1 - 0')[1 - a(O')] da.
(6.6.69)
Now it is clear from (6.6.65), (6.6.66) and (6.6.68), (6.6.69) that (6.6.58), (6.6.59) arise from the choices and and the claim (6.6.57) is a direct consequence of Theorem 6.6.1. The preceding theory and technique for approximating the extreme eigenvalues of the solution PO to (6.5.7) can be applied to obtain estimates of Ix(tW as well as of its integral.
Theorem 6.6.7 Let PO be a symmetric solution to (6.5.7) satisfying (6.6.3) for all t in a subinterval of J with r ~ t. On that subinterval the solution x(' ) to (6.6.1), (6.6.2) satisfies
~~:~
exp( -
f~~:»)
1~12 ~
Ix(tW
~ ~~;;
fP~:»)
exp( -
1~12
(6.6.70)
for all t in the subinterval. Proof. Let vet) = x(t)· P(t)x(t). Since pet) satisfies (6.5.7), it is easy to check that
d~;t)
= -lx(tW.
(6.6.71)
Since
o < ~(t)In
~
pet)
~
P(t)I n
for
r
~
t,
(6.6.72)
6.6
127
Estimates of Solution Norms
successive applications of (6.6.72) to (6.6.71) imply
dv(t) vet) -
dt pet),
--<-which integrates to give
vet)
~
vCr) exp( -
f P~:)
Then another application of (6.6.72) yields
a(t)lx(tW
s
p(r)I~12
(6.6.73)
exp( -
(6.6.74)
.
f p~:) ,
(6.6.75)
and the second inequality of (6.6.70) is clear. A similar approach produces the remaining inequality in (6.6.70) and can easily be checked by the reader. Thus the proof of Theorem 6.6.7 is secured.
EXERCISES
1. In Chapter 5 we estimated the norm of the solution to equations with a constant-coefficient matrix that is at the same time a stability matrix. The techniques and theory given in this chapter provide alternative methods and also cover problems in which the constant matrix is not necessarily a stability matrix. (a) Show that if A(') is a constant stability matrix, then the integral (6.6.10) has a limit as b --+ 00, and that the matrix limit is the one used in deriving the Cetaev estimates of Theorem 5.2.4. (b) Compare the computations involved in the methods of this chapter with those required for application of Theorem 5.2.4 to equations with constant stability matrix coefficients. (c) Compute the estimates in Theorem 6.6.7 on an interval ~ t ~ 2 based on values of k = 0, 2, 4 in Theorem 6.6.5 for the constant-coefficient matrix of Example 5.1.1 and compare the results with that appearing in (5.2.38). 2. In Eq.(6.6.54), choose aCt) = t to get the Airy equation. Compute the terms in the estimates (6.6.57). Compare the results with those obtained by computing the terms in the estimates (6.6.70) and integrating the resulting inequalities. 3. Derive estimates of Ix(tW and its integral for the equation
°
v + t- 1 v = on J
=
(0, 2).
°
128
6. General Linear Equations with Time-Varying Coefficients
4. Derive estimates of Ix(t)l' and its integral for the Bessel equation of zero order
ii
6.7
+ t-'d + u = 0.
Fundamental Matrices of Equations with Timevarying Coefficients
Theorem 6.7.1 The determinant of any n x n matrix sofution 'I' to the diferential equation
Y = A(t)$,
a.e. t E J ,
(6.7.1)
satisfies the equation
det Y ( t ) = e x p ( l tr A(u) d o ) det Y(z),
(6.7.2)
t EJ, z E J.
Proof. Using subscripts to denote rows, in terms of its rows write
1-
(6.7.3)
Then by the n-linearity of the determinant function relative to rows, it follows that
d dt
- det
Y = det
+
... + det
,
(6.7.4)
a.e. t E J . Since Y satisfies the equation -d' I ' = A Y = A dt
a.e. t E J ,
(6.7.5)
129
6.7 Fundamental Matrices for Time-Varying Coefficients
in which Ii is a linear combination of rows r1' r2' ... , rn , excluding rio Hence, on substitution of the derivatives (6.7.6) into the determinants of (6.7.4), by the n-linearity along with the fact that the determinant of a matrix having a repeated row is zero, the resultant equation
d
dt det 'P(t)
= [tt A(t)] det 'P(t),
a.e.
(6.7.7)
tEJ
obtains. This is a linear differential equation whose unique solution is clearly the function given by formula (6.7.2), and the proof is complete.
Remark 6.7.1 It is evident from (6.7.2) that det 'P(t) = 0 at some point t E J if and only if det 'P(') is identically zero on J. Theorem 6.7.2 If 'P and 'i' are n x n matrix solutions to (6.7.1) and 'P is nonsingular, then 'i'(t) = 'P(t)C,
(6.7.8)
tEJ,
for some appropriate time-independent n x n matrix Cover fF. Moreover, each nonsingular matrix solution qi to (6.7.1) is given by (6.7.8) for appropriate nonsingular C. Proof. Let 'P and 'i' be matrix solutions of (6.7.1) with the former nonsingular. Differentiation of the matrix function (t) = 'P- 1(t)'i'(t), (6.7.9)
using the fact that 'P and qi are solutions of(6.7.1), leads to
~ = (_'P- 1d'P '1'-1) qi + 'P- 1~ 'i' dt
dt
=
-'¥-1A'P'¥-1'i'
dt
+ '¥-1A'i' = 0,
a.e.
tEJ.
(6.7.10)
But this implies that is a (constant) matrix over fF; call it C. That is, (6.7.11)
130
6.
General Linear Equations with Time-Varying Coefficients
which proves (6.7.8). The proof of Theorem 6.7.2 is concluded by noting in (6.7.8) that det 'f1 = 0 if and only if det C = O. Remark 6.7.2 Any nonsingular solution '¥(t) to (6.7.1) is called a fundamental matrix of A(·). It follows from Theorem 6.7.2 that the fundamental matrix cD(t, r) of Theorem 6.3.2 can be computed from any fundamental matrix 'P of A( .) according to the formula (6.7.12)
EXERCISES
1. Verify that the solutions to (6.4.6) constitute a finite-dimensional vector space. Determine an ordered basis for the vector space and determine the dimension. 2. Show that if Yl> Y2, ... , Yn is a linearly independent collection ofabsolutely continuous n-vector functions defined on an open interval J, then its elements constitute an ordered basis of the vector space of solutions to the equation x = A(t)x for some n x n locally integrable matrix function A(') oti J,
3. Verify that $'n., the dual of $'n, is a vector space and show that its dimension is n by exhibiting an ordered basis. 4. Prove that cDA(t, 0) = cDA(t, 't)<1lA('t, u) for all t, r, U in J. 5. Verify Remark 6.7.2. 6. For the matrix function A(t) =
[~ t~
J
tEJ =
s,
show that cD A(t, r) ¥= exp(J~ A(u) du) by computing both sides of the claimed inequality. 7. Show that cD~ = <1l AT if A(t) is independent of t and give an example demonstrating thatthis equality can fail for time-varying A(·). 8. Consider the nth-order scalar differential equation x(n)
+ a1(t)x(n-l) + ait)x(n-2) + ... + an(t)x = 0,
in which the coefficient functions a;(t) are locally integrable $'-valued functions defined on an open interval J c ~ (i = 1,2, ... , n). (a) Prove that the scalar equation has a linearly independent collection of solutions
6.7
131
Fundamental Matrices for Time-Varying Coefficients
scalar equation as a first-order system in J x
Show that
ff'n.
cPn(t) ]
cP~I)(t)
cP~n ~ l)(t) is a fundamental matrix for A(·). 9. Let cPI' cP2"'" cPnbe real- or complex-valued functions, (n - Ij-times continuously differentiable on an open interval J. Assume that the matrix function <11('), computed from the cPi as in Exercise 8, is nonsingular on J. (a) Show that <11 is a fundamental matrix for the continuous matrix function A(') associated with the first-order form of a scalar equation
having continuous coefficients a;(') on J. (b) Show that the cPi constitute a basis for the vector space of solutions to the scalar equation. 10. Apply Theorem 6.7.1 to compute det <1I(t) in Exercise 8. 11. Apply Exercise 8 to compute a fundamental matrix and <11A(t, r) for the differential equation tX(2)
+ 5X(I) + (13/t)x =
0
(a) for t E J = ( - 00, 0), (b) for tEJ = (0,1). (Hint: Look for solutions of the type cP(t) = jtj). and split them into real and imaginary parts.) Generalize the problem to the Euler equation tX(2)
+ {3X(I) + (y/t)x
= 0
in which {3, yare real numbers. 12. Dual equations are occasionally used in control theory to simplify models of input-output systems. Consider the equation (6.2.1), x = A(t)x + f(t), for some family of inputs f E f. Let <11* be the n x n matrix solution to the dual equation
1
11
132
6.
General Linear Equations with Time-Varying Coefficients
6.8 Continuous Time, Discrete State Markov Processes
Problem situations involving some element of uncertainty frequently arise in applied work. A significant subset of these can ultimately be modeled, using techniques of applied probability, as special linear differential equations of the type discussed in this book. The brief introduction given here is intended to make the reader aware of this important field of application and to show by example that it is possible to treat some of these problems with little special knowledge. The exposition is kept intuitive and informal. Probability theory systematically attaches a number p[E], between zero and one, called the probability of E, to each event E of concern. This number in interpreted as a degree of likelihood that the event would happen in a trial. It is done in such a manner that (6.8.1) if the occurrence of E 1 has no bearing on the occurrence of E 2 (i.e., if E 1 is independent of E 2 ) , and vice versa. Furthermore, (6.8.2) and, in particular, if it is not possible for both E 1 and E 2 to happen together, then it follows from (6.8.2) that P[E1 or E 2 ]
= P[E 1 ] + P[E2 ]'
(6.8.3)
E 1 IE 2 denotes the event E 1 under the assumption that E 2 has happened, and the probability of that event p[Ell E 2] is called the conditional prob-
ability of E 1 , given that E 2 has happened. Now imagine a situation in which a real scalar variable Set), called the state of the system at time t, evolves with time t E £Ft. It is assumed that Set) takes on only integer values. The Markov property assumed is the following: For each flnite set of times r, < t 2 < ... < t n < tandintegersi 1,i2,···,in,i, p[S(t) = =;:
il S(td = p[S(t)
t., S(t 2) = i2, ... , S(tn) = in]
= i I S(tn) = in]'
(6.8.4)
This property might be described roughly as short memory. The conditional probabilities in (6.8.4) must be considered in modeling, but in applications it is usually the functions Xi(t)
= p[S(t) = i]
(6.8.5)
that are of interest. Differential equations for these functions can be derived for problems in which it can be assumed that (a) p[S(t + L\) = i ISet) = j] = aiit)L\ + O(L\2), (i =1= j) and (b) p[Set + L\) = i ISet) = i] = 1 - aii(t)L\ + O(L\ 2)for all i,j and small L\ > O.
6.8
133
Continuous Time, Discrete State Markov Processes
The coefficient functions a;/t) must be selected to model the specifics of individual problems. The only restraints upon them a;;(t) =
L ait),
j>#i
(6.8.6)
.
provide the probabilistic interpretation that aij(t).::\ is the probability that starting at time t the state makes a transition from state j to state i during an interval of length .::\. The probability that more than one such transition occurs during that interval is assumed to be 0(.::\2), which is to say, (1/.::\)0(.::\2)--+-0 as .::\--+-0. In numerous problems the aij(t) are independent oft.
DERIVATION OF THE DIFFERENTIAL EQUATIONS FOR THE x,(tl
By breaking up the eventS(t + .::\) = i into the conditional events S(t + .::\) = i IS(t) = j (j = 0, ± 1, ... ) and applying (6.8.1), (6.8.3), and (6.8.4), it is
easy to see that p[S(t
+ .::\) = i] =
p[S(t
+L
j>#i
+ .::\) = i I S(t) = i]P[S(t) = i] p[S(t + .::\) = i IS(t) = j]p[S(t) = j].
(6.8.7)
The terms involving P in (6.8.7) can be eliminated by substitution, using (6.8.5) and assumptions (a) and (b), with the result being xi(t
+ .::\)
=
[1 - a;;(t).::\]X;(t) +
L a;/t)x/t).::\ + 0(.::\2).
(6.8.8)
j'i'i
By subtracting x;(t) from both sides of (6.8.8), dividing by .::\, and letting .::\--+- 0, we obtain the differential equations x:;(t) = -a;;(t)x;(t)
+
L a;/t)x/t).
(6.8.9)
j>#;
In summary, we conclude that in applications for which the Markov property (6.8.4) and assumptions (a) and (b) are appropriate, the probabilities (6.8.5) satisfy linear homogeneous equations of the special type (6.8.9). Conversely, a linear homogeneous system x = A(t)x in which A;;(t) = -aii(t) and A;/t) = aij(t) for i 1= j with the aij(t) satisfying (6.8.6) can be given a probabilistic interpretation. The matrix A(t) associated with (6.8.9) need not be offinite size. In problems in which S(t) can take on only a finite number of values with nonzero probability, A(t) can be taken to be finite. Thus the modeling problem is reduced to determining the aij(t). They are usually quite readily obtained from the specific problem considered. Example 6.8.1 Simple Epidemic Model. In a population of size N, assume that there is a probability of ex.::\ that one contact will occur between two in-
134
6.
General Linear Equations with Time-Varying Coefficients
dividuals in the population during time A and that the probability that more than one contact will occur during that time is relatively negligible. Which two individuals meet is determined by drawing them at random from the entire population. An epidemic starts with one infected. person in the population. Whenever an infected person and a noninfected person have a contact, there is a probability p that the noninfected person will become infected. Once a person becomes infected, he stays infected. The problem is to determine the aij(t)swhere S(t) is the number of infected individuals at time t. The solutions to the resultant equations (6.8.9) are then to be computed and examined. In general, the approach to finding the aij(t)sis through determination of what is called the infinitesimal transition scheme; namely, select an arbitrary state j , time t, and interval length A , then consider the states to which a transition is possible, and compute the probabilities of each. As mentioned above, this will determine the aij(t)s. For the problem under consideration, the possible state values (with positive probability) are restricted to j = 1, 2, . . ., N , since initially one individual is infected and no more than the entire population N can become infected. The parameters of the problem p, a, and N are time independent. The transitions that can occur from an initial state j and their respective probabilities can easily be computed with the resultant infinitesimal transition scheme: Transit ions
j
=
Probabilities
1
N-tj
In view of assumptions (a) and (b), which define the aij(t),it is a simple matter to read off from the infinitesimal transition scheme
aij(t)=
jfN-j)
2pa N(N - 1)
lo
for
1 < j < N - 1, i = j ,
(6.8.10)
otherwise
( j = 1, 2, . . ., N ; i = 1, 2, . . ., N ) . The last part of the problem being considered will be solved only for a
6.8
Continuous Time, Discrete State Markov Processes
135
population with N = 3. In this case (6.8.6) and (6.8.10) produce the coefficient matrix (6.8.1 1)
of the system 1 = A x having x e B 3 where it is kept in mind that xi(t) = P [ S ( t ) = i] = P[number infected = i] (i = 1, 2, 3). Since initially only one individual is infected, the initial condition is x,(O) = 1, x2(0) = 0, x3(0) = 0. The ABC formula can be employed to obtain the solution ~ ( t=) [x', AX', A 2 x 0 ] [ y o ,By', B 2 y 0 ] - l y ( t ) .
(6.8.12)
Clearly the eigenvalues of A are Iz = 0, - $pa,- $pa to which is applied the correspondence rule, yielding
This in turn determines yo
=
[;I,
(6.8.14)
and (6.8.15)
With
this provides all the data needed for (6.8.12), which can be computed with
136
6.
General Linear Equations with Time-Varying Coefficients
the result that Xl(t) = exp( -
2~rx
t),
(6.8.16)
(23Prx t) ,
2 X2(t) = 3Prxt exp -
X3(t)
=
1 - exp( -
2~rxt)
-
(6.8.17)
2~rxtexp(
_
2~rxt).
(6.8.18)
Notice that Xj(t) ~ 0 (i = 1,2,3) and xl(t) + X2(t) + X3(t) = 1, as is required of probabilities. Initially Xl(t) is one and X2(t), X3(t) are zero; as t -- 00, Xl(t) monotonically decreases to zero and x 2 (t) builds up for a short time and then converges to zero, as does x 3 (t). This behavior is compatible with the epidemiological interpretation of the problem. The manner in which the parameters rx and P enter the solution also agrees with one's intuition. Remark 6.8.1 The special stochastic processes touched upon in this section of the text (Markov processes) play an important role in diverse fields of application, including birth-death processes and queueing theory. A number of elementary texts such as [2J are available for those interested in learning more about the subject.
EXERCISES
1. Verify that if the conditions' xj(t) ~ 0, L Xj(t) = 1 are satisfied by a solution of the system (6.8.9) at t = to, then they continue to hold for to < t. 2. Compute the explicit solutions to the differential equations (6.8.9) for Example 6.8.1 when N = 2 and N = 4. Correlate the behavior of the solutions with the epidemiological interpretation of the equations. 3. A head of household repairs the home's single dishwasher whenever it breaks down. Assume that the time required before the machine fails again after being repaired each.time is exponentially distributed with rate parameter A. That is, the probability that the working machine will continue to work for at least t more units of time is e-.l.t. Further assume that when the machine fails, the time required to repair it also is exponentially distributed but with rate parameter JI.. Let the state S(t) of the system be the number of machines broken down at time t (i.e.,zero or one). Determine the infinitesimal transition scheme for the problem. Then compute the probability that the machine is operating at time t if it is known to be operating at time zero. 4. In a chemical reaction a molecule of a substance A can react with a
6.8 Continuous Time, Discrete State Markov Processes
137
molecule of a substance B to form a molecule of a compound C. Assume that during a time interval of length L\ the probability that a molecule of A and one of B are in close enough proximity for the reaction to take place is proportional to the product of the numbers of molecules of type A and type B present. Assume that initially there are N molecules of each type, A and B, present. Let the state be the number of molecules of type A left at time t. Write out the infinitesimal transition scheme. Compute the probabilities (6.8.5)for the problem if N = 4. What procedure could be taken if N = loo? N = 108? 5. In a store with one cashier, let j be the number of customers waiting in line (this includes the customer currently being served). Assume that in time L\ the probability of the arrival of a new customer is 2L\. The arrival of more than one customer during that time is assumed to have relatively negligible probability. Similarly, assume that the customer currently being served has probability PL\ of finishing during time interval L\. Assume that whether or not a customer arrives is independent of whether or not service is completed for the customer presently being served. Determine the infinitesimal transition scheme if the state S(t) is taken to be the number of customers waiting in line at time t.
Chapter
7
Commutative Linear Differential Equations
In this chapter, we define and study a class of linear time-varying differential equations (including the constant-coefficient ones) that we call commutative equations. (Another such intermediate class, periodic equations, is dealt with in Chapter 8.) Commutative equations are shown to be solvable in closed form, using the same operations as those required for the constant-coefficient case. Thus the constant-coefficient theory of Chapters 3-5 is found to have an impact well beyond the original motivating applications. The judicious restriction of the class of equations studied permits refinement ofthe general results obtained in Chapter 6 and thereby leads to a more detailed picture of solution behavior.
7.1 Introductory Comments
Linear constant-coefficient equations constitute an important but relatively meager subset of the full class of differential equations with variable coefficients discussed in Chapter 6. It would prove fruitless to seek a method for generating the solutions to the extended class of equations in terms of elementary functions using the finite operations employed in the constant-coefficient situation. More often than not, differential equations themselves, in conjunction with the existence and uniqueness theory, are used to define the functions recurring in applied mathematics. A few special equations, such as Euler's equation, have been solved in closed form. However, a number of those equations are simply constant-coefficient equations disguised by transformations of variables. 138
7.2
139
The Definition and an Example of Commutative Equations
When the elements of the coefficient matrix are analytic in the region of interest, power series methods are sometimes useful. Of course a series is a limit, as opposed to a finite process such as solving a system of linear algebraic equations. If limit processes were to be admitted in computation, then it should be recalled that the existence proof of the previous chapter has already provided a series uniformly approximating the solutions over compact intervals, and, moreover, it requires only local integrability of the coefficient matrix. It becomes apparent, upon leaving the realm of constant-coefficient equations, that the concept of closed-form solution requires further development. One step in that direction is taken in the next section, in which commutative time-varying systems are defined and analyzed. This class of equations significantly extends the constant-coefficient class while continuing to allow construction of the solutions using, basically, only the operations of solving systems of linear algebraic equations and finding roots of polynomials-operations already required for constant-coefficient equations.
7.2
The Definition and an Example of Commutative Equations
Definition Let A(') be a locally integrable n x n matrix-valued function over fF, defined on an open interval J c r!lt. Call A(') commutative on J if there exists an absolutely continuous matrix function B(') on J such that
dB(~) and moreover
dt
= A(t),
A(t)B(t) - B(t)A(t)
a.e. t
=
E
J,
a.e. t E J.
0,
(7.2.1)
(7.2.2)
Such a B( .) is called a commuting antiderivative ofA( .). A differential equation dx at = A(t)x,
(t, x)
E
J x fFn,
(7.2.3)
is called commutative if its coefficient matrix A(') is commutative on J.
Remark 7.2.1 If A(') is commutative on J, then it has more than one commuting antiderivative. This is apparent on noting that the difference between any two such antiderivatives is a constant matrix C for which CA(t) - A(t)C = 0, a.e. t E J. If B(') is a commuting antiderivative for A('), then so is B(') + C where C is any constant matrix (such as In) that satisfies the latter equation.
140
7.
Commutative Linear Differential Equations
Remark 7.2.2 All constant-coefficient equations are commutative. This is demonstrated by the choice B(t) = tA, t E J, for those A(' ) independent of t. Example 7.2.1
The matrix function A(t)
= [ c~s t sin tJ -sm t
cos t
(7.2.4)
is commutative on J = f71. The reader should verify this by selecting an appropriate definite integral of A(') as a commuting antiderivative. The commutative differential equation (7.2.3) associated with (7.2.4) is explicitly solved in the next section to demonstrate the applicability of the theory. The reader might wish to attempt solving the problem by his own devices before reading further. EXERCISES
1. Show that if A(') is commutative on J, then so are the following:
A T( . ), PA(' )p-l, P a nonsingular matrix. (c) CA('), C a constant matrix commuting with A(t), a.e. t E J. 2. Show that if A(') is commutative on J with commuting antiderivative B( '), then A(t) = A(t)B(t) is likewise commutative on J. (Hint: Consider (a) (b)
B(t)
= tB 2(t).)
7.3
Computation of Closed-Form Solutions to Linear Commutative Equations
Theorem 7.3.1
If A(') is commutative on J, then (t, T) E J x J,
(7.3.1)
in which B(') is any commuting antiderivative of A(·). Proof.
The proof commences with an inductive proof of the equations d dt B"(t)
=
kA(t)B"-l(t),
(7.3.2)
a.e. t E J (k = 1,2, ...). By assumption (7.2.1), Eq. (7.3.2) is valid for k = 1. Now assume that (7.3.2) holds for fixed k E {I, 2, ...}. Since B(' ) is assumed to be absolutely continuous, application of (7.2.1), (7.2.2), (7.3.2), and the
141
7.3 Computation of Solutions to Linear Commutative Equations
product rule of differentiation yields
d
d
+ A(t)Bk(t)
dt B"+ let) = B(t) dt Bk(t)
= B(t)kA(t)Bk-l(t)
= (k +
+ A(t)B~t) a.e.
1)A(t)B~t),
t E J.
(7.3.3)
This concludes the proof of (7.3.2) by induction. Since the exponential function is an entire function and (7.2.1) holds at a.e. t E J, it follows that it is possible to do term-by-term differentiation of the series eB(t) =
f
B"(t),
(7.3.4)
k=O k!
using (7.3.2) to get
~
dt
eB(t)
=
f.! ~
k= 1 k! dt 00
= k~l (k -
Bk(t)
1
00
1)! A(t)Bk-l(t)
= A(t)eB(t),
a.e.
1
= A(t) k~O k! B~t)
t E J.
(7.3.5)
From (7.3.5)it readily follows that the matrix function defined by the product in (7.3.1) satisfies (6.3.8) of Theorem 6.3.2 and clearly (6.3.9) as well. Thus the proof of Theorem 7.3.1is completed by appealing to the uniqueness statement in Theorem 6.3.2 regarding Eqs. (6.3.8), (6.3.9).
Theorem 7.3.2 If A(') is locally integrable on J, then it is commutative there
if and only if for
each fixed
BoA(t) - A(t)B o
't" E J
= A(t)
f
there exists a matrix B o over ff such that A(O") do -
f
A(O") do A(t), a.e. t E J. (7.3.6)
(Generally the choice of B o depends on r.) Proof.
Assume that (7.3.6) holds for some B(t) =
s,
+
f
A(O") da,
't"E
J and B o over ff. Define
t E J.
(7.3.7)
From (7.3.6), (7.3.7), and the assumed local integrability of A(') it follows that (7.2.1) and (7.2.2) hold for the B(') defined in (7.3.7). Hence A(') is commutative on J. Conversely, if A(') is commutative on J, then there exists an
142
7.
Commutative Linear Differential Equations
absolutely continuous matrix function B(') satisfying (7.2.1) and (7.2.2). Moreover, B(') must satisfy (7.3.7) for each r E 1 when one takes B o = B(r). Substitution of (7.3.7) into (7.2.2) leads directly to (7.3.6), and the proof is complete. Remark 7.3.1 The characterization of commutative matrix functions given by Theorem 7.3.2 provides a test for commutivity that is sometimes useful in applications. (Other criteria will appear later in this chapter.) Note that computing B o is a .problem of solving a system of linear algebraic equations. Such a rand B o could be used in the formula B(t)
= Bo +
f
(7.3.8)
A(s) ds
to produce an antiderivative B(') as required for computation of A via formula (7.3.1) if A(') turns out to be commutative. Remark 7.3.2 If A(') is commutative on 1 and a commuting antiderivative B(') is given or else computed according to Remark 7.3.1, then the matrix function A providing the general solution to (7.2.3) can be computed by solving a linear differential equation with constant-coefficient matrix containing a parameter, namely, (7.3.9)
A(t, r) = 'P(1, t)'P( -1, r)
in which 'P(a, t) is the unique solution to the initial-value problem
aaa 'P
=
B(t)'P,
(7.3.10)
a E [ -1,1],
for each fixed value of the parameter t e J, (The continuous function 'P that deforms the initial matrix In into the desired function 'P(l, r) = eB(t) as a is varied from to 1 is an example of what in topology is called a homotopy.)
°
The applicability of Theorems 7.3.1 and 7.3.2 is demonstrated by the following example. Example 7.3.1 The problem considered is that of computing the closedform solution to the equation
:t[:J
=
[-:~~:
;~: J[: J
teJ
=~,
(7.3.11)
associated with the time-varying matrix (7.2.4). For this problem, Eq. (7.3.6) reduces to BoA(t) - A(t)B o = 0,
(7.3.12)
7.3
143
Computation of Solutions to Linear Commutative Equations
a.e. t E &t, for each, E &t. Hence Theorem 7.3.2 substantiates the commutivity of(7.3.11) since (7.3.12) is satisfied by B o = 0 or In, for example. Thus choose , = 0, B o = O. Equation (7.3.8) is then applied to compute the appropriate commuting antiderivative B(t) =
rJot A(u) do = [ cost-1 sin t
1 -. cos tJ
(7.3.13)
smt
to be used in (7.3.10). Pursuing the solution of (7.3.10) by the procedure of Chapters 3 and 4 requires computing the eigenvalues of B(t), which are
A.
sin t
=
± i(l
- cos t).
(7.3.14)
As long as 1 - cos t i= 0, the correspondence rule gives y(u)
e lf sin t =;
[
. t elf Sin
cos[u(1 - cos t)]J • [ sin u(1 - cos t)] ,
(7.3.15)
which is differentiated relative to a to produce sin t [ 1 - cos t
cos .t - 1J y (a) = By (o) sin t
(7.3.16)
(where B is the matrix of the ABC method and is not to be confused with B(t)). According to Theorem 4.6.1, elfB(t) = (A l y(o'), A 2y(u)),
(7.3.17)
in which Ai = (e, B(t)e;)(y(O), By(O))-l
(i = 1,2).
(7.3.18)
With B(') computed in (7.3.13), B in (7.3.16), and y(.) by (7.3.15), the Ai in (7.3.18) are computed by elementary matrix calculations and are substituted into (7.3.17) with the result that lfB(t) = e
elfsint [
cos[er(I - cos t)] - sin u(1 - cos t)
[
]
tr
sinEu(1 - cos t)]J cos u(1 - cos t)
[
(7.3.19)
Note that (7.3.19)remains valid for 1 - cos t = 0 by continuity. Substitution of (7.3.19) into (7.3.1) finally provides the fundamental matrix cI> A(t, r) =
e sint - sinr
[c~S( cos
t - COS,) sm(cos t - cos r)
- sinecos t - cos r) J . (7.3.20) cos(cos t - cos r)
In particular, cI>A furnishes the solution to (7.3.11), satisfying Xl(') = x 2(r) = ~2: X1J = [ X2
esint-sint
= [~l ~l
~l'
c~s(cos t - cos r) - ~2 sin(cos t - cos r)J. (7.3.21) sm(cos t - cos r) + ~2 cos(cos t - cos r) (7.3.22)
144
7.
Commutative linear Differential Equations
EXERCISES
1. Verify that the following system is commutative on qj by application of Theorem 7.3.2, and determine a commuting antiderivative B(') of its coefficient matrix A(·). X2
=
+ (1 + 2t)X2'
-tx 1
2. Check the commutivity of the following system on qj and then compute its general solution by the, method discussed in Section 7.3.
3.
Consider the matrix function of qj,
A(t)
O1J {[1 0'
t
~ [~ ~J
irrational, rational.
Find an appropriate commuting antiderivative to demonstrate that A(') is commutative on qj. 4. The matrix function B(t) = sin 2t [
(0. ) sgn sin t
olJ
is absolutely continuous on qj. Why? Compute its derivative A( .) and confirm the commutivity of A(') on qj. Find those (t, r) E qj2 for which A(t)A(-r) - A(-r)A(t) "# 0.
5. Show that if A(') is commutative on J with commuting antiderivative B( '), then for each -r E J, B(-r)A(t) - A(t)B(-r) = A(t)
f
A(u) do -
f
A(u) da A(t),
a.e. t E J. Apply this result to conclude that the equality A(t)
t
A(lT) do
=
fo~(U)
do A(t),
a.e. t E qj, fails for the commutative system of Exercise 4.
145
7.4 Some Sufficient Conditions for Commutivity
7.4
Some Sufficient Conditions for Commutivity
The results of this section provide easy verification of commutivity where they apply.
Theorem 7.4.1 A locally integrable n x n matrix-valued function A(') on J is
if either of the following equivalent conditions are A(t)A(r) = A(r)A(t), a.e. t E J for a.e. r E J, A(t) J: A(O") da = J: A(O") da A(t) for a.e. t E J, all r E J.
commutative
(a) (b)
Proof.
met:
Clearly, (a) implies (b). Now assume (b), which is to say that A(t)
f
A(O") da -
f
A(O") do A(t)
=0
(7.4.1)
for a.e. t E J, all r E J. At each point t E J at which (7.4.1)holds, the left-hand side is an absolutely continuous function of r: therefore the equation can be differentiated relative to r and (a) results. Thus (a) and (b) are equivalent. If (b) is assumed, then, upon setting Bo = 0 in (7.3.6), Theorem 7.3.2 applies and A(') must be commutative. This concludes the proof.
Corollary 7.4.2 Let A o be any n x n matrix over CC and f(z) be a power series centered at the origin, as in Theorem 4.2.2, with radius of convergence p, If Ao is nilpotent, then the formula (7.4.2)
A(t) = f(tA o)
defines a matrix function A(') that is commutative on 9f. If P > 0, then (7.4.2) determines a commutative matrix function on the interval - plr < t < otr in which r is the maximum absolute value of the eigenvalues of A o.
Example 7.4.1 According to Corollary 7.4.2, A(t) =
ao
f'
L: I" A~ k.
= e tAo,
t
E
9f,
(7.4.3)
k=O
is commutative fo.r any n x n matrix Ao over CC. The commutivity of Example 7.2.1 is now transparent when it is recognized to be of type (7.4.3) with Ao the matrix [; ~ A].
Example 7.4.2 series is
Another commutative matrix function defined by a power ao
A(t) =
L:
k=O
t
E (
f' A~
= (In -
tA o)- t,
(7.4.4)
-llr, l/r), where Ao is any n x n matrix over CC whose eigenvalues
2 each satisfy 121
< r.
146
7.
Commutative Linear Differential Equations
If, in lieu of the series in (7.4.4), the inverse matrix in that equation is taken as the defining formula, then the interval can sometimes be extended. In fact, for A o having no real eigenvalues, commutivity occurs on (- 00, (0), and if OCl < OC2 < ... < oc, are the reciprocals of the real nonzero eigenvalues of a matrix A o over re, then intervals over which commutivity occurs can obviously be taken to be anyone of (- 00, ocd, (OCl' O(2), ••. , (OC'-l' oc,), (OC., (0). Although the inverse is defined for a.e. t E ~, the OCi must be excluded from the domain of A(') to maintain the local integrability. A more complete description of the manner in which analytic functions can be used in constructing commutative A(') is given in Section 7.5. Theorem 7.4.3 Let {Ak}O' be a commuting family ofn x n matrices over re and {OCk(')} 0' be a sequence of complex-valued measurable functions defined on an open interval J. If the series CXJ
L
(7.4.5)
locit)IIA k !
k~O
is convergent for a.e. t
E
J and is locally integrable, then the series
A(t)
converges for a.e. t
E
=
CXJ
L
k=O
(7.4.6)
ocit)A k
J and defines a commutative matrix function A(') on J.
Proof. The assumed convergence and local integrability of (7.4.5) imply the same for (7.4.6). Equation (a) of Theorem 7.4.1 follows from writing A(t) and A('l:) as series and applying the commutivity of the Aks. Thus the conclusion of Theorem 7.4.3 follows as a corollary to Theorem 7.4.1.
Series that are not necessarily power series serve as useful devices for producing commutative matrix functions. One example that illustrates the point is the series
Example 7.4.3
•
A(t) =
L CXJ
k~
(1 +-
e- kt [sin ktl I
1
k
)k
Ao ,
t E (0, (0), which clearly satisfies the hypothesis of Theorem 7.4.3. EXERCISES
1.
Consider any matrix function A(') of the type 00
A(t) =
L k~O
ocit)A k,
t
E
[0,
(0),
(7.4.7)
7.4
147
Some Sufficient Conditions for Commutivity
in which the n x n matrices A o' At> A 2 , .•. satisfy AiA j = AjA i for all i,j and the scalar-valued ak(') are locally integrable. (a) Show that if
k~O Loo lak(a) I do IAk ! <00 for some k o E {O, 1,2, ... }, then A(') is commutative on (0, 00). (b) Show that if A o is a stability matrix and as
tao(a)da-oo
t-oo
with
k~lLoolak(a)ldaIAkl
then all solutions to x(t) = A(t)x(t) satisfy lim, ... co x(t) = O. 2. Apply Exercise 1 to prove that all solutions to the system
approach zero as t written as
1: t[ -
3.
Note that the coefficient matrix can be
Hint:
00.
~ - ~] + 1 ~
t2 [
-
~ ~ -
J
Apply Theorem 7.4.1 to prove that A(t) = (tin - A o)-l(tIn + A o)
is commutative on an interval J if the n x n matrix A o has no eigenvalues in J. 4. Investigate the behavior of the solutions x(t) as t - 00 for the system x = A(t)x in which (a)
A(t)
= Lk=o e- kr cos kt At+ 1 with
~ ~],
Ao = (b)
A(t)
-2 -2
= Lk=O cos kt At+ 1 in which Ao
= [
~
0.1
J
0 0.1. -0.1 -0.2 -0.2
o
148
5.
7. Commutative linear Differential Equations
t
Find the closed-form solution to the equation A(t) = A o
when (a)
7.5
Ao = [ _
~ ~
J
(b)
Ao =
x=
A(t)x in which
eO"Ao do
G~l
Matrix Functions and Operations Preserving Commutivity
It is generally not a trivial matter to determine whether or not a given matrix function is commutative. Hence the extent of the class of commutative systems is of interest. One approach is to study the class of all commuting antiderivatives, i.e., all absolutely continuous n x n matrix solutions B(') to the equation B(t)B(t) - B(t)B(t)
=
0,
(7.5.1)
a.e. t E J. Their first derivatives produce all commutative A(' )s. The extent of the solutions to (7.5.1) is illuminated by finding operations that preserve its solutions. For example, all functions of the type B(t)
=
PA(t)P- 1
(7.5.2)
in which P is a constant n x n nonsingular matrix and in which the diagonal matrix A(t) = diag[ A,l(t),A,2(t), ... , A,n(t)] has the A,i(') absolutely continuous on J are solutions since A(·) is obviously a solution and the operation B(') - PB(,)p-l preserves solutions to (7.5.1). In Section 7.4, we have already indicated that functions of matrices are useful in constructing examples of commutative systems. Proper treatment of the subject draws on the elementary theory of functions of a complex variable. That theory provides an elegant solution to a problem that began to emerge in Example 7.4.2, namely, that of defining a function f(A) of a matrix A for which some eigenvalues of A fall outside the open disk of convergence for the power series defining fez). The matrix form of Cauchy's integral formula is now stated along with one of its corollaries. (For further results and proofs see [22]')
rc
Theorem 7.5.1 Let A be an n x n matrix over and consider any complexvalued function fez) analytic in an open set Q containing the collection a(A) of all distinct eigenvalues of A. There exists an open set w whose closure iiJ
149
7.5 Matrix Functions and Operations Preserving Commutivity
satifies wen and whose boundary aw consists of a finite number of simple closed curves that do not intersect and such that u(A) c co. A matrix f(A) is defined by the line integral f(A) =
2~ i. tu
j~w
f(z)(zI - A)-l dz,
(7.5.3)
in which the path of integration is oriented in the standard way. The integral (7.5.3) is otherwise independent of the choice of wand agrees with the power series definition off(A). Corollary 7.5.2 If fez) and g(z) are analytic in an open set containing u(A) and h(z) = f(z)g(z), then h(A) = f(A)g(A).
The Cauchy integral formula is a useful device in the study of commuting matrix functions. Theorem 7.5.3 Let fez) be analytic on an open set nand B(') be an absolutely continuous solution to (7.5.1) on the open interval J. If
u(B(t)) c
n
(7.5.4)
for all t E J, then B( .) = f(B(')) is also a solution that is moreover a commuting antiderivative of A defined as A(t) = !,(B(t))B(t),
t E
J,
(7.5.5)
where the prime denotes the complex derivative. Proof. Fix any t E J. The Cauchy integral formula as described in Theorem 7.5.1 provides the equation
1
f(B(t)) = -2. 1tl
i.j~Wt
f(z) [ zI - B(t)] -1 dz.
(7.5.6)
Generally the open set co, depends on t, but since B( .) is continuous and W t is open, co, can be held fixed for small changes in t. The analyticity of fez) and absolute continuity of B(') then provide sufficient conditions for application of a standard theorem for taking the derivative of (7.5.6) past the integration. The differentiation, aided by (7.5.1), proceeds as 1
i.
d f(B(t)) = -2. dt 1tl j~Wt
= -2 i. 1. 1tl
j~Wt
f(z)[ zI - B(t)] -1 B(t)[zI - B(t)] -1 dz
f(z)[zI - B(t)]-2 dz B(t),
(7.5.7)
a.e. t E J. As with Cauchy's formula for a scalar variable, the integral in the
150
7.
Commutative Linear Differential Equations
last term of (7.5.7) can be written as f'(B(t». The equation
d -
dt B(t)
.
= f'(B(t»B(t),
(7.5.8)
a.e. t E J, is thereby derived. It can be noted in (7.5.7) that f'(B(t» commutes with B(t) since B(t) commutes with B(t), a.e. t E J, and consequently
BB - BB =
f(B)f'(B)B - f'(B)Bf(B)
= 0,
(7.5.9)
a.e. t E J. The absolute continuity of B(') follows from (7.5.8) and the proof is complete. Corollary 7.5.4 Assume that B(') is a commuting antiderivative on J. Then the classical adjoint B*(') inherits the same property. IfB(t) is moreover nonsingular for each t E J, then B- 1( . ) is likewise a commuting antiderivative on J. Proof. By choosing fez) = liz, it follows directly from Theorem 7.5.3 that if B( .) is a commuting antiderivative on J with B(t) nonsingular for each t E J, then B- 1( . ) is a commuting antiderivative on J. To prove the remaining conclusion, recall that the classical adjoint of a matrix B satisfies
(7.5.10) where jBj is the determinant of B. Now assume that B(-) is a commuting antiderivative on J and thus B(t)B'(t)· - B'(t)B(t)
= 0,
(7.5.11)
a.e. t E J. First consider the case in which B(t) is nonsingular at each t E J. Multiplication of (7.5.11) on the left and right by B*(t) and application of (7.5.10) lead to (7.5.12) a.e. t
E
J, and hence to
B'(t)B*(t) - B*(t)B'(t)
a.e. t
E
E
+ B~(t)B(t)
E
- B(t)B~(t)
- B'(t)B.(t)
= 0,
(7.5.14)
J, and in view of (7.5.13) this is the same as B~(t)B(t)
a.e. t
(7.5.13)
J. Differentiation of (7.5.10) gives
B*(t)B'(t)
a.e. t
= 0,
- B(t)B~(t)
= 0,
(7.5.15)
J. Multiplication of (7.5.15) on the left and right by B. and a final
7.5
151
Matrix Functions and Operations Preserving Commutivity
application of (7.5.10) show that
= 0,
- B~(t)B*(t)]
IB(t)I[B*(t)B~(t)
(7.5.16)
and thus B*(t)B~(t)
-
B~(t)B*(t)
=
0,
(7.5.17)
a.e. t E J. All that remains to be done is to show that (7.5.17) would likewise hold at points where IB(t)1 = o. Suppose that IB(r)1 =:= 0 at a point r E J at which the equation in (7.5.11) holds. Then by the continuity of B(' ), for each f. > 0 appropriately small the matrix function Bi«, t) = B(t) + el; will satisfy IB(f., t)1 #- 0 on some subinterval J, of J containing r. But a trivial calculation shows that B(e, t )
dB(f., t) _ dB(f., t) ( )-0 dt dt B f., t - ,
(7.5.18)
a.e. t E J" including the point t = r. It follows from what has already been proved that (7.5.19) a.e. t E J" including the point t = r, Now in view of the fact that the elements of B*(f., t) are polynomials in both f. and the elements of B(t) it is possible to let f. -+ 0 in (7.5.19), with the result that at least at t = r, (7.5.20) Since the argument applies to a.e. r
E
J, as restricted, the proof is complete.
Remark 7.5.1 A matrix function B(') on J is called analytic if it has a convergent power series
B(t) =
Lco k=O
(t
- rl B(k)(r) k!
(7.5.21)
about each point r ~ J. If B(') in Theorem 7.5.3 is analytic, then f(B(')) will likewise be analytic on J. Example 7.5.1 If fez) is any entire function such as eZ , sin z, any polynomial p(z), p(z)eZ , etc., then e", sin B, pCB), p(B)eB, etc., will be commuting antiderivatives of the consequent commutative matrix functions eBB, B cos B, p'(B)B, [pCB) + p'(B)]eBB, etc., for BO any commuting antiderivative. One allowable choice of B( .) would be B(t) = tAo, where A o is any n x n matrix. Theorem 7.5.5
Let B o be any n x n matrix over CC and f(t, z) be defined and continuous on J x E, where J is an open interval and E => a(B o) is an open
152
7.
Commutative linear Differential Equations
subset of~. Further assume that f(t, z) is absolutely continuous relative to t for each fixed z E E and that bothf(t, z) andfl'/, z) are analytic in zfor each fixed t E J. Then the matrix function B( .) defined as B(t) = f(t, B o) is a commuting antiderivative on J and moreover, BoB(t) - B(t)B o = 0, Proof.
t
E
(7.5.22)
J.
The assumptions allow application of the Cauchy integral formula 1 B(t) = -2. tu
1.
jll"',
f(t, z)(zI - BO)-l dz,
t E J.
(7.5.23)
The details are left as an exercise. Corollary 7.5.6 Iff(z) is analytic on an open set nand A o is an n x n matrix over ~ for which a(tA o) c: for all t
E
n
(7.5.24)
J, J an open interval, then the matrix function
A(t) = f(tA o)
(7.5.25)
as well as its derivatives A(k)(t) =
Pk)(tAo)A~
(7.5.26)
(k = 1, 2, ...) are analytic commutative matrix functions on J. Proof.
Apply Theorem 7.5.3 to verify condition (a) of Theorem 7.4.1.
Theorem 7.5.7 Let A o be any n -x n matrix over ~ and f(t, z) be defined on J x E where J is an open interval and E::J a(A o) is an open subset of~. Assume thatf(t, z) is locally integrable relative to t for fixed z and analytic in z for a.e. fixed t E J. Then the matrix function A('), defined as A(t) = f(t, A o), is commutative on J. Proof.
The assumptions allow application of the Cauchy integral formula A(t)
= -2. 1 1
1tl
jll"',
f(t, z)(zI - AO)-l dz,
(7.5.27)
The details are omitted. The next theorem is another example of the many interrelations between linear and nonlinear differential equations and linear algebra.
Theorem 7.5.8 Let B( .) be an n x n matrix-valued commuting antiderivative over
~
on an open interval J. If the characteristic polynomial of B(t),
(7.5.28)
153
7.5 Matrix Functions and Operations Preserving Commutivity
has distinct roots at a.e. t
E
J, then B(') satisfies the differential equation
a.e.
t
E
J.
(7.5.29)
(Subscripts denote partial derivatives.) Conversely, for every monic polynomial pet, 2) - over ((5 with absolutely continuous coefficients, each n x n matrix solution B(') to (7.5.29) will be a commuting antiderivative. If pet, 2) annihilates B(t) and has distinct roots at a.e. t E J, then the roots of the minimal polynomial of B(t) are also distinct at a.e. t E J and the eigenvalues of B(t) are roots of pet, 2) for a.e. t E J. Proof. If B(') is an n x n absolutely continuous matrix function over ((5, then its characteristic polynomial (7.5.28) will obviously have absolutely continuous coefficients. By the Cayley-Hamilton theorem, pet, B(t))
=
t
0,
E
(7.5.30)
J,
which can be differentiated, and when B(') is a commuting antiderivative, that differentiated equation will be of the form pit, B)B
+ pit, B)
=
0,
a.e.
t E J.
(7.5.31)
By a standard theorem of algebra the assumption that p(t,2) has distinct roots implies that pit, 2) #- 0 at those 2 in ((5 where pet, 2) = O. Hence since a(p(t, B)) = p(a(t, B)), the matrix pit, B(t)) is nonsingular at a.e. t E J. This allows (7.5.31) to be rewritten as .
B
=
-pit, B)
-1
a.e.
pit, B),
t
E
J.
(7.5.32)
To prove the last half of the theorem, let pet, 2) be a monic polynomial over absolutely continuous coefficients. Then the right-hand side of Eq. (7.5.29) is defined. Now let B(') be an absolutely continuous solution to that equation. (The existence and uniqueness problems for such nonlinear equations are treated later in the text and are of no direct concern here.) Since B(t) commutes with the right-hand side of(7.5.29), it satisfies (7.5.1) at a.e. t E J and hence is a commuting antiderivative. Suppose that ((5 with
per, B(r)) = 0
at some r
E
(7.5.33)
J. Since B(') is assumed to satisfy (7.5.29),differentiation leads to
d dt pet, B(t))
=
pit, B)
.
+ pit, B)B
= p,(t, B) - pit, B)p;: let, B)p,(t, B) a.e. t E J. = 0,
(7.5.34)
154
7.
Commutative Linear Differential Equations
The only continuous function satisfying (7.5.33), (7.5.34) is pet, B(t)) = 0,
all
t
E
(7.5.35)
J.
If (7.5.35) holds and pet, A) has distinct roots at a.e. t E J, then the minimal polynomial of B(t) divides pet, A) at each t E J, and hence that minimal polynomial must have distinct roots at a.e. t E J. Obviously the eigenvalues of B(t) are roots of pet, A) (with not necessarily the same multiplicity) a.e. t E J. This completes the proof. Corollary 7.5.9 Let B(') be an n x n matrix-valued commuting antiderivative over on an open interval J. If the characteristic polynomial pet, A) of B(t) has n distinct roots Ai(t) (i = 1,2, ... , n), a.e. t E J, they satisfy the scalar differential equation
rc
Z = - Pr(t, z)j pz(t, z), The eigenvalues of A(t) = B(t) are ~.lt)
a.e.
t
E
J.
(i = 1,2, ... , n), a.e. t
(7.5.36) E
J.
Proof. The conclusion that the A;(t) satisfy (7.5.36) on the intervals where they are distinct follows from differentiation of the equations pet, A;{t))
=