METHODS OF NONLINEAR ANALYSIS Volume I
This is Volume 61 in MATHEMATICS IN SCIENCE AND ENGINEERING A series of monographs and textbooks Edited by RICHARD BELLMAN, University of Southern California A complete list of the books in this series appears at the end of this volume.
METHODS OF NONLINEAR ANALYSIS Richard Bellman Departments of Mathematics, Electrical Engineering, and Medicine University of Southern California Los Angeles, California
VOLUME I
@
1970
ACADEMIC PRESS
New York and London
COPYRIGHT 0 1970,
BY
ACADEMIC PRESS,INC.
ALL RIGHTS RESERVED. NO PART OF THIS BOOK MAY BE REPRODUCED I N ANY FORM, BY PHOTOSTAT, MICROFILM, RETRIEVAL SYSTEM, OR ANY OTHER MEANS, WITHOUT WRITTEN PERMISSION FROM
THE PUBLISHERS.
ACADEMIC PRESS, INC. 1 1 1 Fifth Avenue, New York, New York 10003
United Kingdom Edition published by ACADEMIC PRESS, INC. (LONDON) LTD. Berkeley Square House, London W l X 6BA
LIBRARY OF CONGRESS CATALOG CARDNUMBER:78-91424
PRINTED IN THE UNITED STATES OF AMERICA
To EMlL SELETZ Surgeon, Sculptor, Humanitarian, and Friend
This page intentionally left blank
PREFACE
T h e demands of modern science inexorably force the mathematician to explore the nonlinear world. That it is a difficult and often humbling journey with painfully crude maps and rather primitive direction-finders cannot be gainsaid, but in return it can be asserted that it is richly rewarding. The few areas that have been so far examined with any care have been full of surprises and vastly stimulating to the imagination. There is every reason to believe from what so far has been glimpsed that many more surprises lay in store, novel phenomena which will open up undreamt of vistas for mathematics. I t is an exciting prospect in an exciting field in an exciting time. Explicit analytic solutions of nonlinear equations in terms of the familiar, well-tamed functions of analysis are not to be expected, although fortuitous and treasured examples occur here and there. Consequently, if either analytic or computational results are desired, various approximate methods must be applied. By and large, the effective solution of particular problems is an art. However, there d o exist a number of powerful procedures for solving nonlinear problems which have been slowly forged and polished over the last one hundred years. As in all intellectual areas, art combined with method is more effective than untutored art. This book is intended as an introduction to the study of certain systematic and sophisticated techniques. T h e power and versatility of these methods has been tremendously amplified by the digital computer. Even more, this new tool has motivated a careful reexamination of older methods and thus a creation of new techniques specifically designed to exploit the peculiar properties of the electronic calculator. That a good deal of mathematical ingenuity and experience is required to study significant problems with the aid of a computer hardly needs emphasizing. This volunie may also be regarded as a contribution to a new mathematical theory that is slowly emerging, a theory of closure of operations. Abstractly, the general problem may be described in the following terms. We are given the privilege of using a certain limited number of mathematical operations, such as, for example, the solution of finite systems of linear or nonlinear differential equations subject to initial conditions, or the solution of a finite system of linear algebraic equations. The task is then that of solving a particular equation, such as a partial differential equation, a two-point boundary value problem for vii
viii
PREFACE
ordinary differential equations, or an integral equation, to a specified degree of accuracy using only these algorithms. T h e study becomes still more interesting and significant if we impose a constraint on the number of operations of particular types that can be employed, or on the time that may be consumed in the overall calculation. Usually, the computational facilities available automatically impose these constraints. ‘I’he two types of operations mentioned above have been carefully singled out for explicit mention since they correspond to the two major capabilities of the digital computer in the field of analysis. That they are not guaranteed capabilities merely adds to the interest of the zest of using computers to obtain numerical rcsu I ts. We nil1 present a spectrum of methods which can be used for a variety of purposes, ranging from the derivation of a simple exponential or algebraic approximation to a sequence of algorithms of increasing complexity which require a digital computer. At the moment, our orientation, as far as large-scale computing is concerned, is toward a digital computer, which is to say a leaning toward initial value problems. As hybrid computers become more powerful and prevalent, a certain mix of methods involving two-point boundary value problems and initial value problems will occur. In general, the word “solution” must be defined operationally in terms of various technological tools available for obtaining numerical results. T h e arrival of the digital computer has already drastically changed the significance of this term “solution,” and there will be further radical changes over the next twenty-five years. T h e majority of the methods we present here can be applied to the study of partial differential equations and to the still more complex functional equations that the determined engineer and intrepid physicist are forced to face. T h e applications w ithin this broader context are naturally of greater significance than those that can be made using ordinary differential equations. Despite this, we have deliberately refrained from any excursion in force into the area of partial differential equations. In this volume, the first of two, we have discussed only ordinary differential cquations, However, since any complete separation between ordinary and partial differential equations is unnatural, we have broken this self-imposed vow in the second volume. This is particularly the case in the treatment of dynamic programming and invariant imbedding. Since w e are primarily concerned with introducing the reader to a variety of fundamental methods, we feel that there is considerable pedagogical force to keeping the setting as familiar as possible while new ideas are being introduced. Once acquainted with the concepts, the reader can readily apply them to all types of functional equations with a small amount of additional background. References will be found throughout to their utilization in the theory of partial differential equations.
PREFACE
ix
Another strong argument for using ordinary differential equations as a proving ground is that it is relatively easy to provide a number of numerical examples in this area to illustrate different methods. In addition, a large number of interesting analytic results are available as exercises. These have principally been taken from research papers. Having briefly described our overall aims, let us examine the structure of the book. T h e first three chapters contain some of the fundamental results and methods that will serve throughout both as foundation and ancillary tools. Chapter 1 discusses first- and second-order linear differential equations, subject to initial and boundary value problems, with some attention to the Riccati differential equation and a detailed study of the behavior of the physical solutions of nonlinear equations of the form
where p and q are polynomials in their arguments. This last represents a brief account of extensive work by Bore1 and Hardy, very important and useful results which are still not as well known as they should be. Some related results for the Emden-Fowler (or Fermi-Thomas) equation will be presented at the end of Chapter 4 as applications of stability theory. Throughout we have tried to preserve some sort of a careful balance between general methods and particular problems. We have constantly kept in mind the famous dictum of Hurwitz, “It is easier to generalize than particularize.” In Chapter 2, we present a brief account of basic results in algebraic aspects of matrix analysis that will be employed throughout the remainder of the book. The principal contents are the reduction of quadratic forms to canonical forms and associated variational problems and the Perron theorem for positive matrices. Chapter 3 discusses the use of matrices in the study of systems of linear differential equations with both constant and variable coefficients. It is impossible to study multidimensional problems in any meaningful fashion without matrix theory. Chapter 4 contains some basic results concerning stability theory which we will employ in subsequent chapters to validate certain methods of approximation. Following the lines initiated by PoincarC and Lyapunov, we wish to compare the solutions of
T(u) = 0
(2)
T(u) = N ( 4 ,
(3)
with those of where N(u} is “small” in some sense. The most important case is that. where T is a linear operator with the property that T(u) = 0 possesses a convenient
X
PREFACE
solution. Closely connected with this question is the problem of estimating the difference between the solution of (2) and a function w satisfying the inequality
where I/ ... / / denotes some appropriate norm. With these “back-up” results available, we can turn to our principal goal, the study of certain powerful methods of analytic and computational approximation. I n Chapter 5 , we present the Bubnov-Galerkin method, and in Chapters 7 and 8 that of Rayleigh-Ritz. Although in certain special, but important, cases the methods overlap, they are quite different conceptually and extend in different ways. T h e guiding ideas are simple, as are all fundamental mathematical ideas. As always, effort and ingenuity enter in making these procedures work in particular cases. Let us begin with a description of the Bubnov-Galerkin method. Let T(u) = 0 be the equation whose solution is desired. This is equivalent to minimizing the scalar quantity 11 T(u)ll,for any norm, over the class of admissible functions. We now introduce a closure technique by restricting u to some smaller class of functions, for example, one defined by a finite set of parameters. T h e most important example of this is that where the restricted class is defined by N IZ =
akUk k=l
9
(5)
where the uk are fixed functions and the ak are parameters. T h e infinitedimensional problem of minimizing I/ T(u)lI is then replaced by the approximating finite-dimensional problem of minimizing the function
with respect to the ak . This problem may be attacked by any of a number of techniques developed in optimization theory over the last twenty years: search techniques, gradient methods, Newton-Raphson, nonlinear programming, expansion methods, and so on. Let us note that with such methods in mind, we have deliberately refrained from any automatic use of the usual quadratic norm in the foregoing description. In the text, however, succumbing to the lure of analytic simplicity, we have considered principally quadratic functionals. More general nonlinear functionals give us an opportunity to discuss the Newton-Raphson-Kantorovich method and the use of the Lagrange expansion. Closely associated with the Galerkin method are the methods of mean-square
PREFACE
xi
approximation and differential approximation. The first is discussed at the end of Chapter 5, the second in Chapter 6 . The technique of mean-square approximation may be described in the following terms. Let T ( u ) = 0, as usual, be the original equation, and let S(v, u)
=0
(7)
be another equation, depending on the vector parameter a,which is analytically or computationally more tractable than the original equation. Thus, for example, the equation in (7) may be linear with the original equation nonlinear, or it may be a nonlinear differential equation subject to an initial value condition with the original condition linear and subject to multipoint boundary conditions. Alternatively, the original equation may contain stochastic elements, while (7) is deterministic, or conversely. The existence of analog, digital, and hybrid computers, as well as the availability of many powerful analytic theories has considerably altered the concept of “tractable.” A great deal of flexibility now exists. Many different types of mathematical models are available to treat various kinds of physical processes. We have avoided stochastic processes in this volume since a good deal of effort is required to make various useful methods rigorous. We wish to determine the parameter a so that
is small, where u is the solution of T(u) = 0 and some convenient norm is employed. Presumably, this ensures that v, the solution of (7), is close to u. This is a stability question. Interesting complications arise from the fact that u itself is unknown. There are various “bootstrap” methods that can be employed to circumvent this annoying “little detail.” Here we make brief contact with “self-consistent” methods, of such importance in modern physics. A major analytic problem is that of choosing the operator S(v, u ) in such a way that the function v preserves certain desirable properties of u. Very little is known in this area. The method of differential approximation is the following. Let R(u, b ) be a family of operators depending on a finite-dimensional vector b, and let b be chosen so that
is minimized where u is now given implicitly as the solution of T(u) = 0. We then use the solution of
R(v, 6)
=0
(10)
as an approximation to u. Once again, any discussion of the validity of this approach requires stability considerations.
xii
PREFACE
A case of particular importance is that where
with the Rk(u)differential operators. Observe that the aim of this procedure is once again closure. We want to solve complex functional equations using only the algorithms required to solve the more familiar and placid differential equations. We now turn to an entirely different type of artifice. T h e Rayleigh-Ritz method hinges upon the observation that many equations of the form T ( u ) = 0 may be viewed as the Euler equation of an associated functional J(u), By this we mean that a solution of T(u) = 0 is a stationary point for J(u) as u varies over an appropriate space. Let us suppose that we are looking for a minimum value. The question of determining the minimum of J(u) over the original infinite-dimensional space is then replaced by the finite-dimensional problem of minimizing J(u) over a finite-dimensional subspace, each element of which is characterized by a finite number of parameters, say U =
1
bkUk.
k=l
Here the uk are carefully chosen functions. The new problem, that of minimizing the expression
can now be approached in a number of ways. In many cases of significance, J(u) is a quadratic functional, the minimization of which leads to linear equations for the b, . There are, however, numerous difficulties associated with the solution of large systems of linear algebraic equations, which means that the real difficulties often begin at this point. Observe that in both of the principal methods described above, monotone approximation is obtained immediately upon increasing the dimension of the finite space of functions over which the variation is allowed. Thus, if we set A N =
it is clear that
I! c )I ,
min T {a;}
(k:l
PREFACE
xiii
Similarly, if we set
we have
Several fundamental questions immediately present themselves. The first is that of determining when lim A ,
N-CC
= m$
11 T(u)ll,
lim dN = min J(u).
N+m
U
T h e second is that of determining when the function u ( N ) which yields 4 , converges to the function u which yields min, 11 T(u)ll, with the corresponding problem for the Rayleigh-Ritz functional. Under reasonable conditions on T(u),J(u), and the spaces over which u varies, these are not difficult to answer. Far more difficult and important are the associated stability problems of estimating 11 u - u") (1 in terms of A , - A , , or d , - d, , and in determining 11 u - d N I/) as a function of N . These are essential matters when the effective determination of u ( ~is) of importance. A few numerical examples, together with references to extensive work in this area will be given. In view of the considerable effort required to treat the finite-dimensional variational problems when N is large, there is considerable motivation for finding ways of obtaining useful estimates for small N . In a sense, the major problem is the converse. It is one of determining the smallest value of N which yields an acceptable approximation. Questions of acceleration of convergence and extrapolation arise in this connection, with techniques that go back to Euler and Kronecker. We shall touch briefly on these matters. In Chapter 8, we show how a linear equation containing a parameter can be considered to be the Euler equations associated with the minimization of a functional subject to a global constraint. Once again, Rayleigh-Ritz methods can be employed to obtain approximate results. Many of the problem areas discussed in this volume can be further illuminated, or considered by alternative techniques, using the methods of the second volume. There we consider duality as a technique for providing upper and lower bounds, Caplygin's method, and differential inequalities, quasilinearization, dynamic programming, invariant imbedding, the theory of iteration, and truncation techniques. The work was divided into two volumes to prevent the single,
xiv
PREFACE
massive volume that is so forbidding and discouraging to the newcomer into a field. Let us encourage the reader with the flat statement that very little is known about nonlinear analysis and that it is not obvious that major breakthroughs will be made in the near future, or ever. Hundreds and thousands of fascinating and significant problems abound, each of which may require a new theory for its elucidation. I have been fortunate in having three friends read through the book and help considerably with all aspects of preparation of the manuscript: David Collins, Thomas J. Higgins, and Art Lew. I wish to express my appreciation for their help, and to Jeanette Blood and Rebecca Karush for typing the manuscript. RICHARDBELLMAN
Los Angeles, 1969
CONTENTS
vii
Preface
Chapter 1.
First- and Second-order Differential Equations
1.1. Introduction 1.2. T h e First-order Linear Differential Equation 1.3. Fundamental Inequality 1.4. Second-order Linear Differential Equations 1.5. Inhomogeneous Equation 1.6. Lagrange Variation of Parameters 1.7. Two-point Boundary Value Problem 1.8. Connection with Calculus of Variations 1.9. Green’s Functions 1.10. Riccati Equation 1.1 1. T h e Cauchy-Schwarz Inequality 1.12. Perturbation and Stability Theory 1.13. A Counter-example 1.14. Sm If(t)l dt < co 1.15. Sm If’(t)l dt < co 1.1 6. Asymptotic Behavior 1.17. T h e Equation u” - (1 + f ( t ) ) t ~ = 0 1.18. More Refined Asymptotic Behavior 1.19. J m f z d t < 00 1.20. T h e Second Solution 1.21. T h e Liouville Transformation 1.22. Elimination of Middle Term 1.23. T h e WKB Approximation 1.24. T h e One-dimensional Schrodinger Equation 1.25. u” (1 + f ( t ) ) u = 0; Asymptotic Behavior 1.26. Asymptotic Series 1.27. T h e Equation u’ = p(u, I)/q(u, t ) 1.28. Monotonicity of Rational Functions of u and t 1.29. Asymptotic Behavior of Solutions of u’ = p(u,t ) / q ( u ,t ) Miscellaneous Exercises Bibliography and Comments
+
Chapter 2. 2.1. 2.2.
1 2 3 5
7 8 10 11 12 14 16 18 20 21 22 23 24 26 27 29 30 31
33 33 33 35 37 38 39 42 51
Matrix Theory 54 55
Introduction Determinantal Solution xv
CONTENTS
xvi
2.3. Elimination 2.4. Ill-conditioned Systems 2.5. T h e lmportance of Notation 2.6. Vector Notation 2.7. Norm of a Vector 2.8. Vector Inner Product 2.9. Matrix Notation 2.10. Noncommutativity 2.1 I. ‘l’he Adjoint, or Transpose, Matrix 2.12. ‘l’hc Inverse Matrix 2.13. Matrix Norin 2.14. Relative Invariants 2.15. Constrained Minimization 2.1 6. Symmetric Matrices 2.17. Quadratic Forms 2.18. Multiple Characteristic Roots 2.19. Maximization and Minimization of Quadratic Forms 2.20. illin-Max Characterization of the A, 2.21. Positive Definite Matrices 2.22. Ileterminantal Criteria 2.23. Representation for A-’ 2.24. Canonical Representation for Arbitrary A 2.25. Perturbation of Characteristic Frequencies 2.26. Separation and Reduction of Dimensionality 2.27. Ill-conditioned Matrices and Tychonov Regularization 2.28. Self-consistent Approach 2.29. Positive Matrices 2.30. Variational Characterization of h ( A ) 2.3 I . Proof of Minimum Property 2.32. Equivalent Definition of A(A) Miscellaneous Exercises Bibliography and Comments
Chapter 3.
58 59 60 60 61 61
63
64 65 65 67 68 71 72 14 75 76 77 79 81 82 82 84 85 86 88 88 89 91 92 94 101
Matrices and Linear Differential Equations
3.1. Introduction 3.2. Vector-Matrix Calculus 3.3. Existence and Liniqueness of Solution 3.4. ’rhe Matrix Exponential 3.5. Commutators 3.6. Inhomogeneous Equation 3.7. ‘I’he Euler Solution 3.8. Stability of Solution 3.9. Idinear Differential Equation with Variable Coefficients 3.10. Linear Inhomogeneous Equation 3.1 I. Adjoint Equation XB 3.12. T h e Equation X’ = A X 3.13. Periodic hlatriccs: the Floquet Representation 3.14. Calculus of Variations 3.15. Two-point Boundary Condition
+
104 104
105 107 108 110 111 113 114 116 118 118 1 20 121 122
CONTENTS
3.16. 3.17. 3.18. 3.19. 3.20.
Green’s Functions T h e Matrix Riccati Equation Kronecker Products and Sums
AX
+XB = C
Random Difference Systems Miscellaneous Exercises Bibliography and Comments
Chapter 4.
Introduction Dini-Hukuhara Theorem-I Dini-Hukuhara Theorem-I1 Inverse Theorems of Perron Existence and Uniqueness of Solution Poincark-Lyapunov Stability Theory Proof of Theorem Asymptotic Behavior T h e Function q ( c ) More Refined Asymptotic Behavior Analysis of Method of Successive Approximations Fixed-point Methods Time-dependent Equations over Finite Intervals Alternative Norm Perturbation Techniques Second Method of Lyapunov Solution of Linear Systems Origins of Two-point Boundary Value Problems Stability Theorem for Two-point Boundary Value Problem Asymptotic Behavior Numerical Aspects of Linear Two-point Boundary Value Problems Difference Methods Difference Equations Proof of Stability Analysis of Stability Proof T h e General Concept of Stability Irregular Stability Problems T h e Emden-Fowler-Fermi-Thomas Equation Miscellaneous Exercises Bibliography and Comments
5.6.
134 135 138 140 140 142 143 146 148 149 150
152 152 155 156 157 157 158 159 160 161 163 165 165 166 168 168 170 171 182
The Bubnov-Galerkin Method
5.1. Introduction 5.2. Example of the Bubnov-Galerkin Method 5.3. Validity of Method 5.4. Discussion 5.5.
123 123 124 125 127 127 131
Stability Theory and Related Questions
4.1. 4.2. 4.3. 4.4. 4.5. 4.6. 4.7. 4.8. 4.9. 4.1 0. 4.1 1 . 4.12. 4.13. 4.14. 4.1 5. 4.16. 4.1 7. 4.18. 4.19. 4.20. 4.21. 4.22. 4.23. 4.24. 4.25. 4.26. 4.27. 4.28.
Chapter 5.
xvii
T h e General Approach Two Nonlinear Differential Equations
187 188 189 190 190 192
xvi ii
CONTENTS
5.7. T h e Nonlinear Spring 5.8. Alternate Average 5.9. Straightforward Perturbation 5.10. A “Tucking-in’’ Technique 5.1 1. l‘he Van der Pol Equation 5.12. Two-point Boundary Value Problems 5.13. T h e Linear Equation L(u) = g 5.14. Method of Moments 5.15. Nonlinear Case 5.16. Newton-Raphson Method 5.17. Multidimensional Newton-Raphson 5.18. Choice of Initial Approximation 5.19. Nonlinear Extrapolation and Acceleration of Convergence 5.20. Alternatives to Newton-Raphson 5.21. Lagrange Expansion 5.22. Method of Moments Applied to Partial Differential Equations Miscellaneous Exercises Bibliography and Comments
Chapter 6.
193 196 196 198 198 200 200 202 202 204 207 208 210 21 1 212 214 215 222
Differential Approximation
6.1. Introduction 6.2. Differential Approximation 6.3. Linear Differential Operators 6.4. Computational Aspects-I 6.5. Computational Aspects-I1 6.6. Degree of Approximation 6.7. Orthogonal Polynomials 6.8. Improving the ApDroximation 6.9. Extension of Classical Approximation Theory 6.10. liiccati Approximation 6. I I . Transcendentally-transcendent Functions 6.12. Application to Renewal Equation 6.13. An Example 6.14. Differential-Difference Equations 6.15. An Example 6.16. Functional-Differential Equations 6.17. Reduction of Storage in Successive Approximations 6.18. Approximation by Exponentials 6.19. Mean-square Approximation 6.20. Validity of the Method 6.21. A Bootstrap Method 6.22. T h e Nonlinear Spring 6.23. ’I‘he Van der Pol Equation 6.24. Self-consistent Techniques 6.25. T h e Riccati Equation 6.26. Higher-order Approximation 6.27. Mean-square Approximation-Periodic Solutions Miscellaneous Exercises Bibliography and Comments
225 225 226 226 227 228 229 23 1 23 1 232 233 233 236 238 239 240 242 242 242 243 244 244 246 248 24 8 250 25 1 253 255
CONTENTS
Chapter 7.
The Rayleigh-Ritz Method
7.1. Introduction 7.2. T h e Euler Equation 7.3. T h e Euler Equation and the Variational Problem 7.4. Quadratic Functionals: Scalar Case 7.5. Positive Definiteness for Small T 7.6. Discussion 7.7. T h e Rayleigh-Ritz Method 7.8. Validity of the Method 7.9. Monotone Behavior and Convergence 7.10. Estimation of I u z1 I in Terms of J(v) - J(u) 7.11. Convergence of Coefficients 7.12. Alternate Estimate 7.13. Successive Approximations 7.14. Determination of the Cofficients 7.15. Multidimensional Case 7.16. Reduction of Dimension 7,17. Minimization of Inequalities 7.1 8. Extension to Quadratic Functionals 7.19. Linear Integral Equations 7.20. Nonlinear Euler Equation 7.21. Existence and Uniqueness 7.22. Minimizing Property 7.23. Convexity and Uniqueness 7.24. Implied Boundedness 7.25. Lack of Existence of Minimum 7.26. Functional Analysis 7.27. T h e Euler Equation and Haar's Device 7.28. Discussion 7.29. Successive Approximations 7.30. Lagrange Multiplier 7.3 I . A Formal Solution Is a Valid Solution 7.32. Raising the Price Diminishes the Demand 7.33. T h e Courant Parameter 7.34. Control Theory ~
Miscellaneous Exercises Bibliography and Comments
Chapter 8. 8.1. 8.2. 8.3. 8.4. 8.5. 8.6. 8.7. 8.8. 8.9.
xix
259 259 260 261 263 264 265 265 267 268 269 270 27 1 272 273 274 275 277 279 280 281 282 282 283 284 284 286 287 288 288 289 289 290 291 29 1 301
Sturm-Liouville Theory
Equations Involving Parameters Stationary Values Characteristic Values and Functions Properties of Characteristic Values and Functions Generalized Fourier Expansion Discussion Rigorous Formulation of Variational Problem Rayleigh-Ritz Method Intermediate Problem of Weinstein
304 305 306 307 312 313 314 315 316
xx
CONTENTS
8.10. Transplantation 8.1 1 . Positive Definiteness of Quadratic Functionals 8.12. Finite Difference Approximations 8.13. Monotonicity 8.14. Positive Kernels Miscellaneous Exercises Bibliography and Comment
316 317 318 319 320 322 329
Author Index
331
Subject Index
337
METHODS OF NONLINEAR ANALYSIS Volume I
This page intentionally left blank
Chapter 1 FIRST- AND SECONDORDER DIFFERENTIAL EQUATIONS
1.l.Introduction
I n this chapter we wish to prepare the way for the subsequent chapters by considering in some detail a number of results pertaining to firstand second-order ordinary differential equations. T h e pathbreaking consists of two parts. I n the first place, we want to observe the kinds of results that can be obtained in these relatively simple situations, and in the second place we want to note the methods that are employed. I n many cases the same methods can be employed to treat higher order equations. We will consider first the linear first-order differential equation
+p ( t ) u = q(t),
u(0)
u’
=
c,
(1.1.1)
and then the second-order linear differential equation U”
+ p ( t ) u’+ q ( t )
U = f(t),
( 1.1.2)
subject to both initial and two-point boundary conditions. T h e twopoint boundary conditions introduce Green’s functions. Following this, we will analyze the asymptotic behavior of solutions of u”f.(1 +f(t)) u
=
0
(1 .I -3)
as t -+ 00 wheref(t) is “small” in some sense. This will lay the groundwork for the study of the solutions of the more general equation 24’’
fR ( t ) U
=
0,
(1.1.4)
relying upon an ingenious change of dependent variable due to Liouville. T h e Riccati equation plays an important role in the study of (1.1.3) and (1.1.4). Although it appears here only as an artifice, its true fundamental role in analysis will be made apparent in Volume I1 in connection with dynamic programming and invariant imbedding. T h e results obtained in connection with (1.1.3) will permit us to discuss the validity of the WKB approximation. 1
2
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
Finally, we will indicate how elementary, albeit occasionally tedious, analysis can be used to obtain precise asymptotic behavior of the equation (1.1.5)
where p and q are polynomials in their arguments, in the important case where we are considering only solutions which remain finite for all t > t o , the so-called physical solutions. This work was initiated by Bore1 and Hardy. T h e explicit results obtained here can be used occasionally in the later chapters to illustrate the fancier analytic and computational methods designed to handle the more complex functional equations thrust upon u s by biology, economics, engineering, and physics. Although specific examples are absolutely essential, there is no harm done in occasionally minimizing the algebraic and arithmetic labor involved in this exposition. When an equation is displayed in this chapter, it will be tacitly assumed that a solution exists and, furthermore, that this solution is unique. I n Chapters 2 and 3 we will provide the necessary existence and uniqueness theorems validating both the analysis that follows and various statements that are made about the properties of these solutions. 1.2. The First-order Linear Differential Equation
T h e equation u' + p ( t ) u
=
dt),
4 0 ) = c,
(1.2.1)
plays an important role in the theory of differential equations, due equally to its simplicity and to the fact that it is one of the few equations which can be solved explicitly in terms of elementary functions and elementary operations. T o obtain this explicit analytic solution, we multiply by the integrating factor
and integrate both sides between 0 and t. We thus obtain
1.3. FUNDAMENTAL INEQUALITY
or u = cexp
3
1-
Let us denote the expression on the right by T ( p , q, c ) . This is an operation on the functions p and q and a simple linear function of c. Observe that it is also linear in q. Observe further that the positivity of the exponential function permits us to assert that the operation T is monotone in both c and q, that is to say, if c, 3 c2 , T(P, 9, c1> 3 T(P,9, cz), (1.2.4) 91 3 92 * T(P, 91, c) 3 T(p,qz ,4, if These properties will play an important role in the chapter on quasilinearization in Volume 11. Exercises
1. Use the preceding results to show that if u satisfies the differential 0, where v is inequality u' + p(t)u < q(t), u(0) = c, then u < v , t the solution of the corresponding equation.
2. Show that if
c
>, 0, q >, 0, then T ( p , , q, c)
3. How many solutions of
+
< T ( p , , q, c) for p , 3 p , .
u' au = 1 can satisfy the condition that 1imt+- u ( t ) exists ? Consider separately the cases a > 0, a < 0, a = 0.
+
4. Consider the same question for u' a(t)u = f ( t ) , under the assumption that the limits of a(t) andf(t) exist as t -+ 00.
5 . Obtain an explicit representation for the solution of u'
+ p,(t) u = pz(t)U"+1,
u(0) = c
1.3. Fundamental Inequality
From the monotonicity noted above we can readily establish the following basic inequality:
If u(t) < c 3 0, then
Lemma.
for t
+ JAu(tl)v ( t l ) dt, , where c > 0 and u, v 3 0 (1.3.1)
4
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
T o obtain this result from the result of the foregoing section, let rt
Then the integral inequality yields, after multiplication by v(t), the differential inequality ~
dw dt
< cv + vw,
w ( 0 ) = 0,
(1.3.3)
whence, taking account of Exercise 1 following the last section, w
< exp [,: v dt,]
1:
/cu(tl)exp
[-
f'
1
v(t2)dt2] dt,
.
(1.3.4)
Since the integration can be readily carried out, we have w
< c [exp
(1:
v dt,)
-
13.
(1.3.5)
Using this estimate in the original integral inequality, we have
11
=
c exp
(1:.
dt,),
(1.3.6)
the desired result. Exercises
1. Establish the foregoing result starting with the relation
and integrating between 0 and t. (This proof is much shorter, but it masks the reason for the validity of the result.)
2. What can we deduce if u(t) < f ( t ) v >O? 3 . Is the restriction
z1
+ $, u(tl)u(tl)dt, for t 3 0 with
3 0 essential ?
4. Carry through a proof of (1.3.1) by iteration of the inequality.
+
5. Consider the inequality u < h(t) aecbt $, ebL1w(t,)dt, J,,1 eb%u(tl) dt, \< v(t) where v' = heb1 av, u(0) = 0.
+
. Show
that
1.4. SECOND-ORDER LINEAR DIFFERENTIAL EQUATIONS
5
6. Hence, show that if 0 < a < b and h(t) + O
as t-+ co, then f = o(g) to signify that f / g + 0. (Th e notation f = O ( g ) signifies that 1 f 111g 1 is uniformly bounded as t + co.)
Jot e b t w ( t l )dt,
=
o(ebl) as t + 00. We use the notation
+
(1 - a ) J: u(tl) dt, + b as t + co, where 0 can we say about the limiting behavior of u(t) ?
7. If au(t)
+
< a < 1, what
8. If au(t) (1 - a ) So u(tl) dt,/t -+ b as t -+ co, where 0 what can we say about the limiting behavior of u(t) ? 1
+
(1 - a ) C;=, uk/n+ b as n -+ co, where 0 < a can we say about the limiting behavior of u, as n + co ?
9. If au,
< a < 1,
< 1, what
1.4. Second-order Linear Differential Equations
Consider the linear equation U"
+ p ( t ) u' + q ( t ) u = 0,
(1.4.1)
where p and q are assumed continuous in [0, TI. This is a fundamental equation of mathematical physics, arising in analytical mechanics, quantum mechanics, wave propagation, and many other areas. It remains challenging despite all of the work that has been devoted to its study since we cannot read off the properties of the solution with the aid of a general explicit analytic solution in terms of the elementary functions and a finite number of integrations and differentiations. Every solution of this equation is a linear combination of two particular solutions u1 , u2 , the principal solutions, determined by the initial conditions =
1,
u1'(0) = 0,
Uz(O) =
0,
Uz'(O) =
Ul(0)
(1.4.2)
1.
T h e Wronskian of two solutions, u, v, of (1.4.1) is a useful function defined by the determinantal expression W(u,v) =
I
u
v
u)
v'
=
(1.4.3)
uv' - u'v.
It is easily seen that u'
21'
I
=
-p(t)
w.
(1.4.4)
6
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
I n particular, W ( u , , u2) = I in the case wherep principal solutions.
= 0, and
u and
are
Exercises
1. Use the Wronskian relation to obtain the general solution of (1.4.1) given one particular solution.
2. Show that W(u,v) # 0 for any t > 0 if W(u(O),v(0)) # 0.
+
3 . If zil and u2 are principal solutions, show that a,u,(t,) a,u2(tl)= b, , a,u,’(t,) + a,u,’(t,) = b, has a unique solution for the coefficients a, and a 2 .
4. If p ( t ) 3 0, show that W(u, , u 2 ) is a constant for t 3 0. 5. Show that W ( u , , u2) is never zero unless it is identically zero.
6. Show that the general solution of the linear difference equation
where a , , a2 are constants, is un = clrln
+ cZrzn,
where rl and r2 are the roots of the quadratic equation r2
provided that r, #
Y,
+ alr + a2
=
0,
. What is the form of the solution if rl
= r,
?
7. Show that a necessary sufficient condition that all solutions of the foregoing equation approach zero is that the roots of r2 a,r + a2 = 0 be less than one in absolute value.
+
8. Determine the analytic form of r, where
Y , + ~ = (ar, Hence determine the asymptotic form of rn as n + Y, u,&, and pick u, and v ,judiciously. 1
+ b)/(cr, + d ) . 00.
Hint: Set
1.5. INHOMOGENEOUS EQUATION
7
1.5. Inhomogeneous Equation
Consider first the case where the coefficients and u satisfies the inhomogeneous equation u"
p and q are constant
+ pu' + qu = f ( t ) .
(1.5.1)
We can take u(0) = u'(0) = 0 without loss of generality since it is sufficient to obtain any particular solution in order to obtain the general solution. Let us use a method which is particularly well suited to equations with constant coefficients, namely the Laplace transform. I t possesses the merit of extending immediately to many other types of functional equations such as linear differential-difference equations and linear partial differential equations and even to certain types of nonlinear equations containing convolution terms. Write L(u) =
I
m
(1.5.2)
e-%(t) dt
0
assumed convergent for Re(s) > 0. Actually, convergence is of no particular significance here since we use the Laplace transform solely to obtain the desired result. I t is then verified using a direct method. From (1.5.1) we derive by repeated integration by parts L(u) =
Let r1 , r2 be the roots of moment. Then
+ 4)
L ( f1
+ps
(IZ
(1.5.3)
*
+ p r + q = 0, assumed
P . ~
Hence, the inverse transform of (s2
+ PS + q)-'
distinct for the
is (1.5.5)
Let us now employ the fundamental relation for the Laplace transform of the convolution of two functions, L
(St
0
u(t
-
t l ) v(2,) dll)
= L(u)L(v).
(1.5.6)
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
8
1
This means that Jflu(t - tl) v(tl) dt, is the Laplace inverse of L(u)L(v) and thus, referring to (1.5.3), that u
1'
k(t
=
-
t l ) f ( t J dt,
0
.
(1.5.7)
Once the result has been obtained, we can readily verify by direct differentiation that it is indeed a solution. Hence, there is no need to impose any conditions on f ( t ) at t = co or worry about the convergence of the integrals involved. I n particular, if the equation is U"
we have r,
+ w2u = f ( t ) ,
(1.5.8)
and
= mi, r p = - w i
- e-wit
k(t) =
2wi
-
sin w t w
,
(1.5.9)
and thus the frequently useful result u =
-I I
t
sin w ( t -
(1.5.10)
w f l
Exercises
1. Determine the form of K(t) in the case that the roots r , and r2 are equal.
+
+
2. Obtain the general solution of u , + ~ a,u,+, 00 a2u, =f, using the generating function approach, i.e., set u(s) = Cn=ou,sn, etc. 3. Obtain an explicit expression for the Laplace transform of the solution of the differential-difference equation u'(t) = alu(t) a,u(t - l), t 3 1, u(t) = f ( t ) , 0 t 1, and thus a contour integral representation for u(t). For further results, see
< <
+
R. Bellman and K. L. Cooke, DzfSeerential-Dzjference Equations, Academic Press, New York, 1963. 1.6. Lagrange Variation of Parameters
If p and q are dependent on t , it is most convenient in this secondorder case to use a special device rather than either of the general
1.6. LAGRANGE VARIATION OF PARAMETERS
9
approaches based on matrix theory and the use of the adjoint operator given in the following chapter. T h e method employed, due to Lagrange, is useful in other situations and is worth keeping in mind. Let u l ,uz be, as above, the principal solutions of the homogeneous equation and write u =
v1u1
+ vzuz
(1.6.1)
1
where vl and v z are functions to be chosen expeditiously. T h e n
+ vzuz‘ +
u’ = vlul‘
Vlhl
+ vz‘u, .
(1.6.2)
Let us use one free choice to simplify this expression for u‘ by setting vl’ul
+ vz’u2= 0.
(1.6.3)
This yields one relation between vl’ and vz‘. Since this leaves the relation u’ = vlul’ vzu2’, combining (1.6.2) and (1.6.3), we have, upon a further differentiation,
+
u” = v1‘u1’
Hence u“
+ pu‘ + qu
= Vl[g
+
ViU2’
+ vlu; + v,u; .
+ pu,’ + qull +
V,[.,”
+ pu,‘ +
(1.6.4)
QU,]
+ vl’ul’ + vz’uz’ = vl’ul’ + v2‘u2‘
(1.6.5)
upon recalling the definition of u1 and u, . Thus, the equation u”
+ pu’ + qu = f
(1.6.6)
yields a second relation for vl‘ and v,’, vl’u;
+ v2’uz’ = f.
(1.6.7)
Solving (1.6.3) and (1.6.7) simultaneously for the unknown functions q‘, vz’,we have
(1.6.8)
10
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
Hence, ignoring the constants of integration,
(1.6.9) Exercises
1. Why can we ignore the constants of integration?
2. Obtain the general solution of
+
u" w2u = eaLand thus, by differentiation with respect to u, the general solution of u" w2u = t.
+
1.7. Two-point Boundary Value Problem
I n many important cases, such as those arising in the calculus of variations and in many parts of mathematical physics, the solution is not determined by initial-value conditions but rather by conditions at two points such as u(0) = c1 , u ( T ) = c2 , (1.7.1) Sometimes these two-point conditions determine a solution uniquely, and sometimes not, as we shall see. T o obtain the solution of (1.4.1) subject to (1.7.1), we proceed as follows. Let u1 , u2 be the principal solutions. T h e n the general solution has the form u = au, + bu,, where a and b are arbitrary constants. T h e condition at t = 0 determines a, namely, a = c l . Hence we have u = clul
+ bu, ,
(I -7.2)
where the remaining unknown parameter b is determined by the condition at t = T, (1.7.3) c* = clul( T ) bu2(T ) .
+
Hence, if u2(T ) f 0, there is a unique solution. If u2(T ) = 0, then u is a solution if c2 == clul( T ) . I n this case, there is a one-parameter family of solutions. More general boundary conditions can be discussed in a similar way. T h e determination of when u 2 ( T )# 0 is related to questions of characteristic values, which we shall discuss iubsequently. Exercises
+
1. Solve u" u = 0, u(0) = c, , u ( T ) = c2 . Discuss the condition on T to ensure existence and uniqueness.
1.8. CONNECTION WITH CALCULUS OF VARIATIONS
11
u’( T )= c 2 , and when u(0) = cl, u( 7’) = c2 . Discuss the question of existence and uniqueness.
2. Solve u” - au =0, u(0) = c,, 3. Consider the equation u“
+ p(t)u’ + q(t)u = 0 subject to
Let u, , u2 be two linearly independent solutions of the equation. Show that the question of existence and uniqueness of a solution subject to the foregoing conditions is dependent upon the nonvanishing of the determinant
4. Show that
5. Show that the domain of integration may be taken to be 0 < t, < t , < T .
+
6 . Using this result, discuss the problem of solving u” u = 0 subject T u ( t ) elit dt = b, , J i u ( t )eA21dt = b, . For more general results to
so
and the connection with Cebycev systems of functions, see
R. Bellman, “A Note on the Identification of Linear Systems,” Proc. Amer. Math. SOC.,Vol. 17, 1966, pp. 68-71.
7. Consider the parabolic partial differential equation u1 = u,, , u(x,0 ) = g(x), u(0, t ) = u(1, t ) = 0, t > 0. Show that L(u), the Laplace transform of u, satisfies an ordinary differential equation subject to a two-point boundary condition when considered as a function of x.
1.8. Connection with Calculus of Variations
T h e equation U”
+ g(t)u = 0
(1.8.1)
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
12
possesses the fundamental property of being the Euler equation associated with the quadratic functional J(u) =-
(1.8.2)
J T ( u ' 2 - q ( t ) u2) dt. 0
This stationarity property holds regardless of the boundary conditions that are imposed. I n Chapter 8 we will discuss the rigorous aspects in great detail. Here, let us merely indicate the connection which motivates many of the ingenious artifices that are used in the study of (1.8.1). Let u be a solution of (1.8.1), v be another function such that a ' d 2 ( 0 , T ) , and E a parameter. T h e n
+
J(ZL
CV)
Integrating the term
J(u)
:
Jr
+ 2~
IT
[u'v'
-
dt
u'v
1,
f
T ~
Hence
+ €V) = J ( u ) + =
+ €"(a).
(1.8.3)
u'o' dt by parts, we have
j:.w
J(u
q ( t ) uv] dt
0
J(u)
€'J(.)
+ ."(.).
~
2€
T
u"v dt.
0
j
T
o[u"
0
(1.8.4)
+ q ( t ) u] dt (1.8.5)
We have blithely assumed that the integrated term 24'711: is equal to zero. This may be ensured either by virtue of the original restrictions on u and u' or by choice of v ; see Chapter 8 for further details. Exercise
+
+
I . Can one obtain the equation u" p(t)u' q(t)u = 0, p # 0, as the Euler equation of a quadratic functional of the form
1.9. Green's Functions
Let us next consider the inhomogeneous equation u" + g ( t ) u
=f(t),
u(0) = u ( T ) =: 0.
(1.9.1)
1.9. GREEN'S FUNCTIONS
13
Referring to Sec. 1.7, we see that if u2(T)# 0 there is at most one solution. Furthermore, if u 2 ( T )# 0 there is exactly one solution. To obtain this solution we proceed as follows. Using the representation in Sec. 1.6, we have
* = Cl*l
+
CZ*Z
+
J t s(4
tl)f(tl)
4,
(1.9.2)
where c1 and c2 are constants to be determined, and
44 tl)
=
r -w *z@d +
*2@>
*l(tl)l/W(tl
> t2)-
(1.9.3)
Without loss of generality, we can take u1 and u2 to be principal solutions so that W = 1 . Setting t = 0, we see that c1 = 0. T o obtain c 2 , we set t = T, 0 = C2U2(T)
+J
T
q(T9 tdf(t1) 4 .
(1.9.4)
Using this value of c2 in (1.9.2), we have
T h e dependence of the kernel k on T is not usually explicitly indicated. We ordinarily write
*=J
T
4 4 tl)f(tl) dtl ,
(I .9.6)
where
(1.9.7)
14
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
I t follows from this (cf. 1.9.7) that
a most important reciprocity relation. T h e function k(t, t l ) is called the Green’s function associated with the equation and the boundary conditions. Different conditions produce different Green’s functions. Exercises
< <
1. Show that k ( t , t l ) is continuous in t for 0 t T , but that k(t, t l ) has a discontinuous partial derivative with respect to t at t = t , . What is the nature of the discontinuity ?
2. Obtain the Green’s functions for 24’’ = f ( t ) , (a> (b) U” = f ( t ) , (c) 24’’ w2u = f ( t ) , (4 U” w2u = f ( t ) , (el u” wzu = f ( t ) ,
+ + +
the equations u(0) = u ( T ) = 0, u(0) = 0, U ‘ ( T ) = 0, u(0) = u ( T ) = 0 u(0) = 0, u‘(T) = 0, u(0) = 0, U ’ ( T ) bu(T) = 0.
+
3. Consider the Green’s function for Exercise 2(e) as a function of b. Is it continuous as b 4 0 and b -+ co ?
+ u = f ( t ) , u(0) = u ( T ) = 0.
4. Consider the Green’s function for EU” Discuss the limiting behavior as E + 0.
5. Similarly discuss the behavior of the solutions of €24’’ - u = 0, €U“
as
+ tu‘
E 4
--
u =
€24‘‘
0,
+ u‘ = 0, EU“
EU” - u‘ =
+ tu’ + u = 0,
0, €24”
EU”
+ tu‘
+ tu’ = 0, -
u/2 = 0
0. See,
C. E. Pearson, “On a Differential Equation of Boundary Layer Type,” J . Math. Phys., Vol. 47, 1968, pp. 134-154. 1.10. Riccati Equation
T h e change of dependent variable, u
=
exp
(1
z1
dt),
(1.10.1)
1.10. RlCCATl EQUATION
15
replaces the linear second-order equation u” +pu’
+ gu = 0
(1.10.2)
with the nonlinear first-order equation 0’
+
v2
+ g = 0.
+pv
(1.10.3)
This is called a Riccati equation and plays a basic role in the theory of the linear second-order equation. * T h e transformation which appears so formal here is actually well motivated by both analytic and physical considerations, as we will see in the chapters on dynamic programming and invariant imbedding in Volume 11. Exercises
1. If v satisfies a Riccati equation, show that v-l also satisfies a Riccati equation.
+
2. If v satisfies a Riccati equation, show that w
+
= (av b)(cv d) also satisfies a Riccati equation for any functions of t , a , b, c , and d. m
3. If v(t) possesses a power series expansion v ( t ) = En=, antn, and p
and q are analytic in t , show how to obtain a continued fraction expansion for v,
where c1 , c2 are positive integers. Hint: Set v
=
a,
+ a l t c l / v l , etc.
4. If p and q are analytic in a parameter E and v possesses a power series expansion in E, v,(t) q ( t ) ..., show how to obtain a continued fraction expansion for v,
+
+
v=vo+-
+
En1
1
b
E lv,
’
+
where b, is a positive integer. Hint: Set v, = vo E W , etc. (The last two results are connected with the subject of Pad6 approximations, a topic considered in the exercises at the end of the chapter.)
* Also in the study of nonlinear equations. See H. P. F. Swinnerton-Dyer, “On a Problem of Littlewood Concerning Riccati’s Equation,” Proc. Cambridge Phil. SOC., Vol. 65, 1969, pp. 651-662.
16
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
1.11. The Cauchy-Schwarz Inequality
I n the course of various estimations, we will make frequent use of the following basic inequality: (1.11.1) valid for any two real functions for which the right-hand side exists. Let us give two different proofs; each can be extended in different ways. Start with the inequality 0
< (U
b)'
~-
=
+
( 1 .1 1.2)
a2 - 2 ~ b b2,
valid for any two real numbers a and b, equivalent to 2ab
< a2 + b2.
(1.1 1.3)
Now set b
= g/(jTg2dt)lil, 0
(1.11.4)
obtaining (1.1 1.5) Integrating between 0 and T , we obtain (1.1 1.1). I n the second proof, we start with the fact that we have 0
T 0
(af
+ bg)2dt
=
a2
s
T 0
f dt
+ 2ab
1
T
0
f g dt
+ 6 2 s T g 2 dt 0
( 1.1 1.6)
for any real numbers a and b and for any two functions f and g belonging to Z,2(0, T ) . For the quadratic form in a and b to be non-negative for all real a and b, it is necessary that the quadratic equation r ' I S0 T f 2 d t + 2 r / 0' f g d t + s T 0g 2 d t = O
( 1 . I 1.7)
have imaginary roots. T h e usual discriminant condition yields (1.11.1).
1.11. T H E CAUCHY-SCHWARZ INEQUALITY
17
Exercises
1. Show that ab
<
+
a2/2 b2/2 for a, b 3 0 by examining areas in the following figure. What is the case of equality ?
2. I n a similar fashion, establish Young’s inequality, ab < up/p + bq/q for a, b 3 0, p > 1 and q determined by l/p + l / q = 1 . 3. Thus, establish the Holder inequality
4. Establish the triangle inequality
5. Establish the Minkowski inequality
forf,g 3 0 , p 3 1.
6. From the inequality in Exercise 1, show by application to (al and (a3 + a4), where a , , a 2 , a 3 , a4 2 0, that
+ u2)
and hence, generally, that
for N
= 2”
if the a, are nonnegative.
7. By suitable specialization of the a,i show that this general arithmetic mean-geometric mean inequality holds for any integer N . (This
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
18
proof by a combination of backward and forward induction is due to Cauchy.)
8. By suitable specialization of the ai and a limiting procedure derive Young’s inequality given in Exercise 2. 9. Show that
where we employ the norm llfll
=
(JOT
If
l2
dt)1/2.
10. Hence, derive Minkowski’s inequality
I1g + h I1 < I1g I/
+ I1 h I/.
11. Similarly, show that
where
1lp
llfll,
+ 1/q
=
=
1.
(So’ I f I”
d P , Ilfll,
=
(JOT
I g Iq Wq,with p > 1,
12. Let d(x) = (xlP - x2P - ... - x,P)l/P for xi in the region defined x,P)l/P, p 3 1. by xi 3 0, i = 1, 2,..., n, and x1 > (x2P + Show that if x = (xl,x2 ,..., xn), y = ( y l , y 2 ,...,y,), then x y belongs to the foregoing region when x and y do and that 0 . -
4(. Hint: Write +(x)
=
+
+
+ Y >2 4(4 + 4(Y>* n
min,(xi=l xixi) for z in a suitable region.
13. Show that
This is an example of the interesting class of problems which arise in an attempt to show explicitly that an expression is nonnegative. 1.12. Perturbation and Stability Theory
T h e equation u”
+ a(t)u = 0
(1.12.1)
19
1.12. PERTURBATION AND STABILITY THEORY
#
plays an important role in mathematical analysis and in mathematical physics, particularly in connection with the Schrodinger equation. Since the solution cannot, in general, be obtained in terms of the elementary functions of analysis and a finite number of operations of integration and differentiation, we must have recourse to a number of approximate techniques. We shall present only a few of the many ingenious methods that exist in the pages that follow. An important case to begin with is that where a ( t ) is close to a constant, which is to say, the equation has the form u”
+ (1 +f(t)).
= 0,
(1.12.2)
where f is small in some sense. Once we have analyzed equations of this quite special type in detail, we will be in an excellent position to study the behavior of solutions of (1.12.1) with the aid of a simple basic transformation due to Liouville. *
Exercises
+ +
1. Consider the equation u” (1 Ef(t))u = 0, where E is a parameter. Show that we can obtain a formal solution of the form u = uo E U ~ e2u2 + , where
+
+
u;
+ u, = 0,
u;
+ u1 +fu,
= 0,
and so on. We will examine the validity of perturbation expansions of this nature in Chapter 3.
2. Consider the associated Riccati equation v‘
+ v2 + (1 + 4)
Show that we have a formal solution of the form 0
where 0,’
and so on.
= 0,
+ + E’U1
+ + 1 = 0, Oo2
E2V2
Vl’
= 0.
+ .*.,
+ 2v,v1 +f
=
0,
+
3. Consider the equation u“ u = g with the formal solution u = g - U” = g - g(2) + g(4)- .*., Under what conditions does the series represent a solution of the equation ?
* See also K. Stach, “Die allgemeine Eigenschaften der Kummerschen Transformation zweidimensional Raume von stetigen Funktionen,” Math. Rev., Vol. 36, August 1968,
No. 1720.
FIRST- A N D SECOND-ORDER DIFFERENTIAL EQUATIONS
20
1.13. A Counter-example
I t is reasonable to expect that the condition f ( t ) +0 would ensure that all solutions of (1.13.1) u“ ( I +f(t)) u = 0
+
are bounded as t -+ GO * and, furthermore, are well approximated to by the solutions of the tame equation V’’
+
z, =
0.
( 1.13.2)
As the following example shows, the situation is far more complex. Consider the function u = exp
(j‘g(s) cos s ds1 cos t ,
( 1.13.3)
0
where g will be chosen malevolently in a moment. We have 21 21‘
upon setting w u”
=
=
Hence u”
-zu
=
w cos t ,
=
-w sin t
( 1.1 3.4)
+ (g cos2 t ) w,
1
exp(Jog cos s ds), and cos t
-
(g cos t sin t ) w - (2g cos t sin t ) w
+ (g‘cos2 t ) zu
+u
:
-
=
+ (g2
C O S ~t ) w.
+
w cos t[g’ cos t 3g sin t g2 cos2 t ] u[g’ cos t - 3g sin t g2 cos2 t ] , -
+
(1.13.5)
(1.1 3.6)
which means that u satisfies the equation u”
4-(1
+ 4)u
=
( 1.1 3.7)
0,
where (f, = 3s sin t
-
g’ cos t
- g2
cos2 t .
(1.13.8)
Choosing g = cos t i t ,
( 1.13.9)
we see that g, g‘, and g2 all approach zero as t + a.Nonetheless, the integral J” g(s) cos s ds diverges, which means that u increases indefinitely
* As
a matter of fact, a well-known mathematician published a “proof” of this.
1.14.
Jm j
f(r)i dt < co
21
in amplitude as t + co. Hence, the study of (1.13.1) has its interesting aspects in the case whereftt) + 0 as t + CO.
T h e example presented in the preceding section shows that it is not sufficient to require that f ( t ) -+ 0 as t + co if we want all solutions of U" (1 + f ( t ) ) u = 0 bounded as t+co. Let us now show that the condition
+
is sufficient. To do this, we convert the linear differential equation into the linear integral equation u = c1 cos t
t c2 sin t
-
st
sin(t
-
t l ) f ( t l ) u(tl)dt,
0
.
(1.14.2)
Hence IUI
<
Applying the fundamental inequality of Sec. 1.3, we have
I I G (I c1 I
+I
c2
I) exp
(I'
If(t,l)ldt,)
< c3 < a,
(1.14.4)
the desired result. Exercises
1 . Extend the foregoing method to show that if all solutions of U" a(t)u = 0 are bounded as t -+ 03, then all solutions of U" ( a ( t ) + b(t))u = 0 are bounded, under the assumption that J" I b(t)l dt < co.
+ +
2. Show that I U' 1 is bounded as t -+ co under the assumption J" I f 1 dt < co. U" + a(t)u = 0 belong toL2(0, a), then + ( a ( t )+ b(t))u = 0 belong to L2(0,co) if
3. Show that if all solutions of all solutions of I b(t)l c1 < 03.
<
U"
22
FIRST- A N D SECOND-ORDER DIFFERENTIAL EQUATIONS
+ u/tk = 0 cannot be bounded if k > 1 .
4. Show that all solutions of u"
5. Construct a function f ( t ) with the property that maxlGTIf approaches 00 as T ---t co and yet J" 1 f I dt < co.
6. Show that all solutions of u" bounded as t + 03.
7. Show that all solutions of J" jg j dt < a.
+ (1 + b(sin 2t)lt)u = 0
U"
+ gu' + u = 0
I
cannot be
are bounded if
8. Obtain a bound for the difference between the solutions of U" +pu' pu 0 andv" +p,v plv = OoverO < t < T i n t e r m s of the integrals JOT I p -pl I dt, JOT I q - q1 I dt.
+
+
<
J" I f'(t)ldt
1.15.
Next, let us demonstrate that all solutions of u"
+ (1 + f ( t ) ) u = 0
(1.15.1)
are bounded as t + 00 under the conditions f(t)+ 0
t
as
Im l f ' ( t ) l dt <
+ co,
(1.15.2)
00.
Starting with (1.15.1), we have U'U''
Integrating,
+
+ (1 +f(t))
( u ' ) ~ u2 = c,
-
uu' =
(1.15.3)
2 j t f ( t l )uu'dt, 0
=
0.
+ J-"fr(tl)
c1 - f ( t > u2
u2
dt,
0
Without loss of generality, take If(t)l yields
.
< 1/2 for t 3 0.
(1.15.4)
T h e n (1.15.4)
Applying the fundamental inequality, we have u2
the desired result.
< 2c,exp ( 2 J f I f ' 0
I dt,) < c2 < m,
( 1.15.6)
1.16. ASYMPTOTIC BEHAVIOR
23
Exercises
1. Show that I U' I is bounded under the same hypotheses.
2. Show that all solutions of
U"
under the assumptions that J" g(t) + 0 as t 4 GO.
+ (1 + f + g)u = 0 I f I dt < co, J"
are bounded < 00, and
Ig' I dt
3. Is the full force of g ( t ) + 0 as t + co required ? 1.16. Asymptotic Behavior
Now that we have established boundedness under the condition that
J" I f I dt < co, let us obtain some more precise information concerning
the behavior of u ( t ) as t + co. Turning to (1.14.2), we have u = c1 cos t
+ c2 sin t
+ (cos t )
J t (sin
Since I u(t)l is bounded and converge as t -+00. Hence u = cos t [cl
-
(sin t )
Jl
(cos tl)f(tl) u ( t l )dt,
tl)f(tl) 4 t l ) dt,
(1.16.1)
*
J" If(t)l dt < co,we see that both integrals
+ Jm(sin t l ) f ( t l ) u(tl)4 1 0
+ sin t = c, cos
t
[cZ
- Jm
+ o(1)
(cos tl>f(tl> u(tl) dtl]
+ c4 sin t + o(l),*
( 1.1 6.2)
replacing the definite integrals by the constant values. I t is clear that c3 and c4 are linear functions of c1 and c2 . They are, however, quite complex functionals o f f . In the chapter on invariant imbedding in Volume I1 we will pursue the question of their analytic and numerical determination. There are important applications of results of this nature to scattering theory. Exercises
1. Consider the equation integral equation u =
-
U"
+ (1 + p)(t))u= 0
J' sin(t
-
tl) F ( t l > u ( t l ) dt,
0
* Recall
and the associated
our previous use of the o-notation.
,
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
24
where v u =
= c1
v
-
-t c2 sin t . Iterating, show that
cos t
J‘ sin(t
-
t , ) y(tl>v(tl>dt,
0
2. Hence, show that all solutions are bounded if (a)
Jlg)(tl)dt, ,
1
F(tl) sin 2t, dt, , J“,g)(tl) cos 2t, dt, are uniformly bounded for t >, t o . Jto
J“l,
I y ( t l ) sin(t - t,) sin(t, - t2)g)(tl)dt, 1 dt, for some t o . What conditions on y ensure (a) and (b) ? (b)
3 . Show that all solutions of a f 2, b > 1/2.
u”
+ (1 + sin at/t”)u
=
5. Obtain the asymptotic behavior of solutions of u” t -+ cc under the hypothesis that J“ I g I dt < CO. g+o,
J“
/ g ’ l dt
u”
0 are bounded if
+ (1 + sin t2a)u = 0 are bounded
4. Show that all solutions of u” a > 1.
6. Are all solutions of
< k < 1 f o r t 3 to
+ gu‘ + u = 0 bounded
< oo?
1.17. The Equation u”-
(1
if
+ gu’ + u = 0 as if we assume that
+ f(t))u = 0
Let us now consider the equation u”
-
(1 + f ( t ) ) u
=
0
(1.17.1)
under the assumption that f ( t ) + 0 as t + CO. T h e behavior of the solutions of this equation is very much easier to discuss than that of z i ‘ - 1 ( I + f ( t ) ) u = 0 since we can employ the associated Riccati equation, v’
+
212 -
(I
+f(t)) =
0.
( 1.17.2)
We shall carry through the analysis in detail since it will give us an opportunity to introduce techniques that will be employed in more general form in Chapter 3. I n ( 1 . I7.2), set v = 1 4.-w. T h e n w satisfies the equation w‘
+ 2w
1-
w2 - f ( t )
=
0.
(1.17.3)
1.17. THE EQUATION U" - (1
+ f(t))u
25
=0
Regardingf(t) - w2 as a forcing term, we see that w satisfies the integral equation w = ce-2'
+
e-2'
j'
n
e2t1[f(tl)
-
w21 dt,
,
(1.17.4)
where c = w(0). Let us now show that if I c I is sufficiently small (a bound will be given below), the solution of (1.17.4) exists for all t > 0 and w -+ 0 as t -+ co. For our present purposes, we could choose c =0, but it is instructive to give the full result. To do this, we use the method of successive approximations. Set wo = ce-2'
w,+,
=
ce-2'
+ +
e-2'
J' e2t1f(tl) dt, , 0
e-2'
J'
e2t1[f(tl)
- w:] dt,
,
n
0
> 0.
(1.17.5)
<
Without loss of generality, we suppose that If(t)l I c I for t 2 0, since we are interested in the case where f -+ 0 as t -+ co. We wish to show that (1.17.6)
IwnI < 4 1 c l
for n 3 0. We do this inductively. T h e results holds for n
=0
since
Assume that it holds for n. Th en
if I c I < 1/4. T o establish convergence of w, as n -+
CO,
we write
(1.17.9)
26
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
for n 3 I . For n
1, we have
:
J O
< 16 1 c 1'
It
eZt1dt,
eCZt
0
< 8 I c .'1
( 1.17.10)
Iterating the relation in (1.17.9), we have
1 wn+1 - W, I
< 8 1 c IZ(8 1 c j t)./.!
.
(1.17.11)
m
Hence, the series Cn=o1 w,+~- w, 1 converges uniformly in any finite t-interval, and thus w, converges to a function w which satisfies the integral equation in (1.17.4). It remains to show that w -+ 0 as t --t 00. We have w ( t ) = ceCZt
+ eKZtj tn e"ttf(tl) dt,
-
e-"
j t eZt1w2dt, . 0
(1.17.12)
We are now ready to employ Exercise 6 at the end of Sec. 1.3. We have
1w 1
< 1 ce-2t + eCZtJ t e Z f ' f ( t , )dt, I + 4 1 c I eCZtf eZtl I w I dt, , 0
(1.17.13)
0
and we know that the first term on the right approaches zero as t -+ co since f -+ 0 as t co. Hence, since 4 I c I < 1 we have, by the cited exercise, ---f
(1.17.14)
as t -+ co. Returning to (1.17.13), this yields the desired result that w ---f 0 as t co. An alternate proof yielding a stronger result is given below. ---f
Exercise
1. Why can't we use the Riccati equation to treat the asymptotic (1 f ( t ) ) u= 0 ? behavior of the solutions of U"
+ +
1.18. M o r e Refined Asymptotic Behavior
Now that we have demonstrated that w + 0 as t + co, let us see if we can obtain some more accurate bounds. From the equation (1.18.1)
1.19. l m f z d t <
27
00
we have ( 1.1 8.2)
Hence
< lgl+4 Clearly,
Ig I
=
I
ce-2'
+
e-2'
It
(1.18.3)
fi
e2ty dt,
0
dt,
.
(1.18.4)
A direct calculation shows that e-2'
St
e2t11 g
0
I dt,
< I c I e-t + 2e-t f e t l I f I dt, ,
(1.18.5)
0
upon using the right-hand bound on the integral. Hence, for small 1 c 1 we have lg I
< 2 [I c I e-t
+ e-t
fet1 0
I ~dt,]. I
(1.1 8.6)
Exercise
1. Why did we deliberately replace e-Zt by 1.19. J" f Z d t
<
e-1,
a weaker estimate ?
03
If we use the foregoing inequality (1.18.6) and iterate, we can obtain a still stronger result. Without loss of generality, take c = 0 since the J,,t eZL1gdt, in most cases of interest. term ce+l is much smaller than cZL Furthermore, as indicated below, we are really interested in demonstrating the existence of a solution of (1.17.3) with the property that w -+ 0. Then we have
(1.19.1)
FIRST- A N D SECOND-ORDER DIFFERENTIAL E Q U A T I O N S
28
upon using the Cauchy-Schwarz inequality. Hence w
=
ePzt
+ o (It e-(t-tl) I f 1'
1'
eZt'fdt,
0
+
(1.19.2)
dt,).
0
1
Since u'/u = 1 w , we see that it is the behavior of Sow dt, which we want in order to estimate that of u(t) as t -+00. We have
(1.19.3) Integration by parts yields
j t w dt, 0
=
1It
l t f dt, e-2(t-ti)f(t,) dt, 2 0 2 0 ~
j"
Hence U =
0
( s t j Zdt,).
(1.19.5)
c1
as t --?r co. We can thus assert that i f f are two solutions of
+0
as t
+ 03
and
(1.19.4)
0
+ ;j h t , + + o(l))
exp ( t
U"
+
S" f
- (1 4 - f ) u = 0
dt
< 00,
there
(1.19.6)
possessing the asymptotic forms: u1
-
u2 =
exp ( t
+ 1J i f d t , + o(l)),
exp ( - t
-
1
jLfdt, 0
+ o(1)).
( 1.19.7)
We can obtain the behavior of u2 from u1 by using the representation zll J;" dt/uI2for a second solution of (1.17.1) in terms of a first.
u2
Exercises
s"
1. Show that i f f + 0 and 1 f l 3 dt < co, there are two solutions of u" - (1 f ( t ) ) u = 0 possessing the asymptotic expansions
+
29
1.20. THE SECOND SOLUTION
2. Simplify this in the case where f
=
sin t/l/F.
3. Consider the case where f possesses the convergent expansion f = a,/t + a2/t2+ --.for large t, and obtain corresponding expansions for u1 and u2 . Are the expansions convergent ?
+
4. What is the eventual behavior of the solution of v' v 2 - 1 -f(t) = 0 with v(0) > 1 iff + 0 as t + 03 ? Hint: Consider the behavior of v in the (v, t)-plane. 1.20. The Second Solution
T h e equation
+
v'
v2 -
1 -f(t)
=
(1.20.1)
0
appears to have a solution that approaches -1 as t + GO, provided that f -+ 0 as t -+ 03. We can establish this indirectly as above by using the properties of u p . Can we demonstrate the existence of this solution directly ? Set v = - 1
+ w , obtaining the equation w'
=
2w - w2 + f ( t ) .
(1.20.2)
If we convert this into an integral equation in the usual fashion, obtaining w = e2tc
+
e2t
J: e-2t1[f(tl)
-
d]dt,
,
(1.20.3)
we see that the presence of ept causes some trouble. Hence, let us write w = eZtc + e2t -
ezt
J
m
e-ztt'[f(tl>
J't e - z t l [ f ( t l )
Since we want w to approach 0 as t -+ convergent. Choose c so that c
+J
m 0
-
w2] dt,
0
GO,
- to21
dt,
.
(1.20.4)
the infinite integral should be
e-z'l[f(tl) - to21 dt, = 0.
(I .20.5)
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
30
Alternatively, we can begin with the integral equation w
OD
= eZt t
e-2t1[zo2 - f ( t l ) ] dt,
(1.20.6)
and employ the method of successive approximations to establish the existence of the desired solution,
[
'x
w o = -eZt
eUzt'f(t,)dt, ,
(1.20.7)
We leave it to the reader to carry out the details. We shall employ this device again in Chapter 3. 1.21. The Liouville Transformation
Let us now engage in some analytic chicanery which will reduce the equation of (1.12.1) to an equation whose coefficients are close to constants in many cases of significance. T o simplify the algebra, let us take the equation in the form u"
*
d ( t )u
===
0.
(1.21.l)
Perform the change of variable ( 1.21.2)
an idea due to Liouville. T h e n a simple calculation shows that (1.21.1) becomes d2u
-+-ds2
d ( t ) du &u-0. & ( t ) ds
(1.21.3)
We then see that there are two cases of importance where (1.21.3) is close in form to an equation with constant coefficients. T h e first is I in this interval," i.e., where the t-interval is finite and I a'(t)i a(t) is slowly varying. T h e second is where a(t)+ co as t + 00 and 1 a'(t)/a2(t)l 0 as t + co. As we know from what has preceded, a sufficient condition for
<<
--f
* We
shall use the notation a
< 6 to mean that a is very much smaller than b.
1.22. ELIMINATION OF MIDDLE TERM
closeness of the solution of (1.21.3) to the solution of ~ ” ( s ) over the infinite interval is that
31
+ ~ ( s )= 0 ( 1.21.4)
If a’(t) > 0, a(t) > 0, we see that m
f
$#o(t)dt
=
loga(t)]
co
= 03,
(1.21.5)
if a(t).+03 as t + 03. Hence, we require a further transformation of the equation in this case of frequent occurrence. Let us consider first the case u” + a2(t)u = 0. T h e equation with the minus sign is easier to consider. Exercises
1. Use the Liouville transformation to establish the following result of Gusarov: If u” ( I + f ( t ) u = 0 with 1 + f ( t ) >, a2 > 0, J“ lf”(t)\ dt < 00, then u is bounded as t + co.
+
2. Hence, discuss the boundedness of the solutions of U“
for 0
+ (1 + (cos
t”)/tb)
u =
0
< a, b < 1. See
R. Bellman, “Boundedness of Solutions of Second Order Linear Differential Equations,” Duke Math. J., Vol. 22, 1955, pp. 51 1514.
3. Show that if we set u = g exp[i J ( d t ) / ( p g 2 ) ]then , -(pu’)’ + qu = 0 is converted into ((pg’)’/g)- (I/pg4)= q. For applications of this result, see
J. Walter, “Bemerkungen zu dem Grenzpunktfallkriterium von N. Levinson,” Math. Zeit., Vol. 105, 1968, pp. 345-350. J. F. deSpautz and R. A. Lerman, “Equations Equivalent to NonVol. 18, linear Differential Equations,” Proc. Amer. Math. SOC., 1967, pp. 441-444. 1.22. Elimination of Middle Term
Consider the general linear equation u”
+ p(s) u‘ + q(s) u = 0.
(1.22.1)
32
FIRST- A N D SECOND-ORDER DIFFERENTIAL EQUATIONS
T h e change of variable
u = exp ( - s j1, d s ) v
(1.22.2)
converts this into the equation v"
+ (q
-
p'/2 - p74) v
=
(1.22.3)
0,
as a direct calculation shows. Thus, if we set
(1.22.4) we see that v satisfies the equation
(1.22.5) If the integrals
(1.22.6) converge, we can apply the results of Sec. 1.16 to conclude that the solutions of (1.22.5) are asymptotically given by the solutions of wn
+w
==
(1.22.7)
0
as s + co. From this we can determine the asymptotic behavior of the solution of (1.21.1), as we do below. Prior to this, let us see what the conditions of (1.22.6) are in the 1 t-variable. Since s = fo a(tl) dt, , ds/dt = a(t), we see that they are
< 00.
(1.22.8)
Exercises
+
1. Determine the asymptotic behavior of the solutions of U" tau = 0, a > 0; u'' e"'u : 0, a > 0; U" (log t)"u = 0, a > 0. T h e last equation illustrates the fact that it may be necessary to apply the Liouville transformation twice.
+
+
2. Determinc the asymptotic behavior in the cases where a
3. Determine the asymptotic behavior of the solution of ZL"
f(I
+ 1 j t " ) u -= 0,
I > a > 0.
< 0.
1.25.
U”
+ (1 + f ( t ) ) u = 0 ; ASYMPTOTIC BEHAVIOR
33
1.23. The W K B Approximation
Retracing our steps to the original equation, (1.21.1), we see that under appropriate conditions on a(t), e.g., those of (1.22.8), a useful approximation to the solution of u”
+ &(t) u = 0
as t -+ 00 is given by u
exp [i
1’
a(tl)dt,] /a(t)l/z.
(1.23.1)
(1.23.2)
1.24. The One-dimensional Schrodinger Equation
I n quantum mechanics, it is often a question of solving an equation of the form (1.24.1)
where t is constrained to a finite interval. Carrying out the foregoing changes of independent and dependent variable, we obtain the equation (1.24.2)
Hence, if the functions (1.24.3)
are well behaved over the corresponding finite s-interval, we can write m
v
=
v,,
+ C hkvk,
(1.24.4)
k=l
a convergent series, and thus, retracing our steps, obtain an approximation for u. 1.25. u”
+ (1 + f ( t ) ) u = 0; Asymptotic Behavior
Using the Liouville transformation, we can readily determine the asymptotic behavior of the solutions of u”
+ (1 + f ( t ) ) u = 0
(1.25.1)
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
34
in an important case where f
that
+
0 but J" I f I dt
=
co. Let us suppose
I f ' I dt < co.
Jm
(1.25.2)
Performing the change of variable s =
1'(1 0
(1.25.3)
+ f ) l I 2 dt, ,
we obtain the new equation ) zdu+ u = o .
(1.25.4)
T h e condition (1.25.5) is met if (1.25.2) is satisfied and f --t 0 as t + 00. Hence, (1.25.4) has solutions which are asymptotically of the form u1
=
[I
u2 = [l
+ o(l)] cos s, + sin s.
(1.25.6)
0(1)]
This means that (1.25.1) has solutions of the form f 1.25.7)
If, in addition, we suppose that u1
=
[I
u2 = [I
+ 0(1)3
s" f COS
[t
dt
< m, we can write
+ Jtfdt1:2],
+ o(1)l sin [I+ jtfdt1/21. 0
Exercise
1. Determine the asymptotic behavior of the solutions of U"
4-(1
+ l i t u )u = 0
for
a
> 0.
(1.25.8)
1.26. ASYMPTOTIC SERIES
35
1.26. Asymptotic Series
T h e techniques discussed in the foregoing pages enable us to find the principal terms in the asymptotic expressions for solutions of equations of the form 1 U" - ( 1 ta) u = 0. (1.26.1)
+
Suppose that we wish to obtain more refined estimates. I t is tempting to employ the following procedure. Set (1.26.2)
where the ai are constants, and substitute in (1.26.1). Equating coefficients of powers of l / t we obtain the recurrence relation n 3 1,
(1.26.3)
with a, = 1. Since
(
n2 a76 >I n ) a 7 6 - l = .(
-
1) % - I
>
(1.26.4)
we see that a, >, (n - I)! Consequently, the series in (1.26.2) diverges for all t. Nonetheless, it can be used to calculate numerical values of u(t) for large t! It is a particular example of an asymptotic series. These series of paramount importance were introduced into analysis in a systematic fashion by PoincarC and Stieltjes. Let us now give a formal definition. We say that a series S ( t ) = a,
a +2 + -a$ + ...
(1.26.5)
is an asymptotic series for a functionf(t) as t -+ co if the coefficients are obtained in the following fashion: a,
=
limf(t),
a,
=
l i m t ( f ( t )- a,),
an
=
l i m t n [ j ( t )- a, - ... - an-1
t-tm
t+m
t+m
31.
(1.26.6)
FIRST- A N D SECOND-ORDER DIFFERENTIAL EQUATIONS
36
We write (1.26.7)
When we use the notation (1.26.8)
we mean that e - t f ( t ) possesses the asymptotic series Cn=, a,tpn. T h e concept of asymptotic series is of particular significance in the theory of differential equations, where it is often not difficult to show the existence of generalized solutions in the form of asymptotic series. A fundamental problem is then that of establishing, under suitable hypotheses, that these correspond to actual solutions. m
- En=, m
1. Show that i f f
and thatf,
-
Clf
ant-*, g
+ czg -
+
-
[ao/+p (a,&
2. If a, # 0, show tl1atf-'
-
Exercises
+
-+
-Zr=,
m
(clan 11=0
b,t-",
then
+ czb,) t r n
+ a,b,)/t + -..I. [lla,, - a,/a,t
SL
m
+ ...I.
-
a,/t2 a,/t3 *.-, show that f dt, -a,/t - a,/2t2 *... Hence, show that i f f a, -1- a,/t ... and f ' possesses an asymptotic series, then f ' ---ul/t2....
3 . If f
+
4. Show by means of a specific example that f may possess an asymptotic series without f ' possessing an asymptotic series. Hint: Consider a case wheref 0 -1O / t ....
-
+
5. Let
for t
3 0. Show that
and obtain a linear differential equation satisfied by f ( t ) .
1.27. THE EQUATION
0‘
= P(u, t)/q(o, t )
37
6. Show that u” - (1 + I/t2)u = 0 possesses two solutions with the asymptotic series obtained formally by converting the differential equation into a suitable integral equation and proceeding as in the foregoing pages.
7. Obtain the detailed asymptotic behavior of the solutions of U“ - (1 + I / t ) u = 0 . For the foregoing and additional references, see R. Bellman, Stability Theory of DafJerential Equations, Dover Publications, New York, 1969. For asymptotic series associated with differential-difference equations, see the book by Bellman and Cooke previously referred to. For the delicate question of deciding how to use asymptotic series effectively, see
G. M. Roe, “An Accurate Method for Terminating Asymptotic Series,” J. SOC.Indust. Appl. M a t h . , Vol. 8, 1960, pp. 354-367.
1.27. The Equation u’
= p(u, t)/9(u, t)
Let us now show that we can analyze the nature of the solutions of
where p and q are polynomials in their arguments, using quite simple analytic and geometric ideas, provided that we agree to restrict our attention t o those solutions which are continuous for all large t. T h e methods are quite different from those we applied to study the Riccati equation. Our first result is that every such solution is ultimately monotonic. We must show that ti’ cannot vanish for a series of t-values of arbitrary magnitude, unless u = c, a constant. T h e proof is by contradiction. Suppose that u‘ vanishes at the sequence of points (tk>,where t,+ GO. Then the solution u and the curve z, = v ( t ) defined by the equation p(v, t ) = 0 intersect at these points. Since p is a polynomial in z, and t , the curve defined in this fashion possesses only a finite number of branches. Hence, the solution u must intersect one of these branches infinitely often. Without loss of generality, we may suppose that p and q possess no common factor and thus, that q possesses a constant sign in the immediate neighborhood of a solution of p = 0.
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
38
T h e branches of the form
p
=
0 which extend to infinity consist of curves of v
(a)
= c,
(1.27.2)
where q(t) is ultimately monotonic. Let us consider the second case first and show that the solution cannot intersect one of these branches infinitely often. For t sufficiently large, the points of intersection cannot be maxima or minima. Drawing a figure will convince the reader of this. Hence, the points of intersection of u and v are points of inflection. But again, a simple diagram will show that two points of intersection which are points of inflection must be separated by a point of intersection which is not an inflection point-a contradiction. We will provide an analytic proof below to supplement this intuitive geometric proof. 1,et us next consider the possibility of intersection of u with curves of the form v = c. These intersections must again be points of inflection of u and have one of the four forms shown in Fig. I . 1. We can eliminate (a) and (b) since u' changes sign in the neighborhood, but p(u, t)/q(u, t ) does not change sign if u c or u 3 c. If (c) and (d) occur, they can occur only a finite number of times since u can return to intersect v = c only by means of intersections with curves of the form v = v(t)or by means of intersections of type (a) and (b) with curves of the form = c1 .
<
(b)
(C)
Figure 1.1
1.28. Monotonicity of Rational Functions of u and t
We can establish a stronger result using some simple analysis. Let us show that if u is a solution of (1.27.1) continuous for all large t , then any rational function of u and t , (1 -28.I )
1.29. ASYMPTOTIC BEHAVIOR OF SOLUTIONS OF u' = p(u, t)/q(u, t )
39
is ultimately strictly monotone unless Z(u, t) = 0 is satisfied by a solution of (1.27.1) or unless h(u, t)is constant along a solution of (1.26.1). We have dh p - +(u, t) _ -- -ah+ +a-h=
dt
au q
at
(1.28.2)
s(u, t ) *
If dh/dt is not of constant sign as t -+ co, it either vanishes infinitely often or becomes infinite infinitely often as t + co,where u is a solution of (1.27.1). Consider first the case where dhjdt = 0 infinitely often, which means that one branch of Y ( U , t) = 0 has an infinite number of intersections with u. For t t, , this branch has an expansion of the form u = a,tco
+ a#' + ...,
co
> c1 > ...,
a,
(1.28.3)
#0
At the intersection 1bltd'
+ ...,
do
> dl > ..., 6,
f 0
(1.28.4)
upon using the series in ( .28.3). Returning to (1.28.3), we have dv dt
- = aoc0tC"--l
+ alcltC1-l + -...
(1.28.5)
From this we see that for large t we have one of the following conditions: (1.28.6)
The persistent inequalities lead to a contradiction, as we see most easily from a figure. Hence at the points of intersection we must have duldt = dv/dt.If this holds for infinitely many t, we must have bo
= a,~,,
do
= C,
-
1,
6,
=
alcl
, dl = ~1
-
I ,...,
(1.28.7)
which means that dujdt = dvjdt. Hence, Y ( U , t) = 0 contains a solution of (1.27.1), which means that h(u, t) is constant for this solution. T h e case where dhjdt becomes infinite infinitely often is treated in exactly the same way. 1.29. Asymptotic Behavior of Solutions of u'
= p(u, t ) / q ( u , t)
We are now ready to demonstrate that any solution of (1.29.1)
40
FIRST- A N D SECOND-ORDER DIFFERENTIAL EQUATIONS
continuous for t 2 to , is ultimately monotonic, as are all of its derivatives, and satisfies one or the other of the relations (1.29.2)
where p is a polynomial in t, and c is an integer. Consider the expression q(u, t)u' - p(u, t ) which contains terms of the form altmu7' or bltnLunu'.From the foregoing discussion we know that the ratio of any two such terms is ultimately monotone and thus approaches a limit as t co. T h i s limit may be 0, fCQ, or a nonzero constant. Since the equation in (1.29.1) holds, there must be at least two terms whose ratio approaches a nonzero constant. If only one contains u', we obtain --f
UIUnpf
N
c1
.
(1.29.3)
If both or neither contain u', the result is u
N
(1.29.4)
c2t"/Q,
where p , q are rational. We will present the detailed analysis as a series of exercises. Exercises
1. Show that (1.29.3) leads to different results where n n n n
f-1, = -1 , =-I, f - I ,
m#+I, m f f l ,
m=+1, m=+1.
- + --
2. Show that Exercise ](a) leads to unt1/(n I ) and Exercise I(d) to un+'/(n 1 ) c1 log t .
+
3 . Show that Exercise l(b) leads to log u
which requires further consideration. Take 1 the equation
where the
cltl-m/(l
+ dl
+
cltl-m/(l - m ) d, , m > 0 and consider
p i and qi are polynomials in t . Show that
> 0,
m)
-
Y =s
c1 > 0. (Why is it sufficient to consider only the case I
4. Hence if c,
-
-
m
+ 1 if
> 0 ?)
1.29. ASYMPTOTIC BEHAVIOR OF SOLUTIONS
OF
U’
= P(u. t)/q(u, t)
41
for some a. Integrating, show that log Ic
=p(t)
+ CQ log t + O ( l / t ) .
5 . Show that c1 < 0 may be treated by replacing original equation.
zi
by l/u in the
6 . Consider Exercise I(c). There are two terms, atbucu’and dtb--luc+l, of equal order. Show that there is no other term of equal order.
7. Let
7 be any third term. Th en the quotient (atbuCu’ - dtb-1uc+1)/7 tends to a limit as t + CO. There are now two possibilities: (a) There is a third term whose order is equal to that of the difference between the two principal terms. (b) There is no such third term.
8. Consider the first possibility. T h e n either
or
Show that both of these lead to the stated result in (1.29.2).
9. Complete the analysis for part (b) of Exercise 7. More detailed
results can be obtained both for the solutions of polynomial equations of the form p ( t , u , u’)= 0 and for equations of the form u” = p(u, t)/q(u,t ) . See
G. H. Hardy, “Some Results Concerning the Behavior at Infinity
of a Real and Continuous Solution of an Algebraic Differential Equation of the First Order,” Proc. London Math. SOC.,Vol. 10, 1912, pp. 4 5 1 4 6 8 . R. H. Fowler, “Some Results on the Form Near Infinity of Real Continuous Solutions of a Certain Type of Second Order Differential Equation,” Proc. London Math. SOC.,Vol. 13, 1914, pp. 341-371. R. Bellman, Stability Theory of Da8erential Equations, Dover Publications, New York, 1969.
10. Use the foregoing techniques to analyze the behavior of the solutions of 71’ + 7 1 ~- a2(t)= 0 under various assumptions concerning a(t).
FIRST- A N D SECOND-ORDER DIFFERENTIAL EQUATIONS
42
Miscellaneous Exercises
1. Let p , ( t ) and q,,,(t) be polynomials of degrees n and m respectively whose coefficients are determined by the condition that
+
where f ( t ) = f,,+ f i t .--.These polynomials determine a Pad6 table and f ( t ) q,,,/p, is called a Padk approximation. Are p , and q,,, uniquely determined by this condition ?
2. Determine Pad6 approximants for p , ( t ) et
el.
Hint: Start with
+ qm(t) = tm+n+l + ...,
and differentiate to obtain recurrence relations.
+
+
+
+
3. Determine Pad6 approximants for (1
a,t a2t2)1/2.Hint: Uniformize by means of trignometric functions and count zeros and poles.
4. Determine Pad6 approximants for (1
+
a,t a,t2 u $ ~ ) ‘ / Hint: ~. Uniformize by means of elliptic functions. For the foregoing and additional references, see
R. Bellman and E. G. Straus, “Algebraic Functions, Continued Fractions and the Pad6 Table,” Proc. Nut. Acad. Sci., Vol. 35, 1949, pp. 472476.
+
+
5. Consider the differential equation U” au’ bu = 0, a, b > 0. Show that constants a, , a2 can be found such that u2 aim' U ~ U ’ ~ is positive for all nontrivial values of u and u’and, in addition, such
+
+
that
where a 3 , a4 > 0.
u’+ 0 as t -+co and, indeed, that some positive constants, a 5 , a 6 , without calculating the explicit solutions. This is a particular application of the powerful “second method” of Lyapunov we shall discuss in Volume I1 in connection with the topic of differential inequalities.
6. Hence, show that u2
+ ur2< a5ecaG1 for
u,
MISCELLANEOUS EXERCISES
7. Consider the equation u”
+
au’ and suppose that the roots of r2 parts. From the relations
s
m
+ + bu) dt = 0,
~ ( u ” au
0
43
+ bu 0, u(0) = c1 , u’(0) = c, + ar + b = 0 have negative real :
s
m
u’(u”
0
+ au’ + bu) dt = 0,
Sr
plus integration by parts, obtain expressions for ST u2 dt, uI2 dt as quadratic forms in c, ,c2 without using the explicit form of u.
sr
u2 dt as a quadratic form in c, and c2 , derive a necessary and sufficient condition for the roots of r2 ar b = 0 to have negative real parts in terms of a and b.
8. From the expression for
+ +
9. Similarly, derive necessary and sufficient conditions that the roots of r3
+ a1r2+ a,r + aR= 0 have negative real parts. See
A. Hurwitz, “Uber die Bedingungen unter welchen eine Gleichung
nur Wurzeln mit negativen reellen Teilen Besitzt,” Math. Ann., Vol. 46, 1895 (Werke, Vol. 2), pp. 533-545. H. Cremer and F. H. Effertz, “Uber die Algebraische Kriterien fur die Stabilitat von Regelungssystemen,” Math. Ann., Vol. 137, 1959, pp. 328-350.
10. Show that if a(t) 3 0 then all solutions of remain bounded as t --t co. Hint: d(u’2
+ u2)/dt = 2u‘u“ + 2uu’
=
2u’(--a(t)
u”
14‘ -
+ a(t)u’ + u = 0
u)
+ 2uu‘.
11. By consideration of the equation u“ + (2 + d)u‘ + u = 0, show that the condition that a(t) 3 a, > 0 is not sufficient to ensure that all solutions approach zero as t -+ a.
12. By means of a change of variable, show that f ( t ) - 0
as t + co cannot be a sufficient condition to ensure that all solutions of f ( t ) u ” u‘ u = 0 approach a solution of u’ u = 0 as t -+ co.
+ +
+
13. If a ( t ) > 0, show that no solution of u” - a(t)u = 0 can have more than one zero. Hint: Without loss of generality, let u ( t ) be negative between the two points t, and t, where u(tl) = u(tJ = 0. Let t, be
a point where u ( t )assumes a relative minimum value for t, < t < t , . Show that this leads to a contradiction upon using the fact that u“ - a(t)u = 0.
+ +
+
(1 E)U’ u = 0, u(0) = 1, u’(0) = 0, where E is a positive constant. Write the solution as u(t, E ) to indicate
14. Consider the equation EU’‘
FIRST- A N D S E C O N D - O R D E R DIFFERENTIAL EQUATIONS
44
thc dependencc on c. Does lime+ou(t, c) exist? Does it satisfy the differential equation u‘ u = 0 ? What is the initial condition ?
+
15. Consider the Green’s function k ( t , t l ) associated with u” =f, zi(0) -- ZL( T ) 0. Show that T
<
for 0 t , t, , t, kernel”. See
< 7’.
This shows that k ( t , t l ) is a “reproducing
N. Aronszajn, T h e Theory of Reproducing Kernels,” Trans. Amer. Math. IJ’oc.,Vol. 68, 1950, pp. 337-404. X. Aronszajn and K. T . Smith, “Characterization of Positive Reproducing Kernels. Applications to Green’s Functions,” Amel.. J . Math., Vol. 79, No. 3, July 1957, pp. 611-622.
16. Establish this result for the Green’s function of the second-order a(t)u = f,u(0) : u( 7’)= 0. linear differential equation u“
+
17. Show that the Green’s function of the foregoing equation can be characterized by the condition that k ( t , s) is a solution of the linear differential equation in (0, s) and (s, I), with a discontinuity in the derivative a t t
I-=
s, namely,
k’(s
~
0, s)
~
F(S
+ 0, s)
=
1
for 0 -: s .’.. 1.
18. Sho\v that the determinantal equation un
UI
u
21.;
ut?
112
has the general solution ZL -= clwl relations between the solutions of coefficient functions, p and y.
+ c 2 u 2 , and hence zi“ -1 pu‘ + qu = 0
19. Lkterniine the Green’s function associated with 0. ~ ( 0 ) 0 , j”; 11 dt ~
zi“
establish and the
f a% = f ( t ) ,
~
20. If p ( t ) > 0, andf,(t),fi(t) are monotone increasing in
[a,
01, we have
MISCELLANEOUS EXERCISES
45
T h e upper sign holds if fi and f2 are both increasing or both decreasing. T h e lower sign applies if one function is decreasing and the other increasing (Cebycev). For some applications, see
J. A. Shohat and A. V. Bushkovitch, “On some Applications of the Tchebycheff Inequality for Definite Integrals,” J. Muth. Phys., Vol. 21, 1942, pp. 211-217.
+
21. T h e equation u’ - u2 = -2/t2 O ( l / t ) as t $ 0 implies that either u = l/t 0(1) as t $ 0 or u = -2/t O(1) as t $ 0 (Korevaar).
+
22. T h e equation
u‘ - u2 =
+
O(l/t) as t $ 0 implies that either as t $ 0.
u = O(1og l / t ) as t $ 0 or u = - 1 j t
2. Koshiba and S. Uchiyama, “On the Existence of Prime Numbers in Arithmetic Progression,” Proc. Japan Acud., VoI. 42, 1966, pp. 696-701.
23. Let {xk>, {fJ, and {zk},k = 0, 1,..., m be real valued sequences and let { z k )be nonnegative. If for k = 0, 1 ,..., m, X k
+
c
22% 9
Ogi
then
for k
= 0,
1,..., m. See
G. S. Jones, “Fundamental Inequalities for Discrete and Discontinuous Functional Equations,” J. SOC.Indust. Appl. Math., Vol. 12, 1964, pp. 43-57. 24. If u”
+ (1 + f ( t ) ) u
= O,f(t)
- 0 as t
co,J“ l f ’ ( t ) / dt < co,then
FIRST- A N D SECOND-ORDER DIFFERENTIAL EQUATIONS
46
26. If
U"
+ g(t)u = 0, g(t) 3 a2 > 0, g(t)-1/4convex in t, then
(M. Zlamal.)
27. T h e general solution of u" +
is u =
1 -g"(t) -)u=o (m g(t) -
dt,
clg(t)sin
(M. Rab.)
28. Letf(n) be defined for n f(n) = u0
+
=
+ a2n(n
-
0, 1, 2,..., and set
Show that
(Newton Interpolation Formula).
29. Write
Show that
where now
30. Hence show that
+
1)/2!
4
7
2 -
l)(n
-
+ ...
2)/3!
47
MISCELLANEOUS EXERCISES
See
R. Bellman, “A q-version of the Newton Interpolation Formula and Some Eulerian Identities,” Boll. U.M.I., Vol. 16, 1961, pp. 285-287.
+
31. Consider the two equations (a) u” + f ( t ) u = 0, (b) u’’ g(t)v = 0. Suppose that there exist two solutions of (a), u l , u 2 , such that J“ (u12 us2)/f - g 1 dt < co. T h e n every solution of (b) can be written ZI = clul czup o( 1 uI 1 1 u2 I). (Wintner.) See
+
+
+
+
R. Bellman, “On a Generalization of a Result of Wintner,” Quart. Appl. Math., Vol. 16, 1958, pp. 431432.
+ +
32. Consider the equation u” (1 Ag(t))u = 0, where J: I g ( t ) /dt < 00. Let r(h) and e(h) be defined by the asymptotic relation lim {u(t,A) 2-
m
-
r(A) sin(t
+ B(h))) = 0.
Show that if g is real, then r(A) and d(X) are analytic functions of A for real A. See
+ +
Jr
R. D. Kemp and N. Levinson, “On u‘’ ( 1 Ag(x))u = 0 for 1 g(x)l dx < co,” Proc. Amer. Math. SOC.,Vol. 10, 1959, pp. 82-86.
+
b
33. Obtain the solution of y’ ay = J o ydx, , y(0) = c. Hint: First b set Soy dx, = m. Solve for y in terms of m and then obtain an equation for m.
34. Similarly, solve y’
+
Y’
4U Y
UY =
=
j1x1y dx, ,
Y(0) = 1
jlf(xl)y dx, ,
y(0) =: 1 .
+
35. Show that y’ uy = p(x), where p(x) is a polynomial of degree k in x, always has a solution which is a polynomial of degree k,provided that a # 0. What happens if u = 0 ? 36. Obtain the solution of u =
1
+ j1exp(k 1 x 0
-
x1 1) u(xI) dx,
by reducing the equation to a second-order differential equation subject to a two-point boundary condition.
48
FIRST- AND SECOND-ORDER DIFFERENTIAL EQUATIONS
37. Show that the factorization U“
+ u = (-dtd + i)(-
d - i) u dt
is not unique by means of the example d d ( t 4-tan t ) ( dt
-
tan t
Are there any other factorizations of
1
u =
U“
u“
+ u.
+u?
38. If u1 , u2 are two linearly independent solutions of u” + f ( t ) u then we may write c o s ( g ( t ) - el,
=
where r(t)
=
= 0,
(u12
+
u22)1/2,
T h e functions ( u , g ) are occasionally called a “polar pair” and the representation a “polar representation”. For applications of this representation, see
I. I. Kolodner, “Phase Shift of Solutions of Second-Order Linear Ordinary Differential Equations and Related Problems,” J . Math. Anal. Appl., Vol. 4, 1962, pp. 422-439. 39. 1,et sgn t : + 1 i f t > 0, -1 i f t totic behavior of the solutions of (a) u” (b) u” (c) U”
< 0,Oif t
= 0.
Study the asymp-
+ a,u’ + a2zi = sgn(6,u’ + b,u), + (sgn u)u 0, + (sgn u)u’ + u = 0. =
40. Establish Steffenson’s inequality: Let f ( t ) be nonnegative and monotone decreasing in [a, 61. T h e n
lbf b-c
when c
dt
<
Jb fl
f g dt
<
Sfl+‘
a
f dt
b
J,g dt.
Jz
Hint: Introduce the function u(s), defined by fg dt consider the differential equation satisfied by u.
=
Jzf dt, and
MISCELLANEOUS EXERCISES
49
41. Establish the following generalization. Let f ( t ) be nonnegative and , g 3 0 in [a, b] and monotone increasing in [a, 61, f ~ L p [ ab], l/q = 1, p > 1. Then
g ~ L q [ ab,] , where Ilp
where c See
=
+
b
(sag dt)p.
R. Bellman, “On Inequalities with Alternating Signs,” PYOC. Amer. Math. SOC., Vol. 10, 1959, pp. 807-809.
42. Let
u = f ’ ( t ) be monotone increasing,f(O),f’(O)
f
>j
(-l)n+lt(an)
n=l
if a, 3 a2 2 ... 3 a,
> 0 by
(i
1
3 0. Show that
( - 1 ~ + l an
n=l
referring to the figure below.
U
t
0 43. Obtain the result
c n
qc
1,
/ n
(-1)j-l
Wjf(Uj)
\
wjuj
(-1y-1
j=1
j=1
where 1 w1 >, wg 3 -.-3 w, >, 0 (Olkin’s inequality), by suitable specialization of Exercise 40.
44. Show that if u(t), 0 function with
ku(t)”
< t < a,
> (u
-
t)”
Iu t
is any nonnegative Ln-integrable ( a - s) u(s)” ds
<
> 0,
where k, m, n,p , q > 0 and m > n, then a c f ~ ( O ) ~where , c and Y are constants depending only on k, m, n,p and q. For the foregoing and additional references, see
FIRST- A N D SECOND-ORDER DIFFERENTIAL EQUATIONS
50
R. Bellman, “On a Differential Inequality of Cesari and Turner,” Rendiconte del Circolo Matematico di Palermo, Ser. 11Tom0 VII, 1958, pp. 1-3.
+
45. Show that u,, uyy + a2(x,y)u = 0 can be transformed into us, + u t t u = 0 by a suitable change of independent variables provided that log a(x, y ) is a harmonic function of x and y . See
+
+
R. Bellman, “On a Liouville Transformation for u, uyy a 2 ( x , y ) u1 0 , ” Boll. U.M.I., Vol. 13, 1958, pp. 535-538. 46. Obtain the result of Blaschke-Pick. Let u ( t ) and v(t) be convex functions of t for 0 t 1, normalized by the conditions
< <
u(1)
=
0,
v(0) = v(1)
=
0.
u(0)
Then
Ji uv dt 3 112.
=
Hint: u convex means that u“ >, 0 or that u” = f ( t ) , f ( t ) 3 0. 1 Hence u(t) = k(s, t ) f ( s )ds, where k(s, t ) is an appropriate Green’s function. See
so
R. Bellman, “Converses of Schwarz’s Inequality,” Duke Math. J., Vol. 23, 1956, .pp. 429-434. 2. Nehari, “Inverse Holder Inequalities,” J . Math. Anal. Appl., Vol. 21, 1968, pp. 405420. E. F. Beckenbach and R. Bellman, Inequalities, Springer-Verlag, Berlin, 1961.
+
a(t)u = 0, 47. Consider the two-point boundary-value problem u” u(0) = c, , u( T ) = c, , and the following scheme for successive approximations. Let u’(0) = 6, and consider the solution of the initial value problem u” -1 a(t)u = 0, u(0) = c, , u’(0) = b, . Let u’( T ) 6, be determined in this fashion. Consider the initial value problem U” a(t)u 0, u( T ) = cz , u’( T ) = b, , and let u’(0) = b, be determined in this fashion. Does this procedure converge? See :
+
1
R. Bellman, “On the Iterative Solution of Two-Point BoundaryValue Problems,” Boll. U.M.I., Vol. 16, 1961, pp. 145-149. R. Bellman and T. Brown, “On the Computational Solution of Two-Point Boundary-Value Problems,” Boll. U.M.I., Vol. 19, 1964, pp. 121-123.
BIBLIOGRAPHY A N D COMMENTS
51
48. Can one find the convenient analytic expressions for the Pad6 1 /t2)u= 0 ? approximants of the solutions of u” - (1 49. If u(a) = 0, u‘ EL2(a,b), then J: (Opial’s inequality.) See
+ [ uu’ 1 dt < 1/2(b -
JE
1 u’ I 2 dt.
D. W. Boyd, “Best Constants in Inequalities Related to Opial’s Inequality,” J. Math. Anal. Appl., Vol. 25, 1969, pp. 378-387. Bibliography and Comments 31.1. For an elementary account of analytic and computational aspects of ordinary differential equations, see R. Bellman, Modern Elementary Differential Equations, Addison- Wesley, Reading, Massachusetts, 1968. $1.3. This inequality plays a fundamental role in the modern theory of stability of differential equations. See R. Bellman, Stability Theory of Differential Equations, Dover, New York, 1969, where further references to its use may be found. It also plays an important part in the derivation of a priori inequalities in the theory of partial differential equations. See, for example, p. 115 of
R. Lattes and J. L. Lions, The Quasi-Reversibility Method and Applications, Dunod, Paris, 1967. (English translation, American Elsevier, New York, 1969.)
31.7. We shall discuss two-point boundary values again using the theory of invariant imbedding in Volume 11. See
T. Y. Na, “Transforming Boundary Conditions to Initial Conditions for Ordinary Differential Equations,” S I A M Review, Vol. 9, 1967, pp. 204-210. $31.8-1.9. For a detailed discussion of these questions, see
R. Bellman,Zntroductionto the Mathematical Theory of Control Processes: I . Linear Equations and Quadratic Criteria, Academic Press, New York, 1967.
91.10. A powerful method for obtaining analytic approximations to the Riccati equation depends upon the use of continued fractions and, more generally, Pad6 approximations. The method is originally due to Laguerre; see
E. Laguerre, “Sur la Reduction en Fractions Continues d’une Fonction qui Satisfait une Equation Differentielle Lin6aire du Premier Ordre dont les Coefficients Sont Rationnels,” J. Math. Pures Appl., Vol. 1, No. 4, 1885, pp. 135-165. See also
E. P. Merkes and W. T. Scott, “Continued Fraction Solutions of the Riccati Equation,” J . Math. Anal. Appl., Vol. 4, 1962, pp. 309-327. W. Fair, “Pad6 Approximation to the Solution of the Riccati equation,” Math. of Computation, Vol. 18, 1964, pp. 627-634.
52
FIRST- A N D SECOND-ORDER DIFFERENTIAL EQUATIONS
$1.1 1. For a presentation of the modern theory of inequalities, see
G. 11. Hardy, J. E. Littlewood, and G . Polya, Inequalities, Cambridge University Press, New York, 1934. E. 12. Beckenbach and R. Bellman, Inequalities, Springer-Verlag, Berlin, 1961. $1.12. For more intensive treatment, see the book by Bellman cited above in $1.3, and
K. Bellman, Perturbation Techniques in Mathematics, Physics, and Engineering, Holt,
Rinehart and Winston, New York, 1964. L. Cesari, Asymptotic Behavior and Stability Problems in Ordinary Differential Equations, Springer-Verlag, Berlin, 1959.
A powerful approach to thc solution of the second-order linear differential equation is bv(t))u = 0 can be expressed in afforded by the fact that the solution of u” - 2(a terms of the Wiener integrd of the functional exp[-b v ( x ( t l ) ) d t , - a t ] . See
+
Ji
M. Kac, “On Some Connections between Probability Theory and Differential Equations,” Proc. Second Berkeley Symposium on Mathematical Statistics and Probability, University of California Press, Berkeley, 1959, pp. 189-215. T. hl. Gelfand and A. M. Yaglom, “Integration in Functional Spaces and Its Application in Quantum Physics,” /. M a t h . Phys., Vol. 1 , 1960, pp. 48-69. The equation of ( I . 12.1) plays a fundamental role in wave propagation. We will study it in detail using the theory of invariant imbedding in Volume 11. For a discussion of the propagation of waves in a dispersive medium using various approximate techniques, see I<. D. Lewis, “Asymptotic Theory of Transients,” Electromagnetic W a v e Theory, Pergamon Prcss, Oxford, 1967, pp. 845-869.
$1.13. ‘I’liis counter-example is duc to A. Wintner. $1.14. This is a special case of the Dini-Hukuhara theorem given in Secs. 4.2 and 4.3 of Chapter 4. $$1.14-1.15. For the results of this section and the exercises, see the book cited in $1.3. $$1.16-1.18. See the books cited in $1.12. $1.16. Additional results may be obtained by means of the theory of invariant imbedding. See
R. nellnlan and R. Kalaha, “Functional Equations, Wave Propagation and Invariant Imhcdding,” J. M u t h . Mech., Vol. 8, 1959, pp. 683-704. G. M. Lying, “hlathematical Methods Suggested by Transport Theory,” Proceedings of S’yinposirrm on Trumpart Theory (R. Bellman and G. Birkhoff, editors), 1969. F. V. Atkinson, “Wave Propagation and the Bremmer Series,” J. M a t h . Anal. Appl., Vol. I , 1960, pp. 255-276. A papcr which brings to the forefront the physical significance of the W K B approximation is
H. Brernnier, “l’he LVKB Approximation as the First Term of a Geometrical-optical Series,” Thcrory of Z?fectroniagnetic Wavrs, A Symposium, pp. 169-1 70, Wiley (Interscience), New York, 1951.
BIBLIOGRAPHY A N D COMMENTS
53
See also
F. W. Montroll, “Markoff Chains, Wiener Integrals and Quantum Theory,” Comm. Pure and Applied Math., Vol. 5, 1952, pp. 415-453. 51.23. The physical significance of the “WKB approximation,” (inappropriately named, as usual) will be given in Volume I1 in the treatment of invariant imbedding. A more accurate title would include the names Liouville, Green, Jeffreys, and Blumenthal. See, for example,
F. W. J. Olver, “Error Bounds for the Liouville-Green (WKB) Approximation,” Proc. Cambridge Phil. Soc., Vol. 57, 1961, pp. 790-810. 51.24. See J. J. Mahoney, “The Reflection of Short Waves in a Variable Medium,” Quart. Appl. Math., Vol. 25, 1967, pp. 313-316.
If the function we have written as a z ( t ) changes sign in the interval of interest, the situation is far more complex. Study of these problems of great physical interest was initiated by Langer. T h e basic idea is a simple but powerful one. Instead of using U” a2(to)u= 0 as a comparison equation for I t - to I 1 , Langer used the equation u“ c,(t - t,,)u = 0 which can be solved explicitly in terms of Bessel functions. T h e idea can be extended to the case where a 2 ( t )E cz(t - to)*for I t - to I 1 . See
+
<<
+
<<
A. ErdClyi, Differential Equations with Transition Points. I . The First Approximation, Dept. of Math., California Institute of Technology, Pasadena, California, Tech. Report 6, 1955, 22 pp. For further results, see
N. Froman, “Detailed Analysis of Some Properties of the JWKB-Approximation,” A r k i v f o r Fysik, Vol. 31, No. 26, 1966, pp. 381-408. N. Froman and P. 0. Froman, JWKB Approximation, Contributions to the Theory, North-Holland, Amsterdam, 1965. 51.26. For an interesting application of the Lagrange expansion (see Sec. 5.21 of Chapter 5) to the determination of the coefficients in the asymptotic expansion of solutions of u“ (1 (b,/t2))u’ (1 (b,/t2) -) u = 0, see
+ +
+ +
+
C. Hunter, “Asymptotic Solutions of Certain Linear Difference Equations with Applications to Some Eigenvalue Problems,” 1. Math. Anal. Appl., Vol. 24, 1968, pp. 279-289.
Chapter 2 MATRIX THEORY
2.1. Introduction
I n this chapter we wish to consider a number of questions centering about the apparently prosaic problem of solving a system of linear algebraic equations
9 aiixi N
j=1
=
i
b, ,
=
1 , 2,..., N .
(2.1.1)
T h e ability to obtain the unknowns x l , x, ,..., x, by means of a finite number of purely arithmetic operations is one of the foundations of analysis, both theoretical and numerical. I n general, we exert our best efforts to reduce the solution of any given equation to one of the foregoing form. Nonetheless, as we shall see, there are also cogent reasons for trying our best to avoid these equations at certain times. A careful discussion of the obstacles encountered in solving (2.1.1) provides motivation for a good deal of our subsequent work. T h is is particularly so when we encounter ill-conditioned systems. I n the discussion at the end of the chapter we will indicate various points of contact of this type of question with parts of contemporary analysis dealing with improperly posed problems of mathematical physics. One approach to (2.1 .l), susceptible to considerable generalization, is to regard it as the variational equation associated with the minimization of the quadratic function Q ( x ~ x,..., ~ xN) =
C N
i,j=l
N
aijxixj - 2
bpi.
(2.1.2)
i=l
This equivalence holds when aij = aji . Another approach is to regard the solution of (2.1.1) as equivalent to the problem of minimizing the quadratic function
2.2. DETERMINANTAL SOLUTION
55
A slight, but important, modification of this leads to the regularization method of Tychonov. These methods clearly indicate the need for a systematic study of quadratic forms, and thus symmetric matrices, if we are seriously concerned with linear systems. The variational principles encountered in this study play a fundamental role in the Rayleigh-Ritz method analyzed in Chapter 7. Extensions of these variational methods, the Courant-Fischer min-max theorems, will be introduced as a way of studying perturbation techniques. Finally, we will present the basic result of Perron concerning positive matrices and the important variational characterization due first to Collatz, and rediscovered by many others. No attempt has been made to penetrate 'into the remarkable and fascinating theory of matrices in any depth. We have bounded ourselves, albeit reluctantly, with the minimum material needed for subsequent chapters. 2.2. Determinantal Solution
From one point of view, the problem of solving the linear system in
(2.1.1) has long since been resolved. We refer, of course, to the fact that we possess the elegant analytic representation of Cramer. If the determinant of coefficients is nonzero, we may write
a21
022
..'
'N1
aN2
'..
/ /
I
(2.2.1)
~a2NN N
with similar expressions for x2 ,..., x N . A corollary of this result is that a necessary condition for the existence of a nontrivial solution of the homogeneous system N
C aijxj = 0,
i
=
1 , 2,..., N ,
(2.2.2)
3=1
is that the determinant det(aij) = I ai3I
=
0.
(2.2.3)
56
MATRIX THEORY
An inductive argument shows that it is also a sufficient condition. This result is an important tool in many investigations, as the exercises at the end of the section indicate. Satisfying as the Cramer solution is analytically, it possesses severe, indeed crippling, drawbacks computationally. If we were to employ the standard rule for evaluating a determinant of order N , we would be forced to form N ! products and then carry out a total of N ! - 1 additions and subtractions to calculate 1 aij 1 . T w o basic considerations blackball this approach: time and accuracy. T o begin with, note that lo! = 3,628,800 and that 20! is an awesome quantity best appreciated by means of the simple inequality 20!
> 10’0(10!).
(2.2.4)
Even at microsecond or micromicrosecond speeds we cannot contemplate IOO! additions or multiplications. Yet we frequently meet even larger systems of equations in scientific applications. Secondly, on a digital computer, multiplication can only be carried out to a specified precision. At some point, after ten, twenty, or thirty digits, we must “round off”, which is to say truncate. This introduces a systematic error with each arithmetic operation, which means that we must carefully ration the number and type of operations performed. We see then that a critical consideration of the simplest system of equations forces us to accept three basic premises:
I . Numerical solution requires a consideration of errors and thus brings in stability theory. 2. Numerical solution involves a scheduling of operations subject to various constraints. 3. One form of a solution may be far preferable to another for numerical purposes.
Exercises
1. If Ax = b possesses at least one solution for any b, then and the equation possesses a unique solution.
IAI #0
2. Consider the simultaneous polynomial equations p ( x , y ) q(x, y ) 0, where p and q are quadratic in x, 7
+ P d Y b + PAY) = 0, 0. Qo(Y)”2+ d Y ) X +
Po(Y)X2
=
=
0,
2.2. DETERMINANTAL SOLUTION
57
Show that we can eliminate x in the following manner. Multiply both equations by x, obtaining
pox3
+ p1x2 +
=
P,X
0,
Regard the four equations as four linear algebraic homogeneous equations in I, x, x2, x3. Th u s y is a solution of the determinantal equation 0 Po
Po 0
40
P,
P2
Pl
P2
0
40
41
q2
41
42
=
0.
0
Generalize to the case where p and q are polynomials in x of degrees m and n respectively. (Sylvester dialytic method of elimination.) Does every solution of the determinantal equation correspond to a solution of the simultaneous equations p(x, y ) = 0, q(x, y ) = 0 ?
+
+
3. Consider the congruence x2 a,x a, = O(p), a, # 0. Multiply by x, x2,..., X P - ~to obtain p - 1 linear homogeneous equations in the quantities 1, x, x2,..., xPp2, using the Fermat result XP--l 3 I@), x # 0. Obtain in this way the determinantal relation satisfied by a,, a, in order that a solution of the congruence exists.
4. Consider the partial differential equation u t
= u, . Let v be determined by u, = vu. Show that u,, = v,u vu, , uZt = v p vut, uxt= v,,u 2v,u, v u l . T h u s we have four linear homogeneous equations in u, u, , ut , uxt. Hence, obtain the Burger’s equation vt = v,, 2vv, .
+
+
+ +
+
5 , Consider the equation ut = u, , and let v be determined by u, = vu. Find a nonlinear partial differential equation for v.
6. Start with
ut = u ,, (u,,
+ a,u, + a2u, + a3u and set
+ h u , + b)
=
+1u,,
+ + CZU,
CQU).
With suitable choice of the parameters a , , a2 , a 3 , b, , b, , cl, c 2 , c 3 , how close can we get to the Korteweg-de Vries equation, v t = v,, uv,? For an interesting discussion of some questions connected with this partial differential equation, see
+
58
MATRIX THEORY
P. D. Lax, Integrals of Nonlinear Equations of Evolution and Solitary Wuues, Courant Institute, New York University, 1968, NYO1480-87. 2.3. Elimination
I n place of the foregoing determinantal procedure we can use a systematic elimination method to determine x l , xg ,..., x, . Multiply the first equation in (2.1.1) by a2, and the second by alN and subtract. T h e result is (%Vall
- ulN‘21)X1
+ ... +
(a2Nul.N-l
- ulN‘2.N--1)XN-l
= %NbI
- %Nb2
*
(2.3.1)
We then multiply the first equation by a3N and the third by alN and subtract, and so on. I n this way, we obtain ( N - 1) simultaneous linear equations for the ( N - 1) quantities x l ,x2 ,..., xNP1.T h e same procedure may be repeated to eliminate xNP1from these ( N - 1) equations, and so on. T h e total number of multiplications required to determine x l , say, in this fashion is thus 2N(N
-
1)
+ 2(N
-
1)(N - 2 )
+ .*. + 3(2) < 2N3.
(2.3.2)
This is considerably less than N ! for N >, 6, and the algorithm is thus seen to be a feasible one for values of N of the order of several hundred, as far as time is concerned. Nonetheless, the foregoing method by no means disposes of the fear of loss of significance as a consequence of the arithmetic operations. If, for example, the quantities (a2,all - a l N a 2 1 ) and so forth, as well as (a,,b, - al,b,), are small compared to the magnitudes of the aii and bi , we can easily obtain meaningless results. I t follows that there may be considerable merit to eliminating the xi in some specified order which depends on the nature of the particular values of the aij . Th is is indeed the case. References to the pivoting procedure, an important instance of “adaptive computation,” will be found at the end of the chapter. Exercise
1. How many multiplications and divisions are required to determine x1 , xg ,..., x, on the basis of the foregoing method ?
2.4. Ill-CONDITIONED SYSTEMS
59
2.4. Ill-conditioned Systems
Either of the above procedures (that of Sec. 2.2 or Sec. 2.3) shows that the solutions of (2.1.1), the x i , when they exist, are linear functions of the bi , (2.4.1)
If the cii oscillate in sign, with large magnitude compared to the b i , small variations in the b, can produce much larger variations in the xi . We see then that the xi are unstable functions of the b, (in a sense to be made precise in Chapter 4), and we say that the original linear system is ill-conditioned. I n some cases we can predict this unpleasant property well in advance; in other cases we can only wait and worry. I t is because of a fear of ill-conditioning that we strive to avoid linear systems wherever possible. This is a first indication of the modern trend toward nonlinearization in many areas. We will provide an instance of this instability in connection with our discussion of the numerical inversion of the Laplace transform in Volume 11. I n some cases one can apply dynamic programming to the numerical solution of ill-conditioned equations. In other cases, we can apply dynamic programming and invariant imbedding to bypass the task of solving linear systems. These matters will be discussed in Volume 11.
Exercise
1 . Consider the linear system Mx,
b,
x1
-
~2
- Mx, = b,
=
Show that the determinant of the system is 1 - mMN-’. Discuss the sensitivity of the solution to the value of m. See
A. M. Ostrowski, “On Nearly Triangular Matrices,’’ J. Research Nut. Bur. Standards, Vol. 52, 1954, pp. 319-345.
MATRIX THEORY
60
2.5. The Importance of Notation
Finite systems of linear algebraic and differential equations will play an cssential role in much of our subsequent work. T o escape a welter of subscripts and superscripts and symbols of bizarre type when treating multidimensional matters, it is necessary to employ vector-matrix notation, and even then to devote critical care and attention to the choice of notation. I t cannot be sufficiently emphasized that a simple, flexible notation which subliminally guides the intuition and obviates tedious, confusing calculations is an invaluable aid to both research and pedagogy. Let us then turn to vector-matrix terminology. 2.6. Vector Notation
1,ower case letters will be used to denote column vectors, (2.6.1) the only types of vectors we shall employ. T h e quantities x l ,x2 ,..., xN , real or complex, are called the components of x, with N the dimension. When we wish to indicate different vectors, we will use different letters. Generally, x, y , x will denote vectors whose components are variables, and a, 6, c those whose components are constants. T h e relation =_ y (2.6.2) signifies that the components are equal term by term. We define (2.6.3) and subtraction in a similar fashion.* For a scalar c1 we define multiplication of a vector by clx
1
f:::)
(2.6.4)
CIXN
T h e symbol 0 denotes the vector all of whose components are zero. ~
~
~
* T h e ‘‘4-” sign connecting x and y
+”
is not the “ sign of scalar arithmetic. But since it enjoys the same properties, it seems rather higgling to employ a different symbol.
2.8. VECTOR I N N E R PRODUCT
61
Occasionally, when there is no chance of confusion with components, we will use subscripts to denote different vectors; if there is a reasonable possibility of ambiguity, we will reluctantly employ superscripts. 2.7. N o r m of a Vector
As a scalar magnitude of a vector, we have a choice of a number of functions, each convenient for various purposes. Let 11 x 11 , read “norm of x”, denote this quantity. We demand the following properties: (a) 11 x I( >, 0 for all x; (1 x 11 = 0 implies that x = 0. (b) II c1x I/ = I c1 I II x 11, homogeneity. ( c ) /I x
+ Y II < /I x ll + II y (I
(2.7.1)
(triangle inequality),
T h e two most popular candidates are
II x I l l
c I xi I? N
=
i=l
(2.7.2)
T h e second has the great advantage of geometric and physical significance, the first that of analytic simplicity. It is easy to verify that 11 x Ill satisfies the three requirements of (2.7.1). T o establish the triangle inequality for 11 x [I2 , we require Minkowski’s inequality referred to in Sec. 1.11. Exercise
1 . Show that we can use maxi I xi I as a norm. 2.8. Vector Inner Product
A fundamental bilinear function of two real vectors x and y is the inner product N
(2.8.1)
MATRIX THEORY
62
Observe that it is symmetric, (x,y )
=
( y , x), and that for real x,
II x 112 = (x,
(2.8.2)
If (x,y ) = 0, we say that x and y are orthogonal. If we allow complex components, it is necessary to use a modified inner product,
c N
(x,Y ) =
i=l
xdi
(2.8.3)
9
where Tidenotes the complex conjugate of yi . I n this way (x,x) preserves the desired nonnegativity property. Observe, however, that this extended inner product is no longer symmetric. We have (2.8.4)
(%Y>= ( Y T ) .
Unless specifically mentioned or indicated in what follows, all quantities of interest will be real. Exercises
+
+
1. Consider (Y,X y , rlx y ) as a quadratic polynomial in rl . From the fact that it is nonnegative, deduce the Cauchy-Schwarz inequality (x,X)(YlY) 3 (x,Y)2-
2. From this deduce the triangle inequality .(
+ y , x + Y)1’2 < (x,
x)lj2
+ ( Y ,Y)1’2.
3. A set of vectors [xl , x2 ,..., x,,] is said to be linearly independent if any c2x2 - . * ckxk = 0, with the ci relation of the form clxl scalars, implies that c1 = c2 = ... = ck = 0. Show that any set of more than N N-dimensional vectors cannot be linearly independent.
+
+ +
4. If (x,y )
= 0, we say that x and y are orthogonal. Show that we can orthogonalize any set of linearly independent vectors [xl , x2 ,..., xk] in the following fashion (Gram-Schmidt orthogonalization). Set y1 = x1 , y 2 = x, clxl ; determine c1 so that ( y l ,y 2 ) = 0. Set y3 x3 cllyl cI2y2. Determine cI1 and cI2 so that (Y.37Yl) = (Y3 PYA = 0, etc5. What is the advantage of this procedure over setting 7
+
+ +
Y3
=
x3
+
CllXl
etc ? Are there any disadvantages ?
+
C12%
?
2.9. MATRIX NOTATION
63
6 . If x l , x p ,..., xN are N linearly independent vectors of dimension N, we can express any other N-dimensional vector x uniquely as a linear combination of the xi .
7. If the N vectors of dimension N are mutually orthogonal, i.e. (xi , xi) = 0, i # j, then they are linearly independent if nontrivial.
8. If x
=
( x i , xi)
xixlcixi , with the xi orthonormal, that is, (xi , xj = 0, i # j , N
=
1, then ci
=
( x , xi), i
=
1, 2,..., N.
2.9. Matrix Notation
To signify a square matrix, the type we shall deal with almost exclusively, we shall use upper case letters, A
=
(Uij),
i,j
=
1 , 2,.... N.
(2.9.1)
As in the case of vectors, X, Y, 2 will be used as variables, A, 3,C as constants. T h e quantities aij are called the elements of A, and N is called the dimension. Equality of two matrices, A = B, means that corresponding elements are equal, aij = bij . Two matrices of frequent occurrence are the identity matrix
I=
(; ;) ...
(2.9.2)
and the null matrix 0, all of whose elements are zero. Matrices possess a far richer and more interesting algebra than vectors. T h e relations that we introduce here are carefully chosen so that we can employ matrices to study the properties of linear transformations. Many other relations can be introduced for other purposes. Addition is defined in the expected fashion, A =fi.( bijh (2.9.3)
+
+
with subtraction the inverse operation. Multiplication of two matrices is induced by considering the properties of two linear transformations applied one after the other. Consider the linear transformation
c N
yi =
i=1
a,ixj,
i = 1, 2,..., N,
(2.9.4)
64
MATRIX THEORY
and define the product of a vector x by a matrix A so that (2.9.4) may be written y = AX, (2.9.5) that is, Ax is the vector whose components are given by the right-hand side of (2.9.4). Then, if z = By, we have z = By = B(Ax). (2.9.6) If we define B A so that (2.9.6) may conveniently be written (2.9.7)
z = (BA)x,
we see that the ijth element of BA is given by the expression (BA),, =
1b,,a,j .
(2.9.8)
k
Exercises
1 . Show that X + 0 = X for all X and that 0 is the only matrix possessing the property.
2. Show that X I
this property.
3. Let yi
=
= yi(xl
X for all X and that I is the only matrix possessing
, xg ,..., xN) be a set of N functions. Set
J(r,x) = ( a Y i / w , the Jacobian matrix. Show that J ( z ,y ) J(y,x)
=
J ( z ,x).
4. Establish the Jacobi identity
[[A, Bl, CI
+ [ [ C ,A ] ,BI + [ [ B ,Cl, A1
= 0,
where [A,B] = AB - BA, etc. For an interesting geometrical interpretation of this important result, see
W. A, Hoffman, “The Neuron as a Lie Group Germ and a Lie Product,” Quart. Appl. Math., Vol. 25, 1968, pp. 423-440. 2.10. Noncommutativity
It is easy to see by means of simple examples for 2 x 2 matrices that, in general, B A # AB. (2.10.1)
2.12. THE INVERSE MATRIX
65
It is precisely this property which renders matrix algebra so important in science and which makes matrix theory a fascinating and challenging domain. However, despite (2.10.1), it is true that associativity holds, namely, (AB)C = A(BC) (2.10.2) which means ABC is unambiguously determined. We leave the proof of this as a n exercise. If AB = BA, we say that A and B commute. Exercises
1. Show that An for n
=
3, 4,..., is unambiguously determined.
2. If X Y
= Y X for all X , then Y = k l , where k is a scalar. Is the requirement that it hold for all X necessary for this determination of Y?
2.11. The Adjoint, o r Transpose, M a t r i x
An important matrix function of A is the transpose, or adjoint, matrix defined by AT = ( ~ j i ) . (2.11.1)
It is ‘Linduced”using the inner product; namely, we can define AT by the relation (x,AY) = (ATX,Y),
(2.1 1.2)
valid for all x and y. Exercise
1 . Show that AT A = ( ( a ( < &)) ), where a‘<)is the vector whose components are the elements of the ith column of A. 2.12. The Inverse M a t r i x
Let us now return to our original objective, the solution of (2.1.1). If we attempt to invert the relation y = AX,
(2.12.1)
MATRIX THEORY
66
which is to say express x as a linear transformation of y, we are led to the inverse transformation x == A-1Y . (2.12.2) This is our definition of A-l. As we know from the theory of linear equations, the inverse matrix exists and is unique if and only if the determinant of A , written I A l = IaijI, (2.12.3) is nonzero. We can use Cramer’s rule to determine the elements of A-1. If 1 A 1 = 0, A is said to be singular. Exercises
1. From (2.12.1) and (2.12.2) we see that A-l is a left inverse, i.e., x = A-l(Ax) implies that A-lA = I. Show that it is also a right inverse, that is, AA-l = I. 2. Show that (AB)-l
=
B-lA-l when B-l and A-l exist.
3. Let y = S x be a relation which preserves distance, that is, ( y ,y ) = (x,x) for all x.Show that this means that S is an orthogonal matrix, ST = S-I.
4. Let y = Sx be a relation which preserves probabilities, that is, N xi = 1 implies that yi 3 0, N yi = 1 . Show that xi 2 0, N S must be a Markov matrix, that is, sii 2 0, sii = 1 for j = 1 , 2, ..., N . If S merely preserves the property of nonnegativity, then S is a nonnegative matrix, that is, sii 2 0.
zi=l
5. Show that the product of two orthogonal matrices is again an orthogonal matrix and that the product of two Markov matrices is again a Markov matrix, both with and without calculation. 6. Show that A is nonsingular if 1 aii I > 1 aii I for i = 1, 2,..., N . For a history of this important and useful result, as well as extensions, see
0. Taussky, “A Recurring Theorem on Determinants,” Amer. Math. Monthly, Vol. 56, 1949, pp. 672-676. For some quantitative refinements of this result, see A. M. Ostrowski, “Note on Bounds for Determinants with DomiAmer. Math. SOC.,Vol. 3, 1952, nant Principal Diagonal,” PYOC. pp. 26-30.
2.13. MATRIX NORM
67
A. M. Ostrowski, “Note on Bounds for Some Determinants,” Duke Math. J., Vol. 22, 1955, pp. 95-102.
7. T h e “generalized inverse” of an M x N complex matrix A is the unique matrix B such that ABA = A, BAB = B, (BA)* = BA, (AB)* = AB. Here C* = C*, where is the complex conjugate of C . See
R. Penrose, “A Generalized Inverse for Matrices,” Proc. Cambridge Phil. SOC.,Vol. 51, 1955, pp. 406413. 8. Show that Ax = 0 has a nontrivial solution for x if and only if I A I = 0 and that this solution may be taken to be real if A is real. Hint: Use induction on the order of A. 9. Geometrically, in two and three dimensions what do orthogonal matrices correspond to ?
10. If I A I = 0 the columns are linearly dependent, as are also the rows. If I A I # 0 the columns and rows are linearly independent. 2.13. Matrix N o r m
For the measure of the magnitude of A we shall usually employ the quantity
/I A /I1 =
N
C I aii I-
Lj=l
(2.13.1)
Occasionally, one can profitably employ (2.13.2)
Exercises
1. Prove that
II ... Ill -
11 A B 11
< 11 A 11 11 B 11 , 11 Ac 11 = 11 A 11 )I c 11 , using the norm
2. Do these results hold for the norm
3. Introduce the notation tr(A)
N
11
[I2?
uii (read “trace of A”) and the = inner product” of two matrices A and B, (A, B) = tr(AB). Show that tr(AB) = tr(BA) and that 11 A = [tr(AA)]1/2.
MATRIX THEORY
68
4. T h e vector norm can be used to induce a matrix norm (as is common in operator theory) by writing
where / j x I/ is one of the vector norms. Show that norm.
11 A [ I r n
is indeed a
5. Consider the six norms: N,(A)
=
N,(A)
=
ili,(A)
=-
max
I <j
N j=l
1 aij 1,
max (Ax,
(x,*)=l
max
I
I a i j I,
Determine the complex matrices for which the ratios N,(A)/N,(A) attain their upper and lower bounds.
B. J. Stone, Best Possible Ratios of Certain M a t r i x Norms, Technical Report No. 19, Applied Mathematics and Statistics Laboratories, Stanford University, 1962. 2.14. Relative Invariants
Given a transformation, it is always of interest to find its fixed points, which is to say the vectors left invariant under the transformation. I n general, as we shall see below, the only solution of AX = x
(2.14.1)
is x = 0. This is not particularly interesting. Much more interesting, and indeed basic to the theory of linear transformations, is the search for relative invariants.
2.14. RELATIVE INVARIANTS
69
Can we find a scalar A and a vector x, determined u p to a factor of proportionality, such that (2.14.2)
AX = AX.
We can think of x as representing an invariant direction. Writing this relation in the form ( A - Az)x
=
0,
(2.14.3)
we see that a necessary and sufficient condition for the existence of a nontrivial vector x satisfying (3) is the determinantal equation
1 A -XI I
= 0.
(2.14.4)
Th i s is a polynomial equation of degree N , the characteristic equation of
A ; and its N roots, A, ,A, ,..., AN , are called the characteristic roots of A.
I n general. they are complex, even if A is real. T h e corresponding vectors, determined u p to a multiplicative scalar constant, are called the characteristic vectors of A. Although there are always N characteristic roots, simple or multiple, real or complex, there are not necessarily N distinct characteristic vectors. T o illustrate this, consider the 2 x 2 matrix (2.14.5)
It i s easy to see that one is a multiple characteristic root and that there is exactly one characteristic vector
(2.14.6) in the sense that all invariant vectors are scalar multiples of this vector. If there is a full complement of linearly independent characteristic x ( ~ ..., ) , x f N ) , we can, as we know, express an arbitrary vectors, vector x as a linear combination of the x ( ~ ) .
c N
x =
CiX(i),
(2.14.7)
i=l
where the ci are scalars. Th en N
N
(2.14.8)
MATRIX THEORY
70
a simple and useful representation of the effect of the transformation equivalent to A. It is in the hopes of doing this that we study the relative invariants. Exercises
1 . Determine the possible limiting behaviors of Anx as n + co if (2.14.8) holds. 2. If A is a given matrix, show that we can always find a B with 11 B 11 < E such that A + B has distinct characteristic roots.
+
+
3. Show that tr(A) = ELl hi and that tr(A B) = tr(A) tr(B). Are the characteristic roots of A B sums of the characteristic roots of A and B ?
+
4. Show that 1 I
+ EA I = 1 + &(A) + ....
a : i , and hence establish the Cauchy5. Show that tr(AAT) = Schwarz and triangle inequalities for 11 A 1, =
6. Consider the linear system 9x1 qx2
+ +
PX,
= Ax1
7
h,, = Ax, ,
PX, = PX,
qxn-l
=
Ax,.
Determine the characteristic values and characteristic vectors of the associated matrix. This system plays an important role in the study of random walk. See
M. Kac, “Random Walk and the Theory of Brownian Motion,” Arner. Math. Monthly, Vol. 54, 1947, pp. 369-390. T h e determination of the properties of characteristic values and vectors of a particular class of matrices is usually a matter of some difficulty. See, for example, A. M. Ostrowski, “On the Spectrum of a One-Parametric Family of Matrices,” J . Reine und Angew. Math., Vol. 193, 1954, pp. 143160. M. Kac, W. L. Murdock, and G. Szego, “On the Eigen-values of Certain Hermitian Forms,” J. Ratl. Math. and Mech., Vol. 2, 1953, pp. 767-800,
2.1 5. CONSTRAlNED MINIMIZATION
71
R. Bellman, Introduction to Matrix Analysis, McGraw-Hill, New York, 1960. (Second Edition, in preparation.)
7. Are the characteristic roots continuous functions of the elements of
A ? See
A. Ostrowski, Mathematical Miscellanea, XX VII, On the Continuity of Characteristic Roots in Their Dependence on the Matrix Elements, Technical Report No. 2, Applied Mathematics and Statistics Laboratories, Stanford University, 1959. 2.15. Constrained Minimization
T h e problem of determining characteristic roots and characteristic vectors arises in a natural way from the theory of linear differential equations, as we shall see in the following chapter. It also arises from the following variational problem. Consider the quadratic form
c N
Q(x1
Xz
XN) =
V..V
aijxixj
(2.15.1)
9
f+l
and ask for its stationary points on the N-dimensional sphere
c N
xi2 =
(2.15.2)
1.
f=l
Proceeding formally, let us employ a Lagrange multiplier. Form the new function QA(xl ,x2 ,..., x N ) =
c N
N
aiixixj
-
i,j=l
h
1xi2
(2.1 5.3)
i=l
and suppose that the xi are now unconstrained. Then the variational equations are
aQn-- O = or
axi
c N
a t j x j - Ax,
(2.15.4)
j=1
AX = Ax.
(2.15.5)
Using the canonical representation of a quadratic form, we can make the foregoing rigorous in a simple fashion (see Sec. 2.19). At the moment we wish to indicate the fundamental role that matrices occupy in multidimensional maximization problems.
72
MATRIX THEORY
Exercise
1. I n place of using a Lagrange multiplier, consider the quotient
and once again derive (2.15.5). What is the meaning of the characteristic values as derived from this formulation ? 2.16. Symmetric Matrices
There are a number of important classes of matrices which possess a full complement of characteristic vectors. Of these, those most frequently occurring are the real Symmetric matrices. If AT = A we say that A is symmetric. T h e reason for the interest in symmetric matrices lies in the fact that the quadratic form (2.16.1)
can be written in the form
where A = (uii) is a symmetric matrix. Let us begin by showing that characteristic vectors corresponding to different characteristic roots of A are orthogonal if A is symmetric. As agreed, we suppose that all matrices appearing are real unless otherwise specified. From (2.16.3)
we have (2.16.4)
Since (Ax'i),dj)) have
=
(A&),di)),by virtue of the symmetry of A , we (hi --
Aj)(X(J),
=
0.
Since by assumption hi # X j , we have (dip, di)) = 0.
(2.16.5)
2.16. SYMMETRIC MATRICES
73
From this result, it follows readily that the characteristic roots of A are all real. For if A is a complex characteristic root, with associated characteristic vector x, then A, the complex conjugate of A, is also a characteristic root with characteristic vector %. Since, by assumption, A # A, we have from the foregoing the relation (x,5) = 0, a contradiction. Let A, , A, ,..., A, be the N characteristic roots of A and assume for the moment that they are all distinct. Then the associated characteristic ..., dN), can be taken to be real and we can normalize vectors, ~ ( 1 1 ,d2), = 1 . The orthogonality of the x(i) them by the condition ( x f i ) , di)) makes it clear that they are linearly independent. Let R be the matrix whose columns are the di). Then the relations Ax(*)= hi&), i = 1 , 2,..., N , yield the equation AR
=
(2.16.6)
RA,
where A is the diagonal matrix
(2.16.7)
Hence, we have R-IAR = A, A
=
RAR-’,
(2.16.8)
provided that R-l exists. The fact that the columns of R are linearly independent ensures this. However, we can go further. We see that RTR = ((di), dj))).
(2.16.9)
Since the x(*)are orthonormal, we see that RTR = I, which means that R-1 = RT and thus that R is an orthogonal matrix. Hence, (2.16.8) simplifies to RTAR = A,
A
=
RART,
(2.16.10)
a fundamental canonical representation. This yields a parametric representation for the elements of a symmetric matrix that can be used to establish a number of results.
14
MATRIX THEORY
Exercises
1. Show that Ak = RAkR-l for k = 1, 2 ,..., and for k = - 1 , if A is nonsingular.
2. Why can we safely suppose that ati
=
+
-2
,...,
aii in the quadratic form Q(x) ?
3. Consider the perturbed matrix A EB, where A and B are symmetric and E is a scalar. Under what circumstances are the characteristic roots hi(A EB)expressible in the form &(A)+ ~ h l l ) ( A B ,) ? What can happen if multiple characteristic roots of A occur ?
+
+
4. Show that a necessary and sufficient condition that the two symmetric matrices A and B can be reduced to diagonal form by the same orthogonal transformation is that A and B commute.
5. Show that any symmetric matrix with distinct characteristic roots N X,Ei, where the Ei are nonmay be written in the form A = negative definite matrices satisfying the conditions EiEi = 0, i # j , Ei2= Ei , and the hi are the characteristic values of A. T h is is called the spectral decomposition of A. What is the expression for A n ?
I XI - A I = AN that +,(All) = &(BA).
6. Write
-
+,(A)P-l
+ + (-
l)N+N(A).Show
2.17. Quadratic Forms
Returning to the representation (x, A x )
for the quadratic form Q(x) = variable,
N
(2.17.1)
aiixixj , let u s make the change of
x = Ry.
(2.17.2)
Then (x, A x ) = (Ry,A&)
=
( y , RTARy) = ( y , Ay)-
(2.17.3)
Hence N
C
i,+l
N
aijxex3 =
C Xiyz.
(2.17.4)
i=l
Th i s representation as a sum of squares plays an essential role in the study of quadratic forms. Observe that since RTR = I , we also have (x,x)
=
(RY,RY) = (Y, RTRy)= (Y,Y).
(2.17.5)
2.18. MULTIPLE CHARACTERISTIC ROOTS
75
Exercise
1. Show that (x,Ax) = 0 for all x implies that A T = -A. Hence if the quadratic form aiixixj is identically zero for all x, then aij = 0 for all i a n d j , if A is symmetric. 2.18. Multiple Characteristic Roots
T h e fundamental canonical representation of (2.16.8) remains valid in the general case where the symmetric matrix A has multiple characteristic roots. Since we will make no particular use of this result, let us merely sketch two approaches to its demonstration. T h e first proceeds by establishing the result for the perturbed matrix A B, where B is a real symmetric matrix with I(B (1 < E chosen so that A B has no multiple characteristic roots, and then proceeding to the limit as B+0. This is not an esthetically satisfying method since it requires extraneous analytic concepts of limiting procedures in the demonstration of an algebraic result, but it is nonetheless a useful method in many cases. It must be applied in each particular investigation, however, with a certain amount of care since sometimes the limiting procedure is valid, and sometimes not. I n this case, the fact that R is orthogonal, which implies that R-l = RT, hence RRT = I , provides the key. We leave the details to the reader. T h e second approach is inductive and algebraic. If A, is a characteristic root, we begin by showing that we can find an orthogonal matrix R, such that
+
+
(2.18.1)
It is easy to see that ANPIis again symmetric and that it possesses the remaining N - 1 characteristic roots of A. W e now iterate this proce-
dure, using the group property of orthogonal matrices. Again the details are left t o the reader. T h e fundamental fact is that a characteristic root of multiplicity Y has an r-dimensional linear subspace of characteristic vectors. An orthonormal basis may readily be constructed for this subspace. I n what follows we shall suppose that the characteristic roots are distinct unless otherwise specifically stated.
MATRIX THEORY
76
Exercises
I . Establish the Cayley-Hamilton theorem for symmetric matrices: Every symmetric matrix satisfies its characteristic equation.
2. Establish the Cayley-Hamilton theorem for matrices with distinct characteristic roots.
3. Establish the Cayley-Hamilton theorem for arbitrary matrices using a limiting process, or otherwise. (A simple proof uses the Jordan canonical form mentioned in Sec. 2.24. However, it is not simple to obtain this canonical form.) 2.19. Maximization and Minimization of Quadratic Forms
As we shall see in subsequent chapters, a large number of basic problems of analysis can be reduced to the problem of the minimization or maximization of a quadratic form Q ( x ) = (x,Ax)
+ 2(r, b )
(2.19.1)
over some region in x-space. There are two cases of particular importance. T h e first is that where x can vary over the entire N-dimensional space. T h e stationary points are then determined by the linear equation
+b
Ax T h e sccond is that where b
=
=
0.
(2.19.2)
0 and x is constrained to the unit sphere
(.2,s) =
1.
(2.19.3)
A rigorous discussion of the connection between the solution of (2.19.2) and the original variational problem requires the concept of positive definite matrices which we present in Sec. 2.21. Let us consider here the constrained case. ‘There are several ways of proceeding. Probably the easiest is the approach which makes use of the reduction to diagonal form as in (2.17.4). ITTe see that the range of values assumed by (x,Ax) when x tralerses (s,x) = 1 is the same as the range of values of the simpler expression
c 4Yi2 N
2-
1
(2.19.4)
2.20. MIN-MAX CHARACTERIZATION OF THE A*
77
Let us arrange the characteristic values in order of increasing magnitude, (2.19.5) A, < A, < ... < AN .
It follows that A
min
- (.r,x)=l
(x,
(2.19.6)
Ax) =
and that (2.19.7)
Exercises
1. Write &(A),AN(A)to indicate the dependence on A. Show that A,(A AN(A
+B) bA d 4 +A m ,
+ B, <
AN(A)
+
AN(B)*
2. Show that we can employ Rayleigh quotients and write
2.20. Min-Max Characterization of the A,
Let x(1), ..., x(N) be characteristic vectors associated with A, , A, ,..., A,. Then, referring to (2.19.4), we see that we can write (2.20.1)
where the minimum is taken over the y-region defined by y1 = 0. Translating this back into x-space, we may write A,
=
min(x, Ax),
N
yi2 = 1, (2.20.2)
where the minimum is taken over the x-region (x, x(1)) =
0,
(x, x) = 1.
(2.20.3)
?a
MATRIX THEORY
T h e first condition is an orthogonality condition, the geometric meaning of which is clear if we think in terms of an N-dimensional ellipsoid. Similarly, we can write down relations for A, ,..., AN-l . Since the characteristic vectors x(l),..., d N )depend in a complex way on A, these relations are not easily used t o obtain various bounds for A, ,..., AN-l that are often needed to establish other results. To overcome this defect, a basic representation due to Fischer and Courant is extensively used, a min-max characterization. It takes the form A, = min (x,A x ) / ( x ,x), X
A,
=
A,c
=
max min (x,Ax)/(%,x),
( y , y ) = l (x,y)=O
Equivalently,
max
min
( y i , y ' ) = l (x,y')=O
(x,
A x ) / ( x ,x),
i = 1, 2,..., k
max(x, A x ) / ( x , x), ANPI = min max (x, Ax)/(x,x), AN
-
1,
(2.20.4)
=
(2.20.5)
( y , y ) = l (x,y)=O
It is not difficult to verify the foregoing results using the canonical representation, once they have been stated. Their origin becomes quite clear if we consider the geometrical interpretation of the characteristic roots in terms of the lengths of the axes of the multidimensional ellipsoid; see the two-dimensional figure below (Fig. 2.1).
Figure 2.1
As we allow y to vary, it is easy to see that the length OP of x, kept perpendicular to y , becomes a minimum when OP coincides with the minor semiaxis of the ellipse and OQ coincides with the major semiaxis of the ellipse. Observe the desired fact that the domain of variation of the x and y(i)in (2.20.4) and (2.20.5) no longer depends on the matrix A.
2.21. POSITIVE DEFINITE MATRICES
79
2.21. Positive Definite Matrices
If a real symmetric matrix A has the property that (x,
Ax) > 0
(2.21 .l)
for all real nontrivial vectors x, we say that A is positive definite and we write A>0. (2.21.2) If it is merely true that (x, Ax) 3 0, we say that A is positive indefinite, 0. or nonnegative definite, and write A It follows from the decomposition into a sum of squares given in (2.17.4) that a necessary and sufficient condition that A be positive definite is that all characteristic roots of A be positive. T h e condition of positive definiteness introduces a partial ordering in real symmetric matrices. We write A 3 B if A - B 3 0. Clearly A B, B C implies that A 3 C and c1 > 0, A > 0 implies that c,A > 0.
>
>
>
Exercises
1. Show that A 2 3 0 for any real symmetric matrix A.
2. Show that
m:s
...
m
e-(zJz) dx, dx,
3. Show that BABT 3 0 if A
> 0.
4. If A, B 3 0, then h A
-
+ (1
... dx,
if A
= n N / 2 /A[
A) B 3 0 for 0
> 0.
< h < 1.
5. Why do we say “partial ordering” ? Can we compare any two symmetric matrices A and B in this fashion ?
> 0, then A-l > 0. If ( x , A x ) > 0 whenever ( x , B x ) = 0, where B is positive indefinite,
6. If A 7.
then there exists a scalar constant h such that ( x , A x ) positive definite. (Finder.)
+ h(x, B x ) is
8. A necessary and sufficient condition that ( x , A x ) be positive for all nontrivial x satisfying the equations ( x , bi) = 0, i = 1, 2, ..., k, is ( x , 6J2 be positive definite for h I. * that ( x , Ax) + h
>
9. Show that a sufficient condition that ( x , A x ) be positive for all * We
use the notation h
> I to mean that h must be sufficiently large.
MATRIX THEORY
80
nontrivial x satisfying b,x, bordered determinants
...
a11
a12
a21
a22
’..
a,,
Uk2
...
b,
be nonnegative for k
+ B2x, + ... + bNxN = 0
=
b2
a1?i
is that the
b, 62
akk
... 6 ,
b, 0
1 , 2,..., N .
10. If A = (aii), B = (0,) are positive definite, then C positive definite. (Schur.) Hirit: Use Sec. 2.17.
=
(adjbii)is
11. Show that a necessary and sufficient condition that A be positive definite is that tr(AB) > 0 for all positive definite B. 12. If A is positive definite and B symmetric, show that exp[-(x, A x ) - i(x, Bx)]dx
13. Show that dx, dx, ( Z , A d <1
... dx
A r(N/2
&“2j -
=
nN/2
1A
+ iB
/1/2*
\- 1 /2
+ 1)
14. Show 1 A J/i A(i)i= min, (x,A x ) where x is constrained by the condition xi = I and A ( i )is the matrix obtained from A by deleting the ith row and ith column.
15. If I1 is a symmetric matrix with no characteristic value in [a, b], then ( H - a l ) ( H - 01) is positive definite. (Kato’s Lemma.) G. ‘Temple, “An Elementary Proof of Kato’s Lemma,” Mathematika, Vol. 2, 1955, pp. 39-41. 16. I,et { A I 1 )be a sequence of positive definite matrices such that A , < A , < ... < A,, < B. Show that A, converges as n + 00. Hint: ( c , A,,c) is a monotone increasing scalar function for all c. Choose c adroitly.
17. If A is a positive definite matrix, it possesses a unique positive definite
square root 13 = Allz. For a representation of All2 which exhibits some of its properties in a simple fashion, see Chapter 7. For a surprising way in which square roots enter, see E. P. IVigner and 51. Y.Yanase, “Information Content of Distributions,” Proc. Nut. Acad. Sci. U S . , Vol. 49, 1963, pp. 910-918.
2.22. DETERMINANTAL CRITERIA
81
2.22. Determinantal Criteria
Since, however, any property requiring a knowledge of all of the characteristic roots is obviously a difficult condition to verify, it is essential to have a more tractable set of relations. T o obtain these, let us begin with the condition that A > 0 implies that I A 1 > 0. There are several ways of demonstrating this result. Since the characteristic roots, hi , are the solutions of /A-AXII=O (2.22.1) we see that (2.22.2)
Since A > 0 implies that hi > 0, i = 1, 2,..., N , the result follows. An important type of proof using imbedding starts with the fact that A > 0 implies that 1 A 1 # 0. For if I A 1 = 0, there is a nontrivial vector x satisfying Ax = 0. For this value of x, (x,Ax) = 0, a contradiction. Now consider the family of matrices A(t) = t I + (1 - t ) A where 0 t 1. Clearly, A ( t ) > 0 for 0 t 1 . It follows from t 1. Since I A ( l ) [ = 1 I I = 1 > 0, we this that I A(t)l f 0 for 0 see then that 1 A(O)I = I A I > 0. I t follows, upon consideration of the submatrices of A , that a necessary condition that A > 0 is that the determinantal criteria of Sylvester,
< <
< <
< <
(2.22.3)
hold. T hi s is also a sufficient condition, as follows from the representation of Exercise 2 below. A proof without an explicit representation follows, from the PoincarC separation theorem of Sec. 2.26. Exercises
1. Let fl ,fz,..,f N be a set of N real functions each in L2(0,7’).Con-
s:
+
+
x2fz ..- +xNfN)2dt, sidering the quadratic formQ(x) = (XIf show that the determinant fifidt 1 is nonnegative. Show that strict positivity holds if the f i are linearly independent over [0, 7’1.
1s;
2. Show that if Q(xl , x2) = ullx12
Q
= all
(xl
+ 2u12x1x2+ u22xz2,we have
+ 5 x.)” + (azz A) a11
- a11
x22,
82
MATRIX THEORY
and thus show that a necessary and sufficient condition thatQ be positive definite is that a11 a12 > 0. a11 > 0 , a12 a22
I
3. Generally, show that
where Do = 1, D, = I aij I , i , j YN =
=
I
1 , 2,..., k, and
xN
+ C N
yk = xk
ckjxj. (Lagrange.)
j=lc+l
4. Hence, establish the result stated in (2.22.3). 2.23. Representation for A”
Consider the quadratic expression f ( Y ) = W , Y ) - ( Y , AY),
(2.23.1)
where A > 0. Then the minimum off(y) over y is uniquely determined as the solution o f Ay = x, (2.23.2) and minf(y) = (x,A-lx). (2.33.3) Y
Exercises
1. Show that B
> A > 0 implies A-l > B-l > 0 .
2. Does A 3 B imply A2 3 B 2 ? 3. Establish that B > A (2.23.3).
> 0 implies A-l > B-l > 0 without
use of
2.24. Canonical Representation for Arbitrary A
It is easy to extend the foregoing results to complex matrices A which are Hermitian, A T = A. (2.24.1)
2.24. C A N O N I C A L REPRESENTATION FOR ARBITRARY A
83
An immediate reduction to canonical form may be carried out if all of the characteristic roots of A are distinct. We have A
=
T-'AT,
(2.24.2)
where T is orthogonal in the extended sense, T = T-l; A and T are in general complex. A matrix T such that T = T-l is called unitary. If the characteristic roots are not distinct, we use the inductive approach mentioned earlier. With no restriction on the roots, we can carry out an inductive procedure to obtain a triangular form
(2.24.3)
This is useful in connection with linear differential equations. Another useful canonical form, although quite tedious to derive, is that due to Jordan. Consider the matrices of special form
(2.24.4)
T h e n it can be shown that a nonsingular transformation T exists, reducing a general matrix A to the form
+ + +
K, R , = N , the dimension of A. If all of the Fzi are where k , equal to one, we say that A has simple elementary divisors. We will make only a single particular use of (2.24.5) and thus not tarry for its demonstration. -
0
-
a4
MATRIX THEORY
Exercises
1. Show that unitary matrices form a group
2. Let A be a matrix with complex elements. A is called normal if A.*Ir A T A . Show that A is normal if and only if one of the follon i n g holds: :
I? j iC,where R and C are Hermitian and commute. (a) A (b) B has a complete set of orthonormal characteristic vectors. (c) A U T DlJ where U is unitary and D is diagonal. liH ZIU, uhe re C' is unitary and H is Hermitian. (d) A (e) T h e characteristic values of AAT are 1 A, 12, 1 A, 12, ..., 1 AN 12, where A, , A, ,..., Ah are the characteristic values of A. (f) T h e characteristic values of A are ~
~
~
~
'I
+
> ' 2
+
+
&?
>..')AN 1
1 he foregoing desirable properties focus attention on the problem of approximating to a given complex matrix B by a normal matrix A in the senbe of minimizing a suitable matrix norm I/ R A (1 for A a normal matri\. T h e question first was investigated by Mirsky; see also r .
~
I<. I,. Causey, On Closest Normal Matrices, Computer Science l h ision, ~ Stanford University, Technical Report CSIO, June 1964.
2.25. Perturbation of Characteristic Frequencies
I n m a n y situations the characteristic roots of a symmetric matrix A are associated with the natural frequencies of a physical system. We shall disciiss this in the following chapter. Ilihat happens to these frequencies \'i hen the system parameters are altered ? Very often, all that we want to knov is that they are decreased or increased. I n particular, what can we say about the charactcristic roots of A -1 B as compared to those of A when I j is positive definite ? It is intuitively clear that all characteristic roots should be increased. I t is clear that the sinallest characteristic root is increased. We have
(2.25.1 )
2.26. SEPARATION AND REDUCTION OF DIMENSIONALITY
85
With the aid of the foregoing min-max characterization, we can assert that all of the characteristic roots are increased. Furthermore, we can even provide a quantitative result, namely,
+ B, 2
+
(2.25.2)
Exercise
1. Is &(A)a convex or concave function of A ?
2.26. Separation and Reduction of Dimensionality
I n many investigations only the first few characteristic values are desired. Often only the smallest value is important; in other cases, the distance between the first two is significant. If A is high-dimensional, we may wish to try an approximate procedure of the following type. Set
c u,y'", M
x
=
(2.26.1)
k=l
where the y'k) are a set of orthonormal vectors, with M
uk scalars. T he n
< N , and the
c u,u,(y'k', Ay'" M
(x,Ax) =
k,L = l
BMu),
(2.26.2)
where BM = (y(", Ay")). Furthermore, (x, x) = (u, u).
(2.26.3)
What can we say about the characteristic roots of B,? T h e min-max representation given above allows us to state that
This is the PoincarC separation theorem.
MATRIX THEORY
86
Exercise
1. Consider the sequence of symmetric matrices defined by A, i , j : 1, 2 ,..., r , for r = 1, 2,..., N . T h e n
=
(aii),
2 X,(AJ 3 W i + J -
W t + I )
Obtain corresponding inequalities for the other characteristic values. 2.27. Ill-conditioned Matrices and Tychonov Regularization
Let us now return to the problem of obtaining a solution of an illconditioned system Ax = b. (2.27.1) Generally, we regard as a solution of (2.27.1) (l), to a prescribed degree of accuracy, a vector z satisfying the condition / / Az - b /I E , with some convenient norm. This relation is equivalent to
<
AZ - b where I/ y
jl
<
E.
(2.27.2)
=y,
This yields z
=
A-lb
+ A-ly
=X
+ A-ly.
(2.27.3)
If A is ill-conditioned, I/ k l y /I may be excessively large, even if is small. T h e problem of paramount interest in numerous investigation is that of determining x,the solution of (2.27.1), with some desired accuracy. There is no general solution to this problem. If, however, we add some additional information concerning x, information available from analytical, computational, or scientific sources, then we can often achieve success. One method, due to Tychonov, which has been applied with considerable success in recent years is that of regularization. In place of attempting to minimize, say,
11 Ax
-
b
(12
=
(Ax - 6, Ax
-
b),
(2.27.4)
we use the modified norm, (actually, the square of a norm), N ( x ) = (AX- b, A X - 6) + P(x),
(2.27.5)
where ~ ( x )is carefully chosen to take advantage of all that we know concerning x. Thus, for example, we may possess an approximate value
2.27. Ill-CONDITIONED MATRICES A N D T Y C H O N O V REGULARIZATION
87
for the solution of (2.27.1) based o n an entirely different approach to the original problem. Call this approximate value c. Consider then the problem of minimizing N(x, A)
=
(Ax - b, Ax
-
b) + X(x
-
c, x - c).
(2.27.6)
An easy calculation shows that the vector x which minimizes is given by (ATA
+
+ X1)x = (ATb + hc).
(2.27.7)
Observe that ATA XI is positive definite for X 3 0. If X is chosen adroitly (and this is where experience and experimentation count), we will have ATA XI far better conditioned than A , so that x can be el, calculated from (2.27.6), and yet also have (Ax- b, Ax - 6)
+
(x - c,
x - c)
<
Ep
<
.
T h e foregoing procedure gives rise to a method of successive approximations. Let c = x,,, and let x,+~ be determined by the condition that it minimize the expression (Ax
~
b, Ax - b )
+ X(x
-
x, , x
- x,).
(2.27.8)
We leave it to the reader to show that the sequence {xn} determined in this fashion converges to the solution of Ax = b for any h 3 0, provided that A is nonsingular. As will be seen, the rate of convergence of x, to the solution of (2.27.1) is dependent on the size of A. If X is small, it is rapid; if X is large, it is slow. However, if X is large, ATA 3- XI is well-conditioned; if X is small, ATA AI is ill-conditioned. A fine balance must be struck.
+
Exercises
1. Let x(X) denote the vector x which minimizes N ( x , A). What is the analytic structure of .(A) as a function of X ? Can we use extrapolation techniques to determine x(0) from the values of x(h) for sufficiently large A ?
2. Consider the minimization of
with respect to the u i . Show that the matrix of coefficients (the Hilbert matrix) is ill-conditioned. How do we avoid this problem in the determination of the minimum of the integral ?
MATRIX THEORY
88
2.28. Self-consistent Approach
I n other cases we do not possess any useful approximate solution to use for c. We do, however, usually know something about the components x, x l , x., ,..., x N , as functions of the index. For example, we may know that xk is monotone increasing or decreasing in K or convex. Or, we may merely know that xi and xiilmust be reasonably close together in value. In a situation of this type, we can take ~ ( x )in (2.27.4) to have the form V(X)
=
X[(X,
-
xl)'
+ (x3 - x')' + ... +
qJ(n-)
=
,A.([
~-
2x,
+
or
x1)2
(xN - xN-~)'],
4-(x4 - 2x, + x2)2
+ '..I.
(2.28.1) (2.28.2)
Again it is necessary to choose h carefully. I n Volume I1 we will indicate how dynamic programming may be applied to take advantage of the special structure of $(x). 2.29. Positive Matrices
A combination of research in stochastic processes and mathematical economics has produced a considerable interest in matrices with nonnegative elements. We shall restrain ourselves here to the proof of the fundamcntal result for matrices all of whose elements are positive. These are called positiz9e matrices. T h e more general case of nonnegativity, inbestigated first by Frobenius, requires delicate and detailed arguments. A basic result, due to Perron, is the following: Theorem. If A is n positive matrix, there is a unique characteristic root of A , h ( A ) ,which hns greatest absolute value. This root is positive and simple and its associated characteristic vector may be taken to be positive.
There are many proofs of algebraic, analytic, and topological nature of this elegant result. T h e proof we present below is important for our purposes since it provides a variational formulation for h ( A ) which furnishes a means of obtaining upper and lower bounds. I n addition, the characterization can be extended to the more general operator case. We will give an application to integral equations in Chapter 8. 'The root h ( A ) is oftcn called the Perron root. \I;c will wish to consider nonnegative vectors, which is to say vectors all of whose components are nonnegative. We will write x>,o
(2.29.1)
2.30. VARIATIONAL CHARACTERIZATION OF h(A)
89
to signify this. We say x is positive, x > 0, if all of its components are positive. T h e notation x 3 y will mean that x - y 0. T h e point of this is that a nonnegative matrix preserves the region x 3 0, i.e., x 3 0 and A nonnegative imply A x >, 0. Th is invariance plays a fundamental role in what follows. Exercise
1 . Let A be of dimension N . If A-l exists and more than N of the elements of A are positive, then A-l has at least one negative element. T h u s A = P D with P a permutation matrix and D a diagonal matrix with positive elements if and only if A and A-l have nonnegative entries. (R. Spira.) 2.30. Variational Characterization of h(A)
T h e proof of the foregoing theorem will follow as a consequence of: Theorem. Let A be a positive matrix. Let S(X) be the set of nonnegative h for which there exist nonnegative vectors x such that A x 3 Ax. Let T(h) be the set of positive X for which there exist positive vectors x such that Ax Ax. Then A(A) = max A, A E S(A), (2.30.1) = min A, A E T(A),
<
We begin by normalizing all nonnegative vectors we consider. We take N
I/xjl = E x i = 1.
(2.30.2)
i =1
Th i s excludes the null vector automatically. Let us employ the norm
I/ A I/ = for a nonnegative matrix. If Ax
c N
aij
i,+l
(2.30.3)
< A x , we have (2.30.4)
whence (2.30.5)
MATRIX THEORY
90
Hence S(A)is a bounded set, and clearly not empty if A is positive. Let A,
=
x E S(X),
sup A,
and {hi} be a sequence of A's in S(A) converging to A, . Let {#} associated set of vectors, by which we mean hiX'i'
< Ax'i',
i
=
1, 2 ,....
(2.30.6) be an (2.30.7)
Since j / X ( ~ ) = I ~ 1, which implies that all of the xi lie in a bounded region of N-dimensional space, we can choose a subsequence, say {x(j)},which converges to x(O), necessarily nontrivial since Ij x(O)(I= 1. Since, by virtue of (2.30.7), X,x'"' < Ax',', (2.30.8) it follows that A, E S(A), whence the supremum is actually an attained maximum. I t remains to show that the inequality is actually an equality. T h e proof is by contradiction. Suppose that strict inequality holds in the first component. Then, letting do)have the components xl,x2 ,..., xN , we have
4 + x,x, <
c N
a1pj
j=1
, (2.30.9)
where dl > 0. Let us show that (2.30.9) permits us to modify do)in such a way that A, may be increased. Choose (2.30.10)
I t follows from (2.30.9) that
4lY < AY,
(2.30.11)
with strict inequality. This contradicts the maximum property of A,. Hence, (2.30.9) cannot hold and (2.30.8) is an equality. X,x'O'
=
Ax'"'.
(2.30.12)
2.31. PROOF OF MINIMUM PROPERTY
91
Thus, A, is a characteristic root of A , and do)is an associated characteristic vector. We must now show that A, = A(A). We proceed again by contradiction. Assume that there exists a characteristic root A with I A I 2 A,. Let x be its associated characteristic vector. From Az = Az, we have 1x1 1x1 G A l 21, (2.30.13) where I x I denotes the vector whose components are the absolute values of the components of x. It follows from the definition of A, that I X I A, . Hence, we must have I A I = A, . But then the same argument as before shows that (2.30.13) must be an equality. Hence
<
I
I
=
A1 2
I?
(2.30.14)
which means that x = clw, with w > 0 and c1 a complex number. Thus, Az = Ax is equivalent to A w = Aw, whence A is real and positive and thus equal to A,. Let us finally show that w is equivalent to x(O), which means that apart from scalar multiples there is only one characteristic vector associated with A, . Let x be another characteristic vector associated with A(A).As above, we see that it may be taken to be positive. T h e n do)+ EZ for all scalar E is a characteristic vector associated with A, . Decreasing E to some negative value, we meet a first value of E for which one or several components of do) EZ are zero, with the remaining components positive, provided d o ) and x are really linearly independent. This contradicts the relation A(x'0' E x ) = X(A)(x'O' €2) (2.30.15)
+
+
+
since A is positive. Exercises
1. Use the Jordan canonical form to show that A(A) must be a simple root.
2. Let ANPIbe obtained from A by striking out an ith row and a j t h column. Show that A(A) > A(ANA1). 3. Show that A(AT)= A(A). 2.31. Proof of Minimum Property
T h e minimum property may be derived by repeating the foregoing steps. However, it is worth pointing out that it follows readily by using
92
MATRIX THEORY
the adjoint operator, the transpose matrix. Suppose that Ay some y > 0. l’hen for any x > 0, A(z, y )
2 (2,Ay) = (ATz,y ) .
< Ay
for
(2.31.1)
Let z be a characteristic vector of AT associated with A(A).T h e n y)
Since (z,y )
z (ATz,Y ) = W ( z ,Y ) .
(2.31.2)
> 0, we havc X 3 X(A). 2.32. Equivalent Definition of A(A)
T h e definition of X(A) given in the theorem of Sec. 2.30 is equivalent
to the representation
I
=
1
min max I
1
i
1 uijxj/xi
ii:l
(2.32.1)
Judicious choices of the xi yield upper and lower bounds for A(A). We return to this point in Chapter 8, where we give an example of the use of this method in connection with integral equations. Exercises
1. 1,ct c be any nonnegative vcctor with A a positive matrix. T h e n ZI
7
lim [Anc h(A)’”] n ,a
cxists and is a characteristic vector of A associated with X(A).
2. If ,4 is such that a,, > 0, i # j , then the root of A with largest real part, p ( A ) , is real and simple. There is an associated characteristic vector which is positive and unique up to a scalar multiple. Furthermore,
2.32. EQUIVALENT DEFINITION O F X(A)
3. If aij 3 0,
aii
93
< 1 , j = 1 , 2,..., N , then the system
+ c aijxj , N
xz = yi
i = 1,2 )...)N ,
3=1
has a unique solution which is positive if theyi are positive. If aij > 0, then (I - A)-l is positive.
4. Under the same conditions, show that I I - A I > 0. Hint: Consider I I - XA I for 0 < X < 1. 5. Let A be a positive matrix. Then A(A) = maxmin R'
2
R'
I
xi
where R' is defined by xi 3 d > 0, xi = 1, and d is a parameter depending on A. Specifically, we may take
6. X(A) is the unique solution of
R
aiixj
r
j=1
+ A(1
-
xi)
7. X(A) may be obtained as the limit of the sequence (un>determined by
=
min max {...}, R'
i
(xj
with u,, = min,, maxi aijxj). Show that the convergence is monotone. T h e foregoing is closely related to the theory of games, particularly to multistage games. See
R. Bellman and J. P. LaSalle, Non-zero Sum Games and Stochastic Processes, T h e RAND Corporation, RM-271, 1949.
MATRIX THEORY
94
L. Shapley, “Stochastic Games,” Proc. Natl. Acad. Sci. U . S . , Vol. 39, 1953, pp. 1095-1100. R. Bellman, “On an Iterative Procedure for Obtaining the Perron Root of a Positive Matrix,’’ Proc. Amer. Math. SOC.,Vol. 6, 1955, pp. 719-725.
8. Let M be a Markoff matrix with positive elements and consider the recurrence relation xntl = M x , , xn = c, with c a probability N vector, that is, ci 2 0, ci = 1. Show that x, converges as n 4 co a_nd that it converges to the unique probability vector satisfying x = Mx. 9. If we wish to calculate x, is it better to proceed via (Ma)c or via M(Mrn-1~)if we are thinking in terms of time and accuracy ? 10. Let A be a positive symmetric matrix. Use the Rayleigh quotient and the representation of the Perron root given in Sec. 2.32 to obtain upper and lower bounds for h(A). Miscellaneous Exercises
1. If B = limN+mAN, what can be said about the characteristic roots of B ? See
0. Taussky, “Matrices with Cn+O,” pp. 5-10.
/.
Algebra, Vol. 1, 1964,
2. Let Y be a positive definite 2 x 2 matrix. Consider the integral ]( Y ) =
x>n
e-17(xy)IX
ls-3/2
dx,, dx,, dx,,
,
where the notation indicates region of integration is that determined by x l l > 0, xllxT2- x?2 > 0. T h e n
for Re(s) > 3/2. (Ingham-Siegel.) Integrals of this type arise in analytic number theory and mathematical statistics. For further results, see
K. Rellman, “A Generalization of Some Integral Identities due to
Ingham and Siegel,” Duke Math. J., Vol. 24, 1956, pp. 571-578. I. Olkin, “A Class of Integral Identities with Matrix Argument,” Duke Math. J., VoI. 26, 1959, pp. 207-213.
MISCELLANEOUS EXERCISES
95
3. Show that
where the contour of integration in each integral is I xi I suitable hypotheses concerning A.
=
1, under
4. Evaluate 2ni
~f(z)(xI A)-l dz
under suitable assumptions concerning f ( z ) ,A , and the contour C . This contour integral can be used as a basis for the construction of a theory of functions of matrices. See
R. F. Rinehart, “The Equivalence of Definitions of a Matric Function,” Amer. Math. Monthly, Vol. 62, 1955, pp. 395-414.
R. F. Rinehart, “The Derivative of a Matric Function,” Proc. Amer. Math. SOL.,Vol. 7, 1956, pp. 2-5. R. F. Rinehart, “Elements of a Theory of Intrinsic Functions on Algebras,” Duke Muth. J., Vol. 27, 1960, pp. 1-20. = (uii) is a Jacobi matrix, that is, aij = 0 for I i - j J 2 2, under what conditions are the elements aii determined by the characteristic roots ?
5. If A
6. If A is positive definite, then
where the integration is over a k-dimensional linear subspace R of N-dimensional space whose volume element is dVk and the maximum is taken over all R.
7. If A is positive definite, then I A I
< a,,a,,
-*-
uNN.
8. Establish Hadamard’s inequality: If B is a real matrix, then
Hint: Apply the previous result to A = BB‘. For the connection with integral equations and Hadamard’s thoughts concerning this, see
MATRIX THEORY
96
J. Hadamard, The Psychology of Invention in the Mathematical Field, Princeton University Press, Princeton, New Jersey, 1949.
9. If A is a positive definite matrix with the property that aii < 0 for i f j , then A-l has all positive elements. (Stieltjes.) 10. Show that 1 I1 A,
1 1
AN-1 1
... 1
A,
... AN
AN-1
...
2
I l
AN-l
N
the Vandermonde determinant.
1 1. Evaluate the Cauchy determinant
12. If A and B are positive definite, then for all x and y , (A-'x,
x)
+ (B-'y, y ) 2 ( ( A+ B)-'(x + Y ) , + Y ) .
13, Let A be a positive definite matrix and let A, 3 A, 3 ... 3 An be the characteristic values. Then ( x , Ax)(x, A-'x)
+ ($1
G - [(Aj2 4
A*
2
>0
ljZ
Show that the upper bound can be achieved. (Kantorovich.)
14. Let z be an N-dimensional vector such that (x,2) = 1. Determine for these x the minimum of (z- 6,A ( x - b ) ) , where A is a Hermitian matrix. See G . E. Forsythe and G. H. Golub, "On the Stationary Values of a Second-degree Polynomial on the Unit Sphere," J . Soc. Indust. Appl. Math., Vol. 13, 1965, pp. 1050-1068.
15. Let A , B be positive definite. Th en t r [ ( ~ ~ ) 2 " +< ' ] tr[(~2~2)2"] for n = 0, I ,.... (Golden.)
97
MISCELLANEOUS EXERCISES
16. Let F denote the set of nonnegative functions f ( t ) defined over [0, I]. Let T be any nonnegative linear one-to-one mapping of F into itself. T he n Tf(t)= g ( t ) f ( h ( t ) ) , where g is a particular function and h(t) is a permutation of the domain of F. (T. A. Brown, M . L. Juncosa.) 17. Consider the matrix A teristic roots. Then tr(A2)
=
N
(aij). Let A,, A, ,..., h, be the charac-
I uij l2 3
=
N
I hi 12.
(Schur.)
i=l
i,j=l
18. Using the foregoing result and the matrix 0 0
a, 0 .’. 0 0 u2 ...
UN
0
0
where the ai 3 0, establish the arithmetic mean-geometric mean inequality
19. Let A and B be real symmetric matrices. A necessary and sufficient pB,A, p real scalars, condition that the pencil of matrices AA contain a positive definite matrix is that (x, Ax) = 0,(x,Bx) = 0 imply that x = 0, provided that the dimension, N , is greater than 2. (Calabi.)
+
20. Any pencil h A
+
pB for which the foregoing holds can be transformed into diagonal form. See 0. Taussky, “Positive Definite Matrices,” Inequalities, Academic Press, New York, 1967.
21. Let A, B be real symmetric matrices, x real. Consider the values y, = (x, Ax),y2 = (x, Bx) for (x,x) = 1 . Show that the set of points in the ( y , ,y,)-plane is convex. (Brickman.) 22. Show that
f f
..‘ el G.. .G ~ ,
I %(%)I
I +i(.i)l 4 d X 2 ... d,.
=
MATRIX THEORY
98
T h i s useful result is due to C. Andreief. See
G. Polya and G. Szego, Aufgaben und Lehrsatze aus der Analysis, Dover Publications, New York, 1947. For some analogous results, see N. G. deBruijn, “On Some Multiple Integrals Involving Determinants,” J. Indian Math. SOC.,Vol. 19, 1955, pp. 133-1 51. Here integrals of the form
are treated. 23. Using the previous exercise, obtain some necessary and sufficient conditions for the existence and uniqueness of the solution of ”T
See
R. Bellman, “A Note on the Identification of Linear Systems,” Proc. Amer. Math. SOC.,Vol. 17, 1966, pp. 68-71. 24. Let { X }be a linear space of N x N real matrices with the property that every linear combination has only real characteristic values. is a convex matrix function of X and A,,,(X) is a Then A,,,(X) concave matrix function. 25. Let X and Y be two real matrices all of whose linear combinations have real characteristic values and suppose that X < 0. Then the iY have negative real parts. characteristic values of X
+
26. Let { X }be as above. T h e n A,(X), A2(X)..., AAX), the N characteristic roots of X are monotone functions of X . For the foregoing with applications to hyperbolic partial differential equations, see
P. D. Lax, “Differential Equations, Difference Equations and Matrix Theory,” Comm. Pure Appl. Math., Vol. 11, 1958, pp. 175-194.
+
AAx = c. Show that the Liouville27. Consider the linear equation x A2A2c - . * obtained by iteraNeumann solution x = c - AAc tion converges within the circle I A I < 1/1 A, I , where A, is the, or a, characteristic root of A of largest absolute value.
+
+
99
MISCELLANEOUS EXERCISES
28. Show that if A is positive definite and ,A = 1, we can make the change of variable p = h / ( l A), obtaining x(1
-
+ p) + pAx
=c
+
- CP
m
and, by iteration, x = c - Acp Cn=2p n ( A - 1)”( - A c ) , and assert that the new series converges for all A 3 0. See R. Bellman, “On Perturbation Methods Involving Expansions in Terms of a Parameter,’’ Quart. Appl. Math., Vol. 13, 1955, pp. 195-200.
-
29. Letf(8) be a continuous function in (-n, n) possessing the Fourier m m seriesf(8) C,E-m a,eins. Thenf(8) ecik6 a,+keine.Hence N
x,=-m
Thusf(8) is in some sense represented by the infinite matrix What connection, if any, exists between values off (8) and characteristic roots of the finite matrix (+J, i , j = 1 , 2, ..., N ? These matrices are called Toeplitx matrices. See M. Kac, W. L. Murdock, and G. Szego, “On the Eigen-values of certain Hermitean Forms,” J. Ratl. Mech. Anal., Vol. 2, 1953, pp. 767-800. Matrices of this type are of interest in connection with “nearest neighbor” problems of the kind that arise in statistical mechanics, such as the Ising model.
30. Let H be a positive definite Hermitian matrix. Then
where x
=
x
+ iy.
R. Bellman, “Representation Theorems and Inequalities for Hermitian Matrices,” Duke Muth. J., Vol. 26, 1959, pp. 485-490.
loo
MATRIX THEORY
31. A skew matrix is one for which AT = - A . Show that every real
skew matrix is a square root of a negative definite real symmetric matrix and that every skew hermitian matrix is a square root of a negative definite, or semidefinite, hermitian matrix.
R. F. Rinehart, “Skew Matrices as Square Roots,” Amer. Math. Monthly, Vol. 67, 1960, pp. 157-161.
For some geometric aspects of skew-symmetric matrices, see
H. Schwardtfeger, “Skew-symmetric Matrices and Projective Geometry,” Amer. Math. Monthly, Vol. 3, 1944, pp. 137-148.
+
iH,I 3 I H I I whenever H , is a positive 32. Hence show that j H I B 1 3 I A I when definite Hermitian matrix and thus that I A A is a real nonnegative definite matrix and B is skew-symmetric. (A. Ostrowski and 0. Taussky.)
+
33. L e t f ( T ) be a complex-valued function of the variable T ranging over symmetric matrices. Set d --f(T) dT
where c i j
=
1 if i
=j
d
--
dT
and
1T1
=
= (Cij
=
+ if i # j . Show that
1 T 1 T-l i f T-l exists.
R. Bellman and S. Lehman, “ T h e Reciprocity Formula for Multidimensional Theta Functions,” Proc. Amer. Math. SOC.,Vol. 12, 1961, pp. 954-961.
+
34. Let A and B be positive definite matrices. Show that (A2N B2N)112N converges as N + GO to a matrix C = C ( A , B ) such that C 3 A , C >, B. Show that
35. Let A and B be positive definite. Determine the minimum over X of tr X where X >, A , X 3 B. Write M ( A , B ) for the matrix
BIBLIOGRAPHY AND COMMENTS
101
which minimizes. Similarly write M ( A , , A , ,..., AN)for the matrix which minimizes t r X where X 2 A i, i = 1 , 2,..., N . Does M ( A , , A , , A3) = M ( A , , M ( A 2 ,A3))hold ? = c + A x and the method of successive A x , , n 3 0. Under what conapproximations xo = c, x , + ~ = c ditions on A does the sequence {xn> converge to x ?
36. Consider the equation x
+
37. Let us write the equations of approximation in component form
+ C u ~ ~ x ~ i) =, 1, 2,..., N , N
x?+’)
=c
j=1
xio) = ci , and then a modified version in which we “update” the values of the ( x r ) ] ,namely,
y!n+l) = c
+ C u i j y l ) + ,Z uijyj(n), i-1
j=1
N
y‘0’
=
ci ,
3=a
= 1, 2, ..., N . Does the new system yield a convergent sequence, and if so, does it converge more rapidly or more slowly than the original ?
i
38. What conditions must b and c satisfy so that Xb definite ? (Wimmer.)
=
c, with X positive
Bibliography and Comments 52.1. For a general introduction to matrix analysis, see
R. Bellman, Introduction to Matrix Analysis, McGraw-Hill, New York, 1960. (Second Edition, in preparation.)
L. Mirsky, Introduction to Linear Algebra, Oxford University Press, New York, 1955. We have avoided any detailed discussion of the art of solving the linear system A x See
=
b.
J. Todd, “Motivation for Working in Numerical Analysis,” Comm. Pure Appl. Math.,
V O ~8, . 1955, pp. 97-116. G. E. Forsythe, “Solving Linear Algebraic Equations Can Be Interesting,” Bull. Amer. Math. SOC.,Vol. 59, 1953, pp. 299-329. R. Bellman, R. Kalaba, and J. Lockett, Numerical Inversion of the Laplace Transform, American Elsevier, New York, 1966. J. R. Westlake, A Handbook of Numerical Matrix Inversion and Solution Techniques, Wiley & Sons, 1968. One aspect of closure, one of the themes of this volume, is the production of an algorithm which calculates the desired values with as few additional values as possible.
102
MATRIX THEORY
Thus, for example, given the linear system Ax = b, we may want to calculate x1 , the first component of x, without calculating the other values. Cramer’s rule accomplishes this, but not in a practical fashion, as we have noted. In some cases, Monte Carlo techniques can be used. In other types of problems, stochastic approximation can be employed. I n general, the task of constructing an algorithm leading to the desired information and only this information is an intriguing one, and little has been done in this area. 92.2. For Exercises 4, 5 , 6, see
R. Bellman, “Invariant Imbedding and Multipoint Boundary-value Problems,” J. Math. Anal. Appl., Vol. 24, 1968, pp. 461-466. $2.3. We meet ill-conditioned systems in numerical inversion of the Laplace transform and in the solution of Fredholrn integral equations in general. See the Bellman, Kalaba, Lockett book referred to in 92.1 above. We also encounter them in the study of improperly posed problems of mathematical physics, and of inverse problems. See
M. M. Lavrentiev, Some Improperly Posed Problems of Mathematical Physics, Springer, New York, 1967. R. Lattes and J. L. Lions, Mkthode de Quasi-reversibilitd et Applications, Dunod, Paris, 1967. English translation, American Elsevier, 1969. Extensive references to the work of Tychonov will be found there. See also,
R. Bellman, R. Kalaba, and J. Lockett, Systems,” J. Math. Anal. Appl., Vol. R. Bellman, R. Kalaba, and J. Lockett, Systems,” J. Math. Anal. Appl., Vol.
“Dynamic Programming and Ill-conditioned 10, 1965, pp. 206-215. “Dynamic Programming and Ill-conditioned 12, 1965, pp. 393-400.
A discussion of the method of “pivoting” may be found in
G. 8. Uanzig, Linear Programming and Its Extensions, Princeton University Press, Princeton, New Jersey, 1964. What we are desirous of emphasizing is the fact that numerical computation is a control process in which the objective is to minimize the overall error. This means that it should be a “feedback control process,” an adaptive process in which all of the information available at each stage of the computation is used to guide the subsequent calculations. This in turn implies that the computational process may be viewed as a stochastic control process. For further details, see
R. Bellman, Adaptive Control Processes: A Guided Tour, Princeton University Press, Princeton, New Jersey, 1957. 92.4. See
G. Temple, “Linearization and Delinearization,” Proc. International Congress of Mathematicians, 1958, pp. 14-21. 92.13. See
A. Ostrowski, “Uber Normen Von Matrizen,” Math. Zeit., Vol. 63, 1955, pp. 2-18. 92.14. For an extensive survey of far-reaching generalizations, see
F. V. Atkinson, “Multiparametrjc Spectral Theory,” Bull. Amer. Math. SOC.,Vol. 74, 1968, pp. 1-27.
BIBLIOGRAPHY A N D COMMENTS
103
92.15. We are interested in the minimization and maximization of quadratic forms because of their applicability to the study of more general variational problems. See Chapter 8. 92.15. See
R. J. Duffin, “The Rayleigh-Ritz Method for Dissipative or Gyroscopic Systems,” Quart. Appl. Math., Vol. 18, 1960, pp. 215-222. R. J. Duffin, “A Minimax Theory for Overdamped Networks,” J. Ratl. Mech. Anal., Vol. 4, 1955, pp. 221-233. R. J . Duffin, and A. Schild, “On the Change of Natural Frequencies Induced by Small Constraints,” J. Ratl. Mech. Anal., Vol. 6, 1957, pp. 731-758. 92.21. For some ramifications of this concept, see 0. Taussky, “Positive-definite Matrices,” Inequalities, Academic Press, New York, 1967. See the discussion on pp. 108-109 of the book by Bellman cited in 92.1 for references to the far-reaching work of Loewner on monotone matrix functions; i.e., analytic functions such that A B implies f ( A ) > f ( B ) .
$2.29. Perron’s original result was obtained in connection with the study of the multidimensional continued fractions of Jacobi. See 0. Perron, “Zur Theorie der Matrizen,” Math. Ann., Vol. 64, 1907, pp, 248-263.
A resurgence of interest in this area is due to the study of branching processes. See
T. E. Harris, The Mathematical Theory of Branching Processes, Springer-Verlag, Berlin,
1963. G. Birkhoff and R. S. Varga, “Reactor Criticality and Nonnegative Matrices,” J. SOC. Indust. Appl. Math., Vol. 6, 1958, pp. 354-377.
Problems of this nature can also be studied by means of differential equations. See
P. Hartman and A. Wintner, “Linear Differential and Difference Equations with Monotone Solutions,” Amer. J. Math., Vol. 1, 1953, pp. 731-743. A new powerful method for treating positive operators has been developped by G. Birkhoff using the Hilbert projective metric. See
G. Birkhoff, “Uniformly Semi-primitive Multiplicative Processes,” Trans. Amer. Math. SOC.,Vol. 104, 1962, pp. 37-51. R. Bellman and T. A. Brown, “Projective Metrics in Dynamic Programming,” Bull. Amer. Math. SOC., Vol. 71, 1965, pp. 773-775. See the books cited in 92.1 for further references to nonnegative matrices.
Chapter 3
MATRICES AND LINEAR DIFFERENTIAL EQUATl O N S
3.1. Introduction
I n this chapter we wish to discuss the use of matrices in the solution of linear systems of differential equations of the form (3.1.1)
i
=
I , 2, ..., N . Using matrix notation, this takes the compact form dx
-
dt
=
AX + f ( t ) ,
~ ( 0=) C.
(3.1.2)
\.lie will also consider some particular matrix equations of importance, dX - -- A X ,
dt dX -= AX dt
(3.1.3)
+ XB,
(3.1.4)
and the Kiccati matrix equation dt
=
A
+ BX + XC + XDX.
(3.1.5)
T h e first equation is classical; the second has risen in prominence over the last fifty ycars; the third has emerged as one of the fundamental equations of analysis and applied mathematics over the last ten years in connection with control theory, dynamic programming, and invariant imbedding. 3.2. Vector-Matrix Calculus
We wish to consider vectors and matrices which depend on the independent variable t . We write x ( t ) and A(t) to indicate 104
3.3. EXISTENCE AND UNIQUENESS OF SOLUTION
105
this, when necessary, and introduce in the expected fashion the derivatives dx -_
d A - (+) _ at
,
dt
-
(3.2.1)
and the integrals
Using the foregoing notation, we can write the system of linear differential equations in (3.1.1) more simply as dx dt
~ ( 0= ) C.
- = A(t)x + f ( t ) ,
(3.2.3)
It remains to show that there are advantages to this compression of data. We will begin with the homogeneous equation dx _ - Ax, dt
(3.2.4)
x(0) = c,
where A is a constant matrix. Exercises
1. Show that (1
$, x dt, I( < Jb 11 x 11 dt, .
2. Show that d / d t ( X Y ) ( d X / d t )y -1 X(dy/dt).
=
(dX/dt)Y
+
X ( d Y / d t ) , d/dt(Xy)
=
3 . Show that d/d t( X- l) = - X - l ( d X / d t ) X - l . 4 . Show that d/dt(x,y) = ( d x / d t , y )
+ (x, dy/dt).
3.3. Existence and Uniqueness of Solution
Let us begin by demonstrating the existence and uniqueness of the solution of (3.2.4), employing the method of successive approximations.
106
MATRICES A N D LINEAR DIFFERENTIAL EQUATIONS
Set x0 =- c,
x,+,
+
c
:
st
(3.3.1)
A x , dt,.
0
I t is easy to see inductively that x, is a polynomial of degree n in t . We have xo = c,
x1
=
c
+ Act,
x,
--
c
+ Act + ... + Anctn/n!
x,
,- x, = An+1ct?2+1/(n+ I)!
We have and thus
/I xn+l
-
xn /I < I1 A
Iln+lllc
II I t In+l/(n
(3.3.2)
(3.3.3)
+ l)!
(3.3.4)
Hence the series Cn=o11 x,+, - x, 1 1 , and thus also the series m (x,+~ - x,), converges uniformly in any finite t-interval to a function x(t) which satisfies the integral equation m
x =c
+
st
A x dt,
(3.3.5)
0
and thus the original differential equation. T h e right to differentiate (3.3.5) is not immediate since all we know 1 about x from its construction is that it is continuous. But then Ax dt, as an integral is differentiable, which implies that x as given by (3.3.5) is actually differentiable. We shall employ this circuitous route again in Chapter 4. From (3.3.2), we see that we may write
so
(3.3.6) I t remains to establish uniqueness. Let y be another solution of
(3.2.4). T h e n we have successively d dt
-(x
-
x -y = IIx-yIl
I' 0
<Jt
A(x
-
y ) dt, ,
IIAIIIlx-yIldt,
(3.3.7)
107
3.4. THE MATRIX EXPONENTIAL
From the last relation we have (3.3.8)
for any
E
> 0. Using the fundamental )) x
-
lemma in Sec. 1.3, we have
y /) ,< eetllRII.
(3.3.9)
Since this holds for any E > 0, we must have 1) x - y 1) that x = y , the desired uniqueness.
=
0. T h is ' means
3.4. The Matrix Exponential
T h e formal analogy of the series in (3.3.6) to what is obtained in the scalar case, where the solution is eatc, prompts us to introduce the matrix exponential defined by the series (3.4.1)
T h e estimate used in (3.3.4) shows that the matrix series converges uniformly in any finite t-interval. Hence, we see that e A t satisfies the matrix equation -dX _
dt
-
AX,
(3.4.2)
X(0) = I.
T h e uniqueness of solution of (3.4.2) follows from the corresponding result for the vector equation. T h e functional equation eA(t+s)
~
(3.4.3)
eAteAs
can be demonstrated as in the scalar case by series expansion in powers of the exponent and appropriate grouping of terms using the absolute convergence of the series involved. From (3.4.3) we derive the basic result I
Hence,
eAt
= e A ( t - t ) = eAte-At.
is never singular and its reciprocal is
(3.4.4) erAt.
MATRICES A N D LINEAR DIFFERENTIAL EQUATIONS
108
Exercises
1. I t is instructive to regard (3.4.3) as an expression of the law of causality. Derive it from the uniqueness of solution of the equation dX
-~ AX,
X ( 0 ) = eAs.
-
dt
2. Show that we can establish uniqueness for (3.4.2) by viewing it as N vector equations of dimension N or as one vector equation of dimension N 2 .
3 . Using the functional equation of (3.4.3), establish the functional equations for cos t and sin t. 4. Show ~ ( 1 " )
that
-1
we
alu(N-l)
4-
can
establish
+ a,zi
7
existence
and
0, ~ ( ~ ' = ( 0ci ) ,i
by viewing it as equivalent to the system 21' =
u1 , ul'
= u2
,..., u;YPl =
u(0) = co ,
5. Show that
eAf
= lininim ( 1
-aluN--l
Ul'(0)
--
Ant.
uniqueness
0, 1 ,..., N
a2uN--2...
= c1 ,...
~
for 1,
-
aNu,
.
+ At/@)".
6. Consider the differcnce equation xntl X,
-
=
=
Ax,, x,,
= c.
Show that
+
7. Consider the difference equation y ( t A ) = Ay(t),y ( 0 ) = c, where t 0, 4 , 2 4 , .... Examine the convergence of y ( N A ) , where NA = t , as 4 0. ~
---f
3.5. Commutators
T h e relation in (3.4.3) is an analogue of what holds in the scalar case. On the other hand, eAleBl # e ( A + B ) t (3.5.1) for general A and B. If equality holds in (3.5.1) for -co we have, upon expanding,
< t < 00,
3.5. COMMUTATORS
109
Examining the coefficient of t2 on both sides, we have
B2 A2 -+-+AB= 2 2
A2
+ AB + B A 2
+B2
(3.5.3)
or AB
=
(3.5.4)
BA.
Thus, equality can hold only if A and B commute. If A and B do commute, we can group terms as in the scalar case to show that equality holds in (3.5.1). Since eAteBtis nonsingular, we know that it can be written as an exponential (see end of Sec. 3.7). The task of determining the “logarithm” of eAteBtis an interesting and important one. We leave it to the reader to show formally that eAteBt =
exp[(A
+ B)t -1(AB
-
+ ....I
(3.5.5)
BA)t2/2
T h e expression [ A , B ] = A B - B A is called the commutator of A and B and is a basic matrix function of A and B . T h e series obtained in the exponent is called the Baker-Campbell-Hausdorff series and plays a vital role in many investigations. See the exercises after Sec. 3.12 for some results in this area. Exercises
1. Let A and B be matrices which do not necessarily commute. Then &A+B)
=
lim
(eAt/neBt/n)n.
n-tm
See
H. F. Trotter, “On the Product of Semigroups of Operators,” Proc. Amer. Math. SOC.,Vol. 10, 1959, pp. 545-551. This has interesting applications to the study of the Schrodinger equation with Feynman integrals. See W. G. Faris, “The Trotter Product Formula for Perturbations of Semibounded Operators,” Bull. Amer. Math. SOC.,Vol. 73, 1967, pp. 211-215.
+
+
2. Consider the equation X = C € ( A X X B ) , where m parameter. Write X = C + enyn(A,B), where yo(A, B )
=
C,
yn(A, B) = Ayn-i(A, B )
+ ~ n - i ( AB)B, ,
E
is a scalar n
2 1.
110
MATRICES A N D LINEAR DIFFERENTIAL EQUATIONS
Show inductively that B ) = AnC
where
+ (7)A"-lCB + );(
An-2CB2
+ ... + CB",
(2) are the binomial coefficients.
3. Introduce a position operator P with the property that when it operates on a monomial consisting of powers A and B in any order with C somewhere, it shifts all powers of A in front of C and all powers of B after C. Thus, P(AaiBB1 . .. AakBBkBBkCAak+iBflk+i .. . Aa~Bfl,)= AzUiCBz@i.
Further, define P to be additive, P(mi(A, B )
+ m2(4 B ) )
=
P(m,(A, B ) )
+ P(m2(A9 B)),
where m1 and m, are monomials of the foregoing type. Show that %(A, B ) = P ( ( A qnC).
+
+ B)]-l C . E + <(AX+ X B + C X D ) , then
4. Show that X 5. If X
=
=
P[l X
-
=
€(A
+ B + CD)]-lE.
P[I - €(A
3.6. lnhomogeneous Equation Consider next the equation dx
- = AX + f ( t ) ,
dt
~ ( 0= ) C.
(3.6.1)
We obtain the solution using an integrating factor as in the scalar case. Write
(3.6.2)
whence (3.6.3)
3.7. THE EULER SOLUTION
111
Exercise
1. Obtain this solution using the Laplace transform. 3.7. The Euler Solution T h e equation
dx _ -- A x , dt
x(0)
= c,
(3.7.1)
can also be solved in the following fashion. Ignoring the initial condition, let us look for particular solutions of the form x = eAtb,
(3.7.2)
where h is a scalar and b is a constant vector. T h e n (3.7.1) leads to (3.7.3)
hb = Ab,
which means that h is a characteristic value of A and b is an associated characteristic vector. Let us suppose for the moment that the characare distinct, and let T be a matrix which teristic values A,, A, ,...,A, reduces A to diagonal form,
(3.7.4)
Setting x =
(3.7.5)
Ty
we have
dx -- T -dy _ dt
dt
z=
AT9
(3.7.6)
whence (3.7.7) From x(0) = Ty(0) = c we obtainy(0) = T-L. Hence, we see the connection between the matrix exponential solution and the usual solution in terms of exponentials.
112
MATRICES A N D LINEAR DIFFERENTIAL EQUATIONS
We have
[
eAt = T e A t l ' - l
-
T
ehzt
.
..
T-1.
(3.7.8)
If A has multiple characteristic roots, the representation requires more than exponcntials in general. T o obtain the form of eAt we can proceed in either of two ways, use of the Jordan canonical form or a canonical representation in triangular form. Writing
(3.7.9)
where the Ji are the Jordan matrices
(3.7.10)
we see that
We leave the determination of the analytic form of eJa' as an exercise. Alternatively, we can use a representation of the form
3.8. STABILITY OF SOLUTION
113
where the elements below the main diagonal are zero. This is a considerably simpler result to establish. T h e equation for y , where x = Ty as before,
(3.7.13)
shows that if X is a multiple root, then in general, terms of the form tk-leAt will arise when k is the multiplicity of A. We will avoid these more complex cases in what follows and therefore content ourselves with the cursory discussion. Exercises
1 . If A has distinct characteristic roots and A is nonsingular, show that A = eB. 2. Use the Jordan canonical form to establish this exponential representation for general nonsingular A. 3. Obtain the exponential representation without use of the Jordan canonical form.
4. If xntl = Ax, , xo = c, obtain the analytic representation of x, .
+
5 . Consider the vector equation x" Ax = 0 where A > 0. Show that if a11 of the characteristic roots of A are distinct, the general solution is a linear combination of elementary oscillations.
6. Show that this result is true even if A has multiple roots. Hint: Use the first integral (x', x') + (x, Ax). 7. Show that we can obtain approximate values for the characteristic frequencies using variational techniques.
3.8. Stability of Solution
We are now able to answer a most important question: Under what conditions on A do all solutions of dx dt
-= Ax
(3.8.1)
114
MATRICES A N D LINEAR DIFFERENTIAL EQUATIONS
approach zero as t + m ? From what has preceded, we see that a necessary and sufficient condition that this be the case is that all of the characteristic roots of A have negative real parts. If this condition holds, we call A a stability matrix. T h e problem of testing for this condition without explicit calculation of the characteristic roots is a difficult one to which much attention has been directed in recent years.
Exercises
1. If A , B, C are nonnegative definite and either A or C is positive C I = 0 has no roots with positive real definite, then 1 X2A 2XB parts. Hint: Consider the behavior of the solutions of
+
+
+ 2Bx' + CX = 0.
AX"
sr
2. If x' = Ax,x(0) = c, and A is a stability matrix, then (x, Ax)dt can be evaluated directly in terms of the elements of A and the components of c .
3 . If A is a stability matrix, the solution of A y = b may be obtained as the limit of the solution of x' = Ax - b, x(0) = c as t + co.
4. What is the rate of convergence ? 5. Show that if A is nonsingular we can find a permutation matrix P and a diagonal matrix D such that D P A is stable. (Folkman.)
3.9. Linear Differential Equation with Variable Coefficients Let us next consider the equation
dx
- = A(t)x,
dt
x(0)
= c.
(3.9.1)
T o establish the existence and uniqueness of the solution of (3.9.1), we once again convert the differential equation t o an integral equation x
=
c
+
J t A@,)x(tl) 0
dt,
(3.9.2)
3.9. LINEAR DIFFERENTIAL EQUATION WITH VARIABLE COEFFICIENTS
115
and employ successive approximations, xo = c,
x,+1 = c
(3.9.3)
+ Jb4 t l ) Xn(t1) dtl .
T o establish convergence of x, we consider the series We have xn+l-
xn =
It
x:,"=, (x,,+, - x,).
~ ( ~ ( -x xn-1) n dt,
(3.9.4)
and thus
Let us suppose that A ( t ) is continuous for 0 k, = max, ]I A(t)l]. Then, from (3.9.5),
< t < to
and put
(3.9.6)
Since
we obtain inductively
xn=o m
Thus, once again, the series (x,+, -x ), converges uniformly in [0, to] which means that x, converges uniformly to a function satisfying (3.9.2) and thus (3.9.1). T h e proof of uniqueness follows as before. Let X ( t ) be defined as the solution of dX _ dt
-
A(t)X,
X ( 0 ) = I.
(3.9.9)
We can build X ( t ) u p out of N columns satisfying (3.9.1) for appropriate values of c or establish its properties directly proceeding as above. t t o , a fact which is readily T h e matrix X is nonsingular for 0 established by observing that Y = X-' satisfies the linear equation
< <
dY dt
- --
-YA(t),
Y(0) = I.
(3.9.10)
116
MATRICES A N D LINEAR DIFFERENTIAL E Q U A T I O N S
\Vc establish this in reverse. Start with the functions X and Y defined by (3.9.9) and (3.9.10). From (3.9.9) and (3.9.10) we see that d ctX dY (UX)= Y $- -X dt dt dt
-
=
YA(t)X
Y A ( t ) X = 0.
~
(3.9.1 1)
l ‘ h u s , Y X = C , a constant matrix. Setting t = 0, we see that C = I. Hence, Y = X-l. Exercises
I . Show that the solution of X‘ = A(t)X , X ( 0 ) = I , is nonsingular for t 3 0 if A ( t ) is integrable for t >, 0 by demonstrating the result of Jacobi,
I X(t)I = exp
[Jttr(A(t1)) dt,]
.
2. Show that X is nonsingular by showing that the vanishing of I X(t)I at some point t = t , , 0 <-c t, < t, , implies the existence of a nontrivial vector b such that X(t,) b = 0. Hence, y(t) = X ( t ) b is a solution of dy/dt = A(t)y, y(t,) = 0, and thus y ( t ) = 0, for 0 t to . T h i s is a contradiction sincey(0) = X ( 0 ) b = b # 0.
< <
3. Modify the foregoing proof to demonstrate that the integral equation x c J,1 A(t)x dt, possesses a unique solution in any interval [0, t,] where J j A(t)ll is L-integrable, that is, A I/ dt, < co. Suppose that A ( t ) is piecewise continuous. What interpretation do we give the differential equation at a point of discontinuity of A(t)? ~
+
J2II
4. Show that X = exp[J: A(t,) dt,] is the solution of X’ = A(t) X , X ( 0 ) = I if and only if A ( t ) $, A(t,) dt, = A(t,) dt,) A(t).
(Ji
5. Consider the positive definite matrix A(s) with distinct characteristic
values, Xi(s) and associated orthonormal characteristic vectors xi(s). Show that X r 2 ’ ( s ) = (A’(s)x,,, xn), where ’ denotes d/ds, and that
(Kalaba, Schmaedke, Vereeke.) 3.10. Linear lnhomogeneous Equation Let us now consider the inhomogeneous equation dx d t = A(t)x + f ( t ) ,
x(0)
=
c.
(3.10.1)
117
3.10. LINEAR INHOMOGENEOUS EQUATION
To obtain the solution, we can employ the variation of parameters technique. Set x =xy, (3.10.2) where X ( t ) is as in the previous section. Th e n dx dt
-=
Since X '
=
dX (x) Y +x ( dY z ) 4 t ) XY + f ( Q
(3.10.3)
=
A(t)X , this reduces to dY =
dt
x-If,
(3.10.4)
whence
x =c
(3.10.5)
+ jtX ( t ) X(t1)-'f(t1>dt, . 0
Exercises
1 . Let K(t, t l ) = X ( t ) X(tl)-l. Establish the semigroup property, K(t, tl) = K(t, t 2 ) K(t2 , tl). 2. Obtain expressions for the terms in the perturbation expansion e A k B = e A + B1c + ... by considering e A + c B as the value at t = 1 of X ( t ) , where X' = ( A EB)X , and converting the differential equa-
+
tion into an appropriate integral equation.
3. Show that if aij(t) >, 0, ci >, 0, then the solutions of x' x(0) = c are nonnegative for t 0.
=
A(t)x,
4. Show that the same holds under the conditions aii(t) >, 0, i # j , ci 2 0. Hint: Consider the integral equation
i = 1 , 2,..., N . 5. Establish the result using the difference approximation Y,(t
+4
=
(1
-
aii(t) 4
i = 1 , 2,..., N , t = O , L l ,....
YiW
+ c .Zi(t)Yj(t) A , 3#i
iia
MATRICES A N D LINEAR DIFFERENTIAL EQUATIONS
= --a..x. z + C. 3 f 2 . a..x. 23 3 xi(0) = ci , with ci 3 0, xi ci = 1 , >, 0, aij = Ci,j aij , then xi >, 0, Xixi(t) = 1, t 3 0. Equations
6. If xi’ aij
22
I
of this nature are of importance in the modern theory of pharmacokinetics. 3.11. Adjoint Equation
I n dealing with more general linear functional equations, the matrix inverse is replaced by the solution of an adjoint equation. To see how this goes, let Z be a matrix function, as yet unspecified, and consider the integral (3.1 1.1)
Integrating by parts, we have on the left
upon taking x(0) solution of
=
0, as is permissible. Let us then choose Z ( t l ) as the dZ dtl
_ _ --
for 0
-ZA(t,),
Z(t) = I,
(3.1 1.3)
< t , < t. With this choice of 2 we have, from (3.11 . I ) and (3.1 1.2), (3.11.4)
T h e solution of (3.1 1.3) is easily seen to be 2 (3.11.3) is called the adjoint equation. 3.12. The Equation X’
=
X ( t ) X(tl)-l. I n general,
= AX + XB
An equation that arises in a number of interesting situations is dt
=
AX
+XB,
X(0)=
c.
(3.12.1)
Let us first consider the case where A and B are constant. It is easy to see that X = eAtceBt (3.12.2) is a solution. Uniqueness shows that it is the solution.
3.12. THE EQUATION X’ = AX
+ XB
119
T o solve
_ dx at
-
AX
+ X B +F(t),
(3.12.3)
X ( 0 ) = C,
we use the left integrating factor e-At and the right integrating factor e-Bt, obtaining X = eAtCeBt
+
s:
eA(t-ti)F(t,) eB(t-ti) dt, .
(3.12.4)
From the representation in (3.12.2) we obtain an interesting result. Integrate (3.12.1) between 0 and 03 and suppose that A and B are stability matrices so that X -+ 0 as t -+ co. Then
which means that Y=
is a solution of
C
-so
m
=
eAtiCeBti
AY
(3.12.6)
dt,
+ YB.
(3.12.7)
I n Sec. 3.19 below, we will discuss the question of when (3.12.7) has a unique solution. I n the meantime, let us note that the existence of (3.12.6) for all C if A and B are stability matrices means that (3.12.7) has a unique solution in this case. Exercises
1. Obtain the result corresponding to (3.12.4) when A and B are variable. 2. Show that the solution of X’ definitive when C > 0.
=
AX
+ X A T , X ( 0 ) = C is positive
3. Let H be a function of a parameter t , H Then
4. Show that
=
H ( t ) , and
s a
parameter.
120
MATRICES A N D LINEAR DIFFERENTIAL EQUATIONS
5. If [ A ,H(t)]
aH/at, then [ A , f ( H ) ]= a f ( H ) / a t .
:
6. Show that e A B e r A = B
7. Write ez
etAetB,
:
z
+ [A,B] + [ A ,[A,B]]/2! + ..*. xm Z,,tn, 2‘ = ntn-l 2, . Show that m
:n=l
i:erzZ‘(t)ecTZdr
=
A
+ etABectA
and thus obtain recurrence relations for the 2,.I n particular, show -4- B , 2.2= [ A , q / 2 , 2 3 = [ [ A ,[A,BII “A, B ] , B]]/12. (Baker-Campbell-Hausdorff formula.)
+
z,= A
For the foregoing five results and many further results and additional references, see
R. M. Wilcox, “Exponential Operators and Parameter Differentiation in Quantum Physics,” J. Math. Phys., Vol. 8, 1967, pp. 962-982. I. Rialynicki-Rirula, R . Mielnik, and J. Plebanski, “Explicit Solutions of the Continuous Baker-Campbell-Hausdorff Problem ...,” Ann. P h ~ s . Vol. , 51, 1969, 187-200. 3.13. Periodic Matrices: the Floquet Representation
I n general, it is very difficult to obtain representations for the solution of X ’ A(t)X , X ( 0 ) : I . If A ( t ) is periodic, i.e., A(t 1) = A(t), a useful representation exists. We can show that
+
1
X
+
=
P ( t ) eat,
(3.13.1)
where Iz is constant and P(t I ) = P(t). T o see this, observe that X ( t ) and X ( t 1) are simultaneously solutions of the differential equation if A ( t ) is periodic. Hence X(t
+
+- 1)
X(t)C
(3.13.2)
for some constant matrix C. Setting t = 0, C = X(1), a nonsingular matrix. Hence, we can write C : ea for some matrix B. Set X ( t ) = Y ( t )e B t ,
then (3.13.2) yields Y(t
+ 1) eE(t+l)
=
Y(t)eB‘eE,
(3.13.4)
Y(t),
(3.13.5)
whence Y ( t t 1)
the desired result.
=
(3.13.3)
3.14. CALCULUS OF VARIATIONS
121
T h e representation in (3.13.1) is an elegant and important one but is not easy to use for quantitative purposes. There is no readily available way of determining B. Exercises
1. Consider the equation
+
+
where Pl(t 1) = Pl(t), P2(t A) = P2(t)with h irrational. Denote 1, 8) = X ( t , 8 1 ) X ( l , 8). the solution by X ( t , 8). Show that X ( t
+
+
2. Hence, show that
x(t + n, e) = x(t,e + ). 3. Write X ( 1 , 8)
=
X(I, e
+
+
It - 2)
... X(I, e).
e B ( @Then ).
JJ X ( 1 , e + n
n
n
k=O
- 1) X(I, e
-
exp[B(e
k) = k=O
+ n 41. -
Does an expression of the above form possess any simple asymptotic behavior as n -+ 03 ? See
R. Bellman, “A Note on Linear Differential Equations with Quasiperiodic Coefficients,’’ J. Math. Anal. Appl., forthcoming. 3.14. Calculus of Variations
As in the scalar case, variational problems lead to two-point conditions. Consider the problem of minimizing the quadratic functional
over the class of functions subject to the initial condition x(0) = c, and the existence of the integral. Let 3 be a function which renders J ( x ) stationary and consider (3.14.2)
MATRICES AND LINEAR DIFFERENTIAL EQUATIONS
122
where
t
is a scalar. T h e condition of stationarity yields the condition (3.14.3)
for all y . Integrating by parts, this becomes
(z’, y)l,T
+J
”T
[-(x”, Y )
+ (A(t)
9, y ) ] dl
0.
(3.14.4)
Hence, we are led t o conjecture the Euler equation
x”
-
A(t)x
=
0,
(3.14.5)
subject to the two-point boundary condition
X(0)
xyT) = 0
= c,
(3.14.6)
as a necessary condition to be satisfied by the minimizing function. We will discuss the rigorous aspects of this procedure in Chapter 7. 3.15. Two-point Boundary Condition
We solve (3.14.5) subject to (3.14.6) in exactly the same fashion that we treated the scalar case. Let X , and X , be the principal solutions of the matrix equation
x
-
A ( t ) X = 0,
(3.15.1)
that is, X,(O) X,(O)
=
1,
=
0,
Set x := X,b‘l’
X,’(O) X,’(O)
= =
+ X,b@’,
0, 1.
(3.15.2)
(3.15.3)
where .V1) and b(2)are vectors to be determined. T h e n (3.14.6) leads t o (3.1 5.4)
Thus, there is a unique solution for ZP) if and only if X 2 ‘ ( T )is nonsingular. I t is not easy to give general conditions on A which ensure the existence and uniqueness of a solution. O n the whole, we must be guided by
3.17. THE MATRIX RlCCATl EQUATION
123
biological, chemical, economic, engineering, and physical processes which suggest various analytic conditions. T h e theories of dynamic programming and invariant imbedding play a significant role in this study, both analytically and computationally, as we shall discuss in the second volume. 3.16. Green’s Functions
If X,’( T ) is nonsingular, the equation x”
-
A(t)x = f,
x(0)
=
0,
x’(T) = 0,
(3.16.1)
has a unique solution expressible in the form
T h e matrix K is called the Green’s function associated with the equation and the specific boundary conditions. We proceed exactly as in the scalar case to determine its analytic form. Exercise
1 . Find the Green’s functions associated with the conditions (a) x(0) = 0, x’(T) = 0, (b) ~ ( 0= ) x ( T ) = 0, (c) x(0) = 0, x ( T ) Xx’(T) = 0, h a scalar.
+
3.17. The M a t r i x Riccati Equation
An important role in both mathematical control theory and mathematical physics is occupied by the matrix Riccati equation, as we shall see in the chapters on dynamic programming and invariant imbedding in Volume 11. Purely formally, we obtain an equation of this nature in the following fashion. Consider the linear system of matrix equations X‘ = A X Y’ = CX
and the new variable
+ BY, + DY,
X(0) = XI, Y(0)= I ,
z= XY-1.
(3.17.1)
(3.17.2)
MATRICES AND LINEAR DIFFERENTIAL EQUATIONS
124
T h e inverse Y-l exists in some interval [0, t o ] .Then
+
=
( A X 4-B Y ) Y - ‘ - XY-’(CX DY) Y-l AAyk--l+ Z j - Xy-lCXE’-l - x1---1n
=
AZ
-
+B
-
ZCZ
-
ZD,
(3.17.3)
X , . Conversely, we can transform the equation for Z into lvith Z(0) a system of two linear equations of the form of (3.17.1) by means of ~~
(3.17.2).
Exercises
1. Show that 2-1satisfies an equation of the same type.
+
+
B,)(C,Z Dl)-’ satisfies an equation of the same 2. Show that (A,Z form, with and without calculation. 3.18. Kronecker Products and Sums
1,et us now introduce an important matrix product which arises in surprising Mays-the Kronecker product. Let A and B be two matrices of dimension M and N respectively. Define the MN-dimensional matrix A
0R
=
u,,B
=
i , j = 1, 2 ,...,M ,
(uz,B),
(:
uM,B
.‘.
...
“‘““i
. ~MMB
(3.18.1)
’This is called the Krorkecker product of B by A. One way in which the matrix arises is the following. Let x and y respectively undergo the two linear transformations x(‘) = Ax, y(1) = BY, (3.18.2) and consider the linear transformation induced in z , the MN-dimensional vector,
2-
(3.18.3)
3.19. AX
+ XB
=
C
125
I t is easy to see that the MN characteristic roots of A @ B are Xipi where Xi are the characteristic roots of A and the p j are the characteristic roots of B. T h e Kronecker sum of A and B is defined by
A @B
=
A
0IN + IM @ B,
(3.18.4)
where IM and I, are respectively the M and N dimensional identity matrices. If x and y respectively satisfy dx dt
dY
- = Ax,
dt = B ~ ,
(3.18.5)
then z as defined by (3.18.3) satisfies dz -- [ A @ BIZ. dt
(3.18.6)
It follows, thus or otherwise, that the characteristic roots of A @ B are the MN values Ai pj .
+
Exercises
1. Show that ( A x B)(C x D ) = AB @ CD,( A @ B)-l= A-l@ B-l, and ( A @ B)T = A T @ BT. 2. Introduce the “stacking operator,” a transformation from an N x N matrix into an N2-dimensional vector
where the a ( i )are the columns of A. Prove that S(PAQ) = (QT0P ) S ( A ) .
D. Nissen, “A Note on the Variance of a Matrix,” Econornetrica, Vol. 36, 1968, pp. 603-604. 3.19. AX
+ XB = C
As a first application of the Kronecker sum, consider the matrix equation AX+XB=C. (3.19.1)
MATRICES A N D LINEAR DIFFERENTIAL EQUATIONS
126
If we regard this as a linear equation of dimension N 2 for the elements xii and introduce the vector
(3.19.2)
it is easy to show that (3.19.1) takes the form
( A @ B T ) y= d
(3.19.3)
where d is formed from C in the same way that y was formed from X . From this it follows that (3.19.1) has a solution for all C if and only if hi pj # 0 for all i and j . T h i s extends the previous result for the case where A and B were stability matrices, in which case this condition was automatically satisfied.
+
Exercises
1. A necessary and sufficient condition that a real matrix A be a stability matrix is that ATY Y A = -I possess a unique solution which is positive definite. Hint: Let x satisfy dxjdt = A x and u ( t ) = (x,Yx). Consider the behavior of u(t) as a function o f t . Alternatively, use the character of the solution Y in terms of the Kronecker sum. ~
2. Let A be an N x N complex matrix with characteristic roots Xi , A, + A, f 0. T h e n the N x N matrix G, the solution of AG 1- GAT = I , is nonsingular and has as many positive characteristic roots as there are X i with positive real parts. See
0. ‘Taussky, “A Generalization of a Theorem of Lyapunov,” J . SOC. Ind. Appl. Math., Vol. 9, 1961, pp. 640-643.
3 . Show that if all characteristic roots of A are less than one in absolute value, the solution A * X A - X = -0, where A* is the conjugate transpose of A , is given by
MISCELLANEOUS EXERCISES
127
where J denotes integration round the circle I x ] z-plane. Alternately, X
= (2n)-l
JT
=
1 in the complex
(G-l)* QG-l do,
--n
where G
=
A
-
Ieie. See
R. A. Smith, “Matrix Calculations for Lyapunov Quadratic Forms,” J. Dz#. Eqns., Vol. 2, 1966, pp. 208-217. R. A. Smith, “Bounds for Quadratic Lyapunov Functions,” J. Math. Anal. Appl., Vol. 12, 1966, pp. 425-435.
4. Show formally that a solution of X - U X V = W is given by X = UkWVk.When does the series converge and represent the solution. See
xr=o
+
R. A. Smith, (‘Matrix Equation X A BX Math., Vol. 16, 1968, pp. 198-201.
=
C,” SIAM J. Appl.
+
= (C, B ) to be the unique solution of XB B X = C in the case hi hi # 0, Xi the ith characteristic root of B. Show that ( C , ( C , B ) ) = B.
5. Define X
+
3.20. Random Difference Systems
Kronecker products also arise from the consideration of linear difference equations of the form xn+l =
Rnxn,
xo =
(3.20.1)
C,
where the R , are independent random matrices. It is easy to calculate the expected values of the x, , namely, E(xn+l) = E(Rn) E(xn),
E(x0)
=z
C-
(3.20.2)
To calculate the second and higher moments, we require Kronecker products. Detailed results will be found in the references at the end of the chapter. Miscellaneous Exercises
1. Consider the matrix transformation f ( A , T ) = T-lAT
-
dT
-. dt
128
MATRICES A N D LINEAR DIFFERENTIAL EQUATIONS
Show that
f(4S T ) = M A , Y’),
S).
Hint: Start with the differential equation dx/dt = A(t)x, x(0) and consider the equation for y where x = Ty.
=
c,
2. Consider the linear functional equation
f
- -
e
+W f g ) ,
where e is the identity element, h is a scalar, and T is an operator satisfying a relation of the form ( T u ) ( T v )= ”(uT(v))
+ T(u)v
-
Buv,
for any two functions ZL and v where 6 is a fixed scalar. Show that for small A,
(Baxter.) See
F. V. Atkinson, “Some Aspects of Baxter’s Functional Equation.”
J . Math. Anal. Appl., Vol. 6 , 1963, pp. 1-29. J. G. Wendel, “Brief Proof of a Theorem of Baxter,” Math. Scand., Vol. 1 I , 1962, pp. 107-108. It is interesting to point out that operators of this nature are connected with the Reynolds operator of importance in turbulence theory. See
G. C. Rota, “Reynolds Operations,” Proc. Symposia in Appl. Math., Vol. 16, Stochastic Processes in Mathematical Physics and Engineering, Amer. Math. SOC.,1964, pp. 70-83. 3. Consider the differential-integral equation AX =
Use the Laplace transform to find a representation for L(v)
1
m
=
e c s t x ( t ) dt,
0
assumed to exist for Re(s) sufficiently large.
MISCELLANEOUS EXERCISES
129
4. Let X ( t ) be the solution of the matrix equation ~
dX dt
=
AX
+
st 0
K(t
-
t,) X ( t l ) dt, ,
-
tl) K(t,) d t ,
and let Y ( t )be the solution of dY dt
-= Y A
+ jt Y ( t 0
.
What relation exists between X and Y ?
5. Let X ( t ) satisfy the foregoing equation. Show that X(t
+ s)
-
s: [I'
A(tl) K(t
X ( t ) X ( s )= -
+ s - t,
- sl)
4,) ds,]
dt,
.
See J. M. Richardson, "Quasi-differential Equations and Generalized Semigroup Relations," J . Math. Anal. Appl., Vol. 2, 1961, pp. 293-298.
6. Show that if X ( t ) is a differentiable matrix function satisfying the s) = X ( t ) X (s ) for -03 < t , s < 03, functional equation X ( t and if X ( 0 ) = I , then X ( t ) = eAt for some constant matrix A. (Hint: Differentiate with respect to both t and s and compare
+
results.
7. Show that the same result holds if X ( t ) is merely required to be continuous. (This is a classic result due to Polya.)
+
8. Let
u satisfy the equation u = f AT(u), where f is a known function and X is a parameter. Using the relation
T y u ) = T " ( f ) + XT'n+l'(u), show that
and thus obtain a formal infinite continued fraction for u/T(u) and, hence, for u.
9. Apply this to the equation
u =
fraction expansion for et. See
1
+ X sou dt, to find a continued t
MATRICES A N D LINEAR DIFFERENTIAL EQUATIONS
130
R. Bellman and J. M. Richardson, "A New Formalism in Perturbation Theory Using Continued Fractions," Proc. Nut. Acad. Sci. U.S., Vol. 48, 1962, pp. 1913-1915.
10. Let u, be the nth convergent of this infinite continued fraction. Show that
(T. Brown.) S. I. Drobnies, "On a Method of Bellman and Richardson in Perturbation Theory," J. Muth. Anal. Appl., forthcoming. 11. Consider the Fredholm integral equation
4.)
=
f ( 4+ A f b 4 x 7 Y ) 4 Y ) dY.
Let the first approximation be determined by
where a,
=
Jb
a
ul(x) dX/(b - u).
Show that
where D(A) = ( b
-
a) -x
1: f:
k(x,y ) dx dy.
As the nth approximation, take
where a,
-= J b n
Show that
(un(x) - Un&))
dx/(b - a).
BIBLIOGRAPHY AND COMMENTS
131
Discuss the convergence of this method due to Sobolov. See A. Y . Luchka, The Method of Averaging Functional Corrections: Theory and Applications, Academic Press, New York, 1963. 12. Consider the linear integral equations
+ j; k(u9 4 v,(v) dv, g(.) + J: [j: exPC+ - Y ) / k ( % w)3 df4Y ) drl dv,
(1) y(u) = g(u) and
(2) d u , 4
=
where it is assumed that k(u,V ) >, 0 for u, w 3 0. Show formally that limz+m y(u, x) = cp(u). 13. Prove that if (1) has a unique positive solution y(u), then (2) possesses a unique solution which converges monotonically to ~ ( u as ) x + co. 14. For what matrices A and B do we have eAeB= eA+B,in addition to those for which A and B commute ? For the foregoing and additional references, See
R. F. Rinehart, “The Equation eXeY = fl+y in Quaternions,” Rend. Circ. M a t . Polermo. Ser. 11-Tomo VII, 1959, pp. 1-3. Bibliography and Comments 03.1. For the more extensive coverage of the material in this chapter, see Chapters 10 through 13 of
R. Bellman, Introduction to Matrix Analysis, McGraw-Hill, New York, 1960. (Second Edition, in preparation.)
For a survey of numerical techniques, see
G . E. Forsythe, “Today’s Computational Methods of Linear Algebra,” SIAM Review, Vol. 9, 1967, pp. 489-515.
93.3. See B. W. Helton, “Integral Equations and Product Integrals,” Pac. J. Math., Vol. 16, 1966, pp. 277-322.
93.5. See W. Magnus, “Algebraic Aspects of the Theory of Systems of Linear Differential Equations,” Comm. Pure Appl. Math., Vol. 7, 1954. H. Wilcox, “Exponential Operators and Parameter Differentiation in Quantum Physics,” J. Math. Phys., V O ~8,. 1967, pp. 962-982.
132
MATRICES A N D LINEAR DIFFERENTIAL EQUATIONS
Kao-Tsai Clien, “Integration of Paths, Geometric Invariants, and a Generalized BakerIIausdorff I:ormula,” Ann. Math., Vol. 65, 1957, pp. 163-178. Scc also
W. 1,. Miranker and B. iVeiss, “The Feynman Operator Calculus,” S I A M Review, L7oI.8, 1966, pp. 224-232.
43.8. For ;I detailed discussion of what can be done using Kronecker sums and Lyapunov matrices, see A. T. Fuller, “Conditions for a Matrix to IIave Only Characteristic Roots with Negative I
S. I<;iplari and Rl. Ben bliloud, “On Eigenvalue and Identification Problems in the ‘I’heory of I h u g I>istrihution,” USCEE-3 10, University of Southern California, 1968. J. Z. Hearon, ‘“l’lieorcms on Linear Systems,” A m . N. Y . Acad. Sci., Vol. 108, 1963, pp. 36-68. 43.1 1 . For a detailed account of the use of the adjoint operator in differential-difference equations, see
R. Bellinan and K . L. Cooke, “Asymptotic Behavior of Solutions of Differential-Difference Iquations,” Mem. Amev. Math. Soc., No. 35, 1959. R. Ikllman and I<. L. Cooke, Di~ereiitial-Difference Eqzcations, Academic Press, New Yorli, 1963. 43.13. For applications of functional analysis to the establishment of a generalization of the 1:loquct theorem, see A. Stokes, “A Iloquet Theorem for Functional Differential Equations,” Proc. Nut. Acad. Sci. U.S., Vol. 48, 1962, pp. 1330-1334. see also
Y. Sibuya, “Note o n Real Matrices and Linear Differential Systems with Periodic Cneficients,” J . Math. A n d . Appl., Vol. 1, 1960, pp. 363-372. Yu. A. Rlitropol’skiy and A. M. Samoylenko, “Construction of Solutions of Linear Differential 13quations with Quasi-periodic Coefficients,” A N UkrSSR. Matemnticlirskayn fizikn, No. 3, Kievc, Naukova dumka, 1967, pp. 185-198. 1. Z. Shtokalo, “Stability Criterion of Solutions of Linear Differential Equations with Sc;rrly Periodical Coeficients,” Recueil Mathernatiqiie (Matematicesk. II Sbornik), 1946. 53-14. See Chapter 7 and
K. Bellmnn, Introdrictioii to the i\/lattiemutical Theory of Control Processes, Vol. I : Linear Eqrratinns atzd Qnadratic Criteria, Academic Press, New York, 1967.
For a discussion of an important computational question, see
M. E. Fowler and R. M. \!liarten, “Numerical Integration Technique for Ordinary
Differential Equations with Widely Separated Eigenvalues,” I B M Jozrr. of Research rrnd Deaelopmeiit, Vol. 1 I , 1967, pp. 537-543.
BlBLlOG RAPHY A N D COMMENTS
133
J. Legras, “Rksolution NumCrique des Grands Systkmes DiffCrentielles Linkaires,” Numerische Mathematik, Vol. 8 , 1966, pp. 14-28. $3.17. See the book cited above in $14 for a detailed discussion of the Riccati equation. $3.20. See
R. Bellman, “Limit Theorems for Noncommutative Operations-I,” Duke Math. I., Vol. 21, 1954, pp. 491-500. H. Furstenberg and H. Kesten, “Products of Random Matrices,” Annals Math. Stat., Vol. 31, 1960, pp. 457-469. J. H. Pollard, “On the Use of the Direct Matrix Product in Analyzing Certain Stochastic Population Models,” Biometrika, Vol. 53, 1966, pp. 397-415. Also see
M. L. Mehta, Random Matrices, Academic Press, New York, 1967.
Chapter 4
STABILITY THEORY AND RELATED QUESTIONS
4.1. Introduction
I n this chapter we wish to lay a rigorous foundation for a number of approximate methods we will employ in subsequent chapters and to discuss some topics of independent interest. Our theme will be “stability.” I n very general terms, we are interested in what happens to certain properties of the solutions of an equation when the structure of the equation is altered in various ways. Starting with the equation dx
- = Ax, dt
(4.1. I )
x(0) = c,
the properties of whose solutions are readily noted, we wish to study first the boundedness and asymptotic behavior, as t + 00, of the linear equation with “almost-constant” coeficients, dx
dt = ( A
+ B(t))x,
x(0)
=
c.
(4.1.2)
Then we turn to some corresponding questions for the nonlinear equation dx dt
-
=
Ax
+ g(x) + h(t),
x(0) = c,
(4.1.3)
and present results of PoincarC and Lyapunov. We will also study the case where the solution is determined by conditions at two points, say t = 0 and t = T. Equations of this type arise in the calculus of variations in connection with the minimization of functionals of the form
and in mathematical physics in the consideration of transport processes associated with plane-parallel media of finite thickness. 134
4.2. DINI-HUKUHARA THEOREM-]
135
In addition to results of the foregoing nature, we are interested in the behavior of solutions of nonlinear differential equations over finite intervals. I n particular, we wish to study the connections between the solutions of dx
_ dt -- g(x),
40)
=c
(4.1.5)
and that of the perturbed equation
< <
over 0 t to under various assumptions concerning the order of magnitude and analytic properties of h ( y , t ) . These results are basic to our application of the Bubnov-Galerkin technique in Chapter 5. We will also discuss stability theory for difference equations, briefly, in connection with the application of difference methods to the study of partial differential equations. At the close of the chapter we discuss the Emden-Fowler-Fermi-Thomas equation as an illustration of the use of Poincark-Lyapunov results. 4.2. Dini-Hukuhara Theorem-I
T h e first result we wish to demonstrate is: Theorem.
If all solutions of dx - = AX dt
are bounded as t
-+
00, and
(4.2.1)
if (4.2.2)
then all solutions of
dr dt = ( A 4B ( t ) ) y
(4.2.3)
are bounded as t -+ 00. T h e proof we present depends upon the conversion of (4.2.3) into a linear integral equation. We have dY - AY = W y ,
(4.2.4)
136
STABILITY THEORY AND RELATED QUESTIONS
whence, setting y(0) = c, we have (4.2.5)
or
where x is the solution of (4.2.1) subject to x(0) = c. T h e assumption of (4.2.2) guarantees that the solution of (4.2.3) satisfying y(0) = c exists and is unique for any finite t-interval. From (4.2.6), it follows that
where we set
K,
= max
[max 11 x 11, m;x 11 eAt 111. t
(4.2.8)
By virtue of our assumption concerning boundedness of x, we know that
I/ e A t / / is uniformly bounded, and thus that K, < co.
Applying the fundamental lemma of Sec. 1.3, we have
for t 2 0. This completes the proof. Using this result, we can go further. Returning to (4.2.6),we have
Thus, we have not only a stability theorem, but also a stronger result providing an estimate of the difference between x and y. We can also determine asymptotic behavior. Consider the expression in (4.2.5) under the assumption that A possesses distinct characteristic roots. T h e integral J: e A ( t - t l )B(t,) y dt, contains two types of terms, those corresponding to characteristic roots with negative real part and
4.2. DINI-HUKUHARA TH EOREM-I
137
those corresponding to characteristic roots with zero real parts. Thus, we may write
Jr
where in Z, , Re(&) < 0 and in Z2, Re(&) = 0. I t is easy to verify that the fact that I[ By 11 dt,
< co implies
that
(4.2.12)
for any term in 2, . In Z2,write
T h e infinite integral converges since Re(&) = 0 and J" Thus, we can write y
+ Z2AkeAktf
= Z2AkceAkt
m 0
e-"''By dt,
(1 By (1 dt,
+ o(1) = &eAktbk,
(4.2.14)
where the b, are constant vectors linearly dependent on c. The determination of the b, is not a completely simple matter. We shall discuss some aspects of this in the chapter on invariant imbedding in Volume 11. Exercises
1. Let x' = Ax where all the characteristic roots of A have negative B(t))y are bounded real parts. Show that all solutions of y' = (A if [I B(t)l[ k, where k, is sufficiently small and that all solutions approach zero as t -+ co if k, is small enough.
<
+
2. Consider the equation x' = Ax + f ( t ) . Show that all solutions are bounded if all solutions ofy' = Ay are bounded and if J" [ l f [ l dt < 03.
STABILITY THEORY A N D RELATED QUESTIONS
138
3. Show that if A is such that all solutions of x' t + 03, then A has the canonical form
where Re(h,)
=
Re(&)
=
.**
A , have negative real parts.
=
=
Ax are bounded as
0, and all the characteristic roots of
Another way of putting this is that all the Jordan factors associated with roots of zero real part are diagonal.
4. Using the foregoing result, obtain aversionof the result in the foregoing section in which simplicity of characteristic roots is not assumed. 4.3. Dini-Hukuhara Theorem-I I
Using very much the same technique, we can demonstrate a more general result: Theorem.
If all solutions of dx dt
-
are bounded as t
+ CO,
==
A(t)x
if
<4 <
(4.3.2)
WJlI dtl < *,
(4.3.3)
2 dt = (A(t)-I- B(t))y
(4.3.4)
II X(4 W 1 > - l II for t
(4.3.1)
3 t , 3 0, and i f JrnII
then all solutions of
are bounded as t
+ GO.
4.3. DINI-HUKUHARA THEOREM-11
139
T h e proof proceeds as before, We write
and consider the associated integral equation
where, as before, X ( t ) is the solution of the matrix equation
_ dx - A(t)X dt
Hence, as above, I( y
= 0,
11 is bounded for t
X(0) = I .
(4.3.7)
0.
Exercises
1. Show that (4.3.2) is satisfied if all solutions of (4.3.1) are bounded and if tr(A(t)) = 0 for t 3 0. 2. More generally, show that (4.3.2) is satisfied if all solutions of (4.3.1) are bounded and f t tr(A(t,)) dt, is uniformly bounded for t 3 0.
+
+
3. Consider the scalar equation U" ( a ( t ) b ( t ) )u = 0, where f" 1 b(t)l dt < 03. Show that all solutions are bounded if all solutions a(t) u = 0 are bounded. of z)"
+
+
4. Show that all solutions of y' = ( A ( t ) B ( t ) ) y are bounded if all solutions of x' = A(t)x are bounded, if A(t) is periodic in t , and if I1 B(t)ll dt <
S"
140
STABILITY THEORY AND RELATED QUESTIONS
5. Show by considering the systems dY1 dt
-
d ~ 2=
~P
dt
-ay17
[sin log t
+ cos log t
dx, = (sin log t + cos log t dt
-
2a1 y z ,
-
2a) x2 + xlePat,
that the result in the preceding section is not true under the sole condition that all solutions of x' = A ( t )x are bounded as t + 00. See
0. Perron, Math. Zeit., Vol. 29, 1929, pp.129-160. R. Bellman, Stability Theory, Dover Publications, New York, 1969, p. 87.
6. Under the conditions of Exercises 1-3, show that all solutions of y' = A ( t ) y + f ( t ) are bounded as t-+ co provided that J" llfll dt < CO. 4.4. Inverse Theorems of Perron
Suppose that a system is described by the equation dx
-
dt
=4
t ) x +f(t),
x(0)
(4.4.1)
= c,
and that we want to determine something about the structure of the system by observing the behavior of the system, i.e., the solution x, under various classes of forcing functions, f ( t ) . We may, for example, R , < CO, consider thc class of functions satisfying the conditions Ilf(t)ll or j" i i f i l dt, < co, or 1" lifli2 dt < CO, and so on. This type of problem was first investigated by 0. Perron. It is most easily treated by Banach space methods. References will be found at the end of the chapter.
<
4.5. Existence and Uniqueness of Solution
We will establish the existence of a solution of the equations we consider subsequently by means of successive approximations. I n addition to this, we want to guarantee uniqueness of solution. I n many important cases, the question can be readily settled.
4.5. EXISTENCE A N D UNIQUENESS OF SOLUTION
141
Let us suppose that the equation (4.5.1)
has two solutions, x and y in some interval [0, to]. Then
Assume now that g(x) satisfies a Lipschitz condition, that is,
II g(.)
-
dY)l/ G
h I1 x - Y IL
(4.5.3)
where k, depends on the allowed range of 11 x I[ and (1 y 11. T h e n we have, from (4.5.2),
/I x
-
Y
I1 < J t I/ g(4 - g(r)ll ds
< 4 J t II x
-
Y I1 d5
(4.5.4)
where k1 is determined by max (1 x (1, max (1 y 11, as t ranges over [0, to]. As we know, this implies that x = y, the desired uniqueness. There are four important classes of problems associated with differential equations. T h e first is that of establishing existence and uniqueness of solution within some t-interval and some restricted class of functions. T h e second is that of determining the precise interval of existence of this solution. I n our case we are interested in solutions that can be continued for all t > 0. Thirdly, we are concerned with analytic expressions and approximations for the solution. Finally, we wish to obtain various types of computational algorithms. I n general, different methods are used for different types of problems. I n this chapter we concentrate on the first three classes of problems. I n subsequent chapters we will present some computational results. T h e proof of uniqueness is not included solely as a mathematical frill. Simple examples show that without further conditions on g(x), uniqueness will not hold. Consider, for example, the equation u' =
T h e function u
=
dii,
u(0)
= 0.
(4.5.5)
0 is readily seen to be a solution, but so also is u = t2/4.
(4.5.6)
We are not interested here in the weakest conditions which ensure uniqueness but rather in simple conditions which guarantee this property and which in addition permit us to exhibit a constructive demonstration of the existence of a solution.
STABILITY THEORY A N D RELATED QUESTIONS
142
Exercises
1. Show that it is sufficient to consider equations of the form in (4.5.1)
by reducing dyldt = g ( y , t ) to the foregoing form by the change of variable x1 = y1 , x2 = y z ,..., xN = y N ,xN+l = t.
2. Show thFt (4.5.3)may be replaced by /Ig ( x ) - g(y)j( where J: k,(t) dt < co.
< kl(t)/lx - y 11,
4.6. Poincar&Lyapunov Stability Theory
Let us now examine the nonlinear equation dx
-
dt
=
Ax
+,).(g
x(0) = c,
(4.6.1)
where the components of g ( x ) are convergent power series in the components of x , lacking constant and linear terms for, )I x 11 small. We are interested in determining simple criteria which will allow us to use the linear approximation (4.6.2)
as a guide to the qualitative and quantitative behavior of x as t + GO. The fundamental result in this area is a theorem due to PoincarC and Lyapunov: Theorem.
If
(a) all solutions of dyldt = A y approach zero as t 4 GO, (b) the components of g ( x ) are power series in the components of x lacking constant and linear terms, convergent for 11 x 11 suficiently small, (c) )I c I/ is suficiently small,
(4.6.3)
then (a) the solution of ( 1 ) exists for t 3 0, (b) the solution approaches zero as t + CO,
and for some constant k,
.
II x
-
Y
II
< kl II c 112
(4.6.4)
(4.6.5)
4.7. PROOF OF THEOREM
143
Observe that we are simultaneously establishing an existence and uniqueness theorem, a stability theorem, and a result concerning asymptotic behavior.
4.7. Proof of Theorem
T h e proof relies upon the fundamental technique in the theory of differential equations, conversion of the differential equation into an integral equation, a technique we have already employed. Proceeding as in Sec. 4.2, we convert (4.6.1) into the nonlinear integral equation x = e*tc
+
It
(4.7.1)
eA(t-S)g(x(s))ds.
0
We now employ the method of successive approximations since it will permit us not only to establish the stated result but also to obtain important information concerning the asymptotic behavior of x as t -+ co. Write x, = eAtc,
x,+~ = eAtc
+
Jt
eA(t-S)g(xn)ds,
n
0
2 0.
(4.7.2)
T h e proof now proceeds in easy stages. We begin by showing that
I/ %a I1 < 2b, I1 c 11,
n
= 0,l,...,
(4.7.3)
provided that (1 c [I is sufficiently small. Here b, is a constant we shall specify in a moment. T h e hypothesis made above concerning the behavior of g(x) enables us to assert that
provided that (1 x, ( ( is sufficiently small. This condition allows us to establish (4.7.3) inductively. Let us now fill in the details. We have (4.7.5) II xo 1I = II eAtc I1 < I1 eAt I1 I1 c II < I1 c I1 for some rl < 0 and some scalar 6,. This is a consequence of our hypothesis concerning the solutions of dy/dt = Ay. Subsequently, we shall use the full force of (4.7.5). At the moment, we use only the estimate 11 xo 11 b, I1 c 11.
<
144
STABILITY THEORY AND RELATED QUESTIONS
Let us now proceed inductively. Suppose that (4.7.3) is valid for n. Then, using (4.7.2),
I/ %+l I1 ,< II eAtc I1 + j t I1 ea(t-s)g(xn)ll ds
< 6, using (4.7.4). Since (1 x, 112 we have
11 %+I 11
I/ c II
+ 6, j t
erl(t-s)
< 4bI2 11 c 112,
11 g(xn)ll ds
by the inductive hypothesis,
< b1 I/ c /I + 4h3b211 c < bl /I c I/ + 4b13b2I1 c
j'erl(t--s)ds 112 j t erls ds
< b, 11 c 11 + 4b13b211 c 112 e 2bl I1 c
JmerLS ds 0
(4.7.7)
I17
provided that 11 c 11 is sufficiently small. This establishes the induction. T h e next step is to show that the sequence {x,} converges. We have x,+~
x,
~
=
1,:
e A ( t - s ) [g(x,)
(4.7.8)
- g(xnP1)] ds.
T h e assumption concerning g(x), namely, the power series nature of components of g(x), allows us to assert that
for some scalar b, , again under the assumption that
I/ xn+1
-
x,
I1 < b3 /I c I1 j t /I eA(t-s)ll /I x,
- xn-1
11 c 11
< 1 . Hence
/I ds (4.7.10)
From this, we see inductively that
3 0. Thus, the series any finite t-interval.
for n
2, 11 xntl - x, Ij converges uniformly in
4.7. PROOF OF THEOREM
145
A stronger result can be obtained by a slight modification of the foregoing argument. Let
a quantity which we know is finite. Then, from (4.7. lo), (4.7.13)
Hence, if b,lJ c 1) = Y
< I,
we see that u,
C, u, converges. Thus, the series
< b,P,
Y
< 1,
and thus that (4.7.14)
converges uniformly for t 3 0. This, in turn, implies the uniform convergence of Z%(X,+~ -x,) and therefore of x, for t 3 0. Let x denote the limit function, x limn+mx, . Then, uniform convergence yields (4.7.15)
Thus, we see that x satisfies the original differential equation. Since y = eA1c,we have
I1 x - Y It
e J t /I eA(t-s)I1 I1g(4ll < 4b12b2 )I c ]I2 It I] eA(t-s)/I ds 0
st
0
< 4bi2bz 11 \I2 C
11 eAs11 d5
< b, //
C
l12.
(4.7.16)
<
Hence, if 11 c (1 is small, the approximation of x by y over 0 t < 03 is an excellent one. T h e proof that x approaches zero as t -+ co will be deferred until the next section, where we establish a much stronger result. Exercises
1. Is uniform convergence needed to establish (4.7.15) ? 2. Establish the uniform boundedness of any solution of x =y
+
st
0
eA(t-s)g(x)ds,
STABILITY THEORY AND RELATED QUESTIONS
146
where no Lipschitz condition is invoked, by the following argument: t < co, for // c 11 1. We wish to show that /I x // < 2b,J/ c 11, 0 Assume the contrary and let to be the first point at which 11 x 11 = 2b,(l c 11. Then
<
<
2b1 /I c II
=
II x(to)ll
< IIr(to)ll+
1s
t0
eA'to--")g(x)ds
< bl II c II + b, II c 112 whence, a contradiction for 11 c 11
sto
I
II eA(to-s)It ds,
< 1 . (Levinson.)
4.8. Asymptotic Behavior
Having established the uniform boundedness of 11 x(t)ll for t 3 0, we now wish to investigate the asymptotic behavior of x as t -f 00. Let us consider only the simple case where the characteristic roots of A , A,, A, ,..., A,, are distinct. By assumption, 0 > Re(&) 2 Re(A,) 2 Re(&). If complex roots occur, they occur in conjugate pairs, since we have taken A real. Let us simplify the situation still further by supposing that A, is real, since we are primarily interested in pointing out the kind of result that can be obtained rather than deriving the most general result. Since the roots are distinct by assumption, we have eAt =
C eAktAk, N
(4.8.1)
k-1
where the A, are constant matrices. Using this representation in the basic nonlinear integral equation of (4.7.1), we may write x = e"ltA1c
+ 1eAktA N
ds
k=2
(4.8.2)
As we suspect, the principal contribution on the right-hand side is eA1'A1c
+
st
eA1(t-S)Alg(x) ds
0
m
m
+ eA1'J' e-AISAlg(x)ds - eAltf t
= eAltA1c
0
e-AISAlg(x)ds.
(4.8.3)
4.8. ASYMPTOTIC BEHAVIOR
147
T h e infinite integral exists since A, < 0 and [I A,g(x)l[ is bounded by . turns out that the first two terms dominate a term of the form 0 ( e z A i 1 ) It as t + 00, with the third term of lower order of magnitude. Let us now establish this statement. T o begin with, we show that for t >, 0 we have
where b, is a constant. We can do this inductively using the recurrence relation of (4.7.2). T h e result is valid for x1 = y = eAtc, and the induction can be established using the arguments employed in Sec. 4.7. Alternatively, we can start from
(4.8.5)
for some scalar b, independent of c. Hence (4.8.6)
Applying the fundamental inequality, we obtain
<
which yields [ ( x [I bleAltecl where to (4.8.5), we see that
E
is small since (1 c (1 i 1. Returning
11 x 11 < bleAlt+ b,eAlt
so
Since e-Alll(lx dt, converges as t -+ 00, for 11 c (1 sufficiently small we obtain the desired estimate for (1 x I(. Let us now return to (4.8.3). T h e estimate [Ig(x)ll b,1( x [I2 for t >, 0, plus that of (4.8.4), allows u s to conclude that the integral e+lSg(x) ds converges. f
Jr
<
STABILITY THEORY AND RELATED QUESTIONS
140
Furthermore,
/I eA1'
Im e-A1sA4,g(x)ds 11 <
00
eAlt
t
e-"Is
t
< bl0eAlt
m
< blleAlt
m
t
11 A , 11 11 g(x)ll ds
e-"IS /I x
1'
ds
e-A1se2A1sds
t
< bl2eZA1'.
(4.8.9)
Thus, we see that as t + 00 the third term in (4.8.3) is negligible compared to the first two. Similarly, we see that eAL"(-")A,g(x)ds
1 < b13eRe(A,)tIt
e--Re(Ak)s
/I g(x)ll ds
0
eRe(A\L)t
l4
so
e-Re(Ak)s 211s
1
* b14eRe(Ak)t
,*
00
e
--Re(Ak)s
ds
e2A1s ds
=
0
bl,e Re(Ak)t.
(4.8.10)
Collecting the foregoing results, we can assert that the following holds: (4.8.1 1) where we introduce the function ~ ( c to ) indicate that the limit depends upon the initial value c. Let us recall that this was established under the proviso that A, is a simple root. T h e case of multiple roots is not more complex in principle, but the details can become onerous due to the presence of terms such as teAIt, and so on. Exercise
1. Establish the limiting behavior of x in the case that A, is a multiple root. Consider first the case where it is a root of multiplicity two. 4.9. The Function cp(c)
T h e function ~ ( c )is of interest to us in our subsequent work on approximation and closure. Our next assertion is that p)(c) is an analytic
4.10. MORE REFINED ASYMPTOTIC BEHAVIOR
149
function of c for I/ c I/ sufficiently small. T o establish this, we observe that the assumptions concerningg(x) guarantee that each x, is an analytic co and for all t 0. Since x, converges uniformly function of c for ( 1 c (1 to x for t >, 0, it follows that x is an analytic function of c for t >, 0 and I( c I( co . An examination of the estimates we have employed shows that the limit in (4.8.7) is uniform for (1 c (( co . It follows that ~ ( c is ) an analytic function as stated. We have thus established the following theorem:
>
<
<
<
<
Theorem. Let g(x) be an analytic function of x for 11 x /I xo , lacking constant and jirst order terms; let the characteristic roots of A all have negative real parts, and let the one with largest real part, A, , be simple; supposejnally that (1 c (1 is suficiently small. Then the solution of dx
--Ax
dt
+,).(g
40) = c,
(4.9.1)
satisfies the asymptotic relation (4.9.2)
where p(c) is an analytic function of c for (1 c (1 small. At the moment, we do not possess an efficient way of determining p(c). Subsequently, in the chapter on invariant imbedding in Volume I1 we shall present an effective procedure. This determination of 4(c) will provide some information concerning the required degree of smallness of 11 c 1) and, more generally, the domain of stability. Nonetheless, it must be admitted that we do not possess adequate methods for the determination of this domain at the moment. Exercises
1. Consider the scalar equation u’ = -u by using the explicit analytic solution.
2. Use the integral equation
+
+ u2, u(0) = c. Calculate p(c) 6
e-k Joe-(l-tl)u2(t,)dt, to calculate ) means of iteration. the first three terms of the power series for ~ ( c by u =
4.10. More Refined Asymptotic Behavior
With some further effort and an additional assumption, we can establish the following result:
150
STABILITY THEORY AND RELATED QUESTIONS
Theorem. A d d to the foregoing assumptions the condition that the characteristic roots of A , A,, A, ,..., A, are all real and simple with 0 > A, > A, > * * . > A,. Then f o r each k, there is a n ( N - k 1)dimensional set of initial vectors c such that
+
lim xe-Akt t+m
(4.10.1)
= cpk(c)
<
holds. The functions q k ( c ) are analytic f o r 11 c 11 co . T o establish this result, we start with the basic integral equation
and make some suitable adjustments. To eliminate the terms eAlt, eAZt,..., eAk-ll, we wish to set A,C
+ Srne-Al%,g(x)
ds =
0
for r = 1, 2, ..., k integral equation k-1
x=-
s,
1 eAit
a=1
-
o
(4.10.3)
1. T o accomplish this, we start anew with the
00
eCA12(x)ds
N
+ 1 eAit [A,. + i= k
st
eKAiSAig(x)ds].
(4.10.4)
0
T o establish that this equation possesses a solution with the desired properties, the method of successive approximations can be applied as before, starting with the initial approximation x,
N
=
eAitAic.
(4.10.5)
i=k
We shall omit the details, which involve some tedious estimations, since we have no particular application of the results subsequently. Exercise
1. Carry through the proof of the theorem above. 4.11. Analysis of Method of Successive Approximations
Let us pause to analyze our procedure and point out the three basic steps in the application of the method of successive approximations. First, the equation is written in the form x
=
T(x),
(4.11.1)
4.11. ANALYSIS OF METHOD OF SUCCESSIVE APPROXIMATIONS
151
where T is a transformation derived from the original equation. Frequently, some experimentation is required in order to obtain an appropriate form which permits a convergence proof. As we shall discuss below, there are often alternate formulations which possess different advantages as far as accuracy, stability, ease of computation, and so forth, are concerned. In many cases, the choice of an initial approximation is crucial. Generally speaking, T(x) must possess the property of leaving a convenient class of functions invariant. Assuming that we have obtained a suitable transformation, we introduce a sequence of functions by means of the recurrence relation Xn,,
=
T(xn),
(4.1 1.2)
= 0, l)...,with xOprescribed. I n other words, we consider the solution of (4.11.1) to be the steady-state limiting form of the dynamic process defined by (4.11.2), that is, x = limn+mx, . T o establish the convergence of the sequence (x,}, we usually proceed in the following steps. First, we demonstrate that the sequence is bounded in a suitable norm. Following this, we prove the convergence of the sequence by considering the infinite series
n
2 m
(Xn+1 - xn).
(4.11.3)
n=O
I n Volume 11, in our discussion of quasilinearization, we provide an entirely different procedure based on monotone convergence. To obtain an estimate of the terms in this series, we return to the equation in (4.1 1.2) and write (4. I 1.4) %+1 - x n = W,) - w%-l).
If T is linear, this simplifies to xn+l
- xn = T(xn - %-I).
(4.11.5)
I n general, however, we must impose some condition on the transformation T which enables us to estimate 11 T(x,) - T ( X , - ~ )in\ ~terms of 1) x , - x,-~ 11. I n the previous discussions a Lipschitz condition suffices, which is to say a bound on the partial derivatives of the components of
g w
Having established convergence in this fashion, we then turn to the problem of showing that the limit function x = limn+mx , is actually a solution of the original equation. This requires a continuity property of T , namely, - . that T ( x ) = lim T(x,). (4.11.6) n-tm
152
STABILITY THEORY AND RELATED QUESTIONS
This is easy to establish as a consequence of the bound of 11 T ( x )- T(x,)lI in terms of I/ x - x, 11. This does not conclude the proof, however. I n many cases the equation of (4.11.1) is obtained from the original equation by means of some preliminary transformations and it may not be easy to show that these are reversible transformations. T h i s is particularly the case in dealing with partial differential equations.
4.12. Fixed-point Methods
One of the advantages of using the representation
x
=
T(x)
is that it is apparent that the desired solution is a fixed-point of the transformation T . I t is tempting then, stimulated by the finite-dimensional fixed-point theorem of Brouwer, to see if one can establish the existence of a solution using only very general properties of T , such as continuity with respect to a suitable norm. This method, introduced by Birkhoff and Kellogg and greatly extended by Leray and Schauder and others, is a very elegant and powerful one. Existence theorems based upon fixed-point methods play a vital role in the formulation of mathematical models in new domains of science. These existence theorems, derived on the basis of simple characteristics of T , show that a problem is well-posed. I n return, these methods are nonconstructive. They provide no algorithm for the determination of x. As we shall see in subsequent chapters, however, once existence, and hopefully uniqueness, of solution has been demonstrated, a number of paths open for the calculation of the solution. This is particularly so now that digital computers are available.
4.1 3. Time-dependent Equations over Finite Intervals
T h e preceding methods will now be used to obtain an estimate of the norm of the difference between the solutions of the two equations dx
(4 Z (6)
= g(4,
Z =g(Y) + dY
x(0)
w, Y ( 0 )
= c,
(4.13.1) = c,
4.13. TIME-DEPENDENT EQUATIONS OVER FINITE INTERVALS
153
over a finite interval [0, TI. Our basic assumption is that (4.13.la) has a solution over [0, TI. Furthermore, we suppose that g(x) is analytic in x in a neighborhood of x = c, which guarantees uniqueness. As we shall see, the full force of this assumption is not necessary. The analysis can be carried out under various assumptions concerning 11 h 11, where any of a number of different norms may be employed. Results of this type will be used in subsequent chapters, specifically in Chapter 5 . T o bring this problem into a form similar to that already discussed, let us writey = x w where we assume that x exists over [0, TI. Then
+
dw
= g(x dt
+ w ) - g(x) + h(t),
w(0) = 0.
(4.13.2)
Under the assumption that g(x) is analytic in x, we can write, using the multidimensional Taylor expansion,
where J ( x ) is the Jacobian matrix. By assumption, provided that 11 w (1 is sufficiently small,
I1 4%w>ll < bl I1 w 112?*
(4.13.4)
where the choice of b, depends upon the values assumed by x in [0, TI. Thus, w satisfies the equation dw dt
- --
A+
+ 4%4 + 4 t h
40)
= 0.
(4.13.5)
I n order to treat the problem of the estimation of 1) w 11 in terms of 11 h 11, we use the properties of linear systems with variable coefficients as in Sec. 4.3. Let X ( t ) denote the solution of the matrix equation dX _ - J(x)X, dt
X(0) = I .
(4.13.6)
Then the solution of (4.13.5) satisfies the integral equation (4.13.7)
* We now introduce another set of constants b, , 6, ,....
154
STABILITY THEORY AND RELATED QUESTIONS
Let us now employ successive approximations, both to establish the existence of a solution of (4.13.7) and to estimate its norm. Set (4.13.8)
We now show inductively that (4.13.9)
where we have introduced a new norm (4.13.10)
We have
(4.13.11)
Since X(t)-l satisfies the adjoint equation dY dt
Y ( 0 ) = 1,
(4.13.12)
we see that j J X ( t ) - l / /is uniformly bounded for 0 assumption that x exists in [0, TI. Hence
< t < T, under the
- = - YJ( x) ,
II wo IIM G bz II h llM -
(4.13.13)
Let us now proceed with the induction. We have
I1 wn+1 IlM
< (' I1 X ( t )X(t1)-' I1 II WG wn)ll 4 + I1 wo /I
+
G b, I1wn llM2 bz I1h IIM G 4bzZb3