LARGE SCALE EIGENVALUE PROBLEMS
NORTH-HOLLAND MATHEMATICS STUDIES
NORTH-HOLLAND -AMSTERDAM
127
NEW YORK OXFORD *TOKYO
LARGE SCALE EIGENVALUE PROBLEMS Proceedings of the IBM Europe Institute Workshop on Large Scale Eigenvalue Problems held in Oberlech, Austria,, July 8-72, 7985
Edited by:
Jane CULLUM and
Ralph A. WILLOUGHBY Mathematical Sciences Department IBM ThomasJ, WatsonResearch Center Yorktown Heights, New York, U.S.A.
1986
NORTH-HOLLAND -AMSTERDAM
NEW YORK
OXFORD *TOKYO
Elsevier Science Publishers B.V., 1986 Allrights reserved. No part of this publication may be reproduced, stored in a retrievals ystem, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the priorpermission of the copyright owner.
ISBN: 0 444 70074 9
Publishers:
ELSEVIER SCIENCE PUBLISHERS B.V. P.O. Box 1991 1000 BZ Amsterdam The Netherlands
Sole distributors for the U.S.A. and Canada:
ELSEVIER SCIENCE PUBLISHING COMPANY, INC. 52Vanderbilt Avenue NewYork, N.Y. 10017 U.S.A.
Library of Congress Cataloging-in-Publication Data
IBM Europe Institute on Large Scale Eigenvalue Problems (1985 : Oberlech, Austria) Large scale eigenvalue problems. (North-Holland mathematics studies ; 127) Includes.bibliographfes and index. 1. Eigenvalues--Congresses. 2. Eigenvaues--Data processing--Congresses. I. Cullum, Jane K., 193811. Willoughby, Ralph A. 111. Title. IV. Series. ~ 9 3 . 1 2 6 1985 512.9’434 86-13544 ISBN 0-444-70074-9
PRINTED I N THE NETHERLANDS
V
PREFACE
The papers which are contained in this book were presented at the IBM Europe Institute Workshop on Large Scale Eigenvalue Problems which was held at Oberlech, Austria, July 8-12,
1985. This Workshop was one in a series of summer workshops sponsored by the IBM World Trade Corporation for European scientists.
The unifying theme for this Workshop was ‘Large Scale Eigenvalue Problems’. The papers contained in this volume are representative of the broad spectrum of current research on such problems. The papers fall in four principal categories:
(1) Novel algorithms for solving large eigenvalue problems ( 2 ) Use of novel computer architectures , vector and parallel (3) Computationally-relevant theoretical analyses (4) Science and engineering problems where large scale eigenelement computations have pro-
vided new insight.
Most of the papers in this volume are readily accessible to the reader who has some knowledge of mathematics. A few of the papers require more mathematical knowledge. In each case, additional papers on these subjects are available from the authors of these papers. The interested reader can obtain such reprints by writing to the appropriate authors. A complete list of the names and addresses of the authors is included at the end of this book. A corresponding list
of the Workshop speakers who were not able to submit papers is also included. Interested readers should consult both Lists.
Jane Cullum Ralph A. Willoughby Program Organizers April 1986
This Page Intentionally Left Blank
vii
TABLE OF CONTENTS
Introduction to the Proceedings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jane Cullum and Ralph A. WiUoughby
......
1
High Performance Computers and Algorithms from Linear Algebra. . . . . . . . . . . . . . . . . Jack J. Dongarra and Daniel C. Sorensen
15
......
31
..........
51
......
67
Eigenvalue Problems and Algorithms in Structural Engineering. . . . . . . . . . . . . . . . . . . . Roger G. Grimes, John G. Lewis and Horst D. Simon
81
.....
95
The Impact of Parallel Architectures on the Solution of Eigenvalue Problems. . . Ilse C.F. Ipsen and Youcef Saad
Computing the Singular Value Decomposition on a Ring of Array Processors. Christian Bischof and Charles Van Loan
Quantum Dynamics with the Recursive Residue Generation Method: Improved Algorithm for Chain Propagators. . . . . . . . . . . . . . . . . . . . . . . . . . . . Robert E. Wyatt and David S. Scott
A Generalised Eigenvalue Problem and the Lanczos Algorithm. . . . . . . . . . . Thomas Ericsson
Numerical Path Following and Eigenvalue Criteria for Branch Switching. . . . . . . . . . . . Yong Feng Zhou and Axel Ruhe
121
The Lanczos Algorithm in Molecular Dynamics: Calculation of Spectral Densities. . . . . . Giorgio Moro and Jack H. Freed
143
...
Vlll
Table of Contents
Investigation of Nuclear Dynamics in Molecules by Means of the Lanczos Algorithm. . Erwin Haller and Horst Koppel
163
...........
181
Examples of Eigenvalue/Vector Use in Electric Power System Problems. James E. Van Ness
A Practical Procedure for Computing Eigenvalues of Large Sparse Nonsymmetric Matrices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jane Cullum and Ralph A. Willoughby
Computing the Complex Eigenvalue Spectrum for Resistive Magnetohydrodynamics. . Wolfgang Kerner
193
24 1
............
267
Stably Computing the Kronecker Structure and Reducing Subspaces of ............. Singular Pencils A - h B for Uncertain Data. . . . . . . . . . . . . . . . . . . . . . James Demmel and Bo K8gstrom
283
Addresses of Authors and Other Workshop Speakers. . . . . . . . .
............
325
Index to Proceedings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 329
I11 Conditioned Eigenproblems. . . . . . . . . . . . . . . . . . . . . . . . . Francoise Chatelin
Large Scale Eigenvdue Problems J . Cullurn and R.A. WiUoughby (Editors) 0 Elsevier Science Publishers B.V. (North-Holland), 1986
1
INTRODUCTION TO PROCEEDINGS Jane Cullum Ralph A. Willoughby IBM T. J. Watson Research Center Yorktown Heights, New York 10598 USA
We provide a brief summary of each paper contained in this volume, provide some indications of the relationships between these papers, and provide a few additional references for the interested reader.
The papers included in this volume can be classified into the following four categories. (1) Novel algorithms for solving large eigenvalue problems
See the papers by Zhou and Ruhe, by Kerner, and by Cullurn and Willoughby.
( 2 ) Use of novel architectures for solving eigenvalue problems
See the papers by Dongarra and Sorensen, by Ipsen and Saad, and by Bischof and Van Loan. The paper by Dongarra and Sorensen addresses both the question of restructuring the EISPACK library routines [ 19771 of restructuring the LINPACK library routines [1979] for novel architectures.
(3)
Computationally-relevan t theoretical analyses
See the papers by Demmel and Kagstrom, by Chatelin, and by Ericsson.
(4) Examples from science and engineering where large scale eigenvalue and eigenvector computations have provided new insight into fundamental properties and characteristics of physical systems, both those existing in nature and those which have been constructed artificially. See the papers by Grimes, Lewis, and Simon, by Van Ness, by Kerner, by Moro and Freed, and by Haller and Koppel.
2
J. Cullum and R.A. Willoughby
Most of the currently active areas of research in modal analysis are represented in this volume. With the exception of the three papers dealing with novel architectures which are presented first, the papers are presented in an ordering which takes us from the 'easiest' problems, the real symmetric eigenvalue problems, to the most difficult ones, the computation of the Kronecker canonical forms of general matrix pencils.
Many engineering and scientific applications yield very large matrices. Historically, the sizes of the matrices which must be used have grown as the computing power has grown. Therefore, there is much interest in understanding how to exploit the new vector and parallel architectures in such computations. The first paper by Dongarra and Sorensen addresses two basic questions dealing with such architectures. First, they look at the types of computers which are currently available and at those which should be available within the next few years. They then examine the question, how d o we exploit such architectures for linear algebra computations? They include introductory descriptions of classifications for the various arithmetic engines and storage hierarchies. This discussion is followed by a table of advanced computers, proposed and existing, together with tables of characteristics of these machines.
Dongarra and Sorensen include some discussion of data communication and how that relates to algorithm performance. The cost of algorithm execution can be dominated by the amount
of memory traffic rather than by the number of floating point operations involved. A performance classification is given for algorithms on a vector computer; scalar, vector, and super-vector. Data management and synchronization add to the complications in designing algorithms for parallel computers.
The authors also address such issues as program complexity, robustness, ease of use, and portability, each of which plays an important role in the analysis. Certain basic vector and matrix-vector operations are fundamental to many linear algebra algorithms and these are examined very carefully. The authors contend that it is possible to achieve a reasonable fraction
of the peak performance on a wide variety of different architectures through the use of program modules that handle certain basic procedures. These matrix-vector modules form an excellent basis for constructing linear algebra programs for vector and parallel processors. With this philosophy, the machine dependent code is isolated to a few modules. The basic routines provided
Introduction
3
in the linear equation solving package, LINPACK [1979], and in the EISPACK library [ 19771 for eigenvalue and eigenvector computations, achieve vector but not super-vector performance.
The paper by Ipsen and Saad presents a brief survey of recent research on multiprocessor (parallel) architectures and on algorithms for numerical linear algebra coniputations which take advantage of such architectures.
The emphasis is on algorithms for solving symmetric
eigenvalue problems. Basic terminology is introduced and data communication problems such as start-up times and synchronization are discussed. Three loosely coupled architectures; a processor ring, a two-dimensional processor grid and the hypercube are considered.
The paper by Bischof and Van Loan looks at one of the parallel architectures which is available today, the LCAP configuration designed by Enrico Clementi of IBM and at the problem of implementing a block Jacobi, singular value decomposition algorithm on LCAP. LCAP consists of ten Floating Point Systems FPS-l64/Max array processors connected in a ring structure via large bulk memories. The algorithm is a block generalization of a parallel Jacobi scheme which appeared in the paper by Brent, Luk, and Van Loan [1985]. The parallel procedure developed in the Bischof and Van Loan paper could also be applied to the real symmetric eigenvalue problem. The authors however did not achieve the speedups which others had predicted. This paper is a good illustration of the difficulties and considerations encountered in translating an idea for a parallel algorithm into a practical procedure.
Many of the papers in this volume use procedures which rest upon the so-called Lanczos recursion. For more details and background information on this recursion, the reader is referred to Parlett [ 19801 and Cullum and Willougby [ 19851. Both of these books contain bibliographies with references to much of the recent research on Lanczos procedures. A brief survey of recent research in this area is contained in Cullum and Willoughby [1985b].
The paper by Wyatt and Scott provides an example of the use of a real symmetric Lanczos procedure. The objective is to compute time dependent quanta1 transition probabilities. These transition probabilities are obtainable from differences of survival amplitudes for surviving in a particular state at time t given that we started in that state at time t=O. These survival amplitudes can be computed if all of the eigenvectors of an associated Hamiltonian operator are known. However, an eigenvector decomposition of this operator cannot be obtained easily.
J. Cullurn and R . A . Willoughby
4
The Lanczos algorithm provides real symmetric tridiagonal representations of this Hamiltonian which reduce (at least theoretically) the survival amplitude computations to computations of the eigenvalues of tridiagonal matrices and computations of the first components of the eigenvectors of these tridiagonal matrices. Both of these computations are reasonable. Specifically, Wyatt and Scott compute
where M is the size of the Lanczos matrix being used; s l a denotes the first component of the eigenvector of that tridiagonal matrix corresponding to the eigenvalue E,.
Wyatt and Scott use the Lanczos recursion with no reorthgonalization because they have very large matrices and therefore the amount of computer storage which would be required by the Lanczos methods which require reorthogonalization would, be too large. However, if the Lanczos vectors are not reorthogonalized, then extra or 'spurious' eigenvalues can appear among the eigenvalues of the Lanczos matrices. These are not genuine representations of the eigenvalues of the original matrix, and these eigenvalues must be handled appropriately if the results are to have any validity. Such eigenvalues can however be identified very easily, see Cullum and Willoughby [1985], and in earlier papers Wyatt and coauthors were using this identification test. However, the main point of this current paper is that for their particular application it is not necessary to sort these Lanczos eigenvalues. All of the computed quantities can be used in their computations and they still get correct results. They demonstrate this numerically. A partial explanation for this follows directly from the characterization of these spurious eigenvalues given in Cullum and Willoughby [ 19851. The spurious eigenvalues are eigenvalues of a particular submatrix of the Lanczos matrix being considered and because of this the first components of their eigenvectors are pathologically small. Therefore, their contributions to the sum in Eqn.( 1.1) are pathologically small.
Algorithms exist for computing eigenvalues of large, real symmetric generalized eigenvalue problems Ax = XBx where A and B are real symmetric and B is positive definite. However, there are many open questions regarding this problem when neither A nor B is not definite. Grimes, Lewis and Simon focus on this problem. They first provide a survey of the types of eigenvalue/eigenvector problems encountered in structural engineering problems. They then
I n trudicction
5
discuss extensions of procedures designed for the standard real symmetric eigenvalue problem to such problems. When B is not definite, these 'symmetric' problems are genuinely nonsymmetric and many numerical difficulties can be experienced in trying to solve them. Unfortunately, in many structural problems, B is only positive semidefinite.
Grimes, Lewis and Simon outline the two most common classes of structural engineering problems, vibration and buckling analyses. In vibration analyses, the higher frequency modes of vibration are not important because it is unlikely that they will be excited. In buckling analyses usually only the smallest positive eigenvalue and corresponding eigenvector are required. Because of the slow convergence of such eigenvalues in the numerical algorithms designed thus far, it is typical to factor one or more of the matrices involved in the eigenvalue computations and to use such factorizations to transform the desired eigenvalues into eigenvalues with dominant magnitudes. When matrices A
-
oB are factored, the method being used is called a shift
and invert method.
Grimes, Lewis, and Simon also list some of the practical considerations which must be faced by any numerical analyst designing algorithms which are to be used within the constraints of structural engineering packages. For example, typically there are restrictions on the way the required data is stored and can be accessed. For this reason block versions of modal algorithms have a number of desirable features for structural engineering calculations. A detailed discussion of a block Lanczos procedure for the problem Ax = ABx, where A and B are real symmetric and B is positive semidefinite, will appear shortly. See reference 7 in the Grimes, Lewis and Simon paper. The authors point out that in some situations it is necessary to use a model which involves large nonsymmetric matrices and/or solve nonlinear eigenvalue problems. However, satisfactory eigenvalue procedures for these nonsymmetric structural problems have not yet been devised. This is an open area for research.
The paper by Ericsson derives some of the computationally-important theoretical properties of generalized eigenvalue problems Kx = XMx, where K and M are real symmetric matrices. In the first part of his paper he develops the analysis which he needs for examining three types of procedures for computing eigenvalues:
(1) Inverse Iteration; ( 2 ) Power Methods; and (3)
Lanczos Methods. In most of his paper he assumes that K is a nonsingular matrix and that the pencil of matrices (K - AM) is nonsingular. Equivalently, this means that K and M do not have
J. Cullurn and R . A . Willoughby
6
a common null vector. He proves however, that any theorem which is valid under those assumptions must also be valid when K is singular so that there is no loss of generality in his arguments.
Under these conditions these generalized problems can behave very nonsymmetrically and Ericsson illustrates that type of behavior with examples. He then focuses on the problem for
M a positive semidefinite matrix. This type of problem is encountered frequently in structural engineering problems. He uses the analysis which he has developed to look at the ability of the three procedures listed above to compute good approximations to the eigenvectors of the given generalized problem. This analysis points out a basic difficulty with both Lanczos methods and with power methods, namely keeping the eigenvector approximations in the proper part of the space. Since M is singular, the generalized eigenvalue problem has infinite eigenvalues. If the starting vector in the Lanczos procedure contains a nonzero projection on the subspace spanned by the eigenvectors corresponding to these infinite eigenvalues, then this projection may grow and produce significant errors in the resulting Ritz vectors computed. This is a serious problem which must be dealt with numerically. Ericsson also shows that this problem can happen when
M is nonsingular but very ill-conditioned. He also addresses the question of obtaining error estimates for computed Ritz vectors in the case that M is singular.
As mentioned in the Grimes, Lewis and Simon paper, a more accurate representation of a
particular structural problem may require the solution of a nonlinear eigenvalue problem: Find u and h such that G(u, A) = 0 where G is a nonlinear vector function, u is a vector of the same dimension, and h is a scalar.
Zhou and Ruhe examine such problems, not only in the context of solving nonlinear eigenvalue problems, but as a general path following problem. Typically, the manifold of solutions consists of a curve or path as illustrated in several figures in this paper. Bifurcation points in this curve, places where several paths meet, and turning points, where the curve has the hyperplane h = c (c constant) are of interest. These are points where the Jacobian of G with respect to u is singular. In the linear problem there is a bifurcation point at each eigenvalue. They propose a modification to the Euler-Newton path following algorithm which uses the solution of a linear eigenproblem to give both a prediction of the position of singular points and the direction of bifurcating branches. Several examples illustrating this idea are included.
Introduction
I
The next level of difficulty in dealing with algorithms for solving eigenvalue problems is to design procedures which are applicable to complex symmetric problems. For diagonalizable, complex symmetric matrices one can write down a Lanczos recursion which is completely analogous to the real symmetric recursion. The left and the right eigenvectors of a complex symmetric matrix are identical so only one set of Lanczos vectors has to be generated. As is shown in Wilkinson [1965], in the general nonsymmetric case, it is necessary to replace the single Lanczos recursion which is used in the real symmetric case by a set of two such recursions. One of these recursions uses the given matrix and the other recursion uses the transpose of the given matrix. The papers by Haller and Koppel and by Moro and Freed describe applications where eigenvalue and eigenvector computations are used to obtain basic physical properties of molecular systems and the matrices involved are complex symmetric. Haller and Koppel consider both real symmetric and complex symmetric matrices.
Moro and Freed are studying molecular motion.
The information obtained from
spectroscopic or scattering techniques yields only macroscopic responses to external perturbing influences. The objective in these studies is however, to understand the basic mechanisms controlling the molecular motion. Moro and Freed describe the connections between experimental measurements, spectral density computations and the identification of these basic mechanisms.
In practice, different theoretical models for the underlying mechanisms are assumed and then comparisons of the resulting macroscopic quantities are made with experimental measurements. These comparisons require spectral densities.
Under certain assumptions, the computations of the spectral densities can be reduced to the computation of the effect of the resolvent of a certain operator on certain vectors. The form
of the operator and of the particular vectors depends upon the system being studied. The authors develop these relationships. They then show how the Lanczos algorithm can be used to obtain a 'tridiagonal' representation of the operator, and how this representation can be used to reduce the required spectral density computations to computations of continued fractions whose coefficients are simply the entries of the tridiagonal matrices generated by the Lanczos procedure.
The main part of the Moro and Freed paper considers problems where the operator can be symmetrized so that it is either a real symmetric or a complex symmetric operator. In both of
8
J. Cullum and R . A . Willoughby
these cases the Lanczos recursions reduce to a single recursion and the Lanczos tridiagonal matrices are symmetric, either real symmetric or complex symmetric. In the last section of their paper the authors extend some of their ideas to more general nonsymmetric operators.
Haller and Koppel are also looking at problems in molecular dynamics. They have attacked the very difficult and interesting problem of modeling the vibronic coupling in polyatomic molecules. They have obtained a model which reproduces gross features of complex experimental spectra. From this they can make several inferences. The basic computation which is required is the determination of the spectral distribution. Because of the sizes of the matrices involved, it is not possible to use standard eigenelement algorithms for these computations. The Lanczos algorithm with no reorthgonalization plays a critical role in their computations. In this paper the authors consider matrices up to size 40800. However, they want to consider matrices of size up to lo6. They use the computed results to support their theoretical models.
Electric power systems problems yield some of the most difficult nonsymmetric eigenvalue/eigenvector problems. Van Ness summarizes and illustrates these types of problems. Modern power systems consist of many generating stations and load centers connected together by an electrical transmission system. Small disturbances in such systems can be studied by using eigenanalysis on linearizations of the system equations around some nominal operating state. The objective of this analysis is to determine whether or not the linearized system has any eigenvalues with positive real parts, and to determine the sensitivities of such eigenvalues and of the eigenvalues with small negative real parts, to perturbations of parameters in the model. Sensitivity analysis requires the computation of eigenvectors. A history of the study of several oscillation problems in power systems is presented.
Nonsymmetric problems are considered in the paper by Cullum and Willoughby. The objective is to devise a Lanczos procedure for computing eigenvalues of large, sparse, diagonalizable, nonsymmetric matrices. The authors propose a two-sided Lanczos procedure with no reorthogonalization which uses both the given matrix A and its transpose. Two sets of Lanczos vectors are generated and the Lanczos matrices are chosen such that they are complex symmetric and tridiagonal matrices. In exact arithmetic the Lanczos vectors generated are biorthogonal. A generalization of the QL algorithm is used to compute the eigenvalues of these matrices. A generalization of the identification test for spurious eigenvalues which was used in
Introduction
9
the case of real symmetric problems is found to apply equally well here. Several properties of complex symmetric tridiagonal matrices are derived. Numerical results on matrices of size n = 104 to size n = 2961 demonstrate the strengths of this procedure and the type of convergence which can be expected. For the problems up to size n = 656, the Lanczos results are compared directly with the corresponding results obtained using the relevant EISPACK subroutines. All arithmetic is complex even if the starting matrix is real. Because there is no reorthogonalization, this procedure can be used on very large problems and can be used to compute more than just a few of the extreme eigenvalues of largest magnitude.
Kerner addresses the question of the stability of plasmas which are confined magnetically. Such plasmas play a key role in the research on controlled nuclear fusion. This is an application where large scale eigenvalue and eigenvector computations provide new insight into basic physical behavior. The most dangerous instabilities in a plasma are macroscopic in nature and can be described by the basic resistive magnetohydrodynamic model.
A well-chosen
discretization of this model transforms this model into generalized eigenvalue problems: Ax = XBx where A-is a general matrix and B is a real symmetric and positive definite matrix. The eigenvalues and eigenvectors of these systems provide knowledge about the behavior of the plasma. Of particular interest are the Alfven modes, and the author studies the effects of the resistivity upon these modes and upon the sound modes. These eigenvalues are found to lie on curves in the plane, and the eigenvalue and eigenvector computations are performed by using a path following technique which uses inverse iteration and a continuation method. Convergence is demonstrated by performing these computations over finer mesh sizes.
Central to the successful computation of eigenelements are both the theoretical stability of the given problem with respect to perturbations in the data and the numerical stability of the algorithm being used to perform the computations. Chatelin addresses both questions. She focuses on defective eigenvalues. Only multiple eigenvalues can be defective. An eigenvalue is defective if its multiplicity as a root of the characteristic polynomial of the given matrix is larger than the dimension of the subspace of eigenvectors associated with that eigenvalue.
Chatelin first looks at the question of condition numbers of eigenvalues and of invariant subspaces. She then shows that the method of simultaneous inverse iteration is not stable if the
10
J. Cullurn and R . A . Willoughby
subspace being computed corresponds to a defective eigenvalue. She proposes a particular modification of a block Newton method which is stable.
Demmel and Kagstrom present algorithms and error bounds for computing the Kronecker canonical form (KCF) of matrix pencils, A
-
XB, where A and B can be rectangular matrices.
For the standard eigenvalue problem, Ax = Ax, the Jordan canonical form (JCF) provides insight into the behavior of the system under perturbations in the matrix A. The K C F is a generalization of the J C F which can be used to provide similar insight into the generalized eigenvalue problem Ax = ABx and into general systems involving pencils of matrices A - hB where A and B may be rectangular. The KCF is obtained by applying left and right nonsingular transformations which simultaneously reduce the A and B matrices to block diagonal matrices. Each of these diagonal blocks may have one of three forms. These forms together with the matrix transformations characterize the subspace associated with each block.
Computing features of the KCF can be an ill-posed problem; that is, small changes in the data may result in large changes in the answers. By restricting the class of perturbations allowed, Demmel and Kagstrom look at the question of how much the Kronecker structure can change when perturbations are made in the matrices.
These perturbations can occur because of
roundoff errors or because of uncertainties in the input data. They then analyze the errors incurred in algorithms for computing Kronecker structures. They are particularly interested in singular pencils, for example when the determinant of (A
-
XB) = 0. The error bounds ob-
tained can be used for determining the accuracy of computed Kronecker features. Their results have applications to control and linear systems and these relationships are discussed. Wilkinson [ 19791provides additional introductory comments.
Several other talks were given at the Austrian Workshop which, for a variety of reasons, are not included in this volume. In particular, some of them have already been published elsewhere. G. W. Stewart opened the workshop with a survey of the basic theory and algorithms used in eigenelement analysis and computation. Much of this material can be found, for example, in his book Stewart [1973] and in the book by Golub and Van Loan [1983]. Later in the Workshop program, Stewart presented an algorithm for doing simultaneous iterations on the ring processor, ZMOB, which resides at the University of Maryland.
Introduction
11
Kosal Pandey described the Hermitian eigenelement problems which arise in his studies of surface properties of various materials. His results are directly applicable to semiconductor materials. Physical and chemical properties of a surface are determined by its surface structure. Two basic questions addressed are: ( 1 ) Determine the atomic structure at the surfaces of such materials; and ( 2 ) Determine basic physical characteristics such as how the surface will react with various chemicals. Pandey obtains this type of information by computing eigenfunctions of Schrodinger’s equation. Using these computations, he has shown that the accepted buckling reconstruction mechanism for the configuration of atoms at surfaces is valid only for heteropolar surfaces. He has proposed an alternative m - bonding model for homopolar surfaces which fits well with both theoretical arguments and with experimental data. For more details on this work the reader is referred to Pandey [1983, 1983b].
In some applications it is necessary to compute the eigenelements of matrices obtained by simple modifications of a given matrix, for example, a rank one modification. In other applications one or more of the eigenvalues of a system are specified a priori and the user is asked to determine a matrix with those eigenvalues. Gene Golub surveyed some of the work on such problems. Much of his talk is contained in the references Golub [1973] and Boley and Golub [ 19781.
A key question in any eigenelement computation is how do we know that the answers ob-
tained are meaningful? Beresford Parlett presented the material contained in the paper Kahan, Parlett, and Jiang [ 1982.1This paper includes error estimates for nonsymmetric problems which are applicable, for example, to Lanczos algorithms for nonsymmetric problems. Parlett gave two talks. The second talk surveyed the recent research on the question of maintaining semiorthogonality of the Lanczos vectors generated by a Lanczos recursion. The interested reader can find much of this material in the book, Parlett [1980].
A complete list of the authors and coauthors with their full addresses is included at the be-
ginning of this book. A corresponding list of the speakers who do not have papers in this volume is contained at the end of this book. The interested reader can obtain additional references by contacting the authors and speakers directly.
J. Cullurn and R.A. Willoughby
12
REFERENCES D. Boley and G. H. Golub (1978), Inverse eigenvalue problems for band matrices, P a . 7th Biennial Conf. Univ. Dudee: k t u m Notes in Math. 630 , Springer, Berlin, 23-3 1. R. Brent, F. Luk, and C. Van Loan (1983, Computing the singular value decompostion using mesh connected processors, J. VLSI and Computer Systems, 1, 242-270. J. Cullurn and R. A. Willoughby (1985), Lanezos Algorithm for Large Spmetric Eigenwlue Computations, Vol. 1, Z7tmyand Vol. 2, Progrums. Progress in Scientific Computing Series, Vol. 3 and Vol. 4, Eds. S. Abarbanel, R. Glowinski, G. Golub, P. Henrici, and H. 0.Kreiss, Birkhauser-Boston. J. Cullurn and R. A. Willoughby (1985b), A survey of Lanczos procedures for very large real 'symmetric' eigenvalue problems, J. Comput. Appl. Math., 12,13, 37-60. J. J. Dongarra, J. R. Bunch, C. B. Moler, and G. W. Stewart (1979), LZNPACK Users Guide, SIAM Publ., Philadelphia.
B. S. Garbow, J. M. Boyle, J. J. Dongarra, and C. B. Moler (1977), Matrix Eigensptem Routins - EISPACK Guide memion, Lecture Notes in Computer Science, Vol. 51, Springer-Verlag, Berlin. G. H. Golub (1973), Some modified matrix eigenvalue problems, SIAM Review, 15, 3 18-334. G. H. Golub and C. F. Van Loan (1983). Matrix Compututions, The Johns Hopkins University Press, Baltimore. W. Kahan, B. N. Parlett, and E. Jiang (1982), Residual bounds on approximate eigensystems of nonnormal matrices, SIAM J. Numer. Anal., 19, 470-484.
[ 101K. C. Pandey (1983), Reconstruction of semiconductor surfaces: Si( 111)-2 x 1, Si( 111)7 x 7, and GaAs(1 lo), J. Vacuum Science and Technology, Vol. A1(2), 1099-1 100. [ 111 K. C. Pandey (1983b), Theory of semiconductor surface reconstruction: Si( 11 1)-7 x 7, Si(ll1)-2
x
1, andGaAs(llO), PhysicaB, 117-118, 761-766.
1121B. N. Parlett (1980). The Spmetric Eigenwlue Problem, Prentice-Hall, Englewood Cliffs, NJ.
[13] G. W. Stewart (1973), Zniroducfionto Matrix Computations , Academic Press, New York. [14] J. H. Wilkinson (1965), The Algebmic Eigenwlue Problem , Oxford University Press, New York.
Introduction
13
[ 151 J. H. Witkinson (1979), Kronecker’s canonical form and the QZ algorithm, Linear Algebra Appl., 28, 285-303.
This Page Intentionally Left Blank
Large Scale Eigenvalue Problems J. Cullurn and R.A. Willoughby (Editors) Elsevier Science Publishers B.V. (North-Holland), 1986
15
High Performance Computers and Algorithms From Linear Algebra JJ. Dongarra and D.C. Sorensen
Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue Argonne, Illinois 60439
1. Introduction Within the last ten years many who work on the development of numerical algorithms have come to realize the need to get directly involved in the software development process. Issues such as robustness, ease of use, and portability are now standard in any discussion of numerical algorithm design and implementation. New and exotic architectures are evolving which depend on the technology of concurrent processing, shared memory, pipelining, and vector components to increase performance capabilities. Within this new computing environment the portability issue, in particular, can be very challenging. One feels compelled to structure algorithms that are tuned to particular hardware features in order to exploit these new capabilities, yet, the sheer number of different machines appearing makes this approach intractable. It is very tempting to assume that an unavoidable byproduct of portability will be an unacceptable degradation in performance on any specific machine architecture. Nevertheless, we contend that it is possible to achieve a reasonable fraction of the performance of a wide variety of different architectures through the use of certain programming constructs. Complete portability is an impossible goal at this point in time, but it is possible to achieve a level of transportability through the isolation of machine dependent code within certain modules. Such an approach is essential in our view, to even begin to address the portability problem.
2. Advanced Computer Architectures In the past few years there has been an unprecedented explosion in the number of computers in the marketplace. This explosion has been fueled partly by the availability of powerful and cheap building blocks and by the availability of venture capital. We will examine a number of these machines that offer high performance through the use of vector and parallel processing. A much-referenced and useful taxonomy of computer architectures was given by Flynn [8]. He divided machines into four categories:
(i) SISD - single instruction stream, single data stream (ii) SIMD - single instruction stream, multiple data stream (iii) MISD - multiple instruction stream, single data stream (iv) MlMD - multiple instruction stream, multiple data stream
J. J. Dongarra and D. C. Sorensen
16
Although these categories give a helpful coarse division, we find immediately on examining current machines that the situation is more complicated, with some architectures exhibiting aspects of more than one category. Many of today’s machines are really a hybrid design. For example, the CRAY X - M P has up to four processors (MIMD), but each processor uses pipelining (SIMD) for vectorization. Moreover, where there are multiple processors, the memory can be local, global, or a combination of these. There may or may not be caches and virtual memory systems, and the interconnections can be by crossbar switches, multiple bus-connected systems, time-shared bus systems, etc. With this caveat on the difficulty of classifying machines, we list below the machines considered in this report. We group those with similar architectural features.
scalar pipelined (e.g., 7600, 3090) parallel pipelined microcoded FPS 164 FPS 264 Multijow STAR ST-100
vector memory to menory CDC CYBER 205 register to register American Super. Convex C-1 CRAY-I CRAY X-MP-I Amdahl500,l I00,1200,1400 (Fujitsu VP-50,I 00,200,400} Galaxy YH-I Hitachi S-810 NEC S X - 1 , SX-2 Scientifrc Computer Systems cache based r-to-r Alliant FXII
parallel global memory bus connect Alliant FXf8 (vector capability) Elxsi 6400 Encore Multimax Flex132 IP-1 Sequent Balance 8000 direct connect Amer. Super. (vector capability) CRAY-2 (vector capability) CRAY-3 (vector capability) CRAY X-MP-214 (vector cap.) Denelcor HEP-I local memory hypercube Ametek Sysrem I 4 Intel iPSC NCUBE Connection Machine butterfly BBN Butterjly ring-bus CDC CYBERPLUS lattice Goodyear MPP ICL DAP dataflow b r a 1 DATAFLO multilevel memory ETA-I0 (vector capabiliry) Myrias go00
High Performance Computers and Algorithms f r o m Linear Algebra
17
A more empirical subdivision can be made on the basis of cost. We split the machines into two classes: those costing over $1 million and those under $1 million. The former group is usually classed as supercomputers, the latter as high-performance engines. With this subdivision, we can summarize the machines in the following tables. Table 1 Machines Costing over $lM (base system) Machine
Amdahl 1400
Word length
32/64
0s
Maximum Rate
Memory
in MFLOPS
in Mbytes
1142
256
OWU
32
OW
Number of Roc.
1
(Fujitsu VP400) CRAY-1
64
160
CRAY X-MP
64
2lO/pmc
CRAY-2
64
5OO/proc
2000
UNIX
4
CRAY-3
64
lOOO/proc
16000
UNIX
16
CYBER 205
32/64
400
CYBERPLUS
32164
1OO/proc
Denelcor HEP-I
32/64
lO/PEM
ETA-10
32/64
125O/proc
Hitachi S-810/20 Mynas 4000 NEC SX-2
32/64 32/64/128 32/64
128
32
840 ???
1300
4C3) 16/PEM 2048(c)
OWUNIX
1 1,2,4
OW
1
Om
256
UNIX
16@)
OWU
2,4,6,8
256
OW
512/Krate
UNIX
1024/Krate
256W
Om
1
1
(a) Memory per processor (b) 64 processes possible for each PEM; however, effective parallelism per PEM is 8-10. (c) Also 32 Mwords of local memory with each processor (d) Also a 2-Gbyte extended memory The actual price of the systems in Table 1 is very dependent on the configuration, with most manufacturers offering systems in the $5 million to $20 million range. All use ECL logic with LSI (except the CRAY-I in SSI, CRAY X - M P , and HEP in MSI), and all use pipelining and/or multiple functional units to achieve vectorization/parallelizationwithin each processor. For the multiple-processor systems, the form of synchronization vanes: event handling on the CRAYs, asynchronous variables on the HEP, sendreceive on the CYBERPLUS. The CRAY-3 and ETA-10 are not yet available. Both Amdahl and Hitachi systems are TBM System 370 compatible.
J.J. Dongarra and D. C. Sorensen
18
In Table 2 we summarize machines in the lower price category. Table 2 Machines costing under $1M
Machine
Chip
Parallelism
Alliant FX/8
WTL 1064/1065
I+vector
Connection
cross bar (reg to cache) and bus (cache to memory)
plus 10 gate arrays Ametek System 14
80286/80287
256
hypercube
Amer. Super. Comp.
ECL
(vector)
Axiom
LSI
Vector 1
(scalar)
BBN ButterAy
68020/68881
256
butterfly
Connection
VLSI
64000
hypercube
Convex C-l
Gate array
Vector
(vector)
ELxsi 6400
ECL
12
bus
Encore Multimax
32032/32081
20
bus
FlexJ32
32032/3208 1
20
bus
FPS 364
LSI
1
(scalar)
FPS 264
ECL
1
(scalar)
FPS 164+MAX
VLSl
16
bus
bus
Think Machines/
FPS 5000
VLSI
4
FPS MP32
VLSI
3
bus
ICL DAP
ECL
1024
near-neighbor
Intel iPSC
80286/80287
128
hypercube
IP-1
????
8
cross-bar
b r a 1 DATAFM
32016/32081
256
bus
Goodyear MPP
VLSI
16384
near-neighbor
Muitillow
gate array
8
(scalar)
NCUBE
Custom VLSI
1024
hypercube
scs-40
ECULSI
Vecmr
(vector)
Sequent Balance 8000
32032/32081
12
bus
Star ST-100
VLSI
1
(scalar)
Because of the widely differing architectures of the machines in Table 2 it is not really advisable to give one or even two values for the memory. In some instances there is an identifiable global memory; in others there is a fixed amount of memory per processor. Additionally, it may be possible to configure memory either as local or global. A value for the
High Performance Computers und Algorithms f r o m Linear Algebra
19
maximum speed is even less meaningful than in Table 1, since a high Megaflop rate is not necessarily the objective of the machines in Table 2 and the actual speed will be very dependent upon the algorithm and application. In the other aspects quoted in Table 1, all the machines in Table 2 are very similar. All machines, except the FPSs and the SCS (all 64 bit), the DAP, MYP, and Connection (all bit-slice, the first two supporting variable-precision floating point), and the Star (32 bit), have both 32- and 64-bit arithmetic hardware, with most of them adhering closely to the E E E standard. Also, all machines have a version of UNIX as their operating system, except FPS (host system), American Supercomputer and SCS (COS), and Star and Ametek (own system). The machines listed above are representative implementations of advanced computer architectures. Such architectures involve various aspects of vector, parallel, and parallelvector capabilities. These notions and their implications on the design of software are discussed briefly in this section. We begin with the most basic of these, vector computers. The current generation of vector computers exploits several advanced concepts to enhance their performance over conventional computers: Fast cycle time, Vector instructions to reduce the number of instructions interpreted, Pipelining to utilize a functional unit fully and to deliver one result per cycle, Chaining to overlap functional unit execution, and Overlapping to execute more than one independent vector instruction concurrently. Current vector computers typically provide for "simultaneous" execution of a number of elementwise operations through pipelining. Pipelining generally takes the approach of splitting the function to be performed into smaller pieces or stages and allocating separate hardware to each of these stages. With this mechanism several instances of the same operation may be executed simultaneously, with each instance being in a different stage of the operation. The goal of pipelined functional units is clearly performance. After some initial startup time, which depends on the number of stages (called the length of the pipeline, or pipe length), the functional unit can turn out one result per clock period as long as a new pair of operands is supplied to the first stage every clock period. Thus, the rate is independent of the length of the pipeline and depends only on the rate at which operands are fed into the pipeline. Therefore, if two vectors of length k are to be added, and if the floating point adder requires 3 clock periods to complete, it would take 3 + k clock periods to add the two vectors
20
J. J. Dongarra and D. C. Sorensen
together, as opposed to 3 * k clock periods in a conventional computer Another feature that is used to achieve high rates of execution is chaining. Chaining is a technique whereby the output register of one vector instruction is the same as one of the input registers for the next vector instruction. If the instructions use separate functional units, the hardware will start the second vector operation during the clock period when the first result from the first operation is just leaving its functional unit. A copy of the result is forwarded directly to the second functional unit and the first execution of the second vector is started. The net result is that the execution of both vector operations takes only the second functional unit startup time longer than the first vector operation. The effect is that of having a new instruction which performs the combined operations of the two functional units that have been chained together. On the CRAY in addition to the arithmetic operations, vector loads from memory to vector registers can be chained with other arithmetic operations. It is also possible to overlap operations if the two operations are independent. If an addition and an independent multiplication operation are to be processed, the execution of the second independent operation would begin one cycle after the first operation has started. The key to utilizing a high performance computer effectively is to avoid unnecessary memory references. In most computers, data flows from memory into and out of registers; and from registers into and out of functional units, which perform the given instructions on the data. Performance of algorithms can be dominated by the amount of memory traffic, rather than the number of floating point operations involved. The movement of data between memory and registers can be as costly as arithmetic operations on the data. This provides considerable motivation to restructure existing algorithms and to devise new algorithms that minimize data movement. Many of the algorithms in linear algebra can be expressed in terms of a SAXPY operation: y t ytax , i.e. adding a multiple a of a vector x to another vector y. This would result in three vector memory references for each two vector floating point operations. If this operation comprises the body of an inner loop which updates the same vector y many times then a considerable amount of unnecessary data movement will occur. Usually, a SAXPY occurring in an inner loop will indicate that the algorithm may be recast in terms of some matrix vector operation, such as y t y + ~ * x ,which is just a sequence of SAXPYs involving the columns of the matrix M and the corresponding components of the vector x . The advantage of this is the y vector and the length of the columns of M are a fixed size throughout. This makes it relatively easy to automatically recognize that only the columns of M need be moved into registers while accumulating the result y in a vector register, avoiding two of the three memory references in the inner most loop. This also allows chaining to occur on vector machines, and results in a factor of three increase in performance on the CRAY 1. The cost of the algorithm in these cases is not determined by floating point operations, but by memory references. Programs that properly use all of the features mentioned above will fully exploit the potential of a vector machine. These features, when used to varying degrees, give rise to
High Performance Computers arid Algorithms f r o m Linear Algebra
21
three basic modes of execution: scalar, vector, and super-vector [I]. To provide a feeling for the difference in execution rates, we give the following table for execution rates on a CRAY 1: Mode of Execution Scalar Vector Super-vector
I Rate of Execution 0 - 10 MFLOPS 10 - 50 IvLFLOPS 50 - 160 MFLOPS
These rates represent, more or less, the upper end of their range. We define the term MFLOPS to be a rate of execution representing millions of floating point operations (additions or multiplications) performed per second. The basic difference between scalar and vector performance is the use of vector instructions. The difference between vector and super-vector performance hinges upon avoiding unnecessary movement of data between vector registers and memory. The CRAY 1 is limited in the sense that there is only one path between memory and the vector registers. This creates a bottleneck if a program loads a vector from memory, performs some arithmetic operations, and then stores the results. While the load and arithmetic can proceed simultaneously as a chained operation, the store is not started until that chained operation is fully completed. Most algorithms in linear algebra can be easily vectorized. However, to gain the most out of a machine like the CRAY 1, such vectorization is usually not enough. In order to achieve top performance, the scope of the vectorization must be expanded to facilitate chaining and minimization of data movement in addition to utilizing vector operations. Recasting the algorithms in terms of matrix vector operations makes it easy for a vectorizing compiler to achieve these goals. This is primarily due to the fact that the results of the operation can be retained in a register and need not be stored back to memory, thus eliminating the bottleneck. Moreover, when the compiler is not successful, it is reasonable to hand tune these operations, perhaps in assembly language, since there are so few of them and since they involve simple operations on regular data structures. These modules and their usage in the recasting of algorithms for linear algebra are discussed in detail in the next section. With their use, the resulting codes achieve super-vector performance levels on a wide variety of vector architectures. Moreover, these modules have also proved to be effective in the use of parallel architectures. Vector architectures exploit parallelism at lowest level of computation. They require very regular data structures (i.e. rectangular arrays) and large amounts of computation in order to be effective. The next level of parallelism that may be effective is to have individual scalar processors execute serial instruction streams simultaneously upon a shared data structure. A typical example would be the simultaneous execution of a loop body for various values of the loop index. This is the capability provided by a parallel processor. Along with this increased functionality comes a burden. If these independent processors are to work
22
J. J. Dongarra and D. C. Sorensen
together on the same computation they must be able to communicate partial results to each other and this requires a synchronization mechanism. Synchronization introduces overhead in terms of machine utilization that is unrelated to the primary computation. It also requires new programming techniques that are not well understood at the moment. While this situation is obviously more general than that of a vector processor, many of the same principles apply. Typically, a parallel processor with globally shared memory must employ some sort of interconnection network so that all processors may access all of the shared memory. There must also be an arbitration mechanism within this memory access scheme to handle cases where two processors attempt to access the same memoIy location at the same time. These two requirements obviously have the effect of increasing the memory access time over that of a single processor accessing a dedicated memory of the same type. Usually, this increase is substantial and this is especially so if the processor and memory in question are at the high end of the performance spectrum. Again, memory access and data movement' dominate the computations in these machines. Achieving near peak performance on such computers relies upon the same principle. One must devise algorithms that minimize data movement and reuse data that has been moved from globally shared memory to local processor memory. The effects of efficient data management on the performance of a parallel processor can be vely dramatic. For example, performance of the Denelcor HEP computer may be increased by a factor of ten through efficient use of its very large (2K word) register set [12]. The modules again aid in accomplishing this memory management. Moreover, they provide a way to make effective use of the parallel processing capabilities in a manner that is transparent to the user of the software. This means that the user does not need to wrestle with the problems of synchronization in order to make effective use of the parallel processor. The two types of parallelism we have just discussed are combined when vector rather than serial processors are used to construct a parallel computer. These machines are able to execute independent loop bodies which employ vector instructions. The most powerful computers that exist today are of this type. They include the CRAY X - M F line, and a new high performance "mini-super'' FX/8 computer manufactured by Alliant. The problems with using such computers efficiently are of course more difficult than those encountered with each type individually. Synchronization overhead becomes more significant when compared to a vector operation rather than a scalar operation. Blocking loops to exploit outer level parallelism may conflict with vector length etc. Finally, a third level of complication is added when parallel-vector machines are interconnected to achieve yet another level of parallelism. This is the case for the CEDAR architecture being developed at the Center for Super Computing Research and Development at the University of Illinois at Urbana. Such a computer is intended to solve large applications problems which naturally split up into loosely coupled parts which may be solved efficiently on the cluster of parallel-vector processors.
High Performance Coinputers and Algorithms f r o m Linear Algebra
23
3. Performance of Software for Dense Matrix Factorization We are interested in examining the performance of linear algebra algorithms on largescale scientific vector processors and on emerging parallel processors. In many applications, linear algebra calculations consume large quantities of computer time. If substantial improvements can be found for the linear algebra part, a significant reduction in the overall performance will be realized. We are motivated to look for alternative formulations of standard algorithms, as implemented in software packages such as LINPACK 141 and EISPACK [9,111, because of their wide usage in general and their poor performance on vector computers. As mentioned earlier, we are also motivated to restructure in a way that will allow these packages to be easily transported to new computers of radically different design. If this can be accomplished without serious loss of efficiency there are many advantages. In this section we report on some experience with restructuring these linear algebra packages in terms of the high level modules proposed in [ 5 ] . This experience verifies performance increases are achieved on vector machines and that this modular approach offers a viable solution to the transportability issue. This restructuring often improves performance on conventional computers and does not degrade the performance on any computer we are aware of.
Both of the packages have been designed in a portable, robust fashion, so they will run in any Fortran environment. The routines in LINPACK even go so far as to use a set of vector subprograms called the BLAS [lo] to cany out most of the arithmetic operations. The EISPACK routines do not explicitly make reference to vector routines, but the routines have a high degree of vector operations which most vectorizing compilers detect. The routines from both packages should be well suited for execution on vector computers. As we shall see, however, the Fortran programs from LINPACK and EISPACK do not attain the highest execution rate possible on a CRAY 1 [3]. While these programs exhibit a high degree of vectorization, the construction which leads to super-vector performance is in most cases not present. We will examine how the algorithms can be constructed and modified to enhance performance without sacrificing clarity or resorting to assembly language.
To give a feeling for the difference between various computers, both vector and conventional, a timing study on many different computers for the solution of a loOx100 system of equations has been carried out [3]. The LINPACK routines were used in the solution without modification.
J.J. Dongarra and D . C. Sorensen
24
Solving a System of Linear Equations with LINPACK in Full Precision Computer
OS/Compilef
CRAY x - m -1 CDC Cyber 205 CRAY 1s CRAY x-MP-I Fujitsu VP-200 Fujitsu VP-200 Hitachi S-810/20 CRAY 1s CDC Cyber 205 NAS 9060 w/VPF
CFT(Coded BLAS) FTN(C0ded BLAS) CFT(Coded BLAS) CFT(Rol1ed BLAS) Fortran 77(Comp directive) Fortran 77(Rolled BLAS) FORT77/HAP(Rolled BLAS) CFT(Rol1ed BLAS) FTN(Rol1ed BLAS) VS opt=2(Coded BLAS)
Ratiod
MFLOPS'
Time secs
.36 .48
33 25 23 21 19 17 17 12 8.4 6.8
.021 .027 .030 ,032
.54
.57 .64 .72 .74 1 1.5 1.8
,040
,040 .042 ,056 ,082 .lo1
LINPACK routines SGEFA and SGESL were used for single precision and routines DGEFA and DGESL
were used for double precision. These routines perform standard LU decomposition with partial pivoting and backsubstitution. bFull Precision implies the use of (approximately) 64 bit arithmetic, e.g. CDC single precision or IBM
double precision.
'OSlCompiler refers to the operating system and compiler used, (Coded BLAS) refers to the use of assembly language coding of the BLAS, and (Rolled BLAS) refers to a Foman version with, single statement, simple
loops and Comp Directive refers to the use of compiler directives to set the maximum vector length.
dRafio is the number of times faster or slower a particular machine configuration is when compared to the CRAY 1.9 using a Fortran coding for the BLAS. 'MFLOPS is a rate of execution, the number of million floating point operations completed per second. For solving a system of n equations, approximately 2/3n3 + 2n2 operations are performed (we count both additions and multiplications).
The LINPACK routines used to generate the timings in the previous table do not reflect the true performance of "high performance computers". A different implementation of the solution of linear equations, presented in a report by Dongarra and Eisenstat [l], better describes the performance on such machines. That algorithm is based on matrix-vector operations rather than just vector operations. This restructuring allowed the various compilers to take advantage of the features described in Section 2. It is important to note that the
High Performance Computers atid Algorithms f r o m Linear Algebra
25
numerical properties of the algorithm have not been altered by this restructuring. The number of floating point operations required and the roundoff errors produced by both algorithms are exactly the same, only the way in which the matrix elements are accessed is different. As before, a Fortran program was run and the time to complete the solution of equations for a matrix of order 300 is reported. Note that these numbers are for a problem of order 300 and all runs are for full precision.
Solving a System of Linear Equations Using the Vector Unrolling Technique
Computer
OS/Compilef
CRAY X-MP-4 CRAY X-MP-2 Fujitsu VP-200 Fujitsu VP-200 CRAY X-Mp-2 * Hitachi S-810/20 CRAY X-MP-1 + CRAY X-MP-1 CRAY 1-M CRAY 1-S CRAY 1-M CRAY 1-S CDC Cyber 205 NAS 9060 w/VPF
'
CFT(Coded ISAMAX) CFT(Coded ISAMAX) Fortran 77(Comp directive) Fortran 77 CFT FORT77/HAP CFT(Coded ISAMAX) CFT CFT(Coded ISAMAX) CFTfCoded ISAMAX) CFT CFT ftn 200 opt=l VS opt=2(Coded BLAS)
MFLOPS'
Time secs
356 257 220 183 161 158 134 106 83 76 69 66 31 9.7
.05 1 .076 ,083 .099 .113 .115 .136 .172 ,215 .236 ,259 .273 .59 1.9
Comments.
These timings are for four processors with manual changes to use parallel features
' These timings are for two processors with manual changes to use parallel features. +
These timings are for one processor.
Similar techniques of recasting matrix decomposition algorithms in terms of matrix vector operations have provided significant improvements in the performance of algorithms for the eigenvalue problem. In a paper by Dongam, Kaufman and Hammarling
26
J. J. Dongarra and D. C. Sorensen
[6] many of the routines in the EISPACK collection have been restructered to use matrix vector primitives resulting in improvements by a factor of two to three in performance over the standard implementation on vector computers such as CRAY X-MP, Hitachi S8 10/20, and Fujitsu VP-200. Using the matrix vector operations as primitives in constructing algorithms can also play an important role in achieving performance on multiprocessor systems with minimal recoding effort. Again, the recoding is restricted to the relatively simple modules and the numerical properties of the algorithms are not altered as the codes are retargeted for a new machine. This feature takes on added importance as the complexity of the algorithms reach the level required for some of the more difficult eigenvalue calculations. A number of factors influence the performance of an algorithm in multiprocessing. These include the degree of parallelism, process synchronization overhead, load balancing, inter processor memory contention, and modifications needed to separate the parallel parts of an algorithm. A parallel algorithm must partition the work to be done into tasks or processes which can execute concurrently in order to exploit the computational advantages offered by a parallel computer. These cooperating processes usually have to communicate with each other to claim a unique identifier or follow data dependency rules for example. This communication takes place at synchronization points within the instruction streams defining the process. The amount of work in terms of number of instructions that may be performed between synchronization points is referred to as the granularity of a task. The need to synchronize and to communicate before and after parallel work will greatly impact the overall execution time of the program. Since the processors have to wait for one another instead of doing useful computation, it is obviously better to minimize that overhead. In the situation where segments of parallel code are executing in vector mode, typically at ten to twenty times the speed of scalar mode, granularity becomes an even more important issue, since communication mechanisms are implemented in scalar mode.
Granularity is also closely related to the degree of parallelism, which is defined to be the percentage of time spent in the parallel portion of the code. Typically, a small granularity job means that parallelism occurs in an inner loop levels (although not necessarily the innermost loop). In this case, even the loop setup time in outer loops will become significant without even mentioning frequent task synchronization needs. Matrix vector operations offer the proper level of modularity for achieving both performance and transportability across a wide range of computer architectures. Evidence has already been given for a variety of vector architectures. We shall present further evidence in the following sections concerning their suitability for parallel architectures. In addition to the computational evidence, there are several reasons which support the use of these modules. We can easily construct the standard algorithms in linear algebra out of these types of modules. The matrix vector operations are simple and yet encompass
High Performance Computers and Algorithms f r o m Linear Algebra
27
enough computation that they can be vectorized and also parallelized at a reasonable level of granularity [1,2,3,12]. Finally, these modules can be constructed in such a way that they hide all of the machine specific intrinsics required to invoke parallel computation. Thereby shielding the user from being concerned with any changes to the library which are machine specific.
4. Structure of the Algorithms In this section we discuss the way algorithms may be restructured to take advantage of the modules introduced above. Typical recasting that occurs within LINPACK and EISPACK type subroutines is discussed here. We begin with definitions and a description of the efficient implementation of the modules themselves.
4.1 The Modules. Only three modules are required for the recasting of LINPACK in a way that achieves the super-vector performance reported above. They are z=Mw
M
= M - wzT
(matrix x vector). (rank one modification).
and L=
Tz
(solve a triangular system).
Efficient coding of these three routines is all that is needed to transport the entire package from one machine to another while retaining close to top performance. We shall describe some of the considerations that are important when coding the matrix vector product module. The other modules require similar techniques. For a vector machine such as the CRAY-1 the vector times matrix operation should be coded in the form (4.1.1) for j = 1.2,...,n . y(*) t y(*) + M ( * J ) x )
In (4.1.1) the * in the ikst entry implies this is a column operation and the intent here is that a vector register is reserved for the result while the columns of M are successively read into vector registers, multiplied by the corresponding component of x and then added to the result register in place. In terms of ratios of data movement to floating point operations this arrangement is most favorable. It involves one vector move for two vectorfloating point operations. Comparing this to the three vector moves to get the same two floating point operations when a sequence of SAXPY operations are used shows the advantage of using the matrix vector operation. This arrangement is perhaps inappropriate for a parallel machine because one would have to synchronize the access to y by each of the processes, and this would cause busy
28
J.J. Dongarra and D. C. Sorensen
waiting to occur. One might do better to partition the vector y and the rows of the matrix M into blocks
,
Y1 Yz '
=
'
?k,
* .
Y1
Ml
Yz
Mz
+
!k,
'
.
7
Mk
and self-schedule individual vector operations on each of the blocks in parallel: yc t yi + M&
for i = 1,2,._., k.
That is, the subproblem indexed by i is picked up by a processor as it becomes available and the entire matrix vector product is reported done when all of these subproblems have been completed. If the parallel machine has vector capabilities on each of the processors this partitioning introduces short vectors and defeats the potential of the vector capabilities for small to medium size matrices. A better way to partition in this case is
Again, subproblems are computed by individual processors. However, in this scheme, we must either synchronize the contribution of adding in each term Msi or write each of these into temporary locations and hold them until all are complete before adding them to get the final result. This scheme does prove to be effective for increasing the performance of the factorization subroutines on the smaller (order less than 100) matrices. One can easily see this if the data access scheme for LU decomposition shown in Figure 4.1 is studied. We see that during the final stages of the factorization vector lengths become short regardless of matrix size. For the smaller matrices, subproblems with vector lengths that are below a certain performance level represent a larger percentage of the calculation. This problem is magnified when the row-wise partitioning is used.
4.2 Recasting LINPACK subroutines We now turn to some examples of how to use the modules to obtain various standard matrix factorizations. We begin with the LU decomposition of a general nonsingular matrix. Restructuring the algorithm in terms of the basic modules described above is not so obvious in the case of LU decomposition. The approach described here is inspired by the work of Fong and Jordan [7]. They produced an assembly language code for LU decomposition for the CRAY-I. This code differed significantly in structure from those commonly in use because it did not modify the entire k-th reduced submatrix at each step
High Performance Computers and Algorithms f r o m Linear Algebra
29
but only the k-th column of that matrix. This step was essentially matrix-vector multiplication operation. Dongma and Eisenstat [ 11 showed how to restructure the Fong and Jordan implementation explicitly in terms of matrix-vector operations and were able to achieve nearly the same performance from a FORTRAN code as Fong and Jordan had done with their assembly language implementation. The pattern of data references for factoring a square matrix A into PA = LU (with P a permutation matrix, L unit lower triangular, U upper triangular) is shown below in Figure 4.1.1.
L
FZZl
STEP t STEP 2
Figure 4.1.1. Lu Data References
At the k-th step of this algorithm, a matrix formed from columns 1 through k-I and rows k through n is multiplied by a vector constructed from the k-th column, rows 1 through k-1, with the results added to the k-th column, rows k through n. The second part of the k-th step involves a vector-matrix product, where the vector is constructed from the k-th row, columns 1 through k-1, and a matrix constructed from rows 1 through k-1 and columns k+l through n, with the results added to the k-th row, columns k+l though n. One can construct the factorization by analyzing the way in which the various pieces of the factorization interact. Let us consider decomposition of the matrix A into its LU factorization with the matrix partitioned in the following way: L11
30
J.J. Dongarra a n d D.C.Sorensen
Multiplying L and U together and equating terms with A we have:
We can now construct the various factorizations for LU decomposition by determining how to form the unknown parts of L and U given various parts of A, L and U. For example, Given the triangular matrices LI1 and UI1, to construct vectors 1422 = a22 - &u12. must perform, u12= ~ ~ l ' a zI&l , =
1&
and
uI2 and
scalar
uZ2 we
Since these operations deal with triangular matrix LI1 and Ull they can be expressed in terms of solving triangular systems of equations.
Given the rectangular matrices kl and U13, and the vectors izl and u12, we can form vectors I,, and ug3 and scalar uZ2 by forming & = ag3 - /gl~13, u,, = aZ2- /T1ul2, and l32 =
(%2 - L31u12)/u22,
Since these operations deal with rectangular matrices and vectors they can be expressed i n terms of simple matrix-vector operations.
Given the triangular matrix L,,, the rectangular matrix L~~~and the vector ITl we can construct vectors u12 and 132 and scalar uZ2 by forming uI2 = Lil'a12, u22 = q2- ~TIUIZ. 4 2
= (a32 - L ~ I U ~ U Z Z .
These operations deal with a triangular solve and a matrix vector multiply. The same ideas for use of high-level modules can be applied to other algorithms, including matrix multiply, Cholesky decomposition, and QR factorization.
For the Cholesky decomposition the matrix dealt with is symmetric and positive definite. The factorization is of the form A=LLT,
where A = A T and is positive definite. If we assume the algorithm proceeds as in LU decomposition, but reference only the lower triangular part of the matrix, we have an algorithm based on matrix-vector operations which accomplishes the desired factorization.
31
High Performance Computers and Algorithms f r o m Linear Algebra
The final method we shall discuss is the QR factorization using Householder transformations. Given a real m matrix A , the routine must produce an mxm orthogonal matrix Q and an nxn upper triangular matrix R such that
Householder's method consists of constructing a sequence of transformations of the form I
-
(4.2.1)
awwT ,where a wTw = 2.
The vector w is constructed to transform the first column of a given matrix into a multiple of the first coordinate vector el. At the k-th stage of the algorithm one has
and wt is constructed such that (4.2.2) The factorization is then updated to the form
with
However, this product is not explicitly formed, since it is available in product form if we simply record the vectors w in place of the columns they have been used to annihilate. This is the basic algorithm used in LINPACK [4]for computing the QR factorization of a matrix. This algorithm may be coded in terms of two of the modules. To see this, just note that the operation of applying a transformation shown on the left hand side of (4.2.2) above may be broken into two steps: zT = wTA
(4.2.3)
(vector x matrix)
and
A = A - awzT
(rank one modification).
4.3 Restructuring EISPACK subroutines As we have seen, all of the main routines of LINPACK can be expressed in terms of the three modules described in Section 4.1. The same type of restructuring may be used to obtain efficient performance from EISPACK subroutines. A detailed description of this may be found in [6]. In the following discussion we just outline some of the basic ideas
J.J. Dongarra and D. C. Sorensen
32
used there. Many of the algorithms implemented in EISPACK have the following form: Algorithm (4.3.1): For i = 1,.... Generate matrix T, Perform transformation A,, End .
t
TdiF1
Because we are applying similarity transformations, the eigenvalues of A,, are those of A,. Since the application of these similarity transformations represents the bulk of the work, it is important to have efficient methods for this operation. The main difference between this situation and that encountered with linear equations is that these transformations are applied from both sides. The transformation matrices Tiused in Algorithm (4.3.1) are of different types depending upon the particular algorithm. The simplest are the stabilized elementary transformation matrices which have the form T = L P , where P is a permutation matrix, required to maintain numerical stability [9,11,13], and L has the form
The inverse of L has the same structure as L and may be written in terms of a rank one modification of the identity in the following way:
with eTw = 0. If we put
then
B - Ewe:
C + w.7 D
+ wdr - Dwer + (dTw)wer
High Performance Computers and Algorithms from Linear Algebra
33
where cT= eTC, dT = eTD, and el is the f i s t co-ordinate vector (of appropriate size). The appropriate module to use therefore, is the rank one modification. However, more can be done with the rank two correction that takes place in the modification of the matrix D above. In most of the algorithms the transformation matrices Ti are Householder matrices of the form (4.2.1) shown above. This results in a rank two correction that might also be expressed as a sequence of two rank one corrections. Thus, it would be straightforward to arrange the similarity transformation as two successive applications of the scheme (4.2.3) discussed above. However, more can be done with a rank two correction as we now show. Firstly suppose that we wish to form (I-awwT)A(I-puuT), where for a similarity transformation a = p and w = U. We may replace the two rank one updates by a single rank two update using the following algorithm. Algorithm 4.3.2 1. v T = wTA 2. x = A u 3. y' = VT-(PWTX)UT 4. Replace A by A - b u T - a y y T As a second example that is applicable to the linear equation setting, suppose that we wish to form (l-awwT)(I-puuT)A, then as with Algorithm 4.3.2 we might proceed as follows. Algorithm 4.3.3 : 1. v T = wTA 2. xT = uTA 3. yT = v'(pw'u)xT 4. Replace A by A-puxT-awy7 In both cases we can see that Steps 1 and 2 can be achieved by calls to the matrix vector and vector matrix modules. Step 3 is a simple vector operation and Step 4 is now a rank-two correction, and one gets four vector memory references for each four vector floating point operations (rather than the three vector memory references for every two vector floating point operations, as in Step 2 of (4.2.3) ). These techniques have been used quite successfully to increase the performance of EISPACK on various vector and parallel machines. The results of these modifications is reported in full detail in 161. As a typical example of the performance increase possible with these techniques we offer the following table.
J.J. Dongurru and D.C. Sorensen
34
Comparison of EISPACK to Matrix Vector version
Routine
order 50
100
Machine
ELh4HES ORTHES ELMBAK ORTBAK TREDl TRBAK 1 TRED2 SVD nofvectors S V D with/vectors REDUC REBAK
1.5 2.5 2.2 3.6 1.5 4.2 1.6 1.7 1.6 1.8 4.4
2.2 2.5 2.6 3.3 1.5 3.7 1.6 2.0 1.7 2.2 5.8
CRAY 1 CRAY 1 CRAY 1 CRAY 1 CRAY x-Mp-1 CRAY X-MP-1 CRAY x-MP-1 Hitachi S-810/20 Hitachi S-810/20 Fujitsu VP-200 Fujitsu VP-200
(All versions in Fortran) (Speedup of matrix vector versions over the EISPACK routines.)
5. Conclusions As multiprocessor designs proliferate, research efforts should focus on "generic" algorithms that can be easily transported across various architectures. If a code has been written in terms of high level synchronization and data management primitives, that are expected to be supported by every member of the model of computation, then these primitives only need to be customized to a particular realization. A very high level of transportability may be achieved through automating the transformation of these primitives. The benefit to software maintenance, particularly for large codes, is in the isolation of synchronization and data management peculiarities. This desire for portability is often at odds with the need to efficiently exploit the capabilities of a particular architecture. Nevertheless, algorithms should not be intrinsically designed for a particular machine. One should be prepared to give up a marginal amount of efficiency in trade for reduced man power requirements to use and maintain software. There are many possibilities for implementation of the general ideas that are briefly
High Performance Computers and Algorithms from Linear Algebra
35
described above. We are certainly not in a position to recommend a particular implementation with any degree of finality. However, we already have experience indicating the feasibility of both of the approaches discussed here. We believe that a high level of transportability as described above can be achieved without seriously degrading potential performance. We would like to encourage others to consider the challenge of producing transportable software that will be efficient on these new machines.
6. References J.J. Dongarra and S.C. Eisenstat, Squeezing the Most out of an Algorithm in CRAY Fortran, ACM Trans. Math. Software, Vol. 10, No. 3, 1984. J.J. Dongarra and D.C. Sorensen, A Fully Parallel Algorithm for the Symmetric Eigenvalue Problem, Argonne National Laboratory, Report MCS/Tm 62, (January 1986). J.J. Dongma, Performance of Various Computers Using Standard Linear Equations Software in a Fortran Environment, Argonne National Laboratory Report MCS-TM23, (updated August 1984).
J.J. Dongma, J.R. Bunch, C.B. Moler, and G.W. Stewart, LINPACK Users’ Guide, SIAM Publications, Philadelphia, 1979. J.J. Dongarra, J. Du Croz, S. Hammarling, R.J. Hanson, A Proposal for an Exrended Set of Fortran Basic Linear Algebra Subroutines, Argonne National Laboratory Report MCS/TM 41, Revision 1 (October, 1985). J.J. Dongarra, L. Kaufman, and S. Hammarling Squeezing the Most our of Eigenvalue Solvers on High-Performance Computers, Argonne National Laboratory Report ANL MCS-TM 46, (January 1985), to appear in Linear Algebra and Its Applications. K. Fong and T.L. Jordan, Some Linear Algebra Algorithms and Their Performance on CRAY-1, Los Alamos Scientific Laboratory, UC-32, June 1977. Flynn, M. J. Very high-speed computing systems. Proc IEEE, vol. 54, pp. 1901-1909, (1966).
36
J.J. Dongarra and D.C. Sorensen
[9] B.S. Garbow, J.M. Boyle, J.J. Dongma, and C.B. Moler, Matrix Eigensystern Routines - EISPACK Guide Extension, Lecture Notes in Computer Science, Vol. 51, Springer-Verlag, Berlin, 1977. [lo] C. Lawson, R. Hanson, D. Kincaid, and F. Krogh, Basic Linear Algebra Subprograms f o r Fortran Usage. ACM Trans. Math. Software, 5 (1979), 308-371.
[ I l l B.T. Smith, J.M. Boyle, J.J. Dongma, B.S. Garbow, Y. Ikebe, V.C. Klema, and C.B. Moler, Matrix Eigensystem Routines - EISPACK Guide, Lecture Notes in Computer Science, Vol. 6, 2nd edition, Springer-Verlag, Berlin, 1976.
[12] D.C. Sorensen, Buffering f o r Vector Performance on a Pipelined MIMD Machine, Parallel Computing, Vol. 1, pp. 143-164, 1984. [131 J.H. Wilkinson, The Algebraic Eigenvalue Problem, Clarendon Press, Oxford 1965.
Work supported in part by the Applied Mathematical Sciences subprogram of the Office of Energy Research, U.S. Department of Energy under Contracts W-31-109-Eng-38 and DE-AC05-840R21400.
Large Scale Eigenvalue Problems I. Cullum and R.A. Willoughby (Editors) 0 Elsevier Science Publishers B.V. (North-HoUand), 1986
31
THE IMPACT OF PARALLEL ARCHITECTURES ON THE SOLUTION O F EIGENVALUE PROBLEMS Ilse C.F. Ipsen, Youcef Saad Department of Computer Science Yale University New Haven, Connecticut U.S.A.
This paper presents a short survey of recent work on parallel implementations of Numerical Linear Algebra algorithms with emphasis on those relating to the solution of the symmetric eigenvalue problem on loosely coupled multiprocessor architectures. The vital operations in the formulation of most eigenvalue algorithms are matrix vector multiplication, matrix transposition, and linear system solution. Their implementations on several representative multiprocessor systems will be described, as well as parallel implementations of the following classes of eigenvalue methods : QR, bisection, divide-and-conquer, and Lanczos algorithm. 1. Introduction Undoubtedly the most significant impact on research in Scientific Computation, and Numerical Linear Algebra in particular, has come about with the advent of vector and parallel computation. This paper presents a short survey of recent work on parallel implementations of Numerical Linear Algebra algorithms with emphasis on those relating to the solution of the symmetric eigenvalue problem on loosely coupled multiprocessor architectures. Although the concept of parallel computation was well understood in the early years of electronic computing (already Babbage recognised it as a powerful means for speeding up the multiplication of two numbers [4]), it intermittently had to give way to the largely sequential von Neuman Computer. The recent turn to parallel computer architectures is motivated by the serious limitations inherent in the von Neuman model, the most important one being its limits to miniaturisation, imposed by physical constraints, which put a bound on the maximum speed of a logical circuit. Consequently, the only means for increasing computing speed by orders of magnitude is to resort to parallelism. In a parallel computer (also called ‘multiprocessor’) different processors share the computations involved in the solution of a problem. To this end, the computation must be decomposed and broken up into tasks, which can be performed simultaneously by the different processors; and organised co-operation among the processors must be established by means of synchronisation and data exchange. The selection of an algorithm, its decomposition into separate computational tasks and their subsequent assignment to particular processors, z well as the physical channels and protocols by means of which the processors communicate are among the many factors leading to a multitude of parallel implementations for any one problem. The above decisions are aggravated by the need (or perhaps absence) of adequate performance measures. Even if a certain multiprocessor machine is already specified, one still faces the problem of having to decompose a particular algorithm into tasks with the objective of gaining maximal speed-up and a balanced work-load for all processors. Before that, however, reliable criteria for evaluating and comparing the performance of different implementations of an algoritlim on that machine are indispensable. It is also necessary, of course, to be able to
L C.F. Ipsen and Y. Saad
38
compare implementations of different algorithms on one machine, as well as implementations of different algorithms on diflerent machines. These issues are far from being resolved. One of the reasons is that a fair assessment of two architectures must be based on the availability of adequate hardware as well as software. Yet, due to the absence of systematic design techniques the development of software is and will undoubtedly continue to lag behind hardware development. Section 2 presents a brief characterisation of popular parallel architectures and justifies our preference for a loosely coupled multiprocessor architecture. The choice of machine model in turn influences the choice of a parallel algorithm for solving a particular problem. Extensive surveys of parallel algorithms can be found in the articles by Heller [35], Sameh [69, 70, 711, and Ortega and Voigt [60]. Unlike in the single processor case the performance of a parallel algorithm is not only judged by its arithmetic speed but, equally, by the time required to exchange data and coordinate co-operation among processors; understanding of this aspect has only started [29, 30, 611. Accordingly, we present a collection of parallel algorithms for basic Linear Algebra tasks in Section 3 and their application to the parallel solution of eigenvalue problems in Section 4. To conclude, the last section casts a glance at some novel techniques which promise to alleviate the complex problem of parallel algorithm development. 2. Architectures
There exist quite a few classifications of multiprocessor architectures, and we will employ two of them. The first one distinguishes architectures by the way processors relate their instructions to the data while the second one groups machines according to the structure of their communication environment. The two most important categories of parallel architectures are Single Instruction Stream Multiple Data Stream (SIMD) and Multiple Instruction Stream Multiple Data Stream- (MIMD) machines [24]. SIMD machines initiate a single stream of vector instructions, which may be realised by pipelining in one processor or operating arrays of processors [38]. Examples include the CRAY-1, the ICL-DAP [23] and the ILLIAC IV [80]. MIMD machines simultaneously initiate different instructions on (necessarily) different data streams, essentially all multiprocessor configurations are included in this class [38]. Among the MIMD machines one can in turn differentiate between two types :
Shared memory models : processors have very little local or ‘private’ memory; they exchange data and co-operate by accessing a global shared memory. 0 Distributed memory models : there is no global memory, but processors possess a significant amount of local memory (with no access to other processor’s local memory); there are physical interconnections between certain pairs of processors, and data and control information is transferred from one processor to another along a path of these interconnections. 0
2.1. The Shared Memory Model The shared memory model is frequently implemented by connecting k processors to k memories via a large switching network, see Figure 1 (this switching network may be replaced by a global bus when the number of processors k is small). Thus the memory can be viewed as split into k ‘banks’, and shared among the k processors. Variations on this scheme are numerous, but the essential features here are the switching network and the shared memory; examples include the Ultracomputer developed at NYU [32] which uses an Omega network. Programming is greatly facilitated due to transparent data access (from the user’s point of view data are stored in one large memory readily accessible to any processor) and the ability of the switching network to simulate any interconnection topology. However, memory conflicts can lead to degraded performance and the shared memory models cannot easily take advantage of proximity of data in problems with local (data) dependences; these questions
Parallel Architectures apld the Solution of Eigenvalue Problems
39
Figure 1: A Tight,ly Coupled Shared Memory Machine. are addressed in [26, 791. Furthermore, the switching network becomes exceedingly complex as the number of processors and memories increases : the connection of N processors to N memories in general requires a total of O ( N log2 N ) identical 2 x 2 switches. 2.2. The Distributed Memory Model
In the distributed memory model, the processors are identical and the processor interconnections form a regular topology; examples are depicted in Figures 2, 3 and 4. There is no tight global synchronisation, and the computations are data driven (ie, a computation in a particular processor is performed only when the needed operands become available). Examples include the finite element machine [47], tree machines [12], the cosmic cube [78] and systolic arrays (511. Clearly, one of the most important advantages of the second class of architectures is its ability to exploit locality of data dependences in order to keep communication costs to a minimum. Thus, a two-dimensional processor grid as in Figure 3 is perfectly suitable for solving discretised elliptic partial differential equations (eg, by assigning each grid point to a corresponding processor) because iterative methods for solving the resulting linear systems require only interaction between adjacent! grid points. Hence, an efficient general purpose multiprocessor must have powerful mapping capabi6ities, ie, it must be able to easily emulate many common topologies such as grids or linear arrays. 2.3. Hypercube-based Architectures The ‘hypercube’ (boolean cube, n-cube), a distributed memory machine, constitutes an excellent compromise between a linear array and a completely interconnected network of processors. It offers a rich interconnection structure with large bandwidth, logarithmic diameter, and the ability to simulate every realistic architecture with small overhead. This explains the growing interest in hypercube-based parallel machines; commercially available machines at this point (last quarter of 1985) are the 128-processor INTEL iPSC/d7, the 1024-processor NCUBE/Ten, the 64000-processor (bit-sliced) Connection Machine from Thinking Machines and the soon-to-be-available 256-processor Ametek/System 14 [19]. The topology of a hypercube is best described by a simple recursion : a hypercube of dimension 1 consists of two connected processors and a hypercube of dimension n f l is made up of two identical subcubes of dimension n by connecting processors in corresponding positions; an illustration is given in Figure 4 which shows a four-dimensional cube constructed from two three-dimensional cubes. Topological characterisations of the hypercube, in particular with respect to embeddings of different graphs in the hypercube are investigated in (6, 67, 651.
40
I, C F. Ipsen and Y. Saad
Figure 2: A Processor Ring Consisting of Eight Processors.
Figure 3: A 4 x 4 Multiprocessor Grid. 3. Parallel Algorithms for Basic Linear Algebra Computations
When analysing the complexity of parallel methods in numerical linear algebra, one must bear in mind that the total time required to run an algorithm on a multiprocessor system does not only depend on pure arithmetic time but also on the time needed for exchanging data among processors. This implies a great richness in the class of algorithms, in terms of the assignments of tasks to processors and the assumed topology of the processor communication network : for a particular task it is now important when its input data become available as results of preceding calculations, in which processor they are located, and how long it will take to move them to the requesting processor.
Parallel Architectures and the Solution of Eigenvalue Problems
41
Figure 4: A Hypercube of Dimension 4. In many practical applications the number of processors k will usually be much smaller than the ‘problem size’ N (eg, the order of the matrix), and a large variety of algorithms can be found by choosing different ways of assigning matrix elements to the processors. We must take into account that times for data transfer are not negligible and may, in fact, dominate the times for actual arithmetic. A fairly general and yet simple communication model is proposed in (431, and the algorithms are characterised and compared with respect to their requirements for arithmetic as well as communication. It is assumed, that any processor is capable of writing to one of its directly connected neighbours while reading from the other. For purposes of estimating the computation time, processors are considered to work in lock step where one step corresponds to the computation time of the slowest processor, which, in particular, implies that identical tasks will take an equal amount of time if started simultaneously on different processors. This assumption is by no means restrictive and its sole purpose is to simplify the complexity analysis and its results. As a matter of fact, most of the parallel algorithms proposed so far can be viewed as SIMD methods, in the sense that a typical parallel loop comprises k identical tasks to be executed in parallel. It is further assumed that communication and arithmetic are not overlapped, which is the case, for instance, when processors are not equipped with 1/0 co-processors (ie, processors solely devoted to performing input and output). Yet even when this is not true, it is important to have a realistic measure of what exactly constitutes communication and what computation time, in order to judge the efficiency of an algorithm. We define communication (or data transfer) time as the time to execute an algorithm (in lock step mode) under the assumption that arithmetic can be done in zero time (that is, the arithmetic unit is regarded to be infinitely fast). Arithmetic time can then be defined analogously. The corresponding computation time is at most double of the one resulting from overlapped computation and communication [61]. If processor interconnections are capable of transmitting R words per second, then the inverse is denoted by r . In general, each transfer of a data packet is associated with a constant start-up (set-up) time of @, which is independent of the size (the number of words) per packet. Often, the start-up times are (much) larger than the elemental transfer times, that is, /3 >> r . The time to send a packet of size N from a processor to its neighbour is t~ = N r . On a single processor, a linear combination of two vectors of length N takes time t~ = 7 N w , where 7 is the pipe fill time (it is zero for non-pipelined machines), w the time for one scalar operation and 7 2 w (again, the start-up time dominates the elemental operation time).
+
+
42
I. C.F. Ipsen and Y. Saad
+
For any algorithm the sum of its transfer and arithmetic time, t~ t ~ is, simply called its computation time. The following section gives a short overview (with no claims of being complete) over possible implementations of basic Linear Algebra operations on the three loosely coupled architectures. The efficiency of parallel eigenvalue methods depends crucially on the implementation of data transfers, matrix transposition, matrix vector multiplication, and linear system solution. 3.1. Algorithms for the Processor Ring
A multiprocessor ring is one of the simplest interconnection schemes and yet it is one of the most cost-effective architectures when it comes to bridging the gap between future super computers and current vector computers. As suggested in [77] a small number of inexpensive off-the-shelf standard array processors can easily be connected in a ring yielding a machine with the computing power of a CRAY-1. As mentioned before, rings can be emulated without difficulty by most loosely coupled architectures. Time complexities of elementary data transfers and dense matrix vector multiplication on a processor ring are discussed in [43]. These operations are used to implement various algorithms for solution of dense linear systems by Gaussian elimination on a processor ring. Three ways of assigning matrix elements to particular processors are considered : by rows, by columns and by diagonals. A summary of the results obtained follows. The communication times are low order terms compared to the arithmetic operation times when the number of processors k is small compared to the order of the matrix N . Both the arithmetic times and the communication times of the triangular system solution methods are low order terms in comparison with those of Gaussian elimination. Allocating non-adjacent rows or columns of the matrix to a processor results in better arithmetic performance but worse communication performance. Allocation of non-adjacent diagonals to a processor results in poor overall performance. The overhead of pivoting is small compared with the cost of Gaussian Elimination; it is however of the same order of magnitude as that of triangular system solution. Pivoting is less expensive for the row-oriented schemes. In [61] lower bounds on the communication complexity for dense Gaussian elimination on bus and ring oriented architecture are shown : the communication time is of order at least O ( N 2 ) ,independent of the number of processors. Gaussian elimination for dense systems on a multiprocessor ring is discussed in [71]. Lawrie and Sameh [53]present a technique for solving symmetric positive-definite banded systems, which is a generalisation of a method for tridiagonal system solution on multiprocessors; it takes advantage of different alignment networks for allocating data to the memories of particular processors. Implementation of the Cholesky factorisation on a ring is discussed in [34].New algorithms and implementations for the solution of symmetric positive-definite systems are given in [16], as well as minimal time implementations for Toeplitz matrices on linear systolic arrays, which can be easily adapted to loosely-coupled systems. The literature on solution of linear systems arising from partial differential equations is extensive, the reader is referred to the survey by Ortega and Voight [60]. Iterative methods on rings or linear arrays have been considered by Saad and Sameh [62], Saad, Sameh, and Saylor [63] and more recently by Johnsson, Saad and Schultz 146, 681. In [68] it was shown that a speed-up of up to O ( n ) can be achieved when the system arises from an elliptic partial differential equation. Although implementations of direct sparse matrix techniques on vector and parallel computers have been considered by Duff [22, 211, there is very little work dealing with parallel implementations of sparse direct solutions; parallel nested dissection for finite element problems is discussed in [GANNON].
Parallel Architectures and the Solution of Eigenvalue Problems
43
3.2. Algorithms for the Two-Dimensional Processor Grid
A lower bound for the complexity of communication in dense Gaussian Elimination on a two-dimensional grid of processors is O ( N 2 / & ) O ( N & ) [61 when no overlapping of successive steps in Gaussian elimination takes place and O ( N 2 / k ) O(&) for pipelined algorithms. In the spirit of the ‘wavefront concept’ made popular by S.Y. Iiung 1501 data flow algorithms for dense Cholesky factorisation are developed in [59]. Six different implementations of the dense Cholesky factorisation, depending on the arrangement of the three loop indices, on a processor grid are discussed and compared in [27, 341. The preferred variant turns out to be a computation of the Cholesky factor by columns, whereby previously computed columns are accessed columnwise. The idea for transposing dense and banded matrices on two-dimensional architectures was first developed for systolic arrays in [13, 40) and carries over right away to multiprocessor systems; similar ideas can be found in [58]
+
JL +
3.3. Algorithms for the Hypercube The hypercube topology has been the focus of much recent research in parallel computation; since it can easily emulate many other architectures, the first task is the assignment of data to the processors so as to optimise processor utilisation. Saad and Schultz [65] establish general properties of the hypercube, while Bhatt and Ipsen [6] present algorithms for efficient embeddings of trees onto hypercubes for potential employment in adaptive numerical computations. Chan and Saad [14] propose a mapping of the grid points onto the hypercube so as to minimise processor communication in Multigrid methods. Saad and Schultz 1661 discuss the problem of solving banded linear systems on ring, mesh and hypercube architectures. It is concluded that the concept of one best algorithm for a given architecture is no longer valid. For instance, consider a simple banded linear system of half-bandwidth v and order N . It is not realistic to assume, as often done, that the halfbandwidth v matches exactly the number of processors k , ie, that v = k or that u2 = k. In reality the total number of available processors k is fixed, and one has to determine the best way of solving the system for different values of v and N. In the extreme case of a tridiagonal matrix, where v = 1, Gaussian elimination is not parallelisable, and therefore should be excluded, while the cyclic reduction algorithm [44] which is not advantageous for sequential machines is highly parallel and should be selected. At the other extreme, when the half-bandwidth v is very large with respect to k, simple banded Gaussian elimination with the rows of the matrix uniformly distributed over the processors performs best [66]. Similar observations can be made for AD1 methods [46, 641. Thus, it appears that in the standard software packages of tomorrow one will find several different codes for solving the same problem : according to parameters like size of the problem, number of floating point operations per second and communication bandwidth the program would dynamically choose the best alternative. 4. Eigenvalue Algorithms
So far, most of the work on parallel computation of eigenvalues for (symmetric) matrices has seemed to concentrate on the development of systolic array algorithms and hardware mainly with signal processing applications in mind. 4.1. Methods for (Small) Dense Matrices There is a variety of systolic array implementations for the symmetric eigenvalue problem based on the shifted QR algorithm with Givens’ rotations (the exception is [45] which makes use of Householder transformations). One can use either the arrays for orthogonal factorisations [I, 7, 31, 36, 37, 541 (see 15, 411 for the implementation of ‘Fast’ rotations) or the ones constructed specifical!y for the solution of the tridiagonal [36, 571, positive-definite
44
I. C. F. Ipsen and Y. Saad
tridiagonal [28] or banded [75] eigenvalue problem; however no satisfactory way for an efficient shift computation has been found. Other designs for symmetric tridiagonal eigenvalue computations include a doubling version of the QR method 131,methods based on isospectral flows 121, and Newton’s method or bisection [75]. Often a second, different set of arrays is required to reduce by similarity transformations the original matrix to tridiagonal or banded form [36,74, 751. Earlier papers for the solution of the tridiagonal eigenvalue problem on more general parallel architectures suggest the use of multiple Sturm sequences and bisection (491, and computation of the QR iteration via recurrence equations (7’21. Parallel implementations of Jacobi’s method to compute the eigenvalues of dense symmetric matrices have been considered in [49, 521 as early as 1971. Implementations for the ICL DAP [23] can be found in [56] and improved versions for systolic array implementations 191 seem to be most promising in terms of actual physical realisation since no shift computations are necessary and the architectures are simple; a modified algorithm is presented in [76] to deal with problems whose size does not match the number of available processors. A systolic array that solves the generalized eigenvalue problem for dense matrices via the Q Z algorithm is discussed in [Sl. Suggestions for parallel computation of certain instances of the generalized and the nonsymrnetric eigenvalue problem, as well as for Lanczos method, are presented in [70]. The divide-and-conquer approach to the tridiagonal eigenvalue problem introduced by Cuppen [ 151 has been advocated for implementations on tree machines [48], shared-memory architectures [20] and the hypercube [42]. In signal processing applications, the preferred approach is to design arrays that compute the singular values of the Cholesky factors of the covariance matrix directly instead of employing arrays for computing the eigenvalues of the symmetric positive-definite covariance matrix itself. Examples include arrays based on the Jacobi method [lo, 11, 551 and on Givens’ rotations [36, 39, 761. The iterative reduction to triangular form of a square matrix by Schur rotations is suggested in [59] for an architecture similar to the the systolic array in [55]. For general two-dimensional processor grids, a dataflow algorithm similar to the one for Cholesky factorisation is developed [59] for congruence transformations. 4.2. Methods for Large Sparse Matrices Generally speaking large sparse eigenvalue problems have rarely been examined from the parallel point of view, since the parallel algorithms are either trivial extensions of those for solving linear systems or obvious adaptations of sequential algorithms. For instance, a good shift-and-invert technique can be implemented for Lanczos algorithm if the inner loops, which consist of system solutions, can be efficiently parallelised. As a second example, the Subspace Iteration method is a perfectly parallelisable method and might actually constitute a reasonable choice in multiprocessor environments despite its sluggishness on sequential machines. Although actual comparisons between parallel implementations of this technique and Lanczos method in the symmetric case are in order, we still expect Lanczos algorithm to be superior. Along different lines Sameh and Wisniewski (731 and Wisniewski (811 propose an approach based on trace minimisation for solving the generalised eigenvalue problem Ax = XBx, where the trace of WTAW is minimised over all N x M matrices W that are B-orthonormal. An interesting idea by Grimes et al. [33] is the use of Lanczos algorithm, the preferred method for solution of large sparse symmetric eigenvalue problems, to solve even relatively s m a l l a n d dense problems. The tests conducted on a Cray-XMP124 show that Lanczos algorithm performs better on dense matrices of order 219 to 1496 than the Cray-optimised EISPACK routines. Thus, vector and parallel machines may result in unexpected changes regarding the range of applicability of classical algorithms. Large sparse nonsymmetric eigenvalue problems are crucial in the analysis of complex dynamic systems. Due to nonlinearities, the numerical nature of these problems is so intractable that scientists and engineers often abandon them and resort to simplified models.
Parallel Architectures and the Solution of Eigenvalue Problems
4s
In this situation the use of massive parallel computing could be a decisive factor as it might enable the solution of presently intractable problems. Nonsymmetric eigenvalue methods will benefit very strongly from an increase in computational power and parallelism. It is hoped that these difficult problems will be tackled once reasonably reliable and user-friendly parallel machines appear on the market. 5. Outlook
For multiprocessor systems with regular communication topologies, such as the ones discussed here, novel techniques promise ‘automatic’ design of many algorithms. Progress in this area has been made especially for the implementation of algorithms on systolic arrays (surveys can be found in [IS, 251). These approaches are easily adaptable to more general multiprocessor architectures with regular communication topologies. Given sets of recurrence equations (eg, a FORTRAN program consisting of several sets of nested loops) the design methodology developed in [17, 181 delivers the description of provably optimal, regular, parallel architectures for their implementation. Upoii specification of certain constraints for the resulting architecture, eg, avoidance of broadcasting or restriction to nearest-neiglibour communication, the methodology generates an optimal mapping onto a given architecture. Application of this design methodology will allow the numerical analyst to concentrate on the aIgorithm development, particularly on modifications for improved numerical behaviour or pipelinability, rather than on the mapping or implementation process.
Acknowledgements The work presented in this paper was supported by the Office of Naval Research under contracts N000014-82-K-0184 and N00014-85-K-0461.
46
I. C F. @sen and Y. Saad
References 111 Ahmed, H.M., Delosme, J.-M. and Morf, M., Highly Concurrent Computing Structures f o r Matrix Arithmetic and Signal Processing, IEEE Computer, 15 (1982), pp. 65-82. [2] Ang, P.H., Delosme, J.-M. and Morf, M., Concurrent Implementation of Matrix
Eigenvalue Decomposition based on Isospectral Flows, Proc. 27th Ann. Symp. of SPIE, 1983. [3] Ang, P.H. and Morf, M., Concurrent Array Processor for Fast Eigenvalue Computations, PTOC. 1940 IEEE ICASSP, 1984, pp. 34lA.2.1-4. [4] Babbage. H.P., Babbage’s Analytical Engine, Mon. Not . Roy. Astron. SOC., 70 (1910), pp. 517-26. [5] Barlow, J.L. and Ipsen, I.C.F, Scaled Givens Rotations f o r the Solution of Linear Least Squares Problems on Systolic Arrays, SIAM J . Sci. Stat. Comp., (1986). [6] Bhatt, S.N. and Ipsen, I.C.F., Embedding Trees an the Hypercube, Research Report 443, Dept Computer Science, Yale University, 1985. Submitted for publication. [7] Bojanczyk, A., Brent, R.P. and Kung, H.T., Numerically Stable Solution of Dense
Systems of Linear Equations Using Mesh-Connected Processors, SIAM J . Sci. Stat. Comp., 5 (1984), pp. 95-104. [8] Boley, D., A Parallel Method for the Generalized Eigenvalue Problem, Technical Report 84-21, Dept Computer Science, University of Minnesota, 1984. [9] Brent, R.P., and Luk, F.T.,, The Solution of Singular- Value and Symmetric Eigenvalue Problems on Multiprocessor Arrays, SIAM J. Sci. Stat. Comput., 6 (1985), pp. 69-84.
R.P., Luk, F.T., and van Loan, C.F., Computation of the Singular Value Decomposition Using Mesh-Connected Processors, J. VLSI Computer Systems, 1 (1984). To appear. 1111 , Computation of the Generalized Singular Value Decomposition Using MeshConnected Processors, Proc. SPIE Symp. 4% (Real Time Signal Processing VI), 1983, pp. 66-71. [12] Browning, S.A., The Tree Machine : A Highly Concurrent Computing Environment, Technical Report TR-3760, Dept Computer Science, California Institute of Technology, 1980. [ 131 Cappello, P.R. and Steiglitz, I<., Selecting Systolic Designs Using Linear Transformations of Space-Time, Proc. SPIE Symp. 549 (Real Time Signal Processing VII), 1984, pp. 75-85. [14] Chan, T.F. and Saad, Y., Multigrid Algorithms on the Hypercube Multiprocessor, Research Report 368, Dept Computer Science, Yale University, 1985. [15] Cuppen, J.J.M., A Divide and Conquer Method f o r the Symmetric Tridiagonal Eigenproblem, Num. Math., 36 (1981), pp. 318-40. [ 161 Delosme, J.-M. and Ipsen, I.C.F., Parallel Solution of Symmetric Positive Definite Systems with Hyperbolic Rotations, Lin. Alg. Appl., (1986). , Eficient Systolic Arrays for the Solution of Toeplitz Systems : A n Illustration of ~ 7 1 a Methodology for the Construction of Systolic Architectures in VLSI, Research Report 370, Dept. of Computer Science, Yale University, 1985. , An Illustration of a Methodology for the Construction of Efficient Systolic [I81 Architectures in VLSI, Proc. Second Int. Symposium on VLSI Technology, Systems and Applications, Taipei, Taiwan, 1985, pp. 268-73. [19] Dongarra, J.J. and Duff, I.S., Advanced Architecture Computers, Technical Memorandum 57, Argonne National Laboratory, 1985.
[lo] Brent, ~
Parallel Architectures and the Solution of Eigenvalue Problems
47
(20) Dongarra, J.J. and Sorensen, D.C., A Fast Algorithm for the Symmetric Eigenvalue Problem, IEEE ARITH7 Conference, University of Illinois, Urbana, 1985. [21] Duff, I.S., Parallel Implementation of Multifrontal Techniques, Technical Memorandum 49, Argonne National Laboratory, 1985. , The solution of sparse linear equations on the Cray-1, J.S. Kowlik ed., High 1-22] Speed Computation, Springer Verlag, New York, Berlin, 1984, pp. 289-309. NATO AS1 series. (231 Flanders, P.M., Hunt, D.J., Reddaway, S.F. and Parkinson, D., Efficient High Speed Computing with the Distributed Array Processor, High Speed Computer and Algorithm Organization, Academic Press, 1977, pp. 113-28. (241 Flynn, M.J., Some Computer Organizations and their Efeetiveness, IEEE Trans. Comp., C-21 (1972), pp. 948-60. [25] Fortes, J.A.B., Fu, K.S. and Wah, B.W., Systematic Approaches to the Design of Algorithmically Specified Systolic Arrays, Proe. ICASSP, 1985, pp. 8.9.1-4. 1261 Frailong, J.M, Jalby, W. and Lenfant, J., XOR Schemes: A Flexible Data Organization in Parallel Memories., Proc. 1985 Int. Conf. Parallel Processing, IEEE, 1985, pp. 276-83. (271 Funderlic, R.E. and Geist, G.A., Torus Data Flow f o r Parallel Computation of Missized Matrix Problems, Technical Report ORNL-6125, Oak Ridge National Laboratory, 1985.
[28] Gajski, D.D, Sameh, A.H. and Wisniewski, J.A., Iterative Algorithms for Tridiagonal Matrices on a WSI-Multiprocessor, Proe. Int. Conf. Parallel Processing, IEEE, 1982, pp. 82-9. (291 Gannon, D. and van Rosendale, J., On the Impact of Communication Complexity in the Design of Parallel Algorithms, Technical Report 84-41, ICASE, 1984. [30] Gentleman, W.M, Some Complexity Results f o r Matrix Computation on Parallel ProcesS O T S , JACM., 25 (1978), pp. 112-115. [31] Gentleman, W.M., and Kung, H.T., On Stable Parallel Linear System Solvers, Proc. SPIE (Real Time Signal Processing IV), 1981, pp. 19-26. [32] Gottlieb, A,, Grishman, R., Kruskal, C.P., McAuliffe, K.P., Rudolph, L. and Snir, M., The NYU Ultracomputer - Designing an MIMD Shared Memory Parallel Computer, IEEE Trans. Corn?., C-32 (1983), pp. 175-89. [33] Grimes, R.G., Lewis, J.G., Simon, H., Krakauer, H. and Wei, S.H., The Solution of Large Dense Generalized Eigenvalue Problems on the Cray X-MP/.Z4, 1985. Presentation given at, the Second SIAM Conf. Par. Proc. and Sci. Comp., Norfolk, VA. [34] Heath, M.T., Parallel Cholesky Factorisation in Message-Passing Multiprocessor Environments, Technical Report ORNL-6150, Oak Ridge National Laboratory, 1985. Submitted to Parallel Computing. [35] Heller, D.E., A Survey of Parallel Algorithms in Numerical Linear Algebra, SIAM Review, 20 (1978), pp. 740-777. [36] Heller, D.E. and Ipsen, I.C.F., Systolic Networks for Orthogonal Equivalence Transformations and their Applications, Penfield, P. ed., Proc. Conf. Advanced Research in VLSI, 1981, Artech House, Inc., 1982, pp. 113-22. (371 Heller, D.E. and Ipsen, I.C.F, Systolic Networks for Orthogonal Decompositions, SIAM J. Sci. Stat. Comp., 4 (1983), pp. 261-9. [38] Hockney, R.W. and Jesshope, C.R., Parallel Computers, Adam Hilger Ltd, Bristol, 1981. [39] Ipsen, I.C.F., Singular Value Decomposition with Systolic Arrays, Proc. SPIE Symp. 549 (Real T i n e Signal Processing VII), 1984, pp. 13-21.
I. C. F. Ipsen and Y. Saad
48
, Stable Matrix Computations in VLSI, P1i.D. Thesis, T h e Pennsylvania State University, 1983. ,. A Parallel QR Method Using Fast Givens’ Rotations, Research Report 299, (411 Dept Computer Science, Yale University, 1984. [42] Ipsen, I.C.F. and Jessup, E., Solving the Symmetric Tridiagonal Eigenvalue Problem on the Hypercube, Technical Report, Dept Computer Science, Yale University, -. Technical Report in Preparation. [43] Ipsen, I.C.F., Saad,Y. and Schultz, M.H., Complexity of Dense Linear System Solution on a Multiprocessor Ring, Lin. Alg. Appl., (1986). [44] Johnsson, S.L., Odd-Even Cyclic Reduction on Ensemble Architectures., SIAM J . Sci. Stat. Comp, (1986). , A Computational Array for the QR-Method, Penfield, P. ed., Proc. Conference [451 on Advanced Research in VLSI, 1982, Artech House, Inc., 1982, pp. 123-9. [46] Johnsson, S.L., Saad, Y. and Schultz, M.H., The Alternating Direction Algorithm on Multiprocessors, Research Report 382, Dept Computer Science, Yale University, 1985. 1471 . . Jordan, H.F., A Special Purpose Architecture for Finite Element Analysis, Proc. Int. Conf. Parallel Processing, 1978, pp. 263-6. [48] Ihishnakumar, A.S. and Morf, M., A Tree Architecture for the Symmetric Eigenproblem, Proc. 27th Annual Symp. of SPIE, 1983. 1491 Kuck, D.J. and Sameh, A.H., Parallel Computation of Eigenvalues of Real Matrices, Proc. IFIP Congress 1971, North Holland, Amsterdam, 1972, pp. 1266-72. [SO] Kung, S.-Y., Arun, K.S., Gal-Ezer, R.J. and Bhaskar Rao, D.V, Wavefront Array Processor : Language, Architecture, and Applications, IEEE Trans. Comp., (2-31 (1982), pp. 1054-66. [51] Kung, H.T. and Leiserson, C.E., Systolic Arrays (for VLSI), Sparse Matrix Proceedings, SIAM, Philadelphia, PA, 1978, pp. 256-82. [52] Lawrie,D. and Sameh, A.H., On Jacob: and Jacobi-like Algorithms f o r a Parallel Computer, Math. Comput., 25 (1971), pp. 579-90. [53] Lawrie, D. and Sameh, A.H., The Computation and Communication Complexity of a Parallel Banded Linear System Solver, ACM TOMS, 10 (1984), pp. 185-95. (541 Luk, F.T., A Jacobi-Like Algorithm f o r Computing the QR-Decomposition, Technical Report TR-84-612, Dept Computer Science, Cornell University, 1984. (551 , A Triangular Processor Array f o r Computing the Singular Value Decomposition, Technical Report TR-84-625, Dept Computer Science, Cornell University, 1984. [56] Modi, J.J. and Pryce, J.D., Eficient Implementation of Jacobi’s Method on the DAP, Technical Report CUED/f-CAMS/TR.238, Dept Engineering, Cambridge University, 1982. [57] Moldovan, D.I., Wu, C.I. and Fortes, J.A.B, Mapping an Arbitrarily Large QR Algorithm into a Fixed Size Array, Proc. 1984 Conf. Parallel Processing, 1984, pp. 365-73. [58] O’Leary, D.P., Systolic Arrays f o r Matrix Transpose and other Reorderings, Technical Report 1481, Dept Computer Science, University of Maryland, 1985. (591 O’Leary, D.P and Stewart, G.W., Data-FLow Algorithms f o r Parallel Matrix Computations, Technical Report 1366, Dept Computer Science, University of Maryland, 1984. [60] Ortega, J.M. and Voigt, R.G., Solution of Partial Differential Equations on Vector and Parallel Computers, SIAM Review, 27 (1985), pp. 149-240. [401
~
~
~
Parallel Architectures and the Solution of Eigenvalue Problems
49
(611 Saad, Y., Communication Complexity of the Gaussian Elimination Algorithm on Multiprocessors, Research Report 348, Dept Computer Science, Yale University, 1985. ’ . and Sameh, A.H., A parallel Block Stiefel Method for Solving Positive Definite [62] Saad, 1 Systems, Schultz, M.H. ed., Proc. Elliptic Problem Solver Conf., Academic Press, 1980, pp. 405-12. [63] Saad, Y., Sameh, A.H. and Saylor, P., Solving Elliptic Difference Equations on a Linear Array of Processors, SIAM J. Sci. Stat. Comp., 6 (1985), pp. 1049-63. [64] Saad, Y. and Schultz, M.H., Alternating Direction Methods on Multiprocessors : A n Extended Abstract, Research Report 381, Dept Computer Science, Yale University, 1985. , Topological Properties of Hypercubes, Research Report 389, Dept Computer 1651 Science, Yale University, 1985. , Direct Parallel Methods f o r Solving Banded Linear Systems, Research Report 1661 387, Dept Computer Science, Yale University, 1085. , Data Communication in Hypercubes, Research Report 428, Dept Computer 1671 Science, Yale University, 1985. , Parallel Implementations of Preconditioned Conjugate Gradient Methods, ReI681 search Report 425, Dept Computer Science, Yale University, 1985. [69] Sameh, A.H., Numerical Parallel Algorithms - A Survey, Lawrie, D. and Sameh, A.H. ed., High Speed Computer and Algorithm Organization, Academic Press, 1977, pp. 207-28. , An Overview of Parallel Algorithms in Numerical Linear Algebra, Proc. le' 1701 Colloque International sur les Methodes Vectorielles et Paraleles en Calcul Scientifique, Bulletin de la Direction des Etudes e t Recherches, Serie C, Electricite De France, 1983, pp. 129-34. , On Some Parallel Algorithms on a Ring of Processors, Technical Report , Dept (711 Computer Science, University of Illinois at Urbana-Champaign, 1984. 1721 Sameh, A.H. and Kuck, D.J., A Parallel QR Algorithm f o r Symmetric Tradiagonal Matrices, IEEE Trans. Conip., C-26 (1977), pp. 147-53. [73] Sameh, A.H. and Wisniewski, J.A., A Trace Minimization Algorithm for the Generalized Eigenvalue Problem, SIAhl J. Num. Anal., 19 (1982), pp. 1243-59. [74] Schreiber, R., Systolic Arrays for Eigenvalue Computation, Proc. SPIE 341 (Real Time Signal Processing V), 1982, pp. 27-34. , On the Systolic Arrays of Brent, Luk and Van Loan for the Symmetric Eigen1751 value and Singular Value Problems, Technical Report TRITA-NA-8311, Dept Numerical Analysis and Computer Science, Royal Institute of Technology, Sweden, 1983. , A Systolic Architecture for the Singular Value Decomposition, Proc. ler [761 Colloque International sur les Methodes Vectorielles e t Paraleles en Calcul Scaentifique, Bulletin de la Direction des Etudes et Recherches, Serie C, Electricite de France, 1983. [77] Schultz, M.H., Multiple Array Processors f o r Ocean Acoustic Problems, Research Report 363, Dept Computer Science, Yale University, 1985. I781 Seitz, C.L., The Cosmic Cube, CACM, 28 (1985), pp. 22-33. 1791 Shapiro, H.D., Theoretical Limitations on the Eficient Use of Parallel Memories, IEEE Trans. Comp., C-27 (1978), pp. 421-28. [80] Slotnick, D.L, Unconventional Systems, AFIPS Conf. Proc. SO,1967, pp. 477-81. 1811 Wisniewski, J.A., A Parallel Algorithm f o r Solving A z = XBz,Ph.D. Thesis, University of Illinois at Urbana Champaign, 1980.
This Page Intentionally Left Blank
Large Scale Eigenvalue Problems J. Cullum and R.A. Willoughby (Editors) 0 Elsevier Science Publishers B.V. (North-Holland), 1986
51
COMPUTING THE SINGULAR VALUE DECOMPOSITION ON A RING OF ARRAY PROCESSORS Christian Bischof Charles Van Loan Department of Computer Science CorneU University Ithaca, New York 14853
The parallel matrix computation community has lately devoted considerable attention to the Jacobi family of methods for computing eigenvalues and singular values. These methods "map" rather neatly onto various nearest neighbor architectures. If the processors are reasonably powerful and have significant local memory then block Jacobi procedures are appealing because they render a more favorable computation/communication ratio.
We examined the scope of these claims by implementing a block Jacobi SVD procedure on the IBM Kingston Loosely Coupled Array Processor (LCAP) system. The LCAP system consists of ten FPS- 164/MAX array processors connected in a ring via some large bulk memories. Our basic finding is that the algorithm is well-suited to the architecture but that its advantage over a single processor implementation of the Golub-Reinsch procedure is rather unclear.
1. BLOCK JACOB1 SVD
The singular value decomposition (SVD) of a matrix A
E
RmX"(m 2 n) has many impor-
tant applications. In the SVD, real orthogonal matrices U (m
x
m) and V (n
x
n) are sought
T
such that U AV = diag(u,, _..,uJ. See Golub and Van Loan [1983]. In this paper we report
on an implementation of a block generalization of the parallel Jacobi scheme in Brent, Luk, and Van Loan [ 19851.
Assume for simplicity that we have a partitioning of the form
52
C. Bischof and C. Van Loan
(1.1)
wher
A
gi
=
. .
. .
Aij
E
R
mi x nj
2 nj for all i and j.
Jacobi procedures proceed by making A increasingly diagonal by solving a judicious sequence of (i,j) subproblems. Solving the (i,j) subproblem means finding orthogonal U, and V,
(of appropriate dimension) so that if
then
where p
E
[0, 1) is called the "subproblem parameter". It dictates how much the subproblem
is diagonalized. When p = 0, a full SVD is computed.
Another aspect of the subproblem solution concerns the "threshold". Subproblem (i,j) is "skipped" if Aij and Aji are too small, according to a threshold parameter
7
>
0, i.e., if
This manuever is critical for formal convergence.
Once the (i,j) subproblem is solved, the matrices U, and Vo are appropriately applied to A:
The J matrices are block Jacobi rotations in block planes i and j. To illustrate the notation suppose that
53
Singular Value Decomposition on a Ring of Array Processors
J(i,j,Q) designates the k-by-k block orthogonal
is orthogonal, and Q i , and QZ2are square.
matrix that is the identity with Q inserted in block positions (i,i), (i,j), (j,i), and (j,j), e.q.,
With each update the matrix A becomes more block diagonal in the sense that the Frobenius norm of the off-diagonal blocks,
is reduced. In particular, it can be shown that
An important feature of any Jacobi procedure concerns the order in which the subproblems are solved. One approach is to specify an ordering
of the set {(i,j)
until
OFF(A)
I 1 5 i <j 5 k is
small
and choose the subproblems by cycling through i 7 repeatedly enough.
Well-known
is
the
row
ordering,
i.e.,
(1,2),( 1,3),...,(l,k),(2,3) ,...,(2,k),...,(k-1,k). We will be more concerned with something called the "parallel" ordering which we discuss in the next section.
With these concepts and notations we have
C Bischof and C. Van Loan
54
Algorithm 1.1 (Generic Block Jacobi SVD)
Suppose A
E
RmX"is partitioned according to (1.1) and that (i,, j,), ...,(i,, j,) is an enu-
meration of the set {(i,j)
1 I: i < j I: k]. Given termination criteria
I
< E 11 A 11 ,/a, and subproblem parameter p
parameter T that satisfies T algorithm
computes
OFF(UTAV) 5
E
e
orthogonal
U (m
matrices
x
m)
and
> 0, positive threshold E
[O, l ) , the following
V (n
x
n)
such
that
11 A 11 F. A is overwritten by UTAV.
Algorithm 1.1
u
v
I,,
+
I,
+
> E ll A /I F)
While ( O W N
Forr= 1 : K (id *
(4,j,)
+
If ( IIAij 1;
11 Aji 1I
2
71 '1
All;
then
Generate U, and V, by solving subproblem (ij). else
u,
+
I,
v,
+
I.
endif
Set J, = J(i,j, U,) and J, = J(i,j, V,) and performs updates A
+
JYA, A * AJ,, U
+
UJ,,, and V
+
VJ2
end end
The body of the while-loop is called a block swep. Normally, a few block sweeps are required for termination. An easy proof of convergence may be found in Van Loan [1985].
Jacobi methods, particularly for the symmetric eigenvalue problem, have a long history. See Jacobi [1846], Henrici [1958], Hansen [1962], Schonhage [1964], and Rutishauser [1966]. Further discussion and references appear in Golub and Van Loan [1983, p.295ffI. Block Jacobi methods are analyzed in Hansen [1960].
Singular Value Decomposition
oti a
Ring of Arraji Processors
55
In Section 2 we develop a parallel version of Algorithm 1.l. The practical details associated with this parallel scheme and our implementation experiences on the IBM Kingston LCAP system are detailed in Sections 3 and 4. Some general remarks are offered in Section 5 .
We mention in passing that just about everything in this paper applies to the symmetric eigenvalue problem -- just assume in the sequel that A = AT and identify the matrix U with V.
2. BLOCK JACOB1 SVD WITH PARALLEL ORDERING
A parallel version of Algorithm 1 . 1 can be obtained if "nonconflicting" subproblems are solved concurrently. To illustrate what we mean by nonconflicting, assume that k = 8 and that we have four processors P I , .._,P4 with A apportioned among them as follows:
Note that processor i can generate the U, and V, associated with subproblem (2i-l,2i). Ignoring communication details for the moment, assume that A is updated by these 4 pairs of orthogonal matrices. What do we do next?
This is where the parallel ordering comes in. Partition the set of 28 off-diagonal index pairs into seven rotation sets as follows:
56
C Bischof and C. Van Loan
Read left to right, top to bottom, the above is an instance of the parallel ordering. This ordering can be easily derived by imagining a chess tournament among 8 players in which each players plays every other player exactly once. In between rounds (rotation sets) the players (block columns) move to adjacent tables (processors) in musical chair fashion:
etc.
To ready the processor array for subproblems (1,4), (2,6), (3,8), and (5,7) , we must reapportion A as follows:
51
Singular Value Decomposition on a Ring of Array Processors
PI
All
A64
A31
A34 A81
As1
A54
All
All
Notice that solving the "current" (1,2), (3,4), (5,6), and (7,8) problems is equivalent to solving the (1,4), ( 2 , 6 ) , (3,8), and (5,7) subproblems of the unpermuted A
-- precisely
what
we are supposed to do.
Now for some generality. P,,
... , P,.
Suppose k is even and that we have N = k / 2 processors
Let
A = [ A , , ... , Akl
Ai
Rmxni
be a column partitioning of A. If
is a conformable block column partitioning of the n-by-n identity, and we define the permutation
then An, permutes the block columns of A according to the parallel ordering. Let II, be the corresponding m-by-m permutation with the property that n f A permutes the block rows of A according t o the parallel ordering. With this notation we see that we can proceed from rotation set to rotation set on our processor array by performing the update
C. Bischof and C. Van Loan
58
where the symbol + means "assign block columns 2i-1 and 2i of the matrix prescribed by the right hand side to processor Pi.
We can now specify a parallel block Jacobi procedure. The algorithm is
column oriented"
and it will be handy to let U = [U,, ..., U,] and V = [V,, ..., vk] be block column partitionings of U and V with Ui
Suppose A = [A,,
E
Rmxmi and Vi
..., Ak], U
=
E
Rnx ni for i = 1.
[U,, ..., U,] = I,, and V = [V,, ..., vk] = I,. Assume
that we have N = k/2 processors and that processor i contains block columns 2i-1 and 2i of A, U, and V. Given OFF(U~AV)I
E
>0
the following algorithm computes orthogonal U and V such that
1 A 11 F.
Algorithm 2.1
While (OFF(A)
> E II A II F).
For rotationset = 1 : k-1
Processor Pi (i = 1:N) does the following: Solves subproblem (2i-l,2i) thereby generating orthogonal U(i) and V(i). Broadcasts U(i) to P,,
..., Pi-l, Pi+,, ..., PN.
Receives U( l), ..., U(i - 11, U(i
+ I>, ..., U(N).
Performs the updates:
,,
[A2i-
W 2 i - 1,
V,il
+
+
[U2i-19 U2il * [A2i- 1, A2il
diag(U( 1),...,U(N))T[A2i-l, A2ilV(i) [V2i- 1 , V,iIV(i) UJ2i-17
U,iIU(i)
T
+
nrn[A2i- 1, A2il
Global communications: AeAP UeUP VeVP end end
It is assumed that a threshold criteria T and a subproblem parameter p are part of the subproblem solution procedure.
Singular Value Decornpositiorl
oil
a Ring o f Array Processors
59
With this breezy development of the parallel block Jacobi SVD algorithm, we are ready to
look at some important practical details.
3. SOLVING THE SUBPROBLEMS
In the typical subproblem we are presented with a submatrix
n2
"1
and must choose orthogonal U, and V, such that
satisfies 2
IIB~~IIF +
(3.1) for some fixed p
2
IIB2, IIF I
2
P
2
[ I I A ~ ~ I I+ F
2
II&i IIF I
< 1. We mention two distinct approaches to this problem.
Method I. (Partial SVD via Row-Cjzk Jacobi)
Use the row cyclic Jacobi procedure (Algorithm 1.1) to compute U, and V, such that (3.1) holds, That is, keep sweeping until A,, is sufficiently close to 2-by-2 block diagonal form. What is nice about this approach is that it can exploit the fact that the subproblems are increasingly diagonal as the overall iteration progresses. A, more diagonal implies fewer sweeps needed to satisfy (3.1). On the other hand, Jacobi requires square matrices and so A, may have to be padded with zero columns. There are some subtlties associated with this, see Brent, Luk, and Van Loan [1985].
C Bischof and C. Van Loan
60
Methud 2. (Golub-Reinsch SVD with BidiagonalizutionPause)
-
-
x x o o o o o o o x x o o o o o o o x x o o o o U;A,V,
(nl=ml=n2=m2=4)
0 0 0 x b 0 0 0 =
o o o o x x x x o o o o x x x x o o o o x x x x o o o o x x x x -
-
where U, and VB are products of Householder transformations. Note that the "b" entry is all that prevents the reduced matrix from being block diagonal. This suggests that if
I b I is small
enough, then A, is sufficiently close to block diagonal form and we set U, = U, and V, = V,. If
1 b I is too large, we complete the bidiagonalization and proceed with the itera-
tive portion of the Golub-Reinsch algorithm terminating as soon as the absolute value of the current (n,,n1 + 1) entry is sufficiently small.
In contrast to Method 1, Method 2 can handle rectangular problems more gracefully. But note that in the rectangular case, diagonalization of A, does not correspond to block diagonalization: -
-
x o o o O x 0 0 0 0 x 0
ooox 0 0 0 0
-
0 0 0 0 -
The situation is remedied by some obvious permutations of U, and V,.
Singular Value Decompositiow on a Ring of Array Processors
61
After some experimentation we settled on the following subproblem procedure (illustrated by the ml = m2 = 4, n, = n2 = 3 case):
Subproblem Procedure
Step 1. Compute orthogonal Q so that
x x x x x x o x x x x x o o x x x x
0 0 0 0 0 0 o o o x x x
o o o o x x o o o o o x ~
0 0 0 0 0 0
This is a slight variation of the LINPACK QR factorization. For long rectangular problems Chan [ 19821 has demonstrated the advisability of preceding the SVD with a QR factorization. The reason for the "split" R has to do with obtaining close-to-identity U, and V, in the final stages of the iteration. We also mention that after the first rotation set is performed all subsequent subproblems have diagonal blocks, a fact that our split QR routine exploits. Step 2. Glue the pieces of R together and compute the SVD.
UT o o o o x x o o o o o x
1
Steps are taken to insure that U, and V, are close to the identity whenever that is possible through permutation. Step 3. Form U, out of Q and U and form V, out of V.
It is perhaps worth dwelLing on the need for close-to-identity transformations. This corresponds to the choosing of small rotation angles in the scalar case. A brief example serves to il-
C. Bischof and C. Van Loan
62
lustrate why this is important. Suppose k=4, and that each block in A is 2-by-2. Suppose that
A had the following form
where
E
X
E
E
E
E
E
E
E
E
X
E
E
E
E
E
E
E
E
X
E
E
E
E
E
E
E
E
X
E
E
E
E
E
E
E
E
X
E
E
E
E
E
E
E
E
X
E
E
X
X
E
E
E
E
X
E
X
X
E
E
E
E
E
X
denotes a small entry. Suppose that the U, and V, for subproblem (1,2) are close to
the 4-by-4 identity but that in subproblem (3,4) U,
= V,
M
[e,, e4, e l , e,]. It then follows that
A gets updated to X
E
E
E
E
E
E
E
E
X
E
E
E
E
E
E
E
E
X
E
E
E
E
E
E
E
E
X
E
E
E
E
X
X
E
E
X
E
E
E
X
X
E
E
E
X
E
E
E
E
E
E
E
E
X
E
E
E
E
E
E
E
E
X
To get ready for the next rotation set, which involves rotations (1,4) and (2,3), the block columns (A
+
and
rows
of
A
are
shuffled
[El, E,, E,, E,IT A [El, E,, E4, E2I) giving
according
to
the
parallel
ordering
Singular Value Decomposition on a Ring of Array Processors
X
E
E
E
E
E
E
E
E
X
E
E
E
E
E
E
E
E
X
E
E
E
E
E
P
E
E
X
E
E
E
E
E
E
E
E
X
E
E
E
E
E
E
E
E
X
E
E
X
X
E
E
E
E
X
E
X
X
E
E
E
E
E
X
63
Thus the position of the non-neglible off-diagonal block remains fixed thereby slowing convergence immeasurably. (We stumbled into these observations when we encountered a fairly small example that required about 40 block sweeps to converge.)
To guard against this we have incorporated a low-overhead heuristic procedure in Step 3 that permutes all large entries in U and V to the diagonal.
4. IMPLEMENTATION ON THE IBM KINGSTON LCAP SYSTEM
The IBM Kingston Loosely Coupled Array Processor (LCAP) system consists of ten FPS-l64/MAX array processors connected in a ring via five bulk memories manufactured by Scientific Computing Associates (SCA). Each bulk memory is attached to four array processors (AP's) and each A P is attached to a pair of bulk memories. This allows for considerable flexibility. For example, six AP's can be allocated to one user and four to another. Communication is via some message passing primitives provided by the SCA system. To the user these primitives look like calls to Fortran subroutines and are processed by a precompiler.
There are two types of communication required b y Algorithm 2.1. Associated with each rotation set is a broadcast. Each A P must send an orthogonal matrix to every other AP in the ring. This is accomplished in merry-go-round fashion. The U(i) are passed around the ring (clockwise for example) and are applied to the housed block columns at each "stop". After the update, the A matrix must be redistributed around the ring. Here, the nearest neighbor topology is particularly convenient.
C Bischof and C. Van Loan
64
By "piggybacking" information on the U(i) as they circulate around the ring important global information such as OFF(A) and E 11 A 11 can be made available to each AP. This, of course, is critical in order to automate termination and the threshold testing.
We have run several examples and have gathered as much timing information a's the LCAP system permits. Our results for a 636
x
96 example run on a 6-processor ring are fairly typical.
Approximately 5 percent of the execution time was devoted to communication. Seven block sweeps were required and the overall performance rate was in the neighborhood of 3.6 Mflops. If is impossible to compare this with a single processor run as the problem would not fit into the fast memory of a single AP. However, it is clear that we are not achieving significant speed-up as the FPS-164 (without MAX boards) has a peak performance rating of 11 Mflops. (The AP's in the LCAP system are each scheduled to have two MAX boards, but these were not available at the time of our benchmarks.)
Since the block Jacobi procedure is rich in matrix-matrix multiplication, we expect more favorable performance when the LCAP MAX boards are in place. It is clear that communication costs diminish as problem size grows. When the m and n are several hundred, communication costs are quite insignificant.
5. CONCLUSIONS
The problem, as we see it, concerns the algorithm itself. In terms of the amount of arithmetic, two block sweeps is equivalent to one complete Golub-Reinsch algorithm. Thus, it is critical that the number of block sweeps be kept to a minimum. The problems we ran require between 6 and
10 block sweeps in order to reduce OFF(A) to a small multiple of
E
11 A 11 where E is the machine
precision. Thus, we suspect that it will be difficult to implement a parallel block Jacobi SVD procedure that has a decided advantage over a single processor LINPACK SVD routine. The situation is even worse for scalar Jacobi procedures where the communication/computation ratio is less favorable.
The only way to rescue block Jacobi is with an ultrafast update procedure. Fortunately, updating is much more rich in matrix multiplication than the subproblem solution procedure. Thus,
Singular Value Decomposition on a Ring of Array Processors
65
our subsequent work on the LCAP system will involve making optimum use of the MAX boards which are tailored to the matrix multiplication problem.
Acknowledgements. We wish to thank Dr. Enrico Clementi of IBM Kingston who invited us to use the LCAP system. We are also indebted to his research group whose cooperation and friendliness made our experiments possible.
6. REFERENCES
[ 11 R. Brent and F. Luk (1985), The solution of singular value and symmetric eigenproblems on multiprocessor arrays, SIAM J. Scientific and Statistical Computing, 6, 69-84.
[2] R. Brent, F. Luk, and C. Van Loan (1985), Computation of the singular value decomposition using mesh connected processors, J. VLSI and Computer Systems, 1,242-270.
[3] T. Chan (1982), An improved algorithm for computing the singular value decomposition, ACM Trans. Math. Software, 8,72-83.
[4] G. Forsythe and P. Henrici (1960), The cyclic Jacobi method for computing the principal values of a complex matrix, Trans. h e r . Math. SOC.,94, 1-23.
[ 5 ] G. H. Golub and C. Van Loan (1983), Mutrix
Computations, Johns Hopkins University
Press, Baltimore, Md.
[6] E. Hansen (1960), On Jacobi methods and block-Jacobi methods for computing marix eigenvalues, Ph. D. Thesis, Stanford University, Stanford, Calif.
[7] E. Hansen (1962), On quasicyclic Jacobi methods, ACM J., 9, 118-135. [8] P. Henrici (1958), On the speed of convergence of cyclic and quasicyclic Jacobi methods for computing the eigenvalues of Hermitian matrices, SIAM J. Applied Math., 6, 144-162.
[9] C.G.J. Jacobi (1846), Uber ein leiches vehfahren die in der theorie der sacular-storungen vorkommendern gleichungen numerisch aufzulosen, Crelle’s Journal, 30, 5 1-94.
[ 101 H. Rutishauser (1966), The Jacobi method for real symmetric matrices, Numer. Math., 16, 205-223.
[ 111 A. Schonhage (1964), On the quadratic convergence of the Jacobi process, Numer. Math., 6,410-412.
66
C Bischof and C. Van Loan
[ 121 C. Van Loan (1985), The block Jacobi method for computing the singluar value decomposition, Technical Report TR85-680, Department of Computer Science, Cornell University, Ithaca, NY 14853.
Large Scale Eigenvalue Problems J. Cullum and R.A. Willoughby (Editors) 0 Elsevier Science Publishers B.V. (North-Holland), 1986
61
QUANTUM DYNAMICS WITH THE RECURSIVE RESIDUE GENERATION METHOD: IMPROVED ALGORITHM FOR CHAIN PROPAGATORS* Robert E. Wyatt Department o f Chemistry and Institute f o r Theoretical Chemistry The University o f Texas Austin, Texas U.S.A. David S.Scott" Department o f Computer Sciences University o f Texas Austin, Texas U.S.A. *Supported in part by grants from the National Science Foundation and the Robert A. Welch Foundation. "Current Address:
Intel Scientific Computers, 15201 N.W. Greenbriar Parkway, Beaverton, OR 97006 U.S.A.
A new approach is presented f o r the computation of quantum mechanical time-dependent transition probabilities in systems w i t h a large number o f states. Following use o f the Lanczos o f the algorithm t o produce a tridiagonal representation perturbed Hamiltonian, a modification o f the QL algorithm is introduced t o compute eigenvalues and the first r o w (only) of the eigenvector matrix o f 1. All o f t h e eigenvalues and eigenvector coefficients are used t o compute transition amplitudes, even though roundoff error during Lanczos recursion causes spurious eigenvalues t o appear. This contrasts w i t h the original version o f t h e recursive residue generation method (RRGM), where a condensed eigenvalue list was produced (via the CullumWillaughby procedure) before computing squares o f eigenvector coefficients. Eigenvalues, residues, and time dependent transition probabilities computed from t h e t w o methods are found t o be equivalent.
(A)
INTRODUCTION Intramolecular energy transfer, molecular multiphoton excitation and dissociation, spectroscopic pump and probe experiments, quantum beats, and multiquantum NMR are a sample o f the diverse chemical phenomena which require time-dependent quanta1 transition probabilities f o r their prediction or interpretation. Until recently, the largest dynamical calculations on Class VI supercomputers were limited t o fewer than l o 3 states. However, with t h e recently developed recursive residue generation
RRGM,
selected calculations o n systems with 104-105states became possible. Currently, the RRGM is being employed t o study multiphoton excitation pathways in small m o l e c ~ l e s , ~ electronic absorption and resonance Raman ~ p e c t r a ,and ~ t o calculate quantities in nonequilibrium statistical mechanics, including time correlation function^.^
R.E. Wyatt and D.S. Scott
68
The RRGM is based u p o n use o f the Lanczos algorithm6.' t o tridiagonalize the matrix representation o f the total (perturbed) Hamiltonian. From eigenvalues o f the tridiagonal matrix, and those o f t h e reduced matrix (one fewer row and column), squares of eigenvector coefficients (residues) are computed. The eigenvalues and residues are then used t o evaluate transition amplitudes and probabilities. A well known feature of the Lanczos method is that spurious eigenvalues, in the terminology o f Cullum and Willoughby,* may be produced, due t o the propagation of roundoff error. Comparison o f the eigenvalue lists from the original and reduced tridiagonal matrices allows one t o eliminate all o f t he spurious eigenvalues.* In the present study, w e propose and test a different route from the tridiagonal matrix t o the eigenvalues and residues. The new method eliminates the need t o compare lists of eigenvalues. All eigenvalues and residues, even spurious ones, are used t o compute transition probabilities. In addition, the new method is conceptually and computationally simpler than the original version of the RRGM. Over the past fifteen years, recursion methods have been widely used for a number of problems in quantum physics. Prominent among these studies are applications t o the electronic structure and optical properties o f disordered solids. In fact, the RRGM was largely inspired by work during t h e 1970s of the Cambridge solid state physics group.9-'' In Sec. II, the computation o f state-to-state transition amplitudes is reduced t o the evaluation of survivabilities o f t w o different initial states in linear chains. Evaluation o f the chain propagators, using a modification o f the QL a l g ~ r i t h m ' ~ -is' ~presented , in Sec. Ill. Numerical aspects o f b o th th e new method and the original RRGM are described and compared in Secs. IV and V. Results from th e computational study of a model but realistic molecular system are in Sec. VI and a su'mmary is presented i n Sec. VII.
II. DYNAMICS OF LINEAR CHAINS
For times
tcO,
t he quantum system is defined by a Hamiltonian operator Ho and has
stationary eigenstates Im), In), .._with eigenenergies EO,, Eon __.A t time t = 0, a timeindependent perturbation is switched on, such th at the Hamiltonian becomes H = Ho+ V. The states lm), In), ... are no longer stationary and transitions occur between them. Starting from state m at t = 0, this state evolves into the state exp[-iHt]lm) at time t . We say that t he operator exp[-iHt], a time propagator, acts o n Im) and converts it t o the state a t time t. At each time, the evolved state, by design, satisfies the Schrodinger timedependent equation (we w i l l always w o rk in a system of units where 6 = 1): H{e-'Hqm)}=iaa t 6 - ' H t Im)}
Now, at time t, t he amplitude for finding th e evolved state in some other state "n" is just
69
Quantum Dynamics with the Recursive Residue Generation Method
the projection o f exp[-iHt]lm) upon In), An,,l(t) =(nle-'Htlm)
and t h e transition probability is the absolute square o f this, P,,(t) Computing A,,(t) isthe big challenge in chemical dynamics!
= A',,(t)A,,(t)
In order t o compute the time-dependent transition amplitude between initial state m at
time t = 0 and another state n at time t, we first define t w o transition vectors (1)
(n)=(lm)+ ln))/d2 Iv)=(~rn)-~n)~/V"2
The quantum mechanical transition amplitude is then Am,(t) = (nle-
"ltl*n)= [(ule
- IHL
lu) - (vie-
iHt
Iv)1
(2)
Note that the transition amplitude i s the difference between t w o survival amplitudes: for example, (u(exp(-iHt)lu) is the amplitude for surviving in state lu) at time t if we start there at t = 0 If the eigenvectors o f H were known, then each survival amplitude could be easily calculated. Since (we assume an N-dimensional space)
a= I
where Hla) = Eola), t h e survival amplitude in state lu) i s
1 (+jLe-iEat, N
SU(t)= (ule IH7u) = ~
(4)
a= 1
where ( ~ I u )is~referred t o as a residue; it is the residue o f the Green operator (Z-H)-' w i th respect t o vector (u)at th e simple pole E,, (ul
1
N
Z-H Id=
(ulaXalu)
___ (Z - Eo) a= I
The problem wit h this approach is that (for most systems) w e are unable t o calculate the eigenvalues and eigenvectors needed t o evaluate E q . (4). This is because N is very large (>103). In the recursive approach t o computing Su(t), w e define a new basis Iu ,), IUJ, ... , w i t h the starting vector Iu,)= lu) (see Eq. (1)). Given Iu,), the Lanczos algorithm generates a new as t o form a Jacobi tridiagonal matrix representation of H, basis Iuz),Iu3),_._so
R.E. Wyatt and D.S. Scott
“1
Pl
...
0
(5)
...
0
P2
a3
...
where (1, and p, are self-energies and nearest neighbor coupling energies in the onedimensional chain representing H. The diagonal elements or self-energies {a,} are effective masses for ‘pseudo-particles’ in the chain, while t he coupling elements {p,} provide for linkage between nearest neighbor pseudo-particles. A high value for ak impedes the flow of probability (or energy) down the chain, while a high value for pk accelerates this flow. The chain of coupled pseudoparticles is disordered in the sense that the self-energies are all different from one another.
Letting y1 denote the column vector (N elements) representing Iu,), the survival amplitude is simply given by the (1,l) element o f the propagator SU(t)=_ul.exp(-i_Ht).gl
(6)
As a result, the m+n transition amplitude i n Eq. (2) is computed from the difference
between t w o matrix propagators: Amn(t)= t[u;.exp( - it_itbp, -_v:-exp(- igt).v_,I
(7)
In order t o evaluate the t w o survival amplitudes, w e will first focus upon the u-chain, w i th
u2
state vectorsg,, ... Although i s a very large sparse matrix (N rows and columns), if we perform M recursion steps, then A, will be an MxM matrix. In practice, M <
into the chain representation (see Eq. ( 5 ) )
a“9
=Ju
(8)
As a result, the survival amplitude is given in terms of the chain propagator, exp(-iJ,t): SU(t)=uEQ-exp(-ilut)-~tu, (9)
But y,Q, an M columned ro w vector, is just the ’first’ unit vector: @==rl,O,O,
... 0 1 = g
(10)
This arises because Q is a collection o f the m recursion vectors IJ,,y,, ... gm,and glfu,= s,,. As a result, the survival amplitude is the ( 1 , l ) element of the MxM matrix propagator for the u-chain:
71
Quantum Dynamics with the Recursive Residue Generation Method
-1J
Su(t)=(e
(1 1)
t
-I1
and the m+n transition amplitude is given by the difference between t w o chain propagators -Gut
A",Jt)=:{(c
-1.1
),.,--e
t
(12)
-"
Now, let's focus upon the matrix eigenproblem involving just one of the tridiagonal
matrices (of order M, w i th M < N )
-StJS=E=diag(El, -E,, .__, Em),
Then the survival amplitude iseasilv calculated:
o=l
A very important feature o f this equation is that only the first r o w o f 5 is needed If we can efficiently evaluate th e M eigenvalues and the coefficients in the first row of this MxM eigenvector matrix, then the survival amplitude can be easily computed by performing the summation in Eq. (14)
Ill. ALGORITHM FOR RESIDUES
The standard technique fo r computing all o f the eigenvalues of a symmetric matrix is t o use a finite sequence o f orthogonal similarity transforms t o tridiagonalize the matrix followed by the use o f the QL algorithm. The QL algorithm uses a theoretically infinite sequence o f orthogonal similarity transformations which preserves the tridiagonal form and converges t o a diagonal matrix.12-14The resulting eigenvector matrix is the product o f all of t he transformation matrices. If the eigenvectors are desired, then these transformations can be accumulated as they are applied t o the matrix. The details of the algorithm can be found in Chapter 8 o f Ref 14. The QL algorithm costs O(M2) operations t o compute t he eigenvalues b u t also costs O(M3) if all o f the eigenvectors are computed. In our context, the matrix j is already tridiagonal and only the first r o w of the eigenvector matrix (5 in Eq. 13)) is needed. Fortunately, it is possible to obtain the first r o w o f the eigenvector matrix o f j f o r O(M*) operations. To initiate t h e QL algorithm, w e first factor I (now relabeled 1,) into an orthogonal matrix and a lower triangular matrix (with nonnegative diagonal elements): _J ] = 9 ,Lpg;J_, =!I
When these factors are multiplied in reverse order, t h e next Jacobi matrix in t h e sequence is formed: (16)
Jz=L,a,-a;J_,s,
The process is continued by "doggedly iterating."14 At step k, w e have
J, = a,k, so the next tridiagonal matrix is
7
R.E. Wyatt and D.S. Scott
12
(17)
= $4 ls-k
There are several significant features about this iterative process: (1) The tridiagonal form is preserved in the sequence o f 1matrices. (2)The sequence A,, i,, ... ik converges to a diagonal (eigenvalue) matrix
i1”-J2+ ... J k =diag[El, E,,
(3) As Jk becomes diagonal,
... Em’
(18)
5, simultaneously converges t o the eigenvector matrix of J (19)
StJ S =diag(El, .. En,)
-k-l-k
A simple application o f the use o f this algorithm is shown in Table 1.
The initial
tridiagonal matrix is: Table 1. QL Iterations for a 3x3 matrix.
5,
’k
~
2.000
1.000
0.000
0.802
Q k
0.598
0.000
0.802
0.598
0.000
1.000
2.000
1.000
-0.534
0.717
0.447
-0.534
0.717
0.447
0.000
1.000
2.000
0.267
-0.359
0.895
0.267
-0.359
0.895
0.586
0.015
0.000
0.502
0.751
0.429
1.000
0.007
0.000
0.015
2.038
0.229
-0.707
0.071
0.704
-0.007
0.998
0.068
0.000
0.229
3.376
0.498
-0.657
0.566
0.000
-0.068
0.998
0.586
0.000
-0.000
0.500
0.710
0.495
1.000
0.000
0.000
0.000
2.000
0.016
-0.707
0.005
0.707
0.000
1.000
0.005
~~~
0.000
0.016
3.414
0.500
-0.704
0.505
0.000 -0.005
0.586
0.000
-0.000
0.500
0.707
0.500
1.000
0.000
0.000
0.000
2.000
0.000
-0.707
0.000
0.707
0.000
1.000
0.000
0.000
0.001
3.414
0.500
-0.707
0.500
0.000
0.010
1.000
0.586
0.000
-0.000
0.500
0.707
0.500
1.000
0.000
0.000
0.000
-0.707
0.000
0.707
0.000
3.414
0.500
-0.707
0.500
0.000
0.000 0.000
2.000 0.000
1.000 0.000
0.999
0.000 1.000
73
Quantum Dynamics with the Recursive Residue Generation Method
The table shows the 3x3 matrices lk, Qk,and 5, for iteration steps k = 1, 5, 10, 15, and 20. By step 15, all quantities have converged to O(10~3).
In the preceding section, w e emphasized that only the t o p row (sI1,s , ~ ,..., s l m )o f the eigenvector matrix of 1. is needed in order t o compute survlval amplitudes. The QL algorithm, as normally implemented, computes all eigenvectors of 1.; the converged MxM matrix S, isgenerated. However, the algorithm may be modified t o produce only the first r o w of 1. The key observation is th e fact that each new transformation is applied o n the right o f t he current approximation t o the eigenvector matrix. That is, if Q l , Q2,, Q k are the transformations used so far, then the approximation t o the eiqenvector matrix is . . -
(21)
S,=g,Q,..-!3k
wi th the 9 ’ s multiplied in that order: S(new) = S(old).Q(new). Formally, the first r o w of the eigenvector is
elf= (l,O,O,-..,O).Thus, t o obtain the desired row, it is only ~~~
where
necessary t o obtain a
single vector by operating o n the right by each transformation as it is generated. The only extra storage needed is the vector of length M which will hold t h e final result. Applying each transformation costs 4 multiplies and the number o f transformations is a small multiple of M2. (Based on practical experience, W i l k i n s ~ n suggests ’~ that o n the average less than 1.6MZtransformations are needed). In practice, we modified th e efficient EISPACK routine IMTQL2 t o compute only the first
r o w o f 5 (which is named in the routine). Input t o the routine includes: diagonal of A: subdiagonal o f 1: order o f matrix: Output from t he routine includes:
eigenvalues of 1: first r ow of eigenvector matrix:
A(l), ..., A(M) B(1). ._.,B(M-1)
(in array D) (in array E)
M E(1),
..-,E(M)
S(1), .. S(M)
(in array D) (in array 2 )
R.E. Wyatt and D.S. Scott
74
IV. NUMERICALASPECTS
A well-documented feature o f Lanczos recursion is that propagation of roundoff error leads t o a gradual loss o f global orthogonality among the recursion vectors. This feature manifests itself in th e diagonal and off-diagonal elements of 1 such that when eigenvalues are computed, multiple copies o f eigenvalues on the ends o f the spectrum are produced. Depending upon the number o f recursion steps, there may be several copies o f each o f the lowest f e w (and highest few) eigenvalues. In addition, some 'incorrect' unconverged eigenvalues may be imbedded in the approximate spectrum. This eigenvalue list may then be 'purified' by the method of Cullum and Willoughby,s as described in the next section. This is what was done in previous versions of the RRGM; however, there are several alternative approaches. First, the Lanczos vectors may be selectively reorthogonalized as the recursion proceeds.16 This usually requires considerable I/O time, since many long vectors are stored on disk. This approach was not followed here. Second, information from the 'numerically contaminated' 1 matrices (i.e., eigenvalues and squares o f eigenvector coefficients) may be used directly, without purification. Despite the obvious simplicity o f such an approach, a major question needs t o be answered: Are th e resulting transition probabilities accurate? Section VI will provide the answer. V. COMPARISON WITH THE ORIGINAL RRGM The original RRGM and the modified QL methods both use Lanczos recursion t o form the MxM matrixes I, and 1,. However, between this point and the computation of transition amplitudes, the methods differ greatly. In the RRGM, the first r o w and column of I(first -U 1 and then 4") are deleted t o form the (M-l)x(M-1) reduced matrix f . Both matrices (1
and
c)are then sent t o the EISPACK routine TQLRAT, which returns the eigenvalues (one
set for the u-recursion and one set for th e v-recursion). Using the method of Cullum and
Willoughby, the t w o lists o f eigenvalues are compared. If an agreement occurs (within a predefined tolerance), then this eigenvalue is deleted from b o t h lists. From t h e t w o purified lists, residues are then computed. For example, for the u-recursion, let the eigenvalues of I and 1’ be {El} and {E,r}. Then the residue of (2-H)-' a t pole 2 = Ea is computed from the rational fraction',2 Ru(a) =
(E -Ei)....(Ea-ErP- 1)
(23)
(Ea-E,) .... (Ea-Ea-l)(Eo-Ea+,)... (Ea-Ep) '
where p i s the number of purified eigenvalues o f J. Note that there are (p-1) products in both the numerator and th e demonator of Eq. (23). The modified QL aproach differs from th e original RRGM in t h a t all eigenvalues and residues computed from are directly used in computing transition amplitudes. The reduced tridiagonal matrix is not needed, since Eq. (23) is n o t used to compute residues. There is a t least one advantage w i t h the new QL procedure: the CPU time is about 10%
Quantum Dyrzanzics with the Recursive Residue Generation Method
less, since only two diagonalizations are required (for RRGM.
1, and 1") rather than
75
four in the
VI APPLICATION
In this Section, we will compare eigenvalues, residues, and transition probabilities
obtained from the modified QL algorithm with those from the original RRGM The Hamiltonian operator i s that of an anharmonic vibrational mode coupled to four harmonic vibrdtional modes
where at(a) and b,?(b,)are raising (lowering) operators for the anharmonic and harmonic modes, respectively When acting on a (one-dimensional) harmonic oscillator basis function specified by the quantum number n (n = 0, 1.2, ),we obtain a'ln)=ln+ l ) ' l n ) ,
a+aln)=nln).
The parameter values are the same as in our earlier studies (1,2). Basis functions for the five mode system are products of harmonic oscillator functions for the individual modes In) = 1~,)1~,1"2)1~31"4)
These are eigenfunctions of the first three terms in H, which defines H O :
4
The last two terms in Eq. (24) constitute the '.perturbation' V, which causes transitions among the eigenstates of Ha,
1V i ( a t b i + a b t ) + V , ( a +a') A
V=
i=l
The first term in V couples the anharmonic mode with each of the four harmonic 'bath' modes, while the last term couples nearest neighbor states within the anharmonic mode. For the calculations reported here, there are three harmonic oscillator functions in each mode (naor nl = 0 , 1,2), so that the total basis size is N = 35 = 243. Results will be presented ,O,O,O,O). For for the fundamental transition in the anharmonic mode, 10,0,0,0,0)+11 brevity, this will be referred t o as the 0+1 transtion. Both the modified QL and the orginal RRGM calculations were run for M = 100 Lanczos recursion steps. In other calculations not reported here, additional transitions with variable M were examined. These results
76
R.E. Wyatt and D.S. Scott
were in agreement with th e observations reported later i n this Section for the 0+1 transition. We mentioned previously that when eigenvalues are computed from t h e tridiagonal matrix 1, the effects of roundoff error during M steps of the Lanczos recursion leads t o spurious eigenvalues multiple copies o f some eigenvalues, and unconverged eigenvalues which lie between ’good’ eigenvalues This feature is documented on the left side of Table 2, where the lowest 28 etgenvalues of 1, and t h e associated residues (sla2)are listed The lowest eigenvalue is repeated 3 times, the next eigenvalue
(a
= 4) is an unconverged
spurious eigenvalue, and the next 5 eigenvalues each come with one extra copy. Residues computed via the original RRGM are shown in the right column of Table 2. Recall that multiple copies of eigenvalues and other spurious eigenvalues are removed from the eigenvalue list before computing residues with E q (23). I t is very important that the sum o f the residues associated w i t h all multiple eigenvalues (computed directly from 1) equals (within 0(10-6))t he residue produced by the original RRGM. For example, the a = 5 and 6 QL residues add t o 0.2073609,which is within 0.001 % of the original RRGM result. There is clearly a conservation o f residue when the total residue for a given eigenvalue is
partitioned among the multiple copies. If another ’ghost’ joins a set of multiple copies
(by running a f ew more Lanczos recursion steps) then the total residue is just shared among a larger number of nearly degenerate copies. Also note that the residues associated with t he spurious (single copy) eigenvalues are always very small. Cullurn has a proof of this property.” As a result, there is no need t o d o any processing (removal of multiple copies, etc.) on th e modified QL energies and residues before computing time dependent transition amplitudes.
Quantum Dynamics with the Recursive Residue Generation Method
I1
Table 2. Comparison of Residuesfor 0+1 transitiona.
I
RRG M/QL a Ea
residue
1
-0,0036497
0.5605904
2
-0.0036497
0.0000027
3
-0.0036497
4
0.00658 19
5
I
residue sum
E,
residue
0.0000000
0.5605931
-0.0036497
0.5605961
0.00 00000
(s pu r io us)
0.2059913 0.00 13696
0.2073609
0.9280066
0.2073629
0.00205985
0.9748432
0.0205987
0.0376203
0.9991011
0.0376207
0.0059123
1.0214444
0.0059124
I
1.0582480
I 0.1643563
I
6
0.9280066 0.9280066
7
0.9748432
0.0205747
0.9748432
0.0000238
0.999101 1
0.0376203
10
0,999101 1
0.0000000
11
1.0214444
0.0059122
12
1.0214444
0.0000001
13
1.0582480
0.1 64 1047
I
14
I
1
15
I
Original RRGM
1.0582480
I
0.0002497
1.8578429
I
0.0010065
I -
0.1643544
I -
1.8578429
I 0.0010066
16
1.9061222
0.0001859
1.9061222
0.0001 860
17
1.9300472
0.0003244
1.9300472
0.0003244
18
1.9529915
0.0000598
1.9529915
0.0000598
19
1.9540135
0.0000005
20
1.9775073
0.0000497
1.9775073
0.0000497
21
1.9867233
0.00121 19
1.9867233
0.00 12 122
22
2.001 5622
0.0000179
2.0015622
0.0000179
23
2.0241 172
0.0000068
24
2.0364177
0.000 1049
2.0364177
0.0001049
25
2.0463256
0.0000006
E
(spurious)
(spurious) (spurious)
2.0604025
0.0001857
2.0604025
0.0001857
2.0832503
0.0000287
2.083 2503
0.0000287
2.1187666
0.0003770
2.1 187666
0.000377C
a l O O recursion steps were used for the 243 state problem. Eigenvaluesand residues above a = 2 8 were computed, but are not listed here. The initial recursion vector was IU,) = (10)+11))"2.
R.E. Wyatt und D.S. Scott
78
The time dependence o f the 0+1 transition is shown i n Table 3 for O s t l l ns. Table 3. Time dependence of 0+1 transition probability.a
1
t(nsIb
RRGM/QL
Original RRGM
0 .o
0.0000000
0.0000000
0.1
I
I
1-
r1
0.3 0.4
0.5 0.6 0.7 0.8 0.9 1.0
0.0028334
I
0.0003311
0.2
I
I I
0.0047330 0.0006065
I
0.0099961
I
0.0046035
1
0.000331 2
0.0049393 0.001 1585 0.0075851
0.0033699
0.0028335
I
0.0047330 0.0006065 00049393 0.0011584 0.0075853
I
0.0099965
I
0.0033699 0.0046035
1
aProbabilitiesfor other times agree as well as those shown here b~ ns = 1 0-9sec. We note that 1 ns is a long time o n the molecular scale; about lo4 periods of vibrational motion are encompassed. The RRGM probabilities (right column) were computed w i t h 18 eigenvalueslresidues (the residue sum exceeded’ 0.999),while the modified RRGM/QL results were obtained w i th all 100 eigenvalues/residues (including all spurious eigenvalueslresidues). It is encouraging that th e t w o lists of probabilities agree t o within about 1 part in
lo .
VII. SUMMARY The accurate computation o f quantum mechanical time-dependent transition amplitudes recently became possible for much larger systems than had been previously considered. In the original RRGM, before residues (squares of eigenvector coefficients) could be computed, spurious eigenvalues (in the terminology of Cullum and Willoughby) had t o be removed. This was done by comparing eigenvalues of the tridiagonal matrix, formed by t h e Lanczos recursion method, w i t h those of the reduced matrix formed by deleting the first r ow and column from the tridiagonal matrix. A different procedure was introduced i n this study. Emphasis was placed upon directly
computing the survival amplitude o f the first state in each of t w o linear chains (each
Quantum Dynamics with the Recursive Residue Generation Method
19
chain state is coupled only t o its nearest neighbors). Each survival amplitude is of the form (exp(-iJt)),,,, where is the tridiagonal representation o f the perturbed Hamiltonian (HO + V) in t he chain (recursion) basis. This representation is generated through Lanczos recursion, starting separately from each o f th e transition vectors ([m) 2 ln))/d2, where m and n lable (unperturbed) eigenstates o f HO. In order to evaluate the (1,l) matrix element of the chain propagator exp(-iJt), w e modified the Q L algorithm t o produce the eigenvalues (E,} of 1and only the elements in r o w one of its eigenvector matrix {s,,}. Due t o roundoff error in th e Lanczos recursion, some multiple eigenvalues appear, w i t h the concomitant splitting of the exact eigenvector coefficients over the (nearly) degenerate eigenvalues. In spite o f this splitting, the sum of the squares of the eigenvector coefficients over all members of a degenerate set is conserved -- degenerate eigenvectors share the strength. As a result, transition probabilities computed from the QL eigenvalues and residues is,,} agree w i t h those from the original RRGM. Acknowledgements We thank Professor B. N. Parlett, Dr. Jane Cullum and Dr. Ralph Willoughby for helpful com ments. REFERENCES
[l]
[2] [3] [4] [5] [6] [7]
Nauts,A. and Wyatt, R. E., Phys Rev. Lett. 51 (1983)2238. Nauts, A. and Wyatt, R. E., Phys. Rev. A30 (1984)872. Chang, J and Wyatt, R. E., t o be published. Friesner, R. A. and Wyatt, R. E., t o be published. Friesner, R. A . and Wyatt, R. E., J. Chem. Phys. 82 (1985)1973. Lanczos, C., J. Res. Natl. Bur. Stand. 45 (1950)255. Cullum, J. K. and Willoughby, R. A., Lanczos Algorithms for Large Symmetric Eigenvalue Computations, Vol. 1, Theory (Birkhauser, Boston, 1985).
[8] Cullum, J. K. and Willoughby, R. A,, J. Comp. Phys. 44 (1981)329. [9] Haydock, R. in Thorpe, R. (ed.), Excitations in Disordered Systems (Plenum, New York, 1981). [lo] Haydock, R.,Comput. Phys.Commun. 20 (1980)1 1 . [ 111 Haydock, R., Solid State Phys. 35 (1980)2 15. [12] Francis, J. G. F., Comp. J4(1961)265,332. 113) Parlett, B., SIAM Rev. 6 (1964)275. [ 141 Parlett, B. N., The Symmetric Eigenvalue Problem (Prentice-Hall, Englewood Cliffs,
1980). [15] Seep. 162 in ref. 14. [16] Parlett, B. N. and Scott, D. S., Math. Comp. 33 (1979)217. [17] Cullum, J. K., private communication, July, 1985.
This Page Intentionally Left Blank
Large Scale Eigenvalue Problems J . Cullurn and R.A. Willoughby (Editors) 0 Elsevier Science Publishers B.V. (North-Holland), 1986
81
EIGENVALUE PROBLEMS AND ALGORITHMS IN STR UCTUR AL E NGINEERING Roger G. Grimes John G. Lewis Horst D. Simon Boeing Computer Services Co. Tukwila, Washington, U.S.A.
Structural engineering is a predominant source of sparse eigenvalue problems. Yet the numerical analysis literature rarely addresses t h e full generality o f the algebraic characteristics of these applied problems. We describe some o f the more demanding structural engineering eigenvalue problems and the difficulties they pose fo r numerical algorithms.
INTRODUCTION To most numerical analysts the sparse generalized eigenvalue problem AX = BXA comes in t w o forms. One case is the generalized symmetric eigenproblem, i n which A and B are symmetric, w i t h B positive definite. Efficient and robust methods exist for such problems. The other case represents all other problems, which w e treat as generalized unsymmetric eigenproblems, for which our methods are much less efficient and less robust. The eigenvalue problems that arise i n structural engineering are often thought o f as falling int o th e first class of problems, when in fact they fall into the second. However, the most important problems are generalized symmetric, even though the matrix that plays th e role o f B is n o t positive definite. The major emphasis o f this
82
R. C. Grimes et al.
paper is t o describe ways i n which w e can adapt our standard numerical analysis methods t o solve these problems well.
We also present important structural
engineering eigenvalue problems that are n o t generalized symmetric and do not yet have adequate solutions. We begin w i t h a brief overview o f th e origins of the t w o most common classes of structural engineering eigenvalue problems. In doing so w e highlight the algebraic conditions expected i n these problems. This standard material is followed by an outline o f the recent, b u t n o w standard, numerical treatment of the usual vibration problem. Even i n the most restrictive problem we find difficulties n o t yet fully resolved numerically. In the subsequent section w e present a number o f ways in which the standard conditions on t h e symmetric generalized eigenprobtem are relaxed in engineering applications. We also present a number o f practical considerations that must be faced by a numerical analyst implementing algorithms within the constraints o f structural engineering packages. The relaxed conditions and the computing environment present important, often serious, complications. Some of these have reasonable solutions, which are presented cursorily here. Others are difficult open questions, partly addressed by other authors in this proceedings. This presentation is based in large part on our experience in developing a sparse generalized symmetric eigensolver for MacNeal Schwendler Corporation's NASTRAN. We are grateful t o Gary Dilley, Mike Gockel and Louis Komzsik for their support and assistance. Our substantial debt to our numerical analysis colleagues is reflected in the references. ORIGINS OF SPARSE EIGENPROBLEMS IN STRUCTURAL ENGINEERING The analysis o f large structures today usually begins w i t h a finite element model o f the structure. Such models are often o f very high degree; models w i t h tens of thousands of variables are n o t uncommon. W e will discuss t w o common engineering problems that lead t o sparse eigenvalue problems. The most common eigenproblem is vibration analysis, which simplifies use o f the dynamic equations o f motion t o model a structure subjected t o a time-varying force. The second eigenproblem is buckling analysis, used in approximating the behavior o f a structure when subjected t o constant forces at levels that cause the structure t o collapse.
Eigenvalue Puoblem apid Algorithms in Structural Engirieering
83
The motion o f a structure is generally described by a system o f nonlinear ordinary differential equations. For most engineering applications it is appropriate t o make a number o f simplifying assumptions that lead t o a system of linear ODE'S. The common model for structures is LaGrange's equations o f motion. In typical full form [I], these are: M Y(t)
+
(G
+ C) i ( t ) +
(K
+
H) x(t) = f(t)
Two o f t he five matrices in this equation will be familiar t o numerical analysts: the mass (inertia) matrix M and the stiffness matrix K. These t w o matrices, and the damping matrix C are typically symmetric. From physical assumptions both K and M are positive semi-definite. The other t w o matrices, G, the gyroscopic (Coriolis) matrix, and H, t he circulatory matrix, are skew-symmetric. All are sparse because of the local support o f the finite elements. The forcing function f(t) represents the load on the structure. The dynamic equations o f motion simplify when w e consider the equations for free undamped or natural vibration: M Y(t)
+
K x(t) = 0.
The solution t o this problem is obtained easily from the generalized eigenvalue problem M@A = K@
The eigenvectors @ are the vibration mode shapes o f the structure, the eigenvalues hi are the squares o f the natural frequencies mi, and the pair ( hi, @i) is a normal mode o f the structure. Structures exhibit a infinite sequence o f natural frequencies, w i th a limit point at infinity. The finite element models approximate a finite number o f the actual natural modes. The higher frequency approximations are unimportant for t w o reasons: they are the most inaccurate, being most affected by discretization error, and they are unlikely t o be excited in applications like earthquake excitation of buildings. The forcing functions in such applications excite only l ow frequencies; the response o f the higher natural frequencies o f the structure can be represented by simplified approximations. I t follows that the (free) vibration eigenproblem is a sparse generalized symmetric problem, for which only a few, typically the lowest, eigenpairs are sought.
84
R. G. Grimes et al.
Often the engineer can make sufficiently good approximations from the free vibration analysis alone t o understand the motion of the structure. There may be no need t o actually integrate the differential equations, and thus the solution of the eigenproblem is an end result. In other cases the engineer must continue with an analysis of the differential equations. The importance of the free vibration eigenproblem to this analysis is that the eigenvectors @ uncouple simplified equations of motion and that using only a subset of the eigenvectors reduces the order of the problem. For example, under the change of variables
@y = x, the equation
becomes
..y(t)
+ Ry(t) = aTf(t).
Then this decoupled system, which has order n, the number of equations in the original system, can be reduced to a much smaller system by retaining only the equations corresponding t o the p most significant normal modes. The number of significant normal modes is typically quite small. Thus, knowledge of the engineering problem results in a much simpler truncated system of order p < < n. However, it is unlikely that this simple model would be appropriate for real applications. Some more complete models lead t o problems with similarly good numerical behavior. For example, the eigenproblem associated with damped vibration is a quadratic eigenproblem with complex eigenvalues. But it often suffices t o model damping with a matrix that is simultaneously diagonalized with K.and M. This retains the uncoupling properties of the vibration problem and can be realized by choosing some form of proportional damping: C = aK
+
PM
or
C = @ZQTfor somediagonal matrix C
Eigenvalue Problems and Algorithms in Structural Engineering
85
Gyroscopic effects are very important t o t he analysis of rotating shafts. Such problems are relatively uncommon, b u t models th at d o n o t include damping permit a related 2n by 2n generalized symmetric eigenproblem t o be derived. See [ l l for details. Other models d o n o t result i n generalized symmetric eigenproblems. In many o f these cases t he solution o f the vibration problem is used t o provide a reduction i n order f o r the differential equations even though the equations are not decoupled. There are yet other problems for which the solution of an unsymmetic generalized An area o f particular importance in aerospace eigenproblem i s important. applications is structures interacting w i t h control systems. These lead t o large sparse unsymmetric generalized eigenproblems. The stability problem is as important here as it is in other control applications, b u t t h e matrices are of much larger order. With insufficient modeling and numerical tools these and other engineering problems remain unsolved at the present. The buckling eigenproblem arises i n a different engineering context. The usual displacement models fo r constant forces result in the static equation K U = f,
where K is again the stiffness matrix, f the (constant) external force, and u the resulting displacement o f the structure. The static equation is derived o n the linear assumption t hat t h e stiffness o f the structure does not depend o n the force. In the analysis of buckling structures, the stiffness is obviously in a nonlinear regime. The simplest nonlinear analysis o f buckling is Euler or bifurcation buckling. Assume that t he stiffness depends on th e direction in which the force is applied, and consider a first order Taylor’s expansion of the (nonlinear) stiffness matrix as (K
+
AKg(f))u
= hf,
where Kg (f) is t h e differential stiffness matrix, which depends on the structure and on (the spatial distribution of) th e applied load f. Buckling occurs whenever the generalized stiffness matrix (K + AKg (f)) is singular, that is, whenever A is an eigenvalue o f
Here A is the factor o f safety for the applied load f and shape.
I#I
is the buckling mode
R.G. Grimes et al.
86
The buckling eigenproblem differs from the vibration problem in several respects. Although K is again a positive semi-definite matrix, Kg (f) is a symmetric indefinite matrix. The eigenproblem thus has eigenvalues of both signs. It is usually the case that only the smallest positive root is important, since the higher buckling modes only describe the behavior o f a structure that would have buckled under a lighter load and rarely have engineering meaning. In certain contexts, particularly analysis o f pressure vessels, the load can be thought o f as being applied in either direction (which suggests why Ks (f) is indefinite). In this case both t h e smallest positive and negative roots are significant. Although the higher modes seem meaningless, there are engineering applications in which they are desired. It is usually s t i l l the case that fewer modes are desired in the buckling problem than in vibration analysis. NUMERICAL ALGORITHMS FOR STANDARD PROBLEMS The free vibration problem is the prototype for the standard numerical analysis generalized eigenpro blem, A X = BXA,
where A = K and B = M. However, the practice commonly suggested i n the numerical analysis literature is often inappropriate for t h e vibration problem. The desired eigenvalues are usually the smallest eigenvalues, which lie at one extreme of the spectrum of the standard reduced matrix
where B = L B L B ~is the Cholesky factorization o f B. Many numerical analysts would expect t o use a standard sparse eigensolver, such as the Lanczos algorithm, t o compute the eigenvalues o f C. Unfortunately t he convergence will be unacceptably slow. The small eigenvalues, while at one end o f the spectrum, are very poorly separated. Typically, th e smallest eigenvalues o f C are o f order 10’ and the largest eigenvalues are of order lo8 or larger. The relative separation, which governs the convergence, is o f order lo-’ or smaller. I t has long been known t o structural engineers that it is more effective t o compute the largest eigenvalues o f the inverted reduced matrix
Eigenvulue Problem and Algorithms in Structural Engineering
87
D = LK-’ M L K - ~ ,
which give the reciprocals o f th e eigenvalues o f the vibration problem. The inversion transforms poorly separated eigenvalues into well-separated eigenvalues. The convergence rate is very rapid for the desired eigenvalues. Subspace iteration, the most common algorithm found in engineering packages, uses this transformation. The standard Lanczos algorithm can easily be applied t o this transformed problem, as i n Grimes and Lewis [2]. This approach is n o t an efficient way to solve certain vibration problems. Problems in which relatively large numbers o f eigenvalues are desired will require many iterations t o obtain the higher eigenvalues, whose separation is still relatively poor. Similarly, solving problems in which the desired eigenvalues are not the smallest modes will require first computing all eigenvalues below the desired ones. Such problems arise, fo r example, from analysis of the motions induced by rotating machinery attached t o a structure. The solution t o such problems is t o simultaneously shift and invert the problem. For subspace iteration this is achieved by applying simultaneous iterations t o the operator (K - uM)-’ M. The more efficient Lanczos algorithm requires a symmetric operator. The standard formulation for the Lanczos algorithm uses the spectral transformation o f Ericsson and Ruhe [3]. The eigenvalue problem K @ = M@A is replaced by t he eigenproblem
M ( K - aM)-’M @ = M @ Y ,
where Y is diagonal, w i t h u I = l/(Al-u). In the spectral transformation u is a shift value chosen near the desired eigenvalues. The eigenvalues nearest cs will be wellseparated in the transformed problem. If more eigenvalues are desired than will converge rapidly for th e particular shift u, a sequence of such problems w i t h different shifts can be solved. The spectral transformation appears t o be a complicated transformation o f the problem, b u t in actual implementation what appear t o be additional matrix-vector products disappear. The spectral transformation also possesses several important advantages over the ordinary transformations. See Scott 141 for a thorough discussion.
88
R. G. Grimes et al.
For either subspace iteration or t h e Lanczos algorithm, each shift requires computing a factorization of th e shifted operator K-oM, followed by a sequence o f iterations. each o f which requires solving linear systems w i th the shifted operator. The choice of an appropriate sequence o f shifts {oi} depends on the relative costs o f factorization and solution, and on th e convergence behavior o f the nearby eigenvalues. It should be noted that each factorization can also provide t h e inertia o f the corresponding shifted matrix, thereby adding information on the distribution o f eigenvalues that can add substantially t o the reliability of the solution. Although t h e use of the spectral transformation w i t h the Lanczos algorithm combines high efficiency w i th the significant numerical advantages discussed by Scott, it also introduces a new and interesting mathematical problem. The transformed
Lanczos algorithm operates on an M-self-adjoint matrix by using M-orthogonality rather than the usual Euclidean orthogonality. M-orthogonality is uninteresting in the mathematical model problem where the right hand side matrix is positive definite, but it is quite common in engineering problems for M t o be a semi-definite matrix. Indeed, diagonal M w i t h one-half the diagonal entries zero are common. (It is tempting t o invert the diagonal mass matrix instead of the generally sparse stiffness matrix, b u t that leads t o th e standard, ineffective, reduction.) The presence of.rank deficiencies in M means the eigenvalue problem admits infinite eigenvalues. It also means that M induces only a semi-norm instead o f a norm. For the Lanczos algorithm t o compute the required eigenvectors i t i s necessary for all t he Krylov vectors t o remain in the invariant subspace spanned by the eigenvectors corresponding t o the finite eigenvalues of the pencil. Nour-Omid and Parlett [ 5 ] provide a mathematical solution t o this difficulty by a taking particular choice o f starting vector. Ericsson [61 discusses the numerical behavior of this solution and suggests still further improvements in the treatment o f this problem. Alternatively, the case o f semi-definite M in the vibration problem could be solved wit h a proper norm by choosing an appropriate scalar a so that K + a M is a positive definite matrix, and substituting this matrix for M i n the spectral transformation. Since both K and M are semi-definite in the vibration problem, any positive a yields a definite linear combination. There would be numerical considerations in choosing a , b u t it should also be noted that this is a solution only for the vibration problem -- it does n o t generalize t o the buckling problem. Indeed, the spectral transformation o f Ericsson and Ruhe does not apply i n the buckling problem because Kg is indefinite. An alternative transformation is required, i n which
Eigenvalue Problems and Algorithms in Structural Engineering
89
is replaced by the eigenproblem
K ( K + oKg)-’K@ = K@Y, where Y isdiagonal, with ui = hi/(hi-a). This generalized spectral transformation is more complicated in t h a t it has a singularity at zero, which is not an allowable shift.
Otherwise, this transformed
problem has essentially the same implementation requirements as the original spectral transformation. More complete details of this transformation will appear in Grimes, Lewis and Simon [7]. It should be noted that the simple inversion of the spectrum often suffices for the buckling problem, although Scott discusses problems that could arise. MORE DIFFICULTIES Structural engineering eigenproblems present two further sets of hurdles. One is that the solution process must take place in the environment of the structural engineering package that produces the eigenproblem. The second set of hurdles are presented by further relaxations of the standard conditions on the generalized symmetric eigenproblern. The eigenanalysis algorithms in structural engineering packages must abide by the software constraints posed by other modules and requirements of the package. The package’s standard databases are used, which restricts the way the required matrix decomposition and matrix-vector multiply operations can be performed. In particular most packages carry out these operations using secondary storage, even in cases in which the data could be stored entirely in primary storage. On the other hand, decompositions or factorizations are expected; it is not necessary, nor effective, t o use factorization-free algorithms for the eigenproblem. The eigenvalue algorithms of interest perform both a matrix-vector product using the M matrix and the solution of a linear system for K - oM during each iteration. There is a significant cost for accessing this data from secondary storage; in many environments this cost may be a substantial fraction of the cost of the arithmetic operations. However, this cost can be reduced b y performing the matrix operations simultaneously for a number of vectors. Thus, the subspace iteration algorithm is used
R. G. Grimes et al.
90
in preference t o ordinary inverse iteration, since the operations can be performed for all vectors in the trial subspace. Similarly, the block Lanczos algorithm [8,9] enjoys an advantage in data access over th e ordinary Lanczos algorithm. Multiple eigenvalues are frequently encountered when the structural engineering software permits t h e easy modeling o f symmetric structures. Such structures should be modeled in their entirety, and thus many o f the natural frequencies occur multiply. The block algorithms, subspace iteration and the block Lanczos algorithm, have much better convergence properties than their single vector counterparts as long as the block size is as large as the multiplicities. The use o f th e spectral transformation and the concomitant strong convergence of a f ew eigenvalues produces several important problems for a block Lanczos algorithm. Among the problems that need t o be solved are: the generalization o f the analysis o f a tridiagonal matrix [lo] t o the block tridiagonal case, treatment of rank deficiency in generated blocks, and efficiently computing M-orthogonal bases when M is stored on secondary storage. Details o f one implementation will be presented in [71. Frequently engineering models cause additional algebraic difficulties. One common engineering modeling requirement is the presence o f "rigid body modes". Modeling a structure t h a t is free to rotate or translate in i t s entirety, like a spacecraft, yields a stiffness matrix that is singular. That is, the resulting eigenproblem has multiple and exactly zero eigenvalues. Yet, it is often reasonable t o find the natural modes even though both the stiffness and mass matrices are only semi-definite. Two difficulties can arise in this situation. The more difficult, b u t uncommon, situation is that the engineer can produce an erroneous model in which the t w o matrices share a common null vector. In this case t h e eigenproblem is n o t well-posed, and any scalar is an eigenvalue w i t h the common null vector as eigenvector. It is n o t at all obvious h o w t o detect this situation in practice. Although the shifted operator should be singular for all possible shifts, there are few means t o detect numerical singularity, particularly because ill-conditioning i s normal for the shifted matrix. In practice, the eigenvalue algorithms appear to compute the eigenpairs of t h e deflated problem in which the offending common nullspace is removed. This is n o t reliable, nor does it provide a useful warning t o t h e engineering user. The Lanczos algorithm could be used t o compute the nullspace o f one o f the matrices, and the intersection o f this space wit h the nullspace o f th e other matrix could be computed using the singular value decomposition [ l l ] . Such a scheme is far more reliable, b u t i t s expense is such that it is likely to be only practical as an after-the-fact analytical tool rather than a general diagnostic tool.
Eigenvalue Problems und Algorithms in Structural Engineering
91
The more mundane problem posed by rigid body modes is a difficulty for the shifting strategy. The most common vibration applications require finding the lowest modes; zero is an excellent initial shift i f there are no rigid body modes. A shift a t zero when rigid body modes are present is useful for computing the rigid body modes, but nothing else. There is a need in such cases t o have estimates of the lowest nonzero eigenvalues for determining shift values.
Heuristic algorithms for such
estimates have been addressed by the engineering community, but usually in program documentation rather than in the open literature. The presence of rigid body modes in the buckling problem can be far more serious than in the vibration problem. Such problems require the generalization of the spectral transformation addressed above (rather than the simple inversion of the spectrum). Since zero is not allowed as a shift for this transformation, there are no complications t o the shifting strategy. The stiffness matrix K would represent only a seminorm, a problem addressed in [ S ] and [7]. However, such problems can fail t o be generalized symmetric, as observed by Ericsson in [7], and this may represent a signif icant problem. There are other numerical difficulties that arise from modeling approximations. The stiffness matrix results from the displacement formulation, which is known t o be a normal equations approach t o a least squares problem. Thus, ill-conditioning in K is a common phenomenon, but one that usually does not pose serious problems for computing eigenvalues. However, certain modeling constraints in the structural engineering finite element package can exacerbate the ill-conditioning of K. For example, it is common t o model very rigid elements in a structure with very large artificial stiffnesses instead of using a model with constraints. This approximation may be imposed by limits t o the capabilities of the package, but it also has the potential t o destroy any validity in the model. We have seen a number of problems where the condition number of K was so large that the matrix would be determined numerically singular to the precision of the computer, and yet where the computed Such a lowest natural modes were apparently significant t o the engineer. phenomenon is akin t o the success of inverse iteration. Yet, we have also seen problems where increasing the stiffness associated with particular elements destroyed the significance of the computed eigenpairs. It may be that, as the ordinary displacement problem can be reformulated as a sparse least squares problem, the vibration problem also has a dual problem, perhaps as a generalized singular value decomposition. We do not know of such a formulation, nor do we yet have good algorithms for the generalized SVD. Even if such existed, the engineering community’s investment in displacement formulation
R. G. Grimes et al
92
modeling will cause great resistance t o conversion. Problems with ill-conditioned stiffness matrices are likely t o continue t o plague us. Ill-conditioning in
M
is less common, but potentially serious. Again, the
problem can arise because of modeling constraints in the finite element package. A specific and common example arises in earthquake analysis. In modeling the effects of an earthquake on a building the foundation o f the structure is prescribed to move with the ground motions. If the engineering software does not have the capability of adding these as algebraic constraints on the displacement system, they can be approximated by assigning large fictitious masses t o the affected nodes. The effect on eigenanalysis algorithms is that M-orthogonality becomes more difficult t o obtain with the Gram-Schmidt process. It is possible t o design robust codes, as see 171, but Mera [12] gives real examples of failures within at least one well-known and respected finite element package. In conclusion, we and our colleagues have learned that significant numerical problems remain for the solution of structural engineering eigenvalue problems. Some of these problems have been addressed by the developments in the Lanczos algorithm in the last five years, but others remain t o be solved. As computing power and modeling capabilities increase, we are likely t o see more of these difficult eigenproblems, and the need for robust numerical software will increase. REFERENCES [ 11
L. Meirovitch, Computational Methods in Structural Dynamics (Sijthoff &
Noordhoff, Rockville, MD, 1980). [2]
[3]
Lewis, J. G. and Grimes, R. G., Practical Lanczos Algorithms for Solving Structural Engineering Eigenvalue Problems, in: Duff, I . 5. (ed.), Sparse Matrices and Their Uses (Academic Press, London, 1981). Ericsson, T. and Ruhe. A,, The Spectral Transformation Lanczos Method for the Numerical Solution of Large Sparse Generalized Symmetric Eigenvalue Problems, Math.of Comp 35,No. 152 (1980) 1251-1268.
[4]
Scott, D. S., The Advantages of Inverted Operators in Rayleigh-Ritz Approximations, SIAM. J. Sci. Stat. Comp. 3, No. 1 (1982) 68-75.
Eigenvalue Problems and Algorithms in Structural Engineering
93
Nour-Omid, B. Parlett, B. N., Ericsson, T., and Jensen, P.S., How t o Implement t he Spectral Transformation, Math. Comp. (to appear). Ericsson, T., A Generalized Eigenvalue Problem and the Lanczos Algorithm, thisvolume. Grimes, R. G., Lewis, J. G. and Simon, H. D., The Block Shifted Lanczos Algorithm for Structural Engineering Eigenproblems, t o appear. Golub, G. H. and Underwood, R., The Block Lanczos Method for Computing Eigenvalues, in: Rice, J. (ed.), Mathematical Software I l l (Academic Press, New York 1977). Cullum, J. and Donath, W. E., A Block Lanczos Algorithm for Computing the Q Algebraically Largest Eigenvalues and a Corresponding eigenspace o f Large, Sparse Real Symmetric Matrices, i n : Proc. o f the 1974 IEEE Conv. o n Decision and Control, Phoenix (1974). Parlett, B. N. and Nour-Omid, B., The Use o f Refined Error Bounds when Updating Eigenvalues o f a Growing Symmetric Tridiagonal Matrix, Linear Algebra and ItsApplications68 (1985)179-219. Golub, G. H. and Van Loan, C. F., Matrix Computations (The Johns Hopkins University Press, Baltimore, 1983) Mera, A. MSC/NASTRAN Normal Mo d e Analysis w i t h GDR: An Evaluation of Limitations, in: Proceedings of the 1985 MSC/Nastran Users Conference, Pasadena (1985).
This Page Intentionally Left Blank
Large Scale Eigenvalue Problems J. Cullum and R.A. Willoughby (Editors) 0 Elsevier Science Publishers B.V. (North-Holland), 1986
95
A GENERALISED EIGENVALUE PROBLEM AND THE LANCZOS ALGORITHM Thomas Ericsson visiting the Center for Pure and Applied Mathematics University of California Berkeley, California U.S.A.
Some properties of the generalised eigenvalue problem, Kx = X M z , where K and M a r e r e a l and symmetric matrices, and K is nonsingular and M is positive sermdefinite (and singular), a r e examined. We s t a r t by listing some basic properties of the problem, such as the Jordan normal form of K I M . The next p a r t deals with three standard algorithms (inverse iteration, the power method, and the Lanczos method) the focus being on the Lanczos method. I t is shown t h a t if the Lanczos starting vector is contaminated by a component, in a certain subspace, this component may grow rapidly and cause large e r r o r s in the computed approximations of the eigenvectors. Some ways to refine the approximations a r e presented. The last p a r t deals with the problem of estimating the quality of an approximate eigenpair, given a residual T
=m-xm.
I . Introduction The properties of the generalised eigenvalue problem
Kx = hMx where K and M a r e real, symmetric matrices, and M is positive definite, a r e wellknown. Since M is positive definite we can transform the generalised roblem into a symmetric standard eigenvalue problem A z = hz, where A = M-#KM- and 2 = Mx. This implies, for example, t h a t there exist real matrices X and A, where X i s nonsingular and A diagonal such t h a t K X = MXA. Since M is positive definite, x T M y defines a n inner product and (zTMz)% a norm. We can, f o r example, choose X such t h a t the eigenvectors a r e orthogonal with respect t o M ,i.e. XTMX = I . Given a n algorithm f o r the standard eigenvalue problem, we can also construct the corresponding algorithm f o r the generalised problem. The main computation in = Ayk. Reversing the transformathe power method, for example, h a s the form where gi = M-HY,, i = k , k + l , f o r the power tion above, we get the A@k+l = method in the g e n e r a k e d case. However, in practice one often encounters the case when M is only positive semidefinite. When this is the case most of what we did above need not be true. Not only may we lose the properties t h a t depend on the fact t h a t M is positive definite (some of those will remain if K is positive definite) but we can g e t a n eigenvalue problem with new properties, a s shown in the following example.
!f
mk,
Example.
LetK=b Y ] , M = [ i
:].
Here we have one h = 1 and one i n f i n i t e A, (i.e. a zero eigenvalue of K I M . )
T.Ericsson
96
Let
K=
01
[l O].
0 0
M = [o
then K I M =
1i]
In this case Kx = XMx has a defective. infinite. eigenvalue
Here K
I] k], = AM
f o r any A, a n d we s a y t h a t t h e pencil (K, M ) is singular.
To summarise one could say t h a t this paper deals with some of the problems t h a t arise when we have a positive semidefinite a n d singular M . We s t a r t by analysing the eigenvalue problem, in section 2, to find out when we have a defective, singular etc. problem. In section 3 the behaviour of t h r e e algorithms (inverse iteration, power iteration, and the Lanczos method) is examined. Most of the section deals with a M inner product Lanczos algorithm for a shifted and inverted problem. It is shown t h a t problems occur (the Lanczos vectors may become contaminated by large components in a unwanted subspace). Section 4 deals with t h e problems of estimating the accuracy of computed results, given a residual. When Y is positive definite t h e r e a r e several bounds using t h e & -n!-'orm. When M is singular we must, of course, look for other alternatives. This paper does n o t deal with the implementational details of the algorithms (e.g. roundoff is neglected). A more practical view c a n be found in [ 13. For easy reference, we have collected some notation and definitions below.
1.1. Notation.
The notation corresponds to t h a t of Wilkinson [6], with the following additions.
Em'" denotes t h e vector space of all m-by-n r e a l matrices. If A E I R ~ ' " , t h e n the range space T ( A ) = { ye!Rm1 y = Ax for some z E I R " ~ ,and t h e null space of A is n ( A ) = [EIR" s I A z = Oj. If d i m ( ) denotes t h e dimension of a space, then rank ( A ) = d i m (T ( A ) ) ,null ( A ) = dim (n( A ) ) . If A = ( a l , .. . , %), t h e n s p a n ( a l , . . . , %) = T ( A ) . Let A , B E En’", t h e n A , , ( A , B ) is a n eigenvalue of t h e pencil ( A , B ) , i.e. d e t ( A - A k ( A , B ) B ) = 0. If B=Z, i.e. we have a standard eigenvalue problem, we write A k ( A ) instead. If it is clear from the context, what matrices a r e involved, we will use Ak.
If A €IRnXn, A = A T , A j ( A ) > 0, i = 1, . . . , n , we say t h a t A is positive definite. If & ( A ) > 0 , f o r all i. A is positive semidefinite. If A o r -A is positive definite, A is definite. If h i @ ) may be arbitrary, A is indefinite. If A is positive definite, zTAy defines a n inner product, and I ) x ] I A = ( x T A z ) g a vector norm. If zTAy = 0 , we s a y t h a t x a n d y a r e A-orthogonal. The n o r m having A = I is denoted 1 1 1 1 . The subordinate matrix n o r m is denoted in t h e same way, so 1 1 B 1 1 = ( m a z A(B*B))%. A+ is the Moore-Penrose pseudo-inverse of A If & , i = 1.. . . , n a r e square matrices, C = d i a g ( A 1 . .. . denotes the block , = 4. diagonal matrix where Cijis the zero matrix f o r all i f j , and C
,&I
Since we deal with K and M throughout this paper, we will give some often used factorisations:
A Generalised Eigewalue Problem und the Lanczos Algorithm
91
M has t h e eigen decomposition M(R, N ) = ( R ,N ) d i a g (R2, O),where the square matrix ( R ,N) is orthogonal, R = d i a g (a1,. . . , a?), T = rank ( M ) and u, > 0. ( R stands for Range space part, and N for Null space p a r t ) If K is nonsingular then
where H3 is a square matrix of order n u l l ( M ) . H I has the eigen decomposition H I ( Q ~QN) , = (QR,QN)d i a g ( h - ' , 0 ) where (QR, QN) is orthogonal.
C = X-'M, has the Jordan normal form CS = S J . The projections Px,PN and the subspaces X, N a r e defined in section 2.3 2. Some Basic Properties of ( K ,M ) . In this section we will give criteria for when t h e pencil (K, M ) is singular, defective e t c . Some of the theorems a r e well-known, but we have included them for completeness. These subjects a r e also dealt with in [ 5 ] , "71, f o r example, but from a different point of view. A s before we assume t h a t K and M E IRnX", and t h a t M is positive semidefinite and singular. We assume t h a t K is symmetric, but otherwise arbitrary. To make things easier to follow, we will first transform Kz = AMz into a n equivalent problem where the transformed M has a simple form.
Lemma 2.1. Kxi = XiMzi c a n be transformed into Rsi = hi&si,where R, & a r e r e a l = bij then a n d symmetric, a n d fi = d i a g ( I , 0 ) with T a n k (a)= T a n k (M).If z:Mzj s:l@zi = 6, Proof. Using the factorisation of M (please see section 1.1) with U = ( R ,N ) , A = d i a g then Kx = AMx H A-'UTKUA-'AUTz
(n, I ) ,
M = U A d i a g ( I , O)AUT
= A diag ( I ,O)AUTz H
Rz^= ha?, with
R = A - ~ u ~ K u A -z^~=. A U T X . Assume M = d i a g ( I , 0 ) and partition
K as
and let K ( a ) = K+aM, a EIR.
Lemma 2.2. If
k]
has full column rank then there is a a such t h a t K ( a ) is nonsingu-
lar.
Proof. Assume t h e r e is no such a, and let Q = d i a g ( 1 , U ) , where U T K 3 U = d i a g ( A , 0 ) , where A is nonsingular and U is orthogonal. Let K 2 U = ( A , B ) .
2 ’
Having assumed t h a t @ K ( a Q IS singular for all a, there must exist nonzero w T = ( x T , y T , z T ) , s u c h t h a t QK ( a ) @ -0,or
T. Ericsson
98
(Kl+aZ)x+Ay +Bz = 0 A ~ X + A Y=
o
BTz = 0
(22)j
AA-'ATz+Ay = 0. This, together with (1) x ( K l + a l ) z - z T A A - ' A T x = 0, so z T ( K 1 - A A - I A T ) z= - a x T z .
and
(3),
gives
Now, take a such t h a t l a l > m a x I A ( K l - A A - ' A T ) I , then x = 0 is necessary (since -a would be a Rayleigh quotient otherwise).
= 0 a n d ( 2 ) > y = 0 , s o (1) gives Bz = 0 , a n d since w rank.
z
#
0 , B cannot have full column
Using lemma 2.2 i t is easy t o prove:
Theorem 2.3. ( K , M ) singular H equivalent to t h a t
El
does n o t have full column rank. (Which also is
K and M have a nontrivial intersection between their null spaces.)
We leave the proof to the r e a d e r We will now characterise t h e definite case ((K, M ) is said to be a definite pencil if t h e r e exist r e a l numbers a and p such t h a t aK+pM is a definite matrix). If (K, M ) is nondefective t h a t does n o t imply t h a t t h e pencil is definite. Let, f o r example,
K = diag(1,1 , -l), M = diag(l,O,0), aK+@M = d i a g ( a t @ a, , -a) which is indefinite for all a,8 . The pencil is n o t defective, however. We have t h e following theorem.
Theorem 2.4. (K, M ) definite
K3 definite
Proof. If (K, M ) is definite, then K ( a ) is definite for some a , and in particular K3 must be definite.
If, on t h e other hand, K3 is definite (let us assume it is positive definite) take a > I min A(Kl )I (so Kl+aI is positive definite). The block LDLT-decomposition of K ( a ) is given by K ( a ) = LDLT, where D has the diagonal blocks Kl+aI and K ~ - K ~ ( K ~ + c x I ) Since - ' K ~ .K ( a ) and D a r e congruent we can, by taking a large enough, make t h e Dzz-block positive definite (the Dl,-block is already positive definite). H One special case is t h e one t r e a t e d in t h e following corollary
Corollary2.5. If (K, M ) is nonsingular and if K is positive semidefinite then definite.
(K,M ) is
Proof. It is obviously t r u e if K3 is definite. Assume K3 is positive semidefinite and singular. We now prove:
El
has full column r a n k and K3 is singular 3 K has negative
a n d positive eigenvalues. (So if K is positive semidefinite K3 must be positive definite.) Take
z# 0
such
that
K3x = O ,
and
let
z T =(xTKl,uzT),
then
A Generalised Eigenvalue Problem and the Lanczos Algorithm
99
z T K z = z T K ~ K l K z z + 2 ~K1 Ip l12. Since K2z f 0 (we would have rank deficiency otherwise), z ' K z can take any value with a suitable choice of 0 .
=
All the above results hold also when M is nondiagonal ( M is still real, symmetric and positive semidefinite). To generalise the theorems we use the matrix N E RXnU''((iU) introduced before (see section 1.1) and reverse the transformation in lemma 2 . 1 . Our two theorems then become:
( K , M ) singular H KN does not have full column rank ( K , M ) definite H NTKN definite.
(2.6 a ) (2.6 b)
In the coming sections we will not look a t the case when ( K , M) is singular. One way to avoid the singular case is t o assume t h a t K is nonsingular. Another advantage, t h a t results from the assumption, is t h a t we c a n study the transformed problem Cz = u z , where C = K I M , and u = A-'. To justify t h a t assumption we need, however, to show t h a t K being nonsingular does not r e s t r i c t our problem t o o much, a n d t h a t the theorems holding for a nonsingular K also a r e t r u e for a singular K and nonsingular ( K ,M). This is done in the next two paragraphs. In the next chapter we will construct the Jordan normal form, CS = SJ of the matrix C = K ' M . I t will be shown t h a t J = diag (A-l, B), where A is diagonal and non-
I"
singular, a n d B is a block diagonal matrix having and 0 blocks on the diagonal. S ll c a n be partitioned a s S = ( X , R ) , where XTMX = I , so t h a t K x = MXA. For the B-block we have KRB = MR. Now, assume t h a t ( K ,M ) is nonsingular and t h a t K is singular. Then there exists a such t h a t K+aM is nonsingular. From the Jordan normal form of (K+aM)-'M, (K+aM)-'MS = S J , we get M: = MX(A-aZ) and KRB = MR (since B(Z-aB)-' = B). S o t h e only change we can get, when allowing a singular K , is t h a t ( K , M) can have one o r more zero eigenvalues. As we have seen, NTKN is a n important quantity and since M has not changed (when forming K ( a ) ) ,N T K ( a ) N = NTKN.
Putting these things (and those in the next section) together, we get the following picture:
T. Ericsson
100
KN
/\
r a n k deficient
full r a n k
\
singular
( K , M ) is nonsingular
I
N~KN
We will not analyse this branch f u r t h e r
/\
definite
not definite
I
\
(K,M ) is n o t definite
( K ?M ) is definite
I
NTKN
/\
singular
nonsingular
( K , M ) is not
(K,M )
is defective
defective
2.1. The Jordan NormalFormof K I M . In the sections t h a t remain we will assume t h a t K is nonsingular and t h a t M is positive semidefinite and singular. So our problem Kz = AMx can be written a s Cz = u z , where v = A-', C = K ' M . We know t h a t M is singular, s o some u-eigenvalues must be zero (corresponding t o infinite A-eigenvalues). In this section we will cons t r u c t t h e Jordan normal form of C, CS = SJ, where S is nonsingular and J is block diagonal. To s t a r t with we assume t h a t M = diug ( I , 0 ) . Suppose now t h a t M=diag ( I , O), K-’ = H =
Ei 2
then C = K - ' M =
E:
:],where
H , has order T = r a n k (M), and we c a n s t a t e our-main theorem in this seciion. ' Theorem2.7. C has a zero eigenvalue of algebraic multiplicity n u l l ( M ) + n u l l (H,) and The order of the Jordan blocks, corresponding to the geometric multiplicity n u l l (M). defective eigenvalues, is two. The remaining m = n - ( n u l L ( H l ) + n u l L(M))nonzero eigenvalues a r e nondefective, and we c a n find diagonal A E RmXm, a n d X E IRnXm, such t h a t K x = MXA and XTMX = I . Proof. The eigenvalues of C a r e Ak(Hl) and n u l l ( M ) zeroes (looking a t the diagonal blocks C, Czz). Suppose t h a t H 1 has the spectral factorisation
HI(@, QN) = (QR,QN) d i a g
0)
101
A Generalised Eigenvalue Problem and the Lanczos Algorithm
where (QR,QN) is orthogonal, a n d A = diug(hl,. . . , As), for some s = rank(H1),and where t h e hi # 0. Then CX = XA-’, where
PI
*= H z Q ~ A Obviously XTMX = I,and lix = MXA. This t a k e s c a r e of the nonzero eigenvalues of C . The zero eigenvalues (corresponding to t h e Czz-block), have eigenvectors
k],
where
U is any nonsingular matrix of o r d e r null ( M ) . If H , is nonsingular this would be it. But if H1 is singular, we c a n n o t find eigenvec-
t o r s corresponding t o these e x t r a zero eigenvalues, since if u = I
implies H , u , = 0, H z v l = 0. which gives u 1 = 0 (otherwise H
\
l“bl
tI:
, t h e n CV = O
= 0 , a contradiction).
v z m u s t be zero too, since we have used all these vectors (for U-above.)
Another way to s e e the same thing, is by looking a t t h e principal vectors in t h e Jordan of g r a d e o n e , corresponding to the zero eigenn o r m a l form. The values, a r e given QN
n ( C z ) - n ( C )= T ( [ o
1).
The principal vectors of grade two, a r e given by There a r e no principalvectors of higher grade, since
I t follows t h a t t h e Jordan normal form, C S = S J , is given by S =( X , N ) and J = diug(A- ,O) in the nondefective c a s e , a n d ( X , R ) (for some R) and J = diug (A-l,B) where B = diug ( B , , . .
,
,
B,, 0), B1 =
, in the defective case.
Let us now look a t a problem where M # diug ( I , 0) By reversing t h e transformation, of t h e g e n e r a l eigenvalue problem, we g e t the corresponding H,= R R T K I R R , a n d so null ( R T K ’ R ) is t h e interesting quantity. We c a n use the Iollowing lemma to c h a r a c t e r i s e null ( H , ) in t e r m s of K ,r a t h e r t h a n K .
Lemma 2.8. Let Q = lar, then
(al,Qz) be a n orthogonal matrix, and assume t h a t A is nonsingunull (QTA-’ Q1)= null (Q$4Qz)
Proof.
have t h e same nullity.
W e c a n now reformulate our t h e o r e m :
T. Ericsson
102
Corollary2.9. Given Kz = AMz ( M not necessarily equal to d i a g ( I , O ) ) , there a r e + null ( N T K N ) infinite eigenvalues (zero A - l ) of which null ( N T K N ) a r e defective. The remaining eigenvalues a r e finite, nonzero, and nondefective, and we can find h a n d X s u c h t h a t KX = MXh, XTMX = I .
null ( M )
Example. Let K l , K2, and M , E IR"’",
M , positive definite, Kl symmetric, t h e n with
all the eigenvalues a r e infinite (and located in 2-by-2 blocks). In this c a s e
,
#
Let M = d i u g ( l , 1,0,0)
and K =
1 0 0 0 0 1 1 0 0 0 1 1
That there a r e null (M)+ nu11 ( N T K N ) infinite eigenvalues, of which null ( N T K N ) lack eigenvectors, should be interpreted in t e r m s of zero eigenvalues of C. We can t h e n avoid a discussion of eigenvalues in plus a n d minus infinity, as in the following example.
K = diag (1, 1, - l ) , M = diag(Z,O,0 ) . Here c = diug (2, 0,O) I
the
eigenvalues
are
5, + m ,
-m,
but
2.2. Another Word on Definite Pencils. We have already seen t h a t ( K , M ) nondefective does not imply t h a t the pencil is definite, i.e. a linear combination a K t g M , a2+P2 = 1, is positive definite. Let us now consider the equivalent pencil
(R,a)= (-@K+aM, aK+BM).
( R ,a)is a positive definite pencil, then RS 3 = S diag((aA+pI)-X, (aNTKN)-B), sT@3 =I A = diug ((aA+PI)-l(al-@A),-(@/ a ) I ) . Theorem 2.10. If
Proof.
=I
@ ~ Xwhere ,
A Generalised Eigerivalue Problem and the Lanczos Algorithm
103
f2S = ( - p K + a M ) S = -pKS+aKSJ = K S ( - p I + a J )
I@S= ( a K + p M ) S = aKS+pKSJ = K S ( a I + p J ) Since
(R,8)is
definite, aI+pJ must be nonsingular, and hence
h = (aI+pJ)-'(-pI+aJ) = diu ((aA+pI)-'(aI-pA), - ( p / a ) I ) .
RS
=
where
Q
diug_(aA+pI,a N K N ) , so with D = diug ((aA+pI)-%,(aNTKN)-H), 3 = SD, Now STI@S_= ST@S= I , AD = DA. which proves the theorem. I
2-3.Projections.
Since we will r e f e r t o T ( X ) quite often we introduce t h e following notation:
x=T(X)
If (K,M ) is nondefective it will later be shown t h a t X = ' f ( ( K - p M ) - ' M ) . The null s p a c e of (K-pM)-'M is denoted N. Obviously N = n(M).N a n d X a r e orthogonal with r e s p e c t t o K (if K is indefinite we mean t h a t z T K y = 0 for all x ~ X a n yd EN.)This follows f r o m K X = MXA and y T M = 0 if y EN.Also X @ N = iR".This is easily s e e n f r o m the proof of theorem 2.7 or by the fact that if S = (X,N) t h e n
sT= (m(xTm)-l, KN(N~KN)-’).
We also introduce t h e oblique projections
Px = X(XTKX)-'XTK = X X T M pN = N ( N ~ K N ) - ' N ~ K We have Px(Xu+Nb) = X a ,PN(Xa+Nb)= Nb. Also Px+PN = I (since ( P x ,P N )= (X, N)S-' = I ) , P$ = Px? P$ = PN,a n d P$KPN = 0
Example. In the following examples it c a n be s e e n how Xis made u p of components in T (M) a n d n (M). Note that eigenvectors , in general, s h o u l d have components In n (M). Note also t h a t X a n d N a r e orthogonal t o e a c h other with r e s p e c t to K and n o t with r e s p e c t to I. 1)
Let K = I a n d 2M = diug (1, 0 ) , t h e n A = 1 and X = s p u n ( (1, O)T).
t h e n h = 1.5 a n d X = s p a n ( ( 1 ,-)$)T) 3)
t h e n h = 1.5andX=spun((l,-)$,0)T).
If (K, M ) is defective t h e n we will show t h a t X = T((K-pM)-*M(K-uM)-'M). I t is no longer t r u e t h a t X@ n(u)= En (it is easily seen from t h e proof of t h e o r e m 2.7, we
T. Ericsson
104
miss
the
principal
vectors
of
grade
2).
If
we,
however,
define
N = ~ ( ( K - , U M ) - ~ M ( K - C F M ) - ' Mwe ) , have X@ N = En.We c a n no longer define PN using
t h e formula above (since NTKN is singular), but t h e definition for Px still holds a n d s o PN = I - p x .
3. Some Algorithms. In this section we will look a t some standard algorithms (inverse iteration, the power method, a n d a Lanczos algorithm) for solving the generalised eigenvalue problem. We will take t h e algorithms for a positive definite M , a n d then s e e how they behave when M is singular. As we will s e e , the major problem will be to keep vectors in the right subspace X.
3.1. Inverse Iteration. The algorithmis given by (see [4] page 317).
In one s t e p we have 2 = (K-pM)-'& (we assume t h a t K-pM is nonsingular). The algorithm is self correcting (given y 1B x then yk E x f o r k > l o r 2) as shown in the following the0 rem.
Theorem 3.1. T ( ( K - p M ) - ' M ) = X if ( K , M ) is nondefective.
r((K-pulW)-’M(K-~hf)-’M) = X if ( K , M ) is defective Proof. Let C = K I M a n d let C = SJS-' be the Jordan normal f o r m of C .
(K-pH)-'M = (I-pC)-'C = S(I-pJ)-’JS-’ = S diap ((A-pZ)-', 0)S-l. In tne defective c a s e we g e t in t h e same way: (K-~M)-~M(K-UM)-~M = S ( Z - ~ J ) - ~ J ( Z - ~ J ) - ~=J - ~ sdiag ( ( ( A - ~ z ) ( A - ~ I ) ) - ~0 , )s-1 since, with B =
i"
, (I-~B)-’B(I-UB)-~B = 0.
W
This implies t h a t if y, has a nonzero component inX, after at most two iterations, inverse iteration will work a s the standard algorithm (when M is positive definite).
A Generalised Eigenvalue Problem and the Lanczos Algoritlim
105
3.2. The Power Iteration. The algorithm is given by (see [4] page 317). In this section we will assume t h a t t h e problem is nondefective. Given y p1 do For k = 1 , 2 , ... Mz = f K - p k M ) y k Yk+l = = / I / z 11 choose & + I In one step we have M z = ( K - p M ) y say. Now, if M is singular this equation need n o t have a solution. In fact, using theorem 3.1 (assuming K - p M is nonsingular), we see t h a t regardless of what the possible solution z is, t h e known vector y must lie in X. If we have access to ( K - p M ) - ’ M we c a n produce a vector in the right space (X). But if may use the inverted operator we may switch to inverse iteration anyhow. If we do not have access to t h e inverted operator we may consider a n iteration having t h e equation z = A ( K - p M ) y a s its main step. One may be tempted to t r y to use t h e pseudo-inverse, A = M+,of M (if M is diagonal, for example). That choice will n o t work in general, since M + ( K - p M ) y E T ( M ) , a n d we do n o t g e t t h e required components in N. The right choice is to take A = XX* since
X X ~ ( K - ~ M )=YX ( A X ~ M - I . L X ~ M ) Y= X ( A - ~ I ) X ~ M Y a n d any y c a n be written Sin, for some a.This gives
X T M S a = X T M ( X , N ) n = ( I ,0 ) n a n d we would pick o u t the right p a r t of y The drawback with this approach is, of course, t h a t we do n o t know X,a n d t h e r e is no way we c a n find X f r o m M alone. We will r e t u r n to t h e matrix X X T in section 4, but it should be mentioned t h a t i t is a generalised inverse (though not, in general, t h e pseudo-inverse) of M .
3.3. The Lanczos Algorithm We will, in this paragraph, study the Lanczos algorithm for t h e generalised eigenvalue problem (see [4] for more details). We will, however, s t a r t to look a t the algorithm for a shifted and inverted standard problem, i.e given A r e a l and symmetric ( A - p 1 ) - ’ ~= ( A - ~ ) - ’ z AZ = AZ assuming t h a t t h e inverse exists. The Lanczos algorithm for this problem c a n be written:
T. Er icsson
106
Given v l , urvl = 1 , compute
For i = 1 t o p do u = (A-pI)-bi if( i > 1) then u = u -/3i-lv,-l
ai = v r u
u = u-a,v, Pi
=l l ~ l l
if ( pi = 0) then va+1 = 0 stop else
= u/Bi endif The algorithm produces, in exact arithmetic,
V = ( v l , . . . , u p ) , with V T V = I and a tridiagonal matrix T , 'a1
81
81
a2
T=
.
> '
. . . .
. 4
. ap-1 a,-1
a,-1 ap
,
such t h a t
( A - ~ J ) - ' v = VT+ppvp+lepT If ( s , v) is an eigenpair of T , i.e. Ts = v s ,then I I ( A - p u l ) - l V s - v V ~ I / = lt3,~,
1
Suppose t h a t M is positive definite, then the problem Kx = h M x is equivalent to A2 = h z , where z = $z, and A = M-5KM-g. Let us now reverse this transformation in the Lanczos algorithm above by introducing the new vectors 4 + u := M%L and This gives us the following algorithm in M-inner product. ui:=
A Generalised Eigenvalue Problem and the Lanczos Algorithm
107
Given v 1, v ?Mu = I,compute For i = 1 to p do 4 = (K-pUM)-lMVi if( i > 1) then 4 = di-&-lvt-, ai = v,'Mdi d, = d,-aivi Bi
= 114 I I M
i f ( Bi = 0) then stop else vi+l =
di/@i
endif Again we get tridiagonal T and V such t h a t
VTMV = I (K-pM)-'WV =
vT+% e,'
If ( s , u ) is a n eigenpairs of T , then II(K-pM)-'MVs-vVs
llM
=
Isp
I lid, l l d d
(3)
We will now use the above algorithm when M is singular Formulas (1) and (2), will still hold, but (3) will not give us the whole t r u t h (since M does not define a norm over the whole space). I t should also be noted t h a t a, = 0 is possible even if dp f 0 (if dp EN).If this is the case, up+' is not defined, b u t % is. (The only reason we have a n index on d, is to be able to refer to the vector. In a implementation we would use the same vector, a s shown in the Lanczos algorithm for A , ) If I g p > 0 then % = v p + l &., We will use T and V in the coming sections without specific reference t o p If we must refer to submatrices of T or V we use Ti to denote the leading principal submatrix, of order j , of T ,and to denote the first j columns of V (so in particular T = Tp and v = VP). Sections 3.3.1-3.3.4.2 deal with the nondefective case. The defective case is t r e a t e d in section 3.3.5.
3.3.1. Contamination of Vectors. We s t a r t this section by an example.
Example. Let K = d i a g ( h l , . , , l), M = d i a g ( 1 , . . . , l , O ) and take v T = ( l , O , . . . , O , E ) . This gives al = (Al-p)-l, p, = 0, and d , = -(A1--p)-’&en. This means t h a t the eigenvalue is e x a c t but the Ritz vector Vs = v l can be arbitrarily bad depending on E . The Ritz vector c a n be refined, and in this case V s + a ; ' d l = e l is the exact eigenvector. The cause of this problem is t h a t v i X,i.e. in this case PNV,= t e n . We will examine the cause of this problem and a try to find a c u r e f o r it in the following sections.
T. Ericsson
108
We will s t a r t by examining how components n o t being in X may affect t h e algorithm in t h e nondefective case. A s was proved f o r inverse iteration (K -pM )-’%, f o r any y ,will always produce a vector in X, hence
Corollary 3.2. If v EX t h e n all t h e vi and any s a n d y .
4
lie in X, so t h a t Vs a n d Vs + y d , lie in X,for
Proof. Using induction and the f a c t t h a t dk =
((K-/LM)-lMvk--(XkV1,-pk-,vk-l) gives theproof. W We will now look a t how a i , pz, vi, a n d $ change when we use a staring vector
v,@.xx.
have been produced by the Theorem33 Suppose t h a t a i , p i t v i i = 1 , . . . , p , and algorithm using v,EX. Let G i , & , Gi i = 1 , . . . , p , and%= denote the corresponding quantities produced by the algorithm, using t h e starting vector cl = v l + z , where z
EN,z
# 0.
Then
3E. = aI. ipi = pi , 8,= v i + y i z , i = I , . . . , p
ap = 4 + 6 , z
where 7 . = -(a. r-,Ti-,+Bi-27i-2)/Bi-l
6,
If
I
2
sP
8
70
= 0,
71 = 1
= -(“pYp+Bp-IYp-l) pk f
0, then 6 k = 7 k + l p k .
p k a r e formed by a&tf(K-pM)-’Mak and (blmk)# f o r some vectors a, Since Mu = 0 for any u E N,ak and pk a r e n o t affected by components in N. To prove the recursion use:
Proof. ak and and
bk.
= (K-/LLM)-’Kk which gives
6k
=
We leave it to r e a d e r to fill in the details.
-cxkYk-pk7k-l.
To s e e more clearly how Tk.
* -akGk-pk-IVk-,
6k
behaves, we express bA
In
terms of the eigenpairs of
A Generalised Eigenvalue Problem and t h e Lanczos Algorithm
109
Theorem35 L e t Tk have the spectral decomposition TkSi =
ViSi
, i = 1,. . . , k
then
Proof. Let
[4], page 129#gives sliskip'(vt)
and sincep'(vi) =
= @1'...'@k-l > 0 , 1 i
k,
k
(ui-vj) we g e t 3=1 1 #i
6k = P (o)(sliskiP'(ui))-l which proves the theorem. W This expression for bk holds for all i < k , s o let us pick a i such t h a t vi has settled down (converged) so IskiI is small. Let us also assume t h a t vi is well separated (the component of t h e Ritz vector from the other eigenvalues and t h a t s l i = ( VS,)~MV, in vl) is n o t particularly small. (All these assumptions a r e quite reasonable in a practical setting.) Then bk = s g l T k for some -rk, where it follows from the assumptions t h a t 7 k behaves reasonably, and s o I b k I can become quite large. Let us now look a t how the contamination affects a Ritz vector y^ = v s .
Theorem 3.6. Let g
= (rl,. . . ,y p ) then Tg = -bpep fj=v s = v s + z g T s = y - 6 p s p v - 1 z ,
(a) (VfO)
(b)
Proof. We know t h a t P = V + Z ? ~(the e r r o r is always in the same direction). Using this a n d (K-pM)-'MV = v T + a , ep we get (K-pM)-IMV = VT+$ eT+z ( g T T + 6 pepT proving (a). (b) follows directly from (a). H
ck.
From (b) we see t h a t the Ritz vector y^ is not so badly affected by z a s the In a r e less affected t h a n fact, good Ritz vectors (smaller Isp I and usually larger 1.1) bad ones. In the next section we will show t h a t we can get rid of t h e contamination of the Ritz vectors altogether by adding a suitable multiple of the next Lanczos vector.
3.3.2. Correcting the Ritz Vector. We will now look a t a n alternative to v s . Is there any linear combination Pa or $’a++,, t h a t does not have the unwanted z-component?
T. Ericsson
110
Theorem3.7. The vector
g = vs +spv-12p , v
# 0
has no component along z , and gives a residual (K-wM)S E T (M)for any w. Forming this vector is equivalent to taking one s t e p of inverse iteration
g = ~-'(K-puM)-l&Proof. Using ( K - p M ) - ' M v = v T + $
Ts = u s , gives
, :e
(K-~M)-IMVS =
For the residual (K-pM)g = v - ' M V s so ( K - u M ) ~= ( K - , u M ) ~ + ( ~ - c J ) W = M(u-'
VY
+sP
dp = ~g E X
VS + ( p - - ~ ) g )
I
(The vector was introduced in [Z] b u t for a different purpose.) When computing &? in a program t h e r e a r e better ways of doing i t t h a n using t h e formula above. One reason is t h a t s t a n d a r d programs (like t h e ones in EISPACK) may produce a sp t h a t is completely wrong when Isp I is small. See [ l ] for more details. It should be noted t h a t g is not t h e Rayleigh Ritz approximation to (K-pM, M ) from span(V,$ ) o r s p a n ( y , $) (for more details about these topics see [l]). Nor is PxG = g~ in general (T E R).Using (K-pM)-'&- = V $ + S ~ I ? ~= ug w e have
pXg = V X ( A - , U I ) X ~ ~
v-’ap
However, given = 9 s !$ X and a,, g = $+sp is the only (up to a scalar factor) linear combination of y^ a n d a, t h a t lies in X. W e may also note t h a t it is impossible to find linearly independent t,, . . . ,tp such that
P&
E
x,
i = 1,. . . , p
since this implies t h a t gTti is zero for all ti,which is impossible unless g = 0. The reason i t works with ps +sp is because we take a linear combination of p + 1 vectors and get p vectors in X. It would, of course, not be possible t o g e t p + 1 vectors in X by using V,,,. We have unfortunately not found any practical way t o refine the G k . One would like to hit t h e Lanczos vectors by Px = I-PN = I-N(NTKN)-’NTK, but the expense is usually t o high.
.-'ap,
We end this section with a look a t t h e case when T is singular. Since T is unreduced the eigenvalues a r e distinct and s o if T is singular u = 0 is a simple eigenvalue. Due to interlacing and Gcl cannot both be singular. Assume Tk is singular, then from lemma 3.4 it follows t h a t 61, = 0 ( s o 2, EX) and Tg = 0 (from 3.6 a) s o g is a n eigenvector corresponding to v1 = 0 say. Hence psi = Vsi+zgTsi = Vsi E X provided i > 1 and t h e r e is no need to c o r r e c t these Ritz vectors. The Ritz vector vsl does not lie i n X ( b u t it is of little practical interest). Singular T c a n , of course, be produced by the Lanczos a1 orithm. One example is given by K = d i a g (A, -A, I ) , M = diag ( I , I , 0), y = 0 , a n d v f = (aT,a T ,bT), Then all ak = 0 which implies t h a t all Ti of odd order will be singular. (Proof: Let A be a square matrix satisfying uij = 0 if i + jis even. Let C = diag(1,-1, 1 , - 1 , ...) then C A E = -A s o if A z = X z then A ( & ) = -X(Cz), and in o u r tridiagonal case this implies t h a t eigenvalues occur in +- pairs.) For a numerical example see section 3.3.4.
G
111
A Generalised Eigenvalue Problem and the Lanczos Algorithm
3.3.3. The Nonsingular Case. In this section we will t r e a t t h e case when M is nonsingular but ill conditioned. We will limit the study to one simple problem, and we s t a r t by listing some of the properties of the problem. Let
W e a s s u m e t h a t I I K l j l I 1 , 11q11 = 1 , / u I s l , a n d t h a t O < ~ < < l . I t is not difficult to sh?w t h a t (K, M) has o n e large eigenvalue FY u / E (with corresponding eigenvector 2 , say) and n-1 eigenvalues in [ - 2 , 2 ] . If we partition ZT = ( z : , Z N ) , Z N E IR (the notation ZR, Z N should remind about range and null space in the singular case) we can show t h a t
x
/ Z N / // I z R I I
Id/&
This means t h a t z^ is very rich in the eigenvector of M corresponding to X ( M ) = E . This is not true f o r eigenvectors corresponding to the small eigenvalues of ( K , A!?). Here / z N / / lISR//
1/(1-2&)
As will be s e e n in the example in the next section T (G)will function a s N did when was singular. When M was singular we had P N = N ( N ~ K N ) - ~ N ~and K / y k I = / I PNGk I I I I PNG, I I - I . Now when M is nonsingular we define the projection
M
P;. = z^(ZT=)-'STK = z ^ s T M , STM? = 1 a n d provided z^Th?vl# 0
Iz^Tm
1% I = I I P Z V k II IIPP, 11-l = I lZTh,l-' ( ~ i g n ( 7 is~ defined ) below). Letc = VT&, so w i t h g T = ( y l , . . . , y p ) then 1g I = / c I lz^TM-u,I-l,and I I C / I = 1 1 V T M q c 1 1 VTk4lI IIk4z^II = 1
/ / g11 is not bounded in the same way since 119 I / I I z ^ T ~ I-', which can be made arbitrarily large by taking zi almost MI-orthogonalt o z^ Unlike the singular case where the norm of the contamination I I PNck 11 = Iyk I 11 11 1s not bounded ( 1 1 z 1 1 is arbitrary), it is bounded in the nonsingular case. liP;.Vll =
I]
I I ~ c ~ I5I
11Z11 % (min h(M))-#
1, V c a n contain large numbers. This follows from the f a c t Even though / j c t h a t VTMV = I , and s o llzik / / can be a s large as (min X(M))-#. If llvk 1 1 is large then the vector must have a large component along z^. The above conclusions can be eneralised to the case when M has more t h a n one small eigenvalue (if lix = MXA. XM ' X = I then t h e r e is a t least one xk such t h a t jlzk I / 2 (n min x ( A ! ? ) ) - ~ ) .
Another way t o see the similarity between the two cases is by using (K-pM)-'MV = VT+$ e,' which gives (multiply by z^'M) (T-(X-p)-lI)c = -STAT$ ep Taking c = gZTh4u (defining sign(-yk)) (T-(X-p)-'I)g = - Z T ~ a i (ZTMuI)-’ep , which should be compared with t h e singular case (lemma 3.6 a).
T. Ericsson
112
Multiplying (K-pM)-'MV = V T + d , e J by 2^TMIvT& I-' and s we get (with y = V s ) a n expression for the relative growth of 2 in the Ritz vectors
IIPfY/ I 11%~111-1= I Y T E l / v T W - ' = I ~ p + 1 B p S p llu-(x-P)-ll-lM l~p+$pspll~l-l where M hoIds for extreme u. This result can be compared with theorem 3.6 (b). When using f,?' instead of y the component along 2^ will decrease (but not be deleted as when M is singular). Using 3C^TM(K-pM)-'y = U?~A@ we get (provided ZTMy # 0) 77 =
l l w l l ( i l e Y il ilf,?'IlMr1 = l ~ T ~ l ( Il l l f~ , ? 'TI l M~ r '
=(I4R-d
llf,?'llM)-'
IA-Pl IX-PI-'
where the M holds if u has converged (so h M p+u-l for some A). For our special example we would thus get 7 M E IX-pI ] CTI-', quite a substantial reduction. We would also like t o point out t h a t similar phenomena c a n occur when using standard Lanczos on the standard eigenvalue problem Ax = Ax, since we can transform the generalised problem if M is positive definite. An example is presented in the next section.
3.3.4. Two Examples.
In this section we present two examples. In the first Y is singular and in the second Y is nonsingular and moderately ill conditioned. The examples a r e completely artificial and have been included only to give the reader some idea of how the Lanczos algorithm may behave. In both examples n = 30 and
K = d i a g ( 5 - ' , 6-',
...,
32-',
[i
I1
),
M = d i a g ( I Z z eE,)
and the starting vector was v f = (1, 1 , . , , , 1, - 0 . 5 + [ )
T
(where T had been chosen so t h a t vTMu = 1.) So (K, M ) has eigenpairs ((4+i)-', ei), i = 1 , . . . , 28. For the remaining ones we have to look a t the subproblem:
The results below were computed on a VAX in double precision (relative machine precision M 1.4.10-17) using a Lanczos algorithm with full reorthogonalisation implemented in Matlab [3]. We r a n the algorithm for 15 steps, i.c. p = 15. The tridiagonal matrices satisfied: 5.10-'< lak I < 2 and 1.6,10-2 Ifi2 d 0.36. M 1.6.10-'.
3.3.4.1. Singular M Here E = 0 s o the subproblem (*) has eigenpairs (g, (1, -5)') and ( m , (0, l ) T ) . The starting vector had equal components of all eigenvectors corresponding t o finite eigenvalues. The contamination, z = [ T e 3 0 . Since we know t h a t P p k = ykz the size of 6 is not important if we just wish to find yk. (In practice we would like to have a
small ] [ I , of course, i.e. we would like t o have a w 1 t h a t lies in X.As we have seen (theorem 3.1) this can be accomplished by hitting any vector by (K-pM)-'M. Using
A Generaked Eigenvalue Problem and the Lanczos Algorithm
113
this technique on a computer would give a [ depending on the relative machine precision and t h e methods involved in forming (K-pM)M-'r, f o r some T .) Below a r e listed the eigenvalues, v k , of T and the absolute values of the bottom elements. I s p k 1 , of the corresponding eigenvectors (k , t h e index, equals the number of the row in the table). The third column contains the absolute values of the growth f a c t o r s (the 7k). The last column contains 1 1 P N I I ~ 1 1 P N~. ^ ~I I i.e. the relative growth of the contamination in the Ritz vectors. (All t h e values have been put in one table, although they should not necessarily be compared row-wise. A yk is not related to a n y specific u i , for example.)
-',
uk
3.1533d-02 3.4399d-02 3.9433d-02 4.6556d-02 5.5515d-02 6.5894d-02 7.71 1 Id-02 8.8464d-02 9.9504d-02 1.1108d-01 1.2500d-01 1.4286d-01 1.6667d-01 2.0000d-01 2.0000d+00
iSpk
I
7.5432d-02 1.8132d-01 2.7978d-01 3.6344d-01 4.2417d-01 4.5223d-01 4.3532d-01 3.5642d-01 2.1 15 Id-0 1 7.2107d-02 1.2412d-02 1.0479d-03 3.8827d-05 4.4893d-07 2.2286d-23
IYk
1 1 PNGk I I
I
1.0000d+00 3.8661d-01 1.7416d+00 3.8896d+00 8.8019d+00 2.0639d+01 5.0314d+01 1.2781d+02 3.3887d+O2 9.3915d+02 2.7241d+03 8.2803d+03 2.6409d+04 8.8501 d+04 3.1213d+05 1.1606d+06
~
~
~
N
4.4725d+04 9.8550d+04 1.3265d+05 1.4596d+05 1.4286d+05 1.2831d+05 1.0555d+05 7.5329d+04 3.9743d+04 1.2137d+04 1.8566d+03 1.3714d+02 4.3556d+00 4.1967d-02 2.0834d-19
6
1
~
~
In theory PNg = 0 but in this case it is of the order of lo-'' - la-" but t h a t is not of any r e a l significance since the problem is so trivial. Looking a t columns t h r e e and four it c a n be seen how the Lanczos vectors and t h e Ritz vectors a r e affected (theorems 3.5 and 3.6). In particular, it c a n be seen t h a t t h e good Ritz vectors a r e not affected very much.
3.3.4.2. Nonsingular M. Here one change was made, now changed to M fii
E
=
The eigenpairs of the subproblem (*) have
(4.999875.10-', (9.999875.10-', -5.000062.10-1)T) a n d (2.00 0 0 50.10 4 , (5.00 0 0 6 2.10-3,9.99 9 8 75.10 ') T,
(Which c a n be compared with the theory in section 3.3.3: U E - ' = 2.104 and /zNlj l ~ R 1 1 - 1M 2.104.) The [ in v1 does matter in this case (since it is measured by the M-norm) and we took = 0.4. The table below consists of v k a n d ispk 1 a s before. The third c o ~ u m n contains the growth factors as defined in section 3.3.3. The next two columns contain the relative growths of 5 in the Ritz vectors and the n o r r n a l i s e d g-vectors, i.e. using t h e notation in section 3.3.3 the columns contain 1Ipzyk 1 1 llppl 11-l and /)PzgkI/I/Ppll/-l(wherevhasbeennormalisedsothat ]Iglln= 1).
T. Ericsson
114
vk
5.0880d-05 3.1753d-02 3.5610d-02 4.2256d-02 5.1327d-02 6.2274d-02 7.4399d-02 8.6874d-02 9.8957d-02 1.1 101d-01 1.2500d-01 1.4286d-01 1.6667d-0 1 2.0000d-0 1 2.0001d+00
bpk
I
1.291ld-02 9.5617d-02 2.1961d-01 3.2745d-0 1 4.0956d-01 4.5697d-0 1 4.5933d-01 4.0 196d-0 1 2.7082d-0 1 1.1076d-01 2.2542d-02 2.1628d-03 8.8950d-05 1.1252d-06 7.5966d-23
I7k
I I p2Yk 1 1 I I P P 1 II
I
1.0000d+00 3.8646d-0 1 1.7402d+00 3.8842d+00 8.7839d+00 2.0580d+01 5.0098d+01 1.2652d+02 3.2382d+02 7.2302d+02 9.1762d+02 5.3057d+02 1.9481d+02 6.2529d+01 1.8886d+0 1 5.4101d+00
1.3463d+03 2.7726d-01 5.6772d-01 7.1321 d-0 1 7.3425d-01 6.7511d-01 5.6793d-01 4.2558d-01 2.5 171d-01 9.1768d-02 1.6585d-02 1.3922d-03 4.9077d-05 5.1731d-07 3.49 17d-24
I I PSVk I I 1 I p2v 11I
2.9890d+02 4.3600d-04 7.9278d-04 8.3667d-04 7.0076d-04 5.3787d-04 3.7959d-04 2.4418d-04 1.2704d-04 4.1327d-05 6.6339d-0 6 4.8726d-07 1.4723d-08 1.2932d-10 8.7288d-29
Comparing the two examples we see t h a t t h e 7k a r e roughly the same for small values of k . For larger values t h e boundedness forces t h e yk t o decrease in the nonsingular case. If we had t a k e n a smaller ( 1 the values would have followed e a c h other for even larger values of k . Looking a t t h e two last columns one c a n s e e how much b e t t e r fJ is. From section 3.3.3 it follows t h a t t h e quotient, between values in the two columns, is roughly 5.10-5v-’ (except for the first row). (To g e t t h e absolute values of t h e growths (instead of the relative values) use IIPpIjI M 7.4279.10-2 or Iz^TiCLIUl I fil 7.4280.10-4.)
3.3.5. The Defective Case. In this section we will briefly look a t the more complicated situation when ( K , M ) is defective. To make things easier to follow we assume t h a t M = &Lag ( I , 0) and so with o u r previous notation
i 1
= H zQR QRh
We may have unwanted components not only in n (M) b u t also in T (M). The following t h e o r e m describes how these components may grow in t h e v k .The --notation used is t h e s a m e a s in t h e o r e m 3 . 3 . To make the notation easier we have assumed t h a t BP # 0.
A Generalised Eigenvalue Problem and the Lanczos Algorithm
115
and
Since 6 k / tion, instead
E~
i s not constant, the contamination does not stay in the same direc-
pNv^k
=
[ ]
Yk Q N ~ wk
where wk E span ( H z Q N a , b ). From theorem 3.3 we see t h a t the recurrences f o r Y k and &k coincide with the previous r e c u r r e n c e for Yk (in theorem 3.3). SO withg = (yl,...., Tg = -S3p7p+leq. Theorem 3.8 now gives ~ k - l b k - I + $ k b k + B k b k + l = Yk O r with d = (bl,...., 6 p ) , Fd = g -flp6p+lep Using this we could compute P N ~ s . As before the starting vector should be chosen inXwhich can be accomplished by hitting any vector twice by (K-pM)-'M. To refine the Ritz vectors it is n o t enough to a d d a multiple of w , + ~ (that will only ensure t h a t the refined vector lies in T ( ( K - p M ) - ' M ) . If we, however, r u n the Lanczos algorithmp + 1 steps and compute the Ritz vector, y = V p s ,from Vp then
(K-PM)-~H$ = ~ j j + s p $ G p + l (K-pM)-1MVp
=
vp
c1
A T T p +BP +]UP +1ep + I
(1)
(2)
T. Ericsson
116
so ( ( K - p d f ) - ' ~ V ) ~ y=^~ ( K - p B ) - ~ i @ + s ~(& K - ~ J & ) - ' M V ~ +which ~, c a n be computed from (1) and ( 2 ) .
4. A Choice of Norm. When M is positive definite, we have several s t a n d a r d bounds in t h e M-'-norm, e.g. f o r arbitrary y # 0 and p , t h e r e is a n eigenvalue A (where Kx = A M z ) such t h a t
I A - p I l l m l l ~ -II(K-@M)y l~ IlM-1 (see 141 Ch. 15). In our c a s e , when M is singular, this does not hold, of course. It is possible to find relevant norms, I I I I say, also in this c a s e , but t h a t is t h e subject of a coming report. We will, however, quote (without proofs) some of the more practical results from t h a t r e p o r t , and in particular those when W is singular (only positive semidefinite). In t h a t case we have a semi n o r m (there a r e vectors x # 0 s u c h t h a t 115
I / Iy = 0).
When M is singular a n a t u r a l choice of W would, perhaps, be M', the pseudo-inverse of Y. There is, however, a more n a t u r a l choice. If M is ositive definite, t h e n XTMX = I , so M-I = AXT.So one might guess t h a t I I T l l x x ~ = (T XX*r)%could be a reasonable
P
alternative also in t h e c a s e when M is singular. It is not difficult to prove t h a t W = XYT satisfies:
WMW = W , M W M = M when ( K , M ) is nondefective (WMW = W holds in the defective case). So W is a generalised inverse, b u t since WM and MW usually a r e not symmetric matrices, W is n o t in general the pseudo-inverse.
To justify this latter choice of W let us quote the following theorem: Theorem4.1. Given any y
E Rn a n d
IA-PI
r e a l p r there is X = A(K, M ) such t h a t
llMv I l m T g II(K-I,LM)y I I m T
This is not, of course, a particularly practical result, since we do n o t usually know
X.We cannot replace XYT by M', in general, as t h e following example shows. Example. Let K =
1:]. :1 i],
I A-p I 1 1 My I I
a n d y = ( - 1 , 2 - ~ ) ~ Then .
M =
= I 1-p
hold unless p = 1.
1,
and
1 1 ( K - p M ) y 1 1 M+ = 0 ,
and the norm inequality does n o t
If we, however, r e s t r i c t t h e choice of y we c a n regain the inequality: Corollary 4.2. If y E X then IA-pI
llMv I l y c c lI(K-PM)y
IIMt
If y E X then both My and (K-pM)y lie in T ( M ) = T ( M + ) . It may be noted t h a t this bound holds in t h e defective case too. The n o r m of the residual can be computed in practice a t least when M is dia i o d . 1 1 . 1 I M + defines a n o r m on X.In h e following theorem we also use (z , Y ) = ~zTMy which defines a n inner product on X.Using this inner product we c a n get a decomposition of
Q
117
A Generalised Eigenvalue Problem and the Lanczos Algorithm
y into z and e , where z is the eigenvector of the eigenvalue closest t o p and e is a vector orthogonal to z ( ( 2 , e)M = 0). Assuming t h a t y . z , and e have unit length ((5, z ) =~1 , etc.) we can define the error-angle cp such t h a t y = cosp z +sinp e . With these definitions we can prove theorems similar to those in [4] page 2 2 2 .
Theorem4.3. Let y EX, where yTh& = 1, and p be given. Let X be the eigenvalue closest to p and define the gap y = min I X i -p 1 . A' # A
L e t y =coscpz+sincpe,whereKx =hMx,zTMz = l , e T M e = l , a n d e T M x = O . Then the following bound holds for sincp:
I sinv I Y-’ I I (K-PM)Y I I M4 If y equals the Rayleigh quotient, y TKy,it is also true that IA-P I 5 7-' I I (K-PWY I I+; These bounds also hold in the defective case. Using the above theorems we can give bounds on the accuracy of the approximations computed by the Lanczos algorithm. Before applying these bounds to the Lanczos case we prove the following theorem in which the residual of a inverted problem is used to construct another vector f o r the noninverted problem. (This is a slight generalisation of how the vector M was constructed in theorem 3 . 7 . ) From pow on we will only deal with the nondefective problem.
Theorem4.4. Let T = (K-pM)-'My -uy , y T M y = 1, y T& = I I (K-(P+w-1)M)i7 I I M + I I 1 I i!.
w7
Then q(w) is minimised for w o = u+ Ilr Il$u-'
Hi)-'
= 0. Take
g = uy +r
and let
and the minimum value, v(w0), is
IIT I l M ( ~ 2 + I I ~
Proof. Let (v2+11r
US
first note t h a t
Hi)%= I I W I I M + = IlVllnr =
s i n c e y T m = 1. From the definition of r^
5
lI(K-PM)-'h&
IIM f 0
f7 we see t h a t My = ( K - p M ) q , s o
(K-(p+u-')M)V = My -o-'W = (1- u w - ~ ) & -w-’Ah
which gives
1 1 r^ 11*; = (1 - u w - ' ) 2 + W - Z r T m = (w-’(U2+T TMr)?+-u(uZtrT&
)-"Z+
1-u2(
U2+T T M r ) -1
whichisminimisedfor w o = v + u - ' T ~ M T giving I I r ^ I I ~ + = r T A h ( v Z + r T ~ I )-l. (Of course wO1 = gT(K-pM)g/gTA@. Note t h a t u = yTM(K-pM)-'My.) (p+w{’, g ) need not be a better approximation than (p,+u-', y ) . So let us find out when 77(wo) = T
It ( K - W w g ' ) M ) V I I M + I I w I I
= (K-pM)-'IWy-vy
5 I I (K-b+v-')M)y II M + IIM y I I s o (K-(p+u-')M)y = -u-'(K-pM)r and (*) is equivalent to:
1 1 ~ 1 1 ~ ( ~ 2 + 1 l 1l i~ ) - ' s l l ( K - ~ M ) rI I M + 1 4 - ' which is equivalent to I w o I - ' ~ //(K-&Wr l l M + l l M Ilia!
(*)
T. Ericsson
118
In other words, for the inequality (*) to hold, T should have large components in the eigenvectors corresponding to A ( K , M) f a r f r o m y o r I uoI should be large, i.e. g should approximate a n eigenvector corresponding to a n eigenvalue close to y. (In the same way a s was shown in section 3.3.3 we get (provided A ,utu-')
I&?I(I4J@ which shows how
I IIflII)-l
IX-WI
1Ak-PI-l
is affected by other eigenvectors.)
Example. Let K = d i a g ( 5 , 10, loo), M = I , and p = 0. Take y T = (0, 10, l ) l O l - n . In this case we g e t a much better approximation using q T = (0, 10, 0 . 1 ) ~(for some 7). Furthermore, 10-u-’ fil -9.0.10-2, 1O-wO’ W -9.0,10-3, l\(K-(y+u-’)I)y IIy I \ - ' M 8.9, ) V l l M 0.90 and l l ( ~ - ( ~ + ~ O ~ VllVll-l If we, however pick y T = (1, 10, 0)lOl-5 t h e picture changes completely. Now 10-v-' FY 9.8,10-2, 10-wO1 FY 0.19, and g* = (2, 10, 0 ) ~ . II (K-(p+u-l)Z)y 1 1 I I Y 11-1 M 0 . 5 0 , and l I ( K - ( ~ u + w ~ ~ ) I ) f lIIMII-I I FY 0.96 We now apply t h e theorem to the approximations produced by the Lanczos algorithm in the nondefective case (the notation follows t h e theorems above). In this c a s e y = v s and I I T 1 1 = ~ ,3 1s I for some s , so w o = v + B ~ s ~ u -and ' the corresponding q(oo)= a, Isp I (u2+,3;sEf'. %e c a n also see t h a t the changes, when using g , will not be s o dramatic (except f o r the contamination), since:
which would be quite small. Also Iwo-u(
=
IlT
I~~luI-'=fSp2sp2IuI-~
which usually is even less. Since EXwe c a n apply 4.2 and 4.3 and get: Corollary4.5. L e t p = a, I sp I (u2+,3;sE)-'
t h e n sin
I sin p I
is bounded by
I py-'
and there is a n eigenvalue A t h a t satisfies I~-(yu+o;')
I
s min ( p , p2y-')
Acknowledgements. This work was completed during a visit to CPAM a t the University of California, Berkeley. I thank Professor B.N. P a r l e t t for making my stay so pleasant and interesting. This research was supported by the Swedish Natural Science Research Council.
5. References. 113 Ericsson, T., Jensen, P.S., Nour-Omid, B., and Parlett, B.N., How to implement the spectral transformation, Math. of Comp., to appear.
A Generalised Eigenvalue Problem and the Lanczos Algorithm
119
[2] Ericsson, T. a n d Ruhe, A,, The spectral transformation Lanczos method for the numerical solution of large sparse generalised eigenvalue problems, Math. of Comp. 35:152 (1980) 1251-1268. [3] Moler, C., An interactive matrix laboratory, Tech. rep., Univ. of New Mexico (1980, 2nd ed).
[41 Parlett, B.N., The symmetric eigenvalue problem (Prentice Hall Inc., 1980) [5] Uhlig, F., A recurring t h e o r e m about pairs of quadratic forms and extensions: A survey, Lin. Alg. Appl. 25 (1979) 219-237.
[6] Wilkinson, J.H.. The algebraic eigenvalue problem (Oxford U.P., 1965). [ 7 ] Wilkinson, J.H., Kronecker's canonical form a n d t h e QZ algorithm, Lin. Alg. Appl. 28
(1979) 285-303.
This manuscript was prepared using troff a n d eqn a n d printed on a Versatec printer using a 11 point Hershey font.
This Page Intentionally Left Blank
Large Scale Eigenvalue Problems J. Cullum and R.A. Willoughby (Editors) 0 Elsevier Science Publishers B.V. (North-Holland), 1986
121
NUMERICAL PATH FOLLOWING AND EIGENVALUE C R I T E R I A FOR BRANCH SWITCHING
Yong Feng Zhou* and Axel Ruhe** *Wuhan D i g i t a l E n g i n e e r i n g I n s t i t u t e , P . O . Box 2 2 3 , Wuchang , Wuhan , P e o p l e ' s Republic of China **Department o f Computer S c i e n c e , Chalmers U n i v e r s i t y o f Technology, 4 1 2 9 6 Goteborg, Sweden
Methods f o r n u m e r i c a l p a t h f o l l o w i n g f o r n o n l i n e a r +?tract: e i q e n v a l u e problems a r e s t u d i e d . E u l e r Newton c o n t i n u a t i o n a l o n g c u r v e s p a r a m e t e r i z e d by a s e m i a r c l e n g t h i s d e s c r i b e d . C r i t e r i a f o r localizing singular points (turning points o r b i f u r c a t i o n s ) by means o f a l i n e a r eigenproblem a r e i n t r o d u c e d . I t i s found t h a t a n o n l i n e a r v e r s i o n o f t h e s p e c t r a l t r a n s f o r m a t i o n used f o r l i n e a r symmetric eigenproblems g i v e s a s u r p r i s i n g l y a c c u r a t e p r e d i c t i o n of t h e p o s i t i o n o f a s i n g u l a r p o i n t and t h e d i r e c t i o n o f b i f u r c a t i n g b r a n c h e s . P r a c t i c a l a p p l i c a t i o n s a r e d i s c u s s e d and n u m e r i c a l examples are r e p o r t e d .
I n t h e p r e s e n t c o n t r i b u t i o n w e w i l l seek s o l u t i o n s of g(u,X)
(1.1)
=o
where
u
and X
i s a real p a r a m e t e r . Such problems are termed n o n l i n e a r e i g e n -
and t h e n o n l i n e a r f u n c t i o n
g
a r e v e c t o r s of n dimsnsions,
v a l u e problems o r p a t h f o l l o w i n g problems. The m a n i f o l d of s o l u t i o n s t o ( 1 . 1 ) i s i n g e n e r a l of one dimension, and w e g e t a c u r v e o r p a t h t o f o l l o w . When s e v e r a l p a t h s meet w e g e t branch p o i n t s , most o f t e n termed b i f u r c a t i o n s .
L e t u s f i r s t c o n s i d e r some examples. The { l i n e a r ) e i g e n v a l u e problem
(1.2)
g(u,X)
where A and B
5
(A-XB)u, a r e symmetric m a t r i c e s , and B
is positive d e f i n i t e ,
h a s one s o l u t i o n p a t h
r0 =
{ u =0 , X a r b i t r a r y }
which w e c a l l t h e t r i v i a l s o l u t i o n . A t e a c h e i g e n v a l u e A = X k , t h e r e
Yon, Feng Zhou and A . Ruhe
122
i s a bifurcation with a straight line path s e t t i n g out i n the direct i o n of t h e e i g e n v e c t o r ,
rk =
C U = U ~ C,
x = xk 1 ,
where
uk e i g e n v e c t o r ,
(A
-
XkB)uk = 0
c arbitrary, See f i g u r e 1 . 1
1c
r
u ' t
t
G
7
J 13
h,’ A 1
Figure 1.1
When g i s n o n l i n e a r , i t is o f t e n n o r m a l i z e d s o t h a t g ( 0 , O ) = 0 , and w e c a l l t h e b r a n c h s t a r t i n g a t t h e o r i g i n t h e t r i v i a l b r a n c h . W e now g e t c u r v e d b r a n c h e s , and may g e t a more c o m p l i c a t e d s t r u c t u r e w i t h s e c o n d a r y b i f u r c a t i o n s . T u r n i n g p o i n t s , where a s o l u t i o n has t h e hyperplane
X = c o n s t a n t a s a t a n g e n t p l a n e , a r e a l s o of i n t e r e s t . The
s o l u t i o n m a n i f o l d may a l s o be d i s c o n n e c t e d i n t o s e v e r a l p a r t s or i s o l a s . See f i g u r e 1 . 2 .
123
Numerical Path Following and Eigenvalue Criteria for Branch Switching
t
A
Figure 1 . 2 One i m p o r t a n t s o u r c e o f n o n l i n e a r e i g e n v a l u e p r o b l e m s i s l a r g e d e f o r m a t i o n and b u c k l i n g p r o b l e m s i n m e c h a n i c s , see e.9.
[21.
Consider t h e
v e r y s i m p l e s t c a s e o f Euler buckling of a r o d ,
(1.3)
Here
I [ B ( s )8 ' 1 ' + p
sin0 = 0
8 s t a n d s f o r t h e angular d e f l e c t i o n i n t h e p o i n t s. B ( s ) i s t h e
s t i f f n e s s i n t h e p o i n t g i v i n g t h e b e n d i n g moment
The l o a d p , w i t h w h i c h t h e r o d i s c o m p r e s s e d , i s t h e e i g e n v a l u e p a r a -
meter. T h e t r i v i a l s o l u t i o n i s 0 = 0 , c o r r e s p o n d i n g t o a s t r a i g h t rod. I f w e l e t p i n c r e a s e from 0 , t h e r o d k e e p s s t r a i g h t u n t i l p = A , , where it may b u c k l e i n t o t h e f i r s t b u c k l e d s t a t e ( a n U
shaped r o d ) .
I f i t d o e s n o t b u c k l e it s t a y s s t r a i g h t u n t i l p = A 2 , where i t may b u c k l e i n t o t h e s e c o n d b u c k l e d s t a t e (an S s h a p e d r o d ) , and so o n , see f i g u r e 1.3.
124
YongFengZhouandA. Ruhe
Figure 1.3 I n a p r a c t i c a l s i t u a t i o n , t h e o c c u r r e n c e of b u c k l i n g i s g o v e r n e d by small imperfections i n t h e material o r load application, more g r a d u a l o n s e t o f b u c k l i n g , as t h e d o t t e d
leading t o a
l i n e i n f i g u r e 1.3
i n d i c a t e s . W e see t h a t b i f u r c a t i o n i s non g e n e r i c , b u t s t i l l of i n t e -
r e s t i n t h e m a t h e m a t i c a l s t u d y o f t h e m e c h a n i c a l problem. A n o t h e r i m p o r t a n t s o u r c e o f b i f u r c a t i o n problems i s t h e s t u d y o f s t e a d y s t a t e s o f d y n a m i c a l s y s t e m s . A s a s i m p l e c a s e c o n s i d e r a chemical system
ct = F ( c ) + k
(1.4)
where
c
Ac
i s a vector of concentrations of d i f f e r e n t species, F ( c ) a
n o n l i n e a r f u n c t i o n , c o r r e s p o n d i n g t o c h e m i c a l r e a c t i o n s , and A c
is
t h e Laplace o p e r a t o r , corresponding t o d i f f u s i o n . A t s t e a d y s t a t e c t =0, and w e c a n s t u d y t h e s h a p e o f t h e f i n a l s o l u t i o n v e c t o r c f o r d i f f e r e n t v a l u e s o f t h e d i f f u s i v i t y k , which w e now u s e as eigenvalue parameter.
See 131.
F i n a l l y homotopy methods f o r m i n i m i z a t i o n l e a d s t o a s p e c i a l t y p e o f path following [ I ] . (1.5) where cp
C o n s i d e r t h e problem
min ~ ( u )
u i s a r e a l v a l u e d f u n c t i o n . Suppose t h a t w e know t h e minimum
125
Numerical Path Following and Eigenvalue Criteria for Branch Switching
u0
of a n o t h e r
function
'po
,
c o n s i d e r e d as an a p p r o x i m a t i o n t o
U,.
I n t h e s i m p l e s t c a s e we can d e f i n e
Define t h e p a t h o f f u n c t i o n s $ ( u , A ) :=A(cp(u) - c p ( u o ) ) For
A=O
t
(1-A)
t h e minimum i s a t u = u o ,
s o u g h t minimum o f
((P0(U)
and f o r
-cpo(uo)) A=l
it i s a t
u=u*, the
( 1 . 5 ) . Along t h e p a t h i t h o l d s t h a t
One way o f f i n d i n g u * i s now t o f o l l o w a p a t h of s o l u t i o n s of from X = O
to
(1.6)
Such a p a t h may n o t e x i s t i f t u r n i n g p o i n t s
X = l .
o c c u r , b u t t h e r e i s a t e c h n i q u e of c o m p l e x i f i c a t i o n which g i v e s b r a n c h e s b i f u r c a t i n g o u t i n t h e complex p l a n e and t h e n j o i n i n g b a c k a g a i n . See f i g u r e 1 . 4 .
U
$=O Figure 1.4
l=t
Yong Feng Zhou and A . Ruhe
126
The p u r p o s e o f o u r work i n t h i s a r e a i s t o u s e i d e a s t h a t have p r o ved s u c c e s s f u l f o r e i g e n v a l u e c o m p u t a t i o n f o r l i n e a r p r o b l e m s ( S e e 181)
t o c o n s t r u c t a l g o r i t h m s f o r p a t h f o l l o w i n g . T h i s r e p o r t is j u s t
a f i r s t t i n y s t e p o n t h a t way, b u t t h e r e s u l t s t h a t h a v e shown up have e n c o u r a g e d us t o g o f u r t h e r . W e c o n t i n u e i n s e c t i o n 2 by d e s c r i b i n g t r a d i t i o n a l a l g o r i t h m s f o r
p a t h f o l l o w i n g , e s s e n t i a l l y t h e Euler-Newton c o n t i n u a t i o n method.
In
s e c t i o n 3,we d e s c r i b e how e x a c t f a c t o r i z a t i o n s of t h e m a t r i x o f p a r t i a l d e r i v a t i v e s i n one p o i n t , c a n be u s e d a s a p r e c o n d i t i o n i n g i n an i t e r a t i v e s o l u t i o n of t h e l i n e a r s y s t e m s o f t h e Newton i t e r a t i o n , i n l a t e r points.
I n s e c t i o n 4,we
a r r i v e a t t h e problem t h a t i s o u r p r i -
mary c o n c e r n , l o c a t i n g s i n g u l a r p o i n t s . W e show how t h e s o l u t i o n of a l i n e a r eiqenproblem g i v e s b o t h a p r e d i c t i o n o f t h e p o s i t i o n of t h e s i n g u l a r p o i n t s and t h e d i r e c t i o n o f t h e b i f u r c a t i n g b r a n c h e s . W e conc l u d e w i t h s o m e n u m e r i c a l examples i n s e c t i o n 5.
127
Numerical Path Following and Eigenvalue Criteria for Branch Switching
W e s e e k t h e s o l u t i o n of t h e n o n l i n e a r e i g e n v a l u e problem
G(u,A) = O
(2.1)
where
uER
into Rn
n
,
AER
1
i s a n o n l i n e a r t r a n s f o r m a t i o n from Rn x R1
G
I
and assume t h a t
The s t a n d a r d a p p r o a c h i s
dim N ( G U ) = 1
t o use
X I
if
GU is singular.
o n e of t h e n a t u r a l l y o c c u r r i n g
p a r a m e t e r s of t h e p r o b l e m , a s t h e p a r a m e t e r d e f i n i n g s o l u t i o n a r c s , u(X)
.
( u o , Xo)
Those
satisfying
(2.11,
f o r which t h e d e r i v a t i v e Gu
i s n o n s i n g u l a r , a r e r e g u l a r p o i n t s on t h e s o l u t i o n a r c .
is singular,
(uo,Xo)
singular points, i f
If
GU(uO,XO)
i s a s i n g u l a r p o i n t . To f u r t h e r d i s t i n g u i s h
GX$K(Gu)
it i s a t u r n i n g p o i n t , i f
GXER(G ) U
it
is a b i f u r c a t i o n p o i n t . I f w e u s e Euler-Newton c o n t i n u a t i o n t o ( 2 . 1 ) d i r e c t l y , t h i s p r o c e d u r e may f a i l o r e n c o u n t e r d i f f i c u l t i e s a s a s i n g u l a r p o i n t i s a p p r o a c h e d . See [111
f o r f u r t h e r d e t a i l s . T h e s e d i f f i c u l t i e s are o v e r c o m e , f o r
t h e c a s e of t u r n i n g p o i n t s , i f w e i n t r o d u c e a n o t h e r parameter
s
and
impose some a d d i t i o n a l c o n s t r a i n t o r n o r m a l i z a t i o n o n t h e s o l u t i o n . Replace ( 2 . 1 ) b y
and g e t a n e q u i v a l e n t (2.3)
For (2.3) Step 1 .
i n f l a t e d system
F(x,s) = 0
,
t h e Euler-Newton c o n t i n u a t i o n i s : ( F i n d t a n g e n t ) Solve
Yong Feng Zhou and A . Ruhe
128
Step 2 .
(Euler s t e p ) ' Set
(2.5)
x 0 ( s ) = x ( s 0 ) + ( s - s ~* ); ( s o ) = x ( s 0 )+ Aso*X(s0) .
Step 3 .
(Newton i t e r a t i o n ) Solve
(2.6)
Fx(x
V
( s ), s ) * ( x " + ~ (s)-x
and i t e r a t e f o r Here
kIG,i
V = Oll,
V
V
( s ) )=-F(x (s) ,s)
... .
express derivatives with respect t o s.
Some d i f f e r e n t n o r m a l i z a t i o n s a r e : S
11 X ( T ) I /
d?
-
(S-So)
N2 (X,S) = ; ( ( s o ) ( X ( s )
-
X(So))
-
-x(so)) -
qk
N1 ( x , s )
=
T
(2.7)
N ~ ( X ' S )=
N4(x,s) =
e;T
11
-
(x(s)
X(S)
- x ( s 0 ) 11 2 -
(s-so)
See f i g u r e 2 . 1
U
h
u
Figure 2 . 1
(S-So)
(See [ I l l ) (See [ 1 4 1 )
2
(See [ 6 1 )
129
Numerical Path Following and Eigenvalue Criteria f o r Branch Switching
The t r u e arc l e n g t h and W.C.
Rheinboldt
N,
i s d i f f i c u l t t o e v a l u a t e . H.B.
Keller
1 1 4 1 have u s e d t h e p s e u d o a r c l e n g t h s N 2
[I11 and N 3
t o form t h e i r a l g o r i t h m s , r e s p e c t i v e l y . S o l v i n g a l i n e a r s y s t e m by a d i r e c t method i n e a c h i t e r a t i o n i s u n f a v o r a b l e f o r l a r g e problems. D i f f e r e n t i t e r a t i o n s have been proposed. For e x a m p l e , T.F.
Chan and Y .
S a a d [ 5 1 h a v e u s e d a n I n c o m p l e t e Ortho-
g o n a l i z a t i o n Method w i t h p r e c o n d i t i o n i n g , H.Weber [ I 8 1 h a s u s e d MG t e c h n i q u e t o s o l v e a n o n l i n e a r e l l i p t i c e i g e n v a l u e p r o b l e m ; T.F. and K.R. s3.
Chan
J a c k s o n 1 4 1 h a v e p r o p o s e d a new p r e c o n d i t i o n i n g t e c h n i q u e . Mixed Method f o r System S o l u t i o n i n N u m e r i c a l P a t h F o l l o w i n g ............................................................
The I n c o m p l e t e O r t h o g o n a l i z a t i o n Method ( I O M )
,
a s p r o p o s e d by Y.Saad
[ 1 6 1 , i s an i t e r a t i v e p r o c e d u r e f o r s o l v i n g a s y s t e m . F o r l a r g e p r o b -
l e m s t h e main work i n e a c h i t e r a t i o n of IOM i s one m a t r i x - v e c t o r mult i p l i c a t i o n . The number o f i t e r a t i o n s i s r e l a t e d t o t h e c o n d i t i o n number o f t h e c o e f f i c i e n t m a t r i x . T h e r e f o r e some p r e c o n d i t i o n i n g t e c h n i q u e i s needed. Suppose t h a t t h e o r i g i n a l s y s t e m i s (3.11
Ax=b.
W e want t o f i n d a n o n s i n g u l a r m a t r i x
M
so t h a t t h e s y s t e m
which i s e q u i v a l e n t t o ( 3 . 1 ) , c a n be s o l v e d i n f e w e r i t e r a t i o n s i f (3.3)
3 ( M - ~ A ) < 2 (A)
W e have c h o s e n t o LU f a c t o r i z e t h e m a t r i c e s Gu and F,
i n some s e l e c t e d
p o i n t s , and t h e n use t h o s e LU f a c t o r i z a t i o n s as p r e c o n d i t i o n i n g i n a s e q u e n c e o f p a t h f o l l o w i n g s t e p s . When w e h a v e come f a r from t h e
f a c t o r i z a t i o n p o i n t , many i t e r a t i o n s w i l l be needed i n IOM,and t h e n
w e s e l e c t a new f a c t o r i z a t i o n p o i n t .
Yong Feng Zhou and A . Ruhe
130
W e h a v e , q u i t e a r b i t r a r i l y , chosen t o
p r e s c r i b e d v a l u e %ax
refactorize
when ION needed a
i t e r a t i o n s t o converge. I n o u r t e s t s
sax was
chosen a s 15 i n most c a s e s . The convergence of t h e o u t e r Newton i t e r a t i o n ( 2 . 6 ) i s a f f e c t e d by t h e s t e p l e n g t h chosen i n 12.5). and R.
C.D.
H e i j e r and W.C.
Rheinboldt 191
Seydel [ I 7 1 have s t u d i e d t h i s problem. W e have used a method
s i m i l a r t o t h a t proposed by R .
Seydel [ I 7 1
because it i s s i m p l e and
has shown t o be e f f e c t i v e . Our s t r a t e g y i s : (1)
The d e s i r e d number of o u t e r Newton i t e r a t i o n s ( D N I ) f o r ( 2 . 6 ) i s given a p r i o r i , t h e f i r s t s t e p length A s o
(2)
For i = 1 , 2 ,
i s known.
...
where N I i s t h e a c t u a l number of Newton
i t e r a t i o n s t h a t was
needed t o r e a c h xi. (3)
I f t h e s o l u t i o n of ( e . g . 1 5 ) w e set
(4.1)
( 2 . 6 ) d o e s n o t converge a f t e r many i t e r a t i o n s
and go back t o t h e E u l e r s t e p ( 2 . 5 ) .
As.:=Asi/2
_Algorithm _ _ _ _ _A _- _d e_t e_r m_i n_i n_g _t u_r n_i n-g - p-o i n r s - i n
I t i s an obvious f a c t t h a t
c o ~ t ~ n ~ a ~ i ~ n L
a t a t u r n i n g p o i n t . I n numerical
i(s) = 0
x ( s ) by s o l v i n g (2.4) a t e a c h p o i n t . W e monitor t h e s i g n of i t s last component i ( s ) . I f x ( s ) x ( s + l ) < 0 it means
continuation w e g e t
-
t h a t t h e r e i s a t least one t u r n i n g p o i n t on t h e c u r v e segment ( s , s + l ) . Then i f needed w e c a n s e a r c h f o r t h e s i n g u l a r p o i n t u n t i l
- _d e- t-e r-m- i -n i-n -g
(4.2) _Algorithm _ _ _ _ _B _ i n v e r s e i t e r - - - - - - -a t-i o-n .
A t a general singular point
x(s)
so now t h e s m a l l e s t e i g e n v a l u e
Amin
E.
s i n g u l z r_ p_o i n t s b y - m ~ a ~ s - o ~
d o e s n o t n e c e s s a r i l y change s i g n , of
T h i s i d e a formed t h e c o r e of Algorithm B. i n v e r s e power method a t e a c h p o i n t .
1 A (s*) 1 <
has t o be m o n i t o r e d . Xmin was computed by t h e
Gu
Numerical Path Following and Eigenvalue Criteria f o r Branch Switching
131
For solving systems of order n iteratively in the inverse method and solving a system of order n+l ((6.10) in [ 1 1 1 ) the Mixed Method described in § 3 was used. (4.3)
.Algorithm . . . . .C .- .localize . . . . singular . . . . . points . . . .by. iteration . . . . . matrix. . . .
In the tests we found that computing ’min required a large part of the work, especially near the singular point. In Algorithm C we use the vectors computed by the iterative method to compute a Galerkin approximation to the smallest eigenvalue. Assuming the Arnoldi method is used in the iteration we get:
1. 2.
Choose v1
appropriately
4.
(4.4)
v1 1 1 2
= 1
.
Form subspace Vm by Arnoldi algorithm applied to starting at v, , M-l GU Vm
3.
11
=
M-lGu
T VmHm + hm+, ,m vm+l ern
T Let Am = Vm-GUVm eigenpair Amin,
=
T Vm M VmHm (order m Of Am .
and get the smallest
-
Let Amin = Amin, +=V, z , check the condition 11 Gub / I 2 < c . If is taken as the smallest eigenpair of G,. it holds, Xmin,r$ If not, Algorithm C fails and we change to use Algorithm B.
_Localizing _ _ _ _ _ _singular _ _ _ _ _point _ _ _by_ _linear _ - - _eigenproblem. ____
Along a solution path, GU can be considered as a function of the path following parameter s . We can then perform a Taylor expansion
and get a first order approximation to the singular point as s1 = s o w
132
YongFengZhouandA.Ruhe
where
1-1 i s a s o l u t i o n o f t h e l i n e a r e i g e n p r o b l e m
Moreover, i f
GU
i s symmetric a n d GU
i s p o s i t i v e ( o r n e g a t i v e ) defi-
n i t e , w e c a n u s e t h e i n e r t i a t h e o r e m t o f o l l o w how many s i n g u l a r points we pass while following t h e path f o r increasing s
.
In fact
l e t u s c a l l it N N E ( s ) , t h e number of n e g a t i v e e i g e n v a l u e s o f GU(s), a s LDLT when s o l c a n b e found as a by p r o d u c t w h i l e f a c t o r i z i n g GU
ving l i n e a r systems. Thus, i f N N E X ( S , ) - N N E x ( s 2 ) = be oEe e i g e n v a l u e o f
Moreover, l e t
s l = s o -p k
where
( 4 . 3 ) a t the point x(sdand
$*
a real singular point,
(4.4)
GU
(4.5)
GU
(4.6)
If
s,-sB
(4.9)
GU
sl,
s2Er,
it means t h a t t h e r e must
i n t h e i n t e r v a l (sl
GU(s)
(2.1 )
eU, GU
,
,
s,)
,
i . e . one s i n -
.
gular point of
that
1
Qk
i s t h e smallest s o l u t i o n of
pk
b e i t s c o r r e s p o n d i n g v e c t o r ; l e t SB be i s t h e n u l l v e c t o r o f GU(SB) a n d s u p p o s e
a r e bounded n e a r sB and
s,.
W e have
SB)0*= 0 ,
is s m a l l
@ k = @ *c
n
i s small.
Let
+&c2
where From ( 4 . 8 ) and ( 4 . 9 ) , w e h a v e
2
and
$"-@*=
0
Numerical Path Following and Eigenvalue Criteria f o r Branch Switching
133
4
is not small. Because $ is orthogonal to N(GU(sB)) , IIGU(sB) Therefore c2 is small. In fact, c2 =sin ( $ (($k,($*)) [13]. Thus +k is a good approximation of
.
+*
If sl-sB is not small, we cannotget the singular point accurately, but can still switch to another branch,provided sl-sB is not too * large,because just a rough approximation to Q is needed for switching. ylToLiit_hm_ D_. problem)
(Localize approximate singular points by linear eigen-
Step 1
Start with xo
(1 . I )
Solve (2.4) to get
(1.2)
Solve (4.3) at
(1.3)
A
Step 2 points
Compute path
0
:=
A.
xo
xo
to get N N E ( X ~ )
- PNNE(xO)(x,)
A.
xO,xl,....~k by Mixed Method in 53. Find two and NNE(xs) - NNE(xs+,) = 1. As < A 0 I A s + l
such that
xs’xs+l
Step 3 1
T [ (As
- "NE(xs)
. As)
(3.1)
AO: =
(3.2)
0 A s : = ( A -As)
/ A s , u 0 : = uS
(3.3)
Using
($ )
xo =
singular point Step 4
&S*)
Remarks:
=
+AS
+
('S+l -pNNE(xs) ( X s + ~ ) * A s + ~ ) l
us
as the initial guess, get the approximate
*
x(s )
If necessary, take
null vector of
(xs)
by Newton Iteration.
($NNE(xs)(xS) as the left and right
G ~ ( s *,) l e t
(X(S*)- x(s)1 / v"E(xs)
(x,)
to switch branches at the bifurcation point.
1. It is not needed to compute NNE at each point in the algorithms above.
Y o n g F e n g Z h o u a n d A . Ruhe
134
2.
In (4.3)
3.
W e c a n n o t i t e r a t e s t e p 3 of Algorithm D
as i n [ 1 5 1 because
no improved u v e c t o r i s a v a i l a b l e w i t h o u t Newton i t e r a t i o n . However o n l y a moderate a c c u r a c y i s needed f o r branch s w i t c h i n g anyhow. (4.5) 1.
Remarks : -
L
If
-
-
G,
h a s a z e r o ( o r s m a l l enough) e i g e n v a l u e n o t h i n g mo-
r e i s needed. 2.
In t h e searching procedure i n t e r v a l halving
(As:=As/2)
. -.
i s a s i m p l e and f e a s i b l e m e t h o d f i t s convergence i s q - l i n e a r . W e can a l s o u s e a s e c a n t method (As:= l1/ ( ( A 2 A 1 ) / ( s 2- s 1 ) ) ) w i t h a
-
s u p e r l i n e a r r a t e o f convergence. The comparisons between them i n p r a c t i c e a r e i n Table 1 i n 55. 3.
TO s o l v e t h e system ( 2 . 4 )
Step 1
Solve
Step 2
Gu
- y =-G A
i= l *
1
/
w e do t h e f o l l o w i n g :
f o r y. (4.11)
~
1+11 YII
Step 3
G = y *X
.
I n s t e p 2 t h e r e i s a problem a b o u t how t o choose t h e s i g n o f t h e s q u a r e r o o t . W e u s e a h e u r i s t i c method. Suppose t h a t w e want t o d e t e r m i n e
is a t xs now, w e choose t h e a p p r o p r i a t e s i g n i n ( 4 . 1 1 ) s o t h a t i s-1 T k s > 0 , t h e r e f o r e t h e c o n t i n u a t i o n i s i n g e n e r a l i n t h e forward direction.
Numerical $5: __-----__-_ _ _ - _ _ _Examples _____ __ ----------_---______________ W e have tested t h e a l g o r i t h m s d e s c r i b e d h e r e . W e have used a n IBM3081
computer a t Gateborg U n i v e r s i t y Computing C e n t r e . I n a l l r u n s w e have chosen t h e n o r m a l i z a t i o n
N2
in (2.7)
and s t o p p e d t h e i t e r a t i o m
when a p r e s c r i b e d a c c u r a c y was o b t a i n e d 1.
Example 1 was t a k e n from [ 1 2 ] . The problem was
135
Numerical Path Following and Eigenvalue Criteria for Branch Switching - A u = X e x p ( u / ( l + & u ) ) ,e , O ,
in
a=
i n aR
u = o
(0.1)
(0.1)
. f o r m u l a and t h e n e t
The d i f f e r e n c e scheme u s e d was t h e f i v e - p o i n t spacing w a s
X
g i v i n g a d i s c r e t e problem of o r d e r 8
h = 0.1,
Results f o r d i f f e r e n t
6
found by Euler-Newton method ( 2 4 )
- (2.6)
w i t h A l g o r i t h m A a r e shown on F i g 5 . 1 . W e see t h a t when O <
E
< 0.24065
t h e c u r v e h a s two t u r n i n g p o i n t s , t h e l a r g e r
t h e c l o s e r t h e t w o t u r n i n g p o i n t s a r e ; when
no t u r n i n g p o i n t . F o r
E
= 0,
E
> 0.25
is,
E
t h e curve has
t h e only turning point w a s
( u ( 0 . 5 , 0 . 5 ) ,A) = ( 1 . 3 8 , 6 . 7 9 ) . Some s t a t i s t i c s o f t h e r u n s f o r t = 0 a r e g i v e n i n T a b l e 1 . The t u r n i n g p o i n t w a s l o c a l i z e d by e i t h e r a l i n e a r s e a r c h , r e p o r t e d on t h e f i r s t l i n e , o r a s e c a n t method, r e p o r t e d on t h e s e c o n d . W e have a l s o u s e d Euler-Broyden method, a s d e s c r i b e d i n 1 7 1
t h i s example f o r
E
= 0.
,
for
W e l i s t t h e r e s u l t s on t h e t h i r d and f o u r t h
l i n e s i n T a b l e 1 f o r c o m p a r i s o n w i t h t h e Euler-Newton method. Evid e n t l y t h e p e r f o r m a n c e o f E u l e r Broyden i s c o m p a r a b l e t o t h a t of E u l e r Newton a n d t h e f o r m e r i s c e r t a i n l y t h e a l g o r i t h m t o be P r e f e r r e d i n s e v e r a l cases. T h i s q u e s t i o n d e s e r v e s f u r t h e r s t u d y .
Note:
TNP denotes t h e t o t a l number of p o i n t s during c o n t i n u a t i o n from t h e s t a r t p o i n t t o t h e end p o i n t .
NDM denotes t h e number of s t e p s of the d i r e c t method.
NIM denotes t h e number of s t e p s of t h e i t e r a t i v e method.
NSP denotes the number of p o i n t s t e s t e d when searching f o r a t u r n i n g p o i n t . Table 1 .
Y o n g F e n g Z h o u a n d A . Ruhe
136
o0.L
ao
15
tg
5
Figure 5.1
Numerical Path Following and Eigenvalue Criteria f o r Branch Switching 2.
Example 2 [ 1 1 1
Consider t h e u where
137
xx
2 - p o i n t boundary v a l u e p r o b l e m +
f (x,u;X) = 0 , u ( 0 ) = u ( 1 )
f(x,u;X) q(h) p(2)
=
0
= 2 q ( X ) + n2 X p ( u - q ( X ) x ( l - x ) )
= - A2 e -A/2, 2-z2
- 2 3 - . .. - z 8
-22
9
W e h a v e u s e d a 4 t h o r d e r a c c u r a t e d i f f e r e n c e a p p r o x i m a t i o n [ I 0 1 and h = 5 oL , y i e l d i n g a nonsymmetric d i s c r e t e p r o b l e m o f o r d e r 4 9 .
The s i n g u l a r p o i n t s s h o u l d be Ak=k2, k = 1,2, Taking -(2.6)
... .
v 1 = u i n A l g o r i t h m C w e have u s e d Euler-Newton method (2.4) t o compute t h i s problem w i t h t h e i n i t i a l p o i n t (u,A) = ( 0 , O ) .
See F i g 5.2.
Some s t a t i s t i c s of t h e r u n s a r e g i v e n i n T a b l e 2 .
W e see t h a t t h e s t e p s o f t h e d i r e c t method o c c u p i e d a v e r y s m a l l
p r o p o r t i o n of t h e w o r k . Table 2
N o t e : T N S denotes the t o t a l number of singular p o i n t s . The meanings of TNP, DNM and NIM are the same as above.
Yong Feng Zhou and A . Ruhe
138
R e c a l l t h a t t h e o n l y d i f f e r e n c e between A l g o r i t h m B and C
i n 54 is
i n t h e s e a r c h i n g p r o c e d u r e s . W e l i s t t h e amount o f work i n t h e two s e a r c h i n g p r o c e d u r e s i n T a b l e 3. W e see t h a t A l g o r i t h m
C needed s u b s t a n t i a l l y less w o r k , s i n c e o n l y
a v e r y s m a l l m a t r i x Am w a s g e n e r a t e d , w h i l e A l g o r i t h m B needed sev e r a l l o n g IOM r u n s t o p e r f o r m t h e i n v e r s e i t e r a t i o n . The a d d i t i o n a l work o f A l g o r i t h m t i o n t o form
C
a t each p o i n t t e s t e d i s
mN2 + m2N
multiplica-
A m = V m GuVm.
T
Note: N B I ( N B 2 )
d e n o t e s t h e number of s y s t e m s s o l v e d ( I O M i t e r a t i o n
s t e p s ) i n t h e i n v e r s e power method f o r f i n d i n g t h e smallest e i g e n v a l u e of GU u s i n g A l g o r i t h m B ,
rn i s t h e o r d e r of
pB
and
pc
when u s i n g A l g o r i t h m
C,
d e n o t e t h e s m a l l e s t e i g e n v a l u e s of
computed by A l g o r i t h m s B , C Table 3 .
Am
respectively.
GU
Numerical Path Following and Eigenvalue Criteria for Branch Switching
139
Figure 5.2
3.
W e s t i l l u s e d Example 2 . Using a 2nd o r d e r a c c u r a t e d i f f e r e n c e
a p p r o x i m a t i o n [ I 0 1 w e g o t a symmetric d i s c r e t e problem o f o r d e r 4 9 . H e r e GU(s) i s symmetric a n d k u ( s ) i s d i a g o n a l . For e a c h r e g u l a r p o i n t on
ro
& u ( s ) i s n e g a t i v e d e f i n i t e . Thus A l g o r i t h m D i n 54
c a n be u s e d f o r i t . The c o n t i n u a t i o n o f
S t a r t point u(0.5)=0, \r=O
ro
is t h e following.
End p o i n t
TNP
u ( 0 . 5 ) = 0 , \ / x = 5 = 4 27
Note: The meanings of TNP, NDM, N I M
NDM
NIM
NDM NDM+NIM
2
111
1.78
are t h e same as t h a t i n t h e
earlier tables. Table 4 Using A l g o r i t h m D, w e g o t
5 s i n g u l a r p o i n t s on
ro ,
the additio-
n a l amount o f work w a s t o s o l v e t h e l i n e a r g e n e r a l i z e d e i g e n p r o b l e m
Yong Feng Zhou and A . Ruhe
140
( 4 . 3 ) 1 1 times. Table 5 shows the accuracies of the singular points computed by Algorithm D.
I
41
9 . 9 9 5 2 x 1 0-1 1.9989 2.9956 3.9896 4.9795
Table 5 We found that the results of using the linear eigenproblem ( 4 . 3 ) to predict the singularities were exceedingly good. An example:
A 1 j 2 of factorization point A’ l 2 of predicted singular pint 0.9988 1.9997 3.1852
2.9957 3.9894
Table 6 We have used Algorithm D to switch branches at bifurcation point 1 and got 1’+1 and r-l easily.
In addition, Table 4 , in fact, shows continuation along r o using Mixed Method in §3 only. From it we can see that only one matrix of order n and one matrix of order (n+l) are needed as preconditioning matrices on the whole of r o . Acknowledgements The author Mr Zhou would like to thank Ivar Gustafsson, Jacques Huitfeldt, Hans Janestad and Annita Persson for their introducing to the intricacies of the computer facilities. We would like to thank Birgitta Bjorvik Akesson for her nice typewriting this paper. 4.
141
Numerical Path Following and Eigenvalue Criteria for Branch Switching
[I
1
E. Allgower,
B i f u r c a t i o n s A r i s i n g i n t h e c a l c u l a t i o n of C r i t i c a l P o i n t s v i a Homotopy Methods, N u m e r i c a l Methods f o r B i f u r c a t i o n P r o b l e m s , T. KUpper, H . D . M i t t e l m a n n and H. Weber, e d s . , B i r k h a u s e r V e r l a g , B a s e l . B o s t o n . S t u t t g a r t , 1984, pp 1528.
[21
Antman,
S.S.
B i f u r c a t i o n Problems f o r N o n l i n e a r l y E l a s t i c S t r u c t u r e s , A p p l i c a t i o n s o f B i f u r c a t i o n T h e o r y , P.H. R a b i n o w i t z , e d . , Academic Press I n c . , N e w York. San F r a n c i s c o . London, 1 9 7 7 , p p 73-126. E.
Bohl,
Discrete V e r s u s C o n t i n u o u s Models f o r D i s s i p a t i v e S y s t e m s , Numerical Methods f o r B i f u r c a t i o n P r o b l e m s , T . KUpper, H.D. M i t t e l m a n n and H . Weber, e d s . , B i r k h a u s e r V e r l a g , B a s e l . B o s t o n . S t u t t g a r t , 1 9 8 4 , pp 68-78. T.F.
Chan a n d K . R .
Jackson,
N o n l i n e a r l y P r e c o n d i t i o n e d K r y l o v Subspace Methods f o r D i s c r e t e Newton A l g o r i t h m s , SIAM J . S c i . S t a t . Comput., ( 1 9 8 4 ) I pp 533-542. T.F.
Chan a n d Y.
5
Saad,
I t e r a t i v e Methods f o r S o l v i n g B o r d e r e d Systems w i t h A p p l i c a t i o n s t o C o n t i n u a t i o n Methods, SIAM J . S c i . S t a t . Comput., 6 ( 1 9 8 5 ) , pp 438-451. [61
M.A.
Crisfield,
A F a s t Incremented
I t e r a t i v e S o l u t i o n Procedure t h a t H a n d l e s " S n a p - t h r o u g h " , Comp. S t r u c t . , 1 3 (1 981) , p p 55-62.
[71
J.E.
D e n n i s J r , and R.B.
Schnabel,
N u m e r i c a l Methods f o r U n c o n s t r a i n e d O p t i m i z a t i o n and N o n l i n e a r E q u a t i o n s , P r e n t i c e - H a l l , I n c . , Englewood C l i f f s , N e w J e r s e y , 1983. [81
T.
E r i c s s o n and A.
Ruhe,
The S p e c t r a l T r a n s f o r m a t i o n Lanczos Method f o r t h e N u m e r i c a l S o l u t i o n o f L a r g e S p a r s e G e n e r a l i z e d Symmetric Eigenv a l u e P r o b l e m s , Math. Comp., 35 ( 1 9 8 0 ) , pp 1251-1268. [91
C.D.
H e i j e r and W.C.
Rheinboldt,
On S t e p l e n g t h A l g o r i t h m s f o r a C l a s s o f C o n t i n u a t i o n Methods, SIAM J. N u m e r . A n a l . 18 ( 1 9 8 1 ) , pp 925-948. [lo1
P.
Henrici,
D i s c r e t e V a r i a b l e Methods i n O r d i n a r y D i f f e r e n t i a l Ecruat i o n s , John Wiley & S o n s , I n c . , N e w York. London. S y i n e y , 1962.
Yong Feng Zhou and A. Ruhe
142
1111
H.B. Keller, Numerical Solution of Bifurcation and Nonlinear Eigenvalue Problems, Applications of Bifurcation Theory, P.H. Rabinowitz, ed., Academic Press Inc., New York. San Francisco. London, 1977, pp 359-384.
[121
H.D. Mittelman, A n Efficient Algorithm for Bifurcation Problem of Variational Inequalities, Math. Comp., 41 (19831, pp 473-485.
t131
B.N.
1141
W.C. Rheinboldt, Solution Fields of Nonlinear Equations and Continuation Methods, SIAM J. Numer. Anal., 17 (1980), pp 221-237.
1151
A. Ruhe , Algorithms for the Nonlinear Eigenvalue Problem, SIAM J. Numer. Anal., 10 (19731, pp 674-689.
[161
Y. Saad,
[171
R. Seydel, A Continuation Algorithm with Step Control, Numerical Methods for Bifurcation Problems, T. Kiipper, H.D. Mittelmann and H. Weber, eds., Birkhauser Verlag, Basel. Boston. Stuttgart, 1984, pp 480-494.
Parlett, The Symmetric Eigenvalue Problem, Prentice-Hall, Inc., Englewood Cliffs, U.S.A. , 1980.
Practical Use of Some Krylov Subspace Methods for Solving Indefinite and Nonsymmetric Linear Systems, SIAM J. Sci. Stat. Comput. , 5 (1984), pp 203-227.
H. Weber, Multigrid Bifurcation Iteration. SIAM J. Numer.Anal.22 (1985), pp. 262-279.
Large Scale Eigenvalue Problems
143
J. Cullurn and R.A. Willoughby (Editors) 0Elsevier Science Publishers B.V. (North-Holland), 1986
THE LANCZOS ALGORITHM I N MOLECULAR DYNAMICS:
CALCULATION OF SPECTRAL DENSITIES
G i o r g i o Moro Dipartimento d i Chimica F i s i c a U n i v e r s i t a d i Padova, 35131 Padova, I t a l y Jack H. Freed Baker Laboratory o f Chemistry C o r n e l l U n i v e r s i t y , Ithaca, New York 14853.
I n t h e past t h e Lanczos a l g o r i t h m has proven t o be a very u s e f u l t o o l i n t h e s t u d y o f molecular dynamics. I t enables e f f i c i e n t c a l c u l a t i o n o f t h e s p e c t r a l d e n s i t i e s which are e s s e n t i a l i n t h e i n t e r p r e t a t i o n o f spectroscopic o r s c a t t e r i n g experiments. I n t h e present work, t h e use o f t h e Lanczos a l g o r i t h m i n the c a l c u l a t i o n o f t h e s p e c t r a l d e n s i t i e s i s analyzed i n a general fashion. I t s a p p l i c a t i o n t o problems c h a r a c t e r i z e d by complex symmetric matrices, which are n o r m a l l y encountered i n t h e a n a l y s i s o f magnetic resonance experiments, i s t h e n recovered as a p a r t i c u l a r case. A f t e r a d i s c u s s i o n o f t h e f a c t o r s i n f l u e n c i n g t h e convergence i n t h e c a l c u l a t i o n o f s p e c t r a l d e n s i t i e s , t h e implementation of t h e Lanczos a l g o r i t h m w it h non-orthogonal b a s i s f u n c t i o n s i s considered i n r e l a t i o n t o molecular systems having hindered degrees o f freedom. INTRODUCTION The d i r e c t o b s e r v a t i o n o f molecular motion i n condensed phases i s p o s s i b l e o n l y i n s o - c a l l e d Molecular Dynamics (M.D.) "experiments", i.e. i n computer s i m u l a t i o n s o f t r a j e c t o r i e s o f systems o f p a r t i c l e s ( o f t h e o r d e r o f hundreds o r thousands) t o model macroscopic samples. I n f o r m a t i o n about t h e molecular motion i n r e a l systems, m o s t l y obtained by means o f spectroscopic o r s c a t t e r i n g t e c h n i ques, i s i n s t e a d v e r y i n d i r e c t . One i s c o n f i n e d t o t h e measurement o f t h e macroscopic response t o some e x t e r n a l , time-dependent, p e r t u r b i n g f i e l d . According t o l i n e a r response t h e o r y [l], the spectroscopic observation i s c h a r a c t e r i z e d by a p a r t i c u l a r s p e c t r a l density, which i s t h e Fourier-Laplace t r a n s f o r m o f t h e t i m e c o r r e l a t i o n f u n c t i o n ( s ) , f o r t h e dynamical v a r i a b l e ( s ) probed i n t h e s p e c i f i c experiment. Formally, t h e s p e c t r a l d e n s i t i e s are t h e m a t r i x elements o f t h e r e s o l v e n t constructed from t h e t i m e e v o l u t i o n operator, r, a p p r o p r i a t e f o r t h e system observed. By c o n s i d e r i n g a proper b a s i s set, t h e r e s o l v e n t can be w r i t t e n i n terms o f t h e m a t r i x r e p r e s e n t a t i o n o f r, and it can be solved by numerical methods. By means o f spectroscopic techniques l i k e :
-
magnetic resonance (ESR and NMR) dielectric relaxation l i g h t scattering i n f r a r e d and Raman spectroscopy - incoherent neutron s c a t t e r i n g one can have access t o s p c i f i c s p e c t r a l d e n s i t i e s and i n f e r , from a p o s t u l a t e o f r , i n f o r m a t i o n about t h e t r a n s l a t i o n a l , r o t a t i o n a l , and conformational motion o f t h e molecules c o n s t i t u t i n g t h e observed system.
-
The c a l c u l a t i o n o f t h e s p e c t r a l d e n s i t i e s can be seen as a fundamental t o o l i n
144
G.Moro and J.H. Freed
t h e i n t e r p r e t a t i o n o f s p e c t r a l data, s i n c e i t i s a necessary s t e p i n r e l a t i n g e x p e r i m e n t a l measurements and t h e o r e t i c a l models f o r r. T h i s e x p l a i n s t h e need f o r s p e c i f i c a l g o r i t h m s which should be e f f i c i e n t f r o m t h e p o i n t s o f view o f b o t h t h e speed o f c a l c u l a t i o n and t h e s i z e o f t h e m a t r i x t h a t can be handled. The development o f s o p h i s t i c a t e d models o f m o t i o n where t h e c o u p l i n g among s e v e r a l degrees o f freedom i s t a k e n i n t o account, c o n s t i t u t e s a n a t u r a l t r e n d i n chemical p h y s i c s t h a t i s r e q u i r e d by improvements i n e x p e r i m e n t a l techniques which l e a d t o more d e t a i l e d knowledge o f t h e s p e c t r a l d e n s i t i e s . Correspondingly, one must t a c k l e c a l c u l a t i o n s o f s p e c t r a l d e n s i t i e s f r o m l a r g e r m a t r i x r e p r e s e n t a t i o n s , and t h e e f f i c i e n c y o f t h e a l g o r i t h m becomes c r i t i c a l i n making poss i b l e t h e i n t e r p r e t a t i o n o f spectral information. The Lanczos a l g o r i t h m [2-41 c o n s t i t u t e s a n a t u r a l c h o i c e as a method o f c a l c u l a t i o n o f t h e s p e c t r a l d e n s i t i e s , s i n c e t h e m a t r i c e s n o r m a l l y encountered are v e r y sparse [5,6]. Indeed, t h e Lanczos a l g o r i t h m i s p a r t i c u l a r l y s u i t a b l e because i t produces a t r i d i a g o n a l m a t r i x , such t h a t t h e c o n t i n u e d f r a c t i o n r e p r e s e n t a t i o n o f t h e s p e c t r a l d e n s i t y i s e a s i l y generated f r o m t h e elements o f t h i s t r i d i a g o n a1 m a t r i x . T h i s a l l o w s f o r a d i r e c t c a l c u l a t i o n o f t h e f r e q u e n c y dependence o f t h e s p e c t r a l d e n s i t y w i t h o u t t h e need f o r t h e eigenvalues. Moreover t h e r e s u l t s o f t h e Lanczos a l g o r i t h m can be r e l a t e d [ 7 ] t o t h e c o n t i n u e d f r a c t i o n r e p r e s e n t a t i o n o f t h e s p e c t r a l d e n s i t i t e s d e r i v e d b y H. M o r i [8], t h e r e b y c o n n e c t i n g a numerical method t o t h e p r o j e c t i v e f o r m a l i s m s o f t e n used i n s t a t i s t i c a l mechanics [9]. I n the next section, the formal d e f i n i t i o n s o f t h e q u a n t i t i e s r e l a t e d t o s p e c t r a l d e n s i t i e s and s p e c t r a w i l l be given, w i t h p a r t i c u l a r emphasis on c l a s s i c a l systems. A f t e r a b r i e f d i s c u s s i o n o f t h e most common models o f motion, t h e r e s u l t s o f l i n e a r response t h e o r y w i l l be summarized. T h i s w i l l c o n s t i t u t e t h e framework f o r t h e subsequent a n a l y s i s o f t h e r o l e p l a y e d by t h e c a l c u l a t i o n o f s p e c t r a l d e n s i t i e s i n t h e t h e o r e t i c a l i n t e r p r e t a t i o n o f s p e c t r o s c o p i c data. The numerical c a l c u l a t i o n o f s p e c t r a l d e n s i t i e s by means o f t h e Lanczos a l g o r i t h m w i l l be t h e c e n t r a l t o p i c o f t h e t h i r d s e c t i o n . F i r s t , t h e method w i l l be i l l u s t r a t e d w i t h s e l f - a d j o i n t , "symmetrized," t i m e e v o l u t i o n o p e r a t o r s . Then t h e general case w i l l be considered, showing how t h e c o e f f i c i e n t s o f t h e c o n t i n u e d f r a c t i o n r e p r e s e n t a t i o n are c a l c u l a t e d by numerical implementation o f t h e Lanczos a l g o r i t h m . The performance o f t h e a l g o r i t h m w i l l be discussed s p e c i f i c a l l y i n c o n n e c t i o n w i t h t h e convergence o f t h e c o n t i n u e d f r a c t i o n s o l u t i o n . P a r t i c u l a r emphasis w i l l be g i v e n t o t h e f o l l o w i n g r e s u l t : t h e Lanczos a l g o r i t h m i s more e f f i c i e n t i n p r o d u c i n g t h e o v e r a l l shape o f t h e s p e c t r a l l i n e shape t h a n i n computing t h e eigenvalues o f t h e r e l a t e d m a t r i x . The f i n a l p a r t o f t h e t h i r d s e c t i o n w i l l be devoted t o a d i s c u s s i o n o f t h e c r i t e r i a f o r t h e c h o i c e o f t h e b a s i s f u n c t i o n s necessary t o generate t h e m a t r i x r e p r e s e n t a t i o n o f t h e t i m e e v o l u t i o n o p e r a t o r . Such a c h o i c e i s an i m p o r t a n t i n g r e d i e n t i n a c t u a l c a l c u l a t i o n s , and i t must be c a r e f u l l y considered, case b y case, i n o r d e r t o m i n i m i z e c o m p u t a t i o n a l e f f o r t . P a r t i c u l a r emphasis w i l l be g i v e n t o problems c h a r a c t e r i z e d by s t r o n g c o n f i n i n g p o t e n t i a l s , f o r which a l a r g e r e d u c t i o n o f t h e s i z e o f t h e m a t r i c e s can be achieved b y means o f nono r t h o g o n a l b a s i s f u n c t i o n s . T h e r e f o r e t h e i m p l e m e n t a t i o n o f t h e Lanczos a l g o r i t h m w i t h non-orthogonal f u n c t i o n s w i l l a l s o be presented. I n t h e summary s e c t i o n , we s h a l l p o i n t o u t some f u r t h e r a p p l i c a t i o n s o f t h e Lanczos a l g o r i t h m t o t h e s t u d y o f t h e m o l e c u l a r dynamics, and we suggest where f u r t h e r computational developments would be d e s i r a b l e .
SPECTRAL DENSITIES AND WLECULAR
MOTION
In t h i s s e c t i o n , t h e c o r r e l a t i o n f u n c t i o n f o r m a l i s m and t h e r e s u l t s o f l i n e a r
The Lanczos A Igorithin in Molecular Dynamics
145
response t h e o r y w i l l be summarized i n o r d e r t o p r e s e n t a c l e a r d e f i n i t i o n o f t h e q u a n t i t i e s t o be computed by means o f t h e Lanczos a f g o r i t h m . We c o n s i d e r a system d e s c r i b e d by an ensemble o f M c l a s s i c a l s t o c h a s t i c v a r i ables z = (z1,z2, Z M ) ; ( t h e g e n e r a l i z a t i o n t o quantum system w i l l be pres e n t e d s u b s e q u e n t l y ) . The s t a t i o n a r y s t a t e o f t h e system i s d e s c r i b e d b y t h e e q u i l i b r i u m d i s t r i b u t i o n f u n c t i o n Pe ( z ), d e f i n e d as t h e p r o b a b i l i t y d e n s i t y w i t h r e s p e c t t o t h e i n f i n i t e s i m a l vo?ume d& = dz,dz, dzM. The e q u i l i b r i u m average of a dynamical v a r i a b l e d e s c r i b e d b y t h e f u n c t i o n A ( z ) i s then c a l c u l a t i o n according t o the f o l l o w i n g i n t e g r a l :
...,
....
A=Jd'zP
eq
(z)A( z )
(2.1)
One can now i n t r o d u c e t h e H i l b e r t space E c o n s t i t u t e d f r o m t h e ensemble o f dynamical v a r i a b l e s h a v i n g w e l l d e f i n e d average values, and f o r which t h e f o l l o w i n g formal d e f i n i t i o n o f scalar product applies:
=
J dMz A1(
2
)* A2( z
1
The dynamics o f t h e system i s c h a r a c t e r i z e d by t h e t i m e e v o l u t i o n o p e r a t o r which determines t h e b e h a v i o r o f t h e n o n - e q u i l i b r i u m d i s t r i b u t i o n f u n c t i o n P( 2; t ) :
a / a t P( z ; t )
z ;t)
= -rP(
(2.2)
r (2.3)
H e r e a f t e r we s h a l l c o n s i d e r o n l y s t a t i o n a r y Markovian processes [ l o ] ; t h a t i s we assume t h a t r does n o t depend e x p l i c i t l y on t h e time, and we t a k e Peq t o be t h e u n i q u e s t a t i o n a r y s o l u t i o n r:
rPeq = 0
(2.4
F o r a g i v e n p a i r o f dynamical v a r i a b l e s A,( z ) and A,( z ), t h e t i m e c o r r e l a t i o n f u n c t i o n G ( t ) i s d e f i n e d i n terms o f t h e f o l l o w i n g dynamical average [ll] G(t) = A l l z ( t ) I which, f r o m t h e f o r m a l s o l u t i o n o f eq. (2.3), follows:
z
A,[:
(011
(2.5
an be w r i t t e n e x p l i c i t l y as
G(t) = <~,(exp(-Tt) P
A>, eq
The c o r r e s p o n d i n g s p e c t r a l d e n s i t y J(I,J), which i s t h e Four i e r - L a p l ace t r a n s f o r m o f G ( t ) , i s w r i t t e n as:
J(w) z
d t eXp(-iwt) G(t) =
+
r)-l1PeqA,>
(2.7)
The i n t e r n a l m o t i o n o f a m o b i l e group i n a m o l e c u l e can be t a k e n as a model system f o r i l l u s t r a t i n g t h e p r e v i o u s s t a t i s t i c a l concepts. The s t o c h a s t i c v a r i a b l e i s now r e p r e s e n t e d by t h e angle 8 , which i s t h e angle o f r o t a t i o n o f t h e m o b i l e group around a f i x e d a x i s o f t h e r i g i d p a r t o f t h e molecule. The e q u i l i b r i u m d i s t r i b u t i o n f u n c t i o n i s n o r m a l l y w r i t t e n as:
where V ( e ) i s t h e mean p o t e n t i a l a c t i n g on t h e m o b i l e group.
I t r e s u l t s f r o m t h e i n t e r a c t i o n of t h e m o b i l e group w i t h t h e r i g i d p a r t o f t h e molecule. There t h e minima o f V(e) d e t e r m i n e t h e s t a b l e c o n f o r m a t i o n s o f t h e system. See f o r example r e f e r e n c e 12 f o r a p a r a m e t r i c f o r m o f V(e) o f t e n used
G. Moro and J.H. Freed
146
i n t h e s t u d y o f c o n f o r m a t i o n a l dynamics o f a l k y l c h a i n s . The dynamics o f t h e i n t e r n a l r o t o r can be r e p r e s e n t e d by d i f f e r e n t o p e r a t o r s r, t h e c h o i c e depending on t h e t y p e o f c o u p l i n g w i t h t h e thermal bath. A q u i t e common model, j u s t i f i e d on p h y s i c a l grounds when d e a l i n g w i t h b u l k y m o b i l e groups, i s p r o v i d e d by t h e d i f f u s i o n o p e r a t o r , which, f o r t h e i n t e r n a l m o t i o n problem b e i n g used f o r i l l u s t r a t i o n , i s w r i t t e n as f o l l o w s :
r
= -o(a/ae)
Peq(a/ae)
(2.9)
where D i s t h e d i f f u s i o n c o e f f i c i e n t . S i m i l a r d i f f u s i o n o p e r a t o r s can be generated when d e a l i n g w i t h t h e m o l e c u l a r t r a n s l a t i o n a l and r o t a t i o n a l m o t i o n and t h e i r c o u p l i n g [13,14]. A more general c l a s s o f t i m e e v o l u t i o n o p e r a t o r s are r e p r e s e n t e d by t h e s o - c a l l e d Fokker-Planck equations, where t h e c l a s s i c a l streaming m o t i o n i s t a k e n i n t o account by c o n s i d e r i n g as s t o c h a s t i c v a r i a b l e s b o t h t h e c o o r d i n a t e s and t h e c o n j u g a t e momenta [9,13,15,16]. Therefore, s t a t i s t i c a l mechanics p r o v i d e s a h i e r a r c h y o f models o f motion, where t h e c o m p l e x i t y ( i . e . t h e number o f degrees o f freedom e x p l i c i t l y t a k e n i n t o account) grows w i t h t h e l e v e l o f g e n e r a l i t y and w i t h t h e accuracy i n r e p r e s e n t i n g t h e dynamical f e a t u r e s o f t h e p h y s i c a l system (e.g. r e f . 1 7 ) . L e t us now b r i e f l y i l l u s t r a t e , i n t h e c o n t e x t o f c l a s s i c a l systems, t h e l i n e a r response t h e o r y which c o n s t i t u t e s t h e g e n e r a l framework f o r t h e i n t e r p r e t a t i o n o f s p e c t r o s c o p i c measurements [ll]. Such experiments a r e c h a r a c t e r i z e d by t h e presence o f an e x t e r n a l o s c i l l a t i n g f i e l d , h a v i n g s t r e n g t h AEol and f r e q u e n c y ,,,, which i s coupled t o a g i v e n dynamical v a r i o f t h system, y i e l d i n g a The response o f t h e c o n t r i b u t i o n t o t h e t o t a l energy as B( z ) system t o t h e e x t e r n a l f i e l d i s observed b t h e macroscopic average o f a s p e c i f i e d dynamical v a r i a b l e A( z ). D i e l e c t r i c r e l a x a t i o n experiments, f o r example, are c h a r a c t e r i z e d by an o s c i l l a t i n g e l e c t r i c f i e l d E ( t ) = Eo coswt, t h a t i n t e r a c t s w i t h the molecular dipoles ( t ) , which have a time-dependent o r i e n t a t i o n as a consequence o f t h e r o t a t i o n a l m o t i o n o f t h e molecules. Theref o r e B( z ) i s g i v e n by cosg, where t h e s t o c h a s t i c v a r i a b l e @ i s d e f i n e d as t h e angle between p and 'The observed macroscopic v a r i a b l e i s r e p r e s e n t e d by t h e p o l a r i z a t i o n o f t h e medium, i n p r a c t i c e t h e average o f t h e components o f t h e d i p o l e moments along Eo, i.e. A( z )=B( z ) .
1,:
By assuming t h a t t h e e x t e r n a l f i e l d a c t s as a s m a l l p e r t u r b a t i o n w i t h r e s p e c t t o t h e dynamical b e h a v i o r o f t h e i s o l a t e d system, i t i s shown-that t h e macroscopic average o f A( z ) o s c i l l a t e s around i t s e q u i l i b r i u m v a l u e A w i t h t h e same f r e quency o f t h e f i e l d , and t h a t t h e i n t e n s i t y o f t h e s e o s c i l l a t i o n s i s propora c c o r d i n g t o t h e f r e q u e n c y dependent tional t o the f i e l d strength E, s u s c e p t i b i l i t y x ( ~ )[l]. On he o h e r hand, t h e s u s c e p t i b i l i t y X ( W ) i s independent o f t h e e x t e r n a l p e r t u r b a t i o n , and i t i s determined b y t h e dynamics o f t h e i s o l a t e d system d e s c r i b e d b y t h e t i m e e v o l u t i o n o p e r a t o r r , according t o t h e f o l l o w i n g e q u a t i o n [I]: X(W)
= -iwJ(w)
+
& A * 6B
(2.10)
where J ( w ) i s t h e s p e c t r a l d e n s i t y t h a t r e l a t e s t o t h e p a i r o f dynamical That i s : v a r i a b l e s 6A=A( z ) - A and 6B=B( z )-B.
J ( ~ )= <6A\(itrt +
r ) - 1 J peq g ~ >
(2.11)
I t i s i m p l i c i t t h a t each spectroscopy corresponds t o a s p e c i f i c c h o i c e f o r t h e p a i r o f dynamical v a r i a b l e s A and B. I t should be n o t e d t h a t t h e s u s c e p t i b i l i t y i s a complex q u a n t i t y , i t s r e a l and i m a g i n a r y p a r t s b e i n g r e s p e c t i v e l y r e l a t e d t o t h e in-phase and t o t h e out-of-phase response o f t h e system t o t h e o s c i l l a t o r y p e r t u r b a t i o n . O f t e n one measures o n l y one component, m r e p r e c i s e l y t h e i m a g i n a r y p a r t , which i n t h e f o l l o w i n g , w i l l be i d e n t i f i e d w i t h t h e observed s p e c t r a l l i n e s h a p e I(w):
The Lanczos Algorithm in Molecular Dynamics
S p e c t r o s c o p i c experiments: measurement o f
I(W) as
147
S t a t is t ic a l mechanics :
<+>
a
t h e o r e t i c a l models
function of w Figure 1 A s sketched i n F i g u r e 1, t h e c a l c u l a t i o n o f t h e s p e c t r a l d e n s i t y i s an e s s e n t i a l s t e p i n r e l a t i n g t h e e x p e r i m e n t a l r e s u l t s t o t h e t h e o r e t i c a l model f o r t h e dynamical b e h a v i o r o f t h e system. Only b y a c a r e f u l comparison o f t h e f r e q u e n c y dependence o f t h e t h e o r e t i c a l J ( w ) w i t h t h e e x p e r i m e n t a l data, i s i t p o s s i b l e t o o b t a i n a f u l l account o f t h e e f f e c t s o f t h e m o l e c u l a r m o t i o n on t h e observed spectra. I d e a l l y , one would l i k e t o p e r f o r m t h e " i n v e r s e t r a n s f o r m " , v i z . t o reconstruct t h e d e t a i l e d form o f r from a given experimental I ( w ) . Unfortunately, I(W) s u p p l i e s v e r y l i m i t e d and i n c o m p l e t e i n f o r m a t i o n . I n s t e a d one would need a l l t h e c o r r e l a t i o n f u n c t i o n s f o r a complete s e t o f dynamical v a r i a b l e s which span t h e v e c t o r space i m p l i c i t i n eq. (2.2).
More r e a l i s t i c a l l y one compares t h e t h e o r e t i c a l and t h e e x p e r i m e n t a l o r d e r t o achieve t h e f o l l o w i n g two o b j e c t i v e s :
I(w) i n
i ) t h e v e r i f i c a t i o n o f t h e adequacy o f t h e t h e o r e t i c a l model r. I n t h i s cont e x t one o f t e n compares t h e e x p e r i m e n t a l r e s u l t s w i t h t h e t h e o r e t i c a l prof i l e s o f I ( w ) o b t a i n e d w i t h d i f f e r e n t models o f m o t i o n i n o r d e r t o assess which one i s t h e most a p p r o p r i a t e . i i ) The d e t e r m i n a t i o n o f t h e v a l u e s o f t h e t r a n s p o r t c o e f f i c i e n t s , [such as t h e d i f f u s i o n c o e f f i c i e n t D i n eq. ( P . Y ) ] , which e n t e r as parameters i n t h e e x p r e s s i o n o f r. A b e s t f i t between t h e o r e t i c a l and e x p e r i m e n t a l I(w) s u p p l i e s t h e o p t i m a l e s t i m a t e o f t h e s e parameters. I n t h e most f a v o r a b l e s i t u a t i o n s , i.e. when t h e e x p e r i m e n t a l spectrum I(w) i s known a c c u r a t e l y o v e r a wide range o f f r e q u e n c i e s , t h e c a l c u l a t i o n o f t h e spect r a l d e n s i t y p r o v i d e s d e t a i l e d i n f o r m a t i o n on b o t h t h e t y p e o f m o l e c u l a r dynami c s and t h e c h a r a c t e r i s t i c r e l a x a t i o n t i m e a s s o c i a t e d w i t h each elementary motion. The need f o r c a l c u l a t i n g s p e c t r a l d e n s i t i e s i s c l e a r i n t h e c o n t e x t o f s t a n d a r d s p e c t r o s c o p i e s such as magnetic resonance o r d i e l e c t r i c r e l a x a t i o n measurements. The same c o n c l u s i o n a l s o f o l l o w s f r o m t h e a n a l y s i s o f s c a t t e r i n g e x p e r i ments. As an example, t h e d i f f e r e n t i a l c r o s s s e c t i o n f o r i n c o h e r e n t n e u t r o n scattering i s proportional t o the real part o f t h e following spectral density [18]:
J ( ~ =) <exp( - i q
. I (lU+r)-' ' I r )
Peqexp( - i q
. r )>
(2.13)
where r i s now t h e t i m e e v o l u t i o n o p e r a t o r a p p r o p r i a t e f o r t h e s t o c h a s t i c v a r i a b l e r, t h e v e c t o r p o s i t i o n o f t h e s c a t t e r i n g nucleus. The v e c t o r q and4W a r e r e s p e c t i v e l y t h e momentum and energy d i f f e r e n c e s between i n c i d e n t and s c a t t e r e d neutrons. Up t o now we have c o n s i d e r e d c l a s s i c a l s t o c h a s t i c v a r i a b l e s . I n t h e a n a l y s i s o f e x p e r i m e n t s l i k e magnetic resonane (NMR o r ESR) i t i s necessary t o d e a l w i t h quantum degrees o f freedom, i . e . e l e c t r o n i c and n u c l e a r spin, and t h e i r i n t e r a c t i o n w i t h t h e m o l e c u l a r o r i e n t a t i o n a l degrees o f freedom s p e c i f i e d by t h e
G. Moro and J.H. Freed
148
E u l e r angles a = (a,/3,6). The l a t t e r are u s u a l l y t a k e n as c l a s s i c a l s t o c h a s t i c v a r i a b l e s f o r motions i n condensed phases. In t h e s e cases t h e dynamics o f t h e system i s u s u a l l y d e s c r i b e d by t h e s o - c a i i e d s t o c h a s t i c L i o u v i l l e o p e r a t o r [19,20], w r i t t e n as:
(2.14) where rn i s t h e s t o c h a s t i c t i m e e v o l u t i o n o p e r a t o r f o r j u s t t h e o r i e n t a t i o n a l v a r i a b l e s (e.g. t h e d i f f u s i o n o p e r a t o r ) , and HXs[H, 1 i s t h e quantum-mechanical t i m e p r o p a g a t o r ( s u p e r o p e r a t o r ) . Note t h a t t h e c o u p l i n g between t h e quantum and c l a s s i c a l degrees o f freedom, f o r example t h e m o d u l a t i o n o f t h e energy o f t h e quantum s t a t e s induced by t h e r o t a t i o n a l motion, i s i m p l i c i t l y t a k e n i n t o account b y t h e dependence o f H upon t h e o r i e n t a t i o n a l v a r i a b l e D. The c o r r e l a t i o n f u n c t i o n i n eq. (2.6) and t h e c o r r e s p o n d i n g s p e c t r a l d e n s i t y o f eq. (2.7), where A i s now an o p e r a t o r on t h e quantum s t a t e s , are c a l c u l a t e d w i t h t h e p r e v i o u s s t o c h a s t i c L i o u v i l l e o p e r a t o r r, a f t e r r e d e f i n i n g t h e s c a l a r p r o d u c t s as: <....>z
T r Jdn
....
(2.15)
i.e. as a t r a c e over t h e quantum s t a t e s and i n t e g r a t i o n over t h e c l a s s i c a l stochastic variables. F o r t h e sake o f s i m p l i c i t y , i n t h e n e x t s e c t i o n we s h a l l f o c u s on c l a s s i c a l systems, such t h a t t h e t i m e e v o l u t i o n o p e r a t o r can e a s i l y be r e p r e s e n t e d i n a basis o f functions o f the stochastic variables. However t h e numerical methods we d i s c u s s can a l s o be a p p l i e d t o problems d e s c r i b e d b y t h e s t o c h a s t i c L i o u v i l l e o p e r a t o r o f eq. (2.14) i f a proper b a s i s f o r t h e quantum degrees o f freedom i s s p e c i f i c a l l y c o n s i d e r e d [21].
CALCULATION
(IF
SPECTRAL DENSITIES BY E A N S OF THE LANCZOS ALGORITHM
I n t h e a n a l y s i s o f t h e c o m p u t a t i o n a l methods, i t i s c o n v e n i e n t t o make r e f e r e n c e t o t h e s o - c a l l e d "symmetrized" t i m e e v o l u t i o n o p e r a t o r r d e f i n e d as f o l l o w s c22 I.
i; The s p e c t r a l d e n s i t y o f eq. (2.11)
= p-1/2
eq becomes:
rp1/2 eq
J(w) =
(3.1)
gB>
(3.2)
The reasons which d i c t a t e such a t r a n s f o r m a t i o n o f r a r e t w o f o l d . F i r s t o f a l l , when d e a l i n g w i t h a u t o c o r r e l a t i o n f u n c t i o n s , i.e. A=B, t h e c o r r e s p o n d i n g spect r a l d e n s i t y i s s i m p l y t h e d i a o n a l m a t r i x element o f t h e r e s o l v e n t o p e r a t o r w i t h r e s p e c t t o t h e f u n c t i o n Pq12 gA ( i . e . r i g h t - and l e f t h a n d v e c t o r s are H e r m i t i a n c o n j u g a t e s ) . Most oggervables a r e r e l a t e d t o t h e a u t o c o r r e l a t i o n f u n c t i o n s as we p r e v i o u s l y p o i n t e d out, [e.g. eq. (2.13) o r t h e d i s c u s s i o n o f d i e l e c t r i c r e l a x a t i o n i n the previous chapter]. Secondly t h i s t r a n s f o r m a t i o n on r c l a r i f i e s t h e symmetries o f t h e t i m e e v o l u t i o n o p e r a t o r . As an example, i f r i s g i v e n by a d i f f u s i o n o p e r a t o r such as eq. ( 2 . 9 ) , i t s symmetrized f o r m i s s e l f - a d j o i n t w i t h r e s p e c t t o t h e d e f i n i t i o n eq. (2.2) o f t h e s c a l a r p r o d u c t . T h i s p r o p e r t y i s not, i n general, shared by t h e symmetrized Fokker-Planck o p e r a t o r s o r t h e s t o c h a s t i c L i o u v i l l e o p e r a t o r s . But f o r them i t i s p o s s i b l e t o generate complex symmetric m a t r i x r e p r e s e n t a t i o n s by a c a r e f u l c h o i c e o f t h e b a s i s f u n c t i o n s [6,21], and t h i s w i l l s i m p l i f y t h e c o m p u t a t i o n a l a l g o r i t h m t h a t i s needed. We f i r s t c o n s i d e r t h e c a l c u l a t i o n o f t h e s p e c t r a l d e n s i t y under t h e s i m p l i f y i n g c o n d i t i o n s t h a t T i s s e l f - a d j o i n t and t h a t A=B. I f A#B, t h e n it i s easy t o show t h a t J(w) o f eq. (3.2) can be decomposed as a l i n e a r c o m b i n a t i o n o f s p e c t r a l
The Lanczos Algorithm in Molecular Dynamics
149
d e n s i t i e s a s s o c i a t e d w i t h t h e a u t o c o r r e l a t i o n f u n c t i o n s f o r t h e dynamical v a r i a b l e s A-B and A+B [6]. Once a s u i t a b l e b a s i s s e t o f f u n c t i o n s f o r t h e s t o c h a s t i c v a r i a b l e s z has been i n t r o d u c e d , one can a p p l y t h e Lanczos a l g o r i t h m [2-41 t o t h e m a t r i x r e p r e s e n t a t i o n o f P. A l t e r n a t i v e l y , t h e methodology, which i s more g e n e r a l t h a n j u s t t h e n u m e r i c a l a l g o r i t h m , can be a p p l i e d d i r e c t l y t o t h e o p e r a t o r T, i n o r d e r t o produce an o r t h o n o r m a l s e t o f f u n c t i o n s +j ( z ). We s h a l l f o l l o w t h e second r o u t e because o f i t s c o n n e c t i o n w i t h t h e method o f moments [23] and w it h t h e p r o j e c t i v e schemes f r e q u e n t l y used i n s t a t i s t i c a l mechanics [9]. Given t h e r e l a t i o n eq. (3.2) f o r t h e s p e c t r a l d e n s i t y , a n a t u r a l c h o i c e f o r t h e f i r s t element o f t h e b a s i s s e t i s r e p r e s e n t e d by t h e f u n c t i o n P1i2 gA p r o p e r l y normal i z e d : eq
el( z The r e m a i n i n g f u n c t i o n s $ j ( f o l 1owing r e 1 a t i o n :
= phi2 ~ A / < ~ A ( P , ~ J ~ A > ~ / ~
(3.3)
z ) a r e generated i t e r a t i v e l y a c c o r d i n g t o t h e 8,+19,+1
= (l-Pn)hn
(3.4)
where ~ ~ i s +t h el r e a l c o e f f i c i e n t i m p l i c i t l y d e f i n e d by t h e n o r m a l i z a t i o n c o n d i t i o n ( i . e . <+n+ll,+,n+l>=l), and Pn i s t h e p r o j e c t i o n o p e r a t o r on t h e subspace spanned b y f u n c t i o n s $ l , 9 2 , . . . , $n: n (3.5) I t i s e a s i l y demonstrated t h a t ? has a t r i d i a g o n a l r e p r e s e n t a t i o n i n t h e $ - b a s i s ( i . e . < $ j ) r ) $ j l=>0 f o r J J >1) and t h a t :
l’-”l
Therefore, as :
Bj = <$j [Fl+j-1>
(3.6)
a t h r e e t e r m r e c u r s i v e r e l a t i o n i s o b t a i n e d f r o m eq. (3.4), fin+l+n+l = (‘
-
where t h e a j ’ s are t h e d i a g o n a l elements o f aj
I n t h e +-representation,
= <$j
-
an)+n
1-1r
5,
9n-1
r
i n t h e $-basis:
$j
>
written (3.7)
(3.8)
t h e s p e c t r a l d e n s i t y i s w r i t t e n as:
=m
J ( ~ )
[(iw 1
+
T )11,1
(3.9)
c o e f f i c i e n t s a j and ~j p r e v i o u s l y d e f i n e d .
An e l e m e n t a r y a p p l i c a t i o n o f t h e theorem o f m a t r i x p a r t i t i o n i n g , a l l o w s one t o w r i t e down t h e f r e q u e n c y dependence o f J(w) a c c o r d i n g t o t h e f o l l o w i n g c o n t i n u e d f r a c t i o n [24]:
150
G. Moro and J.H. Freed
T h e r e f o r e t h e a p p l i c a t i o n o f t h e Lanczos a l g o r i t h m t o t h e a b s t r a c t H i l b e r t space i n which P i s d e f i n e d , generates t h e c o n t i n u e d f r a c t i o n r e p r e s e n t a t i o n o f t h e s p e c t r a l d e n s i t y . The same r e s u l t , a p a r t f r o m t h e i d e n t i f i c a t i o n o f f w i t h t h e c l a s s i c a l L i o u v i l l e o p e r a t o r , has been d e r i v e d b y H. M o r i i n t h e c o n t e x t o f t h e dynamics o f systems o f i n t e r a c t i n g p a r t i c l e s [ 8 ] . As m a t t e r o f f a c t , t h e same methodology, more s p e c i f i c a l l y t h e r e c u r s i v e e q u a t i o n (3.4), i s t h e f o u n d a t i o n of b o t h t h e Lanczos a l g o r i t h m and M o r i ' s d e r i v a t i o n [7]. It i s interesting t o n o t e t h a t t h i s c o n n e c t i o n between numerical methods and t h e f o r m a l i s m s o f theor e t i c a l physics, can be extended f u r t h e r , by c o n s i d e r i n g on t h e one hand t h e b l o c k Lanczos a l g o r i t h m [25], and on t h e o t h e r hand t h e p r o j e c t i v e scheme norma l l y used f o r d e r i v i n g a d i f f u s i o n o r Fokker-Planck e q u a t i o n f r o m t h e c l a s s i c a l dynamics o f an ensemble o f i n t e r a c t i n g p a r t i c l e s [9,13]. The r e l a t i o n s p e c i f i e d by t h e c o n t i n u e d f r a c t i o n i n eq. (3.10) i s q u i t e g e n e r a l . A n a l y t i c a l c a l c u l a t i o n o f t h e c o e f f i c i e n t s a j and ~j f r o m t h e e x p l i c i t o p e r a t o r f o r m o f f i s i n p r i n c i p l e p o s s i b l e . P r a c t i c a l l y , however, t h i s can be pursued o n l y f o r t h e f i r s t few c o e f f i c i e n t s . T h e r e f o r e n u m e r i c a l implementation o f t h e r e c u r s i v e r e l a t i o n eq. (3.7) i s e s s e n t i a l i n c a l c u l a t i n g enough c o e f f i c i e n t s o f t h e c o n t i n u e d f r a c t i o n f o r an a c c u r a t e s i m u l a t i o n o f t h e I n p r a c t i c e , one generates t h e m a t r i x r e p r e s e n t a f r e q u e n c y dependence o f J(w). t i o n M o f t h e symmetrized o p e r a t o r T i n a g i v e n b a s i s s e t o f orthonormal functions f j ( z ) : (3.11) From eq. (3.7), t h e s t a n d a r d r e c u r s i v e r e l a t i o n o f t h e Lanczos a l g o r i t h m i s r e a d i l y o b t a i n e d i n t h e f o l l o w i n g form:
-
B n + l Xn+l = ( M
w i t h t h e column m a t r i x x,
)
an'
Xn
-
8, Xn-1
(3.12)
c o n t a i n i n g t h e expansion c o e f f i c i e n t s o f $,: $n =
5
( Xn)j f j
(3.13)
The standard computer i m p l e m e n t a t i o n o f t h e Lanczos a l g o r i t h m [3,4] can t h e n be used f o r c a l c u l a t i n g t h e c o e f f i c i e n t s a . and B ' , and f r o m t h o s e t h e frequency p r o f i l e o f J(o) a c c o r d i n g t o t h e c o n i i n u e d f r a c t i o n r e p r e s e n t a t i o n o f eq. (3.10).
In o r d e r t o complete t h i s d i s c u s s i o n we must d e a l w i t h t h e c a l c u l a t i o n o f t h e s t a r t i n g v e c t o r xl. Given eq. (3.3) f o r one can o b t a i n t h e ( x 1 ) j o f eq. (3.13) b y computing t h e s c a l a r p r o d u c t s . T h i s d i r e c t approach has been used f r e q u e n t l y . However, i t u s u a l l y e q u i r e s numerical i n t e g r a t i o n s which can become u n w i e l d y f o r s e v e r a l degrees o f freedom. An a l t e r n a t i v e approach i s t o c o n s i d e r t h e f o l l o w i n g e x p r e s s i o n [See eqs. (2.4) and (3.1)] [26]:
o1
lim [ sl s+o+
-
M
3
xo = c
(3.14)
where xo i s t h e v e c t o r r e p r e s e n t a t i o n o f P I / * and c i s an a r b i t r a r y ( " i n i t i a l " ) v e c t o r . t h i s f o l l o w s because PA/' %! t h e u n i q u e s t a t i o n a r y s o l u t i o n o f f . One s o l v e s eq. (3.14) by m a t r i x inv&-sion t e c h n i q u e s t o o b t a i n xo f o r t h e l i m i t o f v e r y small s. Then
( xl)j =
[
( Xo ) k
(3.15)
The Lanczos Algorithm in Molecular Dynamics
151
Eq. (3.15) i s easy t o e v a l u a t e , s i n c e we u s u a l l y choose b a s i s f u n c t i o n s f j t h a t bear a s i m p l e r e l a t i o n t o t h e 6 A (and 65) o f i n t e r e s t . The c a l c u l a t i o n o f s p e c t r a l d e n s i t i e s f r o m t h e r e c u r s i v e r e l a t i o n eq. (3.12) can e a s i l y be g e n e r a l i z e d t o o p e r a t o r s i; which are not s e l f - a d j o i n t , b u t which have a complex symmetrix m a t r i x r e p r e s e n t a t i o n M. I n p r e v i o u s work [6], we have shown t h a t eq. (3.12) c o n t i n u e s t o hold, b u t w i t h t h e column m a t r i c e s Xn now n o r m a l i z e d w i t h o u t complex c o n j u g a t i o n ( i . e . t h e complex c o n ' u g a t i o n must be removed f r o m t h e d e f i n i t i o n o f s c a l a r p r o d u c t between a r r a y s j . However, we now wish t o r e d e r i v e t h i s r e s u l t i n a more general c o n t e x t , by c o n s i d e r i n g t h e c a l c u l a t i o n o f s p e c t r a l d e n s i t i e s f o r a general f . Moreover we a l s o r e l a x t h e p r e v i o u s l y assumed c o n d i t i o n A=B. T h i s g e n e r a l case i s r e l a t e d t o t h e implementat i o n o f t h e Lanczos a l g o r i t h m t o n o n - s y m e t r i c m a t r i c e s , which l e a d s t o a b i o r t h o n o r m a l b a s i s s e t [27, 281. When we conside t h e f u n c t i o n s $ j t h a t we have i n t r o d u c e d above, we would now have t o r e c o g n i z e t h a t f o r t h e n o n - s e l f a d j o i n t o p e r a t o r , t h e i r i t e r a t i v e g e n e r a t i o n w i l l now l e a d t o a b i o r t h o n o r m a l s e t o f f u n c t i o n s $ j and $ J ' : <$. $ j ' > = 6.
. (3.16) JI J,J' s t a r t i n g from the and $1 c a l c u l a t e d f r o m t h e r i g h t - h a n d and l e f t - h a n d v e c t o r s o f t h e s p e c t r a l d e n s i t y o f eq. (3.2), a c c o r d i n g t o t h e f o l l o w i n g r e l a t i o n s :
o1
(3.17 a) (3.17b) I n s t e a d o f eq. (3.4) we now have a g e n e r a t i n g e q u a t i o n f o r each t y p e o f b a s i s function: (3.18a) (3.18b) where t h e p r o j e c t i o n o p e r a t o r Pn i s w r i t t e n as n
(3.19) and t h e complex c o e f f i c i e n t ~ ~ i s+ determined l by the normalization condition I t i s e a s i l y shown t h a t , i n t h i s new b a s i s , i; may again be <,+,n+l!,+n+l>=l. r e p r e e n t e d by a symmetric, b u t i n g e n e r a l complex, t r i d i a g o n a l m a t r i x T w i t h c o e f f i c i e n t s B~ as o f f - d i a g o n a l elements. T h i s l e a d s t o t h e f o l l o w i n g t h r e e term recursive relations: (3.20 a) (3.20b) with (3.21)
t h e c o n t i n u e d f r a c t i o n which appears a t t h e r i g h t hand s i d e o f eq. (3.10)
can
152
G. Moro and J.H. Freed
a l s o be used f o r r e p r e s e n t i n g t h e f r e q u e n c y dependence o f s p e c t r a l d e n s i t i e s c a l c u l a t e d w i t h o p e r a t o r s which a r e n o t s e l f - a d j o i n t . An a l t e r n a t i v e f o r m f o r J(LI) i s o b t a i n e d f r o m t h e s o l u t i o n o f t h e e i g e n v a l u e problem f o r T
TQ = QA
(3.23)
Reference 29 r e p o r t s on a m o d i f i c a t i o n o f t h e QR a l g o r i t h m f o r complex symmetric m a t r i c e s which can be a p p l i e d t o t h i s problem. The f r e q u e n c y dependence o f J(w) can t h e n be e x p l i c i t l y w r i t t e n as f o l l o w s :
J(w)/6A*6B =
(
Ql,k2/(iw
+
a,)
(3.24)
However t h i s r e l a t i o n i s more u s e f u l i n d i s p l a y i n g how t h e e i g e n v a l u e s o f t h e s t a r t i n g m a t r i x e n t e r i n t o J ( o J ) than f o r p r a c t i c a l purposes. I n f a c t i t i s conv e n i e n t t o c a l c u l a t e J ( w ) d i r e c t l y f r o m t h e c o n t i n u e d f r a c t i o n w i t h o u t any matrix diagonalization. I t should be emphasized t h a t t h e e x i s t e n c e o f t h e c o n t i n u e d f r a c t i o n r e p r e s e n t a t i o n o f t h e s p e c t r a l d e n s i t y i s n o t assured i f i t i s generated by means o f a b i o r t h o n o r m a l s e t o f b a s i s f u n c t i o n s . I t c o u l d happen, t h a t t h e f o l l o w i n g s c a l a r p r o d u c t [ c f . eqs. (3.18)] vanishes:
(3.25) a c c o r d i n g t o eq. (3.16) i s no so t h a t t h e n o r m a l i z a t i o n o f $n+l and l o n g e r p o s s i b l e . When t h e Lanczos method i s used with a s e l f - a d j o i n t F f o r g e n e r a t i n g o r t h o g o n a l b a s i s f u n c t i o n s , a s i m i l a r s i t u a t i o n i s found o n l y i f t h e r i g h t hand s i d e o f eq. (3.4) vanishes. I n t h i s case t h e o p e r a t o r f i s f a c t o r e d w i t h r e s p e c t t o t h e subspace spanned b y f u n c t i o n s 41,$2,...,$n, and t h e continued f r a c t i o n truncated a t the n-th term represents the spectral density completely. O f course, w i t h b i o r t h o n o r m a l b a s i s f u n c t i o n s , i t would a l s o be l e g i t i m a v e $0 t r u n c a t e t h e c o n t i n u e d f r a c t i o n a t t h e n - t h t e r m i f (l-Pn)r$,, o r ( l - p n )i: +n, o r b o t h vanish. However t h e i r s c a l a r p r o d u c t c o u l d v a n i s h s i m p l y because t h e y are o r t h o g o n a l , and i n t h i s case i t would be i m p o s s i b l e t o d e r i v e a c o n t i n u e d f r a c t i o n r e p r e s e n t a t i o n o f t h e s p e c t r a l d e n s i t y . The s p e c t r a l d e n s i t y o f eq. (3.2) w i t h <sn(P$,\&B>=O c o n s t i t u t e s an obvious example o f such a s i t u a t i o n . Normally, i n t h e c a l c u l a t i o n o f a u t o c o r r e l a t i o n f u n c t i o n s r e l e v a n t t o s p e c t r o s c o p i c observables, such anomalous b e h a v i o r i s n o t found. It i s however a d v i s a b l e t o check t h e magnitude o f t h e norm o f t h e f u n c t i o n i n t h e s c a l a r p r o d u c t o f eq. (3.25), when i t equals zero. As i n t h e s i m p l e case t r e a t e d a t t h e b e g i n n i n g o f t h i s chapter, t h e c o e f f i c i e n t s o f t h e c o n t i n u e d f r a c t i o n are cqmputed b y t h e use o f t h e r e c u r s i v e r e l a t i o n s eqs. (3.20) wi t h t h e o j ' s and +J’s expanded i n a g i v e n s e t o f orthonormal b a s i s f u n c t i o n s f j . R e c a l l i n g t h e d e f i n i t i o n eq. (3.11) and eq. (3.13) f o r xn, t h e r e c u r s i v e r e l a t i o n s may be w r i t t e n as f o l l o w s : (3.26a) (3.26 b) where x, i s t h e column m a t r i x c o n s t r u c t e d w i t h t h e expansion c o e f f i c i e n t s o f bn, and:
153
The Lanczos Algoritlini in Molecular Djxamics
C(
n
( x
=
~
( xn+l)t x
M
n+l
)xn ~
(3.27)
1
(3.28)
=
The computer i m p l e m e n t a t i o n o f these r e l a t i o n s r e q u i r e s t h e s t o r a g e o f f o u r v e c t o r s , as w e l l as two m u l t i p l i c a t i o n s o f a square m a t r i x by a column m a t r i x a t each i t e r a t i o n . The c o m p u t a t i o n a l e f f o r t i s n e a r l y doubled w i t h r e s p e c t t o t h e Lanczos a l g o r i t h m w i t h o r t h o n o r m a l b a s i s f u n c t i o n s . There i s however an i m p o r t a n t e x c e p t i o n w i t h complex symmetric m a t r i c e s M i f t h e s t a r t i n g v e c t o r s a r e complex c o n j u g a t e . That i s , f o r n=l, we l e t : x* = n
xn
(3.29)
frpm .
eqs. (3.26) t h a t eq. (3.29) w i l l be v a l i d f o r a l l Then i t i s e a s i l y shown v a l u e s o f n, p r o v i d e d M=M Therefore, o n l y t h e r e c u r s i v e r e l a t i o n (3.26a) needs t o be e x p l i c i t l y computed, and t h e n o r m a l i z a t i o n c o n d i t i o n eq. (3.28) becomes : xTr x
n
n
=
1
(3.30)
T h i s i s e q u i v a l e n t t o t h e implementation o f e q u a t i o n (3.12) w i t h a " E u c l i d e a n form" o f t h e s c a l a r p r o d u c t , i n s p i t e o f t h e complex number a l g e b r a f o r t h e m a t r i x o p e r a t i o n s . E q u a t i o n 3.30) i s e q u i v a l e n t t o t h e pseudo-norm t h a t we have p r e v i o u s l y i n t r o d u c e d [ 6 $ . I t should be n o t e d here t h a t o f t e n problems i n v o l v i n g o p e r a t o r s f which are n o t s e l f - a d j o i n t , such as t h e s t o c h a s t i c L i o u v i l l e o p e r a t o r s c o n s i d e r e d i n magnetic resonance experiments, can be d e s c r i b e d b y complex symmetric m a t r i c e s i f t h e b a s i s f u n c t i o n s f j a r e p r o p e r l y chosen [6,211. I f t h e t i m e e v o l u t i o n o p e r a t o r f is not "symmetrized", t h e n i t w i l l not, i n g e n e r a l , be p o s s i b l e t o r e p r e s e n t i t by a complex-symmetric m a t r i x , f o r which eq. (3.29) i s t r u e , and methods based upon b i o r t h o n o r m a l spaces, as o u t l i n e d above, become e s s e n t i a l . T h i s approach has been discussed i n d e t a i l by Wassam [301. Many aspects o f t h e g e n e r a l a n a l y s i s o f t h e f a c t o r s i n f l u e n c i n g t h e computer performance o f t h e ( r e a l symmetric) Lanczos a l g o r i t h m [3,4] can a l s o be a p p l i e d t o our t y p e o f problem. I n p a r t i c u l a r t h e s p a r s i t y o f t h e m a t r i x i s c r u c i a l i n d e t e r m i n i n g t h e e f f i c i e n c y o f t h e method f r o m b o t h t h e p o i n t o f view o f computer t i m e and memory needed. U s u a l l y t h e m a t r i x r e p r e s e n t a t i o n s o f t h e t i m e evolut i o n o p e r a t o r s c o n s i d e r e d i n t h e p r e v i o u s c h a p t e r have few n o n - v a n i s h i n g e l e ments. Values around 10-20% are t y p i c a l s p a r s i t i e s , b u t i t can be as low as a few p e r c e n t . Moreover, i n general, one f i n d s t h a t t h e r e i s a decrease i n t h e r e l a t i v e number o f non-zero elements when one i n c r e a s e s t h e degrees of freedom i n c l u d e d i n t h e t i m e e v o l u t i o n o p e r a t o r T. I n o t h e r words, t h e e f f i c i e n c y o f t h e Lanczos a l g o r i t h m i s enhanced i n problems w i t h l a r g e - s i z e m a t r i c e s . I t should be emphasized t h a t t h e a p p l i c a t i o n o f t h e Lanczos a l g o r i t h m c o n s i d e r e d h e r e d i f f e r s f r o m i t s s t a n d a r d use i n numerical a n a l y s i s , i n terms o f t h e quanti t y t o be computed. N o r m a l l y t h e Lanczos a l g o r i t h m i s c o n s i d e r e d in t h e framework o f t h e c a l c u l a t i o n o f eigenvalues, w h i l e we need t h e s p e c t r a l d e n s i t y as a f u n c t i o n o f frequency, which i s w e l l d e s c r i b e d b y t h e c o n t i n u e d f r a c t i o n eq. (3.10), ( b u t see Sect. I V ) . C o r r e s p o n d i n g l y t h e convergence o f t h e n u m e r i c a l method must be c o n s i d e r e d i n a d i f f e r e n t manner. I n p a r t i c u l a r we must e v a l u a t e how, by i n c r e a s i n g t h e number o f steps o f t h e Lanczos a l g o r i t h m , i . e . t h e number o f terms o f t h e c o n t i n u e d f r a c t i o n , t h e s p e c t r a l l i n e s h a p e I ( w ) o f eq. (2.12) approaches i t converged form. I n F i g u r e 2 a t y p i c a l ESR a b s o r p t i o n spectrum i s displayed.
G. Moro and J.H. Freed
154
n Figure 2 ESR a b s o r p t i o n spectrum o f a paramagnetic s p i n probe. The magnetic and m o t i o n a l parameters a r e t h e same as case I i n Table I o f r e f e r e n c e 6. I n numerical a n a l y s i s , t h e convergence o f t h e Lanczos a l g o r i t h m w i t h r e s p e c t t o t h e i n d i v i d u a l e i g e n v a l u e s has been c o n s i d e r e d i n d e t a i l [31,32]. We a r e n o t aware o f any g e n e r a l c r i t e r i o n f o r t h e convergence o f t h e s p e c t r a l d e n s i t y . We have found i t c o n v e n i e n t t o use t h e f o l l o w i n g phenomenological d e f i n i t i o n o f rel a t i v e e r r o r En f o r t h e s p e c t r a l l i n e s h a p e computed w i t h n terms o f t h e continued f r a c t i o n [ 6 ] :
Where I(w) i s t h e converged s p e c t r a l l i n e s h a p e . We t h i n k t h i s d e f i n i t i o n o f En i s a u s e f u l one, because i t i s a measure o f t h e o v e r a l l d i f f e r e n c e between I n ( w ) and I(w). Using t h i s q u a n t i t y , we can d e f i n e t h e s u f f i c i e n t number o f s t e p s ns, as t h e s m a l l e s t n which assures an e r r o r En l e s s t h a n t h e r e q u i r e d accuracy f o r t h e s p e c t r a l lineshape. I n general En decreases w i t h n, b u t i t may not be s t r i c t l y monotonic (see r e f e r e n c e 6 f o r some t y p i c a l t r e n d s ) . F o r an ns i s found t o be much l e s s t h a n t h e dimension N o f t h e accuracy En = m a t r i x [ 6 ] . F o r l a r g e s i z e problems, ns i s t y p i c a l l y o f t h e o r d e r o f N/5 o r l e s s . O f course t h i s c o n t r i b u t e s t o t h e o v e r a l l e f f i c i e n c y o f t h e Lanczos a l g o r i t h m by r e d u c i n g t h e number o f i t e r a t i o n s . I n some problems t h e s p e c t r a l d e n s i t y i s dominated by o n l y a few e i g e n v a l u e s . That i s , i n eq. (3.24) o n l y a few o f t h e w e i g h t i n g f a c t o r s Q1 k2 a r e n o t n e g l i g i b l e . I n these cases t h e Lanczos a l g o r i t h m reproduces h i t h comparable accuracy t h e s p e c t r a l l i n e s h a p e and t h e dominant e i g e n v a l u e s . There a r e o t h e r s i t u a t i o n s , l i k e t h e slow m o t i o n a l ESR spectrum d i s p l a y e d i n F i g u r e 2, where t h e l i n e s h a p e i s a c o m p l i c a t e d f u n c t i o n , which must be accounted f o r by a l a r g e coll e c t i o n o f eigenvalues. I n such cases t h e Lanczos a l g o r i t h m i s m r e e f f i c i e n t i n r e p r o d u c i n g t h e o v e r a l l shape o f I ( U ) t h a n i n computing t h e e i g e n v a l u e s [6]. F i g u r e 3 i l l u s t r a t e s t h i s f a c t i n d i s p l a y i n g t h e computed eigenvalues o f t h e ESR problem c o n s i d e r e d i n F i g . 2. The d o t s i n d i c a t e t h e e x a c t e i g e n v a l u e s o f t h e The crosses r e p r e s e n t t h e s t a r t i n g m a t r i x M which has a dimension equal t o 42. 16 eigenvalues o f t h e t r i d i a g o n a l m a t r i x which approximate t h e l i n e s h a p e t o an accuracy of From t h e F i g u r e i t i s c l e a r t h a t t h e r e i s no s i m p l e r e l a t i o n between o v e r a l l accuracy i n t h e l i n e s h a p e f u n c t i o n and accuracy i n t h e approximate eigenvalues. Most o f them, i n fact,cannot be s i m p l y a s s o c i a t e d on a oneto-one b a s i s w i t h p a r t i c u l a r exact eigenvalues. Even when t h i s i s p o s s i b l e , t h e e r r o r i n t h e approximate eigenvalues i s f a r g r e a t e r t h a n t h e accuracy o f
The Lanczos Algorithm in Molecular Dynamics
155
t
Figure 3 D i s t r i b u t i o n o f t h e exact ( d o t s ) and approximate ( t o t h e ESR spectrum d i s p l a y e d i n F i g u r e 2.
@.
) eigenvalues r e l a t i v e
f o r t h e f u l l spectrum ( w i t h t h e o n l y e x c e p t i o n b e i n g t h e e i g e n v a l u e h a v i n g t h e g r e a t e s t i m a g i n a r y p a r t ) . We f i n d t h e r e f o r e , t h a t t h e Lanczos a l g o r i t h m genera t e s c o n t i n u e d f r a c t i o n s which t e n d t o o p t i m i z e t h e o v e r a l l shape o f t h e spectrum, r a t h e r t h a n s e t s of eigenvalues. W h i l e a t f i r s t such a s t a t e m e n t m i g h t appear c o n t r a d i c t o r y , i t i s based on t h e f a c t t h a t t h e s p e c t r a l d e n s i t y i s u s u a l l y dominated by t h e e i g e n v a l u e s o f small r e a l p a r t , and t h e Lanczos a l g o r i t h m i s a b l e t o approximate them i n d i v i d u a l l y , o r i n p r o v i d i n g an "average" t o a c l u s t e r of e i g e n v a l u e s s u f f i c i e n t t o r e p r e s e n t t h e s p e c t r a l d e n s i t y . A q u a l i t a t i v e j u s t i f i c a t i o n o f t h i s b e h a v i o r i s i m p l i c i t i n t h e s o - c a l l e d method o f moments [23], s i n c e t h e subspaces generated b y t h e Lanczos a l g o r i t h m t e n d t o approximate t h e o v e r a l l b e h a v i o r o f T by r e p r o d u c i n g i t s f i r s t moments w i t h respect t o t h e s t a r t i n g vectors. We mention t h a t i n t h e t y p e s of problems we have been d i s c u s s i n g , t h e r e have almost never been s i g n i f i c a n t e f f e c t s due t o r o u n d - o f f e r r o r , which, however, i s t h e main weakness o f t h e Lanczos a l g o r i t h m i n t h e c a l c u l a t i o n o f e i g e n v a l u e s . Q u a l i t a t i v e l y , t h i s can be r e l a t e d t o t h e same f a c t t h a t t h e s p e c t r a l d e n s i t y ccnverges much sooner t h a n t h e c o r r e s p o n d i n g eigenvalues, so t h a t t h e l o s s o f I t i s known, i n f a c t , t h a t s p u r i o u s o r t h o g o n a l i t y i s not yet s i g n i f i c a n t . e i g e n v a l u e s appear a f t e r a l a r g e enough number o f Lanczos s t e p s have been c a l c u l a t e d i n o r d e r t o o b t a i n eigenvalues which are v e r y c l o s e t o t h e i r e x a c t v a l u e s [32]. As n o t e d above, t h i s i m p l i e s many more s t e p s t h a n are needed f o r t h e convergence o f t h e s p e c t r a l d e n s i t i e s . On t h e o t h e r hand, o n l y a t h e o r y , which i s s t i l l l a c k i n g , t h a t i s s p e c i f i c a l l y designed t o analyze t h e convergence o f the spectral densities, could give a q u a n t i t a t i v e estimate o f the e f f e c t s o f t h e round-off e r r o r . The l a s t p a r t of t h i s c h a p t e r w i l l be devoted t o a d i s c u s s i o n o f t h e c h o i c e o f basis functions for representing the time evolution operator. F i r s t of a l l , t h e r e i s a t r u n c a t i o n problem, s i n c e t h e f u n c t i o n s f j ( z 1, which f o r m a
G. Moro and J.H. Freed
156
complete s e t t o r e p r e s e n t t h e H i l b e r t space a s s o c i a t e d w i t h t h e f u n c t i o n s o f t h e s t o c h a s t i c v a r i a b l e s z, are i n general i n f i n i t e i n number. I n computer c a l c u l a t i o n s one can o n l y handle f i n i t e m a t r i c e s . T h e r e f o r e one must t r u n c a t e t h e m a t r i x M, i . e . one r e p r e s e n t s t h e o p e r a t o r f i n a f i n i t e b a s i s s e t o f f u n c t i o n s f,, f 2 , fN, under t h e h y p o t h e s i s t h a t t h e r e m a i n i n g f u n c t i o n s l e a d t o n e g l i g i b l e c o n t r i b u t i o n s . The t r u n c a t i o n o f t h e b a s i s s e t leads t o i n c o r r e c t f e a t u r e s i n t h e s p e c t r a l d e n s i t y , w h i l e t h e computer c a l c u l a t i o n c o u l d become e x c e e d i n g l y u n w i e l d y w i t h a t o o l a r g e b a s i s s e t . I n t h e absence o f theor e t i c a l e s t i m a t e s o f t h e e r r o r i n I ( w ) caused by t h e t r u n c a t i o n , t h e convergence must be v e r i f i e d d i r e c t l y f r o m t h e computed r e s u l t s by comparing IN(^) o b t a i n ed w i t h i n c r e a s i n g values o f N. T h i s i s i n t h e same s p i r i t as t h e a n a l y s i s o f convergence w i t h r e s p e c t t o t h e number o f s t e p s o f t h e Lanczos a l g o r i t h m . Again we can use eq. (3.29) as a measure o f t h e t r u n c a t i o n e r r o r . C l e a r l y , as t h e number o f s t o c h a s t i c v a r i a b l e s increases, t h e t r u n c a t i o n problem becomes more d i f f i c u l t and more t i m e consuming.
...,
U s u a l l y , i n a g i v e n problem, one has a c h o i c e amongst d i f f e r e n t t y p e s o f b a s i s f u n c t i o n s . The b e s t b a s i s f u n c t i o n s would be t h o s e which a l l o w t h e most e f f i c i e n t t r u n c a t i o n , t h u s y i e l d i n g t h e s m a l l e s t m a t r i x t o be handled by means o f t h e Lanczos a l g o r i t h m . There a r e no s i m p l e g u i d e l i n e s f o r such a choice, a p a r t f r o m t h e obvious r u l e t h a t a l l t h e symmetries o f t h e t i m e e v o l u t i o n o p e r a t o r should be t a k e n i n t o account. (Other f e a t u r e s i n s e l e c t i n g a b a s i s s e t i n c l u d e t h e d e s i r a b i l i t y t o maximize t h e s p a r s i t y o f t h e m a t r i x and t h e ease o f c a l c u l a t i n g t h e m a t r i x elements). O n l y by e x p e r i e n c e w i t h each p a r t i c u l a r c l a s s o f problems can one f e e l c o n f i d e n t i n s e l e c t i n g an o p t i m a l s e t o f b a s i s f u n c t i o n s . I n some problems t h e d e s i r e f o r a minimal b a s i s s e t suggests t h e use o f nono r t h o g o n a l f u n c t i o n s . T h i s i s l i k e l y t o happen when d e a l i n g w i t h p h y s i c a l systems c h a r a c t e r i z e d by mean p o t e n t i a l s which c o n f i n e t h e s t o c h a s t i c v a r i a b l e s z around some s t a b l e s t a t e s o r conformations. Correspondingly, smal 1 amplit u d e motions around t h e s t a b l e s t a t e s ( l i b r a t i o n s ) and t r a n s i t i o n s amongst d i f f e r e n t s t a t e s become t h e r e l e v a n t dynamical processes. T h i s t y p e o f problem i s commonly encountered i n such f i e l d s o f r e s e a r c h as chemical k i n e t i c s i n condensed phases [33-351 o r i n t h e s t u d y o f c o n f o r m a t i o n a l dynamics o f c h a i n molec u l e s o r polymers [36-381. From an a n a l y s i s o f t h e a s y m p t o t i c behaviour o f t h e s o l u t i o n s o f t h e d i f f u s i o n equation, i t has been shown t h a t such problems can be c o n v e n i e n t l y s o l v e d w i t h non-orthoganal b a s i s f u n c t i o n s o f t h e f o l l o w i n g t y p e [ 39-42] : (3.32) As long as one i s a b l e t o mimic e f f i c i e n t l y , by means o f t h e f u n c t i o n s g j , t h e fundamental processes o c c u r r i n g i n h i n d e r e d systems, v e r y few elements o f such a b a s i s s e t a r e needed i n t h e c a l c u l a t i o n o f t h e s p e c t r a l d e n s i t y . The r o l e played b y t h e f a c t o r P'/' i n eq. (3.32) should be emphasized, s i n c e i t a l l o w s one t o express P'/'sEqand P1126B o f eq. (3.2) s i m p l y as l i n e a r combinations o f properlFqchosen f u @ t i o n s f j ' s . On t h e o t h e r hand, w i t h orthonormal b a s i s f u n c t i o n s one must expand P 1 l 2 i n a l a r g e s e t o f f u n c t i o n s , as a consequence o f i t s sharp d i s t r i b u t i o n argflnd t h e s t a b l e s t a t e o f t h e system. ( A l t e r n a t i v e l y one may i n t r o d u c e f i n i t e d i f f e r e n c e o r f i n i t e element methods t o choose " l o c a l i z e d " b a s i s s e t s [ 4 3 ] ) . O f course t h e i m p l e m e n t a t i o n o f t h e Lanczos a l g o r i t h m must be changed when p a s s i n g f r o m orthonormal t o non-orthogonal b a s i s f u n c t i o n s . One may w r i t e t h e m a t r i x f o r m f o r t h e s p e c t r a l d e n s i t y eq. 3.2) by con i d e r i n g t h e r e p r e s e n t a t i o n A and P' B i n such a nono f t h e o p e r a t o r r and o f t h e f u n c t i o n s P1 o r t h o g o n a l b a s i s . One q u i c k l y sees thatef!he calculafyon o f the spectral density i s now c l o s e l y r e l a t e d t o t h e g e n e r a l i z e d e i g e n v a l u e problem Ax=xBx 281. A l t e r n a t i v e l y , we can s t a r t w i t h t h e r e c u r s i v e r e l a t i o n o f eq.(3.7) [ o r eq.
$
7
The Lanczos Algoritlzm in Molecular Dynamics
157
( 3 . 1 8 ) ] and implement them w i t h non-orthogonal b a s i s f u n c t i o n s , i n o r d e r t o c a l c u l a t e t h e c o e f f i c i e n t s o f t h e t r i d i a g o n a l m a t r i x . We s h a l l d i s c u s s i n d e t a i l t h i s second r o u t e u s i n g t h e r e c u r s i v e r e l a t i o n eq. ( 3 . 7 ) ; i.e. f o r s p e c t r a l d e n s i t i e s o f eq. (3.2) c h a r a c t e r i z e d by A=B and a s e l f - a d j o i n t o p e r a t o r r. One f i n d s t h a t t h e m a t r i x r e c u r s i v e r e l a t i o n eq. (3.12) c o n t i n u e s t o h o l d i f t h e g e n e r i c column m a t r i x xn r e p r e s e n t s t h e expansion c o e f f i c i e n t s o f Jln on t h e non-orthogonal b a s i s s e t a c c o r d i n g t o eq. (3.13), and i f t h e m a t r i x M i s i m p l i c i t l y d e f i n e d by t h e f o l l o w i n g r e l a t i o n : ? . f .=): M. f J k J k k
(3.33)
One does need t o m o d i f y t h e n o r m a l i z a t i o n c o n d i t i o n and t h e c a l c u l a t i o n o f diagonal c o e f f i c i e n t s , according t o the f o l l o w i n g r e l a t i o n s :
t
Yn xn
1
=
(3.34)
(3.35) where y matrix
9
i s an a u x i l i a r ' y column m a t r i x c a l c u l a t e d f r o m t h e n o r m a l i z a t i o n
sJk .
= < f .f
JI k
>
(3.36)
and t h e a r r a y xn a c c o r d i n g t o t h e e q u a t i o n : Yn =
s
xn
(3.37)
The o p e r a t i o n s o f t h e s t a n d a r d Lanczos a l g o r i t h m are e a s i l y m d i f i e d t o t h e preI n p a r t i c u l a r , d u r i n g an i t e r a t i v e c y c l e one must s t o r e s e n t case [39,42]. t h r e e a r r a y s t o r e p r e s e n t t h e v e c t o r s xn, Xn-1 and yn. By comparison w i t h t h e s t a n d a r d Lanczos a l g o r i t h m , t h e s t o r a g e needed i s t h e n increased by one square m a t r i x ( S ) and one a r r a y (yn), w h i l e t h e c o m p u t a t i o n a l e f f o r t i s i n c r e a s e d by t h e m u l t i p l i c a t i o n o f a square m a t r i x by a column m a t r i x a t each Therefore, t h e use o f s t e p , because o f t h e c a l c u l a t i o n o f Yn f r o m eq. (3.37). non-orthogonal b a s i s f u n c t i o n s i s convenient o n l y when it a l l o w s a c o n s i d e r a b l e r e d u c t i o n o f t h e s i z e o f t h e m a t r i c e s . T h i s i s g e n e r a l l y t h e case i n p h y s i c a l problems c h a r a c t e r i z e d by s t r o n g h i n d e r i n g p o t e n t i a l s [42]. U n l i k e t h e case w i t h o r t h o g o n a l f u n c t i o n s , t h e e f f e c t o f t h e t r u n c a t i o n o f t h e b a s i s t o t h e f i r s t N elements cannot be c a r r i e d o u t s i m p l y by n e g l e c t i n g t h e elements o f M o u t s i d e t h e f i r s t NxN b l o c k . As a m a t t e r o f f a c t , t h e use o f a f i n i t e b a s i s s e t which d e f i n e s an N-dime s i o n a l subspace €N, i s e q u i v a l e n t t o c o n s i d e r i n g i n eq. (3.2) t h e f u n c t i o n P1Y2aA and t h e o p e r a t o r i; p r o j e c t e d onto EN according t o t h e f o l l o w i n g projectorepN:
N
PN =
1 (fj>( j =1
s
-')jk'fk(
C o r r e s p o n d i n g l y , i n t h e r e c u r s i v e r e l a t i o n eq. ( 3 . 7 ) , o p e r a t o r T, by i t s p r o j e c t e d form f .
i;’
and t h e m a t r i x
I
(3.38) one must s u b s t i t u t e t h e
P FP N N
M i s i m p l i c i t l y defined by the f o l l o w i n g equation:
P Ff. = E M f N J k kJk A f t e r s u b s t i t u t i o n o f t h e p r o j e c t i o n o p e r a t o r o f eq. (3.38),
(3.40) one o b t a i n s :
158
G. Moro and J.H. Freed
M = S - ' R w i t h R c o n s t r u c t e d w i t h t h e m a t r i x elements o f
(3.41)
?:
R . = (3.42) Jk Jl-1 We n o t e t h a t eq. (3.41) i n d i c a t e s t h a t t h e c a l c u l a t i o n o f M i s m r e complicated t h a n f o r t h e case o f orthonormal b a s i s sets. There are s e v e r a l methods f o r t h e c a l c u l a t i o n o f M. F i r s t M can be computed d i r e c t l y f r o m eq. (3.41), b u t t h i s would r e q u i r e a l a r g e amount o f computation t i m e because o f t h e i n v e r s i o n o f S . Secondly, as suggested by Jones and coworkers [44,45]; one can implement t h e from r e c u r s i v e r e l a t i o n (3.12) by c a l c u l a t i n g a t each s t e p t h e a r r a y S-lRx, t h e s o l u t i o n o f t h e l i n e a r system o f e q u a t i o n s w i t h Rxn as knoyn c o e f f i c i e n t s . T h i r d l y , i n some d i f f u s i o n a l problems, one can w r i t e r f j as a l i n e a r combination o f b a s i s f u n c t i o n s by c o n s i d e r i n g e x p l i c i t l y t h e o p e r a t o r f o r m o f [39,42]. Thus t h e elements o f M are d e r i v e d by p r o j e c t i n g o u t , a c c o r d i n g t o eq. (3.38), o n l y those f u n c t i o n s which do not belong t o EN [42].
SUmARY We have, i n t h i s review, o u t l i n e d how t h e Lanczos a l g o r i t h m i s capable o f p l a y ing a significant r o l e i n the calculation o f spectral densities that arise i n t h e s t u d y o f m o l e c u l a r dynamics. T h i s i s , i n p a r t , due t o i t s c o m p u t a t i o n a l v a l u e and a l s o t o i t s c l o s e r e l a t i o n s h i p t o i m p o r t a n t t h e o r e t i c a l methods i n s t a t i s t i c a l p h y s i c s . We have p o i n t e d o u t t h a t t h e s e problems can o f t e n be r e p r e s e n t e d by complex symmetric m a t r i c e s , and t h e g e n e r a l i z a t i o n o f t h e Lanczos a l g o r i t h m t o such m a t r i c e s has been g e n e r a l l y s u c c e s s f u l . F u r t h e r work i s c l e a r l y needed i n e s t a b l i s h i n g a b e t t e r u n d e r s t a n d i n g o f how t h e Lanczos algor i t h m e f f e c t i v e l y p r o j e c t s out a useful representation o f the spectral d e n s i t i e s w i t h much l e s s e f f o r t t h a n i s r e q u i r e d t o o b t a i n a good s e t o f eigenvalues. As problems become more complicated, and t h e m a t r i x r e p r e s e n t a t i o n s become l a r g e r , t h e r e i s concern f o r c a r e f u l s e l e c t i o n o f b a s i s v e c t o r s i n c l u d i n g e f f e c t i v e means o f " p r u n i n g " o u t unnecessary b a s i s v e c t o r s . Also, problems due t o accumulated r o u n d - o f f can become more s e r i o u s . Thus e f f i c i e n t t e c h n i q u e s f o r p a r t i a l r e - o r t h o g o n a l i z a t i o n may be c a l l e d f o r [4]. I n systems w i t h s t r o n g t r a p p i n g p o t e n t i a l s , t h e use o f non-orthogonal f u n c t i o n s l o o k s promising. So f a r , however, t h e method has been t e s t e d o n l y i n cases where t h e s p e c t r a l d e n s i t i e s can s t i l l be r e a d i l y c a l c u l a t e d w i t h t h e use o f s t a n d a r d orthonormal b a s i s f u n c t i o n s [39,42]. The a p p l i c a t i o n t o c h a l l e n g i n g problems dependent on s e v e r a l degrees o f freedom r e q u i r e s a search f o r o p t i m a l b a s i s f u n c t i o n s . Otherwise, t h e approach i s s t r a i g h t f o r w a r d apply. Although we have emphasized i n t h i s work t h e c a l c u l a t i o n o f s p e c t r a l d e n s i t i e s , what we have s a i d h e r e g e n e r a l i z e s v e r y n i c e l y t o t h e a n a l y s i s o f t i m e domain experiments on m o l e c u l a r dynamics. I n t h e c o n t e x t o f l i n e a r response theory, t h e F o u r i e r t r a n s f o r m o f t h e s p e c t r a l d e n s i t y ( o r f r e q u e n c y spectrum) i s j u s t t h e t i m e c o r r e l a t i o n f u n c t i o n ( o r t i m e domain response). Thus, many modern time-domain experiments may be d e s c r i b e d by f i r s t c a l c u l a t i n g t h e s p e c t r a l d e n s i t y by t h e above methods and t h e n u s i n g FFT r o u t i n e s t o o b t a i n t h e a s s o c i a t e d c o r r e l a t i o n f u n c t i o n s . T h i s equivalence again emphasizes t h e r o l e o f t h e Lanczos a l g o r i t h m i n s e l e c t i n g o u t and a p p r o x i m a t e l y r e p r e s e n t i n g t h e eigenvalues o f small r e a l p a r t , i . e . t h e s l o w l y decaying components, which u s u a l l y dominate t h e time-domain experiments. There i s , however, a s p e c i a l case o f t i m e domain experiments: v i z . t h e s p i n echo (and i t s o p t i c a l analogues). These experiments may be t h o u g h t o f , t o a f i r s t approximation, as c a n c e l i n g o u t t h e e f f e c t s o f t h e i m a g i n a r y p a r t s o f t h e eigen-
159
The Lanczos Algorithm in Molecular Dynamics
v a l u e s t h a t c o n t r i b u t e t o t h e t i m e domain response and t h e r e b y t o p r o v i d e g r e a t s e n s i t i v i t y i n t h e experiment t o t h e r e a l p a r t s . A n a l y s i s o f spin-echo s p e c t r o scopy by t h e Lanczos a l g o r i t h m has met w i t h some success [46,47], because, as n o t e d above, t h e e i g e n v a l u e s o f small r e a l p a r t t e n d t o dominate t h e experimenta l o b s e r v a t i o n s , and t h e y are t h e ones t h a t are a t l e a s t r o u g h l y approximated by t h e Lanczos a l g o r i t h m . However, f u r t h e r c o m p u t a t i o n a l developments along t h i s l i n e would have t o address how t o o b t a i n b e t t e r e s t i m a t e s o f t h e s e " s m a l l e r " e i g e n v a l u e s by improvements on t h e b a s i c Lanczos t e c h n i q u e . ACKNOWLEDGMENTS T h i s work was supported by NSF S o l i d S t a t e Chemistry Grant DMR 81-02047 and NSF Grant CHE 8319826 (JHF) and by t h e I t a l i a n Research C o u n c i l (CNR) t h r o u g h i t s Centro S t u d i S u g l i S t a t i M o l e c o l a r i R a d i c a l i c i ed E c c i t a t i (GM). REFERENCES
1. R. Kubo, J. Phys. SOC. Japan 12 (1957) 570. 2. C. Lanczos, J. Res. Natn. Bur. Stand. 45 (1950) 255; 49 (1952) 33. 3. B . N . P a r l e t t , The S y m n e t r i c E i g e n v a l u e Problem ( P r e n t i c e H a l l , Englewood C l i f f s , N.J., 1980). 4. 3. Cullum and R. Willoughby, Lanczos A l g o r i t h m s f o r L a r g e Symnetric E i g e n v a l u e Computations, Vol I Theory ( B i r k h a u s e r , Basel, 1985).
5. G . Moro and J.H.
Freed, J. Phys. Chem. 84 (1980) 2837.
6. G. Moro and J.H.
Freed, J. Chem. Phys. 74 (1981) 3757.
7. G. Moro and J.H.
Freed, J. Chem. Phys. 75 (1981) 3157.
8. H. Mori, Prog. Theor. Phys. 34 (1965) 399. 9. J.T. Hynes and J.M. Deutch, i n : P h y s i c a l Chemistry, Vol. 118, eds. H. E y r i n g , 0. Henderson and W. J o s t (Academic Press, New York, 1975). 10. L.E. R e i c h l , A Modern Course i n S t a t i s t i c a l Physics, A u s t i n (1980).
U. of Texas Press,
11. B.J. Berne, i n : P h y s i c a l Chemistry, Vol. 88, eds. H. E y r i n g , D. Henderson and W.
J o s t (Academic Press, New York,
1975).
12. J.P.
R y c k a e r t and A. Bellemans, Phys. L e t t . 30 (1975) 123.
13. L.P.
Hwang and J.H. Freed, 3. Chem. Phys. 63 (1975) 118.
14. G. Moro and P.L.
Nordio, J. Phys. Chem. 89 (1985) 597.
15. M. Fixman and K. R i d e r , J. Chem. Phys. 51 (1969) 2425, 16. A.E.
S t i l l m a n and J.H.
Freed, J. Chem. Phys. 72 (1980) 550.
17. J.H. Freed, i n : G.S.
S t o c h a s t i c Processing, Formalism and A p p l i c a t i o n s , Argawal and S. Oattagupta ( S p r i n g e r , B e r l i n , 1983), p. 220.
18. L. Van Hove, Phys. Rev. 95 (1954) 249.
eds.
G.Moro and J.H. Freed
160
19. R. Kubo, Adv. Chem. Phys. 16 (1969) 101. 20. J.H. P.W.
Freed, i n : Electron Spin Relaxation i n Liquids, A t k i n s (Plenum, New York, 1972), p. 387.
21. E. M e i r o v i t c h , 77 (1982) 3915 22. C.F.
eds. L.T.
Muss and
D. I g n e r , E. I g n e r , G. Moro and J.H. Freed, J. Chem. Phys.
Polnaszek
23. Yu.Y. Vorobyev New York, 1965
G.V.
Freed, J . Chem. Phys. 58 (1973) 3189.
Bruno and J.H.
Method o f Moments i n Applied Mathematics (Gordon and Breach,
24. H.S. Wall, Analytic Theory of Continued Fractions (van Nostrand, P r i n c e t o n , 1948). 25. G.H. Golub and R. Underwood, New York, 1977), p. 361.
in:
Mathematical Software I11 (Academic Press,
26. G.P. Z i e n t a r a and J.H. Freed, i n : Proceedings of the Ninth International Conference on Liquid Crystals (Bangalore, I n d i a , 1982). 27.
L. Fox, Introduction t o Numerical Linear Algebra ( O x f o r d U n i v e r s i t y , New York, 1965).
28. J. W i l k i n s o n , The Algebraic Eigenvalue Problem, (Oxford, London, 1965). 29. R.G. Gordon and T. Messenger, i n : Electron Spin Relaxation in l i q u i d s , eds. L.T. Muus and P.W. A t k i n s (Plenum, New York, 1972), p. 219. 30. W.A.
Wassam, J . Chem. Phys. 82 (1985) 3371, 2286.
31. C.C.
Paige, J . I n s t . Maths. Appl. 10 (1972) 373, 18 (1976) 341.
32. W.Kahan and B.N. P a r l e t t , i n : Sparse Matrix Computations, eds. J . Bunch and D. Rose (Academic Press, New York, 1976), p. 131. 33. H.A.
Kramers, P h y s i c a 7 (1940) 284.
34. J.L.
Skinner and P.G.
35. D.G.
T r u h l a r , W.L.
Wolynes, J. Chem. Phys. 69 (1978) 2143.
Hase and J.T.
Hynes, J . Phys. Chem. 87 (1983) 2664.
36. D. Chandler, J . Chem. Phys. 68 (1978) 2959. 37. M. Fixman, J. Chem. Phys. 69 (1978) 1527; 1538. 38. E. H e l f a n d and J . S k o l n i c k , J. Chem. Phys. 77 (1982) 5714.
39. G. Moro and P.L.
Nordio, Chem. Phys. l e t t . 96 (1983) 192.
40. G. Moro and P.L.
Nordio, Mol. Phys. 56 (1985),
41. G. Moro and P.L.
Nordio, Mol. Phys.;
i n press.
i n press.
42. G. Moro; s u b m i t t e d t o Chem. Phys. 43. A.E.
S t i l l m a n , G.P.
Z i e n t a r a and J.H. Freed, J. Chem. Phys. 71 (1979) 113.
44. R. Jones and T. King, P h i l . Mag. 647 (1983) 481.
The Lanczos Algorithm in Molecular Dynamics 45. R. Jones, i n : The Recursion Method and I t s A p p l i c a t i o n s , eds. D.G. and D.L. Weaire ( S p r i n g e r , B e r l i n , 1985), p. 132. 46. L.J.
Schwartz, A.E.
S t i l l m a n and J.H.
47. G.L. M i l l h a u s e r and J . H .
161
Pettifor
Freed, J. Chem. Phys. 77 (1982) 5410.
Freed, J. Chem. Phys. 81 (1984) 37.
This Page Intentionally Left Blank
Large Scale Eigenvalue Problems I. Cullum and R.A. Willoughby (Editors) 0 Elsevier Science Publishers B.V. (NorthHolland), 1986
163
INVESTIGATION OF NUCLEAR DYNAMICS IN MOLECULES BY MEANS OF THE LANCZOS ALGORITHM Erwin H a l l e r IBM Program P r o d u c t Development Center, S i n d e l f i n g e n ,
Germany
H o r s t Koppel T h e o r e t i s c h e Chemie, U n i v e r s i t y o f H e i d e l b e r g , H e i d e l b e r g , Germany
The s t u d y of multi-mode n u c l e a r dynamics on t h e coupled p o t e n t i a l energy s u r f a c e s o f molecules has become a c h a l l e n g i n g t h e o r e t i c a l and c o m p u t a t i o n a l t a s k . I n t h i s c o n t r i b u t i o n we r e p o r t on o u r main r e s u l t s i n t h i s f i e l d w i t h p a r t i c u l a r emphasis on t h e c o m p u t a t i o n a l problems encountered and on t h e r o l e p l a y e d by t h e Lanczos A l g o r i t h m . The computation
of t h e s p e c t r a l i n t e n s i t y d i s t r i b u t i o n amounts t o t h e diagon a l i z a t i o n o f r e a l symmetric m a t r i c e s which a r e v e r y l a r g e b u t sparse. A s t r a i g h t f o r w a r d m o d i f i c a t i o n of t h e s t a n d a r d Lanczos A l g o r i t h m a l l o w s t h e d i a g o n a l i z a t i o n o f complex symmetric m a t r i c e s . W i t h i n o u r model t h i s i s used t o i n v e s t i gate t h e mixing o f the e l e c t r o n i c species i n t h e v i b r o n i c eigenstates.
INTRODUCTION The i n v e s t i g a t i o n o f t h e v i b r a t i o n a l s t r u c t u r e i n t h e e l e c t r o n i c spectrum o f a m o l e c u l e a l l o w s f o r a v e r y d e t a i l e d i n s i g h t i n t o i t s c o n f i g u r a t i o n and t h e u n d e r l y i n g dynamic processes. E s p e c i a l l y t h e i n t e r a c t i o n o f e l e c t r o n i c and v i b r a t i o n a l m o t i o n i n p o l y a t o m i c molecules interaction"
-
-
commonly termed " v i b r o n i c
o f t e n y i e l d s pronounced c h a r a c t e r i s t i c s t r u c t u r e s i n t h e c o r c s -
ponding s p e c t r a [l]. I n t h i s c o n t r i b u t i o n we r e p o r t on o u r main r e s u l t s i n t h e f i e l d o f n u c l e a r dynamics i n p o l y a t o m i c molecules w i t h p a r t i c u l a r emphasis on t h e c o m p u t a t i o n a l problems encountered. I n t h e f i r s t s e c t i o n a b r i e f account i s g i v e n o f some b a s i c q u e s t i o n s a s s o c i a t e d w i t h m o l e c u l a r spectroscopy. Next, a model w i l l be p r e s e n t e d which a l l o w s f o r a t h e o r e t i c a l i n v e s t i g a t i o n o f v i b r o n i c i n t e r a c t i o n s i n m o l e c u l a r s p e c t r a . The t r a n s f o r m a t i o n o f t h e p h y s i c a l problem t o a n u m e r i c a l one is d i s c u s s e d i n t h e f o l l o w i n g s e c t i o n . The Lanczos A l g o r i t h m i s shown t o be i d e a l l y s u i t e d t o
164
E. Haller and H. Koppel
diagonalize the model Hamiltonian which i s represented a s a large sparse matrix. Finally several i l l u s t r a t i v e r e s u l t s of o u r c a l c u l a t i o n s will be presented f o r s u e c i f i c molecules.
P H Y S I C A L BACKGROUND
The main constituents of the internal energy o f a molecule stem from t h e e l e c t r o nic motion and from the nuclear motion. The l a t t e r i s conveniently separated i n t o vibrational a n d rotational motion. These contributions d i f f e r s i g n i f i c a n t l y i n magnitude: typical e l e c t r o n i c energies l i e in the region of the v i s i b l e and the u l t r a v i o l e t radiation ( 1 10 eV). Vibrational e x c i t a t i o n s require
...
energies of the order o f a tenth of a n electron Volt which corresponds t o the infrared region of the electromagnetic spectrum. Thus, vibrational t r a n s i t i o n s a r e one t o two orders o f magnitude below the energies of e l e c t r o n i c t r a n s i t i o n s . Another two orders of magnitude below - in the microwave region of the r o t a t i o n a l degrees of freedoms a r e observed.
-
excitations
Guided by t h i s f a c t i t i s obvious t h a t one aims a t a separate treatment of t h e e f f e c t s associated with the e l e c t r o n i c , the vibrational and t h e r o t a t i o n a l motion. Indeed, t h i s so-called a d i a b a t i c concept found wide application and success in the study of diatomic molecules [ Z ] . Spectra of diatomic molecules a r e usually very easy t o survey: e l e c t r o n i c bands a r e well separated from each other a n d show a very regular vibrational f i n e s t r u c t u r e . The l a t t e r emerges from d i f f e r e n t vibrational s t a t e s of t h e e l e c t r o n i c s t a t e s involved. Finally, each vibrational band shows a sub-structure due t o r o t a t i o n a l e x c i t a t i o n s . This approach i s appealing in so f a r as i t allows f o r a concise i n t e r p r e t a t i o n of molecular spectra in terms of individual quantum numbers f o r e l e c t r o n i c , vibrational and r o t a t i o n a l s t a t e s [2]. The s i t u a t i o n changes dramatically when we study polyatomic molecules. The f a c t t h a t we a r e dealing here with several vibrational degrees of freedom implies n o t only a q u a n t i t a t i v e change b u t r a t h e r a q u a l i t a t i v e one. Since now the e l e c t r o n i c l e v e l s can vary as a function of several nuclear coordinates t h e probability t h a t two adjacent e l e c t r o n i c surfaces come very close o r even coincide with each o t h e r becomes very high. Thus e l e c t r o n i c energy d i f f e r e n c e s become comparable with t h e vibrational ones, t h e nuclei cease t o be confined t o a s i n g l e potential energy surface and i t i s no longer j u s t i f i e d t o t r e a t the e l e c t r o n i c motion and the nuclear motion separately. The combined action of e l e c t r o n i c a n d vibrational phenomena i s commonly termed "vibronic coupling", the "jumping" of the nuclei between d i f f e r e n t potential
Investigation of' Nuclear Dynamics in Molecules
165
energy s u r f a c e s i s c a l l e d a " n o n a d i a b a t i c " e f f e c t [1,3]. I n a d d i t i o n , t h e v a r i o u s v i b r a t i o n a l degrees o f freedom g e n e r a l l y cannot be t r e a t e d s e p a r a t e l y . T h i s i s known as multimode c o u p l i n g and i s observed i n many p o l y a t o m i c molec u l e s [4]. Consequently, t h e c o m p l e x i t y o f t h e t h e o r e t i c a l d e s c r i p t i o n o f t h e s p e c t r a o f p o l y a t o m i c molecules i s c o n s i d e r a b l y h i g h e r compared t o d i a t o m i c molecules. P a r t i c u l a r l y , i t w i l l be seen t h a t t h e v i b r o n i c l i n e s o f t h e spectrum o f a p o l y a t o m i c m o l e c u l e o f t e n appear c o m p l e t e l y e r r a t i c
-
i f n o t t o say c h a o t i c .
G e n e r a l l y , no s i m p l e r u l e s can be found which d e t e r m i n e t h e p o s i t i o n s o f t h e s p e c t r a l l i n e s and t h e i r i n t e n s i t i e s . V i b r o n i c e f f e c t s a r e n o t o n l y o f i n t e r e s t i n t h e v a r i o u s branches o f s p e c t r o s copy such as p h o t o e l e c t r o n spectroscopy, a b s o r p t i o n spectroscopy, e m i s s i o n s p e c t r o s c o p y o r p r e d i s s o c i a t i o n . Rather, t h e y can p l a n an i m p o r t a n t r o l e a l s o i n chemical processes, i n c o l l i s i o n s i n v o l v i n g molecules, i n resonance Raman s c a t t e r i n g and i n decay processes such as r a d i a t i v e o r n o n r a d i a t i v e decays o f e x c i t e d s t a t e s o f molecules. Here, however, we s h a l l c o n c e n t r a t e on absorpt i o n and p h o t o e l e c t r o n spectroscopy and t o u c h b r i e f l y upon decay processes.
THE THEORETICAL MODEL The H a m i l t o n i a n H i s t h e o p e r a t o r o f t h e energy i n a quantum system. I n t h e case o f molecules i t i s c o n v e n i e n t l y d i v i d e d i n t o t h r e e p a r t s [ 5 ] :
TN and Te a r e t h e k i n e t i c e n e r g i e s o f t h e n u c l e i and t h e e l e c t r o n s , r e s p e c t i v e l y . The t o t a l p o t e n t i a l energy U(q,Q) comprises t h e mutual r e p u l s i o n o f t h e
e l e c t r o n s , t h e mutual r e p u l s i o n o f t h e n u c l e i and t h e a t t r a c t i n g p o t e n t i a l between e l e c t r o n s and n u c l e i . The c o o r d i n a t e s o f t h e e l e c t r o n s and t h e n u c l e i a r e r e p r e s e n t e d b y q and Q, r e s p e c t i v e l y . The e i g e n v a l u e s { E m } and t h e c o r r e s p o n d i n g e i g e n f u n c t i o n s { Y ~ ] a r e t h e s o l u t i o n s o f the Schrodinger equation
Since t h e s o l u t i o n i s d i f f i c u l t t o o b t a i n d i r e c t l y , i t has become customary
[5] t o w r i t e t h e f u l l m o l e c u l a r w a v e f u n c t i o n Y~ as
E. Hailer and H. Koppel
166
where the e l e c t r o n i c wavefunctions o i a r e solutions o f the molecular Hamiltonia n ( 1 ) f o r fixed nuclear geometry ( i . e . , discarding t h e nuclear k i n e t i c energy T N ) . The eigenproblem ( 2 ) i s then reduced - f o r given wavefunctions a i t o a s e t of coupled d i f f e r e n t i a l equations f o r the vibrational wavefunctions x l m ) ( r o t a t i o n a l motion will be ignored in the following). We mention t h a t t h e adiabatic approximation where t h e nuclear motion i s governed by a s i n g l e , well-defined potential energy surface amounts t o retaining a s i n g l e term in the sum of eq. ( 3 ) [3,5]. We seek f o r a simple model Hamiltonian describing the vibrational motion [4]. To t h a t end we use so-called d i a b a t i c e l e c t r o n i c wavefunctions a i [ 6 , 7 ] . These a r e characterized by the requirement t h a t they vary smoothly with the nuclear coordinates even in regions of strong nonadiabatic e f f e c t s . Due t o this smoothness we can assume a very simple expansion of the matrix elements of H :
For the f i r s t two diagonal terms we adopt the harmonic approximation
v
= -
O
1
2
1 ws s
Qs2
and the t h i r d term i s expanded u p t o f i r s t order in the nuclear displacements
E i denotes the v e r t i c a l t r a n s i t i o n energy of t h e e l e c t r o n i c s t a t e labelled " i " . The non-diagonal matrix elements of H which accomplish the coupling bet-
ween the d i a b a t i c s t a t e s a r e a l s o expanded u p t o f i r s t order in the nuclear coordinates :
Group t h e o r e t i c a l considerations show t h a t only molecular vibrations o f c e r t a i n symmetry types can couple two given e l e c t r o n i c s t a t e s .
rs
r 1. x r i
(for ks(i))
Investigation oj'Nuclear Dynamics irt Molecules
r
stands h e r e f o r t h e i r r e d u c i b l e r e p r e s e n t a t i o n o f t h e v i b r a t i o n a l modes
and o f t h e d i a b a t i c s t a t e s i n t h e symmetry group o f a p a r t i c u l a r m o l e c u l e . Due t o t h e s e symmetry r e s t r i c t i o n s s e v e r a l terms i n t h i s f o r m a l expansion v a n i s h . F o r i n s t a n c e , o n l y t o t a l l y symmetric v i b r a t i o n s w i l l appear i n t h e d i a g o n a l o f t h e m a t r i x H a m i l t o n i a n . When t h e i n t e r a c t i n g e l e c t r o n i c s t a t e s have d i f f e r e n t symmetries (and a r e nondegenerate), o n l y n o n - t o t a l l y symmetric v i b r a t i o n s w i l l appear i n t h e non-diagonal elements o f H. Two remarks on t h e a d i a b a t i c p o t e n t i a l energy s u r f a c e s o f t h i s model s h o u l d be added h e r e . By d e f i n i t o n , t h e s e a r e t h e e i g e n v a l u e s o f t h e s t a t i c terms o f t h e H a m i l t o n i a n ( V 6 . . + ( A V ) . . ) . ( A ) Since we have n o t c o n s t r u c t e d o u r diaba0 1J
1J
t i c b a s i s e x p l i c i t l y , we must d e t e r m i n e t h e parameters o f t h e model H a m i l t o n i a n a p o s t e r i o r i . T h i s i s done by a d j u s t i n g t h e p o t e n t i a l energy s u r f a c e s o f o u r model t o ab i n i t i o c a l c u l a t e d p o t e n t i a l energy s u r f a c e s o f r e a l i s t i c molecules. Subsequently, t h e parameter v a l u e s d e r i v e d a r e u s u a l l y r e a d j u s t e d w i t h i n t h e e r r o r range o f t h e ab i n i t o c a l c u l a t i o n i n o r d e r t o improve t h e agreement w i t h experiment [ 4 ] .
( B ) I n t h e case o f two e l e c t r o n i c s t a t e s , f o r example,
dege-neracies o f t h e a d i a b a t i c p o t e n t i a l energy s u r f a c e s emerge when t h e two d i a g o n a l elements become equal and t h e non-diagonal element vanishes. These two c o n d i t i o n s , i n g e n e r a l , d e f i n e a subspace o f dimension N-2 f o r N n u c l e a r degrees o f freedom and w i l l t h u s , i n g e n e r a l , be n o t f u l f i l l e d i n d i a t o m i c molecules (N = 1). Since t h e degeneracy i f l i f t e d i n f i r s t o r d e r w i t h t h e d i s t a n c e f r o m t h e subspace t h e two s u r f a c e s f o r m t h e r e a m u l t i - d i m e n s i o n a l double cone. T h e r e f o r e t h i s t o p o l o g y i s commonly termed " c o n i c a l i n t e r s e c t i o n "
[8,9].
Near c o n i c a l i n t e r s e c t i o n s t h e n o n a d i a b a t i c e f f e c t s a r e known t o be
very strong [4].
As s t a t e d above we a r e m a i n l y i n t e r e s t e d i n t h e v i b r o n i c e f f e c t s i n m o l e c u l a r s p e c t r a . Given t h e complete s e t o f e i g e n v a l u e s { E m } and e i g e n v e c t o r s { y m } o f the Hamiltonian the spectral d i s t r i b u t i o n i s calculated according t o Fermi's Golden Rule [5]: P ( E ) = 2.rr
1 m
167
E. Haller and H. Koppel
168
One gets a sequence of 6-peaks positioned a t the eigenvalues and weighted with the squared f i r s t components of t h e corrresponding eigenvectors ( s e e below). T~ i s t h e matrix element of the t r a n s i t i o n operator T between d i a b a t i c wave-functions and may be termed t r a n s i t i o n amplitude. I t wi 1 be taken as some a r b i t r a r y constant in the following. [4] In experimental spectra one, of course, never f i n d s &-shaped peaks. Among the various mechanisms f o r l i n e broadening me mention the na u r a l l i n e width due t o spontaneous emission, limi ted experimental resolution and Doppler broadening [2]. To account f o r these e f f e c t s we convolute our l i n e spectrum ( 7 ) with Lorentzians of s u i t a b l e width y : 6(E-EM)
__*
L ( E - E ~ ) = rr 1 Y
(Y/2I2
( E- Em '+ ( y / 2
So f a r we have discussed p a r t i c u l a r l y t h e physical aspects of the model used. The next s t e p t o be done now i s the diagonalization of our model Hamiltonian. I n s p i t e of the s i m p l i c i t y of the model, a n a l y t i c a l solutions of the Schrodinger equation can be found f o r very few special cases only. Likewise, q u a n t u m mechanical perturbation theory f a i l s severely i n most cases. The general vibron i c coupling problem, t h e r e f o r e , requires numerical methods f o r i t s solution.
THE NUMERICAL PROBLEM
W e seek f o r the vibrational functions x l m ) a s l i n e a r combinations of products of harmonic o s c i l l a t o r wavefunctions, one f o r each vibrational mode considered. The o s c i l l a t o r wavefunctions a r e chosen t o be eigenfunctions of a "reference" s t a t e described by the Hamiltonian Ho = TN + Vo [4]. By the variational principle [5], the solution of the Schrodinger equation ( 2 ) i s then converted t o the diagonalization of an i n f i n i t e dimensional secular matrix f o r the expansion c o e f f i c i e n t s . This secular matrix must be truncated t o a f i n i t e dimension by introducing maximal occupation numbers f o r the vibrational modes. Furthermore, there are often vibronic symmetries present which permit a rearrangement of t h e matrix t o blockdiagonal form. Figure 1 shows the typical s t r u c t u r e of such a sub-block. Here, 2 e l e c t r o n i c s t a t e s and two vibrational modes a r e considered, one mode entering only i n the diagonal elements of H , t h e o t h e r only i n the off-diagonal element. The points i n d i c a t e in which way the matrix h a d t o be augmented i f the number of basis functions f o r each mode were t o be increased.
4
’0 I
I
I
I
I I
I I' I
I I
6
h
I
I
I
7!
I
I
I
.C
0 - V ~ O l
3-
Investigation of’Nuclear Dynamics in Molecules
I I
I
0 1I , $ I * , k " o o l I
w
0
I
I .*
I
I
I
I I I I
I I l
.
...
169
Figure 1 Structure o f t h e secular matrix
E. Haller and H. Koppel
170 I n g e n e r a l , r e t a i n i n g Ni
b a s i s f u n c t i o n f o r t h e i - t h mode ( 1 -< i -< L ) , one
p Ni w i t h a band-width M 1 [4]. I n t y p i c a l a p p l i c a t i o n s
N/Ni(maX) and
g e t s a banded m a t r i x o f o r d e r N =
=
r o u g h l y 2NL non-zero elements
Ni
a l r e a d y f o r t h r e e modes t h e dimension N i s i n t h e range
lo3
p 10 so t h a t
4
- 10 w h i l e t h e
percentage o f n o n v a n i s h i n g elements i s below 1 f . These numbers make i t c l e a r t h a t t h e Lanczos a l g o r i t h m i s a n e a r l y i d e a l t o o l t o accomplish t h e d i a g o n a l i zation o f the secular m a t r i x [lo] ( a l s o note the r e g u l a r p a t t e r n o f t h e m a t r i x elements i n F i g . 1 which p e r m i t s t h e d e s i g n o f e f f i c i e n t m a t r i x - v e c t o r m u l t i p l i c a t i o n r o u t i n e s r e q u i r e d f o r t h e Lanczos i t e r a t i o n ) . N e v e r t h e l e s s , due t o t h e banded s t r u c t u r e o f t h e s e c u l a r m a t r i x , a l s o s t a n d a r d methods l i k e Jacobi r o t a t i o n s can be c a r r i e d r a t h e r f a r and p e r m i t a check o f t h e Lanczos r e s u l t s i n a few r e l e v a n t cases. We end t h i s s e c t i o n w i t h a remark on o u r c h o i c e o f t h e s t a r t i n g v e c t o r p1 i n t h e Lanczos a l g o r i t h m . Whereas i n p r i n c i p l e t h i s c h o i c e i s r a t h e r a r b i t r a r y i n o u r case i t i s f i x e d by t h e f o l l o w i n g reasoning. W i t h t h e arrangement o f t h e s e c u l a r m a t r i x as i n F i g . 1 t h e s p e c t r a l i n t e n s i t i e s i n eq. ( 7 ) a r e apart from the i r r e l a v a n t o v e r a l l constant
T~
- g i v e n by t h e
-
first components
o f t h e e i g e n v e c t o r s . By s i m p l e r e a s o n i n g one can show t h a t t h e c h o i c e
guarantees t h a t t h e f i r s t elements o f t h e e i g e n v e c t o r s o f t h e t r i d i a g o n a l m a t r i x
Tm generated by t h e Lanczos a l g o r i t h m a r e t h e same as those o f t h e o r i g i n a l m a t r i x . Thus, i t i s n o t necessary t o s t o r e a l l t h e Lanczos v e c t o r s and i t s u f f i c e s t o compute o n l y t h e f i r s t components o f t h e e i g e n v e c t o r s o f Tm.
RESULTS AND D I S C U S S I O N We now p r e s e n t and d i s c u s s some s e l e c t e d r e s u l t s o b t a i n e d w i t h t h e Lanczos a l g o r i t h m . F i g . 2 shows t h e second band i n t h e p h o t o e l e c t r o n ( P E ) spectrum o f e t h y l e n e a s found e x p e r i m e n t a l l y ( F i g . 2a) and a c c o r d i n g t o t h e t h e o r e t i c a l c a l c u l a t i o n ( F i g . 2b). I n drawing F i g . 2b t h e v i b r o n i c c o u p l i n g H a m i l t o n i a n
( 5 ) w i t h f o u r v i b r a t i o n a l modes has been employed [ l l ] . T r e a t i n g one mode i n t h e c o n v o l u t i o n a p p r o x i m a t i o n [4,11]
leads t o a secular m a t r i x o f order
3600 on which 3600 Lanczos i t e r a t i o n s t e p s have been performed.The envelope o f t h e c a l c u l a t e d l i n e spectrum i s seen t o be i n f a i r agreement w i t h experiment. The r e l i a b i l i t y o f t h e u n d e r l y i n g l i n e s t r u c t u r e i s f u r t h e r c o r r o b a r a t e d by ab i n i t i o d a t a f o r t h e parameters a p p e a r i n g i n eq. ( 5 ) which agree w i t h those used f o r F i g . 2b w i t h i n t h e i r e r r o r l i m i t s . To a p p r e c i a t e t h e p h y s i c a l s i g n i -
Investigation of’Nuclear Dynamics in Molecules
>t-
ul
H
Z W
1
I
I
I
b
171
THEORY
t
z
H
W
>
H
t
a
-I W IY
C
CONDON
1
13
12
I O N I Z A T I O N ENERGY Figure 2 Second photoelectron band o f ethylene
eV
172
E. Haller and H. Koppel
f i c a n c e o f t h e c a l c u l a t e d spectrum we show i n F i g . 2c t h e r e s u l t o f a n o t h e r c a l c u l a t i o n w i t h t h e same parameters b u t where t h e n u c l e a r m o t i o n i s a r t i f i c i a l l y c o n f i n e d t o t h e upper p o t e n t i a l energy s u r f a c e , i . e . t h e n o n a d i a b a t i c e f f e c t s a r e suppressed i n t h e c a l c u l a t i o n [ll]. T h i s spectrum e x h i b i t s r e g u l a r v i b r a t i o n a l p r o g r e s s i o n s as i s f a m i l i a r from s p e c t r a o f d i a t o m i c molecules b u t i s seen t o bear l i t t l e resemblance n e i t h e r t o t h e f u l l c a l c u l a t i o n n o r t o experiment. Even t h e v e r y number o f s p e c t r a l l i n e s comes o u t much t o o low, by r o u g h l y two o r d e r s o f magnitude. These
strong
nonadiabatic e f f e c t s , o f
which t h e comparison between F i g s . 2b and c g i v e s evidence, can be t r a c e d t o t h e occurrence o f a c o n i c a l i n t e r s e c t i o n between a d i a b a t i c p o t e n t i a l energy s u r f a c e s . They a r e a genuine multi-mode e f f e c t and v a n i s h i n a single-mode d e s c r i p t i o n o f t h e problem [ll]. I n t h e p r e s e n t example i t p r o v e s p o s s i b l e , though w i t h c o n s i d e r a b l e e f f o r t , t o p e r f o r m a f u l l m a t r i x d i a g o n a l i s a t i o n w i t h t h e method o f J a c o b i r o t a t i o n s and t h u s check t h e Lanczos r e s u l t . The outcome o f t h i s check i s v e r y g r a t i f y i n g , indeed. The s p e c t r a l envelope and p r a c t i c a l l y t h e whole l i n e s t r u c t u r e ( e x c e p t f o r some t i n y l i n e s i n t h e h i g h energy p a r t o f t h e spectrum) c o i n c i d e t o w i t h i n drawing accuracy i n b o t h c a l c u l a t i o n s . A more q u a n t i t a t i v e comparison i s g i v e n i n t h e appendix. While f o r t h e envelope t h e good convergence o f t h e Lanczos scheme c o u l d be expected because o f t h e c l o s e r e l a t i o n between t h e Lanczos procedure and t h e method o f moments, t h e e q u a l l y good convergence on t h e l i n e s t r u c t u r e seems e s p e c i a l l y noteworthy. On t h e o t h e r hand, t h e computing e f f o r t i n c r e a s e s s t r o n g l y f o r t h e method o f Jacobi r o t a t i o n s : b o t h t h e r e q u i r e d s t o r a g e space and t h e CPU t i m e go up by almost two o r d e r s o f magnitude. These a r e t y p i c a l numbers f o r multi-mode v i b r o n i c problems and i l l u s t r a t e t h e power o f t h e Lanczos method f o r o u r purposes. As a s i m i l a r example we p r e s e n t i n F i g . 3 e x p e r i m e n t a l and t h e o r e t i c a l r e s u l t s on t h e v i s i b l e a b s o r p t i o n spectrum o f NO2 [12].
I n a f i r s t approximation t h i s
spectrum can be taken as t h e sum o f two d i f f e r e n t e l e c t r o n i c t r a n s i t i o n s , each o f which l e a d s t o a t w o - s t a t e three-mode v i b r o n i c c o u p l i n g problem. To a v o i d t r u n c a t i o n e r r o r s we have, however, t o i n c l u d e more b a s i s f u n c t i o n s t h a n i n e t h y l e n e and a r r i v e a t m a t r i x dimensions o f 18630 ( F i g . 2 b ) and 24000 ( F i g . 2 c ) T h i s makes t h e Lanczos a l g o r i t h m an i n d i s p e n s a b l e t o o l f o r d i a g o n a l i z a t i o n . P e r f o r m i n g 10000 v i z . 3000 i t e r a t i o n s t e p s one o b t a i n s t h e s p e c t r a d i s p l a y e d i n F i g . 2b and 2c, r e s p e c t i v e l y [12].
The e x p e r i m e n t a l r e c o r d i n g o f F i g . 2a
i s t h e weighted sum o f t h e two p a r t i a l s p e c t r a c o r r e s p o n d i n g t o F i g s . 2b and c w i t h some unknown w e i g h t c o e f f i c i e n t s . One sees t h a t t h e c o m p l e x i t y o f t h e v i s i b l e a b s o r p t i o n spectrum o f N02, f o r which i t has become famous among s c i e n t i s t s [13],
i s caused by t h e t r a n s i t i o n d i s p l a y e d i n F i g . 2b, t h e s o - c a l l e d ZB2-2A1
Investigatiori of Nuclear Dynamics in Molecules
b
1
'
"
'
'
~
"
~
"
'
~
'
~
~
'
"
1
"
'
'
c
I
Figure 3
The v i s i b l e a b s o r p t i o n spectrum o f NO2
'
'
173
114
E. Haller and H. Kappel
t r a n s i t i o n , w h i l e t h e s o - c a l l e d 2 B 1 - Z A 1 t r a n s i t i o n o f F i g . 2c has a r a t h e r regular v i b r o n i c structure. This d i f f e r e n c e i s p r e c i s e l y i n l i n e w i t h the e n e r g e t i c p o s i t i o n of a c o n i c a l i n t e r s e c t i o n which i s l o w - l y i n g i n t h e B2 v i b r o n i c m a n i f o l d b u t h i g h - l y i n g i n t h e B1 v i b r o n i c m a n i f o l d and l e a d s t o strong non-adiabatic effects
( o n l y ) i n t h e B2 v i b r o n i c species, c o r r e s p o n d i n g
t o F i g . 2b [12]. Another i n s t r u c t i v e example i s p r o v i d e d by t h e f i r s t PE band o f BF3 [14]. The 1A;
ground s t a t e of BF;
i n t e r a c t s w i t h t h e second e x c i t e d 3E’ e l e c t r o n i c
s t a t e through two degenerate v i b r a t i o n a l modes w h i l e t o t a l l y symmetric modes p r o v e t o be n e g l i g i b l e f o r t h e c o u p l i n g mechanism. T h i s i n t e r a c t i o n l e a d s t o a s t r o n g v i b r a t i o n a l e x c i t a t i o n i n t h e f i r s t PE band as shown i n t h e l e f t panel o f F i g . 4. For comparison, we d i s p l a y i n t h e r i g h t p a r t o f t h e f i g u r e t h e s p e c t r a where o n l y one o f t h e two v i b r a t i o n a l modes ("3 o r "4)
i s retained.
A p p a r e n t l y , t h e f u l l spectrum i s v e r y f a r from b e i n g t h e c o n v o l u t i o n o f t h e two single-mode s p e c t r a which shows t h a t t h e s t r o n g v i b r a t i o n a l e x c i t a t i o n c h a r a c t e r i s t i c o f t h e two-mode c a l c u l a t i o n i s a mode-mixing e f f e c t . I t can be t r a c e d t o t h e shape o f t h e a d i a b a t i c p o t e n t i a l energy surfaces [14]. I n drawing F i g . 4 ( l e f t p a n e l ) 500 Lanczos i t e r a t i o n s t e p s have been performed on a s e c u l a r m a t r i x o f o r d e r 40800. The convergence has been a s c e r t a i n e d by v e r i f y i n g t h a t t h e spectrum i s v i r t u a l l y t h e same as t h a t o b t a i n e d a f t e r 400 Lanczos i t e r a t i o n steps ( e x c e p t again f o r a few m i n o r l i n e s i n t h e h i g h energy p a r t o f t h e spectrum). T h i s i n d i c a t e s a n o t h e r a s p e c t i n which t h e Lanczos procedure i s i d e a l l y s u i t e d f o r o u r purposes: t h e s i m p l e r t h e v i b r o n i c l i n e s t r u c t u r e t h e s m a l l e r i s t h e number o f Lanczos s t e p s r e q u i r e d t o converge on i t To g a i n f u r t h e r i n s i g h t i n t o t h e s t r e n g t h o f t h e n o n a d i a b a t i c e f f e c t s we a l s o c a l c u l a t e t h e quenching o r w e i g h t c o e f f i c i e n t s qm. These a r e d e f i n e d as t h e percentage c h a r a c t e r of t h e upper e l e c t r o n i c s t a t e , termed No. 2, i n t h e v i b r o n i c e i g e n s t a t e s (we c o n f i n e o u r s e l v e s here t o two i n t e r a c t i n g e l e c t r o n i c s t a t e s ) [4].
Using t h e n o t a t i o n o f eq. ( 3 ) we can w r i t e
A p a r t f r o m t h e i r t h e o r e t i c a l i n t e r e s t , t h e quenching c o e f f i c i e n t s a r e a l s o r e l a t e d t o t h e r a d i a t i v e decay o f t h e v i b r o n i c s t a t e s , a t l e a s t i n a w e l l d e f i n e d l i m i t i n g case [4].
Investigation of Nuclear D.ynamics in Molecules
175
exact calculation
n v)
c, .C
C
1
n L
a
U
>
c, .C
v)
C 0
c, C
n
I 16.0
1
L 16.0
Energy - C ~ V I
Figure 4 F i r s t p h o t o e l e c t r o n band o f BF3
E. Haller and H. Koppel
176
To o b t a i n t h e qm w i t h o u t c a l c u l a t i n g t h e f u l l e i g e n v e c t o r s we use t h e f o l l o w i n g simple t r i c k . We augment t h e v i b r o n i c H a m i l t o n i a n H by t h e s o - c a l l e d r a d i a t i v e damping m a t r i x
I f t h e ( r e a l ) q u a n t i t y y i s much s m a l l e r t h a n n e a r e s t - n e i g h b o r spacings o f t h e o r i g i n a l H a m i l t o n i a n H t h e r e a l p a r t s o f t h e e i g e n v a l u e s w i l l n o t be a f f e c t e d by t h e p e r t u r b a t i o n
r
and t h e i m a g i n a r y p a r t s
y,
w i l l from p e r t u r b a t i o n theore-
t i c arguments be g i v e n b y
The qm can t h u s be o b t a i n e d f r o m t h e e i g e n v a l u e s o f t h e complex symmetric Hamiltonian H +
r
f o r s u f f i c i e n t l y small values o f t h e q u a n t i t y y .
We have g e n e r a l i z e d t h e Lanczos a l g o r i t h m f o r complex symmetric m a t r i c e s
[lo]
as d e s c r i b e d a l s o i n o t h e r c o n t r i b u t i o n s t o t h i s volume and have a p p l i e d i t t o t h e examples o f e t h y l e n e and NO2 d i s c u s s e d above. W i t h s i m i l a r numbers o f m a t r i x dimensions and Lanczos i t e r a t i o n s t e p s as b e f o r e we have found 147 and 175 converged quenching c o e f f i c i e n t s , r e s p e c t i v e l y [4,12].
Figure 5 displays
t h e r e s u l t s as histograms, i . e . p l o t t h e number o f v i b r o n i c s t a t e s w i t h quenc h i n g c o n s t a n t s i n a g i v e n i n t e r v a l . I n t h e absence o f v i b r o n i c c o u p l i n g t h e s e numbers would be e i t h e r z e r o o r one c o r r e s p o n d i n g t o v i b r a t i o n a l l e v e l s o f t h e lower o r upper e l e c t r o n i c s t a t e s . We see f r o m F i g . 5 t h a t t h e v i b r o n i c c o u p l i n g leads t o a s t r o n g m i x t u r e o f t h e e l e c t r o n i c species i n t h e v i b r o n i c s t a t e s . E s p e c i a l l y i n t h e e t h y l e n e c a t i o n t h e d i s t r i b u t i o n i s v e r y narrow and t h e m i x i n g almost complete. I n t h e example o f N02, on t h e o t h e r hand, t h e d i s t r i b u t i o n r a t h e r shows a bimodal b e h a v i o r and a p r e f e r e n t i a l e l e c t r o n i c c h a r a c t e r can s t i l l be i d e n t i f i e d . T h i s shows more c l e a r l y t h a n t h e s p e c t r a t h e d i f f e r e n t s t r e n g t h o f t h e n o n a d i a b a t i c e f f e c t s i n t h e two examples. Concerning t h e r a d i a t i v e decay, t h e d e p a r t u r e o f t h e q, anomalously l o n g r a d i a t i v e decay t i m e s [4,12]
from u n i t y leads t o
which have been w e l l documented
e x p e r i m e n t a l l y f o r NO2 [13].
CONCLUSIONS I n t h i s c o n t r i b u t i o n , we have presented some i l l u s t r a t i v e examples o f v i b r o n i c c o u p l i n g systems i n small p o l y a t o m i c molecules. A s i m p l e model H a m i l t o n i a n
Investigation of Nuclear Dynamics in Molecules
177
30
10
0.1 I
I
0.3 1
- 9
L
Figure 5 Quenching f a c t o r histograms f o r CzH4' and NO2
0.5
1
I
I
r
1
rl
178
E. Huller and H . Koppel
has been used i n t h e c a l c u l a t i o n s w h i c h n e v e r t h e l e s s r e p r o d u c e s t h e g r o s s f e a t u r e s o f complex e x p e r i m e n t a l s p e c t r a c o r r e c t l y . The examples g i v e e v i d e n c e t h a t t h e u s u a l a d i a b a t i c s e p a r a t i o n o f e l e c t r o n i c and n u c l e a r m o t i o n s , whereupon t h e n u c l e i move i n w e l l d e f i n e d p o t e n t i a l e n e r g y s u r f a c e s , may f a i l c o m p l e t e l y : e s p e c i a l l y when d i f f e r e n t e l e c t r o n i c s u r f a c e s i n t e r s e c t each o t h e r , t h e n u c l e i may i n t e r c o n v e r t f r e e l y between t h e s e s u r f a c e s . They a l s o show t h a t t h e dynamic p r o b l e m i s i n t r i n s i c a l l y a m u l t i - m o d e p r o b l e m . The e n s u e i n g enormous d i m e n s i o n o f t h e s e c u l a r m a t r i x , i t s h i g h d e g r e e o f s p a r s i t i y and t h e r e g u l a r p a t t e r n o f n o n z e r o e l e m e n t s make t h e Lanczos a l g o r i t h m a h i g h l y e f f e c t i v e means o f s o l v i n g t h e quantum m e c h a n i c a l e i g e n v a l u e p r o b l e m . T h i s i s f u r t h e r c o r r o b o r a t e d b y t h e o b s e r v a t i o n t h a t t h e Lanczos i t e r a t i o n p r o d u c e s r e l e v a n t i n f o r m a t i o n on t h e s p e c t r a l i n t e n s i t y d i s t r i b u t i o n f a r b e f o r e a numerical convergence i n t h e p r o p e r sense i s a c h i e v e d . I n f u t u r e a p p l i c a t i o n s one w o u l d l i k e t o t r e a t matrices o f s t i l l l a r g e r dimension
(4
6 10 , say) and, i n a s l i g h t l y d i f f e r e n t
c o n t e x t , a l s o o b t a i n s e l e c t e d e i g e n v e c t o r s i n a n a r b i t r a r y r a n g e of t h e e i g e n v a l u e spectrum. The r o l e o f t h e Lanczos o r r e l a t e d a l g o r i t h m s s h o u l d become t h e more i m p o r t a n t and e f f i c i e n t i m p l e m e n t a t i o n s f o r t h e above p u r p o s e s a r e o f great interest.
ACKNOWLEDGEMENT The a u t h o r s w i s h t o e x p r e s s t h e i r g r a t i t u d e t o L.S. Cederbaum and W . Domcke f o r a f r u i t f u l c o l l a b o r a t i o n on t h e p r o b l e m s d i s c u s s e d i n t h i s a r t i c l e .
APPENDIX We w i s h t o compare two l i n e sequences c h a r a c t e r i z e d b y e n e r g i e s Ei
and i n t e n s i -
t i e s Ii, one sequence ( w i t h o u t s u p e r s c r i p t ) b e i n g e x a c t , t h e o t h e r ( s u p e r s c r i p t L ) b e i n g g e n e r a t e d w i t h t h e a i d o f t h e Lanczos i t e r a t i o n scheme. To have a m e a n i n g f u l c o r r e l a t i o n between t h e i n d i v i d u a l l i n e s we c o n s i d e r o n l y t h e s e p a i r s o f l i n e s where t h e e n e r g i e s d i f f e r b y l e s s t h a n a t h i r d o f t h e n e a r e s t n e i g h b o r d i s t a n c e o f t h e e x a c t sequence and t h e i n t e n s i t i e s d i f f e r b y l e s s t h a n a f a c t o r o f two:
The c o r r e s p o n d i n g l y r e s t r i s t e d sum o f e x a c t i n t e n s i t i e s
Investigation of Nuclear Dynamics in Molecules
i s c a l l e d t h e c o n f i d e n c e i n d e x s i n c e i t measures t h a t p a r t o f t h e s p e c t r a l i n t e n s i t y w h i c h i s c o v e r e d b y t h e c o m p a r i s o n ( b o t h sequences o f i n t e n s i t i e s a r e t a k e n t o be n o r m a l i z e d t o u n i t y ) . B e i n g more i n t e r e s t e d i n s t r o n g t h a n i n weak l i n e s we i n t r o d u c e t h e w e i g h t e d e r r o r
T h i s q u a n t i t y i s n o t y e t m e a n i n g f u l b y i t s e l f b u t s h o u l d be r e l a t e d t o a t y p i c a l l i n e s p a c i n g such a s t h e w e i g h t e d a v e r a g e o f s p a c i n g s
( h e r e t h e sum i s n o t r e s t r i c t e d and t h e p r i m e o n t h e summation s i g n h a s t h e r e f o r e been o m i t t e d ) . As t h e c r i t e r i o n f o r t h e q u a l i t y o f t h e Lanczos s p e c t r u m we use t h e w e i g h t e d r e l a t i v e e r r o r
(17)
E = F / D
The f o l l o w i n g T a b l e I l i s t s t h e c o n f i d e n c e i n d e x C and t h e w e i g h t e d r e l a t i v e e r r o r E f o r t h e s e c u l a r problem o f e t h y l e n e ( F i g . 2b) w i t h a dimension o f 3600 and t h r e e d i f f e r e n t numbers o f Lanczos i t e r a t i o n s t e p s . One sees t h a t even a f t e r 1800 i t e r a t i o n s t h e e r r o r E i s q u i t e s m a l l , b u t t h i s i s n o t v e r y s i g n i f i c a n t because o n l y 58% o f t h e s p e c t r a l i n t e n s i t y have a c t u a l l y been compared. A f t e r 5400 i t e r a t i o n s , on t h e o t h e r hand, t h e Lanczos s p e c t r u m can be c o n s i d e r e d a s c o n v e r g e d f o r o u r p u r p o s e s . I t s h o u l d be s t r e s s e d t h a t t h e s e e s t i m a t e s a r e r a t h e r c o n s e r v a t i v e and t h a t f r o m t h e v i s u a l i m p r e s s i o n even t h e s p e c t r u m a f t e r 1800 i t e r a t i o n s l o o k s q u i t e s a t i s f a c t o r y w h i l e t h a t f o r 3600 i t e r a t i o n s l o o k s a l m o s t i n d i s t i n g u i s h a b l e f r o m t h e e x a c t one.
C o n f i d e n c e i n d e x C and w e i g h t e d r e l a t i v e e r r o r E f o r e t h y l e n e ( d i m e n s i o n 3600) Iterations
C
E
3600
5400
0.58
0.92
0.99
0.020
0.0045
0.00083
1800
179
E. Haller and H. Koppel
180
REFERENCES 1) See, f o r example, 1 . 6 . Bersuker, The J a h n - T e l l e r E f f e c t and V i b r o n i c I n t e r a c t i o n s i n Modern Chemistry (Plenum Press, New York, 1984)
2 ) G. Herzberg, Spectra o f D i a t o m i c Molecules (Van Nostrand, New York, 1950) 3 ) G. Herzberg, E l e c t r o n i c Spectra and E l e c t r o n i c S t r u c t u r e o f Polyatomic Molecules (Van Nostrand, New York, 1966)
4 ) H. Koppel, W . Domcke and L.S. Cederbaum, Multimode m o l e c u l a r dynamics beyond t h e Born-Oppenheimer a p p r o x i m a t i o n , Adv. Chem. Phys. 57 ( 1 9 8 4 ) 59
5 ) A.S. Davydov, Quantum Mechanics (Pergamon Rress, New York, 1965) 6 ) H.C.
Longuet-Higgins, Some r e c e n t developments i n t h e t h e o r y o f m o l e c u l a r
energy l e v e l s , Advan. Spectrosc. 2 (1961) 429 7 ) W . L i c h t e n , Resonant charge exchange i n a t o m i c c o l l i s i o n s , Phys. Rev. 131
(1963) 229 8 ) G . Herzberg and H.C.
Longuet-Higgins,
I n t e r s e c t i o n o f p o t e n t i a l energy
s u r f a c e s i n p o l y a t o m i c molecules, Discuss. Faraday SOC. 35 (1963) 77
9 ) T. C a r r i n g t o n , The geometry o f i n t e r s e c t i n g p o t e n t i a l s u r f a c e s , A c c t s . Chem. Res. 7 (1974) 20
10) E . H a l l e r , Mehrmodendynami k b e i konischen Durchschneidungen von P o t e n t i a l f l a c h e n ( T h e s i s , U n i v e r s i t y o f Heidelberg,
1984)
11) H. Koppel, L.S. Cederbaum and W . Domcke, S t r o n g n o n a d i a b a t i c e f f e c t s and c o n i c a l i n t e r s e c t i o n s i n m o l e c u l a r spectroscopy and u n i m o l e c u l a r decay:
C2H4+, J. Chem. Phys. 77 ( 1 9 8 2 ) 2014 1 2 ) E. H a l l e r , E . Koppel and L.S. Cederbaum, The v i s i b l e a b s o r p t i o n spectrum o f N02: a three-mode n u c l e a r dynamics i n v e s t i g a t i o n , J. Mol. Spectrosc.
111 (1985) 377 1 3 ) D.K. Hsu, D.L. Monts and R.N. Zare, S p e c t r a l A t l a s o f N i t r o g e n D i o x i d e (Academic Press, New York, 1978)
1 4 ) E. H a l l e r , H. Koppel, L.S. Cederbaum, W . von Niessen and G. B i e r i , M u l t i mode J a h n - T e l l e r and pseudo J a h n - T e l l e r e f f e c t s i n BF3+, J. Chem. Phys. 78 (1983) 1359
Large Scale Eigenvalue Problems J . Cullum and R.A. Willoughby (Editors) 0 Elsevier Science Publishers B.V. (North-Holland), 1986
181
EXAMPLES OF EIGENVALUE/VECTOR USE I N ELECTRIC POWER SYSTEM PROBLEMS
James E. Van Ness
E l e c t r i c a l E n g i n e e r i n g and Computer S c i e n c e Department Northwestern University Evans t o n , I111i n o i s
U. S. A.
Major
failures
result
of
of
electric
dynamic
disturbances
can
power
systems usually
in
instabilities be
studied
by
the
system.
linearizing
e q u a t i o n s and u s i n g e i g e n v a l u e s / v e c t o r s .
are t h e
the
Small system
If t h e power system
i s r e p r e s e n t e d i n c o m p l e t e d e t a i l , t h e o r d e r of t h e r e s u l t i n g system w i l l be i n t h e t e n s of t h o u s a n d s , handled. solving
which
c a n n o t be
P r e s e n t methods h a v e p r o v e n useful and e f f e c t i v e i n
real
problems,
but
they
have
just
increased
the
d e s i r e t o s t u d y s y s t e m s of h i g h e r o r d e r .
INTRODUCTION A modern power system p r e s e n t s a c h a l l e n g e t o t h e c o n t r o l e n g i n e e r t h a t a r i s e s p r i m a r i l y from i t s huge s i z e .
A c o m p l e t e r e p r e s e n t a t i o n o f t h e system would l e a d
t o d i f f e r e n t i a l e q u a t i o n s whose o r d e r would be measured i n t h e t e n s of t h o u s a n d s . Even w i t h
many
equivalents,
simplifying
assumptions and
t h e dynamic system
combining
t h a t results w i l l
d i f f e r e n t i a l e q u a t i o n s o f o r d e r s e v e r a l hundred. currently
be
of
separate
represented
units by
into
a s e t of
The b a s i c t e c h n i q u e s t h a t are
b e i n g u s e d t o s t u d y t h e s e s y s t e m s h a v e been known for y e a r s b u t o n l y
a p p l i e d t o r e l a t i v e l y small s y s t e m s .
T h i s p a p e r d e s c r i b e s methods of
studying
l a r g e r s y s t e m s and g i v e s examples of s t u d i e s t h a t have been made. A large d i g i t a l computer program c a l l e d PALS [ l ] h a s b e e n d e v e l o p e d t o a p p l y t h e methods t h a t w i l l be d e s c r i b e d h e r e .
The program d o e s l o s e some e f f i c i e n c y once
t h e s i z e of t h e system gets o v e r 300-t.h o r d e r .
T h i s s t i l l i s several o r d e r s of
magnitude l a r g e r t h a n t h e t y p i c a l system s t u d i e d i n a c o u r s e i n f e e d b a c k c o n t r o l .
J. E. Van Ness
182 DESCRIPTION OF THE SYSTEM A modern power
system c o n s i s t s of
many g e n e r a t i n g s t a t i o n s and l o a d c e n t e r s
connected t o g e t h e r by a n e l e c t r i c a l t r a n s m i s s i o n system.
I n many s t u d i e s of t h e
dynamics of t h i s t y p e of system, t h e time c o n s t a n t s of t h e g e n e r a t i n g s t a t i o n s are found t o be t h e c o n t r o l l i n g o n e s i n t h e o v e r a l l system r e s p o n s e .
The t r a n s i e n t s
i n v o l v e d a r e so slow t h a t t h e t r a n s m i s s i o n network c a n be c o n s i d e r e d t o be i n steady-state
operation,
and
the
transmission l i n e
t r a n s i e n t s c a n be ignored.
Likewise , t h e l o a d s a r e normally r e p r e s e n t e d by t h e i r s t e a d y - s t a t e istics. more
character-
T h i s i n part. may be due t o l a c k of s u f f i c i e n t knowledge t o r e p r e s e n t them
adequately.
Thus,
the
first
thing
to
be
is
considered
the
dynamic
c h a r a c t e r i s t i c s of t h e g e n e r a t i n g s t a t i o n . 1
1
B T E A H BOWL
Figure 1.
A Typical Generating S t a t i o n
A t y p i c a l r e p r e s e n t a t i o n of a g e n e r a t i n g s t a t i o n i s shown i n Fig.
t h e r e are many
variations
i n g e n e r a t i n g s t a t i o n s depending upon t h e type of
equipment used and when they were b u i l t . involved,
T h i s f i g u r e does show t h e major elements
The synchronous machine i s r e p r e s e n t e d by t h e b l o c k i n t h e upper r i g h t
hand c o r n e r l a b e l e d "machine e q u a t i o n s " . these
machine
I n t h e example t o be shown l a t e r i n t h i s
equations w i l l
consist
e q u a t i o n and two a l g e b r a i c e q u a t i o n s .
Again,
paper,
Of course,
1.
of
one f i r s t - o r d e r
differential
t h i s c o u l d change i f a d i f f e r e n t
r e p r e s e n t a t i o n of t h e synchronous machine were t o be used.
The v o l t a g e r e g u l a t o r
and e x c i t e r f o r t h i s synchronous machine are shown i n t h e upper l e f t - h a n d c o r n e r of Fig.
1.
The i n p u t t o t h i s r e g u l a t o r
i s t h e t e r m i n a l v o l t a g e ET, and t h e
o u t p u t i s t h e v o l t a g e a p p l i e d t o t h e f i e l d of t h e machine EF.
The a c t u a l v o l t a g e
r e g u l a t o r and e x c i t e r undoubtedly i n c l u d e limiters and s a t u r a t i o n e f f e c t s i n t h e e x c i t e r which would make t h e o v e r a l l system n o n - l i n e a r .
For t h e t y p e s of s t u d i e s
t h a t are d e s c r i b e d i n t h i s paper, t h e v o l t a g e r e g u l a t o r and e x c i t e r system are
l i n e a r i z e d around an o p e r a t i n g p o i n t and r e p r e s e n t e d as shown i n Fig. 1.
Eigenvaluel Vector Use iri Elwrric Power System Probleins
183
T h e mechanical c h a r a c t e r i s t i c s of t h e g e n e r a t i n g system are shown i n t h e lower row
of b l o c k s .
The two b l o c k s o n t h e r i g h t - h a n d s i d e r e p r e s e n t t h e r o t a t i n g i n e r t i a
and damping of t h e synchronous machine.
The v a r i a b l e
represents the v a r i a t i o n
of t h e a c t u a l frequency or speed of t h e machine from a r e f e r e n c e f r e q u e n c y , and t h e angle
6
r e p r e s e n t s t h e a n g l e of t h e machine w i t h r e s p e c t t o t h e o t h e r machines
on t h e system. and t u r b i n e
The t h r e e b l o c k s on t h e l e f t a r e a r e p r e s e n t a t i o n of t h e governor
c h a r a c t e r i s t i c s f o r one
type of
steam t u r b i n e
v a r i a b l e YD a c t u a l l y r e p r e s e n t s t h e power developed by
the
prime mover. turbine.
The In the
diagram t h i s h a s t h e e l e c t r i c a l power t h a t i s f e d t o t h e network s u b t r a c t e d from i t so t h a t t h e d i f f e r e n c e i s t h e a c c e l e r a t i n g power a v a i l a b l e f o r t h e t u r b i n e -
g e n e r a t o r combination.
Again, t h e governor l o o p could i n v o l v e n o n - l i n e a r
terms
t h a t would be l i n e a r i z e d around a n o p e r a t i n g p o i n t f o r t h e r e p r e s e n t a t i o n g i v e n here. The i n t e r a c t i o n o f t h i s g e n e r a t i n g u n i t w i t h t h e e l e c t r i c a l t r a n s m i s s i o n network i s r e p r e s e n t e d by t h e f o u r v a r i a b l e s shown i n t h e upper r i g h t - h a n d c o r n e r .
The
magnitude of t h e terminal v o l t a g e E T and i t s a n g l e 8 T are a result of t h e a c t i o n s of t h e v o l t a g e r e g u l a t o r and governor systems.
Depending upon t h e c o n d i t i o n of
t h e o t h e r g e n e r a t i n g u n i t s and t h e t r a n s m i s s i o n system, a c e r t a i n real and reac-
t i v e power w i l l flow between t h e g e n e r a t i n g system and t h e e l e c t r i c a l t r a n s m i s s i o n network.
The v a l u e of t h e real power P and t h e r e a c t i v e power Q are shown as
i n p u t s i n t o t h e g e n e r a t i n g u n i t r e p r e s e n t a t i o n as they do a f f e c t t h e r e s p o n s e of t h i s unit.
The a c t u a l r e l a t i o n s h i p between t h e s e v a r i a b l e s f o r machine k of N
machines o n t h e network is
where t h e Y ' s are t h e magnitudes of t h e e l e m e n t s of t h e a d m i t t a n c e m a t r i x of t h e
t r a n s m i s s i o n network, variables
t h e &Is are t h e a n g l e s of t h e a d m i t t a n c e s , and t h e o t h e r
are a s d e s c r i b e d above.
These n o n - l i n e a r
e q u a t i o n s are l i n e a r i z e d
around a n o p e r a t i n g p o i n t f o r t h e t r a n s m i s s i o n system and w i l l be r e p r e s e n t e d i n m a t r i x form as a s e t of l i n e a r a l g e b r a i c e q u a t i o n s . THE STABILITY PROBLEM
Modern power systems, such as t h o s e i n t h e U n i t e d S t a t e s , are i n t e r c o n n e c t e d o v e r l a r g e a r e a s so t h a t t h e r e may be o v e r 500 g e n e r a t i n g u n i t s , similar t o t h o s e d e s c r i b e d i n t h e p r e v i o u s s e c t i o n , i n t e r - a c t i n g w i t h each o t h e r . s t a b i l i t y problems are u s u a l l y d e f i n e d .
Two c l a s s e s of
One is c l a s s i c a l l y c a l l e d t h e t r a n s i e n t
J. E. Van Ness s t a b i l i t y problem and i n v o l v e s large d i s t u r b a n c e s of t h e system.
are brought disturbance.
about by
a f a u l t on a
transmission l i n e ,
or
These u s u a l l y
some similar major
I n t h e s e problems t h e n o w l i n e a r c h a r a c t e r i s t i c s of t h e t r a n s m i s s i o n
system, as g i v e n i n E q u a t i o n 1 , are e x t r e m e l y i m p o r t a n t .
These problems u s u a l l y
are s o l v e d by a s t e p b y - s t e p i n t e g r a t i o n of t h e d i f f e r e n t i a l e q u a t i o n s d e s c r i b i n g t h e system.
These s t u d i e s are e x p e n s i v e and time consuming, but e s s e n t i a l t o t h e
o p e r a t i o n of l a r g e power systems.
I n the United States, t h e n o r t h e a s t blackouts
i n November of 1965 and J u l y of 1977 were examples of t h e t y p e of s i t u a t i o n where t h e r e was a l a r g e d i s t u r b a n c e and i t was n e c e s s a r y t o t a k e t h e n o n - l i n e a r i t y
of
t h e system i n t o account. I n t h i s paper a d i f f e r e n t t y p e of s t a b i l i t y problem is c o n s i d e r e d of t h e system t o s m a l l d i s t u r b a n c e s .
-
t h e response
I f t h e d i s t u r b a n c e s are c o n s i d e r e d s m a l l ,
t h e n t h e system can be r e p r e s e n t e d by a s e t of l i n e a r i z e d e q u a t i o n s and l i n e a r system t h e o r y c a n be a p p l i e d .
One of t h e major a p p l i c a t i o n s of t h i s method i s t h e
s t u d y of t h e spontaneous o s c i l l a t i o n s t h a t occur i n some power systems [21. they o c c u r ,
they may b u i l d up t o l a r g e magnitudes.
s t u d y i n g t h e l i n e a r i z e d set of e q u a t i o n s .
When
They c a n be p r e d i c t e d by
The t e c h n i q u e t o be u s e d c o n s i s t s of
examining t h e e i g e n v a l u e s of the c h a r a c t e r i s t i c e q u a t i o n s and d e t e r m i n i n g by t h e i r p o s i t i o n on t h e complex p l a n e t h e c o n d i t i o n of t h e system. The b a s i c t e c h n i q u e used i s one of t h e f i r s t methods p r e s e n t e d i n a b e g i n n i n g course i n feedback control theory.
The problems e n c o u n t e r e d i n t r y i n g t o apply
t h i s method arose from t h e v e r y l a r g e s i z e of t h e system, and t h u s t h e o r d e r of t h e e q u a t i o n s t h a t needed t o be s o l v e d .
It should be noted t h a t i n t h e s y s t e m s
s t u d i e d so f a r t h e c r i t i c a l modes of o s c i l l a t i o n s f o r t h e o v e r a l l system were a f u n c t i o n of t h e i n t e r a c t i o n o f v a r i o u s g e n e r a t i n g s t a t i o n s , and n o t due t o one s t a t i o n by i t s e l f .
Thus,
t h e n e c e s s a r y i n f o r m a t i o n could n o t be o b t a i n e d by
c o n s i d e r i n g a s i n g l e machine o p e r a t i n g i n t o a f i x e d l o a d . I n a d d i t i o n t o f i n d i n g t h e e i g e n v a l u e s f o r l a r g e s y s t e m s such as t h e one d e s c r i b e d h e r e , t h e methods t h a t have been developed c a n be used t o f i n d t h e r o o t l o c i or e i g e n v a l u e l o c i of t h e system and t o f i n d t h e s e n s i t i v i t i e s of t h e e i g e n v a l u e s t o t h e v a r i o u s parameters i n t h e system.
These t o o l s have been most u s e f u l i n d e t e r
mining changes t h a t can be made t o improve t h e s y s t e m s r e s p o n s e .
The e i g e n v e c t o r s
are needed t o f i n d t h e s e n s i t i v i t i e s and are t h u s a v a i l a b l e f o r many o t h e r t y p e s of s t u d i e s on t h e system.
Some of t h e s e w i l l be d e s c r i b e d i n t h i s paper.
Eigenvaluel Vector Use in Electric Power System Problems
185
APPLICATION TO ?HE STABILITY PROBLEM c o n t a i n a high p r o p o r t i o n of
S e v e r a l l a r g e power s y s te m s which generating ope r at i o n .
capacity
have
experienced
spontaneous
oscillations
hydroelectric during
normal
D i f f i c u l t i e s of t h i s type t h a t have been ex p er i en ced i n t h e w est er n
United S t a t e s have been v e r y well documented i n v a r i o u s p u b l i sh ed papers. paper p u b l i s h ed i n 1963 [ 2 ] , A.
R.
Benson and D.
G.
In a
Wohlgemuth d e s c r i b e d i n great
d e t a i l t h e o s c i l l a t i o n s t h a t had o c c u r r e d on t h e Northwest Power Pool d u r i n g t h e p e r i o d from 1955 t o t h e e a r l y 1960's.
I n t h e d i s c u s s i o n s p u b l i sh ed w i t h t h a t
paper, o t h e r s y s t e m s r e p o r t e d s i m i l a r d i f f i c u l t i e s .
L a t e r i n t h e 1 9 6 0 1 s, when t h e
P a c i f i c northwest was i n t e r c o n n e c t e d w i t h t h e rest of t h e w e s t e r n United S t a t e s t o form one system, s i m i l a r o s c i l l a t i o n s were observed throughout t h e t o t a l w est er n United S t a t e s . hydr o - g en er at i n g
Some of t h e work t h a t h a s been done u s i n g first steam and t h e n u n i t s t o damp o u t
t h e s e o s c i l l a t i o n s h as been r e p o r t e d i n a
series of p ap er s by S c h l e i f and o t h e r a u t h o r s [3-51. In
the
earlier
d i f f i c u l t i e s which
involved
only
the
P a c i f i c n o r t h w est ,
frequency of t h e s e o s c i l l a t i o n s was approximately 3 c y c l e s p er minute.
the
Many times
the y would b u i l d up s p o n t a n e o u s l y , l a s t f o r s e v e r a l c y c l e s , and t h e n d i e away.
At
o t h e r times, however, t h e o s c i l l a t i o n s would b u i l d up t o a l e v e l and d u r a t i o n such t h a t c o r r e c t i v e a c t i o n would have t o be taken.
T h i s i n v o l v ed e i t h e r p l a c i n g
a d d i t i o n a l damping i n t h e g o v e r n o r s , b lo c k in g t h e governors, or b l o ck i n g t h e g a t e s on t h e g e n e r a t i n g u n i t s which appeared t o be c o n t r i b u t i n g t o t h e o s c i l l a t i o n s . The o s c i l l a t i o n s t h a t o c c u r r e d when t h e P a c i f i c n o r t h w est was i n t e r c o n n e c t e d w i t h the
power systems i n C a l i f o r n i a ,
Arizona,
and a d j a c e n t t e r r i t o r i e s were of
a
similar nature except t h a t t h e f r e q u e n c i e s sometimes v a r i e d up t o 6 c y c l e s per minute.
Most of t h e s e e a r l i e r o s c i l l a t i o n s are b e l i e v e d t o have been caused by
i n t e r a c t i o n s between t h e g o v e r n o r s of
t h e v a r i o u s u n i t s involved.
Most r e c e n t
cases, however, have been caused by i n t e r a c t i o n s i n v o l v i n g t h e high speed v o l t a g e r e g u l a t o r s now b ein g used o n some of t h e g e n e r a t i n g u n i t s . I n t h e e a r l y stages of t h i s work i t w a s d e s i r e d t o have a n example s y s t e m which could be used t o test t h e programs b e i n g developed.
Working w i t h t h e staff of t h e
Bon n ev i l l e Power A d m i n i s tr a ti o n and t h e Corps of Engineers, a s i m p l i f i e d 8 generating s t a t i o n r e p r e s e n t a t i o n of t h e P a c i f i c northwest was developed.
Many of t h e
pa r am et er s i n t h e models used were e v a l u a t e d o n t h e b a s i s of the i n t u i t i v e judgment of t h e engineers who had worked f o r a l o n g time w i t h t h i s system.
Also, many
g e n e r a t i n g s t a t i o n s were e i t h e r i g n o r e d or lumped t o g e t h e r t o make one of t h e eight
equivalents
t h a t were used.
The results f r m t h i s v er y
crude model
p r e d i c t e d t h e observed o s c i l l a t i o n s and gave a great d e a l of encouragement for t h e
J. E. Van Ness
186
f u r t h e r d e v e l o p e n t of th e s e methods.
L a t e r work showed t h a t t h e r e was a l a r g e
element of pure l u c k i n t h i s o r i g i n a l study because of t h e i n a c c u r a c i e s i n t h e crude model t h a t was used.
However, t h e results o f t h i s e a r l y study w i l l be used
h e r e as a n example of t h e type of study t h a t can be made. The example [61 c o n t a i n s 8 h y d r o e l e c t r i c g e n e r a t i n g s t a t i o n s whose b l o ck diagrams
are of
t h e form shown i n Fig.
2.
I n t h i s study t h e e f f e c t of
the voltage
r e g u l a t o r and synchronous machine e q u a t i o n s were n eg l ect ed s o t h a t t h e upper p a r t of t h e diagram i n Fig. 1 was p o t used. Fig. 2 a r e based o n a hydro-turbine t u r b i n e i n Fig.
1.
The governor and t u r b i n e r e p r e s e n t a t i o n i n
and t h u s d i f f e r from t h o se g i v e n f o r a steam
Also, c o n t r o l s i g n a l s a r e s e n t t o t h e i n d i v i d u a l g e n e r a t i n g These s i g n a l s are shown
u n i t s from a c e n t r a l lo a d frequency c o n t r o l system. coming i n from t h e l e f t of t h e diagram.
I n t h i s particular st u d y , only one of t h e
s t a t i o n s h a s p r o p o r t i o n a l c o n t r o l , so t h a t only one Kp i s non-zero.
REGULATOR
Figure 2.
Block diagram of hydr-turbine
and g en er at o r .
The s t a t i o n s are i n t e r c o n n e c t e d through an e l e c t r i c a l t r a n s m i s s i o n network and a c o n t r o l system which measures t h e power a t d e s i g n a t e d p o i n t s o n t h e system and from
t h i s d et er m i n e s t h e c o n t r o l
s i g n a l s Pc and P,A
f o r each s t a t i o n .
The
r e s u l t i n g system is of 52nd o r d e r . The computer program g i v e s a l i s t of all of t h e e i g e n v a l u e s of t h e system and a l s o p l o t s them u s i n g a p r i n t e r - p l o t t i n g r o u t i n e .
This p l o t i s shown i n Fig. 3.
Since
many of t h e r o o t s f a l l c l o s e t o g e t h e r and n e a r t h e o r i g i n , t h e r e s o l u t i o n of t h i s type of p l o t is not great enough t o separate them. could be found from t h e l i s t e d values. v e r t i c a l scales on t h i s p lo t.
However, t h e i r ex act v a l u e s
Note t h e d i f f e r e n c e i n t h e h o r i z o n t a l and
T h i s i s t y p i c a l of h y d r o e l e c t r i c systems.
For t h e
set of parameters used i n t h i s example, one p a i r of ei g en v al u es i s j u s t i n t o t h e r i g h t half-plane.
It i s of
i n t e r e s t t h a t t h i s pair of
ei g en v al u es seems t o
correspond t o a mode of o s c i l l a t i o n t h a t sometimes b u i l d s up spontaneously o n t h e a c t u a l system.
187
Eigenvaluel Vector Use 111 Electric Power System Problems PlOl OF ElGENVAlUES
I
v
A G
I
N A R V
*
I
* *
- 1 I I
I 1 I
A X I
5 -2.
-3.
-1.
0.
RE4L A X I S
R P 4 O P E R A l l O N S S l U O Y NO
2
F i g u r e 3. P l o t of e i g e n v a l u e s of sample problem.
THE POWERTON 6 STUDY
On May 12, 1976, real and r e a c t i v e power o s c i l l a t i o n s o ccu r r ed o n t h e Commonwealth Edison system.
The s o u r c e of t h e o s c i l l a t i o n s was shown t o be Powerton U n i t 6.
Powerton S t a t i o n c o n s i s t s of
two 992 MVA tandem
g e n e r a t o r s which were i n s t a l l e d i n t h e 1970's.
compound,
3600 rpm t u r b i n e-
The s t a t i o n i s l o c a t e d approx-
i ma t el y 260 kro ( 1 6 0 m i l e s ) southwest of Chicago,
o u t s i d e of Pekin,
Power i s t r a n s m i t t e d t o t h e Chicago area over f o u r 345 kV l i n e s . and t r a n s m i s s i o n c o n n e c ti o n s are shown i n F i g u r e 4.
Illinois.
The g e n e r a t i o n
Each u n i t i s equipped w i t h a
General E l e c t r i c ALTERREX* e x c i t a t i o n system which has a response r a t i o (ESR) o f 2.0
and a General E l e c t r i c Model LA202 power system s t a b i l i z e r .
Both u n i t s are
equipped w i t h e l e c t r o h y d r a u l i c s p e e d - c o n t r o l ( E H C ) systems. P r i o r t o experiencing the o s c i l l a t i o n s , approximately 820 W and 25kV.
both Powerton U n i t s were o p e r a t i n g a t
The l i n e t o Dresden (0302) and a l i n e t o Goodings
Grove (0303) were o u t of s e r v i c e due t o maintenance and t h e bus t i e b r eak er was open.
A d d i t i o n a l l y , n e i t h e r power system s t a b i l i z e r was i n s e r v i c e as they were
awaiting a change i n t h e i n p u t t r a n s d u c e r and f i n a l f i e l d testing. The s t a t i o n o p e r a t o r was a t t e m p t i n g t o i n c r e a s e t h e l o a d on Powerton #6 from 820
J. E. Van Ness
188
TLoCKsmT
Figure 4.
G e n e r a ti o n and Transmission Connections of May 12, 1976.
EIW t o 830 W when real and r e a c t i v e power o s c i l l a t i o n s began a t Powerton and were observed a t o t h e r Commonwealth Edison g e n e r a t i n g s t a t i o n s and on some t i e l i n e s . The o s c i l l a t i o n s
decayed when
Powerton
o s c i l l a t i o n s were n o t i c e d on t h r e e
#6 power o u t p u t was reduced.
The
s u c c e s s i v e a t t e m p t s t o i n c r e a s e l o a d and
decayed each t i m e l o a d was reduced. The o s c i l l a t i o n s observed on s w i tc h b o a r d m e ter i n g a t Powerton were approximately 30 t o 60 W peak t o peak and 10 Mvar on U n i t #6 and approximately 5 MW and 5 Mvar o n U n i t #5.
The frequency of t h e o s c i l l a t i o n s was approximately 1 Hz.
Subsequent
tests s h ar ed t h a t when t h e o s c i l l a t i o n s appeared, they could a l w a y s be e l i m i n a t e d by e i t h e r r ed u ci n g t h e u n i t ' s
power o u t p u t , i n c r e a s i n g t h e u n i t ' s e x c i t a t i o n or
sw it ch i n g t h e au t o m a t ic v o l t a g e r e g u l a t o r out of s e r v i c e . F u r t h e r a n a l y s i s showed t h a t a l t e r i n g some of t h e e x c i t a t i o n system parameters could a l s o s t a b i l i z e Powerton P6.
The major changes were i n t h e forward l o o p and
feedback gains. Following c o r r e c t i o n of the problem by r e s e t t i n g c e r t a i n v o l t a g e r e g u l a t o r g a i n s and t i m e c o n s t a n t s and r e p l a c i n g p a r t of t h e v o l t a g e r e g u l a t o r c o n t r o l s , a post i n c i d e n t a n a l y s i s was s t a r t e d t o simulate t h e i n c i d e n t and see i f t h e problem could have been p r e d i c t e d . PALS Rogram,
S e v e r a l models of t h e system were formed u s i n g t h e
t h e l a r g e s t b e i n g 177th o r d e r .
Th i s a n a l y t i c a l st u d y d u p l i c a t e d
f i e l d o b s e r v a t i o n s of Powerton U n i t #6 i n t h e o p e r a t i n g range of 800 t o 850 MW and 24.5
kV t o 25.5 kV ( . 9 8 t o 1.02 per u n i t ) .
The a n a l y t i c a l and f i e l d o b s e r v a t i o n s
were i n good agreement both i n t h e q u a l i t a t i v e sen se ( s t a b i l i t y vs. i n s t a b i l i t y ) and q u a n t i t a t i v e l y (matching t h e dominant f r e qu en cy ) . p r e s e n t e d i n r e f e r e n c e 7.
'Ihe d e t a i l e d results were
Eigenvaluel Vector (Jse iri Electric Power System Problems
189
OTHER APPLICATIONS While t h e s e methods were developed f o r t h e s t u d y of t h e s t a b i l i t y of a l a r g e power s y s t e m , t h e y have been a p p l i e d t o s e v e r a l o t h e r problems.
arose i n t h e d e s i g n of
One i n t e r e s t i n g problem
l a r g e a c c e l e r a t o r s f o r use i n nuclear physics.
These
a c c e l e r a t o r s have a p e r i o d i c power r e q u i r e m e n t similar t o t h e form shown i n Fig.
5.
I n t h i s particular waveform t h e base power i s 100 megawatts so t h a t t h e t o t a l
power s w i n g o v e r t h e two second p e r i o d i s 282 megawatts. s e v e r e cases.
In the past,
T h i s i s one of t h e more
t h i s f l u c t u a t i o n i n power h a s been accommodated by
u s i n g motor g e n e r a t o r s s e t s w i t h l a r g e f l y w h e e l s t o s e r v e as energy s t o r a g e devices.
The q u e s t i o n was r a i s e d a s t o whether such a l o a d c o u l d be connected
d i r e c t l y t o a modern power system w i t h o u t e x c i t i n g o s c i l l a t i n s i n t h e system t h a t would be u n d e s i r a b l e .
TIME F i g u r e 5.
- SECONDS
Waveform o f l o a d .
S e v e r a l methods of a n a l y z i n g t h i s problem were t r i e d i n v o l v i n g b o t h d i r e c t d i g i t a l s i m u l a t i o n and a n a l o g s i m u l a t i o n of t h e system.
The most s u c c e s s f u l and f l e x i b l e
was a method u s i n g t h e e i g e n v a l u e s and e i g e n v e c t o r s of t h e system [8,91.
The
b a s i c method was one t h a t i s p r e s e n t e d i n a first c o u r s e i n c i r c u i t t h e o r y .
The
waveform shown i n Fig. 5 was broken i n t o i t s harmonic components.
The r e s p o n s e of
t h e system t o e a c h of t h e components could t h e n be e x p r e s s e d simply i n terms of t h e e i g e n v a l u e s and e i g e n v e c t o r s .
The r e s p o n s e of t h e v a r i o u s v a r i a b l e s i n t h e
system was r e c o n s t r u c t e d by a d d i n g t o g e t h e r t h e r e s p o n s e of
those variables t o
J. E. Van Ness
190
each of t h e harmonics. t h e United States.
The method was a p p l i e d t w i c e t o two d i f f e r e n t l o c a t i o n s i n
I n one case i t was d e c i d e d t h a t t h e r e s u l t i n g v a r i a t i o n s i n
v o l t a g e and f r e q u e n c y would n o t be a c c e p t a b l e , w h i l e i n t h e second c a s e they were.
For t h i s second s t u d y t h e power s y s t e m s i n t h e midwestern U n i t e d S t a t e s were approximated by a 1 1 6 t h o r d e r system.
The PALS Program was used t o form t h e
system m a t r i x a n d t o f i n d i t s e i g e n v a l u e s and e i g e n v e c t o r s . v a l u e s and e i g e n v e c t o r s , v a r i o u s waveforms of e f f e c t on t h e system.
U s i n g these eige*
t h e l o a d were s t u d i e d f o r t h e i r
To be sure t h a t t h e worst c a s e had been c o n s i d e r e d , t h e
p e r i o d of t h e waveform was v a r i e d s o t h a t s p e c i f i c modes o f t h e o v e r a l l system, including its voltage
r e g u l a t o r s and g o v e r n o r s , w o u l d be e x c i t e d .
The major
advantage of t h i s approach was t h e a b i l i t y t o i d e n t i f y t h e s e c r i t i c a l modes and t o e a s i l y s t u d y the e f f e c t when t h e y were d i r e c t l y e x c i t e d . Another digital [lO,ll]
i n t e r e s t i n g problem
.
computer was
used
involved t h e e f f e c t t o g i v e load-frequency
of
t h e sampling rate when a control
of
a
power
system
The r e s u l t i n g model i n v o l v e d both d i f f e r e n t i a l and d i f f e r e n c e e q u a t i o n s .
These were t r a n s f o r m e d i n t o a s e t of d i f f e r e n c e e q u a t i o n s which were s t u d i e d f o r
stability. t h e system.
The t r a n s f o r m a t i o n s were based on t h e e i g e n v a l u e s and e i g e n v e c t o r s of
Some of t h e s u b r o u t i n e s i n t h e PALS F’rogram were developed as part of
t h e study of t h i s problem.
It was found t h a t c e r t a i n modes of o s c i l l a t i o n of t h e
system might be e x c i t e d i f t h e sampling r a t e of t h e d i g i t a l computer were i m p r o p e r l y chosen.
However, i t a l s o was found t h a t i f t h e system d e s i g n e r was aware
t h a t t h i s c o u l d happen, he c o u l d a v o i d t h o s e c r i t i c a l sampling rates s o t h a t t h i s
would n o t be a problem o n t h e system. I n r e c e n t y e a r s a n o t h e r type of problem h a s been of g r e a t c o n c e r n t o power system engineers.
I n l o n g ac t r a n s m i s s i o n l i n e s i t i s common t o i n s e r t series c a p a c i t o r s
t o i n c r e a s e t h e power t r a n s m i t t i n g c a p a b i l i t y of t h e l i n e .
The r e s u l t i n g series
L C c i r c u i t may have a r e s o n a n t f r e q u e n c y of 40 H z , f o r example.
If t h i s frequency
corresponds t o one of t h e n a t u r a l f r e q u e n c i e s of t h e spring-mass system formed by t h e g e n e r a t o r and i t s t u r b i n e s , a n o s c i l l a t i o n may result which h a s broken t h e s h a f t between t h e g e n e r a t o r and t h e t u r b i n e s .
Eigenvalue a n a l y s i s i s one approach
used f o r t h i s problem, but s i n c e t h e t r a n s m i s s i o n system can no l o n g e r be repres e n t e d by i t s s t e a d y - s t a t e e q u a t i o n , t h e r e s u l t i n g system of e q u a t i o n s i s of much h i g h e r o r d e r t h a n t h o s e d e s c r i b e d e a r l i e r i n t h i s paper.
For more i n f o r m a t i o n on
t h i s problem, c a l l e d the subsynchronous resonance problem, see r e f e r e n c e 1 2 .
Eigeizvalue/ 17ector U r c
iti
191
Electric Power-System Pr-obleins
REFERENCES 1.
J. E.
Van Ness, "PALS
t h e B o n n e v i l l e Power
-
A Program f o r A n a l y z i n g L i n e a r S y s t e m s , " A r e p o r t t o
published a t Northwestern University,
Administration
March 1969. 2.
A.
R. Benson a n d D.
G. Wohlgemuth, "System Frequency S t a b i l i t y i n t h e P a c i f i c
N o r t h w e s t , " IEEE T r a n s . o n Power A p p a r a t u s a n d S y s t e m s , No. 6 4 , pp. 765-773, F e b r u a r y 1963.
3.
F. R.
Schleif and J . H. White, "Damping f o r t h e Northwest-Southwest T i e - l i n e
Oscillations
An Analogue
-
S y s t e m s , Vol. PAS-85, 4.
F.
R.
S c h l e i f , G.
E.
Study,"
Trans.
IEEE
No. 1 2 , pp. 1239-1247,
Martin,
R.
w i t h a H y d r o g e n e r a t i n g U n i t , " IEEE T r a n s . Vol. PAS-86, 5.
No. 4 , pp. 438-442,
Apparatus
and
December 1966.
Angell,
R.
o n Power
"Damping of System O s c i l l a t i o n s o n Power A p p a r a t u s a n d S y s t e m s ,
A p r i l 1967.
F. R. S c h l e i f , H. D. Hunkins, G. E. M a r t i n , E. E. H a t t a n , " E x c i t a t i o n C o n t r o l
t o Improve P o w e r l i n e S t a b i l i t y , " I E E E Trans. o n Power A p p a r a t u s a n d S y s t e m s , Vol. PAS-87, No. 6 , pp. 1426-1434, J u n e 1968. 6.
A.
R.
Benson,
W.
F.
Tinney,
and D,
G.
Wohlgemuth,
" A n a l y s i s of Dynamic
Response of Electric Power S y s t e m s t o Small D i s t u r b a n c e s , " Proc. I n d u s t r y Computer A p p l i c a t i o n s Conf. 7.
J.
E.
Van Ness,
I n v e s t i g a t o n of
F.
M.
,
B r a s c h , G.
L.
Dynamic I n s t a b i l i t y
1965 Power
pp. 247-259. Landgren, S.
T.
Nauman,
"Analytical
O c c u r r i n g a t Powerton S t a t i o n ,
T r a n s . o n Power A p p a r a t u s and S y s t e m s , Vol. PAS-99,
"
IEEE
No. 4 , J u l y / A u g u s t 1 9 8 0 ,
PP. 1386-95.
8.
J. E.
Van Ness, "Response of Large Power S y s t e m s t o C y c l i c Load V a r i a t i o n s , "
IEEE T r a n s .
o n Power A p p a r a t u s and S y s t e m s , Vol. PAS-85,
No. 7 , J u l y 1966,
PP. 723-727.
9.
J. E.
a
Van Ness, J. A.
C y c l i c Load
P i n n e l l o , "Dynamic Response of a Large Power System t o
Produced by
a N u c l e a r Accelerator," IEEE T r a n s .
A p p a r a t u s a n d S y s t e m s , Vol. PAS-90,
No. 4 , J u l y 1 9 7 1 , pp. 1856-1962.
o n Power
J. E. Van Ness
192
10.
F.
P.
Imad and J .
E b Van Ness,
" F i n d i n g t h e S t a b i l i t y and S e n s i t i v i t y of
L a r g e Sampled S y s t e m s , " IEEE Trans. o n A u t o m a t i c C o n t r o l , Vol.
AC-12,
August 1967, pp. 442-1145.
11.
No. 4,
J. E. Van Ness and R. R a j a g o p a l a n , "Effect of D i g i t a l S a m p l i n g Rate o n System
Stability,
Proc. 5 t h Power I n d u s t r y Computer A p p l i c a t i o n s Conf.,
Pittsburgh,
P e n n s y l v a n i a , May 1967, pp. 41-46.
12.
BEE
Committee
Report,
"A
Bibliography
for
the
Study
Resonance Between R o t a t i n g Machines and Power S y s t e m s ,
"
of
Subsynchronous
IEEE T r a n s . on Power
PAS-95, No. 1 , J a d F e b 1976, pp. 216-218. S u p p l e m e n t , Vol. PAS-98, No. 6, Nov/Dec 1979, pp. 1872-1875. Approach & S y s t e m s , Vol.
First
Large Scale Eigenvalue Problems
193
J. Cullum and R.A. Willoughby (Editors) 0 Elsevier Science Publishers B.V. (North-Holland), 1986
A PRACTICAL PROCEDURE FOR COMPUTING EIGENVALUES OF LARGE SPARSE NONSYMMETRIC MATRICES
Jane Cullum and Ralph A. Willoughby IBM T.J. Watson Research Center, P.O. Box 218 Yorktown Heights, New York 10598 U.S.A.
We propose a Lanczos procedure with no reorthogonalization for computing eigenvalues of very large nonsymmetric matrices. (This procedure can also be used to compute corresponding eigenvectors but that issue will be dealt with in a separate paper.) Such computations are for example, central to transient stability analyses of electrical power systems and for determining parameters for iterative schemes for the numerical solution of partial differential equations. Numerical results for several large matrices are presented to demonstrate the effectiveness of this procedure.
1. INTRODUCTION
Economical procedures for computing eigenvalues and eigenvectors of very large but sparse real symmetric matrices exist. See for example Cullum and Willoughby [ 11and Parlett and Scott [2]. However comparable progress in the computation of eigenvalues and eigenvectors of nonsymmetric matrices has not yet been reported. The algorithms currently available for such computations, see for example Stewart and Jennings [3] and Saad [4] are very useful but somewhat limited in the amount of spectral information which they can obtain. The simultaneous iteration procedure described in [3] can be used to compute a few of those eigenvalues of the given matrix which are largest in magnitude. The procedures in [4] are based on an iterative Arnoldi’s method and use Hessenberg matrices as approximations to the original matrix. Both types of procedures do not modify the given matrix A and use only products of the form Ax. The procedures discussed in [ 11 are based upon variants of the basic Lanczos recursion. For a given real symmetric matrix A, and a starting vector v1 (which is typically generated randomly) this recursion can be used (at least theoretically) to generate orthonormal bases for the Krylov subspaces Xm(A, vl) z {v,, Av,,
... , Am-lvl]
for m = 1, 2, ... , associated with A and v I .
This generation produces a family of real symmetric tridiagonal matrices Tm which (in exact arithmetic) represent orthogonal projections of the given original matrix A onto the corresponding Krylov subspaces. In these procedures, approximations to the eigenvalues of the given
J. Cullurn and R . A . Willoughby
194
matrix A are obtained by computing eigenvalues of one or more of the Lanczos matrices generated, and then selecting some subset of the computed eigenvalues as approximations to eigenvalues of A. The nominal justification for this type of procedure is the “fact” that the Lanczos matrices are projection matrices for A and thus the computed eigenvalues of these matrices T,, are eigenvalues of the operators obtained from A by restricting A to the Krylov subspaces
Sm.
However in practice, this justification is not applicable because the
orthogonality upon which this argument is based does not persist in finite precision arithmetic.
In general the basic Lanczos procedure will not function properly without some modification. Various modifications have been proposed. They are discussed briefly in Chapter 2 of [l]. The interested reader is referred to [ l ] for detailed descriptions of one class of such algorithms.
Because of the observed efficiences and speed which have been obtained using Lanczos procedures on real symmetric matrices, there have been attempts recently to devise Lanczos algorithms for computing eigenvalues of large nonsymmetric matrices. These are based upon the basic two-sided Lanczos recursion for nonsymmetric matrices which is given in Equations (2.1)-(2.2) below. See in particular, Parlett, Taylor and Liu [ 5 ] . This paper and its predecessor do not contain very many numerical results. However they indicate that Lanczos procedures may also be very effective tools for nonsymmetric problems.
In attempting to construct a nonsymmetric Lanczos procedure from the basic equations given in Eqns(2.1)-(2.2), the algorithm designer must be aware of two possible difficulties. First (see Section 2 ) it is possible (at least theoretically) that a normalizing factor which is used in the recursions may vanish. If this were to happen, then the standard Lanczos recursions would have to be terminated. The second possible difficulty is that the roundoff errors caused by the finite precision arithmetic may destroy the theoretical relationship between the original matrix and the Lanczos matrices being generated. The Lanczos procedure proposed in reference [5] is specifically designed to mollify the first possible difficulty. These authors modified the basic twosided Lanczos recursion to incorporate an analog of the block pivoting for stability which was used by Bunch and Parlett [6] in the solution of systems of linear equations. Reference [ 5 ]states that they address the second possible difficulty by the continuous reorthogonalization or biorthogonalization of the Lanczos vectors as they are generated by the recursions.
Compu ring Eigenvalucs of Large Sparse Nonsymmetric Matrices
195
Modal computations for nonsymmetric matrices are inherently much more difficult than similar computations for real symmetric matrices.
Real symmetric matrices are always
diagonalizable. Any real symmetric matrix always has a full set of eigenvectors. Moreover, these eigenvectors can always be chosen to be orthogonal vectors. In fact, the eigenvalue computations for real symmetric matrices are ‘perfectly’ conditioned.
Furthermore, all of the
eigenvalues are real, and all of the eigenvectors are real vectors.
I n general, nonsymmetric matrices A have two sets of eigenvectors, right eigenvectors X
= {x,, . . . , x,)
such that AX = XA, and left eigenvectors Z
= { z , , ... , zJ)
such that
T
A Z = Z h . The matrix A is diagonal, and its nonzero entries are eigenvalues of A. X and Z are
(real) biorthogonal, that is XTZ = 1. but the number J of right or left eigenvectors may be less than the order of A . Furthermore, the individual sets of vectors X and Z need not and probably do not form orthogonal sets of vectors.
DEFINITION 1.1. Any nxn matrix A which does not have a complete set of right and left eigenvectors is called a defective or nondiagonalizable matrix.
Evcry real symmetric matrix is nondefective. Any matrix with distinct eigenvalues is nondefective (diagonalizable). For any diagonalizable matrix A we have that there exists a nonsingular matrix X and a diagonal matrix A such that
Clearly the columns of X in Eqn( 1.1 ) are right eigenvectors of A and the diagonal entries of A are the eigenvalues of A. The interested reader should see for example, Wilkinson [7, Chapter I ] for more background information on defective matrices and for examples of the effects of the
defectiveness upon any numerical procedure for computing eigenvalues. We restrict our considerations to nondefective nonsymmetric matrices. We do not make any claims for defective matrices.
In this paper we present a Lanczos procedure with no reorthogonalization for computing eigenvalues of large nondefective, nonsymmetric matrices. In Section 2 we first restate the general two-sided Lanczos recursion formulas for nonsymmetric matrices given for example in Wilkinson [7, Chapter 61. We then state a general procedure for using these recursions to obtain
J. Cullurn and R.A. Willoughby
196
approximations to eigenvalues .of a nondefective nonsymmetric matrix. In Section 3 we then outline our proposed algorithm which uses complex arithmetic, and derive some fundamental relationships valid in exact arithmetic. In practice when finite precision arithmetic is used, the basic Lanczos procedures as stated in Sections 2 and 3 do not possess the biorthogonality relationship given in Lemma 3.1. In order to obtain a practical procedure, it is necessary to modify the basic procedure. The modification which we use is given in Section 4 along with a justification for it. The behavior of this modified procedure on several large and medium-size problems is then illustrated in Section 5. Computations for small and medium size test problems yielded eigenvalue approximations with accuracies comparable to those obtained using the relevant subroutines contained in the EISPACK Library [8]. One of these examples is given in Cullum and Willoughby [9].
One unusual aspect of our procedure is that the computations are in complex arithmetic even when the original matrix is real. In fact all of the examples we present are real nonsymmetric matrices. Typically in the literature, see for example, EISPACK [8], such computations are modified so that the arithmetic is all real.
2. GENERAL TWO-SIDED LANCZOS RECURSION
Wilkinson [7] gives the following general two-sided Lanczos recursion for any nonsymmetric matrix A. Please refer to Chapter 6 in [7] for additional details. Specifically, let A be a real nxn matrix. Let v I and w I be two nxl vectors with their Euclidean “inner product” v,Tw I = 1 . Note that in general we will take these starting vectors to be complex vectors.
For W,,
i = 1 , 2 , .. . , M
use
the
following
= (wl, ... , wm] and V, = {vl, ... , v,]
recursions
to
and scalars y i + l ,
define and
ai
where T
ai
= wi Avi,
T
yi
= wi-,Avi,
T
T
and pi = viPIA wi.
Lanczos
such that
vectors
Computing Eigenvulues of Large Sparse Nonsymmetric Mutrices
197
The coefficients al,pi, yi are chosen such that for each 1 5 j 5 M, the sets of Lanczos vectors ( V j ) = { v , , ... , v,) and (Wj]
= ( w I , . . . , wj j
are (real) biorthogonal.
That is for each j,
T
V i W , = Ij. Therefore the corresponding vectors aivi and aiwi are respectively, the real T
biorthogonal projections of the vectors Avi and A wi onto the most recently-generated Lanczos v-vcctor v , and w-vector wi. Similarly, the corresponding vectors yivi-l and Biwi-, are respecT
tivcly, the corresponding real biorthogoiial projections of the vectors Avi and A wi onto the next most rccently-generated Lanczos v-vector vi- I and w-vector w,-
I.
Thc corrcspondiiig tridiagonal 1-anczos matrices are defined by the scalars yi+,.
ai,
pi+I ,
and
That is for each m = I , 2, ...
TI,
=
Since these recursions only explicitly biorthogonalize v,+ I and w , + ~respectively w.r.t. w, and , is necessary to prove that in fact w,Tvk = 0 for any j # k. We do not give w,-, and v, and v , - ~ it
a general proof of this fact here. However in the next section we give a very similar proof for the particular variant of Eqns(2.1)-(2.2) which we use in our procedure.
Since the two sets of vectors W, and V j are biorthogonal (in exact arithmetic), we have that for j = 1, 2, . . . the Lanczos matrices in Eqns(2.3) are just biorthogonal projections of A onto the subspaces Vj. That is, T
Tj = Wj AVj.
(2.4)
We can use the recursions defined in Eqns(2.1)-(2.2) to define the following basic Lanczos procedure for computing eigenvalues of nonsynimetric matrices.
BASIC LANCZOS EIGENVALUE PROCEDURE
STEP 1 .
Given a matrix A and a user-specified maximum size M, use the Lanczos recursions in Eqns(2.1)-(2.2) to generate a Lanczos matrix T, of order M.
198
J. Cullum and R.A. Willoughby
STEP 2.
For some user-specified m 5 M compute the eigenvalues of T,. Compute error estimates for each of these eigenvalues and select those eigenvalues with sufficiently small error estimates as approximations to eigenvalues of A.
STEP3.
If convergence is observed on all of the eigenvalues of interest, terminate the
computations. Otherwise enlarge the Lanczos matrix T, and repeat step 2. This basic Lanczos procedure replaces the direct computation of the eigenvalues of the given matrix A by an analogous computation of eigenvalues of the associated tridiagonal Lanczos matrices T,. Observe that the Lanczos recursions do not explicitly modify the given matrix A and only the two most recently-generated pairs of Lanczos vectors vk and wk for k = i, i - 1 are needed at each stage in the recursions. The storage requirements are very small when A is sparse or of a form such that the matrix-vector multiplies can be generated cheaply w.r.t. storage and time requirements.
Theoretically, it is easy to demonstrate that for each i, we have the following relationships between the scalars y i + I and
In other words for each i, the product of these two parameters must be equal to the “inner product” of the two right-hand sides of Eqns(2.1). Furthermore observe that we have used a real “inner product” as Wilkinson [7] does, not the Hermitian inner product. Since there exist complex vectors, for example x = ( 1 , i), for which xTx = 0 even though x # 0, this is obviously only a pseudo-inner product when we apply it to complex vectors.
Observe that this “inner product” can vanish (this would also be true even if we used a Hermitian inner product). If it vanishes then the recursion cannot be continued in its current
form. However, our experience and the experiences of others lead us to believe that this is a rare event. In fact even though Parlett, Taylor and Liu [ 5 ] are specifically addressing this potential difficulty, they were able to construct only one example where this type of failure was observed in practice and that occurred only for a very special choice of starting vector. In fact the test matrix which they were considering was itself very special in that it was a persymmetric matrix.
C o m p u t i n g Eigenvalues of Lnrge Sparse Nonsymmetric Matrices
199
DEFINITION 2.1. A matrix is persymmetric if and only if the'associated matrix B
= AJ where J(i,j) =
1, for i
+ j = n + 1 and otherwise J(i, j) = 0,
is symmetric.
I n other words a matrix is persymmetric if and only if it is symmetric w.r.t to its cross diag-
onal. Observe that if A is persymmetric then A ' = JAJ. In fact one can show that for this type of matrix two Lanczos recursions are not necessary; one will suffice. In the next section we
present a spccialization of the above general Lanczos recursion which seems to have very nice numerical properties.
3. PROPOSED LANCZOS EIGENVALUE ALGORITHM
We propose a double-vector, nonsymmetric Lanczos procedure with no reorthogonalization based upon the variant of Eqns(2.1)-(2.2) which is contained in Eqns(3.2). Nominally, we have three sets of scalars a , , pi, and yi. We can however reduce this number to two sets of parameters if we make the following choice
See Equations (2.1). The penalty for doing this is that even if we start with a real matrix and real starting vectors, depending upon the particular matrix, we can quickly obtain complex scalars and vectors. Furthermore, in our procedure we will always set v I = w I . With this choice wc will see that the only nonsymmetry in the Lanczos matrix generation procedure will be that which is a consequence of the nonsymmetry of A.
First we prove that with this particular choice for the scalars yi+, and
pi+, that the
biorthogonality of the Lanczos vectors is achieved (in exact arithmetic). With these choices, Eqns(2.1)-(2.2) reduce to j3i+lvi+l= Avi - aivi - piviPl= T
/3i+lwj+l = A wi ai = v ai
v (ai T
+
w ai )/2
- L Y ~ W-, /3iwi_, = ti+I and
2
T
= (ri+lti+l) w
T
T
= wi (Avi - /3ivi-l)and ai = vi (A
wi -
200
J. Cullum and R.A. Willoughby
The definition of a,is not what is given in Eqns(2.2) hut is a theoretically equivalent expression which is analogous to the modification which was recommended by Paige [lo] for the real symmetric case.
These definitions yield symmetric but typically complex Lanczos matrices, even if the starting vector is a real vector. Specifically we have that
T,
=
(3.3)
See Chapters 3 and 6 of Volume 1 of Cullum and Willoughhy [ l ] for more details about complex symmetric tridiagonal matrices. Several of the results in [ 11 are repeated here. First we demonstrate that the Lanczos vectors are biorthogonal.
We should observe that for the special case when A is a complex symmetric matrix that the recursions given by Eqns (2.1)-(2.2) and (3.1) reduce to a single recursion.
Formally, this is the ‘same’ recursion as that which is used in the real symmetric case. However, when A is complex symmetric, A and thus the Lanczos vectors v, and the Lanczos matrices T, arc all complex-valued. See Chapter 6 of Volume 1 and Chapter 7 of Volume 2 of reference [ 11 for more details.
LEMMA 3.1. (Exact Arithmetic).
Let A be any nxn nonsymmetric matrix with q distinct
eigenvalues. Let v , and w, be any nxl vectors such that wTv, = 1. Then assuming that for each i that wTvi # 0, we have that the v-vectors and the w-vectors generated using Eqns(3.2) are hiorthogonal. Specifically, for any j # k, T
wi. vk = 0.
(3.5)
Computing Eigenvufui~of-Large Sparse Nonsymmetric Matrices
201
PROOF‘. Clearly, by construction W 2I V I = VZWI T
= 0.
Furthermore we have that
Similarly, we have that
Therefore Tor j
5 2,
Using Eqn(3.6) it is easy to prove directly that
Thc rest of the proof follows by induction. Assume that for any j
Bk = w
T
T
.r
T
T
~ A v = ~ -v,A ~ wk-[ and w1 vk = 0, wkvj = 0, j
Now we show that Eqn(3.7) follows for k
+ 1.
For any j
< k.
(3.7)
- 1 we have that
by the induction hypothesis o n the biorthogonality. For j = k this biorthogonality follows by construction.
Therefore, we have that
T
V ~ + ~ W = ,0
for j 5 k . Similarly, we have that
T
wk+lv1= 0 for j 5 k. Using this biorthogonality we easily obtain the required equivalent forniulas for &+I.
0
Since we have demonstrated the biorthogonality of the Lanczos vectors, we should be able
to use the basic Lanczos procedure given in Section 2 (using Eqns(3.2)) to compute eigenvalues of A. However if we try to do this, we quickly find that the biorthogonality is not preserved when finite precision arithmetic is used. The basic procedure must be modified to obtain a
202
J. Cullurn and R.A. Willoughby
practical procedure. We will do this in Section 4. However first in this section we look at some of the theoretical properties of the quantities involved.
Observc that with the choice of off-diagonal terms given in Eqn(3.1), we have symmetrized the effects of the roundoff errors in our Lanczos matrix generation. Since such symmetrization was important even in the case when the given matrix was real symmetric, see Paige [lo], we expect it to be significantly more important here due to the sensitivity of general nonsymmetric T
eigenvalue computations to perturbations in the matrix. If A = A with A real or complex, then the recursions in Eqns(3.2) reduce to the "symmetric" Lanczos recursions used in [I]. So the procedure which we propose is a natural generalization of our Lanczos procedures in [ I ] .
In matrix form for each m, Equations (3.2) become
where e,,, denotes the vector whose mth component is 1 and whose other components are 0. Thus this procedure maps a given general nonsymmetric matrix into a family of complex symmetric tridiagonal matrices. Two obvious questions about this procedure need to be answered. First we must demonstrate that we have not artificially excluded certain types of eigenvalue distributions by restricting ourselves to complex symmetric tridiagonal Lanczos matrices. The answer is that we have not. See Lemma 3.2 below. In particular we have that for any irreducible tridiagonal matrix there exists a diagonal similarity transformation which maps it into a complex symmetric tridiagonal matrix. Since similarity transformations preserve eigenvalues, Lemma 3.2 tells us that any eigenvalue distribution of any irreducible nonsymmetric tridiagonal matrix can be obtained as the eigenvalue distribution of a complex symmetric tridiagonal matrix.
DEFINITION 3.1. A matrix A is irreducible if and only if there are no permutation matrices P and Q such that the permuted matrix B
= PAQ is block lower triangular.
A matrix P is a per-
mutation matrix if and only if it all of its entries are either 0 or 1 and it contains exactly one nonzero entry in each row and column.
Computing Eigenvulues o f Large Sparse Nonsymrnetric Matrices
203
LEMMA 3.2. Let T, be any irreducible, tridiagonal mxm matrix so that T,(k - 1 , k ) # 0 and T,,,(k, k - 1) # 0 for 2 5 k 5 m. Then the nonsingular diagonal matrix D
= (dij) defined by
the recursion d,,
= d, E d , - , d B , / y ,
-
transforms T,, into the matrix T, define x
= D-lz,
= D-'T,,D,
for j
> 1 and d, = 1,
(3.9)
which is complex symmetric. Moreover if we
then the corresponding relationship between the eigenvectors is given by
Eqns(3.9).
-
T,z = pz if and only if T,x = px.
(3.10)
PROOF. It is a straightforward exercise to demonstrate that
(3.11)
which is obviously symmetric.
The second question is the following.
0
What if anything do we gain in addition to the
symmetrization of the error propagation by our use of symmetric Lanczos matrices? The answer is the ability to work with large matrices. We use a procedure for computing eigenvalues of complex symmetric tridiagonal matrices which requires no more storage than that required to define the tridiagonal matrix. This algorithm is a complex analog of a standard algorithm for real symmetric tridiagonal matrices. This latter algorithm appears in the EISPACK Library [8] as subroutine IMTQLl. We call the complex version CMTQL1, and it is described in Cullum and Willoughby [ 1 11. It has since been pointed out to us that a similar algorithm has been used by Gordon and Messenger [12].
Summarizing, there are three key elements which distinquish our proposed procedure from that in [ 5 ] . First, the two-sided Lanczos recursion is used in such a way that complex symmetric
J. Cullum and R.A. Wilioughby
204
tridiagonal Lanczos matrices are obtained. Numerical experiments indicate that this is an effective mechanism for controlling the errors generated by the use of finite precision arithmetic. (We are looking at theoretical arguments.) We cannot however claim to be the first to use complex symmetric Lanczos matrices.
The NASTRAN User’s Manual [13] contains a de-
scription of a two-sided Lanczos procedure for nonsymmetric matrices which also uses complex symmetric Lanczos matrices.
However that procedure also incorporates the total
reorthogonalization of all of the Lanczos vectors generated. Furthermore, no reasons were given for the choice of this type of Lanczos matrix, and no attempts were made to take advantage of this choice. Second, we use an eigenvalue procedure specifically-designed for complex symmetric tridiagonal matrices which has storage requirements which are linear in the order of the tridiagonal matrix. Third, we do not reothogonalize any vectors as we recurse. Therefore, the storage requirements are small. As with the real symmetric Lanczos procedures described in [ I ] , eigenvalues are computed first, then for user-selected eigenvalues, corresponding eigenvectors are computed using a separate program. Reference [ 5 ] states that the losses in biorthogonality are “handled” by reorthogonalizing Lanczos vectors w.r. t. previously-generated Lanczos vectors. Such reorthogonalizations however require that all of these vectors be kept on some easily accessible storage.
Other properties of complex symmetric matrices which make them particularly attractive are the following. First complex symmetric matrices are balanced a priori. Typically nonsymmetric matrices are balanced prior to the computation of their eigenvalues and eigenvectors. Balancing refers to modifying the matrix in such a way that the sizes of the entries of the matrix are balanced in size. The most popular balancing procedures attempt to make the norms of each corresponding pair of rows and columns approximately the same. Any modifications must be done in such a way that the eigenvalues are preserved.
EISPACK [S] contains the subroutine
BALANC which is used for this purpose. Discussions in Wilkinson [7] and in Parlett and Reinsch [ 141 illustrate the need for balancing matrices. However since our Lanczos matrices are complex symmetric, balancing is not necessary.
Second, once either the left or the right eigeiivector of a complex symmetric matrix is obtained, the corresponding right or left one is also known. In fact for complex symmetric matrices the left and’the right eigenvectors are identical. Thus, when error estimates are computed or Ritz vectors are computed, only one eigenvector of the Lanczos matrices (for each eigenvalue
Computing Eigenvaltres of Large Sparse Nonsymmetric Matrices
being considered) has to be computcd.
205
Moreover, as we will see in the next section the
eigenvectors of complex symmetric tridiagonal matrices have special structure which we will use to our advantage. Please see Section 4 for a discussion of this issue. In the next scction we give
an alternative proposal to reorthogonalization, namely no reorthogonalization at all.
4. A PRACTICAL LANCZOS PROCEDURE
In the real symmetric Lanczos procedures developed in Cullum and Willoughby [ 11 we did not use any reorthogonalization of the Lanczos vectors. Those procedures all rest upon the we a following Lanczos Phenomenon which was observed empirically and for which we b‘ plausibility argument in [I].
LANCZOS PHENOMENON(Conjecture). Given a real symmetric matrix A, if we use the real symmetric Lanczos recursion given for example by Eqns(3.2) with v
= w to generate associated
real symmetric tridiagonal Lanczos matrices T,,,, m = 1 , 2, . . . of size mxm, then for large enough in every distinct eigenvalue of A will appear among the eigenvalues of the Lanczos matrices T,,,.
Thus, this phenomenon indicated that what we wanted was there and that all we had to d o was to figure out a way to distinquish between what we wanted and the extra eigenvalues which appeared because of the losses in biorthogonality. We devised such a test which is used in most
of the algorithms described in [ 11. This identification test sorts the “good” eigenvalues from the “spurious” or extra ones among the eigenvalues of any Lanczos matrix.
For reasons which we do not fully understand at this point in time, this identification test which worked well for the real symmetric procedure has also worked well on the real nonsymmetric test problems which we have run to date. The observed losses in biorthogonality may occur quickly or slowly. This type of behavior appears to be analogous to that observed for real symmetric matrices. In the real symmetric case, the global orthogonality of the Lanczos vectors is lost upon the convergence of one or more of the eigenvalues of the Lanczos matrices being generated to eigenvalues of the original matrix as the size of the Lanczos matrices is increased. We see the same behavior in the nonsymmetric case. We emphasize again that we are restricting our discussion to diagonalizable nonsymmetric matrices.
J. Cullurn and R . A . Willoughby
206
We will give an argument supporting the use of our identification test in the nonsymmetric procedure. However, we first need several definitions and lemmas which we provide below.
DEFINITION 4.1. Let m be fixed and for any 1 5 i 5 j 5 m define submatrices of any Lanczos matrix T, as follows.
Also define 8,
= Yk&,
2 5 k 5 m. Furthermore we use the following notation. A
Tk s
and T,
= T,,,,
for 1 5 k 5 m.
(4.2)
The arguments used to justify our identification test use characteristic polynomials of some of these submatrices.
DEFINITION 4.2. For any tridiagonal matrix T, and any k = 1,2, .. . , m, we denote the A
characteristic polynomials of the submatrices T, and T, defined in Eqns (4.2) by
Observe that Tm-l is the submatrix of T, of order m-1 obtained by deleting the last row and A
the last column of T,. Similarly, T, is the submatrix obtained by deleting the first row and the first column. We have the following formulas for the eigenvectors of T,.
LEMMA 4.1. (Exact arithmetic). For any simple eigenvalue p of a mxm symmetric tridiagonal matrix T, we have the following expressions for the components of the right eigenvectors in terms of the characteristic polynomials of submatrices of T,. For k = 1 , 2 , 3 , ... , m we have that
where gm+l(p) = 1 and a&) following lemma from [l].
E
1. This lemma is proved for example, in [l]. We also need the
Cornpu tirzg Eigenvuhies of Luvge Sparse Nonsyinmetric Mutrices
207
LEMMA 4.2. (Exact Arithmetic). Let TI,, bc a tridiagonal matrix. Let p be an eigenvalue of TI,,.That is a,ll(p) = 0. Then we have the following equality.
From Lemma 4.1 we have that given any simple eigenvalue p of T,, and corresponding eigenvector u that the mth component of u , ti(m), is defined as follows,
For such a pair, { p , u), and each
m , we define a corresponding ‘pair’, Ritz value and Ritz
vectors rcspectively, { p , V,u, W,u) for the original matrix A . Using the matrix form of the Lanczos recursions given in Eqns ( 3 . 2 ) ,we obtain the following relationship for one of the two associated residual norms.
(4.7)
I n Eqn(4.7) Ell, denotes the effect of the roundoff errors
Our argument for our identification test rests upon several assumptions and facts. First and foremost we need to know that the size of the residual norm in Eqn(4.7) and of its counterpart for Wmu. yield an accurate.reflection (in some reasonable sense) of the accuracy of the computed Ritz value. Residual norms are used throughout the literature, see for example Wilkinson [ 71. The “correctness” of the eigenvalue approximations which we generate using our Lanczos
procedure will be measured by these residual norms.
In the real symmetric case it is “easy” to estimate the “correctness” of a given eigenvalue, eigenvector approximation ( p , x) by cxarnining the residual
In fact, we have the following well-known theorem. THEOREM 4.1. Let A be a real symmetric matrix. Given any scalar p and any nonzero vector x, there is an eigenvalue h of A such that
J. Cullurn and R.A. Willoughby
208
Therefore in the real symmetric case, small residuals guarantee small errors in the eigenvalue approximations. Furthermore we know that the eigenvalue computations for real symmetric matrices are perfectly conditioned. A perturbation of size e in a given real symmetric matrix A yields at most a perturbation of size E in the eigenvalues of the matrix.
For the real symmetric Lanczos recursion we have by construction that
)Iv , + ~ 11
=
1 for all
m. Furthermore, Paige [15] proved that in the case the errors Em grow at a rate bounded by
( 1 A 11,
m, and the machine epsilon and that the norms
11 V,u 11 are not “small”, if the recursion
is implemented appropriately. Therefore from Eqn(4.7), in the real symmetric case, we have that for any pair ( p , u) of Ritz value and vector obtained from a Lanczos matrix T,, that the degree of approximation of p to an eigenvalue of A can be estimated by the size of (4.10) where u(m) denotes the mth component of the eigenvector u. These quantities are easily computed.
The nonsymmetric case is much more difficult. For a detailed discussion of error estimation for nonsymmetric problems the reader is referred to Kahan, Parlett, and Jiang [ 161. The basic problcm is that small perturbations in a nonsymmetric matrix may yield significant perturbations in the eigenvalues of that matrix. The eigenvalue computations need not be well-conditioned.
In our discussions we restrict ourselves to diagonalizable nonsymmetric matrices. For such matrices we have the following theorem. THEOREM 4.2. (Bauer and Fike [17] ) . If for the matrix B = A - E, there exists an invertible matrix X and a diagonal matrix A such that B = XAX-’, i.e. B is diagonalizable, then to each eigenvalue h,(A) there exists an eigenvalue h , ( B ) such that (4.11)
Therefore, if we can show that the eigenvalues which we obtain from our Lanczos procedure are approxjmations to the eigenvalues of related matrices B
= A - E where E is “small”,
then
we have a direct but weak relationship between the eigenvalues of A and B. We d o not however
Computing Eigenvalues of Large Sparse Nonsynimetric Mahices
209
have a direct way of getting such an E. Kahan et al [ 161 show that it is possible to get an estimate of the size of such a perturbation E by looking at the size of the residual in Eqn(4.7) and it5
companion expression for the Wlnu vectors. See Theorem 4.3.
For such a result to be useful to us several assumptions must be satisfied. In particular we must assume that the errors Em caused by the finite precision arithmetic enter in a controlled small manner (similar to the behavior of those incurred in the real symmetric case), and that the norms of the RitL vectors x
= V,u
and y
= W,u
are not small. Then we have the following
residual rclationships.
Theorem 2’ in Kahan et al [ 161 statcs that a perturbation matrix E exists such that the matrix A - E has p as an eigenvalue and Vnlu and Wnlu as right and left eigenvectors. The statement
of their Theorem follows. THEOREM 4.3. (Kahan et al [ 161.) Let A and two unit vectors x and y with xTy # 0 be given. For any scalar p define two corresponding residual vectors r
= Ax - px,
and s
E
T
A y - py.
(4.13)
Define the subset of perturbation matrices ( E 1 ( A - E)x = px and (AT - ET)y = py). Then (4.14)
In the case when the scalar 1.1 in Theorem 4.3 is the Rayleigh quotient of A corresponding to the vectors x and y, Kahan et a1 [ 161 give an expression for the minimal perturbation in terms of the residuals r and s. Theoretically, the eigenvalues generated by our Lanczos procedure have
this property. The computed eigenvalue approximations are Rayleigh quotients if the Lanczos vectors V,,, and W, are biorthogonal. However in practice this biorthogonality is not preserved; we do not have this relationship (unless we reorthogonalize the Lanczos vectors).
In our nonsymmetric Lanczos procedure we can use Eqn(4.7) to estimate the size of the residual for V,,u and its analog for Wmu. However, as we said earlier, this does not give us a
210
J. Cullum and R . A . Willoughby
direct measure of the accuracy of the computed eigenvalue approximations. If
)I E 11 is small,
then Kahan et al [ 161 state that
where the condition of the eigenvalue h is defined as follows.
where x and y are the right and left eigenvectors of A corresponding to A. Obviously we do not have a way of estimating the condition without knowing x and y. Estimating it by using cigenvectors of the Lanczos matrices T,, is only valid if the biorthogonality of the Lanczos vectors has been maintained. Since we are not performing any reorthogonalizations, we estimate
1 E 11 from the residuals. All we can state is that if the effects of the finite precision arithmetic E,,, in Eqn(4.7) are small, then small residuals tell us that our eigenvalue approximations are eigenvalues of matrices close to the given matrix. We do not have a measure of how close these values are to the true values. In fact, however, this is the best which we can expect to be able to do.
Now we return to the discussion of our identification test which sorts the “spurious” eigenvalues from the “good” ones. Using first Eqn(4.7) and then Lemma (4.1), we get the following relationship
Then using Lemma 4.2 and Eqn(3. I ) so that for all k , Pk = yk, we obtain m iI
In Eqns(4.7), (4.15) and (4.16). E, represents the roundoff errors caused by the finite precision arithmetic. We have only empirical proof that these errors are controlled and stay small. The argument for the identification test which we give below assumes that these errors are small. In
Computing Eigenvulucs of.Lurge Sparse Nonsymmetric Matrices
21 1
addition we must assume that the Ritz norms ]1Vmu]]and ]] W,u ]] are not ‘small’, and that the norms of the Lanczos vectors
11 v,,,+ II and 11 w,+ I 11 are not ‘big’.
With these assumptions, wc have by Eqn(4.16) that the dependence of the residual norm upon the particular eigenvalue p being considered is centralized in the & ( p ) term in the denominator of this expression. Whenever this term is pathologically small, this residual bound will be large and we expect such an eigenvalue to be extraneous. This is in fact what we see in practice. Any simple eigenvalue which is both an eigenvalue of the Lanczos matrix T, being considcrcd and of its corresponding submatrix T, is “spurious” and caused by the losses in biorthogonality of the Lanczos vectors. Such eigenvalues should be discarded and not used as approximations to eigenvalues of A. This is o u r identification test.
IDENTIFICATION TEST. For any Lanczos matrix generated using Eqn(3.2), a simple eigenvalue fi is classified as “spurious” and discarded from consideration, if it is also an eigenvalue of the associated submatrix T, of order m-1. Eigenvalues which are not “spurious” are accepted as approximation5 to eigenvalues of the original matrix A.
We now state our proposed Lanczos procedure for computing eigenvalues of nonsymmetric matrices. We will then present a sequence of numerical examples to illustrate its performance. Error estimates for the computed eigenvalue approximations are obtained by computing the corresponding Lanczos eigenvector components u(m) for each isolated eigenvalue p. Numerically-multiple eigenvalues of any Lanczos matrix are assumed to be good approximations lo cigcnvalues of the given matrix A, in the sense that each is a n eigenvalue of some matrix close to A. A PRACTICAL LANCZOS PROCEDURE FOR NONSYMMETRIC MATRICES
STEP 1.
Given a diagonalizable matrix A, generate a complex symmetric tridiagonal Lanczos matrix T,,
using the Lanczos recursion given in Eqns(3.2) choosing
T
v, = w l and v , w l = 1 .
STEP 2.
For some m 5 M compute eigenvalues of T,. eigenvalues. Label these as “good” eigenvalues.
Combine numerically-multiple
J. Cullum and R.A. Willoughby
212
h
STEP 3.
Compute the eigenvalues of the submatrix T, corresponding to T,,,. Following the identification test given above, use these eigenvalues to identify any “spurious” eigenvalues of T,,,. Discard any spurious eigenvalues. The remaining eigenvalues
of TI,,are accepted as “good”; that is, as approximations to eigenvalues of the original matrix A.
STEP 4.
Compute error estimates for each simple “good” eigenvalue using inverse iteration on Tnl. If convergence is observed, terminate the computation. Otherwise enlarge the Lanczos matrix T,,, and repeat Steps 2-4.
This procedure does not use any reorthogonalization of the Lanczos vectors generated and Lhcrefore has small storage requirements.
5. RESULTS OF NUMERICAL EXPERIMENTS
This section contains what is perhaps the most important part of this paper, the results of numerical experiments on the procedure defined in Section 4, which were obtained using a variety of test matrices ranging in size from n= 104 to n=2961. These test matrices were obtained from Howard Elman at Yale, Y. Saad at Yale, C. Siewert at North Carolina State, and James E. Van Ness at Northwestern University. All of these test problems are real, nonsymmetric matrices. Therefore, the corresponding eigenvalues are either real or pairs of complex conjugates.
The figures in this section depict the convergence achieved for several typical examples. Each figure consists of 2 sets of data. The two symbols members of the two sets. In each case the symbol
> and + are used
> is used
to distinquish the
to denote the data points corre-
sponding either to the eigenvalues computed using the corresponding sequence of EISPACK subroutines BALANC, ELMHES, and HQR or to the eigenvalue approximations obtained by using a larger size Lanczos matrix. The sizes of the Lanczos matrices used are given in the y-axis label. An overlapping
> and + symbol denote “pictorial” convergence.
With the scales we have
wed in thew pictures, we cannot depict the detailed convergence. Therefore, in the discussion o f each test matrix, specific comments are made about the actual error or error estimates ob-
tained lor some of the eigenvalue approximations.
The real part of each eigenvalue or
Computing Eigenva1ue.c of Large Sparse Norisymmetric Matrices
213
eigenvalue approximation is plotted along the x-axis and the imaginary part is plotted along the y-axis .
The first class of matrices we considered was obtained from Y. Saad, who in turn extracted them from Stewart [ 181. These matrices represent transition matrices of Markov chains which describe random walks on an ( n + l ) x ( n + l ) triangular grid. The matrix of interest is the matrix whose entries are the transition probabilities of passing from a given node to one of its nearest neighbor nodes. For details on the matrix construction see Stewart [ 181. For each such matrix, 1 is always an eigenvalue.
Of interest in the application is the computation of the left
eigenvector corresponding to the eigenvalue 1. The components of this vector (appropriately scaled) give thc steady state probability distribution of the chain.
In this paper we look only at eigenvalue approximations. However, eigenvector computations can be done in a reasonable fashion once the eigenvalues have been computed and they will be discussed in a separate paper. We consider 3 test matrices in this family, SAAD190,
SAAD496, and SAAD1891. The dimensions of these matrices are respectively, 190, 496 and 1891. For SAAD190 we also used the EISPACK subroutines to compute the eigenvalues. Therefore each figure for SAADl90 contains the eigenvalue approximations which were computed using our nonsymmctric Lanczos procedure and the eigenvalues obtained from
EISPACK. For the two larger matrices we have only our Lanczos results. All of the eigenvalues of SAAD I90 are real and in the interval [ - 1 11. The nonzero eigenvalues occur in f pairs. There is also a zero eigenvalue which has a multiplicity of 10. The nonzero eigenvalues with smallest magnitude are h = f ,012706. The gaps between neighboring eigenvalues range from 1.06
x
at h = 2 .051284 to ,0446 at h = f ,93755.
Wc first consider the spectrum near 1. (See Figure 1 ,) By m=30 (a Lanczos matrix of order
30), we have obtained the eigenvalues & 1 to 6-1/2 digits and the eigenvalues
.98 to 4-1/2
digits. Pictorially we see I and .98 as ‘converged’ eigenvalue approximations. That is, the the
>
symbols overlap symmetrically.
+ and
By m=60, we have obtained the 3 pairs of largest
eigenvalues to all digits and the next 4 pairs to 8-1/2 to 9-1/2 digits. Thus, for this problem the extremes of the spectrum are not dilricult to obtain.
J. Cullurn and R.A. Willoughby
214
In considering SAAD 190 we actually looked at the convergence of all of the eigenvalue approximations, not just at the convergence near f 1. Figures 2-7 illustrate the convergence achieved on two interior sections of the spectrum. First consider the interior interval [.3, .7].
In Figure 2 for m=90, we see ‘approximations’ to each ‘cluster’ of eigenvahes on this interval. However none of these approximations has converged pictorially. In Figure 3 for m=150 (the Laiiczos matrix is approximately three-fourths of the size of the original matrix), we see that ‘pictorially’ all of the eigenvalue approximations in the subinterval [.46, .7] have converged. By m=300 (and probably before but we did not examine any T,, for 150 < m
< 300). all of
the
eigenvalue approximations on this interval have converged pictorially. In fact this convergence is genuine and the errors for each of the eigenvalue approximations computed in this interval are
all less than 6
x
lo-”. The sizes of these errors were obtained by directly comparing the
eigenvalue approximations with the eigenvalues computed by EISPACK. Similar convergence is obtained on the interval [.67, 1.01.
For this particular example the most difficult part of the spectrum to compute was near the middle section [.045, ,151. Several of the eigenvalues with the smallest gaps are in this interval. Figures 5-7 illustrate the convergence achieved for the larger interval [ -.05, .3]. Figure 5 depicts the degree of approximation achieved using a Lanczos matrix of size m= 150. At this size nothing in this interval has converged. To conserve space, in Figure 5 the limits on the imaginary axis were set at
.01. There are however two eigenvalue approximations in this interval whose
imaginary parts are outside this range and thus they do not appear in Figure 5 . Their real parts are in the interval [.15, . 2 ] . Figure 6 depicts the eigenvalue approximations obtained o n the same interval but using T300. At this size the eigenvalues with the smallest gaps and sizes are not well-approximated. Figure 7 gives this same part of the spectrum but corresponding to a Lanczos matrix of size m=400. At this size all of the eigenvalues in this section (and in fact in all other sections) of the spectrum are well-approximated (to 10 or more digits) by the eigenvalue approximations obtained using the Lanczos procedure described in Section 4. Observe that the Lanczos matrix required to obtain this degree of approximation is slightly more than twice the size of the original matrix. SAAD190 is not a difficult problem for the Lanczos procedure because the so-called gap stiffness, the ratio of the largest gap between neighboring eigenvalues to the smallest such gap is reasonable, approximately 450. We will consider a more difficult problem, VANNESS, below.
Computing Eigenvalues o f Large Sparse Nonswnrnetric Matrices
215
SAAD190 was considered in detail so that our Lanczos procedures could be compared with EISPACK. However, what is of interest here is the behavior of our procedure on large matrices. Therefore wc consider two larger matrices but only look at that portion of the spectrum which is 0 1 interest in this application. Figure 8 depicts the convergence of the Lanczos approximations near X = 1 for the Markov matrix SAAD496. The plot is of the “good” Lanczos eigenvalues corrcsponding to Lanczos matrices of size SO and of size 300. We see that at rn =
SO the three largest eigenvalue approximations have stabilized pictorially. The approxiniation to X = 1 has converged to 7 digits and the one to h = - 1 has converged to 6-1/2 digits. At
m=80 these approximations have converged to 10-1/2 digits, and the approximations & .99346. & ,97549 and the next pair have stabilized to respectively, 9 digits, 8-1/2 digits, and
5-1/2 digits.
Figures 9-10 depict the corresponding convergence (but at the other end of the spectrum) for the large matrix SAA1891 of order 1891.
Here the comparison is between ‘good’
eigenvalues computed at m=90, 180, and 300. In Figure 9 we sec that at m=90, 4 of the eigenvalue approximations are converged “pictorially.” The approximation to h = 1 has coiiverged to 7-1/2 digits and that to h = -1 has converged to 6-1/2 digits. The approximating pair 2 ,99839 has stabilized to 5-1/2 digits, the pair f .9937 to 4-1/2 digits and the next two pairs to 3-1/2 digits. The corresponding values at m = 180 (see Figure 10) show stabilization of the first 5 largest approximating pairs to 10-1/2 digits, the next 4 pairs to 8-1/2 digits, the next pair to 6-1/2 digits, and the next few pairs to 4-5 digits. Thus, a very reasonable size Lanczos matrix can be used to compute these eigenvalues instead of the original very large matrix.
The second class of matrices we consider are matrices supplied by Howard Elman of Yale. These matrices are obtained by discretizing the following partial differential equation.
where P and y are real scalar parameters used to control the degree of nonsymmetry of the matrices generated. The right-hand side f is chosen so that with Dirichlet boundary conditions the solution is
J. Cullurn and R.A. Willoughby
216
u(x, y) = xe
An nx
x
XY
.
sm(ax) sin(ay).
(5.2)
ny grid is placed on the square and the discretization is obtained by using central
differences for all of the derivative terms. Elman suggested values of
/3 and of y between 0 and
250. For a given matrix the objective is to estimate those cigenvalues with the largest real parts and to determine whether or not there are significant gaps in the spectrum.
Three test problems are discussed.
These are ELN900, ELN4763A, and ELN4763B.
ELN900 corresponds to setting nx = ny = 30, /3 = 20, and y = 0. ELN4763A and ELN4763B are both of size n = 47x63 = 2961. ELN4763A corresponds to setting p = 5 and y = 50. ELN4763B corresponds to setting /3 = 0 and y = 250. All three matrices were generated using code supplied by H. Elman.
Because the test matrices are real, the “converged” eigenvalue approximations must be symmetric with respect to the real axis. However the eigenvalue approximations computed using any Lanczos matrix do not necessarily possess this symmetry. In fact true symmetry appears to occur only once convergence has occurred, and this allows us to mark or track the convergence of pairs (i.e. approximations to conjugate pairs of eigenvalues) by comparing the imaginary parts
of the members of an approximating pair. The eigenvalues of our Lanczos matrices cannot be
expected to have any inherent symmetry w.r.t. the real axis because these matrices are complex symmetric not real.
Figures 11-24 depict our Lanczos computations for ELN900. No EISPACK computations were done for this matrix. Figures 11-16 track the convergence at the upper end of the spectrum, the convergence of the eigenvalues with largest real parts. The Lanczos computations tell us that this matrix has both real and complex conjugate pairs as eigenvalues. The real parts of
the eigenvalues range from ,1735 (which is a real eigenvalue) to 9.443. The imaginary parts are in the interval [ -2., 2.1. The eigenvalues with largest real parts are 9.442873 ? 1.729036 i. The spectrum of ELN900 is of course symmetric w.r.1. the real axis since the matrix is real. Since no EISPACK computations were performed, each of these figures consists of the eigenvalue approximations obtained at the specified size Lanczos matrix together with the corresponding “good” eigenvalues obtained using a Lanczos matrix of size m = 300. In each figure the symbol
Computing Eigenvalites of Large Sparse Nonsymrnetric Matrices
+ denotes symbol
217
an eigenvalue approximation obtained using the smaller Lanczos matrix, and the
> is used to plot the approximations obtained with the Lanczos matrix of size m=300.
For m=30 (see Figure I l ) , the eigenvalue approximations to 9.44287 f 1.729036i have stabilized to 4-1/2 digits. If we increase m to 60 (see Figure 121, we see that the eigenvalue approximations on the leading edge of the spectrum have converged “pictorially”. As we continue to enlarge the Lanczos matrix from m= 9 0 to m=180, see Figures 13-16, we see more and more approximations stabilize. Observe the
+ symbols moving in on the > symbols as
m is
increased. Convergence appears to occur from the outside edges of the spectrum. Observe that there is no genuine symmetry for the
>
points. This is to be expected because the
>
are
eigenvalues of Lanczos matrices. Note however, that the pictorially stabilized eigenvalue approximations are symmetric with respect to the real-axis (as they must be if they are correct).
Figures 17-24 depict the corresponding convergence at the other end of the spectrum. Again these are comparisons of the eigenvalues of Lanczos matrices. The convergence observed at the small “end” is similar to that observed at the large “end” of the spectrum in the sense that the extremes of the spectrum converge first. The convergence on the interval [.1735, 1.31 is however considerably slower than that at the large end of the spectrum. At m = 6 0 (See Figure 17), the approximation to the smallest eigenvalue has stabilized to only 3 digits. We can see other eigenvalue approximations moving in. At m = 90 (See Figure 18) the approximation to the smallest eigenvalue has stabilized to 6.5 digits. At m = 150 (See Figure 19) this approximation has stabilized to 11 digits, the approximation to the next pair has settled down 8-1/2 digits and is followed by an approximation to a real eigenvalue which has stabilized to 5 digits and a n approximation to a pair which has also stabilized to 5 digits. At m=l8O (See Figure 20) the approximation to the smallest eigenvalue has stabilized to 1 I digits, the ones to the smallest pair have stabilized to 9-1/2 digits, the next 3 have stabilized to 7 digits, and the next pair to only 2-1 /2 digits.
Figures 21-24 give the convergence of the approximations at the small end in detail. We should note that the convergence observed o n all these problems is the stabilization of the ‘good’ eigenvalues of the Lanczos matrices as we increase the size of the Lanczos matrix being considered. Recall that all of the eigenvalues which are not labelled ‘spurious’ by our identification test are labelled ‘good’. As the discussion of error estimates given in Section 4 indicated, such
J. Cullurn and R.A. Willoughby
21 8
stabilization is no guarantee that the eigenvalue approximations being computed are accurate reflections of the eigenvalues of the original matrix. We expect however that if the condition numbers of the eigenvalues we are trying to compute are reasonable, then these approximations which we are generating are in fact good approximations to the quantities we want. This has been confirmed numerically for those test matrices where EISPACK computations have been done.
For the matrices ELN4763A and ELN4763B we look only at that end of the spectrum corresponding to those eigenvalues with largest real part.
Each of these matrices has order
n=2961. For these examples the values of the “good” Lanczos eigenvalues obtained using m=40 and m=60 are compared with the corresponding “good” Lanczos eigenvalues obtained using m=100. In Figure 25 for ELN4763A we see that by m = 40 the approximation to the eigenvalue with largest real part has converged “pictorially”. In fact it has stabilized to 6 digits. 4 Its crror estimate is approximately 10- . The next largest has stabilized in 2 digits. By m =
60 (See Figure 26), the approximation to the largest has stabilized to 12 digits with an error estimate 4
x
lo-’.
The second largest has stabilized to 8-1/2 digits, the third to 5-1/2 digits,
the 4th to 4 digits, the 5th to 3 digits and the 6th to 2 digits. The computations indicate that the eigenvalues of ELN4763A on this end of the spectrum are probably all real.
Similar computations for ELN4763B indicate a much more complicated eigenvalue configuration. (See Figures 27 and 2 8 ) . At m = 40 again we have that the approximation to the eigenvalue with largest real part has converged “pictorially”. Here however the eigenvalues are definitely not real. They have large imaginary parts. This is not surprising since choosing /3 = 0 and y = 250 corresponds to making the test matrix as nonsymmetric as possible. By m = 60, 3
pairs of eigenvalues have been approximated pictorially. The approximations to the first
pair have stabilized in the first 4-1/2 and 5 digits. The approximations to the second pair have stabilized in the first 3 and 4 digits. Those for the next pair have stabilized in the first 2 and 3 digits. In each case the member with the positive imaginary part has stabilized slightly less quickly.
If one continues to enlarge the Lanczos matrices more and more eigenvalues
“converge”.
We looked at SAAD190 in detail. We now do similar computations on a much more difficult eigenvalue problem, VANNESS. (See Figures 29-33). This matrix was obtained from J. Van
Computing Eigenvalues o j Large Sparse Nonsymmetric Matrices
219
Ness and was extracted from a power systems analysis. The objective is to determine those cigenvalues with positive real parts. The order is only n= 177 so that the eigenvalues obtained using our Lanczos procedure can be compared directly with those obtained using the sequence of EISPACK [ 8 ] programs BALANC, ELMHES, and HQR. The real parts of the eigenvalues
of VANNESS range from -200. to 5.86. The imaginary parts are contained in the interval [ -60, 601. The majority of the eigenvalues are concentrated near the imaginary axis. See
Figure 29 which is a plot of the eigenvalues computed by EISPACK together with the Lanczos approximations obtained when m = 50.
The eigenvalue gaps range from 3.4
h = ,0996124 and h = ,099646 to 139.1 at h = -200.
x
at
Therefore the gap stiffness for this
matrix is approximately 4 million.
VANNESS has 7 eigenvalues with positive real parts.
They are .00926 f 5.39561
,0504408 ? 5.796151, 5.86125, and ,04688 f 6 . 0 5 . At m = 50 (See Figures 29 and 30) the eigenvalue with largest positive real part h = 5.86 is estimated to 7 digits. The 12 largest eigenvalues on the other end of the spectrum from h = -200, through h = -.02657 f 17.451 are all approximated to 7-1 / 2 or more digits by the Lanczos eigenvalue approximations obtained at m = 50. The blurred portions of Figures 29 and 30 correspond to very dense (on the scale used in these figures) concentrations of eigenvalues. At m = 100 (See Figure 31) the ‘edges’ of the spectrum have converged. These plots are somewhat inaccurate because of the difficulty the graphics program had in depicting points which are very close together. However the overall pictures are basically correct.
Figures 32 and 33 illustrate the convergence of the Lanczos approximations to those eigenvalues of VANNESS which have positive real parts. The two real eigenvalues -.0996124 and -.099646 are depicted as one point. Observe that at m = 200 the approximations to eigenvalues with large magnitudes have converged “pictorially” but none of the approximations to eigenvalues with positive real parts, other than for the eigenvalue h = 5.86, have stabilized. (See Figure 32.)
A larger Lanczos matrix must be used. Figure 33 depicts the corresponding situation when m = 600. For this size all of the eigenvalue approximations have stabilized except the 2 close eigenvalues h = -.0996124 and h = -.099646. Our nonsymmetric Lanczos procedure yields
J. Cullurn and R.A. Willoughby
220
the value p = -.09956.
M = 600 is approximately 3.4n since n = 177. These tests demon-
strate that our nonsymmetric Lanczos procedure can handle very difficult spectrums.
The last set of matrices we considered was supplied by Chuck Siewert of North Carolina State. These matrices have double eigenvalues. Moreover for the example considered here, the cigenvalues occur in groups of 4 and within each group there are two eigenvalues which are very close together. For example the eigenvalues h = ,9995883, ,99903237, ,99903158, and ,9982447 form such a group. With the scales we used in the figures, each of these groups of 4 is plotted as a group of 3 eigenvalues. In this sense the pictures are somewhat misleading. Most of the eigenvalues have magnitudes less than 1 .O. See Figures 34-43.
The particular test problem we considered is CHUCK60C which is of order n=656. Siewert’s initial objective whs to compute those eigenvalues with magnitudes greater than 1. Again however in order to obtain a better understanding of the convergence achievable by our Lanczos procedure, we looked at the convergence in other parts of the spectrum.
The
BALANC, ELMHES and HQR subroutines from EISPACK were used to compute the eigenvalues of CHUCK60C so that we could compare our computed “good” Lanczos eigenvalues with them to confirm the observed convergence.
The largest eigenvalue of h = .000090268.
CHUCK60C is X = 5.5 and the smallest eigenvalue
All of the eigenvalues have positive real parts. The gaps between distinct
eigenvalues range from 4.4 for h = 5.5 to 1 . 1
lo-’ at .000093. We trace (with figures) the
x
convergence of the eigenvalue approximations on two interior intervals of the spectrum [.99, 1.211 and [ S , ,621. We also trace the convergence of the eigenvalues with the smallest magnitudes; that is, those on the interval [O., .02].
The eigenvalues on the interval [1.21, 5.61 converge very quickly. By m=30, all 4 of our eigenvalue approximations agree with the corresponding EISPACK values to 1 1 or more digits. The largest eigenvalue appears as a double eigenvalue of this Lanczos matrix. Thus, we focus o n the interval [.99, 1.211. CHUCK60C has 22 double eigenvalues in this interval but only 7 of these eigenvalues have magnitude greater than 1.0. At m = 30 (See Figure 34) the real
eigenvalue h = 1.97994 is approximated to 7-1/2 digits, and we can see approximations for the cigenvalues X = 1.081 and h = 1.1375
4.98
x
lop4. At m = 60 (See Figure 35), the four
Computing Eigenvalues o f Large Sparse Nonsymmetric Matrices
221
largest eigenvalue approximations on this interval agree with the EISPACK eigenvalues as follows: h = 1.97994 to 11-1/2 digits, h = 1.11395 f 4.98
x
h = 1.081 to 11-1/2 digits, and h = 1.025 to 5-1/2 digits. h = 1.00488 2 3.565
x
to more than 12 digits, The other two eigenvalues
are just beginning to be approximated. Observe from this figure
that there are many eigenvalues near 1 and these are difficult to resolve. By m = 90 the 5th largest eigenvalue has been resolved to 9-1/2 digits, but the 6th and 7th ones on this interval are still not well approximated. At m=150 (See Figure 36), the 5th eigenvalue approximation is good to 12 digits and the 6th and 7th ones have been approximated to 5-1/2 and 6 digits. Pictorially, they have converged. By m = I80 this pair matches the EISPACK values to 8-1 / 2 digits and by m = 210 this match is to 10-1/2 digits. At m = 240 these eigenvalues agree with the EISPACK eigenvalues to 11 digits.
The many eigenvalues near 1.0 are not resolved by m = 240. However if we compare the EISPACK eigenvalues with those ‘good’ eigenvalues computed using a Lanczos matrix of size
m
=
656, then we see that eigenvalue approximations are obtained for all of the distinct
eigenvalues of CHUCK60C and that all of these approximations, including the ones on this interval, have errors less than 2
x
10-“’.
Now consider the interior interval [.S, ,621. See Figure 37. The successive groups of 4 close eigenvalues are displayed as groups of 3 because two of each four are very close to each other. As Figure 37 illustrates, with a Lanczos matrix of size m = 90, we do not obtain good approxi-
mations to any of the eigenvalues in this interval. By m = 150 (See Figure 38), we see rough approximations to each group. Observe that this behavior is similar to the convergence of conjugate gradients. We are picking up an average value for each cluster initially. As we continue to enlarge the Lanczos matrices we will see these values split and we get two approximations
per group. For larger Lanczos matrices these two approximations split into three and we finally begin to get good approximations to the individual eigenvalues in these groups. Figure 39 corresponds to m = 240. Observe that nothing has stabilized yet. If we enlarge to m = 656, then we see in Figure 40 that all of the eigenvalue approximations on this interval have converged “pictorially”. In fact these eigenvalue approximations agree with the EISPACK eigenvalues to less than 2
x
lo-”. Each of the four distinct eigenvalues in each of the groups of four
eigenvalues is resolved accurately.
J. Cullum and R.A. Willoughby
222
Finally we consider the convergence of the small end of the spectrum [O., .02]. At m = 90 (See Figure 41) there are no valid approximations. At m = 150 we get two eigenvalue approximations of the size .9
x
lop5but there is no convergence. At m = 240 (See Figure 42), we see
some pictorial convergence but this is deceptive because what one is seeing are average value approximations and not the individual approximations. CHUCK60C has 4 eigenvalues of size 9
x
lo-’ and 4 of size 8
x
The graphics program maps these 8 points into 4 points in the
picture and the spacing between the points drawn is distorted by the plotting routine. There are similar problems with the picture of the eigenvalues near .002 and near ,004. Thus, these figures are misleading. However, the routine is consistent and provides the same kind of plots on Figures 41-43. Thus, Figure 43 indicates that all of the eigenvalues on this interval have been approximated and this is correct. A comparison with the EISPACK eigenvalues demonstrates that the approximations are all accurate to 10 or more digits.
This combination of examples illustrates the behavior of our nonsymmetric Lanczos procedure. The characteristics of the observed convergence are very similar to those obtained with our real symmetric procedure. Eigenvalue approximations to eigenvalues with large magnitudes and good gaps are obtained easily. The relative location in the spectrum, the sizes of the local eigenvalue gaps, and the size of the gap stiffness ratio are keys to the convergence of the eigenvalue approximations obtained. These results indicate that our proposed procedure is capable of computing eigenvalues of a large variety of real nonsymmetric matrices.
REFERENCES Jane Cullum and Ralph A. Willoughby (1985), Lanczos Algorithms for Large Symmetric Eigenvalue Computations, Volume 1, Theory, Volume 2, Programs; Progress in Scientific Computing, Volumes 3 and 4, eds. S. Abarbanel, R. Glowinski, G. Golub, P. Henrici, H.O. Kreiss, Birkhauser, Basel, January 1985. B. N. Parlett and D. S. Scott (1979), The Lanczos algorithm with selective reorthogonalization. Math. Comp. 33, 21 7-238. William J. Stewart and Alan Jennings (1981), A simultaneous iteration algorithm for real matrices, ACM Trans. Math. Software, 7 ( 2 ) , 184-198. Y. Saad (1980), Variations on Arnoldi’s method for computing eigenelements of large unsymmetric matrices, Linear Algebra Appl., 34, 269-295. B. N. Parlett, D. R. Taylor and Z. A. Liu (1985), A look-ahead Lanczos algorithm for unsymmetric matrices. Math. Comp., 44, 105-1 24. J. R. Bunch and B. N. Parlett (1971), Direct methods for solving symmetric indefinite systems of linear equations, SIAM J. Numer. Anal. 8, 639-655. J. Wilkinson ( 1965), The Algebraic Eigenvalue Problem, Oxford University Press, Oxford, England.
Computing Eigenvalues of Large Sparse Nonsymrnetric Mutrices
223
[8] EISPACK, (1976, 1977) Matrix Eigensystem Routines, ed. B. S. Garbow, J . M. Boyle, J. J . Dongarra and C. B. Moler. Lecture Notes in Computer Science, 6 and 51, Springer, New York. 191 J . Cullum and R. A. Willoughby (1984). A Lanczos procedure for the modal analysis of very large nonsymmetric matrices, Proceedings of the 23rd IEEE Conference on Decision and Control, Dec 12-14, 1984, Las Vegas, Nevada, 1758-1761. [ 101 C. C. Paige (1972), Computational variants of the Lanczos method for the eigenproblem. J. Inst. Math. Appl. 10, 373-38 1 . [ I 1 1 J. Cullum and R. A. Willoughby (1985), A QL algorithm for complex symmetric tridiagonal matrices, IBM Tech. Discl. Bull. 27 (10B) March 1985, 6308-6313. [ 121 R. G. Gordon and T. Messenger (1972), in Electron-Spin Relaxation in Liquids, ed. L.T. Muus and P.W. Atkins, Plenum, New York. [ 131 NASTRAN User’s Manual (1977). NASA Langley Research Center, Hampton, Va. [ 141 B. N. Parlett and C. Reinsch (1969), Balancing a matrix for calculation of eigenvalues and eigenvectors. Numer. Math. 13, 293-304. [ 1.51 C. C. Paige (1976), Error analysis of the Lanczos algorithm for tridiagonalizing a symmetric matrix. J. Inst. Math. Appl., 18, 341-349. [I61 W. Kahan, B. N. Parlett and E. Jiang (1982), Residual bounds on approximate cigensystems of nonnormal matrices. SIAM J. Numer. Anal. 19,470-484. [I71 F. L. Bauer and C.T. Fike (1960), Norms and Exclusion Theorems, Numer. Math. 2, 137-141. [18] G. W. Stewart (1978), SRRIT - A FORTRAN subroutine to calculate the dominant invariant subspace of a real matrix. Technical Report TR-5 14, Computer Science Department, University of Maryland.
U
$
= 2 >
.008.. .006 __ .004.. .002 .. 0. ..
;-.002.. w
-
-.004..
-.006.. -. 000 .. u U f, -.01, lx
FIGURE 1. SAAD190, REAL PART, EIGVAL
J. Cullurn and R . A . Willoughby
224
-
.Ol-
CC
.008.. .OO6 __ .004.. .002 _.
=
2
5
2
-
W
+
+
+
0....>3>>
w
>>>
+ > > .>>$
>f
+'
> > > +>
+
-.004..
+
8
-.006.. Ll -. 008 .. a U E -.Ill,
a
.008 ..
W-
.006 ,004..
2
__
lx
+
.002 ..
; a
o...
>*>>
>
-.002 _. w - .004 .. -.006.. a: -.008-. a
a
U
% c
Whl hl
m
= 2 a g
>
a a El
> s p
+
-.002..
U
++
>+ > > >
w
+ + >>>
**
*3+>
* *
+
.... .
. . 3
3 %
-.01.
.008.. .006.. .004.. .002 _.
o... -.002 .. -.004..
-.006..
-.008..
-.ill,
>*H
*
*...*... ,, ... .... .* i.
225
Computing Eigenvalues of Large Sparse Nonsymmetric Matrices
U
z
.008..+ .006 .. .004 __ .002..
(L
U. m
i U
5
o...>>>>>> -.002.. -.a04
+ + +
+ >
+
+
+
a
= 2 Wm
-.006.. -.008..
U
a
g
f
+ -k
.006 ..
g
3 -
*>
.om..
5
=
>> >>>:>
-.01,
.004.. .002.. 0 . -..z+H*. -.a02 _.
7 Q
>> >> >>+>
+
__
k -.OO6.. g -.008.. a
+
>>>>>>>>>A>>>>=>>>
-.004..
-.Ill,
.
,,m,w
*.**.**
,,w 3 * .
**
33-
..
J. Cullurn and R . A . Willoughby
226
i .002..
U
’ E w
-.006
hl
.008..
.006 _.
& .004.. i .a02 .. U > 0. _. E
i
3%
3
3
*
> > > > >
> > > v
-.002..
-.004..
+
>>>
i
i
>
+
t
-.OM+ -.008
-.01 -1.01
-1.
-.99
-.98 -.97 -.96 -.95 -.94 -.93 FIGURE 9. SAA1891, REAL PART, EIGVAL
d
&
.004.. .a02 _. U > 0. -. -.a02 .. -.004.. -.006..
5
z
5
3
__
$
-
3
3
i>
-.008.. -.01
2
w
=
0. .. -.002.. -.004..
-.008.. -.a1
. 3
3
3
*
* * 3 3
.
-.92
-.91
i
3
3
-
,>p
+
Computing Eigenvalues of Large Sparse Nonsymmetric Matrices
+>
+
1.5 >
>
> >
>
> >
>
>
>
>
> >
>+
>
> > >
> >
>
>
>
>
>
> + >
>
>
>
227
>
> > >
>
>
+
>
>
+
9 3>
> > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > +> >
+
>
-1.51
>+
+ -2.1 5.
1.5
5.5
II+
6.
3
6.5 7. 7.5 8. 8.5 9. FIGURE 11. ELN900, REAL PART, EIGVAL
>+
+
.
>
>
3
%
>
,
10.
3
5
>+
9.5
3
>
>
+ +
>
>,>+ >
>
>
>+
>
+> >
'
>
>
>
>
> + >
+ -2.
+
>
5
3
P 3 3
3
10. I
5.
5.5
6.
6.5 7. 7.5 8. 8.5 9. FIGURE 12. ELN900, REAL PART, EIGVAL
9.5
J. Cullum and R.A. Willoughby
228
2. 3
1.5..
8 M
1. __
m
9; t a
3
> + >
f
”
>
>
>
> >
>
>
> > >
a
-.5.,
a
-1.__
>> >
n
z
>
> >
+
3.
> + +
> > > ++ > >+ > 3 > 7 > > > >+ > > > > > > > > + ’ > > > * $ * -b 3
3
-1.5..
3
9
*3>
0 . ..
W
&
*
>
> > >
_1
=E.
*
3
> + > + >
3
> > > > > > >
.5
3
3. 3
* 3
-2.
5.
5.5
6.
6.5
7.
7.5
8.
8.5
9.
9.5
I
10.
3
i
-b
> >
> >
>
+
P
>
>
>
>
>
*
>
>
>
> >’
*
*
+>+
>
3
>+ >
> +>
+,
d-.
*
3. 3
-2.
* 3
3
10. 1
5.5
6.
6.5 7. 7.5 8. 8.5 9. FIGURE 14: ELN900, REAL PART, EIGVAL
9.5
229
Computing Eigenvalues of Large Sparse Nonsymmetric Matrices
2.T 1.5.. m
1. ..
m
K! m
vl
... r ))
J
U
5 M
w + [Y
a a
.5
M
*
*
*+
0.
> > >>> > +’> +> +> +> ..
3
;>
>
>
*
3
+
> +
3
’+,
>>
+*t
*
,*
3
-1.5.. -2.
9
3
>>’ $ , +
-.5., -1.
>
>
-.
3
* +
>
*
,
*
+ + -5 > > > + > > > > > >+ > > > > >> > > f
U
g
*
*
*
3
3
*
3
3 3
3
3 3
3
_1
2. 3
1.5 Q
m
K! 8)
sz -1
U
>
zw
3
*
1.
m
2
*
* >+
>
>
.5
+
+
*
a3
>+
+
0.
>
> >
2 -.5 w
’+*
U
a a
u
z
-1.
3
-1.5 -2.
3
5.5
6.
6.5
7.
7.5
8.
8.5
FIGURE 16. ELN900, REAL PART, EIGVAL
9.
9:5
lb.
J. Cullurn and R.A. Willoughby
230
2.
3
1.51
>+
.$
+
>
+
a
-1. ..
=
f;
0.
>
+
>
>
.2
.8
.4 -6 FIGURE 17.
1.
> >
>
> >
1.2
>
>
> >
>
>
>
$
>
>
>
> > >
>
+
>
> > >
+ >
-1.5.. -2.
>
>
>
+ 1.6
1.4
I
2.
1.8
ELN900, REAL PART, EIGVAL
i?
3
3
1.5..
1. ._
2
M-
:
>
+ >
hl
+
.5..
+
+
+ +
f
>
i?
>
’
> > >
>
> > >
>
>
>
>
> >
>
2
U
’ E
0. _.
W
+ + +
>
> >
3
>
3
-1.5
t
-2.1 0.
3
> > > >+
> >
+
>
+
3
3
.2
.4
.8
.6 FIGURE 18.
1.2
1.4 1. ELN900, REAL PART, EIGVAL
1.6
I
1.8
2.
Computing Eigenvalues of‘Large Sparse Nnnsymmetric Matrices
2.
3
3
1.5 hl
E
s&
23 1
1.
3
3 3
3
3
>
+
hl
>
+>
.5
>
_1
U
=-
E W
0.
c. M
-.5
a
-1.
U
a m
>
>
’
>
-D
f,
3
3
-1.5 3
-2.
.2
.4
.6
FIGURE 19.
.8
1.
1.2
1.4 ELN900, REAL PART, EIGVAL
1.6
I
1.8
2
3
1.51
3
3
-2.
I
0.
.2
3
.4
3
.6 FIGURE 20.
.8
>>+
1.
+’
1.2
~
1.4
ELN900, REAL PART, EIGVAL
1.6
I
1.8
2.
J. Cullurn and R.A. Willoughby
232
m m
y
.3..
m
Y
t
.2..
ZE
-
_1
U
' z w
8 n I-
.1..
0. _.
>
+
>
-.1..
+
>
> >
-+
- . 2 ._ -.3..
U w ZE
-.4FIGURE 21.
.4
ELN900, REAL PART, EIGVAL
T
*x
> > >
-.l
I-
g -.2 n U
-.3
f; -.4
m m
.15
y
.3..
2
.2..
-
.I..
h)
E
-1
U 2-
z L a U
0. ..
.2
.25 FIGURE 22.
.3 .35 .4 ELN900, REAL PART, EIGVAL
+> 3
'4
-.1.. -.2..
a -.3 .. U x
-
-.4,
FIGURE 23.
ELN900, REAL PART, EIGVAL
.45
-5
Computing Eigenvalues of Large Sparse Nunsymmetric Matrices
03
N!
.3..
2
.2..
hl
x
-1
-
.I..
7
0. --
a
2 k-
c&!
a
3 3
3
-.1..
3
3
3
-.2..
d
w -.3.. a
x F+
-,4* FIGURE 24.
-&
ELN900, REAL PART, EIGVAL
+
m
s l
.1..
g.
>
.05.. a
2 W >
+ -
a
>
>
-1
> >>>>
0 . -.
>
-. 05 ..
*
>
>
> > >
+
P
3
>
IY
p.
-
z
-.1.. -.15, FIGURE 25.
2
hl
6 Y a
-1
2 W
=I
.15.1.. .05.. 0. __
; -.05.. a
> >
> >
a W
-.1_.
+ >
>
>
>
IY
n
ELN4763A, REAL PART, EIGVAL
+
+
>>t> *
9
33.
3
3
233
234
J. Cullurn and R.A. WiNoughby
>
+
>
> > > > >
+ +
>
>
> >
+
+
>
> >
>
-I2
>
>
>
>
t
+
+
-14 1
3.
3.5
4.
4.5
FIGURE 27.
5.
5.5 6. 6.5 7. ELN47636, REAL PART, EIGVAL
7 3
3.
7.5
, 8.
+
0, > >
-2 ..
>
+
> 61
2
> >
-4
61
>
cp x
-
-6
-1
U
__
>
-8 _.
U
2 fi
> >
IY
a
+
>
+
>
>
; W I-
+ +
-10..
+
> >
>
+
-12 __ -14
+
>
>
+
+
>
>+ > > +> > >> + + ' > +$> +> 3
3
~
3.
3.5
4.
4.5
5.
5.5
6.
6.5
7.
Computing Eigenvalues of’Large Sparse Nonsymmetric Matrices
235
60 3
50 __ 3
__ g 30 __ 2 w20 ._ m 40
Y 0
In
&
10 ._
2>
0..
L U
-20 _-
w
-30
*
u
;-10 __ 3
Q
' U
__ __
-40
3 3
-50 ._ -60 -
20
;
-200
-180 -160 -140 -120
-80
-100
-40
-60
T
-20
0
20
3
*
3
>
w
3
> > +
v
>>w>g> >
+
-20
1
-20
-1 5
-10
FIGURE 30.
-5
0
VANNESS, REAL PART, EIGVAL
10 I
5
J. Cullum and R.A. Willoughby
236
u IL
::
z
3
15 __ 10..
W
h)
m
5.3
3
*
3
3.3
%
3
+, :> +
>
* -2 0
-10
-15
-20
FIGURE 31.
-5 0 VANNESS, REAL PART, EIGVAL
+
E
> > )t
LI1
U
x
M
>
+ >
+
5
4
10
>>
H
3 .2 _.
1 .. 0
I
237
Computing Eigenvahtes of Large Sparse Nonsymmetric Matrices
. Oh -.l
-.08
-.06
-.04 FIGURE 33.
-.02 0. .02 .04 VANNESS, REAL PART, EIGVAL
4
.06
.08
.01 ,008 Y 0
.006
2 W
.004
U
a
m
Y
.002
2 >
0.
E
=
+
>
3
-.002
I-
w CII
-.004
-.a06 -.008
-.01
9
l.'Ol 1.'03
1.'05 1.’07 1.’09 1.’11 1.’13 1.’15 1.’17 1.’19 1.21 FIGURE 34. CHUCK6OC, REAL PART, EIGVAL
238
J. Cullurn and R.A. Willoughby
U
g
r
2 1
.008..
.OM.. .004..
.a02 ._ +
0.2m&
w
-.002..
U
-.OM..
a
-.01,
-
El
E
a
-
%
W
2
6)
= 2
6
3
3 3
f
3
3 3
3
-.004..
-.008 __
.008.. .006 __ + .004.. .002..
o..&
-.002.. -.004 __
+
-.OM..
-.008..
g -.a,
El
t
> >>
> >>
+ I Y
a a
> >>
> >>
> >>
> >>
+
-.01
I2 u
-.015
.5
.51
.52
.53 .54 .55 .56 .57 .58 .59 FIWRE 37. CHUCK60C, REAL PART, EIGVAL
.6
.61
.62
Computing Eigenvalices of Large Sparse Nonsymrnetrie Matrices
+ > >>
U
> a
a
> >>
> >>
+
+
6; -.005
+
>>.
239
-.Ol/
+
h
P-4
-.OK]
.5
.51
.52
.53 .54 FIGURE 38.
.55 .56 .57 .58 .59 CHUCK60C, REAL PART, EIGVAL
.6
.61
.62
+ X h
+
+> *>+
>+>>
+ a
g
H
-.0151
.5
.51
.52
.53 .54 FIGURE 39.
.55 .56 .57 .58 .59 CHUCK60C, REAL PART, EIGVAL
.6
.61
.62
.6
.61
.62
... -
-.015
15
.51
.52
.53 .54 FIGURE 40.
.55 .56 .57 .58 .59 CHUCK60C, REAL PART, EIGVAL
J. Cullurn and R.A. Willoughby
240
Wbl 4;
.006 .004
>
+
+
>
+
m a
r
-.oIJ
0.
+4
:: U
a
Wh)
*
=
.a02
.a04
.006
FIGURE 41.
.008
>
>
+
.a1
.012
.a14
.016
CHUCK6OC, REAL PART, EIGVAL
.a18
.01 .008 -006 .004 .002 3
a
-.004
a
a -.008 a a -.a11
E.
0 U
0.
=
2
0.
W_ (0
‘R
>
.004
.006 FIGURE 42.
.008
.a12
.01 .a14 CHUCK60C, REAL PART, EIGVAL
.Ol-
.008.. .006 .. .004.. .002..
2
.a02
= =
3%
3
2 -.002.. -.004.. -.006.. g -.008.. m a -.MA
t
FIGURE 43. CHUCK60C, REAL PART, EIGVAL
.016
.a18
Large Scale Eigenvalue Problems I . Cullum and R.A. WiUoughby (Editors) 0 Elsevier Science Publishers B.V. (North-Holland),1986
24 1
COMPUTING THE COMPLEX EIGENVALUE SPECTRUM
FOR RESISTIVE MAGNETOHYDRODYNAMICS W . KERNER Max-P lanck- Inst i tu t fur P lasmap hysi k Euratom Association, D-8046 Garching, Fed. Rep. of Germany
The spectrum of resistive MHD is evaluted by applying the Galerkin method in conjunction with finite elements. This leads to the general eigenvalue problem Ax = XBx. where A is a general non-Hermitian and B a symmetric positive-definite matrix. .4s this is a stiff problem, large matrix dimensions evolve. The QR algorithm can only be applied for a coarse grid. The fine grids necessary are treated by applying inverse vector iteration. Specific eigenvalue curves in the complex plane are obtained. By applying a continuation procedure it is possible by inverse vector it.eration to map out in succession complete branches of the spectrum, e.g. 4 resistive Alfven modes. for matrix dimensions of u p to 3.742.
1 . Introduction
Many problems in physics and engineering concern the oscillations and the stability of a given system, thus requiring evaluation of eigenvalues. The Schrodinger equation in quantum mechanics is a famous example of an eigenvalue problem where the energy levels are determined by a self-adjoint operator. In general. the physical model includes dissipation, and overstable or damped normal modes evolve which are described by complex eigenvalues. Such a nonHermitian eigenvalue problem arises in dissipative magnetohydrodynamics (MHD). The study of linearized motion has significantly contributed to the understanding of ideal and resistive MHD plasma phenomena such as stability. wave propagation and heating. The most complete pict.ure is obtained by means of a normal-mode analysis.
In the context. of computational physics the characftiristics of the set of equations have he understood before t h e discretization can be set. up and complex eigenvalues can be evaluat,ed. In fusion test devices such as tokamaks or stellarators the plasma is confined by a strong niagnetic field with a specific toroidal structure (“magnetic confinement”). For details ahout fusion devices and about the MHD model we refer t o texbooks, such as Refs. /1,2/. The MHD model combines the fluid equations with Maxwell’s equations. Consequently, the plasma exhibits characteristics of a n ordinary fluid, such as sound waves, as well as special features due to the magnetic field. such as Alfven waves. The ideal, i.e. non-dissipative, MHD spectrum of normal modes has three completely different branches: the fast and slow magnetoacoustic waves and the Alfven waves. The frequency of the fast magnetoacoustic waves tends to infinity with 1.0
242
W. Kerner
increasingly shorter perpendicular wave structure. The sound and Alfven branches usually form continua with zero frequency as end point. Since the fusion plasma is very hot, the resistivity assumes small values. It is therefore appropriate to treat, the dissipation as a small pert,urbation to a self-adjoint operator. Point eigenvalues of the Hermitian system experience o n l y a small change, mostly damping proportional t o the resistivit,y. But the continua are drastically changed even for small resistivity and, in addition, new instabilities occur which cause the plasma t o break away from the magnetic field. The discussion so far has revealed that the problem has quite different spatial and temporal scales which require appropriate and efficient numerical techniques, and that fine grids are necessary for accurate numerical solution. In this respect our model is quite different from the Schrodinger equation, which solves only for a scalar wave function. Consequently, the paper is organized to address the different aspects involved, namely the physical interpretation, the numerical accuracy and the solution of the complex eigenvalue problem. The author has tried t o make the ma.terial presented interesting in various respects: The reader is encouraged to t.ackle a non-Hermitian problem himself by studying the method presented. Serondly, the propert.ies of the discretization are made evident hy discussing the numerical arciirary in t.errns of convergence studies using increasingly finer meshes and by interpret.ing the results, and. last but not least. t h e eigenvalue pattern found may h e regarded a5 attractive and exciting. It ha. been at,temptrd to discuss i.lie different topics somewhat independently. so t h a t readers can pick o u t the parts of most interest to them. While accurate and very efficient solvers are available for treating the self-adjoint eigenvalue problem - we refer to the excellent books of Wilkinson /3:4/, Parlett / 5 , ’ ; and Golub and van Loan / 6 / - the situation is much more difficult for non-Hermitian matrices. Let us assume t h a t the chosen discretization does not lead t o defective matrices. The QR algorithm can then be applied t o compute all the eigenvalues of the system / 3 , 4 , 7 / . However, this algorithm destroys the band structure of the matrices and produces full matrices. T h e storage and CPU time requirements eventually pui, a. limit on the dimension d of the matrices, e.g. on the CRAY-I of IPP Computer Centre d has to be less than 600. However, much finer grids are necessary i n the resistive MHI) model for finding the st.ability limits and scaling properties. Inverse vector it.eration is a very efficient method of computing selected eigenvalues and eigenvectors of general matrices. It preserves the band st,ructure and thus allows very large matrices t o be treated. The slow convergence, which is sometimes considered a severe drawback, is st.rongly improved by a suitable complex shift. Fast convergence is found by restarting the ii.eration with a new shift.. With a continuation procedure in a relevant parameter, only a few shifts are needed to obtain the result.. In the case of Hermitian matrims Sylvester’s theorem (see, for example, Ref. /5/) yields the number of eigenvalues in a given rea.1 interval, and every desired eigenvalue can be found by the bisection method. A generalization of this theorem for general matrices does not, exist, and therefore inverse vector iteration cannot be used as a black box to compute all the eigenvalues in a given complex domain. Hut this also holds for subspace il.eralion /8,’ or for the Lanczos algorithm /9,10/. In practice, we did not find this drawback very rest,rict.ive;the results obtained from a coarse mesh by means of the QR algorithm or from a fine itiesh by continuation provide a good guess a t a suitable shift. Inspection of details of t>heeigenfunctions, such as the nuniber of radial oscillations, even makes it possible to comput,e
The Complex Eigenvalue Spectrum for Resistive Magnetohydrodynamics
243
eigenvalues in a certain domain of the complex X - plane successively. The matrices of the complex eigenvalue problem have a characteristic block-tridiagonal structure, this being the opposite of random initialization. Large matrices with a dimension of up t o 3.742 are treated and extension to even larger matrix dimension is discussed. The paper is organized as follows: The physical model appropriate to simulating the plasma behaviour in magnetic confinement experiments is described in Sec. 2. Section 3 contains t h e numerical method. The Galerkin method used in conjunction with finite elements culminates in numerical solution of the complex eigenvalue problem Ax = XBx , wherr A is a general matrix and B is Hermitian and positive-definite. The solution of the eigenvalue problem is given in Sec. 4 . The algorithm of the inverse vector iteration is presented. The continuation procedure is introduced as defined by a homotopy method. The CPU time and storage requirements are discussed. The results covering the specific properties of the normalmode spectrum of resistive MHD are displayed in Sec. 5. The variety of t h e eigenvalue branches is stressed as well as their physical interpretation. Finally, Sec. 6 contains the discussion and ronclusion. 2 . Physical model
The objertive of cont,rolled thermonuclear fusion research is to derive nuclear energy from the fusion of light nuclei. such as deuterium and tritium. on an economic basis. At the high temperatures involved the gas is ionized; this ionized state of matter is called the fourth st,ate or the plasma state. In order to obi.ain a sufficient number of fusion processes in a hightemperature plasma with T > 10 keV, the product of the particle density and the confinement time should exeed t h e value given by the Lawson criterion.
Fig. 1: Tokamak shematic: A toroidal current is induced in t.he plasma, which acts as the second loop of a transformer. This current creates a poloidal magnetic field, which together with the main loroidal field establishes an equilibrium and which heats the plasma by ohmic heating.
W. Kerner
244
T h e concept of magnetic confinement utilizes the fact that ions gyrate around magnetic field lines, i.e. are tied t o the field. Naturally, many instabilities tend t o destroy favourable confinement configurations. In the tokamak device a toroidal current is induced in the plasma, which produces a poloidal magnetic field. Together with the toroidal field, this current yields an equilibrium and also heats the plasma. The principle of a tokomak is shown in Fig. 1. The ensemble of particles exhibits collective behaviour. The gross macroscopic properties are of special interest. The plasma is described in terms of single-fluid theory. The resistive MHD equations read in normalized, dimensionless form: equation of motion p ( - t V . VV)= - V P i3V
at
Maxwell - Ohm
dB ~
at
=
V
2. (V x
+ ( V x B) x B,
B) - V x (qV x B ) ,
adiabatic law
Maxwell
V.B-0. Here p denotes the density, v the velocity. B the magnetic field, P the pressure and 0 the resistivity; y is the ratio of the specific heats. Note that t h e assumption of incompressibility. C . v = 0. is not made. The adiabatic law is adopted for the equation of state since the dissipation, which is proportional to q , is considered t o be small. Since the resistive modes rapidly oscillate, the compressible set of equations is appropriate. The fast and slow magnetoacoustic waves are retained. These equations are now linearized around a static equilibrium characterized by = 0 and v, 7 0. The equilibrium is then determined by the equation
;Fj
VP,
=
(V
x
B,) x B,.
-
(5)
In straight geometry static, ideal equilibria can be interpreted as resistive equilibria if = 0, with the consequence thai. T ! ~ , , ? ~ , E , const. In toroidal geometry a resistive equilibrium is only possible with flow. i.e. v, I 0 . This flaw, however, is proportional t o q and hence very small. Here we t.ake t.he simplest approach of a constant resistivity ql, irisi,ead o f a constant E,. This simplification does not. constitute any restriction on unstable modes, since the resistivity decouples the fluid from the magnetic field in localized regions where the perturbation matches the field. But also the results for stable modes are o n l y insignificantly changed by using constant resistivity. This model thus gives the basic feature of resistive modes, since we are interested in phenomena which scale as q 3 / 5 or q 1 / 3 ( and as q’l’, like the resistive Alfven modes). For a circular cylinder the equilibrium quantities only have a n r-dependence. With the usual cylindrical coordinates 7 , 0 , z , the equilibrium is determined b\- the eauat.ion dt-‘, 1 d d - - B s - - ( r B e ) - B,-B, dr r dr dr
V x q ( V x B,)
7
~
The Complex Eigenvalue Spectrum for Resistive Magnetohydrodynamics
245
With two profiles given, eq. (6) can b e solved t o give the remaining one. A class of realistic tokamak-like equilibria with peaked current density j, and constant toroidal field is given by
B, T h e tonstant
j0
7
1.
is adjusted t o vary q ( 0 ) . wliere the safety factor is defined as
T h e ratio of the safety factor on surface and on axis is q ( a ) / q ( O ) = v i 1. For Y = 1 the profiles assume t h e form
B,(r) = 1,
Po(r) - 1 ’
To simulate a plasma - vacuum - wall syst,em. it is only necessary to give the resistivity in the “vacuum” a sufficiently large value and hence a small value for the current. The following separation ansatz is suitable for the perturbed quantities :
where X is the eigenvalue. T h e growth rate XR is then defined as the real part of A, i.e. 2x XR = R e ( X ) . With k = - defining a periodicity length, a tokamak with large aspect ratio
1,
is simulated. II corresponding 1.0 the t.oroida1 mode number; m is the poloidal mode number. In ideal M H U X is either real or purely imaginary, which leads to exponentially growing unstable modes or purely oscillating waves. With resistivity included, the frequency can become complex. T h e equat.ions for the pert,urt)ed quantities v , p and b read
Xb = V
i :
(V
x B,)
~
V x ( q V x b).
(13)
Thr divergence condition, eq. (4), For t h e perturbed field, O . t) = 0, is used to eliminate b8 provided rri j 0. The perturbed resistivity is set to zero, thus eliminating the rippling mode. 1~in;tlly.we discuss t h e boundary conditions. It is assumed that the plasma is surrounded by a perfectly conducting wall, which implies the following conditions at the wall :
W.Kerner
246
b,(a) = 0.
(14b)
For finitv resistivity in the plasma the Maxwell equations require that, the tangential component of the electric field vanish at the wall. This implies d - - ( b , ) , = , = 0. dr On the axis
7
= 0 all the quantities are regular
3 . Numerical method
The set of linearized resistive MHD equations is solved by the finite-element method. A state vector u which contains the perturbed velocity, pressure and magnetic field is introduced:
uT = ( ~ ~ . ~ ~ , ~ ~ . p . b ~ , b ~ ) . In order l o reduce the order of derivatives and to obtain the weak form, we take the inner product. of eqs. (11-13) with the weighting funct.ion v , which has to be sufficient,ly smooth. and int,egrat,e over the plasma volume. In the Galerkin method used here the adjoint function v satisfies the same boundary conditions as u. The linear operator in eqs. (11-13) is represented by matrices R and S with spatial dependence only, where in S only the diagonal elements are non-zero and R contains differential operators and equilibrium quantities. The set of equations then reads
R U = XSU.
(15)
The vector u(r) is a weak solution if for any function v(r) of the admissible Sobolev space satisfying the boundary conditions the scalar product satisfies
(Ru,~= ) X(Su,v) The components of u are approximated by a finite linear combination of local expansion functions or shape functions:
3 - 1
Higher-order elcments are used, namely cubic Hermite elements, for the radial velocity and field componen1,s u, arid b , , and quadratic finite dements for ~ 8 v,, , p and b,. This introduces two orthogonal shape funct.ions per interval, raising the order of the unknowns to 2N + 2, where N denotes the number of radial intervals. With this choice, the transverse divergence
can be made to vanish exactly in every interval. and the divergence of b as well. Note t,hat 711
~
rt', and
ti2
~
zwg.
It has been established that this scheme yields for the discretized
The Complex Eigenvalu e Spec tm m for Resistive Magn e tohydrody namics
247
spectrum a pollution-free approximation to the true eigenvalue spectrum. Only with this choice of t h e finite elements can accurate results, as presented later, be obtained. Details of t h e discussion of this discretization are found in Refs. / 1 1 - 1 4 / . The error introduced in the differential equations through the approximation for u k( r ) is orthogonal to every expansion function. T h e Galerkin method eventually leads to the general eigenvalue problem
For details of the numerical method we refer t o Ref. /15,’.
4. -
Eigenvalue problem 4 . 1 . Algorithm
Since the Q R algorithm produces full matrices, it can only be applied up to relatively small matrix diniensions; the resulting coarse discretization of the operator does not yield sufficiently a c c u r a k results. Therefore, a method has t o be found which preserves the structure of the matrices and allows a fine mesh. Wc c h o o x i i i \ c . I s c . tor iteration in conjunction with a cont.inuation procedure. A t first the idea of continuation is explained in the context of a mapping sequence and then the vector iteration is discussed. The general eigenvalue problem Ax =
XBX
(18)
has tlo be solved, t h e eigenvalue X and the eigenvector x of the expansion coefficients being, in general, complex. A is a general, non-Hermitian matrix and B is symmetric and positivedefinit.e. Since in our problem A and B are real matrices, the eigenvalues occur in complex conjugate pairs. A and B have block-tridiagonal structure with a bandwidth b = 48. The dimension of the matrices is given by d = 12N - 2, where K is the number of radial intervals and is usually quite large. In the algorithm presented the band structure of A and 8 , which usually occurs in a finite difference or fini1.e-element discretization, is preserved and utilized. It is assumed that the system (18) can be approximated continuously by a sequence
where A k and B k represent the discretization of the operator with increasingly finer mesh; the dimension of the matrices is
Inverse vector iteration (see below) is used t o solve the system (18b) for given k . The start, value is taken from the previous step = X k - 1 and the start vector x p ) is obtained from xk-1 by interpolation. The iterat,ion is terminated if the change in the eigenvalues is less than a given tolerance. Of course, any other parameter of the system can be used for continuation.
Xp’
W.Kerner
248
Then instead of t h e dimension of the matrices the elements themselves change. In this fashion knowledge of a relevant part, of the spectrum a.t one point in parameter space is used to explore new regions. 4 . 2 . Inverse vector iteration
The initial value A, is considered as a n approximation to the eigenvalue of the system (18), i.e.
+
X = A,
With t h e shift A,
(IAA,l << IX,i).
AX,
(19)
the eigenvalue equation reads
With x, as initial guess, new vectors xi are computed iteratively. Since the matrix A is in general non-Hermitian, a second sequence is defined by the lefthand eigenvect.ors, yielding for i = I .2.3 ..... the i ~ r a t i o i i
Ilere the asterisk denotes the Hermitian conjugate, and the bar the complex conjugate. In the nonsymmetric case an improved eigenvalue is determined from right-hand and left-hand eigenvec tors x, and y t by using the generalized Rayleigh quotient
AX, =
Y'(A
-
2
A,B)x,
Y: Bx,
The system of linear equations is solved by factorization. The shifted matrix is written as a product of triangular matrices:
A'
=
A
-
X,,B
=
L'u.
(24)
This f U decomposition preserves the band structure arid can tw performed efficiently. With the definitions r,-l
=
AAt-1Bx,-,
A X ,-,
=
-
S,-I
=
AA,-]By,
~~
- 1
the following equations are solved for pt and 9;:
=
AA,-l
Xt-].
(25a)
?,-I,
(256)
The Complex Ezgenvalue SpectrLim f o r Resistive Magnefoiiydrod~minics
249
a n d eventually new vectors x, a n d y, are evaluated by
ux, L.y7
Pi, clr.
~
If t h r decomposition of A ’ , eq. (24), IS used, t h e Rayleigh quotient assumes t h e form
where X I = Bx,,defined in eq. ( 2 5 a ) , is needed to co m p u t e t h e new vector ri = AX, X , . T h e iteration is t,erminated if t h e error is srnaller t h an a defined tolerance c. which has 1.0 be larger t h an the machine accuracy. This algorithm is a straightforward extension of t h e solver for a Hcrmitian matrix A given in Refs. /16,17). 4_ _. 3 . lmplement at.iori __ -
T h i s implementation of t h r algorithiri rnakes use of routines from t h e LINI’ACK library T h e matrices A a n d E are given a s INPI-T storrd in t h e usual band-matrix storage mode, so that the zrro elements outside t ~ h rb a n d s d o n o t occur at all. Next the shiftcd matrix A ’ is computed and fact,orized. Kote that A‘ is now considerrd cornplex. even if we begin with real matrices. In order to make full use of the fast execution on t,he CKAY-1 vrclor cornput.rr. ‘18;.
~
the LINPACK routines, CGBFA for factorization and CGBSL for successive solution of linear syst.ems, ar e used. T h e evaluation of Irft,-hand and right-hand vectors is achieved by using t h e same decomposition. T h e vector x,, is usually initialized by random numbers. T h e it,crat,ion i p terminated if t h e error is smaller than the desired tolerance t:
# 0. Termination is also forced if the eigenvalue i s not, monotonically approximated. in ordcr 1.0 avoid pathologic iterat,ion paf,hs. If convergence is achieved in t,he st ep i=m, the final eigenvalue is given by
for A,,
X
A,, t Ax,,
~
a nd the eigenvector by
x
~
x,,
Usually five to ten steps are needed for convergence. T h e niaxirriurn number of st ep s is chosen as,,,,z .: 20. It is found thai. a.n eigenvalur is obtainrd more easily by clioosing a new shift rather t h a n by performing more iterat ions. Tlie rriinirriuru arnouni, of storage rtxluirecl includes tlir matrices A an d B and t.he d e c o n position of A’ =: LU i.ogether with t.he vectors x, arid y t . In a.ddition, a work-spacr Tor t.he pivoting in the linear system has 1.0 br given. These storage requirements can easily be improved b y keeping only thc! minirnum data necessary for t h e algorithm in t h e fast. mernorq and by si.oring d a t a on disk.
W.Kerner
250
T h e storage-improved algorithm then works as follows: 1.
Compute matrix B and store it on disk B
2.
Compute matrix A and perform the shift during computation A' = A
3.
Factorize A'
CU
~
and store f and
U
-
X,B
on disk A .
4. Compute new vectors and keep x , , y l , p tand q, in the fast memory. 5.
Read in
C,21
or
B separately, if needed
This optimized version is a simple extension of the original one. Only one complex matrix in band-mat.rix storage mode is required in the fast memory a t any step together with additional work-space for the factorization with the dimension of the upper band width. Next. we estimate the CPIl time and t h e st,orage necessary for t h e algorithm. The number of operations to factorize a band matrix with bandwidth h = m e - m , and dimension d is
and the number of operations to solve the linear system
NL
%
3 / 2 . d . b.
For m iterations there are then
NI
=
NF
+
m N L zz 0 . 5 d . b ( b
+
3m)
(30)
operations required. Most of them are spent on the decomposition, which accounts for
of them. Fasf evaluation of rigenvalues is achieved. The CPL' time on a CRAY-1 is between Is and 10s. This allows efficient treatment of many caseb.
4_. 4 _Oui-of-core _ ~ - algorithm
The numerical approximation of the state vector u, comprising six components, by two orthogonal (cubic and quadratic) finite elements per interval yields 12 x 12 sub-blocks. Since each finit.e element interacts with its neighbours, a block-tridiagonal structure arises. In the band-matrix storage mode an overall bandwidth b 48 occurs. The dimension of the matrix is given by N of these 12 x 12 matrix blorks, i.e. d = 12 . N - 2, if the boundary conditions are taken into account. T h e first, and last blocks. which take care of t h e boundary conditions, are filled up t o t.he general block size 12; the matrix dimension then becomes d = 12 . N. The maximum t.ractablr matrix dimension is given by t,he available core, in our case d is roughly 4000, which
The Complex Eigenvalur Sprctrum for Resistive Magnrtohydrodj~narnics
25 1
is sufficient for most applications. It is, however, immediately seen t h a t the tractable matrix size becomes quite small if the bandwidth b increases. In the case where L Fourier components are included in the eigenfunction, which is neccesary if poloidal symmetry in the equilibrium is no longer given, the associated sub-blocks have 12 L x 1 2 L elements. The number of unknowns becomes d z 12. L . N and the number of elements inside the bandwidth n = b L . d = h . .V . L 2 . 12. For only 3 Fourier components the necessary core increases almost an order of magnitude. For such cases the algorithm has to take into account these sub-blocks of size 12 L. Again, the number of radial mesh points is large and hence the overall dimension is much larger than the block width b, = 12 L, i.e. d >> b,. It is therefore reasonable to organize the algorithm t o perform the operations successively with increasing radial label i, 1 5 i 5 N . Given the shifted matrix A' = A X n P , which is easily composed by using only sub-blocks of size b,, the factorization is performed hlockwise. A' is again decomposed into a product of triangular matrices (eq. ( 2 4 ) ) :
A'
=
A
x,,e
=
LU.
Note that the first and last, rows have only two blocks A : , , . The factorization can then be cliosen as
t o yield the following algorithm for computing the lower and upper triangular blocks L, and
c,: (31)
which is performed with the LINPACK routine. CGEFA. The evaluation of the quadratic systems requires solution of
by means of the CGESL routine. Event,uallg, t h e matrices I%'? are obtained by solving
L , W , - A ,,*,
1'-1.2...IV-l.
(33)
T h e factorization can t,hen be performed with just the three rows i-1, i and it 1. These subblocks composing the shifted matrix A' as well as its fact,orization are stored on disk. The st.orage c ~ be, reduced t o only three blocks kept, simultaneously in the fast memory and by making 'Iccess to others through 1 / 0 from disk if required. The organization of the pivoting is left o u t in this discussion. The inverse vector iteration with evaluation of new vect,ors x n , . y m r p mand qn, a t each step is logically organized like the storage-improved algorithm described above. These vectors of length d = 1 2 . L . N are partitioned int.0 N part,s and stored
252
W. Kerner
arrordingly. T h e matrix multiplication then involves three matrix sub-blocks acting on three vect,or fractions. If t h e efficiency of the 1/O from and t o disk is put aside, this out-of-core algorithm is more economic because fewer data are involved. Owing to t,he tria.ngular form of t h e L , and U , the evaluation of the blocks K, and M;, only calls for forward solution in contrast t o the usual forward and backward substit,ut,ion. For the decomposition of N rows of blocks with block size b, 7 12 L N p = (2 + 213 + 1)Nbs" = l l i 3 N b ~ (34) operations are required, whereas t h e LINPACK routine. CGBFA, involves
Furthermore, the number of operations for one iteration solving for a new vector x is given by
M,hich is smaller than the number for the LINP.4CK routine, CGBSL:
T h e final prrforniance depends, of course. on t,he degree of vectorization achieved. The ratio of CPI' time spent. on the decomposition versus that of m iterations is
Again, it is evident that most of the CPU time is spent on t h e fact,orization. Therefore, good performance is achieved by successive mesh refinement wit.h increasing dimension of the sysbem, d l < dZ . . . < d,. It is emphasized t h a t the evaluation of the liavleigh quotient
does not require any addit,ional I/O if these vector products are evaluated piecewise together with the evaluation of t h v corresponding vectors. Finally. an overall optimization of the algorithm requires a well-balanced ratio of CPU versus I/O operations. Since I!O operations have to be paid for, it, is not advisable for a fast. vector computer to work with few d a t a in the fast memory and to rely on many I / O operations. Opt irrial performance is achieved by partitioning the available core int,o two pieces a.nd by kcrping as many blocks in memory as fit in1.o one part. The following blocks are then read into the second part. Data transfer can be sped up by using different channels. The algorithm is tuned t o perform 1 / 0 during execution. T h e sparseness within sub-blocks can btl utilized to speed u p this algorithm further. I n particular: the symmetry and sparseness of P arc used to reduce the storage requirement for = B X , ~,. Results obtained b) F as well as to optimize the evaluation of t h e product X,means of this out-of-core algorit,hm will be given elsewhere.
253
The Complex Eigenvalue Spectrum f o r Resistive Magnetohj~drodynamics
5. Results T h e applications display the essential feat,ures of the resistive MHI) sp ect r u m . Moret,he results have to est,ablish t h a t the chosen numerical method, in particular t h e special discretization using higher-order finitr clcments, is a good approxirnat,ion of t h e exact. spcc-
crier,
t r u m . T h i s involves careful inspection of the computed result,s using convergence studies and also comparison with exact analytical findings based on asymptotic boundary laycr theory or W K R J analysis. 5.1. __ Ideal
MHD
T h e first, application is aimed, nat,urally. at, testing t h e performance of the new method by reproducing known results from ideal M H D . T h e entire sp ect r u m of a plasma column with constant toroidal magnetic field and corist,arit. toroidal current dei1sit.y is an int,eresting rase. T h e equilibrium is specified by u 0 i n eq. ( 7 ) , yielding a parabolic pressure profile an d a ~
C O I I ~ I U I s~ a k t y
factor q.
102
I
I
Fig. 2:
0) spect.rum Complct,e, ideal (71 : of t h e constant current equilibriiim (v = 0 iri eq. ( 7 ) ) . T h e square of the eigenvalues ( A = zw)is plotted versus t h e safety fact,or with n = l , ni--2 an d k k 0 . 1 . Three differen(. branches occur, namely fast magriet.oacoiistic, Alfven an d slow magnetoacoustic waves. Negative values for w 2 indicate exponelit ially growing instabilit.ies. The entirc spec-t,riirri is well resolved and n o spurious eigerivaluc~sdue, to numerical coupling of difTerent hrarichc~s occur, i.e. n o "pollution".
I!!! . . . . .I .I .IIIII . . . . .I.!!Ill . . . . I! . . I! . . .I .I.I!.!. . .I!!!!!! . . . . . . .!I . .. .. .. .. .. .. .. .. .. .. .. .. .............. j j j j j j j j i j j j j j j j j ...................
............ .......
10' I
., ! ! I # ' .!
,!i!!!i
-10'6-
-10"
flp
.. . . . . . .. ... ... .: ..; ;:::: _. . . . . . . .' . . ._. . .. .. .. .. .. .. .. . . . . .
-10-q 190
195
200
205
:
In Fig. 2 the spectrum is displayed as a function of the safc,ty factor. T h e s q u a r r of t l i ~ eigmfrequency ( A == zw) is plotted. positive valiies o f w' corresponding to st.able modes and n r g a t i v r values to exponent.ia1ly growing iinstable ones. Th r ee par1.s o f t h e sjwctruni cari lw
clear11 dist ingiiishctl. narriely the discrcte fast, modes, the Alfvm ~ n o d e s ,which for t,his ecluiIibriiirri f o r m a discrele set of modes. and t.he slow-mode cont,inuiirri. If nq is sufficiently closc to - in. thc A1fvc.n rriodes bccornc unstahlc. a.s c a n be sern from I.'ig. 2, a n d f o r nq - rn there, a r e irifinikly rrianv uiistat)lc~triodes. 0 1 1 1a p p r o a c h yields ai this point a5 rriaiiy ii~st~al~ilit~ies as
W. Kerner
254
correspond to the entire Alfven class, namely 1/3 of the spectrum modes. This result holds for all mesh sizes. T h e spectrum presented is in complete agreement with that of Chance el, al. 1191, indicating t.hat we can reproduce the spectrum without, “pollution”/l3,14/, especially t h e marginal points, in agreement with analytical results. I t is emphasized that. our results are obtained from a non-self-adjoint operator in conjunct,ion with cubic and quadratic finite elements, and those of Hcf./IS/ from a completely different self-adjoint operator in conjunction with linear and piecewise constant elements.
5 . 2 . Resistive inst.abilities
As pointed o u t in the introduction, point eigenvalues of ideal MHD, such as the fast modes o r instabilities, experience only a small change proportional to the resistivity. This property is verified numerically, but these resistive normal modes are not really of int,erest. With finite resistivity, the magnetic field is no longer frozen into the fluid. Such resistive instabilities are studied for realistic tokamak-like equilibria wit,h peaked current density and constant toroidal 1 in eq. (7), a.nd hence q(a)/q(o) - 2; it is given explicitly in eq. (9). We field with v concent,rate on the m = 2 mode. ]
o - 1~ A,
[
b
I
i
2.0
marginal point
0.
/ 21
2.2
nq(a1 3
Fig. 3a: Growth rate of the Iriost unstable mode for a bokamak-like current profile (v = 1 i n eq. ( 7 ) ) versus t,he safety factor on the plasma surface for 7 = arid n-1, m--2, k - 0 . 1 . The upper curve refers to the free boundary case r W / u = 1.5, a n d the lower curve to t h e fixed boundary r, = a . Fig. 3b: Growth r a k s of the two most unstable modes for the same c a w o n an enlarged scale.
The Complex Eigenvalue Spectrum for Resistive Magnetohydrodj~namics
255
T h e growth rate of the most unstable mode is plotted versus q ( a ) in Fig. 3a. If the wall is placed directly at the surface r = a, then t h e m :2 tearing mode is unstable for 2.20 5 q ( a ) :<4.0. If the wall is moved away from t h e plasma surface, the internal tearing 2 surface is locakd in the plasma, and i t becomes an mode is riristable as long as the q external kink if the q - 2 surface is in the ‘‘vacuum’’ region. For a finite distance of the wall r,/u = 1.5 this kink becomes stable for q ( n ) ~:1.61. l ’ h e s ~stability limits; obtained with the stability code of Kerner and Tasso ,120/, are indicated in t h e figure a n d are very accurately reproduced. For the internal tearing mode the growth rate becomes much smaller as q(a) approaches the value 2.20. Moreover, it. is particularly worthwhile t o take a closer look at. these instabilities on an enlarged scale near the marginal point. Figure 3b displays t h e growt,h rate of the two most unstable modes. On this scale t h e eigenvalues of the higher modes a r e located on the axis and are therefore omitted. T h e most unstable mode has the global structure for the perturbed field bl i r b , displayed in Fig. 4 . This perturbation has a finite value at t h e singular surface. The normal component of the velocity v1 = r v , is ‘more singular’ and hence more localized around T , and is characterized by one radial node. T h e second unstable mode shows one oscillation around t h e resonant surface in the field perturbation b l . T h e nornial velocity component 21, is very localized at r r.. and has two nodes in the resistive layer. one more than the first mode. I t is evident that these unstable modes represent a Sturmian sequence with increasing niimher of radial nodes inside the layer. Thc fundan~ental mode represenk a current-driven tearing m o d t which for q ( o ) 2.20 turns s m o o t h l y int,o a pressure-driven int.erchange. This t,ra.nsit ion is neatly illustrated in Fig. 3b. Furthermore, the scaling of t h e growth rate XR arid the resistjive layer width 6 w i t h resistivity is est.ablishrd for v 3 / 5 and 6 q 2 l 5 for t.he tearing mode and Ah’ q 1 / 3 values of 7) u p t o lo-’”, namely A R and 6 q l / ’ for the pressure-driven mode 7
~
% ‘
-
-
-
-
c
c r/a 0.0
0.2
0.6
0.6
0.8
0.0
0.2
0.4
0.6
0.8
r/ a 1.0
Fig. 4: 1)) -
Normal component of the perturbed rnagnrtic field h l - rb, and of the velocity r v , for rj = in arbit,rary 1init.s. l’hr singular surface at r , = 0.5a is indicated.
The plots of the eigenfiinct,ions show t.hat a very high resolution is necessary to resolve t h e struclurc. in a layer whose width strongly decreases as the resistivity. We accumulate rnrsh
W. Kerner
256
points at the resonant surface, thereby drastically increasing the numerical resolution. Typical convergence studies are discussed in Ref. / I 1
'
The linearized resistive M H D operat,or is non-self-adjoint, which leads to coiiiplex eigenvalues. Overstable modes can occur only if there is a locally increasing pressure, i.e. d p l d r > 0. .4n appropriate class of tokamak-like equilibria is defined in Ref. / 2 1 / , with the paramet.er a labelling the pressure gradient. The results are displayed in Fig. 5. In the force-free equilibrium ( a = 1; d p / d r z 0) the only instability is the unstable tearing mode. If a assumes smaller values, a < 1 , this mode becomes slightly stabilized, but a second unstable mode emerges from the origin. With increasing pressure gradient, i.e. with decreasing values for a , the growth rate of the most unst.a.ble mode decreases and that of the second most unstable mode increases until both modes merge and an overstable mode evolves. This happens for N c, 0.868. If ci i s further decreased, the growth rat,e of this overstable mode becomes smaller. The oscillat,ory frequency. however, strongly increases. For a -. 0.31 the mod? becomes stable with still finite oscillatory be11aviou r .
15*10T3
i
.
a = 0.4 ** - 0
0 .o
0 .o
i"
= 0'867
iI 5.*10-3
Fig. 5:
Overstable modes for a tokamak-like e q ~ ~ i l i b r i u m with varying pressure gradient. Tho values of a , which governs d p l d r , are given. The pa.rameters are 7 = 2 x 10n = l , m=-2 arid k = 0 . 2 . Re(A) denotes the growth rate, and Im(X) the oscillation frequency. For decreasing values of (Y two unstable modes merge to yield a complex eigenvalue for a < 0.868.
'.
25 7
The Complex Eigenvalue Spectrum jor Resistive Magrietohydrodjinamics 5.4. Resistive Alfven waves
T h e final point is t h e analysis of Alfven modes. In ideal MHD the Alfven waves form a continuum if the profiles are non-uniform. T h e corresponding eigenfunctions exhibit logarithinic singularities. W i th finite resistivity, the ideal continua disappear. For t h e tokamak-like and 17 = 5 Y l o p c ' is equilibrium ( u = 2 in eq.(7) ) t,he Alfven spectrum for 1) T
displayed in Fig. 6. T h e sound mode spectrum has a n even more complicated structure and is concentrated close t o the origin. T h e sound modes are not well resolved o n this scale and a r e therefore omitted from this and t h e following graphs. Th e purely d am p ed modes on the 0 and XR = --00 a r e omitted as well. negative real axis with accumulation p o i n t s a t X,q T h e ideal continuum is approximated a t all points where W A ( T ) has a local extremum. T h e uppermost branch of Fig. 6 is due to modes with eigenfunctions localized near r = a in the limit. 17 + 0. T h e second branch is formed by t h e corresponding modes localized near r = ~
0. T h e lowest branch reaching the origin is d u e to modes localized at t h e resonant surface. T h e eigenfunctions for the modes corresporrding t o all of these branches have different numbers of radial oscillations. For a given branch. t h e number of oscillations increases from the end points lying on t h e imaginark axis until th e number becomes infinite at t h e accumulation points which correspond to X K 0 and - w on t h e real axis. T h e modes with one and two 7
oscillations localized at r = a are examined for varying 7 . Th e dependence of the eigcnvaluc on 17. with 17 ranging between lo-' and 1 0 is shown in Fig. 7. I t is evident t h a t for large 17 the
'.
eigenvalues for different modes lie on different curves in the complex plane which coincide on11 a t t h e end points. Not,e t h a t for 1)-values larger t h a n lo-' the modes are purely damped and no complex eigenvalue occurs. For small ?)-values, 17 5 lo-', t h e eigenvalues lie o n practically identical curves, which is evident from Fig. 6. We skip t h e physical interpretation arid address here t h e problem of numerical accuracy. Therefore, the very simple equilibrium with only a linear dependence of t h e B , field o n the radius is adopted. Then th e ideal Alfven frequency is
Xld
=
ikBz/&
=
1
0.40(1 + 67).
With finite resist.ivity. all t h e modes are d a m p e d . T h e QR algorithm can only be applied u p t,o 51 radial points with a matrix dimension of d RZ 600. T h e eigenvalues for a case with 17 = 1 x are displayed i n Fig. 8a. In t h e case of zero resistivity t h e Alfven modes form a continuum, also indicat.ed in Fig. 8 , with singular eigenfunctions. Especially interesting is
-
the question what happens for small resist.ivity, i.e. in th? limit 0 0. T h e results for sma.ller resist.ivity are quite puzzling; Fig. 8t1 shows t.he spectrum for the sam e mesh as in Fig. 8a bui with smaller 7. With decreasing resistivity t,he location of the eigenvalues i n t h e complex X plane drastically changes. O n the contrary. analytical results suggest t h a t in the limit of vanishing Q the eigenvalues lie on prescribed curves. (see Refs. ,!22.23; and further references
'.
given there). In Fig. 8c the spectrum is displayed for r/ = 2.0 Y 10 t h c same value as in Fig. 8t). b u t computed by applying inverse vector iwration using 313 mesh points. Comparison of the converged results of Figs. 8a and 8c indeed reveals t h at t h e eigenvalues lie on identical curves. T h e eigenfunctions have an incrvasing n u mb er of radial nodes Y = I , 2 . 3 , ..., wit,h I/ 1 being closest t o t h e ideal continuum. T h e u p p er (lower) line of t h e triangle corresponds ~
t)igenfunct,ions with oscillations near ilie hoiindary (near t h e origin). T h e more oscillations there are in the eigenfunrtions, the st.rorigrr t,hc d a m pi n g is. At t h e branch point the oscillations
1,o
W.Kerner
258
Fig. 6: Hesist,ive Alfven rnodc s p e c t r u m for t h e tokamak-like equilibrium (v r 2 in eq. ( 7 ) ) with one singular surface for values of t h e resistivity 7/ = I 0 ’(.) and q 5 . 1 0 p G ( x ) . The sound modes have much 7
. A d
Re h
-0.05
0
0 .o
.o
srnaller eigenfrequencies, as indicated by t h e box close t o t h e origin. T h e purely d a m p e d modes UJI t h e nega1,ive rcal axis have been omit,t ed .
Im X
.
1”
*
Z n d mode
mode 0.10
.. ... . :t * .
‘
*.
i 0.05
Frequency of t h e first t w o modcas frorn t h e uppc~rriosthranch in Fig. 6 for dif-
Fig. 7:
ferent 7 ranging from 71 ~- 1 0 (bot.torn).
rj
=
(toll) to
Re h
-0.05
0 0 .o
.o
The Complex Eigenvalue Spectrum for Resistive Magnetohydrodynamics
259
Im A 3.0
Im A 10
a
b
ZD
..
. .. in
.............-
.........-ID
-2D
Re A
-? 0
0.
-10
Re A
0.
0
Im A
10
C
M
0
...........- 20 Re A
- lo
Fig. 8: The resistivc Alfven spectrum. T h e solid b a r on t h e imaginary axis denotes t h e ideal Alfven continuum (0.40 5 Irn(X,d) 5 2.80). a ) for 17 = and K == 40 intervals. cornpuled by t h e QR algorithm. All cigenvalucs are correct, i.e. converged. b) for 71 = 2 Y and N = 40 intervals, computed by t h e Q K algorithm. A significant nunibrr of t h e eigcnvalues are false owing 1.0 insufficient numerical resolution. C) for 7 2 v 10 and N = 312 inkrvals, succrssivoly rorriput.ed by inverse vector it(:ration. All eigenvalues are correct, i.r. converged. ~
0,
'
W.Kerner
260
occur at, t h e centre and vanish outwards, and further along the eigenvalue curve these oscillations extend over t h e entire radius. Diagram 9 shows three eigenfunctions for three different cigenvalues.
0.
0.6
a2
02
0
0.8
06
01
0.8
r 10
d
I
02
0
OL
06
08
10
0
,
,
,
02
,
,
,
01
,
,
06
,
08
I
10
Fig. 9: Korrrial c o r n p o ~ i c nof~ i.he velocity u 1 = T ? : ~. computed by inverse vector ileration corresponding t o t h e eigcnvaliifts a) X -0.27 + z . 2 . 3 3 for 71 = 2 > 10- which i s the fourth mode of t h e upper branch, 2' . 1 . 4 4 for 7) = 2 x 1 0 ', whicli is the t.ent,h mode of t h e lower branch, 1 ) ) X : --O.(i3 i~ I .57 t 7 . 0 . 3 8 i'or q = 2 1 0 ', w h i c h is tlic' last mode with oscillation, i.e. finitc c) X imaginary p a r t , d) X 0.27 -t T. - 2 . 3 2 for q 2 i 1 0 ',, which i s t h e twelfth mode of t h e upper branch Xotc t.ha.1. t h e real and imaginary p a r t s o f ttic eigcnfiinctions are similar i n structure and cqual in magnitude.
',
-
7
~~
~~
~
Purely d a m p e d modes emerge from a second branch point, on the negative real axis of t.he The eigerifunctions are Bessel function-like with practically constant amplitude but. an
A plane.
Tlz e Coinp lex Eigeriualu c Spec tru tn f o r R esis tiiv Magi i e toh,vdrodynarnics
26 1
increasing number of radial nodes away from this branch point towards the t,wo accumulation points X = 0 an d X = -co. If th e numerical resolution is not good enough, completely false results are obtained for normal modes with eigenvalues between t h e two branch points - like t.hose shown in Fig. 8b. Adequately representing both t h e radial oscillations and the amplitude modulation requires a much finer grid t han t h a t for resolving the purely damped modes. Only with t h e reported fine grids of N = 300 is one able to understand t h e numerical results near the branch points. T h e smaller the resistivity, t h e more eigenvalues lie on the 0. T h e ideal ( r j = 0) curve, b u t t h e curve itself becomes independent of r j in t h e limit r j Alfven continuum is approximated only a t t h e two end points in the limit of r j 4 0 by modes where t h e eigenfunction is peaked in a sniall layer at r = 0 an d r = 1.0. This laver width 6 decreases with r j as h ,I'/~. Figures Y a and 9d display t.he eigenfunctions of two cases
-
-
with almost the s am e eigenvalue b u t for t,wo different values of t h e resistivity. T h e structure of
t h e eigenfunction is similar. b u t for smaller resistivity more radial oscillations occur in a finite radial dorna.in.
Controlled thermonuclear fusion research is aimed a t achieving o n earth economic exploit,ation of fusion energy. which is the source of energy in t h e
btdt\.
I ~ C111i. I I purpose the
plasma conrained in magnetic confiiirnient devices is being extensively studied rxperiment.ally aiid thcorctically. T h e most dangerous instabilit,ies which limit t h e operation of discharges are macroscopic. These can b e described bl- th e MHD model. T h e typical time scale of such gross insl.alJilities in tokamaks ranges from microseconds t o milliseconds. Th e propert.ies of tlrc linearized motion around a n equilibrium s ta te described by resistive MHI) are of special interest and have prompt,ed t h e numerical normal-mode analysis. Since dissipat,ion is included. the Galerkin procedure yields a non-variational form leading to the general eigenvalue problem
Ax = XBx, where A is a general and B il symmetric, positive-dcfinit,e matrix. This allows a ; also t.hc stable p ar t of t h e spectrum is t he numerical search for stable plasma f ~ ~ i i i l i h r ibut relevant for heating. T h e first configuration studied is t h e roilstant-current equilibrium without resistivity. Th e plot of the eigenvalues, either real or purely irnaginar!. clcarly displays three different branches. nairirly fast. and slow magnetoacoustic wa\"cs and -4lfven maves. T h e accumulation point of
t h e fast modes t en d s t o infinity, t h r slow aiid Alfven modes usually form continua approaching m r o frequency. T h e m a in result is t.hai t h e disc-retizaliotr based or1 cubic and quadratic
finite elements gives a n optimal approximation for t h e eniirc spectrum. .4 less sophisticated disc rct izat,ion yields false eigenvalues due t o numerical coupling of t h e different branches ("pollution"). The resistivity h a s small v;ilues for typical tokarriak discharges. Then the infli~ence o f resistivity on t h r ideal fast modes and on ideal instabilities is weak and is therefore not of interest. Important are new, resistive instabilities which decouple t h e fluid from the magnetic field. T h e problem has quite diflerent spatial a n d temporal scales, such as small resistive lavers. which requirc a fine grid for accurate numerical approximation. In t h e applications pressurc and current-driven resistive instal)ilit.ies have been discussed. Owerstable modes, i.c. torriplex eigenvalues. o r
W.Kerner
262
strongly increased by appropriate mesh accumulation. Convergence studies using increasingly finer meshes establish the validity of the result,s. Ideal Alfven and sound mode continua undergo drastic changes if resistivity is included. Resistive Alfven waves are extensively studied. The Alfven rriodes form a point. spectrum lying on a locus which is independent, of resistivity in the limit, 17 - 0 and which intersects the ideal continuum only at its end points and at few, specific interior point~s.T h e corresponding eigenfunctions show radially oscillatory bchaviour. For many oscillations over the entire plasma radius t h e modes are purely damped. Accumulation points on the negative real X axis, i.e. purely d a m p e d modes, occur at X -co and X 40. This part of the spectrum is easily interpreted b o t h numerically and analytically. For sufficiently small eta, 1) < t h e eigenvalues branch away from the negative real axis. T h e modes pick u p an oscillation frequency and X is now complex. Near the imaginary X axis the eigenvalues bifurcate towards specific points of the ideal continuum. For decreasing values of eta the curve itself becomes independent of t h e resistivit,y and t h e point density on this curve increases as 0 - l l 2 . At the same time the number of radial oscillations in t.he eigenfunctioris increases. The corresponding eigenfunctions comprise radially oscillatory an .I i i o i i - o ~illdtory ( components. i.e. resistive and ideal solutions. This requires high nunierical accuracy. S o w , a non-uniforni inesh is not advantageous, this being different from resolving a resistive layer. This numerical study can be performed for valiies of eta u p t,o 10-"(q 1 I [ ) - ' ' \ . v hicii i i ~iifficientfor discussing the asymptotic behaviour +
17 * 0 . These findings have raised many interesting questions concerning global .4lfven waves and t.he role of the continua for heating. Finally, it is emphasized that only a limited number of complex eigenvalues, i.e. modes with damping and oscillation in t.he time dependence, occur for a given value of eta. This number of complex eigenvalues is, naturally, independent of t,hr mesh size. T h e more mesh points are int.roduced for accurate resolution, the smaller this fraction of the eigenvalue spectrum becomes. P u t t i n g aside the physic,al interpretation, we now discuss the general aspects in terms of a n eigenvalue problem. T h e st.artiiig point is a set of linearized differential equations with dissipation. A state vector u cont,aining all t h e linearized perturbat.ions is introduced and the time dependence is expressed hy an eigrnvalue u(x.t ) = u(x)ex’. T h e problem then assumes
tlie form
R
11
-
A5
11.
R and S being matrix opcrators as introduced i n eq (15). Applying the finite-element method t h e general eigenvalue problem A x
AFx
arises. where A is a non-self-adjoint, matrix. Otherwise. the procedure is the same as for a variational form. The QR algorithm, available i i i the ElSPACK library, can be applied to compute all the eigenvalues. However, full rnatrices are produced, which limit,s the tractable dimensions of the matrices, i.e. to d 5 600 in o u r case. This gives only a coarse overview of the spectrum. H u t we are interested in subtle details of the spectrum, which quickly increases the matrix size by an order of magnit,ude. As explained, the very modes wit,h complex eigenvalues which a r e hardest t o compute are the most interesting ones. These modes present a small Cract.ion of the entire spectrum. Consequently, an algorithm which preserves the band structure
The Complex Eigenvalui, Spectrum for Resistive Magnetohydrodynamics
263
of the matrices and allows large dimensions is applied. In this paper results computed by inverse vector itseration in conjunction with a continuation procedure are presented. The algorithm contains factorization and, subsequently, solution of linear systems and is, basically, simple. Most of the computing time is spent on the f U decomposition of a band matrix. With a suitable shift A<, fast convergence is found. Its fast execution allows - in a n interactive manner - efficient analysis of part, of the spectrum or even all of it. There is no guarantee against one or several eigenvalues being missed somewhere in the complex plane. But this is a general problem of the non-Hermitian case. All efficient solvers, such as subspace iteration or the Lanczos algorithm, suffer from this drawback, which has to be lived with. On the other hand, if one eigenvalue of the branch of the spectrum of inkrest is found by means of random shifts or by a continuation procedure as done by the authors, one can indeed find the next one and eventually the entire branch. Careful inspection of the corresponding eigenfunctions, e.g. in terms of radial oscillat,ions, allows one to decide whet,her the nearest eigenvalue has been found. The efficiency of t,he method makes up for the fact t,hat one has to restart several times with slightly different shifts. In the application of t,he algorithm, we do not just show with a few examples that we c a n indeed compute complex eigenvalues and cigenvectors; with the discussion of the ent.ire .4lfven spectrum we also want to point o u t the difticulties involved in a complex spectrum. Near the branch points of the Alfven spectrum a very fine grid is necessary. Only careful convergence studies performed b j inverse vect,or itrratiori allow correct. i.e. converged, rrsults to be separated from false ones. It is cniphasized that t.he complete spectrum can be found h y inverse vector iteration with, admittedly, q u i t e a lot of computer runs, but very fast. ones. 'The successful application of the algorithm implies the extension of inverse vector iteration t.o such large systems that one matrix cannot be kept in memory, even in band-matrix storage mode. Such large systems naturally occur in the normal-mode analysis of two-dimensional toroidal equilibria. In this case the algorithm has t o be split up into tractable pieces. For the f U decomposition this is possible and thus such an extension is straightforward. The eigenvalue patterns discussed make clear t.he demand for a more sophisticated solver tailored t o the simultaneous evaluation of an entire branch. Starting from one eigenvalue, it should then be possible by suitable orthogonalizat.ion in the iteration to find the closest one and hence. eventually, the entire curve. This procedure generates a special subspace. We have t o leave the answering of this problem to t.he oxpert,s. Let. us now discuss the solution for the spectrum of really large systems with , say, d 2 20,000 for 2D equilibria where the eigenfunclion is composed of a superposition of several poloidal Fourier harmonics. The QR will produce no useful approximation at. all. However, the subspace iterations (Refs. i 8 , 9 , 1 0 / ) , which also preserve the structure of the matrices, can provide reasonable knowledge of the spectrum if a proper complex shift is introduced. The possible numerical incompleteness of thc spectrum is not. a drawback, since such "gaps" can be filled by means of the simpler scheme o f continued inverse vector iteration. In this context the detailed analysis of the structured complex eigenvalue spectrum of resistive MHD can serve as a bench-mark test in the development of more general eigenvalue solvers. Consequently, the matrircs A and B are provided as input for other procedures; nt present the Lanczos algorithm o f Cullum and Willoughby is addressed 1.0 our problem. If tackled by many experts, the general iion-symmetric eigenvalue problem will he solvable - even for very large matrices.
W.Kerner
264 Acknowledgements
T h i s subject could only be successfully t,reat,ed in collaboration with many colleagues. T h e a u t h o r wishes t o thank particularly J . Steuerwald for his enthusiastic a n d skilful support i n establishing t h e eigenvalue solver. T h e rc,sulI,s presented were actually computed by K. Lerbinger in t h e context of his 1’h.D. t,hesis: the figures are taken from joint papers (Itels. /11, 12, 2 3 / ) , with t h e sole except,ion of Fig. 5, which is taken from A . Jakoby (Ref. / 2 1 / ) . E . Schwarz significantly contributed to the coding of t h e eigenvalue solver. Credit for the layout and processing of t h e manuscript goes t o E . Schwarz and W . Seifert.
References
~~
~
/
1 ‘ Fusion, Volume 1 Magnet,ic Confinement Part A . edited by E. Teller. Lawrence Liv-
ermore Laboratory. University of California. A C A D E M I C P R E S S 1981 12 1 G.Baterrian .‘MHD Irist.abilities” Tlie M I T Press, Carribridgr. Massachusetts a n d
London. England 1978 ‘ 3 J.H. M’ilkirison. “ T h e Algebraic Eigenvalue Problerri”. Clareridon Press. Oxford 1965
14.: J.H. Wilkinson and C.H.einsch, “Linear A41gebra”.Springer-Verlag. Berlin. Heidelberg. New York 1971 15; B.N. Parlett, “The Symmetric Eigenvalue P r o b l e m ” . Prentice-Hall. Inc.; Englewood
Cliffs, N.J. 07632, 1980 /6/ G.H. Golub and C.F. van Loan “Matrix Computations”. T h e J o h n s Hopkins I;ni-
versit,y Press, Baltimore, Maryland 1983 / 7 / B . T . Smith et al., Matrix Eigensystem Routines - EISPACK Guide, 2nd Edition. Springer-Verlag. Berlin: Heidelberg, New York 1976
/8; Y.Saad. “Chcbyshec acceleration techniques for solving nonsymmetric eigenvaluc problems-‘, Yale Ilniversity. Dep. of Computer Science, Technical Report. 255, December. 1982 ,‘9’
B.K.Parlett: Computing Met,hods
i n Applicd Sciences arid Engineering, V: R. Glewinski. J.1,. Lions (editors). North-Holland I’ublishing Company, INRIA, 1982. D.N. Parlett., D.R.. Taylor arid Z-S Liu (1983). “ T h e look ahead Lanczos algorithm for large uiisymmetric eigenproblems“. I’roc. of INTRlA Sixth International Conference on Computing Methods i n Applicd Scicrices and Engineoring. December 12-16, 1983, Versailles, France
/ 1 0 / J . Cullum a n d R.A. Willoughby: “ A Lancxox procedure for t h e modal analysis of very
large nonsyrnrnetric matrices”, published i n Proceedings of t h e 23rd IEEE Conference on Decision a n d Control. Dec. 12-14. 1984. Las Vegas, Nevada pp 1758-1761 :ll’ W . Kerner,
Ii. Lerbinger, it. Grober and T . Tsuneniatsu, Comp. Phys. C o m m u n .
36, 225 (1985)
’12 ’ M’. Kerner. K . Lerbinger. J . Steuerwald, to a p p e a r in C o m p . Phys. C o m m u n . , I P P 6.’236 j1984)
The Cornplex Eigenvalue Spectrurn for Resistive Magnetohydrod~,r~amics
Is,284 (1975) J . Rappaz. Numer. Math. 28. 15 (1977) C . Strang, G . J . Fix. “An Analysis of thc Finite Element. Met.hod”, I’rentice Hall, Englewood Cliffs, K . J . 1973 R . Gruber. Cornput.. F’hys. Cornrrrr~n.20, 421 (1980) D. Scoti, a n d R. Ciruher, Cornput. l’hy<. C o m ~ r i u n .g ,115 (1981) J . J . Dongerra, C.B. Moler, J . R . Bunch and G.M’. Stewart, “LINPACK USER’S
113,’ K . Appert. D. Berger, R . Gruber, J . Rappaz, J . Comput.Phys. /14/
,!75/
/16/ /17/ !18/
GUIDE” SlAM, Philadelphia 1979 / 1 9 / M.S. Chance, J.M. Greene, R.C. Grirnni, J.L. Johnson, h’ucl. Fusion l7, 65 (1977) 120; W. Kerner a n d H. Tasso, Plasma Physics 24, 97, 1982 /‘21/ W . Kerner, A. Jakoby and K . Lerbinger to appear in Journ. Comp. Phys., 11’1’ 61255 (1385) 122’ Y.P. P a o and ‘N. Kerner Phys. of Fluids 28, 287 (1985) /, ,* , 2,>, W . Kerner, K . Lerbinger and K . Hiedel, to appear in Phys. of Fluids, I P P 6/250 (1985)
265
This Page Intentionally Left Blank
Large Scale Eigenvalue Problems J . Cullum and R.A. Willoughby (Editors) 0 Elsevier Science Publishers B.V. (North-Holland), 1986
267
ILL CONDITIONED EIGENPROBLEMS Francoise Chatelin Centre Scientifique IBM-France 36 Avenue Raymond Poincark F-75 1 16 Paris, France and Universitk de Paris IX - Dauphine The method of simultaneous inverse iterations is unstable when it is used to compute the invariant subspace of a defective eigenvalue. In this paper we propose a stable method, of equal complexity, which is based on a modification of Newton ’s method.
1. INTRODUCTION
When computing a multiple eigenvalue for a differential or integral operator (for a matrix), one ends up in general, after discretization (because of roundoff errors), facing the problem of computing a cluster of eigenvalues. We intend to show that when such an eigenvalue is defective (i.e. the eigenvectors fail to span the whole invariant subspace), the method of simultaneous inverse iterations is unstable. We propose a stable method, of equal complexity, which is based on Newton’s method. The paper is divided into two parts:
i) a complete analysis of the condition number of a set of eigenvalues and its associated invariant subspace.
ii) a study of simultaneous Newton’s iteration versus simultaneous inverse iteration when the desired eigenvalue is defective.
A is an n
x
n complex matrix, and sp(A) denotes the spectrum of A, i.e. the set of its
eigenvalues. If A
E
sp(A), then there is an eigenvector x # 0, such that Ax = Ax. Let m, g,
e
denote respectively, the algebraic multiplicity, the geometric multiplicity and the index of lambda. The eigenvalue A is said to be semi-simde if
e = 1 and defective if e > 1 .
By definition
m is the multiplicity of h as a root of the characteristic polynomial of A. g 5 m is the di-
268
F. Chatelin
mension of the space spanned by the eigenvectors of A corresponding to A, and the index P=m-g+l.
NOTATION A* denotes the conjugute trumpose of the matrix A. We use
11
11 and 11
. 1)
to denote respectively, the sptrul and the Frobenius matrix norm
associated with the Euclidean and the Frobenius vector norms. We define cond2(X) = (1 X I(
(1 X-' I(2 , the condition number (relative to inversion) of
the
matrix X
lin (x,, .. . , xm) denotes the spun of the m vectors x,, . . . , x,. x*y denotes the innerproduct of the vectors x and y. ej denotes the jth caodinute wetor whose jth component equals 1 and whose other components are zero.
I, denotes the identity matrix of order j.
2. STABILITY OF AN EIGENPROBLEM
We study the variation of the eigenvalues and eigenvectors of A in terms of a variation AA of A. We first study the case of a simple eigenvalue. 2.1. ALGEBRAIC MULTIPLICITY, m = l . Let the eigenelements A, x, x. satisfy
AX = A X -
A*x, = AX,
Pl
where 11 x 11 = 1 and x5 x = 1
= xx* is the orthogonal projection on the eigendirection lin (x).
Let ( be the acute angle
between the right and left eigendirections M = lin (x) and M, = lin (x*). See Figure 2.1. Let [x, Q] be a unitary basis of C" such that Q is a basis of M I , and set
z L= Q(B - AI)-'Q*, with B = Q*AQ. The operator C L is the generalized inverse of A - h1 in M I . Finally we set 6 = dist (A, sp(A) - {A)), the distance from A to the nearest distinct eigenvalue in sp(A).
Proposition 2.1. If A is perturbed by AA, then for AA small enough, the first order variation
of A is Ah = x*, AAx, and the eigendirection M rotates through an angle for which the first order approximation is 0 such that tan(@ = 1) Z'AAX
11 2 .
Proof. Use the Rayleigh-Schrodinger series expansions. These converge for
(1 AA 11 small
enough (Chatelin [1986]).
0
Figure 2.1.
X is spc(X) The spectral condition number of the eigendirection lin(x) is spc(x) = 11 8' 1) 2 . Definition 2.1. The spectral condition number of the simple eigenvalue
We recall that -1
6
11 x* 11
I II(B-W
= ( cos < ) - I ,
1) x* )I 2 .
and for small enough 6, 26-
-1
1 l 2 = IIZill25
'
cond2(y) (if B defective)
6-' cond2(v)
(if B diagonalizable)
where !is the index of the eigenvalue of B closest to A, and 4f is the Jordan basis of is defective and the eigenvector basis of B when B is diagonalizable.
We conclude that: if
I
5 is almost x, then h is ill-conditioned,
2 if 6 is small and/or cond2(y) is large, then lin(x) is ill-conditioned.
B when B
F. Chatelin
210
When A is Hermitian or normal, M I is invariant,
6 = 0, 11 xt I(
= 1 and
11 B’ ( 1
= 8-I. In
those cases, the only cause of ill-conditioning of the computation of the corresponding eigenvector x is a small gap 8. When A is not normal, the second cause of ill-conditioning for x is a large departure from normality for B (i.e. cond2(\r) is large). Example 2.1. The spectral condition number of a multiple eigenvalue (of algebraic multiplicity
m) is not defined. When the matrix is perturbed, we usually get simple eigenvalues
If
X is semi-simple, max spc(A’,) may be moderate since h has an orthogonal eigenvector basis, but if h is defective, max spc(hri) is necessarily large since A has less than m independent I
eigenvectors. We conclude that a defective eigenvalue is necessarily ill-conditioned when considered individually (See Example 2.3). A Example 2.2. Consider the 2
x
2 matrix
The spectral condition numbers of a and b are of order E - ’ . When ( b - a) is small, T is close a+b to a matrix with double eigenvalue , which is semi-simple or defective according t o the 2 value of E
i) If b
- a is small, E - ’
ii) If b
- a and
E
is of moderate size and T’
=
(a ~
+ b) I, then I(T - T‘ 11 2
are small then the matrix
T”
I
(b
a
- -E
(b
- a) E
- a) 4
has the defective eigenvalue a and the Jordan basis 2 + -
.=(
1 & 2
1 b-a
(I+-
b-a 2
1
).
= O(b - a)
271
Ill Cotrditiotrcd Eigenproblems
%
If b-a
a)
is moderate in size, cond,(V) is also moderate. cond(V,) increases with the size of
. V has numerical rank 1 (to the order
b-a
). Notice that the departure from normality
of T is then large. A 2.2. ALGEBRAIC MULTIPLICITY, m
> 1.
We consider now a set u = {hi]of eigenvalues
of total algebraic multiplicity m, separated from the rest of the spectrum. We call such a set a
block of eigenvalues, and we denote 6
= minimal distance ( u , sp(A) - u ) = min { 1
h -p
I
,X
E
u, p E sp(A)
- u].
(See Figure 2.2).
X
Figure 2.2
In practice u often consists of a cluster of eigenvalues, or of a single multiple eigenvalue. Let
M be the invariant subspace associated with u, and let [Q, Q ] be an orthonormal basis in Cn where Q and Q are respectively orthonormal bases of M and M I . Define P l = QQ* the orthogonal projection onto M. The matrix B = Q*AQ represents the application A : M
-P
M
in the basis Q : sp(B) = u. Generalizing Equations (2.1), we consider AQ = QB
A*X. = X,B*
= 1, where Q*Q = Q*x*
X, represents the basis in the left invariant subspace M,, which is adjoint to X.
Let
E = diag(&) denote the diagonal matrix of canonical aneles between the subspaces M and M. (See Bjorck-Golub [1973], and Davis-Kahan [ 19681). Set B = Q*AQ, then A is similar to the block-triangular matrix
F. Chatelin
272
(
, t 2B
T may be block-diagonalized:
LQ*AQ
STS-’
-
)=(
+ - k ) = T .
(.-ti:)
where S =
(SF),
and Z is the
solution of the Sylvester equation
The linear map Z-BZ
- ZFJ
defined
on Cmx(n-m) is invertible if
and only if
B)-’ and define sp(B) n sp(B) = 6. We denote its inverse as (l3,
as the block-generalized inverse in M I associated with u. Lemma 2.2. (See Stewart [ 19731 and Varah [ 19791)
Proposition 2.3. If A is perturbed by AA, for
11 AA 11 small enough,
u
and M become re-
spectively u’ and M’ such that, up to the first order:
i) u’ is the spectrum of B’ = B
+ X*, AAQ, and
ii) M’ has the basis X’ normalized by Q*X’ = I, such that X’ = Q - 2’AAQ. Proof. See Chatelin [1986].
0
Corollary 2.4.
where 0 is the diagonal matrix of the canonical angles between M and M‘. Proposition 2.5. For
!, such that
)I AA )I small enough, given any A’ in u’, there exists
h in 2 with index
Ill C7oritlirioned Ij'igctiproblems
273
where V is a Jordan (or eigenvector) basis of B . Proof. This is a consequence of the fact that there exists a h such that
(See Chatelin [1986] for a proof).
Corollary 2.6. For
AA 11 small enough,
dist(u, a') = rpa5 min X€a
XEU
I
A' - h
I I 2 cond2(V) II X* II 2 II AA II 2
1 /m
We now define spectral condition numbers for the case m
.
> 1.
Definition 2.2 The global spectral condition number for a is spc(a)
= cond2(V) II X, II .
The spectral condition number for the corresponding invariant subspace M is
We recall
that
)IX, 11
=
11 ( cos 2 ) - I II
= ( cos &,,ax)-
I
and
11 2' 11
depends on
h-', cond,(V), and cond2(r). (See Example 2.3 below). When A is Hermitian or normal,
X, = Q, V is unitary, cond,(V) = 1, E = 0 , and We illustrate the dependence of example.
Example 2.3. Let
11 Z' 11
IIZ' I I F
=6
-1
upon 6-', cond,(V), and cond2(y) by the following
214
F. Chatelin
r l d I d I d I d 1
Clearly, 6 = u=
I a - 1 I , cond,(V) depends on I
II(B,BI-
i)
1
c
1
, and cond2(V) depends on
1
d
1 . We set
IIF.
Let a = 0.8, and 6 = 0.2.
c= 1
d.
U
d=-1
5
c 0
lo5
2
0 1.6
- 10
-5
-1
lox
2.3
1
lo4
5
lo5
lo8
-15
7
10
lox
100
5
lo6
5
5
lo5
1.3
lo7
Table 2.1
ii)
Let c = 1, d = -1.
Table 2.2
lo8
21s
Ill Coriditioiied Eigenproblems
Table 2.1 (Table 2 . 2 ) shows that a increases when c or d (6-l) increases. There are similar results for matrices B = ( i i ) , a f b .
A
Example 2.4. Consider the particular case u = {A], where h has multiplicity m. If h is semi-
simple, B = XI, and cond2(V) = 1. But if h is defective, cond,(V) may be large (see Example 2.2) and h may be ill-conditioned without
11 X, 11
being large. When cond,(V) 11 X,
11 is
moderate, a defective eigenvalue is globally well-conditioned, in contrast with what happens when it is treated individually (see Example 2.1 .). Cond,(V) is large when there exists a matrix
close to B with a defective eigenvalue having an almost degenerate Jordan basis.
Example 2.5. A =
cond,(V)
(
't)
has eigenvalues { I , 01 and eigenvectors V =
lo4. It is easily checked that
is defective with double eigenvalue 1/2 and the Jordan basis
Cond,(V')
- lo4. The departure from normality of A is of order lo4.
A
2.3. BALANCING OF A NONNORMAL MATRIX.
The quantity
11 X. 11
,is not invariant under a diagonal similarity transformation on
advisable to balance the matrix. The relevant A-'AA such that Example 2.6. Let
and
,
,
A . It is
11 X, 11 is the one which corresponds to the matrix
11 AK'AA 11 is close to its minimum.
F. Chatelin
276
A = ( ' 0 10' - 4 ) ,
then A' = A-'AA =
( A)
. The eigenvectors of A are
x=(
1
-4)
0 - 10
and those of A' are
X'=A
-1
X=('
'>.
0 -1
Balancing A with normality for A.
A has decreased cond2(X), as well as decreased the departure from
A
2.4. GROUPING THE EIGENVALUES.
To compute ill-conditioned eigenvalues and/or eigenvectors one tries to group (i.e. treat simultaneously) the eigenvalues such that a perturbation has more effect on them and/or o n the associated eigenvectors. This should be done to decrease as much as possible the corresponding spectral condition numbers.
When A is normal, spc(a) = 1 and spc(M) = 6-l. Hence, grouping close eigenvalues will decrease spc(M). When A is nonnormal, spc(a) = cond,(V) which depends on cond,(V), cond,(V) and 6-'. decrease 6-' and 11 X,
11 X, 11
and spc(M) =
1) Z*I J F
By grouping certain eigenvalues, one may
11 2 , but cond2(V) and cond2(V) remain unchanged.
Example 2.7. (Stewart [1972]). Consider
It has eigenvalues { l ,0, . 5 ] . The first two are ill-conditioned, whereas the corresponding eigenvectors (1.0, O)T and ( 1, - 1 0-4, O)T are well-conditioned. Indeed
111 Coiitlifiotied Eigrnproblenis
A’ =
I
lo4
o
1.1
0
0
2
0
.5
1
211
has eigenvalues { 1 . I , -0.1, .5 1. The first two eigenvectors are (1/3)
106
1
-1.1
10
x
-4
( - 1/3) x 10
We group the two ill-conditioned eigenvalues:
spc(M)
- lo4 where M
u =
{O, 1). We find spc(u)
- lo4 and
= lin(e,, e,) is the invariant subspace associated with u. Observe that
grouping the eigenvalues has not decreased the spectral numbers since
‘)
B = ( 1 10 0
0
has eigenvectors
v= with cond2(V)
(
1 -4)
0
- 10
- lo4.
One may check that, for A’, the basis X i in the invariant subspace M’, normalized by
Q*X’ = I, with Q = [el, e2] is given b y X’
=
XB where
thus
One may comment that almost parallel but we11 conditioned eigenvectors generate an illconditioned invariant subspace. A
278
F. Chatelin
The reader will find computational criteria to decide when and where to group eigenvalues in the paper by Demmel and Kagtrom in these Proceedings.
3. SIMULTANEOUSNEWTON’S ITERATIONS
We study the relationship between simultaneous inverse iterations and Newton’s iteration in the case of a defective multiple eigenvalue. To set up the framework, we start with a simDle eigenvalue. 3.1. X IS SIMPLE.
Let A be a simple eigenvalue of A. Then for any y, the eigenvector x normalized by y*x = 1 satisfies F(X) = AX - x(y*Ax) = 0
(3.1)
Let u be a starting vector and let y be given such that y*x # 0; The normalization y*x = 1 is linear in contrast with what is usually done (x*x = 1) (see Anselone-Rall[1968]). Newton’s iteration on Equation ( 3 . 1 ) is given by x = u/(y*u), (I
z =x
k+l
k
-x ,
where z satisfies
- x ~ * ) A z- z ( ~ * A x) = - F(x ), k 2 0. k
k
This is well known to be equivalent to a right Rayleigh quotient iteration (See Chatelin [ 19841).
Newton’s method is expensive so we consider the modified Newton’s iteration, where scalar chosen close to A, and u is a starting vector.
- Xk, (I - Xky*)AZ - U Z = - F(Xk), k 2 0 ,
xg = u/y*u, z = Xk+,
or, equivalently
a
is a
Ill Conilirinried Eigenproblerns
279
Proposition 3.1. The modified Newton’s iteration given by Equations (3.2) is equivalent to the inverse iteration on A.
Proof. The vectors qk and xk defined by these iterations are easily seen to be parallel. They correspond respectively to the normalizations
1) qk 11
= 1 and y*Xk = 1 .
The two methods are mathematically equivalent but not numerically. Indeed, Equations (3.2) are a defect correction method (see Stetter [1978]) and higher accuracy can be achieved if the residual F(xk) is computed with higher accuracy. (See Dongarra et a1 [1983] for an implementation). We now turn to a multiple eigenvalue A, with invariant subspace M. 3.2. SIMULTANEOUS INVERSE ITERATIONS.
Let u be close to A. Let U = [u,, . .. , urn] be a set of m independent vectors. The method of simultaneous inverse iterations can be written as the following recursion where k = 0,1,2,...
U = QoRO, Q*oQo = I and R, is a Lower triangular matrix (Schmidt factorization) (3.3) (A - ‘JI)Yk+i = Qk, Yk+l = Qk+iRk+i, k 2 0. For all the theorems stated in the rest of the paper, the proofs are omitted (for length’s sake). They can be found in Chatelin [1986].
Lemma 3.2. If spc(M) is moderate, the error incurred while solving (A
- 0I)Y = U
lies mainly in the wanted invariant subspace M.
Theorem 3.3. Let Q be an orthonormal basis in M. If h is defective, the m vectors which are solutions Z = {z,, ... ,]z,
of
are almost dependent.
Corollary 3.4. If h is defective the simultaneous inverse iterations are unstable.
F. Chatelin
280
Proof. The basis Yk computed by Equation (3.3) is almost degenerate, since the basis Qk tends
to Q.
0
3.3. SIMULTANEOUS NEWTON’S ITERATIONS
Let h be a multiple eigenvalue of A and let M be the corresponding invariant subspace of A
of dimension m. A basis X in M normalized by Y *X = I, is a solution of the quadratic equation F(X) = AX
- X(Y*AX)
= 0.
(3.4)
Conversely, if B = Y*AX is regular, the solution X of Equation (3.4) is a basis in M normalized by Y*X = I,.
B is an m
x
m matrix having a multiple eigenvalue. If h is semi-simple, B = hI,
but if h is defective, B is not normal. Remark. By a change of variable, Equation (3.4) can be transformed into an algebraic Riccati equation, often used in control theory (see Demmel[1985] and Arnold-Laub [1984]).
3.4. MODIFIED NEWTON’S METHOD
The following modified Newton’s method is mathematically equivalent to simultaneous iterations, see Equations (3.3), in the sense of Proposition 3.5 below:
x,= u,
Y * U = I, z = Xk+l - x, - OZ = - F(Xk), k 2 0
(3.5)
(1 - XkY*)AZ
Proposition 3.5. The bases
xk
and Qk computed respectively, by the methods in Equations (3.5)
and (3.3), span the same subspace (A - uI)-,S, where S = lin{U].
To study the convergence of the method defined by Equations (3.5), one may compare it with the simplified Newton’s iteration:
-
-
-
with B = Y*AU. We can interpret Equation (2.5) as the replacement of B in Equation (2.6) by 01. This is legitimate when
-
-
(1 B -
01 (1 is very small. Then the question arises: how well is B
-
approximated by oI? If B is diagonalizable, then we set B = WAW-’, and
-
(IB - 0111~ = II W(A - aI)W-I
I1
I cond,(W) mfx
I
hi - o
I
-
where sp(B) = Ih,, ... , h,).
Ill Conditioned Eigenproblems
28 1
The following proposition demonstrates however, that when A is defective, we cannot expect
-
to be able to approximate B by uI. In this proposition U is an orthonormal set of starting vectors and X is a basis for the invariant subspace corresponding to A. Proposition 3.6. If If A is defective and B diagonalizable, then cond,(W) is necessarily large Y
when
11 U - X 11
is small enough.
The main interest of the simultaneous iterations method given in Equations (3.3) is to keep fixed the matrix A - uI of the system to be solved at each step. This same purpose can be achieved by the following modification of the modified Newton’s method in Equations (3.5) which is stable when A is defective.
3.5. A STABLE MODIFIED NEWTON’S METHOD
-
As we have just seen, when A is defective, B is not well represented by 01 and this fact accounts for the instability of the simultaneous iterations method in Equations (3.3). We can however, obtain a stable method by introducing a SEhur decomposition.
-
Consider the Schur decomposition B = QTQ*, where T = diag (hi) + N, and N is strictly A
triangular.
We set T = uI
+ N,
A
-
A
A
11 B - B 11 = max I hi - u I . This 1 - U 11 is small enough. We are led to
and B = QTQ*, then
quantity is small for any value of cond2(W), when
11 X
the following modified Newton’s method which is stable when h is defective.
The Sylvester equation in (3.7) yields systems with the fixed matrix A
- 01 to solve (see Golub
A lrn et al. [1979] for an efficient algorithm). A natural choice for u in this context is h = x i f ; h i .
REFERENCES
[I]
AhuBs, M; Alvizu, J; Chatelin, F. (1986) Efficient computation of a group of close eigenvalues for integral operators in P m . IUACS World Congm Oslo, 4-9 August SS, North Holland, Amsterdam, (to appear).
F. Chatelin
Anselone, P.M.; RaH, L.B. (1968) The solution of characteristic value-vector problems by Newton’s method. Numer. Math. 11, 38-45. Arnold, W.; Laub, A. (1984) Generalized eigenproblem algorithms and software for the algebraic Riccati equations. Proc. IEEE, 72, (12). Bjorck, A; Golub, G.H. (1973) Numerical methods for computing angles between linear subspaces. Math. Comp. 27, 579-594. Chatelin, F. (1 984) Simultaneous Newton’s iteration for the eigenproblem. Computing, Suppl. 5,67-74. Chatelin, F. (1986) Valeurs Propres de matrices. Masson, Paris (to appear). Davis, C.; Kahan, W. (1968) The rotation of eigenvectors by a perturbation. 111. Siam J. Numer. Anal. 7, 1-46. Demmel, J. (1985) Three methods for refining estimates of invariant subspaces. Tech. Rep. 185, Computer Science Dept., Courant Institute, NY University, New York. Demmel, J; Kagstrom, B. (1986) Stably computing the Kronecker structure and reducing subspaces of singular pencils A - XB for uncertain data, these Proceedings.
[lo]Dongarra, J.J.; Moler, C.B.; Wilkinson, J.H. (1983) Improving the accuracy of computed eigenvalues and eigenvectors. SIAM J. Numer. Anal. 20, 23-45.
[ 111 Golub, G.H.; Nash, S.; Van Loan, C. (1979) A Hessenberg-Schur form for the problem AX
+ XB = C. IEEE Trans. Aut. Control AC-24.909-913.
[12] Golub, G.H.; Van Loan, C. (1984) Matrix commtations. North Oxford Academic, Oxford.
[ 131 Peters, G.; Wilkinson, J.H. (1979) Inverse iteration, ill-conditioned equations and Newton’s method. SIAM Rev. 21, 339-360.
[14] Rosenblum, M. (1956) On the operator equation BX - XA = Q. Duke Math. J. 23, 263-269.
[ 151 Stetter, H.J. (1978) The defect correction principle and discretization methods. Numer. Math. 29,425-443.
[ 161 Stewart, G.W. (1971) Error bounds for invariant subspaces of closed operators. SIAM J. Num. Anal. 8,796-808.
[17] Varah, J.M. (1979) On the separation of two matrices. SIAM J. Numer. Anal. 16, 216-222.
Large Scale Eigenvalue Problems J . Cullum and R.A. Willoughby (Editors) 0 Elsevier Science Publishers B.V. (North-Holland), 1986
283
Stably Computing the Kronecker Structure and Reducing Subspaces of Singular Pencils A - XB for Uncertain Data James Demmel
Courant Institute 215 Mercer Str. New York, NY 10012
USA
Bo K8gstrBm
Institute of Information Promsing University of Ume8 S-90187 Ume8 Sweden
We present algorithms and error bounds for computing the Kronecker structure (generalized eigenvalue and eigenvectors) of matrix pencils A - XB. Applications of matrix pencils to descriptor systems, singular systems of differential equations, and statespace realizations of linear systems demand that we accurately compute features of the Kronecker structure which are mathematically ill-posed and potentially numerically unstable. In this paper we show this can be accomplished efficiently and with computable error bounds. 1. Introduction
1.1 Motivation and short summary During the last few years there has been an increasing interest in the numerical treatment of general matrix pencils A-XB and the computation of the Kronecker canonical form, or KCF. The main reason for this interest is that in many applications, e.g linear systems theory [40], descriptor systems [1, 24, 311 and singular systems of differential equations [4],[46], problems are modeled in terms of linear matrix pencils. Given information about the KCF questions about the existence and the unicity of solutions, the state of a system or even explicit solutions can easily be answered. Algorithms have been proposed by several authors (see section 4) for computing the KCF. These algorithms share the property of backward stability: they compute the KCF (or some of its features discussed in section 1.2) for a pencil C - A D which lies some small (or at least monitorable) distance from the problem supplied as input, A-XB. A natural question to ask is how such a perturbation, measured as 11 ( A , B ) - ( C , D ) [ l E( be it roundoff, a user supplied estimate of measurement error, or a user supplied error tolerance), can affect the KCF or related quantities being computed. In particular a perturbation analysis where the size of the perturbation is a parameter (not necessarily small) is needed to rigorously determine the effects of the uncertainty in the data (see sections 4 and 5). It is not at all obvious that such a perturbation analysis is possible at all, since the mathematical problem is frequently unstable, so that arbitrarily small perturbations in the data may cause large changes in the answer. For some applications (e.g in linear systems theory) it is important to calculate unstable features of the KCF because they still contain pertinent physical information. In fact one sometimes wants to compute as unstable (nongeneric) a KCF as possible, because this in a certain sense allows low dimensional approximations to high dimensional problems, in particular low
284
J. Dernmel and B. Kbgstrom
dimensional approximate minimal realizations of linear systems ([39],[48], see also sections 2.3 and 2.4). In this setting one wants to know what nongeneric pencils lie within a distance E of a given pencil A -XB, in order to pick the one supplying the best approximation (see section 4.5). Before we go into further details we outline the rest of the paper. In section 1.2 we collect the basic algebraic theory of linear matrix pencils, for example regular and singular pencils, the KCF, minimal indices, and generalized subspaces (deflating and reducing subspaces). In section 1.3 we define other notation. Section 2 describes some applications where general matrix pencils A - XB appear, for example descriptor systems (section 2.1), singular systems of differential equations (section 2.2), and statespace realizations of (generalized) linear systems (sections 2.3 and 2.4). Section 3 considers algorithms for computing the Kronecker structure and its relevant features like reducing subspaces. The section starts with a geometric interpretation of a singular pencil in canonical form (KCF)and introduces the RGSVD and RGQZD algorithms ([19, 201) that are reviewed in section 3.2. In section 3.3 the transformation of a singular pencil to a generalized upper triangular form (GUPTRI) is discussed. A new more efficient implementation of the RGQZD algorithm, which is used as the basic decomposition in GUPTRI, is also presented. Section 3.4 is concluded with a short review of other approaches and algorithms, notably Kublanovskaya [16, 171, Van Dooren [39,41], and Wilkinson [46]. In section 4 the perturbation theory of singular pencils is formally introduced. The section begins by reviewing the perturbation theory for regular pencils (section 4.1). Section 4.2 discusses why the singular case is harder than the regular case. Perturbation bounds for pairs of reducing subspaces and the spectrum of the regular part are presented (sections 4.3 and 4.4). For a more complete presentation and proofs the reader is referred to [ll]. Finally in section 4.5 we make some interpretations of the perturbation results to linear systems theory by deriving perturbation bounds for the controllable subspace and uncontrollable modes. We illustrate the perturbation results with two numerical examples. Armed with the perturbation analysis for singular pencils, section 5 is devoted to analyzing the error of standard algorithms for computing the Kronecker structure from section 3. An upper bound for the distance from the input pencil A-XB to the nearest pencil C - A D with the computed Kronecker structure and perturbation bounds for computed pairs of reducing subspaces are presented (sections 5.1 and 5.2, respectively). In section 6 we present some numerical examples (one generic and one nongeneric pencil) and assess the computed results by using the theory from sections 4 and 5. Finally in section 7 we give some conclusions and outline directions for our future work. 1.2 Algebraic theory for singular pencils In this section we outline the algebraic theory of singular pencils in terms of the Kronecker Canonical Form (KCF). Suppose A , B, S and T are m by n complex matrices. If there are nonsingular matrices P and Q ,where P is m by m and Q is II by n, such that A-XB = P (S-AT) Q-' (1.1) then we say the pencils A-XB and S-AT are equivalen~and that P ( . ) Q - l is an
Stably Computing the Kronecker Structure
285
equivalence transformation. Our goal is to find P and Q so that S-XT is in a particularly simple form, block diagonal: S=diag(Sll, . . . ,Sbb) and T=diag(TI1, . . . ,Tbb).We can group the columns of P into blocks corresponding to the blocks of S-AT: P = [P1I * . I Pb] where Pi is m by mi, mi being the number of
rows of Si,-XT,,. Similarly, we can group the columns of Q into blocks corresponding to the blocks of S-AT: Q=[Qll . lQb]where Q,is n by n,, ni being the number of columns Of S,,-A Ti,. The diagonal blocks Sii- ATIi contain information about the generalized eigenstructure of the pencil A-XB and P, and Q, contain information about the corresponding generalized eigenspaces. One canonical decomposition of the form (1.1)is the Kronecker Canonical Form [12],where each block Sii-ATii must be of one of the following forms:
- -
This is simply a Jordan block. ho is called a finite eigenvalue of A-XB
This block corresponds to an infinite eigenvalue of multiplicity equal to the dimension of the block. The blocks of finite and infinite eigenvalues together constitute the regular part of the pencil.
This k by k+ 1 block is called a singular block of minimal right (or column) index k. It has a one dimensional right null space for any X .
This j + 1by j block is called a singular block of minimal left (or row) index j . It has a one dimensional left null space for any A . The left and right singular blocks together constitute the singular part of the pencil. If a pencil A-XB has only a regular part, it is called regular. A-AB is regular if and only if it is square and its determinant det(A-XB) is not identically zero. Otherwise, there is at least one singular block Lk or L,’ in the KCF of A - XB and it is called singular. In the regular case, A-XB has n generalized eigenvalues which may be finite or infinite. The diagonal blocks of S - AT partition the spectrum of A - XB as follows:
286
J. Demrnel and B. Ktigstrorn
UEU(A-AB) =
b
U u(S,,-AT,I) i=1
b
U U, .
1=1
The subspaces spanned by PI and Q, are called left and right deflating subspaces of A-AB corresponding to the part of the spectrum u, [32, 411. As shown in [41], a pair of subspaces P and Q is deflating for A-AB if P=AQ+BQ and dim(Q)=dim(P). They are the generalization of invariant subspaces for the standard eigenvalue problem A-AZ to the regular pencil case: Q is a (right) invariant subspace of A if Q=AQ+Q, i.e. AQCQ. Deflating subspaces are determined uniquely by the requirement that S-AT in (1.1) be block diagonal: different choices of the Pi or Qlsubmatrices will span the same spaces P,and Q1. The situation is not as simple in the singular case. The following example shows that the spaces spanned by the P, and Q,may no longer all be well defined:
As x grows large, the space spanned by Q2 (the last column of Q) can become arbitrarily close to the space spanned by Ql (the first two columns of Q). Similarly the space spanned by P z (the last column of P) can become arbitrarily close to the space spanned by P1 (the first column of P). Thus, we must modify the notion of deflating subspace used in the regular case, since these subspaces no longer all have unique definitions. The correct concept to use is reducing subspace, as introduced in [41]. P and Q are reducing subspaces for A-AB if P=AQ+BQ and dim(P)=dim(Q)-dim(N,), where N , is the right null space of A - AB over the field df rational functions in A. It is easy to express dim(Nr) in terms of the KCF of A-AB: it is the number of L, blocks in the KCF [41]. In the example above, N , is one dimensional and spanned by [1,A,OIT. In this example the nontrivial pair of reducing subspaces are spanned by P 1 and Ql and are well defined. In terms of the KCF,we may define reducing subspaces as follows. Assume in (1.1) that S-AT is in KCF,with diagonal blocks Sll-hTll through &-ATrr of the tfle Lk (right blocks) s r + l,r + l-AT[+ 1,r + 1 through s r + reg ,r+ r e g - A T r + reg,r+ reg regular blocks (Jordan blocks or blocks with a single infinite eigenvalue), and the remaining S,-ATl blocks of the type LF (left singular blocks). Assume for simplicity that there is one 1 by 1 regular block for each distinct eigenvalue (finite or infinite). Then there are exactly 2"g pairs of reducing subspaces, each one corresponding to a distinct subset of the reg eigenvalues. If P,Q are such a pair, Q is spanned by the columns of Ql, . . . ,Qr and those Q, that correspond to the eigenvalues in the chosen subset. P is spanned by the columns of P1,. . . ,Pr and those PI corresponding to the chosen eigenvalues. The smallest subset of eigenvalues is the null set, yielding P and Q of minimum possible dimension over all pairs of reducing subspaces; this pair is called the pair of minimal reducing subspaces. Similarly, the largest subset of eigenvalues is the whole set, yielding P and Q of maximal dimension; this pair is called the pair of maximal reducing subspaces. These pairs will later play a central role in applications. In any event, given A-AB, any pair of reducing subspaces is uniquely identified by specifying what subset of eigenvalues it corresponds to. We illustrate these concepts with the following 5 by 7 example: P-'(A - XB)Q = diag(Lo,LIJz(0),2.Jl(m),LT) =
Stably Coinputing the Kronecker Structure
287
7
.A 1
-A 0
1 -A
1
1 -A
1 Letting el denote the unit column vector with a 1 at position i and zeros elsewhere, we see that the minimal reducing subpaces are given by Qmin
= span{el,ez,e31 and
Pmin
= span{ed
and the maximal reducing subspaces are given by Q, = span(e1,ez,ej,e4,eg,e6,e7} and PmX= span{e1&2,e3&4&5). By choosing two different subsets of the spectrum of A--XB, we may get different pairs of reducing subspaces of the same dimensions. Thus Q = span{el,e2,e3,eq,ed and P = span(el,e~,e3} correspond to the double' eigenvalue at 0, and
Q = s p a n ( ~ l , e ~ w - w and ~ ) P = spadel,e4&5} correspond to the double eigenvalue at m. This concludes our summary of the purely algebraic properties of pencils. Since w e are interested in computing with them as well, we will now describe some of their analytic properties, and explain why computing the KCF of a singular pencil may be an ill-posed problem. Suppose first that A-AB is a square pencil. It turns out almost all square pencils are regular, in other words almost any arbitrarily small perturbation of a square singular pencil will make it regular. We describe this situation by saying that a generic square pencil is regular. Similarly, if A-AB is an m by n nonsquare pencil, it turns out almost all m by n pencils have the same KCF, in particular they almost all have KCFs consisting of Lk blocks alone (if m
288
J. Demmel and B. Ktigstrom
a:=O if ui=O. K ( A )will denote the condition number umm(A)/umin(A)of the matrix A ; this applies to nonsquare A as well. A@B will denote the Kronecker product of the two matrices A and B :A O B = [Ai, B ] . Let COlA denote the column vector formed by taking the columns of A and stacking them atop one another from left to right. Thus if A is m by n , colA is mn by 1 with its first m entries being column 1 of A , its second m entries being column 2 of A , and so on.
2. Matrix pencils in some applications 2.1 Descriptor systems One field of applications is concerned with the solution of the linear implicit system of differential equations Bx '( t )=,4x ( t )+ f ( r ) , x (to)=xo
(2.1)
where A and B are n by n (complex) matrices, possibly singular, f ( t ) is an n component input vector, x ( t ) is an n vector, and to is the initial time with the corresponding initial vector X O . In some applicationsf(t)=Cu(t), where C is an n by k (complex) matrix. Such systems are called descriptor systems [25] since they arise from formulating the system equations in natural physical variables. Implicit or descriptor systems occur for example in control problems (e.g the basic quadratic cost problem), singular perturbation problems and when modeling impulsive behavior of electrical circuits (see [4] for more details, examples and further references). If B is nonsingular then the system (2.1) can be transformed to an explicit system x' ( t ) =Ex ( t )+g( t ) , x (to)=x0 (2.2) where E=B-'A and g = B - ' f , which is the usual linear state model. However if B is ill-conditioned with respect to inversion this transformation will introduce instabilities into the model, and should therefore be avoided. A characterization of the general solution of (2.1) is provided by the KCF of the pencil A-XB ( see [12] and [46]). By premultiplying this system by P - l and introducing new variables Y ( x = Q v ) we can assume that A-XB is in KCF (see equation 1.1). The solution of the equivalent system P - b Q v '( t )=P - 'AQv ( t )+P-'Cu ( t ) , Y( to)=P-?x0 (2.3) then reduces to solving smaller systems determined by the diagonal blocks of the KCF. If A-XB is regular then systems like (2.1) are called solvable because solutions to the descriptor system exist and two solutions which share the same initial value (admissible initial condition, see below) must be identical [31]. So in the following we assume that A-XB is regular and by collecting all blocks in the KCF of A-XB corresponding to the finite and infinite eigenvalues in J and N , respectively, i.e
(2.4)
where y = kf,y$]', it is easy to verify that the solution of (2.4) is given by (2.5a)
(2.5b)
Stably Cornpiiting the Kronecker Structure
289
Here m is the size of the largest Jordan block corresponding to the infinite eigenvalue (in the KCF) of A-XB, i.e N is nilpotent of degree m , and is called the (rdpotency) index of the system. Transformation methods and methods based on the explicit solutions (2.5a-b) do not preserve any sparsity structure of the problem and can of course only efficiently be used for systems that can entirely be handled in the main memory of the computer. Implicit systems are also called diff~ential-algebraic(D/A) systems and from (2.5a-b) we see that the solution of (2.1) includes one part (2.5a) corresponding to the solution of an explicit system (the Dpart) and one part (2.5b) corresponding to sufficiently differentiable (at least m - 1 times) functions u (the A-part). A simple example of an index 3 system with only an A-part is
k 8 :I I;:] t :4:;1 + [cil 0 1 0
Y1’
1 0 0
Y1
=
which has the solution y1 = - c r r ( t ) , y 2 = - c ' ( t ) and y 3 = - c ( t ) . Recently the problem of solving D/A systems has attracted many researchers. Sincovec et al [31] consider difference methods for solving large scale descriptor systems (2.1). Normally we do not know the index of the system (2.1) a priori, so one important part of the problem is to find admissible initial conditions satisfying (2.5b). They propose to use the forward Euler method integrating backwards in time. However in order to use the method they have to know an upper bound for the index of the descriptor system. A backtracking procedure is suggested to resolve that problem (see [31] for details). More general implicit systems are studied by Gear and Petzold [13, 141 and Brenan 131, notably linear systems (2.1) with time-varying coefficients in [13]. For a fixed time the solution can be expressed as in (2.5a-b), but normally the local index (index at a fixed time) changes with time and the behavior of the solution is determined by a global nilpotency index. The above mentioned papers all use the backward difference formulas (BDF) and solve a nonlinear system of equations at each time-step. Campbell [5] also considers systems (2.1) with timevarying coefficients and presents a family of explicit Taylortype methods. It is worthwhile to mention that existing numerical methods for solving linear time-varying D/A-systems can reliably solve only index 1systems, some index 2 systems and very few higher index systems [14]. 2.2 Singular systems of differential equations When det(A-XB)=O or the matrices A and B are rectangular then the pencil A-XB is singular and the corresponding system (2.1) is called a singufur system [46]. The KCF of A-XB will then (except for a possible square regular part) contain nonsquare diagonal blocks corresponding to the minimal indices of A-XB (see section 1.2), and as we will see the singular system (2.1) is not solvable in the sense of the previous section. The nature of the solution corresponding to the variables associated with the regular part will be as before (see section 2.1), so it remains to deal with variables associated with minimal indices. Without loss of generality we do this by considering a left and a right index of degree 2, respectively. Example 2.1 - right index 2 corresponding to a an L2 block
J. Demmel and B. K6gstrom
290
or equivalently, XI’
x2’
+ f1 = x3 + f2
= x2
and we see that x 3 may be chosen to be an arbitrary function and nl,x2 are determined by quadratures. Example 2.2 - left index 2 corresponding to an Lq block
or equivalently, x1’ = f l x2’ = X I + f2
0
= x2+f3
and we see that the variables are completely determined by the compatibility relation
fi = -f2
- f3 .
There are analogous results for Lk blocks and L f blocks of any size, including k=O. In summary, the characteristic parts in the solution of (2.1) introduced by singularity are: (i) the equations corresponding to an Lk block have solutions for all right hand sides f but these are not completely determined by the driving function f or the initial data, and (ii) for. the equations corresponding to an L[ block there is a compatibility relation which the components off must satisfy if a solution is to exist. So a singular system of differential equations may or may not have a solution, and can even have infinitely many solutions dependent on the KCF of the underlying pencil A-XB. 2.3 State-space realizations of linear systems Frequently it is not possible to measure all the state variables x in the linear system of (2.2), but instead one has to measure linear combinations of the state variables. In this case the linear state model (2.2) is extended to the general state space representation x ’ ( t ) = &(t)
+ Cu(t)
(2.6a)
(2.6b) where as before x and u represent the state variables and controls (inputs), respectively, A is an n by n matrix and C an n by k matrix. The p component vector y represents the outputs and D is a p by n matrix. Often the model is further extended by adding a feedthrough term Fu(t) to the outputs in (2.6b). System theorists are interested to know if the system (2.6) is controllable and/or observable. The system (2.6a) is said to be completely controllable if we can choose the Y(t> = W t )
Stably Computing the Kroizecker Structure
29 1
inputs u to give the states x any value in a finite time, and algebraically it is equivalent to the controllable subspace C(A,C) R[CBCB2CI * * h"-'C]
-
having full rank n. Wonham [48] gives a geometric characterization of C(A,C) as the smallest invariant subspace of A containing the range of C. Observability is the dual concept to controllability and the system (2.6) is said to be completely observable if we can determine the states x of the system at time to from the knowledge of the outputs y over a finite time interval [to,tl]. This decision can algebraically be expressed in terms of the unobservable subspace
namely (2.6) is cozpletely controllable if and only if C(A,D)={O}. The geometric interpretation of O(A,D) is the largest invariant subspace of A included in the nullspace of D [48]. Recently there has been much interest in computing the controllable and observable subspaces (see e.g Boley [2], Paige [28] and Van Dooren [40] which contains further references to the control literature). C(A,C) and O(A,D) can also be characterized in terms of matrix pencils [40]. Define the pencils
Then C(A,C) is the finimal left reducing subspace of the-pencil-Al-XBl (see section 1.2), and similarly O(A,D) is the maximal right reducing subspace of the pencil A2-XB,. Since B,=[O,Z] is of full row rank the KCF of A1-XB1 can only have finite eigenvalues and Lk blocks (right minimal indices) and no infinite eigenvalues or Lf blocks. A dual result holds for A2-XB2, i.e since B2 is of full column rank the KCF of A2-XB2 can only have finite eigenvalues and L f blocks. HAl-XB1 andA2-XB, have no regular part then the system (2.6) is said to be completely controllable and observable, respectively. Also the eigenvalues of the regular part of A1-XB1 are of interest in designing control systems. They are called the input decoupling zeroes or uncontrollable modes and are the eigenvalues of the system (2.6a) which cannot be controlled by any choice of feedback u=Kx. Note that similar results hold for the discrete-time versions of the continuous-time systems (2.6). 2.4 State-space realizations of generalized linear systems
By replacing the explicit linear model (2.6a) with a descriptor system we get a generalized state-space realization [45]
+ Cu(t) y(t) = Dx(t) + Fu(t)
Bx'(t)
= Ax(t)
(2.7a)
(2.7b) where B is an n by n matrix. The interesting cases are when B is singular or almost
292
J. Demmel and B. Kdgstrom
so, since otherwise the generalized system can be transformed to a system of the state-space form (2.6). Further we assume that the pencil A-XB is regular so for admissible initial conditions the system of equations (2.7) have a unique solution (see section 2.1). The generalized state-space systems (or descriptor systems, singular systems) allow considerably more generality since the transfer funcrion G(s)=D(A-sB)-'C + F from u to y is not necessarily strictly proper (i.e. G ( s ) 0 as s OD).However in these cases G(s) can be written as the sum of a strictly proper part Gl(s) and a polynomial part F ( s ) [45]. By knowing the KCF of the regular pencil A-XB we can separate the causal part (finite eigenvalues) from the noncausal part (infinite eigenvalues) of the system (2.7): GI($) = D,(J-sZ)-'Cr (strictly proper) (polynomial in s) ~ ( s ) = D,(z-sN)-'c~
-
-.
The concepts of controllability and observability are generalizable to descriptor systems. These concepts are of particular interest for the natural system frequencies at infinity (for a thorough discussion see e.g. [l, 7, 241). The infinite modes are separated into dynamic and nondynamic modes, where a nondynamic mode at infinity is a direction of the descriptor vector x in which there is a purely algebraic relationship between x and the input u and output y . All other infinity modes are called dynamic (or impulsive males) and are directions in which x can exhibit impulsive behavior due to an initial condition with zero input. These modes can directly be identified from the KCF of A-XB; the nondynamic modes are associated with the 1 by 1 Jordan blocks in N and the dynamic modes with Jordan blocks of size at least 2. A straight forward generalization of controllable and unobservable subspaces of statespace systems to generalized statespace systems corresponding to both finite and infinite modes of the system (2.7) is proposed by Van Dooren [40]and he defines C(B,A,C) = inf{X : Y=AX+BX , dim(Y) = dim(X) , R(C)CY} g(B,A,D) = sup(X : Y=AX+BX , dim(Y) = dim(X) , XGN(D)} For B = l we see that these definitions coincide with the controllable subspace C(A,C) and the unobservable subspace O(Ap), respectively, of the state space system (2.6). In words the above geometric characterization says that C@,4,C) is the smallest deflating subspace of A-AB containing the range of C, and O(B,A,D) is the largest deflating subspace of A -XB included in the nullspace of D . These subspaces can also be expressed in terms of matrix pencils. Define the pencils
then C ( B d , C ) is the minimal left reducing subspace o f t h e pencil A1-AB1, and similarly O ( B , A p ) is the maximal right reducing subspace of the pencil A2-XB2. The reducing subspaces corresponding to controllable and unobservable subspaces of state-space or descriptor systems can be computed by applying the algorithms in section 3 to the pencils A1-XB1 and A2-XB2, defined in sections 2.3 and 2.4, respectively. It is also possible to take advantage of the special structure of B , and Bz, respectively, when computing these subspaces [40]. Our perturbation theory for singular pencils (presented later) can also be applied to singular A,- XB, and A2- hB2, respectively, giving perturbation bounds for controllable and unobservable subspaces (see section 4.5).
Stably Cornputirig the Kronecker Structure
293
3. Algorithms for computing the Kronecker structure and pairs of reducing subspaces 3.1 A singular pencil in canonical form - a geometric interpretation In the study of the the Kronecker structure of a singular pencil A - XB the column and row nullities of A and B and their possible common nullspaces play an important role. For example, parts of the singularities in A and B may correspond to zero and infinite eigenvalues of A - X B , and if A and B have a common nullspace there exist minimal right (column) indices in the KCF of A-XB. We illustrate the case of a common column nullspace by considering an m =3 by n = 6 singular pencil, already in canonical form [19]:
I
I
1 0 0 0 0 0 0 0 0 = diag(J2(0),L0,L0,L1) (3.1) 0 0 0 - A 1 Its KCF consists of one 2 by 2 Jordan block with eigenvalue 0 ( J 2 ( 0 ) ) , two right indices of degree 0 (Lo) and one right index of degree 1 (L1).Notice that the 0 by 1 blocks LO only contribute to the column dimension of A-AB. Studying A and B separately and permuting A and B so that all zero-columns of A are the leading ones we obtain: -A
A-XB =
0 0
-A
4
0 0 0 0 1 0
A'=
0 0 0 0 0 1 [o 0 0 0 0
(3.2a)
(3.2b)
The permutations made can be summarized as follows: the LI block is retrieved from columns 3, 5 and row 1, and the J2(0) from columns 4, 6 and rows 2, 3. Finally the two Lo form columns 1and 2. Let n1 denote the column nullity of A and rl be the dimension of the column nullspace of A which is not common to B . Directly from (3.2a-b) we see that n1 is 4 and that A and B have a common column nullspace of dimension 2 (nl-rl), which is the number of Lo blocks in the KCF. We also see that r l = 2 is equal to the number of structure blocks Jk(0) or L, of order k greater than or equal to one. More information about these blocks can be retrieved by first deflating A-XB at row rl and column n l and then analyzing the remaining pencil [0 01-XIO 11 in a similar way. We see that the column nullspace of the new A has dimension n2=2 and the common column nullspace of A and B has dimension l=n2-r2 (r2= l), which now gives the number of L1blocks. Generally we have nl(=4) 2 r1(=2) 2 n2(=2) 2 r2(=1) and r1-n2 is the number of Jl(0) blocks in the KCF, here none. After deflating [0 01-XIO 11 at row r2 and column n2 we are left with the "empty" pencil, and therefore define n3=0. Now r2-n3=l gives the number of J2(0) blocks in the KCF, and we are finished. The above geometric inte retation of the Kronecker structure holds for arbitrary pencils A-XB. Let A(l-l)-ABT-l)denote the deflated pencil from step i-1 (i=1,2, ... andA(O)=A, B(O)=B) which will be studied next, and let
J. DernrneI and B. Kdgstrom
294 nt =
dimN(A('-l))
and
q-ri
=
dim (N(A(i-l))nN(B(f-l)))
.
(3.3)
In general these column nullities expose the Jordan structure of the zero eigenvalue and the right indices of a singular A-XB in the following way. Theorem 3.1: (Geometric interpretation of the structure blocks). The number of Lt blocks equals ni - r, and the number of Ji(0) blocks equals ri -nf+ 1. For a proof see [i9],[39] or [46]. We can also extract the left (row) indices similarly by considering row nullities of A and B , or equivalently column nullities of AT and BT. If we interchange the roles of A and B (i.e. B - pA is considered) instead of the Jordan structure of the zero eigenvalues we now obtain the Jordan structure of the infinite eigenvalues of A-XB. So it is possible with entirely geometric arguments t o extract the Jordan structures of the zero and infinite eigenvalues and the right and left minimal indices. In the next section we make use of this geometric interpretation of the KCF to formulate algorithms for computing the indices nl and r, and transforming (A,B) to a form equivalent to (3.2a-b) via a sequence of equivalence transformations of A - XB. 3.2 The RGSVD and RGQZD algorithms One way of doing a column rangelnullspace separation of a matrix pair (A$) (i.e. to compute n1 and r1) is via the generalized singular value decomposition (GSVD) and we start by formulating the GSVD of the m by n matrices A and B where m r n [19]. The case m
[ u* o v*] 0 FIX=
where D A ,DBE C"
I:$ -
are given by
(3.4) -
(3.5a)
DB
C=diag(cl,c2,
= [OxB]
(3.5b)
I
. , . , c r ) , O S C ~ S C ~ S 1c,11 . . . , s r ) , l r s 1 r s 2 2- . . z s r r O *
*
(3.6a)
(3.6b) S=diag(sl,s2, satisfying C 2 + S 2 = I . Here X A and X B are m by n - r . Since we only need V and X (or U and X) in an equivalence transformation of A-XB, we do not come into the numerical difficulties one encounters when trying to compute the complete GSVD. (See [30, 34, 35, 42, 431. In [36, 371 Sun presents a perturbation analysis of the GSVDproblem, and an improved version of one of his results using straightforward matrix ideas is presented in [29].) By knowing the GSVD (3.4) of A and B we can write A =U
and
01
oc [o
X-l
(3.7a)
295
Stably Coinputirig the Kronecker Structure
(3.7b) Let nl and rl be defined as in equation (3.3). Then the first rl ci VLJS must be zero and therefore the cci-responding si values must be ones (C2+S2=Z). By multiplying A and B in {3.7a-5) with X from the right we see that the columns xi of X span the column nullspace of A and the common nullspace of A and B in the following way: N ( A ) = s P ~ ~ ( ~ I J z* J.~. ,jxnJ
--
N(A)nN(B) = s ~ a n ( x 1 ~ 2 , . nxnl-rl) Now we can make use of V and X from the GSVD of (A$) transformation:
in an equivalence
where M1 and Al are full matrices and B1 is diagonal. In fact the transformed B part of (3.8) is I), in the GSVD (see eq.(3.5b)). Further the right hand side of equation (3.8) is in a form similar to A’ and B’ in equations (3.2a-b). This is the first step in a reduction theorem that apart from the Jordan structure of the zero eigenvalue computes the right minimal indices, similar to what was illustrated in the previous section. In the next step we do a GSVD of ( A l p l ) and decompose Al-XB1 similarly with an equivalence transformation. The deflation process continues until n,=O (dimN(A)=O) or n,#O but r,=O (dim (N(A)nN(B))=O). At the same time we obtain an associated pair of reducing subspaces. We summarize the reduction in the following theorem [20]. Theorem 3.3: Reduction based on GSVD. Let A and B be m by n complex matrices. Then there exists a unitary matrix VECmXmand a nonsingular matrix X € C n X nsuch that (3.9a) where Aor = 0
M1
rl r2 r3
296
J. Demmel and B. Kigstrcm
and BOr= O
0
I
0
Here I,, denote the rk by rk identity matrix. The indices ni and ri satisfy nl 2 r l L n2 2 r2 L * * 2 n, 2 r, = ns+l = 0
-
and provide part of the K c F as follows: nk-rk is the number of Lk-1 blocks and rk-nk+l is the number of Jk(0) blocks. The remainder of the KCF is contained in A,-XB,. We can now apply Theorem 3.3 to A - XB and B* - pA*, respectively, as discussed in section 3.1 and we get the reiterating generalized singular value deflation (RGSVD)algorithm for computing the Kronecker structure of a singular matrix pencil 1201*
It is also possible to do a column (or row) rangdnullspace separation of (A$) in terms of a unitary equivalence transformation that displays nl and n l - r l similarly to GSVD. In [19] a generalized QZ decomposition (GQZD) is presented. Theorem 3.4 (GQZD):Let A$ECmXn where m r n and r=rank[A*$*]*ln. Then there exist unitary matrices U,VECmXm and a unitaq matrix QECnXnsuch that
[Y *;] where TA ,TBE C"
F] E]
(3.10)
Q =
are given by (3.1la) (3.llb)
R , and R, ere upper triangular and satisfy IIRc*Rc + RS*R#IIE = I I W E where Z1is the diagonal matrix of singular values of [A* ,B*]*.
(3* 12)
By comparing the GSVD and GQZD we see that the unitary Q (3.10 and the norm-identity (3.12) take the place of X and the cosine-sine identity (C +S2=Z), respectively in Theorem 3.2. If the first rl ci7sin the GSVD of (A$) are zeros then the first rl columns of Rc will be zero columns [19], so from (3.10) we see that the n l - r l first columns of Q span N(A)r) N(B) and the n1 first columns of Q span N(A). If we make use of V and Q in an equivalence transformation similar to (3.8) we get
1
Stably Computing the Kronecker Structure
297
(3.13)
where Rll is rl by tl, upper triangular and of full rank and B1 has only nonzero elements on and above the diagonal starting at the top left corner. M I , A1 and N1 are full matrices. In the next step we now continue with a GQZD of (Al,Bl) and decompcse A 1 - U 1 similarly and so on. In summary if we replace GSVD by GQZD in the RGSVD algorithm we get the RGQZD algorithm for computing the left and right minimal structures and the Jordan structure of the zero and infinite eigenvalues, respectively (for details see [20]).
As we will see in section 4 the probIem of computing the KCF of a nongeneric pencil is generally ill-conditioned. For example almost all (arbitrary small) perturbations of a nongeneric pencil will turn it into a generic one. However if we make a regularization of the problem by restricting the perturbations in A and B such that the unperturbed and perturbed pencils have minimal and maximal pairs of reducing subspaces of the same dimension, respectively, we can turn the problem to a well-conditioned one. (For more details see sections 4 and 5 . ) So our aim should be to minimize the influence of rounding errors in the computations. In the coming section we therefore use only unitary equivalence transformations in the reduction process to compute the Kronecker structure. 3.3 GUPTRI Transformation to a generalized upper triangular form We will now reduce a singular pencil A-XB to a generalized upper triangular (GUPTRI) form in terms of a unitary equivalence transformation such that
-
(3.14)
P*(A-XB)Q =
-A,-XB, = -
-
*
0
A*-XBo
0 0
0 0
0
0
* * Af-XBf
0 0
*
* * Ai-XBI
0
* * *
* Ai-XB,
where A,- AB, has only the right minimal indices (the L, blocks) of the KCF of A - AB in its KCF. AI- ABl has only the left minimal indices (the L f blocks) of A -XB AreE-ABre8 is the regular part of the KCF of A-XB, and consists of the following three subpaxts: Ao-XBo has the Jordan structure of the zero eigenvalues of A-XB, A,-AB has the nonzero finite eigenstructure of the regular part of A-XB in any desiredorder (det(Af)# 0 and det(Bf) #O), Al -ABI has the Jordan structure of the infinite eigenvalues of A -XB. The diagonal blocks A,-AB, and Al-hB, are block upper triangular and the three diagonal blocks corresponding to the regular part of the pencil A,,,-XB,, are all
.
J. Demmel and B. KLgstrBm
298
upper triangular (in real arithmetic Af-XBf may have two by two diagonal blocks corresponding to pairs of complex conjugate eigenvalues). The stars (*) in (3.14) denote full matrix pencils of appropriate sizes. Further reduction of GUPTRI to a block diagonal form will rely on nonunitary equivalence transformations (see section 4) * Note that A-XB must be nongeneric in order to have both right and left indim, or indices of either kind and a regular part in its GUPTRI form. Since a square generic pencil is regular it cannot have any of the blocks A,-hBr and AI-XBl in its GUPTRI form. If A-XB is a generic and rectangular pencil where mn) it reduces to AI-XBl. (See also sections 1.2 and 4.2.) Since the user controls the ordering of the eigenvalues in Af-XBf it is possible directly from the GUPTRI decomposition to produce a pair of reducing subspaces associated with any part of the spectrum a ( A - X B ) of the regular part of A-XB. By partitioning P and Q as QI] and P = [Pr ,Preg P I ] we see that the minimal and maximal pairs of reducing subspaces, respectively (see section 1.2) are P m i n = S P d P r ) and Qmin = s~an(Qr) and Pmax = sPan([Pr > pregl) and Qmax = SPan([Qr QregI)
Q
=
[Qr
3
Qreg
9
3
3
A further partition of Q,,=[Qo example that
, Qf , Q l ] and PrCg=[Po, P, ,P I ] gives us for
P = spm([Pr Po ,PfI) and Q = Span([Qr Qo QfI) is a pair of reducing subspaces corresponding to the finite eigenvalues of A-XB. If, for example, we want a pair of reducing subspaces associated with the right minimal indices and all eigenvalues in the left complex plane Re X C O (stable modes in connection with linear systems, see sections 2.34) we ask for an ordering of the eigenvalues of Af-ABf such that these eigenvalues will be in the upper leftmost (north-west) corner of A f - X B f . Then we just have to partition Q, and Pr accordingly in order to produce a pair of reducing subspaces associated with the stable modes. However if for any reason we want to mix the zero and/or the infinite eigenvalues together with the nonzero eigenvalues the computed Jordan block structure of Ao-XBo and Al-XB, will be destroyed. Such a reordering can optionally be imposed directly when reducing A-XB to GUPTRI form or by postprocessing. The GupTRl decomposition (3.14) is based on the RGQZDalgorithm (see section 3.2). It utilizes two variants of a GQZD version of the reduction theorem 3.3. The first one, RzsI1I, is a new more efficient implementation of this reduction theorem and computes the right minimal indices (structure) and the Jordan structure of the zero eigenvalues of A -XB. One step of deflation in RZSTR is computed in the following way. Let A(')-XB(') denote the pencil we are to decompose in step i via the unitary transformations PI and Ql such that P~*(A(')-XB('))Q,=
We will use the following notation in the algorithm below:
Stably Computing the Kronecker Structure
299
:= svd(A) computed the singular value decomposition A = KEV* of A with the singular values on the diagonal of X = diag(ak, . . . ,al) in decreasing order. If one of U,X or V is "nil", that means it need not be computed. n := coluxnr~uility@,epsu)is the number of singular values of X which are less than or equal to epsu. < Q p > := qr(A) computes the QR decomposition A=QR. Algorithm RZSTR-kernel: Step 1. Compress the columns of A(i) to give ni 1.1 Cnil,Xf) ,VL’)>:=svd(A (i)) ;n, :=column-nullity(Xf) ,epsu) 1.2 Apply V"' to A(') and B(,): [0 A2]:=A(')V''); [B1B2]:=B(')Vf) (B1and the zero block to the left of A2 each consist of n, columns) Step 2. Compress the columns of B1to give n f - q 2.1 C nil ,Xi[) ,Vh’)> :=svd(B ;n,- r,;=column-nullity(Ei') ,epsu) to B1 and build Q,, the final right transformation in step i: 2.2 [0 B o ] : = B l V ') (Bo has rf columns)
4
Q,:= Vi')
r):]
Step 3. Triangularize [Bo ,B2]to give P f , the final left transformation in step i 3.1 := qr([Bo,Bz])
r "I
0 B ( f + l ) := Rf
3.2 Apply Pi to A2
Next we apply the RZSTR-kernel to A('+l)-XB('+l) and so on until nlfl=O or n,+l#O but r,+l=O, as in theorem 3.3. In steps 1.1 and 2.1 above we use epsu (a bound on the uncertainty in A and B that is given by the user) as a tolerance for deleting small singular values when computing nullities. All singular values smaller than epsu are deleted, but we always insist on a gap between the singular values we interpret as zeros and nonzeros, respectively. In section 5 we further discuss the influence of finite arithmetic to computed results. The second decomposition LISTR computes the left minimal structure and the Jordan structure of the infinite eigenvalues similarly to RZSTR b;r working on B - pA. The main difference is that column compressions are replaced by row compressions and we start working from the lower rightmost (south-east) corner of the pencil. As we said earlier it is also possible to apply RZSTR to B* - pA* to get the left minimal and infinity structures (see section 3.1). However RZSTR working on B*-pA* will produce a block lower triangular form of A-XB, so in order to keep GUPTFU block upper triangular we make use of LISTR. In summary the GUPTFU decomposition is computed by the following 6 reductions: 1: Apply RZSTR to A-XB. Gives the right minimal structure and the Jordan structure of the zero eigenvdue in A,,- XB,,:
ror-ABor* I
J. Dernmel and B. Khgstrom
3 00
0
P1*(A-hB)Q1 =
A1-XB1
The block structure of Ao,-hBo, is exposed by the structure indices nf and rf (column nullities, see theorems 3.1 and 3.3). 2: Separate the right and zero structures by applying RZSTR to Bo,-pAo,. Gives the dght minimal structure in A,-XB,:
YBr* I 0
P z * ( A o ~ - X B O ~ )=Q ~
A ~ - X B ~
Insist on the same right minimal indices as in reduction 1. 3: Apply RZSTR to A2-XB2, which contains the zero eigenvalues of A-XB, giving P ~ * ( A z - X B ~ )= Q ~Ao-hBo Note that reduction 2 destroys the block structure of the zero eigenvalues, but this reduction gives it back by insisting on the same Jordan structure as in reduction 1. Reductions 1 to 3 in summary:
The pencil Al-AB1 comes from reduction 1 and contains the possible left minimal structure and remaining regular part of A -XB. 4: Apply LISTR to A1-XB1. Since A1-XB1 cannot have any zero eigenvalues the reduction produces the block structure associated with &e left indices inAl-ABl:
5 : Apply LISTR to B 2 - g ~ .Gives the Jordan structure of the infinity eigenvalues h Al-XBi:
[" 0"
*
B3
3-
=
P5*(A2-XB2)Q5
Af-Wf
1
6: Apply the QZalgorithm [27]to As- XB3 and a reordering process to the resulting upper triangular pencil to give the desired pencil Af-XBf: Pg* (A3-AB3)Qa = A / - XBf
*I
Let af and be the diagonal elements of Af and B f , respectively. Then the finite eigenvalues of A-XB are given by a f / p f . Reductions 4 to 6 in summary: P6* 0 Ps* 0 0 I] 0 I]
[
[
[
Q5
P4*(A1-XBl)Q4
0
0 I]
[
Q6
0
0 I] =
[
Af-XBf
0
*
Af-XBf * 0 Ai-XBi
Note that the transformation matrices Pi and Q, in the 6 reductions above are all unitary and the composite transformations P and Q in GUPTRI are given by
30 1
Stably Coinpiring the Kronecker Structure
[A 4[A 13][ [ [ o 0 0 I]
Q4
Q = Q,
0 0 I]
Q5
Q6
(3.15a) 0 I]
(3.15b)
where the identity matrices in (3.15a-b) are of appropriate sizes. 3.4 Some history and other approaches During the last years w e have seen an intensive study of computational aspects of spectral characteristics of general matrix pencils A-XB. Already in 1977 Vera Kublanovskaya presented her first paper (in Russian) on the AB-algorithm for handling spectral problems of linear matrix pencils. For more recent publications see [16, 171. The AB-algorithm computes two sequences of matrices {Ak} and {Bk} satisfying AkBk+I = BkAk+I , k = 0 , 1 , . * * ;Ao=A , Bo=B where At+ and Bk+ are blocks (one of them upper triangular) of the nullspace of the augmented matrix c k = [Ak Bk] in the following way: N(C&)=
rBk+lI *
By applying the AB-algorithm to a singular pencil A-XB we get the right minimal indices and the Jordan structure of the zero eigenvalues. Different ways to compute N ( C k ) give rise to different algorithms. Kublanovskaya presents the AB-algorithm in terms of the QR (or LR) decomposition. In [18] a modified AB-SVD algorithm is suggested that more stably computes the rangelnullspace separations in terms of the singular value decomposition. In 1979 both Van Dooren and Wilkinson presents papers on computing the KCF. The paper [46] has already been mentioned in sections 2.1-2, where in algorithmic terms he derives the KCF starting from a singular system of differential equations. In [39] Van Dooren presents an algorithm for computing the Kronecker structure which is a straight forward generalization of Kublanovskaya's algorithm for determining the Jordan structure of A-XI, as further developed by Golub and Wilkinson [15] and Ki3gstrt)m and Ruhe [21, 221. His reduction is obtained under unitary equivalence transformations and is similar in form to the one obtained from the RGQZD algorithm. The main difference is that we in RZSTR and LISTR compute n, and nl-rf (di~n(N(A(~-~))) and dirn(N(A('-l)) nN(B('-I))), respectively) from two column compressions, while Van Dooren computes n, and r, from one column compression and one row com ression. Thus Van Dooren never computes a basis for tLe common nullspace of A(f- and B(I-I). The operation counts for one deflation step in Van Dooren's algorithm and in RZSTR are of the same order. For exam le if m = n the first deflation step counts 2n3+O(n2) for Van Dooren and n3+O(n ) for RZSTR. Note that both algorithms use singular value decompositions for the critical rank/nullity decisions. If one consider less stable and robust procedures for determining a rangelnullspace separation (like the QR or LR decompositions) it is possible to formulate faster variants of our algorithms. However this is not recommended except possibly in cases where we for physical or modeling reasons know that the underlying pencil must have a certain Kronecker structure.
!?
f
J. Demmel and B. Kdgstrom
302
4. Perturbation Theory for Pencils 4.1 Review of the Regular Case The perturbation theory of regular pencils shares many of the features of the theory for the standard eigenproblem A-XI. In particular, for sufficiently small smooth perturbations, the eigenvalues and eigenvectors (or deflating subspaces) of a regular pencil also perturb smoothly. One can prove generalizations of both the Bauer-Fike ([6, 10, 11, 361) and Gerschgorin theorems [33] to regular pencils. In order to deal with infinite eigenvalues, one generally uses the chordal metric x(X,X’) =
IX
- X‘I
(1 + X2)” * (1 + X’2)’/2 to measure the distance between eigenvalues, since this extends smoothly to the case A =
00.
In our context we are interested in bounds on the perturbations of eigenvalues and deflating subspaces when the size of the perturbation E is a (not necessarily small) parameter, possible supplied by the user as an estimate of measurement or other error. Thus, we would like to compute a decomposition of the form A-XB = P(S-XT)Q-’ , S- AT block diagonal as in (1.1),which supplies as much information as possible about all matrices in a ball P(E) { A S E - X(B+F) : II(E,F)II.<E} .
To illustrate and motivate our approach to regular pencils, we indicate how we would decompose P(E) for various values of E, where A-XB is given by
[:1:]
1100 0 1 0 [o 0 ll where -q is a small number. It is easy to see that IT, the spectrum of A - h B , is a={l/q,O,q}. The three deflating subspaces corresponding to these eigenvalues are spanned by the three columns of the 3 by 3 identity matrix. For E sufficiently small, the spectrum of any pencil in P(E) will contain 3 points, one each inside disjoint sets centered at Vq, 0 and q. In fact, we can draw 3 disjoint closed curves surrounding 3 disjoint regions, one around each X € u such that each pencil in P(E) has exactly one eigenvalue in the region surrounded by each closed m e . Similarly, the three deflating subspaces corresponding to each eigenvalue remain close to orthogonal. Thus, for E sufficiently small, we partition u into three sets, ul={l/q}, u2={0} and A-XB=
u3={d*
0 0 0 -A
v
As E increases to q / f i , it becomes im ible to draw three such curves because there is a pencil almost within distance q/ 2 of A-XB with a double eigenvalue at q/2:
E p 1;H] q;J-A
7
where t; is an arbitrarily small nonzero quantity. Furthermore, there are no longer three independent deflating subspaces, because the 5 causes the two deflating subspaces originally belonging to 0 and q to merge into a single two-dimensional deflating subspace. We can, however, draw two disjoint closed curves, one around
Stably Computing the Kronecker Structure
303
Uq and the other around 0 and q, such that every pencil in P(E) has one eigenvalue inside the curve around Vq and two eigenvalues inside the other curve. In this case {Vq} = u1u 0 2 . we partition u={O,q} As E increases to 1, it no longer becomes possible to draw two disjoint closed curves any more, but merely one around all three eigenvalues, since it is possible to find a pencil inside P(r) with q and Uq having merged into a single eigenvalue near 1, as well as another pencil inside P(E) where 0 and q have merged into a single eigenvalue near q/2, In this case we cannot partition u into any smaller sets. This example motivates the definition of a stable decomposition o f a regular pencil: the decomposition in (1.1) is stable if the entries of P , Q , S and T are continuous and bounded functions of the entries of A and B as A-XB varies inside P(E). In particular, we insist the dimensions nI of the SS-XTil remain constant for A-XB in P(E). This b corresponds to partitioning u=U u, into disjoint pieces which remain disjoint for
u
i=1
A - X B in P(E). We illustrated this disjointness in the example by surround each ul by its own disjoint closed curve. For numerical reasons we will also insist that the matrices P and Q in (1.1) have their condition numbers bounded by some (user specified) threshold TOL for all pencils in P(E). This is equivalent to insisting that the
deflating subspaces belonging to different ui not contain vectors pointing in nearly parallel directions. In the above example, as E grows to q / f i , there are pencils in P(E) where the deflating subspaces belonging to q and 0 become nearly parallel:
E
0 11/2.P q;J - A
[;;;]. T O O
The two right deflating subspaces in question are spanned by [0,1,OIT and [0,-V5,llT, respectively, which become nearly parallel as 6 approaches 0. The numerical reason for constraining the condition numbers of P and Q is that they indicate approximately how much accuracy we expect to lose in computing the decomposition (1.1) [8]. Therefore the user might wish to specify a maximum condition number TOL he is willing to tolerate in a stable decomposition as well as specifying the uncertainty E in his data. With this introduction, we can state our perturbation theorem for regular pencils: it is a criterion for deciding whether a decomposition (1.1) is stable or not. Theorem 4.1: Let A - X B ,
E
and TOL be given. Let u
u into disjoint sets. Define xI for l S l b as
b
=
U uI be some partitioning of
i= 1
wherep,, q,, Dif. and Difr will be explained below. The corresponding decomposition (1.1) is stable if the following two criteria are satisfied: max xl < 1
lslsb
and
304
J. Demmel uml B. Kugstrom
For a proof and further discussion see [ll]. The criterion (4.1) is due essentially to Stewart [32]. If we have no constraint on the condition numbers (i.e. TOL=m), then there is a stronger test for stability (see [ll] for details). The quantities pi,q l , Dif,(u,,u-cr,), and Difl(ul,u-u,) play a central role in the analysis of singular pencils, and will be discussed further in the next section. For now let us say Dif,(u,,u-ui) and Difl(ui,u-mi) measure the "distance" between the eigenvalues in u, and the remainder in u- u,,and that p i and q, measure the "velocity" with which the eigenvalues move under perturbations. Thus the factor multiplying E in (4.1) is velocity over distance, or the reciprocal of how large a perturbation is needed to make an eigenvalue of ui coalesce with one from u--0,. This bound is fairly tight, and in fact the factor b.max (Pi,q,) in (4.3) is essentially the least possible value for ,I the maximum of K ( P ) and K(Q) where P and Q block diagonalize A-AB, the center of P( ). The quantities Dif,, Difl, p , , and q1 may be straightforwardly computed using standard software packages. Also, nearly best conditioned block diagonalizing P and Q in (1.1) can also be computed. Therefore, it is possible to computationally verify conditions (4.1)to (4.3)and so to determine whether or not a decomposition is stable as defined above, as well as to compute the decomposition. 4.2 Why is the Singular Case Harder than the Regular Case? Our analysis of the regular case in the last section depended on the fact that eigenvalues and deflating subspaces of regular pencils generally change continuously under continuous changes of the pencil. This situation is completely different for singular pencils. As mentioned in section 1.2, the features of a nongeneric singular pencil may change abruptly under arbitrarily small changes in the pencil entries. In this section we illustrate this with some examples: As stated in section 1.2, almost any arbitrarily small perturbation of a square singular pencil will make it regular. The singular pencils form a lower dimensional surface called a proper variety in the space of all pencils. A variety is the solution set of a set of polynomial equations. In this case the polynomial equations are obtained by equating the coefficients of all the different powers of A in det(A - AB) to zero [44]. In fact, given a square singular pencil one can find arbitrariIy small perturbations such that the resulting regular pencil has its eigenvalues at arbitrary preassigned points in the extended complex plane [46]. For example, the pencil 0 1 0 -A 1 0 0 0 0 - A O O l = 0 0 - A I 0 [ o 0 11 is singular, and it is easy to see that the perturbed pencil
J [:I A]:-
+
[:a
:]
-"b -c-A 0 has determinant dA3+cA2+bA+a. Clearly we can choose a , b , c and d arbitrarily
0
small so that this polynomial has any three roots desired.
Stably Computiiig the Kroriecker Structure
305
Similarly, almost all nonsquare pencils have the same KCF, which will consists entirely of Lk blocks if there are more columns than rows and LT blocks if there are more row than columns. For example, the following pencil is nongeneric because it has a regular part with an eigenvalue at 1:
0 1 0 [0 0 11 -
[i
=
-[: i
but the following perturbation
-[: i
hO1]
+
a\
I!X]
01
has a generic KC.F consisting of a single L2 block for any nonzero a . In control theory the fact that a generic nonsquare pencil has no regular part implies that a generic control system is completely controllable and observable (SW section 2.3). 4.3 Perturbation Results for Pairs of Reducing Subspaces How do we do perturbation theory for nongeneric pencils in the face of the results of the last section? Clearly, we need to make some restrictions on the nature of the perturbations we &ow, so that the structure we wish to preserve changes smoothly. In particular, we will assume that our unperturbed pencil A-XB has a pair of left and right reducing subspaces P and Q, and that the perturbed pencil ( A + E ) - X ( B + F ) has reducing subspaces PEF and QEFof the same dimensions as P and Q, respectively. Under these assumptions we will ask how much PEF and QEF can differ from the unperturbed P and Q as a function of 11 (E,F)(( E: Theorem 4.2: Let P and Q be the left and right reducing subspaces of A-XB corresponding to the subset ul of the set of eigenvalues u of A-XB. Let E = 11 (E,F)II E . Then if ( A + E ) - X ( B + F ) has reducing subspaces PEFand QEF of the same dimensions as P and Q, respectively, where
(where Difu(ul,u-ul), Difl(ul,u-ul), p and q will be defined below) then one of the following two cases must hold: Case 1: emax(P,PEF)Iarctan(x.(p + (p2- I)*~)) and 5
a c t d x * ( q +(q2- 1IU2>). If x
ardan(2.x 1 4 ) . In other words, both angles are small, bounded above by a multiple of the norm of the perturbation E . Case 2: Either ~~JQ,QEF) 5
306
J. Demmel and B. K6gstrom
or
In other words, at least one of the angles between perturbed and unperturbed reducing subspaces is bounded away from 0. Note that the definition (4.4) of x is identical to the definition (4.1) for regular pencils. We will see when we sketch the proof of this theorem below how this result for singular pencils is a natural generalization of the result for regular pencils: Without loss of generality we assume A-AB is in the GUPTRI form defined in section 3:
: :I
A,- XB, A-XB = 0 Arcx-XBrcx 1
0
where A,-XB, has only L, blocks in its KCF Al-XB, has only L,' blocks in subscript I-), A,,,- AB,, is upper triangular and regular, pencil. Now repartition (4.5) as follows: All Al2 A - A B = 0 A,,]
[
(4.5)
AI-XBI
(i.e. all right minimal indices, hence the its KCF (i.e. all left minimal indices), and each * is an arbitrary conforming
-
'[
B11 Bl2 0 B22]
where All-ABll includes all of A,-XB,-and possibly part of Areg-XBrc , and Az2-ABzz contains all of Al-XBl and the remainder of A,,-XB,,,. In partidar, we assume All-XBll and Az2-XBzz contain disjoint parts (al and u 2 = u-ul respectively) of the spectrum u of A-XB. We denote the numbers of rows and columns of All-XBl, by m1 and n,;it is easy to see m+nl (with equality if and only if A,-XB, is null) and rn22n2 (with equality if and only if AI-XB, is null). In this coordinate system our unperturbed reducing subspaces P and Q are spanned by the column of [Zm1IOIT and [Zn1IOIT, respectively. Our first step in computing PEF and QEFis to blockdiagonalize the pencil in (4.6), the upper left block containing crl and the lower right block u2. Thus, we seek P and Q of the following special forms such that P-'(A-XB)Q =
(4.7a) (4.7b)
301
Stably Computirrg the Krorzecker Structure
This is a set of 2rn1n2 linear equations in nln2+mlmz unknowns, the entries of L and R . Since rnl
(A11A22;B11J 2 2 )
umin(Zu)
with the trivial consequence:
11 (LsR)llE
11 (A1Z&12)11
E
Dfa(A11A22$11p22)
*
Dif, is specified merely by choosing u1 and uz=cr-ul, permitting us to write Difu(al,a2) if A-XB is known from context or even just Dif, if u1 is known as well. If A-XB is singular and (4.8) has nonunique solutions, we will choose the (L,R) of least norm since it leads to P and Q as well conditioned as possible. Call this minimum norm solution (Lo,Ro) and denote (1+11 Loll 2)m by p and (I+ 11 Roll 2)uz by q . Just as Dif, is specified only by ul,so are 11 Loll , 11 Roll ,p and q . Now consider the perturbed pencil (A+E)-X(B+F). Premultiplying by P - I and postmultiplying by Q yields the pencil p-l
. ((A+E)-h(B+F))
'
Q
Eal
=
]
A 2El2 2 + ~ 2]2- A ~ l lF2l + F 1 lBz2+F22 Fl2 (4.9)
[A1lCEI1
We now seek PFk and Q E F of the forms
and QEF
=
[
- R ~
.
In,
such that premultiplying (4.9) by Pi; and postmultiplying it by Q E F blockdiagonalizes it. It is easy to see that if we can find such PEF and Q E F the perturbed pair of reducing subspaces P E F and QEFwill be spanned by the columns of
so that if Lz and R2 are small, the-angles between the unperturbed and perturbed subspaces will be small. Performing these multiplications and rearranging the result yields
[
(All+~ll-L1EZl) (I+RlR2)
~ 2 ~ ~ 1 1 + ~ l l ~ - ~ ~+ 2 E21 2 +~ L2El2R2 2 2 ~ ~ 2
(All+Ell)Rl-Ll(A22+E22) + El2 - LlEZlRl (A,,+ E22+LZE12) (I+R2Rl)
I
(4.10)
308
J. Demmel and B. Kbgstrom @11+
F11- W 2 1 )
(I+ R P 2 )
( ~ 1 1 + ~ 1 1 ) ~ 1 - ~ 1 ( ~ 2 2 ++ ~ 2F12 2)
- LlF2lRl
@22+ F22+ J52F12) (Z+R2R1)
1
Setting the upper right and lower left corners of the pencil two zeroes yields two sets of equations: (All+Ell)R1-Ll(A22+E22) = -El2+L1E21R1 (4.11a) (4.llb) (4.12a) J52@11+F11)-
@22+F22)R2
= -F21+L2F12R2
.
(4.12b)
(4.11ab) is a set of 2m1n2 nonlinear equations in n l n 2 + m l r n 2 unknowns, and (4.12ab) is a set of 2m2nl nonlinear equations in the same number of unknowns. If A-XB is regular ( m f = n f ) ,both these systems have the same number of equations as unknowns; this was the case considered by Stewart [32]. He shows that as long as Ef, and F!,are small enough, both (4.11ab) and (4.12ab) have unique solutions in a neighborhood of the origin. If A-XB is singular, (4.1lab) will be underdetermined and (4.12ab) will be overdetermined. Overdetermined systems have no solution in general; that this one does is guaranteed by our assumption that the perturbed pencil (A+E)-X(B+F) have reducing subspaces PEF and QEFof the same dimensions as P and Q. To solve (4.11ab) we make the identifications x = [L1hl], Tx =
(4.13a) (4.13b)
and (4.13~)
so that (4.11ab) becomes Tx = g + $(x) . Expressing T in t e r m of JSronecker products yields
so that [32]
(4.13d)
309
Stably Computing the Kronecker Structure
(4.14a) (4.14b) and (4.14~)
so that (4.12ab) becomes Tx = g
+ +(x)
(4.14d)
Expressing T in terms of Kronecker products yields
I
(All+
EdT@Zrnz
-zn1@
(B 11+ F1d*@zrn2 -1.
I@
( A 2 2 +E22)
(B22 +F Z Z )
Swapping the first mlm2 columns with the last nlnz columns and negating the whole matrix, none of which changes its singular values, yields Znl@(A22
[
+E22) - (All+ E I Y @ Z r n z
Zn,@P22+F22)
J
-( ~ 1 1 + ~ 1 d * @ Z q
which we recognize as the matrix whose smallest singular value is defined as ~22~11+~11;B22+~22,~11+~11)
2 ~f,(A22,A11~22,B,l) -
v5 II ( ~ l l , E 2 2 , F l l 9 F 2 2 ) 1 1 E *
The quantity Difu(A22,A11;82281J does not generally equal DifU(A11P~2$11,B2~) (unless the A,, and B,, are symmetric). In the interests of retaining our coordinate free formulation of our bounds, we therefore define D i f d U l P Z ) = D i f u ('422 sill $22 9Bld where the fact that Dif, depends only on u1 and u2 follows just as for Dif,. Also, one may prove that Dif,(u1,a2)>0 if and only if u1 and w2 are disjoint, as we have assumed. Thus UrninV) 2
~f,(ul,uz)
* II *
.
( ~ 1 1 , ~ 2 2 ~ 1 1 , ~ 2 E2 ) 1 1
Using these bounds on urn&), we can find bounds on the solution L, and R, of (4.11ab) and (4.12ab): Lemma 4.3: [ll] Consider the equations
J. Demmel and B. Khgstrom
3 10
Tx = g
+ +(x)
(4.15a)
and x = T+(g
+
+(.I>
(4.15b)
where T + is the pseudoinverse of T. Assume that amin(T),11 $11 , and K
IIgll 11 $11 *
u&(T) <
1
11 gll
satisfy
4 .
Then equation (4.15b) has a unique solution x inside the ball (4.1%)
This solution x of (4.15b) has the following relationship to the solution 2 of (4.15a):
Case 1: If m=n, 2=x‘ is the unique solution of (4.15a) (in the ball). Case 2: If mn, and if (4.15a) has a solution 2 in the ball, then 2=f. Thus, (4.15b) may have a solution whereas (4.15a) may not. Furthermore, in cases 1 and 3, if (4.15a) has a solution which does not lie inside the ball in (4.15c), it must lie outside the ball: (4.15d) Substituting in values for umin(T), 11 gll , and 11 +I[ from equations (4.13abc) and (4.14abc) yields Theorem 4.2. 4.4 Perturbation Results for the Spectrum of the Regular Part If mZ=n2, then Azz-ABz2 is square and hence regular (otherwise it would have both Lk and LJ blocks in its KCF). Then after blockdiagonalizing the perturbed pencil (A+E)-A(B+F) as in (4.10) we see the perturbed pencil also has a square part [(A22+Ezz+LzE12) - w322+F22+L2F12)1 * (Z+RZRl) * If this square part is regular, we can use our bounds on IIL,II and IIR,II from the last section in conjunction with some of the perturbation theory for regular pencils in section 4.1 to derive bounds on the difference between its eigenvalues and the eigenvalues of the regular part of the unperturbed pencil A22-AB22: Theorem 4.4: Suppose that Case 1 of Theorem 4.2 holds. Suppose further that the block A22-AB22 is regular (i.e. rn2=n2>0). Then the spectrum of the perturbed pencil (A +E) -A (B +F) includes the spectrum of (A22+E’22)-Wzz+F’22) where
v5
II (E‘22J7’22)IlE 5 * 4 * II (E,F)llE . Similarly, if we instead assumeA1l-AB1l is regular (ml=nl>O), the perturbed pencil (A +E) -A (B +F) includes the spectrum of (A11+~’11)-~(~11+~’11) where l l ~ ~ ’ l l J ’ l l ~5 llv E5 . P
- II (EJ)II.
*
then the spectrum of
Stablv Computing the Kronecker Structure
311
See [ll] for a detailed proof. 4.5 Some Applications to Linear Systems Theory
Now let us interpret the results of the last two sections in terms of linear system theory: perturbation bounds for controllable subspaces and uncontrollable modes. Let C ( A , C ) denote the controllable subspace of the pair of matrices ( A , C ) as in section 2.3. As discussed in that section, this sFbsp$ce is simply the left reducing subspace corresponding to crl = 0 of the pencil A-XB = [ C k - X I ] . It is easy to s~ that this pencil can have neither infinite eigenvalues nor L l blocks in its KCF since B = [OF] has full rank; hence it can only have finite eigenvalues and L, blocks in its KCF. Also, one can see that the number of L, blocks is a constant equal to the number of columns of C. Thus, the assumption in Theorem 4.2 about the perturbed pencil having reducing subspaces of the same size is implied by the assumption that the perturbed system (A+EA,C+Ec) has a controllable subspace of the same dimension as C ( A , C ) . Also, the algorithms one uses
Then either Case 1: @,,(C(A,C),C(A+ EA,C+Ec))
5
~ctan(~*(p+(p~-l)~~))
with @,,(c(A,c),c(A+EA,c+Ec)) 5 Xctarl(2.X / p )
if x < l / 2 , or Case 2:
For a detailed proof, see [ll]. It is also easy to see that this result applies immediately to observable subspaces as well by duality [48]. Another feature of a control system ( A , C ) for which we can derive perturbation bounds using this approach is the spectrum of the regular part, also called the input decoupling zeroes or uncontrollable modes of the control system. Theorem 4.4 of the last section yields: Corollary 4.6 Assume we are in Case 1 of Corollary 4.5. Then the uncontrollable modes of the perturbed control system (A+EA,C+EC) are the eigenvalues of the pencil (Azz+E122)-~B22
where
II E’zzll E
5
fi
*
4 . II (EA,&)I1
E
*
312
J. Demmel and B. Ktigshom
In [40],P and Q are chosen in (4.1) so that P-IBQ = [rlO], implying Bz2=Z. Thus the problem of finding the eigenvalues of the perturbed pencil (A22+E122)-AB22 above reduces to perturbation theory for the standard eigenproblem. To illustrate these points consider the pencil
A - A i = [Cb-XI]
=
E5
A 2:
8].
0 0 3-A This pencil is in the canonical form (4.2) with rnl=l, rnz=2, n l = 2 and 4 = 2 . In other words All-ABll = 1-A] has KCF L1 and A22- ABzz = diag(2-A,3-A) is
a regular diagonal pencil with eigenvalues 2 and 3. The left reducing subspace P which is spanned by [1,O,OIT is the controllable subspace C(A,C) of the system (AS). The quantities of interest in Corollary 4.5 are Dif,(0,{2,3}) = Difi(0,{2,3}) .I12 , and p =q = 1. Thus Corollary 4.5 implies that if E A and Ec are such that 1 = dim(C(A,C)) = dim(C(A+EA,C+EC)) and 11 (EA,Ec)ll = .028.x where x < l then either Case 1: B,,(C(A,C) , C(A+EA,C+Ec)) 5 ~ c t a n ( x ) or Case 2: B,,(C(A,C) , C(A+EA,C+Ec)) 2 arctan(l/ = arctan(S77) = 5 2 Thus, for example, if we have II(EA,Ec)llE < .001, then any one dimensional controllable subspace of (A+EA,C+Ec) will either be within .036 radians of C ( A , C ) or at least .52 radians away from C ( A , C ) . A simple example of the latter situation is
e)
,o,
[ ,"
0
[c-kE,-w+E~-Az] = lo-'
2-A
0
3J: in which case the new controllable subspace is spanned by [0,1,OITand so is orthogonal to C(A,C). In case the perturbed controllable subspace falls into Case 1, then we can use Corollary 4.6 to bound the uncontrollable modes of the perturbed system: the perturbed modes are eigenvalues of the matrix
[;: :] +
E'
where 11 E ' l f ~ 5 ~ * I (EA,Ec)ll I E . Gershgorin's theorem supplies the simple bound th the perturbed uncontrollable modes lie in disks centered at 2 and 3 with radii 4 l J ( E , , E c ) l J s 5 xm.04. 4.6 Other Approaches
Stably
C'rirnpii turg
313
the Kroriecker Stntcture
Sun uses a somewhat different approach for deriving perturbation theorems for the eigenvalues of singular pencils [38]. He generalizes his results for regular pencils [37] by relating the perturbation of the eigenvalues of a diagonalizable A - XB and the variation of the orthogonal projection onto the column space R([A,B]*) to each other. His assumption that A-XB is diagonalizable means that A-XB may have neither Jordan blocks of dimension greater than 1 nor singular blocks other than Lo and LOT. Sun must also make restrictions on the perturbations in A and B . His assumptions are that the space W=R(Ap) contains no vectors orthogonal to Z=R(A+Ep+F) and that W'=R(A*(B*)* contains no vectors orthogonal to Z'=R[(A+E)*I(B+F)*]*. By using the chordal metric (see section 4.1) to measure the perturbations of eigenvalues he obtains a Wielandt-Hoffman and a Bauer-Fike type of theorem for singular diagonalizable pencils, showing that the eigenvalues of A-XB are insensitive to small perturbations in A and B. 5. Analyzing the error of standard algorithms for computing the Kronecker structure Using the perturbation analysis from section 4 this section analyzes the error of standard algorithms for computing the Kronecker structure from section 3. Here we focus on the GUPTRI form presented in section 3.3, but the analysis hold for any backward stable algorithm. 5.1 Distance from A-XB to the nearest pencil with the computed Kronecker structure Let (Y be a description of the KCF of A-XB (a list of left minimal indices, right minimal indices, and eigenvalues along with the number and sizes of Jordan blocks for each one). Then we define Sing,(A,B) = min{ll (E,F)IIE : (A+E)-X(B+F) has KCF a} (5.1) i.e. Sing,(A,B) is the smallest perturbation of A-XB (measured as II(E,F)IIE) that makes the perturbed pencil (A+E)-X(B+F) have the Kronecker structure a. One estimate of Sing,(A,B) comes from running one of the algorithms in section 3 for computing the KCF. We focus on the GUPTRI form and its RZSTR kernel discussed in section 3.3. The main contribution to errors caused by the finite arithmetic in RZSTR comes from the computation of nl and r,. For example the computation n,:=column-nullity(A(') , epsu) means that all singular values (Tk of A(') less than epsu are interpreted as zeros. In practice we make column-nullity(.;) more robust as follows: if ak<epsu but a k + l r e p s uwe insist on a gap between these two singular values (e.g. ak+l/ukr100). If this gap is not big enough we also treat ( T k + l as zero. We repeat this until have an appreciable gap between the zero and nonzero singular values. It is straightforward to show (details are omitted here) that the i-th reduction step of RZSTR in finite precision arithmetic can be expressed as
i.e the right hand side of (5.2) is an exact reduction of a perturbed pencil A(,)+,!?(')- X(B(')+F(')). Here IIE(')IIi is the sum of the squares of the n, singular values interpreted as zeros in the column compression of A(') (see step 1.1of RZSTR L
J
314
J. Dernniel and B. Kagstrom
in section 3.3). Similarly 11 F(’)IIi is the sum of the squares of the n i - q singular values of B1 interpreted as zeros in the column compression of the n, columns of B(l)Vf) (see step 2.1 of RZSTR). We get similar contributions from all reductions. Since we only make use of unitary equivalence transformations when computing the GUPTRI form the resulting decomposition can be expressed in finite arithmetic as
P* ((A +E ) -A (B + F ) ) Q
=
(5.3)
where S2 is the sum of the squares of all singular values interpreted as zeros during the six reductions (see section 3.3). The equations (5.3-4) express that the GUPTRI form is backward stable and it computes a Kronecker structure (Y for a pencil C-AD within distance 6 from A-AB. This computed 6 is an upper bound on Sing,(A,B) for the value of the Kronecker structure (Y the algorithm reports. How good is this upper bound? We cannot expect 6 to be smaller than O(epsu max(llAIIE , llBllE)), since epsu is used as the tolerance for deleting small singular values. However if A-AB has a well-defined Kronecker structure (all a,,!are O(rnacheps max((lAIIE, 11 BIIE)) where macheps is the machine are O(l), where machepssepsu), then 6 will be of the roundoff error, and all size O(macheps max(l1 All , 11 Bll E ) ) too. The described situation shows a case where we have overestimated the uncertainties in A and B. On the other hand examples (see [39]) show that delta may be as bad as a root of the true distance (e.g. cU2 versus E), but some function of it may still provide a crude lower bound on &&(A ,B), much as the output of K%gstrt)m’s and Ruhe’s algorithm for the Jordan canonical form [21] provides a lower bound on the distance from the input matrix to one with reported Jordan form. This problem is under investigation. From the backward stability of the GUPTRI form we see that the computed bases for a pair of reducing subspaces (P,Q) correspond exactly to a nearby pencil C-AD within distance 6 . How to compute error bounds for these bases is the topic of the coming section. 5.2 Perturbation bounds for computed pairs of reducing subspaces In this section we apply the perturbation theory of chapter 4 to analyze the results of the GUPTRI algorithm (or any similar backwards stable algorithm) of chapter 3. Let A-AB be a pencil supplied by the user; it may or may not be singular. The algorithm of chapter 3 had the property that it found an exactly singular pencil A’-AB’ within a small distance 6 of A-XB. Theorem 4.2 showed that if we perturb A’-XB’ such that the perturbed pencil (A’+E)--X(B’+F) has reducing subspaces of the same dimension as A’-XB’, and if the norm of the perturbation 11 (E,F)II is small enough, then we can compute bounds on the maximum angle between the unperturbed and perturbed reducing subspaces. Let A denote the upper bound on 11 (E,F)II of Theorem 4.2:
Then clearly if any singular pencil A”-XB” with reducing subspaces of the same dimensions as A’-XB’ is sufficiently close to A-XB so that it is also within A of
315
Stably Cotnputirig the Kronecker Structure
A‘-XB’, the perturbation bounds of Theorem 4.2 will apply to it. For A”-XB” to be sufficiently close to A’-XB’ it clearly suffices that 11 (A“-A,B”-B)IIE < A-6, if A-6>0. This leads to the following algorithm: Algorithm 5.1: Perturbation Bounds for Computed Reducing Subspaces 1) Reduce A-XB to A‘-XB’ of the form GUPTRI using an algorithm such as the one in section 3.3 based on RGQZD. Let 6 bound the perturbation l l ( A f - A , B ’ - B ) \ l E . Let P’ and Q’ be reducing subspaces of A’-XB’ corresponding to the subset u1of the spectrum of A ’ - X B ’ . 2) Compute the bound A given above in equation (5.5). Let q = A-6. 3) I q>O, then if A”-”” is any singular pencil within distance d = 11 ( A ” - A 7 Z 3 ” - B ) ~ ~ E< q of A-XB with reducing subspaces Q” and P” of the same dimensions as Q’ and P‘, one of the two cases of Theorem 4.2 must
hold: Case 1:
and Omax(Q’,Q”)
5
arctan(-
6+d
A
(q+(q2-1)”))
,
with better bounds if A’ ’ -XB’ ‘ is within q/2 of A - XB Case 2: Either
or
4) If 1110,no perturbation bound can be made because the perturbation 6 made by the algorithm is outside the range of our estimate. One other interpretation of chapters 3 and 4 can be made. If the rank decisions made by the algorithm of chapter 3 are sufficiently well defined, i.e. at each reduction step there is a large gap between the largest singular value (Jk+ less than the threshold epsu and the smallest singular value uk greater than epsu: Uk
>>epsu >>uk + 1
then the algorithm will make the same rank decisions for all pencils in a disk around A - X B , and hence will compute the same Kronecker structure for the left and right indices and zero and infinite eigenvalues. In particular, the computed minimal and maximal reducing subspaces will have the same dimensions for all inputs in the disk around A - X B . Combined with Theorem 4.2, this means that if A-XB is intended to be singular, but fails to be because of roundoff or measurement errors, then the minimal and maximal reducing subspaces will be computed accurately anyway. Applied to linear systems theory (see sections 2.3 and 4 . 3 , this means that the algorithms of chapter 3 compute accurate controllable subspaces and uncontrollable modes of uncontrollable systems.
3 16
J. Demmel and B. Khgstrom
6. Numerical examples A Fortran 77 program implementing the GUPTRI form with accuracy-estimates of computed results (see sections 3.3 and 5 ) is under development. The inputs consist of a matrix pair (A,B), a bound epsu on the size of the uncertainty in ( A $ ) , and flags indicating whether a complete or just partial information (e.g. the minimal indices or the regular part of A - X B ) is desired. The output is a GUPTRI form (3.14) which is true for almost all pencils within distance delta of A-XB (the generic case), or else a nongeneric GUPTRI form lying within distance 6 from the input pencil A - X B . The bound 6 is expressed in terms of small singular values that are deleted ( C e p s u ) during the deflation process to compute the Kronecker structure cx (see section 5.1). Here we report results from a Matlab [26] implementation in double precision arithmetic (the relative machine accuracy macheps = 1.4E-17) of the basic reduction RZSTR used to transform A-XB to a GUPTRI form. The inputs are as for the Fortran program while the output of RZSTR is a block upper triangular form
*I
Aor - XBor 0 A1-XB1
[
where Aor-XBo, contains the Jordan structure of the zero eigenvalue and the right minimal indices. The pencil A1-XB1 contains the possible left minimal structure and the remaining regular part of A - X B . This decomposition is the first of six reductions used for computing the GUPTRI form and corresponds to what we get when replacing GSVD by GQZD in the reduction theorem 3.3. 6.1 A generic pencil Let A and B be 7 = m by 8 = n random matrices (generated in Matlab by the RAND-function) and ~ P S U= m a c h e ~ sm a ( ]I A I] E [I B ]I E ) RZSTR computed the following decomposition corresponding to one L7 block (see section 1.2): 9
Since we mainly are interested of the computed block structure we only display numbers in 4 decimals accuracy. However the zeros displayed above are all accurate to the rounding level. RZSTR does not know in advance that A - X B is generic, so it
317
Stably Compzttiiig the Kronecker Structure
has to go through eight deflation steps in order to realize that A - XB is equivdent to one L7 block. (The generic structure of a k by k+ 1pencil is one Lk block.) Below we display ni,r, and the smallest singular value of [A(')*,B(')*]*in each of the seven deflation steps. i n, ri umin([A(')*$(')*]*) 1 1 1 0.38 2 1 1 0.34 3 1 1 0.33 4 1 1 0.21 5 1 1 0.41 6 1 1 0.36 7 1 1 0.67 8 1 0 Locally at each step i the smallest singular value displayed is the 2-norm of the smallest perturbation of [A(')*$(')*]* that makes A(') and B(') have a common nullspace (ni-ri>O). So at least one of these singular values must be interpreted as zero (<epsu) in order to produce a nongeneric Kronecker structure. To sum up, in order to perturb A-XB to a proper variety of nongeneric pencils we have to add quite big perturbations to A and B compared to the specified uncertainty in A and B . Any epsu less than the minimum of the displayed singular values specified as input to RZSTR will cause the generic Kronecker structure to be computed (provided we have appreciable gaps between the zero and nonzero singular values in RZSTR as described in section 5.1). Since A-XB is generic no singular values are deleted during the reduction process and Sing,(A,B) is zero to the rounding level. 6.2 A nongeneric pencil unperturbed and perturbed Without loss of generality we consider a nongeneric pencil A - AB in block upper triangular form, where
-
A =
0.2055 0.6329 0.1249 0. 0. 0. 0.
0.3407 0.7626 0 5552 0. 0. 0. 0.
0.3004 0.9134 0.1967 0. 0. 0. 0.
0.2294 0.6493 0.2089 0. 0. 0. 0. B=
0.5271 1.0435 0.6309 0. 0. 0. 0.
0.8712 0.9253 0.9119 0. 0. 0. 0.
0.4016 0.5897 0.4095 0. 0. 0. 0.
["d' iz]
rl
=
0.3013 0.7512 0.3976 0. 0. 0. 0.
1.2644 1.8991 1.3820 0.9856 0.3355 0.7987 0.5069
1S207 2.3651 1.7303 1.3820 0.6163 1.2348 0.8677
0.7374 1.1086 0.8401 0.6800 0.2280 0.5545 0.4382
0.5971 0.8535 0.6428 0.0489 0.1748 0.0001 0.0764
0.9958 1.3437 1.1080 0.1379 0.4933 0.0001 0.2155
0.4886 0.6911 0.5103 0.0454 0.1622 0 .oooo 0.0709
=
0.5034 0.7699 0.5951 0. 0.
0.7325 0.9922 0.7249 0. 0.
0. 0.
0. 0.
J. Demmel and B. Khgstrom
3 18
are displayed only to four decimals accuracy. The pencils A1l-XB1l and A22-XB22 have the KCF's diag(Lo,L1,J2(0))and diag(LT,2.J1(w)), respectively. Since A - X B is known to full machine precision RZSTR is called with an epsu similar to the one in the preceding example. In order to obtain the complete Kronecker structure RZSTR is called twice, first with A - XB as input and then with the deflated pencil from the first reduction as input (see the discussion in section 3.3 for more details). The following indices determining a nongeneric Kronecker structure a1are computed:
number of L I - blocks
2 1
1 1
number of Lf-l blocks
3 0
0 1
Table 6.1 During the reduction process all singular values that should be zeros are zeros to the rounding level. What happens if we perturb the nongeneric pencil A-XB? Consider where E and F are full matrices and IIEllE is O(machepsV2 IIA116), IIBllE). In section 4.2 we pointed out that almost all (arbitrarily small) perturbations of a nongeneric pencil will make it generic, so ( A + E ) - X(B +F ) ought to be generic, and it is! When we apply RZSTR with the same epsu as for the unperturbed pencil it computes the generic structure as in section 6.1 (which we denote by a2). However if we consider (A+E)--X(B+F) as a pencil with uncertainties in the data of size 11 Ell and 11 FII , respectively, RZSTR computes the nongeneric Kronecker structure a1 corresponding to the unperturbed pencil A-XB. Below we display the computed reduction P*(A+E)Q = 8.4580e-10 1.5835e-9 3.3293e-9 .1.5798e-9 9.4764e-10 4.0688e-9 6.2048e-9
0.0 2.0000e-15 2.4458e-11 5.4961e-9 -6.8016e-10 2.3264e-10 3.0473~-10 3.9444~-9 -2.9634e-10 2.6389e-9 5.7650e-10 -2.3186e-9 0.0 0.0
["" 0
- P*E’Q
=
(6.la)
-7.0593e-1 1.7643 -4.4274 -4.4983e-1 1.4142e-1 3.7705e-1 -7.0718e-2 -5.2545e-1 -4.9481e-2 -6.7643e-3 -6.4429e-8 2.6144e-8 -6.1760e-1 -9.4730e-2 1.5776e-2 -2.5683e-8 1.9761e-8 1.7517 0.0 0.0 -2.2751e-8 1.0899e-8 1.4153 9.8271e-2 0.0 -4.1396e-8 2.0990e-8 3.6086e-1 1.5134e-2 -6.0993e-2 2.8080e-8 -2.7099e-8 1.5303 -4.0147e-2 -3.2022~-2
(6.lb) 0.0 0.0 -2.6917e-9 -1.0018e-9 -4.6944e-9 7.1359e-10 6.7613e-9
6.6146e-1 0.0 0.0 0.0 0.0 0.0 0.0
-1.4960e-2 8.5456e-2 0.0 0.0 0.0 0.0 0.0
8.3731e-1 4.5289e-2 8.0000e-15 -8.7672e-9 3.7934e-9 -8.5628e-9 1.5586e-8
2.5573 4.2848e-1 5.1278e-1 0.0 0.0 0.0 0.0
-2.4824 3.9301e-2 -3.8112e-1 4.8408e-2 -4.3052e-1 1.7979e-2 9.422e-11 6.8512e-10 -5.4277e-11 -3.6715e-10 2.3142e-9 8.3491e-9 6.0093e-1 -1.3563e-1
-4.2721e-2 2.4430e-2 -7.8591e-3 -9.2871e-10 4.3950e-10 6.6088e-9 -3.9087e-2
rows 1 to 4 and columns 1 to 5
of
Stably Compirtiizg t h e Kronecker Structure
319
P*((A+E)-X(B+F))Q and its block upper triangular form exposes the Jordan structure of the zero eigenvalue and the right minimal structure of A+E--X(B+F). The pencil Ail-XBll is identical to rows 5 to 7 and columns 6 to 8 of P*((A+E)-X(B+F))Q and corresponds to the Jordan structure of the infinite eigenvalues and the left minimal indices. Note that since we applied RZSTR to a transposed pencil in the second reduction All-XBll is block lower triangular. The perturbation matrices E’ and F’ in (6.la-b) correspond to the small but nonzero singular values interpreted as zeros during the deflation process to determine the Kronecker structure. However our Matlab implementation of RZSTR does not delete these small singular values in the process of determining the q s in Table (6.1), so the decomposition (6.la-b) is to the rounding level an exact similarity transformation of the pencil A+E--X(B+F) specified as input. How do we interpret the resulting decomposition? One way is to say that (6.la-b) is exactly nongeneric to the accuracy max (IIE‘II. , llF’IIE)-7e-8, with the Kronecker structure of A-XB (the unperturbed pencil). In the context of section 5.1
[
[
Bor BlZ 0 Bill from (6.la-b) is nongeneric to the rounding level and is an exact equivalence transformation of the perturbed pencil (A+E+E’)-X(B+F+F’). 6.3 Assessment of the (nongeneric) computed Kronecker structures In the following figure we try to visualize our nongeneric example: Aor A12 0 Ail] -
63 =
O(E”2)
A’-XB’ 61 = O(E) /”A-XB
-
Figure 6.1 - A surface of equivalent nongeneric pencils (same KCF) Here epsu=O(E) means that w e claim to know the input pencil to full machine accuracy and epsu=O(Ev2) means that we claim to know the input pencil to around half the machine accuracy. Let tii denote the distance between two matrix pencils (See Fig. 6.1) and corresponds to deleted singular values (see equations 5.3 and 5.4). First the unperturbed nongeneric pencil A-XB and epsu=O(E) were specified as inputs to RZSTR, and a nongeneric GUPTRI form with Kronecker structure a1 was computed. This form corresponds exactly to the pencil A‘-XB’ which is very close to A-XB; thus Sing,,(A,B) 5 E1=O(c). Then RZSTR was called with the perturbed (generic) pencil (A+E)-x(B+F) and epsu=O(e) as inputs, and as a result the generic GUPTRI form corresponding to (A+E)’--X(B+F)’ was produced. In this case Sing,,(A+E,B+F) Ia2=O(e)). Finally (A+E)-X(B+F) and epsu=O(Em) were specified as inputs to RZSTR which computed a nongeneric GUPTRI form with Kronecker structure a1 again, and corresponds to (A+E+E’)-X(B+F+F’). Thus Sing,,(A+E,B+F) 5 63=O(em). To summarize, A’-XB’ and
3 20
J. Dernmel and R. Kagstroni
( A+ E + E ’ )- X(B + F + F ’ )
have the same KCF and belong to the proper variety of nongeneric pencils. Let P’ and Q’ denote the computed pair of reducing subspaces corresponding to Aor-XBor in (6.la-b), and P and Q the corresponding pair of A ’ - X B ’ . What can be said about the maximal angle between these two pairs of reducing subspaces associated with the perturbed pencil (A + E) -X(B + F ) and the unperturbed A -XB , respectively? By applying the error bounds as described in section 5.2 the following results were obtained: max(p,q) = 3.2847 Dif, = 0.0315 Difl = 0.0147 A = 0.0013 Case 1: O,,(P,P’) 5 5.3E-5 radians (approximately 0.003 degrees) O,,(Q,Q’) I1.3E-4 radians (approximately 0.007 degrees) or Case 2: Om,(P,P’) 2 0.19 radians (approximately 10.8 degrees) O,,(Q,Q‘) 2 0.08 radians (approximately 4.6 degrees) Since we computed a GUPTRI form of both ( A + E ) - X ( B + F ) and A - X B we can show the actual angles between the computed pairs of reducing subspaces corresponding to the perturbed and unperturbed pencils: O,,(P,P’) = 1E-7 radians O,,(Q,Q’) = 5E-7 radians 7. Conclusions and future work
With applications of matrix pencils to descriptor systems, singular systems of differential equations, and state-space realizations of linear systems in mind (section 2) we have shown how to stably and accurately compute features of the KCF (for example minimal indices, pairs of reducing subspaces, regular part, and Jordan structures) which are mathematically ill-posed and potentially numerically unstable. The stability is guaranteed by a finite number of unitary equivalence transformations of A - X B , giving a generalized upper triangular (GUPTRI)form (sections 3 and 5.1). We make the mathematical problem well-posed by restricting the allowed perturbations E-XF of A - X B such that A - X B and ( A + E ) - X ( B + F ) have minimal and maximal pairs of reducing subspaces of the same dimension (section 4). This regularization leads to perturbation bounds for pairs of reducing subspaces and for the eigenvalues of the regular part (section 4) which can be used for analyzing the error of standard algorithms (section 5). The accuracy of the computed Kronecker features are confirmed via error bounds (sections 5 and 6 ) . Our future work will be concentrated on the completion of the our Fortran 77 program that implements the GUPTRI form and the error bounds. Then we want to experiment with the software on a large and diverse set of applications and encourage other people to make use of it. Hopefully this will lead to feedback permitting us to build in more application-based knowledge and focus the software on different and specialized applications (see sections 2.3-4). The first version of the program will
Stablji Computing the Kronecker Structure
32 1
treat all pencils as a general A - XB problem.
8. References
[l] D.J. Bender, "Descriptor Systems and Geometric Control Theory", PhD Dissertation, Dept of Electrical Engineering and Computer Science, Univ. of California Santa Barbara, September 1985 [2] D.L. Boley, "Computing the Controllability/Observability Decomposition of a Linear Time-Invariant Dynamic System, A Numerical Approach, PhD Dissertation, Dept of Computer Science, Stanford University, June 1981 [3] K.E. Brenan, "Stability and Convergence of Difference Approximations for Higher Index Differential-Algebraic Systems with Applications in Trajectory Control", PhD Dissertation, Dept of Mathematics, Univ. of California Los Angeles, 1983 [4] S.L. Campbell, Singular systems of differential equations, Vol. 1 and 2, Pitman, Marshfield, MA, 1980 and 1982 [5] S.L. Campbell, "The numerical solution of higher index linear time varying singular systems of differential equations", SIAM J. Sci. Stat. Computing, Vol. 6, NO. 2, 1985, pp 334-348 [6] K.E. Chu, "Exclusion Theorems and the Perturbation Analysis of the Generalized Eigenvalue Problem", NA-report 11/85, Dept of Mathematics, Univ of Reading, UK, 1985 [7] D.J. Cobb, "Controllability, Observability and Duality in Singular Systems", IEEE Trans Aut. Contr., Vol. AC-29, No. 12, 1984, pp 1076-1082 [8] J. Demmel, "The Condition Number of Equivalence Transformations that Block Diagonalize Matrix Pencils", SIAM J. Num. Anal., Vol. 20, 1983 pp 599-610 [9] J. Demmel, "Computing Stable Eigendecompositions of Matrices", to appear in Linear Alg. Appl., 1985 [lo] J. Demmel and B. K&strbm, "Stable Eigendecompositions of Matrix Pencils A-XB", Report UMINF-118.84, Inst of Information Processing, Univ of Umea, Sweden, June 1984 [ll]J. Demmel and B. K%gstrt)m,"Computing Stable Eigendecompositions of Matrix Pencils", Technical report #164, Courant Institute of Mathematical Sciences, 251 Mercer Str., New York, NY 10012, May 1985 [12] F.R. Gantmacher, The theory of matrices, Vol. 1 and 2, (Transl.) Chelsea, Ney York, 1959 [13] C.W. Gear and L.R. Petzold, "Differentid Algebraic Systems and Matrix Pencils", in [23] pp 75-89 [14] C.W. Gear and L.R. Petzold, "ODE Methods for the Solution of DifferentiaYNgebraic Systems", SIAM J. Numer. Anal., Vol. 21, 1984, pp 716-728 [15] G.H. Golub and J.H. Wilkinson, "Ill-conditioned Eigensystems and the Computation of the Jordan Canonical Form", SIAM Review, Vol. 18, No. 4, 1976, pp 578-619 [16] V.N. Kublanovskaya, "An Approach to Solving the Spectral Problem of A - X B " , in [23] pp 17-29 [17] V.N. Kublanovskaya, "AB-algorithm and Its Modifications for the Spectral Problem of Linear Pencils of Matrices", Numer. Math., Vol. 43, 1984, pp 329342 [18] B. K%gstrt)m, "On Computing the Kronecker Canonical Form of Regular
322
J. Dernniel and B. Kagstrorn
(A-XB)-pencils", in [23], pp 30-57 [19] B. mgstrdrn, "The Generalized Singular Value Decomposition and the General A - XB Problem", BIT 24, 1984, pp 568-583 [20] B. K%gstrt)rn, "RGSVD - An Algorithm for Computing the Kronecker Structure and Reducing Subspaces of Singular Matrix Pencils A - XB ," (Report UMINF112.83), to appear in SIAM J. Sci. Stat. Comp., Vol. 7, No. 1,1986 [21] B. K%gstrt)mand A. Ruhe, "An Algorithm for Numerical Computation of the Jordan Normal Form of a Complex Matrix", ACM Trans. Math. Software, Vol. 6, NO. 3, 1980, pp 398-419 [22] B. K8gstrt)m and A. Ruhe, "ALGORITHM 560, JNF, An Algorithm for Numerical Computation of the Jordan Normal Form of a Complex Matrix [El" ACM , Trans. Math. Software, Vol. 6, No. 3, 1980, pp 437-443 [23] B. K%gstrBm and A. Ruhe (eds), Matrix Pencils, Proceedings, Pite Havsbad 1982, Lecture Notes in Mathematics Vol. 973, Springer-Verlag, Berlin, 1983 [24] F.L. Lewis, "Fundamental, Reachability, and Observability Matrices for Descriptor Systems", IEEE Trans. Aut. Contr., Vol. AC-30, No. 5 , May 1985, pp 502-505 [25] D.G. Luenberger, "Dynamic Equations in Descriptor Form", IEEE Trans Aut. Contr., Vol. AC-22, No. 3, 1977, pp 312-321 [26] C. Moler, "MATLAE - An Interactive Matrix Laboratory", Dept of Computer Science, Univ of New Mexico, Albuquerque, New Mexico [27] C. Moler and G.W. Stewart, "An Algorithm for the Generalized Eigenvalue Problem", SIAM J. Num. Anal., Vol. 15, 1973, pp 752-764 [28] C.C. Paige, "Properties of Numerical Algorithms Related to Computing Controllability", IEEE Trans. Aut. Contr., Vol. AG26, No. 1, 1981, pp 130138 [29] C.C. Paige, "A Note on a Result of Sun Ji-Guang: Sensitivity of the CS and GSV Decompositions", SIAM J. Num. Anal., Vol. 21, 1984, pp 186-191 [30] C.C. Paige and M.A. Saunders, "Towards a Generalized Singular Value Decomposition," SIAM J. Num. Anal., Vol. 18, 1981, pp 241-256 [31] R.F. Sincovec, B. Dembart, M.A. Epton, A.M. Erisman, J.W. Manke, E.L. Yip, "Solvability of Large Scale Descriptor Systems", Final Report DOE Contract ET-78-GO1-2876,Boeing Computer Services Co.,Seattle Washington, 1979 [32] G.W. Stewart, "Error and Perturbation Bounds for Subspaces Associated with Certain Eigenvalue Problems", SIAh4 Review, Vol. 15,1973, pp 752-764 [33] G.W. Stewart, "Gershgorin Theory for the Generalized Eigenproblem Ax=zBx", Math. of Comp., Vol. 29, No. 130, April 1975, pp 600-606 [34] G.W. Stewart, "A Method for Computing the Generalized Singular Value Decomposition," in [23], pp 207-220 [35] G.W. Stewart, "Computing the CS Decomposition of a Partitioned Orthonormal Matrix," Num. Math., 1982, pp 297-306 [36] J-G. Sun, "Perturbation Analysis for the Generalized Eigenvalue and the Generalized Singular Value Problem", in [23], pp 221-244 [37] J-G. Sun, "Perturbation Analysis for the Generalized Singular Value Problem", SIAM J. NUm. Anal., Vol. 20, 1983, pp 611-625 [38] J-G. Sun, "Orthogonal Projections and the Perturbation of the Eigenvalues of Singular Pencils", Journal of Computational Mathematics, China, Vol. 1, No. 1, 1983, pp 63-74 [39] P. Van Dooren, "The Computation of Kronecker's Canonical Form of a Singular Pencil", Lin. Alg. Appl., Vol. 27, 1979, pp 103-141 [40] P. Van Dooren, "The Generalized Eigenstructure Problem in Linear System
Stably Computing the Kronecker Structure
3 23
Theory", IEEE Trans. Aut. Contr., Vol. AC-26, No. 1, 1981, pp 111-129 [41] P. Van Dooren, "Reducing Subpaces: Definitions Properties and Algorithms", In [23] pp 58-73 [42] C. Van Loan, "Generalizing the Singular Value Decomposition", SIAM J. Numer. Anal., Vol. 12, 1975, pp 76-83 [43] C. Van Loan, "Computing the CS and the Generalized Singular Value Decompositions", Numer. Math., Vol. 46, No. 4, 1985, pp 479-491 [44] W. Waterhouse, "The Codimension of Singular Matrix Pairs", Lin. Alg. Appl., Vol. 57, 1984, pp 227-245 [45] G.C. Verghese, B.C. Levy, T. Kailath, "A Generalized State-Space for Singular Systems", IEEE Trans. Aut. Contr., Vol. AC-26, No. 4, 1981, pp 811-831 [46] J.H. Wilkinson, "Linear Differential Equations and Kronecker's Canonical Form", In Recent Advances in Numerical Analysis, Ed. C. de Boor, G. Golub, Academic Press, 1978, pp 231-265 [47] J.H. Wilkinson, "Kronecker's Canonical Form and the QZ Algorithm", Lin. Alg. Appl., Vol. 28, 1979, pp 285-303 [48] W.M. Wonham, Linear Multivariable Control: A Geometric Approach, Second ed.,New York, Springer-Verlag, 1979 Acknowledgements The authors would like t o thank several sources for their support of this work. Gene Golub provided both financial support and a pleasant working environment to both authors for several weeks during summer 1985. James Demmel has received support from the National Science Foundation (grant number 8501708). Bo K8gstrBm has received support from the Swedish Natural Science Research Council (contract IWRS-FU1840-101). Both authors have been supported by the Swedish National Board for Technical Development (contract STU-84-5481). Finally, both authors wish to thank Jane Cullurn, Ralph Willoughby, and IBM for their support in helping the authors present their results at the Large Eigenvalue Problems conference of the IBM Europe Institute at Oberlech, Austria, in July 1985.
This Page Intentionally Left Blank
325
Names and addresses of authors (and coauthors) who have papers in this volume.
Professor Christian Bischof Department of Computer Science Cornell University 405 Upson Hall Ithaca, NY 14853 USA Dr. Francoise Chatelin (Speaker) Centre Scientifique IBM-France 36 Avenue Raymond PoincarB 75 116 Paris, France Dr. Jane Cullum (Speaker) IBM T.J. Watson Research Center P.O. Box 218 Yorktown Heights, NY 10598 USA Professor James Demmel Department of Computer Science Courant Institute 25 1 Mercer Street New York. NY 10012 USA Dr. Jack J. Dongarra (Speaker) Mathmatics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue Argonne, IL 60439 USA Professor Thomas Ericsson (Speaker) Institute of Information Processing University of Umea S-901 87 Umea, Sweden Professor J.H. Freed Baker Laboratory of Chemistry Cornell University Ithaca, NY 14853 USA Dr. Roger G. Grimes Boeing Computer Services 565 Andover Park West Tukwila. WA 98188 USA
Dr. Erwin Haller (Speaker) IBM Germany SP PE Entwicklung C1 D-7032 Sindelfingen, West Germany Professor Ilse Ipsen (Speaker) Department of Computer Science Yale University Box 2158 Yale Station New Haven, CT 06520 USA Professor Bo Kagstrom (Speaker) Institute of Information Processing University of Umea S-901 87 Umea, Sweden Dr. Wolfgang Kerner (Speaker) Max-Planck-Institut fur Plasmaphysik D-8046 Garching Fed. Rep. of Germany Professor Horst Koppel Theoretische Chemie University of Heidelberg Heidelberg, Germany Dr. John G. Lewis (Speaker) Boeing Computer Services 565 Andover Park West Tukwila, WA 98188 USA Professor Giorgio Moro (Speaker) Dipartimento di Chimica Fisica Universita di Padova 3513 1 Padova, Italy Professor Axel Ruhe (Speaker) Department of Computer Science C h a h e r s University of Technology S-4 12 96 Goteborg, Sweden Professor Youcef Saad Department of Computer Science Yale University Box 2158 Yale Station New Haven, CT 06520 USA
326
Addresses of Airtlzors and Other Workslzop Speakers
Dr. David S. Scott Intel Scientific Computers 15201 N.W. Greenbriar Parkway Beaverton, OR 97606 USA
Professor James E. Van Ness (Speaker) Department of EE and CS Northwestern University Evanston, IL 60201 USA
Dr. Horst D. Simon Boeing Computer Services 565 Andover Park West Tukwila, WA 98188 USA
Dr. Ralph A. Willoughby IBM T.J. Watson Research Center P.O. Box 21 8 Yorktown Heights, NY 10598 USA
Dr. Daniel C. Sorensen Mathmatics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue Argonne, IL 60439 USA
Professor Robert E. Wyatt, (Speaker) Department of Chemistry The University of Texas Austin, TX 78712-1167 USA
Professor Charles Van Loan (Speaker) Department of Computer Science Cornell University 405 Upson Hall Ithaca, NY 14853 USA
Professor Yong Feng Zhou Wuhan Digital Engineering Institute PO Box 223 Wuchang, Wuhan Peoples Republic of China
Addresses of Aiitlrors arid Other Workshop Speakers
Names and addresses of speakers who do not have a paper in this volume. Professor Gene Golub Department of Computer Science and Numerical Analysis Stanford University Stanford, CA 94305 USA
Title Talk:
Some Modified and Inverse Eigenvalue Problems
Dr. Kosal C . Pandey IBM T. J. Watson Research Center P. 0. Box 218 Yorktown Heights, NY 10598 USA
Title Talk: Large Scale Eigenvalue Problems: Surface Physics
A New Algorithm and Application to
Professor Beresford Parlett Department of Mathematics University of California Berkeley, CA 94720 USA
Title Talk:
(1) How to Maintain Semi-Orthogonality Among Lanczos Vectors
Title Talk:
(2) Useable Error Bounds for the Unsymmetric Eigenvalue Problem
Professor G.W. Stewart Department of Computer Science University of Maryland College Park, MD 20742 USA
Title Talk:
( 1 ) An Introduction to the Algebraic Eigenvalue Problem
Title Talk:
(2) Experiments with a Parallel Algorithm for Solving the non-Hermitian Eigenvalue Problem.
327
This Page Intentionally Left Blank
329
INDEX
Jordan Structure, 100, 297 Kronecker Structure, 283 Krylov Subspace, 88, 193 EIGENVALUES/VECTORS
ALGORITHMS Amoldi’s Method, 131, I93 Balancing, 204, 275 Block Jacobi, 51 Block Lanczos. 90 Continuation Method, 241 Chaining, 20 Defect Correction Method, 278 Euler-Newton Continuation, 121, 127 Galerkin Method, 241 Inverse Iteration, 104. 130. 248, 267 Lanczos Recursion, 67, 86, 105, 163, 193 Newton Iteration, 134,278 Numerical Path Following, 121 Perturbation Analysis, 302 Power Iteration, 105 Rank One Modification, 27 Rayleigh Quotient, 209, 248, 278 Simultaneous Iteration, 87, 193 Singular Value Decomposition, 51, 91, 2x7, 294, 301 Spurious Test, 205 Subspace Iteration, 87 ARCHITECTURE, DATA MANAGEMENT AND SOFTWARE Assembly Language, 21 Computer Architecture, 15 Data Dependency Rules, 26 Data Exchange, 37 Data Movement, 20 Distributed Memory, 38 EISPACK, 23.73, 196 Granularity, 26 LINPACK, 23,28, 64,249 Mcmory Conflicts, 22 Multi-Processors, 37 Parallel Processing, 2 1, 5 1 Performance Measure, 37 Pipelining, 19 Portability, 15 Program Modules, 23 Shared Memory, 38 Super-vector, 2 1 Synchronization, 22, 37 Systolic Array, 43 Transportability, 34 Vector Computers, 19 CANONICAL FORMS Generalized Inverse, 269 Jordan Basis, 269
Alfven Modes, 241 Defective Eigenvalue, 195, 267 Eigenvalue Cluster, 271 Eigenvalue Sensitivity. 184 Eigenvectors, 195, 269 Eigenvalue Gaps, 214 Generalized Eigenvalues. 81, 95, 241, 283 Global Spectral Condition, 273 Nonlinear Eigenvalues, 121 Ritz Vectors, 109, 204 Semi-simple Eigenvalue, 267 Spectrum, 213, 243, 267 Vibronic Eigenstates. 174 EQUATIONS, OPERATORS AND TRANSFORMATIONS Algebraic Riccati Equation, 280 Descriptor Systems, 291 Diagonal Similarity Transformation, 275 Dynamic Systems, 124 Electric Power Systems, 18 1, 2 I9 Equivalence Transformation, 285 Fokker-Planck Equations, 146 Fourier-Laplace Transform, 145 Hamilton Operator, 68, 165, 167 Maxwell’s Equations, 244 Molecular Dynamics, 143 Partial Differential Equations, 2 15 Schmidt Factorization, 279 Schrodinger Equation, 68, 165, 241 Singular System of Differential, 283 Equations Stochastic Liouville Operator, 148 Sylvester Equation, 272 MATRICES Biorthogonal, 152, 194 Block Diagonal, 272,2X5 Block Triangular, 202, 272, 297 Canonical Angles, 272 Complex Symmetric, 143, 176, 200 Condition Number, 267 Hermitian Matrix, 243. 273 Hessenberg Matrix, 193 Ill-Conditioned, 91, 267 Ill-Posed, 283 Invariant Subspace, 267 Irreducible, 202 Nonsymmetric Matrices, 85, 193, 241 Normal Matrices, 270 Orthogonal Projection, 193, 268 Orthonormal Bases, 193, 271 Persymmetric, 198 Pseudo-Inverse, 96 Real Symmetric Matrix, 193, 241 Reorthogonalize, 194
Index to Proceedings
330 Schur Decomposition, 28 1 Span, 268 Spectral Condition Number, 269 Spectral Density, 143, 163 Tridiagonal Matrix, 68, 144, 193 Unitary, 273 PENCILS Defective Pencil, 104 Definite Pencil, 98 Generic Pencil, 287 Reducing Subspaces, 286, 305, 315 Regular Pencil, 302 Singular Pencil, 97, 284, 293, 304 SYSTEMS AND THEIR PROPERTIES Adiabatic, 164, 167 Basis Functions, 144 Bifurcation, 121 Complexity, 267 Continued Fraction, 144 Control Theory, 280 Controllable, 290 Dynamic Instabilities, 181 Electrical Transmission Network, 182 Electronic Motion, 164 Feedback Control, 181 Finite Elements, 82, 241
Generating Stations, 182 Linear Systems Theory, 3 11 Low Dimension Approximation, 283 Magnetic Resonance, 143 Markov Chains, 145, 213 Nuclear Motion, 164 Observable, 290 Plasma, 241 Polyatomic Molecules, 164 Radio Active Decay, 174 Root Loci, 184 Rotational Motion, 146, 164 Roundoff Error, 267 Sobolev Space, 246 Spectral Transformation, 87, 121 Spectroscopy, i43, 165 Spontaneous Oscillations, 184 Statistical Mechanics, 147 Structural Engineering, 81 Subsynchronous Response, 190 System Response, 182 Time Constants, 182 Transition Probabilities, 67 Transient Stability, 193 Turning Point, 121 Vector Contamination, 107 Vibrational Motion, 164
