Symposium On The Theory Of Numerical Analysis

Lecture Notes in Mathematics A collection of informal reports and seminars Edited by A. Dold, Heidelberg and B. Eckmann, Z0rich

193 Symposium on the Theory of Numerical Analysis Held in Dundee/Scotland, September 15-23, 1970

Edited by John LI. Morris, University of Dundee, Dundee/Scotland

Springer-Verlag Berlin. Heidelbera • New York 1971

A M S S u b j e c t Classifications (1970): 6 5 M 0 5 , 6 5 M 1 0 , 6 5 M 15, 6 5 M 3 0 , 6 5 N 0 5 , 6 5 N 10, 6 5 N 15, 65N20, 65N25

I S B N 3-540-05422-7 Springer-Verlag Berlin • H e i d e l b e r g • N e w Y o r k I S B N 0-387-05422-7 Springer-Verlag N e a r Y o r k • H e i d e l b e r g • Berlin

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically those of translation, reprinting, re-use of illustrations, broadcasting, reproduction by photocopying machine or similar means, and storage in data banks. Under § 54 of the German Copyright Law where copies are made for other than private use, a fee is payable to the publisher, the amount of the fee to be determined by agreement with the publisher. © by Springer-Verlag Berlin • Heidelberg 1971. Library of Congress Catalog Card Number 70-155916. Printed in Germany. Offsetdruck: Julius Beltz, Hemsbach

Foreword This publication by Springer Verlag represents the proceedings of a series of lectures given by four eminent Numerical Analysts, namely Professors Golub, Thomee, Wachspress and Widlund, at the University of Dundee between September 15th and September 23rd, 1970o The lectures marked the beginning of the British Science Research Council's sponsored Numerical Analysis Year which is being held at the University of Dundee from September 1970 to August 1971.

The aim of this year is to promote the theory

of numerical methods and in particular to upgrade the study of Numerical Analysis in British universities and technical colleges.

This is being effected by the

arranging of lecture courses and seminars which are being held in Dundee throughout the Year.

In addition to lecture courses research conferences are being

held to allow workers in touch with modern developments in the field of Numerical Analysis to hear and discuss the most recent research work in their field.

To

achieve these aims, some thirty four Numerical Analysts of international repute are visiting the University of Dundee during the Numerical Analysis Year.

The

complete project is financed by the Science Research Council, and we acknowledge with gratitude their generous support.

The present proceedings, contain a great

deal of theoretical work which has been developed over recent years. however new results contained within the notes.

There are

In particular the lectures pre-

sented by Professor Golub represent results recently obtained by him and his coworkers.

Consequently a detailed account of the methods outlined in Professor

Golub's lectures will appear in a forthcoming issue of the Journal of the Society for Industrial and Applied Mathematics (SIAM) Numerical Analysis, published jointly by &club, Buzbee and Nielson. In the main the lecture notes have been provided by the authors and the proceedings have been produced from these original manuscripts. is the course of lectures given by Professor Golub.

The exception

These notes were taken at

the lectures by members of the staff and research students of the Department of Mathematics, the University of Dundee.

In this context it is a pleasure to ack-

nowledge the invaluable assistance provided to the editor by Dr. A. Watson, Mr.

IV

R. Wait, Mr. K. Brodlie and Mr. G. McGuire. Finally we owe thanks to Misses Y. Nedelec and F. Duncan Secretaries

in

the Mathematics Department for their patient typing and retyping of the manuscripts and notes.

J. L1. Morris Dundee, January 1971

Contents G . G o l u b : D i r e c t M e t h o d s for S o l v i n g E l l i p t i c D i f f e r e n c e Equations . . . . . . . . . . . . . . . . . . . . . . . . . . I. 2. 3. 4. 5. 6. 7. 8. 9. G.Golub: I. 2. 3. 4. 5. 6. 7. 8.

I

Introduction . . . . . . . . . . . . . . . . . . . . . . . Matrix Decomposition . . . . . . . . . . . . . . . . . . . Block Cyclic Reduction . . . . . . . . . . . . . . . . . . Applications . . . . . . . . . . . . . . . . . . . . . . . The B u n e m a n A l g o r i t h m and V a r i a n t s . . . . . . . . . . . . A ~ c u r a c y of the B u n e m a n A l g o r i t h m s . . . . . . . . . . . . Non-Rectangular Regions . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . Matrix

Methods

in M a t h e m a t i c a l

Programming

2 2 6 10 12 14 15 18 18

. . . . . . .

Introduction . . . . . . . . . . . . . . . . . . . . . . . Linear Programming . . . . . . . . . . . . . . . . . . . . A S t a b l e I m p l e m e n t a t i o n of the S i m p l e x A l g o r i t h m . . . . . I t e r a t i v e R e f i n e m e n t of the S o l u t i o n . . . . . . . . . . . Householder Triangularization . . . . . . . . . . . . . . Projections . . . . . . . . . . . . . . . . . . . . . . . Linear Least-Squares Problem . . . . . . . . . . . . . . . Least-Squares Problem with Linear Constraints ...... Bibliography . . . . . . . . . . . . . . . . . . . . . . .

21 22 22 24 28 28 31 33 35 37

V . T h o m @ e : T o p i c s in S t a b i l i t y T h e o r y for P a r t i a l D i f f e r e n c e Operators . . . . . . . . . . . . . . . . . . . . . . . . . .

41

Preface . . . . . . . . . . . . . . . . . . . . . . . . . 42 I. I n t r o d u c t i o n . . . . . . . . . . . . . . . . . . . 43 2. I n i t i a l - V a l u e P r o b l e m s in L ~ w ~ t h C o n s t a n t C o e f f i c i e n t s . 51 3. D i f f e r e n c e A p p r o x i m a t i o n s in L ~ to I n i t i a l - V a l u e P r o b l e m s with Constant Coefficients . . . . . . . . . . . . . . . . 59 4. E s t i m a t e s in the M a x i m u m - N o r m . . . . . . . . . . . . . . 70 5. On the R a t e of C o n v e r g e n c e of D i f f e r e n c e S c h e m e s . . . . . 79 References . . . . . . . . . . . . . . . . . . . . . . . . 89 E . L . W a c h s p r e s s : I t e r a t i o n P a r a m e t e r s in the N u m e r i c a l S o l u t i o n Elliptic Problems . . . . . . . . . . . . . . . . . . . . . .

of

I. A C o n c i s e R e v i e w of the G e n e r a l T o p i c and B a c k g r o u n d Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 2. S u c c e s s i v e O v e r r e l a x a t i o n : T h e o r y . . . . . . . . . . . . 3. S u c c e s s i v e O v e r r e l a x a t i o n : P r a c t i c e . . . . . . . . . . . 4. R e s i d u a l P o l y n o m i a l s : C h e b y s h e v E x t r a p o l a t i o n : T h e o r y 5. R e s i d u a l P o l y n o m i a l s : P r a c t i c e . . . . . . . . . . . . . . 6. A l t e r n a t i n g - D i r e c t i o n - l m p l i c i t Iteration . . . . . . . . . 7. P a r a m e t e r s for the P e a c e m a n - R a c h f o r d V a r i a n t of Adi 0 . W i d l u n d : I n t r o d u c t i o n to F i n i t e D i f f e r e n c e A p p r o x i m a t i o n s to Initial Value Problems for Partial Differential Equations I. 2. 3. 4. 5. 6.

Introduction . . . . . . . . . . . . . . . . . . . . . . . The F o r m of the P a r t i a l D i f f e r e n t i a l E q u a t i o n s . . . . . . The F o r m of the F i n i t e D i f f e r e n c e S c h e m e s . . . . . . . . A n E x a m p l e of D i v e r g e n c e . The M a x i m u m P r i n c i p l e ..... The C h o i c e of N o r m s and S t a b i l i t y D e f i n i t i o n s ...... Stability, Error B o u n d s and a P e r t u r b a t i o n T h e o r e m .

93 95 98 100 .102 103 106 .107

.111 112 114 117 121 124 .133

VI

7. The y o n N e u m a n n C o n d i t i o n , D i s s i p a t i v e and M u l t i s t e p Schemes . . . . . . . . . . . . . . . . . . . . . . . . . 8. S e m i b o u n d e d O p e r a t o r s . . . . . . . . . . . . . . . . . . 9. Some A p p l i c a t i o n s of the E n e r g y M e t h o d . . . . . . . . . 10. M a x i m u m N o r m C o n v e r g e n c e for L 2 S t a b l e S c h e m e s ..... References . . . . . . . . . . . . . . . . . . . . . . .

138 142 145 149 151

Direct Methods for Solving Elliptic Difference Equations GENE GOLUB Stanford University

i.

Introduction General methods exist for solving elliptic partial equations of general type

in general regions.

However, it is often the ease that physical problems such as

those of plasma physics give rise to several elliptic equations which require to be solved mauy times.

It is not unco~non that the elliptic equations which arise re-

duce to Poisson's equation with differing right hand side.

For this reason it is

judicious to use direct methods which take advantage of this structure and which thereby yield fast and accurate techniques for solving the associated linear equations. Direct methods for solving such equations are attractive since in theory they yield the exact solution to the difference equation, whereas commonly used methods seek to approximate the solution by iterative procedures [12].

Hockney [8] has

devised an efficient direct method which uses the reduction process•

Also Buneman

[2] recently developed an efficient direct method for solving the reduced system of equations.

Since these methods offer considerable economy over older tech-

niques [5], the purpose of this paper is to present a unified mathematical development and generalization of them.

Additional generalizations are given by

George [6].

2.

Matrix Decomposition

Consider the system of equations

= ~ ,

(2.1)

where M is an NxN real symmetric matrix cf block tridiagonal form,

M

A

T

T

A

e

(2.2)

= •

•

W

T

A

The matrices A and T are p×p symmetric matrices and we assume that AT = TA .

This situation arises in many systems•

However, other direct methods which are

applicable for more general systems are less efficient to implement in this case. Moreover the classical methods require more computer storage than the methods te be discussed here which will require only the storage of the vector ~. commute and are s ~ e t r i c ,

Since A and T

it is well known Ill that there exists an orthogonal

matrix Q such that QT A Q = A,

(2.3)

QT T Q = 0 ,

The matrix Q is the set ef eigenvectars of

and A and O are real diagonal matrices.

A and T, and A and n are the diagonal matrices of the p-distinct eigenvalues cf A and T, respectively• To conform with the matrix M, we write the vectors x and ~ in partitioned form,

x

--

° i I

X

~q

Furthermore, it is quite natural to write

xj =

L

X

(2.~_)

~Cj =

.

,

2J

I

x2j

I

pj

I

j

YPJI

System (2.2) may be written

(2.5a) J = 2,3,...,q-1 ,

(2.5b)

T~q_I + AX~q = ~

(2.5e)

.

Frem Eq. (2.3) we have A = Q A QT

and

T = Q O QT •

Substituting A and T into Eq. (2.5) and pre-multiplying by QT we obtain

(,i

=

2,3,...,q-i)

(2.6)

where

-x..i = Q~x ~CI ' Z,i = Q~'~J ' zf~

J = 1,2,...,q.

and ~j are partitioned as before then the ith components of Eq. (2,6) may be

rewritten as

u

N

u

~iXij_l + kiXij + ~ixij+l = ~-~j ,

(j = 2,...,q-~)

,

wiXiq-I + klXiq = Ylq j fer i = 1 , 2 , . . . p p . If we rewrite the equatio~by reversing the rolls of i and J we may write --=

%

ri =

P

6oi

"

Xil

Xiq

-

qxq

-]

N

xi2

Xi

Yil

A

Yi2

1

so that Eq. (2.7) is equivalent to the block diagonal system of equations,

ri~o ~ ,

( i ~ 1,2,...,p).

(2.8)

Thus, the vector ~isatisfies a symmetric tridiagonal system of equations that has a constant diagonal element and a constant super- and sub- diagonal element.

After Eq.

(2.8) has been solved block by block it is possible to solve for ~j = Q~j.

Thus we

have: Algorithm 1 1. Compute or determine the eigensystem of A and T.

2.

0o~pute ij Q~j (J

1,2,...,ql.

3. Solve ri~i = ~

(i = 1,2,...,p).

~. Compute xj = ~j

(j . 1,2,...,q).

It should be noted that only Q, and the yj, j = 1,2,...,q

have to be stored,

A

since _~ oan over~rite the ~ j the ^~ can overwrite the ~ and the ~joan overwrite the

~j. A simple °aleulatien will show that approximately 2plq + 5Pq arithmetic opera-

tors are required for the algorithm

when step 3 is solved

using @aussian el4m4~a-

tion for a tridiagonal matrix when r i are positive definite.

The arithemtic opera-

ters are dominated by the 2p2q multiplications arising from the matrix multiplications of steps 2 + 4.

It is not easy to reduce this re,tuber unless the matrix Q ham

special properties (as in Poisson's equation) when the fast Fourier transform can be

used (see Hookney [8]). For our system ki

wi

ri = "

"

~i

si

ki

and the eigenvalues may be written down as vir = k i + 2~ i cos r_~ q+l

r = 1,2,..., q

er that r i = Z V i ZT , rs~ V i the diagonal matrix ef eigenvalues of r i and Zrs = o s sin ~ . Since r i and rj have the same set of eigenvectors

r i rj = rj ri . Because of this decomposition, step (3) can be solved by computing ~i = Z V~' Z T where the Z is stored for each r i.

This therefore requires of the order of 2pq"

multiplications and this approximately doubles the computing time for the algorithm. Thus performing the fast Fourier transform method in step 3 as well as steps 2 and is not advisable.

3.

Block C,yclic Reductien In Section 2, we gave a method for which one had to know the eigenvalues and

eigenvectors of some matrix.

We now give a more direct method for solving the

system of Eq. (2.1). We assume again that A and T are symmetric and that A and T commute.

Further-

more, we assume that q = m-I and m

= 2k + l

where k is some positive integer.

~.i-2 + A~j-I

Let us rewrite Eq. (2.5b) as follows:

+ ~J

= ~J-l '

TXj_l + A~j + Txj+ 1 ~j

= ~j ,

÷ ~J+l + ~J+2

= ~j+l "

Multiplying the first and third equation by T, the second equation by -A, and addim@ we h a v e T2xj_ 2 + (2T" - A 2)xj + T2xj+ 2 = T~j_I - A~j + T~j+I . Thus if j is even, the new system of equations involves x.'s with even indices. ~j Similar equations held for x• and Xm_ 2.

The process of reducing the equations in

this fashion is known as c2clic reduction.

Then Eq. (2.1) may be written as the

following equivalent system:

F

( 2T2 -A" ) T"

( 2T 2-A" )

T2

•

•

•

o

•

@

o

(2'~'~')

~m_n,

I. k, ~+~

-

-~ (3.1)

e e

and

~j

= Zj + ~(Xj_l + Xi+l)

J = 3,5,...,m-3

(3.2)

Since m = 2k+l , and the new system of Eq. (3.1) involves xj's with even indlcesp the block dimension ef the new system of eqtmticns is 2k-l.

Note that once Eq. (3.1) is

solved, it is easy to solve for the xj's with odd indices as evidenced by Eq. (3.2)• We shall refer to the system of Eq. (3.2) as the eliminated equations. Also, note that Algorithm i may be applied to System (3.1).

Since A and T

commute, the matrix (2Ta-A a) has the same set of eigenvectors as A and T. ~(A) = ki, ~(T) = % , =

-

Also, if

for i = 1,2,...,m-l, .

Heckney [8] has advocated this procedure. Since System (3.1) is block tridiagonal and of the form of Eq. (2.2), we can apply the reduction repeatedly until we have one block.

However, as noted above, we

can stop the process after any step and use the toothed of Section 2 to solve the

resulting equations. To define the procedure recursively, let

A (°) = A, T (e) = T; ~ ~o) = Zj,

(j = 1,2, -" .,m-l).

(3.3)

Then for r = O,l,..,k A (r+l) = 2(T(r)) = _ (A(r)) =,

T (r+z) = (T(r))" ,

(3.~)

~(r-1) = T(r) • (r) j

. (r)

- A(r)

~j+2 r

~J-2 r +

(r)

Yj

•

The eliminated equations at each stage are the solu~on of the diagonal system

(r-l)

A (r-l) X2r_2r_ , = ~2r_2r-,

A(r-1) Xj2r-2r"

- T(r-l) X2r

= ~ j(r-l) 2r-2 r-'

- T(r-1) (xj2 r + x ( j - 1 ) 2 r

)

(3.5)

j = 1,2,...,2 k-r .

A(r-1) ~ . I

(r-l)

_2r., = ~ k + l _ 2 r . , - T(r'l) X2k+l_2r

After all of the k steps, we must solve the system of equations A(k)

. (k)

~2 k -- ~2 k

.

(3.6)

In either ease, we must solve Eq. (3.5) to find the eliminated unknowns, Just a~ in Eq. (3.2).

If it is done by direct solution, an ill-conditloned system may arise.

Furthermore A = A(°)is tridiag~nal A (i) is quindiagonal simple structure of the original system.

and so on destroying the

Alternatively polynomial factorization

retains the simple structure of A. From Eq. (3.1), we note that A (1) is a polynomial of degree 2 in A and T.

By

induction, it is easy to show that A (r) is a polynomial of degree 2r in the matrices A and T, so that

2r-I

A(r) = ~

e(r)2j A2j T2r-2j "~ P2 r(A'T)"

We shall proceed %0 determine the linear factors of P2r(A,T).

Let

2r-I

j--o For t ~ O, we make the substitution

a/~

:

-2

OOS

e

(3.7)

.

From Eq. (3.3), we note that I

(3.8)

p2r÷1(a,t) = 2t2~÷ _ (p2r(a,t))~ It is then easy to verify using Eq~. (3.7) and (3.8), that P2r(a,t) = - 2 t 2r cos 2re , and, consequently 2r

(,

+ 2t

cos -~-2-i ~2~+,~ ,)

,

J=l and, hence,

A (r) = - ~

(A + 2 cos e!r)T)~ ,

(3.9)

01

(r) = (2j_I)~/2~+,

where ~j

Thus to solve the original system it is only necessary to solve the factored system recursively.

For example when r = 2, we obtain A (1) = 2 ~

- A m = (~

T - A)(~

T + A)

whence the simple tridiagonal systems (J: T - A )

~=~

(4~ T + A )

x = w

are used to solve the system A(1)x = ~ • We call this method the cyclic odd-even reduction and factorization (CORF) algorithm.

10

4. Applications Exampie I

Poissen's Equation wit h Dirichlet Boundar~ Conditions,

It is instructive to apply the results of Section 3 to the solution of the finite-difference approximation to Poisson's equation on a rectangle, R, with specified boundary values• u

Consider the equation

+ u ~x

: f(x,y) for (x,y)ER, yy

(~.l)

u(x,y)

: g(x,y) for (x,y)¢aR .

(Here aR indicates the boundary of R.)

We assume that the reader is familiar

with

the general technique of imposing a mesh of discrete points onto R and approximating ~q. (4.Z).

The eq~tion u

+ Uyy : f(x,y) is approximated at (xl,Yj) by

Vi-l.j - 2vi,j + Vi+l.j C~)" = fi,J

vi,j-1 - 2vi. j + vi.j+l (Ay)"

+

(i < i < n-l, I < J < m-i) ,

with appropriate values taken on the boundary VO,J = gC,~'

Vm, j = gm,J

( 1 g J g m-l)

Vi,@ = gi,o'

vi,m : gi,J

(i < i ~ n-l).

,

and

Then vii is an approximation to u(xi,Yj) , and fi,j = f(xi'Yj)' gi,j : g(xl,Yj)Hereafter, we assume that m

-~

2k+l

•

When u(x,y) is specified on the boundary, we have the Dirichlet boundary condition.

For simplicity, we shall assume hereafter that Ax = Ay.

l 1

•-4

(~

© I

•

.

1

and T = I . . l •

1 -4

(n-l)

x (n-l)

Then

11

The matrix In_ I indicates the identity matrix of order (n-l).

A and T are symmetric

and co~ute, and, thus the results of Sections 2 and 3 are applicable•

In addition,

since A is tridlagcnal, the use of the facterization (3.10) is greatly simplified. The nine-polnt difference formula for the same Poisson's equation can be treated similarly when m

-20

4 A

4

O

-20

=

•

,

"~

z

1

4

&

1

T=

(~

•

0

0

.

-20

.

I

1

&

(n-l)×~n-ll

Example II The method can also be used for Poisson's equation in rectangular regions under natural boundary conditions provided one uses au = u(x + ~ . y ) - u(x - ~ . y ) Ox 2h

and similarly ~

• at the boundarie

S,

Example III Poisson's equation in a rectangle with doubly periodic boundary conditions is an additional example when the algorithm can be applied. Example IV The method can be extended successfully to three dimensions for Foissents equation. For all the above examples the eigensystems are k n o w n an~ the fast F o u r i e r transform can ~e applied, Example V The equation of the form (~(x)~)x + ( K Y ) ~ ) y

+ u(x,y) = q(x,y)

on a rectangular region can be solved by the CORF algorithm provided the eigensystem is calculated since this is not generally known.

12

The counterparts in cylindrical polar co-ordinates can also be solved using CORF on the ractangle~ in the appropriate co-ordinates. 5.

The Buneman algorithm and variants In this section, we shall describe in detail the Buneman algorithm [2]

variation of it.

and a

The difference between the Buneman algorithm and the CORF algo-

rithm lies in the way the right hand side is calculated at each stage of the reduction.

Henceforth, we shall assume that in the system of Eqs (2.5) T = Ip, the

identity matrix of order p. Again consider the system of equations as given by Eqs. (2.5) with q = 2k+l-1. After one stage of cyclic reduction, we have

5j-2 +

(21 p - A')~j + 5j+2 = ZJ-I + ZJ+I - A Z J

for J = 2,4,...,q-I with ~e = ~ + l

= ~ ~ the null vector.

(5.1) Note that the right han~

side of Eq. (5.1) may be written as ~J(i) = ZJ-1 + ZJ+I - ~ J

= A(1) A-'~j + ZJ-I + ~J+l - 2A-'~j

(5.2)

where A (1) = (21p- A') . Let us define 2j(i)

:

A-'Zj ; ~J-(1)= ~j-I + ~j+l " 22~I)_

(These are easily calculated since A is a tridiagon~l matrix.)

Z~(1)

=

A(1) £j _(1)

(1)

+ %j

Then

(5-3)

•

After r reductions, we have by Eq. (3.i)

(r+l) j

, (r)

(r)) -A(r) (r)

= ~ j - 2 ~ + ~j+2

~j

.

(5.4)

Let us write in a fashion similar to Eq. (5.3)

(5.5) Substituting Eq. (5.5) into Eq. (5.4) and making use of the identity (A(r)) ' = 21

- A (r+1)

P

from Eq. (3.4), we have the following relationships: (r+l) J

=

2(r) (A(r))_~ , (r) ~(r) ~r)) J ~j_2 r + ~j+2 r -

(5.6a)

13

• ( r~(r) + (r) l ) ^ (r+l) For J = i2 r+l (i = 1,2,...,2k-r-l) with ~!r) = ~(r) (r) = ~(r) = O 2k+l = 2k+l - • Because the number of vectors ~ ~r) is reduced by a factor of two for each successive r, the computer storage requirements becomes equal to almost twice the number of data points.

To compute

(A(r))-'(~J-2(r)r + ~J +2r~(r) _ ~r)) in Eq. (5.6a). we solve the system

of equations A(r) , (r) !,~j

(r+l)

)

- ~,J

(r)r

(r)

== ~ J - 2

+ ~j+2 r -

(r) ~j

'

where A (r) is given by the factorization Eq. (3.9); namely,

A (r)

2r ~ -

=

(A + 2 cos 8(r)j Ip)

,

J=l

o~r)

= (2~ -

~)~/2 r~l

•

After k reductions, one has the equation

.

A (k) x k

=

A(k)

(k)

~2 k

,~(k)

+ ~2k

2

and hence ~(k) (A(k))_1 ~(k) ~2k = ~2k + ~2k

•

Again one uses the factorization of A (k) for computing (A(k)) -I ~I~ ) . Te back solve, we use the relationship ~J -2r + A(r) ~J + ~J +2r = A(r) ~r) + ~r) for J = i2r(l,2,...,2k+l-r-1) with ~o = ~2k+ 1 = ~ • For J = 2r, 3.2r,...,2k+l-2 r, we solve the system of equations A(r)(xj - ~r)) = ~ r ) _ (xj_2r + xj+2r)

,

(5.7)

14

using the factorlzation of A(r); hence

~J

(r))

2~r) =

+

(~J £d -

(5.~3)

"

Thus to summarise, the Buneman algorithm proceeds as follows: 1.

(o)

e

((r) (r)~ Compute the sequence ~ j , ~j } by Eq. (5.6) for r = l,...,k with for J = 0,...,2 k+l , a n a ~O)Z = ~J for j = l, 2,...,2k+l-1.

2. Back-solve for ~j using Eqs. (5.7) and (5.8). The

use of the p(r) and q(r) produce a stable algorithm.

~J

~J

Numerical experi-

ments by the author and his colleagues have shown that computationally the Buneman algorihhm requires approximately 30% less time than the fast Fourier transform method of Hockney. 6.

Accuracy of the BunemanAl~orithms As was shown in Section 5, the Bunemau algorithms consist of generating the

sequence of vectors I~ r), ~J(r)l.

Let us write, using Eqs. (5.12) and (5.13)

£~r) : ~r) + ~J(r) ~J(r)

(6.la)

= ~ Xj -2r + x~j+2r - Afr)

~j(r) '

(6.1h)

where

(6.2) k:l and

S(r) = (A(r-l)...

A(O)) -' .

(6.3)

Then

I1~.~r) - .(r)ll ~il 2

~i

IIs(r)ll2

11.t1'

(6.~)

and

li~ r) - (~j_2r + ~j+2rl12

IIs(r) ACr)il2 i1~1' ,

whe re llVll2 indicates the Euclidean norm of a vector v , IICII2 indicates the spectral norm of a matrix C, and

(6.5)

15

1~1t'. ~ll~jll 2 . j=l : AT

Thus for A

r-1 Ns(r)II2

-~

I](A(J))-III2

j:o and since A( j ) are polynomials of degree 2 j in A we have r-I

lls(r)ll2 Vt max I [P2j(×±) ]"[ j:o [xif

,

where p2j(Xi) are polynomials in Ikil , the eigenvalues of A. For Poisson's equation it may be shown that

(6.6)

lls(r)II2< e'°re where o : 2r-1 and e > O. r

Thus

Hs(r)ll2 _, o and h~ce I12~r) - ~ H 2 ~ 0

That is p

~r)

tends to the exact solution wihh increasing r.

Since it can be shown

that llq~r)N2 remains bounded throughout the calculation, the Buneman

algorithm

leads to numerically stable results.

7. Non-Rectangular Regions In many situations, one wishes to solve an elliptic equation over the region

R

where there are n I data points in R I , n2 data points in R z and ne data points in R, (~ R2.

We shall assume that Dirichlet boundary conditions are given.

When Ax is

16

the same throughout the region, one has a matrix equation of the form m

G

© (7.1) P

~(2)J

pT

@

~c(2)

where "A

T

T

A

© •

G=

©

s

$

B

.

(7.2)

#

H = e

B

.

T

T

A

(~

"

n I xnl

.

S

S

B

n 2 xn=

and P is (noXno). Also, we write

x~z)

x!2~

x (1) =

x(2) •q

I

(7.3)

o

x(a) ,,,r

We assume again that AT = TA and BS = SB. From Eq. (7.1), we see that 0 0

x(1) = ~-I y(1) _ ~-1

. (7.~)

an~

17

pT

x(2) = H-I Z(2) - H-I

0

x(1)

(7.5)

,,.,r

0

Now let us write

G~(1) = ~(1), H~(2) = ~(2) ,

(7.6)

;l ~w(I)

~(2)=

=

o I

(7.7)

"I

oJ Then as -e partition the vectors z (i), z (2) and the matrices W (1) and W (2) as in Eq• (7.3), Eqs• (7.4) and (7.5) becomes

~j(i) = £ ~i) - ~(1) ,,j ~x!2) ,

(j = 1,2,...,r), (7.8)

(2) = (2) _ w!2) x(1) £J

~j

J

,,~

(j = 1,2,..•,s)• ,

For Eq. (7.8), we have

I

w(1)

(1)

r

(7.9)

w~2)

i

z(2)

It can be noted that W ~lj( ~ and W ~2j( ~ are dependent only on the given region and hence the algorithm becomes useful if many problems on the same region are to be considered. Thus, the algorithm proceeds as follows• i. Solve z(I) aria z! 2) using the methods of Section 2 or 3.

18

2. Solve for W (I) and W! 2) using the methods of Section 2 or 3. r

3. Solve Eq. (7.9) using Gaussian elimination.

Save the LU decomposition of

Eq. (7.9). h. Solve for the unknown components of ~(1) and ~(2) •

8.

Conclusion Numerous applications require the repeated solution of a Poisson equation.

The operation counts given by Dorr [5] indicate that the methods we have discussed should offer significant

economies over older techniques; and this has been veri-

fied in practice by many users.

Computational experiments comparing the Buneman

algorithm, the MD algorithm, the Peaceman-Raohford alternating direction algorithm, and the point successive over-relaxation algorithm are given by Buzbee, at al [3]. We conclude that the method of matrix decomposition, the Buneman algorithm, and Hookney's algorithm (when used with care) are valuable methods. This paper has benefited greatly from the comments of Dr. F. Dorr, Mr. J. Alan George, Dr. R. Hockney and Professor 0. Widlund. 9.

References

1.

Richard Bellman, Introduction to Matrix Analysis, McGraw-Hill, New York, 1960.

2.

Oscar Buneman, Stanford University Institute for Plasma Research, Report No.294, 1969.

3.

B.L. Buzbee, G.H. Golub and C.W. Nielson, "The Method of Odd/Even Reduction and Factorization with Application to Poisson's Equation, Part II," LA-h288, Los Alamos Scientific Laboratory. (To appear SIAM J. Num. Anal. )

4.

J.W. Cooley and J.W. Tukey, "An algorithm for machine calculation of complex Fourier series," Math. Comp., Vol.19, No.90 (1965), pp. 297-301.

o

6.

.

8.

F.W. Dorr, "The direct solution to the discrete Poisson equation on a rectangle," to appear in SIAM Review. J.A. George, "An Embedding Approach to the Solution of Poisson's Equation on an Arbitrary Bounded Region," to appear as a Stanford Report. G.H. Golub, R. Underwood and J. Wilkinson, "Solution of Ax = kBx when B is positive definite," (to be published). ~ R.W. Hockney, "A fast direct solution of Poisson's equation using Fourier analysis," J. ACM., Vol.12 No.1 (1965), pp. 95-113.

19

9.

R.W. Hockney, in Methods in Computational Physics (B. Adler, S. Fernbach an~ M. Rotenberg, Eds.), Vol.S Academic Press, New York and London, 1969.

lO.

R.E. Lynch, J.R. Rice and D.H. Thomas, "Direct solution of partial difference equations by tensor product methods," Num. Math., Vol.6 (196A), pp. 185-199.

ii.

R.S. Varga, Matrix Interative Anal2sis, Prentice Hall, New York, 1962.

Matrix Methods

in Mathematical Programming GENE GOLUB

Stanford University

22

I.

Introduction With the advent of modern computers,

matrix algorithms. Simultaneously, programming.

there has been a great development in

A major contributer to this advance is J. H. Wilkinson [30].

a considerable growth has occurred in the field of mathematical

However, in this field, until recently, very little analysis has been

carried out for the matrix algorithms involved. In the following lectures, matrix algorithms will be developed which can be efficiently applied in certain areas of mathematical programming and which give rise to stable processes. We consider problems of the following types: maximize ~ (~) , where ~ = (x,, x,, .. Xn) T subject to

Ax=

b

Gx ~ h where the objective function ~ (~) is linear or quadratic. 2.

Linear Programming The linear programming problem can be posed as follows:

m~x~i,e ~ (~) = ~ subject to

T

A~_ = b

(2.1)

) 0

(2.2)

We assume that A is an m x n matrix, with m < n, which satisfies the Haar condition (that is, every m x m submatrix of A is non-singular). said to be feasible if it satisfies the constraints

The vector ~ is

(2.1) and (2.2).

Let I = lil, i2, .. iml be a set of m indices such that, on setting xj = O, j $ I, we can solve the remaining m equations in (2.1) and obtain a solution such that xij > 0

,

J

=

I, 2, .. m .

Thi8 vector x is said to be a basic feasible solution.

It is well-known that

the vector ~ which maximizes ~ (~) = o T x is a basic feasible solution, and this suggests a possible algorithm for obtaining the optimum solution, namely, examine all possible basic feasible solutions.

23

Such a process is generally inefficient. Dantzig, is the SimylexAl~orithm.

A more systematic procedure, due to

In this algorithm, a series of basic feasible

solutions is generated by changing one variable at a time in such a way that the value of the objective function is increased at each step.

There seems to be no

way of determining the rate of convergence of the simplex method;

however, it works

well in practice. The steps involved may be given as follows: (i)

Assume that we can determine a set of m indices I = liI , i,, .. iml such that

the corresponding x i

are the non-zero variables in a basic feasible solution.

J Define the basis matrix B = [ai , Ai2,

.. aim ]

where the a are columns of A corresponding to the basic variables. --lj (ii)

Solve the system of equations: B~=b

where ~.T= [Xil, Xi, ' .. Xim] (iii)

Solve the system of equations: BT

W

^C

=

where _~T__ [ci,, ci2' .. cim] are the coefficients

of the basic variables in the

objective function. (iv)

Calculate max j £ I

If c r - ~

T

.cj (

~ T w] ~ ~. =

Cr

-

T w ~r -

, say.

w • 0 , then the optimum solution has been reached.

Otherwise, a

is to

~r

be introduced into the basis. (v)

Solve the system of equations: Bt

=

- a --r

If t

~ 0 , k = I, 2, • . m , then this indicates that the optimum solution is unrk

bounded. Otherwise determine the component s for which

xi s tr s

x = min 1 ~k~m

-

~ tr k

trk

0

24

Eliminate the column a i

from the basis matrix and introduce column a r. s This process is continued from step (ii) until an optimum solution is obtained (or shown to be unbounded). We have defined the complete algorithm explicitly, provided a termination rule, and indicated how to detect an unbounded solution.

We now show how the simplex

algorithm can be implemented in a stable numerical fashion. ~.

A stable implementation of the simplex al6orithm Throughout

the algorithm,

solved at each iteration. B~

=

there are three systems of linear equations to be

These are:

b ,

m

BTw =

c ,

Bt = --r

-a -r

Assuming Gaussian elimination is used, this requires about m3/3 multiplications for each system. are available,

However,

if it is assumed that the triangular factors of B

then only O(m 2) multiplications

are needed.

An important considera-

tion is that only one column of B is changed in one iteration, and it seem, reasonable to assume that the number of multiplications

can be reduced if use is made of this.

We would hope to reduce the m3/3 multiplications to O(m 2) multiplications per step. This is the basis of the classical simplex method.

The disadvantage of this method

is that the pivoting strategy which is generally used does not take numerical stability into consideration.

We now show that it is possible to implement the

simplex algorithm in a more stable manner, the cost being that more storage is required. Consider methods for the solution of a set of linear equations.

It is well-

known that there exists a permutation matrix n such that HB

=

LU

where L is a lower triangular matrix, and U is an upper triangular matrix. If Gaussian elimination with partial (row) pivoting is used, then we proceed as follows : Choose a permutation matrix H, such that the maximum modulus element of the

25

first column of B becomes the (I, 1)

-

element of

1"] 1

B.

Define an elementary lower triangular matrix F k as ~

rk =

I

' !

"

k|

-

! f

i

|

". ~

I

'LL I'

l

I'| ~, J

Now~

,

" ".

can be chosen so that P, HI B

has all elements below the diagonal in the first column set equal to zero. Now choose 92 so that 92 r, 9, B

has the maximum modulus element in the second column in position (2, 2), and choose r e so that r= fl~ 1"t H2 B

has all elements below the diagonal in the second column set equal to zero.

This

can be done without affecting the zeros already computed in the first column. Continuing in this way we obtain:

rm-,

~m-,...P2

~ , r, 9, B = U

where U is an upper triangular matrix. Note that permuting the rows of the matrix B merely implies a re-ordering of the right-hand-side

elements.

merely a record kept.

Thus, no actual permutation need be performed,

Further any product of elementary lower triangular matrices

is a lower triangular matrix, as may easily be shown.

Thus on the left-hand side

we have essentially a lower triangular matrix, and thus the required factorization. The relevant elements of the successive matrices F k can be stored in the lower triangle of B, in the space where zeros have been introduced. method is economical in storage.

Thus the

26

To return to the linear programming problem, we require to solve a system of equations of the form B(1)

~

v

=

(3.~)

where B (i) and B (i-I) differ in only one column (although the columns may be reordered)° Consider the first iteration of the algorithm.

Suppose that we have obtained

the factorization: B (°)

=

S (°) U(o)

where the right-hand-side vector has been re-ordered to take account of the permutations.

.i)

The solution to (3 =

with i = 0 is obtained by computing

(L~°))-~ x

and solving the triangular system

v(O)

=

each of which requires m

~

2

,

+ 0 (m) multiplications.

Suppose that the column b (°) is eliminated from B (°) and the column g(O) is S O

introduced as the last column, then BO)

=

[b(O) L

t

b(O) •

~2

. b(O) '

"

~S

"t

bCo) •

~S

0

Therefore, (~(o))

.1

BO)

=

HO)

*1 0

,

where H (I) has the form:

/

{ <

~(o)] '

" "

27

Such a matrix is called an upper Hessenberg matrix.

0nly the last column need be

computed, as all others are available from the previous step.

We require to apply

a sequence of transformations to restore the upper triangular form. It is clear that we have a particularly simple case of the LU factorization procedure as previously described, where r! I) is of the form: i

I i

R

r~I) =

I • '

I

"

' I I

#-~

k_Y

I

11

1

,q~'/ I1 .Ji

"

I I

0

only one element requiring to be calculated.

On applying a sequence of transforma-

tion matrices and permutation matrices as before, we obtain

1)

1)

..

r (1) s

o

H(1) s

=

u (1)

o

where U (I) is upper triangular. Note that in this case to obtain Hj(I) it is only necessary to compare two elements.

Thus the storage required is very small:

(m - So) multipliers gi(I) and

(m - So) bits to indicate whether or not interchanges are necessary. All elements in the computation are bounded, and so we have good numerical accuracy throughout.

The whole procedure compares favourably with standard forms,

for example, the product form of the inverse where no account of numerical accuracy is taken.

Further this procedure requires fewer operations than the method which

uses the product form of the inverse.

If we consider the steps involved, forward

and backward substitution with L (°) and U (i) require a total of m 2 multiplications and the application of the remaining transformation in (L(i)) -I requires at most i(m - I) multiplications.

(If we assume that on the average the middle column of

the Basis matrix is eliminated, then this will be closer to (i/2) (m - I) ).

Thus

a total of m 2 + i (m - I) multiplications are required to solve the system at each

28

stage, assuming an initial factorization is available.

Note that if the matrix A

is sparse, then the algorithm can make use of this structure as is done in the method using the product form of the inverse. 4"

Iterative refinement of the.solution Consider the set of equations B~

=

X

and suppose that ~ is a computed approximation to ~ . Let -- ~ + £ Therefore,

B(~ + 2)

:

v

,

that is, Be_

-- v - B ~

We can now solve for c very efficiently, since the LU decomposition of B is available. acy.

This process can be repeated until ~ is obtained to the required accur-

The algorithm can be outlined as follows: (i)

Compute ~j

=

~ - B~_j

(ii)

Solve B_cj =

r -j

(iii)

Compute ~j+1

=

~J + ~J

It is necessary for --j r to be computed in double precision and then rounded to single precision.

Note that step (ii) requires 0(m 2) operations, since the LU de-

composition of B is available. ~.

This procedure can be used in the following sections.

Householder Trian~ularization Householder transformations have been widely discussed in the literature.

In

this section we are concerned with their use in reducing a matrix A to uppertriangular form, and in particular we wish to show how to update the decomposition of A when its columns are changed one by one.

This will open the way to implemen-

tation of efficient and stable algorithms for solving problems involving linear constraints. Householder transformations are symmetric orthogonal matrices of the form Pk = I -

k UkUk where uk is a vector and Ck = 2/(

).

Their utility in this

29

context is due to the fact that for any non-zero vector 2 it is possible to choos~ u k in such a way that the transformed vector Pk a is element.

Householder

zero except for its first

[15] used this property to construct a sequence of transfor-

mations to reduce a matrix to upper-triangular form.

In [29], Wilkinson describes

the process and his error analysis shows it to be very stable. Given any A, we can construct a sequence of transformations reduced to upper triangular form.

Premultiplying by P

such that A is

annihilates

(m - 1)

O

elements in the first column.

Similarly, premultiplying by PI eliminates (m - 2)

elements in the second column, and so on. Therefore,

em-1 Pm-2 "'PI PoA

=

[ RO ]

'

(5.1)

where R is an upper triangular matrix. Since the product of orthogonal matrices is an orthogonal matrix, we can write (5.1) as QA

=

A=QT[

[ R] 0 R ] 0

The above process is close to the Gram-Schmidt process in that it produces a set of orthogonal vectors spanning E . n

In addition, the Householder transforma-

tion produces a complementary set of vectors which is often useful.

Since this

process has been shown to be numerically stable, it does produce an orthogonal matrix, in contrast to the Gram-Schmidt process. If A = (~I ,...,~n) is an mxn matrix of rank r, then at the k-th stage of the triangularization

(k < r ) we have

A (k)

= Pk-1Pk-2

"'"

PoA= 0

where R k is an upper-triangular matrix of order r.

Tk The next step is to compute

A.k+1.(~ = Pk A'k" ( ~ where Pk is chosen to reduce the first column of T k to zero except for the first component.

This component becomes the last diagonal element

30

of ~ + I

and since its modulus is equal to the Euclidean length of the first column

of T k it should in general be maximized by a suitable interchange of the columns of Sk .

After r steps, T

r

will be effectively zero (the length of each of its

Tk col~Im=~ will be smaller than some tolerance) and the process stops. Hence we conclude that if rank(A) = r then for some permutation matrix H the Householder decomposition (or "QR decomposition") of A is

r Q A ~ = Pr-1 Pr-2 "'" PO A = O where Q = P r - 1 P r - 2

0

"'" PO is an m x m orthogonal matrix and R is upper-triangular

and non-singular. We are now concerned with the manner in which Q should be stored and the means by which Q, R, S may be updated if the columns of A are changed.

We will

suppose that a column ~p a is deleted from A and that a column ~q a is added.

It will

be clear what is to be done if only one or the other takes place. Since the Householder transformations Pk are defined by the vectors uk the usual method is to store the Uk'S in the area beneath R, with a few extra words of memory being used to store the ~k'S and the diagonal elements of R. Q~ for some vector ~ is then easily computed in the form P r - 1 P r - 2 T T for example, PO ~ = (I - ~0~O~0)~ = ~ - ~o(Uo~)Uo . accomplished as follows.

The product "'" PO ~ where,

The updating is best

The first p-1 columns of the new R are the same as before;

the other columns p through n are simply overwritten by columns ap+1, ..., an, aq and transformed by the product P p - 1 P p - 2

(Sp_ I I' I ~ then T \%1

]

"'" PO to obtain a new

is triangularized as usual. p-1

This method allows Q to be kept in product form always, and there is no accumulation of errors.

Of course, if p = I the complete decomposition must be re-done

and since with m ~ a lot of work.

n the work is roughly proportional to (m-n/3)n 2 this can mean

But if p A n/2 on the average, then only about I/8 of the original

work must be repeated each updating.

31

Assume that we have a matrix A which is to be replaced by a matrix ~ formed from A by eliminating column a

and inserting a new vector g as the last column.

As in the simplex method, we can produce an updating procedure using Householder transformations.

If ~ is premultiplied by Q, the resulting matrix has upper

Hessenberg form as before.

Qi

Diagramatically,

=

/

/

<

As before, this can be reduced to an upper triangular matrix in O(m 2) multiplications. 6.

Projections In optimization problems involving linear constraints it is often necessary

to compute the projections of some vector either into or orthogonal to the space defined by a subset of the constraints (usually the current "basis").

In this

section we show how Householder transformations may be used to compute such projections.

As we have shown, it is possible to update the Householder decomposi-

tion of a matrix when the number of columns in the matrix is changed, and thus we will have an efficient and stable means of orthogonalizing vectors with respect to basis sets whose component vectors are changing one by one. Let the basis set of vectors a 1,a2,...,a n form the columns of an m x n matrix A, and let S

be the sub-space spanned by fail •

We shall assume that the

r

first r vectors are linearly independent and that rank(A) = r.

In general,

m > n > r , although the following is true even if m < n • Given an arbitrary vector z we wish to compute the projections u = Pz

,

v = (I - P) z

for some projection matrix P , such that

32

a)

z

=

u

+ v

(b)

2v

= 0

(o)

~ s r (i.e., 3~ ~uoh that ~

(d)

v is orthogonal to S r

=

~)

(i.e., ATv ~ = o)

One method is to write P as AA + where A + is the n x m generalized inverse of A, and in [7~ Fletcher shows how A + may be updated upon changes of basis.

In contrast,

the method based on Householder transformations does not deal with A + explicitly but instead keeps AA + in factorized form and simply updates the orthogonal matrix required to produce this form.

Apart from being more stable and just as efficient,

the method has the added advantage that there are always two orthonormal sets of vectors available, one spanning S

and the other spanning its complement. r

As already shown, we can construct an m x n orthogona~ matrix Q such that r

QA

=

£i

n-r

0S 1

where R is an r x r upper-triangular matrix.

Let

I r W

=

Qz

(6.~)

=

m-r

and define

~

'

X=

(6.2)

~2

Then it is easily verified that ~,~ are the required projections of ~, which is to say they satisfy the above four properties.

Also, the x in (c) is readily shown

to be

In effect, we are representing the projection matrices in the form

33

P Q C:r) (zr o)Q

(6.~)

I-P =QT (im_rO) (OI r)Q

(6.A)

=

and

and we are computing ~ = P z, Z = (I - P)~ by means of (6.1), (6.2) • col,m~R of Q span S

and the remaining m-r span its complement.

The first r

Since Q and R may

r

be updated accurately and efficiently if they are computed using Householder transformations, we have as claimed the means of orthogonalizing vectors with respect to varying bases. As an example of the use of the projection (6.4), consider the problem of finding the stationary values of xTAx subject to xTx = I and cTx = O, where A is a real symmetric matrix of order n and C is an n x p matrix of rank r, with r ! P <~ n. It is shown in [12] that if the usual Householder decomposition of C is r Qc=

(Ro

n-r OS )

th@n the problem is equivalent to that of finding the eigenvalues and eigenvectors of the matrix PA , where P

=

I-P

=

O

O

0

In_ r

Q

is the projection matrix in (6.4). Note that, although PA is not symmetric, since P~ = P , then

PA

= P2A

and further the eigenvalues of P2A are equal to the eigsnvaluee of the s ~ e t r i c matrix PAP.

The dimensionality of the problem is not reduced;

values will be zero. ~.

Linear least-squares problem The least-squares problem to be considered here is'

some of the eigen-

34

m£n

llb

A~_It

-

2

where we assume that the rank of A is n. Since length is invariant under an orthogonal transformation we have

lib

-

Axll 2

=

llQb

-

QA~_II "+

2

where

QA =

[ 1{ ]. 0

2

Let

Qb

=

c --

--

[o_, ].

:

C2

m-

n

Then,

2,

" [~_,]

1{] x

-

[o

U'

-

,

= Ha_,- ~_H"

,

+ lla.il"

and the solution to the least-squares problem is given by =

1{- 1

c,

Thus it is easy to solve the least-squares problem using orthogonal transformations. Alternatively,

the least-squares problem can be solved by constructing the

normal equations A

x = A

D

However these are well-known to be ill-conditioned. Nevertheless the normal equations can be used in the following way. Let the residual vector r be defined by: r

=

b-A~

Then, ATr

=

ATb - A T A ~

=

0

These equations can be written:

[IAA]OIr> (:Jx+ Thus,

0

IATAIiTOIOii IO(r)

I

X

Multiplying out:

(1{7o)

o

C CO/o

(7.~)

:I(:)

35

where ~ = Q E a n d S = Q ~ . This system can easily be solved for ~ and ~.

The method of iterative refine-

ment may he applied to obtain a very accurate solution. This method has been analysed by BJhrck [2]. 8.

Least-squares problem with linear constraints Here we consider the problem minimize ~

- A~_~2 2

subject to

G~ = ~

.

Using Lagrange multipliers ~ , we may incorporate the constraints into equation (7.1) and obtain

0

I

A

GT

AT

0

1

b 0

The methods of the previous sections can be applied to obtain the solution of this system of equations, without actually constructing the above matrix.

The problem

simplifies and a very accurate solution may be obtained. Now we consider the problem minimize llb - A~_~2 2

subject to

Gx ~> h .

Such a problem might arise in the following manner.

Suppose we wish to approximate

given aata by the polynomial y(t)

= ~t ~ + @t 2

such that y(t) is convex. y(')(t)

+

yt +

This implies =

6at + 2~ ) 0 .

Thus, we require 6 a t i + 2~ ) 0 where t. are the data points, (This aces not necessarily guarantee that the polyl

hernial will be convex throughout the interval. ) that

Gx - w = h

where

w ~ _O

.

Introduce slack variables w such

36

Introducing Lagrange multipliers as before, we may write the system as:

h

i O

0

G

-I

0

I

A

0

r

b

GT

AT

0

0

x

0

w

At the solution, we must have

_• ~ o ,

w~o,

T _z_w=0.

This implies that when a Lagrange multiplier is non-zero then the corresponding constraint holds with equality. Conversely, corresponding to a non-zero w i the Lagrange multiplier must be zero.

Therefore, if we know which constraints held with equality at the solution,

we could treat the problem as a linear least-squares problem with linear equality constraints.

A technique, due to Cottle and Dantzig [5], exists for solving the

problem inthis way.

37

Bibliography [11

Beale, E.M.L., "Numerical Methods", in Ngn~.inear Programming, J. Abadie (ed.). John Wiley, New York, 1967;

[2]

pp. 133-205.

Bjorck, ~., "Iterative Refinement of Linear Least Squares Solutions II", BIT 8 (1968), pp. 8-30.

[3]

and G. H. Golub, "Iterative Refinement of Linear Least Squares Solutions by Householder Transformations", BIT 7 (1967), pp. 322-37.

[4]

and V. Pereyra, "Solution of Vandermonde Systems of Equations", Publicaion 70-02, Universidad Central de Venezuela, Caracas, Venezuela, 1970.

[5]

Cottle, R. W. and @. B. Dantzig, "Complementary Pivot Theory of Mathematical Programming", Mathematics of the Decision Sclences~ Part 1, G. B. Dantzig and A. F. Veinott (eds.), American Mathematical Societ 2 (1968), pp. 115-136.

[6]

Dantzig, G. B., R. P. Harvey, R. D. McKnight, and S. S. Smith, "Sparse Matrix Techniques in Two Mathematical Programming Codes", Proceedinss of the S.ymposium on Sparse Matrices and Their Appllcations, T. J. Watson Research Publications RAI, no. 11707, 1969.

[7]

Fletcher, R., "A Technique for Orthogonalization", J. Inst. Maths. Applics. 5 (1969), pp. 162-66.

[8]

Forsythe, G. E., and G. H. Golub, "On the Stationary Values of a Second-Degree Polynomial on the Unit Sphere", J. SIAM, 13 (1965), pp. 1050-68.

[9]

and C. B. Moler, Computer Solution of Linear Algebraic Systems, Prentice-Hall, Englewood Cliffs, New Jersey, 1967.

[10] Francis, J., "The QR Transformation.

A Unitary Analogue to the LR Transforma-

tion," Comput. J. 4 (1961-62), pp. 265-71. [11] golub, G. H., and C. Reinsch, "Singular Value Decomposition and Least Squares Solutions", Numer. Math., 14(1970), pp. 403-20. [12]

and R. Underwood, "Stationary Values of the Ratio of Quadratic Forms Subject to Linear Constraints", Technical Report No. CS 142, Computer Science Department, Stanford University, 1969.

[13] Hanson, R. J., "Computing Quadratic Programming Problems:

Linear Inequality

and Equality Constraints", Technical Memorandum No. 240, Jet Propulsion

38

Laboratory, Pasadena, California, 1970.

[14]

and C. L. Lawson, "Extensions and Applications of the Householder Algorithm for Solving Linear Least Squares Problems", Math. Comp., 23 (1969), pp. 787-812.

[15] Householder, A.S., "Unitary Triangularization of a Nonsymmetric Matrix", J. Assoc. Comp. Mach., 5 (1968), pp. 339-42. [16] Lanozos, C., Linear Differential Operators.

Van Nostrand, London, 1961.

Chapter 3 • [17] Leringe, 0., and P. Wedln, "A Comparison Betweem Different Methods to Compute a Vector x Which Minimizes JJAx - bH2 When Gx = h", Technical Report, Department of Computer Sciences, Lund University, Sweden. [18] Levenberg, K., "A Method for the solution of Certain Non-Linear Problems in Least Squares", ~uart. Appl. Math., 2 (1944), pp. 164-68. [19] Marquardt, D. W., "An Algorithm for Least-Squares Estimation of Non-Linear Parameters", J. SIAM, 11 (1963), pp. 431-41. [20] Meyer, R. R., "Theoretical and Computational Aspects of Nonlinear Regression", P-181 9, Shell Development Company, Emeryville, California. [21] Penrose, R., "A Generalized Inverse for Matrices", Proceedings of the Cambridge Philosophical Society, 51 (1955), pp. 406-13. [22] Peters, G., and J. H. Wilkinson, "Eigenvalues of Ax = kB x with Band Symmetric A and B", Comput. J., 12 (1969), pp. 398-404. [23] Powell, M.J.D., "Rank One Methods for Unconstrained Optimization", T. P. 372, Atomic Energy Research Establishment, Harwell, England, (1969). [24] Rosen, J. B., "Gradient Projection Method for Non-linear Programming. I.

Part

Linear Constraints", J. SIAM, 8 (1960), pp. 181-217.

[25] Shanno, D. C. "Parameter Selection for Modified Newton Methods for Function Minimization", J. SIAM, Numer. Anal., Ser. B,7 (1970). [26] Stoer, J., "On the Numerical Solution of Constrained Least Squares Problems", (private communication), 1970. [27] Tewarson, R. P., "The Gaussian Elimination and Sparse Systems", Proceedings of the Symposium on Sparse Matrices and Their Applications~ T. J. Watson

39

Research Publication RA1, no. 11707, 1969. [28]

Wilkinson, J. H., "Error Analysis of Direct Methods of Matrix Inversion", J. Assoc. Comp. Mach., 8 (1961), pp. 281-330.

[29]

"Error Analysis of Transformations Based on the Use of Matrices of the Form I - 2wwH', in Error in Digital Computation, Vol. ii, L. B. Rall (ed.), John Wiley and Sons, Inc., New York, 1965, pp. 77-101.

[30]

The Algebraic Eigenvalue Problem, Clarendon Press, Oxford,

1 965. [31]

ZoutendiJk, G., Methods of Feasible Directions, Elsevier Publishing Company, Amsterdam (1960), pp. 80-90.

Topics in Stability Theory for Partial Difference Operators VIDAR THOM~E University of Gothenburg

42

PREFACE The purpose of these lectures is to present a short introduction to some aspects of the theory of difference schemes for the solution of initial value problems for linear systems of partial differential equations.

In particular, we shall discuss

various stability concepts for finite difference operators and the related question of convergence of the solution of the discrete problem to the solution of the continuous problem.

Special emphasis will be given to the strong relationship between

stability of difference schemes and correctness of initial value problems. In practice, most important applications deal with mixed initial boundary value problems for non-linear equations.

It will net be possible in this short course to

develop the theory to such a general context.

However, the results in the particular

cases we shall treat have intuitive implications for the more complicated situations. The two most important methods in stability theory for difference operators have been the Fourier method and the energy method.

The former applies in its pure form only

to equations with constant coefficients whereas the latter is more directly applicable to variable coefficients and even to non-linear situations.

Often different

methods have to be combined so that for instance Fourier methods are first used to analyse the linearized equations with coefficients fixed at some point and then the energy method, or some other method, is applied to appraise the error comm~tte~ by treating the simplified case.

We have selected in these lectures to concentrate on

Fourier techniques. These notes were developed from material used previously by the author for a similar course held in the summer of 1968 in

a University of Michigan engineering

summer conference on numerical analysis and also used for the author's survey paper ~361.

Some of the relevant literature is collected in the list of references.

A

thorough account of the theory can be obtained by combining the book by Richtmyer and Morton E28] with the above mentioned survey paper E36S. rain extensive lists of further references~

Both these sources con-

43

I.

Introduction Let ~ be the set of uniformly continuous, bounded functions of x, and let

be the set of functions v with (d/dx)Jv in ~ for J ~ k .

For v ~ ~ set

X

For any v C ~ ) a m y k, and ~ > 0 w e

~1 v is dense in

v//

<£

can f i n d v G ~Ksuch that

.~

~--.

Consider the ~_uitial-value problem

t

.>.0

(1) (2)

If v ~ C ~ this problem admits one and only one solution in

C D

(3) It is clear that the solution u depends for fixed t linearly on v; we define a linear operator Ee(t ) By

where u is defined by (3) and where v C

C~A

The solution operator Eo(t ) has the

properties

and

II ~-~b') v /t

<~ II v//

In particular, the inequality means that a small change in v only causes a small change in the solu~

on

Although for v f

u. ~%

the function u defined by (3) is not a "genuine" or

classical solution for % = O, the integral still converges if v C- C and it is natural to define a "generalized" solution operator by

~>0 •

..

L vL',,? , t - : o , .

/+'t.

•

44

The operator E(t) still has the properties =

lie (~:~~ I/

~

~< /i

0+)

(~)

v I\ ,

and is continuous in t for t ~ O. classio~

"

For this particular equation we actually get a

solutio~ for t ~ o~ even i f

~ is o~y

in C

. e have E ( t ) .

~

(_ - K=O -/~

for t > O, Consider new the initial-value problem ,

(~)

(7) For v g ~

this problem admits one and only one genuine solution, namely

Clearly

(act~mlly we have equality) and it is again natural to define a generalized solution operator, continuous in t by

This has again the properties (~), (5). for t > 0

In this case, the solution is as irregular

as it is for t = O.

Both these problems are thus "correctly posed" in ~ ; they can be uniquel~ solved for a dense subset of ~ We could instead of ~

and the solution operator is bounded.

also have considered ether Basic classes of functions.

Thus let L ~ be the set of square integrable functions with

,,

(LI

1

Consider again the initial-value problem (1),(2) and assume that u(x,t) is a classical solution and that u(x,t) tends to zero as fast as necessary whsm

I~I .-~o

the following to hold.

We then have

~t

Assume for simplicity that u is real-valued.

(8)

for

45

so that for t ~ O,

i~ ~ [., ~-~'~ II

~

II v I\

(9)

Relative to the present framework it is also possible to define

genuine and gene-

ralized solution operators; the latter is defined on the whole of L 2 and satisfies

(~-), (5). For the problem (6), (7) the calculation corresponding to (8) goes similarly~ b'£

One other way of looking at this is to introduce the Fourier transform; for integrable v, set

~o

v

~X

:

(lO)

Notice the Parseval relation, for v ~nx addition in L 2 we have ~

II '~ Ii

=

L a and

/i-~ il v i~.

For the Fourier-transform u(~ ,t) with respect to x of the solution u(x,t) we then get i~itial-value problems for the ordinary differential equations, namely,

for (l), (2) an~ a~

for

(6), (7).

.

~,

A~

~ = Av(~

~ e s e have t h e ~oZut~ons

_~L -~ "~-,~

u

~

(n)

_~

(12)

respectively, and the actual solutions can be obtained, under certain conditions, by the inverse Fourier transform. and (12),

Also by Parseval's formula we have for both (ll) __~I

I

which is again (9). For the purpose of approximate solution of the initial-value problem (1), (2),

where h,k are small positive numbers which we shall later make tend to zero in such a fashion that ~ = k / h 2 is kept constant.

Solving for u(x,t+k), we get

46

.

(z~)

This suggests that for the exact (generalized) solution to (I), (2),

or after n steps

We s h ~ l l prove t h a t t h i s i s e s ~ e n t i ~ l l y c o r r e c t f o r any v ~- ~

if, but only if ~

Thus, let us first notice that if ~ ~ ~ , then the coefficients of ~

are all non-

negative and add up to 1 so that (the norm is again the sup-norm)

or generally

iiE vll .< tlll The boundedness of the powers of ~ Assume now that v 6 ~

is referred to as stability of ~ .

We then know that the classical solution of (i), (2)

exists and if n(x,t) = E(t)v = Eo(t)v , the~ u E g ~ for t '~ 0 an~

We shall prove that~if nk = t, then

To see this let us consider

Notice now that we can write

~ ~

II ~ J

II

½

47

Therefore

"

~-,

E-

which we wanted to prove.

~-

V(~l

¢

We shall new prove that for v not necessarily in

but only in ~ , we still

have for nk = t, when k ~ To see this, let

~~

0 be arbitrary, and choose

'v"

0 .

such that

We then have --K

Therefore, choosing

~ -- ~'z(~il~l')-w~' have for h ~ ~

which concludes the proof. Consider now the case ~ Taking

~o

The middle coeffic ~ n t in ~

is t h ~

X~

we get

so that the effect of ~

is multiplication by ( i - ~ ) .

We generally get

negative.

48

Since

~

>

½ we have 1 - ~ ' ~ ~

-i and it follows that it is not possible to have

an inequality of the form

// T h i s can a l s o be i n t e r p r e t e d

T. t o mean t h a t s m a l l e r r o r s

in the initial

up to an extent where they overshadow the real solution.

d~ta are blown

This phenomenon is oalle~

instability. Instead of the simple difference scheme (13) we could study a more general type of operator, e.g.

If we wa~t this to be "consistent"

with the equation (i) we have to demand that E k

apprexi~tes E(k), or if u(x,t) is a solution, then

(\

Taylor series development gives for smooth u,

or

J Assuming these consistency relations to hold and assuming that all the aj are ~ we get as above

(15) and the convergence analysis above can be carried over to this more general case with few chs~ges.

O,

49

However, the reasons for choosing an operator of the form (14) which is not our old operator (13) would be to obtain higher accuracy in the approximation and it will turn out then that all the coefficients are in general not non-negative.

We cannot

have (15) then, but we may still have

for

some C depend~mg on To

When we work with the L2-norm rather than the maximum norm, Fourier transforms are again helpful; indeed in most of the subsequent lectures, Fourier analysis will be the foremost tool. Thus, let ~ be the Fourier transform of v defined by (lO).

We then have

,.J

3

J

or, introducing the characteristic (trigonometric) polynomial of the operator ~ ,

Jj i~_,,,f we find that the effect of E k on the Fourier transform side is multiplication by a(h~ )n.

n One easily findsthat similarly, the effect of E k is multiplication

a(h ~)n.

Using Parseval's relation,

by

one then easily finds (the norm is now the L z-

norm)

IIE

I

]11

and that this inequality is the best possible. and only if la(~ ) I ~

1 for all real~

.

It follows that we have stability if

We then actually have (15) in the L2-norm We have in this case

Consider again the special operator (13).

2A and a(~ ) takes all values in the interval[l-&/\

L 2 we have stability if and only if 1-4~

~

, i]. We therfore find that also in

-1, that is

~ ~

½.

Difference approximations to the initial value problem (6), (7) can be analysed similarly.

50

We shall put the above considerations in a more general setting and discuss an initial-value problem in a Banach space B. domain D(A) and let v ~ B.

Thus let A be a linear operator with

Consider then the problem of finding u(t) E B, t ~ O,

such that

A~*~t-)

,

E--,

o

(16)

v

(17)

More precisely, we shall say that u(t), t ~ O, is a genuine solution of (16), (17) if

(17)

holds and

(ii)

Ii u(t,<)'-"([-)

uniformly for 0 ~

--

~L~LI.) / / ~ 0 w h e n k - *

O,

t 4 T for any T > O,

Let D O be a subspace of B such that for v 6 Do, the problem (16), (17) has a unique genuine solution,

Then u(t) can be seen to depend linearly on v so that

u(t) = Eo(t)v defines a linear operator with D(Eo(t)) = D O, (16), (17) is correctly posed if D

We say that the problem

can be chosen to be dense in B so that Eo(t) is o

a bounded operator for any t >. O, and for any T > 0 there is a C with

Clearly Eo(t ) then has a uniquely defined bounded linear extension E(t) with D(E(t)) = B such that (18) still holds~ operator.

We call E(t) the (generalized) solution

Thus for v 6 B, E(t)v = u(t) is a generalized solution of (16), (17),

and this solution depends continuously on v. One can show that the solution operator has the semi-group property.

~(t+s) = ~(t)~(s),

s , t ~ 0:

in the terminology of semi-group theory one can define (16), (17) to be correctly posed if the densely defined operator A generates a strongly continuous semi-group for t >~ O.

51

We shall now study the approximation of a solution u(t) = E(t)v of a correctly posed Initial-value problem (16), (17). an approximation ~

of E(k), where ~

We will then for small k,k $ ke, consider

is a bounded linear operator with D(E k) = B

which depends continuously on k for 0

$ k ~ k e.

The thought is then that ~

is going to approximate E(nk)v = E(k)nv. We say that the eperator E k is consistent with the initlal-value problem (16), (17) if there is a set'~ of genuine solutio~ of (16), (17) such that (i)

%

~

~ ~(0~

~ ~

~ ~

~

is dense in B.

for any T > O. If the operator ~

is consistent with (16), (17), we say that it is convergent

(in B) if for any v e B and any t ~ with kj ~-> O, njkj -->t for j ~ ,

O, and any pair of sequences

We say that the operator ~

~ 9

we have

g vI/ o

II

~i

whenj-e

.

is stable (in B) if for any T ~ 0 there is a con-

stant C such that

It turns out that consistency alone does not guarantee convergence; we have the following theorem which is referred to as Theorem

Lax's equivalence theorem [22].

Assume that (16), (17) is correctly posed and that ~

approximation operator. The proof of

is a consistent

Then stability is necessary and sufficient for convergence.

the sufficiency of stability for convergence is similar to the

proof in the particular case treated above; the proof of the necessity depends on the Banach-Steinhaus 2.

theorem.

Ini~al-value problems in L 2 with constant coefficients We begin with some notation.

L ~ = L2(R d) with the norm

We shall work here with the Banach space

52

We define for a multi-index

~< = C~,~ ' j~%~ with ~<~ non-negative integers

setting D = (. ~ ~'~, ~ ..... ~ ~"~'~d we then ~ v e the f o ~ o ~ g

derivatives of order

We denote ~y R d and ~y

~

C~

[~I = ~

no~tion for ~eneral

~J ~ n~ely

the set of infinitely differentiahle complex-valued functions in the su~set of functions with compact support.

We also introduce the

~o

set ~ of u ~ ~

Clearly

Cj

such that for any multi-indices

C J

C

C

~<9 ~

and it is well known that

Co

and

are denae in L*.

For u integrable on R d we define the Fourier• transform

We reoall that if U e J

then ~u

6 S .

Further, for u g ~

inversion formula

~< ~

we have Fourier's

~ ~2

and Parseval 's relation

and as a consequence of the latter, the set

transforms in

Co

Cc~

of functions in

J

with Fourier

is dense in L'

In the sequel we shall consider N-vector valued functions u(x)=(u,(x),...,uN(x)). It is clearly natural %o define u(x) G holds for each component uj, J = I,...,N. to N-vectors e.g.

L', f9 Co~

Single bars will denote norms with respect

~

~ ~

I

and for NxN matrices,

v.o

etc. by demanding that this

Ivi

53

and double bars will indicate norms with respect to L a , so that for the N-vector u(x) 6

L',

~_

\ ~_

For later use we need the following Lemma I

Let ~

be a dense subset of L a and let a(~ ) be a continuous NxN matrix.

Then

il~ V II

veglv

~i v/\

~

Let u(x,t) ~e an N-vector-function defined for x ~ R d and t ~

O.

Consider the

initial-value problem

-~

_

~(,~

=

~.~)~

"5_

~ >~o

where P~ are constant N~N matrices and where we can consider Pu to be defined for u

~ ~

Let

~

We have : Theorem I

The initial-value ~ o b l e m (I), (2) is correctly posed in L" if and onlyif,

for any T ~. O, there is a C such that

Proof

Assume that (3) holds.

Let v ~

a

o~-~ ~ T

~

IL

~

and consider

(~) By differentiation under the integral sign we find that u(x,t) satisfies (i), and so is a solution te (i), (2).

Since for t >I O, u(x,t) ~

in the sense of Lecture land is also unique.

~ ) it is a genuine solution

Thus Eo(t)v = u(x,t) with D

= O

Fourier's inversion formula smd Parseval's theorem

2 . By

54

Since

S

is dense in L ~ it follows that the initial-value problem is correctly

posed°

Jk

We now want to prove the necessity of (3) for correctness. define u(x,t) by (4).

Let now v ~ ~ o

an~

We find at once that u(x,t) satisfies the initial-value

problem (I), (2) and so u(x,t) = E(t)v.

Again, by Fourier's inversion formula and

Parseval' s theorem

[I v// so that by Lemma I,

which proves the necessity of (3) since Co Ex. i

is dense in L z.

Consider the symmetric hyperbolic system

(5) Then the in~ial-value problem for (5) is correctly posed in L 2 for

8

since this is a unitary matrix. Before proceeding to the next example we state a lemma.

For an arbitrary NxN matrix

A with eigenvalues ~ d' j = I,...,N, we introduce

We then have Lemma 2

If A is an NxN matrix we have for t ~ 0

.j --0 Proof Ex~2

See [9]. Consider the system (I) and consider also the principal part P of P which

corresponds to the polynomi~l

55

We say that the system (i) is parabolic in Petrovskii's sense if there is a

~, >

such that .) By homogeneity this is equivalent to the existence of a

~>

0 and a C such that

We then have that if (I) is parabolic in Petrovskii's sense, the corresponding initial-value problem is correctly posed in L 2. 0

For by Lemma 2 we have for

~ t ~ T,

which is clearly bounded.

In particular, the heat equation

clearly falls into this category. Solutions of parabolic systems are smooth for t ~ Theorem 2

0; we have

Assume that (1) is parabolic in Petrovsk~'s sense.

D E(t)v ~ L 2 for any ~

and for ar~y T >

Then for t ~

O,

0 and any°Q there is a C such that

6
Via the Fourier transform and Parseval's relation this reduces to

But this follows at once by (6). Ex.

Consider the Schrodinger equation (N=I) 4

The initial-value problem for this equation is also correctly posed in L 2.

Ex, 4

For

The Cauchy-Riemann equations can be written (d=l)

For these we shall prove the negative result, that the corresponding initial-value problem is not correctly posed in L z.

For here

0

56

-i o

-~

and a simple calculation yields

which is not bounded for any t > Ex. 5

0 whe~

V~

cO .

Although our theory only deals with systems which are first-order systems with

respect to t, it is actually possible to consider also hi~her-order systems by reducing them to first-order systems. lar case.

We shall only exemplify this in one particu-

Consider the initial-value problem (d=l)

.~"~

_-

~-~"~

~ "k >~ ~

(7) ~-~

~

Introducing

(a)

we have for u the initial-value problem

(9)

~ere

ul~o5 = vL~,5. co~

~

= / _~

so that we have that the initial-value problem (9) obtained by the transformation (8) from (7) is correctly posed in L i. In order that an initial-value problem of the type (I), (2) be correctly posed in L 2, it is necessary that it be correctly posed in the sense of Petrovskii, more precisely: Theorem 3

If (i), (2) is correctly posed in L 2 then there is a constant C such that

57

Proof

Follows at once by

We shall see at once by the following example that (I0) is not sufficiemt for correctness in Ls• Ex. 6

Take the initial-value problem corresponding to (d~l)

0

_.~

= -f

T_ -v -l:

We get then

However,a simplecalculationyields

~ 1

which is easily seen to he unbounded for 0 $ t ~

I (take t ~

= i).

Necessary and sufficient conditions for correctness have been given by Kreiss [19].

The main contents in Kreiss' result are concentrated in the following l e n a .

Here for a NxN matrix A we denote by Re A the matrix

Also recall that for hermitian matrices A and B, A ~ B means

for all N-vectors v.

We denote the resolvent of A by R(A;z);

It will be implicitly assume~, when we write down R(A;z ), that z is not an eigenvalue of A. Lemma ~

Let ~

be a f a ~ l y of NxN matrices.

Then the following four conditions are

equivalent

j

?---

(iii) For A A

~ ~

~ A ( A ) <~ 0 and there are two constants C, and C 2 and for each

a matrix S = S(A) such that

58

(, i s i ~ / s - ' ~ )

<- c~

and such that

Sf/ S -~

--

%~. 0

~X.

is a triangular matrix with

\

(iv) There is a constant C > 0 such that for each A ~ ~ there is a hermitian matrix H = H(A) with

C-~

~

I~ ~

Q ~

and

~

~

~ ~

<-0

See [l~].

Proof

To be able to apply this lemma to our problem we need: Lemma 4 g

Assume that (1), (2) is correctly posed in L 2.

xt

and C such that

Pr,oo,,f,

Let C and ~" be such that

Then there exist constants

~

"fO~" 0 5 ~

For arbitrary t > 0 let It] be its integral part.

We have

~ ~< ~

~ ~e

~a ~k~

~t~

~t

which proves the lemma. Combining L e ~ a s 3 and 4 we have at once: Theorem 4

If (1), (2) is correctly posed in L z then there is a constant ~ such

that

satisfies the conditions of Lemma 3. such t h a t ~

On the other hand if there is a constant

satisfies at least one of the conditions of Lemma 3, then (1), (2), is

correctly posed in L z.

59 One commonly used criterion is: Theorem 5

Let P ( ~ ) be a normal matrix.

Then (i), (2) is correctly posed if and

only if (IO) holds. Proof

By Theorem 3 we only have to prove the sufficiency.

Since P( ~ ) is normal we

can find a unitary U(~ ) such that

is diagonal.

Hence

which proves the result. For later use we state: Theorem 6

If (1), (2) is correctly posed in L 2 then (lO) holds and there are posi-

tive constant C I and C a and for each

H(~ )

~ ~

R d a positive definite hermitian matrix

such ~ t

-I and.

(13) l:'roof By Theorem 4 there is a constant 7 such that the family S condition (iv) of Lemma 3 with C = C I.

Thus for each

~g

in (11) satisfies

R d there is a positive

definite H(~ ) such that

But by (12) this implies (13). 3.

Difference appr0ximations in L i to initlal-value problems with constant coefficients Consider again the initial-value problem

ae - ? ( ~ ) ~ : "oe

Z

P,~'~u. , ~ , o

(l)

bzl ._ M

u(',,o) ~ v~×)

(2)

60

For the approximate solution of (i), (2) we consider explicit difference operators of the form

where h is a small positive parameter, ~ = (~, .... ,~d) with ~j integer, e~(h) are NxN matrices which are polynomials in h, and the sunwaation is over a finite set of ~. We introduce the symbol of the operator ~ ,

which is periodic with period 2,U/h in ] transform of ~ v n

is

/k

and notice that for v 6 A

Assume that the initial-value problem (i), ( 2 ) to choose ~

~ ~ the Fourier

is correctly posed.

Pie then want

so that it approximates the solution operator E(k) when k is a positive

parameter tied to h by the relation ~/h ~

=

~

= constant;

we actually want to approximate u(x,nk) = E(nk)v = E(k)nv by ~ v . shall emphasise the dependence on k rather than h and write ~

In the future we

as in Lecture i.

To accomplish this, we shall assume that E k satisfies the condition in the following definition.

(1) ~

We say that ~

is consistent with (i) if for any solution of

C ~o

i f o ( k ) can be ~ p ~ c , ~

by k(~(h~), w, ~ y

that ~

ie a c o ~ a t e of o r d e r f

.

C1~arly

any consistent scheme is accurate of order at least i. We can express consistency and accuracy in terms of the symbol (cf. [35]): Lemma i

The operator ~

is consistent with (I) if and only if

The operator ~ is accurate of order ~ i f and only i f

~

v~

61

The proof of (3), say, consists in proving like in the special case in Lecture 1 that consistency is equivalent to a number of algebraic conditions for the coefficients, which turn out to be equivalent to the analytic functions exp(kP(h -I ~ )) and Ek(h-' ~ ) having the same coefficients for h j ~

up to a certain order.

Using LemmA 1 it is easy to deduce that if ~

is consistent with (1) in the

present sense then we also have consistency in the sense of Lecture 1.

For the set

@ of genuine solutions in the previous definition we can for instance take the ones corresponding to v £ ~

.

From Lax's equivalence theorem it is clear that we want

to discuss the stability of operators ~ Theorem 1

khe operator ~

of the form described.

is stable if and only if for any T >

We notice that

F-,k(~ )n is the

O,

O~

c

,Proof

We have

~ymbol of

~k"

It follows in the same way as

in Lecture 2 that

which praves the theorem. We now turn to the algebraic characterization of stability. the necessity of the yon Neumann condition.

We first prove

For any NxN matrix A we denote byp (A)

its spectral radius, the maximum of the moduli of the eigenvalues of A. Theorem 2

If ~

is stable in L 2, there exists a constant

~

such that

(4) Proof

We have for nk ~

l,

and so

It is easy to prove by counter-examples that (4) is not sufficient for stabili~ Necessary and sufficient conditions for stability have been given by Kreiss [18] and Buchanan [51 ; we quote here Kreiss' result. concentrated in the following Lemma.

The main content in Kreiss' theorem is

Here we have introduced

notation: For H hermitian and positive definite, we introduce

the following

62

IAu Ii~ Recall again that for hermitlan matrices, A ~ B means (Au,u) ~ (Bu,u). Lemma 2

Let ~

be a family of NxN matrices.

Then the following four conditions are

equivalent.

(i) sup ~ ~

(iii) For A ~ ~ ,

~ ~

~ ~ ~o

~-~ ~--~,

~ (A) ~

l, and there are two constants C, and Cm and for each

A @ Jl- a matrix S = S(A) such that

and such that

Sg

=

is a triangular matrix wlth

(iv) There is a constant C ~ 0 such that for each A ~ ~

there is a hermitian

matrix H = H(A) with

c-'T_

~

~

~

CI

and

Proof

see [28].

To be able to apply this lemma to our problem we need the following analogue of Lemma 2.4. Lemma 3 for

Assume that ~

~,~ ( ~

~ ~

is stable in L 2. ~t~)

Then there exists a constant

such that

one has ~O

63

An alternative way of expressing this result is that for some any n we have

Y,k

~ i, and

~

Combining Lemmas 2 and 3 we have at once : Theorem ~

If ~he operator ~

is stable in L i, then there is a ~ such that

satisfies the conditions of Lemma 2.

On the other hand, if there is a constant

such that k F satisfies at least one of the conditions of Lemma 2, then Ek is stable in L ~. One commonly used criterion is : ~eo~m

~

Let ~

be s u ~

t~t ~(~)

is a n o ~ l

matrix.

~ e n ~cn ~ e ~ ' s

con-

dition is necessary and sufficient for stability. Proof

By Theorem 2 we only have to prove the sufficiency.

there is for each k ~ I and

is diagonal.

~ ~

Since ~ ( ~ ) is nor,~al

R d a unitary matrix U k ( ~ ) such that

Her~e

('

I which proves the result.

To see the relation with Lemmas 2 and 3, we could also have

formulated this as fellows.

We have with the same ~ as in (4) for Fk(~ ) = e

that

_ ~

which is diagonal with eigenvalues of modulus ~ i.

Ek(~)

~\

Thus, afortiori it is triangu-

lar, and the estimates in condition (iii) of Lemma 2 hold. As for existence of stable operators, we have (cf. [17]): Theorem ~

There exist L2-stable operators consistent with (i), (2) if and only if

(I), (2) is correctly posed in L z. Proof

We first prove that the correctness is necessary.

the stability that

l~p

~i: ~ f ~ /

"= ~m / ~.~]a~/

It follows by Lemma i, ar~1

6't

which implies correctness. On the other hand if (I), (2) is correctly posed one can construct a consistent difference operator, er which is equivalent, its symbol, by setting

Using Kreiss' stability theorems one can prove that this ~ ~=

is stable for small

K/h~ ~ The part of this operator corresponding to the second term in (5) is

referred to as an artificial viscosity. We shall consider some examples. Consider the initial-value problem for a symmetric hyperbolic system

(6)

We know from Lecture 2 that this problem is correctly posed in L ~.

Consider as

before a difference operator

where for simplicity we assume

e~

independent of h.

We have the following result

by Friedrichs [8]. Theorem 6

If e~ are hermitian, positive semi-definite and

~

= - ' I then

and thus Ek is stable. Proof

We have the generalized Cauchy-Sohwartz inequality

where (u,v) =

~

uj ~j.

Therefore

~-

65

and hence with w =

Ek( ~ )v,

~.

which proves the lemma. As an application, take

We have :,-4 t

so that

,

3

is consistent with (6) and accurate of order i.

It is clear that if

satisfies

o

< ~

.< ~'~ Ca/~l/-~

,

the coefficients are positive semi-definite and so the operator ~ The operator ~

is stable.

can be considered as obtained from replacing (6) by

Consider for a moment the perhaps more natural equation

which gives the consistent operatorj

E~vC~} --- vl~) ÷ ~ ~ aa~ L ~ l ~ * ~ - ~ ( ' ~ - ~ ' ~ with

J

~i

66

We shall prove that this operator is not stable in L ~ if any of the Aj is non-zero. Assume e.g. A, ~ 0 and set ~ j = 0 for J ~ I, ~ Ih =~/2.

With this choice,

which has the eigenvalues

where t h e r e a l numbers # ~ a r e the e i g e n v a l u e s o f A~.

Thus t h e von Neum~nn c o n d i -

t i o n is not satisfied and the operator is unstable for any ~ It can be shown that in general the operator ~ order exactly 1.

.

defined in (8) is accurate of

We shall now look at an operator which is accurate of order 2 in

the case of one space dimension (d=l). =

~

thus have the ~ s t e m

(9)

--

Consider the difference operator =

with

This operator is often referred to as the Lax-Wendroff operator.

We have

and so, E k is consistent with (9), and in general accurate of order 2.

We shall

prove : Theorem

Let~j,

j = 1,...,N, be the eigenvalues of A.

Then the operator Ek in

(lO) is stable in L 2 if and only if

(ll) Pro0f

It is easy to see that the eigenvalues of ~ ( h -I ~ ) are

and we obtain after a simple calculation

if and only if (II) holds.

Since E k is clearly normal, this proves the theorem.

For a NxN matrix A consider the numerical range

67

We have : Theorem 8

then ~

Proof

If ~

is a family of NxN matrices such that

is a stable family, that is there is a constant C such that

We shall prove that condition (ii) in Kreiss' theorem is satisfied.

Clearly

since

we have ~(A) ~ 1 so that R(A;z) exists for

~z~ > I .

Therefore, if w is arbitrary,

and v = R(A;z)w we have

or

which proves the result. Remark

One can actually prove that

IAnl ~ 2, A ~

This result can be used to prove the stability of certain generalizations of the Lax-Wendroff operator to two dimensions (see [2~]). Consider again the symmetric hyperbolic system (6) and a difference operator of the form (7), consistent with (6). Then A ( ~ ) = ~ ( h - 1 ~ ) is independent of h. We say with Kreiss that Ek is dissipative or order O (~

even) if there is a

~~

0

/X)

such that

We shall prove Theorem 9

Under the above assumptions, if Ek is acct~ate of o r d e r ~ -I an~ dissi-

pative of order ~

it is stable in L 2 .

68

Proof

By the definition of accuracy, we have

o,.s

:~ -? O

Let U = U( "~ ) be a unitary matrix which triangulates A( } ) so that

Since B(~ ) is upper triangular it follows that the below-diagonal elements in e x p ( ~ U P ( ~ )U~) are O ( ~ ) .

Since this matrix is unitary, the same can easily be

proved to hold for its above-diagonal terms, and thus the same holds for the abovediagonal terms in B( ~ ) so that .

.

.

.

\o and the s t a b i l i t y

follow~

by c o n d i t i o n

(iii)

in ~eiss t ~eorem, v

Consider now the initial-value ~roblem for a Petrovskii parabolic system .

_

~

~0

so that

We know from Lecture 2 that this problem is correctly posed in L 2.

Consider a

69

aifference operator

We say, foZlo~r.l.ng John [15] and ~ i d l u n d [38] t h a t E i s a p a r a b o l i c d i f f e r e ~ e k operator if there are constants ~ and C, S ~ 0

such that

Notice the close analogy with the concept of a dissipative operator. Theorem 10

Let E

be consistent with (12) and parabolic. Then it is stable in L ~. k We shall base a proof on the following lemma, which we shall also need later for other purposes. Lemma 4

There exists a constant CN depending only on N such that for any NxN

matrix A with spectral radius ~

we have for n ~ N,

!

IP, l Proof

<. C.

+(

See [35].

Proof of Theorem 10

By consistency we have

We therefore have for n ~

N, nk ~ T,

~_Nfj

s which proves the stability. Consider forward difference quotients

and. for a general ~,

We then easily have the following discrete analogue of Theorem 2.2. Theorem ii such that

Assume that E k is parabolic.

Then for any o( and T > O there is a C -- ~

70

,Proof

By Fourier transformation this reduces to proving

._~l

and the result therefcre easily follows by (13).

We know by Lax's equivalence theorem that the stability of the parabolic difference operators considered above implies convergence.

We shall now see

that the difference quotients also converge to the corresponding derivatives, which we know to exist for t >

0 since the systems are parabolic.

Theorem 12

Assume that (12) is parabolic and that ~

parabolic.

Then for any t > O, any o~ , and any v 6 •

~

li b-~

i[

~

,, v

_

~

f~ (,) v ii --> o

~

is consistent with (12) an~ L 2 we have for nk = t, ~, ---.

o,

(~,)

Proo____~f By Theorems 2,2 and ii one finds that it is sufficient to prove (14) for v A~ in the dense subset C~ . But then, by Parseval's relation, • "~ t ~

-

%~

The result therefore follows by the following lemma which is a simple consequence of Lemma i. Lemma ~

If ~

is consistent with (12) then

l e~i_i

~

~'

uniformly for '~ in a compact set.

4.

Estimates in the maximum-norm Consider the initlal-value prohl~n for a symmetric hyperbolic system with

constant coefficients

'~

~i

;)=i

As we recall from Lecture 2, this problem is correctly posed in L ~. is not necessarily the case in other natural Banach spaces.

(1)

However, thls

71

of bounded, uniformly

In this lecture we shall consider the Banach space C continuous functions in R d with norm

In ~

one has the somewhat surprising result b ~ Brenner [2],

Theorem I

The

i

al-

(1), (2) is oo eot

alue probl

posed in

only

if and

if

(3) Let us

commentthat

it is well known that the condition (3) is equivalent to

the simultaneous diagonalizability of the Aj, that is (3) is satisfied if and only if there exists a unitary matrix U such that

is a real diagonal matrix for all J = l,...,d.

This means that if we introduce

,V

u = Uu as a new variable in (i) we can write (i) in the form d

~t--~

=

. -b

~, ~ .~ --~^

(4)

But this is a system of N uncoupled first order differential equations.

Thus,

only

in the case that (i) can be transformed into a system of uncoupled equations is (I), (2) correctly posed in

~.

It can be shown that in the case of non-correctness, that is when (3) is not satisfied, there are no consistent difference operators which are stable in the max,m I J ~ - n o X~le

We shall now consider a very special case of a system of the form (4), namely one single equation and ~=I, ~

~

~

~

real

(5)

We then want to discuss the stability in the maximum-norm of consistent explicit operators of the form

where aj are constants and only a finite number of terms cccuro characteristic polynomial

Ck ~

"-- ~ - ~

~)

=

~

C~ ~

Introducing the

72

we have stability in L" :if and only if la( } )1 ~ 1 f u r r e a l We have Lemma I

Proof

The norm of the operator ~

in

d

is

We clearly have

so that

On the other hand, let v(x) ~ ~ be a function withlv(x)l~ 1 such that

-

~

i.~

o,- t:o

Then

J so that 3

This proves the lemma. We have earlier observed that ~ characteristic polynomial a( ~ ) n

has the symbol ~ ( ~ )n, that is the

If

,I

we therefore have

~i

3

(63 It follows from Lemma i above that

3

ang the discussion of the stability will depend on estimates for the anj. We now state the main result for this problem.

73

Theorem 2

The operator ~

is stable in the maximum-norm if and only if one of the

following two condi~ens is satisfied .)

(~) in

I~ [ ~ i

I~1 r . ~

where

~

< i

e~ept

where I~t~l~-~ ,

is r e a l , Re ~ % >

for at ,ost a finite

n ~ b s r of points'~

For q = l , . . , , Q there are constants

~,

~:Z,...,~

~)~%

~35

O, and'O~ is an even natural number, such that

We shall sketch a proof of the theorem in the case that Ek satisfies the additional assumption "

(8)

We have

Lemma 2

Assume that a ( ~ ) is a trigonometric p o l y n o m i a l such that (8) is s a t i s f i e a

and such that = where

~

~

is real, Re ~ >

O, and ~J

(9)

is even. Then, if anj is defined by (6),

there is a positive constant C independent of n and J such that

Proof

By (8) and (9) there is a

We therefore get~

I ~

~Y~

~ > 0 such that

i ~_,~ I0~(~II

which proves the first half of the lemma.

C~"~ ~

-, ~ ( - ~ " ~ ~'01C~"~

To prove the second half, we define

74

"

~

o,~ t ~

~1

After two integrations ~y parts, using the periodicity of a(V ) we get

Aooor~_'i.nt!l to (9) we have

and it follows for

~\

%

I ~ -%

~l

We thus get

and

~,

~C- ~ ~) ~ ~ + , ,

since

-c , - ~ ~ ~~-

._,~

t-~ f ~

the result follows. We then have CorollaI~

Assume that ~

has the characteristic polynomial a( ~ ) which satisfies

the assumptiens ef Lemma 2.

Then ~

is stable in

~,

1

We have

II~" v,

i~-~f,I-<'~

~

C U '/'>

~-,

,,,.~l~,l

.

i~-<~ ~,t> n y'~

(.~_~<~/

)'

~C~ <~ C(~ -~ ~'~

~

(,,+

whioh proves the corollary. Consider now the n e c e s s i t y

(~) ~&ce.

~

(9).

As~

~at

o f t h e c o n d i t i o n ( 7 ) i which under t h e assumption

(9) i s not ~ t t s f i e d .

We t h ~

must have

75

- :

~

(,4

-

real, q('~ ) r e a l polynomial, q(O) $ O, 1 < ..~ <. ",3 ©

even, Re)~ > O .

By Parseval's relation for periodic functions we have

O

and using (iO), it is easy to deduce from this hhat

.a Using a lemma ~y van der Corput it is also possible to prove

j ~d

so

~

which tends to infinity with n since

t

< ~

~' -

~'~

'

As an application, consider for the solution of the equation (5) the LaxWendroff operator, which in this case reduces to E

We have

and

and so ~

is stable in L 2 if and only i f

.ehave

i°'C

i
i~i~i

On the other hand, i f O
for O <

It follows from Theorem 2 that ~

is unstable in ~ .

By the above proof we have

-K Serdjukova [30],

[31] and Hedstrom [11], [12] have, by using more refined

techniques of estimating the a . above, been able to give more precise estimates of na the growth of II ~ II for the case when ~ is stable in L z but unstable in Co In

76

the particular case of the Lax-Wendroff operator, the exact result is

Ci

-~

..,

more generally, when a(~ ) has the form (i0) one has

c~ ~,~(,-~'/-.,5 .< tl ~11 -< c~_,',~'~('-~/'~

(n)

The instability present here is of course quite weak. The proof of the sufficiency part of Theorem 2 is due to John [15] an~ Stran~ [32].

The proof of the necessity part can he found in [33].

Theorem 2 has also

been generalize~ to variable coefficients in Th°mee [3~] an~ to L p, I ~ p ~ oo, p ~ 2, in Brenner and Thome~e [3].

The analogue of (ii) then reads

c, r,~-~'~°-~'/"~ <./i ~li,_~ .-< C~ r,'~-y~i('- H"/ Consider now the initial-value problem for a system with constant coefficients J

which is parabolic in Petrovskli's sense,

-~

that is

i~i~ H

(12)

(l~) where

We recall from Lecture 2 that this problem is correctly posed in L a and also that derivatives can be estimated in L ~. ~e shall now see that (12), (13) is actually correctly posed in ~

and that

again also the derivatives can be estimated in the maximum-norm. Theorem ~

Assume that (12) is parabolic in Petrovskii's sense.

Then for t >

the solution operator has the form

where

..,

,,ere is .

>O..c,

th.,

(l?)

0

77

The problem is correctly posed i n k .

and for ar~v T )~ 0 and

o~ there is a C such

that

(18) Proo,f

To give a hint of the proof, we recall that if v g

~

then after Fourier

transformation the initial-value problem becomes a problem in ordinary differential equations with solution /k

u (. L*") That this is the case

which is the Fourier transform of (15) if exp(tP(~ )) follows from

(17) which

still remains to prove.

On the other hand, once this is

proved, the estimate (18) is a trivial consequence. need the following extension of (14) to Lem~ ~

complex numbers:

If (12) is parabolic in Petrovskii's sense there are positive constants

an~[ C such that for any

~ ~- ~ ~ ~

.,'x ( Proof

To hint at the proof of (17) we

"1~ ~ ~j~ ~ ~,~d

.<

+ c/

1" +c.

See [7]. To complete the proof of (17) we first notice that by L e n a 2.2 we have for

0
T,

Using this estimate one can see that the domain of integration in (16) may be moved so that for any

/~ ~ ~

or after differentiation,

~hus, by(19) wegetforO

< t S T,w~th

¢ =~.i~

~,~ N-t + W

~

78

where the constants are independent of ."~

. Now choose

Wethen have _~

I

~1~-, (!~_~ m> + c . £ l q i ~

~-(~c t

-

t

J

and the estimate (17) now easily follows. Consider now explicit difference operators

consistent with (12).

We recall from Lecture 3 that ~

-

~C~.

is called parabolic if

~

.\~i\<-~

~>o

that such an operator is stable in L~ and also that difference quotients may be estimated in L 2. We shall now see that a corresponding result holds also in the maximum-norm. Theorem 4

Assume that E k is parabolic, and let

Q where ~ ( ~ )

is the symbol of ~

and Q = I~ ~ I

~ ~I.

The end(h) satisfy the /

following estimates (where difference quotients are taken with respect to ~ ),

The o p e r a t o r Ek i s s t a b l e i n

~

and f o r any ~( and T ~

II "a~ I~. v II ~< C (,nK~ ~ Proof

For details see [39].

II~/l/.)

0 t h e r e ks a O such t h a t

0 4 ,~. ~ -1-

(2l)

The fact that (21) follows from (20) is due to the

can be considerea as a Riemann sum for the integral

79

and therefore this sum is bounded independently of n. To prove (20) one goes through essentially the same steps as in the proof of (17) in Theorem 3, utilizing Le~ma 3.4 instead of L e n a 2.2 and the following lemma instead of L e n a 3. Lemma 4

Assume that ~

is parabolic, and let ~o be given.

constants ~ and C such that for

4 = "~'~"~ /~

~- A,} E:

Then there are positive L~

-

Again, the estimates for the difference quotients can be used to prove their convergence to the corresponding derivatives.

We state the following result without

proof. Theorem 5

Assume that (12) is parabolic and that ~

parabolic.

Then for a~yt ~ 0 ,

il

~

we have for v~ = t,

-

The choice of ~ rary.

any ~ , and any v ~

is ~nsistent with (12) and

as the operator approximating D ~ is again rather arbit-

One can indeed show that the same result as in Theorems ~ and 5 hold for any

difference operators consistent with D ~ Theorems 3, 4 and 5 generalize to variable coefficients.

5.

On the rate of convergence of difference schemes In this lecture we shall work in a slightly more general setting than b e f ~ e .

Let L p = LP(Rd), 1 ~

p <

co, denote the set of measurable functions (or vector-

functions) v such that

and consider the family of Banach spaces

i/w = { Consider also the Sobolev spaces ~

)

of distributions v such that D v 6 W for I~ % ~ o P P This is also a Banach space with norm

80

For 1 ~

p ~

$

(~0

~

this can be thought of as the closure with respect to the norm (1)

Let finally w ~ be the set of v which are in w ~ for all m. This P P means that D v is continuous for any ~ and D v ~ W . The set ~#o is dense in W . P P P of

or

"

Consider again an initial-value problem

~-'~

-

~

~,

~

~

(2) (3)

where as before P~(x) have all derivatives in

(~,

In the sequel we shall demand not only that the initlal-value problem be correctly posed in W , but that it satisfies the stronger requirement of the followP ing definition. We say that the initial-value problem is strongly correctly posed in W

if for any positive m and T, v 6 W m implies P P such that for all v & W m, P

In particular,

E ( t ) v ~ W m and there is a constant C P

this definition implies that

E(t) ]l~°C W °° . P P It can be proved that if P(x,D) has constant coefficients,

or if it is of first

order, then strong correctness in W Further,

is an automatic consequence of correctness. P systems which are parabolic in Petrovskii's sense are strongly correctly

posed in W

for any p with I %

p ~oo.

P Consider difference operators of the same form as before, namely

We have previously defined consistency of ~

with

(2)

to mean that for any suffi-

ciently smooth solution u(x,t) of (2),

J

more precisely, E k is said to be accurate

of

order #

if for such u,

k --> 0 V~en

(2), (3) is

strongly correctly posed in

Wp,

(~)

it is sufficient to assume this

local condition to obtain the following global estimate:

81

Theorem i

Assume that the initial-value problem (2), (3) is strongly correctly

posed in W

and that ~

is consistent with (2) and accurate of order ~

.

Then

P

there exists a constant C such that for any v ~ W m + ~ P

Ii

<.c.

See [ 2 7 ] .

Proof

ii vll

*

The proof consists in expanding E(k)v = u(x,k) and ~ v

in Taylor

series around the point (x,O), using (4), and estimating the remainder terms in integral form.

In doing so it is sufficient to consider v in the dense subset W OO of P

W. P We now easily obtain the following estimate for the rate of convergence: Theorem 2

Assume that the initial-value problem (2), (3) is strongly correctly

posed in W

and that ~ is stable in W , consist~at with (2) and accurate of P P Then for any T > 0 there is a constant C such that for v ~ ~ + ~ nk ~ T, P

Proof

Vie have

n~

and so by the stability of ~ ,

~.

orderf

~-~ -S

,~--0 Theorem i, and the strong correctness,

o

(5)

which proves the theorem. Thus, the situation is that for initial-values in W

we have (by Lax's equivaP lence theorem) convergence without any added information on its rate, and if the initial-values are known to be in ~ + ~ we can conclude that the rate of convergence P is O(h ~) when h ~ 0 . It is natural to ask what one can say if the initial-values belong to a space "intermediate" to W

and ~ + / ~ . To answer this question we shall P P introduce some spaces of functions which are interpolation spaces between W and Wm P P

in the sense of the theory of interpolation of Banach spaces (cf. [27] and ref~enems~ Let s be a positive real number and write s = 8+c~ , S integer, 0 < ~ Set T ~ v ( x ) = v(x+ T).

~

1.

We then denote by B s the space of v ~ W such that the followP P

ing norm is finite, namely

=

iI. -,

i l ll T ~i= S

t:.$o

- b ~"

Thus, B s is defined by a Lipsehitz type condition for the derivatives of order S; P

°

82

these spaces are sometimes called Lipschitz spaces.

For the Heavysi&e function (4=1)

L we have for 1 ~ p < co

and it follows th~ if ~

C~

th~n ¢ ~

~

~

One can prove that B sl C B s2 if s I ~ s2, and that for integer s and ~ > 0 P P arbitrary, B s+~ ~ W s C B s . The main property of these spaces that we will need is P P P then the following interpolation property: Assume that 1 ~ p ~ ~ , m is a natural number, and

m is

a real number with 0 < s ~ m.

that any bounded linear operator A in W

Then there is a constant C such

with P

we have

S_

~< C C , '-~ c ~ ~I~II Theorem 2 and (7) with A = ~-E(nk) Theorem ~

•

(7)

prove immediately the following result:

Assume that the initlal-value problem (2), (3) is strongly correctly

posed in W

and that ~

is stable in W , consistent with (2), and accurate of order p

P •

~ = ~

Then for O ~

s < M+ ~

and T > 0

there is a constant C such that for any V~Bp,

nk~T,

- ~ , . (~) II k' E "~. -~(."~'Y~ v//',,'at, ,< C k~ ' ; - / / v l / ~ '~ ~ X-r,,..t. Notice that ~ =

t,. l", ',;."t"

grows w i t h ~

and lim ~

= 1.

estimate (8) becomes increasingly better for fixed s when ~

This means that the grows.

In other weras,

if for a given strongly correctly posed initial-value problem one can construct stable difference schemes of arbitrarily high order of accuracy, then given a ~ one can obtain rates of convergence arbitrarily close to O(h s) when h * 0 ,

for all

initial-values in B p" s As an application, consider an L 2 stable operator ~ for the hyperbolic equation

~-'-~

=

"-by,

-~

s~ 0

with order of accuracy

85

and l e t v =

~,~where ~

eo

and'~

is the Heavyside function (~).

By above

we have in this case

For dissipative operators ~ ,

stronger results have been obtained in Apelkrans [I],

and Brenner and Thcme~e [4], where also the spreading of discontinuities is discussed. It is natural to ask if for a parabolic system, the smoothing property of the solution operator can be used to reduce the regularity demands on the initial data in Theorems 2 and 3.

This is indeed the case.

Before we state the result we give

the following result, which follows easily from properties of fundamental solutions. v

Theorem A

Assume that (I) is parabolic in Petrovskii's sense.

1 $ p ~ co, any m >

@ and T >

Then for any p with

0 there is a constant C such that

0 -~-k ~ -'f" We can now state and prove the result about the rate of convergence in the parabolic case. Theorem ~ in % ,

Assume that (2) is parabolic in Petrovskii's sense and that E k is stable

consistent with (2) and accurate of order ~

.

Then for any s >

O, T ~

there is a constant C such that for v E B p, s nk ~ T,

t/ Proo___~f For details, see [27]. Here we shall only sketch the proof for the case v C~ B s where ~ ~ Q ~ * ~ t h e other cases can be treated similarly. P We shall use (5). For J = 0 we have by the stability and Theorem l,

,,_,

~ ~Ck ~'~ It~1/ ~*~

L and sc by (7), since s > y s

For J >

0 we have by Theorems i and 4,

O,

84

.<

and hence by (7),

where

Since therefore _~

we get by adding over j S

which proves (9) im the case considered. Investigations by Hedstrom [13], [14], Lefstrom [25], and Widlund [40] have 1 shown that in special cases the factor log ~ can be removed from the middle inequality in (9). Theorem 6

In particular the following result M s Assume that (2) and ~

respectively, and that ~

been proved by Widlund [40].

are parabolic in the sense of Petrovskii an~ John,

is accurate of order ~ .

Then for any T ~

O there is a

constant C such that for V ~ B p ~, nk~T,

The proof of this fact is considerably more complicated than the above and depends on estimates for the discrete fundamental solution.

Using these estimates

it is also possible to get estimates for the rate of convergence of difference quotients to the c rresponding derivatives.

We have thus the following more precise

version of Theorem 4.5. Theorem 7

Assume that (2) and ~

respectively, and that ~

are parabolic in the sense of Petrovskii and John

is accurate of order ~i °

Let

85

be a finite difference operator which is consistent with the differential operator Q of order q and also accurate of order~ .

Then for O <

nk = t ~ T,

In view of the fact that unsmooth initial-data give rise to lower rates of convergence it is natural to ask if the convergence be made faster by first smoothing the initial-data.

This is indeed the case for parabolic systems and we shall des-

cribe a result to this effect(Kreiss, Thome~e and Widlund [21]). We shall consider operators of the form

a,, v where

~

.._ ¢~,-~- v

:9

~,c-,/= ~_a ¢(,<",,/,

is a function such that its Fourier transform satisfies A

.A

Here b~" ~), j = 0,I are such that for some S ~

0, b ~'~ V) and b ~" ) coincide with multi-

pliers on ~ W

for ~ ~ ~ < ~ ~ and I~ 1> ~ respectively. Such an aperator is said to P be a smoothing operator of order ~ in W . Since the multipliers on ~ L 2 are simply P the functions in Lee , the above condition can be seen to be satisfied for p=2 if

and for any multi-index

~9'~

~ } O

:

3

¢C1~-~ ~1~]

~=~ ~

""

uniformly in .~. Special examples of smoothing operators of orders 1 and 2, respectively, in ~he case d=l are

[

k

'1

86

and for general

, a smoothing operator of order ~

can easily be constructed in

the form

where ~

D is a function which is piecewise a polynomial of degree ~ -I and which

vanishes outside

p=2, the operator ~

where

\

i~4~j~,~-. -~

0 ~ ~ ~ Y~

for ~

odd and

L--}x4\jy-~)

for~

even.

For

corresponding to

is a smoothing operator of arbitrarily high order.

In this

case

Smoothing operators in higher dimensions can be obtained by taking products of onedimensional operators

d

where ~ , j is a smoothing operator with respect to xj. The result on the rate of convergence is then the following. Theorem 8

Assume that (2) and ~

respectively, amd that ~

and ~

are parabolic in the sense of Petrovskii and John, are accurate of order ~

. Then there is a constant

C=Cp, T such that for 0 ~ t = ~k ~ T,

We shall complete this lecture by considering the case of the simple hyperbolic equation --y ~

~

~

real constant ~

and a consistent difference operator of the form \

with characteristic polynomial

87

we

shall examine the case when

~

is stable in L a but unstable in ~

; more pro-

cisely we shall assume that I ~I

~

t[~

I for

realpol~o~al

,

t(°~ ~ 0

The following result on the rate of convergence is due to Hedstrom [13] and Brenner

and

Thomde [3].

Theorem ~

Under the above assumptions on the operator Ek, then for s ~

and ~ +,/~ ~ -~1

O, s ~ * ~

~ and ,~ ~ T,

:o,°

It can also be proved that this result is best possible in the sense that the function g(s) above is the largest for which an estimate of the form (I0) holds for all v6 B s. In the stable cases, i.e. when ~3 = f * ~ or p=2~ the order of converP gence is s)~/( )~ +I) when 0 < s < J~- +i im agreement with Theorem 3. In the opposite c a s e t h e e r r o r

is larger

f o r s
For small s, g(s) is then

negative and for s--O we recognize the exponent in e.g. (&.ll) (with some difference in notation)e It is interesting to note that if the irregularity of the initial-function stems from the behavicur at isolated points then the result above may be improved so that for p = ~

we obtain the same result as if ~

formulate this result in terms of the Banach space ~

were stable in ~ .

We shall

of functions v with support in

[-M,O] such that

is finite.

Using the L ~ convergence result, Sobolev's inequality, and the fact that

is continuously embedded in B~+ ~ one can prove the fonowing result

BvS.

88

Theorem lO

Consider a L, stable operator ~

positive M 8Ln~.s ~ ~

÷i end for TLk ~

This result also holds when ~ /

T.

for the equation (5). Then for given

h~,e,~j~./(r+,B)

depends upon x.

89

REFERENCES [i]

M.Y.T. Apelkrans.

On difference schemes for hyperbolic equations with dis-

continuous initial values. Math. Comp. 22 (1968), 525-539. [2]

Ph. Brenner.

The Cauchy problem for sy~netric hyperbolic systems in L . P Math. Scand. 19 (1966), 27-37.

[3]

Ph. Brenner and V. Thomee. difference schemes.

Stability and convergence rates in L

Math. Scand.

P

for certain

To appear.

/

[Z~]

Ph. Brenner and V. Thomee. schemes.

[5]

Estimates near discontinuities for some difference

To appear.

M.L. Buchanan.

A necessary and sufficient condition for stability of diffe-

rence schemes for intial-value problems.

J. Soc.Indust.Appl.Math.

Ii

(1963), 919-935. [6]

R. Courant, K. Friedrichs and H. Lewy.

~er

die partiellen Differenzen-

glelchungen der mathematlschen Physik. Math. Ann. lO0 (1928), 32-74. [7]

A. Friedman.

Partial differential equations of parabolic type.

Prentice-Hall. Englewood Cliffs, New Jersey, 1964. [8]

K. Friedrichs.

Symmetric hyperbolic linear differential equations.

Comm. Pure Appl. Math. 7 (1954), 3&5-392. [9]

I.M. Gelfand and G.E. Schilow,

Verallgemeinerte Funktionen III.

Deutscher Verlag der Wissenschaften, Berlin, 196&.

[lO]

S.K. Godunov and V.S. Ryabenkii. schemes.

Ill]

Introduction to the theory of difference

Interscience. New York, 196~.

G.W. Hedstrom.

The near-stability of the Lax-Wendroff method.

Numer. Math. 7 (1965), 73-77.

[12]

G.W. Hedstrom.

Norms of powers of absolutely convergent Fourier series.

Michigan Math. J. 13 (1966), 393-&16.

[13]

G.W. Hedstrom.

The rate of convergence of some difference schemes.

SIAM J. Numer. Anal. 5 (1968), 363-&06.

[l~]

G.W. Hedstrom,

The rate of convergence of difference schemes with constant

coefficients. BIT 9 (1969), 1-17.

[15]

F. John,

On integration of parabolic equations by difference methods,

Con. Pure AppI. Math. 5 (1952), 155-211.

Ix6]

II

H.0. Kreiss.

Uber Matrlzen die beschrm~kte Halbgrupp~ erzeugen.

Math, Stand. 7

(1959), 71-80.

90

[17]

H.0. Kreiss.

Uber die Losung des Cauchy problems fur lineare partielle

Differentialgleichungen mlt Hilfe yon Differenzengleichungen. Acta Math. lO1 (1959), 179-199. [18]

H.0. Kreiss.

Uber die Stabilitatsdefinitien fur Differenzengleichumgen

die partielle Differentialgleichungen approximieren. BIT 2(1962 ), 153 -181. [19]

H.0. Kreiss.

Uber sachgemasse Cauchyprobleme.

Math. Soand. 13 (1963),

109-128. [20]

H.O. Kreiss.

On difference approximations of dissipative type for hyper-

bolic differential equations. Comm. Pure Appl. Math. 17(1964), 335-353. [21]

H.O. Kreiss, V. Thom~e and O.B. Widlund.

Smoothing of initial data and

rates of convergence for parabolic difference equations. Comm. Pure Appl. Math. [22]

P.D. Lax and R.D. Richtmyer. difference equations.

[23]

To appear.

P.D. Lax and B. Wendroff.

Survey of the stability of linear finite

Comm. Pure Appl. Math. 9 (1956), 267-293. Systems of conservation laws.

Comm. Pure Appl.

Math. 13 (1960), 217-237. [24]

P.D. Lax and B. Wendroff. high order or accuracy.

[25]

J. Lofst~m.

Difference schemes for hyperbolic equations with Comm. Pure Appl. Math. 17 (196~), 381-398.

Besov spaces in theory of approximation.

Ann. Math. Pure Appl. 85 (1970), 93-184. [26]

@.G. O'Brien, M.A. Hymaa and S. Kaplan.

A study of the numerical solution

of partial differential equations. J. Math. and Phys. 29(1951), 223-251. [27]

J. Peetre and V. Thom~e. value problems.

[28]

Math. Scand. 21 (1967), 159-176.

R.D. Richtmyer and K.W. Morton. problems.

[29]

On the rate of convergence for discrete initial-

Difference methods for initial-value

2nd ed., Interscience, New York, 1967.

V.S. P%vabenkii and A.F. Fillipow.

Uber

die Stabilitat yon Differenzen-

gleichungen. Deutscher Verlag der Wissenschaften, Berlin, 1960. [30]

S.I. Serdjukova.

A study of stability of explicit schemes with constant w

real coefficients. Z. Vycisl. Mat. i Mat. Fiz. 3 (1963), 365-370. [31]

S.I. Berdjukeva.

On the stability in C of linear difference schemes with

constant real coefficients. Z. Vycisl. Mat i Mat. Fiz. 6(1966), [32]

W.G. Strang.

Polynomial approximation of Bernstein type.

Trans. Amer. Math. Soc. 105 (1962), 525-535. [33]

V. Thomee.

Stability of difference schemes in the maximum-norm.

J. Differential Equations 1 (1965), 273-292.

&77-486.

91

[3~]

V. Th°mee.

On maximum-norm stable difference operators. Numerical Solution

of Partial Differential Equations (Proc. Syrup.s. Univ. Maryland, 1965), pp. 125-151.

Academic Press. New York.

[55]

V. Theme~e. Parabolic difference operators. Math, Scan~, 19 (1966), 77-107.

[36]

V. Thomee.

Stability theory for partial difference operators.

SIAM Rev. Ii (1969), 152-195.

[57]

V. Thomee.

0n the rate of convergence of difference schemes for hyperbolic

equations.

Numerical Solution of Partial Differential Equations. (Proc.

Sympos. Univ. Maryland~ 1970).

[38]

0.B. Widlund.

To appear.

On the stability of parabolic difference schemes.

Math. Comp. 19 (1965), 1-13.

[39]

O.B. Widlund. norm.

Stability of parabolic difference schemes in the maximum-

Numer. Math. 8 (1966), 186-202.

0.B. Widlund.

On the rate of convergence for parabolic difference schemes,

II. Comm. Pure AppI. Math. 23 (1970), 79-96.

Iteration Parameters

in the

Numerical Solution of Elliptic Problems EUGENE L. WACHSPRESS General Electric Company Schenectady, New York

94

These n o t e s a r e i n t e n d e d to serve as a guide to a d e e p e r stud~ of ~ a t e r i a l presented in a series

of lectures

delivered

i n S e p t e m b e r , 1970 a t t h e U n i v e r s i t y

Dundee as a part of the special one year symposium on The Theory of Numerical Aridlysis.

Subject

Lecture i

A Concise Review of the General Topic and Background Theory Successive Overrelaxation: Theory Successive 0verrelaxation: Practice Residual Polynomials and Chebyshev Extrapolation: Theory Residual Polynomials : Practice Alternating-Direction.Implicit Iteration: Theory Parameters for the Peacems~-Rachford Variant of ADI

Reference text: "Iterative Solution of Elliptic Systems," by Wachspress (Prentice

Hall, 1966).

of

95

i.

A CONCISE REVIEW OF THE @ENERAL TOPIC AND BACKGROUND THEORY We are concerned with iterative approximation to the vector x which satisfies

the system of linear equations Ax

=

(1)

k

m

where:

k is a known m-vector A is a given nonsingular mxm matrix, and x is an m-vector which is to be found. An approximation y to x is acceptable when

//~.-x// //

<

~_ / /

(2)

E,

where E is some prescribed error bound and // . // a designated norm. We shall first categorize various iteration procedures. cribe a measure of efficiency or rate of convergence.

We shall then des-

Having done this, we will

indicate a rather general approach to demonstrating convergence for a wide class of methods for iterative solution of linear systems.

Finally, an example of each of

three major classes of linear iteration procedures will be given. It is convenient to restrict matrix A in (i) to be real, symmetric, and positive definite.

(Our definition of positive definite is such that a real matrix

which is p.d. must be symmetric. )

Although less restrictive conditions are subject

to ar~lysis, the discussion is greatly simplified in this manner. A STATIONARY LINEAR ITERATION procedure is characterized by : xo

=

a known "trial" v e c t ~

~,

=

T ~n-, + R k ,

(3) n = 1,2, . . . .

This procedure is convergent if~ x n ~ x = A-'k for any xo and k. In oraer that x be a stationary point, we require: x = ~ x + ~ k

m

.

m

(J+)

Defining the error vector e_, = ~, - x, and subtracting (4) from (3), we get = w _on I = T" a o

en ~ O_ for arbitrary eo iff Tn ~ O.

,

(5)

96

The spectral Tad_lus of T is r(T) = max/gi(T)/ where the gi are eigenvalues

of ~.

Thus, r(~) muet be lose t ~ n unity for oonverge~e.

From (~), (:-~)A-'k_ =

R k for any k so that (I-T)A -1 = R, and (3) can be written x_, = r ~ _ ,

+ (:-T)A-'~_.

(6)

If we could compute A-'k on the right hand side of (6), we would have no need for iteration.

Thus, T must be such that the right hand side of (6) does not require

computation of A "I or

A-'k.

To clarify this, suppose we can solve the system

Bx = k for x where B approximates A in some sense. B x . = ( B - A ) x._,

+ k

We may attempt to iterate by:

. cr

_x. = (:-B-'A) x._, + B-'k. Here, T = I-B-'A and (I-T)A-' = B-' = R. We note that B is a "good" approximation to A for this iteration when the spectral radius of (I-B -I A) is much less than unity. We will now derive a condition sufficient to assure convergence which has application to many iteration techniques. A is positive definite and hence has a unique positive-definite square root, A ~.

Thus, ( I - B " A ) is similar to (I-A~B-'A~), whose spectral radius is bounded

above by the square root of the largest eigenvalue of , I (I-A'~BT-IA:) I I (I-A:B-'A:) The

=

I-

A ½B'' (B+BT-A)BT-I A ½

(7)

condition sufficient for convergence is that B + B T - A be positive definite.

For then we can define K = A~B-'(B+BT-A) ~ and M = (I-A~B-'A~) and rewrite (7) as MM T = I-KKT; we note here that K is nonsingular, being the product of nonsingular matrices.

Thus

0 ~ g(MM T) = g(l-KE T) = I - g(KKT) < i, and we have proved that

r(I-B-'A) < i if B+B T - A is positive definite.

To convince ourselves that this is

not a necessary condition, we need only find one counterexample : yield

with r(T) = ~ while B+B T - A

is net positive definite.

97

If we choose our norm so that // T // "~ r(T) and if xo = O_, then eo = -x and

II e. /I = / / ~ " ! o / I n

"- log E log r

"

<_.II ~ lln /I ~o II ~ r" II ~_ II so tha~ II ~. /I // x ll

<

~. for

We note that when r is close to u n i ~ the number of iterations

required to satisfy a prescribed convergence criterion varies as i/(l-r). A PARTIALLY STATIONARY LINEAR

procedure is one for which the approximation

is obtained as a linear combination of vectors which could be obtained by a stationary procedure.

Let xj (J = 1,2,...,n) be iterates generated hy (6).

Then the n-th

iterate of a partially stationary procedure based upon this stationary iteration would be

V-n y, = ~ ajn xj.

If we define fn = Yn - Y = y., - x, then

fn = Pn(T) e_o , where P,(T) is a polynomial of degree n in T, normalized to unity when T=I.

(I is the identity matrix of order the same as T.)

Convergence is established by showing that for a given Pn and T: 1 r

=

limit Isup /P,(gi) / ~ n -, co

1

<

I

where the gi are the eigenvalues of matrix T. A NONSTATIONARY LINEAR iterative procedure is one for which the iteration matrix is a function of parameters which may change from iteration to iteration: Xn =

Tnxn_, + R.k or_ .

n Xn = I~I Tj)

n spectral radius of jn I= Tj,

xo

+

n ( I - JI~ Tj)

A-'k_ •

If r. is the I/n

then the asymptotic convergence rate is n~oolimit rn

.

When the Tj all commute, analysis is quite similar to that applied to partially stationary schemes.

When the Tj do not commute convergence theory is often less

definitive. In examining relative merits of iteration procedures, we endeavor first to establish convergence for a range of iteration parameters, second to determine the spectral radius as a function of these parameters, and third to ascertain how these parameters may be chosen to minimise this spectral radius. arises in the analysis of each iterative procedure. will be considered in subsequent lectures.

Thus, a minimax pro~lam

Three commonly used techniques

98

The se are : I. Successive Overrelaxation (stationary) II. Chebyshev Extrapolation (partially stationary) III. ~JLternating-Direction-Implicit

Iteration (nonstationary).

Each of these is well documented in the literature and these notes are intended only as an introduction to the subject rather than a detailed analysis. 2.

SUCCESSIVE 0VERRELAXATION:

THEORY

We may "improve" the value of a component of the vector x by computing as a new value during iteration n that number which yields satisfaction of the p-th equation with values from iteration n-I substituted in the equation for the remaining components of x:

m apjXj = k P

xj

from iteration n-I for j % p nforj =p.

j=l An iteration consists in improvement of all components of x. "Relaxation" or "simultaneous relaxation".

We may call this

On an array computer with many arithmetic

units working in parallel, it would be possible to improve all components simultaneously. We may, alternatively, use new neighbour values as soon as they are computed. Then

~

apjXjn

+ J=~p+lapjXjn-1 = kp ,

p -- 1,2,...,m.

Components are new improved in some order, and we call this "successive relaxation". Although this latter approach is often better than simultaneous relaxation, a more significant gain in efficiency is usually achieved by extrapolation.

If we denote

the unextrapolated result of successive relaxation at point p by Xpn , then before proceeding to the next point during the n-th iteration, we compute for a prescribed extrapolation parameter w :

Xpn =

Xpn-1 +

w(x~~n - Xpn-1 )

.

(8)

Numerical solution of elliptic type difference equations is accomplished with w in (~,2) and since w is greater than unity this is called "successive overrelaxaticn." Among factors responsible for extensive literature on S0R are its simplicity,

99

wide applicability, and firm theoretical foundation. D.M. Young's analysis provided a basis for efficient utilization of SOR.

He

introduced the concept of a "consistent ordering" which is related to what is now known as Young's Property A.

The equations of a system having this property may be

ordered as follows: An index s(p) is assigned to unknown x

associated with the p-th equation. P Then for every nonzero apj, the ordering s(p) is said to be consistent if s(j) = s(p)-i for j < p s(j) = s(p)+l for j > p. Components of x are improved in order of increasing s.

Points (equations) with a

common s-value which are coupled to one another directly must be updated simultaneously.

Many systems arising in practice may be consistently ordered without a need

for simultaneous improvement even though several components may have colmnon s-values. Five-point difference stars are an example. Young established an intimate relationship between eigensolutions of the simultaneous relaxation and consistently ordered $0R iteration matrices derived therefrom.

If g is an eigenvalue of the SR matrix, then there is a corresponding

eigenvalue h of the S0R matrix satisfying:

The optimum extrapolation parameter wb is obtained by solving the minimax problem: H(w,@)

=

maximum

/h(w,g)/

H(Wb,@ )

=

minimum H(w,G). w

The solution to this minimax problem (which is a rather simple minimax problem) is 2 wb = 1 + J1-~ 2 and H(wb,~) = Wb-1. (10) The remarkable gain in efficiency of S0R over SR is evidenced for the case &=-l-r for r << 1 by comparing the relative number of iterations required by the two methods for an error reductions of E: -in E n S R = -in E/r while ns0 R =

2 2~r

"

(II)

100

The potency of the convergence theorem given in the first lecture is illustrated by applying it to the S0R iteration matrix. A

=

D

-

R

- RT

Let the coefficient matrix be

where D is diagonal (and positive since A is positive-definite by

hypothesis) an~ R is strictly lower triangular.

If S0R is applied with the natural

ordering (successive updating of components 1,2,... ) then the S0R iteration matrix is

~. = (I - wD-'R)-'

(I - wl + wD-'R ~) .

This may be written in the alternative form D L = I - (w - R)-' A. Thus, B in equation (7) is equal to D _ R in this case, and w

B + BT

.

A

.

2D/w .

.

R .

RT

D + R + ~T = (2/w - l) D.

The spectral radius of L is less than unity when 2/w - i is greater than zero, or when w is in (0,2). It can also be shown by this approach that S0R converges even when the ordering and the extrapolation parameter are changed each iteration. When one digs deeper into the theory, one finds that the S0R iteration matrix with optimum parameter and consistent ordering does not have a diagonal Jordan form. The resulting eigenvector deficiency has an adverse effect on convergence.

3.

SUCCESSIVE OVERRELAXATION:

PRACTICE

For efficient implementation of SOR, one must choose an appropriate ordering and estimate the optimum parameter w b.

One must also choose a strategy consistent

with the characteristics of the computer for which the iteration program is designed. This latter point is sometimes overlooked.

One illustration is that on the CDC-6600

there is a stack feature which leads to a gain in speed by a factor of ten in the basic arithmetic when one programs the "inner arithmetic loop" in machine language, taking full advantage of the stack feature rather than relying on FORTRAN.

Another

consideration is relative efficiency of getting data in and out of fast memory an~ of computation once the data is in memory. be

On some machines several iterations can

performed in the time it takes to read the data in and out of memory.

The method

of concurrent iterations enables one to perform several iterations with one pass over the equations.

This is particularly important when solving large problems where all

101

the data cannot be contained in memory. Periodic boundary conditions present minor difficulties.

Line relaxation is

beneficial, and the proper choice of lines enables retention of consistent ordering. (I often think in terms of problems arising from discretization of partial differential equations tc obtain five-point or seven-point difference stars. ) A major consideration in any event is choice of w.

As the spectral radius of

SR approaches unity, it becomes increasingly more important to choose a good w. Several elaborate techniques have been described for estimating the extrapolation parameter.

However, a reasonably effective procedt~e which I have had success with

for many years does net require any sophisticated additional programming. One starts with a parameter, wo, chosen deliberately smaller than wb and iterates until am

asymptotic convergence rate is established.

This convergence rate is

measured by comparing successive changes in the vector:

II~

_~o II =

?or Wo < % :

H(Wo, ~), ~

~(wo,~) = (Wo~12 + J ~ ' l ~

~(Wo,~).

+ i - Wo )' .

To estimate wb we estimate, using the above equation:

(1-=')

(l-C,).

(i

- H

=

(w~,G).) "

~'wo

(H(wo,G)n

- (Wo - I)')

H(wo,=).

(12)

We thus estimate after n iterations with Wo : 2 w,=

i+

J(l

-

~)~

"

The presence of higher modes decaying at the rate Wo - i is such that in practice the asymptotic rate is approached from below and w I is less than wb.

Thus, the

extrapolation parameter may be updated periodically in this fashion. If one encounters a class of problems for which wb is quite close to two (say, greater than 1.95), then one should seek alternative procedures to either replace or supplement the SOR iteration.

102 4.

RESIDUAL POLYNOMIALS: CHEBYSHEV

EXTRAPOLATION: T~EORY

We may append to the stationary procedure

~_*. :

(13)

T _~n_, + ( T - T ) A - ' _k

the extrapolation ~_. = _ ~ . ,

+ w. (_~'. - _~., )

(14)

This differs from S0R in that improved values do not appear on the right hand side of (13) and the extrapolation parameter varle s with n.

It is easily shown that

e_. = [j~ (T + wj(T-T))]_eo = Pn(T) ~o where P.(T) is a polynomial of degree n in T (wj~O) and P.(I) = I.

(15) To obtain a

least upper bound on the error norm reduction for arbitrary eo and spectrum g of T, we solve the minimax problem

.p. = sup IP.(gi)I

gi HCn : sin

(16)

Hpn

P.(1) = l This classic Chebyshev problem has as its solution for gi arbitrary in the interval (a,b) with b less than unity:

0.(6)

:

cos(n cos-' 2g-(~+b) ) b-a 2-~a÷bl )

oosh(n cosh -I

The roots of this polynomial(b_a)arecos(2j-l)2n

z. :

h-a

(17)

"

+ (a+b)

2

J = 1,2,...,n.

(18)

By choosing wj =Izl-!- for use in (14), we generate the Chebyshev polynomial of degree ~ after n iterations. This approach has some undesirable features. to compute the set of wj. large.

We must decide upon n in advance

When h is close to unity, some of the wj become quite

In this case, large values of n are often required to yield an acceptable

approximation.

Roundoff error can become detrimemtal.

By utilizing the recursion formulas for Chebyshev polynomials, we can refine the iteration.

A three-term extrapolation formula is now used:

103

xj = xj_ 1

+

pj(xj - xj_ I)

+

qj(xj_ 1

- xj_2)

(20)

where the pj and qj are generated by: U

=

(b-a) 2 - (a+b)

(21)

p, = 2~/(b-~)

q, = 0

&

PJ = (b_aC)j_l(1/u)

and qj = Ci_2(1/u) cj(t/u)

cjU/u)

(22)

for j > I. The pj and qj approach asymptotic values of order magnitude unity as b approaches unity,

roundoff error

i s no l o n g e r a s e r i o u s

p r o b l e m , a n d we n e e d no l o n g e r d e c i d e

i n a d v a n c e upon t h e number o f i t e r a t i o n s .

There are other polynomial extrapolation procedures in common use.

One family

of iteration schemes based on Lanczos' work involves computation of the pj and qj from certain inner products of the xj and x~ vectors.

These include the steepest

descent and conjugate gradient methods. 5•

RESIDUAL POLYNOMIALS : PRACTICE The eigenvalue interval (a,b) upon which the extrapolation parameters are

based may be estimated from observed convergence with assumed bounds.

0s¢illatox,j

error behavior indicates that the assumed lower bound is too large while a uniformly signed error indicates the upper bound is too low (for the case where the assumed bounds lie within the true bounds).

By noting the sign of the error and the rate of

change, one can update the eigenvalue interval. Although it is possible to start a new parameter cycle after each updating, one may use the asymptotic values for pj and qj immediately.

This has proven to be

quite satisfactory for many problems. Systems which satisfy Young's Property A should be treated by the golub-Varga method which reduces the arithmetic by a factor of two. erence text. )

(See Pp. 155-6 in the ref-

104

The Chebyshev and S0R methods are comparable in efficiency.

The Chebyshev

method does often yield a greater reduction in error norm for a given number of iterations, but other factors often outweigh this.

Computer and problem character-

istics often dictate which approach is better. We may examine more precisely means by which the interval [a,b] may be estimated.

Although I have had no occasion to use the procedure which will now be

described, thus making this discussion more "theory" than "practice", the means by which parameters are updated is one of the more practical aspects of iteration and thus falls appropriately in this section. When Chebyahev extrapolation is based upon an assumed eigenvalue interval [a',b'] which contains the true interval [a,b], the asymptotic convergence rate is

ct_l ( ~ ) e

rtlim

~e

=

mE

b t-a!

where ~e

2-(a'+b' )

When [a,b] is not contained in[a',b'], error components associated with eigenvalues outside Is,b] ~ [a',b'] will eventually predominate.

If it is known that

either a' ~ a or b' ~ b, we seek only one bound and the procedure is analogous tc that already described for SOR.

We shall describe a more sophisticated approach for

estimating both a and b. After sufficiently ma~V iterations, s, we suppose that successive iterates satisfy

~+t

~- + ~t z , ( s )

t = ~,2,3,...

+ ~t ~ ( s ) ,

and define for k = 1,2,3, : •

= --%+k-~

k-i

- Us+k = k,

k-I

(l-k,)

X,

+ X,

It is easily shown that

e~ Let w (a,O) -- e, - ae2 + ~ . ~

(! + i

x,

.

i~)~

+ ~

1

~

. =o.

(1-k~)

v~

.

105

We may ascertain values for ~ and ~ which minimize II w (a,~) [12 ~o = C_,,.,~ ) (~.,~) - (,_,.,~ ) (,~.,~) (,~.,~) (m.,~) - (,~.,~)'

and

~o __ (-',-m ) ('~-'~) - (-" .m ) (m-'~) ('~-m ) ('~-m) - ('~ -m )" We may for example, compute =o and ~o after every ten iterations

(s = 10,20,30,... ) until values are obtained which do not change appreciably with s. Having determined ae and Be we may estimate k, and ka by using the relationship : 1

1

r

1

+ ^,r : ao

,

~,x, -

~o

Thus =

~o + 2~4"~°o-~+S~

and

),2

"~ -

=

2~o

are the estimated values for k. Referring to the Chebyshev polynomials with eigenvalue Z of the basic iteration (SR) matrix, we have

k'r

cosh [(s+t) oosh-' (2Z b-'-a'(a'+b'))] c2z - (a'+b')] cosh [(s+t-l) cosh-'~

Now let x = Then

b'-a'

2Z - (a'+~')

bt_a ,

x "- r ( x ÷ ~ ) ,

and we have

x = r/x + X/r 2 The estimates for a and b are obtained by substitution in the above equation: b'-a'

= ½ [Ca'÷b') ÷ ~ a~

a = - , ! , [ ( a ' + b ' ) + ~ b'-a'

(z + ~ ) 3 x, (r_.

~, + r ) ]

if x, ~r,

if

~,~-r

This analysis is intended primarily as a guide to further study of methods for estimating extrapolation parameters. specific problems.

Numerical procedures should be molded to suit

We have indicated how a comparison of observed convergence with

theoretical convergence provides a means for

updating parameters.

106

6.

ALTERNATING-DIRECTION-IMPLICIT ITERATION A family of non-stationary procedures is generated by splitting A into the stem

of two matrices so that a two step procedure of the following type is formed: A=H+V

(H + wjl)xj_~ = -(v - wj~)xj_ I + _k (23) (v + z/)x_j = -(H - ,jl)x_j_~ + _k J = 1,2, . . . . The matrices H and V are chosen so that this iteration is numerically convenient.

F o r example, five-point difference

equations can be handled when H

includes horizontal (along lines of constant y-value) coupling while V includes all vertical (along lines of constant x-value) coupling.

Then (23) involves a correc-

tion of each horizontal line treated as a block followed by correction of each vertical line.

When H and V are both positive-definite and all the wj and sj are

equal to a single positive constant, say w, convergence is easily demonstrated; _e. = T" eo where T = (V+wI)-'(H-wI)(H+wI)"(V-wI). T is similar to T' = K(H) . K(V) where K(X) " (X-wI)(X+wI)-' has a spectral norm less than unity for a~y positive-definite X.

Hence, the spectral radius of T' is

less than unity.

Convergence i s n o t a s s u r e d w i t h o u t f u r t h e r c o n d i t i o n s when wj and z j v a r y with ~.

Pear~y's theorem asserts that by using a large enough number of parameters

in a monotonically noninoreasing sequence (within the interval of the eigenvalues of H and V) one can obtain a spectral radius less than unity.

Repetitive appli-

cation of such a parameter cycle is convergent. The theory for this procedure is not as useful in application as theory for S0R and polynomial extrapolation. The analysis of the model problem, wherein H and V co-,,ute, is quite elegant. Optimum parameters are found by solving the minimax problem: t

g(x,._,_.) -

w~ - x Z~ + X

(25)

107

H

B

= ~imum tg(x,m,~_>.g(y,"_,=_ll a~x~b c ~ y ~ d

H o

o = minimumH

E ,z_ The e x i s t e n c e established

(26)

W Z

E,A

of a unique solution

i n ~y t h e s i s

(see ~IP

to t h i s

minimax ~oblem

'68 proceedings

for

was r e c e n t l y

a concise

summary).

Several

means are available for choosing nearly optimum parameters. It is interesting to review the literature on this problem and note how the theory has been developed during the past fifteen years. w

O

zO

and _

An analytic solution for

found by W.B. Jordan culminated the search for optimum parameters.

Never-

theless, this min~m-x problem was actually solved about i00 years earlier (as observed by J. Todd)!

Jordan first devised a bilinear transformation of variablesto

reduce the problem to an analogous one with identical eigenvalue intervals for both variables.

My thesis could then be used to establish that the set of wj is identi-

cal to the zj (except for order) for the transformed problem.

7.

PARAMETERS FOR THE FEACEMAN-RACHFORD VARIANT OF ADI The optimum parameters are obtained by theory involving modular transformations

of elliptic functions.

Numerical evaluation turns out to be quite easy.

An appro-

ximation valid over a wide range is:

wj

2(a/4b)rj (1 + (a/4b)2(1-rj)).b

(27)

i + (a/@b)2rj

2t

'

J = 1,2,...,t.

To illustrate the mathematical elegance of analysis of convergence rates of (23), we will derive the equations for generating parameters a~

which solve the

minimax problem: 2"

s(x,-_) =j~ Ha = ~imum ._-.Ig(x,a)l a~x~b Multiplying

aj+xa'1"x Ha

(28) O

--

numerator and demoninator of each factor

=

m~n~mum Ha a

.

(29)

-in the product

on t h e r i s h t

108

hand side of (28) by ab/ajx, we obtain

~(x,~) =

ab

ab

x

aj

+

ab

x

j=1

aj

As x varies from a to B, ab/x varies from b to a. o

as the set aj

(3o)

ab

Hence the set

ab/ o /aj

is the same

by virtue of the uniqueness of the parameters for any given eigen-

value interval,

Combining the factors with aj and ab/aj , we get

(x' + ab) - (aj + ~/aj)x (x" + ab) + (aj + ab/aj)~

(aj - ~)(ab/aj - x)

(aj + x)(ab/aj + x)

(x + ab/x) - (a~+ a~/ajl (x + ab/x) + (aj + ab/aj) Now 1,t ~, = ½(~ + ab/x) =d a~ = ½(a:, + ab/aj). ~,n

I~(~,~_)I =

(31)

a' - ~,i J=l

where (ab)½ = a' ( ~' ( b' = (a+b)/2. Continuing in this fashion, we successively reduce the number of factors in the product until we arrive at the one parameter problem:

a(n ) . x(n ) g(x(n),a) _- a!n) + x (.)

a(n ) b(n ) a(")

This is solved by noting that a~ n) a~. )

a(n) ,

x(n ) .

b(n ) .

.

or

= (a (") b(.>)"l"

(32)

We may work backwards to obtain a parameter "tree" by successive solution of quadratics:

(s-~)

a~S)

j(a~)), _ Ca(S)), (33)

aj, (s-1) -. Ca(S))" /

aj,(s-1)

109

Although (27) looks a lot simpler, this technique was developed before the elliptic function solution was known.

There is sn intimate connection between this

process an~ Lan~en transformations for evaluation of elliptic functions.

Introduction to Finite Difference Approximations to Initial Value Problems for Partial Differential Equations

OLOF WIDLUND New York University

This work was in part supported by the U.S. Atomic Energy Commission~ Contract AT(50-I)-1480 at the Courant Institute of Mathematical Sciences, New York University

112

i. Introduction The study of partial differential equations and methods for their exact and approximate solution is a most important part of applied mathematics, mathematical physics and numerical analysis.

One of the reasons for this is of course that very

many mathematical models cf continuum physics have the form of partial differential equations.

We can mention problems of heat transfer, diffusion, wave motion and

elasticity in this context.

This field of study also seems to provide a virtually

inexhaustable source of research problems of widely varying difficulty.

If in

particular we consider finite difference approximations for initial value problems we find a rapidly growing body of knowledge and a theory which frequently is as sophisticated as the theoryfor partial differential equations.

The work in this

field, as in all of numerical analysis, has of course been greatly influenced by the development of the electronic computers but also very much by the recent progress in the development of mathematical tools for problems in the theory of partial differential equations and other parts of mathematical analysis. Much of this progress has centered around the development of sophisticated Fourier techniques.

A typical question is the extension of a result for equations

with constant coefficients, to problems with variable coefficients.

In the constant

coefficient case exponential functions are eigenfunctions and such a problem can therefore, via a Fourier-Laplace transform, be turned into a, frequently quite difficult, algebraic one.

~uch recent work in the theory of finite difference

schemes, including much of that of the author has been greatly influenced by this development.

These techniques are usually referred to as the Fourier method an4 will

be the topic of several of Thom~e's lectures here in Dundee. lectures will be different.

The emphasis of these

W e w i l l concentrate on explaining what is known as the

energy method after a discussion of the proper choice of norm, stability definition etc.

We will also try to make some effort in relating the mathematics to the under-

lying physics and attempt to explain difference approximations.

a philosophy of constructing classes of useful

113

We have decided to use as simple technical tools as possible, frequently concentrating on simple model problems, to illustrate our points.

Some generality will

undoubtedly be lost but it will hopefully make things easier to understand and simplify the notations.

A considerable amount of time will be spent on analysing

the differential equations we are approximating.

Experience has shown that this is

the most convenient way to teach and work with the material.

The properties of the

differential equation are almost always easier to study and a preliminary analysis of the differential equations can frequently be translated into finite difference form.

This is particularly useful when it comes to choosing proper boundary condi-

tions for our difference schemes. The objective of our study is essentially to develop error bounds for finite difference schemes I methods to tell useful from less useful schemes and to give guidelines as to how reliable classes of schemes can be found.

On the simplest

level finite difference methods are generated by replacing derivatives by divided differences,

just as in the definition of a derivative, discretizing coefficient

functions and data by evaluating them at particular points or as averages over sm~ll neighbourhoods.

As we will see there are many choices involved in such discretiza-

tien processes and the quality of the approximate

solutions can vary most drasticalS~

The finite difference approsch has some definite advantages as well as disadvantages.

Thus the most one can hope, using a finite difference

scheme, is to be able

to get a computer program which for any given set of data will give an accurate answer at a reasonable cost.

The detailed structure of the mapping which transforms

the data into the solution will of course in general be much too complicated to understand.

Thus the classical approach giving closed form solutions to differential

equations frequently gives much more information about the influence on the solution of changes in data or the model.

The same is true perhaps to a somewhat lesser

extent, of methods of applied mathematics

such as asymptotic and series expansions.

However finite difference schemes and the closely related finite element methods have proved most useful in many problems where exact or asymptotic solutions are unknown or prohibitively expensive as a computational tool.

114

The main reference in this field is a book by Richtmyer and Morton [1967].

It

is a second edition of a book by Richtmyer [1957] which, in its theoretical part, is based to a great extent on work by Lax and Richtmyer. influenced by the work of Kreiss.

The new edition is heavily

A second part of the book discusses many specific

applications of finite difference schemes to problems of continuum physics.

There

is also a survey article by Kreiss and the author [1967], with few proofs, basea on lectures by Kreiss which still awaits publication by Springer Verlag.

It may still

be available from the Computer Science Department in Uppsala, Sweden.

Also to be

mentioned is a classical paper by Courant, Friedrichs and Lewy [1928] which has appeared in English translation together with three survey articles containing useful bibliographies [1967]. very much worth a study. [1969].

Another

classical paper, by John [1952], is also

Among recent survey articles we mention one by Thom~e

That paper essentially discusses the Fourier method.

2. The form of the partial differenti_al e~uaticns We will consider partial differential equations of the form, 8tu: P(x,t,ax)U , x ¢ Q, t ¢ [O,T], T < eo where u is a vector valued function of x and t.

The variable x = (xl ,...,Xs),

varies in a region O which is the whole or part of the real Euclidian space R s. When 0 is all of R s we speak of a pure initial or Cauchy problem; in the opposite case we have a mixed initial boundary value problem.

The differential operator P is

defined by

P(x,t,ax)

~

= Iv

where {~I = ~ ' i

Av(x,t

)

vl us ax, .... axs

m

and the matrices Av(x,t) have sufficiently smooth elements.

The

degree of the highest derivative present, m, is called the order of the equation. If we let the coefficients depend on u and the derivatives of u as well we say that the problem is nonlinear.

We will restrict our attention almost exclusively to linear problems and to the approximate calculation of classical solutions, i.e. solutions u(x,t) which are smooth enough to satisfy our equation in the obvious sense. In order to turn our problem into one with a possible unique solution we provide initial values u(x,O) = f(x). atu = ~z ' xu

-co

It is thus quite obvious that for the heat equation ~ x ~ co ,

a specification of the temperature distribution at some given time is necessary in order to single out one solution.

Frequently when O is not the whole space we have

to provide boundary conditions on at least part of the boundary 80 of O.

Sometimes

we also have extra conditions such as in the case of the Navier-Stokes equation where conservation of mass requires the solution to be divergence free.

The

boundary conditions, the form of which might vary between different parts of the boundary, have the form of linear (or nonlinear) relations between the different components of the solution and their derivatives.

The essential mathematical

questions are of course, whether it is possible to find a unique continuation of the data and to study the properties of such a solution. Like in the case of systems of ordinary differential equations, we can hope to assure the existence of at least a local solution by providing initial data, etc. It is however, often not immediately clear how many boundary conditions should be supplied.

Clearly the addition of a linearly imdependent boundary condition in a

situation

where we already have a unique solution will introduce a contradiction

which in general leads to a

nonexistence of a solution.

Similarly, the removal of

boundary condition in the same situation will in general lead to a loss cf

uniqueness.

Existence is clearly necessary for the problem to make sense, similarly

to require uniqueness is just to ask for a

deterministic mathematical model.

The

correct number of boundary conditions as well as their form is often suggested by physical arguments (Cf. §3). We now mention some simple examples which will be used in illustrating the theory.

The equation

8tU = 8xU

116

is the simplest possible system of first order, a class of problems of great importance.

We have already mentioned the heat equation

~tu = ~=xu . The equation 8tu = i 8~u ~t

is a simple model for equations of Schrodinger type.

It has several features in

common with a~u = 4 ~ u which arises in simplified time dependent elasticity theory.

Finally we list the

wave equation with one and two space variables: ~u

= 0xU ,

8~u = 8xU + 8yU .

The last three equations do not have the form we have considered until now being second order in t. variables.

This can however be easily remedied by the introduction of new

Thus let v =Stu and w = 8xU ,

Then

o)0x(:I

° (:)=

will be equivalent to the equation 8~u = -8~Uo provided.

Initial conditions have to be

We first note that u and 0tu both have to be given like in the case of

ordinary differential equations of second order.

This is also clear from the

analogy with a mechanical system with finitely many degrees of freedom.

The initial

conditions for w can be formed by taking a second derivative of u(x,0)o The wave equation, in two space variables, can be transformed into the required form in several ways.

We will mention two of these because of their importance in

the following discussion.

Let us first just introduce the new variable v = 0tu and

rewrite the wave equation as

As we will see later on this is not a convenient form and we will instead introduce three new variables

117

u, = a t u ,

u, = a x U ,

u, = ayu

which gives the equation the form

8t

Cu!)(i °Ilulo I •

0

0

o

o

ax

u2

+

u~

Ii

o

o

o

o

8y

ul u2

•

uj

Initial conditions for this new system are provided as above. In order to illustrate our discussion on the number of boundary conditions we consider

~tu:axu,

o: Ix~x~ol

, t~o.

u(x,o)=f(x)

If f has a continuous derivative then f(x+t) will be a solution of the equation for x ~ O, t ~ 0 and it can be shown that this solution is unique. of the origin only.

The boundary consists

If we introduce a boundary condition at this point say u(0,t) =

g(t), a given function, we will most likely get a contradiction and thus no solution. The situation is quite different if O = Ix; x ~ Ol (or what is essentially the same, 8tu = -SxU and O = Ix; x ~ 01).

The solution is still f(x+t) for x + t ~ O, t ~ 0

but in order to determine it uniquely for other points on the left halfline a boundary condition is required.

It can be given in the form u(O,t) = g(t), f(O) = g(O), g(t)

once continuously differentiable. g(x+t).

The solution for 0 ~ x + t ~ t, t ~ 0 will be

Thus different g(t) will give different solutions and the specification of

a boundary condition is necessary for uniqueness°

The condition f(O) = g(O) assures

us that no jump occurs across the line x + t = O.

3. The form of the fini_te difference schemes. We begin by introducing some notations. defined on lattices of mesh points only~ meshes: ~

We will be dealing with functions

For simplicity we will consider uniform

= Ix; x i = nih, n i = O, ~ i, ~ 2,...Io

of the fineness of cur mesh.

The mesh parameter h is a measure

We also discretize in the t-direction: R k = It; t = nk,

118

n = 0,1,2,..o I . When we study the convergence of our difference schemes we will let both h and k go to zero.

It is then convenient to introduce a relationship

between the timestep k and the meshwidth h of the form k = k(h), k(h) monetonicaS/y decreasing when h ~ O, k(0) = Oo

Often this relationship

is given in the form

k = kh m, m = the order of the differential equation and k a positive constant@ The divided differences, which replace the derivatives,

can be written in

terms of translation operators Th, i defined by Th,i~(x ) = ~(x + hei) ,

where e i is the unit vector in the direction of the positive x.-axiso 1

Forward,

backward and central divided differences are now defined by I

D+i ~(x) = ~ (Th,i - I)~(x) D i ~(~) : I (I - T h,i)~(x) #

Doi ~(x) : ~

I

(Th, i - T h, i)~(x) : ~(D+i ÷ D_i)~(x ) o

These difference operators serve as building blocks for our finite difference schemes°

The form of the complete schemes will become apparent as we go along°

We will now look into the mathematical derivation of the heat equation

in

order to illustrate a very useful technique for generating finite difference schemss@ Let us consider heat flow in a one dimensional medium@ temperature by u(x,t)o

Denote the absolute

The law governing the heat flow involves physical quantities,

the specific heat per unit volume K(x,t) and the heat flow constant Q(x,t) o heat energy per unit volume is K(x,t)u(x,t)

at the point x at time to

The

The quantity

Q(x,t)BxU is the amount of heat energy that flows per unit time across a cross section of unit area. Consider a segment between x and x + AXo

The amount that flows into this

segment per unit area per ~lit time is Q(x + Ax,t) 8xU (x + Ax,t) - Q(x,t) axU (x,t) and it must in the absense of heat sources be balanced by

at

x + Ax ]K(x',t) x

u(x',t) dx' o

119

A simple passage to the ]~m~t, after a division by Ax, gives

at(~) -- ax Qaxu • If the slab is of finite extent, lying between x = 0 and I, physical consiaerations lead to boundary conditions.

The heat flow out of the slab at x = 0 is

proportional to the difference between the inside and outside temperature uo.

With

an appropriate heat flow constant Qe we have a flow of heat energy at x = 0 per unit area which is Qe(U - ~ )

QexU +

Q,(u-

and the balance condition is therefore ue) = 0

at x = 0

.

If Qe is very large we get the Dirichlet condition u = ue at x = O.

Similar con-

siderations give a boundary condition for x = I. This derivation already contained certain discrete features.

In order to turn

it into a strict finite difference model we have to replace the derivatives an& integral in the balance conditions by difference quotients and a numerical q u a d r a t i c formula respectively.

We can get essentially the same kind of schemes by starting

off with a discrete model dividing the medium into cells of length Ax giving the discretely defined variable u

the interpretation of an average temperature of a cell.

The relation between the values of the discrete variable , i.e. the difference scheme, is then derived by the use of the basic physical balance conditions. It is clear that we can get many different discrete schemes this way.

In parti-

cular we do not have very much guidance when it comes to a choice of a gooa diseretization of the t derivative

.

We will now examine a few possible finite difference

schemes, specializing to the case K = Q = I and Dirichlet boundary conditions.

The

first two schemes are

u(x,t+k) = u(x,t) + ~_D+u(x,t) and

u(x,t÷k) --u(x,t-k) ÷ 2~_D÷u(x,t) for x = h 9 2h,..., l-h, t E ~ .

We assume that I/h is an integer and we provide the

schemes with the obvious initial and boundary conditions.

120

The first scheme, known as Euler's method or the forward scheme, can immediately be used in a successive calculation of the values of u(x,t) for t = k, 2k, etc.

The

second scheme requires the knowledge of at least approximate values of u(x,k) before we can start marching.

The latter scheme is an example of a multistep scheme.

The

extra initial values can be provided easily by the use of a one step scheme such as Euler's method in a first step.

We could also use the first few terms of a Taylor

expansion in t about t = 0 using the differential equation and the initial value function f(x) to compute the derivatives with respect to t. 8~u(x,O) : 8~f(x), eta.

Thus

atu(x,o) = a~f(x),

The possible advantage in introducing this extra complication

is that the replacement of the t derivative by a centered instead of a forward difference quotient should help to ~ k e the discrete model closer to the original one. Such considerations frequently make a great deal of sense.

We will, however, see

later that our second scheme is completely useless for computations. The difference between a finite difference scheme and the corresponding differential equation is expressed in terms of the local truncation error which is the inhomogenous term which appears when we put the exact solution of the differential equation into the difference scheme.

If the solution is sufficiently smooth we can

compute an expression for this error by Taylor series expansions.

We will later see

that a small local truncation error will assure us of an accurate numerical procedure provided the difference scheme is stable.

Stability is essentially a requirement of

a uniformly continuous dependence of the discrete solution on its data and it is the lack of stabili~ which makes our second scheme useless.

We will discuss stability

at s o m e length in 86. These two schemes are explicit, i.e. schemes for which the value at any given point can be calculated with the help of a few values of the solution at the immed_iate]y

preceeding time levels. Our next scheme is implicit: (I - ~ . D + ) u(x,t+k) -- u(x,t) .

It is known as the backward scheme. system of equations,

Each time step requires the solution of a linear

However this system is tridiagonal and positive definite and

can therefore be solved by Cholesky decomposition or some other factorization method

I21

at an expense which is only a constant factor greater than taking a step with an explicit scheme.

We will see that the backward scheme has a considerable advantage

over the forward scheme by being unconditionally stable, which means that its solution will vary continuously with the data for any relation between k and h.

F~

the forward scheme a restriction k/h 2 ~ ½ is necessary in order to assure stability. This forces us to take

very many time steps per unit time for small values of h.

0ur fourth scheme can be considered as a refined version of the backward scheme

(I - ~k D _D+) u(x,t+k) = (I + ~k D D+) u(x,t). This scheme, known as the CrankoNicclson scheme, is also implicit and unconditionally stable.

It treats the two time levels more equally and this is reflected in a

smaller local truncation error. We have already come across almost all of the basic schemes which are most useful in practice.

We complement the list with the well known Dufcrt-Frankel

scheme:

(I + 2k/h') u(x,t+k) = 2k/h z (u(x+h,%) + u(x-h,t)) + (1 - 2k/h') u(x,t-h) . In order to see that this two step scheme is consistent, which means formally convergent, to the heat equation we rewrite it as (u(x,t+k) - u ( x , t - k ) ) / ~

= (u(x+h,t) - u(x,t+h) - u(x,t-h) + u(x-h,t))/h"

and find by the use of Taylor expansions that it is consistent if k/h , O when h , O. The scheme is unconditionally

stable, explicit, suffers from low accuracy but it is

still quite useful becuase of its simplicity. Another feature of the Dufort-Frankel

scheme, worth pointing out, is that the

value of u(x,t+k) does not depend on u(x+2nh,t) or u(x+(2n+l)h,t-k), n = 0,+ i, _+ 2 .... We therefore

have two independent calculations and we can make a 50% saving ])y

carrying out only one of these using a so called staggered net.

4. An example of diver~ence~

The maximum principle.

We will now show that consistency is not enough to ensure useful answers.

In

fact we will show by a simple general argument that the error can be arbitrarily large for any explicit scheme, consistent with the heat equation, if we allow k to go to zero at a rate not faster than h.

122

Consider a pure initial value problem.

The fact that our schemes are explicit

and that k/h is hounded away from zero implies that only the data on a finite subset of the line t = 0 will influence the solution at any given point.

If now, for a

fixed point (x,t), we choose an intial value function which is infinitely man2 times differentiable, not indentically zero but equal to zero in the finite subset m~ntior~d above then the solution of the difference scheme will be zero at the point for all mesh sizes.

On the other hand the solution of the differential equation

u(x,t) = ~

Fe-(X-Y)'/~t

equals

f(y)dy ,

-o~ an~ thus for amy non negative f it is different from zero for all x and t > O. Using this solution formula we can prove a maximum principle, max lu(x,t)l ~ max If(x)I x

for all t ~ O .

x

Thus, after a simple change of variables, +oo

lu(x,t)l, . -If(x)l x

L, -''/'*t Jf(x-,)ld.,.,

f

~

-co

T h i s shows t h a t t h e s o l u t i o n

maximum norm sense• interpretation.

=

lf(x)l x

varies continuously with the initial

values i n the

This property is most essential and has a natural physical

It means, of course, that in the absense of heat sources the maxi-

mum temperature cannot increase with time.

Similar

inequalities hold for a wide

class of problems known as parabolic in Petrowskii's sense, for Cauchy as well as mixed initial value problems.

Cf. Friedman [196&].

We will now show, by simple means, that our first and third difference schemes satisfy similar inequalities, a fact which will be most essential in deriving usefal error bounds, etc.

First consider Euler's method with the restriction that k ~ ha/2.

The value of the solution at any point is a linear combination of the three values at the previous time level, the weights are all positive and add up to one. the maximum cannot increase.

Thus

For the third scheme we can express the value of the

solution at amy point as a similar mean value of one value at the previous time

123

level and at those of its two neighbours.

Therefore a strict maximum is possible

only on the initial line or at a boundary point.

This technique can be used for

problems with variable coefficients and also in some nonlinear cases.

Unfortunately

it carmot be extended to very many other schemes because it requires a positivity of coefficients which does not hold in general. For the finite difference schemes discussed so far we have had no problems with the boundary conditions.

They were inherited in a natural way from the differential

equation and in our computation we were never interested in using more than the next neighbours to any given point.

We could however be interested in decreasing the

local truncation error by replacing 8xU by a difference formula which uses not three but five or even more points.

This creates problems next to the boundaries where

some extra conditions have to be supplied in order for us to be able to proceed with the calculation.

It is not obvious what these extra conditions should be like.

Perhaps the most natural approach, not always successful, is to require that a divided difference of some order of the discrete solution should be equal to zero. This problem is similar to that which arises by the introduction of extra initial values for multistep schemes but frequently causes much more serious complications. If we go back to our simple first order problem 8tu = 8xU , we see that there are two possibilities.

Either we use a one sided difference such as

u(x,t+k) = u(x,t) + kO+u(x,t) for which no extra boundary condition is needed or we try a scheme like Euler's

u(x,t÷k) = u(x,t) + ~ o u ( x , t ) for which a boundary condition has to be introduced at x = O.

We leave to the

reader the simple proof that, for k/h ~ l, the solution of our first scheme depends continuously on its data in the sense of the maximum norm.

The second scheme is,

as we will later show, unstable even for the Cauchy case.

The problem to provide

extra boundary data however still remains even if we start out with a scheme which is stable for the Cauchy ease.

124

We also mention another method which has some very interesting features; u(x,t÷k) + u(x-h,t+k) + ~_u(x,t,k) = u(x,t) + u(x-h,t) -

~_u(x,t),

uCo,t) = o,

uCx,o) = f(x), o ~ x < ®

.

This difference scheme approximates 8tu = -SxU on the right half line.

It has been

studied by Thome~e, [1962J, and is also discussed in Richtm2er and Morton [1967]. It is implicit but can be solved by marching in the x-dlrection and could therefore be characterized as an effectively explicit method.

It is unconditionally stable.

Finally we would like to point out a class of problems for which the boundary conditions create no difficulties namely those which have periodic solutions.

This

allows us to treat every point on the mesh as though it were an interior point.

In

the constant coefficient case such problems can be studied successfully by Fourier series.

The analysis of a periodic case is frequently the simplest way to get the

first information about the usefulness of a particular difference scheme.

5. The choice of norms and stability definitions In the systematic development of a theory for partial differential equations questions about existence and uniqueness of solutions for equations with analytic coefficients and data play an

important role.

Cf. G~rabedlan [1964].

The well

known Cauchy-Kowaleski theorem establishes the existence of unique local solutions for a wide class of problems of this kind.

As was pointed out by H a ~ r d

[1921],

in a famous series of lectures, such a theory is not however precise enough when we are interested in mathematical models for physics.

We also have to require, amm~g

other things, that the solution will be continuously influenced by changes of the data, which we of course can never hope to measure exactly.

The class of analytic

functions is too narrow for our purposes and we have to work with some wider class of functions and make a choice of norm. considered the ideal one.

In most cases the ~aximum norm must be

We have already seen that for the heat equation such a

choice is quite convenient and that the result on continuous dependence in this nc~m has a nice physical interpretation.

For other types of problems we also have to be

guided by physical considerations or by the study of simplified model problems.

125

Hadamard essentially discussed hyperbolic equations and much of our work in this section will be concentrated on such problems.

A study of available closed

form solutions of the wave equation naturally leads to the following definition. Definitio~

An initial value problem for a system of partial differential

equations is well posed in Hadamard's sense if, (i) there exists a unique classical solution for amy sufficiently smooth initial value function, (ii) there exists a constant q and for every finite T > 0 a constant CT such that

=az

lu(x.t)J

~ CT

x t60,T]

mx

la~u(x,O)l

•

x,l~1 ~ q

One can ask if it is always possible to choose q equal to zero.

A study of our

simplest first order h~perbolic equation 8tu = axu gives us hope that this might be possible, and so ~ e s

an examination of the wave equation in one space variable

a~u --o'a~u,

u(x,o) = f(x) ,

atu(x,o) : g(x)

which, after integration along the rays x = Xo+ct, is seen to have the solution x+ct

u(x,t)

=

f(x+.t) + f(xct) '"

2

i

+

~c

[

g(s)d~.

k-ct In fact these two equations are well posed in Lp, 1 4 p 4 oo, in a sense we will soon specify. We will soon see that a choice of q = 0 is not possible for the wave equation in several space variables.

Before we explain

this further we introduce some

concepts which we will need repeatedly. Because of the linearity of our equations there is, for any well posed intial value problem, a linear solution operator E(t,t I ) 0 g tl g t which maps the solution at time tl into the one at time t. u(x,t) = ~(t,0)f(x)

if

In particular u(x,0) = f(x) .

0he of Hagam~ra's requirements for a proper mathematical model for physics is that the solution operator forms a semigroup i.e. E(t,r) E(r,tl) = E(t,t)

for

0 < t, ~ T ~ t

.

126

When we deal with completely reversible physical processes the semigroup is in fact a group.

Such is, for instance, the case for wave propagation without dissipation.

We now introduce the definition of well posedness with which we will finally choose to work. Definition

An initial value problem is well posed in L

if, P (i) there exists a unique classical solution for any sufficiently smooth

initial value function, (ii) there exists constants C and a such that [[E(t,~ )flip ~ c By the L

exp(,(t-t, ))Llfllp •

norm, of a vector valued function, we mean the L P

norm, with respect P

to x, of the 1 2 - norm of the vector. Littman [1963] has shown, by a detailed study of the solution formulas for the wave equations, that except for one space dimension they are well posed only for p = 2.

His result has been extended to all first order systems with symmetric

constant coefficient matrices by Brenner [1966]. Theorem (Brenner [1966]).

Consider equations of the form

s

atu=~A

A

8x u ,

constant and symmetric matrices.

p # 2 if and only if the matrices A

This system is well posed in the L

P

for

a

commute.

This leaves us with only two possibilities.

Either we only have one space

variable or there is a eommon set of eigenvectors for the matrices A .

In the

latter case we can introduce new dependent variables so that the system becomes entirely uncoupled, consisting of a number of scalar equations. Brenner's proof is quite interesting but too technical Instead we will first show the well posedness in ~

to be explained here.

of symmetric first order systems

and then proceed to show that the wave equation is not well posed in Lco for several space variables. a choice of q = O.

This will of course answer the question about the possibility of

127

We note that most~perbolic equations of physical interest can be written as first order systems with symmetric coefficient matrices. Introducing the standard L 2 inner product we see that for any possible solution to the equation, which disappears for large values of x, s

at(u,u)

2

(Aa=

/~=1

~

s

/J=1

~=1

Therefore S

at(u(t), u(t)) = - 2 L ( u , ( a x

A )u) (

const. (u(t), u(t))

if the elements of A (x) have bounded first derivatives. From this immediately follows [lu(t)[l~ ~ exp((const./2)t)[[u(O)H2 In particular we see that the L 2 norm of u(x,t) is unchanged with t if the coefficients are constant. The restriction to solutions which disappear at infinity is not a serious one. Any L 2 function can be approximated arbitrarily closely by a sequence of smooth functions which are zero outside bounded, closed sets.

A generalized solution can

therefore for any initial value in L 2, be defined as a limit of the sequence of solutions generated by the smooth data.

This is of course just an application of a

very standard procedure in functional analysis, for details el. Richtmyer and Morton. Examining the solution formula for the wave equation in one space variable

we

find that information is transmitted with a finite speed less than or equal to c. This finite speed of propagation is a characteristic of all first order hyperbolic equations.

Thus the solution at a point (x,t) is influenced solely by the initial

values on a bounded subset of the plane t = O. of dependence of the point.

This subset is known as the domain

Similarly any point on the initial plane will, for a

fixed t, only influence points in a bounded subset. We also see, from the same solution formula, that of dependence is of particular importance.

the boundary of the domain

This property is shared by other hyper-

bolic equations. In particular for the wave equation in 3, 5, V ... space variables

128

the value at any particular point on the initial plane will, for a fixed t, only influence the solution on the surface of a certain sphere.

This result, known as

Huygen's principle, can be proved by a careful study of the solution formula of the wave equation.

It is of course also well known from physics.

Cf. @arabedlan [1964S.

We have now carried out the necessary preparations for our proof that the wave equation in three space dimensions cannot be well posed in Leo. equation in the form of a s y ~ e t r i c first order system.

We write the

We choose for all compo-

nents of the solution, the same spherically symmetric class of initial values namely C~

functions which are equal to one for r ~ ~/2 and zero for r ~ ¢ and having valus~

between 0 and 1 for other values of r.

The spherical symmetry of the initial values

will lead to solutions the values of which depend only on the distance r from the origin an~ on the parameter ¢.

It is easy to show that 8tU,axiU ,i = I ,2,3, are

solutions of the wave equation and that they therefore satisfy Huygen's principle. Therefore, the solution at t = I/c will be zero except for values of r between I - E and I + ¢.

We know that the L 2 norm of the solution is unchanged and it is easy to

see that it is proportional to ~ in L

eo

.

.

Now suppose that the equation is well posed

This means not only that the maximum of all components of the solution at

t = I/e is bounded from above by a constant independent of c but also that the norm of the solution is bounded away from zero uniformly.

If this were not the case we

could solve our wave equation backwards and we could not have both well posedness in L

eo

and a solution of the order one at t = O.

Denote by CI the maximum of the component of largest absolute value at t = I/o. This point has to have a neighbour at a distance no larger than constant × cJ the value is less than C~/2. not be of the order ~ 2

where

In the opposite case the L 2 norm of the solution could

because the volume for which the solution is larger than

C~/2 would exceed constant × ¢3.

Our argument thus shows that the signals have to

become sharper, in other words, the gradient of the solution increases.

At t = O

it is of the order I/¢ and it has to be proportional to ~-~ at t = I/o.

This

however contradicts our assumptions because first derivatives of a solution are also solution of the wave equation and their maximum cannot grew by more tham

129

constant factor. posed in L

ao

Thus the wave equation in three space variables is not well

and a choice of q > 0 sometimes has to be made.

The well posedness of the wave equation in L 2 has a nice interpretation in terms of physics.

In one dimension, for example, the kinetic energy is

p/2

[(atu)~dx and J = c 2 = the square of the speed of

the potential energy is T/2 f(SxU)Zdx , where Tip J propagation. The total energy is therefore p/2 /((~tu) 2 + c2(axU)2)dx and it

remains unchanged in time in the absense of energy sources, i.e. inhomogenous or boundary terms.

In fact we note that our proof of the well posedness in L 2 of

symmetric first order system is our first application of what is known as the energy method. The fact that all first order hyperbolic equations have finite speeds of propagation has immediate implications for explicit finite difference schemes.

Thus,

just as for explicit methods for parabolic problems, we have to impose certain restrictions on the relation between k and h in order to avoid divergence. appropriate condition has the form k/h sufficiently small. tive than the condition in the parabolic case.

The

It is thus less restric-

This is known as the Courant-Friedriohs-

Lewy condition and simply means that, for sufficiently small values of h, any point of the domain of dependence of the differential equation is arbitrarily close to points belonging to the domain of dependence of the difference scheme.

It is easy

to understand how in the opposite case we can construct initial value functions which will give us arbitrarily large errors at certain points. Experience also shows that it is advisable to use schemes and k/h which allows us to have the domains of ~ependence coincide as much as possible while satisfying the Courant-Friedrichs-Lewy conditions.

This is related to the particular importance

of the boundary of the domain of dependence mentioned above.

However it should be

pointed out that this is hard to achieve to any great extent when several propa@ation speeds are involved and when they vary from point to point. It should be mentioned that one can show that any first order problem which is well posed i n L 2 (or any Lp space) is well posed in Hadamard's sense. Hadamard's definition is less restrictive than the other one.

Therefore

The proof is by

showing that derivatives of solutions also satisfy well posed first order problems and the use of a so called Sobolev inequality.

130

A result by Lax, [1957] gives an interesting side light on the close relation between the

questions of well posedness, existence and uniqueness.

without a proof.

We describe it

Thus if a first order system with analytic coefficients, and not

necessarily hyperbolic, has a unique solution for any infinitely differentiable initial value then it must be properly posed in Hadamard's sense.

A corollary is

that Cauchy's problem for Laplace's equation, which can be rewritten as the CauchyRiemamm equations and which is the most common example of a problem which is ill posed, cannot be solved for all smooth initial data. Another interesting fact is that it can be shown that homogenous wave motion, satisfying obvious physical conditions such as finite speed of propagation etc., has to satisfy a hyperbolic differential equation of first order.

This gives added

insight into the importance of partial differential equations in the description of nature.

For details we refer to Lax [1963].

We still face a choice between the two definitions of well posedness.

Hadamard's

choice has the advantage of being equivalent to the Petrowskii condition in the cs~e ef constant coefficients.

The Petrowskii condition states that the real part of the

eigenvalues of of the symbol ~ of our differential operator P should be bounded from above.

The symbol is defined by

S

where ~ ¢

an~

< ~,x ~ --

~ixi i=I

and is thus a matrix valued polynomial in ~. natural for the constant coefficient case.

This algebraic condition is most We are ~mmediately led to it if we start

looking for special solution of the form exp(kt) exp i < ~,x > ~, ,

~ some vector .

Thomee will probably discuss these matters in much more detail.

For a proof of the

equivalence between the H~Ismard and Petrowskii conditions of. Gelfan&

[196~].

and Shilov

131

Due to the effort of Kreiss [1959], [1963] four algebraic conditions which are equivalent t o ~ l l posedmess in L 2 are known in the constant coefficient case. full story is quite involved and subtle.

The

We only mention one of these conditions.

Thus a constant coefficient problem is well posed in L z if for some constants a and K and for all • such that Res > 0

ICCs + ~) - ~ , ) ) - '

I, ,, z / R ~

.

We leave to the reader to verify,using this and the Petrowskii conditions,that cur first attempt to rewrite the wave equation is well posed in Had~mAr~'s sense but not in L 2.

The intuitive reason why our second attempt was more successful

is that the

new variable r~turally defined a norm which defines the energy of the system while no similar physical interpretation can be made in the first case. The algebraic conditions just introduced are about the simplest possible criteria we can hope to find to test whether or not a differential equation is well posed. Analogous criteria have been developed for finite difference schemes.

We will now

try to find out if they could be used for problems with variable coefficients as wall . It is known from computational experience that instabilities tend to develop locally and it is therefore natural to hope that a detailed knowledge of problems with constant coefficients, obtained by freezing the coefficients at fixed points, should provide a useful guide to problems with variable or even nonlinear coefficdsnts. The constant coefficients problemscan be treated conclusively by the Fourier transfo I'm. This idea is quite sound for first order and parabolic problems provided our theory is based on the second definition of well posedness. for all problems.

It is however not true

This is illustrated by the following example, due to Strang [1%6]

atu -- i8 x sin XaxU = i sin XSxU + i cos XSxU •

This is well posed i n L 2 because, using the scalar product (u,v) = f u

vdx and

integration by parts, at(uCt), uCt)) = o for any possible solution.

However if we freeze the coefficients at x = 0 we get

132

atu = iaxU which violates the Petrowskii condition. For a more detailed discussion we refer to Strang's paper. The main critisism of thm Petrowskii condition is that it is not stable against perturbations or a change of variables.

This is illustrated by the following exampl~

due to Kreiss [1963], I

8tu

= U(t) /. \0

U(t)

I)

U-'(t)

8xU

9

I

t-cos t sin

o

It is easy to see that the eigenvalues of the symbol, for all t, lie on the ima~na~y axis.

The equation is however far from well posed.

dependent variables by introducing v(t) = U -I (t)u(t).

To see this we change the This gives us a system with

constant coefficients (~

1

Sty = I after some calculations.

a v x

0

-I

I

0

-

v ,

The eigenvalues of its symbol equal

is ~ J ~ ( 1

+ is)

and the Petrowskii condition is therefore violated. In itself there is nothing wrong with Hadamard's definition. much more convenient to base the theory on the other definition.

It is however We will soon see

that an addition of a zero order term, which is essentially what happens in our example above, will not change a problem well posed in L

into an ill posed one. P

It is possible, by present day techniques, to answer questions on admissab!e perturbations for certain problems, even with variable coefficients for which a loss of derivatives as in Hadamard's definition is unavoidable. of this nature is the so called weakly hyperbolic equations. physical interest.

A class of problems

Some of them

are of

These questions are very difficult and we therefore conclude

that if there is a chance, possibly by a change of variables, to get a problem which is well posed in L

we should take it. P

133

One of the main conclusions of this long story is that we have to live with L 2 norms in the first order case. unbounded in L

co

It is well known that an L 2 function might be

and the error bounds in L 2 which we will derive shortly might there-

fore look quite pointless.

At the end of this series of talks we will however see

than an assumption of some extra smoothness of the solution of the differential equation will enable us to get quite satisfactory bounds in the maximum norm as well. In this section we have seen examples of the use of conservation laws; the energy was conserved for the wave equation. derivation of the heat equation.

Similar considerations went into the

There is an on going controversy if the discrete

models necessarily should be made to satisfy one or more laws of this kind.

First

of all, it is of course not always possible to build in all conservation laws into a difference them.

scheme because the differential

equation might have an infinite number of

Secondly a difference should be made between problems which have sufficiently

smooth solutions and those which do not.

In the latter case the fulfilment of the

most important conservation laws often seems an almost necessary requirement especially in nor~inear problems.

When we have smooth solutions we are however

frequently better off choosing from a wider class of schemes. soon to be developed,

The error bounds,

give quite good information on convergence, etc., and it

might even be argued that the accuracy of a scheme, not designed to fulfil a certain conservation law, might conveniently be checked during the course of a computation by calculating the appropriate quantity during the calculation.

6. Stability. error bounds and a perturbation theore m As in the case of a linear differential equation we can introduce a solution operator Eh(nk,n+k), 0 < nl ~ n, for any finite difference scheme.

It is the

mapping of the approximate solution at t = nlk into the one at t = nko

For explicit

schemes the solution operator is just a product of the particular difference operators on the various time levels. Let us write a one step implicit scheme symbolically as

(I + Q_,) u(x,t+k) = (I + kQo) u(x,t)

134 where Qe and Q-I are difference operators, and assume that (I + kQ. I)'I exists and is uniformly bounded in the norm to be considered. Eh(t+k,t) = (I + ~ . , ) "

Then

(I + kQo)

and there is no difficulty to write up a formula for Eh(nk,nlk). A simple device enables us to write multistep schemes as one step systems.

We

illustrate this by changing the second difference scheme of Section 3 into this form. Introducing the vector variables

v(x,t) =

/u(x't+k) ) u(x,t)

/

the difference scheme takes the form

v(x,t+k)

2kD_D+

I

I

0

=

v(x,t)

.

The same device works for any multistep scheme and also when we have a system of difference equations.

When we speak about the solution operator for a multistep

scheme we will always mean the solution operator of the corresponding one step s y s ~ . We will now introduce our stability definitions.

Stability is nothing but the

proper finite difference analogue of well posedness. Definition

A finite difference scheme is stable in L

if there exist constants P

and C such that l~(nk,nlk)fllp, h ~ Cexp(~(nk-n,k)) llfllp,h

o

The finite difference schemes are defined at mesh points only. use a discrete L

Therefore we

norm in this context defined by P

"U"p,h = I ~

hs 'u(x)'P?I/p

.

x~ h It should be stressed, that for each individual mesh size, all our operators are bounded.

Therefore the non trivial feature about the definitions is that the

constants C and ~ are independent of the mesh sizes as well as the initial values. Frequently we will use another stability definition°

135 Definition

A scheme is strongly stable with respect to a norm [II. If Ip,h ,

uniformly equivalent to li.llp,h, if there exists a constant ~ such that

I J l~h((n+1)k,~)el I Ip,h '~ (l~k) 1t ~elllp,h • We recall that ll.llp,h and III-II Ip,h are uniformly equivalent norms if there exists a constant C > O, independent of h, such that

(1/c ) llfllp, h g l llflllp, h ~ CIIIfl[Ip, h for all f ~ Lp, h. It is easy to verify that a scheme strongly stable with respect to some norm is stable.

The strong stability reflects an effort to control the growth of the solutie~

on a local level.

Note that we have already established that certain finite diffe-

rence approximations to the heat equation are strongly stable with respect to the maximum norm.

Our proof that the L 2 norm of any solution of a symmetric first order

system has a limited growth rate gives hope that certain difference schemes for such problems will turn out to be strongly stable with respect to the L2, h norm. In many oases we will however be forced to choose a norm different from llQllp,h in order to assure strong stability.

For a discussion of this difficult subject we

refer to Kreiss [1962] and Richtm~er an~ ~orton [1967]. For any stable scheme, the coefficients of which do not defend on n, there ex~ts a norm with

respect to which the scheme is strongly stable.

This can be shown by

the following trick which the author learned from Vidar Thom~e. The fact that the coefficients of the difference schemes do not depend on time makes Eh(nk,n,k ) a function of n-n, only.

We can therefore write it as Eh(nk-n,k)o

Introduce

IIlelllp,h = ~8 lle'~lk~(~)fllP,h " It is easy to show that this is a normo stability and a choice of i = O,

llfllp, h • lllflllp,h ~ cllfllp, h

It is equivalent to ll.ilp,h because by

136

Our difference scheme is clearly strongly stable because,

[I [Eh(k)fl lip, h {u~ lie -~lk Eh(lk)Eh(k)f[lp, h = su lie -~lk =

eak IIl~lllp,h

Eh((l+l

)k)f[[p, h

•

We could consider using a weaker stability definition.

A closer study gives th~

following analogae of the Hadamard condition. Definition

A finite difference scheme is weakly stable in L

if there exis% P

constants ~, C and p such that

IIEh(nk,n,k)f[Ip, h ( C(n-n,+l )P exp(~(nk-n,k)) I[£1p, h o A theory based on this definition would however suffer from the same weakness as one based on the Hadam~rd definition of well posedness. cf. Kreiss [1962] or Richtmyer and Morton [1967].

For a detailed discuss~n

It is also clear that, in general~

we will stay closer to the laws of physics if we choose to work with the stronger stability definit ionso This far we have only dealt with homogenous problems. genous case is however quite simple.

Going over to the inhcmo-

We demonstrate this for an explicit scheme

u(x,t÷k) = u(x,t) + kQou(x,t) + ~ ( t ) ,

u(x,O) = f(x) Using the solution operator we get n

u(x,nk) = ~(m<,O) f ( x ) + k

~

Eh(nk,vk) F((v-1 )k) .

v--1 If the scheme is stable in L

we get P

Itu(n~)llp, h ~

C(exp(c~nk)

n

Ilfllp, h + k ~ e (~(nk-vk) ×t~[O,nk]maxiIF(t)llp, h) . is.v. !

Now

n k ~e

~(1-e~k)/(1-eak), if ~ i a(nk-vk) qnktf~

=0.

o

137

Notice that this is really essentially a matter of computing compounded interests on

an

original capital llflland periodic savings liF(t)ll. The formalism is known

as Duhamel's principle. Its most common application is to error bounds for finite difference schemes. Put the solution of the differential equation into the difference scheme.

As was

pointed out before we will then get an extra inhomogenous term~ the local truncatian error, of the form kT(x,nk,h).

Introduce the error which is the difference between

the approximate and the exact solution.

Subtract the two difference equations.

The error will then satisfy the same difference equation with the truncation error as an inhomogenous term. gence in L

It is easy to see from our estimate that we have conver-

for 0 g t g T if P

max llT(nk,h)II goes uniformly to zero with the Ognk4T p,h

mesh size and that we have a rate of convergence h r if r(nk,h) = O(h r) uniformly. We recall that w can be computed using just Taylor series. We now turn to the theorem on perturbation mentioned in the previous section. For simplicity we give a proof only for time independent coefficients. Theorem

Consider a finite difference scheme

u(x,(n+1 )k) : Q u(x,~) stable in Lp, statisfying Ilu(nk)llp,h < C exp(amk) llu(O)llp,h . Let Qf be an operator, uniformly bounded in Lp, h.

Then solutions of

vCx,(n+1)k) : (Q+kQf) v(x,~) satisfy

llv(~k)llp, h ~ C e~(~) with

llv(o)llp, h

= ~ + C e-~k ilQ'llp,h .

Proo f

Consider the 2 v terms in the development of (e "~k Q +

e-~

kQ')W,

j of ('I

of these consist of j factors ke-~kQ ' and there at most (j+1) factors of the form (e -~k Q ) ~

~ some natural numbers.

The norm of suck a term is by our assumption bounded by k j C j+1 llQ'ilj e -~kj.

138

Thus

j=O

C(1 + kCe -ak I]Q'll) v ~ C e yvk

where y = C e - a k ]IQ'I] .

Thus we have verified a finite difference version of our perturbation theorem. A differential equation theorem can now be derived by taking, for any particular

differential equations a stable difference scheme, applying the theorem just proved and taking a limit by letting h and k go to zero. We have not shown that a stable scheme always can be found for any well posed differential equation but that is in fact the case.

Also notice that there is

never any chance to derive stronger inequalities for a finite difference scheme thm~ those which hold for the corresponding differential equation.

The most we can hope

is to get exact analogues of what is true in the continuous case. We leave it to the reader to give another proof of our latest theorem using the norm III. If Ip,h which we constructed in our proof that any stable scheme is strongly stable at least with respect to one norm.

7. The yon Netmmnn condition, dissipative and multistep schemes. In this section we will use Fourier series to derive stability conditions and also introduce a number of useful schemes for hyperbolic equations. We first consider the periodic problem atu = 8xU, t ) O, u(x,O) = f(x) = ~ x + ~ If f(x) is sufficiently smooth it can be developed in a convergent Fourier series

f(x) --

%

eiVX

o

M=--GO The solution of the equation takes the form +oo u(x,t) = ~ , % V=--O0

ei~(x+t)

139

The Euler difference scheme vCx,t+k) = vCx,t) + kDo vCx,t) , can be studied in the same way.

v(~,,e) :

where

%

v(x,O) = fCx)

Its solution is

(I + Ix sin ,, h) n e

ivx

k = k/h. In contrast to the differential equation case the amplitude of Fourier com-

ponents will grow,, This is so becuase 11 + ik sin ~ h] = J 1 + k = sin 2 v h > I for v ~ O.

From our discussion of the Courant-Friedrichs-Lewy condition we know that

k = I would be ideal and we see that choosing k equal to a constant will lead to very rapidly growing high frequency components.

It is easy to show that for many

very smooth initial value functions this very strong amplification of high frequm~y components will lead to arbitrary large and wildly divergent approximate solutions. The amplification of the lowest frequency modes is however not very large. This might lead us to the following idea.

Replace the initial value function by a

fixed partial sum of its Fourier series.

If the initial data is sufficiently smooth

we can do this changing the values of the initial value function and the solution by an arbitrarily small amount.

If we use the new initial value for the finite

difference scheme we can see, from the explicit solution formula , that the disc1~te solution will converge to the correct one when h goes to zero=

In fact the same

argument shows that we could proceed with a k larger than I because for any constant k (I + iX sin

vh) n will converge to ei~kn for any fixed value of v.

This approach however suffers from the same weakness as a theory for differential equations based on analytic functions only.

In fact a finite Fourier series

represents an analytic function and much of Hadamard's criticism of the CauchyKowaleski type theory carries over to the finite difference case. To see that something is drastically wrong with our argument above we urge the reader to carry out a few steps with the Euler method using k = 10 and an initial value function which is ~ at one mesh point and zero elsewhere.

The rapid growth

of this special solution will assure us of a totally unacceptable growth of round off errors.

One could say that the error of measurements which played an important

140

part in Hadamard's argument are replaced by the round off errors.

From the error

bound in section 6 we see that we will not be seriously affected by round off errors if a difference scheme is stable.

This is thus a reason, perhaps the most impor-

tant one, why we insist on using only stable schemes for computations. There is a simple remedy for the lack of stability of the Euler scheme namely the addition of a so called dissipation term.

The term corresponds to a finite

difference approximation of yet another term in the Taylor expansion of u(x,t+k) with respect to t.

This way we get the Lax-Wendroff scheme.

v(x,t+k) = v(x,t) + kDov(x,t) + k~/2 D_D.v(x,t)

,

v(x,O) = f(x) .

The coefficient of e ivx is now amplified by a factor i + iX sin ~h - (k2/2) sin~(~h/2)

in each step.

It is elementary to verify that this factor is less than or equal to one in absolute value for k < i.

This clearly ensures strong L 2 stability.

From this, convergence

follows as well as a relative insensitivity to round off errors. We have now developed a simple technical tool which allows us to decide the qualities of all the schemes suggested for the heat equation.

The results will he

reveal ed shortly. The dissipation of the Lax-Wendroff scheme acts to damp out the higher frequency modes of the solution and this is frequently just as well because they must contain rather serious phase errors.

For sufficiently smooth solutions, which means quickly

decreasing Fourier coefficients with increasing v, these modes play a vezyunimportant part in the representation of the solution.

Similar considerations frequently

make very much sense in cases when we do not have constant coefficients.

Heuristi-

cally we can argue that the variability of the coefficients will make various Fourier modes interact in a way which is very hard to analyse. expect serious phase errors for high modes.

However we can

Not only will these components be in

error but they will interact with other components in a totally erroneous way. such a case it seems advisable to damp out such modes in the discrete model.

In

141

Sometimes model.

we however are quite anxious to have an energ~ preserving discrete

This is for instance the case when we have to calculate over long periods of

time and with only very weak forcing functions.

One simple scheme which preserves

the energy for the case atu = 8xU is the leap frog scheme also known as the mid-point rule when it is used for ordinary differential equations,

v(x,t+k) = ~(x,t-k) + 2~. v(x,t) Another one is the Crank-Nicolson scheme (I - k/2 Do) v(x,t+k) : (I + k/2 Do) v(x,t) . Fourier analysis shows that the amplification per step for the Crank-Nicolson scheme is (I + i k/2 sin vh)/(1 - i k/2 sin vh)

and thus that the amplitude is preserved.

For the leap frog scheme we look for solutions of the form v(x,nk) = ~,t eiVX and get, by the solution of a quadratic equation, the two roots ~,,~ J1 - k 2 sin 2 vh

which, for k ~ I, both lie on the unit circle.

= ik sin vh + The multistep

character is reflected in the existence of two independent solutions. A similar analysis for the backward scheme shows that ['I _ ikI sin vh I 4 1 which again implies strong stability. We now turn to aFourier analysis of the schemes suggested for the heat equation. Using k for k/h 2 we find that the amplification factor for Euler's method is I - )J+ sin 2 (vh/2), thus it is stable for k ~ ½. (I

- 2k sin 2

(vh/2))/(1 +

For the Crank-Nicolson scheme:

2k sin 2 (vh/2)); unconditionally stable°

scheme I/(I + 4k sin 2 (wh/2)), unconditionally stable. Ansatz

of the form " ~ e ivx leads to ~ b L =

For the backward

For the mid-point rule an

-4k sin~(vh/2) + ~I + (&k sin2(wh/2))~

which shows that the scheme is unstable for any constant ko

In a similar way we

could also show that the Dufort-Frankel scheme is unconditionally stableG Another interesting method, of fourth order accuracy in time, is Milne's method

(i - y~ kQ) v(x,t+k) = (~ + yI kQ) v(x,t k) + y4 kQv(x, t) where Q stands for Do or D÷D.. are (i kq

_+ J1 +

(kq)"

/ (I

The roots of the corresponding quadratic equation -

i kq) where q stands for i sin vh or -4 sin2~h/2 o

142

Thus the method

preserves energy for the hyperbolic case, provided k ~ ~ ,

an& is

violently unstable for the parabolic case. The instability of the Milne and mid-point methods is closely related to the well known weak stability of these methods when applied to ordinary differenial equations. Cf. Dahlquist K1956], [1963].

In fact parabolic equations are very stiff

equations and weakly stable schemes are therefore quite useless. We are now well prepared for the following definition. Definition

Consider a linear finite difference scheme u.+~ = Qun

and define its symbol by

~)

= e~(

i ~ ~, x , )

Q e~(i

< ~, x 7) .

The difference scheme satisfies the yon Neumann condition if the spectral radius of its symbol is bounded by e ~k for some constant ~. The von Ne,,m~nu condition is the finite difference analogue of the Petrowskii condition.

It is a necessary condition for stability°

It ~as shown by Kreiss

[1962]

to be equivalent to weak stability in L 2 in the case of constant coefficients.

It

is not a sufficient stability condition for problems with variable coefficients for it suffers from the same inadequacies at the Petrowskii condition.

Cf. Kreiss [1962].

8. Semibounde~ operators We now formulate an abstract condition on the differential operator P and its boundary conditions in order to assure well posedness.

We will also see that we

can use the same argument to prove stability for finite difference schemes.

We

begin by forming, using the L 2 inner product I u v dx,

8t(u,u) = (u, Pu) + (Pu, u) = 2Re(Pu, u) o

Definition

An operator P, with its boundary conditions, is semibounded if Re(Pu, u) ~ Const. llull 2

for all sufficiently smooth functions u satisfying the boundary conditions°

143

For a semibounded operator we clearly get the a priori inequality llu(t)ll, ~ exp(Const, t) llu(O)ll~ and the problem is well posed in L 2 if solutions exist. follows from the inequality.

In order to assure existence we have to be sure that

we do not have too many boundary conditions. types of equations it ~

Uniqueness Immediately

Cf. the discussion in §2.

For certain

known that a solution exists if the problem is minlmally

semibounded. Definition

An operator P, with its boundary conditions, is minimally semibounded

if P is semibounded and this property is lost by removing any linearly independent boundary conditions. In periodic cases it is easy to verify that the following expressions are semibounded: A~ x + @xA

A hermitian (the constant = 0);

@

A real, symmetric and positive definite;

A8

x

i@

x

,

X

A@

@x Aax

,

X

'

A hermitian (the constant = O); A skew hermitian (the constant = O)~

A sum of such expression is also semibounded. restricted to only one space variable.

We note that we are in no way

We leave to the reader to verify which of

our examples in §2 are semibounded for periodic cases. To illustrate the theory we consider the heat equation on 0 ~ x ~ I with the boundary conditiens Sou(O) + ~oSxU(O ) = O and u~u(1 ) + ~axU(1 ) = O, ui' ~i real. The corresponding periodic problem is easy to treat because 8t(u(t), u(t)) = 2Re(u, 8~u) = - 2118xUll2 ~ O . No term comes from the boundary because of the periodicity° For the actual case we get 2

8t(u(t), u(t)) = -2Hakull' ÷ 2ReSx;(1)u(1) - 2Reax~(0)u(O ).

This illustrates the necessity of boundary conditions°

The first expression is

144

negative but the L 2 norm of 8 u cannot be used to give a bound for a u at a partix x cular point.

Without the boundary conditions we could therefore not conclude that

the right hand side is bounded by ccnst, llull~ for all u.

Using the boundary eondi-

tions we see that

2o~(~).0 if ~o and ~I ~ 0. ~161 ) 0

= - 2 ~, luO )1 ~ + 2 ~o I~(o)1

) - 2au(o)u(o)

Our first observation is that we clearly have semiboundedmess if

and ~oBe ~ 0.

We now ask the question if we can do without such conditions.

In fact we can because the pointwise

value of a function can, in one dimension, be

estimated by the L 2 norms of the functions and its first derivatives. simple special case of a Sobolev inequality. following calculations :

To get this inequality carry out the x

xm~X lu(x)l 2 - ~ n

This is a

min

lu(x)l 2 = 2Re f USxU dx x

max

where Xmax,(Xmi n) is a value for which the maximum,

(minimum) of lu(x)l 2 is obtained.

A standard inequality shows that x

I 2

max

f

uau~

I

1 ~ 2

X mln ,

/

[uaxul ~ ~ c, la, ull +li/,lluli" ..2

X

2

"

0

Therefore ~.=

x

lu(x)l" ~ ~lla. ull" + I/~ llull" + ~x x 2 2

lu(x)l'

but I

~

tu(~)l • <. [ lu(x)l"

~ = Ilull ~

and thus

max iu(x)l'

~ ~ll~=~ll~ + (1 + 1/~) llull" 2

All we have to do to use this result for our heat equation is to make a choice of a sufficiently small ~. The argument can also be carried out when ~o or ~i, or both are zero. We have used an L 2 norm instead of a maximum norm in this study of the heat equation.

At this point mathematical

convenience has thus been allowed to dominate

145

over our physical intuition. It is obvious that we can prove well posedness in L 2 if we can find any norm, equivalent to the L 2 norm, with respect to which an operator is semiboundedo of the big problems with the

One

energy method is often the difficulty of finding an

appropriate norm for which an operator is semibounded.

The failure to show that a

certain problem, which satisfies the necessary Petrowskii condition, is semibounded can therefore depend on that the problem is ill posed but also on a lack of expertise on our part in dealing with Sobolev inequalities or that we

have chosen the wrong

inner product or even the incorrect dependent variables. We will now indicate how we can use these ideas to construct difference expressions which satisfy similar bounds.

This in fact will provide us with a most

useful guide line for the construction of stable schemes.

It is thus appropriate

to replace an expression of the form A(x)8 x + 8xA(X), A hermitian, by A(x)Do + DoA(x).

In the periodic case this will give us an antisymmetric

for the differential operator. even better, by D_A(x+h/2)D÷.

operator just as

Similarly 8xA(X)8 x could be replaced by DoA(x)Do or Higher order accuracy can also be obtained.

Thus

Do - h2/6 DoD_D÷ is a more accurate approximation to 8 x but it still preserves the crucial quality of being an antisymmetrie operator.

The proof that these difference

operators have the required properties follows easily from sun~ation by parts.

When

the boundary conditions are more involved it is advisable to start by carrying out detailed analysis of the boundary terms for the differential equations to assure semiboundedness and then try to pick sufficiently accurate difference analogues to the boundary conditions to ensure a preservation of semiboundedness. For a very good presentation of these matters we refer to Kreiss [1963].

Cf.

also Richtmyer and Morton [1967].

9. Some application~ of the energy method The purpose of this section is to use the building blocks from §7 and §8, the semibounded difference operators and the stable multistep scheme, in order to construct stable schemes for a wide class of problems.

146

We first consider the backward scheme (I - kQ)u~÷1

= u.

where we assume that Re(Qu. u) ~ C llull2. Take the scalar product of both sides with u.÷1 and use a series of simple inequalities •

liu,,+,ll Jlu,,ll +, ( u . + , , =

2 ll,+,+,.,+,Ii

u,,,) -- ( u . + , ,

- k:(,.+.,,,.,.,, Qu,,+,)

u.+,

~ llu,+.,lJ"

- ~:Q u . + , ) - c k 11 ll..u,+,,. +' .

Therefore. if C k < I

I

flu.lJ

and the scheme is L 2 stable, for small enough k. This calculation also shows that (I - kQ) always has a bounded inverse in L 2 for k sufficiently small if Q is semibounded, a fact most important for all implicit schemes. We now consider the Crank-Nicolscn scheme (I - ~/2Q)un+,

= (I + k/2Q)u.

Rewrite it as ~++

- u. = k/2Q(u.++

+ u.)

and take the scalar product with un+ I + u.. llu.++ll2 - llu.ll2

The left han~ side comes out to be

and the right hand side is less than k/2 C llun÷1 + u.lia, which is

less than or equal to k C (flu.÷ill2 + IlunIl2)o Thus

llu,+,,li" < (I + kc)/(1

-kc)llu,.,ll"

°

Of great interest is the study of schemes of the form (I - kQ, )un+,

~QoU.

+ (I + kQ, )u._,

+

The leap frog and Milne's methods have this form and if Qo = 0 it re@uces to the Crank-Nicolson scheme+

For Qo and QI we want to choose semibounded difference

operators such as those arising from a proper II

.

discretization of the spatial opera,re

from a hyperbolic, parabolic. Schrodlnger equation or from any other problem which

147

is semibounded in L 2.

Frequently we also find both first order antisymmetric

operators as well as even order ones with symmetric in the same problem.

positive definite coefficients

Where should we put these different pieces?

We know from a previous discussion that the mid-point rule is bad for stiff problems such as parabolic ones.

W e should therefore avoid making Qo elliptic.

We

therefore require that Qo is antisymmetric or (Qou,v) = - (u,Qov) for all u and v. This is equivalent, for an operator with real coefficients, to require that (Qeu, u) = 0.

W e note that QI could also contain antisymmetric parts such as in

Milne's method for 8tu = 8xU. bounded.

For the operator QI, we only require that it is semi-

To simplify our arguments we assume that (Q1u, u) ~ O.

We want to establish the L 2 stability of this class of schemes. step scheme the norm (lJUnll2 + ljUn÷lil=)½ suggests itself.

Being a two

However we will find that

Ln = UUn÷111~ + IIunl[2- 2k(Qoun, Un÷1) will be the appropriate expression in the sense that L n is equivalent to the one first suggested and is a non increasing function of n.

We have to put a restriction on Qo namely kilQoll ~ I - 6, 8 a constant > 0 in

order to make Ln strictly positive.

We will see by an example that this is a most

natural restriction° We first show that Ln ~ Ln-1. u.+l

- un-I

:kQt(u.÷,

Rewrite the equation as + u.-t)

+ 2~oU.

and take the scalar product with Un÷t + Un-1.

ilun÷,ll~- llu~_,il~ = (u.÷, + u._,, kQ,(u.~,

Then

+ Un-,) + 2k(un÷, + u._,, Qou~).

The first term on the right hand side is less than or equal to zero because of one of our assumptions.

Rearranging and adding llUnll2 on both sides we get Ln ~ Ln-1 o

To show that Ln is positive and equivalent to the natural L 2 norm we start by observing that

12k(QoUn÷~, u,)l ~ 2(1-8)IlUn÷tll

llu.II ~

(1-6) (llun+,ll2 + Ilu, ll~).

Therefore

6(llu.÷,ll 2 + llu.il2) ~ Ln ~ (2-6) (Ilun+~lJ 2 + ljUnIl2).

148

To see that kIIQell ~ S - 6 is a natural condition consider the case Qo = Do and QI = Oo

This Qe has, as is easily verified, an L 2 norm equal to I/h.

Thus the

restriction just means k/h ~ i - 8, essentially the Courant-Friedrichs-Lewy

condit~n.

This is a natural condition in terms of Qo alone because in the case QI = 0 the method is explicito For a more general discussion and a comparison of the growth rates of the exact and approximate solutions we refer to Johansson and Kreiss [1963]. Schemes of Dufort-Frankel

type can be discussed in very much the same way°

We will now show that these ideas can be used to design stable and efficient schemes, so called alternating direction implicit schemes, for certain two dimensional equations.

We suppose that our problem has the form 8tu = P,u + P2u

and that the operators and the boundary conditions are such that PI and P2 are semibounded.

For simplicity we assume that

Re(u, Pju) ~ o ,

j = 1,2,

and that we have finite difference approximations Re(u, Qju) ~ 0 ,

Qj to Pj, j = I ,2, such that

j = 1,2.

We will consider the following two schemes

(I - kQ,)(I - kQ2)un÷,

= u.

and (I - k / 2 Q I )(I - k/2Q~)un+,

= (I + k/2Q, )(I + k/2Qa)un .

These schemes are particularly convenient if QI and Qz are one dimensional finite difference operators.

In that case we only have to invert one dimensional operators

of the form (I - akQi) and this frequently leads to considerable

savings.

This

becomes clear if we compare the work involved in solving a two dimensional heat equation~ using an alternating direction implicit method with QI = D-zD+:, Q2 = D_~D+9, and the application of the standard backward or Crank-Nicolson scheme with Q = D_zD÷: + D_gD+~©

The former approach only involves solutions of linear

systems of tridiagonal type while the other, in general, requires more work.

149

The L 2 stability o f the first scheme is very simple to prove. i = 1,2, both have inverses the

L 2 norms of which are bounded by I.

stability of the other scheme is more involved.

Y.÷t

Thus I - k Qi' The proof of the

Let

= (1 - k/2Q2)un÷ 1

and z n = (1 4,, k / 2 Q , ) u n

•

Then (1 - k/2Qt )Yn+1 = (1 + k/2Qt ) z . or

y.+,

- :. : k/2 Q,(y..,

+ z.)

o

Forming the inner product with Yn+, + z,, just as in the proof of the stability of the Crank-Nicolseu

method, we get

Ity..,tl"

- II.,,11" = k / 2 R e ( Q , ( y , + t

liy,,,,ll'

=

+ Z n ) , Yr,+, + zn) ~ O.

Now

Ilu,,÷,ll"

- w ' 2 Re(Q.un.,.,, u . + , ) + k 2 / 4

llQ, u,,.,ll'

and

1t.,,ti ~ = Ilu,,ll ~ + k/2 ~e(Q~u,,,

u,,) + z'/~

IlQ~u,,ll ~

.

Therefore~ because Re(Q2u , u) ~ O,

li~..,ti ~ , k~/~

ilQ~u,,,.,II ~ ,~ itu.II ~ + k~/4

liQ~u, II ~ ,,

It is easy to see that this implies L 2 stability if kQ2 is a bounded operator.

If

kQ 2 is not bounded we instead get stability with respect to a stronger norm, a result which serves our purpose equally well. We refer to an interesting paper by Strang [1968] for the construction of other stable accurate methods, based on one dimensional operators@

lO. Maximum norm convergence for L 2 stable schemes

In this section we will explain a result by Strang [1960] whi~'h shows that solutions of L 2 stable schemes of a certain accuracy converge

in maximum norm with

the same rate of convergence as in L 2 provided the solution of the differential equation is sufficiently smooth.

150

Let u . . , = Qu. ,

Uo(X)

-- f ( x )

be a finite difference approximation to a linear problem, 8tu = ~

,

u(x,O) = f(x)

,

well posed in L 2. To simplify matters we assume that the two problems are periodic. assume that we have an L 2 stable scheme.

It is known that if f is a sufficiently

smooth function the solution will also be quite smooth.

We now attempt to establish

the existence of an asymptotic error expansion of the error° u, ix) : u ( x , ~ )

+ hrer(x,~)

We also

+ hr''er.,(x,nk)

where we choose r as the rate of convergence

in L2o

+

Make the Ansatz

....

If we substitute this

expression into the difference equation we find that the appropriate choice for er, er+1, are solutions of equation of the form 8te j = Pej + Lju

ej(O)

= o

where Lj are differential operators which appear in the formal expansion of the truncation error.

The solutiormof a finite number of these equations are under our

assumptions quite smooth. To end our discussion we have to verify that ~n,N(X,h) = u,(x) - u(x,nk) - ~ h

1%(x,nk)

l=r is O(h r ) in the maximum norm for some finite N, i.e. that hre error term.

r

is indeed the leading

This is done by a slight modification of the error estimate of §6@

derive a difference equation for eL, N and find that its L 2 norm is O(h r+N+1 ). assumption we have a periodic problem.

We By

The maximum norm of a mesh function is there-

fore bounded by h -s/2 times its L2, h norm over a period, where s is the number of space dimensions.

This concludes our proof.

151

We remark that an almost identical argument shows that we can relax our stab4_li~ requirements and require only weak stability (Cf. §6) and still get the same results for sufficiently smooth solutions. REFERENCES Brenner, P.; 1966, Math. Scand., V.19, 27-37. Courant, R., Friedrichs, K. and Lewy, H.; 1928, Math. Annal., V.100, 32-7~ also; 1967, IBM J. of Research and Development, V.ii, 213-247. Dahlqulst, G.; 1956, Math. Scand., V.~, 33-53. Dahlqulst, G.; 1963, Prec. Sympes. Appl. Math., V.15, i~7-158. Friedman, A.; 196~, Partial Differential Equations of Parabolic Type. Prentice-Hall. Garabedian, P. ; 1964, Partial Differentiml Equations.

Wiley.

Gelfand, I.M., Shilev, G.E.~ 1967, Generalized Functions, V.3 Academic Press. Haaam~rd, J.; 1921, Lectures on C a u c ~ equations.

problem in linear partial differential

Yale University Press.

Jehanssen, ~, Kreiss, H.0.; 1963, BIT, V.3, 97-107. John, F.; 1952, Comm. Pure. Appl. Math., V.5, 155-211. Kreiss, H.0.; 1959, Math. Scand., V.7 71-80. Kreiss, H.O.; 1962, BIT, V.2, 153-181 Kreiss, H.0.; 1963, Math. Scand., V.13, 109-128. Kreiss, H.0o; 1963, Numer. Math., V.5, 27-77. Krelss, H.0., Widlund, 0.; 1967, Report, Computer Science Department, Uppsala, S ~ Lax, P.D.; 1957, Duke Math. J., V.24 Lax, P.D. ; 1963, Lectures on hyperbolic partial differential equations, Stanford University (lecture notes). Littman, W.; 1963, J. Math. Mech., V.12, 55-68. Richtm2er , R.D.; 1957, Difference methods for initial-value problems. Wiley Interscience. Richtmyer, R.D., Morton, K.W.; 1967, Difference methods for initial-value problems. 2nd Edition Wiley Interscience.

152

Strang, W.G.; 1960, Duke Math. J., V.27, 221-231o Strang, W.Go; 1966, J. Diff. Eqo, V.2, 107-114. Strang, W.G.! 1968, SWAM J. Numer. Anal., V.5, 506-617. Thomele, V.; 1962, J. SIAM, V.lO, 229-245. Thomee, V.; 1969, SIAM Review, V.11, 152-195.