NUMERICAL MATHEMATICS AND SCIENTIFIC COMPUTATION Series Editors G. H. GOLUB A. M. STUART ¨ E. SULI
NUMERICAL MATHEMAT...
48 downloads
1041 Views
11MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
NUMERICAL MATHEMATICS AND SCIENTIFIC COMPUTATION Series Editors G. H. GOLUB A. M. STUART ¨ E. SULI
NUMERICAL MATHEMATICS AND SCIENTIFIC COMPUTATION
Books in the series Monographs marked with an asterix (∗ ) appeared in the series ‘Monographs in Numerical Analysis’ which has been folded into, and is continued by, the current series. For a full list of titles please visit http://www.oup.co.uk/academic/science/maths/series/nmsc ∗
J. H. Wilkinson: The algebraic eigenvalue problem I. Duff, A. Erisman, and J. Reid: Direct methods for sparse matrices ∗ M. J. Baines: Moving finite elements ∗ J. D. Pryce: Numerical solution of Sturm–Liouville problems Ch. Schwab: p- and hp- finite element methods: theory and applications to solid and fluid mechanics J. W. Jerome: Modelling and computation for applications in mathematics, science, and engineering Alfio Quarteroni and Alberto Valli: Domain decomposition methods for partial differential equations G. E. Karniadakis and S. J. Sherwin: Spectral/hp element methods for CFD I. Babuˇska and T. Strouboulis: The finite element method and its reliability B. Mohammadi and O. Pironneau: Applied shape optimization for fluids S. Succi: The Lattice Boltzmann Equation for fluid dynamics and beyond P. Monk: Finite element methods for Maxwell’s equations A. Bellen & M. Zennaro: Numerical methods for delay differential equations J. Modersitzki: Numerical methods for image registration M. Feistauer, J. Felcman, and I. Straˇskraba: Mathematical and computational methods for compressible flow W. Gautschi: Orthogonal polynomials: computation and approximation M. K. Ng: Iterative methods for Toeplitz systems Michael Metcalf, John Reid, and Malcolm Cohen: Fortran 95/2003 explained George Em Karniadakis and Spencer Sherwin: Spectral/hp element methods for CFD, second edition Dario A. Bini, Guy Latouche, and Beatrice Meini: Numerical methods for structured Markov chains Howard Elman, David Silvester, and Andy Wathen: Finite elements and fast iterative solvers: with applications in incompressible fluid dynamics Moody Chu and Gene Golub: Inverse eigenvalue problems: Theory and applications Jean-Fr´ed´eric Gerbeau, Claude Le Bris, and Tony Leli`evre: Mathematical methods for the magnetohydrodynamics of liquid metals Gr´egoire Allaire: Numerical Analysis and Optimization Eric Canc`es, Claude Le Bris, Yvon Maday, and Gabriel Turinici: An Introduction to Mathematical Modelling and Numerical Simulation Karsten Urban: Wavelet Methods for Elliptic Partial Differential Equations ∗
Wavelet Methods for Elliptic Partial Differential Equations Karsten Urban University of Ulm Institute of Numerical Mathematics
1
3
Great Clarendon Street, Oxford OX2 6DP Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide in Oxford New York Auckland Cape Town Dar es Salaam Hong Kong Karachi Kuala Lumpur Madrid Melbourne Mexico City Nairobi New Delhi Shanghai Taipei Toronto With offices in Argentina Austria Brazil Chile Czech Republic France Greece Guatemala Hungary Italy Japan Poland Portugal Singapore South Korea Switzerland Thailand Turkey Ukraine Vietnam Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries Published in the United States by Oxford University Press Inc., New York c Oxford University Press 2009 The moral rights of the author have been asserted Database right Oxford University Press (maker) First published 2009 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, or under terms agreed with the appropriate reprographics rights organization. Enquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above You must not circulate this book in any other binding or cover and you must impose this same condition on any acquirer British Library Cataloguing in Publication Data Data available Library of Congress Cataloging in Publication Data Data available Typeset by Newgen Imaging Systems (P) Ltd., Chennai, India Printed in Great Britain on acid-free paper by CPI Antony Rowe, Chippenham, Wiltshire ISBN 978–0–19–852605–6 1 3 5 7 9 10 8 6 4 2
F¨ ur Almut
This page intentionally left blank
CONTENTS List of Algorithms Preface Acknowledgements List of Figures List of Tables
xii xiii xv xvii xxvi
1
Introduction 1.1 Some aspects of the history of wavelets 1.2 The scope of this book 1.3 Outline
2
Multiscale approximation and multiresolution 2.1 The Haar system 2.1.1 Projection by interpolation 2.1.2 Orthogonal projection 2.2 Piecewise linear systems 2.3 Similar properties 2.3.1 Stability 2.3.2 Refinement relation 2.3.3 Multiresolution 2.3.4 Locality 2.4 Multiresolution analysis on the real line 2.4.1 The scaling function 2.4.2 When does a mask define a refinable function? 2.4.3 Consequences of the refinability 2.5 Daubechies orthonormal scaling functions 2.6 B-splines 2.6.1 Centralized B-splines 2.7 Dual scaling functions associated to B-splines 2.8 Multilevel projectors 2.9 Approximation properties 2.9.1 A general framework 2.9.2 Stability properties 2.9.3 Error estimates 2.10 Plotting scaling functions 2.10.1 Subdivision 2.10.2 Cascade algorithm 2.11 Periodization 2.12 Exercises and programs
vii
1 1 4 6 9 9 9 12 17 24 24 25 26 27 28 29 30 31 35 37 38 39 43 46 46 48 49 50 51 55 57 58
viii
3
CONTENTS
Elliptic boundary value problems 3.1 A model problem 3.1.1 Variational formulation 3.1.2 Existence and uniqueness 3.2 Variational formulation 3.2.1 Operators associated by the bilinear form 3.2.2 Reduction to homogeneous boundary conditions 3.2.3 Stability 3.3 Regularity theory 3.4 Galerkin methods 3.4.1 Discretization 3.4.2 Stability 3.4.3 Error estimates 3.4.4 L2 -estimates 3.4.5 Numerical solution 3.5 Exercises and programs
63 63 64 68 70 74 75 75 76 77 77 78 78 82 83 85
4
Multiresolution Galerkin methods 4.1 Multiscale discretization 4.2 Multiresolution multiscale discretization 4.2.1 Piecewise linear multiresolution 4.2.2 Periodic boundary value problems 4.2.3 Common properties 4.3 Error estimates 4.4 Some numerical examples 4.5 Setup of the algebraic system 4.5.1 Refinable integrals 4.5.2 The right-hand side 4.5.3 Quadrature 4.6 The BPX preconditioner 4.7 MultiGrid 4.8 Numerical examples for the model problem 4.9 Exercises and programs
87 87 89 89 90 94 95 96 97 98 101 101 102 104 108 115
5
Wavelets 5.1 Detail spaces 5.1.1 Updating 5.1.2 The Haar system again 5.2 Orthogonal wavelets 5.2.1 Multilevel decomposition 5.2.2 The construction of wavelets 5.2.3 Wavelet projectors 5.3 Biorthogonal wavelets 5.3.1 Biorthogonal complement spaces
120 120 120 121 124 124 126 129 131 132
CONTENTS
5.4
5.5 5.6
5.7
5.8
5.3.2 Biorthogonal projectors 5.3.3 Biorthogonal B-spline wavelets Fast Wavelet Transform (FWT) 5.4.1 Decomposition 5.4.2 Reconstruction 5.4.3 Efficiency 5.4.4 A general framework Vanishing moments and compression Norm equivalences 5.6.1 Jackson inequality 5.6.2 Bernstein inequality 5.6.3 A characterization theorem Other kinds of wavelets 5.7.1 Interpolatory wavelets 5.7.2 Semiorthogonal wavelets 5.7.3 Noncompactly supported wavelets 5.7.4 Multiwavelets 5.7.5 Frames 5.7.6 Curvelets Exercises and programs
ix
133 134 134 141 143 145 145 147 150 151 152 152 156 157 161 163 164 166 166 167
6
Wavelet-Galerkin methods 6.1 Wavelet preconditioning 6.2 The role of the FWT 6.3 Numerical examples for the model problem 6.3.1 Rate of convergence 6.3.2 Compression 6.4 Exercises and programs
170 170 175 177 177 181 184
7
Adaptive wavelet methods 7.1 Adaptive approximation of functions 7.1.1 Best N -term approximation 7.1.2 The size and decay of the wavelet coefficients 7.2 A posteriori error estimates and adaptivity 7.2.1 A posteriori error estimates 7.2.2 Ad hoc refinement strategies 7.3 Infinite-dimensional iterations 7.4 An equivalent 2 problem: Using wavelets 7.5 Compressible matrices 7.5.1 Numerical realization of APPLY 7.5.2 Numerical experiments for APPLY
186 186 187 190 191 192 194 200 203 205 215 216
x
8
CONTENTS
7.6 Approximate iterations 7.6.1 Adaptive Wavelet-Richardson method 7.6.2 Adaptive scheme with inner iteration 7.6.3 Optimality 7.7 Quantitative efficiency 7.7.1 Quantitative aspects of the efficiency 7.7.2 An efficient modified scheme: Ad hoc strategy revisited 7.8 Nonlinear problems 7.8.1 Nonlinear variational problems 7.8.2 The DSX algorithm 7.8.3 Prediction 7.8.4 Reconstruction 7.8.5 Quasi-interpolation 7.8.6 Decomposition 7.9 Exercises and programs
221 221 224 226 232 232 234 238 239 241 243 246 249 252 254
Wavelets on general domains 8.1 Multiresolution on the interval 8.1.1 Refinement matrices 8.1.2 Boundary scaling functions 8.1.3 Biorthogonal multiresolution 8.1.4 Refinement matrices 8.1.5 Boundary conditions 8.1.6 Symmetry 8.2 Wavelets on the interval 8.2.1 Stable completion 8.2.2 Spline-wavelets on the interval 8.2.3 Further examples 8.2.4 Dirichlet boundary conditions 8.2.5 Quantitative aspects 8.2.6 Other constructions on the interval 8.2.7 Software for wavelets on the interval 8.2.8 Numerical experiments 8.3 Tensor product wavelets 8.4 The Wavelet Element Method (WEM) 8.4.1 Matching in 1D 8.4.2 The setting in arbitrary dimension 8.4.3 The WEM in the two-dimensional case 8.4.4 Trivariate matched wavelets 8.4.5 Software for the WEM 8.5 Embedding methods 8.6 Exercises and programs
257 258 260 262 271 284 287 291 292 292 297 304 307 323 325 327 328 335 342 344 357 372 386 387 390 391
CONTENTS
9
xi
Some applications 9.1 Elliptic problems on bounded domains 9.1.1 Numerical realization of the WEM 9.1.2 Model problem on the L-shaped domain 9.2 More complicated domains 9.2.1 Influence of the mapping – A non-rectangular domain. 9.2.2 Influence of the matching 9.2.3 Comparison of the different adaptive methods 9.3 Saddle point problems 9.3.1 The standard Galerkin discretization: The LBB condition 9.3.2 An equivalent 2 problem 9.3.3 The adaptive wavelet method: Convergence without LBB 9.4 The Stokes problem 9.4.1 Formulation 9.4.2 Discretization 9.4.3 B–spline wavelets and the exact application of the divergence 9.4.4 Bounded domains 9.4.5 The divergence operator 9.4.6 Compressibility of A and B T 9.4.7 Numerical experiments 9.4.8 Rate of convergence 9.5 Exercises and programs
394 394 394 395 401 401 403 406 407
A
Sobolev spaces and variational formulations A.1 Weak derivatives and sobolev spaces with integer order A.2 Sobolev spaces with fractional order A.3 Sobolev spaces with negative order A.4 Variational formulations A.5 Regularity theory
437 437 443 445 446 452
B
Besov spaces B.1 Sobolev and Besov embedding B.2 Convergence of approximation schemes
455 457 459
C
Basic iterations
462
References Index
409 410 412 417 417 418 419 423 425 427 428 429 434
465 477
LIST OF ALGORITHMS 2.1 4.1 4.2 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 7.9 7.10 7.11 7.12 7.13 9.1 9.2 C.1 C.2 C.3 C.4
Cascade algorithms BPX applied to vj = m∈Ij αj,m ϕj,m : wj = BPX[αj , j, t, Φ] MultiGrid-cycle on level j: y j = M G[j, xj , µ, ν1 , ν2 ] wN := APPLY [A, v N , ε] Adaptive Wavelet-Richardson scheme Adaptive Wavelet-Richardson scheme with inner Richardson iteration Thresholding: THRESH RICHRADSON: Adaptive Wavelet-Richardson with thresholding Adaptive wavelet method for linear problems ELLSOLVE: Adaptive wavelet solver for linear elliptic problems General iterative solver Simplified adaptive wavelet solver Core nonlinear evaluation algorithms Prediction Reconstruction to local scaling function representation Local decomposition Adaptive UZAWA method Adaptive divergence operator: [w, Λ] = DIV[v] Conjugate gradient method (CG) Preconditioned conjugate gradient method (PCG) Gauss–Seidel iteration Jacobi scheme
xii
56 104 106 216 221 225 228 228 230 231 237 238 243 246 248 253 413 426 462 463 463 464
PREFACE Wavelets have become a powerful tool in signal and image processing for more than a decade now. Their use for the numerical solution of operator equations has been investigated more recently. By now the theoretical understanding of such methods is quite advanced and has brought up deep results and additional understanding. Moreover, the rigorous theoretical foundation of wavelet bases has also led to new insights into more classical numerical methods for partial differential equations (PDEs) such as finite elements. However, wavelet methods have still only partly reached ‘realistic’ applications in science and technology. This is partly explained by the fact that understanding and applying the full power of wavelets needs some mathematical background in functional analysis and approximation theory. Moreover, many descriptions of wavelet methods are technically complicated which also somehow restricts the group of possible users. The main idea of this book is to introduce the main concepts and results of wavelet methods for solving linear elliptic partial differential equations in a framework that allows us to avoid technicalities to the maximum extent. It should be stressed that this does not mean that the application of wavelet methods for partial differential equations is restricted only to rather academic problems as they are presented in most parts of the book. Also not all technicalities can completely be avoided. We try to find a balance between an easy-to-read introduction and an abstract mathematical description. Several references point towards more detailed descriptions. Wavelets, and in particular wavelet methods for the numerical solution of partial differential and integral equations, are still a highly active field of research. This means in particular that in the current state a textbook cannot be complete and it also cannot aim at describing the most recent state of the art in research. This holds in particular for the chapters on adaptive wavelet methods and on applications. Just recently, several new variants of adaptive wavelet methods have been introduced with different levels of improvement over the existing schemes. We are completely aware of the fact that those adaptive methods that are presented in this book are subject to optimization and improvements. However, we have chosen this selection since we found them most appropriate for an easy introduction. We also hope that this introduction enables the reader also to understand more recent improvements. A similar remark also concerns the treatment of nonlinear methods. Here, we concentrate on a general description of a possible algorithm to evaluate nonlinear functions without going into all details of an algorithm. This book is based upon several lectures that I gave at the RWTH Aachen (Germany), the MRI Master Class “Scientific Computing” at the University of xiii
xiv
P R E FAC E
Utrecht (Netherlands), several summer schools and at the University of Ulm (Germany). It is addressed to readers with just a basic background in numerical methods for linear and nonlinear systems of equations and with a basic knowledge in analysis and linear algebra corresponding to undergraduate level. The book should also be readable by graduate students in physics, computer science, and engineering. Finally, I would like to address all colleagues that are working in the field of wavelet methods in numerical simulation. Many people have been contributing to this field over many years. Certainly, I cannot judge the importance of certain contributions. I tried to the best of my knowledge to quote all contributions that seemed to me relevant for the scope of this book. The probability that I missed some papers, however, is high. If this is the case with one or more of your papers, please accept my apology. It is certainly not your fault, but my ignorance. If you give me a note I will consider it in any new edition (if there is to be one).
ACKNOWLEDGEMENTS Several parts of this book are based upon joint work of the author with several colleagues and friends. I am grateful for their cooperation over several years. I would like to mention (in alphabetical order) Cem Albukrek, Arne Barinka, Kai Bittner, Titus Edelhofer (name at birth Barsch), Stefano Berrone, Silvia Bertoluzza, Claudio Canuto, Philippe Charton, Albert Cohen, Stephan Dahlke, Wolfgang Dahmen, Reinhard Hochmuth, Torben Klint, Angela Kunoth, John Lumley, Wolfgang Marquardt, Dietmar Rempfer, Anita Tabacco, J¨ urgen Vorloeper, and R¨ udiger von Watzdorf. This book is partly based upon a lecture that I gave within the MRI Master Class “Scientific Computing” at the University of Utrecht (Netherlands) which was organized by Rob Stevenson. I am grateful for that opportunity and Rob’s support, especially but not only, in that period. Since 1992 I have been working in the field of wavelet methods for the numerical solution of partial differential equations. Wolfgang Dahmen has been the supervisor of my PhD and continued to be my scientific teacher until I left the RWTH Aachen in 2002. I am deeply grateful for his guidance and for so many inspiring discussions with him over many years. My scientific career has also been influenced by my research visit to Italy. My colleagues Claudio Canuto, Anita Tabacco, and Stefano Berrone have become dear friends; I am very grateful for their support and friendship. Since 2002 I have been working at the University of Ulm, in a very nice environment in the Faculty of Mathematics and Economy and in particular in the Institute of Numerical Mathematics. I thank all colleagues; I enjoy coming to work day by day. Some members of our Institute have particularly contributed to this book, namely Stefan Funken, Michael Lehn, Kai Bittner, Mario Rometsch, and Timo Tonn. Most of the numerical experiments have been done by Alexander Stippler and Christian Pape which I acknowledge a lot. Petra Hildebrand typed most parts of the manuscript in LATEX and sometimes had a heavy fight with my handwriting and all the different figures I asked her to produce. I am particularly grateful to Stefano Berrone, Claudio Canuto, Wolfgang Dahmen, Rob Stevenson, and Anita Tabacco for various helpful comments on previous versions of the manuscript. These comments definitely improved the quality of the book. Finally, I thank Oxford University Press, in particular Alison Jones, for her assistance and her patience. I know that this book should have been finished much earlier. I am grateful for the possibility to publish this book at Oxford University Press.
xv
xvi
AC K N OW L E D G E M E N T S
Last, but definitely not least, I thank my family, my wife Almut and my children Tobias, Niklas, and Annika. This book was also partly written during many evening and night sessions at home, the last part even during vacation. Everybody who ever wrote a book knows that the family has to take most of the burden of such a project. Ulm, October 2007 Karsten Urban
LIST OF FIGURES 2.1 2.2
2.3
2.4 2.5
2.6
2.7 2.8
2.9 2.10 2.11 2.12 2.13 2.14
Scaled and shifted version ϕ[j,k] of ϕ := χ[0,1) for the choices (j, k) ∈ {(0, 0), (1, 6), (2, 8), (−1, 2)}. PjI f and error f − PjI f for the function f (x) := sin(2πx). The first row contains PjI f , j = 0, 1, 2, the second row the corresponding errors. In the third row, we show PjI f , j = 3, 4, 5 and the corresponding errors in the fourth row, and so on. PjI f and error f − PjI f for the function fJ defined in (2.9). The first row contains PjI f , j = 3, 4, 5, the second row the corresponding errors. In the third row, we show PjI f , j = 6, 7, 8 and the corresponding errors in the fourth row, and so on. Convergence history for f (x) := sin(2πx) [sin] and fJ [hat] for the supremum [sup] and Euclidean [euc] norm. PjO f and error f − PjO f for the function f (x) := sin(2πx). The first row contains PjO f , j = 0, 1, 2, the second row the corresponding errors. In the third row, we show PjO f , j = 3, 4, 5 and the corresponding errors in the fourth row, and so on. PjO f and error f − PjO f for the function fJ defined in (2.9). The first row contains PjO f , j = 3, 4, 5, the second row the corresponding errors. In the third row, we show PjO f , j = 6, 7, 8 and the corresponding errors in the fourth row, and so on. Convergence history for f (x) := sin(2πx) [sin] and fJ [hat] for the supremum [sup] and Euclidean [euc] norm. Example of the subdivision scheme using the mask of the Daubechies scaling functions for N = 2 (Table 2.5, page 32). In order to highlight the results of the scheme, we have drawn vertical lines from the computed point value to the horizontal axis. Daubechies orthonormal scaling functions for N = 1, . . . , 8 (first row 1, . . . , 4, second row 5, . . . , 8 from left to right). Cardinal B-splines of order d = 1, . . . , 4. Cardinal B-spline of order d = 1 (top) and dual functions of orders d˜ = 1, 3, 5 (bottom, from left to right). Cardinal B-spline of order d = 2 (top) and dual functions of orders d˜ = 2, 4, 6 (bottom, from left to right). Cardinal B-spline of order d = 3 (top) and dual functions of orders d˜ = 3, 5, 7 (bottom, from left to right). Cardinal B-spline of order d = 4 (left) and dual function of order d˜ = 4, 6, 8 (bottom, from left to right).
xvii
10
12
14 16
19
20 21
33 36 37 41 41 42 42
xviii
2.15 2.16 2.17
4.1 4.2
4.3 4.4
4.5 4.6 4.7 4.8 4.9
4.10 4.11
4.12
5.1 5.2
LIST OF FIGURES
Example d = d˜ = 2, where the largest eigenvalue is not simple. Wrong solution (left) and correct function (right). Examples d = d˜ = 3, d = d˜ = 4 and d = 4, d˜ = 6 (from left to right). First row: results with subdivision, second row with cascade. Periodization of a compactly supported function f . Original function f (left), its 1-periodic extension (middle) and periodization [f ]0,0 . Sparsity pattern of the stiffness matrix for B-splines of orders 2 (left) and 3 (right) both for j = 5. Exact solutions ui , i = 1, 2, 3, for the three test cases. Sin function (left), power function (middle) and solution corresponding to the Delta distribution (right). V -cycle (left) and W -cycle (right) for three and four levels. Iterations of the MGM without preconditioning for the case d = 2. On the left, the (square) norm of the residual is plotted over the number of iterations, on the right, the term (x(k) )T Ax(k) (recall that the solution is x ≡ 0. Iterations of the MGM without preconditioning for the cases d = 2 (top), d = 3 (middle) and d = 4 (bottom). Iterations of the MGM with diagonal preconditioning for the cases d = 2 (top), d = 3 (middle) and d = 4 (bottom). Iterations of the BPX preconditioned CG method for the cases d = 2 (top), d = 3 (middle) and d = 4 (bottom). Iterations of the MultiGrid method for the cases d = 2 (top), d = 3 (middle) and d = 4 (bottom). Iterations of the BPX preconditioned CG method using Daubechies scaling function with N = 3, . . . , 8 (3, 4 top; 5, 6 middle; 7, 8 bottom, from left to right). Iterations of the BPX preconditioned CG method for dual scaling ˜ namely d˜ = 4, 6, 8, 10. functions with d = 2 and different values of d, Iterations of the BPX preconditioned CG method for dual scaling function. First row: d = 3, left: d˜= 7, right: d˜= 9, second row d = 4, d˜= 10. Iterations of the BPX preconditioned CG method. Comparison of primal, dual and Daubechies functions. First row, primal functions for d = 4, second duals with d = 2, d˜ = 4 and third row Daubechies with N = 4. Haar scaling function (left) and Haar wavelet (right). Given piecewise constant approximation (solid) to a given function (top), approximation by a coarse scale part (middle) and complement (bottom).
55 56
58 94
97 105
111 112 113 113 114
116 117
117
118 121
123
LIST OF FIGURES
5.3
5.4 5.5 5.6 5.7 5.8 5.9 5.10
5.11
5.12 5.13 5.14
5.15 5.16 5.17 5.18 6.1
6.2 6.3
6.4
Considering f (x) := sin(2πx) each row shows Pj f (first column), the error f − Pj f (second column), the detail Qj f (third column) and the same detail with respect to an adjusted vertical axis for the levels j = 1 (top row) to 8 (bottom). Daubechies orthonormal wavelets for N = 2, 3 (top) and N = 4, 5 (bottom), both from left to right. Biorthogonal B-spline wavelets for d = 1 and d˜ = 1, 3, 5, 7. Biorthogonal B-spline wavelets for d = 2 and d˜ = 2, 4, 6, 8. Biorthogonal B-spline wavelets for d = 3 and d˜ = 1, 3, 5, 7. Biorthogonal B-spline wavelets for d = 4 and d˜ = 2, 4, 6, 8. Biorthogonal B-spline wavelets for d = 5 and d˜ = 0, 1, 3, 5. Biorthogonal scaling functions and wavelets for d = 6 and d˜ = 8. First row: Cardinal B-spline (left), dual scaling function (right). Second row: primal wavelet (left) and dual wavelet (right). Biorthogonal scaling functions and wavelets for d = 9 and d˜ = 7. First row: Primal (left) and dual scaling function (right). Second row: primal wavelet (left) and dual wavelet (right). Wavelet decomposition of an input vector cj into its detail parts and a coarse scale part. Reconstruction or Inverse Wavelet Transform. Wavelet parts Qj f of the pyramid function defined in (5.23) (also shown in the top row) for the levels j = 1, . . . , 8 using biorthogonal B-spline wavelets of order d = 3 and d˜ = 3. Piecewise linear interpolatory scaling system (left) and wavelet system (right). Deslauriers-Dubuc functions for N = 3 (left) and N = 5 (right). Semiorthogonal wavelets, linear, quadratic and cubic (from left to right). Cubic (left) and quadratic Hermite splines. Sparsity pattern of the stiffness matrix with respect to biorthogonal B-spline wavelets for d = d˜ = 2 (left) and d = d˜ = 3 (right) both for j = 7. Wavelet diagonal scaling preconditioned CG method for d = 2 (top), d = 3 (middle) and d = 4 (bottom). Comparison of the wavelet diagonal scaling preconditioned CG method with the BPX preconditioned CG method for c = 0.1 and d = 2 (top), d = 3 (middle) and d = 4 (bottom). CPU times over error for Wavelet-Galerkin with standard and diagonal preconditioning compared with BPX. The horizontal axis k corresponds to an error of 10−k (left). The right picture shows the CPU time over the level j also including MultiGrid.
xix
125 130 135 136 137 138 139
140
141 143 144
149 159 160 162 165
175 179
180
181
xx
LIST OF FIGURES
6.5
Wavelet coefficients of f (1) (left), f2 (middle), f4 (right) in the first row and corresponding solutions (second row) for j = 10, d = d˜ = 2. (2) (2) Wavelet coefficients of f (1) (left), f2 (middle), f4 (right) in the first row and corresponding solutions (second row) for j = 10, d = 3, d˜ = 5. Function f defined in (7.1) (left) and size of wavelet coefficients (right). Slope of the best N -term approximation. Visualization of all nonzero wavelet coefficients of the function f in (7.1). Form of a classical adaptive method with a posteriori error estimation and (de-)refinement. Note that the finite-dimensional problem on Xh usually cannot be solved exactly. Index cone for an index λ = (j, k). Exact solution (left) and right-hand side f (right) corresponding to Example 7.1. Exact solution corresponding to Example 7.3. Structures of wavelet coefficients of an adaptive scheme based upon Strategy 1 (left) and Strategy 2 (right). Error decay of the ad hoc refinement strategy using Strategy 2 for the refinement. Computed final approximations of the ad hoc refinement strategy for Example 7.1 using ε = 0.1 (left), ε = 0.01 (center) and ε = 0.001 (right). The slope of error reduction in the adaptive approximate operator application APPLY using the data of the periodic boundary value problem for Example 7.1 (left) and 7.3 (right). Convergence history for the adaptive Wavelet-Richardson scheme in Algorithm 7.2 for Examples 7.1 (left) and 7.3 (right). We denote d = 2, d˜= 4 by “+”, d = d˜= 3 by “◦” and d = d˜= 4 by “∗”. The solid line in the picture on the right shows the error of the best N -term approximation for d = 2, d˜= 4. Convergence history for the adaptive Wavelet-Richardson scheme with inner iteration in Algorithm 7.3 for Examples 7.1 (left) and 7.3 (right). We denote d = 2, d˜ = 4 by “+”, d = d˜ = 3 by “◦” and d = d˜ = 4 by “∗”. Coefficients of the iterations 1, 5, 10, 15, 20, sorted with respect to their absolute values. Convergence history of Algorithm 7.7 for Example 7.1 using biorthogonal B-spline wavelets corresponding to d = d˜ = 3.
6.6
7.1 7.2 7.3 7.4
7.5 7.6 7.7 7.8 7.9 7.10
7.11
7.12
7.13
7.14 7.15
(2)
(2)
183
183 187 189 191
193 195 198 198 199 200
200
221
224
225 227 232
LIST OF FIGURES
7.16
7.17 7.18 7.19 8.1
8.2 8.3
8.4 8.5 8.6
8.7
8.8 8.9
8.10
8.11
8.12 8.13 8.14
Numerical approximations (left) and corresponding errors (right) for the adaptive scheme in Algorithm 7.7 for Example 7.1 using biorthogonal B-spline wavelets corresponding to d = d˜= 3. We show the iterations n = 1, 3, 5, 10, 30. Convergence history before and after the thresholding step. The two indices on the left have a tree structure, whereas the two on the right do not. Multiscale covering of Ω = [0, 1] by the index set Γ = {(1, 0), (2, 3), (3, 4), (4, 10), (5, 23), (5, 24)}. Road map for the Wavelet Element Method (WEM). Starting from wavelets on the real line R, construct corresponding bases on [0, 1], ˆ mapping to Ωm and then finally on Ω then by tensor product on Ω, by domain decomposition and matching. Cut-off from ϕ[j,k] resulting in a small piece ϕ[j,k]|I . The vertical dashed line is at the point x = 0. Support of overlapping functions for j = 2 with support size 12 (top) and 34 (bottom). We obtain five overlapping functions with support size 12 and 6 with support size 34 . Graphical visualization of the refinement relation (8.2). ˇT . Structure of the refinement matrix M j,0 Primal modified single-scale functions ϕL 5,k near the left boundary ˜ for d = 3 and d = 5, k = 1, . . . , 3. The polynomial reproduction can be seen near x = 0. Modified dual single-scale functions ϕ˜L 5,k near the left boundary for ˜ d = 3 and d = 5, k = 1, . . . , 5. The polynomial reproduction can be seen near x = 0. Index sets for the interval for d = 3, d˜ = 5, = 4, ˜ = ˜2 = 6. Dual modified dual single-scale functions ϕ˜L 5,k near the left boundary for d = 3 and d˜ = 5, k = 1, . . . , 5 before biorthogonalization. Dual scaling functions ϕ˜5,k , k = 1, . . . , 5 for d = 3, d˜= 5 using Ej = IIj in the biorthogonalization. This means that the primal functions remain unchanged and correspond to Figure 8.6. ˜ j,0 to square Completion of the rectangular matrices Mj,0 , M ˜ ˜ ˜ matrices Mj = (Mj,0 , Mj,1 ), Mj = (Mj,1 , Mj,0 ) that are both ˜ j = II ×I . sparse, invertible, and MjT M j+1 j+1 ˜ Primal wavelets for d = 3, d = 5 at the left boundary. Dual wavelets for d = 3, d˜ = 5 at the left boundary. Primal scaling functions d = 1, d˜ = 3. Only the boundary functions on the left boundary are displayed.
xxi
233 234 244 245
258 259
260 261 271
272
273 275
288
289
294 305 306 306
xxii
LIST OF FIGURES
8.15
Dual scaling functions d = 1, d˜ = 3, before biorthogonalization (left) and after (right). Only the boundary functions on the left boundary are displayed. Primal wavelets d = 1, d˜ = 3, at the left (left) and right (right) boundary. Dual wavelets d = 1, d˜= 3, at the left (left) and right (right) boundary. Biorthogonal system corresponding to d = 1, d˜= 5, primal (first column), dual (second column) scaling functions as well as primal (third column) and dual (fourth column) wavelets. Biorthogonal system corresponding to d = 2, d˜= 2, primal (first column), dual (second column) scaling functions as well as primal (third column) and dual (fourth column) wavelets. Biorthogonal system corresponding to d = 2, d˜= 4, primal (first column), dual (second column) scaling functions as well as primal (third column) and dual (fourth column) wavelets. Biorthogonal system corresponding to d = 2, d˜= 6, primal (first column), dual (second column) scaling functions as well as primal (third column) and dual (fourth column) wavelets. Biorthogonal system corresponding to d = 3, d˜= 3, primal (first column), dual (second column) scaling functions as well as primal (third column) and dual (fourth column) wavelets. Biorthogonal system corresponding to d = 3, d˜= 5, primal (first column), dual (second column) scaling functions as well as primal (third column) and dual (fourth column) wavelets. Biorthogonal system corresponding to d = 3, d˜= 7, primal (first column), dual (second column) scaling functions as well as primal (third column) and dual (fourth column) wavelets. Biorthogonal system corresponding to d = 3, d˜= 9, primal (first column), dual (second column) scaling functions as well as primal (third column) and dual (fourth column) wavelets. Biorthogonal system corresponding to d = 4, d˜= 4, primal (first column), dual (second column) scaling functions as well as primal (third column) and dual (fourth column) wavelets. Biorthogonal system corresponding to d = 4, d˜= 6, primal (first column), dual (second column) scaling functions as well as primal (third column) and dual (fourth column) wavelets. Biorthogonal system corresponding to d = 4, d˜= 8, primal (first column), dual (second column) scaling functions as well as primal (third column) and dual (fourth column) wavelets. Grid points τj,i associated to scaling functions (•) and νj,i associated to wavelets (). Primal single scale functions ϕ5,k induced by Bernstein polynomials without biorthogonalization.
8.16 8.17 8.18
8.19
8.20
8.21
8.22
8.23
8.24
8.25
8.26
8.27
8.28
8.29 8.30
307 308 309
310
311
312
313
314
315
316
317
318
319
320 321 325
8.31 8.32 8.33 8.34 8.35 8.36 8.37 8.38 8.39 8.40 8.41 8.42 8.43 8.44 8.45
8.46 8.47 8.48 8.49 8.50
8.51 8.52 8.53 8.54
8.55
LIST OF FIGURES
xxiii
Dual single scale functions ϕ˜5,k induced by Bernstein polynomials. Piecewise linear wavelets on the interval constructed by Grivet Talocia and Tabacco. Semiorthogonal piecewise linear and quadratic spline scaling functions and spline wavelets. The slope of error reduction in the adaptive approximate operator application. Error of the adaptive algorithm and the best N -term approximation for the first 1D example. The first, third, fourth and sixth approximate solution for Example 8.19. The sets of active indices Λ0 , Λ2 , Λ3 and Λ5 for Example 8.19 The exact solution and the right-hand side for Example 8.20. Comparison between best N -term approximation and adaptive algorithm for Example 8.20. The first, third, fourth and sixth approximate solution and the differences to the exact solution for Example 8.20. The sets of active indices Λ1 , Λ3 , Λ4 and Λ6 for the Example 8.20. Decomposition of an image for one level. Visualization of 2D wavelet coefficients. Decomposition of Ω = (−1, 1) into two subdomains Ω− and Ω+ . Primal scaling functions for d = 2, d˜ = 4 according to Example 8.23. The second function is already an interior one, namely the standard hat function. Boundary dual scaling functions according to Example 8.23 for d = 2, d˜ = 4. Primal boundary wavelets according to Example 8.23, d = 2, d˜ = 4. Dual boundary wavelets according to Example 8.23, d = 2, d˜ = 4. Matched primal and dual scaling functions for the Example 8.23 at the cross-point. Scaling function grid points (bullets) and wavelet grid points (squares). The nonfilled boxes correspond to wavelets that need to be defined as matched functions. Matched primal (top row) and dual wavelets (bottom row) for Example 8.25 at the cross-point. One-dimensional curve in 2D consisting of four pieces. Mapping of grid points labeling scaling functions (bullet) and wavelet (box) from the reference domain to the subdomain. Domain with Dirichlet (solid) and Neumann (dashed) boundary subdivided into six subdomains. The outer boundary of the subdomains needs to coincide with either ΓNeu or ΓDir . Two-dimensional face Γi,i between two subdomains Ωi and Ωi . A one-dimensional face would be one of the two corners of Γi,i .
326 326 327 329 331 332 333 333 334 336 337 340 340 345
346 347 347 348 349
349 354 355 356
358 359
xxiv
LIST OF FIGURES
8.56
ˆ (left), coarse part Sj (Ω) ˆ Set of grid points associated to Sj+1 (Ω) ˆ (middle), and complement Wj (Ω) (right). Scaling function grid points (circles) and wavelet grid points (boxes) in the subdomain Ωi (here for simplicity rectangular). The upper and right parts of the boundary of the subdomain belong to Γ. The filled wavelet grid points are associated to wavelets on Ω constructed according to (8.147). Different 2D matching situations, common side of two subdomains (left), interior cross-point (center) and boundary cross-point (right). Bullets denote scaling function, boxes wavelet grid points. Nonfilled symbols indicate that the corresponding functions are missing depending on the kind of boundary conditions (Dirichlet or Neumann). Matching along an interface of two subdomains. Primal (top row) and dual basis functions (bottom row) defined by (8.166) and (8.167). The first primal wavelet (top left) is only support in I− , the third primal wavelet (top right) is only supported in I+ . Three subdomains meeting at an interior cross-point C. Tensor product matching around a cross-point common to four subdomains. On right (top) the 1D matching without scaling function is displayed; the picture on the right (bottom) is the usual 1D matching. These two are tensorized to obtain the functions on the left around the cross-point. Grid points hC,i (labeled only by i) around a boundary cross-point in the three different cases (from left to right: pure Neumann, mixed and pure Dirichlet case). Note that the cross-point C does only correspond to a scaling function in the pure Neumann case. Construction of a polyhedron corresponding to a cross-point. Left: a cross-point common to three subdomains. Right: definition of the polyhedron PC (solid lines). Matching around a boundary cross-point in 3D. Squared domain with cut-out ellipse. The grid line show the behavior of the parametric mappings. Embedding of a complicated Ω into a simple . Singular solution (9.4), left, of the Poisson problem on the L-shaped domain with corresponding smooth right-hand side f (right). Comparison of numerical approximation and best N -term approximation (solid lines). First iterations of the adaptive wavelet method for the case of a smooth right-hand side. Peak-shaped right-hand side.
8.57
8.58
8.59 8.60
8.61 8.62
8.63
8.64
8.65 8.66 8.67 9.1
9.2 9.3 9.4
361
365
373 373
376 377
383
384
387 388 389 391
396 397 398 399
LIST OF FIGURES
9.5 9.6 9.7 9.8 9.9 9.10
9.11
9.12
9.13
9.14
9.15
9.16
9.17
A.1 B.1 B.2
A Few adaptive iterations for the example of a singular right-hand side. Squared domain with ellipse hole. Solution (left) and contour lines (right) of the solution. Curved quadrilateral domain for the tests in Section 9.2.2. Matrices Dj,0 and Dj,1 for j = 5 and d = d˜ = 3. Scaling functions 3,3 ϕ4,2 , 3,3 ϕ4,3 , 3,3 ϕ4,4 (top) and their exactly computed derivatives being linear combinations of 2,4 ϕ4,k , for k = 0, 1, 2 (bottom). Scaling functions 3,3 ψ4,1 , 3,3 ψ4,2 , 3,3 ψ4,4 (top) and their exactly computed derivatives being linear combinations of 2,4 ψ4,k for k = 0, 1, 2 (bottom). Exact solution for the first example. Velocity components (left and middle) and pressure (right). The pressure functions exhibits a strong singularity and is only shown up to r = 0.001 in polar coordinates. Exact solution for the second example (II) and also for the experiment in Section 9.4.8.3, first and second component of the velocity (left and center) and pressure (right). First, fourth, sixth and eight approximations for Example (II). First and second velocity component (left and middle column) and pressure (right column). Relative error versus number of unknowns for spline wavelets of different order for the discretization of the pressure in the second example. Numerical approximations for the experiment in Section 9.4.8.3 using UZAWAexact for the first velocity component (left) and the pressure (right) for the iterations i = 1, 2, 3, 6. Comparison of the algorithm UZAWAexact with the exact evaluation of the divergence [exa] with UZAWAapprox [app] using APPLY. First velocity component (left) and the pressure (right) are displayed. Circle with cut corresponding to the lower right quadrant. DeVore diagram with Sobolev embedding line. DeVore diagram for convergence rates of linear and nonlinear methods.
xxv
400 402 403 404 425
426
426
428
429
431
432
435
436 447 458 460
LIST OF TABLES 2.1 2.2 2.3 2.4 2.5 2.6 2.7 4.1 4.2 4.3 4.4
4.5 4.6 4.7
4.8 5.1 5.2
Approximation error for f (x) := sin(2πx) (f − PjI f ) for the supremum and Euclidean norm. 13 Approximation error (fJ − PjI fJ ) for fJ in (2.9) for the supremum and Euclidean norm. 15 Approximation error for f (x) := sin(2πx) (f − PjO f ) for the supremum and Euclidean norm. 17 Approximation error for fJ for the supremum and Euclidean norm. 18 Refinement coefficients of Daubechies scaling functions N ϕ for N = 1, . . . , 10. 32 Refinement coefficients of B-spline scaling functions and dual ˜ scaling functions for d = 1, 2 and some values of d. 43 Refinement coefficients of B-spline scaling functions and dual ˜ scaling functions for d = 3, 4 and some values of d. 44 for homogeneous Condition numbers for the stiffness matrix Ahat j Dirichlet boundary conditions for different values of c and j. 91 Condition numbers for the stiffness matrix Ahat in the periodic j case for different values of c and j. For c = 0, the matrix is singular. 92 L2 -errors and convergence factors in the three different cases. 97 H 1 -errors and convergence factors for u1 as solution for the homogeneous Dirchilet problem (3.2) and the periodic boundary value problem (3.1). 98 Condition numbers of the stiffness matrix Aj for the case d = 2 and different values of c and j. 108 Factor between condition numbers of level j and j − 1 (d = 2). 109 Condition numbers of stiffness matrix Aj depending on d for fixed level j = 12. The last column labeled ‘∞’ contains the condition number of the mass matrix (without derivatives). 109 Comparison of CPU times for BPX and MultiGrid using c = 0.1, d = 3 (and d˜= 5 for the restriction). 115 Sizes of the supports of semiorthogonal and biorthogonal spline wavelets. 163 Condition numbers for scaling functions and wavelets of Chui and [0,1] R Wang (ρR Φ and ρΨ ) for L2 (R), and of Chui and Quak (ρΦj and [0,1]
6.1
ρΨj ) for L2 ([0, 1]) and j ≤ 11 in dependence of the spline order d. For comparison, we also display corresponding numbers for biorthogonal spline wavelets on R from [71] and on [0, 1] from [35]. 163 Condition numbers of wavelet-preconditioned stiffness matrix depending on d = d˜= 2 for different values of c and j and factor between condition numbers of level j and j − 1. 178 xxvi
L I S T O F TA B L E S
6.2 6.3 7.1 7.2
7.3 8.1 8.2 8.3 9.1 9.2 9.3 9.4 9.5 9.6
9.7 9.8
9.9
xxvii
Condition numbers of wavelet-preconditioned (diagonal scaling) ˜ stiffness matrix depending on d and d. Comparison of CPU times for d = 3, d˜ = 5 and c = 0.1. Sizes of the active wavelet sets, interior residual (on Λ) and global error. Expected and observed slopes for the error of the adaptive approximate operator application in the periodic case for Example 7.1 (left) and 7.3 (right). Coefficients γd, . ˜ Minimal level depending on d and d. Expected and observed slopes for the error of the adaptive approximate operator application. Sobolev and Besov norms of the exact solution (8.107) to Example 8.20. L∞ -errors computed on a dyadic grid with meshsize h = 2−11 . Rate of convergence Comparision of one (left) and two (right) subdomains of the same domain Ω. Comparison of Algorithm 7.9 (left) and Algorithm 7.6 (right). Results for Example (I). Numbers of adaptively generated degrees of freedom, ratio to best N -term approximation and relative errors. Results for Example (II). Numbers of adaptively generated degrees of freedom, ratio to best N -term approximation and relative error. Minimal level j0 and number of scaling functions NΦ on the minimal level for different order discretizations. Results for the second example with piecewise linear trial functions for velocity and pressure. Note that in this case the number of degrees of freedom for the coarsest level is 243. Numerical results for the experiment in Section 9.4.8.3 corresponding to UZAWAexact . Number of active coefficients, ratio of the error of the numerical approximation and the best N -term approximation and relative error for the first velocity component and the pressure. The results for the second velocity components are similar.
178 182 199
220 251 324 330 334 403 403 405 406 430
430 433
433
434
This page intentionally left blank
1 INTRODUCTION 1.1
Some aspects of the history of wavelets
The origin of wavelets goes back to the beginning of the last century. In fact, it was Alfred Haar in 1910 [121, 122] that constructed the first wavelet, even though he did not call it that. Many years later, wavelets arose in Fourier analysis as an alternative to the standard Fourier transform. Yves Meyer published maybe the first survey [150]. For the Fourier transform it is known that a signal is either localized in time or in space, but not in both. Wavelets in fact offer an alternative here. Most of the early investigations of wavelets in Fourier analysis were done by French researchers who used the French name “ondelette”–little wave, as the later english translation also indicates. However, as in many mathematical ideas and theories, there is not only one origin of wavelet theory. Without claiming completeness, let me just mention fields like Euclidean field theory, classical statistical mechanics, and renormalization [13]. Perhaps the most prominent field of application is signal and image processing. In fact, wavelets allow us to decompose a signal or an image into its components with respect to a whole cascade of levels. This decomposition is done by the fast wavelet transform which is of linear complexity as long as the wavelet is compactly supported. However, for many years the Haar wavelet was the only known wavelet of compact support. Moreover, the Haar wavelet is piecewise constant and hence maybe not well suited for more regular (smooth) signals. Hence, it was a challenge to construct smooth wavelets of compact support. At first glance, this seems to be an easy task since exponentially decaying orthonormal wavelets were introduced earlier by Yves Meyer [150]. These wavelets were simply cut within a window so that the remaining parts were considered negligible. Somewhat surprisingly, it took several years until Ingrid Daubechies published her famous orthonormal wavelets with compact support in 1988 [96]. These functions gave rise to finite masks that could be used for decomposition and reconstruction of signals in linear complexity. Due to orthonormality, decomposition and reconstruction filters are identical. Since this is sometimes too restrictive, biorthogonal wavelets were introduced in 1992 by Albert Cohen, Ingrid Daubechies, and Jean-Christophe Feauveau [71]. In this construction, reconstruction and decomposition filters can be chosen differently with some freedom. The great use of these filters is also indicated by the fact that they are now used in the jpeg2000 imaging standard [133]. Decomposition and reconstruction allows us to transform a signal or an image from one representation to a different one, namely from a single scale to a 1
2
I N T RO D U C T I O N
multiscale representation. However, successive application of these two operations gives back the original signal or image as long as the corresponding filters are chosen appropriately. The reason why wavelets are so successful in signal and image processing lies in the fact that the multiscale representation allows us to modify the signal or image for different purposes. Firstly, it turns out that “reasonable” signals or images have a sparse multiscale representation in the sense that many coefficients in this representation are zero or at least small. Hence, one can neglect these small coefficients. This is the key point for compression. However, just to compress a signal or image is only half of the story. Of course, one would like to change the original information as little as possible when compressing the data. Since wavelets (no matter if they are orthogonal or biorthogonal) allow us to estimate the arising error in terms of the neglected coefficients, it is quite easy to control the error. Hence, any user can choose the compression rate beforehand. The reason for this is that wavelet bases give rise to so-called norm equivalences. This means that the norm of a function (e.g. a signal) is equivalent to the norm of the wavelet coefficients. Finally, such an equivalence not only holds for one single type of norm, but for a whole range. Besides compression, the multiscale representation allows several other possibilities. It allows us to detect textures or edges in an image. This has been used, e.g. in the FBI fingerprint project where a database of fingerprint images has been created that store the images in a sparse format and allow for efficient and reliable identification [38]. Other possible features of the multiscale representation is denoising, which was also used in the transformation of music from old records to modern CDs. At this point there exist also several links between wavelet theory and stochastics. The construction of appropriate wavelets has always been an important task since the properties of the wavelets are also the foundation for the quality of any corresponding method. Hence, construction was and still is an issue. St`ephane Mallat introduced the framework of Multiresolution Analysis (MRA) which also became the most prominent tool for constructing wavelets [149]. Maybe also because of the great success of wavelet techniques in signal and image processing, the idea came up also to use wavelet methods for the numerical solution of differential equations. In particular, their compression properties seem to indicate that wavelets could represent such solutions that have localized features, as for example few peaks but smooth in all remaining areas. A pioneering paper was written by Gregory Beylkin, Ronald Coifman, and Vladimir Rokhlin [31], who realized that not only signals and images but also certain operators have a sparse representation in terms of wavelets. This was the starting point for many contributions. A second issue was preconditioning. It had been noticed that a multiscale structure might be useful to construct preconditioners in such a way that the condition number is independent of the size of the matrix (or, equivalently, the size of the discretization). Hierarchical bases, introduced by Harry Yserentant
S O M E A S P E C T S O F T H E H I S T O RY O F WAV E L E T S
3
[186], gave a great improvement in lower dimensions. Then, different preconditioners introduced by Peter Oswald [162] and James Bramble, Joe Pasciak and Jinchao Xu [43] were proven to be asymptotically optimal. On the other hand, MultiGrid methods (e.g. by Achi Brandt in 1976 [44]) have also been proven to be optimal e.g. by Roy A. Nicolaides [160, 161], Randolph Bank and Todd Dupont [6], and Dietrich Braess and Wolfgang Hackbusch [41]. More or less in parallel, optimal preconditioners have also been investigated for wavelet methods. They have been introduced basically independently by Wolfgang Dahmen and Angela Kunoth [84] as well as by St`ephane Jaffard [134]. In the early days, there was great enthusiasm for using wavelets for numerically solving differential equations. However, the expectations of a breakthrough using wavelet methods were not satisfied very quickly, besides some applications to academic problems. The reason is that signals and images are defined on intervals or squares; a partial differential equation, however, is defined on more general bounded domains in Rn . Hence, using wavelets for a partial differential equation on a bounded domain Ω obviously requires us to construct wavelets on this Ω. Restricting “standard” wavelets to Ω seems to be an easy thing to do, but in fact this is not sufficient. The reason is that properties like compression and error control mentioned above rely on a mathematical framework that is provided by wavelets. Thus, one needs to preserve this framework on Ω. It turned out that this is by far not as easy as for local methods like, e.g. finite elements. At the time when it was realized that constructing wavelets on general domains is a nontrivial task, there was already a kind of competition in particular between wavelet methods and finite elements. Researchers in both fields claimed that their “own method” is the best. One of the reasons for that competition might be that some of the early wavelet researchers in numerical methods claimed that they would come up soon with a wavelet-based numerical method that would beat all known methods. Nowadays, we know that this expectation could not be satisfied. On the other hand, wavelet bases on general domains Ω do exist nowadays. Even though their construction is somewhat more involved than just taking globally continuous piecewise polynomials as for finite elements, there is a complete theory and also corresponding software tools available. Nowadays it is widely accepted that there is a benefit from wavelet methods which also somehow ended the competition. In fact, wavelet-based numerical analysis already had a positive impact on finite elements at least in two areas. First, optimality of an adaptive numerical scheme was proven by Albert Cohen, Wolfgang Dahmen, and Ron DeVore for wavelet methods [66]. Later, this analysis was also done for finite elements by Peter Binev, Wolfgang Dahmen, and Ron DeVore [32]. The second issue was uniform stability (in the sense of the so-called LBB condition) of adaptive methods for the Stokes problem. This was observed for wavelets in papers by Stephan Dahlke, Reinhard Hochmuth, and the author [81] as well as by
4
I N T RO D U C T I O N
Albert Cohen, Wolfgang Dahmen and Ron DeVore [67]. Later, a corresponding finite element method was introduced by Eberhard B¨ ansch, Pedro Morin, and Ricardo Nochetto [7], which was proven to be convergent (however, without rate so far). In the meanwhile, there is a frequent interaction between these communities which is mutually beneficial. Among others let me just mention two very close connections. For some years the coupling of different methods has been investigated. One possibility is by domain decomposition, using different discretization methods on different subdomains and to couple them for example by so-called Mortar methods. In the wavelet context this was investigated, e.g. by Silvia Bertoluzza and Val´erie Perrier [25, 29]. A second close link is the construction of finite element wavelets by Rob Stevenson, partly in joint work with Wolfgang Dahmen [94, 170]. The theory of wavelet methods for elliptic problems has been more or less settled during the last few years. In particular, adaptive wavelet methods have been studied intensively. Starting from the above-mentioned paper by Beylkin, Coifman, and Rohklin, wavelets have been widely used for adaptive numerical solvers, e.g. by Jacques Liandrat and Philippe Tchamitchian [148] as well as Amir Averbuch, Gregory Beylkin, Ronald Coifman, and Moshe Israeli [3]. Silvia Bertoluzza provided a posteriori error estimates [20, 21], but the theory was not complete. The series of papers by Albert Cohen, Wolfgang Dahmen, and Ron DeVore answered all basic questions on convergence and optimality of adaptive wavelet methods [66, 67, 68, 69]. Rob Stevenson (partly with coauthors) has introduced several improvements [110, 173]. However, many questions on the performance of particular schemes, realization, on different applications, and on new and challenging problems still remain open. Of course, wavelets have not only been used for numerically solving elliptic problems. Ami Harten already pointed out the use of a multiresolution hierarchy for hyperbolic problems [128]. Even though the theory of such problems is not as advanced as for the elliptic case, wavelet and multiscale techniques are used in highly complicated fluid computations, see e.g. [159]. Also integral equations, parabolic problems and systems have been investigated, both linear and nonlinear, see e.g. [124, 125]. We will not describe these interesting extensions and applications here since it goes beyond the scope of this book.
1.2
The scope of this book
From this brief historical sketch it should be clear that wavelets is a very wide field with different applications in completely different areas. Hence, I doubt that anybody can write “the ultimate” wavelet book. As the title already says, this book concentrates on wavelet methods for elliptic partial differential equations. There are already some books in that field.
THE SCOPE OF THIS BOOK
5
On one hand, there are some general books on wavelets, mostly with the focus on signal and image processing, that contain some chapters on numerical applications. Usually, these chapters are rather short and do not review the enormous progress of the last decade. On the other hand, there are some research monographs mostly concerned with the theoretical background and foundations of wavelet methods for elliptic partial differential equations. These books are written for researchers or at least advanced graduate students. I am not aware of a textbook that introduces wavelet methods for elliptic partial differential equations and bridges to recent developments, including deep analysis. This book tries to give a contribution to fill this gap. It is supposed to be readable for students with a basic knowledge in analysis and numerical mathematics. The idea is to try to avoid technicalities as much as possible and at the same time to give most of the proofs in detail. This is the reason why we remain in the periodic setting as long as possible. Of course we are aware that periodic problems are mostly of academic interest and are only a very limited subclass of problems. However, the periodic setting is the most convenient framework for wavelets which allows for easy constructions and fast algorithms. On the other hand, the basic techniques that are used in the analysis of the numerical methods are exactly the same as for more general boundary value problems. Hence, we describe the construction of wavelet bases on general domains Ω later and show that they fit into the previously settled framework. Another aspect of this book is that we try to provide as much numerical realization as was possible to us in a limited amount of time. This is done for each chapter at the end in the section entitled Exercises and Programs. We decided not to provide the described software in the form of a CD. The main reason is that software tends to become outdated very quickly and needs regular maintenance. Thus, we provide all software on a corresponding webpage. If this book is used as background material for a corresponding lecture, we highly recommend the use of the software also for exercises. Students (and teachers) can learn a lot by coding themselves. Even though most of the software is provided in C++, we do not expect readers to be experts in C++. In fact, we also provide an open source (BSD license) library called FLENS (Flexible Library for Efficient Numerical Solutions) which allows very easy, flexible but still efficient programing of the introduced methods in C++. A further aspect of the software is the following. Many researchers still believe in the prejudice that wavelet methods cannot be used for nonperiodic problems because the construction of wavelet bases on general domains is so complicated that it cannot be realized efficiently. We describe one possible construction of wavelets on general domains, the so-called wavelet element method, which was introduced by Claudio Canuto, Anita Tabacco, and the author. We also provide a corresponding software package which was basically written at the Politecnico di Torino (Italy) by Stefano Berrone in collaboration with Laurent Emmel and Tomas Kozubek. Even for a reader who might not be interested in the details of the construction, we describe how to use the software almost as a “black box”.
6
1.3
I N T RO D U C T I O N
Outline
We begin in Chapter 2 with the multiscale approximation of given functions and introduce the notion of Multiresolution Analysis (MRA). We start with the simple piecewise constant Haar system and show main properties. We also show the limitation that arises from the lack of smoothness. Next, we investigate piecewise linear systems and show common properties with the Haar system as well as differences. An MRA is basically defined by a refinable function which in turn is given by a mask. These masks will be introduced and investigated and we show how they are used for fast wavelet algorithms. Then, we introduce Daubechies’ orthonormal scaling functions with compact support as well as biorthogonal B-spline scaling functions. Note that wavelets will not be involved until Chapter 5. Since this book is concerned with wavelet methods for elliptic partial differential equations, we introduce elliptic boundary value problems in Chapter 3. We focus on the main theoretical results on weak (or variational) formulations, regularity as well as Galerkin methods as a framework for a numerical scheme. All this is well-known material from advanced functional analysis or partial differential equations. Readers that are familiar with these facts can continue with the next chapter or just look for the notation that will also be used for the rest of the book. Chapter 4 joins the first two chapters in the sense that general MRAs are used for the discretization of elliptic boundary value problems in terms of a Galerkin scheme. Sometimes this is already called the “Wavelet-Galerkin” method in some papers, but as we will see this has nothing to do with wavelets so far. We introduce multiresolution discretization and describe how the arising linear system of equations can be set up in the computer. We also provide the error analysis of the arising numerical approximation. As already mentioned above, the BPX preconditioner, as well as MultiGrid, are also multiscale methods. They are often identified with finite elements, but the only thing that is really needed is a multiscale structure of the discretization. This multiscale structure can also be provided by any MRA. We provide both schemes in the framework of an MRA and also describe numerical experiments for their comparison. Chapter 5 contains the first introduction of wavelets in this book. We describe the idea of building complements and the notion that wavelets form a basis for these complements. Then, different kinds of complements are described. Orthogonal complements lead to orthogonal wavelets, biorthogonal complements to biorthogonal wavelets. The Fast Wavelet Transform (FWT) is not only an important tool in signal and image processing but also for numerical methods. We describe the FWT and also its components decomposition and reconstruction. Then, we investigate the strong analytical properties of wavelets that will be particularly relevant for the analysis of the numerical approximation schemes. We start with vanishing moments and the resulting compression. Then, we show the already-mentioned error control in terms of so-called norm equivalences. Since these norm equivalences play an important role later, one is interested in finding
OUTLINE
7
possibly general criteria to guarantee these equivalences. It was shown in several papers by Wolfgang Dahmen (partly with coauthors) that two well-known estimates from approximation theory provide these criteria. These estimates are known as the Jackson and Bernstein estimates. We show how these estimates give rise to norm equivalences. We briefly sketch other kinds of wavelets. However, since this is not the main focus of this book, this section is not a comprehensive description of all possible wavelet constructions. Moreover, there are also so many recent developments (e.g. curvelets, ridgelets, shearlets, etc.), that a complete description is far beyond the scope of this book. “True” Wavelet-Galerkin methods are introduced in Chapter 6. The decomposition of a multiresolution space into details on various levels offers a multiscale representation of the discrete system. This is used to obtain an optimal wavelet preconditioner by a simple diagonal scaling which is proven. We show then that one should avoid computing and storing the system matrix in multiscale form since this matrix loses the sparsity which is provided by the multiresolution form. It turns out that the FWT is the key to combining sparsity of the multiresolution form and optimal preconditioning by wavelets. We provide corresponding numerical experiments to investigate the quantitative behavior of the method. Chapter 7 is devoted to adaptive wavelet methods and bridges to quite recent developments. We start by adaptive approximations of given functions and by reviewing standard adaptive methods in terms of a posteriori error estimates and (de-)refinement. In terms of wavelets, an ad hoc refinement strategy is obvious which we introduce and test numerically. The next sections contain part of the theory introduced by Cohen, Dahmen, and DeVore for adaptive wavelet methods. The road map is as follows: Using wavelets allows us to reformulate the differential equation as a discrete infinite-dimensional problem for the unknown wavelet expansion coefficients of the solution. Moreover, this discrete problem is well conditioned. An adaptive scheme results by first considering an infinitedimensional iteration and then a computable version arises by introducing finite approximations of the involved operators. We introduce the analysis of these schemes and describe numerical experiments. We also give an outlook to nonlinear variational problems. This part of the book, however, is on a different level concerning the numerical realization. When this manuscript was finished, we were not aware of any software that completely realized an adaptive wavelet method for nonlinear problems which is in a state that allows publication of the software. Moreover, this field still shows very active research so that it is much less settled from the numerical point of view, even though the theory is fixed. All these chapters can be read without any knowledge of how wavelet bases are constructed on general domains. Such a construction is described in Chapter 8. We start by modifying the previously considered periodic setting for MRA and wavelets in order to construct wavelet bases on bounded univariate intervals. In particular, we focus on those relevant properties of wavelets that
8
I N T RO D U C T I O N
allow, for example, norm equivalences. The next step is to construct tensor products of these functions in order to define wavelet bases on rectangular domains in Rn . Finally, we introduce the Wavelet Element Method (WEM) as one possible construction of wavelet bases on general domains. The main idea is to decompose the domain Ω into a nonoverlapping domain decomposition and to map each subdomain to the unit square (2D) or cube (3D). Finally, a matching across the subdomain interfaces provides a globally continuous basis which is required for the discretization of second-order problems. We detail the construction and also give several examples. Chapter 9 contains the description of three applications. First, we consider elliptic boundary value problems on bounded domains, the focus of this book. As one possible reference generalization, we describe saddle point problems. In particular, we show how adaptive wavelet methods omit the usual stability criterion (LBB condition). One specific example of a saddle point problem is the Stokes equation which is a simplified (linear) model for the flow of a viscous incompressible fluid. Finally, we provide three appendices which are meant to give background material which may not be known to some readers. Appendix A contains an introduction to Sobolev spaces and variational formulations of elliptic partial differential equations. Appendix B is devoted to Besov spaces and their relation to the analysis of adaptive numerical methods. Finally, Appendix C contains, for completeness, some basic iterative schemes that are used in the book.
2 MULTISCALE APPROXIMATION AND MULTIRESOLUTION In this chapter, we introduce the concept of multiscale approximation of functions and Multiresolution Analysis (MRA). We also give the proofs of all relevant properties. We will see later that these concepts are an important tool for constructing wavelets. 2.1
The Haar system
The Haar system goes back to Alfred Haar in 1910 [121, 122] and is perhaps the simplest example of multiresolution, leading also to a wavelet system. Let us start with an easy example of approximating a given function f : Ω → R,
Ω := [0, 1],
(2.1)
by piecewise constant functions of step sizes hj := 2−j , 2.1.1
j = 0, 1, 2, . . .
Projection by interpolation
Of course, there exist several possibilities of defining a piecewise constant approximation to a given function. One standard example is interpolation, i.e. (PjI
f )(x) :=
j 2 −1
f (k 2−j ) χ[k 2−j ,(k+1) 2−j ) (x)
k=0
=
j 2 −1
j
2− 2 f (xj,k ) ϕHaar [j,k] (x),
x ∈ Ω,
(2.2)
k=0
where xj,k := k 2−j are the interpolation knots and ϕHaar := χ[0,1)
(2.3)
as well as for any function g : R → R g[j,k] (x) := 2j/2 g(2j x − k), x ∈ R. 9
(2.4)
10
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
6 ϕ[2,8]
2
ϕ[1,6]
ϕ[0,0]
ϕ[−1,2]
1
1
2
3
4
5
6
Fig. 2.1 Scaled and shifted version ϕ[j,k] of ϕ := χ[0,1) for the choices (j, k) ∈ {(0, 0), (1, 6), (2, 8), (−1, 2)}. We use the standard notation χS (x) :=
1, if x ∈ S, 0, else,
for the characteristic function of some set S ⊆ R (and correspondingly in Rn ). Obviously, (2.4) defines a scaled and shifted version of g, where j ∈ Z is the scale or level and k ∈ Z denotes the integer shift. Figure 2.1 shows various instances of ϕ[j,k] note that ϕ ≡ ϕ[0,0] . Obviously, PjI : C 0 (Ω) → SjHaar is a linear projector onto the space SjHaar := {g : Ω → R : g|[k2−j ,(k+1)2−j ) ∈ P0 , k = 0, . . . , 2j − 1},
(2.5)
where Pd is the set of algebraic polynomials of degree at most d ∈ N0 , i.e. d k ak x : ai ∈ R . (2.6) Pd := p : R → R : p(x) = k=0
The error estimate for this approximation is well known from any textbook (e.g. [175]) on interpolation as
f − PjI f sup;Ω ≤ 2−j f sup;Ω
(2.7)
for any f ∈ C 1 (Ω), where · sup;Ω denotes the standard supremum norm on Ω, i.e.
g sup;Ω := sup |g(x)|. x∈Ω
(2.8)
THE HAAR SYSTEM
11
For numerical purposes PjI is attractive since the expansion in (2.2) is very easy to compute. In fact, the computation of the expansion coefficients j
cIj,k := 2− 2 f (xj,k ) for the representation PjI
f=
j 2 −1
cIj,k ϕHaar [j,k]
k=0
just amounts to the evaluation of the given function f . One obvious drawback is that f (xj,k ) needs to be well defined in order to ensure that PjI f is meaningful. Hence, f should be at least piecewise continuous. Moreover, the error estimate (2.7) seems to require that f is continuous, i.e. f ∈ C 1 . However, in order to ensure that the right-hand side of the error estimate in (2.7) is meaningful, one only needs f ∈ L∞ (Ω) which means that f must be “only” in the Sobolev space 1 (Ω) (see Appendix A). W∞ Figure 2.2 (page 12) shows PjI f as well as the error f − PjI f for the function f (x) := sin(2πx). The corresponding numbers are given in Table 2.1. We display the error in the supremum norm defined in (2.8) as well as that corresponding to the Euclidean norm
g euc;Ω :=
Ω
|g(x)|2 dx,
which is approximated by a standard trapezoidal rule. We also show the same quantities for the piecewise linear function J 2 (x − xJ,2k ), for xJ,2k ≤ x < xJ,2k+1 , k = 0, . . . , 2J ; fJ (x) := 2J (xJ,2k+2 − x), for xJ,2k+1 ≤ x < xJ,2k+2 , k = 0, . . . , 2J ;
(2.9)
for a given J ∈ N, in Figure 2.3 and the corresponding norms in Table 2.2 (there for J = 4). Note that f in (2.9) is of course not in C 1 . The convergence history is included in Figure 2.4 (page 16) in a semilogarithmic scale. As we can see, the rate of convergence is the same in all cases. We observe at least three facts: • The error estimate (2.7) is of first order. The reason is obvious: we use piecewise constant approximations. • Even though fJ ∈ C 1 (Ω), we obtain qualitatively the same order of con1 (Ω), and (2.7) vergence as for sin(2π·) ∈ C ∞ (Ω). This is due to fJ ∈ W∞ only requires derivatives in the weak sense. • Measuring the error in the Euclidean norm gives better quantitative results, but the same order of convergence, than using the supremum norm.
12
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
Fig. 2.2 PjI f and error f − PjI f for the function f (x) := sin(2πx). The first row contains PjI f , j = 0, 1, 2, the second row the corresponding errors. In the third row, we show PjI f , j = 3, 4, 5 and the corresponding errors in the fourth row, and so on. 2.1.2
Orthogonal projection
Let us now describe a second possibility of defining a piecewise constant approximation to f , namely the orthogonal projection. The system of functions j := {ϕHaar ΦHaar j [j,k] : k = 0, . . . , 2 − 1}
is orthonormal with respect to the standard inner product (·, ·)0;Ω in L2 (Ω), i.e. (f, g)0;Ω :=
f, g ∈ L2 (Ω),
f (t) g(t) dt, R
which means
Haar ϕHaar [j,k] , ϕ[j,m]
0;R
= δk,m =
1, if k = m, 0, otherwise.
THE HAAR SYSTEM
13
Table 2.1 Approximation error for f (x) := sin(2πx) (f − PjI f ) for the supremum and Euclidean norm. sin(2πx) Level 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
norm Supremum Euclidean 1.000000 1.000000 1.000000 0.706971 0.382506 0.194902 0.097826 0.048876 0.024350 0.012080 0.005944 0.002876 0.001342 0.000575 0.000192 0.000000
128.000000 128.000000 109.111151 57.142484 28.898568 14.485811 7.242827 3.616752 1.803141 0.896262 0.442808 0.216068 0.102674 0.045917 0.017355 0.000000
The corresponding norm (before also called the Euclidean norm) is as usual defined by 2
f 0;Ω := (f, f )0;Ω := |f (t)|2 dt, f ∈ L2 (Ω). R
Hence, we define the mapping PjO f of f onto SjHaar as PjO f
:=
j 2 −1
k=0
f, ϕHaar [j,k]
0;Ω
ϕHaar [j,k] .
(2.10)
It is easily seen that PjO f defined by (2.10) is in fact the orthogonal projection. For vj =
j 2 −1
k=0
Haar cj,k ϕHaar [j,k] ∈ Sj
14
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
Fig. 2.3 PjI f and error f − PjI f for the function fJ defined in (2.9). The first row contains PjI f , j = 3, 4, 5, the second row the corresponding errors. In the third row, we show PjI f , j = 6, 7, 8 and the corresponding errors in the fourth row, and so on.
one has
PjO vj
=
j j 2 −1 2 −1
cj,k
Haar ϕHaar [j,k] , ϕ[j,m]
m=0 k=0
0;Ω
ϕHaar [j,m]
=
j 2 −1
cj,m ϕHaar [j,m] = vj ,
m=0
as well as for any f ∈ L2 (Ω) and all vj ∈ SjHaar
(f −
PjO f, vj )0;Ω
=
j 2 −1
cj,m f, ϕHaar [j,m]
m=0
−
j j 2 −1 2 −1
k=0 m=0
0;Ω
cj,m f, ϕHaar [j,k]
0;Ω
Haar ϕHaar [j,k] , ϕ[j,m]
0;Ω
THE HAAR SYSTEM
15
Table 2.2 Approximation error (fJ − PjI fJ ) for fJ in (2.9) for the supremum and Euclidean norm. f4 (x)
norm
Level
Supremum
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
=
Euclidean
1.000000 1.000000 1.000000 1.000000 1.000000 0.999023 0.499023 0.249023 0.124023 0.061523 0.030273 0.014648 0.006836 0.002930 0.000977 0.000000
j 2 −1
104.511587 104.511587 104.511587 104.511587 104.511587 104.435013 52.179228 26.051332 12.987374 6.455376 3.189338 1.556237 0.739510 0.330719 0.125000 0.000000
cj,m (f, ϕHaar [j,m] )0;Ω
−
m=0
j 2 −1
cj,m (f, ϕHaar [j,m] )0;Ω
m=0
=0 due to the orthonormality. On the other hand, defining the unit interval I = I0,0 = [0, 1) and its scaled and shifted version by Ij,k := 2−j [k, k + 1), we can rewrite (2.10) as PjO f =
k∈Z
2j χIj,k
f (y) dy, Ij,k
16
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
103
102
101
100
10–1
10–2
10–3
10–4
sin sup sin euc hat sup hat euc 0
5
10
15
Fig. 2.4 Convergence history for f (x) := sin(2πx) [sin] and fJ [hat] for the supremum [sup] and Euclidean [euc] norm. which shows that PjO f is in fact a piecewise constant approximation to f , where the pieces are of size hj := |Ij,k | = 2−j . The approximation becomes better and better (i.e. the error gets smaller and smaller) for increasing j. More precisely, we will see later that the error estimate reads −js
f − PjO f 0;Ω <
f s;Ω , ∼ 2
s ≤ 1,
(2.11)
where · s;Ω denotes the Sobolev norm of order s ∈ R+ , see Appendix A. In Table 2.3, we show the errors for the smooth function sin(2πx), in Table 2.4 for the piecewise linear function fJ . The slope of the approximations and the corresponding errors for these two functions are shown in Figures 2.5 and 2.6, respectively. Finally, the convergence history is shown in Figure 2.7 again in a semilogarithmic scale. We obtain results similar to those for the interpolation, namely the rates of approximation are the same for sin(2π·) and fJ even though
PIECEWISE LINEAR SYSTEMS
17
Table 2.3 Approximation error for f (x) := sin(2πx) (f − PjO f ) for the supremum and Euclidean norm. sin(2πx) Level
norm Supremum Euclidean
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1.000000 0.636620 0.636620 0.372923 0.193839 0.097860 0.049048 0.024539 0.012271 0.006136 0.003068 0.001534 0.000767 0.000383 0.000192 0.000096
128.000000 55.710232 55.710232 28.724182 14.473158 7.250544 3.627030 1.813754 0.906946 0.453559 0.226946 0.113804 0.057560 0.030060 0.017355 0.012272
the smoother function is quantitatively better approximated. The reason is that (2.11) is valid only for s ≤ 1. This means that piecewise constant functions cannot approximate better even if the function f is arbitrarily smooth. This is a strong indication to use more regular functions ϕ. Before we proceed, let us introduce an abbreviation that will frequently be used in the sequel. We often use the notation A B to indicate that A ≤ cB
(2.12)
with a constant c > 0 which is independent of the various parameters A and B may depend on (e.g. in (2.11) the constant is independent of j and s). A B is defined in a similar way and A ∼ B means that A B and A B with possibly different constants, of course. 2.2
Piecewise linear systems
It is a well-known fact that one can obtain a better rate of approximation by using more regular functions than piecewise constants. In fact, the rate p = 1 in
18
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
Table 2.4 Approximation error for fJ for the supremum and Euclidean norm. f4 (x)
norm
Level
Supremum
Euclidean
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0.500000 0.500000 0.500000 0.500000 0.500000 0.500000 0.250000 0.125000 0.062500 0.031250 0.015625 0.007812 0.003906 0.001953 0.000977 0.000488
52.255831 52.255831 52.255831 52.255831 52.255831 52.255831 26.127990 13.064145 6.532371 3.266784 1.634587 0.819680 0.414578 0.216506 0.125000 0.088388
(2.7) (note that 2−j = hj ) and in (2.11) is caused by the fact that we consider only piecewise constant approximations. Thus, it comes immediately to mind to replace ϕHaar by a piecewise linear function, the hat function, defined by 1 + x, if x ∈ [−1, 0); (2.13) ϕhat (x) := 1 − x, if x ∈ [0, 1); 0, else, x ∈ R. Since ϕhat is nothing other than the linear B-spline, the definition of P I f by interpolation is absolutely standard and is left as an exercise to the reader. The error estimate is well known and reads
f − P I f sup;Ω ≤
1 −2j 2
f sup;Ω 2
(2.14)
for all f ∈ C 2 (Ω), see e.g. [175] and any other textbook on numerical mathematics.
PIECEWISE LINEAR SYSTEMS
19
Fig. 2.5 PjO f and error f − PjO f for the function f (x) := sin(2πx). The first row contains PjO f , j = 0, 1, 2, the second row the corresponding errors. In the third row, we show PjO f , j = 3, 4, 5 and the corresponding errors in the fourth row, and so on. But how do we define the orthogonal projection PjO ? In fact, this is not obvious. Of course, one could start with j := {ϕhat Φhat j [j,k] |Ω : k = 0, . . . , 2 }
and form an orthonormal basis, e.g. by Gram–Schmidt. However, the resulting functions are supported on all of Ω and thus are not well suited for computational in a little bit more purposes. It pays off to investigate the basis properties of Φhat j detail in order to highlight the similarities and differences of the two bases. It will shorten notation to use j hat hat = {ϕhat Φhat j j,k : k = 0, . . . , 2 } = {ϕj,k : k ∈ Ij },
where ϕhat j,k
:=
ϕhat [j,k] ,
if k = 1, . . . , 2j − 1;
ϕhat |Ω , [j,k]
if k = 0, 2j ,
(2.15)
20
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
Fig. 2.6 PjO f and error f − PjO f for the function fJ defined in (2.9). The first row contains PjO f , j = 3, 4, 5, the second row the corresponding errors. In the third row, we show PjO f , j = 6, 7, 8 and the corresponding errors in the fourth row, and so on. and Ijhat := {0, . . . , 2j }. Similarly, we can write j Haar Haar = {ϕHaar }, ΦHaar j j,k : k = 0, . . . , 2 − 1} = {ϕj,k : k ∈ Ij
(2.16)
Haar and ϕHaar j,k := ϕ[j,k] as well as
IjHaar = {0, . . . , 2j − 1}. As already said, ΦHaar is an orthonormal basis of SjHaar . This means in particular j that
2
Haar
c ϕ = |cj,k |2 =: cj 22 (Ij ) , (2.17) j,k j,k
k∈I Haar
k∈I Haar j
0;Ω
j
PIECEWISE LINEAR SYSTEMS
21
103
102
101
100
10–1
10–2
10–3
10–4
10–5 0
sin sup sin euc hat sup hat euc 5
10
15
Fig. 2.7 Convergence history for f (x) := sin(2πx) [sin] and fJ [hat] for the supremum [sup] and Euclidean [euc] norm. for any coefficient vector cj = (cj,k )k∈IjHaar ∈ 2 (Ij ), where for any countable index set I we use 2 (I) := {c = (ck )k∈I : c 2 (I) < ∞},
c 22 (I) :=
|ck |2 .
k∈I
Of course, (2.17) is nothing other than Pythagoras’ theorem. Obviously, the hat functions are not orthonormal since
hat hat ϕj,k , ϕj,m 0;Ω =
2 3, 13 ,
if m = k ∈ {1, . . . , 2j − 1}; if m = k ∈ {0, 2j };
1 hat 6 , if m = k ± 1 ∈ Ij ; 0, otherwise.
(2.18)
22
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
This means that for any coefficient vector cj = (cj,k )k∈Ij , we observe
2
hat
cj,k ϕ[j,k]
k∈I hat
0;Ω
j
where
=
hat T hat cj,k cj,m (ϕhat [j,k] , ϕ[j,m] )0;Ω = cj Gj cj ,
k,m∈Ijhat
hat hat := (ϕ , ϕ ) Ghat j [j,k] [j,m] 0;Ω
k,m∈Ijhat
is the Gramian matrix of the set of functions Φhat j . Now, we use standard properties of the Rayleigh quotient which is defined for any matrix A ∈ Rn×n by r(A, u) :=
(u, Au) . (u, u)
(2.19)
In fact, it is well known that for symmetric matrices A ∈ Rn×n the estimate λmin (A) ≤
(u, Au) ≤ λmax (A) (u, u)
(2.20)
holds, where λmin (A) and λmax (A) denote the minimal and maximal eigenvalue (in absolute values) of the matrix A, respectively. Moreover, we get a representation for the condition number (which is a crucial quantity for the convergence speed of an iterative method) cond2 (A) =
rmax (A) , rmin (A)
r(A, u),
rmin (A) :=
(2.21)
where rmax (A) :=
max
u∈Rn \{0}
min
u∈Rn \{0}
r(A, u),
see Exercise 2.8. Then, we have
hat hat
λmin (Gj ) cj 2 (Ij ) ≤ cj,k ϕ[j,k]
k∈I hat j
≤
λmax (Ghat j ) cj 2 (Ij ) .
0;Ω
(2.22) In order to estimate the eigenvalues of Ghat j , we use (2.18) and Gerschgorin’s theorem to obtain 1 2 σ(Ghat j ) ⊂ B1/6 ( 3 ) ∪ B1/3 ( 3 ),
PIECEWISE LINEAR SYSTEMS
23
where σ(A) denotes the spectrum of the matrix A and Br (x) := {y ∈ Rn : x − y ≤ r} is the ball of radius r centered around x ∈ Rn . Thus, 1 1 1 1 1 2 1 λmin (Ghat − , − = min , = j ) ≥ min 3 6 3 3 6 3 6 1 1 1 2 1 + , + = max , 1 = 1. λmax (Ghat j ) ≤ max 3 6 3 3 2 Hence, we obtain chat Φ cj 2 (Ijhat )
hat
≤ cj,k ϕ[j,k]
k∈I hat
≤ CΦhat cj 2 (Ijhat )
(2.23)
0;Ω
j
with constants 1 chat Φ = √ ≈ 0.4, 6
CΦhat = 1.
Note that these constants do not depend on the level j. The norm equivalence is uniformly stable. (2.23) means that Φhat j Using this family of bases {Φhat j } for j ∈ N0 we can define a family of projectors Pj : L2 (Ω) → Sjhat , where Sjhat := {g ∈ C(Ω) : g|[2−j k,2−j (k+1)) ∈ P1 } = span(Φhat j ). In fact, such a projector can be represented as µj,k (f ) ϕhat Pj f := [j,k] , k∈Ijhat
where µj,k : L2 (Ω) → R are appropriately chosen linear operators (functionals). Since Φhat is not an orthonormal system, these projectors are not orthogonal. j One could obtain the orthogonal projector PjO : L2 (Ω) → Sjhat in the following way. The orthogonality condition f − PjO f, ϕhat = 0, m ∈ Ijhat , [j,m] 0;Ω
24
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
leads to the following condition hat = ϕhat f, ϕhat [j,m] [j,m] , ϕ[j,k] 0;Ω
k∈Ijhat
0;Ω
µj,k (f ),
i.e. the coefficients µj,k (f ) can be determined by solving a linear system with (which is symmetric positive definite and sparse). the Gramian matrix Ghat j However, if one wants to represent PjO in terms of an orthonormal basis, an orthogonalization process is needed. This, however, would in general destroy the locality of the basis functions. 2.3
Similar properties
Let us collect properties that are similar in the above two examples and that will also turn out to be crucial in more general situations. 2.3.1
Stability
In both examples we have seen the following inequalities:
cΦ cj 2 ≤ cj,k ϕj,k ≤ CΦ cj 2
k∈Ij
(2.24)
0;Ω
for all vectors cj ∈ 2 (Ij ) with respect to an appropriate index set Ij (i.e. IjHaar = {0, . . . , 2j − 1} and Ijhat = {0, . . . , 2j }). Note that the constants 0 < cΦ ≤ CΦ < ∞ depend on the family of bases Φ := {Φj : j ∈ N0 } = {ϕj,k : (j, k) ∈ N0 × Ij } but not on the level j. In the optimal case of an orthonormal basis (such as the Haar basis), one has cΦ = CΦ = 1. If (2.24) holds, the family Φ := {Φj },
Φj := {ϕj,k : k ∈ Ij },
j ∈ N0 ,
is called a family of uniformly stable bases. in the previous paragraph show that the spectrum The considerations on Φhat j of the Gramian matrix Gj := (ϕj,k , ϕj,m )0;Ω k,m∈Ij
S I M I L A R P RO P E RT I E S
25
of a system of functions Φj := {ϕj,k : k ∈ Ij } includes all information on the quantitative stability properties of Φj . In particular, if one can show that λmin (Gj ) ≥ cΦ , λmax (Gj ) ≤ CΦ for some constants cΦ , CΦ independent of the level j, then the system is uniformly stable. These arguments show that the condition number κ(Φj ) := κ2 (Gj ) =
λmax (Gj ) λmin (Gj )
(2.25)
is a measure for the stability of Φj . In fact, if the functions ϕj,k are normalized in L2 (Ω), i.e. ϕj,k 0;Ω ∼ 1, then Φj is uniformly stable if and only if κ(Φj ) ≤ KΦ independent of j. Of course, the optimal case occurs for orthonormal bases with κ(Φj ) = 1. 2.3.2
Refinement relation
Another similarity arises from the fact that a piecewise polynomial with respect to the grid ∆j := {k2−j : k = 0, . . . , 2j } is also piecewise polynomial with respect to the next finer grid ∆j+1 . This means that the corresponding spaces are nested Sj ⊂ Sj+1
(2.26)
for Sj ∈ {SjHaar , Sjhat }, where Sjhat is defined in an obvious way. Since Φj is a basis of Sj , this implies the existence of coefficients {ajk,m }m∈Ij+1 for any k ∈ Ij and all j ∈ N0 such that j ak,m ϕj+1,m . (2.27) ϕj,k = m∈Ij+1
In both of the above examples, we can say even more since ϕj,k coincides with (restrictions) of ϕ[j,k] . Thus, we have a sequence of coefficients {ak }k∈Z , the so-called refinement coefficients, such that ak ϕ(2x − k), x ∈ R. (2.28) ϕ(x) = k∈Z
Equation (2.28) is called the refinement equation or the two-scale relation. For the above-described two cases of ϕHaar and ϕhat , the refinement coefficients are easily obtained, namely = aHaar = 1, aHaar 0 1
aHaar = 0, k ∈ Z \ {0, 1}, k
(2.29)
26
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
as well as hat = 1, ahat = ahat 0 −1 = a1
1 , 2
ahat = 0, k ∈ Z \ {−1, 0, 1}. k
(2.30)
Note that in both cases the sequence {ak }k∈Z of nonvanishing refinement coefficients is finite. Of course, if ϕj,k = ϕ[j,k] , equation (2.28) implies the level-wise formula as in (2.27). In fact, ϕ[j,k] (x) = 2j/2 ϕ(2j x − k) = 2j/2 = 2j/2
am ϕ(2j+1 x − 2k − m)
m∈Z
am−2k ϕ(2j+1 x − m)
m∈Z
1 √ am−2k ϕ[j+1,m] (x), = 2 m∈Z
(2.31)
where we have used the substitution m → m+2k. This shows that (2.28) implies (2.27) with coefficients • which do not depend on the level j; • which do not depend on k ∈ Ij since 1 ajk,m = √ am−2k . 2 These facts allow the construction of highly efficient algorithms which will be described later. 2.3.3
Multiresolution
In the particular case ϕ ∈ {ϕHaar , ϕhat } the following properties are almost trivial. We collect them for further reference, where we will replace ϕ by other functions. Proposition 2.1 With the above settings, we obtain for Φj ∈ {ΦHaar , Φhat j j }, generated spaces Sj and Ω = [0, 1]: (a) The spaces Sj are nested, i.e. Sj ⊂ Sj+1 . (b) The union of all Sj is dense in L2 (Ω), i.e. closL2 (Ω) j∈N0 Sj = L2 (Ω). (c) The intersection of all Sj is trivial: j∈N0 Sj = S0 . (d) Φ is uniformly stable, each Φj is a basis for Sj .
S I M I L A R P RO P E RT I E S
27
Moreover for the Haar system Sj = SjHaar , we have in addition: (e) The spaces Sj arise via dilation by 2, i.e. f ∈ Sj if and only if f (2·) ∈ Sj+1 for all j ∈ N0 . (f ) The spaces Sj are shift-invariant, i.e. we have f ∈ Sj if and only if f (· − k)|Ω ∈ Sj for all k ∈ Ij .
Proof In fact, (a) is trivial and (c) is a simple consequence of (a). However this property will be important under different circumstances. Hence, we note this fact here for completeness. Properties (d)–(f) have been shown for the two examples. Finally, (b) is a consequence of the density of piecewise constant and linear functions in L2 (Ω), respectively. Note that the family S = {Sj }j∈Z of spaces Sj generated by ϕHaar or ϕhat are the simplest examples of a sequence of spaces having the properties mentioned above. Later, we will use these properties of S to define a sequence S called Multiresolution Analysis (MRA). This notation goes back to St´ephane Mallat [149]. As we shall see later, an MRA is an important tool for constructing wavelet bases. It is not too difficult to construct other sequences of nested spaces that satisfy properties (a)–(c) in Proposition 2.1. We will see later that, for example, B-splines can be used to generate such spaces. Property (d) is a very crucial stability property, while (e) shows the relation between succeeding spaces. The shift-invariance in (f) is a very specific property which also corresponds to the particular property of SjHaar , namely that they are generated by translates of only one function ϕHaar . This is the reason why we will also focus on shift-invariant MRAs on L2 (R). 2.3.4
Locality
As opposed to, e.g. approximation by a Fourier series, the basis functions ϕ[j,k] used in the approximation introduced above are locally supported. Let us explain what this means. As usual, we denote the support of a function f : Ω → R by supp f := closR {x ∈ Ω : f (x) = 0}. A function f : Ω → R is called compactly supported if supp f Ω is compact. Here D E means as usual that the set D is compact in E, i.e. in particular that there is a positive distance from D to the boundary of E. Of course, for computational purposes, compactly supported functions are very useful. In fact, the use of global functions requires us to truncate the functions to a bounded domain and one has to carefully analyze the introduced
28
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
truncation error. In the multilevel setting, we require not only compact support, but that the size of the support scales with the level j in the sense the size decreases exponentially with j. Moreover, there should be only a fixed and finite number of functions per level that overlap. Both properties hold in our two examples as the following statement shows. Proposition 2.2 The following identities hold: (a) The basis functions are locally supported, i.e. −j |supp ϕHaar j,k | = 2 ,
1−j |supp ϕhat , j,k | = 2
k ∈ {1, . . . , 2j − 1}.
(b) The translates of ϕ form a partition of unity, i.e. 1≡ 2−j/2 ϕj,k . k∈Ij
(c) The system Φj is locally finite, i.e. #{m ∈ Ij : |supp ϕj,k ∩ supp ϕj,m | > 0} =
1, if ϕ = ϕHaar ; 1, 3, if ϕ = ϕhat .
independent of k and j, where |B| is the Lebesgue measure of B ⊂ R.
For our two cases, these properties are almost trivial, but again we collect them for further reference. 2.4
Multiresolution analysis on the real line
As already said, the Haar and the hat function system are the simplest examples of MRAs. In many applications, however, higher order approximations or smoother functions are needed. Hence, we now generalize some of the concepts, which we have seen, to more regular systems. One crucial point is the availability of efficient algorithms that allow the refinement of a function in Sj to Sj+1 . To a certain extent, this can be achieved if the MRA is formed by integer translates of only one function ϕ. This leads to the consideration of MRAs on L2 (R), i.e. on the whole real line (which may not be the most interesting case for boundary value problems, of course). Later, we will come back to the consideration of bounded domains Ω ⊂ R. One could think that this can be done easily by restricting ϕ[j,k] to the particular domain Ω as we have done for Sjhat and SjHaar with respect to Ω = [0, 1]. However,
M U LT I R E S O L U T I O N A N A LY S I S O N T H E R E A L L I N E
29
this sometimes conflicts with the stability of Φj , particularly in higher dimensions and also causes serious problems for the construction of wavelet bases. In order to minimize technicalities, we first consider the shift-invariant case on L2 (R). Let us start by defining a Multiresolution analysis [149], which contains the properties listed in Proposition 2.2. Definition 2.3 A sequence S = {Sj }j∈Z of spaces Sj ⊂ L2 (R) is called Multiresolution Analysis (MRA), if (a) the spaces are nested, i.e. Sj ⊂ Sj+1 ; (b) their union is dense in L2 (R), i.e. closL2 (R) Sj = L2 (R); (c) their intersection is trivial, i.e.
j∈Z
Sj = {0};
j∈Z
(d) there exists a function ϕ such that each Φj := {ϕ[j,k] : k ∈ Z} is a uniformly stable basis for Sj , j ∈ Z; (e) the spaces arise by scaling: f ∈ Sj ⇐⇒ f (2·) ∈ Sj+1 , j ∈ Z (dilation); (f ) the spaces are shift-invariant, i.e., f ∈ S0 ⇐⇒ f (·−k) ∈ S0 , k ∈ Z.
Remark 2.4 Note that in the above case, the requirement (d) can be replaced by the demand that Φ0 is a stable basis. However, in order to point out the main ingredients also for more general domains, we prefer to use the above formulation. 2.4.1
The scaling function
Again, as an immediate consequence of Definition 2.3, (a) and (d), we obtain the following recursion formula for ϕ: ϕ(x) =
ak ϕ(2x − k),
x∈R
a.e.
(2.32)
k∈Z
for some coefficients ak ∈ R, k ∈ Z. A function ϕ ∈ L2 (R) satisfying (2.32) is called a refinable function, scaling function or generator of the MRA S. The coefficients ak in (2.32) are called the refinement coefficients of ϕ. Note that one can easily derive from (2.32) the version (2.31) of the refinement equation. In our two examples above, there is a closed formula for ϕ available, too. This is, however, not true in general. Closed formulas for a refinable function are often not available, but ϕ is defined implicitly in terms of its refinement coefficients. Thus, the construction of a refinable function amounts often to finding appropriate refinement coefficients. Hence, we focus on the question under which conditions a sequence of coefficients can be used to define a refinable function.
30
2.4.2
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
When does a mask define a refinable function?
It is a nontrivial question under which conditions the mask a := {ak }k∈Z uniquely determines a function ϕ by (2.32) [55, 97]. In the following, we describe a corresponding criterion that is due to Cavaretta, Dahmen, and Micchelli. Let us consider here only finite masks, i.e. a1 = 0, a2 = 0,
ak = 0 for all k < 1
and all k > 2 ,
(2.33)
for two integers 1 < 2 . Some of the subsequent statements in fact also hold for infinite masks. However, since any computer can only store finite arrays, an application of an infinite mask would require a truncation of this mask. This means that we would introduce an error due to the truncation. Any numerical scheme based on such an infinite mask thus requires us to investigate and control the effect of such truncation errors. In order to avoid this additional effort we restrict ourselves to finite masks here. The following tools are helpful for our purpose. We define the subdivision operator Sa : ∞ (Z) → ∞ (Z) by (Sa c)k := ak−2m cm , c ∈ ∞ (Z), (2.34) m∈Z
and the subdivision scheme for an input c ∈ ∞ (Z) by c0 := c,
cm := Sa cm−1 ,
m = 1, 2, . . .
(2.35)
As usual, we define for any countable index set I and 1 ≤ p ≤ ∞ p (I) := {c = (ck )k∈I : c p (I) < ∞} and
c p (I) :=
1/p |ck |p ,
if 1 ≤ p < ∞;
k∈I
supk∈I |ck |,
if p = ∞.
Hence, a subdivision scheme may be interpreted as follows. Given a sequence c representing a function on the lattice formed by the integers, then Sa c is a refined representation of the underlying function at the half-integers using the mask a for the refinement. We say the subdivision scheme converges for c ∈ ∞ (Z), if there exists a function fc ∈ C(R) such that · lim fc ( 2m ) − cm ∞ (Z) = 0.
m→∞
(2.36)
M U LT I R E S O L U T I O N A N A LY S I S O N T H E R E A L L I N E
31
This can be interpreted in the above way, namely that cm converges to fc with respect to a finer and finer grid. Example 2.5 We illustrate the subdivision scheme with an example using the masks a of 2 ϕ given in Table 2.5 below (page 32) and c := δ0 , i.e., ck := δk,0 , k ∈ Z. The iterations are shown in Figure 2.8. According to [55, Thm. 2.1] the convergence of the subdivisions scheme implies the existence of a refinable function with respect to the particular mask. We do not give the proof here and refer to the cited literature. Proposition 2.6 Suppose that the subdivision scheme (2.34), (2.35) converges for all c ∈ ∞ (Z) and for some c∗ ∈ ∞ (Z) we have fc∗ ≡ 0. Then, the finite mask a determines a compactly supported refinable function ϕ. With the normalization ϕ(x − k) = 1, x ∈ R, (2.37) k∈Z
the function ϕ is uniquely determined. Moreover, fc = ck ϕ(2x − k), x ∈ R, k∈Z
holds for all c ∈ ∞ (Z). In the above proposition, the convergence of the subdivision scheme is assumed. It is of course an interesting question to investigate sufficient conditions on the mask coefficients to ensure convergence of the subdivision scheme. We refer to [55, 70] for corresponding results. 2.4.3
Consequences of the refinability
Next, we describe some immediate consequences of the refinability. The corresponding properties will frequently be used in the sequel. First, we easily obtain ϕ(x) dx = R
k∈Z
=
1
k+1
k
0 k∈Z
ϕ(x) dx =
k∈Z
ϕ(x − k) dx = 1,
0
1
ϕ(x − k) dx (2.38)
Table 2.5 Refinement coefficients of Daubechies scaling functions N = 1, . . . , 10. N
ak
N
1
1.00000000000000 1.00000000000000 0.68301270189222 1.18301270189222 0.31698729810778 −0.18301270189222 0.47046720778416 1.14111691583144 0.65036500052623 −0.19093441556833 −0.12083220831040 0.04981749973688 0.32580342805130 1.01094571509183 0.89220013824676 −0.03957502623565 −0.26450716736904 0.04361630047418 0.04650360107098 −0.01498698933036 0.22641898258356 0.85394354270503 1.02432694425920 0.19576696134781 −0.34265671538293 −0.04560113188355 0.10970265864213 −0.00882680010836 −0.01779187010195 0.00471742793907
6
2
3
4
5
7
8
ak
Nϕ
for
N
ak
0.15774243200290 9 0.69950381407524 1.06226375988174 0.44583132293004 −0.31998659889212 −0.18351806406030 0.13788809297474 0.03892320970833 −0.04466374833019 0.00078325115230 0.00675606236293 −0.00152353380560 0.11009943074562 0.56079128362553 1.03114849163620 0.66437248221108 −0.20351382246269 −0.31683501128067 0.10084646500939 10 0.11400344515974 −0.05378245258969 −0.02343994156421 0.01774979237936 0.00060751499540 −0.00254790471819 0.00050022685312 0.07695562210815 0.44246724715225 0.95548615042775 0.82781653242239 −0.02238573533376 −0.40165863278098 0.00066819409244 0.18207635684732 −0.02456390104570 −0.06235020665028 0.01977215929670 0.01236884481963 −0.00688771925688 −0.00055400454896 0.00095522971130 −0.00016613726137
0.05385034958932 0.34483430381396 0.85534906435942 0.92954571436629 0.18836954950637 −0.41475176180188 −0.13695354902477 0.21006834227901 0.04345267546123 −0.09564726412019 0.00035489281323 0.03162416585251 −0.00667962022628 −0.00605496057509 0.00261296728049 0.00032581467135 −0.00035632975902 0.00005564551403 0.03771715759224 0.26612218279384 0.74557507148647 0.97362811073364 0.39763774176902 −0.35333620179411 −0.27710987872097 0.18012744853339 0.13160298710107 −0.10096657119678 −0.04165924808760 0.04696981409740 0.00510043696781 −0.01517900233586 0.00197332536496 0.00281768659020 −0.00096994783986 −0.00016470900609 0.00013235436685
M U LT I R E S O L U T I O N A N A LY S I S O N T H E R E A L L I N E 1.4
1.4
1.4
1.2
1.2
1.2
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0
0
1
–0.2
–0.2
–0.2
–0.4 0
0.5
1
1.5
2
2.5
3
–0.4
0
0.5
1
1.5
2
2.5
3
–0.4
1.4
1.4
1.4
1.2
1.2
1.2
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0
0
0
–0.2
–0.2
–0.2
–0.4
0
0.5
1
1.5
2
2.5
3
–0.4
0
0.5
1
1.5
2
2.5
3
–0.4
0
0.5
1
1.5
2
2.5
3
0
0.5
1
1.5
2
2.5
3
0.5
1
1.5
2
2.5
3
1.4
1.4
1.4
1.2
1.2
1.2
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0
0
0
–0.2
–0.2
–0.2
–0.4
–0.4 0
0.5
1
1.5
2
2.5
3
0
0.5
1
1.5
2
2.5
3
33
–0.4 0
Fig. 2.8 Example of the subdivision scheme using the mask of the Daubechies scaling functions for N = 2 (Table 2.5, page 32). In order to highlight the results of the scheme, we have drawn vertical lines from the computed point value to the horizontal axis. in view of (2.37), i.e. the integral of ϕ is normalized. Let us collect some further properties: Proposition 2.7 The following identities hold: (a) If ϕ is given by the refinement equation (2.32) for a finite mask (2.33), then ϕ is compactly supported, supp ϕ = [1 , 2 ],
supp ϕ[j,k] = 2−j [1 + k, 2 + k]
as well as |supp ϕ[j,k] | ∼ 2−j , j, k ∈ Z. (b) The refinement coefficients are normalized am = 2. m∈Z
34
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
(c) If the integer translates of ϕ are orthonormal, i.e. (ϕ, ϕ(· − k))0;R = δ0,k , k ∈ Z, we have am a2k+m = 2δk,0 , k ∈ Z. m∈Z
Proof By Proposition 2.6, the function ϕ is compactly supported. Let us assume that supp ϕ = [a, b] for a < b. Note that the translates are locally linearly independent which means that the nontrivial restrictions of the basis functions to any compact subset are linearly independent. Since 2
ϕ(x) =
ak ϕ(2x − k),
k=1
we obtain by the local linear independence [a, b] = supp ϕ = =
2 k=1
supp ϕ(2 · −k) =
2 k+a k+b , 2 2
k=1
a + 1 b + 1 , , 2 2
which proves (a). Next, using (2.32), we have 1 ak ϕ(x) dx = ak ϕ(2x − k) dx = ϕ(x) dx. 2 R R R k∈Z
k∈Z
Since by (2.38) ϕ(x) dx = 1 R
(which, in particular, is nonzero), (b) is proven. As for (c), orthonormality and the refinement equation imply δk,0 = (ϕ, ϕ(· − k))0;R 1 = al am (ϕ(2 · −l), ϕ(2 · −2k − m))0;R = am a2k+m , 2 l,m∈Z
which proves the claim.
m∈Z
Remark 2.8 Note that the statement (a) in Proposition 2.7 cannot be reversed. In fact, there exists a compactly supported refinable function with an infinite
DAU B E C H I E S O RT H O N O R M A L S C A L I N G F U N C T I O N S
mask. The following example by Jia and 1, ϕ(x) := 12 , 0,
35
Micchelli can be found in [135]. Let if 0 ≤ x < 1, if 1 ≤ x < 2, elsewhere.
The refinement coefficients ak are given in terms of the symbol a(z) :=
ak z k = (z + 1)
k∈Z
z2 + 2 , z+2
z ∈ C,
which is a Laurent series with infinitely many terms. The reason is that the integer translates {ϕ(· − k), k ∈ Z} are stable but not linearly independent. However, if ϕ has linearly independent integer translates and is refinable, then the mask must be finitely supported. The proof can be found in [14]. Another consequence of Proposition 2.7 is that the bounds of the support of a compactly supported refinable functions must be integers since they coincide with the first and last index of the mask. If supp ϕ = [1 , 2 ],
1 < 2 ,
then N := |supp ϕ| = 2 − 1 ∈ N is an integer. This number N is also the number of translates ϕ(· − k) that overlap an interval of length 1, i.e. #{k ∈ Z : |supp ϕ(· − k) ∩ [0, 1]| > 0} = #{k ∈ Z : |[1 + k, 2 + k] ∩ [0, 1]| > 0} = #{1 − 2 , . . . , −1 } = 2 − 1 = N, where |·| denotes the Lebesgue measure. This means that the system Φ0 is locally finite. The above statements allow us to check if a given mask a gives rise to a refinable function ϕ. We are still left with the problem of determining such masks (beyond aHaar and ahat , of course). We now give some more examples. 2.5
Daubechies orthonormal scaling functions
So far, we have only seen one example of a scaling function with orthonormal translates, namely the Haar function. However, ϕHaar is not even continuous so that among other drawbacks these functions are of very limited use for the numerical solution of elliptic partial differential equations. So immediately the question arises how to construct compactly supported scaling functions with orthonormal translates and any desired degree of regularity.
36
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
1.2 1 0.8 0.6 0.4 0.2 0
0
1.2 1 0.8 0.6 0.4 0.2 0 –0.2 –0.4 0
0.2
1
2
0.4
3
0.6
4
5
0.8
6
1
7
1.2
8
9
1.4 1.2 1 0.8 0.6 0.4 0.2 0 –0.2 –0.4
1.2 1 0.8 0.6 0.4 0.2 0 –0.2 –0.4 –0.6
0
0
0.5
2
1
4
1.5
6
2
2.5
8
3
10
1.4 1.2 1 0.8 0.6 0.4 0.2 0 –0.2 –0.4
0
1.2 1 0.8 0.6 0.4 0.2 0 –0.2 –0.4 –0.6 0
1
2
2
4
3
6
8
4
10
5
12
1.2 1 0.8 0.6 0.4 0.2 0 –0.2 –0.4 1.2 1 0.8 0.6 0.4 0.2 0 –0.2 –0.4 –0.6
0
1
0
2
2
4
3
6
4
8
5
10
6
12
7
14
Fig. 2.9 Daubechies orthonormal scaling functions for N = 1, . . . , 8 (first row 1, . . . , 4, second row 5, . . . , 8 from left to right). This was an open problem for quite some time, and was finally solved by Ingrid Daubechies [96, 97]. Nowadays, these functions are widely used in various applications. There exists a whole family of such functions labeled by a parameter N , which roughly means that the functions are smoother (and less local) for increasing values of N . There are no closed formulas for the arising functions N ϕ.
The functions are only given in terms of their refinement coefficients. The determination of these coefficients is far from being trivial; it can be found in [96]. Some values are listed in Table 2.5. They are also available in the Matlab Wavelet Toolbox and Lawa. The plot of (some of) these functions is shown in Figure 2.9. It can be shown that Daubechies scaling functions are uniquely determined in the following sense. Given a certain order N of polynomial exactness (i.e. polynomials up to that degree can be exactly represented as linear combinations of ϕ(· − k)), then Daubechies scaling functions Nϕ
are the uniquely determined orthonormal scaling functions with the desired degree of exactness and minimal support. Thus, orthonormality heavily restricts the possible choice of scaling functions. Hence, one is interested in a more flexible framework renouncing orthonormality. As can be seen from the pictures in Figure 2.9, the functions N ϕ gain smoothness for increasing N at the expense of growing support (hence longer masks). It can be shown that Nϕ
∈ C r(N ) ,
r(N ) ∼ 0.2 N,
B-SPLINES
37
where C r denotes the space of H¨older continuous functions (see Definition 2.28, page 56). For N = 1, the Haar system is obtained. In this sense, the orthonormal scaling functions N ϕ generalize the Haar function. 2.6
B-splines
One frequently used example of non-orthogonal scaling functions are cardinal B-splines [101]. They offer the advantage that a closed formula is available as opposed to Daubechies functions. To our knowledge, B-splines are the only known scaling functions with an explicit representation. Moreover, well-known algorithms, e.g. for interpolation, are available. The price to pay is that one has to give up orthogonality. There are several equivalent definitions of B-splines available [101]. For our purposes, the following form is most convenient. Definition 2.9 The function Nd : R → R recursively defined by 1 Nd−1 (x − t) dt, where N1 (x) := χ[0,1) (x) Nd (x) := 0
is called a cardinal B-spline of order d. The (centralized) functions Nd are displayed in Figure 2.10 for d = 1, . . . , 4. A huge selection of literature on splines is available nowadays (see, e.g. [101] and references therein). Now, the following is easy to prove [101]. Proposition 2.10 Cardinal B-splines have the following properties: (a) They are compactly supported with supp Nd ⊂ [0, d]. (b) B-splines are nonnegative, i.e. Nd (x) ≥ 0 and Nd (x) > 0 for x ∈ (0, d). (c) They form a partition of unity, i.e. Nd (x) dx = 1, Nd (x − k) = 1. R
1.2
1.2
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
0.2
0.4
0.6
0.8
1
1.2
0
–1
k∈Z
–0.5
0
0.5
1
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 –1
0.7 0.6 0.5 0.4 0.3 0.2 0.1 –0.5
0
0.5
1
1.5
2
0 –2 –0.5 –1 –0.5 0
0.5
Fig. 2.10 Cardinal B-splines (centralized) of order d = 1, . . . , 4.
1
1.5
2
38
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
(d) Nd ∈ C d−2 (R) and Nd (x) = Nd−1 (x) − Nd−1 (x − 1). (e) Cardinal B-splines are refinable with d d Nd (x) = 21−d Nd (2x − k). k=0 k (f ) Nd ( d2 + x) = Nd ( d2 − x). (g) Nd (x) =
x d−1
Nd−1 (x) +
d−x d−1
Nd−1 (x − 1).
(h) We have d 1 k d (x − k)d−1 Nd (x) = (−1) + , k (d − 1)! k=0
where xn+ := (x+ )n := χ[0,∞) (x) xn .
Proof Exercise 2.5.
2.6.1
Centralized B-splines
It will turn out later that it is useful to consider the following centralized version of cardinal B-splines: d ϕ(x)
:= Nd x + d2 ,
(2.39)
which is nothing other than a shifted version of cardinal B-splines in order to obtain a certain symmetry. Then, the following properties easily follow from Proposition 2.10.
Proposition 2.11 The centralized cardinal B-splines satisfy: (a) They are symmetric around x = 0 for even d and around x = odd d, i.e. d ϕ(x
+ µ(d))) = d ϕ(−x),
1 2
for
(2.40)
for x ∈ R with the definition µ(d) := d mod 2. (b) Their support is given by ! " supp d ϕ = 12 (−d + µ(d)), 12 (d + µ(d)) = [− d2 , d2 ] =: [1 , 2 ].
D U A L S C A L I N G F U N C T I O N S A S S O C I AT E D T O B - S P L I N E S
39
(c) The refinement equation reads d ϕ(x)
=
2
21−d
k=1
d k + d2
d ϕ(2x−k)
=:
2
ak ϕ(2x−k). (2.41)
k=1
(d) The derivatives satisfy d d ϕ(x) = d−1 ϕ(x + µ(d − 1)) − d−1 ϕ(x − µ(d)). dx
(2.42)
Moreover, the following will turn out to be crucial, see again [101] for a proof. Proposition 2.12 The cardinal B-spline basis d Φj := {d ϕ[j,k] : k ∈ Z} is uniformly stable, i.e.
ck d ϕ[j,k]
k∈Z
0;R
# ∼
$1/2 2
|ck |
k∈Z
holds with constants which are independent of j.
2.7
Dual scaling functions associated to B-splines
Having a sequence {Sj }j∈N0 of nested spaces and a uniformly stable basis Φj at ˇ j of Sj , e.g. by hand, one can of course always obtain an orthonormal basis Φ Gram–Schmidt. However, this will in general lead to globally supported functions ϕˇj,k which are only of limited use for numerical purposes. One can easily construct examples of functions ϕ which generate a sequence {Sj }j∈N0 of nested spaces but such that the corresponding Φj do not form a stable basis. What might be the specific characteristics of uniformly stable systems? With tools from functional analysis (the Riesz representation theorem) one can show that stability is ensured if there exists a function ϕ˜ ∈ L2 (R) such that (ϕ(· − k), ϕ) ˜ 0;R = δ0,k ,
k ∈ Z.
(2.43)
It is clear that ϕ˜ is not uniquely determined. If Φ0 is an orthonormal system, then one can obviously choose ϕ˜ = ϕ. In general, we have ϕ˜ ∈ / S0 . Of course, if one enforces ϕ˜ ∈ S0 , then ϕ˜ is uniquely determined (see, e.g. [175]), but is in general not compactly supported. The freedom in choosing a particular ϕ˜ out of several possibilities will be of importance in the sequel.
40
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
It was shown in [147] that for a compactly supported refinable function ϕ there always exists a compactly supported refinable function ϕ˜ satisfying (2.43). Thus, ϕ˜ is called a dual scaling function. The generated so-called dual MRA is denoted by S˜ := {S˜j }j∈Z . One often uses the notation primal and dual MRA. We will use the centralized cardinal B-splines as generators of a primal MRA. The dual MRA can be generated by a whole variety of scaling functions that have been constructed by Albert Cohen, Ingrid Daubechies, and Jean-Christophe Feauveau [71]. For any d ∈ N, they constructed a whole family of compactly supported refinable functions ˜ ˜ ∈ L2 (R) N d,d indexed by d˜ such that d + d˜ is even. These functions are dual to Nd , i.e. ˜ ˜(· − k))0;R = δ0,k , (Nd , N d,d and, by shifting for the centralized version
k ∈ Z,
˜ d,d˜ϕ
(d ϕ, d,d˜ϕ(· ˜ − k))0;R = δ0,k ,
k ∈ Z,
for any such d˜ and the regularity (and support length) increases proportionally ˜ Figures 2.11, 2.12, 2.13 and 2.14 contain the graphs of the cardinal B-spline to d. ˜ ˜ for different of order d = 1, . . . , 4 together with different dual functions (i.e. N d,d ˜ Let us summarize the precise statements of the properties of the values of d). centralized dual functions for later reference: Proposition 2.13 The dual functions fulfill: (i)
˜ d,d˜ϕ
has compact support, suppd,d˜ϕ˜ = [1 − d˜ + 1, 2 + d˜ − 1] =: [˜1 , ˜2 ];
(2.44) ˜
2 ˜ = {˜ is refinable with a finitely supported mask a ak }k= ; ˜1 ˜ is symmetric, i.e. d,d˜ϕ(x ˜ + µ(d)) = d,d˜ϕ(−x), ˜ x ∈ R; d,d˜ϕ ˜ ˜ is exact of order d, i.e. precisely all polynomials of degree less d,d˜ϕ than d˜ can be represented as linear combinations of the translates ˜ − k), k ∈ Z; d,d˜ϕ(· ˜ (v) the regularity of d,d˜ϕ˜ increases proportionally with d.
(ii) (iii) (iv)
˜ d,d˜ϕ
D U A L S C A L I N G F U N C T I O N S A S S O C I AT E D T O B - S P L I N E S
41
1.2 1 0.8 0.6 0.4 0.2 0
0
0.2
0.4
0.6
0.8
1
1.2
1.2
1.2
1
1
1
0.8
0.8
0.8
1.2
0.6
0.6
0.6
0.4
0.4 0.4 0.2 0
0
0.2
0.4
0.6
0.8
1
1.2
0.2
0.2
0
0
–0.2
–0.2 –2
–1
0
1
2
3
–0.4 –4 –3 –2 –1
0
1
2
3
4
5
Fig. 2.11 Cardinal B-spline of order d = 1 (top) and dual functions of orders d˜ = 1, 3, 5 (bottom, from left to right). 1.2 1 0.8 0.6 0.4 0.2 0 6 5 4 3 2 1 0 –1 –2 –3 –2 –1.5 –1 –0.5
–1
–0.5
0
0.5
1
2
2
1.5
1.5
1
1
0.5 0.5
0
0
–0.5 0
0.5
1
1.5
2
–1 –4
–3
–2
–1
0
1
2
3
4
–0.5 –6
–4
–2
0
2
4
6
Fig. 2.12 Cardinal B-spline of order d = 2 (top) and dual functions of orders d˜ = 2, 4, 6 (bottom, from left to right).
Since the proof of this proposition requires many details of the construction of ˜ we omit it here and refer the reader to [71, 87]. Some of the refinement d,d˜ϕ, coefficients a ˜k of d,d˜ϕ˜ are listed in Tables 2.6 and 2.7. Remark 2.14 Note that the values for the case d = d˜= 4 in Table 2.7 are not the B-spline coefficients. In fact, in standard software tools, in those cases
42
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
4
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 –1 2
3
1.5
2 1 0
–3 –3
–2
–1
0
1
2
3
4
0
0.5
1
1.5
2 2 1.5
1
1
0.5
0.5
0
0
–0.5
–0.5
–1 –2
–0.5
–1
–4
–2
0
2
4
6
–1
–6
–4
–2
0
2
4
6
8
Fig. 2.13 Cardinal B-spline of order d = 3 (top) and dual functions of orders d˜ = 3, 5, 7 (bottom, from left to right).
0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 –2 –1.5 –1 –0.5
0
0.5
1
1.5
2
80
8
4
60
6
3
40
4
2
20
2
0
0
–20
–2
–40
–4
–60
–6
–4
–2
0
2
4
1 0 –1 –6
–4
–2
0
2
4
6
–2
–8 –6 –4 –2
0
2
4
6
8
Fig. 2.14 Cardinal B-spline of order d = 4 (left) and dual function of order d˜ = 4, 6, 8 (bottom, from left to right). where the dual functions do not have good quantitative stability properties, the refinement coefficients are replaced by other numbers representing more stable systems with the same order of polynomial exactness. This is also the case for d = 9, d˜ = 7 (which is not defined in our above setting) used in the jpeg2000 standard. One easily checks that the symmetry properties (2.40) and (iii), have the following discrete counterparts
M U LT I L E V E L P RO J E C T O R S
43
Table 2.6 Refinement coefficients of B-spline scaling functions ˜ and dual scaling functions for d = 1, 2 and some values of d. d
ak
d˜
1
1.0 1.0
1
1.00000000000000 1.00000000000000
3
−0.12500000000000
a ˜k
d
ak
d˜
a ˜k
2
0.5 1.0
2
−0.25000000000000 0.50000000000000
0.5
1.50000000000000
0.12500000000000 1.00000000000000 1.00000000000000 0.12500000000000 −0.12500000000000 5
0.50000000000000 −0.25000000000000 4
0.02343750000000 −0.02343750000000 −0.17187500000000 0.17187500000000 1.00000000000000 1.00000000000000 0.17187500000000 −0.17187500000000 −0.02343750000000 0.02343750000000
ak = aµ(d)−k ,
0.04687500000000 −0.09375000000000 −0.25000000000000 0.59375000000000 1.40625000000000 0.59375000000000 −0.25000000000000 −0.09375000000000 0.04687500000000
6
a ˜k = a ˜µ(d)−k ,
−0.00976562500000 0.01953125000000 0.06640625000000 −0.15234375000000 −0.24023437500000 0.63281250000000 1.36718750000000 0.63281250000000 −0.24023437500000 −0.15234375000000 0.06640625000000 0.01953125000000 −0.00976562500000 k ∈ Z.
(2.45)
˜ ˜ are not splines. Again, no closed One can already see from the figures that N d,d formula exists; the functions are only determined by their masks. 2.8
Multilevel projectors
As we have seen in our two introductory examples, an MRA is closely linked to a series of projectors Pj : Ω → Sj . Since orthonormal bases Φj are a special
44
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
Table 2.7 Refinement coefficients of B-spline scaling functions and dual ˜ scaling functions for d = 3, 4 and some values of d. d ak
d˜
d
a ˜k
d˜
ak
a ˜k
3 0.25 3 0.093750000000 41 −0.09127176311391 41 0.05349751482200 0.75 −0.281250000000 −0.05754352622794 −0.03372823688600 0.75 −0.218750000000 0.59127176311341 −0.15644653305800 0.25 1.406250000000 1.11508705245689 0.53372823688600 1.406250000000 −0.218750000000 −0.281250000000 0.093750000000
0.59127176311341 −0.05754352622794 −0.09127176311391
5 −0.019531250000
1.20589803647200 0.53372823688600 −0.15644653305800 −0.03372823688600 0.05349751482200
0.058593750000 0.074218750000 −0.378906250000 −0.101562500000 1.367187500000 1.367187500000 −0.101562500000 −0.378906250000 0.074218750000 0.058593750000 −0.019531250000 1 See
Remark 2.14.
˜ j , we will concentrate on the latter ones here. We case of biorthogonal ones Φj , Φ have two families of operators Pj : Ω → Sj defined by Pj f :=
(f, ϕ˜j,k )0;Ω ϕj,k ,
P˜j : Ω → S˜j
P˜j f :=
k∈Ij
(f, ϕj,k )0;Ω ϕ˜j,k .
(2.46)
k∈Ij
˜ j are labeled by the same index set Ij , Note that it is important that Φj and Φ otherwise biorthogonality (ϕj,k , ϕ˜j,m )0;Ω = δk,m ,
k, m ∈ Ij ,
would make no sense. In the shift-invariant case, this is no problem since ˜ j are finite systems (as on bounded Ij = Z = I˜j , but it is important if Φj and Φ hat domains Ω like ΦHaar and Φ on [0, 1]). j j
M U LT I L E V E L P RO J E C T O R S
45
Let us collect some important properties of these operators. Proposition 2.15 The operators Pj and P˜j defined by (2.46) have the following properties: (a) they are projectors, i.e. Pj2 = Pj , P˜j2 = P˜j ; (b) Pj+1 Pj = Pj Pj+1 = Pj as well as P˜j+1 P˜j = P˜j P˜j+1 = P˜j ; (c) Pj = P˜j , P˜j = Pj , where L denotes the adjoint of an operator L.
Proof All arguments for P˜j are completely analogous to those for Pj . Thus, we concentrate on the primal part in the remainder of the proof. By definition, we have for any f ∈ L2 (Ω) that Pj2 f = (f, ϕ˜j,m )0;Ω ϕj,m , ϕ˜j,k ϕj,k m∈Ij
k∈Ij
=
k∈Ij m∈Ij
=
0;Ω
(f, ϕ˜j,m )0;Ω (ϕj,m , ϕ˜j,k )0;Ω ϕj,k ) *+ , =δk,m
(f, ϕ˜j,k )0;Ω ϕj,k ,
k∈Ij
which proves (a) for Pj . Using the refinement relation, we obtain Pj+1 Pj f = (f, ϕ˜j,m )0;Ω ϕj,m , ϕ˜j+1,k ϕj+1,k k∈Ij+1
=
m∈Ij
(f, ϕ˜j,m )0;Ω
k∈Ij+1 m∈Ij
=
l∈Ij+1
(f, ϕ˜j,m )0;Ω
m∈Ij
=
0;Ω
ajm,l (ϕj+1,l , ϕ˜j+1,k )0;Ω ϕj+1,k ) *+ , =δl,k
(f, ϕ˜j,m )0;Ω ajm,k ϕj+1,k
k∈Ij+1 m∈Ij
=
ajm,k ϕj+1,k
k∈Ij+1
(f, ϕ˜j,m )0;Ω ϕj,m = Pj f,
m∈Ij
i.e. the first claim in (b). Similarly, we have Pj Pj+1 f = (f, ϕ˜j+1,m )0;Ω ϕj+1,m , ϕ˜j,k k∈Ij
m∈Ij+1
0;Ω
ϕj,k
46
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
=
k∈Ij m∈Ij+1
=
=
a ˜jk,l (ϕj+1,m , ϕ˜j+1,l )0;Ω ϕj,k ) *+ , =δm,l
a ˜jk,m ϕ˜j+1,m
m∈Ij+1
k∈Ij
l∈Ij+1
f,
(f, ϕ˜j+1,m )0;Ω
ϕj,k
0;Ω
(f, ϕ˜j,k )0;Ω ϕj,k = Pj f,
k∈Ij
which shows also the second part of (b). Finally, as for (c) (Pj f, g)0;Ω =
(f, ϕ˜j,k )0;Ω (ϕj,k , g)0;Ω = (f, P˜j g)0;Ω
k∈Ij
for all f, g ∈ L2 (Ω), i.e. (c) is proven.
2.9
Approximation properties
One motivation to consider more general systems than the Haar or the hat function system is to obtain approximations of higher order. Of course, this has to be made mathematically rigorous. It is well known that the order of approximation is linked to the maximal degree of polynomials that can be represented by the basis functions (this is often called a Bramble–Hilbert type argument). Since all spaces Sj , j ∈ Z, are subspaces of L2 (R), but Pd (R) is not a subspace of L2 (R), we have to consider Sjloc :=
f=
ck ϕ[j,k] : {ck }k∈Z ∈ (Z) = span (Φj ),
k∈Z
where (Z) is the space of sequences on R labeled over Z. For bounded Ω ⊂ R this is of course not needed. The difference with respect to the definition of Sj lies in the fact that we do not require that the coefficient vector is in 2 (Z). Hence, Sjloc ⊂ L2 (R). Now it can be seen that the degree of polynomials contained in Sjloc determines the rate of convergence of the best approximation in Sj . In order to formulate this statement, it is useful to pose some assumptions that are trivially satisfied for the above examples. 2.9.1
A general framework
Since we will frequently use the subsequent approximation result also in more general settings (not only for basis functions ϕ[j,k] and L2 (R)), it pays off to
A P P ROX I M AT I O N P RO P E RT I E S
47
introduce a general framework now. We assume that we have a set of functions Φj = {ϕj,k : k ∈ Ij },
˜ j = {ϕ˜j,k : k ∈ Ij }, Φ
in L2 (Ω), where Ij is a suitable set of indices (e.g. Ij ⊂ Zm ) and Ω ⊆ Rn is a Lipschitz domain. These sets should generate biorthogonal MRAs S, S˜ in L2 (Ω), i.e. Sj = closL2 (Ω) span(Φj ),
˜ j ), S˜j = closL2 (Ω) span(Φ
and ˜ j )0;Ω = (ϕj,k , ϕj,m )0;Ω = IIj ×Ij (Φj , Φ k,m∈Ij (the identity matrix on Ij ). Still we assume that j ∈ Z is a parameter reflecting the level whereas k encodes information on the location of ϕj,k (e.g. the center of its supports), which generalizes the shift-parameter k ∈ Z used so far. To be precise, we pose the following assumptions. For convenience, we abbreviate σj,k := supp ϕj,k ,
σ ˜j,k := supp ϕ˜j,k .
(2.47)
Assumption 2.16 Let j,k = σj,k ∪ σ ˜j,k , k ∈ Ij . Then, we assume ˜ j are locally finite, i.e. (a) that Φj and Φ #{m ∈ Ij : j,k ∩ j,m = ∅} 1 (which means that only a fixed number of functions on the same level overlap); (b) that the size of the support decreases exponentially with the level, independently of k, i.e. |j,k | 2−j ,
k ∈ Ij ;
(c) that the L2 -norm of the translates is uniformly bounded, i.e.
ϕj,k 0;Ω 1,
ϕ˜j,k 0;Ω 1,
k ∈ Ij .
It is readily seen that the above assumptions are trivially satisfied for the system {ϕ[j,k] : k ∈ Z}
48
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
in L2 (R) arising from integer translates of a single compactly supported scaling function ϕ. However, ϕj,k may not be just a shifted scaled version of ϕ as in the periodic or shift-invariant case. 2.9.2
Stability properties
Under the above general assumptions, we start by investigating the stability of the bases. ˜ j are Proposition 2.17 Under Assumption 2.16, we have that Φj and Φ uniformly stable, i.e.
cT Φj 0;Ω ∼ c 2 (Ij ) ,
˜ j 0;Ω ∼ c (I )
cT Φ 2 j
independent of j, where we use the abbreviation ck ϕj,k . cT Φj :=
(2.48)
k∈Ij
Proof Let us abbreviate Ij,k := {k ∈ Ij : σj,k ∩ σj,k = ∅} i.e. the set of those indices corresponding to functions that overlap ϕj,k on level j. Then, we use the triangle inequality and (c) of Assumption 2.16 to obtain
2
cT Φj 20;σj,k = c ϕ k j,k
0;σj,k
k ∈Ij,k
2 |ck |
k ∈Ij,k
|ck |2 ,
k ∈Ij,k
since #Ij,k 1 by (c). Using (a), we obtain by summing over all k ∈ Ij
cT Φj 20;Ω
cT Φj 20;σj,k
k∈Ij
k∈Ij k ∈Ij,k
|ck |2 c 22 (Ij ) .
Furthermore, let vj := cT Φj . Then we have by Assumption 2.16 (a) and (c) 2 |ck |2 = |(vj , ϕ˜j,k )0;Ω |2 < ∼ vj 0;˜σj,k .
A P P ROX I M AT I O N P RO P E RT I E S
49
Again we sum over all k ∈ Ij yielding 2 T 2
c 22 (Ij ) = |ck |2 <
vj 20;˜σj,k < ∼ ∼ vj 0;Ω = c Φj 0;Ω k∈Ij
k∈Ij
which proves the claim. 2.9.3
Error estimates
Now, we consider the error induced by the approximation of a function in a MRA space Sj . Before proving an approximation result, let us recall an important statement from approximation theory which will be needed. It is also known as a Whitney type estimate. For convenience, we formulate this result and also some subsequent statements directly in the multivariate n-dimensional case. Theorem 2.18 If Ih is an n-dimensional cube of side length h > 0, we obtain inf f − p 0;Ih hm+1 f (m+1) 0;Ih
p∈Pd
for all functions f : Rn → R such that the derivative of order m + 1 is in L2 , i.e. f (m+1) ∈ L2 (Ih ) for some 0 ≤ m ≤ d.
Proof See, e.g. [65, Thm. 25.2] and references therein.
Remark 2.19 We will see later that the above result holds in much more generality. In particular, the above formulation seems to require that f ∈ C m+1 , which is of course a very strong assumption. The same estimate can be obtained for piecewise differentiable functions. This leads to weak derivatives and Sobolev spaces, see also Appendix A. Obviously, a Whitney type estimate allows us to control the error of the best polynomial approximation to a given function f . It is important to note that Theorem 2.18 gives a bound on the local approximation. This is very useful for the investigation of the approximation error with respect to an MRA. Proposition 2.20 (Jackson inequality) Let Assumption 2.16 hold. Under the assumption Pd−1 ⊂ S0loc we have −js (s) inf f − vj 0;Ω < ∼ 2 f 0;Ω ,
vj ∈Sj
if f (s) ∈ L2 (Ω).
s ≤ d,
50
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
Proof Since Pd−1 ⊂ S0loc ⊂ Sjloc for any j ≥ 0, one has for any p ∈ Pd−1 that Pj p = p and then we get by the triangle inequality
f − Pj f 0;j,k ≤ f − p 0;j,k + Pj (f − p) 0;j,k ≤ f − p 0;j,k + |(f − p, ϕ˜j,m )0;Ω | ϕj,m 0;Ω . m∈Ij j,k ∩j,m =∅
In view of Assumption 2.16, we have by the Cauchy–Schwarz inequality that inf
p∈Pd−1
|(f − p, ϕ˜j,m )0;Ω | ϕj,m 0;Ω
inf
p∈Pd−1
f − p 0;j,m 2−sj f (s) 0;j,m ,
for s ≤ d, where we have used the Whitney estimate in Theorem 2.18 in the last step. Thus, we finally have
f − Pj f 20;j,k 2−2sj f (s) 20;j,k
f − Pj f 20;Ω k∈Ij
k∈Ij
2−2sj f (s) 20;Ω , since only a fixed number of j,k overlap.
Let us finally comment on the value of d which determines the maximal rate of convergence in Proposition 2.20. For B-splines this is obviously the order of the ˜ B-splines, and for the corresponding dual scaling functions this value is d. The maximal degree of polynomials that can be reproduced by integer translates of Daubechies orthonormal scaling functions N ϕ is N − 1. Using similar arguments applied to derivatives of f gives an analogous estimate also for derivatives of the approximation error. Proposition 2.21 Under the above assumptions we have for Sj H m (Ω) −j(s−m) inf (f − vj )(m) 0;Ω <
f (s) 0;Ω , ∼ 2
vj ∈Sj
⊂
0 ≤ m < s ≤ d, (2.49)
if f (s) ∈ L2 (Ω). Proof Exercise 2.6.
2.10
Plotting scaling functions
Not only for graphical reasons is it important to produce plots of a refinable function. If the numerical solution of a partial differential equation is given as
PLOTTING SCALING FUNCTIONS
51
a linear combination of translates of scaling functions, this solution has to be visualized. Another application is the computation of ϕ(x) for a given point x ∈ R. Of course, if ϕ is given by a closed representation, ϕ(x) can also be computed directly. This is the case, e.g. for cardinal B-spline scaling functions. However, as we have seen, there are important cases, where ϕ cannot be given in closed form but only implicitly in terms of the mask coefficients. For this case, one has to introduce special methods. 2.10.1
Subdivision
Let us consider a compactly supported refinable function ϕ with finite mask a = (ak )1 ≤k≤2 . Using the refinement equation, we obtain for integers m ∈ Z ϕ(m) =
2
2m− 1
ak ϕ(2m − k) =
k=1
a2m−k ϕ(k),
k=2m−2 k∈{1 ,2 }
in other words v = A v,
(2.50)
where v := (ϕ(k))1 ≤k≤2 is the vector of nonzero integer point values of ϕ and A = (a2m−k )m,k=1 ,...,2 is a matrix consisting of the mask coefficients of ϕ. Thus, one can determine all nonzero values ϕ(m), m ∈ Z, by solving the eigenvalue problem (2.50), i.e. one has to compute the eigenvector v of the matrix A corresponding to the eigenvalue 1. This observation also holds in the multivariate case as the following result shows. Proposition 2.22 Let ϕ : Rn → R be an a-refinable function, i.e. ak ϕ(2x − k), a = (ak )k∈Zn , ϕ(x) = k∈Zn
of compact support such that Rn
ϕ(x) dx = 0.
52
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
Then, v := (ϕ(α))α∈Γ , Γ := Zn ∩ supp ϕ, is a finite sequence satisfying the eigenvalue relation v = A v, where again A = (Aα,β )α,β∈Γ , Aα,β := a2α−β .
Proof Exercise 2.7
Now, we turn to the numerical solution of the eigenvector problem (2.50). It is well known that easy and efficient methods are available for the computation of the largest simple eigenvalue. Let us investigate under which condition this is ensured. It turns out that this is related to the subdivision operator Sa defined in (2.34) above. Here we use the index a to identify the dependence of the mask. Proposition 2.23 Let ϕ be an a-refinable function of compact support whose corresponding subdivision scheme Sa converges. Then, v is the unique vector that satisfies (2.50) and vα = 1. (2.51) α∈Zn
Proof Defining (SaT c)α :=
a2α−β cβ = (Ac)α ,
β∈Zn
we obtain (Rc, Sa d)2 (Zn ) = (SaT c, Rd)2 (Zn ) , where (Rc)α := c−α , for α ∈ Zn and all c, d ∈ 2 (Zn ). In fact (Rc, Sa d)2 (Zn ) =
α∈Zn
=
c−α
aα−2β dβ
β∈Zn
α∈Zn β∈Zn
cα a2β−α d−β
(2.52)
PLOTTING SCALING FUNCTIONS
=
#
β∈Zn
53
$ a2β−α cα
d−β
α∈Zn
= SaT c, Rd 2 (Zn ) , i.e. SaT w = w = A w, where w = (wα )α∈Zn is any compactly supported sequence satisfying (2.50) and (2.51). Hence, using (Rc, d)2 (Zn ) = (c, Rd)2 (Zn ) ,
c, d ∈ (Zn ),
we obtain (Rw, Sak d)2 (Zn ) = (SaT w, RSak−1 d)2 (Zn ) = (w, RSak−1 d)2 (Zn ) = (Rw, Sak−1 d)2 (Zn ) = · · · = (w, Rd)2 (Zn )
(2.53)
for all k ≥ 1 and all d ∈ (Zn ). On the other hand, since w is compactly supported and Sa is convergent, we have by (2.36) that
lim (Rw, Sak d)2 (Zn ) = lim
k→∞
k→∞
w−α (Sak d)α =
α∈Zn
= fd (0)
α∈Zn
w−α lim (Sak d)α k→∞
w−α = fd (0)
α∈Zn
by (2.51). Moreover, by Proposition 2.6, we continue fd (0) =
α∈Zn
dα ϕ(−α) =
ϕ(α) d−α = (v, Rd)2 (Zn ) ,
α∈Zn
i.e. lim (Rw, Sak d)2 (Zn ) = fd (0) = (v, Rd)2 (Zn ) , d ∈ 2 (Zn ).
k→∞
Since d ∈ 2 (Zn ) can be chosen arbitrarily, we conclude v = w by comparing the latter equation with (2.53).
Proposition 2.24 Under the assumptions of Proposition 2.23, we have that λmax (A) = 1.
54
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
Proof The convergence of a subdivision scheme Sa implies
A = Sa ≤ 1,
which means that 1 is the largest eigenvalue of A.
Finally, the normalization (2.51) is already implied by (2.37) in Proposition 2.6, so that we are left with the computation of the eigenvector v corresponding to the largest simple eigenvalue λmax (A) = 1 of the matrix A. This can be done by using a relatively simple power iteration. A few iterations suffice to determine the eigenvector up to machine accuracy. In each iteration it amounts to performing a matrix-vector multiplication. Due to the particular structure of A this can be done in an efficient manner. Given the values ϕ(m) at the integers m ∈ Z, we can easily use the refinement equation to compute the values of ϕ at any dyadic grid am ϕ(21−j k − m), ϕ(2−j k) = m∈Z
where the above sum is finite since a and v are finite. Let us end this section with the precise statement for a more general framework also involving derivatives of refinable functions, see Dahmen and Micchelli [89, Thm. 2.1]. Theorem 2.25 Let ϕ ∈ C0m (Rn ) be an a-refinable function with compact support and define (V µ )α :=
dµ ϕ(α), dxµ
α ∈ Zn ,
µ ∈ Nn0 .
Then, V µ = {(V µ )α : α ∈ Zn } is a finite sequence satisfying the eigenvalue relation 2−|µ| V µ = AV µ , where A = (Aα,β )α,β∈Zn , Aα,β := a2α−β and Qµ (−α)(V ν )α = δµ,ν α∈Zn
for all |ν|, |µ| ≤ m and Qµ uniquely defined by α∈Zn
Qµ (α)ϕ(x − α) =
xµ for x ∈ Rn and |µ| ≤ m µ!
PLOTTING SCALING FUNCTIONS 6e + 07 5e + 07 4e + 07 3e + 07 2e + 07 1e + 07 0 –1e + 07 –2e + 07 –3e + 07 –2 –1.5 –1 –0.5
0
0.5
1
1.5
6 5 4 3 2 1 0 –1 –2 –3 2 –2 –1.5
–1 –0.5
55
0
0.5
1
1.5
2
Fig. 2.15 Example d = d˜ = 2, where the largest eigenvalue is not simple. Wrong solution (left) and correct function (right). Here, we use the standard notation |µ| :=
n
µi
i=1
for a multiindex µ ∈ Nn0 . Note that the assumption ϕ ∈ C0m (Rn ) in Theorem 2.25 seems to be very restrictive. In fact, the practice shows that the above statement also holds under weaker conditions. On the other hand, the result is definitely not true for all refinable functions as the following examples show. Example 2.26 Consider the case n = 1, µ = 0 and the dual B-spline scaling function 2,2 ϕ˜ of order d = d˜ = 2. In this case, the eigenvalues are 12 , 1 and − 14 , where 1 is the largest eigenvalue, but it is not simple. The elements of the eigenvector (0, 1, −2, 1, 0)T corresponding to λ = 1 sum up to 0. Moreover, the condition number of the matrix (of dimension 5 × 5) is already 42.75. Figure 2.15 shows the incorrect result that is produced by the algorithm. Example 2.27 In the cases d = d˜> 2 the largest eigenvalue is not 1. This holds also for d = 4, d˜ = 6. Using λ = 1 causes wrong results except in the case d = d˜ = 3. Figure 2.16 shows the corresponding results. 2.10.2
Cascade algorithm
A few subdivision iterations typically suffice for plotting the graph of ϕ with sufficient accuracy. However, this method is based on the eigenvector problem for the computation of the values on the integer lattice. As already mentioned above, this problem is not always uniquely solvable. Hence, we describe in Algorithm 2.1 (p. 56) an alternative which is usually referred to as the cascade algorithm. As a preparation for the next error estimate, we recall the following definition.
56
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
1.5e+14 1e+14 5e+13 0 –5e+13 –1e+14 –1.5e+14 –3
–2
–1
0
1
2
3
4
2.5e+15 2e+15 1.5e+15 1e+15 5e+14 0 –5e+14 –1e+15 –1.5e+15 –2e+15 –2.5e+15 –3e+15
10 8 6 4 2 0 –2 –4 –6 –4
–2
0
2
–8
4
4
40
8
3
30
6
2
20
1
10
–6
–4
–2
0
2
4
6
–6
–4
–2
0
2
4
6
4 2
0
0
–1
–10
–2
–20
–3 –3
–30 –2
–1
0
1
2
3
4
0 –2 –4
–2
0
2
–4
4
Fig. 2.16 Examples d = d˜ = 3, d = d˜ = 4 and d = 4, d˜ = 6 (from left to right). First row: results with subdivision, second row with cascade.
Definition 2.28 Let f : Ω → R be a function and Ω ⊂ Rn an open, bounded domain. For 0 < α ≤ 1 ¯ := sup |f (x) − f (y)| h¨ olα (f, Ω)
x − y α ¯ x,y∈Ω ¯ and lip(f, Ω) ¯ := h¨ ¯ the is called the H¨older constant of f on Ω ol1 (f, Ω) Lipschitz constant. Then, we define the H¨older spaces as ¯ := {f ∈ C m (Ω) ¯ : h¨ ¯ < ∞ for |s| = m}, C m,α (Ω) olα (∂ s f, Ω)
f C m,α (Ω) ¯ :=
∂ s f sup +
|s|≤m
¯ h¨ olα (∂ s f, Ω).
|s|=m
¯ is called Lipschitz continuous. A function f ∈ C 0,1 (Ω)
Algorithm 2.1 Cascade algorithm Require: Refinement coefficients are normalized by
ak = 2.
k
1: Start with the sequence η0 = δ0 , i.e., η0 (n) = δ0,n , n ∈ Z. 2: Compute ηj (2−j n), n ∈ N, by
P E R I O D I Z AT I O N
ηj (2−j n) =
57
an−2 ηj−1 (2−j ),
if n is even;
∈Z
an−1−2 ηj−1 (2−j ), if n is odd. ∈Z
3: Determine a piecewise constant or piecewise linear interpolation of the computed values.
The following statement gives an error estimate for the computational values ηj produced by the cascade algorithm.
Proposition 2.29 If ϕ is a-refinable and H¨ older continuous with exponent α, then there exists a j0 ∈ N such that for j ≥ j0 −αj
ϕ − ηj ∞;R < . ∼ 2
Proof [97, Prop. 6.5.2, p. 205].
General results on the H¨ older smoothness of refinable functions can be found, e.g. in [70]. 2.11
Periodization
As we have seen above, the shift-invariant setting on R is the most convenient framework for the analysis of MRAs. It also allows us to design efficient algorithms. On the other hand, the whole Euclidean space is of course not an appropriate model, e.g. for boundary value problems. However, if a boundary value problem is periodic, i.e. the desired solution is periodic, then one can easily work with periodized shift-invariant spaces. We now describe this periodization. For any function f ∈ C(R), we define [f ]j,k (x) :=
f[j,k] (x − l),
x ∈ [0, 1], j, k ∈ Z,
l∈Z
or [f ]j,k (x) :=
l∈Z
f[j,k] (x − l) χ[0,1) (x),
x ∈ R, j, k ∈ Z.
(2.54)
58
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
1.5
1.5
1
1
1
0.5
0.5
0.5
1.5
0
0
0
–0.5
–0.5
–0.5
–1
–1
–1.5 –1
–1.5 –1
–0.5
0
0.5
1
–1 –1.5 –0.5
0
0.5
1
1.5
1
0
0.2
0.4
0.6
0.8
1
Fig. 2.17 Periodization of a compactly supported function f . Original function f (left), its 1-periodic extension (middle) and periodization [f ]0,0 . Note that the above sum is finite as long as f is compactly supported. Equation (2.54) defines a periodized scaled and shifted variant of f . In fact, it is almost trivial to see that f[j,k] (1 − l)|[0,1) = f[j,k] (l)|[0,1) = [f ]j,k (0). [f ]j,k (1) = l∈Z
l∈Z
Moreover, derivatives are also periodic in the case of existence. In fact, if f ∈ C m (R), m ≥ 0, we have dm (m) 2jm f[j,k] (x − l)|[0,1) , [f ] (x) = j,k dxm l∈Z
so that dm dm [f ] (1) = [f ]j,k (0). j,k dxm dxm Moreover, it is trivially seen that [f ]j,k = [f ]j,k+2j . Then, we define Sjper := span Φper j ,
Φper := {[ϕ]j,k : k = 0, . . . , 2j − 1}, j
m and it is obvious that Sjper ⊂ Cper ([0, 1)] if ϕ ∈ C m (R), where m ([0, 1]) := {f ∈ C m ([0, 1]) : f (l) (0) = f (l) (1), l = 0, . . . , m}. Cper
An example of a periodized function is shown in Figure 2.17. 2.12
Exercises and programs
Exercise 2.1 Determine the piecewise constant approximation PjI f and PjO f according to (2.10) of the following functions (a) f (x) = χ[0,1] (x), (b) f (x) = x χ[0,1] (x),
E X E RC I S E S A N D P RO G R A M S
59
(c) f (x) = x2 χ[0,1] (x), (d) f (x) = sin(2πx) χ[0,1] (x), for arbitrary values of j ∈ Z. Exercise 2.2 Determine the limit of the subdivision algorithm applied to a sequence c ∈ ∞ (Z) for the mask defined by a−1 = a1 = 12 , a0 = 1 and zero otherwise. j Exercise 2.3 Determine the coefficients αr,k in
xr =
j αr,k ϕ˜[j,k] ,
k∈Z
i.e. j αr,k
= R
xr ϕ[j,k] (x) dx
for arbitrary ϕ. ˜ Start with r = 0 and investigate the dependence on the level j. In the next step, develop recursions of the form j αr,k
=
r
j j βl,i αi,l .
i=0 l∈Z
Exercise 2.4 Show the following relations: (a) ϕ[j,k] (x) dx = 2−j/2 , k ∈ Z, R
(b) ϕ[j,k] 0;R = 1, k ∈ Z. Exercise 2.5 Prove Proposition 2.10, and show in particular: (1) suppNd = [0, d]. (2) Nd (x) if 0 < x < d. >0 d d + x = Nd ( − x). (3) Nd 2 2 (4) Nd (x − k) = 1, x ∈ R. k∈Z Nd (x) dx = 1. (5) R
(6) Nd (x) = Nd−1 (x) − Nd−1 (x − 1). x d−x Nd−1 (x) + Nd−1 (x − 1). (7) Nd (x) = d−1 d−1
60
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
d 1 k d (x − k)d−1 (8) Nd (x) = (−1) + , k (d − 1)! k=0 where xn+ := (x+ )n := χ[0,∞) xn . d d 1−d Nd (2x − k). (9) Nd (x) = 2 k k=0
Exercise 2.6 Prove Proposition 2.21. Exercise 2.7 Show Proposition 2.22. Exercise 2.8 Show the relations (2.20) and (2.21) of the Rayleigh quotient. Exercise 2.9 (1) Show that for any d ∈ N and for any polynomial p ∈ Pd−1 there exists a sequence c = (ck )k∈Z such that ck Nd (x − k). p(x) = k∈Z
(2) Show that for f ∈ C m ([a, b]) ∩ L2 (R), m ≤ d, the following estimate holds: inf
p∈Pd−1 (R)
f − p 0;[a,b] ≤ C (b − a)m f m;[a,b] ,
(2.55)
where C only depends on m. Hint: Use the Taylor expansion f (r + x) =
m−1 k=0
xk (k) f (r) + k!
0
x
(x − t)m−1 (m) f (r + t) dt. (m − 1)!
(3) Determine the constant C = C(m) in (2.55). Exercise 2.10 Determine exact formulas for the evaluation of the following integrals by using quadrature: (a) N2 (2j x − k) N2 (2j x − ) dx. R (b) N2 (2j x − k) N3 (2j x − ) dx. R
Exercise 2.11 Let the mask coefficients √ √ √ 1+ 3 3+ 3 3− 3 a0 = , a1 = , a2 = 4 4 4
√ 1− 3 and a3 = 4
(Daubechies with N = 2) be given. Determine the exact values ϕ( k2 ), k ∈ Z, of the corresponding scaling function.
E X E RC I S E S A N D P RO G R A M S
61
Programs Most material presented in this book is supplemented by corresponding software. These programs are written in C++. We provide the source code of the programs on the web page of the book. However, we do not assume that the reader is already an expert in C++. In order to make the usage of the programs as convenient as possible, we also provide an open source numerical software library called FLENS, Flexible Library for Efficient Numerical Solutions, which was mainly written by Michael Lehn and Alexander Stippler. The main idea of FLENS is to provide an environment that enables the user to facilitate the most efficient numerical algebra routines that are provided for basically all computer platforms in BLAS (basic linear algebra subprograms), www.netlib.org/blas. However, often the syntax of BLAS is not really userfriendly. Hence, FLENS provides an efficient and flexible interface to BLAS. This means that the user can program in a syntax which is as close to the formulation of the numerical algorithms as possible and at the same time the library ensures that the most efficient standard subprograms are used. FLENS ensures no loss of efficiency in the communication between user and BLAS. The details, a description and the latest version of FLENS can be found at http://flens.sourceforge.net/ We recommend downloading and installing FLENS before the subsequent codes are downloaded. A user-friendly guide, description and the listing of all programs can be found on the webpage of the book. FLENS serves as a platform: all wavelet related codes are based upon it. Besides the programs below, we provide the following code: • Refinement coefficients of — Daubechies’ scaling functions, — B-spline scaling functions, — dual B-spline scaling functions. • Refinement equations. • Subdivision scheme. • Plotting of refinable functions by using — the eigenvector/eigenvalue problem and subdivision, — cascade algorithm. Program 2.1 The code plot_daub_sf.cc produces plots of the Daubechies orthonormal scaling functions for a given N . Program 2.2 The code periodization.cc produces the periodization of a compactly supported function f as defined in (2.54). Computer Exercise 2.1 Write a code for the piecewise constant approximation of a given function f (that should be left arbitrary) on [0, 1] with respect to
62
M U LT I S C A L E A P P ROX I M AT I O N A N D M U LT I R E S O L U T I O N
a dyadic grid with mesh size h = 2−j . Compute the discrete error on the grid w.r.t. 2 and ∞ and plot the error over the level j. Computer Exercise 2.2 Generalize the program from Computer Exercise 2.1 to piecewise linear approximations. Computer Exercise 2.3 Write a code for the discrete convolution (a ∗ b)k := ak−m bm m∈Z
for finite input vectors a and b with supp a = [la , ua ] and supp b = [lb , ub ]. Computer Exercise 2.4 Write a code for the subdivision scheme and use it to approximate the Daubechies scaling function corresponding to N = 1, 2, 3, 4 and plot the resulting graphs.
3 ELLIPTIC BOUNDARY VALUE PROBLEMS The main focus of this book is the application of wavelet bases for the numerical solution of elliptic partial differential equations (PDE). This chapter is devoted to an introduction to elliptic boundary value problems (BVP). Readers that are familiar with this topic may skip this chapter and continue immediately with Chapter 4. In order to keep notation and technicalities as simple as possible, we start with the univariate case of elliptic two-point boundary value problems in 1D. Following the spirit of the examples of the previous chapter, we consider here homogeneous Dirichlet and periodic boundary conditions. All facts and proofs are formulated in such a way that the relevant ingredients are clearly pointed out so that they can easily be generalized also for other kinds of boundary conditions and to more realistic domain geometries, see also Appendix A. In particular, we do not use any Fourier techniques here even though they would allow some more elegant proofs in the periodic case. 3.1
A model problem
Let us start by reviewing variational formulations of elliptic partial differential equations and Galerkin methods for their discretization We mainly consider the following periodic elliptic boundary value problem. Given f : (0, 1) → R, determine u : (0, 1) → R such that −u (x) + c u(x) = f (x),
x ∈ (0, 1), c > 0,
u(0) = u(1), u (0) = u (1). (3.1)
This problem is also sometimes called the Sturm–Liouville problem and will serve as a model problem. We will also consider the following boundary value problem with homogeneous Dirichlet boundary conditions, namely −u (x) + c u(x) = f (x),
x ∈ (0, 1), c ≥ 0,
u(0) = u(1) = 0.
(3.2)
As the homogenous problem (3.2) amounts to constructing basis functions on [0, 1] (which turns out to be somewhat technical), we will first treat the numerical solution of the periodic problem (3.1) and come to (3.2) later in the book. However, we collect the analytical properties of both problems together in this chapter. Often, the analysis is simpler for the Dirichlet case. 63
64
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
3.1.1
Variational formulation
In several applications, the data f is not continuous, but only of bounded energy, i.e. in L2 . Then, the pointwise interpretation of (3.1) or (3.2) does not make sense. Thus, we consider the weak or variational formulation of (3.1) and (3.2), i.e. we multiply both sides with appropriate test functions and use integration by parts. Let us start with (3.2), the case of homogeneous Dirichlet boundary conditions. For Ω := (0, 1), we define C0∞ (Ω) := {v ∈ C ∞ (Ω) : v(0) = v(1) = 0} i.e. the space of test functions. Multiplying both sides of the partial differential equation with a test function φ ∈ C0∞ (Ω) yields 1 1 1 f (x) φ(x) dx = − u (x) φ(x) dx + c u(x) φ(x) dx 0
0
1
=
u (x) φ (x) dx + c
0
0
1
u(x) φ(x) dx
(3.3)
0
by integration by parts since φ(0) = φ(1) = 0. Note that equation (3.3) is also well-defined if u and φ are only piecewise differentiable. This motivates us to consider derivatives in a weak sense. We now recall the definition of the weak derivative. Definition 3.1 Let u ∈ L2 (Ω). A function v ∈ L2 (Ω) is called the weak derivative of u, i.e. v = u if 1 1 v(x) φ(x) dx = − u(x) φ (x) dx (3.4) 0
0
for all test functions φ ∈ C0∞ (Ω). Let us collect some examples of weak derivatives. Example 3.2 (a) If u ∈ C 1 (Ω) ∩ L2 (Ω), the classical and weak derivatives coincide. (b) Consider the Heaviside function defined by 1, if x ≥ 0, h(x) := 0, if x < 0.
A M O D E L P RO B L E M
65
Hence, h ∈ L1,loc (R), i.e. h ∈ L1 (D) for any bounded interval D := (−L, L) ⊂ R, L > 0. Then, we have for any test function φ ∈ C0∞ (D) that
−(h, φ )0;D = −
L
h(x)φ (x) dx = −
−L
L
φ (x) dx
0
= φ(0) = δ0 , φ, since φ(L) = 0 and with the Dirac distribution δ0 , i.e. h = δ0 . Here, ·, · / L2 (R) so that denotes the application of the distribution. Of course, δ0 ∈ this example shows immediate generalization of the definition of weak derivatives to L1,loc (R). (c) Consider the hat function ϕhat defined in (2.13). With 1, if x ∈ [−1, 0), ψ Haar (x) := −1, if x ∈ [0, 1], 0, else for x ∈ R, the so-called Haar wavelet, we get for any bounded interval D := (−L, L) ⊂ R, L > 1, and all test functions φ ∈ C0∞ (D) that −(φ , ϕhat )0;D = −
0
−1
(1 + x)φ (x) dx −
= −φ(0) +
1 0
(1 − x)φ (x) dx
0
−1
φ(x) dx + φ(0) −
1
φ(x) dx 0
L
φ(x)ψ Haar (x) dx
= −L
since φ is continuous at x = 0, so that we have d hat ϕ = ψ Haar . dx (d) Consider the Dirac distribution δ0 on D ⊂ R, 0 ∈ D. For any test function φ ∈ C0∞ (D), we have (φ , δ0 )0;D =
φ (x)δ0 (x) dx = φ (0).
D
This shows that δ0 is a linear functional, δ0 : C0∞ (D) → R defined as δ0 (φ) := −φ (0).
66
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
Moreover, δ0 is infinitely many times weakly differentiable and (Dm δ0 )(φ) = (−1)m φ(m) (0). Higher order weak derivatives are defined recursively. Also partial derivatives and differential operators like div, grad or ∆ are interpreted in an analogous way in a weak form. Based on this, we can define spaces of weakly differentiable functions.
Definition 3.3 Let m ∈ N. (a) The Sobolev space of order m is defined by H m (Ω) := {v ∈ L2 (Ω) : v (k) ∈ L2 (Ω), 1 ≤ k ≤ m}, where the derivatives are to be understood in the sense of (3.4). A norm on H m (Ω) is defined by #
u m;Ω :=
m
$1/2
v (k) 20;Ω
,
k=0
which is induced by the inner product (u, v)m;Ω :=
m
u(k) , v (k)
k=0
0;Ω
.
Moreover, we define the seminorm |u|m;Ω := v (m) 0;Ω . (b) The Sobolev space with generalized homogeneous Dirichlet boundary conditions is defined as H0m (Ω) := clos·m;Ω (C0∞ (Ω)) . (c) The periodic Sobolev space is defined as m ∞ (Ω) := clos·m;Ω (Cper ). Hper
Let us now come back to (3.3), namely the equation that we obtained by multiplying the original differential equation (3.2) by a smooth test function φ and
A M O D E L P RO B L E M
67
performing integration by parts. Having now the definition of weak derivatives at hand, we see that (3.3) is in fact well-defined for functions in the Sobolev space H 1 (Ω) := {v ∈ L2 (Ω) : v ∈ L2 (Ω)}, H01 (Ω) := {v ∈ H 1 (Ω), v(0) = v(1) = 0}, where the derivative v is to be understood in the weak sense. Using the test and trial space V := H01 (Ω) the weak (or variational ) formulation reads: Find u ∈ V such that a(u, v) = (f, v)0;Ω ,
v ∈ V,
(3.5)
where the bilinear form a : V × V → R is defined by a(u, v) : = (u , v )0;Ω + c (u, v)0;Ω 1 [u (x)v (x) + c u(x)v(x)] dx. =
(3.6)
0
The periodic model problem Now we will see that the variational formulation of the periodic boundary value problem (3.1) differs only slightly from the latter equation (3.6). We now test the differential equations with 1-periodic functions, i.e. ∞ := {v ∈ C ∞ (R) : v(x) = v(x + 1), x ∈ R} φ ∈ Cper
and obtain 1 0
f (x) φ(x) dx = −
1
1
u (x) φ(x) dx + c
u(x) φ(x) dx
0
1
= 0
0
u (x) φ (x) dx − (u (1) φ(1) − u (0) φ(0))
1
+c
u(x) φ(x) dx 0
1
=
1
u (x) φ (x) dx + c 0
u(x) φ(x) dx,
(3.7)
0
∞ , one where the boundary term vanishes due to periodicity. In fact, if φ ∈ Cper easily obtains
φ(k) (x) = φ(k) (x + 1)
68
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
for all x ∈ R and all integers k ∈ N0 . Thus, the variational formulation reads exactly like (3.5), (3.6) with the only difference that the trial and test space V is now defined as 1 ∞ (Ω) := clos·1;Ω (Cper ) V = Hper
with the norm · 1;Ω defined by Definition 3.3. Obviously, if u ∈ C 2 (Ω) solves the original boundary value problem (3.1) or (3.2), then it is also a solution of (3.6). On the other hand, if a function u solves (3.6) and is in addition in C 2 (Ω), then it is a solution of the original boundary value problem. This explains why a solution of (3.6) is also called a weak solution. 3.1.2
Existence and uniqueness
The first question a mathematician poses for a given problem is of course whether a solution to the problem exists and if – in the case of existence the solution is unique. Here, this question can be answered due to the properties of the bilinear form a(·, ·) which we describe now. It is easily seen by the H¨older’s inequality that a(·, ·) is bounded, i.e. a(u, v) ≤ C u 1;Ω v 1;Ω .
(3.8)
Moreover, the bilinear form is also coercive, i.e. a(u, u) ≥ α u 21;Ω .
(3.9)
The constants C and α are also known as the continuity and coercivity constants, respectively. A bounded and coercive bilinear form is called elliptic. As we will see below, these properties ensure the existence and uniqueness of a solution of the variational problem. In order to show (3.9) for the case of homogeneous boundary values, we need the following result, which is known as the Poincar´e–Friedrichs inequality. Proposition 3.4 For any bounded interval I := (a, b), there exists a constant CI = (b − a) > 0 such that
v 0;I ≤ CI v 0;I for all v ∈ H01 (I).
(3.10)
A M O D E L P RO B L E M
69
Proof We start by considering a function φ ∈ C0∞ (I). By the fundamental theorem of calculus, we have
x
φ(x) = φ(a) +
x
φ (ξ) dξ = a
φ (ξ) dξ.
a
Using the Cauchy–Schwarz inequality yields
b
2
b
x
|φ(x)| dx ≤ a
2
x
a
≤ (b − a)2
2
|φ (ξ)| dξ dx
1 dξ a
a b
|φ (ξ)|2 dξ = (b − a)2 φ 20;I .
a
Since, by definition the Sobolev space H01 (I) is the closure of C0∞ (I), the space C0∞ (I) is dense in H01 (I) so that the above inequality extends also to φ ∈ H01 (I) which proves the claim. We use Proposition 3.4 for I = Ω = (0, 1). With this preparation, (3.9) is readily seen. In fact, a(u, u) ≥
u 20;Ω
1 1 ≥
u 20;Ω + u 20;Ω ≥ min 2CΩ2 2
1 1 , 2 2CΩ2
u 21;Ω ,
i.e. (3.9) with α := min
1 1 , 2 2CΩ2
.
Of course, the Poincar´e–Friedrichs inequality does not hold for periodic functions, as the counterexample of a nontrivial constant function shows. Still, the bilinear form a(·, ·) is coercive in the sense of (3.9), which can be seen as follows. 1 (Ω) has an L2 -orthogonal decomposition Any function u ∈ Hper u(x) = u ¯ χΩ (x) + v(x), where u ¯ = u(0) contains the constant part and v ∈ H01 (Ω) (this is, e.g. obtained by the Fourier series of u). Since (χΩ , v)0;Ω = 0 we have ¯2 + v 20;Ω
u 20;Ω = u
70
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
by Parseval’s identity. Using that a(·, ·) is coercive for v ∈ H01 (Ω) we obtain by u = v a(u, u) = u 20;Ω + c u 20;Ω =u ¯2 + v 20;Ω + c v 20;Ω =u ¯2 + a(v, v) u ¯2 + v 21;Ω = u 21;Ω , which is the desired coercivity.
3.2
Variational formulation
Let us now put the above simple examples into a sufficiently general framework which allows us also to study problems in higher dimensions and also on general domains. Let V be a normed linear space and a:V ×V →R be a bounded, symmetric and positive bilinear form, i.e. u, v ∈ V,
(3.11)
u ∈ V, u = 0,
(3.12)
a(u, v) = a(v, u), a(u, u) > 0,
a(u, v) ≤ C u V v V ,
u, v ∈ V.
(3.13)
Then, we consider the following variational problem u ∈ V : a(u, v) = l, v, v ∈ V,
(3.14)
where l : V → R is a bounded linear functional l(v) = l, v ∈ R, i.e. l ∈ V , the dual space of V . The bilinear form ·, · : V × V → R is known as the dual pairing.
(3.15)
VA R I AT I O N A L FO R M U L AT I O N
71
The first step is to characterize the solution of the variational problem as a minimizer of a certain functional.
Theorem 3.5 (Characterization theorem) Let V be a normed linear space and X ⊂ V be a closed, convex subset. Then, u ∈ X solves the problem u∈X:
a(u, v) = l, v,
v∈X
if and only if u = arg min J(v), v∈X
J(v) :=
1 a(v, v) − l, v. 2
(3.16)
Moreover, there is only one minimal solution.
Proof Let u, v ∈ X and t ∈ [0, 1]. Then u + tv ∈ X and by linearity 1 a(u + tv, u + tv) − l, u + tv 2 1 1 = a(u, u) + ta(u, v) + t2 a(v, v) − l, u − tl, v 2 2 1 = J(u) + t[a(u, v) − l, v] + t2 a(v, v). 2
J(u + tv) =
(3.17)
If u ∈ X is a solution of (3.14), then we set t = 1 in (3.17) and obtain 1 J(u + v) = J(u) + a(v, v) > J(u) 2 if v = 0. Thus u is the unique minimizer of J. On the other hand, if u minimizes J, then d J(u + tv)|t=0 = [a(u, v) − l, v + ta(v, v)]t=0 dt = a(u, v) − l, v,
0=
i.e. u solves (3.14)
Existence and uniqueness of the solution of the variational problem (3.14) can be shown under rather general assumptions.
72
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
Definition 3.6 Let H be a Hilbert space with norm · , V ⊆ H. A bilinear form a:H ×H →R is called V -elliptic if (1) it is bounded, i.e. there exists a constant C > 0 (the continuity constant) such that |a(u, v)| ≤ C u H · v H ,
u, v ∈ V ;
(2) it is symmetric; (3) it is coercive, i.e. there exists a constant α > 0 (the coercivity constant) such that a(v, v) ≥ α v 2H ,
v ∈ V.
In the following, we use the standard definition of the norm in the dual space H , namely
l H := sup
v∈H
l, v ,
v H
which is sometimes also called the operator norm. Theorem 3.7 (Lax–Milgram theorem for convex sets) Let V ⊂ H be a closed, convex subset and let a : H × H → R be V -elliptic. Then, the variational problem (3.14) has a unique solution u ∈ V for any l ∈ V . Proof By Theorem 3.5 we have to show that the functional J has a unique minimizer in V . The functional J is bounded from below; in fact 1 α v 2H − l H · v H 2 1 1 (α v H − l H )2 −
l 2 = 2α 2α H
l 2 ≥− H . 2α
J(v) ≥
Thus, there exists a minimal sequence {vn }n∈N . Setting γ := inf J(v), v∈V
VA R I AT I O N A L FO R M U L AT I O N
73
we have α vn − vm 2H ≤ a(vn − vm , vn − vm ) = a(vn , vn ) + a(vm , vm ) − 2a(vn , vm ) = 2a(vn , vn ) + 2a(vm , vm ) − a(vn + vm , vn + vm ) 1 1 a(vn , vn ) − l, vn + 4 a(vm , vm ) − l, vm =4 2 2 . vn + vm vn + vm vn + vm , + 8 l, − 4a 2 2 2 vn + vm = 4J(vn ) + 4J(vm ) − 8J 2 ≤ 4J(vn ) + 4J(vm ) − 8γ, since 12 (vn + vm ) ∈ V by convexity. Because {vn }n∈N is a minimal sequence, we have 1/2 4 4 γ lim J(vn ) + lim J(vm ) − 8 lim vn − vm H ≤ n,m→∞ α n→∞ α m→∞ α 1/2 8 γ γ−8 = = 0, α α i.e. {vn }n∈N is a Cauchy sequence in V ⊂ H. Since H is a Hilbert space, a limit u = lim vn n→∞
exists and since V is closed, we have v ∈ V . Finally, J is continuous and J(u) = J lim vn = lim J(vn ) = inf J(v). n→∞
n→∞
v∈V
In order to show uniqueness, let u1 and u2 be two solutions. Then u1 , u2 , u1 , u2 , . . . is a minimal sequence. Since any minimal sequence is a Cauchy sequence, we have u1 = u2 . Remark 3.8 There are several variants of the Lax–Milgram theorem. In the simplest situation V = H, the result can directly be obtained from Riesz’ representation theorem. There exist also formulations for nonsymmetric bilinear forms a(·, ·). We do not go into detail here. To summarize, Theorem 3.7 guarantees the existence and uniqueness of a solution to our two model problems (3.1) and (3.2). Of course, the abstract setting covers many more applications.
74
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
3.2.1
Operators associated by the bilinear form
Our goal is to consider boundary value problems for elliptic partial differential equations. Later, we will interpret a differential equation in terms of an operator acting on certain Sobolev spaces. This is the reason why we later make use of the following notation. We associate to the bilinear form a:H ×H →R an operator A : H → H defined by Au, v := a(u, v),
u, v ∈ H,
where again ·, · denotes the duality pairing of H and H, i.e. ·, · : H × H → R. If the bilinear form is defined as above, A is a differential operator. But of course also integral operators or integrodifferential operators are covered. The properties of a(·, ·) extend to the following properties of A. • A is linear . • A is bounded :
Au H = sup
v∈H
Au, v a(u, v) = sup
v H v∈H v H
≤ C · u H , where we have used the standard norm of the dual space H
f H := sup
v∈H
f (v) f, v = sup .
v H v∈H v H
• A is self-adjoint: Au, v = a(u, v) = a(v, u) = Av, u, i.e. A = A. • A is positive on V ⊆ H:
Au H = sup
v∈H
a(u, v) a(u, u) ≥ ≥ α u H ,
v H
u H
u ∈ V.
Boundedness and positivity imply that A is boundedly invertible, i.e. α u H ≤ Au H ≤ C u H or, in brief notation,
Au H ∼ u H .
VA R I AT I O N A L FO R M U L AT I O N
3.2.2
75
Reduction to homogeneous boundary conditions
In the case of the Dirichlet problem (3.2), it is readily seen that we can restrict ourselves to homogeneous boundary conditions. In fact, consider the nonhomogeneous boundary value problem Au = f in Ω, u = g on ∂Ω. Let u0 ∈ H be a function with u0|∂Ω = g. Then, we solve the homogeneous Dirichlet boundary problem Au∗ = f − Au0 in Ω, u∗ = 0 on ∂Ω. The function u := u∗ + u0 solves the inhomogeneous problem. In fact Au = Au∗ + Au0 = f − Au0 + Au0 = f, u|∂Ω = u∗|∂Ω + u0|∂Ω = 0 + g = g. This process is sometimes also called homogenization. However, one should be careful since “homogenization” is used in numerical methods for partial differential equations also with a different meaning. 3.2.3
Stability
Besides existence and uniqueness, stability is needed for a mathematical problem to be well-posed. Here, stability means that the solution depends at least continuously on the data. This is ensured by the following result. Proposition 3.9 (Stability) The solution of (3.14) is stable, i.e.
u H ≤
1
l H , α
where α is the coercivity constant.
Proof Choose v = u in (3.14) and use ellipticity as well as the Cauchy–Schwarz inequality to obtain α u 2H ≤ a(u, u) = l, u ≤ l H u H which yields the claim.
Since we only need ellipticity in the proof, we obtain stability for both model problems, namely the periodic (3.1) and the homogeneous Dirichlet problem (3.2).
76
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
3.3
Regularity theory
For the analysis of numerical methods, we often need information on the regularity (the degree of smoothness) of the solution (without assuming that u itself is known, of course). This is provided by results from a well-established field of applied analysis, namely regularity theory. Here, we just list the main results that are needed later in the univariate case. The multidimensional formulation needs some more preparation, in particular on the shape of the domains. The corresponding statements are collected in Appendix A.5. Theorem 3.10 (Regularity theorem, 1D case) Let a(·, ·) be elliptic on a Hilbert space H, where H01 (Ω) ⊆ H ⊂ H 1 (Ω), and Ω = (a, b) is a finite interval. If the coefficient functions a(·), b(·) and c(·) in a(u, v) =
Ω
a(x) ∇u(x) ∇v(x) dx +
Ω
b(x) · ∇u(x) v(x) dx +
Ω
c(x)u(x) v(x) dx
are smooth, then the corresponding solution u of the variational problem satisfies u ∈ H 2 (Ω) provided that f ∈ L2 (Ω). In many relevant cases, the regularity of the right-hand side determines the smoothness of the solution. Roughly speaking, the solution gains as many orders of regularity from the right-hand side as the order of the differential equation. This is called a shift theorem. Theorem 3.11 (Shift theorem, 1D case) Let Ω = (a, b) be a finite interval. If f ∈ H k (Ω),
k ≥ 0,
the week solution of the Dirichlet problem for Laplace’s equation satisfies u ∈ H k+2 (Ω) ∩ H01 (Ω) such that the a priori estimate
u k+2;Ω ≤ C f k;Ω holds.
GALERKIN METHODS
3.4
77
Galerkin methods
The classical approach is to discretize the infinite-dimensional problem by restricting the problem to finite-dimensional subspaces of X. We will briefly review this approach. However, we would like to point out at this point that we will also use an alternative approach later in connection with adaptive wavelet methods. 3.4.1
Discretization
As already said, we typically consider finite dimensional subspaces VΛ ⊂ V, dim VΛ = |Λ| < ∞, where we can interpret Λ as the set of degrees of freedom, which is assumed to be finite in order to be able to treat the corresponding problem numerically. Example 3.12 As an example, we consider again the homogeneous Dirichlet boundary value problem in (3.2) and let ˚hat := S hat ∩ H 1 (Ω) VΛ = S j j 0 be the set of piecewise linear functions with respect to the dyadic equidistant grid, respecting the boundary conditions. Then, obviously we have Λ=˚ I hat = {1, . . . , 2j − 1} = I hat \ {0, 2j }, j j since we have to take the homogeneous boundary conditions into account. Then, the discrete version of (3.5) reads: find uΛ ∈ VΛ such that a(uΛ , vΛ ) = (f, vΛ )0;Ω , vΛ ∈ VΛ .
(3.18)
Still this problem is not completely accessible on a computer; it amounts to having a basis of VΛ at hand, say VΛ = span ΘΛ ,
ΘΛ = {ϑλ : λ ∈ Λ},
and ΘΛ is a family of linearly independent elements of V . Then, the solution uΛ of (3.18) can be written as uΛ = uλ ϑλ =: uTΛ ΘΛ , uΛ := (uλ )λ∈Λ ∈ R|Λ| , (3.19) λ∈Λ
where now the coefficient vector uΛ needs to be determined by uλ a(ϑλ , ϑµ ) = (f, ϑµ )0;Ω , µ ∈ Λ. a(uΛ , ϑµ ) = λ∈Λ
(3.20)
78
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
This can equivalently be formulated in matrix-vector form AΛ uΛ = fΛ ,
(3.21)
where AΛ := (a(ϑλ , ϑµ ))µ,λ∈Λ ∈ R|Λ|×|Λ| is the stiffness matrix and the vector fΛ := ((f, ϑµ )0;Ω )µ∈Λ ∈ R|Λ| is the right-hand side. Remark 3.13 In (3.18) the test space and the trial space coincide, i.e. both are VΛ . This is the classical Galerkin method. Sometimes it is helpful to distinguish between these spaces, i.e. find uΛ ∈ VΛ : a(uΛ , vΛ ) = f, vΛ for all vΛ ∈ WΛ ,
(3.22)
where now VΛ is called the trial space (the basis functions of VΛ are named trial functions) and WΛ is called the test space (the basis functions are called test functions). This method is called the Petrov–Galerkin method. 3.4.2
Stability
Before going to the numerical solution of (3.18), we have to investigate the stability and approximation properties of the Galerkin method. As in the continuous case, we observe the following estimate. Proposition 3.14 (Stability) The solution of (3.18) is stable independent of the choice of the subspace VΛ , i.e.
uΛ H ≤
1
f H . α
Proof Choose vΛ = uΛ in (3.5) and use ellipticity to obtain α uΛ 2H ≤ a(uΛ , uΛ ) = f, uΛ ≤ f H uΛ H
which yields the claim. 3.4.3
Error estimates
Now let u ∈ V be the solution of (3.5) and uΛ ∈ VΛ be the solution of (3.18), i.e. a(u, v) = f, v a(uΛ , vΛ ) = f, vΛ
for all v ∈ V, for all vΛ ∈ VΛ .
GALERKIN METHODS
79
Since VΛ ⊂ V , we can also test the first equation for vΛ ∈ VΛ . Thus, subtraction of these two equations yields a(u − uΛ , vΛ ) = a(u, vΛ ) − a(uΛ , vΛ ) = f, uΛ − f, uΛ = 0.
(3.23)
This equation expresses the so-called Galerkin orthogonality property. Theorem 3.15 (C´ ea lemma) Let a(·, ·) be V -elliptic. Then, we have
u − uΛ H ≤
C inf u − vΛ H . α vΛ ∈VΛ
(3.24)
Proof Let vΛ ∈ VΛ be arbitrary. Then wΛ := vΛ − uΛ ∈ VΛ and by Galerkin orthogonality we have a(u − uΛ , wΛ ) = a(u − uΛ , vΛ − uΛ ) = 0. Then using boundedness and coercivity of a(·, ·) yields α u − uΛ 2H ≤ a(u − uΛ , u − uΛ ) = a(u − uΛ , u − vΛ ) + a(u − uΛ , vΛ − uΛ ) = a(u − uΛ , u − vΛ ) ≤ C u − uΛ H u − vΛ H . Dividing by α u − uΛ H and taking the infimum over vΛ ∈ VΛ on both sides yields the claim. The C´ea lemma shows that the order of approximation of a Galerkin method depends on the approximation power of the trial spaces VΛ . Thus, we need to construct appropriate spaces VΛ in order to achieve an optimal order of approximation with a minimal number of degrees of freedom N . It would of course be helpful to have a criterion at hand to check the order of approximation. The term EΛ (u) := inf u − vΛ H vΛ ∈VΛ
(3.25)
is known as the error of the best approximation in H. The study of this error leads to the field of approximation theory.
80
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
Nowadays, there are several results known for the approximation power of VΛ for several kinds of such spaces, e.g. if they are formed by finite elements. A typical argument is that Pd−1 ⊂ VΛ implies a certain rate of convergence. Here Pd−1 denotes the space of algebraic polynomials of degree d − 1 (order d): Pd−1 :=
p : p(x) =
d−1
cr xr , cr ∈ R .
r=0
Statements of the form that polynomial exactness implies convergence rates are usually called Bramble–Hilbert type arguments. Let us describe such a statement (here directly in the n-dimensional case in Rn ). In this multivariate case, the norm · k;Ω and the seminorm | · |k;Ω are defined in analogy to Definition 3.3, see also Appendix A. Theorem 3.16 (Bramble–Hilbert lemma) Let Y be a linear space normed by · Y and L : H k (Ω) → Y, Ω ⊂ Rn a Lipschitz domain, be a bounded sublinear operator, i.e.
L(v) ≤ L k v k;Ω ,
L k :=
sup v∈H k (Ω)\{0}
L(v) Y ,
v k;Ω
such that L(p) = 0 for p ∈ Pk−1 , i.e. Pk−1 ⊂ ker(L). Then the following estimate holds
L(v) Y ≤ c(Ω) L k |v|k;Ω , with the Poincar´e–Friedrichs constant c(Ω). Proof Let p ∈ Pk−1 . Then we have by L(p) = 0 that
L(v) Y ≤ L(v − p) + L(p) Y = L(v − p) Y ≤ L k v − p k;Ω . Now we use the Poincar´e–Friedrichs inequality (see Proposition 3.4, page 68, in the univariate and Theorem A.8, page 443, in the multivariate case) which
GALERKIN METHODS
81
states that
v k;Ω ≤ c(Ω) |v|k;Ω ,
v ∈ H0k (Ω).
Then, we have
L(v) Y ≤ c(Ω) L k |v − p|k;Ω = c(Ω) L k |v|k;Ω since Dα p = 0 for |α| = k and p ∈ Pk−1 .
This estimate can now easily be used to estimate the error of the Galerkin method. Theorem 3.17 If V = H = H k (Ω), Pk−1 ⊂ VΛ
(3.26)
and PΛ : V → VΛ is a linear projector, then
u − uΛ k;Ω ≤
C C c(Ω) (I − PΛ )u k;Ω ≤ c(Ω) I − PΛ k |u|k;Ω . (3.27) α α
Proof The Ce´a lemma (see 3.24) ensures that C inf u − vΛ k;Ω α vΛ ∈XΛ C ≤ u − PΛ (u) k;Ω . α
u − uΛ k;Ω ≤
Now, we apply the Bramble–Hilbert lemma for Y = X = H k (Ω) and L = I − PΛ and obtain
u − uΛ k;Ω ≤
C C
(I − PΛ )(u) k;Ω ≤ c(Ω) I − PΛ k |u|k;Ω , α α
which proves the claim. The last step to obtain an error estimate is now to develop an estimate for
u − PΛ (u) k;Ω ,
for suitable u. This of course depends on the choice of the trial space VΛ . Using VΛ = S hat j with both periodic or boundary conditions, results in an estimate −j
u − P hat j u 1;Ω 2 |u|2;Ω ,
u ∈ H 2 (Ω).
82
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
Hence, we obtain the error estimate
u − uj 1;Ω 2−j |u|2;Ω ,
u ∈ H 2 (Ω),
i.e. we obtain an error reduction factor of 2 from one level to the next finer one. 3.4.4
L2 -estimates
The above framework gives error estimates in the trial space H, i.e. for elliptic second-order boundary value problems in H01 (Ω). Sometimes, however, one is interested in estimates in the weaker topology of L2 (Ω). This can be done by the following result, known as the Aubin–Nitsche trick . Theorem 3.18 (Aubin–Nitsche trick) Let H be a Hilbert space and V → H be continuously imbedded, VΛ ⊂ V . Then, we have 1 inf ϕg − vΛ V , (3.28)
u − uΛ H ≤ C u − uΛ V sup g∈H\{0} g H vΛ ∈VΛ where uΛ ∈ VΛ is the Galerkin solution, C is the continuity constant of a(·, ·) and ϕg ∈ V is the dual solution for a given g ∈ H, i.e. the solution of the variational problem w ∈ V.
a(w, ϕg ) = (g, w)H ,
(3.29)
Proof We use again Galerkin orthogonality (3.23), i.e. a(u − uΛ , vΛ ) = 0,
vΛ ∈ VΛ ,
and we test (3.29) with w = u − uΛ ∈ V to obtain by continuity of a(·, ·) for any vΛ ∈ VΛ that (g, u − uΛ )H = a(u − uΛ , ϕg ) = a(u − uΛ , ϕg − vΛ ) ≤ C u − uΛ V ϕg − vΛ V . Thus, we obtain by the standard representation of norms in Hilbert spaces
u − uΛ H =
sup g∈H\{0}
(g, u − uΛ )H
g H
≤ C u − uΛ V
sup g∈H\{0}
which proves the claim.
1
ϕg − vΛ V
g H
,
GALERKIN METHODS
83
Note that the above statement can also be proven for nonsymmetric bilinear forms. However, if a(·, ·) is symmetric, (3.29) coincides with the usual variational problem with a different right-hand side. However, the Aubin–Nitsche trick also holds for nonsymmetric problems. The product in (3.28) is the reason why one typically gains an approximation order when replacing · 1;Ω by the weaker norm · 0;Ω . In the case of S hat j , we obtain
u − uj 0;Ω −2j |u|2;Ω , if u ∈ H 2 (Ω), i.e. an error reduction factor of 4 from one level to the next finer one. 3.4.5
Numerical solution
The efficient numerical treatment of (3.21) clearly depends on the properties of the matrix AΛ . These in turn are implied by the type of the underlying differential equation and the trial basis functions ΘΛ . Since the bilinear form in (3.1) is elliptic, AΛ is symmetric and positive definite; in fact uTΛ AΛ uΛ
= a
λ∈Λ
uλ ϑλ ,
µ∈Λ
2
uµ ϑµ ≥ α uλ ϑλ >0
1;Ω λ∈Λ
if uΛ = 0. Note that often the number |Λ| of degrees of freedom may become very large so that direct solvers are ruled out. Moreover, we assume that the basis functions are local in the sense that the basis is locally finite, i.e. only a finite number of functions overlap at a fixed point x; precisely #{µ ∈ Λ : supp ϑλ ∩ supp ϑµ = ∅} 1, see Assumption 2.16 (page 47). This implies that the stiffness matrix is sparse, i.e. #{µ ∈ Λ : a(ϑλ , ϑµ ) = 0} 1 independent of λ ∈ Λ, i.e. only a fixed number of non-zero entries occur in each row and column of AΛ . This means that the application of AΛ to a given vector vΛ can be performed in O(|Λ|) arithmetic operations, i.e. this application is optimal in terms of efficiency. The overall amount of work of an iterative scheme like the conjugate gradient (CG) method (see Algorithm C.1 on page 462) for the solution of a linear system
84
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
with a SPD matrix is determined by the amount of work involved in a matrixvector multiplication (which has to be done in each step of the iteration) and the number of iterations. Since each matrix-vector multiplication can be performed in O(|Λ|) operations, the overall amount is in the order of |Λ| · #iterations. It is well known (see, e.g. [40] or any other textbook on numerical analysis) that the number of iterations is related to the spectral condition number cond2 (AΛ ) =
λmax (AΛ ) , λmin (AΛ )
i.e. the number of iterations is related to cond2 (AΛ ). To be precise, for the CG method, the following estimate is well known for the solution of a linear system Ax = b starting with the initial guess x(0) (k)
x
$k #/ cond2 (A) − 1 − x A ≤ 2 /
x(0) − x A , cond2 (A) + 1
(3.30)
where
x A := xT Ax is the energy norm. From these considerations, it becomes clear that the main issue is to find a basis ΘΛ such that cond2 (AΛ ) is as small as possible or to find a matrix DΛ such that −1/2
DΛ
−1/2
AΛ DΛ
is well conditioned. This issue is called preconditioning and we will come back to this later. It is well known that (3.30) becomes
x(k) − x(0) A ≤ 2
√ k κ−1 √
x(0) − x A , κ+1
(3.31)
with κ = cond2 (D−1/2 AD−1/2 ) for the preconditioned CG method. In the literature, one also finds the version κ = cond2 (AB) with a preconditioner B. In this case, B needs to be symmetric (and positive definite, of course) and BA = AB. Hence, we prefer the representation D−1/2 AD−1/2 where symmetry is guaranteed as long as D is symmetric.
E X E RC I S E S A N D P RO G R A M S
3.5
85
Exercises and programs
Exercise 3.1 For which m ∈ Z the following linear functionals belong to the dual space H −m (R) of H0m (R)? (a) Tg (f ) := f (x) g(x) dx, g ∈ C0 (R). R (b) Tg (f ) := f (x) g(x) dx, g ∈ L1 (R). R
(c) δx f := f (x) the Dirac delta distribution. Hint: For f ∈ C0 (R) one can use the definition. For f ∈ C0 (R) use the continuous extension of the functional. (d) δx f := f (x). Exercise 3.2 Assume that u ∈ C 2 (Ω), Ω = (0, 1), is a weak solution of the periodic boundary value problem (3.2). Show that u is also a classical solution. Exercise 3.3 Consider the Dirichlet boundary value problem (3.1) with boundary conditions u(0) = 1,
u(1) = 2.
Reduce this to homogeneous boundary conditions and compute the algebraic system using piecewise linear hat functions with step size h = 2−4 to generate trial and test spaces. Exercise 3.4 Look up the proof of Theorem 3.3 in the literature and work it out. Exercise 3.5 Show the shift theorem 3.11. Computer Exercise 3.1 Write a program for the periodic two-point-boundary value problem (3.2) on Ω = (0, 1) by using Φhat as basis within a Galerkin j method. Define a sufficiently smooth function u satisfying the boundary conditions and compute (by hands) the corresponding right-hand side f (for later validation). (a) Compute the error u − uj 0;Ω and u − uj 1;Ω , where a suitable approximation of the integrals should be used. (b) Give the number of iterations for the standard CG method using Φj and the preconditioned CG method using the diagonal preconditioner Dj := diag(Aj ). (c) Determine the rate of convergence, i.e. plot
u − uj+1 m;Ω ,
u − uj m;Ω over the level j.
m = 0, 1,
86
E L L I P T I C B O U N DA RY VA L U E P RO B L E M S
Programs FLENS provides the user with some basic routines that are useful for solving elliptic boundary value problems, namely: • Basic linear algebra routines, vectors, matrices, matrix-vector multiplication, etc. • Numerical integration, quadrature. • Solvers for linear systems of equations including — direct solvers: LU, Cholesky, QR; — classical iterative solvers: Jacobi, Gauss–Seidel, — iterative solvers: gradient, conjugate gradients, GMRES, etc. • Plotting interfaces to GNUPLOT, MATLAB. • Condition estimators.
4 MULTIRESOLUTION GALERKIN METHODS In this chapter, we use an MRA for the Galerkin discretization of elliptic boundary value problems. In the literature, this is sometimes called the WaveletGalerkin method . As we will see, no wavelet will be involved here. In Chapter 6 below, we will describe methods based on wavelets. Even though we first concentrate on the periodic setting, we will use a general notation and framework that also allows us to consider bounded domains. 4.1
Multiscale discretization
It has been known for quite some time that the solutions of many differential problems exhibit features living on quite different length-scales. It has been shown that numerical schemes using this knowledge can be highly efficient. Hence, we are concerned with multiscale discretizations. This is usually done by using a whole family of trial and test spaces V0 ⊂ V1 ⊂ · · · ⊂ Vj ⊂ Vj+1 ⊂ · · · ⊂ V. The interplay between these spaces for different values of j is the key to the efficiency of the resulting method. In more classical methods such as finite difference or finite element discretizations, the multiscale hierarchy can easily be formed by a mesh size parameter hj → 0
(e.g. hj = 2−j )
for j → ∞
and Vj as a finite element space corresponding to a mesh of mesh size hj . Then, BPX-type preconditioned CG methods (see Section 4.6 below) or MultiGrid methods (see Section 4.7 below) perform optimally, which means that the Galerkin problem on Vj can be solved in O(dim Vj ) operations, i.e. linear complexity, see Bramble, Pasciak and Xu, e.g. [42, 43]. This means that the condition number of the preconditioned stiffness matrix is uniformly bounded, i.e. −1/2
cond2 (Dj
−1/2
Aj Dj
) = O(1),
87
j → ∞.
88
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
We will use here an MRA S = {Sj }j∈N0 for the discretization, i.e. Vj := Sj . The resulting scheme is called the multiresolution Galerkin method , also abbreviated as MGM or MG method. As already said, one of the ultimate goals of using a multiscale discretization is to obtain efficient numerical methods. Moreover, one asks for a method with optimal complexity. Let us describe this in a bit more detail for the MGM. We assume that the trial and test spaces are given in terms of bases, i.e. Vj = Sj = span{Φj },
Φj := {ϕj,k : k ∈ Ij }.
Thus, the sequence Ij represents sets of degrees of freedom, |Ij | the dimension, such that |Ij | → ∞ for j → ∞, typically in an exponential way, i.e. |Ij | ∼ 2j . Thus the number of operations for the matrix-vector multiplication with Aj = ASj grows (exponentially) with increasing j. Using an iterative scheme, this typically means that the amount of work per step is O(|Ij |). In order to have a best possible numerical scheme at hand, it is hence desirable that the number of iterations of the scheme is bounded independent of j, so that it amounts to the same number of iterations to reduce the error by a given factor independent of how fine the discretization is. Having the error estimate, e.g. for the CG method (3.30) in mind, this means that the condition number should also be bounded independent of j, i.e. cond2 (Aj ) = O(1),
j → ∞.
For a basis consisting of scaling functions, this is not achievable as we will see below. We will need a preconditioner Dj to Aj such that • Dj is symmetric and positive definite; −1/2 to a given vector vj can be realized in O(|Ij |) • the application of Dj operations; −1/2 −1/2 Aj Dj ) = O(1) for j → ∞. • cond2 (Dj
M U LT I R E S O L U T I O N M U LT I S C A L E D I S C R E T I Z AT I O N
89
We will also use wavelets later to achieve this goal which is one of the positive properties of wavelets.
4.2
Multiresolution multiscale discretization
Now we describe how a multiscale discretization can be obtained by using multiresolution spaces Sj as trial and test spaces. Let us start again with some simple examples using the MRAs introduced above. 4.2.1
Piecewise linear multiresolution
We consider now the boundary value problem (3.1) with homogeneous Dirichlet ˚hat for the discretization. boundary conditions. As in Example 3.12 we use S j hat ˚ Straightforward calculations show that for k, m ∈ I j 1 d j/2 d hat d hat d j/2 j j ϕ , ϕ 2 ϕ(2 · −k) (x) 2 ϕ(2 · −m) (x) dx = dx j,k dx j,m 0;[0,1] dx dx 0 = 2j 22j ϕ (2j x − k) ϕ (2j x − m) dx
R
= 22j
R
ϕ (x) ϕ (x − (m − k)) dx
2, if m = k; 2j −1, if m = k ± 1; =2 0, else, as well as by (2.18) hat (ϕhat j,k , ϕj,m )0;[0,1]
= R
ϕ(x) ϕ(x − (m − k)) dx
2 , 3 = 16 , 0,
if m = k; if m = k ± 1; else.
Putting this together, we obtain for the entries ajk,m of the stiffness matrix Ahat = (ajk,m )k,m∈˚ j I hat j
90
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
the following representation
ajk,m
2j+1 + c 23 , 2 hat = a(ϕhat −22j + c 16 , j,k , ϕj,m ) = 0,
if m = k; if m = k ± 1; else,
i.e. a tridiagonal matrix. It is well known from any introduction to numerical linear algebra that a linear system of dimension n × n with a tridiagonal matrix can be solved in O(n) operations by using a sparse LU decomposition. In this situation, we still do not use a direct solver since we will later also consider problems in higher dimensions, where the stiffness matrix does not have such a simple form, ruling out direct solvers. We could estimate the condition of Aj as in the investigation of the stability above by using Gerschgorin’s theorem. However, this would only give an of Φhat j upper bound on the condition number. Instead, we can directly determine the eigenvalues of Aj due to its simple form by Exercise 4.1. We obtain for the kth largest eigenvalue kπ 2 c λk (Aj ) = 22j+1 + c + 2 22j − cos 3 6 2j for k = 1, . . . , 2j − 1. Thus we have λmax (Aj ) = λ1 (Aj ) ∼ 22j λmin (Aj ) = λ2j −1 (Aj ) ∼
c 3
which implies that cond2 (Aj ) = O(22j ),
j → ∞,
(4.1)
so that using CG without any preconditioning is far from optimal complexity. We report the condition numbers in Table 4.1 4.2.2
Periodic boundary value problems
Let us now consider the periodic boundary value problem (3.1). As trial and test spaces we use the periodized multiscale spaces Sjper defined in Section 2.11. We do not specify a particular choice of scaling function here, but just assume that ϕ is sufficiently smooth, i.e. ϕ ∈ H01 (R). Thus it amounts to finding some function uj =
j 2 −1
k=0
cj,k [ϕ]j,k =: cTj Φper j ,
cj := (cj,0 , . . . , cj,2j −1 )T ∈ R2 , j
M U LT I R E S O L U T I O N M U LT I S C A L E D I S C R E T I Z AT I O N
91
Table 4.1 Condition numbers for the stiffness matrix Ahat for homogeneous j Dirichlet boundary conditions for different values of c and j. c/j
3
4
5
6
7
8
9
10
0 100 200 300 400 500
32.2 2.7 1.6 1.2 1.0 1.2
116.5 9.7 5.2 3.6 2.8 2.3
440.7 37.8 19.9 13.6 10.3 8.4
1711.7 149.8 78.5 53.2 40.3 32.5
6743.7 597.6 312.8 211.9 160.3 128.9
26768.0 2387.9 1249.8 846.5 640.0 514.5
106657.7 9547.4 4997.5 3384.7 2558.9 2057.0
425801.6 38182.2 19987.4 13536.9 10234.1 8226.9
106 c=0 c = 100 c = 200 c = 300 c = 400 c = 500
105 104 103 102 101 100
2
3
4
5
6
7
8
9
10
i.e. Ij = {0, . . . , 2j − 1}, such that Aj cj = fj ,
(4.2)
where Aj := (a([ϕ]j,k , [ϕ]j,m ))k,m∈Ij is the stiffness matrix and fj := (f, [ϕ]j,m )0;(0,1) m∈I
j
denotes the right-hand side data.
92
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
We will describe a general method to compute these entries in Section 4.5 below. For the particular example of ϕ = ϕhat , we get 2j+1 + c 23 , if m = k; 2 −22j + c 16 , if m = k + 1, k = 0, . . . , 2j − 2; or m = k − 1, k = 1, . . . , 2j − 1; ajk,m = or m = 0, k = 2j − 1; or m = 2j − 1, k = 0; 0, else, i.e. Aj has cyclic structure. The above estimate (4.1) for the condition number remains unchanged, which will be proven below, i.e. cond2 (Aj ) = O(22j ),
j → ∞.
We illustrate this by a numerical study reported in Table 4.2. Table 4.2 Condition numbers for the stiffness matrix Ahat in the periodic j case for different values of c and j. For c = 0, the matrix is singular. c/j
3
4
5
6
7
8
9
10
100 200 300 400 500
2.9 1.6 1.2 1.0 1.2
10.6 5.5 3.7 2.9 2.4
41.3 20.8 14.0 10.6 8.5
164.2 82.3 54.9 41.3 33.1
655.7 328.0 218.8 164.2 131.4
2621.8 1311.1 874.1 655.7 524.6
10486.1 5243.2 3495.6 2621.8 2097.5
41943.4 20971.9 13981.3 10486.1 8388.9
105 c = 100 c = 200 c = 300 c = 400 c = 500
104
103
102
101
100
2
3
4
5
6
7
8
9
10
M U LT I R E S O L U T I O N M U LT I S C A L E D I S C R E T I Z AT I O N
93
Let us now collect the properties of the linear system of equations (4.2) for the particular periodic boundary value problem that will be relevant for the numerical solution. Lemma 4.1 For the periodic boundary value problem we have: (a) The stiffness matrix Aj ∈ R|Ij |×|Ij | is sparse, i.e. #{(k, l) ∈ Ij × Ij : a ([ϕ]j,k , [ϕ]j,l ) = 0} ∼ 2j . (b) The matrix is ill-conditioned: cond2 (Aj ) ∼ 22j .
Proof The first part is trivially seen since for each k ∈ Ij there is only a finite number of indices l ∈ Ij such that a ([ϕ]j,k , [ϕ]j,l ) = 0 and this number of nonzero values is independent of j since a straightforward calculation shows a ([ϕ]j,k , [ϕ]j,l ) = a ϕ[j,k] (· − m)|[0,1) , ϕ[j,l] (· − p)|[0,1) m∈Z p∈Z
=
2j a ϕ 2j · −2j m − k |
m∈Z p∈Z
=
22j
R
p∈Z
[0,1)
, ϕ 2j · −2j p − l |
ϕ(y − k) ϕ(y − 2 p − l) dy . j
R
[0,1)
ϕ (y − k) ϕ (y − 2j p − l) dy
+c
(4.3)
This means that if 2j > |supp ϕ| (i.e. for sufficiently large values of j), we have a ([ϕ]j,k , [ϕ]j,l ) = 22j ϕ (y − k) ϕ (y − l) dy + c ϕ(y − k) ϕ(y − l) dy R
R
and obviously the number of pairs (k, l) for which this expression is nonzero is independent of j. On the other hand, for those j with 2j ≤ |supp ϕ|, we have that for all k, l ∈ Ij there exists a p ∈ Z such that (supp ϕ(· − (k − l))) ∩ supp ϕ(· − 2j p) = ∅, i.e. a [ϕ]j,k , [ϕ]j,l = 0 for all k and l, which obviously is also independent of j.
94
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
As for the second statement, we use the property (2.21) of the Rayleigh quotient defined in (2.19). Since uT Aj u = a(uT Φj , uT Φj ) which yields by (4.3) rmax (Aj ) ≥ r(Aj , δ0 ) = a([ϕ]j,0 , [ϕ]j,0 ) 22j ϕ 21;R 22j
(4.4)
and for [ϕ]0,0 ≡ 1 ∈ Sjper we obtain rmin (Aj ) ≤ a([ϕ]0,0 , [ϕ]0,0 ) = c 1.
(4.5)
Combining this with (2.21) and (4.4) proves the claim.
Figure 4.1 show the sparsity pattern for Aj in (4.2) for two different examples of ϕ. On the left part, B-splines of order 2 (piecewise linear functions) and on the right part of order 3 (piecewise quadratic functions) are used. The matrix is sparse in both cases as stated in Lemma 4.1. As can be seen, the number of nonzero entries depends on d. This is reflected in the size of the constant in Lemma 4.1. 4.2.3
Common properties
For later reference, let us collect the main common properties of the stiffness matrix Aj in the two examples above. 0
0
5
5
10
10
15
15
20
20
25
25
30
30 0
5
10
15
20
25
30
0
5
10
15
20
25
30
Fig. 4.1 Sparsity pattern of the stiffness matrix for B-splines of orders 2 (left) and 3 (right) both for j = 5.
E R RO R E S T I M AT E S
95
1. Aj is sparse This is due to the fact that the bilinear form a(·, ·) is local in both cases, i.e. |(supp u) ∩ (supp v)| = 0.
a(u, v) = 0 if
(4.6)
This is also true for any bilinear form arising from a differential operator. The second reason for the sparsity is the locality of the basis functions in the sense N (k) := #{m ∈ Ij : |(supp ϕj,k ) ∩ (supp ϕj,m )| = 0} ∼ 1,
(4.7)
independent of k ∈ Ij and of the level j. This means that any function ϕj,k is overlapped by only finitely many ϕj,m independent of k ∈ Ij and of the level j. 2. Aj is ill-conditioned The maximal eigenvalue of the stiffness matrix grows exponentially whereas the minimal eigenvalue is bounded. This results in an exponential growth of the condition number cond2 (Aj ) ∼ 22j ,
(4.8)
so that preconditioning is highly needed. 4.3
Error estimates
Let us investigate the error of the MG method. If a(·, ·) is elliptic on H t (Ω), the C´ea lemma (see 3.24) gives that
u − uj t;Ω inf u − vj t;Ω , vj ∈Sj
where u is the exact weak solution of the boundary value problem and uj is the MG solution. Next, we need the regularity and polynomial exactness of the scaling functions. If Pd−1 ⊂ Sj and ϕj,k ∈ H t (R), the statement of Proposition 2.21 (see 2.49) gives
u − uj t;Ω inf u − vj t;Ω 2−j(s−t) |u|s;Ω , vj ∈Sj
96
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
for u ∈ H s (Ω), t < s ≤ d. By the Aubin–Nitsche trick , one can obtain an L2 estimate
u − uj 0;Ω 2−js |u|s;Ω , for u ∈ H s (Ω), t < s ≤ d. Example 4.2 Let us consider the homogeneous Dirichlet boundary value problem in (3.2) or the periodic one in (3.1), where we have t = 1. If we use the piecewise linear hat functions for the discretization, we know that ϕj,k ∈ H 1 (R) and d = 2. Hence, we obtain
u − uj 1;Ω 2−j(s−1) |u|s;Ω ,
1 < s ≤ 2,
u ∈ H s (Ω),
or, in other words, if u ∈ H 2 (Ω), the maximal result is
u − uj 1;Ω 2−j and by the Aubin–Nitsche trick
u − uj 0;Ω 2−2j . 4.4
Some numerical examples
In this section, we report on some numerical tests that illustrate the above described analysis. We construct three functions that we use as exact solution of the periodic boundary value problem (3.1). The first function is u1 (x) := sin(2π x),
(4.9)
which is a C ∞ -function. Hence, all error estimates hold since in particular u ∈ H 2 . The second function is defined as 1.1 x , if 0 ≤ x ≤ 0.5, (4.10) u2 (x) := (1 − x)1.1 , if 0.5 ≤ x ≤ 1, which is not in H 2 but we can still compute the right-hand side exactly as f2 (x) = −u2 (x) + c u2 (x). We will refer to this function as a power function. In these two cases, we can compute the corresponding errors exactly. As for the third example, we use the delta distribution δ0.5 as the right-hand side f3 and compute the corresponding solution u3 by a high-level discretization. Again, the function u3 ∈ H 2 as in the second case. Thus, the assumptions in the above error estimates are not valid for the second and third example. The functions ui , i = 1, 2, 3 are displayed in Figure 4.2.
SETUP OF THE ALGEBRAIC SYSTEM 1 0.5 0 –0.1 –1
0
0.2
0.4
0.6
0.8
0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 1 0
97
1.08 1.06 1.04 1.02 1 0.98 0.96 0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
Fig. 4.2 Exact solutions ui , i = 1, 2, 3, for the three test cases. Sin function (left), power function (middle) and solution corresponding to the Delta distribution (right). Table 4.3 L2 -errors and convergence factors in the three different cases. u1 j
L2 -norm
3 4 5 6 7 8 9 10
8.43002e − 04 2.21798e − 04 5.59857e − 05 1.40232e − 05 3.50724e − 06 8.76895e − 07 2.19231e − 07 5.48216e − 08
u2 factor
j
— 3 3.801 4 3.962 5 3.992 6 3.998 7 4.000 8 4.000 9 3.999 10
L2 -norm 8.83241e − 04 2.56712e − 04 7.19282e − 05 1.96961e − 05 5.30970e − 06 1.41484e − 06 3.73182e − 07 9.72138e − 08
u3 factor
j
— 3 3.441 4 3.569 5 3.652 6 3.709 7 3.753 8 3.791 9 3.839 10
L2 -norm
factor
1.21784e − 03 3.15193e − 04 8.01060e − 05 2.01915e − 05 5.07219e − 06 1.27486e − 06 3.23275e − 07 8.46738e − 08
— 3.864 3.935 3.967 3.981 3.979 3.944 3.818
In Table 4.3, we show the errors measured in the L2 -norm and the corresponding reduction factors. We see the expected reduction factor of 4 up to numerical precision at least for u1 which satisfies the regularity assumptions in the error estimates. For the second and third case (where the estimate is not rigorously justified), there is a slight loss of convergence speed. We have also performed tests for the homogeneous Dirichlet boundary value problem (3.2) using the hat functions as trial functions. The results are very similar to those reported above for the periodic boundary value problem. In Table 4.4 we show errors and convergence factors measured in H 1 . We expect a reduction factor of 2 as long as the solution is in H 2 . We use u1 and compare the behavior of the homogeneous Dirichlet problem (3.2) and the periodic boundary value problem in (3.1). In both cases, the assumptions of the error estimates are valid. We see the expected factor of 2 in both cases. 4.5
Setup of the algebraic system
If we want to solve the discretized system numerically, we first need to determine the algebraic system that has to be solved, i.e. we need to compute the entries
98
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
Table 4.4 H 1 -errors and convergence factors for u1 as solution for the homogeneous Dirchilet problem (3.2) and the periodic boundary value problem (3.1). u1 Dirichlet
u1 periodic
j
H 1 -norm
factor
H 1 -norm
factor
3 4 5 6 7 8 9 10 11
1.02004 0.50552 0.25207 0.12594 0.06296 0.03148 0.01574 0.00787 0.00393
— 2.018 2.005 2.001 2.000 2.000 2.000 2.000 2.000
0.99703 0.50237 0.25167 0.12590 0.06295 0.03148 0.01574 0.00787 0.00393
— 1.985 1.996 1.999 2.000 2.000 2.000 2.000 2.000
of the stiffness matrix Aj and of the right-hand side fj . To be precise, in the periodic case, we need 1 d d [ϕ]j,k (x) [ϕ]j,m (x) + c [ϕ]j,k (x)[ϕ]j,m (x) dx, (Aj )k,m = dx dx 0 1 (fj )k = f (x)[ϕ]j,k (x) dx. 0
In the case of homogeneous Dirichlet boundary conditions, we need terms like 1 d d (Aj )k,m = ϕj,k (x) ϕj,m (x) + c ϕj,k (x)ϕj,m (x) dx, dx dx 0 1 (fj )k = f (x)ϕj,k (x) dx. 0
In both cases, we need integrals of products (of derivatives) of scaling functions and integrals of a given function times a scaling function. Of course, we could use numerical integration in terms of quadrature. However, there is also an alternative that allows the direct and exact (up to machine accuracy) evaluation of such integrals. This method will be described now. 4.5.1
Refinable integrals
It turns out that integrals of the above mentioned form can be viewed as multivariate functions using the different integer shift parameters as arguments. Then, one can show that such a multivariate function is refinable and thus the computation of such integrals can be reduced to the evaluation of refinable functions
SETUP OF THE ALGEBRAIC SYSTEM
99
on an integer lattice so that we can use the eigenvector–eigenvalue approach from Section 2.10 above. This section is based upon work of Dahmen and Micchelli [89]. A simple example Let us clarify the situation again on the simplest framework, namely the integral of two univariate refinable functions. To this end, let ϕi : R → R , i = 0, 1, be refinable functions with mask ai . Then we consider the function F : R → R defined by the convolution ϕ0 (t) ϕ1 (t − x) dt. (4.11) F (x) := R
It is not hard to see that F is also refinable. In fact F (x) = a0k a1m ϕ0 (2t − k) ϕ1 (2t − 2x − m) dt R
k∈Z m∈Z
1 0 1 a a ϕ0 (s) ϕ1 (s − (2x − (k − m))) ds = 2 k m R k∈Z m∈Z
using the substitution s = 2t − k. Then, we have 1 a0 a1 F (2x − (k − m)) 2 k m k∈Z m∈Z # $ 1 0 1 a a F (2x − k), = 2 k+m m
F (x) =
k∈Z
(4.12)
m∈Z
which shows that F is refinable with mask 1 0 ak = ak+m a1m . 2 m∈Z
If we aim at computing integrals of the form j ϕ[j,k] (x) ϕ[j,m] (x) dx = 2 ϕ(2j x − k) ϕ(2j x − m) dx R
= R
R
ϕ(x) ϕ(x − (m − k)) dx
for k, m ∈ Z, this is obviously equivalent to computing F (n) for n ∈ Z, i.e. to computing the value of F at the integer points. Hence, the original problem is reduced to the computation of the values of a refinable function on integers. For this, we already have an efficient method in Proposition 2.23 by solving a (small) eigenvector–eigenvalue problem.
100
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
The general case Let us also describe the general situation of integrals of products of several derivatives of scaling functions. Since by the chain rule θ(x) := ϕ (x) = 2ak ϕ (2x − k) = 2ak θ(2x − k) k∈Z
k∈Z
also derivatives of refinable functions are refinable, we can reduce ourselves to the consideration of scaling functions leaving derivatives aside. Next, it is easy to see that 0 m m 0 ϕν (2j x − kν ) dx = 2−nj ϕ0 (x) ϕν (2j x − (kν − k0 )) dx, Rn ν=0
Rn
ν=1
so that it suffices to compute integrals of the form m 0 F (x1 , . . . , xm ) := ϕ0 (x) ϕν (t − xν ) dt. Rn
(4.13)
ν=1
In the same manner as above one can derive the following refinement relation for F m m 0 0 ν akν ϕ0 (2t − k0 ) ϕν (2t − 2xν − kν ) dt F (x1 , . . . , xm ) = Rn
k0 ,...,km ∈Zn ν=0
= 2−n
k0 ,...,km
= 2−n
m 0 ∈Zn
1
k0 ,...,km ∈Zn
aνkν
ν=0
l∈Zn
ν=1
# a0l
ϕ0 (s)
Rn m 0
m 0
ϕν (s − (2xν − (kν − k0 ))) ds
ν=1
$2 aνl−kν
F (2x1 − k1 , . . . , 2xm − km ),
ν=1
which generalizes (4.12). Remark 4.3 Note that using the eigenvector–eigenvalue method gives the values for all nonzero entries of F (k1 , . . . , km ) in (4.13), ki ∈ Zn , i = 1, . . . , m, simultaneously. On the other hand, such integrals are restricted to the case where all function live on the same level. Of course, an integral of the form ϕ[j,k] (x) ϕ[j ,k ] (x) dx R
for j = j can be expressed in terms of F in (4.11)by using the refinement equation for ϕ as many times as |j − j|. An easy calculation, however, shows that the cost grows exponentially in |j − j|, so that this approach is no longer efficient. This issue will become crucial later with regard to adaptive wavelet methods. For a multiresolution Galerkin method as described in this chapter this issue is irrelevant since Aj can be computed highly efficiently with the method introduced above.
SETUP OF THE ALGEBRAIC SYSTEM
4.5.2
101
The right-hand side
In order to solve (4.2), it remains to compute the entries (f, [ϕ]j,k )0;(0,1) of the right-hand side for any given function f . The method of choice of course depends on the representation of f . If it is given as a closed formula or by sample values at given nodes, a straightforward approach is to determine an approximation to f , say f≈
fj,k [φ]j,k =: Pˇj f
k∈Ij
with some refinable function φ and coefficients fj,k to be determined. One may think of Pˇj as being a quasiinterpolant or a B-spline approximation (keeping in mind that B-splines are refinable). For the Dirichlet problem, the approach remains the same with [ϕ]j,k and [φ]j,k replaced by ϕj,k and φj,k , respectively. Then, the computation of fj can be reduced to values of the form being described in the previous section. Of course, the particular choice of Pˇj depends on the way f is given and on the desired accuracy. Since this may depend also on the particular application, we do not go into detail here. Let us sketch one particular example where the right-hand side is not given as a closed formula but (e.g. by measurements) as a collection of samples (xi , fi ),
i = 1, . . . , N = 2J .
If the samples are regular, i.e. xi = ih, we can compute PˇJ f as the spline interpolant of the data. Since splines are refinable, we can easily compute the entries of the right-hand side. 4.5.3
Quadrature
At least for B-spline scaling functions, closed formulas for the refinable functions are available (which is not the case for Daubechies functions and the duals for B-splines). If such a closed formula is available, one can of course also use numerical integration to compute integrals of scaling functions. In the case of splines, one can easily use quadrature formulas which are exact since splines are piecewise polynomials. This can also be used for the computation of the entries for the right-hand side. This is also beneficial in the case of “complicated” right-hand side functions f , where a good approximation as above f ≈ Pˇj f
102
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
may require us to replace j by a higher level J. Then, one would have to compute integrals of the form (ϕj,k , ϕJ,m )0;R ,
j < J.
Such terms can of course be computed by using refinable integrals since we can use the refinement equation (J − j) times for ϕj,k in order to reach the same level J which allows us to use a refinable integral. In particular if the level difference J − j is large this approach is no longer efficient since on each level one obtains roughly double the number of functions by using the refinement equation. This means that one has to sum in the order of 2J−j terms. In such a case, it is much more efficient to use numerical integration. The computation of integrals of functions on fairly different levels will be important in the framework of adaptive wavelet methods. We will come back to this point later. 4.6
The BPX preconditioner
A hierarchical preconditioner was introduced by Bramble, Pasciak and Xu and published 1990 [43, 185]. It was proven in [43] that this gives rise to an asymptotically optimal preconditioner Cj in the sense that −1/2
cond2 (Cj
−1/2
Aj Cj
) = O(1),
j → ∞.
We detail this preconditioner here within a multiresolution Galerkin framework, even though it also applies to other discretizations. We assume here that A is an elliptic operator of positive order s = 2t, i.e.
A : H 2t → (H 2t ) , where H 2t denotes a suitable Sobolev space possibly incorporating boundary conditions. Using Sj as trial and test spaces in a Galerkin method and assuming that Sj = span Φj ,
Φj = {ϕj,k : k ∈ Ij }
is a uniformly stable basis of Sj , the discrete operator Aj is given as Aj := A|Sj = ASj = AΦj , Φj . Then, the BPX preconditioner takes the form Cj−1 v :=
j =0
2−2t
k∈I
(v, ϕ,k )0;Ω ϕ,k .
(4.14)
THE BPX PRECONDITIONER
103
For numerical purposes, one can further detail the application of Cj−1 to vj ∈ Sj , i.e. to a function of the form αm ϕj,m , vj = m∈Ij
which we will describe now. Firstly, one needs the entries of the Gramian matrix, i.e. j gk,m := (ϕj,k , ϕj,m )0;Ω
which can easily be computed as refinable integrals. Thus, we obtain for the coefficients in (4.14) of Cj−1 vj on the finest level j j αm gk,m . (vj , ϕj,k )0;Ω = m∈Ij
The coefficients on the next coarser level j − 1 can be computed using the refinement equation for ϕj−1,k as j−1 ak,p (vj , ϕj,p )0;Ω , (vj , ϕj−1,k )0;Ω = p∈Ij
where ajk,m are the refinement coefficients of ϕj,k , i.e. j ϕj,k = ak,m ϕj+1,m , m∈Ij
so that we can compute all coefficients (vj , ϕ,k )0;Ω , = 0, . . . , j, in one loop by using the refinement equation. Then, a reverse loop is applied starting on the coarsest level = 0 to project every coarse level part to the finest level j in order to obtain the full representation. Having the coefficients (vj , ϕ0,k )0;Ω at hand, one uses again the refinement relation, here for ϕ0,k , # $ (vj , ϕ0,k )0;Ω ϕ0,k = a0k,m (vj , ϕ0,k )0;Ω ϕ1,m k∈I0
m∈I1
=:
k∈I0 0 αm ϕ1,m
m∈I1
and the coefficients on level 1 are computed as β1,k = 2−2t (vj , ϕ1,k )0;Ω + αk0 . Thus, two loops are required and the overall application is performed in linear complexity. The complete algorithm is listed in Algorithm 4.1, page 104. We will give numerical comparisons for the periodic boundary value problem (3.1) in Section 4.8 below.
104
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
Algorithm 4.1 BPX applied to vj = wj = BPX[αj , j, t, Φ]
m∈Ij
αj,m ϕj,m :
j := (ϕj,k , ϕj,m )0;Ω 1: compute gk,m j 2: γj,k := αj,m gk,m m∈Ij
3: for = j − 1 downto 0 do 4: γ,k := ak,p γ+1,p p∈I+1
5: end for 6: β0,k := γ0,k 7: for = 1 to j do 8: α−1,k = a−1 m,k β−1,m m∈I
9: β,k := 2−2t γ,k + α−1,k 10: end for 11: wj := βj,k ϕj,k k∈Ij
4.7
MultiGrid
MultiGrid (MG) methods are nowadays among the fastest and most efficient numerical solvers for linear systems of equations. There is a huge literature on MultiGrid methods (see, e.g. [123]) and even an almost complete survey goes far beyond the scope of this book. Thus, we concentrate only on those issues that are closely related to wavelets. A proof of optimality can be found in [41]. Besides the multiscale structure of solutions to several boundary value problems the main ingredient for MultiGrid methods is the observation that classical iterative schemes like Jacobi, Gauss–Seidel or relaxation schemes have a smoothing property. By this is meant that a few iterations of such schemes reduce the high-frequency part (i.e. the high-level part) of the error but the low-frequency part remains. Hence, the idea is to perform few iterations and then switch to the next coarser level and perform few iterations there, and so on. After the so-called coarse grid correction one switches back to the fine grid and performs a few smoothing steps, also called post-smoothing. There are two variations, namely the so-called V -cycle and the W -cycle which are both shown for three and four levels in Figure 4.3. One fixes the number of presmoothing and post-smoothing steps. Next, one needs transition operators between fine and coarse grids, so-called restriction and prolongation operators: for our case of multiresolution Galerkin methods, the definition of these operators is straightforward. The prolongation pj : Sj → Sj+1
M U LT I G R I D
d d A Ad d A A t
d d A Ad d d A A A t A t
2 1 0
d d A Ad d A Ad d A A t
105
d d A Ad d d A A Ad d d A d d d A A A A A t A t A t A t
3 2 1 0
Fig. 4.3 V -cycle (left) and W -cycle (right) for three and four levels.
coincides with the refinement operator, i.e. ck ϕj,k := ck ajk,m ϕj+1,m . pj (vj ) = pj m∈Ij+1
k∈Ij
k∈Ij
For the restriction, we consider the biorthogonal projection Pj : L2 (Ω) → Sj applied to vj+1 ∈ Sj+1 , i.e. rj : Sj+1 → Sj ,
rj (vj+1 ) := Pj (vj+1 ),
namely for vj+1 =
j+1 αm ϕj+1,m ,
m∈Ij+1
we have rj (vj+1 ) = Pj (vj+1 ) =
k∈Ij
=
(vj+1 , ϕ˜j,k )0;Ω ϕj,k
k∈Ij m∈Ij+1
j+1 αm (ϕj+1,m , ϕ˜j,k )0;Ω ϕj,k
106
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
=
j+1 j αm a ˜k,p (ϕj+1,m , ϕ˜j+1,p )0;Ω ϕj,k
k∈Ij m∈Ij+1 p∈Ij
=
k∈Ij
j+1 j αm a ˜k,m ϕj,k ,
m∈Ij+1
which determines the coefficients αkj :=
j+1 j αm a ˜k,m
m∈Ij+1
on level j, given those on level j + 1. In order to formulate the MultiGrid algorithm, we finally need a smoothing operator Σj : Sj → Sj such as the Jacobi relaxation (where I denotes the identity) Σj vj := (I − ωAj )vj − ωbj . The distinction between V - and W -cycles is done by a parameter µ, where µ = 1 indicates the V -cycle and µ = 2 holds for the W -cycle. Fixing the number ν1 of pre- and ν2 of post-smoothing steps, we obtain Algorithm 4.2.
Algorithm 4.2 MultiGrid-cycle on level j: y j = M G[j, xj , µ, ν1 , ν2 ] 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12:
Presmoothing: y1j := Σνj 1 xj compute the defect: dj := bj −Aj y1j {start coarse-grid correction} restriction: bj−1 = rj−1 (dj ) consider the system Aj−1 zˆj−1 = bj−1 if j = 1 then use exact solution zj−1 = zˆj−1 else if j > 1 then compute an approximation zj−1 = M G[j − 1, 0, µ, ν1 , ν2 ] µ times end if {end coarse-grid correction} y2j := y1j + pj−1 (zj−1 ) Post-smoothing: y3j := Σνj 2 y2j y j := y3j
M U LT I G R I D
107
In order to formulate the convergence statement, one usually first defines the two-grid operator by ν1 Sj,j−1 := Σνj 2 (I − pj−1 A−1 j−1 rj−1 Aj ) Σj .
(4.15)
Of course, A−1 j−1 is to be interpreted in a formal way. For µ ∈ N, one recursively defines the MultiGrid operator for m = 1, . . . , j − 1 by µ ν1 )A−1 Sj,j−m := Σνj 2 (I − pj−1 (I − Sj−1,j−m j−1 rj−1 Aj ) Σj .
(4.16)
Again, µ = 2 denotes the W -cycle and µ = 1 stands for the V -cycle. The following form of the convergence statement is not hard to prove, e.g. [120]. Theorem 4.4 If the two-grid operators Sj,j−1 are bounded and contractive, i.e.
Sj,j−1 ≤ c1 < 1,
j = 2, . . . , ,
and ν1
Σνj 2 pj−1 A−1 j−1 rj−1 Aj Σj ≤ c2 ,
j = 2, . . . , ,
with some uniform constant c2 > 0, then there exists some µ ∈ N such that the MultiGrid operators are uniformly contractive, i.e.
Sj,1 || ≤ c < 1,
j = 2, . . . , ,
with a constant c ∈ (0, 1) independent of . Proof By inserting (4.15) into (4.16) , we obtain µ ν1 A−1 Sj,j−m = Sj,j−1 + Σνj 2 pj−1 Sj−1,j−m j−1 rj−1 Aj Σj .
This can be estimated as follows ν1
Sj,j−m ≤ Sj,j−1 + Σνj 2 pj−1 Sj−1,j−m µ A−1 j−1 rj−1 Aj Σj
≤ c1 + c2 ασ .
(4.17)
Since 0 < c1 < 1, there exists some σ ∈ N such that c1 + c2 ασ = α
108
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
has a solution α ∈ (0, 1). Hence, c1 ≤ α and we use (4.17) recursively starting from
Sj,j−1 ≤ c1 ≤ α < 1 to obtain Sj,1 ≤ α.
The latter theorem shows the asymptotically optimal convergence of the W -cycle with sufficiently many iteration steps µ ∈ N. The analysis of the V -cycle (µ = 1) is more complicated, see [41]. In any case, the uniform contraction property of the MultiGrid operator coincides with the statement of an asymptotically optimal preconditioner. Hence, both methods, the BPX-preconditioned CG method and MultiGrid are asymptotically optimal schemes. 4.8
Numerical examples for the model problem
The theoretical results of all methods are similar, i.e. they are proven to be asymptotically optimal. Hence, numerical experiments can show a comparison of the different approaches in quantitative terms. Thus, we report on several experiments in this chapter. To this end, we consider the periodic boundary value problem in (3.1) for different values of c > 0. First we report on the condition numbers of Aj , the multiresolution Galerkin discretization matrix. In Table 4.5 we show the condition number cond2 (Aj ) for c ∈ {0.1, 1, 10, 1000},
j = 2, . . . , 12,
for the case d = 2 (i.e. piecewise linear scaling functions). We clearly see the exponential increase with respect to the level j. We also see the dependence Table 4.5 Condition numbers of the stiffness matrix Aj for the case d = 2 and different values of c and j. j\c
0.1
1.0
10
100
1000
2 3 4 5 6 7 8 9 10 11 12
640.3 2560.3 10240.3 40960.3 163840.3 655360.1 2621422.4 10485451.3 41939973.8 167758286.4 671023809.8
64.3 256.3 1024.3 4096.3 16325.7 65536.2 261759.0 1048540.5 4194000.7 16763786.5 67103042.4
6.7 25.9 102.7 409.9 1635.4 6552.7 26201.7 104853.2 418839.0 1677575.4 6710555.5
1.0 2.9 10.6 41.3 163.9 655.0 2621.5 10479.8 41942.5 167763.8 671020.7
2.5 1.7 1.4 4.4 16.7 65.9 262.5 1048.8 4194.4 16774.2 67038.6
N U M E R I C A L E X A M P L E S FO R T H E M O D E L P RO B L E M
109
on the parameter c. In fact, the smaller c becomes, the larger is the condition number. This is reasonable, since for increasing c, the stiffness matrix becomes more and more the Gramian matrix of the single scale system. In order to simplify the interpretation of the numbers in Table 4.5, we indicate the factors between two successive levels in Table 4.6. These numbers show the expected behavior, namely that the growth is by a factor of 4, at least asymptotically. They also show that the increase is independent on the particular choice of c (the constant of the reactive term). Next, we want to investigate the dependence on the order of the discretization, i.e. on d. In order to do so, we fix j = 12 and display cond2 (Aj ) for different values of d in Table 4.7. We only see a very mild dependence on d, in particular, the condition numbers are slightly decreasing for growing d. However, one should keep in mind that the corresponding stiffness matrices are more populated for Table 4.6 Factor between condition numbers of level j and j −1 (d = 2). j\c
0.1
1.0
10
100
1000
2 3 4 5 6 7 8 9 10
− 3.998 4.000 4.000 4.000 4.000 4.000 4.000 4.000
− 3.984 3.996 3.999 4.000 4.000 4.000 4.000 4.000
− 3.851 3.961 3.990 3.998 3.999 4.000 4.000 4.000
− 2.816 3.654 3.905 3.976 3.994 3.998 4.000 4.000
− 0.674 0.800 3.263 3.774 3.940 3.985 3.996 3.999
Table 4.7 Condition numbers of stiffness matrix Aj depending on d for fixed level j = 12. The last column labeled ‘∞’ contains the condition number of the mass matrix (without derivatives). d\c
0.1
1.0
10
100
1000
∞
2 3 4 5 6 7 8 9 10
6.71e + 08 2.52e + 08 1.81e + 08 1.45e + 08 1.21e + 08 1.04e + 08 9.14e + 07 8.14e + 07 7.33e + 07
6.71e + 07 2.51e + 07 1.81e + 07 1.45e + 07 1.21e + 07 1.04e + 07 9.14e + 06 8.14e + 06 7.32e + 06
6.71e + 06 2.52e + 06 1.81e + 06 1.45e + 06 1.21e + 06 1.04e + 06 9.13e + 05 8.14e + 05 7.32e + 05
6.71e + 05 2.52e + 05 1.81e + 05 1.45e + 05 1.21e + 05 1.04e + 05 9.14e + 04 8.14e + 04 7.33e + 04
6.71e + 04 2.51e + 04 1.81e + 04 1.45e + 04 1.21e + 04 1.04e + 04 9.13e + 03 8.14e + 03 7.33e + 03
3.00 7.50 18.53 45.72 112.81 278.37 686.80 1694.76 4181.27
110
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
growing values of d. Note that due to periodicity, the value of the dual parameter d˜ does not enter here (this changes once we consider a boundary value problem with Dirichlet boundary conditions). The last column labeled by ‘∞’ shows the condition number of the mass matrix (without any derivative). There, we see the expected growth as 22d . Now we want to compare the different solution methods. For a fair comparison, we fix the discretization for all the comparisons, i.e. we fix the order d and use the same periodic multiresolution. For the right-hand side we choose f ≡ 0, i.e. the solution is u ≡ 0. This allows in particular an easy calculation of the error. As initial vector u(0) we choose a random vector. We compare the four methods, namely • • • •
MGM without preconditioning; diagonal preconditioning, i.e. Cj = diag (Aj ); BPX preconditioning; MultiGrid.
for different values of c as above. In all the following figures we label the horizontal axis with the number of iterations and the vertical axes with the logarithm of the error (and residual, respectively). Using this semilogarithmic plot enables us to easily detect the rate of convergence. In fact a decrease as ραk , ρ ∈ (0, 1), should correspond to a straight line with slope −α. In Figure 4.4 we consider MGM without preconditioning for the case d = 2. In the upper part of the figure, we display the (square) norm of the residual versus the number of iterations and on the lower part the term (x(k) )T Ax(k) (recall that the solution is x ≡ 0). We see that the behavior does not differ significantly so that we will concentrate on (x(k) )T Ax(k) in the sequel. As we could expect from the condition numbers, the scheme becomes better for increasing values of c. We also see the very slow convergence due to the ill-conditioning. At iteration k = 2048, there is a jump down almost to machine accuracy. In Figure 4.5 we show the convergence history for the cases d = 2, 3, 4. We find the jump to machine accuracy in all three cases but at k = 1250 for d = 3 and at k = 1050 for d = 4. Since the solution x = 0 is as regular as we wish, a higher order discretization pays off here, which is the reason for the faster convergence. Note that the scales for the horizontal axes are not the same for the three cases (for graphical reasons). In Figure 4.6, we display the convergence history for diagonal preconditioning. We do not obtain any improvement compared to the case of no preconditioning at all. Note that the diagonal entries ajk,k = a(ϕj,k , ϕj,k ) = (ϕj,k , ϕj,k )0;(0,1) + c (ϕj,k , ϕj,k )0;(0,1) = |ϕj,k |21;(0,1) + c ϕj,k 20;(0,1)
N U M E R I C A L E X A M P L E S FO R T H E M O D E L P RO B L E M 1e + 20
111
.1 1. 10. 100. 1000.
1e + 15 1e + 10 100000 1 1e – 05 1e – 10 1e – 15 1e – 20 0
500 1000 1500 2000 2500 3000 3500 4000 4500
1e + 15 .1 1. 10. 100. 1000.
1e + 10 100000 1 1e – 05 1e – 10 1e – 15 1e – 20
0
500 1000 1500 2000 2500 3000 3500 4000 4500
Fig. 4.4 Iterations of the MGM without preconditioning for the case d = 2. On the top, the (square) norm of the residual is plotted over the number of iterations, on the bottom, the term (x(k) )T Ax(k) (recall that the solution is x ≡ 0). are independent of k. Thus, the diagonal is constant and diagonal preconditioning reduces to the multiplication with a constant. This, however, does not change the condition number at all which means that we had to expect that diagonal preconditioning is worthless here. Next, we consider the BPX preconditioner in Figure 4.7. We obtain much faster convergence (as expected, of course). Again note that the horizontal axes are labeled in a different way for the three cases. It turns out that we obtain the fastest convergence for d = 2 and increasing number of iterations for increasing values of d. Moreover, we see that the scheme is faster as c decreases, i.e. the more diffusion-dominated the problem becomes. Finally, we consider the MultiGrid method in Figure 4.8. Note here the values on the horizontal axes which clearly show how much faster MultiGrid is compared to the BPX preconditioner. Moreover, we see that the method is absolutely
112
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S 1e + 15
.1 1. 10. 100. 1000.
1e + 10 100000 1 1e – 05 1e – 10 1e – 15 1e – 20
0
500 1000 1500 2000 2500 3000 3500 4000 4500
1e + 10 .1 1. 10. 100. 1000.
100000 1 1e – 05 1e – 10 1e – 15 1e – 20
0
200 400 600 800 1000 1200 1400 1600 1800 2000
1e + 10 100000 1
.1 1. 10. 100. 1000.
1e – 05 1e – 10 1e – 15 1e – 20 1e – 25 0
200 400 600 800 1000 1200 1400 1600 1800 2000
Fig. 4.5 Iterations of the MGM without preconditioning for the cases d = 2 (top), d = 3 (middle) and d = 4 (bottom).
1e + 10 1e + 08 1e + 06 10000 100 1 0.01 0.0001 1e – 06 1e – 08 1e – 10 2500 0
1e + 15 .1 1. 10. 100. 1000.
1e + 10 100000 1 1e – 05 1e – 10 1e – 15 0
500
1000
1500
2000
.1 1. 10. 100. 1000.
200
1e + 10
400
600
800
1000 1200 1400
.1 1. 10. 100. 1000.
1e + 08 1e + 06 10000 100 1 0.01 0.0001 1e – 06 1e – 08 1e – 10 0
200
400
600
800
1000
1200
Fig. 4.6 Iterations of the MGM with diagonal preconditioning for the cases d = 2 (top left), d = 3 (top right) and d = 4 (bottom). 1e + 10
1e + 15 .1 1. 10. 100. 1000.
1e + 10 100000
1
1 1e – 05
1e – 05
1e – 10
1e – 10
1e – 15 1e – 20 0
.1 1. 10. 100. 1000.
100000
1e – 15 20
40
60
80
100
120
0
50
100
150
200
250
1e + 10 100000 1
.1 1. 10. 100. 1000.
1e – 05 1e – 10 1e – 15
0 50 100 150 200 250 300 350 400 450
Fig. 4.7 Iterations of the BPX preconditioned CG method for the cases d = 2 (top left), d = 3 (top right) and d = 4 (bottom).
114
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S 1e + 20 .1 1. 10. 100. 1000.
1e + 15 1e + 10 100000 1 1e – 05 1e – 10 1e – 15 1e – 20 0
5
10 1e + 20 1e + 15 1e + 10 100000 1 1e – 05 1e – 10 1e – 15 1e – 20 0
15
20
25
1e + 20 1e + 15 1e + 10 100000 1 1e – 05 1e – 10 1e – 15 1e – 20
.1 1. 10. 100. 1000.
0
2
4
6
8
10
12
14 16
.1 1. 10. 100. 1000.
2
4
6
8
10
12
14
Fig. 4.8 Iterations of the MultiGrid method for the cases d = 2 (top left), d = 3 (top right) and d = 4 (bottom).
robust with respect to the choice of c (all marks are at the same position which is the reason that only the filled boxes are visible). Finally, we compare the computing times for BPX and MultiGrid. In order to do so, we fix c = 0.1 and the order to d = 3 For the restriction operator in the MultiGrid scheme, we use d˜ = 5. The CPU times are listed in Table 4.8 and again show the clear dominance of the MultiGrid method. We also compare different trial and test functions: • Daubechies scaling functions; • primal B-spline scaling functions; • dual scaling functions. Since all experiments give quantitatively similar results, we only report on the BPX precoditioned CG method. In Figure 4.9, we show the convergence history for Daubechies scaling functions for N = 2, . . . , 8. In Figures 4.10 and 4.11, we show similar results for the dual scaling functions. Finally, Figure 4.12 contains a comparison of primal, dual and Daubechies functions with comparable properties. The qualitative behavior is the same in all cases. In quantitative terms, Daubechies functions win in particular if the zero-order term is dominant. This was expected due to the orthonormality of the functions. However, one should also keep in mind that plotting the solution would amount to computing
E X E RC I S E S A N D P RO G R A M S
115
Table 4.8 Comparison of CPU times for BPX and MultiGrid using c = 0.1, d = 3 (and d˜= 5 for the restriction). j
MGM-BPX
MultiGrid
10 11 12 13 14 15 16 17 18 19 20 21 22 23
0.03 0.08 0.13 0.29 0.6 1.3 2.67 5.73 12.33 26.42 55.44 118.36 266.18 544.33
0.02 0.03 0.04 0.08 0.17 0.32 0.65 1.38 2.77 5.91 12.59 25.45 53.82 137.61
point values which is less efficient for Daubechies functions due to the lack of a closed formula.
4.9
Exercises and programs
Exercise 4.1 Let A ∈ Rn×n be a tridiagonal matrix of the form
a
c A=
b ..
.
..
.
..
.
..
.
..
.
.
..
.
..
c
b a
with constants a, b, c ∈ R and b · c > 0. Show that the eigenvalues have the form √ λk = 2 bc cos
kπ n+1
+ a,
k = 1, . . . , n
116
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S
1e + 15
1e + 15
.1 1. 10. 100. 1000.
1e + 10 100000
100000
1
1
1e – 05
1e – 05
1e – 10
1e – 10
1e – 15 0
10
20
30
40
50
60
.1 1. 10. 100. 1000.
1e + 10
70
1e – 15
0
.1 1. 10. 100. 1000.
1e + 10 100000
20
30
40
50
60
70
100000 1
1e – 05
1e – 05
1e – 10
1e – 10
1e – 15
0
10
20
30
40
1e + 15
50
60
.1 1. 10. 100. 1000.
1e + 10 100000
1e – 15
0 1e + 15
1e – 05
1e – 05
1e – 10
1e – 10 20
30
40
50
20
30
40
60
1e – 15
0
50
60
.1 1. 10. 100. 1000.
100000 1
10
10
1e + 10
1
0
.1 1. 10. 100. 1000.
1e + 10
1
1e – 15
10
1e + 15
1e + 15
10
20
30
40
50
60
Fig. 4.9 Iterations of the BPX preconditioned CG method using Daubechies scaling function with N = 3, . . . , 8 (3, 4 top; 5, 6 middle; 7, 8 bottom, from left to right). and determine the corresponding eigenvectors. Under what conditions on the parameters is A symmetric and positive definite? Exercise 4.2 Determine an error estimate for u − uj 1;Ω using piecewise quadratic B-spline scaling functions. Exercise 4.3 Determine an error estimate for u − uj 1;Ω using Daubechies scaling functions N ϕ.
1e + 15
1e + 15
.1 1. 10. 100. 1000.
1e + 10 100000
.1 1. 10. 100. 1000.
1e + 10 100000 1
1 1e – 05 1e – 05
1e – 10
1e – 10
1e – 15
1e – 15 0
50
100
150
200
250
300
1e + 15
100000
20
40
60
80
100
1e + 15
.1 1. 10. 100. 1000.
1e + 10
1e – 20 0
120
140
.1 1. 10. 100. 1000.
1e + 10 100000 1
1 1e – 05 1e – 05
1e – 10
1e – 10 1e – 15
1e – 15 0
20
40
60
80
100
120
140
1e – 20 0
20
40
60
80
100
120
Fig. 4.10 Iterations of the BPX preconditioned CG method for dual scaling ˜ namely d˜ = 4, 6, 8, 10. functions with d = 2 and different values of d, 1e + 15
1e + 15 .1 1. 10. 100. 1000.
1e + 10 100000
100000
1
1
1e – 05
1e – 05
1e – 10
1e – 10
1e – 15
0
50 100 150 200 250 300 350 400 450 1e + 15 1e + 10 100000
.1 1. 10. 100. 1000.
1e + 10
1e – 15
0
50 100 150 200 250 300 350 400 .1 1. 10. 100. 1000.
1 1e – 05 1e – 10 1e – 15 0
50 100 150 200 250 300 350 400
Fig. 4.11 Iterations of the BPX preconditioned CG method for dual scaling function. First row: d = 3, left: d˜= 7, right: d˜= 9, second row d = 4, d˜= 10.
118
M U LT I R E S O L U T I O N G A L E R K I N M E T H O D S 1e + 15
1e + 10 .1 1. 10. 100. 1000.
100000 1
.1 1. 10. 100. 1000.
1e + 10 100000 1
1e – 05
1e – 05
1e – 10 1e – 15
1e – 10 0
50 100 150 200 250 300 350 400 450
1e – 15 0
50
100
150
200
250
300
1e + 15 .1 1. 10. 100. 1000.
1e + 10 100000 1 1e – 05 1e – 10 1e – 15
0
10
20
30
40
50
60
70
Fig. 4.12 Iterations of the BPX preconditioned CG method. Comparison of primal, dual and Daubechies functions. First row, primal functions for d = 4, second duals with d = 2, d˜ = 4 and third row Daubechies with N = 4.
Exercise 4.4 Collect from the literature the proof that the BPX preconditioner is asymptotically optimal. Exercise 4.5 Collect from the literature the proof that MultiGrid is asymptotically optimal. Computer Exercise 4.1 Realize the ILU preconditioner. Collect the necessary information from the literature. Compare your preconditioner with the BPX preconditioner. Computer Exercise 4.2 Change the number of pre- and post-smoothing steps in the MultiGrid scheme and investigate numerically how the scheme behaves. Programs Besides the following codes we provide software realizing the following issues: • Refinable integrals. • General interface that allows to introduce own preconditioners.
E X E RC I S E S A N D P RO G R A M S
119
• Numerical integration. • Efficient direct solver for tridiagonal matrices. Program 4.1 The code bpx-solve.cc realizes the BPX preconditioned conjugate gradient method. Program 4.2 The code multigrid-solve.cc realizes the MultiGrid method for the MGM. Program 4.3 The code setup-system.cc computes the linear system of equations for the MGM. The scaling functions and the right-hand side can be chosen. Program 4.4 The code condition-MGM.cc computes the condition number of the stiffness matrix for the MGM and produces the numbers in Table 4.1. Program 4.5 The code spy-MGM.cc computes the stiffness matrix for the MGM and produces spy pictures.
5 WAVELETS So far, we have not used any wavelet in the previous chapters of this book. We have only used MRAs for the discretization and numerical solution of boundary value problems. As already said earlier, an MRA also serves as the main tool for the construction of wavelets. We will make this precise now and describe the construction and the main properties of wavelets for the numerical solution of boundary value problems. 5.1 5.1.1
Detail spaces Updating
As we have seen by the Jackson estimate in Proposition 2.20, we obtain a better multiresolution approximation by using a higher level. However, there is a price to be paid since typically |Ij | ∼ 2j , i.e. the number of degrees of freedom grows exponentially with the level j. If an already computed approximation turns out to be not sufficiently accurate, the complete computation of an approximation on the next higher level would be by far too expensive. One would like to reuse the already computed approximation and only be forced to update this in order to obtain a better approximation. In order to do so, we may ask the following question: Given a function fj+1 ∈ Sj+1 , how can a detailed gj in the sense fj+1 = fj + gj be represented, where fj = Pj fj+1 is the coarse level part of fj+1 ? Hence, we are interested in that part of the space Sj+1 which is not already included in Sj . In particular, we would like to have a representation of these details, e.g. in terms of a basis. In other words, what is a basis for a stable complement Wj of Sj in Sj+1 ? Here, stable means that the angle between Wj and 120
D E TA I L S PAC E S
121
Sj is uniformly bounded. The optimum is of course again if Wj is the orthogonal complement of Sj in Sj+1 , which, on the other hand, is very restrictive. Obviously, there are infinitely many complements of Sj in Sj+1 . This can also be described in terms of the corresponding projectors. Given a projector Pj : L2 (Ω) → Sj , one could define – as an example – the following operators QIj := Id − Pj ,
QII j := Pj+1 − Pj .
The ultimate goal for finding the “right” complement is stability. Then, given bases for the complement spaces, the collection of all these complement bases give rise to a stable basis of the whole space L2 (Ω). 5.1.2
The Haar system again
In the simple example of the Haar system it is readily seen how to characterize the orthogonal complement. In fact, let us define ψ Haar (x) := ϕHaar (2x) − ϕHaar (2x − 1),
(5.1)
the so-called Haar wavelet which is also shown in Figure 5.1. Then, it is easy to see that on Ω = [0, 1] ΨHaar := {ψ[j,k−2j ] : k ∈ JjHaar }, j Haar \ IjHaar JjHaar := {2j , . . . , 2j+1 − 1} = Ij+1
is an ON-basis for Wj defined by Sj+1 = Sj ⊕ Wj ,
Sj ⊥Wj ,
(5.2)
where orthogonality is to be understood w.r.t. the inner product (·, ·)0;Ω . We leave the proof as an easy exercise for the reader. On Ω = R, the above index 1
1
0.5
0.5
0
0
–0.5
–0.5 –1
–1 –0.5
0
0.5
1
1.5
–0.5
0
0.5
1
Fig. 5.1 Haar scaling function (left) and Haar wavelet (right).
1.5
122
WAV E L E T S
set JjHaar has to be replaced by Z, i.e. all integer translates. For the example of the hat function the construction of such orthogonal complements is not at all trivial. We iterate (5.2) by decomposing Sj = Sj−1 ⊕ Wj−1 and so on until we reach a coarsest level, say j = 0. This means that Sj+1 = S0 ⊕
j 5
W .
(5.3)
=0
Hence the spaces W represent the details corresponding to level . For a function fj+1 ∈ Sj+1 , this gives rise to a decomposition of the form fj+1 = f0 +
j
f0 = P0 fj+1 ∈ S0 ,
g ,
g ∈ W ,
=0
where g is the detail of fj+1 on level . Such a decomposition is called a multiscale or multilevel representation. Obviously, this representation can only be stable if the multilevel splitting (5.3) itself is stable, which is trivially the case for an orthogonal decomposition. Figure 5.2 shows a piecewise constant approximation to a given function, its coarse level part and the details represented by Haar wavelets. If Ψj is an ON-basis for Wj , each g can also be written as the orthogonal projection of fj+1 onto W in the following form g = Q fj+1 =
(fj+1 , ψ[,k] )0;R ψ[,k] ,
k∈Z
with Q : L2 (R) → W . Obviously, we have Q = P+1 − P , which can easily be seen by the following direct computations. In fact, we have (P f, (P+1 − P )f )0;R = (P f, P+1 f )0;R − (P f, P f )0;R = 0
D E TA I L S PAC E S
123
Fig. 5.2 Given piecewise constant approximation (solid) to a given function (top), approximation by a coarse scale part (middle) and complement (bottom). because (P f, P+1 f )0;R =
(f, ϕ[,k] )0;R (f, ϕ[+1,m] )0;R (ϕ[,k] , ϕ[+1,m] )0;R
k∈Z m∈Z
1 =√ (f, ϕ[,k] )0;R (f, ϕ[+1,m] )0;R 2 k∈Z m∈Z
× (ϕ[+1,2k] , ϕ[+1,m] )0;R + (ϕ[+1,2k+1] , ϕ[+1,m] )0;R ) *+ , ) *+ , =δm,2k
=
=δm,2k+1
(f, ϕ[,k] )0;R
k∈Z
= 1 < f, √ ϕ[+1,2k] + ϕ[+1,2k+1] 2
0;R
= (P f, P f )0;R . This means that (P+1 − P )f ⊥S for all functions f ∈ L2 . Moreover, it is easily seen that (P+1 − P )w = w for w ∈ W and (f − (P+1 − P )f, (P+1 − P )f )0;R = 0,
124
WAV E L E T S
which shows that P+1 − P coincides with the orthogonal projector Q . Note that we have only used the refinement equation and orthonormality of the integer translates of ϕ. We show f , Pjf , Qjf and the corresponding error for f (x) := sin(2πx) in Figure 5.3. 5.2
Orthogonal wavelets
For simplicity, we start again by considering a MRA on L2 (R) using all integer shifts of a scaling function with orthonormal integer translates. We will consider later the adaptation to bounded intervals and domains. 5.2.1
Multilevel decomposition
We aim at constructing the orthogonal complement Wj of Sj in Sj+1 and in particular at constructing an orthonormal basis Ψj := {ψ[j,k] : k ∈ Z} for Wj . Obviously this implies that the complement spaces Wj and consequently the basis functions ψ[j,k] are mutually orthogonal with respect to different levels j. In fact, for < j, we obtain W ⊂ S+1 ⊆ Sj ⊥Wj and hence (ψ[j,k] , ψ[,m] )0;R = 0
for all j = .
Thus, each g ∈ W is the orthogonal projection of fj onto W , i.e. g =
(fj , ψ[,k] )0;R ψ[,k]
k∈Z
and finally fj = f0 +
j−1
gl
l=0
=
k∈Z
(fj , ϕ[0,k] )0;R ϕ[0,k] +
j−1
(fj , ψ[l,k] )0;R ψ[l,k] .
l=0 k∈Z
Again, we can iterate the decomposition Sj+1 = Sj ⊕ Wj and obtain Sj+1 = Sj ⊕ Wj = Sj−1 ⊕ Wj−1 ⊕ Wj =
j 5 =−∞
W ;
(5.4)
Fig. 5.3 Considering f (x) := sin(2πx) each row shows Pj f (first column), the error f − Pj f (second column), the detail Qj f (third column) and the same detail with respect to an adjusted vertical axis for the levels j = 1 (top row) to 8 (bottom).
126
WAV E L E T S
thus by Definition 2.4 (b) L2 (R) =
∞ 5
Wj ,
j=−∞
so that due to the orthonormality we obtain that Ψ := Ψj = {ψ[j,k] : j, k ∈ Z} j∈Z
forms an orthonormal basis in L2 (R). Then, Ψ is called an orthonormal wavelet basis. 5.2.2
The construction of wavelets
The above statements require knowledge of an orthonormal basis Ψj for Wj , which in turns would be very easy if one could construct an appropriate function ψ ∈ W0 (the so-called mother wavelet). Let us now describe this construction. Since ψ ∈ W0 ⊂ S1 , it amounts to defining appropriate coefficients {bk }k∈Z such that bk ϕ(2x − k), ψ(x) =
(5.5)
k∈Z
where again we only consider cases where b = {bk }k∈Z is a finite vector. In the same way as (2.31) above, (5.5) implies √1 bl−2k ϕ[j+1,l] (x). (5.6) ψ[j,k] (x) = 2 l∈Z
We describe particular choices for the wavelet coefficients bk in the sequel. As we shall see next, all the work to be done for the construction of orthonormal wavelets lies in the construction of an orthonormal scaling function which clarifies why an MRA is such a useful tool for constructing wavelets. In fact, given a compactly supported scaling function with orthonormal integer translates, the corresponding wavelet is uniquely determined as the following statement shows. Proposition 5.1 Given a compactly supported orthonormal scaling function ϕ with mask a, then there exists a uniquely determined compactly supported wavelet ψ which is given by bk = (−1)k a1−k in (5.5).
O RT H O G O N A L WAV E L E T S
127
Proof We first show that ψ defined by the above mask b in fact is an orthonormal wavelet. Let ψ be defined by the mask b. Obviously, we have (ψ, ϕ(· − k))0;R =
am b2k+m =
m∈Z
=
m∈Z
a2l a1−2l−2k −
l∈Z
=
(−1)m am a1−m−2k
a2l+1 a1−(2l+1)−2k
l∈Z
a2l a1−2(l+k) −
l∈Z
a2l a1−2(l+k)
l∈Z
= 0. On the other hand, by Proposition 2.7 and substitution of integration variables (ψ, ψ(· − k))0;R =
bm bl (ϕ(2 · −m), ϕ(2 · −2k − l))0;R
m∈Z l∈Z
=
1 bm bl (ϕ(·), ϕ(· − (2k + l − m)))0;R 2 m∈Z l∈Z
=
1 1 bm bm−2k = a1−m a1−m+2k 2 2 m∈Z
m∈Z
1 am am+2k = 2 m∈Z
= δ0,k for all k ∈ Z. Hence ψ ∈ S1 is a function with orthogonal translates and ψ⊥S0 , hence ψ ∈ W0 . It remains to show that the translates of ψ form a basis of W0 . The refinement relations lead to (Pj + Qj )ϕ[j+1,k] =
(ϕ[j+1,k] , ϕ[j,m] )0;R ϕ[j,m] +
m∈Z
(ϕ[j+1,k] , ψ[j,m] )0;R ψ[j,m]
m∈Z
1 =√ al−2m (ϕ[j+1,k] , ϕ[j+1,l] )0;R ϕ[j,m] ) *+ , 2 m∈Z l∈Z =δk,l
1 +√ bl−2m (ϕ[j+1,k] , ϕ[j+1,l] )0;R ψ[j,m] ) *+ , 2 m∈Z l∈Z =
m∈Z
√1 ak−2m 2
ϕ[j,m] +
m∈Z
=δk,l
√1 bk−2m 2
ψ[j,m]
128
WAV E L E T S
=
1 2 ak−2m
al−2m ϕ[j+1,l]
m∈Z l∈Z
+
1 2 bk−2m bl−2m
ϕ[j+1,l]
m∈Z l∈Z
=
l∈Z
1 2
)
(ak−2m al−2m + bk−2m bl−2m ) ϕ[j+1,l] .
m∈Z
*+
,
=:cj+1 l,k
We would have finished the proof if we can show that cj+1 l,k = δl,k . By definition, we have 1 1 ak−2m al−2m + (−1)k+l a1−k+2m a1−l+2m cj+1 l,k = 2 2 m∈Z
m∈Z
1 = ak−2m al−2m + (−1)k+l a1−k+2m a1−l+2m . 2 m∈Z
Let us first consider the case k = l. Then 1 2 1 2 1 2 cj+1 ak−2m + a1−k+2m = am = 1 k,k = 2 2 2 m∈Z
m∈Z
(5.7)
m∈Z
due to Proposition 2.7 Let us now consider the case k = l. First assume that both are even, k = 2n, l = 2p, n = p. Then, 1 1 cj+1 a2n−2m a2p−2m + a2m+1−2n a2m+1−2p l,k = 2 2 m∈Z
m∈Z
1 = a2n+m a2p+m 2 m∈Z
1 = am a2(p−n)+m 2 m∈Z
= δp,n = 0. The case that both k and l are odd is treated similarly. Finally, let k = 2n + 1 be odd and l = 2p be even. Then, 1 1 cj+1 a2n+1−2m a2p−2m − a2m+1−(2n+1) a2m+1−2p l,k = 2 2 m∈Z
m∈Z
1 1 = a2n+1−2m a2p−2m − a2m+2n a2m+1+2p 2 2 m∈Z
m∈Z
=0 and the case k even and l odd is completely analogous.
O RT H O G O N A L WAV E L E T S
129
Finally, we have to show that ψ is in fact uniquely determined. Let us assume that a second function ψ ∗ is given such that ψ ∗ (x) =
m2
ck ψ(x − k),
m1 < m2 ,
cm1 , cm2 = 0,
(5.8)
k=m1 ∗ : k ∈ Z} is an orthonormal basis for W0 . In fact, since we have already and {ψ[0,k] shown that {ψ[0,k] : k ∈ Z} is an ON-basis for W0 , we can represent ψ ∗ as in (5.8). Then, we obtain
ck = (ψ ∗ , ψ(· − k))0;R and δ0, = (ψ ∗ , ψ ∗ (· − ))0;R =
m2 m2
ck cn (ψ(· − k), ψ(· − ( + n)))0;R
k=m1 n=m1
=
m2
cn c+n .
n=m1
Since m1 < m2 , we can use the latter equation for = m2 −m1 = 0 and obtain 0=
m2
cn cm2 −m1 +n = cm1 cm2 ,
n=m1
which contradicts the assumption cm1 , cm2 = 0 in (5.8).
The orthonormal wavelets are displayed for N = 2, . . . , 5 in Figure 5.4 (page 130). We have already seen that the Daubechies orthonormal scaling functions are to some extent uniquely defined. Thus, also compactly supported orthonormal wavelets are (basically) uniquely defined, are denoted by N ψ, and are called Daubechies wavelets. 5.2.3
Wavelet projectors
In terms of projectors, we consider a wavelet projector similar to the Haar case above by QII j := Pj+1 − Pj . Of course, we expect that QII j is the orthogonal projector onto Wj which can be represented in terms of the orthogonal basis Ψj . This can in fact be seen by the following result.
130
WAV E L E T S 2
2
1.5
1.5
1
1
0.5
0.5
0
0
–0.5
–0.5
–1
–1
–1.5
–1.5
0
0.5
1
1.5
2
2.5
3
0
1.5
1.5
1
1
1
2
3
4
5
0.5
0.5
0 0
–0.5
–0.5 –1
–1 0
1
2
3
4
5
6
7
–1.5
0
1
2
3
4
5
6
7
8
9
Fig. 5.4 Daubechies orthonormal wavelets for N = 2, 3 (top) and N = 4, 5 (bottom), both from left to right.
Proposition 5.2 The projector QII j has the representation QII j f =
(f, ψ[j,k] )0;R ψ[j,k] = Qj f
k∈Z
for all f ∈ L2 (R). Proof Since obviously QII j f = (Pj+1 − Pj )f ∈ Sj+1 and (QII j f, ϕ[j,k] )0;R =
(f, ϕ[j+1,m] )0;R (ϕ[j+1,m] , ϕ[j,k] )0;R
m∈Z
−
(f, ϕ[j,m] )0;R (ϕ[j,m] , ϕ[j,k] )0;R
m∈Z
=
1 √ al−2k (f, ϕ[j+1,m] )0;R (ϕ[j+1,m] , ϕ[j+1,l] )0;R 2 m∈Z l∈Z − (f, ϕ[j,k] )0;R
B I O RT H O G O N A L WAV E L E T S
=
131
1 √ am−2k (f, ϕ[j+1,m] )0;R − (f, ϕ[j,k] )0;R 2 m∈Z
= (f, ϕ[j,k] )0;R − (f, ϕ[j,k] )0;R = 0, II i.e. QII j f ⊥Sj , then we have Qj f ∈ Wj . Finally, since (QII (f, ϕ[j+1,k] )0;R (ϕ[j+1,k] , ψ[j,l] )0;R j f, ψ[j,l] )0;R = k∈Z
=
1 √ bm−2l (f, ϕ[j+1,k] )0;R (ϕ[j+1,k] , ϕ[j+1,m] )0;R 2 k∈Z m∈Z
=
1 √ bk−2l (f, ϕ[j+1,k] )0;R 2 k∈Z
= (f, ψ[j,l] )0;R , we obtain QII j f =
(QII j f, ψ[j,k] )0;R ψ[j,k] =
k∈Z
(f, ψ[j,k] )0;R ψ[j,k] ,
k∈Z
which proves that QII j in fact coincides with the orthogonal projector Qj .
5.3
Biorthogonal wavelets
So far, we have only considered orthonormal wavelets. As already mentioned above, this is a very restricted class of functions. The only compactly supported scaling functions (with maximal degree of polynomials that are reproduced and minimal support) are those of the family of Daubechies scaling functions shown above. Moreover, as we have seen by Proposition 5.1, orthonormal wavelets are uniquely determined by the underlying scaling function. Finally, it turns out that orthonormality poses a serious restriction which is counterproductive in some applications. Hence, we drop the orthonormality but impose that the wavelet basis Ψ is a Riesz basis for L2 (R) which is defined as follows. Definition 5.3 A countable collection of elements H := {hi }i∈I (I ⊆ Z) of a Hilbert space H is called a Riesz basis for H if each element in H has an expansion in terms of H and there exist constants 0 < cH ≤ CH such that
2
|di |2 ≤ d h |di |2 . (5.9) cH i i ≤ CH
i∈I
i∈I
H
i∈I
132
WAV E L E T S
For the wavelets, this means
2
cΨ |dj,k |2 ≤ d ψ j,k [j,k]
j,k∈Z
0;R
j,k∈Z
≤ CΨ
|dj,k |2 ,
or, in shorthand notation with d = (dj,k )j,k∈Z , 1/2
dj,k ψ[j,k] |dj,k |2 = d 2 (Z2 ) .
∼
j,k∈Z
0;R
(5.10)
j,k∈Z
(5.11)
j,k∈Z
Equations (5.10) and (5.11) state that the norm of an expansion can uniformly be estimated from above and below by the sequence norm of the expansion coefficients. Note that an orthonormal basis is a special case of a Riesz basis where in (5.11) “∼” is replaced by “=”. 5.3.1
Biorthogonal complement spaces
In the orthogonal case, Wj is the orthogonal complement of Sj in Sj+1 . In the biorthogonal framework, Wj is still a complement of Sj in Sj+1 but now along S˜j . In this sense, the dual multiresolution controls the angle between Sj and Wj . ˜ j reads Hence, the definition of Wj and W Sj+1 = Sj ⊕ Wj ,
Wj ⊥S˜j ,
˜ j, W ˜ j ⊥Sj . S˜j+1 = S˜j ⊕ W
(5.12)
Consequently, we look for two biorthogonal wavelets ψ(x) := bk ϕ(2x − k) ∈ S1 , k∈Z
˜ ψ(x) :=
˜bk ϕ(2x ˜ − k) ∈ S˜1
k∈Z
(for suitable coefficients bk , ˜bk , k ∈ Z) such that the functions are biorthogonal, i.e. (ψ[j,k] , ψ˜[j ,k ] )0,R = δj,j δk,k ,
j, j , k, k ∈ Z,
(5.13)
and the families Ψj := {ψ[j,k] : k ∈ Z},
˜ j := {ψ˜[j,k] : k ∈ Z} Ψ
˜ j , respectively. In addition, we require that the are Riesz bases for Wj and W collection of all wavelets ˜ := ˜j Ψj , Ψ Ψ Ψ := j∈Z
j∈Z
B I O RT H O G O N A L WAV E L E T S
133
form biorthogonal Riesz bases in L2 (R), i.e. we want to obtain norm equivalences (5.11) for primal and dual systems, i.e.
2
∼ d ψ |dj,k |2 , (5.14) j,k [j,k]
j∈Z k∈Z
0;R
2
˜
dj,k ψ[j,k]
j∈Z k∈Z
0;R
j∈Z k∈Z
∼
|dj,k |2 .
(5.15)
j∈Z k∈Z
The question of under what conditions this can be achieved will be discussed later. Note that the biorthogonality with respect to the level in (5.13) is an immediate consequence of (5.12). The biorthogonality on one level, in particular with ˜ respect to the translates, is subject to the choice of the masks b and b. 5.3.2
Biorthogonal projectors
Again, we define projectors onto the detail spaces by QII j := Pj+1 − Pj ,
˜ II := P˜j+1 − P˜j , Q j
and, as in the orthogonal case, we obtain the following result. ˜ II Proposition 5.4 The projectors QII j and Qj have the representation QII j f =
(f, ψ˜[j,k] )0;R ψ[j,k] = Qj f,
k∈Z
˜ II f = Q j
˜ j f, (f, ψ[j,k] )0;R ψ˜[j,k] = Q
k∈Z
˜ II for f ∈ L2 (R), i.e. QII j and Qj coincide with the biorthogonal projectors Qj ˜ j , respectively. and Q Proof The proof can easily be done following the lines of the proof of Proposition 5.2, see Exercise 5.5.
Proposition 5.5 The wavelet coefficients are determined by the refinement coefficients of the biorthogonal scaling function in the following way bk = (−1)k a ˜1−k ,
˜bk = (−1)k a1−k .
(5.16)
Proof The proof is similar to the proof of Proposition 5.1 above; see Exercise 5.6.
134
WAV E L E T S
5.3.3
Biorthogonal B-spline wavelets
Let us denote by d,d˜ψ,
˜
d,d˜ψ
the biorthogonal wavelets induced by the dual scaling functions d ϕ and mentioned above, the biorthogonal wavelets are defined by bk = (−1)k a ˜1−k ,
˜ d,d˜ϕ.
As
˜bk = (−1)k a1−k ,
where ak are the refinement coefficients of d ϕ and a ˜k those of d,d˜ϕ. ˜ Note that both primal and dual wavelet depend on both parameters d and d˜ which is not the case for the primal scaling function. Since by (5.16) the dual refinement coefficients are used to define the primal wavelet coefficients, the dependence on both parameters becomes clear. Now, we collect several plots of biorthogonal scaling functions and wavelets. The following figures contain detailed plots of primal and dual scaling functions and wavelets: • • • • • • •
Figure Figure Figure Figure Figure Figure Figure
5.5: d = 1, d˜ = 1, 3, 5, 7; 5.6: d = 2, d˜ = 2, 4, 6, 8; 5.7: d = 3, d˜ = 1, 3, 5, 7; 5.8: d = 4, d˜ = 2, 4, 6, 8; 5.9: d = 5, d˜ = 0, 1, 3, 5; 5.10: d = 6, d˜ = 8; 5.11: d = 9, d˜ = 7.
Note that again for d ≥ 4 the biorthogonal wavelets do not coincide with those from Cohen, Daubechies and Feauveau. Moreover, we also display some functions with d˜ ≤ d. The corresponding filters are constructed by the general approach in [71], but in these cases the duals are in general no longer L2 -functions. The filters corresponding to d = 9, d˜ = 7 are used in jpeg2000 . Remark 5.6 As already noted in Table 2.7 (page 44), wavelet coefficients taken from standard software sometimes do not coincide with the B-spline mask coefficients, e.g. in the cases d = d˜= 4 or d = 9, d˜= 7. Since the wavelets are determined in terms of the scaling function coefficients, this fact of course also holds for the wavelet functions.
5.4
Fast Wavelet Transform (FWT)
Now we have two different representations for any fj ∈ Sj , namely fj =
(f, ϕ˜[j,k] )0;R ϕ[j,k] ,
k∈Z
FA S T WAV E L E T T R A N S FO R M ( F W T )
135
1.2 1 0.8 0.6 0.4 0.2 0
0
0.2
0.4
~
0.6
1
0.8
1.2
~
1.2
1
1
0.5
0.5
0
0
–0.5
–0.5
1 0.8 0.6 0.4 0.2 0
0
0.2
0.4
0.6
1
0.8
1.2
1.2
–1 –0.5
0
0.5
1
1.5
1
1
0.5
0.8 0.6
0
0.4
–0.5
0.2 –1
0
–0.2 –1.5 –2 –3 –2.5 –2 –1.5 –1 0.5 0 0.5 1 1.5 2 1.2
–1
–0.5
0
0.5
1
1 0.8 0.6 0.4 0.2 0 –0.2 –0.4 –0.6 –0.8 –1
–1
0
–0.5
0.5
0
0.5
1
1
1
0.6 0.4 0.2 0 –0.2 –5 –4 –3 –2 –1
0
1
2
3
1.2 1 0.8 0.6 0.4 0.2 0 –0.2 –6
–4
–2
0
2
4
1.5
1.5
2
1 0.8 0.6 0.5 0.4 0.2 0 0 –0.2 –0.5 –0.4 –0.6 –1 –0.8 –1.5 –1 –1 0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 4 4 3.5 3 2.5 2 1.5 1 0.5 0 –0.5 1 1 1 0.8 0.6 0.5 0.4 0.2 0 0 –0.2 –0.5 –0.4 –0.6 –1 –0.8 –1 –1.5 –1 0 1 2 3 4 5 6 6 –6 –5 –4 –3 –2 –1 0 1 1
0.8
–0.4
–1.5
–1 –0.5
Fig. 5.5 Biorthogonal B-spline wavelets for d = 1 and d˜ = 1, 3, 5, 7.
136
WAV E L E T S
1.4 1.2 1 0.8 0.6 0.4 0.2 0 –1 –0.8 –0.6 –0.4 –0.2 ~
10
0
0.2 0.4 0.6 0.8
1
1.4 1.2
8
8
1 6
0.8
4
0.6
6 4 2
0.4 2
0.2
0
0
0
–2
–0.2
–2
–4
–0.4
–4 –2
–1.5
–1
–0.5
0
0.5
1
1.5
–0.6 –1 2
2
~
10
–0.5
0
0.5
1
1.5
2
–6 –2
–1.5
–1
–0.5
0
0.5
1
1.5
1.4 1.2
1.5
1
1 0.8 0.6
1
0.5
0.4 0.5
0
0.2 0 –0.2
0
–0.5
–0.4 –0.5 –4
–3
–2
–1
0
1
2
3
4
–0.6 –3 –2.5 –2 –1.5 –1 –0.5
0
0.5
1
1.5
2
1.2
1.6
–1 –2 –1.5 –1 –0.5
1.4
1
1
1.2
0.8
0.8
1
0
–0.2
–0.2
–0.4
–0.4 –4
–2
0
2
4
6
–0.6 –5
–4
–3
–2
–1
0
1
2
–0.8 –2
1
1
1
0.8
0.8
0.6
0.6
–0.2
–0.2
–0.4
–0.4 –8
–6
–4
–2
0
2
4
6
–0.6 –7 8
2
3
4
5
0
0
0
1
0.2
0.2
0.2
0
0.4
0.4 0.4
–1
1.2
0.8 0.6
3
–0.4
1.2
1.2
2.5
–0.2
–0.6 –6
2
0
0
0.2
1.5
0.2
0.2
0.4
1
0.4
0.4
0.6
0.5
0.6
0.6
0.8
0
1.2
–0.2 –0.4 –0.6 –6
–5
–4
–3
–2
–1
0
1
2
–0.8 –2
1
0
1
2
3
4
Fig. 5.6 Biorthogonal B-spline wavelets for d = 2 and d˜ = 2, 4, 6, 8.
5
6
7
FA S T WAV E L E T T R A N S FO R M ( F W T )
137
1.4 1.2 1 0.8 0.6 0.4 0.2 0 –1 ~
6 4 2 0 –2 –4 –6 –2
–1.5
–1
–0.5
0
0.5
1
2
0.6
3
0.4
2
0.2
1
–2 –3
–0.8 –1
3
0.5
1
1.5
2
0.6
1.5
0.4
1
0.2
0.5
0
0
–0.2
–0.5
–0.4
–1
–0.6
–1.5
–0.8 –3 –2.5 –2 –1.5 –1 –0.5
1
0.6
0.8
0.4
0
0.5 1
1.5 2
–0.2
–0.4
–0.4
–0.6
–0.6 –6 –5 –4 –3 –2 –1
0
1
2
3
4
5
–0.8 –5
1.2
0.8
1
0.6
0.8
0.4
0.4
0
0.2
–0.2
0
–0.4
–0.2
–0.6 –6
–4
–2
0
2
4
6
–0.8 –7
0
0
0.5
1
0.5
1.5
2
1
2.5 3
0 –0.5 –1
–4
–3
–2
–1
0
1
2
–1.5
–2
–1
0
1
2
3
4
5
1.5 1 0.5
0.2
0.6
–2 –2 –1.5 –1 –0.5
0.5
1
–0.2
0
–1
0.5
0
0.2
–1.5
1.5
0.2
0.4
–2
2
0.8
–0.4 –8
0
0.8
1.2
0.6
–4
–0.5
~
4
–0.6
–1
1
2
–0.4
–0.5
0
1.5
0
0
–1
1
–1
0.5
–2
0.5
–0.2
1
–3
0
0
1.5
–1.5 –4
–0.5
0.8
0 –0.5 –1
–6
–5
–4
–3
–2
–1
0
1
–1.5 –2 2
–1
0
1
2
3
4
Fig. 5.7 Biorthogonal B-spline wavelets for d = 3 and d˜ = 1, 3, 5, 7.
5
6
7
138
6 5 4 3 2 1 0 –1 –2 –3 –3 4
WAV E L E T S 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 –2 –1.5 –1 –0.5
~
–2
–1
0
0.5
1
1.5
2
~
0
8 1.2 1 6 0.8 4 0.6 2 0.4 0.2 0 0 –2 –0.2 –4 –0.4 –0.6 –6 –2 –1.5 –1 –0.5 0 0.5 1 1.5 2 2.5 3 –3 –2.5 –2 –1.5 –1 –0.5 0 0.5 1 1.5 2 2 3 5 1 4 3 2 0.5 1 0 –1 0 –2 –3 –0.5 –4 –3 –2 –1 0 1 2 3 –3 –2 –1 0 1 2 3 4 3 4 5 –4 5 1 4 3 2 0.5 1 0 –1 0 –2 –3 –4 –0.5 –6 –5 –4 –3 –2 –1 0 1 2 3 –3 –2 –1 0 1 2 3 4 5 6 4 6
1
3 2 1 0 –1 –2 –5 –4 –3 –2 –1 0 1 2 3 2.5 2 1.5 1 0.5 0 –0.5 –1 –1.5 –2 –6 –4 –2 0 2 2
0.8
2
1.5
0.6
1.5 1
0.4
1
0.5
0.2 0.5
–0.5 –1
0
0
0
–8 –6 –4 –2
0
2
4
6
8
–0.5
–0.2
–1
–0.4
–1.5
–0.6 –8 –7 –6 –5 –4 –3 –2 –1 0
1
2
3
–2 –3 –2 –1 0
1
2
3
4
5
Fig. 5.8 Biorthogonal B-spline wavelets for d = 4 and d˜ = 2, 4, 6, 8.
6
7 8
FA S T WAV E L E T T R A N S FO R M ( F W T )
139
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 –2 –1.5 –1 –0.5 0
~
8
0.5
1
1.5
2
2.5
3
6
6
5
2
4
4
1
2
3
0
0
2
–1
–2
1
–2
–4 –1
–0.5
0
0.5
1
1.5
–3 –3
0
2
0
4
0.5
1
1.5
~
3
2
2.5
3
3.5
4
2.5
2
3
2
1.5
2
1.5
0
0
0
–1
–0.5
–0.5
–1
–3 –2.5 –2 –1.5 –1 –0.5 0
0.5
1
1.5
2
–2 –1.5 –1 –0.5 0 2
2
1.5
1.5
–2
–2.5
2.5
0
0.5
1
0.5
1
1.5
2
2.5
3
–3 –2.5 –2 –1.5 –1 –0.5 0
0.5
1
1.5
2
3
4
1.5 1
1
1
0.5
0.5
0.5 0
0
–0.5
–0.5
–1
0 –0.5
–1
–1.5
–1
–1.5
–2 –2.5 –5
–4
–3
–2
–1
0
1
2
3
1.5
4
–2 –4
–3
–2
–1
0
1
2
3
–1.5 –3
1
1
1
0.5
0.5
0
0
0
–0.5
–0.5
–0.5
–1
–1
6
4
2
0
2
4
–1.5 6 –6
–2
–1
0
1
2
1.5
1.5
0.5
–1.5
–0.5
–1.5
–2
–4
–1
–1
–1.5 –3
–1.5
0.5
0.5
–2
–2
1
1
1
–2.5
–1 –1.5 –5
–4
–3
–2
–1
0
1
2
3
–3
–2
–1
0
1
2
3
Fig. 5.9 Biorthogonal B-spline wavelets for d = 5 and d˜ = 0, 1, 3, 5.
4
5
6
140
WAV E L E T S 1.4
1.4
1.2
1.2 1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
–0.2
–0.2
0
2
4
6
8
10
12
14
16
18
–0.4
1.5
1.5
1
1
0.5
0.5
0
0
–0.5
–0.5
–1
0
2
4
6
8
10
12
14
16
18
–1
0
2
4
6
8
10
12
14
16
18
0
2
4
6
8
10
12
14
16
18
Fig. 5.10 Biorthogonal scaling functions and wavelets for d = 6 and d˜ = 8. First row: Cardinal B-spline (left), dual scaling function (right). Second row: primal wavelet (left) and dual wavelet (right).
which is called the single-scale representation and
fj =
(f, ϕ˜[0,k] )0;R ϕ[0,k] +
k∈Z
=:
j−1
j−1
(f, ψ˜[,k] )0;R ψ[,k]
=0 k∈Z
(f, ψ˜[,k] )0;R ψ[,k] ,
=−1 k∈Z
known as the multiscale representation. It will turn out to be a crucial issue to efficiently switch between these two representations. The single-scale representation is often the format in which data is given, e.g. by uniform sampling or in the case of signals and images. The multiscale representation is used to analyze, compress, and denoise data as we shall see below. For simplicity, we describe again the shift-invariant case of an MRA on L2 (R). The case of bounded intervals is quite similar.
FA S T WAV E L E T T R A N S FO R M ( F W T )
141
1.6
1.2
1.4
1 0.8
1.2 1
0.6
0.8 0.6
0.4
0.4
0.2
0.2 0
0
–0.2
–0.2 –3 1.2
–2
–1
0
1
2
3
–0.4
–4
–3
–1 –4
–3
–2
–1
0
1
2
3
4
1.5
1 1
0.8 0.6
0.5
0.4 0.2
0
0 –0.2
–0.5
–0.4 –0.6 –3
–2
–1
0
1
2
3
4
–2
–1
0
1
2
3
Fig. 5.11 Biorthogonal scaling functions and wavelets for d = 9 and d˜ = 7. First row: Primal (left) and dual scaling function (right). Second row: primal wavelet (left) and dual wavelet (right). 5.4.1
Decomposition
Assume that we are given a function fj ∈ Sj in terms of its single-scale representation fj = cj,k ϕ[j,k] ∈ Sj , k∈Z
for some coefficients cj = (cj,k )k∈Z of the form cj,k = (fj , ϕ˜[j,k] )0;R . Since Sj = Sj−1 ⊕ Wj−1 and Φj−1 and Ψj−1 are Riesz bases of Sj−1 and Wj−1 ˜ j−1 and Ψ ˜ j−1 , respectively, we have biorthogonal to Φ (fj , ϕ˜[j−1,m] )0;R ϕ[j−1,m] + (fj , ψ˜[j−1,m] )0;R ψ[j−1,m] fj = m∈Z
=
m∈Z
cj−1,m ϕ[j−1,m] +
m∈Z
m∈Z
dj−1,m ψ[j−1,m] ,
142
WAV E L E T S
where the coarse scale coefficients are given by cj−1,m = (fj , ϕ˜[j−1,m] )0;R 1 =√ a ˜l−2m (fj , ϕ˜[j,l] )0;R 2 l∈Z 1 =√ cj,l a ˜l−2m . 2 l∈Z Analogously, we obtain the detail coefficients by 1 dj−1,m = √ cj,l ˜bl−2m . 2 l∈Z The decomposition formula can be rephrased as follows. Let us define the downsampling operator ↓: (Z) → (Z) as k ∈ Z.
(↓ c)k := c2k ,
(5.17)
Then, we obtain 1 ˜ ), cj−1 = √ ↓(cj ∗ a 2 ˜ = (˜ where a ak )k∈Z is the collection of the refinement coefficients of ϕ˜ (cf. (2.32)), (c ∗ d)k :=
cm dk−m
m∈Z
is the discrete convolution of two sequences c, d ∈ (Z) and (c )k := c−k ,
c = (ck )k∈Z
defines the mirrored sequence. Consequently, the detail or wavelet coefficients are given by 1 ˜ ), dj−1 = √ ↓(cj ∗ b 2 ˜ = (˜bk )k∈Z , are the dual wavelet coefficients (5.5). A corresponding where b program can be found in Program 5.1.
FA S T WAV E L E T T R A N S FO R M ( F W T )
cj
143
- cj−1 - cj−2 - · · · @ @ @ R @ R @ R @
- c1 @ R @
- c0 @ R @
···
d1
d0
dj−1
dj−2
Fig. 5.12 Wavelet decomposition of an input vector cj into its detail parts and a coarse scale part.
˜ are also called low-pass and high-pass filters since a ˜ and b ˜ In signal analysis, a ˜ allows the low frequencies to pass and b the high frequencies. Note that especially the representation of low and high pass filters in terms of the down-sampling operator is frequently used in signal analysis. This is the reason why symmetry of the corresponding filters is often required. The corresponding mapping Tj−1,j : cj → (cj−1 , dj−1 ) is called a Wavelet Transform (WT) or decomposition. Of course, Tj−1,j represents the change of bases from Φj to Φj−1 ∪ Ψj−1 . Then, we can iterate this change of bases and obtain Tj := T0,1 ◦ T1,2 ◦ · · · ◦ Tj−1,j , which is the change of bases from the single-scale basis Φj to the multiscale basis Φ0 ∪ Ψ0 ∪ . . . ∪ Ψj−1 . This is also illustrated in Figure 5.12. 5.4.2
Reconstruction
As already mentioned above, Tj can be used to transform input data (e.g. given by sampling) into a wavelet representation which is more appropriate for processing (compressing, analyzing, transmission) the data. Once this processing is performed, one often has to transform the data back to the single-scale representation, e.g. for visualization purposes. Hence, we need to apply the InverseWavelet Transform −1 Tj−1,j : (cj−1 , dj−1 ) → cj .
In order to derive it, let us consider fj ∈ Sj which is given as fj = fj−1 + gj−1 ,
fj−1 ∈ Sj−1 , gj−1 ∈ Wj−1 .
144
WAV E L E T S
Then, using the refinement relations, we obtain fj = fj−1 + gj−1 = cj−1,m ϕ[j−1,m] + dj−1,m ψ[j−1,m] m∈Z
m∈Z
1 1 √ dj−1,m bl−2m ϕ[j,l] √ cj−1,m al−2m ϕ[j,l] + = 2 2 m∈Z l∈Z m∈Z l∈Z # $ 1 √ = (cj−1,m al−2m + dj−1,m bl−2m ) ϕ[j,l] . 2 m∈Z l∈Z *+ , ) =:cj,l
Obviously, this realizes the desired inverse −1 Tj−1,j : (cj−1 , dj−1 ) → cj
and −1 −1 −1 Tj−1 = Tj−1,j ◦ · · · ◦ T2,1 ◦ T1,0
(5.18)
is the inverse of the multiscale transform. This is often called the Inverse Wavelet-Transform (IWT) or the reconstruction and the scheme is visualized in Figure 5.13. We can also reformulate this process in terms of the involved masks using the up-sampling operator ↑: (Z) → (Z) defined by (↑ c)k :=
cm , 0,
if k = 2m is even, else.
(5.19)
In fact, it is readily seen that 1 cj = √ ((↑ cj−1 ) ∗ a + (↑ dj−1 ) ∗ b) . 2 c0
- c1
- c2
- ···
- cj
d0
d1
d2
···
dj
- cj+1
Fig. 5.13 Reconstruction or Inverse Wavelet Transform.
FA S T WAV E L E T T R A N S FO R M ( F W T )
145
As we see, the decomposition involves the dual coefficients whereas the reconstruction uses the primals. This also shows that Tj−1 = T˜jT ,
T˜j−1 = TjT ,
(5.20)
where the tilde denotes the use of the coefficients for the dual functions. 5.4.3
Efficiency
Let us now turn to the number of operations that are needed to perform the wavelet transform and its inverse. A straightforward calculation shows that the number of operations is linear in the size of the input data and also in the number of nonzero coefficients of a, see Exercise 5.1. To be precise, if the input vector is of length N , both transforms need O(N ) numerical operations, which is obviously optimal. To compare, the Fast Fourier Transform (FFT) needs O(N log(N )) operations. This shows how efficient these transforms are, which is the reason why they are called the Fast Wavelet Transform (FWT). 5.4.4
A general framework
Again, it pays off to consider a general framework as already described for MRAs in Section 2.9.1 (page 46) above. This framework was basically introduced by ˜j W. Dahmen in several papers. Let S and S˜ be biorthogonal MRAs and Wj , W be biorthogonal complements, i.e. Sj+1 = Sj ⊕ Wj ,
Wj ⊥ S˜j ,
˜ j, S˜j+1 = S˜j ⊕ W
˜ j ⊥ Sj . W
˜ j are assumed to be spanned by biorthogonal The complement spaces Wj , W wavelet systems Ψj = {ψj,k : k ∈ Jj },
˜ j = {ψ˜j,k : k ∈ Jj } Ψ
in the sense that Wj = closL2 (Ω) span(Ψj ),
˜ j = closL (Ω) span(Ψ ˜ j ). W 2
Biorthogonality can be rephrased as ˜ j )0;Ω = (Φ ˜ j , Ψj )0;Ω = 0I ×J , (Φj , Ψ j j
˜ j )0;Ω = IJ ×J . (Ψj , Ψ j j
In order to match cardinalities, we assume that |Ij+1 | = |Ij | + |Jj |,
146
WAV E L E T S
i.e. the index sets are isomorphic. Finally, the following locality assumptions similar to Assumption 2.16 (page 47) are posed. We often write the wavelet basis in the form Ψ = {ψλ : λ ∈ J }, where λ = (j, k) and J = {(j, k) : j ≥ 0, k ∈ Jj } is the set of indices. Again, we think of |λ| := j as the level or scale of ψλ and k denotes the position in space, type of wavelet, etc. Moreover, in the shift-invariant case, we again set ψ[j,k] (x) := 2j/2 ψ(2j x − k). Assuming compact support supp ψ = [1 , 2 ] ⊂ R gives supp ψ[j,k] = 2−j [1 + k, 2 + k], in particular |supp ψ[j,k] | = 2−j |supp ψ|, i.e. the size of the support decays exponentially in dependence of the level. To rephrase this in general terms, we require |λ | ∼ 2−|λ| ,
λ := supp ψλ ,
(5.21)
which reflects that functions on higher levels are highly localized. Typically the collection Ψ also contains the scaling functions on the coarsest level Φ0 , i.e. Φ0 := {ψλ : λ ∈ J , |λ| = 0},
I0 := {λ ∈ J : |λ| = 0}.
Even though we pose the existence of such a wavelet basis even on general domains as an assumption here, we are not talking about the empty set. In fact, we will later describe the construction of wavelets on fairly general domains. However, since the description of such constructions require some technicalities it pays off to postpone this and just to accept the existence here (as a black box, say).
VA N I S H I N G M O M E N T S A N D C O M P R E S S I O N
5.5
147
Vanishing moments and compression
Also as a consequence of the polynomial exactness of the multiresolution spaces S˜j , it can be seen that the integral of wavelets multiplied by polynomials of the corresponding order vanishes. These integrals using the monomials are typically called moments. Definition 5.7 For a function f : Ω → R, the term xr f (x) dx Mr (f ) := Ω
is called the rth order moment. The function is said to have d vanishing moments, if Mr (f ) = 0,
0 ≤ r < d,
r ∈ N0 .
The next statement gives an easy criterion to check the order of vanishing moments for wavelets. Proposition 5.8 If Pd−1 ⊂ S˜0loc , we have Mr (ψj,k ) = xr ψj,k (x) dx = 0
(5.22)
Ω
for all j ∈ N and all k ∈ Jj and 0 ≤ r < d. Proof Due to biorthogonality, we obtain xr ψj,k (x) dx = αr,m (ϕ˜0,m , ψj,k )0;Ω = 0, Ω
m∈I0
since S˜0 ⊂ S˜j ⊥Wj , where αr,m := ((·)r , ϕ0,m )0;Ω ˜ 0. are the expansion coefficients of the rth monomial with respect to Φ
˜ Obviously, a similar statement holds for ψ˜ replacing ψ and d replaced by d. We see by Proposition 5.8 that the approximation properties of the dual spaces S˜0 determine the order of vanishing moments of the primal wavelets and vice versa.
148
WAV E L E T S
In the shift-invariant case, we have r j/2 j −j/2 Mr (ψ[j,k] ) = x 2 ψ(2 x − k) dx = 2 2−jr (y + k)r ψ(y) dy R
R
r 1 r = 2−j(r+ 2 ) k r−m y m ψ(y) dy m R m=0 r r 1 k r−m Mm (ψ), = 2−j(r+ 2 ) m m=0 i.e. it suffices to consider the moments of ψ in this case. Vanishing moments yield the possibility of compressing a multiscale representation of a function as can be seen by the following statement. Proposition 5.9 Assume that ψ has d vanishing moments. Then |(f, ψj,k )0;Ω | 2−js f s;supp ψj,k for all f ∈ H s (supp ψj,k ) ∩ L2 (Ω), and 0 ≤ s < d. Proof By the vanishing moment property we have that |(f, ψj,k )0;Ω | =
inf
p∈Pd−1
|(f − p, ψj,k )0;Ω |
inf
p∈Pd−1
f − p 0;supp ψj,k
by the Cauchy–Schwarz inequality and ψj,k 0;Ω 1. Finally, we use the Whitney type estimate in Theorem 2.18 and obtain inf
p∈Pd−1
f − p 0;supp ψj,k 2−js f s;supp ψj,k ,
since diam(supp ψj,k ) 2−j .
0 ≤ s < d,
Again, a similar statement holds for ψ˜ and d˜ instead of ψ and d, respectively. This shows that the wavelet coefficients are small provided that the function is locally smooth. In fact, in regions where f is smooth, one can choose high values for s, so that the right-hand side is small due to the factor 2−js . Hence, wavelet coefficients may be neglected in such areas. This is the key feature for compression. Example 5.10 Let us illustrate this by an example of the multiscale expansion of the function 2 for 0 ≤ x < 12 , x , f (x) := (5.23) (1 − x)2 , for 12 ≤ x ≤ 1, which is drawn in the top line of Figure 5.14. The following pictures (row-wise) show their wavelet parts Qj f , j = 1 . . . 8, using biorthogonal B-Spline-wavelets
VA N I S H I N G M O M E N T S A N D C O M P R E S S I O N
149
Fig. 5.14 Wavelet parts Qj f of the pyramid function defined in (5.23) (also shown in the top row) for the levels j = 1, . . . , 8 using biorthogonal B-spline wavelets of order d = 3 and d˜ = 3. of order d = d˜ = 3. We see that the singularity of f at x = 12 is clearly reflected in the wavelet parts on higher levels. In regions away from this singularity, the details almost vanish. Besides the compression effect this example also shows that wavelets are well suited to detecting singularities, which is of particular importance, e.g. in image processing (edge detection) or numerical simulation (adaptivity). Let us indicate the order of vanishing moments for the examples of wavelets that have been described so far in this book. For the orthonormal wavelets N ψ we have vanishing moments of order N − 1. For biorthogonal spline wavelets, let Sj and S˜j be generated by dual scaling functions with respect to the parameters d ˜ Then, we have and d. Pd−1 ⊂ Sjloc ,
Pd−1 ⊂ S˜jloc ˜
and then it is easily seen as in the proof of Proposition 5.8 that vanishing moments and that d,d˜ψ˜ has d vanishing moments, i.e. R
xr˜ d,d˜ψ˜ dx = 0,
r˜ = 0, . . . , d − 1,
xr d,d˜ψ dx = 0,
r = 0, . . . , d˜ − 1.
R
d,d˜ψ
has d˜
150
5.6
WAV E L E T S
Norm equivalences
Let us now come to an essential property of wavelet systems which is one of the reasons why they are suitable for various applications. This has been introduced by W. Dahmen (partly with coauthors) in various papers. For orthonormal wavelets the set of functions Ψj,J := Φj ∪ Ψj ∪ · · · ∪ ΨJ−1 ,
J > j,
is an orthonormal basis for SJ , i.e. for fJ ∈ SJ , we have fJ =
cJ,k ϕJ,k =
k∈IJ
cj,k ϕj,k +
J−1
dl,k ψl,k ,
l=j k∈Jl
k∈Ij
we have (Parseval’s identity)
fJ 20;Ω =
|cJ,k |2
(5.24)
k∈IJ
=
|cj,k |2 +
k∈Ij
J−1
|dl,k |2 ,
(5.25)
l=j k∈Jl
which means that the L2 -norm of the function coincides with the vector norm of its expansion coefficients. The difference between the two representations lies in the fact that the wavelet coefficients dl,k can be seen as local detail information, see Figure 5.2 above. If we want to compress the representation of fJ in the sense that we want to use as few coefficients as possible to represent fJ (up to a given tolerance), we can do so easily by removing “small” coefficients dl,k . In fact, (5.25) gives us complete control of the error that we introduce. Setting as above Ψ−1 := Φ0 ,
ψ−1,k := ϕ0,k ,
J−1 := I0 ,
and due to the orthogonality, we then obtain
2
dj,k ψj,k
j≥−1 k∈Jj
0;Ω
=
|dj,k |2
(5.26)
j≥−1 k∈Jj
which shows that the L2 -norm of a function coincides with the 2 -norm of its wavelet coefficients
, L := {−1, 0, 1, 2, . . .}.
f 0;Ω = ((f, ψj,k ))(j,k)∈L×Jj 2 (L×Jj )
N O R M E Q U I VA L E N C E S
151
In the biorthogonal case, Ψj,J is “only” a Riesz basis for Sj which implies that (5.24) and (5.25) read
fJ 20;Ω ∼
|cJ,k |2 ∼
k∈IJ
|cj,k |2 +
J−1
k∈Ij
|dl,k |2
(5.27)
l=j k∈Jl
and consequently (5.26) changes to (5.11) for the biorthogonal case, i.e.
2
dj,k ψj,k
j≥−1 k∈Jj
∼
|dj,k |2 .
(5.28)
j≥−1 k∈Jj
0;Ω
Even though the function and sequence norms are no longer equal, they are still equivalent. This means that one can still neglect small coefficients dj,k resulting only in a small error in the function norm. We often use the following abbreviation dj,k ψj,k , d := (dj,k )(j,k)∈L×Jj dT Ψ := j≥−1 k∈Jj
so that the norm equivalence can be abbreviated as
dT Ψ 20;Ω ∼ d 2 (L×Jj ) ,
J := L × Jj .
(5.29)
Such equivalences are usually called norm equivalences. It turns out that besides biorthogonality two properties of the multiresolution analysis S are crucial for obtaining norm equivalences. 5.6.1
Jackson inequality
The first property is known as the Jackson or direct inequality and was already investigated in general terms in Proposition 2.20 (page 49). Since we now have to keep track of several constants (both for the primal and the dual system), we rewrite the statement of Proposition 2.20 in the following form inf v − vj 0;Ω 2−sj v s;Ω ,
vj ∈Sj
v ∈ H s (Ω),
0 ≤ s ≤ dS .
(5.30)
Obviously, (5.30) describes an approximation property of S and the size of dS ∈ R characterizes the highest possible order of approximation. We have already introduced a criterion to check (5.30) in Proposition 2.20, namely (besides others) polynomial exactness. We will also need a Jackson estimate for the dual system which reads inf v − vj 0;Ω 2−sj v s;Ω ,
˜j vj ∈S
v ∈ H s (Ω),
0 ≤ s ≤ dS˜.
(5.31)
152
5.6.2
WAV E L E T S
Bernstein inequality
The second property reads for the primal system
vj s;Ω 2sj vj 0;Ω ,
vj ∈ Sj ,
0 ≤ s < γS ,
(5.32)
and refers to the smoothness of the functions in Sj . Again, the parameter γS ∈ R is characteristic for S, here for the maximal smoothness as the following result shows. Lemma 5.11 Suppose that ϕ ∈ L2 (Rn ) is a compactly supported refinable function and let γ := sup{s ∈ R : ϕ ∈ H s (Rn )}.
(5.33)
Then, the Bernstein inequality holds for all 0 ≤ s ≤ γ. Proof The proof requires several preparations that go beyond the scope of this book and the reader is referred to [82]. Again, property (5.32), which is known as the Bernstein inequality, will also be needed for the dual systems, where Sj is replaced by S˜j and γS by γS˜. It is worth mentioning that Jackson and Bernstein estimates are properties only of the MRA and not of the wavelet spaces. 5.6.3
A characterization theorem
Now we are prepared to state and prove a characterization result that gives sufficient conditions for norm equivalences of the form (5.28) or (5.29) in the more general case for Sobolev spaces. Theorem 5.12 Let S := {Sj }j≥0 and S˜ := {S˜j }j≥0 be dual multiresolution ˜ respectively, such analyses with associated biorthogonal wavelets Ψ and Ψ, that the Jackson and Bernstein inequalities (5.30) and (5.32) hold for S with dS , γS and for S˜ with dS˜ and γS˜. Then, we have 22sj |dj,k |2 (5.34)
dT Ψ 2s;Ω ∼ (j,k)∈J
for all s ∈ (−˜ γ , γ), where γ˜ := min{γS˜, dS˜},
γ := min{γS , dS }.
Note that (5.34) means that the norm equivalences hold for a whole scale of Sobolev spaces H s (Ω). Since γ, γ˜ > 0, (5.34) in particular holds for s = 0, i.e.
N O R M E Q U I VA L E N C E S
153
L2 (Ω). The Sobolev index s enters into the right-hand side of (5.34) by the scaling factor 2sj . We can also write (5.34) as
dT Ψ s;Ω ∼ D s d 2 (J ) ,
(5.35)
where D is the diagonal scaling operator defined as D := diag(2j )(j,k)∈J . The remainder of this section is devoted to the proof of this theorem, where we follow a paper by Dahmen and Stevenson [94]. To this end, we first collect some technical results. We start with some properties of the biorthogonal projectors ˜ j : L2 (Ω) → W ˜ j . Defining Qj : L2 (Ω) → Wj and their adjoints Q ˜ := {Q ˜ j }j≥0 Q
Q := {Qj }j≥0 , and 2,s (Q) :=
∞
j=0
v = (vj )j∈N0 : vj ∈ Wj , v 22,s (Q) :=
22sj vj 20;Ω < ∞
,
we have the following result. ˜ are dual with respect to the Lemma 5.13 The spaces 2,s (Q) and 2,−s (Q) duality pairing v ∗ , v :=
∞
(vj∗ , v)0;Ω
j=0
with equivalent norms. ˜ ⊂ 2,s (Q)∗ . To this end, let v ∗ ∈ Proof We first show the inclusion 2,−s (Q) ˜ Then we have for all v ∈ 2,s (Q) by H¨ older’s inequality 2,−s (Q). v ∗ , v =
∞
(vj∗ , vj )0;Ω ≤
j=0
∞
2−sj vj 20;Ω 2sj vj∗ 20;Ω
j=0 ∗
≤ v 2,s (Q) v 2,−s (Q) ˜ , which is finite and thus v ∗ ∈ 2,s (Q)∗ . To show the converse inclusion, let v˜ ∈ 2,s (Q)∗ . Then, there exists a g ∈ 2,s (Q) such that one has for all v ∈ 2,s (Q) that ˜ v , v = (g, v)2,s (Q) =
∞ j=0
22js (gj , vj )0;Ω =
∞ j=0
(22js gj , vj )0;Ω ,
154
WAV E L E T S
i.e. v˜j = 22js gj , and in particular ˜ v 2,s (Q)∗ = g 2,s (Q) . Thus, we obtain
˜ v 22,−s (Q) ˜ =
∞
2−2js ˜ vj 20;Ω =
j=0
∞
22js gj 20;Ω = g 22,s (Q) = ˜ v 22,s (Q)∗
j=0
˜ and that the norms are equal. which shows that v˜ ∈ 2,−s (Q)
Using Lemma 5.13, we can now prove Theorem 5.12. Proof of Theorem 5.12. The Bernstein estimate states in particular that
wj s;Ω 2sj wj 0;Ω
(5.36)
for all wj ∈ Wj ⊂ Sj+1 and all 0 ≤ s < γS (where the additional factor 2s for wj ∈ Sj+1 is put into the constant). On the other hand, we obtain for all ˜j ∈ S˜j w ∈ H s (Ω) and all u ˜ j w 0;Ω = Q ˜ j (w − u
Q ˜j ) 0;Ω w − u ˜j 0;Ω ˜j u ˜ j are uniformly bounded. Obviously, the since Q ˜j = 0 for u ˜j ∈ S˜j and the Q ˜ latter equation holds for all u ˜j ∈ Sj , so that the Jackson estimate for S˜j yields ˜ j w 0;Ω inf w − u ˜j 0;Ω 2−sj w s;Ω
Q ˜j u ˜j ∈S
(5.37)
for 0 ≤ s ≤ dS˜. Thus, we have for all wj ∈ Wj by definition of the dual norm ˜ for j = , and biorthogonality in the sense Wj ⊥W
wj −s;Ω = =
sup s 0=w∈H ˜ 0 (Ω)
|(wj , w) ˜ 0;Ω |
w
˜ s;Ω
˜ j w) |(wj , Q ˜ 0;Ω | s
w
˜ s;Ω 0=w∈H ˜ 0 (Ω) sup
2−sj wj 0;Ω , where we have used Cauchy–Schwarz and (5.37) in the last step. Thus, (5.36) can be extended to
wj s;Ω 2sj wj 0;Ω ,
wj ∈ Wj ,
−dS˜ ≤ s < γS ,
(5.38)
i.e. a Bernstein type estimate also for negative indices (dual Sobolev spaces). In a similar way, one proves an extended dual Bernstein estimate ˜j 0;Ω ,
w ˜j s;Ω 2sj w
˜ j, w ˜j ∈ W
−dS ≤ s < γS˜.
Now, for any −dS˜ < s < γS choose > 0 sufficiently small such that s ± ∈ (−dS˜, γS ).
(5.39)
N O R M E Q U I VA L E N C E S
155
Then, (5.38) gives for all such s and by Cauchy–Schwarz
2
∞
wj
j=0
∞ ∞ = wj , wj j=0
s;Ω
j=0
∞ ∞
≤ 2
∞ ∞
(wj , w )s;Ω
j=0 =j
s;Ω
wj s+ ;Ω w s− ;Ω
j=0 =j
∞ ∞
2(s+ )j wj 0;Ω 2(s− ) w 0;Ω
j=0 =j
=
∞ ∞
2 (j−) (2sj wj 0;Ω ) (2s w 0;Ω )
j=0 =j
∞
22sj wj 20;Ω ,
(5.40)
j=0
by the H¨ older inequality and 2 (j−) ≤ 1 for ≥ j. This shows that the mapping GQ : 2,s (Q) → H s (Ω),
w := (wj )j∈N −→
∞
wj
j=0
is bounded, i.e.
GQ w s;Ω w 2,s (Q) . In other words
u 2s;Ω
2
∞
= Qj u
j=0
∞
22sj Qj u 20;Ω ,
s ∈ (−dS˜, γS ).
(5.41)
j=0
s;Ω
Again, we can prove a similar statement for the dual spaces, i.e.
u 2s;Ω
2
∞
˜
= Qj u
j=0
s;Ω
∞
˜ j u 2 , 22sj Q 0;Ω
s ∈ (−dS , γS˜).
(5.42)
j=0
The idea is now to show that (5.42) implies ∞ j=0
22sj Qj u 20;Ω u 2s;Ω ,
s ∈ (−γS˜, dS ),
(5.43)
156
WAV E L E T S
and then, that (5.41) and (5.43) imply the statement (5.34) of the theorem. To this end, for any u ∈ H s (Ω) and ˜ ∗ = 2,s (Q), ˜ = (w w ˜j )j∈N ∈ 2,−s (Q) we have that ˜ = u, GQ˜ w
∞
(u, w ˜j )0;Ω =
j=0
∞
˜ (Qj u, wj )0;Ω = (Qj u)j∈N , w,
j=0
i.e. the adjoint of GQ˜ is given by GQ˜ u = (Qj u)j∈N ,
GQ˜ : H s (Ω) → 2,s (Q).
By Lemma (5.13), this mapping is bounded, i.e.
2
∞
∞
2 2sj 2
GQ˜ u 2,s (Q) = 2 Qj u 0;Ω Qj u
j=0
j=0
= u 2s;Ω
(5.44)
−s;Ω
for s ∈ (−γS˜, dS ), i.e. (5.43). Note that both inequalities (5.40) and (5.44) simultaneously hold true for the intersection of the ranges, i.e. for s ∈ (−˜ γ , γ). Finally, by construction Ψj is a Riesz basis for Sj , i.e.
2
2 ˜
Qj v 0;Ω = (v, ψj,k )0;Ω ψj,k ∼ |(v, ψ˜j,k )0;Ω |2 ,
0;Ω
k∈Jj
k∈Jj
where the constants do not depend on j. Now, we combine this with (5.40) for v = dT Ψ, wj = Qj v, i.e.
2
∞
∞
dT Ψ s;Ω = w 22sj wj 2s;Ω j
j=0 j=0 s;Ω 22sj |dj,k |2
(5.45)
(j,k)∈J
for −dS˜ < s < γS . With (5.44), we get both inequalities for the intersection of the two ranges for s. Remark 5.14 The statement in Theorem 5.12 also holds for certain classes of Besov spaces Bqs (Lp (Ω)), but they do not hold for Sobolev spaces Wps (Ω) for p = 2 [65]. For a short definition of Besov spaces, see Appendix B. 5.7
Other kinds of wavelets
Besides orthogonal and biorthogonal wavelets as described above, there is a whole variety of different kind of wavelets with different constructions and tailored to different applications. We mention here some of them having clearly in mind that this list is by no means complete.
O T H E R K I N D S O F WAV E L E T S
5.7.1
157
Interpolatory wavelets
Interpolatory wavelets are useful in some applications since the expansion coefficients are obtained by sampling rather than by integration. This is useful, e.g. for collocation methods [27, 28, 117, 136, 138, 167, 184]. On the other hand, they lose some of the relevant properties. One prominent example is hierarchical bases introduced by Yserentant [187]. We follow here mostly the description of Donoho [104]. Definition 5.15 A compactly supported function ϕ is called (r, d)interpolatory scaling function if (a) it interpolates at the integer nodes, i.e. ϕ(k) = δ0,k , k ∈ Z; (b) it fulfills a two-scale relation ϕ(k/2) ϕ(2x − k); ϕ(x) = k
(c) Pd (R) ⊂ Sjloc , d ∈ N, where Sj = closL2 Φj , Φj = {ϕ[j,k] : k ∈ Z}; (d) ϕ is H¨older continuous, i.e. ϕ ∈ C r (R), r ∈ R+ (for the definition of H¨ older continuity, see Definition 2.28, page 56).
Then, setting ψ(x) := ϕ(2(x − 12 ))
(5.46)
gives rise to an (r, d)-interpolatory wavelet. We now give some details of the construction of interpolatory scaling functions and wavelets. Again this resorts to subdivision schemes. Subdivision schemes and interpolatory wavelets For the construction of interpolatory scaling functions, we are in particular interested in a special subclass of subdivision schemes, namely interpolatory ones, where the limit function interpolates the original control points, i.e. f (k) = λk ,
k∈Z
(5.47)
(see, e.g. [153]). Remark 5.16 One particular example are subdivision schemes that leave the previous control points unchanged while adding new intermediate control points. This is the case if a2k = δk ,
k ∈ Z.
(5.48)
Note that condition (5.48) is sufficient for (5.47), but not necessary. We now quote a result by Micchelli [153] that shows necessary and sufficient conditions for a subdivision scheme to be interpolatory. As above, we use the
158
WAV E L E T S
symbol a(z) of a mask a defined by a(z) :=
z ∈ C.
ak z k ,
(5.49)
k∈Z
Theorem 5.17 (153, Thm. 2.1) Suppose that the subdivision scheme S satisfies (5.48) and converges. Then a(−1) = 0,
a(1) = 2,
(5.50)
and the limit f is given by f (x) =
λk ϕ(x − k),
x ∈ R,
(5.51)
k∈Z
where ϕ is a continuous function of compact support which satisfies the refinement equation ak ϕ(2x − k), x ∈ R, (5.52) ϕ(x) = k∈Z
as well as
k∈Z
ϕ(x − k) = 1, x ∈ R, and ϕ(k) = δk ,
k ∈ Z.
(5.53)
Conversely, if ϕ ∈ C(R) is a solution of the refinement equation (5.52) satisfying (5.53), then (5.50) holds and the subdivision scheme converges to f in (5.51). It is easy to reinterpret (5.50) as a condition on the mask coefficients ak , namely
(−1)k ak = 0,
k∈Z
ak = 2.
(5.54)
k∈Z
We already saw the second condition in Proposition 2.7 (b). In fact, this normalization is necessary for a being a refinement mask. Thus, we are only left with one additional requirement, namely that the sums on even and odd indices coincide (and are both equal to 1). Examples of interpolatory wavelets There are two well-known families of such functions. The first one is interpolatory splines of degree d yielding an (d − 1, d)-interpolatory scaling function. The simplest example of this family is ϕ = N2 , i.e. the piecewise linear B-spline. The corresponding scaling basis (here on (0, 1)) consisting of shifted hat functions is
O T H E R K I N D S O F WAV E L E T S 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
1
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
159
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
1
Fig. 5.15 Piecewise linear interpolatory scaling system (left) and wavelet system (right). displayed in the left picture in Figure 5.15. The functions ψ defined in (5.46) induced by 2 ϕ are shown on the right in Figure 5.15. The second family consists of the Deslaurier–Debuc fundamental functions [102]. They can be described in terms of an interpolation process as follows. Using the standard Lagrange fundamental polynomials Lk ∈ P2N +1 corresponding to the integers k = −N, . . . , N + 1, defined by Lk (x) :=
N +1 0 k=i=−N
x−i , k−i
x ∈ R,
k = −N, . . . , N + 1,
the mask a is defined by for k ∈ Z,
a2k = δk , a2k+1 = L−k ( 12 ),
for k = −N − 1, . . . , N,
a2k+1 = 0,
for k > N or k < −N − 1.
The limit function of the subdivision scheme is denoted by of N θ in dependence of the parameter N is studied in [102].
(5.55)
N θ.
The regularity
Example 5.18 For N = 3 and N = 5 we display the Deslauriers–Dubuc functions in Figure 5.16. For N = 1, we obtain the piecewise linear B-spline, namely the hat function. It is known that there is a close relationship between Deslauriers–Dubuc functions N θ and Daubechies orthonormal scaling functions N ϕ. In fact, N θ is the autocorrelation of N ϕ x ∈ R. (5.56) N θ(x) = (N ϕ ∗ N ϕ)(x) := N ϕ(t) N ϕ(t − x) dt, R
160
WAV E L E T S 1.2
1.2
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
0.2
0.2 0
1
2
3
4
5
6
0 1
2
3
4
5
6
7
8
9 10
Fig. 5.16 Deslauriers-Dubuc functions for N = 3 (left) and N = 5 (right). Moreover, it was shown in [153] that a relation like (5.56) also holds for a large class of interpolatory scaling functions arising as the limit of subdivision schemes corresponding to masks satisfying certain conditions. Wavelet transform As in the case of biorthogonal wavelets, for each f ∈ C0 (R), one obtains a wavelet transform, i.e. coefficients βj0 ,k (j0 being the coarsest level) and αj,k depending only on the samples of f at level j + 1 and coarser, such that f= βj0 ,k ϕj0 ,k + αj,k ψj,k , k∈Z
j≥j0 k∈Z
where the above sum is convergent in the supremum norm [104]. However, it was also shown in [104] that interpolatory wavelets do not form a Riesz basis for L2 (Ω), which can also be seen from the fact that the sampling operator is not well defined on L2 (Ω). Characterization of function spaces However, also interpolatory wavelets allow us to characterize certain Sobolev and Besov spaces. Here Bqs (Lp (R)) denotes the standard Besov space. For a short introduction to Besov spaces we refer to Appendix B. The following result is due to Donoho. Theorem 5.19 (104, Thm. 2.7) Let ϕ ∈ C r (R) be a compactly supported, refinable, interpolatory scaling function which is exact of order d, i.e. αm,k ϕ(x − k), 0 ≤ m ≤ d, x ∈ R, xm = k∈Z
with appropriate coefficients αm,k ∈ R, where the convergence of the sum is to be understood locally. Then we have the following characterization for the
O T H E R K I N D S O F WAV E L E T S
interpolatory wavelets ψ = ϕ(2 · −1) for
d ψ j,k j,k
j,k
∼
1 p
< σ < min{r, d}, 0 < p, q ≤ ∞
# 1
1
2j(σ+ 2 − p )
j
Bqσ (Lp (R))
161
$q 1/q |dj,k |p .
(5.57)
k
Note that the above result also holds for interpolatory spline functions (which are not compactly supported, but piecewise polynomials). In this case, one obviously has r = d − 1. Hence, for p = 2, we obtain that properly scaled (r, d)-interpolatory wavelets form a Riesz basis for Bqσ (L2 (R)) for min(r, d) > σ > 1/2. Note that σ > 0, so that negative ranges are not covered. Moreover, it was shown in [104] that the bound σ > 1/p is in fact sharp. 5.7.2
Semiorthogonal wavelets
A function ψ is called a semiorthogonal wavelet (or prewavelet) if the Ψj are uniformly L2 -stable bases for the orthogonal complement Wj of Sj in Sj+1 . That is, semiorthogonal wavelets are special biorthogonal wavelets, where Sj = S˜j
˜ j. and Wj = W
Therefore, the dual scaling functions ϕ˜ and wavelets ψ˜ are also contained in Sj and Wj , respectively. The dual bases are uniquely determined, but in general not compactly supported. The semiorthogonal setting is of particular interest for the construction of spline wavelets. Using spline wavelets allows us to combine wavelet features with good properties of splines. First, note that there are no orthogonal spline wavelets (of order d ≥ 2) with compact support, while there exist semiorthogonal spline wavelets with minimal support of length 2d − 1 (see Chui and Wang [60, 63, 64]). These wavelets permit the decomposition of L2 into the mutually orthogonal spline spaces Wj . Furthermore, since the dual spaces S˜j are spline spaces, one has an explicit description of these spaces, which is not the case for the compactly supported duals of biorthogonal compactly supported spline wavelets. Finally, semiorthogonality implies that the Riesz bounds in (5.10) of the wavelet basis can be obtained as cΨ = inf cΨj j
and CΨ = sup CΨj . j
Since the Riesz bounds of the single-scale bases Ψj can usually be obtained by straightforward computations, it is much easier to verify (5.10). In particular, it turns out that compactly supported semiorthogonal spline wavelets have much
162
WAV E L E T S
better quantitative L2 -stability properties than the corresponding biorthogonal wavelets introduced above. One essential drawback of semiorthogonal wavelets is the global support of the dual wavelets and scaling functions. Since one can show that these functions and the corresponding refinement coefficients have exponential decay, a straightforward approach is to cut them off. This will introduce a certain approximation error, which might be hard to control, e.g. in an iterative scheme. However, on the interval one can implement an exact, fast wavelet transform by solving a linear system with a banded matrix, which only needs O(N ) operations [163]. A similar approach leads to a fast algorithm for periodic wavelets [16, 34]. However, these computations have to be done on the entire vector, i.e. even if only a few input coefficients are not zero, one has still to compute all output coefficients. This is an essential obstacle for adaptive methods acting only on few nonzero coefficients. Sometimes, however, there is a work-around. Since dual basis functions do not appear explicitly in the Galerkin discretization of a partial differential equation, semiorthogonal wavelets can be used here without loss of efficiency in many cases, as for preconditioning (see also the next chapter) and adaptive wavelet methods (see, e.g. [36]). For particular constructions of semiorthogonal wavelets we refer to [61, 62, 135, 139, 152]. We show linear, quadratic and cubic semiorthogonal spline wavelets in Figure 5.17. As already mentioned above, there are at least two properties of semiorthogonal spline wavelets that make them attractive for numerical simulations, namely their quantitative localization and stability properties. Table 5.1 shows the sizes of the support of semiorthogonal spline wavelets by Chui and Wang [60, 63, 64] as compared to biorthogonal spline wavelets from [71]. Except for the case d = d˜ (which often gives rise to stability problems, see [70]) semiorthogonal spline wavelets have smaller support to biorthogonal wavelets of the same order. Finally, let us show some values for the condition numbers of semiorthogonal and biorthogonal spline wavelets. In Table 5.2, condition numbers for semiorthogonal spline wavelets and scaling functions by Chui and Wang [60, 63, 64] for the real line R and of Chui and Quak [62] for the interval are listed. We compare 0.8
0.3
0.6
0.1
0.2
0.4
0.1
0.2
1 0.5
1
1.5
2
2.5
3
–0.1
–0.2
–0.2
–0.4
–0.3
2
3
4
1
5
2
3
4
5
6
7
–0.1 –0.2
Fig. 5.17 Semiorthogonal wavelets, linear, quadratic and cubic (from left to right).
O T H E R K I N D S O F WAV E L E T S
163
Table 5.1 Sizes of the supports of semiorthogonal and biorthogonal spline wavelets. Type/d
2
3
4
5
6
7
8
d
8 10 12 14 16 2d − 2 7 9 11 13 15 2d − 1 9 11 13 15 17 2d + 1 11 13 15 17 19 2d + 3
Semiorthogonal 4 6 Biorthognal d˜ = d 3 5 Biorthognal d˜ = d + 2 5 7 Biorthognal d˜ = d + 4 7 9
Table 5.2 Condition numbers for scaling functions and wavelets of [0,1] R Chui and Wang (ρR Φ and ρΨ ) for L2 (R), and of Chui and Quak (ρΦj [0,1]
and ρΨj ) for L2 ([0, 1]) and j ≤ 11 in dependence of the spline order d. For comparison, we also display corresponding numbers for biorthogonal spline wavelets on R from [71] and on [0, 1] from [35]. ρΩ Ψj biorthogonal
semiorthogonal d 2 3 4 5 6 7 8
ρR Φ
ρR Ψ
[0,1]
ρΦ j
3.0 2.3 3.0 7.5 3.5 7.6 18.5 5.9 19.3 45.7 10.4 49.8 112.8 18.7 130.4 278.4 33.9 345.0 686.9 61.6 920.7
[0,1]
ρΨj
˜ R d = d,
d˜ = d + 10, R
[0, 1]
10 80 ∞ ∞ ∞ ∞ ∞
4.2 16.0 64.0 256.2 ≥ 1024 ≥ 4096 ≥ 16384
4.1 16.0 64.1 264 ≥ 1024 ≥ 4096 ≥ 16384
2.3 3.5 6.0 11.7 21.2 50.3 158.4
these with the corresponding values for biorthogonal B-spline wavelets for some values of d and d˜ both on R and also on the interval for the particular construction in [35]. Note that in the latter case, the condition numbers only depend ˜ For the interval, we determined these numbers up to level 11, but weakly on d. the behavior of the values for increasing j gives strong evidence that these values [0,1] [0,1] are already close to the true condition numbers ρΦ and ρΨ . Furthermore, we want to mention that we have L2 -normalized these wavelets and scaling functions (in contrast to [62]). 5.7.3
Noncompactly supported wavelets
As already pointed out earlier, we only consider compactly supported wavelets throughout this book even though also noncompactly supported but rapidly decaying functions admit some relevant advantages. One of them is that certain wavelet-type functions allow to diagonalize certain linear operators L in the
164
WAV E L E T S
sense that (LΨ, Ψ)0;Ω is a diagonal operator. Such functions ψ are often also called vaguelettes. Obviously, this allows an efficient application of L to any finite input. However, one still has to face the problem that ψ itself can only be determined by an infinite number of parameters. This means that ψ or its filter has to be truncated at a certain point in order to make it available on a computer. The rigorous study of the committed truncation error is a delicate task which is the reason why we do not consider such functions here. For details, we refer to [105, 109]. 5.7.4
Multiwavelets
Multiwavelets are not generated by only one scaling function but by finitely many functions ϕ1 , . . . , ϕn : R → R which are collected in one so-called scaling vector ϕ = (ϕ1 , . . . , ϕn )T : R → Rn , and that satisfies a refinement equation in the sense ϕ(x) = Ak ϕ(2x − k), x ∈ R a.e.,
(5.58)
k∈Z
where the mask Ak ∈ Rn×n are matrices which here play the role of refinement coefficients. A scaling vector is called a multigenerator if the integer translates from an L2 -stable basis and ϕ satisfies the refinement equation (5.58). Giving up the simple form of the refinement equations allows us to construct multiwavelets with smaller support and higher regularity. We give two examples which are in this form taken from [36], namely cubic Hermite splines defined as follows 1 − 3x2 − 2x3 = (1 + x)2 (1 − 2x), if x ∈ [−1, 0]; 1 − 3x2 + 2x3 = (1 − x)2 (1 + 2x), if x ∈ [0, 1]; ϕ+ (5.59) 1 (x) := 0 otherwise, x + 2x2 + x3 = x (1 + x)2 , if x ∈ [−1, 0]; x − 2x2 + x3 = x (1 − x)2 , if x ∈ [0, 1]; ϕ+ (5.60) 2 (x) := 0 otherwise, and quadratic Hermite splines ϕ− 1 (x)
:=
−6x − 6x2 ,
if x ∈ [−1, 0];
0
otherwise,
(5.61)
O T H E R K I N D S O F WAV E L E T S 1.2
165
1.6 1.4
1
1.2
0.8
1
0.6
0.8 0.6
0.4
0.4
0.2
0.2 0
0
–0.2
–0.2 –1 –0.8 –0.6 –0.4 –0.2 0
0.2 0.4 0.6 0.8
1
–0.4
–1 –0.8 –0.6 –0.4 –0.2 0
0.2 0.4 0.6 0.8
1
Fig. 5.18 Cubic (left) and quadratic Hermite splines. 1 + 4x + 3x2 , 1 − 4x + 3x2 , ϕ− 2 (x) := 0
if x ∈ [−1, 0]; if x ∈ [0, 1];
(5.62)
otherwise.
The corresponding functions are displayed in Figure 5.18. These functions have been investigated in several papers. Biorthogonal mul+ tiwavelets on the interval [0, 1] generated by ϕ+ 1 , ϕ2 have been introduced + in [83]. It is well known that the integer translates of ϕ+ 1 , ϕ2 generate the 1 space of C -continuous piecewise cubic functions on R which interpolate function values and first derivatives at the integers. Based on these functions, biorthogonal divergence-free multiwavelets (also on rectangular domains) have been constructed in [143, 144], and interpolatory divergence-free wavelets in [37]. We collect the two above-mentioned generating functions in one vector + − ϕ1 ϕ1 + − , ϕ := . ϕ := ϕ+ ϕ− 2 2 and look for the mask A := {Ak }k∈Z , Ak ∈ R2×2 in (5.58) which are given by A+ −1
1 2 = 1 − 8
3 4 , 1 − 8
1
A+ 0 =
0
1
0 1 , 2
2 A+ 1 = 1 8
3 4 , 1 − 8 −
(see also [83, 130]) and that ϕ− is refinable with mask matrices
0
A− −2 = 1 − 4
0 0 − = , A 1 −1 0 4
1 0 2 − 1 , A0 = 1 4 4
1 2 − , A1 = 1 1 − 4 0
3 2 . 1 − 4
166
WAV E L E T S
It is known (see, e.g. [83]) that these multigenerators have compactly sup˜+ is given by the mask ported dual multigenerators. For ϕ+ , a dual ϕ 7 1 29 5 3 − − 0 64 64 2 16 32 ˜+ = ˜+ = ˜+ = A A A , , , −2 −1 0 87 31 37 99 15 − − 0 128 64 32 32 8 1 7 3 5 − − 2 16 64 64 ˜+ = ˜+ = A A , . 1 2 99 37 87 31 − − 32 32 128 64 Corresponding biorthogonal multiwavelets can then be determined, see [112]. Note that the masks cannot be computed by the scaling masks in such an easy way as in the case of only one generating function. 5.7.5
Frames
Of course, to require that Ψ is an orthonormal or at least a Riesz basis is a strong analytical property which in particular allows for rigorous analysis. On the other hand, this sometimes might be a too severe restriction that may cause difficulties in constructing Ψ. Moreover, sometimes redundancy is even useful, e.g. in signal transmission on a channel with noise. Considering frames one requires that Ψ spans L2 (Ω) but one drops linear independence. Still, some stability is required, namely AΨ f 20;Ω ≤ |(f, ψλ )0;Ω |2 ≤ BΨ f 20;Ω λ∈J
for all f ∈ L2 (Ω). The constants 0 < AΨ ≤ BΨ < ∞ are called frame bounds. If AΨ = BΨ , the family Ψ is called a tight frame. The theory of frames and in particular of wavelet frames is well advanced, see e.g. [15, 59, 98, 99, 129, 178]. More recently, starting from R. Stevenson [171], wavelet frames have also been used for the numerical solution of partial differential equations by S. Dahlke and coauthors; see, e.g. [79, 80]. The main motivation is to obtain an alternative to the construction of wavelet bases on arbitrary domains Ω which we will describe later. To construct a frame is simpler. The price to pay, however, is that the adaptive method becomes more sophisticated. We will not detail this here. 5.7.6
Curvelets
Curvelets were introduced in 1999 by Candes and Donoho as an effective nonadaptive representation for images (or objects) with edges [47]. The usual
E X E RC I S E S A N D P RO G R A M S
167
approach using a tensor product wavelet basis (or filters) in image processing results in the problem that representing edges that are neither aligned with one of the coordinate axes nor with the diagonal causes a wavelet representation with a huge number of nonneglectable coefficients. Thus, a compression cannot be performed by this approach. The alternative approach is to introduce an angle of rotation as the new parameter for the functions and to use a level difference in the coordinate directions in order to mimic the known h2 -approximation property of curves in 2D. Thus, a curvelet in 2D is defined as (j,) ψj,,k (x) := ϕj Rθ x − xk , where ϕj is some “mother” curvelet, j
• θ := 2π·2− 2 , = 0, 1, 2, . . . , are equispaced rotation angles, 0 ≤ θ < 2π, the orientation; • k = (k1 , k2 ) ∈ Z2 is the shift parameter ; (j,) • xk := Rθ−1 (k1 · 2−j , k2 · 2−j/2 ) the position, and Rθ is the rotation by the angle θ defined as cos θ sin θ Rθ := , Rθ−1 = R−θ . − sin θ cos θ Obviously, one cannot expect to obtain a basis since the rotation introduces redundancy. However, one can construct tight frames formed by curvelets [48]. Nowadays, a long list of articles is available as well as corresponding software. As examples, we mention [46, 106, 169] and the web site [131] being aware that this list is far from being complete. There are also recent extension, e.g. so-called shearlets. 5.8
Exercises and programs
Exercise 5.1 Given a finite input vector c = (ck )k∈Z , i.e. ck = 0 ⇐⇒ k ∈ [A, B], determine the number of floating point operations (flops) of the wavelet and the inverse wavelet transform in dependence of the filter length and the length of the support of the input vector. Exercise 5.2 Given a dual pair of scaling functions ϕ, ϕ˜ with refinement masks ˜ bk = (−1)k a ˜ , show that the wavelets ψ, ψ˜ defined by the masks b, b, ˜1−k , a, a ˜bk = (−1)k a1−k are biorthogonal: (ψ[j,k] , ψ˜[j ,k ] )0;R = δj,j δk,k . Exercise 5.3 Show the following relations: (a) ψ[j,k] (x) dx = 0, k ∈ Z; R
(b) ψ[j,k] 0;R = 1, k ∈ Z.
168
WAV E L E T S
Exercise 5.4 Give best possible estimates for the constants dS , γS and dS˜, γS˜ for biorthogonal B-spline systems of order d and d˜ on R from Cohen, Daubechies and Feauveau [71], Section 5.3.3. Exercise 5.5 Prove Proposition 5.4. Exercise 5.6 Prove Proposition 5.5. Exercise 5.7 Compare the refinement coefficients for the case d = d˜ = 4 with those given in the standard software. −1 Exercise 5.8 Show that the reconstruction Tj−1,j is the inverse operation of the decomposition Tj−1,j .
Exercise 5.9 For α > 0 consider the function |x (1 − x)|α , for 0 ≤ x ≤ 1, fα (x) := 0 else. For which values of m ∈ N0 do we have fα ∈ H m (R)? Computer Exercise 5.1 Write a code to produce an approximation of the Daubechies wavelets for given N . Computer Exercise 5.2 Write a code for determining the wavelet coefficients 0 of a given function f ∈ C2π (R) and a given maximal level j. Write an interface to visualize the size of the wavelet coefficients. Computer Exercise 5.3 Write the same program as in Computer Exercise 5.2 for interpolatory spline wavelets. Computer Exercise 5.4 Use the code in plot_daub.cc to produce plots of the corresponding Daubechies orthonormal wavelets. Computer Exercise 5.5 Write a code to realize the fast wavelet transform (decomposition). Use this code for the following exercise. For α > 0 consider the function |x (1 − x)|α , for 0 ≤ x ≤ 1, fα (x) := 0 else. Use the following quasiinterpolation operator f10 (x) =
1 ∈Z
6
− f 2−10 ( + 1) + 8f 2−10 − f 2−10 ( − 1) N4 (210 x + 2 − )
as an approximation. Compute the coefficients ck and dj,k in f10 =
k∈Z
ck ϕ[0,k] +
9 j=0 k∈Z
dj,k ψ[j,k] ,
E X E RC I S E S A N D P RO G R A M S
169
for α = 12 , α = 1 and α = 2. Use biorthogonal spline wavelets with d = 4 and d˜ = 8. Consider the values Dj = supk∈Z |dj,k |. What do you observe? Computer Exercise 5.6 Write a code for the inverse wavelet transform (reconstruction). (1) Determine a finite approximation by using only the N largest coefficients (in absolute values) for N = 200, 100, 50, 25 for the function fα , (α = 1 2 , 1, 2) as in Computer Exercise 5.5. Plot the resulting approximation. (2) Determine the error with respect to the L2 -norm (use the term f 22,R ≈ A2 A 2−10 Af (2−10 )A as an approximation). (3) Give a graphical visualization of the wavelet coefficients dj,k for the above function, j = −1, . . . , 9 and biorthogonal spline wavelets corresponding to d = 4 and d˜ = 8. Programs Besides the following programs, we provide software for: • Wavelet coefficients of — Daubechies wavelets, — primal and dual B-spline wavelets. • Plotting of wavelets by using — the eigenvector/eigenvalue problem and subdivision, — cascade algorithm. Program 5.1 The code fwt.cc realizes the Fast Wavelet Transform (FWT). Program 5.2 The code inv-fwt.cc realizes the Inverse Fast Wavelet Transform (IFWT).
6 WAVELET-GALERKIN METHODS In this section, we describe the use of wavelets for the numerical solution of boundary value problems. Since we will use wavelets as trial and test functions in a Galerkin approach, the resulting scheme is called the Wavelet-Galerkin Method (WGM). We will describe two main features of the WGM, namely preconditioning and the potential of wavelet discretizations for adaptivity. In particular, we show that wavelets give rise to a diagonal preconditioner that is optimal with respect to the level in the sense that the condition number of the preconditioned stiffness matrix is independent of the level j. We will see that the Fast Wavelet Transform (FWT) plays an important role for the efficiency of the method. The second issue is the observation that the wavelet expansion of the solution of a boundary value problem is sparse in the sense that several coefficients are small and can be neglected in order to obtain a compression. This is one key ingredient for adaptive wavelet methods that will be described later. 6.1
Wavelet preconditioning
So far, we have seen Multiresolution Galerkin Methods for the numerical solution of boundary value problems. Without preconditioning, however, the number of CG iterations grow exponentially with the level, see Lemma 4.1 (page 93). The reason is the fact that the stiffness matrix Aj with respect to the single-scale matrix is ill conditioned, i.e. cond(Aj ) ∼ 22j . The problem of optimal preconditioning was open for quite a while. It was solved independently by Oswald [162], Dahmen and Kunoth [84], and Jaffard [134]. Here we follow [84] with a different notation. The idea to overcome this difficulty is to use the wavelet representation for preconditioning. In fact, as we know, the collection of functions Ψj := Φ0 ∪ Ψ0 ∪ · · · ∪ Ψj−1 is also a basis for Sj = S0 ⊕ W0 ⊕ · · · ⊕ Wj−1 . Hence, we do not change test and trial spaces in the Galerkin method but we only use a different basis. This means that the error estimate in Lemma 2.17 (page 48) still holds true. 170
WAV E L E T P R E C O N D I T I O N I N G
171
Let us introduce some notation. We consider again the periodic boundary value problem (3.1). However, all what is said also holds for the Dirichlet problem (3.2) if a corresponding wavelet basis is available. This is also the reason why we use scaling functions on the coarsest level j0 = 0. In the periodic case, this is just the constant function. The construction of an appropriate basis for the Dirichlet problem will be described later. The stiffness matrix in the wavelet representation will be denoted as AΨj := a(θ, ϑ) θ,ϑ∈Ψj . This can be expressed in an alternative way as follows. The wavelet representation of the differential operator reads as follows A := a(ψλ , ψµ ) λ,µ∈J , where J :=
Jj
j≥0
denotes the collection of all wavelet index sets. Hence, A can be interpreted as a (bi-)infinite matrix. Then J j := I0 ∪ J0 ∪ · · · ∪ Jj−1 denote the wavelet indices up to level j − 1, i.e. Ψj = {ψλ : λ ∈ J j }. The full wavelet basis then reads Ψ = {Ψj,k : j ≥ 0, k ∈ Jj } = {ψλ : λ ∈ J }. With this notation at hand, AΨj is the section of the full (infinite) matrix A, i.e. AΨj = A|J j ×J j . Now we are ready to formulate and prove the following key observation for multilevel wavelet preconditioning. Theorem 6.1 Let Ψ be a wavelet basis in L2 (Ω) such that the following norm equivalence holds
dT Ψ 1;Ω
= dj,k Ψj,k
j≥0 k∈Jj
1;Ω
∼
1/2 22j |dj,k |2
= Dd 2 (J ) ,
j≥0 k∈Jj
(6.1)
172
WAV E L E T - G A L E R K I N M E T H O D S
where D is a diagonal operator, D := (δk,k 2j )(j,k),(j ,k )∈J ,
Dj := D|J j .
Then, we have cond2 (Dj−1 AΨj Dj−1 ) = O(1),
j → ∞,
(6.2)
i.e. Dj2 is an asymptotically optimal preconditioner for AΨj . ˜ by applying a Proof First of all, we develop a dual statement of (6.1) for Ψ simple duality argument. By definition of the dual norm, we have ˜ −1;Ω =
dT Ψ
˜ 0;Ω (u, dT Ψ)
u 1;Ω u∈H 1 (Ω) sup
=
˜ 0;Ω (cT Ψ, dT Ψ)
cT Ψ 1;Ω c∈2 (J )
=
cT d , T c∈2 (J ) c Ψ 1;Ω
sup sup
˜ 0;Ω = I. Then, (6.1) implies where we have used the fact that (Ψ, Ψ) ˜ −1;Ω ∼
dT Ψ
cT d c∈2 (J ) Dc 2 (J )
=
cT (D −1 d) c∈2 (J ) c 2 (J )
sup sup
(6.3)
= D −1 d 2 (J ) , since D is diagonal (hence symmetric) and invertible. Define the differential operator A : H01 (Ω) → H −1 (Ω) as usual by Au, v = a(u, v)
for u, v ∈ H01 (Ω),
where ·, · denotes the duality pairing of H01 (Ω) and H −1 (Ω). Now choose any d ∈ 2 (J ) and abbreviate u := dT Ψ. Then, we have by (6.1) and the ellipticity of a(·, ·)
Dd 2 (J ) ∼ u 1;Ω ∼ a(u, u)1/2 = Au −1;Ω ,
WAV E L E T P R E C O N D I T I O N I N G
173
˜ is a where the last step follows by definition of A. Since by (6.3) the system D Ψ Riesz basis in H −1 (Ω), we have in particular that each element in H −1 (Ω) has ˜ i.e. an expansion in Ψ, ˜ −1;Ω ∼ D −1 (Au, Ψ)0;Ω (J ) ,
Au −1;Ω = (Au, Ψ)0;Ω Ψ
2 where we have used (6.3) in the last step. Finally, straightforward calculations using u = dT Ψ yield
D −1 (Au, Ψ)0;Ω 2 (J ) = D −1 (AΨ, Ψ)0;Ω d 2 (J ) = D −1 AΨ D −1 Dd 2 (J ) , so that we have proven
Dd 2 (J ) ∼ D −1 AΨ D −1 Dd 2 (J ) . for all d ∈ 2 (J ), i.e. cond2 (D −1 AΨ D −1 ) < ∞,
which in particular implies (6.2).
Remark 6.2 In the last step of the above proof, we have used cond2 (B) for an infinite matrix (operator) B : 2 (J ) → 2 (J ). This is defined in the usual way, namely cond2 (B) := B 2 (J ) B −1 2 (J ) and · 2 (J ) is the operator norm induced by the sequence norm on 2 (J ). The above proof obviously holds for all elliptic operators L : H → H in Hilbert spaces H → L2 (Ω) → H and wavelet bases Ψ on L2 (Ω) satisfying
dT Ψ H ∼ Dd 2 (J ) for some invertible symmetric operator D. If D is not symmetric, then (6.3) has to be replaced by ˜ H ∼ D −T d (J )
dT Ψ
2 so that the preconditioned wavelet representation of A has to be replaced by D −T AΨ D −1 .
174
WAV E L E T - G A L E R K I N M E T H O D S
This in particular includes elliptic operators of any order, not only for secondorder operators. Hence, also corresponding operators of negative order (such as integral operators) are covered. Besides the fact that we have proven that AΨj can be preconditioned by a diagonal scaling in an asymptotically optimal way, the above theorem has an important consequence that we note for further reference. Theorem 6.3 The function u ∈ H01 (Ω) solves Au = f for given f ∈ H −1 (Ω) if and only if u ∈ 2 (J ) solves Au = f ,
(6.4)
where A := D −1 AΨ D −1 and u = dT Ψ,
u = Dd,
f = D −1 (f, Ψ)0;Ω .
Moreover, the problem (6.4) is well conditioned, i.e. cond2 (A) < ∞. Proof Equation (6.4) means Au = D −1 (AΨ, Ψ)0;Ω D −1 Dd = D −1 (AΨ, Ψ)0;Ω d = D −1 (Au, Ψ)0;Ω = f = D −1 (f, Ψ)0;Ω and hence (Au, Ψ)0;Ω = (f, Ψ)0;Ω since D is invertible. This however is equivalent to Au = f . This latter result means that the differential problem Au = f posed in a Sobolev space can be stated equivalently as a discrete problem in the sequence space 2 (J ). Moreover, the equivalent problem is well-conditioned. We will come back to this point later. Analyzing the statement and the proof of Theorem 6.1 shows that there are two main ingredients for optimal preconditioning, namely (1) the ellipticity of the operator, or more general, that the underlying operator A : H → H
T H E RO L E O F T H E F W T
175
is boundedly invertible, i.e.
Au H ∼ u H ,
u ∈ H;
(2) the norm equivalence (6.1) for the wavelet system Ψ. The first item is a property of the operator under consideration and defines a class of operators that can be treated. As already said, this class not only involves elliptic partial differential equations but also certain singular integral operators. The second issue is a property of the wavelet system Ψ and we are left with the construction of Ψ in such a way that (6.1) holds. Theorem 5.12 (page 152) gives criteria to ensure corresponding norm equivalences. For the following sections, we always assume that appropriate wavelet bases are available. 6.2
The role of the FWT
Now we have seen that using Ψj as a basis for Sj yields an asymptotically optimal preconditioned system. However, the following statement shows that we cannot use Ψj immediately. Lemma 6.4 The number of nonzero entries of AΨj is O(Nj log(Nj )), where Nj = dim(Sj ).
Proof Exercise 6.1
In fact, Lemma 6.4 shows that AΨj is not sparse but – due to the logarithmic factor – only almost sparse. Figure 6.1 shows the sparsity pattern of AΨj for the periodic model problem. The so-called finger structure is clearly visible. 0
0
20
20
40
40
60
60
80
80
100
100
120
120 0
20
40
60 80 nz = 2622
100
120
0
20
40
60 80 nz = 5038
100
120
Fig. 6.1 Sparsity pattern of the stiffness matrix with respect to biorthogonal B-spline wavelets for d = d˜ = 2 (left) and d = d˜ = 3 (right) both for j = 7.
176
WAV E L E T - G A L E R K I N M E T H O D S
Hence, we are facing the following situation: • the stiffness matrix Aj with respect to the single-scale basis Φj is sparse but ill conditioned; • the wavelet stiffness matrix AΨj is asymptotically optimal preconditioned, but not sparse. Thus, both methods cannot be used immediately. However, a clever combination of the two offers a way out. The appropriate tool is the Fast Wavelet Transform (FWT), Tj : cj −→ dj0 ,
Sj uj = cTj Φj = (dj0 )T Ψj ,
described in Section 5.4 above. Using this, we get cTj Φj = (dj0 )T Ψj = (Tj cj )T Ψj = cTj TjT Ψj , i.e. Φj = TjT Ψj , and we obtain Aj = (AΦj , Φj )0;Ω = (ATjT Ψj , TjT Ψj )0;Ω = TjT a(Ψj , Ψj )Tj = TjT AΨj Tj , or, equivalently T˜j Aj T˜jT = AΨj , where we have used Tj−1 = T˜jT , see (5.20). Thus AΨj dj = fΨj = (f, Ψj )0;Ω can be restated as T˜j Aj T˜jT dj = T˜j fj ,
fj = (f, Φj )0;Ω ,
and in preconditioned form Dj−1 T˜j Aj T˜jT Dj−1 dj = Dj−2 T˜j fj .
(6.5)
The above equation in fact combines both positive effects. We have seen in Theorem 6.3 that Dj−1 T˜j Aj T˜jT Dj−1 = Dj−1 AΨj Dj−1 is asymptotically optimally conditioned. Moreover, note that in a CG-iteration, only the application of the stiffness matrix to a given vector is required. It is not necessary to store the stiffness matrix. In fact, one has to avoid storing (and
N U M E R I C A L E X A M P L E S FO R T H E M O D E L P RO B L E M
177
computing) AΨj since this is already an O(Nj log(Nj )) process, destroying our efficiency demands. However, in the form (6.5) the preconditioned stiffness matrix can be applied in O(Nj ) operations. Indeed, the preconditioner is diagonal, hence O(Nj ) and the FWT was already shown to be O(Nj ). Note that here the dual filters are involved. So far, this was not the case. In Theorem 6.1 where the optimal preconditioning was proven, only the norm equivalence for the primals was used. In particular, this result also holds if the dual functions were globally supported (as, e.g. for certain prewavelets). Moreover, in (6.5), the decomposition with respect to the dual system is needed. Recalling that the decomposition with respect to Ψ needs ˜ it is clear that for the preconditioning only primal filters are the mask of Ψ, needed. This in particular means that semiorthogonal wavelets (prewavelets) can be used here. Finally, Lemma 4.1 shows that Aj can be computed, stored, and applied to a vector in O(Nj ) operations, which concludes that (6.5) can in fact be applied numerically in O(Nj ) operations. 6.3
Numerical examples for the model problem
The above results on preconditioning show that the method is asymptotically optimal. Moreover, these results are qualitative which means that they give an O-estimate without providing information on the particular size of the involved constant. This constant, however, is of crucial importance for the efficiency of the numerical method. The above analysis shows that several kind of constants are involved, namely • constants coming from the problem: continuity and coercivity constant; • constants coming from the wavelet basis: condition number, Riesz constants. We can assume that we get at least estimates for these constants only in academic cases. In general, we will not have information on the size of these constants. Hence, numerical tests and experiments are required to asses the quantitative behavior of the method. In this section, we report on several numerical tests. 6.3.1
Rate of convergence
We start by determining the condition number of the wavelet-preconditioned matrix Dj−1 Aj Dj−1 for different values of c and j; see Table 6.1 for the case d = d˜ = 2. Comparing with the values in Table 4.5 (page 108), we clearly see that the condition numbers are uniformly bounded with respect to the level j. Again, we also investigate the dependence of the condition number on d and ˜ Hence, we fix j = 10 and c = 0.1 and show the condition numbers in Table 6.2. d.
178
WAV E L E T - G A L E R K I N M E T H O D S
Table 6.1 Condition numbers of wavelet-preconditioned stiffness matrix depending on d = d˜= 2 for different values of c and j and factor between condition numbers of level j and j − 1. j\c
0.1
1.0
10
100
1000
j\ c
0.1
1.0
10
100
1000
2 3 4 5 6 7 8 9 10
40.1 51.4 55.8 61.2 65.0 68.2 70.9 73.1 75.0
4.13 5.25 5.68 6.14 6.57 6.88 7.15 7.38 7.55
2.46 2.78 3.13 3.40 3.72 3.87 3.97 4.04 4.14
16.4 22.1 24.2 28.9 32.7 35.3 37.2 38.7 39.8
40 109 189 231 256 306 337 360 378
3 4 5 6 7 8 9 10
1.28 1.09 1.10 1.06 1.05 1.04 1.03 1.03
1.26 1.08 1.09 1.06 1.05 1.04 1.03 1.03
1.15 1.13 1.12 1.06 1.04 1.03 1.02 1.02
1.35 1.10 1.20 1.13 1.08 1.06 1.04 1.03
2.70 1.74 1.23 1.11 1.20 1.10 1.07 1.05
Table 6.2 Condition numbers of wavelet-preconditioned (diagonal scaling) ˜ stiffness matrix depending on d and d. d d˜ − d 0 2 4 6 8 10
2
3
4
5
75.0 95.5 109.3 119.1 126.1 126.1
53.9 53.5 53.7 54.0 54.2 54.5
86.8 85.8 85.6 85.6 85.6 85.6
140.4 138.9 138.6 138.5 138.5 138.5
6
7
8
2176.8 5.1e + 06 3.6e + 08 324.2 4942.2 1.38e + 06 264.4 1150.8 14289.5 264.4 1041.7 4369.5 257.9 1029.2 4138.4 256.9 1027.1 4112.0
9
10
3.6e + 11 1.1e + 09 4.1e + 06 48308.0 17052.3 16497.4
4.1e + 14 1.2e + 12 3.9e + 09 1.6e + 07 183933 67349
The columns correspond to a fixed d and fixed difference d˜ − d, i.e. in the first column we display the numbers for d = 2 and d˜ = 2, 4, 6, 8, 10, 12. We observe an increasing condition number for increasing d and fixed d˜ − d as well as for decreasing d˜ − d and fixed d > 3. This was somehow expected since increasing these parameters means that we increase the order of reproduced polynomials which has to be expected to cause growing condition numbers. The extreme values for some examples of high d are also clear. In fact, it is known that the choice d = d˜ often gives rise to unstable systems. In order to obtain stability the required minimal difference d˜ − d grows with growing d. Of course, we are also interested in a comparison of the wavelet preconditioner to other solution methods such as BPX and MultiGrid that only need the singlescale basis functions. Since the wavelet method acts as a preconditioner, we only show the comparison with the BPX preconditioner. However, the comparison with MultiGrid is straightforward having in mind the results shown in Chapter 4.
N U M E R I C A L E X A M P L E S FO R T H E M O D E L P RO B L E M
179
In order to obtain a fair comparison, we use the same Multiresolution Galerkin discretization for all three methods and compare the number of iterations needed to reach a given tolerance ε starting from a random initial value u(0) using again the right-hand side f ≡ 0, i.e. u ≡ 0, and a random initial vector. In Figure 6.4 below, we show CPU times over the error in the following sense. The horizontal axis indicates the exponent of the error, i.e. a value of k = 10 means that the error is 10−k . Hence, we expect growing curves since a more accurate solution should require more numerical work. Now we consider the wavelet preconditioning. In Figure 6.2, we show the convergence history of the wavelet preconditioned CG method using the diagonal 1e+15
.1 1. 10. 100. 1000.
1e+10 100000 1 1e – 05 1e – 10 1e – 15 0
5
10
15
20
25
30
1e + 15
35
40
.1 1. 10. 100. 1000.
1e + 10 100000 1 1e – 05 1e – 10 1e – 15 0
5
10 15 20 25 30 35 40 45 50
1e + 15
.1 1. 10. 100. 1000.
1e + 10 100000 1 1e – 05 1e – 10 1e – 15 0
10
20
30
40
50
60
70
80
90
Fig. 6.2 Wavelet diagonal scaling preconditioned CG method for d = 2 (top), d = 3 (middle) and d = 4 (bottom).
180
WAV E L E T - G A L E R K I N M E T H O D S
scaling by 2−j . We will refer to this as diagonal scaling. Next, we compare the wavelet-preconditioned CG-method using diagonal scaling with the BPX preconditioned CG method. To this end, we fix c = 0.1 and choose d = 2, d = 3 and d = 4. The corresponding convergence histories are displayed in Figure 6.3. In all cases, the wavelet based method performs quantitatively better. 1e + 15 Wavelet .1 BPX .1
1e + 10 100000 1 1e – 05 1e – 10 1e – 15 0
10
20
30
40
50
60
1e + 15 Wavelet .1 BPX .1
1e + 10 100000 1 1e – 05 1e – 10 1e – 15 0
10
20
30
40
50
60
1e + 15 Wavelet .1 BPX .1
1e + 10 100000 1 1e – 05 1e – 10 1e – 15 0
20
40
60
80
100
120
Fig. 6.3 Comparison of the wavelet diagonal scaling preconditioned CG method with the BPX preconditioned CG method for c = 0.1 and d = 2 (top), d = 3 (middle) and d = 4 (bottom).
N U M E R I C A L E X A M P L E S FO R T H E M O D E L P RO B L E M 1000
300 250
100
200
10
150
1
100
ms_wavelet ms_diag mgm_bpx lt 0
50 0
181
0
2
4
6
8
10
ms_wavelet ms_diag mgm_bpx lt 0 multigrid
0.1 12
0.01
10 12 14 16 18 20 22 24 26
Fig. 6.4 CPU times over error for Wavelet-Galerkin with standard and diagonal preconditioning compared with BPX. The horizontal axis k corresponds to an error of 10−k (left). The right picture shows the CPU time over the level j also including MultiGrid. For the wavelet preconditioner, we also compare two versions, namely the standard diagonal scaling by D −1 mentioned above and a diagonal preconditioner using the diagonal of the stiffness matrix Cj = diag(Aj ). The latter one should perform a little bit better since it uses more information directly from the stiffness matrix. The result in Figure 6.4 shows that both variants of the wavelet scheme perform better then BPX and that the diagonal preconditioner performs better then the standard diagonal scaling. In order to view the full picture, we now add the CPU times for the waveletbased methods to the original Table 4.8 (page 115) in Table 6.3. Again, we fix d = 3, d˜ = 5 and c = 0.1. Again, MultiGrid is the clear winner among the four considered methods. 6.3.2
Compression
One key property of a wavelet representation of a function is the fact that wavelet coefficients are small locally in regions where the function is smooth. We have seen this fact in Proposition 5.9 (page 148) as a consequence of the vanishing moment property. In signal and image processing this property is used to compress a given signal or image. When numerically solving a boundary value problem, the function we wish to compress is not given explicitly but implicitly as the solution of the boundary value problem at hand. In the 1D example we can hope that a smooth right-hand side f implies a smooth solution u and that regions of nonsmoothness of f correlate to such regions of u. Thus, we perform two experiments for (3.1), one with the smooth function f (1) (x) := sin(2πx)
182
WAV E L E T - G A L E R K I N M E T H O D S
Table 6.3 Comparison of CPU times for d = 3, d˜ = 5 and c = 0.1. j
MGM-BPX
MS-Wavelet
MS-Diagonal
MultiGrid
10 11 12 13 14 15 16 17 18 19 20 21 22 23
0.03 0.08 0.13 0.29 0.6 1.3 2.67 5.73 12.33 26.42 55.44 118.36 266.18 544.33
0.02 0.05 0.11 0.2 0.46 0.92 1.95 4.16 9.04 19.94 40.94 98.85 218.68 449.97
0.02 0.05 0.1 0.18 0.38 0.8 1.65 3.54 7.4 15.89 32.62 69.18 155.42 331.44
0.02 0.03 0.04 0.08 0.17 0.32 0.65 1.38 2.77 5.91 12.59 25.45 53.82 137.61
and one with the jump function (2)
fj (x) := χj (x),
where j := [0.5 − 2−j , 0.5 + 2−j ],
i.e. a small box centered at 0.5 with size 21−j . In order to visualize the size of the wavelet coefficients, we use a 2D plot where the horizontal axis corresponds to the location and the vertical axis denotes the level j. We obtain 2j boxes per level j and we color these boxes according to the size of the wavelet coefficient dj,k . Note that these boxes in general do not coincide with the support j,k of the wavelet ψj,k . This is only true for the Haar wavelet. In Figure 6.5, we show in the first row, the wavelet representation of (2) (2) f (1) (left), f2 (middle), and f4 (right) as well as the corresponding solutions for d = d˜ = 2 and maximal level j = 10. For the smooth function f (1) we can hardly see anything. There seems to be some structure in the sense that wavelet coefficients of the right-hand side and of the solution are slightly larger in regions where f (1) changes. This situation is much more pronounced in the other two cases where the wavelet coefficients (2) clearly reflect the boundaries of the piecewise constant piece. At least for f4 , similar behavior is also visible in the wavelet coefficients of the solution. Hence, at least in this case the wavelet coefficients of the right-hand side can be used as a prediction for those of the solution. In Figure 6.6, we show the same data for d = 3, d˜ = 5. The qualitative behavior is similar to the case d = d˜ = 2. However, we see that fewer wavelet coefficients are needed, which indicates that the compression rate is higher in
8
8
7
7
7
6
6
6
5
5
5
4 3
level
8
level
level
N U M E R I C A L E X A M P L E S FO R T H E M O D E L P RO B L E M
4 3
4 3
1
1
0
0
0
–1
–1
–1
8
8
8
7
7
7
6
6
6
5
5
5
3
level
2
1
level
2
level
2
4
4 3
183
4 3
2
2
2
1
1
1
0
0
0
–1
–1
–1
(2)
(2)
8
8
7
7
7
6
6
6
5
5
5
4 3
level
8
level
level
Fig. 6.5 Wavelet coefficients of f (1) (left), f2 (middle), f4 (right) in the first row and corresponding solutions (second row) for j = 10, d = d˜ = 2.
4 3
4 3
1
1
0
0
0
–1
–1
–1
8
8
8
7
7
7
6
6
6
5
5
5
4 3
level
2
1
level
2
level
2
4 3
4 3
2
2
2
1
1
1
0
0
0
–1
–1
–1
(2)
(2)
Fig. 6.6 Wavelet coefficients of f (1) (left), f2 (middle), f4 (right) in the first row and corresponding solutions (second row) for j = 10, d = 3, d˜ = 5.
184
WAV E L E T - G A L E R K I N M E T H O D S
this case. Again, at least in the third case the right-hand side coefficients are a good prediction. We see that the peaks in the right-hand side are clearly visible in the coefficients. Since the solution gains two orders of regularity, these two peaks are not immediately visible in the solution itself. A closer look, however, shows that peaks in the right-hand side correspond to (milder) peaks in the coefficients of the solution. This is the main reason why some adaptive strategies try to use the structure of the wavelet coefficients of the right-hand side as a prediction for the wavelet coefficients of the solution, at least for predicting the indices of the significant coefficients. 6.4
Exercises and programs
Exercise 6.1 Show Lemma 6.4, i.e. determine the number of nonzero terms in the wavelet stiffness matrix. Exercise 6.2 Consider the convection–diffusion–reaction problem −αu (x) + β u (x) + γu(x) = f (x), on (0, 1) with periodic boundary conditions. Derive the weak formulation. Can you construct a preconditioner? Exercise 6.3 Consider the following problem with nonconstant coefficients −(α(x) u (x)) + γ(x) u(x) = f (x), on (0, 1) with periodic boundary conditions. Derive the weak formulation. Can you construct a preconditioner? Exercise 6.4 Assume that you are given a finite wavelet expansion v=
cj,k ψ˜j,k ,
(j,k)∈Λ
where Λ is a finite index set. Determine the expression of v in terms of the primal wavelets. Do you obtain a finite expansion? Computer Exercise 6.1 Determine the solution of −u (x) + u(x) = f (x),
x ∈ [0, 1]
with periodic boundary conditions using the Wavelet-Galerkin method with the CG method for j = 5, . . . , 12 and f (x) = x, f (x) = sin2 (πx) as well as f (x) = sin(2πx). Monitor the number of required CG iterations and estimate the rate of convergence.
E X E RC I S E S A N D P RO G R A M S
185
Computer Exercise 6.2 Use the wavelet library to write a code that computes AΨj for different levels j and different types of wavelet bases. Plot the sparsity pattern. Try also compactly supported orthonormal wavelets. Computer Exercise 6.3 Determine numerically the solution uk (x) of the periodic boundary value problem with the right-hand side f (x) = sin(kπx) for different values of k for fixed j = 12. For the solution plot the following graph: On the horizontal axis use tol ∈ R+ 0 . Plot the number of wavelet coefficients dj,k of the solution uk (x) that are larger then tol in absolute values, i.e. |dj,k | ≥ tol. What do you observe? Programs Besides the following programs we provide the codes listed below. • Computation of the wavelet stiffness matrix. • Point evaluation of a function given as wavelet expansion, primal and dual. • General frame for the preconditioned CG method that allows the use of the FWT in the preconditioner without assembling the wavelet stiffness matrix. • Visualizing wavelet coefficients as in Figure 6.5. • Measurements of CPU times. Program 6.1 The code wavelet_stiffness_matrix_spy.cc determines the wavelet stiffness matrix and produces a figure showing its spy structure. Program 6.2 The code wavelet_galerkin.cc uses the preconditioned Wavelet-Galerkin method to solve the periodic boundary value problem.
7 ADAPTIVE WAVELET METHODS
As we have seen above, wavelet compression allows one to obtain a sparse representation of functions in the sense that wavelet coefficients are small (and hence may be neglected) if the function is locally smooth (see Proposition 5.9, page 148). It is remarkable that one also obtains a sparse representation of a wide class of operators. Recall that the wavelet stiffness matrix AΨj is not sparse but almost sparse since it has O(Nj log(Nj )) entries. We have indicated the famous finger structure. We will further see that one can compress AΨj even more if one is only interested in an approximate application of this matrix (operator) to a given vector. This can be used to design adaptive methods for solving partial differential equations. By adaptive we mean in particular that the degrees of freedom (i.e. here the choice of particular wavelets out of a whole wavelet basis) are selected during the computation without a priori knowledge of the unknown solution. In order to describe the general principle of adaptive wavelet approximations, we start by approximating a given function f . In this framework, the wavelet coefficients of f are known (at least in principle since there may be infinitely many) and one can obtain an adaptive approximation by choosing some of them in an appropriate way. When solving an operator equation, the function we are looking for is only implicitly given as the unique solution of the particular operator equation. Thus, the corresponding wavelet coefficients are not known. In the more classical approach one computes a trial approximation and a local a posteriori error estimator. Then, one refines locally in regions where the error estimator is large. After describing this more classical method, we consider adaptive wavelet methods based upon an approximate operator application. This chapter is mainly based upon several papers by Cohen, Dahmen and DeVore, e.g. [66–69].
7.1
Adaptive approximation of functions
Before we turn to the description of adaptive methods for solving elliptic boundary value problems, let us illustrate the potential of wavelets for the adaptive approximation of given functions. We consider a wavelet representation of the 186
A DA P T I V E A P P ROX I M AT I O N O F F U N C T I O N S 1.2
0.25
1
0.2
0.8
0.15
0.6
0.1
0.4
0.05 0
187
0.2 0
0.2
0.4
0.6
0.8
1
0
2
4
6
8 10 12 14 16 18 20
Fig. 7.1 Function f defined in (7.1) (left) and size of wavelet coefficients (right). function f : [0, 1] → R defined by f (x) :=
x2 , (1 − x)2 ,
if 0 ≤ x ≤ 1/2, if 1/2 < x ≤ 1,
(7.1)
using biorthogonal B-spline wavelets of order d = d˜= 3. In the left part of Figure 7.1, we show the function f itself. It is chosen in such a way that it is not continuously differentiable in the midpoint of the interval. On the right part of that figure, we illustrate the size of the wavelet coefficients for all levels up to j = 8 (i.e. 28 = 256 wavelets are used). We have ordered the coefficients in order of their absolute values starting with the largest. We see that only a few coefficients are “big” and then there is a fast decay so that most of the coefficients are quite small. In other words, only a few coefficients are large and hence significant. 7.1.1
Best N -term approximation
The general idea of an adaptive approximation is now as follows. We fix the number of degrees of freedom (d.o.f.) for an approximation. In our case of a wavelet approximation this means that we fix the number of nonzero wavelet coefficients; we also call these active coefficients. One could think of the situation where the available amount of memory is an upper bound for the number of degrees of freedom that can be used within a numerical method. For simplicity, let us assume that we have complete knowledge of the (infinite) wavelet representation of f , namely we assume that we know f=
dλ ψλ ,
d := (dλ )λ∈J ,
λ = (j, k),
λ∈J
and we wish to measure the quality of the approximation in the L2 (Ω)-norm. We denote by ΣN the subset of all wavelet expansions with at most N ∈ N
188
A DA P T I V E WAV E L E T M E T H O D S
terms, i.e. ΣN :=
v=
dλ ψλ : #Λ = |Λ| ≤ N
.
λ∈Λ
Hence, the best possible approximand to f would be fN = arg min f − gN 0;Ω . gN ∈ΣN
Note that fN may not be unique. In fact, think of a function with N +1 constant wavelet coefficients. Then any choice of N nontrivial linear combinations is a best approximation. Now, we transform this problem posed in a function space to the wavelet coefficients (sometimes also called wavelet space). Since Ψ = {ψλ : λ ∈ J } is a Riesz basis, we have the norm equivalence f = dT Ψ = d λ ψλ . (7.2)
f 0;Ω ∼ d 2 (J ) , λ∈J
Let us now assume that we are allowed to spend N ∈ N degrees of freedom to determine an approximation fN to f . Again, fN can be expanded in terms of wavelets, i.e. fN = cλ ψλ = cT Ψ, c := (cλ )λ∈J , λ∈J
where only N coefficients of c are nonzero, i.e. defining the support of a sequence c ∈ (J ) by supp c := {λ ∈ J : cλ = 0},
(7.3)
we look for an approximand fN that fulfills |supp c| ≤ N. This basically means that we look for a vector cN in RN . For the error, we have
f − fN 20;Ω ∼ |dλ − cλ |2 .
(7.4)
λ∈J
Obviously the right-hand side becomes minimal if cN is chosen as the N largest entries (in absolute values) of d, i.e. dλ , if |dλ |is among the N largest entries in d, cλ := 0, else.
A DA P T I V E A P P ROX I M AT I O N O F F U N C T I O N S
189
It is clear that this choice is not unique (think again of N + 1 constant coefficients), but it is also clear that this is the best possible choice in the sense cN = arg min d − eN 2 (J ) . eN ∈RN
Hence, due to the norm equivalence, the arising function fN = cTN Ψ is comparable to the best N -term approximation to f which is the best approximation to f by taking any linear combination of N basis function in Ψ. Thus we obtain
f − fN 0;Ω ∼ ρN (f ), where the error of the best N -term approximation is defined as gλ ψλ , #Λ = N , ρN (f ) := inf f − gN 0;Ω : gN = λ∈Λ
and Ψ = {ψλ : λ ∈ J } is a given Riesz basis of L2 (Ω), Λ ⊂ J is a subset of indices. In Figure 7.2, the slope of ρN (f ) as a function of N ∈ N is shown for our example (7.1). We see the very rapid decay indicating that very few wavelets suffice to approximate f already quite well. Hence, the above minimization problem is equivalent to determining cN ∈ 2 (J ) such that cN = arg min d − dN 2 (J ) , dN ∈ΣN
where
ΣN := d˜ ∈ (J ) : # supp d˜ = N ∼ = RN
is the space of all finite sequences of length N . 2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0
1
10
100
1000
10000
Fig. 7.2 Slope of the best N -term approximation.
190
A DA P T I V E WAV E L E T M E T H O D S
Obviously, the best N -term approximation is a nonlinear approximation process which can equivalently be described in function spaces instead of sequence spaces. In fact, we determine an approximation to the function f out of the set ΣN similarly as above and formally defined as ΣN :=
u=
uλ ψλ : #Λ = N
λ∈Λ
which is a nonlinear manifold since in general u + v ∈ Σ2N ⊆ ΣN ,
u, v ∈ ΣN .
Thus, we can abbreviate ρN (f ) =
inf
gN ∈ΣN
f − gN 0;Ω .
Note that ρN (f ) of course also depends on the chosen wavelet basis Ψ so that one should write ρN (f ; Ψ). For adaptive wavelet methods, however, one fixes Ψ a priori (usually in such a way that regularity and vanishing moments correspond to the need of the particular application) so that we omit the reference to the wavelet basis Ψ. 7.1.2
The size and decay of the wavelet coefficients
Let us now look more closely at the wavelet coefficients, in particular at their size. In Figure 7.3 we show all nonzero wavelet coefficients of the function f in (7.1) for d = 2 and d˜ = 2. We describe the visualization again in a little more detail since it will be crucial for the remaining parts of the book. The vertical axis corresponds to the level and the horizontal axis corresponds to the shift-index k of ψj,k , i.e. to the location in space. The amount of gray reflects the size of the absolute value of the corresponding coefficient, i.e. the darker the larger. It can be seen that the coefficients on higher levels are large only near the singularity of the derivative of f at x = 1/2. We also see that we obtain a cone of wavelet coefficients around the point x = 1/2. Another aspect is that almost all wavelet coefficients are nonzero, but many of them are quite small. As above j = −1 denotes the scaling function on the coarse level j0 which is the constant function on the interval. This means that the corresponding coefficient is the mean of f . Level 0 consists of one wavelet with support length 1 which is skew-symmetric around x = 0.5. Since f is symmetric around this point, the corresponding wavelet coefficient is zero which can be seen by the white bar corresponding to j = 0 in Figure 7.3.
A P O S T E R I O R I E R RO R E S T I M AT E S A N D A DA P T I V I T Y
191
9 8 7 6 level
5 4 3 2 1 0 –1 0
location
1
Fig. 7.3 Visualization of all nonzero wavelet coefficients of the function f in (7.1). Even though this is no more than an example, let us note two observations that will be made precise later on: • In regions where f is smooth, a few wavelets on coarse scales suffice to approximate f . Wavelet coefficients on high levels are small in smooth regions. • Near a singularity (of the derivative) we encounter large coefficients on high levels. In this sense, the wavelet coefficients reflect the behavior of a function. Obviously, the above properties make wavelets a potential candidate for using them as a basis of an adaptive approximation process which later will be used as a solution method for partial differential equations. In fact, in many cases, the solution is smooth in large regions of the domain Ω and has only singularities at localized regions, e.g. near corners of the domain or caused by singularities of the right-hand side.
7.2
A posteriori error estimates and adaptivity
Before we proceed to introduce adaptive wavelet methods, let us introduce the more classical approach for adaptive schemes which is based on error estimates
192
A DA P T I V E WAV E L E T M E T H O D S
and local refinement. This is useful here in order to explain similarities and differences of the approaches. 7.2.1
A posteriori error estimates
In more classical discretizations like finite elements, adaptivity is driven by so-called local a posteriori error estimates. Roughly speaking, one aims at constructing a functional η:X→R (we change the notation here the from H to X; the reason will become clear soon) that is easily computable at least for a given numerical approximation uH . One requires that η is an efficient and reliable error estimator which means that c1 u − uH X ≤ η(uH ) ≤ c2 u − uH X with absolute constants 0 < c1 ≤ c2 < ∞. Of course, error measures different from
· X have also been considered. Finally, η is constructed in such a way that it consists of local terms, i.e. η=
M
ηi
i=1
where ηi is supported on i ⊂ Ω which are subdomains forming a covering Ω⊆
M i=1
i ,
|Ω| ∼
M
|i |.
i=1
For simplicity, think of a finite element mesh and the error estimator for one element T depends on the values on T and the neighboring elements. Hence, there are some i that overlap, but the union of all these cover Ω. Depending on the size of ηi , one refines the discretization locally in i , i.e. one adds new elements to a triangulation. The rule of thumb is to refine if ηi is large and to derefine (coarsen) if ηi is small. Coarsening is done if performance is an important task in an application. The general form of such a method is shown in Figure 7.4. The construction, analysis and implementation of appropriate a posteriori error estimates and corresponding adaptive methods are a wide field of active research. We will not go into any details here since adaptive wavelet methods follow a different path. One important question is of course whether an adaptive iteration converges. Intuitively, this seems clear since refining a coarse discretization TH to a finer one Th should guarantee that the numerical solution uh on Th is better than uH on TH . Of course, one expects that
u − uh X ≤ u − uH X ,
A P O S T E R I O R I E R RO R E S T I M AT E S A N D A DA P T I V I T Y
193
PDE in variational form
a(u, v) = (f, v)
∀v ∈ X
?
Determine discretization
Xh ⊂ X
?
Solve finite-dimensional problem
a(uh , vh ) = (f, vh )
∀vh ∈ Xh
?
Check quality of the solution uh by a-posteriori error estimates
? H H Satisfied?HH HH H H
No
Refine/Derefine Xh
Yes
? STOP
Fig. 7.4 Form of a classical adaptive method with a posteriori error estimation and (de-)refinement. Note that the finite-dimensional problem on Xh usually cannot be solved exactly.
194
A DA P T I V E WAV E L E T M E T H O D S
but this does not imply that the scheme converges. Typically, the so-called saturation property is assumed, i.e.
uh − uH X ≥ γ u − uH X ,
0 < γ < 1,
which, by orthogonality, is equivalent to
u − uh X ≤
/ 1 − γ 2 u − uH X .
/ Since 1 − γ 2 < 1, this implies convergence. However, it is a priori not clear if the saturation property holds. The first rigorous proof of convergence is due to W. D¨ orfler [107], for piecewise linear finite elements for the Poisson equation. A whole series of papers came after this, see e.g. [7, 32, 156–158]. The first investigations of adaptive wavelet methods for numerically solving partial differential equations followed a somewhat similar path. The early papers were mainly concerned with time-dependent problems. The idea is to predict the significant wavelet coefficients of the solution at the next time step from the current approximation [57, 58, 145, 148]. Other approaches are based upon collocation and interpolatory wavelets [22, 26–28, 30, 108]. Since this is a highly active field of research this list of references cannot claim to be complete. For stationary problems, the first investigation was in 1997 in [76], where a posteriori error estimates have been constructed for wavelet discretizations. 7.2.2
Ad hoc refinement strategies
Before we come to the description of the general framework that allows for a deep analysis (convergence and convergence rates) of the resulting adaptive schemes, we indicate some ad hoc strategies that will also give a good indication of why the whole method works. These strategies are based upon the observation that small wavelet coefficients indicate local regularity whereas large coefficients reflect some discontinuity. We use this observation in order to construct local error indicators as in Figure 7.4. Of course, we cannot hope to give rigorous proofs for such a heuristic approach, but it works astonishingly well. In order to formulate a refinement strategy, we define an index cone which is the set of indices that will be activated once a wavelet labeled λ = (j, k) is marked for refinement. The index cone consists of the neighbors on level j and those wavelets on level j + 1 that overlap ψλ , i.e. C(λ) := {(j + 1, m) ∈ Jj+1 : |(supp ψ[j,k] ) ∩ (supp ψ[j+1,m] )| ≥ 0} ∪ ({(j, k − 1), (j, k + 1)} ∩ Jj ) .
(7.5)
A P O S T E R I O R I E R RO R E S T I M AT E S A N D A DA P T I V I T Y
195
Next, we want to give a precise representation of the index cone C(λ), i.e. we want to determine the corresponding indices in C(λ). In order to do so, let supp ψ = [∗1 , ∗2 ], where ∗1 < ∗2 obviously depend on the particular choice of the (mother) wavelet. Then, supp ψ[j,k] = 2−j [∗1 + k, ∗2 + k], so that |(supp ψ[j,k] ) ∩ (supp ψ[j+1,m] )| > 0 iff m ∈ {2∗1 − ∗2 + 2k, . . . , 2∗2 − ∗1 + 2k}. Thus, noting that Jj = {0, . . . , 2j − 1} is the wavelet index set on level j, we obtain the following representation of the index cone C(λ) = {{(j, k − 1), (j, k + 1)} ∩ Jj } ∪ {{(j + 1, 2∗1 − ∗2 + 2k), . . . , (j + 1, 2∗2 − ∗1 + 2k)} ∩ Jj+1 } .
(7.6)
In Figure 7.5, we have demonstrated this set for one given index λ (marked black). Next, we need to formulate a mechanism to activate an index for refinement. Given a partial differential equation Au = f and a (finite) set of indices Λ ⊂ J denote by uΛ the Galerkin solution corresponding to Λ. The idea is to test the Galerkin solution uΛ with respect to a larger index set in order to check if the current solution might already be accurate
j +1
j λ
Fig. 7.5 Index cone for an index λ = (j, k).
196
A DA P T I V E WAV E L E T M E T H O D S
enough. In order to do so, we define a security zone which is simply the union of all index cones defined above, namely C(λ). C(Λ) := λ∈Λ
Then, we consider the extended residual defined by rC(Λ) := fC(Λ) − AC(Λ)×Λ uΛ , where for Λ, Θ ⊆ J we define by AΛ×Θ := A|Λ×Θ AΛ×Θ , := A|Λ×Θ = (a(ψλ , ψµ ))λ∈Λ,µ∈Θ , the section of an operator and its discrete representation, respectively. Here (the approximation of) the Galerkin solution uΛ of AΛ×Λ uΛ = fΛ has the representation u λ ψλ . uΛ = λ∈Λ
Next, uΛ
and rC(Λ)
are the vectors of wavelet coefficients of the functions uΛ and rC(Λ) , respectively. We consider two approaches detailed in the sequel. Strategy 1: Mark all λ where |rλ | > η. ∗ a reordering of rΛ in decreasing order Strategy 2: Denote by rΛ
|r1∗ | ≥ |r2∗ | ≥ . . . and mark the smallest number Mη of indices such that Mη
|ri∗ |2 ≥ η rΛ 22 ,
i=1
where 0 < η < 1 is some parameter. ˜ All indices λ that are marked by such a strategy form the new index set Λ. Obviously, the first strategy means that all wavelets are refined whose residual coefficients are above certain threshold. The second strategy ensures that the bulk of the error is caught for refinement. A typical choice for the percentage of the bulk is η = 0.8.
A P O S T E R I O R I E R RO R E S T I M AT E S A N D A DA P T I V I T Y
197
Finally, it amounts to defining a suitable stopping criterion. To this end, let ε > 0 be a given tolerance for the final error. In order to formulate a stopping criterion, recall that rC(Λ) = fC(Λ) − AC(Λ)×Λ uΛ . is designed in such a way that it should contain the bulk of the error in the security zone C(Λ). Hence, a good approximation to the overall error should arise if we add the residual corresponding to the active set of indices Λ, i.e. sΛ := fΛ − AΛ×Λ uΛ . Note that rΛ = sΛ if C(Λ) = Λ, which is zero if the discrete system on Λ were solved exactly. Moreover, we also take numerical errors into account. Moreover, note that in general Λ ⊂ C(Λ). The same reasoning is used for the right-hand side so that the stopping criterion reads
rC(Λ) 2 + sΛ 2 ≤ ε fC(Λ) 2 + fΛ 2 . (7.7) Numerical experiments We consider again our model problem, namely the periodic boundary value problem (3.1) for specific choices of the right-hand side f . Throughout the whole chapter we will use these examples to demonstrate properties of the described adaptive schemes. Example 7.1 We construct the right-hand side f of (3.1) such that the exact solution is 2
u(x) = e−ax
(1−x)2
,
(7.8)
which is C ∞ , periodic and has a periodic derivative, i.e. ∞ u ∈ Cper .
Thus, the function u in particular has arbitrarily high Sobolev regularity. One reference value for a is a = 100. The function u and the associated right-hand side corresponding to c = 0.1 are displayed in Figure 7.6. Example 7.2 We consider the function u(x) := sin(2π x),
198
A DA P T I V E WAV E L E T M E T H O D S 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
250 200 150 100 50 0 –50 0
0.2
0.4
0.6
0.8
1
–100 0
0.2
0.4
0.6
0.8
1
Fig. 7.6 Exact solution (left) and right-hand side f (right) corresponding to Example 7.1. 2.5
× 104
2 1.5 1 0.5 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Fig. 7.7 Exact solution corresponding to Example 7.3. ∞ which is obviously in Cper . In this case, the corresponding right-hand side is easily computable, namely
f (x) = (4π 2 + c) sin(2π x), which can easily be used to verify results. Example 7.3 We construct the right-hand side f of (3.1) such that the exact solution is if 0 ≤ x ≤ 0.5, eax , (7.9) u(x) = ea(1−x) , if 0.5 < x ≤ 1, which is C ∞ and periodic. The first derivative is not periodic, but choosing a sufficiently large (we use a = 20) leads to a derivative which is periodic at least up to numerical precision. The function is depicted in Figure 7.7.
A P O S T E R I O R I E R RO R E S T I M AT E S A N D A DA P T I V I T Y
199
9 8 7 6 5
Level
Level
First, we compare the two above-mentioned strategies. Obviously, Strategy 1 is an absolute one since wavelet coefficients are activated just by their size no matter how large or small the other coefficients are. In contrast, the second strategy is a relative one. It is well known that one should use the second option, see e.g. [50]. This is also seen by a numerical comparison. In Figure 7.8, we show index ranges corresponding to the two strategies. It can clearly be seen that Strategy 1 almost leads to a uniform refinement in this case. We have used Example 7.1 here, where the solution has steep gradients close to the borders of the interval and is almost zero in the center. This behavior is reflected by the wavelet coefficients and is also visible for Strategy 2. Next, we perform a quantitative investigation of Strategy 2. In Table 7.1, we give the sizes of the sets of active wavelets Λ that have been selected by the
4 3 2 1 0 –1
9 8 7 6 5 4 3 2 1 0
–1
Fig. 7.8 Structures of wavelet coefficients of an adaptive scheme based upon Strategy 1 (left) and Strategy 2 (right). Table 7.1 Sizes of the active wavelet sets, interior residual (on Λ) and global error. It
|Λ|
Residual
Error
0 1 2 3 4 5 6 7 8 9
4 6 8 11 14 21 26 30 44 52
0.08183 0.09800 0.10162 0.07062 0.01915 0.01490 0.00880 0.00270 0.00257 0.00176
2.37714 1.27443 0.37517 0.21246 0.09356 0.06541 0.03519 0.02076 0.01174 0.00793
200
A DA P T I V E WAV E L E T M E T H O D S 101
100
10–1
10–2
10–3
0
10
20
30
40
50
60
Fig. 7.9 Error decay of the ad hoc refinement strategy using Strategy 2 for the refinement. 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 –0.1
1 0.8 0.6 0.4 0.2 0 –0.2 0
0.2
0.4
0.6
0.8
1
1 0.8 0.6 0.4 0.2 0
0.2
0.4
0.6
0.8
1
0
0
0.2
0.4
0.6
0.8
1
Fig. 7.10 Computed final approximations of the ad hoc refinement strategy for Example 7.1 using ε = 0.1 (left), ε = 0.01 (center) and ε = 0.001 (right). refinement strategy. We also display the residuum on Λ as well as the overall error corresponding to the exact solution. Here, we have chosen ε = 0.001. The slope of the error decay is shown in Figure 7.9. In Figure 7.10 we show the final approximate solution for Example 7.1 using the ad hoc strategy for ε = 0.1, 0.01 and 0.001. 7.3
Infinite-dimensional iterations
Even though the above simple strategy works in a decent manner, at least at the current state of the art, there is a lack of mathematical theory for this approach. In fact, there is no convergence analysis for the above method that would ensure the convergence of the adaptive scheme. Neither are results concerning optimality known. Thus, we now describe a new point of view for adaptive wavelet methods introduced by Cohen, Dahmen and DeVore. We follow their description in [66]. As we have seen in Theorem 6.3 (page 174), an elliptic variational problem in a function space can be transformed into a well-conditioned problem in the sequence space 2 (J ) by using its wavelet representation. Moreover we have shown how this transformation is the basis for preconditioning and it can also be
I N F I N I T E - D I M E N S I O N A L I T E R AT I O N S
201
used to design an adaptive method. The strong analytical properties of wavelets allow a deep analysis of such methods. We will come back to this point later. It pays off to describe this approach in a somewhat more general setting in order to highlight the principles. To this end, let us come back to the consideration of a linear operator equation in Banach spaces Au = f,
A : X → Y,
(7.10)
where f ∈ Y is given, u ∈ X is to be found and A : X → Y is such that the above problem is uniquely solvable. For the moment, we forget that we do not know how to represent elements of abstract (possibly infinite-dimensional) spaces X and Y in a computer. Let us first design an abstract algorithm to solve (7.10) adaptively. Later, we will modify this algorithm so that it is completely realizable and efficient. This is done by introducing computable approximations and controlling the introduced errors. The abstract method reads as follows: Given u(0) ∈ X, we define for n ≥ 0, u(n+1) = u(n) + R(n) (f − Au(n) ) = (I − R(n) A)u(n) + R(n) f,
(7.11)
where R(n) : Y → X,
n≥0
are linear operators to be chosen appropriately. We easily obtain for the exact solution u of (7.10) that u(n+1) − u = I − R(n) A u(n) + R(n) f − u − R(n) (f − Au) = I − R(n) A u(n) − u , since Au − f = 0. Hence, we have
u(n+1) − u X ≤ I − R(n) A L(X,X) u(n) − u X , where as usual the operator norm is defined by
C L(X,X) := sup
v∈X
Cv X ,
v X
C ∈ L(X, X),
so that (7.11) converges if
I − R(n) A L(X,X) ≤ < 1,
n ∈ N.
(7.12)
Let us now describe how to obtain a computable version of this method. Assuming that the initial guess (the starting value for the iteration) u(0) ∈ X
202
A DA P T I V E WAV E L E T M E T H O D S
can be represented by a finite number of degrees of freedom (in a computer), we need to realize the following issues in order to obtain a computable version of (7.10). 1. Determine R(n) such that (7.12) holds. 2. Replace f (or R(n) f ) by an approximation depending on a finite number of degrees of freedom. 3. Replace Au(n) (or R(n) Au(n) ) by an approximation depending on a finite number of degrees of freedom. Now, we are going to investigate these issues. 1. The first issue is obviously related to preconditioning of the operator A. Moreover, the choice of R(n) determines the type of iteration. The choice R(n) = α I,
α ∈ R+ ,
corresponds to a Richardson-type iteration. It is well known that for symmetric positive definite operators A there always exists an α ∈ R+ such that (7.12) holds true. Obviously the choice of R(n) depends on the underlying problem as well as on the specific requirements of the user. We will not go into detail here. Instead, we will just assume that such an R(n) is available and can be applied to any given (finite) input at least up to any desired tolerance in a finite number of operations. 2. For the right-hand side f ∈ Y of (7.10) we always assume that it is known. This is in fact realistic since f typically represents given data. At least, we can approximate f to any desired accuracy. Moreover, since R(n) is assumed to be accessible, also R(n) f can at least be approximated up to any desired accuracy. 3. The third issue listed above is perhaps the most serious one and will not always be solvable. In fact, even if u(n) may be described in terms of a finite number of degrees of freedom, Au(n) is infinite since in general (and basically in all relevant cases) A is an infinite operator. At this point, we may make use of the analytic properties of wavelets. In fact, using the wavelet representation will allow us to approximate a wide range of operators in the sense that the application of A to a finite input u(n) can be approximated by a finite term up to any desired accuracy. Of course, the size of the output will depend on the target accuracy. This setting can basically be applied to general Banach spaces X and Y . Thus, we could try to construct an abstract infinite-dimensional algorithm in the function spaces, i.e. choose X = H01 (Ω) and Y = H −1 (Ω). However, it turns out to be much more convenient (and numerically realizable) to use the norm equivalences and then to consider the boundary value problem as a problem in the sequence space 2 .
A N E Q U I VA L E N T 2 P RO B L E M : U S I N G WAV E L E T S
7.4
203
An equivalent 2 problem: Using wavelets
In the previous section, we have described the most general situation of an operator A acting on some Banach spaces. This, however, is too broad a framework for a rigorous analysis. Now we use Theorem 6.3 which allows us to transform the original problem into a well-conditioned problem in the sequence space 2 (J ), i.e. the iteration will be performed in 2 (J ) = X = Y. Hence, we consider the problem of determining u ∈ 2 (J ) such that A u = f, where A ∈ L(2 (J ), 2 (J )) is a well-conditioned operator, i.e. cond2 (A) < ∞,
cond2 (A) := A A−1 ,
and f ∈ 2 (J ) is given. As in Theorem 6.3, we always assume that the 2 (J )-quantities arise by a wavelet representation of the original problem Au = f , i.e. A = D −1 AΨ D −1 ,
AΨ = (AΨ, Ψ)0;Ω ,
u = Dd, f = D −1 (f, Ψ)0;Ω , where u = dT Ψ, and D is a symmetric, sparse, and invertible operator. Again, we have used the shorthand notation (AΨ, Ψ)0;Ω = ((Aψλ , ψµ )0;Ω )λ,µ∈J ,
(f, Ψ)0;Ω = ((f, ψµ )0;Ω )µ∈J .
At this point, we assume that a wavelet basis Ψ with the required properties is available. Let us again come back to issues 1–3 in Section 7.3, i.e. the requirements for an adaptive method in linear spaces, namely: 1. Determine R(n) . 2. Replace f by a finite approximation. 3. Replace Au(n) by a finite approximation.
204
A DA P T I V E WAV E L E T M E T H O D S
Now, let us comment on these issues for the wavelet-based problem in 2 : 1. First note that we assumed that the operator A is self-adjoint and elliptic, i.e. (Au, u)0;Ω u 21;Ω , which easily implies that A is symmetric and positive definite. Here, we typically only consider second-order problems. Moreover, A is also bounded, so that we have the two-sided inequality cA u 2 (J ) ≤ Au 2 (J ) ≤ CA u 2 (J ) . Then, it is an easy exercise to see that one can use the Richardson iteration, i.e. R(n) = α I, where α ∈ R+ has to be chosen in such a way that 0≤α≤
2 , λmin (A)
= max{|1 − αλmin (A)|, |1 − αλmax (A)|} < 1,2
which then implies
I − αA ≤ , i.e. choosing some < 1 implies convergence of the scheme. The optimal value is α=
2 . λmin (A) + λmax (A)
2. We assume that f is a given function. Hence, the corresponding wavelet coefficients (f, ψλ )0;Ω can at least be approximated by quadrature formulas up to any desired accuracy. Since D is a symmetric, sparse and invertible operator, we can assume that we are able to compute an approximation to f to any accuracy that may be needed. 2 Here λ max (A) and λmin (A) denote the maximal and minimal eigenvalue (in absolute values) of the operator A : 2 (J ) → 2 (J ), respectively. Eigenvalues and eigenvectors of such operators are defined in a straightforward way. If A is bounded, symmetric, and positive definite we obtain positive and finite eigenvalues.
C O M P R E S S I B L E M AT R I C E S
205
Hence, the first two of the above issues in Section 7.3 are solved and it remains to replace Au(n) by a finite approximation. In order to do so, we have to restrict ourselves to operators (we also sometimes talk about matrices having in mind that our “matrices” are with respect to infinite index ranges) that allow such an approximation. Now we introduce a class of such operators. Later, we will show (at least for our model problem) that wavelet representations of elliptic boundary value problems fall into this class. 7.5
Compressible matrices
We have already seen in Figure 6.1 (page 175) the finger structure of the wavelet stiffness matrix AΨj for a fixed level j. Now, we study the properties of the infinite matrix AΨ in detail. We show that the entries have a certain decay which allows us to compress these matrices and construct approximate application schemes. We will concentrate again on our periodic 1D model problem (3.1) in order to keep technicalities low. It should be clear from this context that all what is said can be generalized to realistic frameworks. In order to shorten notation, we consider two indices λ = (j, k), |λ| := j,
µ = (j , k ), |µ| := j
and the corresponding entry of the stiffness matrix a ˆλ,µ = (ψλ , ψµ )0;(0,1) + c(ψλ , ψµ )0;(0,1) , where ψλ := ψ[j,k] ,
ψµ := ψ[j ,k ]
denote two periodized wavelets. Again, we denote by 1 1 −1 A : Hper → (Hper ) =: Hper
the corresponding differential operator. Ellipticity tells us that there exists some τ > 0 such that
Au −1+s ∼ u 1+s ,
1+s u ∈ Hper , 0 ≤ |s| ≤ τ,
(7.13)
s where we denote by · s the norm in Hper . Finally, we use the norm equivalences (5.34) in the form
ψλ s ∼ 2s|λ| ,
s ∈ (−˜ γ , γ).
(7.14)
For convenience, we introduce a shorthand notation for the level difference of two indices, namely A A δ(λ, µ) := A |λ| − |µ| A. (7.15) Using these facts and notations, we obtain the following estimate.
206
A DA P T I V E WAV E L E T M E T H O D S
t Proposition 7.4 Let ψ ∈ Hper be a wavelet, with d vanishing moments and t˜ ˜ a dual wavelet ψ ∈ Hper , having d˜ vanishing moments,
γ := min t, d˜ ,
< = γ˜ := min t˜, d .
Then, |ˆ aλ,µ | 2−σδ(λ,µ) 2|λ|+|µ| for max{1 − γ, −1 − γ˜ } ≤ σ ≤ min{1 + γ˜ , γ − 1}. Proof We use Cauchy–Schwarz, (7.13) and (7.14) to obtain |ˆ aλ,µ | = |Aψλ , ψµ | ≤ Aψλ σ−1 ψµ −σ+1 ψλ σ+1 ψµ −σ+1 2|λ|(σ+1) 2|µ|(1−σ) = 2σ(|λ|−|µ|) 2|λ|+|µ| . Since we can exchange the roles of λ and µ, we obtain the assertion.
Let us comment on the relevance of this estimate. Taking the preconditioning into account, we get for the preconditioned operator A := D −1 AΨ D −1 ,
A = (aλ,µ )λµ∈J
the estimate |aλ,µ | 2−σδ(λ,µ) .
(7.16)
Since σ > 0, we see that this estimate guarantees some decay with respect to the level difference δ(λ, µ) defined in (7.15). Of course we know even more, namely that aλ,µ = 0 if |λ ∩ µ | = 0.
(7.17)
In fact, if the support of two functions is disjoint, their product (of derivatives) is zero and hence the entry in the stiffness matrix vanishes. This is due to the fact that the wavelets are compactly supported and that the differential operator is local. Note that also for some classes of nonlocal operators on Ω ⊂ Rn one can obtain decay estimates. In general, they take the form |aλ,µ | 2−( 2 +α)δ(λ,µ) d(λ, µ)−(n+β) n
(7.18)
C O M P R E S S I B L E M AT R I C E S
207
for some parameters α, β > 0, Ω ⊂ Rn (i.e. n denotes the spatial dimension) and d(λ, µ) := 1 + 2min{|λ|,|µ|} dist(λ , µ ). It turns out that decay estimates like (7.16) or (7.18) guarantee that the matrix A can be approximated by a sparse matrix. Let us now define that class of operators that can be approximated by a sparse matrix. Definition 7.5 An operator C : 2 (J ) → 2 (J ) is called s∗ -compressible if for any 0 < s < s∗ and every j ∈ N there exists a sequence α = (αj )j∈N ∈ 1 (N) and a matrix Cj obtained by replacing all but in the order of αj 2j entries per row and column in C by zero and a positive constant CC such that
C − Cj ≤ CC αj 2−js for all j ∈ N. It should be noted that replacing all αj by 1 (i.e. α ∈ 1 (N)) does not change the class of s∗ -compressible operators. Next, we want to show that matrices satisfying the decay estimate (7.16) are compressible. In order to do so, we need the well-known Schur lemma that we report in our notation. Lemma 7.6 (Schur lemma) Let M = (mλ,µ )λ,µ∈J be an operator such that there exists a sequence (wλ )λ∈J , and a constant 0 < c < ∞ such that ωµ |mλ,µ | ≤ c ωλ and ωλ |mλ,µ | ≤ c ωµ , (7.19) µ∈J
λ∈J
for all λ, µ ∈ J , then M ∈ 2 (J ) is bounded, i.e. M ≤ c. Proof Exercise 7.6. Now, we are ready to prove the already announced result. Theorem 7.7 Let A satisfy the decay property (7.16). Then A is σ-compressible.
208
A DA P T I V E WAV E L E T M E T H O D S
Proof We truncate A in scale in the following way: Given j, we discard aλ,µ if δ(λ, µ) ≥ j and denote the resulting matrix by Aj . We aim at using the Schur lemma with weights wλ = 2−
|λ| 2
.
We have to check the assumption (7.19) keeping in mind that A and thus also M := 2jσ (A − Aj ) is symmetric. Note that M = (mλ,µ ) has the components 0, if δ(λ, µ) < j, mλ,µ := 2jσ aλ,µ , else,
(7.20)
where we again use the abbreviation δ(λ, µ) for the level difference, see (7.15). Then, we have |λ| |µ| wλ−1 wµ (mλ,µ ) = 2 2 2jσ 2− 2 |aλ,µ |. µ∈J
µ∈J δ(λ,µ)≥j
Now, we note that for a given λ ∈ J we have #{µ ∈ J : δ(λ, µ) = j, aλ,µ = 0} ∼ 2j since the number of nonoverlapping functions grows exponentially with the level difference. Using (7.16), we get
2−
µ∈J δ(λ,µ)≥j
|µ| 2
|aλ,µ | 2j
∞
2−σ · 2−(j+|λ|) 2−σj 2−|λ|
=j
since we have a geometrical series. Combining all those estimates gives |λ| |λ| wλ−1 wµ (mλ,µ ) 2jσ 2 2 2−σj 2−|λ| = 2− 2 1, µ∈J
so that due to the symmetry of A the assumption (7.19) in the Schur lemma is proven and the Schur lemma gives the desired result. Before coming to the description of an adaptive approximate operator application for A, let us recall the current state. We have proven the decay estimate (7.16) |aλ,µ | 2−σδ(λ,µ)
C O M P R E S S I B L E M AT R I C E S
209
for the entries of the infinite stiffness matrix A for our model problem (3.1). We have indicated how to obtain such an estimate also for general elliptic partial differential equations. Next, we have seen that the decay estimate (7.16) ensures that A is σ-compressible, i.e. we can approximate A by sparse matrices Aj . In particular, the proof of Theorem 7.7 tells us how to obtain Aj , namely by a simple level-wise cut-off. Hence, we can easily compute these matrices Aj which is one of the core ingredients of the adaptive approximate operator application. Now assume that we are given a finite vector vN ,
N = #(supp v N ) < ∞.
We start by sorting v N with respect to the size of the absolute values of its entries. To this end, denote by N v[j] N the best 2j -term approximation to v N in 2 (J ), i.e. we obtain v[j] by retainj ing the 2 biggest coefficients (in absolute values) of v and setting all other coefficients equal to zero. Then, we compute so-called bins N v[0] , N N v[j] − v[j−1] ,
j = 1, . . . , log N ,
N v[j] := v N ,
j > log N.
(7.21)
It is remarkable that this process can be performed in O(N ) operations even though a complete sorting of the entries in v N would require O(N log N ) operations. In fact, it was shown in [8,10,171] that one can use an approximate sorting that can be performed in O(N ) operations. For a given k ∈ N to be determined later, we compute an approximation of Av N by N N Ai v[k−i] − v[k−i−1] i=0 N N N N N = Ak v[0] + . . . + A0 v[k] . + Ak−1 v[1] − v[0] − v[k−1]
N wkN : = Ak v[0] +
k−1
(7.22)
The precise value of k will be determined in order to reach a desired tolerance, i.e. the bigger k, the better is the approximation wkN to Av N . The idea of (7.22) is obvious: The less accurate bins are multiplied with an accurate approximation of A and the accurate bins with a less accurate approximation of A. The aim is to balance both accuracy and computational complexity at the same time. Let us now start with the analysis of this approximation, in particular with the question of what happens if k is growing. To this end, we need to introduce sequence spaces that characterize the decay of the coefficients. These are the so-called Lorentz spaces.
210
A DA P T I V E WAV E L E T M E T H O D S
Given a sequence v ∈ 2 (J ), we define its decreasing rearrangement v∗ of v = (vλ )λ∈J in the following way. For each n ≥ 1, let vn∗ be the nth largest of the numbers |vλ | and define v ∗ := (vn∗ )∞ n=1 . Definition 7.8 For each 0 < τ < 2 and v ∈ 2 (J ), we define |v|wτ (J ) := sup n1/τ vn∗
(7.23)
w < ∞}. τ (J ) := {v ∈ 2 (J ) : |v|w τ (J )
(7.24)
n≥1
and the weak τ -spaces as
These spaces are special cases of Lorentz sequence spaces. A corresponding norm is defined by
v wτ (J ) := |v|wτ (J ) + v 2 (J ) .
Let us give some equivalent description of these spaces. Proposition 7.9 The following characterizations of w τ (J ) are all equivalent to Definition 7.8. (a) We have v ∈ w τ (J ) if and only if vn∗ ≤ c · n−1/τ ,
n ≥ 1,
and the smallest constant c satisfying this inequality is equal to |v|wτ (J ) defined in (7.23). This is just a reformulation of Definition 7.8. (b) A sequence v is in w τ (J ) if and only if for any ε > 0 #{λ ∈ J : |vλ | ≥ ε} ≤ c ε−1/τ and the smallest such c is equal to |v|τw (J ) . τ (c) We have v ∈ w τ (J ) if and only if for all j ∈ Z #{λ ∈ J : 2−j ≥ |vλ | ≥ 2−j−1 } ≤ c 2jτ and again the smallest such constant c is equal to |v|τw (J ) . τ
C O M P R E S S I B L E M AT R I C E S
211
Proof Exercise 7.2.
It turns out that these spaces play an important role for the analysis of adaptive schemes. This is due to their close relation to best N -term approximations as introduced in Section 7.1.1. There, we introduced the best N -term approximation in the function space L2 (Ω). Using the norm equivalences of wavelets, we transform the boundary value problem to the sequence space 2 (J ). Thus, we also introduce the notion of the best N -term approximation in 2 (J ). Recalling from Section 7.1.1, we define the nonlinear manifold by ΣN := {u ∈ 2 (J ) : # supp u ≤ N } and the corresponding approximation error ρN (v) :=
inf
uN ∈ΣN
v − uN 2 (J ) ,
v ∈ 2 (J ).
As we have seen in Section 7.1.1, a best N -term approximation to v (which is of course not unique) can be obtained by taking the N largest coefficients v. In order to analyze adaptive wavelet methods, we compare them with the best benchmark that is available. This benchmark is the best N -term approximation. If our method converges with a rate that is comparable to the rate of the error of the best N -term approximation, then we say that the method is (asymptotically) optimal. Of course, we still require linear complexity in order to ensure optimal efficiency. Hence, we need to investigate how fast the error of the best N -term approximation decays. This leads to the following framework.
Definition 7.10 The approximation space As is defined as the set of all sequences in 2 (J ) that can be approximated by elements of ΣN with order O(N −s ), i.e. As := {v ∈ 2 (J ) : ρN (v) N −s }. A norm on As is defined by
v As := sup (N + 1)s ρN (v), N ≥0
ρ0 (v) := v 2 (J ) .
Now we are ready to formulate the announced approximation result.
(7.25)
212
A DA P T I V E WAV E L E T M E T H O D S
Proposition 7.11 Given s > 0, define τ :=
1 s+ 2
−1 .
(7.26)
Then we have: (a) The sequence v belongs to As if and only if v ∈ w τ (J ) and
v As ∼ v wτ (J ) with constants depending only on τ for τ (b) If v ∈ w τ (J ), then
0.
ρN (v) N −s v wτ (J ) again with a constant only depending on τ for τ
0.
Proof This proof can be found for example in [103].
Finally, the norm equivalences for wavelets give rise to a close connection of sequence spaces and smoothness spaces of functions.
Proposition 7.12 Let Ψ be a wavelet basis with sufficiently smooth functions ψλ ∈ H t (Ω) ∩ Brsn+t (Lr (Ω)) with d vanishing moments and u = uT Ψ ∈ H t (Ω). Then, u ∈ r (J ) if and only if u∈
Brsn+t (Lr (Ω)),
r=
1 +s 2
−1 ,
0<s<
d−t n
for Ω ⊂ Rn . Moreover, u ∈ w τ (J ) holds if and only if u ∈ Xτ which is a slightly larger space consisting of all functions u whose best N -term wavelet approximation decays like O(N −s ) in the energy norm.
Proof This proof can be found for example in [66].
C O M P R E S S I B L E M AT R I C E S
213
Now we are ready to analyze (7.22), the approximate adaptive operator application, one step further. First, we use a telescopic sum argument. N N N N N − Ak−1 v[1] − v[0] − v[k−1] − · · · − A0 v[k] Av N − wkN = Av N − Ak v[0] N N N N = A v N − v[k] + (A − A0 ) v[k] + · · · + (A − Ak )v[0] − v[k−1] . Next, we apply the triangle inequality and use the assumption that Aj is s∗ compressible in the sense of Definition 7.5. Then N N N
Av N − wkN 2 (J ) ≤ A v N − v[k]
2 (J ) + A − A0 v[k] − v[k−1]
2 (J ) N + · · · + A − Ak v[0]
2 (J ) N N N A v N − v[k]
2 (J ) + α0 v[k] − v[k−1]
2 (J ) N + · · · + αk 2−ks v[0]
2 (J ) .
The next step uses Proposition 7.5 (c) which implies N
v N − v[k]
2 (J ) = ρ2k (v N ) 2−ks v N wτ (J )
where the constant only depends on τ= as τ
s+
1 2
−1
0. Similarly, we obtain by using the triangle inequality N N
v[j] − v[j−1]
2 (J ) ≤ ρ2j (v N ) + ρ2(j−1) (v N ) 2−(j−1) v N wτ (J ) .
Finally, we use the simple inequalities N N
v[0]
2 (J ) ≤ v[0]
wτ (J ) ≤ v N wτ (J )
to continue N N N
Av N − wkN 2 (J ) A v N − v[k]
2 (J ) + α0 v[k] − v[k−1]
2 (J ) N + · · · + αk 2−ks v[0]
w2 (J )
2−ks A v N wτ (J ) + α0 2−(k−1)s v N wτ (J ) + · · · + αk−1 2−(k−1)s v N wτ (J ) + αk 2−ks v N wτ (J ) 2−ks v N wτ (J ) , (7.27)
214
A DA P T I V E WAV E L E T M E T H O D S
since the sequence (αj )j∈N ∈ 1 (N) is summable. Hence, we have proven the following estimate. Proposition 7.13 For wkN defined by (7.22) the following error estimate holds
Av N − wkN 2 (J ) 2−ks v N wτ (J ) for s < s∗ and s=
1 1 − , τ 2
with a constant only depending on τ as τ
0.
So far, we have not specified k in the above result. Now we choose k in order to guarantee a prescribed tolerance ε. In fact, given ε and v N ∈ w τ (J ), we choose k(ε) as the smallest integer that satisfies −1/s
2k(ε) ≥ ε−1/s v N w (J )
(7.28)
τ
with s and τ related as above by 1 1 =s+ . τ 2 Note that here k(ε) depends on the parameter s which is the unknown regularity of the solution. One could of course use a corresponding estimate. However, a refined analysis as in [66,67] allows us to avoid this dependence. Since the analysis is quite sophisticated, we do not go into details here. Choosing k in this way, we have N
2 (J ) 2−k(ε)s v N wτ (J ) ≤ ε,
Av N − wk(ε)
so that we have a guaranteed approximation. Now it of course remains to study the cost of such an algorithm. In order to do so, let us first investigate the number of nonzero coefficients in wkN . This can directly be estimated by the definition of wkN in (7.22) and the s∗ -compressibility of A. In fact, we have by (7.22) and recalling the number of N nonzero entries in v[k] # supp wkN ≤ # rows (Ak ) + # rows (Ak−1 ) + · · · + # rows (A0 ) = αk 2k + αk−1 2k−1 + · · · + α0 2k
C O M P R E S S I B L E M AT R I C E S
215
N by the geometric series and since α ∈ 1 (J ). This means that wk(ε) has 1/s
N 2k(ε) ε−1/s v N w (J ) # supp wk(ε) τ
nonzero entries since k(ε) was chosen as the smallest integer satisfying (7.28). Finally, let us count the number of arithmetic operators to compute wkN . Again from Definition 7.22, we get for the number Nk of arithmetic operations N Nk ≤ # rows(Ak ) · # supp v[0]
N N v[1] − v[0] N N + · · · + # rows(A0 ) · # supp v[k] − v[k−1]
+ # rows(Ak−1 ) · # supp
= αk 2k + αk−1 2k−1 + · · · + α0 2k−1 2k . Putting everything together, we have:
Theorem 7.14 Let A be s∗ -compressible and let some v N ∈ w τ (J ) be given, where 1 1 =s+ , τ 2
s < s∗ .
Then, we can compute for any ε > 0 an approximation Nε wεNε = wk(ε)
of Av N with the following properties: (a) Av N − wεNε 2 (J ) ≤ ε; (b) # supp wεNε ε−1/s v N wτ (J ) =: Nε ; (c) the approximation wεNε can be computed with O(Nε ) operations, i.e. with linear complexity of the number of output variables.
7.5.1
Numerical realization of APPLY
In principle, we now have a completely computable version of the adaptive approximate operator application. However, it turns out that we can improve
216
A DA P T I V E WAV E L E T M E T H O D S
the rough estimate of k(ε) in (7.28). In fact, recall (7.27) N N
Av N − wkN 2 (J ) ≤ A · v N − v[k]
2 (J ) + A − Ak · v[0]
2 (J )
+
k−1
N N − v[k−−1]
2 (J )
A − A · v[k−]
=0 N N ≤ CA v N − v[k]
2 (J ) + αk 2−ks v[0]
2 (J )
+
k−1
(7.29)
N N α 2−s v[k−] − v[k−−1]
2 (J )
=0
=: Rk , where CA is the constant appearing in (7.5). If we assume that CA is known (or, at least an upper bound for it), then Rk is computable. Hence, we can use Rk as an error estimator to determine k = k(ε) for a given accuracy ε > 0. This results in Algorithm 7.1. Algorithm 7.1 wN := APPLY [A, v N , ε] N := supp v N N N N Compute v[0] , v[j] − v[j−1] , j = 1, . . . , log N , N N set v[j] := v , j > log N . Compute k(ε) according to (7.28) for k = 1 to k(ε) compute Rk according to (7.29) if Rk ≤ ε EXIT end for k−1 N N N 9: wN := Ak v[0] + A v[k−] − v[k−−1] .
1: 2: 3: 4: 5: 6: 7: 8:
=0
7.5.2
Numerical experiments for APPLY
We now present some numerical results concerning the adaptive approximate operator application. These results are partly inspired by [9]. Again we consider our periodic model problem (3.1) and in particular we consider the part of the stiffness matrix corresponding to the diffusive part, i.e. A := (ψj,k , ψj ,k )0;Ω j,j ,k,k . (7.30) We choose cardinal B-spline wavelets with parameters d and d˜ from [71], see Section 5.3.3. Since this is a differential operator in H 1 , the diagonal matrix in (3.11) can be chosen as ωλ := 2|λ| .
C O M P R E S S I B L E M AT R I C E S
217
Later, in Section 8.2.8, we compare these results with homogeneous boundary conditions, i.e. (3.2). Compressibility of the stiffness matrix To be able to interpret the results, we have to identify first the range of asymptotic optimality permitted by the above choice of bases. Theorem 7.14 implies that the asymptotic behavior depends on the compressibility of the matrix A. In particular, the parameter s∗ in Definition 7.5 provides a range where optimality is guaranteed. Let us recall Theorem 7.7 which says that A is σ-compressible provided that (7.16) holds. Such an estimate was in turn guaranteed by Proposition 7.4, i.e. we obtain the range 0 ≤ σ ≤ min{τ, t − 1} where ψ ∈ H t,
t ≥ γ,
γ being the upper bound for which the norm equivalence (5.34) holds, and τ bounds the Sobolev regularity. For the above example and biorthogonal B-spline ˜ this means wavelets with parameters d and d, 1 γ =d− , 2
t = γ − 1,
τ = 0,
i.e. σ = 0, which is clearly useless. Of course, the conditions given in Theorem 7.7 and Proposition 7.4 are only sufficient. It turns out that we can improve these estimates by a refined analysis for B-spline biorthogonal wavelets. Lemma 7.15 Let A denote the stiffness matrix in (7.30) obtained by Bspline wavelets of order d as basis functions. Then for any ε > 0 the following compression estimate holds:
A − AJ 2−J(d−3/2−ε) ,
(7.31)
ı.e. A ∈ As for all s < d − 3/2. Proof Again we use the Schur lemma (Lemma 7.6) but this time directly. We want to use (7.19) for the sequence ωλ = 1 for all λ ∈ J . The first step is to estimate the entries in the stiffness matrix corresponding to (3.5),(3.6). Note that (by integration by parts) derivatives of wavelets still
218
A DA P T I V E WAV E L E T M E T H O D S
have vanishing moments, however, of one order less. This implies that we obtain for any polynomial Pλ ∈ Pd−2 (Ωλ ) on Ωλ of degree at most d − 2 and j = |λ | ≥ j = |λ| by using the vanishing moment property d d d d ψλ , ψλ ψλ − Pλ , ψλ = dx dx dx dx 0;Ω 0;Ω
d
d
ψλ ≤ ψλ − Pλ
dx 0;Ωλ dx 0;Ω
d
2j ,
dx ψλ − Pλ 0;Ωλ due to the L2 -normalization of the wavelets. Since d ψλ ∈ H s , s ≤ d − 3/2, dx the Whitney type estimate in Theorem 2.18 yields A A A A d d j −j (d−3/2−ε) A d ψλ , ψλ ψλ AA 2 2 A dx dx dx 0;Ω d−3/2−ε;Ω
(d−3/2−ε)
(d−3/2−ε) j(d−1/2−ε)
2j 2−j 2j 2−j 2(j−j
|ψλ |d−1/2−ε;Ω 2
)(d−3/2−ε) j+j
2
,
so that, taking the preconditioning matrix D into account, we get |aλ,λ | 2(j−j
)(d−3/2−ε)
,
j ≥ j.
(7.32)
,
j < j.
(7.33)
The case j < j can be treated analogously, |aλ,λ | 2(j
−j)(d−3/2−ε)
Moreover, the vanishing moment property of wavelets implies that the only nonvanishing entries aλ,λ correspond to the wavelets ψλ that intersect on spline knot ψλ . In fact, each ψλ is a piecewise polynomial of degree d − 1 so that a wavelet ψλ that is supported completely inside a polynomial piece produces a vanishing integral due to the vanishing moment property. Only if ψλ is supported in at least two polynomial pieces will there be a nonnegative entry. Sometimes, this is also called the singular support.
C O M P R E S S I B L E M AT R I C E S
219
It can easily be shown that the number of these nonvanishing entries does not depend on the refinement level. Consequently, we obtain |aλ,λ | 2−|j−j |(d−3/2−ε) . (7.34) |λ |=j
Next, we use the same truncation as in (7.20). In view of (7.19), we have to show that |aλ,λ | 2−J(m−3/2−ε) . (7.35) |j−j |>J |λ |=j
Let us again first consider the case j > j. By using (7.34), we obtain
j −j>J |λ |=j
|aλ,λ |
∞
2(j−j
)(d−3/2−ε)
j =j+J
2j(d−3/2−ε) 2−(J+j)(d−3/2−ε) 2−J(d−3/2−ε) .
(7.36)
The case j ≤ j can be treated analogously. The second condition in (7.19) can be checked in a similar fashion and hence (7.31) is established. Remark 7.16 It can be shown that the result of Lemma 7.15 is also valid for the case ε = 0, which was shown recently by Schwab and Stevenson [168]. In fact, it was proven there that the matrix is s∗ -compressible for s∗ = ∞. Moreover, the number of nonzero entries in each row and column of Aj is O(J) instead of O(2J ), which yields an enormous improvement of the error analysis concerning optimality. The latter theorem now gives an improved estimate for the decay of the entries in the stiffness matrix. We will use this in the subsequent numerical experiments. Numerical tests for APPLY Now, we present some numerical tests of the adaptive approximate operator application APPLY. Again, we use the periodic setting. In order to numerically test the scheme, we have used the data of Examples 7.1 and 7.3 (pages 197 and 198, respectively) since the knowledge of the exact right-hand side f and
220
A DA P T I V E WAV E L E T M E T H O D S
the corresponding solution of the model problem (3.1) allows us to compute the error of the approximate adaptive operator application exactly. The error estimate in Proposition 7.13 indicates that the approximation power of the adaptive approximate operator application is determined by the parameter s∗ which measures the maximal order of compressibility. In turn, according to (7.31), this parameter is given by s∗ = d − 3/2, where d again is the order of the scaling functions (B-splines). In Figure 7.11, the error
AvN − wjN 2 ˜ namely d = 2, is plotted in a logarithmic scale for different choices of d and d, d˜ = 4 and d = d˜ = 3, 4. We see a quite good matching with the theoretically predicted slopes. As we can see, the results do not match the asymptotic rate at the first few levels, in particular for d = 3, 4. Note that the periodic setting allows us to start with level j0 = 0. On this level, we have to use a scaling function basis which consists of only one constant function supported on the whole interval. Hence, we cannot hope to reach the asymptotic range at early stages. In Table 7.2, we collect the predicted rate of decay of the error and the observed ones. The latter ones are computed by linear regression and are also indicated in Figure 7.11. For d = 2 and d = 3, we see a very good matching. For d = 4, the asymptotic range is range is reached at j = 6. Since the regression takes all values into account the computed slope is less than the expected one. We also observe that the results are basically identical for the two examples, i.e. independent of the vector APPLY acts on. Moreover, we obtained similar results also for other data. The APPLY scheme is, meanwhile, realized in general terms, i.e. for wavelet bases on general domains that will be described in the next chapter. All numerical results that we are aware of undermine the good performance of the method, see also Section 8.2.8. Table 7.2 Expected and observed slopes for the error of the adaptive approximate operator application in the periodic case for Example 7.1 (left) and 7.3 (right). d
s∗
s
ratio
s
ratio
2 3 4
0.5 1.5 2.5
0.51 1.52 2.16
1.02 1.01 0.86
0.51 1.52 2.15
1.02 1.01 0.86
A P P ROX I M AT E I T E R AT I O N S 101
10–1
100
100
–1
10
10–1
–2
10
10–2
10–3
10–3
10–4 10–5
221
d = 2, dt = 4 d = dt = 3 d = dt = 4
1
2
3
d = 2, dt = 4 d = dt = 3 d = dt = 4
10–4 4
5
6
7
8
10–5
1
2
3
4
5
6
7
8
Fig. 7.11 The slope of error reduction in the adaptive approximate operator application APPLY using the data of the periodic boundary value problem for Example 7.1 (left) and 7.3 (right). 7.6
Approximate iterations
Coming back to Section 7.3, we are now ready to formulate our first adaptive algorithm for solving elliptic boundary value problems using the framework of approximate infinite-dimensional iterations. 7.6.1
Adaptive Wavelet-Richardson method
To this end, we choose a Richardson-type method, i.e. R(n) = αI,
α ∈ R+ ,
in (7.11). Moreover, we assume that the wavelet expansion of the right-hand side f is known to us. Finally, we replace AuN by APPLY [A, uN , ε]. Then, we obtain the scheme in Algorithm 7.2.
Algorithm 7.2 Adaptive Wavelet-Richardson scheme Given a finite initial guess u(0) ∈ 2 (J ) and a sequence of tolerances (δn )n≥0 , δn > 0, set u(n+1) := u(n) − α APPLY [A, u(n) , δn ] + αfδn ,
(7.37)
where fε is a finite approximation of f such that f −fε 2 (J ) ≤ ε.
Analysis of the adaptive Wavelet-Richardson method Let us investigate this method. By the triangle inequality, it is easily seen that ¯ (n+1) 2 (J ) + u ¯ (n+1) − u 2 (J ) ,
u(n+1) − u 2 (J ) ≤ u(n+1) − u
222
A DA P T I V E WAV E L E T M E T H O D S
¯ (n+1) denotes the exact iteration, namely where u ¯ (n+1) := (I − αA)u ¯ (n) + αf , u
¯ (0) := u(0) . u
(7.38)
Then, ¯ (n+1) − u 2 (J ) ≤ I − αA n+1 u(0) − u 2 (J )
u by standard estimates similar to Section 7.3. For the first term, we get C B ¯ (n) ¯ (n+1) = u(n) − u ¯ (n) − α APPLY A, u(n) , δn − Au u(n+1) − u + α(fδn − f )
C B ¯ (n) − α APPLY A, u(n) , δn − Au(n) = u(n) − u ¯ (n) − u(n) + α(fδn − f ) + αA u C B ¯ (n) + α Au(n) − APPLY A, u(n) , δn = (I − αA) u(n) − u + α(fδn − f ). Next, by using the error estimate for APPLY and for fε ¯ (n+1) 2 (J ) ≤ I − αA u(n) − u ¯ (n) 2 (J ) + 2αδn
u(n+1) − u ¯ (n−1) 2 (J ) ≤ I − αA ( I − αA u(n−1) − u + 2αδn−1 ) + 2αδn ¯ (0) 2 (J ) ≤ . . . ≤ I − αA n+1 u(0) − u + 2α
n
δ I − αA n−
=0
= 2α
n
δ I − αA n−
=0
¯ (0) . Of course, we have to assume that α ∈ R+ is chosen in such a since u(0) = u way that the abstract iteration converges, i.e. similar to (7.12) ρ := I − αA < 1. At this point we have to assume that the sequence of tolerances δ is chosen in such a way that η := δ ρ−
A P P ROX I M AT E I T E R AT I O N S
223
satisfies η := (η )∈N0 ∈ 1 (N0 ). Then, we have for the overall error of Algorithm 7.2
u(n+1) − u 2 (J ) ≤ 2α
n
δ ρn− + ρn+1 u(0) − u 2 (J )
=0
= ρn
2α
n
δ ρ− + ρ u(0) − u 2 (J )
=0
≤ ρn
2α
∞
δ ρ− + ρ u(0) − u 2 (J )
=0
= ρ (2α η 1 (N) + ρ u(0) − u 2 (J ) ) n
→0
(n → ∞).
This analysis in particular shows that δ has to be chosen small enough. Relaxing the above requirement to δ ≤ ρ also yields convergence since then
u(n+1) − u 2 (J ) ≤ ρn
2α
n
δ ρ− + ρ u(0) − u 2 (J )
=0
≤ ρ (2α(n + 1) + ρ u(0) − u 2 (J ) ) n
→0
(n → ∞).
Numerical experiments Besides the qualitative investigation of the convergence, there is also a quantitative aspect. Of course, several constants are involved in the above analysis and some of these constants are not really known and/or are hard to estimate. Thus, in order to investigate how (good) this method is on a quantitative level, let us perform some numerical tests. We use the data of Examples 7.1 and 7.3 (pages 197 and 198, respectively). In the following graphs, we display the results of numerical experiments for α = 0.5 and u(0) = 0. The sequence δn is chosen as δn = 21−n . The results are shown in Figure 7.12. We see that the scheme converges in all cases, but the convergence speed is not very high. In order to verify this,
224
A DA P T I V E WAV E L E T M E T H O D S 101
101 100
100
10–1 10–1
10–2 10–3
10–2
10–4
10–3 0
200 400 600 800 1000 1200
10–5
0
200 400 600 800 1000 1200
Fig. 7.12 Convergence history for the adaptive Wavelet-Richardson scheme in Algorithm 7.2 for Examples 7.1 (left) and 7.3 (right). We denote d = 2, d˜= 4 by “+”, d = d˜= 3 by “◦” and d = d˜= 4 by “∗”. The solid line in the picture on the right shows the error of the best N -term approximation for d = 2, d˜= 4. we have included in the right part of Figure 7.12 the slope for the best N -term approximation for d = 2, d˜ = 4 (solid line). The distance between the Richardson scheme for the same values of d and d˜ (indicated by “+”) is quite large. Even the higher order wavelet schemes are above that line. Note that the best N -term approximation for lager values of d is well below the solid line shown. We come back to this point later. Basically we see two effects: • The scheme increases the number of active wavelets but the residual is not reduced sufficiently. • The slope is far off the slope of the best N -term approximation. 7.6.2
Adaptive scheme with inner iteration
As expected this simple method does not converge very fast. The reason is the fast growth of the number of degrees of freedom. In fact, APPLY enlarges the set of active indices. On the other hand, the iteration uΛ may still be far away from the best approximation in the linear space SΛ . The reason is that we do not reduce this “inner” residual, e.g. by an iterative scheme. As a first way-out, one could improve each approximation u(n) by using an iterative scheme on Λ(n) := supp u(n) , i.e. which reduces the residual rΛ = fΛ −AΛ uΛ . One could think of a Richardson or a conjugate gradient scheme. As an example, we detail the Richardson scheme with an interior Richardson iteration in Algorithm 7.3. Of course, lines 4–6 can be replaced by any kind of convergent method for symmetric positive definite matrices. Again, we perform some numerical tests for this scheme.
A P P ROX I M AT E I T E R AT I O N S 101 10
0
10–1 10–2 10–3 10–4 10–5 10–6 10–7
0
200
400
600
800 1000 1200
101 100 10–1 10–2 10–3 10–4 10–5 10–6 10–7 10–8 0
200
400
225
600
800 1000 1200
Fig. 7.13 Convergence history for the adaptive Wavelet-Richardson scheme with inner iteration in Algorithm 7.3 for Examples 7.1 (left) and 7.3 (right). We denote d = 2, d˜ = 4 by “+”, d = d˜ = 3 by “◦” and d = d˜ = 4 by “∗”. Algorithm 7.3 Adaptive Wavelet-Richardson scheme with inner Richardson iteration Given a finite initial guess u(0) ∈ 2 (J ), a sequence of tolerances (δn )n≥0 , δn > 0, and approximations fε of f such that f − fε 2 (J ) ≤ ε. Fix some K ∈ N. 1: for n = 0, 1, 2, . . . do 2: u(n+1,0) := u(n) − α APPLY [A, u(n) , δn ] + αf δn 3: Λ := supp u(n+1) 4: for k := 0 to K do 5: u(n+1,k+1) := u(n+1,k) −α APPLY [AΛ , u(n+1,k) , δn ]+αfδn 6: end for 7: u(n+1,k+1) := u(n+1,k) , 8: end for Numerical experiments Again, we report on numerical experiments concerning Algorithm 7.3. The convergence histories are shown in Figure 7.13. The left picture concerns Example 7.1 (page 197). We indicate the slopes for d = d˜ = 3, 4 and the best N -term approximation for d = d˜ = 3. Comparing the numbers, we see that this scheme converges faster than without the inner iteration. However, it is still far off the best N -term approximation. The situation is slightly different in the right part of the figure which shows the data concerning Example 7.3 (page 198). Recall that the corresponding functions are sin-functions and thus are quite regular without strong variations. In this case, we show also the slopes of the best N -term approximation for all three ˜ For the cases d = 2, d˜ = 4 and d = d˜ = 3 the slope is already cases of d and d. relatively close to the best N -term approximation, for d = d˜ = 4 the difference is still large.
226
A DA P T I V E WAV E L E T M E T H O D S
We conclude that the scheme is still not optimal in case of functions with steep gradients like in Example 7.1 and that it does not realize expected higher convergence rates in the case of high order discretizations like for d = 4 here. One option is to replace the inner Richardson iteration, e.g. by a conjugate gradient scheme. We will come back to this point later. 7.6.3
Optimality
As we can see by the above numerical experiments, these approximate infinitedimensional iterations converge but the number of degrees of freedom increases too fast. However, what does “too” fast mean? What could be a benchmark for an adaptive method? Obviously, the best N -term approximation is “the” optimal method. Thus, we call an adaptive method (asymptotically) optimal if the convergence rate is of the same order as the best N -term approximation, i.e.
u − uΛ 1,Ω ∼ ρ#Λ (u) for our model problem in H 1 (Ω). This is the reason why we have compared the two methods (adaptive Richardson with and without inner Richardson iteration) with the best N -term approximation in the above figures. We have seen that the above methods are not optimal. What is the reason for the nonoptimality of these schemes? In order to explain this, we have shown in Figure 7.14 the history of iterations where we have sorted the corresponding wavelet coefficients by their absolute values. The data correspond to the adaptive Richardson scheme for Example 7.1 using d = d˜ = 3. We see that the approximate solutions contain many coefficients which are quite small and – due to the norm equivalences – which do not contribute to uΛ and thus do not increase the accuracy much. However, their presence increases the number of degrees of freedom which results in the graphs in Figure 7.14. In order to optimize the balance of degrees of freedom and accuracy we have to get rid of as many small coefficients as possible without destroying the accuracy too much. At this point, we can benefit from the compression properties of wavelets. This allows a thresholding strategy that can be described in Algorithm 7.4. Then, it is not difficult to show the following statement concerning the properties of THRESH. Proposition 7.17 The output w = THRESH [v, η] of Algorithm 7.4 satisfies
w − v 2 (J) ≤ η, and it is the shortest vector having that property.
(7.39)
A P P ROX I M AT E I T E R AT I O N S
227
1
10
100
0
10
10–0.1
10
–1
10
–2
10
–3
10–4 10
0.2
–5
1
1.5
2
2.5
3
3.5
4
4.5
5
10 0
101
102
100
10
20
30
40
50
60
70
80
90
0
10–1 10
10
10–2
–2
10–4
10–3
–6
10
10–4
10–8
10–5 10
–6
10
–7
10–10 0
50
100
150
200
250 10
300
350
400
10–12
0 100 200 300 400 500 600 708 300 900 1000
2
100 10
–2
10–4 10–6 10
–8
–10
10
10–12 10–14
0
200
400
600
800
1000
1200
Fig. 7.14 Coefficients of the iterations 1, 5, 10, 15, 20, sorted with respect to their absolute values.
Proof Exercise 7.7.
Now, we incorporate this thresholding into the Richardson iteration in Algorithm 7.5. Let us comment on the various parameters that are involved: • η: desired accuracy; • δ: estimate for the initial error
uΛ − u(0) 2 (Λ) ≤ δ
228
A DA P T I V E WAV E L E T M E T H O D S
Algorithm 7.4 Thresholding: THRESH Given a vector v ∈ 2 (J ) and an accuracy η > 0, compute w = THRESH [v, η] as follows 1: Sort the entries vλ according to their size leading to v ∗ = (|vλ1 |, |vλ2 |, . . .) 2: sum := 0, compute v 22 (J ) 3: for k = 1, 2, . . . do 4: sum = sum +|vλk |2 ; 5: if sum ≥ v 22 (J ) − η 2 6: EXIT 7: end if 8: Λ := supp w := {λi }i=1,...,k , wλ := vλ , λ ∈ Λ. 9: end for Algorithm 7.5 RICHRADSON: Adaptive Wavelet-Richardson with thresholding Given an index set Λ, an initial guess u(0) ∈ 2 (J ) and real parameters η, δ, θ, τ, tol , α to be explained later. ¯ τ, tol , α] ¯ Λ = RICHARDSON [Λ, A, u(0) , η, δ, θ, u ¯ 1: v (1) := u(0) ; Kmax := !| log ηδ |\| log θ|" 2: for k = 1, . . . , Kmax do 3: w(k) := APPLY [AΛ , v (k) , τ ]; 4: g (k) := THRESH [fΛ , τ ]; 5: r (k) := w(k) − g (k) ; 6: r (k) := THRESH [r (k) , τ ]; 7: if r (k) 2 ≤ tol uΛ := v (k) then 8: EXIT 9: end if 10: v (k+1) := v (k) − αr (k) 11: τ := τ /2 12: end for
• • • •
with respect to the exact solution uΛ of the discrete problem AΛ uΛ = fΛ ; ¯ reduction parameter; θ: τ : accuracy for approximate operator application; tol: stoping tolerance; α: Richardson damping parameter.
Next, we fix some of these parameters in order to prove the corresponding properties for RICHARDSON in Algorithm 7.5. In order to do so, we need to specify
A P P ROX I M AT E I T E R AT I O N S
229
some of the constants that arise in the problem formulation. We recall the norm equivalence between Au 2 (J ) and u 2 (J ) . This also implies
Au 22 (J ) ∼ uT Au =: u 2A , so that we have the following equivalence cA u 22 (J ) ≤ u 2A ≤ CA u 22 (J ) . Then, we need an estimate κ for the condition number of A, i.e. κ ≥ A · A−1 =
CA . cA
Then, we fix the parameters α :=
1 , CA
1 . θ¯ := 1 − 6κ
(7.40)
Then, one can prove the following result. Proposition 7.18 Assume that the initial value u(0) satisfies
uΛ − u(0) 2 ≤ δ for the exact solution uΛ of AΛ uΛ = fΛ . Then, Algorithm 7.5 choosing ¯ cA η, 2 cA η, α ¯ Λ = RICHARDSON Λ, A, u(0) , η, δ, θ, u 6 3 with α and θ¯ according to (7.40) satisfies the error estimate ¯ Λ 2 ≤ η.
uΛ − u In this case, we abbreviate ¯ Λ = RICHARDSON [Λ, A, u(0) , η, δ]. u
Proof See [66–68].
(7.41)
Now, we have all the algorithmic ingredients to formulate the full adaptive wavelet algorithm for linear problems in Algorithm 7.6 on page 230. At first glance, it may look somewhat complicated. In [66] it was presented in terms of several subroutines. We wanted to present the whole algorithm at one view. Again, we do not describe all technical details of the corresponding analysis, but just quote the result from [66–68].
230
A DA P T I V E WAV E L E T M E T H O D S
Algorithm 7.6 Adaptive wavelet method for linear problems Fix the parameters γ, F, q1 , . . . , q4 , θ and κ according to (i) γ ∈ [0, 1] (ii) F ≥ f 2 1 +q2 +q3 ) (iii) Cq3A + 2(1+γ)(q ≤ q4 γcA √ q3 1 q4 κ + CA ≤ 10 cA 1 q1 + q√ 2 + q3 (1 + κ ) ≤ 20 q3 (iv) q0 := κ + CA B C 20κ (v) θ := 1 − γ 2 CcAA , K := log | log θ| + 1 and choose α and θ¯ according to (7.40). Choose a tolerance ε > 0. E D δ (1) (1) v := 0; δ := F ; M := log √ , Λ(1) := ∅ cA ε 1: for m = 1 to M do √ 2: if δ (m) ≤ cA ε then 3: EXIT 4: end if C B (m) ¯ Λ = RICHARDSON Λ(m) , A, v (m) , δ (m) , Cq3A δ (m) ; 5: u (m,1)
(m)
¯Λ ¯Λ ; 6: Λ(m,1) := Λ(m) ; u := u 7: for k = 1 to K do (m,k) , q1 δ]; 8: w(k) := APPLY [A, uΛ (k) 9: g := THRESH [f , q2 δ]; 10: r (m,k) := w(k) − g (k) ; / 11: c(k) := THRESH [r (m,k) , 1 − γ 2 r (m,k) 2 ]; 12: Λ(m,k+1) := Λ(m,k) ∪ supp c(k) ; A δ then 13: if r (m,k) 2 ≤ c20 (m+1) ¯ (m,k) , 25 δ]; := THRESH [u 14: u 15: r (m+1) := r (m,k) ; ¯ (m+1) 16: Λ(m+1) := supp u 17: EXIT 18: end if (m,k) ¯ (m,k+1) := RICHARDSON [Λ(m,k+1) , uΛ , q0 δ, Cq3A δ] 19: u 20: end for 21: if r (m+1) 2 + (q1 + q2 + (1 + κ−1 )q3 )δ ≤ cA ε then ¯ 22: u(ε) := u(m+1) , 23: Λ(ε) := Λ(m+1) 24: EXIT 25: end if 26: δ (m+1) := 12 δ (m) 27: end for
A P P ROX I M AT E I T E R AT I O N S
231
Theorem 7.19 Algorithm 7.6 always terminates after a finite number of steps. Moreover if A is s∗ -compressible and ρN (u) N −s ,
s < s∗ ,
for the exact solution and some s < s∗ , then Algorithm 7.6 produces a (m) ¯ Λ such that sequence u
u − u(m) (#Λ(m) )−s (m)
−1 ¯ Λ )T DΛ for u(m) defined by u(m) = (u ΨΛ . The algorithm is of linear complexity.
Proof [66–68]
Note that the above version of the adaptive wavelet method basically corresponds to the original version published in [66–68]. After that a series of modifications and refinements have been introduced. For later reference, we can view Algorithm 7.6 also in the form of Algorithm 7.7.
Algorithm 7.7 ELLSOLVE: Adaptive wavelet solver for linear elliptic problems Given a well-conditioned operator A, a right-hand side f and a tolerance ε > 0, one can determine the parameters in Algorithm 7.6 in such a way (m) ¯ Λ = uε satisfies that the output u
Auε − f ≤ ε. We abbreviate this by uε = ELLSOLV E[A, f , ε].
Numerical experiments Now, we present results of some of our numerical experiments. We report on data concerning Example 7.1. In Figure 7.15 we show the convergence history for the two choices d = d˜ = 3, 4 and also compare these with the corresponding best N -term approximations. We see that now the slope is quite close to the slope of the error of the best N -term approximation which underlines the above optimality results.
232
A DA P T I V E WAV E L E T M E T H O D S 101 100 10–1 10–2 10–3 10–4 10–5 10–6
0
50
100
150
200
250
300
Fig. 7.15 Convergence history of Algorithm 7.7 for Example 7.1 using biorthogonal B-spline wavelets corresponding to d = d˜ = 3. In order to demonstrate the performance of the scheme, we show some iterations along with the corresponding errors in Figure 7.16. 7.7
Quantitative efficiency
We have introduced a rigorous analysis of the above adaptive wavelet method which allows quite strong results such as convergence and optimality. The price to pay for this theoretical foundation is the sophistication of the method. However, once a sophisticated method is implemented, the sophistication itself is no drawback. Another aspect, however, turned out to be more crucial. There are several constants and parameters that have to be chosen by the user. It turned out that the performance of the scheme sometimes crucially depends on the choice of these parameters. 7.7.1
Quantitative aspects of the efficiency
The above-mentioned arguments were the main reasons why several researchers investigated quantitative improvements of the original adaptive scheme. A first issue is the conditioning of the wavelet bases. We have seen that upper and lower constants of the stability estimates in fact enter the scheme. Thus, it seems natural to improve the condition number of the wavelet bases. This has already been described above. A second point is the algorithm itself. It has originally been designed in order to prove convergence and optimality. The main focus was not computational efficiency. Hence, it is not surprising that the performance of the algorithm can be improved even without destroying the analytic results. There have been many investigations in this direction and we do not aim at giving a comprehensive overview, but just mention several papers by Stevenson and coauthors, see [110,111,126,172–174]. One observation is that the thresholding (coarsening)
discrete solution (adaptive), N = 5
0.8
0.2
0.6
0.1
0.5 0.4
0
0.3 0.2
–0.1
0.1
–0.2
0 –0.1 1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
difference of exact/discrete solution (adaptive), N =5
0.3
0.7
0
0.2 0.4 0.6 0.8 discrete solution (adaptive), N = 9
1
–0.3
0 0.2 0.4 0.6 0.8 1 difference of exact/discrete solution (adaptive), N = 9 0.02
0.015 0.01 0.005 0 –0.1 –0.2 0
1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
0.2
0.4
0.6
0.8
1
–0.3
0
discrete solution (adaptive), N = 15
0.2
0.4
0.6
0.8
1
difference of exact/discrete solution (adaptive), N = 15 0.008 0.006 0.004 0.002 0 –000.2
0
0.2
0.4
0.6
0.8
1
discrete solution (adaptive), N = 43
–0.004
0 0.2 0.4 0.6 0.8 1 difference of exact/discrete solution (adaptive), N = 43 0.0021
0.00205 0.002 0.00195 0.0019 0.00185 0.0018 0.00175 0
1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0
0.2
0.4
0.6
0.8
1
discrete solution (adaptive), N = 775
0.0017
0 0.2 0.4 0.6 0.8 1 difference of exact/discrete solution (adaptive), N = 775 0.00196 0.00195 0.00194 0.00193 0.00192 0.00191 0.0019
0.2
0.4
0.6
0.8
1
0.00189
0
0.2
0.4
0.6
0.8
1
Fig. 7.16 Numerical approximations (left) and corresponding errors (right) for the adaptive scheme in Algorithm 7.7 for Example 7.1 using biorthogonal B-spline wavelets corresponding to d = d˜= 3. We show the iterations n = 1, 3, 5, 10, 30.
234
A DA P T I V E WAV E L E T M E T H O D S 102 before THRESH after THRESH 100
10–2
10–4
10–6
10–8
10–10 0
2000
4000
6000
8000
10000
12000
Fig. 7.17 Convergence history before and after the thresholding step. is sometimes not necessary. In order to illustrate this, we have performed numerical tests for the above-mentioned example and have depicted in Figure 7.17 the convergence history before and after thresholding. We see that the slope of both curves coincides, which indicates that the rate of convergence is the same for both cases. However, even though the coarsening reduces the number of active wavelets, also the error increases significantly. We would like to point out that several scientists are still concerned with the improvement of the original scheme and there still seems to be room for improvements. It should be stressed that all the new schemes in the above cited papers are also proven to converge and to be optimal. There is no loss of theory. 7.7.2
An efficient modified scheme: Ad hoc strategy revisited
A second path is motivated by our above observations in connection with the ad hoc refinement strategy. This relatively simple strategy worked astonishingly well. On the other hand, so far there is no theoretical foundation for this approach. However, it turned out that one can extend this ad hoc strategy to a quite efficient adaptive wavelet scheme, still without theory. We introduce here a method by Berrone and Kozubek [19] which is relatively easy to describe and realize and gives good results. The main idea is quite close to the ad hoc adaptive refinement scheme and consists of the following steps for a given set of indices Λ which can be interpreted as active wavelets.
Q U A N T I TAT I V E E F F I C I E N C Y
235
1. Determine a “security zone” similar to the index cone defined in (7.5). 2. Test the residual corresponding to Λ for all wavelets in the security zone. The large ones will be activated, i.e. they will be added to Λ. 3. Consider the linear system of equations on the extended index set and use an iterative solver to reduce the residual. 4. Perform a thresholding on the determined approximate solution. Compared to the above adaptive scheme, there are two main differences, namely: • The routine APPLY is replaced by the extension of the active wavelets. • An inner iteration reduces the error. Now, we will describe the concrete ingredients. Security zone Recall the definition of the index cone in (7.5) which was defined by all wavelets on the next higher level that intersect one active wavelet. We will slightly generalize this definition here by introducing a contraction factor c ∈ (0, 1]. We define the contracted support of a wavelet ψλ as c · supp ψλ := [cλ + (1 − c)zλ , cuλ + (1 − c)zλ ], where supp ψλ =: [λ , uλ ] and zλ is the barycenter of supp ψλ . Then, we define the generalized index cone as C(λ; c) := {µ ∈ J|λ|+1 : |supp ψµ ∩ c · supp ψλ | > 0}, where obviously we have C(λ; 1) = C(λ). Accordingly, we set C(Λ; c) :=
C(λ; c).
λ∈Λ
Residual testing Let us denote by Λcheck = Λ ∪ C(Λ; c)
(7.42)
236
A DA P T I V E WAV E L E T M E T H O D S
the index set on which we test the residual. To describe this, for A = (aλ,µ )λ,µ∈J
and f = (fλ )λ∈J ,
we define the finite sections as AΛcheck ×Λ := (aλ,µ )λ∈Λcheck ,µ∈Λ ,
and fΛcheck := (fλ )λ∈Λcheck .
If we consider an approximate solution uΛ on Λ, we compute the extended residual as check := fΛcheck − AΛcheck ×Λ uΛ . rΛ
This extended residual indicates if the current approximation uΛ is sufficiently accurate also on the extended index set. In order to improve the efficiency of the method, the set Λcheck was compressed in [19] by a thresholding step after the enlargement and before the testing of the residual. The arising set Λactivable was there called the set of “activable wavelets”. Iterative solver on Λ As already explained in connection with the APPLY routines it turns out to make sense to reduce the error on the current index set Λ before proceeding to a next (usually finer) index set. As suggested in [19], we do not specify which iterative solver we use since this leaves some additional freedom. For elliptic boundary value problems the stiffness matrix is symmetric and positive definite. Hence, we can use steepest descent, conjugate gradient, or any other appropriate solver. If we nonsymmetric matrices (e.g. for convection–diffusion problems), one could also use BiCG or GMRES here. The only thing that we require is a certain stopping criterion, namely stop initial
rΛ
2 ≤ toliter rΛ
.
We formulate the core algorithm in Algorithm 7.8. Complete adaptive solver Now we can put the above pieces together and formulate the scheme in Algorithm 7.9. We use the same stopping criterion as in (7.7), namely
rn 22 + sn 22 ≤ ε fΛcheck
22 + fΛn 22 . n
Q U A N T I TAT I V E E F F I C I E N C Y
237
Algorithm 7.8 General iterative solver [uΛ , rΛ ] = ITERATE[Λ, uinit Λ , toliter ] 1: 2: 3: 4: 5: 6: 7: 8: 9: 10:
v (0) := uinit Λ , k := 0 r (0) := fΛ − AΛ×Λ v (0) repeat k →k+1 iterate on AΛ×Λ and fΛ update v (k) → v (k+1) r (k+1) := fΛ − AΛ×Λ v (k+1) (k+1) (0) until rΛ
2 ≤ toliter rΛ
(k+1) uΛ := v rΛ := r (k+1)
Let us recall the idea for the choice of this criterion. The security zone is created in such a way that is catches the bulk of the full residuum
f − AJ ×Λn u(n) 2 in the security zone (which is disjoint from the current active set of indices Λn ). Hence, a good approximation of the overall error should be obtained if we add the residual s(n) on the active set, i.e. s(n) := fΛn − AΛn ×Λn uΛn , so that we have
f − AJ ×Λn u(n) 22 ≈ r (n) 22 + s(n) 22 . The same idea holds true with the right-hand side, namely
22 + fΛn 22
f 22 ≈ fΛcheck n so that the chosen stopping criterion ensures an approximation of the relative error criterion
f − AJ ×Λn u(n) 2 ≤ ε.
f 2 This is realized in line 17 of Algorithm 7.9. We will present some performance tests of Algorithm 7.9 in Section 9.2 below where we consider more complicated domains, which are more useful for such comparison purposes.
238
A DA P T I V E WAV E L E T M E T H O D S
Algorithm 7.9 Simplified adaptive wavelet solver [u(ε), Λ(ε)] = S-ADWAV-SOLVE[ε] 1: 2: 3: 4: 5:
Λ0 = ∅, ∂Λ0 = I0 , n = 0 = Λ0 ∪ ∂Λ0 Λcheck 0 u(0) := 0 for n = 1, 2, 3, . . . do for λ ∈ Λcheck n−1 do
6: 7:
if λ ∈ Λn−1 then (n) (n−1) vλ := uλ
8: 9:
else (n) vλ := 0
10: 11: 12: 13: 14: 15: 16: 17: 18:
end if end for (n) , toliter ] [w(n) , s(n) ] = ITERATE[Λcheck n−1 , v (n) (n) u = THRESH[w , tolthresh ] {Alg. 7.4, page 228 } Λn = supp u(n) ∂Λn = C(Λn , c), Λcheck = Λn ∪ ∂Λn n (n) r (n) := fΛcheck − AΛcheck ×Λn u n if r (n) 22 + s(n) 22 ≤ ε fΛcheck
22 + fΛn 22 then n u(ε) := u(n) , Λ(ε) := Λn
19: EXIT 20: end if 21: end for
7.8
Nonlinear problems
We have seen that the above analysis of adaptive methods for linear elliptic problems is sophisticated and yields deep insight into the numerical schemes. However, the framework is still somewhat academic. Hence, we now consider nonlinear problems. As already pointed out in the Preface, adaptive wavelet methods for nonlinear problems are still a very active field of research. Several variants and improvements of methods are still introduced, analyzed, and tested. Hence, we cannot give an ultimate description of this topic in this book. Moreover, most of the recent methods are highly sophisticated and technically challenging. This makes a correct description on a textbook-level basically impossible. Being aware of these facts, we decided to give an overview of the basic techniques that are needed for an efficient evaluation of nonlinear terms. This section is based upon [10, 36, 93]. For further investigations we refer to [8, 68, 69], being aware that this list is not complete.
N O N L I N E A R P RO B L E M S
7.8.1
239
Nonlinear variational problems
In order to be able to understand the specific requirements for adaptive wavelet methods, we review the theory of Wavelet-Galerkin methods for nonlinear variational problems. Consider the nonlinear boundary value problem −∆u = F(u), in Ω, u = 0,
on ∂Ω,
with a given (nonlinear) operator F : H01 (Ω) → H −1 (Ω). The variational formulation amounts finding u ∈ H01 (Ω) such that for all v ∈ H01 (Ω),
a(u, v) = F(u), v where the bilinear form is given by
a(u, v) := (∇ u, ∇ v)0;Ω , and ·, · denotes the standard duality pairing. Any numerical scheme based on some kind of discretization has to evaluate the nonlinearity F(u) at least for a discrete input u. In order to do so, it is maybe most convenient to use a collocation method so that F(u) can be computed using point values, see e.g. [138, 167]. On the other hand, however, collocation methods do not allow us to benefit from the analytical background of Hilbert space theory as it is possible for Galerkin methods. Using a Galerkin method, however, amounts to computing integrals of nonlinear functions as we will now see. In fact, using a Galerkin method based upon a wavelet basis Ψ of H01 (Ω) requires the computation (or at least, the approximation) of terms like (∇uε , ∇ψµ )0;Ω
and F(uε ), ψµ
for a given (finite) approximation uε . Here, finite approximation means that uε is a linear combination of finitely many basis functions, i.e. uε =
uελ ψλ ,
|Λ| < ∞.
λ∈Λ
We have already seen that a sufficiently good finite approximation of the linear part (a(uε , ψµ ))µ∈J
240
A DA P T I V E WAV E L E T M E T H O D S
can be computed with cost O(#Λ), if the entries a(ψλ , ψµ ) of the stiffness matrix can be computed at unit cost. This is the case if the wavelets ψλ are spline functions of compact support (cf. [66]). Then, the APPLY scheme in Algorithm 7.1 offers an approximate adaptive application of the linear part of the given nonlinear problem. This situation changes for the nonlinear terms F(uε ), ψµ as can easily be seen by the simple example F(u) := u2 . For a finite input uε =
uελ ψλ ,
λ∈Λ
we obtain F(uε ), ψµ =
uελ uεν ψλ ψν , ψµ ,
λ∈Λ ν∈Λ
which requires O (#Λ)2 -operations (thus nonoptimal) if one computes these values in a straightforward way. Moreover, the terms ψλ ψν , ψµ are integrals of wavelets on fairly different levels. Such integrals can be computed with an amount of work that is linear in the level difference of the involved functions (see, e.g. [23]), which again is nonoptimal. Hence, one is interested in finding a numerical scheme that is able to approximate F(uε ), ψµ up to any desired accuracy with (asymptotically) optimal complexity. In the sequel, we will consider the more general setting, that F is a mapping from H t (Ω) to H −t (Ω) F : H t (Ω) → H −t (Ω) and we have to find an approximation for the coefficients F(u), ψµ . The problem is solved if we have a suitable approximation g of F(u) with respect to the H −t (Ω)-norm, so that the coefficients dλ = g, ψλ can be computed rapidly. In particular, we want to choose g such that only a finite (possibly small) number of coefficients dλ does not vanish, so that only these coefficients have to be computed. From the Riesz stability we know that the coefficients dλ are a good approximation of F(u), ψλ . In the sequel, we will always assume that F satisfies the following straightforward assumptions.
N O N L I N E A R P RO B L E M S
241
Assumption 7.20 We pose the following assumptions. 1. F is a Lipschitz map, i.e.
F(u) − F(v) −t;Ω ≤ LF (max{ u t;Ω , v t;Ω }) u − v t;Ω ,
(7.43)
where LF : R+ → R+ is a nondecreasing function. 2. F is local and preserves smoothness in the sense that for any U ⊂ Ω s (L (U )) ≤ CF ( u t;U ) u B r (L (U )) ,
F(u) B∞ ∞ ∞ ∞
r − s ≤ 2t, (7.44)
with a nondecreasing function CF : R+ → R+ . 3. If we are able to evaluate any function value of u or its derivatives at fixed cost, then (F(u))(x) can be computed with a fixed number of computations for any x in Ω.
For details of Besov spaces, we refer to Appendix B. These assumptions can be verified, e.g. for many differential operators of the form (F(u))(x) = F (D1α u, . . . , Dnα u),
(7.45)
where F : Rm → R satisfies suitable smoothness conditions (see, e.g. [68, 166]). In particular, for many choices of F (e.g. F (t) = tM +1 ) one obtains CF ( u t;Ω ) = c˜F u M t;Ω ,
(7.46)
where M depends only on F , while c˜F depends also on the chosen Besov norms. 7.8.2
The DSX algorithm
As already said at the beginning of this section, our description here is based on the work in [36,68,93]. The analysis and in particular the realization of adaptive wavelet methods for nonlinear problems is still an active field of research, see e.g. [8, 10, 69]. Hence, we choose to describe only one particular method in this book, being completely aware of the fact that possibly more efficient or more sophisticated procedures may be available in the meanwhile. The main purpose here is to describe one possible way to treat such nonlinear function evaluation. This approach also enables us to describe the main techniques that are used also in more recent papers. We have decided on an algorithm introduced by Dahmen, Schneider and Xu in [93] for evaluating nonlinear functions of wavelet expansions, which we
242
A DA P T I V E WAV E L E T M E T H O D S
will refer to as the DSX algorithm. This algorithm (see Algorithm 7.10 below) consists of four main steps which we will detail in the sequel. We always start with a known finite approximation uε of a function u ∈ H t (Ω), i.e. uλ ψλ ,
u − uε t;Ω ≤ ε, (7.47) uε = λ∈Λu,ε
with a finite index set Λ = Λu,ε ⊂ J . Think of u being the (unknown) solution of a given nonlinear variational problem, and u as the current numerical approximation, which is known and finite. Due to the existing error analysis, we assume that we know how accurate the current approximation is. This motivates (7.47). The goal is to determine a fully computable approximation g of F(u) of similar accuracy to the current approximation to the input u, i.e. dλ ψ˜λ , and
g − F(u) −t;Ω ε, g= ˆ λ∈Λ
where the index set ˆ = Λ(u, ˆ ˜ Λ ε, F, Ψ, Ψ) is as small as possible and will be adaptively chosen. Due to our Lipschitz assumption (7.43) we know already that then
F(u) − F(uε ) −t;Ω ε. Hence, we preserve the current accuracy of the approximation. On one hand, this is the minimal requirement since the evaluation of the nonlinear term should be of the same accuracy (otherwise the subsequent computations would suffer accuracy). On the other hand, it also makes no sense to compute the application of F with a higher accuracy then the current approximation. This would be a waste of computational cost. Therefore, we will consider the equivalent problem of finding a finite approximation dλ ψ˜λ , with
g − F(uε ) −t;Ω ε. (7.48) g= ˆ λ∈Λ
The core algorithm reads as follows (Algorithm 7.10, page 243). These steps are not entirely independent of each other. In particular, the prediction and the quasi-interpolation are linked by an error functional as we will describe in the sequel. Let us now briefly give some background information on these steps.
N O N L I N E A R P RO B L E M S
243
Algorithm 7.10 Core nonlinear evaluation algorithm Input: uε , Λ, ε ˆ ⊂ J of significant indices in (7.48); 1. Prediction: Predict the set Λ 2. Reconstruction: Determine a “local scaling function representation” (to be explained later) of uε , which permits a fast computation of function values. 3. Quasi-interpolation: Based on the prediction, compute a quasiinterpolant g = A(F(uε )) in a finite local scaling function representation, using function values of F(uε ); ˆ of g. 4. Decomposition: Compute the wavelet coefficients dλ , λ ∈ Λ, 7.8.3
Prediction
The prediction is based upon two main ingredients, namely • a computable local error estimator; • an adaptive subdivision of the wavelet index set. We will now describe these ingredients starting with the second one. Adaptive subdivision of the wavelet index set We have to identify in which regions of Ω we need which wavelet functions. Of course, we want to use as few wavelets as possible in order to obtain a highly efficient approximation. It turns out to be useful to associate to each wavelet a certain part of the domain and to consider a subdivision of the domain in order to obtain a subdivision of the wavelet index range. We describe this association first. To any index λ = (j, k) we associate its support cube λ := 2−j (k + ).
(7.49)
This notation is sufficiently general also to include the multivariate case Ω ⊂ Rn . For the univariate periodic case we can simply define λ := 2−j [k, k + 1), i.e. := [0, 1). Note that since Jj = {0, . . . , 2j − 1}, we have that λ Ω = [0, 1] = |λ|=j
and this is (up to sets of measure zero) a disjoint union. This can also be done in the multivariate case in an analogous way.
244
A DA P T I V E WAV E L E T M E T H O D S λ
λ j+1
j+1
j
j λ⬘
Fig. 7.18 The two indices on the left have a tree structure, whereas the two on the right do not.
Definition 7.21 We say that an index set Λ ⊂ J has tree structure if λ ∈ Λ implies λ ∈ Λ for all λ ∈ J such that λ ⊂ λ . To illustrate this, let us again consider the univariate periodic case. Referring to Figure 7.5 (page 195) to explain the covering of Ω by support cubes, Figure 7.18 shows one example of the tree structure (left) and one example where this condition is not valid (right). One can now study the approximation properties of function spaces whose index ranges have tree structure. It turns out that the best N -term tree approximation has almost the same order as without the tree restriction. We do not go into details here. Now, we consider the adaptive refinement of these partitions. Defining the set of all support cubes on a given level j as Lj := {λ = (j, k) : |λ ∩ Ω| = 0} , we obtain the obvious covering ¯= Ω
λ
λ∈Lj
for each j and consequently |Lj | ∼ 2j . Now, we look for a multiscale covering Γ of Ω as indicated in Figure 7.19. Let us make this precise. We call a subset Lj Γ⊂ j≥0
a multiscale covering if ¯= λ and Ω λ∈Γ
A A Aλ ∩ µ A = 0 for λ, µ ∈ Γ, λ = µ.
(7.50)
N O N L I N E A R P RO B L E M S
245
j 5 4 3 2 1 0 1
Fig. 7.19 Multiscale covering of Ω = [0, 1] by the index set Γ = {(1, 0), (2, 3), (3, 4), (4, 10), (5, 23), (5, 24)}. ¯ consisting of mutually disjoint support cubes In other words Γ is a covering of Ω on different levels. The aim of the prediction is to determine a multiscale covering adapted to uε and the nonlinearity F with minimal cardinality. Minimal cardinality means because of |Lj | ∼ 2j that indices on the coarse level are used as much as possible and high levels are used locally only in regions where it is necessary. The selection of Γ is determined by a local error estimator which is described now. Computable local error estimator We want to associate to each support cube a local portion of the error in approximating F(u) which would arise if this support cube would not be refined any more. This local error estimator takes the form e(λ ) = e(λ ; uε , F). Of course, e(·) also depends on the chosen wavelet basis Ψ (and possibly also on ˜ but we do not consider this dependency since we assume that the the duals Ψ) wavelet bases are chosen at the beginning. From the motivation of the error estimator it should be clear that it should decrease if a wavelet is further refined. To express this, define = < M(λ) := µ ∈ J|λ|+1 : µ ⊂ λ , namely those indices on the next higher level |λ| + 1 whose support cubes are contained in that of λ. Then, we require that the following refinement property ν∈M(λ)
e(ν ) ≤ e(λ )
(7.51)
246
A DA P T I V E WAV E L E T M E T H O D S
is satisfied. Moreover, e(λ ) is required to be an upper bound for the local error of the approximation AΓ (F(uε )) ≈ F(uε ). This already gives a hint that one cannot just define the error functional e(·) in a straightforward way. Often, the definition is done in combination with an approximation scheme A. This, in turns, can be done by a quasi-interpolation scheme. We will come back later to this point. Here, we just assume that an appropriate local error functional e(·) is given to us. The aim of the prediction is to find a partition Γ so that for a given δ > 0 E(Γ) := e(λ ) ≤ δ, λ∈Γ
while Γ is small. In [33], a detailed description for a method solving this problem is presented. Prediction algorithm Based on the presented subdivision and under the assumption that an error functional e(·) is given, we obtain a possible algorithm as described in Algorithm 7.11. The main idea of the algorithm should be clear. The reason for checking e(µ ) ≤ 2−|µ| δ is explained as follows. Since |Lj | ∼ 2j this ensures that E(Γ) ≤ δ as desired. Algorithm 7.11 Prediction Given a desired tolerance δ. 1: 2: 3: 4: 5: 6: 7: 8: 9: 10:
7.8.4
Γcheck = {I−1 } Γ=∅ repeat for µ ∈ Γcheck do if e(µ ) ≤ 2−|µ| δ then Γcheck := Γcheck \ {µ} Γ := Γ ∪ {µ} else refine µ end if end for until Γcheck = ∅
Reconstruction
The ultimate idea for evaluating nonlinear functions of wavelet expansions is to avoid multiscale integration of functions on fairly different levels. The goal
N O N L I N E A R P RO B L E M S
247
is to use point values instead. In fact, to compute point values of the nonlinear function itself will not be difficult, think of the example F(u) = u2 or even F(u) = eu . The difficulty comes from the fact that u (or an approximation) is given in terms of its wavelet expansion. If we want to use point values (F(u))(x), we have to determine u(x) for a given x ∈ Ω. This, however, is a costly issue if one uses the multiscale representation of u directly. The single-scale representation in terms of the scaling functions would be more convenient. However, reconstructing uε = u λ ψλ λ∈Λu,ε
in a straightforward way would result in a single-scale representation on the highest level jmax (Λu,ε ) := max{|λ| + 1 : λ ∈ Λu,ε } (the term “+1” is due to the refinement equation) which clearly would waste all possible gains due to adaptivity. Hence, we look for a local scaling function representation. We use the refinement equation in the following form j bk,m ϕj+1,m , ψj,k = m∈Ij+1
which takes a general setting into account. Of course, the sum on the right-hand side is only a local finite one since we use wavelets with compact support. Now we write the input in the following form v= vλ ψλ λ∈Λ jmax (Λ)−1
j=0
λ∈Λj
=
vλ ψλ
jmax (Λ)−1
j=0
k: (j,k)=λ∈Λj
=
vj,k ψj,k .
Let us now consider the contribution on one level and use the refinement equation to this: Qj v = vj,k ψj,k k: (j,k)=λ∈Λj
=
k: (j,k)=λ∈Λj m∈Ij+1
bjk,m vj,k ϕj+1,m
248
A DA P T I V E WAV E L E T M E T H O D S
=
bjk,m vj,k ϕj+1,m
(7.52)
m∈Ij+1 ∩Γj+1 k: (j,k)=λ∈Λj
+
bjk,m vj,k ϕj+1,m .
(7.53)
m∈Ij+1 \Γj+1 k: (j,k)=λ∈Λj
This last splitting has the following meaning. The part in (7.53) contains only indices that are not present in the prediction set Γ. This means that no further high-level wavelets will be added to this part. Only the first sum in (7.52) will be updated by additional terms. On level j + 1 this means that we have to refine the function Qj+1 v + bjk,m vj,k ϕj+1,m m∈Ij+1 ∩Γj+1 k: (j,k)=λ∈Λj
=
vj+1,k ψj,k +
bjk,m vj,k ϕj+1,m .
m∈Ij+1 ∩Γj+1 k: (j,k)=λ∈Λj
k: (j+1,k)=λ∈Λj+1
Algorithm 7.12 Reconstruction to local scaling function representation vλ ψλ Given a function v = λ∈Λ
1: K = ∅, d = 0 2: c0 = v|I −1 3: for j = 0, . . . , jmax (Λ) do 4: compute for all significant m ∈ Ij+1 j+1 j gj+1,m = bk,m vj,k + aj+1 k,m ck k
5: if m ∈ Ij+1 ∩ Γj+1 then 6: K = K ∪ {(j + 1, m)} 7: dj+1,m = gj+1,m 8: else 9: cj+1 m = gj+1,m 10: end if 11: end for
Algorithm 7.12 realizes this local change in the following sense vλ ψλ = dj,k ϕj,k . v= λ∈Λ
(j,k)∈K
N O N L I N E A R P RO B L E M S
7.8.5
249
Quasi-interpolation
Let us first give the definition of a quasi-interpolation operator.
Definition 7.22 We say the linear operators Qj : Lp (Ω) → Sj , form a quasi-interpolation scheme, satisfied:
j ∈ N0 ,
if the following conditions are
(a) Qj preserves polynomials of degree less than d ∈ N, i.e. Qj P = P
for
P ∈ Pd−1 .
(b) The operators Qj are local and uniformly bounded in the following sense: there are a constant CQ < ∞ independent of j and a compact set K so that for each U ⊆ Ω
Qj f Lp (U ) ≤ CQ f Lp (U +2−j K) .
(7.54)
The locality in Definition 7.22 (b) in particular implies that for any fixed x ∈ Ω the value Qj f (x) depends only on values f (y) with |x − y| 2−j . For computational purposes it is particularly useful that the functionals qλ = q(j,k) in the representation Qj f =
qλ (f ) ϕλ
(7.55)
λ∈Ij
are local, i.e. they should only depend on function values in a neighborhood of 2−j k. In the sequel, we will assume that |q(j,k) (f )| ≤ Cq 2j( p − 2 ) f Lp (2−j (k+K )) , n
n
250
A DA P T I V E WAV E L E T M E T H O D S
with a compact set K , which implies immediately that (7.54) is satisfied for K = K + [0, d]n . In [36], an adaptive quasi-interpolation operator for splines was introduced. We review this operator as one example. Let ϕ = Nd be the cardinal B-spline of order d. Following a result of Zheludev [188] one can choose the specific functional d−1 2
q(f ) :=
d ˜ 2 (−1) βd, ∆ 1 f(2)
(7.56)
=0
to obtain quasi-interpolation operators, which are exact for polynomials of degree ˜ denotes the symmetric difference defined by d − 1. Here, ∆ h ˜ 1 f (x) := f ∆ h
h h −f x− x+ 2 2
˜ +1 f (x) := ∆ ˜1 ∆ ˜ and ∆ h h f (x). h
The coefficients βd, are defined via the generating function
2 arcsin 2t t
d =
∞
βd, t2 .
(7.57)
=0
Of course, this latter representation is not well suited for numerical computations. In fact, one would like to avoid the difference operators. Note that the functional q defined in (7.56) can also be expanded in terms of point values of f in the following form
q(f ) =
d−1 =1 γd, f (),
if d is even,
d γd, f ( − 12 ),
if d is odd,
=1
where the weights γd, can be determined explicitly from (7.56). The values for d ≤ 6 are given in Table 7.3. We also show the constant CQ appearing in (7.54). Obviously, this representation is well suited for computational purposes. One easily checks that in this case the compact set K in Definition 7.8.5 can be chosen as K = [1 − d, d − 1] for even d and K = [ 12 − d, d − 12 ] for odd d. This operator is so far defined level-wise, but we have to define it on an adaptive set of indices as K above. Hence, we follow again [36] and describe the extension to an adaptive quasi-interpolation operator. We consider again a Γ in the sense of (7.50) and assume that this is given to us. We could also think of the index set K arising from the local scaling function
N O N L I N E A R P RO B L E M S
251
Table 7.3 Coefficients γd, . d
γd,
CQ
1 2 3
1 1 − 18 , 54 , − 18
4
− 16 , 43 , − 16
5
47 107 319 107 47 1152 , − 288 , 192 , − 288 , 1152 13 7 73 7 13 240 , − 15 , 40 , − 15 , 240
6
1 1 3 2 5 3 179 72 43 15
representation. We set again j0 = min |λ| and J = max |λ|. λ∈Γ
λ∈Γ
Denote now by Ωj =
λ
λ∈Γj
the area covered by the support cubes from level j. Now we define as above Aj f := qλ (f ) ϕλ λ∈∆j
with ∆j := {λ ∈ Ij : |σλ ∩ Ωj | = 0} ,
(7.58)
where σλ is the support of the corresponding scaling function ϕλ . Note that ∆j is the minimal index set for which A A Aj f A = Qj f A . Ωj
Ωj
Starting with AΓ,j0 := Aj0 we want to define successive updates AΓ,j to obtain finally an adaptive operator AΓ := AΓ,J . Then, it is natural to set AΓ,j f := AΓ,j−1 f + Aj f −
(AΓ,j−1 f, ϕ˜λ )0;Ω ϕλ .
(7.59)
λ∈∆j
Without going into the details, we quote some results from [36] concerning the analysis of the above operators.
252
A DA P T I V E WAV E L E T M E T H O D S
Theorem 7.23 (see [36, Theorem 4.1]) The operator AΓ defined in (7.59) is an adaptive quasi-interpolation operator in the sense that (a) it preserves all polynomials P ∈ Pd−1 , i.e. AΓ P = P ; (b) it is local with respect to Γ, i.e. for any open set U ⊂ Ωj we have A A A A AΓ f A = AΓ (χU +2−j K f ) A , U
U
where K is the compact set from Definition 7.22.
This first result tells us that AΓ is a suitable operator. However, we also have to investigate the interpolation error. It turns out that this investigation also gives rise to an error estimator which is required for the prediction.
Theorem 7.24 (see [36, Theorem 4.4]) Let 0 < s < d. Then one has for λ ∈ Γ and f = F(uε ) with uε given in (7.47) 2|µ|(2r+n) |uεµ |2 (7.60)
AΓ f − f 20;λ CF ( uε H t )2 2−|λ|(2s+n) µ∈Λ: |ωµ ∩∗ λ |=0
with ∗λ defined as ∗λ = λ + 2−|λ| K,
(7.61)
and r, s, CF ( uε H t ) from (7.44) and ωµ denotes the support of ψµ . Now, note that the right-hand side of (7.60) is in fact computable as long as we know the constant CF (or at least an approximation to it). However, this is the Lipschitz constant of F so it is not too unrealistic to assume that this is known. Hence we can in fact use the right-hand side of (7.60) as a local error estimator in the prediction. 7.8.6
Decomposition
Using all the previous steps, we now have an approximation of F(u) in terms of a local scaling function representation AΓ (F(uε )) =
˜ (j,k)∈K
cj,k ϕj,k .
N O N L I N E A R P RO B L E M S
253
The last step is to decompose this into its multiscale wavelet representation AΓ (F(uε )) =
dλ ψλ .
˜ λ∈Λ
One starts on the highest level and performs a local decomposition. For the next step, one has to add the contribution of the next lower Γj . But since Γj is a tree, we only add some few coefficients near the leaves of the tree. There, one performs again a local decomposition and iterates this until the coarsest level is reached. To fix notation, let us abbreviate for j ≥ 0 ˜ j := K ˜ ∩ Ij , K i.e. the indices on level j. Then, a possible realization is given in Algorithm 7.13.
Algorithm 7.13 Local decomposition INPUT: function in local scaling function representation: v= cj,k ϕj,k ˜ (j,k)∈K
Note: All subsequent contributions.
sums
have
to
˜ j = ∅} 1: set J := max{j ≥ 0 : K 2: for j = J downto 0 do 3: for m ∈ Ij−1 do j−1 4: cj−1,m = cj−1,m + a ˜m,k cj,k ,
be
restricted
dj−1,m = cj−1,m
k∈Ij
5: 6: 7:
end for do for m ∈ Jj−1 ˜bj−1 cj,k dj−1,m = m,k k∈Ij
8: end for 9: end for OUTPUT: Multiscale representation: v =
(j,k)
dj,k ψj,k .
to
nonzero
254
7.9
A DA P T I V E WAV E L E T M E T H O D S
Exercises and programs
Exercise 7.1 Show that there exists a constant C(τ ) (depending on τ for τ such that
0)
v + w wτ (J ) ≤ C(τ )( v wτ (J ) + w wτ (J ) ) for all v, w ∈ w τ (J ). Exercise 7.2 Prove Proposition 7.9. Exercise 7.3 Show that · As as defined in (7.25) defines a norm and that As = {v ∈ 2 (J ) : v As < ∞} is an equivalent characterization. Exercise 7.4 Prove Proposition 7.17. Exercise 7.5 A function f ∈ L2 (Ω) is assumed to have the following representation f = dT Ψ with respect to a wavelet basis Ψ = {ψλ }λ∈Λ . The N -term-thresholding of f is defined as TN (f ) := dλ ψλ , λ∈ΛN (f )
where ΛN (f ) is the index set of the N largest wavelet coefficients dλ in absolute value. In which relation is the error f − TN (f ) 0;Ω of the N -term-thresholding to the best N -term approximation ρN (f ) =
inf
#supp c≤N
f − cT Ψ 0;Ω ,
(a) if Ψ is an orthonormal basis, (b) if Ψ is a Riesz basis? Exercise 7.6 Prove the Schur lemma, Lemma 7.6. Exercise 7.7 Prove the properties of THRESH formulated in Proposition 7.17. Exercise 7.8 Show that ∗ −1/τ (a) v ∈ w , where (vn∗ )n∈N is the reordering of τ (J ) if and only if vn ≤ c n ∗ v, i.e. vn is the nth largest entry in v in absolute values.. ε > 0, (b) τ (J ) → w τ (J ) → τ +ε (J ), where A → B means that A ⊂ B and x B ≤ c x A with c independent of x ∈ A. (c) v + w wτ (J ) ≤ C(τ ) v wτ (J ) + w wτ (J ) .
E X E RC I S E S A N D P RO G R A M S
255
Exercise 7.9 Show that for s < m the following equivalence holds
s (Ω) ∼ f Lp (Ω) +
f Bp,q
a
q −sq−1
m
0
ω (f, t, Lp (Ω)) t
q1 dt
−s m sup t ω (f, t, Lp (Ω)),
,
if 1 ≤ q < ∞, if q = ∞,
t∈(0,a)
for any s a ∈ (0, ∞]. Exercise 7.10 Show that s s (Ω), (Ω) → Bp,q (a) Bp,q 1 2
0 < q1 < q2 ≤ ∞.
s1 s2 (b) Bp,q (Ω) → Bp,q (Ω), 1 2
0 < q1 , q2 ≤ ∞, 0 ≤ s2 < s1 .
0 (c) f Bp,∞ (Ω) ∼ f Lp (Ω) ,
0 < q ≤ ∞.
Exercise 7.11 Determine s as a function of α ∈ (0, 1) and 0 < p, q ≤ ∞, so that fα (x) :=
xα , 0
for x > 0, else
s is in the Besov space Bp,q ([−1, 1]).
Programs In addition to the below listed programs we provide codes for the following tasks. • Sorting routines. • Data structures for handling adaptive wavelet expansions, e.g. hash maps. • Computation of inner products (of derivatives) of wavelets on fairly different levels. • A core realization of the adaptive scheme for evaluating nonlinear functions. • A realization of the modified adaptive scheme from Section 7.7.2. Program 7.1 The code best-N-term.cc determines the best N -term approximation of a given function. The highest level j can be chosen. Program 7.2 The code ad-hoc-adaptive.cc realizes the described ad hoc refinement strategy for solving the periodic boundary value problem with both strategies.
256
A DA P T I V E WAV E L E T M E T H O D S
Program 7.3 The code apply-periodic.cc realizes the approximate adaptive operator application APPLY for the periodic boundary value problem. Program 7.4 The code quasi-interpolant.cc realizes the above-described quasi-interpolation scheme. Program 7.5 The code adaptive-quasi-interpolant.cc realizes the adaptive quasi-interpolation scheme.
8 WAVELETS ON GENERAL DOMAINS So far we have only used periodic multiresolution analyses and wavelets. This was done mainly to avoid technicalities as much as possible. Of course, periodic boundary value problems are only of limited interest in applications. If one aims at using wavelet methods for solving elliptic partial differential equations on general domains, one needs to construct a wavelet basis on that domain. We have seen so far how the (deep) analytical properties of wavelets crucially enter the deep analysis of wavelet-based methods, in particular for the convergence analysis of adaptive methods. Hence, it is not very surprising that it is not an easy task to construct wavelet bases on general domains in such a way that all analytical properties are preserved. However, nowadays there exist several such constructions of wavelet bases on general domains. Since these constructions are technically sophisticated, it is widely believed that wavelet methods cannot efficiently be used for problems on complicated domains. Of course, local bases like finite elements can be adapted much more easily to complicated domain geometries. However, also grid generation for finite elements is not a completely trivial task. This is usually seen as an automatic preprocessing which is done by software. In some sense, the construction of wavelet bases on general domains is similar to grid generation. First, it is technically sophisticated, but secondly it can be done as a preprocessing once and for all. Finally, nowadays there is software available that does this job. The aim of this chapter is to introduce the construction of wavelet bases on general domains, also giving the technical details. On the other hand, we also show that such a construction can be seen as a black box that can just be adapted also if one is not interested in all the technical details. In particular, readers that are not interested in these details, may skip the technicalities. The road map is as follows. A general domain is decomposed into parametric images of the unit cube. On the unit cube, one uses a tensor product wavelet basis of the unit interval. Hence, we start by multiresolution and wavelets on the interval. We highlight several aspects also for use in numerical methods. Then, we consider tensor products and finally we match the parametric images on the true domains with the aid of the Wavelet Element Method (WEM). This road map is indicated in Figure 8.1.
257
258
WAV E L E T S O N G E N E R A L D O M A I N S [ j, k]
,
...
...
j, k
[0, 1]
0
1
1 ^ Ω = [0, 1]n,
^ j, e, k
0 0
1
^ Ωm = Fm(Ω)
Ω = 8 Ωm m
Fig. 8.1 Road map for the Wavelet Element Method (WEM). Starting from wavelets on the real line R, construct corresponding bases on [0, 1], then ˆ mapping to Ωm and then finally on Ω by domain by tensor product on Ω, decomposition and matching.
8.1
Multiresolution on the interval
We start by considering boundary value problems like (3.2) on a univariate interval and consider MRAs on bounded intervals [a, b]. Clearly, we can restrict
M U LT I R E S O L U T I O N O N T H E I N T E RVA L
259
ourselves without loss of generality to the unit interval I := (0, 1). There are several constructions for an MRA on I available. Here, we follow a joint paper of the author with Dahmen and Kunoth [87] and partly joint work with Canuto and Tabacco [52]. Some of the alternative constructions (or modifications) will be sketched in Section 8.2.6 below. The basic difficulty of constructing an MRA (and corresponding wavelets) on (0, 1) can be described as follows: • The shift-invariant setting by Sj = closL2 span{ϕ[j,k] : k ∈ Z} is highly convenient since • all spaces Sj and Wj are basically generated by only one function ϕ (ψ, respectively); • change of bases can be performed highly efficient (i.e. fast wavelet transform). Hence one would like to use as much of the shift-invariant setting as possible. • A first idea could be to cut off all functions like ϕ[j,k]|I . However, this simple idea does not lead to the desired basis due to two main reasons: • The cut-off procedure could eliminate most of the function and retain only a very small portion, see Figure 8.2. It is clear that this results in stability problems. • We start by dual scaling functions ϕ and ϕ. ˜ Even though this implies (ϕ[j,k] , ϕ˜[j,m] )0;R = δk,m ,
k, m ∈ Z,
this duality is in general not true when replacing the integration domain R by I. [j,k] [j,k]I
0
Fig. 8.2 Cut-off from ϕ[j,k] resulting in a small piece ϕ[j,k]|I . The vertical dashed line is at the point x = 0.
260
WAV E L E T S O N G E N E R A L D O M A I N S
0
1 2–2
Fig. 8.3 Support of overlapping functions for j = 2 with support size 12 (top) and 34 (bottom). We obtain five overlapping functions with support size 12 and 6 with support size 34 . • The duality causes another problem. In general, the sizes of the support differ, without loss of generality |supp ϕ| < |supp ϕ|, ˜ which means that more functions ϕ˜[j,k] overlap I than ϕ[j,k] , see Figure 8.3. Hence, we have to adjust the index sets since Sj and S˜j need to be of the same dimension. This is one of the reasons why the general framework introduced in Section 2.9.1 (page 46) is appropriate here. Just to recall the main points, we have • • • • • • 8.1.1
˜ j = {ϕ˜j,k : k ∈ Ij }; Φj = {ϕj,k : k ∈ Ij }, Φ ˜ j ); Sj = closL2 (Ω) span(Φj ), S˜j = closL2 (Ω) span(Φ ˜ (Φj , Φj )0;Ω = IIj ×Ij ; ˜j,k = supp ϕ˜j,k ; σj,k = supp ϕj,k , σ σj,k | 2−j ; |σj,k | 2−j , |˜
ϕj,k 0;Ω 1, ϕ˜j,k 0;Ω 1. Refinement matrices
We start by describing the consequences of this general setting for the validity of a refinement relation. Throughout the remainder of this section, we adopt Assumption 2.16 (page 47). In the general framework, we cannot hope to get a refinement equation like (2.28), i.e. ϕ(x) = ak ϕ(2x − k), x ∈ R, k∈Z
M U LT I R E S O L U T I O N O N T H E I N T E RVA L
261
with refinement coefficients which are independent of the level. Hence, we are left with the general form (2.27), i.e. j ϕj,k = ak,m ϕj+1,m , (8.1) m∈Ij+1
of the refinement equation. It pays off to rewrite this in a condensed form. We view Φj = {ϕj,k : k ∈ Ij } as column vector where the components are the functions ϕj,k . We say that the family Φ = {Φj }j∈N0 is refinable if there exists an operator Mj,0 ∈ L(2 (Ij ), 2 (Ij+1 )) = [2 (Ij ), 2 (Ij+1 ))], where L(X, Y ) (or [X, Y ]) denotes the space of bounded linear operators from X to Y , such that T Φj = Mj,0 Φj+1 , (8.2) see [54]. If the index sets Ij and Ij+1 are finite, we have Mj,0 ∈ R|Ij+1 |×|Ij | . This means that Mj,0 is a rectangular matrix with roughly double the number |Ij+1 | of rows than columns |Ij |, see Figure 8.4. On might ask why the notation Mj,0 is used, in particular what the meaning of the index “0” might be. The reason is twofold, namely: • We often have to use scaling functions ϕ and wavelets ψ together, for example in a multiscale expansion we always need the scaling functions on the coarsest level. We will later use the abbreviation ψ0 := ϕ,
ψ1 := ψ
=
Φj
T Mj,0
Φj+1
Fig. 8.4 Graphical visualization of the refinement relation (8.2).
262
WAV E L E T S O N G E N E R A L D O M A I N S
to have a unified notation for scaling functions and wavelets. Moreover, in the n-dimensional case, we even have to distinguish 2n − 1 different kind of wavelets. Here, the index “0” is used to indicate that Mj,0 corresponds to scaling functions. Later, we will introduce a matrix Mj,1 for the wavelets. • As we can see in Figure 8.4, the matrix Mj,0 is rectangular. If we want to switch between a single-scale representation of a function and its multiscale representation, this is obviously a change of bases. This means that we have to complete the rectangular matrix Mj,0 to a square matrix which is invertible. This is done by the wavelet matrix Mj,1 , so that the composed matrix Mj = (Mj,0 , Mj,1 ) is the matrix for the desired change of bases. Since all what is said in the sequel also holds for infinite countable sets we prefer the notation Mj,0 ∈ L(2 (Ij ), 2 (Ij+1 )). Often, Mj,0 is also called a refinement matrix (even though it might be an infinite or bi-infinite matrix). Comparing (8.2) with (8.1), we see that the refinement matrix takes the form T = (ajk,m )k∈Ij ,m∈Ij+1 . Mj,0
(8.3)
A refinable system which is uniformly stable in the sense of (2.24), i.e. cΦ cj 2 (Ij ) ≤ cTj Φj 0;Ω ≤ CΦ cj 2 (Ij ) with constants 0 < cΦ ≤ CΦ < ∞ independent of j is also called a single-scale system, generator or scaling system. Remember the shorthand notation from (2.48), i.e. cj,k ϕj,k . cTj Φj := k∈Ij
8.1.2
Boundary scaling functions
Typically, a construction of an MRA on (0, 1) starts with a refinable function ϕ and the generated MRA on R as indicated in Figure 8.1 above. Obviously we need modifications near the two boundary points x = 0 and x = 1. Only in the very easy case of the piecewise linear hat function (see Section 2.2 on page 17) (and without considering the dual functions) this can be done by restricting those ϕ[j,k] that overlap one of the two end points. In the general case, we need a different approach. Let us first fix the goals that we want to achieve, i.e. let us collect those properties of Φj that we require. The guideline for this is given by the results in Section 2.9:
M U LT I R E S O L U T I O N O N T H E I N T E RVA L
263
˜ j must be local in the sense of Assumption 2.16. (1) Φj and Φ ˜ (2) The functions ϕj,k and ϕ˜j,k must have the same regularity as ϕ and ϕ, respectively. ˜ j must represent polynomials of the same degree as ϕ and ϕ, ˜ (3) Φj and Φ respectively. ˜ j must be stable, in particular (4) Φj and Φ • if ϕ has orthonormal translates, then Φj must be orthonormal (the same for ϕ); ˜ ˜ j must be biorthogonal. • if ϕ, ϕ˜ are biorthogonal, then Φj , Φ The general strategy is to consider linear combinations of the form j γk,m ϕ[j,m] |[0,1] ϕj,k = m
ϕ˜j,k =
j γ˜k,m ϕ˜[j,m] |[0,1]
m j j and to choose γk,m , γ˜k,m ∈ R in such a way that the above goals can be achieved. In order to reach the first goal (locality) the target is to have as few coefficients j j , γ˜k,m different from zero as possible. Moreover, we would like to have γk,m j = δk,m , γk,m
j γ˜k,m = δk,m
for as many indices k, m as possible. In fact, this would imply ϕj,k = ϕ[j,k] ,
ϕ˜j,k = ϕ˜[j,k] ,
i.e. original translates. Hence, we follow the strategy only to perform changes near the boundaries. The second goal (regularity) is automatically satisfied since we just form linear combinations which preserve the original smoothness. Now, we consider the third goal, i.e. we look for a construction of the boundary functions in order to get the same order of polynomial exactness as for the original scaling function ϕ. Thus, we start by the representation of monomials on R xr = αr,k ϕ(x − k), x ∈ R a.e. (8.4) k∈Z
which holds by assumption for all r = 0, . . . , d − 1, where xr ϕ(x ˜ − k)dx = ((·)r , ϕ(· ˜ − k))0;R . αr,k := R
We easily obtain useful relations for these coefficients.
(8.5)
264
WAV E L E T S O N G E N E R A L D O M A I N S
Lemma 8.1 Let ϕ, ϕ˜ be biorthogonal scaling functions with masks a = {ak }k∈Z and ˜ a = {˜ ak }k∈Z , respectively. Then, we have αr,k a ˜k−2m , (8.6) αr,m = 2−r−1 = 2r
k∈Z
αr,k ak−2m .
(8.7)
k∈Z
Proof Using the definition of αr,m and the refinement equation for ϕ˜ yields αr,m = a ˜k−2m xr ϕ(2x ˜ − k) dx R
k∈Z −r−1
=2
a ˜k−2m
k∈Z −r−1
=2
R
y r ϕ(y ˜ − k) dy
a ˜k−2m αr,k ,
k∈Z
where we have used the substitution y = 2x. In order to prove (8.7), we use the refinement equation for ϕ to obtain xr = αr,k am−2k ϕ(2x − m) k
=
m
#
m
$
αr,k am−2k
ϕ(2x − m).
k
Now, we use (8.4) applied to y = 2x xr = 2−r (2x)r = 2−r
αr,m ϕ(2x − m).
m∈Z
Since {ϕ(2 · −m), m ∈ Z} is linearly independent, the comparison of the latter two equations proves (8.7) The next step is to use (8.4) and (8.5) to modify some functions ϕ[j,k] near the boundaries in order to reproduce polynomials. Because of the different sizes of the support of ϕ and ϕ, ˜ we need to keep track of the index sets. In order to do so, we need to precisely consider the supports of ϕ and ϕ. ˜ Thus, we assume that ϕ is compactly supported supp ϕ = [1 , 2 ],
−∞ < 1 < 2 < ∞.
M U LT I R E S O L U T I O N O N T H E I N T E RVA L
265
Then, the index set KL = {1 − 2 , . . . , −1 − 1}
(8.8)
denotes those indices where the corresponding translates overlap the left boundary xL = 0, i.e. KL = {k ∈ Z : | 0 ∈ Int(supp ϕ[j,k] )}. Note that this set does not depend on the level j. Now we are ready to define the modified scaling functions to represent polynomials. We will concentrate on the left boundary xL = 0 and do the modifications at the right boundary xR = 1 by a similar construction. To this end, we define the boundary functions as ϕL j,r :=
−1
αr,m ϕ[j,m] |R+ ,
r = 0, . . . , d − 1,
(8.9)
m=1−2
where ≥ −1
(8.10)
is a free parameter to be chosen later. Thus, we have KL ⊆ {1 − 2 , . . . , − 1}, i.e. the above sum contains at least all functions which overlap the boundary xL = 0. The free parameter will be used later to adjust the index sets in order to obtain primal and dual index sets of the same cardinality. It is not a priori clear that the boundary functions are refinable. Of course, we cannot expect to obtain a refinement equation that expresses ϕL j,r just in . Instead, the following refinement relation holds. terms of ϕL j+1,r Lemma 8.2 The boundary scaling functions ϕL j,r defined in (8.9) satisfy the following refinement relation # $ 2+ 1 −1 L −(r+1/2) L αr,m ϕ[j+1,m] ϕj+1,r + (8.11) ϕj,r = 2 m=
+
2+ 2 −2
βr,m ϕ[j+1,m] ,
r = 0, . . . , d − 1,
m=2+1
where
βr,m := 2−1/2
−1 m− q= 2 2
αr,q am−2q .
(8.12)
266
WAV E L E T S O N G E N E R A L D O M A I N S
Proof We use the standard refinement equation for ϕ to obtain −1
ϕL j,r =
αr,m
m=1−2
2 +2m k=1
1 √ ak−2m ϕ[j+1,k] |R+ , 2 +2m
where we have taken the support into account, supp ϕ = supp a = [1 , 2 ]. Now we change the ordering of the two sums and take the supports into account ϕL j,r =
2+ 1 −1
−1
k=1−2 m=1−2
+
2+ 2 −2
1 √ αr,m ak−2m ϕ[j+1,k] |R+ 2
−1
k=2+1 m=! k−2 " 2
=
2+ 1 −1 k=1−2
1 √ αr,m ak−2m ϕ[j+1,k] 2
2+ 2 −2 1 √ 2−r αr,k ϕ[j+1,k] |R+ + βr,k ϕ[j+1,k] , 2 k=2+ 1
where we have used (8.7) for the first sum in the last step. Finally, we split up the first sum 2+ 1 −1
2−(r+1/2) αr,k ϕ[j+1,k] |R+
k=1−2 −1
=
2−(r+1/2) αr,k ϕ[j+1,k] |R+ + 2−(r+1/2)
k=1−2
= 2−(r+1/2)
# ϕL j+1,r +
2+ 1 −1
$
2+ 1 −1
αr,k ϕ[j+1,k]
k=
αr,k ϕ[j+1,k] ,
k=
which proves the assertion.
Let us now come to the scaling functions corresponding to the right boundary xR = 1. Again, we start from (8.4), but now the set of indices corresponding to ϕ[j,k] overlapping xR depends on the level (as opposed to KL in (8.8)). In fact, we get by supp ϕ[j,k] = [2−j (k + 1 ), 2−j (k + 2 )] immediately KjR = {2j − 2 + 1, . . . , 2j − 1 − 1}.
(8.13)
M U LT I R E S O L U T I O N O N T H E I N T E RVA L
267
Even though the index range in KjR depends on the level, the cardinality is independent. Indeed, |KjR | = (2j − 1 − 1) − (2j − 2 + 1) + 1 = 2 − 1 − 1 = |KL |, which is related to the size of the support by |KjL | = |KjR | = |supp ϕ| − 1. In order to obtain as much symmetry as we can, we do not aim at reproducing the monomials xr , r = 0, . . . , d−1, at xR = 1, but we use (1−x)r , i.e. the reflected monomials. Polynomial exactness gives again (1 − x)r = ((1 − ·)r , ϕ(· ˜ − k))0;R ϕ(x − k), x ∈ R a.e. (8.14) k∈Z
Then, by the binomial expansion, we get (1 − x) ϕ(x ˜ − k) dx = r
R
r r i=0
=
i
r r i=0
i
i
(−1)
R
xi ϕ(x ˜ − k) dx
(−1)i αi,k .
As for KjR , we also need to incorporate a dependency on the level in the coefficients and define R ˜ − k) dx αj,r,k := (2j − x)r ϕ(x R
as well as ϕR j,r
:=
2j − 1 −1
R αj,r,m ϕ[j,m] |(0,1) ,
r = 0, . . . , d − 1,
(8.15)
m=2j −q
for a free parameter q ≥ l2
(8.16)
to be chosen later. We start by developing relations similar to Lemma 8.1. By a simple change of variables, we get R = 2j (2j (1 − x))r ϕ(2 ˜ j x − k) dx αj,r,k = R
R
1
2j(r+ 2 ) (1 − x)r ϕ˜[j,k] (x) dx
which means that for x ∈ R a.e. 1 R αj,r,k ϕ[j,k] (x), 2j(r+ 2 ) (1 − x)r = k∈Z
268
WAV E L E T S O N G E N E R A L D O M A I N S
1 am−2k √ ϕ[j+1,k] (x) 2 m∈Z k∈Z $ # 1 R √ αj,r,k am−2k ϕ[j+1,k] (x). = 2 k∈Z m∈Z =
R αj,r,k
Next, we show a useful relation for the coefficients. Lemma 8.3 The following relation holds R R = 2r αj,r,m ak−2m . αj+1,r,k m∈Z
Proof Using (8.14) for level j + 1 gives, on one hand, 1 R 2−(j+1)(r+ 2 ) αj+1,r,k ϕ[j+1,k] (x). (1 − x)r =
(8.17)
k∈Z
On the other hand, we use (8.14) and the refinement equation for ϕ to obtain 1 1 R √ ak−2m ϕ[j+1,k] (x) 2−j(r+ 2 ) αj,r,m (1 − x)r = 2 m∈Z k∈Z =
1 1 R 2−j(r+ 2 ) √ αj,r,m ak−2m ϕ[j+1,k] (x). 2 m∈Z k∈Z
(8.18)
Since the functions {ϕ[j+1,k] : k ∈ Z} are linearly independent, the comparison of (8.17) and (8.18) proves the claim. As on the left part of the boundary, this relation is a key ingredient for the proof of the following refinement equation. Proposition 8.4 The following refinement equation holds ϕR j,r
=
2j+1 + 2 −1−2q
R βj,r,k ϕ[j+1,k]
k=2j+1 +1 −2q
+ 2−(r+ 2 ) 1
2j+1 −q−1
R , αj+1,r,k ϕ[j+1,k] |(0,1) + ϕR j+1,r
k=2j+1 +2 −2q
where R βj,r,k
1 k− 2 1 R √ := αj,r,m ak−2m , 2 m=2j −q
r = 0, . . . , d − 1.
M U LT I R E S O L U T I O N O N T H E I N T E RVA L
269
Proof Let us denote by j j R j j MR j := {2 − q, . . . , 2 − 1 − 1} = Kj ∪ {2 − q, . . . , 2 − 2 }
the set of indices that occur in the definition of ϕR j,r . We use the refinement equation for ϕ to obtain
ϕR j,r =
2 +2m
R αj,r,m
m∈MR j
k=1
1 k− 2
2j+1 −1 −1
=
1 √ ak−2m ϕ[j+1,k] |(0,1) 2 +2m
!
k=2j+1 +1 −2q
k−2 2 m∈MR j
m=
"
1 R √ αj,r,m ak−2m ϕ[j+1,k] |(0,1) , 2
where the bounds in the sum take into account