Introduction to Partial Differential Equations with MATLAB

David

Applied and Numerical Harmonic Analysis Series Editor John J. Benedetto University of Maryland Editorial Advisory Board Akram Aldroubi N I H , Biomedical Engineering/ Instrumentation Ingrid Daubechies Princeton University Christopher Heil Georgia lnstitute of Technology James McClellan Georgia lnsitute of Technology Michael Unser NIH, Biomedical Engineering1 lnstrumentation M. Victor Wickerhauser Washington University

Douglas Cochran Arizona State University Hans G. Feichtinger University of Vienna Murat Kunt Swiss Federal lnstitute of Technology, Lausanne Wim Sweldens Lucent Technologies Bell Laboratories Marfin Vetterli Swiss Federal lnstitute of Technology, Lausanne

Applied and Numerical Harmonic Analysis J.M. Cooper: Introduction to Partial Differential Equations with MATLAB (ISBN 0-8176-3967-5) C.E. D'Attellis and E.M. Fernandez-Berdaguer: Wavelet Theory and Harmonic Analysis in Applied Sciences (ISBN 0-8 176-3953-5)

H.G. Feichtinger and T. Strohmer: Gabor Analysis and Algorithms (ISBN 0-8176-3959-4) T.M. Peters, J.H.T. Bates, G.B. Pike, P. Munger, and J.C. Williams: Fourier Transforms and Biomedical Engineering (ISBN 0-8 176-394 1- 1) A.I. Saichev and W.A. Woyczynski: Distributions in the Physical and Engineering Sciences (ISBN 0-8176-3924-1)

R. Tolimierei and M. An: Time-Frequency Representations (ISBN 0-81763918-7)

G.T. Herman: Geometry of Digital Spaces (ISBN 0-8176-3897-0) A. Prochazka, J. Uhlii, P.J.W. Rayner, and N.G. Kingsbury: Signal Analysis and Prediction (ISBN 0-8176-4042-8) J. Ramanathan: Methods of Applied Fourier Analysis (ISBN 0-8 176-3963-2)

A. Teol is: Computational Signal Processing with Wavelets (ISBN 0-81763909-8) W.O. Bray and C.V. Stanojevic: Analysis of Divergence (ISBN 0-81764058-4)

G.T. Herman and A. Kuba: Discrete Tomography (ISBN 0-8176-4101-7) J.J. Benedetto and P.J.S.G. Ferreira (Eds.): Modern Sampling Theory (ISBN 0-8176-4023-1) P. Das, A. Abbate, and C. DeCusatis: Wavelets and Subbands (ISBN 08176-4136-X)

L. Debnath: Wavelet Transforms and Time-Frequency Signal Analysis (ISBN 0-8176-4104-1) K. Grochenig: Foundations of Time-Frequency Analysis (ISBN 0-81764022-3) D.F. Walnut: An Introduction to Wavelet Analysis (ISBN 0-8176-3962-4)

David F. Walnut

An Introdution to Wavelet Analysis With 88 Figures

Birkhauser Boston Basel Berlin

D a v i d F. W a l n u t Department of M a t h e m a t i c a l Sciences George M a s o n University Fairfax, V A 22030 USA

Library of Congress Cataloging-in-Publication Data Walril~t,,David 12. Ari iritrod~lct~ion t,o wavelet analysis / David I". Walnut, p. crri. (Applied and n~irrlc!ricalllarri~o~lir itrlalysis) Iric111dc:sl)ibliug~-apl~iral refcl-eiices and indt:x. ISBN 0-81763-3962-4 (alk. paper) 1. Wavelets (Matliemat,ic:s) I. Title. 11. Series. QA403.3 .W335 2001 515'.2433 dc21 2001025367 CIP

Prir~t~ed 011 acid-f'rei: paper. @ 2002 Hirkhii~iserBost,ori

Birkhauser

All rig lit,^ reserved. This work irlay riot. I J ~t,la~lslat,eclor copicd ill whole or in part wit.tlout t,l~r:wl.it,t,t!~~ 1~r.1-lllission of t,he p~lblisher(Birkki&liser Bost,ol~,c/o Sprir~gcr-VcrlagNew York, Irir., 175 Fift,h Avenue, New York, NY 10010, USA), except for brief excerpts in corir~ection wit,11 reviews or scholarly analysis. lJse ill conncctiori with any form of information storage and rctricval, elect,ronic adapt,at,iori, cornplit,er soft,ware, or t)y sirnilar or dissirnilar methodology riow kriown or liereafter developed is forbidden. T h c 11sc of gelirral descript,ivo names, t,radc rlanles, tradcmarks, etc., i11 this publication, oven if t,l~eforrrier are not (:specially ide~~t.ified, is not t,o 1 ~ :t,a.kr~las a, sign that silcli ilalnes, as ulltlerstood l)y tlie Trade Marks anti Mcrcharldise Ma1.k~Act,, rrlay accordingly be used freely by anyoni:.

IS13N 0-8176-3962-4 ISBN 3-7643-3962-4

SPIN 10574019

Product,io~~ marlageti t)y Louise Farkas; riian~ifact,~~ring sl~pa.visedt?y .Jacqui Ashri Typc:sc:t, I)y t,lie aiitlior ill LaTeX2c. Printeti and I)oli~id1,y Edwards Rrot,hers, Inr., Arin Artlor, MI. Pririt,ed i r ~t,lit: l1liitc:d St,at,es of America.

L3irkliiiiist:r Host,o~l Basel

Berlin

A rrr ernbcr of /3erlr,lsn1,ur~nSpringer Science+ B7rs,irir ss hI(:d./:nG,nl.bH

To my parents

and to Megan

Unless the LORD builds the house, its builders labor in vain. -

Psalm 1 2 7 ; l a (NIV)

Contents xiii

Preface

I 1

Preliminaries

1

F'linctions and Convergence 1.1 Functions . . . . . . . . . . . . . . . . . . 1.1.1 Bounded (L") Functions . . . . . 1.1.2 Integrable (L1) Functions . . . . . 1.1.3 Square Integrable (L2) Functions . 1.1.4 Differentiable (Cn) Functions . . . 1.2 Convergence of Sequences of Functions . . 1.2.1 Numerical Convergence . . . . . . 1.2.2 Pointwise Convergence . . . . . . . 1.2.3 Uniform (L") Convergence . . . . 1.2.4 Mean ( L 1 )Convergence . . . . . . 1.2.5 Mean-square (L2) Convergence . . 1.2.6 Interchange of Limits and Integrals

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

2 Fourier Series

2.1

Trigonometric Series . . . . . . . . . . . . . . . . . . . . . .

2.1.1 Periodic Functions . . . . . . . 2.1.2 The Trigonometric System . . 2.1.3 The Fourier Coefficients . . . . 2.1.4 Convergence of Fourier Series . 2.2 Approximat.e Identities . . . . . . . . . 2.2.1 hlotivation from Fourier Series 2.2.2 Definition and Examples . . . . 2.2.3 Convergence Theorems . . . . . 2.3 Generalized Fourier Series . . . . . . . 2.3.1 Orthogonality . . . . . . . . . . 2.3.2 Generalized Fourier Series . . . 2.3.3 Complet.eness . . . . . . . . . . 3 The 3.1 3.2 3.3

. . . .

. . . .

. . . .

. . . .

. . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . .

Fourier Transform Motivation and Definition . . . . . . . . . . . . . . . . . . . Basic Properties of the Fourier Transform . . . . . . . . . . Fourier Inversion . . . . . . . . . . . . . . . . . . . . . . . .

3 3

3 3 6 9 11 11 13 14 17 19 21 27 27 27 28 30 32 37 38 40 42 47 47 49 52

59 59 63 65

Contents

viii

3.4 3.5 3.6 3.7 3.8 3.9

Coilvolutior~ . . . . . . . . . . . . . . . . . . . . . . Plancherel's Formula . . . . . . . . . . . . . . . . . The Fourier Trarlsfornl for L~ Functions . . . . . . Smoothiless versus Decay . . . . . . . . . . . . . . Dilation, Translation, ancl Modulation . . . . . . . Bandlirnitetl Functiorls and the Sarrlpling Formula.

. . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . .

4 Signals and Systems 4.1 Signals . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Systerris . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Causality and Stability . . . . . . . . . . . . 4.3 Periodic Signals a r ~ dthe Discrete Fourier Transform 4.3.1 The Discrete Fourier Transform . . . . . . . . 4.4 The Fast Fourier Transform . . . . . . . . . . . . . . 4.5 L2 Fourier Series . . . . . . . . . . . . . . . . . . . .

I1

. . . . . . . .

. . . . . . . .

. . . . . . . . . . . .

68 72 75 76 79 81

87 88 90 95 101 102 107 109

The Haar System

5 T h e Haar System 5.1 Dyadic Step Functions . . . . . . . . . . . . . . . . . . . . . 5.1.1 The Dyadic Intervals . . . . . . . . . . . . . . . . . . 5.1.2 The Scale j Dyadic Step Functions . . . . . . . . . . 5.2 The Haar System . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 T h e H a a r Scaling Functions and the Haar Functions . . . . . . . . . . . . . . . . . . . . . 5.2.2 Orthogonality of the Haar System . . . . . . . . . . 5.2.3 The Splitting Lemma . . . . . . . . . . . . . . . . . 5.3 Haar Bases on [O, 11 . . . . . . . . . . . . . . . . . . . . . . . 5.4 Comparison of Haar Series with Fourier Series . . . . . . . . 5.4.1 Representation of Functions with Small Support . . 5.4.2 Behavior of Haar Coefficients Near J u m p Discontinuities . . . . . . . . . . . . . . . . . . 5.4.3 Haar Coefficients and Global Smoothness . . . . . . 5.5 Haar B a s e s o n R . . . . . . . . . . . . . . . . . . . . . . . . 5.5.1 The Approximation and Detail Operators . . . . . . 5.5.2 The Scale J Haar System on R . . . . . . . . . . . . 5.5.3 The Haar System on R . . . . . . . . . . . . . . . .

115 115 115 116 117

6 The Discrete Haar Transform 6.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.1 The Discrete Haar Transform (DHT) . . . . . . . 6.2 The DHT in Two Dimensions . . . . . . . . . . . . . . . 6.2.1 The Row-wise and Column-wise Approximations and Details . . . . . . . . . . . . . . . . . . . . .

141

117 118 120 122 127 128 130 132 133 134 138 138

. . 141 . . 142 . . 146 . . 146

Contents

6.2.2 6.3 Iinage 6.3.1 6.3.2 6.3.3

I11

ix

The DHT for Matrices . . . . . . . . . . . . . . . . . Analysis with the DHT . . . . . . . . . . . . . . . . . Approximation and Blurring . . . . . . . . . . . . . Horizontal, Vertical, and Diagonal Edges . . . . . . "Naive" Image Compression . . . . . . . . . . . . . .

Ort honormal Wavelet Bases

147

150 151 153 154

161

7 Multiresolution Analysis 7.1 Orthonormal Systems of Translates . . . . . . . . . . . . . . 7.2 Definition of Multiresolution Analysis . . . . . . . . . . . . 7.2.1 Some Basic Properties of MRAs . . . . . . . . . . . 7.3 Examples of Multiresolution Analysis . . . . . . . . . . . . . 7.3.1 The Haar MRA . . . . . . . . . . . . . . . . . . . . . 7.3.2 The Piecewise Linear MRA . . . . . . . . . . . . . . 7.3.3 The Bandlimited MRA . . . . . . . . . . . . . . . . 7.3.4 The Meyer MRA . . . . . . . . . . . . . . . . . . . . 7.4 Construction and Examples of Orthonorrnal Wavelet Bases . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1 Examples of Wavclct Bases . . . . . . . . . . . . . . 7.4.2 Wavelets in Two Dimensions . . . . . . . . . . . . . 7.4.3 Localization of Wavelet Bases . . . . . . . . . . . . . 7.5 Proof of Theorem 7.35 . . . . . . . . . . . . . . . . . . . . . 7.5.1 Sufficient Conditions for a Wavelet Basis . . . . . . . 7.5.2 Proof of Theorem 7.35 . . . . . . . . . . . . . . . . . 7.6 Necessary Properties of the Scaling Function . . . . . . . . 7.7 General Spline Wavelets . . . . . . . . . . . . . . . . . . . . 7.7.1 Basic Properties of Spline Functions . . . . . . . . . 7.7.2 Spline Multiresolution Analyses . . . . . . . . . . . .

163 164 169 170 174 174 174 179 180

8 The Discrete Wavelet Transform 8.1 Motivation: From MRA t o a Discrete Transform . . . . . . 8.2 The Quadrature Mirror Filter Conditions . . . . . . . . . . 8.2.1 Motivation from MRA . . . . . . . . . . . . . . . . . 8.2.2 The Approximation and Detail Operators and Their Adjoints . . . . . . . . . . . . . . . . . . . . . 8.2.3 The Quadrature Mirror Filter (QMF) Conditions . . 8.3 The Discrete Wavelet Transform (DWT) . . . . . . . . . . . 8.3.1 The DWT for Signals . . . . . . . . . . . . . . . . . 8.3.2 The DWT for Finite Signals . . . . . . . . . . . . . . 8.3.3 The DWT as an Orthogonal Transformation . . . . 8.4 Scaling Functions from Scaling Sequences . . . . . . . . . . 8.4.1 The Infinite Product Formula . . . . . . . . . . . . . 8.4.2 The Cascade Algorithm . . . . . . . . . . . . . . . .

215 215 218 218

185 186 190 193 196 197 199 203 206 206 208

221 223 231 231 231 232 236 237 243

x

Contents 8.4.3

The Support of the Scaling Functio~i. . . . . . . . . 245

9 Smooth. Compactly Supported Wavelets 249 . . . . . . . . . . . . . . . . . . . . . . . 9.1 Vanishing Moments 249 Vanishing Moments and Smoothness . . . . . Vanishing Moments and Approximation . . . Vanishing hilomcnts and thc Reproduction of Polynomials . . . . . . . . . . . . . . . . . 9.1.4 Equivalent Conditions for Vanishing h!Iornent.s The Daubechies Wavelets . . . . . . . . . . . . . . . 9.2.1 The Daubechies Polynomials . . . . . . . . . 9.2.2 Spectral Factorization . . . . . . . . . . . . . Image Analysis with Smooth Wavelets . . . . . . . . 9.3.1 Approximation and Blurring . . . . . . . . . 9.3.2 "Naive7' Image Compression with Smooth Wavelets . . . . . . . . . . . . . . . .

9.1.1 9.1.2 9.1.3

9.2

9.3

IV

. . . . 250 . . . .

254

. . . . 257 . . . . 260

. . . . 264 . . . . 264 . . . . 269

. . . . 277 . . . . 278

. . . . 278

Other Wavelet Construct ions

10 Biort hogonal Wavelets 10.1 Linear Independence and Biorthogonality . . . . . . . . . . 10.2 Riesz Bases and the Frame Condition . . . . . . . . . . . . 10.3 Riesz Bases of Translates . . . . . . . . . . . . . . . . . . . 10.4 Generalized Multiresolution Analysis (GMRA) . . . . . . . 10.4.1 Basic Properties of GMRA . . . . . . . . . . . . . . 10.4.2 Dual GMRA and Riesz Bases of Wavelets . . . . . . 10.5 Riesz Bases Orthogonal Across Scales . . . . . . . . . . . . 10.5.1 Example: The Piecewise Linear GMRA . . . . . . . 10.6 A Discrete Transform for Biorthogonal Wavelets . . . . . . 10.6.1 Motivation from GMRA . . . . . . . . . . . . . . . . 10.6.2 The QMF Conditions . . . . . . . . . . . . . . . . . 10.7 Compactly Supported Biorthogonal Wavelets . . . . . . . . 10.7.1 Compactly Supported Spline Wavelets . . . . . . . . 10.7.2 Symmetric Biorthogonal Wavelets . . . . . . . . . . 10.7.3 Using Symmetry in the DWT . . . . . . . . . . . . . 11 Wavelet Packets

289 289 290 293 300 301 302 311 313 315 315 317 319 320 324 328

335 11.1 Motivation: Completing the Wavelet Tree . . . . . . . . . . 335 11.2 Locs.1iza.t.ion of Wavelet Packets . . . . . . . . . . . . . . . . 337 11.2.1 Time/Spatial Localization . . . . . . . . . . . . . . . 337 11.2.2 Frequency Localization . . . . . . . . . . . . . . . . 338 11.3 Orthogonality and Completeness Properties of Wavelet Packets . . . . . . . . . . . . . . . . . . . . . . . . 346 11.3.1 Wavelet Packet Bases with a Fixed Scale . . . . . . 347

xii

Conterlts B .1.2 Wavelets with Rational Noninteger Dilation Factors . . . . . . . . . . . . . . . B . 1.3 Local Cosine Bases . . . . . . . . . . . . . . B . 1.4 The Co~ltinuousWavelet Transform . . . . B . 1.5 Non-MRA Wavelets . . . . . . . . . . . . . B . l.G Multiwavelets . . . . . . . . . . . . . . . . . B.2 Wavelets in Other Domains . . . . . . . . . . . . . B.2.1 Wavelets on Intervals . . . . . . . . . . . . B.2.2 Wavelets in Higher Dimensions . . . . . . . B.2.3 The Lifting Scheme . . . . . . . . . . . . . B.3 Applications of Wavelets . . . . . . . . . . . . . . . B.3.1 Wavelet Denoising . . . . . . . . . . . . . . B.3.2 Multiscale Edge Detection . . . . . . . . . . B.3.3 The FBI Fingerprint Compression Standard

C References Cited in the Text Index

. . . . . 434 . . . . . 434 . . . . . 435 . . . . . 436 . . . . . 436 . . . . . 437 . . . . . 437 . . . . . 438 . . . . . 438 . . . . . 439 . . . . . 439 . . . . . 439 . . . . . 439

441

Preface These days there are dozens of wavelet books or1 the market, sonie of which are destiried t o be classics in the field. So a natural question to ask is: Why another one? I11 short, I wrote this book t o slipply tlie particular rieeds of students in a graduate course on wavelets that I have taught several tirnes since 1991 a t George Mason University. As is typica.1 with sllcli offerings, the course drew an audience with widely varying backgrounds and widely varying expectations. The difficult if not inlpossible task for me, the instructor, was t o present the beauty, usefiilriess. arid matliernatical depth of the sul~jectt o such an auclience. It would be insaiie t o claiin that I have been entirely successful in this task. However, through much trial ant1 error, I have arrived a t sorrle basic principles that are reflected in the stri~ctilreof this book. I believe that this makes this book distinct frorn existiilg text,s. and I hope that others may find the book useful. (1) Consistent assumptions of mathematical preparation. In sonle ways, the subject of wavelets is deceptively easy. It is not difficult to understand and implement a discrete wavelet trarlsforni and from there tlo analyzc and process signals arid irriages with great success. However, the underlying itleas and conrlectiorls that rrlake wavelets such a fascinating subject require some considera1)lc rriathematical sophistication. There have bceil some excellent books written on wavelets cnlphasizirlg their eleinelitary nature (e.g., Kaiser, A I;i-iendlv Glzide to Wavelt:ts; Strang and Nguyen, Wavelets and Filter Banks; Walker, P r i m ~ ron Wavelets and their Scientific Applications: Frazier, Irltroduction to Wavelets through Linear Algebra; Nievergelt, Ilitzvelets Made Easy; Ateyer, Wavelets: Algorithms and Applications). For rriy own purposes. such texts required quite a bit of "filling in the gaps" in order to make some conrlections and to prepare the student for rnore advanced books and research articles in wavelet theory. This book assuriies an upper-level undergraduate semester of advanced calculus. Sufficient preparation would corrle from, for example, Chapters 1-5 of Buck, Advanced Calculus. I have tried very hard not t o depart from this assumption a t any poirit in the book. This has required at times sacrificing elegance and generality for accessibility. However, all proofs are completely rigorous and conta.in the gist of the more general argunient. In this way, it is hoped that the reader will be prepared to tackle niore sophisticated books and articles on wavelet theory. (2) Proceeding from the continuous to the discrete. I have always found it more meaningful and ultirnately easier to start with a presenta-

xiv

Preface

tion of wavelets arid wavelet bases in the continuous dornain and use this to motivate the discrete theory, even thougli the discrete theory liarlgs together in its own right and is easy to understaiid. This can he frustrating for the student whose priniary int,erest is ill applications, but I believe that a better understanding of applications can ultirrlately be achieved by doing things in this order.

(3) Prepare readers to explore wavelet theory on their own. Wavelets is too broad a subject to cover i ~ai single book and is iriost interesting to study when the students have a particular interest in what they are studying. 111 clloosirig what to include in the book, I have tried t o ensure t,hat students are equipped to pursue more advanced topics on their own. I have included an appendix called Excursions in Wavelet Theory (Appendix B) that gives sorrie guidance toward what T consider to he t,he iriost readable articles on sorrle selected topics. The suggested topics in this appendix ca,rl also be used as the basis of serrlester projects for the students.

Structure of the Book The book is divided into five part,s: P~elirni7~n7..ies, Thc Hu,nr. Systcrrl, hfultiresol~~tion, Ar~nlysisand Orthonormal Wavelet Bases, Other Wavelet Constr.uctions, and Applications. Preliminaries Wavelet theory is really very liarcl to appreciate outside tlio context of t,hc language and ideas of Fourier Analysis. Chapters 1-4 of tlie book provide a background in sorrie of these ideas arid include cverytl.iing that is subsequerltly used in the text. These chapters are designed to be more than just a reference but less tllan a 'bbook-witliin-a-book" or1 Fourier analysis. Depenclirig on the background of the reader or of the class in which t,liis book is being used, these chapters are intended to t ~ edipped into eitllcr sliperficially or in detail as appropriate. Naturally there are a great rliarly books on Fourier analysis that cover the same rriaterial better and rnore thoroughly than do Chapters 1-4 and a t the sarne level (more or less) of Inathenlatical sophistication. I will list some of my favorites below. Walker, Fourier Analysis; Ka~nmler,A First Course in Fourier Analysis: Churchill and Brown, h u r i e r Series a i d Boundary Value Problems: Dym and McKean, Fourier Series a,nd Integrals; Korner, Fourier Analysis; and Benedetto, Harmonic Analysis and Applications. The Haar System Chapters 5 and 6 provide a self-contained exposition of the Haar systern. the earliest example of an orthor~orrrialwavelet basis. These chapters could

Preface

xv

be presented as is in a course on a d v a ~ ~ c ecalculus, d or an undergraduate Fouricr analysis coursc. In the context of the rest of the book, these chapters are designed to motivate the search for rrlore general wavelet bases with different properties. and also t o illustrate some of the more advanced concepts such as multiresolutiorl analysis that are used throughout the rest of the book. Chapter 5 contains a description of the Haar basis on [O, 11 and on R, and Chapter 6 shows how t o implernerlt a discrete version of the Haar basis in one and two dimensions. Some exa~rlplesof inlages analyzed with the Haar wavelet are also included.

Ort honor~nalWavelet Bases Chapters 7-9 represent the heart of the book. Chapter 7 contains an exposition of the general notion of a multiresolutiorl analysis (MRA) together with several examples. Next, we describe the recipe that gives the construction of a wavelet basis from an MRA, and then construct corresponding cxarrlples of wavelet orthonorinal bases. Chapter 8 describes the passage from the continuous domain to the discrete domain. First, properties of MRA are the11 used to niotivate and define the quadrature mirror filter (QMF) conditions that any orthonormal wavelet filter must satisfy. Then the discrete wavelet transfornl (DWT) is defined for infinite signals, periodic signals, and for finite sets of data. Finally the techniques used to pass from discrete filters satisfying the QhlF conditions to continuously defined wavelet functions are described. Chapter 9 presents the cor~struction of compactly supported orthornomal wavelet bases due to Daubechies. Daubechies's a.pproa.ch is motiva.ted by a, lengthy disclissiori of the importance of vanishing moments in the design of wavelet filters.

Other Wavelet Constructions Chapters 10 and 11 contain a discussion of two inlportarlt variations on the theme of the const,ri~ct,ion of orthonormw,l wavelet, ba.ses. The first. in Chapter 10, shows what happens wlleri you consider nonorthogonal wavelet systems. This chapter contains a discussion of Riesz bases, and describes the serni-orthogonal wavelets of Chui and Wang. as well as the notion of dual MRA and the fully biorthogonal wavelets of Daubechies. Cohen. arid Feauveau. Chapter 11 discusses wavelet packets. another natural variation on orthnorrnal wavelet bases. The motivation here is to consider what happens to the DWT when the '.full wavelet tree" is conlputed. JVavelet packet functions are described, their time and frequency localizatio~iproperties are discussed, and necessary and sufficierit conditions are give11 u n d e ~which a collection of scaled and shifted wavelet packets constitutes an orthonormal basis on R. Finally, the notion of a best basis is described. and the so-called best basis algoritliln (due to Coifman and Wickerhauser) is given.

xvi

Preface

Applications Many wavelet books have been written emphasizing applications of the theory, most notably, Strang and Nguyen, Wavelets and Filter Banks, and Mallat's comprehensive, A Wavelet Tour of Signal Processing. The book by Wickerhauser, Applied Wavelet Analysis from Theory to Software, also contains descriptions of several applications. The reader is encouraged to consult these texts and the references therein to learn more about wavelet applications. The description of applications in this book is limited to a brief description of two fundamental examples of wavelet applications. The first, described in Chapter 12, is to image compression. The basic components of a transform image coder as well as how wavelets fit into this picture are described. Chapter 13 describes the Beylkin-Coifman-Rokhlin (BCR) algorithm, which is useful for numerically estimating certain integral operators known as singular integral operators. The algorithm is very effective and uses the same basic properties of wavelets that make them useful for image compression. Several examples of singular integral operators arising in ordinary differential equations, complex variable theory, and image processing are given before the BCR algorithm is described.

Acknowledgments I want t o express my thanks t o the rnany folks ~ 1 . 1 0made this book possible. First and foremost, I want to thank my advisor and friend John Benedetto for encouraging me to take on this project and for graciously agreeing to publish it in his book series. Thanks also to Wayne Yuhasz, Lauren Schultz, Louise Farkas, and Shosharina Grossman at Birkhauser for their advice and support. I want to thank Margaret Mitchell for LaTeX advice and Jim Houston and Clovis. L. Tondo for modifying some of the figures to make them more readable. All of the figures in this book were created by me using MATLAB and the Wavelet ToolBox. Thanks to the Mathworks for creating such superior products. I would like also t o thank the National Science Foundation for its support and to the George Mason University Mathematics Department (especially Bob Sachs) for their constant encouragement. I also want to thank the students in my wavelets course who were guinea pigs for an early version of this text and who provided valuable feedback on ~rganiza~tion and found numerous typos in the text. Thanks t o Ben Crain, James Holdener, Amin Jazaeri, Jim Kelliher, Sami Nefissi, Matt Parker, and Jim Tirnper-. I also want t o thank Bill Heller, Joe Lakey, and Paul Salamonowicz for their careful reading of the text and their useful comments. Special thanks go to David Weiland for his willingness to use the manuscript in an u~ldergraduatecourse at Swarthmore College. The book is all the better

Preface

xvii

for his insights, and those of the unnamed students in the class. I want give special thanks to my Dad, with whom I had many conversations about book-writing. He passed away suddenly while this book was in production and never saw the finished product. He was pleased and proud to have a ~ ~ o t h published er author i11 the family. He is greatly missed. Finally, I want t o thank my wife Megan for her constant love and support, and my delightful children John and Genna who will someday read their names here and wonder how thcir old man actually did it.

Fairfax, Virginia

David F. Walnut

Albrecht Diirer (1471-1528), Melencholia I (engraving). Courtesy of the Fogg Art Museum, Harvard University Art Museums, Gift of William Gray from the collection of Francis Calley Gray. Phot,ograph by Rick St,a,fford, @President and Fellows of Harvard College. A detail of this engraving, a portion of the magic square, is used as the sample image in 22 figures in this book. The file processed is a portion of the image file detail.mat packaged with MATLAB version 5.0.

Part I

Preliminaries

Chapter 1 Functions and Convergence 1.1 Functions I . I . 1 Bounded

(L")Functions

D e f i i t i 1 . 1 A pzece,wise C O ~ I L L ~ ~ ~ ~ L U O ~~ UUS, I L C ~ ~ , Of T( Z5 ) defined 01%a n interval I is bounded ( o r L m ) o n I i f there i s a number A1 > O such that If (z)l 5 A1 for allx E I. T h e L m - n o r m of a functzor~f ( 2 ) i s defined b y IlfI2 = s 1 1 p { l f ( x ) I : : rE I ) .

(1.1)

Example 1.2. (a) If I is a closed. firlit,e interval, tlien nriy fil~lctio~l f (cc) contiriuous on I is also Lm 011 I (Theorern A.3). (b) Tlie fiirictiori f (z)

=

1/x is corltinuous and has a finite value at each

point of the irlterval ( O , l ] but is not bounded on (0, 11 (Figure 1.1).

(c) The functiorls f (x:) = sin(z) arid f (z) = cos(z) are Lm on R. Also. the complex-valued function f (z) = eix is Lm on R. 111 fact, 11 sin 1 1 , = 11 cos 1 1 , = Ileirllm = 1.

(d) Ariy polynornial f~lrictionp ( z ) is not Lw on R but is L" on every finite subinterval of R. (e) Any piecewise coritinllous fiirlctiori with orily jump discontirinities is L" ariy firlite interval I.

011

I . 1.2 Integrable (L1) Functions Definition 1.3. A piecewise continuous function f ( x ) &finned o n n,n, i,n,terual I is integrable ( o r of class L' o r simply L') o n I if the integral

is finite. The L'-norm of a function f ( x ) i s defined b y

4

Chapter 1. Functions and Convergence

FIGURE 1.1. Left: f (x) = 1/x is finite-valued but unbounded on (0, 11. Right: sin(x) (solid) and cos(jc) (dashed) are L" on R.

Example 1.4. (a) If f (z) is LW on a finite interval I, then f (z)is L1 on I. (b) Any function continuous on a finite closed interval I is L' on I . This is because such a. function must be Lm on I (Theorem A.3). ( c ) Any function piecewise continuous with only jump discontinuities on a finite closed interval I is L' on I.

(d) For any 0 < a < 1, the function f(z)= Izl-" is L1 on the interval [- 1, I]. Clearly f (z) is picccwise continuous with a11 i~~firiite discontinuity 1 ( f(x)ld z is improper and must be evalua.t,ed at x = 0. Thus the integral as an improper integral a,s follows: Is/-"dz

-

-E

12)"

ds

1

+ lirn

/xi-" dl:

E+O

1

1-a

lim (1 -

E-+O

=

2

The above example shows that an L1 function need not be Lm. If a then f(x) is not L1 on [-I, I]. ( e ) If a > 1, the function f (x) = xp" is L1 on the interval improper Riemann integral

converges.

> 1,

[I,ocl) since the

1.1. Functions

5

( f ) If 0 5 a 5 1, then f (x)= xpLYis not L1 on [ I ,co).But f (x)is L" on [I,co).This shows that an Lm function need not be L' on I if I is infinite. (g) The function f (x)= e-ixl is integrable on R since the improper Riemann integral

1, 00

e-1.1

converges. In fact,

JTme

x dx

d.

= 2.

We present below our first approximation theorem. It says that any function L1 on R can be approximated arbitrarily closely in the sense of the L1-norm by a function with compact support. Theorem 1.5 is illustrated in Figure 1.2.

Theorem 1.5. Let f (rc) be L' o n R, and let E > 0 be given. T h e n there exists a number R such that 2f

Then

Proof: Since f (x) is integrable, the definition of the improper Riemann integral implies that there is a number v such that rr

Hence, given

E

> 0, there

Pick a nurrlber R

Then,

roo

is a number ro

> 4.0, and define

> 0 such that

if r

> ro, then

6

Chapter 1. Functions and Convergence

FIGURE 1.2. Illustration of Theorem 1.5. Left: Graph o f f (x). Area of shaded region is < F. Right: Graph of g(x) with R = 10.

Definition 1.6. A piecewise continuousfunction f (x) defined o n a n interval I is square-integrable ( o r of class L' or simply L') on I i f the integral

i s finite. The

norm of a function

f (x) is defined by

Example 1.7. (a) Any function bounded on a finite interval I is also L2 on I. This includes functions continuous on closed intervals and functions piecewise continuous on closed intervals with only jump discontinuities. (b) Any function that is La and L1 on any interval I (finite or infinite) is also L~ on I.

(c) For any 0 < a < 112, the function f (x) = x-" is L2 on the interval [-I, 11. Therefore an integrable function need not be bounded. If a 112, then the corresponding f (x) is not L2 on [-I, 11.

>

(d) If a > 1/2, the function f (x) = xp" is L2 on the interval [I,m ) . If 0 Q 1/2,then the corresponding f (x) is not L2 on [I,m ) .

< <

Theorem 1.8.

(Cauchy-Schwarz Inequality) Let f (z) and g(z) be L~ o n the

1.1. Functions

7

interval I . Then

Proof: Let us assume first that f (x)and g(x) are real-valued, and let t be an arbitrary real number. Then

This expression represents a real-valued quadratic function of t that is non~legativefor all t . Hence its discriminant must be rlonpositive (the discriminant of a quadratic expression at2 + bt + c is b2 - 4ac). Therefore,

and (1.5)follows. If f (z)and g(x) are not real-valued, then we observe

(Theorem A.4) and then proceed as in the real-valued case.

C]

From the Cauchy-Schwarz inequality, we can say something about the relation between L1 and L~ functions.

Theorem 1.9. Let I be a finite interval. Iff (x) is L2 o n I , t h e n i t is L1 o n I.

Proof: By the Cauchy-Schwarz inequality,

, right side and hence Since I is a finite interval and since f (x) is L ~ the the left side of the above inequality are finite.

8

Chapter 1. Functions and Convergence

Remark 1.10. (a) The conclusion of Theorem 1.9 does not hold if I is l / x if z 1 an infinite interval. For example, let f (x) = Then f (x) 0 ifx<1. is L~ on R but is not L1 on R.

>

(b) The converse of the Theorem 1.9 is false for both finite and infinite intervals. This means that if f (z) is L' on I , then it is not necessarily true that f ( x ) is L2 on I. For example, let f (x) = x-'I2 for x E ( 0 , l ) . Then f (x) is L1 but not L2 on ( 0 , l ) . The following consequence of the Cauchy-Schwarz inequality allows us to conclude that linear combinations of L~ functions are also L~ (Exercise 1.19). It is also known as the triangle inequality for L~ functions; that is, it says that J J f 9 ) J 25 J J f 112 J I g ) ) 2(see Exercise 1.18).

+

Theorem 1.11. internal I . Then

+

(Minkowski's Inequality) Let f ( x ) and g ( x ) be L 2 on t h e

Proof: By the Cauchy-Schwarz inequality,

and (1.6) follows.

The following approximation theorem says that a function f (z) that is

L2 on R can be approximated arbitrarily closely by a compactly supported

no norm.

function in the sense of the The proof is very similar to that of Theorem 1.5 and is left as an exercise (Exercise 1.20).

Theorem 1.12. Let f(x) be L~ on R, and let E > 0 be given. Then there ), exists a n u m b e r R such that i f g ( x ) = f ( x )X I - ~ , ~ ] ( Xthen

1

00

If

( x ) - s(x)12dx =

llf

- gill

< r.

1.1.Functions

1.1.4

9

Dzflerentiable (Cn)Functions

Definition 1.13. Given n E N , we say that a function f (x) defined o n a n interval I is C n on I if it is n-times continuously diflerentiable o n I . C0 o n I means that f (x) is continuous o n I . f (x) is C m o n I if it is Cn o n I for every rc E N .

W e say that f (x) is C," o n I if it i s Cn o n I and compactly supported, C,O o n I if it is C0 o n I and compactly supported, and C y o n I if it is C" o n I and compactly supported.

Example 1.14. (a) If p(x) is a polynomial, then p(x) is Cm on R. (b) The "hat" or "tent" function defined by f (x) = (1 - 1x1) Xr-l,ll (x) is continuous but not differentiable on R since the derivative fails t o exist at x = -1, 0, 1. Thus f (x) is C0 (in fact C:) but not C1 on R (Figure 1.3(a)). (c) We can generate an example of a snloother function by taking the tent function f (x) defined in (b) and taking its antiderivative. That is, we can define the function g(x) by

Putting -m as the lower limit of integration is just a notational convenience since we could just as easily have started the integration a t any constant less than -1. By the Fundamental Theorem of Calculus (Theorem A.5)) g f ( x ) = f (x) for all x E R since f (x) is colltirluous on R. Therefore g(x) is C1 but not C? Note that g ( x ) is not C,' on R. (Figure 1.3(b)). (d) The function g(x) defined in (c) is L" on R but is not L1 or L~ on R. In fact, since g(x) = 1 for all x > 1, g(x) does not even go to zero as x goes t o co.However, by modifying the construction in (c), it is possible to define a function that is C: but not C2 on R. In particular, such a function would be L1 and L2 on R. The idea is to form two shifts of f (x), and subtract one from the other so that the resulting sum has integral zero. Then the antiderivative will vanish after a certain point. Specifically, define the function F(x) by

and the function g(x) by

S(x) =

J(t) d t .

Then g(z) is C1 but not C2 for the sarne reason that g(x) is, and F(x) is CE since it is supported in [-3/2,3/2] (Figures 1.3(c) and (d)).

10

Chapter 1. Functions arid Convergerlce

(e) It is possible to define a sequence of functions with irlcreasiilg snloothness based on the procetlure outlined in (d). First define the function B o ( x ) by BO(x) = )C;-1/2,1/21 (x),and for each n E N. define B,,( 2 : ) by ..I'

(

=(

B l - l ( t

+ 1/21 - (

t - 1/2)) dt

=

J

.I,+1 / 2

1.-1/2

,

B,,- ( t )d t .

Note that B l ( x ) is exactly the tent function defined in (I)). Each B,, (x) vanishcs outside the interval [ - ( ? I 1)/2. (rz + 1)/2] and for n 2 1 is Cn-l but not C" on R (Bo(.r) is not C O ) The . functioil B,,(J.) is called the Bsplinr of order I L a i d is ill fact a pieccwise poly~loilliiil.hlore int,erestirlg properties of spliiie functions arc give11 in Scctioii 7.7.1.

+

FIGURE 1.3. Top left: Graph of "hat" function f ( r )= (1- 1x1) (x). Top right: Antiderivative of f ( s ) . Bottom left: Graph of f (x 112) and - f ( x - 1/2). Bottom right,: Antiderivative of f(s). This function is C: on R but not C'.

+

Exercises Exercise 1.15. Prove each of the statcnients made in Example 1.2.

1.2. Convergence of Sequences of Functions

11

Exercise 1.16. Prove each of the statements rnade in Example 1.4. Exercise 1.17. Prove each of the statements made in Example 1.7. Exercise 1.18. Prove that each of the norms 11 - I / , for p = 1, 2, oo satisfy the norm axioms: Given f (x) arid g ( x ) LP on an interval I: (a)

llf llp 2 0.

(b)

(1 f ( 1 ,

(c)

IIQflip = IQI If ( l p , for every

(d)

11 f' + g1Ip 5 1 flip + llgllp (this is known as the triangle inequality).

=

0 if and only if f (x) = 0. Q

E C.

Exercise 1.19. Prove that the collection of functions Lp on an interval I (p = 1, 2, oo) is closed under the forrriation of linear combinations. That is, show that if f,,(z) is LP 011 I for 1 5 n < N and if a l , (22, . . . , CYN are complex numbers, then a,, f,,(x) is also LP on I.

c:==,

Exercise 1.20.

Prove Theorem 1.12.

1.2 Convergence of Sequences of Functions 1.2.1 Numerical Convergence Definition 1.21.

T h e sequence { a , , ) , , E ~converges to the nurnher a if for every c > 0, there is a n N > 0 such that if n N , then (a,, - a1 < t. In this case, we uirite a , t a as n t m, o r lim,,,, a,, = a . A numerical series, denoted a,,, convcrgcs to a number S i f the sequence N of partial sums { S N ) N € N defined b y s~ = E n = l an converges t o S . I n this case, we write E F = I an = S. W e will frequently denote the series a, b y

>

zr=l

xr=,

-

L n E N

A series

CnEN a , converges absolutely if EILEN la,,I converges.

Remark 1.22. (a) A fundamental property of the real numbers is known as the completeness property. The completeness property for the real numbers says that every set of real numbers bounded above has a supremum (or least upper bound). (b) A sequence of real numbers {an)nENis an increasing sequence if a, 5 a,+l for all n E N. The completeness property of the real numbers implies

"

'Since we assume that f (a)is undefined if f (x) has a discontinuity at x = a,then f (x) s On means "f (x) = 0 a t each point of continuity of f (x)."

12

Chapter 1. Functions and Convergence

that any bounded, increasing sequence of real numbers always converges to its least upper bound. a,, The partial sums of a series with nonnegative terms, (i.e., CnEN where a, 0) form an increasing sequence. Therefore, it follows that if a series of nonnegative terms is bounded, then it converges.

>

(c) A sequence of real or complex numbers is Cauchy if for every e > 0, there is an N > 0 such that if n , m _> N , then la, - a,l < t. Another consequence of the conlpleteness property for real numbers is the following: Every Cauchy sequence of numbers converges.

Example 1.23. (a) Consider the series quence can be computed since

If Iri < 1, then

SN

-+ 1/(1

-

r ) as N

Cr--nr n . The partial ,-

+ 00.Therefore, if

Irl

surn sc-

< 1, then

(b) Consider the series C,q"=l l / n ? Clearly,

Therefore,

for all N. Since each of the terms l / n 2 is positive, { s N l N E Nis a bounded, increasing sequence. Therefore, it converges t o its least upper bound and the series Cr=l1/n2 converges. Note that we have proved that the series converges but we have made no statement about the iralue of its limit. The same argument can be used to show that the series C:=l l/nP converges for every p > 1, but again does not give the value of its limit. (c) The Weierstrass M-test is a well-known test for convergence of a series. Consider the series En a,. The Weierstrass M-test says that if la, 1 5 b, for all n and if CntNLT1 converges, then CTrEN an converges. For example, consider the series cos(n)/n2. Since 1 cos(n)/n21 5 l / n 2 for all n and since Cr=,1/n2 converges, so does C:=, cos(n)/n2. Note that again we have proved that the series converges but have not given the value of its limit.

,,

En=,

1.2. Convergence of Sequences of Functions

13

(d) A consequence of the Weierstrass M-test is the following. If a series converges absolutely, then the original series also converges. Absolute convergence is equivalent to saying that the series converges regardless of the order in which the terms are summed. It is not true that all convergent series are absolutely convergent. For example, it is shown in most calculus books that the series x~=,(-l)"/71 converges but the harmonic series C z = l l / n does not. In dis(e) A doubly infinite sequence is a sequence of the forrn cussing the convergence of such sequences, we look at two lirnits, namely, linl,,-too an and lim,,,, a_,. If both converge to the same number, say, to a , then we write lirnl,,l+, a,,= a.

(f) A doubly infinite series is a series of the forrn C r = - m a,. In discussing cc the convergence of such series, we look at two series, rlxnlely, Cr,=, arband 00 El,=,a P n. If both of these series converge, then tliere is no problem. If C T00L =a, , = St and C:IP=, a p r 1= S ,then Cr=-, a,, = on+St+S- = S . In this case, we write also l i ~ n ~ , ~ ~ , ,r = a,,- =~S. ~ We will frrqllently denote the series a,, by C r L E artZ 01. simply by C,,a,, .

xr=-,

(g) If a doubly infinite series converges absolutely, then it converges regardless of the order in which the terms are surnmed. This is not tkic casc with series that do not convergc absolutely. Consider the series l / n ,where the n = 0 term is understood to be zero. Clearly, this series does not conN verge absolutely. However, because of cancellst ion, s~ = C rl=- 1/71 = 0. Hence, the sgmmetric partial sums corivergc to zcro. However, if we define

x:=-,

N2

S

N

>

~

1/x dz

= 111 N

+ oo

as N + ca. Therefore, if a doubly infinite scries does rlot converge absolutcly, then the form of the partial sums rnust be givcrl explicitly in order to discuss the convergence of the series. This is true of any series that converges but not absolutely.

2

. Pointwise Convergence

Definition 1.24. A sequence of functions {f, ( x ) ) ~ defined ~ ~ N o n a n interval I converges pointwise t o a function f (x) i f f o r each xu E I , the numerical sequence { f n ( x O ) ) n E N converges t o f (xu). W e write f,(x) + f (x) pointwise o n I, as n

4 00.

T h e series

Cr==l l T L ( x= ) f (x) po%nt,wiseo n a n inter~1alI

i f ,for each

XU

E I,

C:y, fn(x0) = f (20). Example 1.25. (a) Let f,,(x) = zn, x E [0,1) for all 71 E N. Then fn(z)+ 0 poilltwise 011 [O, 1) as 12 + a. See Figure 1.4(a).

14

Chapter 1. Furlctiorls and Convergence

(b) Let

2n:c 2 - 2nz 0

+ 0 pointwise on

Theri f,,(.r.)

ifx~[O.l/2n) if T E [1/2n,l / n ) if x E [ l / n , 11.

[0, 11. See Figure 1.4(1-I).

(c) Lct

2n2x

Tlle11 f ,,( J : )

+ 0 pointwise oil

if z E [O, 1/211) if x E [1/21~,l / n ) if x E [ l / n , 11.

[0, 11.See Figlil-t. 1.4(c)

1

zr' = -poir~twiseon (- 1. 1).

(d) The series

1-

r,=O

3(,

COS 'I).[.'

- converges poiritwise on R to its liniit by the

( f ) Tlie serics

11,2

00

COS 7) :X'

-- coilvergcs at odd lrlult,iplesof

( g ) The series r,=l

;.r

(since it reduces

'It '

t o thc: :tltcrnatirig scrics C r = l ( - l ) " / n ) but divergcs a t even m~iltiplcsof sr (since it rcduccls t o the liarrr~o~lic series). In h c t . it can be shown that t h e series coiivcrgcs for all .c that are riot ever1 i ~ i u l t i p l ~ofs K .

1.2.3

Unrifornl ( L m ) Convergence

Definition 1.26.

T h e sequc,rLce { f,, (z)},, E~ converges ~lrliforrnlyon I to t h e hirlctiorl f (.c) i f for r:uery c > 0, t h ~ v is : u n N > 0 .such that i f 11 2 N , t / ~ e , r ~ 1 f,L(x)- f (r)I < 6 for all z E I . We write f r L ( . r )+ f ( x ) ~ n i f o , r m l yo n 1 ass 71 + m . T h e series f , , ( z ) = f ( z ) u n i f o m l ~o n I zf the sequence of partial sums SN(X)

=

xr=,

zit=l fn N

( 2 ) converges

~ ~ n i f o m to l y f (x) on I .

Remark 1.27. (a) With uniforl~lconvelgence. for a given E the sanie N works for all x E I, whereas with pointwise Convergence N may depend on both F arid x. In other words, unifornl convergerice says that given E > 0 there is an N > 0 such that for all n ',N , the maximum difference between f,,(x) and f ( J ) on I is smaller than E . Because of this. uniform convergence

1.2. Convergence o f Sequences of F~lnctions

15

FIGURE 1.4. Top Left: Graph of , f , , ( x ) = .I." on [0, 1) for 11 = 2 , 4.8. Top Right: Graph of f,,(:x:) on [ 0 ,1 ) where f , , ( . ~ : is ) defined in Example 1.25(b). Bottom: Graph of f , , ( x ) on [O. 1 ) where f , , ( s ) is defined in Example 1 . 2 5 ( ~ ) .

is also called LX con?)f>.rgcJncr. Tliat is, f,,(.r.) only if 11 f,, f ,1 + O as 11 + 3 ~ .

--+

f(.r.)

unifori~~ly on I if ant1

-

(11) 111 Exainple 1.25(1)), thr. coilvcrgcnce of' f,, (s)t o 0 is pointwisc but not uniforrn. This is t)ecausc the nlaxirllur~~ cliffcrerlcc between ,f,,(r)ant1 t,hc liniit filrlctiol~f (1:) = 0 is 1 110 ma.t,t,e~. what I , is. 111 otller words. 11 f r ,- f I m = 11 fr,llno = 1 for all n,, aild so I f r ,- f 11% f i O as 11 + m-

(c) 111 Exall~ple1.25(c), t,llc convergc~llrbe of f,, (.I.) t o O is also poiiltwisc 1)ut not ~inifornl.111 fact, in this cast‘, 11 f,, - .f llcx: = Ilf,, llx = 7 1 h r a11 1 1 . Tliereforc 11 f,, - f ,1 + cc as n + m. 111fact,. there. an.c3 110 exainplcs of sequences that converge ul~iforrrllyon a n interval but not poiiltwise. 111 otlier words, the followil~gtheorel11 11olds.

Theorem 1.28. I f f , , (.r) + f

(.c)

in L"

071

a n ~ntcmnlI . then f,,(.r)

+ f (z)

poi7~t7uise an I .

Proof: Exercise 1.44.

An irllportai~ttheorcn~fro111 ;idvailced calc111usis t,he followillg. Its proof

16

Chapter 1. Functions and Convergence

is left as an exercise but can be found in almost any advanced calculus book (for example, Rurk, p. 266, Theorem 3).

Theorem 1.29, If fn(x) + f (x) unzformly on the interval 1 , and if each f , , ( x ) is continzious on I , t h . ~ nf , (z) is con,tin.uo?sson I .

Proof: Exercise 1.45.

Example 1.30. As an illustration of Theorem 1.29, let

Then each f n (x) is continuous on [- 1,1]and { frL( x ) } , , converges ~~ pointwise to the function f ( z ) defined by

which has a jump discontinuity at x = 0 (see Figure 1.5). It car1 be shown directly t,hat f,,(z) does not converge to f (z) in Lm on [-I, 11,but a different argument utilizing Theorem 1.29 would be as follows. If f l L (x) + f (z)in Lm on [-I, 11, then since each f, (z) is continuous, Theorem 1.29 irmplies that f (z)shou,ld also be continuous. Since this i s not

the case, the convergence cannot be i n La.

Example 1.31. (a) The sequence { x n I r L E N converges iiniformly to zero on [-a,a] for all O < a < 1 but does not converge uniformly t o zero on (-111). -

~

(b) The series

1 I" = - uniformly

1-x

n=O

on [-a, a] for all 0

< a < 1, but

not on (- 1 , l ) . 30

(c) The series

1n!

IC -=

ex uniformly on every finite interval I, but not on

n=O

R. rX,

cos n x

- converges

(d) Tlie series n=l

Weierstrass &!-test.

n . .2

uniformly t o its limit on R by the

1.2. Convergence of Sequences of Functions

FIGURE 1.5. Left: Graph of f,,(x) of Example 1.30 for Right: Graph of the limit function f (.r).

1.2.4

71.

17

= 2, 4 , 8.

Mean ( L 1 )Con,veq:rlmce

Definition 1.32.

T h e sequence { f , , ( in ruean to the function f ( s ) 071, I if

x ) ) , ~ ~defined N

o n a n interval I coilverges

W e write f,,(x)+ f(r) in mean orL I a s TL + a. Mean convergence is also referred to as L' convergence because f,,( x ) + f (.r) in m e a n o n I as rl + oo is identical t o the staternent that lim,,,, 11 f,,- f 111 = 0 . The series fi,(x)= f (s) in rnean o n I if the seyuer~ceof yar.tir~lsums N S N (n.) = f,, (x) converges i n mean to f (x) 0 7 , I .

Cr=, CIL=,

Llean convergence can be interpreted as saying that the area between the curves y = f,,(z)and y = f (z) goes t o zero as 7 1 + oc. This type of convergence allows point values of f,,(.~.) and f (x) to differ considerably but says that o n averagc t,hc functions f',,(x) and f ( z ) are close for large n.

Example 1.33. (a) Let J;,(le) = .c". :r* E [O,1)for all r~ E N. As we have seen in Example 1.25(a),this sequence converges to f (z) = 0 poirltwisc on [O, 1) but not uniformly on [0, 1). Since

as n

+ oo:f,,(z)+=O in mean on [o. 1).

18

Chapter 1. Functions andconvergence

-

(b) Consider the sequence {fn(z)),,EN defined in Exarnple 1.25(b). The sequence converges pointwise but not uniformly t o f ( z ) 0 on [0, 11. Since the area under the graph of f,(z) is 1/2n for each n. the sequence also converges in mean t o f (z) on [0, 11. In this example, we can see the character of mean convergence. If n is large, the function f,(z) is close to the limit function f (z) 0 (in fact identical to it) on most of the interval [ O , l ] , specifically on [1/n, 11, and far away from it on the rest of the interval [O. 1/72?. However, 011 average, f, (z) is close t o the lirnit function.

-

(c) The sequence { f , ( ~ ) )defined , ~ ~ in Example 1.25(c) tells a different story. The sequence converges pointwise but not uniformly to f (z) Y 0 on [O. 11, but since the area under the graph of f,, (x) is always 1, f,(z) does not converge to f ( z ) in mean. The width of the triangle under the graph of f (z) decreases t o zero, but the height increases to infinity in such a way that the area of the triangle does not go t o zero. The above examples show that sometimes pointwise convergence and mean convergence go together and sometimes they do not. The proof of the followirlg theorern is left as ail exercise (Exercise 1.47).

Theorem 1.34. If f,(x)

-+ f (x) in Lm

on n finite interval I , then f,(x)

+

f ( x ) in L' on I .

Remark 1.35. (a) The conclusion of Theorem 1.34 is false if the interval I is infinite. Consider fur exarrlple the sequence f , (x) = ( l / n )X [ o , n (x). ~ Then f,(s) i 0 in L" on R but Jyx I f , (z) - 01 dx = 1 for all n, so that f,(z) does not converge to zero in L1. (b) The converse of Theorem 1.34 is also false, as can be seen by considering Example 1.33(b). In this example, f,,(x) convcrges to 0 in L' on [ O , 1 ] but does not converge to 0 in L" on [0, 11. (c) In all of the examples of mean convcrgcncc considered so far, the sequences have also converged pointwise. hfust this always be the case? The by the following example. answer turns out t o be "no," as is ill~st~rated

Example 1.36. Define the interval I i . k by I,i.k = [2-jk, 2 - J ( k + I ) ) , for j E Z+ and 0 5 k 5 2J - 1. Let us make some elementary observations about the intervals I j , k . (a) Each

Ij.k

is a subinterval of [0, 1).

(b) The length of

is 2-3; that is, I I i ~ k= 2 - j .

(c) Each natural number n corresponds t o a unique pair (j,k), j E Z+ and 0 k 5 2.7' - 1, such that n = 2.7' + k. For each n E N, call this pair (j,, , k,). As n --+ co,j, -+ co also.

<

1.2. Convergence of Sequences of Functions

19

{I~.~}:L~'

(d) For each j . the collection of intervals forms a partlition of [O, 1);that is, the intervals are disjoint and cover all of [O; 1).

-

Now, define f , ( z ) = X I J I L , k r L( z ) . Then since I I j 3 , , k r L1 + O as j oo, f n ( z ) + O in mean on [O,l). However, f,(z) does not converge t o zero pointwise because for every x E [0, I), there are infinitely many n for which f,(z) = 1. Therefore, f,(z) does not converge to anything at any point of [0,1). See Figure 1.6.

FIGURE 1.6. Graph of f n ( x ) of Example 1.36 for 1 5 n 5 12.

1.2.5 Mean-square ( L 2 ) Conuergence Definition 1.37. T h e sequence {~,(X)),~~N converges in mean-squa.re to t h e function f (x) o n a n interval I 2f lim n--1s

If,(l) - f(5)12dz

= 0.

W Pvrnte f,(x) + f (x) i n mean-square on I as n + m. Mean-square convergence is also referred to as L~ convergence because f, (x) + f ( z ) i n mean o n I as n + cc is equivalent to the statement that limn,, IIj, f 112 = 0.

20

Chapter 1. Furictions and Convergence

T h e series Cr=,f,(x) = f ( x ) i n mean-square o n I zf the sequence of partial N f n ( 5 ) converges in mean-square to f ( x ) o n I . s u m s s N ( x )=

The proof of the following theorem is left as an exercise (Exercise 1.48). The proof of the first part is similar t o the proof of Theorern 1.34 and the proof of the secvilcl part ~llakesuse of the Caucl~y-Schwarziriequality (Theorem 1.8).

Thcorcm 1.38. (a) If fTL(:c) + f (x) i n La o n a finite interval I , then f n ( x ) + f ( x ) in L~ o n I. ( b ) If f,,(x) I.

+f

( x ) in L~ o n n finite interval I , then f,,(z-)

+f

(x) in L' o n

Remark 1.39. (a) The concl~isionof Tlleorerr~1.38(a) is false if Lm convergerice is replaced by poiritwise convergence. Example 1.33(c) shows a sequence that converges t o zero poirltwise on [O, 11 hiit not in L1 on [r), 11. By Theorem 1.38(b), the sequence does riot coriverge in L2 either, for if it did, then Tl~eorerr~ 1.38(b) would imply that it also converged ill L1.

(b) T11c corlclusion of Theoreni 1.35(a) is false if I is an infinite interval. For exarnple, if f;,(z) = ( 1 / f i ) X,03,,l(z), then f,, (x) + 0 uniformly on R, .rX! tit , 1 j ( ) - 012 tlz = 1 for all iz, so that f (x)does not converge to 0 in L2. (c) Tllc co~iclusionof Tl~eorern1.38(b) is false if I is an infinite interval. For example, if f ,,(x) = (1In) X,,,, ,,](z), tlien since

as n

+ m, f, (x) + 0 in L2 on R. but

for all n so that f,,(z) does riot converge t o 0 in

LI

on R.

(d) The converse of Theorem 1.38(a) is false. To see this, consider Exarnple 1.33(b). In this exarnple, it car1 be shown directly that f,,(z) + 0 in L2 on [ O , 1 ] (Exercise 1.49). However, f,,(z) does not converge to 0 in L" on [o: 11. (e) The converse of Theorerri 1.38(b) is false. To see this, let f,,(z) = J n X ( ~ , l / n(x). ] The11

1.2. Convergence of Sequences of Functions

as n

+ co so that

21

f,,(x)+ 0 in L1 on (0.11. However,

for all n so that f,, (x) does riot converge t o 0 in L2 or1 (0,1]

( f ) Finally, note that Exarriple 1.36 shows that L2 convergence does not irnply pointwise convergence since the sequence defined there also converges to 0 in L~ on [0, 1).

I . 2.6 Interchange of Lim,it.s and In,tegro,l,s A problerrl that we will encounter frequently in this book is the following. Slippose that a sequence of functions { f,,( x ) ) ,on~ an ~ interval ~ I converges in some serise described in one of the previous four subsections t o a fiirlction f (z)on I. Under wliat coiiditiorls is it true t,hat

Silicc we call write f ' ( z ) = lim,,,,

f , , ( . r ) .the above call t)e rewritt,erl as

and this problern is often stated as: W h e n can 7ue e3:chnnge the limit and

the integral? Tlie rliost typical forrn in which this problcrrl nriscs is wllcil the sequence is a sequence of partial surns of a series of fiinctions. In tliis case, thc cquivtilent question is: W h e n cart we intc:y.atc? a series c!f ft1,nctions termhy-term,? To see this, recall that the integral of an finite surri of functiorls is tlie sllrrl of the integrals. so that if sN(z)= C , , = , f,,(.x), then

J

If we could interchange tlie limit and the integral in this case, we would

lim

n=l

N+m

SN

(x)dx

Chapter 1. Functions and Convergence

22

The following theorem gives several conditions under which interchanging the limit and the integral is permitted. Theorem 1.40. (a) If f , (x)+ f (x)in L I

on I , t h c n

(h) If f,, (x)+ f (x)i n La o n

(c)

a, finite

V fTl(x)+ f (x) i n L~ o n a finite

interkvnl I , then

inteninl I , t h e n

dx =

lirn rt+m

Proof: (a) Let

.l;,(:c) + f

(x) in L'

011

1

f (x)dr.

I . Then

(b) By Theorem 1.34, if f,,(z) -+f (z) in L" on I, then it also converges in L1. Then the result follows frorri part (a). (c) By Tlleorenl 1.38(b), if f,,(z) + f ( z ) in L~ 011 I, then it also converges in L1. The11 the result follows from part (a).

If I is an infinite interval, then the cor~clusionsof Theorems 1.40(b) and 1.40(c) are both false, as can be seen by considering the example given in Rernark 1.39(c). In this example, f,,(z) + 0 both in LOC and L2 011 10, m). However, since JI f,, (x) d z = 1 for all n,

However, in the case of infinite intervals. we can prove a useful theorern by making an additional assumption on the sequence { f, ( x ) ) , , ~ ~ .

1.2. Convergence of Sequences of Functions

23

Theorem 1.41. Suppose that for every R > 0, f,, (x)3 f (x) in L" or in L' o n [-R, R ] . T h a t i s , ,for each R > 0 ,

I f f (z)i s L1 on a n interval I and 2f there i s a function g(x), L' o n I , such that for all x E I and all n E N , I f , (x)1 I g(x), then

Proof: If I is a, finitmeinterval, then there is nothing to do by T h e e rem 1.40(b) and (c), so wc Inay assume that I is infinite, and for convenience we will take I = R. By Theorem 1.40(a), it will be sufficient t o prove that f,, (z) + f (z) in L1 on R. Let e > 0. Since f (x)a,nd g ( z ) are L' on R, by Theorem 1.5, there is a number R > 0 such that

If

(o.)( d x

< 1/3

and

/

J,q(x) 1 di < r/3.

rl>R

Therefore, usirig the triangle inequality for the L1-norm (Exercise 1.18(c)),

By 'l'heorem 1.34 and Theorem 1.38(b),if f,,(x) + f (z) in LtX or L~ on [-R, R], then it also converges in L1 on [-R, R]. That is,

Hence, there is an N such that if n

LR1 R

fn

Therefore, if n

> N, then

> N, then

(x) - f ( x i I dz

< d3.

24

Chapter 1. Functions and Convergence

and (1.7) follows.

Next we present a variant of Theorem 1.41.

+ f (x) in L" o r in L~ and a n N E N such that for all

Theorem 1.42. Suppose that f o r every R > 0, f,,(x) o n [-R, R]. If for every n 2N,

t

> 0 , there

is an R

>0

Then

Proof: The proof is the same as that of Theorem 1.41, except that we choose R > 0 and N E N such that for all n 2 N.

Then (1.8) hecomes

from which (1.9) follows.

Exercises Exercise 1.43. Prove each of the statements made in Example 1.25. Exercise 1.44. Prove Theorem 1.28. Exercise 1.45. Prove Theorern 1.29. Exercise 1.46. Prove each of t,he claims made in Example 1.31. Exercise 1.47. Prove Theorern 1.34. Exercise 1.48. Prove Theorem 1.38.

1.2. Convergence of Sequences of Functions

25

Exercise 1.49. Prove that if f,(x) is defined as in Example 1.33(b),then f,(x) 4 0 in L~ on [ O , l ] . Exercise 1.50. (a) A sequence of functions { f , ( ~ ) ) ,defined ~ ~ on an interval I is said to be unzformly Cauchy on I if for every E > 0, there is an N > 0 such that if n, m 2 N then 11 f n - f,lI, < E . Prove that any sequence that converges in L" on I is uniformly Cauchy on I.

(b) A sequence of functions { f n ( x ) j n E Ndefined on an interval I is said to be L1 Cauchy on I if fur every E > 0, there is an N > 0 such that if n , m 2 N , then 11 f, - f,lll < E for all x E I. Prove that any sequence that converges in L1 on I is L1 Cauchy on I . (c) A sequence of functions { f n ( x ) j n E Ndefined on an interval I is said to be L2 Cauchg on I if for every E > 0, there is an N > U such that if n , m 2 N , then I(f , - f, ( I z < E for all x E I. Prove that any sequence that converges in L2 on I is L~ Cauchy on I.

Chapter 2 Fourier Series 2.1 Trigonometric Series 21.1

Periodic Functions

A fur~ctionf (x) defined o n R has period p > 0 Lf f ( r+ p ) = f (x) for all x E R. Such a function is said t o be periodic.

Definition 2.1.

Remark 2.2.

(a) Tlie functions sin(x) arid cos(x) have periocl 2n. The functions sin(ax) and cos(a,z), a > 0. have period 27rla. (b) If f (x) has period p > 0. it also lias period kp. for k E N. IIeilce a periodic function car1 have many periods. Typically the sirlallest period of f (x) is referred to as the pernod of f ( x ) .

Definition 2.3.

Given a function f(z)on R. n.n,d periodization o f f (x) is defined as the fun,ction

0,

number. p

> 0 , the

p

provzded that the sum makes sense. See Figure 2.1

Remark 2.4. (a) It is easy to verify that in fact the function f,,(x) lias period p by nlakirig a change of sunimatiori index in the sun1 on the right side of (2.1). Specifically,

where we have made the change of suinnlatiorl index n e n + 1.

(b) If f (z) is conlpactly supportcd, then the surri in (2.1) will converge poiritwise on R. This is because for each x the slirn will have only finitely riiaiiy terms. (c) If f (x) is supported in an interval I of length p. then f,(a.) is referred to as the period p extension of f (x). This is because for z E I. f,(x) = C,LEz f ( x + np) = f (x) since all terms in the sum besides the n = 0 tern1 are zero. (Whys?)Another way of thinking of this is that we a,re taking ilifi~iitel~ Inany copies of the fuiictiorl f (.c) and placing thein side-by-side on the real line.

28

Chapter 2. Fourier Series

FIGURE 2.1. Top Left: Graph of f (x). Top Right: Graphs of f (z + n p ) for -2 n 2 and p = 1. Bottom: Graph of the 1-periodization of f (z).

< <

Definition 2.5.

Given a

> 0,

the collection of functions (2.2)

{e2711r'T'a )nt~

ss called the (period a ) trigonometric system.

Remark 2.6.

+

(a) Recall Euler's formula: ei.' = cos(x) i sin(x). This formula can be proved by expariding both sides of the equation in a Taylor series (Exercise 2.20). Therefore

and it follows from this that each element in the trigonometric system has period a.

(b) The period a trigonometric system is sometimes given in the form

Systems (2.2) and (2.3) can be obtained from each other by forming simple

2.1. Trigonometric Series

29

linear combinations. Specifically, for n E Z,

and for n E N,

and

(c) A function that can be written as a finite linear combination of elements of the (period a) trigonornctric system is called a (period a ) t r i g o n o m e t ~ i c polynomial. That is, a trigonometric polynorrlial has the forni

for some h1,N E Z and some coefficients c ( n ) .

Theorem 2.7. T h e period a trigonometric s y s t e m ( 2 . 2 ) satisfies the following orthogonality relations:

Proof: Exercise 2.22. Remark 2.8. Note that since the functions e2"inxl" a11 have period a. the integral in (2.4) can be taken over any interval of length a. For example,

A fundamental problem in Fourier series is the following: Given a function f ( x ) with period a

> 0. can we write

,for some choice of coeficients { ~ ( n ) )? , , ~ ~ This problem leads t o three related questions that will be answered in the following subsections:

30

Chapter 2. Fourier Series

( a ) I n order ,for. (2.5) to hold, what m u s t the coeficients c ( n ) be? (b) Assuming we know the answer to question (a), in what serlse doea the s e ~ i e so n the right side of (2.5) converge? (c) Assurniny we know the answers to que.stions ( a ) and ( b ) ,does the series o n the right of (2.5) converge to f (x),or to some other funct,io*n,?

2.1.3 Let

The Fourier Coeficients

11s begin

hy answering question (a) above.

Definition 2.9.

G i v e n a function f ( x ) w i t h period a , the Fourier coefficier~ts o f f ( z ) are defined by

provided that those zntegrals m a k e sense. For example. i f f ( - r ) zs L' on [O. a ] , t h e n the integral in (2.6) converges for each 7 1 .

Remark 2.10. Tlie definition of the Fourier coefficients of a function f (x) is by no means arbitrary. In fact we are essentially forced to define them that way by the followilig a r g u m ~ n l . Sr~ppusethat in fact f(x) = C r l E c(7L) Z e2ai,1x/u. T1lei1 in light of' Theorern 2.7. for rrr 6 Z fixed,

since by (2.4), the only nonzero terni in t,he sum is the rz = m term. Note that the above argument is not a rigorous proof since we interchanged an integral and an irifiriite surn without having any idea liow or even if the slim converged. However. the argument is sufficient motivation for defining the Fourier coefficients as in Definition 2.9.

Definition 2.11.

G w e n a function f ( z ) with period a , L' o n [O, a ] , the Fourier series associated witahf ( z ) is defined as the formal series

where the c ( n ) are defined b y ( 2 . 6 ) . W e refer to (2.7) as a 'fformal series" since w e d o n o t yet know how or i f the series converges. W e write

2.1. Trigonometric Series

31

Remark 2.12. It is possible to rewrite the Fourier series of a function in terms of the real trigonometric system defined by (2.3). To see this, note that

Conversely, a series of the form

can be rewritten as

where

Example 2.13. (a) Let f (x) be the period 2 extension of the f ~ ~ n c t ~ i o n X,-1/2,1/21 ( 2 ) .The Fourier coefficients of f (x) are

=

i

0 if n is even, n # 0, 1 -( - I ) ( ~ - ' ) / ~ if n is odd, nn 1 if n = 0. 2

32

Chapter 2. Fourier Series

The Fourier series associated to f ( x ) is

See Figures 2.2 and 2.3.

(b) Let f ( x ) be the period n extension of the function x X(,,,)(x). Then

and for n

# 0,

Therefore,

f(x,

-.+,

i

C I,

e2inx =

~ E Z

-

C sin (n2 n x ) LEN

See Figure 2.4. (c) Let f ( x ) be the period n extension of the function x X(-,/2,,/2)(x). Then c(O) = 0, and for n # 0, c(n,)= ( - l ) n1;n/2n8, so t,ha,t,

(d) Let f ( x ) be the period 2n extension of the function 1x1 Xc-,,,,

1 . 4

( 2 ) .Then

Convergence of Fourier Series

Definition 2.14. A function f (x) o n a finite inten)u,l I i s piecewise differentiable o n I i f ( a ) f (x)i s piecewise continuous o n I with only j u m p discontinuities (zf a n y ) , ( b )f f ( z ) exists at a21 but finitely m a n y points in I and ( c ) f ' ( z ) i s piecewise continuous o n I with only jump discontinuities (zf a n y ) . A function f (z)i s piecewise dzfferentiable o n a n infinite interval I if i t is piecewise differentiable o n every finite subinterval of I .

2.1. Trigonometric Series

33

FIGURE 2.2. Top left: Graph of f (z) from Example 2.13(a). Top right: Graph of Fourier coefficients of f (z).Bottom lcft: Graph of f (z) from Example 2.13(b). Bottom right: Graph of absolute value of Fourier coefficients of f (z) .

Example 2.15. on I .

(a) Any function C1 on I is also piecewise differentiable

(b) If I is any finite interval, then the function Xr(x) is piecewise differentiable on any i~ltervalJ with I C J . (c) The tent function Bl(x) is piecewise differentiable on R because it is linear on the intervals (-m, -I), (-1, O), (0, I ) , and (1, GO).

(d) Any piecewise polynomial function is piecewise differentiable on R. The following convergence result is due to Dirichlet.'

Theorem 2.16.

>0

and is piecewise diflerentiable on R . T h e n the sequence of partial sums of the Fourier series (Dirichlet) Suppose that f (x)has perzod a

'The proof of Theorem 2.16 will not be given here but can be found for example in Walker, Fourier Analysis, Oxford University Press (1988), p. 19 (Theorem 4.5) and p. 48ff.

34

Chapter 2 . Fourier Series

of f (z),{SN( ~ ) } N E Nwhere ,

-

converges pointwise to the function f (z),where

FIGURE 2.3. Partial sums S N ( X )of the Fourier series f (z) from Example 2.13(a). Top left: A; = 10, top right: N = 20, bottom: N = 60.

-

Note that F ( a ) = f ( a ) if f (z) is continuous at x = a and that f ( a ) is the average value of the l e f t and right-hand limits of f ( x ) at x = a when f ( x ) has a jurrlp discontinuity. If we assume that f (z) has 110 discontinuities, then we can make a stronger statement as in the following heo or em.^

Theorem 2.17.

Suppose that f (z) has period a

>

0 and is continuous and

2 ~ h proof e of Theorem 2.17 can be found in Walker, Fourier Analysis, Theorem 4.4, p. 59.

2.1. Trigonometric Series

35

FIGURE 2.4. Partial sums S N ( Z )of the Fourier series f(x) from Example 2.13(b). top left: N = 10, top right: N = 20, bottom: N = 60.

piecewise dzfferentiable o n R. T h e n the sequence of partial s u m s S N ( X )gzuen b y (2.8) converges t o f ( z ) i n L" o n R.

What if the function f (x) is continuous but not piecewise differentiable? What can be said about the convergence of the Fourier series of such a function? It is by no means obvious that such functions exist, but they do. The most famous example is due to Weierstrass, who constructed a function continuous on R but not differentiable at any point of R. This function is defined by f (x) = CrLCN 3-n ~ o s ( 3 ~ xThe ) . Weierstrass AT-test can be used t o show that this function is continuous, but the proof that it is nowhere differentiable is hard.3 By the Weierstrass M-test, the Fourier series of the Weierstrass function converges uniformly on R. However, this is not the case for all periodic functions, continuous on R. The following theorem is due to ~ u ~ o i s - ~ e ~ r n o n d . ~ 3An example of a continuous, nowhere differentiable function similar t o the Weierstrass function, together with a very readable proof, can be found in Korner, Fourier Analysis, Cambridge University Press (1988). Chapter 11. 4 ~ x c e l l e n texpositions and proofs of this theorem can be found in Korner, Fourier Analysis, Chapter 18. and also in Walker, Fourier Analysis, Appendix A.

36

Chapter 2. Fourier Series

Theorem 2.18.

(DuBois-Reyrnoud) There exists a function f ( : E ) continuous on. R. n.nd e~rifhperiod 27r .ssuch that the Fourier serzes o f f (x) diucrgcs at .r = 0 . T h a t is, lim~,,

SN(O)does not exist where S N ( X )is given by (2.8).

In fact. it is possible to find a continuous, period 27r function whose Fourit>r s e r i ~ sdiverges at every rational nlultiple of 27r.5 Therefore, it is iinpossil~leto make the st,ateiiieilt that the Fourier series of every coiltiliuous fuilctioii coilverges pointwise t o tliat function. The ~ i e x ttheoreill, Theorem 2.19, is due to Fejitr and makes a geilera,l st,ate~nentahout the convergence of the Fourier series of a continllolls function. The idea behind Fejbr's Thcorcrn is tlie following. Instead of looking at tlw part,iixl sums (2.8), coilsider the arithmetic m e a n s of those part)ial suins; that is. coi~sicterthe sequence

It is oftell the case that when the corivergence of a sequence fails due t o oscillatioli ill the terms of the sequence, the arithmetic rrleans of the sequence will have better convergerlee behavior. Take the simple exariiple of the sequeiice { ( ~ ( n ) } ,where ~ ~ . a ( n ) = (- 1 ) " . Clearly lim,,, a ( n ) does not exist hecause the t,erms sirrlply oscillate back and forth between 1 and -1. Homrever. if mre coiisider the sequence of aritllrrletic means, { ~ ( T z ) } ~ ~ ~ . given 1 ) ~ .

so that linl,,, ~ ( 7 1= ) 0 (Exercise 2.25). If t,he original sequence { ~ ( n , ) } ,already , ~ ~ converges. taking the arithmetic means will not affect the convergence; that is, if lim,,,, a ( n ) = a. then also lim,,,, g ( n ) = a (Exercise 2.26). (Ft:j&r'sTheorerri) Let f (x) be a f7~nctionwith period n > O con,tiri,vovs o n R. and define for each 71, E N the functior~a ~ ( x by ) (2.9), where Sl;(cr) i s give77, by (2.8). T11,enCT.~T (z)converges uniformly to f (x) o n R as N -+ GO.

Theorem 2.19.

"bValkcr. Fourier .Arlal,ysis, Xppcrltlix A.

37

2.2. Approximate Identities

Exercises Exercise 2.20.

Prove Euler's formula: For every x E R, em

=

cos(x) +

i sin(x).

Exercise 2.21.

Prove that for every real nilmher o,,

Exercise 2.22. Prove Theorem 2.7. Exercise 2.23.

Prove each of the statements made in Remark 2.12.

Exercise 2.24.

Prove each of the statements made in Example 2.13.

Exercise 2.25.

Show that if a ( n ) = n

1 ~ ( n =) - x a ( k ) = n

n E N, then 0 -l/n

k=l

Exercise 2.26.

Show that if lini,,, where a ( n ) is given by 2.10.

2.2

a(n)

if n is even, if n is odd.

=

a , then liin,,,

a(n)

=

a,

Approximate Identities

The notion of an approximate identity or summability kernel is used extensively in all branches of analysis. The idea is t o make precise the notion of a "delta funct.ionn that is well known and widely used by physicists. engineers, and mathematicians. The delta function, 6(z),has the property that for any continuous function f (x),

or more generally,

f (t) d(x

-

t) d t = f (4

for. every z E R. From sorrle elernenlary considerations he reader. may fill in the details), any function b(t) satisfying (2.11) must satisfy, b(t) = 0, t

#0

and

b(t) dt = 1.

38

Chapter 2. Fourier Series

It is impossible for any ordinary function to satisfy these conditions since the Riemann integral of a function, f (z), vanishing at every 17: # 0 must be zero. This must be true even under more general definitions of the integral (such as the Lebesgue integral). Therefore, 6(t) is not an ordinary function. So the question remains: How are we t o make sense of this concept? There are two ways to do this.

1. Extend the definition of function. This has been done by L. Schwartz who defined the notion of a distribution or generalized f u n ~ t i o n . ~ 2. Approximate the delta by ordinary functions in some sense. This more elementary approach has its natural completion in the theory of distributions alluded to above, but can be understood without any advanced concept,s. The idea is t o replace the single "function" d(t) by a collection of ordinary functions {KT(t)},>" such that for every continuous function f (z),

and more generally,

where the limit is interpreted in some sense and described in Section 1.2. The purpose of this section is to explain the theory of approximate identities.

2.2. I

Motivation from Fourier. S e ~ i e s

In order t o further motivate the notion of an approxirrlate idenlily, let us consider how one might prove Theorems 2.16 and 2.19.

Definition 2.27. For each lc E N , and a > 0, define the Dirichlet kernel Uk (x) b y k

D&)

=

C

,2nimx/a.

(2.12)

m=-k

See Figure 2.5. good exposiliorls uf this theory can be found in Horvath, A n introduction t o distributions, The American Mathematical Monthly, vol. 77 (1970) 227-240, and Benedetto, Harmonic Analysis and Applications, CRC Press (1997).

2.2. Approximate Identities

Theorem 2.28. For each k

E

39

N , and a > 0, the Dirichlet kernel, D k ( x ) , can

be written as Dk(x)=

+

sin(-ir(k l ) x / a ) sin(~x/a) '

and for any period a function f ( x ) ,

Proof: Eqlxakion (2.13) is a,n exercise (Exercise 2.38) and reqi~iresonly t,he formula for summing a geometric series. As for equation (2.14),

Sk ( r )

=

C c ( n )e2K"x/U

The result follows by making the change of variables t F+ x - t in the above int,egral a.nd remembering t,ha.t bot,h Dk (a)a.nd f (z) ha.ve periocl a.

Definition 2.29.

For each n E N , and a > 0, define the Fejhr kernel F,(x)

bY n- 1

CD~(X).

1 ~ ~ (= x ) k=O

(2.15)

See Figure 2.6.

Theorem 2.30. For each n E N , and a > 0 , the Fej6r kernel, F,(x),can he written as

and for any period a function f ( x ) ,

:La

a ( n )( x ) = -

f

(X -

t ) F,(t) d t .

Proof: Equation (2.16) is an exercise (Exercise 2.39) and requires only the formula for summing a geometric series and some manipulation.

40

Chapter 2. Fourier Series

FIGURE 2.5. The Dirichlet kernel D k ( x ) (2.14) for a

= 1.

Equation (2.17) is also a,n exercise (Exercise 2.40), ancl the derivation is similar to (2.14).

17

From Theorems 2.28 and 2.30, we see that the proofs of Theorems 2.16 (Dirichlet) arid 2.19 (Fejer.) amoul~tto sliowing that

pointwise for every period a f~inct~ion f (z),piecewise differentiable on R, ancl

in LDO on R for every period a function f (x) continuous on R. Such convergence results depend on properties of the sequences { D k ( ~ ) ) k E and N { p ? 2 ( ~ ) ) 7Consideration 26N. of tJhe required properties of these sequences leads to the notion of an approximate identity or summability kernel.

2.2.2 Definition and Elcamples Definition 2.31. A collection of functions {Kr(x)),>o o n a n interval I

=

( - a , a ) ( a = m is permitted) is a n approximate identity or a summability kernel

2.2. Approximate Identities

20

i

i

i

i

i

i

i

i

i

'

O

i

i

i

i

i

I s . ~ ~ j ~ - - j - - - ~ - - i - - - ; - - - ~ - - j - - - ; . . . ~,s-.-j---i-..;..;...j...;.. ..l l l l 16--.]---+...t..]...i...t.--

l

l

I I--

I

I

I

I

I

I

I

I

I

I

I

I

I

I

l

l

l

I

l

l

l

6~~~d~-~l...L..A...l..-l~~~l...l...l...

- 0 5 0 . 4 - 0 3 -0.2 -0.1

0

I 1

l

l

l

01 02 03 04 05

I

I

I

I

I

-05 - 0 4 0 . 3 - 0 2 - 0 1 0

i I

01

i

A

;' :1 . -

i --

41

i

-

-. .

0.2 0.3 0 4 0 5

FIGURE 2.6. The Fej6r kernel F,,(x) (2.14) for

a =

1.

on I if the following conditions hold. (a,) For all

7

> 0,

L

KT (x)d z = 1 .

( b ) There exists M > 0 such that for all

7-

> 0,

S_:

/ K T ( x )dl x 5 M

( c ) For every 0 < 6 < a ,

IKT( x ) 1 d x = 0.

lim ~--t'+.

S
Example 2.32. (a) For 0 < r < 1, let K T ( x ) = 1 / ( 2 r )X(-,,,I. { K T(x))O
Then

Then {h',(x))o<,
( c ) More generally, let g ( x ) be any function L1 on R and supported in (-a, u ) ( u = oo is permitted) such that

42

Chapter 2. Fourier Series

For 0 < 7 < 1, let KT(x) = (117) y ( x / ~ ) Tlier~ . {KT(x))o
7

> 0, let K T ( x )= 7

sin(nx/ ( r a ) ) sin(nx/a)

X(-a,a)

(4-

Tlien { I ( , (x)),,~ is an approximate iclentity oil (-a, a). 111 fact, Kll, (x) = F, (x) ?C(-,.) (x), where F, (z) is the Fej6r kernel defined by (2.16). (e) For

T

> 0, let

Then Kllk(z)= D k ( x )X(-,.,) (x), where D k ( x ) is the Dirichlet kernel defined by (2.13). However, {KT(z)),>o is not, a.n a.pproximate identity on (-a, a ) .

(f) For each r

> 0, let

-1

a,

=

[~:,(1

-

t2)'11 dt]

, and let

Then {K,(z)),>o is an approximate identity on (-1,l). Define L, (n;) = Kll,,(x). Then the sequence { L , ( X ) ) , ~ is ~ referred to as the Landau kernel.

2.2.3 Convergence Theorems The idea behind approximate identities is to investigate the validity of the statement f ( t ) K T ( x- t) dt = lim f T ( x ) = f (x).

,to

Proving (2.18) will allow 11s t,o a,pproxima,te a,rbitrary functions f (n;) by functions of the form f, (x) that often can be made to have desirable properties (see Corollary 2.37). In this subsection, we will consider mostly approximate identities on R as those will be most useful t o us later in the book. Let us first consider pointwise approximation.

Theorem 2.33. Let f (x) be L"

o n R and continuous at the point x = a . Suppose that {K,(x)),>o i s an approximate identity o n R. T h e n f ( t )K T ( a - t ) dt = f ( a ) .

(2.19)

2.2. Approximate Identities

43

Proof: Note first that by a change of variables,

f (t) KT( a - t ) dt =

f (a - t ) KT (t) dt.

By Definition 2.31(a),

Therefore, for ally umber 6

> 0,

Let t > 0. Since f ( x ) is continuous at x = a , there is a 6 > 0 such that if It1 < 6, then If (a) f ( a t)l < c/(2116), where 116 is the upper bound in Definition 2.31 (b). Hence for such a delta, -

Since f (z)is L"

on R,

-

If

(y)l

< 11 f /lo

for all t E R. Hence, for any 6 > 0,

for all y E R. Therefore,

Chapter 2 . Fourier Series

44

as

T

-+ of

0

by Definition 2.31(c). Therefore, there is a then

Hence for 0

TO

> 0 such

that if

< r < 70,

and (2.19) follows. Note that if K T ( x ) = (117)g ( x / r ) for some function g ( x ) , L1 on R and compactly supported, then the assumption that f (x) is L" on R is unnecessary. It would only be necessary t o assume that f (x) is bounded on some small interval containing x = a , which follows from the continuity of f ( T ) at n: = a (Exercise 2.42). Now let us consider uniform convergence. Theorem 2.34.

T,et f(z) he L" and unzformly continuous on R . Suppose is an approximate identity o n R. T h e n that {K,(X)),>~

Proof: Making the same estimate as in (2.20), it is sufficient t o show that given r > 0, there is a 6 > 0 and an TO > 0 such that if 0 < T < TO, then

f (x) f (X t ) / -

-

IKT(t)I

dt

+

.I

If

(z)

-

f (X - t)l lKT(t)l df

< E.

It126

(2.22) As for the first term in (2.22), the uniform continuity of f (x) implies that there is a 6 > 0 such that if It1 < b , then for all x E Ft, f (x) - f (x - t)l < € / ( 2 M ) , where M is the bound in Definition 2.31(b). Hence, a.s in tJheproof of Theorem 2.33,

for all r > 0. As for the second term in (2.22), since f (x) is L"

on R,

2.2. Approximate Identities

which converges to 0 as r that if 0 < r < T O , then

45

+ 0+ for any 6 > 0. Hcncc thcrc is a TC,> 0 such

for all z E R, and (2.21) follows. The condition that f (z)is uniformly continuous on R is satisfied if for example f (z) is C: on R. More generally, it is satisfied if f (z) is Co on R and liml,l,, f (x) = 0 (Exercise 2.43). Lemma 2.35 (which we state without proof) establishes a very important property of functions referred to as the continuity of translation for L1 and Lqunctions. This means that the translate of an L1 or L2 function remains very close to the original function provided that the translation is small enough. Here closeness is measured in the sense of the L' or L2 norm. Continuity of translation is used to prove L1 and L2 convergence using approximate identities.

Lemma 2.35.

(Continuity of Translation) Suppose that f (x) is piecewise con-

tznuous o n R . T h e n the followzng hold.

(a) I f f (z)i s L1 o n R, then

(b) I f f ( z ) is L2 o n R, then

f%Lf(x)-

f ( x - t ) 2 dz=O.

(2.24)

As for L1 or L' convergence in (2.18), the following theorem (which we state without proof) holds.

Theorem 2.36. (a) Suppose that f ( x ) is L1 o n R, and that {K,(x)),,o identity o n R. T h e n

zs a n approximate

r

( b ) Suppose that f ( z ) is L2 o n R, and that {K,(X)),>~is a n approzimate identity o n R. T h e n

46

Chapter 2. Fourier Series

FIGURE 2.7. Illustration of Lemma 2.35. Top Left: Graph of function f (x). Top Right: Graph of f (x) and f ( x - t ) where t = 1/16. Bottom: Graph of If (4 - f ( x - t)l. Area of shaded region is small for small t.

The following approximation theorein is a consequence of Theorein 2.36. Corollary 2.37.

( a ) Let f (x) be L' o n R, and let R such that

E

> 0. T h e n there i s a function g(x), C: o n

llf ( b ) Let f (x) be L~ o n R, and let R such that

E

-

9111 < E .

> 0. T h e n there i s a function g(x), C: o n

Ilf

-

9/12 < t.

Proof: (a) By Theorem 1.5, there is a compactly supported function h(x), L1 on R, such that 11 f - hill < €12. Now, let

> 0 (SCC

Examplc 2.32(b)). Thcrl { K T ( ~ ) ) T > isoan approximatc idcntity on R. By Theorem 2.36(a), T

lim h, (x)= lim r+o+

7+0+

h(t)K T ( x - t)dt = h(x)

47

2.3. Gcncralizcd Fourier Series

in L1 on R. Hence there is a 70 > 0 such that Ilh, - hill < ~ / 2 .Let g(x) = h, (x). That g ( x ) is compactly supported follows from Exercise 3.25, and that g (x) is C0 on R follows from Theorem 3.18. The proof of (b) is similar (Exercise 2.44).

Exercises Exercise 2.38. N-1 r n

'1

CrL=O

-

Prove equation (2.13). (Hint: Recall that for any number

1-r N

~

-

1

Exercise 2.39.

Prove equation (2.16).

Exercise 2.40.

Prove equrttiorl (2.17).

Exercise 2.41.

Prove each of the statements made in Example 2.32.

Exercise 2.42. Prove that if f (x) is continuous at x = a:then there is a > 0 such that If (xjl 5 M for all x E [a - b, a 61. b > 0 and a number

+

Exercise 2.43. (a) Prove that i f f (x) is C: on R, then f (x) is uniformly continuous on R. (b) Prove that if f ( x ) is C0 on R and limlslj30 f (x) uniformly contir~uouson R.

=

0, then f (z) is

Exercise 2.44. Prove Corollary 2.37(b).

2.3 Generalized Fourier Series 2.3. I

Orthogonality

Definition 2.45.

A collection of functions { g n ( ~ ) ) , E L~ ~ , o n a n interval I is a (general) orthogonal system o n 1 provided that (a) / gI n ( X ) S r n ( x ) d x = O L f n j i m , and

Part ( b ) says i n particular that none of the gn(x) can be zdentically zero. Th.e collection { ~ , ( x ) ) , ~ Nis a (general) orthonormal system o n I provided t l ~ u ti L ,is ur1 orthogonal system o n I and

48

Chapter 2. Fourier Series

It is not nccessary that the set {gn(x)) be indexed by N, and in fact we have seen an example (the trigonometric system) that is indexed by Z. In all future examples, the index set will either be specified or will be clear from the context. Whenever a generic system of functions is considered, the index set will be assumed t o be N.

Remark 2.46.

(a) Any orthogonal system can be normalized so that it becomes an orthonormal system. Tha,t, is, if {gn(.x)) is a11 orthogonal system, then we may define the functions

Then the system {&(x)) is an orthonormal system. (b) The Cauchy-Schwarz inequality guarantees that each of the integrals in Definition 2.45 exists as a finite number. That is, since f (x) and g(x) are L2 on I,

(c) Throughout t,he hook, we will use inner product notation to represent the integrals in Definition 2.45. That is, we write for any functions f (x), and g(x) L2 on I,

This means in particular that

Example 2.47.

(a) Given any a

> 0, the collection

is an orthogonal system over [-a, a ] . I t is also orthogonal over [O,2a]and in fact over any interval I of length 2a. The collection

is an orthonormal system over [-a, a ] . I t is also ort'honormal over [O, 2a] and in fact over any interval I of length 2a.

2.3. Generalized Fourier Series

(b) Given any a

49

> 0, the collections

{sin(-irnz/a)},~~

and

{cos(~nz/aj},~~

are each orthogonal systems over [-a, a ] . The collections

arc each orthonormal systems over [ a , a ] . (c) Given a > 0, the collection

is an orthogonal system over [0, a], and in fact over any interval I of length a. The collection

is an orthonormal systcm ovcr [0,a], and in fact ovcr any intcrval I of lcngth a.

2.3.2

Generalized Fourier Series

Definition 2.48.

Given a fun,ction f (z), L 2 o n a n interval I , and a n orthonormal system {gn(x)) o n I , the (generalized) Fourier coefficients, {c(n)) of f (x) with respect t o {g,(x)) are defined by

The (generalized) Fourier series o f f (x) with respect to {g,(z)) is

The fundamental problem is to determine under what circumstances the in the above definition becomes a "=" and, if so, in what sense the infinite series on the right side of the equality converges. It turns out that the most convenient form of convergence in this case is L~ convergence on "N"

T. Theorem 2.49. {y,(z))

(Bessel's inequality) Let f (x) be L 2 o n a n interval I , and let be wrt, outhonownul sysLe,ln U,IL I . T J L ~ ~ I L

50

Chapter 2. Fourier Series

The proof of Bessel's inequality will require the following lemma. Lemma 2.50. Let { g , (x)} be an, orthon.orm.al system, on, o,n, in,ten~o,lI . Then for every f (x),L~ on I , and every N E N ,

Proof: The proof is just a calculation making use of thc orthonormality of

{gn(x>>.

+

C C (f'g n ) (f.Smj

n=l m=l

I

~n ( 2 )gm

( x )d:r

which is (2.28).

Proof of Theorem 2.49: Let f ( x ) be given, and let {gn(x)) be an orthonormal system. Then by Lemrrla 2.50, for. each fixed N € N,

2.3. Generalized Fourier Series

51

Therefore, for all N E N,

Since I (f, g,) 1' > 0 for all n , the partial sums of the series CncN( f ,gn) l 2 form an increasing sequence bounded above by JI 1 f (x)I2d x . Thus the series CnEN 1 ( f , gn)I2 converges so that we can allow N to go to infinity Thus;

which is (2.27). Closely related to Lemma 2.50 is another very important inequality that will be very useful in the next subsection.

Lemma 2.51. Let { g , ( x ) } be a n orthonormal system o n I . Then for every f (x),L~ o n I , and every finite sequence of numbers { a ( n ) ) ~ =, ,

Proof: Let f ( x )be given, and let { g n ( x ) )be an orthonormal system. Then

52

Chapter 2 . Fourier Series

by Lemma 2.50.

We are now in a position t o answer the fundamental question about Fourier series, namely: When is an arbitrary function equal to its Fourier series and in, ~r~h,nf; sense does t h t Fourier series converge? The answer lies in the notion of a complete orthonormal system.

Definition 2.52.

Given a collection of functions { g , ( z ) } , L~ o n a n interval 1 , the span of {g,(x)), denoted span{g,(x)), is the collection of all finite linear combinations of the elements of { g , ( x ) ) . In other words, f (z) E span{g,(x)) zf N and only i f f ( x ) -a ( n )y n ( x ) for some finite sequence {a(n)};=, . Note that N is alwal~sfinite but m a y he arbitrarily large.

E x a m p l e 2.53. (a) Let PI denot,e t,he set of all polynomials on the interval I. Then PI = span{xn),"==,. (b) span{e-2TznLjn,,Z is the set of all period 1 trigonometric polynomials. ( c ) Lel p(z) = (1- 1x1) X r l , l l ( x ) .Then span{cp(x - n)InEz is the set of all functions that are ( i ) continuous on R, (ii) linear on intervals of the form

[n, n

+ l ) ,n E Z, and (iii) compactly supported.

R e m a r k 2.54. (a) For any collection of functions {g, (x)} , span{gn (x)} is a linear space: that is, it is closed under the formation of lincar combinations. Specifically, if {f,(x))E=, & span{g,(x)), then for any finite seN quence { a ( n ) ) L , the function f (x) = a(n)f, (x) is in span{gn (z)) (Exercise 2.61). (b) The definition of span involves only finite sums. Without additional assumptions on the collection {g,(x)}, there is no guarantee that any sum of the form CnEN a(n) g,(x) will converge in any sense. For example, if g,(x) = xn-l for n E N, then the series n! x r V o e s not converge except at x = 0, and the series Cr=o 2-n x n does not convcrgc if 1x1 2. See also Theorem 2.55 below.

E,"&

>

(c) Related to the notion of span{g,,(x)) is the notion of the mean-square (or L') closure of span(g,(x)), denoted @ZTi{g,,(x)} which is defined as follows. A function f (x) E SjZZTi{g,(x)) if for every E > 0, there is a function g(z) E span{gn(x)) such that 11 f - gllz < E .

2.3. Generalized Fourier Series

53

As a partial answer to the question of when finite sums can be replaced by infinite sums, we have the following theorem. Theorem 2.55. Lct { g , , ( x ) )bc a n orthonormal s y s t e m on a n zr~tervalI . T h e n a function f (z), L~ on I , i s in SjZiiT{g,(z)) i f and only i f

Proof: (+)

(2.29) is equivalent t o the statement that

Therefore, given

E

> 0,there

is an N

> 0 such that

and f (x) t w { g n (x)).

(==+) Suppose that f (x)E m { g , (z)), and let c > 0.Then by definition there is a finite sequence {a(n)):l,, some NO t N, such that

Since

is a, decrea,sing secliience (Exercise 2.63), it follows t,hat for every N

> No,

54

Chapter 2. Fourier Series

and (2.29) follows. If every function L2 on I has a representation like (2.29), then we say that the collection { g , ( 2 ) )is complete on I. This means that every furiction L~ 011 I is equal t o its Fourier series in L2 on I.

Definition 2.56. Let {g,,(x)) be a n orthonormal system o n I. T h e n {g,(x)} is complete o n I provided that e v e q function f (x),L' o n I, is i n m { g , ,( x ) } . A complete orthonormal system is called a n orthonormal basis. The next theorem gives several equivalent criteria for an orthonormal system to be complete.

Theorem 2.57.

Let {g,(x)} be a n orthonormal system o n I. Then, the ,following are equivalent.

(a){gn(x)) is complete o n I ( b ) For ever9 f (x),L ' on

I,

( c ) Every f u n c t i o n f (z), C: o n I , i s i n span{g,,( z ) } .

(d) For every function f ( x ) , C: o n I,

Remark 2.58. (a) Note that Theorem 2.57(c) is precisely the definition of completeness but with C: functions replacing more general L2 functions. It is often easier t o work with cont,inuous compactly supported filnctions, and the theorem states that this is sufficient.

(b) Theorern 2.57(d) says that Bessel's inequality is an equality for conlplete orthonormal systems. This eyualily is referred t u as Pluszcherel's FOI-mula.

Proof of Theorem 2.57: (a) of Theorem 2.55.

(b). This follows exactly as in the proof

(a) + (c). This follows immediately from the fact that every furiction C: on I is also L2 on I. (a) +== (c). Let f (a)be L~ on I, and let E > 0. Then by Corollary 2.37 there exists a function g(x), C: on I such that I f - 9112 < 612. By (c).

2.3. Generalized Fourier Series

55

there exists N E N such that

Applying Minkowski's inequality, we obtain

Therefore,

so that f (x) E span{g,(x)) and (a) follows. (c)

* (d). By Theorem 2.55, (c) holds if and only if

for all functions f (x) C: on I. But by Lcmma 2.50,

Therefore, (c) is equivalent t o the statement that

and (d) follows. To illustrate an application of this theorem, we will prove the following result about trigonometric Fourier series.

Theorem 2.59.

T h e trigonometric system {e2""x}nEz is complete o n [ O , l ] .

Proof: Wc will usc Thcorcm 2.57(c). To that end, let f (x) be continuous on [0,1] (note that it is also compactly supported), and let E > 0. By

Chapter 2. Fourier Series

56

Exercise 2.65, we call find a functioil f ( x ) that has period 1, is C' on R, and such that - 7112 < t/2. By %er's Theorem (Theorem 2.19), G N ( z ) converges in L" on R t o as N i m, where

If

S(2)

and

1

j ( t ) e-2Tmxdz.

c(n)=

By Theorem 1.38(a), ZN(z) also converges t o Note that

-

f (z) in L~ on [0,11.

(Exercise 2.66). Therefore, for N large enough

arid by the triangle inequality,

But the function

2

(1

-

y)

11") 1.2"znx

n=-N

is in

s p a n { e 2 ~ z r ~ z }nEZ

Hence, f (z) is in span{e2Tinz}n,Ezand by Theo-

rem 2.57(c), the trigonometric system is cornplete on [O, I].

Exercises Exercise 2.60. Prove that if {g,(x)} is an orthonormal systerrl on an interval I and if {a(n));=, is any finite sequence of numbers, then

2.3. Generalized Fourier Series

57

Exercise 2.61. Prove that if {g,(x)) is any systerrl of L2 ~ U I I C L I O I ~Llle11 ~, span{g,(x)) is a linear space (that is, it is closed under the formation of linear combinations, see Remark 2.54 (a)). Exercise 2.62.

f (x)E

Prove that if {g,(x)) is any system of L2 functions, then

(x)) if and only if there is a sequence of fiinctions {fk(z)}

such that fk(2) E span{g,(x)) and such that limk,, 11 f - f k ( I 2 = 0. (Hint: For the "only if" direction, choose f k ( x ) E span{g,(x)) such that 11 f

-

fkll2

< Ilk..)

Exercise 2.63. Prove that if {g,(x)} is an orthonormal system on an interval I, then for any f (x), L2 on I, the sequence

is

il,

decreasing sequence. (Hint: Use Lemma 2.50 or 2.51.)

Exercise 2.64. Show that if {g, ( x ) ) is a complete orthonormal system on an interval I, then (2.30) holds for every f (x), L' on I . Exercise 2.65. Let E > 0, and let f (x) be C0 on [0, 11. Then there is a function f(x) that has period 1, is C0 on R, and such that (J: 1 f d ~ ) ' /<~e. (Hint: For 6 > 0 sufficiently small, you can construct r ( x ) by modifying f jx) only on the interval [l - 6, I ] , and then extending periodically.)

fi2

Exercise 2.66.

Prove equation (2.31).

Exercise 2.67. Let {g,(x)) be a complete orthonormal system on an interval I. Show that the Fourier series of any function f (x), L2 on I, can be integrated term-by-term in the following sense. For any numbers a < b, such that [a,b] I,

(Hint: If the sum converges in 1,' on I , it converges in L2 on [a,b]. Then use Theorem 1.40(c).)

Chapter 3

The Fourier Transform 3.1 Motivation and Definition We have seen that if f (x) is a function supported on an interval [-L. L] for some L > 0, then f (x) can be represented by a Fourier series as f

=

C

e27Mx/2L)

where

l

2L c(n)= -

S_, f ( t ) L

e-"7t(n/2L) d t .

,"

(3.1)

Of course the Fourier series actually equals the 2L-periodization of f (x) (Figure 3.1). What happens to this representation if we let L + co? In order to answer this question, define for each L > 0 and each integer

- 1;

n the number

T((nl2~)

f (t) c-~""('"'")

dt: A

so that f^(n/2L) = (2L) c ( n ). If we were to plot the numbers { f (n/2L) inEz for very large values of L , then the resulting graph would begin to resemble a function of a continuous variable on R (Figure 3.2). This function would naturally be defined by

In addition, we could also write

for large L, since the last sum is a Riemann sum for the last integral. Therefore, we have formally established the duality

fiY)

/

R

f(t) e

dt.

f (x)

-

~ ( 7e2nzx7 ) d7.

(3.3)

The discussion in the remainder of this chapter will focus on two general questions: In what way are the properties of f (x). for example, continuity.

60

Chapter 3. The Fourier Transform

FIGURE 3.1. 2L-Periodizations of a function f (x) assumed to be supported in [-L. L]. Top left: L = 1, top right: L = 2 , bottom: L = 4.

differentiability. int,egral)ility or square-integral~ility. reflect ecl in t h~ corresponding properties of f (y)? and What properties lrl~lstf (s) and f (7) satisfy in order for the *.N" in (3.3) to be replaced by "="'! Let us first make a defiiiitio~~. A

A

Definition 3.1. The Fourier trarlsforrn o f n function f (z). L' on R. i s also n function o n R, denoted f^(-y) defined by

Remark 3.2. The assumption that f (x)is L1 oil R is rrlade in order to ensure that the integral in (3.4) converges for each riuinber 7 . This convergence liolds by virtue of tlie fact that for each 2 E R, we call establish a Cauclly condition on the numbers

La 0

d.r

and

s,

=

f (.r)c - ~ " ~ 'd. ~ . :

cr.

>0

3.1. hlotivation and Definition

61

FIGURE 3.2. Fourier coefficirrrts for the functions graphed in Figure 3.1. Note how the graphs of the sequences begin to resemble the graph of a continuously defined function. as follows. If b

> u > 0, then

<

-

=

lim lim

..e+w

1 dz

- Z . ~ Z ~ X

1,

If(x)ldx

By the completeness property of the complex numbers (Remark 1.22(c)), there exist numbers s+ and s- such t h a t limaico S$ = sf and lim,,, sa S - . Thus, f ( y ) = s+ s-. A

+

A

Example 3.3.

(a) Let

f(x) = Xi-112,1i21 ( 2 ) .Then f ( 7 )= sirl(*7)/~7(Exer-

cise 3.4). (b) Let f (x) = (1 - 1x1) Xl-,,ll (l-). Then

f(y) = sin2( * ? ) / ( T ~ )(Exercise ~ 3.5).

(c) Let f (x)= e-Z"xI.Then f l y ) = l/s(l

+ y2) (Exercise 3.6).

62

Chapter 3. The Fourier Transform A

(d) Let f (z) = e ? T z 2. Then f ( 7 ) = e-"-r2 (Exercise 3.7).

See Figure 3.3.

FIGURE 3.3. Example 3.3. Left: f (x). Right: f^(?).

Exercises A

Exercise 3.4.

Prove that if

f(z) = X[-,,,]

(z), then f

(y)=

sin(2roy) Tiry

3.2. Basic Properites of the Fourier Transform

Exercise 3.5.

Prove that if f (x) = (1 - Ix/al)X[-a.al (x):a

>

63

0. then

A

Exercise 3.6. If f ( z ) = e-"lzI, a

2a > 0, then f (y) = (a,+(2x712) .

= ~ c ( " ~ ) ~(Hint: / ~ Exercise 3.7. If f (x)= ePaz2, a > 0, then f^(?) See, for example, Kammler, A First Course in Fourier Analysis, PrenticeHall (2000) p. 132-133 for the a = 7r case.)

3.2 Basic Properties of the Fourier Transform In this section, we will present two basic properties of the Fourier transform of an L1 function. A

Theorem 3.8. I f f (z)is L' on R, then f (?) is unzformly continuous on R. Proof: Given 71: 7 2 E R,

1fix)l

e-2""~1-~z)x

-

1( dx.

Note that the last term depends only on the difference yl - 7 2 and not on the particular values o l yl a ~ 7~2 . dHer~ceto show unzform continuity on R. it is enough to show that P

We will use Theorem 1.41 t o do this. Since 1 - 1 15 2 ,

If

(x)1 le-2"iax

-

11 5 2 If

(XI/,

and 2 1 f (z)l is L1 on R since f (x)is. By Taylor's Theorem, given any A > 0,

for some [ t [-A, A] and all x t [-A, A].Therefore,

.

64

Chapter 3. The Fourier Transform

for all x E [-A, A]. Therefore, A

1 f (z)J/e-'""""

A

-

11 dx

5 5

Thus, for every A by Theoreni 1.41,

> 0, ( f (x)1 e-2""x

-

lirn 27r)al

a!-0

lim 2 ~ A l a l lfJ111 = 0.

a-0

11

+ 0 in L~ on

[ - A , A ] ,so that

The next Theorem is known as the Riemann-Lebesgue L e m m a and describes the decay a t infinity of f (7). A

Theorem 3.9.

(Riemann-Lebesgue Lemma) I f f (x) is L'

on

R, then

A

IY

lim f ( 7 ) = 0. l--too

Proof: We will present an outline of the proof. The details are left to the reader in Exercise 3.10. S t e p 1. Show that if f (x) = X [ a , b l ( ~ )then , (3.6) holds. This can be done by direct calculation. S t e p 2. Show that if f (x) is a step function of the form

for some coefficients c(n) and intervals [a,, b,], then (3.6) holds. Step 3. Show that if f ( x ) is C: on R, then given E > 0, there is a step function g(x) of the form (3.7) such that 11 f -g(Il < t. Then show that this implies that (3.6) holds for f (x) (cf. Exercise 5.26). S t e p 4. Show t,hat (3.6) holds for any function f (x), L' on R.

Exercises Exercise 3.10. Complete the proof of Theorem 3.9. (Hint: For Steps 3 and 4, use the estimate if^?) 1 5 11 f - 9/11.)

ls(-y)l+

3.3. Fourier Inversion

65

3.3 Fourier Inversion The purpose of this section is to investigate the conditions under which equality holds in (3.3). From the definition of the Fourier transform, we can write

where we have exchanged the order of integration in the double integral. This formal calculation is not valid, strictly speaking, because the integral

does not converge for any particular value of x or t. Nevertheless, this calculation provides a starting point for investigating (3.3). The idea will be to place a "convergence factor" in (3.9) so that it converges for each value of x and t ; that is, we write instead of (3.9),

A

for some function K ( x ) chosen so that its Fourier transform, K ( y ) , forces the integral in (3.10) t o converge and so that equality holds in (3.3) for

K(x). We now obtain

If K ( t ) is some element in an approximate identity, then

which gives us a valid approximate inversion formula for the Fourier Transform. It orlly relllairls lo choose arl approximate identity satisfying the required conditions. There are many valid choices, but a very convenient one is to let ,-.

In this case KT(?) = epm27' (see Exercise 3.7). and the same calculation shows that equality holds in (3.3) in this case, that is, that

66

Chapter 3. Tlie Fourier Transform

It is also easy to see that K,(z) is L' on R for each 7 > 0 and that {KT( x ) ) , , ~ is an approximate identity on R (Example 2.32(c)). Now we are in a position to prove the following theorem.

Theorem 3.11. If f (x) is C O and L1 on R, then for each z

E

R,

Proof: Repeating the calculation in (3.8), we obtain

But since {KT(x)),>o is an approximate identity on R, Theorem 2.33 says that

dt

=

f (4,

for each x E R . A

With an additional assumption on f (y), we can get equality in (3.3) in a pointwise sense.

Corollary 3.12. V f (x) is

SR

for each x 1: R,

CO

and L' on R, a n d iff^(?) i s L' on R, t h e n

f^(y) e2Ti7xdy = f ( : r ) .

(3.14)

Proof: By Theorem 3.11, it will be enough to show that

2

2

But since lim,,o+ e-"' 7 = 1, the proof amounts to justifying the interchange of the limit and the integral in (3.15). This is accomplished using Theorem 1.41 in a similar way to the proof of Theorem 3.8. We leave the details as an exercise (Exercise 3.14). Corollary 3.12 does not cover all the cases that will be of interest to us in this book. For example, in Example 3.3(a), we saw that if f (x) = X~-,,,](x), A

then f (?) = s i n ( 2 ~ a x ) / ( ~ c cIn) . this case, f (x) is L1 but is not continuous, and f^(x) is not L1, though it is L~ (Exercise 3.15). Therefore, neither

3.3. Fourier Inversion

67

Theorem 3.11 nor Corollary 3.12 apply. The answer is to replace pointwise convergence of the limit in (3.13) with L2 convergence. In this case, we have the following theorem.

Theorem 3.13.

If f (z) is L' and L2 on R, and ifT ( y ) i s L' on R, then liln 7+0+

JR~ ( yeTT272 ) e Z n a ydxy = f (x),

(3.16)

Proof: Since f (z) is L1, Theorem 3.18 implies that the function

is continuous on R and Theorem 3.21(a) implies that f T ( x ) is L1 for each r > 0. Since f (z)is L2, Theorem 2.36(b) says that f,(x) i f (x)in L2 on R as r i o+. Therefore,

Since f T ( x ) satisfies all of the hypotheses of Theorem 3.11, it follows that

) e2ni7xdy =

rw

2 y 2 f (?) e2rr-yr A

and (3.16) follows.

d 71

n

Exercises Exercise 3.14.

Complete the proof of Corollary 3.12.

Exercise 3.15.

Prove that the function f (x) = s i n ( ~ x ) / ( ~isz )L2 on R

but not L1 on R.

68

Cl~aptel-3 . The Fourier Transform

3.4 Convolution Definition 3.16.

G i v e n functions f ( x ) and g ( x ) , t h e convolution o f f (x) and g(x), denoted h ( x ) = f * g ( x ) , is defined by

,tuheneve7. the integral m a k e s sense.

Remark 3.17. (a) We have encountered integrals like (3.17) before, namely in thc dcfinition of approxirnate identity. There it was shown that under specific hypotheses on f (x), the integral

is a good approxiirlatiori to f (x) as long as {K,(z))T>o is an approximate identity.

The above observatioil can provide good insight into the action of convolution. Take, for example, the approximate identity defined by (11)

(Example 2.32(a)). In this case, we can see that for any function f ( x ) , the value of f * KT(xo)is just the average value of f (z) on an interval of length T centered at xo. If f (x) is continuous, then these averages are good approxirrlations to the actual point values of f (z). If we consider K T ( z ) = (llr)(1- I x ~ / T ) X,-,,,](x), T > 0, then f * KT(xo) can be interpreted as a "weighted average" of f (z) around the point xo, where points close to zo are given more "weight" than are points further from zo. Thus. the convolution f * g(x) can be interpreted as a "moving weighted average" of f(x),where the "weighting" is determined by the function g(x). See Figure 3.4. By changing variables, it can be shown that convolution is commutative, that is, that f * g(z) = g * f (x) (Exercise 3.22). Then f * g(x) can also be iriterpretecl as a nloving weighted average of g(x),where the weighting is determined by the function f (x). (c) If the function f (x) has large variations, sharp peaks, or discontinuities. then averaging about each point x will tend to decrease the variations, lower the peaks, arid smooth out the discontinuities. In this sense, convolution is often referred t o as a smoothing operation. A more precise statement of this idea is contained in Theorems 3.18 and 3.19.

3.4. Convolution

69

FIGURE 3.4. Illustration of convolution. Top left: Graph of f (x). Top right: Graph of g(x). Bottom: Integral of the product of the solid and dashed function is f * g(1).

Theorem 3.18. If f(x) is L" o n R, and volution f * g(x) is continuous o n R.

zf

g(x) is L' o n R, t h e n the con-

Proof: Given z, y E R,

If * 9 ( x ) - f * 9(y)l

=

li,f s S ,If

=

5

If llm

(t)(9(x - t )

- 9(y -

t))dt

(t)l Ig(x - t ) - 9(y - t)l dt

1 R

9 ( t - ('

-

Y))

-

9(t)l d t .

By Lemma 2.35(a) (continuity of translation for L' functions), lim

if * g(x) f * g(y)I 4 IIf I , -

X+Y

and the result follows.

lim X+?/

j

R

lg(t - (z - Y ) )

- g(t)l dt =

0%

70

Chapter 3. The Fourier. Trarlslurm If f (z)and g ( x ) are both L2 o n continuous o n R.

Theorem 3.19. f

* g ( x ) is

R, t h e n the convolution

Proof: Let e > 0. Then given x, y E R,we calculate as above, but this timc using thc Cauchy-Schwarz inequality,

(1I f

-

(

Idt)

'(

1' 2

9

-

(x - Y ) ) - li(tl12 dt)

R

By Lemma 2.35(b) (continuity of translation for L~ ~ u I I c ~ ~ o ~ ~ B ) ,

and the result follows. We have seen that the convolution of a bounded function with an integrable function and the corlvolutiorl of two L~ functions produces a continuous function. The next theorem addresses the issue of the decay at infinity of a convolution.

Theorem 3.20. ( a ) I f f( x ) and g(z) are both L1 o n

on

R,then the convolution f * g ( x ) is also L'

R,and

llf * 9111 2 Ilf 111 119111. (b) If f ( x ) is L1 o n R,and g ( x ) is L' o n R,t h e n the convolution f L2 o n

(3.18)

* g ( x ) is

R,and llf * 9112 5 llf 111 I d s

( c ) I f f( x ) and g ( x ) are both L2 o n

(3.19)

R,then the convolution f * g ( x ) is Lw o n

R,and

[If * d m 5 Uf ( d ) If f (x) is L"

on Lm o n R, and

R,and

g(z) is

112

119112.

L1 o n R, t h e n the convolution f

IIf * gIIm 5 IIf IIm IlgII1.

(3.20)

* g ( x ) is (3.21)

3.4. Convolution

71

Proof: We will prove (a) and (b) and leave (c) and (d) as exercises (Exercise 3.24). (a) Let f (x) and g(z) be L1 on R. Then

and (3.18) follows. (b) Let f (z)be L1 on R, and g(x) be L~ on R. By the Cauchy-Schwarz inequality,

Therefore,

and (3.19) follows.

Theorem 3.21.

(The Convolution Theorem) If f (x) and g(z) are L' o n R,

then fG-ig(7) =

37).

Proof: Let, f (z) and g(x) be L1 on R. Then

(3.22)

72

Chapter 3. The Fourier Transform f ( t )c~(z- t) e - 2 T i ~dt x dz =

/R IR

Exercises Exercise 3.22. 9

Show that if f ( x ) and g(x) are L1 on R, then f

* g(z) =

* f (4.

Exercise 3.23. Show t h a t under the hypotheses of Theorems 3.18 and 3.19, f * g ( z ) is actually uniform,ly continuou,s on R. Exercise 3.24.

Prove Theorem 3.20(c) and (d)

Exercise 3.25. (a) If f (x) and g ( z ) are compactly supported and L1 on R, prove that f * g(x) is also. (b) If f (z)and g ( z ) are compactly supported and L%n R, prove that f * g(x) is also.

3.5 Plancherel's Formula Theorem 3.26. (Plancherel's Formula) Iff (i-) is L' and L' o n R, t h e n f^(y) i s also L~ o n R and (3.23)

Proof: Define f(z) = f (-z). Then

3.5. Plancherel's Formula

73

where we have made the change of variable x e -z in the last step. Since f (z) is L1 and T , ~on R, so is By the Convolution Theorem (Theorem 3.21),

F(r).

A

A

f * .f'((ri

l2

= F(0)5(7)= I~^((T)) *

y(n:) *7(x)

Since f (z) and are both L1 on R, Theorem 3.20(a) implies that f *F(r) is also L' on R, and since f (x)and f x ( ( r ) are both L~ on R, Theorern 3.19 implies that f is continuous on R . Therefore, we can apply the Fourier inversion formula (3.13) and conclude t,hat for each x E R,

=

IR

f (t) f(x

- t) d t

Evaluating the above equality at z = 0 gives

It remains only to show that in fact,

We will do t,his in t,wa steps.

Step 1. We will show that

then

f^((y)

is L~ OII R by showing that if

74

Chapter 3. The Fourier Transform

contradicting (3.24) in light of the assumption that f (x) is L2. If (3.25) holds, then given any number 111 > 0, there exists a number A > 0 such tl~at

d m ) .But

whenever r > 0 is small enough (specifically, if 0 < r < this is exactly the meaning of (3.26). Therefore, f (y) is L~ on R. A

Step 2. Since f^(?) is L' on R, lf^(y)12is L1 011 R. We leave it as an exercise (Exercise 3.29) to prove that

From this, (3.23) follows.

A related result is the following formula. Theorem 3.27. R, t h e n

(Parseval's Formula) Iff (z) and g(x) are both L' and L~ o n

Proof: Exercise 3.30. One easy consequence of Theorem 3.26 is to sinlplify the statement of the L v o u r i e r inversion formula (3.16). Specifically, we no longer need to state explicitly the llypothesis that f ^ ( ( ~ is ) L2 on R since by Theorem 3.26 tjhis is a,i~t,orna,t,ic given the assumption that f (x) is L1 and L2 on R. Theorem 3.28.

(Theorem 3.13) I f f (z) is L' and L%n R, t h e n

3.6. The Fourier Transform for u unctions

75

Exercises Exercise 3.29. Complete the proof of Theorem 3.26. (Hint: Use Theorem 1.41 and Corollary 3.12.) Exercise 3.30. Prove Parseval's Formula (Theorem 3.27). (Hint: Consider the function g(x) = g ( - r ) , and repeat t,he a,rgllment in the proof of Theorem 3.26 with appropriate modifications.)

Exercise 3.31.

Prove that sin2( t )

dt = T ,

where the first integral is interpreted as

k i sin(t)

dt

= 2 1im )'+a2

sin ( t ) -dt

t

since sin(t)/t is not an Ls function. Hint: Prove the first equality by integrating the second integral by parts, and prove the second equality using Plancherel's Forrxlula and Example 3.3(a,).(See Benedetto, Harmonic AuaI.ysis and Applications, p. 25.)

3.6 The Fourier Transform for L2 Functions Until now, we have been rnaking the assunlption that a function f (x) must be L1 on R in order for its Fourier transform to be defined. This assumption was rnade in order t80gi~a~rantee that the integral in (3.4) converges absolutely for each y. However, we have seen examples that suggest that we need to expand the definition to a larger class of functions. Specifically,

y(?)

f ( L ) is L1 on R, but is not, and in order for equality to hold in both parts of (3.3), we would like to be able to make the statement that if

that is, that

The question is: How do we interpret the integral in (3.29) since it does not converge absolutely?

76

Chapter 3. The Fourier Transform

We have sccn the answer already in Theorem 3.28, which asserts in this case that

in L~ on R. That is, we interpret the nonconvergent integral (3.29) as a limit (in the L~ sense) of convergent integrals. The remaining question is: C a n we do this with any L2 function? The answer is "Yes," but the proof of this assertion is beyond the scope o l this book and involves knowledge of the theory of Lebesgue measure and the Lebesgue integral. We state the relevant theorem for completeness. C L T ~function f (x),L~ o n R, there exists a function Theorem 3.32. Gi~uer~ ,f^(?), L~ o n R (in the sense of Lebesgue), such that

iirn 7+0+

(Z)e-"'

2

2

' e2"77x d s = f ( i )

(3.31)

zn L~ o n R. I n this case, Plancherel's formula holds; that is,

and the Fourier inziersiorz holds in the sense of Theorem 3.28; that is,

3.7 Smoothness versus Decay One of the basic principles of Fourier Transform theory can be loosely stated as follows: T h e smoother f ( z ) is, the more rapidly f^(?) will decay a t infinity, and conversely, the more rapidly f decays at infinity, t h e smoother f (y) will be. There are many ways to measure the smoothness of a given function f (s), but for the purposes of this book, we will measure smoothness of f (x) by counting the number of continuous derivatives it has. We have already seen an illustration of this principle in Theorem 3.8, which asserts that if f (z) is L1 on R (a statement about its decay at infinity), then j^(-y) is uniformly continuous on R ( a statement about its smoothness). In light of the Fourier inversion formula (Corollary 3.12), we can assert that if an L1 function f (z) has an L' Fourier transform (decay of

(XI

A

3.7. Smoothness versus Decay

77

f^(?) at infinity), then f (z) is also uniformly continuous on R (smoothness of f ( x > ) . A more precise statement of this duality starts with the following theorem. Theorem 3.33. (Differentiation Theorem) If f (x) and x f (x) are L' o n R, t h e n f^((r) is continuously differentiable o n R, and

Proof: We wish t o show that for each y,

First, form the difference quotient for

Since

e-2~ihx

f^(?) and calculate.

-

1

lim

= -2nix, h the proof reduces t u justifying the interchange of a limit and an integral. Specifically, we must prove that h-tO

) dx. We will make two estimates on thc quantity ( l l h ) (e-2nihx - 1). First, we expand the function g ( h ) = e-2Tihx about h = 0 in a Taylor series and use Taylor's formula (keeping only one term in the expansion) t o obtain the estimate

Taking now two terms in the expansion, we obtain the estimate

5

I h 2'

-

max

olsih

1

d2 -e

ds2

x

n

2

x

2

(3.37)

Chapter 3. The Fourier Transform

78

Using (3.36), we estimate

By hypotllesis, 277- 1x1 If (x)l is L1 on R. Using (3.37), we note that for any R > 0,

as h 3 0. Therefore, by Theorern 1.41, the interchange of lirrlit and integral is justified and (3.34) follows. The following corollary to Theorem 3.33 can be proved by induction (Exercise 3.37).

Corollary 3.34. f^((y)

If f (z)and x N f (x) are L' o n R for s o m e N E N , t h e n is cN o n R , a n d f o r 0 5 j 5 N ,

We can state ;L partial converse of Theorem 3.33 relating srnoothrless of the Fourier transforrrl of a function to the decay at infinity of the function itself. A

Suppose t h a t f(x) i s L1 o n R, and t h a t following hypotheses.

Theorem 3.35.

( a ) F o r s o m e N E N, (b) B o t h ?(?) and (c)

f^((y)

is

f(?)

cN o n R.

jlN)(?) are L'

f?")(?) or o 5 j 5 N , I Iirn rl-)~

o n R. = 0.

Then Iirn zN f(x) = 0 IX+W

Proof: Consider the function F ( x ) defined by the integral

satisfies the

3.8. Dilation, Translation, and Modulation

79

Integrating by parts N times and using (a) and (c), we obtain

Using ( b ) and the Fourier inversion formula,

Hence F (z)= (2niz) f (x).By (b) and the Riemann-Lebesgue Lernrna (Theorem 3.9), lim,,*, F ( x ) = 0 and (3.39) follows. Finally, we present a theorern relating decay at infinity of the Fourier transform of a function to smoothness of the function.

Theorem 3.36.

If f ( x ) and?(?) are L' o n R and i f T N a y ) is L' o n R for some N E N , the,,&J ( z ) i s c No n R, and for 0 j <: N ,

<

Proof: By the Fourier Inversion Formula (Corollary 3.12), we car1 write f ( x )=

~ ( y~~~~7~ ) dy.

Equation (3.40) follows by repeated application of the argument in the proof of Theorem 3.33. The fact that f ( s )is C N on R follows since ?Jf^(?) is L1 on R and b y applying the argurrlent of Theorern 3.8.

Exercises Exercise 3.37. Prove Corollary 3.34 Exercise 3.38. Prove that if f ( x ) has period 1 and is C K on R, then there is a constant A > 0 such that for all n E Z: Ic(n)l < A J n l - K ,where c ( n ) = J; f (z) e-2T"i"" dx.

3.8 Dilation, Translation, and Modulation Definition 3.39. Given a > 0, the dilation operator, D,, defined o n functions f (x), L' or L' o n R, is given b y

80

Chapter 3. The Fourier Transform

Given b t R, the translation operator, Tb, defined o n functions f (x), L' o r L' on. R, is giuen by (3.42) Tbf(x) = f ( x - b). Given c E R, the modulation operator, Ec, defined o n functions f (x), L I o r L~ o n R, is given b y E , f ( z ) = e2""'" f ( ~ ) . (3.43)

Theorem 3.40.

For any function f (x) L' o n R, A

( a ) For every a > 0 , Daf (7) = ~ 1 / ~ 7 ( 7 ) .

Proof: Exercise 3.44. Remark 3.41. (a) Note that if a > 1, then D, f (x) is a "narrowed down" version of f (x), and if 0 < a < 1, then D, f (x) is a "spread out" version of f ( x ) . Theorern 3.40(a) says that when a function is "narrowed down" by dilation by a > 1, its Fourier transform is "spread out" by dilation by 0 < l / a < 1.

(b) Theorcrrl3.40 says that modulatiori in the time or spatial variable corresponds to translation in the frequency variable. For this reason, modulation is often referred to as a frequency shift or phase shift. Theorem 3.42. (Properties of Dilation and Translation) For every f ( z ) and g ( z ) , L~ o n R, and for e7rery a > 0 , b E R, the following hold. (a) D,Tbf(z) = a"2 f ( a x - 6 )

(e)

( f , DaTb9) = ( T - b D a - l f ,

( f ) (Dof , Dagj =

(f

9

9).

g).

Proof: Exercise 3.45. Theorem 3.43. (Properties of Translation and Modulation) For every f (x) and g ( z ) ,L~ o n R, and for every b, c E R , the following hold.

3.9. The Sampling Formula

(a) TbEcf (x) = e

-27rzbc

(b)

( f , Ecgj

(c)

( f , TbEcg) = e

81

EcTbf (x).

= ( E - c f ,g ) . 27rzbc

(T-bE-,f. g ) .

Proof: Exercise 3.46.

Exercises Exercise 3.44. Prove Theorem 3.40. Exercise 3.45. Prove Theorem 3.42. Exercise 3.46. Prove Theorern 3.43.

3.9 Bandlimited Functions and thc Sampling Formula Definition 3.47. A function f ( x ) , L'

on R, is bandlimited zf there is a number 0 > 0 such that f^(y) is supported in the interval [ - R / 2 , 0 / 2 ] . In this case, the function f (x) is sazd to have bandlirnit 0 > 0. The function f (x) has bandwidth B > 0 if there is an interval I such that II/ = B and such that f (7)is supported in I . A

Remark 3.48. (a) A furlclivn J(x) with bandlimit R > 0 also has bandwidth R > 0. However, in general the numbers are not the same. For example, let f (x) be the function whose Fourier transform f^(?) equals (?). (Whet is f (x) in this case?) Then the bandlirnit of f (x) is 2, whereas the bandwidth is 1.

(b) The bandlimit and bandwidth of a function f (x) are not unique numbers. For example, iff (x) has bandlimit fl > 0, then f (x) also has bandlimit 0' > 0 for any number 0' > fl. Similarly, if f (x) has bandwidth B > 0, then f (x) also has bandwidth B' > 0 for any number B' > B. (c) Intuitively, if f (x) is bandlimited, then f (x) does not contain arbitrarily high-frequency components. The Fourier inversion formula for a function with handlimit fl looks like

82

Chapter 3. The Fourier Transform

That is, f (z) consists only of "frequencies" e2"irz of period 2/R or greater. Thus, one might expect that a bandlimited function would be slowly oscillating and not have any sharp jumps or disconlinuities. In fact, the following theorern holds.

Theorem 3.49. Let f (x) be a bandlimited function with bandlimit R . Then: (a) T h e Fourier inversion formula holds for f (x);that is, for each z E R,

(b) f (x) is C" o n R.

Proof: We will prove (b) first, given the assumptiori that (a) holds. We would like to use Theorem 3.36, since f (x)being bandlirnited iniplies lllat f^(?) is L1 arid that rN is L' for every N F N. However, since we have only assuriied that f (x) is L2 on R and not necessarily L1, we carinot use the theoreni directly. However, if we examine the proof of Theorern 3.36, we see that all that is required is that the Fourier inversion formula hold. Then the argument in the proof of Theorem 3.33 may be applied. But this is exactly (3.44). In proving (a), we agairi run into the difficulty that f (z) has been assumed only to be L2 and not L1 on R. This is certainly not an insurmountable obstacle, but it does require some rather subtle argumentation. According to Theorem 3.32, the Fourier inversion formula holds in the L2 sense for f (x);that is,

y(?)

in L2 on R. By Plancherel's Formula for the L2 Fourier Transform (3.32), we know that f^(?)is L2 on R Since f ( y ) is also compactly supported, Theorem 1.9 says that is also L1 on R. Therefore, we can prove (see Exercise 3.51) that in fact, A

y(?)

lim

r+o+

Y,

in L" on R. Let us call this uniform lirrlil function g ( x ) ; that is,

Thus, we have an L2 limit function f (x) and an L" limit function g ( x )for the same sequence of functions. So we must show that in fact, these limit functions are the same.

3.9. The Sampling Formula

83

In order to do this, define the functions f, (a)by

and fix a number A

> 0.Then by Minkowski's inequality,

The left side of the inequality is independent of r , and the right side can be made as srnall as desired by choosing r > 0 srna.11 enollgh. Therefore for every A > 0, A

which implies that

If

(x)- 9(2)12dz

= 0.

Since f (x) is piecewise continuous by assumption, and g(z)is in fact continuous by the argument used in the proof of 'l'heorem 3.8, f (z) = g(z), except possibly at the discontinuities of f (x).Since there is no problem redefining f (x)at these points, we can conclude that f (x) = y(x) for every x E R. But this is (3.44). One of the fundamental results in Fourier analysis is the Shannon sampling theorem. The theorern asserts that a bandlimited function can be recovered from its sarrlples on a regularly spaced set of points in R, provided that the distance between adjacent points in the set is srnall enough. The formula is also very important in digital signal processing applications. Theorem 3.50. (The Sharlrlorl Sarrlplir~gTheurem) I1 S ( x ) with bandlimit Cl, t h e n f (x)c a n be w r i t t e n as

where the s u m converges in L2 and L"

,is b~7idli,11L%led

o n R.

'This theorem has a long and interesting history that is recounted beautifully in the article by Higgins, Five short stories about the cardinal series, Bulletin of the AMS, vol. 12 (1985) 45-89.

84

Chapter 3. The Fourier Transform

Proof: Since f^(y) is supported in the interval 1-R/2, fl/2], we can expand f^(y) in a Fourier series and obtain

for y E [-fl/2,fl/2], where

But by (3.44), it follows that

Making the change of summation index n

+ -n

leads to

Again applying (3.44), we obtain

where we have used the fact that Fourier series can be integrated term-byterm (Exercise 2.67) and that for any numbers a > 0 and b # 0,

(Exercise 3.4). To see that the convergence of (3.47) is uniform on R, let N, M E Z be fixcd.

3.9. The Sampling Formula

85

where we have used the Cauchy-Schwarz inequality and where the c(n) are t,he Fourier coefficients of f^(y). But since the Fourier series of f (?) , . coriverges to f (y) in L~ on [-012, fl/2], A

The L2 convergerice of (3.47) follows from the fact that the collectio~i

is an orthonormal system on R (Exercise 3.52).

Exercises Exercise 3.5 1. Prove equation (3.46). Exercise 3.52. Prove that the collectiorl

is an ortllonormal system on R. (Hint: Use Parseval's formula.)

Exercise 3.53.

x E R,

Show that if f (x) has bandlimit fl > 0, then fbr every

86

Chapter 3. T h e Fourier Transform

Exercise 3.54. Show that if f ( x ) has bandwidth B > 0, then f (x)can be completely recovered from the samples { f (TL~B)),,~. Exercise 3.55. Let f (x) be L~ on R, and let r > 0 be given. Prove that there exists a handlimited function g(x) such that $(y) is C0 on R and 11 f - g1I2 < r. (Hint: Use Corollary 2.37(b) in the Fourier transform domain.)

Chapter 4

Signals and Systems In the previous chapter, we considered piecewise continuous functions with period 1 and showed that it is possible to represent such functions as an infinite superposition of exponentials en(t) = e2"int, n E Z. Each such exponential has period l / n and hence completes n cycles per unit length (which we can interpret as measuring time). If the exponentials are interpreted as "pure tones" of n cycles per second, then each f (t) has a "frequency representation" of the form

where

We also know that

Conceptually, there is nothing stopping us from changing our perspective and regarding the sequence { f (n)InEz as the object to be given a "frequency representation." In this case, (4.2) is such a representation in ,. which we consider f (n) to be a coritinuous superpositiorl of "pure tones" on Z, e,(n) = e2T"Lx,which complete about one cycle every l / x time steps (indexed by r ~ ) Equation . (4.1) now gives a forrnula for the coefficients in this continuous frequency representation. This new perspective is very well suited for digital signal processing (DSP) applications in which data are necessarily in the form of arrays of numbers. These arrays are of course always finite but can be of arbitrary length. Hence, it is convenient t o regard these objects (to which we will give the name signals) as being infinite sequences. A related perspective regards signals of length N as periodic sequences with period N . This chapter provides a discussion of some of the ternlinology and basic results of the mathematical theory of DSP from both perspectives. A

88

Chapter 4. Signals and Systems

4.1

Signals

Definition 4.1. A signal is a sequence of numbers {~(n)),~z satisfying

Remark 4.2. By basic results on convcrgcnt scrics, any signal must be bounded, that is, there is a number M > 0 such that /x(n)1 < M for all rL E Z. It is also true that any signal satisfies C , Ix(n) 1' < m. Such sequences are said to be (Exercise 4.6).

Definition 4.3.

t2 sequences or somctimes to have finite energy

The frequency domain representation of a signal x(n)is the

function

Remark 4.4. (a) Since En 1x(n)1 < a, the sum defining ?(w) converges ilniformly to a continuous function with period 1.

(b) Recall that the set {e2""}w,,[o,I) is the unit circle in the complex plane. This is because if z = e2"7w,then lzl = 1. Hence the function X(eZKLW) car1 be tlwught of as the restrictioii t o tlie unit circle of some function X ( z ) defined on some portion of the conlplex plane containing the unit circle. Specifically, we can define

wherever the surn makes sense. The function X ( z ) is referred t o as the z t m n s f o r m of z(n,).

Example 4.5.

Also,

(a) Let z ( n )=

1 ifO
Then,

4.1. Signals

89

See Figure 4.1. (b) Let la( < 1, and let

Then,

and CU

X(z) =

C a"

z-n, =

r~=0

whenever lzl

1 1-az-I

-

Z

z-a

> lal. See Figure 4.2.

Exercises Exercise 4.6. Show that every signal is an f 2 sequence and that there are f 2 sequences that are not signals.

90

Chapter 4. Signals and S y s t e m s

FIGURE 4.2. Left: x ( n ) of Example 4.5(b). Right: the real part of z ( w ) .

4.2

Systems

Definition 4.7. ( a ) A s y s t e m is any transformation T that takes a n input signal x ( n ) t o a n output signal y ( n ) . We write T x ( n )= y(n). ( b ) A system T i s linear if

for every pair of signals

XI

( n ) and

5 2 ( n ) ,and

every pair of constants a and b.

( c ) A linear system T is stable i f there is a constant C signals x ( n ),

>

0 such that for all

( d ) For ~ z oE Z , ,we d e f i r ~ el l ~ eLrarlslatiorl operator, rn,,, by

Tn,,x(n)= x ( n - n o ) . ( e ) A linear translation-invariant ( L T I ) s y s t e m is a linear s y s t e m T for which

T(rnox ) ( n )= T~~ ( T x )( n )= T x ( n - n o ) . ( f ) T h e convolution of two signals szgnal y ( n ) given by y(n) = xl

rl

( n ) and x z ( n ) , denoted

* x2(n) =

C x 1 ( I C )x Z ( n

-

li).

XI

* x z ( n ) , is

the

4.2. Systems

91

Theorem 4.8. (a) If x1 ( n ) and x z ( n ) are signals, then so is y ( ~ z ) = X I * x2 (7.1).

( b ) For any pair of signals

XI

( n ) and

2 2 ( n ),

(c) Let h ( n ) be a signal, and define the transformation Th o n signals b y

T h e n TI, is a stable LTI system.

Proof: (a)

(b) Exercise 4.25. (c) By (a), whenever z ( n ) is a signal, so is Thz(n), so that Tjzis a system. That Th is linear is a simple verification (Exercise 4.26). That Th is stable follows from (a) by taking C = C , Ih(n)1. To see that Th is LTI, fix no E Z. Then

by a change of summation index.

Remark 4.9. (a) The r~otiorlof stability defined in Definition 4.10(c) is another way of saying that T is continuous in the sense that a small change in the input z ( n ) results in only a small change in the output y(n) = T z ( n ) .

92

Chapter 4. Signals and Systems

(b) To be more precise about what this means, we define a n o r m on the class of all signals (called the tl-norm) by

We measure the distance between two signals z l ( n ) and z;?(n)as ((zlzz( ( e l ,so that a small change in z ( n ) (to say Ic(n)) means that - lcllPl is small. In this notation, the inequality in Definition 4.10(c) can be written as

IITzIlel I C llll:lel. (c) We can now give a precise definition of continuity as follows: Given > 0, there is a S > 0 such that whenever lllc < 6, IlY'z - T.Zllel < 6 . An equivalent definition (Exercise 4.27) is the following: Let {zk ( n ) ) k , ~ be a sequence of signals converging t o the signal z ( n ) in the sense that 1l.z. - xkllp -+0 as k + m. T h e n T z k ( n ) convenjes to T z ( n ) in the sense that llTz - Tlckllpl + 0 as k + m.

zllel

E

Our next goal is to show that all stable LTI systems have the form Th for some signal h(n) (Theorem 4.12(c)).

Definition 4.10.

T h e unit impulse signal 6 ( n ) is defined b y

See Figure 4.3.

Lemma 4.11. A n y signal x ( n ) can be written as a s u m of shifted impulses; that is, for each n E Z,

Proof: Exercise 4.28. Theorem 4.12. Let T be a stable L T I system. T h e n there is a signal h ( n ) such that

T s ( n )= ( x * h ) ( n )=

a ( k )h ( n - k ) .

(4.5)

kEZ

Proof: Define h ( n ) = (Tb)(n). Since T is a system, h(n) is also a signal; that is, C , J h ( n ) J< m. Given a signal z ( n ) , we can write z(n) =

4.2. Systems

93

FIGURE 4.3. The unit impulse.

C ~ Ex(k) Z rk6(n) by

(4.4). Since T is a stable LTI system,

( T z )(n)

~ ( k( )T T ~( n~ ))

= k€Z

which is (4.5). The assurnptions of stability and linearity are required to justify pulling T inside an infinite sum like we did above. The details for this justification are the content of Exercise 4.29.

Definition 4.13.

Given a stable LTI systern, T , the signal h ( n ) such that T x ( n ) = ( x * h ) ( n ) is called the impulse response of T. T h e impulse response of a stable L T I system i s often called a filter. The frequency representatzon of h ( n ) , h ( w ) , is called the frequency response of T, and the z-transform of h ( n ) ,H ( z ) , is called the system function of T .

A

Remark 4.14. (a) Referring to h ( n )as an impulse response makes sense because according to the proof of Theorem 4.12, (TS)(n)= h ( ~ z )so, that h(n) is the response of the systenl to a unit impulse. (b) That x ( w ) is called the frequency response also makes sense for the following reason. Suppose that the input to the system T was a "pure frequency" z ( n ) = e2"inwo. Even though z ( n ) is not a signal according to our definition, it still makes sense to form the convolution (z* h ) ( n ) .

94

Chapter 4. Signals and Systems

Therefore,

( T s )( n )

s ( k )h ( n - k )

kEZ

x x ( n - k)h(k)

=

k€Z

C

-

e 2 x i w ~ ( n - k )h

k€Z -

e2ninw"

C h(k)

(k) e-2xikwU

k€2 -

e2xinwo

IL(WO)

A

In other words, thc signal z ( n ) passes through the system T unchanged ex,--. cept for multiplication by the constant h ( w o ) .We say that pure frequencies ,-. are eigenvectors of LTI systems with eigenvalues given by h ( w ) . (c) A similar calculation is valid when dealing with real valued signals and justifies some further terminology. Suppose that the input to the system T is a real sinusoid of the form

Moreover, by (b),

Assuming that h ( n ) is real valued, we have that

(Exercise 4.30), so that writing

it follows that Therefore,

A

A

A

h(-wo) = h ( w a )= I h ( w o )1 e - ' Q ( W o ) .

4.2. Systems

95

In other words, the signal z ( n ) passes through the system T unchanged except for multiplication by the real constant Ih(wo)l and a phase shift by O(wo). We refer to lx(w)l as the m a g n i t u d e response of T and to B ( w ) as the phase response of T . A

Remark 4.14(a) can be thought of as a particular case of the following theorem.

Theorem 4.15.

Let

XI

( n ) and x z ( n ) be signals, and let y ( n ) =

* xz)(n).

(zl

Then

G(W)=

A

J;1

( w) $2 (w) .

Proof:

=

z, ( k )z 2 ( n - k ) e-2"zn,w

zl (k)

=

i a ( n- k ) e-2T'n"

Corollary 4.16. Let T be a n L T I system with frequency response x ( w ) . T h e n for every signal x ( n ) , T z ( w ) = z ( w )h ( w ) . h

4.2.1

Causality and Stability

Definition 4.17.

A stable LTI system T is causal

satisfies h ( n ) = 0 for n

2f

its impulse response h ( n )

< 0.

Remark 4.18. (a) The system function for a causal system has the form

which is a power series in z - l . Consequently, wc can talk about the radius of convergence of H, and note that if R > 0, then H ( z ) converges uniformly for 1x1 > R - l .

96

Cliapter 4. Signals and Systems

Obviously, if h(n) is a causal FIR filter (Definition 4.24), then the system function H ( z ) is a polynomial in z-I. (b) If T is causal, then given r ( n ) ,

(Ts)( n )

z ( k ) h ( n - lc)

= kEZ

~ ( nh(0) ) + z ( n - 1)h(1) + z ( n - 2) h(2)

=

+ ....

Hence the output y ( r ~ )at n = no depends only on the input z ( n ) at n = no, no - 1, no - 2, . . .. If we imagine that our input signal z ( n ) is some time series, then T being causal means that the output q ( n o )does not depend on the "future" values of the input (that is, x ( n ) , n > no) but only on the "past" values of the input (that is, s ( n ) , n 2 n o ) In this sense, any realistic systeni rnust be causal. Definition 4.19. A system T is realizable zf the relation between the input x(n) and the output y(n) of T is given by a n equation of the form

for each n E Z

Remark 4.20. (a) Under the assuniption that a(0) note that (4.6) car1 be written for each n as y(n)

=

b ( O ) . x ( r ~ ) + b ( l ) z ( n -1)

+

- a ( o ) ' ( a ( l ) y(n - 1)

+ - - . + a ( K ) y(n

#

0 and b(0)

# 0,

+b(A4)x(n-hf)

+ 4 2 ) y(n - 2)

-

K)).

In other words, the output y(n) a t n = no depends on the current value of the input z ( n a ) and the values of the input M steps in the past, and on the values of the output K time steps in the past. Of course, the output of such a system at times n = 0, 1, . . . will differ depending on the K "initial values" of the output, y(-1), y(-2), . . . , y(- K ) . These values may in principle be computed knowing the values of the input s ( n ) infinitely far in the past, but as a practical matter, it is usually assumed that y(-1) = y(-2) = - . . = y(-K) = 0.

(b) Note that each side of (4.6) is a convolution with the finite filters a ( n ) and b(n). Thus, (4.6) can be rewritten as (a * Y) (n) = (b * z)(n) . By Theorem 4.15,

4.2.Systems

97

Hence the system function for T is given by

where

B ( t )=

x

x --

b(m)z P m

and

A(z) =

a(k) z p k

so that R(x) is a rational function of x-l. (c) Note that if R ( r ) is a rational function in z-', it is clearly also a ratiorial lu~lctionin t . This is true since z N B ( z ) and z N A ( r ) are polynomials in z as long as N 2 max(M, K). Therefore, the system function of a realizable system is completely determined by its poles (the zeros of its denominator) and its zeros (the zeros of its numerator). (d) We know that given n rational function R(z) in wlliclr the degree of the denominator is greater than or equal to the degree of the numerator, we can write the rational function R(z)/z in a partial fraction expansion as follows:

where the pi are the poles of R ( t ) / z , m, is the multiplicity of each pi,and

(e) Next note that if a

# 0, then

whenever 1x1 > la/. Taking the derivative with respect to a , we get

Therefore, whenever 1x1 > /a/,

98

Chaptcr 4. Signals and Systems

Hence we can write

which will converge as long as lzl > max{lpi I).

By the above calculations, we have nearly proved the following theorem.

Theorem 4.21.

If the system function, R ( z ) , of a realizable system T has all of its poles inside the unit circle of C , then T i s causal and stable.

Proof: It follows from Remark 4.20(e) that T is causal with impulse response given by

To show that T is stable, we rnust show that En Ir(n) 1 note that r ( n ) is a finite surn of terms of the f o r n ~

< m. To do this,

and that for each i and j : (n+l?!

$+'-J

j!qn + 1 - j)! n! Ai,j j!(n - j ) ! P?

as n + m . If pi1 < 1 for each i , then for each i and j, and it follows that C , Ir(n) I < m .

Example 4.22.

(a) Let

C , Ir(n,i,j)l < m ,

4.2. Systems

99

This rational function has first-order poles at x = 1/2 and z = 3. Expanding R(x)/x in a partial fraction expansion, we obtain

and

Therefore, T is causal but not stable.

(b) Let

-

This system function has a first-order pole at z = 1/2 and a second-order pole at x = 1/3. Expanding in partial fractions, we obtain

Therefore, T is both causal and stable.

Remark 4.23. (a) Note that in the above example, the impulse response of each systcrrl described is infinite in length, so that it direct computation of T x ( n ) fur some signal x ( n ) would in principle require infinitely many calculations. In fact only finitely many calculations are required since Tx(n) can he realized as the solution to the finite difference equation (4.6). (b) The infinite lengths of the impulse responses of the systems in Example 4.22 arise from the fact that the system function has nonzero poles. A pole at t = 0, cven of high order, will not result in an irlfirlite impulse response. Such a system is called a delay (Exercise 4.31). This leads to the following definition.

Definition 4.24.

The i17zpulse Y ~ S ~ O T of L Sa~ realizable system whose system function has n o poles except possibly at z = 0 is called a finite impulse response (FIR) filter and one with nonzero poles inside the unit circle is called a n infinite impulse response (TTR)filter.

100

Chapter 4. Signals and Systems

Exercise 4.25.

(a) (xl

Let x1 (n), x2(n), and h(n) be signals. Show that

+ x 2 ) * h(n) = X I * h(n) + 2 2 * h ( n ) .

(b) If a is any number, then ( a 21) * h(n) = a (xl* h) (n).

Use (a) and (b) above to show that any system defined as in Theorem 4.8(c) is linear. Exercise 4.26. Prove Theorem 4.8(b). Exercise 4.27. (a) Show that the two definit,ions of continuity given in Remark 4.9(c) are equivalent. That is, show that a systern T satisfies the first definition if arld only if it satisfies the second.

(b) Show that a stable linear system is continuous by showing that such a system satisfies either definition of continuity given in Renlark 4.9(c). (Hint: If you use the first definition, then given 6 > 0, take S = E I C . ) Exercise 4.28.

(a) Prove Lemma 4.11 by showing that for each n t Z, s(k)S(n-k) =x(n).

lirn

N-+co

k--N

(b) Prove that in fact (4.4) holds in the following strong sense:

lirn

N+m

1Ix(n) k=-N C z(k) b(n -

-

k) = 0.

~LEZ

In the notation of Remark 4.9(b), we are being asked to prove that lim IIx - :xNllel,

N+m

where

N

Z N ( ~= )

C

x(k)h(n - k).

k=-N Exercise 4.29. (a) We say that x ( n ) is a finite signal if there exist numbers N, M E Z such that z ( n ) = 0 if n, < I\/[ or n > N. In other words, x ( n ) is a finite signal if it has only finitely many nonzero entries. Prove that, given a signal x(n), there is a sequence of finite signals {xk ( n ) j k E N such that lirnk,, Ilx - xlc = 0.

(b) Note that all calculations in the proof of Theorem 4.12 are legitimate for finite signals ~ ( n ) Show . that the stability of T implies that in fact, Tx(n) = (x * h)(n) for all signals x ( n ) (finite or not).

4.3. Periodic Signals and the DFT

101

A

Exercise 4.30.

Prove that if h ( n ) is a real valued signal, then h(-ci?) =

A

h(4. , Exercise 4.31. Show that if R ( s ) = s P mm system T is givcn by T x ( n ) = z ( n - n ~ ) .

> 0, then tlie correspoiiding

Periodic Signals and the Discrete Fourier Transform A different inode1 for thinking lnathenlatically about finite sigrlals is to corisider the finite signal to be infinite in length but periodic. Iri other words, given a finite data set of leligtli N , ( ~ ( 0 )x(1). . . . . , . r ( N l ) } .define a corresporidirig irifi~iitesequence F(n),n E Z, by F ( n ) = . c ( r ~11iod N ) . Iii this case, . c ( n ) = J:(TI,)wlierlever 0 < rl, < N , so that .?(?I,) is considcrctl an extension of

n.(~i,).

FIGURE 4.4. Left: A signal tension F(n).

~ ( 7 1 )of

length 5. Right: its period-5 ex-

Definition 4.32. G i v e n N E N , a sequence { ~ ( n ) tz ) , ,is a period N sigrial "x(n+ N ) = z ( n ) t o r all 7~ g Z . I n this case x(n) is said to be periodic. Remark 4.33. (a) It is clear that a period N signal, unless it is identically zero, can never be a sigrial in t,he sense of Definition 4.1, since the absolute values of its entries will always surn t o infinity. However, a periodic signal is bounded in the sense that there is a number A f such that I.c(n)1 A1 for all n E Z.

<

102

Chapter 4. Signals and Systems

(b) Even though a periodic signal is never a signal in the sense of Definition 4.1, it is always possible to pass a periodic signal through a stable LTI system and have the result make sense. Since stable LTI systems are characterized by their impulse response filter, this statement amounts to the statement that the convolution of a filter and a periodic signal is well defined. This is the content of the following theorem. Them-err1 4.34. Given u filler. h(71) a n d u. period N signal x(n),t h e convolut i o n x * h(n) i s defined for all n a n d i s a period N signal.

Proof: That z* h ( n )= CktZ ~ ( kh )( n - k) is defined for all n amounts to showing that the sum converges for each n. But this follows from the fact that for sorne M, Ix(n)l5 M for all n and the calculatiorl

To see that z * h,(n) has period N, note that

=

z ( k ) h(n - (k - N))

since x(n)has period N. Since filtering operations are defined on periodic signals, it makes sense to look for an interpretation of such an operation in the frequency dorrlairl analogous to Corollary 4.16. Consequently, we need a notion of frequency representation for periodic signals. This is done via the the Discrete Fourier Transform (DFT).

4.3.1

The Discrete Fourier Transform

Definition 4.35.

G i v e n a period N signal x(n),the (N-point) Discrete Fourier i s t h e period N sequence

T~-ar~slor.rr~ ur. (N-point) D F T of z ( n ) , denoted

defined b y N-1

z(n),

4.3. Periodic Signals and the DFT

103

The DFT is invertible as follows.

Theorem 4.36.

G i v e n a period N sequence x(n)with D F T ;(n),

for each j E Z .

Proof: Note first that for any nurnber r ,

/ir

l-r

'

n=O

so that N-1

In

n=O

Hence, for 0 I j _< N

-

Therefore, for each 0

<j
-

1- e 2 ~ i ( . i - k ) / N

'

1,

-

1,

N-I

Since the sum

1-e2rd~-k)

N-1

N-1

defines a period N sequence in j , the above holds for all j E Z. We can now prove the following theorem relating the DFT of a periodic signal to the DFT of a filtered version of that signal. Theorem 4.37.

L e t h ( n ) be a filter a n d x(n) a period N signal. T h e n

Chapter 4. Signals and Systems

104 where

z(n)is the D F T of z ( n ) and x ( w ) is given by Definition 4.3.

Proof:

,.

Remark 4.38. (a) Notice that since h(w) has period 1, the sequence h(n,/N) has pcriod N . Therefore, it is reasonable to ask if there is some period N signal h(n) wllose N-point DFT is X(n/N), and if there is such a periodic signal, how is it related to the filter h,(n)?

A

(b) To find the period--N signal h(n,),all we need to do is take the inverse DFT of ;(n/N). This gives

BY (4.91, N - l

N - l e2n"".(-~)/N En=,

0 otherwise. Therefore,

= 1 if j -

k. = ,N

for some

V Lt

Z and

4.3. Periodic Signals and the DFT

We say that

h(n)is the

-

105

N-pcriodizntion of h ( n ) .

(c) In terms of h(n), we can write the formula for the convolution of x(n) with h ( n ) as follows.

Since z ( n ) has period N , x ( j - m N ) = x ( j ) for all m and we continue

This suggests the following definition of the convolutiori of periodic signals.

Definition 4.39. Let x(n) a n d y ( n ) be period-N signals. T h e n the circular convolution of x ( n ) a n d

~ ( ' I L ) is

defined b y N-I

x * y(n) =

Remark 4.40. cise 4.42(a)).

(a) Note that x

x ( k ) y ( n - t).

-c ~ ( 7 7 ; ) is

also a period-N signal (Exer-

(b) The sum in Definition 4.39 can be obtained by surrlnlirlg over any N adjacent indices. To see this, let j E Z be given, and let m be the unique integer such that (m - I)N 5 j < m N . Then,

106

Chapter 4. Signals and Systems

Frorn this it can he shown that x * g(n) = g * x(n) (Exercise 4.42(\1)). (c) Circular convolution car1 be realized as nlultiplicatiorl by a matrix whose rows are shifts of one anotlier. Giver1 a period N sequence z ( n ) ,define the matrix X by

If y has period N and r (n) = x * :q(n),define y = [g(0) r = [r.(O) . - . r ( N - I)]. Then r = Xy.

Theorem 4.41.

. . . y (N- I)] and

L e t x(rr) a n d ? / ( r ~he ) period-N sign,al.s, a n d let F ( n ) a n d c(n)

be t h e l r DFTs; t h e n (3:

whew (x * y)"

(TI)

* !J)"

(11)

d e n o t e s the D F T of

=~

:L:

I

L ~ )( T L ) ,

* IJ(TL).

Proof: Exercise 4.43.

Exercises Exercise 4.42.

Let x(n) and y(n) be period-N signals.

(a) Prove that thc circular convolutiorl x * y ( n ) is also a period-N signal. (b) Prove that z

Exercise 4.43.

* y(n) = y * x(n).

Prove Theorem 4.41.

Exercise 4.44. Prove that if 2(n)is the N -point DFT of the period-N signal x (n,), then N-1

N-1

4.4. The Fast Fourier Transform

107

The Fast Fourier Transform

4.4

The N-point DFT can be thought of as a linear transformation on the finite-dilnensional vector space C N ,and hence can be writtcrl as a matrix. That is, given the vector

its DFT,

can 1)e writtcn

Tlie matrix WN dofirled 1-y (WN),,,i= where CI:, = syrnrnctric, ort2logoiia1, and

w$'.

is

Tlic irlversc DFT can he writtc.11

where Wh is the atljoint of WN. Notc tliat all of the cnt,ries in tlie rnatrix WN have absolute value 1. This illearis that to coinpute the DFT of a period N sequcnce . c ( n ) by doing straight rnatrix in~lltiplicationwill require N%ri~llti~lications. This niakcs the tlircct iinplerrlcritatiori of the DFT inipractical for large N . Fortllrlately therc is an algorithrrl known as the Fast Fo7~rler.Tmn,sfonn,or F F T t,h;~t, s p ~ p d su p thc? co~~lputatiorl of thc: DFT c~rlsiclorabl~.~ Tlie idea hehincl the F F T is the following. Lot N E N witli N even. and \Vnr = ?-"dlN. If N = 2 A f , t,heil

Now. given ;-L period N scqlierlce .c(2j 1). Tlleri

+

.C(TL)

(lefiiie ( ~ ( j )= x ( 2 . j ) a~lclb ( j ) =

'The following discussion is adapted frorr~Papolilis, Signal Analysts, hlcGraw-Hill (1977), p. X2ff.

108

Chapter 4. Signals and Systems

where Z ( n ) and a(n)are the W-point DFTs of n and b respectively. Note further that

so that if 0 5 n 5 A1

and if hf 5 r~ 5 N

-

z(71)

-

1 , the11

1, then

1 )+ 1

(71-hI)

=

(

=

ii(7~+ ) win' w;i(rL)

1 -

-b(n

-

hl)

A

=

a ( ri,)

A

-

W; b (n),

A

rernern1)cring that Z ( n ) and b ( n ) have period h l . In inatrix notation, the above discussioll can be sumrnarizcd by the fol. . . ~ ( A I- I ) ] . lowing. Let S = p(0) 6(1) . - . Z ( h 1 - I ) ] and = Tlleil

6(0)Z(l)

(4.10) where flnl is the A4 x h! diagonal matrix giver1 by

and PN is the N x N permutation matrix that separates the even and odd entries of a vector. For example,

4.5. L~ Fourier Series

109

The efficiency of the F F T comes from the fact that the dense rnatrix WN can be factored into a product of sparse matrices as in (4.10). If Ad is a,lso even, then Wnr can also be factored in this way, reducing conlputations still further. The greatest reduction in conlputations is realized when N = 25 for some s > 0. In order t o determine the cornplexity of the F F T , define m ( N ) t o be the number of multiplications required to compute an N-point DFT using the F F T algorithm. Theri since

corrlplitirlg 2 requires rrb(N/2) ~rlultiplicatioilsto cornpute each of 2 and b arid N/2 r~~~~ltiplications to compute W$ b(r1). Thus, A

Thcrcforc, we lii-lve the following theorem.

Theorem 4.45.

If N = 2" f o r sorne s E N , then

Proof: 'l'l1e proof' is by induction. Sirice n1,(4) = 4 the result holds for N = 4. Assurrie that it holds for N. Now. by (4.11), rn(2N)

+N

=

2711(N)

=

2 ((N/2) lo&2(N)) N log2(N) N log, (2)

=

= =

+

+

N(log2( N ) log;?(2)) N log2(2N).

4.5 L~ Fourier Series By Definit,iorl 4.1, a signal z(n,)rnl~st,sa,t,isfyC,,Iz(n,)1 < oo.LVc ha,ve seen t>liatall signals are .!%equences: that is, that C , Iz(n)l\ m, but that not all t%sequerlces are signals (see Remark 4.2 and Exercise 4.6). It turns out that there is a very rich arid clegarit rnathenlatical theory related t o the Fourier analysis of t2 sequences which provides an entryway into the very powerful theory of Hilbert spaces. The full devcloprnent of this tl-icory is beyorid the scope of this book, but we will need one of the main results of the theory (The Riesz-Fischer Theorem, Theorem 4.48) in Chapter 9.

110

Chapter 4. Signals and Systems

Definition 4.46.

e2

Given an sequence z = denoted z ( y ) , is the period 1 function given b y

The series (4.12) is referred to as a n ~"ourier

{ I C ( T L ) ) ~ ~ its Z,

Fourier series,

series.

Remark 4.47. (a) Since z ( n )does not rlecessarily satisfy En Iz(n)l < m, the sum in (4.12) does not necessarily converge uniformly on R. In fact, it is not clear that such a series even converges pointwise. For example, consider the series 02 cos(;;nt) 71=1

This is just (4.12) with x(n) = 1/(2)nJ)for n # 0 a.nd z(0) = 0. Clearly, x ( n ) is t % ~ ~ if tt = 0 (or any even multiple of T),the series reduces to the llarmonic series arld does not converge. (b) In light of Plancherel's formula for Fourier series (Theorem 2.57(d)), it seems reasonable t o expect that the series (4.12) will converge in L2 on [O, 1). However, if we carlrlot even guarantee that the infinite sum (4.12) converges at any given point, then the question arises: How are we t o interpret the s u m (4.12) as u function defined o n [0, l ) ?

(c) It turns out that the proper answer to this question requires us t o allow into the picti~refunctions that are not piecewise continuous. This more general notion of function is referred t o as a Lebesyue measurable function,, and to integrate such functiorls requires a more general notion of integral than does the Rierrlann int,egral, na,mely, the Lebesyue integrul. Both of tliese ideas require the notion of the Lebesgue measure of a set that is beyond the scope of this book. (d) The Riesz-Fischer Theorem (Theorem 4.48) gives the final word on the convergence of (4.12) in L" and Carleson's Theorem (Theorem 4.49) asserts that (4.12) converges pointwise except on a set of Lebesgue measure zero. Such sets includc finite sets of points and countable sets of points but are by no means limited t o that.

Theorem 4.48.

e2

(the Riesz-Fischer Theorem) Given an sequence { c ( n ) ,) , E ~ , there exists a Lebesgue measurable function f (x) on [ O , l ) with the property that

where the integral is the Lebesgue integral. In this case,

4.5. L' Fourier Series

111

Theorem 4.49.

(Carleson's Theorem) Given an l2sequence { ~ ( n ) } , , ~the z, symmetric partial sums N

converge at each point of [ O , 1 ) except possibly on a set of Lebesgue measure zero.

Part I1

The Haar System

Chapter 5 The Haar System In this chapter we will preserit ari exarrlple of an orthoi~oril~al systeni on [O, 1) known as the Huur system. The Haar basis is the simplest and historically the first exarriple of an orthonormal wavelet basis. Marly of its properties stand in sharp contrast t o the corresponding properties of t,he trigonometric basis (Definition 2.5). For example, (1) the Haar basis functions are supported on srnall subintervals of [0, l ) ,whereas the Fourier basis fiinctions are nonzero on all of [ O , l ) ; (2) the Haar basis functions are step functions with jlirnp discontinuities, whereas the Fourier basis functions are C" or1 [0, 1); (3) the IIaar basis replaces the rlotion of frcqucncy (represented by the index rL in the Fourier basis) with the dual rlotions of scale and location (separately indexed by j and k ) ; and (4) the Haar basis provides a very efficient represeritatiorl of filrlctions that corlsist of srnoot,h: slowly varying segments punctuated by sharp peaks and discontinuities, whereas the Fourier basis best represents f~lnctionsthat exhibit long terrri oscillatory behavior. More will be said about this contrast in Sectiori 5.4. Our first goal is the construction of the Haar basis on the interval [0, 1). In the colirse of this presentation, we will introduce many of the concepts required for the uriderstanding of rriultiresolution analysis and for the construction of general wavelet bases.

5.1 Dyadic Step Functions 5.11

The Dyadic Intervals

Definition 5.1. For. each pair of integers j , k E Z7 define the interual

I,,k

by

The collection of all such intervals i s called the collection of dyadic subirltervals of R.

The dyadic intervals have the following useful property.

Lemma 5.2. either

G i v e n j o , ko, j ~k.1, E Z, with, either jo # jl or. k o

# kl, then

116

Chapter 5 . The Haar Systerri

In the latter two cases, the smaller interval i s contained in either the right half o r left half of the larger.

•

Proof: Exercise 5.6.

Definition 5.3. G i v e n a dyadic interval at scale j , I,,,, we write I,,, = I , : . ~U I;,,, where I;,, and are dyadic intervals at scale j + 1, to denote the left half and rzght half of the interval I,,k.. I n fact, I:,, = I J f 1 , 2 k and I;,, = (Exercise 5.13.

5.1.2

1,+1,2k+l

T h e Scale j Dpadzc Step Functions

Definition 5.4. A dyadic step function i s a step ,fi~n,ctionf ( r c ) ulith the property that for som,e j E Z , f ( r )i s constar~to n all dyadic inter?~als1 3 , k , k E Z . W e say in this case that f (x) is a scale j dyadic step fiinction. For a n y interval I , a dyadic step filnctiorl on I i s a dyadic step function that is supported o n I . See Fzgure 5.1. Remark 5.5. (a) khr each j E Z, the collectioil of all scale j dyadic step functiorls is a linear space. That is, any linear conibination of scale j clyadic step functions is also a scale j dyadic stcp f~xnctiori.

(h) For each j E Z arid interval I, the ~ollect~ion of all scale j dyadic step filrlctiorls on I is a linear space. That is, any lirlea,r co1nbina.tion of sci~lcj dyadic step funct,ions on I is also a. scalc j dyadic step fullction on I. (c) If f (z) is a scale j dyadic step function on an interval I, then f (:I;.) is also a scade j' dyadic step furiction on I for ariy j' 2 j .

Exercises Exercise 5.6.

Prove Lerrlnla 5.2.

Exercise 5.7.

Prove that for each j,k E Z ;

~ i % ~ and I;\i = 13+1,2k

Ij+1.2k+1.

Exercise 5.8. Prove that any function f (z) of the form

=

5.2. Tlle Haar Systerrl

117

FIGURE 5.1. Scale j dyadic step fur~ctions.Left,: j = 2. Right: ,j = 4.

car1 he written i11 the form

5.2 The Haar System

Definition 5.9.

Lct p(.r') = X p . l ) (n.), an,$ for.

,TLC/L

j. k E Z, dc,firj,e

pl,k,(.r)= 2 1 ' 2 p ( 2 J . r- X.) = D 2 , 7 ; ~ ) ( . ~ ) .

(5.2)

(For the dej?n,ztion o,f thc d,ll(~,tionoper,ntor D,, c ~ n dthe t ~ a n s l a t i o roperator ~ 7b, see D ~ ~ f i n i t i o3.39.) r~ The collection { p , l , ~ ( ~ ) }i s, r.ef'erre(t l . ~ ~ ~ t o as t h e .system of Haar scaling hliirtions. For ea,ch ,j E Z . th,c rsollection { p , . k ( . r ) } ~ i s ~r.efr.r-r.rjd ~ t o a s the s!lsfrrn of' scale j Haar scaling fil~lctioris.See figure 5.4

Remark 5.10.

(a) For eadl j. k E Z. p,.k(.r.) = 2,1/' t I , , , ( r ) so . that, supported or1 the interval IJ,kant1 tlocs rlot, v a l ~ i s l on i that ilit,ei.vi~l. Therefore, u7e refer to Dhe scaling fullction o , , , k . ( . I : ) as being ussociatcd with,

p , ] . k ( z ) is

the inte,rvc~,lIj,h..

(b) For each j , k E Z,

and

Definition 5.11. Let h , ( z )

=

X 1 0 , 1 / 2 j ( x )- X [ 1 / 2 , 1 ) ( ~ )and , for each j , k E Z,

d efiiz e

h j , k ( z ) = 2J'2 h(2"

-

k) = D2,,Tkh(z).

(5.3)

Th,e collection, { h , j , k ( x ) } , , k c ~ i s referred t o as the Haar system on R. For each j E Z, the collection { h l , k ( x ) } k E is ~r e f e ~ e dt o as the s y s t e m of scale j Haar

functions.

Remark 5.12. (a) For each j , k E Z, hj,/c(x)= 2"'

( X p. I . i( )

-

I

.I

,

(

)

' 2

2'

XI,,,,.^ (x)

-

Y I ~ + ,,,,+,(r)),

so that h,,k(x) is supported on the interval I j , k and does not vanish on that interval. Therefore, wc refer t o tllc Hnur furlctiorl hj,Jlc;.) as being ussociated with the interval I J , k .

(Is) For each j , k E Z, F L , ~ , ~ ( x )is a scale j (c) For each j , k E Z,

+ 1 dyadic step function.

and

5.2.2 Orthogonality of the Haar System Theorem 5.13.

T h e Haar s y s t e m o n R is a n orthonorrnal systern o n R.

Proof: First, we show orthonormality within a given scale. Let j E Z be fixed, and suppose that k , kt E Z are given. By Lemma 5.2,

5.2. The Haar System

119

If k # k', then the product hj,k(x) hj,,/(x) = 0 for all x since the functions are supported on disjoint intervals. Hence, if k # k',

If k = k', then

Nexl, we show orthoriormality between scales. Suppose that j, j' E Z with j # j', say j > j', and let k , k' E Z. Then by Lelrinia 5.2, there are three possibilities.

(1)Ij/,k/ f' Ij,k= 8. In this case, h,,k(x)hj/,k/(x) = 0 for all z and

I;, ,,.

(x)is identically 1 on I:,,,,.Since Ij.*C In this case, I:,,,, it is also identically 1 on I j , , Sirrce hi,,( s )is supported on I,>r,

(2)

C

(3) I j , k c I;, Thus,

k,.

111this case, hj,,kt(x) is identically -1 on Ij;,,, arid on

IJ,k.

Theorem 5.14.

Given any j E Z , the collection of scale j Haar scaling functions is a n orthonormal system o n R.

Proof: Exercise 5.19. Although it is by no means true that the collectiori of all Haar scaling functions is an orthonormal system on R, the following theorem holds.

Theorem 5.15.

Given J E Z , the following hold.

( a ) T h e collection {p.~,k(x), h,,,k(x):j > .I, k E Z ) is a n orthonormal system o n R. (h) T h e collection {P.I,k(x),h.~,k(x): k E Z) is a n orthonormal system o n R.

Proof: Exercise 5.20.

120

5.2.3

Chapter 5. The Haar System

The Splitting Lemma

Lemma 5.16. (The Splitting Lemma) Let j E Z , and let g,(x) be a scale j dyadic step function. T h e n g, (x) can be written as g, (x)= r,-I (x) g,-I ( x ) ,

+

where r, - 1 (x) has the form

for some coeficients { a , -1 ( k ) ) k E z ,and g, -1 (x) i s a scale j - 1 dyadic step function.

Proof: Since gj(x) is a scale j dyadic step function, it is constant on the intervals I j , k . Assume that g j (x) has the value cj (k) on the i~lt~erval Ij,k. For each interval I j W l , k , define the scale j - 1 step function gj-l(z) on I j - l , k by

In other words, on I j - l , k , g3- 1 (x) takes the average of the values of y j (z) on the left and right halves of Ij-l,k(see Figure 5.2(a)). Let r,-l (z)= gj (z) - gjPl(x). By Remark 5.5(a), gj-l (x) is a scale j dyadic step function, and by Remark 5.5(c), so is T ~ - ~ ( xFixing ). a dyadic interval Ij-l,l., recall that lIj-l,k= 2-(i-'). hen'

1 2-(j-l) (cj(2k)

2

=

+ cj (2k + I ) )

0.

Therefore, on Ij-l,k, r j p l ( x ) must be a rnultiple of the Haar function hj-l,k(x) and must have the form (5.4) (see Figure 5.2(c)).

Exercises

5.2. The Haar System

121

FIGURE 5.2. Illustration of one step in the Splitting Lemma. Top left: Solid: Scale 4 dyadic step function y4(z). Dashed: The scale 3 dyadic step function g3(x) constructed as in the Lemma. Top right: Graph of g3(x). Bottom: Graph of the residual rs(x).

Exercise 5.17.

Prove the statements rrlade in Remarks 5.10(a) and 5.12(a)

Exercise 5.18. Prove that po.o(x) = 2-112 P,.,(x)

+ 2-'12

ho,o(") = 2-'12 p 1 , 0 ( x )

-

p1)L,1(2)

and

Exercise 5.19. Prove Theorem 5.14. Exercise 5.20.

Prove Theorem 5.15.

2-112

p1.1 (x).

122

Chapter 5. The Haar System

5.3 Haar Bases on [ O , 1 ] Definition 5.21. For any integer J 2 0, the scale J Haar system on [ O , 1 ] is the collection

W h e n J = 0, this collection will be referred to simply as the Haar system on [0, 11. See Figure 5.3.

Remark 5.22. (a) The Haar system on [O,1] consists of precisely those Haar functiorrs h j , s ( x )corresponding to dyadic intervals Ij,k that are subsets of [0,I], together with the single scaling function po,0(x).

(b) For J > 0, the scale J Haar systern on [0, I] consists of precisely those Haar functions h j , k ( ~corresponding ) to dyadic intervals Ij.k for which j 2 J and that are subsets of [0, I], together with those scale J Haar scaling functions that are supported in [O,l].

Lemma 5.23. Given f(x) continuous o n

[U, 11, and

t

> 0, there is a J

E Z,

and a scale J dyadic step function y (x) supported in [O,1]such that If (x)-y (x)1 < for all x E [0, I]; that is, 11 f - g J ( , < E .

E,

Proof: Exercise 5.26. See also figure 5.5. Theorem 5.24. For each integer J > 0, the scale J Haar system o n [0,I] is a complete orthon,omn,al system o n [0,11. Proof: That the scale J Haar system on [0, I] is an orthonormal system on [0, I] follows from the fact that it is a subset of the collectio~l { p ~ , ~ ( hj,k(z): x), j J , k E Z), which is an orthonormal system on R by Theorern 5.15(a). For completeness, it is sufficient, by Theorern 2.57(c), to show that if f (x) is Ci' on [ O , l ] , then

>

Let E > 0, and let f (x) be C0 on [0, I]. By Lemma 5.23 there exists j and a scale j dyadic step function on [0, I], gj(x) such that

> 0,

Since any scale j dya,dic st,ep function is also a scale j dyadic step function at all higher scales, we can assume that j 2 J.

5.3. Haar Bases on [O, 11

123

FIGURE 5.3. Some uf the Haar functions h j S k ( 5 on ) [O, 1).

By the Splitting Lenima (Lemma, 5.16), g,i(z) m a y be written g j (x) = rJ- 1(x) gj- 1(x), where rj- (x) 1-ias the form (5.4), and is suppurled in [O, 11 and gj-1 (x) is a scale j-1 dyadic step function. Repeating this process j - J times, we conclude that

+

where each r e ( x )can be written

for some constants a p ( k ) and where g~(x)is a scale .7 dyadic step function (Figure 5.6). But this just means that g J ( z ) is a finite linear combination of the collection {pJ , (~s)}:Lil. Thus gi ( z ) is in the span of the scale J Haar system on [ O , l ] and I f - gj1I2 < t.

0

124

Chapter 5. The Haar System

FIGURE 5.4. Some of the Haar scaling functions p , , k ( x ) on [0,1).

Example 5.25. (a) Let f (.c) = X[0.3/4)( 2 ) . Taking J = 0, we see that = 314, and that ( f ,11,,7.k)= O wheiiever Ij,kC [O: 314) or I j , k 5 (f, [3/4,1). This is true for every j > 2 arld all 0 5 k: 5 2J - 1. Thus, the only ilorizero Haar coefficieiits are (f , ho.") = 114 arid ( f ,h l T l )= 2 - v 2 . Notice that the P ( ~ , ~ ( Xterrn ) is simply the average value of the function on 10, I ) , and that the only nonzero Haar coefficients correspond to the Haar functions that "strad(lleVthe discontinuity of f (z). (lr). Again, assuine that J = 0. Then (.f, po,o) = (b) Let f ( z ) = X p l 11/16, which is the average value of f (x) on [0, 11, and ( f ,hj,k) = 0 whenever IjSk C: [O, 11/16) or I j , k C: [11/16,1). This is true for every j 2 4 and all 0 k 5 2j - 1. The only nonzero Haar coefficients are ( f ,ho.o)= 5 . T4, ( f ,h l , l ) = 3 . r 7 i 2 ,(f,h2,2)= T3,and ( f ,h3,5)= 2-5/2. See Figure 5.7.

<

(c) Let f (x) = X [ 0 , 2 / 3 ) ( ~and ) , assume that J = 0. Since 2/3 is not a dyadic rational number (that is, one whose denominator is a power of 2), there will be nonzero Haar coefficierlts for this furlctiorl a t all scales. However, note that ( f ,t ~ j , = ~ )0 if 213 6 I,,k so that at each scale there is exactly one nonzero Haar coefficient. We list the absolute values of several of these coefficients below. Note that as the scale increases, the size of the coefficients

5 . 3 . Haar Bases on [ O , 1 ]

125

FIGURE 5.5. Illustration of Lemma 5.23. Approximation of a continuous function by scale 2, 3, and 4 dyadic step functions.

decreases, but as tlle third colurr~ilof the table indicates, the coefficients are exactly proportional t o 2-j/2. See Figure 5.8. (d) Let 0 x E [0,1/3), 6x - 2 z E [1/3,1/2), .f (.c) = - 6 : ~+ 4 J E [1/2,2/3), 0 x E [2/3, 1).

This function is zero on 10,113) U [2/3, I ) , rises to 1 linearly on [1/3,1/2) and falls to zcro again on 1112,213). Hence, f (x) is C0 on [O,1)with discontinuities in its first derivative at x = 113, 112. a~ltl213. Note tllal nonzero Haar coefficie~ltsare possit~leori1-y when I j , kn [1/3,2/3) # 8.This nlearls that at each scale, about 213 of the Haar coefficients will be zero (See Figure 5.10 and cf. Section 5.4.1). In the previous example, we saw that near the jump discontinuity, the Haar coefficients were proportional t o 2-jl2. Consider the behavior of the Hamarcoefficient,^ near the corner at x = 213. Some values of the Haar coefficients for which 213 E Ij.k are listed in the table in Figure 5.9. Note that the Haar coefficients are approximately proportional to 2-3j/2. Now consider the behavior of those Haar coefficients for which 17,kC [1/2,2/3). It is easy to calculate that each of these Haar coefficients is

126

Chapter 5. The Haar System

FIGURE 5.6. Full decomposition of the function y4(x) of Figure 5.2 using the Splitting Lemma. Down the left column are the functions g:<(x),. q ~ ( x ) , (x),and go(x). Down the right column are the functions 7-3 (.c), T Z (z), r 1 (x),and T O (x).

eqllal to (3/2) 2 - " . 1 / ~ Thus, . the Haar coefficients bchave like 2-"12.

Exercises Exercise 5.26. Prove Lernma 5.23. (Hint: Use the fact that since f (x) is continuous on [O. I], it must be unzform,Zy continuous on [0,I].)

5.4. Comparison of Haar with Fourier

127

FIGURE 5.7. Example 5.25(b).

Exercise 5.27. Prove that every scale j dyadic step Pr~nctionis also a dyadic step functiori at all scales larger than j , and that any linear combination of scale j dyadic step function s is also a scale j dyadic step function. Exercise 5.28. Calculate (numerically or analytically) the Haar coefficients up to scale j = 8 for the function f ( t )= 8 t 2 ( 1 - t ) on [O, 1).Explain the behavior of the coefficients as j increases.

5.4

Comparison of Haar Series with Fourier Series

Note that by Remark 5.12, each Haar function h3,k(x)vanishes outside the interval I j , k .The length of the interval Ij,kis 2-,I SO that if j is large, then the length of Ij,k is small. We say therefore, that the function h,,k(x) is well localized in time (or depending on the context, well localized in space). This property is to be contrasted with the trigonometric basis {e2"i7Lx}nEZ. Note that each element of the trigonometric basis has absolute value 1 for every z E [0, I ) , and so it never vanishes for any x.

128

Chapter 5. T h e Haar System

FIGURE 5.8. Example 5.25(c).

FIGURE 5.9. Example 5.25(d).

5.4.1 Representation of Functions with Small Support One consequence of the good time localization of the Haar basis is that a function f (z) that vanishes outside a small subinterval ( a ,b) of [0, 1 ) will have rrlosL of its Haar coefficients identically zero, since the Haar coefficient ( f ,h j , k )will be zero if does not intersect ( a , b). Note that even if a function f (x) was supported inside a small subinterval of [0,I ) , that most of its Fourier coefficients would be nonzero. Of course, in both cases, we are dealing with infinite series with infinitely many coefficients; so we must be precise about what we mean by "most" or "manynof either the Fourier or the Haar coefficients of a given function. In the case of the Haar series, let us fix an integer j 0 and note that there are 2j functions h j , k ( x )in the Haar system on [0, I). Given a function f (z) supported in an interval ( a ,b), then for this j , the Haar coefficients

>

130

Chapter 5. The Haar Systcm

so that

Therefore, we conclude that t l ~ efraction of possibly rlorlzero Haar cocfficierlts for a fulictio~lvstrlisliing outside an interval is approximately proportional t o the lengtli of that interval (see also Figure 5.10).

5.4.2 Behavior 0.f Huur Coeficients Ncur Jump Disco~~tinuilies Suppose that f (z) is a furictiorl defined on [ O , l ] , with a j u ~ i ~discontinuity p a t 2 0 E ( 0 , l ) arid contiriuous a t all other points in [0, I]. The fact that the Haar functions hj,k(x) have good localization iii tirrie leads us t o ask tlic question: Do the H a a ~coc:ficient.s (f,I L ~ . ~such ) that x0 E 13,k behave differer~tlythan do the H(l,ar. coefjicients S ~ L C that ~ L ~r:o $ IJ,k? In ptirticular, can wc firltl the location of a jump discontin~~ity just by examining tlle ;~l)solutevaluc of tlle Haar coefficients? Wc will sce that in fact wc can do tliis. For sirriplicity, let 11s asswnc that the givcn functio~if ( J ; ) is C2 or1 the iritervals [O, xu] :-tnd [.xu, I]. Tliis nlcarls that both f t ( x ) and f t t ( x ) exist. are co~it,iriuousfunctions, arid lierice are l)ou~idcdon each of these intervals. Fix i~ltegcrsj 2 0 a~itlO 5 k 5 2 5 - 1, anc1 let .c5,k 1)c the rriidpoirit of the interval IJ.k.;that is, .r,?k. - 2 - j ( k : 1/21. Tliere ;ire ~ i o wtwo possibilities; cithtr :x:o E I j . k .or 20 $!

+

Casel: .rao @ I,,,.. If .r.o 6 I j , k , then expancling f (:r) a1)out .x.,,,.by Taylor's fo~~iliulit, it follows tllttt for all .r: E

wliore

Ej,,.

i s s o ~ n epoint, i r ~rJ,k. Now, using thc k c t that

h j q k ( xd)s

-

0,

5.4. Comparison of Haar with Fourier

131

If j is large, then 2-5f12 will be very small compared with 2-"12; so we conclude that for large j ,

Tf xo E I,,*, t,hen either it is in I:,, or it is in I:,. Let us Case 2: r o E assume that xo E I:,,. The other case is similar (Exercise 5.30). Expanding f (x)in a Taylor series about $0, we have

Therefore,

where

Thus,

If j is large, then T 3 j I 2will be very small compared with 2 - j I 2 ; so we conclude that for large j ,

132

Chapter 5. The Haar System

The quantity Izo - 2TJk can in principle be small if s o is close t o the left endpoint of I ; , , arid can even be zero. However, we can expect that in rnost cases, xo will bc iii the middle of so that s o- 2-Jkl F; (114) 2-J. Thus, for large j. 1 ( f , )= f ( - ) - f ( ~ o + ) 2 - . 7 / ~ . (5.8)

11~

1 for 1a.rgej is Cornparing (5.8) with (5.6), we see that the decay of I ( f ,11 considerably slower if zo E Ij.kthan if :co @ IJ,k. That is, large coefficients in the Haar expansion of a fiinction f (.c) that persist for all scales suggest the preserice of a jurnp tliscoritiriliity ill the intervals IJ,k corrt:sporiclirig to the largc coefficients. 5.4.3 H a w Coeficients and Global Smoothness We know that the global silioothrless of a function f (.c) defined on [O. I] is reflected in the decay of its Fourier coefficients. Specifically, if f (z) is periodic and CK 011 R. thcn therc exists a constant A depending on f (z) such that for all Ir E Z, I(., 1 A ~ T I J - ~where . c,, are the Fourier coefficients of f (z) (Excrcisc 3.38). This can be regarded as a stntcrrlcnt about the frequency contcrlt of srnootli fi~nctions~ rlanlcly that snloother filnctions tend to have smaller high-frequency corrlponerlt,~than do filnctiorls that are not smootli. However. no such cstirnat,~holds for tlie Haar series. To see tliis. sirrlply note that tlie f~~nctiori f (n.) = o'"' has period 1 and is Cw on R with all of its dcrivatiw:~1)oilnded by 1 (Show tliis). But by Exercise 5 . 3 2 ,

<

-

ant1 since siri(:c)/~:= 1 a,t .c = 0 and sirice sin((1/4) 2-.I) % (114) 2-J for large j , this means that I(f. I I , . ~ ) (114) 2-"/' for large j . But this is the same rate of decay ol~servedfor furictions continuous lout with a tliscoritiriuous first derivative (Exaniplc 5.25(tl)). Herice, g1ol)al snloothness of a furlctiorl docs riot affect the rate of decay of its Haar coefficierits.

Exercises Exercise 5.29.

Show that i f f (.c) is C" on I , , k , then the Haar coefficients

satisfy 1

I( f . hJ,k)I = - q f f ( ~ J . ~ ) 2 - 3 J ' + 2 pJ,t(.~). where IrJ.r(x)l

1 <max 1 f " ' ( ~ ) 1 2 - ~ ~ / ~ 768 Z E I , . ~

5.5. Haar Bases on R

Exercise 5.30.

I(f,

133

Show that if xo E IT,k, then

hj,k)I

23/2 ) 2 ' ( k

+ 1) - x o ~ ~ ~ ( x o--f)( x o + ) -

Exercise 5.31. Let a l denote the first positive local rrlaximlirri of' the function f (x)= sin2(x)/x. Show that a* = 1.16556.. .. Exercise 5.32.

For integers j

4, (k) =

> 0 and 0 < k < 2J

1,

-

1, let

h,j,k(z) e-2"77'"' ctz.

(a) Sliow that

where

~ : j , k :=

2-.j(k

+ 112) is the rrlidpoirit of I,,c

(b) Show that ( a i ( k ) (is maximized when In( = (314) 2.j, and that the first positive zero of la{l( k )1 occln-s when n = 2J+' (see Exercise 5.31). Hence it is reasonable t o say that ! L ~ , ~ ( Xis) localized at the 2J f r e q ~ ~ e n cies satisfying 2.ipl 5 In(<_ 2.1.

Exercise 5.33. Using MATLAB or sonie other software. explore nurrierically the statement that h j,k ( 2 ) is approximately localized in frequency at the frequencies satsifying 2)-I 5 n l 23. Specifically, with f (z)= cos(2nm,x), for some fixed positive integer m , compute the quantities

<

for various j . Show numerically that Aj is largest when 21-I 5 n-n 5 2,. Do the same when f (x) is some finite linear cornbirlation of cosines at various frequencies.

5.5 Haar Bases on R In this section, we wish to define a system of Haar and scaling functions that form a complete orthonormal system on R.

134

5.5.1

Chapter 5. The Haar System

The Approximution and Detail Operators

Definition 5.34. For each j t Z , define the approximation operator P, on functions f ( z ) , L' o n 11, by

Remark 5.35.

(a) For each j E Z, define the appr.oximation space

1/, by

Since { p , % k(z):k E Z ) is an orthonormal system on R, Lemma 2.51 implies that Pjf (x) is the function in 5 best approximating f (x) in the L2 sense. (h) Since pj,k (x) = 2.i/2 X.lJ,a (x),

In other words, on the interval

Ij,k,

Pjf (cc) is the average value of f (z)on

For this reason, we think of the fuilction Pjf (x) as containing the features of f (x) a t r.esoL7~tionor scale 2 - J . In other words, in passing fro111 f (z)to Pjf (z),the behavior of f (x) on the interval I j , k is reduced to a single nurrlber, the average value of f (x) on Ij,k. Therefore, any data about the small scale variation of f (x) on I , , k is lost. In this sense, P , J ( L ) can be thought of as a blurred ~ ~ e r s i oof'f n (x) at scale 2-J; that is, the details in f (z)of size smaller than 2-J are invisible in the approxi~nationPjf (z), but features of size larger tllarl 2-j are still discernible in Pjf (z). We car1 prove the following facts about the operators Pj

Lemma 5.36. ( a ) For each j E Z , P, is linear; that is, given f (x) and g ( x ) , L~ o n R, and a , P E C, P J ( 0 f+ P 9 ) ( x ) = a P , f ( x ) +pP,g(x). ( b ) For each j E Z , PJ is idempotent; that is, given f (z), L2 o n R,

(c) Given integers j , j' with j

5 j', and g ( x ) E V,,

5.5. Haar Bases on R

(d) Given j

E

Z , and

f(z),

135

L2 on R,

Proof:The proofs of parts (a) and (b) are left as cxcrcises (Exercise 5.46). Part (c) follows frorn (b) and the fact that if g(x) E V,,then P,g(x) = g(x). We will prove (d) in detail. Since {pj,k(x))ktzis an orthonormal system on R, by Exercise 2.60,

By the Caucly-Schwarz inequality,

Thus.

Lemma 5.37. Given f (x)c': on R, (a) lim IIP3f - f l l z = 0 , and J'c=

(h) lim IIP3f112 = 0. 3'00

Proof: Suppose that f (x)is support,ed in an interval of the form [-2N, 2N] for some integer N. By Excrcisc 5.26(b), there is an integer J, and a function g(x) E VJ such that

If j 2 J, then by Lemma 5.36(c), P,g(x) = g(x) and by hlinkowski's inequality and Lemma 5.36(d),

136

Chapter 5. The Haar System

Since

E

Ir/ -

f ll2 < 5.

Combining this with (5.10) proves (a). The proof of part (b) is left as an exercise (Exercise 5.48).

Definition 5.38.

For each j E Z, define the detail operator Q, o n functions

f ( x ) , L2 o n R , by

& ~( f4 = P ~ + l(2) f - Pjf ( 5 ) . Remark 5.39.

(5.11)

(a) For each j E Z, we define the wavelet space W j b y

Since {hj,k(x))kEZis an orthonormal system on R and in light of (5.12), Lemma 2.51 implies that Q j f (x) is the function in W j best approximating f (x) in the L2 sense.

(b) In light of the interpretation of Pj f (x) as the blurred version of f (x) at scale 2-j, we can interpret Q, f (x) as containing those features of f (x) that are of size smaller than 2-1 but larger than 2 - p l . That is, Qj f (x) has those details invisible to the approximation Pj f (x) but visible to the approximation f (x).

Lemma 5.40. ( a ) For each j E Z, Q, is linear; that is, given f ( x ) and g(z), L' o n R , and a , p E C, Q , ( a f + P g ) ( z ) = a Q , f ( x ) +PQ,g(x). ( b ) For each 3 E Z, Q, i s idempotent; that is, given f (x),~ \ n R,

(c) If g(x) E W, and if j' is a n integer with j'

( d ) Given j E Z , and f (x), L'

o n R,

# j,

then

5.5. Haar Bases on R

137

Proof: Exercise 5.47.

Lemma 5.41. Gioen j E Z, and a function f (x)

on

R,

where the sum i s finite.

Proof: To prove this, let j t Z be given and let f (x) be CE on R. Consider QJf (x) for x E Ij.k. Note that

and that

if x E I,,*. For x t I!,,,

and on I:,k,

Since

2j/' -2il2 0

if x E I:,, if x E otherwise,

Q,f (x) = (f;h ~ , khj,k(x) ) and (5.12) follows.

138

Chapter 5. The Haar System

5.5.2 The Scale J Huur Sv.stern on R Definition 5.42.

Let J E Z be given. T h e collection

is called the scale J Haar system on R .

Theorem 5.43.

The scale J Haar system o n R is a complete orthonormal

system o n R .

Proof: Note first that by Theorem 5.15(a), the scale J Haar system on R is orthorlormal on R . To prove compl~t~eness, by Theorem 2.57(c), it is sufficient to show that given f (x), C: on R,

To this end, let E > 0 and let f (x) be C: on R. By Lemrna 5.37(a), there < E. We can take N > J . is an integer N such that J J PfN- f BY (5.11),

Therefore,

where the sum over k is necessarily finite because f (x) is co~npactlysupported. Since

and ( ( P N f -f

11 < E , completeness is proved.

5.5.3 The Huur System on R The Ho,ar system o n R (Definition 5.11) is a complete orthonormal system o n R.

Theorem 5.44.

5.5. Haar Bases on R

139

Proof: By Theorem 5.13, the Haar system on R is orthonornial on R. To prove completeness, by Theorem 2.57(c), it is sufficient t o show that given f (x), C:! on R,

To this elid, let

F

> 0 and let f (x) be C: on R. For ally J E N , by (5.11),

By Lerrlma 5.37(a) and (b), there exists J t N so large that

Therefore, by hlinkowski's inequality,

Also,

wlierc thc slirri over k is iiecessaril-y finite hecallst. f (:r) is conlpactly supported. Since J- I

complctcness is proved.

Exercises Exercise 5.45. Expose the fallacy in the followirlg argument,: Let f (x) be L2 on R. By Theorem 5.44, we may write

140

Chapter 5. The Haar System

Integra,ting both sides, we obtain

f (x) dx = 0. (Hint: Tlie Therefore, every function L~ on R must satisfy fallacy has notliing to do with f (x.). That is, we may assume that f ( L ) is integrable, or infinitely differentiable, or compactly supported and it will not change the argument as written.) Exercise 5.46. Prove Lemma 5.36(a) and (b). Exercise 5.47. Prove Lcmma 5.40. Exercise 5.48. Prove Lemrrla 5.37(b). (Hint: Use Minkowski's inequality ti11tl the fact that f (z) is conlpactly supported.)

Chapter 6

The Discrete Haar Transform 6.1 Motivation Rcctall that a fl~nctiorlf ( x ) ctcfinecl or1 [O. 11 hxs tar1 cxpunsiot~ill terms of Hawl. fiulc-lio~lsa:, follo~~rs. Gi\rc)il ally integcr J 0.

>

iii L2 on [O. 11. 111 ordcr to ~llc)t,iviit,(> i \ (liscrot(>versioli of tliis (~xpi~l~sioli, tllc' D ~ ; s c Y . ~ ~ c Hwar 'Tr7rar1,.s,for.7rl( D H T ) . wc iissllnle tll;it, wc l i i \ v ~only ~ :L finite. tliscret,c t~pproxiniatiwt,o f (.c). 111this ct~sc.tllo liiost natlual sl~cllapproxirllatioll is hY the tlgaclic. step fllrlctioil PAv f (.r). w11el.c N E N and N > J . Tliat is. gia-cli f ( . I . ) . 2r-1

f

(J.)

= p ~ f ( . ~=- C ) ( f . P N . ~ o) ~ . i ( x ) -

(6.2)

k=O

Tl~us.t,lic Haar- cocfficit.tlts of f ( J . ) call l;)e i~ppl.oxirriatcdby t,llr>Hi~ill.coefficicrits of PNf(.x.).That is.

Notc that,

(1) by Tlicorcirl 5.15(a), if j 2 N. tlle~i( f . arid

=

O for 0


5 2J - 1.

(2) sincc liiilN,, PN f ( . r ) = f(.r) in L~ 011 R (Leinrna 5.37(a)). tlie Cauclly-Schwarz ineqlxttlity irriplies tliat linl (PNf'. 11j . k ) = { f .1 1 ~ . k ) a.nd

N-+x

lit11 ( P Nf .

IV- > r n

pJ.k) =

(f.l ) J . k )

(6.4)

(Exercise 6.5). Supposc that we are given ix finite sequence of data of length 2N for some N E N. {cO( I C ) 2\} ~ -1, ~ . Wc assume that for some underlying function f (.c). cao(k)= ( f . ~ ~ Fix , ~ ,I) E. N. tJ < N. and for each 1 < j < tJ, define

142

Chapter 6. The Discrete Haar Transform

It turris out that there is a convenient recursive algorithm that can be used k ) d , ? ( k )from ~ j - ~ ( k This ) . algorithm t o compute the coefficierit,~~ , ~ (and uses the fact, proved in Exercise 5.18, that for each Q, k E Z, 1

(XI

+

P P , ~ = - (~e+i,zlc (x) p~+1.21;+1 (x)),

J'

(k)

Jz

=

( f - ~ ~ - , , . k )

-

1 -

h

1 (fl(r)~-j+l.2k) + - ( f .~

JZ

~ - j + l , 2 k + ~ )

arid by (6.6),

By writing (6.7) arid (6.8) in rriatrix form, it is easy t o see that the calculatiorl is corripletcly reversible. In fact,

6.1.1

The Discrete Haar Transform ( D H T )

Therefore, wc rrlake the followirlg definition.

Definition 6.1. Given J , N E N with .I < N and a finite sequence co = { c g ( k ) } : Z , ' , the ( D H T ) of co is defined by

where

6.1. Motivation

143

The inverse DHT is given by the formula

As with the DFT, thc DHT can be thought of as a linear transformation on a finite-dimensional space and as such can be written as multiplication by a matrix.

Definition 6.2.

Given L

N even, define the ( L / 2 ) x L matrices H L and

E

G L by 1 1 0 0

0 1

...

1

0

...

0

1

HL = 0

...

1

Define the L x L matrix Wr, by

Il'he matrix H L i s referred to as the approximation matrix, the matrix GL as the detail matrix, and the matrix WI, as the wavelet ma,trix.

From here on, we will suppress the subscript L in the matrices H L , G L , and WL in order to clarify the presentation. The value of L will either be clear from context or will be indica,t,ed sepa.ra.t,ely.It is easy to verify that for each L, W is an orthogonal matrix (Exercise 6.6). Hence its adjoint is its inverse; that is,

=

(H* G * ) ( E )

where I is the L x L identity matrix.

144

Chapter 6. T h e Discrete Haar Transform

If we consider our initial sequence of data, co, to be a vector of length L = 2 N , then the DHT algorithm reduces t o rr~atrixmultiplication. Specifically, let co = ( ~ " ( 0 )c o ( l ) - . . ~ ~- 1)): ( and 2 for~1 < j < J, let

and

dj = (d,,(0) dJ (1) . - - d , (2N--'

-

1)).

T h e n t , h e DHT of co is giver1 by

where H and G arc 2N-"2 by

x 2 N - , ~ - 1matrices. Thc inverse DHT is give11

Tlie above car1 be sunlrriarized as follows. Theorem 6.3.

Given .I, N E N with .I


and

(L

vector

c,, = (c(,(O) c.o(l) . . . c(lj2N - 1))

of lenqth 2", the D H T of co zs the liecto~.

where

where H and G are 2N-"2 x 2 N - ~ - matrices as in Definition 6.2, and 1 5 j 5 J . T h e inverse D H T i s given by

Thcorcrri 6.3 is illust,ratcd in the tree diagram in Figure 6.1.

Theorem 6.4.

G i w n N E N , the D H T of a vector of length 2" can be computed with no more than 2N+' multiplications.

Proof: Let c o

=

(co(0) c o ( l ) . . . ~

~- 1)) ( be 2given.~ Computing

6.1. Motivation

145

FIGURE 6.1. Tree diagram for the DHT.

requires one nlultiplication for each 0

< k < 2"-l

-

1, and conlputing

d l (k) = (l/v/Z) (cn(2k) - co (2k + 1 ) ) requires one multiplication for each k for a total of 2" niultiplications. Computing cz and d2 requires 2 " ' multiplicatio~ls.It follows that the total number of rriultiplications is

Exercises Exercise 6.5. Show that (6.4) holds for any function continuous on [0, 11. Exercise 6.6. Prove that the matrix W N defined in Definition 6.2 is symmetric and orthogonal.

Exercise 6.7. Prove that if {ii,(k):l

<j

5 J ; 0 5 I; j 2"-J

-

N

l } ~ { c ~ ( k ) :
is the DHT of a sequence {c~(lc)};=,', then

<2

"

~ 1)~

-

146

Chapter 6. The Discrete Haar Transform

6.2 The DHT in Two Dimensions In many applications, especially image processing, the objects being analyzed are best thought of as matrices, rather than one-dimensiorial finite signals. That is, we are interested in L x &I matrices c of the form c = {c(n,m): 0 5 n 5 L - 1; 0 m - 1). The purpose of this section is to define a generalization of the DHT for matrices.

< <

6.2.1

The Row-wise und Colurrzn-wise A p p r o x i m a t i o n s an,d Details

The generalization of the DHT to two-dirrlensiunal sig~lalsinvolves the separate application of t h e ordinary DHT to the rows and columns of the signal. Given an even nun~berL E N, let H arid G be thc ( L / 2 ) x L matrices dcfiricd in Definition 6.2. Let c be an hl x L matrix of the form

In this notation, ce is the 4th row of c. We define the row--wise approximation rnatriz of c, Hrowc, t o be the M x ( L / 2 ) matrix defined by

We define the row--wise detail matrix of c, GroWc, to be the M x ( L / 2 ) matrix defined bv

In fact.

Clearly, HroWc is the matrix obtained by multiplying ea,ch row of c by the matrix H, and GroWc is the matrix obtained by multiplying each row of c by the matrix G. See Figure 6.3 (left).

6.2. T h e D H T in Two Dimentions

147

Given L F N even, let c be an L x Af matrix of the form

where ce is the f t h column of c. We define the column-wise approximation matrix q f c , Hcolc, t o be the (L/2) x M matrix defined by

In fact, (Hcolc)(n, m) =

1 --

fi

1

c(2n, m ) + - c(2n + 1,m).

JZ

(6.12)

We define the column-wise detail matrix ofc, GroWc, t o be the (L/2) x A1 matrix defined by

In fact. ( c c o l c ) ( n ,m ) =

1 -

Jz

c(2n1rn) -

1 -

Jz

c(2n + 1.m).

(6.14)

Clearly, Hcolc is the matrix obtained by multiplying each colrlrnn of r: by the matrix H ; and Gcolc is the niatrix obtained by rrnrlitplying each column of c by the inatrix G. See Figure 6.3 (right).

6.2.2

The D H T for Matrices

We are now ready t o define the DHT for rnatrices. For sirrlplicity we will assurrie (1) that the matrices we analyze are square, and (2) that the nurriber of rows and columris of these matrices are powers of two, that is, that c is always a 2N x 2 N rnatrix for some N E N. p -1

Definition 6.8. Given J , N E N with J < N and a matrix co = { c ( n ,m ) },,,,,,=,,. For 1 5 j 5 J , define the 2N-' x 2N-.1 matrices c, d;", dl'), and d") .I by

where H C O ~Gcol, , HroW, and Grow are the 2N-3-2 x 2 N - 3 - 1 matrices defined b y (6.9)-(6.13). T h e D H T of co is the collection of rnatrices

148

Chapter 6. The Discrete Haar Transform

FIGURE 6.2. Original "magic square'' image.

Thtl i~ivcrscof t,lie DHT for rilatrices irivolvcs applying tllr atljoiiit of the iliatri~~ Hs iill(1 G row-wise i ~ ~ ~o1111111i-wis~. id Givoil L E N cvc.ri, lct II* wiicl G* 1 ) tllc ~ wtl~oiiit,~ of H ai1c1 G . Lct (. ail Af x ( L / 2 ) matrix of t,hc for111

We define tlic ro2li-wise npproz.irn,at~;onacijoint A1 x L ~riat~rix

r. ~ ' " ~ * cto. 11c

the

We define the ?.ow-wise detail adjoint o,f r , G " ~ ) ~ to * (be~ the , A 1 x L lnat,rix

6.2. The UHT in Two Dimentions

149

FIGURE 6 . 3 . Left: The row-wise approximation and detail matrices applied to image in Figure 6 . 2 . Right: The column-wise approximation and detail matrices applied to image in Figure 6 . 2 .

Hrow*c is the matrix obtained 11y multiplying eadl row of c: by the nia.trix H*, and GroW*c:is the rnatrix obtained by rrlultiplyilig each row of ca by the matrix G*. Given L E N even, let c be an ( L / 2 )x A4 rnatrix of the form

We define tlme colurr~n,l l l i s e detctil (tdjoi71,t of c. GroWc, t,o t ~ et1.w L x 41 matrix G ~ * r.~= l( G*co G*cl . . . G*c*Ai-l ) . (6.19)

the matrix obtained by nnrltiplying each rolu~rlnof i s by tJllr r~iatrixH*, and Gcol*r is the ~rlatrixobtairleii by nmultiplying car11 mlu~ilu of c by the matrix G*. Combining (6.9)-(6.19) we have that H ~ O ~ * is C

and that

150

Cha,pter 6. The Discrete Haar Transform

Since H*H

+ G*G = I.

Theorem 6.9.

~ ~ 0 G1 ,~

T h e inverse D H T for matrices i s given b y

H~ ~ Ol W

Proof: Fix j with 1 5 j

and

G ~ O W are 2 " - 3 - 2

X p - 3 - 1

i~aatricesgiven above.

< J. Then

Similarly,

G r ~ ~ * H ~ ~ l *+dGj r2~) ~ * G ~ ~ l *=d G j 3TOW ) * Grow Since

H ~ O W* H ~ O W cj-1

+ Grow * Grow

Cj-1

-

cj-1,

(6.23) follows.

Exercises Exercise 6.10. Prove that the matrices HroW and Hc0' commute. That is, prove that for any vector c , HroWHcol c = H"O' HroWc .

6.3 Image Analysis with the DHT A digital black-and-white image is stored in a computer as a matrix of numbers { c o ( n ,m)). Each location ( n ,m) in the matrix corresponcls to a picture element or pixel, and the value of c ( n , m) is a nonnegative integer indicating the gray scale value or pixel value a t that location. The pixel values range from 0 t o some maximum value hf.A value of 0 means that

6.3. Image Analysis with the DHT

151

FIGURE 6.4. DHT of "magic square" image with J = 1. The four quadrants contain (clockwise from upper left) el, dl , d(3), d ( 2 ).

,

the intensity at that location in the image is black, and a value of M means that the intensity at that location is white. Numbers in between 0 and &I represent various. shades of gray. If the matrix of pixels is displayed on a grid with each cell of the grid given the appropriate shade of gray, then the eye interprets the display as an image. In this section we will use the DHT for matrices to analyze images represented by matrices of gray scale values. We will give interpretations of the DHT matrices c j , d (1) j , d (2) j , and d (j 3 ) in terms of corresponding features of the image and will indicate the properties of the DHT for matrices tJhat make it effective for image compression.

6.3.1 Approlcirnation a n d Blurring In this subsection, our goal is to provide an interpretation for the approximation matrices cj produced by (6.15). Consider first the 2N-1 x 2N-1

152

Chapter 6. The Discrete Haar Transform

FIGURE 6.5. DHT of "magic square" image with J = 2. Uppcr lcft quadrant contains (clockwise from upper left) c2, d 2(1) , d ,( 3 ), .

approximation matrix e l . Since cl = H'$ 2N-1 for any fixed 0 5 n , m < I

H!$wcol we have by (6.12) that

6.3. Image Analysis with the DHT

153

In this calculation, the pixel values of the four pixels at locations ( 2 n . 2 m ) . (2n. 2rr1. + 1). ( 2 r ~+ 1.2m). and ( 2 r ~ 1 . 2 +~ 1~) are replaced by a single value c z ( n , m ) that is twice the average value of those four pixels. This mearis that ally variation in pixel values within that 2 x 2 block of yixels is lost. In other words, el represents only those features of the image that exist at scale 2 or larger. See Figure 6.4. Computilig the rnatrix c2 irlvolves taking twice the average of the four valucts ci ( 2 n , 2171,), c1( 2 n + 1 , 2 m ) , c1( 2 n , 2 m + 1 ) , and c1(2n 1,21n + 1). Since each of these numbers was cornputed as an average (times 2) of a 2 x 2 block of pixels, each elerrlent of c2 is an average (times 4) of a 4 x 4 block of pixels. Thus, the variatioil in pixel values within that 3 x 4 block of pixels is lost. That is, cz represents only tllosc fcatures of the image that exist at scale 4 or larger. In general, the matrix c j represents those features of the irna.ge that exist at scale 2J or larger. See Figure 6.5. In order to see this "blurring effect" more clearly. we take a 256 x 256 image a i d compute the 128 x 128 matrix c l . the 64 x 64 matrix c z , and the 32 x 32 matrix c:$ (Figure 6.6). We reconstruct a bhlrred version of the original irnagc froin these iiiatrices by itssurni~igthat tlie detail matrices d (j1 are zero for i. j = 1, 2, 3, and the11 applying the illverse DHT for nlat,rices. We see that the blurred images are very blocky arld are fairly unpleasant to look at. We will see in Chapter 9 that this effect can be mitigated by designing wavelet and scaling filters that correspoild to smoother scaling and wavelet functions.

+

+

6.3.2 Hor.ixonta1, Vertical, and Diagonal Edges Intuitively. an edge in an irrlage is a point at which there is a large variation ill pixel value. That is, if the value of a pixel is significarltly different froril the value of orle of its neighbors. theri we say that that pixel is an edge point of the image. Now. each pixel in an irriage lii~seight neighbors. two in the horizontal direction, two in the vertical direction, and four in the diagonal direction. If at a given location, the variatiorl in pixel value is snlall in the vertical dirpctioii but large in the horizorital direction. then that pixel is a vertical edge point of the image. Sirriilarly. if the variation in pixel value is small in the horizo~ltalclirectiori 11ut large in the vertical direction. then that pixel is a horizontal edge point of the iiuage. If the variation is large ill both the horizoiital and vertical directions, then the pixel is a diagonal edge point of the image. Sirlc~the DHT for inatrices illvolves coniputing averages and differences of adjacent pixel values in various combirlations. we can interpret tlie DHT coefficicnt,~as ideiltifyi~igedge poiilt>sof the image. Corlsider for exanlple the 2N-1 x 2N-1 mat,rix d j l ) derived from an

z N -1

image coo = {co (n, rn) I,,,,, =0 by (6.15). Fix 0

5 n. 7 n 5 2 N - 1 . Since

Chapter 6. The Discrete Haar Transform

154

djl) = G ~ B ~ H we ~ % have ~ that C ~by, (6.12) and (6.14), djl)(n,m )

-H

=

row

~ N co(2n,m)

JZ

1

--

JZ

H;O,WC,,

1 (co(2n,fZm) 2

=

-

(2n

+ 1,m )

+ co(2n, 2m + I ) )

If (2n, 2m) is a horizontal edge point of the image co: then the differences co(2n, 2m) - co(2n 1,2m) and co (2n,2m I ) - co(2n 1,2rrz 1) will tend to be large due to the large variation in pixel values in the vertical direction. If (2n, 2m) is a vertical edge point, then these same differences will tend to be close to zero. If (212, 2.172) is a diagorlal edge point, then tlie pixel values will tend to be similar in one of the diagonal directions. That is, at least one of co(2n,2m) - co(2n 1:2m 1) or co(2n,2m + 1)- co(2n 1,2m) will be

+

+

+

+

+

+

+

close to zero. Hence, if (2n, 2m) is a horizontal edge point then dl( 1 ) (n, rn) will tend to be larger than if (2n, 2m) is either a vertical or a diagonal edge. The same argument can be made if the edge point is at (2n,2m + l), (2n 1,2m), or (2n 1:2m 1). Similarly, d l( 2 ) (n. m) will be largest if any of (2n, 2m) : (2n: 2m 1).(2n 1,2m), or (2n 1,2m 1) is a vertical edge, and dY)(n,m ) will be largest if any is a diagonal edge. See Figure 6.4. Since the matrix cj can be thought of as containing the features of the origi~lalimage t,hat are of size 2-r la,rger, t,lia,t is, those features that are larger than scale 2% the matrices d,(1), d,( 2 ) , and d (j 3 ) are interpreted as identifying, respectively, the horizontal, vertical, and diagonal edges at scale 2.7' (See Figures 9.3, 6.7-6.9).

+

6.3.3

+ +

+ +

+

+

"Naive" Image Compression

The key to good image compression is to find a representation of the image with as few numbers as possible. In the language of orthogonal decompositions, this means finding an orthonormal basis in which most of the coefficients of the original image are zero or at least very close to zero. In principle the srnall coefficients can be set to zero without significantly affecting the quality of the image. The purpose of this section is to illustrate some of the principles that make wavelets effective for image compression. The central idea has been

6.3. Image Analysis with the DHT

155

alluded to before, namely that in decomposing a matrix cn into the four matrices cl, d l(1) , d y ) , and dl( 3 ), we have separated the smooth (or slowly varying on a scale of two pixels) parts of the image from the nonsmooth (or rapidly varying on a scale of two pixels) parts of the image. These latter parts are usually interpreted as edge points. If the image consist.^ of large areas of constant intensity separated by edges (which is true of many images), the detail mat,rices will contain many elements that are nearly zero. The same is true when we decompose the matrix cl into c2, d2(1), d(2) , and d r ). This principle is illustrated in Figure 6.10. Here we have taken an image and have computed its DHT with J = 3. We choose various thresholds; that is, fixed numbers below which the DHT coefficients are set to zero, and compute reconstructed images. We see that if 80% of the smallest coefficients are set t o zero, the image is virtually unchanged. If 90% of the smallest coefficients are set t,o zero: most important features of the image are still visible. If 97% are set t o zero, there is significant distortion, but gross features of the image are still recognizable.

FIGURE 6.6. Original image (top left). Reconstruction using only the coefficients (top right), c.2 coefficients (bottom left), and r:c coefficients (bottom right). Reconstructions are increasingly blurred and blocky.

6.3. lrrlage Ai~alvsiswith the DHT

15'7

FIGURE 6.7. Left: Horizontal edges at scale 1. Right: Horizontal edges at scalc 2.

FTGURE 6.8. Left: Vertical edges at scale 1. Right: Vertical edges at scale 2.

158

C1,lapter 6. The Discrete Haar Transfur~n

FIGURE 6.9. Left: Diagonal edges at scale 1. Right: Diagonal edges at scale 2.

6.3. Image Analysis with the DHT

159

FIGURE 6.10. Original image (top left). Compressed image with smallest 80% of DHT coefficients set t o zero (top right). Compressed image with smallest 90% (bottom left) and 97% (bottom right) of DHT coefficients set to zero.

Part I11

Ort honormal Wavelet Bases

Chapter 7 Mult iresolut ion Analysis In Section 5.5, we saw that if h(x) = X[o.l/2)(x) - X[1/2,1)(x), then the collection

forms an orthonormal basis on R. In this chapter, we will see how this construction can be generalized. In particular, we will present a general framework for constructing functions $(x), L2 on R, such that the collection

is an orthonormal basis on R. Such a function $(x) is called a wavelet and the collection {$j,k(x)}j,kEza wavelet orthonormal basis on R. This framework for constructing wavelets involves the concept of a mz~ltiresobution analysis or MRA. Before giving the definition of an MRA, we need to study some properties of c~llect~ions of fi~nctionsof the form

where g(x) is some fixed L~ function. In Section 7.1, we address the following questions: (1)When is the collection {T,g(x)} an orthonormal system? and (2)When does Llle subspace spal?{Tng(x)) admit an orthonormal basis of the form {Tnh(x)) for some possibly different function h(x)? In Section 7.2, we define the notion of an MRA and derive some of its basic properties and in Section 7.3 present some examples of XIRA. In Section 7.4, we give the very simple recipe for corlstructing a wavelet orthonormal basis from an MRA and present some examples of wavelet bases. In Section 7.5. we present a proof that this recipe works, and in Section 7.6, we gather some necessary properties of the scaling and wavelet functions t#hatfollow from the definition of MRA arid the colistruction of the wavelet. Thcsc properties will be useful in later chapters when we explore more examples and generalizations of wavelet, orthonormal bases. Finally, in Section 7.7, we discuss the Battle-LeMarii! construction of spline wavelet orthonormal bases.

164

Chapter 7. Multiresolution Analysis

7.1 Orthonormal Systems of Translates In our study of multiresolution analyses arid their associated wavelet bases, we will frequently encounter orthonormal systems that are integer translates of a single function. In addition to sharing the general properties of orthonormal systems presented in Section 2.3, systems of this form also have special properties that will be valuable in the construction of wavelet bases. In this subsection, we present some of these properties.

Definition 7.1. A n orthonormal system o n R of the f o r m {Tng(~))nE~, where g(z) i s L2 o n R i s called a n orthonormal system of translates.

Example 7.2.

(a) The collection of scale 0 Haar scaling functions

(Definition 5.9) is an orthonormal system of translates by Theorern 5.14. (b) The collection of scale 0 Haar functions {ho.k(z):k E Z ) (Definition 5.11) is an orthonormal systern of translates by Theorem 5.13.

Rernark 7.3. By Theorem 2.55, if (T,g(x)) is an orthonormal system of translates, then it is by definition an ortllornormal basis for the subspace span{Tng(x)). In other words,

f (x) E span{T,g(x)) if and only if

Lemrna 7.4.

T h e collection (T,g(z)) i s a n orthonormal system of translates zf and only if for all y t R,

Proof: Not,e first, t,ha,t, ( T k g , Tl g ) = ( g , Ttpkg ) = 6 ( k - !) if and only if (g, Tkg) = 6(k). By Parscval's formula,

7.1. Orthonormal Systems of Translates

165

(see Exercise 7.10 for a justification of the interchange of the sum and integral in the last step). By the uniqueness of Fourier series,

if and only if x F ( y + n ) / 2 = 1 forall ~ E R . n

Lemma 7.5.

Let { T , g ( x ) ) be a n orthonormal system of translates. T h e n the function f ( x ) t @Zi%(Tng(x)) zf a n d only i f there i s a n t2 sequence { c ( n ) ) such LJLU L

ic-:) = n-a (Ccc1.

e-2x.n~)

n

Remark 7.6. The only assumption being made about the coefficieiits { ~ ( n ) is ) that C , Ic(n)I2 < CCI. Therefore, we cannot conclude necessarily that the Fourier series Enc ( n )e p Z T i nis~ a piecewise continuous function. The most we can conclude, however, is that this Fourier series represents a function L2 on [0,11 in tlie sense of Lebesgue. We know this to be true by the Riesz-Fischer Theorem (Theorem 4.48). This particular technicality does not enter seriously into the proof of Lemma 7.5; so we will not mention it further. In light of (7.3), we know that the coefficients c ( n ) are given by ( f ,T,g) so that by making appropriate assumptions on the functions f ( x ) and g(x), we can say more about the coeficierlLs ~ ( 7 1 )and llle corresponding Fourier series.

Proof:

(=-=+) Suppose that f ( x ) E ~ { T n g ( x ) )By . Theorem 2.55,

in L2 on R.Taking the Fourier transform of both sides and using Theorem 3.40(b),

166

Chapter 7. Multiresolution Analysis

By Bessel's inequality,

so that

sat sifies (7.2). (t=) Suppose that (7.2) holds. Then by the Riesz-Fischer Theorem (Theorem 4.48),

Letting

C

f ~ ( =4

c ( n )Tng(:c),

InllN

it follows that f N ( x ) E span{~,g(x)} and that

Therefore, by Plancherel's formula (Theorem 2.57(d)), Lemma 7.4, and c(n) tip2""?, the periodicity of

and hence f (x) E @ZE{T,g(z)).

7.1. Orthonormal Systems of Translates

167

In some of the examples of multiresolution analyses that follow, we will encounter collections of the form { T , ~ ( X ) )that , ~ ~are not orthonormal systems of translates but that satisfy a weaker version of (7.1), namely that there exist constarlts A, B > 0 such that for all y E R,

For such a system, we wish to consider the subspace ~ { T n g ( x )and ) show that in fact there is an L~ function g(x) such that {T,,g(x)) is an ortllonormal basis for @Zii{T,g(x)). The construction of g ( x ) is referred to as an orthogonalixation of the collection { ~ , , g ( x ) ) . The following lemma shows how to orthogonalize a collection {T,g(x)) satisfying (7.4). To avoid certain technicalities in the proof, we assume that g(x) has compact support. Lemma 7.7. Suppose that g ( x ) is L' o n R with compact support. If the system { T , g ( x ) ) satisfies (7.41, then there is a function ;(XI, L~ o n R, such that: (a) {T,g(z)) is a n orthonormal system of translates and

Proof: Sirice g(x) is compactly supported, Exercise 7.11 and (7.4) implies that the function

is a trigonometric polynomial that never equals zero. Define

Then @(y) is C0 (in fact, Cm) on R and car1 be expanded in a Fourier series as

where the Fourier coefficients satisfy Define the function g ( x ) by

C , Ic(n)l2 < a.

Taking the inverse Fourier transform of both sides, it follows that

168

Chapter 7. Multiresolution Analysis

Since g(x) has compact support, the sum on the right side is finite if x is restricted to any closed finite interval. Hence, on every such interval, g(x) is piecewise continuous and so is piecewise continuous on R. By (7.4), @(y) is L" on R so that ?(?) and F(z)are L2 on R. Since @(y)has period 1,

for each k E Z, and hence

By Lemma 7.4, {T,g(z)) is an orthonormal system of translates and (a) is proved. To see that (b) holds, note that by (7.5) and Lemma 7.5,

By Exercise 7.9, Tkg(x) E span{T,g(x)) for each k E Z and by Exercise 2.61, span{TTLg(z)} is closed under the formation of linear combinations. Therefore span{T,,g(x)) C ~pan{T,g(x)}. Let f (x) E S@Zifi{~,g(x)). This means that given t > 0, there is a function h ( z ) E span{~,ij(x)) such that (1 f - hlla < c/2. Since span{T,g(x)) C: ~p""{T,g(x)), there is a function r ( x ) E span{T,g(x)) such that 11 h-rl2 < €12. Therefore, by Minkowski's inequality,

Hence f (x) E span{T,g(x)) and

A

Since g^(y) = y(?) @-l(-y) the same argument with a l ( y ) replacing @(y) shows that span{T,g(x)) C spnn{T,g(x)} and (b) follows.

Exercises Exercise 7.8. Given any function f (x), L~ on R with J J f 1J2 center of mass of f (x) is defined to be the number

=

I, the

7.2. Definition of MRA

169

provided that the integral exists. Prove that if the center of mass of a function $(x), L2 on R with )($112 = 1, is m, then the center of mass of 4,,k(x)is 2 - j ( r n k ) .

+

Exercise 7.9. Consider the collection of functions (orthonormal or not) {Tng(x)) where g(x) is L~ on R. Show that w { T n g ( x ) ) is invariant under integer translatio~l.T l l a ~is, slluw that if f (x) t span{TT,g(z)), then Tkf(x) E spal?{Tng(x)) fur each k E Z. Exercise 7.10. Justify the interchange of the sum and int,egral in the proof of Lemma 7.4 by showing that if g ( x ) is L2 on R, then the suln

converges in L1 on [0, I].

Exercise 7.11.

(a) Show that if g(x) is L2 on R, then

(b) Conclude that if g(z) is L2 on R and has compact support, then

En

+ n) l2

is a trigonometric polynomial.

7.2 Definition of Multiresolut ion Analysis Definition 7.12. A multiresolution analysis o n R i s a sequence of subspaces { V , ) , E ~of functions L~ o n R satisfying the following properties. ( a ) For all j f Z , T/,

C V,+1.

( b ) I f f ( z ) i s Cz o n R, then f (z) E Sj5iE(V,),Ez. That is, given i s a j E Z and a function g ( x ) E V, such that I1 f - 9112 < E .

E

> 0, there

( d ) A function f (x)E Vo i f and only if D02.1 f (x) E VJ. (F)

There exists a function cp(z),L~ o n R, called the scaling function such that the collection {T,cp(z)) i s a n orthonormal system of tran,slates and

170

Chapter 7. Multiresolution Analysis

Remark 7.13. (a) Typically, an MRA is defined by first identifying the subspace Vo, defining V, by letting

so that Definition 7.12(d) is satisfied, and then proving that Definition 7.12(a), (b), (c), and (e) hold. Vo can be defined by first identifying a function cp(x) such that {T,p(x)) is an orthorlormal system of translates, and then defining

(b) Note that if f (x) E Vo, then by Definition 7.12(e),

By the orthonormality of {T,,cp(x)),

7.21 Some Basic Properties of MRAs In this subsection,

{VJ)

always denotes an MRA with scaling function p(x).

The Approxirrlation and Detail Operators

Definition 7.14.

For each j , k

E

Z define cpj,k(x)b y

For each j E Z, define the approximation operator Pj o n functions f (x),L' o n R by

For each j

E

Z,

define

th,e detail operator Q, o n functions f (x),L2 o n

R

by

Lemma 7.15. For each j E Z, {pj,k(x))j,kEz i s a n orthornormal basis for

7.2. Definition of MRA

171

Proof: Note t h a , t since y O , k ( ~ E) VOfor all k, Definition 7.12(d) implies that D 2 3 ~ 0 , k= ( ~(Pj,k(x) ) E for a11 k . Also, since {cpo,k(x)}is an orthonormal system of translates, Theorem 3.42 (c) implies that,

Hence { c p j , k ( ~ ) ) k E z is an orthornormal system on R. Given f (x) E DZ-/ f (x) E Vo SO that by Definition 7.12(e) and Theorem 3.42(c),

4,

Applying D2J to both sides of Lhe above equation, we obtain

and the result follows by Theorem 2.55.

Lemma 7.16. (a) lim (IP,f

For all f (x),C: o n R: -

3'00

flln =

0, and

Proof: To see (a), let E > 0. By Defirlition 7.12(b), there exists J E Z and g(x) E VJ such that 11 f - g1I2 < ~ / 2 By . Definition 7.12(a), g(x) E and Pig(x) = g (x) for all j 2 J. Thus,

VJ

IIf

-

Pjf 112

Ilf 5 Ilf

g I PJg - P J f l l z

=

9/12 + IIPJ(f- g)112 2llf - 9112

<

-

E,

where we have applied Minkowski's and Bessel's inequality. Since this holds for all j J, (a) is proved.

>

172

Chapter 7. Multiresolution Analysis

To see (b), suppose that f (x)is supported in [-A, A] and let c > 0. By the orthonorrnality of { p j , k ( z ) ) k E z , and applying the Cauchy-Schwarz and Minkowski inequalities,

To do this, let c > O and choose K so large that

Therefore, if 2,'A

< 112, tiler]

Since for each k E Z, limj+-,

.2'A-k J-2JA-k

lq(x)l2dx = 0 ,

7.2. Definition of MRA

Since

E

173

> 0 was arbitrary, (b) follows.

The Two-scale Dilation Equation Lemma 7.17.

There exists a n t2 sequence of coeficients { h ( k ) } such that

in L' o n R. Moreover, we m a y write

where

Proof: Since cp E VoG Vl, and since by Lemma 7.15(a), { q l , k ( ~ ) )isk E Z an orthonorrrlal basis for Vl,

Thus, (7.7) holds with h ( k ) = (cp, c p l , k ) , which is e2 by Bessel's inequality. Equation (7.8) follows by taking the Fourier transform of both sides of (7.7).

Definition 7.18. LeL y3 ( z ) be the scaling function associated with a n MRA {V,}. T h e sequence { h ( k ) ) satisfying (7.7) is called the scaling filter associated with y ( x ) . T h e .function r r ~ ~ defined ( ~ ) by (7.9) is called the auxiliary function associn.ted with ~ ( 2 ) .

Remark 7.19. To call h ( n ) a filter is slightly misleading. According to Definition 4.13, a .filter must satisfy C , ( h ( n 1) < oo.This does not necessarily follow from the definition of h(n)given in Lemma 7.17. It is convenient t o makc this assumption, and we will do so in what fulluws. In fact the sca,ling filter will satisfy I h(n)1 < CX; in every example in this book but one (the Bandlimited MRA) .

En

Chapter 7. Multiresolution Analysis

174

7.3 Examples of Mult iresolut ion Analysis 3

.

The Haar M R A

Let Vo consist of all step functions f (x) such that (1) f (x) is L2 on R and (2) f (x) is constarlt on l l ~ einle~vals for k: E Z. 111other words, Vo is the collection of all scale 0 dyadic step functions, L2 on R. By Exercise 7.26,

where p(x) = XIO,L)(x). Since by Theorem 5.14, {T*(z)) is an orthonormal system of translates, this proves that Definition 7.12(e) is satisfied. For each j E Z, define 4 by Definition 7.12(d); that is, f (x) C T$ if and only if D2-.,f (x)E Vo. By Exercise 7.27, 4 consists of all step functions f (x) such that (1) f (z) is ~ % n R and (2) f (x) is constant on the intervals Ij,k, for k E Z. In other words, V, is the collection of all scale j dyadic step functions, L2 011 R. It remains only to verify Definition 7.12(a)-(c). To see tha,t Definition 7.12(a,) holds, we must prove that if f ( x ) E Ii,, then f ( x ) E y+l for ariy j E Z. Recall that by Definition 5.3, IJ,k = Ij+1,2kU Ij+1,2k+l for all j , k E Z. This means that if f (x) is corlstarlt or1 Ij,k for all k E Z, it is also constant on Ij+l,efor all 4!E Z. Thus, if f (x) is a scale j dyadic step function, it is also a scale j 1 dyadic step function, and Defiriitiorl 7.12(a) is verified. That Definition 7.12(b) holds is a direct consequence of Lemma 5.37(a). To see that Dcfirlition 7.12(c) holds, note that to say that f (x) E ng-, is to say that (1) f (x) is L2 on R and (2) f (z) is constant on thc intcrvals [o,CCI) and (-oo,0). But the only such function is the function identically zero.

+

v.

7.32 The Piecewise Linear MRA Let Vo consist of all functions f ( x ) , L~ and C0 on R., a n d linear on the intervals for k E Z. For each j E Z, define V, by Definition 7.12(d); that is, f (x) E if and only if D2-) f (x) E Vo.By Exercise 7.28, consists of all functions f (x), h2 and C0 on R, and linear on the intervals IJ k , for k c Z. It remains to verify Definition 7.12(a)-(c) and (e). To see that Definition 7.12(a) holds, we must prove that if f (x) E V J 1 then f ( x ) E V,+l for any j E Z. Since IJ,k = IJ+1,2kU IJ+1,21c+l for all for all k E Z, is also lincar on IJ+l,e j, k E Z , any function linear on for all .t E Z. Thus, if f (x) is L2 and C0 on R and linear on IJ,k for all k E Z, it is also L2 and C0 on R and linear on IJ+l,efor all !E Z. Thus, Definition 7.12(a) is verified. In order to prove Definition 7.12(b), let E > 0 and let f(x) be C: on R and supported in the interval [-A, A]. Since f (x) is continuous and has

5

compact support, it is uniformly continuous; so for j large enough, we know that given xo E I j , k , If ( x ) - f ( xo)l < for all x E IJ,kand k E Z. Now, let f j ( x ) be defined as follows. For each k E Z, let 13,k= [a,b) and let b-x x-a f (a') -f ( h ) , f,(4= - a b-a for x E Ij,+.Since b-x x-a = 1, b-a b-a

~/m

,

+

+-

.ir~i-.:~w =

b-x x 7 f

(

b -a b-x

x-a xb -) -

b-x b-u

x-a

b-a x

-

a

for all x E Ij,k.Thus,

This proves Defintion 7.12(b). To see that Definition 7.12(c)holds, note that to say that f ( x ) E ng-,V, is to say that ( 1 ) f ( x ) is L~ and C O on R arid (2) f ( x ) is linear on the intervals [O, m) and ( - O O , ~ ) . But the only such function is the function identically zero. To see that Defintion 7.12(e) holds, we will use Lemma 7.7. Before applying the Lernrna, we will need to establish the following facts. Let

Lemma 7.20.

I f f (x) is then f (x) can be written

CO

o n R and linear o n the intervals Io,k for k E Z ,

where the s u m converges pointwise.

Proof: Let k E Z be fixed, and consider (7.11) for x E 10,k.For any such x , the sum on the right side of (7.11) consists of exactly two terms. Hence the sum converges pointwise and we must verify that in fact f ( x ) = f ( k ) Tkcp(x)

+ f ( k + 1)T k + l v ( ~ ) .

(7.12)

176

Chapter 7. Multiresolution Analysis

Since T,cp(x) is linear on I o , k for all n t Z , it follows that the right side of Since Tkcp(k) = cp(0) = 1, and Tk+lcp(k) = p ( - 1 ) = (7.12) is linear on 0 , equation (7.12) is satisfied when x = k. Since Tkcp(k 1 ) = p ( 1 ) = 0 , and T k + l p ( k + 1) = p ( 0 ) = 1 , equation (7.12) is satisfied when x = k + 1. Thus, the right side of (7.12) is a linear function on I o , k that agrees with f ( x ) at the endpoints. Since f (x) is also linear on it must agree with f ( x ) on the wholc interval. Sincc this holds for any k E Z , (7.11) holds for every x E R.

+

Since we are interested in L~ convergence of (7.11) and since pointwise convergence does not necessarily imply L 2 convergence, we must prove L~ convergence separately.

Lemma 7.21. Suppose that f (z) i s linear o n the i.r~ter.uul [n,rL + 1) Jos some n E Z. Then

Proof: Since f ( x ) is linear on [n,n+ 1 ) , f ( x ) = f ( n )+ ( f (n+1 ) - f ( n ) (x ) n) for x E [n, n 1 ) . Therefore,

+

Because of the inequality 2ab 5 a 2

Therefore.

Also

+ b2, for any real numbers a and b,

7 . 3 . Examples of MRA

177

a'nd (7.13) follows.

Lemma 7.22. Suppose that f ( z ) is L~ and C O o n R and is linear o n the intervals I o , for ~ k E Z. T h e n (7.11) holds in L~ o n R. Proof: Since f ( x ) is L2 on R, (7.13) implies that

In particular limlni,, 1 f (n)I2= 0. Let M , N E N , and consider the partial sum cause (7.l l ) holds pnintwise,

x

f(-M)(x I M+1)

N

f ( n ) T n ~ ( x=)

n=-hl

f (4

f ( N ) ( N +1 - x ) 0

xr=N=_M f (n)T,,q(x). Beif x E [-A(- 1 , - M ) , if x E [-A[, N ) , if x E [ N , N I ) , otherwise.

+

Therefore.

+

lim

+

lim

If

(x>12 dx

I f (412dx

Chapter 7. Multiresolution Analysis

178

Lemma 7.23. Vo = span{T,p(x)). Proof: Since f (z) E 6 is L~ and C0 on R and linear on 1 0 , k for k E Z. Lernma 7.22 says that (7.11) holds in L2. Since every partial sum of the f ( n )Tnv(x) is in span{Tnp(x)}, Lemma 7.22 implies that form f ).( E s p a n { T T L c p ( . ~ ) ) . Conversely, suppose that f (x) E SjT%i{Tncp(x)}. By Exercise 2.62, we can find a sequence of functions { fk(z)}such that f k ( x ) E span{Tn cp(z)) and lirnk,, 11 f - f k 11 2 = 0. We need to show that f (z) is linear on the intervals loe for e E Z, that f ( x ) is C0 on R and that f (x) is L~ on R. Fix & E Z. Since each f k (x) is linear on In,! and since f k (x) converges to f (x) in L~ 011 Exercise 7.30(a) implies that f (x) is also linear on Io,g. Since by Exercise 7.30(b), the convergence is also uniform, f (x) is C0 on R. Finally, since each f k ( x ) is L2 011 R arid since f k ( x ) converges to f (x) in L~ OII R, f (z) is also L~ on R. Hence, f (x) is in Vo.

c:=-~

We are now in a position to prove that Definition 7.12(e) holds.

Theorem 7.24.

There is a functi07~@(LC), L~ o n R, S,UCJZ that:

( a ) ( T , , y ( x ) }i s a n orthonormal system 0.f translates, and

(b) Vo = 3FGi{Tr,F(x)).

Proof: We will prove this theorem by applying Lemma 7.7. To do this, we must first show that cp(.z.) = (I - 1x1) Xr-l,ll (x) satisfies (7.4). In order to do this, note that (Tncp,T,p)

=

213 if n = m , 116 if In - ml = 1, 0 otherwise.

(7.14)

By Exercise 7.11, it follows that

Hence.

for all y E R and (7.4) is satsified. The theorem follows by Lernma 7.7(a) and Lemma 7.23. See Figure 7.1.

7 . 3 . Examples of MRA

7.1. Top left: ~ ( z ) . Top IG(? + n) 12)-'I2. Bottom right:

FIGURE

(C,

7.3.3

right:

F(?).

Bottom

179

left:

A

G(?).

The Bandlimited MRA

Let Ifo consist of all functions f (x) bandlimited with baridlirriit 1 (Definition 3.47). In other words, Vo consists of all functions f (z),L~ on R, such that is supported in the interval [-1/2, 1/21. For each j t Z, define I.; by Definition 7.12(b). That is, f (x) E V, if and only if D z P Jf (x) E Vo By Exercise 7.31, 5 consists of a11 filnctiorls f (x) bandlimited wit21 bandlirriit 2J. To see that Definition 7.12(e) holds, let

y(?)

Then {T,cp(x)} is an orthonormal system of translates by Exercise 3.52. It remains to show that

To see this, note that if f (x) E Vo, then by the Shannon Sampling Theorem (Theorem 3-50), f (x) E s p a n { T , c p ( ~ ) ) , ~ ~For . the opposite inclusion, note that if f (x) E span{T,,,cp(~)}, then by Exercise 2.62 there is a sequence of functions {fk(x)}k6Nsuch that for each k E N, f k ( x ) E span{T,cp(x)),

180

Chapter 7. Multiresolution Analysis

and such that limk,, 11 f - f k I 2 with bandlimit 1. Therefore,

= 0. For each

=

Ilf

k

E

N, Jk(x)is ba~ldlii~iited

2

- fklla

-+ m , we conclude that h71,,,z 1 f ( ? ) I 2 dy = 0 and hence that Y(7) = 0 if 171 > 1/2. Thus, f ( x ) E Voand (7.15) follows. A

Letting k:

. . ,

,

4

Clearly the subspaces are nested so Definition 7.12(a) is satisfied. To verify Definition 7.12(b), let f (x) be C: on R. Since f (z) is also L2 on R, Plancherel's forniula says that f^(y) i d 2 on R By Corollary 2.37(b), there is a function g(y), C: on R such that 11 f - ? I z < E . Since ?(y) is L2 orr R, so is g ( x ) (it's inverse Fourier transform) by Pla,ncherel's fnrrn~lla,.Also by ,. Plancherel's formula, 11 f - gl12 = 11 f - 5112 < E . Since g(y) is supported in an interval of the form [-A, A] for some A > 0, then y (x) E VJ as long as 2 ~ - I> A. Definition 7.12(b) follows. To verify Definition 7.12(c), let f (z) E nJEzVi. Then f (7) is supported in [ - 2 j 1 , 2 i 1 ] for all j E Z. Letting j i m , it follows that f^(?) varrislies everywhere except possibly at y = 0. But since f (7) is L2, this rrrearis that .f (y) must be identically zero so that f (x) is ideritically zero as well. A

A

A

A

7.4

The Meyer M R A

Tliis example of an MRA is due to Y, hleyer, and the correspondiilg wavelet basis is historically the first example of a smooth orthonormal wavelet basis. The idea behind the Meyer MRA is to create a "smoothed" version of the bandlimited MRA by replacing the sharp frequency cutoff function X[-1/2,1/2)(y)by a smoother bell-shaped cutoff function in the frequency domain. The result will be a wavelet with better decay in the time domain (see Section 3.7). To this end, we define below the specific properties required of our smooth cutoff function.

Definition 7.25. Given k E N ( o r k = oo), a ,function b ( z ) is a C" bell function over [- 1/2,1/2] provided that b(z) is C" o n R and satisfies the follouling conditions: ( a ) b ( z )= 1 zf 1x1 5 l / 3 , (b) b ( z ) = 0 if 1x1

> 2/3,

7 . 3 . Examples of MRA

181

(c) 0 5 b ( x ) 5 1 for all x E R, and (d) x l b ( x + n ) 1 2

-- 1.

One way to construct such a function is as follows (see Figure 7.2).

c"'

(1) Define a "bump" function P(n:), (or Cw) on R and supported in the interval [-I, 11. If k is finite, this can be done with the followirlg construction (cf. Example 1.14(e) and Section 7.7.1). Define P o ( 4 = X[-l/(k+t),~/(k+l)](~). (7.16) and let Pn(x) = PO* PrL-l(x), for n E N. By this definition, (Exercise 7.32). Finally, let

(7.17)

PI; is Ck-I on R and is supported in [

1,11

where ck is chosen to guarantee that

(Exercise 7.33). For the k = m case, see Exercise 7.34. (2) Define a "sigmoid" furlctiorl B ( s ) 1,y taking an antiderivative of /?(n)

Then H(x) is C y o r C") on R arid satisfies: (a)

Q(2)=

O i f x 5 -1,

(b) Q(x)= 1 if n:

(c) 0 5 H(x)

2 1, and

< 1 for all n: E R.

(3) Define

(

)

=1

(

'7i

( 6 ) )

and

c ( r ) = cos

(i

~ ( 6 s ) .)

Then s(x) and c(z) are each C v o r CCO)on R and satisfy: (a) s(x) = 0 and c(x) = 1 if x 5 -1/6, (b) s(z) = 1 and c(x) = 0 if n:

2 1/6,

(7.21)

182

Chapter 7. Multiresolution Analysis

< c(x) 5 1 for all x E R, and

(c) O 5 s ( x ) 5 I and 0

+

(d) s 2 ( x ) c2(z) = 1, for all x

E

R.

(4) Define b(z) by

Then b(x) is a Ck bell function over [-1/2: 1/21. The Meyer MRA is defined as follows. Given k E N (or k = oo), let p(x) be such that +(y) is n C k bell function over [-1/2,1/2]. Define

and for j E Z, T(, by f (x) E T(, if and only if D2-If ( x ) E Vb. Thus, Definition 7.12(d) is satisfied. Since +(7) satisfies Definition 7.25(d),

{T,cp(x)) is an orthonormal system of translates by Lemrna 7.4, and Defirliliori 7.12(e) is satsified. In order to verify that Definition 7.12(a) holds, recall that by Lemnla 7.5, f (x) E Vo if and only if there is an t2 sequence of coefficients {a(n)) such that i ( 7 ) = ( x a i n ) e-2x.n-)

pi-) = a:-:

~(7).

(7.22)

Tl

We first show that Vo Vl. Let f (x) E Vb. In order to show that also f (z) E Vl , we must show that Dll2f (x) E VO. By (7.22),

Since @(2y) is supported on the interval [-1/3,1/3] and @(y) = 1 on [-1/3,1/31, 8 2 7 ) = F(27)

@(?I.

Therefore, A

Dl12f (7) = A a ( 2 y ) +(2y) Q(y). Define a(y) to be the period I extension of the function a(2y) @(2(~)). Since 6;(2(~))is continuous on R and periodic, it is bounded. Hence a(?) is L~ on [0, 1) and therefore has t2 Fourier coefficients {Z(n)}. Hence,

7.3. Examples of MRA

183

D l 1 2 f ( z ) E Vl. If for any j E Z, f ( z ) E 4, then D2-Jf ( x ) E V". By the above argument, D2-Jf ( x ) E Vl and hence D2-3-l f ( x ) E VO.Thus, and Definition 7.12(a)holds. f(x) E In order to prove Defiiiition 7.12(b),let E > 0 and let f (x) be C: oil R. By Exercise 3.55, we can find a function g^(y),C: on R and supported in an interval of the form [-A, A] such that 11 f - glla < E . In order to see that g E U J F Z y ,clloose J SO large that A < (1/3)2'. In this case @ ( 2 ~ ~=? 1) 011 [-A, A] SO that ?(?) = c ( y )F(2Ps7y). Let a(?) be the period 2.' extension of G(y).Since 2 > A, G(y) = a ( y )@(2-J y ) , and so Ij(zJ y ) = a(2J y ) $ ( y ) . Since F ( 2 J y ) is continuous on R, c ~ ( 2 ~has y ) t2 Fourier coefficients. Thus, g ( 2 - J x ) E Vo and g ( x ) E V J . To prove Definition 7.12(c),let f ( z )E n j E z V , . This means that for every j t Z, D 2 ,f ( x ) E Vo or that f^(?) has the form T ( y ) = a ( y )@(2-jy).But sirlcc +(2-.jy) = 0 if (2/3)2-1,lctting j + C O , wc scc that f (?) = 0 for every y 1 > 0, which implies that T ( y )isidentically zero and hence that f ( z )is identically zero.

lyl >

A

Exercises Exercise 7.26.

Show that

where p(:x) = Xlo,l)(z), where Vo is given in Section 7.3.1 Exercise 7.27. Prove that a step function g ( x ) is constant on the interfor k t Z if arid only if D2-.,g ( x ) is a step function coristant on vals the intervals for k E Z. Exercise 7.28. Prove that a function g ( x ) is Go on R and linear on the intervals Ij,k, for k E Z if and only if D Z p g 7 ( x ) is C" on R and linear on the intervals for k E Z. Exercise 7.29. Prove equation (7.14). Exercise 7.30. (a) Let {fn(z)),EN be a sequence of linear functions converging to a function f ( x ) in L2 on a finite interval I . Show that f ( x ) must also be linear on I. (Hint: Let f n ( x ) = anx b,. Use the fact that { f n ( z ) ) , is~ L2 ~ Cauchy on I-see Exercise 1.50(c)-to show that {a,,) ,EN and { b n j n E N are Cauchy and hence convergent sequences of numbers. Prove that if lim,,, a, = a and limn,, b, = b, then f ( z ) = ax b on I . )

+

+

(b) Prove that under the assumptions in part (a), f n ( x ) converges uniformly to f ( x ) on I .

184

Chapter 7. Multiresolution Analysis

FIGURE 7.2. From top left: P(x) with k = 2, the sigmoid function O(x), s ( x ) , and c ( x ) . The c2bell function b ( x ) (bottom).

Exercise 7.31. Prove that for any j E Z , f (z) is bandlirnited with bandlimit 23 if and only if D2-3f ( z ) is bandlimited with bandlimit I. (Hint: Use Theorem 3.40(a).) Exercise 7.32. Prove that for each n E N, the function /?,(z) defined by (7.16) and (7.17) is on R and is supported in the interval [-I, 11.

c"'

Exercise 7.33. For each k, find c(k) in (7.18) such that (7.19) is satisfied. (Hint: Consider the Fourier transform.)

7.4. Construction and Examples

185

Exercise 7.34. Show that the function e("-l)/("+1)

Q(x) =

0 1

if < 1, if x < -1, i f n : ? 1,

is a C" sigmoid function. That is, Q(x) is Cw on R, and satisfies (a) 8(x) = 0 if n: -1, (b) O(x) = 1 if z 2 1, and (c) 0 5 8(2) 1 for all x E R. Use tliis fullcliorl t o construct a CCObell functioii on [-I, 11.

<

<

7.4 Constructio~~ and Examples of Ort honormal Wavelet Bases The goal of this section is to provide an algorithm for constructing a wavelet orthonorma,l ba,sis given an MRA. The algorithm is given in the following theorem.

Theorem 7.35. Let {h)be a n M R A with scaling function cp(x) and scaltng filter h(k).' Define the wavelet filter g ( k ) by

and the wavelet @ ( x ) by

Then

{'$J,~(x))J,~Ez

is a wavelet orthonormal basis o n R. Alternatively, given any J E Z ,

is a n orthonormal basis o n R.

Remark 7.36. (a) Since 6 = SjEF{cpl,k(z)),(7.24) implies that $(z) E Vl (b) Taking the Fourier transform of both sides of (7.24) gives that

'We are making the assumption that h ( n ) is a filter, i.e., t h a t convenience only. See Remark 7.19.

lh(n)l

<

m for

186

Chapter 7 . Mi~lt,iresnli~tinn Analysis

where

and mo(y) is defined by (7.8) (Exercise 7.39)

7 . 1 Examples of Wavelet Bases The Haar Wavelet: Recall that scaling function for the Haar MRA was defined by y ( z ) ( z ) . Also note that by Exercise 5.18,

=

Therefore,

and as in the proof of Theorem 7.35,

Therefore, by (7.24),

By Theorern 7.35, { I j l j , k ( ~ ) ) j , k E Z is a wavelet orthonormal basis on R. The Piecewise Linear Wavelet: Recall that the scaling function, defined by

where q ( x ) = (1 - 1x1) X[-I.I]

F(z), for

(2) and

the piecewise linear MRA is

7.4. Construction and Examples

187

Since

p(x) = -

1 -

2

p(2i

+ 1) + p(x) + 51 p(2x

1

1)

1 1 +cp1,o(z) + cp1,1(x), Jz 2Jz

-c P l , - l ( ~ )

2&'

-

-

(7.27)

it follows that (Exercise 7.40)

@(Y)= c 0 s 2 ( ~ 7 / 2P) ( Y / ~ ) Therefore, $(y)

=

Therefore, by (7.26),

6(1 + 2 cus2( n y ) )1/2 y-(y)

(7.28)

188

Chapter 7 . Multiresolution Analysis

Taking the inverse Fourier transform of both sides,

wliere

See figure 7.3.

FIGURE 7.3. Left: The piecewise linear scaling function F ( x ) . Right: The piecewise linear wavelet.

The Bandlimited Wavelet: Recall that tlie scaling function for the bandlimited MRA is defined by ?(y) = Xl-llz,llz)(7').Since p(y/2) = X[-l,l)(y),it follows that

where rno(7/2) is the period 2 extensiori of X~-112,112)(?) SO that mo(?) is the period 1 extension of X[-114,~14) (7). Now, by (7.26), rnl(7) is the period 1 extension of the function

7.4. Construction and Examples

189

so that

By taking the inverse Fourier transform,

?%4=

s i n ( 2 ~ x) COS(TX) n(.?: - 1/21

-

sin n ( z - 112) (1 - 2 sin TX) T ( x - 112)

See figure 7.4.

FIGURE 7.4. Left: The bandlimited scaling function sin(~x)/(;.x)). Right: The bandlimited wavelet.

The Meyer Wavelet: Recall that the scaling function for the Meyer MRA if 171 5 113, .s(y 1/21 if E (1/3,2/3), c(y-112) i f y E (-213,-1/3),

I

+

where s(y) and c(y) are defined by (7.21). Recall also that P(y) is ~ % n R and has the property that 0 < F(y) 1 lor all y E R. Since @(y/2) is identically 1 on the interval [-2/3,2/3], @(y)= mo(y/2) @(y/2), where mo(y) is the period 1 extension of the function $(2y) X,-1/2,1/21(7).Note also that mo(y + 112) e-2ffz(*1/2) vanishes on [-1/6,1 jG] , is identically 1 on 1113, 2/31 and on 1-213, - 1/31, rises from 0 to 1 on [1/6, 1/31, and falls from 1 to 0 on [-113, -1161.

<

190

Chapter 7 . Multiresolution Analysis

By (7.25), $ (x) is defined by

and

' 4 ' , ( ~ ?=

Tllell 0

< $(?)

5 1 for all y E R, and T(y) is ~ " . n R. See Figurc 7.5.

FIGURE 7.5. Top left: The Fourier transform of the Meyer scaling function with k = 2 . Top right: The Fourier transform of the Meyer wavelet. Bottom left: The Meyer scaling function. Bottom right: The Meyer wavelet.

7 . 2 Wavelets in Two Dimensions The goal of this section is to dcfinc an expansion of a function f (x, y), L~ on R2 in terms of an orthonormal collection of functions that is based on dilations and translations of basic wavelets. There are many ways to

7.4. Construction and Examples

191

do this but here we will present a very popular and simple construction called tensor-product wavelets. The construction is based on a single twodimensional scaling function @(x,y) and a set of three two-dimensional wavelet functions P ( l ) ( x ,y), Q ( ~ ) ( xy), , and Q ( ~ ) ( xy). , We will prove the following theorem.

Theorem 7.37.

Let p(z) and $(x) be the scaling and wavelet functions associated with s o m e MRA and define

For each j , F1, k2 E Z, define

T h e n the followzng hold. ( a ) T h e collection

~~:ll~

, k 2 ( X , ~ ) 1 1 < 7 < 3 , , / , k 1. k 2 E z

(7.32)

is a n orthonormal basis o n R2. ( b ) For each J E Z, the collection

i s a n orthonormal basis o n R2

Remark 7.38.

(a) Note lhal in the above construction the dilation is isotropic; that is, it is the sarne in both the x and y direction. This is desirable for a couple of reasons. First, the basis elements are indexed by only three parameters, scale j, and location (kl, k 2 ) , instead of four (we are not counting the index 2 ) . Second, the basis does not "favor" any spacial direction; that is, each basis element does not have any particular orientation but is spread out evenly over its center of mass.

(b) It is possible to define orthonormal wavelet bases of the form (7.32) where only one wavelet is required rather than three. However, such wavelets are not associated with arly MRA so there is 110 corresponding scaling function. Any wavelet orthonormal basis in two dimensions that comes from an MRA will require three wavelet functions.

192

Chapter 7. Multiresolution Analysis

(c) We can provide some insight into why three wavelets are required for this construction by considering natural extensions of the approxirrlatiori and detail operators Pj and Qj to two dimensions. For each j E Z, let us define Pj by

We can realize the approxirnation operator P j as the composition of two operators operating in the x and y directions independently. Define

Then we car1 assert that (2) (1) P j f ( x , Y ) = p' j" ~ (j~ ' f ( . x , t J=) P j P, f ( x , y )

(7.35)

(Exercise 7.41). We also define the detail operators in the x and y directions

and

Then and Therefore we can write

7.4. Construction and Examples

193

where

4:" f

((r,

y)

=

x x(f,

*:il

' @ ~, k ~2 ~ ) l

kl

, k 2 (1, Y).

kz

Proof of Theorem 7.37: Verifying orthonormality of the systems (7.32) and (7.33)is easy and follows from the orthonorrnality of the corresponding one-dimensional wavelets and scaling functions. To verify completeness, note that it can be shown (Exercise 7.42) that for every f (n;, y ) , C: or1 R ~ ,

Then for any such f (x,y),

7.4. ,? I;ocnlzzatzon of Wuvelet Bases Time or Spatial Localization: We have seen in Remark 5.12(a) that if $ ( a ) is the Haar wavelet, then $;,k(x) is supported on the dyadic interval I,,* = [ 2 - i k , 2 - J ( k + 1)).This

194

Chapter 7. Multiresolution Analysis

identifies the Haar basis as a local basis in the sense that if f (x) is supported in an interval (A, B), then its Haar coefficients { (f,Qj,k)) will vanish whenever does not intersect (A, B) (Section 5.4.1). In all of the other examples of this section, the scaling function cp(z) is even (cp(-z) = cp(x)) and has its center of mass at zero. Consequently, +(x) satisfies $(1 - x) = $(x) and has its center of mass at 1 / 2 (Exercise 7.43). Thus, we may think of $(x) as concentrated in the interval = [O, 1) arid of $j.k(x) as co~icentratedin the interval Ij,k. Another point of view is to note that since the center of mass of $(x) is 112, the center of mass of @j,k(x)is 2-j(k + 112) (Exercise 7.8). Therefore, even with non-compactly supported wavelets, we think of the bases as local bases in the sense that the coefficierlt {(f,$I,k)) somehow captures sorne aspect of the behavior of f (z)only near the interval I J , k or near the point 2-j(k 112). In the case of wavelets in two dimensions, we need to consider the coricentration or support of the scaling functions as well. In the Haar case, p(x) is supported on [ O , l ) so that the two-dimensioilal scaling furletions and waveletmu,@(x,y) a n d * ( i ) ( z , g ) , 1: = I , 2, 3 , are supported on the square [0, 1) x [0, 1). Consequently, the scaling functions cDj,k,,kz (z, y) and (i) the wavelet fimctions *j,kl , k z (x, y) are supported on JJ,k, x Ij,k2. The situation is slightly but not substantially different in the other examples considered in this section. As above, we consider y(z) to be concentrated in the interval lo,o - = [-1/2,1/2) and hence c p j . k ( z ) to be concentrated in the interval I , , k = [2-j(k - 1/2),2-j(k 1/2)). We think of the scaling furictiori cD,j,k,,k2 (z,y) as being cori<:entrated on the interval ~ ~ x ,I j , k k2 , ~ , k Z( 2 , y) on the interval Ij,r., x I,sn,, .i2 (x,y) on

+

+

~ :- l i ~

~jZ2,

(3)

the interval l j , k 1 x l j , k , ,and !J!j,k, ,k, (x, y) on the interval I j , k l x I j , k z . Another point of view is to consider the center of mass of the two-dimensional wavelets and scaling functions. This is done in Exercise 7.44. Frequericy Localization: Here we consider the frequency concentration of the scaling functions and wavelets pI,k(x) and II,,,k(x). The question is: What is the best way to do this? If we compute the center of mass of these functions, we quickly run into trouble. Since (at least in all of the examples presented so far) p ( x ) arid ,+(x) are real--valued, the functiorls Ipj.k(?) 1' and (?) l2 are even and hence A

/Y

?j,k(?)12

d~=/T

&.x(r)12 d7 = O

for every j and k. Since in all of our examples F(O) = 1, we car1 safely recognize that @ ( y )l2 resembles a "bump" centered at zero in the frequency domain. However, since q ( 0 ) = 0, we see that in general l $ ( ? ) I 2 will reA

A

7.4. Construction and Examples

195

semble a pair of symmetric bumps in frequency. A proper discussion of frequency localization of thc wavelets should take this into account. A reasonable approach is to consider the bandlimited wavelets as a paradigm for the notion of frequency localization of a wavelet. Specifically, we note that in this case

arid Ihal

4(?)l2= XI-1.-1/21 (7)+ X [ ~ / ~ , L ) ( Y ) .

It follows then that

and that

~ $ ~ , r ( y= ) lx~~ - z ~ , - z ~ - ~ ) ( + Y )X [ ~ ~ - ~ . Z J ) ( Y ) ,

and we define the frequency localization or concentration of the scaling functions y3,k(x) and the wavelets $ j , k (x) to be the intervals [-2j-', 2'-') and [-2j, -2-1-I ) U [2jp1,23) respectively. The accuracy of t#hisparadigm can be verified by computing the Fourier transform of the wavelets and scaliiig fuiictions giver1 in this section. This is the content of Exericse 7.46.

Exercises Exercise 7.39. Prove equation (7.26). Exercise 7.40.

Prove equation (7.28).

Exercise 7.41. Assuming that f (x, y) is C: on R2 and that the scaling function p(t) has compact support on R, show that (7.35) holds.

Exercise 7.42. Show that if f (x, y) is (7.34), then lim llP,f

j-+m

-

C,O on

R 2 and if Pj is given by

f 112 + IIP-3fllz

= 0.

(Hint: The proof is similar to that of Lemma 7.5.)

Exercise 7.43. Show that in each wavelet example of this section (with the exception of the Haar wavelet) the following hold. (a) The scaling function cp(x) is even; that is,

cp(-2)

= cp(x)

(b) The center of mass (see Exercise 7.8) of cp(x) is zero.

196

Chapter 7. Multiresolution Analysis

(c) The wavelet $(z) is symmetric about 112; that is, $(1 - x)

=

+(x).

(d) The center of mass of $(x) is 112.

Exercise 7.44. Suppose that

Show that

Exercise 7.45. Show that if $(x) is a wavelet, then

1$(2jr)l2 = 1 for 3

all y. Exercise 7.46. For each example of a wavelet given in this section, calculate either numerically or analytically its Fourier transform and verify informally that this function is concentrated on the intervals [-I, -112) U [l/2,1).

7.5 Proof of Theorem 7.35 The goal of this section is to prove the first part of Theorem 7.35. This is done in two steps. The first step is Lemma 7.48, which lists three conditions on the wavelet $(z) sufficient to guarantee that { $ J , k ( ~ ) ) J , k E Z forms an orthonormal wavelet basis on R. The second step involves showing that the

7.5. Proof of Theorem 7.35

197

function $(z) given by (7.24) satisfies those three conditions. The proof of the second part of Theorern 7.35 uses the same techniques and is left as an exercise (Exercise 7.50).

7 . 5 1 Suficient Conditions for a Wavelet Basis Definition 7.47. For each j E Z , the wavelet subspace W, is defined by

Lemma 7.48. Suppose that there exists a function +(z) E VI satisfying the following conditions. (a) ( T , @ ( x ) )is a n orthonormal system 0.f translates. I n light of Definition 7.47, this is the same as saying {T,+(x)) is a n orthonormal basis for Wo. (b) (T,@,T, p) = 0 for a11 n, m E Z . Since {T,,p(x)) is a n orthonormal basis for Vo,this i s the same as saying that Vo and Wo are orthogonal subspaces; that is, .Ilf f (x) E Vo and g ( x ) E Wo,then ( f , g ) = 0. This i s often written as Wo IVo. ( c ) Given a function f (z), C: o n R,

QO f ( x ) E SjZE{T,$(x)) = Wo.

is a wavelet orthornormal basis o n R.

Proof: As usual we must show orthonorrnality and completeness. Let us first prove orthonornlality within scales. Let j t Z he fixed. Then

by (a), as requircd. To prove orthonormality between scales, let j, j' t Z with j' < j , and let k , kt E Z be arbit,rary. Since $(z) E Vl, $ o , k / (z) E Vl also (cf. Exercise 7.9). By Definition 7.12(d), $j~,/c'(x)= D ~ ~ / $ o , ~EI5 ( x/ )+1. Since by (b), (qo,,,

( ~ 0 ,= ~ 0)

for all n, rn E Z , it follows that

( D 2 J '$0,n, D2J ( ~ 0 . m= ) ($3,n, ( P j , n l )

for all j , m, n E Z.

= 0,

198

Chapter 7.Multircsolution Analysis

Given f (z)E

4, we know by Lemma 7.15(a) that

Hence for f ( x ) E V,,

Finally, note that by Definition 7.12(a),since j' < j , 4 1 + ~4 and since $ 3 / , k , ( 5 ) E T/n,+l, g j , , k , (x) E V, also. Hence ( g j , k , = 0. Therefore { $ ~ ~ , ~ ( isz an ) )orthonormal ~ , ~ ~ ~ system on R. In order to prove completeness, let f (z) be C: on R. We will show first that (c) implies that for any j E Z , $j/,kl)

To see this. note that

Qjf(x)

=

Pj+l

f

( 2 )-

By (c) (see Remark 7.3),

Therefore,

Pjf (4

7.5. Proof of Theorem 7.35

199

and (7.37) follows. In order to complete the proof, recall that by Lemma 7.16,

For J f Z fixed,

=

Ilf

-

P J +~P-.~f 112

Since for each 3,

& i f(2)

C(f1'$.7.k.)

'd1j.k (lr)>

k.

Therefore by Lemma 7.16,

and { Y ~ . ' J , / ~ ( x )is) ~complete. .~~Z

7 . 5 2 Proof of Theorem 7.35 Let h ( n ) be the scaling filter. Define the wavelet filter g(n) by (7.23) and $ J ( x )E Vl by (7.24). We will show that { $ J , k ( . x ) ) j , k E zis a complete orthonormal system on K. by showing that Lemma 7.48(a)-(c) are satisfied. To prove (a), note that since {T,cp(x)) is an orthonormal system of translates,

200

Chapter 7. Pvlultiresolution Analysis

where we have split the second sum into its even and odd terms, and used the periodicity of m o ( r ) . By (7.26): Iml (?/2)12 lml (?/2 1/2)12 = 1 also and using an argument similar t o the above

+

+

and (a) follows. To prove (b), note that by (7.26), and the orthonormality of {Tnp(x)),

7.5. Proof of Theorem 7.35

201

for all n, !E Z. To prove (c), let f(x) be C,O on R. Since Qof(z)= Plf(z)- Pof(z),and by Lemma 7.15(a), 1

=

f

l

k

l

k

and

f i f ( 2 )=

k

C(f. Po,?+)P " . ~ ( Z ) . k

(7.39) Taking the Fourier transform of both sides of (7.39), we have that

and

Since by Bessel's inequality

the Riesz-Fischer Theorem implies that a,(?) a n d b(y) are functions L2 on [O, 11 in the sense of Lebesgue. By Lemma 7.5, it is enough to show that there is an f2sequence { c ( n ) ) such that (7.42) ( 7 )= ~ Y ) ? ? ( Y ) = m,l(Y/2) $(7/2). If (7.42) did hold, then in light of the definition of Qo, (7.40), and (7.41), E(y) would satisfy

Kf

BY)

Thus, (7.42) would follow if we could find E(7) such that

Chapter 7. Multiresolution Analysis

202

Replacing y by y + 1 in (7.43) and remembering that a(?), b(y), and ?(y) each have period 1, we obtain

Combining (7.43) with (7.44), we obtain the system

(

ml (712) m1(y/2 112)

+

mo;;Pf!/2)

) ( Eh) ) - ( b(y)

a(y/2) a(yI2 + l / 2 )

1.

(7.45)

Since ml(?) =

e-Z7rz(y+l/Z)

mob + 112)

and lmo(y)I2 + Imo(7 + 1/2)12 = 1, the matrix

is unitary for all y

E

R; that is,

Applying this fact to (7.45) gives

so that

It can be verified directly that ?(-(y) has period 1, and since m l ( y ) l 5 1 for all y,that ?(?) is L~ 011 [O, 11 ill the sense of Lebesgue arld Ilence has t 2Fourier coefficients. Therefore, (7.42) holds, (c) is proved, and the result follows from Lemma 7.48.

Remark 7.49. In the course of the preceeding proof, we showed the following facts. (a)

(b) If j

(z))kEz is an ortl~onorrnalbasis for WJ

# j'

then W j IWjl.

(c) For each j E Z,

VjIW j .

7.6. Necessary Properties of the Scaling Function

203

(d) For each j E Z, q+l= V, @ W j . This means that every f ( x ) E y+, can be "split" as f ( z ) = fi (z) f 2 (z), where f (z) E V, and f 2 (x) E Wj . By (c), (f1 , f 2 ) = 0. This fact is t o be compared with the Splitting Lemma (Lemma 5.16) for the Haar system.

+

(e) Every f (z), L2 on R, can be written as a sum

where fj (x) E written as

W'

and by (Is),

(fj,fj1)

=

O if j

# j'.

This is usually

Exercises Exercise 7.50. Prove the second part of Theorem 7.35.

7.6 Necessary Properties of the Scaling Function In this ~ e c t i o nwe , ~ derive sorne propcrtics that thc scaling function, c p ( ~ ) , and the wavelet function, $(+), for a given MRA must satisfy. Throughout the section, it will be assumed that the scaling function is both L' and L2 on R and that the wavelet function defined by (7.24) is also L1 on R.

Theorem 7.51.

Proof: Let f (z) be given so that

I f 11

A

= 1, f (7) is continuous and sup-

ported in the interval [-R, R] for some R

> 0. By Theorem

3.40,

-

2 ~ h material e in this section is adapted from Daubechies, T e n Lectures on Wavelets, Society for Industrial and Applied Mathematics (1992) p. 144.

204

Chapter 7. Multiresolution Analysis

By Parseval's formula,

Since (2 j I 2 e 2 T i k 2 p " y ) k , z is a complete orthonormal system on the interval [-2j-', 2jP1], then as long as 2j-' > R, the above sum is the sun1 of the squares of the Fourier coefficients of the period 23' extension of the function f^(o)@(2-iy). Therefore, by thc Planchcrcl formula for Fourier series.

Sincc cp(z)is L1 on R, @(?) is continuous on R by the Riemann-Lebesgue Lemma (Theorem 3.9). It follows that

un,iforrn,ly on [-R, R]. Therefore, by Theorem 1.40(b),we can take the limit under tlie integral sign and conclude that

Ilf 1 ;

=

lim IPjf l l 2

3-rn

Hence I@(O) I = 1, and since p(z) is L1,

7.6. Necessary Properties of the Scaling Function

205

Corollary 7.52.

Proof: Since @(y)= mo(y/2) @(y/2),where 7 ~ (y) ~ 0is defined by (7.8),and since by (7.46), @(0)# 0, mo(0) = 1. Since by (7.25)

and since by the ortl-lonormality of {Tkcp(z)),

$(o)

mo(l/2) = 0, and hence on R.

= 0. Therefore,

(7.47) holds since $(z) is L'

Corollary 7.53. G(n)= 0 for. all irrtegers n # 0.

Proof: Since {T,,p(z)) is an orthornorrnal system of translates, Leinma 7.4 says that 7L

for all y E R. Letting y

= 0,

By Theorem 7.51, Ig(0) l2

=

this gives

1 so that

and (7.48) follows.

Corollary 7.54.

+

Proof: Note first that the function C , p(z n) is L1 on [0,1) and has period 1. By Corollary 7.53, @(0) = 1 and @ ( k )= 0 for all integers k # 0.

206

Chapter 7. Multiresolution Analysis

Therefore for each k E Z,

The only function with period 1 and Fourier coefficents equal t o b ( k ) is the function that is identically 1 on [0, 1). Therefore, (7.49) follows.

7.7 General Spline Wavelets In Section 7.3.2, we studied an MRA in which the spaces VJ consisted of continuous piecewise linear functions. The wavelet $(z) associated with this MRA is also piecewise linear and continuous, but is not compactly supported. However, since it has rapid decay a t infinity, it is very small outside a relatively small interval. As a result, the piecewise linear wavelet expansion has most of the advantages of the Haar expansion and does a better job of representing smooth functions. In particular, any partial sum of the piecewise linear wavelet series of a smooth function is continuous. We would like to do even better. In this section, we will construct wavelets that are smooth and piecewise polynomial. Specifically for each n E N, we will construct a wavelet that is C7'-I on R and that is a piecewise polynomial of degree n. To do this, we will require some preliminary properties of piecewise polyrlomial functions and specifically of spline functions.

7.7.1 Basic Properties of Spline Functions Definition 7.55.

Let B O ( x )= X [ 1 / 2 , 1 / 2 ~( x ) , and for n E N , define x+1/2

B r L ( x )= BTL-1* B o ( x ) =

S

B , l ( t ) dt.

(7.50)

x- 1 / 2

Thc function B,(x) zs callcd thc B-splinc (or spline) of ordcr n. For n E Z', define ( x ) by (7.51) & ( x ) = B,(x - ( n 1 ) / 2 ) .

g,

+

7.7. General Spline Wavelets

(k) Bl (s)= Bo * Bo(:) = ( 1

Example 7.56.

-

207

1x1)X [ - l , l j ( x )

x

€

[-3/2, -112)

otherwise

1 i 2+,Ix 1 4

+

B3 (4 = B2 * B o ( z )= =

x

[ 3 / 21 / 2 )

otherwise,

2-1/2

Exercise 7.64.

Bob) =

-

B

)

-

(4,

X[0>1]

{

2

z 0

zE[O,l),

: E , 2 ] ,

otherwise.

-

B3( z ) = Exercise 7.64.

Lemma 7.57.

( a ) B,(z) i s supported in [-(n

[O,n+11.

-

T h e functions B, (x)and B,(x) satsify the following properties.

+ 1)/2, ( n + 1)/2], and

&(x)

is supported in

( b ) B, ( x ) and

-

En(z)are cnP1 o n R.

(c) B,(x) is equal t o a degree n polynomial o n intervals of the f o r m [ k ,k k E Z.

gn(y)=

(y)

sin(~y)

n+l

-

A

, and B,(-y)

=

e-Ti'nli'

(?)

sin(7i.y)

+ 11,

n+ 1

.

Proof: (a) Exercise 7.65.

(b) The proof is by induction on n. Clearly B l ( x ) is continuous on R. By (7.50), B,,(x) = B, - (t) d t . By the Fundamental Theorem of Calculus, BA,(x) = B n P l ( x 112) - B,,-l(z - 112).

J~!T~;:

+

By the induction hypothesis, B,,-l (x) is CnP2on R. Therefore, B, (z) is CIL-1 on R. (c) The proof -is by induction on n. Clearly the result holds for B ~ ( X ) . Now assume that B,,(z) = pk(x) on [k, k + 11, k E Z, where pk (x) is a degree n polynomial. Fix k, and let x E [k,k I]. By Exercise 7.63,

+

Since the indefinite integral of a degree n polynomial is a degree n polynomial, we are done.

+1

(d) Exercise 7.66.

7.72 Spline Multiresolution Analyses Given n E N, define the degree n s p l i n e m u l t i r e s o l u t i o n a n a l y s i s by

and for j E Z, define

Note that any function f (z) E Vois Cn-' on R and is a degree n polynomial on each interval k E Z. Any function f (z) E V, is Cn-' on R and is a degree n polynomial on each interval I j , k , I% E Z.

7.7. General Spline Wavelets

209

We need to verify that { V , ) j c z is an MRA. To verify Definition 7.12(a), we need the following lemma.

Lemma 7.58. For each n E N ,

B,(z) satisfies n

E n

where m o ( y ) = 2-'"'

(1

Proof: If n = 0, then

(7) = mo (712) En (74121,

+ e-2"")nf

(7.54)

l.

Bo(z)= Xlo,ll

(x), and by Exercise 5.18,

Taking Fourier transforms gives

A

B,,

A

Since (?) = (go ("r))"+' (Exercise 7.63 and the Convolution Theorem), raising both sides of ( 7 . 5 5 ) to the n 1 power gives (7.54).

+

Since

and Definition 7.12 (a) follows. The verification of Definition 7.12(e) is contained in the following lemma. 1

Lemma 7.59. thonormal basis for

Thcrc cxists a function F(x) such that {T,@(X)}

is a n or-

Vi.

Proof: In light, of T,emrna 7.7, it, will he sufficient t,o show that there exist constants A, B > 0 such that for all y E R,

To see this, note that

210

Chapter 7. Multiresolution Analysis

A

Since CI:(En(?+6)l2 has period 1, it is enough t o show that it is boundcd above and away froni zero on the interval [-1/2,1/2]. For y E [-1/2,1/2],

and

To verify Definition 7.12(b), we require the following lemma. Fix n E N , let f (z) be Da.r~dlirruiLedwilh b a ~ ~ d l i n z R i t > 0, (Definition 3.4'7) and suppose that f (7) is C' on R. 'L'hen

Lemma 7.60.

A

( f , D~~ &) D~~TxEn (x)= f (x)

lim J'S

Proof: Applying Parseval's formula, wc obtain

(f,D~~T, B,)

=

/f

(x) ~

2

)

dx

R

=

~ ( y~) ~ ( 2 -2 j j /~2 e) 2 r i ( 2 - " y ) k d,Y

7.7. General Spline Wavelets

211

Recall that {2-312 e-2TL2-'7 is an orthonormal basis on [ - 2 k 1 , 2,1-']. This means that as long as 2J-I > R, ( f , D2J is the kth Fburier A

fi;) g,,(2-j;).

coefficient of the period 2i extension of the fur~ction

C f^(?+ 2%) 5,(2-j1. + k ) .

Hence

..

-

2i/2

(7.57)

k

In light of (7.57), taking the Fourier transform of the left side of (7.56) gives

The term in the sllrri for m = 0 is

A

and in fact, this term converges to f ( 7 )as follows.

-

A

since limj+, that

As long as 2"' that

11- B , ( 2 - 9 ) l2

=

0 uniforrr~lyon [-R, R].It remains to show

> R, the supports of each term in thc sum arc disjoint so

Chapter 7. Multiresolution Analysis

212

-

A

y(f^(~)

Since iscontinuous and compactly supported, it and also Bn(?) are L" on R. Therefore,

But since

1

5 jj

03

c

m=l

( T

1 (~ 1/2))2n+2

.2-.1

R

J-2-,n

s i ~ l ~ (" ~+ y~dy. )

Since { 2 j f 1 R XI-2.1R,2-J n]( x ) ) , is~ an ~ ~approximate identity arid si~lce ( ~ y is) a, continuous function that vanishes at y = 0: Theorem 2.33 says that ,in1 2~ l 2 - I R sin2"+ 2 ( ~ yd )y

,i+"o

Thus,

2

3R

= 0.

7.7. General Spline Wavelets

213

asj-m. To complete the verification of Definition 7.12(b), let f (z) be C: 011 R. Let F > 0. By Corollary 2.37(b), and