Introduction to Partial Differential Equations with MATLAB

David

Applied and Numerical Harmonic Analysis Series Editor John J. Benedetto University of Maryland Editorial Advisory Board Akram Aldroubi N I H , Biomedical Engineering/ Instrumentation Ingrid Daubechies Princeton University Christopher Heil Georgia lnstitute of Technology James McClellan Georgia lnsitute of Technology Michael Unser NIH, Biomedical Engineering1 lnstrumentation M. Victor Wickerhauser Washington University

Douglas Cochran Arizona State University Hans G. Feichtinger University of Vienna Murat Kunt Swiss Federal lnstitute of Technology, Lausanne Wim Sweldens Lucent Technologies Bell Laboratories Marfin Vetterli Swiss Federal lnstitute of Technology, Lausanne

Applied and Numerical Harmonic Analysis J.M. Cooper: Introduction to Partial Differential Equations with MATLAB (ISBN 0-8176-3967-5) C.E. D'Attellis and E.M. Fernandez-Berdaguer: Wavelet Theory and Harmonic Analysis in Applied Sciences (ISBN 0-8 176-3953-5)

H.G. Feichtinger and T. Strohmer: Gabor Analysis and Algorithms (ISBN 0-8176-3959-4) T.M. Peters, J.H.T. Bates, G.B. Pike, P. Munger, and J.C. Williams: Fourier Transforms and Biomedical Engineering (ISBN 0-8 176-394 1- 1) A.I. Saichev and W.A. Woyczynski: Distributions in the Physical and Engineering Sciences (ISBN 0-8176-3924-1)

R. Tolimierei and M. An: Time-Frequency Representations (ISBN 0-81763918-7)

G.T. Herman: Geometry of Digital Spaces (ISBN 0-8176-3897-0) A. Prochazka, J. Uhlii, P.J.W. Rayner, and N.G. Kingsbury: Signal Analysis and Prediction (ISBN 0-8176-4042-8) J. Ramanathan: Methods of Applied Fourier Analysis (ISBN 0-8 176-3963-2)

A. Teol is: Computational Signal Processing with Wavelets (ISBN 0-81763909-8) W.O. Bray and C.V. Stanojevic: Analysis of Divergence (ISBN 0-81764058-4)

G.T. Herman and A. Kuba: Discrete Tomography (ISBN 0-8176-4101-7) J.J. Benedetto and P.J.S.G. Ferreira (Eds.): Modern Sampling Theory (ISBN 0-8176-4023-1) P. Das, A. Abbate, and C. DeCusatis: Wavelets and Subbands (ISBN 08176-4136-X)

L. Debnath: Wavelet Transforms and Time-Frequency Signal Analysis (ISBN 0-8176-4104-1) K. Grochenig: Foundations of Time-Frequency Analysis (ISBN 0-81764022-3) D.F. Walnut: An Introduction to Wavelet Analysis (ISBN 0-8176-3962-4)

David F. Walnut

An Introdution to Wavelet Analysis With 88 Figures

Birkhauser Boston Basel Berlin

D a v i d F. W a l n u t Department of M a t h e m a t i c a l Sciences George M a s o n University Fairfax, V A 22030 USA

Library of Congress Cataloging-in-Publication Data Walril~t,,David 12. Ari iritrod~lct~ion t,o wavelet analysis / David I". Walnut, p. crri. (Applied and n~irrlc!ricalllarri~o~lir itrlalysis) Iric111dc:sl)ibliug~-apl~iral refcl-eiices and indt:x. ISBN 0-81763-3962-4 (alk. paper) 1. Wavelets (Matliemat,ic:s) I. Title. 11. Series. QA403.3 .W335 2001 515'.2433 dc21 2001025367 CIP

Prir~t~ed 011 acid-f'rei: paper. @ 2002 Hirkhii~iserBost,ori

Birkhauser

All rig lit,^ reserved. This work irlay riot. I J ~t,la~lslat,eclor copicd ill whole or in part wit.tlout t,l~r:wl.it,t,t!~~ 1~r.1-lllission of t,he p~lblisher(Birkki&liser Bost,ol~,c/o Sprir~gcr-VcrlagNew York, Irir., 175 Fift,h Avenue, New York, NY 10010, USA), except for brief excerpts in corir~ection wit,11 reviews or scholarly analysis. lJse ill conncctiori with any form of information storage and rctricval, elect,ronic adapt,at,iori, cornplit,er soft,ware, or t)y sirnilar or dissirnilar methodology riow kriown or liereafter developed is forbidden. T h c 11sc of gelirral descript,ivo names, t,radc rlanles, tradcmarks, etc., i11 this publication, oven if t,l~eforrrier are not (:specially ide~~t.ified, is not t,o 1 ~ :t,a.kr~las a, sign that silcli ilalnes, as ulltlerstood l)y tlie Trade Marks anti Mcrcharldise Ma1.k~Act,, rrlay accordingly be used freely by anyoni:.

IS13N 0-8176-3962-4 ISBN 3-7643-3962-4

SPIN 10574019

Product,io~~ marlageti t)y Louise Farkas; riian~ifact,~~ring sl~pa.visedt?y .Jacqui Ashri Typc:sc:t, I)y t,lie aiitlior ill LaTeX2c. Printeti and I)oli~id1,y Edwards Rrot,hers, Inr., Arin Artlor, MI. Pririt,ed i r ~t,lit: l1liitc:d St,at,es of America.

L3irkliiiiist:r Host,o~l Basel

Berlin

A rrr ernbcr of /3erlr,lsn1,ur~nSpringer Science+ B7rs,irir ss hI(:d./:nG,nl.bH

To my parents

and to Megan

Unless the LORD builds the house, its builders labor in vain. -

Psalm 1 2 7 ; l a (NIV)

Contents xiii

Preface

I 1

Preliminaries

1

F'linctions and Convergence 1.1 Functions . . . . . . . . . . . . . . . . . . 1.1.1 Bounded (L") Functions . . . . . 1.1.2 Integrable (L1) Functions . . . . . 1.1.3 Square Integrable (L2) Functions . 1.1.4 Differentiable (Cn) Functions . . . 1.2 Convergence of Sequences of Functions . . 1.2.1 Numerical Convergence . . . . . . 1.2.2 Pointwise Convergence . . . . . . . 1.2.3 Uniform (L") Convergence . . . . 1.2.4 Mean ( L 1 )Convergence . . . . . . 1.2.5 Mean-square (L2) Convergence . . 1.2.6 Interchange of Limits and Integrals

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

2 Fourier Series

2.1

Trigonometric Series . . . . . . . . . . . . . . . . . . . . . .

2.1.1 Periodic Functions . . . . . . . 2.1.2 The Trigonometric System . . 2.1.3 The Fourier Coefficients . . . . 2.1.4 Convergence of Fourier Series . 2.2 Approximat.e Identities . . . . . . . . . 2.2.1 hlotivation from Fourier Series 2.2.2 Definition and Examples . . . . 2.2.3 Convergence Theorems . . . . . 2.3 Generalized Fourier Series . . . . . . . 2.3.1 Orthogonality . . . . . . . . . . 2.3.2 Generalized Fourier Series . . . 2.3.3 Complet.eness . . . . . . . . . . 3 The 3.1 3.2 3.3

. . . .

. . . .

. . . .

. . . .

. . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . .

Fourier Transform Motivation and Definition . . . . . . . . . . . . . . . . . . . Basic Properties of the Fourier Transform . . . . . . . . . . Fourier Inversion . . . . . . . . . . . . . . . . . . . . . . . .

3 3

3 3 6 9 11 11 13 14 17 19 21 27 27 27 28 30 32 37 38 40 42 47 47 49 52

59 59 63 65

Contents

viii

3.4 3.5 3.6 3.7 3.8 3.9

Coilvolutior~ . . . . . . . . . . . . . . . . . . . . . . Plancherel's Formula . . . . . . . . . . . . . . . . . The Fourier Trarlsfornl for L~ Functions . . . . . . Smoothiless versus Decay . . . . . . . . . . . . . . Dilation, Translation, ancl Modulation . . . . . . . Bandlirnitetl Functiorls and the Sarrlpling Formula.

. . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . .

4 Signals and Systems 4.1 Signals . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Systerris . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Causality and Stability . . . . . . . . . . . . 4.3 Periodic Signals a r ~ dthe Discrete Fourier Transform 4.3.1 The Discrete Fourier Transform . . . . . . . . 4.4 The Fast Fourier Transform . . . . . . . . . . . . . . 4.5 L2 Fourier Series . . . . . . . . . . . . . . . . . . . .

I1

. . . . . . . .

. . . . . . . .

. . . . . . . . . . . .

68 72 75 76 79 81

87 88 90 95 101 102 107 109

The Haar System

5 T h e Haar System 5.1 Dyadic Step Functions . . . . . . . . . . . . . . . . . . . . . 5.1.1 The Dyadic Intervals . . . . . . . . . . . . . . . . . . 5.1.2 The Scale j Dyadic Step Functions . . . . . . . . . . 5.2 The Haar System . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 T h e H a a r Scaling Functions and the Haar Functions . . . . . . . . . . . . . . . . . . . . . 5.2.2 Orthogonality of the Haar System . . . . . . . . . . 5.2.3 The Splitting Lemma . . . . . . . . . . . . . . . . . 5.3 Haar Bases on [O, 11 . . . . . . . . . . . . . . . . . . . . . . . 5.4 Comparison of Haar Series with Fourier Series . . . . . . . . 5.4.1 Representation of Functions with Small Support . . 5.4.2 Behavior of Haar Coefficients Near J u m p Discontinuities . . . . . . . . . . . . . . . . . . 5.4.3 Haar Coefficients and Global Smoothness . . . . . . 5.5 Haar B a s e s o n R . . . . . . . . . . . . . . . . . . . . . . . . 5.5.1 The Approximation and Detail Operators . . . . . . 5.5.2 The Scale J Haar System on R . . . . . . . . . . . . 5.5.3 The Haar System on R . . . . . . . . . . . . . . . .

115 115 115 116 117

6 The Discrete Haar Transform 6.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.1 The Discrete Haar Transform (DHT) . . . . . . . 6.2 The DHT in Two Dimensions . . . . . . . . . . . . . . . 6.2.1 The Row-wise and Column-wise Approximations and Details . . . . . . . . . . . . . . . . . . . . .

141

117 118 120 122 127 128 130 132 133 134 138 138

. . 141 . . 142 . . 146 . . 146

Contents

6.2.2 6.3 Iinage 6.3.1 6.3.2 6.3.3

I11

ix

The DHT for Matrices . . . . . . . . . . . . . . . . . Analysis with the DHT . . . . . . . . . . . . . . . . . Approximation and Blurring . . . . . . . . . . . . . Horizontal, Vertical, and Diagonal Edges . . . . . . "Naive" Image Compression . . . . . . . . . . . . . .

Ort honormal Wavelet Bases

147

150 151 153 154

161

7 Multiresolution Analysis 7.1 Orthonormal Systems of Translates . . . . . . . . . . . . . . 7.2 Definition of Multiresolution Analysis . . . . . . . . . . . . 7.2.1 Some Basic Properties of MRAs . . . . . . . . . . . 7.3 Examples of Multiresolution Analysis . . . . . . . . . . . . . 7.3.1 The Haar MRA . . . . . . . . . . . . . . . . . . . . . 7.3.2 The Piecewise Linear MRA . . . . . . . . . . . . . . 7.3.3 The Bandlimited MRA . . . . . . . . . . . . . . . . 7.3.4 The Meyer MRA . . . . . . . . . . . . . . . . . . . . 7.4 Construction and Examples of Orthonorrnal Wavelet Bases . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1 Examples of Wavclct Bases . . . . . . . . . . . . . . 7.4.2 Wavelets in Two Dimensions . . . . . . . . . . . . . 7.4.3 Localization of Wavelet Bases . . . . . . . . . . . . . 7.5 Proof of Theorem 7.35 . . . . . . . . . . . . . . . . . . . . . 7.5.1 Sufficient Conditions for a Wavelet Basis . . . . . . . 7.5.2 Proof of Theorem 7.35 . . . . . . . . . . . . . . . . . 7.6 Necessary Properties of the Scaling Function . . . . . . . . 7.7 General Spline Wavelets . . . . . . . . . . . . . . . . . . . . 7.7.1 Basic Properties of Spline Functions . . . . . . . . . 7.7.2 Spline Multiresolution Analyses . . . . . . . . . . . .

163 164 169 170 174 174 174 179 180

8 The Discrete Wavelet Transform 8.1 Motivation: From MRA t o a Discrete Transform . . . . . . 8.2 The Quadrature Mirror Filter Conditions . . . . . . . . . . 8.2.1 Motivation from MRA . . . . . . . . . . . . . . . . . 8.2.2 The Approximation and Detail Operators and Their Adjoints . . . . . . . . . . . . . . . . . . . . . 8.2.3 The Quadrature Mirror Filter (QMF) Conditions . . 8.3 The Discrete Wavelet Transform (DWT) . . . . . . . . . . . 8.3.1 The DWT for Signals . . . . . . . . . . . . . . . . . 8.3.2 The DWT for Finite Signals . . . . . . . . . . . . . . 8.3.3 The DWT as an Orthogonal Transformation . . . . 8.4 Scaling Functions from Scaling Sequences . . . . . . . . . . 8.4.1 The Infinite Product Formula . . . . . . . . . . . . . 8.4.2 The Cascade Algorithm . . . . . . . . . . . . . . . .

215 215 218 218

185 186 190 193 196 197 199 203 206 206 208

221 223 231 231 231 232 236 237 243

x

Contents 8.4.3

The Support of the Scaling Functio~i. . . . . . . . . 245

9 Smooth. Compactly Supported Wavelets 249 . . . . . . . . . . . . . . . . . . . . . . . 9.1 Vanishing Moments 249 Vanishing Moments and Smoothness . . . . . Vanishing Moments and Approximation . . . Vanishing hilomcnts and thc Reproduction of Polynomials . . . . . . . . . . . . . . . . . 9.1.4 Equivalent Conditions for Vanishing h!Iornent.s The Daubechies Wavelets . . . . . . . . . . . . . . . 9.2.1 The Daubechies Polynomials . . . . . . . . . 9.2.2 Spectral Factorization . . . . . . . . . . . . . Image Analysis with Smooth Wavelets . . . . . . . . 9.3.1 Approximation and Blurring . . . . . . . . . 9.3.2 "Naive7' Image Compression with Smooth Wavelets . . . . . . . . . . . . . . . .

9.1.1 9.1.2 9.1.3

9.2

9.3

IV

. . . . 250 . . . .

254

. . . . 257 . . . . 260

. . . . 264 . . . . 264 . . . . 269

. . . . 277 . . . . 278

. . . . 278

Other Wavelet Construct ions

10 Biort hogonal Wavelets 10.1 Linear Independence and Biorthogonality . . . . . . . . . . 10.2 Riesz Bases and the Frame Condition . . . . . . . . . . . . 10.3 Riesz Bases of Translates . . . . . . . . . . . . . . . . . . . 10.4 Generalized Multiresolution Analysis (GMRA) . . . . . . . 10.4.1 Basic Properties of GMRA . . . . . . . . . . . . . . 10.4.2 Dual GMRA and Riesz Bases of Wavelets . . . . . . 10.5 Riesz Bases Orthogonal Across Scales . . . . . . . . . . . . 10.5.1 Example: The Piecewise Linear GMRA . . . . . . . 10.6 A Discrete Transform for Biorthogonal Wavelets . . . . . . 10.6.1 Motivation from GMRA . . . . . . . . . . . . . . . . 10.6.2 The QMF Conditions . . . . . . . . . . . . . . . . . 10.7 Compactly Supported Biorthogonal Wavelets . . . . . . . . 10.7.1 Compactly Supported Spline Wavelets . . . . . . . . 10.7.2 Symmetric Biorthogonal Wavelets . . . . . . . . . . 10.7.3 Using Symmetry in the DWT . . . . . . . . . . . . . 11 Wavelet Packets

289 289 290 293 300 301 302 311 313 315 315 317 319 320 324 328

335 11.1 Motivation: Completing the Wavelet Tree . . . . . . . . . . 335 11.2 Locs.1iza.t.ion of Wavelet Packets . . . . . . . . . . . . . . . . 337 11.2.1 Time/Spatial Localization . . . . . . . . . . . . . . . 337 11.2.2 Frequency Localization . . . . . . . . . . . . . . . . 338 11.3 Orthogonality and Completeness Properties of Wavelet Packets . . . . . . . . . . . . . . . . . . . . . . . . 346 11.3.1 Wavelet Packet Bases with a Fixed Scale . . . . . . 347

xii

Conterlts B .1.2 Wavelets with Rational Noninteger Dilation Factors . . . . . . . . . . . . . . . B . 1.3 Local Cosine Bases . . . . . . . . . . . . . . B . 1.4 The Co~ltinuousWavelet Transform . . . . B . 1.5 Non-MRA Wavelets . . . . . . . . . . . . . B . l.G Multiwavelets . . . . . . . . . . . . . . . . . B.2 Wavelets in Other Domains . . . . . . . . . . . . . B.2.1 Wavelets on Intervals . . . . . . . . . . . . B.2.2 Wavelets in Higher Dimensions . . . . . . . B.2.3 The Lifting Scheme . . . . . . . . . . . . . B.3 Applications of Wavelets . . . . . . . . . . . . . . . B.3.1 Wavelet Denoising . . . . . . . . . . . . . . B.3.2 Multiscale Edge Detection . . . . . . . . . . B.3.3 The FBI Fingerprint Compression Standard

C References Cited in the Text Index

. . . . . 434 . . . . . 434 . . . . . 435 . . . . . 436 . . . . . 436 . . . . . 437 . . . . . 437 . . . . . 438 . . . . . 438 . . . . . 439 . . . . . 439 . . . . . 439 . . . . . 439

441

Preface These days there are dozens of wavelet books or1 the market, sonie of which are destiried t o be classics in the field. So a natural question to ask is: Why another one? I11 short, I wrote this book t o slipply tlie particular rieeds of students in a graduate course on wavelets that I have taught several tirnes since 1991 a t George Mason University. As is typica.1 with sllcli offerings, the course drew an audience with widely varying backgrounds and widely varying expectations. The difficult if not inlpossible task for me, the instructor, was t o present the beauty, usefiilriess. arid matliernatical depth of the sul~jectt o such an auclience. It would be insaiie t o claiin that I have been entirely successful in this task. However, through much trial ant1 error, I have arrived a t sorrle basic principles that are reflected in the stri~ctilreof this book. I believe that this makes this book distinct frorn existiilg text,s. and I hope that others may find the book useful. (1) Consistent assumptions of mathematical preparation. In sonle ways, the subject of wavelets is deceptively easy. It is not difficult to understand and implement a discrete wavelet trarlsforni and from there tlo analyzc and process signals arid irriages with great success. However, the underlying itleas and conrlectiorls that rrlake wavelets such a fascinating subject require some considera1)lc rriathematical sophistication. There have bceil some excellent books written on wavelets cnlphasizirlg their eleinelitary nature (e.g., Kaiser, A I;i-iendlv Glzide to Wavelt:ts; Strang and Nguyen, Wavelets and Filter Banks; Walker, P r i m ~ ron Wavelets and their Scientific Applications: Frazier, Irltroduction to Wavelets through Linear Algebra; Nievergelt, Ilitzvelets Made Easy; Ateyer, Wavelets: Algorithms and Applications). For rriy own purposes. such texts required quite a bit of "filling in the gaps" in order to make some conrlections and to prepare the student for rnore advanced books and research articles in wavelet theory. This book assuriies an upper-level undergraduate semester of advanced calculus. Sufficient preparation would corrle from, for example, Chapters 1-5 of Buck, Advanced Calculus. I have tried very hard not t o depart from this assumption a t any poirit in the book. This has required at times sacrificing elegance and generality for accessibility. However, all proofs are completely rigorous and conta.in the gist of the more general argunient. In this way, it is hoped that the reader will be prepared to tackle niore sophisticated books and articles on wavelet theory. (2) Proceeding from the continuous to the discrete. I have always found it more meaningful and ultirnately easier to start with a presenta-

xiv

Preface

tion of wavelets arid wavelet bases in the continuous dornain and use this to motivate the discrete theory, even thougli the discrete theory liarlgs together in its own right and is easy to understaiid. This can he frustrating for the student whose priniary int,erest is ill applications, but I believe that a better understanding of applications can ultirrlately be achieved by doing things in this order.

(3) Prepare readers to explore wavelet theory on their own. Wavelets is too broad a subject to cover i ~ai single book and is iriost interesting to study when the students have a particular interest in what they are studying. 111 clloosirig what to include in the book, I have tried t o ensure t,hat students are equipped to pursue more advanced topics on their own. I have included an appendix called Excursions in Wavelet Theory (Appendix B) that gives sorrie guidance toward what T consider to he t,he iriost readable articles on sorrle selected topics. The suggested topics in this appendix ca,rl also be used as the basis of serrlester projects for the students.

Structure of the Book The book is divided into five part,s: P~elirni7~n7..ies, Thc Hu,nr. Systcrrl, hfultiresol~~tion, Ar~nlysisand Orthonormal Wavelet Bases, Other Wavelet Constr.uctions, and Applications. Preliminaries Wavelet theory is really very liarcl to appreciate outside tlio context of t,hc language and ideas of Fourier Analysis. Chapters 1-4 of tlie book provide a background in sorrie of these ideas arid include cverytl.iing that is subsequerltly used in the text. These chapters are designed to be more than just a reference but less tllan a 'bbook-witliin-a-book" or1 Fourier analysis. Depenclirig on the background of the reader or of the class in which t,liis book is being used, these chapters are intended to t ~ edipped into eitllcr sliperficially or in detail as appropriate. Naturally there are a great rliarly books on Fourier analysis that cover the same rriaterial better and rnore thoroughly than do Chapters 1-4 and a t the sarne level (more or less) of Inathenlatical sophistication. I will list some of my favorites below. Walker, Fourier Analysis; Ka~nmler,A First Course in Fourier Analysis: Churchill and Brown, h u r i e r Series a i d Boundary Value Problems: Dym and McKean, Fourier Series a,nd Integrals; Korner, Fourier Analysis; and Benedetto, Harmonic Analysis and Applications. The Haar System Chapters 5 and 6 provide a self-contained exposition of the Haar systern. the earliest example of an orthor~orrrialwavelet basis. These chapters could

Preface

xv

be presented as is in a course on a d v a ~ ~ c ecalculus, d or an undergraduate Fouricr analysis coursc. In the context of the rest of the book, these chapters are designed to motivate the search for rrlore general wavelet bases with different properties. and also t o illustrate some of the more advanced concepts such as multiresolutiorl analysis that are used throughout the rest of the book. Chapter 5 contains a description of the Haar basis on [O, 11 and on R, and Chapter 6 shows how t o implernerlt a discrete version of the Haar basis in one and two dimensions. Some exa~rlplesof inlages analyzed with the Haar wavelet are also included.

Ort honor~nalWavelet Bases Chapters 7-9 represent the heart of the book. Chapter 7 contains an exposition of the general notion of a multiresolutiorl analysis (MRA) together with several examples. Next, we describe the recipe that gives the construction of a wavelet basis from an MRA, and then construct corresponding cxarrlples of wavelet orthonorinal bases. Chapter 8 describes the passage from the continuous domain to the discrete domain. First, properties of MRA are the11 used to niotivate and define the quadrature mirror filter (QMF) conditions that any orthonormal wavelet filter must satisfy. Then the discrete wavelet transfornl (DWT) is defined for infinite signals, periodic signals, and for finite sets of data. Finally the techniques used to pass from discrete filters satisfying the QhlF conditions to continuously defined wavelet functions are described. Chapter 9 presents the cor~struction of compactly supported orthornomal wavelet bases due to Daubechies. Daubechies's a.pproa.ch is motiva.ted by a, lengthy disclissiori of the importance of vanishing moments in the design of wavelet filters.

Other Wavelet Constructions Chapters 10 and 11 contain a discussion of two inlportarlt variations on the theme of the const,ri~ct,ion of orthonormw,l wavelet, ba.ses. The first. in Chapter 10, shows what happens wlleri you consider nonorthogonal wavelet systems. This chapter contains a discussion of Riesz bases, and describes the serni-orthogonal wavelets of Chui and Wang. as well as the notion of dual MRA and the fully biorthogonal wavelets of Daubechies. Cohen. arid Feauveau. Chapter 11 discusses wavelet packets. another natural variation on orthnorrnal wavelet bases. The motivation here is to consider what happens to the DWT when the '.full wavelet tree" is conlputed. JVavelet packet functions are described, their time and frequency localizatio~iproperties are discussed, and necessary and sufficierit conditions are give11 u n d e ~which a collection of scaled and shifted wavelet packets constitutes an orthonormal basis on R. Finally, the notion of a best basis is described. and the so-called best basis algoritliln (due to Coifman and Wickerhauser) is given.

xvi

Preface

Applications Many wavelet books have been written emphasizing applications of the theory, most notably, Strang and Nguyen, Wavelets and Filter Banks, and Mallat's comprehensive, A Wavelet Tour of Signal Processing. The book by Wickerhauser, Applied Wavelet Analysis from Theory to Software, also contains descriptions of several applications. The reader is encouraged to consult these texts and the references therein to learn more about wavelet applications. The description of applications in this book is limited to a brief description of two fundamental examples of wavelet applications. The first, described in Chapter 12, is to image compression. The basic components of a transform image coder as well as how wavelets fit into this picture are described. Chapter 13 describes the Beylkin-Coifman-Rokhlin (BCR) algorithm, which is useful for numerically estimating certain integral operators known as singular integral operators. The algorithm is very effective and uses the same basic properties of wavelets that make them useful for image compression. Several examples of singular integral operators arising in ordinary differential equations, complex variable theory, and image processing are given before the BCR algorithm is described.

Acknowledgments I want t o express my thanks t o the rnany folks ~ 1 . 1 0made this book possible. First and foremost, I want to thank my advisor and friend John Benedetto for encouraging me to take on this project and for graciously agreeing to publish it in his book series. Thanks also to Wayne Yuhasz, Lauren Schultz, Louise Farkas, and Shosharina Grossman at Birkhauser for their advice and support. I want to thank Margaret Mitchell for LaTeX advice and Jim Houston and Clovis. L. Tondo for modifying some of the figures to make them more readable. All of the figures in this book were created by me using MATLAB and the Wavelet ToolBox. Thanks to the Mathworks for creating such superior products. I would like also t o thank the National Science Foundation for its support and to the George Mason University Mathematics Department (especially Bob Sachs) for their constant encouragement. I also want to thank the students in my wavelets course who were guinea pigs for an early version of this text and who provided valuable feedback on ~rganiza~tion and found numerous typos in the text. Thanks t o Ben Crain, James Holdener, Amin Jazaeri, Jim Kelliher, Sami Nefissi, Matt Parker, and Jim Tirnper-. I also want t o thank Bill Heller, Joe Lakey, and Paul Salamonowicz for their careful reading of the text and their useful comments. Special thanks go to David Weiland for his willingness to use the manuscript in an u~ldergraduatecourse at Swarthmore College. The book is all the better

Preface

xvii

for his insights, and those of the unnamed students in the class. I want give special thanks to my Dad, with whom I had many conversations about book-writing. He passed away suddenly while this book was in production and never saw the finished product. He was pleased and proud to have a ~ ~ o t h published er author i11 the family. He is greatly missed. Finally, I want t o thank my wife Megan for her constant love and support, and my delightful children John and Genna who will someday read their names here and wonder how thcir old man actually did it.

Fairfax, Virginia

David F. Walnut

Albrecht Diirer (1471-1528), Melencholia I (engraving). Courtesy of the Fogg Art Museum, Harvard University Art Museums, Gift of William Gray from the collection of Francis Calley Gray. Phot,ograph by Rick St,a,fford, @President and Fellows of Harvard College. A detail of this engraving, a portion of the magic square, is used as the sample image in 22 figures in this book. The file processed is a portion of the image file detail.mat packaged with MATLAB version 5.0.

Part I

Preliminaries

Chapter 1 Functions and Convergence 1.1 Functions I . I . 1 Bounded

(L")Functions

D e f i i t i 1 . 1 A pzece,wise C O ~ I L L ~ ~ ~ ~ L U O ~~ UUS, I L C ~ ~ , Of T( Z5 ) defined 01%a n interval I is bounded ( o r L m ) o n I i f there i s a number A1 > O such that If (z)l 5 A1 for allx E I. T h e L m - n o r m of a functzor~f ( 2 ) i s defined b y IlfI2 = s 1 1 p { l f ( x ) I : : rE I ) .

(1.1)

Example 1.2. (a) If I is a closed. firlit,e interval, tlien nriy fil~lctio~l f (cc) contiriuous on I is also Lm 011 I (Theorern A.3). (b) Tlie fiirictiori f (z)

=

1/x is corltinuous and has a finite value at each

point of the irlterval ( O , l ] but is not bounded on (0, 11 (Figure 1.1).

(c) The functiorls f (x:) = sin(z) arid f (z) = cos(z) are Lm on R. Also. the complex-valued function f (z) = eix is Lm on R. 111 fact, 11 sin 1 1 , = 11 cos 1 1 , = Ileirllm = 1.

(d) Ariy polynornial f~lrictionp ( z ) is not Lw on R but is L" on every finite subinterval of R. (e) Any piecewise coritinllous fiirlctiori with orily jump discontirinities is L" ariy firlite interval I.

011

I . 1.2 Integrable (L1) Functions Definition 1.3. A piecewise continuous function f ( x ) &finned o n n,n, i,n,terual I is integrable ( o r of class L' o r simply L') o n I if the integral

is finite. The L'-norm of a function f ( x ) i s defined b y

4

Chapter 1. Functions and Convergence

FIGURE 1.1. Left: f (x) = 1/x is finite-valued but unbounded on (0, 11. Right: sin(x) (solid) and cos(jc) (dashed) are L" on R.

Example 1.4. (a) If f (z) is LW on a finite interval I, then f (z)is L1 on I. (b) Any function continuous on a finite closed interval I is L' on I . This is because such a. function must be Lm on I (Theorem A.3). ( c ) Any function piecewise continuous with only jump discontinuities on a finite closed interval I is L' on I.

(d) For any 0 < a < 1, the function f(z)= Izl-" is L1 on the interval [- 1, I]. Clearly f (z) is picccwise continuous with a11 i~~firiite discontinuity 1 ( f(x)ld z is improper and must be evalua.t,ed at x = 0. Thus the integral as an improper integral a,s follows: Is/-"dz

-

-E

12)"

ds

1

+ lirn

/xi-" dl:

E+O

1

1-a

lim (1 -

E-+O

=

2

The above example shows that an L1 function need not be Lm. If a then f(x) is not L1 on [-I, I]. ( e ) If a > 1, the function f (x) = xp" is L1 on the interval improper Riemann integral

converges.

> 1,

[I,ocl) since the

1.1. Functions

5

( f ) If 0 5 a 5 1, then f (x)= xpLYis not L1 on [ I ,co).But f (x)is L" on [I,co).This shows that an Lm function need not be L' on I if I is infinite. (g) The function f (x)= e-ixl is integrable on R since the improper Riemann integral

1, 00

e-1.1

converges. In fact,

JTme

x dx

d.

= 2.

We present below our first approximation theorem. It says that any function L1 on R can be approximated arbitrarily closely in the sense of the L1-norm by a function with compact support. Theorem 1.5 is illustrated in Figure 1.2.

Theorem 1.5. Let f (rc) be L' o n R, and let E > 0 be given. T h e n there exists a number R such that 2f

Then

Proof: Since f (x) is integrable, the definition of the improper Riemann integral implies that there is a number v such that rr

Hence, given

E

> 0, there

Pick a nurrlber R

Then,

roo

is a number ro

> 4.0, and define

> 0 such that

if r

> ro, then

6

Chapter 1. Functions and Convergence

FIGURE 1.2. Illustration of Theorem 1.5. Left: Graph o f f (x). Area of shaded region is < F. Right: Graph of g(x) with R = 10.

Definition 1.6. A piecewise continuousfunction f (x) defined o n a n interval I is square-integrable ( o r of class L' or simply L') on I i f the integral

i s finite. The

norm of a function

f (x) is defined by

Example 1.7. (a) Any function bounded on a finite interval I is also L2 on I. This includes functions continuous on closed intervals and functions piecewise continuous on closed intervals with only jump discontinuities. (b) Any function that is La and L1 on any interval I (finite or infinite) is also L~ on I.

(c) For any 0 < a < 112, the function f (x) = x-" is L2 on the interval [-I, 11. Therefore an integrable function need not be bounded. If a 112, then the corresponding f (x) is not L2 on [-I, 11.

>

(d) If a > 1/2, the function f (x) = xp" is L2 on the interval [I,m ) . If 0 Q 1/2,then the corresponding f (x) is not L2 on [I,m ) .

< <

Theorem 1.8.

(Cauchy-Schwarz Inequality) Let f (z) and g(z) be L~ o n the

1.1. Functions

7

interval I . Then

Proof: Let us assume first that f (x)and g(x) are real-valued, and let t be an arbitrary real number. Then

This expression represents a real-valued quadratic function of t that is non~legativefor all t . Hence its discriminant must be rlonpositive (the discriminant of a quadratic expression at2 + bt + c is b2 - 4ac). Therefore,

and (1.5)follows. If f (z)and g(x) are not real-valued, then we observe

(Theorem A.4) and then proceed as in the real-valued case.

C]

From the Cauchy-Schwarz inequality, we can say something about the relation between L1 and L~ functions.

Theorem 1.9. Let I be a finite interval. Iff (x) is L2 o n I , t h e n i t is L1 o n I.

Proof: By the Cauchy-Schwarz inequality,

, right side and hence Since I is a finite interval and since f (x) is L ~ the the left side of the above inequality are finite.

8

Chapter 1. Functions and Convergence

Remark 1.10. (a) The conclusion of Theorem 1.9 does not hold if I is l / x if z 1 an infinite interval. For example, let f (x) = Then f (x) 0 ifx<1. is L~ on R but is not L1 on R.

>

(b) The converse of the Theorem 1.9 is false for both finite and infinite intervals. This means that if f (z) is L' on I , then it is not necessarily true that f ( x ) is L2 on I. For example, let f (x) = x-'I2 for x E ( 0 , l ) . Then f (x) is L1 but not L2 on ( 0 , l ) . The following consequence of the Cauchy-Schwarz inequality allows us to conclude that linear combinations of L~ functions are also L~ (Exercise 1.19). It is also known as the triangle inequality for L~ functions; that is, it says that J J f 9 ) J 25 J J f 112 J I g ) ) 2(see Exercise 1.18).

+

Theorem 1.11. internal I . Then

+

(Minkowski's Inequality) Let f ( x ) and g ( x ) be L 2 on t h e

Proof: By the Cauchy-Schwarz inequality,

and (1.6) follows.

The following approximation theorem says that a function f (z) that is

L2 on R can be approximated arbitrarily closely by a compactly supported

no norm.

function in the sense of the The proof is very similar to that of Theorem 1.5 and is left as an exercise (Exercise 1.20).

Theorem 1.12. Let f(x) be L~ on R, and let E > 0 be given. Then there ), exists a n u m b e r R such that i f g ( x ) = f ( x )X I - ~ , ~ ] ( Xthen

1

00

If

( x ) - s(x)12dx =

llf

- gill

< r.

1.1.Functions

1.1.4

9

Dzflerentiable (Cn)Functions

Definition 1.13. Given n E N , we say that a function f (x) defined o n a n interval I is C n on I if it is n-times continuously diflerentiable o n I . C0 o n I means that f (x) is continuous o n I . f (x) is C m o n I if it is Cn o n I for every rc E N .

W e say that f (x) is C," o n I if it i s Cn o n I and compactly supported, C,O o n I if it is C0 o n I and compactly supported, and C y o n I if it is C" o n I and compactly supported.

Example 1.14. (a) If p(x) is a polynomial, then p(x) is Cm on R. (b) The "hat" or "tent" function defined by f (x) = (1 - 1x1) Xr-l,ll (x) is continuous but not differentiable on R since the derivative fails t o exist at x = -1, 0, 1. Thus f (x) is C0 (in fact C:) but not C1 on R (Figure 1.3(a)). (c) We can generate an example of a snloother function by taking the tent function f (x) defined in (b) and taking its antiderivative. That is, we can define the function g(x) by

Putting -m as the lower limit of integration is just a notational convenience since we could just as easily have started the integration a t any constant less than -1. By the Fundamental Theorem of Calculus (Theorem A.5)) g f ( x ) = f (x) for all x E R since f (x) is colltirluous on R. Therefore g(x) is C1 but not C? Note that g ( x ) is not C,' on R. (Figure 1.3(b)). (d) The function g(x) defined in (c) is L" on R but is not L1 or L~ on R. In fact, since g(x) = 1 for all x > 1, g(x) does not even go to zero as x goes t o co.However, by modifying the construction in (c), it is possible to define a function that is C: but not C2 on R. In particular, such a function would be L1 and L2 on R. The idea is to form two shifts of f (x), and subtract one from the other so that the resulting sum has integral zero. Then the antiderivative will vanish after a certain point. Specifically, define the function F(x) by

and the function g(x) by

S(x) =

J(t) d t .

Then g(z) is C1 but not C2 for the sarne reason that g(x) is, and F(x) is CE since it is supported in [-3/2,3/2] (Figures 1.3(c) and (d)).

10

Chapter 1. Functions arid Convergerlce

(e) It is possible to define a sequence of functions with irlcreasiilg snloothness based on the procetlure outlined in (d). First define the function B o ( x ) by BO(x) = )C;-1/2,1/21 (x),and for each n E N. define B,,( 2 : ) by ..I'

(

=(

B l - l ( t

+ 1/21 - (

t - 1/2)) dt

=

J

.I,+1 / 2

1.-1/2

,

B,,- ( t )d t .

Note that B l ( x ) is exactly the tent function defined in (I)). Each B,, (x) vanishcs outside the interval [ - ( ? I 1)/2. (rz + 1)/2] and for n 2 1 is Cn-l but not C" on R (Bo(.r) is not C O ) The . functioil B,,(J.) is called the Bsplinr of order I L a i d is ill fact a pieccwise poly~loilliiil.hlore int,erestirlg properties of spliiie functions arc give11 in Scctioii 7.7.1.

+

FIGURE 1.3. Top left: Graph of "hat" function f ( r )= (1- 1x1) (x). Top right: Antiderivative of f ( s ) . Bottom left: Graph of f (x 112) and - f ( x - 1/2). Bottom right,: Antiderivative of f(s). This function is C: on R but not C'.

+

Exercises Exercise 1.15. Prove each of the statcnients made in Example 1.2.

1.2. Convergence of Sequences of Functions

11

Exercise 1.16. Prove each of the statements rnade in Example 1.4. Exercise 1.17. Prove each of the statements made in Example 1.7. Exercise 1.18. Prove that each of the norms 11 - I / , for p = 1, 2, oo satisfy the norm axioms: Given f (x) arid g ( x ) LP on an interval I: (a)

llf llp 2 0.

(b)

(1 f ( 1 ,

(c)

IIQflip = IQI If ( l p , for every

(d)

11 f' + g1Ip 5 1 flip + llgllp (this is known as the triangle inequality).

=

0 if and only if f (x) = 0. Q

E C.

Exercise 1.19. Prove that the collection of functions Lp on an interval I (p = 1, 2, oo) is closed under the forrriation of linear combinations. That is, show that if f,,(z) is LP 011 I for 1 5 n < N and if a l , (22, . . . , CYN are complex numbers, then a,, f,,(x) is also LP on I.

c:==,

Exercise 1.20.

Prove Theorem 1.12.

1.2 Convergence of Sequences of Functions 1.2.1 Numerical Convergence Definition 1.21.

T h e sequence { a , , ) , , E ~converges to the nurnher a if for every c > 0, there is a n N > 0 such that if n N , then (a,, - a1 < t. In this case, we uirite a , t a as n t m, o r lim,,,, a,, = a . A numerical series, denoted a,,, convcrgcs to a number S i f the sequence N of partial sums { S N ) N € N defined b y s~ = E n = l an converges t o S . I n this case, we write E F = I an = S. W e will frequently denote the series a, b y

>

zr=l

xr=,

-

L n E N

A series

CnEN a , converges absolutely if EILEN la,,I converges.

Remark 1.22. (a) A fundamental property of the real numbers is known as the completeness property. The completeness property for the real numbers says that every set of real numbers bounded above has a supremum (or least upper bound). (b) A sequence of real numbers {an)nENis an increasing sequence if a, 5 a,+l for all n E N. The completeness property of the real numbers implies

"

'Since we assume that f (a)is undefined if f (x) has a discontinuity at x = a,then f (x) s On means "f (x) = 0 a t each point of continuity of f (x)."

12

Chapter 1. Functions and Convergence

that any bounded, increasing sequence of real numbers always converges to its least upper bound. a,, The partial sums of a series with nonnegative terms, (i.e., CnEN where a, 0) form an increasing sequence. Therefore, it follows that if a series of nonnegative terms is bounded, then it converges.

>

(c) A sequence of real or complex numbers is Cauchy if for every e > 0, there is an N > 0 such that if n , m _> N , then la, - a,l < t. Another consequence of the conlpleteness property for real numbers is the following: Every Cauchy sequence of numbers converges.

Example 1.23. (a) Consider the series quence can be computed since

If Iri < 1, then

SN

-+ 1/(1

-

r ) as N

Cr--nr n . The partial ,-

+ 00.Therefore, if

Irl

surn sc-

< 1, then

(b) Consider the series C,q"=l l / n ? Clearly,

Therefore,

for all N. Since each of the terms l / n 2 is positive, { s N l N E Nis a bounded, increasing sequence. Therefore, it converges t o its least upper bound and the series Cr=l1/n2 converges. Note that we have proved that the series converges but we have made no statement about the iralue of its limit. The same argument can be used to show that the series C:=l l/nP converges for every p > 1, but again does not give the value of its limit. (c) The Weierstrass M-test is a well-known test for convergence of a series. Consider the series En a,. The Weierstrass M-test says that if la, 1 5 b, for all n and if CntNLT1 converges, then CTrEN an converges. For example, consider the series cos(n)/n2. Since 1 cos(n)/n21 5 l / n 2 for all n and since Cr=,1/n2 converges, so does C:=, cos(n)/n2. Note that again we have proved that the series converges but have not given the value of its limit.

,,

En=,

1.2. Convergence of Sequences of Functions

13

(d) A consequence of the Weierstrass M-test is the following. If a series converges absolutely, then the original series also converges. Absolute convergence is equivalent to saying that the series converges regardless of the order in which the terms are summed. It is not true that all convergent series are absolutely convergent. For example, it is shown in most calculus books that the series x~=,(-l)"/71 converges but the harmonic series C z = l l / n does not. In dis(e) A doubly infinite sequence is a sequence of the forrn cussing the convergence of such sequences, we look at two lirnits, namely, linl,,-too an and lim,,,, a_,. If both converge to the same number, say, to a , then we write lirnl,,l+, a,,= a.

(f) A doubly infinite series is a series of the forrn C r = - m a,. In discussing cc the convergence of such series, we look at two series, rlxnlely, Cr,=, arband 00 El,=,a P n. If both of these series converge, then tliere is no problem. If C T00L =a, , = St and C:IP=, a p r 1= S ,then Cr=-, a,, = on+St+S- = S . In this case, we write also l i ~ n ~ , ~ ~ , ,r = a,,- =~S. ~ We will frrqllently denote the series a,, by C r L E artZ 01. simply by C,,a,, .

xr=-,

(g) If a doubly infinite series converges absolutely, then it converges regardless of the order in which the terms are surnmed. This is not tkic casc with series that do not convergc absolutely. Consider the series l / n ,where the n = 0 term is understood to be zero. Clearly, this series does not conN verge absolutely. However, because of cancellst ion, s~ = C rl=- 1/71 = 0. Hence, the sgmmetric partial sums corivergc to zcro. However, if we define

x:=-,

N2

S

N

>

~

1/x dz

= 111 N

+ oo

as N + ca. Therefore, if a doubly infinite scries does rlot converge absolutcly, then the form of the partial sums rnust be givcrl explicitly in order to discuss the convergence of the series. This is true of any series that converges but not absolutely.

2

. Pointwise Convergence

Definition 1.24. A sequence of functions {f, ( x ) ) ~ defined ~ ~ N o n a n interval I converges pointwise t o a function f (x) i f f o r each xu E I , the numerical sequence { f n ( x O ) ) n E N converges t o f (xu). W e write f,(x) + f (x) pointwise o n I, as n

4 00.

T h e series

Cr==l l T L ( x= ) f (x) po%nt,wiseo n a n inter~1alI

i f ,for each

XU

E I,

C:y, fn(x0) = f (20). Example 1.25. (a) Let f,,(x) = zn, x E [0,1) for all 71 E N. Then fn(z)+ 0 poilltwise 011 [O, 1) as 12 + a. See Figure 1.4(a).

14

Chapter 1. Furlctiorls and Convergence

(b) Let

2n:c 2 - 2nz 0

+ 0 pointwise on

Theri f,,(.r.)

ifx~[O.l/2n) if T E [1/2n,l / n ) if x E [ l / n , 11.

[0, 11. See Figure 1.4(1-I).

(c) Lct

2n2x

Tlle11 f ,,( J : )

+ 0 pointwise oil

if z E [O, 1/211) if x E [1/21~,l / n ) if x E [ l / n , 11.

[0, 11.See Figlil-t. 1.4(c)

1

zr' = -poir~twiseon (- 1. 1).

(d) The series

1-

r,=O

3(,

COS 'I).[.'

- converges poiritwise on R to its liniit by the

( f ) Tlie serics

11,2

00

COS 7) :X'

-- coilvergcs at odd lrlult,iplesof

( g ) The series r,=l

;.r

(since it reduces

'It '

t o thc: :tltcrnatirig scrics C r = l ( - l ) " / n ) but divergcs a t even m~iltiplcsof sr (since it rcduccls t o the liarrr~o~lic series). In h c t . it can be shown that t h e series coiivcrgcs for all .c that are riot ever1 i ~ i u l t i p l ~ofs K .

1.2.3

Unrifornl ( L m ) Convergence

Definition 1.26.

T h e sequc,rLce { f,, (z)},, E~ converges ~lrliforrnlyon I to t h e hirlctiorl f (.c) i f for r:uery c > 0, t h ~ v is : u n N > 0 .such that i f 11 2 N , t / ~ e , r ~ 1 f,L(x)- f (r)I < 6 for all z E I . We write f r L ( . r )+ f ( x ) ~ n i f o , r m l yo n 1 ass 71 + m . T h e series f , , ( z ) = f ( z ) u n i f o m l ~o n I zf the sequence of partial sums SN(X)

=

xr=,

zit=l fn N

( 2 ) converges

~ ~ n i f o m to l y f (x) on I .

Remark 1.27. (a) With uniforl~lconvelgence. for a given E the sanie N works for all x E I, whereas with pointwise Convergence N may depend on both F arid x. In other words, unifornl convergerice says that given E > 0 there is an N > 0 such that for all n ',N , the maximum difference between f,,(x) and f ( J ) on I is smaller than E . Because of this. uniform convergence

1.2. Convergence o f Sequences of F~lnctions

15

FIGURE 1.4. Top Left: Graph of , f , , ( x ) = .I." on [0, 1) for 11 = 2 , 4.8. Top Right: Graph of f,,(:x:) on [ 0 ,1 ) where f , , ( . ~ : is ) defined in Example 1.25(b). Bottom: Graph of f , , ( x ) on [O. 1 ) where f , , ( s ) is defined in Example 1 . 2 5 ( ~ ) .

is also called LX con?)f>.rgcJncr. Tliat is, f,,(.r.) only if 11 f,, f ,1 + O as 11 + 3 ~ .

--+

f(.r.)

unifori~~ly on I if ant1

-

(11) 111 Exainple 1.25(1)), thr. coilvcrgcnce of' f,, (s)t o 0 is pointwisc but not uniforrn. This is t)ecausc the nlaxirllur~~ cliffcrerlcc between ,f,,(r)ant1 t,hc liniit filrlctiol~f (1:) = 0 is 1 110 ma.t,t,e~. what I , is. 111 otller words. 11 f r ,- f I m = 11 fr,llno = 1 for all n,, aild so I f r ,- f 11% f i O as 11 + m-

(c) 111 Exall~ple1.25(c), t,llc convergc~llrbe of f,, (.I.) t o O is also poiiltwisc 1)ut not ~inifornl.111 fact, in this cast‘, 11 f,, - .f llcx: = Ilf,, llx = 7 1 h r a11 1 1 . Tliereforc 11 f,, - f ,1 + cc as n + m. 111fact,. there. an.c3 110 exainplcs of sequences that converge ul~iforrrllyon a n interval but not poiiltwise. 111 otlier words, the followil~gtheorel11 11olds.

Theorem 1.28. I f f , , (.r) + f

(.c)

in L"

071

a n ~ntcmnlI . then f,,(.r)

+ f (z)

poi7~t7uise an I .

Proof: Exercise 1.44.

An irllportai~ttheorcn~fro111 ;idvailced calc111usis t,he followillg. Its proof

16

Chapter 1. Functions and Convergence

is left as an exercise but can be found in almost any advanced calculus book (for example, Rurk, p. 266, Theorem 3).

Theorem 1.29, If fn(x) + f (x) unzformly on the interval 1 , and if each f , , ( x ) is continzious on I , t h . ~ nf , (z) is con,tin.uo?sson I .

Proof: Exercise 1.45.

Example 1.30. As an illustration of Theorem 1.29, let

Then each f n (x) is continuous on [- 1,1]and { frL( x ) } , , converges ~~ pointwise to the function f ( z ) defined by

which has a jump discontinuity at x = 0 (see Figure 1.5). It car1 be shown directly t,hat f,,(z) does not converge to f (z) in Lm on [-I, 11,but a different argument utilizing Theorem 1.29 would be as follows. If f l L (x) + f (z)in Lm on [-I, 11, then since each f, (z) is continuous, Theorem 1.29 irmplies that f (z)shou,ld also be continuous. Since this i s not

the case, the convergence cannot be i n La.

Example 1.31. (a) The sequence { x n I r L E N converges iiniformly to zero on [-a,a] for all O < a < 1 but does not converge uniformly t o zero on (-111). -

~

(b) The series

1 I" = - uniformly

1-x

n=O

on [-a, a] for all 0

< a < 1, but

not on (- 1 , l ) . 30

(c) The series

1n!

IC -=

ex uniformly on every finite interval I, but not on

n=O

R. rX,

cos n x

- converges

(d) Tlie series n=l

Weierstrass &!-test.

n . .2

uniformly t o its limit on R by the

1.2. Convergence of Sequences of Functions

FIGURE 1.5. Left: Graph of f,,(x) of Example 1.30 for Right: Graph of the limit function f (.r).

1.2.4

71.

17

= 2, 4 , 8.

Mean ( L 1 )Con,veq:rlmce

Definition 1.32.

T h e sequence { f , , ( in ruean to the function f ( s ) 071, I if

x ) ) , ~ ~defined N

o n a n interval I coilverges

W e write f,,(x)+ f(r) in mean orL I a s TL + a. Mean convergence is also referred to as L' convergence because f,,( x ) + f (.r) in m e a n o n I as rl + oo is identical t o the staternent that lim,,,, 11 f,,- f 111 = 0 . The series fi,(x)= f (s) in rnean o n I if the seyuer~ceof yar.tir~lsums N S N (n.) = f,, (x) converges i n mean to f (x) 0 7 , I .

Cr=, CIL=,

Llean convergence can be interpreted as saying that the area between the curves y = f,,(z)and y = f (z) goes t o zero as 7 1 + oc. This type of convergence allows point values of f,,(.~.) and f (x) to differ considerably but says that o n averagc t,hc functions f',,(x) and f ( z ) are close for large n.

Example 1.33. (a) Let J;,(le) = .c". :r* E [O,1)for all r~ E N. As we have seen in Example 1.25(a),this sequence converges to f (z) = 0 poirltwisc on [O, 1) but not uniformly on [0, 1). Since

as n

+ oo:f,,(z)+=O in mean on [o. 1).

18

Chapter 1. Functions andconvergence

-

(b) Consider the sequence {fn(z)),,EN defined in Exarnple 1.25(b). The sequence converges pointwise but not uniformly t o f ( z ) 0 on [0, 11. Since the area under the graph of f,(z) is 1/2n for each n. the sequence also converges in mean t o f (z) on [0, 11. In this example, we can see the character of mean convergence. If n is large, the function f,(z) is close to the limit function f (z) 0 (in fact identical to it) on most of the interval [ O , l ] , specifically on [1/n, 11, and far away from it on the rest of the interval [O. 1/72?. However, 011 average, f, (z) is close t o the lirnit function.

-

(c) The sequence { f , ( ~ ) )defined , ~ ~ in Example 1.25(c) tells a different story. The sequence converges pointwise but not uniformly to f (z) Y 0 on [O. 11, but since the area under the graph of f,, (x) is always 1, f,(z) does not converge to f ( z ) in mean. The width of the triangle under the graph of f (z) decreases t o zero, but the height increases to infinity in such a way that the area of the triangle does not go t o zero. The above examples show that sometimes pointwise convergence and mean convergence go together and sometimes they do not. The proof of the followirlg theorern is left as ail exercise (Exercise 1.47).

Theorem 1.34. If f,(x)

-+ f (x) in Lm

on n finite interval I , then f,(x)

+

f ( x ) in L' on I .

Remark 1.35. (a) The conclusion of Theorem 1.34 is false if the interval I is infinite. Consider fur exarrlple the sequence f , (x) = ( l / n )X [ o , n (x). ~ Then f,(s) i 0 in L" on R but Jyx I f , (z) - 01 dx = 1 for all n, so that f,(z) does not converge to zero in L1. (b) The converse of Theorem 1.34 is also false, as can be seen by considering Example 1.33(b). In this example, f,,(x) convcrges to 0 in L' on [ O , 1 ] but does not converge to 0 in L" on [0, 11. (c) In all of the examples of mean convcrgcncc considered so far, the sequences have also converged pointwise. hfust this always be the case? The by the following example. answer turns out t o be "no," as is ill~st~rated

Example 1.36. Define the interval I i . k by I,i.k = [2-jk, 2 - J ( k + I ) ) , for j E Z+ and 0 5 k 5 2J - 1. Let us make some elementary observations about the intervals I j , k . (a) Each

Ij.k

is a subinterval of [0, 1).

(b) The length of

is 2-3; that is, I I i ~ k= 2 - j .

(c) Each natural number n corresponds t o a unique pair (j,k), j E Z+ and 0 k 5 2.7' - 1, such that n = 2.7' + k. For each n E N, call this pair (j,, , k,). As n --+ co,j, -+ co also.

<

1.2. Convergence of Sequences of Functions

19

{I~.~}:L~'

(d) For each j . the collection of intervals forms a partlition of [O, 1);that is, the intervals are disjoint and cover all of [O; 1).

-

Now, define f , ( z ) = X I J I L , k r L( z ) . Then since I I j 3 , , k r L1 + O as j oo, f n ( z ) + O in mean on [O,l). However, f,(z) does not converge t o zero pointwise because for every x E [0, I), there are infinitely many n for which f,(z) = 1. Therefore, f,(z) does not converge to anything at any point of [0,1). See Figure 1.6.

FIGURE 1.6. Graph of f n ( x ) of Example 1.36 for 1 5 n 5 12.

1.2.5 Mean-square ( L 2 ) Conuergence Definition 1.37. T h e sequence {~,(X)),~~N converges in mean-squa.re to t h e function f (x) o n a n interval I 2f lim n--1s

If,(l) - f(5)12dz

= 0.

W Pvrnte f,(x) + f (x) i n mean-square on I as n + m. Mean-square convergence is also referred to as L~ convergence because f, (x) + f ( z ) i n mean o n I as n + cc is equivalent to the statement that limn,, IIj, f 112 = 0.

20

Chapter 1. Furictions and Convergence

T h e series Cr=,f,(x) = f ( x ) i n mean-square o n I zf the sequence of partial N f n ( 5 ) converges in mean-square to f ( x ) o n I . s u m s s N ( x )=

The proof of the following theorem is left as an exercise (Exercise 1.48). The proof of the first part is similar t o the proof of Theorern 1.34 and the proof of the secvilcl part ~llakesuse of the Caucl~y-Schwarziriequality (Theorem 1.8).

Thcorcm 1.38. (a) If fTL(:c) + f (x) i n La o n a finite interval I , then f n ( x ) + f ( x ) in L~ o n I. ( b ) If f,,(x) I.

+f

( x ) in L~ o n n finite interval I , then f,,(z-)

+f

(x) in L' o n

Remark 1.39. (a) The concl~isionof Tlleorerr~1.38(a) is false if Lm convergerice is replaced by poiritwise convergence. Example 1.33(c) shows a sequence that converges t o zero poirltwise on [O, 11 hiit not in L1 on [r), 11. By Theorem 1.38(b), the sequence does riot coriverge in L2 either, for if it did, then Tl~eorerr~ 1.38(b) would imply that it also converged ill L1.

(b) T11c corlclusion of Theoreni 1.35(a) is false if I is an infinite interval. For exarnple, if f;,(z) = ( 1 / f i ) X,03,,l(z), then f,, (x) + 0 uniformly on R, .rX! tit , 1 j ( ) - 012 tlz = 1 for all iz, so that f (x)does not converge to 0 in L2. (c) Tllc co~iclusionof Tl~eorern1.38(b) is false if I is an infinite interval. For example, if f ,,(x) = (1In) X,,,, ,,](z), tlien since

as n

+ m, f, (x) + 0 in L2 on R. but

for all n so that f,,(z) does riot converge t o 0 in

LI

on R.

(d) The converse of Theorem 1.38(a) is false. To see this, consider Exarnple 1.33(b). In this exarnple, it car1 be shown directly that f,,(z) + 0 in L2 on [ O , 1 ] (Exercise 1.49). However, f,,(z) does not converge to 0 in L" on [o: 11. (e) The converse of Theorerri 1.38(b) is false. To see this, let f,,(z) = J n X ( ~ , l / n(x). ] The11

1.2. Convergence of Sequences of Functions

as n

+ co so that

21

f,,(x)+ 0 in L1 on (0.11. However,

for all n so that f,, (x) does riot converge t o 0 in L2 or1 (0,1]

( f ) Finally, note that Exarriple 1.36 shows that L2 convergence does not irnply pointwise convergence since the sequence defined there also converges to 0 in L~ on [0, 1).

I . 2.6 Interchange of Lim,it.s and In,tegro,l,s A problerrl that we will encounter frequently in this book is the following. Slippose that a sequence of functions { f,,( x ) ) ,on~ an ~ interval ~ I converges in some serise described in one of the previous four subsections t o a fiirlction f (z)on I. Under wliat coiiditiorls is it true t,hat

Silicc we call write f ' ( z ) = lim,,,,

f , , ( . r ) .the above call t)e rewritt,erl as

and this problern is often stated as: W h e n can 7ue e3:chnnge the limit and

the integral? Tlie rliost typical forrn in which this problcrrl nriscs is wllcil the sequence is a sequence of partial surns of a series of fiinctions. In tliis case, thc cquivtilent question is: W h e n cart we intc:y.atc? a series c!f ft1,nctions termhy-term,? To see this, recall that the integral of an finite surri of functiorls is tlie sllrrl of the integrals. so that if sN(z)= C , , = , f,,(.x), then

J

If we could interchange tlie limit and the integral in this case, we would

lim

n=l

N+m

SN

(x)dx

Chapter 1. Functions and Convergence

22

The following theorem gives several conditions under which interchanging the limit and the integral is permitted. Theorem 1.40. (a) If f , (x)+ f (x)in L I

on I , t h c n

(h) If f,, (x)+ f (x)i n La o n

(c)

a, finite

V fTl(x)+ f (x) i n L~ o n a finite

interkvnl I , then

inteninl I , t h e n

dx =

lirn rt+m

Proof: (a) Let

.l;,(:c) + f

(x) in L'

011

1

f (x)dr.

I . Then

(b) By Theorem 1.34, if f,,(z) -+f (z) in L" on I, then it also converges in L1. Then the result follows frorri part (a). (c) By Tlleorenl 1.38(b), if f,,(z) + f ( z ) in L~ 011 I, then it also converges in L1. The11 the result follows from part (a).

If I is an infinite interval, then the cor~clusionsof Theorems 1.40(b) and 1.40(c) are both false, as can be seen by considering the example given in Rernark 1.39(c). In this example, f,,(z) + 0 both in LOC and L2 011 10, m). However, since JI f,, (x) d z = 1 for all n,

However, in the case of infinite intervals. we can prove a useful theorern by making an additional assumption on the sequence { f, ( x ) ) , , ~ ~ .

1.2. Convergence of Sequences of Functions

23

Theorem 1.41. Suppose that for every R > 0, f,, (x)3 f (x) in L" or in L' o n [-R, R ] . T h a t i s , ,for each R > 0 ,

I f f (z)i s L1 on a n interval I and 2f there i s a function g(x), L' o n I , such that for all x E I and all n E N , I f , (x)1 I g(x), then

Proof: If I is a, finitmeinterval, then there is nothing to do by T h e e rem 1.40(b) and (c), so wc Inay assume that I is infinite, and for convenience we will take I = R. By Theorem 1.40(a), it will be sufficient t o prove that f,, (z) + f (z) in L1 on R. Let e > 0. Since f (x)a,nd g ( z ) are L' on R, by Theorem 1.5, there is a number R > 0 such that

If

(o.)( d x

< 1/3

and

/

J,q(x) 1 di < r/3.

rl>R

Therefore, usirig the triangle inequality for the L1-norm (Exercise 1.18(c)),

By 'l'heorem 1.34 and Theorem 1.38(b),if f,,(x) + f (z) in LtX or L~ on [-R, R], then it also converges in L1 on [-R, R]. That is,

Hence, there is an N such that if n

LR1 R

fn

Therefore, if n

> N, then

> N, then

(x) - f ( x i I dz

< d3.

24

Chapter 1. Functions and Convergence

and (1.7) follows.

Next we present a variant of Theorem 1.41.

+ f (x) in L" o r in L~ and a n N E N such that for all

Theorem 1.42. Suppose that f o r every R > 0, f,,(x) o n [-R, R]. If for every n 2N,

t

> 0 , there

is an R

>0

Then

Proof: The proof is the same as that of Theorem 1.41, except that we choose R > 0 and N E N such that for all n 2 N.

Then (1.8) hecomes

from which (1.9) follows.

Exercises Exercise 1.43. Prove each of the statements made in Example 1.25. Exercise 1.44. Prove Theorem 1.28. Exercise 1.45. Prove Theorern 1.29. Exercise 1.46. Prove each of t,he claims made in Example 1.31. Exercise 1.47. Prove Theorern 1.34. Exercise 1.48. Prove Theorem 1.38.

1.2. Convergence of Sequences of Functions

25

Exercise 1.49. Prove that if f,(x) is defined as in Example 1.33(b),then f,(x) 4 0 in L~ on [ O , l ] . Exercise 1.50. (a) A sequence of functions { f , ( ~ ) ) ,defined ~ ~ on an interval I is said to be unzformly Cauchy on I if for every E > 0, there is an N > 0 such that if n, m 2 N then 11 f n - f,lI, < E . Prove that any sequence that converges in L" on I is uniformly Cauchy on I.

(b) A sequence of functions { f n ( x ) j n E Ndefined on an interval I is said to be L1 Cauchy on I if fur every E > 0, there is an N > 0 such that if n , m 2 N , then 11 f, - f,lll < E for all x E I. Prove that any sequence that converges in L1 on I is L1 Cauchy on I . (c) A sequence of functions { f n ( x ) j n E Ndefined on an interval I is said to be L2 Cauchg on I if for every E > 0, there is an N > U such that if n , m 2 N , then I(f , - f, ( I z < E for all x E I. Prove that any sequence that converges in L2 on I is L~ Cauchy on I.

Chapter 2 Fourier Series 2.1 Trigonometric Series 21.1

Periodic Functions

A fur~ctionf (x) defined o n R has period p > 0 Lf f ( r+ p ) = f (x) for all x E R. Such a function is said t o be periodic.

Definition 2.1.

Remark 2.2.

(a) Tlie functions sin(x) arid cos(x) have periocl 2n. The functions sin(ax) and cos(a,z), a > 0. have period 27rla. (b) If f (x) has period p > 0. it also lias period kp. for k E N. IIeilce a periodic function car1 have many periods. Typically the sirlallest period of f (x) is referred to as the pernod of f ( x ) .

Definition 2.3.

Given a function f(z)on R. n.n,d periodization o f f (x) is defined as the fun,ction

0,

number. p

> 0 , the

p

provzded that the sum makes sense. See Figure 2.1

Remark 2.4. (a) It is easy to verify that in fact the function f,,(x) lias period p by nlakirig a change of sunimatiori index in the sun1 on the right side of (2.1). Specifically,

where we have made the change of suinnlatiorl index n e n + 1.

(b) If f (z) is conlpactly supportcd, then the surri in (2.1) will converge poiritwise on R. This is because for each x the slirn will have only finitely riiaiiy terms. (c) If f (x) is supported in an interval I of length p. then f,(a.) is referred to as the period p extension of f (x). This is because for z E I. f,(x) = C,LEz f ( x + np) = f (x) since all terms in the sum besides the n = 0 tern1 are zero. (Whys?)Another way of thinking of this is that we a,re taking ilifi~iitel~ Inany copies of the fuiictiorl f (.c) and placing thein side-by-side on the real line.

28

Chapter 2. Fourier Series

FIGURE 2.1. Top Left: Graph of f (x). Top Right: Graphs of f (z + n p ) for -2 n 2 and p = 1. Bottom: Graph of the 1-periodization of f (z).

< <

Definition 2.5.

Given a

> 0,

the collection of functions (2.2)

{e2711r'T'a )nt~

ss called the (period a ) trigonometric system.

Remark 2.6.

+

(a) Recall Euler's formula: ei.' = cos(x) i sin(x). This formula can be proved by expariding both sides of the equation in a Taylor series (Exercise 2.20). Therefore

and it follows from this that each element in the trigonometric system has period a.

(b) The period a trigonometric system is sometimes given in the form

Systems (2.2) and (2.3) can be obtained from each other by forming simple

2.1. Trigonometric Series

29

linear combinations. Specifically, for n E Z,

and for n E N,

and

(c) A function that can be written as a finite linear combination of elements of the (period a) trigonornctric system is called a (period a ) t r i g o n o m e t ~ i c polynomial. That is, a trigonometric polynorrlial has the forni

for some h1,N E Z and some coefficients c ( n ) .

Theorem 2.7. T h e period a trigonometric s y s t e m ( 2 . 2 ) satisfies the following orthogonality relations:

Proof: Exercise 2.22. Remark 2.8. Note that since the functions e2"inxl" a11 have period a. the integral in (2.4) can be taken over any interval of length a. For example,

A fundamental problem in Fourier series is the following: Given a function f ( x ) with period a

> 0. can we write

,for some choice of coeficients { ~ ( n ) )? , , ~ ~ This problem leads t o three related questions that will be answered in the following subsections:

30

Chapter 2. Fourier Series

( a ) I n order ,for. (2.5) to hold, what m u s t the coeficients c ( n ) be? (b) Assuming we know the answer to question (a), in what serlse doea the s e ~ i e so n the right side of (2.5) converge? (c) Assurniny we know the answers to que.stions ( a ) and ( b ) ,does the series o n the right of (2.5) converge to f (x),or to some other funct,io*n,?

2.1.3 Let

The Fourier Coeficients

11s begin

hy answering question (a) above.

Definition 2.9.

G i v e n a function f ( x ) w i t h period a , the Fourier coefficier~ts o f f ( z ) are defined by

provided that those zntegrals m a k e sense. For example. i f f ( - r ) zs L' on [O. a ] , t h e n the integral in (2.6) converges for each 7 1 .

Remark 2.10. Tlie definition of the Fourier coefficients of a function f (x) is by no means arbitrary. In fact we are essentially forced to define them that way by the followilig a r g u m ~ n l . Sr~ppusethat in fact f(x) = C r l E c(7L) Z e2ai,1x/u. T1lei1 in light of' Theorern 2.7. for rrr 6 Z fixed,

since by (2.4), the only nonzero terni in t,he sum is the rz = m term. Note that the above argument is not a rigorous proof since we interchanged an integral and an irifiriite surn without having any idea liow or even if the slim converged. However. the argument is sufficient motivation for defining the Fourier coefficients as in Definition 2.9.

Definition 2.11.

G w e n a function f ( z ) with period a , L' o n [O, a ] , the Fourier series associated witahf ( z ) is defined as the formal series

where the c ( n ) are defined b y ( 2 . 6 ) . W e refer to (2.7) as a 'fformal series" since w e d o n o t yet know how or i f the series converges. W e write

2.1. Trigonometric Series

31

Remark 2.12. It is possible to rewrite the Fourier series of a function in terms of the real trigonometric system defined by (2.3). To see this, note that

Conversely, a series of the form

can be rewritten as

where

Example 2.13. (a) Let f (x) be the period 2 extension of the f ~ ~ n c t ~ i o n X,-1/2,1/21 ( 2 ) .The Fourier coefficients of f (x) are

=

i

0 if n is even, n # 0, 1 -( - I ) ( ~ - ' ) / ~ if n is odd, nn 1 if n = 0. 2

32

Chapter 2. Fourier Series

The Fourier series associated to f ( x ) is

See Figures 2.2 and 2.3.

(b) Let f ( x ) be the period n extension of the function x X(,,,)(x). Then

and for n

# 0,

Therefore,

f(x,

-.+,

i

C I,

e2inx =

~ E Z

-

C sin (n2 n x ) LEN

See Figure 2.4. (c) Let f ( x ) be the period n extension of the function x X(-,/2,,/2)(x). Then c(O) = 0, and for n # 0, c(n,)= ( - l ) n1;n/2n8, so t,ha,t,

(d) Let f ( x ) be the period 2n extension of the function 1x1 Xc-,,,,

1 . 4

( 2 ) .Then

Convergence of Fourier Series

Definition 2.14. A function f (x) o n a finite inten)u,l I i s piecewise differentiable o n I i f ( a ) f (x)i s piecewise continuous o n I with only j u m p discontinuities (zf a n y ) , ( b )f f ( z ) exists at a21 but finitely m a n y points in I and ( c ) f ' ( z ) i s piecewise continuous o n I with only jump discontinuities (zf a n y ) . A function f (z)i s piecewise dzfferentiable o n a n infinite interval I if i t is piecewise differentiable o n every finite subinterval of I .

2.1. Trigonometric Series

33

FIGURE 2.2. Top left: Graph of f (z) from Example 2.13(a). Top right: Graph of Fourier coefficients of f (z).Bottom lcft: Graph of f (z) from Example 2.13(b). Bottom right: Graph of absolute value of Fourier coefficients of f (z) .

Example 2.15. on I .

(a) Any function C1 on I is also piecewise differentiable

(b) If I is any finite interval, then the function Xr(x) is piecewise differentiable on any i~ltervalJ with I C J . (c) The tent function Bl(x) is piecewise differentiable on R because it is linear on the intervals (-m, -I), (-1, O), (0, I ) , and (1, GO).

(d) Any piecewise polynomial function is piecewise differentiable on R. The following convergence result is due to Dirichlet.'

Theorem 2.16.

>0

and is piecewise diflerentiable on R . T h e n the sequence of partial sums of the Fourier series (Dirichlet) Suppose that f (x)has perzod a

'The proof of Theorem 2.16 will not be given here but can be found for example in Walker, Fourier Analysis, Oxford University Press (1988), p. 19 (Theorem 4.5) and p. 48ff.

34

Chapter 2 . Fourier Series

of f (z),{SN( ~ ) } N E Nwhere ,

-

converges pointwise to the function f (z),where

FIGURE 2.3. Partial sums S N ( X )of the Fourier series f (z) from Example 2.13(a). Top left: A; = 10, top right: N = 20, bottom: N = 60.

-

Note that F ( a ) = f ( a ) if f (z) is continuous at x = a and that f ( a ) is the average value of the l e f t and right-hand limits of f ( x ) at x = a when f ( x ) has a jurrlp discontinuity. If we assume that f (z) has 110 discontinuities, then we can make a stronger statement as in the following heo or em.^

Theorem 2.17.

Suppose that f (z) has period a

>

0 and is continuous and

2 ~ h proof e of Theorem 2.17 can be found in Walker, Fourier Analysis, Theorem 4.4, p. 59.

2.1. Trigonometric Series

35

FIGURE 2.4. Partial sums S N ( Z )of the Fourier series f(x) from Example 2.13(b). top left: N = 10, top right: N = 20, bottom: N = 60.

piecewise dzfferentiable o n R. T h e n the sequence of partial s u m s S N ( X )gzuen b y (2.8) converges t o f ( z ) i n L" o n R.

What if the function f (x) is continuous but not piecewise differentiable? What can be said about the convergence of the Fourier series of such a function? It is by no means obvious that such functions exist, but they do. The most famous example is due to Weierstrass, who constructed a function continuous on R but not differentiable at any point of R. This function is defined by f (x) = CrLCN 3-n ~ o s ( 3 ~ xThe ) . Weierstrass AT-test can be used t o show that this function is continuous, but the proof that it is nowhere differentiable is hard.3 By the Weierstrass M-test, the Fourier series of the Weierstrass function converges uniformly on R. However, this is not the case for all periodic functions, continuous on R. The following theorem is due to ~ u ~ o i s - ~ e ~ r n o n d . ~ 3An example of a continuous, nowhere differentiable function similar t o the Weierstrass function, together with a very readable proof, can be found in Korner, Fourier Analysis, Cambridge University Press (1988). Chapter 11. 4 ~ x c e l l e n texpositions and proofs of this theorem can be found in Korner, Fourier Analysis, Chapter 18. and also in Walker, Fourier Analysis, Appendix A.

36

Chapter 2. Fourier Series

Theorem 2.18.

(DuBois-Reyrnoud) There exists a function f ( : E ) continuous on. R. n.nd e~rifhperiod 27r .ssuch that the Fourier serzes o f f (x) diucrgcs at .r = 0 . T h a t is, lim~,,

SN(O)does not exist where S N ( X )is given by (2.8).

In fact. it is possible to find a continuous, period 27r function whose Fourit>r s e r i ~ sdiverges at every rational nlultiple of 27r.5 Therefore, it is iinpossil~leto make the st,ateiiieilt that the Fourier series of every coiltiliuous fuilctioii coilverges pointwise t o tliat function. The ~ i e x ttheoreill, Theorem 2.19, is due to Fejitr and makes a geilera,l st,ate~nentahout the convergence of the Fourier series of a continllolls function. The idea behind Fejbr's Thcorcrn is tlie following. Instead of looking at tlw part,iixl sums (2.8), coilsider the arithmetic m e a n s of those part)ial suins; that is. coi~sicterthe sequence

It is oftell the case that when the corivergence of a sequence fails due t o oscillatioli ill the terms of the sequence, the arithmetic rrleans of the sequence will have better convergerlee behavior. Take the simple exariiple of the sequeiice { ( ~ ( n ) } ,where ~ ~ . a ( n ) = (- 1 ) " . Clearly lim,,, a ( n ) does not exist hecause the t,erms sirrlply oscillate back and forth between 1 and -1. Homrever. if mre coiisider the sequence of aritllrrletic means, { ~ ( T z ) } ~ ~ ~ . given 1 ) ~ .

so that linl,,, ~ ( 7 1= ) 0 (Exercise 2.25). If t,he original sequence { ~ ( n , ) } ,already , ~ ~ converges. taking the arithmetic means will not affect the convergence; that is, if lim,,,, a ( n ) = a. then also lim,,,, g ( n ) = a (Exercise 2.26). (Ft:j&r'sTheorerri) Let f (x) be a f7~nctionwith period n > O con,tiri,vovs o n R. and define for each 71, E N the functior~a ~ ( x by ) (2.9), where Sl;(cr) i s give77, by (2.8). T11,enCT.~T (z)converges uniformly to f (x) o n R as N -+ GO.

Theorem 2.19.

"bValkcr. Fourier .Arlal,ysis, Xppcrltlix A.

37

2.2. Approximate Identities

Exercises Exercise 2.20.

Prove Euler's formula: For every x E R, em

=

cos(x) +

i sin(x).

Exercise 2.21.

Prove that for every real nilmher o,,

Exercise 2.22. Prove Theorem 2.7. Exercise 2.23.

Prove each of the statements made in Remark 2.12.

Exercise 2.24.

Prove each of the statements made in Example 2.13.

Exercise 2.25.

Show that if a ( n ) = n

1 ~ ( n =) - x a ( k ) = n

n E N, then 0 -l/n

k=l

Exercise 2.26.

Show that if lini,,, where a ( n ) is given by 2.10.

2.2

a(n)

if n is even, if n is odd.

=

a , then liin,,,

a(n)

=

a,

Approximate Identities

The notion of an approximate identity or summability kernel is used extensively in all branches of analysis. The idea is t o make precise the notion of a "delta funct.ionn that is well known and widely used by physicists. engineers, and mathematicians. The delta function, 6(z),has the property that for any continuous function f (x),

or more generally,

f (t) d(x

-

t) d t = f (4

for. every z E R. From sorrle elernenlary considerations he reader. may fill in the details), any function b(t) satisfying (2.11) must satisfy, b(t) = 0, t

#0

and

b(t) dt = 1.

38

Chapter 2. Fourier Series

It is impossible for any ordinary function to satisfy these conditions since the Riemann integral of a function, f (z), vanishing at every 17: # 0 must be zero. This must be true even under more general definitions of the integral (such as the Lebesgue integral). Therefore, 6(t) is not an ordinary function. So the question remains: How are we t o make sense of this concept? There are two ways to do this.

1. Extend the definition of function. This has been done by L. Schwartz who defined the notion of a distribution or generalized f u n ~ t i o n . ~ 2. Approximate the delta by ordinary functions in some sense. This more elementary approach has its natural completion in the theory of distributions alluded to above, but can be understood without any advanced concept,s. The idea is t o replace the single "function" d(t) by a collection of ordinary functions {KT(t)},>" such that for every continuous function f (z),

and more generally,

where the limit is interpreted in some sense and described in Section 1.2. The purpose of this section is to explain the theory of approximate identities.

2.2. I

Motivation from Fourier. S e ~ i e s

In order t o further motivate the notion of an approxirrlate idenlily, let us consider how one might prove Theorems 2.16 and 2.19.

Definition 2.27. For each lc E N , and a > 0, define the Dirichlet kernel Uk (x) b y k

D&)

=

C

,2nimx/a.

(2.12)

m=-k

See Figure 2.5. good exposiliorls uf this theory can be found in Horvath, A n introduction t o distributions, The American Mathematical Monthly, vol. 77 (1970) 227-240, and Benedetto, Harmonic Analysis and Applications, CRC Press (1997).

2.2. Approximate Identities

Theorem 2.28. For each k

E

39

N , and a > 0, the Dirichlet kernel, D k ( x ) , can

be written as Dk(x)=

+

sin(-ir(k l ) x / a ) sin(~x/a) '

and for any period a function f ( x ) ,

Proof: Eqlxakion (2.13) is a,n exercise (Exercise 2.38) and reqi~iresonly t,he formula for summing a geometric series. As for equation (2.14),

Sk ( r )

=

C c ( n )e2K"x/U

The result follows by making the change of variables t F+ x - t in the above int,egral a.nd remembering t,ha.t bot,h Dk (a)a.nd f (z) ha.ve periocl a.

Definition 2.29.

For each n E N , and a > 0, define the Fejhr kernel F,(x)

bY n- 1

CD~(X).

1 ~ ~ (= x ) k=O

(2.15)

See Figure 2.6.

Theorem 2.30. For each n E N , and a > 0 , the Fej6r kernel, F,(x),can he written as

and for any period a function f ( x ) ,

:La

a ( n )( x ) = -

f

(X -

t ) F,(t) d t .

Proof: Equation (2.16) is an exercise (Exercise 2.39) and requires only the formula for summing a geometric series and some manipulation.

40

Chapter 2. Fourier Series

FIGURE 2.5. The Dirichlet kernel D k ( x ) (2.14) for a

= 1.

Equation (2.17) is also a,n exercise (Exercise 2.40), ancl the derivation is similar to (2.14).

17

From Theorems 2.28 and 2.30, we see that the proofs of Theorems 2.16 (Dirichlet) arid 2.19 (Fejer.) amoul~tto sliowing that

pointwise for every period a f~inct~ion f (z),piecewise differentiable on R, ancl

in LDO on R for every period a function f (x) continuous on R. Such convergence results depend on properties of the sequences { D k ( ~ ) ) k E and N { p ? 2 ( ~ ) ) 7Consideration 26N. of tJhe required properties of these sequences leads to the notion of an approximate identity or summability kernel.

2.2.2 Definition and Elcamples Definition 2.31. A collection of functions {Kr(x)),>o o n a n interval I

=

( - a , a ) ( a = m is permitted) is a n approximate identity or a summability kernel

2.2. Approximate Identities

20

i

i

i

i

i

i

i

i

i

'

O

i

i

i

i

i

I s . ~ ~ j ~ - - j - - - ~ - - i - - - ; - - - ~ - - j - - - ; . . . ~,s-.-j---i-..;..;...j...;.. ..l l l l 16--.]---+...t..]...i...t.--

l

l

I I--

I

I

I

I

I

I

I

I

I

I

I

I

I

I

l

l

l

I

l

l

l

6~~~d~-~l...L..A...l..-l~~~l...l...l...

- 0 5 0 . 4 - 0 3 -0.2 -0.1

0

I 1

l

l

l

01 02 03 04 05

I

I

I

I

I

-05 - 0 4 0 . 3 - 0 2 - 0 1 0

i I

01

i

A

;' :1 . -

i --

41

i

-

-. .

0.2 0.3 0 4 0 5

FIGURE 2.6. The Fej6r kernel F,,(x) (2.14) for

a =

1.

on I if the following conditions hold. (a,) For all

7

> 0,

L

KT (x)d z = 1 .

( b ) There exists M > 0 such that for all

7-

> 0,

S_:

/ K T ( x )dl x 5 M

( c ) For every 0 < 6 < a ,

IKT( x ) 1 d x = 0.

lim ~--t'+.

S
Example 2.32. (a) For 0 < r < 1, let K T ( x ) = 1 / ( 2 r )X(-,,,I. { K T(x))O
Then

Then {h',(x))o<,
( c ) More generally, let g ( x ) be any function L1 on R and supported in (-a, u ) ( u = oo is permitted) such that

42

Chapter 2. Fourier Series

For 0 < 7 < 1, let KT(x) = (117) y ( x / ~ ) Tlier~ . {KT(x))o
7

> 0, let K T ( x )= 7

sin(nx/ ( r a ) ) sin(nx/a)

X(-a,a)

(4-

Tlien { I ( , (x)),,~ is an approximate iclentity oil (-a, a). 111 fact, Kll, (x) = F, (x) ?C(-,.) (x), where F, (z) is the Fej6r kernel defined by (2.16). (e) For

T

> 0, let

Then Kllk(z)= D k ( x )X(-,.,) (x), where D k ( x ) is the Dirichlet kernel defined by (2.13). However, {KT(z)),>o is not, a.n a.pproximate identity on (-a, a ) .

(f) For each r

> 0, let

-1

a,

=

[~:,(1

-

t2)'11 dt]

, and let

Then {K,(z)),>o is an approximate identity on (-1,l). Define L, (n;) = Kll,,(x). Then the sequence { L , ( X ) ) , ~ is ~ referred to as the Landau kernel.

2.2.3 Convergence Theorems The idea behind approximate identities is to investigate the validity of the statement f ( t ) K T ( x- t) dt = lim f T ( x ) = f (x).

,to

Proving (2.18) will allow 11s t,o a,pproxima,te a,rbitrary functions f (n;) by functions of the form f, (x) that often can be made to have desirable properties (see Corollary 2.37). In this subsection, we will consider mostly approximate identities on R as those will be most useful t o us later in the book. Let us first consider pointwise approximation.

Theorem 2.33. Let f (x) be L"

o n R and continuous at the point x = a . Suppose that {K,(x)),>o i s an approximate identity o n R. T h e n f ( t )K T ( a - t ) dt = f ( a ) .

(2.19)

2.2. Approximate Identities

43

Proof: Note first that by a change of variables,

f (t) KT( a - t ) dt =

f (a - t ) KT (t) dt.

By Definition 2.31(a),

Therefore, for ally umber 6

> 0,

Let t > 0. Since f ( x ) is continuous at x = a , there is a 6 > 0 such that if It1 < 6, then If (a) f ( a t)l < c/(2116), where 116 is the upper bound in Definition 2.31 (b). Hence for such a delta, -

Since f (z)is L"

on R,

-

If

(y)l

< 11 f /lo

for all t E R. Hence, for any 6 > 0,

for all y E R. Therefore,

Chapter 2 . Fourier Series

44

as

T

-+ of

0

by Definition 2.31(c). Therefore, there is a then

Hence for 0

TO

> 0 such

that if

< r < 70,

and (2.19) follows. Note that if K T ( x ) = (117)g ( x / r ) for some function g ( x ) , L1 on R and compactly supported, then the assumption that f (x) is L" on R is unnecessary. It would only be necessary t o assume that f (x) is bounded on some small interval containing x = a , which follows from the continuity of f ( T ) at n: = a (Exercise 2.42). Now let us consider uniform convergence. Theorem 2.34.

T,et f(z) he L" and unzformly continuous on R . Suppose is an approximate identity o n R. T h e n that {K,(X)),>~

Proof: Making the same estimate as in (2.20), it is sufficient t o show that given r > 0, there is a 6 > 0 and an TO > 0 such that if 0 < T < TO, then

f (x) f (X t ) / -

-

IKT(t)I

dt

+

.I

If

(z)

-

f (X - t)l lKT(t)l df

< E.

It126

(2.22) As for the first term in (2.22), the uniform continuity of f (x) implies that there is a 6 > 0 such that if It1 < b , then for all x E Ft, f (x) - f (x - t)l < € / ( 2 M ) , where M is the bound in Definition 2.31(b). Hence, a.s in tJheproof of Theorem 2.33,

for all r > 0. As for the second term in (2.22), since f (x) is L"

on R,

2.2. Approximate Identities

which converges to 0 as r that if 0 < r < T O , then

45

+ 0+ for any 6 > 0. Hcncc thcrc is a TC,> 0 such

for all z E R, and (2.21) follows. The condition that f (z)is uniformly continuous on R is satisfied if for example f (z) is C: on R. More generally, it is satisfied if f (z) is Co on R and liml,l,, f (x) = 0 (Exercise 2.43). Lemma 2.35 (which we state without proof) establishes a very important property of functions referred to as the continuity of translation for L1 and Lqunctions. This means that the translate of an L1 or L2 function remains very close to the original function provided that the translation is small enough. Here closeness is measured in the sense of the L' or L2 norm. Continuity of translation is used to prove L1 and L2 convergence using approximate identities.

Lemma 2.35.

(Continuity of Translation) Suppose that f (x) is piecewise con-

tznuous o n R . T h e n the followzng hold.

(a) I f f (z)i s L1 o n R, then

(b) I f f ( z ) is L2 o n R, then

f%Lf(x)-

f ( x - t ) 2 dz=O.

(2.24)

As for L1 or L' convergence in (2.18), the following theorem (which we state without proof) holds.

Theorem 2.36. (a) Suppose that f ( x ) is L1 o n R, and that {K,(x)),,o identity o n R. T h e n

zs a n approximate

r

( b ) Suppose that f ( z ) is L2 o n R, and that {K,(X)),>~is a n approzimate identity o n R. T h e n

46

Chapter 2. Fourier Series

FIGURE 2.7. Illustration of Lemma 2.35. Top Left: Graph of function f (x). Top Right: Graph of f (x) and f ( x - t ) where t = 1/16. Bottom: Graph of If (4 - f ( x - t)l. Area of shaded region is small for small t.

The following approximation theorein is a consequence of Theorein 2.36. Corollary 2.37.

( a ) Let f (x) be L' o n R, and let R such that

E

> 0. T h e n there i s a function g(x), C: o n

llf ( b ) Let f (x) be L~ o n R, and let R such that

E

-

9111 < E .

> 0. T h e n there i s a function g(x), C: o n

Ilf

-

9/12 < t.

Proof: (a) By Theorem 1.5, there is a compactly supported function h(x), L1 on R, such that 11 f - hill < €12. Now, let

> 0 (SCC

Examplc 2.32(b)). Thcrl { K T ( ~ ) ) T > isoan approximatc idcntity on R. By Theorem 2.36(a), T

lim h, (x)= lim r+o+

7+0+

h(t)K T ( x - t)dt = h(x)

47

2.3. Gcncralizcd Fourier Series

in L1 on R. Hence there is a 70 > 0 such that Ilh, - hill < ~ / 2 .Let g(x) = h, (x). That g ( x ) is compactly supported follows from Exercise 3.25, and that g (x) is C0 on R follows from Theorem 3.18. The proof of (b) is similar (Exercise 2.44).

Exercises Exercise 2.38. N-1 r n

'1

CrL=O

-

Prove equation (2.13). (Hint: Recall that for any number

1-r N

~

-

1

Exercise 2.39.

Prove equation (2.16).

Exercise 2.40.

Prove equrttiorl (2.17).

Exercise 2.41.

Prove each of the statements made in Example 2.32.

Exercise 2.42. Prove that if f (x) is continuous at x = a:then there is a > 0 such that If (xjl 5 M for all x E [a - b, a 61. b > 0 and a number

+

Exercise 2.43. (a) Prove that i f f (x) is C: on R, then f (x) is uniformly continuous on R. (b) Prove that if f ( x ) is C0 on R and limlslj30 f (x) uniformly contir~uouson R.

=

0, then f (z) is

Exercise 2.44. Prove Corollary 2.37(b).

2.3 Generalized Fourier Series 2.3. I

Orthogonality

Definition 2.45.

A collection of functions { g n ( ~ ) ) , E L~ ~ , o n a n interval I is a (general) orthogonal system o n 1 provided that (a) / gI n ( X ) S r n ( x ) d x = O L f n j i m , and

Part ( b ) says i n particular that none of the gn(x) can be zdentically zero. Th.e collection { ~ , ( x ) ) , ~ Nis a (general) orthonormal system o n I provided t l ~ u ti L ,is ur1 orthogonal system o n I and

48

Chapter 2. Fourier Series

It is not nccessary that the set {gn(x)) be indexed by N, and in fact we have seen an example (the trigonometric system) that is indexed by Z. In all future examples, the index set will either be specified or will be clear from the context. Whenever a generic system of functions is considered, the index set will be assumed t o be N.

Remark 2.46.

(a) Any orthogonal system can be normalized so that it becomes an orthonormal system. Tha,t, is, if {gn(.x)) is a11 orthogonal system, then we may define the functions

Then the system {&(x)) is an orthonormal system. (b) The Cauchy-Schwarz inequality guarantees that each of the integrals in Definition 2.45 exists as a finite number. That is, since f (x) and g(x) are L2 on I,

(c) Throughout t,he hook, we will use inner product notation to represent the integrals in Definition 2.45. That is, we write for any functions f (x), and g(x) L2 on I,

This means in particular that

Example 2.47.

(a) Given any a

> 0, the collection

is an orthogonal system over [-a, a ] . I t is also orthogonal over [O,2a]and in fact over any interval I of length 2a. The collection

is an orthonormal system over [-a, a ] . I t is also ort'honormal over [O, 2a] and in fact over any interval I of length 2a.

2.3. Generalized Fourier Series

(b) Given any a

49

> 0, the collections

{sin(-irnz/a)},~~

and

{cos(~nz/aj},~~

are each orthogonal systems over [-a, a ] . The collections

arc each orthonormal systems over [ a , a ] . (c) Given a > 0, the collection

is an orthogonal system over [0, a], and in fact over any interval I of length a. The collection

is an orthonormal systcm ovcr [0,a], and in fact ovcr any intcrval I of lcngth a.

2.3.2

Generalized Fourier Series

Definition 2.48.

Given a fun,ction f (z), L 2 o n a n interval I , and a n orthonormal system {gn(x)) o n I , the (generalized) Fourier coefficients, {c(n)) of f (x) with respect t o {g,(x)) are defined by

The (generalized) Fourier series o f f (x) with respect to {g,(z)) is

The fundamental problem is to determine under what circumstances the in the above definition becomes a "=" and, if so, in what sense the infinite series on the right side of the equality converges. It turns out that the most convenient form of convergence in this case is L~ convergence on "N"

T. Theorem 2.49. {y,(z))

(Bessel's inequality) Let f (x) be L 2 o n a n interval I , and let be wrt, outhonownul sysLe,ln U,IL I . T J L ~ ~ I L

50

Chapter 2. Fourier Series

The proof of Bessel's inequality will require the following lemma. Lemma 2.50. Let { g , (x)} be an, orthon.orm.al system, on, o,n, in,ten~o,lI . Then for every f (x),L~ on I , and every N E N ,

Proof: The proof is just a calculation making use of thc orthonormality of

{gn(x>>.

+

C C (f'g n ) (f.Smj

n=l m=l

I

~n ( 2 )gm

( x )d:r

which is (2.28).

Proof of Theorem 2.49: Let f ( x ) be given, and let {gn(x)) be an orthonormal system. Then by Lemrrla 2.50, for. each fixed N € N,

2.3. Generalized Fourier Series

51

Therefore, for all N E N,

Since I (f, g,) 1' > 0 for all n , the partial sums of the series CncN( f ,gn) l 2 form an increasing sequence bounded above by JI 1 f (x)I2d x . Thus the series CnEN 1 ( f , gn)I2 converges so that we can allow N to go to infinity Thus;

which is (2.27). Closely related to Lemma 2.50 is another very important inequality that will be very useful in the next subsection.

Lemma 2.51. Let { g , ( x ) } be a n orthonormal system o n I . Then for every f (x),L~ o n I , and every finite sequence of numbers { a ( n ) ) ~ =, ,

Proof: Let f ( x )be given, and let { g n ( x ) )be an orthonormal system. Then

52

Chapter 2 . Fourier Series

by Lemma 2.50.

We are now in a position t o answer the fundamental question about Fourier series, namely: When is an arbitrary function equal to its Fourier series and in, ~r~h,nf; sense does t h t Fourier series converge? The answer lies in the notion of a complete orthonormal system.

Definition 2.52.

Given a collection of functions { g , ( z ) } , L~ o n a n interval 1 , the span of {g,(x)), denoted span{g,(x)), is the collection of all finite linear combinations of the elements of { g , ( x ) ) . In other words, f (z) E span{g,(x)) zf N and only i f f ( x ) -a ( n )y n ( x ) for some finite sequence {a(n)};=, . Note that N is alwal~sfinite but m a y he arbitrarily large.

E x a m p l e 2.53. (a) Let PI denot,e t,he set of all polynomials on the interval I. Then PI = span{xn),"==,. (b) span{e-2TznLjn,,Z is the set of all period 1 trigonometric polynomials. ( c ) Lel p(z) = (1- 1x1) X r l , l l ( x ) .Then span{cp(x - n)InEz is the set of all functions that are ( i ) continuous on R, (ii) linear on intervals of the form

[n, n

+ l ) ,n E Z, and (iii) compactly supported.

R e m a r k 2.54. (a) For any collection of functions {g, (x)} , span{gn (x)} is a linear space: that is, it is closed under the formation of lincar combinations. Specifically, if {f,(x))E=, & span{g,(x)), then for any finite seN quence { a ( n ) ) L , the function f (x) = a(n)f, (x) is in span{gn (z)) (Exercise 2.61). (b) The definition of span involves only finite sums. Without additional assumptions on the collection {g,(x)}, there is no guarantee that any sum of the form CnEN a(n) g,(x) will converge in any sense. For example, if g,(x) = xn-l for n E N, then the series n! x r V o e s not converge except at x = 0, and the series Cr=o 2-n x n does not convcrgc if 1x1 2. See also Theorem 2.55 below.

E,"&

>

(c) Related to the notion of span{g,,(x)) is the notion of the mean-square (or L') closure of span(g,(x)), denoted @ZTi{g,,(x)} which is defined as follows. A function f (x) E SjZZTi{g,(x)) if for every E > 0, there is a function g(z) E span{gn(x)) such that 11 f - gllz < E .

2.3. Generalized Fourier Series

53

As a partial answer to the question of when finite sums can be replaced by infinite sums, we have the following theorem. Theorem 2.55. Lct { g , , ( x ) )bc a n orthonormal s y s t e m on a n zr~tervalI . T h e n a function f (z), L~ on I , i s in SjZiiT{g,(z)) i f and only i f

Proof: (+)

(2.29) is equivalent t o the statement that

Therefore, given

E

> 0,there

is an N

> 0 such that

and f (x) t w { g n (x)).

(==+) Suppose that f (x)E m { g , (z)), and let c > 0.Then by definition there is a finite sequence {a(n)):l,, some NO t N, such that

Since

is a, decrea,sing secliience (Exercise 2.63), it follows t,hat for every N

> No,

54

Chapter 2. Fourier Series

and (2.29) follows. If every function L2 on I has a representation like (2.29), then we say that the collection { g , ( 2 ) )is complete on I. This means that every furiction L~ 011 I is equal t o its Fourier series in L2 on I.

Definition 2.56. Let {g,,(x)) be a n orthonormal system o n I. T h e n {g,(x)} is complete o n I provided that e v e q function f (x),L' o n I, is i n m { g , ,( x ) } . A complete orthonormal system is called a n orthonormal basis. The next theorem gives several equivalent criteria for an orthonormal system to be complete.

Theorem 2.57.

Let {g,(x)} be a n orthonormal system o n I. Then, the ,following are equivalent.

(a){gn(x)) is complete o n I ( b ) For ever9 f (x),L ' on

I,

( c ) Every f u n c t i o n f (z), C: o n I , i s i n span{g,,( z ) } .

(d) For every function f ( x ) , C: o n I,

Remark 2.58. (a) Note that Theorem 2.57(c) is precisely the definition of completeness but with C: functions replacing more general L2 functions. It is often easier t o work with cont,inuous compactly supported filnctions, and the theorem states that this is sufficient.

(b) Theorern 2.57(d) says that Bessel's inequality is an equality for conlplete orthonormal systems. This eyualily is referred t u as Pluszcherel's FOI-mula.

Proof of Theorem 2.57: (a) of Theorem 2.55.

(b). This follows exactly as in the proof

(a) + (c). This follows immediately from the fact that every furiction C: on I is also L2 on I. (a) +== (c). Let f (a)be L~ on I, and let E > 0. Then by Corollary 2.37 there exists a function g(x), C: on I such that I f - 9112 < 612. By (c).

2.3. Generalized Fourier Series

55

there exists N E N such that

Applying Minkowski's inequality, we obtain

Therefore,

so that f (x) E span{g,(x)) and (a) follows. (c)

* (d). By Theorem 2.55, (c) holds if and only if

for all functions f (x) C: on I. But by Lcmma 2.50,

Therefore, (c) is equivalent t o the statement that

and (d) follows. To illustrate an application of this theorem, we will prove the following result about trigonometric Fourier series.

Theorem 2.59.

T h e trigonometric system {e2""x}nEz is complete o n [ O , l ] .

Proof: Wc will usc Thcorcm 2.57(c). To that end, let f (x) be continuous on [0,1] (note that it is also compactly supported), and let E > 0. By

Chapter 2. Fourier Series

56

Exercise 2.65, we call find a functioil f ( x ) that has period 1, is C' on R, and such that - 7112 < t/2. By %er's Theorem (Theorem 2.19), G N ( z ) converges in L" on R t o as N i m, where

If

S(2)

and

1

j ( t ) e-2Tmxdz.

c(n)=

By Theorem 1.38(a), ZN(z) also converges t o Note that

-

f (z) in L~ on [0,11.

(Exercise 2.66). Therefore, for N large enough

arid by the triangle inequality,

But the function

2

(1

-

y)

11") 1.2"znx

n=-N

is in

s p a n { e 2 ~ z r ~ z }nEZ

Hence, f (z) is in span{e2Tinz}n,Ezand by Theo-

rem 2.57(c), the trigonometric system is cornplete on [O, I].

Exercises Exercise 2.60. Prove that if {g,(x)} is an orthonormal systerrl on an interval I and if {a(n));=, is any finite sequence of numbers, then

2.3. Generalized Fourier Series

57

Exercise 2.61. Prove that if {g,(x)) is any systerrl of L2 ~ U I I C L I O I ~Llle11 ~, span{g,(x)) is a linear space (that is, it is closed under the formation of linear combinations, see Remark 2.54 (a)). Exercise 2.62.

f (x)E

Prove that if {g,(x)) is any system of L2 functions, then

(x)) if and only if there is a sequence of fiinctions {fk(z)}

such that fk(2) E span{g,(x)) and such that limk,, 11 f - f k ( I 2 = 0. (Hint: For the "only if" direction, choose f k ( x ) E span{g,(x)) such that 11 f

-

fkll2

< Ilk..)

Exercise 2.63. Prove that if {g,(x)} is an orthonormal system on an interval I, then for any f (x), L2 on I, the sequence

is

il,

decreasing sequence. (Hint: Use Lemma 2.50 or 2.51.)

Exercise 2.64. Show that if {g, ( x ) ) is a complete orthonormal system on an interval I, then (2.30) holds for every f (x), L' on I . Exercise 2.65. Let E > 0, and let f (x) be C0 on [0, 11. Then there is a function f(x) that has period 1, is C0 on R, and such that (J: 1 f d ~ ) ' /<~e. (Hint: For 6 > 0 sufficiently small, you can construct r ( x ) by modifying f jx) only on the interval [l - 6, I ] , and then extending periodically.)

fi2

Exercise 2.66.

Prove equation (2.31).

Exercise 2.67. Let {g,(x)) be a complete orthonormal system on an interval I. Show that the Fourier series of any function f (x), L2 on I, can be integrated term-by-term in the following sense. For any numbers a < b, such that [a,b] I,

(Hint: If the sum converges in 1,' on I , it converges in L2 on [a,b]. Then use Theorem 1.40(c).)

Chapter 3

The Fourier Transform 3.1 Motivation and Definition We have seen that if f (x) is a function supported on an interval [-L. L] for some L > 0, then f (x) can be represented by a Fourier series as f

=

C

e27Mx/2L)

where

l

2L c(n)= -

S_, f ( t ) L

e-"7t(n/2L) d t .

,"

(3.1)

Of course the Fourier series actually equals the 2L-periodization of f (x) (Figure 3.1). What happens to this representation if we let L + co? In order to answer this question, define for each L > 0 and each integer

- 1;

n the number

T((nl2~)

f (t) c-~""('"'")

dt: A

so that f^(n/2L) = (2L) c ( n ). If we were to plot the numbers { f (n/2L) inEz for very large values of L , then the resulting graph would begin to resemble a function of a continuous variable on R (Figure 3.2). This function would naturally be defined by

In addition, we could also write

for large L, since the last sum is a Riemann sum for the last integral. Therefore, we have formally established the duality

fiY)

/

R

f(t) e

dt.

f (x)

-

~ ( 7e2nzx7 ) d7.

(3.3)

The discussion in the remainder of this chapter will focus on two general questions: In what way are the properties of f (x). for example, continuity.

60

Chapter 3. The Fourier Transform

FIGURE 3.1. 2L-Periodizations of a function f (x) assumed to be supported in [-L. L]. Top left: L = 1, top right: L = 2 , bottom: L = 4.

differentiability. int,egral)ility or square-integral~ility. reflect ecl in t h~ corresponding properties of f (y)? and What properties lrl~lstf (s) and f (7) satisfy in order for the *.N" in (3.3) to be replaced by "="'! Let us first make a defiiiitio~~. A

A

Definition 3.1. The Fourier trarlsforrn o f n function f (z). L' on R. i s also n function o n R, denoted f^(-y) defined by

Remark 3.2. The assumption that f (x)is L1 oil R is rrlade in order to ensure that the integral in (3.4) converges for each riuinber 7 . This convergence liolds by virtue of tlie fact that for each 2 E R, we call establish a Cauclly condition on the numbers

La 0

d.r

and

s,

=

f (.r)c - ~ " ~ 'd. ~ . :

cr.

>0

3.1. hlotivation and Definition

61

FIGURE 3.2. Fourier coefficirrrts for the functions graphed in Figure 3.1. Note how the graphs of the sequences begin to resemble the graph of a continuously defined function. as follows. If b

> u > 0, then

<

-

=

lim lim

..e+w

1 dz

- Z . ~ Z ~ X

1,

If(x)ldx

By the completeness property of the complex numbers (Remark 1.22(c)), there exist numbers s+ and s- such t h a t limaico S$ = sf and lim,,, sa S - . Thus, f ( y ) = s+ s-. A

+

A

Example 3.3.

(a) Let

f(x) = Xi-112,1i21 ( 2 ) .Then f ( 7 )= sirl(*7)/~7(Exer-

cise 3.4). (b) Let f (x) = (1 - 1x1) Xl-,,ll (l-). Then

f(y) = sin2( * ? ) / ( T ~ )(Exercise ~ 3.5).

(c) Let f (x)= e-Z"xI.Then f l y ) = l/s(l

+ y2) (Exercise 3.6).

62

Chapter 3. The Fourier Transform A

(d) Let f (z) = e ? T z 2. Then f ( 7 ) = e-"-r2 (Exercise 3.7).

See Figure 3.3.

FIGURE 3.3. Example 3.3. Left: f (x). Right: f^(?).

Exercises A

Exercise 3.4.

Prove that if

f(z) = X[-,,,]

(z), then f

(y)=

sin(2roy) Tiry

3.2. Basic Properites of the Fourier Transform

Exercise 3.5.

Prove that if f (x) = (1 - Ix/al)X[-a.al (x):a

>

63

0. then

A

Exercise 3.6. If f ( z ) = e-"lzI, a

2a > 0, then f (y) = (a,+(2x712) .

= ~ c ( " ~ ) ~(Hint: / ~ Exercise 3.7. If f (x)= ePaz2, a > 0, then f^(?) See, for example, Kammler, A First Course in Fourier Analysis, PrenticeHall (2000) p. 132-133 for the a = 7r case.)

3.2 Basic Properties of the Fourier Transform In this section, we will present two basic properties of the Fourier transform of an L1 function. A

Theorem 3.8. I f f (z)is L' on R, then f (?) is unzformly continuous on R. Proof: Given 71: 7 2 E R,

1fix)l

e-2""~1-~z)x

-

1( dx.

Note that the last term depends only on the difference yl - 7 2 and not on the particular values o l yl a ~ 7~2 . dHer~ceto show unzform continuity on R. it is enough to show that P

We will use Theorem 1.41 t o do this. Since 1 - 1 15 2 ,

If

(x)1 le-2"iax

-

11 5 2 If

(XI/,

and 2 1 f (z)l is L1 on R since f (x)is. By Taylor's Theorem, given any A > 0,

for some [ t [-A, A] and all x t [-A, A].Therefore,

.

64

Chapter 3. The Fourier Transform

for all x E [-A, A]. Therefore, A

1 f (z)J/e-'""""

A

-

11 dx

5 5

Thus, for every A by Theoreni 1.41,

> 0, ( f (x)1 e-2""x

-

lirn 27r)al

a!-0

lim 2 ~ A l a l lfJ111 = 0.

a-0

11

+ 0 in L~ on

[ - A , A ] ,so that

The next Theorem is known as the Riemann-Lebesgue L e m m a and describes the decay a t infinity of f (7). A

Theorem 3.9.

(Riemann-Lebesgue Lemma) I f f (x) is L'

on

R, then

A

IY

lim f ( 7 ) = 0. l--too

Proof: We will present an outline of the proof. The details are left to the reader in Exercise 3.10. S t e p 1. Show that if f (x) = X [ a , b l ( ~ )then , (3.6) holds. This can be done by direct calculation. S t e p 2. Show that if f (x) is a step function of the form

for some coefficients c(n) and intervals [a,, b,], then (3.6) holds. Step 3. Show that if f ( x ) is C: on R, then given E > 0, there is a step function g(x) of the form (3.7) such that 11 f -g(Il < t. Then show that this implies that (3.6) holds for f (x) (cf. Exercise 5.26). S t e p 4. Show t,hat (3.6) holds for any function f (x), L' on R.

Exercises Exercise 3.10. Complete the proof of Theorem 3.9. (Hint: For Steps 3 and 4, use the estimate if^?) 1 5 11 f - 9/11.)

ls(-y)l+

3.3. Fourier Inversion

65

3.3 Fourier Inversion The purpose of this section is to investigate the conditions under which equality holds in (3.3). From the definition of the Fourier transform, we can write

where we have exchanged the order of integration in the double integral. This formal calculation is not valid, strictly speaking, because the integral

does not converge for any particular value of x or t. Nevertheless, this calculation provides a starting point for investigating (3.3). The idea will be to place a "convergence factor" in (3.9) so that it converges for each value of x and t ; that is, we write instead of (3.9),

A

for some function K ( x ) chosen so that its Fourier transform, K ( y ) , forces the integral in (3.10) t o converge and so that equality holds in (3.3) for

K(x). We now obtain

If K ( t ) is some element in an approximate identity, then

which gives us a valid approximate inversion formula for the Fourier Transform. It orlly relllairls lo choose arl approximate identity satisfying the required conditions. There are many valid choices, but a very convenient one is to let ,-.

In this case KT(?) = epm27' (see Exercise 3.7). and the same calculation shows that equality holds in (3.3) in this case, that is, that

66

Chapter 3. Tlie Fourier Transform

It is also easy to see that K,(z) is L' on R for each 7 > 0 and that {KT( x ) ) , , ~ is an approximate identity on R (Example 2.32(c)). Now we are in a position to prove the following theorem.

Theorem 3.11. If f (x) is C O and L1 on R, then for each z

E

R,

Proof: Repeating the calculation in (3.8), we obtain

But since {KT(x)),>o is an approximate identity on R, Theorem 2.33 says that

dt

=

f (4,

for each x E R . A

With an additional assumption on f (y), we can get equality in (3.3) in a pointwise sense.

Corollary 3.12. V f (x) is

SR

for each x 1: R,

CO

and L' on R, a n d iff^(?) i s L' on R, t h e n

f^(y) e2Ti7xdy = f ( : r ) .

(3.14)

Proof: By Theorem 3.11, it will be enough to show that

2

2

But since lim,,o+ e-"' 7 = 1, the proof amounts to justifying the interchange of the limit and the integral in (3.15). This is accomplished using Theorem 1.41 in a similar way to the proof of Theorem 3.8. We leave the details as an exercise (Exercise 3.14). Corollary 3.12 does not cover all the cases that will be of interest to us in this book. For example, in Example 3.3(a), we saw that if f (x) = X~-,,,](x), A

then f (?) = s i n ( 2 ~ a x ) / ( ~ c cIn) . this case, f (x) is L1 but is not continuous, and f^(x) is not L1, though it is L~ (Exercise 3.15). Therefore, neither

3.3. Fourier Inversion

67

Theorem 3.11 nor Corollary 3.12 apply. The answer is to replace pointwise convergence of the limit in (3.13) with L2 convergence. In this case, we have the following theorem.

Theorem 3.13.

If f (z) is L' and L2 on R, and ifT ( y ) i s L' on R, then liln 7+0+

JR~ ( yeTT272 ) e Z n a ydxy = f (x),

(3.16)

Proof: Since f (z) is L1, Theorem 3.18 implies that the function

is continuous on R and Theorem 3.21(a) implies that f T ( x ) is L1 for each r > 0. Since f (z)is L2, Theorem 2.36(b) says that f,(x) i f (x)in L2 on R as r i o+. Therefore,

Since f T ( x ) satisfies all of the hypotheses of Theorem 3.11, it follows that

) e2ni7xdy =

rw

2 y 2 f (?) e2rr-yr A

and (3.16) follows.

d 71

n

Exercises Exercise 3.14.

Complete the proof of Corollary 3.12.

Exercise 3.15.

Prove that the function f (x) = s i n ( ~ x ) / ( ~isz )L2 on R

but not L1 on R.

68

Cl~aptel-3 . The Fourier Transform

3.4 Convolution Definition 3.16.

G i v e n functions f ( x ) and g ( x ) , t h e convolution o f f (x) and g(x), denoted h ( x ) = f * g ( x ) , is defined by

,tuheneve7. the integral m a k e s sense.

Remark 3.17. (a) We have encountered integrals like (3.17) before, namely in thc dcfinition of approxirnate identity. There it was shown that under specific hypotheses on f (x), the integral

is a good approxiirlatiori to f (x) as long as {K,(z))T>o is an approximate identity.

The above observatioil can provide good insight into the action of convolution. Take, for example, the approximate identity defined by (11)

(Example 2.32(a)). In this case, we can see that for any function f ( x ) , the value of f * KT(xo)is just the average value of f (z) on an interval of length T centered at xo. If f (x) is continuous, then these averages are good approxirrlations to the actual point values of f (z). If we consider K T ( z ) = (llr)(1- I x ~ / T ) X,-,,,](x), T > 0, then f * KT(xo) can be interpreted as a "weighted average" of f (z) around the point xo, where points close to zo are given more "weight" than are points further from zo. Thus. the convolution f * g(x) can be interpreted as a "moving weighted average" of f(x),where the "weighting" is determined by the function g(x). See Figure 3.4. By changing variables, it can be shown that convolution is commutative, that is, that f * g(z) = g * f (x) (Exercise 3.22). Then f * g(x) can also be iriterpretecl as a nloving weighted average of g(x),where the weighting is determined by the function f (x). (c) If the function f (x) has large variations, sharp peaks, or discontinuities. then averaging about each point x will tend to decrease the variations, lower the peaks, arid smooth out the discontinuities. In this sense, convolution is often referred t o as a smoothing operation. A more precise statement of this idea is contained in Theorems 3.18 and 3.19.

3.4. Convolution

69

FIGURE 3.4. Illustration of convolution. Top left: Graph of f (x). Top right: Graph of g(x). Bottom: Integral of the product of the solid and dashed function is f * g(1).

Theorem 3.18. If f(x) is L" o n R, and volution f * g(x) is continuous o n R.

zf

g(x) is L' o n R, t h e n the con-

Proof: Given z, y E R,

If * 9 ( x ) - f * 9(y)l

=

li,f s S ,If

=

5

If llm

(t)(9(x - t )

- 9(y -

t))dt

(t)l Ig(x - t ) - 9(y - t)l dt

1 R

9 ( t - ('

-

Y))

-

9(t)l d t .

By Lemma 2.35(a) (continuity of translation for L' functions), lim

if * g(x) f * g(y)I 4 IIf I , -

X+Y

and the result follows.

lim X+?/

j

R

lg(t - (z - Y ) )

- g(t)l dt =

0%

70

Chapter 3. The Fourier. Trarlslurm If f (z)and g ( x ) are both L2 o n continuous o n R.

Theorem 3.19. f

* g ( x ) is

R, t h e n the convolution

Proof: Let e > 0. Then given x, y E R,we calculate as above, but this timc using thc Cauchy-Schwarz inequality,

(1I f

-

(

Idt)

'(

1' 2

9

-

(x - Y ) ) - li(tl12 dt)

R

By Lemma 2.35(b) (continuity of translation for L~ ~ u I I c ~ ~ o ~ ~ B ) ,

and the result follows. We have seen that the convolution of a bounded function with an integrable function and the corlvolutiorl of two L~ functions produces a continuous function. The next theorem addresses the issue of the decay at infinity of a convolution.

Theorem 3.20. ( a ) I f f( x ) and g(z) are both L1 o n

on

R,then the convolution f * g ( x ) is also L'

R,and

llf * 9111 2 Ilf 111 119111. (b) If f ( x ) is L1 o n R,and g ( x ) is L' o n R,t h e n the convolution f L2 o n

(3.18)

* g ( x ) is

R,and llf * 9112 5 llf 111 I d s

( c ) I f f( x ) and g ( x ) are both L2 o n

(3.19)

R,then the convolution f * g ( x ) is Lw o n

R,and

[If * d m 5 Uf ( d ) If f (x) is L"

on Lm o n R, and

R,and

g(z) is

112

119112.

L1 o n R, t h e n the convolution f

IIf * gIIm 5 IIf IIm IlgII1.

(3.20)

* g ( x ) is (3.21)

3.4. Convolution

71

Proof: We will prove (a) and (b) and leave (c) and (d) as exercises (Exercise 3.24). (a) Let f (x) and g(z) be L1 on R. Then

and (3.18) follows. (b) Let f (z)be L1 on R, and g(x) be L~ on R. By the Cauchy-Schwarz inequality,

Therefore,

and (3.19) follows.

Theorem 3.21.

(The Convolution Theorem) If f (x) and g(z) are L' o n R,

then fG-ig(7) =

37).

Proof: Let, f (z) and g(x) be L1 on R. Then

(3.22)

72

Chapter 3. The Fourier Transform f ( t )c~(z- t) e - 2 T i ~dt x dz =

/R IR

Exercises Exercise 3.22. 9

Show that if f ( x ) and g(x) are L1 on R, then f

* g(z) =

* f (4.

Exercise 3.23. Show t h a t under the hypotheses of Theorems 3.18 and 3.19, f * g ( z ) is actually uniform,ly continuou,s on R. Exercise 3.24.

Prove Theorem 3.20(c) and (d)

Exercise 3.25. (a) If f (x) and g ( z ) are compactly supported and L1 on R, prove that f * g(x) is also. (b) If f (z)and g ( z ) are compactly supported and L%n R, prove that f * g(x) is also.

3.5 Plancherel's Formula Theorem 3.26. (Plancherel's Formula) Iff (i-) is L' and L' o n R, t h e n f^(y) i s also L~ o n R and (3.23)

Proof: Define f(z) = f (-z). Then

3.5. Plancherel's Formula

73

where we have made the change of variable x e -z in the last step. Since f (z) is L1 and T , ~on R, so is By the Convolution Theorem (Theorem 3.21),

F(r).

A

A

f * .f'((ri

l2

= F(0)5(7)= I~^((T)) *

y(n:) *7(x)

Since f (z) and are both L1 on R, Theorem 3.20(a) implies that f *F(r) is also L' on R, and since f (x)and f x ( ( r ) are both L~ on R, Theorern 3.19 implies that f is continuous on R . Therefore, we can apply the Fourier inversion formula (3.13) and conclude t,hat for each x E R,

=

IR

f (t) f(x

- t) d t

Evaluating the above equality at z = 0 gives

It remains only to show that in fact,

We will do t,his in t,wa steps.

Step 1. We will show that

then

f^((y)

is L~ OII R by showing that if

74

Chapter 3. The Fourier Transform

contradicting (3.24) in light of the assumption that f (x) is L2. If (3.25) holds, then given any number 111 > 0, there exists a number A > 0 such tl~at

d m ) .But

whenever r > 0 is small enough (specifically, if 0 < r < this is exactly the meaning of (3.26). Therefore, f (y) is L~ on R. A

Step 2. Since f^(?) is L' on R, lf^(y)12is L1 011 R. We leave it as an exercise (Exercise 3.29) to prove that

From this, (3.23) follows.

A related result is the following formula. Theorem 3.27. R, t h e n

(Parseval's Formula) Iff (z) and g(x) are both L' and L~ o n

Proof: Exercise 3.30. One easy consequence of Theorem 3.26 is to sinlplify the statement of the L v o u r i e r inversion formula (3.16). Specifically, we no longer need to state explicitly the llypothesis that f ^ ( ( ~ is ) L2 on R since by Theorem 3.26 tjhis is a,i~t,orna,t,ic given the assumption that f (x) is L1 and L2 on R. Theorem 3.28.

(Theorem 3.13) I f f (z) is L' and L%n R, t h e n

3.6. The Fourier Transform for u unctions

75

Exercises Exercise 3.29. Complete the proof of Theorem 3.26. (Hint: Use Theorem 1.41 and Corollary 3.12.) Exercise 3.30. Prove Parseval's Formula (Theorem 3.27). (Hint: Consider the function g(x) = g ( - r ) , and repeat t,he a,rgllment in the proof of Theorem 3.26 with appropriate modifications.)

Exercise 3.31.

Prove that sin2( t )

dt = T ,

where the first integral is interpreted as

k i sin(t)

dt

= 2 1im )'+a2

sin ( t ) -dt

t

since sin(t)/t is not an Ls function. Hint: Prove the first equality by integrating the second integral by parts, and prove the second equality using Plancherel's Forrxlula and Example 3.3(a,).(See Benedetto, Harmonic AuaI.ysis and Applications, p. 25.)

3.6 The Fourier Transform for L2 Functions Until now, we have been rnaking the assunlption that a function f (x) must be L1 on R in order for its Fourier transform to be defined. This assumption was rnade in order t80gi~a~rantee that the integral in (3.4) converges absolutely for each y. However, we have seen examples that suggest that we need to expand the definition to a larger class of functions. Specifically,

y(?)

f ( L ) is L1 on R, but is not, and in order for equality to hold in both parts of (3.3), we would like to be able to make the statement that if

that is, that

The question is: How do we interpret the integral in (3.29) since it does not converge absolutely?

76

Chapter 3. The Fourier Transform

We have sccn the answer already in Theorem 3.28, which asserts in this case that

in L~ on R. That is, we interpret the nonconvergent integral (3.29) as a limit (in the L~ sense) of convergent integrals. The remaining question is: C a n we do this with any L2 function? The answer is "Yes," but the proof of this assertion is beyond the scope o l this book and involves knowledge of the theory of Lebesgue measure and the Lebesgue integral. We state the relevant theorem for completeness. C L T ~function f (x),L~ o n R, there exists a function Theorem 3.32. Gi~uer~ ,f^(?), L~ o n R (in the sense of Lebesgue), such that

iirn 7+0+

(Z)e-"'

2

2

' e2"77x d s = f ( i )

(3.31)

zn L~ o n R. I n this case, Plancherel's formula holds; that is,

and the Fourier inziersiorz holds in the sense of Theorem 3.28; that is,

3.7 Smoothness versus Decay One of the basic principles of Fourier Transform theory can be loosely stated as follows: T h e smoother f ( z ) is, the more rapidly f^(?) will decay a t infinity, and conversely, the more rapidly f decays at infinity, t h e smoother f (y) will be. There are many ways to measure the smoothness of a given function f (s), but for the purposes of this book, we will measure smoothness of f (x) by counting the number of continuous derivatives it has. We have already seen an illustration of this principle in Theorem 3.8, which asserts that if f (z) is L1 on R (a statement about its decay at infinity), then j^(-y) is uniformly continuous on R ( a statement about its smoothness). In light of the Fourier inversion formula (Corollary 3.12), we can assert that if an L1 function f (z) has an L' Fourier transform (decay of

(XI

A

3.7. Smoothness versus Decay

77

f^(?) at infinity), then f (z) is also uniformly continuous on R (smoothness of f ( x > ) . A more precise statement of this duality starts with the following theorem. Theorem 3.33. (Differentiation Theorem) If f (x) and x f (x) are L' o n R, t h e n f^((r) is continuously differentiable o n R, and

Proof: We wish t o show that for each y,

First, form the difference quotient for

Since

e-2~ihx

f^(?) and calculate.

-

1

lim

= -2nix, h the proof reduces t u justifying the interchange of a limit and an integral. Specifically, we must prove that h-tO

) dx. We will make two estimates on thc quantity ( l l h ) (e-2nihx - 1). First, we expand the function g ( h ) = e-2Tihx about h = 0 in a Taylor series and use Taylor's formula (keeping only one term in the expansion) t o obtain the estimate

Taking now two terms in the expansion, we obtain the estimate

5

I h 2'

-

max

olsih

1

d2 -e

ds2

x

n

2

x

2

(3.37)

Chapter 3. The Fourier Transform

78

Using (3.36), we estimate

By hypotllesis, 277- 1x1 If (x)l is L1 on R. Using (3.37), we note that for any R > 0,

as h 3 0. Therefore, by Theorern 1.41, the interchange of lirrlit and integral is justified and (3.34) follows. The following corollary to Theorem 3.33 can be proved by induction (Exercise 3.37).

Corollary 3.34. f^((y)

If f (z)and x N f (x) are L' o n R for s o m e N E N , t h e n is cN o n R , a n d f o r 0 5 j 5 N ,

We can state ;L partial converse of Theorem 3.33 relating srnoothrless of the Fourier transforrrl of a function to the decay at infinity of the function itself. A

Suppose t h a t f(x) i s L1 o n R, and t h a t following hypotheses.

Theorem 3.35.

( a ) F o r s o m e N E N, (b) B o t h ?(?) and (c)

f^((y)

is

f(?)

cN o n R.

jlN)(?) are L'

f?")(?) or o 5 j 5 N , I Iirn rl-)~

o n R. = 0.

Then Iirn zN f(x) = 0 IX+W

Proof: Consider the function F ( x ) defined by the integral

satisfies the

3.8. Dilation, Translation, and Modulation

79

Integrating by parts N times and using (a) and (c), we obtain

Using ( b ) and the Fourier inversion formula,

Hence F (z)= (2niz) f (x).By (b) and the Riemann-Lebesgue Lernrna (Theorem 3.9), lim,,*, F ( x ) = 0 and (3.39) follows. Finally, we present a theorern relating decay at infinity of the Fourier transform of a function to smoothness of the function.

Theorem 3.36.

If f ( x ) and?(?) are L' o n R and i f T N a y ) is L' o n R for some N E N , the,,&J ( z ) i s c No n R, and for 0 j <: N ,

<

Proof: By the Fourier Inversion Formula (Corollary 3.12), we car1 write f ( x )=

~ ( y~~~~7~ ) dy.

Equation (3.40) follows by repeated application of the argument in the proof of Theorem 3.33. The fact that f ( s )is C N on R follows since ?Jf^(?) is L1 on R and b y applying the argurrlent of Theorern 3.8.

Exercises Exercise 3.37. Prove Corollary 3.34 Exercise 3.38. Prove that if f ( x ) has period 1 and is C K on R, then there is a constant A > 0 such that for all n E Z: Ic(n)l < A J n l - K ,where c ( n ) = J; f (z) e-2T"i"" dx.

3.8 Dilation, Translation, and Modulation Definition 3.39. Given a > 0, the dilation operator, D,, defined o n functions f (x), L' or L' o n R, is given b y

80

Chapter 3. The Fourier Transform

Given b t R, the translation operator, Tb, defined o n functions f (x), L' o r L' on. R, is giuen by (3.42) Tbf(x) = f ( x - b). Given c E R, the modulation operator, Ec, defined o n functions f (x), L I o r L~ o n R, is given b y E , f ( z ) = e2""'" f ( ~ ) . (3.43)

Theorem 3.40.

For any function f (x) L' o n R, A

( a ) For every a > 0 , Daf (7) = ~ 1 / ~ 7 ( 7 ) .

Proof: Exercise 3.44. Remark 3.41. (a) Note that if a > 1, then D, f (x) is a "narrowed down" version of f (x), and if 0 < a < 1, then D, f (x) is a "spread out" version of f ( x ) . Theorern 3.40(a) says that when a function is "narrowed down" by dilation by a > 1, its Fourier transform is "spread out" by dilation by 0 < l / a < 1.

(b) Theorcrrl3.40 says that modulatiori in the time or spatial variable corresponds to translation in the frequency variable. For this reason, modulation is often referred to as a frequency shift or phase shift. Theorem 3.42. (Properties of Dilation and Translation) For every f ( z ) and g ( z ) , L~ o n R, and for e7rery a > 0 , b E R, the following hold. (a) D,Tbf(z) = a"2 f ( a x - 6 )

(e)

( f , DaTb9) = ( T - b D a - l f ,

( f ) (Dof , Dagj =

(f

9

9).

g).

Proof: Exercise 3.45. Theorem 3.43. (Properties of Translation and Modulation) For every f (x) and g ( z ) ,L~ o n R, and for every b, c E R , the following hold.

3.9. The Sampling Formula

(a) TbEcf (x) = e

-27rzbc

(b)

( f , Ecgj

(c)

( f , TbEcg) = e

81

EcTbf (x).

= ( E - c f ,g ) . 27rzbc

(T-bE-,f. g ) .

Proof: Exercise 3.46.

Exercises Exercise 3.44. Prove Theorem 3.40. Exercise 3.45. Prove Theorem 3.42. Exercise 3.46. Prove Theorern 3.43.

3.9 Bandlimited Functions and thc Sampling Formula Definition 3.47. A function f ( x ) , L'

on R, is bandlimited zf there is a number 0 > 0 such that f^(y) is supported in the interval [ - R / 2 , 0 / 2 ] . In this case, the function f (x) is sazd to have bandlirnit 0 > 0. The function f (x) has bandwidth B > 0 if there is an interval I such that II/ = B and such that f (7)is supported in I . A

Remark 3.48. (a) A furlclivn J(x) with bandlimit R > 0 also has bandwidth R > 0. However, in general the numbers are not the same. For example, let f (x) be the function whose Fourier transform f^(?) equals (?). (Whet is f (x) in this case?) Then the bandlirnit of f (x) is 2, whereas the bandwidth is 1.

(b) The bandlimit and bandwidth of a function f (x) are not unique numbers. For example, iff (x) has bandlimit fl > 0, then f (x) also has bandlimit 0' > 0 for any number 0' > fl. Similarly, if f (x) has bandwidth B > 0, then f (x) also has bandwidth B' > 0 for any number B' > B. (c) Intuitively, if f (x) is bandlimited, then f (x) does not contain arbitrarily high-frequency components. The Fourier inversion formula for a function with handlimit fl looks like

82

Chapter 3. The Fourier Transform

That is, f (z) consists only of "frequencies" e2"irz of period 2/R or greater. Thus, one might expect that a bandlimited function would be slowly oscillating and not have any sharp jumps or disconlinuities. In fact, the following theorern holds.

Theorem 3.49. Let f (x) be a bandlimited function with bandlimit R . Then: (a) T h e Fourier inversion formula holds for f (x);that is, for each z E R,

(b) f (x) is C" o n R.

Proof: We will prove (b) first, given the assumptiori that (a) holds. We would like to use Theorem 3.36, since f (x)being bandlirnited iniplies lllat f^(?) is L1 arid that rN is L' for every N F N. However, since we have only assuriied that f (x) is L2 on R and not necessarily L1, we carinot use the theoreni directly. However, if we examine the proof of Theorern 3.36, we see that all that is required is that the Fourier inversion formula hold. Then the argument in the proof of Theorem 3.33 may be applied. But this is exactly (3.44). In proving (a), we agairi run into the difficulty that f (z) has been assumed only to be L2 and not L1 on R. This is certainly not an insurmountable obstacle, but it does require some rather subtle argumentation. According to Theorem 3.32, the Fourier inversion formula holds in the L2 sense for f (x);that is,

y(?)

in L2 on R. By Plancherel's Formula for the L2 Fourier Transform (3.32), we know that f^(?)is L2 on R Since f ( y ) is also compactly supported, Theorem 1.9 says that is also L1 on R. Therefore, we can prove (see Exercise 3.51) that in fact, A

y(?)

lim

r+o+

Y,

in L" on R. Let us call this uniform lirrlil function g ( x ) ; that is,

Thus, we have an L2 limit function f (x) and an L" limit function g ( x )for the same sequence of functions. So we must show that in fact, these limit functions are the same.

3.9. The Sampling Formula

83

In order to do this, define the functions f, (a)by

and fix a number A

> 0.Then by Minkowski's inequality,

The left side of the inequality is independent of r , and the right side can be made as srnall as desired by choosing r > 0 srna.11 enollgh. Therefore for every A > 0, A

which implies that

If

(x)- 9(2)12dz

= 0.

Since f (x) is piecewise continuous by assumption, and g(z)is in fact continuous by the argument used in the proof of 'l'heorem 3.8, f (z) = g(z), except possibly at the discontinuities of f (x).Since there is no problem redefining f (x)at these points, we can conclude that f (x) = y(x) for every x E R. But this is (3.44). One of the fundamental results in Fourier analysis is the Shannon sampling theorem. The theorern asserts that a bandlimited function can be recovered from its sarrlples on a regularly spaced set of points in R, provided that the distance between adjacent points in the set is srnall enough. The formula is also very important in digital signal processing applications. Theorem 3.50. (The Sharlrlorl Sarrlplir~gTheurem) I1 S ( x ) with bandlimit Cl, t h e n f (x)c a n be w r i t t e n as

where the s u m converges in L2 and L"

,is b~7idli,11L%led

o n R.

'This theorem has a long and interesting history that is recounted beautifully in the article by Higgins, Five short stories about the cardinal series, Bulletin of the AMS, vol. 12 (1985) 45-89.

84

Chapter 3. The Fourier Transform

Proof: Since f^(y) is supported in the interval 1-R/2, fl/2], we can expand f^(y) in a Fourier series and obtain

for y E [-fl/2,fl/2], where

But by (3.44), it follows that

Making the change of summation index n

+ -n

leads to

Again applying (3.44), we obtain

where we have used the fact that Fourier series can be integrated term-byterm (Exercise 2.67) and that for any numbers a > 0 and b # 0,

(Exercise 3.4). To see that the convergence of (3.47) is uniform on R, let N, M E Z be fixcd.

3.9. The Sampling Formula

85

where we have used the Cauchy-Schwarz inequality and where the c(n) are t,he Fourier coefficients of f^(y). But since the Fourier series of f (?) , . coriverges to f (y) in L~ on [-012, fl/2], A

The L2 convergerice of (3.47) follows from the fact that the collectio~i

is an orthonormal system on R (Exercise 3.52).

Exercises Exercise 3.5 1. Prove equation (3.46). Exercise 3.52. Prove that the collectiorl

is an ortllonormal system on R. (Hint: Use Parseval's formula.)

Exercise 3.53.

x E R,

Show that if f (x) has bandlimit fl > 0, then fbr every

86

Chapter 3. T h e Fourier Transform

Exercise 3.54. Show that if f ( x ) has bandwidth B > 0, then f (x)can be completely recovered from the samples { f (TL~B)),,~. Exercise 3.55. Let f (x) be L~ on R, and let r > 0 be given. Prove that there exists a handlimited function g(x) such that $(y) is C0 on R and 11 f - g1I2 < r. (Hint: Use Corollary 2.37(b) in the Fourier transform domain.)

Chapter 4

Signals and Systems In the previous chapter, we considered piecewise continuous functions with period 1 and showed that it is possible to represent such functions as an infinite superposition of exponentials en(t) = e2"int, n E Z. Each such exponential has period l / n and hence completes n cycles per unit length (which we can interpret as measuring time). If the exponentials are interpreted as "pure tones" of n cycles per second, then each f (t) has a "frequency representation" of the form

where

We also know that

Conceptually, there is nothing stopping us from changing our perspective and regarding the sequence { f (n)InEz as the object to be given a "frequency representation." In this case, (4.2) is such a representation in ,. which we consider f (n) to be a coritinuous superpositiorl of "pure tones" on Z, e,(n) = e2T"Lx,which complete about one cycle every l / x time steps (indexed by r ~ ) Equation . (4.1) now gives a forrnula for the coefficients in this continuous frequency representation. This new perspective is very well suited for digital signal processing (DSP) applications in which data are necessarily in the form of arrays of numbers. These arrays are of course always finite but can be of arbitrary length. Hence, it is convenient t o regard these objects (to which we will give the name signals) as being infinite sequences. A related perspective regards signals of length N as periodic sequences with period N . This chapter provides a discussion of some of the ternlinology and basic results of the mathematical theory of DSP from both perspectives. A

88

Chapter 4. Signals and Systems

4.1

Signals

Definition 4.1. A signal is a sequence of numbers {~(n)),~z satisfying

Remark 4.2. By basic results on convcrgcnt scrics, any signal must be bounded, that is, there is a number M > 0 such that /x(n)1 < M for all rL E Z. It is also true that any signal satisfies C , Ix(n) 1' < m. Such sequences are said to be (Exercise 4.6).

Definition 4.3.

t2 sequences or somctimes to have finite energy

The frequency domain representation of a signal x(n)is the

function

Remark 4.4. (a) Since En 1x(n)1 < a, the sum defining ?(w) converges ilniformly to a continuous function with period 1.

(b) Recall that the set {e2""}w,,[o,I) is the unit circle in the complex plane. This is because if z = e2"7w,then lzl = 1. Hence the function X(eZKLW) car1 be tlwught of as the restrictioii t o tlie unit circle of some function X ( z ) defined on some portion of the conlplex plane containing the unit circle. Specifically, we can define

wherever the surn makes sense. The function X ( z ) is referred t o as the z t m n s f o r m of z(n,).

Example 4.5.

Also,

(a) Let z ( n )=

1 ifO
Then,

4.1. Signals

89

See Figure 4.1. (b) Let la( < 1, and let

Then,

and CU

X(z) =

C a"

z-n, =

r~=0

whenever lzl

1 1-az-I

-

Z

z-a

> lal. See Figure 4.2.

Exercises Exercise 4.6. Show that every signal is an f 2 sequence and that there are f 2 sequences that are not signals.

90

Chapter 4. Signals and S y s t e m s

FIGURE 4.2. Left: x ( n ) of Example 4.5(b). Right: the real part of z ( w ) .

4.2

Systems

Definition 4.7. ( a ) A s y s t e m is any transformation T that takes a n input signal x ( n ) t o a n output signal y ( n ) . We write T x ( n )= y(n). ( b ) A system T i s linear if

for every pair of signals

XI

( n ) and

5 2 ( n ) ,and

every pair of constants a and b.

( c ) A linear system T is stable i f there is a constant C signals x ( n ),

>

0 such that for all

( d ) For ~ z oE Z , ,we d e f i r ~ el l ~ eLrarlslatiorl operator, rn,,, by

Tn,,x(n)= x ( n - n o ) . ( e ) A linear translation-invariant ( L T I ) s y s t e m is a linear s y s t e m T for which

T(rnox ) ( n )= T~~ ( T x )( n )= T x ( n - n o ) . ( f ) T h e convolution of two signals szgnal y ( n ) given by y(n) = xl

rl

( n ) and x z ( n ) , denoted

* x2(n) =

C x 1 ( I C )x Z ( n

-

li).

XI

* x z ( n ) , is

the

4.2. Systems

91

Theorem 4.8. (a) If x1 ( n ) and x z ( n ) are signals, then so is y ( ~ z ) = X I * x2 (7.1).

( b ) For any pair of signals

XI

( n ) and

2 2 ( n ),

(c) Let h ( n ) be a signal, and define the transformation Th o n signals b y

T h e n TI, is a stable LTI system.

Proof: (a)

(b) Exercise 4.25. (c) By (a), whenever z ( n ) is a signal, so is Thz(n), so that Tjzis a system. That Th is linear is a simple verification (Exercise 4.26). That Th is stable follows from (a) by taking C = C , Ih(n)1. To see that Th is LTI, fix no E Z. Then

by a change of summation index.

Remark 4.9. (a) The r~otiorlof stability defined in Definition 4.10(c) is another way of saying that T is continuous in the sense that a small change in the input z ( n ) results in only a small change in the output y(n) = T z ( n ) .

92

Chapter 4. Signals and Systems

(b) To be more precise about what this means, we define a n o r m on the class of all signals (called the tl-norm) by

We measure the distance between two signals z l ( n ) and z;?(n)as ((zlzz( ( e l ,so that a small change in z ( n ) (to say Ic(n)) means that - lcllPl is small. In this notation, the inequality in Definition 4.10(c) can be written as

IITzIlel I C llll:lel. (c) We can now give a precise definition of continuity as follows: Given > 0, there is a S > 0 such that whenever lllc < 6, IlY'z - T.Zllel < 6 . An equivalent definition (Exercise 4.27) is the following: Let {zk ( n ) ) k , ~ be a sequence of signals converging t o the signal z ( n ) in the sense that 1l.z. - xkllp -+0 as k + m. T h e n T z k ( n ) convenjes to T z ( n ) in the sense that llTz - Tlckllpl + 0 as k + m.

zllel

E

Our next goal is to show that all stable LTI systems have the form Th for some signal h(n) (Theorem 4.12(c)).

Definition 4.10.

T h e unit impulse signal 6 ( n ) is defined b y

See Figure 4.3.

Lemma 4.11. A n y signal x ( n ) can be written as a s u m of shifted impulses; that is, for each n E Z,

Proof: Exercise 4.28. Theorem 4.12. Let T be a stable L T I system. T h e n there is a signal h ( n ) such that

T s ( n )= ( x * h ) ( n )=

a ( k )h ( n - k ) .

(4.5)

kEZ

Proof: Define h ( n ) = (Tb)(n). Since T is a system, h(n) is also a signal; that is, C , J h ( n ) J< m. Given a signal z ( n ) , we can write z(n) =

4.2. Systems

93

FIGURE 4.3. The unit impulse.

C ~ Ex(k) Z rk6(n) by

(4.4). Since T is a stable LTI system,

( T z )(n)

~ ( k( )T T ~( n~ ))

= k€Z

which is (4.5). The assurnptions of stability and linearity are required to justify pulling T inside an infinite sum like we did above. The details for this justification are the content of Exercise 4.29.

Definition 4.13.

Given a stable LTI systern, T , the signal h ( n ) such that T x ( n ) = ( x * h ) ( n ) is called the impulse response of T. T h e impulse response of a stable L T I system i s often called a filter. The frequency representatzon of h ( n ) , h ( w ) , is called the frequency response of T, and the z-transform of h ( n ) ,H ( z ) , is called the system function of T .

A

Remark 4.14. (a) Referring to h ( n )as an impulse response makes sense because according to the proof of Theorem 4.12, (TS)(n)= h ( ~ z )so, that h(n) is the response of the systenl to a unit impulse. (b) That x ( w ) is called the frequency response also makes sense for the following reason. Suppose that the input to the system T was a "pure frequency" z ( n ) = e2"inwo. Even though z ( n ) is not a signal according to our definition, it still makes sense to form the convolution (z* h ) ( n ) .

94

Chapter 4. Signals and Systems

Therefore,

( T s )( n )

s ( k )h ( n - k )

kEZ

x x ( n - k)h(k)

=

k€Z

C

-

e 2 x i w ~ ( n - k )h

k€Z -

e2ninw"

C h(k)

(k) e-2xikwU

k€2 -

e2xinwo

IL(WO)

A

In other words, thc signal z ( n ) passes through the system T unchanged ex,--. cept for multiplication by the constant h ( w o ) .We say that pure frequencies ,-. are eigenvectors of LTI systems with eigenvalues given by h ( w ) . (c) A similar calculation is valid when dealing with real valued signals and justifies some further terminology. Suppose that the input to the system T is a real sinusoid of the form

Moreover, by (b),

Assuming that h ( n ) is real valued, we have that

(Exercise 4.30), so that writing

it follows that Therefore,

A

A

A

h(-wo) = h ( w a )= I h ( w o )1 e - ' Q ( W o ) .

4.2. Systems

95

In other words, the signal z ( n ) passes through the system T unchanged except for multiplication by the real constant Ih(wo)l and a phase shift by O(wo). We refer to lx(w)l as the m a g n i t u d e response of T and to B ( w ) as the phase response of T . A

Remark 4.14(a) can be thought of as a particular case of the following theorem.

Theorem 4.15.

Let

XI

( n ) and x z ( n ) be signals, and let y ( n ) =

* xz)(n).

(zl

Then

G(W)=

A

J;1

( w) $2 (w) .

Proof:

=

z, ( k )z 2 ( n - k ) e-2"zn,w

zl (k)

=

i a ( n- k ) e-2T'n"

Corollary 4.16. Let T be a n L T I system with frequency response x ( w ) . T h e n for every signal x ( n ) , T z ( w ) = z ( w )h ( w ) . h

4.2.1

Causality and Stability

Definition 4.17.

A stable LTI system T is causal

satisfies h ( n ) = 0 for n

2f

its impulse response h ( n )

< 0.

Remark 4.18. (a) The system function for a causal system has the form

which is a power series in z - l . Consequently, wc can talk about the radius of convergence of H, and note that if R > 0, then H ( z ) converges uniformly for 1x1 > R - l .

96

Cliapter 4. Signals and Systems

Obviously, if h(n) is a causal FIR filter (Definition 4.24), then the system function H ( z ) is a polynomial in z-I. (b) If T is causal, then given r ( n ) ,

(Ts)( n )

z ( k ) h ( n - lc)

= kEZ

~ ( nh(0) ) + z ( n - 1)h(1) + z ( n - 2) h(2)

=

+ ....

Hence the output y ( r ~ )at n = no depends only on the input z ( n ) at n = no, no - 1, no - 2, . . .. If we imagine that our input signal z ( n ) is some time series, then T being causal means that the output q ( n o )does not depend on the "future" values of the input (that is, x ( n ) , n > no) but only on the "past" values of the input (that is, s ( n ) , n 2 n o ) In this sense, any realistic systeni rnust be causal. Definition 4.19. A system T is realizable zf the relation between the input x(n) and the output y(n) of T is given by a n equation of the form

for each n E Z

Remark 4.20. (a) Under the assuniption that a(0) note that (4.6) car1 be written for each n as y(n)

=

b ( O ) . x ( r ~ ) + b ( l ) z ( n -1)

+

- a ( o ) ' ( a ( l ) y(n - 1)

+ - - . + a ( K ) y(n

#

0 and b(0)

# 0,

+b(A4)x(n-hf)

+ 4 2 ) y(n - 2)

-

K)).

In other words, the output y(n) a t n = no depends on the current value of the input z ( n a ) and the values of the input M steps in the past, and on the values of the output K time steps in the past. Of course, the output of such a system at times n = 0, 1, . . . will differ depending on the K "initial values" of the output, y(-1), y(-2), . . . , y(- K ) . These values may in principle be computed knowing the values of the input s ( n ) infinitely far in the past, but as a practical matter, it is usually assumed that y(-1) = y(-2) = - . . = y(-K) = 0.

(b) Note that each side of (4.6) is a convolution with the finite filters a ( n ) and b(n). Thus, (4.6) can be rewritten as (a * Y) (n) = (b * z)(n) . By Theorem 4.15,

4.2.Systems

97

Hence the system function for T is given by

where

B ( t )=

x

x --

b(m)z P m

and

A(z) =

a(k) z p k

so that R(x) is a rational function of x-l. (c) Note that if R ( r ) is a rational function in z-', it is clearly also a ratiorial lu~lctionin t . This is true since z N B ( z ) and z N A ( r ) are polynomials in z as long as N 2 max(M, K). Therefore, the system function of a realizable system is completely determined by its poles (the zeros of its denominator) and its zeros (the zeros of its numerator). (d) We know that given n rational function R(z) in wlliclr the degree of the denominator is greater than or equal to the degree of the numerator, we can write the rational function R(z)/z in a partial fraction expansion as follows:

where the pi are the poles of R ( t ) / z , m, is the multiplicity of each pi,and

(e) Next note that if a

# 0, then

whenever 1x1 > la/. Taking the derivative with respect to a , we get

Therefore, whenever 1x1 > /a/,

98

Chaptcr 4. Signals and Systems

Hence we can write

which will converge as long as lzl > max{lpi I).

By the above calculations, we have nearly proved the following theorem.

Theorem 4.21.

If the system function, R ( z ) , of a realizable system T has all of its poles inside the unit circle of C , then T i s causal and stable.

Proof: It follows from Remark 4.20(e) that T is causal with impulse response given by

To show that T is stable, we rnust show that En Ir(n) 1 note that r ( n ) is a finite surn of terms of the f o r n ~

< m. To do this,

and that for each i and j : (n+l?!

$+'-J

j!qn + 1 - j)! n! Ai,j j!(n - j ) ! P?

as n + m . If pi1 < 1 for each i , then for each i and j, and it follows that C , Ir(n) I < m .

Example 4.22.

(a) Let

C , Ir(n,i,j)l < m ,

4.2. Systems

99

This rational function has first-order poles at x = 1/2 and z = 3. Expanding R(x)/x in a partial fraction expansion, we obtain

and

Therefore, T is causal but not stable.

(b) Let

-

This system function has a first-order pole at z = 1/2 and a second-order pole at x = 1/3. Expanding in partial fractions, we obtain

Therefore, T is both causal and stable.

Remark 4.23. (a) Note that in the above example, the impulse response of each systcrrl described is infinite in length, so that it direct computation of T x ( n ) fur some signal x ( n ) would in principle require infinitely many calculations. In fact only finitely many calculations are required since Tx(n) can he realized as the solution to the finite difference equation (4.6). (b) The infinite lengths of the impulse responses of the systems in Example 4.22 arise from the fact that the system function has nonzero poles. A pole at t = 0, cven of high order, will not result in an irlfirlite impulse response. Such a system is called a delay (Exercise 4.31). This leads to the following definition.

Definition 4.24.

The i17zpulse Y ~ S ~ O T of L Sa~ realizable system whose system function has n o poles except possibly at z = 0 is called a finite impulse response (FIR) filter and one with nonzero poles inside the unit circle is called a n infinite impulse response (TTR)filter.

100

Chapter 4. Signals and Systems

Exercise 4.25.

(a) (xl

Let x1 (n), x2(n), and h(n) be signals. Show that

+ x 2 ) * h(n) = X I * h(n) + 2 2 * h ( n ) .

(b) If a is any number, then ( a 21) * h(n) = a (xl* h) (n).

Use (a) and (b) above to show that any system defined as in Theorem 4.8(c) is linear. Exercise 4.26. Prove Theorem 4.8(b). Exercise 4.27. (a) Show that the two definit,ions of continuity given in Remark 4.9(c) are equivalent. That is, show that a systern T satisfies the first definition if arld only if it satisfies the second.

(b) Show that a stable linear system is continuous by showing that such a system satisfies either definition of continuity given in Renlark 4.9(c). (Hint: If you use the first definition, then given 6 > 0, take S = E I C . ) Exercise 4.28.

(a) Prove Lemma 4.11 by showing that for each n t Z, s(k)S(n-k) =x(n).

lirn

N-+co

k--N

(b) Prove that in fact (4.4) holds in the following strong sense:

lirn

N+m

1Ix(n) k=-N C z(k) b(n -

-

k) = 0.

~LEZ

In the notation of Remark 4.9(b), we are being asked to prove that lim IIx - :xNllel,

N+m

where

N

Z N ( ~= )

C

x(k)h(n - k).

k=-N Exercise 4.29. (a) We say that x ( n ) is a finite signal if there exist numbers N, M E Z such that z ( n ) = 0 if n, < I\/[ or n > N. In other words, x ( n ) is a finite signal if it has only finitely many nonzero entries. Prove that, given a signal x(n), there is a sequence of finite signals {xk ( n ) j k E N such that lirnk,, Ilx - xlc = 0.

(b) Note that all calculations in the proof of Theorem 4.12 are legitimate for finite signals ~ ( n ) Show . that the stability of T implies that in fact, Tx(n) = (x * h)(n) for all signals x ( n ) (finite or not).

4.3. Periodic Signals and the DFT

101

A

Exercise 4.30.

Prove that if h ( n ) is a real valued signal, then h(-ci?) =

A

h(4. , Exercise 4.31. Show that if R ( s ) = s P mm system T is givcn by T x ( n ) = z ( n - n ~ ) .

> 0, then tlie correspoiiding

Periodic Signals and the Discrete Fourier Transform A different inode1 for thinking lnathenlatically about finite sigrlals is to corisider the finite signal to be infinite in length but periodic. Iri other words, given a finite data set of leligtli N , ( ~ ( 0 )x(1). . . . . , . r ( N l ) } .define a corresporidirig irifi~iitesequence F(n),n E Z, by F ( n ) = . c ( r ~11iod N ) . Iii this case, . c ( n ) = J:(TI,)wlierlever 0 < rl, < N , so that .?(?I,) is considcrctl an extension of

n.(~i,).

FIGURE 4.4. Left: A signal tension F(n).

~ ( 7 1 )of

length 5. Right: its period-5 ex-

Definition 4.32. G i v e n N E N , a sequence { ~ ( n ) tz ) , ,is a period N sigrial "x(n+ N ) = z ( n ) t o r all 7~ g Z . I n this case x(n) is said to be periodic. Remark 4.33. (a) It is clear that a period N signal, unless it is identically zero, can never be a sigrial in t,he sense of Definition 4.1, since the absolute values of its entries will always surn t o infinity. However, a periodic signal is bounded in the sense that there is a number A f such that I.c(n)1 A1 for all n E Z.

<

102

Chapter 4. Signals and Systems

(b) Even though a periodic signal is never a signal in the sense of Definition 4.1, it is always possible to pass a periodic signal through a stable LTI system and have the result make sense. Since stable LTI systems are characterized by their impulse response filter, this statement amounts to the statement that the convolution of a filter and a periodic signal is well defined. This is the content of the following theorem. Them-err1 4.34. Given u filler. h(71) a n d u. period N signal x(n),t h e convolut i o n x * h(n) i s defined for all n a n d i s a period N signal.

Proof: That z* h ( n )= CktZ ~ ( kh )( n - k) is defined for all n amounts to showing that the sum converges for each n. But this follows from the fact that for sorne M, Ix(n)l5 M for all n and the calculatiorl

To see that z * h,(n) has period N, note that

=

z ( k ) h(n - (k - N))

since x(n)has period N. Since filtering operations are defined on periodic signals, it makes sense to look for an interpretation of such an operation in the frequency dorrlairl analogous to Corollary 4.16. Consequently, we need a notion of frequency representation for periodic signals. This is done via the the Discrete Fourier Transform (DFT).

4.3.1

The Discrete Fourier Transform

Definition 4.35.

G i v e n a period N signal x(n),the (N-point) Discrete Fourier i s t h e period N sequence

T~-ar~slor.rr~ ur. (N-point) D F T of z ( n ) , denoted

defined b y N-1

z(n),

4.3. Periodic Signals and the DFT

103

The DFT is invertible as follows.

Theorem 4.36.

G i v e n a period N sequence x(n)with D F T ;(n),

for each j E Z .

Proof: Note first that for any nurnber r ,

/ir

l-r

'

n=O

so that N-1

In

n=O

Hence, for 0 I j _< N

-

Therefore, for each 0

<j
-

1- e 2 ~ i ( . i - k ) / N

'

1,

-

1,

N-I

Since the sum

1-e2rd~-k)

N-1

N-1

defines a period N sequence in j , the above holds for all j E Z. We can now prove the following theorem relating the DFT of a periodic signal to the DFT of a filtered version of that signal. Theorem 4.37.

L e t h ( n ) be a filter a n d x(n) a period N signal. T h e n

Chapter 4. Signals and Systems

104 where

z(n)is the D F T of z ( n ) and x ( w ) is given by Definition 4.3.

Proof:

,.

Remark 4.38. (a) Notice that since h(w) has period 1, the sequence h(n,/N) has pcriod N . Therefore, it is reasonable to ask if there is some period N signal h(n) wllose N-point DFT is X(n/N), and if there is such a periodic signal, how is it related to the filter h,(n)?

A

(b) To find the period--N signal h(n,),all we need to do is take the inverse DFT of ;(n/N). This gives

BY (4.91, N - l

N - l e2n"".(-~)/N En=,

0 otherwise. Therefore,

= 1 if j -

k. = ,N

for some

V Lt

Z and

4.3. Periodic Signals and the DFT

We say that

h(n)is the

-

105

N-pcriodizntion of h ( n ) .

(c) In terms of h(n), we can write the formula for the convolution of x(n) with h ( n ) as follows.

Since z ( n ) has period N , x ( j - m N ) = x ( j ) for all m and we continue

This suggests the following definition of the convolutiori of periodic signals.

Definition 4.39. Let x(n) a n d y ( n ) be period-N signals. T h e n the circular convolution of x ( n ) a n d

~ ( ' I L ) is

defined b y N-I

x * y(n) =

Remark 4.40. cise 4.42(a)).

(a) Note that x

x ( k ) y ( n - t).

-c ~ ( 7 7 ; ) is

also a period-N signal (Exer-

(b) The sum in Definition 4.39 can be obtained by surrlnlirlg over any N adjacent indices. To see this, let j E Z be given, and let m be the unique integer such that (m - I)N 5 j < m N . Then,

106

Chapter 4. Signals and Systems

Frorn this it can he shown that x * g(n) = g * x(n) (Exercise 4.42(\1)). (c) Circular convolution car1 be realized as nlultiplicatiorl by a matrix whose rows are shifts of one anotlier. Giver1 a period N sequence z ( n ) ,define the matrix X by

If y has period N and r (n) = x * :q(n),define y = [g(0) r = [r.(O) . - . r ( N - I)]. Then r = Xy.

Theorem 4.41.

. . . y (N- I)] and

L e t x(rr) a n d ? / ( r ~he ) period-N sign,al.s, a n d let F ( n ) a n d c(n)

be t h e l r DFTs; t h e n (3:

whew (x * y)"

(TI)

* !J)"

(11)

d e n o t e s the D F T of

=~

:L:

I

L ~ )( T L ) ,

* IJ(TL).

Proof: Exercise 4.43.

Exercises Exercise 4.42.

Let x(n) and y(n) be period-N signals.

(a) Prove that thc circular convolutiorl x * y ( n ) is also a period-N signal. (b) Prove that z

Exercise 4.43.

* y(n) = y * x(n).

Prove Theorem 4.41.

Exercise 4.44. Prove that if 2(n)is the N -point DFT of the period-N signal x (n,), then N-1

N-1

4.4. The Fast Fourier Transform

107

The Fast Fourier Transform

4.4

The N-point DFT can be thought of as a linear transformation on the finite-dilnensional vector space C N ,and hence can be writtcrl as a matrix. That is, given the vector

its DFT,

can 1)e writtcn

Tlie matrix WN dofirled 1-y (WN),,,i= where CI:, = syrnrnctric, ort2logoiia1, and

w$'.

is

Tlic irlversc DFT can he writtc.11

where Wh is the atljoint of WN. Notc tliat all of the cnt,ries in tlie rnatrix WN have absolute value 1. This illearis that to coinpute the DFT of a period N sequcnce . c ( n ) by doing straight rnatrix in~lltiplicationwill require N%ri~llti~lications. This niakcs the tlircct iinplerrlcritatiori of the DFT inipractical for large N . Fortllrlately therc is an algorithrrl known as the Fast Fo7~rler.Tmn,sfonn,or F F T t,h;~t, s p ~ p d su p thc? co~~lputatiorl of thc: DFT c~rlsiclorabl~.~ Tlie idea hehincl the F F T is the following. Lot N E N witli N even. and \Vnr = ?-"dlN. If N = 2 A f , t,heil

Now. given ;-L period N scqlierlce .c(2j 1). Tlleri

+

.C(TL)

(lefiiie ( ~ ( j )= x ( 2 . j ) a~lclb ( j ) =

'The following discussion is adapted frorr~Papolilis, Signal Analysts, hlcGraw-Hill (1977), p. X2ff.

108

Chapter 4. Signals and Systems

where Z ( n ) and a(n)are the W-point DFTs of n and b respectively. Note further that

so that if 0 5 n 5 A1

and if hf 5 r~ 5 N

-

z(71)

-

1 , the11

1, then

1 )+ 1

(71-hI)

=

(

=

ii(7~+ ) win' w;i(rL)

1 -

-b(n

-

hl)

A

=

a ( ri,)

A

-

W; b (n),

A

rernern1)cring that Z ( n ) and b ( n ) have period h l . In inatrix notation, the above discussioll can be sumrnarizcd by the fol. . . ~ ( A I- I ) ] . lowing. Let S = p(0) 6(1) . - . Z ( h 1 - I ) ] and = Tlleil

6(0)Z(l)

(4.10) where flnl is the A4 x h! diagonal matrix giver1 by

and PN is the N x N permutation matrix that separates the even and odd entries of a vector. For example,

4.5. L~ Fourier Series

109

The efficiency of the F F T comes from the fact that the dense rnatrix WN can be factored into a product of sparse matrices as in (4.10). If Ad is a,lso even, then Wnr can also be factored in this way, reducing conlputations still further. The greatest reduction in conlputations is realized when N = 25 for some s > 0. In order t o determine the cornplexity of the F F T , define m ( N ) t o be the number of multiplications required to compute an N-point DFT using the F F T algorithm. Theri since

corrlplitirlg 2 requires rrb(N/2) ~rlultiplicatioilsto cornpute each of 2 and b arid N/2 r~~~~ltiplications to compute W$ b(r1). Thus, A

Thcrcforc, we lii-lve the following theorem.

Theorem 4.45.

If N = 2" f o r sorne s E N , then

Proof: 'l'l1e proof' is by induction. Sirice n1,(4) = 4 the result holds for N = 4. Assurrie that it holds for N. Now. by (4.11), rn(2N)

+N

=

2711(N)

=

2 ((N/2) lo&2(N)) N log2(N) N log, (2)

=

= =

+

+

N(log2( N ) log;?(2)) N log2(2N).

4.5 L~ Fourier Series By Definit,iorl 4.1, a signal z(n,)rnl~st,sa,t,isfyC,,Iz(n,)1 < oo.LVc ha,ve seen t>liatall signals are .!%equences: that is, that C , Iz(n)l\ m, but that not all t%sequerlces are signals (see Remark 4.2 and Exercise 4.6). It turns out that there is a very rich arid clegarit rnathenlatical theory related t o the Fourier analysis of t2 sequences which provides an entryway into the very powerful theory of Hilbert spaces. The full devcloprnent of this tl-icory is beyorid the scope of this book, but we will need one of the main results of the theory (The Riesz-Fischer Theorem, Theorem 4.48) in Chapter 9.

110

Chapter 4. Signals and Systems

Definition 4.46.

e2

Given an sequence z = denoted z ( y ) , is the period 1 function given b y

The series (4.12) is referred to as a n ~"ourier

{ I C ( T L ) ) ~ ~ its Z,

Fourier series,

series.

Remark 4.47. (a) Since z ( n )does not rlecessarily satisfy En Iz(n)l < m, the sum in (4.12) does not necessarily converge uniformly on R. In fact, it is not clear that such a series even converges pointwise. For example, consider the series 02 cos(;;nt) 71=1

This is just (4.12) with x(n) = 1/(2)nJ)for n # 0 a.nd z(0) = 0. Clearly, x ( n ) is t % ~ ~ if tt = 0 (or any even multiple of T),the series reduces to the llarmonic series arld does not converge. (b) In light of Plancherel's formula for Fourier series (Theorem 2.57(d)), it seems reasonable t o expect that the series (4.12) will converge in L2 on [O, 1). However, if we carlrlot even guarantee that the infinite sum (4.12) converges at any given point, then the question arises: How are we t o interpret the s u m (4.12) as u function defined o n [0, l ) ?

(c) It turns out that the proper answer to this question requires us t o allow into the picti~refunctions that are not piecewise continuous. This more general notion of function is referred t o as a Lebesyue measurable function,, and to integrate such functiorls requires a more general notion of integral than does the Rierrlann int,egral, na,mely, the Lebesyue integrul. Both of tliese ideas require the notion of the Lebesgue measure of a set that is beyond the scope of this book. (d) The Riesz-Fischer Theorem (Theorem 4.48) gives the final word on the convergence of (4.12) in L" and Carleson's Theorem (Theorem 4.49) asserts that (4.12) converges pointwise except on a set of Lebesgue measure zero. Such sets includc finite sets of points and countable sets of points but are by no means limited t o that.

Theorem 4.48.

e2

(the Riesz-Fischer Theorem) Given an sequence { c ( n ) ,) , E ~ , there exists a Lebesgue measurable function f (x) on [ O , l ) with the property that

where the integral is the Lebesgue integral. In this case,

4.5. L' Fourier Series

111

Theorem 4.49.

(Carleson's Theorem) Given an l2sequence { ~ ( n ) } , , ~the z, symmetric partial sums N

converge at each point of [ O , 1 ) except possibly on a set of Lebesgue measure zero.

Part I1

The Haar System

Chapter 5 The Haar System In this chapter we will preserit ari exarrlple of an orthoi~oril~al systeni on [O, 1) known as the Huur system. The Haar basis is the simplest and historically the first exarriple of an orthonormal wavelet basis. Marly of its properties stand in sharp contrast t o the corresponding properties of t,he trigonometric basis (Definition 2.5). For example, (1) the Haar basis functions are supported on srnall subintervals of [0, l ) ,whereas the Fourier basis fiinctions are nonzero on all of [ O , l ) ; (2) the Haar basis functions are step functions with jlirnp discontinuities, whereas the Fourier basis functions are C" or1 [0, 1); (3) the IIaar basis replaces the rlotion of frcqucncy (represented by the index rL in the Fourier basis) with the dual rlotions of scale and location (separately indexed by j and k ) ; and (4) the Haar basis provides a very efficient represeritatiorl of filrlctions that corlsist of srnoot,h: slowly varying segments punctuated by sharp peaks and discontinuities, whereas the Fourier basis best represents f~lnctionsthat exhibit long terrri oscillatory behavior. More will be said about this contrast in Sectiori 5.4. Our first goal is the construction of the Haar basis on the interval [0, 1). In the colirse of this presentation, we will introduce many of the concepts required for the uriderstanding of rriultiresolution analysis and for the construction of general wavelet bases.

5.1 Dyadic Step Functions 5.11

The Dyadic Intervals

Definition 5.1. For. each pair of integers j , k E Z7 define the interual

I,,k

by

The collection of all such intervals i s called the collection of dyadic subirltervals of R.

The dyadic intervals have the following useful property.

Lemma 5.2. either

G i v e n j o , ko, j ~k.1, E Z, with, either jo # jl or. k o

# kl, then

116

Chapter 5 . The Haar Systerri

In the latter two cases, the smaller interval i s contained in either the right half o r left half of the larger.

•

Proof: Exercise 5.6.

Definition 5.3. G i v e n a dyadic interval at scale j , I,,,, we write I,,, = I , : . ~U I;,,, where I;,, and are dyadic intervals at scale j + 1, to denote the left half and rzght half of the interval I,,k.. I n fact, I:,, = I J f 1 , 2 k and I;,, = (Exercise 5.13.

5.1.2

1,+1,2k+l

T h e Scale j Dpadzc Step Functions

Definition 5.4. A dyadic step function i s a step ,fi~n,ctionf ( r c ) ulith the property that for som,e j E Z , f ( r )i s constar~to n all dyadic inter?~als1 3 , k , k E Z . W e say in this case that f (x) is a scale j dyadic step fiinction. For a n y interval I , a dyadic step filnctiorl on I i s a dyadic step function that is supported o n I . See Fzgure 5.1. Remark 5.5. (a) khr each j E Z, the collectioil of all scale j dyadic step functiorls is a linear space. That is, any linear conibination of scale j clyadic step functions is also a scale j dyadic stcp f~xnctiori.

(h) For each j E Z arid interval I, the ~ollect~ion of all scale j dyadic step filrlctiorls on I is a linear space. That is, any lirlea,r co1nbina.tion of sci~lcj dyadic step funct,ions on I is also a. scalc j dyadic step fullction on I. (c) If f (z) is a scale j dyadic step function on an interval I, then f (:I;.) is also a scade j' dyadic step furiction on I for ariy j' 2 j .

Exercises Exercise 5.6.

Prove Lerrlnla 5.2.

Exercise 5.7.

Prove that for each j,k E Z ;

~ i % ~ and I;\i = 13+1,2k

Ij+1.2k+1.

Exercise 5.8. Prove that any function f (z) of the form

=

5.2. Tlle Haar Systerrl

117

FIGURE 5.1. Scale j dyadic step fur~ctions.Left,: j = 2. Right: ,j = 4.

car1 he written i11 the form

5.2 The Haar System

Definition 5.9.

Lct p(.r') = X p . l ) (n.), an,$ for.

,TLC/L

j. k E Z, dc,firj,e

pl,k,(.r)= 2 1 ' 2 p ( 2 J . r- X.) = D 2 , 7 ; ~ ) ( . ~ ) .

(5.2)

(For the dej?n,ztion o,f thc d,ll(~,tionoper,ntor D,, c ~ n dthe t ~ a n s l a t i o roperator ~ 7b, see D ~ ~ f i n i t i o3.39.) r~ The collection { p , l , ~ ( ~ ) }i s, r.ef'erre(t l . ~ ~ ~ t o as t h e .system of Haar scaling hliirtions. For ea,ch ,j E Z . th,c rsollection { p , . k ( . r ) } ~ i s ~r.efr.r-r.rjd ~ t o a s the s!lsfrrn of' scale j Haar scaling fil~lctioris.See figure 5.4

Remark 5.10.

(a) For eadl j. k E Z. p,.k(.r.) = 2,1/' t I , , , ( r ) so . that, supported or1 the interval IJ,kant1 tlocs rlot, v a l ~ i s l on i that ilit,ei.vi~l. Therefore, u7e refer to Dhe scaling fullction o , , , k . ( . I : ) as being ussociatcd with,

p , ] . k ( z ) is

the inte,rvc~,lIj,h..

(b) For each j , k E Z,

and

Definition 5.11. Let h , ( z )

=

X 1 0 , 1 / 2 j ( x )- X [ 1 / 2 , 1 ) ( ~ )and , for each j , k E Z,

d efiiz e

h j , k ( z ) = 2J'2 h(2"

-

k) = D2,,Tkh(z).

(5.3)

Th,e collection, { h , j , k ( x ) } , , k c ~ i s referred t o as the Haar system on R. For each j E Z, the collection { h l , k ( x ) } k E is ~r e f e ~ e dt o as the s y s t e m of scale j Haar

functions.

Remark 5.12. (a) For each j , k E Z, hj,/c(x)= 2"'

( X p. I . i( )

-

I

.I

,

(

)

' 2

2'

XI,,,,.^ (x)

-

Y I ~ + ,,,,+,(r)),

so that h,,k(x) is supported on the interval I j , k and does not vanish on that interval. Therefore, wc refer t o tllc Hnur furlctiorl hj,Jlc;.) as being ussociated with the interval I J , k .

(Is) For each j , k E Z, F L , ~ , ~ ( x )is a scale j (c) For each j , k E Z,

+ 1 dyadic step function.

and

5.2.2 Orthogonality of the Haar System Theorem 5.13.

T h e Haar s y s t e m o n R is a n orthonorrnal systern o n R.

Proof: First, we show orthonormality within a given scale. Let j E Z be fixed, and suppose that k , kt E Z are given. By Lemma 5.2,

5.2. The Haar System

119

If k # k', then the product hj,k(x) hj,,/(x) = 0 for all x since the functions are supported on disjoint intervals. Hence, if k # k',

If k = k', then

Nexl, we show orthoriormality between scales. Suppose that j, j' E Z with j # j', say j > j', and let k , k' E Z. Then by Lelrinia 5.2, there are three possibilities.

(1)Ij/,k/ f' Ij,k= 8. In this case, h,,k(x)hj/,k/(x) = 0 for all z and

I;, ,,.

(x)is identically 1 on I:,,,,.Since Ij.*C In this case, I:,,,, it is also identically 1 on I j , , Sirrce hi,,( s )is supported on I,>r,

(2)

C

(3) I j , k c I;, Thus,

k,.

111this case, hj,,kt(x) is identically -1 on Ij;,,, arid on

IJ,k.

Theorem 5.14.

Given any j E Z , the collection of scale j Haar scaling functions is a n orthonormal system o n R.

Proof: Exercise 5.19. Although it is by no means true that the collectiori of all Haar scaling functions is an orthonormal system on R, the following theorem holds.

Theorem 5.15.

Given J E Z , the following hold.

( a ) T h e collection {p.~,k(x), h,,,k(x):j > .I, k E Z ) is a n orthonormal system o n R. (h) T h e collection {P.I,k(x),h.~,k(x): k E Z) is a n orthonormal system o n R.

Proof: Exercise 5.20.

120

5.2.3

Chapter 5. The Haar System

The Splitting Lemma

Lemma 5.16. (The Splitting Lemma) Let j E Z , and let g,(x) be a scale j dyadic step function. T h e n g, (x) can be written as g, (x)= r,-I (x) g,-I ( x ) ,

+

where r, - 1 (x) has the form

for some coeficients { a , -1 ( k ) ) k E z ,and g, -1 (x) i s a scale j - 1 dyadic step function.

Proof: Since gj(x) is a scale j dyadic step function, it is constant on the intervals I j , k . Assume that g j (x) has the value cj (k) on the i~lt~erval Ij,k. For each interval I j W l , k , define the scale j - 1 step function gj-l(z) on I j - l , k by

In other words, on I j - l , k , g3- 1 (x) takes the average of the values of y j (z) on the left and right halves of Ij-l,k(see Figure 5.2(a)). Let r,-l (z)= gj (z) - gjPl(x). By Remark 5.5(a), gj-l (x) is a scale j dyadic step function, and by Remark 5.5(c), so is T ~ - ~ ( xFixing ). a dyadic interval Ij-l,l., recall that lIj-l,k= 2-(i-'). hen'

1 2-(j-l) (cj(2k)

2

=

+ cj (2k + I ) )

0.

Therefore, on Ij-l,k, r j p l ( x ) must be a rnultiple of the Haar function hj-l,k(x) and must have the form (5.4) (see Figure 5.2(c)).

Exercises

5.2. The Haar System

121

FIGURE 5.2. Illustration of one step in the Splitting Lemma. Top left: Solid: Scale 4 dyadic step function y4(z). Dashed: The scale 3 dyadic step function g3(x) constructed as in the Lemma. Top right: Graph of g3(x). Bottom: Graph of the residual rs(x).

Exercise 5.17.

Prove the statements rrlade in Remarks 5.10(a) and 5.12(a)

Exercise 5.18. Prove that po.o(x) = 2-112 P,.,(x)

+ 2-'12

ho,o(") = 2-'12 p 1 , 0 ( x )

-

p1)L,1(2)

and

Exercise 5.19. Prove Theorem 5.14. Exercise 5.20.

Prove Theorem 5.15.

2-112

p1.1 (x).

122

Chapter 5. The Haar System

5.3 Haar Bases on [ O , 1 ] Definition 5.21. For any integer J 2 0, the scale J Haar system on [ O , 1 ] is the collection

W h e n J = 0, this collection will be referred to simply as the Haar system on [0, 11. See Figure 5.3.

Remark 5.22. (a) The Haar system on [O,1] consists of precisely those Haar functiorrs h j , s ( x )corresponding to dyadic intervals Ij,k that are subsets of [0,I], together with the single scaling function po,0(x).

(b) For J > 0, the scale J Haar systern on [0, I] consists of precisely those Haar functions h j , k ( ~corresponding ) to dyadic intervals Ij.k for which j 2 J and that are subsets of [0, I], together with those scale J Haar scaling functions that are supported in [O,l].

Lemma 5.23. Given f(x) continuous o n

[U, 11, and

t

> 0, there is a J

E Z,

and a scale J dyadic step function y (x) supported in [O,1]such that If (x)-y (x)1 < for all x E [0, I]; that is, 11 f - g J ( , < E .

E,

Proof: Exercise 5.26. See also figure 5.5. Theorem 5.24. For each integer J > 0, the scale J Haar system o n [0,I] is a complete orthon,omn,al system o n [0,11. Proof: That the scale J Haar system on [0, I] is an orthonormal system on [0, I] follows from the fact that it is a subset of the collectio~l { p ~ , ~ ( hj,k(z): x), j J , k E Z), which is an orthonormal system on R by Theorern 5.15(a). For completeness, it is sufficient, by Theorern 2.57(c), to show that if f (x) is Ci' on [ O , l ] , then

>

Let E > 0, and let f (x) be C0 on [0, I]. By Lemma 5.23 there exists j and a scale j dyadic step function on [0, I], gj(x) such that

> 0,

Since any scale j dya,dic st,ep function is also a scale j dyadic step function at all higher scales, we can assume that j 2 J.

5.3. Haar Bases on [O, 11

123

FIGURE 5.3. Some uf the Haar functions h j S k ( 5 on ) [O, 1).

By the Splitting Lenima (Lemma, 5.16), g,i(z) m a y be written g j (x) = rJ- 1(x) gj- 1(x), where rj- (x) 1-ias the form (5.4), and is suppurled in [O, 11 and gj-1 (x) is a scale j-1 dyadic step function. Repeating this process j - J times, we conclude that

+

where each r e ( x )can be written

for some constants a p ( k ) and where g~(x)is a scale .7 dyadic step function (Figure 5.6). But this just means that g J ( z ) is a finite linear combination of the collection {pJ , (~s)}:Lil. Thus gi ( z ) is in the span of the scale J Haar system on [ O , l ] and I f - gj1I2 < t.

0

124

Chapter 5. The Haar System

FIGURE 5.4. Some of the Haar scaling functions p , , k ( x ) on [0,1).

Example 5.25. (a) Let f (.c) = X[0.3/4)( 2 ) . Taking J = 0, we see that = 314, and that ( f ,11,,7.k)= O wheiiever Ij,kC [O: 314) or I j , k 5 (f, [3/4,1). This is true for every j > 2 arld all 0 5 k: 5 2J - 1. Thus, the only ilorizero Haar coefficieiits are (f , ho.") = 114 arid ( f ,h l T l )= 2 - v 2 . Notice that the P ( ~ , ~ ( Xterrn ) is simply the average value of the function on 10, I ) , and that the only nonzero Haar coefficients correspond to the Haar functions that "strad(lleVthe discontinuity of f (z). (lr). Again, assuine that J = 0. Then (.f, po,o) = (b) Let f ( z ) = X p l 11/16, which is the average value of f (x) on [0, 11, and ( f ,hj,k) = 0 whenever IjSk C: