Fourier and Wavelet Analysis - PDF Free Download

George Bachman Lawrence Narici Edward Beckenstein FOURIER AND WAVELET ANALYSIS Springer George Bachman Lawrence Nari...

Author: George Bachmann | Lawrence Narici | Edward Beckenstein

439 downloads 3081 Views 10MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

George Bachman Lawrence Narici Edward Beckenstein

FOURIER AND WAVELET ANALYSIS

Springer

George Bachman Lawrence Narici Edward Beckenstein

Fourier and Wavelet Analysis

,

Springer

George Bachman Professor Emeritus

Lawrence Narici

of Mathematics Polytechnic University

5 Metrotech Center Brooklyn, NY 11201 USA

Department of Mathematics and Computer Science St. John's University Jamaica, NY 11439 USA

Edward Beckenstein Science Division St. John's University Staten Island, NY 10301 USA

Editorial Board (North America): S. Axler

F.W. Gehring

Mathematics Department

Mathematics Department

San Francisco State UniversIty

East Hall

San Francisco, CA 94132

University of Michigan

Ann Arbor, MI 48109-1109

USA

USA

K.A. Ribet Department of Mathematics University of California at Berkeley

Berkeley, CA 94720-3840 USA

Mathematics Subject Classification (1991): 42Axx, 42Cxx, 41-xx

Library of Congress Cataloging-in-Publication Data Bachman, George, 1929Fourier and wavelet analysis Edward Beckenstein. p.

cm. - (Universitext)

/ George Bachman, Lawrence Narici,

Includes bibliographical references and index. ISBN 0-387-98899-8 (alk. paper) 1. Fourier, analysis. Edward, 1940QA403.5.B28

2. Wavelets (Mathematics) II. Narici, Lawrence.

I. Beckenstein,

III. Title.

IV. Series.

2000

515'.2433--dc21

99-36217

Printed on acid-free paper.

© 2000 Springer-Verlag New York, Inc.

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts jn connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former arc not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone. Production managed by Michael Koy; manufacturing supervised by Jeffrey Taub. Camera-ready copy prepared by the author. Printed and bound by

R.R. Donnelley and Sons, Harrisonburg, VA.

Printed in the United States of America. 9 8 7 6 5 4 3 2 1

ISBN 0-387-98899-8 Springer-Verlag New York Berlin Heidelberg

SPIN 10491067

Not long ago many thought that the mathematical world was created out of analytic functions. It was the Fourier series which disclosed a terra incognita in a second hemisphere. -E. B. van Vleck, 1 9 14 The Fast Fourier transform-the most valuable numerical algo rithm of our lifetime. -G. Strang, 1993 . . . wavelets are without any doubt an exciting and intuitive concept . This con cept brings with it a new way of thinking . . . . -)T. �eyer, 1993 Foreword

Fourier, the nineteenth (and not the last!) child in his family, wanted to join an artillery regiment . His commoner status prevented it and he went on to other things. Goethe's dictum that boldness has a magic all its own found life in Fourier. He was so rash at times that his work was rejected by his peers (see the introduction to Chapter 4) . He never worried about convergence and said that any periodic function could be expressed in a Fourier series. Nevertheless he was so original that others-Cauchy and Lagrange, among them-were inspired to attempt to place his creations on a firm foundation. They both failed. The first proof that Fourier series converged pointwise was Dirichlet 's in 1829 for piecewise smooth functions (Sec. 4.6) . As a result of Dirichlet's work, the idea of function was trans mogrified. No more would it apply only to the aristocratic society of poly nomials, exponentials and sines and cosines; disorderly conduct now had to be tolerated. By the mid-nineteenth century, it inspired (as a trigonomet ric series) Weierstrass's continuous-but-not-differentiable map (Sec. 4.3). It was such a shock at the time that Weierstrass was apparently in no hurry to disseminate it widely. In order to generalize Dirichlet's pointwise convergence theorem for piece wise smooth functions to a wilder sort , Jordan invented the concept of 'function of bounded variation ; ' he proved his pointwise convergence the orem of Fourier series for such functions in 1881 (Sec. 4.6) . As it became necessary to deal with this wider class of functions, the conception of in tegral was also transmuted. At Dirichlet's urging, it went from integral-as antiderivative to being defined as area under a curve. Cauchy developed the integral from this perspective for continuous functions. Riemann extended it to discontinuous functions, although not too discontinuous. Fejer ( 1 904) went beyond functions of bounded variation. He discovered that for many functions I, f can be recovered by summing the arithmetic means of its Fourier series; even if the Fourier series diverges at a poin t , the series o f arithmetic means converges t o (f (t- ) + I (t+)) / 2 (Sec. 4 . 1 5 ) . What happens a t t's where the one-sided limits d o not exist? By removing the requirement concerning the existence of I ( r ) and I (t+) , Lebesgue

vi

Foreword

globalized Fejer's theorem; he showed that the Fourier series for any f E Ll [ -11", 11"] converges (C, 1) to f (t) a.e. The desire to do this was part of the reason that Lebesgue invented his integral; the theorem mentioned above was one of the first uses he made of it (Sec. 4.18). Denjoy, with the same motivation, extended the integral even further. Concurrently, the emerging point of view that things could be decom p osed into waves and then reconstituted infused not j ust mathematics but all of science. It is imp ossible to quantify the role that this perspective played in the development of the physics of the nineteenth and twentieth centuries, but it was certainly great . Imagine physics without it. We develop the standard features of Fourier analysis-Fourier series, Fourier transform, Fourier sine and cosine transforms. We do NOT do it in the most elegant way. Instead, we develop it for the reader who has never seen them before. We cover more recent developments such as the discrete and fast Fourier transforms and wavelets in Chapters 6 and 7 . Our treatment of these topics is strictly introductory, for the novice . ( Wavelets for idiots?) To do them properly, especially the applications, would take at least a whole book. What do you need to read the book? Not a lot of facts per se, but a little sophistication. We have helped ourselves to what we needed about the Lebesgue integral and given references. It's not much and if you don't know them exactly-if you know the analogous results (when they exist) for the Riemann integral-you can still read the book . We use some things about Hilb ert space, too, and we have included a short development of what is needed in the first three short chapters. You can use them as a short course in functional analysis or start in Chapter 4 on Fourier series, referring to them on an as-needed basis. The chapters are sufficiently independent so that you could start in Chapter 5 (Fourier transform) or 6 (discrete, fast) or 7 (wavelets) and refer back as needed. One caveat : To appreciate Chapter 7, you should read the theory of the L2 Fourier transform in Chapter 5. The L2 theory is really quite pretty, anyway. Notation: Our notation is all standard. On some rare occasions we use C for complement . If we say "by Exercise 3," we mean Exercise 3 at the end of the current section; otherwise we say "Exercise 2.4-3," meaning Exercise 3 at the end of Sec. 2.4. Hints are provided for lots (not all) of the exercises. Rather than a separate index of symbols, the symbols are blended into the index. We prepared the book using Scientific Word and Scientific Workplace. The experience has been. . .interesting. We hope that the result is fun.

Contents Foreword 1

2

3

Metric and

1.1 1.2 1.3 1.4 1.5 1.6

v

Normed Spaces

Metric Spaces. Normed Spaces

1

. . . . . .

Inner Product Spaces Orthogonality.

. . .

Linear Isometry..

.

Holder and Minkowski Inequalities;

Lp

and

Qp

Spaces.

35

Analysis

2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10

Balls Convergence and Continuity. Bounded Sets .

.

.

.

.

.

Closure and Closed Sets. Open Sets...

.

..

Completeness .... Uniform Continuity. Compactness

.

. .

Equivalent Norms. Direct Sums.

Bases

3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10

1 9 12 18 24 28

Best Approximation. .

. . . . . . . . . . . . . . . .

Orthogonal Complements and the Projection Theorem. Orthonormal Sequences . Orthonormal Bases .

. .

The Haar Basis .....

.

.

Unconditional Convergence Orthogonal Direct Sums. Continuous Linear Maps Dual Spaces. Adjoints........

.

35 38 49 52 58 60 66 69 75 83 89

90 94 103 107 114 119 123 126 131 135

viii 4

5

Contents

139

Fourier Series

4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12 4.13 4.14 4.15 4.16 4.17 4.18 4.19 4.20

Warmup Fourier Sine Series and Cosine Series Smoothness . The Riemann-Lebesgue Lemma . The Dirichlet and Fourier Kernels. Pointwise Convergence of Fourier Series. Uniform Convergence . The Gibbs Phenomenon. A Divergent Fourier Series Termwise Integration . Trigonometric vs. Fourier Series. Termwise Differentiation Dido's Dilemma . Other Kinds of Summability. Fejer Theory The Smoothing Effect of

(C, 1 )

Summation

Weierstrass's Approximation Theorem. Lebesgue's Pointwise Convergence Theorem . Higher Dimensions Convergence of Multiple Series

The Fourier Transform

5.1 The Finite Fourier Transform . 5.2 Convolution on T . 5.3 The Exponential Form of Lebesgue's Theorem. 5.4 Motivation and Definition. 5.5 Basics/ Examples 5.6 The Fourier Transform and Residues 5.7 The Fourier Map 5.8 Convolution on R . 5.9 Inversion, Exponential Form 5.10 Inversion, Trigonometric Form 5.11 (C, 1 ) Summability for Integrals. 5.12 The Fejer-Lebesgue Inversion Theorem 5.13 Convergence Assistance . 5.14 Approximate Identity . 5.15 Transforms of Derivatives and Integrals . 5.16 Fourier Sine and Cosine Transforms. 5.17 Parseval's Identities.

143 154 159 169 174 188 202 207 210 213 218 221 224 226 235 242 244 245 249 257 263

264 267 273 275 278 284 289 291 294 295 303 306 317 330 334 340 351

5.18 5.19 5.20 5.21 5.22 5.23 6

7

Contents

The L2 Theory . . . . . . . . . . .

356 361 366 372 375 378

The Plancherel Theorem . . . . . .

Pointwise Inversion and Summability A Sampling Theorem . The Mellin Transform . Variations. . .

. .

.

.

383

The Discrete and Fast Fourier Transforms

6.1 6.2 6.3 6.4 6.5

The Discrete Fourier Transform . .

.

The Inversion Theorem for the DFT.

Fast Fourier Transform for N Cyclic Convolution .

. . . . . . . . 2k. .

The Fast Fourier Transform for N =

RC.

=

Wavelets

7.1 7.2 7.3 7.4 7.5

Orthonormal Basis from One Function Multiresolution Analysis. . . . . . .

Construction of

a

.

.

.

Shannon Wavelets.

. .

. .

Riesz Bases and MRAs Franklin Wavelets. Frames . . . . . . . Splines . . . . . . .

. .

The Continuous Wavelet Transform.

Index

.

383 390 396 399 406 411

413 414 419 422

Scaling Function with

Compact Support .

7.6 7.7 7.8 7.9 7.10 7.11

.

Mother Wavelets Yield Wavelet Bases. From MRA to Mother Wavelet

ix

435 448 449 459 464 476 480 497

1

Met ric and N ormed Spaces I t is natural t o think about distance between physical objects-people, say, or buildings or stars. In what follows, we explore a notion of "closeness" for such things as functions and sequences. (How far is f (x) = x 3 from g (x} = sin x? How far is the sequence ( lin) from (2/n 2)?) The way we answer such a question is. through the idea of a metric space. In principle , it enables us to talk about the distance between colors or ideas or songs. When we can measure "distance," we can take limits or "perform analysis." Special distance-measuring devices called norms are introduced for vector spaces. The analysis we care most about in this book involves norms . This type of analysis is known as functional analysis because the vector spaces of greatest interest are spaces of functions.

1.1

Metric Spaces

We define a metric space here, then give some examples. The idea is to abstract the properties of the usual notion of distance.

Definition 1.1.1 METRIC SPACE

Two things are required to define a M ETRIC SPACE: a set X of elements called P O INTS or ELEMENTS and a M ETRIC (or DISTANCE FUNCTION ) d, defined on pairs (x, y) of points that satisfy the following conditions . For all x and y in X , •

• •

(Positive) d( x , y) � 0, and d(x , y ) = 0 if and only if x = y. Thus, not only is the distance from any point x to itself 0, but x is the only point 0 units. away from x. (Symmetric) d(x , y) = d(y, x).

(Triangle Inequality) For any z E X, d(x , y) :s: d(x , z) + d(z, y) . 0

As a rule, the only property that is difficult to verify about a proposed metric is the triangle inequality. We use the following notation throughout the book . • •

R C

denotes the real numbers. denotes the complex numbers.

2

1 . Metric and Normed Spaces •

K=

R or e without specifying which.

Example 1.1.2 THE EUCLIDEAN SPACES (K n ,

d2 ) = £2 (n ) , n E N With distance defined by dl (x, y) = I x - YI , (K, dJ ) is a metric space. Let K 2 denote the Cartesian product K K = {(x, y) x, Y E K } . A 2 :

x

metric d 2 is defined on K by

x (X I , X 2 ) Y ( Y2 , Y2 ). (Xl , X 2 , X 3 ) Y ( YI , Y2 , Y3 ) d2 (x, y) = [ lYI - xl 1 2 + I Y2 - x 2 1 2 + IY3 - x 3 1 2P / 2 .

and = where = Let K 3 = K x K x K. A distance i s defined between the points of K 3 by and =

x

The triangle inequality is verified by using the Minkowski inequality 1 .6 . 3 ( a) with p = 2. In either case d2 is called the EUCLIDEAN or USUAL metric on K 2 and K 3 . The spaces ( K 2 , d2 ) and are called EU CLIDEAN 2-SPACE ( the EUCLIDEAN PLANE if K = and EUCLIDEAN 3-SPACE, re spectively. If we say only or we mean respec and tively. We extend this idea of distance to n-dimensional space K n by taking, for EK n ,

R2 R3 ,

(K3 , d ) R) 2 (R2 , d2 )

(R3 , d2 ),

(x; ), ( Yi )

The triangle inequality follows from the Minkowski inequality with p = 2. The metric d2 is also called the EUCLIDEAN METRIC , the pair d2) EUCLIDEAN n-SPACE. Many authors reserve Eu clidean for d2) and refer to as UNITARY n-SPACE . We say real or complex Euclidean n-space, respectively; we also denote ( K n , by ( n) . 0

(Kn , n (R ,

(e n , d2 )

d2 ) £ 2

In many imp ortant respects , the spaces we care most about in this book can be thought of as though they were Euclidean 2- or 3-space , a point that will become clearer as our story unfolds. For now , you might glance at Exercise 1 .3-8 . I t is common t o view as a "subset" of the plane even though i t is actually x {O} , not that is the subset of R2 . As metric spaces, there is no difference between and R x {O} C (2): For two real numbers a and b compare the distance (a, b) = l a between them with the d 2 -distance between (a, 0) and (b, O) :

R

R R, (R,dl )

R2 ,

dl

£2 - bl

d2 [(a, 0) , (b, 0)] = j(a - b) 2 + (0 - 0) 2 = l a - b l .

1.

Metric and Normed Spaces

3

Likewise , as metric spaces, there is no difference between ( R 2 , d 2 ) and the complex numbers C-distance is measured in exactly the same way in each case. The point is that there can be concretely different things that are the same as metric spaces. Definition 1.1.3 ISOMETRY

Let (X, d) and (Y, d' ) be metric spaces. A map f : (X, d) ---+ (Y, d') is an ISOMETRY if for all x, y E X,

d' (f(x) , f(y)) = d(x , y) . If the isometry f i s onto (surjective) i n the sense that for all y i n Y there is some x in X such that f(x) = y, then the spaces (X, d) and (Y, d' ) are called ISOMETRIC . 0 Notice that isometries f must be 1- 1 (injective) , since

x =F y ¢::::? d (x , y) =F 0 ¢::::? d' (f(x) , f(y)) =F 0 ¢::::? f(x) =F f (y) · Some obvious isometries of R2 onto R2 are translation, rotation through

a fixed angle , and reflection about a line . Indeed , these are the only ones , something we say a little more about in Exercise 1 .5-2 . We commonly identify-i.e ., treat as equal-isometric spaces, as in "R is a subspace of R2 " rather than "R is isometric to a subspace of R2 . " Let Pn denote the set of polynomials p(t) of degree less than or equal to n with real coefficients and distance defined by

n

I)ai - b; ) 2 . ;=0

The space Pn is isometric to Rn + l = £2 ( n + 1) under the mapping

f : Rn + l

(ao,al , . . . , an )

--+

1----+

Pn L�o a; t i '

At times it is preferable to view Pn as a set of polynomials, at other times as a set of ( n + 1 )-tuples. As metric spaces, they are the same. Example 1.1.4 PRODUCT METRICS

The square-root-of-the-sum-of-the-squares method used to create a met ric for the Cartesian products R x R and R x R x R in Example 1 . 1 .2 can be generalized to Cartesian products of arbitrary metric spaces as follows. Suppose that (X, d) and (Y, d' ) are metric spaces. A metric d2 on X x Y = {(x, y) : x E X, Y E Y} is defined by ( 1 . 1)

4

1 . Metric and Normed Spaces

The triangle inequality follows from the Minkowski inequality 1 .6 .3( a) with p = 2. More generally, the Minkowski inequality implies that for 1 � p < 00 , defines a metric o n X x Y. Aside from p = 2 , another imp ortant special case is p = 1: a special case of which is discussed below in Example 1 . 1 .5. For p = 00, The triangle inequality follows from the Minkowski inequality of Exercise 1 .6-2. X Xn Each of the dp can be extended to any finite product Xl x X2 X of metric spaces (Xi , di) as the pth root of the sum of pth powers •

•

.

Are any of the dp to be preferred? In a sense to be made clear in Section 2 . 9 , no, because the convergent sequences are the same with respect to any of them. Suppose that (X, d) and (Y, d') are metric spaces. An imp ortant property that each of these metrics dp on X x Y possesses is that they "extend" the original metrics-for any fixed y in Y, the space X x { y} is metrically a copy of X. Since d' (y, y) = it follows that for any Xl , X 2 E X , by equation ( 1 .2) ,

0,

p 1 dp «X 1 , y) , (X 2 , Y)) = (d(X 1 , x 2t + d, (y, y) ) /P = d (X 1 , X 2 ) , 1 � p �oo. In other words, X {y} is isometric to X under the "projection" map (x , y)�x . 0 In the next example we emphasize the point that it is the pair (X, d) and not just the set X that defines a metric space. x

Example 1.1.5 TAXICAB METRIC

As follows from Example 1 . 1 .4, for X = (Xl , yd and y = (X 2 , Y2 ) in the set R2 ,

d 1 (x, y) = I X 2 - x d + IY2 - yd defines a metric on R 2 . It is called the TAXICAB metric because distance

is measured by adding horizontal and vertical distances, as it would be computed by a taxi ' s odometer on a grid of city streets. It is much easier

1 . Metric and Narmed Spaces

5

to work with than the Euclidean metric, withthe itsorigin cumbersome square(1, roots d1 -didstance from to the point l) in ofthissums of squares. The metric is 2, not V2, the 2 -dis tance; thus, even though the points are the same (namely the points of R2 ), the distances have changed. 0 Example 1.1.6 THE TRIVIAL METRIC

The trivial metric is great for counterexamples (and not much else). On d any points(or yDISCRETE E X , take (x, y) = 1 if =P y and 0 otherwis e; d is setcalledX, fortheanyTRIVIAL . The space (X, d) is called 0 a (not "the") TRIVIAL METRIC SPACE.) METRIC If inn-tuples the spaces '-2 ( ) of Example 1 . 1 .2 we let 00,usually i.e., wecalledgo from to square-summable sequences, we get what is HILBERT SPACE £2 , known also as '-2 (N) or '-2 (00). Be careful about the usage Hilbert "Hilbertspace space,(Definition " because 2.there.1).is a more general type of space also called 6 SQUARE-SUMMABLE SEQUENCES '-2 (N) = £2 (00) = '-2 = £Example 2 (Z) The square-root-of-sums-of-squares of measuring distance is taken about far it(00)can ofgoallin SQUARE-SUMMABLE the space way ( the Hilb ert space, "little ell-two") = £2 £complex 2 = £2 (N)numbers, sequences (an ) such that sequences (an) of real or LneN 1 an 1 2< 00. toWebedefine the distance between two such sequences1 / 2 = (an) and y = ( bn) d2 ( x, y) = ( L 1 bn an 1 2) n eN The triangle inequality follows from the Minkowski inequality 1.6 . 3 (b) for infinite Let Zseries. denote the setto beof integers. ofC a, two-sided series (biseries) m,n_ooTheL�sum lim =E N; for convergent m m k Lbiseries n eZ CnLnis defined terms, limn_oo L�(an) Cn of nonnegative =- n Ck · LneZ Cn =bisequences eZcollection Consider the '(Z) of square-summable neZ 2 such that x,

x

n

n -+

1.1.7

as

as

x

_

n,

and take

6

1. Metric and Normed Spaces

asisa any metric on.e2 (Z). It is easy to show that .e2 (N) is isometric to.e2 (Z); if ( i. e ., 1- 1 and onto map ) ofN onto Z, then (xn ) (xf( n ») fmaps .e bijection Exercise2 (9N. ) 0isometrically onto .e2 (Z). We ask the reader to verify this in By adistance SUBSPACE of a metric space (X, d) we mean a subset S of X in which is still measured using d; we denote such a subspace by s) (S, d i . forof.e any(N ) . > 2; the spaces Rn =.e2 ( ) E N, are (isometric to) subspaces 2 took the "square root of sums of squares" idea to a limit in Example We 1 . 1 .7. In Example 1 . 1 .8 we go further; we go from sums to integrals. Example SQUARE-SUMMABLE FUN CTIO NS L 2 [ a, b) We say thaton two complex-valued functions f and are "equal almost everywhere" the closed interval [a, b) if f (t) = 9 (t) at each t E [a, b) but for ina setSection of Lebesgue measure 0 ( cf. Exercise 15 and the brief dis cussion 1 . 6 ) . We abbreviate "almost everywhere" to "a. e." We treat functions that are equal a.e. as the same ( we "identify" functions that are equal a. e . ) . This means, in particular, that we treat as equal the CHARACTERISTI C (or INDICATOR ) function lQ of the rational numbers Q (lQ = 1 on the rationals and 0 everywhere else ) and the function that is identically O. The convention of identifying almost-everywhere-equal tions can therefore make for some drastic differences and simplify thingsfuncin certain cases. Withspace, a.e . equal treated(foras thethe Lebesgue same function, collection (vector actuallfunctions y) of functions integralthe) L 2 [a, b) = { x [a, b]-+ K lb I x(t) l 2 dt < 00 } istionsknown space of S QUARE-SUMMABLE or SQUARE-INTEGRABLE func on [a,as the b] . The distance between x , y E L 2 [a, b] is defined to be ( ) 1/2 b 2 d(x , y) = l l x(t) - y(t) 1 dt IfL the( R)functions x are square-integrable on R, J�oo I x(t) 1 2 dt < 00 , we write 2 . 1The6 .3 (triangle namely c) . 0 inequality follows from another Minkowski inequality, �

n , n

n

1.1.8

9

:

:

.

Exercises 1 . 1

(X, d)

is a metric space in the problems below.

1.

Metric and Normed Spaces

7

themust only thepossible routes(usualbetween "cities" arethem specified by aalong road map, distances notion) between measured those routes satisfy the triangle inequality? 2. Show that a metric space is still obtain ed if the real numbers are replaced by the complex numbers C in any of the parts of Example 1 . 1 .2. 3 . PSEUDOMETRICS If a function on X fails to be a metric only in that p(x , y) = for certain x # y in X, then is called a PSEUDO METRIC . Let X be any collection of functions on a set T and let t E T. Show that p(x , y) = Ix(t) y(t) l , X , y E X, defines a pseudometric on X . 4. PRO DUCT METRICS NOT ISOMETRIC Show that for p # q , the spaces (X x Y, dp ) and (X Y, dq) of Example 1 .1 .4 are not isometric spaces under the identity map x ..-. x . 5 . POSITIVE MULTIPLES O F METRICS If a > 0, show that da , defined by da (x, y) = ad(x, y) for all x, y E X , is metric. This shows that it is possible to define infinitely many metrics on any metric space. 6. NEW METRICS FRO M OLD Let 1 : [0,00) -+ R be increasing with I(t) > for t > 0, and I( + t) I(s) + I(t) . For any 1metric (0) =d0,show that d' (x, y)r = J[d (x , y)] also defines a metric. Some suitable s are f' inf ( 1 , t) . I(t) = t (0::; 1 ) , In( l + t; ) , tl (l + t) , and 7. EXTENDED REALS Use the result of the preceding exercise to metrize the EX TENDED REAL N UMBERS = Ru {±oo} by means of the map 1 [-1 , 1] , x l.Hxl for x E R, 1 ( -00) = - 1 , I ( 00) = 1 . (For x E ( - 1 , 1 ) , 1 - 1 (x) = xl (1 - I x!) .) For x > 0, what is d' (x, oo)? d' (1, oo)? 8. BIJECTIONS DEFINE METRICS If I X Y is bij ective, then show that 1 . If

p

°

p

-

x

a

°

:

R

r::;

::;

s

R

-+

..-.

:

-+

defines a metric on Y .

9. SOME ISOMETRIES

(a) Show that £2 (N) is isometric to £2 (Z) . (b) isometric For any twoto Lclosed intervals [a , bj and [e, d) show that L2 [a , bj is 2 [e, d) . 10. Identi fy themetric set of space; all pointsthe that t away from a point x in a trivial set ofarealldistance points distance 2 away?

8

1.

Metric and Normed Spaces

MAX AND SUP METRICS ( cf. Example 2.2.8) We explore the metric doo of Example 1.1.4 a little further here. ( a ) Let E N. In real £00 ( ) = (R n , doo), the distance between any two points x = (ai ) and y = (bi ) in Rn is doo(x, y) = m!lX I bi - ai l· VerifyOnthat 1dooforisXa metric. Identi f£y the(3)"unit ball" U = {x E X: £00 d (x, ::; (2) and 00 = ( b ) If x = (a i) and y = (b i ) are points in Roo = I1 i EN R, the space of all sequences ofberealnonumbers, theconsider same idea will not work because there may maximum: x = (0) and y = ( ) If we restrict consideration to the set Roo of all bounded sequences of real or complex numbers and use sup instead of then doo(x, y) = SUPi Ibi - ail defines a metric. 12. THE HAMMING METRIC Let X(n) denote the set of all n-tuples of O's and l 's . For example (without the customary commas and paren theses) , X(3) = {000, 001, 010, 011, 100, 101, 110, I ll } . The HAMMING METRIC d measures distance by similarity: For x, y E X ( n ) , d(x, y) is the number of places where x and y have different entries. For example, d( O O O , 001) = 1 and d(010, 101) = 3. Verify that d is a metric. This metric space has applications in coding and automata theory, combinatorics, and computer science.

11.

n

n

t

.

max,

13.

n .

ULTRAMETRICS

a The metric space indefined here, knowntheory. as the Let BAIRE N ULL SPACE, has applications communication B be the set of all sequences of positive integers. For x = ( a 2 , . .) and y = (b l , b2 , .. .) in B define { 0, ifif iaiis=thebi forfirstallindexi, for which ai =P bi. d(x, y) = As infurther the previous exerci sdifference e, this metricoccurs, is based onclosersimilarity the out the first the x and y are. Veri f y that thi s i s an ULTRAMETRIC in that it satisfies the STRO NG TRIANGLE, or U LTRAMETRIC , INEQUALITY , namely, d(x, y) ::; max {d(x, z), d(z, y)} for all x, y, and

( )

aI,

_zl.'

z.

.

1.

9

Metric and Normed Spaces

b Show that the metric doc of Exercise ll(a) is not an ultrametric. 14. Let N 2 = N N denote the grid of all ordered pairs of positive integers. Does d[(a, b), (c, d)] = I da - b l / ( bd) define a metric? 15. In Example 1 . 1 . 8 we mentioned that we treated functions in L 2 [a, b) that werex -equalifalmost everywhere as the same. Show that the re lation y and only if x = y a. e . is an equivalence relation onclasses. L 2 [a, b]; as such, it decomposes L 2 [a, b] into disjoint equivalence It is these classes that actually constitute the "points" of L ( )

x

c

2 [a, b] .

Hints

and d' ( 1 , 00) = �. 1 3 ( a) . If the distance from x = (aI , a 2 , "') to y = (b l , b 2 , . . . ) in B is Iii, then for j <element i. Now compare x and y to = ( C I' C 2 , ) , whereaj is= abjsuitable of B. 14. What is the distance from ( 1 , 2) to ( 2, 4) ? 7. d' (x, 00 ) = 1/ (1 + x)

z

1.2

z

.

.

•

Normed Spaces

When we say "vector space" we always mean real or complex vector space. we will specify which by saying something like "Let X be a real Sometimes vector space. " If we do not specify, we mean the result to apply to both. The space R or C is denoted generically by K . Real or complex numbers S CALARS . By a SUBSPACE of a vector space we always mean a liare nAnalysis, eacalled rsubspace, a subset thatlimits, is also acannot vectorbespace; we say subse the ability to take doneotherwise in an arbitrary vectort. space.involve In particular, nosums, consideration ofjust infinitesums.seriesIn order is possible because li m i t s of rather than to be able toa they donormanalysis in vector spaces, we introduce a special kind of metric called a vectorintoin theR2 space. or R3 . It generalizes the notion of length, or magnitude, of Definition 1.2.1 N ORM

�

The combination (X, 1 1 · II II : X R isare:called a N O RMED (LINEAR) SPACE . The defining properties of a norm (Positive) I I x i l � 0 for all x E X , and II x i l = 0 if and only if x = O . (Multiplicative) I I axi l = l a l I I x il for all a E K and x E X. •

•

10

1 . Metric and Normed Spaces

+ yl l � I I x l l + I l yl l for all x , y E X. 0 Any normed space is a metric space with d(x , y) = Il x - yl l , since d(x, y) = d(y, x) = I I x - y l l � 0, and II x - y ll = 0 if and only if x = y and (add and subtract y) d(x , z) = I I x - y + y - z l l � I I x - y l l + lI y - z l l = d(x , y) + d(y, ) Whenever we speak of a normed space a metric space, it is always to this norm-in du ced metric that we refer. The prototype normed space is Rn = i2 ( n ) with the usual term-by-term ( also called "pointwise" ) operations and scalar multiplication and the inequality EUCLIDEAN NORM, I I x l b = of addition satisfies the tri a ngle by the Minkowski = (ad l I l l 2 v'L:�=1 ali 11 1 1 2 inequality 1. 6 .3( a ) and induces the nEuclidean metric of Example 1.1. 2 . The a normforonx K= (a( cf.) EExample following 1.1.4) known the n K or sup normalso11 defines namely, , i 11 00 ' (1. 5 ) I Ix l ioo = sup l ai I · i The taxicab norm ( Example 1.1.5) n •

(Triangle Inequality) Il x

z .

as

as

I l x l ll =

� )ail ;=1

max

(1.6)

and the Euclidean norm are each a special kind of p-NORM ( cf. Example ' /' (1.7) II z l l , = (t. l a; I' ) , 1';: p < 00, TheWereason for treating 11·11to00sequences as a p-normandisfunctions, discussed toin the Exercivector se 4.spaces pass from n-tuples i2 (N) and L 2 [a, b] of Examples 1.1.7 and 1.1.8, by taking ( ') 1 2 1/2 ' and I Izl l, = ([ I z(') I d') , I I z l l , = E.. I a n I respectively. These are special cases of the following. Example 1 .2.2 p-NORMS ip ( n ) , ip (N) , and Lp [a, b], 1 � � 00 (a ) ip ( n ) = (K n , 11 ' l Ip ) and ip (N), 1 � p < 00. The space ip ( n ) is the normed linear space Kn of n-tuples of scalars equipped with the p-norm: 1.1.4) ,

/

p

1.

Metric and Normed Spaces p

11

TheNowtriangle followstofrom the Minkowski we passinequality from n-tuples sequences. For 1 ::; inequality let 1.6.3 ( a). < lp (N) = lp = { (an ) E Koo : L l a n l P < oo} , n N the of pTH P O WER SUMMABLE sequences. With pointwise opera tions,collection l is a vector space. We always view l as equip ped with the p-N O RM 00 ,

E

p

p

Toinfinite verifseries. y that 1I · l Ip is a norm, we use the Minkowski inequality 1.6.3(b) for ( b ) Let E be a Lebesgue-measurable set. For ::; 00, the class of K-valued Lebesgue-measurable functions x : E -+1K such< that h l x(tW dt < isE denoted by Lp (E). If E is a finite interval [a, b] we write Lp [a , b] ; if = R, we write Lp (R). The elements of Lp (E) are called pTH POWER SUMMABLE functions on E; L ; (E) and L; (R) denote the subspaces ofreal (E) and L ; (R), respectively. As in Example 1.1.8, valued functions of L;that weformidenti f y functions a vector space, normedarebyequal a.e. As discussed in Section 1.6, they p

00

which is also called the p-N ORM. (c ) The collection loo (N) , or just loo , of all bounded sequences x = (an ) ofnormed elements space with pointwise operations; it is by an E K forms a vector I l x l i oo = sup I an I · n For more details see Section 1.6 and Exercise 1.6-2 ( a) . 0 The following is a very important normed space.

Example 1.2.3 CONTINU OUS FUNCTIO NS C [a, b]

Let Cinterval [a , b] denote the space of continuous K-valued functions on the closed [a , b], ::; a < b::; with sUP or UNIFO RM norm: II x l i oo = sup {I x (t) 1 : t E [0, In . - 00

00 ,

12

1 . Metric and N armed Spaces

Unless we say otherwise, C [a , b] always carries the sup norm. 0 As theI I x l lorigin. = d(x , O), the norm is seen to measure the distance of a vector from vector of norm 1 is said to be a UNIT VECTOR. For any nonzero vectornormsx , xlinduce II x l l is a unit vector. Although metrics, notts every metric defines a(X,norm. For one d) need thing, the underlying set X of poin of the metric space not be a vector space, but even if it is, notice that nonzero vectors x must get bigger2 as they are multiplied by larger scalars: l a l ---+ 00 ==> lI ax l l ---+ 00 . If R carries the trivial metric of 1 . 1 . 6, then d(ax , 0) = 1 for all a =J 0, so metric 2.cannot come from a norm. ofthisExercise 9-7 is have not norm-induced either.Similarly, the bounded metric A

Exercises 1 . 2 q q

q

map is proposed as a norm on a vector space X: 1. The following . O Is a norm? q (x) = 1 for x =J 0, (0) 2. SECOND TRIANGLE INEQU ALITY Show that any norm satisfies the SECOND TRIANGLE INEQUALITY : I l I x l l - I I Y I I I ::; II x - y l l for all vectors x and y. 3 . THE MAX NORM Let {X l , X 2 , , xn } , E N, be a basis for a vector " , a } be scalars. Show that 1 1 2:7:: ai xi l l oo = Let {ala, norm mspacei l aiX.l defines on X; 11 · 11 00 is called the MAX N O RM for X WITH RESPECT TO THE BASIS {Xl , X 2 , . . . , Xn } . denote 4. 1 I ' l Ip on K n , E N. Let 1 I · lI p , 1 p < the p-norm of equation (1. 7 ) on Kn . To see why the notation 11· 11 00 is used to denote the norm (equation ( 1 .5) ) on K n , for any E K n , show that lp I I x l l oo ::; I I x ll p ::; n / I l x l i oo . Deduce from this that I Ixlloo = limp-+ oo I I x lip . .

ax

max

=

n

.

.

•

n

n

1

::;

00 ,

x

Hints 4.

1 .3

Inner Product Spaces

Another way toof generate by means produ ct. It is a generalization the familiarnormsdot isproduct of Rof2 andan inner R3 .

1.

13

Metric and Normed Spaces

Definition 1 . 3 . 1 INNER PRODUCT

vectorKspace K ( = R or C) and an INNER PRO DUCT ( , ) is calledX over an INNER PRODUCT SPACE if the following conditions are met. (Conjugate Symmetry) (x , y) = (y, x) , where the bar denotes com plex conjugation. (Positive Definite) (x , x) 0, and ( x , x) = 0 if and only if x = O. A XxX •

•

•

�

-+

�

(Linear in the First Argument) (ax + by, z ) = o

a (x ,

n

(x , y) =L a i bi

z

) + b (y , ) . z

(1. 8) is =an2inner product; thisin the is therealfamiliar dotconjugate product when K = R and or 3. Note that case, the symmetry reduces tothe(x,mosty) =important (y , x) and that (x , ay) = a (x , y) for any a E R. For many of results aboutspace.innerTheproduct spaces, itofisequation vital that(1.8)we have a complex inner product inner products are essentially the only inner products on Kn (1.5.4). The spaces £2 and spaces with respect toL2 [a, b] of Examples 1.1.7 and 1.1.8 are inner product b (1. 9 ) (x, y) = L a n bn and (x, y) = l x(t)y(t) dt , neN respecti vely.already seen that norms are more special than metrics. Inner We have productsa arenormevenby more induces way ofspecial than norms. Note that an inner product ( 1.10 ) I l x l l =� , weinnerproveproducts after 1.just 3.2. mentioned. The norms Whenever of £2 ( ) £the byin 2 are induced 2 , and11·11 Lsymbol is used the an inner (1.10). productTospace, itthatrefers� to thesatisfies inner theproduct-induced norm weof equation prove triangle inequality, first generalize the following result about the dot product of two vectors x, y in R 2 : I (x , y) 1 = I l I x l l I /y l l cos 01 � I I x l i l l yl i for any i= l

n

a

as

n ,

O.

14

1 . Metric and Normed Spaces

1 . 3 . 2 CAU CHy-S CHWARZ INEQU ALITY For any two vectors x and y of an inner produ ct spa ce l (x , y) 1 � II x l i ll y l i.

X, Proof. If x or y is 0, both sides of the inequali ty are 0, so suppose that neither x nor y is O. For any scalar a , o �

(x - ay, x - ay) .

Fora = (x , y) /(y, y) ,

�� 1 2 0 (1.11) Now weandcanmultiplicative complete theproperties proof thatof inner productsdirectly definefrom norms.the The positive norm follow def inition of inner product: I I x l l = � 0 and I I x l l = 0 if and only if x = 0, and for any scalar a , o � (x - ay, x

_

ay) = II x l 1 2

_

1(

�

l I ax l l = v(ax, ax) = l a l �. �

As for the triangle inequality, since Re (x , y) I (x , y) l , o < I I x + Yl l 2 = (x + y, x + y) I I x l1 2 + 2 Re (x , y) + I I y l 1 2 < I I x l 1 2 + 2 1 (x , y) 1 + l I y l l 2 . By the Cauchy-Schwarz inequality of 1.3.2, it follows that which yields the triangle inequality. Cauchy-Schwarz magni magnitude Inof the dot tude,Whenwhendoesis theequality inner hold productin thea product? In R2 theinequality? of twobetween vectors isthem the same as the product of the lengths if and only thethe angle is 0 or i. e., one vector is a scalar multiple ifofproduct other. As is frequently the case, what happens in R2 can be used to predict what happens generally. In any inner product space, for any scalar a and vector I (x , ax) 1 = l a l ll x ll 2 = II x l i ll ax l l ; thus, equality holds if one vector is a multiple of the other. Conversely, if equality holds for nonzero vectors x and y, then the right side of (1.11) is 0, and x - ay = x - y = 0 , or x = i=.Jll( and only if the vectors are scalar multiples of each other. 0

x,

Y,Y

11",

(X,y)) ( Y,Y

1.

Metric and Normed Spaces

15

Exercises 1 . 3

Inofantheinner productx +space, show that, iftheandlength ofif only + a =sumax isforthesomesumscalar lengths, x y ll = I I ll l I y l l II a � O. Not all norms have this property. Those y that do are called STRICT N ORMS . Show that neither the taxicab norm ( equation (1.6) of Section 1 . 2) nor the max norm ( equation (1.5) of Section 1.2) is a strict norm.

1 . STRICT NORMS

2.

ANG LE

( a)

Define aspace. notionShow of "angl e"your between anyof the twoangle vectors() between in an inner product that notion the vectors x and y in a real inner product space satisfies the law of cosines, i.e ., that

(b) In the real space '-2 (2), show that the �ap A defined by ( � ) (� 0 1 ) ( � ) amounts to rotating a vector counterclock wisesobyin90complex degrees,'- that {Ax , x } = 0 for all x; show that this is not 2 (2). 3. D IFFERENTIABLE FUN CTIO NS The collection C1 [a, b] of continuously differentiabl complex-val uedto functions onaddition the closed intervalmultipli [a, b] is acation. vectorShow spacee that withanrespect pointwise and scalar inner product is defined by (x , y) = lb ( x(t)y(t) + Xl(t)yl (t) ) dt . 4. TRIG O N O METRI C POLYNO MIALS A function x(t) = a + � )a k cos kt + b k sin kt) or L ck e ik t k=l k=-n with complex Tcoefficients is knownpolyasnomials a trigonom etric polyn o mial. The collection of trigonometric obviously tor1, 2,space. Show that ( anticipating Fourier coefficients) forformsanya vec k = 1'lr f�'Ir x (t) dt, a 2 .; D'Ir x(t) cos kt dt , ak bk .; f�'Ir x(t) sin kt dt .

f--+

n

. . . , n,

n

1.

16

5.

Metric and Normed Spaces

Let RH (S l ) denote the space ofrational functions that are holomor phic (analytic) on the circumference S l of the unit circle in C , i. e ., have no pole of magnitude 1. The space RH (31 ) is a vector space with respect to pointwise addition and scalar multiplication. (a) Show that an inner product is defined on RH (Sl ) by 1 1 - dz (f, g ) = 2 . f(z) g (z) -Z . (b) bers Use the ZCauchy integral formula tocircle, show that for complex num strictly inside the unit 2 ( z � Z l ' Z � 2 ) = 1 - �1 2 �z

SI

Zl,

6.

Z

CAUCHy-SCHWARZ INEQU ALITY REVISITED

Z

•

�

(a) Ifo fora, b,allandrealaret , show real numbers with a > 0 and f (t) = at 2 + bt + c 2 that b ::; 4ac. (b) aApply the preceding result to f(A) = I I AX + y l1 2 , where A is complexspace, number and xtheandCauchy-Schwarz y are fixed vectors in an 1.inner product to deduce inequality 3.2. 7 . RIGHT TRIANG LES (cf. Exercise 1.2-2.) Let x, y, and z be three vec tors in an inner product space X. Define "triangle" to mean the triple T (x, y, z) . We say that T is right-angle d at y if (x - y, z - y) = O. (a) Show that T is right-angled at y if and only if I I x - zl l 2 2 2 I I x - y I I + l i z Y1 l . (b) Show that a triangle cannot be right-angled at x and at y . 8. LINES AND PLANES In R3 with its usual inner product, the plane through b, c) normal to the vector (A, B, C) is defined to be thetheset point of all (a,points (x, y, z) such that ( x - a, y - b, z - ) (A, B, C)) = O. Bydefineanalogy withTHROUGH this, in anXoarbitrary real inner product space X we a PLANE NORMAL TO w to be {x E X : (x - Xo , w ) = O}. Similarly, R3 , the line through the point Xo = (a, b , c) parallel to (orthat"in theindirection of") the vector (A, B, C) is the set (x, y, z) such (x, y, z ) t (A, B, C) + (a, b, c) c

=

_

c

=

,

1.

17

Metric and Normed Spaces

the similarity to y = mx + b.) In a general inner forproduct all realspacet . (Note we define the LINE L THROUGH x PARALLEL TO to be the set of all vectors of the form z = ty + x where y # 0 and x and y are fixed vectors and t E These ideas are the basis for the theory of linear programming. In a real inner product space X, (a) Show that if Il x - yll + II y - z ll = II x - z ll , then x, y, and z lie on a line. ( b ) For real k, show that a plane ( x, n) = k divides the space into two parts: (x, ) 2 k and (x, ) < k. ( c ) Show that if a line meets a plane or another line, then the in tersection is a singleton. ( d ) Show that a line z = ty+x meets a SPHERE {v E X: !I v ull = r} oftheradius r > 0 and center u in at most two points. If it meets sphere point w (in which case the line is called a "tangent"in) , only then one ( w - u, y) = O. Draw a picture and interpret this geometrically. A similar result holds for tangent planes to spheres. Compare this to the situation in part (f) . ( e ) The lines z = ty + x and w = tu + are called PARALLEL if y = au for some nonzero scalar a. Show that if the only two possibilities are that that they any meettwoor lines are parallel, dimX = 2. forThelinesproposition meet orthen are parallel is fundamental inbeplane Euclidean geometry. The real ization that there might geometries in which the proposition some isthing false-of there beingin human "non-Euclidean" of a milestone intellectualgeometries-was development. (f) LINES IN NORMED SPACES In any vector space X, define lin e as in the previous exercise. The LINE SEGMENT connecting x and y is [x , y] = {tx + (l - t) y : O :S t :S I } . If X is (apartnormed space, unlike what happens in inner product spaces ( d)) , show that a line can intersect a sphere in infinitely many points. (g) CONVEXITY subset of a vector space is called CON VEX if it contains the ball line insegment two ofIs itsanypoints. Show that the unit a normedthrough space isanyconvex. ball convex? Y

R.

n

n

-

v

A

18

1 . Metric and Normed Spaces

Hints

First, useinequality. the remarkThenaftershow1.3.2thatconcerning equalitynumber in the aCauchy Schwarz for any complex , 11 + a l = 1 + l a l if and only if a � o . 3. Use the fact that if a nonnegative continuous function x(t) is such that J: x(t) dt = 0, then x(t) = of magnitude 1 such that (x , y) 6 (b). Let ,beletat complex be real, andnumber consider A = tao I {x, y) I 8(a) . Use Exercise 1.2-1. 8 (d). This has to do with solutions of quadratic equations. By considering thelinearly linesindependent z = ty and w = tu + show that there 8 (e) . cannot be three vectors {y, u, v} in the space. If dimX = I, then all lines meet. 8(f) . Consider the unit sphere in the real space Roo (2), i. e ., R 2 with norm . 1.

o.

a

ii

v,

max

11· 11 00

1.4

Orthogonality

A wonderful thing about inner product spaces is their "geometry." Among other things, therevectors. is a Pythagorean theorem, a paral lelogram law, and "angles" between We say that two vectors x and y are O RTH O G ONAL if (x , y) = 0; we write x y. Since (ax , y) = a (x , y) for any scalar implies that x is orthogonal to any scalar multiple ay of y, a, x i.e., is yorthogonal to the line [y] = {ay: a EO,K}x) =determined by As 0, so the 0 vector (O, x) = (o + 0, x) = 2 (O, x) , it follows that ( is orthogonal to every vector. Conversely, if x is orthogonal to every vec tor, then (forx , x)showing = 0; since ( , ) is positive definite, x = O. This provides a technique equalityIn Rof2 two vectors x and Show that x - y is orthogonal to every vector. the projection of x asontoa Fourier a unit vector is (x , y) y. As will be seen later, expressing a function series isy very much like writing it a sum of projections of this type. Definition 1 .4 . 1 ORTHOGONAL SETS A subset S of an inner product space X is called an O RTH O G O N A L SET iorthogonal f y for all distinct x, y E S. If is denumerable, we say that S is an SEQUENCE. If S is an orthogonal set of unit vectors, it is called anandORTHONORMAL SET . We say x S if x y for every y E S. For sets S T we say S 1. T if x for each x E S and E T. 0 1.

1.

x

y.

y:

as

x

S

1.

1.

y

1.

1.

y

1 . Metric and Normed Spaces •

•

•

•

•

R2 ,

19

R,"

In (0, 1 ) i s orthogonal t o any vector "in i.e . , any vector ( a, 0) in x {OJ . Similarly, (0 , 0 , ) is orthogonal to any vector in the "plane" { ( a, b, 0) : a, b = R2 X {O J .

R

R

1

E R}

5 0 , any vector of the form (0, 0, a3 , . . . , a5 ) i s orthogonal t o any 0

In vector whose last 48 entries are O.

Let T be any set . The COZERO SET of a scalar-valued function x : T is coz x = {t T : x(t) # OJ . If two functions x and y of L 2 [0, 211"] have disjoint cozero sets, then x y = Hence x and y are orthogonal: ( x , y) = xy = O.

K

E

O.

f�7r

The sets

{ 1 / y'2;, cos

N } and { cos

E N} are orthonormal sequences in L 2 [ -1I", 11"] ; so is { e i n t /y'2; : E Z} . n

t / J?r : n E

n

t / J?r : n

n

A function x is EVEN if x (t) = x (- t ) for all t in its domain , O D D if x (t) = - x ( - t) for all t. In L 2 [ - 1I", 11"] , any even function is orthogonal to any odd function.

The linear subspace [S] SPANNED by a set S, or just the LINEAR SPAN K and of S, is the set of all linear combinations L �= l ai X i where ai S. For finite sets {Xl , X 2 , . . . or sequences we omit the braces and write [Xl , x 2 , . . . , x ] instead of [{Xl , X 2 , . . . , x } ] . Evidently, if x .1 S, then

Xi E

x .l [S] .

, Xn}

n

E

n

Example 1 .4.2 ORTHONORMAL SETS

(a) The so-called STANDARD BASIS for £ 2 ( n ) , the vectors

e i = (0, 0, . . . , 0, 1 , 0, . . . , 0) ,

1�i�

n,

with 1 i n the ith position and O's elsewhere, form an orthonormal set. If the l 's are replaced by arbitrary scalars ai , the vectors so obtained still form an orthogonal set . If we take sequences e i = where is the Kronecker delta, with all entries 0 except the ith, which is 1 , the vectors e i , i are an orthonormal set in £2 ; they are called the STANDARD BASIS for £ 2 . (b) An orthonormal subset of L 2 [ - 11", 11"] is given by the unit vectors

(8in),

E N,

1

sin t cos t sin t cos 2t .,f2-i ' ..ji ' ..ji ' ..ji2 ' ..ji , . . . . (c) In the complex space L 2 [ - 1I", 11"] the vectors

in Xn = e 2 11"t E Z , rn= ' n V

8i n

20

1. Metric and N ormed Spaces

comprise an orthonormal set . 0

=( +

+ ) = II ll +

As to the "Pythagorean theorem" in an inner product space, note that 2 2 2 ( 1 . 12 ) x y, x y x 2 Re x , y y , x y

/ I + ll

so

( ) + lI ll

( 1 . 13)

The Pythagorean relation holds for n orthogonal vectors Xl , . . . , X n 2 n

;= 1

;= 1

(

:

Exercise 5) . In real vector spaces, it follows from equation ( 1 . 12) that two vectors satisfy the Pythagorean equality ( 1 . 13) if and only if they are orthogonal. For any vector x and a unit vector y , observe that x- x , y y is orthogonal to y :

( )

( 1 . 14) ( x - ( x , y) y , y) ( x , y) - ( x , y) /l y ll 2 = 0 . Therefore , x - ( x , y) y is also orthogonal to ( x , y) y . By equation ( 1 . 13) , it =

follows that 2 x = x - x, y y

II

/l l1

( ) + ( x , y) Y ll 2 = II x - ( x , y) y ll 2 + I ( x , y) 1 2 ,

or, equivalently,

( 1 . 15) This is a special case of BESSEL'S EQ U ALITY, a topic we return to in Exercise 6, 3 .3 . 1 , and elsewhere. A LINEARLY INDEPENDENT subset S of any vector space is one such that for any X l , . . . , Xn E S, 1 a; Xi = ° implies that each ai equals 0. If two vectors x and y are not multiples of each other, then they are linearly independent . If S is an orthogonal set of nonzero vectors , then S is linearly independent : For any Xl , . . . , x n E S and any scalars aI , . . . , an and any j, if = 1 a i x i 0 , then

2:�

2::

=

Therefore, each aj is equal to 0 . Since they are mutually orthogonal, it follows that the standard basis vectors

(

ei

=

))

(0, 0, . . . , 0 , 1 , 0 , . ) , i E .

.

N,

of £2 Example 1 .4.2( a are linearly independent. The Gram-Schmidt the orem exhibits a process that converts an arbitrary linearly independent sequence Yl , . . . , Yn , . . . into an orthonormal one.

1 . Metric and Normed Spaces

21

1 .4.3 GRA M-SCHMIDT ff YI , " Yn , . . . is a sequence of lin early indepen dent vectors, then there exists an orth onormal sequence Xl , X 2 , . . " Xn , . . such that the lin ear spans [Xl , X 2 , . . . , xn] and [Y l , . . . , Yn] are equal for every "

.

n.

Proof. L e t a , b , c, d E Two simple observations: ( 1 ) If X and y are linearly independent , then ax + b y = ex + dy if and only if a = c and b = d. (2) For nonzero a and b, the linear spans [x , y] and [ax , b y] are equal. Let Y I and Y2 be linearly independent . The argument for Y I , Y2 demon strates the crucial inductive step . Let Xl = yI / I I Yl ll . Now subtract the projection of Y2 onto Xl from Y2 : Let w = Y2 - (Y2 , X l ) X l . As observed in equation ( 1 . 14) , w 1. X l . Could w be equal to O? If so, then Y I and Y2 would be linearly dependent. Since w =P 0 , we may consider the unit vector X 2 = wi I I w l l j {Xl , X 2 } is an orthonormal set such that [Yl , Y2 ] = [Xl , X 2 ] ' Assume that the result of the theorem holds for sets of n - 1 linearly independent vectors, that {YI , " " Yn } is linearly independent, and that {Xl , X 2 , . . . , Xn - l } is orthonormal and such that the linear spans [Xl , X 2 , . . . . . . , Xn - l] and [Y I , . . . , Yn - l] are equal . Let

K.

Wn = Yn Then for 1 � j �

n

n-l -

(Yn, x ; ) X ; . L ;= 1

- 1,

( wn , Xj ) = (Yn , Xj )

n- l -

L (Yn , X; ) (x ; , Xj ) = (Yn , Xj ) - (Yn , Xj ) = 0 . ;=1

As in the case for two vectors , Wn =P 0 . With

n Xn = W I I wn ll i t i s straightforward t o verify that {Xl , . . . , x n } has the desired properties. o

A BASIS , or HAMEL BASE, in a vector space X is a linearly independent collection B of vectors such that any X in X may be expressed as a (finite) linear combination X = ai X; of vectors from B. The standard basis

2:7= 1

e i = (0 , 0 , . .

.

, O, � O, . . .) , i E N , i th

is a Hamel base for the space cp of sequences all but a finite number of whose entries are OJ the members of cp are also called "finitely nonzero sequences ." Despite the name "standard basis," it is not a Hamel base for £ 2 j however, it is a "Schauder basis" for £ 2 (see Definition 7 . 7 . 1 ) . Note that the possibility of using one finite set of basis vectors to express X and

22

1. Metric and Normed Spaces

another set to express a different vector y is perfectly possible . If there is a finite basis { X l , X 2 , , xn } for a vector space X, then all bases for X have n elements . This fact enables us to define DIMENSION as dim X = n for vector spaces with finite bases; we also say that X is FINITE-DIMEN SIONAL or n-DIMENSIONAL. If B is an infinite basis for X , then any basis for X has the same cardinality as B , and this cardinal is defined to be the DIMENSION of X. Some other facts about bases are : •

• •

•

•

Every vector space has

a

basis .

Any linearly independent set can be extended to a basis.

The proofs of these statements involve what is known as "the standard Zorn's lemma argument" (see, for example, Bachman and Narici 1966 , pp. 1 57-160) . The argument that a vector space X has a basis runs as follows. Choose a nonzero vector Xl E X . If the linear span [xd is not X, there must be a vector X2 E X not in [X l] . If [Xl , X2] :/; X, choose X s fi. [X l , X2]. Then consider the question Is [X l , X 2 , xs] = X. The problem is, Does the process ever end? This is where Zorn 's lemma is used. Zorn's lemma guarantees the existence of a maximal linearly independent set of vectors , which is indeed a basis . As will be seen later, the term "basis" has other distinct meanings (orthonormal, Riesz, and Schauder bases) . As an immediate corollary to the Gram-Schmidt theorem 1 .4.3, it follows that 1.4.4 Every finite- dimensional inner produ ct space has a basis of orthonor mal vectors.

The parallelogram law of plane geometry states that the sum of the squares of the diagonals of a parallelogram is equal to the sum of the squares of the sides . For vectors X, y E R2 , this is equation ( 1 . 16) below. Equation ( 1 . 16) survives in inner product spaces. The formulation below follows directly from equation ( 1 . 12) applied to J l x y J l 2 and J l x - yJ l 2 . The terms ± Re (x , y) drop out.

+

1 .4.5 PARALLELO GRAM LAW For all X and y in an inner produ ct space,

( 1 . 16)

An imp ortant use of the parallelogram law is a test to see whether a norm comes from an inner product . Although an inner product-derived norm satisfies the parallelogram law , it is not satisfied by every norm. For example , consider £ 1 (2) = R2 with the taxicab norm (equation ( 1 . 6) of Section 1 .2 ) ; let X = ( 1 , 0) and y = (0, 1 ) . Then Il x yJ l = J l x - yJ l = 2, so

+

I l x + Yl 1 2 + J l x - y J l 2 = 8 to 2 1 1 x J l 2 + 2 J 1 y J l 2 = 4 .

1 . Metric an d N armed Spaces

23

The taxicab norm is therefore not inner-product-induced. Another feature that distinguishes inner-pro duct-derived norms from general norms is the polarization identity of Exercise 3.

Exercises 1 . 4 1 . Find two vectors x and Y from the complex inner product space £2 (2) for which the Pythagorean relation holds but that are not orthogonal. 2. Orthonormal vectors cannot be too close to each other. For any two orthogonal unit vectors x and y show that I I x - yl l = .../2 .

3 . POLARIZATION IDENTITY (a) Show that any inner product satisfies the P O LARIZATION IDEN TITY

{x, y} =

� { l i x + yl l 2 - l i x - yl 1 2 + i I I x + iyl l 2 - i I I x - iy11 2 }

( 1 . 17)

for all x, y E X . All it takes is fortitude to verify it. (b) The polarization identity provides a way to test a norm for inner product status. Suppose that X is a complex normed space whose norm satisfies the parallelogram law 1 .4.5. Show that equation ( 1 . 17) defines an inner product on X such that I I x l l = �. An inner product space is therefore a normed space whose norm satisfies the parallelogram law . (c) Show that neither .ep (n) nor .ep i s an inner product space for P :F 2 . 4. Show that i f x .1 S, then x i s orthogonal t o any linear combination L:?= l a i X i of vectors Xi from S, i.e . , x .1 [S] .

5.

PYTHAGOREAN THEOREM If {Xl , X 2 , show that 1 I L:?= l x i l l 2 = L:?= l l l x i Il 2 .

.

.

•

, xn } is an orthogonal set ,

6 . BESSEL'S EQUALITY AND INEQU ALITY (cf. 3 . 3 . 1 )

(a) Let {Xl , X 2 , , xn } b e an orthonormal set . Use the Pythagorean theorem of Exercise and argue as in equation ( 1 . 15) to show that for any vector x, BESSEL'S EQUALITY holds : .

•

.

5

24

1 . Metric and Normed Spaces

(b) Deduce BESSEL'S INEQ UALITY from this:

n I (x , x i} 1 2 ::; I I x l l 2 . L ; =1

(c) If (xn ) is an orthonormal sequence and x any vector, show that I (x , x i} 1 2 ::; I I x l l 2 for any n. Deduce from this that I (x , x i} 1 � ° for any x.

E� 1

Hints

3 (c) . See how the parallelogram law works on ( 1 , 0) and (0, 1 ) in ip (2) . 6 . Peek ahead to 3 .3 . 1 .

1.5

Linear Isometry

We have already discussed our criterion for saying that two metric spaces are "the same as metric spaces" in Section 1 . 1 , namely, isometry. We pro mulgate analogous criteria for normed and inner product spaces in this section . By this means, we can prove that for any n E N, there is essen tially only one n-dimensional inner product space: (Exercise 5 ) . We show in 3.4.9 that any infinite-dimensional separable Hilbert space is "in ner product isomorphic" to i2 • This leads to an interesting choice: One can view L 2 [0, 1] as a function space or as the sequence space i2 • For any two closed intervals and the spaces of continuous functions C and C are linearly isometric (Exercise 7) , so there is no loss in focusing attention on C [0 , 1] . Thus, the notion of linear isometry also enables us to exclude many apparently different objects as only trivially different . A map A : X � Y between vector spaces X and Y is LINEAR if for any scalars and and any vectors x and

Kn

[a, b)

[c, d] a

[c, d] ,

b

[a, b)

y,

A (ax + by) = aAx + bAy. An injective linear map is called a LINEAR ISOMORPHISM or j ust an ISO MORPHISM; if a linear bijection exists between two vector spaces, then they are called LINEARLY (or ALGEBRAICALLY ) ISOMORPHIC . Example 1 . 5 . 1 LINEAR MAPS

(a) Let m , n E N. Let T be an m x n matrix and let Tx denote mul tiplication of a column vector x with n elements by T. The map � X 1-+ Tx , is linear . Matrix multiplication is essentially the only linear map between finite-dimensional vector spaces.

Rm ,

Rn

1.

Metric and Normed Spaces

25

(b) Differentiation and integration are linear when defined on vector spaces of differentiable and integrable functions, respectively. ( c) Any n-dimensional vector space over K is linearly isomorphic to K n (see the proof of 1 .5 .4) . Hence , any two n-dimensional spaces over K are linearly isomorphic to each other. If {Xl , x 2 , . . . , xn is a Hamel base for X and ei = (0, 0, . . . , 0, 1 , 0, . . . , 0) , 1 :S i :S n, then the map L:�= l ai Xi -+ L:� l ai ei is a surjective isomorphism . 0

}

Definition 1 . 5 . 2 LINEAR ISOMETRY

Let A : X -+ Y be a linear map . (a) NORM IS OMORPHIC Let X and Y be normed spaces. We say that A is N ORM PRESERVING if (using the same symbol for norm in each space)

I I Ax l 1 = Il x l l

for every X E X j w e also say that A is a LINEAR ISOMETRY o r a N O RM ISOMORPHISM. If A is onto, then X and Y are called LINEARLY ISOM ETRIC or N O RM ISOMORPHIC or NORM ISOMORPHS , and we write X e:: Y . (b) INNER PRO D UCT ISOMORPHIC If X and Y are inner product spaces and

( Ax, Ay) = (x, y)

for all x , y E X , then A goes by the names INNER PRODUCT PRESERVIN G , INNER PRO DUCT ISOMORPHISM, and UNITARY O PERATOR. If A is onto, then X and Y are ISOMORPHIC AS INNER PRODUCT SPACES or UNITARILY EQUIVALEN T. 0 Norm preserving and inn er produ ct preserving are equivalent on inner product spaces.

1.5.3 EQUIVALENCES Let A : X -+ Y be a lin ear m ap of the inner produ ct space X into the inner produ ct space Y. Let d be the norm- derived metric: d (x , y) = I l x - Y I I . Then the following are equivalent: For all x and y , (a) A is an isometry: d (x , y) = d (Ax, Ay) . (b) A is norm preserving: I I x l l = I I Ax l i . ( c ) A is inn er produ ct preserving: (x , y) = (Ax , Ay) . Proof. If A is an isometry, then

d (Ax , 0) = I I Ax l 1 = II Ax - 0 1 1

=

d (x, 0) = I l x - 0 1 1 = I l x l l ,

so (a) implies (b) . We argue that (b) => (c) for complex spaces only. If A preserves norms, then by the polarization identity (Exercise 1 .3-3) ,

4 (x, y)

I I x + Yl 1 2 - l l x - y l 1 2 + i I l x + i y l1 2 - i Il x iy l 1 2 II Ax + Ay l l 2 - I I Ax - Ay l 1 2 + i I I Ax + iAyl 1 2 - i I I Ax - iAy l 1 2 4 ( Ax, Ay) , _

26

1 . Metric and Normed Spaces

so A preserves inner products. The reverse implications are straightforward. o

For any n E N, there is essentially only one n-dimensional real or complex inner product space. 1 .5.4 FINITE- DIMENSIONAL INNER PRO DUCT SPACES Let n E N . Any real or complex n-dimensional inner produ ct space is isomorphic as an inner produ ct space to real or complex 12 (n), respectively.

}

P roof. By 1 .4.4, X has an orthonormal basis {X l , X 2 , . . . , x n . Consider the linear map

Kn , X = :L� l a ; x; ( al , a 2 , . . . , a n ) . By the previous result , it suffices to show that A preserves norms: A:

X

--

Is there essentially (up to linear isometry) only one n-dimensional n o rm ed space? No. Indeed, loo (2) is not linearly isometric to 12 (2) , a point we ask the reader to prove in Exercise 3 . But n-dimensional normed spaces can be identified in a weaker sense: Any two n-dimensional normed spaces over the same field are lin early homeomorphic (Definition 2.9.5) . This nontrivial result of Exercise 2 .9-5 follows from the fact that all norms are "equivalent" in a certain sense, discussed in Section 2.9, on an n-dimensional space (Exercise 2 .9-3).

Exercises 1 . 5

1 . INJECTIVE LINEAR MAPS Let X and Y be linear spaces and let A : X ---4 Y be linear .

=

=

(a) Show that AO O. (Write 0 as 0 + 0.) (b) Show that A is injective if and only if Ax 0 implies X = O. (Write 0 as AO.) 2. ISOMETRIES O F THE PLANE R2 Show that any inner product space isomorphism of 12 (2) onto itself may be represented by a matrix multiplication = Ax where A is of the form

y

( COS

(} - sin (} sin (} cos (}

) ( or

cos (} sin (}

sin (} - cos (}

)

.

Deduce from this that the only linear isometries of 1 2 (2) are rotations through an angle (}, reflection about a line making an angle (} /2 with the x-axis, or compositions of these . If we drop "linear ," there are other distance-preserving maps-translations , for example .

1 . Metric and Normed Spaces

3.

(2) = (R2 , 1 1 · 1 1 2 ) is not linearly isometric to £00 (2) 2 (R , 1 1 ' 11 (0 ) ' 4. Show that £2 (N) is inner product isomorphic to £2 (Z) . Show that real £2

27 =

5 . FINITE-DIMENSIONAL INNER PRODUCT SPACES Show that any n dimensional inner product space X over K is inner product isomor phic to K n . 6 . LINEAR ISOMETRIES O F £2 ( n ) Let

n

E N.

(a) Show that a linear isometry of the real space £2 ( n ) onto £2 ( n ) maps any basis of orthonormal vectors into another basis of or thonormal vectors . (b) Let A : £2 n ) � £2 n ) be a surjective linear map . Show that A is an isometry if and only if the columns of the matrix associated with A are an orthonormal set in K n .

(

(

C [a, b) Let C [a, b) be as in Example 1.2.3. (a) Show that C[O, 1] is linearly isometric to C[l, 2]. (b) Show that the map x (t) x( l + t 3 ) is a linear isometry of C[1, 2] onto C[2, 9] . (c) If h [c, d) � [a, b] is continuous, must the map x x h map C [a, b] linearly isometrically onto C [c , d)?

7. LINE AR ISOMETRIES O F

1--+

:

�

0

8 . Let s denote the linear space (with operations as in £2 ) of all summable sequences (an ) normed by l I (an) 1 I = s UP k 1 2:7= 1 ai .

l

(a) Show that s is linearly isometric to the normed space ( c, 11 . 1 1 (0 ) of ai) E all convergent sequences under the map (an) E s 1--+

c.

x of s into the space ( co , 1 1 - 1 1(0 ) = O} , 11 . 1 1(0 ) of all null sequences is a contin

(b) Show that the identity map x

( {(an) E /{OO an :

�

( 2:?= 1

1--+

uous linear map but not an isometry.

Hints

2.

First note that reflection of a point with polar coordinates ( r, 8) about a line through the origin making an angle with the x-axis takes the point into a point with polar coordinates ( r, 8) . I n Cartesian coordinates, therefore, this amounts to the following multiplication:

cp

( COS 2cp2cp sin

2cp

2cp ) ( x ) ' 2cp y

sin - cos

-

1 . Metric and Normed Spaces

28

(�), (� )

and the vector

(�)

)

a2 b2

Consider the effect of the isometry

on the unit vectors

, which is of length .j2, and solve the

resulting equations for a I , a 2 , bl , and b 2 .

3. Use the polarization identity of Exercise 1 .3-3 .

Holder and Minkowski Inequalities ; Lp and lp Spaces

1 .6

We prove the Holder inequalities in 1 .6 .2. We use them to prove the Minkowski inequalities in 1 .6 .3. The Minkowski inequalities tell us that the and spaces are normed spaces. First, we need the following numerical fact.

fp

1 .6 . 1 Let

aP

b9

-- + - . p q P roof. Let

a,

b � O. If p > 1 and q are such that

1+1

-

-

p

q

=

Lp

1 , then ab <

y = xp - l and let Al and A 2 be the areas: a b aP b9 A l = l xp - l dx = and A 2 = l y9 - 1 dy = 0

--

o

and note that Ij(p

-

1) = q

P

-

q

1 . By examining the diagram below

y

y

=

x

p l -

b �------�--� b i-------:.r-

--+-��----------�--� x a

--+-��----� x

(b)

(a)

The function xp

-

l

.

( a)

The case

a > b. (b)

The case

b > a.

represents the area between the curve and the x-axis and area between the curve and the y - axis

.

In each case

A2

Al

represents the

1 . Metric and Normed Spaces

29

aP - 1 > b and aP - 1 S; b, we see that the desired A simple way to see the validity of 1 .6 . 1 for p = 2 is as follows: (a - b) 2 = a 2 - 2ab + b 2 � 0 , so a 2 + b2 ' ab S; "2 "2

and considering the cases result holds . 0

The Holder inequality tells us when a certain sum of products is less than or equal to something like the product of the sums. 1 = 1, + p q

1 .. 1 .6.2 THE HOLDER INEQU ALITIES For p > 1 and q such that

-

aj , bj , j = 1 , 2, . . . , n ,

the following assertions hold: (a) For complex numbers

-

(an) and (bn) of complex numbers such that E�l l ai /P < q it follows that E� l l a i l l bil < and ( E� l I bil )

(b) For sequen ces 00 and <

x

00 ,

y I x(t)y(t) 1 dt

00

(c) Let and be scalar-valued Lebesgue-measurable functions on the Lebesgu e-measurable set T such that IT < 00 and IT < < 00 and 00 . Then IT

£ I x(t)y(t) 1 dt

I x(t) I P dt

S;

(£ I x(t) IP dt) (£ l y(tW dt) l /p

l y(t W dt

l/q

Proof. (a) We assume that the sums on the right side of the inequality are not zero, the result being trivially true in that case. With reference to 1 . 6 . 1 , let

so that

for every i. Summing over i,

30

1. Metric and Normed Spaces

Now note that the sum on the right is

-p1 + -q1

= 1,

from which the desired result follows. (b) By (a) , for any n ,

It remains only to let n -+ 00 on the left side. ( c ) To get the result for integrals, instead of ( 1 . 18 ) , we start with

I x (t) 1 l p _--,I�y(:.t) ,--...:.I --:-l < I x (t) iP + l y(t W (IT I x (t) i P dt) / (IT l y (t W dt) / q - p (IT I x (t) i P dt) q ( IT l y (t W dt)

and then integrate both sides . 0 As in the Holder inequalities, there are versions of the Minkowski in equality below for finite sums, infinite sums, and integrals. We prove only a) ; assertions (b) and ( c ) follow from (a) as in the previous proof.

( 1.6.3 THE MINKOWSKI INEQU ALITIES For 1 :S p < 00 and scalars a ; and b; , I ( a) (2:;= l l a ; + b; l P f l P :S (2:;= l l a ; I P fIP + ( 2: ;= l l b ; I P f P . then ( b) If2:: l l a ; I P < 00 and ( 2:: 1 I b; I P ) < 2: i = l l a ; + b; I P < 00 ,

00

and

00

(� lai + b;lP ) lip (� lailP ) lip + (� IbilP ) lip 00

:s

00

00

(c) Let 1 :s p < 00 . For scalar-valued Le besgue-measurable functions and on a L ebesgue-m easurable set T such that IT < 00 and , ha ve we 00 < ( IT

x y dt) xI (t) iP dt l y(t W (£ I x (t) + y(t) iP dtr /P (£ I x (t) IP dtr /P + (L ly(t) iP dtr /P :s

31

1 . Metric and Normed Spaces

( I a l + I b l ) it is clear that ( I a l + I b l t = ( I a l + I b D P - 1 l a l + ( I a l + I b I )P - 1 I b l · Replace a by a i , b by bi , and sum over i to get

Proof. (a) By factoring out

Apply the Holder inequality to each term on the right-hand side, and use the fact that (p - l)q = p to get

E;= l ( l ad + I bi l )P

�

(E: 1 ( I ai l + I bi l t f / q1 /(qE: 1 I ai IP f /Pl /p + (E;= l ( I ai I + I bi I t ) ( E;= 1 I bi I P )

( 1 . 19) = 0, then the Minkowski inequality obviously holds; if not, we can divide both sides of ( 1 . 19) by and use the fact that 1 - 1 j q = 1jp to get

lq If (2:�= 1 ( l ad + I bi l t ) /

(2:7= 1 ( I ai l + I bi D P ) l / q

which implies that

(t, la; + b; I') (t, l a; l') + (t, l b; l') 'I,

' I, ,;

' I,

0

It is now clear that

II · lip ) is also denoted by fp (n). 1 Recall that sequences (an) of complex numbers such that (2:: 1 I a iI P ) / P < are called pth power summable. By the Minkowski inequality 1 . 6 .3 ( b ) ,

(

defines a norm on K n . We recall that K n ,

the collection fp = fp (N) , 1 � p < 00 , of pth power summable sequences forms a linear space with respect to poin twise operations, and 00

( 1 . 20)

defines a norm on fp . Actually, fp is a complete normed space, or Banach space, in the sense of Definition 2 .6 . 1 .

32

1. Metric and Normed Spaces

For 1 ::; p < 00 , Lebesgue-measurable K-valued functions defined on < 00 T, where T is a (finite) closed interval or R, such that IT are also called pth power summable; they form a vector space with respect to pointwise operations. By the Minkowski inequality 1 .6 .3( c) , the map

x I x(t) IP dt

( 1 .21) i s almost a norm-it fails only because a nonzero function that i s 0 almost everywhere has 0 "norm." In order that this not happen, as mentioned in Example 1 . 1 . 8 we consider equivalence classes of such functions with respect to the equivalence relation a.e . To add equivalence classes x and fj we consider the class determined by the pointwise sum of any two representatives, i .e . , x + fj +

x=y

= x y.

Scalar multiplication i s managed similarly. The normed space s o obtained is denoted by We refer to members of as functions rather than as classes. The Minkowski and Holder inequalities for integrals hold more generally (see Dunford and Schwartz 1958 , pp. 1 1 9-120) . Let E be a IT-algebra of subsets of a set T and J.l a positive measure on E. Then 1 ::; -+ K such p < 00 , denotes the class of J.l-measurable functions J.l) that IT < 00. As for Lebesgue-measurable functions, actually denotes equivalence classes of functions, two functions f and 9 being equivalent if their difference is 0 everywhere but on a set E E E such that J.l (E) = 0 (we also say f = 9 J.l-a.e .) . With this convention,

Lp (T).

Lp (T)

Lp (T, J.l), x:T Lp (T,

I x(t) I P dt

( 1 .22)

Lp(T, J.l) that makes it a Banach space. Notation If we write Lp [a, b] or Lp (R) , J.l denotes Lebesgue measure and E stands for the Lebesgue measurable sets of [a, b] or R, respectively. Each fp = fp (N) , 1 ::; p < may be viewed as a space Lp (T, J.l) in which one takes T to be the set N of positive integers and one takes the defines a norm on

00 ,

measure J.l defined on the IT-algebra of all subsets of N to be "number ," i.e., for E C N , J.l (E) is the number of elements in E if E is finite, otherwise 00. The norms of equations ( 1 .20) , ( 1 . 2 1 ) , and ( 1 .22) are known as p-N ORMS. The linear spaces J.l) of "essentially bounded" functions described next are similar in some respects to the spaces J.l) , 1 ::; p < 00 . Let T, E , and J.l b e as i n the discussion of J.l) above . A function : T -+ K that is bounded J.l-a.e. on T is called ESSENTIALLY BO UNDED.

1I · lIp

L oo (T,

x

Lp (T, Lp (T,

1 . Metric and Normed Spaces

33

The vector space of all equivalence classes ( modulo equality J-l-a.e . ) of es sentially bounded functions on with pointwise operations is denoted by J-l ) , J-l ) . It is normed by the ESSENTIAL SUPREMUM: for E

x T x Loo (T, I l x l oo = ess sup I x (T)I = inf {k I x l k J-l-a.e. } . As we have done for the Lp spaces, we refer to the members of L oo (T, J-l ) "functions" rather than "equivalence classes." Loo (T,

:

::;

as

Exercises 1 . 6 1 . HOLDER INE QUALITIES FOR P = 1 , q = 00 Prove the following:

(a and ) i a (bi ), if Li l d < 00 and (bi ) is bounded, then Li l ai bil < 00 and

( a) SUMS For n-tuples or sequences of complex numbers

T [a , b] I x (t) 1 dt I x (t)y(t) 1 dt h , x (t)y(t)1 dt ::; (h , x (t)' dt) (ess sup l y (T) I ) .

= or R and let x be a Lebesgue-measurable function (R- or C-valued ) such that IT < 00. If y is an essentially bounded Lebesgue-measurable function defined on T, then IT < 00 , and

( b ) INTEG RALS Let

2 . MINKO WSKI INE QUALITIES FOR P = 00 Prove the following:

(aIib)i . (b; ) I (an ) ; l a;l ·

of complex numbers, and Deduce from this that the + sup ; + b; ::; sup ; sup ; collection of bounded sequences is a vector space normed by = sup

( a) SUMS For bounded sequences

l ai RooI

l a; I I (ai) l oo

( b ) FUN CTIONS For essentially bounded functions

[a , b] or R, ess sup I (x + y) (T) I

on T =

::;

( ess sup

x

and

y

defined

I x (T) I ) + (ess sup l y(T) I ) .

2

Analysis Now that we have some spaces in which to work , we can set about replicat ing the basic structures of analysis-convergence, continuity, etc. The first thing we generalize is the idea of neighborhood of a point.

2.1

B alls

The open neighborhoods of a point x of R are the open intervals containing x; the open neighborhoods of x = (a, b) E R2 are the open disks

We generalize the notion of open disk in the plane to metric spaces by means of the notion of open ball. As shown below , these balls need not be very round. In normed spaces X, the unit ball U about the origin is a micro cosm of the whole space: In many imp ortant respects , if you know U, you know X. Definition 2 . 1 . 1 BALLS AND NEIGHBORHOODS

Let (X, d) be a metric space. Let x E X and r > O. The O PEN and CLOSED BALLS of RADIUS r ABOUT x are given by the sets, respectively,

B(x, r ) = {y E X : d(y, x) < r} and C(x, r) = {y E X : d(y, x) :::; r} .

Any set big enough to contain a ball of positive radius r about a point x is called a NEIGHBORHOOD of x. The open balls B(x, r) about x are called the BASIC NEIGHBORHO ODS OF x. The sub class B(x, l in) , n E N, is an especially imp ortant one because it is denumerable. Among other simpli fying features, this renders mathematical induction possible . The SPHERE S(x , r) OF RADIU S r ABOUT x is the outer shell of C(x , r) :

S(x , r) = { Y E X : d(x , y) = r} . O In the Euclidean spaces R, R2 , and R3 ( 1 . 1 . 2) , open and closed balls

are open and closed intervals, disks, and spheres, respectively. By the U N IT BALL in any normed space X is meant the closed unit ball

U = U(X) = C(O, 1) = {x E X : " x II

:::;

l} .

36

2 . Analysis

Knowledge of the unit ball in a normed space X is very imp ortant infor mation. For one thing, the shape of the unit ball yields the shape of a ball of any radius about the origin, since

C(O, r) = rU for any r > O. The shape of the unit ball U determines the shape of any ball centered at any E X, since for any > 0,

x

r

C (x, r)

C(x, r) = x + rU = {x + ry : y E U} . U

Knowledge of even enables us to reconstruct the norm (Exercise 3 ) . As illustrated i n Example 2.1 .2, the unit ball i n a normed space can b e a cube , a diamond, o r the whole space. I n an ultrametric space (defined in Exercise 1 . 1- 1 3) every point in an open or closed ball is a center (Exercise 4(b)) .

U

Example 2.1.2 SHAPE O F THE UNIT BALL

U

U in ip (2) for p = 1 , 2 , and 00 . Consider the real space ip (2) , i.e . , x = (a, b) , II x li p = ( l a lP + IW ) l /P , 1 � p < 00 . For p = 1 , lI (a, b) l h = l a l + I b l · The metric d (x, y) = II x - y 11 1 induced by 11 · 11 1 is the taxicab metric of Example 1 . 1 .5; the unit ball U consists of (a)

R2 normed by the p-norm: with

points that can be reached by a sequence of horizontal and vertical moves that add up to 1 or less-a tilted unit square (or diamond) centered at the origin with vertices ( - 1 , 0), (0, 1 ) , ( 1 , 0) , and (0, - 1 ) . (In id3) the unit ball is a tilted unit cube centered at the origin.) For p = 2,

U

=

origin. For p = 00 ,

{ (a, b) : Vl a l 2 + I b l 2 I } �

, the unit disk about the

lI (a, b) lloo = m ax { I a l , I b l } ,

and U is a square of side 2, centered at the origin with sides parallel to the axes . Generally, as p gets bigger, the unit ball determined by bulges steadily outward , going from the unit diamond (p = 1) to the unit square (p = 00 ) (Exercise 2) . (b) If R3 carries the trivial metric of Example 1 . 1 .6 , = R3 , while 1) is just the origin. (c) Any orthonormal subset S of an inner product space is a subset of the unit ball. In particular, the subset

1I ·l Ip

U

B(O,

1 sin t cos t sin 2t cos 2t ., ..j2; ' Vi ' Vi ' Vi ' Vi , . . of Example 1 .4.2(b) is a subset of the unit ball of

L 2 [0 , 211"] .

2. Analysis

37

(d) If M is a subspace of the metric space (X, d) , closed balls in M are of the form ( m , r) n M, 0, m E M. Thus, if you view R as a subspace of R 2 = (2) , the unit ball of R, the interval ( - 1 , 1 ) , is the unit ball of R2 intersected with the line R. 0

C £2

r>

Exercises 2 . 1

(

1 . Identify the unit ball in the space X 3 ) of Exercise 1 . 1- 1 2.

Up 1I · lI q 11 ·lIp

denote the unit ball of 2 . p-BALLS INCREASE AS P INCREASES Let ( n ) for any n E {oo} , 1 � p � 00 . Part (b) shows that for is equivalent 1 � p < q < 00, The condition � � to C in ( n ) .

£p

Up Uq £q

NU 1I · lI q 1I · llp ·

(a) In R 2 sketch = � I } for p = ! 3 4 . (You : need only sketch it in the first quadrant to see what it looks like .) (b) For 0 < r � s and nonnegative numbers 1 show r 1/8 �n (�n l/ r 8 t h at ( LA = l ) � LA = l ) .

, ,

U {(a, b) l a lP + IW

a , a 2 , . . . , an,

ai (c) If the series Ei EN a i and Ei EN a i converge, show result of (b) persists for the infinite series. (d) For 1 � p < q � 00 , show that Up Uq . ai

that the

C

3. RE CONSTRU CTIN G THE NORM As mentioned before Example 2 . 1 . 2 , knowledge o f the unit ball of a normed space X i s vital information. = inf Show , in particular, that 0 : E U } so that the norm can actually be reconstructed from its unit ball.

U II x ll

{a > x a

4. ULTRAM ETRIC GEOMETRY Let (X, d) be an ultrametric space (Ex ercise 1 . 1- 1 3 ) . Prove the following:

( a)

For any

x, y, z E X, if d (x, y) # d (y, z), then d (x, z) = max [d (x, y) , d (y, z)] .

(b) Show that any point in an open or closed ball is a center, i.e . , r) . for any E X , and 0, y E B r) => B r) = B (c) If two balls meet , then one of them is a subset of the other. (d) Take "triangle" to mean any triple of distinct points of X. Show that all triangles are isosceles.

x, y

r>

(x,

(x, y, z)

(x,

(y,

38

2. Analysis

Hints 2(b) . Let

d = E?= l ai , and note that

) 1/s E?= l �ir d ((E?=l r) l / s i S

<

2.2

1

Convergence and Continuity

(an)

A sequence of real numbers converges to a if for any r > 0 , there exists a positive integer N such that < r for all n � Nj in other words an E r ) for all n � N. This has the following generalization to metric spaces.

I an - a l

B(a,

Definition 2 . 2 . 1 CON VERGENCE

(xn)

We say that a sequence from the metric space (X, d) CON VERGES to E X if for any r > 0, Xn E r ) for all n greater than or equal to some positive integer N where N usually depends on r . We frequently abbreviate the expression "for all n greater than or equal to some positive integer N" to EVENTUALLY or FOR SU FFICIENTLY LARGE n. Two others 0 ways to denote converges to are Xn and limn Xn =

x

B(x,

x"

�x x. Limits must be unique, for if Xn x and Xn y, then eventually o :S d(x, y) :S d(x, xn) + d(xn, y) < r for any r > O . The only possibility for d(x, y) is therefore O . If (mn) is a sequence from a subspace M of the metric space (X, d), then mn m E M in the subspace (M, di M ) if and only if mn � m in (X, d) as is trivial to verify. Therefore , if Xn � x in £2 (2) then X n � x in £ 2 (n) , for every integer n�2; Xn even converges to x in £2 (N) = £2 . "xn

�

�

�

Though hardly profound, it is useful to note the following alternative descriptions. 2.2.2 EQUIVALENTS O F CONVERG ENCE In any normed space X, the fol lowing statements are equivalent: a) n . (b) For any r > 0, Xn - E rU eventually, where U denotes the closed unit ball of X.

( x �x

(c) xn - x � O . II xn - x I I � O.

( d)

x

2. Analysis

39

(e) For any r > 0, E r) for all but a finite number of values of n . Alternative phrasings are ALMOST ALL E r) E r) and

xn B(x,

EVENTUALLY . (f) For any neighborhood V of

x n B(x,

X n B(x,

x, x n E V eventually.

The following elementary facts are also quite useful. 2.2.3 LIMITS OF SUMS AND PRODUCTS In any norm ed space: converge t o converge to a n d the vect ors (a) SUMS If the vectors � then (b) SCALAR PRODUCTS If the vectors converge to a n d the s calars an converge to then �

Xn x Yn Xn + Yn X + Y ; x Xn a, anXn ax. Proof. (a) If Xn � x and Yn � y, then II xn + Yn - (x + y) II ::; II xn - x ii + ll Yn - y ll � 0 + O. (b) If X n x and an � a, then Il anxn ax i l II (an - a) (xn - ) + a (xn - x) + (an - a) x II ::; I an - a l II xn - x ii l a l II xn - x II + I an - a l II x ll -- 0 . 0 + l a l · 0 + 0 · ll x li . 0 The epsilon-delta formulation for continuity of x R � R at a point a

y,

�

-

x

+

:

is that

I x (t) - x (a) 1 < B (a, 6)

for any f > 0, there exists 6 > 0 such that for all satisfying < 6; in other words, t E E

t It - a l x (t) B (x (a) , f ).

f

==>

The notion o f continuity i s easily extended t o metric spaces (S, d) and (T, We say that : S � T is continuous at E S if

d') .

x

a

d'(x(t), x( a)) < t B (a, 6) x (t) E

for any f > 0, there exists 6 > 0 such that for all satisfying < 6 ¢:::} E

d( a, t)

t f B (x (a) , f ).

=>

In normed spaces the formulation of continuity below ( SEQUENTIAL C O N TIN UITY ) is usually the best one to use, i.e . , usually better than the f-6 description . Definition 2.2.4 CO NTIN U ITY

x:

A mapping S � T where S and T are metric spaces is said to be CON TINUOUS AT THE POINT E S if for any sequence � � A function continuous a t every point o f its domain is called CONTI N U O U S . I f : S � T i s bijective and and are continuous, then i s BICON TIN UO U S , or a HOMEOMORPHISM. 0

x

s

x

x- I

Sn S, x(sn) x(s). x

40

2. Analysis

The sequential description of continuity above is equivalent to the f -b formulation; we ask the reader to prove this in Exercise 4. Continuity of the function at the point s says that the interchange of function and limit is permissible-limits may be computed by substitution :

x

lim n sn = S

If x

==>

n x( sn) x( linm sn) x (s) .

lim

=

=

: S � T is continuous, then the restriction of x to any subspace M of S is also continuous.

Example 2.2.5 CONTINUITY OF BASIC OPERATIONS

Let X be a normed space. •

•

Xn x

II x nll II x ll by the second IIl x ll - II Y II I ::; II x - yll of Exercise 1 .2-2 ) ,

(a) NORM Since � in X implies that (namely, triangle inequality it follows that 1 1 1 1 : X � R is continuous.

�

Xn

x,

� (b) INNER PRODUCT If X is an inner product space and then � for any E X since , b y the Cauchy-Schwarz inequality 1 .3.2,

(xn, y)

y

(x, y)

I ( xn - x, y} 1 ::; II xn - x ll li y li · In other words, for any y E X , the map fy X � K, x ...... (x, y), :

is continuous; we also say that the inner product is continuous in its first argument while the second is held fixed. This is improved to joint continuity in Exercise 3 . •

( c ) S U M AND SCALAR PRODUCT The statements of 2.2.3 are actually � � continuity statements, since statements such as and � can be combined into the one convergence statement Let us elaborate. Equip X x X and K x X with any of the product metrics of Example 1 . 1 .4; on X x X, take doo or

"xn x

y" (x, y).

Yn (x n , Yn)

� EX For 1 p < 00, it follows from the equation above that is dp-convergent to The � E Y if and only if and arguments for K x X , 1 ::; p < 00, and d oo are equally immediate. We may therefore paraphrase 2.2.3 by saying that the maps

::; Yn Y

Xn x (x, y).

(xn , Yn)

XxX

(x, y)

are continuous. 0

X

x+y

and

K x X -- X

(a, y)

I---->

ax

2. Analysis

41

)

The results of Example 2 . 2.6( b and its generalization in ( c ) are very imp ortant . They provide practical ways to verify convergence in higher dimensions as well as an easy way to fabricate new continuous functions from old ones; by ( c ) , since each component map of ( 1 + t , e t ) is continuous, it follows that the map t ...... ( 1 + t , ) is a continuous map of R into R 2 . One way of defining "continuous arc" in the plane is as the continuous image of the closed interval [0 , 1] . As Peano dramatically demonstrated with his space-filling curve-a continuous map from [0 , 1] onto the unit square [0 , 1] x [O , I ] -this definition leads to things that do not look like our usual idea of continuous arcs.

et

Example 2.2.6 CON VERGENCE EXAMPLES

(

a ) In any metric space, any sequence that is eventually constant con verges to that constant . In a trivial metric space Example 1 . 1 .6) these are the only convergent sequences . ( b ) CON VERGENCE vs. COMP ONENTWISE CONVERGENCE Consider the k-dimensional space 1.2 ( k ) , k E N , of Example 1 . 1 .2. We show here that convergence always implies componentwise convergence; the converse is true in the finite-dimensional spaces 1.2 (k) , k E N , but not in the infinite dimensional space 1.2 = 1.2 ( N ) . To see that convergence implies comp onentwise convergence, consider a sequence in 1.2 • Each Xn is a sequence and we denote its jth comp onent by X U) . If X n --+ then for any j ,

(

n

(xn)

li� X n U) =

x,

x U) , since I X nU) - x U ) 1 2 � L I Xn(i) - x(i) 1 2 = II xn - x lI � · iEN

Since for any positive integer k, 1.2 ( k ) is a subspace of 1.2 , w e conclude that if a sequence of k-tuples E 1.2 ( k ) converges to E 1.2 ( k ) , then X --+ in 1.2 ; therefore , it must converge componentwise by the argument above . Conversely, componentwise convergence implies convergence in the finite dimensional case. For any positive integer k , if is a sequence in 1.2 (k) = and liffin for j = 1 , 2 , . . . , k , then given r > 0 , there exist positive integers such that

Xn

x

(xn) xn U ) xU) ni , n 2 , . . . , n k , n 2: nj I Xn U ) - x U ) 1 < :ik' j = 1 , 2, . . . , k. Hence n 2: max {ni , n 2 , . . . , n k } ==}

==}

Thus, for any k E N ,

n x in 1.2 ( k )

X --+

¢=::}

n xn U ) x U)

lim

=

for j = 1 , 2 , . . . , k .

n x

2. Analysis

42

But COMPONENT-WISE CONVERGENCE # CONVERG ENCE IN f2 The sequence e n = (0, 0, . . . , 0 , 1 , 0 , . . .

)

of standard basis vectors does not converge (it is not Cauchy; see Section = 2 .6) even though each component sequence goes to 0: Indeed, limn e n limn Dnj = ° for n > j. (c) CONVERG ENCE ON PRODUCT SPACES Let i = 1 , 2 , . . . , k, be metric spaces and let the Cartesian product X k carry x X2 X one of the metrics =

U)

(Xi , di ) , Xl

d

.

•

X

.

[Lik= l di (Xi , Yi )P] l ip l :S p < oo

maxi di (X i , Yi ) (2 .2) of Example 1 . 1 .4. By essentially the same argument as in (b) , with d (Xn, x) in place of Il xn - x1 I 2 ' it follows that if a sequence X n ( x n ( 1 ) , Xn (2) , . . . , Xn (k)) E Xl X2 . . . Xk , n E N , converges t o X E X l X2 . . . Xk , then it converges componentwise, i .e . , Xn X i n Xl X2 . . . Xk limn xn U) = X (j) for j = 1 , 2 , . . . , k . We establish the converse by replacing I XnU) - xU) 1 by dj (XnU), xU)) in equation ( 2 1 ) Consequently, for any metric space Y , a map Xl X2 . . . Xk I: Y Y (It (y) , . . . , !k (y)) i s continuous a t a point Y if and only if each Ii i s continuous a t y. Thus, =

x

-)0

.

X

X

X

X

X

X

X

x

===>

.

---t

X

1--+

X

x

we can paste new continuous functions together from old ones . The notion of componentwise convergence is generalized to pointwise convergence in Example 2.2.7(b) . (d) PROJECTIONS O N 1 :S P :S 00 , ARE CONTINUOUS The process of extracting the jth component of the n-tuple or sequence ( x n , as we did in (b) and (c) , generalizes the familiar notion of projecting a vector in R 2 onto the x- or y-axis. For X = 1 :S P :S 00 , and a positive integer j , we define the PROJECTIO N O PERATOR

fp ,

)

fp ,

Pj :

fp

( xn

)

---t

1--+

K,

Xj .

fp

By (b) , each of these projections is continuous. Since ( n ) is a metric subspace of the Pj are continuous on ( n ) as well, for any n E NU {oo } .

fp ,

fp

2 . Analysis

43

(e) CONTINUITY OF A LINEAR MAP Generally, continuity at one point has nothing to do with continuity at other points . To illustrate the intimate terms on which points of continuity and discontinuity may be, consider the map : [0 , 1] ...... [0 , 1] defined by

x

, x( t ) = { O1/qforfort irrational t = plq (lowest terms) , p, q E Z.

.

This function is continuous at each irrational number and discontinuous at each rational. Linear maps : X ...... Y between normed spaces X and Y are a different story. If A is continuous at (or any other point) then it is continuous everywhere-in fact, is uniformly continuous (see Section 2.7) : If < 6 ==::} < f (f, 6 > then < 6 ==::} = < f. 0

A

II x ll

II x - y ll

0

II Ax ll

0) ,

II A (x - y) 1I II Ax - Ay ll

For the sake of future use in Fourier analysis, let us compare some kinds [a , b] . of convergence in

L2

Example 2 . 2 . 7 CONVERGENCE IN

L 2 [a , b]

(a) MEAN CONVERGENCE Convergence x n ...... x in 11 ·/ / in the space L 2 [a , b] of Example 1 . 1 .8 is called CON VERG ENCE IN THE MEA2 N ; we denote this special limit as l . i.m.n xn = x . Convergence in the mean is obviously equivalent to

Xn ...... x (xn)

As we show next , it is possible that in the mean even though f+ (t) at every point t. (b) MEAN =/==} POINTWISE A sequence of real- or complex-valued functions with a common domain T converges P O INTWISE to the function as sequences of numbers for each t E T. The sequence if from 1] below converges to 0 in the mean but not pointwise:

Xn (t) x

x xn(t) ...... x(t) L 2 [0,

xn(t) = { 0,1 - nt , t0 >� lit �n. l in , . Since x n (O) = 1 for every n, it is clear that X n f+ 0 pointwise. On the other hand, 2 � t Jo / xn(t) - 0; dt = 3 n ...... O . For t E [0 , 1] ' the analogue of the projection map Pj on ip ( n ) of Example 2.2.6( d) is the evaluation map i : L 2 [0, 1 ] ...... K, x ....... x(t). Unlike the Pj ,

44

2. Analysis

which are all continuous, i is discontinuous when = 0, since = (0) == 1 . 0 respect to but

t

6 (xn) Xn

11 · 11 2

Xn

-->

0 with

Example 2.2.8 UNIFORM CON VERGENCE

Let X be any vector space of numerical-valued functions on a common domain T. Assume also that each function is bounded , alb eit not necessarily by the same bound. Norm X by the UNIFORM N ORM

II x li oo = sup { I x ( t ) 1 : t E T} . This norm induces the max metric of Exercise 1 . 1- 1 1 . Convergence with is called UNIFORM convergence. The space [a , b] of K respect to valued continuous functions of Example 1 .2.3 is a special case. Uniform convergence is a relatively strong form of convergence, as we show in (a) and (b) below. Some reasons for the imp ortance of uniform convergence in [a, b) are that many properties of the individual terms of a sequence persist into the limit. For example , suppose that for each n E N , the function is defined on the closed interval [a, b] . --> uniformly in the sense that --> 0, then the following hold.

11 · 11 00

C

C

Xn If Xn x •

Il xn - x ll oo

x.

Xn

CONTINUOUS FUN CTIONS If each is continuous, then so is This fact provides a good way to construct continuous functions with special properties such as continuous, nowhere differentiable functions or space-filling curves. The idea is to get the "exotic" function as the uniform limit of functions that almost have the exotic property. To generate a continuous, nowhere differentiable function, for example , create a uniformly convergent sequence of continuous functions are not differentiable at progressively more points. such that the (cf. Section 4.3 for an example.)

(x n )

Xn

•

Xn

TERMWISE INTEGRATION If each is a function in Lda , b] , then so is Moreover, the sequence can be integrated term by term in the sense that --> [A much more general result about termwise integration is the LEBESGUE D O MINATED CON VERG ENCE THEO REM (Stromberg 198 1 , p . 268, Rudin 1 976 , p . 305, or Natanson 196 1 , p. 149 ) : For a sequence of real-valued measurable func tions from L 1 (R) and a measurable function (R) such that (t) a.e . , if --> pointwise a.e ., then -->

x.

J: xn(t) dt J: x(t) dt.

(Xn)

I Xn (t) 1 :S y J�oo x(t) dt.

•

Xn

x

y E Ll J�oo Xn(t) dt

TERMWISE DIFF ERENTIATIO N Pointwise limits of differentiable func tions need not be continuous, let alone differentiable . However, if each converges pointwise to on has a continuous derivative (a, b) , and the derived sequence converges uniformly to then is differentiable and = thus, -->

Xn x

x�, Xn (x�) x' y; x� x'.

y,

x

2. Analysis

(a) UNIFORM CON VERGENCE

==::}

45

POINTWISE CONVERG ENCE For any

P E [a, b], I Xn(P) - x(p) 1 :S sup { I xn(t) - x(t) 1 : t E [a, b] } = II xn - x lloo ,

so uniform convergence implies poin twise convergence. To see that point wise convergence does not imply uniform convergence, consider the se n quence from C[O, 1] ; converges pointwise to the = function = 0 for # 1, = 1 . But cannot converge uniformly to because is discontinuous. As another example of such behavior , consider the sequence E C (R) , = 0 for < n and = 1 for 2: n . Clearly, this sequence converges t o 0 pointwise (just wait .until n exceeds any particular t) but not uniformly, because there are always points at which = 1 no matter how large n is. (b) UNIFORM CONVERGENCE ==::} MEAN CON VERGENCE For E [a, b] , -> 0 ==::}

un (t) t n , E N , u t u(l) u Yn(t) Yn

(un) (un) t

u

Yn (t)

t

t Yn (t) Xn , x L2 II xn x lloo II x n - x II; = lb I Xn (t) - x(t) 1 2 dt :S ( b - a) II x n - x II !:, ---> O. The idea of a clust er point of a sequence (xn) in a metric space is a 0

weaker notion than that of limit of a sequence. Definition 2.2.9 CLU STER POINT

x

(xn)

We say that is a CLUSTER P O INT of in a metric space if every neighborhood of contains for infinitely many values of n; we often phrase this latter condition as is FREQUENTLY in any neighborhood of

x.

x

Xn Xn

0

A sequence has at most one limit , but it may have many cluster poin ts . Example 2.2.10 CLU STER POINTS

(a) A limit of a sequence is a cluster point. (b) The sequence 0 , 1 , 0 , 1 , . . . has two cluster points : 0 and 1 . (c) The rational numbers Q may b e written out as a sequence Any neighborhood of any real number contains infinitely many rational 0 numbers; therefore , every real number is a cluster point of

(xn).

x

(x n).

There is a sequential way to characterize cluster points .

x

2.2. 1 1 CLU STER POINTS AND SUBSEQUEN CES In any metric space is a cluster point if and only su ch that has a subsequ ence

Xnk x. ->

of(xn)

X nk x,

if (xn) x

(x nk )

(xn). B(x, 11k), k E N ,

Proof. If Conversely, -> then obviously is a cluster point of let be a cluster point of Each ball contains for infinitely many values of n. Choose the smallest value of n for which 1/2) contains a point such E 1) and call it nl . Likewise such that nk > nk - l for each 0 that > n l ; choose

x

Xn B (x, n2

(x n ).

Xnk E B(x, 11k)

B(x,

Xn

Xn2 k.

2. Analysis

46

Definition 2.2.12 SERIES

A series E n e X n of vectors from a normed space X is said to CON VERGE to x X if the sequence Sn = � l Xi converges to X . We write ne X n = x. For a two-sided sequence ( X n ) ne Z or bisequence we say Xj = x , m , n Note that we use ne Z X n = x if limm n oo 0 not

E N E Ej= - m '

E N

Ej= - n .

E

,

-+

E N.

Ej= - m

Exercises 2 . 2 1 . CONVERGENCE IMPLIES NORM CON VERGENCE For any sequence ( X n ) in a normed space, show that X n -+ x � I I xn l l -+ I l x l l (x E X ) .

2. Prove the equivalences of 2.2.2.

3 . JOINT CONTINUITY Properties ( a ) and ( b ) below are known as J O INT CONTINUITY of a metric and inner product , respectively.

( a) (b )

In a metric space ( X, d) show that if X n

d(xn , Yn ) -+ d(x , y).

-+

x and Yn -+

In an inner product space X, show that if X n then ( X , Yn) -+ (x ,

n

y).

-+

y, then

x and Yn -+ y,

4. (-0 CONTINU ITY Consider metric spaces (S, d) and (T, d') and x : S -+ T. Show that for any convergent sequence Sn -+ S of points from S, x(sn ) -+ x(s) if and only if for all ( > 0, there exists 0 > ° such that d' (x(s) , x( t )) < ( for all t such that d(s , t) < o.

5 . Prove that the function of Example 2.2.6 ( e ) is continuous on the irrationals.

, k,

6 . PROJECTIONS ARE CONTINUOUS Let (Xi , di) , i = 1 , 2, . be metric spaces and endow X l x X2 X X Xk with the metric d oo of equation ( 2.2 ) . Analogously to what was done for ip ( n ) in Example 2.2.6 ( d ) , we define the projection map Pj ( 1 � j � on X l x X2 X X X k as the map that sends ( X l , X 2 , . . . , X k) into Show that each projection Pj , 1 � j � is continuous. .

.

.

k)

•

.

.

k,

.

•

Xj .

7. MAPS O N PRO DUCTS Let X, Y, and Z be metric spaces. Let X carry the metric doo of equation ( 2.2 ) .

( a)

x

Y

If f : X x Y -+ Z is continuous at (xo , Yo ) , show that the map ( defined on X ) x ...- f (x, is continuous at X o . ( b ) Show that the converse is false .

yo)

2. Analysis

47

( e n)

be an orthonormal sequence in 8. UNIQUE REPRESENTATION Let the inner product space X and let be a sequence of scalars. Show that if = 0 , then each = O. Conclude that for any sequence ( of scalars, = implies that = for all i.

(an) ai L i eN ai e i bn) Li e N ai ei Li eN bi ei

a i bi

9 . CONTINUITY O F LINEAR MAPS

(a) Let X and Y be normed spaces. Use the result of Example 2.2.6( e) to show that a linear map A : X ---+ Y is continu ous if and only if A is bounded on the unit ball U of X , i.e . , E U } is bounded. (b) Let D denote the linear subspace of continuously differen tiable functions of (C Use the result of (a) to show that the differentiation operator x � x , is discontinuous.

{ II A ull : u [0, 1]

10.

[0, 1] , 11 ' 11 (0 ) '

11 . 11 (0) denote the sup-normed space of null sequences of scalars. FOR q > p, fp fq. For alI I :::; p < 00 , show that fp co ; show also that for q > p, fp fq . Show that the identity maps x x of (fp , II · l i p ) into (co , 11 . 11 (0) and of (fp , II· li p ) into (fq , 1I ·l Iq) are continuous linear maps that

Let (co , (a) (b)

C

C

C

�

are not isometries.

11. 12.

NON ZERO O N A NEIGHBORHOOD Let X and Y be normed spaces and let f : X ---+ Y be any map . Show that if f is continuous and f(x) 1= then f is nonzero on an open ball about x .

0

UNIFORM CONVEXITY A normed space X is called UNIFO RMLY CON V EX if for any sequences x and of unit vectors If you think of � as the midp oint of such that II � " ---+ the line segment connecting the points and on the surface of the unit ball U , then the only way for the midpoints to approach the surface of U is for the points and to approach each other.

II xn - Ynll ---+ 0

( n) (Yn) X n Yn

1.

Xn

Yn

(a) Show that any inner product space is uniformly convex. (b) Show that none of the spaces ( n ) , :::; p :::; 00 , n uniformly convex.

fp

13.

1

E N , are

ANTIP ODAL POI NTS This exercise demonstrates some of the power of continuity.

(a) Points on a sphere that lie at opposite ends of a diameter are called ANTIPODAL. Assuming that temperature varies continu ously on the surface of the earth , show that there are antipodal

48

2. Analysis

points on any great circle of the earth at which the temperatures are identical. Temp erature was chosen arbitrarily here; we could have chosen any other continuous variable such as altitude above sea level or magnitude of wind velocity at a given time . This is a special case of the Borsuk- Ulam antipodal point theorem. ( b ) Consider a two-dimensional pancake ( the interior of a plane con vex set ) in the plane and a knife placed at an arbitrary angle . Show that the knife can be translated ( its angle with the x-axis does not change) to a place where it bisects the pancake into regions of equal area.

Hints 3 ( b ) . Subtract and add (xn , y) in I {xn , Yn ) - {x, y) l , and use the triangle and Cauchy-Schwarz inequalities. 4. Assuming the (-6 condition , it is easy to deduce the sequential for mulation. Prove the converse by proving its contrapositive, that if the (-6 condition does not hold, then neither does the sequential condition. The failure of the (-6 condition means that there is some ( > ° such that for all 6 > 0, there exists S6 such that d (S6 , s) < 6 but d' (X(S6 ) , x( s )) � L Therefore , for each n there is Sn for which d (sn , s) < lin but d' (x(sn ) , X(S)) � ( . 5 . Show that the denominators of rational numbers close to a particular irrational become arbitrarily large .

7 ( b ) . Let x (s , t) = stl (s 2 + t 2 ) if (s, t ) =P (0, 0) , and x (O, O) = 0; consider x (s, s) for s =P 0. 9 (a) . Look at the proof of 2.3.3. ( b ) The functions t n , n E the unit ball of D [0, 1 ] .

N , belong to

1 3 . Recall the intermediate value theorem for continuous functions:

For any [a, b) C R, any continuous map x : [a, b) ---+ R and any c such that x (a) ::; c ::; x(b) there is some d, a ::; d ::; b, such that x(d) = c. ( a) Let T = T «(}) denote temperature where () is a central angle of the earth . Note that T (O) T (211') . Extend T periodically, and consider T «(}) - T «(} - 11') . (b ) The area of the pancake lying on one side of the knife varies continuously between ° and the whole area as the knife is translated.

=

2. Analysis

2.3

49

B ounded Sets

The generalization of the notion of a bounded subset of R2 is very imp or tant in normed spaces because of its connection to continuity ( 2.3.3 ) for linear maps. A subset S of a metric space X is called BO UNDED if it is contained in a sufficiently large ball B(x, r) about some x X of radius r > 0. In a normed space, a set 8 is bounded if and only if it is contained in a ball B(O, r) about the origin: If y 8 C B(x, r) , then by the triangle inequality, l I y l l ::; l I y - x ll + l l x l l ::; r + l l x l l ; therefore 8 C B(x , r) ==> S C B(O, r + l l x l l ) ·

E

E

Example 2.3.1 BO UNDED SETS

( a) Finite subsets S Take

= {Xl > . . . , xn } of any metric space (X, d) are bounded.

Then S C B(X 1 ' r) . ( b ) Any subset of a bounded set is bounded. ( c ) The finite union of the bounded subsets 81 , . . . , 8n of a normed space is bounded. For each i 1 , 2, . . . , n, assume that 8i C B(O, rd . Let r max: 1 $ j $n rj . Then U7= 1 8i C U7= 1 B(O, ri ) C B(O , r) . ( d ) The terms of a convergent sequence Xn -> X in any metric space constitute a bounded set by the following argumen t. For some integer N , { X N+ 1 , . . . } C B(x, 1 ) , so the tail {X N + 1 , " ' } of the sequence is bounded ; the first part {Xl , . . . , X N } is bounded because it is finite. The terms of the sequence comprise the union of these two sets. Boundedness does not imply convergence, however: Consider the sequence 0, 1 , 0 , 1 , . . . ( e ) Any trivial metric space ( Example 1 . 1 .6) is bounded. (f) Any metric space (X, d) can be re-metrized in such a way as to be bounded using the metric d' dl ( 1 + d) of Exercise 2.9-7 ( a ) ; moreover, (X, d) and (X, d') have the same convergent sequences . 0

=

=

=

2.3.2 BOUNDED SE Q U ENCE CHARACTERIZATI ON If each denumera ble subset of a subset B of a normed spa ce X is bounded, then B is boun ded.

EN

E

Proof. If B is unbounded, then for every n there exists Xn B such that I Ixn l l � n. Thus, any unbounded set B has an unbounded denumerable subset . 0

Continuous functions need not map bounded sets into bounded sets , since the map X : ( 0 , 1 ) -> R, t � l i t , is continuous. Generally, maps that take bounded sets into bounded sets are called BOUN DED MAPS . As noted above , continuous maps need not be bounded , but continuous lin ear maps are . In fact , bounded is interchangeable with con tinuous for linear maps .

50

2. Analysis

A

2.3.3 CONTINUITY AND BOUNDEDNESS A lin ear map : X -4 Y of the normed space X into the normed space Y (a) is bounded if a n d only if it is bounded on the closed unit ball U of X; (b) is continu ous if and only if it is a bounded m ap. Proof. (a) B is bounded if and only if B is contained in rU for some For any > 0, (rU) (U) , so

r

A

= rA

sup I I A (rU) 1 I �

rk

¢}

sup II A (U) I I

� k (k > 0) .

r > O. (2 .3)

(b) If A is continuous, there exists 6 > 0 such that I l x l l � 6 => II Ax l l � 1 . For any x E U, 1 1 6x l l � 6. Therefore II A (6x) 1 1 � 1 and I I Ax l l � 1/6. Conversely, if sup I I A ( u ) 1 I k for some 0, and { 0 then sup II A ( f U) II � f It follows that A is continuous at 0 and therefore continuous by Example 2.2.6(e) . 0

k>

�

>

Boundedness is one measure of smallness of a set. In the Euclidean spaces it is the only one that is needed. In other metric spaces the stronger notion of total boundedn ess or the notion of compactness is needed, a topic we return to in Section 2.8 .

.e2 ( n ) , n

E N,

Exercises 2 . 3 1 . I n the space C [0, 1] of continuous functions on [0, 1] with sup norm show that the set I I x l l oo = sup { I x ( t ) 1 : t

E [0, In,

{ x E C[O, 1] 11 I x(t ) 1 :

is unbounded.

dt � 1

}

2 . Identify the bounded linear subspaces of a normed space. 3 . SUMS AND PROD UCTS Let (X, 1 1 · 1 1 ) be a normed space.

=

(a) SUMS If A and B are bounded subsets of X, show that A + B {x + Y : x E A, y B} is bounded. (b) CARTESIAN PRODUCTS Let ( Y, I I · I I ' ) be a normed space. Show that l I (x, Y ) l I oo max: ( lI x ll , lI y lI ') defines a norm on the Carte sian product X x Y (cf. Section 2.9) . If A is a bounded subset of X and B a bounded subset of Y, show that x B is a bounded subset of X x Y . (c) PROJECTIONS With X x Y as i n (b) , show that a subset B of X x Y is bounded if and only if each of its projections PI (B) and P2 (B) ( "projection" as in Exercise 2.2-7) is bounded.

E

=

A

2 . Analysis 4. DIAMETER The D IAMETER d space (X, d) is

51

(A) of a nonempty subset A of a metric

= sup {d (x , y) : X , y E A } if the set {d (x, y) : x, y E A} is bounded, d (A) = 00 otherwise. Show that a set is bounded if and only if it has finite diameter and that d(A) = 0 if and only if A is a singleton. d (A)

5 . SOME CONTINU OUS LINEAR MAPS Here are some applications of 2.3.3.

(a) For any fixed vector y from an inner product space X , the map x 1-----+ (x , y) is continuous. (Use the Cauchy-Schwarz inequality.) Thus, the map f : £2 � K, (an ) 1-----+ L ne N an/n i s continuous. (b) Consider the subspace cp of "finite sequences" (all but a finite number of entries are 0) of the inner product space £2 . Show that the restriction of the map f of part (a) to cp is continuous.

6 . BOUNDED NESS AND NULL SEQUENCES

(a) Show that a subset B of a normed space X is bounded if and only if for any sequence (Xn ) from B and any sequence (an ) of scalars that tends to 0, anXn � o . (b) Let X and Y b e normed spaces. Show that a linear map : X � Y is continuous if and only if it maps null sequences Xn � 0 into bounded sequences.

A

7. CONTINUOUS "HOMO GENEO US" IMAGE OF A BOUNDED SET We already know that the continuous linear image of a bounded subset of a normed space is bounded (2.3.3). This result is a little stronger. For normed spaces X and Y, let f : X � Y be continuous and such that for some fixed r > 0, f (ax) a r f (x) for all a > 0 and x E X . Show that f maps bounded sets into bounded sets.

=

Hints 1 . Functions with small integrals can have large peaks.

6 . (a) If B is bounded and {xn } C B , then there exists an r > 0 such that I I xn II ::; r for all n. Thus, I l anxn I I ::; I an I r � O. Conversely, if B is unbounded, it must contain a sequence (Xn) for which I I xn l l � n for each n. (b) If A is continuous, then it maps null sequences into null sequences. Conversely, if A is discontinuous, then there exists

52

2 . Analysis

a sequence ( Yn ) and a number r > 0 such that II Yn l 1 � 1 / n 2 , but I I A Yn l l � dor every n. Thus, Xn = nYn -+ 0, but I I Axn l 1 = n I I A Yn l l � n r , so (Axn ) is unbounded. 7. Show that f (0) = O . Note the result of Exercise 6(a) and consider an -+ 0 and f l an I 1 / r xn .

(

2.4

)

Closure and Closed Sets

Readers of this book surely know the arithmetic of the real numbers addition, subtraction, etc. Right? How exa ctly do you compute V2 + V3? Or 1re? The key word is exa ctly-no approximations. The fact is that most people, even very well educated people, know the same arithmetic as a well educated person of the seventeenth century. There is no need, for practical purposes, to know how to compute things like V2 + .J3 exactly. This is possible because numbers like V2 can be approximated by rational numbers (numbers that can be written as the ratio of integers) to any desired degree of accuracy. How many kinds of functions can you name?-p olynomials, trigono metric functions, logarithms , exponentials, . . . What else? As these are all continuous-infinitely differentiable , in fact-surely there are many others. As with numbers, we can approximate the more complicated functions by the relatively simple ones just mentioned. This section introduces a frame work for understanding the phenomena of approximation in a more general context. Definition 2.4.1 ADHERENCE POINTS , CLO SURE , AND DENSITY

Let S be a subset of the metric space (X, d) . •

A point x is called an ADHERENCE POINT of S if every open ball B ( x , r) , r > 0, meets (has nonempty intersection with) S.

•

The collection cl S, or 5, of all adherence points of S is the CLOSURE of S. If S cl S then S is called CLOSED .

•

=

If cl S

= X then S is DENSE in X .

0

As there are no points adherent to 0 , the null set is closed-how could any set have nonempty intersection with 0? It is also clear that the whole space X is closed. For any subset S of X , cl S is closed, since an open ball B of positive radius about any point x of cl( cl S) must contain a point of cl S; therefore , B must also contain a point of S, and it follows that x cl S.

E In any metric space (X , d) , x E cl S if

2 .4.2 CLO SURE AND SEQU ENCES and only if there is a sequence (xn ) of points of S that converges to x .

2. Analysis

x n x,

53

x contains almost all of the Xn so x E cl S, then B(x, l In) contains a

Proof. If --+ then every ball about every such ball meets S. Conversely, if point of S for every n . 0

Xn

Example 2 .4.3 CLO SURES

X.

C

(a) Let S be a subset of a metric space Then S cl S, since any ball about contains x. An imp ortant consequence of this trivial observation is that to show a set to be closed, it suffices to show that cl S S. (b) The closure of an open interval R is the closed interval b] . and let r be positive. The closure (c) Let X be a normed space, let of an open ball is the closed unit ball any closed ball is a closed set. To see that = cl ) and let let w cl be a sequence from that converges to w. Since < for every n , the continuity of the norm implies that

x

C (a, b).C [a, x E X, B(x, r) C(x, r) ; C(x, r) C (x, r) B(x, r ), E B(x, r B(x, r) II xn - x II r

(xn )

(2 .4) II xn - x II = I l li� Xn - x ii = I I w - x II ::; r , so w E C(x, r); therefore cl B(x, r) C C(x, r ). Conversely, if l l w - x l l = r, then Yn = X + ( 1 - ; ) (w - x) E B(x, r ) for every n , and Yn w , i.e . , W E cl B(x, r). (d) If S is a nonempty subset of an inner product space X , the set S 1. = {x E S x S} is called the O RTHOGO NAL COMPLEMENT or O R THO COMPLEMENT of S. S 1. is pronounced "S perp." It is always closed : If (xn ) is a sequence from S1. and Xn --+ x, then for any E S and n E N , 0 = (xn, ) --+ (x, s ) so ( x, s) = O. 0

li�

--+

:

1.

s

s

The following result is easy to prove and very useful.

X

--+ Y is 2.4.4 If X and Y are norm ed spaces and A : lin ear map , then the null space of A, A - I (0) , is closed.

a

con tinu ous

x E cl A- 1 (0) , then there exists a sequence (xn) from A- I (0) such Xn x. Since A is continuous, Ax = lim n Axn = limn O = O. 0

Proof. If --+ that

Example 2.4.5 DENSE SETS

(a) THE RATIO NALS The closure of the set of rational numbers Q in R is R by the following argument. As there is a rational number between any two reals, every open interval - l I n , + l In) about a real number contains a rational number Clearly, --+ A similar argument shows that Q x Q is dense in R2 . (b) POLYNO MIALS IN By the Stone- Weierstrass theorem [see Section 4 . 1 7] the closure of the set of polynomials on 0 1 ] is the space 1] of K-valued continuous functions on 1] when it carries the uniform norm 1 1 1 1 00 of Example

(x xn. C [a, b]

C[O,

1.2.3.

x Xn x. [0,

x

[,

2. Analysis

54

(c) DENSE SUBSPACES O F Lp [- 1I" , 1I"J , P � 1 [Natanson 1 96 1 , pp. 1 72, 200] The linear spans of the following sets are dense in Lp [-1I", 11"] . •

• •

{ eint

E Z} Ani kelement of the linear span of { e int : n E Z} is of the form 2:::: � = - n a k e t and is called a TRIG ONOMETRIC P O LYNOMIAL. SINES AND COSINES sin nt , cos (n - 1 ) t , n E N . :n

CONTINUOUS FUNCTIONS C[-1I", 1I"] .

Definition 2.4.6 STEP FUN CTIO NS O N R Consider real numbers

Co < C l < . . . < Cn ·

A fun ction x : R --+ R that is constant on each of the int ervals ( Ci ' Ci + d , i = 0, - 1 , is defined arbitrarily o n the Ci , a n d vanishes for t < Co and t > Cn is called a STEP ( or SIMPLE ) FUNCTION ON R. (d) STEP FUN CTIONS DENSE IN Lp SPACES , 1 :S p < 00 The collection of step functions defined on R is dense in Lp (R) , 1 :S p < 00 [Stromberg . . . , n

1981 , p. 342] . The fact that the step functions on [a , b] are dense in Lp [a , b] follows as an immediate corollary. More generally, suppose that T is a space with a positive measure p, defined on a u-algebra of subsets of T. Lp (T, p,) , 1 :S p denotes the linear space of K-valued pth power summable functions x on T, i.e., p,-measurable functions x : T --+ K such I normed by I l x l l p = [ IT I x l P dp, ] / P . (There is a brief that IT I x l P dp, discussion of Lp (T, p,) at the end of Section 1 .6. As noted in Section 1 .6 , if T = R or [a , b] and p, is Lebesgue measure, we write just Lp [a , b] or Lp (R) , respectively.) Let A I " ' " An be pairwise disjoint p,-measurable subsets of T such that U7= 1 A; = T. In this context , a STEP (or SIMPLE ) FUN CTION O N T is a linear combination x = 2:::: 7= 1 a lA " a K , = 1 , 2 , . . . , n , of . For 1 characteristic functions l A , :S p the step functions x on T such that p, (coz x) are dense in Lp (T, p,) [Rudin 1 974, p. 70] . (e) CONTINUOUS FUNCTIONS WITH COMPACT SUPPORT Cc (R) A func tion x : R --+ R that vanishes outside some closed interval is said to be of COMPACT SUPPORT . The linear space of continuous functions on R with compact support is denoted by Cc (R) . For alI I :S p Cc (R) C Lp (R) , and Cc (R) is dense in Lp (R) [Stromberg 198 1 , p . 342.] . 0

< 00, < 00,

< 00

iE

i < 00,

i

< 00,

Density results such as the density of Cc (R) in Lp (R) are very useful. As one illustration , consider the proof of 2.8.9 on "continuity in the mean." As is evident from Example 2.4.5, a linear subspace of a normed space need not be closed-it could be dense. The closure of a linear subspace M is again a linear subspace, however: If (xn ) and ( Yn ) are sequences from M that converge , respectively, to X , Y cl M , then for any scalar + Yn --+ ax + Y by 2.2.3; thus, cl M is closed with respect to the formation of linear combinations. A type of subspace that is always closed the orthogonal complement-is discussed in Section 3.2.

a, aXn

E

2. Analysis

55

Example 2.4.5 is about approximating exotic functions by simpler ones . If a metric space X has a countable (i.e . , denumerable or finite) dense sub set , we say that X is SEPARABLE. Thus R is separable because Q C Rj so are C[-1I", 1I"] and L 2 [0, 211"] because they contain, respectively, the de numerable dense subspaces of polynomials and trigonometric polynomials with rational coefficients. Example 2.4.7 SEPARABLE SPACES

( a) The space l2 of square-summable sequences ( 1 . 1 .7) is separable. Why? Consider the linear subspace cp of all "finite" sequences, sequences that are o except for a finite number of entries. Let x E l2 . For each n E N , let Xn be the sequence whose first n entries are the same as x , 0 thereafter. Clearly, X n -+ x, and cp is seen to be dense in l2 (real or complex) . If we restrict the entries in cp to be rational or Gaussian rational (i.e., of the form a + bi where a and b are rational) , it is easy to see that this countable subset is dense in l2 as well. (b) NON SEPARABLE Separability is a measure of smallness of a space. If a space is big enough to have an uncountable number of pairwise disjoint open balls, then it is not separable , because each such ball would have to contain at least one distinct point of any dense subset. Consider the space loo of all bounded sequences with sup norm 1 1 · 1 1 00 (cf. Exercise 1 . 1l 1 (b) ) . The subset X of sequences that have only O 's and l 's as entries is an uncountable set , since it is in 1-1 correspondence with binary decimals , and binary decimals are in 1- 1 correspondence with the real numbers between o and 1. Clearly, distinct elements of are of distance 1 apart. The open spheres B(x, 1 /2) about the points x of are therefore mutually disj oint, and it follows that X is not separable . What does this say about the larger space loo ? Though we do not prove it, any subspace of a separable metric space must be separable . Therefore , loo is not separable . 0

X

X

Exercises 2 . 4

X

1 . Let S b e a subset of a metric space and let x E S. I s it true that the closure of the set difference S - { x } is S?

2. Let X and Y be metric spaces and let I, 9 : X -+ Y be continuous functions. Show that if I = 9 on D then I = 9 on cl D .

e X,

3. DENSE SUBSETS O F DENSE SUBSETS Show that if A is dense in B, then A is dense in cl B. In particular if A is dense in B and B is dense in the metric space then A is dense in Application: The subspace Cc (R) of continuous functions on R with compact support is dense in Lp (R) , 1 :::; p 00 (Example 2.4.5) . Therefore , to show

X <

X.

56

2. Analysis

that step functions on R are dense in Lp (R) , it suffices to show that the step functions are dense in Cc (R) . 4. In any metric space X, show that singletons, 0, and X are closed sets. 5 . CLO SURE MONOTONE If A

C

B then show that cl A

C

cl B .

6 . CLO SURE O F PRODUCTS If X and Y are metric spaces and X x Y carries any of the metrics 1 � p � 00 , of Example 1 . 1 .4, show that cl (A x B) = cl X x cl Y for any subsets A and B of X and Y , respectively. Thus, if A and B are closed, then so is A x B .

dp,

7. TRANSLATES A N D MULTIPLES O F CLO SED SETS For any subset E of a normed space X show that

E

E

E

(a) for any x E X , if is closed, then x + is closed . Indeed , is closed if and only if x + is closed for all x E X . (b) cl aE = a cl being closed implies for any scalar Thus, that is closed. (c) For subsets E and F of X, + F = {x + y : x E E, y E F}. Make up two closed subsets E and F of R such that E + F is not closed.

aE

E

E,

a.

E

E

8 . SOME CLO SED SETS Here are some applications of 2 .4.4. Direct arguments are cumbersome for these sets.

= o } is closed . 12 : (a) Show that the set (b) Let I{J be the subspace of finite sequences of 12 . Show that the set M = { (an ) E cp : = o } is a closed subspace of

{ (an) E

I{J .

En e N an/n En eN an/n

9. Let Co denote the subspace of loo (Exercise 1 .1- 1 1(b) ) of all N ULL SEQUENCES , i.e . , sequences that converge to O.

(a) Show that Co is closed. (You can use 2 .4.4 to do this.) (b) Show that lp , 1 ::; p < 00 , is not a closed subspace of co . Note that lp carries the sup norm 1 1 · 1 1 00 in this context, not the p norm. 1 0 . Show that a separable metric space has cardinal number at most c, the cardinal number of R. 1 1 . (c, I I · l I oo ) SEPARABLE Show that the space (c, I I · l I oo ) of convergent sequences is separable .

12. lp SEPARABLE For 1 � p <

00

show that lp is separable.

2 . Analysis

57

1 3 . Prove that a countable union of separable subsets of a metric space is itself a separable space. 14. CONTINU OUS IMAGE OF SEPARABLE SPACE Let f : X --+ Y be a continuous map of the separable metric space X onto the metric space Y. Prove that Y is separable. 15 . Prove that in any metric space, the closure of a separable subspace is separable . 1 6 . DIAMETER AND DISTANCE Show that the closure of a bounded set is bounded. Recall that the DIAMETER d (S) of a nonempty subset S of a metric space (X, d) is

d (S) = sup {d(x , y) : x, Y E S} (possibly d (S) = (0 ) and show ( a) that d( S) = d ( cl S) and (b ) that x E cl S if and only if the DISTANCE FRO M x TO S, d(x , S) = inf {d(x , s) : s E S} , is O. Note that this notion of distance from a point x to a set S generalizes the established convention for distance between a point and a line in R3: The DISTANCE from a point to a line or plane in R3 is the shortest distance , the perpendicular distance. ( c ) For any x , y E X show that I d (x , S) - d (y, S) I � d (x, y) .

Hints 1 . Suppose S = {x} . 3 . If bn E B --+ x E cl B , choose an E A such that d (a n , bn ) all n E N . 7 ( c ) . Let E = {n : n E

< lin for

N , n 2: 2} and F = { - n - lin : n E N , n 2: 2 } .

8 . Use Exercise 2.3-5.

10. Use the fact that N�o = c and count how many sequences-convergent or not-you can make from a denumerable set. 1 1 . Consider sequences with finitely many rational coordinates and the rest equal to some fixed rational. 16 ( c ) . Show that

d (x , S)

<

inf {d (x, y) + d (y, s) : S E S} d (x, y) + inf {d (y, s) : s E S} d (x, y) + d (y, S) .

58

2. Analysis

2.5

Open Sets

A subset G of a metric space (X, d) is O PEN if its complement CG is closed. 2 . 5 . 1 OPEN SETS G is open if and only if for all x ball B(x, r) , r > 0, such that B(x, r) C G.

P roof. Note that

E G, there is an open

CG is closed ¢} cl CG C CG.

The latter condition is equivalent to the existence, for each x r > 0 such that

E G, of some

B(x, r) n CG = 0,

which is equivalent to

B(x, r)

C

G. 0

Example 2.5.2 EXAMPLES O F OPEN SETS

In any metric space (X, d) : (a) OPEN BALLS ARE OPEN Any open ball B(x , r) , r > 0 , is an open set . For y B(x , r) , r ' = r - d(x , y) is positive. Therefore , for any z B (y, r ' ) ,

E

d(z , x) ::; d(z , y) + d (y, x)

< r - d(x , y) + d(y, x) = r,

E

and the result follows from 2.5 . 1 . The open balls B (x, r) , r > 0 , are called the BASIC O PEN SETS of X . (b) X and 0 are open. (c) UNIONS AND INTERSECTIONS Any union and any finite intersection of open sets is open. The statement about unions is easy. As for finite intersections, if G 1 ,G 2 , , Gn are open and x n 7= 1 Gi then there exist positive numbers rl ,r 2 , . . . , rn such that B(x, ri ) C Gi for each i. Let r = mini ri o Then B(x, r) C n 7= 1 Gi . Infinite intersections of open sets need not be open: n : l ( - lln, lin) = {O} , which is not open since it is too small to contain an open ball in R. (d) INTERI OR For any G C X, the INTERIOR OF G, the set •

int G = {x

•

E

•

EG

:

B(x, r)

C

G, for some r > O}

is open. A point of int G is called an INTERIOR POINT of G . I f x E int G, then B(x, r) C G for some r > O. Therefore , by ( a) , each point of B(x, r) is also in int G; therefore , int G is an open set. Indeed, G is open if and only if G = int G. If G is open , then for each x G, there exists rx > 0 such that B(x , rx ) C G. Therefore G= (2 . 5) B(x , rx ) .

E

U

xEG

2. Analysis

59

Thus, the open balls are the building blocks of any open set , hence the name BASIC open sets. In a normed space the representation of equation (2.5) is even simpler , since B (x, =x+ 1). 0

rx )

rx B(O,

Definition 2.5.3 EXTERIOR

The EXTERIOR of a subset A of a metric space, ext A , is int CA. 0 Nonempty sets can have empty interior : Consider the rationals Q C R, for example . Sets A that are so sparsely distributed that int(cl A) = 0 are called RARE or NOWHERE DENSE. The Hilbert cube of Exercise 5 is an example of a rare set. The totality T of open subsets of a metric space Xis called the TOPOLO GY on X. A reason for interest in the topology is that convergence can be characterized in terms of open sets without reference to the metric : For any open set G to which x belongs,

x n --+ x if and only if X n

E G eventually.

Exercises 2 . 5 1 . CLO SED V S . OPEN The notions closed and open are dual concepts, not opposites : Failure to be open does not imply closed, nor vice versa. Sets can be both (X and 0, for example) or neither ([0 , 1 ) C R, for example) . Those that are both open and closed are called CLOPEN . Show that in any ultrametric space (Exercises 1 . 1-13 and 2.2-4) every ball, closed or open, is clopen. 2. PRO DU CTS O F OPEN SETS For any two open subsets U and V of the metric spaces X and Y , respectively, show that U x V is an open subset of X X Y when it carries the max metric doc of Example 1 . 1 .4 (or any other of the metrics dp , 1 � p < 00 , of Example 1 . 1 .4, for that mat ter ) . 3 . CONTINUITY For metric spaces X and Y , show that I : X --+ Y is continuous if and only if

(

(

a) I maps closures into closures: For any subset A of X , I cl A) cl / (A) . (b) I takes open sets back into open sets: For any open subset G Y, 1 - 1 (G) is open.

B.

C

C

4. INTERIOR If A C B, then show that int A C int In a normed space show that for any A C int cA = c int A for any scalar c =1= 0 .

X

X,

60

2. Analysis

5. HILBERT CUBE The subset C = { (an ) £2 : I an :::; of the space £2 of Example is called the HILB ERT CUBE. (a) Show that C is closed. (b) Show that C has empty interior , i . e . , C is a rare set .

E

1.1.7

I 1 In}

Hints 5 . (a) Show that if

x rI. C, then x rI. cl C.

(b) Show that any ball B(x , r) about any point x E C also meets the complement , i.e., that there exists a sequence from B(x, r) such that > for some in particular, choose > 2/r.

I b k l 11k

2.6

(bn) k

k;

Completeness

(X, d),

In a metric space if Xn - x , then the Xn are eventually close to are close to each Since points (xn and xm ) close to the same point x. other by the triangle inequality

(x)

it follows that for any

i

> 0, d(xn, x m )

<

i

for sufficiently large n and

m.

(2.6)

When a sequence has this latter property (2.6) , it is called a CAUCHY SEQUENCE. As we have just seen, convergent sequences are Cauchy. For the sequence (ed = E N, of standard basis vectors in £2 , =P j � - ej 1 1 2 = .)2, so it is not Cauchy; hence it is not convergent . Does Cauchy imply- convergent? Generally, no. Consider the rational numbers Q as a metric space with its usual metric . The sequence of ratio nals

(0, 0, . . . , 0, 1, 0, . . . ), i i lI ei

1, 1.4, 1.41, 1.414, . . . ,

which converges to v'2 is a Cauchy sequence (it is a convergent sequence in R) but does not converge to a point of Q . Hence it is a nonconvergent Cauchy sequence in Q . Consider the open interval as a metric space with its usual distance . The sequence ( l in) is a Cauchy sequence of points of but does not converge to a point of Spaces in which Cauchy sequences may not converge are therefore "incomplete" in the sense that they lack these limits.

(0, 1)

(0, 1)

(0, 1) .

2. Analysis

61

Definition 2.6.1 COMP LETENESS

A metric space (X, d) is called COMPLETE if every Cauchy sequence con verges. Complete normed and inner product spaces are called, respectively, BANACH and HILBERT spaces. 0 Complete metric spaces are of special imp ortance in analysis. For one thing, in such spaces there is a criterion for convergence other than find ing the limit-something for which there is no general procedure. But the reasons are deeper than that. Let us mention one. THE BAIRE C ATEGORY THEOREM : If a complete metric space is the denumerable union of closed sets, then at least one of them contains a nonempty open ball.

This is a profound result . One of its consequences is an elegant demon stration of the existence of a profusion of continuous, nowhere differen tiable functions (see , for example, Bachman and Narici 1966, pp. 76-82) . The Baire category theorem figures prominently in many other proofs of existence. But back to some examples. The real numbers R are complete. This is a deep fact that requires knowing how R is constructed. Roughly, one starts with the integers Z , then adjoins reciprocals to get the rationals Q . Then add in limits for nonconvergent Cauchy sequences of rationals- 1 , 1 .4, 1 .41 , 1 .414, . . . ---+ �, for example . The result is R. The process of providing the limits is called COMPLETI O N ; it is a general one that applies to any metric space X . The idea is to create a space Y in which the holes have been filled, to provide limits for the non convergent Cauchy sequences but to do it in a "minimal way." The "minimal way" is achieved by requiring that X be dense in Y . A development of the reals c an b e found in many places ; one is Stromberg 1 981 . The general process of completing a metric space can be found in many places as well, in particular Bachman and Narici 1966 , pp. 53-58. The completion of a normed or inner product space is a Banach space or a Hilb ert space, respectively ( ibid. pp. 1 18-121 and pp. 141 - 142) . It follows from Example 2.4.5 that the completion of the space of polynomials on is The completion b] with respect to the uniform norm of with respect to is L2 likewise , the completion of the trigonometric polynomials and the step functions on with respect to is L2

[a,

C [a, b]

/1·/1 2

[a, b].

11 - 11 2

/ 1 · /1 C [a, b]. [a, b]; 00 [a, b]

Example 2.6.2 EXAMPLES ON COMPLETE SPACES

(a) COMP LETENESS O F ip ( n ) , n

p � 00 .

E NU {oo} , Lp (R) , and Lp [a, b] , 1 �

These spaces (Example 1 .2.2) are Banach spaces. Here i s an instance in which the Lebesgue integral is superior to the Riemann integral. The space Rp of Riemann-integrable functions x such that I x(t) I P dt < 00 with the

f:

62

2. Analysis

Lp [a, b)

p-norm is not complete , but is. More generally, if T is a set with (T, J.l) a positive measure J.l defined on a u-algebra of subsets of T and ) is as in Section (T, J.l is a Banach space for all � p � 00. The proofs of completeness of (T, J.l ) are widely available ; see, for example , Bachman and Narici p. Naylor and Sell 1 982, p. 589; or Dunford and Schwartz 1 958, p. The completeness of follows as a special case. (b) HILBERT S PACES l2 , n are L 2 (R) , and Hilbert spaces. We prove the completeness of l2 = ( R 2 , 1 1 · 1 1 2 ) . Let (xn) = ( ( n , n ) ) be a Cauchy sequence in R2 . Since (xn ) is Cauchy, so is ( n ) , since

1.6, Lp Lp 1966; 115, 146. lp (n) E NU {oo},

a

a b

Lp

1

L 2 [a, b]

(2)

Similarly, (bn ) is a Cauchy sequence of real numbers as well. Therefore , there are real numbers and b such that

a

an � a and bn

2.2.6(b))

�

b,

from which it follows (Example that Xn � x . L 2 (R3n ) is a Hilb ert space, too; this space constitutes the "quantum-mechanical states" of a system of particles. (c) CONTINU OUS FUNCTIONS (C b) , 1 1 . 1 1 (0 ) The space of con tinuous functions on the closed interval with sup norm 1 1 · 1 1 00 (Example is a Banach space. A Cauchy sequence (Xn ) in this space is a uni form Cauchy sequence of continuous functions (for any f > 0 there exists N such that � N � I I xn (t) - X m (t) 1 1 < ( for all It therefore converges uniformly to a continuous function. The same argument shows that the space (R ) of bounded continuous functions is complete. , viewed (d) 1 1 · 1 1 2 ) INCOMPLETE The inner product space as a subspace of L 2 with

n

[a,

1.2.3)

C [a, b)

[a, b]

m, n

(C [a, b] ,

t E [a, b)).

C

C [-1, 1]

[-1, 1]

{n

is not a Hilbert space. Consider the following Cauchy sequence of ramp functions: 0, � t � 0, t, 0 < t � xn ( t ) =
n, m � N

-1 1, l/n �

l/n, 1.

1/N, since that is the area of the rectangle of base 1/ N and height 1. It is easy to see that the sequence converges pointwise to the discontinuous function x equal to 0 on [-1 , 0] and 1 on ( 0 , 1] ; furthermore, Xn x with respect to 1 1 · 1 1 2 . The 1 1 · 1 1 2 limit x is not uniquely determined on [-1, 1]; it may be II Xn - xm l l 2

�

2. Analysis

63

redefined arbitrarily on a subset of [- 1 , 1] of measure O. As there is no way to redefine x on a set of measure 0 that will yield a continuous function , [a , b] , 1 1 · 1 1 2 ) is not complete . 0

(C

Some elementary uses of the Cauchy criterion for convergence are illus trated in the following example.

Example 2.6.3 USIN G THE CAUCHY CRITERION

(a) ABSOLUTELY CON VERGENT ==> CON VERGENT Let (xn ) be a se quence of vectors in a Banach space X that is ABS OLU TELY CON VERG ENT (also known as ABSOLUTELY SUMMABLE ) in the sense that E n e N I I xn l l converges. Does E neN Xn converge? With Sn = E�= l X i , then , for n > m ,

II sn - Sm l l

=

n

L

i =m + l

Xi

Since E n e N I l xn l l converges, its partial sums tn = E7= 1 1 1 x i l l form a Cauchy sequence: For { > 0 there exists an integer N such that for n , m >

N,

II tn - t m l l = Since

n

n

L

i =m+ l

II x i i I

<

L

n

by the triangle inequality, the partial sums (sn ) form a Cauchy sequence; therefore the original sequence E neN X n converges. Thus, in any Banach space, absolute convergence implies convergence. (b) E n e N lin = 00. To show that E ne N lin does not converge, we show that the sequence of its partial sums is not Cauchy. For any n , 1 1 l I n > = -. +...+ I S 2 n - sn l = n + 1 + -n+2 2 2n - 2n --

-

-

Therefore , (sn ) does not converge. We can say a little more in this case. Since (s n ) is increasing, it must be unbounded (a bounded increasing se quence of real numbers must converge) ; thus E neN l i n = 00 . (c) UNIFO RMLY CON VERG ENT ¢} UNIFO RMLY CAU CHY L e t (Xn ) be a sequence of functions mapping a set T into a Banach space (X, 1 1 · 1 1 ) . Then X n -+ X uniformly if and only if it is a uniform Cauchy sequence, i .e . , for all { > 0 there exists N such that m , n � N => II xn (t) - X m (t) I I { for all t E T. Proof. If Xn -+ X uniformly, then given f > 0, there exists N such that I I xn - x (t) I I {/2 for all t T and for all n � N . Hence , for n , m � N ,

<

(t)

<

E

64

2. Analysis

I I Xn (t) - X m (t) 1 I t E T.

:S

I I xn (t) - x (t) 1 I + II x (t) - X m (t) 1 I < < /2 + < /2 for all

Conversely, if (x n ) is uniformly Cauchy and ( > 0, choose N such that ::} I I xn (t) - Xm (t) 1 I < ( for all t E T. At each t, (xn (t)) is a Cauchy sequence in and therefore converges to a limit we shall call x (t) . Now, by continuity of the norm, let m -+ 00 in I I xn (t ) - X m (t) 1 I < f . 0 (d) T HE WEIERSTRASS M-TEST Let (xn) be a sequence of functions mapping a set T into a Banach space X. If there are constants Mn , n E N , such that I I xn (t) 1 I :S Mn for all t and L: n E N Mn < 00, then L: n E N Xn converges absolutely and uniformly. Proof. The absolute convergence follows by a comparison of the partial sums of L: n E N II xn l l and L:n E N Mn . Since L: n E N Mn converges, for f > ° and sufficiently large n and all p E N , I Mn + . . . + Mn +p l < f . By the triangle inequality, m, n � N

X

I I xn (t ) + . . . + xn +p (t) 1 I :S I I xn (t) I I + ' + lI xn +p (t) 1 I :S Mn + . . + Mn+p < ( for all t E T. In other words, the sequence of partial sums of L: n E N Xn is uniformly Cauchy, and the result follows from (c) . 0 Next , we consider an alternative characterization of completeness . Re call that the DIAMETER d(A) of a subset A of a metric space is d(A) = sup E A} or d (A) = 00 . Let E cl A and choose sequences (an) and (bn) from A that converge to and respectively. By the joint continuity of the metric (Exercise 2.2-3(a)) it follows that

{d(a, b) : a, b

a

a, b

b,

d ( b) = limn d ( an , bn) :S d (A) . Therefore, d (cl A) :S d (A) . Since A cl A, the reverse inequality holds well, and it follows that d (A ) = d (cl A) . a,

C

as

2.6.4 NESTED SEQU ENCES A metric space (X, d) is complete if and only if every "n ested sequ e n ce " Fl :J F2 :J . . . of nonempty clos ed sets wh ose diameters d (Fn) go to 0 has nonempty intersection. Proof. Let (Fn ) be a nested sequence of nonempty closed subsets of the complete space such that the diameters (Fn ) -+ 0 . For each n E N , choose Xn E Fn . Since d(Fn) -+ 0 , it follows that (xn) is Cauchy. Let x = limn xn . Since {Xn,Xn + l " " } C Fn for every n , it follows that x E cl Fn = Fn for every n , i.e . , that n n E N Fn :J { x } . Conversely, suppose that has the "nested sequence property" of the theorem and that (xn ) i s Cauchy. For each n let Fn = cl {Xn,Xn + l , . . . } . Since (Xn ) i s Cauchy, d(Fn) -+ 0. Therefore , there exists x E n n E N Fn . Since (x, xn) :S (Fn) it follows that Xn -+ x . 0

X

d

X

d

d

2 . Analysis

65

Exercises 2 . 6 1 . ORTHO N O RMAL SEQUENCES NONCO N V ERG ENT Show that no or thonormal sequence (i.e . , m ,# n => Xn 1.. X m and II xn l l = 1 for each n N ) in an inner product space is Cauchy.

E

2. SUMS AND SCALAR MULTIPLES If (an ) is a Cauchy sequence of scalars and (xn ) and ( Yn ) are Cauchy sequences of vectors from a normed space, show that (anxn ) and (xn + Yn ) are Cauchy. 3. ONE-DIMENSIONAL SUBSPACES COMP LETE Show that a one-dimen sional subspace of any normed space is complete , indeed that it is of the form Kx for some vector x and is therefore linearly isometric to

4.

K.

INFINITE- DIMENSIONAL TRIANGLE INE QUALITY For any summable sequence (xn) in a normed space, show that II L neN Xn I ::; L n e N I I xn l l · (Allow for the possibility L neN I I xn ll = 00 .)

5 . If (x n ) is an orthonormal sequence in an inner product space X and (a n ) E £ 2 , show that Sn = L �= l a i X i is a Cauchy sequence. Thus, if X is a Hilbert space, then any such series LieN a i X i converges. 6 . A COMPLETE SPACE Show that the linear span

M = {a cos t + b sin t a, b E C } :

o f {cos t , sin t } i n the Hilbert space L 2 [O, 211"] is complete. 7. B [a, b] COMPLETE Let B [a , b] denote the space of all bounded scalar valued functions on the closed interval [a , b] (or any set T) with point wise operations and sup norm (as in Example 1 .2.3) . Show that B [a, b] is complete .

11 · 11 00

8.

(C

11·11 2 )

C

INCOMP LETE Let (R) denote the subspace of contin (R) , uous square-summable functions of the Hilbert space L 2 (R) . Show that (R) is not a Hilbert space by showing that for any a 0,

C

>

I t I ::; a, It I > a, i s a non convergent Cauchy sequence; indeed, Xn -+ 1 [ - 4.41 , the de cidedly discontinuous characteristic function of [ - a, a] . 9 . CAUCHY SE QUENCES IN ULTRAMETRIC SPACES In an ultrametric space (X, d) (Exercises 1 . 1-13 and 2 . 1 -4) the Cauchy criterion sim plifies considerably : Show that a sequence (xn) from X is Cauchy

66

2. Analysis

d (X n, Xn + l)

if and only if -+ O . For contrast , show that there is a sequence of real numbers such that x -+ 0 that does not converge.

(X n )

IXn+ l - n l

1 0 . BAIRE NULL SPACE COMPLETE Show that the ultrametric Baire null space of Exercise 1 . 1- 1 3 is complete. 1 1 . CAU CHY IMPLIES BOUNDED Show that a Cauchy sequence in any metric space is bounded. 12. 13.

X

COMPLETE IF AND ONLY IF U COMP LETE Show that a normed space is complete if and only if its unit ball U is complete .

X

fp COMP LETE

For 1

::; p ::; 00 , show that fp is a Banach space.

14. CON VERGENT SUBSEQ UEN CE In any metric space, if a subsequence of a Cauchy sequence converges, then the sequence must converge to the same limit . 1 5 . WHEN CLO SED IMPLIES COMPLETE Show that a closed subset of a complete space is complete. What about the converse? i .e . , does complete imply closed?

Fn) n n EN Fn =

of closed subsets of R 1 6 . Show that there exists a nested sequence ( 0 and a nested whose diameters do not go to 0 for which sequence of nonclosed sets whose diameters go to zero with empty intersection.

Hints 1 1 . Let f = 1 and consider the sequence in two parts, an early part and a tail.

Xn)

p

1 3 . For < 00, if ( is Cauchy, show that for any fixed j , the sequence of jth components (j) ) is Cauchy. For 00, use the fact that Cauchy sequences are bounded.

(X n

p=

14. Closeness is a transitive relation .

2.7

Uniform Continuity

Let and T, be metric spaces. A function x : S -+ T is U NIFO RMLY S, CONTINUOUS on S) if for any c > 0 there is 6 > 0 such that for all s ,

(S, d)

(

( d')

d(s, s ' ) < 6

s' E

==::}

d' ( x ( s ) , x ( s ' )) <

€.

2. Analysis

67

Example 2 . 7 . 1 EXAMPLES ON UNIFORM CONTINUITY

(a) As is well known (any advanced calculus book) , a continuous real valued function defined on a closed bounded interval is uniformly con tinuous. The results of (b) show how fragile this result is, that it is the conj unction of continuity with the closedness of the interval that yields the uniform continuity. (b) The map t � lIt of (0, 1] into R , for example , is easily shown not to be uniformly continuous; neither is the map t 1---7 t 2 of R into R. Note that in the first case the domain is not closed; in the second it is not bounded . (c) LINEAR MAPS Example 2.2.6(e) shows that linear maps between normed spaces are continuous if and only if they are uniformly continuous. (d) BO UNDED DERIVATIVE If f : ( a, -+ R is differentiable and has a bounded derivative-there exists M such that for all x ( a, I f' (x) 1 � M-then f is uniformly continuous. By the mean value theorem, for any x , y E ( a , b) , there exists c (x, y) such that

b)

E b),

E

I f (x) - f ( Y) I = I f' ( c) l l x - Y I � M l x - y l . 0

If x : S -+ T is uniformly continuous and (sn ) is a Cauchy sequence in S, then so is (x(sn ) ) , as follows directly from the definition. This has an imp ortant consequence . If is uniformly continuous and T is complete , then x may be extended in a unique way to a uniformly continuous map x defined on c l D by the following process known as EXTENSION BY CONTIN UITY : For S cl D, let (sn ) be a sequence from D that converges to s. Since (sn ) must be Cauchy, so is (x(sn )) , and here is where it is crucially important that x be uniformly continuous. Since T is complete , we may define the EXTENSION x of x BY CONTINU ITY as x (S ) = lim n x(sn ) .

E

There is one problem. Suppose (s� ) is a different sequence of points from D that converges to s. What if

lim n x(s� ) i lim n x(sn )? If this could happen, x would not be well-defined . Fortunately, it cannot occur . Observe that since (sn ) and (s� ) converge to S, d(sn , s� ) -+ 0 by the triangle inequality. Since x is uniformly continuous, this implies that

d' (x(sn ) , x(s� ) ) -+ O. Since

d' (x(s� ) , x(s)) � d' (x(s� ) , x(sn)) + d' (X(sn ) , x ( S ) ) , it follows that limn x(sn ) = limn x(s� ) .

68

2 . Analysis

As to the uniform continuity of x , given f > 0 , the uniform continuity of

x implies the existence of a 6 > 0 such that for all s, s' E D , d(s, s') < 6 d' (x(s), x(s')) < f . Now consider any two points s, s' in cl D such that d(s, s') < 6/3 and sequences (Sn) and (s � ) from D such that S n s and s� s'. By the ==>

triangle inequality, for sufficiently large n ,

-+

-+

d(sn , s�) � d(sn, s) + d (s, s' ) + d(s', s�) < 6.

Since metrics are jointly continuous (Exercise 2.2-3(a) ) , it follows that

d' ( x ( s), x( s' )) = limn d' (x( sn), x( s� )) � f .

The uniqueness of x follows from the fact that if two continuous functions agree on a set , then they agree on its closure (Exercise 2 .4-2) . An imp ortant application is the following. If : D -+ Y is a continuous linear map defined on a dense linear subspace of a normed space into a Banach space Y , then since A is uniformly continuous, A has a continuous extension A to Indeed,the extension is lin ear, since for E D such that -+ and -+ and E K,

A

X. .A x y, a Xn Yn lim .A (ax + y) n A (axn + Yn) = a.Ax + .Ay.

X xn , Yn

=

Exercises 2 . 7 When Cartesian products of metric spaces are considered in the exercises below , they may carry any of the metrics of equation (2.2) of Section 2.2 (cf. Example 1 . 1 .4) , since any of the produce the same convergent sequences (2.9.3 ) . It is usually simplest to endow the product with the max metric

dp dp

doc .

1.

RESTRI CTIONS AND COMP OSITES Show that restrictions and com posites of uniformly continuous maps are uniformly continuous.

2. MAPS INTO PRO DUCTS Let X, Y, and Z be metric spaces. Show that a map I----+of X into Y x Z is (a) continuous at x if and only if f and 9 are continuous at x. (b) uniformly continuous if and only if f and 9 are uniformly contin uous.

x

(f (x) , g (x))

3 . METRICS UNIFO RMLY CONTINUOUS For any metric space (X, show that is a uniformly continuous map of X x X into R.

d

d)

2. Analysis

69

4. PROJECTIONS UNIFORMLY CONTINU OUS Show that the proj ection operators of Example 2.2.6( d) are uniformly continuous. 5 . DISTA N CE UNIFO RMLY CONTINUOUS Let A be a nonempty subset of a metric space (X, d) and let d (x , A) = inf {d (x, a) : a E A } denote the distance from x to A of Exercise 2.4-16. Show that the map x 1-----+ d ( x , is uniformly con tin uous.

A)

6 . EVEN-ODD Assume that the differentiable function f has an inte grable derivative f' . Show that is even on ( -r, r ) if and only if its derivative f' is odd there. Show that is odd if and only if f (0) = ° and f' is even.

f

f

Hints 2. Use 2.2.6( c) for the continuity and the definition of uniform continuity for the uniform continuity. 3. Note that I d (x, y) - d (x ' , yl ) 1 ::; d (x , x ' ) + d (y, y' ) .

4 . Projections are linear and continuous.

2.8

Compactness

The idea of compactness embodies two concepts: smallness and neatness . As we shall see, closed intervals [a, b] are compact, but half-open intervals ( a , b] are not. The half-open interval is even smaller than the closed interval but the fuzzy edge at the left endpoint disqualifies it. Unbounded sets are never compact . There are several ways to characterize compactness, but as with continuity, the sequential way is usually the best one to use in normed spaces. We first need the notion of an (-net. Definition 2.8.1 {- NETS

With { > 0, a subset E of a metric space (X, d) is called an {-NET for X if for any x E X, there exists y E such that d (x , y) ::; (. Equivalently, E is an {-net for X if and only if for any ( > 0 , X is "covered" by the closed balls C (x, ( ) in the sense that X = UXEE (x, ( ) . 0

E

C

Note that a dense subset of a metric space is an (-net for any positive (. The integers are a ( 1/2)-net for R. The finite set {O, lin , 2/n , . . . , nln} Spaces constitutes a lin-net for the closed interval [0, 1] for any n E such as [0 , 1] in which finite {-nets exist for every ( > ° are of special imp ortance.

N.

70

2. Analysis

Definition 2.8.2 TOTAL BOUNDEDNESS

A metric space (X, d) is called TOTALLY BOUNDED (or PRE COMPACT ) if it has a finite {-net for any { ° or, equivalently, a finite number of closed (or open) balls of radius f cover X for any positive f . 0

>

[0, 1]

We observed above that is totally bounded; so is any bounded closed (or open) interval. R is not totally bounded, and neither is any unbounded metric space (X, by the following argument. If X is totally bounded then it is covered by a finite number of bounded sets, so it must be bounded. Bounded sets need not be totally bounded, as we shall see after 2.8.7.

d),

M

2 . 8 . 3 SUBSETS AND CLOSURES For a n y subset of a totally bounded metric space (X, and cl M are totally bounded.

d), M

X and consider the subset { X I , X 2 , . . . , x n } = {x E E : d (x, M) < f/2} , where d (x, M) is the distance inf {d (x, m) m E M } from x to M. Choose arbitrary points { m l , m 2 , . . . , mn } from M such that d (xj , mj ) < f/2 for each j. For any point m E M, d(m, x) :::; f/4 for some x E E. Therefore , x = Xj E { X l , X 2 , . . . , xn } , and d (m, mj ) :::; d (m, xj ) + d (xj , mj ) :::;
:

inequality implies that an f/2-net for M is an f-net for cl M . 0 For any

1 :::; p < 00 ,

(2 .7) since for

x = (aj ) E Rn , )

Hence an (f/ n l / p -net for 1 1 · 1 1 00 is an {-net with respect to 1 I · l lp . Con sider real (n) = (Rn , IH l oo ) . We commented above that is totally bounded because the set l/n, 2/n, . . . , n/n} (n E N) constitutes a l/n net for By considering the grid of pairs of numbers + i/n , c + j/k) where i, j, n , k are appropriately chosen integers , rectangles x [c, are seen to be totally bounded subsets of ( R2 , 1 1 · 1 1 00 ) ' Similarly, CELLS Cartesian products of a finite number of closed intervals-are totally bounded in (Rn , 1 1 · 1 100 ) and therefore totally bounded in the Euclidean spaces ( Rn , 1 1 - 1 1 2 ) as well. Since any bounded subset of (Rn , 1 / · 1 1 00 ) is contained in an appro priate "cube" about the origin, boundedness is seen to be the same as total boundedness in (Rn , 1 1 · 1100 ) and therefore i n Rn , 1I · l l p for any n E N and any 1 :::; p :::; 00 .

foo [0, 1].

[0, 1]

{O,

(a

(

)

[a, b]

d]

2. Analysis

71

2.8.4 BOLZANO-WEIERSTRASS THEOREM A ny bounded sequ e n ce (xn ) in 1 ::; p ::; 00 , for any k E N , has a convergent subsequ e n ce .

(Rk, 11·llp ) ,

11·11 00 .

The P roof. B y Inequality (2.7) , i t suffices t o prove the theorem for idea is the same for any k , but it is easier to visualize in R2 so we argue that case. Since (xn ) is bounded , it is contained in some closed ball-a square S= for some 0 , in this case-about the origin. Since S x is totally bounded, it follows from the argument immediately preceding the theorem, it is covered by a finite number of squares of side a/2. At least one of those squares, SI say, must contain Xn for infinitely many different values of n, i.e . , SI contains a subsequence of (xn ) . Since SI is totally bounded , it is covered by a finite number of squares of side /4 . Let S2 be one of them that contains a subsequence of the subsequence in SI . For each positive integer n, let Sn be a square of side a/2 n that contains a subsequence of the subsequence in Sn - l . It is now possible to choose X i " E Sn such that n < m ==> i n < im . By Example 2.6 .2( a) , 2 is complete ; therefore the nested sequence of closed sets Sn has nonempty intersection by 2 . 6.4. Since the diameters (Sn ) -+ the intersection nn e N Sn must be a singleton {x} . For any n, since Xi " , X E Sn , it follows that

[- a, a] [ - a, a]

a>

a

d

R

0,

SO X i ,, -+ X . O Thus, another way to describe a totally bounded set is as one that is small in the sense that sequences cannot wander around too much . The notion of compactness defined next is usually called sequ ential compactness. Definition 2.8.5 COMPACTNESS J{

( X, d)

A subset K of a metric space is COMPACT if every sequence from has a subsequence that converges to a point of K. 0

Example 2.8.6 COMPACTNESS EXAMPLES

Let • •

•

•

(X, d) be a metric space. ( a) A trivial metric space is compact if and only if it is finite. (b) Any finite set S e X is compact . Any sequence (Xn ) from a finite S must be such that Xn = x E S for infinitely many values of n. These Xn constitute the convergent subsequence.

U

(c) Let (xn ) be a sequence in X . If Xn -+ x , then {xn } {x} is com pact , since any subsequence of a convergent sequence must converge to x, too.

R

(d) is not compact , since the sequence ( n ) has no convergent sub sequence (any subsequence is unbounded) .

72

2. Analysis •

[a, b]

(e) Any closed interval is compact by the Bolzano- Weierstrass theorem, but (0, 1] is not compact, since ( l in) has no subsequence that converges to a point of (0, 1 . 0

]

The next result links compactness and total boundedness. 2.8.7 COMPACT SPACES The metric space if it is comp lete and tot ally bounded.

(X, d) is compact if and only

(Fn) 0; (xn)

Proof. Assume that X is compact and let be a nested sequence of nonempty closed sets whose diameters go to for each n E N , choose E Fn . Since X is compact , the sequence must have a subsequence with limit say. Since E for k � n, it follows that E cl F ( = for each n E N . The completeness of X follows from 2.6 .4. Suppose that is not totally bounded and E X . Since X is not totally bounded, there is some { > ° for which there is no finite { -net . In particular, the closed ball {) cannot cover so there exists E X such that � L Since C {) and {) cannot cover X either, there must exist EX such that � f and � L We continue this process and generate a sequence for which � { whenever n =f:. m. Since such a sequence cannot have a Cauchy subsequence, it cannot have any convergent subsequence; therefore, X is not compact. Conversely, assume that is a sequence in the complete , totally bounded space We now proceed as in the proof of the Bolzano-Weierstrass theorem (2.8.4) . Since is totally bounded , a finite number of closed balls of radius must cover X . At least one of them, say, must therefore con tain a subsequence of i.e . , for infinitely many different values of n. As a subset of a totally bounded space, FI is totally bounded (2.8.3) , so it is covered by a finite number of closed balls of radius 1/2. As was the case for one of these, say, must contain a subsequence of the subsequence in Continuing in this way, we generate closed balls Fn of radius 2 - n and a subsequence of the original sequence with E Fi for every n E Fi , it follows that Since j > implies that :::; 2 - for is Cauchy. Since X is complete, the subsequence j, k � n, so therefore converges to some x. 0 Thus, if K is a subset of a complete metric space, then K is compact if and only if K is closed and totally bounded. It is now easy to give examples of bounded sets that are not totally bounded .

d (Fn )

Xn

X k Fn Xl C (Xl , X, X2 (x I , C (X 2 ' d (X 3 , X 2 ) d (X 3 , X I ) (x n) d (x n , x m ) x,

X.

1

FI , Fl.

i.

i (x n ,)

•

x

n Fn) X

d (X l , X 2 ) X3

(xn)

X (xn),

F2 (xn ,)

Xn

xnJ

(C [0, 1] , 11 · 11 ) X n (t) t n 00

FI

Xn• d (X nk ' XnJ

(xn,)

The unit ball of is bounded but not totally bounded. The sequence has no 1 1 · l I oo -convergent subsequence , since = a uniform limit is a pointwise limit and the pointwise limit of any subsequence of this pointwise convergent sequence is discontinuous.

2. Analysis •

73

The unit ball U of 1.2 is bounded but not compact , since the sequence of standard basis vectors ei E N , with 1 in the ith position, has no convergent subsequence (it has no Cauchy subsequence) .

= (0, 0, . . . , 0, 1, 0, . . . ), i

As it happens, compactness of the unit ball is a characteristic of finite dimensional spaces in the sense that a normed space is finite-dimensional if and only if its unit ball is compact (Narici and Beckenstein 1985, p. 92) . The next result generalizes the familiar one that a continuous function on a closed bounded interval is uniformly continuous. It shows that , gener ally, the mixture of continuity and compactness is potent enough to yield uniform continuity. 2.8.8 UNIFORM CONTINUITY ON COMPACT SETS A continu ous fun ction f : X � Y m apping a compact metric space X into a metric space Y is uniformly continu ous.

Proof. 0,

d

We use to denote the metric on each space and argue by contradic tion. If f is not uniformly continuous, there is some { > ° for which, for all 8 > there always exist E X with 8 but (f f �L In particular, for each n E N , there exist such that lin but (f f 2: L By the compactness of X we see that there must exist subsequences and of and respectively, such � that Un � U and We still have that lin and lin, it follows that U = � dor every n . Since d (f ,/ v. The con tin uity of f implies that I and I both converge to f Since they both converge to I ,I this contradicts (f 2: { for all n E N . 0

x, y

d (x, y) < d (x) , (y)) d (Xn, Yn ) < x n , Yn d (x) , (y)) (un) (vn ) (Xn) ( Yn ), Vn v. d (un , vn) < d (un , vn) < (un) (Vn)) ( un ) ( u). ( vn ) (u), d (un) ( Vn ))

As an application of 2.8.8, we prove the intuitively plausible result that the function fh = f (t � ° in (R) , This has a � f as number of imp ortant uses in Fourier analysis.

(t)

+ h)

h

( Lp

II · l ip )

.

2.8.9 CONTINUITY IN THE MEAN For f E (R) , 1 :::; p 00, let be any real number and let fh t Then I( � ° as �

Proof.

0,

(t) =

+ h).

Lp

11 1 - fh ll p

<

h

h 0.

For { > b y Example 2.4.5 there exists a continuous function 9 defined on R with compact support such that {/3. Assume that 9 vanishes outside [- b, b) . By 2.8.8, 9 is uniformly continuous, so there exists a 8, ° 8 such that

11 1 - gll p <

< < 1, ) l ip gI (t + h) - g (t) l < af ( 2 (b 1+ 1) for l h l < 8.

Therefore,

00 Ig (t + h) - g (t) IP dt = 1b + l Ig (t + h) - g (t) IP dt < (�) p , 1- 00 3 -(b+ l)

74

2. Analysis

or

Il gh - g llp < ( /3. It is easy (change of variable x = t + h) to prove that for any

u

E

Lp (R) ,

Hence , for Ih l < 6,

Exercises 2 . 8

1.

FINITE UNION S Show that a finite union of compact sets is compact. What about infinite unions?

2. SUBSPACES Show that a subspace of a compact space is compact if and only if it is closed.

3. Show that a totally bounded space is separable. 4. PRO DUCTS If X and Y are compact metric spaces, show that the Cartesian product (X x Y, is compact.

doo )

5 . Show that a compact metric space must be of finite diameter. 6 . If ( xn ) is a sequence from a compact metric space with only one cluster poin t x , show that X n --+ x.

7. HILBERT CUBE COMPACT Show that the Hilbert cube

mentioned in Exercise 2.5-5, is compact . 8 . IMAGES O F TOTALLY BOUNDED SETS

(a) Show by an example that the continuous image of a totally bounded set need not be totally bounded . (b) Show that the uniformly continuous image of a totally bounded set is totally bounded. (c) Deduce from (b) that the continuous linear image of a totally bounded set is totally bounded.

2. Analysis

75

9. IMAGES OF COMPACT SETS Let f : X -+ Y be a continuous function mapping a compact metric space X into a metric space Y. ( a) Show that f has compact range f ( X ) .

10. CLOSED MAP Show that a continuous map f : X -+ Y mapping a compact metric space X into a metric space Y is CLOSED in the sense that it maps closed subsets of X into closed subsets of Y. (a) If Y = R, show that f achieves a maximum and a minimum value. 1 1 . POSITIVE DISTANCE BETWEEN COMPACT AND CLO SED Show that if I< is compact and B is closed and I< n B = 0 , then the distance = inf ( a, : a E I<, b E B} is positive. By means of an ( I<, example , show that this need not be true for disjoint closed sets.

B)

d

{d b)

Hints 3. Consider lin-nets, n E N .

((Xn, Yn)) E X Y, consider convergent sub ( Yn). 6 . I f x i s not a limit , there is some r > 0 for which a subsequence belongs x

4. For a sequence ( wn ) = sequences of ( xn ) and

to the complement CB(x, r ) of the open ball B(x, r ) of radius r about x.

k

Ln>k

2

7. Given t > 0, choose such that 1 / n < t / 2 and consider an t / 2-net for the Cartesian product n�= l [ - l/k, l / k] .

2.9

Equivalent Norms

2

In the plane R the Euclidean metric

the

max

metric doo (x, y) = max

and the taxicab metric

{ 1 (X 2 - x d l , I (Y2 - yd l } ,

76

2. Analysis

are EQUIVALENT in the sense that they determine the same families of convergent sequences : Any sequence that converges with respect to one of them will converge with respect to any of the others. Thus, any "analytic" result proven for one of them is valid for the others as well. Another (log ically equivalent) imp ortant way in which they are equivalent is that they each determine the same class of open sets, since the open balls that they generate-the circles, squares, and tilted squares, respectively, of Example 2 . 1 .2-can be fitted inside one another . Therefore , any set that is open with respect to one of them can be written as a union of open balls generated by any other one, as in equation (2.5) of Section 2.5. None of them is equiva lent to the trivial metric , since some of the trivial metric 's open balls are singletons. We develop a criterion for two norms to be "equivalent" and (they determine the same convergent sequences) in equation (2. lO) . Any 1 I · I I t-open ball = EX : < about in the normed space X is the translate + l (O, 1) of Therefore , to is equivalent to show that it suffices to show that the open unit ball 1 ) may be written as a union of 1 1 · 1 1 2 -open balls. A nontrivial result about equivalent norms is that all norms on a finite-dimensional space are equivalent (Exercise 3), from which it follows that all linear maps on a finite-dimensional domain are continuous (Exercise 4) . Having a great many neighborhoods of a point or having very "small" ones makes it more difficult for a sequence to converge. For a sequence to converge with respect to the trivial metric , for example, the sequence must be eventually constant , so convergence with respect to the trivial metric is hardest of all. We use relative ease of convergence to formalize comparisons of norms in the definition below.

B l (O,

11 · 11 1 11·11 2 B l (x, r) {y Il y - x ii i r} , r > 0, x rB B l (0, r). 11 · 11 1' 11·11 2

x

Definition 2 . 9 . 1 STRONGER NORMS

If 11 · 11 1 and 11·11 are norms on the same vector space X and 1 1 · l l cconvergence X n x implies I2H l 2-convergence X n x, then we say that 11 · 11 1 is STRO NGER than 11 · 11 2 or that 11·11 2 is WEAKER than 11 · 11 1 ; we write 11 · 11 1 � 11 · 11 2 . Clearly, 11 · 11 1 � 11 · 11 2 1 1 · 1 1 1 � 1 1 · 11 z ; as we prove below (2.9.2) , this is essentially the only way in which one norm can be stronger than another . 11 · lIp � 11 · 11 00 on K nl forp any n E N and lalI I :s p < 00 Let (ad E K n . Since l aj I = ( I aj n / :s ( 2:7= 1 ( lad P )) / p for each j , it follows that (2.8) 11 · 11 00 :s 11·ll p ' 1 :s p < 00 . Therefore, 11 · l l p is stronger than 11 · 11 00 . 11·11 1 � 11 · l b CUI U2 for some c > O. Next, we show that 11 · 11 1 � 11 · 11 2 implies that a positive multiple of the closed unit ball Ul = {x E X : II x ll l � I} determined by 11 · 11 1 is contained in the closed unit -+

-+

0

=>

•

•

=>

C

0

77

2 . Analysis

ball U2 determined by 1 1 · 1 1 2 ' By the definition of continuity, 1 1 · 1 1 1 !': 1 1 · 1 1 2 means that the identity map

X

1----+

X.

is continuous. Since I is linear , it is continuous if and only if i t is continuous at (Example 2.2.6(e) ) . Therefore, the continuity of I implies that CUI C I - I (U2 ) = U2 for some c > by the { - o formulation of continuity at (Exercise 2 .2-4) . 0

0

0

0

We collect these observations together in the following proposition. 2.9.2 CHARACTERIZATIO N O F STRO NGER NORMS Let 1 1 · 1 1 1 and 1 1 · 1 1 2 be norms on the vector space X . Then the following statements are equivalent: (a) 1 1 · 1 1 z :5 1 1 · 1 1 1 ; (b) CUI C U2 for som e c > 0 ; ( c ) c 1 1 · 1 1 2 � 1 1 · 1 1 1 for some c 0 ; (d) Every 1 1 · 1 1 2 - open ball is 1 1 · l l l - open.

>

Proof. We have already seen that (a) implies (b) . If (b) holds and x =P 0,

then

O.

which yields (c) , the result being trivially true for x = That (c) implies (a) is clear , since c I Ixn - x l 1 2 � I I xn - x I I I ' Let Bl = {x E X : I I x l l l < I } and B 2 = {x E X : I I x l 1 2 < It is easy to see that Bl is the interior , int Ul , of Ul ; similarly, B 2 = int U2 . We show below that (b) implies that B 2 can be written as a union of 1 1 · l l l -open balls. Since any 1 1 · 1 1 2 -open ball about x E X is of the form x + rB 2 , for some r > 0, it will follow that every 1 1 · I I z -open ball can be written as a union of 1 I · l I copen balls and is therefore 1 I · l I copen. By (b) , CUI C U2 for some c > so (Exercise 2.5-4)

I}.

0,

int cUl = c Bl

C

int U2 = B 2 .

Since B 2 is 1 I ' 1 I 2 -open, it can be written as a union of 1 I · 1 I 2 -open balls about each of its points by Example 2.5.2(d) : For each x E B 2 , there exists rx > such that

0

Since x + rx cBl e x + rx B 2 for all x , it follows that (2.9)

78

2 . Analysis

Therefore , B 2 is 1 I · l I copen, and we see that (b) implies (d) . Last, we show that (d) implies (b) . With = 0 and reading equation (2 .9) from right to left , it follows that for k = cro ,

x

Now take closures. 0

implies that 1 I · l I l -Cauchy sequences are By (c) it is clear that 1 1 · 1 1 2 � 1 I · 1 1 2 -Cauchy sequences . In Exercise 2 . 1-2 we noted that p < q implies in By 2.9 .2(b ) , C for p < q . By this observation and (2.8) , it i t follows that � follows that for 1 S; P S; q S; 00 . � We say that 1 1 · 1 1 1 and 1 1 · 1 1 2 are EQUIVALENT, 1 1 · 1 1 1 '" 1 1 · 1 1 2 ' if each is stronger than the other , in other words if they determine exactly the same class of convergent sequences. Geometrically, this means that each unit ball may be shrunk so as to fit into the other one. By 2.9 .2(a,c) (twice) it follows if and only if there are positive constants such that that 1 1 · 1 1 1 '"

1I·lIq

1I·l h

Up Uq iq.

1I·lIp 1I · lIq 11·lIp

a, b

1I·l b

(2. 10) The roles of 1 1 · 1 1 1 and (2. 10) is equivalent to

1I·lb may be interchanged in the above expression : (l/b) 1 I ·lb S; 1 1 · 1 1 1 S; ( l/a ) 1 1 · 1 1 2 '

so it makes no difference whether one says 1 1 · l i I '" 1 1 · 1 1 2 or 1 1 · 1 1 2 '" 1 1 · 1 1 1 ' Another way of describing equivalence is that there exist b > 0 , such that < ll.:!k < b . - 11·111 -

a,

a

Il x lip =

(�n l ai lP ) lip

S;

(n

II x ll � ) l /p = n l /p II x l i oo , 1 S; p < 00 ,

so 1 · I � 1 1 · 1 1 00 ' Combined with Inequality (2.8) , we have the following result .

I lp

2.9.3

Kn . Kn .

1I · lIp

Kn , n

EQUIVALENCE OF p- NORM S O N on E N, 1 1 · 1 1 00 for all 1 S; P S; 00 ; by the evident transitivity of equivalence of norms, it follows that all p-norms are equivalent on '"

This is a special case of the fact that all norms are equivalent on any finite-dimensional space (Exercise 3) . Even though p-norms are equivalent in the finite-dimensional case, they are not in the infinite-dimensional case.

2. Analysis

79

Example 2.9.4 INEQUIVALENCE O F p- NORMS

(a) For q > p � 1, 1 I · l l p + 1 1 · l l q on £p . it follows that As observed after 2.9.2, for q > p � 1, C in belongs to £p C £ , and it is a proper inclusion : In particular, = £g , but not to £2 . If we restrict 1 I · l I g to £2 , it is not equivalent to 1 1 · 1 1 2 since the sequence with kth entry

q

(xn )

Up Uq £q; l x (n - / 2 )

x n (k) = { k - l / 2 kk >� n,n, 0,

'

nEN

is 1 I · l I g -Cauchy but not 1 I · 1 1 2 -Cauchy. (b) 1 I · l l p + 1 1 · 1 1 00 on Co for any 1 � p < 00. that are eventually 0 , and of sequences from sequences . Observe that

K

cp C

£p

C Co ·

As in the finite-dimensional case, 1 1 · 1 1 00 ::5 1 I · l I p on for any p �

1,

£p

( n l1/p ) E

Co

but

( n l1/p )

Co

Let cp denote the set the space of all null

£p for any p � 1 . Also ,

� £p ,

so is a proper subset of Co . Note that (cp, I I · l I oo ) is dense in ( co , 1 1 - 1 1 (0 ) ' since any point of Co is the limit of the sequence obtained by taking to be those of for � n and 0 for > the components of Since 1 1 . 1 1 (0 ) : therefore (£p , 1 1 . 1 1 (0 ) is not (cp, 1 1 . 1 1 (0 ) is dense in ( co , 1 1 - 1 1 (0 ) ' so is closed in ( c o , 1 1 - 1 1 (0 ) ' Since complete subspaces must be closed , it follows is a that (£p , 1 1 . 1 1(0 ) is not complete as a subspace of ( co , 1 1 - 1 1 (0 ) ' Since Banach space with respect to 1 I ' l I p , 1 I · l I p is not equivalent to 1 1 · 1 1 00 . (c) 1 I · l I p + 1 1 · 1 1 00 on Consider the space (Example 1 .2 . 3) and the norms

x X n

x(£ , k (xn ) p

k

n.

£p

C [a, b] .

I I x l l oo = sup I

C [a, b]

x [a, b] 1 and I I x l ip =

(lb l x(t) IP dt) l /p

Since

it follows that

x

II l i p

� (b - a) l / P I x l oo ,

so 1 1 · 1 1 00 is stronger than 1 I · l I p ' But they are not equivalent: The sequence = from 0 , 1 ] is 1 I · l l p -convergent to 0 for any � p < 00 , but does not converge uniformly to O. 0

Xn tn (n E N )

C[

1

In Example 1 . 1 .3 we considered the notion of isometry of metric spaces. The notion of "homeomorphism" is similar, but weaker. If two metric spaces

80

2 . Analysis

X and Y are isometric under an isometry I : X Y , then a sequence ( x n ) converges in X if and only if (f(xn ) ) converges in Y . If a bijection I has only -+

this convergence-preserving property, then I is called a HOMEO MORPHISM. A homeomorphism I, in other words, is a bicontinuous ( f and 1 - 1 are is a homeomorphism of continuous) bijection . The map I-> (1 + R onto the open interval ( - 1 , 1) that is not an isometry. In Section 1 .5 we discussed lin ear isometry of normed spaces. If two normed spaces are linearly isometric, they are called "geometrically" the same; if they are linearly homeomorphic (defined below) , they are called "topologically" the same.

x

Ix l)

xl

Definition 2.9.5 LINEAR HOMEOMORPHISM

X

A X

Let and Y be normed spaces. A bicontinuous linear bijection : -+ Y is called a LINEAR H OMEO MORPHISM . If a linear homeomorphism exists between and Y , we say that X and Y are LINEARLY HOMEOMORPHI C .

X

o

Exrunple 2.9.6 LINEAR HOMEOMORPHISMS

(a) A linear isometry is a linear homeomorphism. The converse is false : the map I-> 2t is a linear homeomorphism of R onto R that does not preserve distance . (b) Linear homeomorphisms preserve completeness . (Proof? Consider the linear image of a Cauchy sequence.) , (c) If "' I I · I I 2 then the identity map I : -+ I-> is a linear homeomorphism. Hence , by 2.9.3, for any n E N , the spaces fp ( n ) and ( n ) are linearly homeomorphic for all p :s q 00 . ( d ) I t follows from Example 2.9.4 that none o f the identity maps I-> below are linear homeomorphisms.

t

x x,

(X, II · II l )

11 · 11 1

fq

(X, 11 . 11 2 ) 1 ::; ::; x x

(fq , II · II q ) for q > p � I , ( Co, II · 11 00 ) , ( C [a, b] ' II · IIp ) . O Exercises 2 . 9

1.

(X, II · II ). 11 · 11 11 · 11 00 .

Write Let {X 1 , X 2 , . . . , Xn } be a basis for a normed space E as for appropriate E I<, i = 1 , 2, . . . , n. Define lad · Show that = = I :S

x X 2:::7= 1 ai x i I x lioo 112:::7= 1 ai x ill oo maxi

ai

2 . FINITE-DIMENSIONAL SUBSPACES COMP LETE Show that any finite dimensional subspace of a normed space is complete. Conclude that finite-dimensional subspaces are always closed, in particular, that ( n ) is a closed subspace of f2 ( N ) for any n E N .

X

f2

2 . Analysis

81

3 . NORMS EQUIVALENT O N FINITE-DIMENSIONAL SPACES

( a) (b )

Show that all norms are equivalent on a finite-dimensional vector space X . What about the converse? i.e., if all norms are equivalent o n a vector space X, must X be finite-dimensional?

4. FIN ITE-DIMENSIONAL IMPLIES CONTINUOUS

( a)

Show that any linear map A : X � Y defined on a finite dimensional vector space X into any normed space Y is con tinuous. In particular, any linear functional i.e . , Y = on a finite-dimensional space is continuous. b If all linear functionals are continuous on the normed space X, show that X is finite-dimensional . then and only then the identity map x 1-+ x c If of (X, onto (X, is discontinuous. Thus, any time one norm is not dominated by another , you have a discontinuous linear map .

(

() ( ) 1I·l b 1. 11 · 11 1 ' 11 · 11 1 )

K)

)

( 11 - 11 2 )

5 . FINITE-DIMENSIONAL SPACES ARE LINEARLY HOMEOMORPHIC Show that any two normed spaces X and Y of the same finite dimension are linearly homeomorphic.

6 . EQU IVALENT METRICS If d and d' are metrics on X , we say that d and d' are EQUIVALENT if there are positive constants a , b such that

( y) < d' ( x , y) < bd( x , y)

ad x , for all

x,

y E S. Note that these inequalities are equivalent to ( l / b) d' ( x , y) < d( x , y) < ( l / a ) d' ( x , y),

so there is no favoritism shown to either metric . Indeed, show that "equivalence of metrics" is an equivalence relation among the metrics on any given set. 7. BOUNDED METRIC

( a)

That there exists a metric on a set that makes it into a space of finite diameter is clear: The trivial metric is one such metric . Significantly, given a metric space (X, d there is an equivalent metric d' that makes X into a bounded metric space. Show that if d is a metric on X , then so is the BOUNDED METRIC d' = d 1 + d , that the d'-diameter of X i s finite, and that d i s equivalent to d' ; in other words, (X, d is homeomorphic to (X, d' .

)

)

/(

)

)

82

2. Analysis

( b ) If {dn : n E N } is a family of metrics on X, then show that dn(x, y) d(x, y) = L � 2 n 1 + dn (x, y ) nEN is a metric .

Hints 2 . By the result of Exercise 1 on the finite-dimensional subspace M, hence it suffices to show that is complete . Show that is complete by imitating the argument of Example 2 .6 .2 b . Let be a basis for X and let = be a Cauchy sequence in For each E 2, . . . , n , show that for each i. Show that is a Cauchy sequence. Let = limk with respect to -+

(M, 11·11 00 ) 11·11 :5 11·11 00 ; (M, II·ll oo ) ( ) X l , X 2 , . . . , Xn Yk 2.:7= 1 aik x i X. } i {I, bi (a ik ) aik Yn 2.:7= 1 bi X i 11·11 00 · 3. ( a) . Let X l , X 2 , . . . , Xn, be a basis for X and let 11 · 11 00 be as in Exercise 1 . Show that any norm 11 · 11 i s equivalent to 11 · 11 00 . By Exercise 1 , 11·11 :5 11·11 00 ' so it only remains to show that 11·11 00 :5 11 · 11· For each i, consider the closed ( Exercise 2.4-2) linear subspace Mi spanned by {X l , X 2 , . . . , Xi - l, X i + l , · · · , xn}. The set X i + Mi is closed by Exercise 2 .4-7( a) . Show that ° rf. X i + Mi , so that for some open neighborhood B (0, ri ) , ri > 0, of 0, B (0, 7'i ) n (Xi + Mi ) = 0 . It follows that there must be some ri > ° such that , for all m E Mi , Il x i + m il � ri . Let X = 2.:7= 1 aj xj , where aj E K , j = 1 , 2 , . . . , n . For any nonzero scalar ai , n n a· Il x ll = L= aj Xj = l ail X i + '" L...J ...L Xj � l a d ri � l a il m in ri j l j = l,j ;t i ai so I 2.:i= I aj X I � (mini ri ) Il xi l ( b ) The converse is true. Let B be a Hamel base for X. For X E X write X = 2.:7= 1 a i X i for appropriate X i E B and scalars a i . Each of the following defines a norm on X: Il x ll l = 2.: 7= 1 l a i l and Il x ll oo = maxi l ai l · If X is infinite-dimensional, show that 11 · 11 1 1:. 11·11 00 . 4. ( a) . Let {X l , X 2 , . . . , Xn} be a basis for X and let 11 · 11 00 be as in Ex ercise 1 . By Exercise 3 , we may assume that X carries 11 · 11 00 . Let {X l , X , . . . , Xn} be a basis for X, so that for any X E X, X 2.:7= 1 2ai X i , for appropriate scalars ai . Then . Z

j

OO"

'

2. Analysis

83

Now use the criterion for continuity of a linear map of Exercise 2.2-9 . (b) . If dim X = 00 , construct a linear functional that i s unbounded on the closed unit ball.

d' is a metric , note that for all x, y, E X , d(x, y) d(x, y) 1 + d(x, y) � 1 + d(x, y) + d(y, ) To see that it is equivalent to d, note that the function t tf (1 + t), t > 0 , has a unique inverse.

7 . (a) . In order to show that

z

"

z

�

2 . 10

Direct Sums

Y

Can two normed spaces X and be combined to yield a new normed space? Can two inner product spaces be combined to get a new inner product space? Moreover, can the combining be done in such a way that we do not lose the original spaces entirely? Specifically, can the combination be formed so that it contains copies of the original spaces? We considered the analogous question for metric spaces (X, in Example 1 . 1 .4. The new space was the Cartesian product X x metrized by any of the metrics

d l ), (Y, d2 ) Y

or

doo (x, y) = max [dl (x, y) , d2 (x, y)] .

y

In each case the original space X is recoverable as (isometric to) X x { } for any fixed in For linear spaces X and over the same field K , we make X x into a vector space by defining addition and scalar multiplication as follows: For E X, E and a E K,

y Y.

Xl , X2

Y

Y

Y2 , Yl Y, Y

$Y

$Y

So equipped, X X is denoted by X or X (ext) and called the EXTERNAL ( ALGEBRAIC ) DIRECT SUM of X and Y. If (X, { " ' } l )and (Y, ( are inner product spaces, the EXTERNAL INNER PRODUCT DIRECT E X SUM is X equipped with the following inner product: For and E

'}2 ) $Y Yl , Y2 Y,

"

Xl , x2

Y

The original spaces X and may be recovered as inner product isomorphs and of X x { O } ( = X {O}) and {O} x respectively. If (X,

$

Y,

11 · 11 1 )

(Y, 11-I1 2 )

84

2. Analysis

are normed spaces, we have choices about how to norm X and y E Y , norms are defined by

x

Y : For

(2.11)

or any of

(1.6.3

xEX (2.12)

1.6-2)

The Minkowski inequalities and Exercise validate the triangle inequality in each case. It makes little difference which norm is selected , since the 00 , are equivalent ; in other words, �p< q

11·lI p , 1

(X

Ef)

Y,

�

1I·lIp )

is linearly homeomorphic to

x x.

(X

Ef)

(

Y,

11 · 11 ) q

1I · ll p ) (2.11)

under the identity map 1---7 Thus, although the spaces X Ef) Y, are obviously not linearly isometric as p varies, they do have the same conver gent sequences . X Ef) Y equipped with any of the norms of equation or i s called the EXTERNAL TOPOLOGICAL DIRECT SUM of X and Y; other aliases are DIRECT PRO DUCT and TOPOLO GICAL PRO D U CT. Any finite number Xl , X , Xk of normed spaces is easily accommo dated by the same technique; ED := l Xi denotes the direct sum in this case. c) , a We endow with one of the norms 00 . As in p sequence n E N , from ED := l Xi converges = to if and only if � i) for each no matter what norm 00 , is chosen for ED := l Xi . It follows from this observation � p that

(2.12)

2,

•

.

.

11 · lIp ' 1 � � x Xn (xn(1), n(2), . . . , xn(k)), x Xn (i) x( 1�i�k 1I·ll p , 1 � • •

•

2.2.6(

The external direct sum of Banach spaces is a Banach space.

.

{O} x . . for all j .

x

{O}

x

Xj

x

{O}

x . . . x

{ O } is a closed subspace of ED := l Xi

of ED := l Xi onto Xj The projection maps = are continuous for each j; indeed, they are uniformly continuous linear maps .

Pj [(X l , X 2 , . . . , Xn)] Xj N

Now suppose that M and are linear subspaces of a normed space X . For appropriate M and N (i.e . , big enough M and N) , can w e synthesize X from M and N by this technique of pasting spaces together? As we shall see, it is easy to put X back together algebraically but it may not be possible to recover X as a normed space from component subspaces . More specifically, the norms p 00, on M x N may not yield a space that is even linearly homeomorphic to X, let alone linearly isometric to it. If M n N = {O} and X = M + N in the sense that any vector in X can be written as

11 · lIp , 1 � �

x

x = m + n for

m

E M, n E N,

85

2. Analysis

M

N,

M

N

then X is called the INTERNAL DIRECT SUM of and and and are called ALGEBRAIC COMPLEMENTS of each other. Sometimes this is written X

= M EEl N (int) .

It can be shown that every subspace M has an algebraic complement to a Hamel base B' for the whole space; a extend a Hamel base B for complement N will be the linear span of the "new" basis vectors , the basis vectors of B ' not in B. Furthermore, every algebraic complement of M has the same dimension , and this dimension is called the CODIMENSION of M, denoted by codim M. Not only can every in X be written in the form for some and but the and are unique: if

M

x = m+n

then

x = M EEl N m E M n E N, m x = m + n = m' + n',

n

m - m' = n' - n E M n N = {OJ .

Algebraically, there is no difference between external and internal direct sum. If X N (int) , then consider

= M EEl

M' = M {OJ

N' = {OJ N. It now follows that X is linearly isomorphic to M' EEl N' (ext) : The map M' EEl N' (ext) ---. M EEl N (int) , m E M, n E N, (2.13) ((m, O) , (O, n)) m + n, x

x

and

�

is a linear isomorphism . For X N (int) , since the representation N, is unique, the PROJECTION of X ONTO M

= M EEl

x = m + n, m E M, n E Pl Pl : X = M EEl N (int) ---. M, x=m+n m, is a well-defined linear map; so, of course, is P2 x = n, the PROJECTION of X O NTO N. Unlike projections on external direct sums, however, proj ec �

tions on internal direct sums do not have to be continuous. For example , there is no continuous projection of (bounded sequences) onto Co (null sequences) . (The argument can be found in Narici and Beckenstein p . 87.) Let (int ) . Suppose (ext) is normed by one of the norms :::; p :::; 00 , of equations and and consider the map (with "s" for "sum" )

£00

X = M EEl N 11 · lIp , 1 S:

M EEl N (2.11)

M EEl N (ext) ---. X, (m, n) m + n. �

1985,

(2.12)

(2.14)

It is routine to verify that S is a linear bijection ; it is continuous because

86

2. Analysis

2.2.6(

by Example c) , and the continuity of addition in a normed space But it need not be bicontinuous, i.e . , its inverse

(2.2.3).

need not be continuous, something we say a little more about in

2.10.1.

2 . 1 0 . 1 WHEN 8 IS A LINEAR HOMEOMORPHISM 8 is a lin ear bijective homeomorphism if and only if one of the projections and is contin uous.

PI

P2

Proof.

Since 8 is a continuous linear bijection, the question concerns only the continuity of the map 8 - 1 , the map � For any metric spaces X, Y, and Z, a map � (f 9 of X into Y x Z (with any of the usual product metrics) is continuous at a point x if and only if f and 9 are continuous at x, since by Example c) ,

x (PI X, P2 X) . (x) , (x)) 2.2.6 ( (f (x n ) , 9 (xn)) � (f (x) , 9 (x)) I (xn) � I (x) and 9 (xn) � 9 (x) . Therefore, in particular, x (PIX, P2 x) is continuous if and only if PI and P2 are continuous. But one of PI and P2 is continuous if and only if the other one is, since 8 = PI + P2 , 8 is continuous, and the difference of X

�

�

continuous maps is continuous.

0

Definition 2 . 10.2 TOP OLO GICAL DIRECT SUM

The normed space X is the TOPOLOGICAL DIRECT SUM of the subspaces M and N, X = M (top) , if the map S of is a linear homeomor phism. In this case and are called TOP OLOGICAL C OMPLEMENTS (or TOPOLO GICAL SUPPLEMENTS ) of each other, and M (and are said to be TOP OLOGICALLY COMPLEMENTED . 0

EEl N M

N

(2.14)

N)

PI

is If M and N are TOPOLO GICAL COMPLEMENTS of each other , then continuous and N = so is closed. Thus, a necessary condition for a subspace to possess a topological complement is that it be closed. There are "uncomplemented" closed subspaces , however. For example (N arici and Beckenstein 1985, pp. the closed subspace Co of null sequences has no topological complement in the Banach space of Exercise 1 . 1- 1 1(b) . Thus, if N is any algebraic complement of in the map 8 : Co x N � ( m , n ) � m + n , cannot be bicontinuous. Three imp ortant special cases in which complements do exist are the following:

Pl- l (O), N 86-88),

too ,

•

•

too Co too ,

In any normed space X , if f : X � K is a nontrivial continuous linear form, then 1 - 1 has a topological complement . (N arici and Beckenstein 1 985, p. 8 9.)

(0)

N

If M and are closed algebraically complementary subspaces of a Banach space X , then they are topological complements . This follows

2. Analysis

87

from the open mapping theorem-which implies that a continuous linear bijection between Banach spaces is bicontinuous-applied to the map S. For a closed subspace M of a Hilbert space X, things really simplify : X = M EEl M a topic w e return t o i n Section 3.2. If X , Y, and W are normed spaces, and A : X - W and B : Y - W •

l. ,

are continuous linear maps , then

A $ B : X $ Y (ext) (x, y)

�

1---+

W

Ax + By

determines a continuous linear map called the direct sum of A and

B.

Exercises 2 . 1 0 In the exercises below, direct sums are endowed with any of the norms of equation (2 . 1 1 ) or (2. 12) . 1 . n-DIMENSIONAL HILBERT SPACE Suppose X is an n-dimensional Hilbert space, and that , is an orthonormal basis of vectors for Show that is isomorphic as an inner product space to the inner product space EEl f= 1 (ext) , where denotes the linear span of for = 1 , 2 , . . . , n .

{X l , X 2 , x n } X. X [X i ] [X i] KX i X i i 2 . I s the real space Roo (2) = (R2 , 11 - 11 (0 ) linearly isometric to ( R x {O}) EEl ({O} x R) (ext)? (If the direct sum carries the max norm 11 · 11 00 of equation (2. 1 1 ) , yes; otherwise, no.) .

•

•

3 . D IRECT SUM OF COMPLETE SPACES

X M

X$Y

(a) If and Y are Hilb ert or Banach spaces, show that (ext) is a Hilb ert or Banach space, respectively. (b) If and N are complete orthogonal subs paces of an inner prod uct space show that M + N is complete.

X, 4. Let Xl , X2 , Xn be normed spaces. Show that: (a) {O} x . . . x {O} x Xj {O} x . . . {O} is a closed subspace of EB �= 1 Xi (ext) for all j. (b) The projection maps Pj (X I , X 2 , , X n) = Xj of EB �= I Xi (ext) onto Xj are uniformly continuous linear maps. 5. D IRECT SUM O F SEPARABLE SPACES Let X l , X2 , . . . , Xn be sepa rable normed spaces. Show that EB� 1 Xi (ext) is separable . •

•

•

,

x

x

•

.

•

88

2.

Analysis

6. If M and N are orthogonal subspaces of an inner product space X , show that any vector x in M + N = {m + n m E M, n E N} has a unique representation as x = m + n ( i.e . , the m E M and n E N are :

unique) .

7 . If M and show that

N are closed, orthogonal subspaces of a Hilbert space X , M + N is closed.

Hints

1. Consider the map ( X l , X 2 , . , xn) X l + X 2 + . . . + X n ; cf. 1.5.4. 7. For w E cl (M + N), choose Xn E M and Yn E N such that x n + Yn w. Use the Pythagorean theorem to show that (x n) and ( Yn ) are . .

1---+

�

Cauchy sequences.

3

B ases The classical subject of Fourier series is about approximating periodic func tions by sines and cosines, specifically, about expressing an arbitrary peri odic function as an infinite series of sines and cosines. (Any function that vanishes outside some interval can be viewed as a periodic function on by merely extending it periodically.) The sines and cosines are the "basic" periodic functions in terms of which we express all others. To use a chemi cal analogy, the sines and cosines are the atoms; the other functions are the molecules. Unlike the physical situation, however, there can be other atoms, other functions, that can serve as the "basic" functions just as effectively as sines and cosines. We consider in this chapter a more general setting in which to view de composition into elemental comp onents . Specifically, we investigate how to write a vector as an infinite series of "basic" vectors in Hilb ert and Ba nach spaces. We show (3.4.8) that , in any separable Hilbert space there is an orthonormal sequence such that any E X can be written as Such a sequence is called an O RTHONORMAL = BA SIS for (The collection of sines and cosines (suitably normalized) constitutes an orthonormal base for the Hilbert space the HAAR FUNCTIONS of Section 3.5 are an orthonormal basis for The ex istence of a canonical way to represent an arbitrary element is very useful of any information. For example , the representation = in a separable Hilb ert space makes it possible to "digitize X," to view E as a sequence Put another way (3 .4.9) , any infinite dimensional separable Hilbert space is linearly isometric to Thus, one has a choice: one can, for example , view as a function space or as the sequence space We cannot do as well in separable Banach spaces . Suppose that is a sequence in an infinite-dimensional B anach space uniquely If every E can be written as = with the determined, then is called a SCHAUDER BASE (or BASIS ) for Exam The ples? Any orthonormal base is a Schauder base: take = Haar system mentioned above is a Schauder base for each of the ::; p < 00 . Does every separable Banach space have a Schauder base? This was a long-standing open question known as THE BASIS PRO BLEM . We have j ust noted that the spaces have Schauder bases; the function spaces (D) of all analytic functions on the disk D = E C : < I } , such = [J JD < 00, normed by that J JD for p = 00 , have Schauder bases; so does for ::; p < 00 , and by (C In fact, all the separable Banach spaces known before 1 973

R

(xn)

x En eN (x, Xn) xn · X.

x

X.

X « ( x, x n} ).

x X

X,

x

(xn)

L 2 [0, 271"]; L 2 [0, 1].) x En eN ( x, xn) X n £2 .

L 2 [0, 1]

£2 . (xn) x X (x n )

x En e N anxn,

an X. an (x, xn). Lp [O, 1],

{z

Izl I I (z W dxdy] l i p ,

(xn)

1

Hp

Lp

I I (zW dxdy 1 11·11 00 [0, 1] , 11 ' 1 1 00 ) '

I

11 / 11

90

3. Bases

had Schauder bases. These facts , together with the knowledge that every separable Banach space is linearly isometric to a closed linear subspace of [0 , 1] , led to the consensus that all separable Banach spaces possessed a Schauder base. But they do not . The solution came not from looking "big" -at larger and more exotic spaces-but to look "small, " inside Co . En flo ( 1973) showed that certain subspaces of and for 2 < p < 00 do not have Schauder bases. An improved version of Enflo's argument may be found in Lindenstrauss and Tzafriri 1 977; it is only for the decidedly more advanced (intrepid?) reader.

C

(co, 11 · 11 00 )

3.1

ip

Best Approximation

When we speak of approximation, as in "a real number x can be approx imated by rational numbers," we mean that there are rational numbers that are arbitrarily close to it. This implies that there is a sequence of rational numbers converging to x-choose a rational X n from the inter val - lin, + lin) for each n E But there is no sequence of vectors in the plane that converges to the point z = ( 0 , 0 , 1 ) . The vector in that comes closest to is the origin 0 (0, 0, 0). It happens that the distance from z to 0 is the same as the (perpendicular) distance from to

(x n)

(x

N.

x R2

R2 R2 :

z

=

z

d (z, 0) = d (z, R2 ) = inf { li z - x II : x E R2 } .

We take the minimal distance requirement imation in a normed space.

as

the criterion for best approx

Definition 3 . 1 . 1 BEST ApPROXIMATION

X

x

Let M be a linear subspace of the normed space and let be a vector not in M. We say that E M is a BEST APPROXIMATION TO X FRO M M if = inf : E We call the DISTANCE from x to M. As an in!, Ilx - mil 0 for all

mo

II x mo ll d(x, M) m E M.

{ ll x m il m M} = d(x, M). II x mo ll :::; -

Best approximations do not always exist-What is the best rational ap proximation to Vi? If they do exist , they need not be unique. What is the best approximation to the origin from the surface E = I } of the unit ball in R3 ? (cf. also Exercise 2 and Example 3.2 . 6) . If a linear subspace of a normed space is such that there exist unique approxi mations to any then is called CHEBYSHEV . We provide examples of Chebyshev subspaces in 3 . 1 . 2 and 3 . 1 .3. For Chebyshev subspaces M , we can speak of the best approximation to E from M.

{x R3 : I lx ll

M

x E X,

M

X

x X

3 . 1 . 2 FIN ITE-DIMENSIONAL SUBSPACES O F INNER PRO D U CT SPACES Let { X l , . . be an orthonormal set in an inner produ ct spa ce Then .

, xn}

X.

3. Bases

91

the best approxim a tion to any x E X from M = [X l , X 2 , . . . , Xn) is given by L:7= 1 ( x, Xi ) X i · Proof. Consider X = (a, b , ) E R3 and let e l = ( 1 , 0 , 0) , e = (0, 1 , 0) and e 3 = (0, 0 , 1 ) . Note that a = (x, e l ) and b = (x, e 2 ) and that2 x - a e l - b e 2 .l R2 . The distance from x to R2 is given by c

{X l , . . . , xn} M

This idea extends to general inner product spaces. Let be an orthonormal set in an inner product space X and let denote the linear span of Finding the best approximation of from is equivalent to finding scalars such that

M

[X l " ' " xn) {X l , " " xn }.

{a I , . . . , an} n X - 2: ai X ; i= l

X

ai = (x, X i ) . First, a, ) - a (X- l , X ) + ali----II x I1 22 - (x, X l---Il x ll - (X, X l ) (X, x r) + (X, X l ) (X, X l ) X l ) - a (X l , X) + a li (X, 2 II x I1 - I (x, x r) 1 2 + I (x, X l ) - a l 2 . The last expression is clearly minimized when a = ( X, X l ) ' When aX l is replaced by L:� 1 a i X i , we get more terms like the ones in the expression is minimized . We show that it is minimized when the observe that for any scalar

-Ii

Ii

above. We get

2 n n n X - 2: ai X i = II x I1 2 - 2: I (x, x i ) 1 2 + 2: I (x, Xi ) - ai l 2 . i= l Clearly, the best approximation to x is obtained by setting a i = (x, X i ) for each i. The best approximation to x by vectors in the linear span M = [X l , X 2 , . . . , xn) is therefore the "sum of its projections" (x, X i ) X i on the subspaces [X i ): n rna = 2: (X, X i ) X i . 0 ;=1 Let {X l , " " x n } be an orthogonal subset of an inner product space X and let X E X. Then the vector X - L: ?= l (x, X i ) X i is orthogonal to each Xi , for i = 1 , 2, . . . , n. Therefore , it is orthogonal to the linear span M = [X l , X 2 , . . . , xn). This suggests that in more general circumstances the best approximation to x by a vector rn a E M is an rn a such that X - rna .1 M. We show in 3 . 1 .3 that a unique best approximation to any X E X from a complete (linear, as usual) subspace M of an inner product space X ;=1

92

3. Bases

exists no matter what the dimension of M is. Since a closed subspace of a complete space is complete , below applies to closed subspaces of Hilbert spaces, and this is its most frequent application. The conclusion of the theorem fails for closed subspaces of incomplete spaces (see Exercise and Example

3.1.3

3.2.6).

2

M X x E X. mo E M x,

3 . 1 . 3 BEST ApPROXIMATION FRO M COMP LETE SU BSPACES Let be a complete lin ear su bspace of the inner produ ct space and let Th en (a) there is a uniqu e best approximation i. e., a unaqu e to E such that

mo M

II x - mo ll = inf { ll x - m il : m E M} = d(x, M); (b) x - mo M and mo is the only point in M such that x - m o .1 M. Proof. (a) Let d = d(x, M). We want a vector m o E M such that Il x - m o ll = d. We create a sequence of vectors mn E M that almost have this property, then take a limit. Since d(x, M) is an infimum, for each n E N there exists mn E M such that d � lI x - mn ll < d + 1/n. (3.1) Since the m n are all close t o x, they are close t o each other; indeed (mn) is Cauchy. To verify this, consider a "parallelogram" with sides x - mn and x - m k and diagonals 2x - mn - m k and m k - mn , the sum and the difference. It follows from the parallelogram law (1.4.5) that 2 11 x - mn ll 2 + 2 11 x - m k l1 2 = 11 2x - mn - m k l1 2 + II m k - mn ll 2 • Since M is a linear subspace, t (mn + m k ) E M so, for any n and k, .1

II x - mn ll < d + l/n, I I x - mn ll 2 < d2 + 2d/n + 1/n22 . For c > 0 we 2d/n < c/2 and 1/n < c/2, so that 2 2 n n, k � N , d II x - mn ll � + c lI (mk - mn) 1I 2 � ' (2 d2 + 2c) + (2d2 + 2 c ) - 4d2 = 4c . It follows that (mn) is Cauchy. Since M is complete , there exists mo E M such that mn � mo. By the continuity of the norm it follows from inequality (3.1) that II x - mo ll = d. Suppose m E M also satisfies II x - m il = d. Since t (m o + m) E M, the parallelogram law implies that 2 11 x - mo l1 2 + 2 11 x - m l1 2 - 11 2x - mo - m ll 2 11 m - mo ll 2 = 2d2 + 2d2 _ 4 1 I x - t (mo + m) 11 2 � 4d2 - 4d2 = 0, Since can choose N to simultaneously satisfy for � N. Therefore , for

93

3. Bases

m m o. m

from which it follows that = (b) For any nonzero point of M and any nonzero scalar

B y expanding the left-hand term and using the fact that follows, for all nonzero t, that

I t I 2 II m il 2 - 2 Ret- (x - m o , m» 0 Suppose that (x - m o , m ) =p 0 and let t = (m, x - mo)

real. The above inequality becomes

t,

II x - mo ll = d, it

8,

where

8

=p 0 is

which implies that

8 2 ! ! m ll 2 - 28 > 0 for all real nonzero 8. For 8 = 1 , then II m ll 2 > 2 for any m E M , which is imp ossible . We conclude that (x - mo, m ) = 0, i.e . , that x - mo M . Finally, suppose that m' E M also has the property that x - m' M . Then, for any m E M, (x - mo, m) = 0 = (x - m' , m ) . This implies that (m' - mo, m ) = 0 for every m E M. Therefore, m' - m o m' - mo; this yields the desired uniqueness of mo . 0 1..

1..

1..

More generally, 3 . 1 .3(a) is true for closed convex subsets of uniformly convex Banach spaces (see Exercise for the definition of uniformly convex and Exercise 1 of this section; see Narici and Beckenstein 1 985 , p . 363, ( 1 6 . 1 . 5) , for the more general result) . Hilbert spaces are a special kind of uniformly convex space.

2.2-12

Exercises 3 . 1 1 . UNIQUE VECTO R O F MINIMAL NORM This generalizes the "linear subspace" of 3 . 1 .3 to "convex set" (defined in Exercise 1 .3-8g) . For any complete convex subset of an inner product space

(a) (b)

K X: There is a unique vector w E K of smallest norm, i .e . , such that !! w ll = d (0, K) = inf { ! ! x ll : x E K }. (This is the best approxi mation to 0 from I<. ) For any x E X, there i s a unique vector w E K such that d (x, K) = II x - w I! = inf { ! I x - y ll : y E K}.

94

3. Bases

2.

No BEST ApPROXIMATION Show that

co, x (bn) M Il x - l = d x, 3. L e t (X, II·I D be a normed space with Schauder basis (x n ). For x = l: nEN an x n , show that Il x ll * = SUP n 1I l:�= 1 ai x ill defines a norm and is a closed subspace of the Banach space ( 1 1 · 1 1 00 ) of null sequences. Show that if = (j. M, then there is no m E such that mi ( M).

that 1 1 · 1 1* is stronger than 1 1 · 1 1 ·

Hints l(a) . Imitate the proof of Choose E J{ such that = inf E J{ } and show that is Cauchy by considering the parallelogram of sides and The convexity of J{ implies that ( 1 /2) + E J{ for all n and m.

{ lI x ll : x (x n x m )

3.1.3. Xn (xn) Xn X m .

II xn ll --+ d

of the con 2. To see that M is closed, show that it is the null space n (the series tinuous linear map f Co --+ K, (an) 2 an l: nEN converges absolutely by the comparison test) ; since f is bounded on :

1--+

the unit ball by the Holder inequality (Exercise 1 .6- 1(a) ) , it is con tinuous. Let for n be as above. Let = and for n > Let Show = / that E and --+ from above. Thus, M) If m E is such that show that = < which is contradic tory.

a l:i EN 2 - i bi , an = a � k, an = 0 k. X k (2 k (2 k 1)) (an) - (bn). X k M II x - x kll l a l d (x, � il a l . = (an ) M x - m il � l a l , ll:i EN 2 - bi / II ll: iE N 2 -i (bi - ai ) 1 ::; l:i EN 2 - i I bi - ail l a l , (bn)

3.2

Orthogonal Complements and the Proj ection Theorem

2.10,

X

As discussed in Section a normed space may possess a closed linear subspace M that has no topological complemen t, i.e., no subspace for which X = Ef) (top) . It follows from 3 . 2 .4 that a closed subspace of a Hilbert space is "complemented ."

M N

N

3. Bases

95

Definition 3 . 2 . 1 ORTHO G O NAL COMPLEMENTS

S S1. = {x S

Recall that if is a nonempty subset of an inner product space X , the set E : x 1. is called the ORTHOGONAL CO MPLEMENT or O RTHO CO MPLEMENT of S. is pronounced "S perp." •

•

•

Clearly, other.

S} S1.

0

{O} and the whole space X are orthocomplements of each {O} x R R2 . R3

Rx {O} {(O, On x R i s the R2 . In £2 (Example 1 . 1 .7) the orthocomplement of the subspace M of sequences whose first three entries are 0 is the subspace of sequences whose entries are 0 after the first three . This idea extends to L 2 [0 , 211"] : The orthocomplement of the subspace M of functions that vanish on The " y-axis" is the orthocomplement of the "x-axis" o r any subset thereof i n Likewise, the z-axis orthocomplement in of any subset of the plane

[0, 11"] consists of the functions that vanish on [11", 211"] .

The most basic properties of orthocomplements are the following. 3.2.2 PRO PERTIES O F ORTH O CO MPLEMENTS For any subsets S and of an inner produ ct space X : (a) i s a closed lin ear subspace. (b) C (c) ===? C (d)

S1. S S1.1. . S e T T1. S1. . (S1.1.) 1. = S1. .

T

S1.

Proof. We prove only (a) to illustrate the technique. That is a linear subspace follows immediately from the linearity of the inner product in its first argument : For any scalar a, any x , y and z E

E S1. S, { ax + y , z} = a { x , z} + {y , z} = O. To see that S1. is closed, let ( xn ) be a sequence from S 1. such that Xn x E cl S1. . Then for any z E S, by the continuity of the inner product ->

(Example 2 .2 . 5( b) ) ,

The next result sharpens the result that the only vector perpendicular to every vector is

O.

S

3.2.3 ORTHO C OMPLEMENT OF A DENSE SET If is a dense subset of the inner p roduct space X , then = (The converse is true, too, if X is a Hilbert space (3.2.5).)

S1. {O}.

96

3. Bases

Proof.

y

Let x E 81. . For any r > 0, since 8 is dense, there exists E 8 such that I I x < r. By the Pythagorean relation (equation ( 1 . 12) of Section 1 .4) , -

y ll

Since r is arbitrary, this implies that x = o .

0

The following imp ortant result is a corollary to 3 . 1 .3. 3.2.4 T HE PROJECTION THEOREM If M is a complete subspace of the inner produ ct spa ce, X then: (a) X = M $ M1. (top) . (b) M = M 1. 1. . Most often , we apply this to a elosed subspace M of a Hilbert space.

Proof.

(a) Clearly, M n M1. = {OJ . Let x E X , and let rn a E M be the best approximation to x from M of 3 . 1 .3. Since x E M 1. and -

x=

rna

rna + (x rna) , -

=

M and M 1. are algebraic complements. To finish the proof that X M $ M1. (top) (Definition 2. 10.2) we use the criterion of 2 . 1 0 . 1 and show that the projection on M along M 1. is continuous. In other words, we show that

X i = rni + ni E M $ M1.

�

X = rn + n

This follows from the Pythagorean relation

(b) By 3.2.2(b) , M

C

==>

rni

� rn .

M 1. 1. . Now suppose that x E M 1. 1. and write x =

rn + rn' E M $ M1. by (a) . Since x and rn each belong to the linear subspace M 1.1. it follows that rn ' = x rn E M 1.1. , well. Since rn' E M 1. n M 1.1. , it follows that rn' = O . 0

-

as

The completeness of X is vital in 3.2.5, as shown by Example 3.2.6. 3.2.5 ORTHO CO MPLEMENTS IN HILBERT SPACES space M of a Hilb ert space X (a) M 1. 1. el M . (b) M 1. = { O J if and only if M is dense i n X .

For any lin ear sub

=

Proof.

(a) Since M C M 1.1. and M 1. 1. is elosed, i t follows that e l M C M 1.1. . Since el M is complete , by the projection theorem we may decompose the Hilbert space M 1. 1. = el M $ (el M) 1. , where (el M) 1. is computed within M 1. 1. . Since M C el M, it follows that (el M) 1. C M1. . If x E (el M) 1. , then x E M 1. n M1.1. = {OJ, and M1.1. = el M $ (el M) 1. collapses to M 1. 1. el M .

=

3. Bases

( b) We have already proved the "if" assertion M.L = {o} then l M = M.L.L = {O} ol = X . 0

97

in 3.2.3. Conversely, if

e

In 3 . 1 .3 we showed that if M is a complete subspace of an inner product space then given any x there is a unique best approximation rno E M to x. If M is not complete, the result can fail , even for closed subspaces . If, for example, M M .L.L and x M.L ol - M, then there is no best approximation to x from M by the following argument . As shown in 3 . 1 .3, any such best approximation rn o M is such that x - rno M.L . Since x and rno belong to Mol.L , it follows that x - rno M.Lol . This leads to the contradictory result x - rno = Closed subspaces M such that M f. M.L.L are given in Example 3.2.6 and Exercise 9. The following example demonstrates the necessity of having a Hilbert space in 3 . 1 .3, 3 .2.4, and 3.2.5.

X,

E X, f.

E

E

E

E

0.

Example 3.2.6 No BEST ApPROXIMATION

X

There is an incomplete inner product space which has a closed proper subspace M such that (a) There is no best approximation in M to any x - M. (b) cl M M f. M.Lol . (c) M.L {O} but M is not dense in

EX

= =

X. 2 Discussion . Consider z = (1/i ) E £2 and let X be the subspace of £ 2 of sequences that are eventually 0. Since is dense in £2 ( the truncated sequences of any in £ 2 belong to and converge to x in £ 2 ), it is incom plete. Since the inner product is linear in the first argument, the nontrivial cp

x

cp

map

is linear . It is bounded on the unit ball of

n , and consider the vectors

=

i

with

i2

f.

=

Wi = ( 0, 0, . . , i2 , .

. , O, - (n + 1) 2 , 0, 0, . . . .

appearing in the ith position and

( + 1 )st position. Since n

.

ai =

ai E

) , 1 � i � n,

- (n + 1 ) 2

2 ! (Wi ) = ( Wi ' Z) = �Zi2 - (n(n ++ 1)1) 2 = 0,

appearing i n the

98

3. Bases

it is clear that each

Wi E M. Since

Since this implies that each

y

.1.. M, it follows that

ai = 0 it follows that

y

= o.

0

If M is a nontrivial closed subspace of a Hilbert space X , then X = (top) , so any vector in X has a unique representation of the form = + n where E and n E M by the projection theorem (3.2.4) . Since X M EEl (top) , we know that the projection on M along M 1. is a continuous linear map . This special type of projection is called the O RTHOGONAL PROJECTION ON M. It is easy to see that the range of is M and that if is applied twice, nothing happens the second time: for every E X . The property is called IDEMP O TEN CE . Let I denote the identity map 1-+ of X onto X . Since (I = n, I is seen to be the orthogonal projection on An orthogonal projection on a closed subspace of a Hilb ert space behaves like an orthogonal proj ection in in several imp ortant respects. A con sequence of 3.2. 7( c) is that the orthogonal projection on a one-dimensional space is just z z

M EEl M 1.

x m

x m M M1.

=

1.

P P P(Px) = p 2 x = Px -P -P)x = x-Px

x

PM

x x

p2 = P

M1. .

R3 ( x, / II z ll ) / II z ll .

[z]

3.2.7 ORTHO GONAL PROJECTION BEST ApPROXIMATION Let M be a closed lin ear su bspace of the HilbfOrt space X , let be the orthogonal pro iection on M , and let E X. Then: (a) .1.. M, and is the only vector with this property. (b) is the best approximation to x from M (see Definition 3. 1. 1): M ) and for all =f:. < in M. } (c) ORTHO G ONAL PRO JECTION AND INNER PRO DUCT Let ... be an orthonormal set of vect ors in X. Let = . . , n ] . (Since any finit e- dimensional su bspace of a normed space is closed, M is closed.) Then the orthogonal proiection P on M is given by = [For an infinite-dimensional version, see 3.3 .4( d).]

x - Px Px Il x - Px ll = d (x,

Proof.

x

P

Px m Px II x - Px ll Il x - mil {X l , X 2 , , xn M [X l , X 2 , . x Px :L�== l (x, X i ) X i .

x

x Px M M 1.; m M mo m Px M. Px x m II x - ll II il :L�== [X x - l (x, Xi ) x i l , X 2 , . . . , x n], :L�== l { x, X i } X i . 0

x - Px x-m Px = mo Px

For any E X , = E M 1. . +nE EEl therefore, .1.. M By (3 . 1 .3) the only vector E with the property that is the best approximation to x from M. It follows that and that A direct computation for all =f:. in < = shows that .1.. so, by 3 . 1 .3 again, The context " closed subspace of a Hilb ert space" is imp ortant in Example 3.2.7. In Example 3.2.6, the space

x m

m

M

X

M1.

3. Bases

99

Example 3.2.8 BEST MEAN ApPROXIMATIONS Approxima tion with re spect to in the Hilb ert space is called MEAN APPROXIMATIO N o r APPROXIMATION IN THE MEAN. (a) Sh ow that there is a uniqu e best mean approximation to sin t in by a polynomial of degree ::; (b) What is this approximation ?

L2 [a, b]

1 ·1 2

L2 [0 , 1]

x=

5.

M

5

Sol ution . Clearly, the space of polynomials of degree ::; is of dimension 6. By 1 .5.4 and Example 2.6.2( a) , is a 6-dimensional Hilbert subspace of 1] . By the projection theorem 3.2.4 and 3 .2.7, it follows that the orthogonal projection on M is the best approximation to sin by a polynomial of degree ::; 5 : For all v

L2[0 ,

M

Px t E M, I x - Px II 2 = l 1 Isin t - px(t) 1 2 dt � l 1 I sin t - (t ) 1 2 dt. To see what Px is, use the Gram- Schmidt process 1 .4.3 to convert the polynomials {1, t, . . . , t5} into an orthonormal basis {X l , X 2 . . . , X6} for M. Px is then given by 2:f= l [fo1 Xi (t) sin t dt] Xi . 0 v

Exercises 3 . 2

1 . PRO PERTIES O F ORTHOCO MPLEMENTS For any subsets S and an inner product space X, show that

T of

S C S1.1. . [S]) 1. = S-L . ([S] denotes the linear span of S. ) S C T ===} T1. C S1. . (d) ( S1.1. ) 1. = S1.. 2 . If M and N are linear subspaces of the inner product space X, show that M1.1. + N1. C (M + N) l..L . 3 . WAY TO COMP UTE ORTHOGONAL PRO JECTION Let M = [X l , X 2 , . . . , (a) (b) (cl (c)

l.

xn ] be the Xlinear span of the subset {X l " ' " Xn } of a Hilbert space Px of x on The point of this exercise is to show that one way to do it is by solving a certain set of linear equations. Since X - Px M, it follows that (PX, Xj) = (x , Xj) , j = 1 , 2 , . . . , n. Since Px E M, there must b e scalars that satisfy equation (3 .2) below. It turns out that these are the only scalars that satisfy equation M?

X, and let E X. How do we compute the orthogonal projection

.1

Ci

(3.2) . The imp ortance of this metho d of computing the orthogonal proj ection is illustrated in Exercise 4 below .

100

3. Bases

(a) If the scalars

C 1 , C2 , , Cn satisfy the equations

(t;=1 Ci Xi , Xj ) = ti= l Ci {Xi , Xj} = (X, Xj ) , •

.

•

j

= 1 , 2 , . . . , n,

(3.2) is the orthogonal projection of on

2:7= 1 Ci X i {C 1 , C2 , . . . , cn {X l , X 2 , . . X n {X l , X 2 , Xn Ci = (X, Xi } / (Xi , Xi } , i =

X

then show that M. (b) Show that a solution } of equation (3.2) must exist . Show that it is unique if the vectors . , } are linearly independent ; if } is orthogonal, then show that ..., 1 , 2, . . . , n .

4.

SOLVING THE UNSO LVABLE We apply the results of Exercise 3 to the problem of "solving" incompatible linear equations m

L Cj X ij = bi ,

X ij

j=l

i = 1 , 2, . . . , n, j

= 1 , 2, . . . , m,

(3.3)

Ci

and bi are known scalars, the are unknowns and the where equations are incompatible . For such a system of equations, we use the best mean approximation , since no exact solution is available. Let A be the matrix i 1 , 2, . , n. with column vectors = Equation (3.3) Let x be the column vector (b;) E (n) thus becomes m

(X ij )

Xj (X ij), = = (Kn , 11·11 2 ) ' L CjXj = X. j=l '-2

.

.

(3 .4)

We take as the best solution of equation ( 3. 4 ) the best approximation , the orthogonal projection E M , ], is minimal. We know by Exercise 3a that it suffices because to find that satisfy equation (3 .2) to find the orthogonal projection We illustrate in some special cases.

Px.

l i z - x II 2 Ci

z = Px = 2:7=1 Cj Xj

= [X l , . . . x n

(a) Find the best solution z to the system of equations

a + 2b 2a + 4b

=

and compute the percentage error

X= ( ! ).

1, 3,

(3.5)

li z - x I1 2 / l i x il

x

100, where

(b) Find the best solution z to the system of equations

2a + 3b C a + b + 3c 3a + 4b + 2c

5,

9,

15.

(3 .6)

Compute the percentage error l i z

UJ xl, X , . . . , Xn

-

x II 2 I II x l!

x

3. Bases

101

1 00,

x=

where

2 = [X , X , . . . , Xn]. a , . . , an l l 2 X . . . , n? ( x, Xj) = aj

belong to the Hilbert space X and consider the 5. Let subspace M be scalars. When is . Let for j = 1 , 2 , there E X such that As we discuss below, when there is such an x, there is one whose norm is minimal .

( a)

Show that if the set is orthogonal, then "'n .·_ l .... { a ....=-r is a solution to the system. � - \X" Z.l

{X l , . . . , xn}

Xi

x=

{X I , . . . , X n } aI , (c) Suppose that for scalars a I , . . . , a n there exists x E X such that (x, X i ) = ai , for i = 1 , 2, . . . , n . Show that the orthogonal pro jection of onto M is the vector z of minimal norm such that ( z, Xj) = aj for j = 1 , 2, . . . , n. 6 . BEST MEAN ApPROXIMATIO N TO t 4 Use the technique of Example 3.2.8 to compute the best mean (i.e . , 11 · 11 2 ) approximation to x = t 4 by a polynomial y of degree :::; 3 . Sketch your solution and see whether

(b) Show that if the vectors are linearly dependent , then there exist scalars . . . , a n such that there is no solution x.

x

it looks like a good approximation .

N

7. SUMS O F COMP LETE SUBSPACES If M and are complete orthog onal subspaces of a Hilbert space X , then (a) M + is closed; (b) the orthogonal projection PM + N PM + PN .

=

N

8 . SUMS O F CLOSED SUBSPACES Unlike what happens in a Hilbert space, as in the preceding exercise, the sum of closed orthogonal subspaces of an inner product space need not be closed. Let (0, 0, . . . , 0, 1 , 0 , . . . ) , E N , be the standard basis vectors for let . . ] . Let M be the linear sub ( l/i), and let X space of X whose odd entries are 0, and N the subspace whose even entries are 0. Show that M and N are closed orthogonal subspaces of X but that M + N is not closed.

i = [x, e l , e 2 , . . . , e n ,

x=

ei = £2 ,

.

9 . Let cp be the space of finite sequences of Example 3.2.6 endowed with the inner product of Let

£2 .

=

Show that M1. cpo By Exercise 2.4-8(b) , M is closed in therefore a closed subspace for which M =P M

1.1. .

cp o

M is

102

3. Bases

Hints 3(a) . Show that x - z

{X l , X 2 , , n

1. M and use the result of 3.2.7.

3 ( b) If x } is linearly independent, then there is only one way to write the orthogonal projection x on M. .

4(a) .

.

•

•

P = E?=l Ci X i The system has no solution because 2a + 4b = 2 (a + 2b) but 3 =J 2 · 1 .

In vector form the system may be written

By equation (3.2) , we want to find scalars a, b such that

and

Since the second equation is merely double the first, it suffices to solve the first one. To get a solution, set b 0 to get a = 7/5. Thus, the orthogonal projection of

(!)

=

on M

=

[ ( ; ) , ( � )]

=

[( � )] ( ; ) + O · ( � ) = ( ( ! ) , � ( ; ) ) (� ( � ) ) , (i) is given by

( 7/5)

where the term on the right is more obviously the projection of on M. The percentage error is 14.1%.

{X I , . . , X m } is a maximal linearly independent subset of {X l , " " xn }. Then,. for each k > m, X k can be written as a linear combination of {X l , . , x m } . Therefore , for k > m, each a k can be written as a linear combination of { a I , . . . , a m } . Choose a k so that it

5 ( b) . Suppose

.

.

.

violates this latter condition.

5(c) . Px and ( 1 - P) x denote the orthogonal projections on M and M l. , respectively. Show that V [xl + M l. contains all vectors such that j = 1 , 2, . . , n . By 3 . 1 . 3 and Example 3.2.7, x (1 = Px is the vector of minimal norm in V.

= {z, Xj} = aj , - P) X By 3 . 1 .3, PM + N X is the only vector such that X - PM + N X Show that X - PM X - PNx M + N.

z

.

7.

1.

1.

M+

N.

3. Bases

3.3

103

Orthonormal Sequences

In R3 one routinely expresses a vector as the vector sum of its projections onto the standard unit basis vectors i, j, k, along the coordinate axes . The standard unit basis vectors may be replaced by an orthonormal sequence in many imp ortant spaces �uch as [ - 11", 11"] . We consider the basic properties of orthonormal sequences in this section. We considered Bessel 's equality and inequality in finite-dimensional spaces in Section 1 .4 and Exercise 1 .4-6 .

L2

(xn ) bein anX theorthonor series

3.3.1 BES SEL'S EQU ALITY A N D INEQU ALITY L e t mal sequence in an inner produ ct space X. For a n y converges and satisfies

x

LneN I{ x, Xn ) 1 2 (3.7) L: I{ x, X n ) 1 2 :::; I x II 2 (Bessel's inequality) , neN and for convergent L neN (x , X n ) X n , 2 x - L: (x, Xn ) Xn + L: I{x, Xn )1 2 = I x I 2 (Bessel'S equality) , neN neN (3.8)

) x = L: (X, Xn ) Xn L: l{x,xn ) 1 2 = I x l 2 ( Parseval's entity n eN neN

Moreover,

<==>

.d

1

.

.

(3.9)

The last equality is one version of Parse val's identity or PARSEVA L'S EQUAL ITY ; another version is given in 3.3.4(f) .

by Example , . ' " X(xn . XForXany. TheX, Pythagorean X l , X 2L?: x- L?=l (x, Xi ) Xi i) i , l II x I 2 Il x - L:;= l (x, Xi ) Xi 1 22 + 1 L:: 1 (X, Xi ) X i l 1 2 Il x - L:;= l (x, Xi )2Xi 1 + L:;= l I { x, xi) 1 2 > I L;= l {X,Xi) Xi I1 = L:;=1 1{x,Xi) 1 2 , which yields the convergence of L nEN l{ x, x n ) 1 and Bessel 's inequality. If Ln e N (x, Xn ) Xn = x, then we may utilize the continuity of the norm and take limits on the second line to obtain the Bessel equality ; equation (3.9) follows immediately from equation (3.8) . If, conversely, L I{ x, Xn ) 1 2 Xn = I x I 2 , n EN

Proof.

Consider the first n vectors 3.2.7, is orthogonal to relation therefore implies that for any n,

104

3 . Bases

then taking limits on the second line shows that

x = L nEN (x, Xn) Xn. 0

Example 3.3.2 CLA SSICAL FO URIER COEFFICIENTS G O TO 0

The sequen ces

(Xn) = CJ;t ) and ( Yn) = ( s�t ) are orthonormal sequences in L [-11", 11"] . The FOURIER COEFFICIENTS of x E L 2 [-1I", 11"] are ao = ;: f�1< x(t)2 dt, an = -11"1 1 1< X(t ) cos nt dt = 1 (x, x n) , n E N , _ 1< and 2 bn 11"1 1 1< x (t) sm. nt dt = 1 (x, Yn) , n E N. r;;; y 1l"

=

By Bessel's inequality,

li�

r;;; y 1l"

-

0

1: x(t) nt dt sin

=

0

and li�

11< x(t) cos nt dt = 0 , 1<

so an , bn 0, a fact known as the RIEMANN-LEBESGUE LEMMA; for a much more general version see 4.4. 1 . 0 �

Our next result illustrates the strong similarity between orthonormal sequences and the usual basis vectors in R3 . A big differ ence is that not every vector in X can necessarily be represented as x LnEN (x, xn) Xn -only those in c l [X l, X 2 . . . , Xn, . . .] can. 3.3.4 ORTHO N O RMAL SEQUENCES IN HILBERT SPACES For any or thonormal sequence (Xn) in a Hilbert space X : ( a) ( RIESZ-FIS CHER) Ln EN anxn converges if and only if L n E N l an l 2 < 00 , i. e., (an) E 1.2 • This is equivalent to: For any orthogonal sequence ( Yn ), Ln E N Yn converges if and only if Ln EN ll Yn 11 2 < 00 . (b) If L n EN anxn = x, then an = (x, xn) for all n. (This is true in any inner product space.) It follows that the coefficients an are uniqu e. ( c ) For all x E X, the series Ln N (x, xn) Xn = Y converges-not nec essarily to x, but to something in cl [(xn ) ] = cl [X l, X 2 , " " X n , . . .]. (d) For all x E X, x - Ln ( x, Xn) Xn J.. cl [( xn )) so Ln EN (x, Xn) Xn = 3. 2. 7. Px, the orthogonal projectionENof x on M = cl[(x n)), by Example (e ) If x E cl [(xn )) , then x = LnEN (x, xn) Xn. (f) PARSEVAL'S IDENTITY ( equation (3.9) is also known as Parseval 's identity) . For convergent series x = L n EN anxn = L nEN (x, xn) Xn and Y = LnEN bnxn = Ln EN ( y, xn) Xn, Example 3.3.3

=

(x, y) = L anbn = L (x, xn) ( y, xn) nEN nEN

(Parseval's identity) .

(3.10)

3. Bases

105

( g) PYTHAG OREAN THEOREM If(Yn) is an orthogonal sequence of nonzero L n e N Yn, then (3. 1 1 ) II x II 2 L ll Yn 11 2 . neN Proof. ( a) Let Sn = L 7= 1 ai X i· The sequence ( L7= 1 l ail 2 ) is Cauchy if and only if (sn) is Cauchy since n 2 = sn s II m ll =L l ai l 2 . i m+ l ( b) Let Sn L7= 1 ai Xi . For any n > j , {Xj , sn } = aj so limn {Xj , sn} = aj . By the continuity of the inner product , it follows that {Xj , x} aj for each j. ( c ) By 3 .3 . 1 , LneN I {x, Xn} 1 2 converges in any inner product space. Therefore , L neN {x, Xn} Xn converges by ( a ) . As a limit of vectors in [(Xn)], this sum belongs to cl[ ( xn )) . ( d) It is trivial to verify that if x S, then x cl S. Therefore , it suffices to show that x - L n e N ( x, xn) Xn [(X n)). This follows from the fact that for any j, using the Kronecker delta 6n j , vectors and x =

=

=

=

..L

.1

.1

( e ) If x E cl [( xn )] , then by ( d ) , Px = x Ln eN { x, xn} xn. (f) Let x = Ln e N anXn and Y = L ke N b k x k . It follows from the mutual orthogonality of the (X n ) and the continuity of the inner product that =

To get the previous version, equation (3.9) , of the Parse val identity, let

x. (g ) Rewrite L n e N Yn as Ln eN ( II Yn ID ( Yn/ ll Yn I! ) Since ( Yn/ ll Yn II ) is an orthonormal sequence, it follows from (b) that {x, Yn/ II Yn ll} = II Yn l! ' n E N . It follows from (f ) that II x I1 2 = L ll Yn 11 2 . 0 n eN y =

.

106

3. Bases

Exercises 3 . 3

1.

(xn) is an orthonormal sequence from L 2 [0, 2 ], 1r

CHANGE O F SCALE If show that the functions

are orthonormal on

[0, b).

(xn) and conx Ln EN anXn Y Ln EN bn xn in a Hilb ert (x, y) = L anbn n EN by Parseval's identity 3.3.4 ( f) . Show that Ln EN anbn converges ab

2. ABS OLUTE PARSEVAL For an orthonormal sequence vergent series and = space

solutely.

x = Ln EN Xn and ( y, Xn) = 0 for E N, ( y, x ) 4. Let (xn) be a sequence of vectors in a Hilbert space X whose linear span [(xn)) is all of X . Show that X is finite-dimensional. 5 . Let (xn ) be an orthonormal sequence in an inner product space X . ( a) For any > 0, show that the set { Xn : II x II 2 < f I (x, X n) 1 2 } 3. In any inner product space, if every n show that = O.

finite.

{

IS

( b) Improve the result of (a) . Show that for any k E N the set

has at most

k elements .

Ld -1r, x E L 2 [-1r, ] x E Ld ] 7. If {X l , X 2 , . . . , Xn} is an orthonormal set in L 2 [a, b), show that Yij ( t) = Xi ( 8 ) Yj (t) , i, j E N , is an orthonormal set in L 2 ([a, b) [a, b]) .

1r) We know that the Fourier 6 . RIEMANN-LEBESGUE LEMMA FOR 1r go to 0 by Example 3.3.2. Show that the coefficients of same result holds for - 1r , 1r . x

8,

3. Bases

107

Hints 2. Replace

an and bn by l an l and I bn l .

4. Use the Gram-Schmidt process ( 1 .4.3) to orthornormalize an infinite linearly independent subset of to get an orthonormal sequence ( Yn ) , and consider

(xn) LnEN { l /n)Yn.

5 . Use 3 .3 . 1 for both parts. 6 . The step functions are dense in

Ld - 1r, 1r]. Or peek ahead to 4 .4 . 1 .

7. Write the double integral as an iterated integral.

3.4

Orthonormal Bases

Let X be an inner product space. We now know that orthonormal sequences in X behave much like the usual basis vectors in in that we may write for many vectors x E X. The defect is that there x = (x , may not be sufficiently many for every vector to be expressible in terms of them. The notion we consider next characterizes orthonormal sets that are large enough to make this representation universally possible . Orthonormal sets that are not properly contained in any other or thonormal set are special. Such a maximal orthonormal set is called an O RTHONORMAL BASE ( BASIS ) or a COMPLETE ORTHONORMAL SET . We prove in 3 .4.8 that orthonormal bases in separable Hilbert spaces are the infinite-dimensional analogue of the standard basis vectors in If A is an orthonormal sequence but not an orthonormal base , then it is properly contained in another orthonormal set, so there must be some unit {x d maximal? If not , there must be Is vector such that some unit vector such that { x d . An inductive style argument like this, using Zorn's lemma ( Bachman and Narici 1966, pp. 149-150) , shows that

L nEN Xn) Xn

R3

Xn

S

Rn .

Xl

• •

•

X2

X l J.. A . A u X 2 J.. A u

Orthonormal bases exist in any inner product space. Any orthonormal subset is a subset of an orthonormal base . We also say that an orthonormal subset can be extended to an orthonormal base . All orthonormal bases are of the same cardinality ( Bachman and N ar ici 1966, p . 166) . This common cardinality is called the ORTH O G O N A L DIMENSION of the space; it is the same as the lin ear dimension, the cardinality of any Hamel base , only in finite-dimensional spaces .

108

3. Bases

3.4.1 ORTHO N O RMAL BASES IN INNER PRODUCT SPACES A n orthonor mal subset S of an inner product space X is an orthonormal base (a) if and only if SJ. = {O}. (b) if the lin ear span [5'] is dense in X.

Proof.

S

{O} ,

{O}.

(a) If J. =p there exists x E SJ. Since S is properly contained in the orthonormal set S , S is not an orthonormal base. Conversely, if SJ. = then S is not a proper subset of another orthonormal set. (b) Suppose that is dense in X. If x 1. clearly then .1 . Since is dense in X, there is a sequence from that converges to x. Since .1 n = 0 for every n. By the continuity of the inner product, = Thus = 0 , and the completeness it follows that 0 = limn follows from (a) . 0 -

u {xl I Ix ll }

{O},

[S] x [5'] , ( x, x )

[S]

( x,

(x n ) xn) II x 1l 2 .

S, [5'] x

x [5']

In a Hilb ert space X, cl = X constitutes a criterion for completeness of orthonormal sets S. We prove it for denumerable S below .

[5']

3.4.2 COMP LETE ORTHONORMAL SEQUENCES IN HILBERT SPACES A n orthonormal sequence (xn) i n a Hilbert space X is a n orthonormal base if and only if any one of the equivalent conditions below is satisfied. (a) cl [(xn)] = cl [X l X 2 . . . , Xn, .] = X .

(b) = (c) The Parseval identity = for every (d) = ,

.

.

[(xn )]J. {O}. II x I1 2 � n E N I (x, Xn) 1 2 holds for all x E X. x E X. x �n E N (x, xn) Xn Proof. (a) If cl [( xn )] = X , then (Xn) is an orthonormal basis by 3 .4. 1(b). Conversely, if (x n ) is an orthonormal basis, let M = [X l ,X 2 . . . , X n , . . .]. Then M J. = {O}. Therefore , by 3.2.5, cl M = MJ.J. = X . (b) A linear subspace M of a Hilbert space is dense if and only if M J. = {O} by 3.2.5(b) . Therefore cl[(xn )] = X if and only if [(xn)]J. = {O}. ( c ) Assume that Parseval's identity holds for all x E X . I f x [(xn)], then II x II 2 = L I (x, Xn )1 2 = 0 n EN s o X = 0 and (Xn) i s an orthonormal base b y (b) . Conversely, i f (x n ) i s an orthonormal base , then cl [ ( xn )] = X by (a) . Therefore Parseval's identity holds by 3 .3 .4(e, f) . (d) The validity of Parseval's identity for all x E X is equivalent to the ability to write every x = � n EN (x, xn) Xn by 3.3. 1 ; the desired result follows from (c) . 0 1.

We utilize the criteria of the previous theorem in the next two examples.

3. Bases

109

Example 3 .4.3 RADEMACHER FUN CTIO NS NOT A BASE

(1'n), 0, L 2 [0, ] 1 [0, 1] ' 1'1 1 [0, 1'2 [0, [�, 1]. 1'3 , [0, 1]

The functions n � from I are known as the RADEMACHER = on FUNCTION S : = on t) and - I on Next, divide into four equal parts and define = I on t ) and [t , �), and To get = - I on [ t , t ) and divide into eighths and let be on the first eighth, - I on the second, and so on. A compact description is afforded by means of the SIGNUM FUN CTI O N sgn t , which is for � ° and for < O . The Rademacher functions are then given by = sgn ( sin 2 7r , ° � �

1'2

[0, 1]

1'0

1

1

t 1'n (t)

-1

t n t)

[t , l].

1'3

t 1.

05 ·0.

-0 5 . -1 .

The Rademacher function

r2

The Rademacher functions are an orthonormal set but not an orthonor = To see that the mal base: Clearly, are mutually orthogo and n, p E N . On any subinterval nal, consider k 2 is constant , on which must change sign an even number of times. The integral of the product on all such subintervals is therefore = To see that they are incomplete, and it follows that we use the criterion of 3 .4.2( d) and exhibit a function x E for which x i= (x , Consider the function x that rises linearly from o to on then falls linearly from to ° on Now consider and Since the First, compare what happens on integrands represent triangles, we can compute the areas by geometry :

1'n f01 1'; (t) dt 1. 1'n 1'n +p , [(k - 1) /2n , / n ) 1'n +p 1'n 1'n 1'n p 1'( n , 1'n +p ) + 0. 0, L 2 [0, 1] 1' 1' Ln>o n) n· 1 [0, f] 1 [t, l]. 1 [0, i] [� , 1] . f0 X(t)1'2 (t) dt. 1/4 [Jo X(t)1'2 (t) dt 2"1 · 41 · 41 ' while 1 1 1 1 13/ 4 X(t)1'2 (t) dt = - -2 . -4 . -4 , =

110

3 . Bases

clude that

I; 1 . Thus ,

o for all n �

I1 /; x(t)r2 (t) dt

- I:/24 x(t)r2 (t) dt . We con x(t)r2 (t) dt = O. The same idea shows that x(t)rn (t) dt

so their sum is O. Similarly,

L ( x , rn) rn =

n ?O

=

[1 1 x(t)ro (t) dt] ro � ro 0

=

I01

=

=

�.

Since Ln>o ( x , rn ) rn is constant, it is certainly not equal to x; the Rademacher functions are therefore not an orthonormal base by 3.4.2( d) .

0

The following result looks like a strong form of continuity. Its corollary 3 .4.5 is that even if Parseval's identity holds only for all x in a dense subset of a Hilbert space, then the orthonormal sequence (xn ) is an orthonormal basis. 3 .4.4 PA SSAGE TO THE LIMIT Let (xn ) be an orthonormal sequence in the inner product space X . If Yn -- Y and

Yn

L (Yn , Xk) xk kEN

=

for every n, then Y=

L (y, Xk) Xk. kEN

Proof. Given € > 0 , choose N such that l lYn - y ll < € / 3 for n � N. By hypothesis YN = L k E N (YN , Xk ) Xk , so there exists M such that n

L (YN , Xi) x ; - YN i= 1

< €/3 for n � M.

By the triangle inequality,

I L�= l (y , Xi) Xi - L�= l (YN , Xi) Xi I + II L�= l (YN , Xi) Xi - YN I I + l l YN - Y I I · (3. 12) By the orthonormality of the Xi and the Bessel inequality of 3 .3 . 1 we have I L�= l (y, Xi) Xi - y l n

<

L (y - YN , Xi) Xi

i= l

2

n

=

I { y - YN , x ; } 1 2 ::; l I y - YN I I 2 , L ;=

1

so the first term in inequality (3. 12) is also less than €/3.

0

111

3. Bases

3.4.5 PARSEVAL O N A DENSE SUBSET Let (x n ) b e a n orthonormal se quence in a Hilbert space X. If Parseval 's identity, equation (3. 9) of 3. 3. 1,

lI y ll 2 L I {y, x k } 1 2 , keN =

or, equivalently,

Y = L (y, X k ) x k , keN

holds for each y in a dense subset D of X , then basis for X.

Proof.

(xn) an IS

orthon ormal

We use the criteria of 3.4.2(b, c) to show that Parseval's identity holds for all in X . Let E D be such that -+ By hypothesis

Yn y. Yn Yn L (Yn, X k ) X k ke N for every n, so the desired result follows from 3.4.4. 0 y

=

y'2;i) , The density criterion for completeness of 3 .4.2( a) implies that n E Z, is an orthonormal base for L 2 [ - 1I" 1I" , as outlined in Example 3 .4.6{b) .

( e i nt /

, )

Example 3 .4.6 ORTHO NORMAL BASES FOR

(a) STANDARD BASIS IN

ei

=

£2 ( n )

£2 AND L 2 [ - 1I", 1I"]

The standard basis vectors

(0, 0 , . . . , 0, 1 , 0, . . . , 0) , 1 ::; i ::;

n,

Kn , N, { a e ) ai . x

with a 1 i n the ith position , are an orthonormal base for as are the sequences = (0 , 0, . . . , 0, 1 , 0 , . . . ), i E for £ 2 . For any = E £2 (n) , Thus, if is ::; n ::; 00 , ( j ) , i = orthogonal to the linear span [( then = 0 . /V2i) , n E Z, is an orthonormal basis for L 2 [ - 1I", 11"] . (b) Orthogonality follows from the fact that for n i m ,

ei x (aj ) (e i nt

1 ei )],

111' ei(n - m)t dt - 11'

=

z

.

n - m )t l 1l' i( e _ (n - m) 1

1I'

=

0.

[(e i nt /V2i»), n E N , of

As noted in Example 2.4.5(c) , the linear subspace trigonometric polynomials n E is dense in the desired result follows from 3 .4.2(a) . (c) Essentially, the facts used in (b) prove that

L:�= _n a k e i k t ,

N,

x

L 2 [-1I", 1I"] , so

, ( / J1i) sin t, ( 1 / J1i) sin 2t, . . . , (1/ J1i) cos t, (1/ J1i) cos 2t , . . . is also an orthonormal base for real L 2 [ - 1I" , ] 0 /

1 .,f2; 1

11" .

Example 3 .4.7 demonstrates the fragile nature of completeness for or thonormal sets.

112

3 . Bases

Example 3.4.7 REMOVE ONE-NoT A BASE

Consider

S = { sin nt j.fi : n 2: 2 } U { cos nt j.fi : n 2: 1 } u {lj.J2;} L 2 [-'II" , 'II"] . Since sin t S, it follows that S1. # {O} and S is a not an orthonormal base. 0 C

.1

In a separable inner product space, orthonormal bases are countable. 3.4.8 SEPARABLE <=> COMPLETE ORTHONORMAL SEQUENCE a An inner product space X is separable if and only if it has a complete

()

(xn), i. e., a sequence (xn) that is an orthonormal (b ) ( FO URIER SERIES ) In a separable Hilbert space any x E X can be

orthonormal sequence base.

written uniqu ely in the form

x = L (x, Xn) Xn n EN for any orthonormal basis (xn). The values (x, xn) are called the FO URIER C O EFFICIENTS of x. The series E nEN (x, x n) Xn is called the FO URIER for x. Thus, in separable Hilbert spaces we recover the ability to write a vector as the "sum of its projections "on the basis vectors.

SERIES

11

11

Proof. ( a) If (Xn) is an orthonormal base , then linear combinations of the Xn with rational or Gaussian rational (a + bi, where a and b are rational) coefficients constitute a countable dense subset of X . Conversely, if X is sep arable , let {x n } be a countable dense subset . Let { Yn } be the sequence ob tained from the {x n } by the Gram-Schmidt process ( 1 .4.3 ) . Since the linear spans [{X n }] and [{ Yn }] are equal, it follows that cl [{yn }] = cl [{x n }] = X j therefore , {yn} is an orthonormal basis b y 3.4 . 1 ( b ) . (b ) Let X be a separable Hilbert space. By ( a) there exists an or thonormal basis (xn) in X . By 3.4.2 ( a) , cl [(xn)] X . By 3 .3.4 ( e ) , x EnEN (x, xn) xn · The uniqueness of the coefficients (x, Xn) follows from 3.3.4 ( b ) . 0 We characterized finite-dimensional Hilbert spaces X over K in 1 .5 .4: Essentially, K n is the only one. We chose an orthonormal basis X l, X 2 , . . . , Xn for X , then showed that the map E � 1 a i X i (ai ) was a linear isometry of X onto K n . We use a similar technique in the corollary below . As a consequence , a function x from L 2 [ - 'II" , '11"] may be characterized by "digital information ," namely the sequence «( x, x n) ). =

=

1-+

Corollary 3 .4.9 SEPARABLE HILBERT SPA CE � '- 2 Any infinite dimensional separable Hilbert space X is inner product isomorphic

to '-2 .

3. Bases

Proof.

£2

113

B y 1 .5 .3 w e only have t o show that X and are norm isomorphic. By 3 .4.8(a) , it follows that X has an orthonormal basis By 3 .4.8(b) it follows that any E X can be written in the form = Consider the map

x

A:

X

--

X = LnEN (X, Xn} Xn

1---+

(x n ). x LnEN (x, xn) xn.

£2 ( (X, xn})

The linearity of the inner product in the first argument yields the linearity of A. If = 0, then Since ° for every n E so 1.. is an orthonormal base , it follows that = 0, and A is seen to be injective. To see that A is surjective, suppose that < 00. By the Riesz- Fischer theorem, 3 .3.4(a) , converges to some x; by so 3.3.4(b) , a = = By the Parseval identity, equation (3.9) of 3 .3 . 1 , it follows by 1 .5.3 that

(xn)

N , x [(x n)]. x LnEN I an 1 2 L n eN anxn n ( x, x n ), Ax (an ).

Ax

(x, xn) =

II A x lI; = 1I ((x, xn}) II � = L l (x, xn } 1 2 = II x 11 2 . 0 n EN Exercises 3 . 4

1.

ORTHO N O RMAL BASES FOR - 11", 11"] Which of the following are or thonormal bases for -11", 11"] (a) L �= - n ak cos kt, n E N . (b) �= - ak sin kt , n E (c) L �=_ n (a k cos kt + b k sin kt) , n E

C[ , I H I 2 )?

(C [

N.

L n

N.

2. ORTHO N O RMAL BASES FOR 11"] Show that each of the sequences (cos nt) , n E U {O} , and {sin nt} , n E suitably normalized, are orthonormal bases for real 11" . Note : 11"] , not 11" .

N

L 2 [0, L 2 [0, ]

N, L 2 [0, L 2 [-1I", ] 3 . Show that any n orthonormal vectors in Rn (n E N ) form an or thonormal base .

(xn)

be an orthonor 4. CONTINUITY O F FOURIER COEFFICIENTS Let mal sequence in a Hilbert space X . Show that each of the C O EF FICIENT FUNCTIONALS = x, defines a continuous linear functional on X. (Hint: Cauchy-Schwarz .)

In (x) ( x n)

5 . DIRECT SUMS O F ORTHONORMAL BASES Recall (Section 2 . 1 0) that the direct sum X Ef) Y of Hilbert spaces X and Y is a Hilbert space. If are countable orthonormal bases for X and Y, re and 0) , (0, spectively, show that } is an orthonormal basis for

(x n)

X Ef) Y .

( Yn )

{(xn '

Yn)

114

3. Bases

x E L 2 [-7I", 71"] , show that J�'If I x (t) 1 2 dt Ln eZ 2� I J�'If x (t)2 e - int dt l 2 2 2� I J�'If I x (t) 1 2 dt l + L2 n eN � I J�'If x (t) cos nt dt l + � 1 f"'If x (t ) sin nt dt I .

6 . For any

Hints Suppose that E L 2 [0, 7I"] is orthogonal to all terms of the cosine sequence. Extend to L 2 [-7I", 71"] by defining Show that the extended function is orthogonal to both the cosine and the sine sequences.

2.

3.5

x

x

x (-t) = x (t).

The Haar Basis

The standard Fourier coefficients

'1f

an = -71"1 1 x(t) cos nt dt, n E NU {O} , and bn

1

1 k x(t) cos n t dt , n E N, 71" involve integrals of products of the function x E L 2 [-7I", 71"] with sines and - 'If

= -

a

cosines. As the reader undoubtedly knows, such integrals can be difficult to compute . One gets simpler integrals using Haar functions (Definition 3 .5 . 1 ) . Though this simplicity is desirable, the Haar functions are discontinuous; so ultimately, we find ourselves in the strange situation of expressing a continuous function in terms of discontinuous ones . We show in this section that the Haar functions form an orthonormal basis for L 2 [0, 1], a fact that we shall use in Chapter 7 on wavelets. We remind the reader that elements of L 2 [0, 1] that differ on a set of Lebesgue measure ° are considered to be the same. The function 9 = f i� called a DILATION O F the function f by The first Haar function that we introduce is

° is to convert the cozero set [ 0, 1 ) of

(t)

(bt)

l ib).

(t) =

(t)

b.

(t),

Ib):

(bt

(bt - k) = l [k / b ,(k+l)/ b) (t) .

Thus,

(t) =

(the scaling identity)

(3 . 13)

3. Bases

The MOTHER WAVELET (see Section 7.3)

1P (t) = t,b (2t) - t,b (2t - 1 ) = f ( x ) =

looks like a truncated sine wave.

{ �1,

115

[O, i) [i,

on - I on 1) elsewhere

0,

o.� o�

-0 . 2

0.2

0.4

0.6

0.8

1 .2

-0 . 5

The Haar Mother Wavelet 'ljJ

Definition 3 . 5 . 1 HAAR FUNCTIO NS

The mother wavelet

1P generates

We refer to j as the G ENERATION of 1Pj,k . The collection together with t,b is called the set of HAAR FUNCTIONS. 0

\II

of the 1Pj ,k

j I , we get two functions 1P 1,O (t) = 2 1 /2 1P (2t) and 1P 1,1 (t) 12 /For 2 1P (2t - 1 ) ; 1P 1,O and 1P , 1 have greater amplitude (so that they have norm 1) than 1P but smaller cozero sets (of length i): =

1/; 1 , 0 (t) =

-<>.

{

0.1

1

on [0 , �), Vi -..;2, - l on [ � , i ) , elsewhere, 0,

0.2

0.3

0.4

.,pI,. = 2l/2.,p (2t)

5

0,6

and

1P 1,1 (t) =

0.2

{

on [i , �) , Vi on [ � , -Vi, elsewhere. 0,

-1

0.4

0.6

.,pl,l (t) = 21/2 (2t -

0.8

1)

1 ).

116

3. Bases

j 2,

[0, 1]

At the next generation, = we divide into fourths; on each fourth (half-open interval) we define a function that is on the first eighth and on the second; the amplitudes increase from v'2 to and the cozero sets shrink from t to ! in length : are, respectively, given by , 3 ) . The cozero set of each for any k is of length The number of functions as varies from = ° to n is 1 + + + . . . + are Evidently,
2 2, -2 '1/;2 ,0 , . . . , '1/;2 ,3 2'1/; (4t) 2 '1/; (4t - 1) , 2 '1/; (4t - 2) , 2 '1/; (4t 'l/;i ,k 1/2i . n n 1 'l/;i ,k j 2 4 2 2 + - 1. 'l/;j ,k . 'l/; ,k (j, k) 1= (j', k') j 1= j' i 1= k'. j j' 1= k', 'l/;i ,k 'l/;i ,k ' 'l/;j , k 'l/;j , k ' {'I/;j , k , 'I/;j ,k /} f�oo 'l/;j , k 'l/;i ' ,k 1 dt 0. j 1= j', 'l/;i ,k 'l/;j / ,k l 'l/;j , k

L 2 [0, 1].

L 2 [0, 1],

(n2 -i , (n 1) 2 - j ) , jE N , n 0, 1, . . , 2n - 1. L 2 [0, 1] 1974, 71, 2.4.5( [iii] .
is denoted by

'1/; , '1/; 1 , 0 , '1/;1 , 1 , '1/; 2 ,0 , '1/;2 ,1 , '1/;2 , 2 , h I , h 2 ' h3 ,

.

.

.

[0, 1].

.

[\If]

..

•

We index the discontinuity points of a finite collection of them by increasing size . For example , the discontinuities of are at t , i , and we index them not in the order they appear with their functions but in = !, = = t, ascending order as = i , and = When w e add = (which is on the first fourth , 1 on the second, then to the list , we introduce a new point of discontinuity at = �. We reindex the discontinuities as 3 = = 8' = 4' = 2 ' = 4 ' and =

O, ! , h I , h 2 , h3 , h4 , a l 0, a 2 a3 a4 as 1 . 1 0) hs 1/;2 , 0 t a l 0, a 2 1 a3 1 a 4 1 as a6 1. The point of the next two results is to show that any x E C [0, 1] can be l written as a series E nEN bnhn , with bn = fo x (t)hn (t) dt, which is uni n formly convergent to x on [0, 1] except on { k / 2 : ° ::; k ::; 2 n , n = 0, 1, 2, . . . } , 1;

-

3. Bases

0.

117

a set of measure Since uniform convergence almost everywhere on a set of finite measure implies L 2 -convergence, it will follow that is dense in L2 and therefore that W is an orthonormal basis for L 2 by

[w] [0, 1] 3.4.2.

[0, 1] ,

Let a 1, a 2 . . . , an, an + 1 be the discontinuity points of the first n elements h 1 ' h 2 ' . . . , hn of W arranged in ascending order. If s and t are between successive discontinuity points ale , a k + 1 , then

Lemma 3.5.2

otherwise,

Proof.

2:7= 1 hi (t) hi ( ) = 0. s

1, the sum above is just h 1 (t) h 1 ( ) {O, I} = {a l , a 2 }. It is trivial to verify

We proceed by induction. For n = and the points of discontinuity are that

s

hn + 1 '

Let us move on to bigger things. Assume the theorem for n, add in and assume that the cozero set of in the existing (up lies in to set of discontinuity points; inserts a new discontinuity point d right in the middle of If one of 8 , belongs to and the other does not , then ( s ) = Thus,

[alc, a k + 1 ] hn + 1 hn + 1 [ale, a k 1]. t [alc , a k + d 0. hn + 1 (t)+hn + 1 n n +1 L: hi (t) hi ( 8 ) = L: hi (t) hi ( ) ;= 1 i =1

hn)

s

,

and the result follows from the induction hypothesis . Now suppose that both 8 and belong to Further, suppose that s and t are on opposite sides of the midp oint d, i.e . , for example , that < < Assume that = "pm ,j , so that its cozero set is an interval of length Since d is the point at which changes sign , it follows that the product

t

s ::; ak + 1 .

[alc, a k + 1 ]. hn 1 112 m . +

By induction,

so

n+1 L: h ; (t) hi ( ) = 2m - 2m = 0, ; =1 s

hn + 1

ale ::; t d [alc , a k + 1]

,

118

3 . Bases

which is the desired result, since s and t are not between cbnsecutive points of discontinuity of Finally, when s, E [ak , are on the same side of the midp oint d, then hence (s) =

h 1 , h 2 , . . . , hn, hn + 1 .

hn + 1 (t) hn + 1

t

aH 1 ] 2n - 1 ;

[1Ji] C [0, 1] Let x E C [0, 1] and bn = J01 x(t)hn (t) dt, N. Ln EN bnhn converges uniformly to on [0, 1] \ {k/2n : 0 � k � 2n , n = 0, 1, 2, . . . } , the complement of the dyadic points in [0, 1] . Proof. Consider the nth partial sum n b.h. (t) " xn (t) L.Ji= 1 L1�_ 1 ( J(o x(s)hi (s) dS) hi (t) 1 L�= 1 1 x(s)hds ) hdt) ds.

IN 3.5.3 DENSITY O F nE Then the series

x

I

I

Since x is uniformly continuous (being continuous on a closed interval) , then given f > there must exist 6 > such that

0,

It - t' l < 6

==>

0 sup I x (t) - x (t') 1 < f . /t -t' / < 6

1 a . . . , an, an + 1 h l , h 2 , . . . , hn2 h 1 , h2 , , hn

,

Since the discontinuity points a , (arranged i n ascending order) of the elements of IJi are the dyadic points j 1 k an integer m can be chosen such that for n > m, the distance between suc cessive discontinuity points of is less than 6. For a non dyadic point E (ak , a H 1 ) , by the lemma,

t

•

.

2

•

1] '

Note that this is the average value of x on the interval [ak , a H so that the approximation to x is formed by step functions whose value is the average value of x on the dyadic intervals. By the choice of 6 and m, it follows that

for any nondyadic point t.

0

3. Bases

119

Exercises 3 . 5

1.

[iii] L 2 [0, 1] [iii] . L 2 [0, 1]. Show that the dyadic step functions D-functions constant on dyadic intervals [m/2 k , (m + 1) 12 k ) , m, k E Z-are dense in S. Show that the linear span Hn of the first 2 n elements of [iii ]

DENSITY O F IN Here is another way t o demonstrate the density of As noted in Example 2 .4.5( d), the class S of all step functions is dense in

(a) (b)

coincides with the linear span of the dyadic step functions Dn that are constant on the intervals of length 2 -n .

Hint

1.

{t l ,i 2 . . . t

xES

(a) Let , n } be the discontinuity points of that are not dyadic points . Surround each of these points by a small interval and define a dyadic step function Y D to agree with x outside these small intervals in such a way that f. (b) Hn C Dn and dim Hn = dim Dn .

E II x - y I1 2 <

3.6

Unconditional Convergence

A REARRANGEMENT of a series L neN Xn of vectors Xn from a normed space X is a series L neN Yn in which each Xn appears as some (unique) Yj · More formally, L n e N Yn is a rearrangement of L neN Xn if there exists a bijection f : N -+ N such that Yn = Xf( n ) for every n N . Riemann made the remarkable observation that the series L neN ( - l ) n In converges, but can be rearranged so as to converge to any real number whatsoever. Virtually any advanced calculus text has the argument ; it is outlined in the hint to Exercise a) . Series that converge on condition that the order in which they appear be left intact but that may diverge if rearranged are called CONDITIO NALLY CON VERG ENT . If the series converges to the same limit no matter how the terms are permuted, then it is called COMMUTA TIVELY or UNCOND ITIO NALLY CON VERG ENT . In this section we examine some connections between absolutely convergent, convergent and uncondi tionally convergent series. The gist of it is that in any infinite-dimensional Banach space, there is the following hierarchy of implications:

E

3(

absolute

'\./ convergent

unconditional

120

3. Bases

In finite-dimensional Banach spaces, unconditional convergence implies ab solute convergence-this is essentially the result of Exercise 3 . In a Hilb ert space of arbitrary dimension, convergence of a sequence of orthogonal vec tors implies unconditional convergence (3.6.2) . in We observed in 2.6.3(a) that an absolutely convergent series a Banach space must converge. We strengthen this now . We show next that an absolutely convergent series in a Banach space is commutatively convergent .

(xn)

3.6.1 ABS OLUTE CON VERGENCE IMPLIES UNCONDITIONAL CON VERG ENCE

A n absolutely convergent series ally convergent.

Proof.

N N

(xn) in a Banach space X is uncondition

( Yn) (x /(n ) ) be a rearrange n n Sn = L Xi and tn = L Yi . i= l i= l Since each X i must eventually appear in the sequence (Yn), for any n E N we may choose m � n sufficiently large that Let f : -> be a bijection and let = ment of the sequence Consider the partial sums

(xn).

m

n

2:7= 1 Il y; l1

i s therefore bounded above , s o it must The increasing sequence converge. Since X is a Banach space, the absolute convergence of and implies that it is convergent (2.6.3(a) ) . To see that the limits of are the same, we show that -> O. Since the partial sums are a Cauchy sequence, for t > 0 there exists such that

Sn - tn

. . . , m}

mEN

tn - Sn is a sum of terms x 0

Let k = max 1- 1 ( { 1 , ). For n � k, j> it follows from ( * ) that :S t .

m;

I i tn - sn ll

2: EN Yi Sn i t n 2:7= 1 Il x ill j

with

What about the converse? Does unconditional convergence imply abso lute convergence? For series of real numbers, unconditional convergence implies absolute convergence (Exercise 3). Indeed, (same exercise) the ar gument can be extended to show that unconditional convergence implies absolute convergence in any finite-dimensional normed space. But there it stops (Example 3.6 .3) . A profound result of Dvoretzky and Rogers ( 1950) asserts that if unconditional convergence implies absolute convergence in a Banach space X, then X is finite-dimensional! The proof can be found in Swartz 1 992 , pp. 440-4, Lindenstrauss and Tzafriri 1977, p. 1 6 , or for a very different approach, Grothendieck 1 955 , p . 149, or Diestel 1984, p .

3. Bases

121

59. Warning: These arguments are not for the faint of heart. I n Hilbert a series spaces, as we show next , for orthogonal sequences converges if and only if it converges unconditionally.

L n EN Xn

(xn),

3.6.2 UNCONDITIO NAL CONVERGENCE IN HILBERT SPACES Let (x n) be an orthonormal sequence in a Hilbert space X. Suppose (an) is a sequen ce from K an d L n EN a n Xn converges. Then L n e N anXn is uncon ditionally convergent. ( Note that any orthogonal sequence (mn) of nonzero vectors can be rewritten in the form (anxn) Il mn ll (mnl li mn I! ) We can therefore say that if (Yn) is a summable sequence of orthogonal nonzero vectors, then Ln e N Yn converges unconditionally.) .

=

Proof. Let x = L n EN anxn · Since L n EN anXn converges, L n eN l an l 2 < 2 00 by 3 .3 .4(a) . SinceL n E N l an l is an absolutely convergent series of real numbers, it converges unconditionally. By (3.3.4) (a) again, it follows that any rearrangement L n EN bn Yn of L nEN anXn converges as well. Suppose Ln EN bnYn = y . Does x y? For any i, for sufficiently large m, =

X i ) ai for all i. Thus, by Y L xn ) Xn L anXn x. 0 n EN n eN

so by the continuity of the inner product, (y, 3 .3 .4(e) , = = (y,

=

=

Example 3.6.3 UNCONDITIONAL CONVERGENCE DOES NOT IMPLY AB S O LUTE CON VERGENCE

(xn)

Let be a denumerable orthonormal basis for a Hilbert space X. converges by Since the coefficients are square-summable , = 3.3 .4(a) ; hence it converges unconditionally by 3 .6.2. Since does not converge absolutely. the series

Ln EN xnln Ln eN II xnln ll 0

lin L n e N xnln

Ln eN l i n,

Exercises 3 . 6

1.

(xn)

b e a denumerable orthonormal basis for a Hilbert space X . Let Imitate the argument of 3.6.3 t o show that for any square-summable sequence the series is unconditionally convergent.

(an),

LnEN anXn 2. SEPARABLE HILBERT SPACE £2 (Z) In 3.4.9 we showed that any separable infinite-dimensional Hilbert space is linearly isometric to £2 ( N ) . Show that £2 ( N ) can be replaced by £2 (Z) by showing that £2 ( N ) is linearly isometric to £2 (Z) . �

122

3.

3. Bases

UNCONDITIONAL IMPLIES ABSOLUTE in

Rn R Rn E N .

(a) Show that any unconditionally convergent series in is abso lutely convergent . ,n (b) Show that the result remains valid if is replaced by (c) Show that the result remains valid in any n-dimensional space over

R

K.

4.

2.6.3(

CRITERION F O R COMPLETENESS We observed i n a) that if X is a Banach space, then absolute convergence implies convergence. Show the converse, that if absolute convergence implies convergence in a normed space X, then X is a Banach space.

(Xn)

5 . CRITERION FOR UNCONDITIO NAL CON VERGENCE Let be a se is unconditionally quence in a Banach space X. Show that convergent if and only if for all f > 0, there exists a finite subset J of such that for any subset H of N for which J n H = 0 , s: f .

EnEN Xn

II En E H Xn I

N

Hints Consider a bijection f : -+ Z such as f ( n ) = % if n is even and for n odd. By then f (n) = if (Z). I--t < 00 . Now consider the map

N 3.6.1, (an) E £2 ( N ), En EN laf ( n ) 1 2 (an) (af ( n ) ) E £2 3. (a) Prove the contrapositive , that if EnEN l an l 00 then there is a rearrangement E n EN dn that is divergent. If E nEN an diverges, there is nothing to prove so suppose that it converges. Since E n EN an < 00 , there must be infinitely many positive terms and negative terms. Let bn and Cn be the subsequences of nonnegative and nega tive terms of (an) in the order in which they appear . For each n, then there are positive integers p� and qn such that E �=l a i (b 1 + . . . + bp n ) + ( C l + . . · + cq J . If EnEN bn 00 then E nEN Cn must converge well because E nEN an does and therefore so would E n l an l = (b 1 + . . . + bp J - (C l + . . . + cq n ) · Consequently, L n EN bn EN00 , and, similarly, E nEN Cn -00 . Now choose enough b's to make their sum � 1, then add in enough c's to make the sum S: O. I n this way create a rearrangement that oscillates and therefore does not converge. (b) For (a i , bi ) E R2 , note that E �= da i' bi ) = ( E �= l a i , L �= l bi ). Now use Example 2.2.6(b) and the result of (a) above . 2.

l ;n

=

<

as

=

=

=

3. Bases

123

(xn) II xn - x m ll Y Xnk - Xnk_1 LnEN Yn Xnk .

4. The idea is to show that a Cauchy sequence has a convergent sub sequence. Choose an increasing sequence ( nk ) such that :::; and k = 2- k for n , � nk . Let = for k > 1 . The is kth partial sum of the absolutely convergent series

m

YI Xn,

5. To show sufficiency, argue as in 3.6 . 1 . Argue necessity by contradic tion to get , for some > 0, disjoint finite subsets (H k ) of N such that IILn EHk Xn I � for every k. Define a bijection f N N such that €

€

:

L n E N xJ ( n ) is not convergent .

3.7

-+

Orthogonal Direct Sums

We considered external and internal direct sums of a finite number of inner product spaces in Section 2 . 1 0 . We consider infinite direct sums of Hilb ert spaces in this section. If is a countable orthonormal base for a Hilbert space X, then by 3 .4.2(a) , X = cl ( ) ] , where denotes the linear span of . . . } . Furthermore, the subspaces are mutually or thogonal and closed. This inspires the following definition.

(xn)

{X l , . . . , Xn,

[ xn

[(x n )l [x n l

Definition 3.7.1 INTERN AL ORTHOGONAL DIRECT SUMS

ffi n EN Mn

The INTERNAL O RTHOGONAL DIRECT SUM ( HILBERT SUM ) (or (int)) of a countable family of mutually orthogonal closed subspaces of a Hilbert space X is cl

ffi n E N Mn

(Mn ) [Un EN Mn] . 0

If is an orthonormal base for a Hilbert space X , then X = Moreover, by 3 .3 .4(e) , any E can be written as in other words, can be written as an infinite series of orthogonal projec Our next result shows that a vector tions on the subspaces from an orthogonal direct sum may be written as the infinite sum of its orthogonal projections on the

(x n )

( x, xn) Xn

X

X ffi nEN [xnl [xnl. ffin EN Mn Mn .

ffinEN [xnl. Ln EN (x, xn) Xn ;

3.7.2 PRO PERTIES O F DIRECT SUMS (a) SQUA RE-SUMMABILITY

For (Mn) and X as in Definition 3. 7. 1, and x E ffi nEN Mn (int) there exist mn E Mn such that x = LnEN mn and II x I1 2 = LnEN li mn 11 2 . Conversely, for mn E Mn such that L n EN II mn l1 2 < 00, then L nEN mn converges. For each E N , let Pn denote the orthogonal projection of X on Mn. Then, for any x E X, x = L nEN Pnxn. n

(b) UNC O N D ITIO NALLY CONVERG ENT SUMS

ffinEN Mn (int) = L n EN Mn , mn E Mn .

the set of all unconditionally convergent sums L nEN mn with

124

3. Bases

N [Un EN Mn] . x ffin EN Mn X n Mk ( n ) I I x - xn ll X n Mi + . . . + Mk ( n ) Xn -- x; Mi , Pn X Mn . 3 2 7( ) Pn x Mn . 3.2 7( ) P Pi + . . ' + Pk (n ) ' PiX + . . . + Pk (n ) X Mi + . , . + Mk (n ) ' X n Mi + . . . + Mk (n ) ' Proof.

For each n E there = cl (a) Let E exists k (n) E N and E such that < lin . Hence there exist vectors we can assume E such that that the sum starts in provided that we take some 0 vectors . Let denote the orthogonal projection of on the complete (closed subset of a complete space) subspace By . b , is the best approximation to x from Hence By Exercise - b M1 + .. +Mk ( n ) = is the best approximation to x from Therefore , since E .

(Pnx) x L nEN Pnxn . 2 3.3.4(g) II x II Ln EN II Pnx I1 2 . mn Mn L nEN mn L n EN li mn 11 2 M 3.3.4(a). mn ffi n] [U Ln E N n EN Mn · n EN 3.6.2, x ffi n EN Mn , mn Mn x L nEN mn · (mn ) 3.6.2. x ffin EN Mn LnEN Mn . Mn mn L nEN . Ln EN x ffin EN Mn 3.3.4(a). Ln e N I I mn l1 2 0

Since is an orthogonal sequence, it follows Thus, = from that = E is such that Conversely, if < 00 , then E cl = converges by Clearly, (b) By there is no difference between convergence and uncondi tional convergence of series of orthogonal vectors in a Hilbert space. By (a) , if E then there exist E such that = Since is an orthogonal sequence, the series is unconditionally con vergent by Hence C Conversely, let = E As a convergent series of orthogonal vectors , it by follows that < 00 by Therefore, E (a) above. To define external Hilbert sums, we use square-summability again . Definition 3.7.3 EXTERN AL ORTHOGONAL DIRECT SUMS

We define the EXTERNAL ORTHOGO NAL DIRECT SUM ffi n EN Xn (or ffi n EN Xn (ext)) of a countable family (Xn , ( - , ' } n ) of Hilbert spaces to be the collection of all sequences (Xn) from the Cartesian product Il nEN Xn such that L nEN II xn ll 2 < 00 . ffi n EN Xn (ext) is also called the EXTERNAL

HILBERT SUM. We add such sequences and multiply them by scalars in the usual pointwise way to make a vector space of them. We define the inner product of two such sequences and to be

(x n ) ( Yn ) {(Xn), (Yn) } = L {xn, Yn } n ' 0 (3.14) n EN As we prove below , not only does equation 3 . 14 define an inner product , ffin eN Xn i s a Hilbert space. 3. 7.4 EXTERN AL ORTHOGONAL DIRECT SUMS

direct sum ffi n E N Xn of a countable family Hilbert space.

The external orthogonal of Hilbert spaces is a

(Xn)

3. Bases

125

Proof.

The closure with respect to the operation of addition follows from the Minkowski inequality 1 .6.3(b) with p = 2. The only remaining problem of any substance about the proposed inner product is the convergence of We show that is (absolutely) the series convergent by using the Cauchy-Schwarz inequality 1 .3.2, followed by the Holder inequality 1 .6.2(b) with p =

L n EN (Xn, Yn} n

L n EN (Xn, Yn} n ·

2:

The remaining properties o f the inner product are trivial t o verify. To see that is a Hilbert space, consider a Cauchy sequence = = Xn . Let us now "look down the columns" . . . from from at n = 3, say, at the sequence For any k and j, It follows that , with ; :S � = is a Cauchy sequence from the Hilbert space Xj . Let n fixed, and for each n. We now show that = E = limj --4 Since is a Cauchy sequence in given > 0 there exists N such that for k, < . = Taking the limit as --4 00 ,

Xl ffin N Xn (X1 n )' X 2 (XE 2n )' ffin EN X3 . (X k 3 ) kEN 2 II X k 3 - xj 3 11 Ln E N II X k n - Xj n ll II X k - Xj 11 . (X, n)jE N Xj n Yn Y ( Yj ) ffinEN Xn Xn y. (Xn) ffi2 nEN Xn , f m 2: N, II X k - x mll Lj II X kj - X mjll� f 2 m (3. 15) L II X kj - Yj ll� :S f 2 . j Thus , X k - Y E ffi n E N Xn. Since Y = (y - X k ) + X k for every k, it follows that Y E ffi n E N Xn. That Xn Y now follows from equation ( 3 . 1 5 ) . 0 Thus, the elements of external direct sums are sequences; the elements of internal direct sums are sums. Let (Mn) be a collection of mutually --4

orthogonal closed subspaces of a Hilb ert space. It routine (but tedious) to verify that the map

A : ffinEN Mn (ext) (mn)

---*

�

ffin EN Mn (int) L n EN mn

(3. 16)

is a surjective inner product isomorphism .

Exercises 3 . 7

X

1. PASTING ORTHONORMAL BASES TOGETHER Let the Hilb ert space be decomp osed as = (int) and let be an orthonormal base for for each n E Show that is an orthonormal basis for

Mn X.

X ffin EN Mn N.

Bn UnEN Bn

126

2.

3. Bases

Show that the inner product of Definition uct axioms.

3.7.3 obeys the inner prod

3. Verify that the map of (3.16) is an inner product isomorphism . 3.8

Continuous Linear Maps

Let X and Y be normed linear spaces. A linear map : X ---> is contin uous if and only if it is bounded on the closed unit ball U of X by 2.3.3; i.e . , for some � 0,

A

Y

k

II A x li ::; k for all x E U {::::> II A x li ::; k II x li for all x E X. (3.17) The set L (X, Y) of all continuous linear maps (also called OPERATORS or BOUNDED OPERATORS) of X into Y is a vector space with respect to the pointwise operations: for any A, B E L (X, Y) and any scalar a, we define A + B and aA at any x E X to be ( A + B) x = Ax + Bx and (a A) x aAx.

=

With

=

II A II = sup II A (U) II , L (X, Y) is a normed space, as we verify next . Clearly, II A II � 0 and II A II 0 if and only if A = O . For any scalar and any x E U, II (aA) x II = l a l ll A x ll so aA is bounded on the closed unit ball, and since nonnegative multiples a

come "through" the sup,

lI aA I i = l a l ll A II · For A, B E L (X, Y ) , and x E U, II ( A + B) x II = II Ax + Bx ll ::; II A x li + II Bx ll · The triangle inequality follows immediately. are given in Some alternative ways to compute

II A II

3.S.1 T H E NORM OF A LINEAR MAP

For A

3.8.1.

E L (X, Y ) ,

sUPl :9 11 Ax ll = sUP l l xx ll = l I I Ax i l = inf {k � 0 II A xl l ::; k for all x E U} . Proof. Let I<. = {k � 0 for all x E U, II A x li ::; k}. Clearly II A II ::; inf I<. . Since I I A x II ::; II A II for all x E U, it follows that I I A II E I<. The refore II A II = inf I<. . Since sup i s monotonic (the bigger the set, the bigger the sup) , c = sUP l x l l = l l1A x li ::; II A II . Suppose that 0 :5 c < I IA II . If so, then there must II A ll

:

:

3. Bases

127

x

be someSincewithx 0 << 1,II xthis l < 1 such that c < I A x l i . Thus, I A (xl I x lDI l ::; c < I Ax l i . I l l means that II Ax l ::; c I Ix l < I Ax l l x l l ::; I A x l · It follows from this contradiction that c = sUPl l x l = l l 1 Ax l l = I I AI I . 0 We compute the norm of an orthogonal projection next.

Example 3.8.2 NORM OF ORTHOGONAL PROJECTION

=

1

Let M be a closed nontrivial linear subspace of the Hilbert space X . By the projection theorem, 3.2.4, X = M$M .L (top). Let Pbe the orthogonal projection x E X, there are unique elements m E M and n E M.L such that xon M.m For + By the Pythagorean theorem, I x l l 2 = I mll 2 + I n l 2 � I ml l 2 = II Px ll 2 so I I Px ll 2 ::; I x 1 l 2 ; therefore, II P II ::; 1. Since there must be unit vectors x E M for which P x = x, it follows that I I P II = 1 . 0 =

n.

3.8.3 NORM O F AN INTEGRAL OPERATOR ON REAL C [a, b] Let C [a, b] denote the real Banach space of real-valued continuous functions on [a, b]. For every s E [c, d] , let the real-valued function D (t, s) be continu ous for all t E [a, b]. For E C [a, b], define

x

Then

Ax (s) = lb x (t ) D (t, s) dt.

A is a continuous linear map of C [a, b] into C [c, d], and I A I = s esup[c ,d] Ja[b I D (t, s) 1 dt.

D is called the K ERNEL or K ERNEL FUNCTION of the integral operator A . Proof. The map A is clearly linear and maps C [a, b] into C [c, d]; A is

bounded since l Ax (s)1 = Ja[b x (t ) D (t, s) dt (s esup[c,d] Ja[b I D (t, s) 1 dt) II x l i oo (x E C [a, b] ). Thus, [c,d] f: I D (t, s ) 1 dt . In the remainder of the argu II Dthat= sDUPiss e the mentConsider weI Ashow the ramp functionsnorm of A. { -1 t ::; -lIn, Un (t) nt, I t I < l I n, , n E N , 1, t � l In, ::;

::;

=

'

3. Bases

128

= Un Let X n for any fixed this composite of continuous functions of is clearly continuous in For all and the product

(D (t, s)); s, t t. t s, lin xn(t, s)D (t, s) = { nI DD(t,2 (t,s)s)l , :S I D (t, s) l , II DD (t,(t, s)s) 1l >:;;: lin (t, s)

is nonnegative. Since continuous functions are Lebesgue measurable,

la b xn(t, s) D (t, s) dt

AX n (s) =

2:

{

J{tE[a , bl:ID(t ,s)l � l/n}

I D (t, s) 1 dt (3.18)

Since

lb I D (t, s) 1 dt

{

I D (t, s) 1 dt + {1D(t,s )15.l/n I D (t, s) 1 dt J 1 1ID(t,s )l ? l/n I D (t, s) 1 dt + lID(t,s )l $ l/n -n dt,

J1D(t,s )I? l/n

<

equation

( 3 18 ) implies .

b A x n (s) 2: la I D (t, s) 1 dt - -n1 (b - a)

for every n and

s. Therefore ,

b

I Axn (s) 1 2: SE[supc ,d] la I D (t, s) 1 dt. nEN,sE[ c ,d] sup

Since

IX n (t, s) 1

I IAII

sup

=

1I "'1 I 00 5.

:S

1 for all t, n and s, it follows that

sup I D (t, s) 1 dt = D, I Axn (s) 1 2: s sup II Ax ii oo 2: nEN,sE[c,d] l E[c ,d] la

b

which completes the proof.

0

Exercises 3 . 8

1. 2.

A:

X � Y b e a linear isometry between NORM OF A N ISOMETRY Let the normed spaces X and Y . Show that =

II A II 1.

Let x b e a nonzero element of the normed space X and consider the linear map g", Show that the map ...... g", is a � X , ...... linear isometry of X onto X) .

:K

a

ax. L (K,

x

3. Bases

129

loo {U l , U 2 , " " un} A : loo N . C l , . . . , Cn K. loo loo a I , a 2 , . . . , an, A ( E� l ai u i ) = E7= 1 Ci a i U i , Ci {U l , U 2 , . . . , un} II A II = maxi l ei l · 4. FINITE- DIMENSIONAL CASE : THE NORM OF A : l2 ( n ) l2 ( n ) , n E N . This seems as though it should be easy to compute, but it is not. We discuss the results and provide some details in the hints . Let { U l , . . . , un } be an orthonormal basis for l2 ( n ) and let (aij ) be the matrix representation for A with respect to this basis. We denote the rows (ai l , a i 2 , . . . , a i n) of (a ij ) as a i and assume that none of them is 0; we denote the columns by Ci . Note that this means that for x E lz ( n ) , 3.

( n ) --+ THE NORM O F ( n ) Let be a basis ( n ) by (n) , n E E Let ( n ) -+ for Define A : taking, for any scalars so that the matrix representation for A with respect to the basis is a diagonal matrix with on the main diagonal. Show that

loo

--+

Ax =

( ��: :� )

(an , x) We begin with some bounds on II A II . (a) B O UNDS Show that

n max { miax ll aill , mF ll eill } :S II A II :S L l aij l 2 . i ,j = l

(b) PARALLEL Rows If the rows are "parallel" in the sense that for then there are constants . . . , n, Thus, for example ,

VE?'j =1 I aij I 2 .

ki ' i = 2, 3, 1 1 1 2 2 2 3 3 3

a i = ki a l ,

II A II =

= )42.

(c) ORTHOGONAL Rows If the rows are mutually orthogonal (as column vectors in ( n )) , then (d) ORTHO GONAL GROUPS OF PARALLEL Rows If the rows can be grouped in sets where within each the rows are parallel but rows from different groups are orthogonal , then is a group of rows for which where

l2

II A II = maxi lI a; ll .

Ml , M2 , ' . . , Mk Mj IIAII = VEa,EMJ Il ail1 2 , VEa,EMj Il ail1 2 is maximal.

Mi

130

3. Bases

Xl, X 2 , , Xn is an orthonormal basis of £2 ( n ) of eigenvectors of A £2 ( n ) - £2 ( n ) , then its matrix representation is a diagonal ma trix with its eigenvalues C l , C 2 , , Cn on the diagonal. Use (d) of the previous exercise to show that II A II = maxi l ed .

5. If

.

•

.

:

.

•

•

Hints

4.

AUi = Ci for each i and conclude that II cd l :S II A II·

(a) Show that N ext, note that

bi - l Il ai II bi + l bj

bn

Il ad l :S I (ai , x) 1 lI aill :S Il x ll 1, I l ad l ll x ll Il aill for each i. This implies that II Ax l1 :S /'L 7= 1 1I aiI1 2 J'L�j = l l aij 1 2 . (b) Consider an orthonormal basis {x 1 , X , . . . , xn} for £ ( n ) in which X l a l · Thus (aj , X k ) = 0 for j, k 2:2 2. For a unit2 vector X = 'L7= 1 bi X i , the jth row of the2 column vector Ax is bj kj (a l , a l ) with k l = 1. Thus II Ax ll 2 = I bd 11 a l l1 4 'L7= 1 I ki 1 2 . This fraction is maxi mal when 1 � 1 2 is maximal, which-since is a unit vector-occurs when I h I = 1. Therefore , n n 2 2 2 Ik = L a j l2 . II A II = II a ll1 L =i l il i ,j = l l i (c) {a l , a 2 , . . . , an} must be an orthogonal basis for £ 2 ( n ) . For x 'L7= 1 bi a i , Ax is the column vector whose jth row is (aj , x ) = bj Il aj l1 2 . Assume that is a unit vector and that Il a l ll 2: Il aj II for all j. Show that n 2 2 = a + Ax II l1 II l ll L I bd 2 ( 11 ai11 4 - ll a d I 2 1I ad I 2 ) . i= 2 Since Il a lll is maximal, it follows that Il a il1 4 - II a ll1 2 11 a ill 2 :S 0 for i = 2, 3, . . , n . Therefore, II A x ll 2 achieves its maximum lI a l l1 2 when bi 0 for i 2: 2. for appropriate and every i (the b's depend on i) . Therefore, for each i. < = by the Cauchy-Schwarz inequality For

II A II

=

=

=

x

x

.

=

=

3. Bases

3.9

131

Dual Spaces

L

Y)

A s i n the preceding section, ( X , denotes the normed linear space of all continuous linear maps of the normed space X into the normed space i.e . , the linear forms are A special case of great interest is that of = to X ' . The normed space the linear functionals on X. We shorten (X, is called the (CONTINUOUS) DUAL or (CONTINUOUS) CONJUGATE of X. The study of the interplay between X and X' is called DUALITY THEORY . We compute some norms of linear functionals i n the next example .

Y.

Y K, L K)

X'

Example 3 . 9 . 1 NORMS OF SOME LINEAR FUN CTIONALS

(a) PROJECTION O N A VECTOR: Let z be a nonzero = vector in an inner product space X and consider the map Iz : X --+ The map Iz is linear by the linearity of the inner product in 1--+ the first argument ; it is continuous because it is bounded on the unit ball for U of X by the Cauchy-Schwarz inequality: l iz it = any E U. Hence z = sUP II�II = l l /z Since l iz follows that z (b) EVALUATIO NS. For 0 , 1 define the (linear) evaluation map i to be the map 1--+ of 0 , into Clearly, I i I = ( ) for any x, so II i ll As there are continuous functions x of norm 1 such that = 1 , II i ll = (c) INTEG RAL OPERATOR ON REAL Let be the space of continuous real- or complex-valued functions on with sup norm For let I ( ) = = sup dt . For any the linear form defined by

" z ll

II {·, z} 1I

K,

x {x , z} .

(x) 1 � II z ll ll x ll � II z ll (z) 1 II z 11 2 , (x) 1 � II z ll .

II l " II l " = II z ll · t E [ ], x x (t) C [ 1 ] K. � 1. 1. x (t)

x

I x t 1 � II x 11 00

C [a, b] C [a, b] [a, b] { l x (t) 1 : t E [a, b]), x E C [a, b]. x E C [a, b] ' D E C [a, b]

II x lioo f: x(t)

is continuous, and

c = d.

(x)

0

x

I (x) = lb x(t)D (t) dt

11/11 = f: I D (t ) 1 dt. This follows from 3 . 8.3 by taking

The following result , known as the Hahn-Banach theorem, is very im portant. We omit its proof but it can be found in almost any book on functional analysis (for example , B achman and Narici 1966 , p . 176f) . 3.9.2 HAHN-BANACH THEOREM If f is a bounded linear form defined on a linear subspace M of a normed space X then there is a continu ous extension F of f to X such that II F II = 11/11 .

F: X

I

I:

I M

'\.

---+

K

and

11/11 = II F II

3. Bases

132

Two imp ortant consequences of the Hahn- Banach theorem are the fol lowing: • •

=P {O} . If all continuous linear forms vanish on a given vector x, then x = O. The dual of a nontrivial normed space X is not trivial : X '

Information about the dual of a space can be very enlightening. Hilbert spaces X, for example, are essentially their own duals. Before we prove that, let us make an observation about the enormity of the null space of a linear functional. 3.9.3 NULL SPACE O F A LINEAR FUNCTIO NAL If f is a nontrivial linear form on the vector space X, then any algebraic complement of its null space N 1 - 1 (0) is one-dimensional, i. e., of the form [m] = Km for some vector m =P o. =

P roof. Does N have an algebraic complement? Since f =P 0, there exists m E X such that I (m) =P O. For any x E X , consider z x - (f (x) / I (m)) m. Clearly, I (z) = 0 so z E N and x z + (f (x) /f (m)) m E N + [m]. Obviously, N n [m] = {O} so [m] is an algebraic complement of N. If L is any algebraic complement of N and y is any nonzero vector in L, then X = N [y] , exactly in the argument above . Consequently, if z E L, then we can write it in the form z + ay for some E N and scalar a. Since z - ay E N and z - ay E L, it follows that z - ay 0, i.e . , that z E [y]. Therefore , L = [y] and the proof is complete. 0 We observed in Example 3.9 . 1(a) that maps fz (x) (x, z) are contin uous linear functionals of norm II z ll . In Hilbert spaces, these are the only continuous linear functionals, as shown next . The ability to write any con tinuous linear functional on a Hilbert space in the form Iz (x) = (x, z), =

=

Ef)

as

= n

= n

n

=

=

where z is uniquely determined by f, is known as the RIESZ REP RESENTA TION THEOREM; it is subsumed by 3.9 .4.

3.9.4 HILBERT SPACES ARE (ESSENTIALLY ) SELF-DUAL Let X be a real

or complex Hilbert space. For any z E X, the map Iz , continuous linear functional on X. If X is real,

A:

X

--+

z ....... ( . , z ) ,

is a

, Iz ,

X'

then A is a surjective linear isometry. If X is complex, then A is a "conju gate lin ear" isometry of X onto X ' , where by "conjugate lin ear" we mean A (a x) = aAx for any scalar a and any x E X .

A

f II A z ll = II lz II

Proof. It is trivial to verify that the map : X � X ' , Z 1--+ z is conj ugate linear or linear , depending upon whether X is real or complex, respectively. = We observed in Example 3.9 . 1(a) that is norm-preserving:

A

3.

li z I

Bases

133

, i.e . , it is an isometry. To see that A is onto, let I be a nontrivial continuous linear functional on The null space N = (0) is closed by = N $ N .L by the projection theorem 3 .2 .4. By 3.9.3, 2.4.4. Therefore, there is some unit vector such that N .L = [ . By Example 3.2.7(a, c) , we can therefore write any E X as = n + for some unique n E N. Hence

X

1- 1

X.

m

x

m] {x, m} m

x

I (x) = I (n + (x, m } m) = (x, m} / (m) = ( x ' / (m)m ) . In other words, I = Ij ( m ) m . Finally, note that I (m)m is unique, for if y were also such that

{ x , y} = I (x) = ( x, I (m)m ) , for all x E X, then y - I (m)m would be orthogonal to every vector x and would therefore have to be O. 0 Exercises 3 . 9

1.

R3 R Iz ( ) { . , }

Let I : � b e defined by . = z . z such that

(a, b , c)

�

a

3 -b+

I (x y) x, y X] a R x Let I : R3 R be the linear map (a, b, c) continuous and compute 11/11 .

c. Find the vector

I (x) I (y) X R. (ax) a l (x)

2 . Let I be a continuous, additive [i.e. , + = for all + Show that E map of the real normed linear space into J is necessarily SCALAR HOMO GENEOUS , i.e. , I for all = E and E X. 3.

�

1---+

a + b. Show that l is

4 . By Exercise 2.9-4, all linear functionals on a finite-dimensional normed space X are continuous. Let , } be a basis (not neces Then, for any sarily an orthonormal basis, just a Hamel basis) for Let I be a E there exist scalars such that = in the following cases: linear functional on Find

{X l , X 2 ,

x X,

•

.

.

xn X. x L� l a i X i .

ai 11 / 11 (a) Norm X by taking II x ll maxi l a i l . 1/2 (b) Norm X by taking II x ll = IIL?=l a i x i ll [L ?=1 I a i I 2 ] . 5 . On the space £ 1 of absolutely summable sequences x = (an) (i.e . , (a n ) such that Ln eN l an l < 00 ) , consider the map I defined by taking I (x) = Ln e N an · (a) With II x ll 1 = L n eN l an l , show that I is a continuous linear X.

=

form and compute its norm.

=

134

3. Bases

(b) If i 1 is normed by taking discontinuous.

6.

II (an) lI oo

=

S UPn

l an l , show that I is

DUALS ARE COMPLETE For any normed space X, show that (a) The dual X ' is a Banach space. (b) If Y is a Banach space, then so is X.

L (X, Y) for any normed space

7. THE SECOND DUAL Let x E X where X is a normed space and define --+ I f-+ I (x) . x'

: X' K,

( a) Show that x , is a bounded linear functional on X' . (b) The dual X" of X, is called the SECOND DUAL or SECOND CONJUGATE of X. With x' as in (a) , show that the map into X". --+ f-+ x ' , is a linear isometry of J : (c) If the map J of (b) is onto, then is said to be REFLEXIVE. Show that all finite-dimensional spaces and Hilbert spaces are reflexive. (We are not asking you to prove this, but we mention that the ip for 1 < p < 00 are reflexive but i 1 is not .)

= L (X', K) X X", X

8.

X

X

Let Ibe a linear functional on the Hilbert space X with null space N. (a) Show that if l is not continuous, then N is dense in (b) Prove that I is continuous if and only if N is closed .

X.

9. L e t X be a Hilbert space. If x , y E X are such that for any contin uous linear functional I on X, I (x) I (y) , then show that x y. =

=

(It follows from the Hahn-Banach theorem that this is true for all normed spaces. )

Hints 2. First show that I (nx) = nl (x) for integers n; then show that I (x I n) = ( l in) I (x) . Conclude that for any rational number r, I (rx) = r I (x) . Finally, use the density of the rationals in the reals .

4.

ai X i·

Let x = E?= 1 (a) I / (x) 1 � I l x I l E?= 1 1 / (xi ) l , so 1 1/1 1 � E?= 1 1/ (xi Now let ai = I/ (xi ) l /l (xi ) for I (xi) 0, = 0 otherwise. (b) By the triangle and Holder inequalities,

f. ai

a d (X i ) I < E?- 1 l ai I I I (X i ) I < I l x l l (E?- 1 I I (X i ) 1 ) 1 / 2

IE?- 1 IIxll so 1 1 / 1 1

-

IIxII

-

� (E i=n 1 1 1 (X i ) 1 ) 1/ 2 . Now let ai I (X i ). =

--

Ilxll

'

3. Bases

135

5 . (a) Use the triangle inequality. ( b ) Consider the sequence from the unit ball = ( 1 , 1 , . . . , 1 , 0, 0, . . .

xn

)

whose first n entries are 1 and 0 thereafter. 6(a) . For a Cauchy sequence im

l n In (x) .

8.

Un ) from X'

and

x

E X , define

I (x)

=

X

(a) . Prove the contrapositive. Assume that cl N =P and choose a unit vector w E (cl N) 1. . Show that 1 0 = - . ( w ) w .

(

I

)

(b) We need only prove that N closed implies f continuous. This follows from ( a) . 9 . Use 3 . 9 . 4 to show that

3 . 10 Let Fix

x - y E X 1. .

Adj oints

X be a Hilbert space and let A : X � X be a bounded linear map . y E X and consider the map Iy (x) = (Ax, y) , x E X .

As the composite of continuous linear maps , f is clearly a continuous linear functional on As such , by the Riesz Representation theorem 3 .9 .4, there exists a unique E such that

X. y* X (Ax, y) = ( x, y* ) for all x E X

This procedure induces a mapping

A* : X ---+ X, Y 1----+ y* , called the ADJOINT A* of A , where A*y = y* is defined by (3.19) (Ax, y) = (x, A*y) , for all x E X . It i s routine t o verify that A* i s linear and that , for any bounded linear maps A, B X X and any scalar a, A** A (see 3 . 1 0 . 1 ) , (A + B) * A* + B* , CiA* , (aA) * (AB) * B*A*. If A = A* , then A is called SELF-ADJ OINT. :

�

136

3. Bases

3 . 1 0 . 1 ADJOINTS Let X be a Hilbert space and lei A X bounded lin ear map. The adjoint A* is a bounded linear map with :

X be a II A* II =

II A II · Proof. We first show that adjoint is norm-reducing, i.e. , that II A* II � II A II . To this end , note that for all x E X, (A*x, A*x) ( AA*x, x) II AA* x ll · ll x ll < II A II II A*x ll ll x ll For nonzero A*x, this implies that II A * x ll � II A ll ll x ll so II A * II � II A II · Thus, since A* is bounded, we can consider its adjoint A** as defined by (A*x, y) = {x, A**y} for all X, y E X. Since (x, Ay) = (Ay, x) = ( y, A*x) = ( A * x, y) = (x, A ** y) for all and y, it follows that A**y Ay for all y and therefore that = A* * A . Since the norm of an adjoint is no larger than the norm of the x

original map , we have

=

=

II A II II A** II � II A * II .

0

We show next that the adjoint of a linear map defined by matrix multi plication is determined by the transpose of the original matrix in the real case; otherwise, it is the conjugate transpose . Example 3.10.2 ADJOINT OF

A : R2 R2 _

"

= " TRANSPOSE O F

A

Let a, b, d E R. In this example we make repeated use of the fact that if axi + bX 2 = 0 for all X l , x 2 E R, then a = b = O. Consider the linear map A : R2 R2 , ( : � ) ( � � ) ( : � ) (matrix multiplication) . Since A * also maps R2 into R2 , we see that there must b e a 2 2 matrix ( ; { ) such that A* ( �� ) = ( ; { ) ( �� ) for all ( � � ) E R2 . For all ( � � ) and ( �� ) , by the definition of adj oint, c,

_

f-+

x

or

(a YI + CY2 - e YI - !Y2 ) X l + (bYI + dY2 - gYI - h Y2 ) x 2

which implies that

(a - e) YI + (c - f) Y2 (b - g) YI + (d - h) Y2

0,

O.

=

0,

3. Bases

Therefore, a

= e, C =

the transpose of any

n

x

n

I, etc. We see that

( � ! ).

A*

=

137

( � � ) ( � ! )t,

We ask the reader to prove that the adj oint of

matrix is its conj ugate transpose . 0

Exercises 3 . 1 0

1.

X, A X X

EIGEN VALUES O n a vector space let : -> b e a linear map . If x is a nonzero vector and a is a scalar such that x ax , then we say that a is an EIGENVALUE of A; x is called an EIGENVECTOR of A. Suppose that is a Hilbert space and that E IS self-adjoin t .

X

A = A L (X, X)

(a) Show that all eigenvalues of A are real. (b) If x and y are eigenvectors of A associated with the distinct eigenvalues a and b, respectively, then x is orthogonal to y.

4

Fo urier Series Fourier uses his mathematics with the delightful freedom and naIvete of [one] who trusts in a mathematical providence. -E. B . van Vleck N OTE . Except for Section 4.19, we restrict consideration to real valued functions in this chapter. The expression "function I" . means "I : R -+ L:; [-11", 11"] and Ll [-11", 11"] denote the subspaces of real-valued elements of L 2 [ - 11", 11"] and Ll [-11", 11"], respectively. The process of understanding is always facilitated if more complicated structures are known to be synthesized from simpler ones-matter from molecules, molecules from atoms , atoms from quarks, organisms from cells , integers from primes. In mathematical analysis we don't usually get a full decomposition into the simpler things but an approximation of the more complex by the more elementary-real numbers by rationals, for example . When you truncate the Taylor series expansion of a function, you approx imate the function by a polynomial. The quality of the approximation de pends on the number of terms you take. Of course, for a function to have a Taylor series, it must (among other things) be infinitely differentiable in some interval, and this is a very restrictive condition. Sines and cosines serve as much more versatile "prime elements" than powers of t. Sines and cosines may be used to approximate not only nonanalytic functions ; they even do a good j ob in the wilderness of the wildly discontinuous. And even though sines and cosines are periodic, this does not constitute a very serious limitation. In most situations there is usually some interval in time or space on which interest centers. Functions can be truncated to the interval of in terest , and then periodically extended . As we shall see, sines and cosines triumphantly approximate functions I from the wide class Ll [-11", 11"] of integrable functions on [-11", 11"] in the sense that their FOURIER SERIES

R;"

�o + L an cos nt + L bn sin nt nEN nEN

(

4 .1)

(C, I), etc.) , to I, ( 4 .2)

converge somehow (in the mean, pointwise, uniformly, where ={O} , I(t) cos nt dt , n E

an

and

1

Nu 1"2 bn = -1 1 "- I(t) cos nt dt , n E N ; 11"

_ ,,-

11"

0

( 4 .3)

140

4. Fourier Series

bn

are called the FO URIER COEFFICIENTS of f. the terms a n and The attempt to approximate functions by trigonometric series began in connection with the problem of vibrations of strings, i.e . , attempts to solve the wave equation

fJ 2 w - a 2 8 2 w 8t 2 - 8x 2 ·

(4.1))

Daniel Bernoulli ( 1 753) proposed a trigonometric series (equation as a solution. Euler and a young Lagrange also made contributions t o the problem. In connection with a trigonometric series that arose in a different problem, Euler published the integral form of the and of equation ( . 1 in 1757 . The problem of how heat diffuses in a continuous medium defied the mathematics of the day in "the best of times . . . the worst of times," the early 1 800s. Jean Baptiste Joseph Fourier invented powerful new techniques to attack the problem, which included "Fourier series." (We should point out that "convergent trigonometric series" and "Fourier series" are not the same thing [see Section 4. 1 1] . ) Fourier's efforts were based on a shaky, intuitive foundation: He did not prove that his series converged . His con temporaries, Lagrange , Legendre, and Laplace, objected-not a pedantic complaint when you consider some of the nonsense that can occur when infinite processes are treated too casually. Fourier asserted that any peri odic function whatsoever could be written as a trigonometric series. Alas (Section .9 , some functions are too tempestuous to permit such a repre sentation. But even so, Fourier was mostly right. It is imp ossible to conceive of the signal processing upon which the modern world depends, from CDs to satellites, without Fourier series and its modern offshoots-the Fourier transform, the fast Fourier transform ( "the most valuable numerical algo rithm in our lifetime" [Strang 1 993 , p. 290) ) , and the wavelet transform (see Section 7 . 1 1 ) . An unintended consequence of Fourier's insightful but rash forays essentially brought analysis from its childhood to adolescence during the nineteenth century. Much of the gestation had a direct root in an attempt to j ustify Fourier's manipulations, or to solve other problems related to Fourier series. Dirichlet was the first to prove pointwise convergence of Fourier series for a large class of functions. In memoirs in 1829 and 1 837 , he showed that the Fourier series of a piecewise smooth function 1 on [-11", 11"] converges pointwise at every to the average of the limiting values

an

4 )

bn

4 )

(4.6.2)

t l (r) + / (t + ) 2

Dirichlet suspected that the theorem was true for a wider class of functions but was never able to prove it. Jordan did in 188 1 . To do so he defined functions 1 of bounded variation, and showed that the Fourier series of such an 1 converges pointwise to at every Fejer ( 1904) discovered that for any integrable function 1 o n [-11", 11"] , 1 (t)

(I (t - ) + / (t + )) /2

t (4 . 6.6).

4. Fourier Series

141

can be recovered by summing the arithmetic means of its Fourier series; even if the Fourier series for does not converge at a point the series of arithmetic means converges to /2 (4. 15.3) , provided that + the one-sided limits exist . Riemann was inspired by Fourier series too, and made deep contributions to their theory. When Riemann first considered the problems, "integral" was used in the sense given it by Cauchy in 1823: "Integrable" meant con tinuous. Riemann decided that a more general integral was needed, and he created the Riemann integral for functions that were required only to be bounded-no formal continuity requirement . He did this in his Habilitation sehrift at Gottingen in 1854 entitled Uber die Darstellbarkeit einer Funk tion du reh eine trigonometrisehe Reihe; it was not published until 1867, after his death. As an application of his new integral, Riemann showed that the Fourier coefficients of equations (4.2) and (4.3) of a and Riemann-integrable function approach 0 as n -+ 00 , an early version of the Riemann-Le besgue lemma 4.4 . 1 . Riemann also discovered the "localization principle (4.5. 1 1) ," that the behavior of the Fourier series of at a point depends only on the values of in an arbitrarily small open interval about t; he also proved the first version of the Parseval identity. In terms of the intrinsic character of a function whose Fourier series con verges "well," the smoother (continuous, continuously differentiable , etc.) the function is, the "better" its Fourier series converges (the more rapidly the coefficients approach 0) . Stokes ( 1 847) and Seidel ( 1 848) discovered the infinitely increasing slowness of convergence of the trigonometric series at a point of jump discontinuity. They observed that the discontinuity can not be enclosed in any interval in which the convergence of the Fourier series is von gleieh em Grade (uniformly convergent) . Through Weierstrass, and his students , especially Heine (as in "Heine-Borel" ) , the concept of uniform convergence percolated into analysis generally. In the presence of uniform convergence, certain attributes (e.g . , continuity) of each term of a sequence or series persist to the limit , and series can be integrated term by term. For example, if a series of continuous functions converges uniformly, then its limit is continuous. Heine proved in 1 870 that the Fourier series of a piecewise smooth 21r-periodic function converges uniformly in any interval that does not contain discontinuities of the function (4. 7.4(b ) ) ; in particular, if f is continuous, then its Fourier series converges uniformly and absolutely on every Closed interval (4.7.4(a) ) . Riemann's notion of integral prevailed until the arrival of the Lebesgue integral in the early twentieth century. Lebesgue's invention was also moti vated by problems concerning Fourier series. (Perhaps "motivated" is too strong, but he certainly was actively involved in research on Fourier series, and the first applications he made of his new integral were to the theory of Fourier series. ) Now continuity restraints were greatly relaxed (Riemann integrability is equivalent to continuity a.e.) , and there was much more free dom to take limits through the integral by means of Lebesgue's dominated

f

(f (t - ) f (t + ))

bn

an

t

t,

f

f (t)

f

142

4. Fourier Series

convergence theorem (4.4.2). In 1903 Lebesgue improved Riemann 's result that the Fourier coefficients an and of a bounded Riemann-integrable function j approach 0 as n � 00; he broadened it to include Lebesgue integrable functions j, even if j is unbounded. He also extended Rie mann's localization principle to Lebesgue-integrable functions. The subject of "Fourier series" -also known as harmonic analysis--- c ontinues to develop today in a much broader context known as abstract harmonic analysis. We begin our development of Fourier series in Section 4 . 1 by considering Fourier analysis in the REAL Hilbert space L� [-1I", 11"] of square-integrable functions on [-11", 11"] . Square-integrable functIons include a wide variety of functions that occur in practical applications such as the piecewise con tinuous functions on the closed interval b] . In the L5 [-1I", 11"] setting, we consider convergence of the Fourier series of a function E L5 [-11", 11"] "in the mean" -in the L 2 -norm 1 1 ' l b-to the limit function. Mean convergence of the Fourier series for I E L5 [-1I", 11"] signifies that if is the nth partial sum of the Fourier series, then the mean square error

bn

[a,

I

Sn

I: II (t) - Sn (t) 1 2 dt I

goes to 0 as n � 00 . For the step function of Example 4 . 1 .3, it means that the shaded area of the figure below shrinks to 0 as n � 00 .

A

square

wave and

84

Generally, mean convergence does not imply pointwise convergence (Exam ple 2.2.7(b)) . In Sections 4.6 and 4.7 we consider smoothness assumptions on the function that guarantee pointwise and uniform convergence of the as sociated Fourier series. In regard to pointwise convergence, no smoothness assumptions are really necessary. A deep result of Carleson ( 1 966) shows

4. Fourier Series

143

fE

that the Fourier series of any function L:; [- 7I', 7I'] converges pointwise a.e . on [-71', 71'] . We move to the wider context of Fourier series for functions in Section [-71', 71'] for all p � 1 , this is more general and -71', 71'] C Since quite useful. For functions we consider pointwise , uniform, and (C, l ) convergence o f the Fourier series. To illustrate how different the L l and theories are , consider the following point. Unlike the a.e. convergence of the Fourier series of any 71'] , Kolmogorov gave an example of 71'] whose Fourier series diverges at every point. We say more an f E about the phenomenon of divergent Fourier series in Section

Lp [

4.4.

Ll

L2

f E L 2 [ - 7I',

L 1 [- 7I',

4. 1

L'i

Ll

4.9.

Warmup

A real- or complex-valued function h defined on all closed intervals except possibly for a finite number of points in each of them is p-PERIODIC if there exists a p > 0 such that h p) = h (t) for all t and t + p where h is defined. If h is p-periodic, then the function 9 defined for all by 9 = h ( pt /271') is 271'-periodic : As t advances by 271', pt/271' increases by p. Periodic phenomena are commonplace; they include the rotation of the earth on its axis , the motion of the moon about it, planetary motions generally, vibrations of strings or tuning forks, pendulums, tides , and electromagnetic w aves. Any real- or complex-valued function h defined on a closed interval [-p, p] can be made into a periodic function by PERIODIC EXTENSION or 2p-PERIODIC EXTENSION as follows: Write x R as x = 2 p t for appro priate N and [-p, p] ; then define

(t +

nE

t

(t)

n +

E

tE

h (t

+ 2 np) = h (t) , n E Z, tE R.

Since we are considering Lebesgue integrals, if we change the values of a function at a finite number of points, this will not affe ct the integrals that produce the Fourier coefficients (equations and This gives us some options .

(4.6)

(4.7)).

Remark PERIODIC EXTEN SION AND THE ENDP OINTS

[- 71', 71']

h

If a real- or complex-valued function h is defined on and (-71') # h (71') , then we have choices about its periodic extension. We could omit the values of h at -71' and 71' or redefine the function so that h (-71') = h ( 71' ) and then extend h periodically. For example , if h (t) = on we might redefine it to be at - 7I'-or declare it to be undefined there-and then periodically extend it by defining h (t) = h 271') for every x. No matter what we do, if h #- h the periodic extension of h will have discontinuities at odd multiples of 71'. We consider what the Fourier series of h converges to at jump discontinuities of the periodic extension of h in 0 Section

71' (-71')

4.6.

t

(71'),

(t +

[-71', 71']

144

4. Fourier Series

A TRIG O N OMETRIC SERIES is a series of the form

IS

�O + L an cos nt + L bn sin nt , t E R, neN

neN

(an) n>O

and (bn ) n > 1 are complex sequences . It is easy to convert where the the nth partial sum n n Sn + k cos kt + bk sin kt = k= l k=l

(t)

La

�

L

of the trigonometric series into exponential form n Sn = k= - n

(t) L Ck e ik t

by taking bo

=

0, and for all k 2: 0,

Ck = a k -2 ibk

an d L k -

_

a k + ibk

(4.4)

2

ak

To convert from exponential to trigonometric form, use Ck + L k and bk = - L k ) . Generally, one says that a two-sided series (or from a normed space CON VERGES to the of vectors = x . This is not what is customarily vector x if limn , k -+oo L� = - k done in the special case of the series L e however. We say that converges (somehow) t o s ( t ) if the symmetric partial sums = L�= converge to (t) as --> 00 .

i (Ck eries) Ln eZ Xn

Xn

Xm

Ln eZ cn e i n t Un (t) n Ck e ik t

s

bis

n Z cn e i nt , n

Definition 4 . 1 . 1 FO URIER SERIES AND COEFFICIENTS

If

I E L1 [-11", 11"] , then

1 1 "" I (t) e - z.nt dt, n E Z,

I ( n) = 211" �

_ ,,"

I,

( 4.5) e i nt I

is called the nth FO URIER COEFFICIENT for and L n e Z fen) is called the EXPONENTIAL (or COMPLEX) FORM OF THE FO URIER SERIES FOR Another name for fen) is the FINITE FOURIER TRANSFORM of evaluated at (cf. Section 5 . 1 ) . Equivalently,

n

an = :;1 1 "" I(t) cos nt dt, n E Nu {O} _,,"

and

1 I"" I(t) sin nt dt , n E N,

bn = 11"

_ ,,"

I.

4. Fourier Series

are called the nth FOURIER COSINE AND SINE COEFFICIENTS F O R spectively, and

145

f, re

f (t) = � + L an cos nt + L bn sin nt nEN n EN

is also called the TRIG ONOMETRIC FORM O F THE FOURIER SERIES F O R are related to /( n ) by equation ( 4.4) . 0 The and

an

bn

f.

Even though the exponential form provides the most symmetric, elegant formulation of Fourier series, we use the trigonometric form throughout this chapter because we think that it is easier for the first-time reader. Fourier series provide highly effective means to investigate periodic func tions. Using them, you can dissect a sound into component parts called harmonics; the process is known as harmonic analysis. The is the neutral position, cos sin sin the fundamental tone , cos which go to 0 with the first octave and so on. The amplitudes and increasing n, determine the imp ortance of the overtones in toto. In essence, Fourier analysis of periodic phenomena enables reduction of the problem to determining the response to one harmonic at a time , then taking a limit of a linear combination ( superposition ) of these . Not only that , as van Vleck 19 14] observed:

al t + bl t

a o /2 a 2 2t + b2 2t bn,

an

[

. . . we mathematicians may well query . . . whether it may not have conditioned the form of physical thought itself-whether it has not actually forced the physicist often to think of compli cated physical phenomena as made up of oscillatory or harmonic components , when they are not inherently so composed . To what extent did Fourier series accelerate the development of the wave theories of modern physics? Who would have thought of an electron as a wave? We compute some Fourier series for functions f E 11"] . Since

L2[ - 1I", I } U { sin nt , cos nt .. n E N } B {_ y 1l" y 1l" y 211" -

ro=

;;;

;;;

11"] ( 3 .4.6 ( c )) , any is an orthonormal basis for written as the infinite series of its projections ,

L2[-1I",

f E L2[-7I",

f = L (f, e } e .

)

eEB

The terms of the series are of the form nt (f, e ) e - I\ f ' cos .Ji -

t

cos n .ji or

I\ f' sin nt ) sin.jint . .Ji

] may be

11"

146

4. Fourier Series

For historical reasons and ease of notation, we deviate from the conven tion (3.4.8(b)) about what we call the Fourier coefficients ) of f in the cos Hilbert space 11-]. Instead of (f, ) ft = we change the normalizing factor from 1 / J1i to 1/11". As in Definition 4 . 1 . 1 , w e take as the Fourier coefficients of f

(f, e e = ( /, e t ) J;r D" / (t) nt dt,

L2[-1I",

1 an = ;:

and

f"" f(t) cos nt dt, L

1 bn = -

11"

1

"

_"

n E NU {O}

f (t ) sin nt dt, n E N .

(4.6)

--+

(4.7)

It follows from the Riemann-Leb esgue lemma 3.3.2 that an , bn 0; in deed, by Parseval's identity 3.3. 1 , the sequences (a n ) and (bn ) are square summable (belong to £ 2 (00)). The Fourier series for f E is then the 1 I · l b-convergent series

L2[-1I", 1I"]

�o + L (an cos nt + bn sin n t ) .

(4.8)

nEN

The a n and bn are unique by 3 . 3 .4(b) ; if (an ) and (bn ) are square-summable sequences , then the trigonometric series of equation (4.8) is the Fourier series of some E by the Riesz-Fischer theorem 3 3 4 (a)

/ L2[-1I", 11"]

.. .

Historical Note THE MEANING O F DEFINITE INTEGRAL In 1 807 there was controversy about the meaning of the definite integrals in equa tions (4.6) and (4.7) . At the time, determination of the antiderivative to compute area was deemed essential. Since an antiderivative cannot always be found in terms of elementary functions, as with f d ? It was for example , what meaning should be attached to f12 Fourier's idea to bypass the antiderivative and define the definite in tegral as the area under the curve . Dirichlet later realized that not every function could be integrated even in this sense, that not every function has a meaningful area between it and the horizontal axis ; the characteristic function l Q of the rational numbers Q, now known as the DIRICHLET FUNCTION , is an example of such a function. 0

e -t 2 t

e - t 2 dt,

We always get something extra with Fourier series. If we start with a function f defined on [- 11" 11" , the Fourier series automatically produces a periodic extension of f because of the periodicity of the sine and cosine. Our attitude to this largesse is this: As long as it converges somehow to on [- 11", 11"] , who cares what it does elsewhere? Let us get some concrete experience .

, ]

f

4. Fourier Series Example 4. 1 . 2 SQUARE WAVE I

function

f

O,

(t) = {

147

Compute the Fourier series for the step

1r t

- :5 < 0, . I , 0 :5 t < 1r.

[-1r, 1r]

Solution . As noted above, the Fourier series for this step function on actually converges to a square wave on R. By equation (4.6) , its cosine = coefficients are 1 and "" = = 0, = -1 f( ) cos cos

ao

an 1r 1

t

_ ,,"

nt dt -1r1 10 "" nt dt

n E N.

cos n1r 1 = bn = -1 1 "" sm. nt dt = 1 - n1r n1r ( I - ( - l t ) , n E N .

Its sine coefficients are 1r

0

The Fourier series of f is therefore

�+� � L...J

sin nt (1 - (- It)

�+� � L...J

n - l) t .

sin (2 2n - 1

(4. 9) 7r n e N This converges to f in the Hilbert space Lli[-7r, J i.e . , it converges to f in the mean. What does it converge to pointwise? That depends on the point t. By the pointwise convergence theorem (4.6.2) , this series converges to f (t) at points of continuity. At t = 0, it converges to 1/2, the average of 2

1r ne N

n

=

2

7r ,

the limiting values f (0 - ) and f (0 + ) . Also (and this is generally true) the series converges more slowly at points where the function is less smooth, at points of discontinuity or sharp corners . 0

t -t - (t) .

t,

A real-valued function f is EVEN if f ( ) = f ( ) for all i.e . , if it is symmetric about the y-axis, as etc. , and cos t are , for example ; a real-valued function f is ODD if J ( ) = f Linear combinations of odd powers of t are odd and so are linear combinations of sin where is an integer. It is trivial to verify that the product of two even or two odd functions is even; only even x odd odd. Equally trivial to verify is the fact that the integral of an odd function over a symmetric interval is If f is an odd periodic function with period then its integral over any interval of length p is 0 (4.2.2) . The practical use of these observations is that the sine coefficients of any even function are 0, as are the cosine coefficients of any odd function .

t 2 , t4, -t

kt

=

O.

p,

Example 4 . 1 . 3 SQUARE WAVE II Find 9

( )=

t

{ - I, I,

the Fourier series for

- 1r t t

< < 0, 0 < < 7r .

k

[-c, c]

148

4. Fourier Series

an

Solution . Since 9 is odd, the cosine coefficients are all 0, whereas '11" 2 2 = - ( 1 - cos sin n 0 odd, n even . 0,

n1r) n1r n1r ' n sin (2 n - 1 ) t · serIes lor 9 IS t here£ore 4 '" Th e FO U rIer . ;: L..m E N 2n 1 bn = -1r 1 g (t) .

l"

t dt

{�

•

_

-4

0

4

A square wave and

84

Remark LINEARITY O F FOURIER COEFFICIENTS AND SERIES

Since the integral operators .; ( (cos or sin n are linear , the Fourier cosine (for example) coefficients of + for 9E any scalars a and b, are given by

J�'II" ) -

nt t) dt en af bg, f, L2[-1r, 1r],

dt + -b 1 '11"'II" g(t) cos nt dt = aan + ba� , where an and a� are the Fourier cosine coefficients of f and respectively. It immediately follows that the Fourier series for af + bg is a (Fourier series for J) + b (Fourier series for g). 1r 1

�

-'11"'II"

/(t) cos nt

1r

g,

X

x

This observation can sometimes save some computation . 0 Example 4 . 1 .4 SQUARE WAVE I I I

f (t) = {

O,

- 1r < t < °

1, ° < t <

1r

Show that the Fourier series of

- l) t .

. 1 2 " sin (2n + 2 2 -1 EN -; n

IS

L.J

n

4. Fourier Series

149

Sol ution . Forget Example 4. 1 .2 where we computed the Fourier series of the step function for the moment. Let us use the fact that we know the Fourier series of the square wave 9 of Example 4 . 1 .3. We can use that and linearity to get the Fourier series for Since = l ( 1 + its Fourier + series is given by l l (Fourier series of or

f

f.

g ),

f

g)

!2 + �71" L...tn EN sin 2(2nn--1 1) t . "

0

Example 4 . 1 . 5 AN EVEN SAWTO OTH Sh ow that the Fourier series for is t - ;. ( cos + CO;23t + CO;25 t + · · l on

[ -71", 71"] t Solutio n . Since f is even, its sine coefficients b n are 0 for every n E N . The cosine coefficients are an = -71"2 1'"0 t cos nt dt. f (t) = I t I

Integrating by parts , we obtain

2 2 cos n 7l" - 1 ) an = 7I" n (

=

{

-4 -2 ' n odd, 7I" n n even, 0,

a o = ( 2 / 71" ) fo'" t dt = 71". The Fourier series for I t I is therefore cos 3t + cos 5t + . . 4 -. . 0 - - - ( cos t + -2 71" 32 52 ) Remark PERIOD 2p INSTEAD OF 271" Of course, there is nothing special about [-71", 71"] . If [-p, p], p > 0, is any closed interval and if f E L2[- p, p], the Fourier coefficients for f are given by 1 P n7l"t (4. 10) a n = - j f(t) cos - dt, n E N U {O} p P and

71"

-P

P n7l"t bn = -p1 j f(t) sm. dt, n E N . P -P The Fourier series for f is ao + (an cos n 7l"t + bn sin n 7l"t . L P P ) nEN and

2

{ t,-6 - t

(4. 1 1 )

0

(4 . 1 2)

Example 4 . 1 . 6 AN ODD SAWTO OTH Sh ow that the Fou rier series for the sawtooth < < -3' ' = -3 � � 3 , 3 3 ". is 2 4 (sin ". t ..!.. sin t + ..!.. sin !1l!:! on )

f (t)

[- 6 , 6]

".2

6

- 32

-6 t t 6 - t, 5: t 5: 6, 52 6

6

.

.

•

.

150

4. Fou rier Series

2

A saw-tooth Sol utio n . Since f is odd, we need only compute the sine coefficients bn since f (t) sin [( rt) /6] is even, we can double the expression in equation (4.11) and integrate from 0 t o 6: 26 [ 6 f (t) sm. mrt dt bn 10 fi 1 [3 t sm. mrt dt + 1 [ 6 (6 - t) sm. n7l"t dt, n E N . a la fi a 13 fi With w = 7I"t/6, 12 1 '1r (71" - w) sin nw dw. 12 1 '1r / 2 w sin nw dw + 2" bn = 2" 71" a 71" / 2 j

m

Integrating by parts and simplifying gives

'Ir

24 . n 7l" bn = 22 7I" n sm -2 '

24 ( . 7I"t 1 . 371"t 1 . 571"t 71" 2 sm "6 - 3 2 sm (; + 5 2 sm (; - . . .) . 0 Example 4. 1 . 7 A PARABOLIC CUP Sh ow that the Fourier series for f (t) = t 2 on [-71", 71"] . + 4 " L.me N ( _ l)Rn cos n t so the Fourier series for the sawtooth is

IS

Solution .

"3 'lr 2

2

•

271"2 and an = -I j'lr t 2 cos nt dt for n E N . ao = -71"I j'lr t 2 dt = 3 71" - 'Ir

- 'Ir

4. Fou rier Series

=

151

Integrating the latter by parts yields

-2 an n 1'" t sin nt dt . 7r

_ ...

Another integration by parts yields

2 1'" cos nt dt 4 (- 1t t n ... n n Since 1 is even , each bn is O. Thus, the Fourier series for I (t) is " (- 1 r cos nt 7r 2 +4 L.J 3 n2 . nEN an

=

2

-2- cos t L ... n

7r

-

27r

=

_ ...

2

0

Exercises 4 . 1

1 . EXP ONENTIAL FORM

Ln EZ cn e i nt OF FOURIER SERIES

(a) Verify the relationships of equation (4.4) . (b) FINITE FOURIER TRANSFORM Show that if 1 E L1 even or odd, then

=

[

]

- 7r , 7r

is

I( t ) e - int dt , n E Z , ... is real or purely imaginary, respectively, for all n E Z. j(n)

� I:.

2

2. AMPLITUDE-PHASE FORMULATION

Show that the series

� + L an cos nt + L bn sin nt nEN n EN

may be written in AMPLITUDE-PHASE form

!'f +

L dn cos (nt + ipn ) . nEN The coefficient dn is called the AMPLITUDE and i.pn the PHASE (PHASE ANGLE) of the nth component . Express an and bn in terms of Cn and ipn and vice versa. 3. SHIFTS AND MULTIPLES Let 1 E L'2 [- 7r , 7r] . (a) SCALE CHANGE T

=

D ILATION Given the Fourier series

+ L an cos nt + L bn sin nt nEN nEN

for the function 1 and some constant C series for 1 (

ct)?

# 0, what is the Fourier

152

4. Fourier Series

)

I 2, Ln EN dn (nt if'n) I (t a) � + L dn cos (nt + [if'n + na]) . n EN

ao/2

(b TIME SHIFT If the function has the Fourier series + cos + of Exercise show that the Fourier se ries + is given by

4.

I (t) = {

Use the linearity of Fourier coefficients as in Example pute the Fourier series of

io,

O� � � �. O ,

4.1.4 to com

5 . Find the Fourier series of the following functions. (a (b (c (d

6. 7.

) ) ) )

I (t) = 0 for -1r ::; t < 0 , I (t) = t for 0 ::; t ::; 1r. I (t) = t for - 1r ::; t ::; 1r. I (t) = t 2 for 0 ::; t ::; 21r. I (t) = 112 (1r 2t - t3 ) , -1r ::; t ::; 1r. IF ( note: IF; cf. 4.6.2) the series converges pointwise, show that ;� 1 ;3 + 5\ - . . . . =

-

Find the exponential Fourier series expansions of:

( a) I (t) = at/21r, 0 < t < 21r, where (b ) I (t) = t,
- 1r

a

is constant.

DERIVATIVES AND INTEGRALS OF PERIODIC FUNCTIO NS Let the function be p-periodic.

I

) ) If I

I

I' be pe (x) = f: I(t) dt

(a Assuming that is differentiable , must the derivative riodic? (b is integrable over all closed intervals is F periodic? What if

f: +P I(t ) dt =

8 . Find the Fourier series for

O?

[a, b],

I (t) = { 01 ,, � << tt ::;< tl '. 2

9 . Find the Fourier series for the triangular wave

2n - 1 ::; t ::; 2n, , n E N . I (t) = { t2n -2nt - t, 2n ::; t ::; 2n + 1 , -

-

2'

1 0 . PARSEVAL'S IDENTITY Energies of waves are related to the squares of their amplitudes. The Parseval identity equation (3.9) of 3 .3 . 1 ) is useful in evaluating these. Let

( I E L2[-1r, 1r). ( a) Show that � r:.,. I / (t) 1 2 dt = a2� + Ln EN (a� + b�) , where an and bn are the Fourier coefficients of I of equations (4.6) and (4 . 7) .

4. Fourier Series

(b) With

i (n)

1 ( a) , show that 2 2� f:'7r I/ (t) 1 2 dt = Ln EZ Ii (n) 1 .

as in Exercise

(c) Use the result of Exercise 5(c) to show that 7r4 /90

11.

153

= L n EN 1/n4.

Does there exist an I E L; [- 7r, 7r] whose Fourier coefficients are : (a) (b)

an = bn = lifo with a o = O? an = bn = l/ n with ao = O?

(t) = (-t)) 12.

12. Any function I can be written as the sum of an even function Ie -I +I /2 and an odd function 10 = For any I E L2 [-7r, 7r] , show that

(f (t)

(-t))

� f:'7r P (t) dt

=

(t) (f (t)

� f:'7r {; (t) dt + � D7r {; (t) dt.

Does this relationship among squares look familiar? Can you explain it? 1 3 . PRODUCTS Let I, 9 E L; [-7r, 7r] have Fourier cosine and sine coeffi cients for I and for g , respectively. Show that

(an , bn )

(cn, dn)

Since we have considered Fourier series only for functions from L; [-7r, 7r] , the Fourier series for the product Ig falls outside our purview-it may not be square-summable-for now. The product of two L; functions is an Ll function, however, and we consider Fourier series for Ll functions in Section 4.4.

t,

n 1) t, . . . is an orthonormal

14. Show that sin sin 3t , sin 5t , . . . , sin (2 + basis for L; [O, 7r/2] .

1 5 . PERIODICITY If there exists a smallest period Po > 0 for a function Which of the I, then Po is called the FUNDAMENTAL PERIOD of functions below is periodic? If it is periodic, what is the fundamental period? (a) sin 7rt (b) sin tl5 (c) sin 3tl5 (d) sin 3t + cos 5t (e) sin tl5 + sin tl3 (f) I (t) = sin lit , t # 0 , and 1 (0) = 0 (g) e it + e i 7r t .

f.

16 .

EXISTENCE O F FUNDAMENTAL PERIOD Let I be a nonconstant con tinuous function on R. Show that if I is periodic with period then I has a fundamental period.

p

154

4. Fourier Series

Hints/ Answers

dn 'Pn ), an = dn 'Pn , dn Ja� b� 'Pn = (an/dn). n sinnn-t · " EN cos(n2n -1?1)t - " 5 A nswers. ( a) ". - ;-2 wn wn EN ( - 1) (2 (b) 2 (sin t - t sin 2t + i sin 3t - � sin 4t + . l " E Z - {O} ( -ni ) " ei nt . 6(b) A nswer: . wn 8. See 4 . 2 . 1 ( c). 9 wn EN cos(( 22nn--1 1)�".t . 11. Use 3 .3.4. 1 2. It looks like the Pythagorean relation: 11/11 2 = II g l1 2 + II h ll 2 . If M denotes the subspace of even functions, show that M 1. is the subspace of odd functions. Conclude that L5[-1r, 1r] = M � M 1. . 14. For any 1 E L5[O, 1r/2] orthogonal to each of these functions, extend 1 so that it is even with respect to /2 (i.e . , symmetric about the line t 1r /2) on [0, 1r]. Then extend this function to an even function on [ - 1r, 1r] and compute its Fourier coefficients. 15. Answers. (c) 101r/3. (d) 21r. (e) 301r. (f) not periodic. (g) Note that e it and e i". t are linearly independent . 1 6 . If 1 doesn't have a fundamental period, let (Pn) be a decreasing se quence of positive numbers decreasing to 0 such that 1 (t + Pn) = 1 (t) for each n and all t. Then 1 (t + mpn) = 1 (t) for all integers m, n � 1. For a given s, let mn be the largest integer such that mnPn :::; Then I - mnPn l :::; Pn. Since 1 is continuous and Pn 0, conclude that 1 (t + s ) = 1 (t) for a fixed t and arbitrary s . This contradicts the nonconstancy of f. 2.

cos Given the amplitude-phase series (the and the sin Conversely, = + and cos - 1 This is easy to remember by drawing a right triangle.

bn = -dn 'Pn . 4"

·

·

•

·

Z

=-1 " 2 ".

1r

=

s

s.

4.2

-+

Fourier Sine Series and Cosine Series

(4 . 10)-(4.12), if a square-summable function 1 is [- p, p], its Fourier series is given by n1rt + bn sm. n1rt ) , aO L..J ( an cos "2 + '" (4.13) P P nEN

As noted in equations defined on an interval

4. Fourier Series

155

mrt dt, n E N U {O} , (4 . 14) an = p-I jP f(t) cos P -P and mrt dt, n E N . (4. 15) bn = p-I j-P f(t) . P P If f is defined only on [0, p], we can extend f to [-p, p] as an even or as an odd function or, depending on f, neither (see Example 4 . 2 . 1 (c )) . If we consider the ODD EXTENSION fo-taking fo on [-p, O] to be the negative of what f is on [0, p] and taking its 2p-periodic extension-then all the Fourier cosine coefficients an are 0, n E N u {O} , and equation (4. 13) is a series consisting only of sines, a FOURIER SINE SERIES for f: where

Sill

f

Likewise , if had been extended as an even function, then all the sine coefficients drop out, and we get a FOURIER COSINE SERIES representation for We illustrate these HALF-RANGE or HALF-INTERVAL EXPANSIONS next .

f.

Example

4.2.1

SINE AND COSINE SERIES Expand the fun ction

f (t) = { 0,I, � << tt << �1,, 2

.

a s (a) a Fourier sine series; (b) a Fourier cosine series; and (c) a Fou rier serzes.

{,

Solution . (a) Fou rier sine series. The odd extension of

f is

t _ 12 ' fo (t) = -1, 1 , 0 <�
IS

-1 < < -

2'

even ,

1 11 2 mr ( 1 - cos -) . bn = 1- I fo(t) sin mrt dt = 2 0 / 2 sin mrt dt = mr 2

We get the following pattern

b1 = -11"2 , b2 = -11"2 , b3 = -311"2 , b4 = 0, . . . .

156

4. Fourier Series

Therefore, the Fourier sine series

(

I is

-- --

)

2 sin 271"t sin 371"t sin 571"t 2 sin 671"t . . . 2 . sm 7l"t + . + + + + 71" 2 3 5 6

I t E �) , t E ((-�, - I, - V U (�, I).

(b) Fourier cosin e series. The even extension of is �

Je

_

(t) -

{ 0,

1,

f; / 2 dt = 1 and 1 1 / 2 cos n7l"t dt = -2 sin 11 n7l" , E N. an = 2 /e (t) cos n7l"t dt = 2 2 n n7l" o 0 The Fourier cosine series for I is therefore The sine coefficients bn are all 0 ; a o = 2 fol le (t) dt = 2

!2 + !71" ( cos1 7I"t _ cos3371"t + cos5571"t _ . . .) .

(c) Fourier series. The periodic extension of I happens to be neither even nor odd with period 2p = 1, so 2n7l"t d 1 1 / 2 cos 2n7l"t dt , n E Nu {O} , 1 1 / 2 I(t) cos -t=2 1 0 - 1/2 1 1 / 2 sin 2n7l"t dt , n E N . 1 11 / 2 2n7l"t I(t) sin -- dt = 2 bn = / 1 2 - 1/2 1 0

1 an = 1/2 and

This yields the Fourier series

(.

)

sin 671"t sin 107l"t sin 1471"t . . . . 0 2 1 + - sm 2 7I"t + -- + + 3 5 7 + 71" Lemma 4.2.2 shows that if we integrate a 2p-periodic function over any interval of length 2p, we get the same value as over any other such interval. If is continuous, it is easy to prove this by differentiating:

I

lx + 2P /(t) dt = I (x + 2p) - I (x) = 0 for any x , x from which it follows that f: + 2P I(t) dt is constant. More generally, d dx

4.2.2 INTEGRAL SHIFT Iff is 2p-periodic and int egrable over [-p, p] , then for any real numbers a and b,

I b /(t) dt = I b+2P /(t) dt and jP I(t) dt = la + 2P I(t) dt . a +2p a a -P

4. Fourier Series

157

Proof. For any a and b, with s = t +2p and the fact that f ( s - 2p) = 1 ( s ) , b I(t) dt = b+2P f(s - 2p) ds = b+2P I(s) ds , Ia+2p Ia+2p Ia which proves the first equality. Now replace a by a - 2p and b by a to get then let

a = -po

b+2P ra+2P la+2p = la ;

0

Exercises 4 . 2

( (t) = t

1. Find a) the Fourier cosine series and (b) the Fourier sine series for 1 on [0, .

2.

'11"]

Find the Fourier (a) cosine series for I (t) = sin t , 0 0 t p. (b) cosine series for f (c) sine series for = a constant, for (d) sine and cosine series for 1 = 0

(t) = t 2 , I (t) c,

3. HALF-RANGE COEFFICIENTS general form of the coefficients (a)

(b)

0 � t � p.

(t) t, < t < 2. If f i s defined o n [O, p], determine the

an in its cosine series expansion. bn in the sine series expansion .

4. SHIFT BY

'II"

Suppose you know that the Fourier series for f E

L2[- , '11"] is ao/2+ E n eN an cos nH E n eN bn sin nt. Find the Fourier series for 9 (t) = 1 (t - '11" ) . EXP ONENTIAL SERIES For any constants a and p, find the exponen 'II"

5.

� t � '11" . � �

tial Fourier series (Exercise 4.1- 1) for (a)

(b) (c)

I (t) = at/2'11" , 0 < t < 2'11" . I (t) = t, < t < '11" . I (t) = at/2p, 0 < t < 2p. - 'II"

L2[-'II" , '11"] , ao Ibn I � ao I an I bn '11") , I I � n b for

6. SIZE OF FOURIER COEFFICIENTS For nonnegative f E show that the Fourier coefficients satisfy and � for all n E If 1 is odd and f � 0 in (0, then n . ..

= 1, 2,

N.

.

1

158

4. Fourier Series

f (t) = cos at where a is not an integer. ( a) Show that the Fourier series for f is

7. Let

Substituting t

= 7r and dividing by sin a7r,

2 a ( 2 \ + �12 + 2 1 2 2 + . . .)

cot 7ra = With w

we obtain

a

".

a -

a

-

.

= 7ra we get _n + _1n_ ) , cot w - w + " ( L-nEN w - ". w + ". 1

.!.

-

_

which is the partial fraction expansion of the cotangent. (b ) Use the expansion in a) to show that

(

_si n1_w -_ w + " ( _ I) n ( _ L-w - n ". + w +n ". ) . nEN 1

.!.

__ 1_

_

Hints/ Answers 1

(

1r. . a) 2

n n _ .i". " L--n E N cos(( 2n2- -1)21 )t . ( b ) 2 " L--n EN ( _ 1) + 1 s i nnn t .

2. Answers: ( a) - .i ( COS 2t + cos 4 t + cos 6 t + . . . ) 2-n1t · Note - ". _ ". L-- n EN cos n 1 ". ". 3 2 5 35 4 that this returns I sin t I . (b) � - � 3 ". 2 ( cos ".Pt - 4 cos 2".tP + cos 3". t _ . . .) . (c ) 4".c ( sin ". t + sin 3". t + 5 sin 5 ".t ). P 3 P P (d ) cosine series·. 1 - ". 2 ( cos 2 + 32 cos 3 ".t + 52 cos 5 ".t + . . . ) . 2 2 3. (a) a n = foP f(t) cos u t dt, n = 0, 1, . . (b ) bn = foP f(t) sin u t dt, P P P n N P 4. The cosine coefficients are ( - 1 r an the sine coefficients are (-1 r bn. 5 . (a) Use 4.2.2 to assert that Cn = 21". !�". f (t) e - i n t dt . in ". /p in ( b ) i I:nEZ - { O } (-�t e t . ( c ) Cn = (1/2p) !:' p f (t) e - t dt. 6 . For the second statement, use the fact that I sin nt l ::; n i sin t l . �

- �

1

E

.

£

J(

1

..§..

1 9

1

".t

P

_

.

.

.

...L

...L

.

;

.i "

.

£

J(

4. Fourier Series

4.3

159

Smo othness Not long ago many thought that the mathematical world was created out of analytic functions. It was the Fourier series which disclosed a terra incognita in a second hemisphere. -E. B . van Vleck , 1 9 1 4

1

For I E L [ - 'II" , '11"] , what, if anything, does the nth partial sum

a; + L an cos nt + L bn sin nt nEN n EN

( 4. 16)

of the Fourier series for I converge to pointwise? In many cases-wave propagation , for one-it is important to know how good the approxima tion of the wave by its harmonics is at specific points, i.e . , to know about pointwise convergence. The problem of pointwise convergence of Fourier series is an old one . Fourier first used such series in his Th eone de la propaga tion de la ch aleur dans les solides (reproduced with comments in Grattan-Guiness 1972) , a manuscript submitted to the Institut de France on 2 1 December 1807 , to a committee consisting of Laplace, Lagrange , Lacroix, and Monge. (Whew! Poisson was only the committee's clerk .) Lagrange questioned the conver gence of such series. Fourier offered the following trigonometric series for I (t) = 1 on [- 1 , 1] 'll"t 1 3'11"t 1 5'11"t 1 ht cos "2 - 3" cos 2 + 5" cos 2 - 7 cos 2 + . . .

(

4 . 17)

How could this converge for all t? A series of real numbers converges ab solutely if and only if it converges unconditionally (see 3.6. 1 and Exercise 3.6-3) . Since the series of absolute values of the coefficients of the series of equation ( 4 . 17) , 1 1 1 1 + - + - + - + · · · ----. 00 3 5 the terms cannot be rearranged-if the series converges at all, it can only do so conditionally. Not only that, in Example 4 . 1 . 3 we computed the Fourier series of the square wave

'

7

I (t) = as

±

{

'"

'II" nL.J

-1, 1,

EN

-'II" < t < 0 , 0 < t < '11" ,

sin (2n - l ) t . 2n - 1

160

4. Fourier Series

does not even con verge conditionally. Though the others would probably have consented to the publication of Fourier's article, Lagrange was adamant. Simeon Poisson tersely (5 pages) summarized the committee's findings in 1808 which were as follows: Re jected on the grounds that it contained nothing new or interesting. Fourier went back to his desk . He submitted a revised version, M emoire sur la prop agation de la chaleur, to essentially the same group of j udges at the end of 1 8 1 1 as a candidate for the Grand prix de matMmatiques for 1 8 12. Though they awarded him the prize , their reservations were such that the article was not published in the Memoires de l 'A cademie des Sciences. Fourier resented this bitterly. In 1824 Fourier became Secretary of the Aca demie and published his 1 8 1 1 paper , practically unchanged, in the Memoires. Nevertheless, the problems about convergence of his series were serious . Poisson published an erroneous argument about their convergence in 1 820 . Fourier kept trying, and Cauchy published a flawed proof in 1 826 . In 1829 the 23-year old Prussian mathematician of French origin, Peter Gustav Lejeune Dirichlet pointed out Cauchy's mistake (a rather courageous act , considering Cauchy ' s influence and crotchetiness) ; Dirichlet gave sufficient conditions for pointwise convergence of Fourier series. After Gauss's death in 1855 , Dirichlet succeeded Gauss as professor of higher mathematics at Gottingen. It is essentially Dirichlet's proof that we give in 4.6.2. It asserts that the Fourier series of a piecewise smooth integrable function converges at each point to + 2 Hence , the Fourier series converges to at points of continuity and to the average of the limit Ing values at a jump discontinuity. We look at Jordan's ( 1 881) improvement of Dirichlet 's pointwise convergence theorem in 4.6.6: If the piecewise smoothness condition on is weakened to bounded varia tion (Definition 4.3 .4) , its Fourier series still converges to t + at every point t. There is no hope of weakening the smoothness condition to mere continuity. As we show in Section 4.9, there are continuous functions whose Fourier series diverge at certain points. Indeed, there are continu ous functions on [- '11' , '11'] whose Fourier series diverge on a dense subset of [- '11' , '11'] . Of course, since such functions cannot even be of bounded varia tion, they are pretty wild. The key to convergence of the Fourier series of the integrable function is the smoothness of The smoother is (i.e . , the more continuous and its derivatives are ) , the better the convergence of its Fourier series to If is merely continuous at a point, its Fourier series may diverge, but (4.6.2) if is continuous and piecewise smooth, the series converges uniformly and absolutely to (4.7.4) . The convergence is also faster if the function is smoother. The essential content of 4 . 1 2.3 is that if is continuously differentiable k times, then its Fourier coefficients go to 0 faster and than

f

t

f (t + ) f (t - ) f (t) f

f.

f

(f (t - ) f (t + ))

f f f. f

f

f

li n k.

an

f

bn

161

4. Fourier Series

Before demonstrating the benefits of smoothness in regard t o conver gence, let us look at the other side of the coin first, at the deleterious effect of discontinuity on pointwise convergence. Dirichlet invented the function 1 Q of Example 4 .3 . 1 , which is as discontinuous as a function can be and its Fourier series has correspondingly atrocious pointwise convergence behav ior . As a comment on Dirichlet ' s bold ingenuity, at the time of his invention entities like 1 Q were not even considered to be functions. It forced math ematicians to confront the question: What is a function? EXaIllple 4 . 3 . 1 AWFUL POINTWISE CONVERGENCE

(

The Dirichlet FUNCTION not the Dirichlet kerne0 1Q

�atio.nal, E Lr2 [-1I" , 11"] (t) = { 0, tt IS�s IrratIOnal, I,

has a Fourier series that does not converge pointwise to the function at any rational number. Since we can disregard the set Q (of rational numbers , which has Lebesgue measure 0,

an = -11"1 1 " 1 Q ( t ) cos t dt = -11"1 1Q _

n

"

n [ - " , "l

l

)

td

cos n t

=O

,

n

E N,

and O. Likewise , 1 Q (t) sin n 0 for all n. The Fourier .;. series is therefore identically 0 and does not converge to 1 Q (t) at any rational number t. 0

ao =

bn = J�"

t dt =

[1914] n cnx

As E. B . van Vleck put it, " . . the power series obtained by com bining terms of the form define the most civilized members of math ematical society-the so-called analytic functions-which are most orderly in their behavior , being continuous throughout their domains, possessing derivatives of all orders." Fourier series can represent some of the most pathological elements of mathematical society, even those schizophrenic functions that are continuous everywhere but differentiable nowhere. Weier strass proved that functions defined by series of the form

f

f (t) = n=L bn sin (an t) , 0 < b < I , a E N, ab > I , 00

O

(

(4.18)

are continuous everywhere by Weierstrass's M-test, the series converges uniformly but differentiable nowhere this is not so easy to prove . To see how wildly these functions oscillate, consider the graphs of some partial sums of

Sn

)

£; 00

( l )k 2

(

sin

)

(5kt) .

( 4. 19)

1 62

4. Fourier Series

86

(t)

As we discuss in Section 4.1 1 , convergent trigonometric series

�o + L an cos nt + L bn sin nt neN neN and Fourier series are not the same thing; there , we give an example of a convergent trigonometric series that is not the Fourier series of any function in [ -11", 11"] . But the series of equation (4.19) is a Fourier series (by 4.7. 1 , any uniformly convergent trigonometric series is a Fourier series) . Even so, wild as the limit is, the Fourier series represents it deftly ; the Taylor series cannot touch it. For the sake of the pointwise convergence theorems to follow, we need to elaborate on the theme of smoothness. Specifically, we need to discuss piecewise continuity, one-sided derivatives, and functions of bounded vari ation.

Ll

4. Fourier Series

163

Definition 4 . 3 . 2 PIECEWISE CONTINUITY A N D SMOOTHNESS

I

[a, b]

A real-valued function defined on the closed interval except for possibly a finite number of points is PIECEWISE (SECTIONALLY ) CONTIN UOUS if it is continuous at all but a finite number of points and the right and left-hand limits and exist at every point except the endpoints where only and must exist ; in other words , is continuous except for a finite number of finite jump discontinuities . We de note the space of all such functions by If is composed of a finite number of differentiable components and f' and f' exist at every point E [ a , except the end points where only and must exist , we say that I is PIECEWISE SMOOTH; in other words, is piecewise 0 smooth if f ' E a,

c

I (c+ ) I (c - ) c E [a, b], I (a + ) I (b - ) I, PC[a, b]. I (c+) +(c-) b] I' (a ) f' (b-) I PC [ b].

Step functions are piecewise continuous-indeed, they are piecewise smooth. In fact , any function that is continuous except for a finite number of jump discontinuities is piecewise continuous. The definition of piecewise conti nuity rules out highly oscillatory functions such as sin ( l ft) for '# 0, 1 (0) 0 , on any interval containing 0 ; f is not piecewise continu ous on because it does not approach a limit as ---+ 0 from either side. The function 9 1 ft for 0, 9 (0) = 0 on [0 , 1] is not piece wise continuous for the same reason: 9 does not exist . By breaking the closed interval a , up into an appropriate finite number of subintervals , it is easy to see that c for any � 1 . Since the step functions are dense in (Example 2.4.5) , it follows that is dense in [a , as well. It is simple to show (basically, you connect the j umps with steep , but not vertical, straight lines) that piecewise smooth functions are dense in , too.

t

= [ - 11', 11']

t f. (0 + ) [ b] PC [a, b] L; [a, b] Lp [a, b] L; b] L; [a b],

I (t) = t

(t) =

p

PC [a, b]

Definition 4.3.3 ONE-SIDED DERIVATIVES

Let I be defined in an interval containing TIVE of at a point c is

I: (c) I

.fI

Jr

c. A RIGHT-HANDED DERIVA

( ) - hl....' O+ I ( c + h)h - I ( c ) . C

_

un

(4.20)

Ii. (c) . Remark A SUBTLE DISTIN CTION f' (c + ) f. I: (c) If I is piecewise smooth on the closed interval [a, b], then I' is continuous at all but a finite number of points of [a , b] , and f' (c + ) and f' (c - ) exist at every point c E ( b) . Note the distinction between the right-handed derivative I: ( c) and f' ( c+): f' (c + h) . I: (c) f. I' ( c+ ) = lim .... O+

A similar definition applies to LEFT-HANDED DERIVATIVES

a,

h

0

164

4. Fou rier Series

I, as we discuss in Exercise 1 , · I(c + h) - / (c+) ' 1' ( c+ ) - 11m h-+O+ h 0 for i .e . , with I (c+) in the numerator instead of I (c) . If I (t) t < 0 and I (t) 1 for t > 0 , then f' (0+) and f' (0- ) both exist ; neither fl (0) nor I: (0) exists because I is not even defined at O. We For piecewise smooth

=

=

say a little more about this in Exercise 2.

Dirichlet's 1829 pointwise convergence theorem 4.6.2 shows that the Fourier series of a piecewise smooth periodic function converges to I at every point Dirichlet believed that weaker conditions would suffice , but was unable to prove it. Jordan took up the cause and focused his attention on functions that were not smooth-functions such as the nondif ferentiable function of equation 4 . 18 , for example . He noted that the wild oscillation that destroyed differentiability also meant infinite arc length . He defined and investigated functions of "bounded variation" Definition 4.3.4 , then improved Dirichlet 's result in 1 8 8 1 . He showed that Fourier series of functions of bounded variation converge to at every point 4.6.6 .

+ (t+))

I

t.

�(I (t - )

( )

(

)

t ( l (t - ) + /(t+))

I t( )

Definition 4.3.4 BOUNDED VARIATION

Let h be a complex-valued function defined on the ( finite ) closed interval [a, b] and let P {t o , h , . . , tn}, t o = a < t l < . . . < tn b, be a partition of [a, b]. The VARIATIO N OF h WITH RESPECT TO P is n-l Vp (I) = L I h (t ; + d - h (t ; )I · =

(

)

=

.

;=0

h over [a, b] is V (h, a, b) V (h) = sup p Vp

The TOTAL VARIATIO N of

=

[a, b], BV [a, b]

as P ranges over all partitions of and we allow the possibility that V = 00. If < 00 , then we say that is of BOUNDED or FINITE VARIATION on The set of all real-valued functions of bounded variation on is easily verified to form a real vector space. 0

(h)

V (h) [a, b]. [a, b]

(

h

[a, b]

)

Clearly, any monotonic function on a closed interval is of bounded variation there-indeed, = Since the monotone func tion I = on (0, 1 is of unbounded variation , it is essential that the domain be a closed interval to conclude that a monotone function be of bounded variation.

(t) lit

]

V (I)

II (b) - I (a) l .

4. Fourier Series

165

What connection is there between continuity and bounded variation? The uniformly continuous function I (t) =

{ 0,

t cos h t E (0 , 1] , t = 0,

is not of bounded variation (Exercise 3) , so not even uniform continuity implies bounded variation. But suppose I is smoother. Suppose that I is continuous on and has a bounded derivative on the open interval then I must be of bounded variation (Exercise 6(a) ) . Conversely, how smooth must a function of bounded variation be? By 4.3.6 and the following theorem due to Lebesgue [Royden p. or Natanson pp. pretty smooth.

[a, b]

(a, b);

1968, 96,

2 1 1-212]:

1961,

4.3.5 MONOTONE ==> DIFFERENTIABLE A .E . A ny monotone function I on a closed interval is differentiable a. e., and the derivative I' is integra ble.

Not only are monotone functions of bounded variation, there is even the following result in the converse direction (the proof is outlined in Exercise

7):

1961, 218] [a, b] f V a,

4.3.6 T HE JORDAN DECOMPOSITION THEOREM [Natanson p. A ny function I 01 bounded variation on the closed interval can be writ ten as the difference of two increasing functions, namely (t) = (I, t ) (I, t) - I (t)) .

(V a,

Consequently, a function of bounded variation must be differentiable a.e. and have an integrable derivative. Thus, the nondifferentiable functions of equation are of unbounded variation. Since differentiable a.e. implies continuous a.e . , monotone functions are continuous a.e .-hence Riemann integrable , therefore also Lebesgue integrable . Some other facts about func tions of bounded variation are given in the Exercises.

(4.18)

Exercises 4 . 3

1.

THE VALUE I' (a + ) (a) (b)

2.

Suppose that I is differentiable in the open

(a, b), and that limt -+a+ f' (t) exists. Show that I (a + ) exists. f (a + u) - / (a + ) . f' (t) = 'Ulim f' ( a+ ) = t lim -+a+ -+O+ U

interval

ONE-SIDED DERIVATIVES Suppose that I is defined on the open interval is differentiable in and that limt -+ a + I' (t) exists. The right-handed derivative I: depends on the of equation existence of I and is generally different from

(a, b),

(a),

(a, b), (a)

(4.20) + f' (a ) .

166

4. Fourier Series

f' (a + ).

(a) I: (0)

(a) Show that if I: exists, then so does The converse is not true by the Remark after Definition 4.3.3, a case where exists, but does not. 2 (b) For I (t) = t sin ( l/t) for t E and = show that I i s differentiable everywhere (even at but that limt -+ o+ f' (t) does not exist . Therefore I is not piecewise smooth.

f' (0 + )

[- 71", 71"] {O} ,

1 (0) 0, 0)

{

3. UNIFORM CONTINUITY # BOUNDED VARIATION Show that

0, t = 0, I (t) = t cos � 0 < t _< 1, 2t '

is continuous-actually, uniformly continuous since but not of bounded variation on

[0, 1] .

4. PRODUCTS For I, 9 E BV [a, b], show that

[0, 1] is closed

Ig E BV [a, b] .

5. LIPSCHITZ CONDITION AT A POINT c A condition in between differ

entiability and continuity-i.e. , stronger than continuity but weaker than differentiability-is the Lipschitz condition . Let be a closed interval. The function � R is said to satisfy a LIPSCHITZ CONDITION OF ORDER 1 at E if there is some neighborhood r) n r > and some constant K such that - r,

[a, b] I : [a, b] c [a, b] c + [a, b] , 0, II (t) - I ( c) I � K It - cl for all t E (c - r, c + r ) n [a, b] ,

(c

or its equivalent formulation

I/ (c + u) - l (c) 1 � K l u i for l u i < r, and c + u E [a, b] with the obvious adjustments at the endpoints.

(a) If I is continuous on the closed interval and f' and I' exist at E show that I satisfies a Lipschitz con dition of order 1 at (b) Show that

(c- )

c (a, b), c.

differentiable at and

c

(c+ )

[a, b],

::}

Lipschitz condition at

Lipschitz condition at

c

::}

continuous at

c

c

and that neither of the implications is reversible .

6 . UNIFORM LIPSCHITZ CONDITION Let I be an interval. The function I I � R is said to satisfy a UNIFORM LIPSCHITZ CONDITIO N on I if there is some constant K such that :

x I (y) 1 � K I x - y l

II ( ) -

for all

x, y E I.

4 . Fourier Series

167

f

(a) UNIFORM LIPSCHITZ CONDITION =} B V Show that if satisfies a uniform Lipschitz condition, then is of bounded variation on

( b)

f

I.

B OUNDED DERIVATIVE =} LIPSCHITZ If f is continuous on and has a bounded derivative on the closed interval show that satisfies a uniform Lipschitz condition on In particular, the function ( t = t sin for t E 11" 11" and 1 0) = 0 of Exercise 2(b) i s o f bounded variation o n 0 c) PIECEWISE SMOOTH =} BV Show that any piecewise smooth function on the closed interval is of bounded variation.

I

(

[a, b]

(a, b), [a, b]. [- , ] - {O} [ , 1].

f ) 2 (1/t)

(

7. THE VARIATION FUN CTION

[a, b] Let [a, b] be a closed interval. For I, 9 E

BV [a, b] show that: ( a) The variation (of Definition 4.3.4) V (I) is 0 if and only if I is constant. ( b ) For a < e < b, V (I, a, b) = V (I, a, e) + V (I, e, b) . (c) For x, y E [a, b], let V (I, x, y) denote the variation of f on [x, y] with V (I, a, a) = O. Show that for any a ::; x ::; y ::; b, V (I, a, y) = V (I, a, x) + V (I, x, y). Conclude that V (I, a, x) is an increasing function of x. ( d ) Show that p (x) = V (I, a, x) - f (x) is increasing. Since I (x) = V (I, a, x) - (V (I, a, x) - f (x)), conclude that f can be written as the difference of two increasing functions and therefore that functions of bounded variation are differentiable a.e .

8 . BOUNDS O N FOURIER COEFFICIENTS BY TOTAL VARIATION A mea sure of the rapidity of convergence of a series is how quickly its terms go to Show that:

O.

( a)

The Fourier sine and cosine coefficients

LH-1I", 1I"] satisfy 1 (I, [ l en I ::; -V n 1l"

])

- 11", 11"

en of f E BV [ n

,

E

]

- 11" , 11"

C

N.

( b) These estimates cannot be improved. f

9 . ABSOLUTE CONTINUITY What functions may be recovered as the integral of their own derivatives (to within an arbi trary constant) ? They are precisely the a solutely continu ous func tions. A real-valued function f defined on the closed interval is ABSOLUTELY CONTINUOUS on if for every t > 0 there ex ists 8 > 0 such that for any n E N , for all E such that < < and < < 8, we

f: f'(t) dt

[a, b] a l b l ::; a 2 b2 ::; . . . ::; an bn ,

b

[a, b]

ai , bi [a, b] L:?= l (bi - a i )

168

4. Fou rier Series

2:7=1 I I (bi ) - I (a i ) 1

have < f. Since n could be 1 , absolutely contin uous functions must be uniformly continuous. We denote the set of absolutely continuous functions on by Prove the following:

[a, b] AC [a, b].

(a) PRODUCTS The product of absolutely continuous functions is absolutely continuous. (b) AC => U C Absolute continuity implies uniform continuity. (c) If satisfies a uniform Lipschitz condition, then is absolutely continuous. (d) If I has a bounded derivative on then is absolutely con tinuous on a , (e) AC => BV An absolutely continuous function is of bounded variation. (Note that this implies the existence of continuous functions that are not absolutely continuous such as x sin ( l /x) on By the remarks about functions of bounded variation after it follows that absolutely continuous functions have integrable derivatives: If is absolutely continuous on then is differentiable a.e. , and f' E The next result is in the converse direction . (f) INTEGRALS OF FUNCTIONS ARE ABSOLUTELY CONTINUOUS If then 9 (x) is absolutely continuous. (g) INTEGRATION BY PARTS [Stromberg 1 9 8 1 , p. By (e) and (f) of this exercise, absolutely continuous functions are dif ferentiable a.e . and have integrable derivatives. If and 9 are absolutely continuous on then

I

I

[a, b],

[ b].

(0, a).) 4 . 3.6

I

I

I E L'i [a, b],

Ll

I

[a, b],

LHa, b].

= f: I(t) dt

6.90, 323] I

[a, b], f: I (t) g ' (t) dt + f: f' ( t) 9 (t) dt = I ( b) 9 (b) - I ( a) 9 (a) .

Hints

(f (tn))

tn

f' (a + ) b.

a. a

1 . (a) . Show that is Cauchy whenever --+ Since exists, has a bounded derivative in r) for some < r < By Example 2.7. 1(d) it follows that is uniformly continuous in ( a , r) . Hence , by the discussion in Section 2.7, maps Cauchy sequences into Cauchy sequences. By the uniform continuity, if --+ and --+ limn then limn = (b) Define let --+ and apply the mean value theorem to on r] for some r >

I

I

(a,

I

I (tn) = I (Yn). I (a) I (a + ), tn a, 0. l(tn) [a, (a) . If I: (a + ) exists, show that I (a + ) I (a). Yn a,

2.

=

tn

a

4. Fourier Series

3. Consider the partitions [0 , 1] .

169

Pn = {O, 1/2n , 1 / (2n - 1) , . . . , 1/3, 1 / 2 , 1 } of

6 . (a) . Use the mean value theorem. (b) Use (a) o n each of the subin tervals. (c) Let I satisfy a uniform Lipschitz condition on b] . For . .< E [ , b] , [{

[a, Xn a ( I (X k + 1 ) - I (X k )) � (X k + 1 - X k ) . Xl < X2 < 7. (d) . For X < y , show that p ( y) - p ( x ) = V (I, X, y) - (f ( y) - I (x)) . Note that I ( y) - / (x) � V (j, a, x). 8 . (a) . an = t D" I(t) cos nt dt = n1" f:'" I (t) d (sin nt) . Integrating by parts , we get an = �; f:'" sin nt dl (t) . (b) Consider Example 4 . 1 .3. .

9 . (f) . Make use of the following property of the integral [Nat anson 196 1 , p. 148] : If I E b] then, for any > 0, there exists 6 > ° such that for any measurable set E C b] of measure less than 6, I (t) dt E l I

L'1 [a,

f

4.4

[a,

< f.

f

The Riemann-Leb esgue Lemma

We enlarge the class of functions for which we compute Fourier series from to in this section. For I E the products (t) sin nt l and If (t) cos nt l are each less than or equal to l l (t) l , so the integrals that define the Fourier coefficients still converge; therefore it makes sense to consider the Fourier series of such functions. One reason to consider the theory for functions instead of functions is that there are more functions: As noted in Exercise 1 , the space is a proper subset of b] for all > 1 ; indeed, generally, as p increases , the spaces decrease in size. We cannot really go beyond for if rf: 11"] , what sense could we make of the Fourier coefficients

L2 L1

L1

LH-1I", 1I"],

L2

p

II

L1 Lp [a, b] Lda, Lp [a, b] L 1 , I Ld-1I",

an = -11"1 1" f (t) cos nt dt? Another reason to enlarge the theory to L1 functions is that it is fairly easy to do. As noted in the Remark before Example 4.1 .4, the Fourier coefficients of a linear combination of L; functions are the corresponding _"

linear combination of the Fourier coefficients of the original functions. Since this result depends only on the linearity of the integral, it remains true for functions. Also, for functions the product Ig is in 71"] , since

L1

L2

2 41g = (f + g)

L1 [-7I",

-

(j - g) 2 E L1[-1I", 11"] ,

so we can now speak of the Fourier series of the product of two another convenience.

L2 functions,

170

4. Fourier Series

L1

Although we consider functions from now on , we do not consider con vergence in the norm. Rather, we consider various kinds of convergence: pointwise , uniform, (e, 1), and Abel convergence. Lagrange , who was so troubled by the erratic behavior of Fourier co efficients , would perhaps have been mollified by the Riemann-Leb esgue lemma 3 .3.2; it demonstrates at least that the Fourier coefficients of any go to 0, i.e . , E

L1

f L;[-7I", 71"] 11" an = 71"1 1 f ( t ) cos n t dt -

- 11"

11"

---+

O, and bn = -71"1 1-11" f (t) sin nt dt

---+

O. (4.2 1 )

Intuitively, as the oscillation of the sinusoids increases , it puts as much of the function's area above the axis as below ; we mention two more instances where the increasing oscillations of a kernel function drive an integral to 0 in 4.5.4 and Exercise 2( a) . For our development of the theory of Fourier series for functions, we generalize the Riemann-Lebesgue lemma to functions fE b] , -00 � � b � 00 , and to more general kernel functions than sinusoids; the key to 4.4 . 1 is the density of the step functions in mentioned in 2.4.5( d).

L'i LHa,

a

L'i [a, b] ,

4.4.1 THE GENERALIZED RIEMANN-LEBESGUE LEMMA For any f E -00 � ::; 00 , and any bounded measurable fun ction defined on R it follows that

Li:[a, b],

then

h

�a b

c

l c h (t ) dt

lim ! .... ± oo c 0

limoo

w-+

-+

0

( the averaging condition) ,

l b f (t) h (wt) dt = O . a

Remarks Sinusoids are bounded, measurable (indeed, continuous) func tions, and, using sin t (t) for example,

=h �I l c sin t dt l = I � (1 - cos C)I ::;

I� I

-+

It therefore follows from 4.4. 1 that for any f E lim

W -+ OO

0 as c -+ ±oo.

Li:[a, b],

l b f (t) coswt dt = lim lb f (t) sinwt dt = O. W -+ OO

a

a

L1

(4.22)

In particular, it follows that Fourier coefficients of functions ap proach Note , too, that oscillation is not the driving force for the conclusion of 4.4. 1 ; rather the averaging condition is. Clearly,

O.

h (t) = {

�:t , IItt II

> ::;

1, 1,

4. Fourier Series

171

h( )

satisfies the conditions of 4.4. 1 . As w - 00 , the graph of y = wt becomes a horizontal line the t-axis with a peak of 1 at t = 0 , it does not oscillate at all.

(

Proof.

b

)

I(

< a,

If a or is finite, define t ) to be 0 for t > b or t respectively, so that we can restrict attention to E L� (R) . Suppose that l [ c,d] is the characteristic function of C [0 , (0 ) . Then, letting x = wt ,

[e, d]

l1 ( )

100 l[c,d] (t ) h (wt ) dt

l

I

d

dw

-wI

o

By hypothesis ,

1

dw

() Thus, 1000 l[c, d] ( t ) h ( wt ) dt any step function g , J uW

0

1

h x dx - O and �

-

h wt dt

0 as w

l0

-

h ( x ) dx - -wl

cW

1

c

0

W

()

h x dx .

()

h x dx - O as w - oo.

00 . It follows by linearity that for

r oo 9 (t ) h (wt ) dt = O.

W -+ OO 10 lim

As noted in 2.4.5, the step functions are dense in L'i (R) . Hence, if C is a bound for h , then for any E L'i (R) and { > 0 , there exists a step function {/2C. For sufficiently large t - 9 t dt 9 such that {/2, so w , 1J000 9 t ) h wt dt

I l

(

10(00 I I) ( ) < l

I

( ) 1 � II I gil l <

1 1 00 I (t ) h (wt ) dt l

<

100 II ( )

l

( ) l l h (wt ) 1 dt 1 1 00 9 (t ) h (wt ) dt � 2C C + 2 = c. A similar argument applies to I� oo I ( t ) h ( wt ) dt . Consider a more general question . If a trigonometric series a 0 + L an cos nt + L bn sin nt 2 n eN n eN converges, must an , bn O? As this need not be a Fourier series ( see o +

€

t -9 t

€

0

-

Section 4.1 1 ) , you do not know how the coefficients were obtained. Cantor showed that if the series above converges for all t in a closed interval, then - O. Lebesgue then generalit;ed Cantor's result to sets of positive measure . Since we refer to it in the proof of 4.4.3, we state Lebesgue's dominated convergence theorem here for reference.

an, bn

172

4. Fourier Series

1961,

161;

p. 4.4.2 DOMINATED CONVERGENCE THEOREM (Natanson is a sequence Rudin p . 305) If of measurable functions defined on the m easurable set E that is domin at ed on E by the integra ble function ---+ a. e. for every and 9 in the sense that a. e. on E, then is integrable, a n d

1976,

I

(In) l in (t) 1 � g (t)

n E N,

In (t) I (t)

n JfE In (t) dt JfE I (t) dt. 4.4.3 THE CANTOR-LEBESGUE THEOREM If a trigonometric series a o /2+ Ln EN an cos nt+ L n EN bn sin nt converges on a set E of positive Lebesgu e measure, then a n , bn ---+ O . lim

=

Jl.

Jl. [a, b]: Jl.

Proof. Let denote Lebesgue measure . We can assume that E is of finite measure , and even that E is a subset of a closed interval If (E) = 00, w e can choose the first positive integer m such that (E n [- m , mn > 0, and use E n m instead of E i n the argument . A s i n Exercise we can rewrite the series in amplitude-phase form as

[-m, ]

4.1-2,

�o + L: dn cos (nt - !f'n ) , where dn = Ja� + b� and 'Pn cos - 1 (an/dn) . nEN For t E E , dn cos (nt - !f'n) ---+ O. I f dn -f+ 0 then there exists f. > 0 such that dnk � f. > 0 for some subsequence of (dn). Hence cos (n k t - !f'nk) goes to 0, and therefore so does cos 2 (n k t - 'Pnk) ' Since I cos 2 (n k t - !f'nk) I � 1 on E, by the Lebesgue dominated convergence theorem 4.4.2, it follows that =

we can pass to the limit under the integral:

2 2 k JfE cos (n k t - 'Pnk) dt = JfE limk cos (n k t - 'Pnk ) dt

lim But

L cos 2 (n k t - !f'nk) dt

Now consider the integral

=

-

=

O.

L [1 + cos 2 (n k t - 'Pnk)] dt t [Jl.(E) + L 2 (n k t - 'Pnk ) dt] . t

L cos 2 (nkt - 'Pnk ) dt. Let I = 1 on E , and

f = 0 on [a, b] n CE; clearly, I E L'i [a, b]. Now

4. Fourier Series

173

0

As the latter two integrals go to by the Riemann-Leb esgue lemma 4.4. 1 , i t follows that the limit is p. (E) 1 2 , which contradicts the previous equality. o

Exercises 4 . 4

1. Lp GETS SMALLER AS P GETS BIGGER Show that (a) L q [a, b] is a proper subset of L p [a, b] for all 00 � > p � 1 , the exact opposite of what happens for the lp spaces. Does this remain true for infinite intervals? (b) There are functions I, g E L2[-7I", lr], such that Ig fi. L2[-7I", 71"]. 2. RIEMANN-LEBESGUE LEMMA VARIATIONS Let [a, b] be a closed in terval, and let I E Ll [a, b]. (a) Divide [a, b] into n subintervals of length (b - a) In, and define gn = ±1 on alternate subintervals . Show that for continuous I, limn co f: I (t) gn(t) dt = O. (You can use 4.4. 1 for this, but it is easy to give a more revealing elementary argument .) (b) OTHER ORTHONORMAL BASES Show that if ( gn) is a uniformly bounded (i .e . , for some M, I gn (t)1 � M for all t, and every ) countable orthonormal basis for L2[a, b], a, b finite, then nlim � oo ta I (t) gn(t) dt = O. 3 . ALTERNATIVE PRO OF OF RIEMANN-LEBESGUE Let I E Ll ( R) . (a) Show that lim6 o f�co I I (t + 6) - I (t)1 dt = O. (b) With g (w) = f�oo e iwt l (t) dt , without using 4.4. 1 , show that limw 9 (w ) = O. q

.....

n

.....

.....

o

Hints

L [a, b], split [a, b] into {t E [a, b] : I / (t)1 < I } U {t E : I / (t)1 � qPI}, and use the fact that I / (tW I / (tW on the latter . Since I / I � max { I , lf n , it follows that I /P I is integrable . Since q p � 1 , q p + for some O. Let 9 (t) = II (p + ) on

1 . (a) . For I E

[a, b]

>

>

c

c

>

�

c

(0, 1], 9 (0) = O. Then 9 E Lp [0, 1] but 9 fi. L q [0, 1]. The inclusion L q Lp for q � p does not hold on infinite intervals: Consider I (t) C

lit on [1 , 00) .

=

174

4. Fourier Series

2. (a) . Use the uniform continuity of f, and consider Riemann sums.

-g

3 . (b) . Since (w) = it follows that

f�oo eiw (t +1r /w ) I (t) dt = f�oo e iwt I (t - 7r/w) dt,

2g (w) = Therefore,

4.5

f�oo eiwt [I (t) I (t -

-

7r/w)]

dt.

Ig (w) 1 � � f�oo I I (t) - f (t - 7r/w ) 1 dt.

The Dirichlet and Fourier Kernels

This section paves the way for the pointwise convergence theorem 4.6.2 by providing certain ways to express the nth partial sum of a Fourier series (4.5. 1 ) . The device by which we accomplish this is the simpler way provided by equation (4.24) to express a sum of cosines as a ratio of two sines . In practice, Fourier series are always truncated to approximate a function as a finite sum of harmonics (t) = sin kt . cos k t Through the use of the nth DIRICHLET KERNEL

Bn

+ L � = l bk

ao/2+ L� = l a k

(4.23)

Bn

(see also equation (4.25) ) , we can theoretically calculate (t) in just one integration (4.5. 1 ) . Clearly, for n 1= 0, is an even 27r-periodic function . As illustrated in the figure below , (t) becomes more and more highly concentrated at t = 0 as n increases, becoming like a pulse 2n 1 units high on a shrinking pedestal of width 47r / (2n 1) . The shrinking base and increasing height are such that the area under the curve remains constant (Example 4.5.3) . (To sketch it, use the representation of equation (4.24) , and treat 1/ [sin (t/2)] as the envelope.)

Dn

Dn

+

+

The Dirichlet kernel for

n

=

3 and

n

=

7

4. Fourier Series

175

, Dn =

For t = 0 , ±21r, ±47r, . . . (t) 2n + 1 by substitution. To calculate # 0, we can avoid the summation by showing that for any n E N,

Dn (t) for t

sin ( n + t ) t , t # 0, ±21r, ±47r , . . . . sin (t/2) (4.24) To see this, add the equalities below for k = 1 , 2, . . . , n : _

1 + 2 cos t + 2 cos 2t + · · · + 2 cos nt -

sin

(k �) +

( �)

t - sin k -

t = 2 cos k t sin

�.

The sum on the left telescopes t o yield just the end terms:

which implies that

For t # 0, ±21r, ±47r, . . . , we can divide both sides by sin (t/2) to get equation (4 .24) . (An alternate derivation is suggested in Exercise 1 .) By I 'Hopital 's rule, for any 0, ±21r, ±47r, . . . ,

p=

. sin (n + l/2) t hm . sm ( t /2)

t -+ p

{

= 2n + 1 = Dn (p) .

Hence , the function defined as sin (n + l/2) t , t # 0, ±27r, ±47r, . . . , sin (t/2) otherwise, 2n + 1 ,

n E NU

{O} ,

(4.25)

Dn

is continuous and agrees with (t) as defined in equation (4.23) for all t. When we write in the sequel, we do so with the understanding that it has the value 2n + 1 when t = 0, ±21r, ±47r, . . .. Various expressions for the nth partial sum of a Fourier series are given in 4.5 . 1. As we shall see, as n -- 00, (t) behaves like Dirac 's 6 function in the integrals below , sieving out the value of the function f (t) when the argument of (t - ) is 0, i.e . , when = t; in other words, (t) -- f

Si:�:t;��)t

Dn

x

Dn x

Dn

Sn

(t) .

4.5 . 1 THE nTH PARTIAL SUM USIN G For any n E N U { O } , the n th partial sum of the Fourier series for f E L1 [-1r, 7r] is given by any of the convolution integrals below:

Sn

176

4. Fourier Series

-211'1 1_�� f(x) Dn (t - x) dx = -211'1 1_� f(x) Dn (x - t) dx, Sn (t) = (b) -211'1 1_�� f (t - x) Dn (x) dx = -211'1 1_� f {t + x) Dn (x) dx, � (c) 1 ior [f (t + x) + f (t - x)]Dn (x) dx. 211' Proof. The statements are all trivial for = 0, so we assume i- 0 in the arguments below . (a) The nth partial sum of the Fourier series of f is � Sn (t) = 211'1 1_ f (x) dx + � L:� = l cos kt I: f (x) coskx dx "" nk = l sin kt 1 � f (x) sin kx dx + 2:.11' L...-; [cos kt cos kX + sin kt sin kx] dx 1;: 1 � f (x) ( 21 + {; _� ) ) n ( 1 ;:1 1_� f (x) 2 + {; cos k (t - x) dx -211'1 1_�� f (x) Dn (t - x) dx -211'1 1_�� f (x) Dn (x - t) dx because Dn i s even. (b) With w = t - x, the penultimate equation becomes Sn (t) = -2111' I-t + � f (t - w) Dn (w) dw. t� Since f and Dn are each of period 211' , by 4.2.2 , the value of the integral remains the same as long as the range of integration is 211'; hence Sn (t) = 211'1 1_� f (t - w)Dn (w) dw. � With w = x - t, the equation Sn (t) = � 1 � f (x) Dn (t - x) dx becomes 211' _ � -t � Sn (t) = 211'1 1-� -t f (w + t) Dn (w) dw = 211'1 1_ � f (w + t) Dn (w) dw. � (c) We split the first integral 21 J::� f (t - x ) Dn (x) dx in (b) to get � 1 10 Sn (t) 211' _ f (t - w) Dn (w) dw + 211'1 ior f (t - w) Dn (w) dw. � (a)

n

=

n

4. Fourier Series

Now replace w by to get

177

-u in the first integral , and use the fact that Dn is even

sn (x) = 211"1 Jr [f (x + u) + f (x - u)] Dn (u) duo o We need the estimates of 4.5.2 to prove Dirichlet ' s theorem 4.6.2 about 0

the pointwise convergence of Fourier series.

4.5.2 Two UNIFO RM BOUNDS ON INTEGRALS (a) THE DIRICHLET KERNEL For any 0 :S :S b :S {O} , b sin n t t < 411". . a sm

NU

( + l /2) d ( t/2)

l

a

11" ,

a n d any n E

-

(b) THE CONTINU OUS FO URIER KERNEL For all real w and t , the qu a n tity cp (t , w ) = sin wt t for t 0, cp (O, w ) w , is called the CONTIN U O U S FOURIER KERNEL. We write sin wt t sometimes, but with t h e understan d ing that sin wt t is defined to be w when t O. For any 0 :S <

/

=P

/

= =

/

lb -sin wt dt a

t

<

a b,

11" .

Proof. (a) If > 0 and AI , . . . . . are numbers that alternate in sign and decrease in absolute value, then dominates the partial sums: For any k = 1 , 2 , . . . , n ,

Al

. , An ,

Al

= 1 , . . . , n , the areas l k1r /(n + l / 2) ( n + 1/2) t . { positive, A k = k l n l sinsm . (t/2) dt negative, ( - ) 1r/( + / 2) sin ( n + 1/2) t under the various lobes of Dn (t ) 0 sin (t/2) For k

IS

7

3

if k is odd, if k is even, < t <

11",

consti-

178

4. Fourier Series

tute a set of numbers of alternating sign and decreasing absolute value , since sin is increasing. Since is contained in the rectangle of base [0 , (n and height 2n it follows that � 1r . For any [0 , 1r , either there exists some � n such that

(t/2) 1r / + 1/2)] aE ]

Al + 1, Al 2 k a E [(k - 1) 1r/ (n + 1/2) , h/ (n + 1/2)] , or a E [n1r / (n + 1/2) , 1r] ( take the interval on the right if a belongs to two of them) . As the latter argument is quite similar, we argue only the former case. If k is odd, A k > 0, and A k < 0, so for some E [0, 1] , 10a Dn(t) dt = Al +A2 + · · +Ak - l + cA k � Al +A2 + · ·+Ak - l � Al � 2 11". Consequently, for any ° � a � b � 1r, and any n E N u {O}, c

If k is even,

br

Ak

sin u duo A k = jk I 'II" -(-) u

The alternate in sign and decrease in absolute value-indeed, by a geometric argument as in a , It follows that � k , for each = ( 'II" Sl�. t for any n , 11", and therefore that

( ) I Ak I 1/ k E N. n dt < J E k = l Ak < Al o 1 '11" sin wt sin t 1 6 -< t d t - 0 t dt < 1r .

a

0

Example 4.5.3 CONSTANT AREA UNDER DIRICHLET KERN EL O N [0 , 11"] Sh ow that

1 '11" sin (.n + 1/2) t dt = 1 '11" Dn (t) dt = 11", for all n E NU {O} .

( t-/2) 0 Solution. Since fo'll" cos n t dt = ° for every n , o

sm

1o '11" Dn (t) dt = 1'11" (1 + 2 t cos kt ) dt = 1r. 0

k= l

0

4. Fourier Series

179

Dn

(t) = S i :�:t;Wt jumps up to 2n + 1 at t = 0, The Dirichlet kernel then stays at about constant amplitude between t = 7r / (n + 1 / 2 ) and t = 7r while becoming increasingly more oscillatory. We might expect that the oscillations of are enough to make

Dn

1,..,./.(n + l/ 2)

f (t)

Dn (t) dt

--->

0 as n ---> 00

f E Ll [- 7r, 7r] , and indeed this is what happens. 4.5.4 T HE RIEMANN -LEBESGUE PROPERTY O F Dn L e t Dn ( t ) th e Dirichlet kernel. For any f E Ll [ - 7r, 7r) , and r E (0, 7rJ , '" lim ! n f (t) Dn (t) dt = O.

for any

den ote

Proof. Since r > 0, then sin (t/2) � sin r/2 > 0 for all r S; t S; 7r. It follows r

f?j

E Ll [r, 7r) . Hence, by the Riemann-Lebesgue lemma 4.4. 1 , as that . sm t 2 ) n � oo ,

Definition 4.5.5 DISCRETE FO URIER KERNEL

Suppose we replace the sin (t/2) in the denominator of the Dirichlet kernel (t) = [sin (n + 1/2) t) / sin (t /2) by t /2, i.e . , consider

Dn

sin (n + l/2) t sin (n + l/2) t instead of t/2 sin (t/2) . . N U {O} , we call the contmuous function � n (t) =

sin (n + 1/2) t t/2 for t '# 0, �n (0) = 2n + 1, the DIS CRETE FOURIER KERNEL of order n . sin (n + 1/2) t . sin (n + 1/2) t . IS we mean that If we WrIte mstead of t/2 /2 defined to be 2n + 1 at = O. 0 For n E

t

t

�n ,

�n 2; �7, �7

The discrete Fourier kernel provides a very good approximation to the Dirichlet kernel for It I < and are sketched below. Eventually, (t) = sin the discrete Fourier kernel / (t/2) decreases more rapidly than but and are essentially indistinguishable until about = 2. (To sketch treat 1/ (t/2) as the envelope.)

Dn

D7,

�7 D7 �n (t),

D7 (7.5t)

t

4 . Fourier Series

The Dirichlet kernel

The Fourier kernel

Dr (t)

and

D7 (t)

7 (t)

r ( t)

=

=

�in 7 . 5 t sm(t/2}

sin 7 . 5t t/2

4. Fourier Series

181

AnotherFourier observation concerning the behavior similarityasbetween thein integral Dirichlets . The and discrete kernels concerns their kernels essentialby content of 4.Fourier 5 . 6 and kernel 4. 5 . 9 isas that you can replace the Dirichlet kernel the discrete n Let 1 E L1: [0, 7r] 4.5.6 sin (t/2) CAN BE REPLACED BY t/2 as n and r E (0, 71"]. Then, assuming that the limits exist, limn for I (t) sins�: �:;)2)t dt limn for I (t) sin (nt;21 /2) t dt. (4. 2 6) �i; -+ 00 .

=

sin(

In addition, the discrete Fourier kernel

-+ 00

/ 2)t

LEBESGUE PROPERTY : 11

has the ''RIEMA NN

o.

l� 1" 1 (t) sin ( nt721 /2) t dt Proof. By two applications of l' H 6pital's rule, lim..... o (_ sin1 t - �)t = Therefore, 9 (t) � / is continuous everywhere if we define it to beis the0 atproduct t = O. Itoffollows that 9 isfunction integrableandanda bounded boundedintegrable on [0, 71"] . function, Since Ig an integrable Igwithis integrable on [0, 71"] . Thus, by the Riemann-Lebesgue lemma 4.4 .1 h (t) sin ( n + 1 /2) t , l� 1 " [I (t) Cin (lt/2) - t;2 ) ] sin (n + 1 /2)t dt 0 , or, assuming "that the limits exist, + 1 /2) dt = limn 1" 1 (t) sin (n + 1/2) t dt. limn 1 1 (t) sinsm�n (t/2) t/2 By an identical" argument + l/2) dt = limn " l (t) sin(n + 1 /2)t dt. limn 1r l (t) sinsm�n (t/2) t/2 1r By 4. 5 . 4 , however, limn Ir" 1 (t) :(;(i/N dt 0, and the desired result follows. For what 1 do the limits of equation (4. 2 6) of 4. 5 . 6 exis t? What is the limit? For certain 1 (piecewise smooth, for one type) the limi t is 7r 1 (0 + ) . This fact is the essence ofthese the pointwise convergence theorems 4. 6 . 2 andon 4.integrals. 6 . 6 . BeforeIn dealing we consider questions, we consider some variants with infinite series, wesomething allow absolutely convergent and nonabsolutely convergent ones. We do similar for integrals for more flexibility. =

o.

t

=

s i n( /2) - t 2

=

=

0

0

0

si

i

)

=

182

4. Fourier Series

-+ , pv , a < b 1a -6 1-+6a 1-+ a 6 1a b we say thatconvergence. f is integrable on [a, b], we mean that f: I f (t)1 dt < i. e-If. , absolute If f is integrable, f+ (t) = max:(f (t) , 0), and f (t) = min (f (t) , 0), then f = f+ -f - , and f: f(t) dt f: f+(t) dt f: F(t) dt. If we abandon absolute convergence and demand only that f be integrable on [a, c] for all c < b and that li c-+ f: exist, then we represent the limit as fa-+ b • Similarly, b and -+ = lim d = lim 1 1-+ba c-+ a c 1-+ a 6 c-+ a , d--+ 1c b kernel sin tit is not The distinction is important. For example, the Fourier in L'i (R+) since (4.48) ) r oo I sint t l dt ( eqUation of Section 4.9 ' Jo but the DIRICHLET INTEGRAL - 00 sin t 1o t dt 11"/2 (4. 5 . 8 (b )) . If f is integrable on [a, b], then 1a b f(t) dt = 1a -+ b f(t) dt = 1-+ba f(t) dt. For c E ( a, b) , the CAU CHY PRINCIPAL VALUE of f: is defined to be ( -€ PV { b lim r + 1 ) . Ja €-+ o Ja c+6 € By this convention, even though neither f'::' r 3 dt nor r:o t-3 dt exists, °O -€ PV 1 t-3 dt = lim ( 1 r 3 dt + J oo r3 dt ) = lim ( -� + -4) = 0, €-+o 2f 2f €_ o - 00 - 00 € whereas if we take 2f in the second integral, we get -1 --1 ) ' hm. ( e-O 2f 2 + 2 (2f) 2 sob, then the symmetric approach is crucial. Likewise, if a Cl < C2 . . . < Cn ::; Definition 4.5.7 IMPRO PER INTEGRALS

,

,

- 00

::;

::;

00 .

00 ,

=

-

m

6

= 00

=

=

= - 00

::;

<

4. Fourier Series

183

Last, if lima_ oo I� exists, we designate it to be the CAU CHY PRINCIPAL of I�oo ' and write °O l PV 1 = lim - 00 If I� oo and 1�000 exist,� i.e ., if f is absolutely integrable on (-00, 0] and then I oo = I oo ' and 1000 = 10..... 00 If the integrals I� oo' and 10- 00 exi[0, s00),t, then 00 00 PV 1 = ..... - 00 1-+- 00 A consequence of taking Cauchy principal values is that if function-f(t) t or f (t) = sint, say-then for all 0, If�(t)f is(t)anydt =odd0, t dt nor I:�oo sin t dt exists. If f is so PVI�oo f (t) dt = neither I�C: integrable, f E L'i ( b) , then I ..... b f, t+ f, I:: f, PVI: f exist and are equal to I: f. nelWenext.consider integrals of the Dirichlet kernel and continuous Fourier ker 4.5.8 (a) For any r E (0, 11"] , limn iro sinsm�n +(t/2)1/2) t dt = limn 10 sinsm�n +(t/2)1/2) t dt = 11". (b) THE DIRICHLET INTEGRAL For any r 0, l r sinwt dt -- 1 ..... 00 sint dt - 1I"/2 . 1m _ w oo 0 t o t Proof. (a) By Example 4.5.3, dt 11" for every n. Hence, for any r E (0, 11"] and any n , 10'" Si:�:�;Wt 1/2)t dt 11" = ior sin(sinn +(t/2) sin(n + 1 /2) t 1'" sin (n + 1/2) t ior sin (t/2) dt + r sin (t / 2 ) dt. with f (t)the=continuous By(b)4.5.4Consider 1, I: � n 00. Fourier kernel (t, w) defined to be w at t = 0, and

VALUE

a

a -+ OO

_

a

.

•

.

=

a,

0

a >

OJ

a

a

'"

I

>

·

-

=

°

-

as

�

a

a

a

4. Fourier Series

1 84

Since l it is integrable onJ. [a,sinwt r] , it follows from the Riemann-Lebesgue . r lemma 4.4.1 that hII1w .... oo -t- dt = 0. We may therefore assume that r E ( 0 , 11"]. For b, w 0, w b sm u u = .... -b sm t · 1 sm w t dt = 11m · 1 -I 1m w_oo 0 U 10 t dt . w - oo t Sincesin C)t (t, l) = sin t lt is continuous, I01 cp (t, 1) dt < Now integrate I1b -t- dt by parts: cos t dt. sin t dt = --cos b + cos 1 - 1 b -16 t b t2 1 Observe that limr_oo ( cos b) /b = 0. Since I(cost) /t 2 1 1/t 2 , it follows that absolutely, and thereforesithat I1.... � dt converges; It �so dtdoesconverges oo si hence Io � dt. What does Io.... oo � dt converge to? Since r E (0, 11"] , it foll ows from ( a) and 4.5.6 that lr sin ( + 1/2) t dt = 1m lT sin ( + 1/2) t dt = 11". n t/2 o sin (t/2) Hence, substituting = (n + 1/2) t, we obtain sin u duo sin u du = 1 .... -lr sin ( + 1/2) t dt = hm.n 1(n+ 1/ 2 )r -11" = hm n t U 2 0 U o o It is easy to deduce a criterion for pointwise convergence of a Fourier senes. >

a

•

•

--

d

00

0

00 .

1

00 c

t

•

n

t

t

1·

n

-

�

t

c

•

0

u

n

00

0

4.5.9 CRITERION FOR POINTWISE CON VERGENCE The Fourier series of the 211"-periodic function f E LH-1I", 11"] converges at a point t if and only if for some r E ( 0 , 11"] , the limit

limn !11" Jor [/ (t +

u

) + / (t _ u)]

sin ( n + 1/2) u du u

(4.27)

exists; if it does exist, the limit is the sum of the Fourier series at t.

Proof. By 4 . 5 .1 ( c ) , the nth partial sum of the Fourier series for f may be written in terms of the Dirichlet kernel Dn (u) = si n!�n+"'%2]u as

12 11" Jto ' [f (t + u) + f (t - u)] Dn (u) du. Sn (t) Since / (t + u) + f (t ) is integrable, the result follows from 4.5.6. -

u

0

4. Fourier Series

185

Knowing that the Fourier series for 1 converges at t does not say that the limit is 1 (t), which is really what interests us. We get a little closer to what the limit can be in 4.5 . 10. Let D..I (t, u) = 1 (t + u) + 1 (t - u) - 2c. Note thattheif c1ofis4.5.continuous, and l u i is small, then D..I (t, u) 21 (t) - 2c. Though 10 is arbitrary, the most important applications are for c = [ - 1 (t + )] /2, which is f (t) at points t of continuity. �

/ (t ) +

4.5.10 CON VERGENCE TO A PARTICULAR VALUE the 21r-periodic function E - 1r , 1r converges to only if there exists r E ( 0 , 1r such that

1]

Li[

]

The Fourier series of c at a point t if and

hm. -1rl 1 r [/ (t + u) + / (t - u) - 2c] sin ( +U l /2) u du = O. (4.28) Proof. Apply 4.5.9 to 9 (t) = 1 (t) - c . Conclude that the Fourier series for 9 (t) = 1 (t) - converges to 0 at t. By linearity, the Fourier series for I (t) converges to c. on thechange values atof 1a through outThetheFourier intervalcoefficients on whichofitaisfunction defined.1 Ifdepend the values point or aoffinite number of points or on a set of measure 0, the Fourier coefficients I do notmeasure change.forSome valuescoefficients of I must tochange on a setHowever, of positive Lebesgue its Fourier be altered. Rie mann' s LO CALIZATION PRIN CIPLE 4.5 . 1 1 asserts that the behavior of the Fourier series of I at a point t depends only on the values of I in an arbi trarily small open interval about t: If we alter I outside an open interval containing t to get g , say (subject to the constraint that 9 still belong to Li [- 1r , 1r] ) , then the Fourier series for 9 the Fourier for 1 converges to I (t) at t, ifconverges to 1series (t) at t, and ifdiverges the Fourier diverges at t , at t series for I Note how different thisneighborhood is from the ofbehavior oft, power series:musttwobe power series coincide in a a point then they equal everywhere in their common region of convergence. n

n

0

c

0

{

If

4.5 . 1 1 RIEMANN 'S LO CALIZATIO N PRIN CIPLE Extend I , 9 E Li[ - 1r, 1r] to be 21r-periodic. If = 9 a. e. on an open interval (t - r, t + r), r > 0 , about t, then their Fourier series both converge or both diverge at t . If both Fourier series converge at t, then their sums coincide.

1

P roof. For h = I - g , we have h (t u) = 0 a. e . for u E ( 0 , r), and the result follows from 4.5 . 10 with integrand h (t + u) + h (t - u) - 2 . O . ±

0

18 6

4. Fourier Series

The integrability condition of Dini's test 4.5 . 12 provides a sufficient con dition at t. for the Fourier series of a function I to converge pointwise to I (t) 4.5.12 DINI 'S TEST II for the 21r-periodic function I E is some r E (0, 1r] and some fixed t such that

Ll [ - 1r, 1r] there

f ( t + u) + / (t - u) - 2f (t) E Lr [o , r] , u then the Fourier series for I converges to I (t) at t. :......:. --!.-�----'-....� ...:.

1

By hypothesis and the Riemann-Lebesgue lemma 4.4. 1 , limn .!.1r iro [J (t + u) + f (t - u) - 21 (t )] sin ( n +u 1/2) u du = 0 , and the result follows from 4.5 . 10 . Lipschitz' s test 4.5f. 13toprovides another sufficient condition for the Fourier series of a function converge pointwise to I (t) at t. We say that f satisfies LIPSCHITZ CONDITION of order a 0 at the point t if there exist positive aconstants r and b such that (4.29) I I (t + u) - I (t) 1 � b l u l a for l u i < r. Ifvalue I has a bounded derivative f' on [- 1r , 1r] , 1 f' 1 b, say, then by the mean theorem, for all t E [0 , 21r] and sufficiently small l u i , there exists d E [t, t + u] such that I I (t + u) - I (t) 1 = I f' (d) l l u l � b l u i . As mentioned in Exercise 4.3-5, the Lipschitz condition of order 1 at a point t is stronger than continuity but weaker than differentiability at t. Proof.

0

>

�

Corollary 4 . 5 . 1 3 LIPS CHITZ 'S TEST

If the 21r-periodic function I E

Ll[-1r, 1r] satisfies a Lipschitz condition at t then the Fourier series for f converges to I (t) at t. Proof. Let a, b, and r be positive numbers such that I f (t + u) - I (t) I < b l u l a for l u i < r. Then adding and subtracting I (t), I I (t + u) + I (t - u) - 2 1 (t) 1 ::; 2b l u l a for l u i < r.

Since

r I I (t + u) + I (t - u) - 2 1 (t) 1 du ::; 2 r

bu a du 2bra , a io io u it follows that [/ (t + u) + / (t - u) - 2/ (t)] /u E Ll [O, rJ, and the result u

follows from 4.5 . 12 .

0

=

4 . Fourier Series

187

Exercises 4 . 5 1. ALTERN ATIVE COMPUTATION O F note that

Dn (t)

1 + 2 cos t + 2 cos 2t + . . . + 2 cos nt

To verify equation (4.24) ,

n k= -n

L eik t,

=

n E NU {O} .

Now sum the geometric series on the right. 2. SUMMIN G "EVEN" COSINES Prove that for t =p ±k1r, and any n E N,

t cos (2kt) k=l

=

k = 0 , 1 , 2, . . . ,

sin [ (2� + 1) t] ! . _ 2 sm t 2

3 . SUMMIN G SHIFTED COSINES Prove that for w =p ±2k7r, and any n E N ,

k

=

0, 1 , 2 , . . .,

� _ sin [(2 n + l) (w/2) + a] - sin (w/2 + a) . L..J cos w + a ) 2 sin ( /2)

k=l

(k

w

Note that taking a 0 almost yields equation (4.24) , i.e . , that this is almost a translated version (by a ) of equation (4.24) . =

4. SUMMING "ODD" COSINES Prove that for t =p ±k1r, k = 0 , 1 , 2, . . . , and any n E N, � sin 2nt . · L..J cos [2k - 1] t -2-sm t k= l =

5 . SUMMIN G "ODD" SINES Prove that , for t =P ±k-n", . and any n E N, sm . 2k - 1] t _ 2 nt " sm - -.- . L..J [

n k=l

=

sm t

6 . SU MMING SINES Show that for t =P ±2k1r,

� .

k

k

=

0, 1 , 2, . . . ,

cos t /2 - cos ( n + !) t _ sm t . k 2 sin (t/2) �

0, 1 , 2, . . . ,

188

4. Fourier Series

Hints 5.

E�= l sin ([2 k - 1] t) by sin t, and use the fact that 2 sin x sin y cos ( x - y) - cos ( x + y) . 6. Multiply E�= l sin kt by 2 sin (tj2), and use the fact that 2 sin x sin y cos ( x - y) - cos ( x + y) Multiply

=

=

to obtain a telescoping sum.

Pointwise Convergence of Fourier Series

4.6

(4. 5 . 9 , 4 .5 .10, 4. 5 .12, f 4. 6 .6.

In the previous section we obtained some results and concerning criteria for the Fourier series of to converge at a point. In this section we deal with what the series converges to. We follow the chronological order of development and prove D irichlet's pointwise convergence theorem then Jordan's improvement Dirichlet showed that the Fourier series of reasonably smooth functions converge pointwise to

4.5 .13)

4. 6 . 2 , f (r ) + f(t+) 2

f

t.

for all Using Bonnet's mean value theorem, Jordan extended Dirichlet's result to functions of bounded variation. It is, of course, not necessary that a function be of bounded variation for its Fourier series to converge point wise. The nondifferentiable functions of equation are of unbounded variation, yet their Fourier series converge uniformly! The idea of both pointwise convergence theorems and is as follows. By c ) , the nth partial sum of the Fourier series for f may be si n��n+,.%2 )U of equation written in terms of the Dirichlet kernel as

4.5 .1(

(4. 25) sn (t)

(4 .18) 4. 6 . 2

Dn (u) =

4.6.6

1 ( " [f (t + u) + f (t - u)] Dn (u) du 27r Jo -21 i0 ll" f (t + u) Dn (u) du + -27r1 i0 ll" f (t - u) Dn (u) du. We will show that under suitable hypotheses the last two terms converge to f(t - ) f (t+) and --' respectively, as 2 2 7r

n -+ 00 .

4. Fourier Series

Recall (Section

189

4.5 ) what Dn (u) looks like: 2n+ l

The Dirichlet kernel

Dn (u)

at u = 0, then stays at about constant amplitude Dn (u) jumps up to after the first crossing at u 7r / ( n + 1/2) while becoming increasingly more oscillatory. If we break the integral 21 fa7r f (t + u) Dn (u) du in to

2 n+ 1

1

=

1 7r /(n + 1/2)

7r

1 f (t u) D (u) du + + n 2 7r 17r/(n+1/2) 7r f (t + u) Dn (u) du, 0 the second integral goes to 0 by the Riemann-Lebesgue property 4. 5 . 4 of Dn. Since limu _ o+ f (t + u) = f (t + ), f (t + u) should hover around f (t + ) as gets larger in the (increasingly smaller) interval [0, 7r/ ( + 1/2)], so that 1 1 7r/(n+1/2) f (t + u) Dn (u) du � -1 f (t + ) 1 7r/(n + 1/2) Dn (u) duo 2 7r 0 27r 0 Finally, f; /(n + 1/2) Dn (u) du � 7r for any The net effect is that Dn -27r n

n

n.

ultimately behaves like Dirac 's delta function in the integral

1 1 7r 27r 0 f (t + u) Dn (u) du, sieving out f ( t + ) /2 We show in 4. 6 .1 that this is what happens at 0 for sufficiently smooth f. It is then easy to translate this to other points t we do in equations (4. 32), and (4. 34). By 4. 5 . 6 we know that the kernel behavior of the discrete Fourier kernel
=

as n � 00 .

n

190

4 . Fourier Series r

sin(n + l/2)t n � 00, namely, for any E (0, 71'] and I E Lrl [0 , 71'] , sin (t/2) Imn lro , (t ) sin(sm.n +(t /2)l /2)t dt = I1m'n lo r , (t ) sin (nt+/2l/ 2)t dt . (4. 30) But what is.1.theAs noted limit? inIt Exercise is I (0+),4.3assuming that I' (0+) exists, asandwe show in 4. 6 , if I is differentiable in (a, b) limt-+ a+ f' (t) exists, then I (a+) exists,1and lim I(a + u)U- / (a+) . f' (a+) = t -+lima+ f' (t) = u-+O+ ' (a+)Seeis not right-hand derivative4.I:3 .3,(a)andof IExercise at O-that aMoreover, different flimit. the theRemark after Definition 4. 3-2.is 4.6.1 SELECTO R PROPERTY O F [sin ( n + 1/2) u] /u As usu al, we define sin (n +then1/2)foru] any /u toa Ebe(0,(n71'+] , 1/2) at u = O. If I E L'i [0 , 71'] , and f' (0+) [exists, limn .!.71'. 1r0 I (u) sin (n +u 1 /2) u du limn .!.. 1r0 I (u) sin (n +u 1/2) u du 1 (0+) 2 (4. 3 1) Proof. Add and subtract f (0+) in � fo'" I (u) si n (n � 1/2)u du to get .!.. r I (u) sin (n +u 1/2) u du 71' 10 sin (n l /2)u s n(n + l /2) u = 1 (0+) 71' 10["" +u du + .!.71'. 1r0 [/(u) _ / (0+)] i u du. By (4. 5 .8)(b ), 1mn 10 '" sin(n + 1 /2)u du -_ -71'2 ' so it remains to show only that � f;[1 (u) - I (0+)] Sin(n� 1/2)u du O. Toproaches do this,0 wen �split00 .foSince '" into f; + fr"' , and show that each integral ap f'on(0+)(0, exists, choose an E (0, 71') such that ) Then (f (u) - f(0+)) /u is bounded sin(n + l /2) u du .!.. r [/ (u) _ / (0+)] u 'II' h = .!.'II' . l(� U (u) _ / ( 0 +)] sin (n +u l/ 2 ) u du + .!.71'. 1r [/(u) - I (O+)] sin(n +U 1 /2) u du o as

1·

71'

1·

as

U

r .

r

�

4. Fourier Series

191

I (u) - / (0+) '11") , '" f r [I (u) - 1 (0+)] /u (0, (0, f; 0 n a ( l /'II") fo'" = ( l /'II") f; + ( l/ 'II") f:; 0fo , n I E LHO, '11"] (t+) t E [0 , '11"] . sn (t) -2'11"1 1° '" [/(t + u) + / (t - u)] Dn (u) du = 2'11"1 Jro l (t + u) Dn (u) du + 2'11"1 Jor l (t - u) Dn (u) du, and replace I (0+) by I (t+) in the proof of 4.6 . 1 . The exact same argument shows that I(t+) . (4 . 32) ·n -1 1 "' f (t + u ) sin(n + 1/2) u du = -IIm 2 'II" ° U It is also easy to show that if I E LH-'II" , 0], and I' (t - ) exists for some t E [-'11" , 0], then l (t - ) · -1 1 ° / (t + u ) sin( n + 1/2) u du = --. I1m (4.33) n 'II" U 2 Now suppose that I E LH-'II" , '11"] . If we replace u by - U in the integral of equation (4.33 ) , then f� ... becomes fo"' , and we have that r lim n !.'II" Jro f(t _ u) sin (n +u 1/2) u du = / (2 ) . (4.34) Equations ( 4.3 2) and (4.34) enable us to deduce Dirichlet's 1829 result 4 . 6 . 2 relating pointwise convergence of the Fourier series of a function f to its smoothness at the point from the criterion for pointwise convergence

Since I S clearly integrable on ( r , it follows from the Riemann-Lebesgue property 4.5.6 of the discrete Fourier kernel that 00. Since is bounded on r ) , it is integrable on o as -+ as -+ 00 by the Riemann-Leb esgue lemma 4.4. 1 . r) . Therefore As t o the statement about note that the second integral goes to as -+ 00 by 4.5.6. 0 Now suppose that and that f' exists for some Consider -+

n -+

_ ...

4.5.9.

I [-'II" , '11"] . I' (t+) 1I

4.6.2 DIRICHLET 'S POINTWISE CONVERGENCE THEOREM Let be the 2'11" - periodic extension of an integrable function on If and f' exist everywhere-as happens for any piecewise smooth function then t h e Fourier series ( equation ( 4. 1 6) of Section 4.3 ) for converges to

(t - )

�«(f (r ) + I (t+))

a t every t; a t points t of continuity the series therefore converges t o

I (t).

192

4. Fourier Series

Taylor series are named for Brook Taylor ( 1685-1731) because he was the first to publish about them in his Methodus incrementorum in 1715. Among several others who had essentially discovered such series was James Gregory ( 1638-1675) , who calculated the early terms of the power (i.e . , Taylor) series for tan - 1 x in 1671 as

x3 x 5 x7

tan - 1 x = x - - + - - - + . . . . 7 3 5 By letting x = 1 (assuming that the series converges to tan - 1 x at x = 1 ) , this implies that 1 1 1 1 - - + - - - + · · · = -' 3 5 7 4 which is called the GREGORY SERIES . In those days, this served as a useful way to compute (the convergence is very slow, so it is not in use today) . We use the pointwise convergence theorem 4.6.2 to sum the Gregory series next.

1r

1r

Show that

Example 4.6.3 T H E GREGORY SERIES

1r

1 1 1 1 - 3 + "5 - 1" + . . . = 4" . Sol ution . By Example 4.1 .3, the Fourier series for the piecewise smooth

function

-1r � t � 0 . -4 L sin (2n - 1) t (t) = { -I, 1, 0 < t < 1r n EN 2n - 1 By 4.6.2, the series converges to (f (t - ) + 1 (t+» /2 for every t E [-1r, 1r]. At t = -1r, 0, and 1r, it therefore converges to O. At t = 1r/2, we get 1

IS

•

1r

1 = .! ( 1 - 1 + . 1 - 7I + . . .3) , 1.e., 5

Jordan wanted to extend Dirichlet's result to a wider class of functions. For what kinds of functions 1 does f' not exist? As noted in Ex = 2 sin ( l ) for t 1= 0, ercise 4.3-2(b), the function = is differentiable; since f' does not exist , pointwise convergence of the Fourier series for 1 is not covered by 4.6 .2 at = O. It is the oscillation of 1 that causes the problem. To exclude functions that have infinitely many extrema in an interval, Jordan defined functions of bounded varia tion. (Since 1 = sin 1 has a bounded derivative , it is of bounded variation by Exercise 4.3-6(a,b) .) Even though Jordan wanted to relax the smoothness requirements for pointwise convergence of Fourier series, functions of bounded variation turned out to pretty smooth anyway:

(t+) /t I(t) t (0+) t (t) t 2 ( l /t) , (0) = 0,

1 (0) 0,

4. Fourier Series

193

By 4.3.5 and The 4.3.6,keyfunctions of bounded variation4. 6are. 6 isdifferentiable almost everywhere. to Jordan' s improvement to establish equa tion (4. 3 1) (selector property of the discrete Fourier kernel for f E Li [0, 11"] such that variation; f' (0+ ) exists) forthisthe incontinuous Fourier kernel and4.6.5, functions of bounded we do 4. 6 . 5 . In turn, the key to the in gredient that Dirichlet was lacking, is the SECOND MEAN VALUE THE O REM FOR INTEGRALS 4. 6 . 4 . We say more about it in Exercises 9-11; one place it is proved is Carslaw 1930, pp. 107-112. Another name for 4. 6 . 4 iswhere BO NNET 'S MEAN VALUE THEOREM after Ossian Bonnet, who proved it (1849, 1850) inconvergence connection theorem with an attempt to simplify Dirichlet' s proofinof the pointwise for Fourier series; it was discovered dependently by Weierstrass his lectures, and du Bois-Reymond, who(presented published inversions in 1869butandnever1875.published), 4.6.4 SE COND MEAN VALUE THEOREM FOR INTEGRALS If 9 is continu ous on the closed interval [a , b] and / � 0 is increasing [a , b], then there exists a point c E ( b) such that l b f (t ) g (t ) dt = f (b) l b g (t) dt. Now we can=establish the selector property of the continuous Fourier kernel (t, w ) sinwt/t. 4.6.5 SELECTO R PROPERTY OF sinwt/t If / is of bounded variation on [0 , s], 0, then dU /(0+) -11"1 10 8 / ( u) sinwu 2 U Proof. Since f is of bounded variation, it is integrable on [0 , s] and can becanwritten as the difference offunction two increasing functions (4.3.6). Since weto deal with one increasing at a time and then use linearity combine our results, we may assume that / is increasing. As monotone + ). functions have one-sided limits everywhere, it is proper to speak of / (0 d Add and subtract f (0+) in .; f; /(u) SInUWU u to get S /(0+) is sinwu du !11" l f ( u ) sinwu d u U 0 0 u is ! du o + 0 [/ ( u) - / (o+ ) ] sinwu U It follows from 4.5.8(b) that sinwu du = - . 1m is0 -u 2 on

a,

s >

1·

--

IIllw -+ oo

=

11"

11"

1·

w -+ oo

11"

4. Fourier Series

194

Itend,remains to show only that � I;[f (u) - f ( o +)] "in"w " du 0. To that suppose 0 > 0, and choose r E (0, ) such that f (u) - f (0 + ) < 0/2 on (0, r] . Now split I; : sinwu .!.7r r [f (u) - f ( o + ) ) u du io -+

s

Since integrable, [f (u) - f (0+ )) /u is integrable on ( r, ) and it follows from thef is Riemann-Lebesgue lemma 4.4. 1 that I I: I < 0/2 for sufficiently + large w. Since (u) ) is f -f ( 0 an increasing, nonnegative function of u, and sinwu on value [0, r) when defined to be w at (hence integrable /utheis continuous there), second mean theorem (4.6.4) implies that for some a E (0, ) r .!. r [f (u) - f ( O + ) ) sinwu du '!' [f ( ) - f (O + )) l sinwu du o u 7r 7r io U Since I I: sin"w" dul -s 7r by 4.5.2( b ) , it follows that s ,

°

r

,

=

r

a

The exact same steps as beforeconvergence lead to equations (4.32)-(4.34) , and ul timately to Jordan' s pointwise theorem 4.6.6. 4.6.6 POINTWISE CON VERG ENCE FOR BY If f is the 27r-periodic exten sion of a function of bounded variation on [ - 7r , 7r) ( which implies that f is integrable on [ -7r , 7r) ) , then for any t, its Fourier series converges to

� (J (C) + f (t + )) . The twentieth century has seen a dramatic enlargement in the class of functions f whose Fourier series converge pointwise to f. Carleson [1966) proved with no smoothness assumptions that the Fourier series of any 1 E L 2 [-7r, 7r) converges pointwise a. e . to f (t) . Hunt [1968) extended Carleson ' s result tocanallbe1 found E L [- 7r , 7rJ, 1 < < 00. Proofs of Carleson' s and Hunt' s results in Mozzochi 1971 and J!Ilrsboe and Mejlbro 1982. In Fejer showed (4.15.3) that for any f E Ld-7r, 7r) at any point t at 1904 which 1 (t - ) and 1 (t + ) exist, the Fourier series for f is (C, 1) summable to [/ (t - ) + f (t + )) /2. p

p

4 . Fourier Series

195

Exercises 4 . 6

1. S U2 M S O F INVERSE SQUARES Prove Euler's formula: l:n E N 1/ n 2 11" / 6 . 2 2. SUMS . . . = 11"O2F/ 8IN. VERSES O F SQUARES Show that 1 + 1/3 + 1 / 5 2 + 3. wise PIE CEWISE SMO OTHNESS Which of the following functions is piece smooth? (a) I(t) = { sin0, 1/t, t0 =< O.t :::; 1, tsin 1/t, t0 =< Ot . :::; 1, (b) I (t) = { 0, (c) I(t) = { 0,1, tt rational, irrational. { t2 0 :::; t < 1/2, 4. Let I(t) = 0, ' t = 1 /2 . Compute f' (t + ) , and f' (r) · -Ii, 1/2 < t 1. 5. periodic SAME FO URIER COEFFICIENTS If I and 9 are piecewise smooth 211" that have the same Fourier coefficients, then show that I (t)functions = 9 (t) at all t where they are both continuous. { 6. Let / (t) = 0,In(� /t) ' 0- 11"< t :::;t :::;1I" , O. . :::; (a) Show that:::; I satisfies the conditions of the Jordan theorem1I", 4. 6 .6 for-11" -11" a0 < 0 < b :::; i.e ., that I E BV [a, b) LH- 11"] for :::; a < < b :::; (b) Show that [I (u) + I ( - u)] /u f!. LHO, r] for any r 0, i. e ., that I does not satisfy the condition [f (t + u) + I (t - u) - 2/(t)] /u E L1[O, r] of Dini's test, 4. 5 .12, at t = O .

=

ODD

,

11" .

:::;

C

11" ,

>

7 . D E L A VALLEE-POUSSIN CRITERION F O R POINTWISE CON VERG EN C E

(

a) Let extended, let E R, and let I E AILH-(x,1I",t)1I"]=beI 211"+-periodically (x t) + I (x - t) - 28 , x, t E R. Fix x E R. If (x, t) t A 8

v

=

f; / (x, u) du

196

4. Fourier Series

is ofbj forbounded as a function0 oft t in0a, then closedshowinterval some avariation 0 , and (x, t) that [a, the Fourier series of f converges pointwise to s at x. 's (b) pointwise Use the deconvergence la Vallee-Poussin criterion of (a) to deduce Jordan of bounded theorem 4.6.6 for functions f variation. (c) Use the de la Vallee-Poussin criterion to deduce Dini's test 4 .5 . 12 . 8 . FIRST MEAN VALUE THEOREM Let 9 be continuous, and let f be increasing interval [a, bj. Prove that there exists c E [a, bj such that on the closed J: 9 (x) df (x) = 9 ( c) (f (b) - f (a)) . 9 . SE COND MEAN VALUE THEOREM I Let 9 be continuous, and let f be increasing on [a, bj. (a) Prove that there exists c E [a, bj such that >

v

->

as

->

J: f (x) dg {x) f (a) J: dg (x) + f { b) J: dg { x) . Show that there exists c E [a, bj such that J: f (x) 9 (x) dx = f (a) J: 9 (x) dx + f (b) J: 9 (x) dx. =

(b)

increason ing function on the closed interval [a, b],Letand9 belet afcontinuous be nonnegative [a, bj . (a) Prove 4.6 .4, namely, that there exists a point c E (a, b) such that J: f (x) 9 (x) dx = f (b) J: g{x) dx. ( b) If f is decreasing on [a, b], then show that there exists a point c E (a, b) such that J: f ( x ) g {x) dx = f (a) J: g (x) dx. (c) Show that (a) or (b) can fail if we omit the requirement f � o. 1 1 . SE COND MEAN VALUE THEOREM III Let 9 be continuous, and f be increasing of Exerciseson9 (theb ) , closed and 10.interval [a, bj. Prove the following extensions (a) Show- thatd, forthereanyexists real numbers and d such that c ::; f (a + ) ::; r E [a, bj such that 10 . SECOND MEAN VALUE THEOREM II

c

f (b ) ::; J: f {x) g (x) dx = c J: g (x) dx + d t g {x) dx.

4. Fourier Series

(b)

197

If f � 0, then show that there exists a point r E [a , b] such that J: I (x) g (x) dx = d J: g (x) dx.

Consider the trigonometric series (4.35) S (t) = � + L: an cos nt + L: bn sin nt, neN neN where are real sequences. Now consider the power senes ( an)n � o and (bn)

12. CONJU GATE SERIES

(

4 . 36 )

(a) Show that for

cn _- { aoan/2- , ibn , nn =� 01 , ' ,and z = eit , the real part of L: �= o cnzn is (t), and the imagi n nary part of L:�= o cnz is THE CONJUGATE SERIES O F (t), (t) = L: an sin nt - bn cos nt. n eN If the series in equation (4.35 ) is the Fourier series of 1 E s

s

0"

Li [ - 7I", 7I"] ,

so that

an = .; D,. I(t) cos nt dt, n E N u {O} ,

and bn = .; J�,. I(t) sin nt dt, n E N , then( , t),werespectively; will denote wethedenote series the(t)nthandpartial(t) sums by (f,of t) (f,andt) andf (f, t) by Sn (f, t) and O"n (f, t), respectively. (b) Let dn (t) = L: � = 1 sin kt, and note that dn is odd. Show that s

0"

0"

s

O"n (f, t) = -./ D,. I(x )dn (x - t) dx,

and also that

O"n (f, t ) = -,.l Io" ( f (t + x) - / (t - x)) dn (x) dx.

( c)

s

0"

Show that and

dn (t) -- cos t / 2-2 sinco(st(/n2+) 1 /2 }t dn (t) - 12-tancost /n2t + 2 · lli!..!!.1

( 4.37 )

( 4.38 )

( 4.39 ) ( 4.40 )

198

4. Fourier Series

(d) Show that there is some constant c such that "./2 Sl. � 2 n t dt -< e ln n . sm t

Jro

(e) Show that there is some constant k such that

� J�". l dn (t) 1 dt = � Jo"' l dn (t) 1 dt �

kin n . (f) pleted Prove thean earlier followingresultresultof Fejer due to(Zygmund Lukacs (1920) , which com 1959, p. 107) . The presentation here .follows Bary 1964. Let f E LH-7I", 7I"] be peri odically extended If has a jump discontinuity at t , and f (t + ) f + ( and f (t- ) exist, let d = f t ) - f (r). Then show that limn UnIn(t,n t) = -71"d .

Hints

4

2

The Fourier( cosseries for f (t) = t 2 on the closed interval [0 , 271"] is ; + 4� LmeN n 2nt ". sinn n t ) . Consider the expansion at t = 271" 2. By Example 4.1 .5, the Fourier series for f (t) = It I on [-71", 71"] is � ;. ( cost + '; + o; + . . .) . Now set t = O. 7. (a). The Fourier series of f converges at x to s if li![l J; D"f (x, t) sin(n� 1/2)t dt = 0 for some 0 < r < 71" by 4.5 . 10. Now, A f( x, t) - 8(tv(x,t)) 8t - v t 8v8t almost everywhere. Thus J; v (x, t) si n(n � 1/2)t dt J; � f (x, t) si n (n � 1/2)t dt in( n l/2)t + Jrro t 8v 8t s +t dt . Toof bounded treat the variation, first integralandJ;v v(x,(x,t)t)-+sin(0n�as1/ t2)t dt ,O.noteNowthatarguev(x ,ast) inis the proof of Jordan's pointwise convergence theorem 4.6.6. For the integral lr 8v sin (n + 1/2) t 1.

_

C 23t

C 25 t

+

u

-+

t ut o !:j

t

dt ,

note thatandsince v (x, t) is of bounded variation, �� exists almost every where is integrable. Now argue as in the proof of Dini's theorem, 4.5. 12.

4. Fourier Series

199

(b)(f . Since f is of bounded variation, �f (x, t) = f (x + t)+ f (x - t ) (x - ) + f (x + )] is of bounded variation in [a, b] , a 0, for any x E [-11", 11"] . It follows that v (x, t) is of bounded variation in an to the right oft = 0 (prove). Now show that interval lim t f; (f (x + t) + f (x - t) - f (x + ) - f (x - )) dt = 0 By(f the+ ) result of (a) , the Fourier series of f converges at x to (x) = (x + f (x - )) /2. ( c ) . If �f (x, t) /t E Ll [O , r], then h (x, t) f; A f C: . u ) du is abso lutely continuous on [0 , r], and therefore is of bounded variation there. Now consider >

t-O

s

=

v (x, t) = t f; ud� h (x, u) du h (x, t) - t f; h (x, u) duo

=

8.

In thisfunction hint and the one= cforonExercise that any USe the 9factdefined stant [a , b] and9(aany) Wefunction on [con a , b] f (x) c the Riemann-Stieltjes (x) is equal to ( g (b) 9 (a)). sup f9:(f(x) Let inf 9 ( [a, b]), Mintegral [a , b] ).dgThen -

m =

=

m (f (b) f (a)) ::; f: 9 (x) df (x) ::; M (f (b) f (a)) . Since the result and observe thatis trivial if f (b) f (a), suppose that f (b) :/; f ( a), f ( b ) �f ( a ) f: g (x) df (x) -

-

=

is an intermediate value of the continuous function 9 on [a , b]. 9. gration ( a). Integration by parts is justified by Exercise 4. 3 -9(g). Using inte by parts and Exercise 8, f: f (x) dg (x)

10.

f (b) g (b) - f (a) g (a) - g (c) (f (b) - f (a)) f (a) f: dg (x) + f (b) f: dg (x) . ( b ) . Consider the continuous function h (x) f: g (t) dt, and USe ( a) . ( a) . Let w (x) a < x ::; b, and let w (a) 0, and USe Exerci se 9(b) withf (x)w inforplace of f. (c ) Let f (x) g (x) x on

=

=

=

== =

[-1, 1]. 1 l . ( a) .. Let w (x) = f (x) for a < x < b, w (a) c, and w (b) = d. Now use Exercise 9 (b) with w and g. Note that this result is true eVen for 9 E L l [ a, b]j cf. McShane 1947, p. 210. =

200

12.

4. Fourier Series

(b) . The result of equation (4.37) uses arguments like those of 4.5 . 1 . As for equation (4.38) , let in equation (4.37) . Then

w x-t -.,.11 r::;� t f( t + w)dn (w) dw -.,. f::1r f(t + w)dn (w) dw -.,.1;f�,'lt J(t + w)dn (w) dw - fo f( t + w)dn (w) dw.

=

w

Now let = - u in the first integral, and use the fact that (c) . See the hint to Exercise 4.5-6 .

dn is odd.

1::;= 1 sin ( 2k 1 ) t Si�� t Hence /2 2 1::;= 1 fo'lt /2 sin ( 2k - 1 ) t dt J('Ito Si�smtnt dt �n L- k = l 2k 1 1 :::; �n L- k = l k1 :::; c In since 1:: ; = 1 1 j k is asymptotic to In in the sense that [1::;= 1 1 j kl / In 1 as � 00 (look at lower sums for In n l j ( ) fI t dt ) (e) . Since 1 - 2sin(t/2) 1 - - 1 tan t 2 tan(t/2) is 0 ( 1 ) for t E [0, 11"] , it follows from part (c) that 1 - cosnt dn (t) + sin2nt 2tan( /2) t - cosnt + 0 (1) ( 4.41) 2ll tan(t/2) cosnt + 0 ( . 1) 2sin(t/2) Thus i �2 ( nt/ 2) + 0 ( 1) , l -:-cosnt t /2) + 0 (1) s sm(t/2) 2sm( =

-

(d) . By Exercise 4.5-5 ,

.

n,

_

n

.

n

n

�

n

_

With

2 (n t /2 ) 0 1 ) dt + ( Jo('It I dn (t) 1 dt 1r Jo('It smsm(t/2) =

1.

w t j 2 this becomes 2 nw dt 0 ( 1 ) . /2 'It Si� ( 'It Jo sm w +

=

.

1.

Now use (d) . (f) . By assumption, f )-f ) as By equation (4.38) we have �

x 0+ .

4"

=

and it follows that 1. 'It

"2

(t + x

=

(t - x d + f ( x ) where f ( x ) 0 �

4. Fourier Series

Show first that

n n n ];0" dn(x) dx

lim , I

_

=

Observe that

l.

( 4.43)

n -kk",- I 0" - - L.. " dn ( x ) dX -- - � � nk = 1 ( cas -kk-" L.. k = 1 cas

Jfo

_

For m = (n - 1) /2 this becomes 2 1 + 1 + . . . + 1_

(

3

201

-

1)

k

.

_ ) 2m + l

which is asymptotic to

2 (ln m - t ln m) = ln m .

In turn, this is asymptotic to In n , which establishes (4.43) . Next , we show that lim , 1 ];0" ( 4.44) =

n n n dn(X) f (x) dx O. For any r > 0 there exists 0 > 0 such that I E (x) 1 < r for 0 < x < o. Hence , by (e) , there is some constant k' such that _

To consider f.s" By equation (4.39) �

x 1r.

dn(X) f (x) dx we must get a bound on dn (x) for 0 �

I dn (x) 1 � � for 0 < 0 � x < 1r. Since sin x � 2x/1r for 0 � x � 1r/2, it follows that I dn (x) 1 � 1r/x for O < o � x < 1r � 1r /0 for 0 < 0 � x < 1r. sin / 2

Therefore,

fo" dn(X) f (X) dx � f fo" If (X) 1

f

[-1r, 1r).

dx,

which is finite, since E L� This establishes equation (4.44) . From equations (4.42) , (4.43) , and (4.44) , it follows that lim �

n In n

=

-

.4.

"

.

202

4. Fourier Series

4.7

Uniform Convergence

Since sines and cosines are continuous, if a trigonometric series converges uniformly, its limit must be continuous. If a function f is discontinuous, its Fourier series cannot converge uniformly, so this is an easy way to make up convergent series that do not converge uniformly. Knowing when a se ries converges uniformly is imp ortant, because it enables termwise integra tion and with more hypotheses termwise differentiation . By Dirichlet 's pointwise convergence theorem 4.6 .2 we know that the Fourier series of a periodic, piecewise smooth, continuous function f converges pointwise to f everywhere. We can do much better than pointwise convergence, how ever. We show in 4.7.4 that the Fourier series of such a function converges uniformly. We show in Example 4 . 1 1 . 1 that not every convergent trigonometric senes sin + cos +

(

)

�

L an nt L bn nt neN n eN is a Fourier series. ( L nEN sin ntl In n is the counterexample .) Theorems

4.7 . 1 and 4.7.2 provide sufficient conditions for a convergent trigonometric series to be a Fourier series.

4.7.1 UNIFO RMLY CON VERGENT TRIG ONOMETRIC SERIES ARE FO URIER SERIES If the trigonometric series

�o + L an cos nt + L bn sin nt n EN neN converges uniformly on [-11", 11"] , then the limit is a (211"-periodic) continu ous function whose Fourier coefficients are an and bn .

(t)

o / Ln EN an nt L n EN bn nt

Proof. Clearly, 9 = a 2+ sin is continu cos + ous because it is the uniform limit of a series of continuous functions. Since it converges uniformly, given c: > 0, there exists a positive integer N such that for m � N ,

k b e a nonnegative integer, and consider the series 9 (t) cos kt �o cos kt + L an cos nt cos kt + L bn sin nt cos k t . ( 4.45 ) nEN n EN

for all t E [-11", 11"] . Let

=

Since

Ig (t) cos kt -

Sm

(t) cos kt l

::;

I g (t) - Sm (t) 1 <

c:

4. Fourier Series

203

for all E [ - 11", 11"] for m � N, the series of equation (4.45) converges uniformly. Therefore (Example 2.2.8) it can be integrated term by term; hence the Fourier cosine coefficients for 9 are the A similar argument applies to the bn . 0

t

a n.

As a corollary, the next result shows that the terms of any absolutely summable sequence are the Fourier coefficients of a continuous function. 4.7.2 ABSOLUTELY CON VERG ENT COEFFICIENTS If (an)n-?o, and ( b n ) are absolutely summable sequences of real numbers, then

� + L.: an cos nt + L.: bn sin nt n eN

n eN

(4.46)

converges uniformly, and absolutely to a 211"-periodic continu ous function whose Fourier coefficients are (an)n >O and ( bn) .

P roof. Since for every

n,

I an cos nt + bn sin nt l � I an cos nt l + I bn sin nt l � I an I + I bn I ,

the trigonometric series (4.46) converges uniformly and absolutely by the Weierstrass M-test of Example 2 .6.3 d) . The desired result now follows from 4.7. 1 . 0 The Fourier coefficients 4/ 1) 11" of the square wave of Example 4 . 1 .3 comprise a divergent series. In 4.7.4 we show that the Fourier coefficients and bn of a piecewise smooth continuous (which the square wave is not) pe riodic function are even absolutely convergent : 00 . + b We conclude from 4.7.2 that the Fourier series of such a function converges uniformly to it. As we have noted in Exercise 4.3-9(g) , integration by parts is valid for the Lebesgue integral 4.7.3 of absolutely continuous functions.

(

(2n

-

an

L n e N ( I an - I I I n I ) <

4.7.3 INTEGRATION BY PARTS If f and 9 are absolutely continu ous on the interval [a, b], then I and 9 are differentiable a. e., and

I

l b f (t) g'(t) dt = I (t) 9 (t) l: - lb I' (t) get) dt.

If is piecewise smooth, it is absolutely continuous; thus, the integration by-parts formula above is applicable to piecewise smooth functions. The uniform convergence theorem 4.7.4 illustrates again that the smoother is, the better the convergence of its Fourier series. An improved version appears in 4.7.6.

I

4.7.4 UNIFORM CON VERG ENCE FOR PIECEWISE SMO OTH FUN CTIO NS

Let I be a piecewise smooth 211"-periodic function. (a) If f is continuous, then its Fourier series converges uniformly and absolutely to f. (b) Even iff is not continu ous, its Fourier series converges uniformly to f on every closed int erval not containing a point of discontinuity of f.

204

4. Fourier Series

P roof. ( a) By the continuity of I and 4. 6 . 2 , the Fourier series for I converges pointwise togoalI (t)is toat every t. Let (an ) n >O and (bn ) be the Fourier coefficients ofandI .then Our apply show that (an )n >� and (bn ) are absolutely summable, 4. 7 . 2 . Let dn = � J::", f'(t) sin nt dt, n E N, be the Fourier sine coefficients for I' (t). It follows from 4 . 7 . 3 and the periodicity of I that ;:1 1 '" I (t) cosnt dt � I (t) sin ntl :' ", - � '" f' (t) sin nt dt -1I"ndn /n for any n E N.1I"n 1 Similarly,is bpiecewise = cn/ n , where Cn is the nth Fourier cosine coefficient of I' (t) . n Since E L2 [-1I", 11"] . Therefore, by Parse val' s iden I tity, equation (3.9) ofsmooth, 3.3 . 1 , itf'follows that LnEN (d; + c; ) < 00 . Since 2 I cnnl + n1 n E N, 0 ::; ( I cn l - -n1 ) = cn2 - 2and the analogous result holds for dn , it follows that J cn I + I dn I - � (cn2 + dn2 ) + �2 n E N . n' n n 2 Consequently, the series _

'"

_ '"

2'

<

converges. (b) Let g denote the 211"-periodic extension of the function g (t) = t on ( -11", 11") . The Fourier series of g is 2 ( sin1 t sin2 2t + sin33t . . .) . Since g is discontinuous at t = ±k1l", k odd, the Fourier series does not converge uniformly on [-11",on any 11"] . As we show next, the Fourier series for g does converge uniformly interval [-a, a] for any 0 < a < 11". Let Sn (t) = 2 ei� t Si� 2t + Si�3t . . . + (- I t + 1 si:nt ) . Multiply sin ( a + b)both to getsides by cos t/2 use the identity 2 sin a cos b = sin (a - b)+ cos 2"st n (t) _

_

_

_

and

4. Fourier Series

205

0,

[ a] , cos t/2 � cos a/ 2 b and Isin O I ::; 1 for all I Sn +p (t) - Sn (t) 1 ::; b1 ( (n + l )1(n + 2) + . . . + (n + p) (n1 + p + 1) ) t

Since for E

-a,

=

p � 1. Since the terms on the right are independent o f t and En EN n (n1+ l ) converges, it follows that for any > 0, I Sn +p (t) - Sn (t) 1 < for sufficiently large n, all p � 1 , and all t E [- a, a] . The Fourier series therefore converges uniformly on [- a, a ] . We can now use 9 to smooth f out. The function 9 (t c) is continuous except at t = c ± k7r, k odd, where it j umps from 7r to -7r. Let t 1 , . . . , tn be the points of [-7r, 7r] where f is discontinuous, and let Ji = f (tt) - f (t n , i 1 , 2 , . . . , n. By considering the differences h (t t ) h (t;) , it is easy to see that the for all

€

€

-

-

function

h (t) E�= l 1; g (t +

=

is continuous. Since is obviously piecewise smooth and 27r-periodic, its Fourier series converges uniformly by (a) . As we have argued above , the Fourier series for 7r converges uniformly on any closed, bounded interval not containing one of the It follows that the Fourier series for converges uniformly to on every closed and bounded interval 0 not containing a point of discontinuity of

- ti ) f

f

ti .

f.

Discuss the convergence ( mean, pointwise, uniform) of the 27r-periodic extensions of (a) f (t) = -t , and (b) g (t) = cos t 2 on [-7r, 7r) .

Example 4.7.5

/

f

Discussion . Both and 9 are square-integrable, so their Fourier series con verge in the mean to and respectively. By 4.7.4, the Fourier series for converges uniformly to on intervals that do not contain odd integer multiples n7r of 7r; at odd-integer multiples of 7r the series converges point wise to = 7r. Since the periodic extension of 9 is continuous everywhere, the Fourier series for 9 converges uniformly, and 0 absolutely to

f

g, f (f (n7r + ) + f (n7r - )) /2 g. f

In 4.7.4 we showed that piecewise smooth functions have uniformly con vergent Fourier series on closed intervals not containing a discontinuity. We shed the piecewise smoothness in the following uniform convergence theorem. We outline its proof in Exercise

1.

206

4. Fourier Series

4.7.6 UNIFO RM CONVERGENCE FOR CONTINUOUS FUN CTIONS Let / E LH-1I", 1I"] be 211"-periodically extended. Iff is a continuous function 0/ bounded variation on (a, b) C [ -11" , ] , then th e Fourier series 0/ / converges uni /ormly to / on every closed subinterval [a + r, b - r] , r > 0, 0/ [a, b] . 11"

We return to the topic of convergence of Fourier series in Sections. 4.15 and 4 . 1 8 . Here is a summary of some things that we know so far for real valued 211"-periodic functions on R. If

/ is

its Fourier series converges

/ . . / (t + ) + / (r ) pomtwlse to in the mean to

piecewise smooth

piecewise smooth

.:.-0.---'-_.:.-0.--'-

2

uniformly, and absoutely to on closed intervals without discontinuties of /

/

Table 4.7- 1 Convergence of Fourier series

Exercises 4 . 7 1 . UNIFORM CON VERGENCE FOR CONTINU OUS FUN CTIO NS Prove 4.7.6. Let Ll [-1I", 1I"] be extended to a 211"-periodic function on all of R. Let [ a, b] C [- 11" , 11"] , and assume that [a, b] . If is continuous on ( a , b) , then show that the Fourier series for converges uniformly on every closed subinterval [a + r, b - r , r > 0, of [a, b] .

/E

/ E BV / ]

/

Hint

/ is increasing on [a, b] as in the proof of 4.6 .6. Let A / (t, x) = (f (t + x) - / (t)) - (I (t)' - / (t - x)). For any f > 0, the continuity of / enables us to choose r > 0 such that 1 ( 1 (t + x) - / (t)) 1 < f and 1 - (I (t) - / (t - x)) 1 < f for t E [a + r, b - r] and 0 ::; x ::; r' . lt follows that I f;' � [sin ( + 1/2) x]A / (t, x) dx l is uniformly small for all t E [a + r, b - r] for sufficiently small r. For fixed r, show that 1 1� � [sin ( + 1/2) x] A/ (t, x) dx l is uniformly small for t E [a + r, b - r] when n is sufficiently large .

1 . Assume that

n

n

4. Fourier Series

4.8

207

The Gibbs Phenomenon

By 4.7.4(b) , the Fourier series of a piecewise smooth function f converges uniformly to f on any closed subinterval not containing a point of discon tinuity of f. How does the series behave on intervals containing points of discontinuity? In an 1899 letter to Nature (vol. 59, p. the American mathematician Josiah Willard Gibbs, famous for the development of vector notation among other things, considered the function

606)

h (t) = with Fourier series

{

-1r - t -1r ::; t < 0, 0,1r -2 t t = 0, 2 ' 0 < t < 1r, L

n EN

h

sin n t n

.

The function and the seventh partial sum in the figure below.

S7

of the series above are shown

of immediately to the right of 0 is a bit higher than The peak hfact,(0+) = 1r /2; on the left , the minimum of is below h (0 - ) = -1r /2. In i + (. 0 9)1r = i + ( . 0 9 ) (h (0+) - h (0 - )) , that is, overshoots h (0+) by about 9% of the total j ump 1r. A similar thing happens to the left of O. Not only that, this overshoot persists even as 00, despite the pointwise convergence of n (t) to h (t) throughout [-1r, 1r] Unknown to Gibbs, Henry Wilbraham had made similar observa . tions 5 1 years earlier ( Camb. and Dublin Math. 3 ( 1 848) , p . 198) . The American mathematician Maxime Bacher went considerably farther than Gibbs or Wilbraham in 1906 (Annals of Math. 7) . He showed that this 8 7 m ax

87

87

S 7 m ax �

87

n

-->

S

J.

overshoot of Fourier series by 9% of the total jump is a general property

208

4. Fourier Series

of Fourier series in the vicinity of a jump discontinuity. Thus, even though Gibbs's statement was not the first, not accompanied by any proof, and dealt only with a special case, this quirk of nature is universally known today as the GIBBS PHEN OMENON . Such are the vagaries of history. Consider what happens to a step function.

-4

4

The Gibbs Phenomenon

4.8.1 GIBBS PHENO M ENON FOR A STEP FUN CTIO N

Suppose a <

b, and

(t) = { b,a, O-7r:5 :5t :5t <7r • 0, . Then the maximum and minimum of the nth partial sum Sn of the Fourier series for s occur, respectively, at 7r/2n and -7r/2n. Moreover, the peaks ( ) (7r /2n) to the right of 0 "retain their overshoot: t S (0 + ) + [ (0 + ) - s (0 - )] ( � 1 " Si: dt ) limn Sn ( :) 2 b + [b - a] (0.089) , consider

S

sn

max

=

Sn

11

s

while

�

( ;: ) = a - (b - a) (� 1"

dt) a - (b - a) (0.089) . Proof. We will do the problem for a = - 1 and b = 1 . It is easy to convert to general and b using the technique of Example 4.1 .4. As computed in l i;n s n

a

Exercise 4 . 1 .3, the Fourier series for s is i "'" sin For any

:

si t

(2n - 1 ) 7r nL..JN 2n - 1 E

�

t.

n E N, let Sn denote the nth partial sum of the Fourier series; then dSn = 4 � ; L..J cos (2 k - 1 ) t . dt k =l

4. Fourier Series

209

By the result of Exercise 4.5-4, for t rt 1I"Z, d8n - -4 � dt 11" kL-= l cos (2k 1) t _- -11"4 sinsini2nt , reveals that we have which is at0 when2 t and = ±11"minima /2n. Another at -11"/2n.differentiation maxima The locations ±11"/2nSubstituting of the peaks n +11" / obviously approach 0 from the right and left, respectively. t = 1I"/2 n we get �11" Lnk = l sin [(2k2k--1)1I"1 /2n] � " n sin [(2k - 1)1I" / 2n] . � 1I" L-k = 1 (2k - 1)1I"/ 2n n · The latter sum is a Riemann sum for sint/t on [0, 11"] of gauge 1I"/ n. As n 00, it therefore converges to sint t. -11"2 10 11" t Thetheintegral Si (x) = f; [(sint)/tj dt, pronounced "sigh of x," is known SINE INTEGRA L , and its values are tabulated in many places (e. g ., Jahnke et al. 1960). The value of � foil" [(sin t)/tj dt is approximately 1.179, so 0.179 lim8n (1I"/2 n) 1.179 = 1 + 0.179 2 · 2 = (0+ ) + 2 [8 (0+ ) - 8 ( 0 - )] . The argument for limn 8n ( /2n) is essentially the same. Nowdiscontinuity suppose thatat 1c Eis [-11", any 11"piecewise smooth periodic function with(t) =a jump ] . With 8 the step function of 4. 8 .1 (8 8 (t) =discontinuity -1 for t E [-11",at c0)),by and [0, 11"] , and we1 forcant Eremove the jump taking= [I ( c+ ) - 1 (c )] /2 _

-

--

-+

d

as

�

n

s

0

- 11"

J

( c+) = 1 ( c+ ) +2 1 (c ) , then is continuous at c. Since 1 is piecewise smooth, we can choose r > 0 small enough to avoid other jump discontinuities of I, and make continuous onseries [c - forr, c + rj.converges By the uniformly uniform convergence theorem 4.7.4(b) the Fourier to on [c r, c + rj . Since J 8 (t - c) is just a constant multiple of a shifted version of 8, the peaks Taking

v

( c ) = (c - ) v

= v

v

v

v

v

-

210

4. Fourier Series

of its Fourier series remain at J + (2J) . ( 1 . 1 7 9/2) as _ c+ , the peaks of the Fourier series for

t

1

t - c+ . Hence , as

(t) = (t) + J S (t - c) v

remain at approximately

=

v (c) + J + J ( 1 . 1 7 9)

I (c+) + [/ (c+ ) - / (c - )] ( 1 . 1 7 9/2) .

More precisely, the Fourier series Sn for 1 is such that

)

(

lim sup s n c , c + r = n

1 ( c+ ) + [I (c + ) - 1 ( c - ) ] (1 . 1 7 9/2)

and lim n inf Sn (c, - r )

C

= 1 (c+) - [t (c+ ) - 1 (c - ) ] ( 1 . 1 7 9/2) .

The Gibbs phenomenon is thus seen to occur at any jump discontinuity of a piecewise smooth function. We consider a method of summation 1) summation) i n Section 4. 16 that eliminates the overshoot.

((C,

4.9

A Divergent Fourier Series

Where does a Fourier series converge pointwise? At least , where does the Fourier series of a continuous function converge pointwise? The question has stimulated science (not just mathematics) for two centuries. Various attempts to solve it have led to many diverse imp ortant discoveries. At the end of Dirichlet 's 1829 article in which he proved his pointwise convergence theorem 4.6.2, he commented that he believed that the Fourier series of every continuous function converges pointwise to the function at every point. Such greats as Riemann, Weierstrass, and Dedekind expressed similar beliefs over the next 40 years. They were all wrong. In 1876 P. du Bois-Reymond gave an example of a continuous function whose Fourier series does not converge at the points of a dense subset of [- 71" , 71"] . Fejer simplified du Bois-Reymond 's construction in 1909, and gave other exam ples of continuous functions whose Fourier series diverge at certain points. Korner ( 1 988, pp . 67-7 3) elaborates one of du Bois-Reymond 's examples, and produces a continuous function 1 on [-71", 71"] whose Fourier partial sums Sn evaluated at 0 are such that lim su Pn ISn (0) 1 - 00 ; Hardy and Rogosinski ( 1950) give further examples. As a consequence of these re sults and Kolmogorov ' s 1926 proof that there is a function 1 E Ld-7I", 7I"] whose Fourier series diverges everywhere (Bary 1964, p . 455; Zygmund 1959, p. 3 10; Katznelson 1968, p. 59) , it was believed by many that it was only a matter of time until it was shown that there were continu ous functions whose Fourier series diverged everywhere. The consensus was

(t)

4. Fourier Series

211

wrong again. The Swedish mathematician Lennart Carleson [1966] proved that the Fourier series of a 211"-periodic continuous function has to con verge pointwise to the function almost everywhere. Indeed, for the spaces Lp [-1I", 1I"] , 1 :S p < 00, and X [a, b) (and others) , there is the following dichotomy:

X

=C

=

X

Either there exists f E whose Fourier series diverges every where or the Fourier series of every f E X converges almost everywhere (Katznelson 1968 , p. 59) . As noted above, Kolmogorov showed that Ld-1I", 1I"] is of the former typ e . Lusin had conjectured i n 1915 that L 2 [-1I", 1I"] i s o f the latter type, that Fourier series of L 2 functions converge almost everywhere. Carleson con firmed Lusin's conjecture in 1966 ; then Hunt [1968] extended Carleson's result to all the Lp [-11", 11"] for > 1. Kahane and Katznelson (Katznelson 1968 , p. 58f) demonstrated that given any set E C [-11", 11"] of measure 0 , there is a continuous function on [-11", 11"] whose Fourier series diverges at every point of E. We present Kolmogorov's marvelous argument here that there is a con tinuous function whose Fourier series diverges at certain points . We do not construct the function; we only show that such a function must exist . The principal tool of the proof is the Banach-Steinhaus theorem, also known as the principle of uniform boundedness. The proof is widely available (Bach man and Narici 1966 , p . 251) . We may take the following special case as its statement .

p

4.9.1 PRIN CIPLE O F UNIFORM BOUNDEDNESS If (An) is a sequence of continu ous lin ear maps of a Banach space X into a normed linear space Y that is bounded at every x E X ,

n II An x l1 < 00,

sup

then s UP n II An ll < 00 . As the domain space X of the theorem, we take the sup-normed Ba nach space C [-11", 11"] of real-valued continuous functions on [-11", 11"] ; for the codomain we take Y = R. Now use the nth Dirichlet kernel

Dn (t)

=1

+

n

L 2 cos kt = k =l

212

4 . Fourier Series

By 4.5 . 1(a) , the nth partial sum of the Fourier series for a function Ll [ -11", 11"] is given by

=

xE

Sn (t) 21 1" X (u) Dn (t u) du, so ( since Dn is even ) , An x = Sn (0) . By Example 3.9. 1 ( c ) , we know that the An are continuous on C [ ] and that " II An ll = 21 1 I Dn (t) 1 dt. If we can show that II An II -- 00, it will follow from the Banach-Steinhaus theorem that there is some continuous function x such that I An (x) 1 = I Sn(O) 1 -- 00, i.e . , that the Fourier series for x diverges at t = O. Since sin (t/2) :S t/2 for 0 :S t :S " 1 II An ll 2 1 I Dn (t) 1 dt .!. 1" sin ( n + 1/2) t dt sin (t/2) I 0 � 1" sin (n + l/2) t l dt. > t 0 Making the substitution u (n + 1/2) t, we see that for any n E N 2 1 ( n + l /2)" -I sin u l duo dt 2: ( 4.47 ) (t) Dn 1 I 21 1 " U 0 But consider the function I sin u l /u for > 0, -

_"

11"

- 11" , 11" ,

_"

11"

11",

11"

_

"

11"

=

11"

11"

-

_"

11"

u

t n 2,

with triangles inscribed in each lobe. For 2: 11", the peaks of the triangles occur at odd multiples of hence, for 2: the altitudes of the triangles are - 1 ) 11". Thus, summing the areas of the triangles,

2/ (2n

11"/2;

foo I sin u l 2 " ! 11" 2 L...J 2 (2n ) Jo u du > n� 2 -

-

1

11"

= 00 .

( 4.48 )

213

4. Fourier Series

Hence, the terms o n the right of inequality (4.47) g o t o infinity as n � 00, and the argument i s complete.

4.10

Termwise Integration

2:n EN fn of integrable functions with limit f defined on [a, b],

For general series the closed interval if

L fn = f uniformly, then

n EN

la b nLEN fn (t) dt = nLEN lab fn(t) dt . f (t) t/2 [

But consider the 211"-periodic extension of = on -11" , 11"] . It is inte grable , and of bounded variation on [-11", 11"] . Therefore , by Jordan's theorem 4.6.6, for any (-11", 11") ,

tE

t=

L nEN

( _ 1 )n + 1

sin n .

( 4.49)

The series converges to 0 for t = - 11" , 11" . For what sides from 0 to to get

t can we integrate both

2

n

t

t

t2 = 4

( - 1 t + 1 ( 1 - cos t) ? L 2 nEN n n

Since the series of equation (4.49) has a discontinuous limit, it certainly does not converge uniformly on [-11", 11"] . It is therefore not clear that termwise integration is permissible . In a case like this where the limit of the series is known, and the function is easily integrated, the ability to integrate term by term is just a curiosity. But suppose that all we know about a function is its Fourier series. Then the ability to integrate termwise is much more imp ortant. In the case of Fourier series, termwise integration abets convergence be cause it puts n 's in the denominators of the terms in the series (differentia tion puts n's in the numerators) . We prove in 4.10.1 that the Fourier series of any f Lt [-1I", 1I"] may be integrated term by term; not only that , but the termwise integrated series converges uniformly and is the Fourier series of the integrated function. Before we prove 4.10 . 1 , recall a few things about absolute continuity that were first mentioned in Exercise 4.3-9:

E

1. A function f defined on the closed interval [a, b] is ABS OLUTELY CON TINUOUS on [a, b] if for every > 0 there exists 6 > 0 such that for any n E N , for all a i , b i E [a, b] such that a 1 < b 1 � a 2 < b 2 � . . . � an < bn and 2:7=1 (bi - ai ) < 6, we have 2:� l l f (bi ) - f (ad l < f. (

214

4. Fourier Series

n

Since could be 1 , absolutely continuous functions must be uniformly continuous.

2. An absolutely continuous function is of bounded variation. Since any function 1 of bounded variation has an integrable derivative f' [see the remarks after 4.3.6] , if 1 is absolutely continuous, then f' Li [a, b].

E

3. If 1 E L'i [a, b], then g is absolutely continuous, and = J: g' = 1 a.e. [Exercise 4.3-9(f) , Natanson 196 1 , pp. 252-3] .

I(t) dt

(x)

4. 10.1 TERMWISE INTEGRATED SERIES CON VERGE UNIFORMLY

E LH-7r, 7r] have the Fourier series �o + L: an cos nt + L: bn sin nt , n

27r-periodic extension of 1

eN

Let .the

neN

which may or may not converge pointwise. Then: bn converges. (This (a) The sine coefficients bn are such that L: n N e n has an interesting application in Section 4. 1 1 .) (b) The Fourier series for J: 1 is obtained by termwise integration of the Fourier series for f, namely,

(t) dt

an . - bn cos n x ) , ao-x2 + L: bnn + L: ( -smnx n n eN n eN n and it converges uniformly to J: 1 (t) dt on R. (c) For any finite interval [a, b], l b I(t) dt = �o (b - a) + L: [� (sin nb - sin na) - � (cos nb neN -

Proof. By Observation

-

3 before the theorem,

-

cos

na)] .

(4.50)

F (x) lx (/ (t) - �) dt is absolutely continuous. Therefore , F E BV[-7r, 7r] LH-7r, 7r] . It is 27r periodic, since F (x + 2 7r) = Jro (/(t) - �o ) dt + lx+ 21< (/ (t) - �o ) dt F (x) + 1 1< (I (t) - �o ) dt [4. 2 .2] F (x) + . l� 1 (t) dt - 7rao = F (x) . =

C

1<

X

4. Fourier Series

215

Since F i s absolutely continuous, the Fourier series for F,

� + L Cn cos nx + L dn sin nx, n EN

nEN

(4.51 )

converges uniformly to F everywhere by 4.7.4(a) , and we may integrate by parts by Exercise 4.3-9(g) ; with = cos the cosine coefficient

dv

nx dx,

-1r1 1 " F (x) cos nx dx 1 sinnnx " - -n1r1 1" (f (x) - -ao2 ) sm. nx dx -F 1r (x) -bn n E N . - " ---, n Similarly, the sine coefficient dn of F is a n/n, n E N . Substituting these Cn

_"

_"

values in equation (4.51 ) , we get

F (x) = or

Co + ,, ( ann sm. nx - -bnn cos nx ) , � 2 n EN -

-

lx f (t ) dt = 2 bn cos nx ) . an sm. nx - -; aox + "2Co + ,, ( -;�

o

With

nEN

(4.52)

x = 0 i n equation (4.52) w e see that (4.53)

o /2 by Ln EN bn/n in equation (4.52) , we get lx f (t ) dt = bn + " ao x + " an sm. nx - bn cos nx, (4.54) � � n 2 o nEN n nEN n which proves (b) . The result of (c) follows from the observation that f: = f: - foa . D which proves (a) . Replacing

c

-

-

Example 4.10.2 SUMS OF ALTERNATIN G INVERSE SQUARES

Show that

216

4. Fourier Series

(4.49), the Fourier series for f (t) t/2 on (-11", 11") is _ I )n+ l t '" ( sin nt . n 2 nL...,; eN Integrating both sides from 0 to t, we get t 2 _- '" ( _I ) n + l - '" ( l ) n + l cos nt . "4 e N n2 nEL...,;N n 2 nL...,; Now integrate termwise again, this time from -11" to 11" : =

Sol ution . By equation

-

Example 4.10.3

=

Show that

", 1 cos (2n - 1 ) t. I t I -11"2 - -11"4 L...,; n e N (2n - l) 2

=

Sol utio n . By Example 4 . 1 .3, the Fourier series for

, -11" t 0 , . 4 L sin (2n - 1) t ' f (t) = { -I 1, 0 t 11", 11" n E N 2 n - 1 and the result follows from equation (4.54) and the result of Exercise 2(b), namely that 1 11" 2 · 0 nELN (2n - l) 2 S <

<

<

<

IS

-

=

Exercises 4 . 1 0 1 . Use the technique of Example 4. 10.2 to show that :

(a

) (t3 - 1I"2 t ) / 1 2 l:n eN ( -I t (sin nt) /n3 , -11" t 11".

=

<

<

(b) l:neN 1/ (2n - 1) 2 11"2 /8. 2. Periodically extend f(t) (11" - t) /2 , 0 t 211", and find its Fourier

=

=

<

<

expansion. Then use the termwise integration theorem 4 . 1 0 . 1 to show that 1!:1 :t: = '" , 0< < W

l n 2 nt 2 - 4 n eN - cos

t 211"

•

4. Fourier Series

3.

217

The functions below are assumed t o b e 2 71"-periodically extended.

(a) Show that the Fourier series of

f ( t) = { s0in t , 0- 71"�
Use this to show that g (t) =

{ 01 '-

IS

- 71" < t <

0 � t < 71" ,

cos t ,

2

1 1 .

- + - sm t- 71" 2 71"

cos 2nt

. L 2 4n 1 neN

0,

has the Fourier expansion " sin 2nt 1r + 12 12 cos t 1.1r neN n(4n L.J 2 -I) ·

1.

_

_

(b) From the Fourier cosine expansion

1r2 - 21rt 8

=

" cos(2n + l)t , 0 < t < 71" , L.J (2n + l)2 n�O

show that

1r2t_1rt2 8

_

" sin(2n + l)t , 0 < t < nL.J� O (2n + l )3

71" •

4. D IRICHLET 'S TEST Dirichlet ' s test for the convergence of infinite se ries of constants of the form l: neN X n Yn [Marsden and Hoffman 1993, pp. 287, 309; Taylor and Mann 1972, p . 639] says that if Yn > 0 mono tonically decreases to 0 and Il:�=l Xi 1 � for all then l: neN X n Yn "n cos(22 n-l)t converges. U se thOIS to prove t h a t L.J eN ( n - l) converges pomtwise for all t =P ±h, k E N. [Hint: Show that

M

n

k'fl

cos (2 k -

n,

•

1) t = s��i�n/ for t =P ±h, k E Nu {O} .]

5. Let f (t) = - In 12 sin (t/2) 1 . Show that :

fE

(a) LH- 7I" , 71"] . (b) - ln I2 sin (t/2 ) 1 = cos t + � + � + . . . for t ( c) Use (b) to show that In 2 1 t + i- � + . . . . (d) Use (b) again to show that

c 2 t cO 3 t = -

In 1 2 cos U) 1 = cos t for t

=P 2 (k + 1 ) 71", k E Z .

co; 2t + c�3t -

=P 2h, k E Z . •

.

.

218

4. Fourier Series

t

(e) Use (b) , and (c) to prove that for 0 < < 7r,

'" cos (2 k + l )t - .=.l In (tan t/2) 2 ki..J=0 2 k + l 00

-

•

6 . Use 4.10.1 to show that

sin n t - t <- 27r - Jor t ln (2 sin (1)) 2 dt = i..J n e N n 2 ' 0 < '"

Similarly, show that

J� In ( 2 cos U) )

4. 1 1

dt = Ln e N ( -I t + 1 si�ft ,

-7r

::;

.

t ::; 7r.

Trigonometric vs. Fourier Series

If a trigonometric series (i.e . , a series of sines and cosines) converges point wise to f, must the series be the Fourier series for f? Is there some condition on the coefficients of a convergent trigonometric series that guarantees that the series is a Fourier series? We know by the Riemann-Leb esgue lemma 4.4 . 1 that the Fourier coefficients of a Fourier series must converge to But this is true of any convergent trigonometric series by the Cantor-Lebesgue theorem 4.4.3. Indeed, there is no general way to detect from the coefficients of a convergent trigonometric series that the series is the Fourier series of its limit. There are some sufficient conditions for a trigonometric series to be a Fourier series, namely, uniform convergence or square-summability of the coefficients : If the trigonometric series converges uniformly on [-7r, 7r] , for example , then it must be a Fourier series by (4.7. 1 ) . If ( a ) E then ( /."fii) by the Riesz-Fischer theorem 3.3.4(a) , converges (in the mean) to some E L2[-7r, 7r] , and is the Fourier ( /."fii) series for As a consequence of 4 . 1 0 . 1 we know that the termwise integrated (J�) Fourier series for a function E Ll [-7r, 7r] must converge uniformly to J� If it doesn't , then the original series is not a Fourier series. More over, the Fourier sine coefficients of must be such that converges by 4 . 1 0 . 1 (a) . Fatou used this in 1906 to show that even though the trigonometric series

O.

I.

I(x) dx.

L keZ Ck L keZ Ck

I

k f (Z), e ik t 2 ei k t

I

(bn ) I,

L n eN bn/n

is uniformly convergent on [a, 27r - a] for any a E (0, 7r) , it is not a Fourier series, a point we take up in Example 4.1 1 . 1 . Before doing that we note the following convergence test : Dirichlet's test [Marsden and Hoffman 1993, pp. 28 7 , 309] for the uniform convergence of (t) , t E T c R :

Ln eN In (t) Un

4. Fourier Series

219

Assume onthatT.UIfn )gand(t)(g�n )0aredecreases sequencesto of0 R-valued functions defined monotonically and n uniformly, and for some M, I E; = l /dt) 1 M for all n and E t E T, then neN In (t) gn (t) converges uniformly on T . Example 4. 1 1 . 1 For any a E (0, 11" ) , the trigonometric series sinnt L.J n�2 In n converges uniformly on [a, 211" - a) but is not a Fourier series. Proof. We verify that the conditions are right to apply Dirichlet ' s test. Let T = monotonically [a, 211" - a], where a E (0, 11") . Clearly, the positive terms 1 / In n decrease to o. The partial sums of the sines are uniformly bounded on(and[a ,uniformly) 211" - a) because cost/2 - cos (n + t) t (Exercise 4.5-6) � . sm kt 2 sin (t/2) k= l for t rJ. 211"Z , so for t E [a, 211" - a], 1 :::; 1 . k t :::; I c ost/2 1 + l cos (n + I/2) t l :::; � sm � 21 sin(t/2) 1 I sin(t/2) I sin (a/2) " By Dirichlet's test, therefore, En>2 sin nt/ In n converges uniformly for all t E [a , 211"-Consequently, a); since the series converges uniformly, its limit is a continuous En � 2 sin nt/ In n converges to a function that is function. continuous at every point of [a, 211" - a). If En>2 s·l nn nnt is a Fourier series, :::;

"

L.J

_

_

-3

3

The first 7 terms of the trigonometric series

then EneN 1/ (n In n) must converge by 4.1O.1(a). By the geometry of the

220

4. Fourier Series

situation, for every n

E N, 1 > n ln n

--

and

r

12

oo

In

n+l

dt t In t

dt = lim In (ln t) I � = oo . I t n t b _ oo

The series L: n > 2 1/(n ln n) diverges. Therefore L:n > 2 sin nt/ ln n is not Fourier series. -0

a

As an indication of the delicacy involved here we mention (Stromberg 1 9 8 1 , p.516) that even though L: n > 2 sin nt/ In n is not a Fourier series, L:n > 2 cos nt/ In n is! In 1875 P. du B ois-Reymond showed that if a trigono metric series converges to a Riemann-integrable function f in (- 71" , 71") , then the series is the Fourier series for I n 1 9 12 de la ValIee-Poussin improved the result to Lebesgue-integrable functions:

f.

If a trigonometric series converges pointwise everywhere to a finite-valued function, then the series is the Fourier series of the function .

Ll

(The series L: n > 2 sin ntl In n is therefore not integrable on ( - 71" , 71" ) . ) For the du Bois-Re ymond-de la ValIee-Poussin theorem, see Natanson 1961 , vol. II, p . 52, Bary 1964, pp. 326 and 382; or Rees, Shah, and Stanojevic 1 9 8 1 , pp. 209-2 12. For series with monotonically decreasing coefficients, we have (Rees, Shah , and Stanojevic 198 1 , p. 217) the following result . 4 . 1 1 . 2 If (an ) is a sequence of posahve numbers that decreases mon oton ically to 0, and

f (t) = �o + I: an cos nt , and n eN

9 (t) =

I: an sin nt ,

neN then the cosine series is a Fourier series if and only if f E L l [- 71" , 71"] . The analogous statement holds for g .

The fact that not every everywhere-convergent trigonometric series i s the Fourier series of its sum was considered to be something of a defect of the Lebesgue integral, and this motivated Denjoy to generalize the Lebesgue integral to the "Denjoy" integral. Once again , Fourier series motivated mathematicians to expand the concept of integral.

4 . Fourier Series

4.12

221

Termwise Differentiation

By equation (4.49) of Section 4.10, the Fourier series for the periodic ex tension of I (t) = t on [-11", 11"] is

2

L n eN

( _ l) n + l sin nt . n

Termwise differentiation yields

2

(-It + 1 cos nt. L neN

Since cos nt f+ ° for any t E [-11", 11"] as n --+ 00 , the latter series does not converge, so termwise differentiation fails. This I, however, has disconti nuities at odd multiplies of 11". Sufficiently smooth functions have termwise differentiable Fourier series. 4.12.1 TERMWISE DIFFERENTIATION Let f be a continuous, piecewise smooth 211"-periodic function on all of R with Fourier series

�o + L an cos nt + L bn sin nt. neN neN

If f' is piecewis e smooth, then the series can be differentiated term by term to yield the following pointwise convergent series at every point t: I'

(t + ) + I' (t - ) = L (nbn cos nt - nan sin nt) . 2 neN

Proof. By the uniform convergence theorem 4.7.4 for piecewise smooth func tions, the Fourier series for I converges uniformly and absolutely to I (t) on [-11", 11"] . The cosine coefficients of f' are

Co = -11"1 j 1r1r I' (t) dt = -11"1 [J (1I") - f (-1I")] = 0, -

and for n � 1 ,

=

I-; j-1r1r f' 1

-; f (t) cos nt n bn

·

1 1r- 1r

(t) cos nt dt +

; j1r f (t) sin nt dt - 1r

dn - nan .

Similarly, the sine coefficients of f' are given by = The Fourier nb (cos nt - nan sin nt) of f' therefore converges pointwise series by the pointwise convergence theorem 4.6 .2. 0 to (I' (t+ ) + I'

Ln eN n (t )) /2

222

4. Fourier Series

The rate of convergence of a series is directly related to how rapidly the terms of the series approach In order to compare rates of convergence, the following terminology is helpful.

O.

Definition 4.12.2 SMALLER ORD ER

9

9

We say that (n) is of SMALLER ORDER than h (n) , or (n) goes to 0 0 , and we write o ( h ) ; "g o ( h)" faster than h (n) , if limn-+oo " is pronounced g is little oh of h." 0

�� =

9=

= an = /' .

Let / be a piecewise smooth continuous 27r-periodic function with Fourier coefficients an and bn . We saw in the proof of 4.12.1 that dn l n and cn l n , where cn , and dn are the Fourier coefficients of By the Riemann-Leb esgue lemma 4.4. 1, � 0, and dn � Hence , for a piecewise smooth continuous periodic function, nan = a n i (lin) � In other words, � 0 faster than lin; similarly, b � 0 faster than lin as well; equivalently, the Fourier coefficients of a piecewise smooth continuous periodic function are of smaller order than lin. (More generally, this is true for absolutely continuous functions; see Exercise 4.12-3.) If / is smoother, then its Fourier coefficients approach 0 even more quickly:

bn

=

O.

Cn

n

an

O.

4.12.3 SMO OTHN ESS AND SPEED O F CONVERGENCE Let / be a con tinuous 27r-periodic function with Fourier coefficients an and bn. If for k E N , /' , . . . , /( k - l ) are continuous and /(k) is piecewise continu ous, then nkan 0 and nk bn 0 as n � 00. �

�

Exercises 4 . 1 2 1 . Assume that /, /' E L�;[-7r, 7r] with

J::'". / (t) dt = O. Show that

2. SQUARE SUMMABILITY IMPLIES ABSOLUTE CON VERGENCE If a sequence cn goes to 0 faster than 1/ n 2 , i .e . , limn 1/�2 0 , show that

E n eN Cn converges absolutely.

=

3. MAG NITUDE OF FOURIER COEFFICIENTS Let / be a 27r-periodic function. (a) If / is absolutely continuous with Fourier coefficients an and bn , show that = o ( l / n ) and bn = o ( l / n ) . (b) Let If /(k - l ) is absolutely continuous, then show that an (link) , and bn (link) .

an k E N.

=0

=0

4. Fourier Series

4. BIG OH FOR FUN CTIONS For real-valued functions

I�I is bounded as t � a, f g.

(g),

223

f and g, if

where a could b e ±oo , then w e write = 0 and say f i s "big oh" of or f is AT MOST OF ORDER If limx--+ a then w e write (as � ) Show that :

g,

f '" Lg

x a.

f (x) / g (x) = L,

(a) f 0 as t � if and only if is bounded in a neighborhood of a . (b) sin t 0 as t � for any c) sin t '" t as t � O. t 2! + t 4 4! + 0 (t6) as t � O. (d) cos t ", (e) sin t - l/t = 0 on the interval (0, 1r/2) .

=

(

(1) = (1)

a

a

1 - 2/

1/

f

(1)

a.

/

(xn),

( Yn ) n '" L Yn

5 . BIG OH FOR SEQUENCES For sequences of real num and bers we say =0 if � J( for n � N , and say Xn is "big oh" of If limn (as � ) = then we write X Show that:

Yn '

Xn

( Yn) X n / Yn xn/ Yn L,

x a.

=

(1) (x n ) is bounded . n = ( Yn), 2:7= 1 X i = 0 (2:7=1 Yi ) . 2:7; 1 1/i = In n + 0 ( 1) . Suppose that Xn � 0 for all E N . Then 2: n E N Xn converges if and only if 2:�= 1 X i = 0 ( 1 ) .

(a) X n 0 if and only if (b) If X 0 then (c) (d)

n

Hints 1 . Use Parseval's identity 3.3 . 1 and the comments after the termwise differentiation theorem 4.12. 1 .

f

f' f'(x) dx

3 . If i s absolutely continuous, then exists almost everywhere on [-1r, 1rj ; define f' (t) = 0 at points t where the derivative does not exist . Then f ( - 1r) + f" for t E [-1r , 1rj . Integrate by parts as in the proof of 4 . 1 2 . 1 .

f (t) =

224

4. Fourier Series

4 . 13

Dido 's Dilemma

One measure of the size of a parcel of land is how long it takes to walk around i t . This method avoids the problem of the calculation of the area of an irregular shape. But figures of the same perimeter may enclose very different areas. As Vergil tells it in the A eneid (retold in Henry Purcell 's opera Dido and A eneas) when the Phoenician princess Dido fled her mur derous brother Pygmalion's rampage in Tyre (Lebanon) , she and a few of the faithful arrived on the North African coast circa 900 B . C . in the area that later became Carthage. The stingy local monarch, King Jarbas, said he would allow her to buy as much land as the skin of an ox could surround. Her minions set to work and cut an ox hide into the thinnest strips they could. Having decided that one side would be the Mediterranean coast , in what shape should the "cord" be strung so as to enclose the most area? One of the last scholars of the Neo-Platonic school at Athens, Proclus, described such problems in about 450 A . D . in his commentary to Euclid's first book. He called them isoperimetric problems: Of all curves of a given length, find the shape that encloses the most area. Zenodorus's thoughts on the subject (ca. 100 B.C.) included the following: 1 . Of all polygons with n sides and the same perimeter, the regular polygon encloses the most area.

2. Of regular polygons with the same perimeter, the more angles, the greater the enclosed area.

3. The circle encloses a greater area than any polygon of the same perimeter. It is not known what shape Dido chose, but Carthage did prosper for a time , and she was its Queen. Her love life did not fare as well. When Vergil's hero Aeneas stopped in Carthage, she fell in love with him to Jupiter's intense displeasure . Jupiter told Aeneas to leave . Aeneas came out all right, and subsequently founded Rome. Dido committed suicide. Though Zenodorus had the right idea, his proofs were incorrect . In 1 836 Jakob Steiner thought he had solved the problem, had proved the circle to enclose more area than any curve of the same length. He argued as follows: 1 . Suppose a closed curve all curves of length

L.

C of length L encloses the most area among

2. Then the enclosed area must b e convex (the line joining any two points in the region must lie wholly within the region) .

A

B

C

3 . If two points and are chosen on that produce arcs of equal length, then the straight line bisects the area.

AB

4. The arcs of part 3 must be semicircles.

4. Fourier Series

225

Dirichlet demurred. The problem lay in assuming that a solution existed . Steiner had only proved that if s u c h a c u r v e C exists, then must be a circle. Its existence must be established for the argument to be valid. Some years later Weierstrass did prove that a circle enclosed more area than any other curve of the same perimeter. Our interest is in A . Hurwitz 's 1902 solution to the problem using Fourier series. Suppose that a closed curve is defined by

C

C

x = x (s) , y = Y (8) ,

0�

8 � L,

x t / L, L. x (t) y (t) (bn) (dn ), n N, (ndn, , n N,

where 8 denotes arc length along the curve , and and y are continuous functions with piecewise smooth derivatives. Let = 27r8 so that varies between 0 and 27r as 8 varies between 0 and Suppose the Fourier cosine coefficients for the 27r-periodic extensions of and are ) and denote their respectively ; let n E and E respective sine coefficients. By 4.12. 1 , the Fourier coefficients for and are , - n cn ) respectively. Since ) and E

N U {O}, dy/dt (nbn -nan 2 2 ( �� ) + (*) = 1 , i t follows that ( dxdt ) 2 + ddty 2

(cn),

t (an

dx/dt

()

By the previous equality and Parseval's identity 3 .3 . 1 ( see also Exercise 4 . 1-10 ) ,

( �)') dt ; t (e:)' n 2 ( a� b� c� d�) . +

Ln E N

+

+

+

Using the version of Parseval's identity of 3.3.4(f ) , we can compute the enclosed area as an inner product :

A

Consequently,

L 2 -47rA = 27r 2 L [(nan - dn) 2 + (nbn + cn) 2 + ( n 2 - 1 ) (c� + dn) 2 ] 2:: O. nN ( 4.55 ) Thus, no matter what C is, the upper bound for A is L 2 / 47r. This fact , L2 A - 47r ' is known as the IS OPERIMETRIC INEQ UALITY . Of course, if C is a circle of E

<

radius r , then

226

4. Fourier Series

but could any other curve maximize the area? In order to have L 2 /41r = A , the right side of equation (4.55) must b e O . This means that

and Hence

an = bn = en = dn = 0 for ' t , an d x = 2"O + a 1 cos t + b I sm

a

which satisfies

C

(x

_

2 �O ) + (y

_

n

� 2.

y = 2bo - b l cos t + al sm. t ,

�)

2

= a� + b�,

so must be a circle. Another way of saying this is that among all plane figures of the same area, the circular disk has minimal perimeter. Steiner tried the three-dimensional analogue of the problem as well , and argued that the surface that encloses the greatest volume is the sphere by way of the three-dimensional isoperimetric inequality

Steiner's original argument , even though conditioned on the assumed exis tence of a solution, is easy to follow, and very appealing. Nice discussions of it, and further references to isoperimetric problems of a more general na ture, can be found in Courant and Robbins 1941 , pp . 376f, and Hildebrandt and Tromba 1985, pp. 44-46 and 145-149.

4 . 14

Other Kinds of Su mmability

In practice, convergent infinite series are always truncated; the limit is ap proximated by a finite sum. That being the case, let us consider some other ways of achieving the approximation through a finite sum. For example , consider the divergent series

1-1+1-1+

.

.

·

.

The partial sums oscillate between 0 and 1 , and their "average" is sort of Is there a sense in which this series "converges" to p In G . H . Hardy 's droll phrase, we say that a series of real numbers converges in a PICKWICKIAN sense,

�.

L n eN Xn

L Xn = x (P) ,

n eN

4. Fourier Series

227

Ln EN Xn is P-SUMMABLE to x, if it adheres to the following code

or that of conduct :

(P I ) (Regularity) If LnEN Xn = x (in the usual sense) , then L nEN Xn x (P) . (We want to extend the notion of convergence.) ( P 2 ) (Linearity) If LnE N Xn = x (P) and Ln EN Yn = Y (P), then for any scalars a and b, L (axn + bYn) = ax + by (P) . nEN (P 3 ) (Decapitation) If Ln EN Xn = x (P), then Ln � 2 Xn = x X l (P) . You can view P-summability as an extension problem. We seek a way to assign a real number (the "sum" ) to a sequence. The linearity requirement means that we are seeking a linear functional on a linear space of cer tain real sequences; regularity commands that the space contain the vector space of summable real sequences and must extend the linear functional defined by ordinary summability on we have a method of summation that obeys these rules , and if

X

M

M.

If

1-1+1-1+

.

.

.

= s

(P) ,

it immediately follows from decapitation that -1

+ 1 - 1 + . . . = s - 1 (P) .

Now multiply by - 1 (by linearity) to get 1

-

1+1-1+...= 1

from which it follows that 1 1

-

-

s

-

s

(P) ,

= s , or s = t, i.e . ,

1 1+1-1+. .= .

2 (P) .

-

An imp ortant special case was developed by the Italian mathematician Ernesto Cesiuo in 1890. An important application of Cesaro summability is Fejer's result 4 . 1 5 .3 that the Fourier series of any function 1 E is Cesaro summable to at all points t where the one-sided limits exist , even when the Fourier series for 1 (t) does not converge! More over, as we illustrate in Section 4.16, Cesaro summation has a smoothing effect-it suppresses the Gibbs overshoot of Fourier series.

[/ (t - ) + l (t + )]/2

L'i [-11", 11"]

=

4. Fourier Series

228

Definition 4.14.1 CESARO SUMMABILITY

(xn), let Sn = L:?=1 X i , n E N ; the nth CESARO n 1 (J'n = -n L Si ' i= l Thus, (J' 2 t (S l + S 2 ) = t (2X 1 + X 2 ) , and (J'3 i- (3X 1 + 2X 2 + X 3 ). Generally, l n = - L . = (n - (i - l)) x i n n t (l i - I) = L . = l 1 - -- X i . n t We say that L:n eN Xn is (C, 1) SUMMABLE to x, L X n = X (C, 1) , neN i f (J'n x, i .e . , 1 n n ( 1) 1 � � L s i = li�L 1 - � X i = S . ; =1 ;= 1 It is easy to verify that the (C, 1) sums of � + L: n e N an cos nt+ L: n e N b n sin nt are (starting at n = 0) n (t) = a; + � ( 1 n : 1 ) (a k cos kt + bk sin kt) . We verify next that (C, 1 ) summability satisfies ( P t} - ( P 3 ) . Linearity is clear . As to decapitation, suppose that L: n eN Xn = X (C, 1), and consider L:n � 2 Xn · Some computation with the series yields terms: X l , X 2 , · · · , Xn, · · . partial sums Sn: X l , X l + X 2 , · . . , x l + x 2 + . . . + Xn, . . . Some computation with the decapitated series yields terms: X 2 , . . . , Xn, · · · partial sums U 1 = X 2 , U 2 = X 2 + X 3 , . . · , Un X 2 + X 3 + . . . + Xn + 1 , · · · sums of sums tn = ""�= U i = X 2 +(X 2 +X 3 )+ " ' +(X 2 +X 3 + " , +xn + 1 ) ' .L..,.. t l = (S l - x I ) + (S 2 - X l ) + . . . + (Sn + 1 - xt}. = S l + S 2 + . . . + Sn + 1 - nX 1. tn S + S 2 + . . . + Sn + 1 - X l averages Tn = - = l n n For a numerical sequence SUM is the mean

=

-+

.

(J'

0

_

-

=

4. Fourier Series

n+I = -- (jn - xl n

�

X - Xl as n

�

229

00.

Rather than prove regularity now, we will prove it for a more general method (Toeplitz) below that includes (e, 1) summability as a special case. First , consider two examples. Example 4.14.2

Show that the series

L: ( - It + l = I - I + I - I + . . . = �

nEN

Sol ution . terms: 1 , - 1 , 1 , 1 . . . partial sums S n : 1 , 0, 1 , 0, 1 , 0, . . . sums of sums L:�=1 S i : 1 , 1 , 2 , 2 , 3 , 3, . . . = ; E� l Si : 1 , t , � , � , i , � , averages

(jn Thus the (jn

(e, I ) .

···

with even subscripts 2n are all n/2n; the odd ones (the D I )st terms) are n/ (2n - 1). Therefore , (2n � t.

(jn

Not every series is (e, 1) summable, as shown by the following example .

Example 4.14.3

Show that the series 1 -2+3-4+ ··· =

L: (- It + l n

nEN

is not (e, 1 ) summable. Sol ution . terms: 1 , -2, 3 , -4 . . . partial sums Sn : 1 , - 1 , 2, -2, 3 , -3, . . . sums of sums 1 , 0, 2, 0, 3, 0, . . . averages : 1 , O , � , 0, i , 0, * O, � , . . . ' Thus the with even subscripts are all 0; the odd ones (the (2n - I )st terms) are n/ (2n - 1 ) � t . The sequence ( ) therefore does not converge.

(jn (jn

(jn

D

Cesaro summability is concerned with the convergence of a certain se quence of averages. In the following situation, we consider "weighted" av erages. Definition 4. 14.4 TOEPLITZ SUMMABILITY

A TOEPLITZ MATRIX is an infinite matrix

A = ( an k ) that satisfies:

(Td liffin = ° for every k E N (terms of any column � 0) . (T2 ) There exists such that E kEN l l :::; for every n E

an k

M

formly bounded summa ble rows) .

an k M

N ( uni

230

4. Fourier Series

E keN an k = 1 (row sums 1 ) . Consider a sequence (cn ) with partial sums Sn = E ?= l Ci . Now sum along -+

( Ta) liIlln

the nth row of

A

to get the nth Toeplitz partial sum

(Tn = L an k S k , keN

where we assume that the series on the right converges for every Cn is T-SUMMABLE to s, and we write we say that

(Tn -+

S,

E n eN

L n eN Since A

�

Cn

= S (T) . 0 1 0

U

n.

If

0

0 0 1

...

.

)

(Tn = Sn ,

is obviously a Toeplitz matrix with Toeplitz partial sums ordinary summability is a special case of T-summability. ( G, 1) summability is a special case of T-summability, too. To see this, consider the matrix

A

with entries

_

-

(

) . �� .

1 0 1/2 1/2

0 0

... ..

0 0

�. : : :

.

l/n, k � n , k > n. 0,

The nth Toeplitz partial sum is

=

which is the (G, 1) partial sum. We show next that Toeplitz summability is regular. It then follows as a special case that (G, 1) summability is regular.

=

4.14.5 RE GULARITY O F T OEPLITZ SUMMABILITY x (T) .

then E n e N X n

Sk

Proof. Let denote the kth partial sum of Toeplitz sum

If

En eN Xn

x,

En eN Xn, and consider the

4. Fourier Series

L kEN an k

23 1

S k -> x,

for We first show that i s convergent for each n . Since > 0 there exists m E N such that + Hence , for any p � m , and , n E

€

N,

I Sm l ::; I x i €.

( )

l anp l l an ml L kEN an k S k (Tn x. € k = S k - x -> S k € k x (Tn (Tn = X L an k L an k € k· kE N kEN By ( T3 ) , x L ke N an k -> x as n -> 00 . To prove the theorem it therefore suffices to show that dn = L an k € k -> O . kEN Given € > 0 , choose m E N such that h i < €/2M for k > Since limn an k = 0, we may choose q E N such that and � q . l an k l < 2 ( L�= l h I ) for k = 1, 2, By (T 2 ) again ,

By T 2 , we may assume m sufficiently large that +...+ < < The partial sums of therefore form a Cauchy sequence. I t remains t o show that -> Note that to 0 b y hypothesis ; replace b y + i n get +

m.

(

. . .,m

n

L kEN l an k l ::; M, it follows that I dn l < € for n � q . 0 Corollary 4.14.6 REGULARITY OF (C, I) SUMMABILITY If L nEN Xn = x, then Ln EN Xn = x (C, I). Since

Definition 4 . 1 4 . 7 ABEL SU MMABILITY

(rn)

Let be a sequence of positive numbers such that entries of the Toeplitz matrix we take

rn -> 1 - . As the

an k = (1 - rn)r� , n, k � 0 Clearly, limn an k = 0 for every k, so (T I ) is satisfied . As for ( T 2 ) and (T3 ) ,

for every n ,

232

4. Fou rier Series

L n>o Xn

Sk .

Now consider a series with partial sums The Toeplitz partial sums are called ABEL -sums in this case, after the Norwegian mathe matician Niels Abel (as in "Abelian" ) . They are

(Tn

Ln>o Xn x

= (A) or is SUMMABLE IN THE SENSE O F ABEL if We say that A the ABEL PARTI L SUMS

(Tn = L X k r� k �O

�

x.

For an alternative characterization of Abel summability, see 4. 14.8. 0 Since it is a special kind of Toeplitz summability, Abel summability is regular; the fact that summability implies Abel summability is a well-known theorem of analysis known as ABEL'S CONTINU ITY ( LIMIT ) THEOREM . converges for every n , and If = then since conv�rges for all E (0, 1) by standard --+ I , it follows that properties of power series. We then have the following result .

rn

Ln>o Xn x (A),

Lk>O S k rk

L k>O s k r�

r

Ln>o x n = x (A) if and only if Ln >o xnrn < n L n�o xnr x jor all 0 < r < 1 . Example 4.14.9 Show that 1 - 1 + 1 - . . . = � (A) . Solution . Note that the partial sums Sn of the series are 1 , 0, 1 , 0, . . . Let (rn )n �O be a sequence of positive numbers such that rn 1 - . Then '" S k rnk Vn � k>O'" rn2 k - '" (rn2 ) k �k>O � k>O 1- � 1 0 · 1 + r� 2 4. 14.8 ABEL SUMMABILITY = 00, and limr-+ l -

-

--+

�

--

-

(C, I)

We saw in Examples 4.14.2, and 4. 14.9 that the and A-sums of are each 1/2. These are special cases of 4.14. 1 0 , which says that the 1) sum must be the same as the A-sum. The converse is false : the series 1 - 2+ 3 4 + · . . = � and this series i s not 1 ) summable by Example 4. 14.3.

LnEN (-It + 1 (C, 4.14.10

x (A) .

-

(A),

(C,

(C, 1 ) IMPLIES ABEL If L n � o Xn = x (C, I), then Ln�o Xn =

4. Fourier Series

=

233

P roof. Let S n L � = o S k , and let O"n = L � = o sk i (n + 1) denote the nth Cesaro sum of Ln> o x n . First we show that Ln>o x n r n converges for any r E (0, 1). Since O"n x, there must be some > 0 such that 100n i � for (n + 1) r n all n E N. The radius of convergence of the power series Ln>o (n + 1) . . IS gIven b y I'Imn (n + 2) = 1 . Th erefiore ,

---+

I MM

M

1

M M

(4.56 ) L: ( n + 1) O"n rn , 0 < r < 1 , n�O i s convergent , since i t i s dominated b y the series Ln > (n + 1 ) M r n . M ul tiply the series in equation (4.56) by ( 1 - r ) , and set -O"_ l = O. Then n � ( n + 1) O"n rn ( 1 - r) � �n>O �n�O (n + 1) O"n r - Ln> o (n + 1) O"n rn + 1 = � [ (n + 1) O"n - nO"n - d r n �n>O L: n o [ L� = O S k - L �:� S k r n - s n rn . L:n�O If w e multiply b y ( 1 - r ) again , we get n n ( 1 - r )2 � ( 1 - r) � �n� O (n + 1) O"n r �n>O sn r n+I n � �n>O S n r �n>O Sn r -- � (4.57) � - (s n - sn_ I } ;n �n>O - x n rn , L:n�O so L n>o x n r n converges. Now we have to show that it converges to the (C, 1) llmit x limn n ' As we ask the reader to prove in Exercise 4 , ( 1 - r)2 L: ( n + 1) rn 1, (4.58) n�O 0

]

:

=

0"

=

so Therefore, by equation (4.57) ,

( 1 - r )2 L: ( n + I ) ( O"n - x ) rn = L: x n rn - X .

n�O

n�O Given > 0 there exists an integer p such that 100n - xl < £/2 for n Now split Ln�o ( n + 1) ( O"n - x ) rn into two parts : p- l n � (1 - r) 2 � � n�O x n rn - x �n _- O (n + I ) ( O"n - x ) r + n + ( 1 - r)2 � �n� p ( n + l) ( O"n - x ) r . £

�

p.

4. Fourier Series

234

We can make the first term less than c/2 by taking r sufficiently close to 1 . The second is small because I Un - x l < c/2 for n �

p:

< <

(l _ r) 2 " n>p (n + l) -C rn 2 L.J ) rn 1 ( 1 _ r) 2 � " + (n o L.Jn> 2 c "2 (equation (4.58) . 0

It follows from 4.14.6 and 4.14.10 that summability implies Abel summa bility, which is another way to prove the Abel continuity theorem mentioned above. As mentioned before 4. 14. 10, the converse of 4.14.10 is false . The converses of the regularity statements are obviously false as well. Sometimes it is possible to get a partial converse, something like "Abel summability + another condition" implies summability. Theorems of this type are known as TAUBERIAN theorems after A. Tauber, who proved such a result relating Abel summability to ordinary summability.

Exercises 4 . 1 4 1 . With notation for Un as in Example 4.14. 1 , verify that the nth ( C, I ) sum of Ln e N X n is n n n S i L (1 - i -n 1 ) X i = .1n ;L (n + 1 - i) X i . Un .1n ;L ;=1 =1 =1

=

=

2. Show that if 1 - 2 + 3 - 4 + . . . 3. Show that 1 - 2 + 3 - 4 + . .

.

= S (P), then S = t .

= t (A).

4. For 0 < r < 1 , show that (1 - r) 2 L k � O (k + 1) r k

= 1.

Hints 3. Show that

1 - 2r + 3r 2 - 4r3 + . . . = � ( 1 + r)

(ordinary con vergence).

4. By standard results about power series, Ln >o (n + 1) rn converges for I rl < 1 liffin n� l Now multiply the series by r and subtract .

=

.

4. Fourier Series

4.15

235

Fejer Theory

We discussed in Section 4.9 that there are functions f E L1 [-1I", 1I"] whose Fourier series diverge everywhere. Even so (4. 15.3) , the Fourier series of any f E L1 [-1I" , 11" is (G, summable to [f at any point +f exist . The Hungarian mathematician Leopold and f where f Fejer ( 1880-1959) made this remarkable discovery in 1904, and the body of related results is known as Fejir theory. (We are told that Fejer is pro nounced "fay-ha.") Lebesgue (4. 18.5) proved that the Fourier series of any f E L1 [-1I", 11"] is ( G, 1) summable to f almost everywhere. To get Fejer 's theorem, we proceed in a manner analogous to that used to get Dirichlet 's theorem on pointwise convergence 4.6.2.

t

(t - )

(t - )

1) (t + )

]

(t + )] j2

(t)

1 . We use trigonometry to find a simple expression for the average of the first n + 1 Dirichlet kernels (equation (4.61) and Theorem 4. 1 5 . 1 ) . The average of the first n + KERNEL, denoted by

Fn (t) .

1 Dirichlet kernels i s called the FEJ ER 1)

2 . Then we use information about the Fejer kernel to evaluate. the (G, partial sums o f equation (4.60) i n 4.15.3. I n particular, the Fejer kernel has a selector property identical to that of the Dirichlet kernel (cf. 4.6 . 1 , and 4.5.6) . For any 211'-periodically extended f E L1 [-1I', 11'] for which f exists,

O'n

(t + )

lim -.!.. 211'

n

r f (t + x) Fn (x)

10

dx = f (2t+ ) .

Do (t) = 1 and sin (n + l/2) t , n E N , (4.59) L.J 2 cos k t = Dn (t) = 1 + � sin (t/2) k= l where the latter fraction is defined to be 2n + 1 when t is an integer multiple Recall (Section 4.5) the Dirichlet kernels:

of 211". By 4.5 . 1(b) we can express the kth partial sum of the Fourier series (of the 211'- periodic extension) of f E L1 [-1I", 11"] as

S k (t ) = 211"1 l_1r f (t + x) Dk (x) dx, k E NU {O} . 1r The ( G, 1) partial sum, the average n of the first n + 1 (starting at n = 0) partial sums Sn , is therefore 1 l 1r 1 En Dk (x) dx. ( 4 .60) f (t + x) O'n (t) = 2 n + 1 k=O The bracketed term, the average of the first n+ 1 Dirichlet kernels, is known as the nth FEJER KERNEL Fn (t). For n � 0 we have the following options (note that Fo (t) = 1) for expressing Fn (t): 0'

11'

- 1r

[

--

1

236

4. Fourier Series

1 ", n Dk (t) n +1 1 � nk = o sin (k + l /2) t ( 4 . 6 1) ", = n + 1 � k O sin (t/2) where si:f�t;') t is defined to be 2k + 1 when t is an integer multiple of 21r. Since Dk (21rm) = 2k + 1 for any E Z and any k � 0, it is easy to Fn (t)

m

verify that

Fn (2 1r) = n + 1 for any n E N and any E Z . Using the Fejer kernel, w e can rewrite equation ( 4 60 ) as (4.62) un (t) = 21r1 Lr1f f (t + x) Fn (x) dx. Since the Dirichlet kernels D k are even, so is every Fn. Consequently, we can split I::,.. into I� ,.. + 10"' , replace x by - x in I� ,.. , and rewrite equation 4.62 as (4.63) un(t) = 21r1 10 '" [! (t + x) + f (t - x)] Fn (x) dx. m

m

.

As with the Dirichlet kernel of equation (4.59) , a little trigonometry enables . . · · m us to CIrcumvent t h e addlhon

l /2)t . " L.,.. nk = O sinsin( k (+t /2)

4.15.1 ANO THER EXP RESSION FOR THE FEJER KERN EL and 21rZ, sin 2 l ) sin 2 (t/2)

t r;.

n

For E Nu {O}

Fn (t) = (n +[(n + 1) (t/2)] .

P roof. Since

and sin 2 0 to to get

n

2

2 sin or

2 sin (t/2) sin(k

+ 1/2)t = cos kt - cos (k + 1) t

= 1 - cos 20, we can calculate the telescoping sum from k = 0

(t/2) t sin(k + 1/2)t = 1 - cos (n + 1) t = 2 sin 2 (n � 1) t , k=O · 2 (n + 1) t n sm 2 L sin(k + 1/2)t = . t k=O sm 2

4. Fourier Series

Therefore , using equation

237

(4 .6 1 ) ,

n 1 ( n + 1) 1sm· (t /2) 2:sink 2=O(nsin+ (kl ) +t /21/2 )t ( n + l ) sin (t /2) sin (t/2) . 0

Fn (t)

------�--�

.

The Fejer kernels look a lot like the Dirichlet kernels (which can be negative) . They are highly concentrated around 0, rising to (n + 1) on a shrinking base of width 1r/ (n + Their effect as a kernel in an integral is to sieve out the value of the function at = 0 as n -+ 00 , something that we formalize in Fejer's theorem on pointwise convergence.

4

-4

1) . (4.15.3)

t

-2

2

The Fej er kernel for

4.15. )

n

=

t

3 and

4

n

=

7

Fn (t)

We show in 2 (c that the narrowing width of is sufficiently offset by the increasing height to keep the area under the curve constant . 4.15.2

PROPERTIES O F THE FEn �R KERNEL

Fn (t) = {

and

( (n + l ) /2 ) 2 , (n + l) (t/2) 1

(n + 1) ,

sin

sin

t

The Fejir kernels

Fo (t) = 1

, n � 1,

have the following properties: For every n E N , ( a) is an even 21r-periodic function; (b) � 0 for all t, and -+ 0 uniformly outside [ -r, r) for all o < :S 1r ; 1r (c) =2 (t) 21r. r

FFn (t) Fn (t) n J:1r Fn (t) dt 1 Fn dt =

238

4. Fourier Series

Proof. The result of (a) is obvious, as is the nonnegativity assertion of (b ) . As to the uniform convergence outside any interval about the origin , let r be positive. For any such that 0 < r � � 11",

t

as

n

-+

1 (

Fn ( t ) = (n + 1)

It I

sin

\n + 1) t/ 2

sm (t/2)

)2 - (n +1 1) (_sm._1_r/2 )2 -+0 <

00 .

(c)

i: Fn (t) dt = i: (1 + n ! 1 t (n + 1 - k) cOS kt) dt = 211". 0

So much for the preliminaries. We can now prove Fejer 's analogue of the pointwise convergence theorem.

f E L1[-1I", 11"] f (t - ) and f (t + ) exist, the Fourier series for f (t) is (C, 1) summa ble to [f (t - ) + f (t + )] /2. (b) UNIFO RMLY (C, l) SUMMABLE FOR CONTINUOUS f If f is contin uous on [c, d], then the (C, partial sums O"n of equation (4. 60) converge uniformly to f on [c, d] . Proof. (a) Suppose that f :/; and let t be a point at which f (t - ) and + f (t ) exist . By equation (4.63), O"n(t) = 211"1 [f (t + x) + f (t x)] Fn (x) dx. 4.15.3 FEJER ' S THEOREM For 211"-periodically extended (a) SUMMABILITY A t every point t where

(C, l)

1)

l1r

0,

:

-

0

To prove the theorem, we show that the Fejer kernel has the following selector properties: lim �

n

and

r [f (t - x) Fn (x)]

211" Jo

dx = f (r) 2

� l 1r [J (t + x) Fn (x)] dx = f (� ) . +

� 2

lm

.1 . (

Since the arguments are quite similar , we show only the latter . By 4 5 2 c) ,

f (t + ) = f (t + ) l 1r rn ( X ) dx, 2

211"

0

D

1 11r0 [f (t + x) - f (t+ ) ] Fn (x) dx --+ O.

so it suffices to show that

211"

4.

Fourier Series

239

(t, c) > 0 such that 0 < x < r I f (t + x) - f (t + ) I < c. " Split � f; [f (t + x) - f (t + )] Fn (X) dX into � f; + � fr . By 4 1 5 .2 ( c ) 2 2 2 and the choice of r, r < 2� l Fn (x) dx (4.64) < 211"1 c1l" = 2c ' while there exists N = N (r) = N (t, c) such that I Fn (x) 1 � 1I"c/2 11 f 11 1 for � N and x > r ; hence c 1 + 2· (4.65) dx (t x) (t Fn (X) f ) + [! � 1 � ] I 1 1 211"1 r "1 211" 1r " (b) Since f is continuous on [c, d] and closed, bounded intervals are com pact , f is uniformly continuous on [c, d]. Therefore , the r chosen in the proof of (a) is independent of the point t, and so is the N. 0 c

To that end, let b e positive, and choose r = r implies

.

n

If a series converges, the sum, and (e, 1) sum must agree by 4. 14.6 . This yields the following pointwise convergence theorem.

For periodically extended f ) and f (t + ) exist, and the is convergent, the Fourier series converges to

Corollary 4.15.4 POINTWISE CONVERGENCE E at every point t where ( t

f LH- 1I", 1I"],

Fourier series for f (t)

-

Another immediate consequence of Fejer 's theorem 4. 15.3(b) is the following: 4.15.5 UNIFORM CON VERGENCE If the 211"-periodic function f is contin uous on R, then f is uniformly approximated on any closed interval by the arithmetic means of the partial sums of its associated Fourier serzes.

In other words , any continuous periodic function approximated by trigonometric polynomials

ak ,

bk

f can b e

uniformly

f,

where and are the Fourier coefficients of a classical result of Weier strass. Another way of saying this is that the trigonometric polynomials are As dense in the space of continuous periodic functions with respect to another intriguing application of Fejer's theorem, we deduce Weierstrass's

11 11 00 .

240

4. Fourier Series

classical result on the uniform approximation of continuous functions by ordinary polynomials in Section 4.17. Since (G, 1 ) summability implies Abel- or A-summability, we have the following result: 4.15.6 A-SUMMABILITY O F FO URIER SERIES

For 27r-periodically extended

I E Ll [-7r, 7r], at every point x where I (t - ) and I (t + ) exist, the Fourier series lor f is A-summable to [I (t - ) + I (t + )] 12. In more detail this means that

ao

2

n-l . n I (t - ) + I (t + ) L.J (a n cos nt + bn sm nt) r -+ + """" 2

k=l as r -+ l- , O < r < l .

f (t - )

I (t + ) [,]

With the frequent hypothesis that exist , i t is pertinent and to recall that monotone functions have one-sided limits everywhere. Since a function of bounded variation on the closed interval a b can be written as the difference of monotone functions, it too, has right- , and left-hand limits everywhere in a b]. By Fejer ' s theorem, therefore , the Fourier series of any function of bounded variation is (G, I ) summable to everywhere. Moreover, the Fourier coefficients a n and bn of a function of bounded variation (Exercise 4.3-8) on 7r] are such that there must be a constant c such that I a n I :::; and I bn I :::; for every Hence , there is some constant [{ such that I a n cos bn sin for every This is useful because of the Hardy and Landau criterion for (G, 1) summability to imply summability. We outline its proof in the hint to Exercise

[,

[/ (t - ) + / (t + )] /2

cln

constant K, I cn l

:::;

[-7r, cln n. nt + nt l :::; KIn

4.15.7 HARDy-LANDAU THEOREM

n. l.

1f2:/:=0 cn = c (G, 1 ) an d iffor some

KIn for all n E N , then 2::=0 Cn = c.

2::= 0 Cn is A-summable and Icn l :::; KIn for all K , then 2::=0 Cn converges to the Abel sum.

There is a much stronger result due to Hardy and Littlewood, namely , if for some constant

nEN A consequence of 4 . 1 5 .7 is that if I E BV [ - 7r, 7r], then the Fourier series of I converges to [I (t - ) + I (t + )] 1 2 everywhere. This provides us with an

alternative treatment of Jordan 's theorem 4.6.6.

The Fourier series of the 27r-periodic extension of I E converges to [/ (r) + / (t + )] /2 everywhere. [f in addition f is continuous, then the Fourier series of f converges uniformly to f. 4.15.8 JORDAN I I

BV[-7r, 7r] c LH-7r, 7r]

4. Fourier Series

241

Exercises 4 . 1 5

1.

HARDy-LANDAU THEOREM If = (e, ::; Min for every n E constant M such that (in the usual sense) .

:L�=o Ck C

I cn l

Un n E N.

1) and there is some N , then :L�=o Ck = C

(4.63).

be as in equation 2. BOUNDS Let Show that if there is some (t) 1 ::; M for all constant M such that t ::; M for all t, then t and any

3.

I Un

SERIES O F SINES AND COSINES Show that for 0 ::; r < I , '2

1+

and

4.

IJ ( ) 1

"

L-

n eN L"

neN

n

r cos n () -

_

rn sin n ()

1 ( 1 -2rl_r2 cos 9 + r 2 )

'2

= 1 -2rr cossin 99 +r2 .

THE POISSON KERNEL The quantity

1 -2r cos(t-x)+ r 2

4 . 15.6, + n / ( ) +/ ( - ) u (t , r) = � + :L (an cos nt + bn sin nt) r - t 2 t n eN as r - 1 - . Show that u ( t , r ) = 2� r:7r J (x) ( 1 2 r c�;2 t )+r 2 ) dx . is known as the POISSON K ERNEL. Under the circumstances of

Hints

+ S ,,+ + ··+ S ,, ± k n ( Un + k -2k Un ) + k±.!. 1 . k Un + k - ""fUn S ,, + 1

""f

( lkn) dn ,k " Un - C, k dn, k " = 8n + f-n i:L=l" (kn - i + 1) Cn + i , so k" I dn, k" - sn l ::; iL:=1 I Cn +i l ::; !+l M . Finally, let k n = [w] , the greatest integer in Then I dn, k" - Sn I ::; EM. Since dn, k " - C , it follows that Sn - c . (kn)

Let be any sequence of integers such that the sequence n is bounded. Since - C, but it follows that

w.

242

4. Fourier Series

En eN z n , z = r ( cos () + i sin ()) . Substitute the integral expressions for an and bn , and use the result

3. Consider the series 1/2 + 4.

of Exercise 3.

4.16

The Smo othing Effect of (C, 1) Summation

Recall how the partial sums sin (2k - 1) t 8n ( t ) -_ ±7r � � k = l 2k - 1 of the Fourier series for the square wave (Example 4 . 1 .2) converge. Consider for example (t) = (4/7r) (sin t + sin 3t/3 + sin 5t/5),

83

4

-4

The Gibbs Phenonemon

(C, 1) sums Un = 8 0 + 81 + . . . + 8n

with the Gibbs overshoot . The

n+l

approach more smoothly. To illustrate the point, consider

4. Fourier Series

(13 t =

()

243

.!.1r ( 3 sin t + 2 sin 3t l 3 + sin 5t 1 5 )

-4

4

The Fejer sum a3 and the square wave

or

(15

(t) = (2) 31r

(

5 sin t +

-4

i sin 3t + O.6 sin 5t + � sin 7t + .!. sin 9t 3

-2

9

7

2

)

.

4

The Fejer sum as and the square wave The suppression of the Gibbs overshoot is not a coincidence. Suppose

I E L'i [-1r, 1r] is bounded , and consider the expression ( equation ( 4 . 62 ) of Section 4.15 ) I (In (t) 1 :S 2� 1: I I (t + x) Fn (x) 1 dx :s "�I�oo 1: Fn (x) dx = 11 / 11 00 ,

since f�1r

Fn ( ) dx = 21r by 4.15.2 ( c ) . x

244

4. Fourier Series

4.17

Weierstrass 's Approximation Theorem

Weierstrass proved that any continuous function on a closed interval can be uniformly approximated by a polynomial-in other words, that the poly nomials are dense in = [ , b] . We deduce this from Fejer's b] theorem 4 . 1 5 .3 in this section. Though it takes some work , Fejer 's theorem can be deduced from the Weierstrass theorem as well. It suffices to consider the interval [- 1 , 1] . For any I E C[- I , I] , note that 9 (t) = I (cos t) E (R) is an even 27r-periodic function. Suppose the Fourier series for is cos nt . 2" + " L..J nEN

(C [a, , 11 11 00 ) C a C

9

ao

an

B y 4.15.5, the nth (C, 1 ) partial sums Un (t)

= ao2 + t ( 1 - n +k _1 ) a k cos kt _

k=l

uniformly approximate I (cos t) for all t . Thus, replacing any > 0, and sufficiently large n,

f

I (t)

- a20 - t ( k= l

1 - __ n+1

k

) ak

t by arccos t , for

cos (k arccos t)

<

f.

for - 1 ::; t ::; 1 . It remains for us to prove that cos (k arccos t) = Pk ( t) is actually a polynomial . 4.17.1 CHEBYSHEV POLYNOMIALS AND COSINES For any integer n 2: 0 there exists a polynomial Pn , called the nth CHEBYSHEV POLYNO MIAL, such that Pn (cos s) cos ns lor all s E R.

=

=

=

Proof. If n = 0 , and 1, take Po (t) 1, and PI (t) t , respectively. Suppose the statement holds for all n < m where m 2: 2. Then, for any s E R ,

cos ms

cos ms + cos ( m - 2) s - cos (m - 2) s cos [(m - 1 ) s + s] + cos [(m - 1 ) s - s] - cos ( m - 2) s 2 cos s cos (m - 1 ) s cos (m - 2) s . -

Since cos (m - 1 ) s and cos ( m - 2 ) s may b e expressed as polynomials in cos s, the result follows, and we see that the polynomials must satisfy the following recursion relation

Pm (t) = 2t pm - 1 ( t) - Pm- 2 ( t) . 0

Using the recursion relation, the next few Chebyshev polynomials are t 2 - 1 , P3 (t) 4t 3 - 3t , P4 (t) = 8t4 - 8t2 + 1 . They are tabulated

P2 (t)

=2

=

4. Fourier Series

245

in several mathematical handbooks and software. They form an orthogonal system on [- 1, 1] "with respect to the weight function (1 - t 2 ) -1 / 2 , " i. e ., [11 Pn (t) Pm(t) (1 - t 2) - 1 / 2 dt = 0 for =P are often such employed forandfunctions as et .in algorithms used by calculators to compute values n

4. 18

m,

Leb esgue 's Pointwise Convergence Theorem

Fejer's Theorem 4.15.3 (a) states that for 211"-periodically extended I E LH-1I", 1I"], at every point t where I (t - ), and I (t + ) exist, the Fourier series forthe IFourier (t) is (e, l) summable to [! (t - ) + / (t + )] /2. Lebesgue showed that series for any I E Ll[-1r, 1I"] converges (e, 1) to 1 (t) a. e . The

method of proof is quite similar to that of Fejer's theorem. By equation 63) ofbySection 4.15 the nth (e, 1) partial sum of the Fourier series for 1 is(4.given 11r l 1r [/ (t + x) + / (t - x)] Fn (x) dx O"n(t) 21 1r 11r l 1r l (t - x) Fn (x) dx. (x) dx l (t + x) Fn + 2 2 11" l We want to show that each integral converges to 1 (t) /2 for almost every + t. Instead of subtracting 1 (t ) /2 from the integral with 1 (t + x) in the integrand (as we did in the proof of Fejer' s theorem), subtract 1 (t) /2 to get 11r l 1r [! (t + x) Fn (x)] dx - / (t) /2 2 1 1 1r = 21r [/ (t + x) - / (t)] Fn (x) dx . Thus, we want to show that o1r [/ (t + x) - / (t)] Fn (x) d x 0 as I 00 . After choosing an r E (0, 1r) to satisfy a certain condition, we split Io1r [I (t + x) - 1 (t)] Fn (x) dx into 1r ) [I (t + x) - 1 (t)] Fn (x) dx . + 1 (lr Since I: [I (t + x) - I (t)] dx is bounded (because 1 E Ll ([-1r, 1r))), and Fn (t) 0 uniformly for t > r by 4.15. 2 (b), f,.1r [I (t + x) 1 (t)] Fn (x) dx is;small for forsufficiently large So, argument the argument dependsthaton showing that is small large (A similar will show I -1 l 1r [! (t - x) Fn (x ) ] dx -+ / (t) /2 0

0

a

a

a

-T

->

n.

n.

21r

a

-

n

->

246

4. Fourier Series

as n ---+ 00 . ) For the sake of demonstrating that is small, we need to discuss the idea of Lebesgue point, a concept related to differentiability of the indefinite integral J: .

J;

Definition 4 . 1 8 . 1 LEBESGUE POINTS

x

x+h

If and belong to the closed interval b] if LEBESGUE P O INT of E

1

lim h ..... O h

I Ll [a,

[a, b] , then x is called a

1 x + h I/ (t) - / (x) l dt = lim -1 1 h I / (x + t) - / (x) l dt = O. h ..... O h 0

x

o

The connection between Lebesgue points and differentiability is con tained in the following theorem.

( Xo (a, X o , F' (xo) = I (xo).

4.18.2 DIFFERENTIABILITY OF THE INTEGRAL Nat anson I 1961 , Theo rem 4, p . 255) If I E b] , and E b) is a Lebesgue point of f, then and is differentiable at

LHa,

F (x) = J: I(t) dt

Points of continuity are Lebesgue points. 4.18.3 II point.

I E Ll [a, b] is continu ous at x E (a, b), f
Proof. Given > 0 choose r > 0 such that Then, for 0 r,

then

x

is a Le besgu e

I I (x + t) - I (x) 1 < f for I t I

h 1 I / (x + t) - / (x) l dt < f h = f . h 1

"h

°

<

r.

0

x

On the other hand, no point of jump discontinuity is a Lebesgue point, since > 0 implies that

I/ (x + ) - l (x) 1 = J

Jh dt h = J for small h > O. o 4.18.4 (N at anson 1961 I , Theorem 5 , p . 255) If I is int egrable on the closed interval [a, b] then almost every point of [a, b] is a Lebesgue point of I· 1 {h I I (x + t) - I (x) 1

hi

�

Now we are ready to prove Lebesgue's theorem on (C, 1) summability. 4.18.5 LEBESGUE 'S POINTWISE" CO NVERGENCE THEOREM Th e Fou rier series of the 21r-periodic extension of any E L'i [- 1r, 1r] is ( C, 1) summa ble to I alm ost everywh ere.

(t)

I

4. Fourier Series

t be a Lebesgue point of f, so that 1 1 h I f (t + x) - f (t) 1 dx = o. lim h h ..... h > 0 , choose T E ( 0 , 11") such that for all 0 < � * l h I f (t + x) - f (t) 1 dx < f .

247

Proof. Let

O

Given

I:

(4.66)

a

T,

(4.67)

fr1l" [f (t + ) - f (t)] dx is bounded (because f E Ll ([- 1I", 1I"])) and Fn (t) ---4 0 uniformly for t > T by 4.15.2(b), fr'" (f (t + x) - f (t)] Fn (x) dx is small for sufficiently large n. We can therefore choose an integer N > l/T - 1 such that J,.'" (f (t + x) - f (t)] Fn (x) dx < for n 2: N. In the remainder of the argument we show that f; (f (t + x) - f (t)] Fn (x) dx < 511"2 1:/4. To that end, suppose that n 2: N , and split r 1 1/(n + l ) + lr . [f (t + x) - f (t)] Fn (x) dx into l /(n + l ) To see that f 1/(n + l ) is small, note that sin x < x for x > 0 , and sin x > 2x / 11" for 0 < ox < 11"/2 (consider the straight line connecting the origin to (11"/2, 1)), so that 1 < -11" for 0 < x < 11"/2. (4.68) -.smx 2x Thus, for 0 < x < 1/ (n + 1), 2 1I"2 (n + 1) sin 2 ( n + 1) x/2 [(n + 1) x/2] 2 (�) Fn (x) = (n + 1) sin2 x/2 -< (n + 1) x = 4 . Hence , since 1/ ( + 1) < [O l/(n + l) < 11"2 ( n4 + 1) [ l /( n + l ) I f (t + x) - f (t) 1 dx In Jo l /(n + l ) 2 11" 1 [ I f (t + x) - f (t) l dx 42 1/ (n + 1) Jo 11" f . < 4 Since

x

I:

l

a

a

n

f;/(n + l )'

T,

xl 1 x , then considering inequality (4.68), 1/ (n + 1) x sin2 (n + 1) x/2 � 1 ( 11" ) 2 (n + l ) -; . ( n + ) sin 2 x/2

since Isin � for all As for we see that for � � T, l

248

4. Fourier Series

Thus

2 r I f (t + x) - f (t) l dx. 11 /(n + l ) ::; (n + 1) 11 /(n + l ) x2 Let J; l f (t + x) - f (t) 1 dx, and integrate b y parts with dv I f (t + x) - f (t) 1 dx to get g (t, x) . � r n+1 x 2 l /(n + l ) + 2 11 /(n + l ) x 3 By equation (4.67 ) , g (t, x) < for 0 < x ::; r, so I J;/( n + l ) 1 is less than 11"2 -x 2 dx + 1 x l /(n+l) + 2 l /(n + l ) -

g (t , h)

r

=

( g (t , X) l r f.X lr f. 1

[ f. l r

n

11"

dX )

f

If all the Fourier coefficients of a function f are 0, must = O? Or could there be some strange nonzero whose every Fourier coefficient is O? If all the Fourier coefficients of [-11", 11"] are 0 , then for all E NU {O} ,

f

an, bn f E Ll

n

(Tn (t)

=

(Tn - f

�o + � ( 1 - n : 1 ) (ak cos kt +

h

sin

kt) = O . f

Since a.e. b y Lebesgue's theorem 4 . 1 8 .5 , it follows that = O. A related question in the converse direction is, Can an arbitrary function be represented by more than one trigonometric series? Riemann had consid ered the problem, but did not solve it. Cantor proved that the coefficients were indeed unique. Then he and others considered further refinements . If instead of a Fourier series vanishing everywhere, suppose

�o + L (an cos nt + bn sin nt) = 0 for all t E [-11", 11"] - D,

D

neN

where is a certain sparsely populated subset of [-11", 11"] . Must all the coefficients be O? Cantor showed that if D is any closed countable set , then the coefficients indeed vanish. The British mathematician W . Young showed that if D is any denumerable set then all the coefficients must be O. Cantor's work motivated him to investigate properties of sets of points generally, to lay the foundations for point-set topology. This is one more

4. Fourier Series

249

instance of the profound influence of Fourier series on the development of mathematics. After these successes, the natural question was, Could the sparse set be any set of measure O? No! Incredibly enough, Menshov showed in 1 9 1 6 that there are nontrivial (i.e . , not all coefficients 0) trigono metric series that converge to 0 a.e . ! For more details on this fascinating material, we must refer you to Bary 1964 or Zygmund 1 959 . Since (e, 1) summability implies Abel summability (4. 14. 10) , we have the following consequence of Lebesgue's theorem.

D

Ll[ -'II" , '11"] is A-summa ble to

Corollary 4.18.6 The Fourier series 01 I E I (t) lor alm ost every real number t; that is, il u ( r, t)

then

u

=

� + L ( an cos nt + bn sin nt) rn , 0 < r < 1 , neN

( r, t) � I (t) as r � 1 lor almost every t.

Without going into the details, we mention that u ( r, t) -+ I (t) as r � 1 even for points that are not Lebesgue points. In the table below I stands for the 2'11"- periodic extension of I to R. If

I E LjJ-'II" , '11"]

is

its Fourier series converges

l ( r ) , / (t+ ) exist

(e , 1 ) to I a.e.

4.18.5

( r ) + / (t+) (e , 1) t l

4 . 1 5 .3

0 2 I (t- ) + / (t+ ) to

BV [-'II" , '11"]

4 . 1 5 .8

2 everywhere

B V [-'II" , '11"] n e (R)

uniformly to

I

4 . 1 5 .8

Table 4. 18-1 Convergence of Fourier series

4 . 19

Higher Dimensions

Assuming familiarity with the basic facts about Lebesgue measure on Rn , n E and the Lebesgue integral of measurable functions defined on sub sets of R in this section we illustrate the theory of Fourier series for complex-valued, square- summable functions of n variables; we discuss con denote the vergence for real-valued functions in Section 4.20 . Let Cartesian product

N, n ,

[-'II" , 'II"] n

[

[ -'11" , '11"] x -'11" , '11"]

,

x · · · x [-'11" , '11"]

n factors v

f

250

4. Fourier Series

f L2

n

f

and let E ( [- 71" , 7I"] ) . If there are numbers a and b such that (x + a , y) = f (x, y) and f (x, + b) = f (x, y) for all x and y, then we say that f is D O UBLY PERIODIC. For example, (x, y) = sin x cos 2y is doubly periodic.

y

f

Remark PERIODIC EXTENSION As we have done for functions of one vari able , we will extend functions of several variables periodically, too: If is defined on [-al , al] x . . . x [ a , a ] , we extend it with pe riod 2a i in each direction (holding the other variables fixed) . If f is defined j ust on [0, ad x . . . x [0 , a n ] , we can extend f to be an even or an odd function in each of its arguments . This leads to multiple Fourier cosine and sine series, a particular case of which is illustrated in equation (4.70) .

f

-n n

L: meN L:n eN X mn

The idea of convergence of a double series of vectors from a normed space is that if both indices are sufficiently large , the partial sums = L:� = l L:� = l are arbitrarily close to some vector x : Given > 0 there exists N such that

f

Sp q

X mn

II Spq - x II

<

f for p, q � N.

"Absolute convergence" and "unconditional convergence" have the obvious analogues for double series; absolute convergence of a double series implies unconditional convergence if the normed space is complete (cf. Section 3.6). o f a convergent series are all nonnegative , This means that i f the terms then the series can be summed by rows or columns :

X mn L: mEN L:nEN X mn

( ) mEN n eN

� � X m n = � � x mn

meN n eN

(

� � x mn

n eN m e N

)

Also, for nonnegative terms, if any one of the series above converges, then so do the other two, to the same sum. Comparison and integral tests (using double integrals) also remain valid. Thus,

m +n � � ( ml 2) n 2 cos mx cos n y m EN n eN -

L:meN L:n e N min 2 •

is convergent by comparison with A double integral fIra , bl x [c , dJ f (x , y) dx dy (we do not distinguish be tween dx dy, and dy dx in the double integral) on a rectangle [a, b] x [c, d] is usually evaluated by means of iterated integrals f: (x, dy) dx . The process is j ustified by a combination of the Fubini and Tonelli theo rems 4.19.1 and 4 .19.2, respectively. We quote versions appropriate for our situation below . One of the many places in which proofs of the theorems can be found is Goldberg 19 76. Fubini's theorem provides the means to

(t f y)

4. Fourier Series

25 1

evaluate double integrals by iteration. The Tonelli theorem is a criterion for integrability in terms of iterated integrals: If is measurable and one of the iterated integrals is absolutely convergent , then is integrable , and the iterated integrals yield the same value , regardless of the order of inte gration. Although we state the theorems for f they are valid on finite etc . , intervals b] and namely, for b] x

[a, I E Ll ([a,

[e, d], [e, d]).

ffra,b]x[c,d]

1=

I

I

fR2 '

f: [ t I (x, y) dy] dx,

E L 1 (R2 ), f y) I I (x,Ll y)(R)1 dy x R, (x, �oo x,

x E R, y;

then for almost all 4 . 1 9 . 1 FUBINI 'S THEOREM Ifl < 00; in other words, is defin ed, i. e. , for almost all fixed E I E as a function of also, is integrable as a function of and

f�oo I (x, y) dy J�oo f (x, y) dy

IIR2 I dx dy = I: [I: I (x, y) dY] dx.

I is measura ble on R2 and one of I: [I: I/ (x, y) 1 dY] dx, I: [I: I / (x, y) 1 dX] dy

4.1 9.2 TONELLI 'S THEOREM If

exists, then: (a) so does the other, (b) and (c) ( by Fu bini 's theorem)

f E L l (R2 ) ,

IIR2 / (X, y) dxdy

I

=

[a, [e, d] [a, [e, d]

I: [I: (x, y) dy] dx I: [I: I (x, y) dX] dy.

L ([a, b] [e, d]) denotes the Hilb ert I (x,2 y) that are square-summable on

Let b] and be closed intervals ; space of all complex-valued functions b] x in the sense that

x

lb l d I I (x, y) 1 2 dydx < 00

with the inner product

(I, g) = lb l d I (x, y) 9 (x, y) dydx.

The following theorem provides a general way to manufacture orthonormal bases for as products of orthonormal basis elements for b] x b] and

L 2 [a,

L 2 ([a, [e, d]) L 2 [c, d].

252

4. Fourier Series

, (In ), (Un ) hmn (x, y) 1m (x) Un (y) ,

4.1 9.3 ORTHONORMAL BASIS FOR L 2 ([a, b] x [ c d]) Let [a, b] and [ c , d] be closed intervals. If and are orthonormal bases for L 2 [a , b] , and L 2 [c, d] , respectively, then =

i s a n orthonormal basis for L 2 ([a, b]

x

[c, d])

.

m,

n

E N,

, hmhmn n (hij , hmn ) ld16 /i (x) uj (y) /m (x) un (y)dxdY 16 Ii (x) 1m (x) dx ld Uj (y) Un (y) dy 8im 8jn (hmn ) m,n w L2 ( [a , 8rs 3.4.2(b). ( w, hm n ) 0 o l d 16 (x, y) 1m (x) Un (y) dydx 16 1m (x) (ld w (x, y) Un (y) dY) dx, (x) t w (x, y) Un (y)(x,dyy) .iUn (y) 1mx.; it x [a, 0 t (x, y) Un (y) dy y. w (x, y) x w (x, y) y. [a, [, L 2 ([-11", 11"] 2 ) ii(n ntm/$) Z} e { Z L 2 [-1I", 11"] , {e t+ s /211" Z } i L 2 ( [-11", 11"] [-11" , 11" ) . L 2 ([-11", 1I"] n ) , x Rn ) (,) (k L2�= l ki X i , {k, X ) k (ki ) znL2, (X[- , 1I"(Xni )) . Rn . { ei / (211"t/ } 11"

Proof. It follows from the Tonelli theorem 4.19.2 that each summable on [a, b] x [c d]. As to the orthonormality of the for any positive integers i, j, m , and n ,

is square ( m , n E N) ,

=

where is the Kronecker delta. To see that E N constitutes an orthonormal basis, we use the criterion of Suppose E b] x [c, d] , and = for all m and n . Then for all m and n , W

so v = is orthogonal to every follows that v = 0 a.e. Consequently, w for almost every Fix E b] . Since = w for all n , it follows that = 0 a.e. as a function of Thus = 0 for almost every and almost every Since the measure of a Cartesian product of measurable sets is the product of the measures, it follows that w = 0 a.e. on b] x c d] . 0 Example 4.1 9.4 BASIS FOR

Let denote the integers . Since basis for it follows that mal basis for X ] 0

:

n

:

E

m, n

E

is an orthonormal s an orthonor

Example 4 . 1 9 . 5 BASIS FOR

Let carry its usual inner product where = = E E orthonormal basis for ] 0

so that Then

=

is an

4. Fourier Series Example 4.19.6 BASIS FOR

253

L2 ( [- 11" , 1I"F )

m, n E N , an orthonormal basis for the real space L2([ - 1I", 1I"F) is

With given by

1 cos mx sin mx cos ny sin ny cos mx cos ny 211" ' -/211" ' -/211" ' V211" ' V211" ' sin mx cos ny cos mx sin ny sin mx sin ny To pass from [-11", 11"] [-11", 11"] to [-a, ] [-b, b] requires only a simple 11"

a

X

o

x

change of variable. There is no loss of generality in using symmetric inter vals [ - a , a] because we can define the function to be 0 appropriately.

L2 ([-a, ] x [-b, b)) Let and b be positive. With m, n E N , an orthonormal basis for L2 ( [ a] [-b, b)) is given by 1 1 m 1l"x 1 sm. m 1l"x 1 n 1l"y 1 . n 1l"y cos --' a v'2cJ a - ' v'2cJ cos -b- ' ..j2;b sm -b- ' 1 m 1l"x n 1l"y 1 . m1l"x n 1l"y 1 m 1l"x . n1l"y -- cos -- cos -- ' -- sm -- cos -- ' -- cos -- sm -- ' -j;b b a b yQ a b yQ a 1 . m 1l"x . n 1l"y -- sm -- sm -- . 0 b a yQ a

Example 4.19.7 BASIS FOR a

-a,

x

2-j;b ' ..j2;b

The exponential form of the Fourier series ( cf. Definition 4.1 . 1) of a function E C is given by

f L 2 [- 1I", 1I"] L 1 [-11", 11"] � e e i n t , where n - �11" f(t) e - int dt . L..t n nEZ We define the Fourier series of f E L 2 ([-11", 1I"] n ) to be f(t) e - i { k, t ) dt, (4.69) L ek e i { k , t ) , where ek = � 1 (211") [ ] kE Z where t E [-11", 1I"t , and (k, t ) denotes the usual inner product on Rn . Since L 2 ([-11", 1I"t) is a Hilbert space for every n, the Fourier series of equation (4.69) converges in the I Hk norm ( in the mean ) to the function f; the e

n

2 17r

- 7r

- 7r 7r n '

analogues of Parseval's identity, Bessel's inequality, the Riemann-Leb esgue lemma, etc . , are valid. In the specific setting of 4.19.3, the Fourier series for E x d)) is given by

f L 2 ([a, b] [e, L amn fm (X) gn ( y) , m ,n EN

where

amn = U, fmgn) .

254

4. Fourier Series

f (x, y) = xy

(x, y)

Example 4.19.8 Find the Fou rier series for for E with respect t o the orthonormal basis of Example 4. 1 9. 6 for the real space

[_1r, 1r] 2

L5 ([-1r, 1rF) .

Discussion . First note that for

function,

m,

n E N U {O} , since y cos ny is an odd

(f, cos m:cos ny ) � [: (x cos mx) ([: Y COS ny dY) dx = O. =

Likewise , for any

m, n E N U {O} ,

=P 0 and n =P 0 , am n -1 1 " 1" xy sin mx sin ny dxdy - x -sin mx dx ysin ny dy ;:� 1_:" " 1_"" -1r4 1"0 x sin mx dx 10 " y sin ny dy � ( :) ( _ I) m+ l ( � J (-It + 1 � ( _ I) m +n . mn f Thus, the Fourier series for (x, y) = xy is m n sin mx sin ny . 4 L ( _ I) + mn Finally, for

m

m,n EN

[- 1

1

0

f (x, y) = xy

(x, y)

Example 4.19.9 Sh ow that the Fourier series for for E , ]x with respect t o the orthonormal basis of Example 4. 1 9. 7 for the real space is x

[- 2, 2] L5 ([-1, 1] [- 2, 2])

Solution . Since f is odd in each variable (cf. the previous example) , the

Fourier series reduces to

sin m1rX sin n1ry 1 -a mn L 1 2 ' .;r2 m ,n EN

4. Fourier Series

where for

255

m -# 0 and n -# 0, 1 1 X sin m1rX dX 2 Y sin n1ry dY 1- 2 VI · 2 1 1 4 1 x sin m1rX dx 22Y sin n1ry dY 10 2 v 1 · 2 0 m +1 4 ' (_I) 4 (-It + 1 VI . 2 m1rm + n n1r 16 (_I) VI . 2 ' mn1r2 . 1

-1

110

1

The associated Fourier series is therefore

(_I) m+ n sm. mx s . 2 ny ' L 1r2 m,n e N mn 8

m

0

Remark ODD IN EACH VARIABLE Generally, if the real-valued function f is square-sumrnable on x and odd in each vari = and = able separately (i.e . , for all then the Fourier series for with E x respect to the basis in Example is j ust

(x, y)

[-a, a] [- b, b] f (-x, y) -f (x, y), f (x, -y) -f (x, y) (x, y) [-a, a] [- b, bD, f 4.19.7 1 . m1rX . n1ry " a mn sm (4.70) s m -, a b vab m,� neN rl

where

amn = v4ab l0 rJ

--

a

-

m1rX . n1ry

1b f (x, y) sm. -a - sm -b- dxdy. 0

Exercis es 4 . 1 9 PRO DU CTS 1-+ PRO DUCTS Show that if E and = with = and = then the Fourier series of with respect to the basis of Exam ple is

f, 9 L 2 [- 1r, 1r] f (x, y)i n m i x e h ey) Ln EZ bn e y , g (x) LmeZ am f (x, y) f (x, y) L L am bn ei ( mx+n y) . mEZ n EZ 2. VARIABLE MISSING (This is a special case of the preceding exercise.) Show that if f (x, y) = g (x) with g (x) = L n EZ an e i n x , then f (x, y) L ane i nx . n EZ 1.

g (x) h (y) 4.19.4

=

=

256

4. Fourier Series

3. Assume that the following functions are defined on [_ 11', 11'] 2 and have been 211'-periodically extended in each variable. Expand each in real double Fourier series.

l (x, y) = xy2 . I (x, y) = x + y. l (x, y) = l xy l .

(a) (b) (c)

4. 5. 6.

View the functions of Exercise 3 as defined on [-1, 1] x [-2, 2] , peri odically extended, and find their real Fourier series. Expand in complex (i.e., in exponential form) double Fourier series: (a) I (x, y) (b) 1 (x, y) X

= xy on [ 1, 1] x [-2, 2] . = xy2 on [_ 11', 11'] 2 . -

-1I' < t < 0, g (t) = { -1, 1, 0 < t < 11', 4 L sin (2n - 1) t (Example 4.1.3). By Example 4.1.5, the 11' neN 2 n - 1 Fourier series of the even saw-tooth 1 (t) = I t I on [-11', 11'] is 2 ( cos t + 323t + 52 + . . .) Find the Fourier series of Ig with respect to the basis of Example 4.19.6. 7. Find the Fourier series of the function 1 defined on [_11', 11'] 2 by y . 1 (x, y) = { 0,I ' xy << x. SQUARE

IS

TRIANGLE The Fourier series for the square wave

-

l!:.

_

1. 'If

cos

�

-

•

'

8 . EVEN IN EACH VARIABLE Find the Fourier cosine series of a function x that is even in each variable (when the other 1E

L2 ([-a, a] [-b, b])

9.

is held fixed). BASES FOR L; ([0, 1I'F) Show that suitably normalized versions of {sin mx sin ny m, n E N } and {cos mx cos ny : m, n E N} constitute orthonormal bases for L; ([0, 1I'F ) . :

10. BESSEL'S INEQUALITY State Bessel's inequality ( L n e N I { x, xn) 1 2 � I I x ll 2 of 3.3.1) for the orthonormal base of Example 4.19.6 for L; ([-11', 11'] 2

Answers

. 3(a). (2 11'2 /3) L meN ( l)m+l sm mx 8 Lm,n e N ( ml )m+ n 2 ,. sin mx cos ny. m l )m + ,. e i ( m x+ n y) � m e Z { } (_ml ) m e imx+ 2 i L.. �m ,n e Z ,m ;tO , n ;tO (_mn 5(b). ( 1I'2 i/ 3) L.. 2 -

-

O

-

-

•

4. Fourier Series

4.20

257

Convergence of Multiple Series

We approach pointwise convergence of double Fourier series in a way paral lel to that of single series. For example, we saw that the nth partial sum S n of a Fourier series of a real-valued function of one variable can be written using the Dirichlet kernel Dn (Section 4.5) , ", n ikt Dn (t) L...,; k = - n e 1 + 2 cos t + 2 cos 2t + . . . + 2 cos nt (4. 7 1 ) sin ( n + 1/2) t , t =P 0, ±211", ±411", . . . sin (t/2) 2n + 1 , otherwise

{

as

(4.5.1)

j ""

1 I (t - x) Dn (x) dx. 211" I t is routine t o show that the pqth partial sum Spq , p , q E N , of the dou ble Fourier series of functions in L; ([-11", 1I"F) of Example 4.19.6 may be written 1 I (x - w, y - v) Dp (w) Dq (v) dw dv . Spq (x , y) = --2 (211") - ,," - ,," Sn (t) =

_ ,,"

j"" j ""

as

The analogue 4.20.1 of the pointwise convergence theorem 4.6.2 also pos sesses the feature that smooth functions have Fourier series that converge pointwise. We state 4.20.1 for continuous functions on [-11", 11"] 2 ; the modi fication to [- a , a] x [-b, b) is straightforward.

4.20.1 MULTIPLE POINTWISE CON VERGENCE THEOREM If the real-valu ed function f is continuous on [-11", 11"] 2 and has bounded partial derivatives Ix and Iy , then: ( a) If there is some neighborhood of the interior point (x, y ) E [-1I", 1I"F such that Ixy exists at every point in that neighborhood of (x, y), then the Fourier series of I converges to I (x , y) at (x , y) . ( b ) If I is 211"-periodic in each variable and has continu ous partial deriva tives lx , Iy , and Ixy , then the Fourier series of I converges to the 211" periodic extension of I everywhere. Thus, the Fourier series of I (x , y ) = x y ,

4

sin mx sin ny _ ( 1) m+n , L mn m neN ,

of Example 4. 1 9. 8 converges to the periodic extension of I everywh ere. Let I = [-11", 1I"] n , and let I E L; ( 1) be periodically extended. We il lustrate next how to get analogues of expressions for a partial sum of a

258

4. Fourier Series

multiple Fourier series for / using the Dirichlet kernel (a la 4.5.1) in equa tions (4.72) and (4.73); we consider the Fejer sum in equation (4.75). We continue to use the notation max 1/1 ( 1 ) for continuous / on I, 11 / 11 00 kIl II = max { I k ll , . . . , I kn I } for k = (ki ) E zn , 00

and

11 / 11 1 = 1 1/ (x ) 1 dx for / E L� (1) . We remind the reader that 11 11 00 -convergence is uniform convergence (Ex ample 2.2.8) . As in equation (4.69) of Section 4.19, the Fourier series of / E L'i ( 1) is x, t E l, and ( . , . ) is the usual inner product on Rn . For p E N, we take the

pth partial sum to be

Sp (x) = I kLl oo$p Ck ei ( k,x) .

(k l , . . . , kn), x = ( X l . . . ' xn ) , and t = (t l , . . . , tn), ( L e - i ( k, t -x ) J e t ) dt Sp (x ) = +11" ) l ( ji l kLl oo$Pe - ik ,(t,- Xt ) . . L e - ik ,, (t ,,-x,,) Jet) dt . +11" I k "l $ p ( ) I l k , l $p (4.72) Using the Dirichlet kernel Dp (t) L� = - p e ik t , P E N, we can rewrite this as n Sp ( x ) = (2 11"1 )n 1 P Dp (tj - Xj ) J e t) dt . (4.73) = l I Now for the analogue of the (C, 1 ) sum. First, recall the Fejer kernel of Section 4 . 1 5 , Fo (t) = 1, and for p E N, sin 2 (p + l) tj2 Fp (t) = (p + l) sin 2 tj2 ' t rf; 2 11" Z , With k =

'

.

=

J

{

P + 1,

otherwise.

We define the n-DIMENSIONAL FEJER K ERNEL to be the product

n Hp (t) = II Fp (tj ) , t = (t l , . . . , tn ) E Rn , n, p E N. j=l

(4.74)

4. Fou rier Series

259

We can write the pth ( C, 1) partial sum, the average up of the first p + (starting at 0) partial sums sp of equation (4.73), as

1

(2!t j Hp (x) f(x + t) dx (4.75) t) H (x f(x) dx. p = (2!t 1 As noted in 4.15.2, the ordinary Fejer kernel Fp (t) has the following Up (t)

=

properties. For every p E N:

Fp (t) is an even 271"-periodic function. (b) Fp (t) � 0 for all t E R, and Fp (t) 0 as p 00 uniformly outside [ -r, r] for all r > O . 1f ( c ) I Fp (t) dt = 271". 1f The n-dimensional Fejer kernel Hp has quite similar characteristics : 4.20.2 T H E n-DIMENSIONAL FEJER KERNEL Hp Let Hp (t) = II Fp (tj) , t = (t 1 , . . . , tn) E Rn , n, p E N . (a)

�

�

n

j=l

p E N: Hp is 271"-periodic in each variable; (b) Hp (t) � 0 for all t E Rn , a n d Hp ( t ) 0 as p 00 uniformly outside Ir = [-r, rr for all r > 0; (c) 1 Hp (t) dt = (2 71" t . The results of (a) and the first part of (b) are obvious. The uniform convergence to 0 outside of Ir follows by noting that if t E I - Ir , then for at least one coordinate t i , I t i I � r; (c) follows b y splitting II into the product of n integrals. Part (a) of 4.20.3 is the n-dimensional analogue of Fejer's theorem 4 . 15.3(b). 4.20.3 THE n-DIMENSIONAL FEJER THEOREM L e t n , p E N , let I = [-71", 7I"] n , let I den ote the 271"-periodic extension of f E L'i (1) , a n d let (t) = (2!t 1 Hp (x) I(x + t ) dx, x, t E Rn (equation (4.75)) be the n th (C, 1) partial sum of the Fourier series for f. Then: (a) If I is continu ous on I, then II I - u l l � O. (b) If f E L1 (I) , then 11 1 - up ll 1 o . p oo For every

(a)

�

Up

�

�

260

4. Fourier Series

I

Proof. ( a) Since is continuous on the compact set I, I is uniformly con tinuous there ( 2.8.8 ) . Since all norms are equivalent on Rn , for > 0 there exists r > 0 such that for every E l , < t / 2 for all such I + ::; r, i.e. , for all E Ir = [-r, r]n . Now, for p E N , that

t

t

I /(t x) - (t )1

x

x II x l i oo l up (t ) - I (t) 1 = (2!f 1 1 Hp (x) I /(t + x) - I (t)1 dx l ·

Split II into IIr + II -I r . The first integral is small by the choice of r, and 4.20.2 ( c ) :

·r ( 27rf 1Ir

1

Hp (x) I /(t + x) - l (t) 1 dx ::; (27r1 f 1rIr Hp (x) 2" dx < 2" . 1. The second is small because Hp (t) -+ 0 uniformly p -+ 00 on 1 - Ir (4.20.2 ( b )) and I(t + x ) - I (t) is bounded (1 / (t + x) - I (t) 1 ::; 2 11 / 11 (0 ) . ( b) Observe that t

E

as

III - up l 1

l l up (t) - I (t) 1 dt < 1 ( 2�r 1 1 Hp (x) ( J (t + x) - I (t)) dx l dt < ( 2!f 1 Hp (x) dx 1 1 / (t + x) - /(t)1 dt. With 9 ( x ) = II I / ( t + x) - I (t) 1 dt, this is ( 2!f 1 Hp (x) g (x) dx.

x,

Since 9 is continuous on I ( Exercise 1), 27r-periodic in each argument of and 9 ( 0 ) = 0, the expression above represents the pth (e, 1) sum of the Fourier series for 9 evaluated at = O. By ( a) , therefore,

t ( 2!f 1 Hp (x) 9 (x) dx Hence I I - up l 1 1 -+ o. 0

-+

0 as p

-+

00 .

Some sources that go beyond this glimpse of multiple Fourier series are Nikolsky 1977, Hobson 1957, and Zygmund 1959. Exercises 4 . 20

1.

9t

CONTINUITY IN THE MEAN Show that the function of 4.20.3 ( b ) is continuous. ( Hint: The idea is that for I E Ll (/) , if is close to then the translated function It = I + t) is close to It o = I + more precisely, that the map E l ....... It E Ll (/) is continuous. Use

t

(x

to, (x t o ),

4. Fourier Series

26 1

the fact that for any I E L1 ( I) and > 0 there exists a continuous function h on I such that III h il l = II II (x) - h (x) 1 dx < Then, for t, to E I, note that -

I g (t)

-

9 (to) 1

Finally, add and subtract

::;

i

II I I (x + t) - I (x + to) 1

L

dx.

h (x + t) and h (x + to).) [Hint: See 2.8.9.]

5 T he Fo urier Transform NOTE. In this chapter, unless otherwise indicated, all functions are complex-valued functions of a real variable.

t , f, and k, the function g (w) = L f (t)k(t , W) dt for some set E is called an INTEGRA.L TRANSFORM of f with KERNEL k (t, w) of the trans form . By "transforming" both sides of certain equations, we can sometimes Given integrable functions of

(

)

convert them into simpler ones-differential equations to algebraic equa tions, for example. Assuming that we can solve the transformed equation, then we convert its solution back into the original situation, this latter pro cess being called "inversion." The practicality ( ease, doability ) of inversion is the raison d 'etre of the transform method: If you cannot invert with reasonable ease, why do it? Up till now we have considered Fourier series representations of functions that vanish ( either naturally or by the brute force of truncation ) outside a finite interval, and of periodic functions. Assuming that

f

7r n J 211" f(t)e -i t dt is known for every n we recover f through L j ( n ) ei n t some kind of convergence . n EZ Now suppose that f is defined for all real t, and w e cannot truncate i t t o a finite interval for some reason. For f (R) , instead of integrating from j(n) = �

- 7r

E N,

(

-11"

to

11" ,

we integrate over R:

)

E L1

(5 . 1)

f.

The latter expression is called the FOURIER TRANSFORM of Rather than from a series, we seek to recover from the integral ( the inversion formula ) . (w) -1

2

f 1 00 f e,w t dw - 00 �

11"

.

As we shall see, the inversion formula holds for a large class of functions

f E L1 (R).

264

5. The Fourier Transform

We recast some things that we did for Fourier series into exponential form in Sections 5.1-5.3. The reason for this is to motivate analogous re sults for the Fourier transform, which we introduce in Section 5.4. Guided principally by the quest for analogues of results for Fourier series, we in vestigate the Fourier transform in this chapter. In Section 5.18 we consider the Fourier transform of functions I E L 2 (R) . This is the most symmetric theory. The map I 1-+ j is practically a linear isometry of L 2 (R) onto L 2 (R); it misses being an isometry by a factor of V21r: 2 V21r 1l / 1l 2 (see Plancherel's theorem 5.19.3) .

1I �1

=

The Finite Fourier Transform

5.1

p.

Convention We normalize Lebesgue measure taking p./27r. Thus, for I, E L 2 [7r, 7r] ,

9

in Sections

5.1-5.3 by

(f, g) = 217r 1 '" I (t) 9 (t) dt, and 11 / 11 22 = 27r1 1 '" I I (t) 1 2 dt. For I E L 1 [7r, 7r] , 11 / 11 1 = 2� f�". 1 1 (t) 1 dt. Any trigonometric series �o + L an cos nt + L bn sin nt , t E R, n eN neN where the (an) n �o and (bn) n � l are complex sequences may be converted into exponential form L: n eZ cn e i n t by taking bo = 0 , and for all n � 0, -

_ ".

-

-

_ ".

(5.2)

i

To convert from exponential to trigonometric form, use an = Cn + C - n and bn = (cn - c - n ) . Note, too, that for every n � 0, Sn (t) = ,",� n+ L:�= 1 ika k cos kt + L:� = 1 bk sin kt = wk= - n Ck e t ,

We say that L: n eZ Cn e i n t converges (somehow) to s ( t ) if the symmetric partial sums Un (t) = L: �= - n Ck e ik t converge to s (t) as n The expo nential form of the Fourier series of I E L 1 [-7r, 7r] is given by � 00 .

I(t)e - in t dt. 1'" L i{n) ein t , where j(n) = � 27r - ". n eZ

The relationships between the j (n) and

(5.3)

2 an = 7r1 1 '" I(t) cos nt dt, n � 0, and bn = 7r1 1 I(t) cos nt dt, n � 1 , -

_ ".

-

0

".

5. The Fourier Transform

5.

265

( t

are as in equation ( 2 ) . We call the bisequence j (n) of complex E Z Fourier coefficients of f the FINITE FOURIER TRANSFORM ( FINITE FT ) of Ij we say "finite" because the domain of integration [ -71", 71"] is finite, in contrast to the Fourier transform of equation of Section 5.4 in which we employ (Sometimes we get less formal , and call just j(n) the finite Fourier transform rather than j (n) n ) By making changes EZ of variable, i t i s routine t o verify that the finite Fourier transform has the following properties.

(5. 15)

f�oo .

( ) .

5 . 1 . 1 FINITE FOURIER TRANSFORM BASICS If have: function finite FT

I E Ld - 7I", 7I"] ,

then we

f (t) f (-t)

j(n) j C -n) I (t - a ) e - i an fen) eik t l (t) j(n - k)

(t)

j

j(-n)

Products are more complicated. 5 . 1 .2 PRODUCTS OF L 2 FUNCTIONS (cf. Exercise 4.1-13) Let f, E

9

L 2 [-7I", 71"]

have the Fourier transfo rms j and g, respective/yo Then the finite FT of E at any k E Z is given by

Ig L I [-7I", 7I"]

fg (k) = Proof.

B y 5 . 1 . 1 , i t follows that

fg

L j(n) g (k - n) . n EZ

--

g et) = g (-n) and g e i k t = g (- (n - k)) = g ( k - n ) . Since L 2 [- 7I", 7I"] is a Hilbert space, we may use Parseval's identity 3.3.4(f) to get 1 I(t)g (t) e - ik t dt fg (k) 271" 1 11" 1 11" I(t)g (t)eikt dt 271" 1Ln EZ11" j(n) g (k - n) . .=..

--

- 11"

Definition 5 . 1 . 3 THE CIRCLE GROUP T

0

: =

The points of the unit circle T {z E C I z l I} of the complex plane form a multiplicative group known as the CIRCLE GROUP , and the map R/271"Z (R modulo 271") =

t

--+

I--->

266

5. The Fourier Transform

is a group isomorphism of the additive group R/211"Z onto T . From now on we view 211"-periodic complex-valued functions E Lt [ - 1I", 1I"] or L 2 [ - 1I", 1I"] as having T as their domain, and denote L 1 [-11", 11"] and L 2 [- 1I", 11"] by L 1 (T) and L 2 (T) , respectively. 0 Since { e i n t /.J27r n E Z } is an orthonormal base for the Hilbert space L 2 (T) of square-summable, complex-valued functions on T, it follows from the general theory of Hilbert spaces (3.4.9) that the map

/

:

(5 .4) is a surjective inner product isomorphism ; in particular, every square summable bisequence (an) of complex numbers is the Fourier transform of some I E L 2 (T) , and

This equality is frequently called PARSEVAL'S IDENTITY (compare it to the version in 5.17.2, where the domain is R rather than T) . For any I E L 1 (T) , by the Riemann-Lebesgue lemma 4.4.1, le n) 0 as n ±oo . Thus, associated with each E L1 (T) is a null bisequence = 1 To discuss what happens for L1 functions, recall that the space Co (Z) (or j ust co ) of all complex bisequences (a n) n eZ such that limn-d oc an = 0 with the sup-norm 11 · lloc is a Banach space. Now consider the process of converting L 1 functions into bisequences j. Definition 5 . 1 .4 FINITE FOURIER MAPPING F1 The map

/

(1 (n))

->

->

/

F1 :

L 1 (T) /

� �

is called the FINITE FOURIER MAPPING. 0

co � ) ,

/,

(5 .5)

We show in 5 . 1 . 5 that the properties of the finite Fourier mapping F1 are similar to those of the map F2 of (5 .4) . 5.1.5 T H E FINITE FOURIER MAPPING F1 The finite Founer mapping Fi is lin ear, continuous, and injective, i. e., il j = 0 lor every n, then 1 = 0 a. e., or, equivalently, il j(n) g (n) lor every n E N, then I = 9 a. e.

(n)

=

P roof.

The linearity is clear. As to the continuity of F1 , note that

l i
j'"_". I / (t) 1 dt = 1 / 1 1 for every n ,

267

5. The Fourier Transform

I l i ( n )IL

so Il Fd lloo = ::; 11 / 11 1 ' and F1 is continuous by 2.3.3. As observed after Lebesgue's pointwise convergence theorem 4.18.5, if = 0 for every then every Cesaro sum (Tn (t) == 0; since the Cesaro sums converge to I a.e. by 4.18.5, it follows that I = 0 a.e . , in other words, that as a member of L 1 (T) , I = O . 0

n,

i (n )

F1 is not onto

A way in which F2 is different from the finite Fourier mapping F1 is in the surjectivity. The map F2 of (5.4) maps L 2 (T) onto 1.2 ( Z)-indeed , F2 is a Hilbert space isomorphism , guaranteeing that L 2 (T) and 1.2 (Z) are the same as Hilbert spaces. The finite Fourier map F1 , however, does not map L 1 (T) onto co . As a consequence of the bounded inverse theorem (Bachman and Narici 1966, p. 271), a continuous linear bijection between Banach spaces is bicontinuous. The finite Fourier map F1 of (5.5) is linear , continuous, and injective, and L 1 (T) and (Z) are Banach spaces; so, if by the criterion for F1 were onto, F1- 1 would be continuous. Therefore, continuity for linear maps of 2.3.3, since F1- 1 is obviously linear , there would be a [{ > 0 such that Co

11 / 11 1 = II F1- 1 (Fd) 11 1 � [{ Il Fd ll oo = [{ I �L for every I E L 1 (T) . In particular, for the D irichlet kernel I = Dn (t) = E�= - n ei k t , for every n E NU {O}, ( 5.6 ) Computing a k is;. (k) by equation ( 5 . 3) , we see that F1 (Dn) is the bisequence a k 1 for k = 0, ±1, ±2 , . . . , ±n, and a k = 0 otherwise; hence 11 F1 (Dn) lI oo = 1 for every n. As follows from the argument after ( 4.47 ) of Section 4.9, however, II Dn ll l � 00 . Therefore, F1 is not onto i.e . , there are bisequences a n --+ 0 as n � ±oo that are not the Fourier coefficients of any L 1 function. =

Co ,

Convolution on T

5.2

For a complex-valued function I E L 1 (T ) , the nth partial sum of its Fourier senes IS

or, in terms of the Dirichlet kernel Dn (t) =

E� = - n ei k t , Sn (t) = 2111" 1 11" I (x) Dn (t - x) dx. _ 11"

(

5.7 )

268

5. The Fourier Transform

The expression 21.,.. J�.,.. I (x) Dn (t - x) dx is denoted by I * Dn (t) and called the CONVOLUTION of I and Dn. The order is unimportant: 1 * Dn (t) Dn * I (t). Integrals like this arise so frequently in harmonic analysis that we single them out for further study.

=

Definition 5.2.1 CONVOLUTION

For any I, g E L 1 (T) , we define the CONVOLUTION of I and g to be

l * g (t) = 211"1 j"" / (t - x) g (x) dx. _.,.. We justify the existence of the integral in 5.2.2 below. 0 Note that products of L 1 functions need not be L 1 functions. Consider the square of the integrable function I (t) = t - 1 / 2 on (0, 1]. This is why (L 1 (T) , +, . ) , where · represents pointwise multiplication, is not an alge bra. The properties of convolution are such that (L 1 (T) , + , * ) is a Banach algebra ( recall that Ld-1I", 1I") is a Banach space) , however.

(L 1 ( T) , + , * ) A BANACH ALGEBRA For any I, g, h E L l (T) , * g ) (t) exists for almost every t, belongs to L 1 (T) , and II (J * g ) 11 1 � 11 / 11 1 II g l1 1 . (b ) (L 1 ( T ) , + , * ) forms a commutative algebra with respect to pointwise

5.2.2 ( a) ( J

addition and scalar multiplication and convolution. In particular,

I*g g * f, (J * g ) * h, 1 * (g * h) a (J * g) = I * ag (a E C) , (a t) * g 1 * ( g + h) 1 * g + I * h. Proof. ( a) Since f i s periodic, D.,.. I I (t - x) 1 dt = 211" 11 / 11 1 for any x by 4.2.2. Hence, i: I I (t - x) g (x) 1 dt = I g (x) 1 i: I I (t - x) 1 dt = I g (x) 1 (211") 11 / 11 1 · Therefore, integrating with respect to x, i"".,.. (i"".,.. I/ (t - x) g (x) 1 dt ) dx = i>2g (x) 1 dx (211") 11 /111 (2 Jr ) 1I g I1 1 11 / 11 1 It follows by the Tonelli theorem 4.19 . 2 that I (t - x) g (x) is integrable on [_11", 11"] 2 ; by Fubini's theorem 4.19.1, J�.,.. I / (t - x) g (x) 1 dx is defined for <

00 .

5. The Fourier Transform

269

ahnost every t. Hence, with one more change in the order of integration,

211" 11 1 * gil l

1: 1 (f * g) (t) l dt 1: 1 2� 1: I (t - x) g (x) dx l dt < 211"1 1-'1flI" l_'1flI" I / (t - x) g (x) l dx dt < 2� 1: I g (x) 1 (1: I/ (t - x) 1 dt) dx < 211" g /11 . Il 1l 1 11 1

(b) All the verifications are routine. The argument for commutativity illustrates the procedure. With w t - x, =

(f * g) (t)

= =

1 I ll" / (t - x) g (x) dx 211" _ lI"- lI" - 1 11 1 (w) 9 (t - w) dw 2 11" 11++ lI" 1 lI" I (w) g (t - w) dw 211" 11 -'1f lI" 1 1 / (w) g (t - w) dw by 4.2.2 211" _ (g * f)lI"(t) . 0

The following result is the most imp ortant property of convolution . Com pare it to 5.1.2 . 5 .2.3 CONVOLUTIONS TO PRODUCTS For any

j ( n ) g ( n) . Proof.

I, 9 E L 1 (T)

,

/;-g

(n) =

By the Fubini theorem 4.19.1, we can change the order of integration

so that

1 : in 211" 1 e - t [I * 9 (t)] dt 1 n 1 211" I_ll"lI" e . [ 211" Ill"_ lI" / (t - x) g (x) dx] dt 1 n 211" Ill"_". g (x) ( 2111" Ill"_ lI" e - o· tl (t - x) dt ) dx. In the inner integral f�lI" e - i n t 1 (t - x) dt, with u = t - x, 1:�: e - in (u +x ) / (u) du = 1: e - in (U +X) / (u) du = e - i n x 1: e - i n u 1 (u) duo /-:g

(n)

-o

t

270

5. The Fourier Transform

j 1l" . 271" j1l" m. / (u) du = /� (n) g (n) . I-* g (n) = 271" g (x) m dx -

The original double integral therefore is the product 1 e- x . 1 e- u _

_ 11"

11"

0

Exercises 5 . 2

1. Split I E L 1 (T) into its real and imaginary parts: I = 9 + ih. Show

j 2. For I E Lt {T), define f* (t) -f* = -;::.I ·

that if = 0 then Ii and 9 are O.

3. 4.

=

I(

-

t ) Show that .

f = ( i) *

and

TRANSFORMS OF DERIVATIVES Let I E L 1 (T) be absolutely con tinuous so that its derivative f' exists a.e. Use integration by parts to show that f ( n ) = inj(n) .

Show that if the Fourier series Ln E Z i (n) ei n t of the continuous func tion I E L 1 (T) converges uniformly, then L n E Z j(n) ei n t = I (t) for all t . 5 . PROPERTIES OF CONVOLUTION Verify that for any I, g, h E L l (T) and any scalar a ,

6.

1 * (g * h) ( a t) * 9 1 * (g + h)

(f * g ) * h, a (f * g ) , I * g + I * h.

9 E Lt [- 7I", 71"] , with 9

CONVOLUTION WITH EVEN FUNCTIONS For I, even, show that 1 * (t) = 1 I (t + x) (x)

9

271"

j1l"

_ 11"

9

dx.

7. Show that if p (t) is a trigonometric polynomial L�=_ n cj ei k t , then so is 1 * for any I E Ll (T). 8. Let e n (t) = ei n t . Determine (a) e n * em (t) . (b) I * e m (t) for I E L 1 (T). 9. THE TRANSLATION OPERATOR Ta For any I E L 1 (T) , define the translation operator Ta for any a E T to be Tal (t) I (t - a ) . For any 9 E L 1 (T) show that: (a) Ta (f * g) = Ta l * 9 + I * Tag. (b) Ta l * ng = Ta + b l * 9 for any b E T. (c) TJ (n) = e-i n a i (n) for any n E Z. p

=

5. The Fourier Transform

271

5.2 . 3

10. L1 (T, * ) HAS NO MULTIPLICATIVE IDENTITY Use the convolutions

and the Riemann- Leb esgue lemma 4.4.1 to products theorem to show that there is no u E L 1 (T) such that f * u = f for all f E L l (T). 1 1 . ApPROXIMATE IDENTITY Although L 1 (T) has no identity, it has APPROXIMATE IDENTITIES . An "approximate identity" is a sequence (bn) of functions from L 1 (T) that behave like Dirac's delta function in the limit (see (ii) of (a)) but serve as an identity in the limit in the sense of (c) . (a) An APPROXIMATE IDENTITY for L 1 (T ) is a sequence elements from L 1 (T) such that i. SUP n II bn ll l < 00 ; 1 bn (t) dt = 1 ; ii. limn

21r 111" iii. limn 1 I bn (t) 1 dt = 0 for any r r �l t l::;1I" _ 11"

E

(bn)

of

(0, 1r ).

(b) Let Dn (t) denote the Dirichlet kernel �� = - n e i k t , n E NU { O } . Show that the Fejer kernels Fn (t) = ifI � � =o Dn (t) , n :::: 0, constitute an approximate identity for t (T ) . (The Dirichlet kernels do not because II Dn 11 1 as follows from the argument after inequality (4.47) of Section 4.9 .) (c) Show that if (bn) is an approximate identity for L 1 ( T ) , then -+ 00

lim n bn * f =

f

for all f E Ll ( T ) , where the limit is taken with respect to 11 · 11 1 . We continue our discussion of approximate identities in Section 5 . 14.

Hints

4. The function g (t) = � n EZ j(n) e int is continuous by the uniform convergence of the series. The uniform convergence also permits term wise integration, so j(n) = g (n) for all n; therefore , f = 9 a.e. by 5 . 1 .5. Since f and 9 are continuous, it follows that f = 5 . With a change in the order of integration, we get

g.

21r 1 11" f (t - x) h] (x)1I" dx 2� L: f (t - x) [ 2� L 1I" g (x - w) h (w) dW] dx �111" g (x - w) h (w) dX] dw. 21r f (t - x) [�111" 2 1r 1

_ 11"

_ 11"

[g *

_ 11"

272

5. The Fourier Transform

u = x - w, this becomes 1 1 '1f / (t - u - w) [ 1 1 '1f'If- w y(u)h(w) du dw ] 271"1 _ 7r 271" _ _ w -(271"1 ) 2 1'1f 1 '1f (t - u - w) y (u) h (w) du dw 271"(f*1'1fy)- 'If*I*h(t).y (t- w)h(w) dw 9. ( c ) . T-a l ( n) = -2171" 1'1f I(t- a)e - m. t dt, but 7r 7r n n (t - a) dt -i t i e 1 e � = dt 1 (t) t ( � 271"eina - 7r'1f 271" 7r - 1 I (t - e -i n t dt. 271" 10. IfL 1 such a u exists, then ;;;-/ (n) = u(n)j( n ) = j(n) for all I E (T) and n E Z. Hence (n) = 1 for all n E Z, which violates the Riemann-Lebesgue lemma 4.4. 1 . 11. which (b) . As shown i n 4.15.2( c ), (1/ 2 71" ) f Fn (t) dt = 1 for every n, from (i) and (ii) follow ; (iii) follows from (see 4.15.1) the fact that - r �lmaxt l � 7r Fn (t) - (n l) sin1 2 (r/2) (c) . First show that limn I bn * I - 1 11 00 = 0 for all continuous I on To see this, consider bn */ (t) -/(t) 2171" 17r- 7r bn (x) dx = 2171" j-'lf'If bn (x)(f(t - x) - /(t)) dx . With = 21'1f f::'If bn (x) dx and Tx the translation operator of Exer cise 9, With

I

- 'If

- 'If

- 'If

I

I

a)

- 'If

- a)

u

:: 'If

o<

<

.

+

T.

an

0, E (0,71"] I Tx l - 111 00 for I x l f71" j7r l bn (x) I I Tx l- /l oo dx = f (Jlfx l � b 1b�lx l � 7r ) .

For f > choose b such that Now split the integral in two:

< f

+

- 'If

71"

::;

b.

5 . The Fourier Transform

In the first integral and every

n,

273

l i T.,! - !1 I 00 < fj by (i) , for some constant 1( ,

1 f 2 7r 111" I On ( x) 1 dx :::; J(f . 2 7r J[1"I 5,b I On ( x) I II T.,f - ! 1 I 00 dx :::; - 11"

As to the second integral

by (iii) . Therefore, lim suPn Since is arbitrary and an o.

=

f

l i on * ! - an!l l oo

-+

:::;

KL

1 by (ii) , it follows that limn li On * ! - ! 1 I 00

For any ! E Ll (T) and f > 0, choose a continuous function o n T such that < L Since L (T) is a Banach algebra, and (i) holds ,

g l l i on * ! - On * gi l l :::; l i On 11 1 I I ! - g il l :::; J(f , where K is a constant such that l i on I ! K for all E N. Since l i on * g - gl l oo < f for sufficiently large by the previous result, it follows that l i On * g - gi l l < 2i1O11"nJ�1I"* I-On * g - gl dt l g gl l oo < I ! - g il l

:::;

n

Therefore, for large

n,

n

L

and therefore

5 .3

The Exp onential Form of Leb esgue 's Theorem

As in equation (5.7) of Section 5.2, the nth partial sum of the Fourier series for (complex-valued) E L (T) may be written

! l

where Dn (t) denotes the Dirichlet kernel (Section 4.5) : n Dn (t) = L eikt .

k=- n

274

5. The Fourier Transform

By considering real and imaginary parts separately, it is easy to extend Lebesgue's theorem 4.18.5 on pointwise convergence to complex-valued functions; hence, the Cesaro means Un satisfy (t) + (t) + . . . + 8n (t) Un (t) = 1 (t) a.e. n+ l Since convolution is distributive, 1 Un (t) t + (t) + . . . + (t)) -n + 1 (so ( ) 8

0

--+

81

S1

=

1 n+1

8n

(f (t) Do + . . . + 1 (t) Dn) " n Dk (t) . I (t) � n + 1 L..J k =O *

=

*

*

Thus, in terms of the Fejer kernel 1 "n D (t) Fn (t) n + 1 L..J k =o k 1 " n " k ez. mt n + 1 L..J k = o L..J m = - k "n ei k t , L..J k = - n 1 � n+ 1 We can write Un (t) 1 * Fn (t)

(

� 211"

(

_

)

)

) eik(t- X)] dx �= n (1 - � 111" 1 ( ) [kL..J n+ 1 - 11"

( (

x

) � [1_11" 1 11" )

-

e i k t . 211" E k= - n 1 - 1L n+1 "n i (k) e ik t . L..J k =- n 1 - � n+1 n

(x)

e - ik x dX]

(5 .8)

Consequently, We can write Lebesgue's theorem 4.18.5 in the following way: 5.3.1 LEBESGUE'S POINTWISE CONVERGENCE THEOREM II Let T de note the unit circle in the complex plane, and let 1 E Then

E j(k) ei k t = I (t) (G, I)

kEZ

L 1 (T).

a. e. ;

for every Lebesgue point t of /, limn E: _ n ( 1 - * ) i ( k ) e , k = f (t) = ¢:::::>

t

(n + 1 lim � n + 1 k= n n

t

- I k l ) i (k) e ik t = f (t) .

5. The Fourier Transform

275

Similarly, by 4.6.6, we have the following:

If f E L1 (T) is of bounded variation, k i t then limn L � = - n j (k) e = t (f (t - ) + f (t + )) for every t E T. I t follows from the Riemann-Leb esgue lemma that the sequence ( j(n) ) is in Co ( Z) . What if it satisfies the stronger requirement ( j (n) ) E .e1 (Z)? We showed in 4.7.2 that any sequence (cn) E .e1 (Z) comprises the Fourier coefficients of a continuous function. If the Cn are Fourier coefficients j (n) of some f E L1 ( T) , Lebesgue's theorem implies that L j(n) ei nt = f (t) (G, l) . n eZ For (cn) E .e1 ( Z) , L n eZ j (n) e i n t is absolutely convergent-hence con vergent. By the regularity of (G l) summability (4. 14.6) , the ordinary 5.3.2 JORDAN COMPLEX FORM

,

sum must be the same as the (G, 1) sum, i.e., if f E L1 (j(n) ) E .e1 ( Z), then f is continuous, and for all t E T,

(T) is such that

L j(n) e i nt = f (t) . n eZ

5 .4

Motivation and Definition

Trigonometric Approach

The Fourier coefficients for f E L t [- p , p] are

and

P n7rt dt (n E N U {O}) an = P-1 j f (t) cos P -P

P n 7rt dt (n E N) bn = P-1 j f (t) sm. P -P

where an and bn are complex numbers. The Fourier series for the 2p periodic extension of f is

n7rt + '" a o � an cos . n7rt � bn sm -. 2" + n'" p p neN eN

If, for example, f is piecewise smooth, then (4.6.2, complex version) f may be recovered at any point t of continuity from the Fourier series as ( pointwise convergence)

n7rt n 7rt � bn sm. -. ao + '" � an cos - + '" f (t) = 2" p p n eN neN

5. The Fourier Transform

276

f�p

Substituting an = � f (t) cos nt dt and bn = � expression for f (t), and using the fact that n1r P

n1r P

f�p f (t) sin t dt into this n

cos - (t - x) = cos - t cos -x + sm -t sm -x, we get

f (t) = 2p1 jPp f (x) dx +

-

n1r P

. n1r P

. n1r P

Ln EN [l j-Pp f(x) cos - (t - x) dx] . n1r P

P

oo? i.e., what happens if the period of f is infinite? f�oo If ( x ) I dx < 00 , or if f satisfies the weaker pvjOO f (x) dx = P _p f(x) dx < 00 , then 2� f�p f (x) dx -+ 0 p -+ 00 , and maybe What happens as p If f E Ll (R), so that condition

-+

li.� jP

- 00

li.� nLE N [ l j-Pp f(x) cos - (t - x) dx] .

f (t) = p

as

p

n1r p

(5.9)

N)

Let us look more closely. Let Wn = n1rlp ( n E and �w n = Wn + l - Wn = 1r Ip. With this change of notation the series above becomes

Ln E N [ 1 j-PP f(x) cos - (t - x) dx] = nLE N [-;-W j-PP (x) p

With h (w)

n 1r P

�

f

cos wn (t - x)

= � f�p f( x) cos W (t - x) dx the expression becomes

dx] .

( 5 .1 0 )

Since �wn = 1r lp 0 as p -+ 00 , this looks like a Riemann "sum" for fooo h (w) dw (note that the expression in equation ( 5 .1 0 ) is an infinite se ries, not a sum) , which leads us to suspect that -+

f (t) = � 100 [I: f (x) cosw (t - x) dX] dw. (5.11) The expression in equation (5.11) is called FOURIER'S INTEGRAL FORMULA not to be confused with the Fourier integral. If we expand cosw (t - x) in equation ( 5.11), we get f (t) = 100 [a (w) cos w t + b (w) sinwt) dw, (5.12)

277

5 . The Fourier Transform

a (w) = -�1 1 00 f (x) cos wx dx, and b (w) = �1 1 00 f (x) sinwx dx.

where

-

- 00

- 00

The expression in equation (5.12) looks more like the continuous analogue of the Fourier series

L

f (t) = ;0 + L an cos nt + bn sin nt. nEN n EN

An integral appears now instead of a summation, and the discrete and is replaced by a continuously varying parameter

w.

bn

n in a n

Exponential Form

The form of equation (5. 15) below is the most common one, and the one that we shall most frequently utilize. To motivate it, we start with the exponential form of a Fourier series, and then take the same sort of limits that were considered for the trigonometric form. The exponential form of the Fourier series for E Ld-p, is

f

Ln EZ j(n) eimrt/p , 1_pP f(t)e-imrt/p dt, n

where

p]

E Z. j(n) = � 2p For sufficiently smooth f we may recover f at any point t of continuity from the transform j (n) f (t) = j(n) e imr t /p . nEZ as

Assuming that the limits exist

L

p --+ 00 ,

" ! (n) e i mr t / p = l im " [ � 1 P f(t) e - i mr t / P dt ] e i n 1f t / p . f (t) = plim oo p -+ oo nEL...Z., 2p - P -+ nL... E Z., With Wn = n�/ p (n E Z) and .6.wn = Wn+ 1 - W n �/ p this becomes as

=

With h

(w) = J�P f(t) e - iw t dt we have (5. 13)

278

5 . The Fourier Transform

Llw n = 0 p 00 , this resembles a Riemann sum for �2 f�oo h (w) dw (as with the trigonometric approach above , the ex pression in equation is an infinite series, not a finite sum) . This

eiw7rt/p (5. 13) I (t) 217r /-0000 /(w ) eiwt dw . . -217r /00- 00 [ -0000 I(y) e- 'wy dy] e·wt dw. (5. 14) I (R)FOURIER , f�oo I(y) e- iw y dy TRANSFORM I: I). (5. 1 5) I(w) i: I(t)e - i w t dt (5. 14) I (t) = -217r /00- 00 I (w) e,wt dw (5. 16) 5. 5 . 3 , f�oo5. 5 .4), 4. 5 . 7 fnot:�oo f�oo . I(w) I L 1 (R), I(w) 0 1/27r (5. 7 . 2). w 1/v'2ii Since

{:}

-

-

leads us to suspect that

-

For any E L1 define it to be the

h

exists, and we

the inner integral

of

=

(the Fourier transform of

In this notation equation

becomes

�

.

(the inversion formula) .

Since integrable functions can have nonintegrable Fourier transforms (Ex amples and we will consider various possibilities in the in terpretation of in the inversion formula, namely some of the variants of Definition such as or the Cauchy principal value PV Even though integrability is one of the attributes of Fourier transforms for E each is uniformly continuous and decays to as The factor may be bundled with the inversion formula - 00 as we have done or with the Fourier transform, or can appear in front of the transform and the inversion formula to provide a symmetric appearance. All these approaches are found in the literature . Because of the spectacular applications of the Fourier transform in elec trical engineering, we frequently refer to as time and to as frequency. Another form for the Fourier transform of used frequently in prob by ability theory, replaces in this case, if is the probability density function of the random variable X, then

t I, e-iwt eiwt ; I h x i: I(t) e i x t dt CHARACTERISTIC FUNCTION I. ( )

is called the

5.5

w

=

of

Basics/Examples

We need some concrete things to work with . To indicate the correspon dence between a function E = and its Fourier transform

I

L 1 (R)

I (w)

5. The Fourier Transform

J�oo f(t) e - iwt dt, we use the notation f (t)

�

279

j(w) .

We develop some general properties of Fourier transforms in 5 .5 . 1 , then apply these general principles to some special cases. 5.5.1 FOURIER TRANSFORM BASICS For f bers a and h, ( a) SCALIN G ( DILATION ) For a 1= 0,

f (at )

�

E L l (R)

and any real num

1!l j ( �) .

(b) TIME-SHIFT 1-+ FREQUEN CY MODULATION

f (t - h)

_

(c) EXP ONENTIAL MODULATION

e - iw h j(w) .

1-+

FREQUENCY SHIFT

ei llt f (t) - j(w - a) .

(d) COSINE MODULATION

[cos at] f (t) - "21 f� (w - a) + "21 f� (w + a) . (e) CHANGE O F RO OF If 9 E L l ( R) , then I: j(t) g (t) dt = I: f (t) g (t) dt.

Remark

These integrals are finite because the Fourier transform j (w) of

L l function is uniformly continuous and bounded (see 5. 7. 2). P roof. (a) With at, l: f(at) e - iwt dt f (at) �l a l 1°O f(u) e - iw (u / Il) du an

u =

1-+

- 00

1! l j ( � ) ·

Parts (b) and (c) require only a simple manipulation. To illustrate , we prove (c) .

e i llt f (t)

1-+

I: f(t) e i llt e - iw t dt

i: f(t) e - i (W -Il)t du

j (w - a) .

280

5. The Fourier Transform

( d ) By ( c ) , �

( cos at) I (t)

2

I(t)ei at + �2 I(t) e - i at

1-+ 2/ (w - a) + "2 / (w + a) . 1�

1�

(e)

i: !(x) g (x) dx i: [I: I (t) e -ixt dt] 9 (x) dx i: i: I (t) 9 (x) e - ixt dx dt. =

( 5 . 17 )

Since the iterated integral

is equal to

i: Ig (x) 1 dx · i: I I (t) 1 dt <

00 ,

( )

the Tonelli theorem 4 . 1 9 .2 permits us to write equation 5 . 17

()

It follows from 5.5. 1 a that

I( at) e1b. t

I (at)

1 !1 !(w/a) . By (c ) , therefore , 1 (W b ) a . l a l I --

1-+

�

1---+

as

-

The first two examples illustrate that integrable functions need not have integrable transforms. Definition 5.5.2 THE UNIT STEP FUN CTIO N U STEP FUNCTION to be

U

(t)

=

1 [ 0 ,00)

=

{ 0I, ,

(t)

We define the UNIT

t 2: 0, t < 0.

This is not a "step function" in the sense used in Definition 2.4.6 because it does not vanish outside a bounded interval; we use the term unit step function because this is what U is customarily called. 0

5. The Fourier Transform

Show that for any a

Example 5.5.3 EXPONENTIAL

integrable)

Sol ution .

f (t) = e - a t U (t)

1--+

281

> 0 (so that f

IS

1 . . a + zw

--

Since limt-+ oo e - ( a + iw )t = 0,

a + iw With f as in Example 5.5.3 and a = 1 ,

lew) 1 w , I l (w ) 1

= __ .

1 + zw

and

I j (w) I = �. 1 + w2

(

For large 1/ Iw l so i rf. L1 R) and this for an f that decays to 0 in magnitude exponentially! The rectangular pulse of Example 5 . 5 .4 does not approach 0 rapidly, it is 0 outside [ -a , a] ; still, it does not have an integrable transform. Thus, even very tame behavior of a function does not guarantee the integrability of its transform. Functions such as the rect angular pulse l[ - a,a] of Example 5 .5 . 4 that vanish outside a closed interval are called TIME-LIMITED . Note that the transform 2 sin aw /w of 1 [ -a,a] does not vanish outside any closed interval (is not BAND-LIMITED , in the sense that its nonzero values are limited to a band of frequencies - b ::; w ::; b) ; in fact, the greater the duration (namely 2a) of the pulse, the greater the "concentration" of the transform (see the figures after 4.5 .4) and the more closely it resembles Dirac's delta function. �

Example 5 . 5 .4 RECTANGULAR PULSE

I [ - a , a]

(t) = {

I,

0,

-

For the characteristic function

It I < a, I t I :; a,

, a > 0,

. -sm aw sm aw . on a, a] , show that I [- a,a] = 2-- = w w/2 .

-

Solution .

f

a . -e - zwt I [ - a ' a] (w) = -a

dt = e

i aw - e -i aw .

ZW

=

2

sin aw

--

W

.

By the argument after inequality (4.47) of Section 4.9, sin w/w rf. L 1 (R). Thus, for a = 1 (actually, for any a f. 0) , here is another instance of a . nonintegrable transform. 0

282

5. The Fourier Transform

The hat function k (t) = ( 1 - It I ) 1 [ - 1,1 ] (t) of the next example plays an imp ortant role in Section 5 . 1 2 , where we discuss (5. 12.3) the Fourier transform analogue of Lebesgue ' s theorem namely, that the Fourier series of any 1 E L 1 [-71'", 71'"] is Cesino summable to 1 a.e . For that reason, the hat function is also known as the CESARO KERNEL.

4 . 1 8.5 ,

Example 5.5.5 THE HAT FUN CTIO N Show that

k (t) = (1 - It I) 1 [ - 1,1] (t ) Sol ution .

Since k is even,

2 2

2

.0

(1 -

-2

0

( : - C � I:

Integrating by parts , we get =

Si W

s wt

2 - ( 1 - cos w) w2 . W -2 sm 2 - 0

w4

( sin�/t2) ) 2

ill k(t) cos wt dt 1 1 t) cos1 wt dt sm w 1 t cos wt dt . -w

k (w)

k (w )

f---+

-

�2 (1 - C OS W )

)

2.

A rare case of symmetry between function and transform is exhibited

next.

Example 5.5.6 GAU SSIAN TO GAUSSIAN Sh ow that

I (t) =

_1_ e - t 2 /2 -$

f---+

e - w 2 /2 •

Solution . Since power series converge uniformly within all circles of conver gence, and termwise integration is valid for uniformly convergent series,

j (w)

=

=

5. The Fourier Transform

The integral 8(b))

283

f�oo e -t 2 / 2 tn dt is 0 if n is odd; if n = 2m , then (see Exercise

Thus,

i(w)

,", 00

_ iw) 2n (2n)!n ( L...tn = o (2n) ! n!2

Exercises 5 . 5 Find the Fourier transforms of the functions in Exercises 1 - 5; Ix denotes the characteristic function of the set

X.

(t)

f (t) = e - a 1t l , a > O . 2 . f (t) = (1 - t 2 ) 1[ - 1,1 ] (t) . 3. f (t) = e - a 2 t 2 , a > O . 4. f (t) = e 2it l [ _1 , 1J (t) . 0 < t < 1, 5. f (t) = -1, -1 < t < 0, 0, I t I > 1. 6 . For f E L l (R), express the Fourier transform of f (at - b ) , a, b E R, 1 . THE ABEL KERNEL

{ I,

in terms of

i(w) .

7 . REAL-VALUED FUNCTIONS : REFLECTIONS 1-+ CONJUGATES Show that is real-valued if and only if

f E L l (R)

i { -w) = i (w).

8 . THE GAMMA FUNCTION Like the Fourier transform, the gamma function is defined by an improp er integral dependent upon a parameter: 00 r (x) = x > o.

1

e -t tx- 1 dt,

One reason for interest in the gamma function is that for n E N, r (n + = n! I t provides a continuous extension of the factorial function and a way to compute factorials.

1)

284

5. The Fourier Transform

(a) Show that

(b)

_1_ ] 00 e -t 2 / 2 t 2m dt = 2m r (m + � ) . 2 � - 00 .,fi (Hint: Let y = t 2 12.) Show that r ( m + 1 ) _ .,fi (2m) ! 2" - m ! 4m ' m 2: 1 .

Hence , using (a) ,

1 --

�

(2m) ! . ]-0000 e _ t2/ 2t 2m dt - -m!2m _

r + t) = the fact that r ( t ) = Vi· )

( Hint:

r

(

m

2m2- 1

(

m

-

t) .

Then use induction, and

Answers

2al (w 2 + a 2 ) 3. f (w) = ( Vila) e - w 2 /4a 2 . 4. i (w) = 2 sin (w - 2) I (w - 2) . 1 . i(w) =

5.6

•

The Fourier Transform and Residues

Recall from Definition 4.5.7 some of the variants of the notion of improp er integral, namely and 00 . If I is -00 � 2: integrable , I E Ll b), then I, I exist and are I, equal to I. The method of residues can be helpful in evaluating Fourier transforms as well as some extensions of the Fourier transform of I such e - w I (t) dt instead of I (t) dt . First , we summarize some as basic facts . If I is analytic throughout a neighborhood of except at itself, then is called an isolated singularity off At an isolated singularity, I may be represented by a Laurent series

f:

a b� fa-+b , f� a ' f'::, PVf: , -+b (a, fa f!" a f':: PV f:

I,

PV f: i t

f: e - iwt

Zo

Zo

Zo

" an (z - zo) n + b 1 + b2 2 + . . . , 0 I z - zo l < r, z L..J ( Z - Zo ) (z - zo) n� O <

I( ) =

for some r > 0 , where

I

I

(z) dz and bn 1 (z) n dz 1 an = .1 1 + 1 27rZ (z - zo) - + 1 27rZ (z - zot C

= -.

C

5. The Fourier Transform

285

Zo

for any closed contour C around in the positive (counterclockwise) sense , provided that I is analytic on C and the region (which satisfies some de cent topological conditions, such as no holes) it contains except at The coefficient of the first negative power

Zo .

1

b 1 = Res (f (z) , zo) = -. 2 1n

zo o

f I (z) dz

le

is called the residue of I at The RESIDUE THEOREM states that contour may be evaluated by summing the residues inside the contour integrals (the integral is 0 if there are no singularities inside) and multiplying by 2 71" i.

Ie

RESIDUE THEOREM If C is a closed contour within and on which a function I is analytic except for a finite number of (per force isolated) singular po m ts Z l , . . within C, and R l ' · . · ' Rn are the residues of I at those points , then

. , Zn

Zo

If I is analytic everywhere within and on C, and is any point interior to the region enclosed by C, then we get CAUCHY'S

INTEGRAL FORMULA

1e zJ-(z)Zo dz = (

)

2 71"zl .

(zo) .

The following result also converts the computation of an integral into summing residues. It is a limiting case of the residue theorem taken on semicircles with diameters on the real axis as the radius goes to 00; the value of the function on the curved part is eventually o. If I Ll (R) , then e- i wt PV e- iw t = so the theorem provides a way to calculate i ( w ) .

I (t) dt I�oo

I�oo

J (t) dt,

E

5.6.1 RESIDUE THEOREM FOR THE UPPER AND LOWER HALF PLANES Suppose that I ( ) is (a) analytic in the open upper half plane H = {z : Im z > O} except for a finite number of poles at Z I , . . . , Zn E H, (b) analytic on the real axis R = {z 1m z = O} except for a finite number 01 simple poles at rl , . . . , rm , and (c) f r l m z > 0, II ( ) 1 ::; M/ l z l k when I z l > R o , where M and k > 1 are · constants such that II (z) 1 -+ 0 as I z l in the upper half plane. Then, for w > 0, z

:

o

z

PV

i: eiwt J (t) dt

-+ 00

=

286

5. The Fourier Transform

(z) eizw, ) denotes the residue of I (z) eizw at and pv l-°O00 eiwt I (t) dt = l I(t)e - iwt dt denotes the Cauchy principal value of f� eiw t I (t) (see Definitio n The corresponding result for the lower half plane is also valid, but this time we get some minus signs: For w - 2'11"i "nm (J (z) eizw , ) _'II"i " (J (z) eizw , ) where Res (J

Zk

Zk ,

R

lim

R-+ oo

-r

4· 5. 7).

oo

< 0,

L..t k = 1 Res

Zk

L..t k = 1 Res

Tk .

This technique enables us to evaluate certain Fourier transforms. Example 5.6.2

Show that the Fourier transform of

I I

I(t) = t 2 + a2 ' a > is f(w) = �'II" e -aw . I E L l (R), P V 1 e - iw tl(t) dt = 1 e -iw tl (t) dt = J(w) . : :2 ) ( I z + I(z) ia, I(z) a > I I (z) 1 2/ =I z l 2 I z l > J2a. ;zwia _ e2-;aw , e z w e; (J (z) , ia) z + I z= i a a . 00 ezwtl (t) dt = 2'11"i ( e -aw ) _ e -aw w > 2za a 1- 00 w t2�a2 dt = � � , 00 1 t 2 + a 2 dt = w -w > 00 e-iwt I (t) dt ?!.. e aw. 1- 00 a 1

0,

Sol ution . Note first that

�

so

( z ) = 1/ a2 , 0 . In the upper half plane , the only pole of is at and is analytic for Im z 0, so conditions (a) and (b) of 5 .6 . 1 are satisfied . Moreover, ::; for Since Let

Res

=

•

it follows that

-.-

Since f

tan - 1

0.

holds for this equality also 'II"

= 0:

1

If

< 0, then

�.

- 00

0 , and therefore

=

Finally, since

'II"

for

=

287

5. The Fourier Transform

we have

i (w)

= 1-0000 e - ;wt f (t) dt = �a e - a l w l . - lwt (t) dt. e f a f: e - iwt ( ) dt 5.23. 0

�

.

In the next example we do not compute f (w) , but PV b f Some authors consider PV f t as an "extended Fourier trans form," a topic we return to briefly in Section Example

5.6.3 Show that for a > 0 , PV

1 ":' e- I.Wt

dt+ a2 ) = a2 ( 1 - e -a1w I ) 2 (t t - 00 - 1ri -

P roof. The function

f (z)

1

- z (z 2 + a 2 )

---;--;::-::7"

satisfies the conditions of 5.6 . 1 and has simple poles at Moreover, Res (!

Res Therefore ,

)

±ia and

z

= o.

-

1

( )

21r i � 2"

1ri

Note, too, that

=

e;zw , ia) = z (zei+zwia I z =ia e2-:: ' eizw (! z e i z w , 0 ) = 2 2 z + a I z=o - a 2 · ( _e-aw ) + ( a2 ) o. a (I e-aw )

(z)

while

z

sgn w .

1

1ri

( 5 . 18 )

for w >

-

I: eiwt (t) dt : _ a ( 1 e-aw) . = < 0, (5.18) (5.18) 1-0000 e -iwt (t) dt = a2 (1 _ eaw ) . PV

f

- i

Since f is odd, equation holds for w O. For w can replace w by -w in equation to get PV It follows that PV

I: e - iw t f (t) dt

-w >

1r i

f

- :z

a (1 -

e-a1w l )

sgn w .

0

0 , so

we

288

5. The Fourier Transform

We showed in Example 5.6.2 that I (z ) =

1

z

2 + a2

�

I I

_11"a e -a w = I�(w) , a >

O.

Contrast this with the almost symmetric result of Example 5.6 .4. Note , too, that even though does not have an integrable transform (Example 5 .5 . 3) , does.

e-1 t / e-t U (t)

Example 5.6.4 THE ABEL KERNEL

O. Show that for any

a E R,

e -1 at l

e - 1 a1 t Ue -(t)1 at l U f (t)I a l ++ iw)/ (-t)

�

The A bel kernel is / (t) =

2 1a l a2 + w 2

e- a 1 t l , a >

(t) ( I a l - iw).

= denote the unit step function. By Example 5.5.3, 1 = � 1/ 1/ ( . By 5.5 . 1(a) , / = which will not affect the except at = Now , transform (the functions are equal almost everywhere) , so we can use the linearity of computing transforms to get Sol ution . Let

(-t) t 0,

�

e -1 at l

1-+

e1 a 1 t U (-t)

2 1a l 1 1 al l + iw + l a l - iw = a 2 + w 2

.

0

Exercises 5 . 6 1 . Find the Fourier transforms of

t - �t + 2 · [Ans. i (w) = 1 . [Ans. 1 (w) = 11" (b) / (t) 2 2 ( t 2 + 1) 1 ( c) 1 (t) = t4 + 1 . Find PV f�oo 1 (t) dt for w > 0, where (a) I ( t ) =

2

=

2.

�

1I"e - 1w l - i i w .J _ e-1w l + Iw l ) (cos w

sn )

.J

(1

eiwt

I (t) = } . [A ns. 7ri.J t-1 (b) I ( t ) = t (t 2 + 1) " For a > 0 and any constant b, find the Fourier transforms of cos bt � 11" I + I I) J (a) -- . [Ans. / (w) = - ( I 2a + t 2 2a (a)

3.

e-a w- b e-a w +b

.

289

5. The Fourier Transform

(b)

sin bt . a2 + t 2

4. Show that the function I (t ) = cos� ". t is in and find j. [Hint: To find j, integrate c�:�"':z around the rectangle with base [ - R, R] and upper vertices at ( - R, -R + i) , (R, R + i) . The only pole of the function within the enclosed region is a simple pole at i/2 , and the residue of the function there is

L 1 (R),

Thus, by 5 .6 . 1 , R

e- iw t

L R cosh 1rt

e - iw ( i/ 2 )

ew/ 2

sinh 1r (i/2)

1rZ

1 coshe-iw(1r (RR+iy)+ iy) dy + IR- R coshe -iw(Hi) dt 1r ( t + i)

dt + Io +

10 1

ew/2 e- iw ( - R+iy) . . . dy = 21ricos h 1r ( - R + zy) 1rZ

Io1 I1° ()

go to 0, which implies that and As R -+ 00 , the integrals w/ e eW j(w ) (1 + ) = 2 2 or j w = cOs h W/ ) " ] 2

5.7

(

The Fourier Map

In equation (5 .5) of Section 5 . 1 we considered the finite Fourier mapping

F1 : L 1 (T) I

---+

f----+

Co

LZ) ,

I.

We now investigate its continuous analogue, the FOURIER MAPPING

IE

L 1 (R)

f----+

j (w ) =

I: I(t ) c iw t dt ,

F1 ,

also commonly called the FOURIER TRANSFORM. After selecting an appro priate analog of Co (Z) as codomain , we show in 5 .7.2 that the Fourier map is a linear , continuous map . For I E (T) , lim lnl-+oo j(n) = 0 , by the Riemann-Lebesgue lemma 4.4 . 1 ; the Riemann-Lebesgue lemma also implies that for any I E liml w l -+oo j (w) = 0. We single out this latter property for further consider ation.

L1

L 1 (R),

Definition 5.7.1 FUN CTIONS THAT VANISH AT INFINITY ; COMPACT SUP and Cc If lis a real- or complex-valued function defined on such that lim I (t) = 0,

o (R),

PORT ; C

R

(R)

t -++oo

290

5. The Fourier Transform

ff

then is said to VANISH AT +00, with a similar convention for VANISHES AT -00 . If vanishes at +00 and at -00 , then we say that VANISHES AT 00 . The collection of continuous functions that vanish a t infinity with sup norm is denoted by Co ( and is a Banach space-the subscript 0 denotes that the functions decay to 0 as I � 00 . Functions that vanish at infinity need not decay to 0 rapidly enough to be integrable : Consider = Functions defined on that vanish outside a closed (finite) interval are said to have COMPACT SUPPORT. stands for time and has compact support , then we say that is TIME-LIMITED . For E L 1 if j(w) has compact support, we say that is BAND-LIMITED . Clearly, any function with compact support vanishes at infinity. The subset of Co of functions with compact support is denoted by Cc It happens that 0 Co is the completion of (Cc (Rudin p.

11 11 00

f

R)

It

1/Jt2+I.

(R)

R

f f 1ft

(R) , 11 11 00 )

f (t) (R), f f(t)

( R). 1974, 72).

(R)

We have already noted that the Fourier transform i of any function f E L 1 ( ) vanishes at 00 ; we show in our next result that Fourier transforms i are uniformly continuous and that if two functions are close to each other as members of L1 ( ) then their transforms are close to each other as members of Co This property is useful when we want to approximate a function that is difficult in some way (e.g . , hard to integrate) by one with more manageable properties.

R

(R).

R,

5.7.2 T HE F OURIER

M AP The Fourier map F:

R

L1 ( )

f

is a continuous linear map, and for any tinuous.

f

Proof. By the Riemann-Lebesgue lemma

4.4 . 1, for any

R , i is uniformly con

E L1 ( )

i (w) = I: f(t)e -iwt dt

f

R,

E L1 ( )

(R), we must show that i is con

vanishes at infinity. To see that i E Co tinuous. To that end, for arbitrary E

w , h R, consider

Hence

i (w + h) - i (w) = I: f(t ) e- iw t ( e- iht - 1) l i (w + h) - i (w) 1 ::;

dt . [: I f(t) l l e -iht 1 1 dt . -

w.

Note that the term on the right does not depend on So, if we can show that this limit is 0 as � 0, uniform continuity follows. To complete

h

5. The Fourier Transform

(t) l hn In (t) = f (t) (e- ihn t i: I(t)e- iwt (e-i h n t 1) dt = 1-0000 n [J(t)e - iwt (e -i hnt - 1)] dt = (hn ) h l i (w h) - i (w) 1 = i I f l oo :::; I I f l l w E R, l i(w) 1 i: I f(t)e -iwt I dt = I fl l ·

291

4.4.2

the argument , we use Lebesgue's dominated convergence theorem on passage to the limit under the integral sign. Since 1 1 :::; , the integrand is dominated by an integrable function. For any sequence --+ 0 , 1) --+ O . Therefore ,

2 11

I/(t) l l e - iht

-

-

limn

-

O.

lim

Since

is an arbitrary null sequence, it follows that lim .....

O

+

0,

and is seen to be uniformly continuous. As an integral operator, the linearity of F is obvious . The continuity of F, that F p follows from the observation that for any 0

:::;

5.12.5

We shall see in and injective but not surjective.

5.12 . 6, respectively, that the Fourier map is

Exercises 5 . 7

f, fn L l (R) R, "in - �I oo

1 . UNIFORM CONTINUITY O F THE FOU RI E R MAP F If E and --+ --+ 0 , show that uniformly on i.e . ,

I /(Hint: n f il l -

--+

O.

in i l in (w) - i(w) 1 :::; i: I /n (t)

-

f(t) 1 dt . )

Actually, whenever a lin ear map is continuous, it is uniformly contin uous; this is a special case.

5.8

Convolution on R

By analogy with what we did on the unit circle in Definition 5 .2 . 1 for 9E we define the CON VOLUTION of E Ll to be

I,

L l (T)

T

f, 9 (R) I*g(t) = i: f (t - X)9 (X) dX.

5. The Fourier Transform

292

2

Note that this time we omit the factor 1/ 1r in front of the integral. The existence of the integral is j ustified by the following argument :

I (t - x) 9 (x) i s integrable on f�oo I (t - x) g (x) dx is defined for almost

I t follows b y the Tonelli theorem 4 . 1 9 .2 that by Fubini's theorem 4 . 1 9 . 1 , every Moreover,

R2 j

t.

1: IU

II I gi l l *

<

*

g ) (t) I dt

f�oo / f�oo I (t - x) 9 (x) dX / dt f�oo f�oo I f (t - x) 9 (x) 1 dx dt f�oo I g (x) 1 f�oo I I (t - x) 1 dt dx II g ll 1 11/11 1

It is also routine (change variables) to verify that convolution is commu tative , associative, and distributive . In other words, as was the case for , + , * ) is a commutative Banach algebra.

L 1 (T) (5.2.2), (L 1 (R)

Example 5.S.1 RECTANGULAR PULSE AND SMO OTHING

I L 1 (R) -a, a , I

This example shows that convolution of a function E with a rectangular pulse l a , the characteristic function of [ ] results in an in tegral of that is generally smoother than f. Thus, if has spikes caused by noisy transient phenomena (such as flickering caused by atmospheric dis turbances of relatively short duration in astronomical observations) , these can be eliminated by convolving with a rectangular pulse whose duration would be determined by the phenomenon. For the pulse la , with =

f

I

1: f (t - x)

u t - x,

1a

(x) dx

1: I (t - x) dx t + a I (u) duo 0 It- a

For the unit step function U (which is not integrable on

R), convolution

5. The Fourier Transform

with

f

293

reduces to a simple integral:

f*U(t)

100 f (t - x) dx

=

[too f eu) duo

The most important theorems (so much depends on them) about Fourier transforms are the inversion theorems and the convolutions to products theorem below. 5.8.2 CONVOLUTIONS TO PRODUCTS

For any

Proof.

f,

9 E L 1 (R),

f7g

=

jg.

Since the integrals below are all absolutely convergent , we may freely interchange the order of integration by the Tonelli and Fubini theorems, 4 . 1 9 .2 and 4 . 1 9 . 1 .

[: e -iwt [f * 9 (t)] dt [: e - iwt [I: f(t - x)g ( x) dX] dt = [: g ( x) ([: e-iw t f (t - X ) dt ) dx. f�oo e- iwt f (t x) dt u t - x, [: e -iw(u +x) ! (u) du e -iwx [: e -iwu! (u) du = e - iw x !(w) . -

I n the inner integral

with

=

The original double integral is therefore the product

Exercises 5 . 8

(f * g)

1 . Prove that convolution is commutative , associative , distributive , and = thatfor any scalar and any 9 E L 1 ( l) 9 =

f * (ag) .

a

f,

(R) , a

*

a

294

5. The Fourier Transform

5.9

Inversion, Exp onential Form

nt , ] , ei (n) j L: nE Z f E L 1 [-1I", 11"] t [-11" 11" f(r ) +2 f (t+) = L j( n) eint = �211" L [Lr" I(t) e -int dt] eint n EZ nEZ j(n) 2� J�" I(t) e-nt dt. I E L l (R) f = f j(f = 0 I L l (R)), I E L l (R) II I E L l (R) is 01 bounded variation in some neighborhood [t - s, t + s], s 01 t R, then I +2 f (t+) 2111" 1-00 I(w )eztw dw = oo 2111" l_P I(w) eztw dw . p -+ 00 p S �= ' n J�P p L: -n � l P [ f(w) ] ei tw dw Sp (t) 211" 00 � 211" 1_P [1- 00 I(x) e -iwx dX] eitw dw. I(x) e-iwx E LI (R2 ) 4. 1 9. 1 � 00 I (x) [l P e - i w (x -t) dW] dx Sp 211" 1- 00 oo _p (x - t) x ( �211" ) 2 Lr oo I (x) x -t d sinYpy dy [y = x -t]. -11"1 1-0000 I(t + y) -(y, p) py y 1 y= -11"1 1-0000 I (t - y) --Ypy dY (5. 9) 1 211" ( f ( ) 2<1> (p, -)) (t) .

of a function

By Jordan's theorem 4.6.6, the Fourier series of bounded variation is such that for all E

(pointwise) ,

Next, we present the analogous result where = for Fourier transforms of functions of bounded variation . Note that it implies that if a function of bounded variation has a 0 Fourier transform, then 0 a.e. as a member of in other words, that the Fourier map 1-+ is injective on the class of functions of bounded variation. We extend this to all integrable functions in 5 . 1 2 .5 . 5.9.1 INVERSION F O R BY

> 0,

(r ) . �� = - P V �.--:.-:in this case. For

lim -

.

�

Proof. Instead of a partial sum

gral

E

=

�

.

we consider a "partial" inte

> 0, consider -I

Since , Fubini's theorem the order of integration and write

permits us to reverse

(t)

sin p

For use later , note that since the continuous Fourier kernel sin / (defined to be when 0) is even, the last integral may be written as a convolution: sin 1

-

*

5. The Fourier Transform

295

J�oo into J� oo + Jooo . Then, with a change of variable and s as above , sin py d Sp (t) = -1r1 1 00 (f (t + y) + f (t - y)] Y y sin py dy -1rl iS [f(t + y) + f (t - y)] y sin py + -1r1 1 00 (f (t + y) + f(t - y)] y dy. By the selector property of the Fourier kernel 4.6.5, sin py f (t+) + f (t ) , dy = lim .!. r ( f (t + y) + f (t y)] oo p_ 1r io y 2

Split

0

0

S

-

_

1 [f (t + y) + f (t - y)] --Y dy =

whereas by the Riemann-Leb esgue lemma 4.4 . 1 , 1 00 sin py . hm 1r S

p- oo

5 . 10

-

o.

0

Inversion, Trigonometric Form

f can be recovered from f (t) = �l-> oo [l: f(x) cosw (t - X) dX] dw (5.20)

In this section we develop conditions under which the FOURIER INTEGRAL FORMULA

or its equivalent

f (t) = 1 -> 00 [a (w) coswt + b (w) sinwt] dw,

(5 . 2 1)

(w) = ;:1 1-0000 f(t) cos wt dt and b (w) = ;:1 1-0000 f(t) sin wt dt . (You might want to look back at equations (5. 1 1 ) and ( 5 . 12 ) of Section 5 .4.) The version of the Fourier integral formula in equation (5.21) is especially where

a

useful if f is even or odd. In Section 4.5 we obtained the following criteria for recovery of a function from its Fourier series. 4.5.9 CRITERION FOR CONVERGENCE The Fourier series of the 21r-periodic function 1r] converges at a point if and only if there exists r (0, such that

t fE E Ll1r][-1r, sin ( + 1/2) u d lim .!. r [f (t + u) + f (t _ u)] 1r io u u n

n

296

5. The Fourier Transform

exists; the limit Fourierifseries for 1exists, at t. the limit is the value of the sum of the 4.series 5 .10 ofCONVERGENCE T O A PARTICULAR VALUE The Fourier 1 E L 1 [-11", 11"] converges to c at a pointthet if211"and-periodic only iffunction there exists r E (0, 11"] such that r [/( ) ( ) 2 ] sin( n + 1 j2)u d 1 1 u = 0. Imn 11" t + u + I t - u - c U 1·

0

T

4.some 5 .12rDINI'S TES If for the 211"-periodic function 1 E L1 [-1I", 11"] , E (0, 11"] , and some fixed t, 1 (t + u) + 1 (t - u) - 2c ;:....., --'-..:... u '-...: --'- '--- E L 1 [0 , r] , then the Fourier series for 1 converges to at t. The effectseriesof 4.in5 .terms 9 is toofconvert the integral nth partial sum of a Fourier the Dirichlet kernelforDthe n (t) = L:�= - n eilet to an integral with analog the discrete Fourier kernel

0, sinwu du hm. -11"1 1r [! (t + u) + 1 (t - u)] -U exists. If it does exist, then � 1- 00 [1: 1 (x) cosw (t - x) dX] dw sinwu duo = hm. -11"1 1r [I (t + u) + 1 (t - u)] -U c

w� oo

w _ oo

0

0

5. The Fourier Transform

297

Let r bew a positive number. To prove the theorem, we show that liIIlw--+oo l [1: 1 (x) cos v (t - x) dX] dv . . oo r smwu = lun l [/ (t + u) + / (t - u)] -- du. w--+ 0 u For any 4v,t. 1 9E. 1 R,enables - x) 1 the Iorder / (t) l ; ofsinceintegration 1 E LdR), Fubini' s I / (t)uscosvto (tchange theorem as follows: w 00 1 (x) cos v (t - x) dv dx 00 r 1 I(x) cos v (t - x) dx dv 1- 00 lsinw 10 - 00 1 1 (x) 0 -(t x- x) dx . t - 00 We split the last integral into three: (I t - r + 1t +r 1 00 ) ( sinw � - ) ) - 00 t- r + t +r I (x) t x dx. Since 1 is integrable, so is 1 (x) / (t - x) for x � t + r. We can apply the sinw) (t - x) dx Riemann-Lebesgue lemma 4.4. 1 and conclude that ftc::.r {�si xJ�i�-x dx 0 as o w 00. A similar argument shows that J�: 1 (x) n w Now for the middle integral ftt�: = ftt_ r + ftt +r . With the change of variable u = t - x in fL r ' and u = - (t - x) in ftt+r , we get 1t +r 1 (x) sinwt -(t x- x) dx r [f (t - u) + f (t + u)] sinwu o 0 d u u t- r 10 It is now easy to deduce the continuous analogue of 4.5. 10. Proof.

::s

x

as

�

�

�

00 .

=

Corollary 5. 10.2 CONVERGENCE TO A PARTICULAR VALUE

L 1 (R) , the Fourier integral formula

For f E

converges to if and only if for some r > 0,

sinwu du = o. hm. -1r1 10r [f (t + u) + 1 (t - u) - 2c] -w--+oo U Proof. If 00 � 1--+ [1: 1 (x) cosw ( t - x) dX] dw then by 5 . 1 0 . 1 there exists a positive number r such. that smwu du. . r [f (t + u) + /(t - u)] -C1r = lun w--+oo u l c

= c ,

0

(5 . 22)

�

298

5. The Fourier Transform

hm. l r 2c s. Uwu du = C1r.

By 4 . 5 . 8 (b), Therefore,

W -+ OO

0

m --

hm. lr [/ (t + u) + /(t - u) - 2c] �nwuu du = 0.

W -+ OO

--

0

Conversely, if equation (5. 22) holds, then r [! (t + u) + I (t - u)] -sinwu du C7r = hIllw u . -+oo l = � l -+oo [l: /( x )cosw (t - X ) dX] dw by 5.10 . l . 0 A condition that suffices Iffor theE L1recovery of I (t) from the Fourier integral formula is differentiability. (R) is differentiable at t, and C = I (t), I then l (t + u) + / (tu - u) - 2/(t) = I(t + u)u - /(t) + I(t - u)u - / (t) isEquation a bounded-hence integrable-function of in [0 , r] forlemma some4.4.r >l . O . (5. 2 2) then follows from the Riemann-Lebesgue Finally, there is the continuous analogue of Dini's integrability test. o

u

ANALO GUE OF DIN I 'S THEOREM For ICorollary E L dR) ,5il. 10.3 ICONTINUOUS (t + u) + I (t - u) - 2c E L1 [0 , r] (5. 23) u lor some r > 0, then the Fourier integral lormula

c. (The most interesting case is where c = I(t).) Proof. By the Riemann-Lebesgue lemma 4.4. 1 , equation (5. 22) holds, and the result follows from 5.10. 2 . 0 As notedderivative in 4.5 .13in( Lipschitz' s testabout ) condition (5. 2 3) is satisfied if I has a bounded some interval t or, more generally, if I satisfies a Lipschitz condition of order I I (t + u) - I (t) 1 b l u l a for l u i r, where b, and r are positive constants; if so, then I (t + u) + I (t - u) - 21 (t) E L [0 , r] . By Dini 's theorem 5.10.3, this yields the following corollary. converges to

a,

a,

::;

"-'----'--=--.:'-----'---'---'--'u

<

1

5. The Fourier Transform

299

IE

Corollary 5 . 1 0 .4 CONTINUOUS ANALO GUE OF LIPSCHITZ 'S TEST If Ll satisfies a Lipschitz condition at t, then the Fourier integral formula

(R)

- 00 [I: f x cosw (t - x dX] dw 1 � converges to f (t). We have already proved a continuous analogue of Jordan 's theorem 4. 6 .6 in namely, if I E (R) BV [a, b], then at any t E (a, b), f (r ) +2 f(t+) �211" pV1°O_ j(w) eitw dw. 00of the continuous analogue of Here is the Fourier integral formula version Jordan' s theorem. Once again, the key ingredients are the boundedness of : sinuwu du on any interval [a, b], the Riemann-Lebesgue lemma, and the J to make an integral small by restricting the range of integration. ability ANALOGUE OF JORDAN'S THEOREM For I E (R) BV [a, b], b ::; and t E (a, b), I (r ) +2 I (t+) !11" r-oo [1 00 f cosw (t dX] dw . 10 - 00 Our goal is to show that ! r [I(t + U) +f (t _ U) _ 2 ( f (t - ) +f (t+) )] si n wu du = O lim 2 u w _ oo 11" 10 for some 0; the result then follows from 5. 1 0.2 with ( ) I (t+) . c f r +2 ( )

Ll

5.9. 1 ,

)

n

=

::;

Ll

5.10.5

-

00

a

<

00,

(x)

=

-

x

n

)

Proof.

r>

=

Since I is of bounded variation, we write ReI and Im f as the difference ofa time, two increasing functions byincreasing. 4. 3 . 6 ; by linearity, weE (a,mayb) and consider one at so suppose that I is Choose t r > 0 such r, t + r) (a, b). Let w(t,u) = f(t+ u) - I(t+). For u E [O , r] , nonnegative function of u. Consider wthatis an(t -increasing, 1 r w (t , u) sin;u du 1r [f (t + u) 2 ( f �+) )] sinuwu du o For ><0,t/1I"choose a::;positive number r' r such that I w (t, u) 1 = f (t + u) for 0 u ::; r' . By the second mean value theorem 4. 6 . 4 there I(t+) is some c E (0, r') such that l r l w (t, u) -sin w u du = w (t, r') lr l sin wu du o can

C

_

=

<

{

o

u

U

--

c

300

5. The Fourier Transform

1 1r' s ; l and any R by . 1 r ' smwu in u

Since for any r' > c � 0 ,

wE

du <

�

4.5 .2(b) ,

w (t , r')

it follows t�at

-- du < U

c

f.

In other words, I I;' w (t , u) si nuwu du l < f . The integral I; w (t , u ) si nuw u du follows as w; w uby) si nthewu Riemann-Lebesgue Nowlemma let 4.4.(t, u)1 . =It ftherefore that I u du 0 as w ( t - ) - f ( t - u) . ( t, A similar argument shows that sinwu du 0 as w lr ( t , u) -and the proof is complete. 0 The function < t 1 / �, f (t) = { t sin �, 0, t ::; 0 or t > 1 / �, satisfies the condition (5.23) of 5 . 10.3 at t = with c = f (0) : �2 ] ' f (0 + u) + f (0 - u) - 2f (0) _ u sin ( l/u) E L 1 [0 ' u u but is not of bounded variation in any neighborhood of O . Thus, 5 . 10.5 is not applicable to f, but we can still recover f ( 0) by 5 . 10 .3 . Likewise there exist functions of bounded variation that do not satisfy the integrability condition of 5 . 10 .3 ( see Exercise 1). Recall theorem on uniform convergence: o

-+

-+ 00

-+

o

v

-+ 00 .

v

-+

U

°

-+ 00 ,

::;

°

4.7. 6

4 . 7 . 6 UNIFORM CONVERGENCE FOR CONTINUOUS FUN CTIONS fE d n b] [a, b] C [ - , ] f b) f [ + r, b - r] , r > 0, [ b].

For be extended to � � let. on alLl of- �,R.�lIf BVis [a,continuous athen2�-periodic function on (a, Fourier series for f converges uniformly closedthesubinterval of a , to on every a Though we dotonottheprove it, the ofcontinuous analogue of 4.7.6 is also true ifa,inb)jaddition hypothesis 5 . 10.5 , we take f to be continuous on in that case, (

f ( t) =

� 1- 00 [1:

]

f (x) cos w (t - x) dX dw ,

5. The Fourier Transform

30 1

and � f (x) cos w t x) dX dw converges uniformly as w -+ 00 on every closed subinterval r :S t :S + r, r > 0, of We now look at some applications of these results. This first example yields another way to show that dw = /2.

fo- oo [J�oo

( a

]

-

-

a

[a, b).

fo- oo si� w

5.10.6

1f

RECTANGULAR PULSE Use the Fourier integral formula representation of equation (5. 21) of the function ( t ) to show that

Example

1o -00 sin wwcos wt dw _

-

1[- 1 , 1]

{ 1f

1f / 2, I t I < 1 , /4 , t = ± 1 , , .

ItI > 1 .

0,

Solution . In the notation of equation ( 5.21 ) ,

w ( ) 1f2 1 1 cos wt dt = 2 sin 1fW ,

a w

=

-

--

and

0

b (w) = 1f1 1

1 sm. wt dt = 0 . -1

-

1[- 1 , 1] is of bounded variation on R. By 5 . 10 .5 , 1[- 1 , 1] (t- ) + 1[-1 , 1] ( t + ) = � ( - OO sin w cos wt dw ;

The integrable function therefore , at any point t ,

2

1f h

w

in other words,

1f /2, I t I < 1, 1f/4 , t = ±1,

0,

The special case t = 0 yields

5.10.7

fa- oo si �w dw = 1f/2.

1[0, 00 )

I t I > 1. 0

Let U = denote the unit step function. Use the Fourier integral formula representation of equation (5. 21) of the functzon f ( t ) e- t U ( t ) to show that

Example =

{_ OO w sin wt + cos wt dw = 1 + w2 io

{

t < 0, t = 0, t > O.

Solution . We can evaluate

a (w ) = 1f1 10 00 e -t cos wt dt by integration by parts ( or by recognizing that a ( w ) is the Laplace trans form of cos t evaluated at s = 1 ) . This yields a ( w) = � 1 lw 2 . Similarly, + b (w ) = � 1 ':w 2 • Therefore, by 5 . 10.5 , -

1f / 2, 1o -00 w sin 1wt++w 2cos wt dW = { O�e-t " , ,

t < 0, t = 0, t > 0.

302

t

5. The Fourier Transform

fo- oo

dwoo = 7r/2. Indeed, since 1 / (1 + w 2 ) is fo- = fooo , and we get the well known

If = it follows that 1 ;w 2 absolutely convergent , we have 1 0 result that = 7r 1 +W 2

0,

fooo

dw

12.

Exercises 5 . 1 0

1.

f (t) =

Inl� /t) ' ( et) 2 '

0 < t � lie ,

t > li e ,

f (t) f (-t) Show that ( a) f E L 1 (R). ( b) f is of bounded variation in a neighborhood of 0 ( so it satisfies the condition of 5.10.5 at 0) . ( c) f does not satisfy condition (5.23). (Hint: f is bounded and monotone .) ( d) f behaves like II [- In t] near the origin. ( a) Find the Fourier integral representation of f(t) = e - k 1 t l , k > O. Use the result t o show that

with

2.

{

B V BUT NOT DINI Consider the function

=

.

100 k2

ut du = 7r e - k t for t > O. o u2 2k e - k t for 2: 0, and take .f (-t) - f (t). Find the +

cos

f (t)

t

= ( b) Define Fourier integral representation of

3.

1-o 00 ku2

+

sin

f, and use it to show that =

ut 7r k u 2 du = "2e - t for t > O.

< f (t) = { -21, ' 0-1<
Find the Fourier integral representation of

Use this to show again that

fo-oo SI� U du = t.

4. FOURIER COSINE AND SINE INTEGRALS Suppose that

and that f satisfies the conditions of Jordan's theorem

f E L 1 [0 , 00 )

5.10.5, namely

:::; a that f E L l (R) n BV [ b] , for t > O. Show that for any t E ( -b,

5. The Fourier Transform :::;

-00 < b Define f ( - t ) ) ( b) , 00 00 f (r + f ( t+) "'--'. --'-)-"'--'--'- = -2 1-+ 1 f ( x) cos wt cos w x dx dw . a,

U

-a

a,

=

00 .

303 f (t)

7r This is called the COSINE INTEGRAL REPRESENTATION of f . The SINE INTEGRAL REPRESENTATION is obtained similarly. If f is extended as an odd function, then show that for any t E (-b, - a ) U ( a , b) 2

f ( t- ) + f ( t+ )

=

2

5. Show that

a

a

00 00 2 [-+ [

:;

10

10

!

_ 00 sin w cos wt [ dw w 7r 10

=

,

f ( x) sin wt sin wx dx dw .

{ �i 0,

0 :::; t < 1, 2, t 1 , =

t > 1.

6 . Find the Fourier sine integral representation of f ( t)

=

{ 01 ,,

0 :::; t :::; 1 , t > 1.

e- t cos t for t > O. Use the Fourier sine integral represen tation to show that - 00 w3 sin wt dw = 7r e - t cos t for t > O. w +4 2

7 . Let f ( t) =

1 a

-

4

8. Solve the integral equation fooo f (w) cos wt dw = e -t for t > O.

Hints

1 (0,1 ) of ( O, 1 ) , and use the Fourier cosine integral representation of Exercise 4 .

5. Consider the characteristic function 8 . Use Exercise 2 ( a) .

5.11

(C, 1 ) Sumrnability for Integrals

Let Si = E � = o X k , i � 0, denote the ith partial sum of the series E:= o Xn of real or complex numbers. Recall from Definition 4. 14. 1 that (C, 1) summa bility, E :=o X n = x (C, 1), means that (Tn

=

n

n

1 -L: S = L: + 1 i = O i ;�n n

(

. )

1 - _z_ n+1

X

i

�

x.

304

5. The Fourier Transform

:2:::'= - 00 Xn = :2:n eZ Xn, w e use symmetric partial sums Si = :2:� = -i X k , i � 0. We say that :2:::'= - 00 Xn = x (G, 1) if n n Un = n +1 1 L Si = L ( 1 - n I+i i 1 ) X i --+ X. i = O i = -n

For biseries

--

--

For integrals, we have the following analogue.

Definition 5 . 1 1 . 1 (G, 1) SUMMABILITY FOR INTEGRALS

For I E Ll PV

[-r, r] for all r > 0, we say that i: I (t) dt = S

i: ( 1:1 ) I (t) dt 1-

=

(G, 1) if

r�� J:r ( 1 - 1:1 ) I(t) dt = s .

0

Some elementary observations about (G, 1) summability of integrals fol low. Part (a) appears unusual at first because we convert a single integral into a double integral. 5 . 1 1 . 2 ELEMENTARY PROPERTIES OF (G, l)-SUMMABLE INTEGRALS (a) ALTERNATIVE CHARACTERIZATION II E L l lor all 0,

then

00 1- 00 I (t) dt = S (G, 1)

I

-¢=::>

l r j u I(t) dt du = s . r-+oo r 1 -u lim

-

(G, I ) .

Proof.

= s,

then

r>

i: I (t) dt

(a) We show that the "partial integrals" are equal:

! r r 10

l-uu I(t) dt du = j-rr ( 1 - IHr ) I(t) dt.

Since I is absolutely integrable on Fubini's theorem 4. 1 9 . 1 ,

! r r 10

0

i: I (t) dt

(b) REGULARITY Ifl E L l (R) and P V s

[-r, r]

It =t =u-u I(t) dt du

[-r, r], it is integrable on [0 , r], so by

=

5 . The Fourier Transform

(b ) Let f E L l (R). For r > 0, let tion of [ -r, r] , and define

305

I[ - r , r] denote the characteristic func

fr (t) = ( 1 - -;:I t l ) f(t) I[ ;- r, r] ' I f (t)l, and limr -+ oo Ir (t) = f (t) for all t E R. Since

I fr (t) 1 � f E L l (R), the following exchange of limit and integral is permissible by

Clearly,

the dominated convergence theorem 4.4.2:

Since it follows that

r�� I: fr (t) dt = I: f (t) dt = s . r I: fr (t) dt = I r ( 1 - 1:1 ) f(t) dt , r ) j lim r --+ oo - r ( 1 _ l!lr f(t) dt = s . 0

Exercises 5 . 1 1

1 . (C, 1) SUMMABILITY IS LINEAR Suppose J�oo f (t) dt = p (C, 1 ) . Show that:

q

2.

J�oo 9 (t) dt (C, 1).

(C, 1),

J�oo [I (t) + 9 (t)] dt = p + ( b ) For any scalar a, J�oo af (t) dt = ap (C, 1) . oo (C, 1) SUMMABILITY FOR Jo If I E L l (0, r ) for all r > 0, then we oo define Jo f (t) dt = s (C, 1) if ( a) If

= q

then

) rlim -+ oo Jro ( 1 - :r f (t) dt = s. Show that Jooo f (t) dt = s (C, 1) if and only if r � l [lU f (t) dt] du -+ as r -+ 00 . S

Hint: Use Fubini's theorem 4.1 9 . 1 to write

306

5.12

5. The Fourier Transform

The Fejer-Lebesgue Inversion Theorem

We will prove the continuous analogue of Lebesgue's pointwise convergence theorem 5.3 . 1 for the Fourier transform in 5.12.3, namely that for I E LI (R) and j ew) = I(t) e - iw t dt,

i:

I (t) = 211r 1 eiw t j(w) dw (e , l ) a.e., by showing that at any Lebesgue point t of I, r limr oo : i ( 1 - I: ') e iw t j(w) dw I (t) 1r r PV � 1 00 ( 1 - ti ) e i w t j ew) dw. 2 1r r The Fejer kernel Fn , n � 0, played a key role in the proof of Lebesgue's 00

- 00

....

- 00

pointwise convergence theorem. Its continuous analogue Kr , r > 0, is the key to the version (5. 12.3) for the Fourier transform. The Continuous Fejer Kernel

Lebesgue's pointwise convergence theorem (4. 18 .5) proclaims that at any Lebesgue point t of I E L1 [- 1r, 1r] , the Fourier series for I is (e, l) summable t o I (t) , i.e . , the nth ( e , 1) sum Un (t) of the Fourier series for I satisfies 1 I (t - x) Fn (x) dx n (t ) 21r (5.24) (f * Fn) (t ) � I (t) as n where the continuous function sin2 (n + 1) t/2 t :/; 0, , n � 0, Fn ( t) = (n + 1) sin 2 t/2 ' n + 1, 0, denotes the nth Fejer kernel. We will show that the continuous Fejer kernel sin2 rt/2 ° t , r > 0, 2 :/; ' Kr (t) = r (t/2) 0, r, functions as an analogue of the nth Fejer kernel. Rather than a partial sum Un ( t) , we show (5. 12.1) that the "partial" (e, 1) integral satisfies

1 1(

u

- 1(

� 00 ,

{

{

Sr (t)

t

t

=

=

(5 .25)

5. The Fourier Transform

30

and that (5.12.3) at any Lebesgue point t of I, Sr (t) I (t) r -+ We begin by establishing equation (5. 2 5). 5.12.1 WAYS TO WRITE THE PARTIAL (C, 1) INTEGRAL L e t I E L 1 (R) For r > 0 and J(r (t) = S!(:;;fl for t i 0, Kr (0) = r, at any t E R, Sr (t) �211" l-rr ( 1 - tir ) eiwt i{w) dw 2 � 1 °O I(x) r [ sinr(X - t) /2 ] dx 2111" - 00 r(x - t) /2 211" ( f J(r ) ( t) . Proof. Rewrite S r (t) by substituting for i{w) in equation (5. 2 5) to get -+

as

00 .

*

Since ftheorem E L l (R) , the iterated integral above exists, and we may use the Tonelli 4.19.2 to interchange the order of integration to get Sr (t) = �211" 1-00 I (x) [l-rr ( 1 - tir ) e - iw (x -t ) dW] dx. 00 We show next that the inner integral satisfies 2 (x lr ( 1 ti) e -iw ( x-t ) dw = r [ sin r - t) /2 ] r (x - t) /2 -r r Realize that J� r ( 1 - ¥) e - iw(x -t ) dw is the Fourier transform of the hat function k ( t/r) = (1 - I t I /r) 1 [- 1 , 1 ] (t/r) = ( 1 - It I /r) 1 [- r,r] (t) byevaluated at x - t: By Example 5.5 .5 the Fourier transform2 of k (t) is given sin 2) k (t) = (1 - It I) 1 [- 1 , 1] (t) is ( �/t ) By the scaling theorem 5.5 .1(a), for r > 0, -( r sm rW/2 ] 2 k : ) (w) = rk (rw) = [ (5. 2 6) rw/2 r _

•

308

5 . The Fourier Transform

Therefore, k

-(!.)r (x

Hence

1-rr ( 1 1:1r ) e - iw (x - t) dw = r [ Sinr (xr (x--t)t)/2/2 ] 2 (5.27) Sr ( t ) = �21r 10000 ( x) r [ sinr (rx(x--t )t)/2/2 ] 2 dx .

_

t) =

_

-

1

[ Si�[}�;)W2 ] 2 is an even function of x - t, equation (5 .27) is the . of 1 wIth. r [ Sirnxr/2x/ 2 ] 2 : convolutIOn

Since

Sr (t)

in r ] 2 ) [ (t) (I (x) r S X/2 2 1r (rx/2)

�

1 (f * 21r

Kr

*

(5.28)

) (t) . 0

Compare equation (5.28) to the expression for the nth (C, 1) sum of equa tion (5.24) :

Because of this analogy, we define the CONTINU OUS rTH FEJER K ERNEL

Kr (t) for r > 0 to be r at t = 0, and for t =j:. 0 (since I - cos rt = 2 sin2 rt/2) , L\. r (t) = r�k ( rt ) = r [ sin(rtr/2)t /2 ] 2 = 2 ( I -rt2cos rt) ' (5 .29) T/

As the following figures show, the continuous Fejer kernel Kn + 1 is quite similar in appearance to the discrete Fejer kernel Fn .

2

-4

-2

2

t

4

The Continuous Fej er Kernel KlO

(t)

5. The Fourier Transform

-4

-2

The Discrete Fej er Kernel Fg

309

(t)

Compare the following result about the discrete Fejer kernel corresponding result 5 .12.2 for the continuous Fejer kernel Kr .

Fn to the

4.15.2 THE DISCRETE FEJ:ER KE RNEL The Fejer kernels 1 and

Fo (t)

=

{ _(n+1 _1) ( sinsm(�+t/21)t/2 ) 2 ' t fJ.

2 7r Z , E 27rZ,

n � 1, t have the following properties for every n E N: (a) Fn (t) is an even 27r-periodic function; (b) Fn (t) � 0 for all real t , and Fn (t) 0 uniformly outside all closed intervals [ - r, r] for all r > 0; (c) I�1f Fn (t) dt = 27r. Fn (t)

=

n + 1,

�

5.12.2 THE CONTINUOUS FEJER KERNEL

Let

in equation (5. 29). For all r > 0, (a) 1{r (t) is an even function of t; (b) 0 ::; 1{r (t) ::; 4/rt 2 for all real t, so Kr (t) closed intervals [-b, b] , for all b > 0; (c) 1{r E L 1 (R) and I�oo Kr (t) dt = 27r.

[{r (t) = �

2(1�� r t ) be as

0 uniformly outside all

The result of ( a) is clear , while that of (b) follows from the fact that Isinl ::; l . (c) Since 1{r is continuous, and bounded on [0, 1], it is integrabl� there . Since 1{r (t) ::; 4 / rt 2 for all real t, it follows that Kr is integrable on [ 1 , 00 ) . Therefore, Kr is integrable on [0, 00), and 1000 Kr (t) 10-+00 Kr (t) dt. Since I{r (t) is even, I� b Kr (t) dt = 2 f; Kr (t) dt, so it suffices to consider Proof.

=

310

5. The Fourier Transform w

PV

J{r

limb-+(letoo 2 J;=/{rt)r (t) dt to compute J�oo (t) dt = J�oo Kr (t) dt. Note that [ 00 /{r (t) dt = 2 [00 1 - cos rt dt = 2 [ 00 1 - cos 10 w 2 10 rt 2 10oo -+ si�w Since J - 2 dw) yields = � by 4. 5 . 8(b), an integration by parts (with dv = cos - 1 / 00 + [-+ 00 sin d [ 00 1 - c�s d 10 sin 0 W 1=0 -7r . 0 1 -+ 00 -2 Beforetheorem proving5 . 12.3, the continuous analogue4.18.1) of Lebesgue' s pointwise conver gence recall (Definition that t is a Lebesgue point of f if [ h I f(t + x) - f (t) 1 dx = 0, I lim 10 h-+ O halmost and (Section 4.18) that every point of the domain of an integrable function is a Lebesgue point; every poin t of the domain of a continuous func tionn t.is a Lebesgue point, and no point of jump discontinuity is a Lebesgue poi 5.12.3 FEJER-LEBESGUE INVERSION THEOREM At any Lebesgue point t of any f E Ll (R) , (a) 1 00 eiwt !(w ) dw (C, I) � f (t) 27r - 00 r limr -+ oo �27r l- r ( 1 - tir ) eiwt !( w) dw 1 00 . P V- 1 e z wt f (w ) dw 27r - 00 limr-+oo 2� (f Kr) (t) . (b) UNIFORM CONVERGENCE FOR CONTINUOUS f If f is continu ous in a closed interval [-b, b] conta ining t, then r � l ( 1 - ti ) i (w) dw = f (t) lim r -+ oo 27r r w

o

dw

w

w

w

w

w

o

w

w

w

d

w

w

dw .

w

�

=

*

uniformly in [- b, b] .

-r

e;wt

Proof. The proof of (b) involves a slight modification of that of (a), so we prove (a) and give a hint to (b) in Exercise 3. Let t be a Lebesgue point of f. Byonlyequations (5.27) and (5. 2 9), since convolution is commutative, Sr (t) = -1 1-00 f (t - x) 1 -rxcos2 rx dx. 00 7r

5 . The Fourier Transform

Since

311

l -rc",o� r '" is an even function, this is the same (t) = -1 1-00 f (t + x) 1 rxcos rx dx. 00 as

-

Sr

7r

2

Adding these two equations, we obtain

2sr (t) = -7r1 1-00 [f (t + x) + / (t - x)] 1 rxcos rx dx. 00 Since the integrand is an even function of we have f�oo = 2 fooo , so sr (t) = -7r1 10 00 [f (t + x) + / (t - x)] 1 - cos rx dx. By 5.12.2(c), sr (t) - f (t) = -7r1 1 00 [/ (t + x) + / (t - x) - 2/ (t)] 1 -rxcos rx dx. (5.30) oo Our goal is to show that r (t) I (t) 0 as r -+ Split fo of equation (5.30) into f; + fboo . As we show next, f; is small because t is a Lebesgue point; the second integral is small because [{r -+ 0 uniformly outside [-b, b] (5.12.2(b)). Let Af (t, x) = l f (t + x) + / (t - x) - 2 / (t) 1 and h (x) = 1'" A / (t, y) dy. 2

-

x,

n

2

2

0

S

-

-+

00 .

Since t is a Lebesgue point of I,

h (x) = lim .!. f'" AI (t, y) dy = O. X ", .... 0 x Jo Hence , for arbitrary f > 0, we may choose b > 0 such that 0 � x � b h (x) � fX . For positive r such that l/r < b, ! f b A I (t, x) 1 - c� rx dx < ! f b A / (t, x) 1 - c � rx dx rx rx 7r Jo Jo r ! f 1 / AI (t, x) 1 co; rx dx rx 7r Jo b 1 - cos rx 1 + l r A / (t, x) rx dx. / It follows from the Taylor series expansion for cos rx that (5.31) ", .... 0

lim

1r

:::::}

-

2

312

5. The Fourier Transform

Therefore,

1 _ 7r

11 /r A I (t , x) 1 - C0S rx dx 0

rx

L.l.

::;

Since 1 - cos rx

2

<

1 1/r A I (t, x ) dx = r h ( 1 / r) 2 7r ;2 1r

0

r 1 < -� 27r r

L.l.

� . =27r

2,

l1/r

.!:. [ b .6./ ( t ) I - cos rx d � b .6. / (t , x) dx. X ::; 'X rx2 rx2 7r 7r

11 /r

Integrating by parts, with dv = .6.1 (t , x) dx, we have

Consequently, since 0 ::; h (x) ::; �x , and rb > 1 , 1 b A I ( t , x ) 1 - cos rx dX < -2 -� + -4� - 7rr b 7r rx 2 7r -

1 1/ r

1b dx 2€ 4€ 6€ rx2 1/r < -7r + -7r - -7r .

L.l.

Therefore

lb .!:. 1 1/r + .!:. lb 7r 0 7r 1/r

.!:. 7r 0

<

_

�

�

-< 27r + 27r

I!..

= 2 7r .

As for It , since

I!7r 1[b 00 [/ (t + x)

+ I (t

and

it follows that

_

x)

_

'>0 1 cos r x d � [ .6./ (t , x) dx 21 (t)] x ::; rx 2 x2 7rr 1b

l

.6./ (t, x) E ---"-� x2 �

L 1 (b , 00 ) ,

� [ 00 .6.1 (t, x) d lim x = o. 0 ..... oo 7rr 1b x2

r

5.12.4 INVERSION WHEN 1 E (R) If I and 1 are integrable on R, then at any Lebesgue point t of I, in particular at any point t of continuity of I, 1 e·w t dw. I (t) = 2 7r 00 f (w)

L1

J

OO

-

�

.

5. The Fourier Transform

313

The integrability of j implies that I: j(w) eiwt dt = s istherefore finite. By the regularity of (C, 1) summability (5.11. 2 (b)), and 5.12.3, f (t) limr ....00 2� J:r ( 1 - I� I) eiwt j(w) dw 1 211" 1- 00 ezwt f (w) dw. 0 Now we can easily demonstrate the following result. Proof.

oo

•

�

Th e Fourier m ap F : L l (R) -- Co LR) , f f,

5.12.5 INJECTIVITY OF THE FOURIER MAP

of 5. 7. 2 is injective. Proof.

1--+

By 5.12. 3 ,

f (t) = 2� I: eiw t j(w) dw (C, l ) a . e . , so if j (w) = 0 , then f (t) = 0 a. e . Since the Fourier map is linear, this . implies that if j(w) = g (w) a. e . then f = 9 a. e . or that the Fourier map F is injective. 0

Supposegivenwe want to solve an integral equation of the convolution type, namely, g , k E Ll (R), find f E L l (R) such that I: k (t - x) f (x) dx = 9 (t) for all real t . By the convolutions-to-products theorem 5.8 .2, (w) . k (w ) 1 (w) = 9 (w) or j (w) = � k (w) exists hmap, E L l (R) such that h (w) = 11k (w), then, by the injectivity ofIf there the Fourier j (w) = It (w) g (w) = ';;-g (w) , so f (t) = I: h (t - x) 9 (x) dx. OfUsing course,thethebounded probleminverse is thattheorem, in manywecases, h does not exist. proved after 5.1.5 that the finite Fourier map Fl : L l (T) -+ Co (Z) is not onto. The same holds for the Fourier maD F.

314

5. The Fourier Transform

5.12.6 THE FOURIER MAP F : L1 ( R)

---+

Co ( R) Is NOT ONTO .

Proof. We exhibit a g E Co (R) that is not the Fourier transform of any

I E L 1 (R) . First , note that I is odd if and only if j is odd : By the definition of j, if I is odd, then j is odd; by 5 . 1 2.3, if j is odd, then I is odd. Let e denote the base of the natural logarithm. We will show that if f E L l ( R )

fe a

is odd , then

j ew ) W

dw < 2 '11" 1 1 / 1 1 1 for any

a > e,

E

then produce an odd function 9 Co ( R) that does not satisfy this inequal ity to conclude that 9 cannot be the Fourier transform of any ( R) . First note that if I E ( R ) is odd, then

I E L1 L1 OO I(t) sinwt dt = -2i (XJ I(t ) sin wt dt . j ew ) _i Jo -00 For any a > e , j ew ) /w is continuous on. fe , a], so we can invoke Fubini's

j

=

theorem 4 . 1 9 . 1 to interchange the order of integration as follows:

00 -2i le a �1 1n[0 I(t) sinwt dt dw oo -2i 10[ I(t) Je[ a sinwwt dw dt .

r j cw) dw

1e

w

t

With u = w , this becomes

By 4.5.2(b) , for all 0

for all

::;

00 sin u duo - 2i 1o I(t) dt le a t -

t

c < d, I t si: u du l <

fe a

U

'11"

,

so

j cw ) dw < 2 '11" 1 1 / 1 1 1 w

(5 .32)

{ 11 ,

a > e . Now consider the odd, continuous function w/e ,

9 (w ) Since

limw_ ± oo 9 (w )

=

0, 9 E Co (R) . However, for all a > e ,

r g (w )

Je

w

=

0 ::; w ::; e , w > e, nw - g ( - w) , w < o .

dw

r dw

1Ine ( lnw wIn) wl � a

I n ( In )

---+ 00

as

a

---+ 00 .

5. The Fourier Transform

It follows from inequality

(5. 32) that

9 :/;

o

i f r any f E Ll (R) .

315

0

Evaluation of Integrals

5 . 1 2.4

We can use to evaluate some rather difficult integrals. For example, let f (t) be the hat function

(t) = (1 - It I ) By Example 5 . 5 . 5 , its transform is given by sin (W/2) ) 2 , k (w ) ( w /2 which is integrable because it is continuous on [- 1, 1] and bounded by the integrable function 4/ 2 on 1]. Since k is continuous everywhere, every point t is a Lebesgue point of k. Therefore, by 5. 1 2 . 4 , for all 1211" 100 ( sinw(/w2/2) ) 2 dw ( I - It I ) 00 k

1[- 1 , 1 ] .

=

R\ [- l ,

w

For t

t,

e

-

iw t

=

1[- 1 , 1 ] .

= 0, it follows that

100 ( sinW(w/2/2) ) 2 dw - 211". - 00 / 2 in this last integral yields 100 sin--22 -y dy = 11". - 00 Y -

Substituting y = w

Exercises 5 . 1 2

1. If t is a Lebesgue point of f l then show that 00 (x) cos u - x) dX] du = f (t) lim ! Jr (1 - �) [ 1 r 11" o ( cf. equation (5. 20) of Section 5. 1 0). [Hint: ; J; ( I - �) [J�oo f(x) cosu(t -x) dx] du � I: [lr (1 - ;) [cosu(t - x)] dU] dx. E L ( R) ,

.....

oo

r

f (x )

Integration by parts yields

_

f

oo

(t

316

5. The Fourier Transform

.; g ( 1

- *) [f�oo I (x) cos u (t - x) dX] du � 1 00 ( 1 - cos (t � X » ) d I ( x) 1r - 00 1'(t - x) x, which is just S r (t) by equation (5.27) ; Sr (t) ---* I (t) as Fejer- Lebesgue inversion theorem 5.12.3(a) .] l'

l' � 00

by the

2. EXTENDED FEJER-LEBESGUE THEOREM The Fejer-Lebesgue in version theorem 5.12.3(a) says that at any Lebesgue point of E L1 (R) , 1 1 - cos as 1r Prove the following extension: Instead of requiring that E L 1 (R) , show that if L1 for all > 0 and I{�:�I < 00 , then at any Lebesgue point t, 1 1 - cos d ) as 1' � oo. 1r - 00 )

t I

Sr (t) - 1-0000 I (x) 1'(t (tx)-2 x) dx f (t) I d I E (- 1', 1') f�oo x sr (t) = - 100 I(x) 1' (t - (tx -2 x) x ---* f (t [Hint: Let Ie (x) = l[ - e, e] (x) I (x) , c > 0, and let c be positive. It is easy to see that l ie (x)1 11 xc22 I f (x) 1 for all x E R. Since f�oo I{�:�I dx < this implies that Ie E L1 (R) . Hence, by 5 . 12.3(a) , �1r 100- Ie (x) 1 -1'cos(t - (tx)� x) dx fe (t) a.e . as 00. 00 Thus, for It I c, 1 e I(x) 1 - cos 1' (t - x) dx = f (t ) a.e ., � lim r --+oo 1r - e (t x) 2 and f (x)12 dx 0 r I(x) 1 - cos 1'(t � x) dx � r I 1' }l x l > e (t - X) 1'(t - x) }Ix l > c as ---* since the integral is finite. Hence, for I t I c, -1r1 1-0000 I (x) 1 -1'(tcos- (tx)-2 x) dx � f (t) a.e. as � l'

=

�

l' ---* 00 .

l'

l'

::;

+

+

00,

::;

l'

�

l'

l' ---*

_

::;

l'

�

::;

00

l'

Since c > 0 is arbitrary, this completes the proof.]

l'

00 .

5. The Fourier Transform

317

3. UNIFORM CONVERGENCE F O R CONTINU OUS I Prove 5 . 12.3(b) . Show that if I E Ll (R) is continuous in a closed interval [-b, b] containing t, then r � � e iw t i (w) dw = I (t) lim r_oo 211" - r 1 - r uniformly in [ b , b] . [Hint: In the proof of 5.12.3 ( a) , note that AI (t, x ) is uniformly continuous in [-b, b] , so there is a 6 > 0, independent of t, such that h ( x ) :::; f.X for all 0 :::; x :::; 6.]

l

-

5.13

)

(

Convergence Assistance

Many integrable functions have nonintegrable Fourier transforms. The func tion I (t) e - t U (t), for example, has 1/ (1 + iw) as its Fourier transform (Example 5.5.3 ) ; the magnitude of the transform is I/Vl + w 2 1/ lw l for large I w I . 1 is not integrable. If 1 is not integrable, then =

as

�

� 211" 1-0000 f (w) e1wt dw

1 -

•

may not exist , and we cannot recover I (t) from it. Nevertheless, regardless of the integrability of 1, if we massage the integrand a little and consider (C, 1 ) summability, then (5. 12.3)

100- eiw t 1(w) dw (C, l ) a.e. 00 J ( 1 - I� I) eiwt 1(w) dw a. e . 2� :r

1 2 limr - oo

f (t)

11"

� (f

limr_oo 211"

where with k (t)

= ( 1 - I t I ) 1 [ - 1 , 1] ,

*

Kr ) (t) a.e.,

rW/2 ] Kr (w) = rk (rw) = r [ sin(rw/2) �

2

Somehow the factor

makes the integral converge. We will investigate other kernels k (w) in this section such that 1 lim r_oo 2 11"

jr k ( W ) e1w t f� (w) dw = / (t) a.e. .

-

-r

r

318

5. The Fourier Transform

In particular, in 5.13.4 we generalize the Fejer-Lebesgue inversion theorem to one in which (1 - I Ir) is replaced by a more general "convergence factor." Let us examine more closely the properties of the hat function

wl

k (t) = (1 - It I) 1 [ - 1 , 1) (t) ,

also known given by

as

the CES A RO KERNEL. By Example 5.5.5, its transform is

k (w) = ( sin��t 2) r

(w)

w

Since k is continuous on [- 1 , 1] and bounded by 4 1 2 for I integrable. Moreover, by 5 . 12 .2 ( c) ( with r = 1 ) ,

00 1 k (w) dw = l. 1r 00

1 2

w l > 1 , it is

-

Obviously, k (w) is even . We abstract these properties of the Cesaro kernel now . Definition 5.13.1 SUMMABILITY KERNELS

k

A real-valued, even function E L 1 ( R) is called a SUMMABILITY KERNEL if its Fourier transform satisfies

k (w)

100

1 k (w) 2 1r - 00

dw = l.

0

Obviously, the Cesaro kernel is a summability kernel. Here are two other examples. Example 5.13.2 GAUSSIAN AND ABEL KERNELS

( a) The ABEL KERNEL is By Example 5.6 .4, its Fourier transform is

2 �k (w) = -12·

+w

Since J�oo 12 a:::2 2 1r , is a summability kernel. + 2 ( b) By Example 5.5.6, the Fourier transform of e - t / 2 is V2iie - w 2 /2 . Therefore, replacing by .;
=

k

t

k (t) =

�

=

5. The Fourier Transform

319

Because of the hypothesis i n 5. 13.4 below , that k (w) is decreasing f?r 0, we note here that the Abel and Gaussian kernels each have this property. Let k (t) = ( 1 - I t I ) 1[ 1 , 1 ] (t), the Cesino kernel. We noted in equation ( 5.26 ) , in the proof of 5 . -1 2 . 1 , that

w>

rw/2 ] I
2

r � l ( 1 - l::ir ) eiw t j(w) dw 211" -1r 00 e,w t f (w) dw. = 211"1 00 k (W) r By the change of roof property, 5.5 . 1 ( e ) , 1 1 00 ( ) e,w t i 2� I:00 [ k (� ) (w) e xt f(w) f (w) dw 211" - 00 k r f (w) dw 1 1 2111" - 00 rk ( r (w - t)) (w) f (w) dw 2 11" (f * I
(t)

Sr

.

-

W

.

�

-

�

-

-

For arbitrary summability kernels, things are quite similar.

For any summability kernel k, and

5.13.3 SUMMABILITY KERNEL BASICS r > 0, let Kr x r rx . Suppose

f E L l ( R) . Then ( a) 1 1 i 2 7r I: j(w) k (�) e w t dw = 2 7r (f * I
0

By 5.5 . 1 ( a, c ) , the effect of scaling (u the transform of k ( ) is as follows: u

....... u/r) and multiplication by eiut on

320

5. The Fourier Transform

Since k is an even function, so is and

Kr ; therefore, Kr (w - t) = Kr (t - w ) ,

I: f (w) [k ( ; ) eiU{(w) dw

(b) With the change of variable w

1

00 1 2 71" - 00

so we can write I (t) 1 271"

k

=

(f * J{r ) (t) .

= ru,

I<.r (u) du =

=

I: I (w ) Kr (w - t) dw

=

1

1 1

00

2 71" - 00 00 1 2 71" - 00

rk (ru) du k (w) dw = 1 ,

f (t) 21", I�oo Kr (u) du , and

(f * I<.r ) (t) - / (t) = 2171" 1 (f (t - u) - / (t)) I
- 00

Since is even, we can compute (f * I
1 1 1 00 K (t) (f I (t) = ) (f (t + u) - I (t)) I<.r (u) du o r * 2 71" 271" - 00

Adding the last two equations and dividing by 2, we obtain

Since Kr (u) and ill (t, u) are each even functions of u,

I�oo = 2 It , i.e . , 1 1 00 1 (f ) (t) (t) = / * I<.r il f (t, u) I<.r (u) du. o 2 71" 71" 0

The Fejer- Lebesgue inversion theorem 5 . 12.3 used the Cesaro kernel. We generalize the Fejer-Lebesgue inversion theorem in 5 . 13.4 to arbitrary summability kernels k such that k is dominated by a decreasing integrable function on the nonnegative real numbers R+ . 5.13.4 A DOMINATED INVERSION THEOREM Let k be a summability ker nel. For r > 0, let Kr (x) = rk (rx). II there is a nonnegative decreasing function D L 1 (R+ ) such that

E

I k (w ) 1 ::; D (w) lor all w � 0,

5 . The Fourier Transform

.

32 1

then at any Lebesgue point t of f E Ll ( R) , as r --+ 00 , a n d the convergence is uniform i n every closed interval i n which f is con tinuous. P roof. The result is trivially true if D = 0, so assume that D =f:. O. Since is decreasing, the geometry of the situation dictates that for w > 0,

� D (w) � 1rw/2 D (y) dy.

Since D is integrable, r:/ 2 D (y)

dy

--+

0 as

W --+ 00 .

Thus,

wD (w) --+ 0 as w Let A / (t, ) = ! (f (t + ) + I (t ) - 2 1 (t)), and g (x) = fox I A / (t, y) 1 dy for x � O.

( 5 .33 )

--+ 00 .

u

u

D

- u

�

( 5.34 )

Since t is a Lebesgue point of I, � = I; I A / (t, y) 1 dy is small for sufficiently small x > 0: Indeed, for f > 0, there exists 6 = 6 ( t) > 0 such that ( 5.35 ) g (x) < ( f / Il D Il 1 ) x f r 0 � x � 6, where 1 1 11 1 denotes the norm in L 1 ( R ) . By 5 . 1 3.3 ( b ) , f,

o

We show that I; and It approaches 0 as r

1-1r 16 A/ (t, x) Kr (x) dx 0

<

--+ 00 .

For I; , consider

� foli6 I AI (t, x) 1 r I k (rx) 1 dx

f I AI (t, x) 1 rD (rx) dx f D (rx) dg (x) equation 5.34

< .!:. 1r 10

6

)) . ( ( 1r 10 Since D is decreasing, it is of bounded variation; 9 is absolutely continuous because it is defined an integral. Hence, we can integrate by parts with !:.

as

322

5. The Fourier Transform

dv = dg to get -11"r 10 6 D ( rx) dg ( x)

° :; D (rx) 9 (x ) I � + 1 9 (x ) d ( - D (rx))] ° = :; D (r6) 9 (6) + 1 9 (x) d ( - D (rx))] . Since I g (x) 1 < ( { / l i D I ) x for 0 ::; x ::; 6 , o -;r Jfo O D ( rx) dg (x) ::; -;r [D ( r6) I D{ I 1 6 + II D{ I 1 Jfo xd( - D ( rx))] . Since 11" > and -D is increasing ( hence of bounded variation ) , we can integrate I� x d ( -D (rx)) by parts to get -11"1 1 ° Af (t, x) Kr Cx ) dx < 1 1;1 1 1 (D (r6) r6 + r ( - xD (rx) I � ) + 1 ;1 1 r 1° D (rx) dx ° 1 ;1 1 / 1 D ( rx ) dx. Since 1; ° D ( w ) dw ::; I D I I 1 ' with w = rx, this implies that (0 -;1 Jfo O A f( t , x) Kr (x) dx ::; 1 1 ; 1 I 1 Jo D (w) dw ::; €. For It , since I Kr (x) 1 = Irk(rx ) 1 ::; rD (rx) and D is decreasing, I � 100 Af (t, x) Kr (x) dx l = 1 � 100 [f (t + x ) + f (t - x) - 2 f ( t ) ] Kr ( x) dx l oo 21 (Xl < 11" J I f (t + x) + f ( t - x )I I Kr (x)l dx + 2 I f (t)I f rD (rx) dx Jo 2 /ir6D (r6) < 2 '( � 6 2 1 I f I l 1 + 2 I f (t)l l� D ( W) dW) . By (5.33), r6 D ( r 6 ) --+ 0 as r --+ so the first term in the parentheses above goes to O. As for the second, since D is integrable, 2 1 f (t)l [oo D (w) dw --+ 0 as r =

1

I,

0

=

00 ,

ro

--+ 00 ,

5. The Fourier Transform

323

and it follows that

Kr )

1 (f * (t) - f (t) - 0 as - 00 . 211" If f is continuous in a closed interval, then each point of the interval is a Lebesgue point of f. Moreover, since such a function is uniformly continu ous, we can choose the 8 of inequality (5.35) independently of t . Therefore, the r > 0 such that r8D (r8) is small is independent of t as well. The con vergence of (1/211") (f * ) (t) to f (t) is therefore uniform on any closed interval. 0 The Abel and Gaussian kernels, k = and k = respectively, of Example 5.13.2 have decreasing, integrable transforms on R+ , so it follows from 5.13.4 that at any Lebesgue point t of f E L (R) , r

Kr

f (t) =

1 lim 211"

r_oo

e- t 2 ,

e- 1 t l

1

l

00

�

f (w) - 00

( e -w1w2 /1 /rr2 ) .

e,wt dw .

e_

If 1 E L l (R) , then we can bring the limit inside the integral by Lebesgue's dominated convergence theorem (4.4.2) to get , in either case, as in 5 . 12.4, 1 f ( t) = 2 11"

1

00

- 00

f (w) e,wt dw �

.

(5 .36)

at any Lebesgue point t of f-in particular, at any point t of continu ity. Although the Cesaro kernel k (t ) = (1 - It I) 1 [ - 1 , 1 ] has a transform k (w ) = ( Sin�/t 2 ) ) 2 that does not decrease monotonically to 0 for w > 0 , its transform is dominated by the decreasing function 4/w2 there . By 5 .12.4, if f and 1 are integrable, then 1 f (t ) = 211"

1

00

- 00

w

f (w) e , t dw a . e . �

•

But when is f E

L l (R) ? Our next result provides a criterion. 5.13.5 CRITERION FOR INTEGRABILITY OF i Let f E L l (R) be such that for some M, and some positive d, I f (t) 1 � M a. e. in [-d, d] . If f ? 0, then fE L l (R) . P roof. Consider the Abel summability kernel k (t) = By Example 5.6.4, k ( w) = 2/ (1 + w 2) . Let r be positive, and let (x) = r k ( rx ) . By

5.13.3(a) ,

1tl . eKr

324

5. The Fourier Transform

Since k is even, with a change of variable we have

( / (x) * rk (rx)) (t)

i: I (t - x) rk (rx) dx i: I (t + x) rk (rx) dx

i: I (t + � ) k (x) dx.

With t = 0, we get

i: j(w) e - 1w l /r dw I: I (�) k (x) dx. =

J�oo into r::d + f�d + frO;; · Since I I (t) 1 S; M a.e. for I t I S; d and k � 0, d J:r d l (�) k (x) dx S; M i: k (x) dx = 21rM. Since 2x/(1 + x 2 ) is bounded on R, xk (x) 1 +2xx 2 S; C for some constant C > 0, say. For x � 0, Split

�

=

With

w = x/r, this becomes C 00 C d J[ I I (w) 1 dw S; d 11 / 11 1 '

d

Likewise, for some positive constant C ' ,

Therefore, there is some constant f{ , independent of r, such that

The monotone convergence theorem [Stromberg 1981, Theorem 6 . 20, p . 266] asserts that if (In) is a n increasing sequence of real-valued functions

5. The Fourier Transform

In

325

I In e- / w / / a e- / w l / b,

integrable on R such that liIDn J�oo (t) dt < 00 and ( t ) is defined to be limn In (t) a.e., then I E L 1 (R) and J�oo I ( t ) dt = limn J�oo (t) dt . the Since � 0 by hypothesis, and a < b implies � monotone convergence theorem permits us to make the following inter change:

j(w)

I: j(w) dw = r�� i: j(w) e - /w l /r dw

� I<. 0

I

Two imp ortant cases in which is bounded a.e. on a neighborhood of 0 are: ( a) is continuous at 0; ( b ) I E L oo ( R) ( i.e., Lebesgue-measurable such that for some constant MJ, I I (t) 1 � MJ a.e.) . In either case, if � 0, then by 5.13.5, E L 1 ( R) and

I

j

j

I

1 1 � e . wt I (t) = I (w) , dw 211" -00 at every Lebesgue point t of I by 5.12.4. 00

Exercises 5 . 1 3

e-a/ t /,

a > O. By Example 5 .6.4, the Abel 1 . A NEW WAY TO WRITE > kernel 0, has the integrable Fourier transform ( 2 a ) / (a 2 + a Use equation ( 5 .36 ) to show that at each t E R,

e-a 1 t /,

1 00

w2).

a2 + w2 dw - 2a e - a / t / . 2. T E SELECTOR PROPERTY OF e - [( t - x )/ 2bj2 For I E L 1 ( R) and b > 0 , show that for almost every t, 1 00 I( x ) e - (1_,.) 2 dx -+ / (t) as b -+ O+ . 2 v 1I"b -00 [Hint: Consider the Gaussian kernel k (x) = e- x 2 . By 5.13.4, 00 1 ( I (x) * rk� (rx) ) (t) = 1 1 / (x) rk� (r [t - x]) dx -+ / (t) a.e. H

cos wt

1

rl

211"

_ .!!...

o

4&

211"

_ oo

r -+ 00 . Now let b = 1/r2 .] 3. We used the Abel kernel k ( t ) = e-/ t / in the proof of 5 . 13.5. Show as

that we could equally well have used the Gaussian or Cesino kernel.

326

5. The Fourier Transform

4. NORM CONVERGENCE For suitable summability kernels k , we showed ( 5 . 13.4) that

t

at every Lebesgue point of f. In other words, we were looking at a form of pointwise convergence. In this exercise and the next we ex plore norm convergence. Let E L 1 (R) be such that f�oo = 1 . Let > O. Show that for any f E L l (R), =

g rt), r t) rg gr ( ( 1 2� (f * gr ) - f i l l

-+

0 as r -+ 00 .

21"

g (t) dt

[Hint: By hypothesis,

2� f�oo gr (t) dt = 2� f�oo rg ( rt) dt = 1 . Now, 2� (f * gr ) (t ) - f (t) 21" f�oo [f (t - x) - f (t)) rg (rx) dx,

so

f�oo f�oo I f (t - x) - f (t) 1 r I g (rx) 1 dx dt 1 21" (f * gr ) - f il l =:S 2211,.". f� w (x) r l g (rx)l dx, where w ( x) = f�oo I f (t - x) - f(t) 1 dt. Since f E L l (R), by con tinuity in the mean 2.8.9 ( namely, that I l f (t + h) - f (t) 1 1 -+ 0 as h -+ 0 ), w ( x) 0 as O. Given ( > 0, there exists 6 > 0 such that I x l 6 implies 0 < w (x) < ( ( 2 11" ) / I l g 1 1 ' Hence 1 2� (f * gr ) - f il l :S I + J, :S

-+

x -+

where

J

21" �xl � 6 w (x) r lg (rx) 1 dx and = 21" �x l>6 w (x) r l g (rx) 1 dx. Let rx = y. For an upper bound c for w (x), I J I :S 2� �YI > r6 Ig (y) I dy -+ 0 as r -+ 00 . 1=

Next,

5. NORM CONVERGENCE AND SUMMABILITY KERNELS Let k be a summability kernel . Show that for any f E L l (R),

1 2� f (x) (rx) - f il l ----. 0 as r -+ uk

00

5. The Fourier Transform

327

1 ( 1 00 ) f (x) rk (rx) (t) = f (w) k (w l r) e , w t dw. * 2w 2w 1-

and

00

�

.

�

[Hint: Use the preceding exercise with 9 = k . Then use

6. Show that

5. 1 3.3(a).]

2 /4 cos xt dx = e - t 2 . [Hint: Use equation (5. 36) with k (t) = e-t 2 ; k (w) = .../ie -w 2 / 4. ] 1

.../i

[ 00 e -x

Jo

7. COMPOSITION

( a) Use the convolutions-to-products theorem and 5.12.4 on inver

sion when 1 E L 1 (R), then 1 2w

L 1 (R) to prove that if f, g , and lu belong to

00 1_00oo f�(w) u (w) e,wt dw = U * g) (t) = 1_oo f (t - x) g (x) dx .

for almost every t E R. ( b ) Use ( a) to show that for

a, b > 0, 00 dx w 2 2 1- 00 (x 2 + a ) (x + b2 ) ab (a + b) ' [Hint: Let f (t) = a > and g (t) = b > O. Then an d 9 ( w ) - 2 Now use ( a) and take t = INTEGRAL EQUATION For a > and f E L 1 (R), solve the

e- a 1 t l , 0 0.]

8 . AN

integral equation

00

1- 00

�

-

e- b1 t l , b2 +bw 2

0

1 f (t - x) f (x) dx = -a 2+ t-2 '

[Hint: Take the Fourier transform of both sides , and use the convolution to to-products theorem and Example . 6 . 2 ( [ 4 ] = (wi a) get l (w) 2 = so l w) (.../il y'ii) Then use 5 . 12.4 on inversion when 1 E L 1 (R) to get f (t) = 2 fi ( )

(

e - a 1 w I /2 .

5 a 2 ! e -a 1 w l , a2�4t2 .]

e- a 1w l )

328

5. The Fourier Transform

9. AN INTEGRAL EQUATION Assuming that integral equation

1 0000 a2 f+(y)(t _dyy) 2 = b2 +1 t 2 ' 0 -

[Ans. f (t) :b ( b _ba)f+ t 2 ') =

10. AN INTEGRAL EQUATION Assuming that integral equation

[A ns. f (t)

=

f E L l (R), solve the
b.

f E L l (R), solve the

12

_ 2 .) J2�

1 1 . CONVOLUTIONS TO PRODUCTS II For f, g , j, g E L I (R), show that 1 211" for almost every Hence

10000 (�f * g) (w) ezwt. dw -

=

21fJ (t) g (t)

t E R. [Hint: Since 1, g E L1 (R) , j * g E L 1 (R).

w = x - t. Then the right-hand integral equals 1" J�oo J�oo j( w) g (t) ei(t+w)u dw dt 2 et) eitu dt · 211" 21" J�oo i ( w) ei w u dw 2211"� gJ�(u)oo fg (u) a.e. ) 12. PRODUCTS TO CONVOLUTIONS I If f, g, 1 E L I (R), show that fg E L 1 (R) and that fg = (1/211") j * g. [Hint: The function f is bounded by 5 .12.4 and 5 . 12.5. Hence fg E L 1 (R). Now f[ ,(w) = f(t) g (t) e- iw t dt. Let

Since i E

1:

L 1 (R), by 5.12.4, the integral above is 1 fg (w) 2 " J�oo ( J�oo i (y) e iy t dyi() 9 y)(t) e - iw t dt - w - t dt dy 2�1 J�oo J�(y)oo iC(wy) 9 (t)y) edy.) 2 " J�oo i g -

5. The Fourier Transform

329

13. Use the results of Exercise 12 and Example 5 .6 .2 to show that the Fourier transform of (t 2 + 1 -4 is

)

2� 1re -1 w l * 1re -1 w l = � e - I w l * e -1w l . [Hint: Find the Fourier transform of (t 2 + 1) -2 . By considering cases, show that this equals (1r /2 ) e -1 w l (1 + I w l ) .] 14. PRODUCTS TO CONVOLUTION II If f, g, i E L l (R) (so that f7g = Pi E L l (R)), show that 1 00 � 21r 1- 00 f (w) g (w) etwt dw = (J * g) (t) a.e. [Hint : The integral equals 21" J�oo f7g (w) eiw t dw. By 5. 12.4, this is just (J * g ) (t) a.e.] .

3 .3 .1 3 3 .

15. PARSEVAL'S IDENTITIES We saw Parseval's identities in the context of Hilbert spaces in and . 4 (f ) ; we consider versions that use the Fourier transform rather than the finite Fourier transform here and in Section 5.17. If f, g, i E L l (R), show that: (a)

1-0000 f (t ) g (t) dt = �2 1r 1 °° i(w) g (w) dw. 1-00 If (t) 1 2 dt = 211r 1-00 l i (w) 1 2 dw < 00 . 00 00 - 00

(b)

E L l (R) n L 2 (R) and i E L 1 (R), then i E L2 (R) . [Hint to (aJ: By Exercise 12, 1: f (t) g (t) e - iwt dt = 2� i(y) [g (;: y) ] dy.

Note that if f

1:

g (y) = g (-y).] Use the Parseval identity of Exercise 15 ( b ) to evaluate the integral 100 ---" dt "72 , a > 0. 2 + (a t 2 ) Let w = 0, and use the fact that

16.

- 00

[Hint: By Example 5.6.2, the Fourier transform of f (t) = (a 2 + t 2 ) - 1 , a > 0 , is i(w) = (1r/a) e - a 1 w l . By the Parseval identity, 00 dt _ 1 00 1r2 e -2al w l dw - 1r ] 2a3 · 1 (a2 + t 2 ) 2 - 21r 1- a2 - 00

00

_

330

5. The Fourier Transform

5 . 14

Approximate Identity

(L 1)( T)

)

As in the case of , +, discussed in Exercise 5.2- 1 1 , the Banach algebra (R) , + , does not have an identity with respect to convolu tion: If there exists E (R) such that = I for all I E (R), then in particular, = so (U) 2 = U. Thus, U (w) is 0 or 1 . Since U E Co (R), we conclude that u (w) 0 for all w. Hence = 0 a.e. by the inversion theorem 5 . 12.4, which is contradictory. As we pointed out in Exercise 5.2- 1 1 , however, has approximate identities, an "approximate identity" being a sequence of functions from that behave like Dirac 's delta function in the limit. Specifically, an APPROXIMATE IDENTITY for of elements from is a sequence 1 such that: (a) 11 1 < 00 . 1 '" . (b) l (t) dt = l . 211" (c) limn t dt = 0 for any r E (0, 11") . rS ltlS'" The imp ortance of approximate identities is that if is an approximate identity for (T) , then

(L1

*

u

u * u

L1

* ,

L 1 (T) L (T) SUPn l i On n 1 On 1 I On ( ) 1 L1 lffi

1

u,

=

L1

* u

L1 (T)

L 1 (T)

(on)

u

( On)

_ ".

( On)

n l i On 1 = 1 11 1 = O . As mentioned in Exercise 5.2- 1 1 , the Fejer kernels Fn (t) constitute an approximate identity for L1 (T), but the Dirichlet kernels do not because I Dn ll l � 00 . Definition 5.14.1 ApPROXIMATE IDENTITY FOR L 1 (R) A sequence (on ) of functions from L 1 (R) is an APPROXIMATE IDENTITY for L 1 ( R) if: (a) Supn l i On 1 1 � [{ < 00 . (b) limn I: On (t) dt = c for some constant c. (c) limn [ I On (t) 1 dt = 0 for any > O . 0 J1t l ? r lim

*

r

Approximate identities always exist . For example, consider the functions

n n , l nj. On = "21[-1 / /

I n this case w e can take [{ = 1 = c. Theorem 5 .14.2 establishes the rea son for the name "approximate identity." The conclusion of the theorem, limn 1 - c i = 0 , also holds for I E (R) , 1 < p < 00 (see Exercise 1) .

l i On

*

/l l

Lp

5. The Fourier Transform

33 1

5.14.2 If (bn) is an approxzmate identity for L l (R), then with c as in Definition 5. 14 . 1 , for any I E L I (R) , lim n li on * I - c/i l l = O. Proof. For I E L l (R) and h E R, let Ih (t) I (t + h). By continuity in the mean (2.8.9), II lh - 1 11 1 -+ 0 h -+ O. Thus, with [{ as in Definition 5.14.1, for > 0 there exists > 0 such that II/- h - 1 11 1 2 [{f for Ih l By 5.14.1(c), there exists an integer N such that f for n 2: N. r IOn (t) 1 dt 4 I l I II 1 J1 t l ? r With Cn = J�oo On (h) dh, n 2: N, and any t E R, f

r

=

as

<

<

<

r.

-

Hence

II I * bn - cnl l b

<

=

< <

1: 1: I I (t - h) I (t) l I on (h) 1 dh dt 1: 1: I f (t - h) - f (t) l I on (h) 1 dt dh 1: II/- h - 111 1 IOn (h) 1 dh f 2 } , Jf1 h l < r IOn (h) 1 dh + 2 J1rh l -> r 11 / 11 1 IOn (h) 1 dh f f -

{

2J{ [{ + 2 11 / I1 l 4 11 / I 1 l = f . Therefore , limn II I * On - cnl il l = O. By property (b) of Definition 5.14.1, it follows that limn C =-+ c . Therefore II/ * On - c/ li l = II / dn - cnl + (cn - c) / lI l :S II / * on - cnl ll l + ! cn - c l ll / ll l · 0 An immediate consequence of 5. 14.3 is that if k is a summability kernel, then (n k (nx)) is an approximate identity. 5.14.3 MANUFACTURING ApPROXIMATE IDENTITIES Suppose g E L l (R) is such that 1 1 00 21r - 00 g (t) dt = 1 and gn (t) = n g (nt), n E N. Then ( gn ) is an approximate identity, and limn J�oo gn (t) dt = 21r. <

332

5 . The Fourier Transform

Proof. For any such 9 and any n ,

i: gn (t) dt i: g (nt) dt i: g (w) dw =n

=

=

211".

Likewise, for every n ,

and properties ( a) and ( b ) of Definition 5 . 14.1 are satisfied. For any o and n E N ,

J[1 t l?d I gn (t) 1 dt

[ I g (nt) 1 dt 1.Iwl? dn I g (w) 1 dw n

-l-

d>

J1tl? d

0 as n

-l-

00. 0

It follows from Exercise 5 . 13-4 that if 9 is such that � f�oo 9 (t) and gr (t) = r g (rt) for r > 0, then for any f E Ll (R) ,

2

21'/r f�oo

dt = 1

(t) dt

A variant of Exercise 5.13-4 runs as follows. If 9 is such that 9 = 1 and = n n , n E N, then is an approximate identity by

gn (t)

g ( t)

(gn)

5. 14.3; it therefore follows from 5.14.2 that

If k is a summability kernel, then � f�oo k (w) dw = 1 , and an approximate identity by 5 . 14.3. Therefore, by 5. 14.2,

2

00 1 211"

By 5.13.3, however, 1

_ oo

� f

(nk (nx)) is

. 1 ( � ) (w) k (W) � e· w t dw = 211" f (x) * nk (nx) (t) ,

which is the essential content of Exercise 5 . 1 3-5. Since the Cesaro kernel k (t) = ( 1 It I ) (t) is a summability kernel, its Fourier transform, -

1[- 1 , 1]

5 . The Fourier Transform

produces the approximate identity with nth term

On (w) = n k ( nw )

=

( n

.!. sin nw I 2 w l2

333

)2

Similarly, the Fourier transform y'1ie - w 2 /4 of the Gaussian kernel k (t) = e -t2 also produces an approximate identity. Exercises 5 . 1 4 1 . ApPROXIMATE IDENTITY FOR L 1 (R) AND f E Lp, 1 < Show that if (on ) is an approximate identity for L 1 (R),

100 On (t) dt

p

< 00 .

c, then the conclusion of 5. 14.2 remains valid for f E Lp (R), 1 < p < 00 , namely that limn liOn * f - cf ll = O. [Hint: Let p be as above, and let q be a solution to lip + l /?q = 1 . Let h E Lp (R). The map ip defined at each f E L q (R) by IP (f) = J�oo f (x) h (x) dx is a continuous linear functional on L q (R), and IIIPII = II h ll p . [Dunford and Schwartz 1958 , IV. 8 . 1 , p. 286] . Let f E Lp (R). As in the proof of 5.14.2, given f > 0, choose > 0 such that II !- h - f lip < 2]( for I h l < lim n

-00

= C for some constant

r

f

r.

Choose an integer N such that

4 11 ! llp Jr � l h l l on (h) 1 dh < for n � N, and let en = J�oo On (t) dt. It can be shown that if f E Lp (R), then ! * On - cn! E Lp (R) (Exercise 5 .20- 1 ) . By the Holder inequality (1.6.2(c) ) , ( f * On - cn!) g E L 1 (R) for any g E L q (R). Consider the linear functional IPn defined on L q (R) by R, IPn : Lq (R) g � J�oo [ f * On (t) - cnf (t)] g (t) dt, of norm I I IPn l1 = Il f * On - cnf ll p . For n � N, and any E L q (R), I IPn (g) 1 = I J�oo J�oo (f (t - h) - f (t)) On (h) g (t) dh dt l � J�oo J�oo I f (t - h) - ! (t) l l on (h) l l g (t) 1 dt dh. f

--*

g

B y the Holder inequality ( 1 .6.2(c) ) , i t follows that Il f- h - f lip IOn (h) l ll g ll q dh IIPn ( g ) 1 < < � 2+ 2 I l gfll q IOng (h) 1 ndh(h) dh Il ll p Il ll q IO 1 < � ]( II g ll q + 2 11 f ll p II g llq = 2

J�oo �hl
41 fl p

f

Il g ll q .

334

5. The Fourier Transform

It follows that II lf'n ll = that

II I * On - cn/ lip < ( for n � N , in other words, l� II I * On - c/ li p = 0 . ]

2. SOME SPECIAL CASES Determine the approximate identities associ ated with the Gaussian and Abel kernels e - t 2 and e -1t l , respectively. 3. Show that for every n, On (t) (nl../270 e - n2 t 2 / 2 E L1 (R) , IIOn l l 1 = 1 , and that ( on ) is an approximate identity for L1 (R) . =

5.15

Transforms of Derivatives and Integrals

In this section we consider Fourier transforms of derivatives and integrals. It is imp ossible to overstate the imp ortance of 5.15. 1 . It demonstrates that differentiation in the time domain corresponds to multiplication in the fre quency domain. This property is what gives the Fourier transform its im portance in solving differential equations.

......

5.15.1 TIME DERIVATIVES AND MULTIPLICATION BY iw: I' (t) iwj(w) If I E L1 (R) is absolutely continuous in every closed finite subinterval, and the kth derivative I ( k ) is in L1 (R) for k = 0, 1 , 2 , . . . , n, and each I( k ) van ishes at infinity, then for each k 0, 1 , 2,

I( k ) (t)

=

1--+

...

, n,

(iw) k few) .

Proof. It suffices to prove this for k = 1 ; the rest follows by induction on k. Let I E L1 (R) be absolutely continuous in every closed finite interval so that its derivative !, exists a.e. In addition, suppose that I vanishes at infinity, and that !, E L1 (R) . Then integration by parts ( with dv = I' (t) dt) yields

1: !' (t) e - iwt dt I(t) e - iwt eoo iw 1: I(t) e - iwt dt 0 iw j (w ) . Since liffiw_ ±oo 1: I'(t) e - iw t dt 0 by the Riemann-Leb esgue lemma =

=

+

0

+

=

4.4. 1, it follows from 5.15 . 1 that

(W )

j w _lim ±oo I w l j(w) = lim ±oo 1 l I w 1 = 0, w

....

so not only does j (w) vanish at infinity for any I E L1 (R) , for differentiable I, it goes to 0 faster than II Iw l. In the notation of Definition 4. 12.2, j(w) is of smaller order than II I w l:

j(w) = 0 C� I ) .

5 . The Fourier Transform

335

The smoother I is, the more rapidly ! (w) 0: If I( k ) E LI (R) for k = 0, 1 , 2 , . . . , n, and the derivatives I ( k ) vanish at infinity for k = 0 , 1 , 2 , . . . , n, then �

(5 .37)

which is the continuous analogue of 4.12.3. Dual to the correlation between time derivatives and multiplication by iw is 5 . 15 . 3 , which relates frequency derivatives d! (w) /dw to multiplying I by -it. Before establishing 5 . 1 5 .3 , we state a result on differentiation under the integral sign that will be used in many circumstances; it says that under fairly mild restrictions on the integrand,

(t)

d 1 00 I (w, dw = 100 I (w, dw. -d - 00 - 00

-aat t)

t)

t

This is sometimes called Leibniz 's rule. For a proof, see Bartle 1 995 ,

p.

46 .

5.15.2 DIFFERENTIATION UNDER THE INTEGRAL SIGN Suppose that w o ) is an integrable function oft on R for some W o E [a, b), that ah/ exists on R x [a, b], and that there exists g E L 1 (R) such that I �: I ::; g (t) for all w E [a, b) (so that the domin ated convergence theorem 4.4. 2 can be used). Then F (w) = J�oo h (t, w) dt zs differentiable, and

h

aw

(t,

F'

00 -a h (t, w) dt. - 00 aw

(w) = 1

Consider what this means for Fourier transforms

1: I(t) e- iwt dt. (t) e - iwt, which is differentiable everywhere

!(w) = The integrand h of 5 . 1 5 .2 is I with respect to w , and

l a:/(t) e-iwt l = I -itl (t) e - iwt l = I tl (t) l , which is integrable if t l Hence, a sufficient condition for dif ferentiability of !(w) is that tl If I can be retrieved from the inversion formula,

E L1 (R) . E L 1 (R) .

00

1 1 I (w) e ·w t dw, I (t) = 211" �

.

- 00

and w ! (w) E L1 (R) , then 1 100 � I' (t) = I (w) (iw) 2 11" -

00

e·wt dw . .

Compare this to 5 . 1 5 . l . We use 5 . 15.2 to calculate Fourier transforms of moments

tn I (t) of I (t) .

336

5. The Fourier Transform

5.15.3 FREQUENCY DERIVATIVES AND MULTIPLICATIO N BY -it If f, tf E L l (R) , then j (w) is differentiable, and -itf (t) d�� ) , or tf (t) i d�� ) . Ift n f E L l (R) , then (-it t f (t) � jC n ) (w) , n E NU {O} . Proof. Note that a (J (t) e - iw t ) law = -itf (t) e - iw t . Since tf E L I ( R) (i.e., I tf (t) 1 is the 9 of 5 . 15.2) , j(w) = i: f(t) e - iw t dt �

�

is differentiable and can be differentiated under the integral sign, and the desired result follows. The extension to t n f (t) follows by induction. 0 We saw in 5.15.1 the correspondence between the time derivative f' of f and multiplication of j (w) by iw . Ton go backwards from the Fourier transform of an nth time derivative f e ) (t) to the Fourier transform of f(n - l ) (t), we divide f( ;)(w) by iw. 5. 15.4 TRAN SFORMS O F INTEGRALS AND DIVISION BY iw If f , tf E L l (R) and f�oo f(t) dt = 0, then 9 (t) = f� oo f (x) dx E Ll (R) , t f (W f (x) dx 1--+ . ) , w # 0, zw -

1

and

�

00

- i: tf(t) dt. Proof. To see that 9 (t) = f�oo f (x) dx E L l (R) , we show that fooo 9 (t) dt and f� oo 9 (t) dt exist . Consider ft 9 (t) dt. Since 0 = f�oo f = f� oo f + !too f for any t, it follows that 1 00 Ig (t) 1 dt = 1 00 11 00 f (x) dx l dt. Interchanging the order of integration and using the fact that xf (x) E L l (R) , we obtain 9 (0) =

5. The Fourier Transform

337

J� Ig (t) dt < 00 , so 9 E Ll (R) . Integrating by parts (take e iw t dt)oo, for w 1:/; 0, t g {w) 1: e -iwt [l oo f (u)oo dU] dt e��wt jt f (u) du l _ ( _� JOO f{t) e- iwt dt zw ) _ oo zw 00 00 i (w)

Similarly, dv = -

iw

If w = 0, then

(w :/; 0) .

t l: l oo f {u) du dt 00 00 f ( u) du dt + ° Itoo f ( u) du dt - 1oo 1 1O 00 r r f { U) dt du + j 1U° f (u) dt du h oo h 00 rJo uf (u) du - j-O00 uf- (u) du - 1: uf {u) duo 0

g {O)

_

_

We have already computed the Fourier transform of the hat function in Example 5.5.5. We use the techniques of this section to recompute it now . Example 5.15.5 HAT FUNCTION

Solution .

5.5.4,

Let

p

Sh ow that

h (t) = (I - I t i) 1 [- 1 , 1] 1---+ ( sin�/;2» )

(t)

2

denote the rectangular pulse 1 [- 1/2 , 1/2] . By Example

( 2 sin {w/2) . p w) w By the time-shift theorem 5.5.1(b), �

_

(t + 2"1) I---+ e w / 1) w/ p (t - p

and

;

2

I---+ e

_i

2

/)

2 sin ( w 2

2

w

i /2 )

2 s n (w

w

( 5 . 38)

' .

5. The Fourier Transform

338

Therefore ,

eiW /2 2 sinw(w/2) - e _iw / 2 2 sinw(w/2)

p (t + t) - p (t - t)

2 sin (w/2) (2i sin w/2) w

4i sin 2 (w/2) w

o. -1

-0.5

The Hat Function We get the hat function

h (t )

h

(t)

=

(1

-

2.5

It

- 1 1 ) 1 [0,2)

h as the integral of k (t) = P (t + t) - p (t - t) :

= {too [p (x + �) - p (x - �)] dx J�oo k = 0 tk (t)

1[- 1 , 1] ' = ( I t l - 1 ) 1[- 1 , 1 ] (t) E

= ( I - It I )

and By 5. 15.4, since k is such that . . . above is ob = the Fourier transform of the integral h tained by dividing the transform of the integrand by iw :

L 1 (R),

(t) J� oo

h

(w)

=�

4i sin 2 (w/2) w zw

sin 2 (w/2) . (w/2) 2

5.15.6 Sh ow that - _" w e - W 2/4 -; te-t2 ' Since - 1 ( e- t 2) = t e - t 2 , by 5 . 1 5 . 1 2 -1 iw! (w) , te-t2 T

Example

Z

1---+

Solution .

=

r:;r

0

•

�

1---+

i

2

-e w 2(t) = e- t .

where is the Fourier transform of ! By Example 5 . 5 . 6 the / 2 j by 5 . 5 . 1 ( a) (replace by J2t) , Fourier transform of e- t 2 / 2 is ..,fii

t

5. The Fourier Transform

the Fourier transform of

339

f (t) = e - t2 is ..,fie -w 2 /4 . Therefore ,

A s done i n Example 5 . 1 5 .6 , w e could use Example 5 . 5 . 6 and 5 . 5 . 1 ( a) to find the Fourier transform of e-b t 2 ; instead , in the next example we differentiate under the integral sign and use 5 . 1 5 .3. Example 5.15.7

Show that for b > 0 , j(w) = 1: e- bt2 e - iw t dt.

Solution . Let

Since

tf E Ll (R) , it follows from 5 . 1 5 .3 and an integration by parts that dj - i 1 00 e - bt 2 t e - iwt dt dw -w 1- 0000 e - bt 2 e -iwt dt 2b - 00 -w -u f (w) . �

Solving the differential equation, we conclude that Since 00 � = e - bt 2 , b - 00 it follows that

f (O)

1

�

Example 5.15.8 - iw ( 1 +4w 2 )2 .

dt = Ii-

Show that the Fourier transform of f (t)

Sol ution . By Example 5 . 6 .4,

By 5 . 1 5.3,

j(w) = j(O) e - w2 /4b .

te - I t I

2 . e - I t l l---+ --1 + w2 .

d 2 = -4iw . dw 1 + w 2 ( 1 + w 2 ) 2

1---+ z - --

0

340

5. The Fourier Transform

Exercises 5 . 1 5

I E L 1 (R) show that f (t) 1-+ / ( -w). For any I, 9 E L 1 (R), if I' E L 1 (R), show that f 9 is differentiable and (J g)' = f' g. [Hint: Use 5 . 15.2 to differentiate under the

1 . CONJUGATES For 2.

*

*

*

integral sign.]

3. LINEAR COMBINATIO NS OF DERIVATIV ES As in 5 . 1 5 . 1 assume that is absolutely continuous in every closed finite interval, 9 E that g(k) E for k = 0, 1 , 2 , . . . , n , and that the g(k) vanish at infinity. For any constants k = 0, 1, 2, . . show that

L 1 (R)

L 1 (R)

ak ,

.

, n,

4. Find the Fourier transforms of (a) (b)

t 2 e-t 2 • t e - a 1 t l , a > 0.

5. MODULATIO N Find the Fourier transform of for 6.

j I E L 1 (R). For f E L l (R), show that (a) f ( a t - b ) 1-+ ! l j(�) e - i ': b . 1

I (t) sin at in terms of

(b) Use the result of (a) and Exercise 5.5-5 to obtain the Fourier transform of the characteristic function l[c,dJ of the closed inter val [c, d].

7. FORM O F ODD TRANSFORM Show that for f then = - 2i

1 00 I(t) sin wt dt.

j (w)

5 . 16

E

L l (R), if j is odd,

Fourier Sine and Cosine Transforms

We consider only real-valued functions in this section. Let The Fourier transform

I E Ll (R). j(w)

i: I(t) e -iw t dt

i: I(t)

cos

wt dt - i i: I(t) sin wt dt

5. The Fourier Transform

341

simplifies to

00 00 i (w) = 1 I(t) cos wt dt = 2 10 I(t) cos wt dt (5 .39) - 00 if I is even. Likewise, if I is odd, we retain only the sine integral. Many functions I (t) of interest are CAUSAL in the sense that they are 0 prior to a certain t; for example , a function I such that I (t) = 0 for t < 0 This gives us some choices for Fourier transforms of such functions: we can define on all of R by extending it as an even or an odd function. This leads us to consider the following transforms; they play an imp ortant role in signal analysis, and heat flow problems.

I

Definition 5.16.1 SINE AND COSINE TRANSFORMS

I E Ll (R) , ic (w) I E Ll (R+ )

Let R+ denote the nonnegative real numbers; obviously, if then ( R+ ) . The Fourier COSINE TRAN SFORM of

I E LI

Fc ! (w) = ic (w) = l oo I(t) cos wt dt; the Fourier SINE TRAN SFORM is (w) is

IS

F. / (w) = is (w) = l oo I(t) sin wt dt.

Clearly, ic is even and is simple observation is that for any

(w)

0

(w) is odd for any I E Ll (R+ ) . Another I E Ll (R) , if I is even, i (w ) = 2ic � ) , (5 .40) -2il. (w) , if I is odd. . Any function I E Ll (R) can be written as the sum of an even and an odd function, I = Ie + 10 ' where Ie (t) = I (t) +2 I ( - t) and 10 (t) = I (t) -2I (-t) .

{

Thus, by equation (5.40) ,

I = Ie + 10

Therefore,

f---+

i = Ie + 10

2Fc (fe) - 2iF. (fo ) .

342

5. The Fourier Transform

As integral operators, the sine and cosine transforms are obviously linear . Suppose that 1 (R) is such that E (R+ ) ; then we can differ entiate under the defining integral of the transforms with respect to by 5 . 15.2 to get

E L2

tf L1

w

Is (w) = Fe (tf (t)) and fe (w) = -Fs (tf (t)) . - I

_ I

Some other elementary properties are the following:

5.16.2 SINE-CO SINE TRANSFORM BASICS For f E L1 (R + ) and any a > 0; (a) SCALE ( DILATION )

( �) � 1.(b) SHIFT. and

Fe

Fe [f (at)] (w) = �1 fe ( �w ) , and Fs [J (at)] (w) = �

[J (t + a) + f ( I t - a l )] (w) 2!c (w) cos aw =

[J ( It - a I ) - f (t + a)] (w) = 2 1. (w) sin aw. (c) MOD ULATIO N For any scalar b 1 Fe [J (t) cos bt] (w) = "2 [fe (w + b) + fe (w - b) ] Fs

�

and

Fe [1

�

(t) sin bt] (w) = � [1. (w + b) - 1. (w - b) ] .

The analogous results hold for Fs .

Proof. Since (a) requires j ust a change of variable, we prove only (b) and (c) . (b) SHIFT Let 9 denote the even extension of to R. Then

f Fe [g (t + a) + g (t - a)] (w) = 1 00 9 (t + a) coswt dt o + 1 00 9 (t - a) cos wt dt. Let u = t + a in the first integral and u = t - a in the second. The integrals

become

100 g (u) cos w (u - a) du + 1: g (u) cos w (u + a) duo

Expand the cosine in each, and split the second integral into get

oo to

I� a + Io

5. The Fourier Transform Fc [g (t

+ a) + 9 (t - a)] (w) cos wa

1a00

+ cos wa + cos wa which equals

0

g (u) cos wu du + sin wa

1a 00

343

0

g (u) sin wu du

L a g (u) cos wu du - sin wa L a g (u) sin wu du

1 00

g (u) cos wu du - sin wa

fooo a = fo

100

g (u) sin wu du,

= f�oo

Since 9 is even, 9 (u) cos wu du 9 (u) cos wu du and 9 (u) sin wu du - 9 (u) sin wu du so we can write the displayed ex pression as

f� a

cos wa

[L:

]

g (u) cos wU du +sin wa

The second term is

[l

[(100 1a -100 ) +

]

9 (u) Sin w u du .

1°O 9 (U) Sin wu du] = 0. oo Similarly, since f�oo 9 (u) cos wu du = (f� oo + fo ) 9 (u) cos w u du , the first sin wa

°O

g (u) sin w u du -

term can be rearranged to yield

Fc [g (t + a) + 9 (t - a)] (w) =

2!c (w) cos aw .

Since a > 0, for t - a 2 0, we have 9 (t + a) 1 (t + a) and 9 (t - a) 1 (t - a) ; 9 (t - a) = 1 (a - t) if t - a < o. Thus, for t > 0, 9 (t + a) + 9 (t - a)

Therefore , Fc

(g (t + a) +

9

(t - a»(w)

=

= 1 (t - a) + 1 ( It - a D . = Fc [I (t + a) + 1 ( It - a l )] (w ) = 2!c (w) cos aw , for a > O .

The second shift formula follows similarly. (c) MO DULATI O N For any b E R,

Jro oooo f (t) cos (w + b) t dt

!c (w + b) =

1

f (t) cos wt cos bt dt -

Fc [I (t) cos btl (w)

-

Fs

1 00 f (t) sin wt sin bt dt

[! (t) sin bt] (w) .

=

344

5. The Fourier Transform

Replacing

b by -b yields lc (w - b) = Fe [! (t) cos bt] (w) + Fs [I (t) sin btl (w) .

Adding,

[/ (t) cos bt] (w) = � [!c (w + b) + !c (w - b) ] . The expression for Fe [I (t) sin btl (w) is obtained similarly. 0 Suppose that I E Li (R+ ) , and we extend I to e E L'i (R) as an even function. As noted in equation ( 5 .40) , e(w) = 2!c (w). If I is of bounded variation on [a , b], 0 � a < b, and is continuous at t E ( a , b), then I (t) can Fe

00 e(w) eitw dw 1 211' - 00 1 211' 00- 00 / (w) ez tw dw 1 1 -PV !c (w ) wt dw 00 ; 1 00 lc (w) cos wt dw. -

be recovered by the inversion theorem 5 .9 . 1 : -pv

1

I (t)

- pv

1

11'

�

.

2 e

cos

....

(5.41 )

a

A similar result holds for the Fourier sine transform. We considered Fourier transforms of derivatives in 5.1 5 . 1 . In particular, if L l (R) is twice differentiable and and are integrable and vanish at infinity, then f" (t) 1-+ What about Fourier sine and cosine transforms of derivatives? To simplify the argument, we assume slightly stronger conditions than are necessary.

IE

(iw) 2 !(w).

I

I'

5.16.3 COSINE T RAN SFO RM OF DERIVATIVES Suppose that I E L'i (R+ ) is twice differentiable, that each derivative is integrable, and that limt ..... I (t) 0 = limt ..... f' (t). Then

=

oo

oo

Fe

(f" (t)) (w) = -w 2 !c (w) - I' (0) .

Proof. Integrating by parts, we obtain Fe

(I" (t)) (w)

r oo f" (t) cos wt dt -f' (O) + w Jo

Another integration by parts yields

10 f'(t)sin wtdt. 00

-,' (0) - w 2 �1o °O I (t) coswt dt -f' (0) - w 2 Ie (w) . 0

5 . The Fourier Transform

f', . . . , f( iv ) are integrable ill f, . , f vanish at + 00, then Fc ( f( i V ) (t) ) (w) = w4 ic (w) + w 2 f' (0) - /' " (0) , etc.

A very similar argument shows that if ..

345

and

Analogous results hold for the Fourier sine transform. In particular (same assumptions about integrable derivatives that vanish at as above) , Fs

and

Fa

( I" (t)) (w) = - w 2 1. (w) + wf (O) ,

+ 00

(f( iV ) (t)) (w) = w 4 1. (w) - w 3 f (0) + w /" (0) .

iw;

The Fourier transform of an integral is obtained by dividing by division by w and switching from sine to cosine transform or cosine to sine transform accomplish this for the sine and cosine transforms.

5.16.4 TRAN SFORMS O F INTEGRALS If f, belong to L1 (R+ ), then for w f- 0, Fc

J; f(u) du,

and

!too f{u) du

(1 00 f{u) dU) (w) = �1. (w)

Fs ( It f{ U) dU ) (w) = � ic (w) . Proof. We prove only the first statement . Since !too f( u) du is integrable and

and the cosine is bounded, their product is integrable, and we can switch the order of integration as follows: Fc

(!too f{u) du) (w)

l

100 [1 00 f(U) dU cos wt dt 100 [lU cos wt dt f( u) du

1 100 w-1 0 sin wuf(u) du w-fs (w) . 0 �

We illustrate the use of these general principles in some examples. Example 5.16.5 EXP ONENTIAL I transforms of e - a t , > are

a 0,

Fc ( e- a t ) (w) =

W

2

a + a2

Show that the Fourier cosine and sine and Fs (e- a t ) (w) =

W

w + a2 . 2

346

5. The Fourier Transform

P roof. By Example 5.5.3

e -at U (t) _1 a +_zw._ , 1------+

i .e . ,

1 00 e -at cos wt dt - i 100 e- at sin wt dt 1 a + iw a - zw a2 + w2 · Example 5.16.6 GAUSSIAN Fa (k e- t 2/2) ( w) = w e -w 2/2 . By Example 5 .5.6 we know that the Fourier transform of e-t2 /2 is e- w 2 / 2 . Since e-t2 /2 is even, e -w 2/ 2 = 2 Fc ( vh e -t2/2) (w ) . Differentiating Fe (J;,.. e -t2/2) (w) under the integral s ig n (justify) with respect to w, we get Fa (.Jk:. e - t2 /2 ) (w) = w e - w 2/2 . 0 1, o t ::; a, , a > Example 5.16.7 RE CTANGULAR PULSE Let f (t) = { 0 , t > a, O . Then, for w =1= 0 , . n aw . 0 a cos wt dt = -lc ( w) lo w Example 5.16.8 FIRST MOMENTS O F EXP ONENTIALS For a > 0 show that the Fourier cosine and sine transforms of t e-at are, respectively, a2 _ w2 and 2aw . (w 2 + a2) 2 (w 2 + a2) In Example 5 . 1 6 . 5 we saw that for a > 0 , a os -at d c e wt = --::-_---=t Fe (e- at) (w) = 100 + w2 a2 o and Fa (e- at) (w) = 1 00 e -at si n w t dt = w2 w+ a 2 . 0

::;

Sl

-----,,-2

Solution .

o

5. The Fourier Transform

347

Fe under the integral with respect to w, we get 00 sm w t dt = - 2aw = ( -Fa -1 (w 2 a 2 ) 2

Differentiating

a

Differentiating

te - at ·

t e - at) .

+

Fa, it follows that

For a, b > 0, show that the

Example 5 . 16 . 9 SHIFTED EXP ONENTIALS

Fourier cosine transform of e- al t - bl is

a ( 2 cos w w 2 + a2

b

- e -ab )

.

Solution . By Example 5.1 6.5,

By the linearity of the cosine transform,

Fe

(e - a(tH) )

(w) =

e - ab

a . 2 w + a2

Hence , by the shifting property 5 . 16 .2(b) ,

W 2 ci a2 cos bw - e -aeb-awb2)+. a2 a

2a

w 2 + a2

( 2 co bw s

-

Example 5 . 1 6 . 1 0 MODULATED EXP ONENTIALS cosine transform of cos a > E R, is

e - at

bt,

0, b

0

Show that the Fourier

Solution . By Example 5 . 1 6 . 5 , the cosine transform of

e-at

is

By the modulation property 5 . 1 6 .2( c) , the effect on the cosine transform of of multiplying by cos is to replace

e-at

bt

348

5. The Fourier Transform

Le. , 1 Fe ( e -at cos bt ) (w) = 2

[ (w + b)a2 + a2 + (w - b)a2 + a2 ] .

0

The next example illustrates the utility of the differentiation property for computing Fourier sine or cosine transforms. Example 5 . 1 6 . 1 1 EXP ONENTIAL I I

Use 5. 1 6 . 3 to show that the Fourier

a > 0, is

cosine transform of I ( t ) = e-at ,

Fe {e -a t) (w ) = Proof. Let I

(t) = e- at . Since f" ( t ) =

a w 2 + a2 .

a 2 e- at = a 2 f (t ) ,

it follows from 5 . 1 6 .3 that

a 2 lc (w ) = Fe (t" (t)) (w) = f' (0) - w 2 lc (w) . Since f' (0) = -a, a 2 I�e (w) - a - w 2 I�e (w) , -

_

(w 2 + a2 ) Ie� (w) = a or Ie� (w) = w 2 +a a2 . 0 We computed the sine transform of f ( t ) = e - at , a > 0, as is (w) [wi (w 2 + a 2 ) ] in 5 . 1 6.5 . So, by 5 . 1 6 .4 on transforms of integrals, w = 1 . Fe ( 1 00 e- au du ) (w) = ! is (w) = ! 2 w w w + a2 w 2 + a2

it follows that

t

In other words,

( a ) (w) = w 2 +1 a2 '

Fe ! e-at or

a w2 + a Recall from Example 5 .6.2 that for a > 0, Fe (e-at) (w) = 1 I ( t ) = __

t2 + a2

�

2 .

0

�e - a lw l .

a

Since I is even, it follows that

lc (w) = .!.. 2a e -a1 wl , w > O .

=

5. The Fourier Transform

Exercises 5 . 1 6 In all the exercises

1.

f

E

L1 (R

349

+) .

MORE ON MODULATION Extend the modulation result 5 . I 6 .2(c) follows: For scalars a > 0 and any show that:

b,

Fe (f (at) cos bt) (w) = 21a [ic ( �) + ic ( W � b ) ] . (b) Fe (f (at) sin bt) (w) = 1a [1. (�) - 1. (W� b) ] . 2 (c) Fs (f (at) cos bt) (w) = 1a [1. (�) + 1. (w�b) ] . 2 (d) Fs (f (at) sin bt) (w) = ;; [ ic (�) - ic (W�b) ] .

as

(a)

2. Use the mo dulation property of Exercise I above to find , for a > 0 and any

b:

Fe ( e- a t sin bt) . Fs ( e-at cos bt) . Find Fe (1�t 2 ) . (a) (b)

3.

4. Assuming that the integrals all exist, show by differentiating under the integral sign (justify) that

Fe (-t 2 f (t) ) (w) = fl/ (W) . (b) Fs (_t 2 f (t) ) (w ) = fll (w) . n ( c ) Fe ( - I t t 2 n f (t)) (w) = jJ 2 ) (w) . n (d) Fs ( - I t t 2n f (t)) (w) = j} 2 ) (w) . (e) Fe ( - It t 2n + 1 f (t)) (w) = j} 2 n + l ) (w) . 5 . Assuming that f' E L1 ( R+ ) and that limt _HlO f (t) = 0 , show that Fs (f' (t)) (w) = -wic (w) . t 6 . Find fo...... oo si�a coswt dt, a > O. (a)

7. Under appropriate smoothness and uniqueness assumptions, use the Fourier cosine transform to solve f" = 0, 2: 0, where f' (0) = (0) 1. Use the Fourier sine transform to solve the same problem.

f =

(t) - f (t)

t

350

5. The Fourier Transform

E Ll (R+ ) , prove that 2lc (W ) ge (w) = Fe [100 l (u) [g (t + u) + g ( l t - u l )] dU] (w) ,

8 . CONVO LUTIO N PRO PERTY For 9

2!s (w) gs (w) . [Hint:

and derive a similar formula for Extend and consider the convolution 9 as even functions and

Ie

ge ,

I and

Take the Fourier transform of both sides of this equation, and use the fact that Ie = = Finally, show that and

(w) 2lc (w) ge (w) 2ge (w). I: ge (t - u) Ie (u) du = 1 00 I (u) (g (t + u) + 9 I t - u l ) du.]

9. INVERSION In equation ( 5.41 ) we noted an inversion result for even

I E L'i (R+ ) is of bounded variation on the closed [a, a < b, and is continuous at t E (a, b), then I (t) = 2- Jo Ie� (w ) cos wt dw. Now suppose that I � 0 decreases on R+ , vanishes at +00, and is integrable on ( 0, a) for all a > O. Show that at any t > 0 : ( a) HI (t - ) + I (t + )] = � J':ooo [ Jo-+ oo I (u) coswu du] coswt dw. ( b ) H/ (t - ) + / (t + )] = � Jo-+ oo [ Jo--+ oo I (u) sin w u du] sin w t dw . [Hint: ( a) By the second mean value theorem Exercise 4.6- 1 0 ( b ) , for all 0 � c < d there is some E (c, d) such that I t I (u) cos wu du l = I / (c) J: cos wu dul � 2/�e) , W > O. This "Cauchy condition" guarantees that Jo--+ oo I (u) cos wu du is uni formly convergent for 0 < a � w � b. Therefore , for any t > 0 , I: [10--+ 00 I(u) coswu du] coswt dw 10-+ 00 I( u) I: cos wu cos wt dw du t 10--+ 00 I(u) (;: cosw (t - u) + cosw (t + u) dw du s a (t u ) I(u) si n tb(t-u 2 Jo u ) m t - u- ] du Si n a (t+ U) ] du o + 2 J(--+o 00 I(u) [ si n tb(t+U) t +u +u extensions: If interval b] , 0 � 7r

-+00

7r

r

1 ( -+ 00 1

_

_

5 . The Fourier Transform

351

Consider any one of these last four integrals for arbitrary 0 < c < d. For example , by the second mean value theorem, Exercise 4.6- 10(b) again , for some s E d) ,

(c,

l Ied I( U ) sin :£t;u ) I du = f(c) l IeS sm :�;u ) 1 du ::; KI (c) d for some constant since I fe s m :£tu- u) I du is bounded (4.5.2(b) ) . K,

K

f (c)

B y hypothesis, for f > 0, < f / for sufficiently large each of the four integrals of the form

I f-· oo f (u) si n :£t; u) I du,

c. Thus,

l Ie--+OO f (u) si n :itu- u) I du, etc.

is less than f for sufficiently large c and all 0 < a ::; selector property of the Fourier kernel 4.6.5,

b. Using the

b�� I I; 1 (u) si n:£t; u) I du = � [ I (r ) + 1 (t + ) ] ,

while by the Riemann-Lebesgue lemma 4.4. 1 ,

.1'" 1 Jrco f (u) si n t+b(t+u u ) du l = 0 blim oo --+

and

(b) The proof is virtually the same as for (a) . In this case, however, since for d > c > 0, there is some instead of we can write d) such that a E

(c,

fo--+ oo

r:ooo ,

I fed f (u) sin wu du l

<

f (c) I cos w e;:;cos wa I WE '

while for some constant I f; (u) sin wu du I ::; I<. From these esti mates of I fo-+oo f (u) sin wu du l and because of the sin w u factor , the w-integral becomes that is, the w-integral is absolutely convergent at the lower limit. This argument fails for the integral in (a) , since the w-integral factor is cos wu.

K,

f fo--+ oo ;

f':ooo

5.17

w

Parseval 's Identities

Our goal in this short section is to show that if

f E L1 (R) n L2 (R) , then

i E L2 (R) . We accomplish this by establishing Parseval's identity for such

f

in 5. 17.2(a) . The following technicality demonstrates the continuity of convolutions of functions 9 E L1 (R) L2 (R) (hardly surprising for an integral).

f,

n

352

5. The Fourier Transform

If f, 9 E L 1 (R) n L 2 (R), then h (t) = l: f (t - x) g (-x) dx = l: f (t + x) g (x) dx is a bounded, continuous, int eg rable function. Proof. With (t) = 9 (t), we have h = f E L 1 (R). The equality of the last two integrals follows by replacing x by - x. By the Holder inequality ( 1 .6.2(c) ) , h is bounded: For any t E R, I h (t) 1 < 1: l f (t + x) g (x) l dx /2 2 /2 < (I: I f (t + x) 1 2 dx r (I: I g (x) 1 dx r II f ll 2 11 g 11 2 . As for the continuity of h, for any r, I h (t + r) - h (t) 1 /2 2 /2 < ( I: I f (t + r + x) - f (t + x) 1 2 dX r ( I: I g (x) 1 dX r /2 = (I: I f (t + r + x) - f (t + x) 1 2 dX r 11 g 11 2 ' /2 0 as r 0 Since f E L 2 (R) , ( I�oo I f (t + r + x) - f (t + x) 1 2 dx f 5 . 1 7. 1 CONTINUITY OF AN INTEGRAL

u

* u

-+

-+

by continuity in the mean 2.8.9. 0

Both of the equalities of 5 . 17.2 are known as Parseval's identity. Compare them to the Hilbert space versions of 3 .3 . 1 and 3.3.4(f) , and to those of Exercise 5 . 13- 15. In regard to the hypothesis, note that any continuous function with compact support belongs to

f

L 1 (R) n L 2 (R).

5 . 1 7.2 PARSEVAL'S IDENTITIES (a) If

f E L l (R) n L 2 (R), then j E L 2 (R), and II f ll � = 2� I � I : . (b) If f, 9 E L 1 (R) n L 2 (R), then I: f (t) g (t) dt = 2� 1: j(w) g (w) dw. Proof. (a) Clearly, (t) = f (-t) E L 1 (R) L 2 (R). Since f, L 2 (R) , by 5 . 1 7 . 1 , h (t) = f * v (t) = 1: f (t + x) f(x) dx v

n

v

E

L 1 (R) n

5 . The Fourier Transform

353

is bounded and continuous. Also, it = jii and ii (w ) = f-( -t) = j(w). Thus it E L1 (R). Therefore, by 5 . 12.4, since h is continuous, atHence, everybyt E5.13.5, R, h(t) = -211"1 1-0000 �h (w) ezwt dw. Hence -211"1 1-0000 �h (w) ezwt dw .

12 .

1 1 00 i (w) e iw t dw. 211" - 00 I

With t = 0,

(b) Let h (t) = l: f (t + x) g (X) dX = l: f(t - x) g (-X) dx . With v (t) = g(-t), we have h = f E LI (R). By (a), j and -g belong to L 2 (R). It therefore follows from Holder's inequality ( 1.6.2) that j-g E L 1 (R). Since h is continuous (5.17.1 ) and it = jg E L1 (R), we can use 5.12.4 to recover h from its transform: * v

h (t)

1 1 00 j(w) g (w) e iw t dw 2 11" - 00

1: (t + x) (x) dx for all t E R. f

9

The desired result now follows by setting t = O. Appli cations Example 5.1 7.3 Show

thai

0

1: (Si: w ) 2 dw = 11".

354

5. The Fourier Transform

Solution . By Example 5 .5 .4, the Fourier transform of

/ = 1 [ - a , a] , a > 0, is

/ E L 1 (R) n L 2 (R), it follows from the Parseval identity 5 . 1 7 .2(a) that j E L 2 (R), and II / II� = 2� I � I: , j (w) = 2 sin aw/w . Since

1 00 ( 2 sin aw ) 2 dw la dt = 2a, w 211' - 00 -a or sin aw ) 2 100 ( -- 00 w dw - lI'a . The special case a = 1 yields sin w ) 2 1-00 ( -:;dw = 11'. 0 �

that is,

=

_

Example 5 . 1 7.4

00

looo ( : ) Si W

Show thai

4

dw =

i'

Sol ution . Consider the Fourier transform (Example 5 .5 .5)

i (w) =

(

)

sin w/2 2 w/2

(t) = - I t I )

L 1 (R) L 2 (R), it j L 2 (R), and 1 � 1 00 ( sin w/2 ) 4 / dw = ( 1 - l t l ) 2 dt = � . 3 w/2 211' - 00 -1

of the Cesaro kernel f 1 [ - 1 , 1 ] ' Since / E (1 follows from Parseval's identity 5 . 1 7 .2(a) that E

n

By a change of variables, it follows that

1: ( : r Si w

Example 5 . 1 7.5

dw

=

Show thai

� or lo oo ( Si:w ) 4 dw = i' [ 00 dw !:. .

Jo

(1 + w2)

2

_

0

4

= e - I t l is i (w ) 2 2/ ( 1 + w ) by Example 5 .6.4. Since / E L 1 ( R ) n L 2 (R), it follows from Parseval's equality 5 . 1 7 .2(a) that i E L 2 (R), and � 100 ( �) 2 1 00 e - 2 lt l

Sol ution . The Fourier transform of the Abel kernel / (t)

211'

- 00

1 +w

dw =

- 00

dt = 1 . 0

=

5. The Fourier Transform

355

sin3 w dw dw = 311" . S Jo w 3 Solution . Let f (t) = 1[ - 1 , 1 ] and 9 (t) = ( 1 - I t /2 1 ) 1[ - 2 , 2]. By Examples 5.2sinw/w, 5 .4 , andand5.5.5g(w)the= Fourier functions are i(w) = 2 w/wtransforms 2 . Both f ofandthese 2 sin 9 belong to L 1 (R)nL 2 (R). Therefore, it follows from Parseval's identity 5.17 .2(b) that Example 5 . 1 7.6

Show t h at

rOO

1 (1 - l t /21 ) dt = � 1 °O 2sinw ( 2sin2 w ) dw. 211" - 00 W w 2 1 Therefore, 2 1 1 (1 - t/2) dt = -32 = -11"2 1-00 sin3w W dw, 00 or 1 00 sin3w w dw = 3411" . 0 - 00 3 Thus,

-1

o

3-

Exercises 5 . 1 7

1. Generalize Example 5.17.4 by showing that for [Hint: Take f (t) = ( I - It I /1' ) 1 [- r , r] (t).]

l'

> 0,

a,

2. Generalize Example 5.17 .5 by showing that for b > 0 , 2 b ( + b) [Hint: Take f (t) = e- a1tl and 9 (t) = e- b 1 t l , and use Parse val's iden tity 5. 1 7 . 2 (b).] 3. Show that: sin5 1'w w = --11511"1'4 lor >- 0 . (a) 1o 00 --w5 384 sin6 1'w dw = -1 1 11"1'5 for >- 0 . (b) 1 00 -w6 40 a

d

o

l'

l'

l'

a

'

356

5.18

5. The Fourier Transform

The L2 Theory

For finite intervals (Exercise 4.4- 1 ) :

Lp (R)

[a, b], the bigger gets, the smaller Lp [a, bj becomes q > p L q [a, bj Lp [a , bj . =?

p

C

The spaces do not descend as p increases. For example , with U denoting the unit step function,

I (t) = � U (t - 1) belongs to L 2 (R), but not to L 1 (R). Hence, even though a function I E L 2 (R), the Fourier transform integral I: I(t) e -iwt dt I L 2 (R),

need not exist . To define a Fourier transform for functions E we must therefore proceed differently. The idea of the Fourier transform for functions is classic analysis: We sneak up on it . We choose a dense subset X of norm (dense in the , we on which the Fourier transform is defined. Then , for E select a sequence from such that fn --+ and define to be limn In · Problems? Is well-defined? If E , does this new agree with what we get by the usual method of calculation? First , for the dense subset of (R) on which the Fourier transform is already defined, the obvious choice is but is it dense in )? It is, because contains the continuous functions n Cc with compact support, and Cc is dense in Some other imp ortant dense subsets are considered in Exercise 5 . 1 9- l . For let

L2

L 2 (R) L2 I L 2 (R) I, j (In) X j I L 1 (R) n L 2 (R) j L2 L 1 (R) n L 2 (R), L 2 (R L 1 (R) L 2 (R) (R) (R) L 2 (R). I E L 2 (R), In (t) = I (t) l[ -n ,n ] (t) , n E N . As functions with compact support, each of the In belong to L 1 (R) L 2 (R). Since In - I E L 2 (R) , it is also clear that I l /n - 1 11 2 o. 11 - 11 2 )

--+

n

We show next the crucial fact that the sequence verges to a function in (R) .

(In) of transforms con

The sequence

constructed above

L2

5 . 1 8 . 1 CON VERG ENCE O F

(Tn)

converges to a function in L 2 (R) .

(Tn)

5. The Fourier Transform Proof. Since each In belongs to

357

L 1 (R) n L 2 (R), each in is in L 2 (R) by

(In)

Parseval's identity 5.17.2(a) . We will show that is Cauchy in the Hilbert space L 2 (R). Since 1m--::-In = 1:. - In, by Parseval's identity,

For m > n ,

and this approaches 0 as m , n -+ 00 . 0 We now define the FOURIER TRANSFO RM j of I 11 1 1 2-limit j = lim n In.

E L 2 (R)

to be the (5.42)

Defined this way, j is also called the PLAN CHEREL TRAN SFORM . To em phasize that the limit is computed with respect to 1 1 · l b , we denote it by l.i.m . for "limit in the mean." Thus, j = l.i.m.n

i- n f (t) e- iw t dt. n

Since 1 I · l b -limits are unique only in the sense of equality almost everywhere, it is important to note that this defines only j (w ) almost everywhere. This quirk occasionally causes some difficulties . There is nothing special about the sequence (fn ) of truncated f's. Any sequence from L 1 (R) nL 2 (R) that converges to f may be used to compute j as the following result demonstrates. 5 . 18.2 WELL-DEFINED If (fn ) and ( g n ) are sequences from L 1 (R) n L 2 (R) that converge to f E L 2 (R), i. e., l.i.m .n In = l.i.m .n gn , then

l.i.m-n In = l.i.m .n g,;.

Proof. By Parse val's identity 5.17.2(a) ,

Since

358

5. The Fourier Transform

it follows that

1 I y,;- - Tnt O. Therefore, l.i.m.n Tn = l.i.m .n y,;-. ---+

0

To See that this mode of computation caU SeS no change in the Fourier transform for functions E calculated in the usual way, consider I(t) e - iw t dt . =

j

I L 1 (R) n L2 (R),

j(w)

I:

In (t) (t)I1[- n,In jfor(t) all, nnEENN. Byandthealmost dominated convergence theo every w E R, n :::; OO j(w) = limn j- 00 In ( t ) e - iwt dt.

=I Let rem 4.4.2, since

This being a pointwise limi t , i t says that Tn ---+ pointwise a.e. A s we note in Exercise 2, mean convergence of a sequence of functions implies the convergence of a subsequence pointwise a.e. to the same limit . Since any subsequence of a pointwise convergent sequence converges to the original limit, it follows that We get the same transform a.e. by either method of computation. Passage to the limit enables us to get versions of many results for functions. Theorem 5 . 18.3 illustrates the procedure. Note that 5 . 1 8 .3(a) establishes the continuity of the linear map 1-+ of into indeed, Were it not for the 27r, it would establish an isometry.

j

L2

L1

I

j

L2

L2 ;

5.18.3 PARSEVAL'S IDENTITIES FOR L 2 FUN CTIO NS (a) If I E L 2 (R), then I / I � = 2� I �I: . (b) If I, 9 E L 2 (R) , then I: I(t)g (t) dt = 2� I: j(w) g (w) dw. (a) Let In (t) = I (t) 1[ - n , n j (t), n E N . Since l j - l 1 I Tn 2 0 , it follows from the second triangle inequality that O . Since l 1 2 1 12 IITn � In E L 1 (R) n L2 (R), it follows from Parseval's identity, 5 . 17.2(a) , that Proof.

---+

---+

Hence, by continuity of the norm,

h�. l /nl1 22 = 1 / 1 22 = j�· 27r1 1 _In 1 22 = 27r1 I 11� 1 22 '

5. The Fourier Transform

359

1.4-3,

(b) By Exercise we can express an inner product by means of the polarization identity; hence , by (a) ,

(f, g )

=

I: I (t) g(t) dt � (II I + g ll� - II I - g ll� + i II I + ig ll� - i II I - ig ll�) + i l i + i9 1 l : - i l i - i g l l : ) �12�/ �( l I i) + g l1: 1- 00Ii i �- gi l :211" \

I, g = 2 11"

- 00

I (w) ( g (w)) dw.

0

Ll

Another basic property of the Fourier transform of functions that survives for functions is the change of roof property of 5 .5 . 1 ( e) .

L2

5. 1 8.4 CHANGE O F RO OF FOR L2 FUN CTIONS If I, g E L 2 (R) then I: l (t) Y(t) dt = i: f(i)g (t ) dt.

n E N , let fn (t) = f (t) 1[-n,n ] (t) , n E N , and gn (t) = g (t) 1 [-n,n ] (t) , n E N . Since II g - gn ll 2 --+ 0, it follows from the Holder inequality (1.6.2) that for Proof. For

I i: j;. (t) gn (t) dt - I: J::. (t) g (t) dt l

any m ,

<

I: IJ::. (t) (gn (t) dt - g (t)) 1 dt

<

1 I J::. 1I 2 11 gn - g li 2 --+ ° as n --+

00 .

Hence , for any m E N , li� Similarly, since

I: j;. (t) gn (t) dt = I: J::. (t) g (t) dt.

I i - j;. 1 2 0, for every n, --+

I: r;. (t)

(t) dt = I: i {t) gn (t) dt. Since In and gn belong to L l (R) n L2 (R), it follows from 5 . 5 . 1(e) that I: 1m (t) g;; (t) dt = 1: j;. (t) gn (t) dt for all m , n E N . l�

gn

360

5. The Fourier Transform

Therefore , for every n, limm

I: 1

m

I: I (t) (t) dt I: 1m (t) gn (t) dt I: j (t) (t) dt.

(t) 9,; (t) dt

9,;

limm

gn

In other words, for every n,

I: I (t)

9,;

(t) dt = I: 1 (t) (t) dt, gn

and it remains only to take the limit as n --+ 00 . 0

Exercises 5 . 18 1.

L2

L1 I L2

TRANSFORM BASICS Many familiar properties of the Fourier transform survive for functions. The results of ( a) and ( b ) , for example , are trivial; in ( d ) , we have to take a limit. For E ( R) , show that :

( a) (b) () c

(d)

(e )

L2

CON J U GATES

7 (t) (w) = j( -w).

I{-:t) (w) = j( -w). SC ALIN G For a # 0, f(!;t) (w) = ( II l a D j(wla) . For any r E R show that j (w) = l.i . m . n f�:�r I (y) e - iw y dy. TRANSLATES For any r E R show that I (t+ r ) (w) = eiw r j(w). [Hint : For n E N , let hn (t) = I (t + ) 1[ - n,n ] (t) . Then REFLECTIONS

r

Now take the limit in the mean as n --+ 00, and use (d) .]

E ( R) , � 2. NORM AND POINTWISE CON VERGENCE For and if 1, E show that if --+ with respect to --+ 9 a.e. , then = 9 a.e. [Hint: If , then --+ with respect to such --+ in measure ; therefore there exists a subsequence that --+ a.e. Since --+ 9 a.e . , = 9 a.e.]

n N, I In I Ink I

In I

Ink

In

I I

I, In Lp p In II lip I lip (Ink)

5. The Fourier Transform

5 . 19

36 1

The Plancherel Theorem

L2 5.19.2).

We need a way to invert Fourier transforms. The following result essen tially provides it (see The reciprocity theorem says, in essence, that. if you take the conjugate transform twice, you return to the original function

j (w)

f.

5 . 1 9 . 1 THE RECIPRO CITY THEOREM

then a s elements of L 2 (R) ,

If f E

L 2 (R)

and 9 (w) =

j(w),

f (t) = 211"1 g (t) . -

Proof. Consider

1: (f (W) - 2� g (w)) (7 (w) - 2� g (w)) dw Il f ll � - 2� 1: f (w) g (w) dw - 2� 1: f (w) g (w) dw 4�2 IIglI � · +

By the change of roof property 5 . 18.4, the definition of identity 5. 18.3( a) ,

1: f (w) g (w) dw

( 5 .43 ) g , and Parseval's

1: j(w) g (w) dw

1: j(w) j(w) dw I l jl l :

211" " f"� . Consequently,

1: f (w) g (w) dw 1: f (w) g (w) dw =

Now,

I I gll ;

=

211" II f ll � ·

2 11" II g II;

[Parseval 5 . 1 8 .3( a))

411" 2 11 f ll�

[Parse val 5 . 1 8 .3(a)) .

362

5 . The Fourier Transform

Thus, by equation

(5.43), 11 / 11 22 271"1 271" 11 1 11 22 271"1 271" 11 / 11 22 + 4�2 471"2 11 / 11 ; -

-

o.

I (w) = (1/271") g (w) as elements of L 2 (R). 0 Now we can prove the inversion theorem for L 2 functions. 5.19.2 INVERSION THEOREM FOR L 2 FUNCTIONS II I E L 2 (R), then n I (t) = l .i . m .n 2. j j (w) e iw t dw. 271"

Therefore

Proof. Let 9 (w)

= j(w). B y 5 . 1 9 . 1 ,

-n

1 -g (t) 271" n 1 I .l.m· n j- 9 (w ) e - iw t uW 2 71" n iw 1 I. 271" .l.m ·n i-nn 9 (w ) e t uW 1i � w . I .l.m ·n 271" - I (W ) e i t uw. 0 n The Fourier transform for L 2 functions has more symmetry than that for L 1 functions. For example (5.12.6), the Fourier map F L 1 (R) Co (R) , I j, is not onto. The L 2 version F : L 2 (R) L 2 LR) , I (t)

.

J

-

J

J

�

:

�

I

-+

I->

I,

[1(:) ] . By

I E L 2 (R), and let h (w) = the reciprocity theorem 5 1 9 1 , 1 = (1/271") h or 1 = (1/271") h = h/271". is surjective. To see why, consider .

.

L 9 2 L2

The Fourier transform for functions is also injective by the inversion theorem for if I and are functions with (almost everywhere) equal transforms and then

5.19.2,

j g, I (t) = l.i.m.n 2� I: j(w) e iw t dw = 9 (t) . The aggregate of these results about the L 2 Fourier transform is usually

called Plancherel's theorem.

5 . The Fourier Transform 5 . 1 9.3 PLAN CHEREL ' S THEOREM

5. 18,

The

L2

363

Fourier transform of Section

F : L 2 (R) f

j(w ) = Li.m ·n 1: f(t)e -iw t dt,

where

has the following properties. (a) F is a continu ous linear bijection (5.18.3 , and

5.19.1). 5 .18.3 ) (j, y) = 2 7r {!, g) or l � 1 2 = .)2; lI fI 1 2 • (c) (5.18. 2 ) If f E L l (R) n L 2 (R), then the Fourier transform j is the same whether computed as an L l function or an L 2 function. (d) ( Inversion 5.19. 2 ) n 1 it f (t) = Li . m .n � 27r - n j (w ) e w dw . Now you can see why the Fourier transform f E L l (R) is frequently defined 1 1 00 f(t)e - l.w t dt. f (w ) = V2i -00 This sacrifice for symmetry places a 1/V2i in front of the integral in the inversion theorems instead of 1/ 27r and makes the Fourier map F for L 2 functions a linear isometry-hence a Hilbert space isomorphism by 1.5.3. (b) (Parseval' s identities

as

�

Since surjective linear isometries of complex inner product spaces are called unitary operators, this version is sometimes referred to as the UNITARY FOURIER TRANSFORM of functions. For this formulation we have

L2

n � 1- n f(t)e - iw t dt j(w ) = l.i.m.n v27r n � - n j (w ) eiw t dw . f (t) = l.i.m .n v27r

and

1

Exercises 5 . 1 9

1.

S f E N U {O},

Let (R) denote L. Schwartz 's space of infinitely differentiable func tions on R that are RA PIDLY DECREASIN G in the sense that for all

m, n

I t m JC n ) (t) 1 < ER

sup

t

00 ,

where the superscript ( n ) denotes the nth derivative . The subspace C S (R) of infinitely differentiable functions on R that vanish

D (R)

364

5. The Fourier Transform

outside some closed (bounded) interval is dense in for p ?: 1 ; therefore, is dense in for p ?: 1 , too (see, for example, Szmydt 1977) . Since is dense in we could use in place of n in the definition of the Fourier transform for functions. Show that :

S (R) Lp (R) S (R) L 1 (R) L 2 (R)

L2

Lp (R)

L2 (R),

S (R)

I E S(R),2 then tm f(n ) (t) -+ 0 I t l -+ indeed, I tm I(n ) (t)1 Kj ( 1 + t ) for some constant K . ( b ) It follows from (a) that tm f( n ) (t) belongs to L 1 (R) for all m, E NU {O} . By 5.15.3 on derivatives of transforms, it follows that i is infinitely differentiable , and dn j(w)n = / 00 (- it t f(t) e - iwt dt. dw - 00 Show that i E S (R). [Hint: Consider I(m) (t) E L 1 ( R); f(m ) (t) -+ 0 as I t I -+ 00. Thus, by 5.15.1 on the transform of a derivative, the Fourier transform of I(m) (t) is (iw)m i(w), and (a) If

as

<

00 ;

n

But

dn i(w)n / 00 (- itt I(t) e - iwt dt. dw - 00 =

Hence

(c)

iwm (i) (m) (w) i

�

M for some constant M.] Show that the map f i of S into S is surjective . [Hint: By (a) and (b) , fJ E L 1 ( R) , and f is continuous. By the inversion theorem for I such that i E L 1 ( R ) , 5 . 1 2.4, Therefore,

>-+

so

f(t) = -211!' 1-0000 f� (w) e'.Wt dw, e -iyt dy. ] f (t) / 00 j(-y) 2 =

(d) For

- 00

I, 9 E S ( R) , verify that :

1!'

5. The Fourier Transform

II / I I � =

I I

365

�:· I (t) (t) dt 2� f�oo l (w) yew) dw. f� oo [Hint: Use Parseval ' s identity 5.17.2.] (e ) Prove that if I, E S (R) , then 1 * E S (R) . [Hint: Since I, E (R) , it follows from (b) that i, y E S (R) . It is straightforward to verify that i · y E S (R) . Now 1· y i.

ii.

21"

9

=

9 9 S

9

=

2.

r:;g E S (R) . It follows from the inversion theorem for I such that 1 E L 1 (R) , 5.12.4, that 1 * 9 E S (R).] (f) Use the dense set S (R) of L 2 (R) to define the Fourier transform i of I E L 2 (R) , and then establish Plancherel's theorem 5 . 19 . 3.

INVERSION AND DERIVATIVES Let L 2 (R) . Show that

IE

i be

the Fourier transform of

I� (w) = -ddw 1-0000 I(t) 1 - zte. - iwt dt a.e. and I(t) = -211r dtd 1-0000 I(w) e;wtzw. - 1 dw a.e . [Hint: Let In (t) = I (t) l[- n ,n ] (t). Then Tn (w) = I: In (t) e - iw t dt and f; f�oox Inn (t)(t) ee -- iiww tt dwdt dwdt f�00oo fo I 1 - e -itx f- oo In (t) it dt. Since inner products ( t , g) are generally continuous ( Example 2.2.5) and ( 1 - e -iw t ) / iw E L 2 (R) , conclude that l x Tn dw = lim 100 I - �- itx dt, lim n o (w) n n (t) zt or lx 1 00 I (t) - �-itx dt , i (w) dw o - 00 which implies d 1 00 I(t) 1 - e. - itx dt a.e . I� (x) = dx 00 zt �

1

- 00

=

-

1

d

The second part follows by a similar argument and Plancherel 's the orem 5 . 1 9 .3 d) .

( ]

366

5. The Fourier Transform

!.:: j

3. THE UNITARY FOURIER TRAN SFORM Let n = l.i.m.n f(t) e - iw t v 21r - n be the unitary Fourier transform of f E L 2 (R) discussed at the end of this section. Its inverse is given by, for 9 E L 2 (R) , n e i w t dw . (t) = l.i.m. n 9 v 21r - n Show that = 1 , the identity map = and that . L L onto of (R) f (R) 1-+ 2 2

Gf (w)

(G - 1 g ) Gf (t) (G - 1 f) (-t)

f

dt

!.:: j

(w) G4

4. SINE AND COSINE TRANSFO RMS In Section 5 . 1 6 we defined the Fourier sine and cosine transforms lc for E L1 (R+ ) and 1. (real-valued) . Now let f E L2 (R+ ) , define = f ( t ) l [ O,n ] , and let = Li .m .n rn f ( ) cos = l.i.m.n [Tnt,

(w)

Fe (w)

(w) f fn (t) t wt dt

[Me

fn

where denotes the usual Fourier cosine transform of E Ll ( R + ) n L 2 (R + ) . Show that the limit i n the mean exists and defines as an element of L 2 (R+ ) (hence a.e.) . Do the corresponding develop ment for the Fourier sine transform Fs In particular, show that :

(w) L It f (t) SIntt dt . t �1t Iooo Fe (w) s, :w t dw . Fs (w) d: It f (t) 1 - c�s wt dt . (t) �1t It Fs (w) l - c;s wt dw.

Fe (w)

(w).

= ( a) Fe (b) f ( ) = = (c) = (d) f

[Hint: Apply the corresponding results for the Fourier transform of f E L 2 (R) in the case where is even, and then odd. ]

f

5 . 20

Pointwise Inversion and Surnmability

f

Note that a complex-valued function is of BOUNDED VARIATIO N if and only if the real and imaginary parts of f are of bounded variation. In 5 .20.2 and 5 .20.3 we get analogues for L 2 functions of the following theorems. 5 . 9 . 1 If f E L1 (R) is of bounded variation in some neighbor hood [t - s, t + s] , s > of E R, then

0, t

f {t - ) + f(t+ )

2

1 ei tw dw PV I�oo 2 1r P limP-+OO l 2 ,.. I- p f e i tw dw .

j(w )

�

(w)

5. The Fourier Transform

367

5 . 12.3 (Fejer-Lebesgue theorem) At any Lebesgue point t of E L r (R), � 1: eiw t [ (w ) dw (C, I) I (t) 2 -d- l r ( 1 - tir ) eiwt [(w ) dw limr .... oo 1r -r = limr .... oo � (f * J{r ) (t) 2 where k (t) denotes the Cesaro kernel (1 - It I) 1 [ - 1 , 1] (t), and I<.r (w ) = rk ( rw ) . 1

The key ingredients of our proofs of the inversion theorems for Fourier series and Fourier transforms of 1 L1 (T ) or 1 L1 were the ability to split a certain integral I�oo into a left, a middle, and a right oo part, say I�: + I� r + Ir . The left and right integrals go to 0 by the Riemann-Lebesgue lemma (4.4. 1 ) , while the middle one yields something like (t - ) + by the selector property of some kernel (Dirichlet, Fejer, Fourier) . As we show in equations (5 .44) and (5.45 ) , it happens that if 1 (x) / (1 + Ix l ) E L1 (R) , then we retain these key features. The imp or tance of this for inversion theorems for L2 functions is that if 1 E L2 then (x) / ( 1 + I x l ) E L1 (R) . In the proof of 5 .9 . 1 , in the integral

E

[I

E

(R)

1 (t + )] /2

(R),

f

.!. 1°O 1r

we moved the "x -

- 00

I (x)

X

-t

sin p (x - t)

dx,

t" underneath f (x) to get 1 (x)

x-t N ow suppose that

I (x) E L I ( R) . 1 + Ix l

Certainly, this assumption implies that 1 is locally integrable , i.e. , inte grable in any finite interval Moreover, since for any t ,

[a, b].

x+l

it is clear that

(1

-- � x-t

/

1,

as x � ±oo,

2 for sufficiently large I x l , or that I x � t I < 1 :Ixl

+ Ix l ) I x - t l <

368

5. The Fourier Transform

II(x)(x) (x+-I xt)l ); hence is integrable on t I-x l . , it follows 00 f (x) sin p ( x - t) dx = 0 and lim ] t - 8 I (x) sin p (x - t) dx = O. 1 lim p -+ oo t 8 x - t (5.44) x-t P -+ OO - 00 + Ifproperty f is of bounded variation on [t - s, t + s] , s > 0, then using the selector of the Fourier kernel

0, then so is / from the Riemann-Leb esgue lemma 4.4. 1 that

[t +

n

have that 4.6.5,

p

t +s f (x) sin p (x - t) dx = f (t - ) + f (t+) . (5.45) .!. I p -+ oo 7r t 8 2 -t In summary, we have reestablished the selector property of the Fourier kernel for f such that f (x) / (1 + I x l ) E L 1 (R). This is the first step toward establishing the inversion theorem 5 .20.2 for L 2 functions. 5.20.1 SELECTOR PRO PERTY OF THE FO URIER KERNEL If f is of bounded variation in some neighborhood [t - s, t + s] , s > 0, of t E R , and I(x) 1 + I x l E L dR) , then f(r ) + f (t+) = plimoo .!. j OO I(x) sin p(x - t) dx . 2 -+ 7r - 00 -t Now suppose that f E L 2 (R). Since 1 / ( 1 + I x l ) E L 2 (R) and products of L 2 functions are integrable (i .e . , L I), it follows that I(x) E L d R) . 1 + Ix l lim

X

X

This yields the following.

L2 EFUNR, CTIO NS If f E L 2 (R) is of bounded vari then 1 P � + f (t+)--,1 f (t...!. - )_:... � ezw t dw. OO f(w) t ezw PV= dw ...o.. = plimoo - j f(w) ::...--<. J 27r _ 27r 2 -+ p -00 By Parseval ' s identity 5.18.3(b) for L 2 functions, I: I(x)g(x)dx = 2� I: j (w) g (w) dw . Replacing g (x) by g ( -x) and using Exercise 5 . 18-1(a, b ) , we get ]-00 I (x)g (-x) dx = 1 ] 00 I(w) � g (w) dw. (5 .46) 27r - 00 00 5.20.2 INVERSION FOR

ation in a neighborhood of t

Proof.

.

.

5. The Fourier Transform

369

L 2 functions is onto (5. 19.3(a) ) , if g (w) = e iwt l [_ p ,p] (w) E L 2 (R) , p > 0, t E R,

Since the Fourier transform for

we can recover 9 by 5 . 1 9 .3( d)

as

I

1

1

By equation (5.46 ) , this means that 00 f (x) sin p (t - x) dx = 1r (t x) 00

dw

1 �w J... j j (w) 1 21r

_

-

w

w

sin p (x + t) . 1r (x + t)

=

1

n

. e iwt 1 [ - p , p] ( ) eiw x d I .l.m ·n 2 1r - n J... P e iw (x + t ) 21r _ p

9 (x)

00 f ( ) e iw t 1 [ _ p, p] ( ) dw 00 P

w

-

21r _ P

e iwt

dw.

Since sin py/y is an even function, the result follows from the selector prop erty of the Fourier kernel 5.20 . 1 :

OO f (x) sin p (x - t) dx = f (t - ) + f (t + ) . .!. j P-+OO 1r -t 2

X 00 Next, we consider the version of the Fejer-Lebesgue theorem 5 . 1 2 .3 . We begin b y considering a "partial integral" Sr (t ) as we did i n Section 5.12, but for f E (R):

lim

-

L2

L2

Sr (t) = J...

l ( 1'1)r r

21r - r

Let

( I� I )

g (w) = 1 By 5 . 1 9 .3(d) ,

1-

e iw t l[ _ r , r]

(w) E L 1 (R) n L 2 (R) , t E R.

1-

Li.m. n

r2

2 1r

-r

1-

r

-r

1-

r

e iwt l[ _ r , r ]

dw. r > 0,

[

e - iw (x+t) d - r Sin r (x + t) /2 r ( x + t ) /2

w

_

(w) e iw x dw

ei w t e iwx

r

As in the proof of 5 . 1 2 . 1 , it follows that fo r

l ( 1'1)

jw

e iwt ( ) dw , r > O.

� I: ( I� I ) J... j ( 1'1)

9 (x)

0

]2

T ( X + t) , - Hr _

/

370

5. The Fourier Transform

Kr ( u) =

( ru/2) [r (u / 2) 2 ]

where r sin 2 I kernel of Section Thus,

5.12.

denotes the continuous rth Fejer

2 r(x+t l 1 Kr (x + t) . -1r4 sinr (x + 2t) 2 = 2 1r Hence , by equation (5.46), for I E L 2 (R) , r � J Sr (t) 21r ( 1 - �r2 )r eiw t !(w ) dw 1 00 I (x) -4 sin ( -x2 + t2l dx - 00 oo 1r r (-x + t) 1 21r J 00 l (x) Kr (-x + t) dx.

9 (t) =

-r

(5.47)

-

(5.28) 5.12 I E L 1 (R) 1 1 00 1 Sr (t) = 2 1r (f * Kr ) (t) . 2 1r 00 l (x) Kr (-x + t) dx = Since I (x) I (1 + I x l ) E LdR), analogously to how we obtained 5.20.2 we can make some minor adjustments in the proof of the Fejer-Lebesgue inversion theorem 5.12.3 for L 1 functions to conclude that Sr (t) -- I (t) at every Lebesgue point t of f. In summary: 5.20.3 FEn�R-LEBESGUE INVERSION FOR L2 FUNCTIONS If f E L 2 (R), then at any Lebesgue point t E R of I, Iw l ) I� (w ) ezw. t dw = lim -1 (f * Kp ) (t) . I (t) = plim -1 P ( 1 - p ..... oo 2 1r ..... oo 2 1r I_p P In other words, we have arrived at the same point we did in equation of Section for , that -

Exercises 5 . 20

1.

Lp

CONVOLUTION BETWEEN L 1 AND FUN CTIO NS For and 9 1 < p < 00 , show that for almost every fu-n ction w = (x belongs to ,

E Lp (R), (y) I - y) g ( y)

f E L l (R) x E R, the

L 1 (R)

and

(f * g) (x) = J�oo I (x - y) 9 (y) dy belongs to Lp (R) , and also that II I * g ll p � 11/11 1 II g ll p ' where lip + l / q = 1. [Hint: Let h E L q (R), so that gh E L 1 (R) by the Holder inequality (1.6.2). In

order to evaluate

J�oo J�oo I I (x - y) 9 (y) h (x) 1 dx dy

5. The Fourier Transform

by Tonelli's theorem 4.19.2, with

u = x - y,

371

consider

by Holder's inequality with v = - u ) . This is clearly equal # 0 for (R) such that h to . Now choose any R. Then we see by the above that is finite a.e . , so (R) . It is also clear that the linear functional

(x) g (x h E Lq (x) II f ll 1 11 g ll p Il h ll q xE f�ooo I f (x - y) g (y) I dy f (x - y) g (y) E L l

C, F : L q (R) ) (x) h (x) dx, h (f � g * f oo is bounded, and Il F ll q :S Il f l1 1 1l g ll p . Since the continuous dual of L q (R) is Lp (R), it follows that f * g E Lp (R) and II I * g ll p :S 11 1 11 1 II g l i p . ] 2. A CON VOLUTIO N THEOREM Let I E L 1 (R) and g E L 2 (R) . Show that h = 1 * g E L 2 (R) , and that h (w) = j(w) g (w) . [Hint: By the preceding exercise with p = 2 , it follows that h E L 2 (R) . For gn (t) = g (t) I [ - n,n] (t), II g - gn l1 2 ---+ 0, and l.i.m .ng,;- = g. Now let hn = f * gn. Since gn and hn belong to L 1 (R) n L 2 (R) , -+

1-+

it follows that

-;;;, = jgn .

By the preceding exercise,

Therefore , by the Parseval identity 5 . 1 8 . 3 ,

Since

I � E Co (R) , I� i s bounded b y some number M , and 1 -;;;, - jg l 2

I jg,;- - jg ll

:S M 11 g,;- - gl1 22 ---+ 0 as n ---+ 00 .

=

Combining these results, we get

a.e .]

II h - jg ll 2

L2

=

0 so

h (w) = j (w) g (w)

3 . SUMMABILITY KERN ELS AND FUN CTIO NS Here is a version of 5 . 1 3 . 4 on summability kernels for ( R) be functions. Let k such that k (R) n (R) is an even function with

E L1

L2

1 2 11'

00

L2

L oo k (w) dw

f, E L 2

�

=

1.

372

5. The Fourier Transform

[{rex) = rk (rx), r > 0 , show that 11 (1/27r) (f [{r) - 1 11 2 -+ 0 r -+ and

With as

*

00 ,

[ Hint: By Exercise 5 . 19- 1 and 5 . 1 8 .4 we have

J�oo 1 (t + y) [{r ( y) dy 2� J� 1 (t - y ) [{r ( y ) dy 2�� (f oo [{r ) (t) a.e . , 2 *

where 2�

(f [{r ) E L 2 (R) by Exercise 1 . The fact that *

follows from Exercise 5 . 1 4- 1 .]

5.21

A S ampling Theorem

(tn), E N ,

If you know infinitely many values 1 n of a function, do you know I ? Of course not. There are obviously infinitely many distinct ways of connecting the points so that a function is defined by the ensuing graph. Nevertheless, there are many ways in which a denumerable amount of information about a function suffices to determine it: If 1 is a sufficiently smooth integrable periodic function, for example, and you know the Fourier coefficients ; or if 1 is analytic in a region and you know all its derivatives at some point Zo Thus, there is some precedent to believe that if you know something about the character of 1 (integrable , periodic, analytic) as well as a denumerable amount of information, then 1 can be reconstituted. In an intriguing application of transform theory we show in 5 .2 1 . 1 that if 1 (R) is band-limited in the sense of Definition that vanishes outside some interval [- K , K] , then 1 can be recovered from knowledge of the values 1 K ) , by means of

(tn , / {tn))

E D.

j ew)

E L2

D,

L2

5.7.1,

(n7r/ ( n7r ) sin ( [{t - n7r) . 1 (t) = '" 1 L..J n EZ [{ Kt - n7r

Theorems like this-recovering 1 from certain o f its values-are known as sampling theorems (also interpolation theory) , and they go back a long way. The result of this section was essentially first discovered by Cauchy in 1841 , then rediscovered by Whittaker in 1 9 15 . Shannon made remarkable applications of sampling theory to communication theory ( 1 948 ) . Beutler 's 1961 article is of interest in this evolving and imp ortant area.

5. The Fourier Transform

373

5.21 . 1 SAMPLING THEOREM If 1 E L 2 (R) is continuous and band-limited, > K for some constant K, then 1 is determined by its

i (w) = 0 for I w l

values at a discrete set of points:

sin �f{t f (t) = L 1 (n�) l\ t - mr 1\

mr

)

nEZ

.

Proof. Since i (w) is band-limited (of compact support) , i E L 1 (R) n L 2 (R) , and by the inversion theorem for L 2 functions ( 5 . 1 9 . 2) , 1 n � 1.m 1 (t) n 211" - n I (w ) eiwt dw (5 .48) K 1 ! (w) e iwt dw a.e. (i E L I (R) ) . 2 11" .

l.

.

j

-

j

-K

Since J!:K i (w) e iwt dw defines a continuous function of t (see the proof of 5.7.2 that i (w) is continuous) , and since 1 is continuous, equation (5 .48) holds for all t . Since i E L 2 [-K , KJ , we can expand it in a complex Fourier series

i (w) = where

K

�

.

�

( ;;) = ;1 ( ;; ) .

By equation (5.48 ) , however, 211"

so

(5 .49)

n EZ

2\ j

dn = 1 ? } 1 -

L dn e - in rr w /K ,

-K

i ( w ) e i n rrw/ K dw .

j-KK I (w) e,wn rr/K dw =

dn = { 2 11" 1 2

1

(n1l" -) , K

Substituting the Fourier series of equation (5 .49) into equation (5.48) yields 1

(t)

j

1 K � . 1 (w ) e, wt dw 2 11" - K 1 / � - wt (inner product) 1, e , 211" \ dn e- i n rr w / K , e -iwt . /" 211" \ L...- n E Z

�

.

)

)

374

5. The Fourier Transform

Since the inner product is continuous (Example 2.2.5(b) ) ,

f (t)

d ( e -i m r w /K , e -i wt ) J.-211" " L...t n EZ n K 1 __ " n EZ f (�) j- e iw (K t -mr)/K dw K 2K L...t K ) ( e i w( K t - n 1r) /K K n1l" 1 f [( i ([(tn1l")- n1l") I -K 2[( L n EZ K n1l" ([(t sin ) ( " f . L...t n E Z

K

0

K t - n1l"

The quantity aT = 11"/ [( is called the NYQUIST INTERVAL , its reciprocal v = [(/11" is the NYQ UIST RATE , and the result is often written in terms of [( [(

" L...t

them as

naT) f (t) = f (naT) sin (t (t--naT) .

nEZ

fE

f

Suppose that L 2 (R) is time-limited rather than band-limited: (t) = It I > K for some real number K . Then f E Ll (R) n L 2 (R), and it is easy to modify the preceding argumen t to show that is determined by its values at = n1l" / K, n E

o for

i (w)

Z.

w

Another Approach

There is another way to view the sampling theorem that is j ust irre sistible. Consider the functions 11" -i n1r w /K l [ -K , K] ( ) e gn (W )

- - /{

w.

_

Then

n .l.m ·n 211r j n U-n (W ) e iwt uW J.- j K � e -i n1rw /K e i wt dw 211" /{

Un (t)

I.

J

-K

sin (Kt - n1l") /{t - n1l"

In other words, the quantities si nk��1r ) are j ust the inverse Fourier trans forms of the g.;. Since the e -i n 1r w /K are an orthogonal basis for L 2 [- K, K] , it follows from Plancherel's theorem 5 . 19.3 that the sin �-t -n n1r) 1r constitute an

k

orthogonal basis for B = E L 2 (R) = 0 for > K . The sam pling theorem then gives a pointwise expansion for the continuous members of B in terms of this basis as well as 1 1 1 1 2 -convergence.

{f

: i(w)

Iw l

}

5. The Fourier Transform

5 . 22

375

The Mellin Transform

f (t) e - iwt. 1 t8 - f )

The Fourier transform involves integrating The Mellin trans form is concerned with integrals of ( t , t E e , i.e . , with "moments" of I t resembles the Fourier transform in some respects, and because it involves moments , it has many applications in mechanics and statistics. For those familiar with the idea, we mention that the Mellin transform of is the same as the bilateral Laplace transform of

f.

f (t)

f (e - t ) .

Definition 5.22 . 1 THE MELLIN TRANSFORM

f and the complex number s are such that (M f) ( s) = f (t) t s - 1 dt exists, then M f is the MELLIN TRANSFORM of f. More generally, if f is integrable over any finite interval ( a , b), 0 < a < b, and f (t) t s - 1 dt (M f) ( s) = converges for some complex number s r + iv , we take M f to be the MELLIN TRANSFORM of f. As equation (5 .5 1 ) below illustrates , M f can be expressed in terms of If the function

100

1 -+0 00 -+

0

=

Fourier transforms. For sufficiently well behaved functions, we have a fairly symmetric inversion formula.

5.22.2 INVERSION OF MELLIN TRANSFORM Let s = r iv denote a complex variable, let f be of bounded variation in a neighborhood of t E R, and suppose that r - 1 f ( ) E L 1 ( R+ ) for some r E R. For s such that -

x

x

(5.50)

exists,

Proof. With

f (t-) ; f (t + ) = -1 .

t = eY, and B (Mf) (r

2

= r -

- iv

iv)

lr+ia (Mf)

11"1 a-+ oo r - i a

=

lim

(s )

r s dB.

(r as above) in equation ( 5 . 50) , we have (5.51) 1: f( eY)erYe-iy'IJ dy,

which i s the Fourier transform w ( v ) of w bounded variation in a neighborhood of =

(y) = f(eY )ery . Since f i s of t eY, it follows by 5 .9 . 1 that �pv1°O w� ( v ) eiy'IJ dv = w (y- ) + w (y+ )

2

11"

- 00

2

'

376

5. The Fourier Transform

or

J... PV 21r

j oo00 M f ( r

_

-

iv )

e iy v dv = W (Y - ) +2 W (y+ ) .

e - ry , we obtain J...21r pv joo00 Mf ( r zv ) e - (r - iv )Y dv = e _ry w (y- ) +2 w (y+ ) . Since t = eY and s = r - iv , � j r +i a lim (M f) (s) r S ds = f (r ) +2 f (t + ) . a_ oo 21rZ r - i a 5.22.3 MELLIN TRANSFORM BASICS Assuming that the integrals exist for f and for any complex numbers a and b, (a) LINEARITY M (af + bg) = a Mf + bMg. (b) SCALING For a > 0, 1 [Mf (at)] (s) = a s (Mf) (s) . Multiplying both sides by

_

.

-

0

g:

(c) TRANSLATION For any complex number

a,

[M W f (t))] (s) = (Mf) (s + a) . Proof. We use It below, rather than Io--+ oo ; the same argument is valid in

either case. (b) Consider the Mellin transform

( M f (at)) (s) = 1 f (at) tS - l dt . 00

With W =

(c)

Now

at this becomes (M f (at)) (s) (Mt a f (t)) (s)

1

ws - 1 f ( w ) dw s-1 a a f a s (Mf) (s) . 00

--

r OO t a f (t) tS - l dt

Jo 1 oo ta+ s-1f (t) dt

(M f) ( + a) . (Mf) (s) = j --+0 OO f (t) t S- l dt. --+ s

0

5. The Fourier Transform

With 9

(y) = I

( e Y ) , this becomes

(MI) (s) =

377

L�:g(y) eSYdY.

If h is integrable over every finite interval and the improp er integral exists, it is customary to call

T (s) = 1 -+-00 h (t) e - s t dt -+ 00 the BILATERAL LAPLACE TRAN SFORM of h.

B y contrast, the usual (one-sided) Laplace transform of a function defined on (0 , 00) and integrable over every finite interval is

h

(ef) (s) = 1-+ 00 h (t) - s t dt. e

Exercises 5 . 22

1.

I( ) =

Find the Mellin transform of t t a u (t - e) , where U is the unit step function, e is a real constant , and a is a complex number. [ A n . s) for Re s < Re (s) _ ea + s

s - a.] (Mf) = f(a + 2 . Let (M f) ( + i v) be in L1 (R) as a function of v and also be a func tion of bounded variation as a function of v in some neighborhood of v = Show that if 1 lim J,r!".zai a ( Mf) (s) t -S ds, s = r + i v, l (t) = 2 . a-+oo r r

y.

11"1

then

-+ oo

alim ft,a l ( t ) tr + iv - 1 dt =

t [(M f) ( + iy+ ) + (M f) (r + i - )] .

y

r

[Hint: As in the proof of 5 .22 .2, use 5 .9 . 1 , and make an appropriate change of variable.]

a

3. Let I W ) , > 0 , and let e < d. Show that (a) (b)

(1 ft ) I (1 f t) be integrable over all (e, d) , ° <

M (f W»( s) = (I fa) (Mf) (sfa) . M (f (I f t) ft) ( s) = (Mf) (I - s ) .

4. MELLIN CON VOLUTION THE OR EM Under suitable integrability con ditions on I and prove that

g,oo

M (L: I ( w) 9 ({;; ) d: ) (s) = (M f) (s) (M (s) .

g)

378

5. The Fourier Transform

5 . DERIVATIVE O F MELLIN TRANSFORM Under suitable integrability conditions on I, show that

M (In tI (t)) ( B )

5 . 23

= ddB (Mf) ( B ) .

Variations

L l (R) L� (R+ ), fI. L l (Roo+ ), Jo_

We consider some ways to enlarge the class of functions, or that have Fourier transforms or Fourier sine and cosine transforms. I t is possible (see Examples 5 .23 . 1 and 5 .23 .2) that I but that I (t) sin wt or I (t) cos wt f(t) sin wt dt or that or I(t) cos wt dt exists. (Consider the effect of a sine or a cosine in the integrand exhibited in the Riemann-Lebesgue lemma 4 .4. 1 . ) We reserve the use of the notation

E L I (R+ ),

E L I (R+ ) oo Jo-

i (w) , lc (w) , 1. (w) for the cases where I is appropriately integrable as before ; in all other cases, we write the integral instead. The fundamental properties of linearity, scaling, shifting, etc . , are of course shared by these extended integrals. Example 5.23.1 For a > 0, let I (t) but I (t) sin wt ( R+ ) .

L l (R+ )

E Ll

=

( lit) e -a t , t >

O. Th e n

I fI.

Proof. B y Example 5 . 1 6 . 5 , the Fourier sine transform of e - at is given by

Fa (e - a t ) (w) = Jo[00 e- a t sin wt dt = w 2 w+ a 2 .

Integrate both sides of this equation with respect to a from a to 00 , then interchange the order of integration on the left side (justify) to get

1o 00 _e- at

_

t

or

sin wt dt =

1o 00 e - at

-- sin wt dt t

1a00

w -:_-:-2 da 2 w +a

1f' a - - tan - 1 - ' 2 w

=

w tan - 1 - . a

It is common to see this called the Fourier transform of I (t) to see it written as (w) tan - 1 (wla ) . 0

i =

= e - at It and

We leave the details of the following example to Exercise 3 .

379

5. The Fourier Transform Example 5.23.2 For f (t) = ( 1 / ..Ji) f (t) cos wt (j:. L 1 ( R + ) , but

r-+ oo cos wt dt

10

..ji

=

,

we have

t > 0,

rrr- , V�

w>

f (j:. L 1 (R+ )

and

o.

Exercises 5 . 23 1 . ALTERN ATIVE ARGUMENT FOR EXAMPLE 5 . 2 3 . 1 . Show that 1 4' SIn . wt dt e-t- tan - a ' a > 0 , Jor oo W

by differentiating under the integral sign. [Hint: Let 9 (w ) =

e-4' . Jor oo t SIn wt dt .

Differentiation with respect t o w then yields

g' (w)

foooa e- a t cos wt dt a 2 +w 2 ,

which implies that 9 (w) = tan - 1 (w / a) + C where C is an arbitrary constant . Since 9 (0) = 0 and tan - 1 0 = 0, we get C = O. Thus , 9 (w) = tan - 1 (w/a) .J 2. Use the result of Example 5 . 6 . 3 to find fooo t(�i2n+,:t2) dt for a >

O.

3 . Show that fo-+ oo C (�t dt = ,.fE, w > O. [Hint: Let 9 (z) = e-Z z a - 1 , 0 < < 1 , which is analytic for Re z 2: 0 , Im z 2: 0, except at z = O . We integrate 9 around the contour C below

a

c.

00

380

5. The Fou rier Transform

Izl Ie e - Z z a - l dz = z

z

from r to R where 0 < r < R, then along = R from 0 ::; arg ::; 'Ir/2, then along the imaginary axis from iR to ir, then over =r from arg z = 'Ir/2 to arg O . Now 0 by the Cauchy integral theorem. Since � 0 as � 0, it follows that the integral along the small circular arc approaches 0 as r � 0 (See, e.g., Sansone and Gerretson 1 960 , p . 130) . Along the large circular arc , the integral becomes

z= zg (z)

and

<

and

"i Ra - 1 ( 1 - e - R ) ,

which approaches 0 as R � 00 . Thus, we get r (a)

1 -+ 00 e - x x a- 1 dx

00 i 1[ -+ e - iy (iyt - 1 dy 0

Hence

[ - 00 cos y -l i sin y dy 10 y -a

_ _

=

i-a r (a)

(cos 'Ira - z. sm. 'Ira ) r ( a ) . T

Finally, comparing real and imaginary parts gives

1 -+ 00 :�� dy = (cos �a ) r (a)

T

Izl

5. The Fourier Transform

and

1 .... 00 :��: dy = (sin 7i"a ) f (a) .

Now let y = wt, w > 0, and

r

Jo

and

1-00

a = t to get

2

.... oo cos wt dt = f1r V 2::

Vi

1 .... 00 --

4. Use the previous exercise to find, for t > 0 and 0

o

cos wt d -t an d t1-a 0

< a < 1,

sin wt d t. t1-a

6

T he D iscret e and Fast Fo urier Transfo rms The Fast Fourier transform-the most valuable numerical algo rithm of our lifetime . -G. Strang, 1 993 For some functions I, it is impractical (indeed, well-nigh imp ossible) to evaluate the Fourier transform

j(w)

=

1: I(t)e -iwt dt.

I n such a case w e truncate the range of integration t o an interval [a , b] and then approximate the integral for j (w) by a finite sum such as

j (w)

�

N- l

L I (t k ) e - iwtk D.. t .

k =O

This latter sum is called the discrete Fou rier transform D I of /, and it is very useful indeed . As it happens, f can be computed as a matrix product. Certain terms and patterns recur in the matrix. A method that exploits this recurJ,'ence is called the fast Fourier transform algorithm. The efficacy of the fast Fourier transform is prodigious; it can, for example , reduce a billion computations to about a million-to less than a thousandth of the original number! It is one of the major technological breakthroughs of the twentieth century. One thing that Fourier series and Fourier transforms have in common is that the functions on which they operate are defined on groups-the circle group T , R, or Rn . When we discuss the discrete Fourier transform we consider a different group as the common domain, but this time it is finite (the integers modulo an integer) . The crucial thing is the ability to recover the function from its transform.

D

Z

6.1

The Discret e Fourier Transform

As our first approach to the discrete Fourier transform, we treat it as an approximation. The cases of greatest interest concerning the Fourier trans form occur when I Ll (R) is not a familiar function but a complicated

E

384

6. The Discrete and Fast Fourier Transforms

signal, one such that

i (w ) =

1: f(t) e- iw t dt

cannot be evaluated in closed form. For sufficiently large a

< 0 and b > 0 ,

i s a good approximation t o i( w ) for any E (R) . To approximate this integral, we sample the signal at a finite number of equally spaced times =a O. Let

f L1

to

tN - 1

b-a D.. t = � and t k = a + kD.. t ,

Then the approximation tf; of i is given by

k

=

0, 1 , 2 , . . . , N.

" N=o- 1 f (t k ) e - iw t k D..t L....J k N kb N e - iw a " L....J k = O1 f (t k ) e -iw ( - a)/ D.. t .

( )

tf; w

(6. 1)

We further take the time duration [a, b] into account by focusing attention on the points ( frequencies )

Wn = -b27rn -a' where n is an integer. At these points

N- 1 e - i aw n L f (t k ) e -i 27r n k /N D.. t . k=O Neglect multiplication by the constant e - i aw n D.. t and focus attention on the N-periodic function Df Z C, N- 1 Df (n) = L f (t k ) e - i 27rn k /N , n E Z , k=O tf; (w n ) =

:

-+

or ( the same thing )

N- 1 Df (n) = E f (t k ) w - n k where w = e 27ri /N , n E Z . k=O

( 6 . 2)

f

Let us view things from a "discrete" perspective . Forget that was de fined on As we are dealing with the values of at only a finite number of

R.

f

6. The Discrete and Fast Fourier Transforms

385

points , suppose that f is defined on { O , I, . . . , N - I } . In other words, view f as an n-tuple of complex numbers: f = (f (0) , f ( I ) , . . . , f (N - I ) ) E C N . Or consider f as defined on the cyclic group of integers modulo the positive integer N , Z N = Z/ (N) (Z modulo N) where (N)

and

f:

ZN k + (N)

--->

�

c,

=

{kN : k E Z } ,

f (k) .

Another possibility is to view f as the N-periodic function defined on Z by taking f ( k + nN) = f (k) on Z , k = O, I , 2, . . . , N - I , n E Z ,

which is quite analogous to the 27r-periodic functions of Chapter 4. Definition 6.1.1 DISCRETE FO URIER TRANSFORM

Suppose that the finite set ZN is equipped with the discrete measure (all subsets are measurable and the measure of any subset is the number of elements in it) . Since Z N is finite, any function defined on it is integrable ; thus, Ll (Z N ) = L 2 (Z N ) = C N , the collection of all functions f : Z N -+ C . (a) The DIS CRETE FOURIER TRANSFORM (D FT ) of f : Z N -+ C is Df (n) =

N- l

L f ( k) e - 27rikn/N , n E ZN .

k =O

We depart from standard notation here in the use of D f, i being the common designation. As DJ is N-periodic, we can view it, too, as defined on Z N . (b) DISCRETE FO URIER TRAN SFORM OPERATOR The map

is called the DIS CRETE FO URIER TRANSFORM MAP or OPERATOR or (un fortunately) also the discrete Fourier transform. 0 The DFT in Matrix Form We have defined the DFT of f as

Df (n) =

N- l

L f (k) w - n k ,

k =O

where w

=

e 27ri /N .

386

6. The Discrete and Fast Fourier Transforms

In order to achieve a certain symmetry between function and transform (see the inversion theorem 6.2.1) , two commonly used variations of the definition of the DFT use 1/ N or I/Vii as normalizing factors : DI ( n ) =

and

� L l ( k) e - 27ri k n/N N- l k =O

1 DI ( n ) = __

N- l

L 1 ( k) e - 2 7ri k n/ N .

Vii k = O

(As we mentioned earlier, some authors define the Fourier transform with a 1/$ in front of the integral for j(w) for reasons of symmetry.) Another variant is to replace Z N by the cyclic group of Nth roots of unity. We could also view 1 and D 1 as vectors 1=

(

1 (0) : I (N - 1)

)

and DI =

(

DI (O) : DI (N - 1 )

)

.

With

MN =

1 1 1

1 e - l · 2 7r i/ N e - 2 . 2 7r i/ N

1 e - 2 · 2 7r i/ N e - 2· 2 . 2 7r i/ N

1 e - (N- l ). 27r i/ N e - 2 (N - 1 ) 2 7r i/ N

1

e- (N - l )· 2 7r i / N

e - 2 ·(N- l).2 7ri /N

e - (N- l)' . 27r i / N

(6.3) then DI = MN f. MN is also called the NTH ORDER D FT matrix. With 7r N , w = e2 i/

MN =

1 1 1

1 ii; ii; 2

1 ii;2 ii; 4

1 ii;N- l ii; N- 2

1

ii;N- l

ii;N- 2

ii;

As mentioned above , the DFT is sometimes defined so that MN /Vii is obtained, thereby becoming a unitary operator . Since the DFT map (Definition 6 . 1 . 1) is a matrix product , it is clearly a linear operator. The analogs of some of the basic properties of (w) of 5 .5 . 1 are proved in 6 . 1 .2.

j

6 . 1 . 2 DISCRETE FO URIER TRANSFO RM BASICS For 1 : Z N � C , with as the independent variable of f in each case, and any (fi x ed) j E Z,

n

6. The Discrete and Fast Fourier Transforms

(a) TIME SHIFT

Df (n) e -27rijn /N j

1-+

EXP ON ENTIAL MULTIPL ICATION

f (n) e i 2 7rjn / N

f (n - j)

387 1---+

D f (n - j ) ; (c) MODULATION f ( n) cos 21rjn/N 1-+ � [Df (n - j) + Df (n + j)] . Proof. (a) Time Shift. Let 9 (n) = f (n - j) . Then N- l N- l n N ik f (k - j) e - 2 7r ik n / N . / 7r -2 e Dg (n) = L 9 (k) L k=O k=O With m = k -j the latter sum becomes E ;;::�j j f ( m ) e -27r i ( m +j ) n / N . Split "", N l - j . (b) FREQUEN CY SHIFTING

1-+

=

t h e sum

Since

L.J m=- - j

mto

N - l -j +L m=L-j m=O -1

f is N-periodic, -1 m +j ) n / N NL- l f ( m) e - 2 7ri (m +j ) n / N . i ( 7r -2 e ( f ) m m=L- j m=N -j =

Thus

Dg (n)

1 N- l j ", L..J m- = -l J. f ( m) e - 27r i ( m+j )n / N + ", L..J m N=o - fj ( ) e - 2 7ri (m +j ) n /N m nN ", L..J mN =N - J. f ( m ) e -2 7ri ( m+j ) n / N + ", L..J m =-lo f ( m ) e - 2 7r i ( +j ) / (DfL(n)::e�-f27r(imjn)/Ne-2. 7rimn /N) e - 27rij n /N (b) Frequ ency shift. The DFT of f (n) e i 27rjn / N is E f=-OI f (k) e 27rij k /N e -27rik n /N E f;OI f (k) e -27r ik (n - j ) fN f (n m

=

D - j) .

( c) Modulation. Since f (n) cos 21rjn/ N = � ( f (n) e 27r ij n / N + f (n) e - 27r inj / N ) , it follows from (b) that the DFT of f (n) cos 21rj n/ is � Df (n - j) + � Df ( n + j) . N

0

Some examples are in order.

388

6. The Discrete and Fast Fourier Transforms

I : Z4 --+ C be 1 at (1, 2, 2, 1).

Example 6. 1 .3 D F T O F CONSTANT FUN CTION L et 01 1 and 9 =

n = 0, 1, 2, 3: f = (1, 1, 1, 1) . Find the DFT Sol ution . In this case

(1, 2, 2, 1), then Dg = (6 , - 1 - i, 0, -1 + i) . Example 6. 1 .4 Find the DFT 1 9 = (1, i, i 2 , i3 ) = (1, i, -1, -i) . Solution . We can use the frequency shift property 6 . 1 . 2( b ) to find the DFT. If we take 9 =

0

0

With f the constant function of Example 6 . 1 . 3,

9 (k) it follows that

= 1 ( k) e 21fik/ 4 = 1 (k) ik , k = 0, 1, 2, 3,

Dg (0) = Df (0 - 1) = Df (3) = 0, Dg ( l ) = DI (I - I) = DI (O) = 4, Dg (2) = DI (2 - 1) = DI (I) = O, Dg (3) = Df (3 - 1) = DI (2) = 0. Example 6. 1 .5 Let h : Z C be d efined by taking h (k) = 9 (k - 1) 4 2 where 9 = ( l , i, i , i3 ) = ( l , i, - I , -i) is as in Example 6. 1.4. Find th e DFT ol g . Solution . Instead of computing the DFT directly as in Example 6 .1 . 3, we use the time shift property 6.1.2(a): Dh (n) = Dg (n) e -21f i n / 4, n = 0, 1, 2, 3. 0

--+

Hence

Dh = (0, -4i, 0, 0) .

0

Exercises 6 . 1

DI denotes the discrete Fourier transform of f. 1. CONJ U G ATES Let 1 : ZN R.

In the exercises below,

--+

6. The Discrete and Fast Fourier Transforms

DI(n) = DI ( -n) N - n.) D

2.

RO OTS OF UNITY For a positive integer N , let that

"-n" in Z N

w = e 2 1Ti / N . Show

= 1. k w = w - k w N+ k = wN - k for any integer k . ( c ) 1 + w + w2 + . . . + wN- 1 = 0 , N > 1 . C b e given by 1(1) = 1 and I(n) = 0 for n =P 1 . Let I : ZS (a) ww (b)

3.

n.

(a) Show that for all (Note that can be taken to be (b) Show that if 1 is even, then 1 is real and even.

389

=

�

(a) Find Df.

= 1(2) = 1 , I(n) 0 for n =P 1, 2. e - an , for some constant a . Show that 4. Consider I : ZN C, n N DI(n) = 1 -1e-- ae--.. a2 1Tn N 5 . Let N = 4 and let I : Z 4 � C be given by 1 = (0, 1 , 1 , 1) . Find Df. 6. TRAN SFORM OF I (n) = n Find Df for the following 1 : Z N C . Show that for any N, when f(k) = k , k = 0 , 1 , . . . , N - 1 , ) DI(n) = -N/2 + 2 (i1N-sincos(2(2 nn/ N/ N)) (b) Find DI for 1( 1 ) �

=

1---+

.

/

�

7r

7r

7. DFT OF A CONSTANT As a generalization of Example 6 . 1 .3 find the D F T DI of � C where 1 for all

I : ZN

I(n) =

n.

8. VANDERMONDE MATRIX The Vandermonde matrix is 1

Zo z� V (zo, . . . , zN-d = ZoN - 1

1

ZN l Zh _1ZNN_- 1l

(zo , . . . , zN - I ) = I1 ��i�lO .j < i (Zi - Zj ). Show that : MN = V (1 , w , w 2 , . . . , W N - 1 ) where w = e 2 1Ti N . det MN =P O .

Its determinant det V ( a) (b)

1

Z1 z 12 Z 1N - 1

/

390

6. The Discrete and Fast Fourier Transforms

Hints 2 . (c) . Note that 1 + x + X 2 + . . . + X N- 1 = ( 1 - X N ) / ( 1 - x) . 6 . Use the fact that

which equals - N/ ( 1 - r ) if rN = 1 . Then use this result with r = w- n = e - 2 1r in/ N and simplify.

6.2

The Inversion Theorem for the DFT

By an "inversion theorem" we mean a way to recover f from its discrete Fourier transform. We consider one in 6 .2. 1 . Some other avenues to inver sion (same result, different proofs) are considered in equations ( 6 . 5) , (6 .7) , (6 .9) , and (6 . 1 1 ) . Note that the space L 1 (Z N ) of all functions f : Z N � e with respect to the inner product { f, g} =

N- 1

L f( k )g ( k ) ,

k =O

f, g

E L 1 (Z N ) ,

is just e N with its usual inner product and is therefore a Hilbert space. Our first inversion theorem 6 .2. 1 formally resembles the inversion theo rem for the Fourier transform 5 . 12.4. Before proving it, we need the elemen tary observation that l + x + x 2 + . . ' + X N- 1 = (1 - x N ) / (1 - x) for x i- I ; it follows that for any Nth root of unity w = e 2 1r i k / N , k = 1 , 2, . . . , N - 1 , 1 + w + w2 + . . . + wN- 1 6.2.1 INVERSION THEOREM For f : Z N

Df(n) =

�

=

O.

e, w = e2 1r i/ N , with DFT

N- 1

N- 1

k=O

k =O

L f(k)e - 2 1ri k n/ N = L f(k) w - k n ,

we have f(n)

=

� NL- 1 Df(k)e 2 1ri k n/N = � N-L1 Df(k) w k n . k =O

(6 .4)

k =O

39 1

6. The Discrete and Fast Fourier Transforms Proof.

N1 kn N L -Ol Df(k)w ) �� ( N 1 N L k = O LJ. =-O f(j)w - kj W n k ) N- 1 ( N - l N LJ. = O f(j) L k =O W ( n - j ) k . If n = j, then E :';Ol w ( n - j ) k = N , while if # j, then E�;Ol w ( k - j ) n = 0 N1 N L k =-O Df(k) e 2 1ri k n / N

.1..

.1..

.1..

.1..

n

by equation (6 .4) . Hence

N- l L Df(k) e 2 1rik n / N = �N f(n)N = f(n). N k=O

�

0

( 6 . 5)

It follows from the invertibility of the DFT that it is a linear bijection. 6.2.2 DFT MAP IS A LINEAR BI�ECTION The discret e Fourier transfo rm map (Definition 6. 1 . 1)

is a lin ear bijection.

W = e 2 1r i / N and consider the functions Wj : ZN C with m wj (m) wj for m,j = 0, 1, ... , N - 1 , or , in vector notation , Wj ( m ) = ( 1 , , -W2j , . . . , ;;-;;(W N - l )i ) . (6 .6) Lemma 6.2.3 AN O RTHONORMAL BASIS FOR L 2 ( ZN ) = L 1 ( Z N ) Th e functions {Wj lIN j = 0, 1 , . . . , N - 1 } form an orthonormal basis for L 1 ( ZN ) (see Section 3 .4 for the definition of orthonormal basis). P roof. Since L 1 ( Z N ) is N-dimensional , it suffices to show that the Wj /.,fN Next, let =

-+

=i OUT

:

are orthonormal. First, note that

( .,fN Wj , ..jN1 Wk ) 1 N�- l Wj (n)w-k (n) 1

Obviously,

=

N

N- l k ' n 1 w ( -J) . = N

�

(l/N) E �;Ol w( k - j )n = 1 if k = j, and by equation (6.4) , N- l ( l/N) L w ( k - j ) n = 0 if k # j. n= O 0

392

6. The Discrete and Fast Fourier Transforms

{ Wj / VN : j = 0, . . . , N - 1 } is an orthonormal basis for L 1 ( Z N ) , so is the set of conjugates {wj/VN : j = O , . . . , N - l } . Thus, any I E L 1 ( ZN ) can be written N- 1 1 1 (6.7) Wj . 1 = r; I, VN Wj VN Since ", N-1o I(m) -j;; wj (m) L....J Nm = ", -1 / ( m ) 1 w -mj L... .J =O TN '" W - l ( m) 1 e - 2 1r i mj / N , L....J m =o / TN Since

)

(

( I, J& Wj ) J& DI(j)

it follows that

=

, j

= 0, 1 , . . . , N - 1 ,

( 6 . 8)

DI ( ) VN (I, -j;; Wj) I Dl ( ) I E L 2 [-11", 11"] , e: = j(n) e in t , I (t) = L I, ve: 2 11" v 2 71" nLE Z nEZ where j(n) = 2� J�1r I(t) e - i n t dt is the usual Fourier coefficient of I · Since DI(n) = L �;Ol l(k) e -2 1r i n k / N , with W k (n) = e -2 1r ik n / N , we can write DI(n) L:�Ol I( k )wn( k ) wn ( k ) . VN ",N-l ) I( k =o L....J k ..IN Consequently, since {wd VN j = 0 , . . . , N - I} is an orthonormal basis for L 2 ( Zn ) = L 1 ( Z N ) by 6 .2 . 3, we have

and times the coefficient in equation ( 6.7) . Thus, j is may be written as a linear combination in which the scalars are mul tiples of j , analogous to the Fourier series expansion of a function

)

(

:

or

I(n)

=

N

1

(DI, wn) � L :�Ol DI(k)wn (k)

( 6 . 9)

6. The Discrete and Fast Fourier Transforms

393

which is another way to prove the inversion theorem 6 .2 . l . To emphasize the analogy with the Parseval identities for Fourier trans forms ( 5 . 1 8 . 3 ) , we denote by each being equal to in 6.2 .4.

eN ,

L 1 (ZN ) L 2 ( ZN ),

f, 9 E L 2 ( ZN ). Then: l l (a) l: :: f(k)g(k) = � l: ::O D f(k)Dg(k) . Ol (b) l: ::o l f(k) 1 2 = � l::: l ID f(k) 12 . O

6.2.4 PARSEVAL'S IDENTITIES Let

Proof. (a) By equation ( 6 . 9 ) ,

l:::: f(n)g(n)

�N - l ( 1 �N-l Df(k)wn (k) ) ( 1 �N - l Dg(j)wn (j) L..t n = o N L..t k =O N L..ti = o N - l -1 � N - l --� N - l Dg (j) L..t k =O Df(k) N1 � L..t n =o wn(k)wn(j) N L..ti =o -1 �N - l �N - l Df(k)Dg(j)8 ki =o =o k L..t N L..t i -- . 1 �N- l Df(j)Dg(j) N L..ti = o (b) Follows immediately from (a) by setting 9 = f. Another way to obtain the inversion theorem utilizes the inverse of MN . Recall that with W = e 2 1r i / N ,'Df = MN f, where -

0

MN =

1 1 1 1

1

iiJ

iiJ 2

iiJN- l iiJ N - 2

iiJ

( 6 . 10)

L 1 ( ZN ) ( 6.2.3 ) (l/VN) MN (Wk (j)) = (w ki ) , or

wdVN, i = 0 ,

Th e orthonormal basis vectors are the rows of the unitary matrix (with k as the row variable) ,

Clearly ,

iiJN1 - 1 iiJN -2

iiJ12 iiJ4 .

. . , N - 1 , for =

Wk (j) = wkj = Wj (k). Thus MN = Mfv , its transpose .

394

6. The Discrete and Fast Fourier Transforms

With

A = (aij ) we denote the adjoint A* of A by A* = (aj ; ) . Since the

w;fVN are orthonormal,

N N

it follows that

N o o

0 N 0

0 0 N

o o o

o

0

0

N

= NI

where I is the x identity matrix. Thus ( l /N) MN DI MN I, it follows that

=

Nl .

M-

1 = MN l D I = � MN (DJ) .

· S mce (6 . 1 1 )

Thus, we have a third way of obtaining the inverse of the DFT.

Exercises 6 . 2 1 . EVEN TRANSFORM Prove the converse of Exercise 6 . 1- 1 (b) : Show that if E L and D is even, then is real and even.

I I I l ( ZN ) 2. ON D 2 I For any I E L 1 (Z N ), use the inversion theorem 6 . 2 . 1 show that for any n E Z N , D [DIl (n) = NI (-n).

to

3 . Using the inversion theorem 6.2 . 1 o r equation (6 .9) o r (6. 1 1 ) , calculate I from in Examples 6 . 1 .4 and 6 . 1 .5.

DI

I = 2:f=-Ol a/wj /VN,

Wj is as in 6.2.3. N, j = 0, . . . , N -

4. PERIO D O GRAMS Let where The PERIOD O G RAM of I at the frequencies Aj 2n-j / 1 , is the set o f values I(Aj ) =

=

l aj 1 2 . ( a) Show that l aj l 2 = 1:1 1 2: �=-01 l(k) e -2 1f ij k /N I 2 = 1:1 I DI(j) 1 2 . (b) Show that 1111 1; = 2:f=�l I( Aj ) .

The periodogram plays an imp ortant role in stationary discrete Lz-processes because I(Aj ) can be expressed in terms of the sample covariance function of the process.

N N matrix is ROW CIRCULANT if it is of the form CN - 1 Co C l C2 C CN - 2 CN - l o C1 CN -3 , CN - I ) = CN -2 CN - l Co C = circ(co , C l , C l Cz C3 Co

5. An

x

·

·

·

6. The Discrete and Fast Fourier Transforms

395

Successive rows are obtained by shifting the elements of the row above one place to the right with wraparound at the last column . Since the columns can be obtained in the same way, we can just as well say that the matrix is COLUMN CIRCULANT:

C = circ

CI C2 Co CN- I Co CI CN - 2 CN - I Co

Co CN- I CN-2

CI

CI

C2

CN- I CN - 2 CN- 3 Co

C3

Since it does not matter , we simply use the term CIRCULANT. Davis 1979 discusses circulant matrices in detail. (a) Let S be the SHIFT MATRIX circ(O, 1 , 0, . . . , 0) and let A be an N x N matrix. Show that AS shifts the columns of A one place to the right, and SA shifts the rows upward one place with wraparound at the edges. (b) Show that if C is any circulant matrix, then CS = SC.

1 and (c) Show that the characteristic polynomial of S is ).N consequently its roots are w, . . . , w N - 1 where, w = e 2 7r i / N . (d) Show that any N x N circulant matrix C can be written in the form C = Col + CI S + C2 S2 + . . . + CN_ 1 SN - 1 , -

where I stands for the identity matrix. (e) Let FN = -iN MN , where

1

MN =

1 1

1

iiJ iiJ2

1

iiJ 2 iiJ4

1 iiJN - I iiJ N - 2

1

iiJ N - l iiJ N - 2

iiJ

is the DFT matrix of equation ( 6 . 10) . Show that FN diagonalizes S in the sense that FN SF'N = D,

where D is a diagonal matrix whose entries are 1 , W, w 2 , . . . , wN- 1 down the main diagonal. FiV stands for the adjoint of FN (which in this case is j ust the conj ugate, since MN is symmetric) . (f) Show that FN Si FiV = Di for all positive integers j. (g) Show that FN diagonalizes any N x N circulant matrix C.

6. The Discrete and Fast Fourier Transforms

396

6.

Let FN be

as

in Exercise 5 .

(a) Show that

(FN ) 2 = (F;:. ) 2 =

1 0 0 0 0 0 0 0 0

0 0 0 1 1 0

0 1 0

0 0

(b) Show that (F;:') 4 = (FN )4 = I. (c) Show that the distinct eigenvalues of FN are

h

± 1 , ±i.

7. CHARACTERS Let : Z N --+ C be N-periodic, such that I h (k) 1 = 1 for all k , and additive: ( x + y ) = ( x ) + ( y) for all x and y. Such an is called a CHARACTER . Show that for any character

h

(a) (b) (c)

h

h

h

h:

h (O) = 1 . h ( -k ) = h (k) - 1 for all k . h (k) N = 1 for all k.

Hints

1. 5.

Use Exercise

6.1-1 ( b ) and

6.2. 1.

(f) Use (e) . (g) B y (d)-(f) ,

FN CF;:'

c 1 FN F;:' + C2 FN SFN + . . . + cN FN SN - 1 FN = C1 l + C2 D + . . . + cN DN - 1 .

For more detail, see Barnett

6.3

the inversion theorem

1990.

Cyclic Convolution

The imp ortant thing about convolution is the property that the Fourier transform of a convolution offunctions from L 1 (T) or L 1 (R) is the product of the transforms (see 5.2. 3 and 5.8.2 ) , i.e . , = 19. To see what happens for the DFT, we first define a notion of convolution of functions f, 9 E L 1 (ZN ) . We show in 6. 3.3 that the DFT of a convolution is the product of the DFTs.

r;-g

6. The Discrete and Fast Fourier Transforms

CYCLIC CONVOLUTION

Definition 6.3.1

f

The DISCRETE (CYCLIC) CONVOLUTION, * 9 of at each E by

k ZN

N- l f * g (k) = "L. f(j)g(k - j) . j =O

f, 9 E L l ( ZN ) is defined ( 6 . 12)

0

The basic properties of the cyclic convolution are listed in 6.3 .2 and

6.3.2 CONVOLUTION BASICS L e t (a) f * g = g * f; (b) f * * = ( f * * (c) (f * g ) = ( f ) * ( for any ( d) 1 * + = 1 * 9 + 1 *

f, g , h E L 1 ( ZN ).

a

397

(g h) a (g h)

g ) h; ag)

h.

6.3.3.

Th en:

constant a ;

Proof. Since this i s so straightforward, w e prove only part (a) . By definition,

,,\,jN - l l(j)g(k - j) � =o ,,\, - ( N - l) I(k - )g (m) � m=O

k - N+ l I(k - m ) g ( m) "L.m=k ,,\, N - l I(k - m) g (m) . 0 �m =O 6.3.3 CONVOLUTIONS To PRODUCTS For I, g E L 1 (Z N ), the DFT D (f * g ) (n) is e q a l to DI(n)Dg (n) for all n. m

u

Proof.

N1 "L. k =-O 1 * g (k) e - 27r ik n / N ,,\, N - l 1 * g (k)w k n (w = e 2 7ri /N ) � k= O ,,\, N - 1 ( ,,\,jN - 1 l(j) g (k - j ) -Wk n �k = O � =O ) ( 6 . 13 ) N- 1 N l n k "L.J. =O l(j ) ( "L. k=O g ( k - j)-W ) ,,\,N - l f U )-WJ. n ( ,,\, N -0l g (k - j)-W( k -J. )n � k= �j = O ) = D 1 ( n) Dg (n) . 0 We noted for f E L l (R) and L l (T) that there does not exist an identity for the convolution operator (see Exercise 5.2-10 for L l (T) and at the D (f * g ) (n)

398

6. The Discrete and Fast Fourier Transforms

beginning of Section 5.14 for Ll (R)). This is not the case for L 1 (Z N ) . The function e ( k ) = (1 , 0, . . . , 0) ( 6. 14) satisfies 1 * e = I for all

I E L 1 (ZN ) . This fact is used in the exercises.

Exercises 6 . 3

1.

Complete the proof of 6.3.2.

(a) determine 1 * f. (b) For g (k) = k , k = O , I , . . . , N - l , find / * g.

We say that I E Ll (Z N ) is INVERTIBLE if there exists 9 E L l (Z N ) such that 1 * 9 = e.

3 . INVERTIBLE ELEMENTS IN Ll (Z N ) (a) Show that vanishes.

I

E L 1 (Z N ) i s invertible if and only if

DI never

I

(b) For noninvertible E L 1 (Z N ) , find all 9 such that 1 * 9 = o. (c) We say that 1 =P 0 is a divisor 01 0 when there exists 9 =P 0 such that 1 * 9 = o. Show that I is a divisor of 0 if and only if I is not invertible . (d) For invertible I, show that = for every

[DI - l ] (n) I/DI (n)

4. POWERS A N D THE (a) Let I : Z 2 where e =

DI.

�

n.

D FT

I

C be given by = (0, 1) . Show that 1 * 1 = e, the identity for convolution . Also, determine

( 1 , 0),

I

(b) Let I : Zg � C be given by I = (0, 0, 1 ) . Show that * 1 = e, where e = ( 1 , 0, 0), the identity for convolution. Determine (c) Generalize the results of ( a) and (b) for 1 : Z n � C is given by 1 = (0, . . . , 0, 0, 1 ). (d) In general, characterize all 9 : Zn � C such that the n-fold convolution 9 * 9 * . . . * 9 = e in terms of the DFT Dg .

DI.

6. The Discrete and Fast Fourier Transforms

399

Hints

3.

(a) . First note that e(n) = 1 for all n . Then use 6.3.3 on D (J * g ) = DfDg . (b) Find f, g E L 1 (Z N ) with (DfDg) (n) = Df(n)Dg ( n) = 0 for all while f :j:. 0 and 9 :j:. O.

n

4.

(d) . Use the convolution to product theorem

6.4

Fast Fourier Transform for N

6.3.3. =

2k

Modern signal processing requires the ability to process huge numbers of bits rapidly. A color TV picture, for example , requires about eight million-a megabyte-per second. To evaluate the discrete Fourier trans form D f = MN f, f E e N , requires N 2 multiplications and N (N 1) additions-a formidable thicket of computations for large N . The FAST FOURIER TRANSFORM (FFT) algorithm for N = 2 k reduces the N 2 multi plications to something proportional to N log 2 N . For example (see 6 4 . 3 ) , for N = 2 1 = 32, 768, it reduces -

5

.

approximately one-thousandth of the multiplications. Using it makes it effectively possible to force the flow of a fire hose through a garden hose , as in forcing a video signal through a telephone line , for example . Though the applications to image processing, optics, geology, etc . , are contemporary, the idea of the FFT is not . Indeed , Heideman et al. [ 19 84] trace the notion back to Gauss. They mention that he expressed it in a clumsy notation and that his work on the subject was published only posthumously. The mo dern development begins with the seminal article of Cooley and Tukey [ 1965] . In this section we investigate the FFT technique known by the various aliases INDEX REPRESENTATIO N , RADIX-SPLITTING , or TIME D ECIMATION . There is a symmetric approach-FREQUENCY DECIMATIO N -in which the roles of rows and columns are exchanged . This symmetry is related to the symmetry of the DFT matrix MN of equation ( 6.3) of Section 6.1. The approach is a classic-split a large process into groups of smaller (half as many at each stage) subprocesses. We count how many steps the FFT procedure takes in 6.4.3. To discuss the process we need the notion of even and odd parts of a function .

6. The Discrete and Fast Fourier Transforms

400

Ie and 10 of I E e N For I E e N and N even we define I" e v en " and l "oM' to be Ie = ( 1 (0) , / (2) , . . . , I ( N - 2)) and 10 = ( 1 (1), / (3), . , I ( N - 1 )) .

Definition 6.4. 1 EVEN AND ODD PARTS

..

The FFT algorithm consists of repeatedly decomposing a function

e N into even and odd parts as follows:

lee /

I

/

Ie

'\.

!

/

leo

!

'\.

1

'\.

loe

!

!

Ieee leeo

10

'\.

leoe

leoo

loee

100

0

IE

( 6.15)

loeo

We compute the DFT of functions like lee o e and then synthesize D I from them. Consider what happens for N = 4 in the following example . Example 6.4.2 USIN G

For

I E e4 ,

DIe

( (

AND

D lo

DI

)( ) )

TO GET

we compute its DFT directly as

DI =

1 1 1 1

1 1 1 -i - 1 i -1 1 - 1 - 1 -i i

1 (0) 1 ( 1) 1 (2) 1 (3)

(6 . 16) 2 (1) [/(0) + 1 ( )] + [/ + 1 (3)] [/(0) - 1 (2) ] - i [/ ( 1) - 1 (3)] . [/ (0) + 1 (2)] - [/ ( 1 ) + 1 (3)] [/ (0) - 1 (2)] + i [ /(1) - 1 (3)] Note that each row of DI has 1 (0) , I (2)-the terms of Ie-and the terms of 10 , I (1) and I (3) . Rewriting the matrix equality of equation ( 6 . 16 ) as _

four equalities , we get

DI (0) DI ( I) For

= = =

[/ (0) + 1 (2) ] + [/ ( 1 ) + 1 (3) ] , [/ (0) - / (2) ] - i [f ( 1 ) - / (3) ] ,

Ie = ( 1 (0) , / (2)) and 10 = ( 1 ( 1 ) , / (3)) , D Ie = M2 !e =

( � !1 ) ( �� ) ( �� g� ) =

(6. 17)

40 1

6. The Discrete and Fast Fourier Transforms

and

D 10 = Mdo = Thus

(6.17)

( � �1 ) ( � gj ) = ( � gj � � ��j ) .

DIe (0) = 1 (0) + 1 ( 2) , DIe (1) = 1 (0) - 1 (2) , becomes

DI ( O) DI ( l )

etc . , and equation

DIe (0) + D lo (0) , DIe ( 1 ) - iD lo (1) ,

(6 . 18)

Thus, knowing DIe and D lo permits us to calculate Df. Equation (6.18) is a special case of the butterfly equations (see equations (6.25)-(6.27)) . 0 In 6 .4.3 we count how many multiplications are needed to compute

DI .

6.4.3 FFT FOR N = 2 k For N = 2 k and I E e N , the number of multi plications re quired to compute DI is 2N log 2 N = 21:+ 1 . k. Proof. We prove this b y induction o n

DI

��

k. For k = 1

and

I E e2 ,

Md = -

_

) ( � ��j ) 1 (0) + (1) , 1 ( ) - 1 (1) ) �1

0

I

which involves 4 multiplications or 2 . 2 1 log 2 2. Assume that the result holds for N = 2 k - 1 , i.e., that 2 · 2 k- 1 1 0g 2 2 k - 1 multiplications are required for N = 2 k- 1 , and let N = 2 k . With q as a variable, let Wq = e2 1f i/ q , so that the DFT of I is

N- 1 DI ( n) = L I (k) wN" , n = 0, 1 , 2, . . . , N - 1 . (6 . 19) O = k Divide k by 2 and n by N /2 and write each in terms of a quotient ( ko and no, respectively) and a remainder (k 1 and nd: k = 2ko + k 1 , n = (N/2) no + n1 .

Since k and n vary between 0 and N - 1 , the possibilities for quotients and remainders are ko = 0, 1 , . . . , N/2 - 1 , k 1 = 0, 1 , no = 0, 1 , and n 1 = 0, . . . , N/2 - 1 . Hence

402

6. The Discrete and Fast Fourier Transforms

w� k o n o

(WN WN raised to any multiple of N is 1), we can . 1 N/ 2- 1 n k o o � DI ( n ) = w L I ( 2 ko + k d vl{/2) k , n o w;t 0 n ' WN L k , = O k o =O or, by setting k l successively to 0 and 1 , wNNk o n o L Nk /2= O- 1 I (2k0 ) w2Nk o n , DI(n) o N/2 - 1 ,,\, N/2 n o k n n (6.20) n Nk ( o o w + N k (2 I 0 + 1 ) wN ) w2N o , wN' . L...t k o = O Since w'jy WN/ 2 , we can rewrite the w;t° n , in each summand as w�/�' to get Since = 1 or rewrite equation (6 1 9 ) as

k ' '' ' ,

=

=

DI(n)

WN ( N/ 2)n o WN Il , WN Il• This simplifies w NNk o n o L Nk o/=2O- 1 I (2 k0 ) W-k N/2 o n , (6.22) N/2 - 1 1 (2 ko + 1) W k o n , , + WN Il L N / 2 ko=O

l

Replace ( N/2 ) no + n by the second summation:

DI ( n ) no = 0, 1 , n l

=

=

n;

then

=

0, 1 , . . . , N/2 - 1 . The terms

N/2 - 1 N/2- 1 n k o , I (2 k o + 1 ) WN/2 k o n , and o W ) (2k L 1 N/ 2 L k o =O k o =O

are the rows of

D ie = MN/ 2

( ) 1 (0) 1 (2) :

and

I (N - 2) so we can rewrite equation ( 6.22) as

D lo

=

MN/ 2

( )

( 6.23)

/ ( 1) 1 (3)

:

'

I (N - l)

Now let us count what is needed to get the values of DI ( n ) for every To get

n.

6. The Discrete and Fast Fourier Transforms

403

for every nl we need 2 (N/2) log 2 N/2 = N log 2 N/2 multiplications by the induction hypothesis . We need the same number to calculate D fo (nd, for a total of 2N log 2 N/2. For a fixed n, we need two multiplications (by w�ko n o and WNIl) to get Df (n) . Thus, for n = 0, 1 , . . . , N - 1 , we need 2N more multiplications, bringing the count to Remark In calculating the number of multiplications in 6.4.3 to be 2N log 2 N, we counted multiplications by 1 , so that

requires 4 operations. If we do not count multiplications by 1 , we get ( N/2 ) log 2 N multiplications, one-fourth of 2N log 2 N. The result also appears in this form. The key to the FFT algorithm is equation (6.24) , which expresses terms of Dfe and Dfo . With no = 0 and 1 , respectively, equation yields what are known as the BUTTERFLY RELATIO NSHIPS

Df(n)

=

Dfe (n) + wNIlDfo ( n), O :S n :S N/2 - 1

Df in (6 .24) (6.25)

and

Df(N/2 + n)

= w�ko Dfe (n) + WNN/ 2 + n Dfo(n),

O :S n :S N/2 - 1 . (6.26)

1 and wN/ 2 -1 , this simplifies to = Dfe (n) + wN"Dfo (n) , O :S n :S N/2 - 1, Df(n) Df ( N/2 + n) Dfe (n) - wNl1Dfo ( n) , O :S n :S N/2 - 1 . ( 6.27) The FFT algorithm continues to split fe and fo into even and odd parts fee , feo , foe , foo , etc . , until we get two-dimensional vectors . Then we synthesize D f from the DFTs of the parts . We outline the procedure for N = 8 . For f = ( 1 ( 0 ) , / (1) , . . . , / (7)) E C8, (6.28) Since w�ko =

=

=

we have

fe = ( 1 (0) , f(2) , f(4) , f(6)) and fo = ( 1 ( 1 ) , f(3), f(5), f(7)) . (6.29) Split fe and fo into its 2-dimensional even and odd components : fe e = ( 1 (0), f(4)) , and foe = ( 1 (1) , / (5)) , (6.30) fo o = ( 1 (3) , f( 7 )) . feo ( 1 (2) , f(6)) , Now compute Dfee = Mzfe e and Dfe o = Mzfeo and use equations (6 .27) to obtain D fe . Similarly, use D fo e and D foo in equations (6.27) to obtain D fo . Finally, use D fe and Dfo in equations (6 .2 7) to obtain D f. We present an actual calculation in Example 6.4.4. =

404

6. The Discrete and Fast Fourier Transforms

Example 6.4.4 A D FT CALCULATIO N FOR Calculation . Apply M2 =

(�

�1

)

I E C8 .

to

lee = ( 1 (0) , / (4)) , and loe = ( 1 ( 1 ) , / ( 5)) , 100 = ( 1 (3) , / ( 7 )) , . leo = ( 1 (2) , / (6)) , to obtain

-

D lee = D le o

_

�

1 (0) + 1 (4) 1 (0) - 1 (4) 1 ( 2) + 1 (6) 1 (2) - 1 (6)

)

'

and

'

-

-

Dloe = D loo

_

�

1 ( 1) 1 ( 1) 1 (3) 1( 3)

Now use equations ( 6.27) to compute Dle and Dlo :

= Dlee (n) + � D leo (n) , 0 Dle (n) Dle ( n + 2) = D lee (n) � D leo (n) , 0

( (-

D Ie _

Similarly,

( 1 ( 0) + 1 (4) ) + ( 1 (2) + 1 (6) ) ( 1 (0) - 1 ( 4) ) - i ( l( 2) - 1 (6)) ( 1 (0) + 1 (4) ) - ( 1 (2) + 1 (6)) ( 1 (0) - 1 (4) ) + i ( l (2) - 1 (6))

( 1 ( 1 ) + 1 (5)) + ( 1 (3) + 1 (7)) ( 11 ) - 1 (5)) - i ( l (3) - 1 (7)) Dlo ( 1 ( 1 ) + 1 (5)) - ( 1 (3) + 1 ( 7 )) ( 1 ( 1 ) - 1 (5)) + i ( l (3) - 1 ( 7 ) ) Finally, w e use equations (6 .27 ) t o compute DI : _

= Dle (n) + WS'l Dlo (n) , 0 D I (n) D I(n + 4) = Dle (n) - WS'l D lo (n) , 0

S; S;

+ 1 (5) - 1 (5) + 1 ( 7) - 1 (7)

S; S;

S; S;

n n

) .)

), .

1, 1.

.

n n

S; S;

3, 3.

With W8 = e2 7r i/ 8 in equations (6 .27 ) we get D 1 =

[/ (0) + 1 (4)] + [/ ( 2) + 1 (6)] 1 (0) - 1 ( 4) - i [/ ( 2) - 1 (6 )] 1 (0) + 1 ( 4) - [/( 2) + 1 (6) ] 1 (0) - 1 (4) + i [/ ( 2) - 1 (6)] 1 (0) + 1 (4) + [/ (2) + 1 (6) ] 1 (0) - 1 (4) - i [/ (2) - 1 (6) ] 1 (0) + 1 ( 4 ) - [/ ( 2) + 1 (6)] 1 (0) - 1 (4) + i [/ (2 ) - 1 (6) ]

+ [/ ( 1 ) + 1 (5)] + [ / (3) + 1 (7) ] + e - 2 7r i/ 8 ([I ( 1 ) - 1 ( 5) ] - i [/ (3) - 1 ( 7 )] ) + e - 4 7r i/ 8 ( [/( 1 ) + 1 (5) ] - [/ (3) + 1 (7) ] ) + e - 67r i !8 ([/ ( 1 ) - 1 (5)] + i [ / (3) - 1 (7)] ) - ([1 ( 1 ) + 1 (5)] + [/ (3) + / (7) ] ) _ e - 27ri/ 8 ([I ( 1 ) - 1 (5) ] - i [ / (3) - 1 ( 7 ) ]) _ e - 4 7r i/ 8 ( [/( 1 ) + 1 (5) ] - [/ (3) + 1 ( 7 ) ] ) _ e- 6 7r i / 8 ( [/( 1 ) - 1 ( 5) ] + i [ / (3) - 1 (7) ] ) (6 . 31 )

6. The Discrete and Fast Fou rier Transforms

405

Buneman's Algorithm

The pairs of terms in the rows of (6 .3 1 ) , ( / (0) , I (4)) , ( / (2) , I (6) ) , etc. , are the terms of lee. leo , loe and 100 ' respectively. Forget the I for a moment and just look at the arguments of I: 0 , 4, 2 , 6 , 1 , 5 , 3 , 7. The technique known as BUNEMAN 'S ALGORITHM permits us to calculate these arguments directly. It consists mainly of multiplying by 2 and adding 1: ( 0, 1 ) � (0, 2 ) � ( 1 , 3 ) co�n e (0 , 2, 1 , 3 ) . This yields the order of the terms in the rows of D I for I E C4 ( see Example 6 .4.2 ) . For N = 8, as in Example 6 .4.4, ( 0, 2 , 1 , 3 )

x 2 ( 0, 4, 2, 6 )

--+

+1

--+

( 1 , 5, 3 , 7

) combin e --+

( 0, 4, 2 , 6 , 1 , 5 , 3 , 7 ) .

Thus lee = ( / (0) , / (4)) , leo = ( / (2) , / (6)) , loe = ( / ( 1) , / (5 )) , and 100 = ( / (3) , / ( 7 ) ) . As a simple exercise, use the algorithm on ( 0 , 4, 2 , 6 , 1 , 5 , 3 , 7 ) to see what happens for N = 16. The numbers 0 , 1 , . . . , 7 written in binary form are 000, 00 1 , . . . , 1 1 1 . Buneman ' s algorithm transformed (0 , 1 , . . . , 7 ) into (0, 4, 2, 6 , 1 , 5 , 3 , 7 ) . In particular, the entry 1 in position 1 = 00 1 2 ( we begin the count at 0 , not 1) ended up in position 4 = 1 00 2 . The binary representation of its final location is the reverse of its original. This general phenomenon is known as BIT REVERSAL. For example , if N = 16, 1 ( 13 ) = 1 ( 1 1 0 1 2) will ultimately be in position 1011 2 = 1 1 .

Exercises 6 . 4 1 . Show that the column vector in equation (6 .31) is indeed equal to Ms/ . 2. ( a ) Use equations (6.27) to show that when multiplication by 1 is not counted, it is possible to evaluate D I in ( N/2 ) log 2 N steps. ( b ) With both ( N/2 ) log 2 N and 2N log 2 N as the number of re quired operations instead of N 2 , find the percentage savings in operations for N = 21°, 2 2 0 , 23° .

406

6. The Discrete and Fast Fourier Transforms

3 . Use Example

6.4.4 to obtain Df when f is given by f (n) =

{ n,I ,

O :S n :S 5 , 6 :S n :S 7. .

4. The fact that MN is symmetric and that M;;/ = (1/ N) MN ( Section 6 .2, equation ( 6.11 ) ) supports the statement that there is a process by which one obtains f from D f similar to that of Example 6 .4.4. Write the details of the process of Example 6.4.4 and use it to show that the inverse of D f for D f( n ) 1 for 0 :S n :S 7 is the multiplicative =

identity of the convolution operator

1 o e=

o o Hints 2. ( a) . Begin by determining that Md requires only one operation and proceed by induction. (b) Recall that MN f would be performed with N 2 operations.

6.5

The Fast Fourier Transform for N

=

RC

We consider the less dramatic improvement in calculation of the DFT for the case where N factors but is not a power of 2, in this section . Let N = RC, where r or C is not a power of 2. We can view the calcu lation of D f = MN f as the computation of N dot products-each row of D f is (row of MN) • f. The FFT of 6 .5.2 reduces the N dot products to R dot products of vectors of dimension C, or C dot products of vectors of dimension R; we illustrate in Example 6.5 . 1 . This reduction will not occur if N does not factor, however ( Exercise 2 ) . Example 6.5.1

REDUCTION WHEN

N

=

6

We group terms in M6f in two ways, by reversing the roles of 3 and 2 . Method 1. With w = e 2 1ri/ 6 and f (k) = for k = 0 , 1 , . . . , 5 ,

ak

6. The Discrete and Fast Fourier Transforms

M6 / =

I I I I I I

I I I3 I I

w w2 w3 wt WS

P

u

w wt I I w3 2 w I wt w3

wt w2 I wt w2

wS wt w3 w2 w

407

ao at a2 a3 a4 as

Thus

M6 /=

ao + at + a 2 + a 3 + a4 + as ao + atw + a 2 w2 + a 3w3 + a4wt + asws ao + alw 2 + a 2 wt + a 3 + a4w2 + aswt ao + alw3 + a 2 + a 3 w3 + a 4 + asw3 ao + alwt + a 2 w 2 + a3 + a4wt + as w 2 ao + atwS + a 2 wt + a3 ws + a4w2 + asw

( 6 .32 )

Since 6 = 2 · 3 , we group the terms in M6 / in two groups of three terms. In the first of the two groups we put those terms whose subscript divided by two has remainder O-namely, ao , a 2 and a4-in the second, we put those whose subscript divided by 2 has remainder I-at , as and as-as in

M/ 6

(ao + a 2 + a 4 ) + (at + a3 + a s ) (ao + +a 2 w2 + a4wt) + (at w + a 3 ws + asw5 ) (ao + a 2 wt + a4w2 ) + (at w2 + a 3 + aswt) (ao + a 2 + a4) + (alwS + a 3 w3 + asw3 ) (ao + a 2 w 2 + a4wt) + (at wt + as + asw2 ) (ao + +a 2 wt + a4w2 ) + (at wS + asw3 + asw)

=

( 6 .33 )

Since WS k = e - 2 1ri 6k / 6 = I for any integer k, we can factor the terms as below: o

M6 / =

(a o + a 2 +�) + (a1+as+a s ) I (ao + a 2w2 + a4wt ) + w(at + asw 2 + aswt) 2 (ao + a 2wt + a4w2 ) + w2 (al + as wt + asw 2 ) . 3 ( a o + a 2 + a4 ) + w3 ( a t + a3 + aS ) 4 (ao + a 2 w2 + a4wt) + wt(at + a 3 w2 + aswt ) 5 (ao + +a 2 wt + a 4w2 ) + w S (at + a 3wt + asw2 )

. ( 6 . 34 )

Group rows whose remainder is the same when divided by 3-namely rows and 3, I and 4, and 2 and 5. In rows 0 and 3 , note the recurrence of (ao + a 2 + a4 ) and ( al + a3 + as) . In rows and 4, (ao + a 2 w2 + a 4 wt) and (a t + asw2 + aswt) recur . Similarly, in rows 2 and 5 , (ao + a 2 wt + a 4w 2 ) and (al + a 3wt + as w 2 ) recur. Later, we make use of the fact the parenthesized sums need only be calculated once .

o

I

Method 2.

Now we consider an alternative grouping. We collect the terms of M6 / by grouping terms whose subscripts, divided by 3 , yield the same remainder:

408

6. The Discrete and Fast Fourier Transforms

( 0 , 3 ) , ( 1 , 4) and ( 2, 5 ) . Thus, (ao + a3 ) + (al + a4) + (a 2 + a 5) (ao + a3 w3 ) + (alw + a4tv4 ) + (a 2 w 2 + a5 w5 ) (ao + a3 ) + (al w 2 + a4w2 ) + (a 2 tv4 + a 5tv4 ) ( ao + a 3 w3 ) + (alw3 + a4) + (a 2 + a 5w3) (ao + a 3 ) + (al tv4 + a4tv4 ) + (a 2 w2 + a 5 w2 ) (ao + a3 w3 ) + (altuS + a4w2 ) + (a 2 tv4 + +a5w)

(6 .35)

(a O + a3 ) + ( a l + a4) + ( a 2 + a5) 1 (ao + a3 w3) + Weal + a4w3 ) + w2 (a 2 + a 5 w3 ) 2 ( a o + a3 ) + w2 (a1 + 84) + tv4 ( a 2 + a5 ) 3 (ao + a 3 w 3 ) + w3 (al + a4w3 ) + (a 2 + a 5 w3 ) 4 (a o + a3 ) + tv4 ( a l +84) + w2 (a 2 + a5) 5 (ao + a3 w3) + w5 (al + a4w3 ) + tv4(a 2 + +a5 w3)

(6 . 36)

o

=

Now consider pairs of rows whose remainders are the same when divided by 2; that is, rows 0, 2, and 4, and rows 1, 3, and 5. Note the recurrence of (ao + a3 ) , (al + a4 ) , and (a 2 + a 5 ) . 0 It is time to consider the general result.

IF

N = RG, the DFT MN I of any I E 6.5.2 FFT FACTOR FORM L l ( ZN ) can be computed with N ( R + G ) multiplications. Proof. For I : ZN

D/ ( M)

N- l =

-+

C, the DFT of I is

� I ( K ) wK M , M = 0, 1 , 2,

.

. . , N - 1 , w = e 2 1ri/ N . (6.37)

K =O Dividing the exponent K by G enables us to write K = Gk + k o , where k = 0, 1 , . . . , R

-

land ko = 0, 1 , . . . , G

-

1.

Similarly, dividing the row variable M by R, we get

M = Rn + n o , n Now,

WKM

=

= =

0, 1 , . . , G .

- I, no = 0 , 1 , . .

.

, R - 1.

W
Since wN kn = 1 for all k and n , we can substitute for wKM in equation (6 . 37) to get

DI (M) =

[ � �

C - l R- l

k o =O k=O

I ( k G + ko )

1

wCk n o wk o( n o + n R) .

(6 .38)

6. The Discrete and Fast Fourier Transforms

With WR = e 2 1t i/ R and

Wc

409

= e 21t i/ C ,

View

f (kC + ko) = g (k, ko) , k = O, I , . . . , R - l , as a member of Z R and consider the DFT h (k o ) of 9 (k, ko ) :

R- I ko) wn o C k h (ko) (no) = " L..,.. k =O 9 (k, (6 .39) R- 1 k L k =O g (k, ko) �O , no = O , I , . . . , R - l . We can replace the bracketed sum of equation (6 .38 ) by h (ko) (no ) : D f (M ) =

C- l

C- I

k o =O

k o =O

L h (ko) (no ) wk o n ouyk o n R = L h (ko) (no) wk o n o w�o n .

( 6 .40 ) In equation ( 6 .39 ) we need R multiplications for each no ; since no assumes R values, complete knowledge of h (ko ) (no ) requires R2 multiplications . Since there are C values of ko , we need CR 2 multiplications to determine all the h (ko ) (no ) . Now consider equation (6 .40 ) for a particular M . Each value D f ( M) can be calculated by adding C values of h (ko) (no ) wk o n o wk o n R . Since there are N = R C rows, there is a total of C x N = C x RC = R C 2 additional products. The grand total of multiplication operations is therefore CR 2 for the h (ko) (no) plus RC 2 for D f (M) :

C R2 + RC 2 = RC (C + R) = N (C + R) .

0

We note without proof that there is a corresponding reduction in the number of additions and that if N = C1 C2 . . . Ck , then the DFT can be computed in N (CI + C2 + . . . + Ck) multiplications. A composite number N can usually be factored in many ways, and some methods reduce N (C + R) more than others. If N = 900, the direct method for calculating the DFT requires 900 2 = 8 10, 000 multiplications . Since 900 factors into 30 · 30, the method of 6 . 5 . 2 reduces this to 900 ( 30 + 30) = 54 , 000, one-fifteenth of the first value . For N = k 2 , the bigger N is, the more significant the reduction; indeed,

410

6. The Discrete and Fast Fourier Transforms

Exercises 6 . 5 1 . Obtain

MN for N = 3 and N = 5 . Then find MN f for 1=

(0

�d / =

(�)

Is it possible to reorganize the entries in f in either case to reduce the multiplications from 3 2 or 5 2 as was done in Sections 6 .5 and 6 .4? What can be concluded when N is a prime number? 2 . THE NEED FOR COMPOSITE N (a) Show that with N = 3 the number of multiplications necessary to perform M31 cannot be reduced from 9. Do the same for M5 • (b ) Multiply a 12 7 2 1

(

! � !

)( )

: .

Can you do this with fewer than 3 2 = 9 multiplications? 3 . Verify that the procedure of Example 6 .5. 1 can be done with 6( 3 + 2) rather than 6(6) multiplications. 4. Let f : Z6 -+ C , f(n) = n for all n. Use equation (6.34 ) of Example 6 .5 . 1 to find the DFT D I of f.

7 Wavelet s

.

. . . wavelets are without any doubt an exciting and intuitive concept . This concept brings with it a new way of thinking . . . -Yo Meyer, 1993 By a DILATIO N of I ( t ) we mean I ( k t ) for some constant k . When we con sidered Fourier analysis in L 2 [ - 1I", 11"] , we made extensive use of the fact that normalized dilations 1 .J21i) e i n t , n E Z, of e it constitute an orthonormal basis for L 2 [- 1I", 11"] . The procedure permits any function f E L 2 [- 11", 11"] to be represented as an infinite series of complex exponentials in the sense of convergence to f in the L 2 -norm. As there are other orthonormal bases for L 2 [- 11", 11"] , it would be profligate to ignore them. Wavelet analysis not only e i n t , it uses bases that are employs orthonormal bases other than not orthonormal ( Riesz bases ) and basis-like collections ( frames ) that are not even linearly independent. In some cases (fingerprints ) wavelet analysis is much better than Fourier analysis in the sense that fewer terms suffice to approximate certain functions. What is a wavelet? When you hear "wavelets," think "basis" as in orthonormal basis or Schauder basis. Starting with one function, a MOTHER WAVELET tP, from some class of functions, for example L 2 (R) , we want to be able to generate some kind of basis by dilation and translation of tP: The WAVELETS are the functions

(/

(1/.J21i)

tPa , b (t) = lal - 1 /p tP

C � b ) , p > 0, a, b E R, a 1= 0 ,

(7. 1 )

with 2 being the most frequently used value of p. Indeed, we usually consider functions like tPi , k (t) = 2i 1 2 1/J 2i t - k) , j, k E Z.

(

For example (Example 7.3 . 1 ) , consider the Baar mother wavelet tP (t) =

1[0, 1 ) ( 2 t ) - 1[0,1 ) (2t - 1) E L 2 (R) . o.

�"2��OT2�O·4�OC6-'O.'-'--'1 2 �.

The Ha.ar Mother Wavelet 1/J

412

7. Wavelets

Then the HAAR WAVELETS

L2

0,

are an orthonormal basis for (R) . For k = consider the collection { tjJj , O (t) = 2j / tjJ (2jt ) : j E Z } . Notice that the bigger j is, the smaller the cozero set 1 / 2j is. To see how efficient it can be to use wavelets, suppose we do not translate the Haar wavelets, but consider only the Haar functions tjJj , k { 1[0 , 1) } ' j 2: and k = 0, 1 , . . . , 2j - 1 , as we did in Section 3 .5 ( Definition 3 .5 . 1 ) ; the Haar functions are an orthonormal basis for [0 , 1 ] . If f E [0 , 1 ] and f (t) vanishes for t 2: t , then we only need to use one-fourth of the Haar basis elements to represent f, whereas for the Fourier series for f, we would need all of the sines and cosines. To approximate f we would need many more sines and cosines than Haar functions. This leads to a much better "compression ratio" for a signal f E [0, 1 ] than that provided by Fourier analysis in the sense that less data is needed to reconstitute the signal. Wavelets serve a similar purpose in image compression. Indeed, the compression ratio for certain wavelet expansions is so superior to Fourier analysis in fingerprint reconstruction that the American Federal Bureau of Investigation ( F.B.I. ) uses them to store and transmit their extensive data base ( Cipra 1993 ) . They also have significant applications to filtering noise from signals and speech recognition; see Cipra 1993 , Strang 1993 , Strichartz 1993 and Walker 1997. Meyer [1993] surveys the development of the theory. Since Fourier series converge relatively slowly in the vicinity of spikes, jagged functions require many terms in their Fourier approximations to become recognizable. Through the use of bases with discontinuities ( such as the Haar functions ) and sharp corners, fewer terms need be employed. The essence of wavelet analysis is the judicious choice ( depending on the situation ) of an orthonormal basis other than sines and cosines for the ultimate goal of approximating a function. We first consider "multiresolution analysis" ( Sections 7.2-7.8) , in which we get orthonormal bases for (R) by dilation and translation of a mother wavelet. We forgo orthogonality in Section 7.9 , and thereby uniqueness of the coefficients in the series as we consider frames, sequences in a separable Hilbert space X that satisfy

[ 0,

2

]

U

0

L2

L2

L2

L2 LnEN anxn,

(xn)

A II x II 2 �

L

nEN

I( x, Xn) 1 2 � B Il x 11 2 , for all x E X.

We retain the ability to write members of the space as of vectors from the frame

(Xn).

L nEN anXn in terms

7. Wavelets

7. 1

413

Orthonormal Basis from One Function

We investigate primarily orthonormal bases for L 2 ( R) . One way to create an orthonormal basis for L 2 ( R ) is to convert an orthonormal basis for L 2 [0, 1] into an orthonormal basis for L 2 ( R) by translation, dilation, and modulation ( i . e . , multiplication by e 2 1Ti k t ) . By Example 3 .4.6 ( b ) , the family { e21Ti k t : k E Z } is an orthonormal basis for L 2 [0, 1 ] . Now consider truncated translates of the e 2 1T i k t :

gk,n (t) = e 2 1Ti k t 1[o,l] (t - n ) , k , n E Z,

(7 .2)

where 1[0,1] is the indicator ( = characteristic ) function of [0, 1 ] . For n fixed, knowing that { e 2 1T i k t : k E Z } is an orthonormal basis for L 2 [0, 1 ] , it re quires only a change of variable to see that gk,n , k E Z, is an orthonormal basis for L 2[n, n+l ] . We use this fact in the theorem below to paste together an orthonormal basis for L 2 (R) . Since we will be using it, recall the following criteria for an orthonormal set to be an orthonormal base:

3.4. 1 An orthonormal subset S of an inner product space X is an orthonormal base ( a) if and only if S.L = ( b ) if the linear span [S] is dense in X .

{O};

7 . 1 . 1 ORTHO N O RMAL BASIS F O R L 2 ( R ) Th e {gk,n : k, n tion {7. 2} is an orthonormal basis for L 2 ( R) .

E Z}

of equa

Proof. If (k, n ) =f:. (k' , n'), then (gk ,n , gk',n') = 0 by the following argument. If (k, n ) =f:. (k' , n' ) , then n =f:. n' or k =f:. k' . If n =f:. n', then gk,n and gk ' ,n ' have disjoint cozero sets so the integral defining the inner product is 0; if n = n' and k =f:. k' , the orthogonality argument is the same as for the e 2 1Ti k t on [0, 1 ] . It is trivial to verify that I Igk,n ll 2 = 1 for all n and k in about the same way as the argument for e 2 1T i k t on [0, 1 ] . To see that the gk ,n are an orthonormal basis for L 2 ( R) , we use the criterion of 3.4. 1 and show that if f is orthogonal to all of them, then f = O . Note that for any fixed n, {gk ,n : k E Z i s an orthonormal basis for L 2 [n, n + 1 ] , s o (f, gk,n) = 0 for all k implies that f = 0 on [ n, n + 1 ] ; since n is arbitrary, it follows that f = 0 ( as a member of L 2 ( R)) . 0

}

Unfortunately, the gk,n of equation ( 7 .2) do not constitute the sort of basis that we seek . We desire orthonormal bases for L 2 (R) that are much more selective in characterizing the local behavior of certain functions. In connection with the manufacture of bases by modulation and translation of a single function h, the BALIAN-Low THEOREM 7 1 . 2 shows that Ih l 2 2 or �h ( where �h denotes the Fourier transform of h) must have an infinite second moment.

11

.

414

7. Wavelets

7.1.2 NECESSARY CONDITIONS TO MANUFACTURE BASES If

L 2 -Fourier transform of h E L 2 ( R) .

Let Ii be the

h k , n (t) = e 2 1ri k th ( t - n ) , k , n E Z,

is an orthonormal base for L 2 ( R) , then either

00 1- 00 t2 1h (t) 1 2 dt =

00

or

00 w2 11i (w) 1 2 dw = 1- 00

00 .

Since we will not make use of 7.1 .2, we do not prove it; for a proof, see Hermindez and Weiss 1 996 , p. 7f.

Exercises 7 . 1

1.

SOME UNITARY MAPS Show that each of the following linear opera tors of L 2 (R) onto L 2 ( R ) are unitary (i.e . , inner product preserving ) . For f E L 2 ( R) :

( a) TRANSLATIO N Ta f (t) = f (t a) , a E R. ( b ) MODULATIO N Ea f ( t ) = e 2 1r i at f ( t ) , a E R. ( c ) DILATION Da f (t) = l a l - I / 2 f ( t a ) , a =p 0, a E R. -

f

7.2

M ultiresolu tion Analysis

We consider the idea of a MULTIRESOLUTION ANALYSIS (MRA) in Defini tion 7.2.1. MRAs produce ( 1 ) an orthogonal direct sum decomposition of L 2 ( R) , and (2) a "mother wavelet" 1/J ( t) such that the "wavelets" 1/Jj, k ( t ) = 2j/ 2 1/J (2jt k) , j, k E Z , comprise an orthonormal basis for L2 (R) ( Section 7.4 ) . If, for example , coz 1/J = [0, 1], then coz 1/Jj , k = [2 - j k, 2 - j (k + 1)] ; so the wavelets become very highly concentrated as j increases. The orthonor mal basis of wavelets enables us to approximate a function from L 2 ( R ) by a finite sum of wavelets to arbitrarily high precision; the concentration per mits the use of fewer of them in some imp ortant cases. The course we follow is to go from MRA to wavelet bases. In a sense "most" wavelets arise from MRAs, a point taken up in Hernandez and Weiss 1996 , Chapter 7. -

Definition 7.2.1 MULTIRESOLUTION ANALYSIS

A sequence of closed subspaces { Vj : j E Z} of L 2 ( R) together with a function r.p E Va is called a MULTIRESOLUTION ANALYSIS ( MRA ) if it satis fies the following conditions: (a) (Increasing )

.

.

, C

V- I

C

Va

C

VI

C . . ..

7. Wavelets

415

( b ) (Densit y) cl Ui e Z \rj = L 2 (R) .

( c ) ( Separation) ni e Z \rj =

{O}.

(d) ( S caling ) f ( t ) E \rj if and only if f (2 t ) E \rj + 1 ' (e) (Orthonormal basis ) There exists a S CALING FUNCTION ( of the MRA ) E Va whose integer tr anslates (t - n ) : n E Z are an 0 orthonormal base for Va .

cp

{cp

}

The scaling condition shows that there is really only one space, Va say: all the others are scaled versions of that prototype, different "resolutions" of Va . To illustrate the point , we show in 7.2.4 that ( e ) implies that an orthonormal base for any \rj , j E Z, is given by translates of normalized dilations 2j f 2

cp

cp

Example 7.2.2 A MULTIRESOLUTION ANALYSIS

The subspaces of square-integrable, piecewise constant functions Vj =

{ ! E L 2 ( R ) : f constant on [2-i n, 2 -i (n + 1 ) ) for all n E Z } , j E Z,

cp

with scaling function = 1 [ 0 , 1 ) are an MRA. Va is the closed linear subspace of L 2 ( R ) of piecewise constant functions on intervals [ n , n + l ) for all n E Z having jumps ( possibly ) at integer values. V1 consists of functions constant on intervals [n /2 , (n + 1 ) / 2) of length t , V- 1 of functions constant on intervals of length 2, and so on. Clearly, the functions { ( t - n) = l[n,n + 1 ) ( t ) : n E Z } are an orthonormal set . Let [ ] denote linear span and observe that

cp

Va

=

cp

cl [ ( t - n ) : n E Z] =

{cp

{ L ancp (t L la l oo} . neZ

- n) :

}

neZ

n2<

I t follows from 3 .4. 1 (b ) that (t - n ) : n E Z is an orthonormal basis for Va . A similar argument on intervals [2 -i n , 2-i (n + 1)) using

416

7. Wavelets

instead of I{) yields that Vi = cl [l{)j ,A: : j, k E Z] .

The increasing property ( Vi C Vi + 1-the bigger j is, the smaller the inter val on which the functions are constant ) and scaling property E Vi if and only if f E Vi + d are trivial to verify. As to the separation condi tion, if E Z Vi , then f is constant; since E LdR) , this means that f = 0 a.e. Therefore , = 0 as a member of L 2 ( R) . The density condition (cl U · E Z Vi = L 2 (R) ) follows from the fact that any E L 2 (R) can be appr �ximated in the L 2 -norm by a piecewise constant function with jumps at the binary rationals ( see Exercise 3 .5- 1 ) . 0 We make two elementary general observations about MRAs I{) ) in 7.2.3 and 7.2 .4.

(2t) f ni E

(f (t)

f

f

f

((Vn),

Since the Vn are increasing, U n EZ Vn = Un >k Vn for any integer k, it follows that cl Un� k Vn = L 2 (R) for any integer-k . Using an argument similar to that of 7 . 1 . 1 ( i.e ., fix and let j vary ) , it is 7.2.3

n

easy to show that dilated translates of the scaling function yield orthonor mal bases for the various

Vn .

7.2.4 S CALIN G FUNCTIO N YIELDS ORTHONORMAL BASES

mal base for Vi for any j E Z is given by Proof. By the change of variable u =

Vo

f ( t)

--+

2t,

A n orthonor

{2i / 2

Vi ,

..,fif (2t) , is an inner product isomorphism . Hence { ..,fi

is an orthonormal base for Vi .

(7.3)

�

is an or

0

Vo VI {..;21{) (2t - n : n

Since I{) E and C ) E Z } is an orthonormal base for we can write in terms of its own dilated translates:

VI ,

k EZ

( 7.4 )

Ck are the Fourier coefficients of

where the

n

7. Wavelets

41 7

Note that the bisequence ( Ck ) belongs to £2 (Z) . If we let

then

ck ..fi = 2

=

ak

cp (t) =

1:

If'

(t) cp (2t - k) dt,

L a k CP (2t - k)

keZ

in the L 2 -norm.

By the Pythagorean theorem 3.3 . 4( g ) , 1

=

I! cP l!; L Icd = L l a; 1 =

so

(7 .6 )

2

keZ

keZ

,

L l a k l 2 = 2.

(7. 7)

keZ

As we discuss in Exercise 1 , equation ( 7.7) can be extended to

L ama2 k+m 2 0k , =

me Z

O

( Kronecker delta) .

(7.8)

MRAs Produce Orthogonal Direct Sums We show in equation (7 . 12 ) that an MRA ((Vn ) , cp) produces an orthog onal direct sum decomposition of L 2 ( R) = ffi ne Z We show in Section the mother wavelet that there exists a function such that the family 7.4 ) ( - ) E Z} of translates of t/J is an orthonormal basis for Wo o It then follows ( Section 7.3 ) that - k ) : k E Z } is an orthonormal basis j E Z . Therefore , since L 2 (R) = ffi n e Z the for - k) , as j and k vary, are an orthonormal basis for L 2 ( R) . Since the closed subspace Vo is contained in VI , by the projection theorem 3 . . 4, Vi decomposes into the orthogonal direct sum

Wn. t/J

{t/J (t n : n Wi ,

{2j f 2 t/J (2i t

2

VI

=

W,

2j f 2 t/J (2i t

Vo EEl Wo ,

where Wo is the orthogonal complement vl of Vo in VI . Since VI have V2 = VI EEl I , where I = V{ in V2 . Thus,

W

W

VI EEl WI = (Vo EEl Wo ) EEl WI . We can continue like this: If Wn Vnl. in Vn + then Vn + I = Vn EEl Wn = . . . = Vo EEl Wo EEl WI EEl · · · EEl Wn , V2

=

I,

=

n E N.

C

V2 , we

(7.9)

The closed subspaces Vo , Wo , WI , . . . are mutually orthogonal by their con struction; we show next that M, their orthogonal direct sum ( Definition 3 .7. 1 ) , is L 2 ( R) -namely, with some abuse of notation, that M

�

Vo
(�w")

�

L, (R) .

(7 . lO )

418

7. Wavelets

3.2.5, it suffices to show that M .L = {O} . If f E L 2 ( R) is orthogonal to Wn , n � 0, it is orthogonal to all the Vn , n � 0, well by equation (7.9); therefore [by Exercise 3.2-1] ' f cl U n Vn = L 2 (R) , so f = 0, which establishes equation (7.10). By the projection theorem 3.2.4, Vo decomposes into the orthogonal di rect sum Vo = V- I EEl W-1 , where W- 1 = V.:\ in Vo . Now split V- I into orthogonal subspaces V- 2 and By

Vo and all the

.1..

>

O

as

W- 2 = V!-2 in V- I to get

Continuing the process ,

To see that

(7.11)

we show that the orthogonal complement of : W- n in Vo is {O} . To this end, suppose that 9 E Vo is orthogonal to : W- n · Since 9 E 9 E W:n for every E Since each W- n equals V!-n in V- n+ it follows that 9 E V-n . Since the Vi are increasing , by the separation property ( Definition 7 c )) ,

[61:= 1 W_ n ] .L , 1,

n N. n:=o .2.1(

61 =1 61 =1

00

n V- n = n

n =O

nEZ

Vn = { O } ,

which establishes equation ( 7. 1 1 ) . Substituting tion it follows that

(7.10),

L 2 ( R) =

ED Wn ,

nEZ

where each Wn

61:=1 W- n for Vo in equa =

Vn.L in Vn + l .

(7.12)

Exercises 7 . 2

1.

(7.8). =P 0 (t) (2t n : n

f�oo (t (t)

(t) dt

cp - k) cp Verify equation (Hint: Use the fact that = for k and substitute for cp (t - k) and cp using the fact that cp - k) ; expand, using the orthogonality = L k E Z ck"\,/2cp of {cp - ) E Z}.

o

(2t

7. Wavelets

41 9

Mother Wavelets Yield Wavelet Bases

7.3

((Vn ), CP), Wo Vl 7.4

1/J

{1/J (t

Z}

Given an MRA - n) : n E is an a function such that orthonormal basis for = in VI is called a MOTHER WAVELET. In this section we show that a mother wavelet produces an orthonormal basis we consider a way to tease a mother wavelet out for (R) . In Section of an MRA. Let be a mother wavelet for the MRA ((Vn ) , cp). Consider the linear isometry ! � V2 ! of Vo onto VI of equation of Section Since

L2

1/J

1

00

- 00

(t)

!

(2t)

(7.3)

7.2.

(t) 1/J (t n ) dt = "21 1 00 ! (2t) 1/J (2t - n ) dt = 0, n E Z, ! E Vo , -

-oo

1/J (2t - n {V21/J (2t n : n Z} { 2i/2 1/J ( 2j t - k) : k E Z } is an orthonormal basis for Wj , j E Z. Since L 2 ( R ) = ffi jEZ Wj ( equation (7 . 12) , Section 7.2), it follows that the

it follows from the scaling property that ) E Viol in V2 for every n and that ) E is an orthonormal basis for Wt ; similarly, -

WAVELETS (O NDELETTES )

(7.13 )

are an orthonormal basis for R) . 1 t is called an O RTHONORMAL WAVELET BASIS with MOTHER WAVELET Since : j, k E is an orthonormal basis for (R) , any ! E (R) can be written in (f, !=

{1/Jj , k

Z}

L2 ( 1/J.

L2 11 · 11 2 ·

L2 (7.14)

L L 1/Jj ,k) 1/Jj , k kEZjEZ Under suitable restrictions on cp and 1/J, the series converges pointwise at ev ery Lebesgue point, hence almost everywhere ( Hernandez and Weiss 199 6 , p . 22 8 ). Now let (7.15) so that {CP O,k : k E Z} denotes our original orthonormal basis for Vo . Since Vo (ffi:=o Wn) = L 2 ( R) by equation (7.10) of Section 7.2, another or thonormal basis for L 2 ( R ) is {1jJj,k j , k E Z , j � O} U {CPO , k : k E Z} . (7 . 16) Ef)

:

To see what happens in a particular case, consider the MRA of Example

7 . 2.2.

420

7. Wavelets

Example

7.3.1

HAAR WAVELETS

7.2.2 consists of the closed subs paces Vi = { J E L 2 ( R) : f constant on [2- i n, 2 - i (n + 1 ) ) for all n E Z } , j E Z, with scaling function cp = 1 [ 0,1 )' The MRA of Example

Let

(7.17)

t/J (t) = cp (2t) - cp (2t - 1 ) . o.�

-0. 2

0.2

u

0.4

0.6

0.8

1 .2

-0.1

The

Haar Mother Wavelet t/J

We show that t/J is a mother wavelet for ((Vn), cp) , i.e., that {1/J (t ): n E Z} is an orthonormal basis for Wo = Vl in V1 · Clearly, t/J E V1 , t/Jl.. cp , and t/Jl.. cp (t - n) for all n E Z; therefore, t/Jl.. Vo and t/J E Wo o To see that {t/J (t - n) : n E Z} is an orthonormal basis for - n

Wo , consider

t/J (t - n) = cp (2t - 2n) - cp (2t - 2n - 1) E V1 . It is easy to verify (t/J changes sign while cp remains constant where they

are nonzero ) that Thus For f E Suppose

n vl

cl [t/J (t - n) : n E Z] C V1 = Wo o since f E f is constant on all intervals [n/2, ( n + Since f l.. cp , we have on and f = b on [

v1 nvl , f=a

t/J (t - n) .1. cp (t - k) for all n, k E Z.

[0, t) 1

V1 ,

t, 1).

a

1o f(t) dt = 0 = 2 + 2 b

=?

b=

1) /2) .

-a,

f = at/J on [0, 1). Likewise , for some constant C 1 , f = c 1 t/J (t - 1) on [1, 2), etc. In other words, there must be a bisequence (Ck ) such that f (t) = L C k t/J (t - k) pointwise a.e. and in 1I · lb . keZ

so

n

7. Wavelets

n

42 1

n

Therefore , ! E cl ["p (t - ) : E Z] = Wo . Since {"p (t - n ) : E Z } is an orthonormal basis for Wo and L 2 (R) = EB n eZ Wn (Section 7.2, equation (7. 12)), it follows that the HAAR WAVELETS

{ "pi,k (t) = 2i / 2"p (2i t - k)

:

j, k E

Z

}

are an orthonormal basis for L2 (R) . Since Vo Ef) ( EB::'= o Wn ) = L 2 ( R) ( Section 7.2, equation that Ni,k : j, k E Z, j � O} U {If'O ,k : k E Z}

(7. 10) , it follows

is another orthonormal basis for L 2 (R) . If we restrict the functions to [0, 1], then the family

{ "pi ,k j , k E Z, j � 0, k = 0, 1 , . . . , 2i - l } U {If'} :

is an orthonormal basis for o

L 2 [0, 1], the result we obtained in Section

3.5.

Exercises 7 . 3 AND PROJECTIONS Let ((Vn ) , If' ) b e an MRA with mother wavelet "p . For each j E Z, let Pi be the orthogonal projection of L 2 (R) on \tj , and let Qi be the orthogonal projection of L 2 ( R) on Wi Show that:

1. MRAs 0

( a) Pi - Pi - l = Qj - l , for all j E Z. ( b ) Writ e ! E L 2 ( R) as ! = Lj e Z Lke Z ( f, "pj,k) "pj,k as in equa tion (7. 14) . For m E Z, let

!m Show that

Z.

m- l =

I: I: ( f, "pj,k) "pj,k o

j = - oo ke Z

(fm , If'm ,k ) = (f, If'm ,k ) and Pm! = !m for all k, m E

2. Suppose that the dilation equation If' (t) = Lk e Z a k lf' (2t - k) is a finite sum and that If' E Ll (R) with f�oo If' (t) dt :/; 0. Show that L n e Z a n = 2. (Hint: Integrate the dilation equation termwise. )

422

7. Wavelets

7.4

From MRA t o Mother Wavelet

7.3

'IjJ

The results of Section show that a mother wavelet yields an orthonor mal basis : j, E Z} for L 2(R) , something that will permit us to write everything in terms of the Thus, if we have a mother wavelet, we have something quite useful. But how can we get one? More fundamentally, is is there E Wo = VI n such there one?-given an MRA ( ( Vn ) , that {'IjJ : E Z} is an orthonormal basis for Wo ? After some general observations we show that a mother wavelet is obtained from the scaling function by

{'IjJj , k k

(t - n)

'ljJj , k .

tp),

n

a 'IjJ (7.4.8)

Vl

'IjJ(t) = L (v'2) C l_k (-1) k tp(2t - k), kEZ

where

Ck = ( tp (t) , v'2tp (2t - k) ) = v'2 1: tp (t)

for almost every w E R . Proof. Suppose first that the Fourier transform of from Parse val's identity

nEZ

{J(t - n} : n E Z} is an orthonormal family . Since f(t - n) is /(w)e-iwn ( Exercise 5.18-1), it follows (5.18.3, (f, g) = 21" (1, g)) that 1: f(t)f(t - n) dt (f(t), f(t - n) ) (7.18) 1 ), f(w ) e -iw n ) -(f(w 2 11" 00 2 ( w ) 1 einw dw. � 1 1 1 2 00 �

-

7. Wavelets

423

where on , O denotes the Kronecker delta. Now split f�oo into a sum: 2 (k+ 1)1r oo 2 2 Thus

e i nw dw =

J-00 Il(w) 1 1

L k EZ 27r 1

L

1

Il(w ) 1

kE Z 2 k1r

1 2(k+I)1r Il(w) 1 2 einw dw 2 k1r r 2 1r

As a consequence of equation

2 i e nw dw

2)

w + 2k7r) 1 27r L kE Z Jo Ij( 1 r 2 1r � I j(w + 2k7r) 1 27r Jo

(

e i nw dw.

(7. 19)

e i nw dw .

(7. 19), the 27r-periodic function 2

L I l(w + 2 k7r)1 kEZ h as Fourier coefficients Cn equal t o zero for n :j:. 0 and Co = 1 . Thus 2 L Il(w + 2 k7r)1 = 1 a.e . kE Z

Reversing the steps in the previous argument proves the converse. 0 Since the integer translates {If' - k) : k E Z} of a scaling function form an orthonormal family an orthonormal base for Vo , we have the following corollary.

(

Corollary 7.4.2 of an MRA, then

(t

)

If (j5 is the L 2 -Fourier transform of a scaling function If'

L 1 (j5(w + 2k7r ) 1 2 = 1 for almost every w E R.

kEZ

(

)

If a linear electrical network linear system passes signals of certain frequencies and attenuates others, it is called a FILTER. Suppose that the response v of the system to an input u (t) is given by a convolution v = h * u for some h associated with the system (h is usually called the "impulse response" . Then, taking Fourier transforms,

(t)

)

v(w ) = h (w ) u (w ) .

( 7 . 20 )

h (w ) = 0 for I w l 2: Wo , then the system is called a "low-pass filter" ; if h (w) = 0 for I w l :s Wo , then the system is called a "high-pass filter." If h (w) = 0 for WI :S I w l :S Wo , then it is called a "band-pass filter ." Because they figure in equations like equation (7.20) ( see 7.4.5, for ex ample ) , we call the functions obtained in Definition 7.4.3 filters. If

424

7. Wavelets

Definition 7.4.3 FILTERS

((Vn) ,
Let be an MRA. Any g E VI can be written in terms of the orthonormal basis - k) E Z } for VI as

{ V2
Since

=

L: kEZ I bk 1 2 < 00, we can form the 27r-periodic function ( 7.21 )

mg

where T represents the circle group. The function is called the FILTER associated with g. The filter of a scaling function is often called the LOW-PASS FILTER associated with 0

mcp

<po

Thus, for example , if the dilation equation , (t) = for the scaling function

L:k EZ ck V2

its low-pass filter is

mcp (w ) = 21 + 21 e

- iw

.

Example 7.4.4 T H E HAAR FILTER

The scaling function for the Baar system of Example 7.3 . 1 is

(t)
Therefore, as above, the associated filter is

mcp (w ) = 21 + 21 -

mg

_e

.

- zw

•

0

mg

The filter of g enables us to express 9 in terms of and (j). This fact plays an imp ortant role in the subsequent development. 7.4. 5 FILTERS AND FOURIER TRANSFORMS Let ((Vn ) ,
7. Wavelets Proof. Choose a bisequence (bk ) E £ ( Z) such that

2 g(t) = L bk V2
- k) .

By Exercise 5 . 1 8- 1 , the L 2 -Fourier transform of

Hence , by the continuity of the map f 1---7

- k)

425

(7 .22) is

j (Plancherel's Theorem 5 . 1 9 .3 ) ,

g(w) = "'" k Z bk\l2 � e- iw k / 2 ip( �2 ) L....J E 2 = L k E Z � e - i W k / 2) ip( i )

(

or

= mg ( ) ip( ) i i ,

g(2w)

m g (w) ip (w ) . 0 The scaling identity of 7.4.6 is imp ortant in the sequel.

( 7 .23)

=

7.4.6 THE SCALING IDENTITY For an MRA « Vn ) ,
1 = I mrp (w ) 1 2

Hence mrp (w)

+ I mrp (w + 11") 12 a. e.

=P 0 or mrp (w + 11") =P 0 for almost every w E R.

Proof. By 7 .4.2 we know that l:k E Z lip(w

wE

(scaling identity) .

R. Hence l: k E Z l ip( 2w + 2 h ) 1 2

7.4.5, therefore

=

+ 2 h ) 12 = 1 for almost every

1 a.e. as well . By the filter equality

2 L I mrp (w + h ) 1 I ip( w + h ) 1 2 = kEZ

1 for almost every w E R.

Breaking the summation into sums over the even and odd integers , we get that 1

= L k E Z I mrp (w + 2 h ) 1 2 I ip(w + 2 h ) 12 + L k E Z I m rp (w + (2k + 1 ) 1I" ) 1 2 Iip (w + (2k + 1 )11") 1 2 .

Since mrp is 211"-periodic, 1=

I m rp (w) 12 L k E Z lip (w + 2 h ) 12 + Im rp (w + 11") 12 L k E Z l ip(w + (2k + 1 )11") 1 2

.

7. Wavelets

426

The sums in the above equation simplify: By 7.4.2,

L I�(w + 2h) 1 2 = L I �(w + (2k + 1)11") 1 2 = 1 for almost every E R. kEZ kEZ W

Therefore,

1 = I mrp(w) 1 2 + I mrp(w + 11" ) 1 2 for almost every w E R.

0

( 7. 24 )

E Wo , the form of mg simplifies. 7.4.7 REPRESENTATION OF THE FILTER mg FOR 9 E Wo For an MRA ((Vn) ,
P roof. We

claim that

mg(w)mrp(w) + mg(w + 1I")mrp(w + 11") = 0 for almost every w E R. (7.26) To prove this, note that since 9 J.. Va , 0 = I: g (t)
-

-

11"

and summation,

o

I: y(w)�(w)k einw dw

r 2 ( + I ) ?r y(w) �(w) e inw dw

L Z J2

E �: + 2 h)f(w + 2h),'l:' a kEZ

dw .

L e y(w + 2h)�(w + 2h) are zero for n E Zj w k2w,Z L y(2w + 2h)�(2w + 2h) = 0 for almost every w E R . ( 7 . 28 ) kEZ B y the filter equality 7.4.5, y(2w) = mg (w )�(w) and � (2w) = m rp (w) � (w), Thus, the Fourier coefficients of all therefore, replacing by

so equation ( 7.28) implies

o

L kEZ mg(w + h)�(w + h )mrp(w + h)�(w + h) a.e . L kEZ mg(w + h) I�(w + h) 1 2 mrp(w + h) a.e .

7. Wavelets

427

Breaking this sum into sums over the even and odd integers , we obtain mg ( w + 2 k1r) Icp(w + 2 k1r) 1 2 mrp (w + 2 k1r) mg (w + 2 k1r + 11" ) Icp (w + 2 k 1r + 11" ) 1 2 m-rp"""(w1I"-+-1I"-"-) = 0 a.e . + +-2-kSince mg and mrp are 211"-periodic, the mg and m rp factor out to yield

Lk E Z L k EZ

It follows from 7.4.2 that each of the summations is 1 and therefore that

---,- ---,.-

W

which verifies equation ( 7 .26....) .. _ By 7 .4. 6 , mrp (w ) 0 or mrp ( w + 11" ) 0 for almost every E R, so the vector (mrp ( w ) , mrp ( w + 11" )) is nonzero almost everywhere. By equation = (7.26) and the definition of the inner product in C 2 ( namely, {(a, b) , ( c , ac + bd) , the vector ( mg (w) , mg ( w + 11" )) is perpendicular to the vector (mrp ( w ) , mrp (w + 11" ) ) a.e . But �rp ( w + 11" ) , - mrp (w ) is also perpendicular to (mrp ( w ) , mrp (w + 11" )) everywhere. Since C 2 is 2-dimensional , it follows that there exists a function a ( w ) such that

=P

=P

(

)

or

d))

(7.29)

and

mg ( w + 11" ) = - a (w ) mrp (w ) a.e . Since mrp is 211"-periodic, equation (7.29) implies that

(7.30)

Therefore , by equation ( 7.30 ) , a(w + 1I" ) m rp (w ) = - a(w ) mrp ( w ) a.e. Since mrp (w)

=P 0 or mrp (w + 11" ) =P 0 for almost every w E R (7.4.6) ,

a (w + 11" ) = - a ( w ) and a ( w + 211" )

=

- a(w + 11" ) = a(w ) a.e .

Since a (w + 1I" ) = - a(w ) = e - i 1f a(w ) a.e . , it follows that q (w ) = ei(1f+ w ) a(w ) has period 11". Consequently, there is some 211"-periodic function f3 such that a(w ) = e - i(w+ 1f ) f3 (2w ) a.e. Substituting for a in equation (7.2 9 ) , we obtain (7.3 1 )

428

7. Wavelets

L 2 (T) , we show that a E L 2 (T) . Since mg E L 2 ( T ) , I I mg ll� 1 2 1f la (w )12 I m

To see that f3 E

i a (x + 1I") 1 2 Im

I m
Since 1 =

l1f l a (w) 1 2 dw,

which completes the proof since a is of period 11". 0 We now proceed with the construction of a mother wavelet from an MRA Since we w.ant a function tf; E Wa = n vl , by 7.4.7, there must be some 211"-periodic function f3 E L (T) such that

((Vn) , cp).

VI

2 ) m.p (w) = e-i(w+ 1f f3(2w)m
Let us arbitrarily take f3 == 1 . Then, by 7 .4 . 5,

(7 .32)

(7.33)

We show in 7.4.8 that this tf;, the one whose transform satisfies equation cp) . (7.33) , with this arbitrary choice of f3, is a mother wavelet for The function tf; can then be found ( in theory) by the inversion formula of 5 .19.2: tf; (t) = Li .m . ;j (w) eiwt dw. 211" - n

((Vn) ,

n J.- j

7.4.8 MOTHER WAVELET THEOREM

sider

tf; (t) =

such that

n

Let

((Vn) , cp) be an MRA, and con

L Cl - k (_ 1 ) k v'2cp ( 2t - k) E Va ( ( Ck ) E £2 (Z »

kEZ

� tf;(w ) = e - '·( w + ) m
11"

W

Then tf; is a mother wavelet for the MRA is an orthonormal basis for Wa = n

W

((Vn ) , cp), i. e., {tf;( t - n ) : n E Z} VI Vl.

7. Wavelets

42 9

P roof. We start by assuming that we know the transform -if; and work backward to Since rp E L 2 ( R) and i s bounded a.e . (7.4.6), i t follows that -if; E - : L 2 ( R) j consequently, ,,p E L 2 ( R) . We next show that the family E is orthonormal. To prove this, it suffices to establish equation (7.34) below, and use the criterion of 7.4. 1 . Observe that

"p.

n

I m

Z}

{"p(t n )

L nEZ I -if;(w + 2 mr) 1 2 2 LnEZ I m

By 7.4.2 and 7.4.6, this yields

-n

LnEZ I -if;(w + 2n 1l") 1 2

t-n

= =

I m
1 a.e.

(7.34)

To show that each "p ( ) is orthogonal to Vo , we show that each ) is orthogonal to every ( - k). By Plancherel's theorem (5 . 19 . 3) , it is sufficient to show that ( "p ('t-:: k)} = 0 for all k, E To show that is orthogonal to Vo , we show that k )} = 0 for all E Since the Fourier transforms of "p and - k) are equal to -if; - i w and - k , respectively, the result will follow by a simple change of variables . By the filter equality 7.4.5, = so

"p ( t

n

Z. e (w) n

"p(t)

rp(w ) e i w

cp t

n Z. n), cp(i:: (i{i), cp(t:: (t - n) cp(t rp(w) m
(rp(w)e-ikw, -if;(w)}(rp(w) e - ik w, ¢(w)) - i: m
11"

)

7. Wavelets

430

y + 2mr, since m

x

=

1l"

21r (
an

n

L kEZ � e -i 1r( k + l ) e -i (W/2)( k + l )
7. Wavelets

43 1

1/J E VI ' Hence 1/J(t - n ) E VI for all n E Z. To see that Wo ( the orthogonal complement of Vo in VI ) is an orthonormal basis for Wo , suppose that g E Wo o By 7.4.7 and 7.4.5, g(w) = e -i ( � +") ,8(w)m cp (!f + 7r) �( � ) = ,8 (w) J;(w) a.e. Since ,8 E L 2 (T), we can write it a Fourier series, ,8 (w) = L n EZ an e i n w , so that g(w) = a k ei k w J;(w) = a k e i kw J;(w). kEZ kEZ Since the in verse of e i nw J;(w) is 1/J (t + n ) , it follows that g (t) = a k 1/J(t - k), kEZ which demonstrates that any g E Wo can b e expressed in terms of the and therefore that

{1/J(t - n ) : n E Z }

C

as

(L )

L

L

translates of 1/J.

0

7.4.4, the dilation equation for the Haar scaling

-

-

-

A s shown in the body of Example 5 .15 .5, sin w /2 cp�(w ) - e -iw /2 2 w /-. _

Knowing

1/J, we compute J; directly as oi.( _ _ i w / 2 sin 2 w/4 w ) - ze w /4 . .

'P

(7.36) (7.37)

mcp (w) = � + (�) e -iw , it is now easy to verify that J;(w ) = m l/J (w/2) � (w/2) e -i(w /2+") m cp (w/2 + 7rW(w/2) a.e. (7.38)

With

=

432

7. Wavelets

Exercises 7 . 4 In the exercises below ,

((Vn) , ip ) denotes an MRA.

1. Verify equations (7.37 ) and (7 . 38 ) .

2.

PROPERTIES OF THE LOW-PASS FILTER Let m", denote the low-pass filter associated with ip. Assume that tP(O ) # o .

E kEZ Ck V2ip (2t - k) E kEZ Ck = V2. En EZ (-It an = O . Let T denote the circle group . Show that if g E L 2 (T) , then g tP E (a) Consider the dilation equation ip (t) = Show that ( equation (7 .4) of Section (b) If m", is continuous, show that

7.2).

3.

L 2 (R) .

4. REPRESENTATION IN

Va =

Va Show that

{f E L 2 ( R) : f(w) = ex(w}tP(w) for some ex E L 2 ( T ) } .

5 . BOUNDEDNESS AND ORTHO N O RMAL FAMILIES Show that if

{f(t

E

fE

L 2 (R) is such that n) : n Z} is an orthonormal family, then :S 1 for almost every R.

6.

l i(w) 1

-

wE Use Exercise 4 and Definition 7.2.I( d ) to determine V1

Fourier transforms as was done in Exercise 4 for

{i : f E

in terms of

E Z, let Wj = Vji. in 0 + 1 0 Wj .

S} . For each j 7. For S C L 2 ( R ) , let S = = so that Vi l = E8 Wj . Show that

8.

Va .

Vi + ! + Vi Let 'lj; be the mother wavelet of 7.4.8. Show that 00 2 I tP(w ) I 2 = L 1 'lj;(2'w) 1 for almost every w E R. i= l

�

E8

.

9 . DECOMPO SITIO N AND RECONSTRU CTION

{ip (t

E Z } and {'lj; (t - k) : j E Z} are orthonor

( a) Since - n) : n mal bases for Va and

{ ip (t

Wa = Vl n Vl , respectively, -

n) , 'lj; (t - k) : n, k E Z}

7. Wavelets

433

Vi Vo Wo. Since {-J'it.p (2t - n) : Vi , for any f E Vi there (an , d, (an , o), (bn,o) such that f (t) = � L.....Jn E Z an ) 1\12 t.p (2t - n) = � n an , ot.p (t - n) + � �n E Z bn ' o'lj; (t - n) . L....J E Z By the dilation equation ( equation 7. 4 , Section 7.2), t.p (t - n) = ..J2 L Ck t.p (2t - 2n - k) . kEZ

is an orthonormal basis for = EEl E is also an orthonormal basis for exist bisequences and

n Z}

B y 7 .4.8,

'Ij;

(t - n) = ..J2 L ( _ I) k Ci_ k t.p (2t - 2n - k) . kEZ

Show that

a n,1 = L kEZ Cn - 2 k a k,O + L kEZ (-I t Ci - n +2k b k ,O ( the reconstruction)

and

bn , 0 = � ( _ I) k Ci - k +2na k ) 1 E Z L... .J k '( the1 and decomp osition) .

an 0 = � � kE Z Ck -2n a k

'

I

( b ) Extend these results to any \tj .

Hints

2.

(a) It follows from the filter equality of 7.4.5 that

434

7. Wavelets

E Vo , then f(t) = EkeZ bk rp(t - k ) with E keZ I bk l 2 < 00 . Then j(w) =
4. If f

5 . Use 7.4 . 1 .

8 . B y 7 .4.5 and 7.4.8,

1
I� . 2 � .,p(2'w) 1 for almost every w E R. 1
( 7.39)

By equation (7.39) and 7.4.2, it follows that

�(2 k w) 1 2 :::; 1
(E�= I I �(2k w) r )

n

E

N.

is a bounded increasing sequence, so it con

n w) 1 2 exists. Since
-+

By Fatou 's lemma,

1: li� 1
n -+ 00 .

n 1: 1
< li

m

1
Hence , limn yields the result.

7. Wavelets

435

9 . ( a)

1

" �"kE Z V2Ck CP (2t k- 2n - k) �n E Z an ' 0 " " +� n E Z bn ' o � kE Z V2 (-1) C 1_k CP (2t - 2n - k) .

(t)

2n + k = j we get " 1 (t) �n E Z an ' " �J. E Z Cj - 2n V2CPi(2t - j) +" �J. E Z V2 (_l) 1 -j +2nCP (2t - j) . �n E Z bn ' 0 " Since 1 (t) = " nE a n ' 1 V2cp (2t - n), by the uniqueness of the co� Z With

=

°

C

efficients ,

an ,l L Cn -2 k a k ,O + L ( I t C1 - n +2kbk , O . kEZ kEZ =

Next ,

an,O

-

{j, (t - n) ) ( I, L kEZ V2Ck CP (2t - 2n - k) ) L kEZ Ck a 2n + k ,1 " � kE Z Ck - n a k ' The metho d for the expression for bn,o is similar. ( b ) Recall that {2i/2 cp ( 2i t - n ) ; n E Z } is an orthonormal basis for Vj , while Vj = V} - 1 El1 Wi- 1 and cP

=

2

1·

is also an orthonormal basis for Vj . With

C k , l = Ck ,

Now continue as in ( a) .

7.5

Construction of a Scaling Function with Compact Support

The SUPPORT of a function 1

;

R

�

C is the set

supp i = cl coz l = cl {t E

R ; / (t) # O } .

436

7. Wavelets

If supp f is compact , we say that f has COMPACT SUPPORT or is COM PACTLY SUPP O RTED . Functions that vanish outside a closed interval such as the scaling function I{J = for the Haar wavelets ( Example 7.3 . 1 ) have compact support. A scaling function I{J is one that generates subspaces (Vn ) such that ((Vn ) , I{J ) is an MRA. In this section we exhibit a way to fabricate scaling functions with compact support . The key to the process is the associated filter an essential fact being that if I{J has compact support , then its associated filter is a trigonometric polynomial. ( The converse is true, too: If is a trigonometric polynomial, then I{J has compact support. A proof is sketched in Exercise 1 .) First, we develop some simple properties of scaling functions with com pact support. The results are summarized in 7. 5 . 1 . We then work backward from these properties to get a scaling function. We first show the crucial property that if I{J has compact support , then its associated filter is a trigonometric polynomial. Given an MRA ( ( Vn ) , I{J ) , the scaling function I{J must satisfy the dilation equation ( equa tion (7.4) of Section 7.2) ,

1[0,1 )

m
m
l{J(t) =

L ck V2I{J ( 2t - k), where Ck = V2 j-OO00 l{J ( t )I{J ( 2t - k) dt . kEZ

If I{J has compact support , then supp I{J implies that coz
_

C

[- n ,

] for some

k) [k; k; ] C

n ,

n

n

.

E

n

(7.40)

N . This

k

Clearly, if k is sufficiently large, then the integrand 3 n , and 1{J(2t - k) 3n. Similarly = 0 for k < -3 n Thus, if I{J has compact support, there exists a positive integer M such that is a trigonometric polynomial:

Ck

(k

O. Ck

Ck

.

O.

m
M Ck ' "'" kW m

.

I{J

-

7. Wavelets

43 7

7.4.5, m

=1

the product as

n -+ 00 ,

, n

we obtain

(7.41) We summarize the above discussion: 7.5.1 PROPERTIES OF m

7· 4 · 6) ; (c)

m
If ip has compact support , then its mother wavelet is quite simple: Since the are zero for � n for some n E the mother wavelet of

N, ln 1/J (t) = L c l _ d -1) k J2ip (2t - k) , k l -n Ikl

Ck

7.4.8,

+

=

is a linear combination of functions with compact support; consequently, it has compact support, too. Now we retrace our steps. We work backward from a trigonometric poly nomial satisfying the conditions of to a scaling function ip with compact support. Suppose that = v'2) is a trigonometric polynomial satisfying conditions 1 a c Motivated by equation we define

m
7.5.1 e - ik w

m

TIi eN m

(7 . 41),

(7.42)

The point of the next result is to deduce convergence and continuity of

7.5.2 UNIFORM CONVERGENCE O F THE PRO DUCT

If

7. Wavelets

438

is a trigonometric polynomial satisfying the conditions of 7. 5. 1, then the product TI i EN converges uniformly on bounded subsets ol R . < 1 for all P roof. By 7.5.1, 1= and w E R. Hence

mcp(w/2i) mcp(w) -

mcp(w) - mcp(O) I mcp(w)1 ", n = bl l e - ikw 1 1 L.J k - n v'2 ", n = bl l ( e ikw / 2 - ie - ikw / 2 ) e - ikW / 2 2 i l L.Jk -n v'2 2 k v'2 �:= _ n I C k I sin ; I < v'2 �: = _n I C k I k; I < C lw l , where C = L:� = - n i c kl l k l /v'2. Let Pn {w) = TI;= 1 mcp{w/2i) be the nth partial product. Since I m cp(w)1 1 for all w E R, <

_

�

Thus , adding and subtracting some terms, we obtain

I Pn + m (w)

-

Pn {w) 1

< <

m

N

C ( 1 2:+ 1 1 + 1 2:+ 2 1 + . . . + 1 2 n:m I ) C I; I

for n , E It is now clear that the sequence on bounded sets. 0 .

(Pn{w)) converges uniformly

It follows from this uniform convergence on bounded sets that rp is con tinuous; since (0) = 1 , it follows that = 1 . What we would like to do now is compute the inverse transform of rp to get To do that , however, we need to know that rp L 2 (R) .

rp{O) cpo E 7.5.3 rp BELONGS TO L2(R) Let mcp (w) = L:� = - n ( Ck /v'2) e - i kw satisfy conditions 7. 5. 1 (aJ-(cJ, and let rp (w) = TIi EN mcp ( ; ) Then rp E L 2 (R), its inverse L 2 -Fourier transform cp E L 2 (R), and I cp l 2 1. Proof. Let Pn(W) = TIj= 1 mcp(w/2i) be the nth partial product, and define mcp

� .

7. Wavelets

Since

439

m

w +2n 1r

By changing the variable from w to in the integral J:,:';'" the limits become 0 and 2 n 1r ; moreover,

I Pn(w) 1 2 dw ,

so

=

2n" fo ! Pn _ i(w) 1 2 ( I m

Iteration yields

w by w + 1r in the integral J� " I m
Replacement of

=

21r by 7.5 . 1 ( b ) .

Therefore, applying I Pn (w) 1 2 I cp(w) 1 2 as (In) is a sequence of nonnegative integrable J In < then J limn inf In ::; limn inf J In <

Finally, we note that Fatou's lemma [namely, if functions for which limn inf

---+

n ---+ 00 .

00 ,

7. Wavelets

440

(Pn(w)1 [ -2 " 1f , 2 " 1fj(w») yields �2 i: I sO(w) 1 2 dw 2171' i: li�2 "I".Pn(w) 1 2 1 [_ 2 " ". , 2 " ".j (w) dw 1 2 2 71' liInn 1-2 " ". I Pn(W) 1 dw 2� liInn In = 1. Consequently, sO E L 2 (R). Hence , b y Parseval's identity ( 5. 18 .3 ) , 00 ] to the sequence

<

ip E L 2 (R), which completes the proof. Now that we know that sO E L 2 (R), it makes sense to talk about its inverse transform. Since sO(w/2) = ilj: l mrp(w/2j + l ), it follows that mrp (�) iljeN mrp ( 2i': 1 ) sO(w) mrp (�) sO (�) ("L...Jnk = - n �v'2 e - i kW/2) sO ( �2 ) . 0

and

Hence , taking the inverse Fourier transform,

1, ip

n ip(t) = L ck\/2ip(2t - k). k=-n

(7.43)

By Exercise has compact support; therefore , so does the associated mother wavelet of 704.8

n 1jJ (t) = L V2Cl _ k (-1) k ip(2t - k). k=-n that ip must satisfy is that {ip(t )

A condition - n : n E Z} be or thonormal . We show in the next example that there are filters that are trigonometric polynomials satisfying the conditions of 7.5 . 1 for which n ) : n E Z} is not orthonormal.

mrp

{ip(t -

Example 7.5.4 TRIG ONOMETRIC POLYNOMIAL NOT SUFFICIENT TO GEN ERATE WAVELET

mrp(w) = (1 + e - 3iw )/2 and sO(w) = ilj e N l t e -;'W/ 2J I mrp(w) 1 2 = "21 (1 + cos 3w)

Let see that

•

It is easy to

7. Wavelets

44 1

I m
B y Exercise 2 and the hint t o Exercise 2,

2J e iw/ 2 sinw /w2/ 2 = II 1 + e-iw/ 2 _

This can be rewritten

je N

as

1 - e-iw zw Consequently,

II 1 + e; iW /�J

j eN

�

and it follows that

1
-

n

:

-

n

n

0

: n

m
orthonormality. We show in 7.5.7 that if then the orthonormality condition of 7.4. 1 ,

g (w) =

L IsO(w + 2 k11') 1 2 = 1 a .e. keZ

is satisfied. ( Note that the filter of Example 7.5 .4 in particular, 11' 3 ) = 0.) First , we establish the following lemma.

m
does vanish on [-11' / 2 , 11' / 2] ;

=

L� = - n ( Ck / V2') e-i kw satisfy conditions 7. 5. 1 (a)-(c), and let � (w ) = ilje N m

7.5.5 L e t m

P roof. Clearly, g ( w ) is 211'-periodic. Since the partial sums of

L I �(w + 2 k11') 1 2 k eZ

2

7. Wavelets

442

are nonnegative, we can interchange summation and integration as follows:

[o 2 ... Ig (w) 1 dw

=

J

[ 2 ... L Icp(w + 2k7r) 1 2 dw 10o kEZ 12 = L kE Z ... I cp(w + 2k7r) 1 2 dw 1 (2k +2) 2 dv (v = w 2k7r) + L kE Z 2 h I cp(v) 1 i: I cp( v) 1 2 dv < by 7.5 .3. 0

'"

0

00

g (w) is a trigonometric polynomial. Lemma 7.5.6 Let mrp (w) = L � = - n (C k /.../i) e - ik w satisfy conditions 7. 5. 1 (a) (c), and let cp (w) = TIjE N mrp ( ; ) . Then g (w) = L kE Z I cp(w + 2k7r) 1 2 Next , we show that

-

is a trigonometric polynomial.

( )

P roof. By 7.5.5, 9 E L 1 T . Arguing as in 7.5.5, we compute the standard Fourier coefficients of g. By the dominated convergence theorem,

2 ... e - in t [ 27r Jo g (t)1 (2 k +2dt) 1 n 2 7r L kE Z 12 �� 2...k +2) ... e - z v cp( v )cp( v) dv 1 n (7.44) 2t lL� E Z .2h [e - z v cp(v) ] cp(v) dv n 27r 00 ( e z v cp (v) ) cp (v) dv i: cp(t - n)cp(t) dt [Parseval's identity (??)]. As already observed, cp has compact support . As in the argument following equation ( 7 .40) , we conclude that Cn = 0 for almost all i.e . , that 9 is a ...!:...

--

.

"'

.

--

_

-

trigonometric polynomial.

-

0

n,

mrp

{cp(t - n) : n

We now consider a condition on that makes E Z} orthonormal . For other sufficient conditions , see Prasad and Iyengar 1997 . )

(

{cp(t - n) : n E Z} Let n m rp (w) = L (ck / h) e - ik w k=-n satisfy conditions 7. 5. 1 (a)-(c), and let cp (w) = TIjEN mrp ( ; ) Ifmrp(w) t O for w E [-7r/2, 7r/2], then {cp(t - n) : n E Z} is an orthonormal family. 7.5.7 ORTHONORMALITY OF

.

7. Wavelets

443

L (R) , we can use 7.4. 1 to demonstrate the orthonormal ) 2 E Z} , namely, we show that g(w) = L lti?(w + 2 h) 1 2 = 1 a.e. kEZ A s a trigonometric polynomial, 9 is continuous; therefore, g (w) = 1 a.e . is equivalent to g( w) 1 . We show that ti?(2h) = I1j E N mcp (2h/2j ) = 0 for k :j:. O. Let k = 2Sq where q is odd. Consider I1j= l m cp (2 S + 1 - j q1l') . For 2 s + 1 , this product contains the term m cp (q1l') = m cp ( 1I' ) = 0 (7. 5 . 1 ) . Hence ti?(2h) = 0 for P roof. Since cp E ity of { cp(t n : -

n

==

n

k :j:. O. Consequently,

g(O) = L 1 ti?(2h) 1 2 = 1ti?(0) 1 2 = 1 . kEZ To show that 9 (w) = 1 for all w in [ -11', 11'] w e first show that g (2w) = I mcp (w) 1 2 g(w) + I mcp (w + 11') 1 2 g (w + 11') .

( 7.45 ) ( 7 .46 )

As in various arguments in Section 7.4, we prove this by splitting the summation into even- and odd-indexed terms. By the filter equality 7.4.5, it follows that =

ti?(w) mcp(w/2) ti?(w/2), g(2w) = LkEZ 1 ti? (2 (w + h)) 1 2 = L kEZ I mcp (w + h ) 1 2 1 ti? (w + h) 1 2 . Since m cp is 211'-periodic, by breaking the sum into its even and odd parts we have g(2w) = L kEZ I m cp (w) 1 2 1ti? (w + 2h) 1 2 +L kEZ I mcp (w + 1I') 1 2 1 ti? (w + 11' + 2h) 1 2 . Factoring I m cp (w) 1 2 and I m cp (w + 11' ) 1 2 we get equation ( 7.46 ) : g(2w) = I mcp (w) 1 2 g (w) + I mcp (w + 11') 1 2 g (w + 11') . Since g(w) is continuous, we may choose W o E [ -11', 11'] such that g(wo ) = min_ll'< w < ll' g (w) a . If g(wo/2) > a , use the fact that m cp (wo/2) :j:. 0 ( by hypot hes�) together with equation ( 7.46 ) to get the contradictory result ,

=

that

444

7. Wavelets

Thus = a. Repeating the argument, it follows that =a for all j E N . Since 9 is continuous and 9 (0) = 1 (equation (7.45)) it follows that (7.47) min g(O) = 1 .

g(wo/2i )

g (wo/2)

g(w) = a = We now proceed similarly with b = max { g (w) : w E [ ] } Let W l E ] be such that ) = b. Assume that 9 (wd2) < b. Since mil' (wd2) =f:. g(W l '11" - 'lr :::; W � 'Ir

- 'II" , 'II"

[ - 'II" , 0, equation (7.46) implies that

.

g (w l /2) = b. Proceeding as above , we have max { g (w) : w E [ ] } = b = g (O) = 1 . (7.48) It follows from equations (7.47) and (7.48) that g (w) = 1 for all w E R. Let mil' and

- 'II" , 'II"

0

{
Vo = cl [{

n

: n

C

-

The proof of the general result is now clear .

2.

The scaling property of an MRA is clear by definition.

n

: n

7 . Wavelets

445

3. The orthonormality property follows from the definition of Va .

4. That n n E Z Vn = {O} and cl U n E Z Vn

= L2(R) are left to Exercise 3 .

Exercises 7.5 1 . COMPACT SUPPORT If the dilation equation representation o f a scal

ing function

W

(c) If ip is continuous at 0 and cp(O)

=

1 , then cl Uj E Z Vi

= L2 (R) .

Hints 1 . The proof follows that of Daalhuis (Koornwinder 1993) . Alterna tive proofs using the theory of distributions appear in Daubechies 1992, Prasad and Iyengyar 1997, and Hernandez and Weiss 1 996 . Let g a (t) = 1 [ _ 1.2 ' l.2 l (t) . Define recursively gj (t)

= .J2 L Ck9j _ l (2t - k ) , j E N. kEZ

I t can be shown (see Daubechies 1 988) that n + . Let na, - = - � and 1 - 2 ' C onSI' d er n a, + gl (t)

= J2

L

n_ �k�n+

ckga (2t - k) .

446

7. Wavelets

9o {2t - k) will vanish unless n o, - = - t ::; 2t - k ::; t = n o, + or (n o, - + n _ ) /2 ::; t ::; (n o, + + n + ) /2. Hence , the support supp 9 1 is contained in [n ,_ , n l, + ], where n l, - = (n o, - + n _ ) /2 and n l, + = (no , + + n + ) /2. l In general, let nj,_ = (nj - l, - + n _ ) /2 and nj, + (nj - l, + + n + ) /2. B y the same type of argument as above w e obtain supp 9j [nj, _ , nj, + l. Now ,

=

C

_

�

�

C

.

•

.

•

.

•

=

.

=

map , F (I) = j, gives

jj (w) = 2 - i / 2 i ( ; ) = L Cj,k e - i k w sO{w) = mj (w) sO{w), keZ where mj {w) = L: keZ Cj,k C i /C w . Clearly, mj (w) E L 2 { T ) and Il mj 1 1 2 = L: keZ i cj , kl 2 = 1. Now , i( ; ) = 2i / 2 jj (w) = 2j / 2 mj (w) sO{w), so i(w) 2j f 2 mj (2j w) sO(2j w), j � 1 . B y Holder's inequality (1.6.2(c)), 1/ 2 1/ 2 J2� l i(w) 1 dw ::; 2j f 2 (J2� IsO(2j w) 1 2 dW ) ( J2-t; I mj (2j w) 1 2 dw) ) . =

7. Wavelets

447

2j w in the integrals on the right. This yields f2-t; / f(w) / dw Let u =

1 IImj ll 2 = 1 , f2-t; / f(w) / dw ::; ( I2�+ 1 ". 1 �(u) 1 2 dU ) /2 o. Therefore , f(w) = 0 for almost every w E � 211", 411"]. If the same argu ment is repeated beginning with f:: 2j / 2 I f(2j w) / dw for j E Z, we get f(w) = 0 for almost every w E [2i + 1 11", 2i+ 2 11"] . Therefore , f( w) = 0 for almost every w E (0, 00) . We can also begin the argument with [-411", -211"] and obtain f(w) = 0 for almost every w E (-00, 0 ) . Hence f and f are both O . This proof is due to Hernandez and Weiss 1996 . (b ) We show first that W is invariant under dyadic translations , transformations of the form Tm /2 n , m, n E Z , where for f E L 2 (R) , Tm / 2 nf (t) = f (t - m/2n ). Let l E W and let > 0 be arbitrary. Then there exist jo E Z and h E Vi a such that II I - h ll 2 < By our assumption on the Vi , h E Vi for all j � jo , and it follows from Since

�

{

L

scaling and orthonormality that

h (t) = L aj , k ip ( 2j t - k) keZ

with convergence in the L 2 -norm. Thus

Tm / 2 n h (t) For j �

h (t - m/2n ) = L keZ aj , k ip (2i (t - m/2n ) - k) .

n,

since i n this case

2j - n m - k E Z. Since Tm / 2 nl E W. m/2n a, II Tm /2 nl - Ta f l1 2 l E W, Tm / 2 n l E W;

Tm n h E a E /2

and since Vi and c is arbitrary, we get that Now let R b e an arbitrary real number and let c b e positive. We can find m and n such that is arbitrarily close to so by continuity in the mean 2 .8 . 9 , we can make < c (* ) . By the above for since ( * ) holds for arbitrary { > 0,

Ta l E W.

7. Wavelets

448

(c). By assumption, (j is continuous at 0 and (j (O ) = 1 . Therefore , there exists > 0 such that :/; 0 for E Now suppose = w :/; that cl Ui e Z Then W is a closed , proper sub space of By the projection theorem there exists 9 E such that g :/; O and 9 .1 W. Thus 9 .1 for all l E W. By (b) , however, W is invariant under translation. Hence

(

for all

u

8

V; )

L 2 (R).

(j (w) L 2 (R).

w (-8, 8).

L 2 (R)

I

f�oo I(t + u)g (t) dt = 0 E R and l E W. Now, (T-::f ) (w) = e i u w j(w) , yew) =

g(-w). It follows from Parseval's identity 5 . 1 8 .3(b) that f�oo e ; uw j ew) g( -w) dw = 0 for all E R . But j(w) g(-w) E L 1 ( R) , and by the uniqueness theorem for the L 1 - Fourier transform, j (w ) 9 ( -w) = 0 for a.e . w E R. In particular" take I (t) = 2i ip (2i t) . Then I E V; W and j ew) = (j (w/2i ) . Hence (j (w/2i ) g( -w) = 0 for a.e. w E R. But w/2i E (-8, 8) for sufficiently large j , so (j (w/2i ) :/; O. Thus g( -w) = 0 a.e. for I w l < 2i 8. If we let j we get g(w) = 0 a.e . ; consequently, 9 = 0 as an element of L 2 (R) . This contradiction implies that W = cl Ui e Z V; = L 2 (R). u

C

00 ,

7.6

Shannon Wavelets

In this section we consider the band-limited Shannon wavelets. Example

7.6.1

SHANNON WAVELETS

ip (t)

Instead of beginning with the (time-limited) scaling function as we did with the Haar wavelets (Example 7.3. 1 ) , we begin with the frequency-limited Fourier transform

1[O, 1 ) (t)

?r iw _ sin 1rt . t td 1 e ip (t) -- � 21r -?r Since L: n e Z l (j(w + 2n1r) 1 2 = 1 for a.e . w E R, {ip(t - ) orthonormal family in L ( R) by 7.4 . 1 . To correspond to (j(w) = 1 [- ?r , ?r ) (w), we take2 Vo to be the space Vo = { I E L 2 ( R) : j( w) = 0 for Iw I > 1r } whose inverse is

7rt

n

E Z} is an the fact that

: n

7. Wavelets 00

of band-limited functions. By the Sampling theorem 5.21 . 1 , if

- n)) I(n) sin(7r(t I(t) = " L...J (7r t - n ) - 00

449

I E VO , then

( 1 I 1 I 2 -convergence)

{!pet - n) : n [1 Vi = cl [{ !pi , n = 2j/2 !p(2i t - n) : n E z}] , j E Z .

and the series converges pointwise as well. Thus, E Z } is an orthonormal basis for Vo . We take (with denoting linear span)

Thus,

= { I E L 2 (R) : j(w) = 0 , I w l > 2j 7r } . It is easy to see that { Vi j E Z} is an MRA with scaling function !pet) = (sin 7rt ) /ri. The increasing, separation, scaling, and orthogonal properties are obviously satisfied from the form of each Vi , That cl ( Ui � o Vi ) = L 2 {R) follows from the facts that the map I j is a bijection from L 2 {R) to itself, and Parseval's identity 1 I 1 2 = $ 1 1 /1 1 2 (Plancherel 's theorem 5 . 1 9 .3) , as we now show. Let I E L 2 {R). Then j E L 2 (R), and let t:. (w) = j(w) 1 [ - 2 " 11' , 2 " 11' ] (w). Then In E Vn and l - 1 0 as it:. � 2 n Therefore , II /n - 111 2 0 as n Vi

:

�

�

00 .

�

By the filter equality 7.4.5,

�

1-+

�

00 .

cp{w) = mcp (w/2) cp{w/2) = 1[ _11', .. ) (w) . Generally, mcp E L 2 (T), so m cp is the 27r-periodic extension of 1 [ - 11'/ 2 ,11' / 2 )' By the mother wavelet theorem 7.4.8,

�(w) =

=

e - i (w / 2+ 1I') m cp (w/ 2 + 7r) cp{w/ 2 ) a.e. e - iw /2 (1[ -2 11', - 1I') (w) + 1 ( 1I' , 24w))

is the transform of a mother wavelet. Hence we can take as the mother Shannon wavelet - sin 27r(t - 1 /2) . '!/J{t) = sin 7r(t - 1/2) 7r(t - 1 / 2 )

7.7

0

Riesz Bases and MRAs

!p {t - n E

« Vn) , !p)

If n) , Z , of the scaling is an MRA, then the translates function are an orthonormal basis for Vo . A kindred notion to that of or t h o n o rm a l basis is basis. When we substitute Riesz basis for orthonor mal basis in the definition of MRA, we get a RIESZ MRA Specif ically, we require that Z} be a Riesz basis for There is

Ri esz {h(t - n) : n E

((Vn) , !p): Va .

7. Wavelets

450

(( n) , h)

((Vn) , cp).

a close connection between Riesz MRAs V and MRAs In particular, if - n ) : n E Z} is a Riesz basis for then there exists a function cp E Vo such that t - n ) : n E Z} is an orthonormal basis for Vo (7.7. 8 ) .

{h(t

{cp(

Yo,

Definition 7 . 7 . 1 SCHAUDER BASIS

(xn )

A sequence is a SCHAUDER BASIS for a Banach space ( V, 1 1 1 1 ) if (1 1 1 1 convergence) where every E V can be written as = the scalars are uniquely determined . 0

x

x En EN anXn

an

Clearly, if V has a Schauder basis, then V must be separable : If V is =P 0 for finitely many n } is a count real, then E Q, able dense subset of V . In the complex case, the set E Q + iQ, =P 0 for finitely many n } is a countable dense subset of V .

an

{ E nEN anXn : an

{ E n EN anXn : an

an

Definition 7.7.2 EQUIVALENT SCHAUDER BASES

(x n )

(Yn) are called EQUIVALENT if they have the

Schauder bases and mutual convergence property:

L anXn converges L anYn converges. nEN nEN <=>

0

Equivalence is characterized in 7.7.3. We leave its proof to Exercise 3 . 7.7.3 EQUIVALENT BASES Schauder bases (xn) and (Yn) for V are equiv alent if and only if there exists a bounded linear bijection T : V - V such that TXn = Yn for all n.

We discuss

Riesz bases below i n the context of separable Hilbert spaces.

Definition 7. 7.4 RIESZ BASIS

(X n )

A Schauder basis for the Hilbert space V is a RIESZ BASIS if it is 0 equivalent to an orthonormal basis

( Yn ).

Obviously, any orthonormal basis is a Riesz basis. There are many alter native characterizations of Riesz bases (see Young 1980 ) . The most imp or tant for our purposes is 7.7.5. Others are indicated in the exercises.

If V is a separable Hilbert space, then (X n ) is a Riesz basis if and only if each x E V can be expressed uniquely as x = E n EN anXn and there exist positive constants A and B , the RIESZ CONSTANTS, such that 7.7.5 RIESZ BASIS CHARACTERIZATION

I

l1

2 2 f A L l an l � anxn � B L l an l 2 n= l nEN n EN

for all x E v.

( 7 . 49 )

7. Wavelets

45 1

Proof. Assume that (xn) is a Riesz basis and write x E V uniquely as x = Ln EN anxn · Since (xn) is a Riesz basis, there exists an orthonormal basis ( Yn ) such that the map T defined by T ( Ln EN anXn) = L n EN an Yn is a bounded linear bijection of V onto V. Since I I Tx I I 2 ::; II T I I 2 11 x II 2 and ( Yn ) is an orthonormal basis, it follows that 2 L anYn = I an 1 2 = I I Tx II 2 ::; I IT I1 2 11 x l1 2 . nEN n EN Thus ( 1 / II T II 2 ) Ln EN I an 1 2 ::; II x 11 2 , which proves the left half of (7.49) .

L

The right half follows from a consequence of the bounded inverse theorem (Bachman and Narici 1966, p. 27 1 , Theorem 1 6 .6) , which says that a con tinuous linear bijection between Banach spaces is bicontinuous. In this case it means that T - 1 is bounded or T- 1

(Ln EN anyn) nLEN anXn

::; II T - 1 1 1

L an Yn .

nEN

(Xn) Ln EN anxn · (Yn) = Ln EN an Yn ; L n EN anYn 2 L n EN l an l ::; (I/A) I Ln EN anXn l 1 2 . II Tx II 2 Ln EN l an l 2 ::; (l/A) I ILnEN anXn l 1 2 . = = L n EN an Yn , an = Y = Ln EN an Yn

Conversely, suppose satisfies (7 .49) and that each x may be uniquely Since V is a separable Hilbert space, all or represented as thonormal bases are denumerable (3.4.8 and 3.4.9) . Let be an orthonor T is well-defined because of the mal basis for V, and let Tx uniqueness of the representation. The series converges because by equation (7 .49) , T is clearly lin ear ; it is bounded because = Clearly, T is injective, because if Tx 0 0 for then all n . To see that T is onto, we observe that if E Y, then < 00 . As a consequence of (7.49) , the sequence of finite sums ZN is a Cauchy sequence. Hence, x E V and 1 Tx It follows from (7 .49) that T- 1 is bounded. 0

Ln EN l an l 2 = L;;= anXn = Y = Ln EN an Yn .

= Ln EN anXn

We now extend the notion of MRA by substituting "Riesz basis" for "orthonormal basis" as a property of the scaling function. Because a Riesz basis is equivalent to an orthonormal basis and convergent series of orthonormal vectors converge unconditionally (3.6.2 ) , a Riesz basis i s also unconditional i n that sense . This j ustifies writing sums in material to come with indices belonging to Z, i.e . , as rather than

( Yn)

Ln EZ anXn

Ln EN anxn ·

Definition 7.7.6 RIESZ M RA

A RlEsz MULTIRESOLUTION ANALYSIS (RIESZ M R A ) is a sequence of closed subspaces Vi , j E Z, of (R) with the following properties:

L2

452

7. Wavelets

V- I Vo VI . . . . (b) (Density) cl Uj e Z V; = L 2 ( R) . (c) ( Separation ) nj e Z V; = {O} . (d) (S caling) f (t) E Vi if and only if f ( 2 t ) E V; + I . (e) (Riesz Basis) There exists a RIESZ S CALING FUNCTION h E Vo such that {h (t ) E Z} is a Riesz basis for Vo . (a) (Increasing)

- n

Vn , h

C

...

C

C

C

0

: n

Our first goal (7.7.8) is to show that if we start with a Riesz MRA ( ) , we can find a function cp such that {cp (t - n ) : n Z is an We begin by proving 7.7.7. orthonormal basis for

({Vn ) ,

Vo.

E Vo

E }

7.7.7 Let h) be a Riesz MRA. Let A and B be the Riesz con stants {7.49} of the Riesz basis { h ) E Z } for Let

(

L: n e z

I h (w + 2mr ) r )

1 / 2 , w E R. (tThen

- n

VA �

Vo. gh (w)

: n

gh (w) � VB a. e.

gh E L 2 (T) . P roof. Note that h (w + 2 mr ) is the Fourier transform of e - 2 mr t h (t) . The series by which gh is defined converges by Parseval's identity ( 5 . 1 8 .3) and (7.49) . Clearly, gh is 2 71"-periodic. To see that gh E L 2 (T) , consider [10 2 " 1 9h (w)1 2 dw = 1o 2,. L I h (w + 2 ) 1 2 dw ke Z 2" 2 L k e Z 10[ 1 h (w + 2k71") 1 dw. If we change variables in the preceding equation and replace w + 2k7l" by w , we obtain 1 2,. 1 9h (W ) 1 2 dw = Lk e Z 12h( 2k + 2),. I �h (w) 1 2 dw 1: Ih (w) 1 2 dw and

i

k71"

O

I h l : = 271" Il h ll � .

by Parseval's equality 5 . 18.3(a) . Thus

gh E L2 (T) .

7. Wavelets

453

In the notation of equation (7. 15) of Section 7.3, we write

(a k h e z

Let 5 . 1 9 .3 ,

ho,dt) = h (t - k) , k E Z. E £2 ( Z) and consider L keZ a k ho , k . By the Plancherel theorem

We write the above as

2

�

r ( 2n +2) 1r L a k e - ik W h (w) dw. = L 27r nE Z i2 n1r kEZ Replacing w by w + 2n7r in the previous integrals, we obtain 2

2

2 n ik ( ) L ak e - w +21r h (w + 27rn) dw kEZ 2

] [ (w) = L keZ a k e - ikw and gh (w) = LnE Z I h (w + 27rn) I 2 2 1 f 2 1r 2 (7.50) = 27r io lu (w) 1 g� (w) dw.

1/2

Then , with u

By hypothesis ,

2

2

A L l anl 2 :::; L an h ( t - n) nEZ nEZ 2

n EZ

'

454

7. Wavelets

Also, 2'11'" L ne Z Thus

l an l 2 = II Ln eZ an e - i nw l l � , where Il l b is the L 2 (T)-norm. 2

A lI u ll � S; 2'11'" L an h o , n n eZ

Using equation

2

(7.50), we obtain

or

(7.51 )

(7.51)

where the functions and norms in equation are L 2 (T) functions and norms. Choose n E Since {a kh eZ E £2 ( Z) is arbitrary, let be fixed and choose the scalars ak such that

N.

y

By some trigonometry,

(

1

2'11'" ( n +

)1)

1 Fn { y - w),

sin 2

{

1) (W - y) / 2 (w - y) / 2

+ 2 sin n

(7.52)

2'11'"

Fn(w) is the nth Fejer kernel (Section 4.15). Inserting this into equa (7.51), using the fact that Fn(w) is 2'11'"- periodic as well f�oo Fn(x) dx = (4.15.2), we obtain 1 j 1r Fn (y - w) g� {w) dx S; B. A S; 2

where tion 2'11'"

as

'II'"

Hence

- 1r

(Fn * gO (w) S; B. Now , g� E Ll (T) ; hence as in equation (5.8) of Section 5.3 and Lebesgue's pointwise convergence theorem 5.3.1, I f w e let

or

A S;

n

---+ 00 i n the previous inequality, w e obtain

A S; g� (w) S; B a.e . ,

VA S; gh

(w) S; Vii a.e.

0

7. Wavelets

455

(t - n) : n Z}

If {h E is an orthonormal basis for Va ,it follows from the Pythagorean theorem that

2 Thus, i n this case

g�

2 n EZ

A = B = 1 and

(w) = L I h (w + 2mr) 1 2 n EZ

=

1

a.e. (cf. 7.4 . 1 ) .

7.7.8 RIESZ BASIS T O ORTHONORMAL If { h (t - n) : n E Z} is a Riesz basis for the closed subspace Va of L 2 (R), then there exists I{J E Vo such that {I{J (t - n) : n E Z} is an orthonormal basis for Va . P roof. By 7.7.7,

� l /gh (w) � l/VA a.e., where 1/2 2 gh (w) = I h (w + 2mr) n EZ Since l/gh ( ) is bounded a.e., ip = high E L 2 (R), and therefore E L 2 (R). It is also clear ithat there exist (an) and (bn) in £ 2 ( Z) such that l / gh (w) = Ln EZ an e - nw and gh (w) = Ln EZ bn e - i nw ( 9 h (w ) E L 2 (T) ) . Thus an e - inw and h ew) = ip (w) bn e - i nw . ip (w) = h ew) n EZ n EZ 0 < l /VB

(L

1)

I{J

w

Hence

l{J(t) =

L Ln EZ anh(t - n)

L

and

which shows that I{J E Vo and cl [l{J(t Finally, we note that

h(t) =

Ln eZ bnl{J(t - n),

n) : n E Z] = Va .

2 L l ip(w + 2n1r) 1 2 = g� �w) L I h (w + 2n1r 1 1 a.e . , n eZ nEZ which shows that the family {1{J(t - n)} n EZ is an orthonormal family by 7 .4 . 1 . 0 =

The proof of 7.7.8 follows Hernandez and Weiss 1996 .

7. Wavelets

456

Exercises 7 . 7

(xn)

1 . COEFFICIENT FUN CTIONALS CONTINUOUS Let b e a Schauder basis for the Banach space V. Any E V can be written uniquely as = Show that the linear coefficient functionals = are continuous and in fact there exists M > 0 = such that 1 � � M for all n E

x

x E n e N anxn· In (x) ( a Xn) an in En e N n N. I I xn ll ll /nl l 2. Let (xn) and (Yn) be Schauder bases for the Banach space V. Show that the maps Tn ( E n e N a i x i ) = E �= 1 a i Yi , E N , are linear and n

continuous.

3 . EQUIVALENT SCHAUDER BASES Verify the assertion of 7.7.3 that Schauder bases and for V are equivalent if and only if there for exists a bounded linear bijection : V -+ V such that = all n .

(xn)

( Yn )

4. RELATIO NS BETWEEN (R) .

L2

T

TXn Yn

.e2 (Z) ELEMENTS AND h E L 2 (R)

Let h E

E keZ I h ( w + 2br) 1 2 converges almost everywhere. Let (an) E .e2 ( Z), and let h k (t) = h (t - k) for k E Z. Show that

(a) Show that (b)

o,

5 . INEQUALITY O N Uh EXTENDS TO Uh ( w ) =

.e2 ( Z)

(keZL Ih

(w

For h E

+ 2 k1 r) r

)

L 2 (R) and 1 /2

as in 7.7.7, choose A , B > 0 such that v'if � U h ( w ) � Vii a.e . Show that for it follows that E

(an) neZ .e2 ( Z),

2 2 A L l a kl � L a k h (t - k) n eZ keZ 2

6 . Show that the result in Exercise 5 generalizes the part of 7.4 . 1 that

L2

L keZ I h (w + 2 k1 r) 1 2

states that for h E (R) , if { h (t n ) : n E is orthonormal. -

Z}

= 1 a.e . , then

7. Wavelets

457

Hints 1. Norm the vector space W = ing

{(an) : 2: nEN anXn converges } by tak n sup L a i X i . lI (an) 1I = nEN i= 1 It can be easily verified that this is a norm and that W is a Banach space. Define T W V as T « a n )) = 2: n EN anxn. Clearly T is a linear bijection. By the continuity of the norm on V, T is also :

continuous:

�

n aX L II T « an)) 1I = L anXn :::; nsup EN i =1 i i nEN

Since a continuous linear bijection between Banach spaces has a con tinuous linear inverse (Bachman and Narici 1966 , p. 27 1 , Theorem 1 6 .6 ; this is sometimes called the open mapping theorem) , is continuous. With =

x 2:n EN anxn, I fn (x) 1 = l an l anxn ll = Il II Xn ll 2:7=1 a i X i 2:7;;11 a i xi l 1 = II Xn ll

T- 1

-

But

11 2:�= 1 a , x , - 2:�:11 a,x, II II x n ll

I fn (x) 1 :::; 2 ii T- 1 i i ll x l I / II xn ll · Thus, each fn is bounded, and II fn ll :::; 2 ii T - 1 i / lI xn ll . With M = 2 i i T- 1 ii , we have II xn ll ll fn ll :::; M, E N . Finally, 1 = fn(xn) :::; II x n ll ll fn ll :::; M, E N. n

n

458

7. Wavelets

Tn

2. Note that is a linear combination of coefficient functionals, each of which is continuous by Exercise l .

(xn) and ( Yn) are Schauder bases for the Banach space V T V --+ V is a bounded linear map such that TXn = E N . Assume that L n eN anXn converges and consider Yn L n eN an Yn · For m > n , IIL�= 1 a kYk - L � = 1 a kYk ll = IIL�=1 ak Tx k - L �= 1 a k Tx k ll l i T ( L�= 1 a k X k - L � = 1 ak x k ) 1I < II T II IIL�=1 a k X k - L � = 1 a k x kll · Since L n eN anXn converges, IIL �= 1 a k X k - L � = 1 a k x kll can be made arbitrarily small for sufficiently large n , and the result follows. Since T- 1 is bounded (open mapping theorem) , it follows similarly that if Ln eN anYn converges, then Ln eN anXn converges. Conversely, suppose that (xn) and ( Yn) are equivalent bases. Write x E V uniquely as x = Ln e N anxn· By hypothesis Y = L n eN an Yn converges, and we define T V V by taking T ( L n eN anXn) = L n eN an Yn · It is easy to verify that T is a well-defined linear bijec tion. To show that T is bounded, let Tn (L n e N anXn) = L �= 1 a i Yi be the bounded linear map of Exercise 2. Clearly, Tx = limn Tnx, and by the Banach-Steinhaus theorem (quoted in Chapter 4 as 4.9 . 1 , or Bachman and Narici 1 966, p . 25 1) it follows that T is bounded. 2 1 / 2 E L (R) by (a) Follows since ( w ) = (L keZ I h ( w + 2 h ) 1 ) 2 7.7.7; only the fact that h E L 2 (R) was used in the proof. (b) This was also proved in 7.7.7, assuming only that h E L 2 (R) .

3 . Suppose and that for all n

:

:

4.

--+

gh

5 . Use 4(b) and the hypothesis to obtain

2

2 < - 1 11" B

- 2 71"

6.

0

2

dt.

2 2 k i But J:1I" IL keZ a k e - t l dt = 271" L keZ l a k l 2 . If L keZ I h (w + 2 h ) 1 2 = 1 a.e . , then (w ) = L keZ I h (w + 2k7l") 1 2 = 1 a.e . Thus, with A = B = 1 in Exercise 5 , 2 gh

(a k h eZ E £2 ( Z ) .

2

for all This is equivalent to the family n being orthonormal .

E Z}

{h (t - n ) :

7. Wavelets

7.8

459

Franklin Wavelets

An example of a Riesz basis for the space Vo of piecewise linear continuous - n) : n E functions on the intervals [k , k + 1] , k E is given by = (1 where is the hat function (see Exercise 3) . We - 1 1) provide an elementary direct method of obtaining an orthonormal basis for Vo from the n : n E independent of the general metho d of conversion for Riesz bases of 7.7.8 (cf. Exercise 3).

h(t)

- It {h(t

-

-0.5

-,

Z,

l[ o, 2j (t) ) Z},

The Hat

Function

h

{h(t

Z}

2.5

(t)

=

(1 - I t - 1 1 ) 1 [0, 2]

A fact we shall need is the following. 7.S . 1

1

1

sin

2 "TrW = L n E Z (7rW + 7rn ) 2

a. e.

Proof. We know that the Fourier transform of the Haar scaling function (see Example 5 . 1 5 .5) , and that I [O,lj (t) is tP(w ) = n) : n E is orthonormal. Thus, by 7.4. 1 ,

g(t ) = {cp(t

e - iw /2 Si:ir

Z}

-

Now replace

W by 27rw .

4 sm· 2 "2W ", n ( +2n ". ) 2 a.e . L..." EZ w 1

=

0

Example 7.S.2 FRANKLIN WAVELETS

Let Vo be the set of piecewise linear continuous functions on the intervals [k , k + 1] for all k E Z, which are in L 2 (R). Any f E Vo may be written as

(see Exercise 3)

L a k h(t k) , kEZ h(t) = ( 1 - I t - 1 1 ) l[o, 2j (t) f(t) =

where

-

460

7. Wavelets

is the hat function. The collection Vj = {f E L 2 (R) : f 2 - i t E is an increasing ( Vj C Vj + l ) family of closed subspaces of L 2 (R) . Since {h t - n ) : n E Z } is clearly not orthogonal, h is not a scaling function. By Example 5.5.5 and 5.5. 1(b) , the Fourier transform of h is

) Vo l

(

it h (w) = e-iw ( si: 2

)

(

2 (7.53)

[Or observe that h( t ) = l[O, l](t) * l[O, l ](t) (see Exercise 1 ) .] Since {h(t - n ) : n E Z } spans Vo , any cp E Vo can be written

cp (t) =

In

L an h ( t

ne Z

)

- n .

the remainder of the argument, we obtain a way to choose the a k so that

{ cp ( t - n ) : n E Z } is an orthonormal base for Vo. By the continuity of the L 2 -Fourier transform map F (I) = i (5. 19 . 3) , (7.54) �(w) = a n e-iw n h (w) = pcp (w)h(w) n eZ wn where Pcp (w) = L neZ a n e -i . By 7.4. 1 , in order for {cp ( t - n ) : n E Z }

L

to be orthonormal, we must have

L I�(w + 2mr) 1 2 = 1 a.e.

ne Z Thus we want 1

= =

o r (note that

Since

h (w)

=

LneZ I�(w + 2 mr) 1 2 2 I pcp (w) 1 2 LneZ I h (w + 2mr) 1 a.e. ,

L neZ I h (w + 2 mr) 1 must be nonzero a.e .) 1 Ipcp (w) 1 2 = 2 a.e . 2mr) (w + 1 LneZ I h

(7.55)

2

e -iw

(Si:iF) 2 by equation (7.53) , '" sm[( w +2n7r){21 1 4 L.... n e Z I (w +2n ". ) / 2 w

. sm 4 2

Lne Z (w/2 + mr) 4 · 1

(7.56)

(7.57)

7 . Wavelets

Thus

46 1

(7. 5 8)

By 7.8 .1, Differentiating twice, we obtain (7.59) 2'Tr2 (sin2 'TrWsin+4 3'TrWcos2 'Trw ) = L ('TrW 6'Tr+ 2'Trn )4 . nEZ Replacing 'TrW by w/2 in equation (7.59) and using some basic trigonometry, 1 - G) sin 2 W /2 " 1 (7.60) � sin4 w/2 -- n�E Z (w/2 + n'Tr)4 a.e . By equation (7. 58), we see that we want -

- .,---'--

We choose

1 . PIP(w) = ( 1 - ( 3'2 ) sin2 w/2) - / 2 eSw • By equation (7.54), 2 . 2 w/2) - 1 / 2 [ sin2 W/22 ] (7.61) � (w) PIP (w)h� (W ) ( 1 - ( 3' ) sm (w/2) ' In terms of the filter mIP(w), by the filter equality 7. 4 . 5 , I{J

so

=

=

sm. 22w ( 1 - -sm 2 ' 2 w) - 1 / 2 -w 3 2 sin (w/2)2 ( 1 - -sm 2 . 2 w / 2) -1/ 2 3 (w/2)2 sin2 w (1 - (2/3)Sin2 2w/2) 1 / 2 4sin (w/2) 1 - (2/3)sin w 1/2 2 1 (2/3) sin w /2 cos2 ( �2 ) ( 1 - (2/3) sin2 w )

(7. 62)

462

7. Wavelets

By the mother wavelet theorem 7.4.8, the Fourier transform of the FRANKLIN MOTHER WAVELET is

which may be computed by substituting for rp and mcp from equations ( 7.61 ) and ( 7.62) into this equation. We determine 'IjJ by inversion and numerical techniques. Alternately, rewrite equation ( 7.61 ) as

By equation

( 7.53)

and the translation property, it follows that

cp ( t) =

l: bn h (t + n + 1 ) .

n EZ

We determined the coefficients bn b y expanding

( 1 - 2 . 2 w ) - 1 / 2 = V(3"2 ( 3" sm "2

)

W - 1/2

1 + cos "2

in powers of cos w/2 = (eiw/ 2 + e - iw/ 2 ) /2. We mention that the Fourier coefficients bn are bounded by e - a1 n l ( bn = O ( e - a1 n l )) for some a > o (Hernandez and Weiss 1996, Walter 1997) . 0

From the nature of the hat function h ( t ) = ( 1 - I t - 1 J) 1[o, 2] ( t ) used in Example 7.8.2, the Franklin wavelets should produce effective approxi mations of functions with slowly changing derivatives (described nicely by straight line approximations) . A large number of basic properties of

Exercises 7.8

1.

Verify that

-

l[O, l ] (t) * l[ O , l] ( t) = ( 1 I t - I J )1[o, 2]( t) = h ( t ) .

Use this to give an alternative proof that the Fourier transform of the hat function

h(t) = (I - I t - I J)1[o, 2]( t) is h(w ) = e - iw

(si:iP r .

2 . Give an alternate proof of 7.8 . 1 by expanding e - iw t in a Fourier series, and then using the Pythagorean theorem for Hilbert spaces (3.3.4(g)) .

7. Wavelets

3 . TRANSLATES O F HAT FUNCTION FORM RIESZ BASIS Let - I I)1[ o , 2] (t) be the hat function of Example 7 . 8 .2.

(I - I t

(a) Show that for any scalars such that

463

h(t)

=

f E Va there is a unique sequence (an) n E Z of f(t) =

L a nh(t - n).

n EZ

Moreover,

L I f(k) 1 2 � II f ll ; � L I f(k) 1 2 . � kEZ kEZ

{h( n) : n E Z} � (w) (w) (w).

( n) : n Z}

is a Riesz basis by a) , consider the (b) Since tassociated orthonormal basis {cp(t E of 7.7.8 with =h Compare this with the � of Example 7.8.2. /gh

Hints 2.

f( t ) = e - iw t = Ln EZ i(n)e in t , where sin 7r(w + n) 1 i in f�( n ) -- 271' 1 " - w t e - t dt -- 71' (W + n ) , n E Z . e

_"

Thus

+ n) e int , e - iwt - L sin7r (7r(w w + n) n EZ _

Divide

-71' < t < 71'.

both sides b y $ t o normalize, and use the Pythagorean theorem to get 1

_ '" sin 2 7r(w + n) - L..J

-

2 7rw nL..J EZ 7r2 (w + n) 2 ' '"

sin

n EZ 7r2 (w + n) 2 3 . (a) . I t is easy to verify that any f E Va has the form f( t ) = L f(k)h(t - k + 1 ). kEZ To verify the inequalities , note that since f is linear o n [k, k + 1 ] , r k + 1 I f(y) 1 2 dy = r1 ((1 t) f(k) + tf(k + 1)) 2 dt .

Jk

Jo

-

_

464

7. Wavelets

Thus

lkk + 1 1f( Y) 1 2

d

Y - f(k) _

It is also easy to see that

f(k) 2 + f(k + 1 ) 2

�

f( k) 2 + f(k + 1) 2

+

6

and

3

2 + f(k + 1 ) 2 3

+

f(k)f(k + 1) .

f(k) 2 + f(k + 1 ) 2 3

f(k)f(k + 1) 3

�

3

+

( 7 63 ) .

f(k)f(k + 1) 3

f(k) 2 + f( k + 1) 2 . 2

It follows from (7.63) and these inequalities that �

II f ll� � L I f(k) 1 2 . kEZ This, incidentally, also shows that (f( n)) E f2 ( Z ).

� kLEZ I f(k) 1 2

Frames

7.9

Every separable Hilbert space has a countable orthonormal basis. In L 2 (R), for example , there is the sequence

Umn - e 27rim( t- n ) 1 [n,n + l ) , m , n E Z . Not only can every vector x be written as an infinite series of its orthogo nal projections on the U mn , x L m, n EZ (x, umn) U mn ' but the coefficients (x, Umn) are unique; that is where the orthogonality is used. Riesz bases are -

=

more general, but we still demand uniqueness of the representation. Is the uniqueness that imp ortant? By sacrificing uniqueness, we gain more flexibil ity in the basis vectors , and this is what FRAMES are about . It follows from Parseval's identity 3 .4.2 that an orthonormal sequence in a Hilbert 2 1 = for space X is an orthonormal basis if and only if every E X . Frames provide a variation on this theine ; instead of equality, it is required only that there be constants 0 < � such that

(x n) LnEN I (x, Xn) II x ll 2 (xn) A B A II x II 2 � L I (x, Xn) 1 2 � B II x I1 2 . nEN

x

Nevertheless, we retain the crucial representation property (7.9.9(c) ) , that every vector may be written

x

x = I:: ( x, S- l Xn) Xn, n EN

7. Wavelets

465

where S is the FRAME OPERATOR defined below . Essentially, S is defined as the sum of the projections of a vector v on the

Xn :

Sv =

L ( xn) xn· n eN v,

We show in 7 . 9 . 1 2 that a certain kind of frame (an exact frame) is a Riesz basis. We do not restrict consideration to (R) , the so-called finite-energy < 00 ) , one-dimensional signals. We remain in the gen eral separable Hilbert space environment except for the Weyl-Heisenberg frames of 7.9 . 13 .

L2

( f�oo I f (t) 1 2 dt

Definition 7.9.1 FRAMES

(xn )

A sequence of vectors in a Hilbert space V is a FRAME if there exist constants A, B > 0, called the FRAME BOUNDS , such that for any E V,

A

A II x I1 2 � L I (x, Xn }1 2 � B II x II 2 nEN

x

(frame inequality ) .

(7 . 64)

Xn

If = B , then the frame is called a TIGHT FRAME. If the removal of one renders the collection no longer a frame , then it is called an EXACT FRAME. D

An orthonormal basis (xn) is a tight, exact frame by Parse val's identity (l:n eN l (x, xn} 1 2 = II x 1I 2 , (3.3.4(f)) . We show in 7.9 . 12 that a sequence (xn) of a separable Hilbert space is a Riesz basis if and only if it is an

exact frame-in particular, therefore , any Riesz basis is a frame. Without exactness, a frame is not generally a Riesz basis, since the elements of a frame need not be linearly independent (Example 7.9.4 and Exercise 7.9- 1) . By analogy with the nomenclature for orthogonal bases in Hilbert spaces, we adopt the following convention. Definition 7.9.2 COMPLETE SEQUENCES-DENSE LINEAR SPAN

(xn)

A sequence of vectors-any sequence, not necessarily a frame-in a Hilbert space V is COMPLETE if cl V , where denotes linear span. D

[{ X n}] =

[]

We omit the easy proof of the following analogue of 3 .4.2:

{Xn} .L = {O} A sequence (Xn) of vectors in a Hilbert only if {X n} .L = {O}. The left half of the frame inequality, A II x II 2 ::; l: n e N I ( x, Xn} 1 2 , implies that if (X n ) is a frame , then {x n } .L = {O}, i.e . , that frames are complete . 7.9.3 COMPLETE {::} space V is complete if and

466

7. Wavelets

Therefore , the Hilbert space V in which it resides must be separable as was the case for Riesz bases. In Examples 7.9.4-7. 9.6, is an orthonormal base of a Hilbert space

(x n )

V.

,

,

Example 7.9.4 TI G HT NOT E X A C T NOT LINEARLY INDEPENDENT

If the orthonormal basis . . is a tight frame:

Xn, Yn,

.

{ Yn } is disjoint from {x n } , then X l , YI , X 2 , Y2 , . . . ,

L l (x, xn ) 1 2 + L I (X, Yn) 1 2 = 2 11 x II 2 , x E V.

n EN

nEN

It is not exact. Removing one vector will change the frame bounds by converting the equality above into a strict inequality as a consequence of Bessel's inequality 3.3 . 1 . But it is still a frame. If we remove say, then

Xl ,

0

Its elements are clearly not linearly independent. Example 7. 9 . 5 EXACT , NOT TIGHT

The sequence

2X I , X 2 , . . . , Xn, . . . is an exact frame . It is not tight because I ( X I , 2X I ) 1 2 + nL I (X I , Xn) 1 2 = 2 II X lll 2 , ;?: 2

but for all k :l 1 ,

I (X k , 2X I ) 1 2 + nL I (x k , X n) 1 2 = II X k ll 2 . ;?: 2

0

Example 7.9.6 NOT A FRAME

The sequence

X l , x 2 /2, X3/3, . . . , Xn/ n, .

.

. is not a frame since for all

Definition 7.9.7 POSITIVE OPERATORS

k,

A bounded linear map T V V on a Hilbert space V is POSITIVE if (Tx, X) 2: ° for all X E V . We indicate this by writing T 2: 0. If 5, T V V are self-adjoint bounded linear maps such that 5 T 2: 0, we write T ::; 5 or 5 2:. T; this means that (Tx, x) ::; ( 5x, x ) for all X E V. If T is positive, then (Tx, x) = (Tx, x) = (x, Tx). Moreover, by po larization [Bachman and Narici, 1966, p . 368] (Tx, y) = (x, Ty) for all :

-+

-

:

0

-+

7. Wavelets

467

E Vj in other words, T is self-adjoint (i.e . , T = T* ) . For an orthonor mal basis any vector can be reconstructed from its projections as = = We consider sums for frames in 7.9.8.

x, y (xn) x LnEN (x, xn) xn ·

x

Sx LnEN (x, xn) Xn

7.9.8 T H E FRAME OPERATOR For an arbitrary sequence of vectors (xn) in the Hilbert space V, the following are equivalent: (a) (x n) is a frame with frame bounds A and B; (b) The FRAME OPERATOR Sx = L nEN ( x, Xn) xn (x E V) is a bounded positive operator with AI ::; S ::; BI where I V -+ V is the identity map x x for all x E V.

1-+

:

P roof. ( a ) ::} (b) . Let be a frame with frame bounds and B . We first show that the infinite series converges for all E V. With = and m � n ,

(x n ) A x Ln EN (x, xn) Xn Sn LJ= l (x, Xj) Xj II sm - sn ll 2 SUP l y l = l l (sm - sn , y) 1 2 2 SUP l y ll = l I LJ=n + l (x, Xj ) (Xj , y) 1 < SUP l y l = l (LJ::n + l l (x, Xj ) 1 2 ) (LJ::n + l l (xj , Y) 1 2 )

by the Holder inequality 1.6.2(a) . This expression is

::; SUP l y l I =l ( LJ=n + l l (x, Xj) 1 2 ) B lI yII 2 = B LJ= n + l l (x, Xj ) 1 2 0 as m , n since L n EN I (x, xn ) 1 converges by the frame inequality. As a Cauchy se quence in a Hilb ert space, (sn) therefore converges to a vector Sx of V. The linearity of S is clear . To see that S is bounded, consider II Sx II 2 SUP l yl l = l I (Sx, y) 1 2 SUP l yl l ::l ILn EN (x, X n) (Xn , y) 1 2 < SUP l yl l = l (L nEN I (x, Xn) 1 2 ) (L nEN I (xn, Y) 1 2 ) < SUP l y l =l B II x II 2 B lI y l 1 2 B 2 11 x 1I 2 . Consequently, S is bounded, and II S II ::; B. By the continuity of the inner -+

product ,

(

)

-+

00 ,

( SX, x) = L (x, xn) Xn, x = L I (x, Xn) 1 2 � 0 n EN nEN for every x E V, s o S � 0, and w e can rewrite the frame inequality (7.64) as A (x, x) ::; (Sx, x) = L I (x, Xn } 1 2 ::; B (x, x) for all x E V, nEN

468

7. Wavelets

or

AI � S � BI. (b) ( a ) If AI � S � BI, then for all x E V, (Alx, x) = A (x, x) � (Sx, x) = L I (x, xn) 1 2 � (Blx, x) = B (x, x) . nEN A consequence of AI � S � Bl for tight frames, A = B, is that (Sx, x ) = A ll x ll 2 • Interestingly enough , every frame produces a dual frame, namely (S - 1 x n ). �

0

A most imp ortant consequence of this is that every vector c an b e written in 7.9.9(c) .

as

7.9.9 DUAL FRAME REPRESENTATIO N Let S be the frame operator of the frame (xn) with frame bounds A and B in the Hilbert space V. Then: (a) S - 1 exists and is positive on V, and B - l 1 � S - 1 � A - I I; (b) (S - lx n ) is a frame called the DUAL FRAME with frame bounds l iB and 11A; (c) A ny E V can be written in terms of the dual frame x

x = L ( x, S - l xn) Xn = L (x, xn) S- l xn . n EN nEN P roof. (a) Since AI � S � BI by 7.9.8, 1- (II B) S � 0 and (II B) S � (AI B) I. Thus, by Exercise 5 ,

1-

1-

11 1- (II B) SII � 11 1- (AI B) 1 1 1 = 1 - � < l . By Exercise 3 , [(lIB) S] - 1 exists on all of V and is a bounded operator. The same is therefore true of S- I . As a positive operator, S is self-adjoint. By (7.9.8 ) , for any x E V , therefore ( S- l x, x) = (S- l x, S (S- I ) x) = ( S (S - l x) , S- l x) � A II S - l x 11 2 � o . Thus, S - 1 i s positive, too. Generally, i f positive operators T and U com mute , then TU � 0 (Bachman and Narici 1966 , p . 4 1 5) . Since S com mutes with S- 1 and I and AI � S ¢} O � S - AI, it follows that o � (S - AI) S - 1 0 � 1 - AS - I , or S- 1 � A - II. Similarly, since S � BI, it follows that B - 1 I � S- 1 � A - I I. (b) Since S - 1 is positive, it is self-adjoint, so LnEN ( x, S- l xn) S - l xn = (since S- 1 is bounded) ¢}

7. Wavelets

469

Thus 8- 1 is bounded and linear , and � � 7.9.8, (8- 1 is a frame with frame bounds 1 / B and 1/ (c)

B - 1 I 8- 1 A - 1 I. Therefore , by A.

Xn)

x

8 (S - 1 X) 8 (L nEN (x , 8 - 1 xn) 8 - 1 xn ) (shown in (b) ) L n EN ( x, 8- 1 xn) Xn b y continuity of 8.

We get the other equality in part (c) by considering

x = S- 1 (Sx) .

0

Removing one vector from an orthonormal basis vitiates it-it is no longer complete. (The removed vector is orthogonal to what is left but is not 0.) What happens if a vector is removed from a frame? Sometimes nothing: We might still have a frame, as we show in 7.9 . 10 . To prove this, we first establish equation (7.65 ) . To that end let be the frame operator of the frame let E V, and let

S

(xn ), x

By 7 . 9 . 9( c) , we can write

LnEN ( x, S- 1 Xn ) X n Ln EN anXn . Suppose that there exist scalars bn such that x = Ln EN bnxn . If so, we x

contend that

(7.65) The proof depends on being able to write 8- 1 is self-adjoint,

( x, S- 1 x ) in three ways. Since

Thus, by the continuity of the inner product ,

( x , 8 - 1 x) However,

470

7. Wavelets

Lne N anbn since S- 1 is positive. Now , Ln eN l an l 2 + Ln eN I an - bn l 2

which is the same as

N ow we can get to the theorem. 7.9.10 FRAME MINUS ONE Let S be the frame operator of the frame (x n ) in the Hilbert space V with frame bounds A and B. The removal of one vector X m from (xn) leaves either a frame or an incomplete set as follows: If '# 1 , then (Xn) n ;e m is a frame, 1 , then (xn)n;e m is incomplete, and (Xm, S- I Xm ) is therefore not a frame.

{

=

X m = L (X m , S- I Xn) Xn L an (x m ) X n · neN neN However, in terms of the Kronecker delta om, n , X m = L n eN om, n xn. Hence =

by equation (7.65) ,

L IOm ,n l 2 = L I an (x m ) 1 2 L I an (x m ) - Om ,n l 2 , +

neN

ne N

n eN

which reduces to

n;em n ;em Consider two cases: a m (X m ) = 1 and a m (x m ) '# 1 . (1) If am (x m ) 1 , then equation (7.66) implies that L n ;e m I an (X m ) 1 2 O . This means that an (X m ) (S - I X m , Xn) 0 for n '# m or that S- l x m Xn for n '# Since am (x m ) 1 , S- l xm '# 0, so (xn)n ;em is incomplete. Therefore, (xn)n;e m is not a frame. (2) If a m (x m ) (X m , S- I X m ) '# 1 , then since X m L n eN an (X m ) X n , we get (1 - a m (x m )) X m = L a n (X m ) xn, n;e m =

=

=

m.

=

=

..L

=

=

7. Wavelets

47 1

or

Therefore , for any

x E V,

where C = I l - am1(Xm W Thus

L:n #m I an (xm ) 1 2 , by the Holder inequality 1 .6 .2(b) .

Since the frame bounds are A and B ,

while

n #m nN Therefore , (x n)n #m is a frame with frame bounds AI ( 1 + C ) and B . If a frame (xn) is exact, then as a consequence of 7.9. 10, a m (x m ) = (S- l x m , xm) = 1 for all m E N. And l (S- l X m , X m ) = 1 (see (1) in the proof) implies that an (X m ) = (X m , S - Xn) = 0 for n 1= We therefore E

0

m.

have the folowing result.

7.9. 1 1 EXACT FRAMES AND BIORTHOGONAL SEQUENCES If (xn) is an exact frame, then (x n) and the dual frame (S - l xn) are biorthogonal se quences of vectors: (x m , S - l Xn) = Om,n ( m , n E N) .

We can now prove the equivalence of Riesz bases and exact frames in separable Hilbert spaces. 7.9.12 EXACT FRAMES RIESZ BASES IN SEPARABLE SPACES A se quence (x n) in the separable Hilbert space V is a Riesz basis if and only if it is an exact frame. =

Proof. If (x n ) is a Riesz basis, then it is equivalent to an orthonormal basis (Yn). Therefore , by 7.7.3, there exists a bounded linear bijection T V --+ V :

7. Wavelets

472

such that

TYn = Xn for all n E N . Hence Ln EN I (x, TYn ) 1 2 L nEN I (T*x, Yn) l 2 II T*x II 2 (by Parseval's identity ? ? 3 .4.2(c) ) .

(7 67) .

so

Thus, by equation (7.67) ,

(x n )

II Z Il' 5 E I ( z, z.) I' 5 1I 1" II' lI z ll ' , l I (T* ) - I ' n EN

so is a frame . As a Riesz basis, it is a Schauder basis; hence if we remove any we get an incomplete set, which cannot be a from frame. Now suppose that is an exact frame with frame bounds A and B . We show that is a Schauder basis. By 7.9.9( c) , we can write any E V as then by 7.9. 1 1 If we also have = = (here is where w e use exactness) ,

Xm

(Xn) (x n ) (x n ) l x LnEN ( x, S - x n) Xn.

x LnEN anxn,

x

( x, S- l X m ) = L an (xn ' S- l Xm ) = am , nEN so the representation is unique; therefore , (xn) is a Schauder basis. Since (S- l X n ) is a frame (7.9.9) with frame bounds 1/ B and I/A, � II x 1I 2 ::; nLEN I ( x, S- l xn) 1 2 ::; � II x I1 2 ,

or

where basis.

x = L nEN ( x, S- l x n) xn · It follows from 7.7.5 that (Xn) is a Riesz 0

Some interesting general techniques for generating frames can be found in Aldroubi 1995 .

473

7. Wavelets

We briefly introduce the notion of a WEYL-HEISENBERG frame , one that can be generated by translation and modulation of one function; we give a method for obtaining such frames in 7.9.14. Other methods and specific examples are given in Heil and Walnut 1989, Young 1980, and Auslander et al. 1990. Recall the unitary operators of Exercise 7.1- 1 : For I E L 2 (R), TRANSLATION : Ta l (t) = I (t - a) , a E R. MODULATION : Ea l (t) = e 2 1r i a t I (t) , a E R. DILATION : Da l (t) = l a l - 1/ 2 I (tla) , a =p 0,

a

E

R.

Definition 7.9.13 WEYL-HEISENBERG FRAMES

Given I E L 2 (R) and a, b > 0, we say that (I, a, b) GENERATES A WEYL HEISENBERG , or W-H , FRAME for L 2 (R) if {Emb Tna l : m , n E Z} is a frame for L 2 (R) ; Clearly,

0

I is called the mother wavelet.

Emb Tn a l (t) = e 2 1ri mb t I (t - na)

while, reversing the operators,

Tn a Em d (t) = e 2 1ri m b ( t - na ) I ( t - na) .

Consequently,{Em b Tna l : m , n E Z} is a frame if and only if {Tna Em d m , n E Z} is a frame. A key fact about W-H frames is the following. 7.9.14 GENERATIN G W-H FRAMES Given I E suppose that (a) there are real constants A and B such that °

<

A � L I I (t - na) 1 2 � B

a. e.

L 2 (R)

tE

and

a,

b

>

:

0,

R;

nEZ

(b) I has compact support with supp I C I , where I is a n interval 01 length lib. Then (I, a , b) generates a W-H Irame lor L 2 (R) with Irame bounds b - 1 A and b - 1 B .

Proof. Let Ea denote the function Ea (t) = e 2 1r i a t ; thus, Em b (t) = e 2 1r i mbt . Let 9 E L 2 (R) , and consider the function g Tn a f (n E Z ) . Clearly,

n l)

supp ( g T a

C

In

=

I + na = {t + na : t E I} ,

2 which is again an interval of length lib. By (a) , ° � II (t) 1 � B for almost every t E R; thus , I is bounded a.e. , and g T a f E L 2 ( In ) . The

n

7. Wavelets

474

Z}

Emb

functions { b l / 2 form an orthonormal basis for :mE by Parseval's identity 3 .4.2( c) ,

L 2 (In). Thus ,

Hence,

By (a) ,

which implies that Alb and Bib. 0

{Em b Tna f

:

m, n

E

Z} is a frame with frame bounds

Exercises 7 . 9 1 . Give an example of a frame that is neither tight nor exact.

(X n ) be a sequence of vectors in a Hilbert space V. Show that : (a) If (x n ) is a frame , then ((x, xn)) E i2 ( N ) for all x E V , and the map T : V -+ idZ) ' x 1-+ ((x, xn)) , is a bounded , linear 1- 1 map with a bounded inverse. (b) If T V i2 ( N ) , x ( (x, xn)), is a bounded, linear , 1- 1 map with a bounded inverse, then (Xn) is a frame.

2. Let

:

-+

1-+

3 . CLO SE TO I => INVERTIBLE Let A : V -+ V be a bounded linear map on the Banach space V . Show that if III - A l l < 1 , then A - I exists and is a bounded linear map on all of V. 4. REPRESENTATIO NS WITH TIGHT FRAMES Use 7.9.9 to show that if S is the frame operator of the tight frame with frame bounds A = B, then:

(xn)

(a) S = A I. (b) S - 1 = A - I I.

7. Wavelets

475

x E V, then x = A - I ( Ln EN {x, Xn } xn) . 5. T ::; U II T II ::; 11 U 11 Use the fact that if a linear map T is bounded and self-adjoint, then II T II = s UP l x l = I I{ Tx, x } 1 (Bachman and Narici 1966 , p . 378) to show that if T, U V V are positive maps on the Hilbert space V, then T ::; U II TII ::; 11 U 1i . 6 . Let (Xn) be an exact frame in the Hilbert space V with frame bounds A and B. Show that A ::; II x ml1 2 ::; B for all m E N . 7. For the unitary operators Ta , Ea , and Da defined on L 2 (R) (men tioned before 7 . 9 . 13), show that for any a E R: (c) If

=>

=>

(a) (b) (c)

:

�

( j, Ta g) = {L a l, g} . ( j, Ea g) = { E- al, g}. ( j, Da g) = (D I / al, g) for a =P O.

8 . In the notation of the proof of 7.9 . 14, show that

(gTn a /, Em b ) .

{g , EmbTn a /}

Hints 2. (b) . Since T is bounded,

E l{ x, xn } 1 2 = II Tx 11 2 ::; II T 1I 2 11 x 1I 2 .

n EN I Since T - is bounded, II x II 2 = II T- 1 ( { x, xn } ) 1 2 ::; II T- 1 11 2 E I {X, xn }l 2 . nEN 3 . Use the facts that L ( V, V) i s a Banach space and that A - 1 = 1 + E (1 - At n EN converges with respect to the operator norm i n L ( V, V) . 6 . Apply the frame inequality to S- I X m to get < L n EN I (S - I X m , X n) 1 2 = I ( S- l xm , X m) 1 2 by 7.9 . 1 1 < II S - I x m I 1 2 1I x m Il 2 . Therefore , A ::; II x m ll 2 for all m E N . Now, by the frame inequality, II x mll 4 I {xm, X m} 1 2 ::; Ln EN I{ x m , Xn } 1 2 < B ll x mll 2 .

476

7. 10

7. Wavelets

Splines

So far, we have considered the discontinuous Haar wavelets and the contin uous Franklin wavelets. The smoothing process can be continued by using spline wavelets, an idea we briefly introduce in this section. Splines re duce to polynomials on intervals [n , n + 1] ( n E N ) , and the meetings with other intervals are sufficiently seamless as to leave a differentiable function overall. First, we define a basic spline. Let

N1 (t) = 1[0, 1 ) (t) be the scaling function for the Haar wavelets ( Example tion with itself,

7 . 3 . 1 ) . Its convolu

N2 (t) = N1 (t) * N1 (t) = ( 1 - I t - 1 1 ) 1[0, 2) (t) is the hat function used in the development of the Franklin wavelets in Example 7.8.2. For n � 2 , we define the nT H ORDER BASIC SPLINE ( B SPLINE) recursively:

Nn (t) = Nn - 1 (t) * N1 ( t) =N1 (t) * . . . * N1 (t) . n factors

{

,

V

.I

(7.68)

We leave it as an exercise to show that

N3 (t) = N2 ( t ) * N1 (t) =

t 2 /2, o � t � 1, - ( t - 3/2) 2 + 3 /4, 1 � t � 2, 2 � t � 3, ( t - 3) 2 /2 , elsewhere. 0,

(7.69)

For n E N , we take Cn (R) , to be the space of n-fold continuously differentiable functions on R. We take Co (R) and C- 1 (R) to be the spaces of continuous and measurable functions on R, respectively. Pn denotes the collection of polynomials f of degree less than or equal to n: deg f � n .

7.10.1 SP LINES The space Sn or Sn (R) of SPLINES of order n E N with KNOT SEQUENCE Z is the collection of all f E Cn - 2 (R) whose restrictions f l[k ,H 1] to the intervals [k , k + 1] , k E Z, belong to Pn - 1 . 0 For example, if n = 2, then for f to belong to S2 , f must be continuous and piecewise linear on every interval [k , k + 1] , k E Z. Dilations and trans lations of the hat function h (t) = ( 1 - It - 1 1 ) 1[0, 2) (t) , for example , belong to S2 . We develop a few fundamental properties of basic splines next . Definition

477

7. Wavelets

Since

with x

I:

Nn (t)

Nn + 1 (t)

=t

1

-y

Nn + 1 (t)

Ndt)

- y) N1 (y) dy Nn (t - y) dy ,

1

we get

*

Nn (t

i-t 1 Nn (x) dx. t

=

7.10.2 For any n E N, we have supp Nn t E (O, n ) .

=

(7.70)

[0 , n] and Nn (t) > 0 for

0,

Proof. First we show that supp Nn C [0 , n] . Clearly, supp N1 = supp 1 [ 1 ) = [0 , 1] . Assuming that supp Nn (t) C [0 , n] , then equation (7.70) implies that Nn + 1 (t) = 0 for t < 0 and t - 1 > n ; hence supp Nn + 1 (t) C [0 , n + 1]. We prove that (0, n) C supp Nn by showing that Nn (t) > 0 for t E (0, n) by induction. Clearly, (t) = 1 [ 1 ) (t) > 0 for t E [0 , 1 ) . Assume that Nn (t) > 0 on (0, n). If t > 0 or t < n + 1 (t - 1 < n ) , it follows from equation (7.70) that Nn + 1 (t) > O. 0

0,

N1

Next , we show that basic splines are rather smooth.

E N. Proof. Clearly, N1 E Sl . If Nn E Sn , then Nn E Cn - 2 (R) , while equation (7.70) shows that Nn + 1 E Cn - 1 (R). Recall that Nn is a polynomial of degree at most n - 1 in any interval [k - I, k] , k E Z , and let t E [k , k + 1] . 7.10.3 Nn E Sn for all n

Split the integral of equation (7.70) to write Nn+ l (t)

Nn

=

(J[kk-1

+

Jtk Jt-1 k -1 _

)

Nn (x) dx.

Since (x) is a polynomial of degree at most n - I on both [ k - I , k] and [k, k + 1] , (t) is a polynomial of degree at most n on [k , k + 1] . 0

Nn+ 1

Another basic property is the following.

7.10.4 U NI TY PROPERTY OF BASIC SPLINES For every n

L Nn ( t - k ) k EZ

=

1 for all t

E N,

E R.

P roof. By 7 . 1 0 . 2 , supp Nn = [0 , n] . Therefore , for any given t, all but a Nn (t - k) are 0 ; the series is therefore finite number of summands in

L k EZ

478

7. Wavelets

convergent . Now we argue by induction . The result clearly holds for n = l . Assuming that l: k e Z Nn ( t - k ) = 1 for n E N , consider

l: k e Z Nn +1 (t - k)

=

l: k e Z 11 Nn (t

1o 1 l: Nn (t

-k

k

-

-

-

y) dy

y) dy

ke Z

The Fourier transform of the Baar wavelet -

Nl ( w

Nl

=

1[0, 1 )

) _- l _ e - iw _ _ iw / 2 sin w/2 . - e

.

zw

is

w /2

It follows from the convolution theorem that (7.7 1 ) Thus Nn (w) is continuous and vanishes at Now consider

00 .

(7.72) By 7 . 1 0 .2 and 7 . 1 0.3, Nn E Von . The same is true of translates (Exercise 5 ) ; namely, Nn (t - k) E Von for all k E Z. Next , we consider the space of splines of order n with knot sequence 2 - i Z, j E Z. Note that S� = Sn . We have the sequence

S�

Now let Then

. . . C

S:; 2

C

S:; 1 C S� = Sn C S� C S; C . . . .

Vp = cl [S� n L2 ( R) ] .

Vr C V2n C . . . . In fact , this is a Riesz MRA with scaling function Nn . For the details that {Nn (t - k) : k E Z} is a Riesz basis for Von , see Wojtaszczyk 1 997. Since {( Nn (t - k) : k E Z)} is a Riesz basis for Von , we could follow the procedure of Section 7.7 to obtain I{J such that {l{Jn (t - k) : k E Z} is an orthonormal basis for Van . The process leads to the BATTLE-LEMARn� WAVELETS . Something nice is lost in the process, however, since the I{Jn (t - k) and associated wavelets do not have compact support as did the Nn (t - k ) . . . . C

V�2

C

V�1

C

Von

C

7. Wavelets

479

Exercises 7. 1 0 1.

{ t 2/2, 2 N (t) = N2 (t) * N (t) = - (t - 23/2) + 3/4, l :::; t :::; 2,

Verify that

O :::; t :::; I ,

3

1

(t - 3) /2, 2 :::; t :::; 3, elsewhere. 2. NnBy -differentiating equation (7. 70), show that N� (t) = Nn - 1 (t) 1 (t - 1 ) . 3. ofUseitsinduction support, tothatshowis, that Nn is symmetric about t = n/2, the center 0,

4. forShowsomethatconstant if f E Sn (n E N) and f 1 [- 1 ,0]= 0 , then f 1[0 , 1] (t) = ctn - 1 5 . Show that (see equation (7. 7 2)) Nn (t - k) E Von for all n E N and k E Z. 6 . Replace W by 2w in equation (7.71) and show that I: 1 N: (2w + 271'k ) 1 2 = (sin2n w) I: (w + 11!'k) 2n ' kE Z kE Z The special case n = 2 yields equation (7.57) . 7. Use Exercise 3 to show that i: Nn (y + k) Nn (y) dy = N2n ( n + k) . c.

Hints

All left-handexistderivatives of f are 0 at t = O . But f E Sn , so all n - 2 4. derivatives 0; therefore they are all O. Since f 1[ - 1 , 0] is a at polynomial of degree at most n - I, the result follows. 5 . Since Nn (t) E Sn , it is cle ar that Nn (t - k) E Sn for all k E Z . Also, since k and t > k + n, so n (t) to[0, Ln) ,(R)Nn for(t -alk)l k E0 Zforandt
=

7. Wavelets

480

7. 1 1

The Continuous Wavelet Transform

f E L l (T) , i ( n ) = i: f(t) e - int dt = ( I, eint ) , E Z}. produces a bisequence of projections of f onto the orthogonal basis { e i n t The Fourier transform of f E L l (R) n L 2 (R) , 00 f(t) e - iwt dt = (I, eiwt ) , i (w ) = � 211" 1The finite Fourier transform of

:

n

00

is a continuous version of the finite Fourier transform. In this section we consider a function h (R) and the CONTINUOUS WAVELET TRANSFO RM OF f (R) ,

E L2

E L2

t --b ) dt, a =P 0, b E R . (Wh f) (a, b) = l a l - 1 / 2 1-00 f (t) h ( a 00 The continuous wavelet transform can effectively treat signals f (t) with

spikes whose Fourier series would require many high-frequency exponen tials. In the course of our investigation we consider a more general uncer tainty principle than the well-known one of Heisenberg concerning momen tum and p osition. First, we consider a time-lo calized Fourier transform. Definition 7 . 1 1 . 1 WINDOWED FO URIER TRANSFORM

E L2

tg E L2

( Tg,b f ) (w ) =

i: f(t) e - iwt g (t - b) dt

A function 9 (R) such that (R) is called a WIN D O W or WIN D O W FUNCTION. The WINDOWED FOURIER TRANSFORM WFT it induces is

(

=

( I , e iwt g (t - b)

)

.

Occasionally, Tg,b is abbreviated to just n . 0 As mentioned in Exercise 1 , any window function has the property that (R) . 9 (R) and 9

E LI It 1 1 / 2 E L2 Example 7.11.2 T H E GABOR TRAN SFORM If, as a particular win dow, we take th e Gaussian fun ction g a (t) = );rae -t 2 / 4 a , then we get the GABOR 2 1 , b f) (w) = G (a, b; f) (w) 1-0000 f(t) e - ·wt 2y1l"a � e - ( t- b ) /4a dt.

TRANSFORM

( Tg

=

2

0

In this context we have the following analogues of mean and standard deviation.

7. Wavelets

48 1

Definition 7 . 1 1 . 3 CENTER, RADIU S , WIDTH

9 are, respectively, 1/2 1 (1 00 1 1 00 ' ) 2 2 2 = . and I::. t*) dt t (t) t* = dt (t) (t g g g I 1 II 9 II 2 - 00 II g ll 22 -00 I 1 The WIDTH of 9 is 21::.. g . The center of the Gaussian window ga (t) = 2Fa e -t 2 / 4 a is t* = 0 ; its width 21::.. g a is 2Ja. If 9 is a window, then 9 E L 2 (R) . If wg E L 2 (R) , i.e . , if 9 is a window function, too, then we define the center w * , radius 1::.. 9 , and width 21::.. 9 of 9 we did for g . The CENTER and RADIU S of a window function

0

as

Definition 7. 1 1 .4 SHORT-TIME FOURIER TRANSFORM

9

(Tg ,b/) (w) is called the SHORT ) Ub,p (t) = eipt g (t - b) ,

If and 9 are windows, then the WFT TIME FOURIER TRANSFORM STFT . 0

(

With

by the standard properties of the Fourier transform we have

9

9

Ub;

(w) = e -i b (w - p ) g (w - p) .

If and are windows, we take the point of view that the time and frequency information (the latter is also called spectral information) is con centrated, respectively, in +b+b+ and +p+p+ The net result is t o produce a time-frequency win dow

[t* I::..g , t* I::.. g] [w* 1::.. 9, w* 1::.. 9]. [t * + b - I::..g , t* + b + I::..g] [w * + p - 1::.. 9, w" + p + 1::.. 9] of fixed area 41::.. 9 1::.. 9 , regardless of the signal f (t). For better localization, we want the time-frequency window to be of small area, but we cannot make both I::.. g and 1::.. 9 small: The area of the time-frequency window is limited by the un certain ty principle 7. 1 1 .6 , namely, 1::.. 9 1::.. 9 � t ; the optimal choice (namely 1::.. 9 1::.. 9 = t) is a Gaussian window. In order to prove 7 . 1 1 .6 , x

w e use the following lemma.

f =1= 0, and f, f', tf E L 2 (R) , then I Re 1: tf(t)f' (t) dt l 2 < (1: Itf(t) l lf' (t) 1 dt) 2 < (1: I tf(t) 1 2 dt ) (1: I f' (t) 1 2 dt ) . Equ ality holds if and only if f is Gaussia n. 7. 1 1 . 5 If

482

7. Wavelets

P roof. The first inequality is clear ; the second is a consequence of the Cauchy-Schwarz inequality. We leave the assertion about equality for Ex ercise 4. 0

We prove the uncertainty principle for functions from the space S (R) of infinitely differentiable functions on R that are rapidly decreasing in the sense that for all m, n E N U {OJ ,

f

It

sup m f( n ) teR

(t) 1 < This space was first discussed i n Exercise 5.19-1. For the general result, see 00 .

Dym and McKean 1972.

9

g

7. 1 1 .6 T H E UNCERTAINTY PRINCIPLE If E S ( R) and are win dows, then � � . Equ ality holds if an d only if 9 is Gaussian.

6.g 6:9

P roof. We only prove the first statement; we leave the second for Exercise 5. By a change of variables , we may assume that the centers of and are each O. Consequently,

9

( 6.g 6.g) 2 =

1 ( [: t 2 1 g (t) 1 2 dt ) II g ll � I 1 g ll � 00 (t) 2 dt II g ll /2 11 g 11 22 (1- 00 e l g 1 ) �

([: w 2 Ig (w) 1 2 dW ) (1 l iwg (w) 1 2 dW ) .

By ?? on the Fourier transform of derivatives Parseval's identity 5 . 18.3(a) (namely comes 00

00

- 00

(f (w) =

II f ll � = ( 1/211")

(iw) k f(w)) and

I �I : )

)( 1 ( (1 ) ) (1 1 II ll 2

twice , this be

t 2 1 g (t) 1 2 dt 211" I g ' (t) 1 2 dt \ 2 11" 11 9 1 1 I I g ll 22 - 00 - 00 00 00 2 - � t 2 1 g (t) 1 2 dt I g ' (t) 1 dt . g =

- 00

g

00

)

- 00

gg' + gg' = 2 Re gg' , this is 2 > � I Re l °O tg (t) g, (t) dt I II g ll 2 00- 00 1 1 1 1 t d Ig (t) 1 2 dt 1 2 I I g lI � "2 - 00 dt

By 7 . 1 1 .5 and the fact that

(7.73)

Integrating the last expression by parts and using the fact that 9 E S (R) , so that 0 as 00 , we get that

t I g (t) l -+ I t l -+ °O 2 dt ) 2 = � , (t) ( 6. g 6. g) 2 2: � g ( 1 I � I II gll 2 - 00

7. Wavelets

483

which proves the first assertion. The second follows by observing that in equality occurs only at (7.73) and then using 7 . 1 1 . 5 0

.

Definition 7 . 1 1 . 7 THE CONTINUOUS WAVELET TRANSFORM

h

( a ) fb ( Wh f) (a, b) = l a l - 1 / 2 1: f(t) h C� b ) dt.

Let E L 2 (R) and let 1= 0 and be real numbers. The CONTINUOUS WAVELET TRANSFORM C WT of E L 2 (R) is given by

h the MOTHER WAVELET . With t � b) , 2 ha,b (t) = l a l 1 / h ( the terms (Whf) ( a , b) = (f, h b ) are called the CONTINUOUS WAVELET COEFFICIENTS OF f. h are windows, then we can speak of centers t* and w* and radii f:,.about hIfofh and h and f:,.h of h. The CWT (Whf) (a, b) has localized information f in a time window [b + at* - af:,.h, b + at* + af:,. h), a window that diminishes as a decreases and increases as a increases. As for the frequency window, by the basic properties of the L 2 -Fourier transform, 1 (w) = lal - 1 / 2 l a l e - ' bW h (aw) . 2 71" ha,b By Parseval ' s identity 5 . 1 8 .3(b) , it follows that h (aw) dw . ( Wh f) ( a , b) = I a21 71"1 / 2 1-00 i(w) ei bw -00 Note that 9 (w) = h (w + w*) has center O . However, g (aw - w*) h (aw - w* + w*) h (aw) . Thus (Whf) (a, b) = , aJ� 2 1: i(w) eibw g (a (w - :* ) ) dw , so it is clear that ( Wh f) (a, b) also gives localized spectral ( = frequency) behavior in the frequency window [w * / a - f:,.h/ a, w* / a + f:,. h / a ] We there fore have a time-frequency window [b + at* - af:,.h, b + at* + af:,.h) x [w * /a - f:,.h/a,w * /a + f:,.h/a] We call

-

a,

·

0

�

.

7. Wavelets

484

t-w

in the plane of width 2a.6.h. This feature makes the CWT a more flexible tool than the WFT for many purposes. Some elementary properties of the CWT include:

f, 9 E L 2 (R) and e , d E C , (Wh [ef + dgD ( a, b) = (Wh f) (a, b) + d (Wh 9) ( a, b ) . ( Translation) For f E L 2 (R) and e E C , (Wh [f (t - e) ) ) (a, b) = (Wh f) ( a, b e) . ( Scaling) For f E L 2 (R) and d > 0 , ( Wh [d 1 /2 f (dt) ] ) (a, b) = (Wh f) (da, db) .

1. ( Linearity ) For

e

2.

-

3.

To obtain reconstruction formulas, we imp ose an ADMISSIBILITY CONDI TION on the mother wavelet

h:

00

1 I h (w) 1 2 dw < - 00

Iw I

00 .

A particular consequence of the admissibility condition when h and windows is that = O.

( See Exercise 7.)

1: h (t) dt

a>0: (Wh f) (a, b) = a - 1 / 2 1 f (t) h t : b ) dt.

Consider the CWT again with

If we let

(

:

a and b assume only the discrete values a = at b = kbo at j, k E Z,

then we obtain the DISCRETE WAVELET TRANSFO RM ( DWT )

(Wh f) (at kboai ) = a r; 2 1: f (t) h (a oi t - kbo ) dt. il

We are therefore considering the countable set of functions

a -i/2 o h ( a o-i t - kb0 )

,

.

J

,

k E Z.

(7.74)

h

are

7. Wavelets

With

485

ao = � and bo > 0 , we have the functions

and

( 2i kbo /2i ) = (f, hi , k ) , j, k E Z, which is a set of discrete wavelet coefficients for I E L 2 (R). Can we recover I if we know just these coefficients? Can it be done in a "numerically stable" ( Wh l) 1 / ,

manner? , i .e . , in such a way that small changes in the discrete wavelet coefficients produce a small change in I? It can be done if forms a frame for (R) . Thus, we require that there exist frame bounds A, B > 0 (R), (Definition 7 .9 . 1) such that for all I E

L2

B y Exercise

{hi , k }

L2 A II / II � S; L l (f, hi ,k ) 1 2 S; B II / II � · i , keZ

7 . 9-2, the map

£ (Z) , «(f, hi2, k ))i ,keZ ' is a bicontinuous linear 1- 1 map . Since T is injective, I is completely de -l

termined by its discrete wavelet coefficients ; the continuity of T implies that this can be done in a numerically stable way. Alternatively, consider the frame operator of 7.9.8

SI = L ( f, hi ,k) hi , k . i , keZ By 7.9.9, S- 1 is bounded, (S - l hi , k ) is a frame, and 1 = L ( t, S- l hi ,k ) hi , k = L (f, hi ,k ) S - 1 hi , k . i , keZ i , keZ The continuous wavelet transform ( W f) ( a , b) can usually b e calculated h

only by numerical techniques, not in closed form. There is a "fast" way (as in "fast Fourier transform" ) to compute the discrete wavelet transform.

Exercises 7. 1 1 1 . WINDOW FUN CTIONS For a window function g , (i.e . , g , tg E show that: (a)

It 1 1 /2 9 E L 2 (R) .

L 2 (R»

7. Wavelets

486

( b) 9 E

L t { R) .

II ga ll ; = ( 8 11"a) - 1 / 2 . ll ga = va. If f, f' E L 2 ( R) , show that f (t) --+ 0 as t ( a) (b)

3.

--+ 00 .

4. Show that equality holds i n 7. 1 1 .5 if and only if

f i s Gaussian.

5 . Complete the proof of 7 . 1 1 .6 . 6 . Prove the linearity, translation, and scaling properties o f the CWT. 7. For a mother wavelet h satisfying the admissibility condition, equa tion (7.74) , and such that h and h are windows, show that J�oo h =

(t) dt

o.

Hints

I t l ) g (t)

It I)

1 . ( b) . Clearly, 1 / ( 1 + E LdR ) . By hypothesis , ( 1 + E L 2 ( R) . Now use Holder's inequality to conclude that 9 E Ll ( R ) . 3 . Note that

f2 (t) = f2 (0) + lo t (P)' (u) du = f2 (0) + 210t (ff' ) (u) duo By hypothesis and Holder ' s inequality, it follows that ff' E Ll ( R) . It therefore follows from the equation above that limt ..... oo f (t) exists. Now show that the limit is O.

4. If equality holds in 7 . 1 1 .5, then - Re

c

and for some =f. 0,

tf (t) f' (t) = I t ! (t) l l f' (t) 1 , I t f (t) 1 = e l f' (t) l ·

(7.75) (7.76)

Retf (t) f' (t)

Equation (7.75) is one of two possibilities, the other being = As we will see, the choice we made leads to an f E L 2 (R) ; the other does not . Equation (7.76) follows from the gen eral condition for equality to hold in the Cauchy-Schwarz inequality. e i u (t ) , where is a = Now = implies that real-valued function of Since

I t f (t) l l f' (t) l ·

, It f (t) I e l f' (t) 1

tf (t) cf' (t) t. - Re ( tf (t) f' (t) ) = I t f (t) f' (t) I ,

u

7. Wavelets

487

tl ( t ) I' (t) ::; O.

it follows that

Consequently, - e I f' ( t) 1 2 eiu(t ) 2: O . Hence eiu(t ) = - 1 , and therefore tl (t) = - e l' (t) . Thus I (t) = ke - t 2 / 2 c for some constant k i O .

7. By Exercise 1 , h, Ii E L l (R) , and Ii ( w ) i s continuous. The admissi bility condition implies that Ii (0) = O. Thus 0 = Ii (0)

= lilllw -+ o Ii (w ) h (t)e -iwt dt = lilllw -+ O h ( t) dt =

J�oo

J�oo

by the dominated convergence theorem.

7. Wavelets

489

References 1. M. ALDROUBI 1 9 9 5 , Portraits of frames, Proc. Amer . Math. Soc . 123, 1661-1668 .

2 . J . M . ASH 1 9 8 9 , Uniqueness of representation by trigonometri c se ries, Amer. Math. Monthly 96, 873-885.

3 . R . ASH AND M. GARDNER 1 97 5 , Top ics in stochastic processes, Academic Press , New York . 4.

L . AUSLANDER , T . KAILATH , AND S . MITTER (EDITORS ) 1 9 9 0 , Signal processing theory, Springer-Verlag, New York; see especially Wavelets and frames by C. Rei!.

5 . G . BACHMAN AND L . NARICI 1 96 4 , Elements of abstract harmonic analysis, Academic Press, New York .

6 . G . BACHMAN AND L . NARICI 1 9 6 6 . Fun ctional analysis, Academic Press , New York . Reprinted by Dover Publications, Mineola, NY , 1 998 . 7. S . BARNETT 1 9 9 0 , Ma trices-meth ods and applications, Clarendon Press , Oxford.

8 . R. BARTLE 1 99 5 , The elements of int egration and Le besgue measure, Wiley Classics Library, New York.

9 . N K BARY 1 964, A treatise on trigonometric series I, Macmillan , New York. .

.

10. J . BENEDETTO 1 9 97 , Harmonic analysis and applications, CRC Press , Boca Raton . 1 1 . J . BENEDETTO A N D M FRAZIER 1 9 9 3 , Wavelets: Ma th ematics a n d applications, CRC Press , Boca Raton. 1 2 . G . BERGLAND 1 967, The fast Fourier transform recursive equ ations for arbitrary length records, Math. Comp o 21 , 236-238. 13. F. BEUTLER 1 96 1 , Sa mpling theorems and bases in a Hilb ert space, Information and Control 4 , 97-1 17.

14. M. BO CHER , 1 9 0 6 , Introdu ction t o the theory of Fourier's series, Annals of Math. 7 .

1 5 . S. BOCHNER AND K CHANDRASEKHARAN 1 9 4 9 , Fourier transforms, Annals of Mathematics Studies 1 9 , Princeton University Press, Prince ton, NJ . .

490

7. Wavelets

16.

O . BONNET, 1 850, Memoires couronnes et memoires des savants Etrangers publies par l 'a cadimie royale des sciences, des lettres et des beaux- arts de Belgique, 23 .

17.

D . BRESSOUD 1 994, A radical approach to real analysis, The Mathe matical Association of America, Washington, D.C. Uses Fourier series as basis for exploring analysis with good historical notes.

18.

W . BRIGGS AND V . E . HENSON 1 995, The DFT, an owner's m a n u al for the discrete Fourier transfo rm, SIAM.

19.

G. CANTOR 1 870, Beweis, das eine fur jeden reellen Wert von x durch eine trigonometrische Reihe gege bene Funktion f (x) sich nur auf eine einzige Weise in dieser Form darstellen lasst, J. f, d. reine u. angew. Math. 72, 139-142; also in Gesammelte A bhandlungen, Georg Olms , Hildesheim 1962, 80-83.

20.

L. CARLESON 1 96 6 , On convergence and growth of partial sums of Fourier series, Acta Math. 1 1 6 , 1 35-157.

21.

L. CARSLAW 1 93 0 , Introdu ction to the theory of Fourier's series and integrals, Macmillan, New York ; reprinted by Dover, New York 1952. Thorough and readable.

22. M . CARTWRIGHT 1 99 0 , Fourier methods, Ellis Horwood , Chichester. 23. A . L . CAUCHY 1 841 , Memoire sur diverses formula es de analyse, C. R. Acad . Sci. Paris 12, 283-298. 24. C . CHUI 1 9 92 A , A n introdu ction to wavelets, Academic Press , San Diego.

25.

C . CHUI 1 992B , Wavelets: A tutorial in theory and applications, Academic Press , San Diego.

26.

B . CIPRA 1 993, Wavelet applications come to the fore, SIAM News, November.

27.

R. COOKE 1979 , Th e Cantor-Lebesgu e theorem, Amer . Math. Monthly 86, 558-565.

28.

J . COOLEY AND J . TUKEY 1 965, A n algorithm for the mach in e calculation of complex Fourier series, Math. Comp o 1 9 , 297-301.

29.

J . CORRADO 1 984, Vibration signatures are easy to rea d with FFT, Design News 1/23/84, 125-128. How the FFT is used to eliminate vibrations in machinery.

30.

R. COURANT AND H. ROBBIN S 1 94 1 , What is mathematics '? Oxford University Press , New York .

7. Wavelets

49 1

3 1.

I . DAUBECHIES 1 988, Orthonormal bases of compactly supported wavelets, Comm, Pure and Applied Math. 4 1 , 909-996.

32.

I . DAUBECHIES 1 991 , The wavelet transfo rm: A method of tim e frequ ency localization in A dvances in spectrum analysis and array processing I, edited by S. Haykin, Prentice-Hall, Englewood Cliffs .

33.

I . DAUBECHIES 1 992 , Ten lectures o n wavelets, CBMS-N SF Regional Conference Series in Applied Mathematics, SIAM , Philadelphia.

34.

I. DAUBECHIES , A. GROSSMAN AND Y. MEYER 1 986, Painless nonorthogonal expansions, J . Math. Phys. 27, 1271-1283.

35.

P . DAVIS

1 979,

Circulant matrices, Wiley, New York .

36. CH . DE LA VALLEE-POUSSIN 1912, Sur l 'uniciti du developpement trigon ometrique, Bull. Acad. Roy. de Belg. 702- 718.

37.

J. DIESTEL 1 984, Sequences and series in Banach spa ces, Springer Verlag, New York.

38.

P . DIRICHLET 1 82 9 , Sur la convergence des series trigonometriqu es qui servent a representer une function arbitraire entre des limites donnees, J. fur d. reine u. angewandte Mathematik . 4, 157-1 69.

39. A. DVO RETZKY AND C . RO GERS 1 9 5 0 , A bsolute and un con ditional convergence in normed lin ear spaces, Proc . Nat. Acad. Sci. U . S .A . , 3 6 , 192- 1 97.

40 . N . DUNFORD AND J . SCHWARTZ Interscience , New York.

41. 42.

1 958.

Linear operators, Part I ,

H . DYM AND H . McKEAN 1 97 2 , Fourier series and int egrals, Aca demic Press , San Diego.

R. EDWARDS York .

1967 , Fourier series Holt, Rinehart and Winston , New

H.

43.

FEICHTINGER AND K GRO CHEN 1 992 , Gabor wavelets and the Heisenberg group in Wavelets: A tutorial in theory and applications edited by C. Chui, Academic Press, San Diego.

44.

L . FEJER 1 904, Untersu chungen tiber Fouriersche Reihen, Mathe matische Annalen. 58 , 5 1 -69.

45.

G . FOLLAND 1 9 9 5 , A course in abstract harmonic analysis, CRC Press , Boca Raton.

492

7. Wavelets

46. J . FO URIER, The an alytical theory of heat, Dover, Mineola, New York , a translation of the 1 822 version of Theorie analytique de la chaleur published by Didot, Paris. The 1822 version is an improvement of Fourier's 1 8 1 1 version. Theorie de la propaga tion de la chaleur dans les solides, reproduced in Grattan-Guiness 1972. 47. W . GENTLEMEAN AND G . SANDE 1 9 6 6 , Fast Fourier transfo rm for fun and profit, Proc. 1 966 Fall Joint Computer Conference AFIPS 29, 563-578. 48 . A. GLUCHOFF 1 99 4 , Trigonometric series and theories of int egration, Mathematics Magazine 67 , 3-20. How the integrals of Cauchy (con tinuous functions) , Riemann (continuous a.e.) , and Lebesgue were invented to address problems concerning Fourier series and their ap plications to those problems inunediately afterward. 49 . R. GO LDBERG 1 9 7 6 , Methods of real an alysis, Wiley, New York .

5 0 . 1 . GRATTAN-GUINEs s 1 9 7 2 , Joseph Fourier, 1 768-1 830; a survey of his life and work, MIT Press , Cambridge, MA. Fourier's original 1807 paper with excellent annotation. 5 1 . A . GROSSMAN AND J . MORLET 1 9 8 4 , Decomposition of Hardy func tions int o square-integrable wavelets of constant shape, SIAM J . Math. Anal. 1 5 , 723-736.

52. A. GRO THENDIECK 1 955, Produ its tensoriels topologiques et espaces nucieaires, Memoirs of the American Mathematical Society 16, Prov idence, R.1. 53. M . HEIDEMAN , D . JOHNSON , AND C. BURRUS 1 9 8 4 , Gauss and the history of the fast Fourier transform lEE ASSP Mag. (October) 14-2 1 ; also in Arch . Hist. Exact Sciences 34 ( 1985) 265-277.

54. C. HElL AND D . WALN U T 1 98 9 , Continuous and discrete wavelet transforms, SIAM Review 3 1 , 628-666.

55. E . HERNANDEZ AND G . WEISS 1 99 6 , A first course on wavelets, CRC Press , Boca Raton.

56. J . R . HIGGINS 1 9 9 6 , Sampling theory in Fourier and signal analysis, Oxford University Press , New York. 57. S . HILDEBRAN DT AND A. TRO MBA 1 98 5 , Math ematics and optimal form, Freeman, New York . 58. E . HOBSON 1 95 7 , Th e theory of functions of a real variable, Dover, New York .

7. Wavelets

493

59. M . HOLS CHNEIDER 1 9 9 5 , Wavelets: An analysis tool, Oxford , Lon don. 60 . R . HUNT 1 96 8 , Orthogonal expansions and their continu ous an a logu es, Southern Illinois University Press , Carbondale , Illinois, 235255. 6 1 . E. JAHNK E , F. EMDE , AND F. L O S CH 1960 , Tables of higher func tions, 6th ed., McGraw-Hill, New York . 62 . O . J0RSBOE AND L . MEJLBRO 1 98 2 , The Carleson -Hunt theorem on Fourier series, Lecture Notes in Mathematics 9 1 1 , Springer-Verlag, New York. 63. G . KAISER 1 994, A friendly guide to wavelets, Birkhiiuser, Boston.

64. Y . KATZNELS ON 1 968, A n introdu ction to harmonic analysis, Wiley, New York .

6 5 . T . KAWATA 1 9 7 2 , Fourier analysis i n probability theory, Academic Press , New York .

66 . H . KO ORNWINDER 1 9 9 3 , Wavelets: A n elementary treatment of the ory and applications, edited by T. Koornwinder, World Scientific Press. 67. T . KORNER 1 988 (reprinted 1 992) , Fou rier analysis, Cambridge Uni versity Press , Cambridge-New York . 6 8 . J . LINDENSTRAU SS AND L. TZAFRIRI 1 977, Classical Banach spa ces I, Springer- Verlag, New York . 69. F . LUKACS 1 9 2 0 , fiber die Berstimmung des Sprunges einer Funk tion aus ihrer Fourierreihe, J . fur Mathematik 150, 107- 1 1 2. 70. S . MALLAT 1 989, Multiresolution approximation and wavelet or thonormal bases of L 2 (R) , Trans. Amer . Math. Soc. 3 1 5 , 69-8 8 . 7 1 . J . MARSDEN AND M . HOFFMAN 1 993 . Elementary classical analysis, W. H. Freeman, New York. 72. E. M C SHANE 1 947, Int egration, Princeton University Press , Prince ton, NJ .

73 . Y . MEYER 1 9 9 0 , Ondelettes et op erations I and II, Hermann, Paris.

74. Y . MEYER 1 9 9 3 , Wavelets-algorithms and applications, (translated and revised by R. Ryan) SIAM . 75 . Y . MEYER 1 9 9 3 , Book review, Bull. Amer . Math. Soc . 28, 350- 360.

494

7. Wavelets

76 . N . MORRISON 1 99 4 , Introdu ction to Fourier analysis, Wiley, New York. 77. C . MOZZO CHI 1 97 1 , On the pointwise convergence of Fourier series, Lecture Notes in Mathematics 199, Springer-Verlag, New York .

78. 1 . NARICI AND E . BECKEN STEIN 1 98 5 . Topological vector spaces, Marcel Dekker, New York .

79. I . NATANSON 1 96 1 , Theory of functions of a real variable I and II, Ungar, New York.

80. A. NAYLOR AND G. SELL 1 98 2 . Linear operator theory in engineer ing an d science, Springer-Verlag, New York . 8 1 . S . NIK O LSKY 1 977, A course of mathematical analysis, Mir Publish ers, Moscow. 82. M . PLANCHEREL 1 924-5 Le developpement de la thiorie des series trigonometriques dans Ie dernier quart de siecle, L'Enseignement Math. 24.

83. 1. PRASAD AND S. IYENGYAR 1 997, Wavelet analysis with applica tions to image processing, CRC Press . 84. C . REEs , S . SHAH , AND C . STANOJEVIC 1 98 1 , Theory and applica tions of Fourier series, Marcel Dekker, New York . 85. C . REIN D AND T . PASSIN 1 99 2 , Signal processing in C, Wiley, New York. 86. B. RIEMANN 1 8 6 7 , Uber die Darstellbarkeit einer Funktion durch eine trigonometrische Reihe, Gottingen, Abh. Ges. Wiss. 1 3 .

87. H . ROYDEN 1 96 8 , R e a l an alysis, 2nd e d . , Macmillan, New York.

88. W . RUDIN 1 9 6 2 , Fourier analysis on groups, Wiley-Interscience, New York. 89. W . RUDIN 1 974. Real and comp lex analysis, 2nd ed ., McGraw-Hill, New York . 90. W . RUDIN 1 976 , Principles of mathematical analysis, 3rd ed., McGraw Hill, New York . 9 1 . M . RU SKA I , G . BEYLK IN , R. COIFMAN , I . DAUBECHIES , S . MAL LAT , Y . MEYER, AND L. RAPHAEL , EDS . 1 992, Wavelets and their applications, Jones and Bartlett, Boston. 92. G. SANSONE 1 95 8 , Orthogonal functions, Wiley, New York.

7. Wavelets

495

93. G. SANSONE AND I. GERRETSON 1 96 0 , Lectures on the theory of functions of a complex varia ble, Noordhoff, Groningen. 94. C . SH ANNON 1 948 , A mathematical theory of communication, Bell System Technical Journal. 95. G . SIMMONS 1 963. Introdu ction to t opology and modern analysis, McGraw-Hill, New York. 96. G . STRANG 1 9 9 3 , Wavelet transfo rms versus Fourier transforms, Bull. Amer . Math. Soc . 28, 288-305. 97. R. STRICHARTZ 1 99 3 , How to m ake wavelets, Amer . Math. Monthly 100, 539-556. 98. K . STROMBERG 1 98 1 . A n introdu ction to classical real analysis, Wadsworth, Belmont, California . 99. Z . S Z MYD T 1 97 7 , Fourier transformation and lin ear differential equa tions, Reidel Publishing Co. , Boston. 100. B . S Z . - N AGY 1 9 6 3 Introdu ction to real functions and orthogonal ex pansions, Oxford University Press, Oxford. 101. C . SWARTZ 1 99 2 , A n introdu ction to functional analysis, Marcel Dekker, New York . 102. A . TAYLOR AND W . MANN 1 97 2 , A dvanced calculus 2nd ed., Xerox Publishing Co., Lexington, MA. 103. E . C . TITCHMARSH 1 9 3 9 , The theory of functions, 2nd ed., Oxford University Press , Oxford. 104. G . P . TOLSTOV 1 9 7 6 , Fourier series (translated by R. Silverman) , Dover, New York. This is a reprint of the 1962 version published by Prentice-Hall, Englewood Cliffs , New Jersey. 105. C . VAN LOAN 1 992, Computational frameworks for the fast Fou rier transform, SIAM . 106. E . B . VAN VLECK 1 9 1 4 , Th e influ ence of Fourier's series upon the development of mathematacs, Science 39, no. 995, 1 1 3- 124. An in teresting informative article. In an illustration of how times have changed, another article in the same issue of Science is "Scripture on Stuttering and Speech." 107. J. WALKER 1 9 9 1 , Fast Fourier transforms, CRC Press, Boca Raton. 108. J . WALKER 1 99 7 , Fourier analysis and wavelet analysis, Notices Amer . Math. Soc. 44, 658-670.

496

7. Wavelets

109. G . WALTER 1 994, Wavelet and other orthogonal systems with appli cations, CRC Press , Boca Raton.

1 10. R . L WHEEDEN AND A . ZYGMUND 1 977, Measure and int egral, Dekker, New York . .

1 1 1 . E . T . WHITTAKER 1 9 1 5 , On the functions which are represented by the expansions of the interp olation theory, Proc . Roy. Soc . Edinburgh , Sec. A (35) , 1 8 1-194. 1 12. P. WO JTA SZCZYK 1 99 7 , A mathem atical introdu ction to wavelets, Cambridge University Press , London.

1 13. R. YOUNG 1 98 0 , A n introdu ction to nonharmonic Fourier series, Academic Press, San Diego. 1 14. A . ZYGMUND 1 95 9 , Trigonometric series, 2nd ed., Cambridge Uni versity Press, New York .

Index

A

band-limited, 28 1 , 290 base = basis Hamel, 21 Riesz , 450 Schauder , 450 basis problem, 89 BattIe-Lemarie wavelets, 478 Bessel equality , 20, 23 , 103 inequality, 24, 103, 256 best approximation, 90, 98 does not exist , 94, 97 from complete subspace, 92 best mean approximation, 1 0 1 bicontinuous, 39 big oh notation, 223 bijective , 7 bisequences, 5 biseries, 5 bit reversal, 405 Bolzano-Weierstrass theorem, 7 1 bounded linear map , 50 map , 49 metric , 8 1 operator, 126 set, 49 bounded variation, 164 B-spline, 476 Buneman's algorithm , 405 butterfly relationship , 403

A* , adjoint, 135

a.e . , 6 Abel continuity theorem, 232 kernel, 283, 288, 3 1 8 summability, 23 1 absolutely continuous, 167, 213 convergent, 63 summable , 63 adherence point, 52 adjoint in Hilb ert space, 136 map A* , 135 admissibility condition , 484 algebraic complement , 85 almost everywhere, 6 angle in inner product space, 1 5 antipodal points , 47 approximate identity for L 1 (R) , 330 for L1 (T) , 271 approximation best, 90 best mean , 99

B

B(x , r) , open ball, 35 BV [a, b], 164

c

Baire category theorem, 6 1 null space, 8 Baire null space complete, 66 ball closed, 35 open , 35 Balian-Low theorem, 413 Banach space, 6 1 -Steinhaus theorem, 2 1 1

C , complex numbers, 1 C [a , b], 1 1 C1 [a, b] , 15 C(x , r ) , closed ball, 35 co , 47 Cc (R) , 54 CA , complement of 58 (C, 1) summable , 228, 304 Co (R) , 289

A,

497

498

Index

Cc (R) , 289 Cantor- Lebesgue theorem, 172 Carleson, 2 1 1 pointwise convergence theo rem, 194 Cauchy principal value , 182 sequence, 60 Cauchy-Schwarz , 14, 16 equality in, 14 causal, 34 1 Cesaro kernel, 282, 318 Fourier transform of, 3 1 8 Cesaro summability, 228 character, 396 characteristic function , 6 , 278 Chebyshev polynomial, 244 subspace, 90 circle group T, 265 circulant, 394 cl, closure operator, 52 clop en , 59 closed map , 75 set , 52 closure , 52 cluster point, 45 co dimension , 85 coefficient functionals, 456 commutatively convergent , 1 19 compact , 71 support , 54, 290, 436 sequentially , 7 1 complete, 6 1 orthonormal set , 107 sequence, 465 completeness and nested sequences, 64 completion, 6 1 conditionally convergent , 1 1 9 conjugate series, 197 space, 131 conjugate symmetry, 13

continuity , 59 in the mean, 73, 260 joint, 46 of a function, 39 of a linear map , 43 of basic operations, 40 of linear map , 47 of projections , 46 continuous dual, 1 3 1 Fourier kernel, 177 not differentiable , 1 6 1 wavelet transform (CWT) , 483 convergence commutative, 1 19 componentwise , 41 conditional, 1 1 9 equivalents , 3 8 i n metric space, 3 8 in the mean, 43 mean, 43 of series, 46 of two-sided series, 46 on products, 42 pointwise, 43 unconditional, 1 19 uniform, 44 convex, 17 convolution, 268, 328, 370 cyclic , 397 Fourier cosine transform, 350 on 29 1 theorem, 269, 293, 328, 37 1 , 397 countable, 55 coz x , cozero set of x , 19 cozero set, 19 cyclic convolution, 397

R,

D

doo , 4 d(x , S) , 57 de la VaIee-Poussin pointwise convergence theo rem, 195

Index

decapitation, 227 decomposition, 432 dense, 52 subspaces of Lp [ - 11", 11"] , P 2: 1 , 54 derivative one-sided, 165 DFT, 385 matrix, 386 diameter , 57, 64 finite for compact space, 74 diameter (of a set) , 5 1 Dido, 224 dilation, 1 14, 4 1 1 dilation equation, 416 dimension of vector space, 22 Dini test, 186 Dini theorem, 298 direct product, 84 direct sum, 123 external, 83, 124 inner product, 83 internal, 85, 123 topological , 86 Dirichlet function, 1 6 1 integral, 182-183 kernel, 1 74, 178 pointwise convergence theo rem, 1 9 1 test for series of constants, 217 test for series offunctions, 218 discrete Fourier transform, 385, 391 inversion , 390 discrete wavelet transform (DWT) , 484 distance from a point to a set, 57 distance function , 1 dominated convergence theorem, 44, 1 72 dual second, 134 space, 1 3 1

499

complete, 134 frame representation , 468 Dvoretzky-Rogers theorem, 120

E

e i , standard basis element , 19 eigenvalue, 137 eigenvector, 137 EnRo, 90 E-net , 69 equivalent metrics, 8 1 norms, 78 Schauder bases, 450 essential supremum, 33 essentially bounded, 33 Euclidean norm, 10 Euclidean space, 2 evaluation map , 131 even function, 1 9 , 147, 153 odd derivative, 69 even part of function , 400 extension by continuity , 67 exterior of set , 59 external direct sum, 124

F

Fn (t) , Fejer kernel, 235

i (w) , Fourier transform, 278 fast Fourier transform algorithm (FFT) , 400 Fatou's lemma, 439 Fejer kernel, 235 n-dimensional, 258 continuous, 308 Fejer theorem, 238, 259 n-dimensional, 259 FFT algorithm, 400 for N = 2 k , 40 1 for N = RC, 408 filter , 423-424 and Fourier transform, 424 compact support, 437 Haar, 424

500

Index

low-pass, 423-424 orthonormality of translates, 442 finite Fourier mapping, 266 finite Fourier transform, 265 finite FT, 265 finite sequences , 5 1 Fourier coefficients, 104, 1 12, 144 absolute convergence, 203 products, 153 size , 157, 167, 222 Fourier cosine integral, 302 Fourier cosine series, 1 55 Fourier cosine transform, 341 Fourier sine transform, 341 Fourier cosine transform in L2 (R) , 366 Fourier integral formula, 276, 295297 Fourier kernel continuous = sin wt /t , 177 n discrete - sin( +/ 2l / 2 )t ' 179 t selector property , 190, 193, 368 Fourier map , 289-290 finite, 266 for L 2 functions, 363 injective , 3 1 3 not onto, 3 14 Fourier series, 144 A-summability, 240, 249 amplitude-phase , 1 5 1 complex form, 144 convergence, 2 1 1 termwise differentiation, 221 divergent , 2 1 1 exponential form, 144, 1 5 1 , 264 higher dimensions, 249 in Hilbert space, 1 12 of dilation, 151 period 2p, 149 pointwise convergence, 184, 239, 246, 257 -

pointwise convergence for BV, 194 several variables , 253 speed of convergence, 222 table of convergence, 206, 249 termwise integration, 2 1 4 time-shift , 152 uniform convergence, 203, 206, 239, 300 Fourier sine series, 155 Fourier transform, 278, 289 conj ugates, 340 discrete, 385 finite, 144, 265 in f E L 2 (R) , 357 integrability of 323 inversion for BV , 294 of derivative, 334 of integral, 336 windowed, 480 frame, 465 and Riesz basis, 47 1 bounds, 465 dual, 468 exact, 465 inequality, 465 operator, 465, 467 tight, 465 Weyl-Heisenberg, 473 Franklin mother wavelet, 462 Franklin wavelets, 459 frequently, 45 Fubini theorem251

j,

G

Gab or transform, 480 Gamma function, 283 Gauss and FFT, 399 Gaussian kernel, 3 1 8 Gibbs phenomenon , 208 suppressed, 242 Gram-Schmidt, 2 1 Gregory series, 192

Index

H

Holder inequality , 29, 33 Haar functions, 1 15 density of in C [ 0, 1 ] , 1 18 density of in L2 [ 0, 1 ] , 1 1 9 Haar wavelets, 420 filter , 424 wavelets, 412 Hahn-Banach theorem, 131 half-range expansion, 155 Hardy-Landau theorem, 240 harmonics , 145 hat function, 307 Fourier transform, 282, 337 Hilbert cube, 60 compact , 74 Hilbert space, 5, 6 1 dual of, 132 Hilbert sum, 124 homeomorphism, 39, 80 linear, 80 Hunt , 194

I

-b ' f-a r b Ja- '

182 182 idempotent, 98 identity none for L1 (T) , 271 ( , } inner product, 13 indicator function, 6 , 1 14 inner product , 13 inner product isomorphic , 25 integral transform, 263 integration by parts , 168 interior of set , 58 interior point, 58 internal direct sum, 123 inversion theorem dominated, 320 Fejer-Lebesgue, 310, 316, 370 Fejer-Lebesgue for L2 functions, 370 for E L1 (R), 312 for L2 functions, 362, 368

j

50 1

for BV functions, 294 for DFT, 390 Fourier cosine transform, 350 Jordan, 299 trigonometric form, 298 invertible element, 398 isometric, 3 isometry , 3 isoperimetric inequality , 225

J

joint continuity of metric and inner product, 46 Jordan decomposition theorem, 165 pointwise convergence theo rem, 194, 240, 275, 299 K K=

R or C , 2

J{r (t) , continuous Fejer kernel, 306,

308 kernel, 127, 263 L

i2 , 5 ioo , 8 ip (N) , 10 ip ( n ) , 10 Lp (T, f.l ) , 32 L1 (T) , 266 L1 (R), 1 1 i2 = i2 ( 00 ) = i2 (N) , 5 L2 ( ,R) , only real in Ch. 4 . , 6 i2 ( 00 ) , 5 i2 ( n ) , 2 i2 (N) , 5 i2 ( Z) , 5 Lebesgue pointwise convergence theo rem, 246 point, 246 Leibniz 's rule, 335 l.i.m . , limit in the mean, 43

502

Index

limit in the mean , 43 linear homeomorphism, 80 linear isometry , 25 of Rn , 27 of C [a , b] , 27 linear isomorphism , 24 linear span, 1 9 linearly independent, 20 Lipschitz condition, 1 66, 186 uniform, 166 Lipschitz test, 186, 299 little oh notation, 222 localization principle, 1 85 low-pass filter , 432 (E) , 1 1 (R) , 1 1 (R) , real-valued, 1 1 (E) , real-valued, 1 1 [a, b] smaller for bigger 173 L (X, Y ) , 126

Lp Lp L; L; Lp

M

p,

mean approximation, 99 mean value theorem Bonnet , 193 first, 196 second, 193, 196 second = Bonnet , 1 93 Mellin transform, 375 Menshov, 249 metric , 1 bounded, 8 1 equivalent, 8 1 Euclidean, 2 Hamming, 8 max = sup , 8 product , 3 space, 1 taxicab , 4 trivial , 5 uniform continuity of, 68 mg , filter associated with g , 424 Minkowski inequality, 30, 33 monotone convergence theorem, 324 mother wavelet, 41 1 , 4 1 9

for MRA , 428 for Haar system, 1 15 MRA, 414 multiresolution analysis (MRA ) , 414 Riesz , 45 1

N

1 1 1 1 00 , 10 10 neighborhood basic, 35 of a point, 35 nondifferentiable function, 1 6 1 norm, 9 equivalent , 78 max, 12 of too ( n ) --+ too ( n ) , 129 of A : t2 ( n ) --+ t2 ( n ) , 129 of a linear map , 126 of a projection, 127 of integral operator , 1 27 reconstruct from unit ball, 37 stronger, 76 uniform, 1 1 weaker, 76 norm isomorphic, 25 norm preserving, 25 normalized Lebesgue measure , 264 normed space, 9 nowhere dense, 59 null sequence, 56 null space, 53 Nyquist interval, 374 rate , 374

II

li p ,

A:

o

lQ , indicator of rationals, 6 0, smaller order, 222 0, big oh, 223 odd function, 1 9 , 147 odd part of function, 399-400 ondelette, 419 open set, 58

Index

operators, 126 orthocomplement, 53, 95 in Hilbert Space , 96 properties, 95, 99 orthogonal, 18 complement , 53, 95 dimension, 107 direct sum, 123 projection , 98, 99 set , 18 orthonormal sequence, 18 set , 18 orthonormal basis, 107 for C [- 71" , 71"] , 1 13 for L1 (ZN ) , 391 for La ( [- 71" , 7I"] n ) , 252 for La ([a, b) x [c, d]) , 252 for La [0 , 71"] , 1 13 for X EEl Y , 1 13 for band-limi ted, 374 for fa and La[ - 7I", 7I"] , 1 1 1 for L a (R) , 413 from scaling function, 416 pasting together, 125 orthonormal wavelet basis, 419 orthonormality of translates, 422, 432

p PV f: ,

182 p-norm, lO equivalent in finite-dimensional case, 78 inequivalent , 79 P-summable , 227 pth power summable , 1 1 parallelogram law , 22 Parseval identity and ener�y , 152 for E Ll (R) , 329 for E Ll (R) n La (R) , 352 for Ll (T) , 266 for L2 functions, 358 for L2(ZN ) , 393

I, g, / I

503

in Hilbert space, 103-104, 106 108 on dense subset, 1 1 1 Parseval ' s equality, 103 periodic, 143 doubly, 250 extension, 143, 250 fundamental, 153 periodic extension odd, 155 periodic function integral, 156 periodogram, 394 'P, 2 1 Pickwickian, 226 piecewise continuous, 163 smooth, 163 Plancherel theorem, 362 transform, 357 pointwise convergence theorem de la VaJee-Poussin, 195 Dirichlet , 191 Fejer, 239 Jordan , 1 94, 240, 275 Lebesgue, 246, 274 multiple, 257 Poisson kernel, 241 polarization identity , 23 positive operator, 466 precompact, 70 principle of uniform boundedness, 211 product metrics, 3 projection onto subspace, 85 operator, 42 orthogonal, 98 uniform continuity of, 69 projection map , 46 projection theorem, 96 pseudometric, 7 Pythagorean theorem, 20, 23, 105

504

Index

Q

Q , rationals, 6

R

R, real numbers, 1 R + , nonnegative real numbers , 320 Rademacher functions, 109 rapidly decreasing, 363 rare set , 59 rearrangement of series, 1 19 reciprocity theorem, 36 1 reconstruction, 432 reflexive space, 134 RH ( 8 1 ) , 16 Riemann-Lebesgue lemma, 1 04 for E Ld-1r, 1r] , 106 for E L 1 [a, b] , 170 for E L 1 (R), 173 variations, 1 73 Riemann-Lebesgue property Dirichlet kernel, 179 Fourier kernel , 181 Riesz basis, 450 translates of hat function, 463 and orthonormal basis, 455 multiresolution analysis, 45 1 scaling function, 452 Riesz representation theorem, 132 Riesz-Fischer theorem, 1 04 right angle in inner product space, 1 6 right-handed derivative, 163

j j j

s

[8] , linear span of 8, 1 9

summable sequences , 27 8 1. , ortho complement , 53, 95 8 (R) , 363 sampling theorem, 373 saw-tooth, 149 scalar homogeneous, 133 scaling function, 415 compact support , 445 Riesz, 452 s,

scaling identity, 1 15, 425 Schauder base = basis, 89, 450 equivalent , 450 sectionally continuous, 163 self-adjoint, 135 separable iff complete orthonormal se quence , 1 12 space, 55 separable Hilbert space � £2 , 1 1 2 � £ 2 (Z) , 121 Shannon wavelets, 448 shift matrix, 395 short-time Fourier transform, 48 1 sgn t , signum function , 109 signum function, 109 simple function, 54 sine integral, 209 smaller order, 222 solving linear equations, 100 spanned, 19 spectral , 48 1 sphere , 35 spline, 476 basic, 476-477 square wave, 147-148 square-integrable functions, 6 square-summable functions, 6 sequences , 5 step function, 54 strict norm, 15 subspaces sums of, 1 0 1 summability kernel , 3 1 8 and norm convergence, 326 for L 2 functions, 371 8(x , r) , sphere, 35 supp j, 436 support, 435 T

T, circle group , 265

Index

Tauberian theorems, 234 taxicab metric , 4 time-limited, 281 , 290 Toeplitz matrix, 229 Toeplitz summable, 230 Tonelli theorem, 251 topological complement , 86 supplement , 86 product , 84 topology, 59 totally bounded, 70 transpose, 136 trigonometric polynomial , 1 5 , 1 1 1 density of, 54 trigonometric series, 144, 171, 197 not Fourier, 219

U

uncertainty principle, 482 U(t) , unit step function, 280 ultrametric, 8 Cauchy sequences in, 65 geometry , 37 unconditional convergence does not imply absolute, 121 implied by absolute, 120 implies absolute in 122 in Hilb ert space, 121 uniform convergence, 44 convexity, 47 norm, 1 1 , 44 uniform continuity on compact sets, 73 uniform convergence implies mean, 45 trigonometric series, 202 uniformly Cauchy, 63 continuous, 66 convergent , 63 convex, 47, 93 unit ball, 35 increasing with p, 37

Rn ,

505

shape, 36 unit step function, 280 unitarily equivalent , 25 unitary operator, 25 space, 2 unitary Fourier transform, 363, 366

V

vanish at 00 , 290 variation, 164 bounded, 164 total, 1 64 variation function, 167

W

wavelet, 4 1 1 , 419 Battle-Lemarie, 478 transform, 480, 483-484 Franklin , 459 Shannon, 448 Weierstrass M-test, 64 nondifferentiable function , 1 6 1 theorem, 244 W-H frame, 473 Weyl-Heisenberg frame , 473 WFT, 480 window function, 480, 485 center, radius, width, 48 1 windowed Fourier transform, 480

x

y,

x 1.. 18 X Ef) Y , direct sum, 83 X' , dual space, 131

z

Z, integers , 5 ZN = Zj (N) , 385

Fourier And Wavelet Analysis

Read more

Fourier and wavelet analysis

Read more

Fourier analysis and wavelet

Read more

Fourier and wavelet analysis

Read more

Fourier and Wavelet Analysis (Universitext)

Read more

Wavelet analysis and applications

Read more

Fourier analysis

Read more

Fourier analysis

Read more

Fourier analysis and convexity

Read more

Fourier Analysis and Convexity

Read more

Fourier analysis and approximation

Read more

Fourier analysis

Read more

Fourier Analysis

Read more

Fourier Analysis

Read more

Fourier Analysis

Read more

Fourier analysis

Read more

Fourier Analysis and Its Applications

Read more

Fourier Analysis and Its Applications

Read more

Fourier analysis and its applications

Read more

Fourier Analysis and Its Applications

Read more

Fourier Analysis and Its Applications

Read more

Fourier Analysis and Its Applications

Read more

Fourier analysis and its applications

Read more

Fourier Analysis and Its Applications

Read more

Fourier Analysis and Its Applications

Read more

Fourier analysis and its applications

Read more

Fourier Analysis and Its Applications

Read more

Fourier analysis and its applications

Read more

Microlocal Analysis and Complex Fourier Analysis

Read more

Fourier Analysis and its Applications

Read more

Recommend Documents

Fourier And Wavelet Analysis

Fourier and wavelet analysis

Fourier analysis and wavelet

walker.qxp 3/24/98 2:16 PM Page 658 Fourier Analysis and Wavelet Analysis James S. Walker I n this article we will co...

Fourier and wavelet analysis

Fourier and Wavelet Analysis (Universitext)

...

Wavelet analysis and applications

Applied and Numerical Harmonic Analysis Series Editor John J. Benedetto University of Maryland Editorial Advisory Boar...

Fourier analysis

Ibookroot October 20, 2007 FOURIER ANALYSIS Ibookroot October 20, 2007 Princeton Lectures in Analysis I Fourier A...

Fourier analysis

Fourier analysis and convexity

Fourier Analysis and Convexity

Applied and Numerical Harmonic Analysis Series Editor John J. Benedetto University of Maryland Editorial Advisory Board...