Real Mathematical Analysis

then I f I

E n.

Proof The function
�--+

I f (x) I is Riemann D

Riemann Integration

Section 2

1 69

If a < c < b and f : [a , b] ---+ ffi. is Riemann integrable then its restrictions to [a , c], [c, b] are Riemann integrable, and

28 Corollary

1b

f (x) dx =

1 f(x) dx + [ f(x) dx . b

c

Conversely, Riemann integrability on [a , c] and [c, b] implies Riemann integrability on [a , b ]. Proof See Figure 7 4. The union of the discontinuity sets for the restrictions of f to the subintervals [a , c], [c, b] is the discontinuity set of f. The latter is a zero set if and only if the former two are, and so by the Riemann­ Lebesgue Theorem, f is Riemann integrable if and only if its restrictions to [a . c], [c, b] are. Let X ra .cJ • X [c,bl be the characteristic functions of [a , c ], [c, b ] . By Corol­ lary 23 they are integrable, and by Corollary 25, so are the products X [a .cl · f and X [c.bl • f. Since

f=

X ra .c]

· f + X (c , b] • f,

the addition formula follows from linearity of the integral, Theorem

a

c

16. 0

b

Figure 74 Additivity of the integral is equivalent to additivity of area.

29 Corollary If f : [a , b] ---+ [0, M] is Riemann integrable and has inte­ gral zero then f (x) = 0 at every continuity point x of f. That is, f (x) = 0 almost everywhere.

Proof Suppose not: let x0 be a continuity point of f and assume that f(xo) > 0. Then for some 8 > 0 and each x E (x0 8 , x0 + 8 ) ,

f (x)

?:

-

f (xo)/2. The function f (xo) g (x) = 2 0

I

if X

E (Xo - 8 , Xo + 8 )

otherwise

Functions of a Real Variable

1 70

Chapter 3

satisfies 0 ::=:: g (x) ::=:: f(x) everywhere. See Figure 75. By monotonicity of the integral, Theorem 1 7 ,

f(xo) fl =

1b g (x) dx 1b j ( ) dx ::=::

a contradiction. Hence f (x ) =

x

= 0, 0

0 at every continuity point.

graph g

b

a

Figure 75 The shaded rectangle prevents the integral of f being zero. Corollary 26 and Exercises 34, 36, 46, 48 deal with the way that Riemann integrability behaves under composition. If f E R and ¢ is continuous then ¢ o f E R, although the composition in the other order, f o ¢, may fail to be integrable. Continuity is too weak a hypothesis for such a "change of variable." See Exercise 36. However, we have the following result.

Iff is Riemann integrable and l/1 is a bijection whose inverse satisfies a Lipschitz condition then f o l/1 is Riemann integrable. 30 Corollary

Proof More precisely, we assume that f : [a , b] grable, l/1 bijects [c, d] onto [a, constant K and all s , t E [a , b],

b], lfr (c)

=

---+

ffi. is Riemann inte­ and for some

a , l/J (d) = b,

l lfr - 1 (s) - l/l - 1 (t) l ::::: K Is - t ! .

We then assert that f o l/1 is a Riemann integrable function [c, d] ---+ R Note that lfr - 1 is a homeomorphism. For it is a continuous bijection whose domain of definition is compact. Let D be the set of discontinuity points of f. Then D ' = l/f - 1 (D) is the set of discontinuity points for f o l/J. Let E > 0 be given. There is an open covering of D by intervals (a; , b; ) whose total length is ::=:: E K . The homeomorphic intervals (a; , b; > = lfr - 1 (a; , b; ) cover D ' and have total length

I

".t b�t - a�z < " < L..t K (b·z - a z· ) _ L.

E.

Therefore D ' is a zero set and by the Riemann Lebesgue Theorem, is integrable.

f o l/1

0

17 1

Riemann Integration

Section 2 31

Corollary If f E R and 1/f : [c, d] then f o 1/f is Riemann integrable.

---+

La . b] is a C 1 diffeomorphism

Proof The hypothesis that 1/f is a C 1 diffeomorphism means that it is a

continuously differentiable bijection, whose derivative is nowhere zero. We assume that 1/f ' (t) > 0. Since 1/f ' is continuous and positive on [c, d] there is a constant K > 0 such that for all 0 E [c, d], 1/f'(O ) =:: K . The Mean Value Theorem implies that for all u , v E [c, d], there exists a 0 between u and v such that 1/f (u) - 1/f (v) = 1/f ' (O ) (u - v ) . Thus,

For any

s, t

e

1 1/f (u ) - 1/f (v) l � K l u - v i . [a , b], set u = 1/f - 1 (s) , v = 1/f - 1 (t). Then I s - t i � K J l/f- 1 (s) - 1/f - 1 (t) J ,

which is a Lipschitz condition on 1/f - 1 with Lipschitz constant K = K - 1 • By Corollary 30, f o 1/f is Riemann integrable. D Versions of the preceding theorem and corollary remain true without the hypotheses that 1/f bijects. The proofs are harder because l/t can fold infinitely often. See Exercise 39. In calculus you learn that the derivative of the integral is the integrand. This we now prove.

32 Fundamental Theorem of Calculus If f : [a . b]

lx f {t) dt

integrable then its indefinite integral F (x)

=

---+

lR

is Riemann

is a continuousfunction ofx. The derivative of F (x) exists and equals f (x) at all points x at which f is continuous. Proof #1 Obvious from Figure 76. Proof #2 Since f is Riemann integrable, it is bounded; say for all x . By Corollary 28 I F (y) - F (x) l =

l l y f (t) dt l .s:

M ly -

l f (x) l

_::::

M

xl .

Therefore, F is continuous : given E > 0, choose o < E f M, and observe that I Y - x i < o implies that I F (y) - F (x) l < Mo < E . In exactly the same way, if f is continuous at x then

Chapter 3

Functions of a Real Variable

172

F(xJ

X

a

h

x+h

Figure 76 Why does this picture give a proof of the Fundamental Theorem of Calculus?

F (x + h) - F (x) h

as

h

---+ 0. For if

=

!._h

l X

x

+

h

f (t) d t

---+

f (x)

m (x , h) = inf { f (s ) : Is - x l s l h l } M (x, h) = sup{ f (s) : Is - x i s lh l }

then

m (x , h)

= s

1

h ]

h

lx+h m (x , h) dt 1 lx+h f (t) dt lx+h M(x . h) t M (x , h) . s

x

h

x

d =

x

When f is continuous at x, m (x , h) and M(x , h) converge to h ---+ 0, and so must the integral sandwiched between them,

1

h (If h

1x+h f(t) dt. X

< 0 then t fx +h f (t) dt is interpreted as - t J:+h f(t) dt .) x

f (x)

as

D

The derivative ofan indefinite Riemann integral exists almost everywhere and equals the integrand almost everywhere.

33 Corollary

Riemann Integration

Section 2

173

Proof Assume that f : [a , b] ---+ JR. is Riemann integrable and F (x) is its

indefinite integral. By the Riemann-Lebesgue Theorem, f is continuous almost everywhere, and by the Fundamenta1 Theorem of Calculus, F ' (x) exists and equals f (x) wherever f is continuous. D A second version of the Fundamenta1 Theorem of Calculus concerns an­ tiderivatives. lf one function is the derivative of another, the second function is an antiderivative of the first.

Note When G is an antiderivative of of g : [a , b] ---+ JR., we have

G ' (x) = g (x) for all x E

[a , b], not merely for almost all x E [a , b] .

34 Corollary

Every continuous function has an antiderivative.

Proof Assume that f : [a , b] ---+ JR. is continuous. By the Fundamental Theorem of Calculus, the indefinite integral F (x) has a derivative every­ D where, and F ' (x) = f (x) everywhere. Some discontinuous functions have an antiderivative and others don't. Surprisingly, the wildly oscillating function

f (x)

=

{ 0. if x ::::: 00 --{ oi 00 rr

sm ­

x

if X >

has an antiderivative, but the j ump function

g (x) does not. See Exercise 42.

if X :::::

if x >

35 Antiderivative Theorem An antiderivative of a Riemann integrable function, if it exists, differs from the indefinite integral by a constant.

Proof We assume that f : [a , b] ---+ JR. is Riemann integrable, that G is an antiderivative of f , and we assert that for all x E [a , b],

G (x) =

{x f (t ) dt

+

C,

Functions of a Real Variable

1 74

where C is a constant. (In fact, C

a = Xo < and choose

tk E [xk - I . xk]

Chapter 3

= G (a) .) Partition [a , x]

X1

< · · · < Xn

as

= X,

such that

Such a tk exists by the Mean Value Theorem applied to the differentiable function G. Telescoping gives

G (x) - G (a)

=

n

n

k=l

k =l

L G (xk ) - G (xk - l ) = L f (tk ) D. xk ,

which i s a Riemann sum for f on the interval [a , x ] . Since f is Riemann integrable, the Riemann sum converges to F (x) as the mesh of the partition tends to zero. This gives G (x) - G (a) = F (x) as claimed. D

36 Corollary

are valid.

lb

Standard integral formulas, such as 2 b3 - a 3 x dx = , 3

a

Proof Every integral formula is actually a derivative formula, and the Anti­ derivative Theorem converts derivative formulas to integral formulas.

lx -1 dt .

D

In particular, the logarithm function is defined as the integral, log x

=

1

t

Since the integrand 1 j t is well defined and continuous when t > 0, log x is well defined and differentiable for x > 0. Its derivative is l j x . By the way, as is standard in post calculus vocabulary, log x refers to the natural logarithm, not to the base 1 0 logarithm. See also Exercise 1 6. An antiderivative of f has G ' (x) = f(x) everywhere, and differs from the indefinite integral F (x) by a constant. But what if we assume instead that H ' (x) = f (x) almost everywhere? Should this not also imply H (x) differs from F (x) by a constant? Surprisingly, the answer is "no."

There exists a continuous function H : [0, 1 ] ---+ lR whose derivative exists and equals zero almost everywhere, but which is not con­ stant. 37 Theorem

Riemann Integration

Section 2

175

Proof. The counter-example is the Devil's staircase function, also called the Cantor function. It is defined as H (x ) -

1 /2

if X E [ l j3 , 2 j 3]

1 /4

if X E [ l j9 , 2j9]

3j4

if X E [7 /9 , 8j 9 ]

See Figures 77, 78.

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·

3 4

2 I

· · · · · · · · · · · · · ·

I

4

· · · ·

�---�

�-

9 I

0

�

9 2

3

3

9

2

I

7

9 8

Figure 77 The devil' s staircase. On each discarded interval in the Cantor set construction, H is constant. Thus H is differentiable with derivative zero at all points of [0, 1 ] \ C, and since C is a zero set, this implies H ' (x) = 0 for almost every x . To show that H is continuous we use base 2 and base 3 arithmetic. [f x E [0, 1 ] has base 3 expansion x = (.x1x2 h then the base 2 expansion of y = H (x) is (-YI Y2 . . . h where •

Yi =

��

Xi 2

if 3k


such that Xk

•

•

=1

= 1 and !Jk < i such that xk = 1 if xi = 0 or xi = 2 and !Jk i such that Xk = 1 .

if Xi

<

You should ask: is this a valid definition of H (x)? If two different base 3 expansions, (.x1x2 h and (.x;x2 . . . h represent the same number x , do the two base 2 expressions for (x) represent the same number y? Two •

•

•

H

1 76

Functions of a Real Variable

Chapter 3

]

Figure 78 The devil' s undergraph.

base 3 expansions of x represent the same number x if and only if x is an endpoint of C; one of its base 3 expansions ends in O's, the other in 2's. For example, (.x 1 x2 . . . xeOZh = x = ( . x , x2 . . . xe lOh . If for some (smallest)

k

::::: e , xk =

(-YI Y2 · · - h =

I then

(" 2 2 X r X2

.

.

)

Xk - 1 . -- l O 2 2 -

unambiguously. If none of Xk with k :::: f equals expansions corresponding to H (x ) are

1

then the two base 2

(.� � . x�, ) ( . - - . . . 1 0) . oT

. .

X J x2

Xe - !

-

2 2

-

2

2

2

These two base 2 expansions represent the same number y . The same rea­ soning applies to the only other ambiguous case, (.xr x2 . . . xe lZh =

x

= (.x1 x2

. . . xe20h .

The point y = H (x) is well defined. Continuity is now easy to check. Let E > be given. Choose k such that 1 /2k :::: E . If j x - x ' < I j3 k then there are base 3 expansions of x , x '

0

i

Riemann Integration

Section 2

whose first k symbols agree. Therefore the first k symbols of agree, which implies that

H (x')

I H (x )

- H ( x ' ) l�

2

LI < E .

1 77 H

(x )

and

0

A yet more pathological example is a strictly monotone, continuous func­ tion whose derivative is almost everywhere zero. Its graph is a sort of Devil's ski slope, almost everywhere level but also everywhere downhill. To construct start with and extend it to a function fi : JR. --+ JR. by setting + n) = + n for all n E z and all X E ro. I ] . Then set

J

H(x

J,

H (x)

H

00 H k J (x ) = L (34k x) . k=O The values of fi (3 k x) for x E [0 , 1 ] are� 3 k , which is much smaller than the denominator 4k . Thus the series converges and J (x) is well defined. According to the Weierstrass M -test, proved in the next chapter, J is con­ tinuous. Since fi (3 k x) strictly increases for any pair of points at distance > l f 3 k apart, and this fact is preserved when we take sums, J strictly in­ creases. The proof that J'(x) = 0 almost everywhere requires more and deeper theory. Next, we justify two common methods of integration.

If f E R and g : [ c, d ] --+ [a , b] is a continuously differentiable bijection with g ' > 0 ( g is a C 1 diffeomorphism) then f (y) dy = f (g(x))g ' (x) dx.

38 Integration by substitution

lb

[d

Proof The first integral exi sts by assumption. By Corollary 3 1 , the com­ posite f o g E R, and since g ' is continuous, the second integral exists by Corollary 25 . To show that the two integrals are equal we resort again to Riemann sums. Let P partition the interval [ c, dl as C =

and choose

tk E

Xo < X I

<

· · ·

<

Xn = d

Lxk -I · Xk] such that

The Mean Value Theorem ensures that such a tk exists. Since g is a diffeo­ morphism we have a partition Q of the interval [a , b]

a

=

Yo < Y I

< ··· <

Yn = b

Functions of a Real Variable

1 78

Chapter 3

where Yk = g (xk). and II P I I � 0 implies that II Q II � 0. Set sk = g (td. This gives two equal Riemann sums n

n

k=l

k=l

which converge to the integrals J: f (y ) dy and fcd f (g (t)g'(t) dt as II P II � 0. Since the limits of equals are equal, the integrals are equal. D Actually, it is sufficient to assume that g ' 39 Integration by Parts

l f (x)g' (x) dx

R then

b

Iff, g : [a , b]

E

R.

� JR. are differentiable and f ' ,

= f(b)g(b) - f (a)g(a) -

l f' (x)g(x) dx .

g' E

b

Proof Differentiability implies continuity implies integrability, so J, g E R. Since the product of Riemann integrable functions is Riemann in­ tegrable, f' g , fg ' E R, and both integrals exist. By the Leibniz Rule, (fg)'(x) = f(x)g'(x) + J '(x)g(x ) everywhere. That is, fg is an an­ tiderivative of f ' g + fg ' . The Antiderivative Theorem states that fg differs from the indefinite integral of f ' g + fg ' by a constant. That is, for all t E [a , b] ,

f (t)g (t) - f(a)g(a) = = Setting t = b gives the result.

11 f' (x)g (x) + f(x) g' (x) dx l J' (x)g(x) dx lr f (x)g' (x) dx . +

r

D

Improper Integrals

Assume that f : [a , b) � JR. is Riemann integrable when restricted to any closed subinterval [a , c] c [a , b) . You may imagine that f (x) has some unpleasant behavior as x � b, such as lim supx �b 1 / (x) l = oo and/or b = oo . See Figure 79. If the limit of J: f (x) dx exists (and is a real number) as c � b then it is natural to define it as the improper Riemann integral

l f (x) dx b

a

= lim

c �b

l f (x) dx. c

a

The same idea works of course on an interval to the left of a . In order that the two sided improper integral exists for a function f : (a , b) � JR. it is

Section

3

Series

1 79

I

a

Figure 79 The improper integral converges if and only if the total undergraph area is finite. natural to fix some point m E (a , b) and require that both improper inte­ grals fam f (x) dx and J� f (x) dx exist. Their sum is the improper integral J: f (x) dx. With some ingenuity you can devise a function f : IR --+ IR whose improper integral J� f (x) dx exists despite the fact that it is un­ bounded at both ±oo. See Exercise 7 1 .

3

Series

A series is a formal sum L ak where the terms ak are real numbers. The n th partial sum of the series is

The series converges to A if A n

--+

A as

n

--+ oo, and we write

A series that does not converge diverges. The basic question to ask about a series is: does it converge or diverge?

Functions of a Real Variable

1 80

Chapter 3

< 1 then the geometric series

For example, if A is a constant and I A I 00

L Ak = 1 + ). + . . . + A n + . . .

k =O

converges to L / ( 1 - 'A). For its partial sums are

An = 1 + A + A 2 +

·

·

·

n 1 - ). + l n + A = --1 - 'J....

n k and ). + 1 -+ 0 as n -+ oo. On the other hand, if l A I :::: 1 , the series L A diverges. Let L an be a series. The Cauchy Convergence Criterion from Chapter 1 applied to its sequence of partial sums yields the CCC for series

L ak converges if and only if VE

>

0 3N such that m ,

n

:::: N

==}

One immediate consequence of the CCC is that no finite number of terms affects convergence of a series. Rather, it is the tail of the series, the terms ak with k large, that determines convergence or divergence. Likewise, whether the series Leads off with a term of index k = 0 or k = 1 , etc. is irrelevant. A second consequence of the CCC is that if ak does not converge to zero as k ---+ oo then L ak does not converge. For Cauchyness of the partial sum sequence (An) implies that an = An - An- I becomes small when n -+ oo. If I A I � 1 the geometric series L ). k diverges since its terms do not converge to zero. The harmonic series

gives an example that a series can diverge even though its terms do tend to zero. See below. Series theory has a large number of convergence tests. All boil down to the following result . 40 Comparison Test Ifa series L bk dominates a series L ak

thatfor all sufficiently large k, l ak I convergence of L ak .

�

in the sense

bk. then convergence of'L bk implies

181

Senes

Section 3

Proof Given E > 0 , convergence o f L bk implies there i s a large N such

that for all m , n 2: N , L�=m bk n

< E . Thus n

n

and convergence of L ak follows from the CCC.

0

Example The series L sin(k) j2k converges since it is dominated by the geometric series L 1 j2 k . A series L ak converges absolutely if L lak I converges. The comparison test shows that absolute convergence implies convergence. A series that converges but not absolutely converges conditionally: L ak converges and L lak I diverges. See below. Series and integrals are both infinite sums. You can imagine a series as an improper integral in which the integration variable is an integer,

More precisely, given a series L ak o define f

f(x) See Figure 80. Then

=

ak tf k - 1 < x

00

:L ak = k =O

:

[0, oo) _:::

1 00 f (x) dx .

---7-

lR by setting

k.

O

The series converges if and only if the improper integral does. The natural extension of this picture is the

41 Integral Test Suppose that and and L ak is a given series.

J000 f(x) dx is a given improper integral

(a) If lak l .::S f (x) for all sufficiently large k and all x E (k - 1 , k] then convergence of the improper integral implies convergence of the series. (b) lf l f (x) l .::S ak for all sufficiently large k and all x E [k , k + 1 ) then divergence of the improper integral implies divergence of the series.

Functions of a Real Variable

1 82

0

2

4

5

k- l

0

2

4

5

k

Chapter 3

k+ l

k+ l

k+2

Figure 8 0 The pictorial proof o f the integral test.

Proof See Figure 80. (a) For some large N0 and all N 2: N0 we have

t

��+ 1

iak l

::=::

1 N f(x) dx 1 00 f (x) dx , ::=::

�

0

which is a finite real number. An increasing, bounded sequence converges to a limit, so the tail of the series L la k l converges, and the whole series L lak I converges. Absolute convergence implies convergence. 0 The proof of (b) is left as Exercise 73.

Example The p-series, L 1 j kP converges when p > 1 and diverges when p ::::: l . Case 1 . p > 1 . By the fundamental theorem of calculus and differentiation rules b 1 -P - I l fb 1 = )1 xP 1 - p -----* p - I as b ---+ oo . The improper integral converges and dominates the p-series, which implies convergence of the series by the integral test. Case 2. p ::=:: 1 . The p-series dominates the improper integral h

1 dx { lt x P

=

{

log b

b 1 -P -

if p = 1 1

if p < 1 . 1-p As b -----* oo, these quantities blow up, and the integral test implies divergence of the series. When p = 1 we have the harmonic series, which we have just shown to diverge.

Series

Section 3

1 83

The exponential growth rate of the series L ak is ot

= lim sup � . k-+ oo

42 Root Test Let a be the exponential growth rare of a series L ak . If

< 1 the series converges. root test is inconclusive.

ot

Proof If ot

if ot

> I the series diverges. and if ot =

1

the

< 1 , fix a constant {3 , ot < f3 < 1 .

Then for all large k, l ak i i / k ::::: {3 ; i.e.,

which gives convergence of L ak by comparison to the geometric series k L: P . If a >

1 , choose {3 , 1 < f3

<

ot.

Then l ak l :::: pk for infinitely many k. Since the terms ak do not converge to 0, the series diverges. To show that the root test is inconclusive when ot = 1, we must find two series, one convergent and the other divergent, both having exponential growth rate ot = 1 . The examples are p-series. We have log

( -1 �

) I fk

=

- p log(k)

k

'"""

- p log(x) X

,....,

- p ix I

__

,...., 0

by L'Hospital's rule as k = x ---+ oo . Therefore a = limk -+ oo O i k P ) 1 f k = 1 . Since the square series L 1 I k2 converges and the harmonic series L 1 I k diverges the root test is inconclusive when ot = I . 0

43 Ratio Test Let the ratio beTWeen successive Terms of the series L ak be rk = l a k+ I f ak l . and set

li m sup r = p .

lim inf rk = A k � oo

If p < 1 the series converges, the ratio test is inconclusive.

if A

k --'> 00

>

k

1 the series diverges, and otherwise

Functions of a Real Variable

1 84

Proof lf p i.e. ,

<

l , choose f3 , p

<

f3

<

Chapter 3

l . For all k � some K Iak + I I ak l < f3 ; .

k k i ak i :S f3 - K i a K I = Cf3 where C 13 -K l a K I is a constant. Convergence of L ak follows from comparison with the geometric series L Cf3 k . If A > 1 , choose /3, 1 < f3 < k A . Then lak l � f3 I C for al1 1arge k, and L ak diverges because its terms do not converge to 0. Again the p-series all have ratio limit p = A = 1 and demonstrate the inconclusiveness of the ratio test when p = 1 or A = 1 . D

Although it is usually easier to apply the ratio test than the root test, the latter has a strictly wider scope. See Exercises 56, 60. Conditional Convergence

If ( ak) is a decreasing sequence in lR that converges to 0 then its alternating series converges. For, A 2n = (a l - a 2 ) + (a3 - a4 ) + . . (a 2n- 1 - a 2n ) . and ak- J - ak is the length of the the interval h = (ak . ak - l ) . The intervals h are disjoint, so the sum of their lengths is at most the length of (0, a0) , namely a0 . See Figure 8 1 . .

0

Figure 81 The pictorial proof of alternating convergence. The sequence ( A 2n ) is increasing and bounded, so limn oo A 2n exists. The partial sum A 2n+ 1 differs from A 2n by a 2n + 1 , a quantity that converges to 0 as n -+ oo , so A2n = nlim A2n+l • nlim --). oo �oo and the alternating series converges. When ak = 1 I k we have the alternating harmonic series, 00 k 1 I I (- 1 ) + l 1 +3-4+... .L k = 2 k =l which we have just shown is convergent. --->

1 85

Series

Section 3 Series of Functions

A series of functions is of the form 00

where each term A series

:

(a , b )

�

lR is a function. For example, in a power

the functions are monomials ck x k . (The coefficients ck are constants and x is a real variable.) [f you think of )... = x as a variable, then the geometric series is a power series whose coefficients are 1 , L x k . Another example of a series of functions is a Fourier series

L ak sin(kx) + bk cos (kx) . 44 Radius of Convergence Theorem If L ckx k is a power series then there is a unique R, 0 :S R :S oo, its radius of convergence, such that the series converges whenever lx I < R, and diverges whenever lx I > R. Moreover R is given by the fonnula

R-

1

lim sup V"'cJ .

----===

k--+ 00

Proof Apply the root test to the series L ck x k . Then lim sup k--+ oo

If l x I

<

.j J ckx k J = l x l lim sup V"'cJ = �R k--+ oo

R the root test gives convergence. [f lx I > R it gives divergence.

D

There are power series with any given radius of convergence, 0 :S R :S = 0. The series L x k I a k has The series L kk x k has = a for k 0 < a < oo. The series L x I k ! has R = oo. Eventually, we show that a function defined by a power series is analytic: it is has all derivatives at all points and it can be expanded as a Taylor series at each point inside its radius of convergence, not merely at x = 0. See Section 6 in Chapter 4. oo.

R

R

1 86

Functions of a Real Variable

Chapter 3

Exercises

f : � ---+ � satisfies 1 / (t) - f (x) l :::::: I t - x l 2 for all t, x . Prove that f is constant. 2. A function f : (a , b) ---+ � satisfies a HOlder condition of order a if a > 0, and for some constant H and all u , x E (a , b), l f (u) - f(x ) l :::::: H l u - x l a . 1 . Assume that

The function is said to be a-HOlder, with a-HOlder constant H . (The terms "Lipschitz function of order a" and "a-Lipschitz function" are sometimes used with the same meaning.) (a) Prove that an a-Holder function defined on (a . b) is uniformly continuous and infer that it extends uniquely to a continuous function defined on [a . b J . Is the extended function a-HOlder? (b) What does a-Holder continuity mean when a = 1 ? (c) Prove that a-Holder continuity when a > I implies that f is constant. 3. Assume that f : (a . b) ---+ � is differentiable. (a) If f' (x) > 0 for all x , prove that f is strictly monotone increas­ ing. (h) If f'(x) =:: 0 for all x , what can you prove? 4. Prove that .Jn+1 - Jn ---+ 0 as n ---+ oo. 5 . Assume that f : � ---+ � is continuous, and for all x "1- 0, f'(x) exists. H lim ! ' (x) = L exists, does it follow that f' (0) exists? x� o

6.

7.

8.

9.

Prove or disprove. If a differentiable function f : (a , b) ---+ � assumes a maximum or a minimum at some () E (a , b) , prove that j'(()) = 0. Why is the assertion false when [a , b ] replaces (a , b ) ? In L'Hospital 's Rule, replace the interval (a , b) with the half-line [a , oo) and interpret "x tends to b" as "x ---+ oo." Show that if fi g tends to 010 and f ' I g ' tends to L then fIg also tends to L also. Prove that this continues to hold when = oo in the sense that if ! ' I g' ---+ oo then fIg ---+ oo. In L' Hospital 's Rule, replace the assumption that fi g tends to 010 with the assumption that it tends to oo I oo. H f' I g' tends to L, prove that fl g tends to L also. [Hint: Think of a rear guard instead of an advance guard.] [Query: Is there a way to deduce the ooloo case from the 010 case? Na ;tvely taking reciprocals does not work.] (u) Draw the graph of a continuous function defined on [0, 1] that is differentiable on the interval (0, 1 ) but not at the endpoints.

L

Exercises

1 87

(b) Can you find a formula for such a function? (c) Does the Mean Value Theorem apply to such a function? 1 0. Let f : (a , b) � lR be given. (a) If exists, prove that

f"(x)

r

h1_!?1

f(x - h) - 2f(x ) + f(x + h) - f" (X ) h2

.

(b) Find an example that this limit can exist even when (x) fails to exist. 1 1 . Assume that ( - 1 , 1) � lR and exists. If an , fJn -+ 0 as n � oo, define the difference quotient

f"

f:

f'(O)

Dn

=

J (fJn

) - J (an )

fJn - CXn

.

(a) Prove that lim Dn = f ' (0) under each of the following conn -4 oo ditions (i) CX n < 0 < fJn · (ii) 0

<

CXn

<

fJn and

fJn :S fJn - CXn

M.

(iii) j' (x) exists and is continuous for al] x E (- 1 , 1 ) . = 0 . Observe that = x 2 sin ( l jx) for x i= 0 and (b) Set is differentiable everywhere in ( - 1 , 1 ) and (0) = 0. Find an , fJn that tend to 0 in such a way that Dn converges to a limit unequal to / ' (0) . Assume that and are rth order differentiable functions (a , b) � JR, r :=: I . Prove the rth order Leibniz product rule for the function

f(x )

f(O)

f

1 2.

f'

f

f · g,

g

(f . g )(r)(x) = t (�) f (k) (x ) . g (r -k)(x ) . k=O

where (�) = r ! j (k ! (r - k) !) is the binomial coefficient, r choose k. [Hint: Induction.] 1 3. Assume that lR � lR is differentiable. (a) If there is an L < 1 such that for each x E JR, (x) < L, prove that there exists a unique point x such that = x . [x is a fixed point for (b) Show b y example that (a) fails i f L = 1 . 1 4 . Define lR � lR by

f:

f' f(x)

e:

f.]

e(x )

=

{eo-I /x

if > 0 if X :S 0

x

Functions of a Real Variable

1 88

Chapter 3

(a) Prove that e is smooth: that is, e has derivatives of all orders at all points x . [Hint: L' Hospital and induction. Feel free to use the standard differentiation formulas about ex from calculus.] (b) Is e analytic? (c) Show that the bump function

fJ (x) = e 2 e ( I - x) · e(x + 1 )

is smooth, identically zero outside the interval ( - 1 . 1 ) , positive inside the interval ( - 1 1 ) , and takes value 1 at x = 0. (e 2 i s the square of the base of the natural logarithms, while e(x) is the function just defined. Apologies to the abused notation.) (d) For l x l < 1 show that ,

fJ (x) = e - 2x2f (x2 - l ) .

Bump functions have wide use in smooth function theory and differ­ ential topology. The graph of /3 looks like a bump. See Figure 82.

-I

--{).5

0

0.5

Figure 82 The graph of the bump function /3 . * * 1 5 . Let L be any closed set in JR. Prove that there i s a smooth function f : :IR -+ [0, 1 ] such that f (x) = 0 if and only if x E L . To put it another way, every closed set in IR is the zero locus of some smooth function. LHint: Use Exercise 1 4(c).] 16. log x is defined to be ft l f t dt for x > O. Using only the mathematics explained in this chapter.

Exercises

17. 1 8.

1 89

(a) Prove that log is a smooth function. (b) Prove that log(xy) = log x + log y for all x , y > 0. [Hint: Fix y and define f (x ) = log(xy) - log x - log y . Show that f (x) = 0.] (c) Prove that log is strictly monotone increasing and its range is all of R Define f (x) = x 2 if x < 0 and f (x) = x + x 2 if x :::_ 0. Differenti­ ation gives f" (x) = 2. This is bogus. Why? Recall that the K-oscillation set of an arbitrary function f : [a , b] --+ lR is

DK

=

{x E [a , b] : oscx CJ)

:::_

K}.

(a) Prove that DK i s closed. (b) Infer that the discontinuity set of f is a countable union of closed sets. (This is called an Fa -set.) (c) Infer from (b) that the set of continuity points is a countable intersection of open sets. (This is called a G8-set.) * 1 9. Baire's Theorem (page 243) asserts that if a complete metric space is the countable union of closed subsets then at least one of them has non-empty interior. Use Baire's Theorem to show that the set of irrational numbers is not the countable union of closed subsets of R 20. Use Exercises 1 8 and 19 to show that there is no function f : lR --+ lR which is discontinuous at every irrational number and continuous at every rational number. **2 1 . Find a subset S of the middle-thirds Cantor set which is never the discontinuity set of a function f : lR --+ R Infer that some zero sets are never discontinuity sets of Riemann integrable functions. [Hint: How many subsets of C are there? How many can be countable unions of closed sets?] **22. Suppose that fn : la , b] --+ lR is a sequence of continuous functions that converges pointwise to a limit function f : fa , b] --+ R Such an f i s said to be of Baire class 1 . (Pointwise convergence is discussed in the next chapter. It means what it says: for each x , fn (x) converges to f (x ) as n --+ oo . Continuous functions are considered to be of Baire class 0, and in general a Baire class k function is the pointwise limit of a sequence of Baire class k - 1 functions. Strictly speaking, it should not be of B aire class k - 1 itself, but for simplicity I include continuous functions among B aire class 1 functions. It is an interesting fact that for every k there are Baire class k functions not of Baire class k - I . You might consult A Primer of Real Functions by Ralph B oas.)

Functions of a Real Variable

1 90

Chapter 3

Prove that the K-oscillation set of f is nowhere dense, as follows. To arrive at a contradiction, suppose that DK is dense in some interval (a , {3) c [a , b]. By Exercise 1 8, DK is closed, so it contains (ex , {3 ) . Cover lR b y countably many intervals (a e , ht ) of length < K and set Hf. = fpre (af. , bt ) .

(a) Why does U e He = [a , b ] ? (b) Show that no Hl contains a subinterval of (ex , {3 ) . (c) Why are Ff.m n = {x E [a , b] : al + EtmN =

n Fem n

1

m

< fn (X )

<

bl

1

- -} m

n�N

closed? (d) Show that

U EemN · m. NEN (e) Use (a) and Baire's Theorem (page 24 3 ) to deduce that some EtmN contains a subinterval of (a, {3 ) . ( f) Why does (e) contradict (b) and complete the proof that DK is nowhere dense? 23. Combine Exercises 1 8 , 22, and Baire's Theorem to show that a Baire class 1 function has a dense set of continuity points. 24. Suppose that g : [a . b] ---+ lR is differentiable. (a) Prove that g' is of Baire class 1 . [Hint : Extend g to a differen­ tiable function defined on a larger interval and consider He =

fn (X ) =

g (x + 1 / n ) - g (x ) 1 /n

for x E [a . b ] . Is fn (x ) continuous? Does fn (x ) converge point­ wise to g' (x ) as n ---+ oo?] (b) Infer from Exercise 23 that a derivative can not be everywhere discontinuous. It must be continuous on a dense subset of its domain of definition. 25. Consider the characteristic functions f (x) and g (x ) of the intervals [ 1 , 4] and [2. 5]. The derivatives f ' , g' exist almost everywhere. The integration by parts formula says that 1' 3

Jo

f (x ) g ' (x ) dx = f ( 3)g ( 3 ) - f (O) g ( O) -

J{o 3

f ' (x) g (x ) d x .

Exercises

191

But both integrals are zero, while j (3 ) g ( 3) - j (O)g (O) = 1 . Where is the error? 26. Let Q be a set with a transitive relation � - It sati sfies the conditions that for all w, , wz , w3 E Q , w, � w, and w, � Wz � w3 implies that w 1 � w3 • A function f : Q --+ JR. converges to a limit L with respect to Q if, given any E > 0 there is an Wo E Q such that Wo � w implies l .f (w) - L l < E . We write lim n .f (w) = L to indicate this convergence. Observe that • When f (n) = an and N is given its standard order relation :;:: , limn �cx.. a n means the same thing as limN .f (n ) . • When JR. is given its standard order relation :;:: , limHoo .f (t) means the same thing as limJR f ( t) . • Fix an x E JR. and give JR. the new relation t1 � t2 when l tz - x l :::: I tt - x l . Then limHx .f ( t) means the same thing as limcJR. :o> f (t) . (a) Prove that limits are unique: if lim n f = L t and lim n f = Lz then L, = L z . (b) Prove that existence of lim n f and lim n g imply that lim (/ + cg) = lim f + c lim g n n n lim (/ · g) = lim f · lim g n n n lim (// g) = lim f/ lim g n n n where c is a constant and, in the quotient rule, lim n g f= 0, g t= 0. (c) Let Q consist of all partition pairs ( P , T); define ( P , T) � (P' , T ' ) when P' is finer than P, mesh P' :::: mesh P . Observe that � is transitive and that lim n R ( j, P, T) = I means the same as lim mesh P---. o R (/, P , T) in the definition of the Riemann integral. (d) Review the proof of Theorem 1 6 and use (b) to justify the fact that linearity of Riemann sums with respect to the integrand, R ( f + cg , P, T) = R ( f, P , T) + cR (g , P, T ) , actually does imply linearity o f the integral with respect to the integrand. (e) Formulate this limit definition for functions from Q to a general metric space in place of R

1 92

Functions of a Real Variable

Chapter 3

27. Redefine Riemann and Darboux integrability using only dyadic par­ titions. (a} Prove that the integrals are unaffected. (b) Infer that Riemann's integrability criterion can be restated in terms of dyadic partitions. 28. In many calculus books, the definition of the integral is given as

� f(xi ) b - a

n ---+ oo L

lim

k= l

--

n

where xi is the midpoint of the interval

k(b - a) (k - 1 ) (b - a) . a+ ]. n n See Stewart's Calculus with Early Transcendentals, for example. [a +

29.

30. 31 .

*32.

(a) If f is continuous, show that the calculus book limit exists and equals the Riemann integral of f. [Hint: This is a one-liner. ] (b) Show by example that the calculus style limit can exist for functions which are not Riemann integrable. (c) Infer that the calculus style definition of the integral is inade­ quate for real analysis. Suppose that Z C R Prove that the following are equivalent. (i) Z is a zero set. (ii) For each E > 0 there is a countable covering of Z by closed intervals [a i . bd with total length L b; - a; < E . (iii) For each E > 0 there i s a countable covering of Z by sets S; with total diameter L diam S; < E . Prove that the interval LO, 1 ] i s not a zero set. [Hint: B e careful; this is not entirely trivial.] The standard middle-quarters Cantor set is formed by removing the middle quarter from ro. 1 ] , then removing the middle quarter from each of the remaining two intervals, then removing the middle quarter from each of the remaining four intervals, and so on. (a) Prove that it is a zero set. (b) Formulate the natural definition of the middle /3-Cantor set. (c) Is it also a zero set? Prove or disprove. Define a Cantor set by removing from [0, 1 ] the middle interval of length 1 14. From the remaining two intervals F 1 remove the middle intervals oflength 1 I 1 6. From the remaining four intervals F2 remove the middle intervals of length 1 164, and so on. At the nth step in the construction p n consists of zn subintervals of p n - l

Exercises

33.

34.

*35.

*36.

1 93

(a) Prove that F = np n is a Cantor set but not a zero set. It is often referred to as a fat Cantor set. (b) Infer that being a zero set is not a topological property: if two sets are homeomorphic and one is a zero set, then the other need not be a zero set. [Hint: To get a sense of this fat Cantor set, calculate the total length of the intervals which comprise its complement. See Figure 49 and Exercise 36.] Consider the characteristic function of the dyadic rational numbers, j (x ) = 1 if x = kf2n for some k E Z and n E N, and f (x) = 0 otherwise. (a) What is its set of discontinuities? (b) At which points is its oscillation ::::: K ? (c) Is i t integrable? Explain, both by the Riemann-Lebesgue The­ orem and directly from the definition. (d) Consider the dyadic ruler function g (x) = 1 12n if x = kf2n and g (x) = 0 otherwise. Graph it and answer the questions posed in (a). (b), (c). (a) Prove that the characteristic function f of the middle-thirds Cantor set C is Riemann integrable but the characteristic func­ tion g of the fat Cantor set F (Exercise 32) is not. (b) Why is there a homeomorphism h : [0. 1 ] ---+ LO. 1] sending C onto F? (c) Infer that the composite of Riemann integrable functions need not be Riemann integrable. How is this example related to Corollaries 26, 30 of the Riemann-Lebesgue Theorem? See also Exercise 36. Assume that 1/1 : [a . b] -+ JR. is continuously differentiable. A critical point of 1/1 is an x such that 1/1' (x) = 0. A critical value is a number y such that for at least one critical point x, y = j (x ) . (a) Prove that the set o f critical values is a zero set. (This is the Morse-Sard Theorem in dimension one.) (b) Generalize this to continuously differentiable functions JR. -+ JR. Let F c [0, I ] be the fat Cantor set from Exercise 32, and define 1/f (x )

=

fox

dist(t , F) dt

where dist (t, F) refers to the minimum distance from t to F.

Functions of a Real Variable

1 94

Chapter 3

(a) Why is 1/f a continuously differentiable homeomorphism from [0, 1 ] onto [0, L] where L = 1/f ( 1 ) ? (b) What is the set of critical points of 1/f ? (See Exercise 35.) (c) Why is 1/f (F) a Cantor set of zero measure? (d) Let f be the characteristic function of 1/f (F). Why is f Riemann integrable but f o 1/f not? (e) What is the relation of (d) to Exercise 34? 37 Generalizing Exercise 30 in Chapter 1 , we say that f : (a , b) --+ JR. has a jump discontinuity (or a discontinuity of the first kind) at c E (a , b) if

f(c-)

=

lim f(x)

X -+ C

and

f (x + )

=

x �c+

lim f(x)

exist, but are either unequal or are unequal to f(c) . (The three quan­ tities exist and are equal if and only if f is continuous at c.) An oscillating discontinuity (or a discontinuity of the second kind) is any non-jump discontinuity. (a) Show that f : JR. --+ JR. has at most countably many jump discontinuities. (b) Show that 1 sm ­ if x =:: O

f (x)

=

{

.

0

x

if x ::::;: 0

has an oscillating discontinuity at x = 0. (c) Show that the characteristic function of the rationals, X Q . has an oscillating discontinuity at every point. *38. Recall that P ( S ) = 25 is the power set of S, the collection of all subsets of S and R is the set of Riemann integrable functions f : [a b ] --+ JR.. (a) Prove that the cardinality of R is the same as the cardinality of P (JR.) , which is greater than the cardinality of JR.. (b) Call two functions in R integrally equivalent if they differ only on a zero set. Prove that the collection of integral equivalence classes of R has the same cardinality as JR., namely zN . (c) Is it better to count Riemann integrable functions or integral equivalence classes of Riemann integrable functions? (d) Show that f, g E R are integrally equivalent if and only if the integral of I f g I is zero. 39. Suppose that 1/f : f c, d] --+ fa , b] is continuous and for every zero set Z c [a , b], 1jfP'e (Z) is a zero set in [c, d] . ,

-

Exercises

1 95

(a) If f is Riemann integrable, prove that f o 1/r is Riemann inte­ grable. (b) Derive Corollary 30 from (a) . 40. Let 1/f (x) = x sin l jx for 0 < x ::::: 1 and 1/f (O) = 0 . (a) If f : [ - 1 , l ] --+ lR is Riemann integrable, prove that f o 1/f is Riemann integrable. (b) What happens for 1/f (x) = ,JX sin l fx ? *4 1 . Assume that 1/f : [c . d] --+ [a . b ] is continuously differentiable. (a) If the critical points of 1/f form a zero set in [ c, d ] and f is Rie­ mann integrable on [a . b] prove that f o 1/f is Riemann integrable on [ c , d] . (b) Conversely. prove that if f o 1/f is Riemann integrable for each Riemann integrable f on [a , b], then the critical points of 1/J form a zero set. [Hint: Think in terms of Exercise 35.] (c) Prove (a) and (b) under the weaker assumption that 1/f is con­ tinuously differentiable except at finitely many points of L c. d] . (d) Derive part (a) of Exercise 36 from (c). (e) Weaken the assumption further to 1/J being continuously dif­ ferentiable on an open subset of [c, d] whose complement is a zero set. The following assertion, to be proved in Chapter 6, is related to the preceding exercises. If f : [a , b] --+ lR satisfies a Lipschitz condition or is monotone then the set of points at which f' (x) fails to exist is a zero set. Thus: "a Lipschitz function is differentiable almost everywhere," which is Rademacher's Theorem in dimension one, and a "monotone function is almost everywhere differentiable," which is the last theorem in Lebesgue's book, Lefons sur / 'integration et Ia recherche des fonctions primitives. See Theorem 39 and Corollary 4 1 i n Chapter 6 . 42. Set f (x) =

l

o_

rr

sm ­

x

if X ::::: 0 if x > 0

and

g (x ) =

{�

i f X ::::: 0

if X > 0.

Prove that f has an antiderivative but g does not. 43 . Show that any two antiderivatives of a function differ by a constant. [Hint: This is a one-liner.] 44. (a) Define the oscillation for a function from one metric space to another, f : M � N .

1 96

45 .

46.

47.

48 .

**49.

50.

51.

Functions of a Real Variable

Chapter 3

(b) Is it true that f is continuous at a point if and only if its oscil­ lation is zero there? Prove or disprove. (c) Fix a number K > 0. Is the set of points at which the oscillation of f is ::: K closed in M? Prove or disprove. (a) Prove that the integral of the Zeno' s staircase function described on page 1 6 1 is 2/3 . (b) What about the Devil's staircase? In the proof of Corollary 26 of the Riemann-Lebesgue Theorem, it is asserted that when ¢ is continuous the discontinuity set of ¢ o f is contained in the discontinuity set of f . (a) Prove this. (b) Give an example where the inclusion is not an equality. (c) Find a sufficient condition on ¢ so that ¢ o f and f have equal discontinuity sets for all f E R (d) Is your condition necessary too? Assume that f E R and for some m > 0, 1 / (x ) l ::: m for all x E [a , b] . Prove that the reciprocal of f, 1 /f (x ) , also belongs to R. If f E R. 1 / (x ) l > 0. but no m > 0 is an underbound for 1 / 1 . prove that the reciprocal of f is not Riemann integrable. Corollary 26 to the Riemann-Lebesgue Theorem asserts that if f E R and ¢ is continuous, then ¢ o f E R. Show that piecewise continuity can not replace continuity. [Hint: Take f to be a ruler function and ¢ to be a characteristic function.] Assume that f : [a , b] ---+ [ c . d] is a Riemann integrable bijection. Is the inverse bijection also Riemann integrable? Prove or disprove. [Hint: The graph of f - 1 is the same as the graph of f, viewed with the axes reversed. A function is Riemann integrable if and only if its graph is squeezed tightly between graphs of step functions. Draw a picture of this and rotate your paper by 90 degrees.] If f, g are Riemann integrable on [a , b] , and f (x ) < g (x ) for all x E La , b], prove that J: f (x) dx < J: g (x ) dx . (Note the strict inequality. ) Let f : la , b ] ---+ lR b e given. Prove or give counter-examples t o the following assertions. (a) f E R => 1 / 1 E R. (b) I f I E R => f E R. (c) f E R and 1 / (x ) l ::: c > O for all x => 1 // E R (d) f E R => f 2 E R. (e) / 2 E R => f E R.

Exercises

1 97

f 3 E n ==> J E n. f (x ) ;:: 0 for all X ==} f E R. f2 E R and [Here f 2 and f 3 refer to the functions f (x ) · f (x) and f (x ) f (x ) f(x ) , not the iterates.] 52. Given f, g E R, prove that max (/, g) , min(/, g) E R, where max(/, g) (x ) = max ( j (x ) , g (x)) and min(/, g) (x) = min( f (x) . (f) (g)

·

g (x) ).

53. Assume that f, g : [0, I ] --+ JR. are Riemann integrable and f (x ) = g (x) except on the middle-thirds Cantor set C. (a) Prove that f and g have the same integral. (b) Is the same true if f (x) = g (x) except for x E Q? (c) How is this related to the fact that the characteristic function of Q is not Riemann integrable? 54. Prove that if an ;:: 0 and L an converges, then L
.

.

•

converges. (I call this the block test because it groups the terms of the series in blocks of length 2k- I .) 58. Prove that L 1/ k log(k) P converges when p > 1 and diverges when p .:S 1 . Here k = 2, 3, . . . . [Hint: Integral test or block test.] 59. Concoct a series L ak such that ( - l ) k ak > 0, ak --+ 0, but the series diverges. 60. (a) Show that if a series has ratio lim sup p then it has exponential growth rate p . Infer that the ratio test is subordinate to the root test. (b) Concoct a series such that the root test is conclusive but the ratio test is not. Infer that the root test has strictly wider scope than the ratio test.

1 98

Functions of a Real Variable

Chapter 3

6 1 . Show that there is no simple comparison test for conditionally con­ vergent series: (a) Find two series L ak and L bk such that L bk converges con­ ditionally, aklbk ---+ 1 as k ---+ oo, and L ak diverges. (b) Why is this impossible if the series L bk is absolutely conver­ gent? th 62. An infinite product is an expression n Ck where Ck > 0. The n partial product is Cn = c1 · · · Cn . If Cn converges to a limit C =I= 0 the product converges to C. Write ck = 1 + ak. If each ak ::::: 0 or each ak ::::: 0 prove that L ak converges if and only if n Ck converges. [Hint: Take log's.] 63 . Show that conditional convergence of the series L ak and the product n ( 1 + ak ) can be unrelated to each other: k (a) Set ak = ( - 1 ) I -Jk. The series L ak converges but the corre­ sponding product n ( 1 + ak ) diverges. (b) Let ek = 0 when k is odd and ek = 1 when k is even. Set bk = ek I k + ( - 1 ) k I -Jk. The series L bk diverges while the corresponding product n ( l + bk ) converges. 64. Consider a series L ak and rearrange its terms by some bijection f3 : N ---+ N, forming a new series L a,B(k) · The rearranged series converges if and only if the partial sums a.a co + · · · + a.a
Exercises

1 99

(a) Prove that Y is closed and connected. (b) If Y is compact and non-empty, prove that L bk converges to Y in the sense that dn ( Yn , Y) -+ 0 as n -+ oo, where dn is the Hausdorff metric on the space of compact subsets of JR. and Yn is the closure of { Bm : m ?:: n } . See Exercise 2. 1 24. (c) Prove that each closed and connected subset of JR. is the set of subsequential limits of some rearrangement of L ak. The article, "The Remarkable Theorem of Levy and Steinitz" by Peter Rosenthal in the American Math Monthly of April 1 987 deals with some of these issues. * * *67 . Let V be a Banach space - a vector space with a norm such that V is complete with respect to the metric induced by the norm. (For example, JR.m is a Banach space.) If L Vk is a convergent series of vectors in V such that L II vk I I diverges, what is the generalization of Exercise 66? In particular, is Y convex? *68. Absolutely convergent series can be multiplied in a natural way, the result being their Cauchy product,

where ck = ao bk + a 1 bk -1 + · · · + akbo . (a) Prove that L ck converges absolutely. (b) Formulate some algebraic laws for such products ( commutativity, distributivity, and so on.) Prove two of them. [Hint for (a) : Write the products aibi in an oo x oo matrix array M, and let An , Bn , Cn be the n th partial sums of L ai , L bi , L Ck . You are asked to prove that (lim A n ) (lim Bn ) = lim Cn . The product of the limits is the limit of the products. The product An Bn is the sum of all the aibi in the n x n comer submatrix of M and en is the sum of its anti-diagonal. Now estimate An Bn - Cn . Alternately, assume that a n , bn ?:: 0 and draw a rectangle R with edges A , B . Observe that R is the union of rectangles Rij with edges ai , bi .] * *69. With reference to Exercise 68, (a) Reduce the hypothesis that both series L ai and L bi are ab­ solutely convergent to merely one being absolutely convergent and the other convergent. (Exercises 68 and 69(a) are known as Mertens' Theorem.) (b) Find an example to show that the Cauchy product of two con­ ditionally convergent series may diverge.

Functions of a Real Variable

200

Chapter 3

* *70. The Riemann � -function is defined to be � (s) = :L� 1 n- s where > 1 . It is the sum of the p-series when p = s . Establish Euler's

s

product formula,

oo

� (s ) = kn =l

71.

72.

73.

* * *74.

l

I - Pk-s

where Pk is the kth prime number. Thus, p 1 = 2, P2 = 3 , and so on. Prove that the infinite product converges. [Hint: Each factor in the infinite product is the sum of a geometric series 1 + p;' + ( p;' )2 + . . . . Replace each factor by its geometric series and write out the n th partial product. Apply Mertens' Theorem, collect terms, and recall that any integer has a unique prime factorization.] Invent a continuous function f : JR. -+ JR. whose improper integral is zero, but which is unbounded as x -+ - oo and x -+ oo. [Hint: f is far from monotone.] Assume that f : JR. -+ JR. and that restricted to any closed interval, f is bounded. (a) Formulate the concepts of conditional and absolute convergence of the improper Riemann integral of f. (b) Find an example that distinguishes them. Let f : [0, oo) -+ [0. oo) and I > k be given. Assume that for all sufficiently large k and all x E [ k , k + 1 ) , f x ) ::=: ak . Prove that divergence of the improper integral J000 f (x) dx implies divergence of L: ak . Let f : JR. -+ JR. be given. Assume that the square and cube of f are smooth. Is f smooth? That is, if f · f E c oo and f f f E C00, does it follow that f E C 00 ?

(

·

·

4

Function Spaces

1

Uniform Convergence and

C0 La , b]

Points converge to a limit if they get physically closer and closer to it. What about a sequence of functions? When do functions converge to a limit function? What should it mean that they get closer and closer to a limit function? The simplest idea is that a sequence of functions fn converges to a limit function f if for each x , the values fn (x ) converge to f (x ) as n -4 oo. This is called pointwise convergence: a sequence of functions fn : [a , b] -4 JR. converges pointwise to a limit function f : [a , b] -4 JR. if for each x E [a , b]. lim f (x ) = f (x ) . n -> oo n

The function f is the pointwise limit of the sequence Cfn ) and we write fn -4 f

or

lim fn = f.

n ---> 00

Note that the limit refers to n -4 oo, not to x -4 oo. The same definition applies to functions from one metric space to another. The requirement of uniform convergence is stronger: the sequence of functions fn : La , b] -4 JR. converges uniformly to the limit function f : [a . b ] -4 lR if for each E > 0 there is an N such that for all n :=: N and

202

Function Spaces

Chapter 4

all X E [a , b ] , l fn (X ) - f (x ) l

(1)

< E.

The function f i s the uniform limit of the sequence fn

::::::t

f

or

Un ) and we write

unif n lim fn = f.

-+ oo

Your intuition about uniform convergence is crucial. Draw a tube V of vertical radius E around the graph of f . For n large, the graph of fn lies wholly in V . See Figure 83. Absorb this picture !

b

Figure 83 The graph of fn is contained in the E -tube around the graph of f .

It is clear that uniform convergence implies pointwise convergence. The difference between the two definitions is apparent in the following standard example. n Example Define fn : (0. 1 ) � lR by fn (x ) = x . For each x E (0, 1 ) it is clear that fn (x ) � 0. The functions converge pointwise to the zero function as n � oo. They do not converge uniformly: for take E = l / 1 0. The point Xn = ,ifl/2 is sent by fn to 1 /2 and thus not all points x satisfy ( 1 ) when n is large. The graph of fn fails to lie in the E -tube V . See Figure 84.

The lesson to draw is that pointwise convergence of a sequence of func­ tions is frequently too weak a concept. Gravitating toward uniform conver­ gence we ask the natural question: Which properties offunctions are preserved under uniform convergence?

The answers are found in Theorem 1 , Exercise 4, Theorem 6, and Theorem 9. Uniform limits preserve continuity, uniform continuity, integrability, and - with an additional hypothesis - differentiability.

Uniform Convergence and C 0 [a , b]

Section 1

203

0

0

Figure 84 Non-uniform. pointwise convergence. 1 Theorem If fn � f and each fn is continuous at xo , then f is continuous at xo. In other words, the uniform limit of continuous functions is continuous. Proof For simplicity, assume that the functions have domain [a , b] and target JR. (See also Section 8 and Exercise 2.) Let E > 0 and x0 E [a , b] be given. There is an N such that for all n ::::_ N and all x E [a , b] ,

l fn (X ) - f( x ) l The function fN is continuous at l x - xo I < � implies

If lx

- xo I

<

x0

<

E 3·

and so there is a �

0 such that

� then

l f (x ) - f (xo) l

:S:

l f (x ) - !N (x ) l + I JN (x ) - fN (xo) l + I JN (xo) - f (xo) l

E E E < - + - + - = E. 3 3 3

-

Thus

>

f is continuous at x0 E

D

[a , b] .

Without uniform convergence. the theorem fails. For example. define n : [0, 1 ] � lR as before, fn (x ) = x . Then J,, (x ) converges pointwise to the function

fn

f(x )

�

{�

if O :s: x < l if x = L

204

Function Spaces

Chapter 4

The function I is not continuous and the convergence is not uniform. What about the converse? If the limit and the functions are continuous, does pointwise convergence imply uniform convergence? The answer is "no," as is shown by x n on (0, 1 ) . But what if the functions have a compact domain of definition, [a , b] ? The answer is still "no.' ' Example John Kelley refers to this as the growing steeple,

if O if !n if ln

<x <x <x -

! n l n < - l.

< < -

See Figure 85 .

0

Figure 85 The sequence of functions converges pointwise to the zero

function, but not uniformly. Then lim n ---+ oo In (x) = 0 for each x . and In converges pointwise to the function I = 0. Even if the functions have compact domain of definition, and are uniformly bounded and uniformly continuous, pointwise conver­ gence does not imply uniform convergence. For an example, just multiply the growing steeple functions by 1 / n . The natural way to view uniform convergence i s i n a function space. Let Cb = Cb ( [a , b] , .IR) denote the set of all bounded functions [a , b] """""* JR.

Section 1

Uniform Convergence and C 0 [a , b]

205

The elements of Cb are functions f, g, etc. Each is bounded. Define the sup norm on Cb as l l f ll = sup{ l f (x) l : x E [a , b] } .

The sup norm satisfies the norm axioms discussed in Chapter I , page 27 . I I f I I ::: 0 and I f I = 0 if and only if f = 0. l l cf l l = l c l l l f ll II I + g il :::: II I II + I I g il . As we observed in Chapter 2, any norm defines a metric. In the case at hand, d ( f, g) = sup{ l f (x) - g (x ) l : x E [a , b] }

is the corresponding metric on Cb . See Figure 86. To distinguish the norm I I I I I = sup l f (x) l from other norms on Cb we sometimes write ll f l l sup for the sup norm.

I

, '

I

Figure 86 The sup-norm of f and the sup-distance between the functions f and g .

The thing to remember is that Cb is a metric space whose elements are functions. Ponder this. 2 Theorem Convergence with respect to the sup-metric d is equivalent to uniform convergence. Proof If d ( fn , f)

j�

=:::t

----+ 0 then sup{ l fn x - f x l f, and conversely.

3 Theorem Cb is a complete metric space.

x

E

[ a , bJ }

----+

0, so D

Function Spaces

206

Proof

Chapter 4

Let (fn) be a Cauchy sequence in Cb . For each individual x0 E [a . b] fn (xo) form a Cauchy sequence in Jl{ since

the values

i fn (Xo) - fm (Xo) i

S

sup{ l fn (x) - fm (x ) i : X E [a , b] } = d ( fn , fm) .

Thus, for each x E [a , b],

nlim -+oo fn ( X )

exists. Define this limit to be f (x ) . It is clear that fn converges pointwise to f. In fact, the convergence is uniform. For let E > 0 be given. There exists N such that m, n ::::: N imply

Also, for each x E [a , b] there exists an m =

i fm (X ) - f (x ) i If n :::::

m (x )

::::: N such that

E < 2·

N and x E [a , b] then i fn (X) - f (x ) i

S

l fn (X) - fm (x) (x ) l

E E < - + - = E.

2

+

l fm (x) (X ) - f (x ) l

2

Hence fn :::4 f. The function f is bounded. For !N is bounded and for all x , i fN (x ) - f (x ) i < E . Thus f E Cb . By Theorem 2, uniform con­ vergence implies d-convergence, d ( fn , f ) -4 0, and the Cauchy sequence Un ) converges to a limit in the metric space Cb . D The preceding proof is subtle. The uniform inequality dUn , f ) < E is derived by non-uniform means: for each x we make a separate estimate using an m (x ) depending non-uniformly on x . It is a case of the ends justifying the means. Let C 0 = C 0 ( [a , b], Jl{) denote the set of continuous functions [a , b J -4 Jl{_ Each f E C 0 belongs to cb since a continuous function defined on a compact domain is bounded. That is, C 0 c Cb . 4 Corollary C 0 is a closed subset of Ch · It is a complete metric space. Proof Theorem l implies that a limit in Cb of a sequence of functions in

lies in C 0 . That is, C 0 is closed in Cb . A closed subset of a complete D space is complete. C0

Uniform Convergence and C 0 [a , b]

Section 1

207

Just as it is reasonable to discuss the convergence of a sequence of func­ tions we can also discuss the convergence of a series of functions, L fk · Merely consider the nth partial sum Fn (x) =

n

L /k (x) . k=O

It is a function. If the sequence of functions ( Fn ) converges to a limit function F then the series converges, and we write F (x ) =

00

L fk (x ) . k=O

If the sequence of partial sums converges uniformly, then so does the series. If the series of absolute values L I /k (x ) I converges, then the series L A converges absolutely. 5 Weierstrass M-test If L Mk is a convergent series of constants and if fk E C satisfies I fk I ::::: Mk for all k, then L fk converges uniformly and b absolutely. Proof If

n

telescope as

>

m

then the partial sums of the series of absolute values

n

n

k=m+ l

k=m+ l

Since L Mk converges, the last sum is < E when m , n are large. Thus ( Fn ) is Cauchy in C . and by Theorem 3 it converges uniformly. 0

b

Next we ask how integrals and derivatives behave with respect to uniform convergence. Integrals behave better than derivatives. 6 Theorem The uniform limit of Riemann integrable functions is Riemann integrable, and the limit of the integrals is the integral of the limit,

lim

{ b fn (x ) dx

n ---?>- 00 Ja

=

b

limf (X) dx . n ---?>- 00 n J{a unif

Function Spaces

208

Chapter 4

In other words, R is a closed subset of Ch and the integral functional f 1---7 J: f(x) dx is a continuous map from R to JR. This extends the regularity hierarchy to

Theorem 6 gives the simplest condition under which the operations of taking limits and integrals commute. Proof Let fn E R be given and assume that fn ::::::t f as n --+ oo. By the Riemann-Lebesgue Theorem, fn is bounded and there is a zero set Zn such that fn is continuous at each x E fa , b ] \ Zn . Theorem 1 implies that f is continuous at each x E [a , b] \ U Zn , while Theorem 3 implies that f is bounded. Since U Zn is a zero set, the Riemann-Lebesgue Theorem implies that f E R. Finally

1 1b 1b

j( } x dx

:S

- 1b

l 1 1b

fn (x ) dx =

f (x) - fn (x ) dx

l f (x) - fn (x ) l dx :S d U, fn ) ( b - a ) --+ 0

l

as n --+ oo. Hence the integral of the limit is the limit of the integrals. 7 Corollary unifonnly,

1 1x

If fn

E R and fn

- 1x

Proof As above, f ( t ) dt

:::::4

0

f then the indefinite integrals converge

l

fn ( t ) dt :S d Un . f) (x - a ) :S d Un , f) ( b - a ) --+ 0

when n --+ oo.

0

8 Term by Term Integration Theorem A unifonnly convergent series of integrable functions L fk can be integrated tenn by tenn in the sense that

b1 �OO

fk (x) dx =

� 1b fk (x) 00

dx .

Uniform Convergence and C 0 La , b J

Section 1

209

F, converges uniformly to L fk . Each F, belongs to R since it is the finite sum of members of R. According to

Proof The sequence of partial sums

Theorem 6,

�1 Ti

b

fk (x) dx =

1

b

1 � fk (x) dx . b OO

F, (x) dx

�

This shows that the series L I: fk (x) dx converges to I: L fk (x) dx . D

The uniform limit of a sequence of differentiable functions is differentiable provided that the sequence of derivatives also converges uni­ formly. 9 Theorem

Proof We suppose that f, : [a , b] � lR is differentiable for each n and that f, ::::4 f as n � oo. Also we assume that f� ::::4 g for some function g. Then we show that f is differentiable and in fact f ' = g . We first prove the theorem with a major loss of generality: we assume that each f� is continuous. Then f� . g E R and we can apply the fundamental theorem of calculus and Corollary 7 to write

f (a) +

1x g(t) dt .

Since f, ::::4 f we see that f(x) = f (a) + I: g(t) dt and, again by the fundamental theorem of calculus, f' = g. In the general case the proof i s harder. Fix some x E [a , b] and define _



=

{ {

f, (t) - f, (x) t-x

f� (x)

f (t) - f (x ) 1 -X

g (x)

if t f= X if t = X

if t f= X

if t = X .

Each function ¢, is continuous since

_


_

J.' (O ) n

210

Function Spaces

Chapter 4

for some () between t and x . Since f� ::=t g the difference J;, - f� tends uniformly to 0 as m , n --+ oo. Thus (¢n ) is Cauchy in C 0 • Since C0 is com­ plete. ¢n converges uniformly to a limit function lfr . and lfr is continuous. As already remarked, the pointwise limit of ¢n is ¢, and so lfr = ¢ . Continuity 0 of lfr = ¢ implies that g (x) = j' (x ) . 10 Term by Term Differentiation Theorem A uniformly convergent se­ ries of differentiable functions can be differentiated term by term, provided that the derivative series converges uniformly,

Proof Apply Theorem 9 to the sequence of partial sums.

0

Note that Theorem 9 fails if we forget to assume the derivatives converge. For example, consider the sequence of functions fn : [ - 1 1 ] --+ � defined by ,

Figure 87 The uniform limit of differentiable functions need not be

differentiable. See Figure 87. The functions converge uniformly to f (x) = lx I , a non­ differentiable function. The derivatives converge pointwise but not uni­ formly. Worse examples are easy to imagine. In fact, a sequence of ev­ erywhere differentiable functions can converge uniformly to a nowhere differentiable function. See Sections 4 and 7. It is one of the miracles of the complex numbers that a uniform limit of complex differentiable functions is complex differentiable, and automatically the sequence of derivatives con­ verges uniformly to a limit. Real and complex analysis diverge radically on this point.

Power Series

Section 2

2

21 1

Power Series

As another application of the Weierstrass M -test we say a little more about the power series L ck x k . A power series is a special type of series of func­ tions, the functions being constant multiples of powers of x. As explained in Section 3 of Chapter 3, its radius of convergence is 1 R- ----= · lim sup {ICk

k�oo

Its interval of convergence is ( - R, R) . If x E ( - R, R), the series converges and defines a function f (x) = L ck x k , while if x fj. [- R , R], the series diverges. More is true on compact subintervals of ( - R , R) . 1 1 Theorem If r < R , then the power series converges uniformly and absolutely on the interval [ -r, r ]. Proof Choose {3 , r < f3 < R . For all large k, � < 1 / f3 since f3 Thus, i f l x l .::: r then

<

R.

These are terms in a convergent geometric series and according to the M -test 0 L ck x k converges uniformly when x E [ -r. r ] .

A power series can be integrated and differentiated term by term on its interval of convergence. 12 Theorem

For f(x) = L: ck x k and l x l

r f(t) dt =

Jo

<

fk+1 k=O

R this means

� x k+ l

00

and f ' (x) =

L kckx k - 1 • k=l

\

Proof The radius of convergence of the integral series is determined by the

exponential growth rate of its coefficients,

Since (k - 1 ) / k --+ 1 and k - l f k --+ 1 as k --+ oo, we see that the integral series has the same radius of convergence R as the original series. According to Theorem 8, term by term integration is valid when the series converges

Function Spaces

212

Chapter 4

uniformly, and by Theorem I 1 , the integral series does converge uniformly on any interval [ - r, r] C (- R , R) . A similar calculation for the derivative series shows that its radius of convergence too is R. Term by term differentiation is valid provided the series and the derivative series converge uniformly. Since the radius of convergence of the derivative series is R, the derivative series does converge uniformly on any [ - r, r] c ( - R , R). D 13 Theorem

Analytic functions are smooth, cw

c

C 00 •

Proof An analytic function f is defined by a convergent power series. According to Theorem 1 2, the derivative of f is given by a convergent power series with the same radius of convergence, so repeated differentiation is D valid, and we see that f is indeed smooth.

1 �-l jx

The general smooth function is not analytic, as is shown by the example e (x ) =

if x > 0

if x ::::: 0

on page 1 49. Near x = 0, e (x ) can not be expressed as a convergent power series. Power series provide the clean and unambiguous way to define functions, especially trigonometric functions. The usual definitions of sine, cosine, etc. involve angles and circular arc length, and these concepts seem less fundamental than the functions being defined. To avoid circular reasoning, as it were, we declare that by definition 00

k

exp x = " � � koft.O

k!

.

sm x

=

oo (- l ) k x 2k+ t L (2k + I ) ! k=O

oo ( - l txzk cos x = L ---­ k=O

(2k) !

We then must prove that these functions have the properties we know and love from calculus. All three series are easily seen to have radius of conver­ gence R = oo. Theorem 1 2 justifies term by term differentiation, yielding the usual formulas, exp' (x) = exp x

sin ' (x) = cos x

cos ' (x) = - sin x .

The logarithm has already been defined as the indefinite integral J( I I t dt. We claim that if lx I < 1 then log( l + x ) is given as the power series log ( I + x) =

oo

L

( - l ) k+l x k

k

Compactness and Equicontinuity in

Section 3

C0

213

To check this, we merely note that its derivative is the sum of a geometric series, (log( l

1

00

00

+ x))' x +1 -l = 1 - (-x) = '"'C-x)k = '"'c-llxk. k=O k=O =

-

LJ

LJ

The last is a power series with radius of convergence 1 . Since term by term integration of a power series inside its radius of convergence is legal. we integrate both sides of the equation and get the series expression for log ( l x ) as claimed.

+

3

Compactness and Equicontinuity in

C0

The Heine-Bore} theorem states that a closed and bounded set in IR.m is 0 compact. On the other hand, closed and bounded sets in C are rarely compact. Consider for example the closed unit ball B =

{f E C 0 ([ 0 , 1 ] , JR.)

:

ll f ll �

1 }.

fn (x) xn .

To see that B is not compact we look again at the sequence = It lies in B. Does it have a subsequence that converges (with respect to the converges to f in C 0 then metric d of C 0 ) to a limit in C 0 ? No. For if = lim Thus = 0 if x < 1 and f ( l ) = 1 , but this k-+ oo 0 function f does not belong to C • The cause of the problem is the fact that C 0 is infinite-dimensional. In fact it can be shown that if V is a vector space with a norm then its closed unit ball is compact if and only if the space is finite-dimensional. The proof is not especially hard. Nevertheless, we want to have theorems that guarantee certain closed and bounded subsets of C 0 are compact. For we want to extract a convergent subsequence of functions from a given sequence of functions. The simple condition that lets us go ahead is equicontinuity. A sequence of functions in C 0 is equicontinuous if

j(x)

C fn)

fnk(x).

VE > Is -

0 38

fnk

f (x)

>

0

such that

l fn (s ) - fn(t)l < E . t i and n E N The functions fn are equally continuous. The depends on E but it does not depend on n . Roughly speaking, the graphs of all the fn are similar. For < 8

=}

8

total clarity, the concept might better be labeled uniform equicontinuity, in contrast to pointwise equicontinuity, which requires

VE

> 0 and Vx E

l x - ti

[a , b]

< 8 and n E N

38

>

=}

0 such that

l fn (X ) - fn (t) i < E.

Function Spaces

214

Chapter 4

The definitions work equally well for sets of functions, not only sequences of functions. The set £ c C 0 is equicontinuous if VE > 0 315 > 0 such that < t5 and

�

l f(s ) - f(t) i < E . The crucial point i s that t5 does not depend on the particular f E £. It is valid for all f E £ simultaneously. To picture equicontinuity of a family £, Is - t l

f

E

£

imagine the graphs. Their shapes are uniformly controlled. Note that any finite number of continuous functions [a , b] --+ JR. forms an equicontinuous family so Figures 88 and 89 are only suggestive.

Figure 88 Equicontinuity.

Figure 89 Non-equicontinuity.

The basic theorem about equicontinuity is the 14 Arzela-Ascoli Theorem Any bounded equicontinuous sequence offunc­ tions in C 0 ( [a , b] , JR.) has a uniformly convergent subsequence.

Compactness and Equicontinuity in C 0

Section 3

215

Think of this as d compactness result. H (fn) i s the sequence of equicon­ tinuous functions, the theorem amounts to asserting that the closure of the set { fn : n E N} is compact. Any compact metric space serves just as well as [a , b], and the target space lR can also be more general. See Section 8.

[a , b] has a countable dense subset D = { d1 , d2 , } . For instance we could take D = Ql n [a , b] . Boundedness of Un) means that for some constant M, all x E [a , b], and all n E N, l fn (x) i :::; M. Thus Un (di )) is a bounded sequence of real numbers. Balzano-Weierstrass implies that some subsequence of it converges to a limit in JR, say Proof

•

.

•

The subsequence fi . k evaluated at the point dz is also a bounded sequence in JR, and there exists a sub-subsequence fz. k such that fz. k (d2 ) converges to a limit in JR, say fz . k (dz) --+ yz as k --+ oo . The sub-subsequence evaluated at d 1 still converges to y1 • Continuing in this way gives a nested family of subsequences !m , k such that

Um . k ) is a subsequence of Um - I , k ) j :::S m � fm . k (dj) --+ Yi as k

Choose k(m) 2:::

m large enough that if j

--+ oo . :::; m and k(m) :::; k then

The superdiagonal subsequence gm (x) = !m , k (m J (x) converges to a limit at each point x E D. We claim that 8m (x) also converges at the other points x E [a , bJ and that the convergence is uniform. It suffices to show that (gm ) is a Cauchy sequence in C 0 . Let E > 0 be given. Equicontinuity gives a 8 > 0 such that for all s, t

E [a , bJ ,

Choose J large enough that every x E [a , b ] lies in the 8-neighborhood of some di with j :::; J. Since D is dense and [a , b] is compact, this is possible. See Exercise 1 9. Since {d 1 , , d1 } is a finite set and gm (dj) converges for each di , there is an N such that for all i , m :::: N and all j :::; J , •

•

•

Function Spaces

216 If l ,

m

:::: N and x E [a , bJ , choose di with j dj - x i < 8 and j ::S J . Then

gm (x) - g (x) l e

Hence

Chapter 4

::S

j gm (x ) - gm (dj ) j + j gm (dj ) - gf (dj ) j + j ge (dj ) - 8t (X

E E E < - + - + - = E. 3 3 3

(8m ) is Cauchy in C 0 , it converges in C 0 , and the proof is complete.

0

Part of the preceding development can be isolated as the

Pointwise convergena of an equicontinuous sequence of functions on a dense subset of the domain propagates to uniform convergence on the whole domain. 15 Arzela-Ascoli Propagation Theorem

0

Proof This is the E /3 part of the proof.

The example cited over and over again in the equicontinuity world is the following:

16 Corollary Assume that fn : La , b] --+ lR is a sequence of differentiable functions whose derivatives are uniformly bounded. Iffor some Xo, fn (xo ) is bounded as n --+ oo, then the sequence (fn) has a subsequence that converges uniformly on [a , b].

Proof Let M be a bound for the derivatives I f� (x) I , valid for all n E N and x E [a . b] . Equicontinuity of Un ) follows from the Mean Value Theorem:

Is - t i

< 8

==>

l fn (s) - fn (t ) i = l f� (B ) j l s - t i

::S M8

for some e between s and t. Thus, given E > 0, the choice 8 = Ej (M + 1 ) shows that Un ) is equicontinuous. Let C be a bound for l fn (xo) l . valid for all n E N. Then

l fn (x) l

l fn (x) - fn (xo ) l + l fn (xo) l ::S M l b - a l + C ::S

::S

M lx - .\ol + C

shows that the sequence Un ) is bounded in C 0 . The Arzela-Ascoli theorem 0 then supplies the uniformly convergent subsequence. Two other consequences of the same type are fundamental theorems in the fields of ordinary differential equations and complex variables.

Uniform Approximation in

Section 4

C0

217

(a) A sequence o f solutions to a continuous ordinary differential equation in IRm has a subsequence that converges to a limit, and that limit is also a solution of the ODE. (b) A sequence of complex analytic functions that converges pointwise, converges uniformly (on compact subsets of the domain of definition) and the limit is complex analytic. Finally we give a topological interpretation of the Arzela-Ascoli theorem.

A subset £ c C 0 is com­ pact if and only if it is closed, bounded, and equicontinuous. 17 Heine-Borel Theorem in a function space

Proof Assume that £ is compact. By Theorem 2.56, it is closed and totally

bounded: given E > 0 there is a finite covering of £ by neighborhoods in C 0 that have radius E/3, say �1 3 ( fk ) , with k = 1 . . n . Each fk is uniformly continuous so there is a 8 > 0 such that .

If f E

Is - t l

< 8

=>

.

.

l fk (s ) - fk ( t) l

E < 3·

£ then for some k, f E �;J ( fk ) , and Is - t l < 8 implies

l f(s) - f (t) l

:S

l f (s) - fd s) i + i fk (s) - fk (t) l + l fk (t) - f (t) i

E E E < - + -+ - = E

3 3 3 Thus £ is equicontinuous. Conversely, assume that £ is closed, bounded, and equicontinuous If < fn ) i s a sequence i n £ then by the Arzela-Ascoli theorem, some subsequence < fn k ) converges uniformly to a limit. The limit lies in £ since £ is closed D Thus £ is compact.

4

Uniform Approximation in C0

Given a continuous but nondifferentiable function f, we often want to make it smoother by a small perturbation. We want to approximate f in C 0 by a smooth function g . The ultimately smooth function is a polynomial, and the first thing we prove is a polynomial approximation result 18 Weierstrass Approximation Theorem in C 0 ([a , bl , JR) .

The set ofpolynomials is dense

Density means that for each f E C 0 and each E > 0 there is a polynomial function p (x) such that for all x E La, b],

l f (x) - p(x) i < E . There are several proofs of this theorem, and although they appear quite

Function Spaces

218

Chapter 4

different from each other, they share a common thread: the approximating function is built from f by sampling the values of f and recombining them in some nice way. It is no loss of generality to assume that the interval [a, bj is [0 . 1 ] . We do so.

Proof #1 For each n E

N, consider the sum

Pn (x) =

t (:)ckx k (l - xt-k , k=O

- ).

where ck = f ( k j n) and G) is the binomial coefficient n ! / k ! (n k ! Clearly Pn is a polynomial. It is called a Bernstein polynomial. We claim that the nth Bernstein polynomial converges uniformly to f as n ._ oo. The proof relies on two formulas about how the functions

rk (x ) = (: ) x k (l - xt -k

shown in Figure 90 behave. They are n

(2)

n

(3)

:�:)k - nx ) 2 rk (x) = nx ( I

k=O

In terms of the functions

(6)k xk

- x).

rk we write

.....

Figure 90 The seven basic Bernstein polynomials of degree six.

(1

- x)

6 -k

,k=0

6.

Uniform Approximation in C 0

Section 4

n Pn (X) = .L::Ck rk (X ) k=O

219

n f (x ) = L f(x)rk (x) . k=O

Then we divide the sum Pn - f = L: Cck - f)rk into the terms where k/n is near x , and other terms where kIn is far from x . More precisely, given E > 0 we use uniform continuity of f on [0, I ] to find 8 > 0 such that i t - s l < 8 implies i f (t) - f(s) i < E/2. Then we set K1 = {k e {O, . . , n } : .

This gives

l � - x � < 8}

K2 = {0, . . . n } \ K 1

and

.

n

The factors i ck - f (x) i in the first sum are < E/2 since ck = f(kj n) and kjn differs from x by < 8 . Since the sum of all the terms rk is 1 and the terms are non-negative. the first sum is < E /2. To estimate the second sum, use (3) to write

nx ( I - x)

n 2 = L(k - nx) 2 rk (x) :::: L (k - nx) rk (x) k=O

:::: since k

e

L (n8) 2 rk (x) , k E K2

keK2

K2 implies that (k - nx ) 2 :::: (n8) 2 . This implies that

L rk (x ) -<

kEK2

nx ( l - x) 1 < 2 2 4n8 (n8)

--

since max x ( l - x) = 1 /4 as x varies in [0. 1 ] . The factors i ck - f(x) i in the second sum are at most 2M where M = I f 11 . Thus the second sum is E "" l ck - f (x) i rk (x) :::: M 2 :::: 2 L Z n8 k E K2 when n is large, completing the proof that I Pn (x) - f (x) I < E when large.

n

is

220

Function Spaces

Chapter 4

(2) and ( 3 ). The binomial coefficients

It remains to check the identities satisfy

(4)

x.

which becomes if we set = 1 On the other hand, if we fix y and differentiate (4) with respect to once, and then again, we get

(2)

(5) (6)

y

x

-

n(x + y) n -1 = t (�) kx k -l yn - k , k=O n (n - l)(x + y t - 2 = t (�) k(k - l )x k-Z y n - k . k=O

Note that the bottom term in (5) and the bottom two terms in (6) are 0. Multiplying (5) by and (6) by and then setting = 1 in both equations gives

x

x2

y

-x

nx = t (�)kx k (I - x) n -k = t krk(x), k=O k=O k (8) n(n - l)x2 = t ( � )k(k - l)x k (l - x )n- = t k(k - l)rk(x). k =O k=O The last sum is L k2rk(x) - L krk(x). Hence (7), (8) become n n k2rk(x) = n(n l)x2 krk(x) = n(n - l)x2 + nx. + (9) L L k=O k=O (7 )

Using (2 ),

(7 ), (9), we get

n (k - nx)2rk(x) L k=O n = L k2rk (x) - 2nx L krk (x) + (nx)2 L rk(x) k=O k=O k=O = n(n - l)x2 + nx - 2(nx)2 + (nx)2 = -nx2 + nx = nx(I - x), a s claimed i n ( 3 ) n

.

n

0

Uniform Approximation in C 0

Section 4

22 1

Proof #2 Let f E C 0 ( [0, 1 j , JR) be given and let g (x) =

where

j(x ) - (m x + b )

f ( l ) - f (O)

and b = f (O) . 1 Then g E C 0 and g ( O) = 0 = g ( l ) . lf we can approximate g arbitrarily well by polynomials, then the same is true of f since mx + b is a polynomial. In other words it is no loss of generality to assume that / (0) = f ( l ) = 0 in the first place. Also, we extend f to all of lR by defining f (x) = 0 for all x E lR \ [0, 1 ] . Then we consider a function

m =

n f3n ( t ) = bn ( l - t 2 )

- 1 ::=:: t ::=:: 1 ,

where the constant bn is chosen so that J� 1 f3n (t ) dt = 1 . As shown in Figure 91 , f3n is a kind of polynomial bump function. Set Pn (X) =

/_: f

(x + t ) f3n ( t ) d t .

This i s a weighted average o f the values o f f using the weight function f3n . We claim that Pn is a polynomial and Pn (x) =t f (x) as n --+ oo .

Figure 91 The graph of the function f36 (t) = 1 .467 ( 1

-

t2 ) 6 •

To check that Pn is a polynomial we use a change of variables, u = x + t . Then Pn (x ) =

r+l f (u )f3n (U - x) d u lx-1

=

Jot f ( ) f3 u

n (X

- u ) du

since f = 0 outside of [0, 1 ] . The function f3n (x - u ) is a polynomial in

Function Spaces

222

Chapter 4

x

whose coefticients are polynomials in u . The powers of x pull out past the integral and we are left with these powers of x multiplied by numbers, the integrals of the polynomials in u times f (u) . In other words, by merely inspecting the last formula, it becomes clear that Pn (x) is a polynomial in x . To check that Pn =t f as n --+ oo , we need to estimate f3n (t) . We claim that if 8 > 0 then

(10)

f3n (t)

=t

0 as n --+

oo and 8 ::::;

ItI

::::;

1.

This is clear from Figure 9 1 . Proceeding more rigorously, we have

,

n n ----> oo (1 - 1 / n ) we see that for some constant c and all n ,

Since l fe = lim

See also Exercise 29. Hence if 8 ::::; I t I ::::;

1

then

due to the fact that Jn tends to oo more slowly than (1 - 8 2 ) - n as n --+ oo . This proves ( I 0) . From ( 1 0) we deduce that Pn =t f, as follows: Let E > O be given. Uniform continuity of f gives 8 > O such that l t l < 8 implies l f (x + t) - f(x) l < E /2. Since f3n has integral 1 on L- 1 , 1 ] we have

I Pn (X ) - f(x) l ::S

=

I L: (f(x + t) - f(x))f3n (t) dt '

L: l f(x f(x) l f3n (t) dt 1 l (x + t) - f(x) l f3n (t) dt + 1 =

+ t) -

l r i
f

l r l=o:o

l f(x

+ t ) - f (x) l f3n (t) dt.

The first integral is < E /2, while the second is at most 2M � r l =o:o f3n ( t) dt . By ( 10) , the second integral is < E/2 when n is large. Thus Pn =t f as claimed. D

Uniform Approximation in C 0

Section 4

223

Next we see how to extend this result to functions defined on a com­ pact metric space M instead of merely on an interval. A subset A of C 0 M = C0 (M, JR) is a function algebra if it is closed under addition, scalar multiplication, and function multiplication. That is, if f, g E A and c is a constant then f + g, cf, and f · g belong to A. For example, the set of polynomials is a function algebra. The function algebra vanishes at a point p if f (p) = 0 for all f E A. For example, the function algebra of all polynomials with zero constant term vanishes at x = 0. The function algebra separates points if for each pair of distinct points p 1 , p2 E M there is a function f E A such that f (p J ) i= f(pz) . For example, the function algebra of all trigonometric polynomials separates points of [0, 2:rr ) and vanishes nowhere.

If M is a compact metric space and A 0 is a function algebra in C M that vanishes nowhere and separates points then A is dense in C 0 M. 19 Stone-Weierstrass Theorem

Although the Weierstrass approximation theorem is a special case of the Stone-Weierstrass theorem, the proof of the latter does not stand on its own ; it depends crucially on the former. We also need two lemmas. 20 Lemma IfA vanishes nowhere and separates points then, given distinct points PI , pz E M, and given constants c 1 , c2 , there exists afunction f E A such that j (p J ) = CJ and f(pz) = cz.

g 1 , gz E A that satisfy g1 ( pd i= 0 i= gz (pz). Then g = belongs to A and g (pt ) i= 0 i= g (pz) . Let h E A separate P I · p z , g f + gi and consider the matrix

Proof Choose

H=

(p J ) [ac abcd] [gg (pz) =

By construction a , c i= 0 and b i= d. Hence det H H has rank 2, and the linear equations

ac(d - b) i= 0,

a� + abT} c� + ed T]

J

g ( p J )h (p J ) . g (pz)h (pz ) acd - abc

= Ct =

cz

have a solution (� . TJ). Then f = �g + T}gh belongs to A and f (p1 ) = c1 ,

f(pz)

=

D

cz .

21 Lemma

The closure ofa function algebra in C 0 M is a function algebra.

Proof Clear enough.

D

Function Spaces

224

Chapter 4

Proof of the Stone- Weierstrass Theorem. Let A be a function algebra in

C 0 M that vanishes nowhere and separates points. We must show that A is dense in C 0 M: given F E C0 M and E > 0 we must find G E A such that for all x E M, F (x) - E < G (x) < F (x ) + E .

(1 1) First we observe that

(12)

f EA

=>

lfl E

A

where A denotes the closure of A in C 0 M. Let E > 0 be given. According to the Weierstrass approximation theorem, there exists a polynomial p (y) such that E ( 1 3) sup{ l p(y) - l y l l : I Y I � ll f ll l < 2 After all, I y I is a continuous function defined on the interval [- I f I , I f II ] . The constant term of p (y) i s at most E/2 since l p (O) - 101 1 < E/2. Let q (y) = p (y) - p (O) . Then q (y) is a polynomial with zero constant term and ( 1 3) becomes ( 14)

l q (y) - l y / 1 < E .

Lemma 2 1 states that A i s an algebra, so g E A . t Besides, i f x E M and y = f (x) then

l g (x) - l f (x) / 1 = l q (y) - I Y I I < E . Hence I f I E A = A as claimed in ( 1 2). Next we observe that if f, g belong to A, then max (f, g) and min(f, g) also belong to A. For max(f, g) = min(f, g) =

f+g If - gl + 2 2 f g If gl

;

�

-

_

t Since a function algebra need not contain constant functions. it was important that q has no constant term One should not expect that g = ao + u1 f + + an f n belongs to A. · · ·

Uniform Approximation in C 0

Section 4

225

Repetition shows that the maximum and minimum of any finite number of functions in A also belongs to A. Now we return to ( 1 1 ) . Let F E C 0 M and E > 0 be given. We are trying to find G E A whose graph lies in the E -tube around the graph of F. Fix any distinct points p, q E M. According to Lemma 20 we can find a function in A with given values at p, q , say Hpq E A satisfies Hpq (p)

=

F (p)

and

Hpq (q ) = F (q ) .

Fix p and let q vary. Each q E M has a neighborhood Uq such that x E Uq

( 1 5)

::::::>

F (x) - E < Hpq (x ) .

For Hpq (x ) - F (x ) + E is a continuous function of x which is positive at x = q . The function Hpq locally supersolves ( 1 1 ) . See Ftgure 92.

Hpq(p) = F(p)

q

Figure 92 In a neighborhood of q ,

Hpq supersolves ( l l ) in the sense of

(1 5). Compactness of M implies that finitely many of these neighborhoods Uq cover M, say Uq 1 , , Uq" . Define •

•

•

G p (X )

Then Gp E A and

=

max (Hpq 1 (x ) , . . . , Hpq" (x ) ) .

Function Spaces

226

G p (p)

( 1 6)

for all x

=

F (p)

and

F(x) - E

Chapter 4

G p (x)

<

E M. See Figure 93.

q,

Figure 93 Gp is the maximum of Hpq; • i

=

1,

. . . , n.

Continuity implies that each p has a neighborhood VP such that { 1 7)

x E Vp

==>

Gp (x) < F (x) + E .

See Figure 94. By compactness, finitely many of these neighborhoods cover

p

Figure 94 Gp (p ) = F (p) and Gp supersolves { 1 1 ) everywhere.

Uniform Approximation in C 0

Section 4

227

Figure 95 The graph of G lies in the € -tube around the graph of

F

M,

say VPI ' - - - . VPm ' Set G (x) = min(Gp 1 (x) , . . . , G Pm (x ) ) . We know that G E A and ( 1 6), ( 1 7) imply ( 1 L ) . See Figure 95.

22 Corollary Any 2rr -periodic continuous function of x formly approximated by a trigonometric polynomial

T (x) =

E

0

lR can be uni­

n

n

L ak cos kx + L bk sin kx

.

k=O k=O Proof Think of [0, 2rr) parameterizing the circle S 1 by X !---+ (cos X ' sin x ) . The circle is compact, and 2rr -periodic continuous functions o n lR become continuous functions on S 1 . The trigonometric polynomials on S 1 form an algebra T c C 0 S 1 that vanishes nowhere and separates points. The 0 Stone-Weierstrass theorem implies that T is dense in C 0 S 1 . Here is a typical application of the Stone-Weierstrass Theorem: Consider a continuous vector field F : � -+ JR2 where � is the closed unit disc in the plane, and suppose that we want to approximate F by a vector field that vanishes (equals zero) at most finitely often. A simple way to do so is to approximate F by a polynomial vector field G . Real polynomials in two variables are finite sums

P (x . y) =

n

L ciix ; yi

i , j =O

where the Cij are constants. They form a function algebra A in C 0 ( � , JR) that

Chapter 4

Function Spaces

228

separates points and vanishes nowhere. By the Stone-Weierstrass Theorem, A is dense in C 0 , so we can approximate the components of F = (F1 , F2 ) by polynomials F1 = P Fz = Q .

The vector field ( P , Q) then approximates F . Changing the coefficients of P by a small amount ensures that P and Q have no common polynomial factor and F vanishes at most finitely often.

5

Contractions and ODE 's

Fixed point theorems are of great use in the applications of analysis, includ­ ing the basic theory of vector calculus such as the general implicit function theorem. If f : M --+ M and for some p E M, f( p ) = p, then p is a fixed point of f . When must f have a fixed point? This question has many answers, and the two most famous are given in the next two theorems. Let M be a metric space. A contraction of M is a mapping f : M --+ M such that for some constant k < 1 and all x, y E M,

f(y)) � kd(x, y). 2 3 Banach Contraction Principle Suppose that f M d(f (x),

: --+ M is a con­ traction and the metric space M is complete. Then f has a unique fixed point p and for any x E M, f" (x) = f o f o · · · o f(x) � p as n --+ 00.

Brouwer Fixed Point Theorem Suppose that uous where Bm is the closed unit ball in Rm. p E Bm.

f

:

Bm --+ Bm

Then

f

is contin­ has a fixed point

The proof of the first result is easy, the second not. See Figure 96 to picture a contraction. Proof #1 of the Banach Contraction Principle Beautiful, simple, and dy­

namical ! See Figure 96. Choose any xo claim that for al1 n E N

E M and define Xn = f" (xo). We

( 1 8) This is easy:

d(xn , Xn+l ) � kd(f(Xn-J ), f(X71 ) ) � k2 d(f(Xn-z ), f(Xn-1 )) n � � k d(xo , xi ) . ·

·

·

From this and a geometric series type of estimate, it follows that the se­ quence (xn ) is Cauchy. For let E > 0 be given. Choose N large enough

Section 5

Contractions and ODE's

\

� X

•

•

\

fi

•

'1

/

Figure 96

Xo •

•

I

y

• Jy

•

Xn

••

r

•

229

/ fM

M

� -----

•

p

"

�

f contracts M toward the fixed point p

that

kN < E -1 - k d(xo, xt) . Note that ( 1 9) needs the hypothesis k < 1 . If N ::::: ( 1 9)

::::

m

n then

d(Xm , Xn) ::::: d(Xm , Xm+l) + d(Xm+ l • Xm+2 ) + · · · + d(Xn-t . Xn ) ::::: k"'d(xo, X J ) + km +ld(xo, xt ) + + kn -ld(xo, Xt ) ::::: km (l + k + · · · + kn - m -l)d(xo, X t) k :::: kN ( L kl ) d(xo, xt ) = 1 _N k d(xo, Xt) < E . l=O Thus (xn) is Cauchy. Since M is complete, Xn converges to some p E M ·

·

·

oo

as n --* oo . Then

d(p, f(p )) ::::: d( p, Xn) + d(Xn , f(xn)) + d(f(xn ), f ( p )) ::::: d(p , Xn ) + kn d(xo, xt) + kd(xn , p ) 0 as n --* --*

00 .

Since d (p, f(p ) ) i s independent of n, d( p , f( p)) = 0 and p = f(p ) . This proves the existence of the fixed point. Uniqueness is immediate. After all, how can two points simultaneously stay fixed and move closer together? D

230

Function Spaces

Chapter 4

Proof #2 of the Banach Contraction Principle - sketch Choose any point xo E M and choose ro so large that f(M,.0 (xo)) C Mr0 (xo ) - Let B0 =

= t n (Bn - d - The diameter of Bn is at most kn diam(B0 ) , and this tends to 0 as n --+ oo . The sets Bn nest downward as n --+ oo and f sends Bn inside Bn +I · Since M is complete, this implies that n Bn is a single point, say p, and f ( p ) = p . 0

Mr (xo ) and Bn 0

Proof of Brouwer's Theorem in dimension one The closed unit 1 -ball is the interval L - 1 , 1 ] in JR. If f : [ - 1 , l ] --+ [ - 1 , 1 ] is continuous then

so is g(x ) = x - f(x). At the endpoints ± 1 , we have g( - 1 ) ::::= 0 ::::= g( l ) . By the intermediate value theorem, there is a point p E [- 1 , 1 ] such that 0 g(p) = 0. That is, f(p) = p.

The proof in higher dimensions is harder. One proof is a consequence of the general Stokes ' Theorem. and is given in Chapter 5 . Another depends on algebraic topology, a third on differential topology. Ordinary Differential Equations

The qualitative theory of ordinary differential equations (ODE's) begins with the basic existence/uniqueness theorem in ODE's, Picard's Theorem. Throughout, U is an open subset of m-dimensional Euclidean space :!Rm . In geometric terms. an ODE is a vector field F defined on U . We seek a trajectory Y of F through a given p E U , i.e., Y : (a , b) --+ U is differentiable and solves the ODE with initial condition p . (20)

Y ' (t) = F(Y (t)) and Y (O) =

p.

See Figure 97. In this notation w e think o f the vector field F defining at each x E U a vector F (x) whose foot lies at x and to which Y must be tangent. The vector Y ' (t) is (Y; (t) , . . . . Y� (t)) where Y 1 , Y m are the components of Y . The trajectory Y (t) describes how a particle travels with prescribed velocity F. At each time t, Y (t) is the position of the particle; its velocity there is exactly the vector F at that point. Intuitively, trajectories should exist because particles do move. The contraction principle gives a way to find trajectories of vector fields, or - what is the same thing to solve ODE ' s. We will assume that F satisfies a Lipschitz condition - there is a constant L such that for all points x, y E U I F(x) - F (y) l ::::: L l x - y j . .

Here, I

I refers to the Euclidean length of a vector.

F,

x,

.

.

•

y are all vectors

Contractions and ODE's

Section 5

23 1

Figure 97 Y is always tangent to the vector field

F.

in :!Rm . It follows that F is continuous. The Lipschitz condition is stronger than continuity, but still fairly mild. Any differentiable vector field with a bounded derivative is Lipschitz.

Given p E U there exists an F -trajectory Y (t) in This means that Y : (a . b) � U solves (20). Locally, Y zs

24 Picard's Theorem

U through unique.

p.

To prove Picard's Theorem it is convenient to re-express (20) as an inte­ gral equation and to do this we make a brief digression about vector-valued integrals. Let's recall four key facts about integrals of real valued functions of a real variable, y = f(x ) , a ::::: x ::::: b. (a) J: f (x) dx is approximated by Riemann sums R = L f (tk) D.xk. (b) Continuous functions are integrable. (c) If f'(x) exists and is continuous then J: f'(x) dx = f(b) - f(a) .

jJ:

l

(d) f (x) dx ::::: M(b - a) where M = sup J f (x) J . The Riemann sum R in (a) has a = x0 ::::: · · · ::::: Xk- 1 ::::: Xn = b and all the D.xk = Xk - Xk - 1 are small. Given a vector-valued function of a real variable

a

:::::

x

tk ::::: xk

:::::

· · ·

:::::

j(x ) = ( /1 (x) . . . . . fm (x)) .

:::::

1b f (x) dx (1b j, (x) dx , . . . , 1b fm (x) dx)) .

b, we define its integral componentwise as the vector of integrals =

Function Spaces

232

Chapter 4

J:

Corresponding to (a) - (d) we have (a') f(x) dx is approximated by R = (R 1 , . . . , Rm ), with Rj a Riemann sum for [j . (b') Continuous vector-valued functions are integrable. (c') If f'tx) exists and is continuous, then J: f'(x) dx = f(b) - f(a).

l

I J:

(d' ) f(x) dx ::=: M (b - a) where M = sup l f(x) l . (a'), (b' ), (c') are clear enough. To check (d') we write

R = =

L Rjej = L L [j (tk) D.. xkej j k L L [j (tk )ej D.. Xk = L f(tk) D.. xk k k j

where e1 , . . . , em is the standard vector basis for :!Rm . Thus,

IRI

:S

L l f(tk ) l D.. xk k

::=:

L M D.. xk = M (b - a ) . k

By (a' ), R approximates the integral, which implies (d'). (Note that a weaker inequality with M replaced by ,JfiiM follows immediately from (d). This we aker inequality would suffice for most of what we do - but it is inele­ gant.) Now consider the following integral version of (20),

Y ( t) = p +

(2 1 )

1 ' F(Y (s) ) ds.

A solution of (2 1 ) is by definition any continuous curve Y : (a. b) � U for which (2 1 ) holds identically in t E (a , b). By (b') any solution of (2 1 ) i s automatically differentiable and its derivative i s F ( Y (t)) . That is. any solution of (2 1 ) solves (20) . The converse is also clear, so solving (20) is equivalent to solving (2 1 ). Proof of Picard's Theorem Since

F is continuous. there exists a compact (p) and a constant M such that I F (x) I ::=: M for all

neighborhood N = N r > 0 such that

x E N . Choose r (22)

r M ::=:

r

and r L < 1 .

Consider the set (3 of all continuous functions respect to the metric

Y

[-r, r]

d( Y , a) = sup{I Y (t) - a (t) l : t E [- r. r ] }

�

N.

With

Contractions and ODE's

Section 5

233

the set e is a complete metric space. Given y E e. define

(Y ) (t) =

p

+ for F(Y (s)) ds.

Solving (2 1 ) is the same as finding Y such that (Y) = Y. That is, we seek a fixed point of . We just need to show that is a contraction of e . Does send e into itself? Given Y E e we see that (Y ) (t ) is a continuous (in fact differentiable) vector-valued function of t and that by (22),

l (Y) (t) - P i =

1 1 1 F (Y (s)) ds l .:s rM

Therefore, does send e into itself. contracts e because

d( (Y ) , (u )) = s�p � �

�

r .

1 1 t F (Y (s)) - F (u (s)) ds l

T

sup I F (Y (s)) - F(u (s) ) l

r

sup L I Y (s) - u (s) l � rLd(Y , u ) s

and T L < 1 by (22). Therefore has a fixed point Y . and (Y) = Y implies that Y (t) solves (2 1), which implies that Y is differentiable and solves (20). Any other solution u (t) of (20) defined on the interval [ - r, r] also solves (2 1 ) and is a fixed point of , ( u ) = u . Since a contraction mapping has a unique fixed point, Y = u , which is what local uniqueness means. 0 The F -trajectories define a flow in the following way: To avoid the pos­ sibility that trajectories cross the boundary of U (they "escape from U") or become unbounded in finite time (they "escape to infinity") we assume that U is all of JRm . Then trajectories can be defined for all time t E JR. Let Y (t , p) denote the trajectory through p. Imagine all points p E JRm moving in unison along their trajectories as t increases. They are leaves on a river, motes in a breeze. The point PI = Y (ti . p ) at which p arrives after time t i moves according to Y (t, pi). Before p arrives at P I . however, p 1 has already gone elsewhere. This is expressed by the flow equation

Y (t, PI ) = Y (t + t 1 , p).

See Figure 98. The flow equation is true because as functions of t both sides of the equation are F -trajectories through PI , and the F -trajectory through a point

Function Spaces

234

_ _

/

/

y

I

I

/

�

/

/

/

.,

.-

Chapter 4

_

PJ

_...

/

/

/ / ,. / P2

/

/

p

Figure 98 The time needed to flow from from p to

p2

times needed to flow from p to P I and from

is the sum of the to P2 ·

PI

is locally unique. It is revealing to rewrite the flow equation with different notation. Setting qJ1 (p) = Y (t, p) gives

(/Jt +s (p) = (/Jr ( (/Js (p)) for all t, s E ffiL

(/)1 is called the t-advance map. It specifies where each point moves after time t . See Figure 99. The flow equation states that t �---+ (/Jr is a group /

/

/

/

/

Jf

/

/

"

/ /

"

/ /

Jf

/

Jf

/

"

Jf

Figure 99 The t-advance map shows how a set A flows to a set qJ1 (A ) .

homomorphism from ffi. into the group o f motions o f ffi.m . In fact each (/)1 i s a homeomorphism o f ffi.m onto itself and its inverse i s qJ_1 • For (/J- r o (/)1 = ({Jo and q;0 is the time-zero map where nothing moves at all, ({Jo = identity map.

Analytic Functions

Section 6*

6*

235

Analytic Functions

Recall from Chapter 3 that a function f : (a , b) � JR. is analytic if it can be expressed locally as a power series. For each x E (a , b) there exists a convergent power series L ck h k such that for all x + h near x,

f (x + h )

00

=

:I:Ck hk . k =O

As we have shown previously, every analytic function is smooth but not every smooth function is analytic. In this section we give a necessary and sufficient condition that a smooth function be analytic. It involves the speed with which the r th derivative grows as r � oo. Let f : (a , b) � JR. be smooth. The Taylor series for f at x E (a , b) is

Let I = [x - a, x + a ] be a subinterval of (a , b) , a > 0, and denote by Mr the maximum of l !(r ) (t) l for t E I . The derivative growth rate of f on I is

.

a = hm sup

r ---> 00

Clearly,

r i t (x ) l f r !

::S:

�Mr !r

-.

:./Mr fr!, s o the radius o f convergence

R=

------,====

lim sup

r I J< r > (x) l r!

of the Taylor series at x satisfies 1 - ::S:

a

R.

In particular, if a is finite the radius of convergence of the Taylor series is positive. 25 Theorem If a a < 1 , th e interval I.

un

then the Taylor series converges uniformly to f

Function Spaces

236

Proof Choose 8 > 0 such that (a + 8)a <

Chapter 4 I.

The Taylor remainder formula from Chapter 3 , applied to the (r - l ) st order remainder, gives

f (x + h )

r- 1 - " L., k=O

f (k) (X ) k h k!

=

f (r ) ((} )

r!

hr

M (( M ) a)

for some (} between x and x + h. Thus, for r large

f(x + h )

f (k) (x ) k h k'

r-1

-

L k=O

•

� -f a r =

r.

-f

1 /r

r.

r � ( (a + 8 )a Y .

Since (a + 8) a < 1 , the Taylor series converges uniformly to f (x + h ) 0 on / . 26 Theorem If f

is expressed as a convergent power series f (x + h ) = k L qh with radius of convergence R > a, then f has bounded derivative growth rate on I. The proof of Theorem 26 uses two estimates about the growth rate of factorials. If you know Stirling ' s formula they are easy, but we prove them directly. (23)

Taking logarithms, applying the integral test, and ignoring terms that tend to zero as r � oo gives 1

1

1

- (Iog r r - 1og r !) = 1og r - - (I og r + log(r - 1 ) + · · · + log 1 )

r

�

log r - 1

r

1

r

r 1

log x dx = log r - - (x log x - x)

1

r

= 1 - -,

r

which tends to

I

as r

�

oo. This proves (23).

l

r 1

Analytic Functions

Section 6*

f G) "Ak f k (k - l ) (k k=r 00

237

To prove (24) we write "A = e-J.L for Jl > 0, and reason similarly: =

k =r

< - '"""' e e- J.L r .' � k l

(

2� · !

k

.

. (k - r +

l -

r .'

1 00

l)

k e - J.L

X r e -J.LX dx

r r rxr - l r (r - 1 ) x r -1 x x = e - J.L - + -- + Jl2 Jl Jl 3 r! ] r+ l 1 < -! e -w ( r + 1 ) r r - r min( 1 , Jl)

=r

(

)

-

2

r! + · · · + -Jl r + l

) 1 00 r

According to ( 23 ) the rth root of this quantity tends to e 1 -J.L I min ( 1 , Jl) as r -+ oo, completing the proof of (24 ).

Proof of Theorem 26 By assumption the power series L ck h k has radius of convergence R and a < R Since 1 I R is the lim sup of as k -+ oo, there is a number "A < 1 such that for all large k, ! q a k :::=: "A k . Differentiating

00

the series term by term with l h I :::=: a gives

! J (r) (x + h ) !

:S

!

-YfCkT

L k (k - 1 ) (k - 2 ) . . . (k - r + 1 ) ! ck h k r ! -

k=r

for r large. Thus,

According to ( 24) ,

and f has bounded derivative growth rate on I .

D

From Theorems 25 and 26 we deduce the main result of this section. 27 Analyticity Theorem A smooth function is analytic locally bounded derivative growth rate.

if and only if it has

Function Spaces

238

Chapter 4

(a , b) --+ lR is smooth and has locally bounded derivative growth rate. Then x E (a , b ) has a neighborhood N on which the derivative growth rate a is finite. Choose a > 0 such that I = Lx - a , x + a ] C N and a a < 1 . We infer from Theorem 25 that the Taylor series for f at x converges uniformly to f on l . Hence f is analytic. Conversely, assume that f is analytic and let x E (a . b) be given. There is a power series L ck h k that converges to f (x + h) for all h in some interval Proof Assume that

f

(- R . R), R > 0. Choose a , 0 < a < R . We infer from Theorem 26 that 0 f has bounded derivative growth rate on I .

28 Corollary

bounded.

A smooth function is analytic if its derivatives are uniformly

M

An example of such a function is f (x) = sin x .

I f(r) ( fJ) I ::=:: for all r and f) then the derivative growth rate of f is bounded, in fact a = 0 and R = oo. 0

Proof [f

lf f (x) = L ckx k and the power series has radius of convergence R, then f is analytic on ( - R , R). 29 Taylor's Theorem

Proof The function f is smooth, and by Theorem 26 it has bounded deriva­

tive growth rate on I

c

( R , R ) . Hence it is analytic. -

0

Taylor's Theorem states that not only can f be expanded as a convergent power series at x = 0, but also at any other point x0 E (- R , R) . Other proofs of Taylor's theorem rely more heavily on series manipulations and Mertens' theorem. The concept of analyticity extends immediately to complex functions. A function f : D --+ C is complex analytic if D is an open subset of C and for each z E D there is a power series

such that for all

z

+ i; near z ,

f (z + i;)

=

00

L::Ck i; k . k=O

The coefficients ck are complex and so is the variable i; . Convergence occurs on a disc of radius R . This lets us define ez , log z, sin z, cos z for the complex

Section 6*

239

Analytic Functions

number z by setting log ( l + z) = cos z =

=

L k=

( - l ) k + I zk

l

k

when l z l < 1

oo ( - 1 ) k z 2k z= (2k ) ! k=O

It is enlightening and reassuring to derive formulas such as

e i e = cos () + i sin () directly from these definitions. (Just plug in z = i() and use the equations i 2 = - 1 , i 3 = -i , i 4 = 1 , etc.) A key formula to check is e z + w = e'e w .

One proof involves a manipulation of product series, a second merely uses analyticity. Another formula is log(e') = z . There are many natural results about real analytic functions that can be proved by direct power series means ; e.g., the sum, product, reciprocal, composite. and inverse function of analytic functions are analytic. Direct proofs, like those for the Analyticity Theorem above, involve major series manipulations. The use of complex variables leads to greatly simplified proofs of these real variable theorems, thanks to the following fact.

Real analyticity propagates to complex analyticity and complex analyticity is equivalent to complex differentiability. t

For it is relatively easy to check that the composition, etc., of complex differentiable functions is complex differentiable. The analyticity concept extends even beyond
x ' = Ax in calculus. A is a given m x m matrix and the unknown solution x = x (t) is a vector function of t , on which an initial condition x (0) = x0 is usually imposed. A vector ODE is equivalent to m coupled, scalar, linear ODE's. The solution x (t) can be expressed as

x (t) = e t A x0 t A function f : D ---+ C is complex differentiable or holomorphic if D is an open subset of C and for each z E D, the limit of

!'if l'iz

exists as

=

f (z

+

l'iz) - / (-:..} l'iz

/'i z --+ 0 in C. The limit, if it exists, is a complex number.

Function Spaces

240

Chapter 4

where

oo k I 1 (I + t A + - (t A ) 2 + · · · + - (t A ) n ) = " !.._ A k . e t A = nlim � oo 2! n! � k!

I i s the m x m identity matrix. View this series a s a power series with kth coefficient t k I k ! and variable A . ( A is a matrix variable ! ) The limit exists in the space of all m x m matrices, and its product with the constant vector x0

does indeed give a vector function of t that solves the original linear ODE. The previous series defines the exponential of a matrix as eA = L A k I k ! . You might ask yourself - i s there such a thing a s the logarithm o f a matrix? A function that assigns to a matrix its matrix logarithm? A power series that expresses the matrix logarithm? What about other analytic functions? Is there such a thing as the sine of a matrix? What about inverting a matrix? Is there a power series that expresses matrix inversion? Are formulas such as log A 2 = 2 log A true? These questions are explored in nonlinear functional analysis. A terminological point on which to insist is that the word "analytic" be defined as "locally power series expressible." In the complex case, some mathematicians define complex analyticity as complex differentiability, and although complex differentiability turns out to be equivalent to local ex­ pressibility as a complex power series, this is a very special feature of
7*

Nowhere Differentiable Continuous Functions

Although many continuous functions, such as lx l , $, and x sin ( l lx) fail to be differentiable at a few points, it is quite surprising that there can exist a function which is everywhere continuous but nowhere differentiable. 30 Theorem There exists a continuous function f derivative at no point whatsoever.

:

lR

�

lR that has a

Proof The construction is due to Weierstrass. The letters k, m , integers. Start with a sawtooth function a0 : lR � lR defined as a0 (x )

=

l

x - 2n 2n + 2 - x

if 2n :::=: x s 2n + l if 2n + 1 ::::: x ::::: 2n + 2.

n denote

Section 7*

Nowhere Differentiable Continuous Functions

cr0 is periodic with period 2;

compressed sawtooth function

if

crk (x ) =

t =

x

+ 2m

then cro (t)

(�) k cr0 (4kx )

24 1

cro (x ) . The

has period :rrk = 2j4k . If t = x + m:rrk then crk (t) = crk (x) . See Figure 1 00.

ao(x)

Figure 100 The graphs of the sawtooth function and two compressed

sawtooth functions. According to the limit f, and

M -test, the series L crk (x) converges uniformly to a f (x) =

00

L crk (x) k=O

is continuous. We claim that f is nowhere differentiable. Fix an arbitrary point x, and set 8n = 1 /2 · 4n . We will show that

Function Spaces

242

Chapter 4

f (x ± 8n ) - f (x )

f).j

does not converge to a limit as 8n The quotient is

f

f).j = /).x k=v ---"

---+

0, and thus that f ' (x ) does not exist.

O"k (X ±

Dn) 8n

O"k (X )

.

There are three types of term in the series, k > n , k = n , and k < n . If k > n then o-k (x ± 8n ) O"k (x ) = 0 - for 8n is an integer multiple of the period of o-k o -

�

_

On -

1

__

2 4n •

4k-(n + l )

_ -

.

� 4k

-

_

4

k-(n + I ) . "k· ...-

Thus the infinite series expression for f).jI f).x reduces to a sum of n + I terms

The function O"n is monotone on either [x - 8n , x ] or [x , x + 8n ] , since it is monotone on intervals of length 4 -n and the contiguous interval [x - 8n x , x + 8n ] at x is of length 4 -n . The slope of a-n is ±3n . Thus, either ,

or The terms with k k ±3 : Thus

I I f).j f).x

�

<

n

are crudely estimated from the slope of O"k being

3 n - (3 n- l +

which tends to oo as 8n

---+

·

·

·

+ 1 ) = 3n

-

3n 1 1 n � = 2 (3 + 1 ) , -

0, so f ' (x ) does not exist.

D

Weierstrass showed that a nowhere differentiable continuous function exists by simply writing a formula for it. Yet more amazing is the fact thar most continuous functions (in a reasonable sense defined below) are

Section 7*

Nowhere Differentiable Continuous Functions

243

nowhere differentiable. If you could pick a continuous function at random, it would be nowhere differentiable. Recall that the set D c M is dense in M if D meets every non-empty open subset W of M, D n W f= 0. The intersection of two dense sets need not be dense; it can be empty, as is the case with Q and Qc in JR. On the other hand if U, V are open dense sets in M then U n V is open dense in M. For if W is any non-empty open subset of M then U n W is a non-empty open subset of M, and by denseness of V, V meets U n W ; i.e., U n V n W is non-empty and U n V meets W . Moral Open dense sets do a good job o f being dense.

The countable intersection G = nc n of open dense sets is called a thick (or residual t ) subset of M, due to the following result, which we will apply in the complete metric space C 0 ([a , b] , JR.) . Extending our vocabulary in a natural way we say that the complement of a thick set is thin (or meager). A subset H of M is thin if and only if it is a countable union of nowhere dense closed sets, H = U Hn . Clearly, thickness and thinness are topological properties. A thin set is the topological analog of a zero set (a set whose outer measure is zero).

31 Baire's Theorem Every thick subset of a complete metric space M is dense m M. A non-empty, complete metric space is not thin: if M is the union of countably many closed sets, at least one has non-empty interior.

If all points in a thick subset of M satisfy some condition then the con­ dition is said to be generic, we also say that most points of M obey the condition. As a consequence ofBaire 's theorem and a modification ofWeier­ strass' construction we will prove

32 Theorem

The generic f E C 0 = C 0 ([a , b ] , JR.) is differentiable at no point of [a . b ], nor is it monotone on any subinterval of l a . b 1 -

Using Lebesgue's monotone differentiation theorem (monotonicity im­ plies differentiability almost everywhere ), the second assertion follows from the first, but below we give a direct proof. Before getting into the proofs of Baire's theorem and Theorem 32, we further discuss thickness, thinness, and genericity. The empty set is always thin and the full space M is always thick in itself. A single open dense subset is thick and a single closed nowhere dense subset is thin . JR. \ Z is a thick subset of JR. and the Cantor set is a thin subset of JR. Likewise JR. is a

t "Residual" is an unfonunate choice of words. It connotes smallness, when !he opposite.

n

should connote just

Function Spaces

244

Chapter 4

thin subset of �2 • The generic point of � does not lie in the Cantor set. The generic point of �2 does not lie on the x-axis . Although � \ Z is a thick subset of � it is not a thick subset of �2 • The set Q is a thin subset of �­ It is the countable union of its points, each of which is a closed nowhere dense set. Qc is a thick subset of R. The generic real number is irrational. In the same vein: (a) The generic square matrix has determinant =I= 0. (b) The generic linear transformation �m ---+ �m is an isomorphism. (c) The generic linear transformation �m ---+ �m - k is onto. (d) The generic linear transformation �m ---+ JR.m +k is one-to-one. (e) The generic pair of lines in JR3 are skew (nonparallel and disjoint). (f) The generic plane in �3 meets the three coordinate axes in three distinct points. (g) The generic n th degree polynomial has n distinct roots . In an incomplete metric space such as Q, thickness and thinness have no bite: every subset of Q, even the empty set, is thick in Q.

Proof of Haire's Theorem If M = 0, the proof is trivial, so we assume M =I= 0. Let G = nG n be a thick subset of M; each G n being open dense in M. Let Po E M and E > 0 be given. Choose a sequence of points Pn E M and radii rn > 0 such that rn < 1 / n and

M2q ( p i ) c M"' (po ) M2r2 ( P2 ) C Mr1 (p d n G t M2rn ( Pn + d

C

Mrn (Pn ) n G t · · · n Gn .

See Figure 1 0 1 . Then

ME (po )

:J

Mr J (PI )

:J

Mrz (P2 )

:J

·

·

·

·

The diameters of these sets tend to 0 as n ---+ oo . Thus (pn) is a Cauchy sequence and it converges to some p E M, by completeness. The point p belongs to each set M rn ( Pn ) and therefore it belongs to each G n . Thus p E G n ME (p0 ) and G is dense in M. To check that M i s not thin, w e take complements. Suppose that M = UKn and Kn is closed. If each Kn has empty interior, then each G n = K� is open-dense, and

a contradiction to density of G.

0

245

Nowhere Differentiable Continuous Functions

Section 7*

Figure 101 The closed neighborhoods Mrn (pn ) nest down to

33 Corollary

and thin.

a

point.

No subset ofa complete non-empty metric space is both thick

Proof If S is both a thick and thin subset of M then M \

S is also both

thick and thin. The intersection of two thick subsets of M is thick, so

0 = S n (M \ S) is a thick subset of M. By Baire's theorem, this empty set D is dense in M, so M is empty.

To prove Theorem 32 we use two lemmas. 34 Lemma

The set P L ofpiecewise linear functions is dense in C 0 .

Proof If � :

---* IR is continuous and its graph consists of finitely many line segments in :IR2 then � is piecewise linear. Let f E C0 and E > 0 be given. Since [a , b] is compact, f is uniformly continuous, and there is 8 > 0 such that I t - s l < 8 implies l f (t) - f (s) l < E . Choose n > (b - a)/8 and partition [a , b] into n equal subintervals /i = [xi - l , xd, each of length < 8. Let � : [a , b] ---* :IR be the piecewise linear function whose graph consists of the segments joining the points (xi 1 , f (xi 1 )) and

[a, b]

_

_

(xi , f (xi )) on the graph of f. See Figure 102 . The value of � (t) for t E h lies between f (x i - d and f (x i ) . Both these numbers differ from f (t) by less than E . Hence, for all t E [a , b],

Function Spaces

246

Chapter 4

X;

Xi - 1

Figure 102 The piecewise linear function ¢ approximates the continuous

function f.

l f (t) - f/J (t) l < E. In other words, d (f, ¢) < E and P L is dense in C 0 .

D

35 Lemma If ¢ E P L and E > 0 are given then there exists a sawtooth function a such that II a II ::; E, a has period ::; E, and I

min{ l slope a l } > max{ l slope ¢ 1 } + - . Proof Let () = max{ l slope ¢ 1 } and choose

E

c large. The compressed saw­ tooth a (x ) = Ea0 (cx) has l l a ll = E , period rc = l jc, and slope s = ±Ec. D When c is large. rc ::; E. and l s i > () + l jE .

Proof of Theorem 32 For n E

Rn = { f E C 0 : Vx E [a , b L n = { f E C 0 : Vx E [a +

N define

�]

3h

> 0 such that

� · b ] 3 h < 0 such that

I D..: I I D..: I

>

n}

>

n}

1 G n = { f E C 0 : f restricted to any interval of length - is non-monotone} ,

D..

n

where f = f (x + h) - f (x) . We claim that each of these sets is open dense in C0 .

Section 7*

Nowhere Differentiable Continuous Functions

247

For denseness it is enough to prove that the closure of each contains P L , since by Lemma 34 the closure o f P L i s C 0 Let cp E P L and E > 0 be given_ According to Lemma 35, there is a sawtooth function a such that l l a l l � E , a has period < 1 / n , and _

min{ l slope a I }

>

max{ l slope c/J I } + n .

Consider the piecewise linear function f = cp + a . Its slopes are dom­ inated by those of a , and so they alternate in sign with period < 1 j2n . At any x E [a , b + 1 / n ] there is a rightward slope either > n or < - n _ Thus f E Rn . Similarly f E L n . Any interval I of length 1 / n contains in its interior a maximum or minimum of a , and so it contains a subinterval on which f strictly increases, and another on which f strictly decreases. Thus, f E Gn . Since d(j. cp) = E is arbitrarily small this shows that Rn , L n . and G n are dense in C 0 . Next suppose that f E Rn is given. For each x E [a . b - 1 /n ] there is an h = h (x) > 0 such that

Since f is continuous, there is a neighborhood Tx of constant v = v > 0 such that this same h yields

I

f (t + h) h

-

x

in [a . b] and a

I >n+v

f(t)

for all t E Tx . Since [a , b - 1 / n J is compact, finitely many of these neigh­ borhoods Tx cover it, say Tx1 , • • • , Txm . Continuity of f implies that for all t E T Xi ' (25)

I

f (t + h i ) - f (t) h I-

I

::: n + vi ,

where hi = h (xi ) and vi = v (xd . These m inequalities for points t in the m sets Tx, remain nearly valid if f is replaced by a function g with d (j g) small enough; (25) becomes (26) which means that g E Rn and Rn is open in C 0 • Similarly L n

is

open in C 0 •

Function Spaces

248

Chapter 4

Checking that G n is open is easier. If (jk ) is a sequence of functions in G� and fk :::::::t f then we must show that f E G� . Each fk is monotone on some interval h of length 1 In. There is a subsequence of these intervals that converges to a limit interval I . Its length is l 1 n and by uniform convergence, f is monotone on I . Hence G� is closed and G n is open . Each set Rn . L n . G n is open dense in C 0 . Finally, if f belongs to the thick set

then for each x E

[a, b] there is a sequence h n

I

f (x + hn) - f (x) hn

I

f= 0 such that

>

n.

The numerator of this fraction is at most 2 II f 11. so hn -4 0 as n -4 oo. Thus f is not differentiable at x . Also, f is non-monotone on every interval of length lin. Since each interval J contains an interval of length 1/n when D n is large enough, f is non-monotone on J. Further generic properties of continuous functions have been studied, and you might read about them in the books A Primer of Real Functions by Ralph Boas, Diffe rentiation of Real Functions by Andrew Bruckner, or A Second Course in Real Functions by van Rooij and Schikhof.

8*

Spaces of Unbounded Functions

How important is it that the functions we deal with are bounded, or have domain [a, b] and target JR? To some extent we can replace [a, b] with a metric space X and lR with a complete metric space Y . Let :F denote the set of all functions f : X -4 Y . Recall from Exercise 2.84 that the metric dy on Y gives rise to a bounded metric

p (y, y ) '

=

1

dy ( y , y ' ) . + dy ( y . y')

where y, y ' E Y . Note that < 1 . Convergence and Cauchyness with p respect to and dy are equivalent. Thus completeness of Y with respect to dy impliespcompleteness with respect to p . In the same way we give :F the metric d d(f, g ) = sup y (j(x) , g (x)) XEX

1 + dy (f(x), g (x ) )

Spaces of Unbounded Functions

Section 8*

249

A function f E :F is bounded with respect to dy if and only if for any constant function c, supx dy (f(x) , c) < oo; i.e., d(f, c) < I . Unbounded functions have d (f, c) = 1 .

In the space :F equipped with the metric d, Uniform convergence of (fn) is equivalent to d-convergence. Completeness of Y implies completeness of F. The set :Fb of bounded functions is closed in F. The set C 0 ( X , Y) of continuous functions is closed in F.

36 Theorem

(a) (b) (c) (d)

f = unif lim means that dy (fn (x) , f (x)) � 0, which means n -HXl that d(fn , f) -+ 0.

Proof (a)

(b) If Un) is Cauchy in :F and Y is complete then, just as in Section 1 , f(x) nlim -+ oo fn (x ) exists for each x E X. Cauchyness with respect to the metric d implies uniform convergence and thus d ( fn , f) -+ 0. (c) If fn E :Fb and d(fn , f) -+ 0 then supx dy (fn (x), f (x )) -+ 0. Since fn is bounded, so is f. D (d) The proof that C 0 is closed in :F is the same as in Section 1 . =

The Arzela-Ascoli theorem is trickier. A family £ C :F is uniformly equicontinuous if for each E > 0 there is a 8 > 0 such that f E £ and dx (x , t) < 8 imply dy ( J (x ) , f(t)) < E . If the 8 depends on x but not on f E £ then £ is pointwise equicontinuous. 37 Theorem

Pointwise equicontinuity implies uniform equicontinuity if X

is compact. Proof Suppose not. Then there exists E > 0 such that for each 8 = 1 / n we have points Xn , tn E X and functions fn E £ with dx (Xn , tn) < 1 /n

and

Xn

dy (fn (Xn ) , fn C tn)) 2: E . By compactness of X we may assume that -+ xo . Then tn -+ xo , which leads to a contradiction of pointwise D

equicontinuity at x0 .

38 Theorem Ifthe sequence offunctions fn : X -+ Y is uniformly equicon­ tinuous, X is compact, andfor each x E X, (fn ( ) ) lies in a compact subset of Y, then (fn) has a uniformly convergent subsequence. x

D. Then the proof of the Arzela Ascoli Theorem in Section 3 becomes a proof of Theorem 38.

Proof Being compact, X has a countable dense subset

D

250

Function Spaces

Chapter 4

The space X is u -compact if it is a countable union of compact sets, X = m UX i . For example Z,
39 Theorem If X is a -compact and if
Proof Express X as UXi with Xi compact. By Theorem 37 Un l x) is uniformly equicontinuous and by Theorem 38 there is a subsequence ft.n that converges uniformly on X 1 , and it has a sub-subsequence fz.n that con­ verges uniformly on X2 , and so on. A diagonal subsequence (gm) converges uniformly on each Xi . Thus (gm) converges pointwise. If A c X is com­ pact, then (gm l A ) is uniformly equicontinuous and pointwise convergent. By the proof of the Arzela Ascoli propagation theorem. (gm l A ) converges uniformly. D

is a sequence of pointwise equicontinuous functions IR -+ IR, and for some xo E IR.
Proof Let [a . b] be any interval containing x0 . By Theorem 37, the restric­ tions of fn to [a , b] are uniformly equicontinuous, and there is a 8 > 0 such that if t, s E [a . b] then I t s l < 8 implies that l fn (t) - fn (s) l < 1 . Each point x E [a , b] can be reached in .:::: N steps of length < 8, starting at x0 , if N > (b - a) j8. Thus l fn (x ) i .:::: l fn (xo) l + N . and < fn (x ) ) is bounded for each x E JR. A bounded subset of IR has compact closure and Theorem 39 gives the corollary. D -

Exercises

25 1

Exercises

In these exercises, C 0 = C 0 ([a , b] , IR) is the space of continuous real valued functions defined on the closed interval [a, b ] . It is equipped with the sup norm, 11 / 11 = sup{ l f (x) l : x E [a , b] } . 1 . Let M, N be metric spaces. (a) Formulate the concepts of pointwise convergence and uniform convergence for sequences of functions fn : M � N. (b) For which metric spaces are the concepts equivalent? 2. Suppose that fn � f where f and fn are functions from the metric space M to the metric space N. (Assume nothing about the met­ ric spaces such as compactness, completeness, etc.) If each .fn is continuous prove that f is continuous. [Hint: Review the proof of Theorem 1 .] 3 . Let fn : [a , b] ---+ !R be a sequence ofpiecewise continuous functions, each of which is continuous at the point x0 E [a , b J. Assume that fn � f as n � oo . (a) Prove that f is continuous at xo. [Hint: Review the proof of Theorem 1 .] (b) Prove or disprove that f is piecewise continuous. 4. (a) If fn : lR � lR i s uniformly continuous for each n E N and if fn � f as n � oo, prove or disprove that f is uniformly continuous. (b) What happens for functions from one metric space to another instead of lR to lR? 5 . Suppose that fn : [a , b] � lR and fn =::t f as n � oo . Which of the following discontinuity properties (see Exercise 37 in Chapter 3) of the functions fn carries over to the limit function? (Prove or give a counter-example.) (a) No discontinuities. (b) At most ten discontinuities. (c) At least ten discontinuities. (d) Finitely many discontinuities. (e) Countably many discontinuities, all of jump type. (f) No jump discontinuities. (g) No oscillating discontinuities. * * 6 (a) Prove that C 0 and lR have equal cardinality. [Clearly there are at least as many functions as there are real numbers, for C 0 includes the constant functions. The issue is to show that there are no more continuous functions than there are real numbers ]

Function Spaces

252

Chapter 4

(b) Is the same true if we replace [a , b] with lR or a separable metric space? (c) In the same vein, prove that the collection T of open subsets of lR and lR itself have equal cardinality. 7. Consider a sequence of functions fn in C 0 • The graph G n of fn is a compact subset of JR2 • (a) Prove that Un ) converges uniformly as n --+ oo if and only if the sequence (Gn ) in lC (JR2 ) converges to the graph of a function f E C 0 . (The space /( was discussed in Exercise 2. 1 24.) (b) Formulate equicontinuity in terms of graphs. 8. Is the sequence of functions fn : lR --+ lR defined by

fn (x)

=

cos (n + x) + log( l

+

1 . n .Jn+2 s m2 (n x) ) n+2

equicontinuous? Prove or disprove. : lR --+ lR is continuous and the sequence fn (x) = f (n x ) is equicontinuous, what can be said about f? 10. Give an example to show that a sequence of functions may be uni­ formly continuous, pointwise equicontinuous, but not uniformly equi­ continuous, when their domain M is noncompact. 1 1 . If every sequence of pointwise equicontinuous functions M --+ lR is uniformly equicontinuous, does this imply that M is compact? 1 2. Prove that if £ c C 0 (M, N) is equicontinuous then so is its closure. 1 3. Suppose that Un ) is a sequence of functions lR --+ lR and for each compact subset K c IR, the restricted sequence Un I K ) is pointwise bounded and pointwise equicontinuous. (a) Does it follow that there is a subsequence of Un) that converges pointwise to a continuous limit function lR --+ lR? (b) What about uniform convergence? 14. Recall from Exercise 78 in Chapter 2 that a metric space M is chain connected if for each E > 0 and each p, q E M there is a chain P = Po, . . . . Pn = q in M such that

9. If f

d ( Pk - J , Pk ) < E for

1 :::5: k ::::: n .

A family F of functions f : M --+ lR i s bounded at p E M if the set { f (p) : f E F} is a bounded in JR. Show that M is chain connected if and only if pointwise boundedness of an equicontinuous family at one point of M implies pointwise boundedness at every point of M.

Exercises

15. A continuous, strictly increasing function

253 JL

: (0, oo) --+ (0, oo) is a modulus of continuity if JL (s) --+ 0 as s --+ 0 . A function f : [a , b] --+ IR has modulus of continuity JL if l f (s) - f (t) l :S

JL ( Is - t l) .

1 6.

17.

18.

1 9.

(a) Prove that a function is uniformly continuous if and only if it has a modulus of continuity. (b) Prove that a family of functions is equicontinuous if and only if its members have a common modulus of continuity. Consider the modulus of continuity JL (s) = L s where L is a positive constant. (a) What is the relation between C�-' and the set of Lipschitz functions with Lipschitz constant ::=:: L ? (h) Replace [a , b] with IR and answer the same question. (c) Replace [a , b] with N and answer the same question. (d) Formulate and prove a generalization of (a). Consider a modulus of continuity JL (s ) = H s a where 0 < a ::=:: 1 and 0 < H < oo. A function with this modulus of continuity is said to be a-HOlder, with a-Holder constant H . See also Exercise 2 in Chapter 3. (a) Prove that the set c a (H) of all continuous functions defined on [a , b] which are a-HOlder and have a-Holder constant ::=:: H is equicontinuous . (b) Replace [a , b] with (a , b ) . I s the same thing true? (c) Replace [a , b] with R Is it true? (d) What about Q? (e) What about N ? Suppose that Un ) is an equicontinuous sequence in C 0 and p E [a , b] is given. (a) If fn (p) is a bounded sequence of real numbers, prove that Un ) is uniformly bounded. (b) Reformulate the Arzela-Ascoli Theorem with a weaker bound­ edness hypothesis. (c) Can [a , b] be replaced with (a , b) ?, Q?, IR?, N ? (d) What is the correct generalization? If M is compact and A is dense in M, prove that for any 8 > 0 there is a finite subset {a! , . . . , ak} C A which is �-dense in M in the sense that each x E M lies within distance 8 of at least one of the points aj , j = l , . . . , k.

Function Spaces

254

Chapter 4

20. Give an example of a sequence of smooth equicontinuous functions

21.

22.

23.

: [a , b] � lR whose derivatives are unbounded. Suppose that (fn ) is an equicontinuous sequence of functions : lR --+ JR, such that (0)) i s a bounded sequence in R (a) Does there exist a subsequence that converges uniformly? Prove or give a counter-example. (b) What if lRm replaces JR? (c) What about N ? Suppose that £ c C 0 is equicontinuous and bounded. (a) Prove that sup { f (x ) : f E £ } is a continuous function of x . (b) Show that (a) fails without equicontinuity. (c) Show that this continuous sup-property does not imply equicon­ tinuity. (d) Assume that the continuous sup-property is true for each subset F C £ . Is £ equicontinuous? Give a proof or counter-example. Let M be a compact metric space, and let be a sequence of isometries : M � M. ( a ) Prove that there exists a subsequence that converges t o an isometry as k � oo . (b) Does the inverse isometry 1 converge to - I . (Proof o r counter­ example. ) ( c ) Infer that the group of orthogonal 3 x 3 matrices is compact. LHint: Each orthogonal 3 x 3 matrix defines an isometry of the unit 2-sphere to itself.] (d) How about the group of m x m orthogonal matrices ? Suppose that f --+ is a contraction, but M is not necessarily complete. (a) Prove that f is uniformly continuous. (b) Why does (a) imply that f extends uniquely to a continuous rna£_ f : M --+ M, where M is the completion of M ? (c) I s f a contraction? Give an example of a contraction of an incomplete metric space that has no fixed point. Suppose that f : M --+ M and for all x , y E M, if x "I y then d ( f (x) , f ( y)) < d(x , y ) . Such an f is a weak contraction (a) Is a weak contraction a contraction ? (Proof or counter-example . ) (b) If M i s compact is a weak contraction a contraction? (Proof or counter-example.)

fn

fn

Un

Un )

in

in k

i

24.

25 . 26.

i�

:M

M

i

Exercises

27.

28. 29.

30.

255

(c) If M is compact, prove that a weak contraction has a unique fixed point. Suppose that f : JR. --+ JR. is differentiable and its derivative satisfies l f ' (x) l < 1 for all x E R (a) Is f a contraction ? (b) A weak one? (c) Does it have a fixed point? Give an example to show that the fixed point in Brouwer's Theorem need not be unique. On page 222 it is shown that if bn J� 1 ( 1 - t 2 t dt = 1 then for some constant c, and for all n E N, bn ::S c -Jii . What is the best (i.e. , smallest) value of c that you can prove works? (A calculator might be useful here.) Let M be a compact metric space. and let c Lip be the set of continuous functions f : M --+ JR. that obey a Lipschitz condition: for some L and all p. q . E M .

l f(p) - f (q ) l 0

::S

Ld(p, q) .

*(a) Prove that C LiP is dense in C ( M , JR.) . [Hint: Stone-Weierstrass.] * * * (b) If M = [a , b) and JR. is replaced by some other complete metric space, is the result true or false? * * * (c) If M is a general compact metric space and Y i s a complete metric 0 space, is C LiP ( M , Y) dense in C ( M , Y)? (Would M equal ro the Cantor set make a good test case?) 3 1 . Consider the ODE x ' = x on R Show that its solution with initial condition x0 is t f--+ e 1 x0 . Interpret e t+s = e ' e s in terms of the flow property. 32. Consider the ODE y ' = 2JTYT where y E R (a) Show that there are many solutions to this ODE, all with the same initial condition y (O) = 0. Not only does y (t ) = 0 solve the ODE, but also y (t) = t 2 does for t ::=: 0. (b) Find and graph other solutions such as y (t) = 0 for t ::S c and y (t) = (t - c) 2 for t ::=: c > 0. (c) Does the existence of these non-unique solutions to the ODE contradict Picard's Theorem? Explain. * (d) Find all solutions with initial condition y (O) = 0. 33. Consider the ODE x ' = x 2 on R Find the solution of the ODE with initial condition xo . Are the solutions to this ODE defined for all time or do they escape to infinity in finite time?

Function Spaces

256

Chapter 4

Suppose that the ODE x' = f (x ) on lR is bounded, I f (x ) 1 ::::: M for all x . (a) Prove that no solution of the ODE escapes to infinity in finite time. (b) Prove the same thing if f satisfies a Lipschitz condition, or, more generally, if there are constants C, K such that (x ) C l x l + K for all x . (c) Repeat (a) and (b) with JRm i n place of JR. (d) Prove that if : JRm ---+ JR.m is uniformly continuous then the condition stated in (b) is true. Deduce that solutions of uni­ formly continuous ODE's defined on JR.m do not escape to in­ finity in finite time. * * 3 5 . (a) Prove Borel's Lemma : given any sequence whatsoever o f real numbers (ar ) there is a smooth function : lR ---+ lR such that J (O) = ar . [Hint: Try = L fJk (x)akx k f k ! where fJk is a well chosen bump function. ) (b) Infer that there are many Taylor series with radius of conver­ gence = 0. (c) Construct a smooth function whose Taylor series at every x has radius of convergence = 0. [Hint: Try L fJk (x) e (x + qk) where {qJ , qz , . . . } = Q.] *36. Suppose that T C (a . b) clusters at some point of (a , b) and that J, g : (a , b) ---+ lR are analytic. Assume that for all t E T, ( t ) =

34.

If

I :::::

f

f

f

R

R

f

g (t) .

(a) Prove that = g everywhere in (a , b). (b) What if and g are only C00? (c) What i f T i s an infinite set but its only cluster points are a and

f

f

b?

* * (d) Find a necessary and sufficiem condition for a subset Z c (a , b) to be the zero locus of an analytic function defined on (a , b), Z = {x E (a , b) : f (x ) = 0}. [Him: Think Taylor. The result in (a) is known as the Identity Theorem. It states that if an equality between analytic functions is known to hold for points of T then it is an identity, an equality that holds everywhere.] 37. Let M be any metric space with metric d. Fix a point p E M and for each q E M define the function = d(q , x) - d(p, x ) . (a) Prove that i s a bounded, continuous function of x E M , and that the map q �---+ J sends M isometrically onto a subset Mo of C 0 (M, JR.) .

f

fq

fq (x)

q

Exercises

257

(b) Since C 0 (M. JR.) is complete, infer that an isometric copy of M is dense in a complete metric space, the closure of M0 , and hence that we have a second proof of the Completion Theorem 2. 76. 38. As explained in Section 8, a metric space M is cr -compact if it is the countable union of compact subsets, M = U M; . (a) Why i s it equivalent to require that M is the monotone union of compact subsets,

i.e., M1 C M2 C . . . ? (b) Prove that a a -compact metric space is separable. (c) Prove that Z,
tJ


(f) Assume that M is a * -compact, M =

tJ int {M; ) with each M;

compact. Prove that this monotone union engulfs all compacts in M , in the sense that if A C M is compact, then for some i ,

A C M; .

(g) If M = M; and each M; is compact show by example that this engulfing property may fail, even when M itself is compact. * * (h) Prove or disprove that a complete a -compact metric space is a * -compact. 39. (a) Give an example of a function f : [0. I] x [0, 1] --+ JR. such that for each fixed x, y � f (x , y) is a continuous function of y, and for each fixed y , x � f (x , y) is a continuous function of x, but f is not continuous. (b) Suppose in addition that the set of functions

tJ

£ = { x � f (x . y) : y

E

[0, I ] }

i s equicontinuous. Prove that f i s continuous. 40. Prove that JR. can not be expressed as the countable union of Cantor sets. 41 . What is the joke in the following picture?

258

Function Spaces

Chapter 4

e

y' = ll(y)

More Pre-lim Problems

f and fn . n E N. be functions from JR. to JR. Assume that fn (xn ) f(x) as n - oo and Xn - x . Show that f is continuous. LHint: Equicontinuity is irrelevant since the functions fn are not assumed

I . Let

2.

to be continuous.] Suppose that E

fn

C 0 and for each x

!1 (x)

n -> oo fn (X)

and lim

3.

:=::

E

[a , b ] ,

fz (x ) :=:: . . .

,

= 0. Is the sequence equicontinuous? Give a proof

fn (x)

or counter-example. [Hint: Does converge uniformly to 0, or does it not?] Let E be the set of all functions : [0, 1 ] ---+ JR. such that (0) = 0 and satisfies a Lipschitz condition with Lipschitz constant 1 . Define
u

u

u


(u ) =

11 (u(x)2 - u (x)) dx.

Prove that there exists a function E E at which
u

x

u

259

Exercises

5. Let (Pn ) be a sequence of real polynomials of degree � 10. Suppose that Pn (x) converges pointwise to 0 as n � oo and x E [0, 1 j. Prove that Pn (x) converges uniformly to 0. 6. Let ( an ) be a sequence of nonzero real numbers. Prove that the se­ quence of functions

has a subsequence converging to a continuous function. 7. Suppose that f : JR. -4 JR. is differentiable, f (O ) = 0, and f ' (x) > f(x) for all x E R Prove that f (x ) > 0 for all x > 0. 8 . Suppose that f : [a , b] -4 JR. and the limits of f (x) from the left and the right exist at all points of [a . b]. Prove that f is Riemann integrable. 9. Let h : [0. 1 ) -4 JR. be a uniformly continuous function where [0, 1 ) i s the half open interval. Prove that there is a unique continuous map g : [0. 1 ] -4 JR. such that g (x ) = h (x) for all x E [0. 1 ) . 1 0. Assume that f : JR. -4 JR. is uniformly continuous. Prove that there are constants A , B such that I f (x) I :::::: A + B lx I for all x E JR.. 1 1 . Suppose that f (x) is defined on [ - 1 , 1 ] and that its third derivative exists and is continuous . (That is, f is of class . ) Prove that the series

C3

L (n ( f ( l jn ) - f ( - 1 /n )) - 2J ' (O) ) 00

n =O

1 2.

1 3.

converges. Let A C IR.m be compact, x E A. Let (xn ) be a sequence in A such that every convergent subsequence of (xn ) converges to x . (a) Prove that the sequence (xn ) converges . (b) Give an example to show if A is not compact, the result in (a) is not necessarily true. Let f : [0, 1 ] -4 JR. be continuously differentiable, with f ( O ) = 0. Prove that

1 /11 2 :::::: 1 \f'(x ) )2 dx

1 /1 1 = sup { l f (t ) l : 0 :::::: t _::: I } .

where 14. Let fn :

fn (0)

=

JR. JR. be 0 and I f� (x) I -4

differentiable functions, n = _::: 2 for al l n , x . Suppose that

nlim -+oo fn (x) = g (x)

for all x . Prove that g is continuous.

1 , 2,

. . . with

Function Spaces

260

Chapter 4

15.

Let X be a non-empty connected set of real numbers. If every element of X is rational, prove that X has only one element. 1 6 . Let k � 0 be an integer and define a sequence of maps fn lR lR as

:

�

xk

fn (X) = � x +n

n

= 1 , 2, . . . . For which values of k does the sequence converge

uniformly on JR? On every bounded subset of lR? 1 7 . Let f [0, 1] lR be Riemann integrable over [b , 1] for every b such that 0 < b (a) If f i s bounded, prove that f is Riemann integrable over [0, 1 ] . (b) What if f i s not bounded? 1 8 . (a) Let S and T be connected subsets of the plane JR2 having a point in common. Prove that S U T is connected. (b) Let { Sa } be a family of connected subsets of JR2 all containing the origin. Prove that u sa is connected. 19. Let f : lR --+ lR be continuous. Suppose that lR contains a countably infinite set S such that

:

� :::: 1 .

iq f

(x) dx = 0

if p and q are not in S. Prove that f is identically zero. 20. Let f lR lR satisfy f (x) :::: f (y) for x y. Prove that the set where f is not continuous is finite or countably infinite. 2 1 . Let (gn ) be a sequence of Riemann integrable functions from [0, 1 ] into lR such that l gn (x) l :::: 1 for all n , x . Define

:

�

::::

1x gn (t) dt.

Gn (X ) =

Prove that a subsequence of (G n ) converges uniformly. 22. Prove that every compact metric space has a countable dense subset. 23. Show that for any continuous function f [0, 1] lR and for any E > 0 there is a function of the form

:

g (x ) = for some

n

n

L Ck x k k =O

E for all x in [0, 1 ] . lR --+ lR having all three of the

E N, and l g (x) - f(x ) l <

2 4 . Give an example of a function

following properties:

f

�

:

26 1

Exercises

(a) f (x ) = O for all x < O and x > 2. (b) = 1, (c) has derivatives of all orders. JR JR whose 25 . (a) Give an example of a differentiable function derivative is not continuous. (b) Let f be as in (a). If < 2 < prove that (x = 2 for some x E [0, 1 ] . 26. Let U C JRm be an open set. Suppose that the map h U --+ JRm is a homeomorphism from U onto JRm which is uniformly continuous. Prove that U = JRm . 27. Let be a sequence of continuous maps [0, 1 ] JR such that

f'(l) f

f : --+

f'(O)

f'(l)

f ) '

:


--+

fo\ fn ( Y )) 2 dy :S: 5 for al1

n.

Define

1 1 �X + Y fn (y) dy

gn : [0, 1 ] --+ JR by gn (X ) =

(a) Find a constant K :=: 0 such that :::; K for all n . (b) Prove that a subsequence of the sequence converges uni­ formly. 28. Consider the fol1owing properties of a map JRm JR. (a) is continuous. (b) The graph of is connected in JRm x JR. Prove or disprove the implications (a) :::::} (b), (b) ::::::} (a) . b e a sequence o f real polynomials o f degree :::; 10. Suppose 2 9 . Let that lim =0

l gn (x ) l

f

f:

(gn )

--+

f

(Pn )

n-+ oo Pn (x ) for al] x E [0, 1 ] . Prove that Pn (x) :::t 0, 0 :::; x say about Pn (x ) for 4 :::; x :::; 5 ?

:::; 1 . What can you

30. Give an example o f a subset of JR having uncountably many connected components. Can such a subset be open? Closed? Does your answer change if JR2 replaces JR? 3 1 . For each (a , b, c ) E JR3 consider the series

Determine the values of a, b, and c for which the series converges absolutely, converges conditionally, diverges.

262

Function Spaces

Chapter 4

Let X be a compact metric space and f : X � X an isometry (That is, f (y)) = y) for all y E X.) Prove that f (X ) = X 3 3 . Prove or disprove:
32.

d(f(x) ,

d(x,

f

x,

.

/_: l f(x) l dx

< oo .

(xn)

Xn

Show that there is a sequence in IR such that ---* oo, ---* 0, and ---* 0 as n ---* 00. 3 5 . Let : [0, 1] ---* lR be a continuous function. Evaluate the following limits (with proof)

Xnf(xn) f

Xn f( -xn)

n nlim ---. oa }t 0 x f(x) dx

(a)

x n f(x) dx. nlim ---> 00 n lt o

(b)

36. Let K be an uncountable subset of lRm . Prove that there is a sequence of distinct points in K which converges to some point of K . 37. Let be a sequence of twice differentiable functions on [0, 1 ] such that for all n , (0) = 0 and (0) = 0. Suppose that l g� I :::=: 1 for all n , Prove that there is a subsequence of which converges uniformly on 3 8 . Prove or give a counter-example: Every connected locally pathwise connected set in IRm is pathwise connected. 39. Let be a sequence of continuous functions ro. 1 1 ---* IR such that � 0 for each x E [0, l j . Suppose that

(gn) x.

gn

g�

(gn )

LO, 1 ] .

Un > fn (x)

1 1 ' fn (x) dx I

(x)

:S:

K

J01 fn (x ) dx

for all n where K is a constant. Does converge to 0 as n � oo? Prove or give a counter-example. 40. Let E be a closed, bounded, and non-empty subset of IRm and let f : E ---* E be a function satisfying I f (y) < - y I for all x , y E E, x t=- y . Prove that there is one and only one point E E such that f(xo) = 4 1 . Let : 2.rr I ---* lR b e a continuous function such that

(x) - f

Xo.

f LO,

for all integers

I

n

�

1 2:n: f(x) sin(nx) dx

1.

=0

Prove that f is identically zero.

lx

x0

Exercises

263

u

42. Let E be the set of all real valued functions : [0, I ] -+ IR satisfying = 0 and y ) l � l x - y l for all x , y E [0, 1 ] . Prove that the function

u(O)

lu(x) - u (

11 ((u(x) 2 - u (x)) dx


achieves its maximum value at some element of E. 43 . Let !I , fz , . . . be continuous real valued functions on [0, 1 ] such that for each x E [0, L ], f1 (x ) :::: fz (x ) :::: . . . . Assume that for each x , fn (x) converges t o 0 a s n -+ oo. Does fn converge uniformly to 0? Give a proof or counter-example. 44 . Let f : [0, oo) -+ [0, oo) be a monotonically decreasing function with

X-+ 00

(

1 00

j (x ) dx < oo .

Prove that lim xf x ) = 0 . 45 . Suppose that F

:

!Rm -+ IRm

is continuous and satisfies

IF(x) - F ( y) l

:::: A l x - Y l

F

for all x , y E !Rm and some constant A > 0. Prove that is one-to­ one, onto, and it has a continuous inverse. 46. Show that [0, 1 ] cannot be written as a countably infinite union of disjoint closed sub-intervals. 47. Prove that a continuous function f : IR -+ IR which sends open sets to open sets must be monotonic. 48. Let f : [0, oo) -+ IR be uniformly continuous and assume that ..... oo

lim

b

r b f (x ) dx

Jo

x ..... oo

exists (as a finite limit). Prove that lim

f (x )

= 0.

49. Prove or supply a counter-example: If f and g are continuously differentiable functions defined on the interval 0 < x < which satisfy the conditions

1

x---> 0

lim

f(x )

x-+ 0

= 0 = lim g (x )

and

J ' (x ) . an d 1 f g and g ' never vant· sh , then 1 Im · -- = x ..... o g ' (x) of L' Hospital's rule. )

f(x)

x---> 0 g (x )

lim c.

(Th"IS

IS ·

=c a converse

264

Function Spaces

Chapter 4

50. Prove or provide a counter-example: If the function f from JR to lR has both a left and a right limit at each point of JR, then the set of discontinuities is at most countable. 5 1 . Prove or supply a counter-example: is a non-decreasing real val­ ued function on [0 , 1 1 then there is a sequence n = 1 , 2, . . . of continuous functions on [0, 1 ] such that for each in [0, 1 ] , lim =

IT f

n -+oo fn (X)

j(x). f

fn ,

x

52. Show that if is a homeomorphism o f [0, 1 ] onto itself then there is a sequence of polynomials (x ) , n = 1 , 2, . . . , such that � uniformly on [0. 1 ] and each is a homeomorphism of [0. 1 ] onto 1 itself. [Hint: First assume that is C .] 53. Let be a C 2 function on the real line. Assume that is bounded with bounded second derivative. Let A = supx and B = supx I !" (x } I · Prove that

Pn Pn f

f

sup X

54. Let

I J (x) l '

Pn

f

f I f (x) I

_:::: 2 -Ji:B .

( k)

f be continuous on lR and let n l 1 fn (X) = ;; L J X + -n · k=O

fn (x)

Prove that converges uniformly to a limit on every finite interval [a , b] . 5 5 . Let be a real valued continuous function on the compact interval [a , b ] . Given E > 0, show that there is a polynomial such that

f

p

p(a) = f(a), p ' (a) = 0, and l p(x) - f(x) l < E.

,

for all x E [ b] . 5 6 . A function : [0, 1 ] � lR i s said t o b e upper semicontinuous if, given x E [0, L J and E > 0, there exists a 8 > 0 such that l y <8 implies that (y) < (x) E. Prove that an upper semicontinuous function on [0, 1 ] is bounded above and attains its maximum value at some point E [0, 1 ] . 57. Let and be functions from lR to R Assume that � as n � oo whenever � Prove that is continuous. (Note: the functions are not assumed to be continuous.)

a f

f

f

fn

fn

f

+

Xn

x.

- xI

p

f

fn (xn)

f (x)

Exercises

f(x), x :::; J'(x).

265

5 8 . Let 0 :::; l, be a continuous real function with continuous < 1. derivative Let M be the supremum of 0 :::; Prove: for n = 1 , 2 , . . . -l 1 n

;; L f k=O

(k)- - 1' f (x) dx o

n

lf'(x)l,

:S

x

M 2n

(

59. Let K be a compact subset of JRm and let Bi) be a sequence of open balls which cover K . Prove that there is an E > 0 such that each E -ball centered at a point of K is contained in at least one of the balls

Bi .

60. Let

f

( t (x)

f (t) dt ) f(x) = 0.

be a continuous real-valued function on [0, oo) such that lim x � oo

+

r

Jo

exists (and is finite) . Prove that limx-4 oo 6 1 . A standard theorem asserts that a continuous real-valued function on a compact set is bounded. Prove the converse: if K is a subset of JRm and if every continuous real-valued function defined on K is bounded, then K is compact. 62. Let F be a uniformly bounded equicontinuous family of real valued functions defined on the metric space X. Prove that the function

g(x) = u { f (x) : f s p

E F}

is continuous. 63. Suppose that Un ) is a sequence of nondecreasing functions which map the unit interval into itself. Suppose that lim = n -+ oo pointwise and that f i s a continuous function. Prove that uniformly as n oo . Note that the functions are not nec­ essarily continuous. 64. Does there exist a continuous real-valued function 0 :::; :::; 1 , such that

f(x)

fn (x) f (x) fn (x) --+ fn f(x), x

--+

1 ' xf(x) dx

= l

and

for all n = 0, 2 , 3, 4, 5 , . . . ? Give a proof or counter-example. 65 . Let f be a continuous, strictly increasing function from [0, oo ) onto [0, oo) and let = (the inverse, not the reciprocal) . Prove that

g f-1 loa f(x) dx + fob g (y) dy

2:

ab

Function Spaces

266

66. 67 .

68.

69.

70.

Chapter 4

for all positive numbers a , b, and determine the condition for equality. Let f be a function 1 ] --+ IR whose graph { (x , f (x) ) : x E 1]} i s a closed subset of the unit square. Prove that f i s continuous. Let ) be a sequence of positive numbers such that L converges. Prove that there exists a sequence of numbers --+ oo as n --+ oo such that L converges. Let f (x , y) be a continuous real valued function defined on the unit square 1] x 1 ] . Prove that g (x) = max {f (x , y) : y E 1]} i s continuous. Let the function f from 1 ] to 1 ] have the following properties. 1 It is of class C , f(O) = = / ( 1 ) , and f ' is nonincreasing (i.e., f is concave) . Prove that the arc-length of the graph of f does not exceed 3 . Let A b e the set of all positive integers that do not contain the digit 9 in their decimal expansions. Prove that

[0,

[0,

(an

Cn

Cn a n

[0,

[0,

an

[0,

[0, 0

[0,

I: -1 < 00 . a a EA That is, A defines a convergent sub-series of the harmonic series.

5

Multivariable Calculus

Thi s chapter presents the natural geometric theory of calculus in n dimen­ sions.

1

Linear Algebra

It will be taken for granted that you are familiar with the basic concepts of linear algebra - vector spaces, linear transformations, matrices, de­ terminants, and dimension. In particular, you should be aware of the fact that an m x n matrix A with entries aii is more than just a static array of mn numbers. It is dynamic. It can act. It defines a linear transformation TA : IR" ---+ _�Rm that sends n-space to m -space according to the formula

TA (v) =

m

11

L L aij Vj ei i= l j =l

, e, is the standard basis of .IR" . where v = L v jej E .IR" and e 1 , (Equally, e 1 , . . . , e is the standard basis for _�Rm . ) m The set M = M (m , n ) of all m x n matrices with real entries aii is a vector space. Its vectors are matrices. You add two matrices by adding the corresponding entries, A + B = C where aii + bii = cii . Similarly, if 1.. E lR is a scalar, I.. A is the matrix with entries J... aii . The dimension of the vector space M is mn , as can be seen by expressing each A as L aii Eij where •

•

•

268

Multivariable Calculus

Chapter 5

is the matrix whose entries are 0, except for the (ij ) th entry which is 1 . Thus, as vector spaces, M = �m n . This gives a natural topology to M . The set £ = £(JRn , JRm ) of linear transformations T : �n --+ JRm i s also a vector space. You combine linear transformations as functions, U = T + S being defined by U ( v ) = T (v) + S (v ) , and A T being defined by ( A T ) ( v ) = A T (v). The vectors in £ are linear transformations. The mapping A �---+ TA is an isomorphism T : M --+ £. As a rule of thumb, think with linear transformations and compute with matrices. As explained in Chapter 1 , a norm on a vector space V is a function I : V --+ lR that satisfies three properties: (a) For all v E V, I v i ::: 0; and I v i = 0 if and only if v = 0. (b) I A v l = I A I I v l . (c) l v + w l ::S l v l + l w l . (Note the abuse of notation in (b) ; l A I is the magnitude of the scalar A and l v l is the norm of the vector v . ) Norms are used to make vector estimates, and vector estimates underlie multi variable calculus. A vector space with a norm is a normed space. Its norm gives rise to a metric as d(v, v' ) = v - v' ·

E;j

l

l

Thus a normed space is a special kind of metric space. If V, W are normed spaces, then the operator norm of T : V --+ W is

I T il

= sup{

I Tv

l

w

--

lvlv

: v :F 0 } .

The operator norm of T is the maximum stretch that T imparts to vectors in V . The subscript on the norm indicates the space in question, which for simplicity is often supprc�ssed. t

1 Theorem Let T : V --+ W be a linear transformation from one normed space to another. The following are equivalent: (a) II T II < oo. (b) T is uniformly continuous. (c) T is continuous. (d) T is continuous at the origin.

Proof Assume (a) ,

II T I I < oo . For all v, v ' E V , linearity of T implies that T v - T v' ::: II T II I v - v ' ,

l

l

l

t If II T 11 is finite then T is said to be a bounded linear transformation. Unfortunately, this terminology conflicts with -;- being bounded as a mapping from the metric space V to the metric space W. The only linear transformation that is bounded in the latter sense is the zero transformation.

Section 1

Linear Algebra

269

which gives (b), uniform continuity. Clearly, (b) implies (c) implies (d) . Assume (d) and take E = 1 . There is a � > 0 such that if u E V and l u i < � then

I Tu l

For any nonzero and

E V , set u

v

I Tvl

--

lvl

<

1.

= A. v where A. = �/2 1 v l . Then l u i I Tu l

= -- <

lui

1

-

lui

=

�/2 < �

2 �

= -

which verifies (a).

D

Any linear transformation T JRn it is an isomorphism then it is a homeomorphism.

2 Theorem

:

---+

W

is continuous, and if

Proof The norm on JRn is the Euclidean norm

IvI and the norm on W is I Express v E JRn as v = L

=

Jvf + · · · + v�

Let M = max { I T ( e ) 1 w . . . , I T ( en ) l w } . 1 vi e i . Then l vi l � l v l and

lw·

n

n

j= l

j=l

,

which implies that II T II � n M < oo . By Theorem 1 , T is continuous . Assume that T is an isomorphism. Continuity of T implies that the image of the unit sphere, T (sn- 1 ) , is a compact subset of W. lnjectivity of T implies that the origin of W does not belong to T (sn- 1 ) . Thus. there is a constant c > 0 such that

We observe that

�

= T - 1 (w) l < 1 . For, if not, then t = 1 / r < 1 , and we have I T - 1 (t w ) = t r = 1 , contrary to the fact that l tw l < c. Thus II r - 1 11 � 1 / c and by Theorem 1 , r - I is continuous. A bicontinuous

r

bijection is a homeomorphism.

D

In the world of finite dimensional normed spaces, all lin­ ear transformations are continuous and all isomorphisms are homeomor­ phisms. In particular, T M .C is a homeomorphism. 3 Corollary

:

---+

Chapter 5

Multivariable Calculus

270

Proof Let V be an n -dimensional normed space. As you know from linear algebra, there is an isomorphism H : JRn ---* V . Any linear transformation T : V ---* W factors as T = (T

o

H)

o

H- 1 •

Theorem 2 implies that since T o H is a linear transformation defined on it is continuous, and H is a homeomorphism. Thus T is continuous. If T is an isomorphism, then continuity of T and T - 1 imply that T is a D homeomorphism.

ffi.n ,

A fourth norm property involves composites. It states that (d) li T o S ll :S I I T II II S II for all linear transformations : U ---* V and T : V ---* W. Thinking in terms of stretch, (d) is clear: stretches a vector u E U by at most II S II , and T stretches S(u) by at most II T 11 - The net effect on u is a stretch of at mo st II T II II S II . Corresponding to composition of linear transformations is the product of matrices. If A is an m x matrix and B is a x n matrix then the product th matrix = AB is the m x n matrix whose (ij ) entry is

S S

k

P

k

Pij = a;Iblj + Proof For each

... +

k

a;kbkj = L: airbrj· r=1

er E JRk and e E JRn w e have i

TA ( )

er

=

m

L a;rei i=l

k TB(ej ) = L brjer. r=l

Thus,

k

=

k

m

TA(TB(ej)) = TA ( L brjer) L brj L airei r=l i=l r=l m k = L L airbrjei = TAB ( ej ) . i=l r=l Two linear transformations that are equal on a basis are equal.

D

Theorem 4 expresses the pleasing fact that matrix multiplication corre­ sponds naturally to composition of linear transformations. See also Exer­ cise 6.

Derivatives

Section 2

2

27 1

Derivatives

A function of a real variable y

= f (x) has a derivative f' (x) at x when f (x + h) - f (x) (1) lim = f ' (x) . h-+0 h If, however, x is a vector variable. ( 1 ) makes no sense. For what does it mean to divide by the vector increment h ? Equivalent to ( 1 ) is the condition R (h) f (x + h) = f (x) + f ' (x)h + R (h) ::::} lim = 0, h-+0 l h l

which is easy to recast in vector terms.

Definition Let f : U � :!Rm be given, where U is an open subset of :!Rn . The function f is differentiable at p E U with derivative ( Df) P = T if T : :!Rn � IRm is a linear transformation and (2)

f(p + v) = f (p) + T (v) + R (v)

::::}

R (v) = 0. l v l -7 0 l v l lim

We say that the Taylor remainder R is sublinear because it tends to 0 faster than l v l . When n = m = 1 , the multidimensional definition reduces to the stan­ dard one. That is because a linear transformation IR � lR is just multipli­ cation by some real number, in this case multiplication by f ' (x ) . Here is how to visualize Df. Take m = n = 2. The mapping f : U � distorts shapes nonlinearly; its derivative describes the linear part of the distortion. Circles are sent by f to wobbly ovals, but they become ellipses under ( Df) P . Lines are sent by f to curves, but they become straight lines under ( Df) P . See Figure 103 and also Appendix A.

JR2

v ' •

P ', /

EB

Figure 103

(Df)p

( Df ) P

is the linear part of f at

p.

Multivariable Calculus

272

Chapter 5

This way of looking at differentiability is conceptually simpl e. Near p. i s the sum o f three terms : a constant term f(p), a linear term and a sublinear remainder term Keep i n mind what kind o f an object the derivative is. It is not a number. It is not a vector. No, if it exists, then is a linear transformation from the domain space to the target space. P

f

(Df )p v,

R (v).

( Df)

If f is differentiable at p, then it unambiguously determines (Df)p according to the limit formula, valid for all u E ffi.n , f(p + t u ) - f(p) . (3) lim ( Df)p (u) = t-+0 t Proof Let T be a linear transformation that satisfies (2) . Fix any u E ffi.n and take v = tu. Then f(p + tu) - f(p) T (tu) + R (tu) = T u ) R (tu ) u . ( + tu l i t t 5 Theorem

=

l l 0, which verifies (3 ) . Limits, when

The last term converges to zero as t � they exist, are unambiguous, and therefore if T' is a second linear transfor­ D mation that satisfies (2) then T (u ) = T' ( u), so T = T'.

6 Theorem

Differentiability implies continuity.

Proof Differentiability at

l f(p + v ) - f(p) l as p +

v �

=

p implies that

I C DJ) pv + R (v) l

::=

I I < Df) p ll l v l + I R(v) l � 0 D

p.

Df is the total derivative or Frechet derivative. In contrast, the (ij)th partial derivative of f at p is the limit, if it exists, afi (p) a xj

= lim

t-+0

fi ( P + tej ) - fi ( P ) . t

If the total derivative exists, then the partial derivatives exist, and they are the entries of the matrix that represents the total derivative.

7 Corollary

Proof Substitute in (3) the vector u = both sides of the resulting equation.

ei

and take the ;th component of

D

As is shown in Exercise 1 5 , the mere existence of partial derivatives does not imply differentiability. The simplest sufficient condition beyond the ex­ istence of the partials - and the simplest way to recognize differentiability - is given in the next theorem.

Derivatives

Section 2

If the partial de riva tives of f uous, then f is differentiable.

8 Theorem

273

: U � IR_m

exist and are contin­

Proof Let A be tbe matrix of partials at p, A = [aji ( p) jax i ] , and let T : IR.n � IR.m be tbe linear transformation that A represents. We claim that ( Df ) P = T . We must show tbat tbe Taylor remainder

R ( v ) = f (p + v ) - f (p) - A v

an ] from p to q = p + v tbat is sublinear. Draw a patb a = [a1 , consists of n segments parallel to tbe components of v . Thus v = L v i e i and •

is a segment from P i-1 = p Figure 1 04.

•

•

•

+ L k < i v k ek

to p1 = Pi- l

+ vi e i .

See

Figure 104 The segmented patb a from p to q . By tbe one-dimensional chain rule and mean value theorem applied to tbe differentiable real-valued function g ( t ) = fi o ai ( t ) of one variable, tbere exists t;i E (0, 1 ) such tbat J.f.'i c P i )

- Ji c Pi - t ) = g c 1 ) - g (o ) = g ' C tij ) =

where P ii = ai (tij ) . Telescoping ji (p

R, (v)

=

ji (p

af; ( P;i ) vi , a Xi

+ v) - ji (p ) along a gives

+ v ) - /; (p ) - ( Av ) ;

vi � ( /; (pi ) - /; (Pi-d - --. a Xl i =l :t { afi (P;i ) a.{; (p) J vi . = =

a.{; (p)

�

)

_

i= l

axi

axi

Continuity of the partials implies tbat tbe terms inside curly brackets tend 0 to 0 as I v i � 0. Thus R is sublinear and f is differentiable at p .

Multivariable Calculus

274

Chapter 5

Next we state and prove the basic rules of multivariable differentiation

Let f and g be differentiable. Then (a) D(f + cg) = Df + cDg. (b) D (constant) = 0 and D(T(x)) = T.

9 Theorem

(c) D(g o f) = Dg o Df. (chain rule) (d) D(f g) = Df g + f Dg. (Leibniz rule) •

•

•

There is a fifth rule that concerns the derivative of the nonlinear inversion operator Inv : T r+ T - 1 • It is a glorified version of the formula

d x-I

� = -X -2 , and is discussed in Exercises 32 - 36.

Proof ( a) Write the Taylor estimates for the Taylor estimate for f + cg.

f and g and combine them to get

f(p + v) = f(p) + ( DJ)p (v) + Rt g(p + v) = g(p) + (Dg)p (v) + Rx (f + cg) (p + v) = (f + cg) (p ) + ( (Df)p + c(Dg) p) (v) + R f + cRx . Since R f + eRg is sublinear, (DJ)p + c(Dg)p is the derivative of f + cg at p. n (b) If f : lft --+ lftm is constant, f (x ) = c for all x E lft n , and

if 0 : lft --+ lftm denotes the zero transformation then the Taylor re­ = mainder is identically zero. Hence constant )p = 0 . n T : lft � lftm is a linear transformation. If then substi­ = tuting itself in the Taylor expression gives the Taylor remainder v) = which is identically zero. Hence Note that when n = m = 1 , a linear function is of the form (x ) = ax , and the previous formula just states that a x ' = a . (c) Tacitly, we assume that the composite makes j (x ) = sense as x varies in a neighborhood of E U . The notation refers to the composite of linear transformations and is written out as

n

R(v)

D(

f(p +

T

J(p + v) - f(p) - O (v)

f(x) = T(x),

R(v) (DJ)p T. f

f(p) - T(v),

( )

p

go

g ( f(x)) Dg o Df

D (g o J)p = ( Dg)q o (DJ)p where q = This chain rule states that the derivative of a composite is the composite of the derivatives . Such a beautiful and natural formula must be true. See also Appendix A. Here is a proof:

f (p) .

Derivatives

Section 2

275

+ - f (p ) T ( v)

It is convenient to write the remainder R ( v) = f (p v) in a different form, defining the scalar function e( v) by

e( v )

I R (v ) l lvl

=

I0

-

if v =I= 0 if

v

=0

Sub linearity is equivalent to lim e( v) = 0. Think of e as an error factor.

v--.0

The Taylor expressions for f at p and

g at q = f (p) are

+ Av + R t g( q + w) = g (q ) + B w + R8 f ( p + v) = f (p )

where A = (Df) p and B = ( D as

g ) q as matrices. The composite is expressed

g o f (p + v) = g (q + A v + Rf ( v)) = g (q ) + BAv + B Rf (v) + R8 ( w ) where w = Av + R1 ( v) . It remains to show that the remainder terms are sub linear with respect to v . First

is sublinear. Second,

Therefore,

Since eg ( w) --+ 0 as w --+ 0 and since v --+ 0 implies that w does tend to 0, we see that R8 ( w ) is sublinear with respect to v. It follows that f )) p = B A as claimed. (d) To prove the Leibniz product rule, we must explain the notation v • w. In there i s only one product, the usual multiplication o f real numbers . In higher dimensional vector spaces, however, there are many products, and the general way to discuss products is in terms of bilinear maps. A map f3 : V x W --+ Z is bilinear if V, W, Z are vector spaces and for each fixed v E V the map f3 (v ) : W --+ Z is linear, while for each fixed w E W the map {3 ( . , w ) : V --+ Z is linear. Examples are

(D(g o ffi.

•

.

Multivariable Calculus

276

Chapter 5

(i) Ordinary real multiplication (x , y) t---+ xy is a bilinear map JR. X JR. ----* R (ii) The dot product is a bilinear map IR.n x IR.n ----* R (iii) The matrix product is a bilinear map M (m x k) x M (k x

M (m x n) .

t

n)

�

The precise statement of (d) is that if fJ : JR.k x IR. ----* IR.m is bilinear while f : U ----* JR.k and g : U ----* JR.�' are differentiable at p , then the map x t---+ fJ (f(x) , g (x)) is differentiable at p and

(DfJ (f, g)) p (v)

=

fJ ((Df) p ( v ) , g (p)) + {J ( f ( p ) , (Dg) p ( v )) .

Just as a linear transformation between finite-dimensional vector spaces has a finite operator norm, the same is true for bilinear maps:

ll fJ II

= sup{

! fJ ( v , w ) ! : v, w lvl lwl

::/= 0} < oo . t

To check this, we view fJ as a linear map T : JR.k ----* .C (ffi. , JR.m ) . Accord­ 13 ing to Theorems 1 , 2, a linear transformation from one finite dimensional normed space to another is continuous and has finite operator norm. Thus the operator norm T is finite. That is,

13

But

I I T13 (v) ll

ll fJ I I < oo .

I I T13 ll

= max{

= max{ lfJ ( v ,

II

T���) II

:

v

w) l I ! w ! : w

::/= 0} < oo . ::/= 0}, which implies that

Returning to the proof of the Leibniz rule, we write out the Taylor esti­ mates for f and g and plug them into fJ. Using the notation A = (Df) p , B = ( D g) P , bilinearity implies

fJ (f ( p + v) , g (p + v ) ) = fJ ( f(p ) + A v + R1 , g (p) + B v + Rg ) = fJ (f ( p ) g ( p ) ) + fJ ( A v g ( p )) + fJ (f( p ) , B v ) + fJ (f ( p ) , Rg ) + fJ (A v , B v + Rg) + fJ ( R1 , g ( p ) + B v + Rg ) . ,

,

The last three terms are sublinear. For

l fJ ( f ( p ) , Rg) l l fJ ( Av, B v + Rg) l l fJ C R t . g(p) + B v + Rg l

�

llfJ II I f (p) I I Rg l � ll fJ II II A I I I v i i B v + Rg l � llfJ II I R t l l g ( p ) + Bv + Rg l Therefore fJ ( f, g) is differentiable and DfJ (f, g) = fJ ( Df g) + fJ (f, Dg) ,

as claimed.

0

Here

are

277

Derivatives

Section 2

some applications of these differentiation rules :

10 Theorem A function f : U ---+ IRm is differentiable at p E U if and only if each of its components /;, is differentiable at p. Furthermore, the derivative of its i 1h component is the i 1h component of the derivative. m Proof Assume that f is differentiable at p and express the i component

of f as /;, = rri o f where rri : IRm ---+ lR is the projection that sends a wm ) to Wj . Since rri is linear it is differentiable. By vector w = ( w 1 the chain rule, fi is differentiable at p and •

.

.

.

,

( D/;, ) p = (D rrd o ( Df ) p

= ni

o ( Df)p .

0

The proof of the converse is equally natural.

Theorem 1 0 implies that there is little loss of generality assuming m = 1 , i.e., that our functions are real-valued. Multidimensionality of the domain, not the target, is what distinguishes multivariable calculus from one-variable calculus.

segment [p, q ] where M =

:

U ---+ IR m is differentiable on U and the is contained in U, then

11 Mean Value Theorem If f

l f (q ) - f ( p) l ::;::

sup{ I I ( D/ ) x ll :

Proof Fix any unit vector g (t )

u

x

E

E

U }.

M

pi ,

lq -

lRn . The function

= ( u , f (p + t (q

-

p)))

is differentiable, and we can calculate its derivative. By the one-dimensional Mean Value Theorem, this gives some () E {0, l ) such that g ( l ) - g (O) = g ' (() ) . That is,

( u , f (q ) - f (p ) ) = g ' (() ) = ( u ,

( Df ) p +O (q -p) (q -

p) } ::;:: M l q

- pl .

A vector whose dot product with every unit vector is no larger than M l q - p i 0 has norm ::;:: M l q - p l .

Remark The one-dimensional Mean Value Theorem is an equality, f (q ) - f (p)

= f ' (()) (q - p) ,

and you might expect the same to be true for a vector-valued function if we replace J' (() ) by (Df) e . Not so. See Exercise 1 7 . The closest we can come to an equality form of the multidimensional Mean Value Theorem is the following:

Multivariable Calculus

278

Chapter 5

U -4- IRm is of class C 1 (its derivative exists and is continuous) and if the segment [p, qJ is contained in U, then 12 C 1 Mean Value Theorem

Iff :

f (q ) - f ( p)

(4)

=

T (q - p )

1 ( Df)p+t(q-p) dt .

where T is the average derivative of f on the segment 1

T=

Conversely, if there is a continuousfamily of linear maps Tpq (4) holds, then f is of class C 1 and (Df) p = Tpp ·

E

£for which

Proof The integrand takes values in the normed space £(1Rn , JRm ) and is a continuous function of t. The integral is the limit of Riemann sums

L ) Df ) p+tk(q-p) !:,. tb k

which lie in £. Since the integral is an element of £. it has a right to act on the vector q - p. Alternately, if you integrate each entry of the matrix that represents f along the segment, the resulting matrix represents T. 1 Fix an index i , and apply the Fundamental Theorem o f Calculus to the C real-valued function of one variable

D

g (t) where

a (t) = p + t (q - p) J; (q) - J; (p)

=

fi o a (t) ,

parameterizes

=

=

[p, q ] . This gives

1 1 g' (t) dt

1 1 � aJ; (a (t)) (q · - p · ) dt

g ( l ) - g (O)

1

0 � j=1

-�

- � j= 1

=

ax J-

J

J

aJ; (a (t ) ) d t (qj - Pj ) , a XJ· 0 1

which is the ; m component of p). To check the converse, we assume that (4) holds for a continuous family of linear maps Take q = p + The first-order Taylor remainder at p is

T (q -

Tpq ·

v.

f( p + v) - f (p) - Tpp ( v ) (Tpq - Tpp ) ( v) , which is sublinear with respect to v . Therefore ( Df)p Tpp· R ( v)

=

=

=

D

279

Higher derivatives

Section 3

Assume that U is connected. Iff : U ___,. !Rm is differentiable = 0, then f zs constant and for each point E U, 13 Corollary

x

(Df)x

Proof The enJoyable open and closed argument is left to you as Exercise 20.

D We conclude this section with another useful rule - differentiation past the integral. See also Exercise 23.

Assume that f [a . b] x (c, d) is continuous and that aj(x, y)jay exists and is continuous. Then F (y ) = 1 b f (x, y) dx is of class C1 and dF 1b aj(x, y) = dy ay dx. C1 F (y + h) - F(y) 1 1b (1 1 af(x, y + th ) dt ) h dx . h h ay f y y y + h. aj(x, y) jay h 0, dF jdy aflay. D :

14 Theorem

___,.

lR

(5)

a

Proof By the

Mean Value Theorem, if h is small, then

------ = -

a

0

The inner integral is the average partial derivative of with respect to along the segment from to Continuity implies that this average converges to as ___,. which verifies (5) . Continuity of follows from continuity of See Exercise 22.

3

Higher derivatives

In this section we define higher-order multivariable derivatives. We do so in the same spirit as in the previous section: the second derivative will be the derivative of the first derivative, viewed naturally. Assume that : U ___,. !Rm f) exists at each E U and the is differentiable on U . The derivative map x �---+ x defines a function

(D x

( Df)

x

f

The derivative D is the same sort of thing that is, namely a function from an open subset of a vector space into another vector space In the case

f

f

Multivariable Calculus

280

Chapter 5

of Df, the target vector space is not IR;_m , but rather the mn dimensional space £. If Df is differentiable at p E U then by definition

( D (Df)) p

=

(D 2 f) p = the second derivative of f at p,

and f is second-differentiable at p. The second derivative i s a linear map from IR;_n into £. For each v E IRn , ( D 2 f) P ( v) belongs to £ and therefore is a linear transformation IR;_n � IR;_m , so (D 2 f) p ( v ) (w) is bilinear, and we write it as (D 2 f) p ( v , w ) . (Recall that bilinearity is linearity in each variable separately. ) Third- and higher derivatives are defined i n the same way. If f is second differentiable on U, then x �---+ ( D 2 f) x defines a map where £2 is the vector space of bilinear maps IR;_n x IR;_n ---+ IR;_m . If D 2 f is differentiable at p, then f is third differentiable there and its third derivative is the trilinear map (D 3 f) p = (D(D 2 f)) p . Just as for first derivatives, the relation between the second derivative and the second partial derivatives calls for thought. Express f : U � IR;_m in component form as f (x) = (fl (x) , . . . , fm ( x)) where x varies in U. L S Theorem

p exist, and

If (D 2 f) P exists then (D 2 fk ) P exists, the second-partwls at a 2 k (P) (D 2 fk ) p (e; , e J) = ---'-f--'ax; ax1

Conversely, existence of the second-partials implies existence of (D 2 f) P • provided that the second-partials exist at all points x E U near p and are continuous at p. Proof Assume that ( D 2 f) P exists. Then x = p and the same is true of the matrix

x

�---+

( Df) x is differentiable at

that represents it; x �---+ Mx is differentiable at x = p. For, according to Theorem 1 0, a vector function is differentiable if and only if its components

Higher derivatives

Section 3

28 1

are differentiable; and then, the derivative of the kth component is the kth component of the derivative. A matrix is a special type of vector, its compo­ nents are its entries. Thus the entries of Mx are differentiable at x = p, and the second-partials exist. Furthermore, the k th row of Mx is a differentiable vector function of x at x = p and

(Dfk )p +r ei ( ej ) - ( Djk )p ( ej ) ( D(Dfk ))p ( ei )( ej ) = ( D 2 fk )p (ei , ej ) = rlim --+ 0 t th The first derivatives appearing in this fraction are the j partials of fk at p + t e and at p. Thus, a2 fk ( p ) jaxi axj = (D 2 fk )p (ei , ej ) as claimed. Conversely, assume that the second-partials exist at all x near p and are continuous at p. Then the entries of Mx have partials that exist at all points q near p, and are continuous at p . Theorem 8 implies that x 1--+ Mx is differentiable at x = p; i.e. , f is second-differentiable at p. D i

The most important and surprising property of second derivatives is sym­ metry.

If (D 2 f )p exists then it is symmetric: for all v , w E JR:n, (D 2 f )p ( v, w) = (D 2 f )p ( w , v). Proof We will assume that f is real-valued (i.e. , m = 1 ) because the symmetry assertion concerns the arguments of f, not its values. For a variable t E [0, 1], draw the parallelogram P determined by the vectors t v , t w , and label the vertices with ± 1 's as in Figure 1 05 . 16 Theorem

p + tw

p + tv + tw

p + tv

p

Figure 105 The parallelogram

P has signed vertices.

The quantity

D. = !::i (t,

v , w ) = f( p + t v + t w) - j( p + tv) - f( p + t w) + f( p )

Chapter 5

Multivariable Calculus

282

is the signed sum of f at the vertices of P . Clearly, respect to v, w ,

Ll (t,

!::!..

is symmetric with

V, W ) = Ll ( t , W , V).

We claim that

. b. (t, v , w ) , (D 2 f) p (v , w ) = hm t ---+ 0 t2 from which symmetry of D 2 f follows. Fix t, v, w and write .6. = g ( l ) - g (O) where (6)

g(s ) = f(p + t v + st w) - f(p

+ st w) .

Since f is differentiable, so is g . By the one-dimensional Mean Value Theorem there exists () E (0, 1 ) with .6. = g ' (O) . By the Chain Rule, g ' (()) can be written in terms of Df and we get

b. = g ' (()) = (Df) p+tv+Ot w (t w) - (Df) p+Ot w (t w) . Taylor's estimate applied to the differentiable function u gives

1---+

( D/)u at u = p

(Df ) p+x = (Df) p + ( D 2 f) p (x , . ) + R (x, . ) where R (x , . ) E .C (ll�n , JR:m ) is sublinear with respect to x . Writing out this estimate for (Df) p+x first with x = tv + Ot w and then with x = Ot w gives

�

=

=

� {[ (Df) p (w) + (D2 f) p (t v Ot w, w) R (tv Ot w, - [ (Df) p (w) + (D2 /) p (Ot w , w) + R (Ot w , w) J} +

(D 2 f) p ( V , W ) +

+

R (t v + Ot w , w) 1

+

w)

]

R (Ot w, w) - ---t

Bilinearity was used to combine the two second derivative terms. Sublin­ earity of R (x , w) with respect to x implies that the last two terms tend to 0 as t -+ 0, which completes the proof of (6). Since (D 2 f) P is the limit of a symmetric (although nonlinear) function of v, w it too is symmetric. D Remark. The fact that

D2 f can be expressed directly as a limit of val­

ues of f is itself interesting. It should remind you of its one-dimensional counterpart,

_ 1.

f " (Y ) - lm h---+ 0

f(x + h) + f(x - h) - 2f(x) h2

.

Section 3

Higher derivatives

283

Corresponding mixed second-partials of a second-differen­ tiable function are equal, 17 Corollary

Proof The equalities

follow from Theorem 1 5 and the symmetry of D 2 f .

D

The mere existence of the second-partials does not imply second-dif­ ferentiability, nor does it imply equality of corresponding mixed second­ partials. See Exercise 24.

h derivative, if it exists, is symmetric: permutation of does not affect the value of(D' f ) p ( V J , . . , v, ) . Cor­ responding mixed higher-order partials are equal.

18 Corollary The r 1 the vectors V J , . . . , v,

.

D

Proof The induction argument is left to you as Exercise 29.

In my opinion Theorem 1 6 is quite natural. even though its proof i s tricky. It proceeds from a pointwise hypothesis to a pointwise conclusion: when­ ever the second derivative exists it is symmetric. No assumption is made about continuity of partials. It is possible that f is second-differentiable at p and nowhere else. See Exercise 25 . All the same. it remains standard to prove equality of mixed partials under stronger hypotheses, namely that D 2 f is continuous. See Exercise 27. We conclude this section with a brief discussion of the rules of higher­ h order differentiation. It is simple to check that the rt derivative of f + cg is D' f + cD' g. Also, if fJ is k-linear and k < r then f (x) = fJ (x , , x) has D' f = 0. On the other hand. if k = r then ( D' f)p = r ! Symm (fJ ) , the where Symm({J) i s the symmetrization o f fJ . See Exercise 28. th The chain rule for r derivatives is a bit complicated. The difficulties arise from the fact that x appears in two places in the expression for the first order chain rule, (D g o j)x = (Dg)J(x> o ( Df ) x and so differentiating this product produces .

.

.

.

284

Chapter 5

Multivariable Calculus

(The meaning of f) ; needs clarification.) Differentiating again produces four terms, two of which combine. The general formula is

(D

(Dr g

0

f) x =

r

L �)Dkg ) f(x ) k= i

ll

0

(D11 n x

where the sum on Jl is taken as f.l runs through all partitions of { I , . . . , r } into k disjoint subsets. See Exercise 4 1 . The higher-order Leibniz rule i s left for you a s Exercise 42.

4

Smoothness Classes

A map f : U --+ �m is of class cr if it is r order differentiable at each p E U and its derivatives depend continuously on p. (Since differentia­ bility implies continuity, all the derivatives of order < r are automatically h continuous; only the rt derivative is in question.) If f is of class for all r, it is smooth or of class c oo . According to the differentiation rules, these smoothness classes are closed under the operations of linear combination, product, and composition . We discuss next how they are closed under limits. Let ( fk ) be a sequence of functions fk : U -4 �m . The sequence is (a) Uniformly cr convergent if for some C' function f : U -4 �m ,

th

cr

cr

as k -4 oo.

(b) Uniformly cr Cauchy if for each E > 0 there is an N such that for all k, f. � N and all x E U,

lfk(x) - fe(x) i < E l i (Dfk)x - (Dfdx l < E 19 Theorem

Uniform

cr convergence and Cauchyness are equivalent.

Proof Convergence always implies the Cauchy condition. As for the con­ verse, first assume that r = 1 . We know that fk converges uniformly to a

Section 4

Smoothness Classes

285

continuous function f, and the derivative sequence converges uniformly to a continuous limit Dfk =:::t G .

We claim that Df = G. Fix p E U and consider points q in a small convex 1 neighborhood of p. The C Mean Value Theorem and uniform convergence imply that as k � oo,

fk (q) - fk (P)

=

tt

f (q ) - f (p)

=

11

(Dfk)p+r (q- p J dt (q - p)

1 1 G (p

tt +

t (q - p)) d t (q - p) .

This integral of G is a continuous function of q that reduces to G (p ) when p = q . By the converse part of the C 1 Mean Value Theorem, f is differen­ tiable and Df = G. Therefore f is C 1 and fk converges C 1 uniformly to f as k ---+ oo, completing the proof when r = 1 . Now suppose that r 2: 2. The maps Dfk : U ---+ £ form a uniformly C' - 1 Cauchy sequence. The limit, by induction, is C' - 1 uniform; i.e., as

k ---+

00,

Ds (Dfk )

=:::t

D·' G

for all s ::S r - I . Hence fk converges C' uniformly to f as k completing the induction. The cr norm of a C' function f : U � IRm is

ll f ll ,

= max f sup xEU

I I , makes C' (U, �m)

normed vector space.

oo, D

l f (x ) l , . . . , sup II (D ' nx II }. xEU

The set of functions with l l f ll , < oo is denoted C (U. �m ) . 20 Corollary II

�

a

Banach space -

a

complete

Proof The norm properties are easy to check; completeness follows from Theorem 19. D

is a convergenT series of constants and if II fk II , ::S Mk for all k, then the series offunctions L fk converges in C' ( U , JRm ) to a function f. Term by term d(ffe rentiation of order ::S r is valid, D' f = L k D' fk · 21 cr M -test If "'E. Mk

Proof Obvious from the preceding corollary.

D

5

Chapter 5

Multivariable Calculus

286

Implicit and Inverse Functions

Let f : U ---+ JR.m be given, where U is an open subset of JRn x JRm. Fix attention on a point (xo , Yo) E U and write f (xo , Yo) = zo. Our goal is to solve the equation

f(x , y) = zo

(7)

near (xo, Yo ) . More precisely, we hope to show that the set of points (x , y) nearby (xo, Yo) at which f (x , y) = zo, the so-called zo-locus of f , is the graph of a function y = g(x). If so, g is the implicit function defined by (7) . See Figure 1 06. f= Zo

Yo

f = Zo

Figure 106 Near

(xo , Yo) the zo-locus of f is the graph of a function y = g (x ) .

Under various hypotheses we will show that g exists, is unique, and is differentiable. The main assumption. which we make throughout this section, is that

the m x m matrix

B=

[ afi (xoayj, Yo) ]

is invertible.

Equivalently the linear transformation that B represents is an isomorphism

]Rm ---+ ]Rm .

If the function f above is C, 1 .:::: r .:::: oo, then near (xo, Yo). the zo-locus of f is the graph of a unique ju."lction y = g (x). Besides, g is C. 22 Implicit Function Theorem

Proof Without loss of generality, we suppose that (x0 , y0) is the origin (0, 0) in JRn x JRm , and zo = 0 in JRm . The Taylor expression for f is

f (x , y) = Ax + By + R where A is the m

x

n matrix

Section

5

287

Implicit and Inverse Functions

[

(x A = aj; o. Yo)

]

a xj and R is sublinear. Then, solving j(x , y) = 0 for y = gx is equivalent to solving

y = - B -1 (Ax + R (x , y)) . In the unlikely event that R does not depend on y, (8) is an explicit formula (8)

for gx and the implicit function is an explicit function. In general, the idea is that the remainder R depends so weakly on y that we can switch it to the left hand side of (8), absorbing it in the y term. Solving (8) for y as a function of x is the same as finding a fixed point of

Kx : y 1--+ - B -1 ( A x + R (x , y)),

so we hope to show that Kx contracts. The remainder R is a C 1 function, and (DR) (O.O) = 0. Therefore if r is small and lx l , IY I ::::: r then

I B - 1 11

1 a R �:· y) I ::::: � ·

By the Mean Value Theorem this implies that

I Kx Cy i ) - Kx ( Yz) i ::::: I B - 1 II I R(x, yt ) - R (x , Yz) i ::::: I I B - 1 1 1

for

1 �; I IYt - Yz i

:::::

� IY1 - Yz i

lx l , IYI I , iYz i ::::: r . Due to continuity at the origin. if lx l ::::: T « r then I Kx CO) I :::::

I

2.

Thus, for all x E X , Kx contracts Y into itself where X is the T -neighborhood of 0 in JR.n and Y is the closure of the r -neighborhood of 0 in JR.m . See Fig­ ure 1 07 .

Figure 107

Kx contracts Y into itself.

Multivariable Calculus

288

Chapter 5

By the Contraction Mapping Principle, Kx has a unique fixed point g (x) in Y . This implies that near the origin, the zero locus of f is the graph of a function y = g(x) . It remains to check that g i s c r . First we show that g obeys a Lipschitz condition at 0. We have l gx l = I Kx (gx ) - K, ( O) + Kx CO) I � Lip( Kx ) l gx - 01 + I Kx CO) I 1 1 1 � (x , 0)) x l + x B1 g � (A + R I 1 2 2 lgx l + 2L lx l

where L condition

I B -1 11 11 A I I and l x I is small. Thus

g

satisfies the Lipschitz

l gx l �

4L l x l . In particular, g is continuous at x = 0. Note the trick here. The term l g x I appears on both sides of the inequality but since its coefficient on the r.h.s. is smaller than that on the l.h.s., they combine to give a nontrivial inequality. By the chain rule, the derivative of g at the origin, if it does exist, must satisfy A + B(Dg)0 = 0, so we aim to show that ( Dg)o = -B- 1 A . Since gx is a fixed point of Kx , we have g x = - B 1 A (x + R ) and the Taylor estimate for g at the origin is -

l g (x) - g (O) - (- B - 1 Ax ) l = I B- 1 R (x , g x) l � II B - ' II I R (x , gx ) l � II B - ' 11 e (x , gx ) ( l x l + l g x l ) � II B - ' II e (x , gx) ( l + 4L) I x l where e(x , y) --* 0 as (x, y) --* (0, 0). Since g x --* 0 as x --* 0, the error factor e(x . gx ) does tend to 0 as x --* 0, the remainder is sublinear with respect to x, and g is differentiable at 0 with (Dg)0 = - B-1 A .

All facts proved at the origin hold equally at points (x . y ) on the zero locus near the origin. For the origin is nothing special. Thus, g is differentiable at x and (Dg)x = - B; ' o Ax where

aj (x , gx ) ax

aj(x, gx) . ay Since gx is continuous (being differentiable) and f is C 1 , Ax and Bx are continuous functions of x . According to Cramer's Rule for finding the in­ verse of a matrix, the entries of B; 1 are explicit, algebraic functions of the Ax =

---­

Bx =

entries of Bx . and therefore they depend continuously on x . Therefore g is C 1 • To complete the proof that g is c r , we apply induction. For 2 � r < oo, assume the theorem is true for r - I . When f is c r this implies that g is

Implicit and Inverse Functions

Section 5

289

cr- l . Because they are composites of cr- l functions, Ax and Bx are cr- l . Because the entries of B; 1 depend algebraically on the entries of Bx . B; 1 is also cr - l Therefore ( D g) X is cr- l and g is cr If f is c oo ' we have just D s hown that g is c r for all finite r, and thu s g is c oo . 0

0

Exercises 35 and 36 discuss the properties of matrix inversion avoiding Cramer's Rule and finite dimensionality. Next we are going to deduce the Inverse Function Theorem from the Implicit Function Theorem. A fair question is: since they tum out to be equivalent theorems, why not do it the other way around? Well, in my own experience, the Implicit Function Theorem is more basic and flexible. l have at times needed forms of the Implicit Function Theorem with weaker differentiability hypotheses respecting x than y and they do not follow from the Inverse Function Theorem. For example, if we merely assume that B = af (xo , Yo)fay is invertible, that af (x , y)jax is a continuous function of (x , y) , and that f is continuous (or Lipschitz) then the local implicit function of f is continuous (or Lipschitz). It is not necess ary to assume that f is of class C 1 • Just as a homeomorphism is a continuous bij ection whose inverse is continuous, so a cr diffeomorphism is a cr bij ection whose inverse is C. (We assume 1 � r � oo.) The inverse being C is not automatic. The example to remember is f (x) = x 3 . It is a C00 bijection JR. -4 JR. and is a homeomorphism but not a diffeomorphism because its inverse fails to be differentiable at the origin. Since differentiability implies continuity, every diffeomorphism is a homeomorphism. Diffeomorphisms are to cr things as isomorphisms are to algebraic things. The sphere and ellipsoid are diffeomorphic under a diffeomorphism JR.3 -4 JR. 3 , but the sphere and the surface of the cube are only homeomor­ phic, not diffeomorphic.

If m = n and f : U � JR.m is C , I � (Df) p is an isomorphism, then f is a C diffeomorphism from a neighborhood of p to a neighborhood of f(p). 23 Inverse Function Theorem r � oo, and if at some p E U,

f(x ) - y. Clearly F is C, F (p, fp ) = 0, and the derivative of F with respect to x at (p, fp) is (Df) p . Since (Df)p is Proof Set F (x ,

y)

=

an isomorphism, we can apply the implicit function theorem (with x and y interchanged ! ) to find neighborhoods Uo of p and V of fp and a C implici t function h : V � Uo uniquely defined by the equation

F (hy , y)

=

0.

Chapter 5

Multivariable Calculus

290

Then f (hy) = y, so f o h = idv and h is a right inverse of f . Except for a little fussy set theory, this completes the proof: f bijects U1 = {x E Uo : f x E V } onto V and its inverse is h , which we know to be a cr map. To be precise, we must check three things, (a) U1 is a neighborhood of p. (b) h is a right inverse of f u 1 • That is, f u o h = idv . 1 (c) h is a left inverse of f u1 • That is, h o f u1 = id u1 • See Figure 108.

l

l

l

l

h

Figure 108 f is a local diffeomorphism.

(a) Since f is continuous, Ut is open. Since p E U0 and fp E V , p belongs to U 1 · (b) Take any y E V . Since hy E Uo and f (hy) = y, we see that hy E U1 . Thus, f u1 o h is well defined and f u o h (y) = f o h (y) = y . 1 (c) Take any x E U1 . By definition of U1 , fx E V and there is a unique point h (fx) in Uo such that F (h (fx ) , fx) = 0. Observe that x itself is just such a point. It lies in Uo because it lies in U1 , and it satisfies F (x , fx) = 0 0 since F (x , fx) = fx - f x . By uniqueness of h, h (f (x)) = x .

l

l

Upshot If (Df)p is an isomorphism, then f is a local diffeomorphism

at p.

6*

The Rank Theorem

The rank of a linear transformation T : IRn � IRm is the dimension of its range. In terms of matrices, the rank is the size of the largest minor with nonzero determinant. If T is onto then its rank is m . If it is one-to-one, its rank is n . A standard formula in linear algebra states that rank T + nullity T = n

where nullity is the dimension of the kernel of T. A differentiable function f : U � IRm has constant rank k if for all p E U the rank of (Df)p is k.

The Rank Theorem

Section 6*

29 1

An important property of rank is that if T has rank k and I S - T I is small, then S has rank 2: k. The rank of T can increase under a small perturbation of T but it cannot decrease. Thus, if f is C1 and (Df) p has rank k then automatically ( Df) x has rank 2: k for all x near p. See Exercise 43 . The Rank Theorem describes maps of constant rank. It says that locally they are just like linear projections. To formalize this we say that maps f : A -4 B and g : C -4 D are equivalent (for want of a better word) if there are bijections a : A ----* C and f3 : B -4 D such that g = f3 o f o a - 1 . An elegant way to express this equation is as commutativity of the diagram f

A

B

g

c

D.

Commutativity means that for each a E A , {3(/(a )) = g (a (a )) . Following

the maps around the rectangle clockwise from A to D gives the same result as following them around it counterclockwise. The a. f3 are "changes of variable." If f, g are C and a, f3 are C diffeomorphisms, l :::::: r :::::: oo , then f and g are said to be cr equivalent, and we write f � g . As cr r maps, f and g are indistinguishable. 24 Lemma cr

on rank.

equivalence is an equivalence relation and it has no effect

Proof Since diffeomorphisms form a group, �r is an equivalence relation. Also. if g = f3 o f o a- 1 , then the chain rule implies that

Dg = D/3

o

o

Df

Da - 1 •

Since D/3 and Da - 1 are isomorphisms, Df and Dg have equ al rank.

P : !Rn -4 IRm

The linear projection P (x r

•

0

.

.

.

, Xn ) = (x r

•

.

.

[hxk OJ

, Xk o 0, . . . , 0)

.

has rank k. It projects !Rn onto the k-dimensional subspace IRk x 0. The matrix of P is 0

0

.

25 Rank Theorem Locally, a cr constant rank k map is cr equivalent to a linear projection onto a k-dimensional subspace.

Multivariable Calculus

292

Chapter 5

As an example, think of the radial projection n : JR3 \ {0} -+ S2 • where n (v) vj l v i . It has constant rank 2, and is locally indistinguishable from linear projection of JR3 to the (x . y )-plane. =

U -+ IR.m have constant rank k and let p E U be given. We will show that on a neighborhood of p , f � r P . Step 1 . Define translations of IR. n and IR.m by ---

Proof Let f :

r

'

: ]Rm -+ ]Rm

Z I-+ Z - fp I

I

The translations are diffeomorphisms of JR.n and !Rm and they show that f is cr equivalent to r ' 0 f 0 r . a cr map that sends 0 to 0 and has constant rank k. Thus, it is no loss of generality to assume in the first place that p is the origin in IR.n and fp is the origin in IR.m . We do so. Step 2. Let T : IR.n -+ IR.n be an isomorphism that sends 0 x !Rn - k onto the kernel of (D /)0 . Since the kernel has dimension n - k , there is such a T . Let T ' : IR.m -+ IR. m be an isomorphism that sends the image o f ( DJ)o onto JR.k x 0. Since ( DJ )o has rank k there is such a T'. Then f � r T ' o f o T . This map sends the origin i n IR.n t o the origin i n IR.m , its derivative at the origin has kernel 0 x JR.n -k, and its image JRk x 0. Thus, it is no loss of generality to assume in the first place that f has these properties. We do so. Step 3. Write --f (x , y )

=

Ux (x , y ) ,

fy (x ,

y) )

E IR

k

x

IR.m - k _

We are going to find a g � r f such that

g (x , 0) = (x , 0) . The matrix of (D /)0 is where A is k x k and invertible. Thus, by the Inverse Function Theorem. the map a : �--+ fx (x , 0)

x

is a diffeomorphism a : X -+ X ' where X, X' are small neighborhoods of the origin in JR.k . For x' E X'. set h (x' )

=

1 Jy (a - (x ' ) . 0) .

Secti on 6*

The Rank Theorem

This makes h a cr map

293

X' ---+ JR.m-k . and h ( a (x ) )

= fy (x . 0) .

The image of X x 0 under f is the graph of h . For f (X

X

0) = { f (x , 0) :

x

E X } = { { fx (x , 0) , fy (x , 0)) :

x

E X}

= { {fx (a - 1 (x ' ) . 0) , jy (a - 1 (x ') . 0) ) : x' E X'} = { (x' , h (x')) : x ' E X' } .

See Figure 1 09. y

X

X

Figure 109 The image of X

For (x ' ,

y ')

Y'

f

E X'

O"X

X'

x 0 is the graph of h.

x JR.m -k , define 1/f (x', y ' ) = ( a - 1 (x' ) , y ' - h (x ' ) ) .

Since l/f is the composite of cr diffeomorphisms,

(x' . y ' ) �---+ (x' , y ' - h (x')) �----* ( a - 1 (x ) , y ' - h (x') ) . '

it too i s a C diffeomorphism. (Alternately, you could compute the deriva­ tive of l/f at the origin and apply the Inverse Function Theorem.) We observe that g = l/f o f � f satisfies

r

g (x , 0) = l/f o ( fx (x , 0) , jy (x , 0 ) ) = ( a - 1 o fx ( x , 0) . Jy (x , 0) - h (fx (x , 0)) = (x , 0) . Thus, it is no loss of generality to assume in the first place that f (x , 0) = (x , 0) . We do so. Step 4. Finally, we find a local diffeomorphism q; in the neighborhood of O in JR.n so that f o q; is the projection map P (x , y ) = (x , 0) . The equation fx ( � , y ) - x = 0 defines � = � (x , y ) implicitly in a neighborhood of the origin; it is a cr map from JR.n into JR.k and has � (0, 0) = 0. For, at the origin, the derivative

294

Multivariable Calculus

Chapter 5

of fx (l; , y) - x with respect to � is the invertible matrix that ({J (x . y) = (Hx , y) , y )

hx k · We clrum

is a local diffeomorphism of JRn and G = f o (/) is P . The derivative of � (x , y ) with respect to x at the origin can be calculated from the chain rule (this was done in general for implicit functions) and it satisfies a F a� aF d F(�(x . y ) . x . y ) a� 0 = -+ hxk - hxk• a� ax dx ox ax -

[hxk

­

That is, at the origin a � ;ax is the identity matrix. Thus,

(D({J)o =

0

* l(n- k ) x (n- k )

J

which is invertible no matter what * is. Clearly