Mathematical Analysis A Concise Introduction
Bernd S. W. Schroder Louisiana Tech University Program of Mathematics and Statistics Ruston, LA
BtCLNTENNIAL
WILEY-INTERSCIENCE A John Wiley & Sons, Inc., Publication
This Page Intentionally Left Blank
Mathematical Analysis
THE W l L E Y BICENTENNIAL-KNOWLEDGEFOR
GENERATIONS
6
ach generation has its unique needs and aspirations. When Charles Wiley first opened his small printing shop in lower Manhattan in 1807, it was a generation of boundless potential searching for an identity. And we were there, helping to define a new American literary tradition. Over half a century later, in the midst of the Second Industrial Revolution, it was a generation focused on building the future. Once again, we were there, supplying the critical scientific, technical, and engineering knowledge that helped frame the world. Throughout the 20th Century, and into the new millennium, nations began to reach out beyond their own borders and a new international community was born. Wiley was there, expanding its operations around the world to enable a global exchange of ideas, opinions, and know-how.
For 200 years, Wiley has been an integral part of each generation's journey, enabling the flow of information and understanding necessary to meet their needs and fulfill their aspirations. Today, bold new technologies are changing the way we live and learn. Wiley will be there, providing you the must-have knowledge you need to imagine new worlds, new possibilities, and new opportunities.
Generations come and go, but you can always count on Wiley to provide you the knowledge you need, when and where you need it! rc'\
U WILLIAM J. PESCE PRESIDENT AND CHIEF
EXECUTIVEOFFICER
PETER B O O T H WlLEV CHAIRMAN
OF
THE BOARD
Mathematical Analysis A Concise Introduction
Bernd S. W. Schroder Louisiana Tech University Program of Mathematics and Statistics Ruston, LA
BtCLNTENNIAL
WILEY-INTERSCIENCE A John Wiley & Sons, Inc., Publication
Copyright C 2008 by John Wiley & Sons, Inc. All rights reserved Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-601 1, fax (201) 748-6008, or online at http://www.wiley.com/go/permission.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic format. For information about Wiley products, visit our web site at www.wiley.com. Wiley Bicentennial Logo: Richard J. Pacific0 Library of Congress Cataloging-in-Publication Data:
Schroder, Bernd S. W. (Bemd Siegfried Walter), 1966Mathematical analysis : a concise introduction / Bernd S.W. Schroder p. cm. ISBN 978-0-470-10796-6 (cloth) 1. Mathematical analysis. I. Title. QA300.S376 2007 5 15-dc22 2007024690 Printed in the United States of America.
Contents Table of Contents
V
Preface
xi
Part I: Analysis of Functions of a Single Real Variable 1 The Real Numbers 1.1 Field Axioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Order Axioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Lowest Upper and Greatest Lower Bounds . . . . . . . . . . . . . . . 1.4 Natural Numbers, Integers. and Rational Numbers . . . . . . . . . . . 1.5 Recursion. Induction. Summations. and Products . . . . . . . . . . .
1 1 4 8 11 17
:2 Sequences of Real Numbers 2.1 Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Limit Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 CauchySequences . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Bounded Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Infinite Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25 25 30 36 40 44
:3 Continuous Functions 3.1 Limits of Functions . . . . . . . . . . . . . . . . . . . . . . . . . . .
49 49
3.2 Limit Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 One-sided Limits and Infinite Limits . . . . . . . . . . . . . . . . . . 3.4 3.5 3.6 4
Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Properties of Continuous Functions . . . . . . . . . . . . . . . . . Limits at Infinity . . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
. . 66
Differentiable Functions 4.1 Differentiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Differentiation Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Rolle’s Theorem and the Mean Value Theorem . . . . . . . . . . . . V
52 56
69
71 71 74 80
vi
Con tents
5 The Riemann Integral I 5.1 Riemann Sums and the Integral . . . . . . . . . . . . . . . . . . . . . 5.2 Uniform Continuity and Integrability of Continuous Functions . . . . 5.3 The Fundamental Theorem of Calculus . . . . . . . . . . . . . . . . 5.4 The Darboux Integral . . . . . . . . . . . . . . . . . . . . . . . . . .
85 85 91 95 97
6 Series of Real Numbers I 101 6.1 Series as a Vehicle To Define Infinite Sums . . . . . . . . . . . . . . 101 6.2 AbsoluteConvergenceandUnconditionalConvergence . . . . . . . . 108 7
Some Set Theory 7.1 The Algebra of Sets . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Countable Sets . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Uncountable Sets . . . . . . . . . . . . . . . . . . . . . . . . .
117 117 122 124
8
The Riemann Integral I1 8.1 Outer Lebesgue Measure . . . . . . . . . . . . . . . . . . . . . . . 8.2 Lebesgue’s Criterion for Riemann Integrability . . . . . . . . . . . 8.3 More Integral Theorems . . . . . . . . . . . . . . . . . . . . . . . 8.4 Improper Riemann Integrals . . . . . . . . . . . . . . . . . . . . .
127 127 131 136 140
9 The Lebesgue Integral 145 9.1 Lebesgue Measurable Sets . . . . . . . . . . . . . . . . . . . . . . . 147 9.2 Lebesgue Measurable Functions . . . . . . . . . . . . . . . . . . . . 153 9.3 Lebesgue Integration . . . . . . . . . . . . . . . . . . . . . . . . . . 158 9.4 Lebesgue Integrals versus Riemann Integrals . . . . . . . . . . . . . 165 10 Series of Real Numbers I1 10.1 Limits Superior and Inferior . . . . . . . . . . . . . . . . . . . . . . 10.2 The Root Test and the Ratio Test . . . . . . . . . . . . . . . . . . . . 10.3 Power Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
169 169 172
11 Sequences of Functions 11.1 Notions of Convergence . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Uniform Convergence . . . . . . . . . . . . . . . . . . . . . . . . . .
179 179 182
12 Transcendental Functions 12.1 The Exponential Function . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Sine and Cosine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3 L‘H6pital’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
189 189 193 199
13 Numerical Methods 13.1 Approximation with Taylor Polynomials . . . . . . . . . . 13.2 Newton’s Method . . . . . . . . . . . . . . . . . . . . . . . 13.3 Numerical Integration . . . . . . . . . . . . . . . . . . . . .
203 204 208 214
.
175
Con tents
vii
Part 11: Analysis in Abstract Spaces 14 Integration on Measure Spaces 14.1 Measure Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
225 225
14.2 Outer Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3 Measurable Functions . . . . . . . . . . . . . . . . . . . . . . . . . .
230 234
14.4 Integration of Measurable Functions . . . . . . . . . . . . . . . . . . 235 14.5 Monotone and Dominated Convergence . . . . . . . . . . . . . . . . 238 14.6 Convergence in Mean. in Measure. and Almost Everywhere . . . . . 14.7 Product a-Algebras . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 242 245
14.8 Product Measures and Fubini’s Theorem . . . . . . . . . . . . . . . . 251
15 The Abstract Venues for Analysis 15.1 Abstraction I: Vector Spaces . . . . . . . . . . . . . . . . . . . . . . 15.2 Representation of Elements: Bases and Dimension . . . . . . . . . 15.3 Identification of Spaces: Isomorphism . . . . . . . . . . . . . . . . 15.4 Abstraction 11: Inner Product Spaces . . . . . . . . . . . . . . . . . 15.5 Nicer Representations: Orthonormal Sets . . . . . . . . . . . . . . 15.6 Abstraction 111: Normed Spaces . . . . . . . . . . . . . . . . . . . . 15.7 Abstraction IV: Metric Spaces . . . . . . . . . . . . . . . . . . . . . 15.8 LP Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.9 Another Number Field: Complex Numbers . . . . . . . . . . . . .
. . . .
.
255 255 259 262 264 267 269 275 278 281
16 The Topology of Metric Spaces 16.1 Convergence of Sequences . . . . . . . . . . . . . . . . . . . . . . . 16.2 Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3 Continuous Functions . . . . . . . . . . . . . . . . . . . . . . . . . .
287 287 29 1 296
16.4 Open and Closed Sets . . . . . . . . . . . . . . . . . . . . . . . . . . 16.5 Compactness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
301 309 316 322 330 333
16.6 The Normed Topology of Rd . . . . . . . . . . . . . . . . . . . . . . 16.7 Dense Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.8 Connectedness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.9 Locally Compact Spaces . . . . . . . . . . . . . . . . . . . . . . . .
17 Differentiation in Normed Spaces 341 17.1 Continuous Linear Functions . . . . . . . . . . . . . . . . . . . . . . 342 17.2 Matrix Representation of Linear Functions . . . . . . . . . . . . . . . 348 17.3 Differentiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 17.4 The Mean Value Theorem . . . . . . . . . . . . . . . . . . . . . . . 360 17.5 How Partial Derivatives Fit In . . . . . . . . . . . . . . . . . . . . . 362 17.6 Multilinear Functions (Tensors) . . . . . . . . . . . . . . . . . . . . 369 17.7 Higher Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 17.8 The Implicit Function Theorem . . . . . . . . . . . . . . . . . . . . . 380
...
Vlll
Con tents
18 Measure. Topology. and Differentiation 18.1 Lebesgue Measurable Sets in Rd . . . . . . . . . . . . . . . . . . . . 18.2 Cco and Approximation of Integrable Functions . . . . . . . . . . . . 18.3 Tensor Algebra and Determinants . . . . . . . . . . . . . . . . . . . 18.4 Multidimensional Substitution . . . . . . . . . . . . . . . . . . . . .
385 385 391 397 407
19 Introduction to Differential Geometry 42 1 19.1 Manifolds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421 19.2 Tangent Spaces and Differentiable Functions . . . . . . . . . . . . . . 427 19.3 Differential Forms. Integrals Over the Unit Cube . . . . . . . . . . . 434 19.4 k-Forms and Integrals Over k-Chains . . . . . . . . . . . . . . . . . . 443 19.5 Integration on Manifolds . . . . . . . . . . . . . . . . . . . . . . . . 452 19.6 Stokes’ Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458 20 Hilbert Spaces 20.1 Orthonormal Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.2 Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.3 The Riesz Representation Theorem . . . . . . . . . . . . . . . . . . .
463 463 467 475
Part 111: Applied Analysis 21 Physics Background 2 1.1 Harmonic Oscillators . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Heat and Diffusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Separation of Variables. Fourier Series. and Ordinary Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.4 Maxwell’s Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 21.5 The Navier Stokes Equation for the Conservation of Mass . . . . . . .
483 484 486
22 Ordinary Differential Equations 22.1 Banach Space Valued Differential Equations . . . . . . . . . . . . . . 22.2 An Existence and Uniqueness Theorem . . . . . . . . . . . . . . . . 22.3 Linear Differential Equations . . . . . . . . . . . . . . . . . . . . . .
505 505 508 510
23 The Finite Element Method 23.1 Ritz-Galerkin Approximation . . . . . . . . . . . . . . . . . . . . . . 23.2 Weakly Differentiable Functions . . . . . . . . . . . . . . . . . . . . 23.3 Sobolev Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23.4 Elliptic Differential Operators . . . . . . . . . . . . . . . . . . . . . 23.5 Finite Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
513 513 518 524 532 536
Conclusion and Outlook
490 493 496
544
Con tents
1x
Appendices A Logic A.l Statements. . . . . . . . . . . . . . . . . . . . , . . . . . . . . . A.2 Negations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
545
. ,
. 545
. 546
B SetTheory 547 B.l The Zermelo-Fraenkel Axioms . . . . . . . . . . . . . . . . . . . . . 547 B.2 Relations and Functions . . . . . . . . . . . . . . . . . . . . . . . . . 548
C Natural Numbers, Integers, and Rational Numbers C.1 The Natural Numbers . . , . . , . . . . . . . . . . . . . . . . . . .
549 . 549 C.2 The Integers . . . . . . . . . . . . . . . . . . . . . . . , . . . . . . . 550 C.3 The Rational Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 550
Bibliography
55 1
Index
553
Con tents
X
Theon
systems
Background: Brieflv in Amendices
Chapter 6 senes I I
I
Part I: Analysis on R
Counrabdii)
Chapter 5 RlemB""
+
Chapter 8 RlCma""
Chapter I I Sequencesof Functions
Integral I
Integral II
Chapter 13
Chapter 9 Lcbeigue Integral
Chapter 12 Trmscendenral
Chapter 21 Ph>iics Background
Chapter 22 Ordmay Differential Equations
4
h"lIleClCd
Methods
4
c
F""Ctl0"S
Part 11: Abstract Analysis
Part 111: Applications
Chapter 21 Panial DEr. Finite Elements
Figure 1: Content dependency chart with minimum prerequisites indicated by arrows. Some remarks, examples, and exercises in the later chapter might still depend on other earlier chapters, but this problem typically can be resolved by quoting a single result. Details about where and how the reader can "branch out" are given in in the text.
Preface This text is a self-contained introduction to the fundamentals of analysis. The only prerequisite is some experience with mathematical language and proofs. That is, it helps to be familiar with the structure of mathematical statements and with proof methods, such as direct proofs, proofs by contradiction, or induction. With some support in the right places, mostly in the early chapters, this text can also be used without prerequisites in a first proof class. Mastering proofs in analysis is one of the key steps toward becoming a mathematician. To develop sound proof writing techniques, standard proof techniques are discussed early in the text and for a while they are pointed out explicitly. Throughout, proofs are presented with as much detail and as little hand waving as possible. This makes some proofs (for example, the density of C [ a ,b] in L P [ a ,b]in Part 11)notationally a bit complicated. With computers now being a regular tool in mathematics, the author considers this appropriate. When code is written for a problem, all details must be implemented, even those that are omitted in proofs. Seeing a few highly detailed proofs is reasonable preparation for such tasks. Moreover, to facilitate the transition to more abstract settings, such as measure, inner product, normed, and metric spaces, the results for single variable functions are proved using methods that translate to these abstract settings. For example, early proofs rely extensively on sequences and we also use the completeness of the real numbers rather than their order properties. Analysis is important for applications, because it provides the abstract background that allows us to apply the full power of mathematics to scientific problems. This text shows that all abstractions are well motivated by the desire to build a strong theory that connects to specific applications. Readers who complete this text will be ready for all analysis-based and analysis-related subjects in mathematics, including complex analysis, differential equations, differential geometry, functional analysis, harmonic analysis, mathematical physics, measure theory, numerical analysis, partial differential equations, probability theory, and topology. Readers interested in motivation from physics are advised to browse Chapter 21, even if they have not read any of the earlier chapters. Aside from the topics covered, readers interested in applications should note that the axiomatic approach of mathematics is similar to problem solving in other fields. In mathematics, theories are built on axioms. Similarly, in applications, models are subject to constraints. Neither the axioms, nor the constraints can be violated by the theory or model. Building a theory based on axioms fosters the reader's discipline to not make unwarranted assumptions.
xii
Preface
Organization of the content. The text consists of three large parts. Part I, comprised of Chapters 1-13, presents the analysis of functions of one real variable, including a motivated introduction to the Lebesgue integral. Chapters 1-6 and 10-13 could be called “single variable calculus with proofs.” For a smooth transition from calculus and a gradual increase in abstraction, Chapters 1-6 require very little set theory. Chapter 1 presents the properties of the real line and limits of sequences are introduced in Chapter 2. Chapters 3-5 present the fundamentals on continuity, differentiation, and (Riemann) integration in this order and Chapter 6 gives a first introduction to series. Chapters 6-8 are motivated by the desire to further explore the Riemann integral while avoiding the excessive use of Riemann sums. This exploration is done with the Lebesgue criterion for Riemann integrability. Although this criterion requires the Lebesgue measure, the payoff is that many proofs become simpler. To quickly reach this criterion, the first presentation of series in Chapter 6 is deliberately kept short. It presents enough about series to allow the definition of Lebesgue measure. Chapter 7 presents fundamental notions of set theory. Most of these ideas are needed for Lebesgue measure, but, overall, Chapter 7 contains all the set theory needed in the remainder of the text. Chapter 8 finishes the presentation of the Riemann integral. With Lebesgue measure available, it is natural to investigate the Lebesgue integral in Chapter 9. This chapter could also be delayed to the end of Part I, but the author believes that early exposure to the crucial ideas will ease the later transition to measure spaces. The analysis of single variable functions is finished with the rigorous introduction of the transcendental functions. The necessary background on power series is explored in Chapter 10. Chapter 11 presents some fundamentals on the convergence of sequences of functions and Chapter 12 is devoted to the transcendental functions themselves. Chapter 13 discusses general numerical methods, but transcendental functions provide a rich test bed for the methods presented. Part I of the text can be read or presented in many orders. Figure 1 shows the prerequisite structure of the text. Prerequisites for each chapter have deliberately been kept minimal. In this fashion, the order of topics in the reader’s first contact with proofs in analysis can be adapted to many readers’ preferences. Most notably, the intentionally early presentation of Lebesgue integration can be postponed to the end of Part I if so desired. Throughout, the author intends to keep the reader engaged by providing motivation for all abstractions. Consequently, as Figure 1 and the table of contents indicate, some concepts and results are presented in a “just-in-time’’ fashion rather than in what may be considered their traditional place. If a concept is needed in an exercise before the concept is “officially” defined in the text, the concept will be defined in the exercise and in the text. Part 11, comprised of Chapters 14-20, explores how the appropriate abstractions lead to a powerful and widely applicable theoretical foundation for all branches of applied mathematics. The desire to define an integral in d-dimensional space provides a natural motivation to introduce measure spaces in Chapter 14. This chapter facilitates the transition to more abstract mathematics by frequently referring back to corresponding results for the one dimensional Lebesgue integral. The proofs of these results usually are verbatim the same as in the one-dimensional setting. Moreover, this early introduction makes LP spaces available as examples for the rest of the text. The abstract venues of analysis are then presented in Chapter 15, which provides all examples
Preface
...
Xlll
for the rest of Part 11. The fundamentals on metric spaces and continuity are presented in Chapter 16. As with measure spaces, for several results on metric spaces the reader is referred back to the corresponding proof for single variable functions. Proofs are no longer verbatim the same and abstraction is facilitated by translating proofs from a familiar setting to the new setting while analyzing similarities and differences. In a class, the author suggests that the teacher fill in some of these proofs to demonstrate the process. Chapter 17 presents the fundamentals on normed spaces and differentiation. Again, ideas are similar to those for functions of a single variable, but this time the abstraction goes beyond translation. With all three fundamental concepts (integration, continuity, and differentiation) available in the abstract setting, Chapter 18 shows the interrelationship between concepts presented separately before, culminating in the Multivariable Substitution Formula. The second part is completed by a presentation of the fundamentals of analysis on manifolds, together with a physical interpretation of key concepts in Chapter 19 and by an introduction to Hilbert spaces in Chapter 20. The remaining chapters give a brief outlook to applied subjects in which analysis is used, specifically, physics in Chapter 21, ordinary differential equations in Chapter 22, and partial differential equations and the finite element method in Chapter 23. Each of these chapters can only give a taste of its subject and I encourage the reader to go deeper into the utterly fascinating applications that lie behind part 111. The mathematical preparation through this text should facilitate the transition. It should be possible to cover the bulk of the text in a two course sequence. Although Chapters 14-16 should be read in order, depending on the available time, the pace and the choice of topics, any of Chapters 17-23 can serve as a capstone experience.
How to read this text. Mathematics in general, and analysis in particular, is not a spectator sport. It is learned by doing. To allow the reader to “do” mathematics, each section has exercises of varying degrees of difficulty. Some exercises require the adaptation of an argument in the text. These exercises are also intended to make the reader critically analyze the argument before adapting it. This is the first step toward being able to write proofs. Of course the need for very critical (and slow) reading of mathematics is nicely summed up in the old quote that “To read without a pencil is daydreaming.” The reader should ask himherself after every sentence “What does this mean? Why is this justified?’ Making notes in the margin to explain the harder steps will allow the reader to answer these questions more easily in the second and third readings of a proof. So it is important to read thoroughly and slowly, to make notes and to reread as often as needed. The extensive index should help with unknown or forgotten terminology as necessary. Other exercises have hints on how to create a proof that the reader has not seen before. These exercises require the use of proof techniques in a new setting. Finally, there are also exercises without hints. Being able to create the proof with nothing but the result given is the deepest task in a mathematics course. This is not to say that exercises without hints are always the hardest and adaptations are always the easiest, but in many cases this is true. Finally, some exercises give a sequence of hints and intermediate results leading up to a famous theorem or a specific example. These exercises could also be used as mini-projects. In a class, some of them
xiv
Preface
could be the basis for separate lectures that spotlight a particular theorem or example. To get the most out of this text, the reader is encouraged to not look for hints and solutions in other background materials. In fact, even for proofs that are adaptations of proofs in this text, it is advantageous to try to create the proof without looking up the proof that is to be adapted. There is evidence that the struggle to solve a problem, which can take days for a single proof, is exactly what ultimately contributes to the development of strong skills. “Shortcuts,” while pleasant, can actually diminish this development. Readers interested in quantitative evidence that shows how the struggle to acquire a skill actually can lead to deeper learning may find the article [4] quite enlightening. A better survival mechanism than shortcuts is the development of connections between newly learned content and existing knowledge. The reader will need to find these connections to hisher existing knowledge, but the structure of the text is intended to help by motivating all abstractions. Readers interested in how knowledge is activated more easily when it was learned in a known context may be interested in the article [5].
Acknowledgments. Strange as it may sound, I started writing this text in the spring of 1987, as I prepared for my oral final examination in the traditional Analysis I111 sequence in Germany. Basically, I took all topics in the sequence and arranged them in what was the most logical fashion to me at the time. Of course, these notes are, in retrospect, immature. But they did a lot to shape my abilities and they were a good source of ideas and exercises. In this respect, I am indebted to my teachers for this sequence: Professor Wegener and teaching assistant Ms. Lange for Analysis I, Professor Kutzler and teaching assistant Herr Bottger for Analysis 11-111 as well as Professor Herz in whose Differential Equations class I first saw analysis “at work.” With all due respect to the other individuals, to me and many of my fellow students, the force that drove us in analysis (and beyond) was Herr Bottger. This gentleman was uncompromising in his pursuit of mathematical excellence and we feared as well as looked forward to his demanding exercise sets. He was highly respected because he was ready to spend hours with anyone who wanted to talk mathematics. Those who kept up with him were extremely well prepared for their mathematical careers. Incidentally, Dr. Ansgar Jungel, whose notes I used for the chapter on the finite element method, took the above mentioned classes with me. The thorough preparation through these classes is the main reason why most of this text was comparatively easy to write. If this text does half as good a job as Herr Bottger did with us, it has more than achieved its purpose. It was thrilling to test my limitations, it was humbling to find them and ultimately I was left awed once more by the beauty of mathematics. When my abilities were insufficient to proceed, I used the texts listed in the bibliography for proofs, hints or to structure the presentation. To make the reader fully concentrate on matters at hand, and to force myself to make the exposition self-contained, outside references are limited to places where results were beyond the scope of this exposition. A solid foundation will allow readers to judiciously pick their own resources for further study. Nonetheless, it is appropriate to recognize the influence of the works of a number of outstanding individuals. I used Adams [2], Renardy and Rogers [23], Yosida [33] and Zeidler [34] for Sobolev spaces, Aris 131, Cramer’s http: //www.navier-stokes .net/,and
Preface
xv
Welty, Wicks and Wilson [31] for fluid dynamics, Chapman [6] for heat transfer, Cohn [7] for measure theory, DieudonnC [8] for differentiation in Banach spaces, Dodge [9] and Halmos [ 131 for set theory, Ferguson [ 101, Sandefur [24] and Stoer and Bulirsch [28] for numerical analysis, Halliday, Resnick and Walker [ 121 for elementary physics, Hewitt and Stromberg [14], Heuser [15], [16], Johnsonbaugh and Pfaffenberger [20], Lehn [22] and Stromberg [29] for general background on analysis, Heuser [17] for functional analysis, Hurd and Loeb [18] for the use of quantifiers in logic, Jiingel [21] and Solin [25] for the finite element method, Spivak [26], [27] for manifolds, Torchinsky [30] for Fourier series, Willard [32] for topology, and the Online Encyclopaedia of Mathematics http : / /eom.springer.d e / for quick checks of notation and definitions. Readers interested in further study of these subjects may wish to start with the above references. The first draft of the manuscript was used in my analysis classes in the Winter and Spring quarters of 2007. The first class covered Chapters 1-9, the second covered Chapters 11 and 14-18 (with some strategic “fast forwards”). This setup assured that graduating students would have full exposure to the essentials of analysis on the real line and to as much abstract analysis as possible without “handwaving arguments.” I am grateful to the students in these classes for keeping up with the pace, solving large numbers of homework problems, being patient with the typos we found and also for suggesting at least one order in which to present the material that I had not considered. The students’ evaluations (my best ever) also reaffirmed for me that people will enjoy, or at least accept and honor, a challenge, and that an ambitious, motivated course should be the way to go. Devery Rowland once more did an excellent job printing drafts of the text for the classes. Aside from the referees, several colleagues also commented on this text and I owe them my thanks for making it a better product. In particular, I would like to thank Natalia Zotov for some comments on an early version that significantly improved the presentation, and Ansgar Jiingel for pointing out some key references on Sobolev spaces. Although I hope that we have found all remaining errors and typos, any that remain are my responsibility and mine alone. I request readers to report errors and typos to me so I can post an errata. My contacts at Wiley, Susanne Steitz, Jacqueline Palmieri, and Melissa Yanuzzi bore with me when the stress level rose and their patience made the publishing process very smooth. As always, this work would not have been possible without the love of my family. It is truly wonderful to be supported by individuals who accept your decision to spend large amounts of time reliving your formative years. Finally, I was sad to learn that Herr Bottger died unexpectedly a few years after I had my last class with him. Sir, this one’s for you.
Ruston, LA, August 30,2007 Bernd Schroder
Part I Analysis of Functions
of a Single Real Variable
Chapter 1
The Real Numbers This investigation of analysis starts with minimal prerequisites. Regarding set theory, the terms “set” and “element” will remain undefined, as is customary in mathematics to avoid paradoxes. The empty set 0 is the set that has no elements. The statement “e E S” says that e is an element of the set S. The statement “ A G B” says that every element of A is an element of B . Sets A and B are equal if and only if A C B and B C A . The statement “A c B” says that A E B and A # B . Subsets will be defined as “ A = {x E S : (property)},”that is, with a statement from which set S the elements of A are taken and a property describing them. The union of two sets A and B is A U B = {x : x E A o r x E B } , theintersectionis A n B = {x : x E A andx E B ) .
u n
Union and intersection of finitely many sets are denoted
j=l
n
A j and
n
A j , respec-
j=1
tively, and the relative complement of B in A is A \ B = {x E A : x @ B ) . Further details on set theory are purposely delayed until Section 7.1. Until then, we focus on analytical techniques. Any required notions of set theory will be clarified on the spot. To define properties, sometimes the universal quantifier “V” (read “for all”) or the existential quantifier “3” (read “there exists”) are used. Formal logic is described in more detail in Appendix A. Finally, the reader needs an intuitive idea what a function, a relation and a binary operation are. Details are relegated to Appendices B.2 and C.2. The real numbers R are the “staging ground” for analysis. They can be characterized as the unique (up to isomorphism) mathematical entity that satisfies Axioms 1.1, 1.6, and 1.19. That is, they are the unique linearly ordered, complete field (see Exercise 1-30). In this chapter, we introduce the axioms for the real numbers and some fundamental consequences. These results assure that the real numbers indeed have the properties that we are familiar with from algebra and calculus.
1.1 Field Axioms The description of the real numbers starts with their algebraic properties.
1
2
1. The Real Numbers
Axiom 1.1 The real numbers R are a field. That is, R has at least two elements and there are two binary operations, addition : R x R + R and multiplication . : R x R -+ R,so that
+
1. Addition is associative, that is, for all x , y , z
(x
R we have
E
R we have
+ y ) + z = x + (y + z).
2. Addition is commutative, that is, for all x , y x
E
+y = y +x.
3. There is a neutral element 0 f o r addition, that is, there is an element 0 t h a t f o r a l l x E R we havex + 0 = x .
4. For every element x x + (-x) = 0.
E
R so
E
R there is an additive inverse element (-x) so that E
R we have
6. Multiplication is commutative, that is, for all x , y E
R we have
5. Multiplication is associative, that is, for all x , y , z (x . y ) . z = x . ( y . z ) . x ‘ 4 ’ = y .x.
7. There is a neutral element 1f o r multiplication, that is, there is an element 1 E R so that for all x E R we have 1 . x = x . 8. For every element x E t h a t x . x - l = 1.
R \ { 0 }there is a multiplicative inverse element x - l so
9. Multiplication is (left) distributive over addition, that is, f o r all a , x , y have a . (x y) = a, .x + a . y .
+
E
R we
As is customary for multiplication, the dot between factors is usually omitted. Fields are investigated in detail in abstract algebra. For analysis, it is most effective to remember that the field axioms guarantee the properties needed so that we can perform algebra and arithmetic “as usual.” Some of these properties are exhibited in this section and in the exercises. The exercises also include examples that show that not every field needs to be infinite (see Exercises 1-7-1-9).
Theorem 1.2 The following are true in R: 1. For all x
E
R,we have Ox
= 0.
2. 0 # 1. 3. Additive inverses are unique. That is, i f x E property in part 4 ofAxiom 1.1, then x’ = X.
4. For all x
E
R,we have (- l)x
= -x.
R and
x’ and
X both have the
3
1.1. Field Axioms
Proof. Early in the text, proofs will sometimes be interrupted by comments in italics to point out standard formulations and proof techniques. To prove part 1, let x E R. Then the axioms allow us to obtain the following Ax.3
Ax.6
Ax.9
Ax.6
equation. Ox = (O+O)x = x(O+O) = xO+xO = Ox +Ox. This implies
as was claimed. The proof of part I shows how every step in a proof needs to be just$ed. Usually we will not explicitly justify each step in a computation with an axiom or a previous result. Howevel; the reader should always mentallyfill in thejusti3cation. The practice offilling in these justiJcations should be started in the computations in the remainder of this proot To prove part 2, first note that, because R has at least two elements, there is an x E R \ ( 0 ) . Now suppose for a contradiction (see Standard Proof Technique 1.4 below) that 0 = 1. Then x = 1 . x = 0 . x = 0 is a contradiction to x E R \ ( 0 ) . For part 3, note that if x’and X both have the property in part 4 of Axiom 1.1, then x’ = x’+O = x’+(x+X) = (x’+x)+X = (x+x’)+X = O+X = X+O = X.Note that the statement of part 3 already encodes the typical approach to a uniqueness proof (see Standard Proof Technique 1.5 below). Finally, for part 4 note that x (- l ) x = l x (- 1)x = (1 (- 1)). = Ox = 0. Because by part 3 additive inverses are unique, (- l)x must be the additive inverse -x of x . The last step is a typical application of modus ponens, see Standard Proof Technique 1.3 below.
+
+
+
To familiarize the reader with standard proof techniques, these techniques will be pointed out explicitly in the early part of the text. The techniques presented in Chapter 1 are general proof techniques applicable throughout mathematics. Techniques presented in later chapters are mostly specific to analysis.
Standard Proof Technique 1.3 The simplest mathematical proof technique is a direct proof in which a result that says “ A implies B” is applied after we have proved that A is true. Truth of A and of “ A implies B” guarantees truth of B . This technique is also called modus ponens. An example is in the proof of part 4 of Theorem 1.2. 0 Standard Proof Technique 1.4 In a proof by contradiction, we suppose the contrary (the negation, also see Appendix A.2) of what is claimed is true and then we derive a contradiction. Typically, we derive a statement and its negation, which is a contradiction, because they cannot both be true. For an example, see the proof of part 2 of Theorem 1.2 above. Given that the reasoning that led to the contradiction is correct, the contradiction must be caused by the assumption that the contrary of the claim is true. Hence, the contrary of the claim must be false, because true statements cannot imply false statements like contradictions (see part 3 of Definition A.2 in Appendix A). But this means the claim must be true. We will usually indicate proofs by contradiction with a starting statement like “suppose for a contradiction.” 0
4
1. The Real Numbers
Standard Proof Technique 1.5 For many mathematical objects it is important to assure that they are the only object that has certain properties. That is, we want to assure that the object is unique. In a typical uniqueness proof, we assume that there is more than one object with the properties under investigation and we prove that any two of these objects must be equal. Part 3 of Theorem 1.2 shows this approach.
Exercises 1-1. Prove that (-1). (-1) = 1. 1-2.
. is right distributive over +. Prove that for all x , y , z
E
R we have (x + y ) z = xz + yz.
1-3. Multiplicative inverses are unique. Prove that if x E W and x' and X both have the property in part 8 of Axiom 1.1 then x' = X. 1-4. Prove that 0 does not have a multiplicative inverse.
1-5. Prove that if x , y # 0, then ( x y ) - ' = y - l x - ' .
Conclude in particular that x y # 0.
1-6. Prove each of the binomial formulas below. Justify each step with the appropriate axiom
+ b ) 2 = a* + 2ab + b2 (a + b ) ( a - b ) = a2 - b2
(b) ( a - b)* = a 2 - 2ab
(a) ( a (c)
+ b2 +
1-7. Prove that the set (0, 1) with the usual multiplication and the usual addition, except that 1 1 := 0, is a field. That is, prove that the set and addition and multiplication as stated have the properties listed in Axiom 1.1. 1-8. Prove that the set (0, 1. 2 ) with the sum and product of two elements being the remainder obtained
when dividing the regular sum and product by 3 is a field.
1-9. A property and some finite fields (a) Let F be a field and let x , y
E
F . Prove that x y = 0 if and only if x = 0 or y = 0
(b) Prove that the set [O. 1, 2. 3 ) with the sum and product of two elements being the remainder obtained when dividing the regular sum and product by 4 is not a field. (c) Prove that the set (0, 1, . , , , p - 1) with the sum and product of two elements being the remainder obtained when dividing the regular sum and product by p is a field if and only if p is a prime number.
1.2 Order Axioms Exercises 1-7-1-9c show that the field axioms alone are not enough to describe the real numbers. In fact, fields need not even be infinite. However, aside from executing the familiar algebraic operations, we can also compare real numbers. This section presents the order relation on the real numbers and its properties.
Axiom 1.6 The real numbers R contain a subset R+,called the positive real numbers such that 1. For all x , y
2. For all x
E
Either x E
E
R+,we have x + y
E
E%+ and x y
E
E%+,
R, exactly one of the following three properties holds. R+ or -x E Rt or x = 0.
5
1.2. Order Axioms
A real number x is called negative if and only if -x E R+. Once positive numbers are defined, we can define an order relation. As usual, instead of writing y (-x) we write y - x and call it the difference of x and y. The binary operation “-” is called subtraction. The phrase “if and only if,” which is used in definitions and biconditionals, is normally abbreviated with the artificial word “iff.”
+
Definition 1.7 For x,y E R,we say x is less than y, in symbols x < y, i f f y -x E R+. We say x is less than or equal to y, denoted x 5 y, ifSx < y or x = y. Finally, we say x is greater than y, denoted x > y, i r y < x,and we say x is greater than or equal to y , denoted x 2 y, ifsy 5 x. The relation 5 satisfies the properties that define an order relation.
Proposition 1.8 The relation 5 is an order relation on R.That is, 1. 5 is reflexive. For all x
E
R we have x 5 x,
2. 5 is antisymmetric. For all x,y x = y,
E
R we have that x 5 y and y 5 x implies
3. 5 is transitive. For all x, y , z E X , we have that x 5 y and y 5 z implies x 5 z. Moreovel; the relation 5 is a total order relation, that is, f o r any two x,y E have that x 5 y or y 5 x.
R we
Proof. The relation 5 is reflexive, because it includes equality. For antisymmetry, let x 5 y and y 5 x and suppose for a contradiction that x y . Then x - y E R+ and -(x - y ) = y - x E R+,which cannot be by Axiom 1.6. Thus < must be antisymmetric. For transitivity, let x 5 y and y 5 z . There is nothing to prove if one of the inequalities is an equality. Thus we can assume that x y and y < z , which means y - x E Rf and z - y E R+.But then R+ contains (7, - y) ( y - x) = z - x, and hence x < z . We have shown that for all x,y. z E R the inequalities x 5 y and y I :z imply x 5 z , which means that 5 is transitive. For the “moreover” part note that if x,y E R,then y - x E R and we have either y - x E R+,which means x < y , or y - x = 0, which means y = x, or x - y = - ( y - x ) E R + , w h i c h m e a n s y < x . Thereforeforallx,y E R o n e o f x 5 y or y 5 x holds, and hence 5 is a total order.
+
+
Once an order relation is established, we can define intervals.
Definition 1.9 An interval is a set I C R so that for all c , d E I and x E R the inequalities c < x < d imply x E I . In particular for a , b E R with a < b we define 1. [ a , b] := (X E
R : a 5 x 5 b},
2. ( a , b ) := (x E R : a < x < b ] , ( a , 00) := (x E R : a < x}, (-00. b ) := (X E R : x < b}, (-w, 00) := R,
6
1. The Real Numbers
3. [ a ,b ) := {x E
R :a 5 x
4. ( a , b ] := {X
R :u
E
< b } , [ a , 00) := { X E R : u 5 x},
< x 5 b ] , (-w, b ] := ( X E R : x 5 b].
The points a and b are also called the endpoints of the interval. A n interval that does not contain either of its endpoints (where &m are also considered to be "endpoints") is called open, An interval that contains exactly one of its endpoints is called half-open and an interval that contains both its endpoints is called closed. For the first part of this text, the domains of functions will almost exclusively be intervals. Because analysis requires extensive work with inequalities, we need to investigate how the order relation relates to the algebraic operations.
Theorem 1.10 Properties of the order relation. Let x , y , z E R. 1. The number x is positive ifsx > 0 and x is negative c r x < 0. 2. I f x 5 y , then x
+ z 5 y + z.
3. I f x 5 y and z > 0, then xz 5 y z . 4. I f x 5 y and
z
< 0, then xz 2 y z .
5. l f 0 < x 5 y , then y-' 5 x-'. Similar results can be proved f o r other combinations of strict and nonstrict inequalities. We will not state these here, but instead trust that the reader can make the requisite translation from the statements in this theorem.
Proof. Parts 1 and 2 are left to the reader as Exercises 1-10a and 1-lob. Throughout this text, parts of proofs will be delegated to the reader to facilitate a better connection to the material presented. For part 3 , let x 5 y and let z > 0. Then, y - x E R+ or y = x. In case y = x, we obtain y z = x z and thus, in particular, xz 5 y z . In case y - x E R+,note that z > 0 means z E R+, and hence y z - xz = ( y - x ) z E R+.By definition, this implies xz < y z , and in particular xz 5 y z . Because we have shown xz 5 y z in each case, the result is established. All proofs in this section are done with the above kind of case distinction (see Standard Proof Technique 1.1 1). For part 4, let x 5 y and let z < 0. Then, y - x E R+ or y = x. In case y = x, we obtain y z = x z , and hence xz 2 y z . In case y - x E Rf,note that z < 0 means -2 E B+, and hence xz - y z = ( x - y ) z = ( y - x ) ( - z ) E R+.By definition, this implies y z < xz, and hence y z 5 xz,which establishes the result. For part 5, first note that there is nothing to prove if x = y . Hence, we can assume that x < y . Suppose for a contradiction that x - l < y-' . Then by part 3 we have that 1 = x - l x < y - ' x , and hence x < y . 1 < y y - ' x = x,contradiction. Standard Proof Technique 1.11 When several possibilities must be considered in a proof, the proof usually continues with separate arguments for each possibility. The proof is complete when each separate argument has led to the desired conclusion. This 0 type of proof is also called a proof by case distinction.
1.2. Order Axioms
7
We conclude this section by introducing the absolute value function and some of its properties.
Definition 1.12 For x
E
R,we set Ix I = x;
i f x 1.0, and we call it the absolute -x; i f x < 0,
value of x . Theorem 1.13 summarizes the properties of the absolute value. The numbering is adjusted so that properties 1,2, and 3 correspond to the analogous properties for norms (see Definition 15.38). We will formulate many results in the jirst part of the text to be analogous or easily generalizable to more abstract settings, but we will usually do so without explicit forward references. In this fashion many abstract situations will be more familiar because of similarities to situations investigated in the jirst part.
Theorem 1.13 Properties of the absolute value. 0. For all x
E
R,we have Ix I > 0,
1. For all x
E
R,we have 1x1 = 0 i y x
2. F o r a l l x , y
ER,wehave
= 0,
lxyl = Ixllyl,
3. Triangular inequality. For all x,y E R,we have Ix 4. Reverse triangular inequality. For all x , y
E
+ y I 5 lx I + I y 1.
R,we have 1 Ix I - I y I 1
I Ix - y I.
Proof. For part 0, let x E R.In case x > 0, by Definition 1.12 we have /x1 = x > 0. In case x < 0, we have x @ R+ and by part 2 of Axiom 1.6 we conclude -x > 0. Because in this case Ix I = -x > 0, part 0 follows. Throughout the text, the two implications of a biconditional “ A iff B” will be referred to as “+,”denoting “if A, then B ” and “+,”denoting “if B , then A.” For part 1, note that the direction “+=” is trivial, because (01= 0. For the direction “jlet , x” E R be so that /xI = 0 and suppose for a contradiction that x 0. If x > 0, then 0 < x = 1x1 = 0, a contradiction. (Note that the previous sentence is a shortproof by contradiction that is part of a longer proof by contradiction.) Therefore x < 0. But then 0 < -x = 1x1 = 0, a contradiction. Hence, x must be equal to 0. For part 2 , let x , y E R. If x 2 0 and y 1. 0, then by part 3 of Theorem 1.10 xy 1. 0, and hence lxyl = x y = I x / / y l . If x 2 0 and y < 0, then by part 4 of Theorem 1.10 we infer xy 5 0. Hence, (xyl = -xy = x ( - y ) = J x J J y The J . case x < 0 and y 3 0 is similar and the reader will produce it in Exercise 1- 11a. Finally, if x < 0 and y < 0, then by part 4 of Theorem 1.10 we obtain x y > 0. Hence, /xyl = xy = (-l)(-1)xy = ( - x ) ( - y ) = ixllyl. To prove the triangular inequality, first note that for all x E IR we have that x I /x1. This is clear for x 1. 0 and for x < 0 we simply note x < 0 < -x = 1x1.Moreover, (see Exercise 1-llb) for all x E R we have -x I1x1. Now let x,y E R. If the inequality x y 2 0 holds, then by part 2 of Theorem 1.10 at least one of x. 4’ is greater than or equal to 0. (Otherwise x < 0 and y < 0 would imply x y < 0.) Hence,bypart2ofTheoreml.lOIx+yI = x + y l I x l + y ~I x I + I y I . I f x + y ( 0 ,
+
+
+
1. The Real Numbers
8
then at least one of x and y is less than 0. Hence, by part 2 of Theorem 1.10 we obtain Ix y l = -(x y ) = --x (-y) < I -XI (-Y) i I --XI I - Y I = 1x1 IYI. Finally, for the reverse triangular inequality, let x, y E R.Without loss of generality (see Standard Proof Technique 1.14) assume that ( X I 3 IyI. (The proof for the case 1x1 < lyl is left as Exercise 1-llc.) Then 1x1 = Ix - y yl i Ix - yI lyl, which w implies 11x1 - 1y11 = 1x1 - IYI i Ix - Y I .
+
+
+
+
+
+
+
+
Standard Proof Technique 1.14 If the proofs for the cases in a case distinction are very similar, it is customary to assume without loss of generality that one of these similar cases is true. This is not a loss of generality, because it is assumed that what is presented enables the reader to fill in the proof(s) for the other case(s). In this text, the omitted part is sometimes included as an explicit exercise for the reader. 0
Exercises 1- 10. Finishing the proof of Theorem 1.10 (a) Prove part 1 of Theorem 1.10. (b) Prove part 2 of Theorem 1.10. 1-1 1. Finishing the proof of Theorem 1.13.
(a) L e t x , y ~ W . P r o v e t h a t i f x > O a n d y ~ O , t h e n I x y l = I x l l y l . (b) Prove that for all x E R we have --x 5 1x1.
1
(c) Prove that if 1x1 < Iyl, then 11x1 - ( y / 5 Ix 1-12. Let I , J G
R be intervals. Prove that I n J
= {x E
-
y/.
W :x
E I and x E J ] is again an interval
1-13. Let a < b and letx, y E [ u , b].Prove that In - yI 5 b - a 1-14. Prove that none of the fields from Exercise 1-9c can satisfy Axiom 1.6 by showing that for these fields part 2 of Axiom 1.6 fails for n = 1. Note. This result shows that Axiom 1.6 distinguishes R from the finite fields of Exercise 1-9c.
1.3 Lowest Upper and Greatest Lower Bounds A structure that has the properties outlined in Axioms 1.1 and 1.6 is also called a linearly ordered field. The rational numbers satisfy these properties just as well as the real numbers. Thus we are not done with our characterization of R.The final axiom for the real numbers addresses upper and lower bounds of sets.
Definition 1.15 Let A be a subset ofR. E R is called an upper bound of A iff u 2 a f o r all a E A. has an upper bound, it is also called bounded above.
1. The number u
2. The number I E R is called a lower bound of A i f f 1 5 a f o r all a a lower bound, it is also called bounded below. A subset A
E
If A
A. If A has
R that is bounded above and bounded below is also called bounded.
9
1.3. Lowest Upper and Greatest Lower Bounds
Among all upper bounds of a set, the smallest one (if it exists) plays a special role. Similarly, the greatest lower bound plays a special role if it exists.
Definition 1.16 Let A C R. 1. The number s E R is called lowest upper bound of A or supremum of A, denoted sup(A), iffs is an upper bound of A and f o r all upper bounds u of A we have that s 5 u. 2. The number i E R is called greatest lower bound of A or infimum of A, denoted inf(A), iff i is a lower bound of A and f o r all lower bounds 1 of A we have that 1 5 i . Formally, it is not guaranteed that suprema and infima are unique, but the next result shows that this is indeed the case. Note that the statement of Proposition 1.17 follows the standard pattern for a uniqueness statement.
Proposition 1.17 Suprema are unique. That is, ifthe set A and s , t E R both are suprema of A, then s = t.
R is bounded above
Proof. Let A G Iw and s , t E R be as indicated. Then s is an upper bound of A and, because t is a supremum of A, we infer s 2 t . Similarly, t is an upper bound of A and, because s is a supremum of A, we infer t 2 s. This implies s = t . Standard Proof Technique 1.18 (Also compare with Standard Proof Technique 1.14.) When, as in the proof of Proposition 1.17, two parts of a proof are very similar, it is common to only prove one part and state that the other part is similar. Throughout the text, the reader will become familiar with this idea through exercises that require the construction of proofs that are similar to proofs given in the narrative. The proof that infima are unique is similar (see Exercise 1-15). Because suprema and infima are unique if they exist, we speak of the supremum and the infimum. The final axiom for the real numbers now states that suprema and infima exist under mild hypotheses.
Axiom 1.19 Completeness Axiom. Every nonempty subset S of R that has an upper bound has a lowest upper bound. Although the Completeness Axiom formally only guarantees that nonempty subsets
of
R that are bounded above have suprema, existence of infima is a consequence.
Proposition 1.20 Let S 5 R be nonempty and bounded below. Then S has a greatest lower bound. Proof. Let L := {x E R : x is a lower bound of S}. Then L f 0. Let s E S. Then for all 1 E L we have that 1 Is. Because S f: 0 this means that L is bounded above. Because L f: 0, by the Completeness Axiom, L has a supremum sup(L). Every s E S is an upper bound of L , which means that s 2 sup(L) and so sup(L) is a lower bound of S . By definition of suprema, sup(L) is greater than or equal to all elements of L ,
10
1. The Real Numbers
that is, it is greater than or equal to all lower bounds of S. By definition of infima, this means that sup(L) = inf(S). rn We will see that suprema and infima are valuable tools in analysis on the real line. The next result shows that in any set with a supremum we can find numbers that are arbitrarily close to the supremum. This fact is important, because analysis ultimately is about objects “getting close to each other.”
Proposition 1.21 Let S c R be a nonempty subset of R that is bounded above and let s := sup(S). Thenfor every E > 0 there is an element x E S so that s - x < E . Proof. Suppose for a contradiction that there is an E > 0 so that for all x E S we have that s - x 1 E . Then for all x E S we would obtain s - E 1 x, that is, s - E would be an upper bound of S. But s - E < s contradicts the fact that s is the lowest upper bound of S. rn Although the supremum and infimum of a set need not be elements of the set, we have different names for them in case they are in the set.
Definition 1.22 Let A be a subset of R. 1.
If A is bounded above and sup(A)
E A, then the supremum of A is also called
the maximum of A, denoted max(A).
2. If A is bounded below and inf(A) minimum of A, denoted min(A).
E
A, then the injmum of A is also called the
Although the distinctions between suprema and maxima and between infima and minima are small, the notions are distinct. For example, the open interval (0, 1) has a supremum (1) and an infimum (0), but it has neither a maximum, nor a minimum.
Exercises
1-15. Let A g W be bounded below and l e t s , f E W both be infima of A. Prove that s = t . 1-16. Approaching infima. State and prove a version of Proposition 1.21 that applies to infima. Is the proof significantly different from that of Proposition 1.21? 1-17. Let S g W be bounded above. Prove that s E W is the supremum of S iff s is an upper bound of S and for all E > 0 there is an x E S so that Is - x / < E . 1-18. Suprema and infima vs. containment of sets. (a) Let A. B C W be bounded above. Prove that A (b) Let A , B g W be bounded below. Prove that A
1-19. Let A g
5 B implies sup(A) 5 sup(B). g B implies inf(A) ? inf(B).
W be bounded above. Prove that inf(x E R : - x
E
A] = - sup(A).
1.4. Natural Numbers, Integers, and Rational Numbers
11
1.4 Natural Numbers, Integers, and Rational Numbers Although Axioms 1.1, 1.6 and 1.19 uniquely describe the real numbers, they do not mention familiar subsets, such as natural numbers, integers, and rational numbers. This is because these sets can be constructed from the axioms as subsets of the real numbers. We start with the natural numbers, which are the unique subset with properties as stated in Theorem 1.23. While their existence is easy to establish, the uniqueness of the natural numbers can only be proved in Theorem 1.28 after some more machinery has been developed.
Theorem 1.23 There is a subset N
G R,called the natural numbers, so that
1. 1 E N .
2. For each n E N the number n
+ 1 is also in N.
3. Principle of Induction. If S s have n 1 E S,then S = N.
+
N is such that 1 E
S and f o r each n E S we also
Proof. Call a subset A G R a successor set iff 1 E A and for all a E A we also have a 1 E A . Successor sets exist, because, for example, R itself is a successor set. Let N be the set of all elements of R that are in all successor sets. Because 1 is an element of every successor set, we infer 1 E N. Moreover, if n E N, then n is in every successor set, which means n 1 is in every successor set, and hence n 1 E N. Finally, any subset S C N as given in the Principle of Induction is a successor set. Because the elements of N are contained in all successor sets, we conclude that N G S , and hence N = S. 1
+
+
+
Of course, we will denote the natural numbers by their usual names 1, 2, 3, . . . As algebraic objects, natural numbers are suited for addition and multiplication (see Proposition 1.24), but they are not so well suited for subtraction (see Proposition 1.25). Although all results until Theorem 1.28 are stated for N,they hold “for every subset of R that satisfies the properties in Theorem 1.23.” The reader should keep this in mind and double check, because we will need it in the proof of Theorem 1.28. To avoid awkward formulations, the results up to Theorem 1.28 are formulated for N,however.
Proposition 1.24 The natural numbers are closed under addition and multiplication. That is, i f m , n E N,then m n and mn are in N also.
+
Proof. The key to this result is the Principle of Induction. Let m E W be arbitrary and let S, := { n E N : m+n E N}.Then m E N implies m+ 1 E N,and hence 1 E S,. Moreover, if n E S,, then m n E N,and hence m ( n 1) = ( m n ) 1 E N, which means that n 1 E .S, By the Principle of Induction we conclude that S, = N. Because m E N was arbitrary, this means that for any m , n E N we have m n E W. 1 The proof for products is similar and left to the reader as Exercise 1-20.
+
+
+ +
+ + +
Readers familiar with induction recognize the part “1 E S,” of the preceding proof as the base step of an induction and the part “n E S, jn 1 E S”, as the induction step. In this section, we use the “induction on sets” as done in the preceding proof. The more commonly known Principle of Induction is introduced in Theorem 1.39.
+
12
1. The Real Numbers
Proposition 1.25 Let m , n
E
N be such that m
> n. Then m - n E
N.
Proof. We first show that if m E N,then m - 1 E N or m - 1 = 0. To do this, let A := { m E N : m - 1 E N o r m - 1 = 0 ) . Then 1 E A a n d i f m E A , then ( m 1) - 1 = m E A C N,which means m 1 E A . Hence, A = N by the Principle of Induction. Now let S:= { n E N:(Vm E N : m > n implies m - n E N)}. If n = 1 and m E W satisfies m > 1, then m - 1 > 0 and so by the above m - 1 E N,which means 1 E S. Let n E S. If m > n 1, then m - 1 > n , and hence m - ( n 1) = (m - 1) - n E N, which means n 1 E S. By the Principle of Induction we conclude that S = N,and hence for all m , n E N we have proved that m > n implies m - n E N.
+
+
+
+
+
Proposition 1.26 shows that the natural numbers are positive and the smallest difference between any two of them is 1.
Proposition 1.26 For all n E N,the inequality n 2 1 holds and there is no m that the inequalities n < m < n 1 hold.
+
E
N so
Proof. The proof that all natural numbers are greater than or equal to 1 is left to Exercise 1-21. Now suppose for a contradiction that there is an n E N and an rn E N so that n < m < n 1. Thenm - n E N a n d m - n < 1, acontradiction.
+
The Well-ordering Theorem turns out to be equivalent to the Principle of Induction (see Exercise 1-22).
Theorem 1.27 Well-ordering Theorem. Every nonempty subset of N has a smallest element. Proof. Suppose for a contradiction that B 5 N is not empty and does not have a smallest element. Let S := { n E N : (Vm E N : m I n implies m $ B ) } . By Proposition 1.26, 1 is less than or equal to all elements of N,so 1 # B, and hence 1 E S. Now let n E S. Then all m E N with m 5 n are not in B. But then n 1 E B would by Proposition 1.26 imply that n 1 is the smallest element of B . Hence, n 1 # B and we conclude n 1 E S.By the Principle of Induction, S = N and consequently B = 0, a contradiction.
+
+
+
+
Now we are finally ready to show that the natural numbers are unique.
Theorem 1.28 The natural numbers N are the unique subset of R that satisfies the properties in Theorem 1.23. Proof. Examination of the proofs of all results since Theorem 1.23 reveals that any set S E Iw that satisfies the properties in Theorem 1.23 must also have the properties given in these results. It may feel tedious to go back and verify the above statement. Howevel; mathematical presentations more often than not will ask a reader to use a modification of a known proof toprove a result (also see Standard Proof Technique 1.14). When this occurs, the
13
1.4. Natural Numbers, Integers, and Rational Numbers
reader is expected to verifL that the result(s)can indeed be proved with similar methods as were used for earlier results. Now suppose for a contradiction that there is a set S # N with properties as in Theorem 1.23. Then S is a successor set, so M S. Let B := S \N = {s E S : s # N). Then B # 0, and hence by the Well-ordering Theorem, which is valid for S, B has a smallest element b. Because 1 E N we infer b 1 , and hence by Proposition 1.25, which is valid for S, we have b - 1 E S. But then b - 1 @ N,because this would imply b = ( b - 1) 1 E N.Hence, b - 1 E B , which is a contradiction to the fact that b is the smallest element of B .
+
Once we have constructed the natural numbers, the next number system to consider are the integers.
Definition 1.29 The set Z := { m E R : m E of integers.
N or m
= 0 or
- rn
E
N)is called the set
We leave several proofs of natural properties of the integers to the reader.
Proposition 1.30 The integers are closed under addition, subtraction and multiplication. Moreovel; for any two integers k , 1 with k > 1 we have that k - 1 >_ 1, every nonempty set A 5 Z that is bounded below has a minimum, and every nonempty set A C Zthat is bounded above has a maximum. Proof. To prove that Z is closed under addition, let m , n E Z. In case both are natural numbers or in case one of them is zero, there is nothing to prove. Moreover, in case -m, -n E N we have rn n = -((-m) ( - n ) ) , which is in Z, because (-m) ( - n ) E N. Now consider the case m E N and -n E N. If m = -n, we obtain m n = 0 E Z. If m > -n, then by Proposition 1.25 we conclude that m n = m - (-n) E N 5 Z. Finally, if m < -n again by Proposition 1.25 we conclude that -(m n ) = (-n) - m E N,which means by definition of Z that m n E Z.The case -rn E M and n E N is treated similarly (see Exercise 1-23a). Closedness under subtraction and multiplication as well as the claim about differences are left to Exercises 1-23b-1-23d. Now let A E Z be nonempty and bounded below. Then, because A C R, it has an infimum a . By the version of Proposition 1.21 for infima, there is an integer rn E A with m - a < 1. Because the absolute value of the difference between any two distinct integers is at least 1, rn is the only integer in [a,a 1 ) . Hence, m is below all elements of A that are not in [a,a 1). Because m is the only element of A in [ a ,a l), m must be the minimum of A. The proof of the corresponding result for nonempty subsets A 5 Z that are bounded above is left to Exercise 1-23e.
+
+ +
+
+
+
+
+
+
+
A key property of the natural numbers is that any real number is exceeded by a natural number. To prove this, we need the usual fractions, which are easily introduced. 1 R \ { 0 )we set - := a-l and call it the reciprocal of a. n -. b 1 For b E W and a E W \ ( 0 )we set - := b . - = ba-' and call it a fraction. a U
Definition 1.31 For all a
E
1. The Real Numbers
14 1 1 Because - + - = 2-' 2 2 following.
+ 2-'
= (1
+ 1) . 2 - '
Theorem 1.32 For every x E R, there is an n
E
= 2 .2-' = 1 we can now prove the
N so that n 2 x.
Proof. For a contradiction, suppose that x is such that for all n E N we have that n < x. Then B := { y E R : (Vn E N : n < y ) ) is not empty. Moreover, B is bounded below by all n E N. By the Completeness Axiom, B has an infimum, call it b. Then 1 1 1 b - - # B, which means there is an n E N with n 2 b - -. But then n 1 2 b 2 2 2 is a lower bound of B, a contradiction to b = inf(B).
+
+
Because N C Z and because subsets of Z that are bounded below have a minimum, we infer that for every real number x there is a unique smallest integer that is greater than or equal to x. Similarly there is a unique largest integer that is less than or equal to x. These numbers are useful when we need integers instead of real numbers, so we define the following.
Definition 1.33 For every x E R, let [XI be the smallest integer greater than or equal to x. Moreovel; let 1x1 be the largest integer less than or equal to x. Asfunctions from IR to Z, r.1 is called the ceiling function and 1.1 is called the floor function. The last subset of R that we introduce is the set of rational numbers. Rational numbers are naturally defined as fractions.
ca
1
Definition 1.34 The set Q := - : n E Z, d E N is called the set of rational numbers. The set R \ Q := {x E R : x # Q]is called the set of irrational numbers. Proposition 1.35 The rational numbers are closed under addition, subtraction and 4 multiplication. Moreovel; i f q , r E Q and r 0, then - E Q. r
+
m
n
Proof. Let m , n E Z, let c , d E N and consider the rational numbers - and - . C d Then Q is closed under addition because m n = mc-' + nd-' = mdd-'c-' + ncc-'d-' - + c d
mn For multiplication, note that - - = mc-'nd-' cd der is left to Exercise 1-24.
= mnc-ld-'
=
mn The remaincd
-.
Rational numbers can be found between any two real numbers and Exercise 1-45 will establish a similar result for irrational numbers.
Theorem 1.36 Let a , b thata < q < b.
E
IR with a < b. Then there is a rational number q
E
Q such
15
1.4. Natural Numbers, Integers, and Rational Numbers
1 Proof. By Theorem 1.32, there is an n E N so that 0 < -< n. By part b-a 1 5 of Theorem 1.10, we obtain - < b - a. Now let u := min n 1 u 1 Then - - - 2 b - a > -, which means n n n n 1+1 u '+'
similarly let I := max m E
Z: - 5 a
I
,
We conclude with a simple looking result that is actually at the heart of a standard proof technique (see Standard Proof Technique 2.7). Exercise 1-25 extends Theorem 1.37 to inequalities.
Theorem 1.37 Let x
E
E
R.r f x 3 0 and for all E
> 0 we have x 5
E,
then x = 0.
Proof. Let x be as indicated and suppose for a contradiction that x > 0. Then x . x . 1 := - is positive and x 5 E = - implies 1 5 -, a contradiction. 2 2 2
Exercises 1-20. Prove that if m. n E N, then mn E N. Hint. Same idea as the first part of the proof of Proposition 1.24 with sets S,
:= [n E
N : mn
-m E
N and n
E
N}.
1-21. Prove that if n E N,then n 2 1. Hint. Use S := ( n E N : n ? 1). 1-22. Use the Well-ordering Theorem to prove the Principle of Induction. 1-23. Finish the proof of Proposition 1.30 by proving the following. Finish the proof that then m n E Z.
+
Z is closed under addition. That is, prove that if
Prove that Zis closed under subtraction. That is, prove that m
-n E
Prove that 2.is closed under multiplication. That is, prove that mn Prove that for any two integers m , n with m > n we have m Hint. Find a contradiction to Proposition 1.26.
-n
E
Z for all m ,n
E
Z.
Z for all m ,n
E
Z.
? 1.
Prove that every nonempty set A 2 Z that is bounded above has a maximum. 1-24. Finish the proof of Proposition 1.35. That is.
(a) Prove that Q is closed under subtraction. (b) Prove that if q , r
E
Q and r f 0, then
Hint. First show that for n 1-25. Prove that if a , b
E
E
4
- E
r
Z\ ( 0 )and d
R are such that for all E
0. E
N we have that
> 0 we have a 5 b
+
E,
(;)-I
n then a 5 b.
1-26. Prove that for every real number x there is an integer n so that n 5 x. 1-27. Prove that for any real numbers x, E > 0 there is an n E N so that Hint. Theorem 1.32. 1 1 1 1-28. Prove that - - - = 1. 3 3 3
+ +
X -
n
=d
< E.
E
N.
I. The Real Numbers
16
1-29. A rational number r is called a dyadic rational number iff there are p E Z and n E N so that P r = -. Dyadic rational numbers are useful in analysis because they can provide a sequence of 2” “grids” such that each new grid contains the old one (see part 1-29a below), the whole set is the union of the “grids” (see part 1-29b) and between any two real numbers there is a dyadic rational number (see part 1-29c). Let D be the set of dyadic rational numbers and for each n E W let Dn :=
K
-
:p E
Z].
(a) Prove that for all n E N we have Dn c Dn+l.
uDn m
(b) Let
:=
{ x E W : (3n E N : x
co
E Dn)
] and prove that D
=
u D, n=l
n=l
(c) Prove that for any x , y E
R with x
< y there is a dyadic rational number d so that x < d < y.
1-30. In this exercise, we will prove that the real numbers are the (up to isomorphism) unique linearly ordered complete field. That is, we will prove that every mathematical object that satisfies Axioms 1.1, 1.6, and 1.19 is in a certain sense (defined below) “the same as R.” First notice that, similar to the proof of Theorem 1.28, all results proved so far hold for any object that satisjfies Axioms 1.1, 1.6, and 1.19 (because the results are derivEd fzom t h e y axioms). That is, every set W that satisfies Axioms 1.1, 1.6, and 1.19 contains subsets N, Z, and Q that have the properties that we have proved up to now for the natural numbers, the integers and the rational numbers. (a) Prove that for all x E
W we have that x
= sup(r E
Q : r 5 x).
0
(b) Now let Jk be a set that satisfies Axioms 1.1, 1.6, and 1.19 and let 6, 2, and be subsets of R that have the properties that we have proved up to now for the natural numbers, the integers and the rational numbers, including Exercise 1-30a. i. Define a function f : Q + @ as follows. For n E W, let f ( 1 ) := i and once f ( n ) is defined let f ( n 1) := f ( n ) + l . Also let f ( - n ) := l i t . For n E Z and d E W let
+
Q the above definition is not self-contradictory by d proving that it assigns exactly one value to each x E Q.Then prove that f ( x ) E for each x E Q and that f preserves the order, that is, if x < z , then f ( x ) 4 f(z). f
(E)
:= $.Prove that for all x E d
0
ii. For x E R let f ( x ) := sup { f ( r ) : r E Q and r 5 x ] . Prove that for all x E R the above definition is not self-contradictory by proving it assigns exactly one value to each x E
w.
(Formally this says that f is well-defined.) iii. Prove that the above function does not map any two points to the same image by proving that for all x, y E R the inequality x f y implies that f ( x ) # f ( y ) . (Formally, this says that the function f is one-to-one or injective.) iv. Prove that the above function “reaches” every element of Jk by proving that for all 1 E there is an x E R so that f ( x ) = 1. (Formally, this says that the function f is onto or surjective.)
fi
v. Prove that the above function is consistent with the algebraic operations by proving that for all x , y E W we have that f ( x y ) = f ( x ) T f ( y ) and f ( x y ) = f ( x ) - f ( y ) . (Formally, this says that f is a field isomorphism.)
+
vi. Prove that the above function is consistent with the order relation by proving that for all x , y E W we have that x 5 y implies that f ( x ) i f ( y ) . (Formally, this says that f is an order isomorphism.) The above steps show that the points and operations in W and in Jk can b_e identified with each other in such a way th5t it does not matter if we are working in R or in R.Thus for all intents and purposes, W and W are “the same.” This is the essence of saying that the real numbers are up to isomorphism the unique linearly ordered, complete field.
17
1.5. Recursion, Induction, Summations, and Products
1.5 Recursion, Induction, Summations, and Products A recursive definition defines an entity X, that depends on a natural number n first for n = 1 and then it defines Xn+l in terms of X , . By the Principle of Induction the set S = { n E N : X, is defined ] is equal to N,which means that a recursive definition defines the entity X, for all natural numbers n . In this fashion, the sum of finitely many numbers can be defined. I
Definition 1.38 For each j E N let
aj E
R. Define the sum c
a j := a1
and for
j=l
n+l
n
E
Ndefine the sum x a j := a n + l
n
-m
j=l
j=l
+ Caj.Form E N U {0},set x
j=l
a j := 0. The
parameter j is also called the summation index. In particular, note that a sum whose index starts at 1 and ends at a number smaller than 1 is always zero. It is also called an empty sum. Summations that start at numbers other than 1 are defined similarly (Exercise 1-31). By their nature, recursive definitions are closely linked to induction. Unlike what is stated in Theorem 1.23, induction normally is used to prove statements about natural numbers. This is possible, because a proof that a statement is true for all natural numbers is the same as a proof that a certain set is equal to N.
Theorem 1.39 Principle of Induction. Let P(n) be a Statement about the natural number n. I f P ( 1) is true and iffor all n E W truth of P(n) implies truth of P(n 1), then P (n) holds for all natural numbers.
+
Proof. Let P be as indicated and consider the set S := { n E W : P(n) is true }. Then 1 E S. For every n E S the statement P(n) is true, hence P(n 1) is true, which means n 1 E S . By Theorem 1.23 we conclude S = N and thus P(n) is true for all n E N. w
+
+
Standard Proof Technique 1.40 In the form of Theorem 1.39, induction is a standard proof technique. It involves a two-step process. In the first step, called the base step, P ( l ) is proved. Then, in the induction step, P(n) is used to prove P(n 1). In this context, P(n) is also called the induction hypothesis. All proofs in this section rely on induction. Moreover, Exercise 1-32 exhibits another way to carry out an induction (sometimes called strong induction). 0
+
n
Example 1.41 For all n
E
W, the summation formula j=l 7
1 j = -n(n 2
+ 1) holds.
7
1
Proof. The statement is P(n) = j=1
cj 1
Basestep. Weprove P(n) forn = 1.
j=l
1 = 1 = -1(1 2
+ l ) , so P ( l ) holds.
18
1. The Real Numbers
c n
Induction step. Under the induction hypothesis
1 j = -n(n
2
j=1 nil
I
j = 7 (n
+ 1) we must prove
+ l)((n + 1) + 1). A standard step in induction for recursively defined
L,
j=1
quantities is to split off the last term. This is done in the first step here.
cj
n
n+l
=
(n+l)+Cj
j=l
j=1
1 1 1 + 1) + -n(n + 1) = -2(n + 1) + -n(n + 1) 2 2 2
=
(n
=
1 -(n 2
+ 2)(n + 1). 0
Further examples of similar inductions can be found in Exercise 1-33.
Similar to sums we can define products. Although products occur less frequently than sums, they are useful to define powers.
n 1
Definition 1.42 For each j E N,let aJ
n
E
R. DeJine the product
n+l
f o r all n E
N dejine the product
n
aJ := an+l .
j=1
fi
n
aJ := a1 and
5= 1
a ] . For all m E
N U [O}, set
J=1
a] := 1. The parameter j is also called the product index.
J=1
Products that start at numbers other than 1 are defined similarly (Exercise 1-31). Products that end at an index that is smaller than the starting index are set to 1 and are also called empty products.
n n
Definition 1.43 For all a
E
R,and all n
E NU{O}, we dejine
the nth power an :=
a.
J=1
Aside from integer powers of numbers, we want to work with rational powers. To define rational powers, we need nth roots of nonnegative real numbers. To formally prove their existence, we need the Binomial Theorem. As a start we need binomial coefficients and one of their key properties.
n n
Definition 1.44 For all n
E
N U {O}, we dejine n ! :=
J=1
of n. For all n , k E n! := k ! ( n - k ) ! ’
(el
j and call it the factorial
N U (0) with k 5 n, we dejine the binomial coefficient as
1.5. Recursion, Induction, Summations, and Products
Theorem 1.45 The equation
19
(k:l)+(F)=(nll)
holdsforalln,k E
N
with k 5 n.
Proof. This result can be proved by direct computation.
=
(krl)+(;)
-
n! n! ( k - l ) ! ( n- ( k - l))! + k ! ( n - k ) ! n!k n !( n 1- k ) - n ! ( k +n 1- k ) k !( n 1- k ) ! k !( n 1-k ) ! k !( n 1 -k ) ! (n l)! k!(n 1 - k ) ! '
+ + +
+
+ +
+
+
Now we are ready to prove the Binomial Theorem.
Theorem 1.46 The Binomial Theorem. For all real numbers a , b we have ( a
+ b)"
n
, ,
E
R,and all n
E
N,
= k=O
Proof. Throughout the proof we will freely use the properties of sums proved in Exercise 1-34. The proof is by induction on n , with P ( n ) being the statement about (a b)". Base step. For n = 1, note that
+
which proves the base step. Induction step. Assuming that the result holds for n , we must prove it for n First note that it follows easily from Definition 1.43 that for all x E R and all m we have x . x m = xrn+'.
=
(a
=
(a
+ b)(a + b)" + b) f:(I) akbnPk k=O
=
2( i )
ak+lbn-k
k=O
+
2 (:> k=O
akbn+l-k
+ 1. E
N
20
1. The Real Numbers
=
(
;),.+lbn+l-intl,
+(
[( ) + (
: ) U O ~ ~ + I - ~ + ~
j=1
j-1
7)]dbn+l-j
With the Binomial Theorem, we can prove that nth roots exist. The proof of Theorem 1.47 is the first proof in this text in which we have to choose a number to make another number smaller than a given bound. That is, this is our first proof with a distinct analytical flavor. Theorem 1.47 Let n E N.For every nonnegative real number a, there exists a unique nonnegative real number r such that r n = a.
Proof. We first prove the existence of r . Let R := {x E R : x _> 0 and x n 5 a } . Then 0 E R and R is bounded above by max{l, a } . Let r := sup(R). To show that rn = a , we will show that rn # a and r" # a . First, suppose for a contradiction that rn < a . Then there is a 6 > 0 so that r n 6 < a. By Theorem 1.32 (or Exercise
+
1-27), for each k E (1, . . . , n } we can find an mk
E
N so that
(i)
m := max{ml, . . . , m,,). Then by the Binomial Theorem we conclude
k=l
mk
i
r n - k a n
s
.; Let
i
1.5.Recursion, Induction, Summations, and Products
The above shows that r
21
1 +E R , contradicting the fact that r = sup(R). Hence, m
rn # a . The proof that rn 3 a is similar and left to the reader as Exercise 1-36. For uniqueness, suppose for a contradiction that there is another b >_ 0 with b” = a . Then b < r or b > r . But if b > r. then with S := b - r we obtain a = b“ = ( r
+ S)n
= rn
+
rnPkSk> a , a contradiction. Hence, b < r . But k=l
then with 6 := r - b we have a = r n = ( b
+ 6 ) n =bn +
bnPkSk> a , a con. .
tradiction. Therefore r is unique. We conclude by defining rational powers of nonnegative numbers and by proving some of their properties.
Definition 1.48 Let n E N and let a E IR be nonnegative. The unique nonnegative real For n = 2 the root number r such that rn = a is called the nth root of a, denoted G. is called the square root, denoted &. Existence of nth roots is another property that distinguishes R from Q.Although Theorem 1.36 indicates that there are “many” rational numbers, the rational number system has some shortcomings when it comes to powers.
Proposition 1.49 There is no rational number r such that r 2 = 2. Proof. We first prove by induction as stated in Exercise 1-32 (strong induction) that if n2 = 2z for some z E N,then n = 22’ for some z’ E N. The base step for n = 1 is vacuously true. That is, because the hypothesis l 2 = 2 2 leads to the contradiction 1 = l 2 = 22 = z z > 1, the hypothesis is never true, which means that the implication is automatically true (see Definition A.2 in Appendix A). For the induction step, first note that the result is trivial for n = 2, because 2 = 2.1. Now assume that n > 2 and the statement has been proved for all natural numbers less than n. Then 22 = n2 = ( n - 2 2)2 = ( n - 2)2 4(n - 2) 4 implies that ( n - 2)2 = 22 for some Z E N.By induction hypothesis, we conclude that n - 2 = 2: for some Z E N,and hence n = 22 2 = 22’ for some z’ E N. This proves that if n2 = 2z for some z E N,then n = 22’ for z’ = 2 1 E N.
+
+ +
+
+
+
Now suppose for a contradiction that there are n E Z and d E N so that (;)2 =2 and such that there is no k E N \ 11) such that n = nk . k and d = dk . k . But by the above n2 = 2d2 implies n = n2 . 2 . Consequently, 2d2 = (n2 . 2)2, that is, d 2 = n; ’ 2 , which implies d = d2 . 2, a contradiction. We conclude from Theorem 1.47 and Proposition 1.49 that f i is irrational. For odd natural numbers (that is, natural numbers of the form n = 2k l), it is possible to define the nth root of a negative number a < 0 as @ := For the most part, powers are considered for nonnegative numbers, though.
+
-m.
1. The Real Numbers
22
Definition 1.50 For all real numbers a 2 0, all m E N U {0), n E N and all q E with q > 0 we dejine 1. a ; := ~.
That is, the
(i)th power
Q
of a is the nth root of a.
2. a: := ( a " ) : .
3.
a-q
:= (a">-
1
1 = -fora Uq
+ 0.
Theorem 1.51 For all positive numbers a and b and all rational numbers x and y , the following power laws hold: aXaY
=(p+Y
(q=
(ab)x = a X b X
aXY
Proof. We first prove (ab)x = axbx. For exponents n E N,this is an easy induction. The base step n = 1 is trivial and the induction step from n to n 1 is (ab)n+' = ab(ab)n = abanbn = aanbb" = an+'bn+'. n For rational exponents - with n , d E N,we have that (ab)a = (ab)n and d ( ~ z b ;= ) ~ = anbn = (ab)n. Note that in both equalities we used the dejinition of fractional powers, not the power law that we are currently proving. Because all numbers involved are positive and dth roots are unique, we conclude that n n n (ab)a = a a b a . For x = 0, the equality (ab)x = a X b Xis trivial. Finally, for all positive x E Q we note (ab)-xaxbx = (ab)-x(ab)x = 1. Therefore (ab)-x is the multiplicative inverse of u X b x ,that is, ( d P= xa-xb-x. Thus (ab)x = aXbXfor all a , b > 0 and all x E Q. To prove that uX+Y = a X a y we proceed similarly. For exponents m , n E N,the proof for arbitrary m is an induction on n. The base step amal = ama = urn+' follows straight from the definition of powers with natural exponents. For the induction step from n to n 1, note that aman+' = amana = a m f n a = a m + ( n f 1 )which , proves the result for exponents m , n E N. For positive rational exponents x and y , note that there are m , n , d E N so that m n x = - and y = -. Then, using the equality we already proved, we obtain d d
+
(
(b;)d
'!>"
+
ni
n
aXaJ = a a a a =
(.")f
1
= (a"an)a = (am+n
)
f
-
=a ~ + ~ .
The equality is trivial if one of x and y is zero. In case both exponents are negative, note that for all positive x,y E Q we have aX+Ja-Xa-J= ~ ~ a J ' a - ~ a -= - Y1, which means a-Xa-Y = ~ - ~ f ( - y ) as was to be
23
1.5. Recursion, Induction, Summations, and Products
proved. This leaves the case in which one exponent is positive and the other is negative. Let x,y E Q be positive and consider ax-)'. If the inequality 1x1 > IyI holds we have ax-4' a-v a --x = &Ca-x = 1, which means that ax-y = axa-y. If 1x1 < IyI we have aL'-xaxa-.i = a y a - y - 1, which means that ay-' = aya-'. If 1x1 = IyI the claim is trivial. Thus a x + y = d a y for all a > 0 and all x,y E Q. We leave the remaining three equalities as Exercise 1-37. Power laws for a 5 0 and b 5 0 (as applicable) can be proved similarly. To conclude, note that the results presented in this chapter guarantee that the real numbers have the properties we expect them to have. We will therefore use the usual notation (fractions, etc.) and laws of algebra throughout this text without further qualms about the need to justify that we are indeed allowed to do so.
Exercises 1-31. Let k . rn
E
m
Z and for each j
E
Zlet a j
E
n in
W.Define the sum
a , and the product j=k
a,.
j=k
1-32. Let P ( n ) be a statement about the natural numbern. Prove that if P(1) is true and if for all n E N\[1) truth of P ( l ) , . . . , P ( n - 1) implies truth of P ( n ) , then P ( n ) holds for all natural numbers. This type of induction is sometimes called strong induction. Hint Consider S := n
{
W : ( V k < IZ : P ( k ) holds
E
) }.
1-33. Prove each of the following by induction.
n
(b)
1
j 3 = ;n2(n
+
(d) Bernoulli's inequality. Prove that for all real numbers x that (1 +x)" z 1 + n x .
1-34. Properties of sums and products. Let c E
W and for all j
(a) Prove that for all n E N we have x ( a j
+bj) =
n
# 0 and n 2 2 we have
-1, x
N let a j and bj be real numbers aj
j=1
j=1
(b) Prove that for all n E
E
P
+
x
bj
j=l
n
N we have E ( c a j ) = c
nj J=1
j=1
n
1 =n
( c ) Prove that for all n E N we have j=1
(d) Prove that for all n
E
N we have
fi
(a, . b j ) =
j=1
1-35. Reindexing sums. Lets
E
Z, n
E
N and for j
[fi [fi aj )
j=1
E
Zlet a,
E
.
bj
)
j=1
5s. Prove that
s+n
n+I
1=s
i=l
C a, =
nkfs-l.
1. The Real Numbers
24 1-36. Finish the proof of Theorem 1.47 by showing that r" # a . Hint. Suppose r" > a and prove that then for some E > 0 and all 6
E
(0, E ) we have r" - S @ R
1-37. Finish the proof of Theorem 1.51. That is, let a and b be positive real numbers, let x , y E prove each of the following. (c)
Q and
(ax)' = ax'
1-38. Let 0 5 a < b and let q z 0 be rational. Prove that aq < bq. 1-39. Let a,x
E
(0, 00) and let x be a rational number.
(a) Prove that if a > 1 and x > 1, then ax > a .
P and compare a p and a4 Hint. Let p , q E N be so that x = 4 (b) Prove thatifa < 1 andx < 1, thenax > a . (c) Prove that if a > 1 and x < 1, then ax < a . (d) Prove that if a < 1 and x > 1, then a x < a . 1-40. Letn
E
IV,Provethat
(a>
= 1 andthat
(:>
=I
1-41. Prove that there is no rational number r such that r 2 = 3. 1-42. Prove that for any n real numbers X I ,
. . . , xn the inequality
a+b 1-43. Prove that for all a,b t 0 the inequality 4% 5 -holds. 2 1-44.
(a) Prove that for all a , b
E
W we have d;;2+b2 5 / a /+ /bl.
(b) Prove that for any a l , . . . , an
E
W we have
1-45. Let a , b E W with a < b. Prove that there is an irrational number x E W \ Q such that a < x Hint. Use that f i is irrational and Exercise 1-27 and mimic the proof of Theorem 1.36.
4
b.
Chapter 2
Sequences of Real Numbers Convergence is the fundamental concept of analysis. It explores what happens when two quantities get close to each other, or when a quantity grows beyond all bounds. This chapter exhibits these ideas for sequences, with special emphasis on standard proof techniques.
2.1 Limits We start by defining sequences.
Definition 2.1 A sequence of real numbers is a function f from the natural numbers to the real numbers. To emphasize their discrete nature, we denote sequences as with the understanding that a, = f ( n )for all n E N. Similar to sums and products, a sequence can actually start at any integer k (Exercise 2-1). The limit of a sequence should be the place where the sequence “stabilizes” for large n. Definition 2.2 encodes this property by demanding that for every given tolerance E , there is a threshold N so that once the running index n has gone past the threshold N ,the sequence can only deviate from the limit by less than the tolerance E .
Definition 2.2 Let {a,}=, be a sequence of real numbers. Then L E R is called limit of [ u , } g , ifffor all E > 0 there is an N E N so that for all n 2 N we have that la, - L J < E (see Figure 2). A sequence that has a limit will be called convergent, a sequence that does not have a limit will be called divergent. Fintrely man) n mapped below L - E w
1
w
All
n
IV i u “la1l.’ of w) mapped to ( L - 8 , L
-
w
L
w
-&
L
+
Finitely many n mapped aboie L
+
F)
w
\
w
L
f
a
w
r
E *
E
Figure 2: Visualization of convergence to L . For every E > 0 a “tail” of the sequence is in ( L - E , L E ) .
+
25
2. Sequences of Real Numbers
26
Remark 2.3 It can be helpful to restate the definition using quantifiers (see Definition A.3 in Appendix A). L
E
R is a limit of {a,}Eliff Vs
> 0 : 3N E
N : V n 2 N : la,
- LI <
E.
Formal statements with quantifiers enforce the rule that a variable must be defined or quantified before it can be used. We will usually enforce the same rule in natural language. Although the prose becomes a bit rigid this way, in nested quantifications like the definition of limits it is clearer to say “. . .for all x we have P ( x ) . . . ” than to 0 say “. . . P ( x ) holds for all x . . . ” Note that we did not speak of the limit of a sequence in Definition 2.2. This is because we have not proved yet that every convergent sequence has only one limit. The next theorem shows that this indeed the case.
Proposition 2.4 Limits of sequences of real numbers are unique. That is, if(^,}^^ is then L = M . a sequence of real numbers and both L and M are limits of
{a,}zl
Proof. Let be a sequence and let L and M be limits of {a,}El. We need to prove that L = M . By Theorem 1.37 we can do so by showing that for all E > 0 the inequality I L - MI < E holds. Let E > 0 be arbitrary but fixed. Then there is an N1 E N such that for all n 2 N1 & we have that la, - L / < -. There also is an N2 E N such that for all n 2 N2 we 2 & have la, - M I < -. Let N := max{Nl, N2]. Because N 2 N1, for all n 2 N we 2 & & have (a, - L / < -. Because N 2 N2, for all n 2 N we have la, - MI < -. Then 2 2 by adding and subtracting U N and applying the triangular inequality we obtain (with n =N )
Because for arbitrary E > 0 we have the inequality lL - MI < E , by Theorem 1.37 we conclude that I L - MI = 0, and hence L = M . We will ultimately read and produce proofs that are much more complicated than the proof of Proposition 2.4. Therefore it is only appropriate to analyze how such proofs can be conceived. The standard proof techniques discussed later in this section reveal that certain details are indeed standard techniques which simply need to be internalized and used at the right time. Other than that, the novice usually is impressed & . by the sometimes “strange” choices for E . The reason why - is chosen in the proof of 2 Proposition 2.4 is that the proof is actually created backwards. Consider the following. To show that L = M we first note that because the a, are eventually close to L and close to M , we can put an a, with a sufficiently large index between L and M . After applying the triangular inequality
27
2.1. Limits
the resulting differences should be small. By Theorem 1.37, if for all E > 0 we can make the difference IL. - MI less than or equal to E then IL, - MI = 0, that is, L = M . So we want to make the sum of the differences lL - a, I and (a, - MI smaller than E . & It is most natural to make each of the two terms smaller than - to obtain 2
jL - MI = IL - UN
+
- MI 5 IL. - a N 1
+ IUN - MI
E
+
E
- = E. 2 2 This argument provides the final few lines of the proof. Note that up to here we have not chosen any N1,N2, or N . However, now that we have the “meat” of the argument, it is easy to create the “header.” E To make la, - L J and la, - MI smaller than -, we use that by the definition 2 of convergence there are N1 and N2 as mentioned in the proof and choose N to be their maximum so that both required inequalities hold for indices beyond N . Note that even though the “header” is the first thing we encounter, it is often the last thing that materializes as a proof is created. So, to set up an analysis proof it is standard practice to start by working with inequalities. Once the inequalities work, we create a “header” with the appropriate choices for E , n , and so on. UN
< -
Standard proof techniques in analysis. Certain steps occur so frequently in analysis proofs that they should become second nature. In this fashion, communication becomes more effective because memory is less strained to recall details of proofs. This is a cognitive technique commonly known as “chunking” of data. By internalizing certain standard “chunks,” larger amounts of data can be recalled, because we only need to recall which chunks are involved rather than all details. Unlike the standard proof techniques listed so far, from here on most standard proof techniques will be specific to analysis. The standard techniques used in the proof of Proposition 2.4 are listed below. Standard Proof Technique 2.5 It is common practice to rearrange terms and to add and subtract the same term to obtain more manageable expressions. When working with absolute values, this is often done in conjunction with the triangular inequality. In Proposition 2.4 this is the step IL - MI = J L - U N
+
UN
- MI 5 IL - UN I
+
jUN
- MI.
+
Such a step is usually abbreviated as ( L - MI 5 IL - U N I la^ - MI. In other computations, we will see that it can also be useful to multiply and divide by the same nonzero term. 0
Standard Proof Technique 2.6 If finitely many numbers N1, . . . , Nk E N are such that for all n 2 Ni a certain inequality holds, we can choose N := max{N1, . . . , Nk}. Then for all n 2 N all these inequalities hold. We usually claim directly that such an O N exists, skipping the intermediate Ni . Standard Proof Technique 2.7 To prove that two quantities are equal, we can prove that for any E > 0 the absolute value of the difference is less than E . This is usually done without explicit reference to Theorem 1.37. To prove an inequality a 5 b we often prove that for all E > 0 the inequality a 5 b + E holds (see Exercise 1-25). 0
28
2. Sequences of Real Numbers
Standard Proof Technique 2.8 In many analysis proofs we prove results about a universally quantified variable, often denoted E . To prove such results, we pick one such E that is “arbitrary, but fixed,” throughout the proof. It must be fixed throughout the proof so we can uniquely define quantities that depend on it, and it must be arbitrary so that we really prove something about all variables in the scope of the universal quantification. Once the result is proved, we can conclude that “Because E was arbitrary we have proved . . . for all such E.” This final statement reiterates that, even though we made specific choices for the E in the proof, we can indeed make these choices for all E , which proves the universally quantified statement. Because this approach is so common, the bracketing statements about the variable E are usually left out or abbreviated. 0 Standard Proof Technique 2.9 Finally, statements like “We need to prove L = M,” that are put at the start to remind the reader what we will prove are often left out. Similarly, statements put at the end to reiterate what we have proved are often left out, too. 0 To phase in these techniques, we will first carry them out explicitly and give a reference to the above list. Then we will omit the explicit step, but still refer to the appropriate entry in the above list. Eventually a proof for something like Proposition 2.4 will condense to the following.
“Expert Proof” of Proposition 2.4. Let E > 0. Then there is an N E W such that & & for all n 2 N the inequalities la, - L ( < - and (a, - MI < - hold. Therefore 2 2 IL - MI 5 ( L - a N I
+ la^ - MI
E
<-
2
+ E2 = E , -
which implies L = M . The “expert proof’ contains all relevant information needed to communicate the idea. It does not list the details of standard techniques, which are assumed to be known. In this fashion the reader may have to fill in more “steps between the lines,” but as long as these steps can be considered manageable, communication becomes more effective.
Definition 2.10 Because the limit is unique, we speak of the limit of a sequence. The exists and is equal to the notation lim a, = L will indicate that the limit of n+w number L. The next result shows that the first finitely many values of a sequence do not affect its convergence behavior.
Theorem 2.11 Let [ u , ) and ~ ~ ( b n } g lbe sequences of real numbers and let K E W be so that for all n 2 K we have a, = b,. Then { a , } z l converges ifs (bn}E1 converges and in this case the equality lim a, = lim b, holds. n-cc n+w Proof. Left to Exercise 2-2.
29
2.1. Limits
We conclude this section with an explicit proof that a limit is a certain number. This exercise is good for learning how to choose more exotic values for N . Remember that while the proof is presented in linear fashion, it is actually created by working with the inequalities that we see at the end. In fact, N was found by solving the inequality 17 < E , which arises naturally from the computation at the end. 4n 10
+
3n-1 3 -Example 2.12 Prove that lim -n-+m2n+5 2’ Let E > 0 be arbitrary but fixed, and let N for all n 3 N the following holds.
E
N be such that N
12-10 >
4
3n-1 3 Because E > 0 was arbitrary, this proves that lim -- n-tm2nf5 2
. Then
0
Exercises 2-1. Let k E Zand for each n E Zwith n 2 k let a, E B.Define the sequence it means for L E W to be its limit.
and define what
2-2. Prove Theorem 2.1 1. 2-3. Write out the argument that produced the choice for N in Example 2.12. 2-4. Let {an]F=, and (bn)r==l be sequences such that for all n if { u , ) ~ =converges ~ then so does
2-5. Let {a,)?=,
{bn)r-’, and n+cu lim a ,
E
N we have la,
= Iim bn.
- bnl
<
I
-. Prove that n
n+cu
be a sequence. Prove that lim a, = L iff lim la, - L / = 0. n+ce
n+m
2-6. Prove each of the following.
(d)
l i m m - & = O
n+m
2-7. The usefulness of quantifiers. ” the (a) State the negation of the statement “ L is the limit of the sequence ( a n ) ~ = ; Cby= lfinding negation of Definition 2.2.
(b) State the negation of the statement “L is the limit of the sequence { u , ] ~ = ~by” finding the negation of the quantifier version of Definition 2.2 in Remark 2.3. Hint. Appendix A.2.
30
2. Sequences of Real Numbers
Limit Laws
2.2
Example 2.12 shows how to prove that a limit is a certain number. However, to write such a proof we must know the limit in advance. It would be nice to have a computational tool that provides the limit. It would be even nicer if the computation somehow could justify (prove) that the result of the computation really is the limit. This can be done with the limit laws discussed in this section. Because we are investigating the theory, it will be important to understand how the limit laws come into being. Applying the limit laws is fairly simple (see Exercise 2-8). To use the limit laws, we must know the limits of certain standard sequences.
1 Theorem 2.13 lim - = 0. n+m
y2
Proof. Exercise 2-9. The limit laws state that limits can be moved into the familiar algebraic operations. The proof consists of several similar arguments. The argument for sums is spelled out in great detail. The argument for differences is almost verbatim the same argument as for sums and it is left to the reader in Exercise 2-10. The argument for products is proved closer to the fashion of the “expert proof’ on page 28, but with remarks interspersed. For the argument for quotients, we only present the final inequalities, leaving the remaining parts to the reader in Exercise 2- 1 1.
Theorem 2.14 Limit lawsfor sequences. Let sequences. Then the following hold. 1. The sum {a,
{a,}Eland [ b , } E l be two convergent
+ b n } z i converges and n-+m lim a, + b, = lim a, + lim b,. n+co n+co
2. The difference [a,
-
00 bn},,l converges and lirn a, -b, = lirn a, - lirn b,
n-m
n+oc
3. The product {anbn}Ei converges and lim a, . b, = lim a,
. n+cc lim b,.
n+m
n+m
4. Ifall b,
# 0 and lim b, n+m
+ 0, then the quotient
n+m
converges and
an limn--tcoan lim - = b, b, ’
n-30
Proof. Throughout this proof let L := lim a, and let M := lim b,.
+
n+m
n+co
To show for part 1 that the sum {a, b,,}E1converges to the limit L + M , we must show that for all E > 0 there is an N E N so that for all n 2 N we have the inequality (a, bn) - ( L M)I < E . Let E > 0 be arbitrary but fixed (see Standard Proof Technique 2.8). Because E lirn a, = L there is an NL E N so that for all n 2 NL we have la, - L / < -. Simn-m 2 ilarly, because lirn b, = M there is an NM E N so that for all n 2 NM we have
1
+
+
n+w
Ib,,
&
-. Let N
:= max(NL, NM}. Then (compare with Standard Proof Tech2 & & nique 2.6) for all n 3 N the inequalities la, - LI < - and Ib, - MI < - hold. By 2 2 -
MI <
2.2. Limit Laws
31
rearranging the terms and applying the triangular inequality (compare with Standard Proof Technique 2.5) we obtain the following for all n 2 N .
1 (an + b,)
- (L
+ M )I
=
Ian - L
<
& & - f - = & .
+ bn
-
MI I la, - L I
+ Ibn - MI
2 2 We have proved that for arbitrary E > 0 we can find an N E N so that for all n 2 N we have that (a, b,) - ( L M ) < E (see Standard Proof Technique 2.9). Therefore by the definition of the limit, lim a, b, = L M . n-+m Part 2 is Exercise 2-10. To show how an abbreviated proof still contains the standard statement of the de$nition of the limit, the key parts of the definition are in the proof of part 3. We also intersperse comments in italics on how certain choices arise. For part 3, let The choices for E look a bit strange. They are made so that the inequalities later on work out and they can be motivated by reading the final estimates. Recall the discussion after the proof of Proposition 2.4. The first typed part of this proof were the $nu1 inequalities. Because lim a, = L there is an N L E N so that for all n 2 N L we have that
I +
+ I
+
+
n+-cc
E
. (We cannot divide by ]MI alone, because it could be zero.) 2(lMl 1) Similarly, because lim b, = M there is an N M E N so that for all n 2 N M we have la, - L1 <
+
E
n+cc
lim a, = L there is a K1 E N so that for all 16, - MI < 2(ILI 1) . Finally, because n-cc n 2 K1 we have la, - LI < 1. Consequently, for all n 2 K1 we obtain by the reverse triangularinequality la,] - ILI 5 llanl - I L / / 5 la, -LI < 1 andthus lanl < ILI 1. Let . Then (compare with Standard Proof Technique 2.6)
+
+
the inequalities la,l
< /LI
Ian - LI <
E
2(IMI
+ 1 hold.
&
+ 1)’
Ibn - MI <
2(ILI
+ 1 ) and
To “connect” the anbn and the L M , we add and subtract the term a,M. This allows us to apply the triangular inequaliv (compare with Standard Proof Technique 2.5) to split the original difference lanbn - LMI into two summands. Each of these summands contains one factor that can be made small. For all n 3 N , we obtain the following.
a,MI
<
lanbn
=
lanllbn -MI
-
+ la,M - LMI + Ian -LIIMI
2. Sequences of Real Numbers
32
which proves part 3 (see Standard Proof Technique 2.9). For part 4, the final estimates, which are the hard part to figure out, are
la,M - LMI -
I
<
+ ILM - Lb,l
Ibn I IMI
+
LIIMI ILIlM - bnl Ib* I IMI la, - LllMl lLllM - bnl la,
-
+
IMI
E I M ~2 21LI &M2 -M 2 2(21LI 1) < &. 4 /MI
+-
+
To train the reader in generating the appropriate “header,” the complete proof of part 4 (including final estimates) is left to Exercise 2-1 1.
Standard Proof Technique 2.15 In an analysis proof, it can be necessary to divide by a nonnegative quantity lal. Usually the quotient will be multiplied by that same quantity later in an estimate and the goal is to cancel it (consider the proofs of parts 3 and 4 of Theorem 2.14). However, we cannot divide by zero. To avoid any undue distractions here, when defining certain quotients, we will usually add 1 to nonnegative
la’ < 1 still allows us to quantities in denominators. In this fashion, the fact that la1
+1
“cancel” the term in an estimate. This is more effective than to separately consider the 0 case that a quantity is equal to zero. The limit laws allow us to efficiently establish convergence and compute the limits for many complicated looking sequences. For instance, finding the limit of the sequence in Example 2.12 reduces to the following.
{ ;;; ;}Il.
Example 2.16 Find the limit of -
Using Theorem 2.13 and the limit laws we obtain the following: 3n - 1 3 n-1; lim - = lim n-+W 2n 5 n-ta2n+5
+
3-,
1
1
= lim -n+-m
2+5
3-lim,,,; 2+lim,,,;
1
3 - -
5
2’
3 Note that by the limit laws, the computation also proves that the limit is - . That is, 2 we will not need to perform the tedious argument given in Example 2.12.
2.2. Limit Laws
33
For limits of powers, we will consider integer powers here, leaving rational powers to Theorem 3.42 and real powers to Theorem 12.11.
Theorem 2.17 Let
{a,,}Elbe a convergent sequence. Thenfor all k E N the sequence
. Ifnone of the terms are zero and the limit is not equal to zero, the result holds for all k E Z. converges and lim a," = n i x
Proof. Fork E N the proof is an induction on k. The base step fork = 1 is trivial. For the induction step k -+ k 1, assume that for a given k 3 1 the sequence
+
oo
converges and that lirn a," = {a,"]n = l nioo converges with lirn a,"" = lim n+cc n+oo as follows.
(
=
. We must prove that . By part 3 of Theorem 2.14, we can argue
lim a, .a," = lim a,"+',
n+oo
n+cc
where each expression in the chain of equations is guaranteed to exist because the expression preceding it exists. This completes the induction step and the proof of the result for k E N. Now for k E Z,let all terms and the limit be nonzero. For k > 0, we just proved the result and for k = 0 the result is trivial. For k < 0. note that
where again each expression in the chain of equations is guaranteed to exist because the expression preceding it exists.
Standard Proof Technique 2.18 Many statements about limits implicitly assert two things. First, that the limit exists, and second, what the limit is. This means a proof will need to verify both these claims. To simplify the language and to shorten proofs, it is customary to dispense with an explicit existence proof for the limit. Instead, the computation of the limit can be used to establish the existence of the limit as in the proof of Theorem 2.17. If we start with an existing quantity and finish with the quantity in question, then the existence claim in known limit laws establishes the existence of all intermediate quantities from left to right. Exercise 2-23 shows that the order of progression is important. The limit of a sum, a diference, a product, or a quotient can 0 exist even when the sequences of the individual terms diverge.
34
2. Sequences of Real Numbers
Standard Proof Technique 2.19 An induction as in the first part of the proof of Theorem 2.17, which takes a result for an operation on two objects and turns it into a result for an operation with k E N objects, is often omitted by saying “a simple induction argument shows ” The proofs will all be similar. The base step is usually simple. For the induction step, split off the last element, apply the induction hypothesis to the first k elements, then apply the original result and thus conclude the proof. 0
...
Although it would feel natural to prove that Theorem 2.17 holds for real exponents, real exponents must be postponed until we define real exponentiation in Definition 12.8. Our main focus is on the abstract underpinnings and fundamentals of analysis, not on certain specifics. Hence, we postpone the introduction of transcendental functions until we have all the machinery to introduce them very efficiently. Their introduction will also make us more fully appreciate the power of the abstract tools we build. We could at least prove Theorem 2.17 for rational exponents, but at this stage the proof would be cumbersome. We will obtain the result easily in Theorem 3.42, once we have developed some more tools. We will use this approach throughout. Zfa result can be obtained easily late< then (unless an important technique needs to be introduced) we will postpone the result rather than set up an unnecessarily complicated argument. We conclude this section by investigating the relationship between limits and inequalities.
Theorem 2.20 Let {a,}Eland { b n } z lbe two convergent sequences of real numbers so that for all n E N the inequality a, 5 b, holds. Then lim a, 5 lirn b,. n+m
n+m
Proof. Let L := lim an and let M := lim b,. We will prove (see Standard Proof n+m n+m Technique 2.7) that for every E > 0 the inequality L - M < E holds, which proves that L - M 5 0, that is, L 5 M . Let E > 0. Then (see Standard Proof Technique 2.6) there is an N E W so that & & for all n 2 N we have la, - LI < - and Ib, - MI < -. But, using Standard Proof 2 2 Technique 2.5, this implies L-M
L-a,+b,-M+a,-b,
=
&
5 lL-a,I+lb,-MI+a,-b,
&
-+-+O=&. 2 2
<
We have proved that for all E > 0 we have L - M < proves that lirn a, = L 5 M = lim b,.
E,
which by Exercise 1-25
,--too
,--too
The next theorem is a refinement of the fact that limits preserve existing inequalities. If a sequence {b,}r=l is “trapped” between two convergent sequences with the same limit, then must converge to that limit also. The novelty is that we need not assume that { b , ] E l converges.
{&}El
Theorem 2.21 The Squeeze Theorem for sequences. Let { u , } ~ ={bn)n=l, ~ , cc { c , }cc ,=~ be sequences of real numbers so that for all n E N the inequalities a, 5 b, 5 c, hold and so that lim a, = lim c., Then { b n ) E lconverges and the limits are equal, that n+m n+cc is, lim b, = lim a, = lim cn. n-m
n-m
n+m
2.2. Limit Laws
35
Proof. Let L := lim a, = lim c,. We need to prove that for all E > 0 there is an n+co n+cc N so that for all n 2 N the inequality Ib, - LI < E holds. Let E > 0. Because {a,}zl and both converge to L , there are N, E N and Nc E N so that for all n 2 N, we have la, - LI < E and so that for all n 2 Nc we have / c , - Li < E . Let N := max{N,, N c } . Then for all n 2 N we obtain L - E < a, and c, < L E . Therefore, for all n 2 N we conclude L - E < a,* 5 b, 5 c, < L E, which means that Ib, - LI < E . We have proved that for all E > 0 there is an N E W so that for all n 2 N we have the inequality Ib, - LI < E . This means that {bn)zl converges to L .
N
E
(cn}zl
+
+
Standard Proof Technique 2.22 To prove that the limit of a sequence {brz]r=l is zero, 1 it is common to prove that lbnl is bounded by some sequence, such as a multiple of -, n that goes to zero itself. The Squeeze Theorem then guarantees that the limit of {b,}gl also is zero. In particular (compare with Exercise 2-5), we often prove that lirn a, = L by n+cc
proving that la, - L I is bounded by a sequence that goes to zero.
Exercises 2-8. Use the limit laws to compute the limit of the sequence.
2-9. Prove Theorem 2.13. Hint. Exercise 1-27 2-10. Prove part 2 of Theorem 2.14. 2-1 1. Prove part 4 of Theorem 2.14. Explain all choices for N and for upper or lower bounds. Hint. There are three inequalities that must hold for all n 2 N . 2-12. Prove that if
(an)g, converges, then lim n-tx
2-13. Prove that if a, 2 0 for all n
E
/ a n /=
lirn a,
n+m
1.
N and ( a , ] g l converges, then n lim -kw
=
4s..
Y"
2- 14. Let q > 0. Prove that lirn - = 0. n+3o n ! rz! 2-15. Conjecture the value of lim - and then prove your conjecture. n-cc n" 2-16. Let
{an]gl be a sequence of real numbers and let ( p n ) g l be a sequence of positive real numbers 1
so that lim ___ = 0. n-tm c g , l Pk c;=i Pkak
(a) Prove that if (an]r=O=l converges to a E R, then lim
n++m
(b) Give an example to show that the convergence of convergence of
{
=a
C;,lPk
"='
I, =, oii
Pkak Pk
need not imply the
2. Sequences of Real Numbers
36
Figure 3: An injective function maps all elements of the domain to distinct images, but some elements of the range may not have a preimage ( a ) . For a surjective function, every element of the range has a preimage, but some elements of the domain may be mapped to the same image ( b ) . A bijective function maps all elements of the domain to distinct images and each element of the range has a preimage (c). This is why the existence of a bijection between two sets indicates that the two sets are “of the same size’’ (see Definitions 2.25 and 7.1 1).
2.3 Cauchy Sequences A sequence so that for all E > 0 all elements with a sufficiently large index are within E of each other should converge. Indeed, this condition guarantees that the elements with large indices cluster ever more tightly. However, the number system may have a hole just where these elements cluster. For example, the sequence 1.4, 1.41. 1.414, 1.4142, . . . of successively better decimal approximations of h does not converge in Q because the value that the sequence approaches is not in Q (see Exercise 2-17). This section shows that this problem does not arise in the real numbers. Sequences for which elements with large indices cluster ever more tightly play an important role in analysis. They are called Cauchy sequences.
{a,}zl
Definition 2.23 Let be a sequence of real numbers. Then {an)rT1is called a Cauchy sequence i r f o r all E > 0 there is an N E N so that for all m , n 2 N we have that laTl- a, 1 < E . In the real numbers, convergence and being a Cauchy sequence are equivalent. Before we can prove this result, we need to define finite and infinite sets.
Definition 2.24 Let A , B be sets and let f : A -+ B be a function. Then f is called injective or one-to-one ifffor all x , y E A the inequality x # y implies f (x) # f (y). f is called surjective or onto ifSfor all b E B there is an a E A with f ( a ) = b. Finally, f is called bijective iff f is both injective and surjective. Figure 3 gives a visualization of injective, surjective, and bijective functions and some properties of injective and surjective functions are investigated in Exercises 2-1 8 and 2-19. Once we have bijective functions, we can define finite sets as “sets of size n,” where n E N U {O}.
2.3. Cauchy Sequences
37
Definition 2.25 A set F is called finite iff F is empty or there is an n E N and a bijective function f : { 1, . . . , n } -+ F. Sets that are notjnite are called infinite. For finite sets F # 5 we set 1 FJ:= n with n as above and we set 101 := 0. For injinite sets ZwesetlII:=co. Lemma 2.26 Let F be ajnite set and let I be an injinite set. Then I \ F is injinite.
Proof. In case F is empty, there is nothing to prove. In case F is not empty let n E N be so that there is a bijective function f : ( 1, . . . , n } -+ F . Suppose for a contradiction that I \ F is finite. Then there are a natural number m E N and a bijective function g : (1, . . . , m } + I \ F . Define the function h : (1, . . . , m n } + I by if j 5 n , Then it is easy to show that h is bijective (Exercise h ( j ) := f ( A ; g ( j - n ) ; if j > n. 2-20). But this means that I is finite, a contradiction.
+
[
Theorem 2.27 A sequence ( u , } ~of~real numbers converges iff it is a Cauchy sequence.
Proof. For the direction “jlet , L” := lim a,. We need to prove that for all E > 0 n-oo there is an N E N so that for all m , n 2 N we have \a, - a, I < E . & Let E > 0. Then there is an N E N so that for all n 2 N we have la, - LI < -. 2 Therefore for all m , n 2 N we obtain
We have proved that for all E > 0 there is an N E N so that for all m , n > N we have la, - a,J < E . Hence, {a,}r=l is a Cauchy sequence. Once more we have used Standard Proof Technique 2.5. It is so common, and we have used it often enough, that we will no longer explicitly refer back to it. “+:”Let ( u , ) be ~ ~a Cauchy sequence. We need to prove that there is an L E R so that for all E > 0 there is an N E N so that for all n 2 N we have la, - L1 < E . We first need to find a suitable number L. Because ( a , ) ~isl a Cauchy sequence, for E := 1 there is an N E N such that for all m , n > N we have la, - a, I < 1. In particular, for all n 3 N we obtain la, - a N 1 < 1, and hence a, < a~ 1. Therefore the set ( n E N : a, 5 a N 1) is infinite and thus {x E R : ( n E N : a, 5 x } is infinite } # 0. For all n 2 N we also have that a, > a~ - 1, so that for all x 5 a N - 1 the set ( n E N : a, 5 x} is finite. Therefore {x E R : ( n E N : a, 5 x} is infinite } is bounded below. This means L := inf (x E R : {n E N : a, 5 x} is infinite } exists by Proposition 1.20. The idea how to obtain L is visualized in Figure 4. To prove that L is the limit of we need to prove that for all E > 0 there is an N E N so that for all n 2 N we have la, - L / < E . Let E > 0. By definition of L , the set H- := FI E N : a, 5 L - - is finite 2 and the set H+ := n E N : a, 5 L - is infinite. Therefore, by Lemma 2.26 the 2 is infinite. Because relative complement H+ \ H.- = n E N : a, E
+
+
(a,}zl
1
[
+7
{
7
2. Sequences of Real Numbers
38 CUlOff polnr Infimtely man) n mapped to an) (-m. x ] filth x > L Finilely many n mapped to any (-m. XIwith x < L
-
Fmrel) many n mapped below a N - 1 1
w
w
-I
FmtzI) many n mapped abore a N
w
w ,
aN - 1
w
L
w
-
w
1
w
\
,
a
w
+1
- c
aN f 1
aN
Figure 4: Visualization of the construction of L in the proof of Theorem 2.27.
{U,},X~=~ is a Cauchy sequence, there is an N E N such that for all in, n 2 N we have & & la, - a,/ < -. Moreover, because n E N : a, E ( L - i,L + is infinite, there
2 is a k 3 N so that
lak -
LI 5
6
z]} 2
{
-. Therefore for all n 2 N we obtain 2
Because E was arbitrary we have proved that for all E > 0 there is an N E W so that for all n 1 N we have la, - LI < E . Hence, lim a, = L . fl+W
Standard Proof Technique 2.28 Application of the Completeness Axiom to get the infimum or supremum of the “right” set is a standard technique on the real line. We will also see this approach in the proofs of Theorems 2.37, 2.41, 3.34, and 8.4. In abstract spaces, this technique is not available and we usually substitute compactness (see Definition 16.57), a property which, for closed and bounded intervals on the real line, is a consequence of the Completeness Axiom (see Theorems 2.41 and 8.4). 0 Convergence of Cauchy sequences is a fundamental analytical property called “completeness,” which is introduced in Section 16.2. Another way of formulating Theorem 2.27 is to say that the real numbers are complete. Although Axiom 1.19 is already called the Completeness Axiom of the real numbers, this terminology makes sense, because Exercise 2-25 shows that Theorem 2.27 and Axiom 1.19 are equivalent. This means that either one of them could rightly be called the Completeness Axiom. There are many other equivalent formulations of the Completeness Axiom. We will encounter some more of them in Theorems 2.37, 2.41, and 8.4 as well as in Exercise 2-5Oc. Whenever we encounter one of these formulations, there will be an exercise similar to Exercise 2-25 to show that the new result is equivalent to one of the equivalent formulations of the Completeness Axiom that we already know. Aside from the fundamental importance of completeness, Theorem 2.27 has immediate value for showing if a sequence is divergent. By Definition 2.2 a sequence is divergent iff (using negations as stated in Appendix A.2) for every real number L there is an E > 0 such that for all N E W there is an n 3 N so that la, - LI 2 E . This is a four times nested quantification that would require us to show for every real number that it is not the limit of the sequence. Theorem 2.27 reduces proofs of divergence to showing the sequence is not a Cauchy sequence.
Example 2.29 The sequence { (-
diverges.
2.3. Cauchy Sequences
39
To prove that the sequence diverges, we need to prove that it is not a Cauchy sequence. This means (see Appendix A.2) we must find an E > 0 so that for all N E N there are m , n >_ N so that la, - a,\ > E . But with E := 1 for every N E N we have that i(-l)N - (-l)N+li = 2 > 1. Hence, { (- l),}Zlis not a Cauchy sequence and therefore it diverges.
0
Standard Proof Technique 2.30 In Example 2.29, we had to negate the statement “for all E > 0 there is an N E N such that for all m , n 2 N we have la, - a, 1 < E.” When negating such a complicated statement, it is helpful to write the statement in quantifiers and then negate it. In this fashion, the definition of a Cauchy sequence is VE > 0 : 3N E
N : V m , n 2 N : la, - a,(
< E,
and the negation becomes (see Appendix A.2) 3~ > 0 : V N E
N : 3m, n
>_
N : la,
- a,
I 2 E,
which is what was needed in Example 2.29. The schematic way in which quantified statements can be negated is very helpful, especially the first few times one negates a complicated statement. However, the quantifiers must not become a crutch. It is advisable to first try the negation verbally and then double check with quantifiers.
Standard Proof Technique 2.31 To prove that a sequence of real numbers converges, we often simply prove that it is a Cauchy sequence. To prove that a sequence of real numbers diverges, we prove that it is not a Cauchy sequence. 0
Exercises 2-17. Prove that number.
{
1
x. is a Cauchy sequence of rational numbers whose limit is not a rational
n=l
2- 18. Let A . B be sets and let f : A -+ B be a function. Prove that f is injective iff for all bl , b2 E B we have that f ( b 1 ) = f ( b 2 ) implies bl = b2. 2-19. Compositions of injective and surjective functions. Let A , B , C be sets and let f : B -+ C and g : A -+ B be functions. The composition of f and g is defined by f o g ( a ) := f ( g ( a ) ) for all a E A.
(a) Prove that if f and g are injective, then so is f o g . (b) Prove that i f f and g are surjective, then so is f o g 2-20. Prove that the function h in the proof of Lemma 2.26 is bijective. 2-2 1. State the definition of a convergent sequence using quantifiers 2-22. For each of the following sequences, prove that it converges or prove that it diverges
40
2. Sequences of Real Numbers
2-23. Existence of the limit on the left side of a limit law as in Theorem 2.14 does not imply the existence of the limits on the right side. (a) Use {a,]:,
:= { ( - l ) , } g 1
1
and [b,]El:= (-1)"'l
cr3
ln=l
to show that lirn a, n+oo
+ b,
can exist without either sequence being convergent. (b) Show that lirn a, - b, can exist without either sequence being convergent. n+m (c) Show that lirn a, . b, can exist without either sequence being convergent n+m
(d) Show that lim
?!
n+m b,
can exist without either sequence being convergent
2-24. Can a Cauchy sequence have two limits? Explain your answer. 2-25. Use the fact that Cauchy sequences converge in the real numbers and the axioms for R except for Axiom 1.19 toprove Axiom 1.19. Hint Let S G R be bounded above and not empty. Construct a Cauchy sequence so that for all x E S there is an N E N so that for all n 1 N the inequality an 2 x holds.
2.4 Bounded Sequences If the elements of a sequence cannot become arbitrarily large, we speak of a bounded sequence. Unlike being a Cauchy sequence, boundedness is not equivalent to convergence, but it still has some important consequences.
Definition 2.32 A sequence is called bounded above iff there is a number A E R such that for all n E N the inequality a, 5 A holds. In this case, A is also called an upper bound of the sequence. A sequence {a,}Klis called bounded below iff there is a B E R such that for all n E N the inequality a, 2 B holds. In this case, B is also called a lower bound of the sequence. Finally, a sequence is called bounded #it is bounded above and bounded below and we call it unbounded if not. Example 2.33 The sequence
is bounded, while the sequence
{n)El is not
-
U
bounded.
Proposition 2.34 Any convergent sequence of real numbers is bounded. Proof. Let { a n ) z lbe a convergent sequence and let L := lim a,. We need to n+cc prove that there is a number M E R so that for all n E N the inequality /a,/ 5 M holds. Let E > 0. Then there is an N E N so that for all n 2 N we have la, - L / < E . Let M : = m a x ( J L l + & , la11 . . . . , I a M - I I } . Thenforalln < Nwetnviallyhavela,I 5 M and for n L N we obtain lan] 5 la, - LI jLI < E ILI 5 M . We have proved that is bounded below by -M and above by M . w
+
+
In general, the converse of Proposition 2.34 is not true as the next example shows.
Example 2.35 The sequence { (-l)"}zl is bounded, but it does not converge (see Example 2.29). 0
2.4. Bounded Sequences
41
For monotone sequences, however, boundedness does imply convergence.
Definition 2.36 Let { a n } E i be a sequence. Then [a,}Elis called nondecreasing ifs for all n E N we have a, 5 an+l. It is called nonincreasing ifffor all n E N we have a, 2 a,+l. rf{a,}EI is either nonincreasing or nondecreasing, then it is called monotone. Moreovel; is called (strictly) increasing zxfor all n E N we have a, < a,+l and it is called (strictly) decreasing ifffor all n E N we have a, > a,+l. The sequence { n ] z lshows that nondecreasing sequences can grow beyond all bounds. But if this is not the case, a monotone sequence converges. The key to the proof is Standard Proof Technique 2.28.
Theorem 2.37 Monotone Sequence Theorem. If { u , , } ~is ~bounded and monotone, then [a,}Elconverges. More precisely 1.
v{a,]Elis bounded above and nondecreasing, then it converges.
2.
Zf{a,},X=l is bounded
below and nonincreasing, then it converges.
Proof. We only prove part 1. The proof of part 2 is similar (Exercise 2-30). To prove part 1, let be bounded above and nondecreasing. Then the set {a, : n E N] is bounded above, and hence by Axiom 1.19 it has a supremum L . To prove that L is the limit of the sequence, we must prove that for every E 0 there is an N E N so that for all n 2 N the inequality la, - LI < E holds. Let E > 0. By Proposition 1.21 there is an N E N with U N > L - E . But then for all n N we have L - E < U N 5 a, 5 L , and hence ~ U N- LI < E . We have proved that for every E > 0 there is an N E N so that for all n 2 N we have la, - LI < E . Hence, lim a, = L . H n+o3
Although boundedness does not imply convergence, it forces the sequence to “cluster” in some places. To make this idea more precise, we need the notion of a subsequence.
Definition 2.38 Let A , B , C be sets and let f : B -+ C and g : A -+ B be functions. The composition o f f and g is defined by f o g ( a ) := f ( g ( a ) )for all a E A. Definition 2.39 Let [a,]Elbe a sequence of real numbers and let { n k ] g l be a strictly increasing sequence of natural numbers. Then { a f l k } Eis 1 called a subsequence of [ u , } ~ Formally, ~ . a subsequence is the composition of the function that maps k to nk and the function that maps n to a,. Convergence is what happens when the indices get large. To obtain a notion that is useful to analyze convergence behavior, in the definition of a subsequence we had to specifically demand that the nk are strictly increasing. If we had allowed nk = n k + l , then by choosing [ n k } & to be a constant sequence, any sequence would have infinitely many convergent “subsequences.” This would be counterintuitive, because a sequence such as
{ k]
00
would have a “subsequence” that converges to 1 even though the n= 1
sequence itself converges to 0.
2. Sequences of Real Numbers
42
With the definition as it is, subsequences behave sensibly when the sequence converges.
{a,}zl
Proposition 2.40 Let be a convergent sequence of real numbers with limit L . Then every subsequence { a n k } z also l converges to L.
{a,}zl
Proof. Let be a convergent sequence of real numbers with limit L and let be a subsequence. We must prove that for all E > 0 there is a K E N so that for all k 2 K we have lank - Li < E . Let E > 0. Because nk < nk+l for all k E N,an easy induction shows that rZk 2 k for all k E N (Exercise 2-31). Because { a , } z l converges, there is an N E N so that for all n 2 N we have la, - LI < E . Therefore for all k 2 N we obtain nk 2 k 2 N , and hence lank - LI < E . We have proved that for all E > 0 there is a K E N so that for all k 2 K we have /ank- L / < E . Hence, { a n k } E 1converges to L , too. H
{U,,}~QS_~
The precise statement of the idea that a bounded sequence of real numbers must “cluster” somewhere is that a bounded sequence of real numbers must have a convergent subsequence. This is an important property of the real numbers, which is ultimately encoded in the notion of compactness (see Section 16.5). The proof of the Bolzano-Weierstrass Theorem utilizes Standard Proof Technique 2.28.
of
Theorem 2.41 Bolzano-Weierstrass Theorem. Any bounded sequence real numbers has a convergent subsequence.
Proof. Let be a bounded sequence and let b > 0 be such that for all n E N we have ]a,] < b. Then b is a real number so that { n E N : a, 5 b ] = N and {n E N : a, 5 -b} = 0. Therefore b is contained in the set of real numbers {x E R : { n E N : a, 5 x} is infinite }. Moreover, this set is bounded below by -b. Hence, the infimum L := inf { x E R : ( n E N : a, 5 x } is infinite } exists by Proposition 1.20. We will prove that L is the limit of a subsequence of {U,},X=~. To do this, we will employ the Standard Proof Technique 2.22 and find a subsequence { a n k } E lso that /ank- LI is bounded by a sequence that goes to zero. For each k E N the set
1
Hk+ := n
E
N : a, 5 L
+ -kl l is infinite and the set HC
is finite. Therefore the set Tk := n E N : a,%E ( L -
{
1
i ,L
1 + ‘1)
:= n
E
N : a,
7
5L -k
= Hk+ \ H L is ink finite for each k E N (see Lemma 2.26). Construct {nk}‘& inductively as follows. Because TI is infinite, it is not empty and we let n 1 E T I . Once rZk was chosen, let nk+l be any natural number in Tk+l that is greater than ilk. Such a natural number exists, because T k + l is infinite. Then {a,,}& is a subsequence of and for all 1 k E N we have /ank- LI 5 -, because nk E Tk. By the Squeeze Theorem this means k H lim la,, - L / = 0, and hence lim ank = L.
{a,]zl
k-m
k+w
43
2.4. Bounded Sequences bounded
+:(1 + (-1)")n t
convergent subsequence
bounded and monotone f:
convergent
ti
monotone
c
finite or infinite limit
(-1)"
pi: n
Figure 5: Implications between the various notions related to convergence that are introduced in this chapter. Implications are indicated with arrows and the examples near the arrows indicate that the opposite implication does not hold. The Bolzano-Weierstrass Theorem is a useful tool in single variable analysis. We will see examples of its use in the proofs of Theorem 3.44 and Lemma 5.19. For many sets of properties, it is instructive to explore which property implies which other properties and which properties are equivalent. Figure 5 summarizes the properties introduced in this chapter (including those from the next section) and how they are related to each other.
Exercises 2-26. For each of the sequences below determine if it is bounded, and then prove your claim.
2-27. For the given sequence (a, nk is as indicated.
]El,find an expression for the terms of the subsequence { unk)Elwhere
2-28. Use Proposition 2.40 to prove that each of the sequences below diverges.
2-29. Explain why we could have chosen E = 1 in the proof of Proposition 2.34. Then explain why we cannot choose E = 1 in a general convergence proof. 2-30. Prove part 2 of the Monotone Sequence Theorem. 2-31. Perform the induction mentioned in the proof of Proposition 2.40. First state exactly what it is that you prove, then execute the proof.
2. Sequences of Real Numbers
44
2-32 Sketch a visualization for the construction of L in the proof of the Bolzano-Weierstrass Theorem that is similar to Figure 4. 2-33 Let x E R and let (x,)?=~ be a sequence of real numbers that does not converge to x. Prove that there is an E > 0 and a subsequence { x n k } E lso that for all k E N we have xnk - x 1 E .
1
1
2-34 Let [x,)~=~be a sequence of real numbers that has no convergent subsequence. Prove that for each x E R there is an E~ > 0 so that { n E N : Ixn - x / < cx } is finite. be a sequence of real numbers such that every subsequence has a subse2-35 Let L E R and let quence that converges to L . Prove that ( x ~ } ? = ~converges. 2-36 A well-known convergent sequence. n
(a) Let a , b E Wand n
E
N.Prove that b n f l -an+'
= (b- a )
ajbn-j. j =O
(b) Let a , b
E
(c) Prove that
:>")I,
lFt with 0 5 a < b and let n E
{
(1
+
N.Prove that
b"+l - .n+l
b-a
< (n
+ 1)b".
is increasing and that it converges.
Hint To prove it is increasing, bring an+' to the right in part 2-36b and use a = 1
1 and b = 1 + - , To prove the sequence is bounded above, use a = 1 and b = 1 n
+ n +1 1 ~
1
+. 2n
2-31 Use the Monotone Sequence Theorem and the axioms for 1.19.
R except Axiom 1.19 to prove Axiom
2-38 Use the Bolzano-Weierstrass Theorem and the axioms for 1.19.
R except Axiom 1.19 to prove Axiom
2-39 Use the Bolzano-Weierstrass Theorem and the axioms for W except for Axiom 1.19 to prove directly (that is, without using Theorem 2.27 or its proof) that every Cauchy sequence of real numbers converges. 2-40 Use the Bolzano-Weierstrass Theorem and the axioms for Monotone Sequence Theorem. 2-4 1 Let
R except
for Axiom 1.19 to prove the
[U,),X=~ and {bn]E1 be bounded sequences.
+
(a) Prove that (a, b,]?=;O=l is bounded. Hint. Unlike for convergence, the bound need not be one number M . An upper bound of the form Ma M b would work just fine.
+
(b) Prove that ( a n b n ] ~ =isl bounded.
(c) Prove that if there is a S
0 so that b, > S for all n
E
N then
is bounded
2-42. Let x E [O, I]. Prove that the sequence defined recursively by a0 := 0 and the recurrence 1 relation a,+l := an - (x - a,' ) converges to Jr;. 2 Hint. Prove by induction that the sequence is bounded above by &.Then prove that it is nondecreasing.
+
2.5
Infinite Limits
Although unbounded sequences diverge, they can display some types of regular behavior, which are explored in this section.
2.5. Infinite Limits
45
Definition 2.42 Let {a,,}z1 be a sequence of real numbers. Then we say that the limit of ( a , } z l is infinity ifffor every M E R there is an N E N so that for all n 2 N the inequality a, >_ M holds. In this case, we write lirn a, = 00. A limit of negative n-+ x infinity is defined similarly (Exercise 2-43) and denoted lirn a, = -00. ,--too
Intuitively an infinite limit should mean that eventually the sequence gets close to infinity. Similar to Definition 2.2, this idea is encoded in Definition 2.42 by saying that for any given bound M , there is a threshold N so that once the running index n goes past the threshold N ,the sequence will not drop below the bound M any more. It is also said that a sequence with lirn a, = 00 grows beyond all bounds. ,--too
Example 2.43 L e t x > 1. Then lim x n = 00. n+cc
Clearly, for all n E N the inequality xn+' > x n holds. Suppose for a contradiction that { x " } z l does not go to infinity. Then (refer to Standard Proof Technique 2.30 as necessary) there is a B > 0 so that for all N E N there is an rn > N with x m 5 B . Let n E N. By the above, there is an rn 1. n so that X" 5 x m 5 B . Hence, the sequence { x n } z l is bounded above. Let M := sup { x n : IZ E N} . Now M M by Proposition 1.21 because - < M , there is an n E N so that x n > -. But then X
X
M xntl = x . x n > x- = M , a contradiction. Thus lim x n = 00. n-m
X
With some exceptions, discussed at the end of this section, the limit laws for infinite limits are similar to those for convergent sequences. be such that lim a, = co.
Theorem 2.44 Limit laws involving co. Let
n+oo
1. If {bn},X==l is a bounded sequence, then lirn a,, n-+m
+ b, =
00
and if all a, are
bn nonzero, then lim - = 0. n+W a,
2.
If nlirn +x
b, =
then lim a, + b , = m and lirn anbn = 03. ,-too
n+cc
3. I f c > 0 is a real number, then lim can = 03. n--tx
4. I f c
4
0 is a real number, then lirn can = -co. n+m
Proof. To prove part 1 let lirn a, = 00 and let (b,],X==,be bounded. Let B E R be n i x such that for all n E W the inequality Ib, I < B holds. First consider the sum. We need to prove that for all M E R there is an N E N so that for all n > N we have a, + b, 2 M . Let M E R. There is an N E N so that for all n 2 N we have a, 2 M + B . But then for all n 2 N we obtain a, b, > M B - B = M , and hence lirn a, b, = 00. n i x Now consider the quotient. We need to prove that for all E > 0 there is an N E W
+
+
la": I
so that for all n 2 N the inequality - <
+
E
holds.
2. Sequences of Real Numbers
46 Let
E
IBI R so that < M , which
> 0. By Theorem 1.32 there is an M E W
1Bl means - < M
E.
&
Now there is an N E W so that for all n >_ N we have a, 2 M . Thus < E , and hence lim
for all n >_ N we obtain
bn
- = 0.
n-+m a,
.. ,
To prove part 2 let lirn a, = lirn b, = 00. n-tcc n-00 For the sum, we need to prove that for all M E R there is an N E N so that for all n 3 N the inequality a, b, 2 M holds. Let M E R and note that there is an N E N so that for all n 2 N we have the M M inequalities a, 2 - and b, 2 - (see Standard Proof Technique 2.6). But then for all 2 2 M M n 2 N we have a, b, 1. - - = M . Hence, lirn a, b, = 00. 2 2 n+co For the product, we need to prove that for all M E R there is an N E N so that for all n 2 N the inequality anbn 2 M holds. Let M E R and note that there is an N E N so that for all n 2 N we have the inequalities a, 2 and b, 2 But then for all n 2 N the product exceeds M because anbn 2 IM I 2 M . Hence, lim a, b, = co. n+oo To prove part 3, let lim a, = co and let c > 0. We need to prove that for all n-00 M E R there is an N E N so that for all n 2 N the inequality can 2 M holds. M Let M E R.There is an N E W so that for all n 2 N we have a, 1. -. But then C M for all n 2 N we obtain can >_ c- = M . Hence, lim can = 00. C n+co Finally, to prove part 4 let lirn a, = 00 and let c < 0. We need to prove that for nim all M E R there is an N E N so that for all n 2 N the inequality can IM holds. M Let M E R.There is an N E N so that for all n 2 N we have a, 2 --. But then
+
+
+
+
m
m.
mJln/ll=
(
i)
for all n 2 N we obtain can I c --
Icl
= M . Hence, lirn
n+m
Can
= -m.
Infinite limits can also help indirectly to establish the existence of finite limits.
Example 2.45 Let x > 0. Then lim x f = 1. n+cc The result is trivial for x = 1. We first consider x > 1. Suppose for a contradiction that lirn xf f 1. For every n E n+cc
Thus if
I
N we have 1 < x h
=
(.;)
*
< xf .
00
xi
' ln=l
does not converge to 1, then (refer to Standard Proof Technique 2.30
as necessary) because xj > 1 for all n E N E N there is an n 2 N with x: > 1
+
N,there is an E.
E
Because (x;]
+
> 0 so that for every 00
n=l
is decreasing this
would mean that 2; > 1 E for all n E N,But then for all n E N we would have (1 + E ) , < (x:) = x, contradicting the fact that lim (1 E ) , = 00 (see Example 2.43). Thus for all x > 1 we have lirn n+m
I
xi
,-too
= 1.
+
2.5. Infinite Limits
47
The proof for 0 < x < I is deferred to Exercise 2-46. Just as for infinite limits, there are limit laws for limits that equal negative infinity. be such that lim a, = -cc.
Theorem 2.46 Limit laws involving -m. Let
n-cc
1. I f { b n } E l is a bounded sequence, then lim a, n-tcc
+ b, = -00
and
if all a,
are
bn nonzero, then lim - = 0. ,+aa,
2. I f lim b, = -.co, then lirn a, n+cc
,403
+ b, = --oo
and lirn anbn = 00. n+w
3. I f lim b, = cc,then lim anbn = -w. n-cc
,300
4. I f c > 0 is a real numbel; then lirn can = -cc. 11'00
5. I f c
40
is a real numbel; then lim can
00.
n+m
Proof. The proof of Theorem 2.46 is similar to that of Theorem 2.44. It is thus left to the reader as Exercise 2-44. The addition of two sequences such that one has limit 00 and the other has limit -m is absent from Theorems 2.44 and 2.46. This is because by Exercise 2-48b the sum need not converge. Exercise 2-48c shows that even if there is a limit, it is not the same number in all cases. The situation is similar for the product of a sequence with infinite limit and a sequence with limit zero (see Exercises 2-48d and 2-48e), as well as for the quotient of two sequences with infinite limits (see Exercises 2-48f and 2-488). These types of limits are called indeterminate forms and they are discussed in more detail in Section 12.3.
Exercises 2-43. State the definition of a sequence whose limit is negative infinity. 2-44. Prove Theorem 2.46. That is, prove each of the following.
+
is a bounded sequence, then lim a, bn = -oa and if all n+x b an are nonzero, then lim -!! = 0. n + w a, (b) If lim a, = lim bn = -w, then lirn a, b, = --oo and lim a,b, = oc. (a) If,5mxa,
n+x
= -m and [bn):=,
n+x
n e w
+
n+x
(c) If lim a, = -oa and lim b, = oa,then lim a,b, n+x
n+x
n+x
= -oa
(d) If c > 0 is a real number and lirn a, = -m, then lim can = --oo n+m
n+m
(e) If c < 0 is a real number and lirn a,, = - w , then lim can = 00. n+x
n+x
2-45. Let 0 < x < 1 (a) Prove that lim x n = 0 by using the limit laws n+m
(b) Prove that lirn x n = 0 by mimicking the proof in Example 2.43 n+m
2. Sequences of Real Numbers
48 1
1. Prove that lirn x i = 1
2-46. Let 0 < x 2-47. Prove that
n+x
I,"=,
{
is decreasing and that the limit is 1
( + :In 5
Hint. Use Exercise 2-36c to show that 1 (n
+ 1) &I
n
-
n for all large enough n and derive the inequality
from this. For the limit L , use the result from Example 2.45 to show that L = L2.
i
2-48. A first encounter with indeterminate forms. (a) For
[ u , ) ~:=~( n ] g l
and
( b n ] g l :=
lim b, = --o= and that the sequence (a,
n-x
[ - n2 ),"=,,
+ b,}r=l
prove
(n]E1 a n d ( b , ] z l := ( - n ) z l , p r o v e t h a t n+oo lirn and that the sequence (a, + b n ) r =l converges.
(b) F o r ( a n ) z l := (c) Let c E
W.Find sequences
+ b,
lim a,
n-x
lim an = x ,
that
,-roo
diverges.
[~,)r=~
an = x , n-oo lim bn = -00
and [ b n ) r z lso that lirn a, = 00, lirn bn = -m and n i w
n-a3
= c.
(d) Find sequences ( a n ) z l and [ b n J z 1so that lirn a, = x , lim bn = 0 and ( ~ n b n ] E i n+oo
n-x
diverges. (e) Let c E
W. Find sequences (an},X=l and ( b n ] ; P 1so that n+m lim
a, = 03,
lirn b, = 0 and
n+x
lim anbn = c.
n-oo
(0Find
3cI
and (b,)r=l so that lim a, = lirn b, = m and
sequences
n-oo
n+x
di-
verges. (g) Let c E lim
n-m
W. Find sequences
2 = c. b,
2-49. Prove that if (a,)?=l and (b,]?=l an N
E
are sequences so that lirn a, = x and there are an
N so that bn > E for all n 2 N , then n-oo lim
n-oo
E
> 0 and
anbn = 00.
2-50. A characterization of divergent sequences
(a,]zl
I,"=,
be an unbounded sequence. Prove that there is a subsequence (ank (a) Let lim ank = 00 or lim ank = -a.
so that
k+co
k-x
be a bounded divergent sequence. Prove that there are two convergent subse(b) Let quences (a!, such that lim a!, and lim ank exist, hut are not equal. and
{ank}gl
m+x
k+x
(c) Let be a sequence. Prove that [ u , ] ~ = ~diverges if and only if there is a subsequence {ank},& such that lim ank = x or lim ank = --x, or there are two subse-
}r=l
quences (aim
and
k-m
{ L Z , , ~ } :such ~
k+oo
that lirn al, and lim ank exist, but are not equal. m+m
k+x
(d) Use the characterization of divergent sequences in part 2-5Oc and the axioms for R except for Axiom 1.19 to prove the Bolzano-Weierstrass Theorem. 2-51. Prove Cauchy's Limit Theorem. That is, let (b,]?=, be a strictly increasing sequence of positive numbers that goes to infinity and let [a,]?=, be a sequence. Prove that if the sequence an converges to c, then lim - = c. n - x b, Hint Exercise 2-16 with p , := b,, - b,-l and another appropriate sequence
Chapter 3
Continuous Functions Functions are the central objects of analysis. This chapter defines limits and continuity for functions of a real variable and it presents some consequences of continuity. To avoid problems with complicated domains (see Exercise 3-32 and the end of Section 16.3.1 for some details), in this chapter functions are usually considered on intervals or on intervals from which at most finitely many points were removed. These domains are sufficient to build the traditional calculus of functions of one variable. More complicated domains are handled in metric spaces in the second part of the text.
3.1 Limits of Functions The limit of a function at a point x is supposed to express what happens near x, but not necessarily at x. This is similar to the running index n of a sequence never actually becoming 00. While 00 is not in the domain of a sequence, a real number x can be in the domain of a function. Hence, we must explicitly remove x from consideration. In this section, functions are defined on an open interval from which a point x has been removed. In this fashion, we assure that each function is defined “close to the left of x” and “close to the right of x,” which is what we need to investigate for (two-sided) limits. Because convergence of sequences is already defined, we can use sequences to define convergence of functions.
Definition 3.1 Sequence formulation of the limit of a function. Let I g R be an open interval and let x E I . The number L E R is called the limit of the function f : I \ (x} -+ Iw at x zTfoor all sequences (zn}El with zn E I \ {x} for all n E M and lim zn = x we have lim f ( z , ) = L. In this case, we denote lim f ( z ) := L and we n-rffi n m ’ XZ ’ also say that f converges (to L ) at x. Similar to Theorem 2.1 1, the limit of a function at x is only affected by the values of the function near x .
49
50
3. Continuous Functions
Theorem 3.2 Let I 2 IR be an open interval, x E I , and let f ,g : I \ {x} + R be functions. If there is a number 6 > 0 so that f and g are equal on the subset { Z E I \ { x } : Iz - X I < 8 ) o f I \ {x},then f converges at x ifsg converges at x and in this case the equality lirn f ( z ) = lim g ( z ) holds. Z'X
Z'X
Proof. E: cise 3-2. By Theorem 3.2, Definition 3.1 also defines the limit at x for any function that is defined on a set D that contains a set I \ (x},where I is an open interval that contains x. Formally, we define the following.
Definition 3.3 Let D , R , S be sets with R D and let f : D -+ S be a function. The restriction of thefunction f to R, denoted f I R , is defined by f I R ( x ) := f (x) for all x E R. Definition 3.4 Let f : D -+ IR be a function and let x E R be so that there is an open interval I with x E I and I \ { x }C D. We define the limit off at x as the limit of the restriction f l ~ \ { a~ t~ x and denote it lim f ( z ) := lim f l ~ \ { ~ ) ( z ) . X Z'
Z'X
All the following results on functions f : I \ {x} -+ R also apply to functions with larger domains. Strictly speaking we would need to apply all definitions and results to the restriction of the function to a set I \ {x},where I is an open interval and x E Z. We will usually avoid this simple formality. Ultimately, Definition 16.33 will encompass this situation as well as some situations in which, for single variable functions, we use one-sided limits (see Definition 3.15 below).
Example 3.5 1. For all x E
R,we have lirn z
= x.
Z'X
2. For all x E R,we have lirn IzI = 1x1. X Z'
3. The function f (x) :=
{
1; 0; -1;
f o r x > 0, for x = 0, does not have a limit at x = 0. f o r x < 0,
Part 1 is trivial and part 2 follows from Exercise 2-12. To see that the function in part 3 does not have a limit at x = 0, it suffices to produce two sequences {yn}E1and {zfl}glthat converge to zero so that yn f 0 and zn f 0 for all n E N and { f ( y n ) ) z J
for all n
E
W we obtain n+x lirn f
(:)
= 1 f -1 = lim f n+x
(- :),
which completes
the argument.
:StandardProof Technique 3.6 If a sequence that converges to a number x from the is usually a good choice. For a sequence that converges
51
3. I . Limits of Functions a2
to x from the left,
{
x--
is usually a good choice. This idea is extended in
Standard Proof Technique i 3.8. l n =
0
l
There are at least two ways to define the limit of a function. We have already seen the formulation with sequences and we give the formulation with E and S below. The two formulations are equivalent, and hence either one could serve as the definition. With both formulations available, we can choose which one to use. Depending on the situation, one formulation may be preferable over the other to produce a simpler statement or proof. The proof of Theorem 3.19 is a good example of how each formulation is better suited for certain settings than the other.
Theorem 3.7 E-6 formulation of the limit of a function. Let I C R be an open interval and let x E I . Then L E R is the limit of thefinction f : I \ {x} + R at x iff for all E > 0 there is a 6 > 0 such that for all z E I \ {x} with ) z - x / < 6 we have that ( z ) - LI < E .
If
Proof. For “+,”let lim f ( z ) = L . Suppose, for a contradiction, the statement on XZ ’
the right is false. Then there is an E > 0 so that for each 6 > 0 there is a z E I \ {x] with / z - x J < 6 and l f ( z ) - LI 2 E (if necessary, use Standard Proof Technique 1 1 2.30 for the negation). Then for S := - there is a z , E I \ {x] with Iz, - X I < - and n n l f ( z n ) - Li 2 E . But then lim zn = x, while lim f (z,) either does not exist, or n+3o n+cc if it exists, then lirn f (z,) L (see Exercise 2-5). Either way we have amved at a n+m
+
contradiction. For “e,” let f : I \ {x} + R be such that for all E > 0 there is a 6 > 0 such that for all z E I \ {x}with Iz - X I < 6 we have that I f ( z ) - LI < E . We need to prove that for each sequence with zn E I \ {x} for all n E N and lirn zn = x we have
{z,}r=,
n+cc
that lirn f ( z , ) = L . n+m
Let { z ~ } : = ~ be a sequence with zn E I \ ( x } for all n E N and n+co lim zn = x, and let E > 0. Then there is a 6 > 0 such that for all z E Z \ {x] with Iz - X I < 6 we have I f ( z ) - Ll < E . Moreover, for S there is an N E N so that for all n 3 N we have Izn - x J < 8. But then for all n 2 N we infer l f ( z , ) - LI < E , and hence lim f ( z , ) = L . This proves that lirn f (z) = L . 17 + 30
Z’X
Standard Proof Technique 3.8 In the ‘‘+”part of the proof of Theorem 3.7, for all S > 0 there is a z with Jz - x J < 6 and other properties. To obtain a sequence {~,,},x7_~ that converges to x so that each z, has the other desired properties, it is standard 1 1 practice to use 6 := - and then pick an appropriate element zn with / Z n - X I < -. n n
Exercises 3- 1, Explain why after Definition 3.1 it is not necessary to prove that the limit of a function is unique 3-2. Prove Theorem 3.2.
3. Continuous Functions
52 3-3. Let m , b
E
W.Prove that Z'lirn X
mz
+ b = mx + b.
3-4. Let I be an open interval and let x E I . Prove that if f : I \ ( x ) -+ W does not converge to L at x, then there are an E z 0 and a sequence [ z ~ ) : = ~ so that lirn zn = x , zn E I \ (x} for all n E 8 and n-x
1 f ( z n ) - L 1 > E for a11n E N.
3-5. Prove that if m E Zand 1.1 is the floor function, then lirn l z ] does not exist. z+m
2-3
1
3-6. Prove that lirn -- -
6'
2-312-9
3-7. Prove that the Dirichlet function f ( x ) =
''
for Q' does not converge at any x E R. 0; f o r x $ Q,
3-8. Explain why, with the present definition, lirn &is not defined. Then state what the limit should be
z+o
and how we could circumvent our purely formal problem.
Nore. This problem will be resolved in Exercise 16-28.
3.2 Limit Laws Just as for limits of sequences, we are interested in how limits of functions relate to the algebraic operations, because this should simplify the computation of limits. First note that all algebraic operations on functions are defined pointwise.
Definition 3.9 Let D E R be a set and let f , g : D -+ R be functions. For all x E D we define ( f g)(x) := f ( x ) g(x), ( f - g)(x) := f ( x ) - g(x), and f (x) ( f . g)(x) := f (x) . g(x). For all x E D with g(x) f 0 we dejne (x) := -
+
+
($)
Theorem 3.10 Limit laws for functions. Let I Iw be an open interval, let x E I and let f , g : I \ {x} -+ R be functions such that lim f ( z ) and lirn g ( z ) exist. Then the Z'X
Z'X
following hold. lim( f X Z'
+ g ) ( z ) = lim f ( z ) + lim g ( z ) . Z'X
XZ'
lim( f - g ) ( z ) = lim f ( z ) - lim g ( z ) .
Z'X
lim( f X Z'
Z"X
g ) ( z ) = lim f ( z ) . Z'X
XZ'
JL:
g(z).
Each equation implicitly asserts that the limit on the left side exists (see box on p . 33). Moreovel; formally, in part 4 we would need to demand also that g ( z ) # 0 for all z E I \ {x). But lirn g ( z ) # 0 implies that g ( z ) # 0 for all z E I \ {x} that are near Z'X
x. Hence, i f g has zeros, we implicitly assume that g has been restricted appropriately
rather than worry about zeros that do not affect the convergence behavior:
53
3.2. Limit Laws
Proof. Throughout this proof let L f := lim f (z) and let L , := lim g(z). Z’X
X’Z
In this proof we will use Definition 3.1 as well as Theorem 3.7. Although it will turn out that Definition 3.1 in conjunction with the limit laws for sequences is more effective for this prooj it will also be instructive to see how 8-6proofs are constructed. The reader will compare the two approaches by proving each part with the respective other approach in Exercises 3-9 and 3-10. To prove part 1, we use Theorem 3.7. We need to prove that for all E > 0 there is a 6 > 0 s o t h a t f o r a l l z ~I\{x)withIz-xl < 6 w e h a v e I ( f + g ) ( z ) - ( L f + L , ) l <E. Let E > 0. Then there are 6f > 0 and 6, > 0 so that for all z E I \ {x) with
If
(z) - L f l < - and for all z E I \ {x) with Iz - X I < 6, we 2 E have Ig(z) - L , < -. Let 6 := min{bf, 6,) (compare with Standard Proof Technique 2 2.6). Then for all z E I \ {x) with Iz - X I < 6 we obtain via the triangular inequality that
Iz - X I
< Sf we have
&
1
This means we have proved that for all E > 0 there is a 6 > 0 so that for all I \ {x) with Iz - X I < 6 we have f g)(z) - ( L f L g ) l < E . Consequently, lim( f g ) ( z ) = L f L,.
z
E
Z’X
+
I( +
+
+
To prove part 2 using Definition 3.1, we need to prove that for all sequences {z, with Z, E I \ {x) for all n E N and lim zn = x we have lirn f (z,) = L .
(z,}zl be a sequence in I \ {x} with lim n m ’
Let
Because
n-cc
n-cc Z, = x. By
{z,)zl was arbitrary this implies lim( f
)El
Theorem 2.14 we infer
-g)(z) = L f
-
L,.
X’Z
The proofs of parts I and 2 show that to prove the limit laws, Definition 3.1 is more effective than Theorem 3.7. Nonetheless, both ways are actually equally complex overall. If we compare the proof of part I with the proof of part 1 of Theorem 2. I 4 we see striking similarities in the arguments. This means that the complexity of a proof using Dejinition 3.1 is simply delegated to the proof of an earlier result (Theorem2.14). The reader will have the chance to analyze the similarities and the direrences in the proofs for part 3 in Exercise 3-9. The similarity between the proofs here and the proofs for Theorem 2. I 4 can be used to translate the proof for part 4 into a complete proof of part 4 of Theorem 2.14. The rather complicated choices in the header were of course made after the final inequalities had been analyzed carefully. The proof of part 3 is left to the reader as Exercise 3-9. For part 4, we use Theorem 3.7. So we need to prove that for all E > 0 there is a f L S > O s o t h a t f o r a l l z E Z \ {x} with Iz - X I < 6 we have l;(z) < E.
$1
Let E
1
0. Then there is a 6f > 0 such that for all z
have f ( z ) - L f l <
7 ‘ILX1. Moreover, there is a 6,
>
\ (x)with Iz - x I < Sf we 0 such that for all z E I \ {x)
E
I
3. Continuous Functions
54
with
/z-XI
2 ( 2 ILf
We have proved that for all
Iz
ILg I*
< 6, we have / g ( z ) - L,/ <
- x / < 6 we have
E
I + 1)
. Finally, there is a v
> 0 there is a 6 > 0 so that for all c
E.
z
E
> 0 so
I \ (x)with
Therefore the limit of the quotient exists and
Just like limits of sequences, limits of functions preserve inequalities and there also is a Squeeze Theorem.
Definition 3.11 Let D 5 R be a set and let f , g : D + R be functions. We say 3f is pointwise less than or equal to g, that is, i r f o r all x E D the inequaliv f _< g 1 f ( x ) 5 g ( x ) holds. Theorem 3.12 Let I 5 R be an open interval, let x E I and let f , g : I \ (x}+ R be functions. I f f 5 g on I \ {x} and f and g converge at x, then lim f ( z ) 5 lim g ( z ) . Z'X
Z'X
Proof. Exercise 3- 1 1. Theorem 3.13 The Squeeze Theorem for functions. Let I C R be an open interval, let x E I and let f , g , h : I \ {x} -+ R be functions. I f f 5 g 5 h on I \ {x) and f and h converge at x with lim f ( z ) = lirn h ( z ) , then g converges at
x and lim g ( z ) = lim f ( z ) = Z'X
Z X'
i% h(z).
Z'X
Z'X
55
3.2. Limit Laws
Proof. Exercise 3- 12. Finally, convergence is also preserved by the composition of functions.
Theorem 3.14 Let I , J 5 R be open intervals, let x E I , let g : I \ {x) -+ IW and f : J + R befunctions with g [ I \ {x}] J , and let lirn g ( z ) = L E J . Assume that Z"X
lirn f ( y ) exists and that g [ I \ (x}] E J \ { g ( x ) } ,06 in case g(x)
E g[Z
J-+L
\ (x}],that
lirn f ( y ) = f ( L ) . Then f o g converges at x and lirn f o g(x) = lim f ( y ) .
J-tL
y+L
Z'X
Proof. Let M := lirn f ( y ) and let
be a sequence in the set I
\ {x} so
Y+L
that lirn zn = x. Then lim g ( z n ) = L . If no zn satisfies g ( z n ) = g(x), we obtain n+oo
n-cc
lirn f ( g ( z n ) ) = M , while if some g ( z n ) are equal to g(x), then M = f(L)and we
n-+w
can infer lirn f ( g ( z n ) ) = f ( L ) = M in this case also. Because the sequence {z,},"=, n+w was arbitrary the result is established.
Exercises 3-9. Completing the proof of Theorem 3.10. Let x E I and let f . g : I \ {x)--f lim f ( z ) and lim g(z) exist.
R be functions such that
Z"*
Z"X
(a) Prove part 3 using Definition 3.1.
(b) Prove part 3 using Theorem 3.7.
3-10. Alternative proofs for the proved parts of Theorem 3.10. Let x E I and let f,g : I \ {x] + functions such that lirn f ( z ) and lirn g(z) exist. (a) Prove that Jir+mx(f
R be
X Z'
X Z'
+ g)(z) = lirn f ( z ) + Jim g(z) (part 1) using Definition 3.1 XZ'
X Z'
(b) Prove that lirn ( f - g)(z) = lim f ( z ) - lirn g ( z ) (part 2 ) using Theorem 3.7. z+x
Z'X
(c) Prove that if lirn g(z) f 0, then ZX'
Z+X
!s(5) ( z )
= l i m Z ~ xf ( z ) (part4) using Definition 3.1. limz+x g(z)
3-11. Prove Theorem 3.12. (a) Using Definition 3.1.
(b) Using Theorem 3.7.
3-12. Prove Theorem 3.13. (a) Using Definition 3.1
(b) Using Theorem 3.7
3-13. Explain the similarities between the proof of part 1 of Theorem 3.10 as presented and the proof of part 1 of Theorem 2.14. 3-14. Computation of limits. (a) Let I be an open interval, let x E I and let f : I \ (x) + Jim f(z) = lim f ( x h ) . i-x
+
h-0
(b) Compute each of the following limits x 2 - 5x 6 i. lirn x 1 3 x 2 - x - 6 x-9 iii. lim X"9X2( 9 + & ) x + r n
+
ii. lim x-2
W be a function. Prove that
3x3 - x* - 12x x2 - 4
+4
56
3. Continuous Functions
3.3 One-sided Limits and Infinite Limits Section 3.5 will show that it is advantageous to consider continuous functions (see Definition 3.23) on closed intervals. But that means we also need a notion of convergence at the endpoints of a closed interval. One-sided limits provide just that.
Definition 3.15 Sequence formulation of the left limit of a function. Let a < b. The number L E R is called the left limit of the function f : [ a ,b ) + R at b iff for all sequences in [ a ,b ) with lirn zn = b we have lirn f (z,) = L. In this case, n+cc n+cc we denote lim f ( z ) := L and we say f converges (to L ) at b from the left.
{zn]zl
z+b-
The right limit at a for a function f : ( a , b ] -+ E% is defined similarly. It is denoted lim f ( z ) , and we say f converges at a from the right.
ziaf
I f f : D -+ R is a function and the domain D contains an interval [ a ,b ) with a c b, we dejine lim f (2) := lirn f I [ a , b ) ( z ) if it exists. (Exercise 3-15 shows that z+b-
z+b-
these left limits are well dejined.) Right limits are defined similarly. Similar to limits, we prove most results for functions defined on half-open intervals. These results are also valid for functions with larger domains. We simply apply them to the appropriate restrictions.
Example 3.16 Let m E Z.For thejoor and ceiling functions of Definition 1.33, we have lirn l z ] = m - 1, lim LzJ = m, lim rzl = m, and lirn [zl = m 1. 0 z+m-
z-+m+
zim-
z+m+
+
Definitions 3.15 and 3.1 differ in only one way. For a one-sided limit, the sequences must all stay on one side of the number, while in Definition 3.1 the sequences can have values on either side. To emphasize the ability to approach from either side, limits of functions are sometimes called two-sided limits. With such strong similarity in the definitions, it is only natural that the theorems that govern one-sided limits are similar to the theorems that govern (two-sided) limits.
Theorem 3.17 Limit laws for one-sided limits. Let f , g : [a,b ) -+ R be functions such that lim f ( z ) and lirn g ( z ) exist. Then the following hold. z+b-
zib-
I . lim ( f + g ) ( z ) = lim f ( z ) z-tb-
z+b-
+ z-tblim
g(z).
Each equation implicitly asserts that the limit on the left side exists. (See box on page 33.) Moreovel; because lim g ( z ) # 0 in part 4 implies that g ( z ) # 0 for z near b z-b-
(where it matters), we did not demand g ( z ) hold for right limits.
+ 0 for all z E [ a ,b). Similar limit laws
57
3.3. One-sided Limits and Infinite Limits Proof. Exercise 3-16.
Theorem 3.18 8-6formulation of the left limit of a function. The number L E R is the left limit of thefunction f : [ a ,b ) + R at b ifSfor all E > 0 there is a 6 > 0 such thatforallzE [ a , b ) w i t h l z - b l < 6 w e h a v e ( f ( z ) - L (< E . Proof. Exercise 3- 17. Theorem 3.19 connects one-sided limits to (two-sided) limits. Note that to make the proof efficient, we use the sequence formulation of the limit for one direction and the E-8 formulation for the other direction.
Theorem 3.19 Let I g R be an open interval, let x E I and let f : I \ {x) + R be a function. Then lim f ( z ) exists iff lim f ( z ) and lirn f ( z ) both exist and are equal. z'x-
Z'X
Z'X+
In this case the limit is equal to the left and the right limit.
Proof. For
"+,"let
for all sequences
L := lirn f ( x ) . Using Definition 3.15 we must prove that Z'X
{z,}:~ in I with zn
< x for all n E
N and nlim -ffi
Z, = x
{z,}Elin I with
lim f (z,) = L and we must prove the same result for all sequences > x for all n E N and lirn Z , = x.
fl'cc
zn
we have
rl-m
Let {z,},"=, be a sequence in I with Z , < x for all n E N and lirn Z , = x. By Definition 3.1 we have lirn f (z,) = L , which was to be proved. Sequences with n-+ffi zn > x for all n E N are treated similarly. Hence, lim f ( z ) = lirn f ( z ) = L . fl'ffi
-x'z
Z'X+
For "+,"let lim f ( 2 ) = lirn f ( z ) =: L . By Theorem 3.7, we must prove that Z'x-
Z'X+
> 0 there is a 6 > 0 so that for all l f ( z ) - L / < &.
for all
E
z
E
I
\
{x} with
Iz
-
XI
< 6 we have
Let E > 0. By Theorem 3.18 there is a 61 > 0 so that for all z E I \ (x} with z < x and Iz - X I < 61 we have l f ( z ) - LI < E . By the corresponding result for right limits, there is a 6, > 0 so that for all z E I \ { x } with z > x and ) z - X I < 6, we have I f ( z ) - LI < E . Let S := rnin{&, &}. Then for all z E I \ {x} with / z -xi < 6 we infer f ( z ) - L / < E . By Theorem 3.7, this proves Z'lim f (z) = L. X
1
Standard Proof Technique 3.20 To prove that a function has a limit at x often proves that the left and the right limits exist and that they are equal.
E
R one
Infinity and negative infinity are not numbers, so formally they do not qualify as limits. But knowing that a function grows beyond all bounds near a point gives more information than a statement that the limit does not exist. Hence, we extend the language to allow infinite limits.
Definition 3.21 Let f : [ a ,b ) + R be a function. Then the left limit off at b is said to be infinity iff f o r all sequences {zn}Klin [ a , b ) with lim zn = b we have lim f ( i n= ) 00. We denote lim f ( z ) := 00. fl'ffi
n
'x:
z+b-
58
3. Continuous Functions
Infinite right limits at a f o r a function f : ( a , b] -+ R and infinite (two-sided) limits of a function f : I \ (x} 3 E% at x E I , where I is an open interval, are defined similarly. Limits equal to negative infinity are also defined similarly and they are denoted by -m. Finally, as before, infinite one-sided and two-sided limits of functions with larger domains are defined via the limits of appropriate restrictions.
We chose to put one-sided infinite limits in the spotlight in Definition 3.21, because functions often have different behavior to the left and to the right of a point. For 1 1 example, lirn - = 00 and lirn - = -m.
z
Z'O+
z+o- z
It is not surprising that there is a formulation of infinite limits that is similar to the E-8 formulation of finite limits.
Theorem 3.22 M-8 formulation of infinite left limits. The left limit of the function f : [ u , b ) + R at b is infinite iff f o r all M E R there is a 6 > 0 such that f o r all z E [ a ,6 ) with Iz - bl < 6 we have f (2) > M .
Proof. Exercise 3-18. Of course, similar results also hold for right-sided and two-sided limits. Limit laws for infinite limits and a version of Theorem 3.19 are given in Exercises 3-22 and 3-23.
Exercises 3-15. Let f,g : [a. b ) -+ B be functions. Prove that if there is a number S t 0 so that f and g are equal on the subset [ z E [ a ,b ) : / Z - bl < S ] of [ a , b ) ,then the left limit o f f at b exists iff the left limit of g at b exists and in this case we have lirn f ( z ) = lirn g(z).
z- b-
i-b-
3-16. Prove Theorem 3.17. That is, let f,g : [a. b ) + X be functions such that lim f ( z ) and lirn g ( z ) z-tb-
z+b-
exist and prove each of the following. (a) - : ~ - ( f+ g ) ( z ) = lim f ( z ) z+b-
(b)
lirn :+b-
+ z+blim
(f - g ) ( t ) = lirn f ( z ) - lirn z+b-
g(z). g(z)
z-b-
lirn (f . g ) ( z ) = lim f ( z ) . lirn g ( z ) z-tb-
(d) If lirn g ( z ) z-tb-
z+b-
# 0, then lirn z+b-
('1 &'
(z)=
lim,,b-
f(z)
lim,,b-
g(z)
3-17. Prove Theorem 3.18.
3- 18. Prove Theorem 3.22. 3-19. Prove that f ( x ) = 3-20. Prove that Iim
z+o+
x+3; (x + 10;
h ,
forx
1, has a limit at x = 1 and state its value forx = 1,
= 0. Explain why this is a satisfactory resolution of the formal problem in
Exercise 3-8 or why it is not. 3-21. A function f is called nondecreasing on I C W iff for all x i < x2 in I we have f ( x 1 ) 5 f ( x 2 ) . Let f : [ a , b ] -+ R be a nondecreasing function.
59
3.4. Continuity (a) Prove that for every x E ( a , b] we have lim f ( z ) = sup
{ f(z):z<x }
Z+S-
Hint. Mimic the proof of the Monotone Sequence Theorem. (b) Prove that for every x E [ a , b ) the right limit lim f ( z ) exists and state its value ZX '+
3-22. Limit laws involving infinite limits of functions. Let f . g : [ a , b ) + W be functions, and let lim f ( z ) = m. Prove the following. :+b-
1
(a) If g is bounded (that is, there is an M > 0 so that for all z E [a. b ) we have g(z) then lirn f ( z ) + g(z) = x and if all f ( z ) are nonzero, then lirn z+b-
z+b-
(b) If lim g(z) = m, then lirn f ( i ) :+b-
z-b-
+ g(z) = co and z lim ib-
I < M),
go = 0. f(z)
f ( z ) g ( z ) = 00.
(c) If c > 0 is a real number, then lirn c f ( z ) = x. z+b-
(d) If c < 0 is a real number, then lim c f ( z ) = -m z-b-
JR be an open interval, let x E I and let f : I \ [ x ] 3-23. Let I lim f ( z ) =miff lirn f ( z ) = lim f ( z ) = 00. ZJ'
Z'X-
--f
R be a function. Prove that
ZX '+
3.4 Continuity Continuous functions are usually defined as functions for which the limit is computed by substituting the value. Because we may need to take care of endpoints, the formalization of the elementary definition from calculus requires an extra item (see number 4 below).
Definition 3.23 Let D 2 R be an interval of nonzero lengthfrom which at mostfinitely many points have been removed and let f : D -+ R be a function. Then f is called continuous at x
iff
1. f (x) is dejned, that is, x E D, and
2. lirn f ( z ) exists, and Z-+X
3. lim f ( z ) = f ( x ) ,and Z"X
4. I f x is an endpoint of D, use left or right limits in 2 and 3, as appropriate. f is called continuous (on D ) iff f is continuous at every x
E
D.
We could also define continuity for functions defined on sets for which every point of the domain is contained in an interval of nonzero length. Exercise 3-32 shows that with the present definition this idea is a bit too simple to produce a sensible result. This is not a problem, because in the early part of the text the only functions whose domains are not intervals are rational functions. For these functions the pathology of Exercise 3-32 is not an issue. Therefore we relegate all concerns regarding more complicated domains to Section 16.3.
3. Continuous Functions
60
__ I_ T
r---------
/
X
For theorems, we will usually work with functions that are defined on intervals, because if D is an interval from which at most finitely many points were removed, then f : D +. R is continuous at x E D iff f 1 is continuous at x, where I is a maximum-sized (with respect to containment) interval contained in D that contains x.
Example 3.24 1. Constantfunctions are continuous at every x E
W.
2. The function f (x) = x is continuous at every x
E
R.
3. Thefilmtion f ( x ) = 1x1 is continuous at every x E
R.
Parts 1 and 2 are trivial and part 3 follows from part 2 of Example 3.5.
0
It is useful to incorporate the definition of limits directly into a characterization of continuity. With such a characterization it is not necessary to resolve the multiple parts of the original definition whenever we work with continuity. Figure 6 shows graphical interpretations of the conditions.
EX be an interval, let x Theorem 3.25 Let I Then the following are equivalent.
E
I and let f : I -+ R be a function.
1. f is continuous at x.
2. For every sequence lim
n+m
f(zn) =
{zn},"=, with zn
E
I for all n E
N and n+m lim zn
= x, we have
f(x).
3. For every E > 0, there is a 6 > 0 such that for all z E I with Iz - x I < S we have
If(z)- f
< &.
3.4. Continuity
61
Proof. We will prove “1+2,” “2=+3”and “3+1.” The remaining implications follow because logical implications are transitive. That is, the implications “ I +2” and “2=+3”imply “l+3,” the implications “2=>3”and “3+l” imply “2+1,” and so on. We will assume throughout that x is not an endpoint of the domain. The arguments are easily modified (by using appropriate one-sided limits and theorems) for the case that x is an endpoint. For “1+2,” we need to prove that for every sequence { Z n } E i with Zn E I for all n E Nand lirn zn = x we have lim f(z,) = f ( x ) . n’cv n+cc Because f is continuous at x, we have lim f ( z ) = f (x). By Definition 3.1, this
(zn}zl with z ,
XZ ’
I \ {x) and n-too lirn Z n = x we have that (zn}gl be a sequence with Zn E I and lirn Zn = x . n+cv n+oo If there is an N E N so that zn = x for all n 2 N, then there is nothing to prove. Otherwise let (zn,}El be the subsequence of all elements with zllk E I \ {x}. Then lirn znk = x, and hence lim f (z,,) = f (x). Now let E > 0. Then there is a K E N means that for all sequences lim f (z,) = f (x). Now let
k+ cv
E
k+oc
so that for all k 2 K we have I f (z,,) - f (x)( < 8. Then for all n 2 nK either zn = x and f (2,)- f (x)] = 0 < E orthereis a k 2 K withn = nk and f ( z n k ) - f (x)l < E . This means lirn f (2,) = f (x) for every sequence with Z, E I for all n E N n-oo and lirn Z, = x, which establishes this part of the proof. n-oc The proof of “2+3” is similar to the proof of “=>” in Theorem 3.7 (Exercise 3-24). For “3+1,” note that, because f is defined on I and x E I , f is defined at x. The condition in part 3 implies by Theorem 3.7 that lirn f ( z ) = f (x), which completes the
1
{z,}zl
1
X’Z
proof. Because of the formal problems with limits of functions outlined at the end of Section 16.3.1, in general settings one of the conditions in Theorem 3.25 is normally used to define continuity.
Standard Proof Technique 3.26 To prove the equivalence of several conditions, it is standard practice to prove that the first implies the second, the second implies the third, and so on, and finally that the last condition implies the first. All other implications 0 follow from the transitivity of logical implications. The proofs of parts 5 and 6 of Theorem 3.27 serve as good examples how the conditions in Theorem 3.25 can be used in conjunction.
Theorem 3.27 Let I R be an interval, let x at x E I . Then the following hold. 1. f
E
I and let f , g : I + R be continuous
+ g is continuous at x.
2. f - g is continuous at x.
3. f . g is continuous at x .
4. I f g ( x ) # 0 f o r all x
E
f
I , then - is continuous at x. g
3. Continuous Functions
62
5. max{f ,g } is continuous at x. (The maximum is dejinedpointwise.)
6. min{f,g } is continuous at x. (The minimum is definedpointwise.) Proof. The first four parts are direct consequences of the corresponding limit laws. For part 5 , let x E I. We will establish continuity of max{f, g } at x by proving that for all sequences with Z, E I for all n E N and lim zn = x we have
{z,}zl
,--too
lim max { f ( z , ) , g ( z n ) } = max { f b ) ,g w } . n-oc Let be a sequence in I with lim zn = x. In case f ( x ) # g ( x ) , assume
{z,}zl
n+Cc
without loss of generality that f ( x ) > g ( x ) . To prove that the limit of the image sequence is lim max{f, g } ( z , ) = max{f, g } ( x ) = f ( x ) , let E > 0. Because f and n-+m .. .. g are continuous at x, there is a 6 > 0 so that for all z E I with Iz - X I < 6 we f ( x ) - g ( x ) . In have I f ( z ) - f ( x ) l < min E , f ( x ) and I g ( z ) - g ( x ) l < 2 particular, for all z E I with'lz - X I < 6 we dbtain
1
ig(')}
and hence max{f, g } ( z ) = f ( z ) . For 6, find an N Jz, - X I < 8.Then for all n 2 N we infer
E
N so that for all n 2 N we have
I maxIf3 g } ( z n ) - maxIf, g } ( x ) l = I f ( z n ) - f ( x > l <
E,
and hence max{f, g } is continuous at x when f(x) > g(x). This leaves the case f ( x ) = g ( x ) . Let E > 0. Because f and g are continuous at x, there is a 6 > 0 so that for all z E I with Iz - xi < S we have that l f ( z ) - f ( x ) l < E and I g ( z ) - g ( x ) l < E . For 6, find an N E N so that for all n 2 N we have that Iz, - X I < 6. For all n 2 N , the maximum max{f, g } ( z n ) is equal to f ( z , ) or g(z,). Because Iz, - X I < 6, if max(f, g } ( z n ) = f(z,) we infer max{f, g } ( z n ) - maxIf9 g } ( x ) l = If(z,> - f ( x ) J < E andifmaxIf, g } ( z n >= g(z,) we infer max{f, g}(z,) - max{f, g } ( x ) l = Ig(z,) - g ( x ) / < E . Thus the maximum max{f, g } is continuous at x when f ( x ) = g ( x ) . The proof of part 6 is left as Exercise 3-25b.
I
1
There is a faster proof of part 5 that relies on Theorem 3.30 below and on an algebraic representation of the maximum (see Exercise 3-26a). Because our main focus is on standard techniques in analysis, the longer proof was presented here. Theorem 3.27 gives access to two standard examples of continuous functions.
Example 3.28 A polynomial is a function p : R -+ JR for which there are an n n
and ag, . . . , a , E R so that a, f 0 and for all x
E
E
W
Iw we have p ( x ) = x a ; x ; . The ;=0
number n is called the degree of the polynomial. The constant function p ( x ) = 0 is also considered to be a polynomial. Its degree is defined to be -m. Every polynomial is continuous on R. (See Exercise 3-28.)
3.4. Continuity
63
Example 3.29 A rational function is a function r for which there are two polynomials p ,q : R
R so that for all x
P(X)
R for which q ( x ) f 0 we have r ( x ) = -.
By q(x) Theorem 3.27 and Example 3.28, every rational function is continuous on its domain { x E R : q ( x ) f O}. (We implicitly use here that every polynomial has at most finitely many zeroes.) 0 --f
E
Continuity is also preserved by compositions.
Theorem 3.30 Let I , J R be intervals, let g : I -+ R be continuous at x E I , let g [ I ] C J and let f : J -+ R be continuous at g ( x ) . Then f o g : I -+ R is continuous at x . Proof. We will prove that for every E > 0 there is a 6 > 0 so that for all z E I with Iz - X I < 6 we have ( g ( z > )- f ( g ( x ) ) l < E . Let E > 0. Because f is continuous at g ( x ) , there is a 6f > 0 so that for all y E J with / y - g ( x ) < 6 f we have ( y ) - f ( g ( x ) ) < E . Because g is continuous at x , for Sf there is a 6 > 0 so that for all z E I with jz - X I < 6 we have I g ( z ) - g(x)l < 6f. But then because l g ( z ) - g(x)j < 6 f we infer f ( g ( z ) ) - f ( g ( x ) ) l < E . Therefore we have proved that for every E > 0 there is a 6 > 0 so that for all z E I with 1 - X I < 6 we have l f ( g ( z ) )- f ( g ( x ) ) l < E , which means that f o g is continuous at x . H
If
I
I 1
If
We conclude this section by characterizing discontinuities.
Definition 3.31 Let D be an interval of nonzero length from which at most finitely many points X I , . . . , x,, have been removed. Ifthefunction f : D -+ R is not continuous at x E D U ( X I , . . . , x,}, we speak of a discontinuity at x . There are several types of discontinuities. (For visualizations, see Figure 7,for examples, see Exercise 3-31.) I . I f lim f ( z ) exists, or i f x is an endpoint of D U ( X I , . . . , x,,} and the appropriate Z'X one-sided limit exists, but the limit is not equal to f ( x ) ; or if x is not an endpoint of D U { X I , . . . , x,} and f is not defined at x , we speak of a removable discontinuity. 2. I f lim f ( z ) and lim f ( z ) exist, but they are not equal, we speak of a jump Z'X-
Z'X+
discontinuity. 3. I f (at least) one of lim f ( z ) and lirn f ( z ) does not exist and Z'X-
Z'X+
sequence (z,}:, in D that converges to x and limn+oo of an infinite discontinuity.
1f(z,)] =
if there is a 00,
we speak
4. Iftat least) one of lim f ( z ) and lirn f ( z ) does not exist, ifthere is a 6 > 0 so z'x-
Z'X+
{z,,}zl
that f is bounded in { z E D : / z - x I ia} and ifthere are two sequences and ( w , ] that ~ ~ converge to x such that for all n E W we have z,,, w, < x (or z,, w, > x ) and such that both lirn f ( z , ) and lirn f (w,)exist, but they are n+oo ,--too not equal, we speak of a discontinuity by oscillation.
3. Continuous Functions
64
Figure 7: Visualization of the possible discontinuities of a function.
Theorem 3.32 Let D C R be an interval of nonzero length from which at most$nitely many points XI,. . . , x, have been removed and let f : D + R be a function. Then eveiy discontinuity x E D U {XI,. . . , x,} of f is of one of the four types listed in Dejinition 3.31. Proof, Let x E D or let x be one of the finitely many elements that were removed and assume that f is not continuous at x. Let the discontinuity at x be neither a removable discontinuity, nor a jump discontinuity, nor an infinite discontinuity. We will prove that it must be a discontinuity by oscillation. If lirn f (x) and lirn f ( x ) both Z’X-
Z’X+
existed, then they would either be equal and the discontinuity would be removable, or not, in which case the discontinuity would be a jump discontinuity. Hence, one of the one-sided limits does not exist at x. If x is the supremum or the infimum of D , then f is defined at x and one of the two one-sided limits does not exist by default. In this case, the respective other one-sided limit also must not exist, because otherwise there would be a removable discontinuity at x. By symmetry, we can assume without loss of generality that lim f (x) does not exist and f is defined on some interval [x - 6 . x). zx’-
Because the discontinuity at x is not an infinite discontinuity, there is a u > 0 so that V f is bounded on [x-u, x). Let zn := x - -. Then lirn z , = x. First consider the case n n’oo that lim f (z,) =: L exists. Because lim f ( z ) does not exist, there must be an E > 0 n+oo
ZX’-
and a sequence {vk}&
1
1
so that lim
k+m
Uk
= x, and for all k E
N we have x - v 5
uk <
x
oc
and f ( U k ) - L > E . Because f is bounded on [x - u , x), the sequence { f ( U k ) J k Z l is bounded. By the Bolzano-Weierstrass Theorem, it has a convergent subsequence co {f(Uk,)},=1. Then n-oo lim f(vk,) L , and hence {wn}E1:= and {zn1Z1 are sequences as required in the definition of a discontinuity by oscillation. oo If { f (z.)},=~ does not converge, note that it is bounded, and hence it has a convergent subsequence { f Now f has a discontinuity by oscillation at x,because we can replace with { z n k } E in I the above argument and then repeat it. W
+
{w,}zl
(znk)}E1. {zn]zl
Standard Proof Technique 3.33 When an argument requires a convergent sequence and all that is guaranteed for a given sequence ( z ~ } ? = ~is a convergent subsequence, then one often assumes without loss of generality that ( z , , ] ~ = converges. ~ This is because, just like at the end of the proof of Theorem 3.32, the given sequence can be replaced with a convergent subsequence that (usually) has all the properties of (Z,},X=~, plus it converges. 0
{zn]zI
3.4. Continuity
65
Although Theorem 3.32 characterizes all possible discontinuities for functions from
R to R,other types of discontinuities exist for functions with infinite dimensional range (see Exercise 16-36).
Exercises 3-24. Prove part “ 2 j 3 ” of Theorem 3.25. 3-25. Completing the proof of Theorem 3.27
(a) Give all details of the proofs of parts 1-4. (b) Prove part 6. 3-26. Alternative proofs of parts 5 and 6 of Theorem 3.27. Let I be an interval and let f,g : I + R be functions.
(a) Use parts 1 and 2 and Theorem 3.30 to prove that if f and g are continuous at x
E
I , then
max(f, g ) is continuous at x .
Hinr. First prove that for all a , b
E
1 2
W the equality max[a, b ] = - ( a + b + la - b / ) holds.
(b) Give a similar proof of part 6 of Theorem 3.27. 3-27. Prove that for all n E N the function f ( x ) = x n is continuous on W.Then prove that for all m E W the function f ( x ) = Y r n is continuous on I% \ ( 0 ) . 3-28. Prove that all polynomials are continuous on E. Hint. Induction on the degree. 3-29. Alternative proofs of Theorem 3.30.
(a) Prove Theorem 3.30 using Definition 3.23 and Theorem 3.14. (b) Prove Theorem 3.30 using part 2 of Theorem 3.25. 3-30. Explain why we demand that the interval in Definition 3.23 must have nonzero length. 3-31. Examples of discontinuities.
2
(a) Prove that f ( x ) = - has a removable discontinuity at 0 X
(b) Prove that f ( x ) = rx] has a jump discontinuity at 0. 1 (c) Prove that f ( x ) = - has an infinite discontinuity at 0. x
(d) Let g(x) := +2;
.
1 f o r O S x i -, 2 1 for - < x 5 1. 2 -
,
has a discontinuity by oscillation at 0
u [k - L. ‘1 m
3-32. Let
u :=
10n ’ n
,,=I
f : [-l,O] (a) (b) (c) (d)
UU
--f
:= ( x E
B :(3n
I
E N :x E
0;
[
-
A,:])I
and let the function
forx E [-l,O], E U.
W be defined by f ( x ) := 1: forx
Prove that f is continuous on every interval I that is contained in its domain D := [- 1 , OIUU. Prove that every point in D is contained in an interval of nonzero length. Explain why the function still should not be considered to be continuous on D . Suggest a generalization of the definition of continuity that would allow domains such as D and that would make f discontinuous at 0.
This function will be revisited in Exercise 16-29.
66
3. Continuous Functions .xA approach c
I
,
from the left
sn approach c from the nght
b
-'
"
1
f(x)>Ofor
Figure 8: Visualizing the Intermediate Value Theorem. Intuitively ( a ) it is clear that an unbroken graph that goes from a point below the x-axis to a point above the xaxis must cross the x-axis at least once (solid graph) and that it could even cross the x-axis multiple times (dotted continuation past the first intercept). Part ( b ) gives a visualization of the proof. Because the sequences { x , ) and ~ ~ {xA},"=, meet in the middle at (c, f ( c ) ) , we conclude f ( c ) = 0.
3.5 Properties of Continuous Functions Aside from their obvious connection to limits, continuous functions are interesting because they have several useful properties.
Theorem 3.34 Intermediate Value Theorem. Let a < b and let f : [ a ,b] -+ R be a continuousjknction. I f f ( a ) < 0 and f ( b ) > 0 (or vice versa) then there is a c E ( a , b ) such that f ( c ) = 0. (Also see Figure 8(a).) Proof. Assume without loss of generality that f ( a ) < 0 and f ( b ) > 0. The proof for the other case is similar. The set G := {x E [ a ,b ] : f ( x ) > 0) contains b and it is bounded below by a . Let c := inf(G). We will show that c is as claimed in the theorem. First, we show that c $ { a ,b ) . For a contradiction suppose that c = a . Then by the version of Proposition 1.21 for infima, for each n E N there would be an
xn
E
(a,a
+ :)
with f (x,) > 0. (Note that we are using Standard Proof Tech-
nique 3.8 here.) But then lim x, = a , and by continuity of f we could infer that n-+m
0 > f ( a ) = n+m lim f(x,) 3 0, a contradiction. The inequality c f: b is proved similarly (see Exercise 3-33). Because c # b, again by the version of Proposition 1.21 for infima, for each n E N there is an x, E c
c, c
( # a , there is an N
E
+-
3
with f ( x , ) > 0. In particular, lim x, = c. Because n-cc
1 is greater than n = c and f(xA) 5 0 for
W so that for all n 2 N the number XI, := c -
XA
or equal to a. Hence, the sequence {xA},"==, satisfies lim n+cc all n E N.For a visualization of the sequences, consider Figure 8.
-
3.5. Properties of Continuous Functions
67
Because f is continuous, we infer lim f ( x n )= lim f (x;) = f ( c ) . But the inn-oc
n-oc
(a)
I 0. Thus equalities for f ( x n ) and f (x;) show that lim f ( x , ) 3 0 and lim f n-+w n-cc f ( c ) must be greater than or equal to zero and less than or equal to zero, which implies f ( c ) = 0.
The Intermediate Value Theorem immediately implies that continuous images of intervals are intervals, too.
Definition 3.35 Let A , B be sets, let f : A -+ B be a function and let C the image of C under f is dejned to be f [ C ] := { f ( c ) : c E C}. Theorem 3.36 Let I is an interval.
G R be an interval and let f
:I
s A.
Then
+ R be continuous. Then f [ I ]
Proof. Let I , u E f [ I ] with 1 < u and let m E (1, u ) . Then there are a , b E I with f ( a ) = 1 and f ( b ) = u . Without loss of generality assume a < b. The function g ( x ) := f ( x ) - m is continuous on [ a , b ] . By the Intermediate Value Theorem there is a c E ( a , b) so that g ( c ) = 0. But then f ( c ) = g ( c ) in = in. Because i n , 1, u were arbitrary we have proved that for any two elements 1 < u of f [ Z ] and any element m between them we have that m E f [ Z ] .Therefore f [ I ] is an interval.
+
We now turn to inverse functions.
Definition 3.37 Let A , B be sets and let f : A i B be a bijective function. Then the inverse function o f f is the unique (see Exercise 3-34)function f : B + A that maps each b E B to the unique a E A so that f ( a ) = b.
-'
Theorem 3.38 Let I 2 R be an interval and let f : I -+ R be a continuous injective function. Then the inverse function f : f [ I ] i I is also continuous. Proof. Clearly, f maps Z bijectively onto f [ Z ] ,and hence f has an inverse function f - ] that is defined on f [ Z ] . By Theorem 3.36, f [ Z ] is an interval. All that is left is
1
oc
to show that if a sequence { ~ ~ ] n h in 3 , f~ [ I ]converges to y E f [ Z ] ,then f - ( yn ) l n = l converges to f - ' ( y ) . Let(yn}E1beasequenceinf[I]with lim yn = y . F o r n ~ N , l e t x ,:= f - ' ( y n ) n+oo
and let x := f - ' ( y ) . For E > 0, let J := { z E I : Iz - X I < E } , Ji := { z E J : z 5 x) and J , := { z E J : z 2 x}. Then f [ J ] ,f [ J l ] and f [ J u ] are all intervals that contain j = f ( x ) . Because f is injective, the only point f [ J l ] and f [ J u ]have in common is y . Therefore y is the maximum of one of the intervals f [ J l ] and f [ J u ] and it is the minimum of the other. If both f [ J / ]and f [ J u ]have more than one element, then there is a 6 > 0 so that ( y - 6, y 8) f [ J ] . In this case, because {yn)Z1 converges to y , there is an N E N so that for all n 3 N we have yn E ( y - 6 , y 6) E f [ J ] .Consequently, for all n 2 N the point x n is in J , which means Ixn - x /< E . If f [J,] has exactly one element, then J , has exactly one element and x is the largest point of I . Therefore f [ J / ]has more than one element and there is a 6 > 0
+
+
68
3. Continuous Functions
+
so that ( y - 6 , y ] E f [ J l ] or [ y , y 6) & f [ J l ] . Without loss of generality assume ( y - 6 , y ] E f [ J l ] . We claim that then y is the largest point of f [ I ] . Suppose for a contradiction that there was a y' > y in f [ I ] . Then a := f - ' ( y ' ) < x and there would also be a b < x with f ( b ) < y . By Theorem 3.36 there would be a c # x between a and b so that f ( c ) = y = f ( x ) , contradicting the injectivity of f . Thus y = sup f [ I ] .Because ( y n } z lconverges to y there is an N E N so that for all n 2 N we have yn E ( y - 6, y ] 5 f [ J l ] . Thus for all n 2 N we infer x, E J l , which means
Ix, - x / < E . The case in which f [ J / ]has exactly one element is handled similarly. We have shown that in each of the above cases f - ' lim y,) = lim f - ' ( y n ) , which implies that f
(
r 2 . 3 ~ -
-' is continuous.
,
n.fOg-
Now that we know that inverses of continuous functions are continuous, we can establish limit laws for powers with rational exponents.
Definition 3.39 A number n E Z is called even irthere is a k it is called odd iff there is a k E Z so that n = 2k 1.
+
E
Z so that n = 2k and
Corollary 3.40 Let d E N. Then f ( x ) = x : is continuous on [0,00) i f d is even and it is continuous on JR i f d is odd. Proof. Use Theorem 3.38 (Exercise 3-35). Corollary 3.41 Let r E Q be positive. Then f (x) = x" is continuous on [0,CO), and i f r can be represented as a fraction with odd denominatol; f is continuous on R. For negative r E Q,f ( x ) = x" is continuous on ( 0 ,CO), and i f r can be represented as a fraction with odd denominatol; f is continuous on R \ {O}. Proof. Use Exercise 3-27 and Corollary 3.40 (Exercise 3-36). Theorem 3.42 Let f o r all r
E
lirn Q we have n+cc
be a convergent sequence of nonnegative numbers. Then aI; =
(n+oc lim
Proof. Exercise 3-37. We conclude this section by showing that on closed and bounded intervals continuous functions assume an absolute minimum and an absolute maximum.
Definition 3.43 Let I E R be an interval and let f : I + JR be a real valued function. The number y, is called the absolute minimum value o f f (in I ) if and only ifthere is an x,, E I with f (x,) = y, and f o r all x E I we have f (x) > f (x,). The number y~ is called the absolute maximum value off (in I ) ifand only ifthere is an X M E I with f ( x ~ = ) Y M and for all x E I we have f ( x ) 5 f ( x ~ ) .A value that is the absolute maximum or the absolute minimum is also called an absolute extremum. Theorem 3.44 Let f : [ a ,b] -+ Iw be continuous. Then there is an x that,forall z E [ a ,bl we have f ( x ) 2 f ( z ) .
E
[ a ,b] such
69
3.6. Limits at Infinity
Proof. First we show that f is bounded above on [ a ,b ] . For a contradiction, suppose that for every n E N there is an x, E [a,b ] such that f (x,) 2 n. Then by the Bolzano-Weierstrass Theorem there is a convergent subsequence { xnk},“=, with limit x E [ a ,b ] . But then for all k we have f ( x n k ) L nk while at the same time lim f ( x n k ) = f (x) < 00, which is not possible. k-+ rn
Thus f is bounded above. Let M := sup { f ( z ) : z E [ a , b ] } . For each n E N 1 find x, E [ a ,b] with f (x,) 2 M - -. Again by the Bolzano-Weierstrass Theorem n ~ limit x E [ a ,b ] . For all k we have there is a convergent subsequence { x , , } ~ with 1 A4 - - 5 f ( x n k ) 5 M , which means by the Squeeze Theorem that nk
f (x) = ,lhlf
= M = SUP { f ( z ) : z E [ a ,b l } .
Exercises + b in the Intermediate Value Theorem
3-33. Prove that c
3-34. Prove that i f f : A b E B.
+
B is bijective and g, It : B -+ A are inverses o f f , then g(b) = h ( b ) for all
3-35. Prove Corollary 3.40 3-36. Prove Corollary 3.41 3-37. Prove Theorem 3.42. 3-38. Let f : [ a . b] + B be continuous. Prove that there is an x inequality f ( x ) 5 f ( z ) holds.
E [ a , b] such
that for all
z
E [a. b] the
3-39. Let I 5 W be an interval. Give an example that shows that even if f : I + W is continuous, f [ I ] need not be bounded. Then explain why this example does not contradict Theorem 3.44. 3-40. Let I be an interval and let f : I + R be continuous and injective. Prove that f is either increasing (that is, for all x1 < x 2 we have f ( x 1 ) < f ( x 2 ) ) or decreasing (that is, for all X I < x2 we have f(x1) > f ( x 2 ) ) . 3-41. Although the contrapositive is often used to clarify a mathematical statement, some contrapositives can be quite confusing. Determine if the statement “If f ( n ) 0 for all c E [a, b ] ,then f ( a ) > 0 or f ( b ) 5 0 or f is not continuous on [a, b].” is true or false by analyzing its (much simpler) contrapositive.
+
3.6 Limits at Infinity For functions defined on an interval ( t , 30) it is sensible to investigate the behavior as the argument x gets large. The resulting notion of a limit at infinity is set up in exactly the same way as the other limits discussed in this chapter. Thus it is not surprising that similar laws hold.
Definition 3.45 Let L be a real number and let f : ( t , 00) + R be a function. We say f converges to the limit L at 30 and write lim f ( z ) = L iff for every sequence
{Z,},X=~ in ( t , 30) such that n+m lim
ZO ’D
Z,
= 00 we have lim f (z,) = L. ,--too
70
3. Continuous Functions
Limits at --oo are dejined similarly. For functions with domains that contain intervals ( t , 00) or (--00, t ) the limits at f - 0 0 are dejined as the limits of the appropriate restriction.
Theorem 3.46 Limit laws for limits of functions at -00. Let f , g : ( t , 00) + R be g ( z ) exist. Then the following hold. functions so that limZ+CCf ( z ) and limZ--rCC 1. lirn ( f Z M '
+ g ) ( z ) = lim f ( z ) + lim g ( z ) . Z'CC
Z'CC
2. lim (f - g ) ( z ) = lim f ( z ) - lim g(z). Z'ffi
Z'CC
Z'CC
3. lirn (f . g ) ( z ) = lim f ( z ) . lim g ( z ) . Z'W
Z'CC
Z'CC
Each equation implicitly asserts that and in this case the equality holds.
if the right side exists, so does the left side
Proof. Exercise 3-42. Theorem 3.47 E-M formulation for the limit of a function at infinity. Thefinction f : ( t , 00) -+ R converges to the limit L at infinity ifand only iffor every E > 0 there is an M such that for all z 2 M we have f ( z ) - L < E .
1
I
w
Proof. Exercise 3-43.
Moreover, infinite limits at &00 can be Similar results hold for limits at -a. defined similar to infinite limits at a finite number and similar limit laws hold. By now, the reader is sufficiently familiar with the underlying ideas to formulate these definitions independently (see Exercise 3-44).
Exercises 3-42. Prove Theorem 3.46. (a) Prove part 1 of Theorem 3.46.
(b) Prove part 2 of Theorem 3.46.
( c ) Prove part 3 of Theorem 3.46.
(d) Prove part 4 of Theorem 3.46.
3-43. Prove Theorem 3.47. 3-44. Infinite limits at infinity. (a) State the definition of lim f ( z ) = oc z+m
(b) State limit laws for infinite limits at oc.
(c) State and prove a result similar to Theorem 3.47 for infinite limits at infinity.
3-45. Explain why we do not define left and right limits at infinity.
Chapter 4
Differentiable Functions
Geometrically speaking, differentiable functions have unbroken graphs without corners. This “smoothness” of differentiable functions is useful in applications. In the present chapter differentiability is introduced in Section 4.1, the relation between differentiability and the common operations on functions is considered in Section 4.2, and some geometric consequences of differentiability are provided in Section 4.3.
4.1 Differentiability The derivative provides the slope of the tangent line, if the function has a tangent line at the point. Definition 4.1 encodes this idea by demanding that as we fix one point and move the other one closer to this “base point,” the slopes of the secant lines through these points approach a limit. The left part of Figure 9 shows that this convergence of the slopes means that the secant lines tilt towards a “limiting line,” the tangent line. It should be noted that differentiability typically is considered on open intervals so that secant lines “in both directions” can be used to obtain the limit.
Definition 4.1 A function f : ( a , b ) -+ lim
z+x
-
2--x
R is differentiable at x
exists. In this case we set f ’ ( x ) := lim Z’X
E
( a , b ) zrthe limit
(‘) - f ( x ) and call it the
z-x
d f ( x ) and D f ( x ) . derivative o f f at x. Other notations for the derivative at x are dx Similar to limits offunctions, if D C R, a function f : D --f JR is called differentiable at x E D iff there is an open interval ( a , b) D so that x E ( a ,b ) and so that 71
72
4. Differentiable Functions
f I ( a , b ) is differentiable at x. Moreover, similar to what we did so far, we will mostly work with functions defined on open intervals, trusting that the reader can make the jump to larger domains.
Example 4.2 The following are verified with routine computations (Exercise 4-1). I . At every x
E
R thefunction f (x) = x is dizerentiable with f’(x) = 1.
0
2. The function f (x) = 1x1 is not differentiable at x = 0.
As with continuity, the local property of differentiability can be demanded at every point of the domain to obtain a global definition.
Definition 4.3 Let D E R.Afunction f : D + IR is differentiable, or differentiable on D, iff it is differentiable at every x E D. Continuity at x means that lirn f ( z ) - f(x) = 0. Theorem 4.4 affirms that conZ X ’
vergence of the quotient
(‘) -
z-x
to a number is a stronger condition.
Theorem 4.4 Let f : ( a , b ) + R be dizerentiable at x . Then f is continuous at x. Hence, every differentiable function is continuous. Howevel; not every continuous function is dizerentiable.
Proof. Let f be differentiable at x. Then lim Z X ’
0 = lim(z - x) lim Z’X
Z X ’
f ( z ) - f (x) = lim(z - x) z-x
X’Z
(‘) -
z-x -
z-x
exists. This implies = lim f ( z ) - f(x), Z’X
+
that is, lirn f (z) = lirn (f( z ) - f (x)) lim f (x)= 0 + f (x) = f (x),and f is conZ’X Z’X Z’X tinuous at x. To see that not every continuous function is differentiable, consider f (x) = 1x1, H which is continuous, but it is not differentiable at x = 0 . Ultimately, we will generalize differentiability to higher dimensions. In higher dimensions, division is not possible, but it is possible to define entities that are similar to tangent lines. Theorem 4.5 shows the difference between differentiability at x and continuity at x without using division. A differentiable function f can be approximated by a straight line g ( z ) = f(x) f’(x)(z - x) through (x,f (x)) in such a way that the difference between f and g goes to zero faster than Iz - x 1. Geometrically, this means (see Exercise 4-2a and Figure 9 ( b ) )that, no matter how small the width, near x the differentiable function f will enter all “cones” which are centered at (x,f (x)) and symmetric about the line g . This idea ultimately leads to the definition of differentiability in higher dimensional spaces (see Definition 17.24). For a continuous function f,the difference between f and any straight line through (x,f (x)) goes to zero (see Exercise 4-2c), but the function need not enter arbitrarily narrow cones about the line.
+
4.1. Differentiability
73
(b)
Figure 9: Two ways to view differentiability. In ( a ) , as in Definition 4.1, the slopes of the secant lines through (x,f ( x ) ) and (z, f ( z ) ) approach a number, which is the slope of the tangent line. In ( b ) ,as in Theorem 4.5 and Exercise 4-2a, the function and a certain straight line, the tangent line, are such that for any width E > 0, near x the function will ultimately enter the “cone of width E” about the tangent line.
Theorem4.5 Let f : ( a , b ) + R beafunctionandletx E ( a ,b). Then f isdigerentiable at x iff there is an L E JR so that for every E > 0 there is a 6 > 0 such that for Moreovel; all z f x with Iz - X I < 6 we have I f ( z ) - f ( x ) - L ( z - x)/ < E J Z -XI. in this case f ’ ( x ) = L .
1
2-x
-L <
E.
Multiplying both sides by
8
L = lim Z’X
-
z-x
Iz
-
XI
turns this inequality into
-
and f is differentiable at x with f ’ ( x ) = L .
Exercises 4-1.
(a) Prove that the function f ( x ) = x is differentiable at every x E
W and that f ’ ( x ) = 1
(b) Prove that the function f ( x ) = 1x1 is not differentiable at x = 0.
4-2. Differentiability versus continuity. (a) Let f : (u,b) + R.Prove that f is differentiable at x iff there is an L E > OthereisaS > 0 so that for allz E R with / z - x i < 6 wehave
( A x )+
u z -XI
) - El2 --XI 5 f ( z ) 5
(f(x)
+ L(z - x )
E
W so that for every
) +ElZ
-XI
Also prove that in this case f’(x) = L . Use this result to explain the right part of Figure 9.
14
4. Differentiable Functions (b) Prove that for any m
E
W we have lim Iz/ - mz = 0, even though f ( x )
(c) Let f : ( a , b)
0
’i
tiable at x = 0
W be a function that is continuous at x
i
1 f ( z ) - [ f ( x ) + m(z - x)l I = 0.
J&x
E
= 1x1 is not differen-
W and let m
E
W. Prove that
4-3. For f : ( u , b ) + B and x E ( a , b ) we define the left-sided derivative of f at x via the left limit ~ ’ f ( x:=) lim z’x-
- f ( x ) if the limit exists and we define the right-sided derivative o f f at z-x
x via the right limit D ” f ( x ) := lim f ( z ) <‘X+
z-x
if the limit exists.
(a) Prove that f is differentiable at x iff the left-sided and right-sided derivatives exist at x and D‘f(x) = D‘f(x). (Compare with Standard Proof Technique 3.20.) (b) Prove that for f ( x ) = 1x1 we have that D ‘ f ( 0 ) = 1 and d f ( 0 ) = -1.
4.2 Differentiation Rules The results in this section show how differentiability relates to the algebraic operations for functions as well as to composition. So far, whenever limits were concerned, we worked with estimates that ultimately made certain differences smaller than E . However, we can also use the available limit theorems and we will do so in this section. The following computations reemphasize the fact that facility in symbolic computation remains important in abstract mathematics.
Theorem 4.6 Let the functions f : ( a ,b ) -+ E% and g : ( a ,b ) -+ R both be dzfferentiable at x E ( a , b ) and let c be a real numbei: Then the functions f g , f - g and cf are all differentiable at x and
+
(f
+ g)’(-x)
(f
- g)’(x)
= =
(Cf)’(X)
=
f ’ G )+ g’(x), f’(-x) - g’(-x), Cf’(X).
Proof. We use the definition of the derivative with the goal of extracting the difference quotients for f and g , respectively.
(f
+g)’(x) =
lim ii
x
(f
+ g ) ( z I - ( f + g>(-x) = lim f (2) + g ( z ) - f ( X I - g ( x ) z - X
z+x
z--x
4.2. Differentiation Rules
75
The remainder of the proof is left to the reader as Exercises 4-4 and 4-5.
H
Derivatives are also well-behaved when products, quotients, powers and compositions are involved. We first prove the quotient rule, which is a bit harder than the product rule and we leave the proof of the product rule as an exercise.
Theorem 4.7 Quotient Rule. Let the functions f,g : ( a , b) + R be diTerentiable at x E ( a , b) and let g ( x )
+ 0.
f is. diflerentiable at x with Then the quotient g
Proof. Similar to the proof of Theorem 4.6 we compute the limit of the difference quotients. The computations are a bit more involved than before.
$(z) lim X’Z
- gf ( x )
z-x
4. Differentiable Functions
76
AA=Af
g+Ag. f+AfAg
Figure 10: Visualization of the Product Rule. The growth rate of a rectangle with side lengths f and g can be obtained from the picture above by dividing the formula for
+
g ' f . The term A f A g does A A by At and letting A t go to zero. It is A' = f ' g At not contribute to the rate because in its numerator two quantities are going to zero. The proof of the product rule (Exercise 4-6) makes this idea more precise.
w Theorem 4.8 Product Rule. Let f,g : ( a , b) + R be differentiable at x E ( a , b). Then f g is differentiable at x with ( f g ) ' ( x ) = f ' ( x ) g ( x ) g ' ( x )f ( x ) . (For a visualization, consider Figure 10.)
+
Proof. Exercise 4-6. Now that products and quotients are taken care of, we can consider powers. The Power Rule could have been proved earlier in a more direct fashion, but the present proof is faster.
+
Theorem 4.9 Power Rule. For every integer n 0, thefunction f ( x ) = X" is diyerd entiable with -xn = nxn-' at evely x E R for which the right side is dejined. dx Proof. For n > 0, we use induction on the exponent n. For the base step with d 2-x n = 1, note that -xl = lim -= 1 for all x E R. dx z-xz-x d For the induction step n + ( n + l), we need to prove -x"+' = ( n 1)x" for all dx d x E 1w and we can use the induction hypothesis that -xn = nxn-'. Via the Product dx d d Rule we obtain -xn+' = - ( x . x " ) = 1 . X" nx"-' . x = ( n 1)x". dx dx This establishes the power rule for positive integer exponents n. For any rn E N, we have -rn < 0 and we can differentiate x P m as follows.
+
+
+
4.2. Differentiation Rules
77
ipeed g’(n) in the g-direction
,peed f ’ (g(x) ) g’(x) n the f-direction
,’slope g‘(xj
,‘slope i ‘ ( g ( x 1 )
I
X
*
-
__c
-
“g” c
speed 1 in the x-direction speed g’(x) in the g-direction The output of g becomes the input o f f . Position and speed are preserved.
Figure 11: Visualization of the chain rule. The derivative can also be understood as a magnification factor for speeds. If a particle at x moves at unit speed along the horizontal axis, then its image particle under g moves at speed g’(x) along the vertical axis. Now, if a particle at g ( x ) moves at speed g’(x) along the horizontal axis then its image particle moves at speed f ’(g(x))g’(x)along the vertical axis. The above shows that the power rule holds for all nonzero integers. Now that we have the power rule, the last computation can only be viewed as a clumsy way to compute the derivative of powers with negative exponents. Howevel; in this proof we had no choice, because we were proving the very rule that abbreviates this computation. We will revisit the Power Rule in Theorems 4.22 and 12.10. With algebraic operations taken care of, we turn our attention to composition. Figure 11 shows a kinematic way to explain the Chain Rule. Theorem 4.10 Chain Rule. Let g : ( a , b ) -+ LR and f : ( c ,d ) + R be functions with g [ ( a ,b ) ] ( c ,d ) and let x E ( a , b ) be such that g is dzjterentiable at x and f is dzjterentiable at g ( x ) . Then f o g : ( a , b ) -+ R is dijterentiable at x and the derivative is ( f 0 g)’(x) = f’(g(x))g’(x). Proof. First, consider the case that there is no sequence
{znlzl with ,hhr Zn
and zn # x and g ( z n ) = g ( x ) for all n E W. In this case, we proceed as follows. lim Z X ’
f
0
g(z)- f
z-x
0
g(x)
=x
4. Differentiable Functions
78
=
lim u*g(x)
=
f( u ) - f
. lim
u -g(x)
Z’X
g(z) - g(x) z -x
f’(g(x))g’(x).
{z,},“=, with n+m lirn zn
= x and
Because f is differentiable at g ( x ) , there is a u > 0 so that for all u
# g(x)
This leaves the case in which there is a sequence
zn # x and g ( z n ) = g ( x ) for all n
with 1u - g(x)l < u we have
1
E
N.In this case,
- g(x)
reverse triangular inequality) ’(’) E
- f l ( s ( x ) ) l < 1, and hence (by the
1
I
+ 1. Moreover, for all
- (g(x)) < f’(g(x))i - g(x) 0 there is a 6 > 0 so’ that for all z f ’ x with Iz - X &
If’ ( g ( x ) )1 + 1
I
< 6 we have that
(because g’(x) = 0) and / g ( z ) - g(x)l < u (by
Theorem 4.4). Formally, we would need tofind a 6 so that thefirst inequalig holds and another so that the second one holds and then use the minimum of the two. We used a simple modification of Standard Proof Technique 2.6 to abbreviate this step. Therefore for all z # x with Iz - x / < 6 we obtain the following. In case g ( z ) = g ( x ) , we have g(z) g ( x ) = 0. In case g ( z ) # g ( x ) ,we have I g ( z ) - g ( x ) / < v , and hence z-x
Because E was arbitrary we conclude
and the proof is complete.
4.2. Differentiation Rules
79
With compositions taken care of, it would be natural to also consider inverse functions. Because we need Rolle's Theorem to dispose of a technicality, consideration of inverse functions is postponed to Theorem 4.2 1.
Example 4.11 Maxima and minima do not preserve diyerentiability. Even though f (x) = x and g ( x ) = -x are both differentiable, the functions max{f , g } ( x ) = 1x1 and min{f , g ] ( x ) = --/XI are not differentiable at 0. Thus, while differentiability is compatible with the natural algebraic operations for functions, it is not compatible with the natural order-theoretical operations for functions. 0 If a function f : ( a , b ) -+ R is differentiable on ( a , b ) , then the derivative f' is a function in its own right. Hence, we can consider continuity and differentiability for
.f I . Definition 4.12 Let D R and let f : D -+ R be afunction. Then f is called continuously differentiable iff it is diyerentiable on D and the derivative f ' : D + R is continuous. The function f is also often considered to be its own "zeroth derivative" f (1' := f . The function f is called n times differentiable i y i t is n - 1 times derivative f (n-l) : D + R is d8erentiable. The nth diTerentiable and its (n d d" derivative o f f is f ( " ' ( x ) := -f("")(x), also denoted -f . The function f is dx dxn called n times continuously differentiable iff it is n times differentiable and its nth derivative f ( n ) : D + R is continuous. Finally, f is called infinitely differentiable iy fo r all n E N it is n times differentiable. The differentiation rules we have derived here immediately show the following.
Example 4.13 Polynomials and rational functions are infinitely diyerentiable on their
0
domains.
Example 4.14 For every n E N, the function f (x)= times continuously differentiable, but it is not ( n + 1)-times differentiable. (Exercise
0
4-7.)
Exercises 4-4. Prove that if f,g : ( a , b ) -+ R are both differentiable at x E ( a , b ) , then f - g is differentiable at x and (f - g)'(xj = f ' ( x ) - g'(x). 4-5. Prove that if f : ( a , b ) -+ and ( c f ) ' ( x ) = c f ' ( x ) .
W is differentiable at x
E ( a , b ) and c E
W,then cf is differentiable at x
4-6. Prove the product rule. Hint. It's similar to the proof of the quotient rule, but simpler. 4-7. Prove the claim in Example 4.14. Hint. Use induction and at x = 0 use Exercise 4-3.
4-8. Prove that f : ( u , b j + 5s is differentiable at x E ( a . b ) iff the limit lim and that in this case f ' ( x ) = lim h-0
f(x
+h) - f(x) h
h+O
f(x
+hj - f(xj h
exists
80
4. Differentiable Functions
4-9. Let n E N. Use the Binomial Theorem and Exercise 4-8 to prove the Power Rule for f ( x ) = x" without using induction.
4-10. Prove that the derivative of f ( x ) = f i is f ' ( x ) =
-.
1
2 f i 4- 1 1. Compute the derivative of each of the following functions.
(b) f ( x ) = (x2
+ 5>1 d
x (You may use Exercise 4-10.)
(c) f ( x ) =
4-12. Use induction
(-l)"n!x-"-'
4-13. Prove that if f , g : (a,b ) + differentiable and ( f g ) ( " ) =
W are both
2 (;>
for all x
# 0.
n times differentiable, then the product f g is n times
f(kIg(n-k),
k=O
Hznr. Mimic the proof of the Binomial Theorem.
4.3 Rolle's Theorem and the Mean Value Theorem One of the main applications of derivatives is to use the sign of the derivative to compute relative extrema of a function and intervals where a function is increasing or decreasing. The formal justification follows from Rolle's Theorem and the Mean Value Theorem.
Definition 4.15 Let f : ( a ,b ) + R be a function. Then f is said to have a relative (or local) minimum at x, @there is a 6 > 0 such that f (x,) 5 f ( x )for all x E (x, - 6 , x, + 6 ) . f is said to have a relative (or local) maximum at x~ ifand only ifthere is a 6 > 0 such that f ( X M ) 2 f (x)for all x E ( x -~6, x~ 6). I f f has a local maximum or a local minimum at the point c we also say that f has a relative (or local) extremum at c.
+
Intuitively, relative extrema are the locally highest or lowest points of the graph. (Note, however, that stagnation also is possible, see Exercise 4-14.) At the location of a relative maximum there cannot be an incline in any direction. Hence, the derivative should be zero at a relative maximum.
Theorem 4.16 Let f : ( u , b ) + R be a function and let m E ( a ,b). I f f is differentiable at m and f has a relative maximum at m, then f ' ( m ) = 0. Proof. Because f has a relative maximum at m there is a positive number 6 so that f (z) - f (m) f ( z ) - f ( m ) 5 0 for all z with Iz - ml < 6. We infer that lim 5 0 z+m+ z -m and lim z+m-
-
z-m
( m ) 3 0. Because f is differentiable at m , these two limits must
4.3. Rolle’s Theorem and the Mean Value Theorem
rn
(a) a
b
-+-T--+
81
b
(b) a
Figure 12: Rolle’s Theorem states that if a differentiable function starts and ends at the same height, then it must have a flat tangent in between ( a ) . The Mean Value Theorem states that some tangent must be parallel to the secant through the starting point and the ending point ( b ) . be equal to f’(rn). Therefore f ’ ( m ) is greater than or equal to 0 and smaller than or equal to 0. This implies f ’ ( m ) = 0. Rolle’s Theorem states that if a function’s values are equal at the endpoints of an interval, then the function must have a horizontal tangent line in the interval (see Figure 12). Note that the proof is a collection of direct proofs (modus ponens), which is possible because we can rely on strong results that we proved earlier.
Theorem 4.17 Rolle’s Theorem. Let f : [ a , b] + Iw be differentiable on the open interval ( a , b ) and continuous on the closed interval [ a ,b]. I f f ( a ) = f (b),then there is an m E ( a , b ) with f ’ ( m ) = 0. Proof. Because the result is trivial if f is constant, we can assume f is not constant. By Theorem 3.44, f assumes an absolute maximum and an absolute minimum on [ a ,b ] . Because f is not constant, one of these values is not equal to f ( a ) . Assume without loss of generality that the absolute maximum value is greater than f ( a ) . Let m E [ a ,b] be so that f ( m ) 1: f ( x ) for all x E [a, b ] . Because f ( m ) > f ( a ) = f ( b ) we infer m E ( a , b ) . Hence, f is differentiable at m. Because f ( m ) 2 f ( x ) for all x E ( a , b ) , f has a relative maximum at m . Now by Theorem 4.16 we conclude f ’ ( m ) = 0. The Mean Value Theorem generalizes Rolle’s Theorem by no longer demanding that the values at the endpoints are equal. It guarantees that there is a point in the interval so that the tangent line at that point is parallel to the secant line through the endpoints. In the proof, we subtract the line I(x) = (x - a ) f ( b ) - f ( a ) from the b-a function f . This reduces the proof to an application of Rolle’s Theorem.
Theorem 4.18 Mean Value Theorem. Let f : [a, b] + R be differentiable on the open interval ( a , b ) and continuous on the closed interval [ a ,b]. Then there is a numf ( b )- f ( a ) ber c E ( a , b ) so that = f’(c). b-a Proof. The function g(x) := f ( x ) - ( x - a ) f ( b ) is continuous on [ a ,b ] , b-a differentiable on ( a , b ) and g ( b ) = f ( a ) = g(a). By Rolle’s Theorem there is a c in the
82
4. Differen tiable Functions s-’/
ieflecr across the diagonal
c
I
’
I’
?
c
x
*
I
Figure 13: Visualization of Theorem 4.21. Reflection across the diagonal produces the graph of the inverse function and the slope of the reflected line is the multiplicative inverse of the slope of the original line.
interval ( a , b ) such that g’(c) = 0. But that means f ’ ( c ) that is, f ’(c) =
f (b)- f (a) b-a
f ( b )- f ( a ) / = g ( c ) = 0, b-a
‘
With the Mean Value Theorem available we can now prove a sufficient criterion for when a function is increasing or decreasing.
Definition 4.19 Let I 2 R be an interval and let f : I + R be a function. Then f is called (strictly) increasing on I if and only if for all x1 < x2 in I we have f ( x l ) < f ( ~ 2 ) f. is called (strictly) decreasing on I ifand only iffor all x1 < x2 in I we have f ( x 1 ) > f ( x 2 ) . Moreovel; f is called nondecreasing on I if and only iffor all x1 < x2 in I we have f ( x l ) 5 f (x2) and f is called nonincreasing on I ifand only iffor all x1 < x2 in I we have f ( X I ) 2 f ( ~ 2 ) . Theorem 4.20 Let f : [ a ,b] + R be differentiable on the open interval ( a , b ) and continuous on the closed interval [ a ,b]. If f ’ ( x ) > 0 for all x E ( a , b), then f is increasing on [ a ,b]. Proof. Let f ’ ( x ) > 0 for all x in ( a , b ) . Suppose, for a contradiction, that there are points X I ,x2 in [ a ,b] such that X I < x2 and f ( X I ) 2 f (x2). Then by the Mean Value Theorem there is a c in the interval ( X I , x2) (and therefore in ( a , b ) ) such that f’(c)=
(x2)x2 - X I
<
0, which is a contradiction.
w
Exercise 4-2 1 provides a similar criterion for nonincreasing and nondecreasing functions. Rolle’s Theorem also enables us to prove that the inverses of differentiable functions are differentiable. For a visualization, consider Figure 13.
4.3. Rolle's Theorem and the Mean Value Theorem
83
-'
Theorem 4.21 Let f : ( a , b ) + R be a continuous injective function, let f be its inverse and let x E f [ ( a ,b ) ] be such that f is differentiable at f - ' ( x ) with 1 f ' ( f - I ( x ) ) f 0. Then f is dgerentiable at x and ( f - I ) ' ( x ) =
-'
f' (f- 1
(XI)
'
Proof. By Theorem 3.36 f [ ( a , b ) ] is an interval and by Theorem 4.16 x is neither its maximum nor its minimum. Thus there is a 6 > 0 so that the containment ( x - 6, x 6) f [ ( a ,b ) ] holds. Hence, we can talk about differentiability of the function f - ' at x. Let be a sequence in f [ ( a , b)]\ { x } so that lirn zn = x . By Definition 3.1,
+
{zn}zl
n-m
f-'(zn) - f-'(x>
1
-
. For each f ' (f-'w) and let y = f - ' ( x ) . Then yn = f-'(z,) f f - ' ( x ) = y for
we are done if we can show that lim n-m
Zn - X
n E M let yn := f - ' ( z , ) all n E M.By assumption f is continuous, and hence by Theorem 3.38 so is f - ' . This means that lim yn = lirn f-'(z,) = f - ' lirn zn = f ( x ) = y , and therefore n-+m
(n-+m
n-m
=
lim n-Q3
-
Yn - Y
f(yn)-f(y)
1
= lim n+m
1 f(Yn)-f(Y) Yn -Y
1
-f'(y) f'(f-'(x>)
.
Theorem 4.21 allows us to extend the Power Rule to rational exponents.
Theorem 4.22 Power Rule. Let r E Q \ {O}. Then f ( x ) = x' is differentiable with d -Xr = rxr-1 at every x for which the right side is defined. dx Proof. Let m E N and consider f ( x ) = x i . Then f is the inverse function of g ( x ) = x m . By Theorem 4.21 for all nonzero x E R for which x i is defined, we
Now let n E Zand m
.
E
,
M. Bv the Chain Rule. for all nonzero x for which x t is
Exercises 4-14. Geometry versus utility in the definition of relative extrema. (a) Explain why by Definition 4.15 every point in the interval (0. 1) is a relative maximum and a for x 5 0, relative minimum of the function f ( x ) = for x E [O. I], ( x - 113; f o r x > 1.
84
4. Differentiable Functions (b) Sketch the graph of the function and comment whether intuition agrees that f has a relative maximum and a relative minimum at each x E (0, 1). (c) Use the proof of Rolle’s Theorem to explain why the definition of relative maxima as in Definition 4.15 is preferable to a definition of relative maxima that requires the value at a relative maximum to be strictly larger than the values of the function near the relative maximum.
f
4-15. Prove that if f ‘ ( m ) = 0.
: ( a , 6) -+
R is differentiable and has a relative minimum at m
E ( a , b ) , then
4-16. Let f : [a, b] + W be differentiable on the open interval (a, b ) and continuous on the closed interval [ a , b]. Prove that if f ’ ( x ) i0 for all x E ( u , b ) , then f is decreasing on [ u , b ] . 4-17. Let f : [ u , b] -+ I 3be differentiable on the open interval ( u , b ) and continuous on the closed interval [ a , b ] . Prove that if f ’ ( x ) = 0 for all x E (a, b), then f is constant on [a, b]. 4-18. Give a direct proof of Theorem 4.20. That is, give a proof that does not argue via contradiction 4-19. Let f ( x ) := u x 2 decreasing on
+ bx + c be a quadratic function defined on R. Prove that if u
(
-m.
3
--
G
a
State and prove a similar result for a < 0.
+
> 0 then
f is
)
- -, m .
and increasing on
+
+ +
4-20. Let f ( x ) := ax3 bx2 c x d be a cubic function defined on W with u 0. Prove that if 4b2 - 12uc > 0, then f has two relative extrema. For a > 0 and for a < 0 separately describe where f is increasing and where it is decreasing. 4-21. Let f : [ a , b] + R be differentiable on (a, b ) and continuous on [ a , b ] . (a) Prove that f is nondecreasing iff for all x equivalence.)
E ( a , b ) we
have f ’ ( x ) 2 0. (Note that this is an
(b) Prove that f is nonincreasing iff for all x E ( a , b ) we have f ’ ( x ) 5 0. (Note that this is an equivalence.) (c) Give an example that shows that the condition in Theorem 4.20 is not equivalent to f being strictly increasing. 4-22. Use Theorem 4.21 to prove that the derivative of f ( x ) = f i is f ’ ( x ) =
-.
1
2JT;
4-23. Explain with the following is not a proof for Theorem 4.21.
“Proof.” We know that x = f (f-’( x ) ) . Differentiating both sides with respect to x gives d
1= -x
dx
=
d
-f
= f ’ ( f - ’ ( x ) ) ( f - ’ ) ’ ( x ) , so ( f - ’ ) ’ ( x ) =
dx
1
f‘ (f -1( X I )
4-24. Let f : ( a . b ) + iW be continuous and let x E ( a , b). Prove that if f ’ ( z ) exists for all z and lim f ’ ( z ) exists, then f is differentiable at x with f ’ ( x ) = lim f ’ ( z ) .
E
?? ‘
( a , b) \ ( x )
Z’X
Z’X
4-25. Let f : ( a , b ) + W be differentiable. Prove that f ’ (which need not be continuous) has the intermediate value property. That is, prove that for all c < d in (a, b ) and all u between f ’ ( c ) and f’(d) there is an m E (c. d ) so that .f’(m)= u .
{
r;
and h ( x ) := f o r x i d , are for x = c, forx = d, continuous on [c, d ] , that g ( d ) = h ( c ) , apply the Intermediate ValueTheorem to one of them, then apply the Mean Value Theorem to f.
Hinr. Show that g ( x ) :=
4-26. Prove that for n 2 2 the nth derivative of f ( x ) = & is -Zn-l (2n - 2)! f ‘ n ’ ( x ) = (-1)”+l x 2 = (-,)n+’ 2n-l(n - 1)!2n
1.3.5...(2n-3) 2”
x
-2n-I 2
4-27. For each function, find an expression for f ( n ) ( x ) and prove that it is the nth derivative o f f I
(a) f ( x ) =
-
fi
(b) f ( x ) =
%
Chapter 5
The Riernann Integral I
The geometric goal of integration is to compute the area under a graph. In Riemann integration this is done by approximating the area with rectangles. This chapter presents the idea behind Riemann integration and some integration criteria, examples and theorems. Although intuitively the Riemann integral seems to be the right idea, by the end of the chapter we will need more machinery to fully characterize Riemann integrability and we will also have exposed some key weaknesses. An equivalent criterion for Riemann integrability will be presented in Theorem 8.12. The observed weaknesses of the Riemann integral will be addressed in Chapter 9.
5.1 Riemann Sums and the Integral To define the area of rectangles under the graph of a function, we first need to determine the base for each rectangle. This is done with a partition of the interval.
Definition 5.1 Let [ a ,b ] be a closed interval. Then anyjnite set P [a,b ] such that a , b E P will be called a partition of [ a ,b]. Because the order of the points will be important, we also write P = {a = xo < xi < . ' < xn = b } when working with a partition P . 1
With the partition giving the bases of the rectangles, we still need to determine the heights. Each height will be a value that the function assumes within the respective interval of the partition (see Figure 14(a)).
Definition 5.2 Let [ a ,b ] be a closed interval and let f : [ a ,b ] -+ R be bounded. For any partition P = {a = xo < x1 i. . . < xn = b } a set T = { t l , . . . , t n ) such that f o r all i E { 1, . . . , n } we have that ti E [xi-l,xi] will be called an evaluation set. We 85
5. The Riemann Integral I
86 Riemann sum
“upper sum”
“lower sum”
Figure 14: The Riemann integral approximates the area under the graph of a function with the areas of rectangles. Definition 5.2 demands that the height of each rectangle is the value of the function at some point in the base interval (one point marked in ( a ) ) . Of particular importance are the lower sum (see Definition 5.13) of the areas of the largest rectangles that can be fit under the graph of the function (b) and the upper sum (see Definition 5.13) of the areas of the smallest rectangles that contain the graph of the function ( c ) . dejine the Riemann sum off with respect to the partition P and evaluation set T to be
c n
R ( f , P , T ) :=
f (tj)(xi - xi-1). We will also use the notation Axi := xi - X i - 1 .
i=l
Clearly, a Riemann sum can only accidentally be equal to the area under the graph. However, the narrower we make the rectangles, the closer the Riemann sums should be to the actual area. The norm of a partition gives a uniform measure of how narrow the rectangles in a partition are.
Definition 5.3 Let [ a ,b ] be a closed interval in the real numbers. For a partition P = { a = xo ix1 i’ . . < x, = b ) , we define the norm of the partition P to be 11 P / / := max { (xi - xi-1) : i = I , . . . , n } . We now say that a function is Riemann integrable iff all Riemann sums get close to one value, the integral, as the norm of the partitions is made small.
Definition 5.4 Let D be a set. A function f : D -+ E% is called bounded iff there is an M E R so that f (x) 5 M f o r all x E D.
1
I
Definition 5.5 The function f : [ a ,b] + R is called Riemann integrable (on [ a ,b ] ) i f f f is bounded and there is a number I such that f o r all E > 0 there is a 6 > 0 so that f o r all partitions P with 11 PI1 < 6 and all evaluation sets T the inequality
c
f(ti)Axj - I
i=l
i
I
= IR( f , P , T ) - I <
E
holds. The number I will be called the
Riemann integral off and it will also be denoted
I”
f (x) d x := I . The function f
is also called the integrand and a and b are called the bounds of the integral, with a being the lower bound and b being the upper bound.
5.1. Riemann Sums and the Integral
87
The definition of the Riemann integral allows us to compute the value of the integral, if it exists, as a limit of a sequence of Riemann sums. Note that in Lemma 5.6 below the only condition on the partitions is that their norm goes to zero and that we are free to choose the evaluation sets any way we want to.
Lemma 5.6 Let f : [ a ,b] +- R be Riemann integrable. Then for any sequence [ Pk } E1 of partitions of [a,b] with lirn (1 pk I( = 0 and any associated sequence of k-tw
b
evahation Sets { T k ) E l we have lim R( f, pk, Tk) = k+
03
f (x)dx.
Proof. Let [ p k } E l be a sequence of partitions of [ a ,b] with lim IIPkII = 0, let k - t co
[ T k ) E l be an associated sequence of evaluation sets and let E > 0. There is a number 6 > 0 so that for all partitions P of [ a ,b] with 11 P 11 < 6 and any associated evaluation set T we have R ( f , P , T ) -
Ib -=I
f (x)dx < &. Because lim I/Pk 11 = 0 there is a
K E W so that for all k 2 K we have IIPkII
/ R ( f $ pk, Tk) -
lb
f (x)dxl
k+w
6. Hence, for all k 2 K we infer that
rn
< &.
The idea in Lemma 5.6 is very useful for numerical integration (see Exercise 1325) and for the proof of the Fundamental Theorem of Calculus (see Theorem 5.23). To demonstrate how Lemma 5.6 is applied, consider the following example.
I'
1 x dx = -. 2 If f is Riemann integrable, then by Lemma 5.6 for any sequence {Pn}F=l of par-
Example 5.7 If f (x) = x is Riemann integrable on [0, 11, then titions with evaluation sets Tn we have lim R(f, P, :=
{1 j n
:
n-tw
= O , . . . , n } andT, :=
=
-
lim
n-tw
{f
:j = 1
-Cj 1
n2
j=1
I n lim - - ( n + a2 2
n-tw
f ( x ) d x . We choose
1
n2+n 1) = lim -n-tco 2n2 2'
Example 5.7 exhibits a key problem that we will ultimately resolve in Theorem 8.12. Although we may be able to compute a value that should be the Riemann integral, it may not be clear if the function actually is Riemann integrable.
88
5. The Riemann Integral I
The following results show that it is reasonably simple to prove Riemann integrability if the value of the integral is known. Note that the strong similarity between Theorem 5.8 and the appropriate parts of Theorems 2.14 and 3.10 is not accidental. The definition of the Riemann integral is similar to the definitions of the limits of sequences and functions. Hence, similar “limit laws” hold and they are proved with similar methods. Unfortunately the similarity does not apply to integrals of products and quotients. (Products are addressed in Exercises 5-21 and 5-22 and quotients are typically treated as products in integration.)
Theorem 5.8 Let f , g : [ a ,b] + R be Riemann integrable and let c and cf are Riemann integrable and the following equations hold. 1.
2.
lb(f + lb Ib g)(x) d x =
cf ( x )d x = c
b
f
( x ) dx
+
l
E
R. Then f
+g
b
g ( x )d x .
f (x) dx.
+
Proof. We will only prove that f g is Riemann integrable and that equation 1 holds, leaving the rest to Exercise 5-3. Let E > 0. Then there is a 6 > 0 so that for all partitions P of [ a ,b] with I/P 11 < 6 and all associated evaluation sets T we have
12
f (ti)&
-
i=l
Ib
f ( x ) dxi <
4
and , k g ( t i ) A x i i=l
for all partitions P of [ u , bl with II P I/ < 6 and all associated evaluation sets T we obtain
<
& & - + - = &
2
2
By Definition 5.5, this implies that f
+ g is Riemann integrable and the equation
The prototypes with which we approximate the area under functions are rectangles, that is, areas under step functions. Therefore it is only natural that step functions should be Riemann integrable and that their integrals should be the sum of the areas of the rectangles involved. Definition 5.9 Let M be a set and let S M be a subset. Then the indicator function 1; f o r x E S, of S is thefunction ls(x) := 0; f o r x # S.
5.1. Riemann Sums and the Integral
89
Proposition 5.10 Let a < b and let c , d integrable over [ a , b ] and
Proof. Let
E
Lb
E
[ a , b ] with c < d. Then f [ c , d ) is Riemann
l [ c , d ) dx = d - c.
> 0, let 6 := min
{ i,T},
let P = {a = xo <
be a partition with II P )I < 6 and let T =
{tl,
...
< x, = b }
. . . , t , } be an associated evaluation set.
d-c
Because 6 5 -, there are j , k E 11,. . . , n } with j i k so that l r c , d ) ( t i ) = 1 iff 3 i E { j ,j 1 , . . . , k } and l f c , d ) ( t i = ) 0 otherwise. Then c E ( t j - 1 , t j ] G [ x j - 2 , x j ] , Or, if j = 1, C E [Xo, t i ] 5 [ X j - l , X j ] . Similarly, d E [ x k - I , x k + l ] , or, if k = n ,
+
d
E
[x,-l,
x,].
Either way,
&
IXj-1
- c J < - and IXk 2
- dl <
&
-. Hence, 2
I
I i=i -&-I)-
By Definition 5.5, l
(d - c )
~ ~ ,isd Riemann )
integrable and the integral is (d - c).
rn
Standard Proof Technique 5.11 A sum of differences as in the proof of Proposition 5.10, for which the negative part of one term cancels the positive part of the previous or the next term, is also called a telescoping sum. Telescoping sums are typically used to collapse sums of many terms into shorter sums (see the proof of Theorem 5.23), or, conversely (see, for example, the proofs of Theorems 13.14 and 17.33) to make a connection between two terms using differences that are easier to work with. Proposition 5.12 Let Q := { and f o r k
and
E
11, . . . , m ) let
lb
eakl[zk-I,zk) k=l
a =
ak E
zo
<
z1
< “m
<
zm
= b } be a partition of [ a ,b ]
R. Then f := ~ a k l ~ z k - , ,isi kRiemann ) integrable k= 1
in
dx = z a k ( Z k - z k - 1 ) . k=l
Proof. Combine Theorem 5.8 and Proposition 5.10 (Exercise 5-5).
rn
Exercises 5-1. Prove that Lemma 5.6 can be turned into a biconditional. That is, let f : [ a , b] + W be a bounded function and prove that f is Riemann integrable iff there is an I E R so that for every sequence of partitions of [a, b] with lim llPk 11 = 0 and any associated sequence of evaluation sets
(&)p=,
k+cc
5. The Riemann Integral I
90 5-2. Prove each of the following
(a) If the function f ( x ) = x 2 is Riemann integrable on [0, I], then (b) If the function f ( x ) = x3 is Riemann integrable on [0, I], then (c) If the function f ( x ) = x4 is Riemann integrable on [0, 11, then
s,I
x2 dx =
6'
1 3
-. 1
x3 d x = -. 4
s,I
x 4 dx =
1 5
-.
Hint Use summation formulas from Exercise 1-33. 5-3. Prove part 2 of Theorem 5.8. m
5-4. Prove that if fl , . . . , fm : [ a , b]
--f
R are Riemann integrable on
k=l
lb5
integrable on [a. b] and
fk is Riemann
[a, b ] , then
m
k=l
b
fk(x) dx
fk(x)dx = k=l
5-5. Prove Proposition 5.12. You may use the result of Exercise 5-4. 5-6. Let f,g : [ u ,b]
l (f lb
+
b
- g ) ( x )dx =
R be Riemann integrable. Prove that f
l
b
f ( x )d x -
l
-
g is Riemann integrable and
b
g(x) d x .
5-7. Let a < 6 and let c, d E [ a , b] with c < d . Prove that l[,,d] is Riemann integrable over [ a , b] and l[,,d]
dx = d
- C.
5-8. Prove that if c is a constant, then f c ( x ) := c is Riemann integrable and 5-9. Let f : [ a , 61 + R be Riemann integrable and let m , M E m 5 f ( x ) 5 M . Prove that m(6 - a ) 5
s,"
1"
fc(x) dx = c ( b - a ) .
W be such that for all x
E [ a , b] we
have
f ( x ) dx 5 M ( b - a ) .
5-10. In the definitions of the Riemann integral and of Riemann sums we demand that the function is bounded. We could define Riemann sums for unbounded functions also, but this exercise shows that the Riemann integral cannot be defined for unbounded functions. Let f : [a,b] --f R be a function. Prove that if there is a number I such that for all E > 0 there is a S > 0 so that for all partitions P with 11 P // < S and all associated evaluation sets T we have R(f,P. T ) - I < E , then f must be bounded. Hint. Assume the function is not bounded and choose the right partitions and evaluation sets to obtain a contradiction to Riemann integrability.
1
1
f : [a. b] + 4 be a function. Prove that if for all z 0 there is a S z 0 so that for all partitions P ,Q with I/ P 11, 11 Q I/ < S and all associated evaluation sets T , U we have 1 R(f,P ,T ) - R(f,Q , U ) 1 < E . then f must be Riemann integrable.
5-1 1. Cauchy Criterion for Riemann integrability. Let E
5-12, Riemann-Stieltjes integrals. Let [ u ,61 be a closed interval, let g : [ a , 61 + W be nondecreasing and let f : [a, 61 -+ W be bounded. For any partition P = (a = xg < x1 < . . . < xn = b) and associated evaluation set T = ( t l , . . . , t n ) , define the Riemann-Stieltjes sum of f with respect to n
the partition P,evaluation set T and integrator g as S,(f, P,T ) :=
f ( t j ) ( g ( x i )- g(xi-1)
).
i=l
We will also use the notation Agi := g ( x j ) - g ( x j - 1 ) . The integrand f is called Riemann-Stieltjes integrable on [ a , 61 with respect to the integrator g iff f is bounded and there is a number I such that for all E > 0 there is a 6 z 0 so that for all partitions P with lIPl1 < S and all associated evaluation sets T we have S g ( f , P ,T ) - I < E .
1
The number I will be called the Riemann-Stieltjes integral of f ,denoted
1
lb
f dg
:= I .
5.2. Uniform Continuity and Integrability of Continuous Functions
91
(a) Prove that if fl , f2 are Riemann-Stieltjes integrable with respect to g, then f1 +f2 is Riemann-
Jd
Stieltjes integrable with respect to g and
b
(fl
Jdb
+ f2) dg =
Jd
+
fi dg
(b) Prove that if f is Riemann-Stieltjes integrable with respect to g and c Riemann-Stieltjes integrable with respect to g and
Sb
(c) Prove that if g : [0, 11 + R is the indicator function 1
not Riemann-Stieltjes integrable with respect to g .
cf dg = c
Jd f
b
E
f2 dg.
R,then
cf is
b
dg.
[ 1 4 , then the function f
=1
5.2 Uniform Continuity and Integrability of Continuous Functions Unlike continuity and differentiability, Riemann integrability is not easily checked. Therefore, we need to invest some effort into finding criteria for Riemann integrability that are simpler than the definition. The first such criterion is that continuity implies Riemann integrability. The fundamental idea behind the Squeeze Theorem (see Theorem 2.21) is that if a sequence is trapped between two other sequences that converge to the same limit, then the sequence in the middle must converge to that limit, too. To use this idea in our investigation of Riemann integrability, we define lower and upper bounds for all Riemann sums associated with a given partition.
Definition 5.13 Let f : [ a ,b] -+ R be bounded, let P = { a = xg < a partition of [ a ,b] and for each i E (1, . . . , n } let mi
:= inf { f (x) : x
E [xi-l,
x;]}
Mi := sup { f (x) : x
and
. . . < x,
E [xi-l,
c
= b } be
xi]}
n
We define the lower sum o f f with respect to P to be L ( f , P ) := n
define the upper sum o f f with respect to P to be U (f , P ) :=
mi Ax; and we
i=l
M ; Ax;. i=l
Because the supremum and the infimum in Definition 5.13 need not be assumed by
f,lower and upper sums need not be Riemann sums of f . However, the next result shows that the Riemann sums are "trapped" between appropriate upper and lower sums.
Lemma 5.14 Let f : [ a ,b] -+ R be bounded and let P = { a = xg,. . . , x, = b } be a partition of [ a ,b]. Then f o r all associated evaluation sets T = { t l , . . . , t n } the inequalities L ( f , P ) 5 R( f , P , T ) L U ( f , P ) hold. Proof. Because ti
E [xi-l, x i ] n
for all i
E
n
all i E (1, . . . , n ) . Hence, c m i Axi p i=l
11, . . . , n } we have mi 5 f ( t ; ) 5 Mi for
f (tj)Axi 5 i=l
c n
Mi Axi, as claimed.
i=l
To prove that a function is Riemann integrable, we would need to show that lower and upper sums get closer to a certain limit as the norm of the partition tends to zero. The following monotonicity properties are a first step in this direction.
92
5. The Riemann Integral I
Definition 5.15 Let P , Q be partitions of [ a ,b]. Then Q is called a refinement of P ifSP C Q. Lemma 5.16 Let f : [ a ,b] -+ E% be bounded and let P , Q be partitions of [ a ,b ] so that Q is a rejinement of P . Then L ( f , P ) f L ( f , Q ) f U (f , Q ) f U (f , P ) . Proof. From Lemma 5.14, we know that L ( f , Q) f U (f , Q). Therefore we can focus on the other two inequalities. Let {zl,. . . , z k } := Q \ P . Then with Qo := P and Q; := Qj-1 U { z j ) for j = 1, . . . , k we have that each Q j is a refinement of Q;-1 and Q k = Q. Thus we are done if we can prove the outer inequalities for Q = P U { z ] for some z E [ a ,b ] \ P . We will only prove L ( f , P ) 5 L ( f , Q ) , leaving the other inequality to Exercise 5-13. Let P = { a = xo < x1 < . . . -= xn = b } and let j E { 1, . . . , n } be such that z E [x;-1,xj].Then
Lemma 5.17 Let f : [ a ,b] + R be bounded and let P , Q be partitions of [ a ,b]. Then L ( f , Q ) 5 U ( f , PI. Proof. Because P U Q is a refinement of P and of Q , the proof follows from Lemma 5.16: L ( f , Q ) I L(f , P U Q ) I U (f , P U Q ) 5 U (f , P ) . Turning to continuous functions, we note that continuity guarantees that for every point x and every E > 0, there is an interval (x - 6,x 6) so that the inequality sup{ f ( z ) : z E (x - 6 , x 6 ) ) - inf { f ( z ) : z E (x - 6 , x 6 ) ) < E holds. With this difference expected to be small, the difference between upper and lower sums of continuous functions should also become small as the partitions become finer. Unfortunately, while the norm of a partition is given for the whole interval, the S may vary from point to point. To prove that continuous functions are Riemann integrable, we need to show that for any E > 0 we can choose a 6 as above that works at every point of the closed interval [ a ,b ] . This is the idea behind uniform continuity.
+
+
+
Definition 5.18 Let J C R be an interval. The function f : J + R is called uniformly continuous zrfor every E > 0 there is a 6 > 0 such that f o r all u . u E J with Iu - U I < 6 we have I f ( u ) - f ( u ) l < E.
93
5.2. Uniform Continuity and Integrability of Continuous Functions
Lemma 5.19 Let f : [ a ,b] -+ IR be continuous. Then f is uniformly continuous. Proof. Suppose for a contradiction that f is not uniformly continuous. Then there is an E > 0 such that for all 6 > 0 there are u , u E [a,b] such that ( u - U I < 6 1 and l f ( u ) - f ( u ) / >_ E . Therefore, for all k E N with 6 k := - we can find numbers k 1 U k , U k E [ a ,b] so that Iuk - u k / < - and / f ( u k ) - f ( u k ) l 2 E . By the Bolzanok Weierstrass Theorem, the bounded sequence { u k } L , has a subsequence { uk, that 1 the inequality Iuk, - Uk, < converges to a t E [ a ,b ] . Because for all m E km holds, we conclude that lim U k , = lim U k , = t . But then, because f is continuous,
1
m+co
lim f
m+m
(uk,)
m-co
= f ( t ) = lim f (uk,) , which means lim I f (uk,)
a contradiction to
/
m+co f( u k ) - f ( u k )
m-co
-
f (uk,)l
= 0,
I 2 E for all natural numbers k .
Uniform continuity was the last piece in the puzzle to establish that continuous functions on closed and bounded intervals are Riemann integrable.
Theorem 5.20 Let f : [ a ,b] -+ R be continuous. Then f is Riemann integrable. Proof. By Theorem 3.44, there is a real number M such that f (x) 5 M for all [ a ,b ] . Then for all partitions P of [a,b] we have that L ( f,P ) 5 M ( b - a ) . Set I := sup { L ( f , P ) : P is a partition of [a,b ] } .Because by Lemma 5.17 for any partitions P , Q of [ a ,b] we have L(f,Q) 5 U ( f , P ) , we infer L ( f , P ) 5 I 5 U ( f , P ) for all partitions P of [a,b ] . By Lemma 5.14 for all evaluation sets T , we have L(f,P ) 5 R ( f , P , T ) 5 U (f,P ) , so by Exercise 1-13, for all partitions P of [ a ,b ] the inequality l R ( f , P , T ) - I / 5 U ( f , P ) - L(f,P ) holds. Now let E > 0. By Lemma 5.19, there is a 6 > 0 such that for all x, y E [ a ,b] with E I x - j I < 6 wehave lf(x) - f ( y ) l < -.Let P = {a = xo < x1 < . . . < xn = b } b-a be any partition of [ a ,b] with /I P 11 < 6. Because f is continuous on [ x j - l , x i ] , by Theorem 3.44 and Exercise 3-38 there are tf , t u E [xi-1,xi] so that f t f = mi and
x
E
0
)
f ( t y = M i . Therefore n
By Definition 5.5 f is Riemann integrable with
n
Ib
f ( x ) dx = I .
We conclude this section by proving that for continuous functions the “right” choice of an evaluation point will produce the value of the integral. Proposition 5.21 is a lemma needed to establish the result, which is stated in Theorem 5.22.
5. The Riemann Integral I
94
Proposition 5.21 Let f , g : [a, b ] + IR be Riemann integrable. I f f (x) I g ( x )for all
l
b
x
E
b
f ( x ) dx I g ( x ) d x .
[ a , bl, then
Proof. Let E > 0 and let 6 > 0 be such that for all partitions P = { a = xo < x1 < . . . < xn = b } with 11 PI1 < 6 and all evaluation sets T = ( t l , . . . , t n }we have
Idb
f ( x )dx -
2
f(ti)Axi
g(x)dx -kg(tj)Axi i=l
i=l
P being a partition with 11 P 11 < S and T being an evaluation set we obtain
which proves the result (see Standard Proof Technique 2.7).
Theorem 5.22 Mean Value Theorem for the Integral. Let f : [ a ,b] + IR be continuous. Then there is a c E ( a , b ) so that
l6
f ( x ) d x = f ( c ) ( b- a ) .
Proof. Let M = max{f(x) : x E [ a ,b ] } ,m := min{f(x) : x E [ a ,b ] }and let x,, X M E [ a ,b ] be such that f ( x , ) = m and f ( X M ) = M . Then the constant ) M are continuous, and hence Riemann intefunctions k , ( x ) := m and k ~ ( x := grable over [ a ,b ] . Moreover, k , ( x ) 5 f ( x ) 5 k ~ ( x for ) all x E [ a ,b ] and it is easy to show (see Exercise 5-8) that if k is a constant, then
Therefore by Proposition 5.21 we obtain m ( b - a) 5 then f (x,)=m 5
1,"f (x) d x b-a
Ib
lb
k d x = k(b -a).
f ( x )dx I M ( b - a ) . But
5 M = f ( x ~and ) by the Intermediate Value Theorem
there must be a c between x, and X M so that f ( c ) =
1,"f (XI d x b-a
'
Exercises 5-13. Finish the proof of Lemma 5.16 by proving that if f : [ a . b] -+ W is bounded, P is a partition of [ a . b] and Q = P U ( z ) for some z E [ a ,b] \ P , then U ( f , Q ) 5 U ( f , P). 1
5-14. Prove that f : (0, 11 + W defined by f ( x ) = - is continuous but not uniformly continuous. Then X
explain why this example is not a contradiction to Lemma 5.19. 5-15. A function f : [ a ,b] -+ W is called Lipschitz continuous iff there is an L x,y E [ a , 61 the inequality f ( x ) - f ( y ) 5 L ~ x yl holds.
i
1
t
0 such that for all
(a) Prove that i f f : [a. b] + W is Lipschitz continuous, then f is uniformly continuous.
(b) Prove that i f f : (c. d ) + W is differentiable and f ' is bounded on [ a ,b] C (c. d ) , then f is Lipschitz continuous on [ a ,b ] .
5.3. The Fundamental Theorem of Calculus
95
(c) Prove that f ( x ) := f i is uniformly continuous on [0, 11, but not Lipschitz continuous 5-16. Prove that if f : [a, b] --f [0, 00) is a continuous nonnegative function with
f ( x ) = 0 for all x
E [a, b].
5-17. Let f : [ a , b] +
R be Riemann integrable and let
Ib
f ( x ) dx = 0, then
[ P k ] r = , be a sequence of partitions with
rb
5-18. Prove that i f f : [ a , b ] + [0,co)is a nonnegative Riemann integrable function with
lb
f ( x ) d x > 0, so that f ( x ) 1 6 for all x E [c, d ] . Hint. For a contradiction, suppose the opposite. Use lower sums to show the integral would be zero. then there are an E > 0 and c
id
5-19. Let f : [ a , b] + R be bounded and let g : [ u ,b] + J R be nondecreasing. With mi and Mi as in n
n
Definition 5.13, let Lg(f, P ) := xrnj Agj be the lower sum and let Ug(f, P) :=
E M , Agi be i=l
i=l
the upper sum o f f with respect to g and P . (a) Let P , Q be partitions of [a. b] so that Q is a refinement of P. Prove that Lg(f,p ) 5 Lg(f,Q ) 5 Ug(f, Q) 5
ug(f, P).
(b) Prove that i f f is continuous, then f is Riemann-Stieltjes integrable with respect to g. (c) Let f1, f2 : [a, b] +
R be Riemann-Stieltjes integrable with respect to
f l ( x ) 5 f z ( x ) for all x E [ a , bl, then
lb s,” fl dg 5
f 2 dg.
(d) Mean Value Theorem for Riemann Stieltjes Integrals. Let f : [a, b] --f Prove that there is a c E ( a , 6 ) so that
(e) Let c E (a, b ) and let g := l [ , , b ] . i. Prove that i f f : [ a ,b] -+
with respect to g and
g . Prove that if
R be continuous.
Ib
f dg = f ( c ) ( g ( b )- g ( a ) ) .
W is continuous at c, then f is Riemann-Stieltjes integrable
Ib
f dl[,,b] = f ( c ) .
ii. Prove that if f : [ a , b] + R is not continuous at c, then f is not Riemann-Stieltjes integrable with respect to g . (f) Prove that if f is continuous and g is continuously differentiable, then, with the integral on b
f dg =
the right being a Riemann integral, we have
/
a
b
f ( x ) g ’ ( x )d x .
Note. Integrals against d g are often used to abbreviate Riemann integrals as on the right side.
5.3 The Fundamental Theorem of Calculus The Fundamental Theorem of Calculus connects differential and integral calculus by stating that differentiation and integration are essentially inverses of each other.
Theorem 5.23 Fundamental Theorem of Calculus, Antiderivative Form. Let [ a ,b ] be contained in ( c ,d ) and let F : ( c ,d ) +. R be a differentiablefunction whose derivative f is Riemann integrable on [ a , b]. Then
Ib
f (x)d x = F ( b ) - F ( a ) =: F ( x )
96
5. The Riemann Integral I
Proof. Because f is Riemann integrable, the integral exists. Hence, we can use Lemma 5.6 to prove the equation. For k E N,let Pk = a = xf' < . . . < x$) = b 1 be a partition with (1 Pk 11 = 7 . By the Mean Value Theorem (see Theorem 4. 18), for
I
1
K
each k E €V and each i E { 1, . . . , nk} there is a point ti(k) E [x!!)~,xr)] such that F x?)
F xi'!),
( xi(k)-- x i -(,
theequation f (ty'>= F f (t(k') =
holds. Therefore we obtain
(k)
'i
(x:!)~)
I
f (t?)) Ax(k) = F (x:")) - F . Let Tk := t!k) : i = 1, . . . , nk . Then Tk is an evaluation set for Pk and by Lemma 5.6 we conclude via a telescoping sum
which proves the result.
Definition 5.24 Because of the Fundamental Theorem of Calculus, if F and f are functions with F' = f , then F will also be called an indefinite integral o f f , denoted F =
1
f (x) dx. One way to compute Riemann integrals is to evaluate an indefinite
integral at the upper and lower bound and to compute the difference. The hypothesis that the derivative of F is Riemann integrable feels quite artificial. Nonetheless, this hypothesis is best possible. Exercise 12-24 will exhibit a differentiable function whose derivative is bounded, but not Riemann integrable. Although this example is a bit pathological, it points out a weakness of the Riemann integral that motivates the development of the Lebesgue integral. The Antiderivative Form of the Fundamental Theorem of Calculus for the Lebesgue integral, which has no artificial looking hypotheses, will be proved in Exercise 23-8. The Derivative Form of the Fundamental Theorem of Calculus is proved in Theorem 8.17 for the Riemann integral and in Exercise 18-6 for the Lebesgue integral.
Exercises 5-20. Power Rule for integration. Let r E
Q \ [-
positive or both be negative. Prove that
b. In case r i0, let a and b either both be 1 1 x r dx = -brcl o r + ' . Then explain why
1) and let a
Ib
i
r+1
~
r f 1 i0.
we needed to require a and b to be both positive or both negative for r
5-21. Integration by Parts. Let [ a , b] c (c, d ) and let F , g : (c, d ) + W be continuously differentiable with derivatives f and g'. Prove that
Ib
f ( x ) g ( x ) dx = F(b)g(b)- F ( a ) g ( a ) -
Ib
F ( x ) g ' ( x ) dx.
5.4. The Darboux Integral
97
5-22. Integration by Substitution. Let [a, b] c (c, d ) , let g ; (c, d ) + W be continuously differentiable with derivative g’ and let F be continuously differentiable with derivative f such that the domain of F contains g [ [ u , b]
1. Prove that
b
f
(g(x)
) g ’ ( x ) dx = F ( g ( b ) ) - F
(g(a))
.
5-23. What would we need to prove so that the hypothesis that the derivatives are continuous in Exercises 5-21 and 5-22 can be replaced with the hypothesis that the derivatives are Riemann integrable?
5.4 The Darboux Integral Lemma 5.6 is an efficient tool to establish properties of the Riemann integral, provided all functions involved are Riemann integrable. It is now time to look for a similarly efficient criterion to prove that a function is Riemann integrable. Riemann’s Condition below is inspired by the idea of trapping the Riemann sums between lower and upper sums, which was already used in the proof that continuous functions are Riemann integrable. Riemann’s Condition is simpler to verify than Definition 5.5 of Riemann integrability, because, for E > 0, instead of working with all partitions of sufficiently small norm and all evaluation sets, we only need to find one partition so that the upper and lower sums are closer together than E . The price is paid in the proof, where for the ‘‘e” direction, our only tool is one partition and we must prove something for all partitions of sufficiently small norm.
Theorem 5.25 Riemann’s Condition. Let f : [ a ,b] + Jft be a boundedfunction. Then f is Riemann integrable on [a,b ] i#for all E > 0 there is a partition P of [a,b] such that U (f , P ) - L ( f , P ) < E. Proof. For “+,”let f : [ a ,b] + EX be bounded and such that for all E > 0 there is a partition P of [ a ,b] such that U ( f , P ) - L ( f , P ) < E . By Lemma 5.17 the set B := { L ( f , P ) : P is a partition of [ a ,b ] }is bounded above. Let C := sup B and let E > 0. Let P = {a = xg < x1 < . . . < x, = b } be a partition of [ a ,b ] such that E U (f,P ) - L ( f , P ) < -. Then for all refinements Q of P and all evaluation sets T for 2 Q , Lemmas 5.14 and 5.16 imply
IR(f,Q , TI
u(f, P ) - L ( f 3P ) < 5.
- 131 I u ( f ,Q ) - L ( f , Q ) I
&
To show that f is Riemann integrable, let M := sup {If (x)l : x E [ a ,b ] }and let & Ax1 &?.! . We will now show that for all partitions S 6 := min ( 4 n ( M + 1 ) ’ 3 ’ ” ’ 3 with IlSIl < 6 and all associated evaluation sets TS we have l R ( f , S, Ts)- CI < E . Let S = a = xo < x1 < . . . < x& = b be any partition of [ a ,b] with IlSll < 6. s s Then Q : S U P is a refinement of P . Therefore, for any evaluation set T for Q & we have R ( f , Q , T ) - CI < -. Moreover, by choice of 6, every interval [x:-~, xs] 2 contains at most one point of P that is not in S and any two intervals [x;-, , x/”]that contain such a point do not intersect. Let TS be any evaluation set for S and let T be an evaluation set for Q that contains Ts.Then T \ TS = { t i , , . . . , t i k } with k < n , because
1
1
I
I
98
5. The Riemann Integral I
the addition of at most n - 1 points to S that are all in distinct intervals at most n - 1 intervals that do not already have an evaluation point assigned. Therefore 1 R ( f ,Q , T ) - R
( f 9
S,
Ts)1
and hence
IR ( f , S , Ts)- CI
I IR
( f , S , Ts) - R ( f , Q, T ) l + I R ( f , Q , T ) - Cl
& & -+-=&.
<
2
2
Thus f is Riemann integrable and its Riemann integral is C. For ''=+," let f : [ u , b] -+ R be Riemann integrable and let E > 0. Then there is a number I such that for all E > 0 there is a 6 > 0 such that for all partitions & P with IIPII < 6 and all evaluation sets T we have that I R ( f , P , T ) - I1 < -. Let 4 P = ( a = xo < x1 < . . . < x, = b] be such a partition and for each i E 11.. . . , n } & & f i n d t f , t y E [xi-l,x;]suchthatf t,! I m i + and f (ty) 2 M ; - -. 4n Axi 4n Axi L e t T L : = { t , ! : i = l , . . . , n a n d T ' : = { t ~ : i = l , . . . , n}.Then
I
0
U ( f >P ) - L ( f . P ) <
[ U ( f ,P ) - R
(f,P , T')]
+ / R (f,P , T')
+ 11 - R ( f , P , T L ) i + [ R ( f , P , T L )
-
-
L ( f ,PI]
99
5.4. The Darboux Integral
n
<
&
C-AXi+z+C4n Axi i=l
&
n i=l
&
&
AX,- 4nAxi ‘ - 4
+ -2 + -4 = &
&
E,
which was to be proved. Riemann’s Condition shows that if the lower and upper sums of a function f can get arbitrarily close to each other, then f is Riemann integrable. The only way this cannot happen is if the function oscillates too much, that is, if the function is highly discontinuous.
Example 5.26 The Dirichlet function f (x) = 0; f o r x E [O, 11 \ Q> is Riel ; for x E [0,l ] n Q, mann integrable on [0,I ] . All lower sums of the Dirichlet function are zero and all upper sums are 1. By Theorem 5.25 the Dirichlet function is not Riemann integrable. 0 It is natural to ask now how much oscillation or discontinuity there can be without losing Riemann integrability. Moreover, there are further important results about Riemann integrals that could be presented here. Theorem 5.25 could be used to prove these results, but the proofs would involve substantial work with Riemann sums. We will avoid this work by first using Theorem 5.25 to prove the Lebesgue criterion for Riemann integrability in Theorem 8.12. This criterion makes the proofs of further results about Riemann integrals, presented in Section 8.3, more effective. Moreover, the Lebesgue criterion will answer how discontinuous a Riemann integrable function can be. To formulate the Lebesgue criterion, we need to introduce the Lebesgue measure of a set, which is fundamental for more advanced analysis. To define Lebesgue measure we need “infinite summations” (introduced in Chapter 6) and some set theoretical notions of “size” for sets (introduced in Chapter 7). In Chapter 8, we will continue where we leave off here. To conclude this chapter, we note that the approximation of the area under a function with lower and upper sums is also known as Darboux integration. To acquaint the reader with the language involved, we formally introduce Darboux integration, which is equivalent to Riemann integration.
Definition 5.27 Let f : [a,b] -+ R be a boundedfinction. We define
Cf 24,
P ) : P is a partition of [ a ,b ] } , := sup {L(f, := inf { U (f , P ) : P is a partition of [ a ,b ] } .
Cf is also called the lower integral o f f and Uf is also called the upper integral of f . We will say that f is Darboux integrable on [a,b] iff Cf = 24f. In this case Cf = 24, is also called the Darboux integral o f f .
Proposition 5.28 Let f : [ a ,b ] + R be a bounded function. Then Cf 5 Uf.
5. The Riemann Integral I
100
Proof. Easy consequence of Lemma 5.17 (Exercise 5-24). Theorem 5.29 Let f : [ a ,b ] -+ R be a bounded function. Then f is Darboux integrable on [ a ,b] iff f is Riemann integrable on [a,b]. Moreovei; the Darboux and Riemann integrals are equal in this case, that is,
Lb
f ( x ) d x = C = 24.
Proof. For “jlet ,” f : [ a ,b] ++R be Darboux integrable and let E 0. Then & U = C and there are partitions Q and R of [a,b] such that C - - < L ( f , Q ) and 2 & U(f,R) < U + - . N o w P : = QURisarefinementof QandRandbyLemma5.16 2 we infer
C-
&
- < L ( f , Q ) 5 L ( f ,P) I C=U I U ( f ,P ) 5 U ( f , R ) < U +
2 = C + i, &
&
2 and hence U (f , P ) - L ( f , P ) < E . By Theorem 5.25 f is Riemann integrable. For “+,”let f : [ a ,b ] -+ R be Riemann integrable. Then by Theorem 5.25 for all E > 0 there is a partition P of [ a ,b ] such that U ( f , P ) - L(f,P ) < E . Hence, for any E > 0 we infer U - C 5 U (f , P) - L ( f , P ) < E , which means U = C. The “moreover” part follows upon examination of the “+”part of the proof of Theorem 5.25, which also shows that the Riemann integral o f f is equal to C.
Exercises 5-24. Prove Proposition 5.28. 5-25. Let f ,g : [a,b] + R be Darboux integrable. Prove that then f
cf+g= L f + c g .
5-26. Let f : [ a ,b] --f
W be Riemann integrable on [a.b ] .
(a) For each n with 11
+ g also is Darboux integrable and
E
[5
W, let P - a = x $ ) < . . . < 1‘“) = b } be a partition of the interval [ a . b ] kn n - 1
11 < -1 and let sn
:=
n
1
m j ” ) = inf f ( x ) : x E [xf:),
k,i
~
1
1
m ~ ) l ~ X ; ~ > , , X+~m ) )t ) l
, x?)]}
i. Prove that for all n E N and all x ii. Prove that i f f is continuous at x iii. Prove that lim n-30
l
b
1 f - sn 1
(b) Prove that there is a sequence {cn};$
E E
-,I’;.,
(n) [Xkn
’ where
[ a ,b] we have sn ( x ) 5 f ( x ) X converges to f ( x ) . [a, b ] ,then { s , ( x )
dx = 0
of continuous functions on [a,b ] such that for all n
E
W
we have jcn/ 5 I f / and so that 5-27. Prove Riemann’s Condition for Riemann-Stieltjes integrals. That is, let f : [ a ,b] +. X be bounded. let g : [a.b ] + E be nondecreasing and prove that f is Riemann-Stieltjes integrable on [a.b ] with respect to g iff for all F z 0 there is a partition P of [a,b] such that U,(f,P ) - L g ( f ,P ) < E .
Chapter 6
Series of Real Numbers I Series facilitate the computation of “sums with infinitely many terms.” This first introduction mostly showcases the results needed to define outer Lebesgue measure (see Definition 8.1). Further results about series will be presented in Chapter 10.
6.1 Series as a Vehicle To Define Infinite Sums Of course it is impossible to add infinitely many numbers. But we can consider the convergence behavior of a sequence of finite sums.
Definition 6.1 Let ( a j } c l be a sequence of real numbers. The partial sums of the n
a2
series c a j of real numbers are dejined to be sn :=
la2
a j . The series is said to
i=l
j=1
converge ifs the sequence of partial sums ( e a j j=1
converges and it is said to
n=l
c ..
diverge otherwise. For a convergent series, the limit is usually denoted
aj
j=1
Series that start at numbers other than 1 are defined similarly (Exercise 6-1). This section is devoted to introductory examples and some fundamental results. Geometric series are the prototypical examples of convergent series. Because there is a simple formula for the partial sums, geometric series nicely fit the mold of Definition 6.1. Figure 15 gives an indication why geometric series are called “geometric” and Exercise 6-2 gives some examples of computations.
Theorem 6.2 Geometric sums and geometric series. Let a , q E R and let q f 1. n 1 -qn a q j = aq -holds. Moreover;for Then for all n E N the summation formula 1-9 j=1
c
101
102
1
6. Series of Real Numbers I
I I I I
1
-
1 1 1 1 -+-+-+-+...=I 2 4 8 1 6
t -
2
I I I I
Figure 15: The “sum” of infinitely many terms can be finite. Each rectangle inside the square above is one-half of the size of the previous rectangle and the sum of all their areas should be the area of the square.
c
all real numbers 1q I < 1 the series
c oc
30
aqJ converges and
j=l
j=l
aq a q J = -. 1-q
Proof. The first statement follows from the following telescoping sum argument. n
n j=1
;=1
j=1
n
r z t1
;=l
j=2
= aq - a q n f l . a;
For 141 < 1, note that
n
x a q j
1 -qn
= lim
;=1
n e w
aq
.
m
;=l
Series with nonnegative terms play an important role in analysis. Their partial sums are nondecreasing. Therefore, by the Monotone Sequence Theorem, whenever the terms of a series are nonnegative and the partial sums are bounded, the series converges. cc
Lemma 6.3 For all j
E
M, let a j 2 0. Then the series
quence of the partial sums
{2 J=1
a;
1
a j converges $the sej=1
is bounded above.
n=l
Proof. For 3,” recall that by Proposition 2.34 any convergent sequence is bounded, and hence, in particular, it is bounded above.
103
6.1. Series as a Vehicle To Define Infinite Sums
c 00
For
"+,"let
aj be a series such that all aj are nonnegative and such that the
j=1
c c c
sequence n+l have
00
of its partial sums is bounded above. Because for all n E
aj
N we
{ j l l
aj = an+l 2 0 , the sequence of the partial sums is nondecreasing.
aj -
j=1
j=1
Thus by the Monotone Sequence Theorem series converges.
{
c
30
must converge, and hence the
aj In=1
j l 1
Series can easily be added, subtracted, and multiplied by numbers. Multiplication of series is more complicated and is therefore deferred to Theorem 10.14.
c c 00
Theorem 6.4 Let
x)
bj be convergent series of real numbers and let c be j=1 j=1 a real numbel: Then the following hold. aj and
c c c 00
1. The series
aj
c c 00
+ bj converges and
aj
j=l
j=1
00
2. The series
j=1
33
c
bj .
aj
j=l 00
aj -
j=1
00
caj converges and
00
00
aj - bj =
j=l
00
c +c c c
j=1
00
aj - bj converges and
j=l
3. The series
+ bj =
bj.
j=1
c 00
caj = c
j=l
aj .
j=1
Proof. To prove part 1, note that 00
n
x
C a j+ c b j j=1
=
n-oo lim
c a j j=1 n
j=l
=
n
+ n+m lim
n
I1
c b j =)i-ncaj + c b j j=l j=l j=l 30
lim c a j + b j = c a j + b j , n+w j=l j=l
where in each step the existence of the quantity on the left side of the equal sign implies
c c c +c 00
c+ x
c 00
cc
00
aj and
the existence of the quantity on the right side. Hence, if
j=1
bj converge,
j=1
00
bj . j=l j=1 j=1 j=1 The remaining parts are left to the reader as Exercises 6-3a and 6-3b.
then
aj
bj converges and we have
aj +bj =
aj
6. Series of Real Numbers I
104
Similar to the algebraic operations, comparabilities can be moved through the summations.
c c c c cc
w
Theorem 6.5 Let
cc
00
Then
j=1
aj 5
j=1
bj be convergent series so that a j 5 bj for all j E
a j and
N.
j=1
bj.
j=1
Proof. Exercise 6-4. The most concrete example of series in our daily experience is so fundamental, it is often not even recognized as a series. Geometric series are the foundation for the decimal expansion of real numbers. Formally, decimal expansions are an identification of numbers with sequences of integers. The connection is made via series with nonnegative terms as shown below.
Proposition 6.6 Let D be the set of all sequences { d j ) p of integers in the set (of digits) (0, 1, 2, 3 , 4 , 5 , 6 , 7 ,8,9) such that there is a k E &=with dk # 0 and so that for every n E N there is an m 2 n so that d,,, # 9. Then for each { d j ) g l E D the series O0 d. converges and the function r : D + (0, 1) is bijective. r ( { d j ) T l ) := j=1 For every x E (0, l), we call r-l (x) the decimal expansion ofx.
c
Proof. Let {dj]T=o=l E D . For all n 05
E
N,we infer the inequalities
c- c-
1 9 = 9101 10 1j=1 loJ - j=1 dj
<
n
&
9 1 10 2 10
<--=I -
. co
dj are nonnegative, Lemma 6.3 guarantees that d j conBecause the terms 10J 105 j=1 verges. Moreover, because at least one dk is positive and not all dk are equal to nine,
c 00
we infer that 0 <
dj < 1 for all { d j } E 1E D . -
101 i=l To prove that r is injective, let { c j } p l f: {dj}y=l be sequences in D. Let n E R? be the smallest natural number so that cn $r d,, and assume without loss of generality that cn > d,. Then 00 cj - dj 30 1 r ( { ~ j )-c rl ( {)d j ) p l ) = 2 - -k 10J 1o n j=n+l j =n
C
1
-
c7 C -=--c--
1On
" O 9 j=n+l 101
1 10n
9 1 10 10n 1 -
-1-
&
1
lon
-
1 lon
O 0 9 1 j = l 10" 10J 1 = 0: 10"
6.1. Series as a Vehicle To Define Infinite Sums which implies that r ( { c ; For surjectivity, let x
} E l )# r E
105
({dj}zl).
(0, 1) and construct {d;}TZl recursively as follows. Set the
digit do := 0. Once dn has been defined so that 0 5 x the largest integer k so that 0 I x-
c
inequalities
o5x -C j=l
The series
O0
;=l
j=1
dj
--
j=1 nfl
105
2 di
j=l
1 < -,
10n
let dn+l be
dj 1 - < -hold. 105 lon+'
d. -L converges to x,because for all n 1oj
dj
101
k 10n+" Then dn+l is at most 9 and the
1 < -. Finally, for each n 101 10n j=1 dm # 9, because otherwise we would have that 0 p x -
2 dj
E
j=l
N we have by construction
N there must be an m
9
dj
03
E
1 9+3
j=n+l
;=l
> n with
1 10
which cannot be. In particular, the representation in Proposition 6.6 can be used to convert infinite repeating decimals into fractions (see Exercise 6-5). Of course not every series converges. For example, if all terms are equal to 1, the sum is infinite. The limit test below is a criterion for divergence, because it says (Exercise 6-6) that if the terms do not go to zero, the series diverges. c*3
Theorem 6.7 Limit Test. Ifthe series
aj
converges, then lim j-co
;= 1
aj
= 0.
co
Proof. Note that if
C
aj
converges, then
j=l n
n-1
00
c4
Caj - C a j = 0.
lim a, = lim x u ; - x u ; = n-co n-co ;=l j=1 j=1
j=1
If Theorem 6.7 was a biconditional, the theory of series would be very easy indeed. The next example shows that this is not the case.
Example 6.8 The harmonic series
O01 diverges, even though the terms converge to C J
;=l
zero.
106
6. Series of Real Numbers 1
The partial sums sn = subsequence { s n l ni = 2‘.
c
n 1
- form an increasing sequence. If we can show that a
j=1 j
}Elgoes to infinity, then the sequence { s n } z 1 diverges. We choose
i-1
i k=O
k=O
Because the
cf x
that
~ 2 go i
to infinity, the sequence of partial sums diverges, which means
.
0
diverges.
We will be confronted with “infinite sums” like the harmonic series throughout measure theory. Therefore it is useful to formally define infinite sums.
c c x
Definition 6.9 Let
n
n+oo
a.- co and call it infinite or an
j=l
j=l
infinite sum ifs lim
c Ly,
a j be a series. We write
n
00
C a j =-aifs lim C a j =-a.
a j = 00. we write
j=1
n-+Ly,
j=l
j=l
Exercises 6-1. Let k
E
Z and for each j
E
Zlet a ,
E
W.Define the series
x.
k
a ] and
a] j=-w
j=k
6-2. Compute the value o f each of the series below.
6-3. More on the arithmetic of series (Theorem 6.4) (a) Prove part 2 of Theorem 6.4 (b) Prove part 3 of Theorem 6.4.
x j=1 Cc
00
aj
j=I
nor
b j converges. j=l
x a j and
(c) Give an example of two series
j=1
c rn
b j such that
aj
j=1
+ b j converges, but neither
6.1. Series as a Vehicle To Define Infinite Sums
c m
(d) Is it possible to find two series
c m
actly one of
j=1
c
c
j=1
c m
00
a j and
b j such that
j=1
a j
+ b j converges, and ex-
j=1
m
aJ and
c
b j diverges?
j=1
00
(el IS there a series
107
c
m
00
a , and a c E R such that
j=l
a j diverges?
c a j converges, and
j=1
j=l
6-4. Prove Theorem 6.5 6-5. Convert each of the following infinite repeating decimals below into a fraction. A bar over a set of digits means that these digits repeat indefinitely. (b) 0.25
(a) 0.25
(c)
0.9462
(d) 0.1473
(e) 12.004%
c m
6-6. Why does the limit test say that if ,lim a j f 0, then the series 5-00
a j diverges?
j=l
6-7. Another proof for the summation formula in Theorem 6.2. Let q f 1 and a be real numbers. Prove
c
?I
by induction that
c
j=1
1-9"
'
aqJ = aq 1-GJ 00
00
6-8. 2k test. Let
a j be a nonincreasing sequence with nonnegative terms. Prove that
j=1
a j converges j=1
k= 1
Hint. Use Example 6.8 as guidance.
c m
6-9. Prove that
converges by showing that the partial sums form a Cauchy sequence
(-
j
j=1
5
1 -
m
+ -.n1 Be careful to distinguish all possible combinations of
n . m being even or odd.
6-10. Translating between sequences and series. Let ( u , ) ~ = ~be a sequence of real numbers. Prove that m
converges iff the series c ( a , + l
- a j ) converges and that in this case we obtain the limit
j=l oi
as lim an = a1 n-rm
+C(aj+l
-aj).
j=1
6- 1 1. Use Lemma 6.3 to prove the Monotone Sequence Theorem
c 00
6-12. Let
a j be a convergent series, let ( j k ] &
be a strictly increasing sequence of natural num-
j=1
c
jk-1
bers with j o = 1 and for all k
E
N let
Ak :=
j=jk-l
c 00
aj.
Prove that
Ak converges and
k=l
6. Series of Real Numbers I
108
6.2 Absolute Convergence and Unconditional Convergence For series, there is no simple condition that is equivalent to convergence. Therefore we need criteria that imply convergence. This section presents the criteria that are needed to define and work with outer Lebesgue measure. More criteria will be presented in Section 10.2. Cauchy sequences play an important role throughout analysis, so it is natural to relate the convergence of series to Cauchy sequences. Cc
Proposition 6.10 Cauchy Criterion. A series
a j converges lfSfor all E > 0 there j=1
is an N E
N so that f o r all n 2 rn 2 N we have 00
Proof. The series
c a j
converges iff the sequence
{s,}zl of its partial sums
j=1
converges, which by Theorem 2.27 is the case iff it is a Cauchy sequence. This is the case iff for all E > 0 there is an N E N so that for all n 3 m 3 N the inequality
Isn - sm-11
iE
holds. Since
1
I we have proved the result.
a , = Is, - s-1, 1j;m
The Alternating Series Test shows that there are indeed many convergent series.
Theorem 6.11 Alternating Series Test. Let
{ b j } F 1 be
a nonincreasing nonnegative
00
sequence such that lim
bj
(- l ) j + ' b j converges.
= 0. Then
J-00
j=1
Proof. Let E > 0 be arbitrary. There is an N E b, i-. Then for all rn > n 2 N we obtain 2 I m
I
m-n
1 j=n
I
i=O
E
M so that for all n
i =O
,
i
use bn+2,+1?bn+2(j+1)and a telescoping sum
3 N we have
109
6.2. Absolute Convergence and Unconditional Convergence
rn
and by the Cauchy Criterion we conclude that the series converges.
In many situations we will work with series of nonnegative numbers. If negative summands occur, it is natural to take absolute values and hope the sum still converges. This is not always the case, but the idea of absolute convergence is fundamental.
c
c 00
00
Definition 6.12 A series
converges absolutely iffthe series
a j
j=1
laj
1 converges.
j=1
Absolute convergence is a strictly stronger condition than convergence. That is, absolutely convergent series converge, but the converse is not true.
c c J II c 00
Proposition 6.13 I f the series
converges absolutely, then it converges. More-
a j
j=l
Ix
ovel; the triangular inequality
x
a. < l a j I holds. ij=l - j=1
c c 1 00
00
Proof. Let E > 0. Because
a j
converges absolutely,
j=l
laj
I converges. Thus
j=1
n
there is an N E
N so that for all n 2 m 2 N we have that
c n
all n 2 m 2 N we obtain
c
5
laj
< E . But then for
j =m laj
I<
E.
Therefore by the Cauchy Criterion
j=m
x
the series
a j
converges.
j=l
c n
Moreover, for all n E W we have
l
a j
j=1
I1
n
I
c 00
laj I
I
j=1
la;
1, which implies the
j=l
rn
triangular inequality. 00
Example 6.14 By the Alternating Series Test, the series
cI x
However,
(-I);+'
j=l I
f J
~
'
=
O01 j=1
e(-l)j+l j=1
1 converges. j
= 00, so it does not converge absolutely.
0
J
Absolute convergence of a series is often established with the Comparison Test.
c c a2
Theorem 6.15 Comparison Test. Let
a j
j=1
30
for all j
E
N.I J ? b ~j converges, then j=1
M
bj be series with 0 5 a j I bj
and ;=l
M
a j
j=1
converges, too.
110
6. Series of Real Numbers 1
is a nondecreasing sequence and for all n we have
c c c n
j=l
5
00
rn
n
aj
b j
5
b j
< co. This means that
is bounded above by
j=l
j=l
00
c 00
bj.
By the Monotone Sequence Theorem, the series
j=1
a j
converges.
j=l
The Comparison Test allows us to establish convergence and divergence for series that can be compared to series with known convergence behavior. For example, O =1 converges because 2 j12 < -
(k)'.
j=l
2J2
Similarly, O 01 diverges because 1 > -.1 We
A
i-1 A - j will use the Comparison Test extensively in Section 10.2. Moreover, it should be noted that the Comparison Test can also be applied when there is a k G N so that the comparability 0 5 a j 5 b j holds only for j 2 k (Exercise 6-16). J
- I
We conclude this section with the subtle, but nonetheless necessary, notion of unconditional convergence. Assume that the terms of a series are provided in no specific order. (This is the case in the definition of outer Lebesgue measure.) Then we could add the terms in any order we choose. It would be catastrophic if the value of this summation depended on the order in which we sum the terms. Unfortunately, for 30 1 arbitrary series this can actually happen. For example, - 1)J + l - converges. How-
c(
J=1
j
c
C 1 ever, a rearrangement can destroy the convergence. Consider that O - 00 and r=l 2 j + 1
c( 30
I
- 1) - = --oo (Exercise 6-13). Suppose we first sum enough odd numbered terms
j=1
2j
1 1 to get a number > 1, say, - - > 1, then add the first even numbered term, so we 1 3 1 1 get - then add enough odd numbered terms to get a number > 2, say, 1 3
+
+ + (-:),
> 2, then add the second even numbered term, so we
1 get 1
k)
+ -31 + (- +
get a number > 3, say
20
1
+ (-
),then add enough odd numbered terms to
j=2
+-+ 1 3
254
-
j=2
k=21
1
6.2. Absolute Convergence and Unconditional Convergence
111
then add the third even numbered term, so we get 254
j=2
1
k=21
and proceed in this fashion indefinitely. This process uses all the available numbers and it produces a sequence of sums that goes to 00. Of course we will need to use more and more odd numbered terms to make up for the one new even numbered term and the increment of 1 , but nonetheless, in one arrangement the series sums to a finite number, while in another the sum is infinite. We will make this idea more precise in the proof of Theorem 6.18 and we will push it to its full extent in Exercise 6-23. Series in which we can rearrange the terms in any way without affecting the “sum” are called unconditionally convergent.
c 00
Definition 6.16 The series
aj converges unconditionally @for all bijective func-
j=l
co
a2
co
tions u : W + W the series C a , ( j ) converges and x a , ( i ) = x a j . Zfa series i=l i=l j=l converges, but not unconditionally, we will say it converges conditionally. For series of real numbers, Theorem 6.18 below shows that unconditional convergence and absolute convergence are equivalent. This is very helpful, because it is a lot easier to check whether one series of absolute values converges than to check if all rearranged series converge to the same limit.
c co
Definition 6.17 Let
c co
be the series
aj be a series and let B 5 N. We dejine the series
j=1
c
a; to
jeB
b j , where b; :=
a,;
for j
E
B,
;=l
c =o
Theorem 6.18 The series
a, converges absolutely @it converges unconditionally.
;= 1
Proof. For “j,” let E > 0 and let N E N be such that for all n 2 rn 2 N we have
2
c co
la;/ <
;=m
E . Then 2
&
lajl 5 - < E . 2
j=N Now let u : W -+ N be an arbitrary bijection. Then there is an I (1, . . . . N - 1) a[{l,. . . , Z } ] . Hence,forallrn p Z weobtain lm
c
o
I
I
I
E
W so that
112
6. Series of Real Numbers I
c
c oc
00
which means
a,(i) =
uj.
j=1
i=l
M
For
"+,"we prove that if
x a j does not converge absolutely, then it does not j=1
c co
converge unconditionally. There is nothing to prove if
c cc
can assume
does not converge, so we
aj
j=1
aj
converges, but not absolutely.
j=1
For j E
N,let uj'
:= max{aj, 0) and let a; := - min{aj, 0). We first claim that
oc
ZC
33
x u ; =
c u r = co. Suppose for a contradiction the series X u :
j=l
j=1
c c c
c c
oc
00
Then the series
a; =
00
00
laj I =
- ( U j - u t ) would converge, so
n=l
j=1
did converge.
j=l
j=1
uJ' +a;
j=1
00
would converge, and hence
a j would converge absolutely, a contradiction.
j=1
We now recursively construct a bijection a : N +
N so that
c n , ( i ) diverges. i=l
ffi ~~
Because X u : = co there are n1 E N a n d a(1) < a(2) <
. . . 4 a(n1) so that for
J=1
all i E (1, . . . , n l } we have a,(i) > 0, so that for all j 5 a(n1) with a j > 0 we have ni
nl-1
j = a(i) for some i E { l ,. . . , n l } and so that
Because
a,(i) 5 1 < i=l
i=l x
~ u j = c ~ t h e r e a r e mE lN a n d o ( n 1 + 1 ) < a ( n 1 + 2 ) < j=1
+
+
...
+
for all i E {nl 1 , . . . , n1 m l } we have a,(i) I 0, so that for all j I a(n1 m l ) with a j 5 0 we have j = a(i) for some i E ( n l 1, . . . ,n1 m l } and so that
+
+
i=I
i=l
For the recursive step, let no := mo := 0 and assume that nl < . . . < n k , m 1, . . . , mk E N and distinct a ( l ) ,. . . , a(nk mk) E have been chosen as follows.
+
k
Foralli E P k : = U { n p - l + m p - l + l , . . . , np}wehavea,(i) > ~ , f o r a l l lj a ( n k ) p=l I1 1
i=l
i=l
6.2. Absolute Convergence and Unconditional Convergence
u k
Moreover, for all i E
j 5
a(nk
+ mk) with
N k :=
{ n pi-1, . . . , n p
+ m p } we have u,(i)
5 0, for all
p=l
c
Ui
5 0 we have j = a(i) for some i E
nkfmk-1
the inequalities
113
Nk
and the sums satisfy
nk+mk
u,(i) > k 2
u,(~). i=l
i=l
+ mk + 1) < . . . < O ( n k + l ) greater than a ( n k ) so that for all i E &+I := u { n P - l + mp-1 + 1, . . . , n p }we have u,(i) > 0, for all Choose natural numbers a ( n k k+l
p=l
j 5
a(nk+l)
with
Uj
nktl-1
ties
+1<
a,(i) 5 k i=l
c
> 0 we have j = a(i) for some i E Pk+l and the inequali-
u,(i) hold. (This can be done because i=l
U,(i)
uj’ = 00.)
j=1
nk+mk
Because k 2
c co
nk+l
we infer
> nk
nk+l
+ mk.
Choose the natural numbers
i=l
a(nk+l
+ 1) < . . . < o ( n k + l
+mk+l)
greater than a ( n k + m k ) such that for all indices
k+l
i
E N k + l := U ( n p + l, . . . , np+mp}wehaveu,(j)
10,forallj
sa(nk+l+mk+l)
p= 1
with U j 5 0 we have j = a(i) for some i
c
nk+l + m k + l - 1
u,(i) > k
i=l
Because k
c
E Nk+1
and the sums satisfy the inequalities
nk+l fmk+l
+1<
c
+12
cUT 30
u,(i). (This can be done because
= 00.)
j=l
i=l
nk+l
a,(i) we infer m k + l
2 1.
i=l
Then a : N +. N as constructed is injective by construction, surjective because in every recursive step at least one more positive term and at least one more nonpositive 30
term of xu;are added to the set
. . . , u , , ( ~ ~ + ~and ~ ) }the , construction shows
(uD(l),
is unbounded, and hence divergent.
Standard Proof Technique 6.19 In the proof of “=+”of Theorem 6.18, our estimates
c co
only yielded a nonstrict inequality
luj
I 5 . . .. By making the sum less than or
j=N E
8 .
equal to - we made it strictly less than E . Because E can be replaced with - in any 2 2 argument, in analysis it is usually not a problem if an estimate only yields a nonstrict inequality 5 E rather than a strict inequality < E . 0 Sometimes we need to partition an infinite sum into infinitely many “chunks” (see the proof of part 3 of Theorem 8.6). For this type of rearrangement, we use double
6. Series of Real Numbers I
114
series. For brevity’s sake, we limit ourselves to double series of nonnegative terms in Proposition 6.22.
Definition 6.20 For i , j E N,we define the ordered pair ( i , j ) := { i , { i ,j ] } and we define N x N = { ( i , j ) : i , j E N}.A (doubly indexed) family of numbers { ~ ( i , j ) ] Z = ~ is a function from N x N to R. The definition of ordered pairs guarantees that the order in which the numbers are listed matters (Exercise 6-14). Definition 6.20 also indicates that it is time to investigate set theory in more detail. We will do so in the next chapter.
Definition 6.21 Let series
{a(i,j)}$=l be
7, M
a doubly indexed family of numbers. The double
M
M
a ( i , j )is called
convergent $for all i
E
a(,,j ) converges
W the series
i = l j=1
j=1 00
also converges. We will also denote the
and furthermore the series i=l
cooc
m o c
double series as well as its limit by
ai3jinstead i=l j=1
of
a(i,j). i=l j=1
Proposition 6.22 shows that for double series of nonnegative numbers the order of summation is immaterial.
Proposition 6.22 Let
{ ~ ( i , j ) ] E ,be = ~a family
of nonnegative numbers. Then the dou-
m o o
ble series
a ( i , j ) converges
Ifsf o r all bijections a : N x N
+W x
N the sum
i=l j=1 a330
7‘7, a,(i,j) converges. Furthermore, in this case the values are equal. j=l
j=1
Proof. The direction E
prove “+,”let
‘‘e” is trivial, because we can choose a ( i , j )
:= ( i , j ) . To
M
77,a ( i , j )be convergent and let a : N x N + N x N be a bijection. i = l j=1
c
x
m
Let i E W. Then for any m E
c
N we have
a,(i.;)
m
5
j=1
a ( k , ; ) .Hence,
by Lemma
k = l j=1
..
6.3 each series
a,(i,j)
converges.
j=1
0000
For the convergence of the rearranged double series to
a(i,j ) , let E > 0 be i = l j=1
arbitrary but fixed, and let m E N. Then for all i
E
{ 1, . . . , m ] there is an Ni so that
115
6.2. Absolute Convergence and Unconditional Convergence &
Qu(i,j) j=N,+l
< -. Let N := max { N i : i E (1, . . . , m } } .Then rn /
N
Because E was arbitrary, for all m
E
m
m
m
m
N the inequality
N
cccc
m c c
>:7;
au(i,j )
i = l j=1
5
7 7;a(i,
j)
i = l j=1
corn
holds. In particular, this means that the double series
a u ( i , j )converges
and
i=l j=1 cccc
i = l j=1
i=l j=1
To show that the inequality actually is an equation, note that the roles of the double series can be reversed in the above argument to arrive at the reversed inequality (see Exercise 6-24). 00cc
0 0 0 0
a(i,j ) , which completes the
au(i,j ) =
Therefore, we conclude that
We will revisit the summation of double series that also have negative terms with more sophisticated tools in Exercise 14-50.
Exercises
c 30
6-13. Prove that
j=1
cc
1 ~
2j
1
= 00 and c ( - l ) -
+1
j=l
= -00.
2j
6-14. Let i . j , m ,n E N.Prove that (i, j ) = (m.n ) iff i = m and j = n
c c 00
6-15. Cauchy Criterion for absolute convergence. Prove that a series
a , converges absolutely iff for
j=1
n
all E > 0 there is an N E
N so that for all n
c c c X
6-16. Comparison Test revisited. Let k E
c c
N and let
X
j ? k . Prove that if
b j converges, then
j=1
laj 1 <
? m ? N we have 33
bj be series with 0 5 a j 5 bj for all
a j and
j=1
cc
J=1
a j converges, too.
j=1
33
6-17. Prove that a series
a j converges absolutely iff the sequence
J=l
E.
j =m
6. Series of Real Numbers I
116
6-18. Prove that the sum of two absolutely convergent series also converges absolutely
c 00
6-19. Give an example of an absolutely convergent series
so that
j=1
c c
c c x
00
aj
aj f
j=1
laj 1
j=1
x
6-20. Let x
a j be an absolutely convergent series and let ( b j ] z l be a bounded sequence. Prove that
J=1
a j b j converges absolutely.
j=1
6-21. Determine which of the following series converges. If it converges, determine if it converges absolutely.
j=1
c c x
X
6-22. Limit Comparison Test for series. Let
j=1
if lim J+W
b j be series with positive terms. Prove that
a j and
j=1
9 = c > 0, then either both series converge or both series diverge. bj
Hint. For E > 0 there is an N
E
N so that for all j
p N we have (c - e ) b j 5 a j 5 (c
+E)bj,
00
a , be a conditionally convergent series and let z
6-23. Let j=1
c
E
W.Prove that there is a bijective function
00
u : W + N such that
a,(j)
= z.
j=1
Hint. Mimic the proof of the “e’’ part of Theorem 6.18. Oscillate about z rather than increasing
c n
with k . Use that the limit of the terms is zero and that the distance of the partial sum
a,(jj
to z
j=1
ultimately is always bounded by some la, 1 with rn being large.
cc
a,(;, j ) and
i=l j = l
i=l j=1
cc x00
cow
6-24. Reverse the roles of
x x
a(;,j ) as indicated in the proof of Proposition 6.22
;=I j=1
i = l j=1
cc 1 x x
6-25. Let { ~ ( j , j j ] ; = ~be a family of real numbers so that the double series
cc x
Prove that
1 converges.
x
a ( ; , j ) converges to a number L and for all bijections a : N x W
;=I j=1 xDs
a , ( j , j ) converges to the same number L . i=l j=1
a(;,jj
;=I j=1
+ W x N the sum
Chapter 7
Some Set Theory Up to this point our development of analysis had minimal need for details of set theory. However, to define the outer Lebesgue measure of a set (see Definition 8.1), we will cover the set with an infinite family of intervals and sum the lengths. To be able to compute the sum as a series, we can use at most as many intervals as there are natural numbers. Surprisingly enough this is not trivial, because infinite sets can come in different sizes. This chapter presents the fundamentals on arbitrary families of sets and on the notion of size for sets. These ideas will be needed immediately for outer Lebesgue measure and they will be useful throughout our work with measures. All sophisticated set theory needed in this text is presented in this chapter.
7.1 The Algebra of Sets Recall from the introduction to Chapter 1 that the terms “set” and “element” remain undefined. To be able to work with arbitrary families of sets, we need to formally define them and show some fundamental properties. Because every family of sets must consist of subsets of another set, we start with the power set of a set.
Definition 7.1 Let S be a set. The power set P ( S ) of S is the set of all subsets of S. Definition 7.2 Let S be a set. A family of sets C is simply a set of subsets of S , that is, P ( S ) . An indexed family of sets is denoted {Ci}iEz,where it is understood that there is a function from the index set I to P ( S ) that maps each i E I to Ci.
C
So while families of sets do not introduce any new set theoretical ideas, we call them “families of sets” rather than “sets of sets” to indicate that we are now considering objects whose elements are set theoretically more complicated than just elements that cannot be decomposed any further. A family of sets can always be turned into an indexed family of sets by making each set its own index. Conversely, an indexed family can be turned into a nonindexed family by considering the set C := {Ci : i E I } , but this process will collapse sets Ci = Cj with i j into one set in C. In our work, we may need repeated sets, so most of the families of sets we consider will be indexed.
+
117
118
7. Some Set Theory C1
3
@
( a ) c2
c , n c* 8 1 c3
Figure 16: Venn diagrams of the intersection ( a ) and the union (b)of three sets. Nonetheless, proofs for both kinds of families are very similar and we will present both notations in this section. The most elementary operations for families of sets are unions and intersections (also see the Venn diagrams in Figure 16).
Definition 7.3 Let C be a family of sets in S. Then we defne the union of C as the set := {x E S : (3C E C : x E C ) } .Zf[Ci)iEz is an indexed family, the union is
UC denoted U C i := {x E S : (3i E I : x E C i ) } . i€f
Definition 7.4 Let C be a family of sets in S. Then we defne the intersection of C as the set := {x E S : (VC E C : x E C ) } . Zf[Ci}ie~ is an indexed family, the
nC n
intersection is
~i
:= {x E
s : (vi E I : x E ~ i ) } .
ieZ
The relation between complements (see page 1) and unions/intersections is described by DeMorgan’s Laws.
Theorem 7.5 DeMorgan’s Laws. Let C and [Ci}i,~ be families of sets and let X be another set. Then I.
x \ Uci = iEZ
2.
x \ ciand x \ Uc = n i x \ c : c E el. i€I
x \ n c i = U x \ cia n d X \ ief
nc
=
U{x \ c : c E el.
ief
Proof. Proofs f o r indexed and nonindexed families are very similar: Hence, we will only present one proof f o r each part. For part 1, we will prove the containments X \ G n { X \ C : C E C} and
u
UC
X \ C 1 n [ X \ C : C E C},which (see introduction to Chapter 1) shows the two sets are equal. For the containment relation X \ C C n { X \ C : C E C},consider an arbitrary
x
E
X\
u
UC. Then x 6 UC,which means that x
6
C for all C E
C. But
then
x E X \ C for all C E C, which means x E n { X \ C : C E C}.This proves that X \ C is contained in n { X \ C : C E C}.
u
7.1. The Algebra of Sets
119
For the reversed containment relation X \ arbitrary x
E
n { X \ C :C
x $ C. But then x @
E
uC 2
C). Then for all C
UC,which means x
E
X
E
\
n { X \ C : C E C),consider an C we have x
X \ C, that is,
E
UC. This proves that X \ UC
contains n i x \ c : c E el. Because each set is contained in the respective other set, the sets must be equal. It is also possible to prove part 1 with a chain of equalities, like part 2 below (Exercise 7-la). The proof of part 1 for indexed families of sets is similar (Exercise 7-lb.). We prove part 2 for indexed families.
=
{X E
x
: (3i E I : x E
x \ ci)}= UX\ Ci. icI
It is also possible to prove part 2 with a mutual containment argument as done for part 1 (Exercise 7-lc). The proof of part 2 for (nonindexed) families of sets is similar (Exercise 7-ld).
Standard Proof Technique 7.6 To prove equality of two sets A and B it is common to prove mutual containment, that is, A C B and B 5 A . This is similar to proving that two numbers are equal by proving that one number is both greater than or equal to and less than or equal to the other one. 0 Ordered pairs and products also have formal set theoretical definitions.
Definition 7.7 Let A and B be sets and let a E A and b E B. Then we define the ordered pair ( a , b ) to be ( a , b ) := { a , { a ,b } } . ZfA1, . . . , A, are sets and ai E Ai f o r i = 1, . . . , n, we dejine ( a l , . . . , a,) := { ( a l , . . . , a,-i), { ( a l , . . . , a n p i ) ,a,}} and call it an ordered n-tuple. Definition 7.8 Let A 1 , . . . , A, be sets. Then the product A 1 x . . . x A, of these sets is dejined to be the set of all ordered n-tuples ( a l , . . . , a,) so that f o r all i = 1, . . . , n
n n
we have ai
E
Ai. The product is also denoted
n
Ai. Moreovel; for j
E
{ 1, . . . , n ) we
i=l
n
define the natural projection T A , :
Ai + A j onto A j by
T A (~a l ,
. . . , a,)
:= a j .
i=l
We will also abbreviate the natural projection onto the jthfactor as T,. For a visualization of the product of two sets, see Figure 17. The definition of a product is fundamental for the formal definition of functions and relations. Because we will not need this level of detail in this text, these definitions have been relegated to Appendix B.2.
Theorem 7.9 Let ( C i j i , ~be a family of sets and let A be a set. Then the equalities Ci = A x Ci hold. A x Ci = A x Ci and A x
u u
n
7. Some Set Theory
120 f ~
7r2(a) = a 2
I
----
---
Figure 17: The product of two sets is a “rectangle’’ with the natural projections ni providing the “coordinates” of an element.
Proof. Left to Exercise 7-2. The distributive laws express the relationship between unions and intersections.
Theorem 7.10 Distributive laws. Let { A i } i eand ~ { B j } j eJ be indexedfumilies of sets. Then Ai n Bj = Ai n Bj and Ai U Bj = Ai U B j . ieI jeJ (i,j)eIxJ ieI jcJ ( i , j ) E I xJ
u u
u
Proof. For the first equation let x
E
u nu Ai
iel
x
E
Ai and a j
E
J so that x
For the reverse inclusion, let x thatx E Ai n B j
U A in
E
B j , which means x
Ai
E
Ai f?B j . Then there is an (i, j )
E
n
E
I , then x
x E Ai,
u B j for all j
Ai U
Ai C
E
ieI E
ieI
J , and hence x
B j , then x
Ai U
I x J . For the reverse inclusion, let x
for all i
Ai
nBj.
E
I x J so
E
Ai U Bj
Bj.
ieI E
u
n Bj C
I so that
( i ,j ) E I x J
For the second equation, first note that if x for all (i, j )
E
( i , j ) e I xJ
u
E
u
B, , Then there are an i
jeJ
jeJ
iE1
n
0
jeJ E
n
Ai U B j . If x
E
Ai
( i .j ) E I x J
0 Bj and if x
@ Ai, for some io
n cn
E
I, then
jEJ
E
Bj
jeJ
Ai U
ieI
Bj. jEJ
Exercises 7- 1. Proving DeMorgan’s Laws. (a) Prove part 1 of Theorem 7.5 for families of sets using a chain of equalities. (b) Prove part 1 of Theorem 7.5 for indexed families of sets.
(c) Prove part 2 of Theorem 7.5 for indexed families of sets using a mutual containment argument.
7.1. The Algebra of Sets
121
(d) Prove part 2 of Theorem 7.5 for nonindexed families of sets 7-2. Prove Theorem 7.9. That is, let { C i ) i €be ~ a family of sets and let A be a set. (a) Prove that A x
u u Ci =
icI
A x Ci.
(b) Prove that A x
icI
nCi n
A x Ci.
=
icI
icI
7-3. Let A , B , C be sets. Rove each of the following. (a) C \ ( A n B ) = (C \ A ) u (C \ B ) (c) (e)
c u ( A n B ) = (cu A ) n (cu B ) ( A \ B ) n c = ( A n c)\ ( B n c)
c n ( A u B ) = (cn A ) u (cn B )
(d)
(0 If A , B 5 C, then A \ B
X be a set. An algebra is a set of sets A g P ( X ) such that 0 E A, if A n
and if A j E A for all j = 1, . . . , n , then
= A n (C \ B ) .
C = 0 so that any two sets in the family intersect.
7-4. Give an example of a family of sets with 7-5. Let
(b) C \ ( A U B ) = (C \ A ) n (C \ B )
u
E
A, then X \ A
E
A,
A j E A.
j=l
n
(a) Prove that if A j
E
A for all j = 1, . . . , n , then
A,
E
A.
j=l
(b) Let X be a set. Prove that the power set of X is an algebra. (c) Let X be a set. Prove that A := { A g X : A or X \ A is finite ) is an algebra. (d) Prove that an algebra need not contain countable unions of its elements.
g X and B g Y be sets. Prove that (X x Y ) \ ( A x B ) = [ ( X \ A ) x (Y \ B ) ] U [ (X \ A ) x B ] U [ A x ( Y \ B ) 1.
7-6. Let A
7-7. Let A , B , C, D be sets. Prove that ( A x B ) n (C x D ) = ( A n C) x ( B n D ) . 7-8. Let f be a function whose domain is contained in X and whose range is contained in Y . X x Y : y = f ( x ) ) ] is the domain o f f .
(a) Prove that nx
[ {(x, y ) E
(b) Prove that n y
[ {(x,y ) E X
x Y : y = f ( x ) ) ] is the range of f.
7-9. Unions and intersections versus functions. Let X , Y be sets, let f : X -+ Y be a function, let { X i ] i c l be a family of subsets of X and let { Y i ) i € be ~ a family of subsets of Y . Prove each of the following.
(e) For f ( x ) := x 2 there are sets A , B with f [ A n Bj f f[A]n f [ B ] . injective .
7. Some Set Theory
122
7.2 Countable Sets Two sets are considered to be of the same size iff their elements can be matched oneby-one without any leftovers on either side. Recall that bijective functions were defined in Definition 2.24.
Definition 7.11 Two sets A and B are called equivalent iff there is a bijective function f : A + B. With this language, Definition 2.25 says that a nonempty set is finite iff it is equivalent to a subset { 1, . . . , n ) of N. In Figure 3 on page 36, only the two sets on the right are equivalent. For analysis, the most important size distinction between infinite sets is if they are countable or uncountable. Countable sets are, roughly speaking, sets for which it is possible to "count" the elements, where the counting process may stop or it may not.
Definition 7.12 A set C is called countably infinite iff there is a bijective function f : N -+ C. A set C is called countable iff C is finite or countably infinite. Subsets of a set cannot be larger than the set, so it is natural that subsets of countable sets are countable.
Theorem 7.13 I f C is countable and S
C , then S is countable.
Proof. If S is finite, there is nothing to prove. Hence, we can assume that S is infinite. Let f : N + C be a bijection and let n1 := min f - ' [ S ] . Fork E W, we define nk+l recursively. Once n l , . . . , nk are chosen, let nk+l:=min f - ' [ ~ l \ { n l ,. . . , n a ) ) .
(
Define g : N -+ S by g ( k ) := f ( n k ) . Because all nk are in f - ' [ S ] , g maps N into S. Because no two nk are equal and f is injective, g is injective. Finally, suppose for a contradiction that g is not surjective. Let b be the smallest element of f-'[S \ g[N]] and let the number of elements of f - ' [ S ] n { l , . . . , b - 1) be k . Then f - ' [S] n { 1, . . . , b - 1) = { 1, . . . , n k } (with no = 0 in case k = 0) and b = min f - ' [ S ] \ { n l ,. . . , n k ) ) , which means b = nk+l, a contradiction.
(
Theorem 7.13 says that to prove that a set is countable, it is enough to embed a copy of it into a countable set. In the language of analysis, it is good enough to find an "upper bound" that is countable. We will use this idea repeatedly in the remainder of this section. Interestingly enough, even sets that look uncountable may be countable.
Lemma 7.14 The set N x W is countable. Proof. The function f : N x W + W defined by f (in, n ) := 2m3n is injective. Thus the set N x N is equivalent to a subset of W, and hence by Theorem 7.13 W x W is countable. Note how Theorem 7.13 allowed us to avoid the detailed construction of a bijective function between N and W x N.Defining such a function is not trivial. Exercise 7-10 explicitly presents a bijective function between N x N and N.
7.2. Countable Sets
123
Definition 7.15 Two sets A and B are called disjoint ifSA f l B = 0. A family {Ci}i,r is culled pairwise disjoint @for all i # j we have Ci n Cj = 0. Theorem 7.16 Countable unions of countable sets are countable. Proof. Let {C,}EE1 with a E N U (00) be a countable family of countable sets. For each C,, let B, := C, \
u
u u ‘‘2” u a
a
n-1
C,. The containment ‘‘5”
B, =
C j . We claim
j=1
n=l
n=l
a
follows from B,
C, for all n E
N. For
let x
smallest natural number so that x E C,. Then x
E
C,. Let n
E
N be the
n=l
B,, which proves ‘‘2.” Moreover,
E
u
m-1
\
the B, are pairwise disjoint, because if m < n , then B, = C,
Cj L C, and
j=l
u
n-1
B, = C, \
I
Cj G Cn \ C,. Now for each n let B, = bk : k
j=1
E
I,],where Zn is N
or a set of the form 11, . . . , m,}. Then f ( n ,k ) := bf:is a bijective function between
u a
{(n,k)E N x
N
: n 5 u,k E
I,) G N
x Nand
a
UC,.
B, =
n=l
n=l
00
Thus u C n is n= 1
countable. Maybe the most surprising fact about countability is that the rational numbers are countable. The reason is that the order in which the rational numbers are counted has nothing to do with their natural ordering.
Theorem 7.17 The rational numbers Q are countable.
u{ n ”-
M
=- 0) =
N] , that is, the positive rational d numbers are a countable union of countable sets, and hence by Theorem 7.16 they are countable. Similarly, ( r E Q : r < 0 ) is countable and of course {0}is finite. Now Q = ( r E Q : r > 0 } U {r E Q : r < 0 ) U (0} is countable by Theorem 7.16. Proof. We have {r E Q : r
- :n E
ff=I
Exercises 1 7-10. Prove that the function f : N x W + W defined by f ( m . n ) := - ( m 2 bijective. (For a visualization, consider the middle of Figure 18.)
+ n - l)(m + n - 2) + n is
7-11. Prove that the set of integers Z is countable. 7-12. Prove that the set of integers Z is countable by constructing a bijective function f : N + Z. Hint Figure 18(a). 7-13. Prove that the set of dyadic rational numbers is countable. 7-14. Use Theorem 7.16 to prove Lemma 7.14. 7-15. Give a direct proof that the union of TWO countable sets is countable. 7-16. Prove that if C1, , . . , C, are countable, then C1 x C2 x
.. . x
C, is countable.
Figure 18: Some standard visualizations for countability arguments. Part ( a ) shows the construction for a direct proof that the union of two countable sets is countable. Part ( b ) shows an explicit bijective function between W and the product of two countable sets. Part ( c ) shows the idea behind the proof that (0, 1) is not countable.
cc 3030
7-17. Let ( a ( i , j ) ] E = lbe a family of nonnegative numbers. Then the double series
a ( i , j ) con-
i=l j = 1 30
verges iff for all bijections
D
:W
+ N x N the sum
a o ( i ) converges. Furthermore, in this case i=l
the values are equal. Hint. This is similar to the proof of Proposition 6.22.
7.3 Uncountable Sets Not all sets are countable. In fact, (see Exercise 7-18) there is an infinite hierarchy of sizes for infinite sets, because any time we form a power set, we obtain a set that is strictly larger than the set we started with.
Theorem 7.18 I f X is a set, then X is not equivalent to its power set
Pix).
Proof. Suppose for a contradiction that f : X --f P ( X ) is a bijection. Define B := {x E X : x # f ( x ) } . Because f is surjective, there is a b E X with B = f ( b ) . Now b E B would imply b E B = f ( b ) and by definition of B this would mean b $ f (bj = B . Thus we infer b # B . But then b # B = f (b),which by definition of B forces b E B , a contradiction. For analysis, we typically do not need the full hierarchy. Instead we only need to distinguish sets that are countable from those that are not.
Definition 7.19 A set U is called uncountable i f f it is not countable. The real numbers, which are fundamental for analysis, are not countable.
Theorem 7.20 The interval ( 0 , 1) is uncountable.
125
7.3. Uncountable Sets x
,Itl,=, 1
.i j In=] CD
w; !
*
Figure 19: Cantor sets are the intersection of a sequence of unions of closed intervals where at each step only the left and right segments of each interval are kept.
Proof. Suppose for a contradiction that (0, 1) was countable. Then there is a sequence such that for every x E (0, 1) there is an n E N so that x = x,.
[X~}F=~
For each n
E
N,let
1
00
xAk)
I k l
be the decimal expansion of x, as in Proposition 6.6.
For each n E N,let y , be a number in the set [ I , 2, 3 , 4 , 5 , 6 , 7 , 8) \ [xf)] . Then ( y n } z l is a decimal expansion of a number y E (0, 1). However, for all n E N we have that yn f x;), and hence y # xn,contradiction. The remainder of the proof that W is uncountable is left to Exercise 7-19b. The uncountability of the real numbers shows that, even though in real life we work mostly with rational numbers, there are many more irrational numbers than there are rational numbers.
Theorem 7.21 The set W \
Q of irrational numbers is uncountable.
Proof. By Exercise 7-19b, the real numbers are uncountable. Now for a contradiction suppose that W \ Q was countable. Then R = (R\ Q) U Q would be countable by Theorem 7.16, contradiction. We conclude this section by defining Cantor sets. These sets are very useful to construct counterexamples which show that certain hypotheses in analytical theorems cannot be dispensed with. Because these counterexamples can be considered a bit pathological, we defer their construction to the exercises and we will only refer to Cantor sets when necessary. Cantor sets are constructed from a sequence of unions of closed intervals so that in each step we remove the middle of each interval. Figure 19 shows the first six stages in the construction of the ternary Cantor set, which is constructed by successively removing the middle third of the intervals at each stage.
1 Definition 7.22 Let [ a ,b ] be an interval and let 0 < q < -. Then we define the left 2 part o f [ a ,b ] to be the interval L q [ a ,b ] := [a, a q ( b - a ) ] and the rightpart to be the interval & [ a , b ] := [b - q ( b - a ) , b ] .
+
7. Some Set Theory
126
1 For a sequence Q := {qn]zl with 0 < qn < - f o r all n E N define CF recursively 2 as follows. Let Cf := ZFo := [0, 11 and once the set Cf is dejined as a union ofpair2”
wise disjoint closed intervals
u
Zi,n, Q f o r i = 1 , . . . , 2n let Z2i-l,n+l Q
:= Lqn+l[Ii:],
i=l
u
2n+l
:= Rqn+l[ Z e ] and let Cn+l Q := let Z2i,n+l Q
00
Zj:+l.
Then C Q :=
i=l
Cf is called n=l
the Cantor set associated with the sequence Q.
Even though the construction looks like it should only leave the boundary points of the intervals, Cantor sets are in fact uncountable. The details can be explored in Exercise 7-25.
Exercises 7-18. Construct a sequence (Pn}F=l of infinite sets so that no two sets P, and Pn+l are equivalent, but for each n E N there is an injective function f n : Pn -+ P,+1. 7-19. Containment and uncountable sets (a) Let U ,V be sets. Prove that if U is uncountable and U C V, then V is uncountable (b) Prove that B is not a countable set. 7-20. Prove that every uncountable set contains a countably infinite subset 7-21. Prove that the set of all functions from W to (0, 1) is uncountable 7-22. Let { a , ] j G lbe an uncountable family of positive numbers. Prove that there are an so that ain P E for all n E W. countable subfamily {qn
E P
0 and a
7-23. Let F : [ a , b] + R be a nondecreasing bounded function. Prove that F can have at most countably many discontinuities. Hint.Suppose the set of discontinuities is uncountable and use Exercise 3-21 to conclude that there must be an E so that the set of all x with lim F ( t ) - lim F ( z ) > E is uncountable. z+x+
z+x-
7-24. Prove that for every countable subset A g R there is a nondecreasing function f : B + [O, 11 that is continuous on W \ A and discontinuous at every a E A. Hint. With A = {a, : n
E
W) set f ( x ) := ,:an i x
2”
7-25. Cantor sets (a) Prove that for any sequence Q = (q,)F=l of numbers qn
E
tion from C Q to the set of all sequences of zeroes and ones. Hint. Forx
E
C Q andn 3 1, seta,(x) := Oiffx E I ?
Jsn
(
0,
:>
-
there is a bijective func-
= Lq,,
thatis, iffwehave
to turn “left” at the nth stage of the construction to keep x in the interval. (b) Prove that CQ is uncountable. Hint.Exercise 7-21. (c) Prove that the set of endpoints of the intervals I f n that make up the C :
is countable.
Q . (d) Prove that every x E C Q is the limit of a sequence of endpoints of intervals I 1.11
Chapter 8
The Riernann Integral I1 The Dirichlet function in Example 5.26 shows that functions with too many discontinuities may not be Riemann integrable. This is because for a discontinuity d there is an E > 0 so that, independent of 6, the numbers inf { f (x) : x E (d - 6 , d 6 ) ) and sup { f (x) : x E (d - 6 , d a)} will always be at least E apart. By Theorem 5.25, too many such discontinuities will cause the function to not be Riemann integrable. At the same time, Proposition 5.12 shows that a function can have some discontinuities and still be Riemann integrable. To determine when a function is Riemann integrable, we need to determine “how many” discontinuities are acceptable. For a graphical motivation, consider Figure 20 on page 133. Section 8.1 introduces outer Lebesgue measure, which is the tool to measure “how many” discontinuities a function has. Section 8.2 introduces Lebesgue’s integrability criterion and Section 8.3 shows how this criterion allows us to easily obtain new results about the Riemann integral. We conclude in Section 8.4 with improper integrals.
+
+
8.1 Outer Lebesgue Measure Outer Lebesgue measure covers a set with open intervals and assigns the infimum of the sums of the lengths of these intervals as the measure (“size”) of the set. With respect to Riemann integrability, we should note that if all discontinuities of our function are trapped in a union of intervals, then outside of this union of intervals the function is continuous and Riemann integrability should not be a problem there.
Definition 8.1 For an open interval I = ( a ,6 ) in R,we define I I1 := b - a. For any set S C R,we define the outer Lebesgue measure of S to be cx: j=1
j=1
where we set h ( S ) = oc, ifnone of the series in the set on the right converge.
127
8. The Riemann Integral II
128
The proof of Proposition 8.5 will show why we use open intervals in the definition of outer Lebesgue measure. We first turn our attention to sets with outer Lebesgue measure zero. These sets, and properties that hold on the complement of such a set, will be of particular importance for the Riemann integral.
Definition 8.2 A set of outer Lebesgue measure zero is called a set of measure zero or a null set. A property P ( x ) such that h ( ( x E D : P ( x ) is not true }) = 0 is said to hold almost everywhere in D. Almost everywhere is also abbreviated as a.e. Countable sets are considered “small” in set theory and they are also “small” with respect to outer Lebesgue measure.
Proposition 8.3 Countable subsets of EX have outer Lebesgue measure 0. Proof. Let C = {cl, c2, . . .} be a countable subset of R and let E > 0. For j E N let 00 00 00 E l 1 I . ‘= c . - - . ThenCsUZjandCIIil=Cs-=s.Thus 2 2j 2J j=1 j=l j=l h ( C ) = 0.
(,
Although the proof of Proposition 8.3 is quite simple, it takes a little to get used to the result. Recall that Q is countable, which means it is a null set! Exercise 8-3d will show that null sets can be uncountable, too. Of course, not all subsets of R are null sets. To prove that outer Lebesgue measure provides the “right” measure for intervals, we need the Heine-Bore1 Theorem. The conclusion of the Heine-Bore1 Theorem is the inspiration for the topological definition of compactness (see Theorem 16.72). Until we formally introduce compactness in Section 16.5 we will rely on Standard Proof Technique 2.28 as used in the proof of the Heine-Bore1 Theorem.
Theorem 8.4 Heine-Bore1 Theorem. Let [ a ,b] c R and let Z be a family of open intervals with [ a ,b] 2 I . Then there are$nitely many intervals I1, 1 2 , . . . , I,, E Z
u
n
SO
that [ a ,b]
IEZ
c U Zj. j=l
Proof. Suppose for a contradiction that
Let c := inf C. Because a E I for some I E Z we infer c > a. Let J E Zbe an open interval with c E J. If c < b there is a 6 > 0 with (c - 6, c 6) 5 J n [ a ,b ] . If c = b there is a 6 > 0 with (c - 6, b] 2 J n [ a ,b]. By definition of c, there
+
are finitely many I1, 1 2 , . . . , Z,
E
Z so that [ a ,c - 61 s
u n
j=1
I j . But then if c
b we
i
8.1. Outer Lebesgue Measure
+S] C J U
obtain [ a , c
u
u
129
n
Zj,
contradicting the definition of c. Hence, c = b and we
j=l
n
obtain [ a ,b] C J U
Zj, implying
C = 0, a contradiction.
j=1
Proposition 8.5 Let a , b E R with a < b. Then h ( [ a ,b ] ) = b - a. Proof. Let
E
> 0. Because [ a ,b] 2
b
+ E4 ) U
+
fi (--
-) 2 . 2 n 92 . 2 "
n=2
&
&
we
obtain h ( [ a ,b ] ) < ( b - a ) E . Because E > 0 was arbitrary, h ( [ a ,b ] ) I b - a. To show the reverse inequality, let [ Z j } c l be a countable family of open intervals
u 00
so that [ a ,b] E
Z j . By
the Heine-Bore1 Theorem, there is a finite number of inter-
j=l
u n
vals
Zj,,
. . . , Zj,
SO
that [ a ,b] C
Zj,.
For k = 1, , . . , n let
Zjk
= ( a k , bk). Without
k=l
loss of generality assume that no interval Z j k is contained in another. Reorder the intervals so that for k = 2, . . . , n we have bk-1 5 bk. Then for all k = 2, . . . , n we have a k - 1 P ak. Moreover, bn > b, a1 -= a and for all k = 2, . . . , n we infer ak ibk-1. n
Hence,
bk - ak > bl - a1
n
bk - bk-1 = bl - a1
+ bn - bl
1 b - a , which
k=2
k=l
co
IZ, 1
implies
+
> b - a , and hence h ( [ a ,b ] ) 3 b - a .
J=1
Aside from its use related to Riemann integration, outer Lebesgue measure also is the foundation for Lebesgue integration. We conclude this section with some of the properties of outer Lebesgue measure, which will also be helpful for some exercises.
Theorem 8.6 The properties of outer Lebesgue measure h. With 00 defined to be greater than all real numbers and the sum of a divergent series of nonnegative numbers being 00 we have the following. 1. h ( 0 ) = 0. 2. ZfA C B , then h ( A ) 5 h ( B ) . 3. Outer Lebesgue measure is countably subadditive. That is, for all sequences 00
{An]El ofsubsets An 5 R the inequality h Proof. For part 2, let A & B . Then we obtain the following.
130
8. The Riemann Integral II
u 30
IZj 1 : A G
5
inf
r
each Z j is an open interval
u
Zj,
each
00
IZj
I :B
s
j=l
=
Zj,
j=1
j=1
Zj
is an open interval
j=1
h(B).
For part 1, note that for all n E
N we have 0
1- ,:
I 1
L 7c
I
2 and thus h ( 0 ) 5 -, which
71
-71
via part 2 implies h(0) = 0. For part 3, first note that there is nothing to prove if the right side is infinite. So assume the right side is finite and let E > 0. For each n E M, find a countable family
{
I u uu 00
Lemma 7.14 the family Zn of this family is such that
1 j.n=l
is a countable family of open intervals. The union " 0 0
00
An 5
n=l
u xi
Zj" =
n=l j=1
Zj". By Proposition 6.22, the
j,n=l
convergence behavior and value of a doubly infinite sum of nonnegative numbers do not depend on the order of summation and by Exercise 7-17 it does not matter if we represent the sum as a double sum or a single sum. Thus we can conclude
Because E was arbitrary, this proves part 3.
Exercises 8-1. Let [ A n ) E lbe a countable family of null sets. Prove that h
8-2. Let a. b E W with a ib. Prove that h ( ( a ,b ) ) = b - a . Hint. For ''z"approximate the open interval "from the inside" with closed intervals
8-3. Let Q = ( q n ] 2 1be a sequence of numbers qn E
and let C Q be the associated Cantor set
as in Definition 7.22. We will use the notation of Definition 7.22 throughout this exercise.
j=l
8.2. Lebesgue ’s Criterion for Riemann Integrability
131
Hint.Prove that for a finite union of painvise disjoint intervals the outer Lebesgue measure is the sum of the lengths of the intervals. This requires repeated use of the argument in the proof of Proposition 8.5. (b) Prove that
{ fi
2q,} converges.
j=1
n n
(c) Prove that h (CQ) = n
%$24j
j=1
Hint. For ‘‘2,”first consider a family Z of open intervals so that C Q E VZ1,12,. . . , In E Z : C Q n [O, XIg‘
UZ.Prove that
= 0, by assuming it is j=1
not empty and showing that inf C E CQ, which leads to a contradiction similar to the proof of the Heine-Bore1 Theorem. (It also helps to look ahead to the proof of Lemma 8.11 for a similar argument.) Then show that for any countable family
u
{Zj]gl with C Q
u 03
I, there
j=1
m
are intervals Iji , . . . , Zjk with C Q
I jk . Conclude that there must be an n E W so that
k=l
m
k=l
(d) Prove that for any q
E
(0, -:>
the constant sequence Q = {q)z, yields a Cantor set C Q of
measure zero. Note. By Exercise 7-25b in Section 7.3 Cantor sets are uncountable. This means there are uncountable sets of measure zero. (e) Use Q =
{
2n;;
n=l
~
to prove that there are Cantor sets that are not of measure zero
n
n+l
n
Hint. Prove that for all n
E
N we have
-.
2qj 2 1 -
j=i
j=2
2J
(0Prove that there are Cantor sets whose Lebesgue measure is arbitrarily close to 1. Hint. Fork
E
W fixed, use n
+ k instead of n in Exercise 8-3e.
8-4. Use the Heine-Bore1 Theorem and the axioms for W except for Axiom 1.19 to prove the BolzanoWeierstrass Theorem. Hint. We will ultimately do this in a more abstract setting in Theorem 16.72. 8-5. Prove that i f f : [a, b] --f all x E [ u , b ] .
R is continuous and A ( ( x
E [ a ,b] : f ( x ) # 0)
) = 0, then f ( x ) = 0 for
8-6. Prove that if f,g : [a. b] + B are continuous almost everywhere, then f everywhere.
8.2
+ g continuous almost
Lebesgue’s Criterion for Riemann Integrability
The oscillation of a function is a quantitative measure “how discontinuous” the function is. It is the last tool we need to characterize Riemann integrable functions.
8. The Riemann Integral II
132
Definition 8.7 Let f : [ a ,b ] -+ R be a boundedfunction and let x E [ a ,b]. For any interval I let w f ( I ) := sup { If(y) - f ( z ) / : y , z E I n [a,b ] }be the oscillation off over the interval I . Define the oscillation o f f at the point x E [ a ,b] as the infimum w f ( x ) := inf { w f ( ~ :)x E J , J is an open interval }. Exercise 8-8 shows that the oscillation measures the height of “jumps” in the function and Exercise 12-23 shows that the oscillation is also a measure of the size of oscillations. Regarding the details of the definition, Exercise 8-10 shows that it is important that we use open intervals in the definition of the oscillation wf (x) at a point.
Theorem 8.8 The boundedfunction f : [a,b ] -+ R is continuous at x = 0.
E
[a,b ] ij7
Of(X)
0. Then there is a Proof. For “+,”let f be continuous at x E [a,b] and let E & S > 0 such that for all y E [ a ,b] with Iy - X I < 6 we have f ( y ) - f (x)l < -. Then 2 for all y , z E ( x - 6, x + 6 ) r l [a,b] we obtain
1
+
Hence, w f ( x ) 5 w f ( ( x - S, x 6)) 5 E and because E was arbitrary we conclude that w f ( x ) = 0. Conversely, for “e,” let x E [a,b] with w f ( x ) = 0 and let E > 0. Then there is an interval (c, d ) with x E (c, d ) such that w f ( ( c ,d ) ) < E . Let 6 := min{x - c, d - x} unless x = a , in which case we set 6 := d - x, or x = b, in which case we set S := x - c. Then for all y E [a,b] with lx - y / < 6 we have y E (c, d ) , and hence l f ( x ) - f ( y ) / < E . Thus f is continuous at x. The next two lemmas show that, when it comes to Riemann integrability, small oscillation means that lower and upper sums can get close to each other, while nonzero oscillation on a set of positive outer Lebesgue measure prevents Riemann integrability (also see Figure 20).
Lemma 8.9 Let f : [a,b ] --+ R be bounded. I f w f ( . x ) < & f o rall x E [ a ,b], then there is apartition P of [ a ,bl so that U ( f , P ) - L ( f , p ) < 61b - al.
+
Proof. For every z E [ a ,b ] ,there is an open interval Iz = ( z - a,, z 6,) so that w f ( I , ) < E . By the Heine-Bore1 Theorem, there are finitely many z1, . . . , zm E [ a ,b]
. Let P
:= ( a = xo < x1 < . . . < x n = b}
j=l 6. be the set comprised of a , b and the endpoints z j k 2 that are in [ a ,b ] . Then each 2 interval [xi-l, xi] is contained in an interval Izl and consequently, with notation as in Definition 5.13, we infer Mi - mi < E . But then n
U ( f , P ) - L ( f ,p )
=
n
Mi Axi i=l n
n
mi Axi = c ( M i - m i )Axi i=l i=l
8.2. LebesgueB Criterion for Riemann Integrability
133
A
4
(“1
(h)
Figure 20: The area of the boxes indicates the difference between the lower and the upper sum of the function for the given partition. Boxes are tall where the function has large slopes or discontinuities. Comparison of ( a ) and ( b ) shows that as the norm of the partition goes to zero, the height of the boxes goes to zero where the function is continuous. Where the function is discontinuous the height remains bounded away from zero (consider the two discontinuities). Thus f can only be Riemann integrable if the discontinuities (unavoidable tall boxes) can be trapped inside a set of intervals whose total length is small. This is the idea for the Lebesgue criterion. which finishes the proof.
Lemma 8.10 Let f : [ a ,bl -+ 24 be bounded. Ifh({.x theti f is not Riemann integrable.
Proof. Because { x
E
[ a .b ] : w f ( x ) > 0 } =
x
[ a .61 : w f ( x ) > 0 ) ) > 0,
E
E
[a. b] : w f ( x ) >
/=I
is an E > 0 so that L := h ( { x E [ u , b] : w , f ( x ) > E } ) > 0. Let P = { a = xo < X I < . . . < xn = b } be any partition of [a.b ] . Define the set D := { x E [ a .b] \ {xo, . . . , x,?} : w f ( x ) > E } . By Theorem 8.6, we obtain
+ CA ( { x j ~2 ) A 11
L
2 A(D)= A ( D ) 2
h ( { x E [u, b] : w f ( x ) >
=L,
E})
i 1 D
j=1
uUIxj}
which means h ( D ) = L . With i l , . . . , ik E { 1 , . . . , n } being the indices of the intervals ( x i - ] ,x i ) that ink
tersect D we obtain D
k
u(x;,-l. xi,)
j=1
Axi,
and
2 h ( D ) = L . But then in each
j=1
interval ( s i , - l .x;,) there is an s j with w , f ( s j ) > E , and hence w f ( x i , - l , x i , ) > each j = 1. . . . , k we infer M;, - ni,, > E and thus t1
E.
k
U ( f . P ) - L ( f . P ) = C ( M i - m;)Axi 2 C ( M i , - m;,)Ax;, > E L > 0. i=l
j=1
For
134
8. The Riemann Integral II
Because P was arbitrary we have shown that for any partition P of [ a ,b ] we have U (f ,P ) - L ( f ,P ) > E L > 0, where E and L are fixed. By Theorem 5.25 this means that f is not Riemann integrable. Lemma 8.10 shows that only functions that are continuous almost everywhere have a chance to be Riemann integrable. Lemma 8.9 shows that if we can “trap” the discontinuities in a small enough set, a function should be Riemann integrable. The only obstacle left is that outer Lebesgue measure works with countably many open intervals. To obtain a complement that is made up of closed intervals, it would be nice if we could use finitely many open intervals to cover the set of discontinuities. The next lemma shows that this is possible.
Lemma 8.11 Let f : [ p , b]- -+ E% be a bounded function and for each p > 0 define B, := {x E [a,b ] : w f ( x ) 2 p } . l f Z is a family of open intervals such that B,
c
u
n
I , then there arefinitely many 11,1 2 , . . . , I,
E
u
Zso that B ,
IEZ
Zj.
j=1
Proof. Suppose for a contradiction that
Then c := inf C 5 b. We first claim c E B,.
There is a sequence { a , } z l of elements of B, so that
a, < c and lim a, = c. Let
E
n+o3
> 0. Then there is an n
E
N so that la,
- CI <
&
-. 2
E + -)) 2 p . Because of 2 6 c ( c - E , c + E ) we infer - c +E)) 2 p . the containment a, - -, a, + ( 2 Because for all 1 < c r there is an > 0 with ( c - c + E ) g (1, r ) we conclude
Moreover, because a,
E
E
B, we have that w f ( ( a , - ?, a,
W ~ ( ( C E,
E
i
E,
that w f ( c ) 2 p . Because c E B,, there is an open interval Z E Z that contains c. Clearly, c a. But then x := max { a ,inf(I)} E [ a,b] and because x # C, there are in-
+
u n
tervals Z1,. . . , In E
Zso that B , n [ a,XI 2
I ] . But then for all y
E
I with y 2 c
J=1
we obtain B,
n [ a ,y ] C Z U
u n
Zj.
If c
b this means inf C 2 sup(I) > c and if
i
j=1
c = b this means C = 0, a contradiction either way. This proves the result.
Now we can characterize Riemann integrability as follows.
Theorem 8.12 Lebesgue’s criterion for Riemann integrability. The bounded function f : [ a ,b ] + R is Riemann integrable on [ a,b] iff f is continuous a.e. on [ a ,b ]. Proof. The part
“+”is the contrapositive of Lemma 8.10.
8.2. Lebesgue’s Criterion for Riemann Integrability
135
For “+,”let f be continuous a.e. on [ a , b] and let E > 0. Choose rn E N so that b-a E . Then h(X,) = 0, and there is a -< -. Let X, := x E [ a ,b] : o f ( x ) 2 m 2 rn sequence
u
of open intervals with X, 5
c oc
m
(Zj}cl
Z j and
j=1
j=1
lZjI <
&
2 ( q ( [ a ,bl)
+ 1)’
u k
. . . , Zj, so that X, C
By Lemma 8.11, there are finitely many open intervals Z j l , and without loss of generality we may assume the
Zj,
Zj,
1=1 are pairwise disjoint. For each
u k
Zjl
let Cjl := Zjl U { sup(Zj,), inf(Zjl)}. The set [ a , b] \
Ij, is a finite union of closed
1=1
intervals
J1,
. . . , J p and singleton sets {sl},. . . , { s q } . By Lemma 8.9 for each Ji,there
lJ.1 Set P := is a partition Pi so that U (f , Pi) - L(f,Pi) < 2. rn
.
1
5 <
c
u P
Pi U { a , b}. Then
i=l
c k
P
+
I Ji I O f( [ a ,b ] ) I Zj, I i=l 1=1 b-a & & & -+ w f ( [ a ,bl) <-+-=&. rn 2 ( q ( [ a , b l ) 1) 2 2 rn
+
By Theorem 5.25 f is Riemann integrable on [ a , b]. Lebesgue’s integrability criterion shows some of our earlier examples in a different light. Clearly, the Lebesgue criterion implies that continuous functions are Riemann integrable, so it supersedes Theorem 5.20. Similarly, any function with finitely many discontinuities is Riemann integrable, so Proposition 5.12 also is an easy consequence of the Lebesgue criterion. Regarding the Dirichlet function we can say the following.
Example 8.13 (Example 5.26 revisited.) The Dirichlet function is not Riemann integrable on [0,11. In light of Theorem 8.12, the function f ( x ) = 0; for x E [O, 11 \ Q is not 1; for x E [0, 11 n Q, Riemann integrable, because it is discontinuous at eviry point of [Ol 11 and by Proposition 8.5 we have h([O,11) = 1 > 0. 0
I
8. The Riemann Integral II
136
Exercises 8-7. Let f : [ a ,bl
+R
be bounded and let I , J be intervals with I g J . Prove that w f ( Z ) 5 w f ( J ) .
8-8. The oscillation as a measure of "jumps" in the function.
(a) Let f ( x ) := 1 1 0 , ~ Prove ). that w f ( 0 ) = 1 (b) Let
f
: [ a , b] + E% be bounded and let x E (a. b ) be so that lim f ( z ) and lim f ( z ) both z'x-
Z'X+
exist. Prove that wf(x) 2 Z'X-
(c) State when the inequality in Exercise 8-8b actually is an equality and prove your claim. 8-9. Let f : [ u , b] + R be bounded and let x E [a, b]. Prove that w f ( x ) = lim wf
{ O;
n+oo
'
for 8-10, Prove that for f ( x ) := O' we have inf { w f ( J ) : 0 E J , J is an interval 1; forx < 0, though f is discontinuous at 0. The explain why this does not contradict Exercise 8-8b.
}
= 0, even
8-1 1. Prove that any nondecreasing function F : [a. b] + R is Riemann integrable. Hint. Exercise 7-23. 8-12. Let a < b. A function f : [ a , b ] + R is said to be of bounded variation iff
(a) Prove that every function of bounded variation is the difference of two nondecreasing functions. Hinr. Use V , " ( f ) := sup
I"
1 f ( a i ) - f(ai-1) 1 : a = a0 < a1 < . . . < a,
=x
i=l
1
.
(b) Prove that every function of bounded variation has at most countably many discontinuities. Hint. Exercise 1-23. (c) Prove that every function of bounded variation on [ a , b] is Riemann integrable on [ a , b ] . (d) Prove that if f and g are of bounded variation and (Y variation, too.
E
R,then f
+ g and af are of bounded
8-13. A function f : [ a ,b] + R is called piecewise continuous iff there are a = zo < z1 < ' ' < Z, = b so that for all i = 1, . . . , n the restriction f l ( z , - l , z i ) is continuous. Prove that piecewise continuous bounded functions are Riemann integrable. 8-14. Prove that if A , N g Rand N is a null set, then h ( A \ N ) = h ( A ) . 8-15. Prove that if C Q is a Cantor set with nonzero measure (see Exercise 8-3e),then Ice is not Riemann integrable. Hint. Prove that l Cis ~ discontinuous at every x E CQ.
8.3 More Integral Theorems The most tedious part in any proof involving Riemann integrable functions is to prove Riemann integrability. Lebesgue's integrability criterion is a powerful tool to address this issue, Without it, the proofs of Theorems 8.14 and 8.16 would be a lot longer.
Theorem 8.14 Let f,g : [ a ,b ] + E% be Riemann integrable. Then 1. The product f g is Riemann integrable on [ a ,b],
8.3. More Integral Theorems
137
I
f . 2. Ifthere is an E > 0 so that I g ( x ) 2 E for all x E [ a ,b ] , then the quotient - is g
Riemann integrable on [a,b ] ,
3. The absolute value If 1 is Riemann integrable and the triangular inequality
Proof. For part 1, note that if f and g are continuous a.e., then the set of discontinuities { x : wfg (x) 0 ) is contained in the union { x : ~f (x) # 0 } U { x : wg(x) O } , which has measure zero. Thus f g is continuous a.e., and hence Riemann integrable. The remaining parts of this proof are left as Exercise 8-16.
+
+
It is now easy to see that once f : [a,b] + R is Riemann integrable, then it is also Riemann integrable over any closed subinterval. Formally, we define the following.
Definition 8.15 Let f : [ a ,b] + IR and let c , d E [ a ,b] with c < d. Then f is called Riemann integrable over [c,d ] ifSthe restriction f I[c,d] is Riemann integrable. In this case we set
l
d
f ( x ) dx :=
l
d f l [ c , d ~ ( ~d )x .
Theorem 8.16 Let f : [ a ,b] + R and let m E ( a ,b). Then f is Riemann integrable over [ a ,b] iff f is Riemann integrable over [a, m ] and over [ m ,b]. In this case, the integrals satisfy the equation
Ib
f ( x )d x =
lm
f (x) d x
+
/
m
b
f ( x )d x .
Proof. Exercise 8-17. We can now consider integrals in which the upper bound is an independent variable. This idea provides another connection between derivatives and integrals. For any continuous function f , definite integrals produce a function G with G' = f.
Theorem 8.17 Fundamental Theorem of Calculus, Derivative Form. Let f be a Riemann integrablefunction on [a,b]. Then the function G ( x ) := formly continuous on [ a , b ] and iff is continuous at x
I"f ( t )dt is uni-
Ja
E
( a ,b ) , then G is dixerentiable
at x with G ' ( x ) =
1
I
Proof. Let B be an upper bound so that f (x) < B for all x E [a,b ] . To see that &
G is uniformly continuous on [ a ,b ] ,let E > 0. Set 6 := -. Then for all x , z E [a,b] B with Ix - zI iS we obtain (assuming without loss of generality that x < z )
<
&
B-=E B '
which means that G is uniformly continuous.
8. The Riemann Integral II
138
Now let x E ( a , b ) , let f be continuous at x and let E > 0. Then there is a 6 > 0 such that for all z E [a,b ] with Iz - X I < 6 we have ( f ( z ) - f ( x ) l < E . Hence, for all z > x with z E [a,b ] and Iz - X I < S we obtain
For z < x,the proof is similar, and hence lim
G ( z )- G ( x )
Z'X
z-x
rn
= f (XI.
Functions as mentioned in Theorem 8.17 sometimes are also defined with an integral such that a > x . If we went through a partition from right to left rather than left to right, then all the bases of the rectangles in a Riemann sum would have negative length. Thus it makes sense to define the following. Definition 8.18 Let f : [a, b ] +. R be Riemann integrable. Then we define the integral with the reversed bounds to be
la
b
f (x)d x := -
f (x) dx.
Corollary 8.19 Let f : [ a ,b] -+ IR be Riemann integrable and let xo the function G ( x ) :=
Jc:
f ( t ) dt is uniformly continuous on [ a ,b ] and
d", (I:f ( t ) dt 1
uous at x E ( a ,b), then -
E [ u , b].
iff
Then
is contin-
= f (x).
Proof. Exercise 8- 18.
rn
Exercises 8-16. Proving Theorem 8.14 (a) Prove part 2 of Theorem 8.14. (b) Prove part 3 of Theorem 8.14. Hint.Use Lemma 5.6 to prove the inequality (c) To see how valuable the Lebesgue criterion is, use Riemann's Condition (Theorem 5.25) to prove that I f 1 is Riemann integrable over [u,b]. Then compare this proof with the proof using the Lebesgue criterion and state which proof is simpler. Hint.Use a partition P with U ( f , P ) - L ( f , P ) < E . 8-17. Proving Theorem 8.16 (a) Prove Theorem 8.16. Hint. In each direction of the proof use the Lebesgue criterion to establish Riemann integrability. Use Lemma 5.6 for the equations, making sure that the point m is an element of each partition P k .
8.3. More Integral Theorems
139
(b) To see how valuable the Lebesgue criterion is, use Riemann’s Condition (Theorem 5.25) to prove that if f is Riemann integrable over [a, b],then f is Riemann integrable over [a, nz]. Then compare this proof with the proof using the Lebesgue criterion and state which proof is simpler. Hint. Use a partition P with U ( f , P ) - L ( f , P ) iE . 8-18. Prove Corollary 8.19. 8-19. Use Riemann’s Condition (Theorem 5.25) to prove that if f and g are Riemann integrable over [a, b],then f g is Riemann integrable over [a, b]. Then compare this proof with the proof using the Lebesgue criterion and state which proof is simpler. 8-20. Determine which result from this section that was used in the proof of Theorem 8.17 was not available in Section 5.3. (This prevented us from placing Theorem 8.17 right after the Antiderivative Form of the Fundamental Theorem of Calculus.) 8-21. Mean Value Theorem for the Integral. Let the function f : [ u , b] + W be continuous and let the function g : [a, b] + [0, co)be nonnegative and Riemann integrable. Prove that there is a c E [a, b]
so that the integral satisfies 8-22 Let g : [ a , b] + [O.
00)
l
b
b
f ( x ) g ( x )d x = f ( c )
be Riemann integrable and let f : [a, b] --f b
there is a c
f ( x ) g ( x )dx = f ( a )
[ a , b] so that
E
g(x) dx.
[
g(x) dx
W be nondecreasing. Prove that
+f(b)
h
b
g ( x ) d x . You may use
that nondecreasing functions are Riemann integrable (see Exercise 8-1 1). 8-23 Integration by Parts. Let [ a , b] c (c. d ) and let F , g : (c, d ) -+ W be differentiable functions with derivatives f and g’ that are Riemann integrable on [ a , b]. Prove that the integral satisfies
lb
f ( x ) g ( x ) d x = F ( b ) g ( b )- F ( a ) g ( a ) -
lb
F(x)g’(x)dx.
8-24. Integration by Substitution. Let [ a , b] c (c, d ) , let g : (c, d ) + R be differentiable, let its derivative g’ be Riemann integrable on [a, b ] ,let ( u , u ) 2 g [ [ a ,b] ] and let F : ( u , u) + R be differentiable with continuous derivative f. Prove that
lb
f ( g ( x ) ) g ’ ( x ) dx = F ( g ( b ))
-
F (g(a) ) .
8-25. Let a > 0. A function f : [-a, a ] + W is called even iff f ( x ) = f ( - x ) for all x E [ - a , a ] . A function f : [-a, a ] + R is called odd iff f ( x ) = - f ( - x ) for all x E [-a, a ] .
La L
(a) Prove that if f is even and Riemann integrable, then
f ( x ) dx = 2
(b) Prove that i f f is odd and Riemann integrable, then
f ( x ) d x = 0.
La
f ( x ) dx
(c) Prove that any function f : [-a, a ] + W is the sum of an even and an odd function Hint. f ( x ) + f ( - x ) is even, ‘ 2 8-26. Compute the derivative. (Should the integrands be unknown, simply note that they are continuous.) (a)
& 1’
et2 dt
8-27. Let f : [a, b] + W be continuous and let I , u : (c, d )
(l:;)
f(t)
d f ) = f ( u ( x ) ) u’(x) - f (I@)
8-28. Construct a function f : [0, 11 + R so that integrable.
-+ [ a , b] be
differentiable. Prove that
) l’(x).
I f 1 is Riemann integrable and
f is not Riemann
8-29. A function f : [ a , b] + W is called absolutely continuous iff for every E > 0 there is a S > 0 so that for all sequences (a1 , b l ) , . , , , (a,, b,) of pairwise disjoint open intervals the inequality n
n
x ( b j - ai) i6, implies i=l
i=l
1 f ( b i )- f(ai) 1 < E.
8. The Riemann Integral 11
140
(a) Let f : [a, bl + R be a Riemann integrable function. Prove that the function G : [a, b] + W defined by G ( x ) :=
s,^
f ( t ) df is absolutely continuous on [ a , b].
(b) Prove that every absolutely continuous function is uniformly continuous. (c) Prove that f ( x ) =
1
-
X
is continuous, but not absolutely continuous on (0, 11
8-30. Results for Riemann-Stieltjes integrals. Let g : [a, b] + R be nondecreasing, and let the functions f,h : [ a , bl + Iw be bounded and Riemann-Stieltjes integrable on [ a , b] with respect to g.
(a) Prove that i f 1 is Riemann-Stielfjesintegrable on [a, b] with respect t o g and that the triangular inequality
ilb / If1 f dgi 5
b
dg holds.
(b) Prove that for all m E [a,b] the function f is Riemann-Stieltjes integrable with respect to g b
over [a,m ] and over [m,b] and that
f dg
=
/
rn
a
f dg
+
/f b
rn
dg.
(c) Prove that the product f h is Riemann-Stieltjes integrable on [ a , b] with respect to g Hint (for all). Use the Riemann Condition for Riemann-Stieltjes integrals (see Exercise 5-27)
8.4 Improper Riemann Integrals The Riemann integral allows us to compute the “area” under bounded functions defined on closed and bounded intervals. Sometimes we are interested in the area under functions that are defined on infinite intervals or that are unbounded. These areas can be approximated with Riemann integrals.
Definition 8.20 Let a restriction f I[a,c~
E
R and let f
: [ a ,00) -+ R be such that for all c > a the
is Riemann integrable. Ifthe limit lim r+cz,
l
t
f (x) d x exists, it is called cc
the improper Riemann integral of f over [ a , 00) and it is denoted Improper Riemann integrals for functions f : (Exercise 8-31).
(-00,
f ( x )d x .
b ] -+ R are dejned similarly
Example 8.21 The p-integral test for integrals over infinite intervals. Let p > 0 be 1 rational. Then f ( x ) = - is improperly Riemann integrable over [ 1, 00) zfs p > 1. xp
1
, while for
Finally, we need to show that the improper integral does not exist for p = 1. To do 1 1 this, note that f ( x ) = - is greater than -on each interval [ m ,m 1). Therefore, X m+l
+
141
8.4. Improper Riemann Integrals
for all n
E
N we infer f L
C" k +1 l
-l[k,k+l),
and hence
k=l
The latter sum is unbounded, so
0
d x does not exist.
Note that because we have not yet defined logarithms, the last argument in Example 8.21 is unavoidable. Similarly, we had to restrict ourselves to rational powers, because powers with arbitrary real exponents have not been defined yet. Of course, the p integral test ultimately holds for real exponents p and we will not restate it once real powers are defined. Improper integrals can also be defined for (potentially) unbounded functions. Definition 8.22 Let a , b E W with a < b and let f : [a,b ) + W be suck that for all
c
E
( a ,b ) the restriction f
is Riemann integrable. Ifthe limit lim t+b-
lt
f ( x ) dx
exists, it is called the improper Riemann integral off over [ a ,b ) and it is denoted
lb
f ( x ) dx. Improper Riemann integrals forfunctions f : ( a ,b] -+
W are defined
similarly (Exercise 8-32). For Riemann integrable functions on [ a ,b ] , the improper Riemann integral over [ a ,b ) agrees with the Riemann integral over [ a ,b ] . Proposition 8.23 Let a , b E R with a < b and let f : [ a ,b ] + R be Riemann integrable. Then the improper Riemann integral off over [ a ,b ) exists and it is equal to the Riemann integral off over [ a ,b]. &
Proof. Let B > 0 be an upper bound of I f / and let E > 0. Then for S := - and all B t E [ a ,b ) with It - bl < 6 we obtain
Riemann integrable functions are not the only functions that are improperly Riemann integrable. Example 8.24 shows that there are unbounded functions for which the improper Riemann integral exists. Example 8.24 The p-integral test for improper integrals over (0, 11. Let p > 0 be 1 rational. Then f ( x ) = - is improperly Riemann integrable over ( 0 , I ] iff p < 1. XP
Mimic the proof for Example 8.21. (Exercise 8-33.)
0
8. The Riemann Integral II
142 Note that the improper integrals
r
I'
-
:x
d x converge for p < 1, while the integrals
- d x converge for p > 1. Irrespective of this important difference, for both
xlP types of improper integrals similar laws hold and there also is a Comparison Test that is similar to the Comparison Test for series.
Theorem 8.25 Let f , g : [ a ,b ) + R (where b could be integrable over [ a ,b ) and let c E R.Then 1. f
00)
be improperly Riemann
+ g is improperly Riemann integrable over [a,b ) and
lb(f +
g ) ( x )d x =
l
b
f (x) d x
+
l
b
g(x)dx.
2. cf is improperly Riemann integrable over [a,b ) and l b c f ( x )d x = c
Ib
f(x) dx.
Proof. Similar to the proof of Theorem 6.4. (Exercise 8-34.) Theorem 8.26 Let f : [a,b ) -+ R (where b could be 00) be Riemann integrable over all intervals [ a ,c] 5 [a,b). Then f is improperly Riemann integrable over [ a ,b ) if and only if for all c E [ a ,b ) the finction f is improperly Riemann integrable over
1
b
[c. b ) and in this case
f (x)d x =
lc
f (x) d x
b
+
f (x) d x .
Proof. Exercise 8-35. Theorem 8.27 Comparison Test for improper integrals. Let f ,g : [ a ,b ) -+ E% (where b could be 00) be such that 0 5 f 5 g , f is Riemann integrable over every closed interval in [ a ,b ) and g is improperly Riemann integrable over [ a ,b). Then b
b
f is improperly Riemann integrable over [ a ,b) and
Proof. The function F ( t ) := [ a ,b ) and it is bounded by
f (x)d x 5
g ( x )dx.
lt
f (x)d x is continuous and nondecreasing on
g(x) d x . The reader will show in Exercise 8-36 that
lim F ( t ) = sup { F ( t ) : t E [ a ,b ) } 5
g ( x ) d x to complete the proof.
t+b-
Theorem 8.28 Let f : [ a ,b) + R (where b could be 00) be Riemann integrable over every closed interval in [ a ,b). If I f 1 is improperly Riemann integrable over [ a ,b), then f is improperly Riemann integrable over [ a ,b ) and the triangular inequality
143
8.4. Improper Riemann Integrals
t
Figure 21: In the integral test, the improper integral of a nonincreasing function is related to the series that give the Riemann sums with left and right endpoint evaluations for the partition with step length 1. The integral test says that for an improperly integrable function the series obtained by right endpoint evaluation cannot be infinite ( a ) , while for a function that is not improperly integrable the series obtained by left endpoint evaluation cannot be finite (b).
Proof. Exercise 8-37. Finally, we should note that the occurrence of series in Example 8.21 is not an accident. The Integral Test connects the convergence of certain series to the convergence of improper integrals over infinite intervals.
Theorem 8.29 Integral Test. Let f : [ 1, 00) -+ [0,00) be a bounded nonincreasing 00
ifs the improper integral
f ( j ) converges
nonnegative function. Then the series j=1
f (x)dx converges (also see Figure 21). 00
Proof. Throughout the proof let g ( x ) =
f ( j ) l [ j , j + I ) ( x ) .(For
every x 3 1,
j=1
this sum has at most one nonzero term.) m
For
“+,”let
f ( j ) be convergent. Then g as defined above is improperly Riej=1 00
w
[
mann integrable over [ l , 00) with
f(j)
g ( x )d x =
i00.
Because 0 5 f 5 g ,
J I
j=l
the Comparison Test for improper integrals implies that Conversely, for
r
La
“e,” let
f ( x - 1) d x =
Lrn
L
00
f ( x ) d x converges.
f ( x ) d x be convergent. Then the improper integral
f ( x ) d x converges. Because g ( x ) 5 f ( x - 1) for all x 2 2,
by Comparison Test for improper integrals the integral
g ( x ) d x converges, which
144
8. The Riemann Integral II C X
f ( j ) converges.
means the series j=2
The connections and similarities between integrals over infinite intervals and series need to be considered with caution. For example, Exercise 8-38 shows that there is no Limit Test for improper integrals over infinite intervals. Finally, improper integrals can also be defined for functions on the whole real line and for functions with multiple singularities. a1 < a2 < . . . < a,-l < a, = b (where a could be and b could be 00) and let f : ( a , b ) \ { a l , . . . , a n - l } -+ E% be Riemann integrable over all closed subintervals of its domain. We define the improper Riemann integral off as follows. Let r1 < 12 < < r, be so that aj-1 < r j < a , and define
Definition 8.30 Let a = a0 < -00
Ib
f (x) dx :=
s . .
2 [I::,
f (x) dx
+
j=1
1;
f (x) dx] ifall the summands exist.
Exercises 8-31. Let b E
W and let f
: (-m, b] +
W be such that for all a
integrable. Define the improper Riemann integral
< b the restriction f [ [ a , b ]is Riemann
s_, b
f ( x ) dx o f f over (--00,
b].
8-32. Let a , b E iW with a < b and let f : (a. b] + W be such that for all c E ( a , b) the restriction f l [ c , b ] is Riemann integrable. Define the improper Riemann integral 8-33. Prove the claim in Example 8.24.
lb
f ( x ) dx o f f over [a, b]
8-34. Prove Theorem 8.25. 8-35. Prove Theorem 8.26. 8-36. Finish the proof of Theorem 8.27 by proving that lirn F ( t ) = sup
+If1
8-37 Prove Theorem 8.28. Hint. 0 5 f
t+b-
[ F(t): t
E [a, b )
}.
5 2lfl.
8-38 Construct a function f : [ I , m ) + [0, I] that is improperly Riemann integrable, but does not 1 converge to zero as x --f 03. Hint. Triangles of height 1 and area 2n 8-39 Cauchy Criterion for improper Riemann integrability. Let f : [a, b ) + W (where b could be 00) be Riemann integrable over all intervals [a, c ] 2 [ a , b). Prove that f is improperly Riemann integrable over [ a , b ) iff for all E > 0 there is an M E [a, b ) , so that for all c, d E ( M , b ) we have
8-40 Limit Comparison Test for improper integrals. Let
be Riemann integrable over all intervals [ a ,c]
s,"
f,g :
[ a ,b )
+ [0, 03) (where b could be x)
[a, b ) . Prove that if
lim x+b-
f(x)
fo =K g(x)
> 0, then
d x converges iff
Hinr. Close to b we have g ( x ) ( K
~
E)
5 f(x) 5 g(x)(K
+E).
A
8-4 1 Prove that the function f : ( a , b] + W is improperly Riemann integrable over ( a , b] iff the function
[
improperly Riemann integrable over a , m ) and that in this case the integrals are equal.
Chapter 9
The Lebesgue Integral
The geometric idea behind integration is to approximate the area under the graph of a function with areas that are easier to compute. In the Riemann integral, we partition the x-axis and erect a rectangle over each partition interval to approximate the area under the graph. However, Lebesgue’s criterion for Riemann integrability shows that geometric rectangles will not approximate the area well if the function “oscillates” too much. That is, if the differences between the possible choices for the heights of the rectangles do not shrink to zero, then we cannot uniquely identify the area. Equivalently, in the Darboux formulation (see Section 5.4) excessive oscillations force the upper and lower approximations of the area with rectangles to stay a finite distance apart. If we change our point of view and partition the y-axis instead of the x-axis, the problem with different choices for the height goes away (see Figure 22). For a set S = { x E [ a , b] : yi-1 5 f ( x ) < yi }, all sensible values for the height of a generalized rectangle with base S are between yi-1 and yi. Because the difference between these values can be made small, oscillations are not an issue. However, this approach requires that the bases of our generalized rectangles are no longer intervals. The area of such a generalized rectangle will be the Lebesgue measure of the base set times the height of the rectangle. In this fashion, we retain all benefits of the geometric motivation for integration, while being able to integrate many more functions.’ This chapter introduces the fundamentals of Lebesgue integration. These fundamentals will be revisited in Chapter 14 when we generalize our work to arbitrary measure spaces. Our presentation is designed to readily translate to the more abstract setting of Chapter 14. Before we start, it is time to extend our arithmetic from the real numbers to the ‘The Lebesgue integral also remedies a more abstract problem with the Riemann integral. Spaces of Riemann integrable functions are usually not complete (see Exercise 16-15d), while spaces of Lebesgue integrable functions are (see Theorem 16.19). Completeness is such a fundamental abstract property that this may be the main reason why the Lebesgue integral is preferred.
145
146
9. The Lebesgue Integral
k f l ___
2"
k 2" k-l 2"
k-2 2"
dashed:
~
n2"
k-3 2"
S
=
k-1 7 l A k - l
k=l 7
-
2"
Ak-1
Figure 22: The idea behind the Lebesgue integral is to partition the y-axis instead of the x-axis and to approximate the area with simple functions. This figure shows that this can lead to "scattered" base sets on the x-axis, which we will treat first in this chapter. The proof of Theorem 9.19 uses the partition of the y-axis into intervals with dyadic rational endpoints. real numbers with 30 and -m included. This is sometimes called the extended real number system. In the extended real number system, every set has a supremum. This will make our definitions of measures and of the Lebesgue integral simpler, because we will not need to explicitly distinguish between bounded and unbounded sets.
Proposition 9.1 In the extended real number system [--00, co] := R U {oc.- 3 0 ) , 30 is the greatest element and --oo is the smallest element. Consequently, every set has an in$mum and a supremum. Proof. Let A C [--00, 001. The supremum of {-30) is -30, and the supremum of 0 also is -00, so we can assume that A # 0, {-cm}.If A is bounded above by a real number, then A n Iw # 0 has a supremum in the real numbers, which is also the supremum of A in [-30, 301. If A is not bounded above by a real number, then sup(A) = 00. Infima are treated similarly. Proposition 9.1 assures that from here on, we can take suprema and infima of sets of numbers fairly indiscriminately, as long as we know how to handle infinite values 301 is the same as on R with the additional convenalgebraically. Arithmetic on [-a, tions of Definition 9.2. These conventions are inspired by the corresponding limit laws in Theorems 2.44 and 2.46.
147
9.1. Lebesgue Measurable Sets
Definition 9.2 Arithmetic involving 00. Let c E R. Then c-00=-00,
c+co=00,
undefined; c.(-m)=
{
00;
:nzfined;
i f c > 0, i f c < 0, i f c = 0, i f c > 0, i f c < 0, i f c = 0,
00
C
- = 0: --oo
m . (-00) = -00,
m*co=co, ( - 0 0 ) . (-00)
C
- = 0,
= 00.
All other attempts at “arithmetic with infinity” lead to what is called indeterminate forms. For these, the result can be any number, and hence rules of arithmetic cannot be stated. We will consider indeterminate forms in Section 12.3.
9.1 Lebesgue Measurable Sets We will work with generalized rectangles whose bases are no longer intervals. Therefore we need to measure the size of sets that are more complicated than rectangles. Outer Lebesgue measure is a reliable upper bound for the (one-dimensional) “volume” of a set. For all examples we have seen, it gives the right “volume.” Hence, we can (and do) consider it the right way to measure the “outer volume” of a set. Unfortunately, there are complications. It is possible to split a set T into two sets so that the outer Lebesgue measures of the two sets add up to more than the outer Lebesgue measure of T . If we were to involve such pathological sets in a definition of integration, the integral would not even be a reliable measure of the area of generalized rectangles. It would make no sense if the total “length” of the base would depend on how we split the base. The definition of Lebesgue measurable sets is designed to safeguard exactly against this problem (see Theorem 9.11). Because all our sets are subsets of R (and later of a measure space M ) , we introduce abbreviated notation for the complement.
Notation 9.3 When there is one underlying set X that contains all sets that we currently investigate (as is the case in measure theory) and S s X , then we denote the 0 complement of S in X also as S’ := X \ S. Definition 9.4 A subset S R is called Lebesgue measurable i f f o r all T C R the equality h ( T ) = h ( S n T ) + h (S’ n T ) holds. We will also call the set T a test set. We denote the set of Lebesgue measurable subsets of R by EL. The existence of non-Lebesgue measurable sets, which would cause the abovementioned problems, is equivalent to the Axiom of Choice. That is, whether or not nonLebesgue measurable sets exist depends on what axiomatic system of set theory is used.
9. The Lebesgue Integral
148
In practical terms it means that non-Lebesgue measurable sets do not occur in physical phenomena. Hence, from an applied point-of-view we need not be too concerned with nonmeasurable sets and we will not consider them any further in this text. Exercise 9-7 illustrates the problems mentioned above with a simpler measure of ‘‘size,’’ the Jordan content, for which even simple sets can behave badly. The construction is more complicated for Lebesgue measure and the interested reader can find such constructions in [14] and [29]. For the remainder of this section, we explore the properties of the set of Lebesgue measurable subsets of R.
Definition 9.5 Let S R be a Lebesgue measurable set. Then the outer Lebesgue measure h ( S ) of S is also called the Lebesgue measure of S. We first note that “half’ of the definition of measurability is always satisfied.
Corollary 9.6 For all subsets S , T 5 R,we have h ( T ) 5 h ( S n T ) + A. (S’ n T ) Proof. This follows from part 3 of Theorem 8.6 with A1 := S n T , A2 := S’ n T and A,, := 0 for n 2 3. It is now easy to see that null sets are Lebesgue measurable.
Proposition 9.7 I f h ( S ) = 0, then S is Lebesgue measurable. Proof. Let S be a null set. By part 2 of Theorem 8.6 for all sets T R we obtain 0 5 h ( S f l T ) 5 h ( S ) = 0. Hence, for all subsets T E R we have the inequality h ( S n T ) h (S’ n T ) 5 h (S’ n T ) 5 h ( T ) , which by Corollary 9.6 is all we need to establish Lebesgue measurability of S .
+
In the following, we establish that certain set theoretical operations preserve measurability. Theorem 9.10 summarizes the most important facts. Although there are more results, the properties listed in Theorem 9.10 suffice for our purposes. Set systems that satisfy these properties are called a-algebras (see Definition 14.1). These set systems are fundamental for measure theory.
Lemma 9.8 If A and B are Lebesgue measurable sets, then the intersection A n B is Lebesgue measurable. Proof. Proofs of Lebesgue measurability typically involve the appropriate rewriting of terms and the use of the right test sets. To show that A n B is Lebesgue measurable, let T R be any subset of R. Then h ( ( An B ) n T ) =
+ h ( ( A n B)’ n T ) h ( A n B n T ) + h ((A’ U B’) n T ) +
+
h ( A n B f l T ) h ( A n (A’ U B’) f l T ) h (A’ = A ( B n A n T ) + h (B’ n A n T ) + A (A’ n T ) =
n (A’ U B’) n T )
149
9.1. Lebesgue Measurable Sets
= h(AnT )
+ h (A’ n T )
= h(T).
Because T was an arbitrary subset of measurable.
R we have proved that
A
nB
is Lebesgue
rn
The next lemma will be useful in two ways. It is a step toward proving that countable unions of Lebesgue measurable sets are Lebesgue measurable. It also is a step toward proving that if a Lebesgue measurable set consists of countably many pairwise disjoint Lebesgue measurable pieces, then the Lebesgue measure of the set is the sum of the Lebesgue measures of the pieces.
Lemma 9.9 Let
{An}Elbe a sequence of painvise
u
disjoint Lebesgue measurable
co
sets. Then the union
A, is Lebesgue measurable and for all T &
IR we have
n=l
Proof. Let T E R. We first prove by induction that for all k k
A(T)=
E
N we have that
)
C A (A, n T ) + A
n T . The base step with k = 1 follows
n=l
from the Lebesgue measurability of A 1. For the induction step k -+ ( k l), we can assume that the induction hypothe-
+
k
sis A(T) =
C A (A, n T )+A n=l
k+l
A.(T) =
CE. (A, n T ) + A
n=l
induction hypothesis.
is true, and we need to prove that
((5 n=l
An)’ n T ) . We start the induction step with the
9. The Lebesgue Integral
150
/ k+l
k
\
n=l
=
& A n n T ) + h ( ( ; A n ) ’ nnT= l) , n=l
which finishes the induction stem k
Thus for all k E
N we have h ( T ) 2
h (A,
n T )+h
n=l
letting k go to infinity we obtain the following.
2
h(T).
The above establishes measurability of the union as well as the desired equality. W
Theorem 9.10 The set CA of Lebesgue measurable subsets of properties. 1. 0
E
2. I f S
R has the following
C).. E C h , then
S’
E
C,.
u x
3. @ A n E CAf o r all n E W, then
An
E
Xi,.
n=l
Proof. Parts 1 and 2 are left to the reader as Exercises 9-la and 9-lb. For part 3, let An E CAfor all n E N and let T R. Define B1 := A 1 and then inductively for n E
N set Bn+l
that for all n
E
:= A n + l r7
n n
(dB1l’
= A n f l r7
Bi. An easy induction shows
,=1
N the set Bn is Lebesgue measurable (use part 2 and Lemma 9.8),
that B1, . . . , Bn are pairwise disjoint and that
u n
A, =
r=l
u n
2=1
B, (see Exercise 9-lc). But
9.1. Lebesgue Measurable Sets
u u x
then
151
00
Bi (see Exercise 9-ld) and the latter set is Lebesgue measurable by
Ai =
i=l
i=l
Lemma 9.9. Now that we have established that countable unions preserve Lebesgue measurability, it is reassuring to note that Lebesgue measure is additive for countable unions of pairwise disjoint Lebesgue measurable sets. This is exactly what we expect from a sensible measure. The size of a whole set is the sum of the sizes of its pairwise disjoint parts.
Theorem 9.11 Let
u rn
sets. Then
{An]El be a sequence of painvise disjoint Lebesgue measurable ffi
A, is Lebesgue measurable and h
n=l
Proof. The union is Lebesgue measurable by Lemma 9.9 and if we apply Lemma
u ffi
9.9 to T :=
A,, we obtain
n=l
which proves the result. The next result is a nice application of the countable additivity we just proved and it will also be needed when we show that sums of Lebesgue integrable functions are Lebesgue integrable. be a sequence of Lebesgue measurable subsets of W so
Theorem 9.12 Let
(,,u =, 100
that A,
A,+lforall n
E
N.Then h
Proof. Set B1 := A1 and for all n
u
N E
N the equality
I 1 =1
u N
N
B, =
E
\
A,
= lim h(A,). n-.+rn
N define
Bn+l := An+l
\ A,. Then for all
n=l
u
u cc
so
A, = A N holds, which means
B, =
n=l
A,.
n=l
N
lim
N+W
C ~ ( B , 1’)im= AN). n=l
N+rn
The whole idea of Lebesgue measurability is only useful if it indeed allows us to extend the idea of Riemann integrability. For that to happen, intervals must be Lebesgue measurable. We have delayed this result to the end of the section, because now we can use some of the machinery built so far.
9. The Lebesgue Integral
152
Proposition 9.13 Intervals are Lebesgue measurable and the Lebesgue measure of an interval is its length. Proof. Because singletons are null sets and thus Lebesgue measurable, and because countable unions of Lebesgue measurable sets are Lebesgue measurable, intervals are shown to be Lebesgue measurable if we can prove that open intervals of finite length are Lebesgue measurable. So let A = ( a , b ) be an open interval of finite length and let T E R. By Corollary 9.6, we only need to prove h ( T ) 2 h ( A n T ) + h (A’ n T ) . The inequality is trivial if h ( T ) = co, so we only need to prove the inequality if h ( T ) < 00. Let T C R be a set of finite outer Lebesgue measure, let E > 0 and let
[Zj}pl be a family of open intervals with T
u 00
00
Zj
and
IZjl
5 h(T)
j=1
j=l
+ !.2
Then
Z := { Z j n A : j E N} is a countable family of open intervals whose union contains A n T . Moreover, 0
:=
{Zj \ [ a , CO)
u
:j E
a - -&, a +
{(
8
N} U { Z j \ (-co,b] : j
-), (b&
i& ’ b +
E
i)] &
N]
8
is a countable family of open intervals whose union contains A’ n T . Thus h ( A f’ T )
=
+ h (A’ n T )
h(T)+&
+
Because E was arbitrary this proves that h ( T ) 1 h ( A r l T ) h (A’ n T ) and we have proved that A is Lebesgue measurable. Regarding the Lebesgue measure of intervals, by Proposition 8.5 for closed intervals in R we have h ( [ a ,b ] ) = b - a . For closed, unbounded intervals of the form [ a , co), we infer for all b > a that h ( [ a ,co)) 2 h ( [ a ,b ] ) = b - a , and hence h ( [ a ,00)) = 00. Intervals of the form (-00, b] are handled similarly. For open and half-open intervals, note that the singleton sets consisting of the endpoints have measure zero. This means (see Exercise 9-2) that adding or removing these points does not affect the Lebesgue measurability of the set or its Lebesgue measure.
Standard Proof Technique 9.14 Note that the inequality marked with (*) in the proof of Proposition 9.13 can actually be shown to be an equality. This is not necessary
9.2. Lebesgue Measurable Functions
153
because we only need the inequality. In complicated estimates, it can happen that an inequality sign is put between quantities that are actually equal. Usually, this happens when the equality would not have helped in the proof (as in the example just mentioned) and when the writer did not want (the reader) to spend extra effort to think about why the quantities may be equal. 0
Exercises 9-1. Finish the proof of Theorem 9.10. That is,
(a) Prove that 0
E
(b) Prove that if S
ZA. ZA,then S’
E
ZA.
E
(c) Perform the induction mentioned in the proof of part 3. (d) Prove that if (Ai):,
u u n
Ai =
i=1
and (Bi)iOO,l are countable families of sets so that for all n E N we have
n
u u 00
B i , then
03
Ai =
i=l
i=l
Bi.
i=l
9-2. Let A be a Lebesgue measurable set and let N be a null set. Prove that A \ N is Lebesgue measurable and that h ( A \ N ) = ,L(A). Him. Use Theorem 9.10.
9-3. Let A , B 9-4. Let
u
B be Lebesgue measurable sets. Prove that A \ B is Lebesgue measurable. be a finite sequence of painvise disjoint Lebesgue measurable sets. Prove that the union
N
An is Lebesgue measurable with A
u )
(ny1
n=l
An
N
=
2 h(An). n=l
n 00
9-5. Let (An)E1 be a sequence of Lebesgue measurable sets. Prove that
An is Lebesgue measurable
n=l
and that for all k E W we have h
n ) (Il An
5 h(Ak).
9-6. Let C Q be a Cantor set. Prove that CQ is Lebesgue measurable. Hint. Use Exercise 9-5. n
9-7. For all S C [0, 11, let J(S) := inf j=1
dan content of S . Prove that J ( [ O , 11 n Q ) = 1 and J (10, 11 \
Q ) = 1.
9-8. Let A G B C B and let A (but not necessarily B ) be Lebesgue measurable Prove that h ( B ) = h ( A ) h ( B \ A )
+
9.2 Lebesgue Measurable Functions We now introduce the functions for which the Lebesgue integral can be defined. By first defining what (potentially) integrable functions should look like, we avoid the Riemann integral’s conceptual complications that are characterized in Lebesgue’s criterion (see Theorem 8.12). Existence or nonexistence of the Lebesgue integral, defined in Section 9.3, will then merely be a question of whether there is too much area under the graph of the function. We will revisit the original motivation of partitioning the y-axis after the
9. The Lebesgue Integral
154
proof of Theorem 9.19. To approximate areas, in Lebesgue integration indicator functions take the place of rectangles. Recall that by Definition 5.9 the indicator function 1; f o r x E S, of a set S R is l s ( x ) := 0; f o r x $ S. Just as Riemann integrable functions can be approximated a.e. with step functions (see Exercise 5-26), Lebesgue integrable functions will be approximated with functions that are constant on measurable sets and which only assume finitely many values.
Definition9.15 A function s : R + R is called a simple Lebesgue measurable function, or, a simple function, iff there are numbers a1 , . . . , a, E R and pairwise n
disjoint Lebesgue measurable sets A1,
. . . , A, 5 so that s =
aklAk. k=l
For functions f that assume more than finitely many values, we consider the positive and negative parts of f separately.
Definition 9.16 For f : R + [-m, co],we define f + ( x ) := max { f (x), 0 ) and f - ( x ) := - min { f ( x ) , 0 )f o r all x E R. Because we will successively approximate measurable functions from below we want to speak of sequences of functions.
Definition 9.17 A family { f n J n E ~of functions will also be called a sequence of functions, denoted { fn}r=l. Definition 9.18 A function f : R + [0,001 is called Lebesgue measurable iff there is a sequence {sn)rE1of simple functions s, : Iw + [0,co)such that f o r all x E R the sequence { s, ( x ) } E l is nondecreasing and lim s, (x) = f (x). A function f : R + [--00,-00] both Lebesgue measurable.
n-+m
is called Lebesgue measurable iff f + and f - are
The key problem in Riemann integration is that for some functions the approximations from above and below will not “meet.” Definition 9.18 does not simply circumvent this problem by only focusing on approximations from below. Exercise 9-9 shows that a bounded function (only bounded functions are considered in Riemann integration) is Lebesgue measurable iff it can be approximated from above with simple functions. That is, Definition 9.18 may look biased, but for bounded functions the concept of Lebesgue measurability could also be defined with approximations from above. Because we are also interested in unbounded functions, we choose to work with approximations from below throughout. Because Lebesgue measurability is a key concept, it is useful to have several equivalent formulations available.
Theorem 9.19 Let f : R -+ lent.
[--00,
m] be a function. Then the following are equiva-
1. The function f is Lebesgue measurable.
9.2. Lebesgue Measurable Functions
155
2. For all a
E
R,the set {x E R : f (x) > a } is Lebesgue measurable.
3. For all a
E
R,the set {x E R : f (x) 5 a } is Lebesgue measurable.
4. For all a
E
R,the set {x E R : f (x) < a } is Lebesgue measurable.
5. For all a E
R,the set {x E R : f (x) L a } is Lebesgue measurable.
Proof. We first prove the result for a function f : R + [0, w]. For the implication “1=+2,” let a E R and let {s,]:=~ be a sequence of simple functions such that for all x E R the sequence {sn(x)}Z1is nondecreasing and converges to f ( x ) . For all x E R,if f ( x ) > a , then for some n E N the inequality s,(x) > a holds. Conversely, if for some n E N we have that s n ( x ) > a , then because { S , ( X ) } ~ is~ nondecreasing we must have f ( x ) > a . This means that
u {x 30
{x E R : f ( x ) > a } =
E
R : s n ( x ) > a } . But each set {x
E
R : sn(x) > a )
i1=1
is a union of finitely many Lebesgue measurable sets, which means it is Lebesgue measurable. Therefore, as a countable union of Lebesgue measurable sets, the set {x : f ( x ) > a } is Lebesgue measurable. For “2+3,” let a E R.Then {x E R : f ( x ) 5 a } = R \ {x E R : f ( x ) > a } , which is the complement of a Lebesgue measurable set. For “3=+4,” note that for all real numbers a the set {x E R : f ( x ) < a } is equal to the union {x
E
R :f(x)
u7x
E
7
R : f ( x ) 5 a - - and the latter set is
k a countable union of Lebesgue measurable sets. For “4+5,” let a E R. Then {x E R : f ( x ) 2 a } = R \ {x E R : f ( x ) < a } , which is the complement of a Lebesgue measurable set. For “5+1,” by Exercise 9-10 the set {x E R : f ( x ) = 00) is Lebesgue measurable. For each n E N,define the simple function k= 1
Then each s, is nonnegative, because for all k , n E
N we have
To see that each { s n ( x ) } Z lis nondecreasing, let x E R and let n E N. The claim is trivial if f ( x ) = 00, so we can assume f ( x ) < w. If f ( x ) 2 n , then k-1 s , ( x ) = 0 5 s,+l(x). If f ( x ) < n , find k E { 1 , . . . , n2”} so that s,(x) = -. 2” k-1 k 2 ( k - 1) k- 1 - -= s,(x). Finally, Then -5 f ( x ) < -, and hence s,+1(x) 2 2” 2, 2, each ( s n ( x ) } z lconverges to f ( x ) because, if f ( x ) E [ N - 1, N ) , then for all IZ 2 N 1 the inequality l f ( x ) - s , ( x ) / < - holds, and if f ( x ) = 00, then f ( x ) = n . 2”
2n+l
9. The Lebesgue Integral
156
Finally, consider f : R + [--00, co]. For “1+2” let f be Lebesgue measurable. Then f - and f + are Lebesgue measurable. But then for a 2 0 we have that {x E R : f ( x ) > a } = {x E R : f + ( x ) > a } , which is Lebesgue measurable, and for a < 0 we have {x E R : f ( x ) a } = (x E R : f - ( x ) < - a } , which is also Lebesgue measurable. Parts “2+3,” “ 3 j 4 , ” and “4+5” are similar to what was done for nonnegative functions. To prove part “ 5 j 1 , ” first note that for all a > 0 we have {x E R : f + ( x ) 2 a } = { x E R : f ( x ) 2 a } , which is Lebesgue measurable and for all a 5 0 we have { x E R : f + ( x ) 2 a } = R, which is also Lebesgue measurable. Hence, f +is Lebesgue measurable. Considering the negative part f-,for all a 2 0 we have that { x E R : f - ( x ) 5 a } = { x E R : f ( x ) 2 - u } , which is Lebesgue measurable and for a < 0 we have {x E R : f - ( x ) 5 a } = 0, which is also Lebesgue measurable. Hence, f - is Lebesgue measurable and because we already proved that f’ is Lebesgue measurable we have proved that f is Lebesgue measurable. rn The underlying idea of the proof of part “ 5 j 1” for nonnegative functions f is to 1 partition the interval [O, n ) on the y-axis into intervals of length -. The proof shows 2n that the area under the functions that are used to approximate f should approximate the area under f , which means the idea of partitioning the y-axis can lead to a sensible notion of integration. Once measurable functions are characterized, it is helpful to determine how measurability relates to common algebraic operations.
+
Theorem 9.20 Let f , g : R -+ [--00, 001 be Lebesgue measurable functions. I f f g is dejined everywhere, then f g is Lebesgue measurable. Similarly, i f f - g or f . g is dejined everywhere, then it is Lebesgue measurable. Moreovel; f + , f- and I f I are Lebesgue measurable.
+
+
Proof. To see that f g is Lebesgue measurable, let a E R. We will use Theorem 1.36 to show that the set { x E R : ( f g ) ( x ) < a } is a countable union of Lebesgue measurable sets. If (f g ) ( x ) < a , then there is an E > 0 so that (f g ) ( x ) 26 < a. By Theorem 1.36, there are rational numbers r and s so that f ( x ) < r < f ( x ) E and g(x) < s < g ( x ) E . This means that f ( x ) g ( x ) < r s < ( f g ) ( x ) 2s < a , which proves the containment ‘‘S”in the equation below. The containment ‘‘2’’ is trivial.
+
+
+
+
+
+ +
+ + +
Because the latter set is a countable union of Lebesgue measurable sets, the set g)(x) < u } is Lebesgue measurable. Because a E R was arbitrary this means that f g is a Lebesgue measurable function. The proofs that f - g and f . g are Lebesgue measurable functions are similar (see Exercise 9-1 1). The functions f + and f - are Lebesgue measurable by Definition 9.18 and the Lebesgue measurability of If1 = f + f - follows from the Lebesgue measurability of sums of Lebesgue measurable functions. rn { x E R : (f
+ +
+
9.2. Lebesgue Measurable Functions
157
Exercises 9-9. Prove that a bounded function f : W -+ [0, 00) is Lebesgue measurable iff there is a sequence w ( s n ) z , of simple functions S n : W -+ R such that for all x E R the sequence ( s n ( x ) }n=l is nonincreasing and lim s n ( x ) = f ( x ) . n+oc
Hint. Mimic part ‘‘5+1” of the proof of Theorem 9.19 for nonnegative functions. 9-10. Let f : W -+ [-m, cc]be a Lebesgue measurable function. Use part 4 of Theorem 9.19 to prove that ( x E W : f ( x ) = cu } is Lebesgue measurable. 9- 1 1. Finish the proof of Theorem 9.20. That is, (a) Prove that if f,g : W + [ - x , m] are Lebesgue measurable and f - g is defined everywhere, then f - g is Lebesgue measurable. (b) Prove that i f f , g : R -+ [-m, co]are Lebesgue measurable and f . g is defined everywhere, then f g is Lebesgue measurable. Hint. This one is complicated because of negative signs. Prove the result first for f,g > 0, then use f = f+ - f- and g = g t - g - . 9-12. Prove that the sum of two simple functions is again a simple function. 9-13. Prove that f : R + [--00, co]is Lebesgue measurable iff for any two numbers a < b in W the set [ x E W : f ( x ) E [ a , b ) ] is Lebesgue measurable. 9-14. Use Definition 9.18 to prove that if f,g : f g is Lebesgue measurable.
+
W
-+ [0, co] are Lebesgue measurable functions, then
9-15. Let f , h : W -+ [-cc,x] and let f be Lebesgue measurable. Prove that i f f = h a.e., then h is Lebesgue measurable. 9-16. Let f .g : R -+ [-x,co]be Lebesgue measurable functions.
+
f,g : W + [-cc,co] are Lebesgue measurable and f ( x ) g ( x ) is de’ fined almost everywhere, then (f + g ) ( x ) := f ( x ) + g ( x ) ; if f ( x ) + g ( x ) is defined, IS lo: otherwise, Lebesgue measurable. Hint Apply Exercise 9-15 to the right auxiliary functions and then use Theorem 9.20.
(a) Prove that if
(b) Prove that if f,g :
R
+ [-m,
fined almost everywhere, then Lebesgue measurable (c) Prove that if
co] are Lebesgue measurable and f ( x ) - g(x) is deis defined, 1s ’ f ( x ) - g ( x ) ; if f ( x ) 0; otherwise,
(f - g ) ( x ) :=
l
f,g : W + [-x.m] are Lehesgue measurable and f ( x ) g ( x ) is defined almost
everywhere, then ( f g ) ( x ):=
if f ( x ) g ( x ) is defined’ is Lehesgue measurable. otherwise,
9-17, Let f,g : W +. [-m, 001 be Lebesgue measurable functions. (a) Prove that (b) Prove that (c) Prove that
{ x E W : f ( x ) = g ( x ) } is Lebesgue measurable. [ x E W : f ( x ) 5 g ( x ) ] is Lebesgue measurable. [ x E R : f ( x ) ig ( x ) } is Lebesgue measurable.
9-1 8. Let f . g : W + [-cu, x]be Lebesgue measurable functions (a) Prove that max(f, g ) (defined pointwise) is Lebesgue measurable. (b) Prove that min(f, g ) (defined pointwise) is Lebesgue measurable. 9-19, Let f : W + W be a nondecreasing function. Prove that f is Lebesgue measurable
9. The Lebesgue Integral
158
9.3 Lebesgue Integration Independent of whether the base is an interval or a potentially more scattered Lebesgue measurable set, the area of a “rectangle” should be the measure of the base times the height. This idea is behind the Lebesgue integral of simple functions.
Definition 9.21 Let A1, . . . , A , C JR be painvise disjoint Lebesgue measurable sets, n
let
al,
. . . , a,
E
[o, 00)
be nonnegative numbers and let s = n
function. We dejne the Lebesgue integral o f s by
UklAi,
k= 1
be a simple
n
ak1.4, d h :=
C akh(Ak). k=l
n
By Exercise 9-20a for any given simple function s the value
akh(Ak) does not k=l
7c
depend on the representation s =
ak1.4, that was chosen for s. Hence, Definition k=l 9.21 is sensible and we can proceed to more general functions. For a more general function, the Lebesgue integral is defined by approximating the area under the function from below with the area under simple functions.
Definition 9.22 Let f : R + [0,001 be a Lebesgue measurable function. We dejne the Lebesgue integral o f f to be f d h := sup
{
s d h : s is a simple function with 0 5 s 5 f
and we will call f Lebesgue integrable ifSthe supremum isfinite. A function g : R + E% will be called Lebesgue integrable i#g+ both Lebesgue integrable. We set
4
g d h :=
’8
dh -
1 and g- are
g- d h and call it the
Lebesgue integral of g . Continuing our comparison with the Riemann integral, Exercise 9-2 1 guarantees that for bounded functions that differ from zero only on a set of finite Lebesgue measure (the Riemann integral is defined for bounded functions on bounded intervals) there also is an approximation from above that will give the value from Definition 9.22. That is, unlike the RiemandDarboux integral (see Definition 5.27 and Theorem 5.29), for bounded functions that differ from zero on closed and bounded intervals the Lebesgue integral does not have any problems with an upper and a lower approximation not being equal. As noted after the definition of Lebesgue measurable sets (see Definition 9.4), nonmeasurable sets are quite hard to come by. Similarly, although we will always need to prove measurability for functions that we want to integrate, nonmeasurable functions are not expected to arise in practical applications. This means that the only possible problem in the definition of the Lebesgue integral is the potential for infinite area under
9.3. Lebesgue Integration
159
the graph. This is not a problem, because functions whose graphs enclose an infinite area cannot have a finite integral, independent of what notion of integration is used. Now that we have a sensible notion of integration that (as it ultimately turns out) does not have the weaknesses of the Riemann integral, we can establish some theorems about Lebesgue integrals.
Theorem 9.23 Let f , g : R + [-m, 001 be Lebesgue measurable functions. I. If0 5 f
5 g a.e. and g is Lebesgue integrable, then f is also Lebesgue
IRI r
integrable and
r
f dh 5
g dh.
R
2. f is Lebesgue integrable iff 1 f I is Lebesgue integrable and in this case the tri-
angular inequality 3.
Iff
>O,then
1
f d i i I I f I d h holds.
1
f d h = O z f f f =Oa.e..
Proof. For part 1, let N := {x E R : f ( x ) 2 g ( x ) } . By hypothesis, N is a null set. Let s : W + [0, 00) be a simple function with 0 5 s 5 f . Then s l ~ 5\ g~ is a simple function also and
sup
{b
s dh =
s l ~ d\h .~Hence,
s d h : s is a simple function with 0 5 s 5 f
= sup 5
1 1
sup
[1 {1
S~R\N
I
d h : s is a simple function with 0 5 s 5 f
s d h : s is a simple function with 0 5 s 5 g
1
.
I
Because g is Lebesgue integrable, the latter supremum is finite, and hence f is Lebesgue integrable. Because the suprema are equal to the respective Lebesgue integrals, we conclude that
b
f dh 5
b
g dh.
For part 2, first note that by Theorem 9.20 the function If1 is Lebesgue measurable. The direction "+"of the claim follows straight from part 1, because if 1 f I is Lebesgue integrable, then 0 5 f+ 5 I f 1 and 0 5 f - 5 I f I imply that f+ and f - are both Lebesgue integrable, which means by Definition 9.22 that f is Lebesgue integrable. For the direction "+,"let f : R + [--00. 001 be Lebesgue integrable. Then f f and - f - are Lebesgue integrable. To prove that I f [ = f + - f - is Lebesgue integrable I1
lets = x U k l & beasimplefunctionwith0 5 s 5 I f l . L e t P := {x E R :f ( x ) ? 0) k=l and N := { x E R : f (x) < 0). Then P U N = R and P and N are disjoint. Therefore n
n
160
9. The Lebesgue Integral
and 0 5 s- 5 f
-. But this means that
f - d h < co and thus 1 f
Therefore we conclude that i s Leoesgue integratxe. Finally, because f -, f
+
1
5 If 1 we conclude by part 1 that
L
For part 3, let f 2 0. First, consider the direction obtain 0 5 f 5 0 a.e. and by part 1 we conclude that P
f d h = 0. IB Conversely, for the direction
“+.”Because f f dh 5
= 0 a.e., we
0 d h = 0 , which
means
“+”let
s,
f d h = 0 and suppose for a contradiction
that A( {x E R : f (x) > 0 ) ) > 0. Then, because the countable union of null sets is again a null set and {x E R : f (x) > 0 } =
n
, there must be an
n and s
1 := - 1 we ~ obtain
n by part 1. Therefore we conclude that f = 0 a.e..
Exercise 9-15 shows that if a function is equal a.e. to a Lebesgue measurable function, then it must be Lebesgue measurable, too. Part 1 of Theorem 9.23 can be used to show that if two Lebesgue measurable functions are equal a.e., then their Lebesgue integrals must be equal, too (see Exercise 9-26). Basically this means that for integration null sets are insignificant. The following definition is therefore sensible because independent of how we extend the function f to all of R,either all extensions are Lebesgue measurable or none of them are (because for all a > 0 the set {x E R : g(x) < a } with g as in Definition 9.24 below differs from {x E R : f ( x ) exists and f ( x ) < a } at most by a null set) and either all extensions are Lebesgue integrable with the same Lebesgue integral or none of them are Lebesgue integrable (by part 1 of Theorem 9.23).
Definition 9.24 I f the function f : R g(x) :=
{ ofl(x)’
--f
[--30,
001 is defined a.e. and the function
is Lebesgue measurable, then we will call f is i f f ( x ) is not defined,
9.3. Lebesgue Integration
161
Lebesgue integrable iff g is Lebesgue integrable and we define the Lebesgue integral P
P
o f f to be /R f d h := /R g dh.
Theorem 9.25 shows that the Lebesgue integral is well-behaved with respect to the linear operations of multiplying with a real number and addition. Because of Exercise 9-16a and Definition 9.24 and because the set where the sum f g is undefined is a null set (see Exercise 9-27), we do not need to place any additional hypotheses on the functions in part 2 of Theorem 9.25.
+
Theorem 9.25 Let f ,g : R -+ [--00, a E R. The the following are true:
001
be Lebesgue integrablefunctions and let
1. af is Lebesgue integrable and
2. f
+ g is Lebesgue integrable and
s, + f
g dh =
s,
f dh +
g dh.
Proof. For part 1, note that by Theorem 9.20 with g(x) = a the function af is Lebesgue measurable. If f 2 0 and a 2 0, then af is Lebesgue integrable because af d h
= sup = sup
-
[s, [ [1
s d h : s is a simple function with 0 5 s 5 af
I
as d h : s is a simple function with 0 5 as 5 af s d h : s is a simple function with 0
a sup
5s5f
I
But this means that for any Lebesgue integrable function f and any a 2 0 the functions (af )+ = af + and (af ) - = a f - are Lebesgue integrable and
1
af d h
=
k(af)+dh-k(af)-dh=a
1
f'dh-a
1
f-dA
= a / Rf dh
Finally, for any Lebesgue integrable function f and any a < 0 the functions ( af )+ = -af- and (af ) - = --af + are Lebesgue integrable and
For part 2 first note that by Exercise 9-16a, Definition 9.24 and the preceding discussion, we can assume that f g is defined and finite everywhere and that f g is
+
+
9. The Lebesgue Integral
162
Lebesgue measurable. (Simply set f and g equal to zero where the sum is not defined.) Also note that part 2 is easily proved for simple functions (Exercise 9-20b), so we can use the additivity of the Lebesgue integral for simple functions in the following. We will first prove the result for nonnegative f and g. To see that f + g is Lebesgue integrable, suppose for a contradiction that f g is not Lebesgue integrable. Then for
+
each n E
{
N there is a simple function s, with 0 5 s, 5
F := x E
R :f(x) 3
For each n E
f(x)
+
2
[
f
and G := x E
'(')}
N one of the inequalities
1 1: :
-S,1F
Without loss of generality, assume that
+ g and
k
s, d h > n. Let
R : g(x) 2
+2
a k;
d h > - or
g(x)
n
1.
--s,lc; d h > - holds.
4
n - s , 1 ~d h > - holds for all n 4
E
N.Because
1 0 5 -S,*lF 5 + 1~ 5 f this implies that f is not Lebesgue integrable, which is a 2 2 contradiction. Hence, f g must be Lebesgue integrable. Now if s1 is a simple function with 0 5 s1 5 f and s2 is a simple function with 0 5 s2 5 g, then s1 s2 is a simple function with 0 5 s1 + s2 5 f + g. This
+
1
implies
s1
+
dh
+
arbitrary, we obtain
s2
I
sl
dh =
f dh
+
L
+ s?:d h 5
g dh 5
L
f
s,+ + f
g d h and because
s1, s2
were
g dh.
1
1 For the reversed inequality, let C, := x E R : - 5 f ( x ) g(x) 5 n for each { n n E N.We first prove that lim f g d h . Let E > 0 and let (f g ) lc, d h = n+m R k s d h > k f + g d A - - . N Eo t e s be a simple function so that 0 5 s 5 f g and 2
s
u CT:
that
C, = { x E
R : f ( x ) + g(x)
n=l
subsets A C {x E R : f ( x ) m
+
+
+
L
+
> 0 ) . Hence, by Theorem 9.12 for all measurable
+ g(x) > 0 ) we obtain n+Q3 lim h ( A n C,)
= h ( A ) . With
uJ l ~and , all aJ > 0 this implies
s = J=1
m
m
slc, d h = lim C a j h ( A j n C,) = C a j h ( A j ) = n+m j=1
Therefore there is an n E But then 0 5 slc, 5 ( f
j=1
N so that the inequality k . 1 ~ .d h
+ g ) l c , , and hence
L(f+g)lc,,dhz
s,
s dh.
> k s dh -
& -
2
holds.
163
9.3. Lebesgue Integration
{
Because f and g are both nonnegative, the sequence JR (f nondecreasing and we conclude that lim n+oo
Now let
E
> 0 and let n E <
{I,
Iw ” + (f
N be so that
(f
+ g) lc, d k }
is n=l
g ) lc, d h = E
+ g) lc, d h >
2
I
6
. Because f and g are bounded by n on Cn, the 4 ( W d + 1) proof of part “5+1” of Theorem 9.19 shows that there are simple functions sf and sg so that 0 I sf If l c , , 0 Isg F glc, and for all x E R the inequalities f ( x ) l c , (x) - s f ( x ) < u and g(x)lc, ( x ) - sg(x) < u hold. Therefore, Let u := min
-,
Because E > 0 was arbitrary, this proves the additivity of the Lebesgue integral for nonnegative functions. For not necessarily nonnegative functions, note that because If gJ5 I f / /gl, the above and part 2 of Theorem 9.23 show that f g is Lebesgue integrable when f and g are Lebesgue integrable. For the equality of the integrals, first notice that if f1, f 2 , gl, g2 are nonnegative,
+
+
integrable and satisfy we can conclude that
(f
+ g)’
- (f
+ g)-
f1
- f2 = gl fl
=f
dh -
- g2, then via f2
dh =
+ g = (f’ + g’)
1 +k fl
gl d h - (f-
+ g-)
g2 d h =
+
1+ f2
gl d h
g2 d h . Therefore with
we obtain
9. The Lebesgue Integral
164
which completes the proof.
+
+
The approximation of the integral of f g with integrals of functions ( f g) lc,, in part 2 of the proof of Theorem 9.23, shows that sequences of functions should be powerful tools. Convergence of sequences of functions is discussed in Chapter 11 and the fundamental limit theorems for (Lebesgue) integration are introduced in Section 14.5. Moreover, Exercise 14-33 gives a more efficient proof of part 2 using limit theorems for integrals.
Exercises 9-20 Integration of simple functions. (a) Let s be a simple nonnegative function, let 4'1, . . . , ym E (0, 03) be the nonzero values that s assumes, let a l , , , . ,a, E [O, M), and let A l , . . . , An G B be pairwise disjoint Lebesgue measurable sets so that s =
n
n
m
k=l
k=l
j=1
C a k l A k . Prove that C a k h ( A k ) = C y j h
(5-1
(yj1) .
(This proves that the integral in Definition 9.21 does not depend on the representation of s . ) Hint. Each of the ak must be a yj or zero. Group the first sum so that equal values U k are contiguous and then prove that the union of the corresponding sets Ak is the inverse image of the appropriate yj. (b) Prove that if $1 and s2 are simple functions, then
s,
$1
+ s2 dh =
s,
sl d h
+
s2 dh
Hint. Find Lebesgue measurable sets A 1 . . . . , An so that $1 and s2 are constant on each A j . 9-2 1 Let f : X + [O, mj be bounded and Lebesgue measurable so that h finite. Prove that f is Lebesgue integrable and
k 9-22 Let f :
f d h = inf
( {x
{k
iff the supremum S := sup
W
: f(x) > 0
} ) is
1
s dh : s is a simple function with f 5 s .
R + [O, 031 be a Lebesgue measurable function. Prove that
Lebesgue integral of f .
E
f is Lebesgue integrable
1
{s,
min(f, n ) l [ - n , n l d h : n E W is finite and that in this case S is the
9-23 Let f : R + [O, m] be Lebesgue integrable, let a > 0 and let A g R be Lebesgue measurable and SO
that f
-a
lA
> 0. Prove that
k
f
-
a l A dh =
f dh - uh(A).
9-24 Construct Lebesgue integrable functions f : B + R and g : R + W so that ( f and ( f + g)- f fg-.
+
9-25 Let a1 , . , , , a, E sets and let f =
+ gjf
f f'
+ gf
R,let A 1 , , , . , A n g B be (not necessarily pairwise disjoint) Lebesgue measurable
5
UklAk
k= 1
be a simple function. Prove that
s,
f dh =
5
a k h ( A k ) . Then explain
k=l
how this result differs from the result in Exercise 9-20a and why we could not have used it to prove Exercise 9-20a. Hint. For the proof of the equation, you may use Theorem 9.25.
9.4. Lebesgue Integrals versus Riemann Integrals
165
9-26. Let j , g : R + [-co,001 be Lebesgue measurable functions so that f = g a s . Use only Theorem 9.23 to prove that f is Lebesgue integrable iff g is Lebesgue integrable and that in this case we have IRjdA.=I
gdh. R
9-27. Let j : W
+-
[-co,co]be Lebesgue integrable. Prove that { x E JR : f ( x ) = cm } is a null set.
9-28. Let f,g : R +- [ - m , co]be Lebesgue integrable functions. (a) Prove that max(f, g) (defined pointwise) is Lebesgue integrable. (b) Prove that min(f, g } (defined pointwise) is Lebesgue integrable. (c) Prove that f - g is Lebesgue integrable and
s,
f - g dh =
s,
j dh -
g dh.
9-29. Let C Q be a Cantor set. Prove that l C p is Lebesgue integrable. Hinr. Exercise 9-6. 9-30. Prove that the Dirichlet function f ( x ) = 1~,[0,11is Lebesgue integrable.
9.4 Lebesgue Integrals versus Riemann Integrals We conclude this chapter by establishing the relationship between the Lebesgue integral and regular as well as improper Riemann integrals. The Lebesgue integral truly is an extension of the Riemann integral of bounded functions on closed and bounded intervals. To see this, we first show that Riemann integrable functions are Lebesgue integrable and that for these functions the Lebesgue integral equals the Riemann integral. Theorem 9.26 also shows how to overcome the small nuisance of Riemann integrals being defined on sets [ a ,b ] ,while the Lebesgue integral is defined on R.
Theorem 9.26 I f f is Riemann integrable over [ a ,b], then f R : E% + E% dejined by
{
f W ( x ) := ;()';
for "[ otherwise,
blJ
is Lebesgue integrable and the Riemann integral of
f is equal to the Lebesgue integral of
fR.
That is,
1
fps d h =
Proof. First, let f 2 0. Let P = {a = xo < . . . [ u , bl. With mi and Mi as in Definition 5.13, lets;
Ib
f dx.
xn = b } be a partition of
i
n
+
:= ~ m k l [ x r - l , x k mnl(xnl j and k=l
p XU
.-.-
n
+
M k l [ X k - , , X k ) M n l { x n l . Then 0 5 sf I f 5 sup. k=l
For all n E
N,consider the partition P,,
: k = 0, . . . ,2"
:=
I
. Then
sp 5 sF+' for all n E N and by Lebesgue's criterion for Riemann integrability (Theorem 8.12) we infer that lim s,p"(x) = f ( x ) a.e. Because fps and the s p are zero n+w outside [ a ,b ] ,by Exercise 9-15 this means that fw is Lebesgue measurable. Because f~ 5 Ml[,,bl for some M > 0, we conclude that fps is Lebesgue integrable. Moreover, for all n E
N we have L ( f , P,,)
=
1s p
dh 5
f R dh 5
s$ d h =
U (f , Pn).
166
9. The Lebesgue Integral
For each n E
N,there is an evaluation set T,
[
= t?),
integers k = 1, . . . ,2" the inequality f (t;)) - mk <
. . . , t2" 1
~
n(b - a )
(n)
I
so that for all
holds. But then we
1 infer R ( f , P,, T,) - L ( f , P,) < -. Because lim 11 Pn 11 = 0 we obtain n n-cc
lim L ( f , P,) = lim R ( f , Pn, Tn) = n+co
n+co
Similarly, we can prove lim U (f , P,) = n+oc
l
b
f (x)d x .
lb
f (x) dx. But this means
which establishes the desired equality. For f being an arbitrary Riemann integrable function on [ a ,b ] ,note that because f is bounded there is a real number B so that h := f B l [ , , b ] 2 0. Then h x is Lebesgue integrable, so f~ = h~ - B l [ , , b ] is Lebesgue integrable and
+
b
=
l h - B d x =
Ib
fdx.
The Lebesgue integral also incorporates parts of the improper Riemann integral as the next result shows. For improperly Riemann integrable functions as in Theorem 9.27, we also often say that the improper Riemann integral converges absolutely.
Theorem 9.27 I f f is Riemann integrable over every closed subinterval of [ a ,b ) and 1.f I is improperly Riemann integrable over [ a ,b) (where b is either a number or infinity), then fw : R + R dejinedpointwise by f R ( x ) := i;(x): f o r x E [ a ,b), 1s . otherwise,
{
b
Lebesgue integrable and the integrals are equal, that is,
fR d h =
f dx.
A similar result holds for functions that are improperly Riemann integrable over an interval ( a , b ] (where a is either a number or negative infinity) or over a set of the form ( a , b ) \ I a i , . . . ,a,}.
Proof. We will prove the result for b E ( a , co) and leave the case b = 00 to Exercise 9-31b. For Lebesgue measurability, first consider the case that f 2 0. For b-a : k = 0 , . . . , 2, and with mi and Ml as in Defall n E N,let P, := a k2,
{
inition 5.13, let
1
+
2"-1
SZ :=
a+(k-l19,a+k9). k=l
Then for all n E
N
we have
9.4. Lebesgue Integrals versus Riemann Integrals
167
s i 5 ";s I f and by Lebesgue's criterion for Riemann integrability (Theorem 8.12) for almost all x E [ a ,b) we infer that lim s E ( x ) = f ( x ) . Because f~ and the s; are n+cc
zero outside [ a ,b ) , by Exercise 9-15 this means that if f 2 0, then f~ is Lebesgue measurable. Applying this result to f f and f - separately implies that f is Lebesgue measurable regardless of what sign it takes. Now for all simple functions s with 0 5 s 5 If1 and all n E N the inequalities
l
b- 1
l
b
I f ( x ) /d x < 00 hold. Because n E N was s l [a ,b - arbitrary we can conclude that for all simple functions s with 0 5 s 5 I f ] we have the dh 5
inequality
1
I f ( x ) (d x 5
b
sdh 5
lf(x)l d x < 00 (Exercise 9-31a). Hence, I f ~ is l Lebesgue
integrable, so f x is Lebesgue integrable and
h l f I ~ Lb1
To prove that the two integrals are equal, let
1-8 b
dh 5
E
1
f ( x ) dx <
00.
> 0 and find a 6 > 0 so that
&
I f ( x ) /d x < -.Then 2
b
I 2 1 - 8 ( f ( x ) ld x < E .
Because E was arbitrary the two integrals must be equal. Note that not every improperly Riemann integrable function is Lebesgue integrable (Exercise 9-34). Moreover, not every Lebesgue integrable function can be integrated with a regular or improper Riemann integral (Exercises 9-29, 9-30, 12-248). Hence, neither integral can formally be replaced by the other. In fact, because cancellations as in Exercise 9-34 are sometimes desired in Lebesgue integration, improper Lebesgue integrals can be defined similar to improper Riemann integrals (Exercise 9-35). If we consider the earlier development of the Riemann integral in Chapter 5, it would be natural to target the Fundamental Theorem of Calculus as the next big result after the fundamental properties and examples that were presented in this chapter. Unfortunately, although it is quite beautiful, the Fundamental Theorem of Calculus for the
168
9. The Lebesgue Integral
Lebesgue integral has a rather technical proof. Exercise 9-36 is the first lemma for this result. Further lemmas are presented in Exercises 10-7, 11-20, 14-36 and 14-38 before the result itself is proved in Exercises 18-6 and 23-8.
Exercises 9-31. Finish the proof of Theorem 9.27. (a) Prove that if a , b
E
R, a < b, M
n E W we have that
z 0 and s is a nonnegative simple function so that for all dh
M , then
i
s dh 5
M.
Hint. Recall that simple functions are bounded. (b) Prove Theorem 9.27 f o r b = co. 1 9-32. Let f ( x ) = - 1(0,11.Prove that
8is Lebesgue integrable while f is not.
1 9-33. Let f ( x ) = - l [ ~ , ~ Prove ) . that f 2 is Lebesgue integrable while f is not. 30
9-34. Let f ( x ) := ~ ( - l ) n + l i l [n,n+l),
where the sum is taken pointwise. Prove that the improper
n=l
Riemann integral of f over [ 1, co)exists and that f is not Lebesgue integrable. Hint Harmonic series and the Alternating Series Test. 9-35. Define the improper Lebesgue integral of a function f : [ u , b ) + R,where b 9-36. Use the following steps to prove that if E with E
g
E
W U (00)
[ u , b] and Z is a family of closed subintervals of [a. b]
C UZ,then there is a finite, pairwise disjoint subfamily 3 g Z so that
lI I IEF
z -61h ( E ) .
I I
(a) For any closed interval I,let I * be the closed interval with the same midpoint and I * = 5 1 I I . Prove that if I , J E 1,I n J # 0 and 111 < 2 1 4 , then I G J*. 6 (b) Let So := sup { I f / : I E Z let I1 E Z be so that If11 > 2.Assume pairwise disjoint 2
1,
u n
I1, . . . , I, are chosen so that, with A , :=
Ij, for each I E
Zwe have I
f' An = 0
or
j=1
u I:. n
I
Prove that if there is an I
E
Z with I n A n = 0, we can continue the construction.
j=l
Hint. Use6,+1 := sup
[ Ill : I E Z, I n An = 0 }
(c) Prove that, independent of whether the construction in part 9-36b terminates in finitely many steps or not, if 3 is the family of intervals constructed in part 9-36b, then h ( E ) < 5 1 I1 . 1€3
(d) Prove that part 9-36c yields the requisite intervals.
Chapter 10
Series of Real Numbers I1 The essentials on series were introduced in Chapter 6 to facilitate the “summation of infinitely many numbers.” This chapter presents further aspects of the theory of series. Particular attention is given to power series, which are needed to define the transcendental functions in Chapter 12.
10.1 Limits Superior and Inferior The terms of a series are usually easier to handle than the partial sums. Hence, it would be useful to have convergence criteria based on properties of the terms. The first obstacle for devising such criteria is that sequences obtained from the terms of a series need not converge. The limits superior and inferior defined in this section can be computed for any sequence and they describe the sequence’s limiting behavior.
Proposition 10.1 Let {a,}?=l be a sequence of real numbers that is bounded above. ’x Then the sequence { sup{aj : j 2 n}},=l converges to a real number or to -a. Proof. The sequence of suprema is nonincreasing. If it is bounded below, then it converges to a real number. If not, then it converges to -m. Proposition 10.1 and a simple modification (see Exercise 10-1) show that the following definition is sensible.
Definition 10.2 Let {a,}Elbe a sequence of real numbers that is bounded above. The limit superior of {an}Elis defined to be lim sup a, := lim sup{aj : j 2 a } . For n+cc
n-+m
sequences that are not bounded above, we say that the limit superior is co. If is a sequence of real numbers that is bounded below, then the limit inferior of {a,}?=l is defined to be liminfa, := lim inf{aj : j 2 n } . For sequences n+m n+’x that are not bounded below, we say that the limit inferior is -m.
{a,}zl
The relationship between limit superior and limit inferior is the obvious one. 169
10. Series of Real Numbers II
170
Proposition 10.3 Let
{a,}El be a sequence in R.Then lim inf a, n+oc
Proof. Clearly, for all n E implies lim inf a, 5 lirn sup a,. n+oo
N we have inf{aj : j
5 lirn sup a,. ,-too
p n } 5 sup{aj : j 2 n } , which
n+oo
Exercise 10-3 shows that precise arithmetic with limits inferior and superior is impossible for most operations. At least for negative signs we have a simple “reversal.”
Proposition 10.4 For any sequence of real numbers lim inf -a, = - lirn sup a, and lirn sup -a, = - lim inf a,. n+cc
n+oo
n+m
n-oo
we have the equations
Proof. Recalling Exercise 1-19 we note that
The other equation is proved similarly. Finally, we should establish the relationship between the limit (if it exists) and the limits superior and inferior.
{a,}zl
Lemma 10.5 Let be a sequence of real numbers and let { b n } E 1be a convergent sequence of real numbers so that f o r all n E N the inequality b, 5 a, holds. Then lim b, 5 liminfa,. n-cx
n+oo
Proof. Let E > 0 and let L = lirn b,. Then there is an N E N so that for all n+oc n z N we have Jb, - L J < E , and hence a, p b, L - E . This means that for all n 2 N we have inf{aj : j 2 n } > L - E , and hence liminfa, 2 L - E . Because n+oo E > 0 was arbitrary we conclude lirn inf a, 3 L , which was to be proved. n+oc
Theorem 10.6 A sequence of real numbers converges ifs its limits superior and inferior are equal and real, that is, lim sup an = lim inf a, E R.In this case, we n+cc
n+oo
have the equalities lim a, = lirn sup a, = lim inf a,. n+m
n-oc
n+m
Proof. For “+,”let be a convergent sequence. Then by Lemma 10.5 and Exercise 10-4a we obtain lim sup a, 5 lim a, 5 lim inf a, E R.By Proposition 10.3, n+oo
n+oo
n+m
we also have lirn inf a, 5 lirn sup a,, which implies the two must be equal to each other n-toc
n-+x
and to the limit of the sequence. let lirn sup a, = liminf a, =: L E R and let For “e,” n-oo
n+bc
E
> 0. There is an N
so that for all n 2 N we have inf{aj : j 2 n } > L - E and sup{aj : j 2 n } < L converges to L . In particular, for all n 2 N we infer la, - L / < E , so The equation at the end of the theorem is proved in either of the above parts.
E
N
+ E. W
10.1. Limits Superior and Inferior
171
Exercises 10-1. State and prove a version of Proposition 10.1 that can be used to justify the definition of the limit inferior.
10-2. Limit superior, limit inferior, and subsequences. Let {a,]:=l
be a sequence of real numbers.
is the largest S E [-co,x] such that there is a (a) Prove that the limit superior of {a,]:=, subsequence { a n k } & with lirn ank = S. k+oc
(b) Prove that the limit inferior of ( u , ] ~ =is~ the smallest S ] ~ lirn ~ ank = S. subsequence [ u , ~ with
E
[-co,001 such that there is a
k-tm
10-3. Let
and { b n ] r z 1be bounded sequences of real numbers.
(a) Prove that lirn inf a, n-tm
+ b,
+ lim inf b,. n+w
2 lim inf a, n+bo
(b) Give an example that shows that the inequality can be strict. (c) Give examples that show that the limit inferior of a product of sequences can be greater or smaller than the product of the individual limit inferiors. 10-4. Let
{ a n ] g land
be sequences of real numbers so that a n 5 b, for all n
E
N.
(a) Prove that if {bn]gl converges, then limsupa, 5 lim b,. n-tm
n- - tm
(b) Prove that in general it is possible to have lirn sup a, > lirn inf b,. n+oc
n+m
10-5. Let { L Z , ] ~be =a ~ bounded sequence of positive numbers so that the sequence of the reciprocals is bounded, too. Prove each of the following. 1 1 (a) liminf - = n+m a, limsup,,,
a,
1 (b) limsup - = n--tm
’
a,
1
a,
liminf,,,
’
10-6. Let be a sequence of positive numbers and let (b,]r=l be a convergent sequence of positive numbers with nonzero limit. Prove each of the following. limn+= bn
(b) limsupa,b” = limsupa,
(a) lirn sup anbn = lim sup a, lim b,. n+w
n-sm
n-oc
n-+m
(n+m
1
10-7. Lebesgue’s Differentiation Theorem states that for any function f : [ a , b] + B of bounded variation the derivative f ’ ( x ) exists a.e. We will prove this result using the following steps.
(a) For any function f : ( a , b ) + B andx
E
(a. b),define D + f ( x ) := lirn sup
f(x
+h) - f ( x ) h
h, - f ( x ) , where “h + O+” means that h approaches zero form h the right ( h > 0) and “h -+ 0-” means that h approaches zero form the left ( h < 0). Prove that f is differentiable at x iff D + f ( x ) = D + f ( x ) = D - f ( x ) = D - f ( x ) and that in this case these values are equal to f ’ ( x ) . Note. The four values above are also known as the Dini derivatives of f at x. D - f ( x ) := liminf f ( x
+
h+O-
(b) Let f : [ a , b] + R be of bounded variation, let A := { x E (a. b ) : D - f ( x ) < D + f ( x ) }. let B := [ x E ( a, b ) : D - f ( x ) > D + f ( x ) }, and let C := [ x E ( a . b ) : D + f ( x ) = co Prove that f is differentiable at every x E ( a , b ) \ (A U B U C ) . Note. In particular, this means we have proved Lebesgue’s Theorem if we can prove that A , B and C are null sets. The remaining parts of this exercise are devoted to this task.
1
1
1.
10. Series of Real Numbers I1
172
(c) Suppose for a contradiction that h ( A ) > 0 and obtain a contradiction as follows. 1 1 i. Prove that h x E A : D- f (x) < 4 - - < q < q - < D + f ( x ) > 0 for some n n q E Q and n E N. 1 1 ii. With q , n as above let E := x E A : D- j ( x ) < 4 - - < q < 4 + - < D+ f (x) , n n 1 let g(x) := f ( x ) - q x , let E := - and let a = xo < x1 < . . . < xn = b be so n
(1
1)
+
I
I
n
t h a t x /g(xk)-g(xk-l)
1 > Vig-
k=l
I EZ
Eh ( E) .Pr o v et h at f o r al l x 6
1 )I1 > - h ( E ) and use this family to obtain the contradiction 6
E
En(xk-1,xk)
n
VXx-, g > V t g . k=l
(d) Use D- f (x) = - D - ( - f ) ( x ) and D + f ( x ) = - D + ( - f ) ( x ) toprove that h ( B ) = 0. 6Vbf 0 to a contradiction by letting M := -2.-and finding for each
(e) Lead the assumption h ( C ) x E C an hx with
I f ( x + h,)
h(C)
- f (x) 1 > Mh,.
Use Exercise 9-36 as in part 10-7(c)iv and arrive at a contradiction in a similar way.
10.2 The Root Test and the Ratio Test When it works, the Ratio Test for the convergence of a series is computationally convenient, because it only involves a division. Note that by using the limit superior and the limit inferior we avoid any issues with convergence of the sequence of quotients. ffi
Theorem 10.7 Ratio Test. Let
a j be a series with a j
# 0 for all j
E
W.
j=1
cc
a j converges absolutely. ffi
> 1 then
aj diverges.
If neither of the above conditions holds, then the series might converge or diverge. We also say that the Ratio Testfailed in this situation. < 1. Then there is a J
Proof. For part 1, let q := such that for all j 2 J we have
laj+ll
E
W
< qlajl, and hence lajl < qj-'/a~(. Let
10.2. The Root Test and the Ratio Test
173
laJ 1 laJ I M := J . Then for all j 3 J we obtain 0 5 laj 1 < q J - = M q j . By Comparison 4 qJ
c 00
Test (Theorem 6.15 and Exercise 6-16),
I *I
a j converges absolutely.
j=l
> 1, then there is a J E W so that for all j 3 J aj we have laj+ll > lajl. Hence, for all j > J we obtain lajl > laJl > 0 and the terms of the series do not converge to zero. Thus the series diverges. The last statement will be illuminated in Example 10.10.
For part 2, note that if lirn inf J+00
The limit superior and the limit inferior remove any analytic concerns about convergence of the quotient from the Ratio Test. But algebraic concerns remain regarding the possible division by zero. The Ratio Test cannot be applied directly to any series for which a subsequence of the terms is zero. The Root Test does not involve divisions. Thus it can be applied directly to any series. 30
a j be a series.
Theorem 10.8 Root Test. Let j=1
1. lflim sup
m < 1, then m > 1, then
m
aj
diverges.
j=1
j+m
Zf lim sup
a j converges absolutely. j=1
j-+m
2. Zflim sup
30
= 1, then the series might converge or diverge. We also say that
j-m
the Root Testfailed in this situation. < 1. There is a J E
Proof. For part 1, let q :=
N such that
c 30
laJ I < q J for all j 2 J . By Comparison Test,
aJ converges absolutely.
J=1
For part 2 , note that if lim sup .I/l.Ji > 1, then for any n
E
N there is a j
> n with
J+m
laJ 1 > 1. Hence, the terms do not converge to zero, and so the series diverges. The last statement will be illuminated in Example 10.10.
w
In theoretical investigations, the Root Test usually is preferred over the Ratio Test, because we do not need to worry about terms that are equal to zero. Exercise 10-8 shows that for any series for which convergence can be proved with the Ratio Test, convergence can (in theory) also be proved with the Root Test. This is another indication that the Root Test is preferable when developing a theory. The p-series test below provides examples of convergent and divergent series for which the Root Test and the Ratio Test both fail.
10. Series of Real Numbers II
174
Q.The sum
Theorem 10.9 p-series test. Let p E
c
OC1
- converges ifsp > 1.
j=l J P
1 1 Proof. If p 5 1, then for all j 2 1 we have - > .: Because the harmonic series
c
Jp
O01
- diverges (Example 6
j=l
j
4 , by Comparison Test
c -J
3 0 1
- diverges for p 5 1.
j=l J P
c+ k
This leaves the case p > 1. Note that the sequence of partial sums Sk =
is
j=1
increasing. Hence, we are done if we can show that it is bounded. To do this we use a chunking argument similar to the proof in Example 6.8.
.
2'+'
m-1
1
1D-1
Because the right side of this inequality does not depend on m , the Sk are bounded H and the series converges for p > 1.
5j,
Example 10.10 The Root Test and the Ratio Test both fail for the harmonic series M 1 which diverges, and for the series ?, which converges. For the Ratio Test,
c
/=I
i=l
J
this is a simple computation. For the Root Test, we need to use that lirn 1'30
postpone the verification to Exercise 10-16c.
f i = 1. We 0
Exercises x
a, be a series with a j f 0 for all j
10-8. The Root Test versus the Ratio Test. Let j=1
Explain why part 10-8a shows that if the Ratio Test proves that a series converges, then the Root Test also proves that the series converges.
10.3. Power Series
175
c m
10-9. Construct a convergent series
a j with
j=1
10-10. Prove the p-series test (Theorem 10.9) using the Integral Test (Theorem 8.29). 10-11. Use the Ratio Test or the Root Test to determine which of the following series converge. If a series converges, determine if it converges absolutely.
10-12. Use any of the convergence tests introduced so far (including those from Chapter 6) to determine which of the following series converge. If a series converges, determine if it converges absolutely.
10.3 Power Series With series giving access to “sums with infinitely many terms” it is natural to consider what happens when we let the sum in the definition of polynomials have infinitely many terms. The resulting notion of a power series is very versatile. In particular, it will enable us to introduce the transcendental functions in Chapter 12. 00
is called a Ck(X k=O power series centered at a (or a power series about a ) with coeficients C k . The power Definition 10.11 For a sequence
{Ck}z,,
the expression
c N
series is called convergent at x
E
ifand only
if lim N-KC
Ck(X
- a )k exists. Other-
k=O
wise it is called divergent at x .
Theorem 10.12 shows that power series converge on a symmetric open interval ( a - R . a R ) about the center point (where R could be infinite), and they diverge outside [a - R , a R ] . So only if R is finite are there two points, a R and a - R for which the convergence behavior is unknown. Note how in Theorem 10.12 R is defined in terms of the Root Test. Exercise 10-13provides an equally useful conceptual characterization of R.
+
+
+
x
Theorem 10.12 Let R :=
Ck(X
- a ) k be a power series.
1 lim supk+x?
x
If lim sup
iflim sup
$%TI’
= 00 let R := 0 and iflim sup
k+m
- a ) k converges absolutely f o r all x k= 1 and it diverges f o r all x E R with lx - a1 > R.
R :=
CC.
Then
c C k ( X
E
(0, m) let
k+x?
k=O
= 0 let
k - + 00 E
R with Ix - a1 < R
10. Series of Real Numbers 11
176
1 Proof. We apply the Root Test. Let the formal quotient - be 00. For x # a , 0 = Ix - a1 lim sup the limit superior lim sup is less than 1 iff
/-
k+m
1
Ix-al <
lim SUPk+00 This proves the result.
m
k+
00
and it is greater than 1 iff Jx-al >
c
1 limSUPk+00
00
Definition 10.13 Let
1 0
- a ) k be apower series. Then, with - := 00 this once,
Ck(X
k=O R :=
1
lim suPk-+oo
rn.
c 00
is called the radius of convergence of
Ck(x - a ) k .
k=O
Theorem 10.12 shows that power series define functions on intervals. These functions are limits of polynomials of increasing degree. To find out more about the analytic properties of these functions we need to investigate sequences of functions, which is done in Chapter 11. We conclude the present section with some remarks on the arithmetic of power series. Sums, differences and constant multiples of power series are straightforward (see Exercise 10-14). The multiplication of power series is similar to that of polynomials. We want to multiply each term of the first series with each term of the second and then collect terms with equal exponents. The product of a term with exponent i in the first power series with a term with exponent k - i in the second power series gives a term with exponent k for the product. We use this observation to first define a product for series.
c
00
x7
Theorem 10.14 The Cauchy Product of two series. Let
Uk
and
k=O
bk be absok=O
c cc c
00
converges absolutely and
c n
Pk =
k=O
n
7
Uibk-i. Then
Pk
k=O
i =O
m
k=O
k=O
Proof. First note that
a
ak
pk =
c oc
k
lutely convergent series and for each k 1. 0 dejine pk := bk.
k=O
k
k=O i=O
aibk-j =
c
ajbl =
Osj+l5n
n n-k
y,
akbi for
k=O i=o
00
every n
E
N. To see that c p k converges absolutely, note that for all n k=O
inequalities
E
& theI
177
10.3. Power Series
x
hold. Now let E > 0 and assume without loss of generality that each series has at least 00
one nonzero term. Then there is an even N E
N so that
lakl <
k= f 00
E
n
3 CEO Ibkl
and
. Therefore for all n 2 N we obtain
M
M
k=O
k=O
n
0
k=O
i=O
n-k
k=O i=O n
n-k
k=O n
i=o
k=O
-
E
I
I 0
00
00
i=n-k+l
k=n+l
1
n
00
00
0
1
k=O
k=O
k=O
k=O
0
1
1
k=O
F E E --+--+--=&.
3
3
3
co
Hence,
x
o c 0 0
Pk
=
k=O
bk, which completes the proof.
ak
k=O
k=O
It can be shown that in Theorem 10.14 it is enough to demand that one series converges (not necessarily absolutely) and the other converges absolutely. Because the proof is much more involved, we demanded here that both series converge absolutely.
c 00
Corollary 10.15 Let
c 00
Ck(X
- a ) k and
k=O
dk(X - a ) k be power series with radii of
k=O
k
convergence R, and Rd, respectively. With Pk := c c i d k - i f o r k p 0 , the radius
x
i =O
00
of convergence of
pk(X - a ) k is at least R, := min{R,, R d ] and f o r all x with k=O Ix - a 1 < R , the product of the power series is the power series with coeficients P k , 00
that is,
pi ( x - a ) =
k=O
(5
ck ( X - a ) k )
k=O
(5
dk ( x -
k=O
.
10. Series of Real Numbers
178
II
Proof. Exercise 10-15.
Exercises
c x
10-13. Prove that the radius of convergence of a power series
is the largest R
U k ( X - a$
c
E
[O, m] so
k=O
X
that
a k ( x - a ) k converges absolutely for all x with
/x- a / iR
k=O
Note. This conceptual formulation is often more helpful than the formula in Definition 10.13
c x
X
c k ( x - a ) k and
10-14. Let k=O
d k ( x - a ) k be power series with radii of convergence Rc and Rd
k=O
m
(a) Prove that the radius of convergence of C ( c k + dk)(x
C(C + dk)(x ~
c x
X
that
- a)k=
k=O
ck(x - a ) k
is at least min(R,. R d ] and
x
+
k=O
- a)k
c
k=O
dk(x - a j k for all x for which both se-
k=O
ries converge. Hint. Use the result from Exercise 10-13. (b) State and prove a result similar to part 10-14a for the difference of the two power series.
c x
(c) Prove that if b
E
c
W \ (01, then the radius of convergence of
c x
X
x for which
ck(x - a ) k converges, we have
k=O
bq(x
bck(x
k=O - a)k = b
k=O
- a)k
is Rc and for all
c x
ck(x - a j k .
k=O
10-15. Prove Corollary 10.15
c x
10-16. Consider the power series
k= 1
Xk
-. k
(a) Use the Ratio Test to show that the radius of convergence is 1 = 1.
(b) Use Theorem 10.12 to show that lim
n+w
(c) Prove that the Root Test fails for
c c
j=1
and
J
j=l
-
J2'
10-17. Use the formula from Definition 10.13 to compute the radius of convergence of the given power series. You may use the result from Exercise 10-16b.
Chapter 11
Sequences of Functions In Section 10.3, we considered power series as series of numbers for fixed x E R. Another way to look at them is as sequences of polynomials. This change in pointof-view is the first step toward function spaces, which are very important in analysis. This chapter describes ways in which sequences of functions can converge. Formally, a sequence of functions is a map from the natural numbers into a function space. Until we encounter function spaces in Example 15.3, a sequence offunctions will simply be, as in Defnition 9.17, a sequence whose “values” are functions, not numbers.
11.1 Notions of Convergence The most obvious way in which a sequence of functions can converge is at every point.
Definition 11.1 Let S be a set and for every n E N let f n : S -+ R be a function. The sequence of functions ( is called pointwise convergent to the function f : S -+ R i f f o r all x E S we have lim f n ( x ) = f ( x ) .
f,}rz1 n+m
Example 11.2 For n E N,let f n ( x ) := xt. On the set (0, 1) the sequence {fn],X==l converges pointwise to the constant function f (x) = 1.
By Exercise 2-46, for all x
E
1
(0, 1) the sequence x
1 3 0
ln=l
converges to 1.
Example 11.3 By Defnition 9.18 every real valued Lebesgue measurable function f is the pointwise limit of a sequence of simple functions. This is because if {sn},XE1is a sequence of simple functions that converges pointwise to f t and (tn)E1 is a sequence of simple functions that converges pointwise to f - , then the sequence {s, - t n ] E lconverges pointwise to f = f f - f - . 0 Pointwise convergence means the sequence of functions converges at every point, but the “speed” of convergence can vary from point to point. Uniform convergence requires that convergence happens at a uniform minimum speed (see Figure 23). 179
180
11. Sequences of Functions
f+E f f --E
0
I
I
I
I
Definition 11.4 Let S be a set and jor ever): n E N let j,, : ?, -+ lK be ajunction. is called uniformly convergent to the function The sequence of functions ( fn]r=l f : S + E% iff for all E > 0 there is an N E N so that for all n 2 N and all x E s we have lf,,(x> - f ( x ) J < E . It a sequence ot tunctions only converges (pointwise or unirormly) on a part 1 of the common domain S, we also say that the sequence converges (pointwise or uniformly) on T . It is easy to see that uniform convergence implies pointwise convergence (Exercise 11-1). The converse is false, as the next example shows.
Example 11.5 For n E N,let f,,(x) := xi. Then for any 6 > 0 on the interval (6,1) the sequence ( fn]TT1 converges uniformly to the constant function f (x) = 1. Howevel; on the set (0, 1) the sequence ( fn}r& does not converge uniformly to the constant function f (x) = 1. For the first claim, let 6 E (0, 1). Then for all points x E (6, 1) we have that 6; < x i < 1. If E > 0 is given, there is an N E N so that for all n 1. N we have 1 - E < 6; < 1. Consequently, for all points x E ( 8 , 1) and all n 2 N we infer 1 - E < 6i ixi < 1. Therefore on the interval (6, 1) the sequence { fn]T=l converges uniformly to the constant function f ( x ) = 1. To prove that the sequence { does not converge uniformly to f (x) = 1 on (0. l), suppose for a contradiction that it does. Then for every E E (0, 1) there is an N E N so that for all n 2 N and all x E (0, 1) we have 1 - E < xi < 1. But
j”,,}zl
y := (1 - E ) is~ in (0, 1) and fiy ((1 - E ) ~ = ) ((1 - E ) ” ) ~ = 1 - E 3 1 - E , a contradiction. Therefore, on the set (0, 1) the sequence { f,,}Z1does not converge uniformly to the constant function f (x) = 1. 0
181
11.1. Notions of Convergence Power series are important examples of uniformly convergent series.
c M
Theorem 11.6 Let
c
Ck(X
- a ) k be a power series with radius of convergence R.
k=O
ffi
Then
Ck(X
- a ) kconverges uniformly on any closed subinterval of ( a - R , a
+ R).
k=O
+
Proof. Any closed subinterval of ( a - R , a R ) is contained in a symmetric closed subinterval [a - r, a r ] with 0 < r < R. Thus we are done if we can prove the result for symmetric closed subintervals [a - r, a r ] E (a - R , a R ) . Let r E (0, R ) . For all x E [a - r, a r ] and all k E N U {0},we have that I C k ( X 5 Icklrk. For
+
+
+
u)~I
+ E [a - r, a + r ] , let P ( x ) := c c k ( x - a ) k be the limit of the power series. c13
each x
k=O 30
By Theorem 10.12,
Ck((a
+ r ) - a )k converges absolutely.
k=O
there is an N x
E
[a - r, a
N so that for all
E
n
>
n
I
I
k=O
I
E
> 0
co
N we have
lCklrk
< E . Therefore for all
k=n
+ r ] and all n 1. N we obtain
I
Thus for any
lk=n+l 30
and hence the power series converges uniformly on [a - r, a
+r].
More notions of convergence for sequences of functions are discussed in Section 14.6.
Exercises 11- 1. Prove that every uniformly convergent sequence of functions is pointwise convergent. I
11-2. In Example 11.5, consider the proof that the f n (n) = x i do not converge uniformly to the constant 1 . function f ( x ) = 1 on (0, 1). Could we have used E = - in this part of the proof? Explain. 2 11-3. Prove that for each S > 0the sequence {fn):zl with f n ( x ) = x” converges uniformly on (0, 1 - S ) , but it does not converge uniformly on (0, 1). 11-4. Prove that a bounded function f : B --t R is Lebesgue measurable iff it is a uniform limit of simple functions. Hint. Part “5=+1”of Theorem 9.19. 11-5. Give an example of a sequence {fn)r=, of bounded functions on an interval I that converges pointwise to an unbounded function f : I + W 11-6. A sequence of functions (fn)T=l defined on a set S is called pointwise Cauchy on the set S iff
]El
for all x E S the sequence [ f n ( x ) is a Cauchy sequence. Prove that every pointwise Cauchy sequence of functions is pointwise convergent to a function f : S + W.
11. Sequences of Functions
182
11-7. Approximating continuous functions with differentiable functions. Let [ a , b] be an interval (a) For continuous f : [a. b] -+ R and all x E R,let f ( x ) :=
f(x);
if x E [ a , b].
f ( b ) ; i f x z b.
For n E
N and all x E R, let fn (x):= n
x+;
f ( z ) d z . Prove that fn is differentiable on B.
(h) Prove that for every continuous function on [a, b] there is a sequence of functions that converges uniformly to f and so that every function in the sequence is continuous on [ a , b] and differentiable on ( a , b ).
[fn]Fz1
11-8. A sequence of functions defined on a set S is called uniformly Cauchy on the set S iff for all E > 0 there is an N E N so that for all n , m 2 N and all x E S we have fn ( x ) - f m ( x ) < E . Prove that every uniform Cauchy sequence of functions is uniformly convergent to a function f : S + W.
c cc
if
E
f
j=l
: ( a , b ) + IW be continuous, let
Prove that
[e,d ]
c ( a , b ) he a closed subinterval and let N
3
f
: (a,b ) +
andlet N :=
Ib 1
-1 ,’;/.
:= -
x + - converges uniformly to f on [c. d ] .
{gn]gN defined by g n ( x ) := f
Hint Uniform continuity. 1 1 - 1 1 . Let
1
I
c
j=1
11-10, Let
1
: [ a , b] + R be so that a, := sup { f , ( x ) : x E [a, b] ] < m. Prove that co a, converges, then fj converges absolutely at every x E [a, b] and uniformly on [ a , b].
11-9. For each j
N,let f j
I
R be continuously differentiable, let [e,d ] c
d ] . Prove that the sequence
( a , b) be a closed subinterval
f (x + ;) - f ( x ) [gn]EN defined by g n ( x ) := 1 n
converges uniformly to f ’ on [e,d ] . Hint Mean Value Theorem
11.2 Uniform Convergence If the function f is the (pointwise or uniform) limit of a sequence of functions [ f n } z l , it is natural to ask which properties of the functions in the sequence are inherited by the limit f . It turns out that pointwise convergence preserves neither continuity (see Exercise 11-12) nor Riemann integrability (see Exercise 11-13). Uniform convergence on the other hand preserves continuity (see Theorem 11.7), as well as Riemann integrability (see Theorem 11.9).
Theorem 11.7 Let { fn}Elbe a sequence of functions on [a, b] that converges uniformly to f : [a, b] + E% and let all f n be continuous at x E [a, b]. Then f is continuous at x. Proof. Let E > 0. Because { f n } , X , 1 converges uniformly to f on [a, b], there is an
N
N so that for all z
I
1
&
[a, b] we have f N ( z ) - f ( z ) < -. Moreover, because f N 3 is continuous at x, there is a S > 0 so that for all z E [a, b] with Iz - x /< 6’ we have & lf~(z)- f N ( x ) / < -.Thenforallz E [ a , b ] w i t h l z - x l <Sweconclude 3 E
I f ( z ) - f(x)l
5 <
E
/f(z) &
-
&
fN(Z)I &
-+-+--=&,
3
3
3
+ / f N ( Z ) - fN(X)I + 1f N ( X ) - f(x)l
11.2. Uniform Convergence
183
and hence f is continuous at x.
w
Corollary 11.8 Let [ f n } ~ z obe = l a sequence of functions on [a, b ] that converges uniformly to f : [ a ,b] + R and let all f, be continuous on [a, b]. Then f is continuous on [ a ,b]. Proof. Apply Theorem 11.7 at every point in [ a ,b].
w
Theorem 11.9 Let { f n j n E N be a sequence of Riemann integrable functions on the interval [ a ,b ] that converges uniformly to the function f : [ a ,b] + R. Then f is Riemann integrable and
l
b
f (x) d x =
.I
hl
b
f, (x) d x .
Proof. For each n E N,the function fi? is Riemann integrable. Therefore by Theorem 8.12 each set B, := {x E [ a ,b ] : w f ( x ) > 0} is a null set. Hence, by countable subadditivity of outer Lebesgue measure (part 3 of Theorem 8.6), the set oc
B :=
U B, is a null set.
NOW
for every x
E
[ a ,b] \ B all functions f, are continuous
n=l
at x, and by Theorem 11.7 we infer that f is continuous at x. Because B is a null set, by Theorem 8.12 f is Riemann integrable. For the integrals, let E > 0 and find an N E N so that for all n 3 N and all E x E [ a ,b ] we have f n ( x ) - f (x)l < -. Then for all n 2 N we obtain b-a
I
which establishes the equality.
w
Lebesgue’s integrability criterion for Riemann integrals reveals that being Riemann integrable is a substantially weaker property than continuity. It would be nice if this weaker property would be preserved by a weaker notion of convergence, such as pointwise convergence. However, Exercise 11-13 shows that the pointwise limit of Riemann integrable functions need not be Riemann integrable. This is another situation in which the Lebesgue integral has an advantage over the Riemann integral. For pointwise limits of (Lebesgue) integrable functions, please consider Section 14.5. With continuity and integrability investigated, we turn to differentiability. For sequences of differentiable functions, uniform convergence does not guarantee the differentiability of the limit.
Example 11.10 Uniform limits of direrentiable functions need not be differentiable. 2nt2 For every n E N,the function f n ( x ) := x2n+l is diflerentiable on (-1, l), but the uniform limit of (f,}Elis f (x) = 1x1,which is not dcfferentiable (also see Figure 24). It is clear that the f, are all differentiable. To see the uniform convergence to the absolute value function, let E > 0. For all x E (- 1, 1) we have
11. Sequences of Functions
184
I
Figure 24: A sequence of differentiable functions that converges uniformly to Ix 1. For an indication how pathological the situation can become, consider that by Exercise 117 every continuous function is the uniform limit of differentiable functions, while by Exercise 11-21 there are continuous functions that are not differentiable at any x E R.
5 1 and 1x1 < 1. Moreover, lim r k = 1 for every posin+m tive real number r . Let E > 0 and let N E W be such that for all n 2 N we have -11 < ~ . T h e n f o r a l l x ~ ( - ~ , ~ ) a n d a l l n ~ N w e o b t a i n 1 = 6, while for all x E (- 1, 1) \
(-E,
E)
and all n
> N we obtain
We have proved that for all E > 0 there is an N E N so that for all n 2 N and for all x E (-1, 1) we have fn(x) - f ( x ) l < E , which means that the sequence { 0 converges uniformly on (-1, 1) to f (x) = 1x1.
I
fn}Ei
To obtain a differentiable limit function, we require continuous derivatives and we impose a uniform convergence criterion on the derivatives. Theorem 11.11 Let { fn}El be a sequence of dzrerentiable functions on ( a , 6 ) so that for all n E N the derivative f; is continuous on ( a , b). I f ( convergespointwise to f : ( a ,b ) + R and the sequence { converges uniformly to the function g : ( a , b ) -+ IR,then f is direrentiable and f ’ = g .
f,!,)zl
fn}El
Proof. By Theorem 11.7, the function g is continuous, because (on every closed subinterval of ( a , b ) ) it is the uniform limit of continuous functions. To show that
11.2. Uniform Convergence
f' = g, let x
185
( a , b ) and let E > 0. Let 6 > 0 be such that for all y E ( a , b ) & with Iy - X I < 6 we have I g ( y ) - g(x)l < -. Let z E ( a , b) \ { x } with Iz - X I < S 3 be fixed. Because converges uniformly to g, there is an Ng E N so that for E
(f,'}zl
{fn}z1
&
all n 2 N g and for all y E ( a , b ) we have ]f,'(y) - g ( y ) l < T . Because converges pointwise to f,there is an Nf E N so that for all n $ Nf the inequality
I
'(')
-
- f n ( z ) - fn(x)
2 - x
2 - x
I
5
<
Value Theorem there is a c between
3
holds. Let n 2 max(Nf, Ng). By the Mean
z and x so that
f n (2) - f n (x) 2 - x
we conclude
&
&
&
-+O+-+-=&. 3 3 3
<
with Iz - x / < 6 we have
R=
= f,'(c).Therefore
-
2 - x
m'
Then
E.
Hence, f is differentiable at x
c
c
Ck(X - a ) k is dlfSerentiable on ( a - R , a
k = ~
33
the derivative is
- g(x) <
X
1 lim suPk+co
I
kck ( x - a)k-
'.
k= 1
Proof. First, note that (by Exercises 10-6 and 10-16b)
1
1
+ R ) and
11. Sequences of Functions
186
c cc
which means that the termwise derivative
c cc
of convergence as
c k ( x - u ) ~ .Let r
k=O
d + r ) we have dx
all x E ( a - r, a
(0, R). Then for every n
E
c
derivative is continuous on (a - r, a
+ r ) . Moreover,
22
X
N and and the
1
32
ck(x - a ) k
converges
n=l
c 30
converges uniformly to
c c
+ r ) and the derivative is
kCk(x -
on (a - r, a
+r ) .
k=l
cc
ln=1
Therefore by Theorem 11.11 the function val ( a - r. a
E
+ r ) and the sequence of the derivatives
ck(x - a ) k on ( a - r, a
k=O
E
[2
k:l
c
kck(x -
=ckck(x-
- a)
Ck(X
(k10
pointwise to
has the same radius
kck(x -
k=l
ck(x - a ) k is differentiable on the inter-
k=O cc
Because r
kCk(x -
E
(0, R ) was
k=l X
ck(x - a ) k is differentiable at every point x
arbitrary, k=O
E (a -
R,a
+ R ) and the
c __
ix)
derivative at x is
kck(x -
k=l
In particular, Corollary 11.12 shows that the derivative of a power series is again a power series. Therefore Corollary 11.12 can be applied to derivatives and higher derivatives of power series and we obtain the following.
c cc
Example 11.13 Any power series f ( x ) =
ck(x - a ) k with radius of convergence
k=O
R > 0 is infinitely differentiable on the interval (a - R , a derivative at x = a is f ( k ) ( a )= k ! C k .
+ R ) . Moreover, the kfh
0
In light of Example 11.13, it is natural to ask if every infinitely differentiable function is a power series. Lemma 18.8 will show that this is not the case.
Exercises 11-12. Let n
E
1 for 0 5 x 5 1 - -,
W. Prove that f n ( x ) :=
1
for 1 5 x 5 2, 0; f o r O i x i 1 , is not continuous on defines a continuous function on [O. 21 and that f ( x ) = 1; for 1 5 x 5 2,
1:
11.2. Uniform Convergence
187 converges pointwise to f.
[0, 21. Then prove that
11-13. Let { 4 j ) E 1be an enumeration of all rational numbers in [0, 11 and let n E
'
I
N.Prove that each func-
[O' I q l ' ' ' ' ' 4n1' is Riemann integrable on [0, 11. Then prove that tion f n ( x ) := O; for 1; f o r x E ( q 1 , . . . , q n l , the sequence { converges pointwise to the Dirichlet function f ( x ) = 0; for x E [O, 11 \ 0, 1; for x E [O, 11 n Q, which is not Riemann integrable on [O, 11. 11-14. Prove Theorem 11.11 as follows. Fix xo E ( a , b ) . Prove that g is Riemann integrable from xo to any x E ( a , b). Then use the Fundamental Theorem of Calculus to prove that for all x E ( a , b ) we have
l:
g ( t ) d t = n lim - x f n ( x ) - f n ( x o ) . From this finding, conclude that f ' = g
11-15. Explain why in the proof of Theorem 11.11 the hypothesis that all with the hypothesis that g is continuous.
fi are continuous can be replaced
[fn]r=l
11-16. A sequence of functions f,,: [ a , b] + R is called equicontinuous at x iff for all thereisaS z Osothatforalln E Nandallz E [a. 61 with It-XI < 6 wehave f n ( Z ) - f n ( x )
1
E
1
> 0 <
E.
(a) Prove that if { fn):=l is equicontinuous at x and converges pointwise to f : [ a , b] + B,then f is continuous at x.
(b) Prove that if [fn),X=l is a sequence of continuous functions that converges uniformly to the continuous function f,then {fi2),"=1 is equicontinuous at every x E [ a ,b]. 11-17. Give an example of a sequence of functions [f,],"=, on [0, 11 that converges pointwise on [0, 11 to . .
1
1
a Riemann integrable function f : [0, 11 + X and
f ( x ) d x # n
%$1
1
fn(x) dx.
11-18. Prove that if ( fn}F=l is a sequence of bounded functions on an interval I that converges uniformly to the function f : I --+ B, then f is bounded also. s
a3
c k ( x - a ) k and
11-19. Let k=O
let
E
x
d k ( x - a ) k be two power series with positive radius of convergence and k=O
x
E
N we have C k
- a$
for all x E
X with 1x - a J <
E.
Prove that
k=O
k=O
then for all k
dk(x
ck(x - a ) k =
z 0 be so that
= dk. Hint. Derivatives at a .
fn)FZ1
1 1-20. Fubini's Differentiation Theorem states that if { is a sequence of nondecreasing functions on [ a , b] so that f n 5 f n + l for all n E N,fn+l - f n is nondecreasing for all n E N and so that converges pointwise to f : [a. b] + R, then f ' ( x ) = lim f L ( x ) for almost all x E [a. b ] .
[fn]r=l
n+m
We will prove this result using the steps below. (a) Prove that f is nondecreasing. (b) Prove that there is a set A & [ a , b] so that h ( [ a , b] \ A ) = 0 and all the derivatives involved exist at every x E A . Conclude that we only need to prove the equality for almost all x E A. Hint. Lebesgue's Differentiation Theorem (Exercise 10-7). (c) Prove that for all n E N and all x E A we have f,'(x) 9 Hint Consider that fn+l = f n ( f n + l - f n ) .
+
fi+l(x).
(d) Prove that for all n
E
N the function f - fn is nondecreasing.
(e) Prove that for all x
E
A we have f i ( x ) 5 f ' ( x ) cc
[ f ( b )- f n k ( b ) ]converges and prove that
(f) Construct a subsequence [ f n k ] E sol that k=l
x
C [f(.x) k=l
-
f n k (XI] converges for all x
E [ a , b]
188
11. Sequences o f Functions m
(g) Apply what we have proved so far to g ( x ) :=
[f(x)
-
f n k ( x ) ] to prove that the series
k=l
c4
[f'(x) k= 1
fik ( x )1 converges at almost every x E A with limit less than or equal to g ' ( x ) .
{
Conclude that f ' ( x ) -
fik(x)\ k=m 1 converges to zero at almost every x
(h) Prove that lim f A ( x ) = f'(x) at almost every x "+a,
E
A , which establishes the result. x;
1 1-21. A continuous nowhere differentiable function. Let r (x) :=
and let s ( x ) := t ( x )
c13
+
(t(x-j )
+ t ( x + j ) ).
E A.
For all n
1-x; 0; E
1 forO<x<-, 2 1 for-<x
N let f n ( x ) := --s
i=l m
fn.
and define f := n=l
Prove that f is continuous because it is the uniform limit of continuous functions. 1 so that Prove that for every x E R and n E N there is a z E R with /z- x / = 4 ' 1000" -f(x) > and conclude that f is not differentiable at any x E W.
1
I
1
z-x
~
Hint.The nth sawtooth function f n is made up of straight line segments with slopes 1 1 0 " on 1
intervals of length __ 2 ' 1000" ' Prove that for every continuous f : [ a .b] + R that is differentiable on ( a , b ) there is a of continuous functions that are not differentiable at any point in ( a , b ) , sequence . v J l i n ~ l . L ~ ~ ~ ~ ~ ~ . ~ i f fn r. m l i q ' n ,
(fn)F=,
11-22. By Exercise 4-17, if the derivative of the function f : ( a , b ) -+ R is zero for all x E ( a , b ) ,then f is constant. This exercise shows that if the derivative is only zero almost everywhere, then f need not be constant even if it is continuous. The function @ below is called Lebesgue's singular function.
{ i]
co
Let Q =
, let C Q be the ternary Cantor set and use the notation of Definition 7 . 2 2 .
n=l
L e t n ~ N . P r o v e t h a t f o r a l l i ~. {. l. . . 2 " ) w w e h a ~ e ~1Z ~ ~ = ~
Let n
E
W. For every x
{ 1 , . . . , 2"}
E
[O. 11 \ 114" let ix
so that x is an upper bound of
E
I2 and
{ 1, . , 2") be the largest number in x # I s . For x E I f , , let i, := 0. ,
,
Define
Prove that $,, : [0, 11 + [0, 11 is continuous, surjective and @,
[6
Il?]
= [0, 11
i=l
Prove that [$n),"=l
[ I
converges uniformly to a continuous, nondecreasing function $
Prove that @ C Q = [O. 11 and that $ is constant on every subinterval of [O, 11 \ C Q ,which means tlr' = 0 a.e.
Chapter 12
Transcendental Functions In this chapter, we use the machinery built so far to define the transcendental functions and to investigate their properties. It would have been possible to define the transcendental functions earlier, but the proofs would have been harder.
12.1 The Exponential Function The question “Is there a nonzero function that is equal to its own derivative?” can be tackled with Corollary 11.12. If there is a power series with this property, then for 1 1 all k E we infer kck = Ck-1. That is, C k = -Ck-l = . . . = -CO. The most natural k k!
w
nonzero value for co is 1. Thus we are interested in the power series
Definition 12.1 For x
E
R,we define exp(x)
:=
c OC
k=O
c 3o
k=O
xk
-.
k!
xk
-.
k!
Theorem 12.2 The series that defines exp(.) has infinite radius of convergence. Thered fore exp(.) is differentiable on R with - exp(x) = exp(x) for all x E R. dx
Proof. Because the last convergence
terms of k ! exceed
, we obtain for the radius of
which proves that the radius of convergence is infinite. The differentiability of exp(.) on d R follows from Corollary 11.12. To see that - exp(x) = exp(x) we also use Corollary dx 11.12 and obtain the following. 189
190
12. Transcendental Functions
c 3o
d
- exp(x) =
dx
xk-l
k-
k=l
k!
= k=l
c5 oc
X
-
= exp(x).
1
n =O
Aside from being equal to its own derivative, the function exp(.) has interesting algebraic properties.
Theorem 12.3 For all x,y
E
R,we have exp(x) exp(y) = exp(x + y ) .
Proof. Let x ,y E R. Then via Theorem 10.14 (or Corollary 10.15) and the Binomial Theorem we obtain
Theorem 12.4 For all rational numbers r , we have exp ( r ) = (exp( 1))'. Proof. With an easy induction it can be proved that for all n E N and x E R we have exp (nx)= (exp(x))" (Exercise 12-1). It is also easy to see that exp(0) = 1. Moreover, for all x E R we have exp(x) exp(-x) = exp(x - x) = exp(0) = l , which means 1
. In particular, for all n E z we infer exp (nx)= ( exp(x))n. exp(x) P with . p E Z and q E N we obtain Hence, for all rational numbers -
that exp(-x) =
~
9
Theorem 12.4 shows that on Q the function exp(.) is an exponential function. Exponentials with irrational exponents have remained undefined so far. Because exp(.) is differentiable, it is continuous. The most sensible way to extend an exponential function defined on Q to all of R is so that the resulting function is continuous. Thus the following definition is reasonable.
Definition 12.5 Euler's number is dejined to be e := exp( 1). The function exp(.) is called the natural exponential function and f o r all x E JR we set ex := exp(x). By Theorem 12.4 for x E Q,the equality exp(x) = ex holds with the power on the right side being a power as in Definition 1.50. Next, we want to define powers with real exponents for all positive real bases. To do so, we first need to prove that e" has an inverse function.
12.1. The Exponential Function
191
Proposition 12.6 The natural exponential finction is a strictly increasing, bijective function from JR to (0, 00). Proof. Let x < y . By Definition 12.1 for any z > 0, we have ei > eo = 1. But this means eJ = eyPx+' = eYPxex > e x , so the natural exponential function is strictly increasing and, in particular, injective. Moreover, its values are greater than or equal to 1 on [0, 00). 1 For z < 0 note that e z e v 2 = e z P z= 1, which means that ez = 7 must be greater ethan 0. Therefore, ex maps R into (0. m). Now note that for x > 0 we have ex > 1 x , and hence lim ex = 00. ConseX'OO 1 quently, lim ex = lim - = 0. By Theorem 12.2, the natural exponential funcx+-x x--f--oo e-x tion is differentiable, and hence continuous. The above limits and the Intermediate Value Theorem show that the natural exponential function is surjective onto (0, 00). (MentallyJill in the details.) w
+
Definition 12.7 We define the natural logarithm function In : (0, m) + JR to be the inverse function of exp : R + (0, 00). The natural logarithm allows us to define powers of positive bases with arbitrary real exponents. With an argument similar to the proof of Theorem 12.4 we can show that this definition agrees with Definition 1.50 for rational exponents. (Exercise 12-2.)
Definition 12.8 Let a > 0 be a positive real number and let r E R.Then we dejine the rth power of a to be a' := exp ( r ln(a)) = erln('). Theorem 12.9 For all positive numbers a and b, and all real numbers x and y the following hold.
Proof. All properties follow directly from corresponding properties of the natural exponential function. The first property is proved as follows. u.xa~
= ex In(a)ey M a ) = ex W ) + yMa) = e ( x + ~M )a ) = ax+r
Working rows left to right we obtain the following for the second property. (ab)x = -
(eln(a)eln(b))x
= (eln(u)+ln(b))x= eln(el"(a)cl"(b) )x
e( In(a)+ln(b))x = eln(a)x+ln(b)x = ,1n(a)xeln(b)x = a X b X .
The third property follows from a x - J a J - x = 1 and ax-)'ay = a x . The remaining w properties are left as Exercise 12-3. Of course, the properties of the natural logarithm function are also of interest,
12. Transcendental Functions
192
Theorem 12.10 The natural logarithmfunction is diferentiable on (0,00) with deriva1 d tive - ln(x) = -. Consequently, allpowerfunctions f (x) := x r with r E R \ Q are dx X d dgerentiable on (0,00) and the Power Rule -xr = r 2 - l holds. dx Proof. By Theorem 4.21, the natural logarithm function is differentiable at every d 1 1 x E (0, 00) and - ln(x) = -= -. For the derivative of powers, the Chain Rule dx eln(x) x implies -xr d = d e r l n ( x ) = e r l n ( x ) r -1 = r x r - l . H dx dx x Because the natural logarithm function is differentiable, it is continuous, which allows us to extend the limit law for powers to arbitrary exponents.
be a convergent sequence of nonnegative numbers and
Corollary 12.11 Let let r
E
R.Then n+oo lirn a;
unless r < 0 and lirn an = 0.
=
n+oo
Proof. Exercise 12-4b.
H
Further properties of the natural logarithm function are exhibited in Exercises 12-5 and 12-6.
Exercises N we have exp(n) =
)" as claimed in the proof of
12-1. Prove by induction that for all n Theorem 12.4.
E
12-2. Prove that for all x > 0 and all r 12.8 agree.
Q the definitions of the power x r in Definition 1.50 and Definition
E
(exp(1)
12-3. Finish the proof of Theorem 12.9. That is, prove the following for all a , b > 0 and x , y (a)
(i)
a x
=
a" bX
(b) (a")'
= ax)
E
R.
(c) ax > 0
12-4. The limit law for powers (a) Prove that
lim ex = 0.
x+-m
(b) Prove Corollary 12.11 . Be careful with sequences that go to zero. 12-5. Prove that the natural logarithm function is a strictly increasing bijective function from (0, co) to E with lim In(x) = m and lim ln(x) = -co. x+o+
x+m
12-6. Let u , u > 0. Prove each of the following. (b) ln(e) = 1
(a) In(1) = 0 id) In
(:)
= ln(u) - W
u)
(c) ln(uu) = ln(u)
(e) In ( u L )= u ln(u)
12-7. Limits of nth roots (a) Prove that if nlirn an = a > 0, then n lirn +m +m Hint. Exercises 2-16 and 12-6.
g
m=a
+ ln(u)
193
12.2. Sine and Cosine (b) Prove that lim @ = 1. n-oc
1
1
12-8 Compute the integral
x3ex2 d x
12-9 Gronwall's Inequality. Let u , u : [ a , b] + [0,00) be continuous functions and let c ? 0 be so that for all x
E
[ a , b] we have u ( x ) 5 c
+ /'
u ( t ) u ( r ) d r . Prove that for all x E [ a , b] we have
Hint. Divide by the right side, multiply by u ( x ) and integrate.
1
co
12-10 The function r ( a ) :=
x 0 l - ' C x d x defined for CY z 0 is called the Gamma function.
(a) Prove that the improper integral
LW
xa-l e -' d x converges for a p 1.
(b) Prove that the improper integral also converges for 0
a < 1.
i
(c) Prove that the improper integral diverges for CY 5 0. (d) Prove that r(1)= 1. ( e ) Prove that for all a 2 1 we have
r ( a + 1) = a r ( a ) .
(0Prove by induction that for every natural number n we have T ( n ) = ( n - l)!. For this reason, the Gamma function is also referred to as the generalized factorial function. 12-11. Compute the following parameter dependent indefinite integrals. These integrals are useful for the integration of rational functions after a partial fraction decomposition.
12.2 Sine and Cosine Similar to the natural exponential function, the sine and cosine functions are defined via power series that have the right derivatives at the origin. Of course, this is a bit of "reverse engineering," because to obtain these derivatives we would need to quote arguments that rely on the geometric definition of these functions. oc ( - l ) k x 2 k + '
Definition 12.12 For x
E
R, we define sin(x) := k=O 30
sine function, and cos(x) := k=O
(- l)kx*k
(2k)!
(2k
+ 1)!
, which is called the
, which is called the cosine function.
Theorem 12.13 The power series that define sin(.) and cos(.) have infinite radius of convergence. Therefore, both sin(.) and cos(.) are direrentiable and moreover d sin(x) = cos(x) and - cos(x) = - sin(x)for all x E R. dx dx
12. TranscendentalFunctions
194
Proof. We prove the result for sin(.), leaving cos(.) to the reader in Exercise 12-12. For the sine function, note that for every x E Iw we have (- 1 ) k + l x 2 ( k + ” + 1
(2(k+l)+l)!
lim
kj.m
X2k+3
(2k
(-l)kx*k+’
(2k+l)!
+ 3)!
(2k + l)! xZk+’
= lim k+ca
X2
(2k
+ 2)(2k + 3)
= 0.
Hence, by the Ratio Test, the power series converges for all x E R,so its radius of convergence is infinite. By Corollary 11.12, the sine function is differentiable on R and
x1
=
(- l)kx2k
k=O
(2k)!
= cos(x).
The following identities are useful when working with sine and cosine.
Theorem 12.14 For all x, y E R the following identities hold.
+ cos2(x) = 1 (trigonometric law of Pythagoras) 2. sin(x + y ) = sin(x) cos(y) + cos(x) sin(y) 3. cos(x + y ) = cos(x) cos(y) - sin(x) sin(y) 1. sin2(x)
Proof. To prove the first identity, we proceed as follows. sin2(x)
+ cos2(x) (_1)kX2k+’ (2k
= n=l
2
+ l)!
(- 1); (- l)k ( 2 j + I)! (2k+ I)! 2j+1+2k+l=2n
( c
( - l ) J (-l)k
(-1)J
(-l)k
-1 n=l
k=O
1
12.2. Sine and Cosine
195
The remaining two identities are left for Exercise 12-13. The smallest positive zero of the sine function also has a special place in mathematics. Of course, we first must show that the sine function has a positive zero.
Proposition 12.15 Thehnction sin(x) ispositive on
Proof. For all x
E
(0,
&) , we have X4k+l
X2k+l
x
and it is negative at 4.
sin(x) = C ( - l l k (2k k=O
+
l)! = C ( 4 k + 1 ) ! k=O
)
X2
(4k
+ 2)(4k + 3) > o .
On the other hand, for x = 4 we obtain
42k+ 1
x
sin(4)
=
k=O C ( - l l k (2k + l)!
-
where the first term is negative because
42k+ I
4
D- ilk (2k + l ) !
k=O
= 4--
43
45
-
4 '
49
1.2.3 1.2.3.4.5 1.2.3.4.5.6.7 1.2.3.4.5.6.7.8.9 5.43 4.43 9.46 2.46 - 4-1.2.3.5 1 . 2 . 3 4 1.2.3.5.6.7.9 1.2.3.5.6.7-9 7 .43 .43 6.9.43 = 41.2.3.5.6.9 1'2.3.5.6'7.9 +
+
+--
+
12. Transcendental Functions
196 = 4-
472 118.43 =4-4.-<0. 1.2.3.5.6.9 405
Definition 12.16 The smallest positive zero of the function sin(x) is called n. We conclude by proving that the sine and cosine functions are periodic.
Definition 12.17 Let p > 0 be a real number: A function f : R -+ R is called periodic with period p zrfor all x E R we have f ( x p ) = f(x).
+
Theorem 12.18 Both the sine and the cosine function have period 2n Proof. Note that sin(2n) = sin(n + n) = sin(n) cos(n) + cos(n) sin(n) = 0. This implies cos2(2n) = 1 - sin2(2n) = 1 and because cos(2n) = cos(n
+ n)= cos(n>cos(n) - sin(n) sin(n) = cos2(n) > 0, R we obtain
we infer that cos(2n) = 1. Hence, for all x E
+ 2n) cos(x + 2n)
+
=
sin(x) cos(27r) cos(x) sin(2n) = sin(x) = cos(x) cos(2n) - sin(x) sin(2n) = cos(x).
sin(x
Exercises 12-12. Prove that the power series that defines the cosine function has infinite radius of convergence and d that - cos(x) = - sin(x). .t% 12-13, Finish the proof of Theorem 12.14. (a) Use power series to prove sin(x
+ y) = sin(x) cos(y) + cos(x) sin(y) for all x .
(b) Use the above and the law of Pythagoras to prove cos
(?) n = O a n d s i n ( ?3)7
J E
(c) Use the power series to prove sin(-n) = - sin(x) and cos(-x) = cos(x) for all x (d) Prove that sin (e) Prove that cos
(5 (-2
7r
- x ) = cos(x) for all x
E
B.(Use parts
= sin(x) for all x
E
B.
- .x)
(0, Prove that cos(x + y),=
R
= 1. E
8.
12-13a, 12-13b and 12-13c.)
c o s ( ~ ) , c o s i ~ ~sinixi,sin(xi,for ,all x . y E
B
12-14. Prove that n > 3. 12-15. A function f : [0, 27r] + B of the form f ( x ) = a0
+
n ( a j cos(jx) i--.l
trigonometric polynomial. (a) Prove the following product-to-sum formulas 1 i. cos(x) cos(y) = - [ cos(x y) cos(x - y) ] 2 1 ii. sin(x) sin(y) = - [ cos(x - y) - cos(x y) ] 2
+ +
+
+ bj sin(jx) ) is called a
12.2. Sine and Cosine
197
1 [ sin(x y) sin(x - y) ] 2 (b) Prove that the product of two trigonometric polynomials is a trigonometric polynomial,
iii. sin(x) cos(y) =
+ +
-
12- 16. Inverse trigonometric functions.
[
-1.
n n
Prove that the sine function is injective on - - , 2 2
[ 5,]; n n that - arcsin(x) = -f o r allx E (--, -1. 2 2
The inverse of the sine function restricted to -
is called the arcsine arcsin(,). Prove
d
d c 2 dx Prove that the cosine function is injective on [0, n].
The inverse of the cosine function restricted to [0, n]is called the arccosine arccos(.). Prove 1 d for all x E (0,n). that - arccos(x) = -dx qF2 sin(x) 12-17. The tangent function is defined to be tan(x) := cos(x) (a) Prove that the tangent function is differentiable on its domain and
d dx
- tan(x)
1 cosZ(x)
=-
n (- n2 , -). 2 The inverse of the tangent function restricted to (- 5, );
(b) Prove that the tangent function is injective on
-
is called the arctangent arctan(,). 1 d Prove that - arctan(x) = dx 1 +x2' (d) (Another integral for integration with partial fraction decompositions.) Compute the inte1 gralS dx. (c)
~
12-18. (The last integral needed for integration with partial fraction decompositions.)Proceed as follows to prove that for all natural numbers n > 1 we have
(a) Use integration by parts on
1
1 (x2 + b2)n dx =
1
1 . ( x 2 + b2)" d x X2
(b) The resulting equation contains an integral
(x2
+ b2)""
(b)
1
d x . Expand the numerator with
i b 2 and cancel what can be canceled. (c) Solve the equation for
s
(x2 + b2)"fl
dx.
12-19. Compute each of the integrals below (a)
/
Cxsin(x) dx.
.rex2 sin (x2) dx.
12-20. A representation of n (a) Prove that
l
b
1 .
sin"(x) dx = - - sin"-'(x)cos(x)
sinnP2(x) dx for all nat-
ural numbers n E N and all real numbers a < b. Hint. Use sinn(x) = sin"-'(x) ( 1 - cos2(x) ) and use integration by parts for the summand ( sinn-2(x) cos(x) ) cos(x).
198
12. Transcendental Functions Prove by induction that
1'.
2n-1 2n-3 IT sin 2n (x) dx = __ . -. . . - - for all n 2n 2n-2 2 2
2n 2n - 2 d x = -. -. . . - for all n E 2 n + 1 2n-1 3
Prove that
,7
-
2
= lim
-
n - + ~2n
n
k= 1
~
E
W.
N.
(2k)2 This is called Wallis' Product Formula. (2k- 1 ) 2 '
s,'
(x)dx ?
sin2"(x) dx ?
1%
sinZnf1(x) dx for all n
E
N,
IT
substitute the above expressions and divide by the expression in front of the -. 2
R is Riemann integrable and has a Riemann integrable derivative, then
that if f : [0, n ] +
12-21.
I
-1"
(x -
+ bzn
f ( k ) = L n f(x) dxf
ler's Summation Formula. Hint. Note that J';l (x
i)
-
f'(x) =
1
dx.This formula is called Eu-
f'(x)
2
. _
[ f (0) + f (1) ] -
/
1
f ( x ) dx and that the last integral
0
on the right side of Euler's Summation Formula can be turned into a sum of integrals like these by integrating from one integer to the next and applying the appropriate shift. n
12-22 An asymptotic expression for n ! . To see the idea for the first step, note that ln(n!) =
ln(k). k=1
(a) Apply Euler's Summation Formula to f ( x ) = ln(x
{
(b) Prove that / n (x Hint.
1
0
1' i) (x
X
1
- 1x1) __ dx]
converges.
't 1; + I' k)
x+l
dx =
-
+ 1) to prove that for all n E W we have
n=l
(x -
1) 2 x
(x
i k
n!e" (c) Let bn := __ Prove that (bn}r=lconverges to a limit b nnJTi'
1 b
(d) Use Wallis' Product Formula to show that - = lim
E
-
2
1
dx.
R
1 5 = ~.&
n+m b:
n!e"
(e) Prove that lim -n+m nn&
-
This result is called Stirling's Formula. It is often written as n ! read "is asymptotically equal to." 12-23
($1'
where "->, -.en fin&=
is
for # O3 Prove that w ~ ( 0=) 2. for x = 0.
12-24 A differentiable function with bounded, but not Riemann integrable, derivative (see Figure 25) (a) Prove that for all 8 > 0 there is an xg
(b) Prove that f g ( x ) :=
'OS
(4)
E
;
part 12-24a, is differentiable on R, but
( 0 , 8 )so that 2x cos
(:)
+ sin (:)
=0
for x 5 0, for (O' x8), where xg is a positive number as in
'
fi is not continuous at x = 0.
12.3. L’H6pital’sRule
199
If
IIII IIV V I l
v
IIII I V
w vI
IIII I1 v
v II
IIII
Figure 25: The differentiable function h in Exercise 12-24 oscillates so that the derivative h’ is discontinuous on a Cantor set (marked by blocks).
(c) Prove that g a ( x ) :=
L
f4 (XI;
6 forx 5 -
’ is differentiable on W,g; is discontinuous at forx z -, 2 5 (0.8).
fs (8 - x ) :
0 and at 6 and
(x
: ga(x)
#0]
(d) For any open interval ( a , b ) , let g ( , , b ) ( x ) := gIb-,l ( x - a ) , Prove that g(,,b) is differentiable, g;u,b) is discontinuous at a and at b and (x E W : g(,,b)(X) # 0) g ( a . b ) . cx
(e) Let C Q =
n
Cf be a Cantor set of nonzero Lebesgue measure, which exists by Exercise
n=l
8-3e. For each n
E
N let D i , i
tervals so that [O. 13 \ CR =
= 1, . . . , 2n - 1 be a sequence of pairwise disjoint open in-
-1
u
2”
,=I. .
2“-1
0;;.Let hn :=
gn:, , Prove that h := lim hn (taken n+m
, = l. .
pointwise) is differentiable on W and h‘ is discontinuous on C Q (0 Prove that h‘ is bounded, but not Riemann integrable on [0, I ]
(g) Prove that h‘ is Lebesgue measurable, and hence Lebesgue integrable on [0, I]. Hint. Represent h‘ as a sum of measurable functions and obtain (h’)-’ or intersection of preimages of ( u , 00) under these functions.
[ ( a , x ) ] as a union
12.3 L’H6pital’s Rule Limits of functions involving transcendental functions can be hard to compute. The algebra either involves power series, or it might even be so complicated that it is virtually impossible. L‘H8pital’s Rule is a way to replace the limit of a quotient with the limit of the quotient of the derivatives, which may be easier to compute. The idea behind oc
L‘H8pital’s Rule is easily explained with power series. If f ( x ) = x
f k ( x - a ) k and k=O
gk(x - a)k are power series with f ( a ) = g ( a ) = 0 and if g ’ ( a ) f 0, then
g(x) = k=O
200
12. Transcendental Functions
That is, the limit of the quotient of the functions is the quotient of the derivatives. Because not every function is a power series and because derivatives cannot be defined at infinity, in general we expect the limit of the quotient of the derivatives to f (XI f’(x) . . be on the right side. To prove lirn -- lim -, it is tempting to apply the X t “ g(x) x+a g’(x) Mean Value Theorem to the quotient, using that f ( a ) = g ( a ) = 0. But arguing
f (XI
f(x)- f ( a )
lirn -- lim x+a g ( x ) x+a
X -a
g(x)-g(a)
= lim x+a
f/o is problematic. g’(cg)
If c f and cg approach a
X-U
at different rates, then the limit need not be lirn - To get the right limit, we need x+a g’(x) cf = cg, which can be achieved with a stronger form of the Mean Value Theorem.
Theorem 12.19 Generalized Mean Value Theorem. Let f , g be functions that are continuous on [ a , b] and di#erentiable on ( a , b) and let g ( a ) f: g(b). Then there is a - f’(c> number c E ( a , b ) such that f ( b ) - f ( a ) g(b) - g(a> g’(c) . Proof. Let h ( x ) := [ f ( b )- f ( a ) ] g ( x )- [g(b) - g ( a ) ] f ( x )and apply Rolle’s Theorem. (See Exercise 12-25.) Now we are ready to prove L‘HBpital’s Rule.
Theorem 12.20 L’H8pital’s Rule. Let a E [-a, co]and let f,g be differentiable functions dejned on an interval ( z , 00) ( i f a = GO), or (-co,z ) ( i f a = -00), or (a - 6, a + 6 ) \ { a ) ( i f a E R). Ifthe limits o f f and g satisfy lim f ( x ) = lirn g ( x ) = 0 x+a
or both limits are in {hm}and lim x+a
f
X+a
f’(x> exists as a number or is infinite, then g’(x)
(XI
lim -- lim -. This rule also applies to one-sided limits.
g(x)
x+a
x--ta
g’(x)
f’o. First consider a E (-m, m] and L + g’(x)
Proof. Let L := lirn x+a
-00. We claim
f
(x)
that for all yo < L there is an xo < a so that for all x E (xo, a ) we have g(x) > Let yo < L . Then there is an x1 < a such that for all x E
f
’(XI
a ) we have g’(x) > In case lirn f ( x ) = lirn g ( x ) = 0, by the Generalized Mean Value Theorem for x-+a
all x
E (XI,
(XI,
x+a
a ) there is a c
E
( x , a ) so that
f ( x > f ( x >- f ( a ) - f’(c) > yo, which g(x>- g ( x ) - g ( a ) g’o
means in this case the claim holds with xo := XI. In case lirn f ( x ) = lim g ( x ) = 00,we can assume without loss of generality that X-+a
x--ta
f and g are positive on
[XI,
a ) . Let
E
> 0 be such that ___ < L . Find an (1 - &)2
12.3. L’H6pital’s Rule x2 E ( X I , a )
an xo
20 1
so that for all c
E (x2, a )
E (x2, a )
so that for all x
f’ ( c ) we have that -> g’(c)
E (xo, a )
we have that
~
(1 - E ) 2 ’
f(x) - f(X2)
Then find
> 1 - E and
- g (x2 ) > 1 - E . Then for all x E (xo, a ) by the Generalized Mean Value Theg(x) orem there is a c E ( x 2 , x ) so that f(x) - f ( x 2 ) - f’(c) >and hence for g ( x > - g(x2) g’(c) (1 - E l 2 ’ all x E (xo,a ) we obtain
f (x) f ( x )- f ( x 2 ) f(x) g ( x ) - g(x2) f’(c) > -(1 g(x) g ( x ) - g(x2) f ( x > - f ( x 2 ) g(x) g (c)
-El
2
> yo,
which proves the claim in this case. The other cases for the limits of f and g being infinite are proved similarly, so the claim is proved. The claim proves the result for L = 00 for left limits at a E R and for limits at infinity. Similar to the above we can prove that for L # 00 for all yo > L there is an xo < a so that for all x E (xo, a ) we have - < yo. This proves the result for L = -cm for g(x>
left limits at a E R and for limits at infinity. Putting the two results above together, for L E R and every xo < a so that for all x E (xo, a ) we have L - E < ’(’)
E
> 0 there is an
+ E , which proves the
result for L E R for left limits at a E R and for limits-at 60. Repeating the above process for a E [-m, 00) to the right of a establishes the result for a = -00, for right limits at a E R and for two-sided limits at a E R. Exercise 12-28 shows that L‘HBpital’s Rule is not a one-for-one swap. The limit lim f ( ’ ) can exist even if lim
x+a
g(x)
x-+a
f’(X> fails to exist. g’(x)
Aside from the obvious applications to quotients, L‘HBpital’s Rule also allows us to derive a well-known representation for the exponential function.
Theorem 12.21 For all x Proof. For all x
E
E
R we have ex
= lim (1 n+cc
+ x--)n .
R,we obtain
Because the natural exponential function is continuous, the result follows.
202
12. TranscendentalFunctions
Exercises 12-25 Prove Theorem 12.19. Remember to prove that Rolle’s Theorem can be applied 12-26 Compute each of the limits below. (a) lirn
2x4
(g)
+
-t x 2 x x 2 - 16
- 4x3
x+4
-
276
(b)
lim xln(x)
1 1 lim -- ___ sin(x) cos(x) - 1
12-27 Prove by induction that for all n E W we have lim x”e-’ X’cs;
12-28. Prove that for f ( x ) = 2x lirn x+m
ln(x)
(c) &mm
x+o+
= 0.
+ sin(x), g(x) = 2x - sin(x) and a = co we have
f‘cx, does not exist. g
lirn x+w
f (x) ~
g(x)
= 1, but
(XI
12-29. Let f : (0,co) + R be differentiable. Does lirn f’(x) = 0 imply that lirn f ( x ) exists? Justify x +30 X+lX your answer. 12-30, Is there a differentiable function f : (0, m) + W with lim f ’ ( x ) = 0 so that for every real number
(x~)?=~that goes to infinity and so that n+lim X + M
L there is a sequence
f ( x n ) = L? Justify your answer.
12-31. Creating summation formulas. This exercise shows one way in which summation formulas for
c n
powers of integers (see Exercise 1-33) can be discovered. Let f ( x ) :=
ekx.
k=l
c n
(a) Prove that for all p E
N we have
kP = f@)(O).
k= 1
n
(b) Prove that for all x E
W we have
ekx = k=l
(c) Prove h a t for all p
E
W we have
f(P’(0)
ex
- e(n+l)x 1-ex
dP = lim xiodxp
’
- e(n+l)x 1-ex
’
n
k =
(d) Use l’H8pital’s Rule to verify that k=l
22 (n + 1) for all n
E W.
(e) Use a computer algebra system to generate a closed formula for
k5. k=i
12-32. Use Cauchy’s Limit Theorem (see Exercise 2-51) to give an alternative proof of 1’Hbpital’s Rule.
Chapter 13
Numerical Methods
Many problems cannot be solved exactly. Therefore it is natural to consider computational approaches to mathematics. Numerical analysis is a wide field. For any problem that can be solved exactly (under good circumstances), there is at least one numerical method to provide an approximate solution in case exact methods fail. Usually, a numerical method contains a parameter, call it n , that indicates the computational effort required to obtain the approximation. With enough computational effort, a numerical method should provide approximations close to the exact solution. More formally, this means that as n goes to infinity the limit of our approximations should be the exact solution. But just having approximations that converge to the exact answers usually is not enough. We want to obtain good approximations with as little computational effort as possible. This means we not only need to assure that, given enough computational effort, the approximations converge to the correct result. We also must analyze how fast the approximations converge.
In the language that we have developed, this means that just showing that for every N E N so that for all n 2 N the nth approximation is within E of the exact solution is not enough. We also want our estimates to be sharp enough so that when, say, N = 10 guarantees a desired accuracy, we do not use a larger N and waste computational effort. So, where in proofs so far we were satisfied with the fact that N exists, in numerical analysis we want to know what N is. Where in proofs so far we were satisfied that estimates ultimately showed that a certain difference is smaller than E , in numerical analysis we want to perform the estimate with an N that is as small as possible. E
> 0 there is an
For this reason, this chapter will emphasize error analysis. We present numerical approaches for three typical tasks: The representation of functions in Section 13.1, the solution of equations in Section 13.2 and the computation of integrals in Section 13.3. 203
204
13. Numerical Methods
13.1 Approximation with Taylor Polynomials It seems mundane, but the most fundamental numerical task is the computation of the values of functions such as exponential and trigonometric functions. The exact values of these functions can only be computed for certain special input values x. For all other input values, we need to use approximation techniques. The most fundamental of these techniques is the approximation with Taylor polynomials. There are several ways to motivate the use of polynomials. Most importantly, polynomials are easy to compute, which is a paramount concern in numerical analysis. Moreover, for each of the many functions defined as power series there is a sequence of polynomials that converges to it. Geometrically, we can argue that the tangent line of a differentiable function at a point a has the same value and the same first derivative as f at a and, locally, it approximates f rather well. We have reason to hope that, by increasing the number of derivatives that agree with the derivatives of the function at a, we can enlarge the interval on which we have a good approximation for f . To increase the number of derivatives that agree at a , we need to use polynomials of degree greater than 1.
Theorem 13.1 Let the function f be n times diTerentiable at a. Then the polynomial f(j)(a) ( x - a)’ is such that thejrst n derivatives of Tn at a are equal T,,(x) := j!
c
~
i =O
to thefirst n derivatives off at a. That is, Tjk)(a)= f ( k ) ( afor ) k = 0, . . . , n.
Proof. Prove by induction that for 1 Ik In the kth derivative of T,(x) is T,‘k’(x) =
c ~
j=k
f ( j ) ( a ) j. (
j !
j - 1). .. ( j - k
+ l)(x - a ) j - k . (Exercise 13-1.)
Theorem 13.1 motivates the definition of Taylor polynomials and Taylor series.
Definition 13.2 Let thefunction f be n times diyerentiable at a. Then the polynomial T n ( x ) :=
c
f(j)(a)
~
i =O
j!
(x - a ) j is called the nth Taylor polynomial of f at a. f(j)(a) j!
___ (x - a ) j
is infinitely difSerentiable at a, the series T ( x ) := j=O
If f
is called the
Taylor series o f f at a. The definitions of the exponential and the trigonometric functions guarantee that the Taylor polynomials at a = 0 ultimately provide good approximations for these functions (see Exercise 13-2). However, as mentioned in the introduction, for numerical purposes it is not sufficient to just know that for some degree n the nth Taylor polynomial of f is close to f .We need to know how close T,, is to f .
+
Theorem 13.3 Taylor’s Formula. Iff is ( n 1) times continuously differentiable on ( a - R , a + R ) , then for all x E E% with Ix - a / < R we have
13.1. Approximation with Taylor Polynomials
205
if Ix - a / < R and M is such that for all x E IR with Ix M have f ( " " ) ( x ) I M , then ] f ( x ) - T,(x) I 5 Ix - aIn+l.
In particulal;
I
x
I
(n
- a1 <
R we
+ l)!
Proof. This proof is an induction on n. For the base step n = 0 note that for all R , a R ) we have
+
E (a -
For the induction step, we use integration by parts on the induction hypothesis. Let R ) and let f be n+2 times continuously differentiable on (a - R , a R ) . Then
x
E (a - R , a
f(x)
+
+
=
T,(x)
+ (x - a)n+'
=
Tn(x)
+ (x - ay+'
I'l y f ' " + ' ' + (a
(1 - U),+l ( n l)!
+
f(n+l)(a
u(x - a ) )du
+ u(x - a))
The remainder of the proof is left to Exercise 13-3. There are two ways to use error estimates like Taylor's formula. In an a posteriori or after the fact estimate, the polynomial and the interval are given and we estimate the error. This is straight substitution into the formula (see Exercise 13-4). In an a priori or before the fact estimate, we have a desired accuracy for an interval and need to find an n that guarantees this accuracy.
Example 13.4 Determine the degree n so that the nth Taylor polynomial off (x)= e" about a = 0 is within loe4 of ex for all x E [-2, 21. To bound the error by the maximum acceptable error of we make the upper bound of the error from Theorem 13.3 less than the specified acceptable error. This makes the actual error less than the acceptable error. In the notation of Theorem 13.3, we have a = 0, Ix --a I 5 2, and n is to be determined. All derivatives of ex are again e" and ex is increasing. Hence, the upper bound M for the (n l)Stderivative on [-2,2]
+
206
13. Numerical Methods
Figure 26: Taylor polynomials are local approximations to the function. The left graph shows the exponential function and its first (dashed), second (dotted) and third (dashdotted) Taylor polynomials. Note how the approximation gets better as the degree of the polynomials increases. The right graph shows the exponential function and its eleventh Taylor polynomial as demanded in Example 13.4. e2 2n+1 i where the inequality we need ( n l)! to solve is marked with an exclamation sign. The above inequality cannot be solved algebraically for n . However, n ! grows much faster than 2". Thus by substituting values for n and checking if the inequality is satisfied we find that n = 11 is large enough to guarantee the desired accuracy. For a visualization of the Taylor polynomials, consider Figure 26. 0
is e 2 . We set I f ( x > - T,(x)/ 5
~
+
Remark 13.5 Not every function can be approximated well with Taylor polynomials. For example, Lemma 18.8 exhibits a function that is infinitely differentiable, not identical to zero, and yet all its Taylor polynomials at a = 0 are identical to zero. 0 Remark 13.6 Early operating systems, such as the one on the Commodore 64 in the 1980s, used Taylor polynomials of sufficiently high degree to compute many functions. Nowadays, computational schemes that are faster than Taylor polynomials, but also more memory intensive, are used to compute functions. The reason is that memory is not as much of an issue as it was in the early days of computing, while speed remains a crucial concern. 0 Remark 13.7 Taylor polynomials are also used in physics to obtain low-order approximations of complicated functions f.Typically, i f f is to be evaluated at x Ax, where Ax is small, the exact expression f ( x Ax) is replaced with the approximately equal expression f ( x ) f'(x)Ax. This is feasible because, as Taylor's Formula shows, the difference is often bounded by C ( A X ) ~where , C is a constant. If Ax is small, terms of the order (Ax)* are usually negligible. The determination what is small and what is negligible is made based on practical, nonmathematical considerations. A posteriori, if the approximate formula correctly predicts an experiment, then the approximation must have been permissible. A priori, one could say that if other effects influence the quantity given by f by, say, 0.1% (of the underlying base unit), and Ax is at most I%, then (Ax)' is less than 0.01%, so it can be ignored because other effects will have greater influence. If a first order approximation as indicated does not work, higher0 order Taylor polynomials can be used in more sophisticated models.
+
+
+
13.1. Approximation with Taylor Polynomials
207
Functions can also be approximated with trigonometric polynomials. This idea is motivated by problems as described in Section 21.3. We will present the corresponding series, called Fourier series, in Section 20.2. The powerful tools available by then allow for a more efficient presentation than what would be possible now.
Exercises 13-1 Prove Theorem 13.1,
c m
13-2 Prove that if f ( x ) =
ckxk is a power series with nonzero radius of convergence, then the Taylor
k=O
c m
series o f f about a = o is
Ckxk
k=O
+ 1) times continuously differentiable on (a - R , a + R ) , then for all x E R 1 1 ( n +M I)! / x - a / " + ' , where M is such that for all 1x - a1 < R we have 1 f ( " + ' ) ( x ) I < M .
13-3 Prove that if f is ( n with / x
-
a / < R we have that f ( x ) - T n ( x ) 5
~
13-4 Find an upper bound for the error incurred when approximating f on [ l , I ] with its nth degree Taylor polynomial at a . (a) f ( x ) = e x , a = 0, n = 10, [ l , rl = [-5,51 (b) f ( x ) = sin(x), a = 0, n = 7, [ L , r ] =
[22 'T "1
13-5 Use induction to prove that the given expression is the nth derivative of the given function.
(a) f ( x ) = In 1x1,f ' " ) ( x ) = (bj f ( x ) = 2', f'"'(xj = 2'
- l)!
(-l)"+'(n
for n ? 1
X"
(ln(2)
)"
+ ne" 1 - a" f ( x ) = x e a x , f ( " ) ( x ) = anxeaX + -eax 1-a
(c) f ( x ) = x e x , f ( " ) ( x ) = .ex
(d)
13-6 Determine the smallest n so that polynomial of f about a .
1 T,(x) - f ( x ) 1
<
E
for all x
E
[ l , r ] , where T, is the nth Taylor
(aj f ( x ) = e x . a = 0,[ L , r ] = [-lo, 101, E = l o r 5 (b) f ( x ) = cos(x), a = 0, [ l , r ] = [-n, TI, (cj f ( x ) =
A, a = 1, [ l , r ] = [.5. 1.51,
E
E = = lo-''
(d) f ( x ) = In@), a = 2, [ l , I ] = [ l , 31, E = lo-* 13-7 Prove that for any a > 0, the Taylor polynomials of f ( x ) = In 1x1 at a converge for x E ( 0 , 2 a ) and they diverge for lx - a1 > a . 13-8 Second Derivative Test. Let f : (a, b ) + R be twice continuously differentiable and let x E ( a, b ) be so that f ' ( x ) = 0. (a) Prove that if f " ( x ) > 0, then there is an E > 0 so that for all z i:x with 1z - X I < E we have that f ( x ) < f ( z ) , (b) Prove that if f " ( x ) < 0, then there is an E > 0 so that for all z # x with ) z - X I iE we have that f ( x ) > f ( z ) .
208
13. Numerical Methods (c) State and prove a similar result for f : ( u , 6) + IF?, being n times continuously differentiable with f ’ ( x ) = f ” ( x ) = . . . = f ( n - l ) ( x ) = 0. (Distinguish even and odd n.) n
13-9. Efficient evaluationof polynomials. Let p ( x ) =
u j x j be a polynomial. j =O
(a) Provethatp(x1 = a ~ + x ( u l+ x ( u 2 + . . . + x ( u n - l + x ( a n ) ) . . . ) ) . n
(b) Count the number of operations in the evaluation of the sum
a j x j and in the evaluation j=O
in part 13-9a to prove that evaluation as in part 13-9a takes fewer operations (and is thus more efficient) than evaluation of the original sum. Hint. Evaluating u j x J takes j floating point multiplications and floating point multiplications take much more time than floating point additions. (c) State an n step recursive procedure that evaluates polynomials as in part 13-9a. Hint Start with Hn := an and define H n - l , . . . , H1 in such a way that H1 = p ( x ) .
13.2 Newton’s Method Solving equations is a common numerical task. The Intermediate Value Theorem guarantees that for equations f ( x ) = 0 the issue usually is not ifwe can find solutions, but rather how fast we can compute them.
Example 13.8 The bisection method. Let f : [a, b] + R be continuous with f ( a ) f ( b ) < 0. Then by the Intermediate Value Theorem f has a zero in [ a , b ] . To simplify the presentation, assume without loss of generality that f has a unique zero z in [ a , b]. We will recursively construct a sequence [ x , } ~ =that ~ converges to z . Let xg := a , x1 := b, and j ( 1 ) := 0. For the recursive construction, let xg,. . . , xn b-a and j ( n ) E ( 0 ,. . . , n - 1) be SO that f ( ~ ~ ) f ( x j (<~ )0 )and /xn- xj(n) = -.2n-1
I
Define xn+l := x n + x j ( n ) . If f(x,+l) = 0, stop the recursion because xn+l is the 2 desired zero. Otherwise, if f ( x n + l ) f ( x j ( n ) ) < 0 set j ( n 1) := j ( n ) , and if b-a f ( x n + l ) f ( x n ) < 0 set j ( n 1) := n. In either case, Ixn+l - . ~ j ( ~ + = l ) l-, 2n and we can continue the recursion. By the Intermediate Value Theorem and the uniqueness of z , for each n E N we b-a obtain Ix, - zI 5 -, and hence lim x, = z . 2n- 1 n+oo If f has multiple zeroes in [ a , b] the argument becomes a bit messier, but the sequence will converge to one of the zeroes of f.
+
+
Aside from establishing convergence, the estimate that in the nth step we are within b-a 2”-I of a zero of f tells us how many steps we need to execute to obtain an estimate of a given quality. For example, if b - a = 1 and we want a number that approximates the first 15 decimal places of the zero z , we would need to execute 51 steps of the bisection method (2-51+’ = 2-50 = 1,024C5 < 10-15).
209
13.2. Newton’s Method
Figure 27: An illustration of Newton’s method that will be analyzed precisely in Exercise 13-12. This is a large number of steps to carry out by hand. Nonetheless, to solve a single equation, a program would find the desired approximation rather quickly. However, when these methods were first designed, computers had not been invented, so a faster computation was desirable. For today’s applications, speed remains an issue, because in large scale computations we may need to solve billions of equations. This means every step we save would be magnified by a large factor, leading to a more efficient use of computational resources. Newton’s method reduces the (hard) computation of the zero of a differentiable function to the (easier) computation of the zero of the tangent line at a point. Say we are currently at a point x,. Then the tangent line of f at x, is the unique straight line that goes through the point (x,, f(xn)) and has slope f’(x,). Setting its equation y = f(x,) f’(x,)(x - x,) equal to zero and solving for x gives the expression
+
x=xn--
(”) for the zero of the tangent line. Of course, the zero of the tangent
f ’(x,)
line is not the zero of the function, but we would hope that it is closer to the zero of
f than x, was. Hence, we set x,+l := x, place of x, (see Figure 27).
-
’(“) and repeat the above with
f ’(xrl)
x,+1 in
Definition 13.9 Let f be a diferentiable function and let xo be a point in its domain. Then Newton’s method started at xo generates a sequence dejked recursively
f(x,) f o r n 2 0. by X,+I := X, - -
f ’(xn)
If the sequence generated by Newton’s method converges, then under mild hypotheses the limit is a solution of the equation f(x) = 0.
Theorem 13.10 Let f be a continuously dzferentiable function. If the sequence generated by Newton’s method converges to a limit L in the domain off and f ’ ( L ) # 0, then f ( L ) = 0.
210
13. Numerical Methods
Proof. Because the sequence of the recurrence relation.
lim
n+cc
{xn}Zn converges, we can take limits on both sides .
xn+l
-
= lim n-oo
(x,
- - leads to the equation
L = L - - f ( L ) , which clearly implies f ( L ) = 0. f ’(L)
w
The reader will prove in Exercise 13-10 that L is also a zero of f if f ’ ( L ) = 0. Another way of looking at the proof of Theorem 13.10 is to say that the limit L is a fixed point of the function F ( x ) = x - -. Fixed points are a standard tool in f’(x) numerical and abstract mathematics. We will encounter fixed points again in Theorems 17.64, 17.65, and 22.6. Theorem 13.10 says that ifNewton’s method converges, the limit is a zero of the function. It does not guarantee that the sequence from Newton’s method converges. However, there are mild conditions that guarantee convergence.
Lemma 13.11 Let q E [0, l), let F : ( a , b) -+ IR be a difSerentiablefunction so that F ’ ( x ) < q f o r all x E ( a , b ) and let p E ( a , b) be so that F ( p ) = p ( p is also called a fixed point of F ) . Then p is the onlyfixedpoint of F in ( a , b ) and f o r all xo E ( a , b ) the sequence generated by the recursive equation x,+l := F ( x , ) converges to p .
1
I
Proof. To prove that there is no F E ( a , b) \ { p } with p” = F ( 3 note that if F = F ( 3 , then by the Mean Value Theorem there would be a c between p” and p so that IF- p / = / F ( 3- F ( p ) l = IF’(c)/ IF- pl < IF- p i , whichis acontradiction. Hence, p is the only fixed point of F in ( a , b). Now let xo E ( a , b). Because the powers qn converge to zero, the result is proved if we can establish the inequality lx, - pi 5 q“ 1x0 - pI for all n E N.The inequality is proved by induction on n. For the base step n = 1 note that by Mean Value Theorem between xo and p there must be a c so that 1x1 - p ( = IF(xo) - F ( p ) l = IF’(c)/ 1x0 - pi I qlxo - p J . For the induction step note that there must be a c between x, and p so that w Ix?I+l-PI = l F ( x , ) - F ( p ) / = / F ’ W Ixn-pl 5 4 (qnIXO-pI) = qn+lIxo-PI.
Theorem 13.12 Let f : ( a , b ) -+ R be twice continuously diTerentiable. Then f o r each zero z off in ( a , b ) with f ’ ( z ) # 0 there is a 6, > 0 so that f o r ever)! starting point xg E ( z - 6,, z 8,) the sequence generated by Newton’s method converges to z.
+
Proof. We apply Lemma 13.11 to F ( x ) := x - f ( x ) . First note that
f ’(XI
In particular this means that F ’ ( z ) = 0, and hence there is a 6, > 0 so that 1 I F ’ ( x ) / < - < 1 for all x E ( z - 6,, z 6,). But then by Lemma 13.11 every se2 quence generated by Newton’s method started at any xo E ( z - 6,,z SZ) converges to z . w
+
+
21 1
13.2. Newton's Method
From Theorem 13.12 and its proof we infer that the closer xo is to a zero of f , the faster Newton's method will converge. But when Newton's method converges, the numbers xn will get ever closer to a zero of f . Hence, as Newton's method is executed, the speed of convergence should accelerate. Theorem 13.14 below makes this statement more precise.
Lemma 13.13 Let f : ( a , b ) + E% be continuously differentiable and let y > 0 be so that for all x,z E ( a , b ) we have that l f ' ( z ) - f ' ( x ) ( < y l z - XI. Then for all
Y ( z - x(2 holds. x,z E ( a , b ) the inequality 1 f ( z ) - f ( x ) - f ' ( x ) ( z - x) 1 < 2 Proof. Without loss of generality assume that x < z . Then
=
Y -x)2 -(t
2
JX
Y - x)2 = -(z 2
Theorem 13.14 Let f : ( a , b ) -+ R be a continuously differentiable function so that f ' ( x ) # 0 for all x E ( a , b). Assume there are xg E ( a , b) and a , j3, y > 0 so
1 :,(; 1
1
1 that - 5 a, so that for all x E ( a , b ) we have f'(x) i B, so that for all
I
( a , b ) we have f ' ( z ) - f'(x)/ 5 ylz -XI, so that h := - < 1 and so that 2 a with r := -we have [XO - r, xo r ] G ( a , b). Then 1-h
x,z
E
+
I . Each recursively dejinedpoint xn+l := xn - f ( x , ) is in (xo - r, xo + r ) . f'(xn) 2. The sequence (xn}ZO converges to a point u E [ X O -r, xo+r] with f ( u ) =O. 3. Foralln>Owehaveju-x,I
h2"-1 ( a - 1 - h2".
Proof. We first prove by induction that for all n E N the point xn is well defined, < ah2"-1-1 , and Ixn - xo/ < r . For n = 1 the above is trivial. For the lxn - xn-ll induction step, n + n 1 first note that because (xn- XOI < r the number xn+i is well defined. The definition of xn implies that f ( ~ ~ - 1+) f'(xn-l)(xn - x n - l ) = 0 , which implies
+
212
13. Numerical Methods But then, using a telescoping sum, we obtain the following.
which finishes the induction. To prove that {x~}:=~ converges, let m 2 n. Then lXm
- Xj-1
5
c m
lXk
- Xk-11 5 a
k=n+l
c m
h2k-'-1 = a
m-n-1
h2"-1h2"(2'-1) j=O
k=n+l
1 goes to zero as n goes to infinity, so 1 - h2" {x~}:=~ is a Cauchy sequence. (Mentallyfill in the argument.) In particular, (x,},"=~ converges to a number u , which by Theorem 13.10 must satisfy f ( u ) = 0. Letting m 1 go to infinity in the above estimate also shows that Iu - x, I 5 ah2"-' which 1 - h2" ' finishes the proof. Because 0 < h < 1 the bound ah2"-'-
~
Theorem 13.14 shows that once Newton's method is "close enough" to a zero of a function, it converges quite rapidly. Indeed, near a zero u of a continuously differentiable function f , the hypotheses of Theorem 13.14 can be satisfied if f ' ( u ) # 0, because we can make a small by starting near u . For comparison with the bisection 1 22"-1 method, suppose, for argument's sake, ct = j3 = y = 1. Then Iu - X n 1 I
ton's method six time; to obtain & approximation that gives the first 15 digits behind the decimal point of u. Another nice feature of Theorem 13.14 is that it can be generalized to several variables (see Exercise 17-44). Finally, note that even though Newton's method is only applicable to differentiable functions, Exercise 13-13 shows that it can be modified to provide a method that is applicable to all functions.
Exercises 13-10, Let f be a continuously differentiable function. Prove that if the sequence generated by Newton's method converges to a limit L in the domain of f and f ' ( L ) = 0, then f ( L ) = 0.
1
I
13-1 1. Let q > I, let F : ( u , b ) + W be a differentiable function so that F'(x) > q for all x E (a. b ) and let p E ( u , b ) be so that F ( p ) = p . Prove that p is the only fixed point of F in ( a , b ) and that for all xo E (a, b) \ [ p } the sequence generated by the recursive equation xn+1 := F ( x n ) terminates after finitely many steps with a value that is not in ( a , b).
13.2. Newton’s Method 13-12. Let f : ( a , b ) -+
213
R be twice differentiable.
(a) Let z E ( a , b ) be so that f ( z ) = 0, f’(x) > 0 for all x E ( z , b ) (that is, f is increasing on ( z , b ) ) ,and so that f ” ( x ) > 0 for all x E ( z , b ) (that is, f is concave up on ( z , b)).Prove that the sequence generated by Newton’s method started at any xo E ( z , b) converges to z . Hint. Prove that z 5 xn+l 5 xn for all n E N, Note. Figure 27 provides a geometric visualization of the claim in this exercise. Second note. In particular, it is allowed that f’(z) = 0 in this exercise. (b) Prove that i f f ” is continuous, then for each z E ( a , b ) with f ( z ) = f ’ ( z ) = 0 and f”(z) # 0 there is a 6, z 0 so that for every starting point xo E (z - 6,,z 6,) the sequence generated by Newton’s method converges to z.
+
13-13. Let
f be a function, let x0,xi
E
R with
f(x0)
# f ( x 1 ) and consider the recursively defined
(a) Show that the recursive formula is obtained by taking the equation of the secant line of f through ( ~ ~ - 1f(x,-l) , ) and (x,, f ( x n ) ) and computing its unique zero. (b) Prove that if the sequence generated with this method converges to L and bounded for x ,z near L , then f(L)= 0.
f ( z ) - f(x)
is
z-x
+
f ,z < x1 < xo, f is twice differentiable on ( z - 6,xo 8 ) for some 6 z 0, and f is increasing and concave up on ( z - 6,xo 6) (that is, f ” ( x ) > 0 for all x E (z - 6,xo 6)), then the sequence generated by this method converges to z . Hint. Prove that z 5 xn+1 5 X , for all n E N. (d) Prove that in a situation as in part 13-13c the sequence [ x , ) ~ = ~converges at least as fast to z (c) Prove that if z is a zero of
+
+
‘x
iz x,L o x,” for
as the sequence x N
Hint. Prove that 5
generated with Newton’s method and
5
xt =
XO.
all n
13-14. Explain why xo = 1 is not a useful starting point for finding the zeroes of f ( x ) = x 3 - 3x + 4 with Newton’s method. 13-15. Explain why xo = 1 is not a useful starting point for using Newton’s method to find the zeroes of 1. Sketch a rough graph of f and of the tangent lines used to the function f ( x ) = x 6 - 4x2 - x compute x1 and x2 to illustrate your point. 13-16. For the function f ( x ) = x 3 - 5 x - 5 , execute Newton’s method started with xo = -2. Use a calculator or a computer.
+
(a) Find x1,x2,x3, x4. (b) Find the first n such that your computer shows x, = xn+1. Explain why n is so large.
+
13-17. Apply Newton’s method to f ( x ) = 6x4 - 18x2 - 6 x 1 with xo = -1. Explain why the limit is not the zero o f f that is closest to the starting point. 13-18. The limit L computed with Newton’s method does not always give f(L)= 0.
+
Apply Newton’s method with xo = 1. Use a com(a) Let f ( x ) = (lO1oo los0) x puter and call the apparent limit L . (b) Compute f(L).Is f(L) zz O? (c) Compute the zero o f f exactly and compare it to L. Is L close to the zero? (d) Explain why Newton’s method cannot produce a better approximation to the actual zero o f f .
13-19. Square roots. (a) Use Newton’s method and the fact that &is the solution of the equation x 2 - a = 0 to devise a recursive method to approximate square roots. Note. This is one algorithm that is used in computers to approximate square roots. (b) Prove that for any xo > 0 the sequence generated in part 13-19a converges to & and for any xo < 0 the sequence generated in part 13-19a converges to
-A.
13. Numerical Methods
214
13.3 Numerical Integration We conclude our introduction to numerics with numerical integration. Although the Darboux and Lebesgue integrals are useful, even essential, for the development of integration theory, the Riemann integral and the Fundamental Theorem of Calculus are the best tools for numerical considerations. The key to numerical integration is to replace the function we want to integrate with a function that is close to it and which can easily be integrated. The simplest geometric idea is to choose points on the function and let a polynomial go through these points. This section will focus solely on this idea. For other approaches to numerical integration, consider [28].
n
Definition 13.15 Let X O , . . . , x, Li(X) :=
kfi
X
- Xk
__ .
xi - xk
E
R with xi # X k for i # k and define the polynomial
Then Li is called a Lagrange polynomial.
Theorem 13.16 Lagrange’s Interpolation Formula. Let n E N,xo, . . . , x , E R with xi # X k for i # k , and let f o , . . . , f n E R. Then the polynomial defined by n
thatfor all i = 0 , . . . , n we have P,,(xi) = fi.
Proof. The equation follows easily from the fact that for all j E {0, . . . , n } we have O’ if j ” Regarding uniqueness, suppose that Q was 1, if j = i .
*
another polynomial of degree 5 n with Q ( x j ) = f , for all j = 0, . . . , n . Then P - Q is a polynomial of degree 5 n with n 1 zeroes. By the Fundamental Theorem of Algebra (see Exercise 16-74 for a proof), this means that P - Q = 0 or P = Q.
+
With the polynomial P, going through the points (xi, f i ) the most natural way to approximate a function f is to set fi = f ( x i ) . We first note that if we start with a polynomial f of sufficiently low degree, then we simply get f back.
Corollary 13.17 Let f be a polynomial of degree 5 n and let P,, be a polynomial computed as in Theorem 13.16 with numbers f i = f ( x i )for distinct numbers xi E R, i = 0 , . . . , n. Then P,(x) = f ( x )for all x E R. Proof. Both P, and f are polynomials of degree 5 n that go through the points ( x i , f ( x i ) ) . The equality follows from the uniqueness of P,. To obtain a convenient integration formula, we need to express the integral of P, in terms of the fi = f (xi). Interestingly enough, as long as the xi are equidistant, the coefficients in the formula do not depend on the interval over which we integrate.
Proposition 13.18 Newton-Cotes formulas. Let n d-c tewal. Define h := -, f o r i = 0 , . . . , n let n
Xi
E
N and let
:= c
[c,d ] be an in-
+ i h , let f j
E
R and
13.3. Numerical Integration
215
't
Figure 28: Visualization of the approximation of an integral over two subintervals with Riemann sums with evaluation at left endpoints ( a ) , trapezoidal sums ( b ) and Simpson's Rule (c).
n
Pn(x) d x = h
fiai. i =O
x-c Proof. The key to the proof is the substitution t := h '
The most obvious way to approximate an integral is to approximate a function
f : [a. b ] -+ R with a polynomial of sufficiently large degree and then use the appro-
priate Newton-Cotes formula. However, for n 1. 7 some of the ai become negative (see Exercise 13-20), which leads to problems with cancellations of digits. Moreover, the a; are hard to compute for large n. Therefore an interval [ a ,b] is usually first partitioned into shorter subintervals, the formula from Proposition 13.18 is applied on each subinterval and then the results are added. This type of integration formula is called a composite integration formula. 1 1 For example, for n = 1, we obtain a0 = - and a1 = -. If we now partition the 2 2 b-a interval [ a ,b] into N subintervals of length Ax := -and apply Proposition 13.18 N on each subinterval we obtain the approximation
216
13. Numerical Methods
which is called the trapezoidal rule, also shown in Figure 28(b). In the trapezoidal rule, we need to evaluate the function N 1 times. The effort in numerical integration is proportional to the number of times the function f is evaluated. To compare the performance of two numerical integration formulas, it is thus important that both formulas evaluate the function equally many times. For this reason, we will construct composite Newton-Cotes formulas so that f is evaluated N 1 times. For n = 2, note that each additional subinterval of length Ax requires two additional evaluations of the function. Thus for n = 2 we must demand that N is even. 1 1 4 With a0 = -, a1 = - and a2 = - we obtain the approximation formula 3 3 3
+
+
N-1
”
\
\
k=l
which is called Simpson’s Rule, also shown in Figure 28(c). In Exercise 13-21, the reader will state composite integration formulas based on Newton-Cotes formulas for n = 3-6. We now turn to the error analysis once more. Peano’s error representation (Theorem 13.19) shows that a more “abstract” point-of-view can have benefits for concrete tasks, which makes it a perfect conclusion for Part I of this text and a lead-in for the more abstract Part 11. Consider that the integral as well as the Newton-Cotes formulas are functions themselves. They map functions on an interval to real numbers. Moreover, the integral as well as the Newton-Cotes formulas are linear. That is, sums and constant factors can be moved through the integral (see Theorems 5.8 and 9.25) and also through the Newton-Cotes formulas (easy computation). Linear functions on vector spaces (like vector spaces of integrable functions) will be important in abstract analysis (see Chapter 17). Because we adopt this more abstract point-of-view, the error representation actually is valid for any numerical approximation of integrals that is linear, that gives exact results for low-degree polynomials and that can be “moved into the integral” as described in the hypothesis of Theorem 13.19 (see [28] for examples beyond the Newton-Cotes formulas). By Corollary 13.17 the nth Newton-Cotes formula is exact for polynomials up to degree n. Because it only evaluates the function at select points X k , multiplies the values with numbers, and adds the results, it is linear and it can be moved into the integral. Hence, Peano’s error representation applies to the nth Newton-Cotes formula. Although the hypotheses for Peano’s error representation look complicated, the proof shows that they are just what is needed to get an estimate. Finally, Peano’s error representation also shows that we need to establish more abstract results to fully justify more concrete results, like error estimates. In the proof of Peano’s error representation, we work with double integrals and we reverse their order. Formally, we have not proved yet that this is possible. Fubini’s Theorem (see Theorem 14.66) will show that this reversal is indeed allowed. For the specific case of double Lebesgue integrals, Fubini’s Theorem is stated in Exercise 16-80. Because Theorem 13.19 is not used to prove these results, we will reverse the order of integration in the proof of Theorem 13.19, anticipating that this can be justified.
13.3. Numerical Integration
217
Theorem 13.19 Peano's error representation. Let n E N,c < d and let F ( . ) be an integration formula which is lineal; that is, F ( a f ,Bg) = a F ( f ) ,BF(g)for all a , E R and f , g : [c,d ] -+ R,which for polynomials p of degree at most
+
n gives F ( p ) =
+
Id
p ( x ) d x , and which for all continuousfunctions g : [c,d ] -+ R g ( t ) F ( ( x - t)nlx-tzO) dt. Forevery
g ( t ) ( x - t)nlx-r20 dt
R,let R( f ) := F ( f ) -
Riemann integrable function f : [c,d ] -+
kd
f ( x ) d x be 1 the error of the numerical integration and let K ( t ) := - R ((. - t)nl.-tzo). Then for n! every function f that for some 6 > 0 is n 1 times continuously diferentiable on the
+
interval ( c - 6 , d
+ 6 ) we have R ( f ) =
called the Peano kernel.
kd
f ( " + ' ) ( t ) K ( t )d t . The function K is also
Proof. By Taylor's Formula, we obtain the following for all x
f (x)
+
=
T n ( x ) (x - c)"+l
=
T,(x)
+
I'
f
("+l)
(c
+
1
U(X
E
- c))?(l n.
[c,d ] .
- u)" du
ld
f('+l)(t)+(x - t)nlx-r10 d t , n.
where T,(x) is the nth Taylor polynomial of f at c. Because the integration formula is linear, we infer for all functions g , h that R(g+h)
=
F(g+h)-
=
F'(g1-L
=
R(g)
d
g+hdx
Id
gdx+F(h)-k
d
hdx
+ R(h).
Now because the integration formula gives exact results for polynomials of degree 5 n we know that R(T,,) = 0. Because the integral in Taylor's formula above is a function of x we obtain the following. R(f) =
R(T,)
+R
1 f ' " + ' ) ( t ) - i ( x - t)'lx-tz0 dt n.
13. Numerical Methods
218
F =
( (X
- t)" ln-tzo
)-
ld
ld(x
- t)nlx-r20
1
dx d t
f ( " + ' ) ( t ) K ( t )d t . H
Although Peano's error representation is very versatile, we need something more concrete for applications. One key weakness of Peano's error representation is that it depends in a rather complicated way on the interval of integration. The next result makes this dependency more manageable for the Newton-Cotes formulas by reducing the error to a power of half the length of the interval, times a constant, times a value of an appropriate derivative.
Theorem 13.20 Let n E N and let f : [c. d ] +-R be n+ 1 times continuously differentiable ifn is odd and n+2 times continuously dgerentiable ifn is even. Let RE'd1(f ) be
f ( t ) d t with the nth Newton-Cotes for-
the error w9hen approximating the integral
/" d
inula F n ( f ) , that is, R F ' d l ( f )= F , ( f ) -
f ( t ) d t . Foroddn, there i s a
< E ( c ,d )
r
E
( c ,d ) so that R F . d l ( f )=
(n
+ 2)!
Proof. For odd n , by its definition the nth Newton-Cotes formula gives exact results for polynomials of degree n. For even n , the nth Newton-Cotes formula even gives exact results for polynomials of degree n 1 (see Exercise 13-22). Because the proofs for even and odd n are very similar, we will assume throughout that n is fixed and odd. The idea for the proof is a substitution that changes the domain of the integral from [c. d ] to [-1. 11. For all real numbers c < d , let KE.dl denote the Peano kernel for the nth NewtonCotes formula for the interval [ c ,d ] . This Peano kernel can be reduced to the Peano kernel KL-'.'] on [-1, 11 as follows. A simple substitution (similar to the computation
+
below, see Exercise 13-23) shows that KF.dl ( t
+
T)
d-c
d-c
= K,[-7'T1
( t ) for all
219
13.3. Numerical Integration t E
[
d-c --,
2
-1
d-c
2
. Moreover, for all x
E
[--1, I] we have
n!K,
d-c
=
F,
-
d-c -FA-','] 2
d-c
((
-9 1: (-Ud
--
2 __
-c 2
2
- -dX )- c
'I d - c
2
du
d-cx,O
T"-T -
1
((. - x), l.-xzo) -
11
(U - x )'
l u - x r ~du
1
To complete the error estimate, we need to use that the Peano kernels KE . d l ( t ) for the Newton-Cotes formulas on [c,d ] are of constant sign. The proof of this fact is geometrically obvious for n = 1 (see Exercise 13-24b)and still manageable for n = 2 (see Exercise 13-24c), but the algebra becomes increasingly tedious. Because we will focus only on n = 1 and n = 2 (trapezoidal and Simpson's Rule) in the examples, the presentation remains (somewhat)self-contained. For a general argument that also holdsfor other integrationformulas, consider [lo],which uses matrix methods toprove that the Peano kernel does not change its sign. Because the Peano kernel for the Newton-Cotes formulas does not change sign, by the Mean Value Theorem for the Integral as in Exercise 8-21 there is a [ E (c, d ) so that
=
f('"'(t)/
* d-c
K k . d l( u
-7-
1>
+ci-d
du
13. Numerical Methods
220
Note that the proof of Theorem 13.20 (and hence its conclusion) will work for any linear integration formula which can be moved into the integral, which gives exact results for polynomials up to degree n and for which the Peano kernel does not change sign. The reader will verify this for the Midpoint Rule in Exercise 13-25. With Theorem 13.20 providing a bound for the error for individual Newton-Cotes formulas, bounds for the error for composite Newton-Cotes formulas are now an easy consequence.
+
Corollary 13.21 Let n E N, a < b, 6 > 0 and let f : ( a - 8, b 6) + R be a function. If n is odd let f be n 1 times continuously difSerentiable and let M := max f ("+')(x) : x E [ a ,b] . I f n is even let f be n+2 times continuously d$
1
11
{
I
ferentiable and let M := max If""(x)I
+
I.
: x E [ a ,b ]
value of the error when approximating the integral
Let C F ' b l (f ) be the absolute
Lb f ( t ) d t with the nth composite
Ja
+
+
Newton-Cotes formula with N = n j intervals, that is, N 1 = n j 1 evaluations, ( b - a)n+2 nfl+l R[-"'1 n (xn+l) where j E N.Then ifn is odd we have C:3b1 ( f ) 5 M Nn+l ( n 1)!2"+2 '
+
Proof. If n is odd, by Theorem 13.20 we obtain
(b - a)n+2 nn+l R[-"'] =
Nnfl
(n
(~n+l)
+ 1)!2n+2
and a similar computation gives the result for even n. Note that in the error bounds in Corollary 13.21 only M depends on the function f , only ( b - a ) depends on the interval and only N depends on the number of evaluations. The remainder, although it looks complicated, is merely a constant. Therefore, as N --f 00 the numerical approximation of an integral of an n 1 times (if n is odd) or
+
13.3. Numerical Integration
22 1
+
n 2 times (if n is even) continuously differentiable function with the nth composite Newton-Cotes formula will converge to the actual integral. Of course, more concrete formulas will be better for given fixed n. For the trapezoidal rule and Simpson’s Rule, we obtain the error bounds indicated below.
Corollary 13.22 Let f : ( a - 6 , b
+ 6 ) + R be a function.
(b - a)3 , where 12N2 [ a ,b ] } . This is the error formula for the trapezoidal
1. I f f is twice continuously dgerentiable, we have Cy’bl(f ) I :K
{I
K = max f ” ( x ) I : x rule with N intervals.
E
~
(b - a)5 180N4 ’ where C = max f (i”)(x) : x E [ a ,b ] ] . This is the error formula for Simpson’s Rule with N intervals.
2. I f f is four times continuously dgerentiable, we have CF,bl(f ) 5 C
[1
~
1
Proof. For n = 1, we obtain
(b- a)3 which proves C p 3 b 1f () 5 K . The computation for Simpson’s Rule is similar 12N2 and left to the reader as Exercise 13-26. ~
If the function, the interval and N are given, the aposteriori error estimate is simply a substitution into the formula (see Exercise 13-27). If the function, the interval and a desired accuracy are given, the a priori estimate is obtained by demanding the error bound is less than the desired accuracy and solving for N .
Example 13.23 Find the number of intervals needed to estimate
s_:
e
2
-2
d x with the
trapezoidal rule so that the error is less than lop4. 2 12 First, note that with f (x) = e - T we have the derivatives f ’ ( x ) = - x e - T and 12
f ” ( x ) = ( x 2 - 1) e - T . The maximum of K-
( b - a13
12N2
I
2 1 0 - ~ means I-
23
12N2
<
1f”l
is assumed at x = 0, so K = 1. Now
2 or N 2 > -lo4. Hence, for N we obtain 3
I 7
81.65 and so N d ; always need to round up. N > 100 -
%
= 82 would work. Note that in the last step, we
222
13. Numerical Methods
Exercises 13-20. Compute the coefficients a; in the Newton-Cotes formulas for n = 7. Use a computer. 13-21. For each given n compute ao, . . . , a, for the Newton-Cotes formula. Then state the corresponding composite integration formula for N divisible by n. Finally, compute the error for the Newton-Cotes formula and for the composite integration formula. For the computation of the cq,a computer is recommended. 3 (a) n = 3. This is called the --rule. 8 (b) n = 4. This is called Mitne’s Rule. (c) n = 5. This rule does not have a specific name.
(d) n = 6. This is called Weddle’s Rule. 13-22. Prove as follows that for even n the nth Newton-Cotes formula gives accurate results for polynomials of degree n 1.
+
(a) Use the linearity property to prove that if the nth Newton-Cotes formula gives the exact integral for one polynomial of degree n 1, then it gives the exact integral for all of them.
+
(b) Prove that the coefficients a; of the nth Newton-Cotes formula satisfy ai = an-; for all i E (0,. . . , n ) . (cj Prove that if n is even, the nth Newton-Cotes formula gives the exact integral for the polynomial p ( ~=) x
(
-
c:d)n+l
-
and conclude that the nth Newton-cotes formula gives the
exact integral for all polynomials of degree n
+ 1.
(d) To illustrate that the result does not work for odd n , prove that the first Newton-Cotes formula does not give the integral of f ( x ) = x 2 on [-1, 11. 13-23. Prove that for all t
E
we have K F ’ d l ( t
c+d +T )= K n[-+.?I
(tj
13-24. The sign of the Peano kernel for the Newton-Cotes formulas (a) Explain why proving that the Peano kernel KA-”” for the nth Newton-Cotes formula on [-1, I] does not change its sign on [-I, 11 proves that all Peano kernels K F ’ d l for the nth Newton-Cotes formula on [c, d ] do not change signs on [ c , 4. (bj Prove that for n = 1 the Peano kernel KA-’“] is nonnegative on [-1, I]. Hint.Direct computation of the integrals, using that the approximating polynomial is a straight line.
’
(cj Prove that for n = 2 the Peano kernel KA- “ I is nonnegative on [ - 1, 13. Hint.This computation is more tedious. Make sure you use (x - t)31,-,z0 computations for t > 0 and t 5 0.
and do separate
(dj In a computer algebra system implement a short program that graphs the Peano kernel KL-” on [-I. 11 for arbitrary n. Note. While these graphs do not formally prove that the Peano kernel does not change its sign, the graphs and the implementation are instructive. 13-25. Let f : [a. b]--f
R be twice continuously differentiable. Prove that when approximating
b
f ( xj d x
with the midpoint rule, which uses Riemann sums with evaluation in the middle of the interval to ap( b - a)3 , where K = max { l f ” ( x ) I : a 5 x 5 b ) . proximate the integral, the error is bounded by K 24n2 Hint. Use Peano’s representation of the error and use the fact that the midpoint rule is accurate for polynomials of degree n 5 1. Then emulate the rest of the proof for the Newton-Cotes formulas, including the proof that the Peano kernel does not change its sign. ~
13.3. Numerical Integration
223
13-26. Prove part 2 of Corollary 13.22. 13-27. In each part, give an upper bound for the error of the approximation of the given integral with the given rule and the given number of intervals.
e - g dx,trapezoidal rule, N = 20
(a) (b) (c) (dj
/-:
e - g dx,Simpson’s Rule, N = 20
l5
sin ( x2 ) d x , trapezoidal rule, N = 50
s,’
sin ( x2 ) d x , Simpson’s Rule, N = 50
13-28. Compute the number N of intervals needed to approximate the integral with the indicated rule so that the error is at most the given v . (a) (bj (cj
l:
e - g dx,Simpson’s Rule, u 5
l2 i4
dx,trapezoidal rule, u i. lo-’
In(xj& d x , Simpson’s Rule, v i. lo-*
13-29. Approximate the integral with the indicated rule so that the error is at most the given u Hiizt. Use the error bounds in Corollary 13.22 to compute the number of intervals. Then compute the requisite sum with a computer, (a) (bj (c)
ll Ll‘ L2 1
1 2/5;;e
-2 2 d x , trapezoidal rule. u 5 lo-*
1 2 Z e - T dx,Simpson’s Rule, v 5
2
z
sin
(e)
1
e
(:)
-2 2 d x , Simpson’s Rule, u 5 lops
dx,trapezoidal rule, u 5
10
(0
1‘
f i e X d x , Simpson’s Rule, v
5
I
13-30. Let the function f : [ u ,b] + B be continuously differentiable. Use Euler’s Summation Formula (see Exercise 12-21j and an appropriate substitution to prove that the approximations of the integral
lb
f ( x ) dx with the trapezoidal rule with N trapezoids converge to
Ib
f ( x ) d x as N + w .
Part 11 Analysis in Abstract Spaces
Chapter 14
Integration on Measure Spaces Throughout this second part of the text, we need to integrate multivariable functions. Therefore we start our investigation of the more abstract realms of analysis with integration. Recall that the fundamental idea behind Lebesgue integration was the partition of the range (see Figure 22). Moreover, when we worked with Lebesgue measure, we were concerned with properties stated in terms of sets. We rarely used the fact that these sets were subsets of the real numbers. Therefore integration of functions from a more general space to [-m, m] should be similar to the theory developed in Chapter 9. Indeed, Sections 14.1-14.4 basically recast and sort the results of Chapter 9 to show how these ideas generalize to arbitrary measure spaces. In particular, this generalization makes it possible to talk about Lebesgue integration in Rd.Sections 14.5 and 14.6 provide fundamental results on sequences of integrable functions. These results are the cornerstone for the proofs of many important facts about integrable functions. Finally, Sections 14.7 and 14.8 show how measures on products (such as R2 = R x R) are related to measures on the factors. In this chapter, we experience the full power of abstraction for the first time. Once the abstract core of concrete results for the real numbers is identified, the familiar results from the real numbers can be established in much more general contexts, sometimes even with the same proof. It is important to realize, however, that this generalization comes at a price, We must very carefully check that we did not use any specific properties of the real line in the proof of the concrete result. The most important property of the real line that is lost in the general setting is the linear ordering. Proofs and results that depend on this ordering do not generalize easily. This is why the linear ordering of the real line was used sparingly in the first part of the text.
14.1 Measure Spaces To be most widely applicable, abstract integration is defined on sets equipped with a structure that makes them a “measure space.” In this fashion, we do not need to worry about details regarding the shape and dimension of the domain. The fundamental idea for integration in arbitrary spaces is the same as for Lebesgue integration. Partition the
225
14. Integration on Measure Spaces
226
/&
Ak+l
Ak-1
Figure 29: For integration in more complicated spaces than the real line, we retain the main idea from Lebesgue integration. We partition the range (here the z-axis) and measure the size of the inverse images Ak of intervals in the range (the z-axis). The sum of these sizes times the corresponding heights
should approximate the
“volume” under the graph of the function. range and approximate the “area” or “volume” under the function with “generalized rectangles” or “generalized boxes” whose bases are sets for which we can determine the “measure” (see Figure 29). It was noted after Definition 9.4 that even on the real line there are sets for which we cannot determine a sensible “measure” using outer Lebesgue measure. Thus, it is not surprising that on a general set M we need to consider the subset of the power set P ( M ) that contains all subsets for which we can determine the “measure.” The properties that define these subsets are directly inspired by properties of Lebesgue measurable sets (see Theorem 9.10).
Definition 14.1 Let M be a set. A subset C & P ( M ) is called a sigma algebra or a-algebra ifs 1. 0
E
c,
2. I f S E C, then S’ E C, 32
3. I
~
c for~ all n E W,then U A ,
EA
n=l
E
C.
14.1. Measure Spaces
227
Our most important examples of a-algebras so far are the power set itself and the set of Lebesgue measurable subsets of R.
Example 14.2 1. Let M be a set. Then the power set
P(M)of M
is a a-algebra.
2. The set Ch of Lebesgue measurable subsets of R is a a-algebra.
0
Part 1 is trivial and part 2 is Theorem 9.10.
More examples of a-algebras are given in Exercise 14-1 and Theorem 14.25. Because a-algebras are newly introduced entities, we need to prove that the properties we know from Lebesgue measurable sets also hold in this general context.
Proposition 14.3 Let M be a set and let C E P ( M ) be a 0-algebra. I f for every W
n
N we have A ,
E
E
C,then
An
E
C.
n=l
Proof. For each n
E
N by part 2 of Definition 14.1, we infer M \ A ,
of Definition 14.1, we obtain
n
E C . By part 3
M \ An E C.By DeMorgan’s Laws, this means that
n=l
co
M \
u co
n co
A , E C . Therefore by part 2 of Definition 14.1
n=l
A, E C,which concludes
n=l
w
the proof.
Proposition 14.4 Let M be a set, let X
u
_C
P ( M ) be a a-algebra, let N
E
N and let
N
A ] , . . . . A N E C.Then
A,
E
C.
n=l
Proof. For n 2 N
+ 1, let A ,
u N
:= 0. Then
n=l
u 00
A, =
A , E C.
w
n=l
Further properties of a-algebras are presented in Exercise 14-2. Recall that for the definition of Lebesgue measurable functions we never referred to the measure itself. Thus for some purposes it will be sufficient to work with a set and its measurable subsets. This is the idea behind a measurable space.
Definition 14.5 A measurable space is a pair ( M , C ) consisting of a set X and a a-algebra C 5 P ( M ) . The sets in C will also be called C-measurable sets. Finally, a measure is a function that assigns each measurable set its “measure.” The only conditions for a sensible “measure” function are that the empty set has no volume and that the volumes of pairwise disjoint sets are added to obtain the volume of their union.
228
14. Integration on Measure Spaces
Definition 14.6 Let ( M , C) be a measurable space. Then p : C + [O, measure iJj’
001
is called a
1. ~ ( 0 = ) 0, and 2. ,u is countably additive, that is, sets A ,
E
C, then p
( fi
if
is a sequence of painvise disjoint
!x
An) =E w ( A n ) .
The definition of a measure space connects the “measurable sets” with a function that assigns the measure.
Definition 14.7 A measure space is a triple ( M , C , p ) consisting of a set M , a sigma algebra C P ( M ) and a measure p : C + [O,CQ]. Example 14.8 With h denoting outer Lebesgue measure and Ch denoting the sigma algebra of Lebesgue measurable sets, (R,Ch,h ) is a measure space. We have already noted that Ch is a 0-algebra. Trivially h ( 0 ) = 0, and the countable additivity of Lebesgue measure is given by Theorem 9.1 1. 0 Example 14.9 Let M be a set. For A M , we define the counting measure y~ ( A ) to be the number of elements of A i f A isfinite and co i f A is infinite. Then ( M , P ( M ) , y ~ ) is a measure space. The reader will prove this in Exercise 14-3. Note that counting measure will allow us to model absolutely convergent series as integrals in ExamQles 14.36 and 14.37 below. 0 In Chapter 9, we defined Lebesgue integration over R,but we never formally defined Lebesgue integration over subsets of R such as intervals. The formalism of measure spaces allows us to easily fill this gap. Every measurable subset of a measure space can be equipped with the structure of a measure space.
Example 14.10 Let ( M , C , p ) be a measure space and let 52 E C be measurable. Let C’ := ( S E C : S a}and let pa := pl,n. Then (5 2 , C”, p a ) is a measure space. 0 The reader will prove this in Exercise 14-4. Because measure spaces are newly introduced, we must prove their properties “from scratch.” Specifically, we cannot use familiar properties of Lebesgue measure without first proving them for measure spaces. Nonetheless, the next three propositions should be familiar from Lebesgue measure.
Proposition 14.11 Let ( M , C ,F ) be a measure space and let A l , . . . , A N be painvise
(u /
disjoint sets in C. Then p
N
n=l
\
N
A,) =
p(An). n=l
14.1. Measure Spaces
229
u N
Proof. By Proposition 14.4, we have that
+ 1, let An := 0.
An E C. For n 2 N
n=l
Proposition 14.12 Let ( M , C , p ) be a measure space and let A , B Then k ( A ) I F(W.
E
C with A & B.
Proof. By Exercise 14-2b we have B \ A E C . Therefore @.(A)I d A ) L4B \ A ) = P ( A lJ ( B \ A ) ) = FUB).
+
Definition 14.13 Let ( M , C , p ) be a measure space. Then A E C is called a set of measure zero or a null set iff p ( A ) = 0. Aproperty is said to hold almost everywhere in M ipthe subset of M f o r which the property does not hold is of measure zero. Almost everywhere is also abbreviated as a.e. Proposition 14.14 Let ( M , C , p ) be a measure space and f o r all n
u co
A , E C be a null set. Then
E
N let the set
A , also is a null set.
n=l
Proof. By Proposition 14.12, subsets of null sets are null sets also. For n E N,
u
Aj.
Then each B, is a null set and
j=1 1 w
1
u u co
n-1
define B, := An \
m
Bn =
n=l
A,. But
n=l
co
p ( B n ) = 0, which establishes the result.
W
We conclude this section with a result that shows that the measure of the union of a nested sequence of sets is equal to the limit of the nondecreasing sequence of the measures of the sets
Theorem 14.15 Let ( M , C , p ) be a measure space and let
N.Then p
sets in C so that An E A,+lforall n E
u 1
100
be a sequence of
\
An = lim p ( A n ) .
(n=l
*-+m
Proof. Mimic the proof of Theorem 9.12. (Exercise 14-5.)
Exercises 14-1. Let M be a set. Prove that the set of all subsets S C M so that S is countable or M \ S is countable is a u-algebra.
14-2. Let M be a set and let C g P ( M ) be a o-algebra.
n N
(a) Prove that if A 1 , . . . , A N
E
C , then
n=l
(b) Prove that if A , B
E
X,then A \ B
E
C.
An
E
C
230
14. Integration on Measure Spaces
14-3. Counting measure. Let M be a set and let y~ : P ( M ) + [0, M ] be counting measure on M (a) Prove that ~ M ( A=) 0 iff A = 0. (b) Prove that y~ is a measure. 14-4. Measures on subsets. Let ( M , C , p ) be a measure space, let R E Z, let C" := {S E C : S a n d l e t p a := p l x n .
5 Q]
(a) Prove that C" is a a-algebra. (b) Prove that p~ is a measure. 14-5. Prove Theorem 14.15. 14-6. Let ( M , C . p ) be a measure space and let (An}:=, be a sequence of sets in C . Prove the inequality
14-7. Let M be a set. Prove that a subset C 2 P ( M ) is a o-algebra iff 0 E C; if S E C , then S' E C ; and 'x.
if An E C for all n E
N,then
A,, E C . n=l
14-8. L e t ( M , C , p ) b e a m e a s u r e s p a c e a n d l e t A , B E C . P r o v e p ( A ) - p ( B ) = p ( A \ B ) - p ( B \ A ) 14-9. The measure of nested intersections. (a) Let ( M , C , p ) be a measure space and let {An},",l be a sequence of sets in C such that for all n E N we have An 2 An+l and such that for some rn E W we have p(A,) < 00. Prove
(b) Show that the condition p ( A m ) < m for some m E W cannot be dropped Hint. Let ypq be counting measure on W and let An := { i E W : i 1 n } . 14-10. Let ( M , C , p ) be a measure space. (a) Prove that Z, := [ A that contains C.
gM
(b) Prove that for all A
C, and all E , F E Z. with E
E
: ( 3 E ,F E C : E
2 A 2 F , p ( F \ E ) = 0 ) } is a o-algebra g A g F and p ( F \ E )
= 0 we have
P ( E ) = W(F).
(c) F o r a l l A E C , , l e t E . F E C w i t h E g A g F a n d p ( F \ E ) = O a n d d e f i n e p ( A ) : = p ( F ) . Prove that iI : C, + [0, m ] is a measure. (You also need to show that p is well-defined.) (d) Prove that for all B E C we have p ( B ) = p ( E ) .
(e) Prove that the measure space ( M , X p , p) is complete. That is, prove that if N E X, is so that p ( N ) = 0 and S & N , then S E C., The o-algebra Z p is also called the completion of Z. and p is also called the completion of
p
14.2 Outer Measures Although not all subsets of the real numbers are Lebesgue measurable, outer Lebesgue measure is defined for all subsets of R. The idea of an outer measure can be transplanted to an abstract space. This section shows that outer measures produce a measure space similar to how outer Lebesgue measure produced a measure space. In particular, we will obtain a Lebesgue measure on d-dimensional space.
23 1
14.2. Outer Measures
Definition 14.16 Let M be a set. Then p : P ( M ) + [0,co] is called an outer measure 1 3 I . p ( 0 ) = 0, 2. ZfA
G B , then p ( A ) 6 p ( B ) ,
3. Thefunction p is countably additive, that is, for all sequences {An]:=, of sets
(u 00
in M we have p
n=l
00
An) 5 C p ( A , ) . n=l
By Theorem 8.6, outer Lebesgue measure on R is an outer measure. To integrate in Rd (and via Example 14.10 on subsets C2 E Rd)we need to define an outer measure on RBd that is similar to outer Lebesgue measure on R. The definition is very much the same as for the real line, except that instead of open intervals we use d-dimensional boxes. The thus defined outer measure is also called outer Lebesgue measure and it is also denoted with h.
Definition 14.17 Let d 2 1 and for i = 1, . . . , d let ai < bi. Then a set of the form d
d
B := n ( a i , bi) is called an open box in Rd.We define IBJ := n ( b i - a i ) . For a set i=l i=l S Rd we define the outer Lebesgue measure of S to be 30
I
U B j , eachBjisanopenboxinRd , j=1
j=l
where we set h ( S ) = cc ifnone of the series in the set on the right converge.
It would be nice to show here, similar to Proposition 8.5, that the outer Lebesgue measure of a box is exactly the volume of the box. We postpone this result to Theorem 16.81, where we can use compactness for an efficient proof. The argument that outer Lebesgue measure on Rd defines a measure on its measurable sets is exactly the same as for R.Thus the following results and the definition of measurable sets can be seen as a recap of the appropriate parts of Section 9.1.
Theorem 14.18 Outer Lebesgue measure on Rd is an outer measure. Proof. Mimic the proof of Theorem 8.6. (Exercise 14-11.) The motivation and definition of measurable sets in the abstract setting are the same as for Lebesgue measure (see Definition 9.4). We must prevent that the sum of the measures of the pairwise disjoint measurable parts of a set is different from the measure of the whole measurable set.
Definition 14.19 Let M be a set and let p : P(M)+ [0,00]be an outer meaM is called p-measurable iff for all T g M we have that sure. A subset S p ( T )= p ( S n T ) + p (S' n T ) . The set of p-measurable subsets of M is denoted C,.
14. Integration on Measure Spaces
232
As in Definition 9.5 the Lebesgue measure is obtained by restricting outer Lebesgue measure to the set Ch of Lebesgue measurable sets.
Definition 14.20 Let S C Rd be a Lebesgue measurable set. Then the outer Lebesgue measure h ( S ) of S is also called the Lebesgue measure of S. As before, “half’ of the equality for measurability is always satisfied. Moreover, the proof that outer measures induce measure spaces runs along the same lines as the proofs of the corresponding results in Section 9.1.
Proposition 14.21 Let M be a set and let p : P ( M ) -+ [0,CQ] be an outer measure. For all subsets S , T G M , we have p(T) 5 p(S n T ) p (S’ fl T ) .
+
w
Proof. Mimic the proof of Corollary 9.6. (Exercise 14-12.)
Proposition 14.22 Let M be a set and let p : P ( M ) -+ [0,001 be an outer measure. Ifp(S) = 0, then S is p-measurable.
w
Proof. Mimic the proof of Proposition 9.7. (Exercise 14-13.)
Lemma 14.23 Let M be a set and let p : P ( M ) -+ [0,001 be an outer measure. I f A and B are p-measurable, then the intersection A n B is p-measurable.
w
Proof. Mimic the proof of Lemma 9.8. (Exercise 14-14.) Lemma 14.24 Let M be a set, let p : P ( M ) let
-+
[0,001 be an outer measure and
{A,,]zl be a sequence of painvise disjoint p-measurable sets.
u cc
Then
A,, is
Proof. Mimic the proof of Lemma 9.9. (Exercise 14-15.) Theorem 14.25 Let M be a set, let p : P ( M ) -+ [0,001 be an outer measure and let C, be the set of p-measurable sets. Then ( M , C,, p ) is a measure space. Proof. Mimic the proof of Theorem 9.10 to prove that C, is a a-algebra and then mimic the proof of Theorem 9.11 to prove that p is countably additive on C,. (Exercise 14-16.) rn In particular, Theorem 14.25 shows that the triple Rd,Ch,h is a measure space. Lebesgue measure is the standard measure for integration in d-dimensional space. Thus, as we proceed to define measurable and integrable functions, we are constructing a theory that allows us to integrate on Rd and its subsets.
(
Exercises 14-11. ProveTheorem 14.18
)
14.2. Outer Measures
233
14-12. Prove Proposition 14.21 14-13. Prove Proposition 14.22 14-14. Prove Lemma 14.23. 14-15. Prove Lemma 14.24. 14-16. Prove Theorem 14.25. 14-17. Let d 2 1 and for i = 1, , . . , d let the numbers ai < bi be dyadic rational numbers. Then a set of
n d
the form D :=
(ai , b i ) is called a dyadic open box in Rd. Prove that for any set S
g Wd we
i=l
u 30
IDj I : S C
D j , each D j is a dyadic open box in Rd
j=1
1
14-18. Prove that if A , B g R and h ( A ) = 0 (where h is Lebesgue measure on JR), then h ( A x B ) = 0 (where h is Lebesgue measure on R2).
I
14-19. Looking at Riemann integrals from a measure theoretic point-of-view. Let [ a ,b] be an interval. For a set S & [a, b] we define J ( S ) := inf
n
("
IIjI : S &
j=1
u
I j , each I j is an open interval
j=1
and
call it the Jordan content of S. Note that the only difference between the Jordan content and outer Lebesgue measure is that the sums over which the infimum is taken are finite. (a) Prove that for any closed interval [c, d ] g [a, b ] we have J
( [c, dl ) = d - c.
(b) Prove that the Jordan content is nor an outer measure. Hint: Q. (c) Let Ji (S):=sup
In
n
Ibj - a j 1 : S 2 U [ a j ,b j ] , a l 5 bl 5 a2 5 b2 5 . .
j=1
. sa, 5 bn
j=1
1
, and call
it the inner Jordan content of S g [ a ,b]. Prove that for any closed interval [c, d ] E [ a ,b] we have Ji ([c, d ] ) = d - c. (d) Prove that the inner Jordan content is not an outer measure
E W Jordan measurable iff J ( S ) = J i ( S ) . An algebra of sets satisfies the first two properties of a a-algebra, but it is only closed under finite unions rather than infinite of Jordan measurable subsets of [ a ,b] forms an algebra. unions. Prove that the set
( e ) Call a set S
z[[n,b] J : z[~,b] + [O, m) is a finitely additive measure. That is,
(f) Prove that the Jordan content
J(0) = 0 and if (An),N_lis a finite sequence of pairwise disjoint sets
An E
q~.b], then
(g) Prove that a set S g [ a , b] is Jordan measurable iff its indicator function 1s is Riemann
integrable. Note. This exercise shows it is fair to say that the problem with the Riemann integral is that its
associated notion of a content is only finitely additive. 14-20. Let a < b be real numbers and let g : [ a ,b] + W be nondecreasing. For numbers c, d E ( a , b ) so that c < d define (c, d ) := lim g(x) - lirn g ( x ) . define [ a ,d ) := lirn g(x) - g ( a ) ,
Ig
1
1
and define (c, b]
x-td-
Ig := g ( b ) -+,,.. lim , ,
x+c+
1
g(x). Set [ a , b]
I
Ig
I
x+d-
:= g ( b ) - g ( a ) . Let Z be the set of all
Ir
subintervals of [ a , b ] that are either open, closed at a and open on the right, closed at b and open on the left or equal to [a, b]. For any set S c [a, b ] ,we define the outer Lebesgue-Stieltjesmeasure
u 30
lljlg : S &
of S to behg(S) := inf j=1
j=1
I j , each I j is i n Z
I
.
234
14. Integration on Measure Spaces (a) Prove that the outer Lebesgue-Stieltjes measure really is an outer measure (b) Prove that open intervals ( c , d ) C [ a . b] are hg-measurable. Hint. Compare with Proposition 9.13. (c) Prove that for c < d both in [ u , b] we have hg ([c, d ]
)=
lim g ( x ) - lim g ( x ) , where .xed+
x’c-
the one-sided limit is understood to be the value of g if c = a or d = b.
1;
forx
> c.
Prove that Ag(S) :=
(e) Construct a nondecreasing function g : [a, b] + hg ( [ c ,dl ) > g ( d ) - gic).
O; 1:
’
if for any set s 2 [ a , b ] . if c E S, ’3
B and an interval
[c, d ]
g
[ u , b] so that
(f) Prove that the function f : [ u , b] -+ W is Riemann-Stieltjes integrable with respect to g iff hg ( ( x : f is discontinuous at x ) ) = 0.
14.3 Measurable Functions As we did on the real line, we first define the functions for which there is a chance that the integral exists and then we define the integral. Indicator functions (see Definition 5.9) are once more our “rectangles” and simple functions are composed of finitely many disjoint “rectangles.” From here on, algebraic operations on functions will always be understood to be pointwise, that is, for example, the sum f g of two functions f and g with the same domain is defined as ( f g ) ( x ) := f ( x ) g ( x ) for all elements x of the domain.
+
+ +
Definition 14.26 Let ( M , C) be a measurable space. A simple measurable function is a function s : M -+ R such that there are n E N,a l , . . . , a, E R and painvise n
disjoint sets A 1,
. . . , A,
E
C so that s =
ak1.4,. We will also call these functions k=l
simple functions. For measurable functions, we consider the positive and negative parts separately.
Definition 14.27 Let M be a set. For f : M -+ [-m, 001 we dejine the positive part f + ( x ) :=max{f(x),O} andthenegativepart f - ( x ) := -min{f(x),O}. Definition 14.28 Let ( M , C) be a measurable space. The nonnegative function f : M -+ [0,001 is called measurable if there is a sequence of simple functions s,, : M + [0,00) such that f o r all x E M the sequence { s n ( x ) } Z lis nondecreasing and lim s, (x) = f (x).
{s,}zl
n+oc
A function f : M -+ [-a, 001 is called measurable fz f + and f - are both measurable. lf it is necessary to distinguish between several a-algebras, we will also call these functions C -measurable. Once again, measurable functions have many characterizations.
Theorem 14.29 Let ( M , C ) be a measurable space and let f : M + [-XI, function. Then the following are equivalent.
XI] be a
14.4. Integration of Measurable Functions
235
1. f is measurable,
2. For all a
E
R,we have {x E M
3. For all a
E
R, we have {x E M : f (x) 5 a } E C ,
4. For all a
E
R,we have {x E M : f
5. For all a
E
R,we have {x E M : f (x) 1. a } E C.
:f
(x) > a } E C ,
(x) < a }
E
C,
Proof. Mimic the proof of Theorem 9.19 (Exercise 14-21). The characterizations of measurable functions can be used to prove that certain operations preserve measurability.
Theorem 14.30 Let ( M , C ) be a measurable space and let f , g : M -+[-GO, co]be measurable functions. r f f g is defined everywhere, then f g is measurable. If f - g or f . g is defined everywhere, then it is measurable. Finally, f +, f - and I f I are measurable.
+
+
Proof. Mimic the proof of Theorem 9.20. (Exercise 14-22.)
Exercises 14-21. Prove Theorem 14.29 14-22. Prove Theorem 14.30. 14-23. Let ( M , C ) be a measurable space. Prove that a bounded function f : M -+ [0, co) is measurable iff there is a sequence of simple functions sn : M + [0, 00) such that for all x E M the is nonincreasing and lim s n ( x ) = f ( x ) . sequence { s , ( x )
I,"=,
n e w
Hint. Mimic part ''5=+1" of the proof of Theorem 14.29 for nonnegative functions.
14-24. Let ( M , C )be a measurable space and let f , g : M + [-m,
{x {x
E
M : f ( x ) = g(x) }
E
C.
(b) Prove that
E
M : f ( x ) 5 g(x) }
E
C.
(c) Prove that
{x
E M : f ( x ) < g(x)
}
E
C.
(a) Prove that
001 be
measurable functions.
14.4 Integration of Measurable Functions Once measurable functions are defined, the definition of the integral also is similar to that of the Lebesgue integral.
N,let A1, . . . , A ,
Definition 14.31 Let ( M , C ,p,) be a measlire space, let n
E
pairwise disjoint, let a1 , . . . , a,
a k l be ~ ~a simple function.
n
E n
[0,co)and let s =
k= 1
E
C be
14. Integration on Measure Spaces
236
The fact that the integral in Definition 14.31 does not depend on the representation of s can be proved just as for the Lebesgue integral in Exercise 9-20a.
Definition 14.32 Let ( M , C , p ) be a measure space. Let f : M measurable function. We dejine the integral o f f to be f d p := sup I M
[O, co] be a
+.
{ IM
s d p : s is a simple function with 0 5 s 5
f
and we call f integrable iff the supremum isjinite. A function g : M -+ [-m, 001 is called integrable z r g + and g - are both integrable. Its integral is defined to be
IM 1, g d p :=
g+ d p - /M g - dp.
The integral on measure spaces is very versatile. Examples 14.33 and 14.34 show that integration on the real line as well as on d-dimensional spaces are special cases. Examples 14.36 and 14.37 show that absolutely convergent series and absolutely convergent double series are in one-to-one correspondence with integrable functions on the right measure spaces.
Example 14.33 With M = R, C = C;, and p = A, Definition 14.32 gives the 0 Lebesgue integral on the real line. Example 14.34 With M = Rd,C = C;, and p = A, Definition 14.32 gives the Lebesgue integral in d-dimensional space. We will investigate in Section 14.8 how the 0 Lebesgue integrals in various dimensions are related to each other. For the mentioned examples on series, we first need to establish that a function is integrable iff its absolute value is integrable. As was (by now) to be expected, the properties of the abstract integral are proved in exactly the same way as for the Lebesgue integral.
Theorem 14.35 Let ( M , C, p ) be a measure space and let f , g : M measurable functions. 1. I f 0 5 f 5 g a.e. and g is integrable, then so is f and
+.
[-co,001 be
lM IM f dp 5
g dp.
2. f is integrable z f f 1 f I is integrable and in that case the triangular inequality I
n
3. I f f 2 0 , t h e n SM
f d p = 0 i f f f =Oa.e.
Proof. Mimic the proof of Theorem 9.23. (Exercise 14-25.)
237
14.4. Integration of Measurable Functions
Example 14.36 With M = 0;:
N,a series
aj
N,C
= P( N ) and p = yw being counting measure on
converges absolutely iff the function f : N + [--00,
001 defined by
j=1
f(j)=
aj
is integrable over the measure space (N, P ( N ) , yw). Moreover, for every
integrable function f on this measure space we have
jN dyw f
co
=
f(j). j=1
Example 14.37 Let (a(i,j ~ } $ = be ~ a doubly indexed countable family of numbers.
7
occo
We say the double series
a(i,j) converges
absolutely iff the double series
i = l j=1 moci
1 converges.
la(j,j)
With M = N x N, C = P ( N x N)and p = mxwbeing counting measure on N x N, 000;:
a double series
7
a(i,j)converges
absolutely iff the function f (i, j ) := a ( i , j ) is
i = l j=1
integrable on the measure space
(N x N, P ( N x N), y w x ~ ) . For every integrable
function f on this measure space, we have
S,
cocc
.t-dywxrv=~~f(i,.j).
XW
0
i = l j=1
As for Lebesgue measure, we consider null sets to be insignificant. Therefore with the same motivation as given for Definition 9.24 we define the following.
Definition 14.38 Let ( M , C , p ) be a measure space. I f . f : M -+ [--00, -001 is defined iff (') is defined? is integrable. a.e., we call it integrable z r g ( x ) := i f f (x) i s not defined,
{ 6(x)'
As long as a function is constructed from other integrable functions, like the sum in Theorem 14.39, measurability of the set where the function is not defined is not an issue. However, even if this was a problem and the function is undefined on a subset of a null set, we could simply switch to the completion of the measure (see Exercise 14-10) and note that subsets of null sets are also insignificant. Overall, with the same conventions as for the Lebesgue integral we can prove that the integral over a measure space has the right linearity properties.
Theorem 14.39 Let ( M , C , p ) be a measure space and let f ,g : M + integrablefunctions. Then the following hold. 1. For all a E
R,the scalar multiple af is integrable and
IM
2. The sum f
+ g is integrable and
f dp
f
+g d p =
[--00.
-003
IM 1, + lM af d p = a
g dp.
be
f dp.
14. Integration on Measure Spaces
23 8
Proof. Mimic the proof of Theorem 9.25. (Exercises 14-26a and 14-26b.)
Exercises 14-25. Prove Theorem 14.35. That is, let ( M , C,p ) be a measure space and let f ,g be measurable functions. (a) Prove that if 0 5 f 5 g a.e. and g is integrable, then so is f and (b) Prove that f is integrable iff
(c) Prove that if f
> 0, then
jM
I f 1 is integrable and that in this case
f dp 5
1
/
g dp.
M
[M f dw
5
~
IM
1f l
dp.
J, f d p = 0 iff f = 0 a.e..
14-26. Let ( M , C . p ) be a measure space and let f , g : M + [-m, 031 be integrable functions. Prove that if
ci
E
W,then the scalar multiple af is integrable and /M
Prove that the sum f
+ g is integrable and
IM + f
g dl* =
cif
dp =a
IM + IM 1 f dp
lM
f dw.
g dp
Note. Exercise 14-33 gives a more effective proof than mimicking the proof of Theorem 9.25. Prove that f - g is integrable and /M f - g d p = /M f d p -
g dp.
M
Prove that max(f, g ] (defined pointwise) is integrable. Prove that min[f, g ] (defined pointwise) is integrable. Give an example that shows that the product o f f and g need not be integrable.
14-27. Markov’s inequality. Let ( M , o,g ) be a measure space and let f : M + [O. 1 Prove that for all c z 0 we have p ( { x E M : f ( x ) > c ] ) 5 f dp. C
1
30)
be integrable.
M
14.5 Monotone and Dominated Convergence Sequences have been a standard tool throughout our investigation of single variable functions. Sequences play an equally important role in more abstract settings. In Exercise 11-13, we have seen that pointwise convergence of a bounded, monotone sequence of Riemann integrable functions need not produce a Riemann integrable limit function. The two fundamental results about pointwise convergence in measure theory are the Monotone Convergence Theorem, which says that the pointwise limit of a nondecreasing sequence of nonnegative integrable functions is integrable if the limit of the integrals is finite (see Theorem 14.41), and the Dominated Convergence Theorem, which says that the pointwise limit of a sequence of integrable functions is integrable provided all functions are below one integrable function that dominates them all (see Theorem 14.43). Hence, when it comes to pointwise convergence of functions, the Lebesgue-type integral has more favorable properties than the Riemann integral. Because every subset A E C of a measure space ( M , C , /A) can be turned into a measure space, and because by Theorem 14.35 for any integrable function g, the function g l A is also integrable, we can define integrals over subsets.
Definition 14.40 Let ( M , C , p ) be a measure space, let A E C and let the function ,f : A4 + [-m, 001 be integrable. Then we define the integral o f f over the subset A
239
14.5. Monotone and Dominated Convergence as
I,
g d p :=
lb
SM
glA d p . For Lebesgue integrals over intervals, we will also write
f ( x ) d h ( x ) :=
Lb
f d h :=
la,b, f dh.
Because the integral is defined as a supremum of integrals of simple functions, it should not be too surprising that the integral is well-behaved with respect to monotone sequences of functions.
Theorem 14.41 Monotone Convergence Theorem. Let ( M , C , p ) be a measure bela sequence of nonnegative measurable functions so that space and let { f f l } z { f f l ( x ) } z l is nondecreasing for almost all x E M and lim f n ( x ) exists for almost all x
n+m
E
M . Let f : M + [0,00] be a measurablefunction so that f (x) = fl+m lim f f l ( x )
IM P
a.e. Then
P
f d p = lirn
fn
f l J M
dl*.
Proof. First assume that for all x limit lim f n ( x ) exists.
E
M we have 0 5 f 1 (x) 5 f 2 ( x ) 5 . . . and the
fl+X
reversed inequality, let s =
c
ak1.4, be a simple function such that 0 5 s 5 f and let
k=l
t E ( 0 , 1). For n E
all n
E
N,define E,
cc
:= ( x
N we have the containment E ,
E
M : f i l ( x ) 2 t s ( x ) } .Then M =
fl=l
En+l, and
m
m
k=l
k=l
Let E > 0. Then there is an N E
u
N so that
E,, for
240
14. Integration on Measure Spaces
Because
E
SM SM f5y- 1,
was arbitrary we obtain
t s d p i lirn
fn
n+w
d p , and because the
number t E (0, 1) was arbitrary, we can let t approach 1 and obtain sdp =
s d p 5& :%
t
S,
dp.
fn
Because s 5 f was an arbitrary simple function, we conclude that SM
{
f dF = sup s,/
d p : s simple, 0 5 s 5 f } 5 lim n+oo
,/
fn dp.
Now assume that 0 i fi(x) 5 f 2 ( x ) I . and n+cc lirn f n ( x ) = f (x) for almost all x E M . Let N E C be a null set so that 0 5 f l ( x ) 5 f 2 ( x ) 5 . . . and lirn f n ( X ) = f (x) for all x E M \ N. Changing a function on a null set does not n+cc change its integral. Therefore we can replace f with f ~ M \ and N each f n with fnlM\N and apply what we have proved above to obtain the equation for the integrals. rn ‘
1
We need the next lemma for the proof of the Dominated Convergence Theorem.
{fn)rE1
Lemma 14.42 Fatou’s Lemma. Let ( M , C , p ) be a measure space and let be a sequence of nonnegative measurable functions. Then lirn inf f n (defined pointwise) n+w
is measurable and
f n d p 5 ln+cc iminfh fn dp.
Proof. Because the limit inferior is liminf f n = lim inf{f j : j 3 n ] , where the n+m n+co infimum and the limits are taken pointwise, we first consider the sequence of functions p n := inf{f j : j 2 n ) .
u {x 00
Let n
E
N and a
E
R. Then {x E M : p n ( x ) ia } =
E
M : fj(x) < a}.
j =n
Because the union on the right side is measurable and a was arbitrary we conclude that p n is measurable. Moreover, clearly we have pn 5 pn+l. By Exercise 14-28a, this means that liminf f n = lim p n is measurable and by n+oo
n+oo
the Monotone Convergence Theorem numbers n
E
N we have p n
p n d y . Now for all
I f n , which by Lemma 10.5 means
liminf f n d p = lim n+w
1
p n d p 5 liminfIM fn d p . n+m
The inequality in Fatou’s Lemma can be strict (Exercise 14-29). Finally, as long as all functions’ absolute values are bounded (“dominated”) by one integrable function, pointwise limits preserve integrability and the limit can be moved out of the integral.
Theorem 14.43 Dominated Convergence Theorem. Let ( M , C ,p ) be a measure be a sequence of measurable functions and let f be a measurable space, let ( fn]rG1 functionsuch that f (x) = lirn f n (x)for almost all x. Moreovel; let g be an integrable n+oo
24 1
14.5. Monotone and Dominated Convergence function such that for all n
f is integrable and
SM
E
N and almost all x
f dp=
,hI/,
fn
If
I
1
I
E M we have f n (x) 5 g ( x ) . Then
dp.
I
Proof. Because ( x ) l = lirn f n ( x ) 5 g(x) for almost all x E M , by part 1 of n+cc Theorem 14.35 I f 1 is integrable and then by part 2 of Theorem 14.35 f is integrable. Because changing a function on a null set does not change the integral, we can assume that l f n ( x ) (5 g(x) and (x)l 5 g(x) for all x E M . Now for all n E N we have f n (x) g(x) 2 0 and thus by Fatou’s Lemma we obtain
If
+
lim lirninf
F
n-cc
which means that functions (- f n ) sition 10.4)
IM
jMf
dp 5
fa
,/
fn
hnkf /M
+ g 2 0 gives
+ g d p = /M
-f d p
liminf(f, n-+m
+ g d p = liminf n-cc /M
+ g) d p
f n d p -t
/Mgdir’
f n d p . The same argument applied to the
5 liminf
-f n d p , that is (recall Propo/M
f n d p . By Proposition 10.3 and Theorem 10.6, we
f dp 2
r
The hypothesis that all functions f n in the Dominated Convergence Theorem are below an integrable function g cannot be dropped. For example, on (0, 1) equipped l o o
converges point{ [M);),=, wise to zero, but all integrals are equal to ln(2). Moreover, Exercise 14-28 shows that
with Lebesgue measure, the sequence of functions 1
the demand that the pointwise a.e. limit is measurable cannot be dropped in general, but that it can be dropped for Lebesgue measure.
Exercises 14-28. Pointwise almost everywhere limits of measurable functions. Let ( M , C , p ) be a measure space and be a sequence of measurable functions. let (fn)r=u=l Prove that if f ( x ) = lim f n ( x ) for all x E M , then f is measurable. n-tcc
Give an example that shows that the pointwise a.e. limit of a sequence of measurable functions need not be measurable. Hint. There is an example with M = (0, 11, C = (0, M )and p = 0. Prove that if ( M , C , p ) is a complete measure space, then f ( x ) = lirn f i l ( x ) a x . implies n-cc
that f is measurable. Prove that (&id, Xi,h
) is a complete measure space.
14-29. Give an example that the inequality in Fatou’s Lemma can be strict. Then explain why this is not a counterexample to the Dominated Convergence Theorem.
14-30. Let ( M , C , p ) be a measure space and let g : M + [O, pg : C + [O,
cc)defined by p g ( A ) :=
031
be integrable. Prove that the function
14. Integration on Measure Spaces
242
14-31. Let ( M , Z. p ) be a measure space and let ( f n } , X , l be a sequence of integrable functions that converges uniformly to a measurable function f. (a) Prove that if p ( M ) < co,then f is integrable and
sM
f dw
= LlmE
sM
f n dpL.
(b) Give an example that shows this result need not hold for infinite measure spaces. be a sequence of integrable functions that con14-32. Let ( M , Z, p ) be a measure space and let {fn]r=l verges pointwise to a measurable function f and for which there is a B E JR so that for all x E M we have f n ( x ) 5 B . (a) Prove that if p ( M ) < co,then f is integrable and
f d p = lim
n+x M
f n dp.
(b) Give an example that shows this result need not hold for infinite measure spaces
sM sM
14-33. Let f,g : M + [-co,301 be integrable functions. Give a more effective proof that f integrable and
IM + f
g dp =
f dp +
+ g is
g d p than was given in Theorem 9.23.
Hint. Use the Monotone Convergence Theorem and two nondecreasing sequences of simple functions that converge to I f 1 and / g (respectively to prove that I f 1 + / g /is integrable. Use a similar idea to prove the equation for the integrals. Do not use the Dominated Convergence Theorem (why?). 14-34. Explain how the hypotheses of the Dominated Convergence Theorem prevent the occurrence of examples as in Exercise 11- 17 or as mentioned at the end of this section. 14-35. Prove that for every Lebesgue integrable function f : SS -+ JR there is a sequence of Riemann integrable functions rn : R + R such that lim
n+w
s
w if
- rn
I d h = 0.
14.6 Convergence in Mean, in Measure, and Almost Everywhere Theorem 15.50 will show that the “distance” between two functions can be measured with the integral of the absolute value of the difference. Because convergence with respect to this distance is fundamental in analysis, we investigate this notion of convergence and some of its consequences here.
Definition 14.44 Let ( M , C ,p ) be a measure space, let [ fn]F=l be a sequence of measurable functions and let f : M + [-w, w] be a measurable function. If
1 fn
-
f 1 d p = 0 then we say that ( fn]zl converges in mean to f
I
Convergence in mean is near the heart of the definition of integration.
Theorem 14.45 Let ( M , C ,p ) be a measure space and let f : A4 + [-w, w] be an of simplefunctions that converges integrablefunction. Then there is a sequence (sn]El in mean to .f. Proof. By definition of the integral, for every n tions 0 5 ;s
5 f - and 0 5 :s
5 f + so that
E
N there are simple func1 n
14.6. Convergence in Mean, in Measure, and Almost Everywhere
-1 + -1= - 2,
<
which means that {s.},"=~
243
n
n
n
converges in mean to f .
Unfortunately, convergence in mean does not even imply pointwise a.e. convergence, so the visualization of convergence in mean is a bit complicated.
Example 14.46 On the interval [0, 11, for n E N and k E {0, . . . , 2n - 1) define the function f p + k := 1F k ~k + l, ~Then ] . the sequence { fm}F==l converges in mean to 0, but
0
it does not converge at any point in [0, 11.
We can say that convergence in mean implies that the measure of any set on which the difference Ifn - f I is greater than a given number must go to zero.
Proposition 14.47 Let ( M , C,p ) be a measure space. I f the sequence { fn}r=l of measurable functions converges in mean to the measurable function f . then for all E > 0 we have lim p ({x E M : I f n ( x ) - f ( x ) l > E } ) = 0. n+cc
Proof. Let E > 0. Then
w
and the latter sequence goes to zero.
The condition derived in Proposition 14.47 is called convergence in measure. Convergence in measure implies at least that a subsequence converges a.e.
f,,}rz1
Definition 14.48 Let ( M , C ,p ) be a measure space. The sequence { of measurable functions converges in measure to the measurablefunction f ifffor every E > 0 we have lim p ({x E M : f n ( x ) - f ( x ) l > E } ) = 0. n+cc
1
fn}ry1
Proposition 14.49 Let ( M , C , p ) be a measure space. I f the sequence { of measurable functions converges in measure to the measurable function f , then it has a subsequence that converges a.e. to f .
14. Integration on Measure Spaces
244
Proof. Define
as follows. Let no := 1 and for k L 1 let
(nk}.&
x E M : I f n k(x) - f ( x )I > that
(fnkJE1
s o t h a t p ({x be so that I
Suppose for a contradiction
does not converge a.e. to f . Then there are an m E E M : (VK
E
w l
m and
W : 3k 3 K :
nk > nk-1
Ifnk(x)
- f(x)l >
N and an E
> 0
m
< E . Then
k=l
but the measure of the latter set is less than
E,
a contradiction.
Exercises 14-36. Let ( M , C , p ) be a measure space and let g : M + [O, m] be integrable
(b) Prove that for every E > 0 there is a S > 0 so that if p ( A ) i8, then
Hint. First prove the result for bounded g.
L
gdp
iE .
(c) Prove that with M = [a,b ] , p being Lebesgue measure h on [ u , b] and C being the set Cj, of Lebesgue measurable subsets of [ u , b ] ,the function f ( x ) :=
continuous. 14-37. Pointwise convergence and convergence in measure. (a) Give an example that shows that, in general, pointwise convergence does not imply convergence in measure. (b) Prove that if ( M . C , p ) is a measure space with p ( M ) < 00, [fn]:={ is a sequence of measurable functions that converges ax. to the real valued measurable function f,then converges in measure to f.
[fn]F=l
(c) Prove Egoroff's Theorem, which states that if ( M , C,p ) is a measure space with p ( M ) iCQ, is a sequence of measurable functions that converges a x . to the real valued measurable function f,then for each E > 0 there is a B E C so that p ( B ' ) iE and converges uniformly to f on B .
[fn}r=l
14-38. First results on derivatives of integrable functions. (a) Let f : [a,b] -+ W be a Lebesgue measurable function that is differentiable a t . . Prove that f' is Lebesgue measurable. You may use the result of Exercise 14-28c. (b) Let f : [ a , b] + JR be a nondecreasing function. Prove that f' (which exists a x . by Exercise 10-7) is Lebesgue integrable on [a,b ] and
f ' ( x ) dh 5 f ( b )- f ( a ) .
14.7. Product a -Algebras
Him. Apply Fatou’s Lemma to fn (x) :=
245 f (x
+ b) 1
-
f(x)
(where we set f ( b + e ) := f ( b )
for all 6 > 0) and use Exercise 7-23, Theorem 9:26 and an adaptation of the proof of the Derivative Form of the Fundamental Theorem of Calculus for Riemann integrals.
14.7 Product a-Algebras One motivation for introducing measure theory was the desire to define an integral on multidimensional spaces. Integration with respect to d-dimensional Lebesgue measure provides such an integral as an abstract entity. Computationally, it is standard practice to compute higher dimensional integrals as iterated lower dimensional integrals. Sections 14.7 and 14.8 show that this is also possible in measure theory. As a consequence, we not only progress towards showing that d-dimensional integrals can be computed as iterated integrals, but we also are able to establish some results on double series (see Exercise 14-50). The relevant constructions actually start with the construction of the product space of two measure spaces. For this product space, we need a a-algebra that contains certain sets. This section will provide the a-algebra on the product space and Section 14.8 will provide the product measure. As before, to safeguard against nonmeasurable sets, we must make sure that our a-algebras do not get too large. The smallest a-algebra that contains certain sets is generated as follows.
Definition 14.50 Let M be a set and let U P ( M ) . Then the a-algebra generated by U is defined to be the intersection of all a-algebras that contain U. Proposition 14.51 Let M be a set and let U by U is a a -algebra.
P(M).Then the a-algebra generated
Proof. Exercise 14-39. Exercise 14-41a shows that the a-algebra of Lebesgue measurable sets is generated by a family of fairly simple sets. To prove the equality of two measures on the a algebra generated by the set U we want to concentrate on the sets in U . Unfortunately, because the definition of a measure involves unions of pairwise disjoint sets, this is not as straightforward as it might seem. Instead of being equal on a-algebras, measures usually agree on Dynkin systems.
14. Integration on Measure Spaces
246 Definition 14.52 Let M be a set and let V system iff
P ( M ) . Then D is called a Dynkin
_C
1. M E D ,
2. I f A , B E D a n d A 2 B, then A \ B
E
D, and
u cc
3. I f { A n } F I is a sequence in D with A ,
An+l for all n E
N,then
A , E D.
n=l
Definition 14.53 Let ( M , C , p ) be a measure space. Then the measure p is called finite z f f p ( M ) < co. Proposition 14.54 Let ( M , C) be a measurable space and let p and v befinite measures on M so that p ( M ) = v ( M ) . Then { S E C : p(S) = v(S)} is a Dynkin system. Proof. Exercise 14-43a (for part 3 use Theorem 14.15). As with a-algebras, the intersection of all Dynkin systems that contain a given set of sets U is again a Dynkin system, called the Dynkin system generated by U (see Exercise 14-44). The connection between a-algebras and Dynkin systems is made via n-systems.
Definition 14.55 Let M be a set and let U G P(M).Then U is called a n-system for all A , B E U the intersection satisfies A f?B E U.
zx
Theorem 14.56 Monotone Class Theorem or Dynkin’s Lemma. Let M be a set and let U be a n-system that contains M . Then the a-algebra generated by U equals the Dynkin system generated by U. Proof. Let C be the a-algebra generated by U and let D be the Dynkin system generated by U.Because every a-algebra also is a Dynkin system, we infer D C. For the reversed inclusion, first consider D1 := { A E D : A n U E D for all U E U}. BecauseU contains M we infer M E D1.If A , B E D1 and A 2 B , then for all U E U we have ( A \ B ) f l U = ( A n U ) \ ( B n U ) E V (see Exercise 7-3e), which means A \ B E D l . Moreover, if is a sequence of sets in D1 with A , G A,+l for all n E
N,then for all U
u cc
means that
n=l
E
U we obtain
(6
An) nU =
n=l
co
u
(A,
n U ) E V ,which
n=l
A, E D1. Hence, D1 is a Dynkin system. Because U is a n-system, we
infer U D1 and therefore 2) G D1, which implies that V = D1. Now define D2 := { A E D : A n D E D for all D E V}.Because D1 = D we infer U 2 232. Similar to the above we can prove that 272 is a Dynkin system, which means that D = V2.We conclude that if A , B E D , then A Ti B E D. Now we can prove that V is a a-algebra. First, note that because M E D , for all sets S E D the complement S’ = M \ S is in D. In particular, we trivially obtain 0 = M’ E D. Now let { B n } E lbe a sequence of sets in D. For n E N,define the
247
14.7. Product a -Algebras
/ / / /
’//
X
X
X
Figure 30: The “horizontal sections” S J of a set S C X x Y are obtained by intersecting the set with “horizontal lines” X x { y } . The “vertical sections” S, are obtained by intersecting the set with “vertical lines” {x} x Y .
u u u n
set A , :=
Bi = M \
E 2). Then
A , 2 A,+1 for all n E
N and so
i=l
oc
x
B, =
A,
E
D and 2)is a a-algebra. This proves C
D , hence C = D.
,=l
17=1
The Monotone Class Theorem now allows us to prove that under certain circumstances, two measures on the same a-algebra must be equal.
Theorem 14.57 Let M be a set, let U C P ( M ) be a n-system, let C be the a-algebra generated by U and let p , v C + [0,001 be measures with kiu = vlu.If there is a in U so that all p ( A , ) = v(A,) arejnite, A , An+l for all n E N sequence
u 3c,
and
A,z = M , then p = v.
Proof. First prove that each { A E C : p ( A n A , ) = u ( A n A , ) } is a Dynkin system. Then apply Theorem 14.56 to show p ( A f’ A , ) = u ( A n A , ) for all A E C and n E N. cc
Finally, show p = v by using A0 := 0 and p ( A ) =
K ( A f l ( A , \ A,-i))
and
n=l C-C
u ( A f l ( A , \ A,-I)) for all A
v(A)=
E C.
(Exercise 14-45.)
n= 1
To define a product of two measurable spaces we now simply form all the products of sets in the respective a-algebras and consider the o-algebra generated by these products. We also define the intersections of sets with “horizontal lines” or “vertical lines” and the intersections of graphs of functions with “vertical planes.” These intersections will allow us to reduce integrals on the product of two measure spaces to iterated inte-
248
14. Integration on Measure Spaces
“I
“t
Figure 31: The “y-sections” f Y of a function f : X x Y + [--00, m] are obtained by intersecting the graph of the function with “vertical planes X x {y} x [--00, 001 in the X-direction at y.” The “x-sections” f x are obtained by intersecting the graph with “vertical planes (x} x Y x [-m, 001 in the Y-direction at x.” grals. To emphasize the visualization as x- and y-coordinates, for the remainder of this chapter, the first factor space will be X and the second factor space will be Y.
Definition 14.58 Let ( X , C ) and ( Y , r) be measurable spaces. A subset S of X x Y is called a rectangle with measurable sides iff there are subsets A E C and B E r such that S = A x B. The a-algebra generated by the set of all rectangles with measurable sides is called the product a-algebra C x r of C and r. Let x E X and y E Y . For all sets S E C x r in the product a-algebra we dejine the sets S, := n y [ { s E S : n x ( s ) = x}] and SY := n x [ { s E S : n y ( s ) = y } ] (see Figure 30). For all C x r-measurablefunctions, we dejine f x (y) := f (x,y) and f’(x) := f (x, y) (see Figure 31). The standard approach to prove results about product a-algebras is to establish the result for rectangles with measurable sides and then to show that the sets for which the result holds form a a-algebra. We start by making sure that the sections S,, SY, f, and f Y of measurable sets and functions are measurable.
Proposition 14.59 Let ( X , C ) and ( Y , r) be measurable spaces and let x E X and y E Y. Then for all sets S E C x the set S, is r-measurable and the set S’ is C-measurable. Moreover, for all C x r-measurable functions, the function f x is r-measurable and the function f Y is C-measurable. Proof. First, we prove that for each y E Y and S E C x r we have SY E C. To do this let y E Y and let BY be the set of all S E C x r so that SY E ’c. For all A E C and B E r the rectangle with measurable sides A x B is in BY, because if y E B we
14.7. Product B -Algebras
249
have ( A x B)Y = A E C and if y # B we have ( A x B)Y = 0 E C. To prove that BJ = C x r we need to show that BY is a a-algebra, because then it must contain C x r (see Exercise 14-42) and must hence be equal to it. To see this, first note that 0 = 0 x 0 E BY. Now if S E BY, then (S')'
so S'
u
E
s' : ny(s)= y}]
=
nx [{s
=
x \ nx[{s E S : ny(s)= y}]
BY. Finally, if
E
= nx[{s #
s : ny(s)= y}]
= (S')' E C ,
{Sfl}Zl is a sequence in BY, then
33
so
S, E BY. Thus BY is a a-algebra that contains the rectangles with measurable
n=l
sides and is contained in C x r, which means that BY = C x r. Hence, for all y E Y and all S E C x r we have that SY E C. Similarly, we prove that for all x E X and all S E C x l- we have that Sx E r. Now let f : X x Y + [--00, -001 be a C x r-measurable function. To prove that for every y E Y , the function f Y is C-measurable, let y E Y and a E R.Then (f')-"(a,CO]]
= {x E x : f ( x , y ) > a } =nx[{z E
xxY
: f(z)>a,nr(z)=y}]
Similarly, we prove that for every x E X the function fx is r-measurable. We conclude this section by showing that the constructions presented here give reasonable results for Lebesgue measure.
+
Proposition 14.60 Let m , n , d E N be such that m n = d and let h,, h, and hd denote Lebesgue measure on Rm,Rn and Rd,respectively. Then Chd 2 Xi, x Ch, and f o r all A E and B E Ch, we have hd(A x B ) 5 h,(A)k,(B). Proof. By Exercise 14-41b, Ch, x Xi,,is generated by the sets of the form A x B , where each A and B is either an open box or a null set. The proof that products of open boxes A x B (which are of course open boxes themselves) are hd-measurable is similar to the proof of Proposition 9.13 (see Exercise 14-40). Now let A E Ch,, let B E Ch, and let E > 0. Find sequences of open cc
{Zj)Eland { K l ] E l so that the containments A E u Zj, B C uKi and the c4
boxes
j=l
1=1
250
14. Integration on Measure Spaces
c x
inequality
00
II j I
j=1
I K1 I
< h,
(A)hn(B)+ E hold. Then
1=1
j,l=l
J=l
1=1
and for all A E Chm and B E Chn we obtain hd(A x B) I h,(A)h,(B). In particular, this means that for A E Ch,, and B E Ch, such that one of A , B is a null set we have that hd(A x B) = 0, which means A x B is Ad-measurable. Together with the Ad-measurability of products of open boxes, this means Chd 2 Chm x Ch,, because Chm x C,in is generated by products of open boxes or null sets in Rm with open boxes or null sets in Rn. Proposition 14.60 has two apparent shortcomings. First, the a-algebras are not shown to be equal to each other. Exercise 14-47 shows that the containment is indeed strict, so this part of the result cannot be improved. Second, we did not show that hd(A x B) and h,(A)h,(B) are equal, even though they should be equal. Recall that in Proposition 8.5 we needed the Heine-Bore1 Theorem to prove that the Lebesgue measure of an interval is its length. To prove that hd(A x B) and h,(A)hn(B) are indeed equal, we need a higher dimensional version of the Heine-Bore1 Theorem. This version will be provided in Theorems 16.72 and 16.80. Thus we delay the proof that hd(A x B) = A,(A)h,(B) to Theorem 16.81. To keep notation simple, unless the distinction of different dimensions is necessary, we will denote Lebesgue measure in all dimensions by h.
Exercises 14-39. Prove Proposition 14.51 14-40. Prove that every d-dimensional open box is hd-meaSUrabk. 14-41. Generating the Lebesgue measurable sets. For d E boxes and all d-dimensional null sets.
N let Gd be the set of all d-dimensional open
(a) Prove that C A ~the, 0-algebra of Lebesgue measurable sets in E d , is generated by Gd. Hint. Gd g C A by ~ Exercise 14-40. Prove that every S E X i d with h ( S ) < m differs by a null set from a countable intersection of countable unions of open boxes. Then use 0 -finiteness. (b) Prove that form, n E
N the 0-algebra generated by ( A x B : A
E G,, B E G,] is Chm x C h H .
(c) P r o v e t h a t i f m + n = d a n d A E C ~ ~ , t h e n t h e r e i s a BE Ch, X C A , withhd(A\BUB\A) = 0.
+
(d) Prove that if m n = d and the function f : Wd + [-m, m] is Zhd-measurable, then there is a Z A , x~ Cj,, -measurable function g with j = g a.e. 14-42. Let M be a set and let U be a set of subsets of M . Prove that the a-algebra generated by U is contained in all a-algebras that contain U . 14-43. On the equality of measures. (a) Prove Proposition 14.54.
(b) Prove that the 0-algebra generated by all the singleton subsets of C := (Cg W : C or W \ C is countable 1.
W is the set
14.8. Product Measures and Fubini's Theorem
25 1
(c) Construct two finite measures on C that agree on all singletons, but which are not equal 1 Hint.Set F ( [k) ) := - fork E N and set it equal to zero for all other singletons. 2k
(d) Construct a measurable space ( X , C) and two finite measures /1. and u on C so that the equality l ( X ) = u(X) holds, but { S E C : w ( S ) = u(S) } is not a a-algebra. Hint There is a finite example. 14-44. Prove that if ('Dj]iclis a family of Dynkin systems on the set M , then
n'Di
is a Dynkin system.
iel
14-45. Prove Theorem 14.57. 14-46. Prove that if is a measure defined on the a-algebra 8 generated by the finite open intervals such that F ( ( a , b ) ) = b - a for all finite open intervals, then p must be equal to the restriction of Lebesgue measure to B. 14-47. The containment of the a-algebras in Theorem 14.60 is strict. To see this, prove that for m < d every subset of W m (considered as a the subset Rm x [O)d-m of Rd) can be a section of a Lebesgue measurable set in Rd.
14.8 Product Measures and Fubini's Theorem We are now ready to define a measure on the product a-algebra. Because Theorems 14.56 and 14.57 are needed to construct an unambiguous product measure in Theorem 14.62, we introduce the notion of a-finiteness. Definition 14.61 A measure space ( M , C , k ) is called a-finite
u
iff
there is a sequence
a3
of subsets A; & M ofjnite measure so that M =
A ; . We will also sometimes call
j=1
the measure a -3nite.
Clearly, Lebesgue measure is a-finite, so we will be able to use the results from this section for d-dimensional integration. Theorem 14.62 Let (X, C , k ) and ( Y , r, u ) be a-jnite measure spaces. Then there is a unique measure k x u on C x r so that with the convention 0.00 = 0 for all A E C and B E we have ( k x u ) ( A x B ) = k ( A ) u ( B ) . Proof. The natural idea for the product measure of a set S E C x r is to integrate the u-measures of the sections S, over X. To do this we first need to prove that for all S E C x r the function x H u (S,) is C-measurable. Let X be the set of sets S E C x r so that the function x H u (S,) is C-measurable. We need to prove that X = C x r. For all A E C and B E r we have that u ( ( A x B),) equals u ( B ) when x E A and 0 when x 9 A . Therefore, the function x H u ( ( A x B),) = U(B)lA(X) is C-measurable for all A x B E C x r. To prove X is a a-algebra, first note that the rectangles with measurable sides form a rr-system (see Exercise 7-7) and X x Y E X. To be able to apply Theorem 14.56, we will now prove 'H is a Dynkin system when u is finite. If S,F E 'H and S C F , then for F \ S we have u ( ( F \ S),) = u (F, \ S,) = u(F,) - u(S,), which implies that F \ S E 'H. Now let be a sequence in X with A, 5 A,,+1 for all n E N.
252
Then u
(ifi
An),) = u
n=l
14. Integration on Measure Spaces
(c
( A n ) , ) = ,l&l u ( ( A , ) , ) by Theorem 14.15, so by
Exercise 14-28a li is closed under the formation of increasing unions and thus li is a Dynkin system. Therefore, by Theorem 14.56 7-l = C x r. Hence, if u is finite, for all S E C x r the function x H u(S,) is C-measurable. To prove the result for a-finite measure spaces ( Y , u , r), let { F n ) r = i be a se-
u CQ
quence of pairwise disjoint sets in u, (S) := u ( Fa
n S).
r
with u(F,) <
00
so that
F, = Y . Define
n=l
Then each u, is a finite measure and so each function u, (S,) is co
C-measurable. But then u(S,) = which proves that the functions x
u,(S,) is also C-measurable (Exercise 14-28a), n=l H
u(S,) are C-measurable.
Now we prove that ( p x u ) ( S ) := Clearly, (w x u)(0) =
s,
u (0,) d p =
s, s,
u (S,) d p defines a measure on C x
0 d p = 0. Now let [ S n ) ~bel a sequence
of pairwise disjoint sets in C x r. Then for all x sequence of pairwise disjoint sets in r. Therefore
Thus ( p x u ) defines a measure on C x have ( p x u ) ( A x B ) =
s,
r.
E
X we have that { (Sn),]zl is a
r. Moreover, for all A
u ( ( A x B),) d k =
s,
E C and B E
r we
u ( B ) 1~ dw = p ( A ) u ( B ) ,where
the overall result is zero if one of A or B has measure zero. Because ,LL and u are both a-finite, any measure with the above property must be a-finite. Thus by Theorem 14.57 p x u is the unique measure on C x r so that ( p x u ) ( A x B ) = k ( A ) u ( B )for a all A E C and B E r.
Definition 14.63 The measure of p and u.
x u in Theorem 14.62 is called the product measure
Theorem 14.62 not only proves the existence of the product measure, its proof also provides a representation of p x u. Because the product measure is unique and we could have switched the roles of the two factors, there are two representations. These two representations are needed so that we can ultimately switch the order of integration.
253
14.8. Product Measures and Fubini’s Theorem
Corollary 14.64 Let ( X , C, p), ( Y , r, v ) be a-Jinite measure spaces and let S E C x r. Then a ( x ) := v (S,) is C-measurable, b ( y ) := p (SY) is r-measurable and the equality ( p x v ) ( S ) =
s,
v (S,) d p =
s,
p (S’) d v holds.
Proof. Similar to the proof of Theorem 14.62 we can prove that for all S E C x the function y
H
p
(SY)is r-measurable and
(p x
v)(S) =
p
r
(SY)d v .
We are now ready to prove that integrals with respect to product measures can be computed as iterated integrals.
Proposition 14.65 Let ( X , C, p ) and ( Y , r, v ) be a-Jinite measure spaces and let the function f : X x Y -+ [0,001 be C x r-measurable. Then the function x is C-measurable and the function y
H
J, f
H
f x dv
d p is r-measurable. Moreovel;
Proof. We first prove the result for f = 1s where S E C x r. By Proposition 14.59, for all y E Y the function fY = (1s)y = 1s) is C-measurable. Now
and similarly we can prove that By Theorem 14.39 the equalities must hold for all simple C x r-measurable functions and by the Monotone Convergence Theorem the equalities hold for all nonnegative C x r-measurable functions.
Theorem 14.66 Fubini’s Theorem. Let ( X , C , ,x) and ( Y , r, v) be a-finite measure spaces and let f : X x Y + [-m, 001 be p x v-integrable. Then for p-almost all x E X the function f , is v-integrable and for v-almost all y E Y the function f’ iff, is v-integrable, is is @-integrable.The function x H otherwise,
14. Integration on Measure Spaces
254
Proof. By Proposition 14.65 we have that the functions x
x
H
s,
H
s,
( f+)x du and
( f -), du are pintegrable, so f x is