Measure Theory and Probability Theory

Springer Texts in Statistics Advisors: George Casella

Stephen Fienberg

Ingram Olkin

Springer Texts in Statistics Alfred: Elements of Statistics for the Life and Social Sciences Athreya and Lahiri: Measure Theory and Probability Theory Berger: An Introduction to Probability and Stochastic Processes Bilodeau and Brenner: Theory of Multivariate Statistics Blom: Probability and Statistics: Theory and Applications Brockwell and Davis: Introduction to Times Series and Forecasting, Second Edition Carmona: Statistical Analysis of Financial Data in S-Plus Chow and Teicher: Probability Theory: Independence, Interchangeability, Martingales, Third Edition Christensen: Advanced Linear Modeling: Multivariate, Time Series, and Spatial Data—Nonparametric Regression and Response Surface Maximization, Second Edition Christensen: Log-Linear Models and Logistic Regression, Second Edition Christensen: Plane Answers to Complex Questions: The Theory of Linear Models, Third Edition Creighton: A First Course in Probability Models and Statistical Inference Davis: Statistical Methods for the Analysis of Repeated Measurements Dean and Voss: Design and Analysis of Experiments du Toit, Steyn, and Stumpf: Graphical Exploratory Data Analysis Durrett: Essentials of Stochastic Processes Edwards: Introduction to Graphical Modelling, Second Edition Finkelstein and Levin: Statistics for Lawyers Flury: A First Course in Multivariate Statistics Ghosh, Delampady and Samanta: An Introduction to Bayesian Analysis: Theory and Methods Gut: Probability: A Graduate Course Heiberger and Holland: Statistical Analysis and Data Display: An Intermediate Course with Examples in S-PLUS, R, and SAS Jobson: Applied Multivariate Data Analysis, Volume I: Regression and Experimental Design Jobson: Applied Multivariate Data Analysis, Volume II: Categorical and Multivariate Methods Kalbfleisch: Probability and Statistical Inference, Volume I: Probability, Second Edition Kalbfleisch: Probability and Statistical Inference, Volume II: Statistical Inference, Second Edition Karr: Probability Keyfitz: Applied Mathematical Demography, Second Edition Kiefer: Introduction to Statistical Inference Kokoska and Nevison: Statistical Tables and Formulae Kulkarni: Modeling, Analysis, Design, and Control of Stochastic Systems Lange: Applied Probability Lange: Optimization Lehmann: Elements of Large-Sample Theory (continued after index)

Krishna B. Athreya Soumendra N. Lahiri

Measure Theory and Probability Theory

Krishna B. Athreya Department of Mathematics and Department of Statistics Iowa State University Ames, IA 50011 [email protected]

Editorial Board George Casella Department of Statistics University of Florida Gainesville, FL 32611-8545 USA

Soumendra N. Lahiri Department of Statistics Iowa State University Ames, IA 50011 [email protected]

Stephen Fienberg Department of Statistics Carnegie Mellon University Pittsburgh, PA 15213-3890 USA

Ingram Olkin Department of Statistics Stanford University Stanford, CA 94305 USA

Library of Congress Control Number: 2006922767 ISBN-10: 0-387-32903-X ISBN-13: 978-0387-32903-1

e-ISBN: 0-387-35434-4

Printed on acid-free paper. ©2006 Springer Science+Business Media, LLC All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excepts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed in the United States of America. 9 8 7 6 5 4 3 2 1 springer.com

(MVY)

Dedicated to our wives Krishna S. Athreya and Pubali Banerjee and to the memory of Uma Mani Athreya and Narayani Ammal

Preface

This book arose out of two graduate courses that the authors have taught during the past several years; the first one being on measure theory followed by the second one on advanced probability theory. The traditional approach to a first course in measure theory, such as in Royden (1988), is to teach the Lebesgue measure on the real line, then the differentation theorems of Lebesgue, Lp -spaces on R, and do general measure at the end of the course with one main application to the construction of product measures. This approach does have the pedagogic advantage of seeing one concrete case first before going to the general one. But this also has the disadvantage in making many students’ perspective on measure theory somewhat narrow. It leads them to think only in terms of the Lebesgue measure on the real line and to believe that measure theory is intimately tied to the topology of the real line. As students of statistics, probability, physics, engineering, economics, and biology know very well, there are mass distributions that are typically nonuniform, and hence it is useful to gain a general perspective. This book attempts to provide that general perspective right from the beginning. The opening chapter gives an informal introduction to measure and integration theory. It shows that the notions of σ-algebra of sets and countable additivity of a set function are dictated by certain very natural approximation procedures from practical applications and that they are not just some abstract ideas. Next, the general extension theorem of Carathedory is presented in Chapter 1. As immediate examples, the construction of the large class of Lebesgue-Stieltjes measures on the real line and Euclidean spaces is discussed, as are measures on finite and countable

viii

Preface

spaces. Concrete examples such as the classical Lebesgue measure and various probability distributions on the real line are provided. This is further developed in Chapter 6 leading to the construction of measures on sequence spaces (i.e., sequences of random variables) via Kolmogorov’s consistency theorem. After providing a fairly comprehensive treatment of measure and integration theory in the first part (Introduction and Chapters 1–5), the focus moves onto probability theory in the second part (Chapters 6–13). The feature that distinguishes probability theory from measure theory, namely, the notion of independence and dependence of random variables (i.e., measureable functions) is carefully developed first. Then the laws of large numbers are taken up. This is followed by convergence in distribution and the central limit theorems. Next the notion of conditional expectation and probability is developed, followed by discrete parameter martingales. Although the development of these topics is based on a rigorous measure theoretic foundation, the heuristic and intuitive backgrounds of the results are emphasized throughout. Along the way, some applications of the results from probability theory to proving classical results in analysis are given. These include, for example, the density of normal numbers on (0,1) and the Wierstrass approximation theorem. These are intended to emphasize the benefits of studying both areas in a rigorous and combined fashion. The approach to conditional expectation is via the mean square approximation of the “unknown” given the “known” and then a careful approximation for the L1 -case. This is a natural and intuitive approach and is preferred over the “black box” approach based on the Radon-Nikodym theorem. The final part of the book provides a basic outline of a number of special topics. These include Markov chains including Markov chain Monte Carlo (MCMC), Poisson processes, Brownian motion, bootstrap theory, mixing processes, and branching processes. The first two parts can be used for a two-semester sequence, and the last part could serve as a starting point for a seminar course on special topics. This book presents the basic material on measure and integration theory and probability theory in a self-contained and step-by-step manner. It is hoped that students will find it accessible, informative, and useful and also that they will be motivated to master the details by carefully working out the text material as well as the large number of exercises. The authors hope that the presentation here is found to be clear and comprehensive without being intimidating. Here is a quick summary of the various chapters of the book. After giving an informal introduction to the ideas of measure and integration theory, the construction of measures starting with set functions on a small class of sets is taken up in Chapter 1 where the Caratheodory extension theorem is proved and then applied to construct Lebesgue-Stieltjes measures. Integration theory is taken up in Chapter 2 where all the basic convergence theorems including the MCT, Fatou, DCT, BCT, Egorov’s, and Scheffe’s are

Preface

ix

proved. Included here are also the notion of uniform integrability and the classical approximation theorem of Lusin and its use in Lp -approximation by smooth functions. The third chapter presents basic inequalities for Lp spaces, the Riesz-Fischer theorem, and elementary theory of Banach and Hilbert spaces. Chapter 4 deals with Radon-Nikodym theory via the Riesz representation on L2 -spaces and its application to differentiation theorems on the real line as well as to signed measures. Chapter 5 deals with product measures and the Fubini-Tonelli theorems. Two constructions of the product measure are presented: one using the extension theorem and another via iterated integrals. This is followed by a discussion on convolutions, Laplace transforms, Fourier series, and Fourier transforms. Kolmogorov’s consistency theorem for the construction of stochastic processes is taken up in Chapter 6 followed by the notion of independence in Chapter 7. The laws of large numbers are presented in a unified manner in Chapter 8 where the classical Kolmogorov’s strong law as well as Etemadi’s strong law are presented followed by Marcinkiewicz-Zygmund laws. There are also sections on renewal theory and ergodic theorems. The notion of weak convergence of probability measures on R is taken up in Chapter 9, and Chapter 10 introduces characteristic functions (Fourier transform of probability measures), the inversion formula, and the Levy-Cramer continuity theorem. Chapter 11 is devoted to the central limit theorem and its extensions to stable and infinitely divisible laws. Chapter 12 discusses conditional expectation and probability where an L2 -approach followed by an approximation to L1 is presented. Discrete time martingales are introduced in Chapter 13 where the basic inequalities as well as convergence results are developed. Some applications to random walks are indicated as well. Chapter 14 discusses discrete time Markov chains with a discrete state space first. This is followed by discrete time Markov chains with general state spaces where the regeneration approach for Harris chains is carefully explained and is used to derive the basic limit theorems via the iid cycles approach. There are also discussions of Feller Markov chains on Polish spaces and Markov chain Monte Carlo methods. An elementary treatment of Brownian motion is presented in Chapter 15 along with a treatment of continuous time jump Markov chains. Chapters 16–18 provide brief outlines respectively of the bootstrap theory, mixing processes, and branching processes. There is an Appendix that reviews basic material on elementary set theory, real and complex numbers, and metric spaces. Here are some suggestions on how to use the book. 1. For a one-semester course on real analysis (i.e., measure end integration theory), material up to Chapter 5 and the Appendix should provide adequate coverage with Chapter 6 being optional. 2. A one-semester course on advanced probability theory for those with the necessary measure theory background could be based on Chapters 6–13 with a selection of topics from Chapters 14–18.

x

Preface

3. A one-semester course on combined treatment of measure theory and probability theory could be built around Chapters 1, 2, Sections 3.1– 3.2 of Chapter 3, all of Chapter 4 (Section 4.2 optional), Sections 5.1 and 5.2 of Chapter 5, Chapters 6, 7, and Sections 8.1, 8.2, 8.3 (Sections 8.5 and 8.6 optional) of Chapter 8. Such a course could be followed by another that includes some coverage of Chapters 9– 12 before moving on to other areas such as mathematical statistics or martingales and financial mathematics. This will be particularly useful for graduate programs in statistics. 4. A one-semester course on an introduction to stochastic processes or a seminar on special topics could be based on Chapters 14–18. A word on the numbering system used in the book. Statements of results (i.e., Theorems, Corollaries, Lemmas, and Propositions) are numbered consecutively within each section, in the format a.b.c, where a is the chapter number, b is the section number, and c is the counter. Definitions, Examples, and Remarks are numbered individually within each section, also of the form a.b.c, as above. Sections are referred to as a.b where a is the chapter number and b is the section number. Equation numbers appear on the right, in the form (b.c), where b is the section number and c is the equation number. Equations in a given chapter a are referred to as (b.c) within the chapter but as (a.b.c) outside chapter a. Problems are listed at the end of each chapter in the form a.c, where a is the chapter number and c is the problem number. In the writing of this book, material from existing books such as Apostol (1974), Billingsley (1995), Chow and Teicher (2001), Chung (1974), Durrett (2004), Royden (1988), and Rudin (1976, 1987) has been freely used. The authors owe a great debt to these books. The authors have used this material for courses taught over several years and have benefited greatly from suggestions for improvement from students and colleagues at Iowa State University, Cornell University, the Indian Institute of Science, and the Indian Statistical Institute. We are grateful to them. Our special thanks go to Dean Issacson, Ken Koehler, and Justin Peters at Iowa State University for their administrative support of this long project. Krishna Athreya would also like to thank Cornell University for its support. We are most indebted to Sharon Shepard who typed and retyped several times this book, patiently putting up with our never-ending “final” versions. Without her patient and generous help, this book could not have been written. We are also grateful to Denise Riker who typed portions of an earlier version of this book. John Kimmel of Springer got the book reviewed at various stages. The referee reports were very helpful and encouraging. Our grateful thanks to both John Kimmel and the referees.

Preface

xi

We have tried hard to make this book free of mathematical and typographical errors and misleading or ambiguous statements, but we are aware that there will still be many such remaining that we have not caught. We will be most grateful to receive such corrections and suggestions for improvement. They can be e-mailed to us at [email protected] or [email protected]. On a personal note, we would like to thank our families for their patience and support. Krishna Athreya would like to record his profound gratitude to his maternal granduncle, the late Shri K. Venkatarama Iyer, who opened the door to mathematical learning for him at a crucial stage in high school, to the late Professor D. Basu of the Indian Statistical Institute who taught him to think probabilistically, and to Professor Samuel Karlin of Stanford University for initiating him into research in mathematics. K. B. Athreya S. N. Lahiri May 12, 2006

Contents

Preface

vii

Measures and Integration: An Informal Introduction

1

1 Measures 1.1 Classes of sets . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Measures . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 The extension theorems and Lebesgue-Stieltjes measures 1.3.1 Caratheodory extension of measures . . . . . . . 1.3.2 Lebesgue-Stieltjes measures on R . . . . . . . . . 1.3.3 Lebesgue-Stieltjes measures on R2 . . . . . . . . 1.3.4 More on extension of measures . . . . . . . . . . 1.4 Completeness of measures . . . . . . . . . . . . . . . . . 1.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

9 9 14 19 19 25 27 28 30 31

2 Integration 2.1 Measurable transformations . . . . . . . . . 2.2 Induced measures, distribution functions . . 2.2.1 Generalizations to higher dimensions 2.3 Integration . . . . . . . . . . . . . . . . . . 2.4 Riemann and Lebesgue integrals . . . . . . 2.5 More on convergence . . . . . . . . . . . . . 2.6 Problems . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

39 39 44 47 48 59 61 71

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

xiv

Contents

3 Lp -Spaces 3.1 Inequalities . . . . . . . . . . 3.2 Lp -Spaces . . . . . . . . . . . 3.2.1 Basic properties . . . 3.2.2 Dual spaces . . . . . . 3.3 Banach and Hilbert spaces . . 3.3.1 Banach spaces . . . . 3.3.2 Linear transformations 3.3.3 Dual spaces . . . . . . 3.3.4 Hilbert space . . . . . 3.4 Problems . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

83 . 83 . 89 . 89 . 93 . 94 . 94 . 96 . 97 . 98 . 102

4 Differentiation 4.1 The Lebesgue-Radon-Nikodym theorem 4.2 Signed measures . . . . . . . . . . . . . 4.3 Functions of bounded variation . . . . . 4.4 Absolutely continuous functions on R . 4.5 Singular distributions . . . . . . . . . . 4.5.1 Decomposition of a cdf . . . . . . 4.5.2 Cantor ternary set . . . . . . . . 4.5.3 Cantor ternary function . . . . . 4.6 Problems . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

113 113 119 125 128 133 133 134 136 137

5 Product Measures, Convolutions, and Transforms 5.1 Product spaces and product measures . . . . . . . . 5.2 Fubini-Tonelli theorems . . . . . . . . . . . . . . . . 5.3 Extensions to products of higher orders . . . . . . . 5.4 Convolutions . . . . . . . . . . . .
. . . . . . . . . . 5.4.1 Convolution of measures on R, B(R) . . . . 5.4.2 Convolution of sequences . . . . . . . . . . . 5.4.3 Convolution of functions in L1 (R) . . . . . . 5.4.4 Convolution of functions and measures . . . . 5.5 Generating functions and Laplace transforms . . . . 5.6 Fourier series . . . . . . . . . . . . . . . . . . . . . . 5.7 Fourier transforms on R . . . . . . . . . . . . . . . . 5.8 Plancherel transform . . . . . . . . . . . . . . . . . . 5.9 Problems . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

147 147 152 157 160 160 162 162 164 164 166 173 178 181

6 Probability Spaces 6.1 Kolmogorov’s probability model . . . . 6.2 Random variables and random vectors 6.3 Kolmogorov’s consistency theorem . . 6.4 Problems . . . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

189 189 191 199 212

7 Independence

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

219

Contents

7.1 7.2 7.3

xv

Independent events and random variables . . . . . . . . . . 219 Borel-Cantelli lemmas, tail σ-algebras, and Kolmogorov’s zero-one law . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

8 Laws of Large Numbers 8.1 Weak laws of large numbers . . . . . . . . . . . 8.2 Strong laws of large numbers . . . . . . . . . . 8.3 Series of independent random variables . . . . . 8.4 Kolmogorov and Marcinkiewz-Zygmund SLLNs 8.5 Renewal theory . . . . . . . . . . . . . . . . . . 8.5.1 Definitions and basic properties . . . . . 8.5.2 Wald’s equation . . . . . . . . . . . . . 8.5.3 The renewal theorems . . . . . . . . . . 8.5.4 Renewal equations . . . . . . . . . . . . 8.5.5 Applications . . . . . . . . . . . . . . . 8.6 Ergodic theorems . . . . . . . . . . . . . . . . . 8.6.1 Basic definitions and examples . . . . . 8.6.2 Birkhoff’s ergodic theorem . . . . . . . . 8.7 Law of the iterated logarithm . . . . . . . . . . 8.8 Problems . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

237 237 240 249 254 260 260 262 264 266 268 271 271 274 278 279

9 Convergence in Distribution 9.1 Definitions and basic properties . . . . . . . . . . . . . . . 9.2 Vague convergence, Helly-Bray theorems, and tightness . 9.3 Weak convergence on metric spaces . . . . . . . . . . . . . 9.4 Skorohod’s theorem and the continuous mapping theorem 9.5 The method of moments and the moment problem . . . . 9.5.1 Convergence of moments . . . . . . . . . . . . . . . 9.5.2 The method of moments . . . . . . . . . . . . . . . 9.5.3 The moment problem . . . . . . . . . . . . . . . . 9.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

287 287 291 299 303 306 306 307 307 309

10 Characteristic Functions 10.1 Definition and examples . . . . . 10.2 Inversion formulas . . . . . . . . 10.3 Levy-Cramer continuity theorem 10.4 Extension to Rk . . . . . . . . . 10.5 Problems . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

317 317 323 327 332 337

11 Central Limit Theorems 11.1 Lindeberg-Feller theorems . . . . . . . . 11.2 Stable distributions . . . . . . . . . . . . 11.3 Infinitely divisible distributions . . . . . 11.4 Refinements and extensions of the CLT

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

343 343 352 358 361

. . . . .

. . . . .

. . . . .

xvi

Contents

11.4.1 The Berry-Esseen theorem . . . . . . . . 11.4.2 Edgeworth expansions . . . . . . . . . . 11.4.3 Large deviations . . . . . . . . . . . . . 11.4.4 The functional central limit theorem . . 11.4.5 Empirical process and Brownian bridge 11.5 Problems . . . . . . . . . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

361 364 368 372 374 376

12 Conditional Expectation and Conditional Probability 12.1 Conditional expectation: Definitions and examples . . . 12.2 Convergence theorems . . . . . . . . . . . . . . . . . . . 12.3 Conditional probability . . . . . . . . . . . . . . . . . . 12.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

. . . .

383 383 389 392 393

13 Discrete Parameter Martingales 13.1 Definitions and examples . . . . . . . . . . . . . 13.2 Stopping times and optional stopping theorems 13.3 Martingale convergence theorems . . . . . . . . 13.4 Applications of martingale methods . . . . . . 13.4.1 Supercritical branching processes . . . . 13.4.2 Investment sequences . . . . . . . . . . 13.4.3 A conditional Borel-Cantelli lemma . . . 13.4.4 Decomposition of probability measures . 13.4.5 Kakutani’s theorem . . . . . . . . . . . 13.4.6 de Finetti’s theorem . . . . . . . . . . . 13.5 Problems . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

399 399 405 417 424 424 425 425 427 429 430 430

14 Markov Chains and MCMC 14.1 Markov chains: Countable state space . . . . . . . . . . . 14.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . 14.1.2 Examples . . . . . . . . . . . . . . . . . . . . . . . 14.1.3 Existence of a Markov chain . . . . . . . . . . . . . 14.1.4 Limit theory . . . . . . . . . . . . . . . . . . . . . 14.2 Markov chains on a general state space . . . . . . . . . . . 14.2.1 Basic definitions . . . . . . . . . . . . . . . . . . . 14.2.2 Examples . . . . . . . . . . . . . . . . . . . . . . . 14.2.3 Chapman-Kolmogorov equations . . . . . . . . . . 14.2.4 Harris irreducibility, recurrence, and minorization . 14.2.5 The minorization theorem . . . . . . . . . . . . . . 14.2.6 The fundamental regeneration theorem . . . . . . 14.2.7 Limit theory for regenerative sequences . . . . . . 14.2.8 Limit theory of Harris recurrent Markov chains . . 14.2.9 Markov chains on metric spaces . . . . . . . . . . . 14.3 Markov chain Monte Carlo (MCMC) . . . . . . . . . . . . 14.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . 14.3.2 Metropolis-Hastings algorithm . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

439 439 439 440 442 443 457 457 458 461 462 464 465 467 469 473 477 477 478

. . . . . . . . . . .

. . . . . .

. . . . . . . . . . .

. . . . . .

. . . . . . . . . . .

. . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

Contents

xvii

14.3.3 The Gibbs sampler . . . . . . . . . . . . . . . . . . . 480 14.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481 15 Stochastic Processes 15.1 Continuous time Markov chains . . . . . . . . . . . . . . 15.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . 15.1.2 Kolmogorov’s differential equations . . . . . . . . 15.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . 15.1.4 Limit theorems . . . . . . . . . . . . . . . . . . . 15.2 Brownian motion . . . . . . . . . . . . . . . . . . . . . . 15.2.1 Construction of SBM . . . . . . . . . . . . . . . . 15.2.2 Basic properties of SBM . . . . . . . . . . . . . . 15.2.3 Some related processes . . . . . . . . . . . . . . . 15.2.4 Some limit theorems . . . . . . . . . . . . . . . . 15.2.5 Some sample path properties of SBM . . . . . . 15.2.6 Brownian motion and martingales . . . . . . . . 15.2.7 Some applications . . . . . . . . . . . . . . . . . 15.2.8 The Black-Scholes formula for stock price option 15.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

487 487 487 488 489 491 493 493 495 498 498 499 501 502 503 504

16 Limit Theorems for Dependent Processes 16.1 A central limit theorem for martingales . . 16.2 Mixing sequences . . . . . . . . . . . . . . . 16.2.1 Mixing coefficients . . . . . . . . . . 16.2.2 Coupling and covariance inequalities 16.3 Central limit theorems for mixing sequences 16.4 Problems . . . . . . . . . . . . . . . . . . .

. . . . . .

. . . . . .

509 509 513 514 516 519 529

17 The Bootstrap 17.1 The bootstrap method for independent variables . . . . . 17.1.1 A description of the bootstrap method . . . . . . . 17.1.2 Validity of the bootstrap: Sample mean . . . . . . 17.1.3 Second order correctness of the bootstrap . . . . . 17.1.4 Bootstrap for lattice distributions . . . . . . . . . 17.1.5 Bootstrap for heavy tailed random variables . . . . 17.2 Inadequacy of resampling single values under dependence 17.3 Block bootstrap . . . . . . . . . . . . . . . . . . . . . . . . 17.4 Properties of the MBB . . . . . . . . . . . . . . . . . . . . 17.4.1 Consistency of MBB variance estimators . . . . . . 17.4.2 Consistency of MBB cdf estimators . . . . . . . . . 17.4.3 Second order properties of the MBB . . . . . . . . 17.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . .

533 533 533 535 536 537 540 545 547 548 549 552 554 556

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

18 Branching Processes 561 18.1 Bienyeme-Galton-Watson branching process . . . . . . . . . 562

xviii

Contents

18.2 BGW process: Multitype case . . . . . . . . . . . . . . . . . 18.3 Continuous time branching processes . . . . . . . . . . . . . 18.4 Embedding of Urn schemes in continuous time branching processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . .

564 566

A Advanced Calculus: A Review A.1 Elementary set theory . . . . . . . . . . . . . . . . . . . . . A.1.1 Set operations . . . . . . . . . . . . . . . . . . . . . A.1.2 The principle of induction . . . . . . . . . . . . . . . A.1.3 Equivalence relations . . . . . . . . . . . . . . . . . . A.2 Real numbers, continuity, differentiability, and integration . A.2.1 Real numbers . . . . . . . . . . . . . . . . . . . . . . A.2.2 Sequences, series, limits, limsup, liminf . . . . . . . . A.2.3 Continuity and differentiability . . . . . . . . . . . . A.2.4 Riemann integration . . . . . . . . . . . . . . . . . . A.3 Complex numbers, exponential and trigonometric functions A.4 Metric spaces . . . . . . . . . . . . . . . . . . . . . . . . . . A.4.1 Basic definitions . . . . . . . . . . . . . . . . . . . . A.4.2 Continuous functions . . . . . . . . . . . . . . . . . . A.4.3 Compactness . . . . . . . . . . . . . . . . . . . . . . A.4.4 Sequences of functions and uniform convergence . . A.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . .

573 573 574 577 577 578 578 580 582 584 586 590 590 592 592 593 594

568 569

B List of Abbreviations and Symbols 599 B.1 Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . 599 B.2 Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 600 References

603

Author Index

610

Subject Index

612

Measures and Integration: An Informal Introduction

For many students who are learning measure and integration theory for the first time, the notions of a σ-algebra of subsets of a set Ω, countable additivity of a set function λ, measurability of a function, the definition of an integral, and the interchange of limits and integration are not easy to understand and often seem not so intuitive. The goals of this informal introduction to this subject are (1) to show that the notions of σ-algebra and countable additivity are logical consequences of certain natural approximation procedures; (2) the dividends for the assumption of these two properties are great, and they lead to a nice and natural theory that is also very powerful for the handling of limits. Of course, as the saying goes, the devil is in the details. After this informal introduction, the necessary details are given in the next few sections. It is hoped that after this heuristic explanation of the subject, the motivation for and the process of mastering the details on the part of the students will be forthcoming. What is Measure Theory? A simple answer is that it is a theory about the distribution of mass over a set S. If the mass is uniformly distributed and S is an Euclidean space Rk , it is the theory of Lebesgue measure on Rk (i.e., length in R, area in R2 , volume in R3 , etc.). Probability theory is concerned with the case when S is the sample space of a random experiment and the total mass is one. Consider the following example. Imagine an open field S and a snowy night. At daybreak one goes to the field to measure the amount of snow in as many of the subsets of S as

2

Measures and Integration: An Informal Introduction

possible. Suppose now that one has the tools to measure the snow exactly on a class of subsets, such as triangles, rectangles, circular shapes, elliptic shapes, etc., no matter how small. It is natural to try to approximate oddlyshaped regions by combinations of these “standard shapes,” and then use a limiting process to obtain a measure for the oddly-shaped regions and reach some limit for such sets. Let B denote the class of subsets of S whose measure is obtained this way and let λ(B) denote the amount of snow in each B ∈ B. Call B the class of all (snow) measurable sets and λ(B) the measure (of snow) on B for each B ∈ B. It is reasonable to expect that the following properties of B and λ(·) hold: Properties of B (i) A ∈ B ⇒ Ac ∈ B (i.e., if one can measure the amount of snow on A and knows the total amount on S, then one knows the amount of snow on Ac ). (ii) A1 , A2 ∈ B ⇒ A1 ∪ A2 ∈ B (i.e., if one can measure the amount of snow on A1 and A2 , then one can do the same for A1 ∪ A2 ). (iii)  If {An : n ≥ 1} ⊂ B, and An ⊂ An+1 for all n ≥ 1, then limn→∞ An ≡ ∞ n=1 An ∈ B (i.e., if one can measure the amount of snow on An for each n ≥ 1 on an increasing sequence of sets, then one can do so on the limit of An ). (iv) C ⊂ B where C is the class of nice sets such as triangles, squares, etc., that one started with. Properties of λ(·) (i) λ(A) ≥ 0 for A ∈ B (i.e., the amount of snow on any set is nonnegative!) (ii) If A1 , A2 ∈ B, A1 ∩ A2 = ∅, λ(A1 ∪ A2 ) = λ(A1 ) + λ(A2 ) (i.e., the amounts of snow on two disjoint sets simply add up! This property of λ is referred to as finite additivity). (iii) If {An : n ≥ 1} ⊂B, are such that An ⊂ An+1 for all n, then ∞ λ(limn→∞ An ) = λ( n=1 An ) = limn→∞ λ(An ) (i.e., if we can approximate a set A by an increase sequence of sets {An }n≥1 from B, then λ(A) = limn→∞ λ(An ). This property of λ is referred to as monotone continuity from below, or m.c.f.b. in short). This last assumption (iii) is what guarantees that different approximations lead to consistent limits. Thus, if there are two increasing sequences

{An }n≥1 and {An }n≥1 having the same limit A but {λ(An )}n≥1 and

{λ(An )}n≥1 have different limits, then the approximating procedures are not consistent.

Measures and Integration: An Informal Introduction

3

It turns out that the above set of reasonable and natural assumptions lead to a very rich and powerful theory that is widely applicable. A triplet (S, B, λ) that satisfies the above two sets of assumptions is called a measure space. The assumptions on B and λ are equivalent to the following: On B B(i) : ∅, the empty set, lies in B B(ii) : A ∈ B ⇒ Ac ∈ B (same as (i) before) B(iii) : A1 , A2 , . . . ∈ B ⇒ ∪i Ai ∈ B (combines (ii) and (iii) above) (Closure under countable unions). On λ λ(i) : λ(·) ≥ 0 (same as (i) before) and λ(∅) = 0. ∞ λ(ii) : λ(∪n≥1 An ) = n=1 λ(An ) if {An }n≥1 ⊂ B are pairwise disjoint, i.e., Ai ∩ Aj = ∅ for i = j (Countable additivity). Any collection B of subsets of S that satisfies B(i) , B(ii) , B(iii) above is called a σ-algebra. Any set function λ on a σ-algebra B that satisfies λ(i) and λ(ii) above is called a measure. Thus, a measure space is a triplet (S, B, λ) where S is a nonempty set, B is a σ-algebra of subsets of S and λ is a measure on B. Notice that the σ-algebra structure on B and the countable additivity of λ are necessary consequences of the very natural assumptions (i), (ii), and (iii) on B and λ defined at the beginning. It is not often the case that one is given B and λ explicitly. Typically, one starts with a small collection C of subsets of S that have properties resembling intervals or rectangles and a set function λ on C. Then, B is the smallest σ-algebra containing C obtained from C by various operations such as countable unions, intersections, and their limits. The key properties on C that one needs are: (i) A, B ∈ C ⇒ A ∩ B ∈ C (e.g., intersection of intervals is an interval). (ii) A ∈ C ⇒ Ac is a finite union of sets from C (e.g., the complement of an interval is the union of two intervals or an interval itself). A collection C satisfying (i) and (ii) is called a semialgebra. The function λ on B is an extension of λ on C. For this extension to be a measure on B, the conditions needed are (i) λ(A) ≥ 0

for all A ∈ C

(ii) If A1 , A2 , . . . ∈ C are pairwise disjoint and A = ∞ λ(A) = n=1 λ(An ).

 n≥1

An ∈ C, then

4

Measures and Integration: An Informal Introduction

There is a result, known as the extension theorem, that says that given such a pair (C, λ), it is possible to extend λ to B, the smallest σ-algebra containing C, such that (S, B, λ) is a measure space. Actually, it does more. It constructs a σ-algebra B ∗ larger than B and a measure λ∗ on B ∗ such that (S, B ∗ , λ∗ ) is a larger measure space, λ∗ coincides with λ on C and it provides nice approximation theorems. For example, the following approximation result is available: If B ∈ B∗ with λ∗ (B) < ∞, then for every  > 0, B can be approximated by a finite union of sets from C, i.e., there kexist sets A1 , . . . , Ak ∈ C with k < ∞ such that λ∗ (AB) <  where A ≡ i=1 Ai and AB = (A∩B c )∪(Ac ∩B), the symmetric difference between A and B. That is, in principle, every (measurable) set B of finite measure (i.e., B belonging to B∗ with λ∗ (B) < ∞) is nearly a finite union of (elementary) sets that belong to C. For example if S = R and C is the class of intervals, then every measurable set of finite measure is nearly a finite union of disjoint bounded open intervals. The following are some concrete examples of the above extension procedure. Theorem: (Lebesgue-Stieltjes measures on R). Let F : R → R satisfy (i) x1 < x2 ⇒ F (x1 ) ≤ F (x2 ) (nondecreasing); (ii) F (x) = F (x+) ≡ limy↓x F (y) for all x ∈ R (i.e., F (·) is right continuous). Let C be the class of sets of the form (a, b], or (b, ∞), −∞ ≤ a < b < ∞. Then, there exists a measure µF defined on B ≡ B(R), the smallest σalgebra generated by C such that µF ((a, b]) = F (b) − F (a)

for all

− ∞ < a < b < ∞.

The σ-algebra B ≡ B(R) is called the Borel σ-algebra of R. Corollary: There exists a measure m on B(R) such that m(I) = the length of I, for any interval I. Proof: Take F (x) ≡ x in the above theorem. This measure is called the Lebesgue measure on R. Corollary: There exists a measure λ on B(R) such that  b 2 1 e−x /2 dx. λ((a, b]) = √ 2π a Proof: Take F (x) =

x −∞

2 √1 e−u /2 du, 2π

x ∈ R.

2

Measures and Integration: An Informal Introduction

5

This measure is called the standard normal probability measure on R. 2 Theorem: (Lebesgue-Stieltjes measures on R2 ). Let F : R2 → R be a function satisfying the following: (i) (Monotonicity) For x = (x1 , x2 ) , y = (y1 , y2 ) with xi ≤ yi for i =
 1, 2, ∆F (x, y) ≡ F (y1 , y2 ) − F (x1 , y2 ) − F (y1 , x2 ) + F (x1 , x2 ) ≥ 0. (ii) (Continuity from above) F (x) =

lim

yi ↓xi ,i=1,2

F (y) for all x ∈ R2 .

Let C be the class of all rectangles of the form (a, b] ≡ (a1 , b1 ] × (a2 , b2 ] with a = (a1 , a2 ) , b = (b1 , b2 ) ∈ R2 . Then there exists a measure µF , defined on the σ-algebra B ≡ B(R2 ), generated by C, such that   µF ((a, b]) = ∆F (a, b). The above theorems have a converse that says that every measure on (Rk , B(Rk )) that is finite on bounded sets arises from some function F (called a distribution function) and is, therefore, a Lebesgue-Stieltjes measure. Here is another simple example of a measure space (with discrete S). Example: Let S = {s1 , s2 , . . . , sk }, k ≤ ∞, and let B = P(S), the power set of S, i.e., the collection of all possible subsets of S. Let p1 , p2 , . . . be nonnegative numbers. Let pi IA (si ), λ(A) ≡ 1≤i≤k

where IA is the indicator function of the set A, defined by IA (s) = 1 if s ∈ A and 0 otherwise. It is easy to verify that (S, B, λ) is a measure space and also that every measure λ on B arises this way. What is Integration Theory? In short, it is a theory about weighted sums of functions on a set S when the weights are specified by a mass distribution λ. Here is a more detailed answer. Let (S, B, λ) be a measure space. Suppose f : S → R is a simple function, i.e., f is such that f (S) is a finite set {a1 , a2 , . . . , ak }. It is reasonable to k define the weighted sum of f with respect to λ as i=1 ai λ(Ai ) where Ai = f −1 {ai }. Of course, for this to be well defined, one needs Ai to be in B and λ(Ai ) < ∞ for all i such that ai = 0. k Notice that the quantity i=1 ai λ(Ai ) remains the same whether the ai ’s are distinct or not. Call this the integral of f with respect to λ and denote

6

Measures and Integration: An Informal Introduction

  this  by f dλ.  If f and g are simple, then for α,β ∈ R, (αf + βg)dλ = α f dx + β gdλ. Now how should one define f dλ (integral of f with respect to λ) for a nonsimple f ? The answer, of course, is to “approximate” by simple functions. Let f be a nonnegative function. To define the integral of f , one would like to approximate f by simple functions. It turns out that a necessary and sufficient condition for this is that for any a ∈ R, the set {s : f (s) ≤ a} is in B. Such a function f is called measurable with respect to B or B-measurable or simply, measurable (if B is kept fixed throughout). Let f be a nonnegative B measurable function. Then there exists a sequence {fn }n≥1 of simple nonnegative functions such that for each s ∈ S, {fn (s)}n≥1 is a nondecreasing sequence converging to f (s). It is now natural to define the weighted sum of f with respect to λ, i.e., the integral of f with respect to λ, denoted by f dλ, as 

 f dλ = lim

fn dλ.

n→∞

An immediate question is: Is the right side the same for all such approximating sequences {fn }n≥1 ? The answer is a yes; it is guaranteed by the very natural assumption imposed on λ that it is finitely additive and monotone continuous from below, i.e. λ(ii) and λ(iii) (or equivalently, that λ is countably additive, i.e., λ(ii) ). One can strengthen this to a stronger result known as the monotone convergence theorem, a key result that in turn leads to two other major convergence results. The monotone convergence theorem (MCT): Let (S, B, λ) be a measure space and let fn : S → R+ , n ≥ 1 be a sequence of nonnegative Bmeasurable functions (not necessarily simple) such that for all s ∈ S, (i) fn (s) ≤ fn+1 (s),

for all

n ≥ 1,

and

(ii) lim fn (s) = f (s). n→∞

Then f is B-measurable and



f dλ = lim

n→∞



fn dλ.

This says that the integral and the limit can be interchanged for monotone nondecreasing nonnegative B-measurable functions. Note that if fn = IAn , the indicator function of a set An and if An ⊂ An+1 for each n, then the MCT is the same as m.c.f.b. (cf. property λ(ii)). Thus, the very natural assumption of m.c.f.b. yields a basic convergence result that makes the integration theory so elegant and  powerful. To extend the definition of f dλ to a real valued, B-measurable function f : S → R, one uses the simple idea that f can be decomposed as f = f + − f − where f + (s) = max{f (s), 0} and f − (s) = max{−f (s),  0}, s ∈ S. Since both f + and f − are nonnegative and B-measurable, f + dλ and

Measures and Integration: An Informal Introduction



7

f − dλ are both well defined. Now set    f dλ = f + dλ − f − dλ,

provided at least one of the two terms on the right is finite. function f  The + dλ and f − dλ is said to be integrable with respect to (w.r.t.) λ if both f  are finite or, equivalently, if |f |dλ < ∞. The following is a consequence of the MCT. Fatou’s lemma: Let {fn }n≥1 be a sequence of nonnegative B-measurable functions on a measure space (S, B, λ). Then   lim inf fn dλ ≤ lim inf fn dλ. n→∞

n→∞

This in turn leads to (Lebesgue’s) dominated convergence theorem (DCT): Let {fn }n≥1 be a sequence of B-measurable functions from a measure space (S, B, λ) to R and let g be a B-measurable nonnegative integrable function on (S, B, λ). Suppose that for each s in S, (i) |fn (s)| ≤ g(s)

for all

n≥1

and

(ii) lim fn (s) = f (s). n→∞

Then, f is integrable and    fn dλ = f dλ = lim lim fn dλ. n→∞

n→∞

Thus some very natural assumptions on B and λ lead to an interesting measure and integration theory that is quite general and that allows the interchange of limits and integrals under fairly general conditions. A systematic treatment of the measure and integration theory is given in the next five chapters.

1 Measures

Section 1.1 deals with algebraic operations on subsets of a given nonempty set Ω. Section 1.2 treats nonnegative set functions on classes of sets and defines the notion of a measure on an algebra. Section 1.3 treats the extension theorem, and Section 1.4 deals with completeness of measures.

1.1 Classes of sets Let Ω be a nonempty set and P(Ω) ≡ {A : A ⊂ Ω} be the power set of Ω, i.e., the class of all subsets of Ω. Definition 1.1.1: A collection of sets F ⊂ P(Ω) is called an algebra if (a) Ω ∈ F, (b) A ∈ F implies Ac ∈ F, and (c) A, B ∈ F implies A ∪ B ∈ F (i.e., closure under pairwise unions). Thus, an algebra is a class of sets containing Ω that is closed under complementation and pairwise (and hence finite) unions. It is easy to see that one can equivalently define an algebra by requiring that properties (a), (b) hold and that the property (c)



A, B ∈ F ⇒ A ∩ B ∈ F

holds (i.e. closure under finite intersections).

10

1. Measures

Definition 1.1.2: A class F ⊂ P(Ω) is called a σ-algebra if it is an algebra and if it satisfies

An ∈ F. (d) An ∈ F for n ≥ 1 ⇒ n≥1

Thus, a σ-algebra is a class of subsets of Ω that contains Ω and is closed under complementation and countable unions. As pointed out in the introductory chapter, a σ-algebra can be alternatively defined as an algebra that is closed under monotone unions as the following shows. Proposition 1.1.1: Let F ⊂ P(Ω). Then F is a σ-algebra if and only if F is an algebra and satisfies

An ∈ F. An ∈ F, An ⊂ An+1 for all n ⇒ n≥1 ∞ Proof: The ‘only if’ part is obvious. nFor the ‘if’ part, let {Bn }n=1 ⊂ F. Then, since F is analgebra, An≡ j=1 Bj ∈ F for all n. Further, An ⊂ An+1 for all n and n≥1 Bn = n≥1 An . Since by hypothesis ∪n An ∈ F, ∪n Bn ∈ F. 2

Here are some examples of algebras and σ-algebras. Example 1.1.1: Let Ω = {a, b, c, d}. Consider the classes F1 = {Ω, ∅, {a}} and F2 = {Ω, ∅, {a}, {b, c, d}}. Then, F2 is an algebra (and also a σ-algebra), but F1 is not an algebra, since {a}c ∈ F1 . Example 1.1.2: Let Ω be any nonempty set and let F3 = P(Ω) ≡ {A : A ⊂ Ω},

the power set of

Ω

and F4 = {Ω, ∅}. Then, it is easy to check that both F3 and F4 are σ-algebras. The latter σ-algebra is often called the trivial σ-algebra on Ω (Problem 1.1). From the definition it is clear that any σ-algebra is also an algebra and thus F2 , F3 , F4 are examples of algebras, too. The following is an example of an algebra that is not a σ-algebra. Example 1.1.3: Let Ω be a nonempty set, and let |A| denote the number of elements of a set A ⊂ Ω. Define. F5 = {A ⊂ Ω : either |A| is finite or |Ac | is finite}.

1.1 Classes of sets

11

Then, note that (i) Ω ∈ F5 (since |Ωc | = |∅| = 0)), (ii) A ∈ F5 implies Ac ∈ F5 (if |A| < ∞, then |(Ac )c | = |A| < ∞ and if |Ac | < ∞, then Ac ∈ F5 trivially). Next, suppose that A, B ∈ F5 . If either |A| < ∞ or |B| < ∞, then |A ∩ B| ≤ min{|A|, |B|} < ∞, so that A ∩ B ∈ F5 . On the other hand, if both |Ac | < ∞ and |B c | < ∞, then |(A ∩ B)c | = |Ac ∪ B c | ≤ |Ac | + |B c | < ∞, implying that A ∩ B ∈ F5 . Thus, property (c) holds, and F5 is an algebra. However, if |Ω| = ∞, then F5 is not a σ-algebra. To see this, suppose that |Ω| = ∞ and {ω 1 , ω2 , . . .} ⊂ Ω. Then, by definition, Ai = {ωi } ∈ F5 for all ∞ i ≥ 1, but A ≡ i=1 A2i−1 = {ω1 , ω3 , . . .} ∈ F5 , since |A| = |Ac | = ∞. Example 1.1.4: Let Ω be a nonempty set and let F6 = {A ⊂ Ω : A is countable or Ac is countable}. Then, it is easy to show that F6 is a σ-algebra (Problem 1.3). Suppose {Fθ : θ ∈ Θ} is a family  of σ-algebras on Ω. From the definition, it follows that the intersection θ∈Θ Fθ is a σ-algebra, no matter how large the index set Θ is (Problem 1.4). However, the union of two σ-algebras may not even be an algebra (Problem 1.5). For the development of measure theory and probability theory, the concept of a σ-algebra plays a crucial role. In many instances, given an arbitrary collection of subsets of Ω, one would like to extend it to a possibly larger class that is a σ-algebra. This leads to the following definition. Definition 1.1.3: If A is a class of subsets of Ω, then the σ-algebra generated by A, denoted by σA, is defined as σA = F, F∈I(A)

where I(A) ≡ {F : A ⊂ F and F is a σ-algebra on Ω} is the collection of all σ-algebras containing the class A. Note that since the power set P(Ω) contains A and is itself a σ-algebra, the collection I(A) is not empty and hence, the intersection in the above definition is well defined. Example 1.1.5: In the setup of Example 1.1.1, σF1  = F2 (why?). A particularly useful class of σ-algebras are those generated by open sets of a topological space. These are called Borel σ-algebras. A topological space is a pair (S, T ) where S is a nonempty set and T is a collection of subsets of S such that(i) S ∈ T , (ii) O1 , O2 ∈ T ⇒ O1 ∩ O2 ∈ T , and (iii) {Oα : α ∈ I} ⊂ T ⇒ α∈I Oα ∈ T . Elements of T are called open sets.

12

1. Measures

A metric space is a pair (S, d) where S is a nonempty set and d is a function from S × S to R+ satisfying (i) d(x, y) = d(y, x) for all x, y in S, (ii) d(x, y) = 0 iff x = y, and (iii) d(x, z) ≤ d(x, y) + d(y, z) for all x, y, z in S. Property (iii) is called the triangle inequality. The function d is called a metric on S (cf. see A.4). Any Euclidean space Rn (1 ≤ n < ∞) is a metric space under any one of the following metrics: (a) For 1 ≤ p < ∞, dp (x, y) =

 n

|xi − yi |p

1/p .

i=1

(b) d∞ (x, y) = max |xi − yi |. 1≤i≤n

(c) For 0 < p < 1, dp (x, y) =

 n

 |xi − yi |p .

i=1

A metric space (S, d) is a topological space where a set O is open if for all x ∈ O, there is an  > 0 such that B(x, ) ≡ {y : d(y, x) < } ⊂ O. Definition 1.1.4: The Borel σ-algebra on a topological space S (in particular, on a metric space or an Euclidean space) is defined as the σ-algebra generated by the collection of open sets in S. Example 1.1.6: Let B(Rk ) denote the Borel σ-algebra on Rk , 1 ≤ k < ∞. Then, B(Rk ) ≡ σ{A : A is an open subset of Rk } is also generated by each of the following classes of sets O1 O2

= {(a1 , b1 ) × . . . × (ak , bk ) : −∞ ≤ ai < bi ≤ ∞, 1 ≤ i ≤ k};

O3

= {(a1 , b1 ) × . . . × (ak , bk ) : ai , bi ∈ Q, ai < bi , 1 ≤ i ≤ k};

O4

= {(−∞, x1 ) × . . . × (−∞, xk ) : x1 , . . . , xk ∈ Q},

= {(−∞, x1 ) × · · · × (−∞, xk ) : x1 , . . . , xk ∈ R};

where Q denotes the set of all rational numbers. To show this, note that σOi  ⊂ B(Rk ) for i = 1, 2, 3, 4, and hence, it is enough to show that σOi  ⊃ B(Rk ). Let G be a σ-algebra containing set A ⊂ Rk , there exist a sequence of O3 . Observe that given any open  sets {Bn }n≥1 in O3 such that A = n≥1 Bn (Problem 1.9). Since G is a σalgebra and Bn ∈ G for all n ≥ 1, A ∈ G. Thus, G is a σ-algebra containing all open subsets of Rk , and hence G ⊃ B(Rk ). Hence, it follows that B(Rk ) ⊃ σO1  ⊃ σO3  =

G:G⊃O3

G ⊃ B(Rk ).

1.1 Classes of sets

13

Next note that any interval (a, b) ⊂ R can be expressed in terms of half spaces of the form (−∞, x), x ∈ R as

(a, b) =

∞

[(−∞, b) \ (−∞, a + n−1 )],

n=1

where for any two sets A and B, A\B = {x : x ∈ A, x ∈ / B}. It is not difficult to show that this implies that σOi  = B(Rk ) for i = 2, 4 (Problem 1.10). Example 1.1.7: Let Ω be a nonempty set with |Ω| = ∞ and F5 and F6 be as in Examples 1.1.3 and 1.1.4. Then F6 = σF5 . To see this, note that F6 is a σ-algebra containing F5 , so that σF5  ⊂ F6 . To prove the reverse inclusion, let G be a σ-algebra containing F5 . It is enough to show that F6 ⊂ G. Let A ∈ F6 . If A is countable, say ∞A = {ω1 , ω2 , . . .}, then Ai ≡ {ωi } ∈ F5 ⊂ G for all i ≥ 1 and hence A = i=1 Ai ∈ G. On the other hand, if Ac is countable, then by the above argument, Ac ∈ G ⇒ A ∈ G. Definition 1.1.5: A class C of subsets of Ω is a π-system or a π-class if A, B ∈ C ⇒ A ∩ B ∈ C. Example 1.1.8: The class C of intervals in R is a π-system whereas the class of all open discs in R2 is not. Definition 1.1.6: A class L of subsets of Ω is a λ-system or a λ-class if (i) Ω ∈ L, (ii) A, B ∈ L, A ⊂ B ⇒ B \ A ∈ L, and (iii) An ∈ L, An ⊂ An+1 for all n ≥ 1 ⇒ n≥1 An ∈ L. Example 1.1.9: Every σ-algebra F is a λ-system. But an algebra need not be a λ-system. It is easily checked that if L1 and L2 are λ-systems, then L1 ∩ L2 is also a λ-system. Recall that σB, the σ-algebra generated by B, is the intersection of all σ-algebras containing B and is also the smallest σ-algebra containing B. Similarly, for any B ⊂ P(Ω), the λ-system generated by B, denoted by λB, is defined as the intersection of all λ-systems containing B. It is the smallest λ-system containing B. Theorem 1.1.2: (The π-λ theorem). If C is a π-system, then λC = σC. Proof: For any C, σC is a λ-system and σC contains C. Thus, λC ⊂ σC for any C. Hence, it suffices to show that if C is a π-system, then λC is a σ-algebra . Since λC is a λ-system, it is closed under complementation and monotone increasing unions. By Proposition 1.1.1, it is enough to show that it is closed under intersection. Let λ1 (C) ≡ {A : A ∈ λC, A ∩ B ∈ λC for all B ∈ C}. Then, λ1 (C) is a λ-system and C being a π-system, λ1 (C) ⊃ C. Therefore, λ1 (C) ⊃ λC. But λ1 (C) ⊂ λC. So λ1 (C) = λC.

14

1. Measures

Next, let λ2 (C) ≡ {A : A ∈ λC, A ∩ B ∈ λC for all B ∈ λC}. Then λ2 (C) is a λ-system and by the previous step C ⊂ λ2 (C) ⊂ λC. Hence, it follows that λ2 (C) = λC, i.e., λC is closed under intersection. This completes the proof of the theorem. 2 Corollary 1.1.3: If C is a π-system and L is a λ-system containing C, then L ⊃ σC. Remark 1.1.1: There are several equivalent definitions of λ-systems; see, for example, Billingsley (1995). A closely related concept is that of a monotone class; see, for example, Chung (1974).

1.2 Measures A set function is an extended real valued function defined on a class of subsets of a set Ω. Measures are nonnegative set functions that, intuitively speaking, measure the content of a subset of Ω. As explained in Section 2 of the introductory chapter, a measure has to satisfy certain natural requirements, such as the measure of the union of a countable collection of disjoint sets is the sum of the measures of the individual sets. Formally, one has the following definition. Definition 1.2.1: Let Ω be a nonempty set and F be an algebra on Ω. Then, a set function µ on F is called a measure if (a) µ(A) ∈ [0, ∞] for all A ∈ F; (b) µ(∅) = 0; (c) for any disjoint collection of sets A1 , A2 , . . . , ∈ F with µ



n≥1

 n≥1

An ∈ F,

∞  An = µ(An ). n=1

As discussed in Section 2 of the introductory chapter, these conditions on µ are equivalent to finite additivity and monotone continuity from below. Proposition 1.2.1: Let Ω be a nonempty set and F be an algebra of subsets of Ω and µ be a set function on F with values in [0, ∞] and with µ(∅) = 0. Then, µ is a measure iff µ satisfies (iii)a : (finite additivity) for all A1 , A2 ∈ F with A1 ∩A2 = ∅, µ(A1 ∪A2 ) = µ(A1 ) + µ(A2 ), and (iii)b : (monotone continuity from below or, m.c.f.b., in short) for any collection {An }n≥1 of sets in F such that An ⊂ An+1 for all n ≥ 1

1.2 Measures

and

 n≥1

15

An ∈ F, µ



 An = lim µ(An ). n→∞

n≥1

Proof: Let µ be a measure on F. Since µ satisfies (iii), taking A3 , A4 , . . . to be ∅ yields (iii)a . This implies that for A and B in F, A ⊂ B ⇒ µ(B) = µ(A) + µ(B \ A) ≥ µ(A), i.e., µ is monotone. To establish (iii)b , note that  if µ(An ) = ∞ for some n = n0 , then µ(An ) = ∞ for all n ≥ n0 and µ( n≥1 An ) = ∞ and (iii)b holds in this case. Hence, suppose that µ(An ) < ∞ for all n ≥ 1. Setting Bn = An \ An−1 for n ≥ 1 (with A0 = ∅), by (iii)a , µ(Bn ) =µ(An ) − µ(A n−1 ). Since {Bn }n≥1 is a disjoint collection of sets in F with n≥1 Bn = n≥1 An , by (iii) µ



 An

= µ

n≥1



∞ N  Bn = µ(Bn ) = lim [µ(An ) − µ(An−1 )]

=

N →∞

n=1

n≥1

n=1

lim µ(AN ),

N →∞

and so (iii)b holds also in this case.   Conversely, let µ satisfy µ(∅) = 0 and nn }n≥1 be  (iii)a and (iii)b . Let {A a disjoint collection of sets in F with i≥1 Ai ∈ F. Let Cn = j=1 Aj for n ≥ 1. Since F isan algebra,Cn ∈ F for all n ≥ 1. Also, Cn ⊂ Cn+1 for all n ≥ 1. Hence, n≥1 Cn = j≥1 Aj . By (iii)b , 

   = µ Aj Cn = lim µ(Cn ) µ j≥1

=

=

n≥1 n

lim

n→∞ ∞

n→∞

µ(Aj )



(by (iii)a )

j=1

µ(Aj ).

j=1

Thus, (iii) holds.

2

Remark 1.2.1: The definition of a measure given in Definition 1.2.1 is valid when F is a σ-algebra. However, very often one may start with a measure on an algebra A and then extend it to a measure on the σ-algebra σA. This is why the definition of a measure on an algebra is given here. In the same vein, one may begin with a definition of a measure on a class of subsets of Ω that form only a semialgebra (cf. Definition 1.3.1). As described in the introductory chapter, such preliminary collections of sets are “nice” sets for which the measure may be defined easily, and the extension to a σalgebra containing these sets may be necessary if one is interested in more general sets. This topic is discussed in greater detail in the next section.

16

1. Measures

Definition 1.2.2: A measure µ is called finite or infinite according as µ(Ω) < ∞ or µ(Ω) = ∞. A finite measure with µ(Ω) = 1 is called a probability measure. A measure µ on a σ-algebra F is called σ-finite if there exist a countable collection of sets A1 , A2 , . . . , ∈ F, not necessarily disjoint, such that

An = Ω and (b) µ(An ) < ∞ for all n ≥ 1. (a) n≥1

Here are some examples of measures. Example 1.2.1: (The counting measure). Let Ω be a nonempty set and F3 = P(Ω) be the set of all subsets of Ω (cf. Example 1.1.2). Define µ(A) = |A|,

A ∈ F3 ,

where |A| denotes the number of elements in A. It is easy to check that µ satisfies the requirements (a)–(c) of a measure. This measure µ is called the counting measure on Ω. Note that µ is finite iff Ω is finite and it is σ-finite if Ω is countably infinite. Example 1.2.2: (Discrete probability measures). Let ω1 , ω2 , . . . , ∈ Ω and ∞ p1 , p2 , . . . ∈ [0, 1] be such that i=1 pi = 1. Define for any A ⊂ Ω P (A) =

∞

pi IA (ωi ),

i=1

where IA (·) denotes the indicator function of a set A, defined by IA (ω) = 0 or 1 according as ω ∈ A or ω ∈ A. For any disjoint collection of sets A1 , A2 , . . . ∈ P(Ω), P

∞

 Ai

=

i=1

∞ j=1

=

=

=

∞

pj I
∞ (ωj ) i=1 Ai

∞

pj j=1 i=1 ∞ ∞

 IAi (ωj )

 pj IAi (ωj )

i=1 ∞

j=1

P (Ai ),

i=1

where interchanging the order of summation is permissible since the summands are nonnegative. This shows that P is a probability measure on P(Ω).

1.2 Measures

17

Example 1.2.3: (Lebesgue-Stieltjes measures on R). As mentioned in the previous chapter (cf. Section 2), a large class of measures on the Borel σ-algebra B(R) of subsets of R, known as the Lebesgue-Stieltjes measures, arise from nondecreasing right continuous functions F : R → R. For each such F , the corresponding measure µF satisfies µF ((a, b]) = F (b) − F (a) for all −∞ < a < b < ∞. The construction of these µF ’s via the extension theorem will be discussed in the  next section. Also, note that if An = (−n, n), n = 1, 2, . . ., then R = n≥1 An and µF (An ) < ∞ for each n ≥ 1 (such measures are called Radon measures) and thus, µF is necessarily σ-finite. Proposition 1.2.2: Let µ be a measure on an algebra F, and let A, B, A1 , . . . , Ak ∈ F, 1 ≤ k < ∞. Then, (i) (Monotonicity) µ(A) ≤ µ(B) if A ⊂ B; (ii) (Finite subadditivity) µ(A1 ∪ . . . ∪ Ak ) ≤ µ(A1 ) + . . . + µ(Ak ); (iii) (Inclusion-exclusion formula) If µ(Ai ) < ∞ for all i = 1, . . . , k, then µ(A1 ∪ . . . ∪ Ak )

=

k i=1

µ(Ai ) −



µ(Ai ∩ Aj )

1≤i<j
+ . . . + (−1)k−1 µ(A1 ∩ . . . ∩ Ak ). Proof: µ(B) = µ(A ∪ (Ac ∩ B)) = µ(A) + µ(B \ A) ≥ µ(A), by (a) and (c) of Definition 1.2.1. This proves (i). To prove (ii), note that if either µ(A) or µ(B) is finite, then µ(A∩B) < ∞, by (i). Hence, using the countable additivity property (c), we have µ(A ∪ B)

= µ(A) + µ(B \ A) = µ(A) + [µ(B \ A) + µ(A ∩ B)] − µ(A ∩ B) = µ(A) + µ(B) − µ(A ∩ B).

(2.1)

Hence, (ii) follows from (2.1) and by induction. To prove (iii), note that the case k = 2 follows from (2.1). Next, suppose that (iii) holds for all sets A1 , . . . , Ak ∈ F with µ(Ai ) < ∞ for all i = 1, . . . , k for some k = n, n ∈ N. To show that it holds for k = n + 1, note that by (2.1), µ

n+1



= µ

i=1 n

i=1

 Ai 

Ai

+ µ(An+1 ) − µ

n i=1

 (Ai ∩ An+1 )

18

1. Measures

=

 n

µ(Ai ) −

 µ(Ai ∩ Aj ) + · · · + (−1)n−1 µ(A1 ∩ . . . ∩ An )

1
i=1

+ µ(An+1 ) −

 n

µ(Ai ∩ An+1 ) −



+ · · · + (−1)

n−1

=

n+1 i=1

µ(Ai ) −

µ(Ai ∩ Aj ∩ An+1 )

1≤i<j≤n

i=1



 µ(A1 ∩ . . . ∩ An+1 )

µ(Ai ∩ Aj ) + · · · + (−1) µ n

1≤i<j≤n+1

n+1

 Aj .

j=1

By induction, this completes the proof of Proposition 1.2.2.

2

In Proposition 1.2.1, it was shown that a set function µ on an algebra F is a measure iff it is finitely additive and monotone continuous from below. A natural question is: if µ is a measure  on F and {An }n≥1 is a collection of decreasing sets in F with A ≡ n≥1 An also in F, does the relation µ(A) = limn→∞ µ(An ) hold, i.e., does monotone continuity from above hold? The answer is positive under the assumption µ(An0 ) < ∞ for some n0 ∈ N. It turns out that, in general, this assumption cannot be dropped (Problem 1.18). Proposition 1.2.3: Let µ be a measure on an algebra F. (i) (Monotone continuity from above) Let {An }n≥1 be  a sequence of sets in F such that An+1 ⊂ An for all n ≥ 1 and A ≡ n≥1 An ∈ F. Also, let µ(An0 ) < ∞ for some n0 ∈ N. Then, lim µ(An ) = µ(A).

n→∞

(ii) (Countable subadditivity) If {An }n≥1 is a sequence of sets in F such  that n≥1 An ∈ F, then µ

∞ n=1

 An

≤

∞

µ(An ).

n=1

Proof: To prove part (i), without loss of generality (w.l.o.g.), assume that n0 = 1, i.e., µ(A1 ) < ∞. Let Cn = A1 \ An for n ≥ 1, and C∞ = A1 \ A.
Then Cn and C∞ belong to F and Cn ↑ C∞ . By Proposition 1.2.1 (iii)b ,
(i.e., by the m.c.f.b. property), µ(Cn ) ↑ µ(C∞ ) and by (iii)a , (i.e., finite additivity), µ(Cn ) = µ(A1 ) − µ(An ) for all 1 ≤ n < ∞, due to the fact µ(A1 ) < ∞. This proves (i).

1.3 The extension theorems and Lebesgue-Stieltjes measures

19

n  To prove part (ii), let Dn = i=1 Ai , n ≥ 1. Then, Dn ↑ D ≡ i≥1 Ai . Hence, by m.c.f.b. and finite subadditivity, µ(D) = lim µ(Dn ) ≤ lim n→∞

n→∞

n

µ(Ai ) =

i=1

∞

µ(An ).

2

n=1

Theorem 1.2.4: (Uniqueness of measures). Let µ1 and µ2 be two finite measures on a measurable space (Ω, F). Let C ⊂ F be a π-system such that F = σC. If µ1 (C) = µ2 (C) for all C ∈ C and µ1 (Ω) = µ2 (Ω), then µ1 (A) = µ2 (A) for all A ∈ F. Proof: Let L ≡ {A : A ∈ F, µ1 (A) = µ2 (A)}. It is easy to verify that L is a λ-system. Since C ⊂ L, by Theorem 1.1.2, L = σC = F. 2

1.3 The extension theorems and Lebesgue-Stieltjes measures As discussed earlier, in many situations, one starts with a given set function µ defined on a small class C of subsets of a set Ω and then wants to extend µ to a larger class M by some approximation procedure. In this section, a general result in this direction, known as the extension theorem, is established. This is then applied to the construction of Lebesgue-Stieltjes measures on Euclidean spaces. For another application, see Chapter 6.

1.3.1

Caratheodory extension of measures

Definition 1.3.1: Let Ω be a nonempty set and let P(Ω) be the power set of Ω. A class C ⊂ P(Ω) is called a semialgebra if (i) A, B ∈ C ⇒ A ∩ B ∈ C and (ii) for any A ∈ C, there exist sets B1 , B2 , . . . , Bk ∈ C, for some 1 ≤ k k < ∞, such that Bi ∩ Bj = ∅ for i = j, and Ac = i=1 Bi . Example 1.3.1: Ω = R, C ≡ {(a, b], (b, ∞) : −∞ ≤ a, b < ∞}. Example 1.3.2: Ω = R, C ≡ {I : I is an interval}. An interval I in R is a set in R such that a, b ∈ I, a < b ⇒ (a, b) ⊂ I. Example 1.3.3: Ω = Rk , C ≡ {I1 × I2 × ×Ik : Ij is an interval in R for 1 ≤ j ≤ k}. Recall that a collection A ⊂ P(Ω) is an algebra if it is closed under finite union and complementation. It is easily verified (Problem 1.19) that the smallest algebra containing a semialgebra C is A(C) ≡ {A : A = k i=1 Bi , Bi ∈ C for i = 1, . . . , k, k < ∞, }, i.e., the class of finite unions of sets from C.

20

1. Measures

Definition 1.3.2: A set function µ on a semialgebra C, taking values in ¯ + ≡ [0, ∞], is called a measure if (i) µ(∅) = 0 and (ii) for any sequence R  of sets {An }n≥1 ⊂ C with n≥1 An ∈ C, and Ai ∩ Aj = ∅ for i = j, ∞  µ( n≥1 An ) = n=1 µ(An ). Proposition 1.3.1: Let µ be a measure on a semialgebra C. Let A ≡ A(C) be the smallest algebra generated by C. For each A ∈ A, set µ ¯(A) =

k

µ(Bi ),

i=1

k if the set A has the representation A = i=1 Bi for some B1 , . . . , Bk ∈ C, k < ∞ with Bi ∩ Bj = ∅ for i = j. Then, k (i) µ ¯ is independent of the representation of A as A = i=1 Bi ; (ii) µ ¯ is finitely additive on A, i.e., A, B ∈ A, A ∩ B = ∅ ⇒ µ ¯(A ∪ B) = µ ¯(A) + µ ¯(B); and (iii) µ ¯ is countably additive  on A, i.e., if An ∈ A for all n ≥ 1, Ai ∩Aj = ∅ for all i = j, and n≥1 An ∈ A, then ∞  

An = µ ¯(An ). µ ¯ n≥1

n=1

Proof: Parts (i) and (ii) are easy to verify. Turning to part (iii), let each  kn  kn n ≥ 1, An = j=1 Bnj , Bnj ∈ C, {Bnj }j=1 disjoint. Since n≥1 An ∈ A  k then exist disjoint sets {Bi }ki=1 ⊂ C such that n≥1 An = i=1 Bi . Now  

Bi = Bi ∩ An = (Bi ∩ An ) n≥1

=



kn

n≥1

(Bi ∩ Bnj ).

n≥1 j=1

Since for all i, Bi ∈ C, Bi ∩ Bnj ∈ C for all j, n and µ is a measure on C µ(Bi ) =

kn

µ(Bi ∩ Bnj ).

n≥1 j=1

Thus, 

 µ ¯ An = n≥1

k i=1

µ(Bi ) =

kn k i=1 n≥1

j=1

 µ(Bi ∩ Bnj )

1.3 The extension theorems and Lebesgue-Stieltjes measures

=

kn k

=

 µ(Bi ∩ Bnj )

i=1 j=1

n≥1



21

µ ¯(An ),

n≥1

since An

=

An ∩

k

Bi

i=1

=

kn k



(Bi ∩ Bnj ).

i=1 j=1

2 Thus, the set function µ ¯ defined above is a measure on A. To go beyond A = A(C) to σA, which is also the same as σC (see Problem 1.19 (ii)), the first step involves using the given set function µ on C to define a set function µ∗ on the class of all subsets of Ω, i.e., on P(Ω), where for any A ∈ P(Ω), µ∗ (A) is an estimate ‘from above’ of the µ-content of A. Definition 1.3.3: Given a measure µ on a semialgebra C, the outer measure induced by µ is the set function µ∗ , defined on P(Ω), as   ∞

∗ µ (A) ≡ inf (3.1) µ(An ) : {An }n≥1 ⊂ C, A ⊂ An . n=1

n≥1

Thus, a given set A is covered by countable unions of sets from C and the sums of the measures on such covers are computed and µ∗ (A) is the greatest lower bound one can get in this way. It is not difficult to show that on C and A, this is not an overestimate. That is, µ∗ = µ on C and on A, µ∗ = µ ¯. Now suppose µ(Ω) < ∞. Let A ⊂ Ω be a set such that the overestimates of the µ-contents of A and Ac , namely, µ∗ (A) and µ∗ (Ac ), add up to µ(Ω), i.e., µ(Ω) = µ∗ (Ω) = µ∗ (A) + µ∗ (Ac ).

(3.2)

Then, there is no room for error and the estimates µ∗ (A) and µ∗ (Ac ) are in fact not overestimates at all. In this case, the set A may be considered ‘exactly measurable’ by this approximation procedure. This observation motivates the following definition. Definition 1.3.4: A set A is said to be µ∗ -measurable if µ∗ (E) = µ∗ (E ∩ A) + µ∗ (E ∩ Ac ) for all E ⊂ Ω.

(3.3)

In other words, an analog of (3.2) should hold in every portion E of Ω for a set A to be µ∗ -measurable.

22

1. Measures

It can be shown (Problem 1.20) that µ∗ defined in (3.1) satisfies: µ∗ (∅) A⊂B

= 0, ⇒ µ∗ (A) ≤ µ∗ (B),

(3.4) (3.5)

∞  An ≤ µ∗ (An ) .

(3.6)

and for any {An }n≥1 ⊂ P(Ω), µ∗



n≥1

n=1

¯ + ≡ [0, ∞] satisfying Definition 1.3.5: Any set function µ∗ : P(Ω) → R (3.4)–(3.6) is called an outer measure on Ω. The following result (due to C. Caratheodory) yields a measure space on Ω starting from a general outer measure µ∗ that need not arise from a measure µ as in (3.1). Theorem 1.3.2: Let µ∗ be an outer measure on Ω, i.e., it satisfies (3.4)– (3.6). Let M ≡ Mµ∗ ≡ {A : A is µ∗ -measurable, i.e., A satisfies (3.3)}. Then (i) M is a σ-algebra, (ii) µ∗ restricted to M is a measure, and (iii) µ∗ (A) = 0 ⇒ P(A) ⊂ M. Proof: From (3.3), it follows that ∅ ∈ M and that A ∈ M ⇒ Ac ∈ M. Next, it will be shown that M is closed under finite unions. Let A1 , A2 ∈ M. Then, for any E ⊂ Ω, µ∗ (E)

= µ∗ (E ∩ A1 ) + µ∗ (E ∩ Ac1 ) (since A1 ∈ M) = µ∗ (E ∩ A1 ∩ A2 ) + µ∗ (E ∩ A1 ∩ Ac2 ) +µ∗ (E ∩ Ac1 ∩ A2 ) + µ∗ (E ∩ Ac1 ∩ Ac2 )

(since A2 ∈ M).

But (A1 ∩ A2 ) ∪ (A1 ∩ Ac2 ) ∪ (Ac1 ∩ A2 ) = A1 ∪ A2 . Since µ∗ is subadditive, it follows that µ∗ (E ∩ (A1 ∪ A2 )) ≤ µ∗ (E ∩ A1 ∩ A2 ) + µ∗ (E ∩ A1 ∩ Ac2 ) + µ∗ (E ∩ Ac1 ∩ A2 ). Thus

µ∗ (E) ≥ µ∗ (E ∩ (A1 ∪ A2 )) + µ∗ (E ∩ (A1 ∪ A2 )c ).

The subadditivity of µ∗ yields the opposite inequality and so, A1 ∪ A2 ∈ M and hence, M is an algebra. To show that M is a σ-algebra, it suffices to show that M is closedunder monotone unions, i.e., An ∈ M, An ⊂ An+1 for all n ≥ 1 ⇒ A ≡ n≥1 An ∈ M. Let B1 = A1 and Bn = An ∩ Acn−1

1.3 The extension theorems and Lebesgue-Stieltjes measures

23

for all n ≥ 2. Then, ∞for all n ≥ 1, Bn ∈ M (since M is an algebra),  n B = A , and n j=1 j j=1 Bj = A. Hence, for any E ⊂ Ω, µ∗ (E)

= µ∗ (E ∩ An ) + µ∗ (E ∩ Acn ) = µ∗ (E ∩ An ∩ Bn ) + µ∗ (E ∩ An ∩ Bnc ) + µ∗ (E ∩ Acn ) (since Bn ∈ M) ∗

∗

∗

= µ (E ∩ Bn ) + µ (E ∩ An−1 ) + µ (E ∩ Acn ) n = µ∗ (E ∩ Bj ) + µ∗ (E ∩ Acn ) (by iteration) j=1

≥

n

µ∗ (E ∩ Bj ) + µ∗ (E ∩ Ac )

(by monotonicity).

j=1

Now letting n → ∞, and using the subadditivity of µ∗ and the fact that  ∞ j=1 Bj = A, one gets µ∗ (E) ≥

∞

µ∗ (E ∩ Bj ) + µ∗ (E ∩ Ac )

j=1 ∗

≥ µ (E ∩ A) + µ∗ (E ∩ Ac ). This completes the proof of part (i). To  prove part (ii), let {Bn }n≥1 ⊂ M and Bi ∩ Bj = ∅ for i = j. Let ∞ Aj = i=j Bi , j ∈ N. Then, by (i), Aj ∈ M for all j ∈ N and so µ∗ (A1 )

µ∗ (A1 ∩ B1 ) + µ∗ (A1 ∩ B1c ) (since B1 ∈ M) = µ∗ (B1 ) + µ∗ (A2 ) = µ∗ (B1 ) + µ∗ (B2 ) + µ∗ (A3 ) (by iteration) n = µ∗ (Bi ) + µ∗ (An+1 ) (by iteration) =

i=1

≥

n

µ∗ (Bi )

for all n ≥ 1.

i=1

Now letting n → ∞, one has µ∗ (A1 ) ≥ µ∗ , the opposite inequality holds and so µ∗ (A1 ) = µ∗

∞ i=1

∞ i=1

 Bi

=

µ∗ (Bi ). By subadditivity of

∞

µ∗ (Bi ),

i=1

proving (ii). As for (iii), note that by monotonicity, µ∗ (A) = 0, B ⊂ A ⇒ µ∗ (B) = 0 and hence, for any E, µ∗ (E ∩ B) = 0. Since µ∗ (E) ≥ µ∗ (E ∩ B c ), this

24

1. Measures

implies µ∗ (E) ≥ µ∗ (E ∩ B c ) + µ∗ (E ∩ B). The opposite inequality holds by 2 the subadditivity of µ∗ . So B ∈ M, and (iii) is proved. Definition 1.3.6: A measure space (Ω, F, ν) is called complete if for any A ∈ F with ν(A) = 0 ⇒ P(A) ⊂ F. Thus, by part (iii) of the above theorem, (Ω, Mµ∗ , µ∗ ) is a complete measure space. Now the above theorem is applied to a µ∗ that is generated by a given measure µ on a semialgebra C via (3.1). Theorem 1.3.3: (Caratheodory’s extension theorem). Let µ be a measure on a semialgebra C and let µ∗ be the set function induced by µ as defined by (3.1). Then, (i) µ∗ is an outer measure, (ii) C ⊂ Mµ∗ , and (iii) µ∗ = µ on C, where Mµ∗ is as in Theorem 1.3.2. Proof: The proof of (i) involves verifying (3.4)–(3.6), which is left as an exercise (Problem 1.20). To prove  (ii), let A ∈ C. Let E ⊂ Ω and {An }n≥1 ⊂ C be such that E ⊂ n≥1 An . Then, for all i ∈ N, Ai = (Ai ∩ A) ∪ (Ai ∩ B1 ) ∪ . . . ∪ (Ai ∩ Bk ) where B1 , . . . , Bk are disjoint sets in k C such that j=1 Bj = Ac . Since µ is finitely additive on C, µ(Ai )

=

µ(Ai ∩ A) +

k

µ(Ai ∩ Bj )

j=1

⇒

∞ n=1

µ(An )

=

∞ n=1 ∗

µ(An ∩ A) +

k ∞

µ(An ∩ Bj )

n=1 j=1 c

≥ µ (E ∩ A) + µ∗ (E ∩ A ), since {An ∩ A}n≥1 and {An ∩ Bj : 1 ≤ j ≤ k, n ≥ 1} are both countable subcollections of C whose unions cover E ∩A and E ∩Ac , respectively. From the definition of µ∗ (E), it now follows that µ∗ (E) ≥ µ∗ (E ∩ A) + µ∗ (E ∩ Ac ). Now the subadditivity of µ∗ completes the proof of part (ii). To prove (iii), let A ∈ C. Then, by definition, µ∗ (A) ≤ µ(A). If µ∗ (A) = of ∞, then µ(A) = ∞ = µ∗ (A). If µ∗ (A) < ∞, then by the definition  ‘infimum,’ for any  > 0, there exists {An }n≥1 ⊂ C such that A ⊂ n≥1 An and ∞ µ∗ (A) ≤ µ(An ) ≤ µ∗ (A) + . n=1

1.3 The extension theorems and Lebesgue-Stieltjes measures

25

  But A = A ∩ ( n≥1 An ) = n≥1 (A ∩ An ). Note that the set function µ ¯ defined in Proposition 1.3.1 is a measure on A(C) and it coincides with µ on C. Since A, A ∩ An ∈ C for all n ≥ 1, by Proposition 1.2.3 (b) applied to µ ¯, µ(A) = µ ¯(A) ≤

∞

µ ¯(A ∩ An ) ≤

n=1

∞

µ ¯(An ) =

n=1

∞

µ(An ) ≤ µ∗ (A) + .

n=1


Alternately,w.l.o.g., assume that {An }n≥1 are disjoint. Then A = A ∩ Since A ∩ An ∈ C for all n, by the countable n≥1 An = n≥1 (A ∩ An ).   ∞ ∞ additivity of µ on C, µ(A) = n=1 µ(A ∩ An ) ≤ n=1 µ(An ) = µ∗ (A) + . 2 Since  > 0 is arbitrary, this yields µ(A) ≤ µ∗ (A). Thus, given a measure µ on a semialgebra C ⊂ P(Ω), there is a complete measure space (Ω, Mµ∗ , µ∗ ) such that Mµ∗ ⊃ C and µ∗ restricted to C equals µ. For this reason, µ∗ is called an extension of µ. The measure space (Ω, Mµ∗ , µ∗ ) is called the Caratheodory extension of µ. Since Mµ∗ is a σalgebra and contains C, Mµ∗ must contain σC, the σ-algebra generated by C, and thus, (Ω, σC, µ∗ ) is also a measure space. However, the latter may not be complete (see Section 1.4). Now the above method is applied to the construction of LebesgueStieltjes measures on R and R2 .

1.3.2

Lebesgue-Stieltjes measures on R

Let F : R → R be nondecreasing. For x ∈ R, let F (x+) ≡ limy↓x F (y), and F (x−) ≡ limy↑x F (y). Set F (∞) = limx↑∞ F (x) and F (−∞) = limx↓−∞ F (y). Let     C ≡ (a, b] : −∞ ≤ a ≤ b < ∞ ∪ (a, ∞) : −∞ ≤ a < ∞ .

(3.7)

Define µF ((a, b])

= F (b+) − F (a+),

µF ((a, ∞))

= F (∞) − F (a+).

(3.8)

Then, it may be verified that (i) C is a semialgebra; (ii) µF is a measure on C. (For (ii), one needs to use the Heine-Borel theorem. See Problems 1.22 and 1.23.) Let (R, Mµ∗F , µ∗F ) be the Caratheodory extension of µF , i.e., the measure space constructed as in the above two theorems.

26

1. Measures

Definition 1.3.7: Let F : R → R be nondecreasing. The (measure) space (R, Mµ∗F , µ∗F ) is called a Lebesgue-Stieltjes measure space and µ∗F is the Lebesgue-Stieltjes measure generated by F . Since σC = B(R), the class of all Borel sets of R, every LebesgueStieltjes measure µ∗F is also a measure on (R, B(R)). Note also that µ∗F is finite on bounded intervals. Conversely, given any Radon measure µ on (R, B(R)), i.e., a measure on (R, B(R)) that is finite on bounded intervals, set ⎧ if x > 0 ⎨ µ((0, x]) 0 if x = 0 F (x) = ⎩ −µ((x, 0]) if x ≤ 0. Then µF = µ on C. By the uniqueness of the extension (discussed later in this section, see also Theorem 1.2.4), it follows that µ∗F coincides with µ on B(R). Thus, every Radon measure on (R, B(R)) is necessarily a LebesgueStieltjes measure. Definition 1.3.8: (Lebesgue Measure on R). When F (x) ≡ x, x ∈ R, the measure µ∗F is called the Lebesgue measure and the σ-algebra Mµ∗F is called the class of Lebesgue measurable sets. The Lebesgue measure will be denoted by m(·) or µL (·). Given below are some important results on m(·). (i) It follows from equation (3.1), that µ∗F ({x}) = F (x+) − F (x−) and hence = 0 if F is continuous at x. Thus m({x}) ≡ 0 on R. (ii) By countable additivity of m(·), m(A) = 0 for any countable set A. (iii) (Cantor set). There exists an uncountable set C such that m(C) = 0. An example is the Cantor set constructed Start with I0 =  as follows:  1 2 [0, 1]. Delete the open middle third, i.e., 3 , 3 . Next from the closed 1 2 intervals I11 = [0,  3 ] and  I12 = [ 3 , 1] delete the open middle thirds,  i.e., 19 , 29 and 79 , 29 , respectively. Repeat this process of deleting the middle third from each of the remaining closed intervals. Thus at stage n there will be 2n−1 new closed intervals and 2n−1 deleted open intervals, each of length 31n . Let Un denote the union of all the deleted open intervals  at the nth stage. Then {Un }n≥1 are disjoint open sets. Let U ≡ n≥1 Un . By countable additivity

m(U ) =

∞ n=1

m(Un ) =

∞ 2n−1 = 1. 3n n=1

Let C ≡ [0, 1] − U . Since U is open and[0,1] is closed, C is nonempty. ∞ It can be shown that C ≡ {x : x = 1 a3ii , ai = 0 or 2} (Problem

1.3 The extension theorems and Lebesgue-Stieltjes measures

27

1.33). Thus C can be mapped in (1–1) manner on to the set of all sequences {δi }i≥1 such that δi = 0 or 2. But this set is uncountable. Since m([0, 1]) = 1, it follows that m(C) = 0. For more properties of the Cantor set, see Rudin (1976) and Chapter 4. (iv) m(·) is invariant under reflection and translation. That is, for any E in B(R), m(−E) = m(E) and m(E + c) = m(E) for all c in R where −E = {−x : x ∈ E} and E + c = {y : y = x + c, x ∈ E}. This follows from Theorem 1.2.4 and the fact that the claim holds for intervals (cf. Problem 2.48). (v) There exists a subset A ⊂ R such that A ∈ Mm . That is, there exists a non-Lebesgue measurable set. The proof of this requires the use of the axiom of choice (cf. A.1). For a proof see Royden (1988).

1.3.3

Lebesgue-Stieltjes measures on R2

Let F : R2 → R satisfy F (a2 , b2 ) − F (a2 , b1 ) − F (a1 , b2 ) + F (a1 , b1 ) ≥ 0,

(3.9)

F (a2 , b2 ) − F (a1 , b1 ) ≥ 0,

(3.10)

and ¯ 2 by appropriate limiting procedure. for all a1 ≤ a2 , b1 ≤ b2 . Extend F to R Let C2 ≡ {I1 × I2 : I1 , I2 ∈ C}, (3.11) where C is as in (3.7). Next, for I1 = (a1 , a2 ], I2 = (b1 , b2 ], − ∞ < a1 , a2 , b1 , b2 < ∞, set µF (I1 × I2 ) ≡ F (a2 +, b2 +) − F (a2 +, b1 +) − (F (a1 +, b2 +) + F (a1 +, b1 +)), (3.12) where for any a, b ∈ R, F (a+, b+) is defined as F (a+, b+) ≡

lim

a
↓a,b
↓b

F (a , b ).

Note that by (3.10), the limit exists and hence, F (a+, b+) is well defined. Further, by (3.9), the right side of (3.12) is nonnegative. Next extend the definition of µF to unbounded sets in C2 by the limiting procedure:
 (3.13) µF (I1 × I2 ) = lim µF (I1 × I2 ) ∩ Jn , n→∞

where Jn = (−n, n]×(−n, n]. Then it may be verified (Problems 1.24, 1.25) that

28

1. Measures

(i) C2 is a semialgebra (ii) µF is a measure on C2 . Let (R2 , Mµ∗F , µ∗F ) be the measure space constructed as in the above two theorems. The measure µ∗F is called the Lebesgue-Stieltjes measure generated by F and Mµ∗F a Lebesgue-Stieltjes σ-algebra. Again, in this case, Mµ∗ includes the σ-algebra σC ≡ B(R2 ) and so (R2 , B(R2 ), µ∗F ) is also a measure space. If F (a, b) = ab, then µF is called the Lebesgue measure on R2 . A similar construction holds for any Rk , k < ∞.

1.3.4

More on extension of measures

Next the uniqueness of the extension µ∗ and approximation of the µ∗ measure of a set in Mµ∗ by that of a set from the algebra A ≡ A(C) are considered. As in the case of measures defined on an algebra, a measure µ on a semialgebra C ⊂ P(Ω) is said to be σ-finite if there exists a countable collection {An }n≥1 ⊂ C such that (i) µ(An ) < ∞ for each n ≥ 1 and (ii)  n≥1 An = Ω. The following approximation result holds. Theorem 1.3.4: Let A ∈ Mµ∗ and µ∗ (A) < ∞. Then, for each  > 0, there exist B1 , B2 , . . . , Bk ∈ C, k < ∞ with Bi ∩ Bj = ∅ for 1 ≤ i = j ≤ k, such that

 k

∗ Bj < , µ A j=1

where for any two sets E1 and E2 , E1  E2 is the symmetric difference of E1 and E2 , defined by E1  E2 ≡ (E1 ∩ E2c ) ∪ (E1c ∩ E2 ). Proof: By definition of µ∗ , µ∗ (A)< ∞ implies that for every  > 0, there exist {Bn }n≥1 ⊂ C such that A ⊂ n≥1 Bn and µ∗ (A) ≤

∞

µ(Bn ) ≤ µ∗ (A) + /2 < ∞.

n=1

Since Bn ∈ C, Bnc is a finite union of disjoint sets from C. W.l.o.g., it can be assumed that {Bn }n≥1 are disjoint. (Otherwise, ∞ one can consider the sequence B1 , B2 ∩ B1c , B3 ∩ B2c ∩ B1c , . . ..) Next, n=1µ(Bn ) < ∞ implies ∞ that for every  > 0, there exists k ∈ N such that n=k+1 µ(Bn ) < 2 . k  Since both A and j=1 Bj are subsets of n≥1 Bn , 









 k ∞ ∞ ∗ c ∗ µ A Bj Bj Bj . ≤µ A ∩ +µ ∗

j=1

j=1

j=k+1

1.3 The extension theorems and Lebesgue-Stieltjes measures

29

 But since µ∗ is a measure on (Ω, Mµ∗ ), µ∗ ( n≥1 Bn ) = µ∗ (A) + µ∗ (( n≥1 Bn ) ∩ Ac ). Further, since µ∗ (A) < ∞, µ∗



  Bn ∩ Ac

n≥1

= µ∗



 Bn − µ∗ (A))

n≥1

≤ =

∞ n=1 ∞

µ∗ (Bn ) − µ∗ (A), µ(Bn ) − µ∗ (A),

(since µ∗ is countably subadditive) (since µ∗ = µ on C)

n=1

<

 2

(by the choice of {Bn }n≥1 ).

Also, by the definition of k, ∗

µ



∞ j=k+1

 Bj

≤

∞ j=k+1

µ∗ (Bj ) =

∞ j=k+1

µ(Bj ) <

 . 2

k Thus, µ∗ (A  [ j=1 Bj ]) < . This completes the proof of the theorem. 2 Thus, every µ∗ -measurable set of finite measure is nearly a finite union of disjoint elements from the semialgebra C. This was enunciated by J.E. Littlewood as the first of his three principles of approximation (the other two being: every Lebesgue measurable function is nearly continuous (cf. Theorem 2.5.12) and every almost everywhere convergent sequence of functions on a finite measure space is nearly uniformly convergent (Egorov’s theorem) cf. Theorem 2.5.11). One may strengthen Theorem 1.3.4 to prove the following result on regu
larity of Radon measures on Rk , B(Rk ) (cf. Problem 1.32). See also Rudin (1987), Chapter 2. Corollary 1.3.5: (Regularity of measures). Let µ be a Radon measure on (Rk , B(Rk )) for some k ∈ N, i.e., µ(A) < ∞ for all bounded sets A ∈ B(Rk ). Let A ∈ B(Rk ) be such that µ(A) < ∞. Then, for each  > 0, there exist a compact set K and an open set G such that K ⊂ A ⊂ G and µ(G \ K) < . The following uniqueness result can be established by an application of the above approximation theorem (Theorem 1.3.4) or by applying the π-λ theorem (Corollary 1.1.3), as in Theorem 1.2.4 (Problem 1.26). Theorem 1.3.6: (Uniqueness). Let µ be a σ-finite measure on a semialgebra C. Let ν be a measure on the measurable space (Ω, σC) such that ν = µ on C. Then ν = µ∗ on σC.

30

1. Measures

An application of Theorem 1.3.4 yields an useful approximation result known as Lusin’s theorem (see Theorem 2.1.3 in Section 2.1) for approximating Borel measurable functions by continuous functions.

1.4 Completeness of measures Recall from Definition 1.3.6 that a measure space (Ω, F, µ) is called complete if for any A ∈ F, µ(A) = 0, B ⊂ A ⇒ B ∈ F. That is, for any set A in F whose µ measure is zero, all subsets of a set A are also in F. For example, the very construction of the Lebesgue-Stieltjes measure for a nondecreasing F on R, discussed in Section 1.3, yields a complete measure space (R, Mµ∗F , µ∗F ). The Borel σ-algebra B(R) is a sub-σ-algebra of Mµ∗F and (R, B(R), µ∗F ) need not be complete. For example, if µF is the Lebesgue measure, then the Cantor set C (Section 1.3.2) has measure 0 and hence M ≡ the Lebesgue σ-algebra contains the power set of C and hence has cardinality larger than that of R. It can be shown that the cardinality of B(R) is the same as that of R (see Hewitt and Stromberg (1965)). For another example, if F is a degenerate distribution at 0, i.e., F (x) = 0 for x < 0 and F (x) = 1 for x ≥ 0, then Mµ∗F = P(R), the power set of R (Problem 1.28), and hence (R, B(R), µF ) is not complete. The same is true for any discrete distribution function. However, it is always possible to complete an incomplete measure space (Ω, F, µ) by adding new sets to F. This procedure is discussed below. Theorem 1.4.1: Let (Ω, F, µ) be a measure space. Let F˜ ≡ {A : B1 ⊂ A ⊂ ˜ set B2 for some B1 , B2 ∈ F satisfying µ(B2 \ B1 ) = 0}. For any A ∈ F, µ ˜(A) = µ(B1 ) = µ(B2 ) for any pair of sets B1 , B2 ∈ F with B1 ⊂ A ⊂ B2 and µ(B2 \ B1 ) = 0. Then ˜ (i) F˜ is a σ-algebra and F ⊂ F, (ii) µ ˜ is well defined, ˜ µ (iii) (Ω, F, ˜) is a complete measure space and µ ˜ = µ on F. Proof: ˜ there exist B1 , B2 ∈ F, B1 ⊂ A ⊂ B2 and µ(B2 \B1 ) = (i) Since A ∈ F, = µ(B2 \ 0. Clearly, B2c ⊂ Ac ⊂ B1c , and B1c , B2c ∈ F and µ(B1c \ B2c )  ˜ Next, let {An }∞ ⊂ F˜ and A = B1 ) = 0 and so Ac ∈ F. n=1 n≥1 An . ⊂ Then, for each n there exist B1n and B2n in F such that B1n  An ⊂ B2n and µ(B2n \ B1n ) = 0. Let B1 = n≥1 B1n andB2 = n≥1 B2n . Then B1 ⊂ A ⊂ B2 , B1 , B2 ∈ F and B2 \ B1 ⊂ n≥1 (B2n \ B1n ) ∞ and hence µ(B2 \ B1 ) ≤ n=1 µ(B2n \ B1n ) = 0. Thus, A ∈ F˜ and hence F˜ is a σ-algebra. Clearly, F ⊂ F˜ since for every A ∈ F, one may take B1 = B2 = A.

1.5 Problems

31

(ii) Let B1 ⊂ A ⊂ B2 , B1 ⊂ A ⊂ B2 , B1 , B1 , B2 , B2 ∈ F and µ(B2 \ B1 ) = 0 = µ(B2 \ B1 ). Then B1 ∪ B1 ⊂ A ⊂ B2 ∩ B2 and (B2 ∩ B2 ) \
(B1 ∪ B1 ) = (B2 ∩ B2 ) ∩ (B1c ∩ B1c ) ⊂ B2 ∩ B1c . Thus µ([B2 ∩ B2 ] \ [B1 ∪ B1 ]) = 0. Hence, µ(B2 ) = µ(B1 ) + µ(B2 \ B1 ) = µ(B1 ) ≤ µ(B1 ∪ B1 ) = µ(B2 ∩ B2 ) ≤ µ(B2 ). By symmetry µ(B2 ) ≤ µ(B2 ) and so µ(B2 ) = µ(B2 ). But µ(B2 ) = µ(B1 ) and µ(B2 ) = µ(B1 ) and also all four quantities agree. (iii) It remains to show that µ ˜ is countably additive and complete on ˜ Let {An }∞ be a disjoint sequence of sets from F˜ and let A = F. n=1  n≥1 An . Let {B1n }n≥1 , {B2n }n≥1 , B1 , B2 be as in the proof of (i). ∞ ∞ Then, the fact that {An } n=1 are disjoint implies {B1n }n=1 are also disjoint. And since B1 = n≥1 B1n and µ is a measure on (Ω, F), µ(B1 ) ≡

∞

(B1n ).

n=1

˜(An ) for all n ≥ 1, and by Also, by definition of B1n ’s, µ(B1n ) = µ (i), µ ˜(A) = µ(B1 ). Thus, µ ˜(A) = µ(B1 ) =

∞ n=1

(B1n ) =

∞

µ ˜(An ),

n=1

establishing the countable additivity of µ ˜. ˜ Next, suppose that A ∈ F and µ ˜(A) = 0. Then there exist B1 , B2 ∈ F such that B1 ⊂ A ⊂ B2 and µ(B2 \ B1 ) = 0. Further, by definition of ˜(A) = 0. If D ⊂ A, then ∅ ⊂ D ⊂ B2 and µ(B2 \ ∅) = 0. µ ˜, µ(B2 ) = µ ˜ µ Therefore, D ∈ F˜ and hence (Ω, F, ˜) is complete. Finally, if A ∈ F, then take B1 = B2 = A and so, µ ˜(A) = µ(B1 ) = µ(A), and thus, µ ˜ = µ on F. Hence, the proof of the theorem is complete. 2

1.5 Problems 1.1 Let Ω be a nonempty set. Show that F ≡ {Ω, ∅} and G = P(Ω) ≡ {A : A ⊂ Ω} are both σ-algebras. 1.2 Let Ω be a finite set, i.e., the number of elements in Ω is finite. Let F ⊂ P(Ω) be an algebra. Show that F is a σ-algebra. 1.3 Show that F6 in Example 1.1.4 is a σ-algebra.

32

1. Measures

1.4 Let  {Fθ : θ ∈ Θ} be a family of σ-algebras on Ω. Show that G ≡ θ∈Θ Fθ is also a σ-algebra. 1.5 Let Ω = {1, 2, 3}, F1 = {{1}, {2, 3}, Ω, ∅}, F2 = {{1, 2}, {3}, Ω, ∅}. Verify that F1 and F2 are both algebras (in fact, σ-algebras) but F1 ∪ F2 is not an algebra. 1.6 Let Ω be a nonempty set and let A≡ {Ai : i ∈ N} be a partition  of Ω, i.e., Ai ∩ Aj = ∅ for all i =j and n≥1 An = Ω. Let F = { i∈J Ai : J ⊂ N} where, for J = ∅, i∈J Ai ≡ ∅. Show that F is a σ-algebra. 1.7 Let Ω be a nonempty set and let B ≡ {Bi : 1 ≤ i ≤ k < ∞} ⊂ P(Ω), B not necessarily a partition. Find σB. k (Hint: For each δ = (δ1 , δ2 , . . . , δk ), δi ∈ {0, 1} let Bδ = i=1 Bi (δi ), whereBi (0) = Bic and Bi (1) = Bi , i ≥ 1. Show that σ(B) = {E : E = δ∈J Bδ , J ⊂ {1, 2, . . . , k}}.) 1.8 Let Ω ≡ {1, 2, . . .} = N and Ai ≡ {j : j ∈ N, j ≥ i}, i ∈ N. Show that σA = P(Ω) where A = {Ai : i ∈ N}. 1.9 (a) Show that every open set A ⊂ R is a countable union of open intervals. (Hint: Use the fact that the set Q of all rational numbers is dense in R.) (b) Extend the above to Rk for any k ∈ N. (c) Strengthen (a) to assert that the open intervals in (a) can be chosen to be disjoint. 1.10 Show that in Example 1.1.6, Oj ⊂ σOi  for all 1 ≤ i, j ≤ 4. 1.11 For k ∈ N, let O5 ≡ {(a1 , b1 ] × . . . × (ak , bk ] : −∞ < ai < bi < ∞, 1 ≤ i ≤ k} and O6 ≡ {[a1 , b1 ] × . . . × [ak , bk ] : −∞ < ai < bi < ∞, 1 ≤ i ≤ k}. Show that σO5  = σO6  = B(Rk ). 1.12 Let OS ≡ {{x} : x ∈ Rk } be the class of all singletons in Rk , k ∈ N. Show that σOS  is properly contained in B(Rk ). (Hint: Show that σOS  coincides with F6 of Example 1.1.4). ¯ ≡ R ∪ {+∞} ∪ {−∞} be the extended real line. The Borel 1.13 Let R ¯ denoted by B(R), ¯ is defined as the σ-algebra on R ¯ σ-algebra on R, generated by the collection B(R) ∪ {∞} ∪ {−∞}.   ¯ = A ∪ B : A ∈ B(R), B ⊂ {−∞, ∞} . (a) Show that B(R) ¯ (b) Show, however, that the  σ-algebra on R generated by B(R) is given by σB(R) = A ∪ B : A ∈ B(R), B = {−∞, ∞} or B=∅ .

1.5 Problems

33

1.14 Let A1 , A2 and A3 denote, respectively, the class of triangles, discs, and pentagons in R2 . Show that σAi  ≡ B(R2 ). Thus, the σ-algebra B(R2 ) (and similarly B(Rk )) can be generated by starting with various classes of sets of different shapes and geometry. 1.15 Let Ω be a nonempty set and B be a σ-algebra on Ω. Let A ⊂ Ω and BA ≡ {B ∩ A : B ∈ B}. Show that BA is a σ-algebra on A. The σ-algebra BA is called the trace σ-algebra of B on A. 1.16 Let Ω = R and F be the collection of all finite unions of disjoint intervals of the form (a, b] ∩ R, −∞ ≤ a < b ≤ ∞. Show that F is an algebra but not a σ-algebra. 1.17 Let Ω be a nonempty set and {Ai }i∈N be a sequence of subsets of Ω such that Ai+1 ⊂ Ai for all i ∈ N. Verify that A = {Ai : i ∈ N} is a π-system and also determine λA, the λ-system generated by A. 1.18 Let Ω ≡ N, F = P(Ω), and An = {j : j ∈ N, j ≥ n}, n ∈ N. Let µ be the counting measure on (Ω, F). Verify that limn→∞ µ(An ) = µ( n≥1 An ). 1.19 Let Ω be a nonempty set and let C ⊂ P(Ω) be a semialgebra. Let k  

Bi : Bi ∈ C, i = 1, 2, . . . , k, k ∈ N . A(C) ≡ A : A = i=1

(a) Show that A(C) is the smallest algebra containing C. (b) Show also that σC = σA(C). 1.20 Let µ∗ be as in (3.1) of Section 1.3. Verify (3.4)–(3.6). ∈ N, then find, for (Hint: Fix 0 <  < ∞. If µ∗ (An ) < ∞ for all n  ∞ each n, a cover {Anj }j≥1 ⊂ C such that µ∗ (An ) ≤ j=1 µ(Anj )+ 2n .) 1.21 Prove Proposition 1.3.1. 1.22 Let F : R → R be nondecreasing.  Let (a, b], (an , bn ], n ∈ N be intervals in R such that (a, b] = n≥1 (an , bn ] and {(an , bn ] : n ≥ 1} are disjoint. Let µF (·) be as in (3.8). Show that µF ((a, b]) =  ∞ n=1 µF ((an , bn ]) by completing the following steps: (a) Let G(x) ≡ F (x+) for all x ∈ R and let G(±∞) = F (±∞). Verify that G(·) is nondecreasing and right continuous on R and that for any A in C, µF (A) = µG (A). (b) In view of (a), assume w.l.o.g. that F (·) is right continuous. Show that for any k ∈ N, F (b) − F (a) ≥

k i=1

(F (bi ) − F (ai )),

34

1. Measures

and conclude that F (b) − F (a) ≥

∞

(F (bi ) − F (ai )).

i=1

(c) To prove the reverse inequality, fix η > 0. Choose c > a and dn > bn , n ≥ 1 such that such that F (c) − F (a) < η, [c, b] ⊂  n n≥1 (an , dn ) and F (dn ) − F (bn ) < η/2 for all n ∈ N. Next, apply the Heine-Borel theorem to the interval [c, b] and the open cover {(an , dn )}n≥1 and extract a finite cover {(ai , di )}ki=1 for [c, b]. W.l.o.g., assume that c ∈ (a1 , d1 ) and b ∈ (ak , dk ). Now verify that F (b) − F (a) ≤ ≤

k j=1 ∞

(F (bj ) − F (aj )) + 2η (F (bj ) − F (aj )) + 2η.

j=1

1.23 Extend the above arguments to the case when (a, b] and (ai , bi ], i ≥ 1 are not necessarily bounded intervals. 1.24 Verify that C2 , defined in (3.11), is a semialgebra. 1.25 (a) Verify that the limit in (3.13) exists. (b) Extend the arguments in Problems 1.22 and 1.23 to verify that µF of (3.12) and (3.13) is a measure on C2 . 1.26 Establish Theorem 1.3.6 by completing the following: (a) Suppose first that ν(Ω) < ∞. Verify that L ≡ {A : A ∈ σC, µ∗ (A) = ν(A)} is a λ system and use the π-λ theorem. (b) Extend the above to the σ-finite case. 1.27 Prove Corollary 1.3.5 for Lebesgue measure m(·). 1.28 Let F be a discrete distribution function, i.e., F is of the form F (x) ≡

∞

aj I(xj ≤ x),

x ∈ R,

j=1

where 0 < aj < ∞, P(R).

 j≥1

aj = 1, xj ∈ R, j ≥ 1. Show that Mµ∗F =

(Hint: Show that µ∗F (Ac ) = 0, where A ≡ {xj : j ≥ 1}, and use the fact that for any B ⊂ R, B ∩ A ∈ B(R).)

1.5 Problems

1.29 Let

⎧ ⎨ 0 x F (x) = ⎩ 1

35

for x < 0 for 0 ≤ x ≤ 1 for x > 0.

Show that Mµ∗F ≡ {A : A ∈ P(R), A ∩ [0, 1] ∈ M}, where M is the σ-algebra of Lebesgue measurable sets as in Definition 1.3.7. 1.30 Let F (·) = 12 Φ(·)+ 12 FP (·) where Φ(·) is the standard normal cdf, i.e., x ∞ k (k) 2 Φ(x) ≡ −∞ √12π e−u /2 Au and FP (x) ≡ k=0 e−2 2k! I(−∞,x] , x ∈ R. Let F1 = Φ, F2 = FP and F3 = F . Let A1 = (0, 1), A2 = {x : x ∈ R, sin x ∈ (0, 12 )}, A3 = {x : for some integers a0 , a1 , . . . , ak , k < ∞, k i i=0 ai x = 0}, the set of all algebraic numbers. Compute µFi (Aj ), 1 ≤ i, j ≤ 3. 1.31 Let µ be a measure on a semialgebra C ⊂ P(Ω) where Ω is a nonempty set. Let µ∗ be the outer measure generated by µ and let Mµ∗ be the σ-algebra of µ∗ -measurable sets as defined in Theorem 1.3.3. (a) Show that for all A ⊂ Ω, there exists a B ∈ σC such that A ⊂ B and µ∗ (A) = µ∗ (B). (Hint: If µ∗ (A) = ∞, take B to be Ω. If µ∗ (A) < ∞, use the definition of µ∗ to show that for each n ≥ 1, there exists {B } ⊂ C such that A ⊂ Bn ≡ j≥1 Bnj , µ∗ (A) ≤  ∞ nj j≥1 1 ∗ j=1 µ(Bnj ) < µ (A) + n . Take B = n≥1 Bn .) (b) Show that for all A ∈ Mµ∗ with µ∗ (A) < ∞, there exists B ∈ σC such that A ⊂ B and µ∗ (B \ A) = 0. (Hint: Use (a) and the relation B = A ∪ (B \ A) with A and B \ A = B ∩ Ac in Mµ∗ .) (c) Show that if µ is σ-finite (i.e., there  exist sets Ωn , n ≥ 1 in C with µ(Ωn ) < ∞ for all n ≥ 1 and n≥1 Ωn = Ω), then in (b), the hypothesis that µ∗ (A) < ∞ can be dropped. (Hint: Assume w.l.o.g. that {Ωn }n≥1 are disjoint. Apply (b) to {An ≡ A ∩ Ωn }n≥1 .) (d) Show that if µ is σ-finite, then A ∈ Mµ∗ iff there exist sets B1 , B2 ∈ σC such that B1 ⊂ A ⊂ B2 and µ∗ (B2 \ B1 ) = 0. (Hint: Apply (c) to both A and Ac .) This shows that Mµ∗ is the completion of σC w.r.t. µ∗ . 1.32 (An outline of a proof of Corollary 1.3.5). Let (R, Mµ∗F , µ∗F ) be a Lebesgue Stieltjes measure space generated by a right continuous and nondecreasing function F : R → R.

36

1. Measures

(a) Show that A ∈ Mµ∗F iff there exist Borel sets B1 and B2 ∈ B(R) such that B1 ⊂ A ⊂ B2 and µ∗F (B2 \ B1 ) = 0. (Hint: Take C to be the semialgebra C = {(a, b] : −∞ ≤ a ≤ b < ∞} ∪ {(b, ∞) : −∞ ≤ b < ∞} and apply Problem 1.31 (d).) (b) Let A ∈ Mµ∗F with µ∗F (A) < ∞. Show that for any  > 0, there exist a finite number of bounded open intervals Ij , j = 1, 2, . . . , k k such that µ∗F (A  j=1 Ij ) < . (Outline: Claim: For any B ∈ C with µF (B) < ∞, there exists an open interval I such that µ∗F (I  B) < . To see this, note that if B = (a, b], − ∞ ≤ a < b < ∞, then one may choose b > b such that F (b ) − F (b) < . Now, with I = (a, b ), µ∗F (I  B) = µ∗F ((b, b )) = µF ((b, b )) ≤ F (b ) − F (b) < . If B = (b, ∞) and µF (B) < ∞, then there exists b > b such that F (∞) − F (b −) < . Hence, with I = (b, b ), µ∗F (I  B) = µ∗F ([b , ∞)) = F (∞) − F (b −) < . This proves the claim. Next, By Theorem 1.3.4, for all  > 0, there exist k B1 , B2 , . . . , Bk ∈ C such that µ∗F (A  j=1 Bj ) < /2. For each Bj , find Ij , a bounded open interval such that µ∗F (Bj Ij ) < 2j . Since (A1 ∪ A2 )  (C1 ∪ C2 ) ⊂ (A1  C1 ) ∪ (A2  C2 ) for any A1 , A2 , C1 , C2 ⊂ Ω, it follows that µ∗F

k 

k k  

  Bj  Ij < µ∗F (Bj  Ij ) < . 2 j=1 j=1 j=1

k Hence, µ∗F (A  [ j=1 Ij ]) < .) (c) Let A ∈ Mµ∗F with µ∗F (A) < ∞. Show that for every  > 0, there exists an open set O such that A ⊂ O and µ∗F (O \ A) < .  > 0, there ex(Hint: By definition of µ∗F , for every  ∗ B ist {Bj }j≥1 ⊂ C such that A ⊂ j≥1 j and µF (A) ≤ ∞ ∗ j=1 µF (Bj ) ≤ µF (A) + . Now as in (b), there exist open ∗ j intervals Ij such that Bj ⊂ Ij and ∞ µF (Ij \ Bj ) < /2∗ for all ∞ j ≥ 1. Then A ⊂ j=1 Bj ⊂ j=1 Ij ≡ O. Also, µF (O) = ∗ ∗ ∗ µ∗F (O \ A) ⇒ µ µ∗F (A) +  F (O \ A) = µF (O) − µ (A) < 2 (since ∞ ∞ ∗ ∗ ∗ ∗ µ (O) ≤ j=1 µ (Ij ) = j=1 µF (Bj ) +  ≤ µF (A) + 2 < ∞).) (d) Extend (c) to all A ∈ Mµ∗F . (Hint: Let Ai = A ∩ [i, i + 1], i ∈ Z. Apply (c) to Ai with  and take unions.) i = 2|i|+1 (e) Show that for all A ∈ Mµ∗F and for all  > 0, there exist a closed set C and an open set O such that C ⊂ A ⊂ O and

1.5 Problems

37

µ∗F (O \ A) ≤ , µ∗ (A \ C) < . (Hint: Apply (d) to A and Ac .) (f) Show that for all A ∈ Mµ∗F with µ∗F (A) < ∞ and for all  > 0, there exist a closed and bounded set F ⊂ A such that µ∗F (A \ F ) <  and an open set O with A ⊂ O such that µ∗F (O \ A) < . (Hint: Apply (d) to A ∩ [−M, M ] where M is chosen so that µ∗F (A ∩ [−M, M ]c ) < . Why is this possible?) Remark: Thus for all A ∈ Mµ∗F with µ∗F (A) < ∞ and for all  > 0, there exist a compact set K ⊂ A and an open set O ⊃ A such that µ∗F (A \ K) <  and µ∗F (O \ A) < . The first property is called inner regularity of µ∗F and the second property is called outer regularity of µ∗F . (g) Show that for all A ∈ Mµ∗F with µ∗F (A) < ∞ and for all  > 0, there exists a continuous function g with compact support (i.e., g (x) is zero for |x| large) such that µ∗F (A  g−1 {1}) < . (Hint: For any bounded open interval (a, b), let η > 0 be such that µF ((a, a + η]) + µF ([b − η, b)) < . Next define ⎧ if a + η ≤ x ≤ b − η ⎨ 1 0 if x ∈ (a, b) g (x) = ⎩ linear over [a, a + η] ∪ [b − η, b]. Then g is continuous with compact support. Also, g −1 {1} = [a + η, b − η] and (a, b)  g −1 {1} = (a, a + η) ∪ (b − η, b). So µF {(a, b)  g−1 (1)}| < , proving the claim for A = (a, b). The general case follows from (b).) (h) Show that for all A ∈ Mµ∗F and for all  > 0, there exists a continuous function g (not necessarily with compact support) such that µ∗F (A  g−1 {1}) <  (i.e., drop the condition µ∗F (A) < ∞). (Hint: Let Ak = A ∩ [k, k
+ 1], k ∈ Z.  Find gk : R → R continuous with supportin k − 18 , k + 98 such that µ∗F (IAk =  . Let g = gk ) < 2|k|+1 k∈Z gk . Note that for any x ∈ R, at (x) = 0 and so g is continuous. Also, µ∗F (IA = g) ≤ most two g k  ∗ k∈Z µF (IAk = gk ) < .) 1.33 Let C be the Cantor set in [0,1] as defined in Section 1.3.2. (a) Show that ∞   ai C= x:x= , a ∈ {0, 2} . i 3i i=1

38

1. Measures

and hence that C is uncountable. (b) Show that C + C ≡ {x + y : x, y ∈ C} = [0, 2].

2 Integration

2.1 Measurable transformations Oftentimes, one is not interested in the full details of a measure space (Ω, F, µ) but only in certain functions defined on Ω. For example, if Ω represents the outcomes of 10 tosses of a fair coin, one may only be interested in knowing the number of heads in the 10 tosses. It turns out that to assign measures (probabilities) to sets (events) involving such functions, one can allow only certain functions (called measurable functions) that satisfy some ‘natural’ restrictions, specified in the following definitions. Definition 2.1.1: Let Ω be a nonempty set and let F be a σ-algebra on Ω. Then the pair (Ω, F) is called a measurable space. If µ is a measure on (Ω, F), then the triplet (Ω, F, µ) is called a measure space. If in addition, µ is a probability measure, then (Ω, F, µ) is called a probability space. Definition 2.1.2: (a) Let (Ω, F) be a measurable space. Then a function f : Ω to R is called F, B(R)-measurable (or F-measurable) if for each a in R
 (1.1) f −1 (−∞, a] ≡ {ω : f (ω) ≤ a} ∈ F. (b) Let (Ω, F, P ) be a probability space. Then a function X : Ω → R is called a random variable, if the event X −1 ((−∞, a]) ≡ {ω : X(ω) ≤ a} ∈ F

40

2. Integration

for each a in R, i.e., a random variable is a real valued F-measurable function on a probability space (Ω, F, P ). It will be shown later that condition (1.1) on f is equivalent to the stronger condition that f −1 (A) ∈ F for all Borel sets A ∈ B(R). Since for any Borel set A ∈ B(R), f −1 (A) is a member of the underlying σ-algebra F, one can assign a measure to the set f −1 (A) using a measure µ on (Ω, F). Note that for an arbitrary function T from Ω → R, T −1 (A) need not be a member of F and hence such an assignment may not be possible. Thus, condition (1.1) on real valued mappings is a ‘natural’ requirement while dealing with measure spaces. The following definition generalizes (1.1) to maps between two measurable spaces. Definition 2.1.3: Let (Ωi , Fi ), i = 1, 2 be measurable spaces. Then, a mapping T : Ω1 → Ω2 is called measurable with respect to the σ-algebras F1 , F2  (or F1 , F2 -measurable) if T −1 (A) ∈ F1

for all A ∈ F2 .

Thus, X is a random variable on a probability space (Ω, F, P ) iff X is F, B(R)-measurable. Some examples of measurable transformations are given below. Example 2.1.1: Let Ω = {a, b, c, d}, F2 = {Ω, ∅, {a}, {b, c, d}} and let F3 = the set of all subsets of Ω. Define the mappings Ti : Ω → Ω, i = 1, 2, by T1 (ω) ≡ a for ω ∈ Ω 

and T2 (ω) =

a if ω = a, b c if ω = c, d.

Then, T1 is F2 , F3 -measurable since for any A ∈ F3 , T1−1 (A) = Ω or ∅ according as a ∈ A or a ∈ A. By similar arguments, it follows that T2 is F3 , F2 -measurable. However, T2 is not F2 , F3 -measurable since T2−1 ({a}) = {a, b} ∈ F2 . As this simple example shows, measurability of a given mapping critically depends on the σ-algebras on its domain and range spaces. In general, if T is F1 , F2 -measurable, then T is F˜1 , F2 -measurable for any σ-algebra F˜1 ⊃ F1 and T is F1 , F˜2 -measurable for any F˜2 ⊂ F2 . Example 2.1.2: Let T : R → R be defined as  sin 2x if x > 0 T (x) = 1 + cos x if x ≤ 0.

2.1 Measurable transformations

41

Is T measurable w.r.t. the Borel σ-algebras B(R), B(R)? If one is to apply the definition directly, one must check that T −1 (A) ∈ B(R) for all A ∈ B(R). However, finding T −1 (A) for all Borel sets A is not an easy task. In many instances like this one, verification of the measurability property of a given mapping by directly using the definition can be difficult. In such situations, one may use some easy-to-verify sufficient conditions. Some results of this type are given below. Proposition 2.1.1: Let (Ωi , Fi ), i = 1, 2, 3 be measurable spaces. (i) Suppose that F2 = σA for some class of subsets A of Ω2 . If T : Ω1 → Ω2 is such that T −1 (A) ∈ F1 for all A ∈ A, then T is F1 , F2 measurable. (ii) Suppose that T1 : Ω1 → Ω2 is F1 , F2 -measurable and T2 : Ω2 → Ω3 is F2 , F3 -measurable. Let T = T2 ◦ T1 : Ω1 → Ω3 denote the composition of T1 and T2 , defined by T (ω1 ) = T2 (T1 (ω1 )), ω1 ∈ Ω1 . Then, T is F1 , F3 -measurable. Proof: (i) Define the collection of sets F = {A ∈ F2 : T −1 (A) ∈ F1 }. Then, (a) T −1 (Ω2 ) = Ω1 ∈ F1 ⇒ Ω2 ∈ F. (b) If A ∈ F, then T −1 (A) ∈ F1 ⇒ (T −1 (A))c ∈ F1 ⇒ T −1 (Ac ) = (T −1 (A))c ∈ F1 , implying Ac ∈ F. −1 (c) If A1 , A2 , . . . , ∈ F, then,  T (Ai ) ∈  F1 for−1all i ≥ 1. Since F1 −1 is a σ-algebra , T ( A ) = (An ) ∈ F1 . Thus, n n≥1 n≥1 T  n≥1 An ∈ F. (See also Problem 2.1 on de Morgan’s laws.)

Hence, by (a), (b), (c), F is a σ-algebra and by hypothesis A ⊂ F . Hence, F2 = σA ⊂ F ⊂ F2 . Thus, F = F2 and T is F1 , F2 measurable. (ii) Let A ∈ F3 . Then, T2−1 (A) ∈ F2 , since T2 is F2 , F3 -measurable. Also, by the F1 , F2 -measurability of T1 , T −1 (A) = T1−1 (T2−1 (A)) ∈ 2 F1 , showing that T is F1 , F3 -measurable. Proposition 2.1.2: For any k, p ∈ N, if f : Rp → Rk is continuous, then f is B(Rp ), B(Rk )-measurable. Proof: Let A = {A : A is an open set in Rk }. Then, by the continuity of f , f −1 (A) is open and hence, is in B(Rp ) (cf. Section A.4). Thus, f −1 (A) ∈

42

2. Integration

Rp for all A ∈ A. Since B(Rk ) = σA, by Proposition 2.1.1 (a), f is 2 B(Rp ), B(Rk )-measurable. Although the converse to the above proposition is not true, a result due to Lusin says that except on a set of small measure, f coincides with a continuous function. This is stronger than the statement that except on set of small measure, f is close to a continuous function. For the statement and proof of Lusin’s theorem, see Theorem 2.5.12. Proposition 2.1.3: Let f1 , . . . , fk (k ∈ N) be F, B(R)-measurable transformations from Ω to R. Then, (i) f = (f1 , . . . , fk ) is F, B(Rk )-measurable. (ii) g = f1 + . . . + fk is F, B(R)-measurable. k (iii) h ≡ i=1 fi is F, B(R)-measurable. (iv) Let p ∈ N and let ψ : Rk → Rp be continuous. Then, ξ ≡ ψ ◦ f is F, B(Rp )-measurable, where f = (f1 , . . . , fk ). Proof: To prove (i), note that for any rectangle R = (a1 , b1 )×. . .×(ak , bk ), f −1 (R)

= =

{ω ∈ Ω : a1 < f1 (ω) < b1 , . . . , ak < fk (ω) < bk } k

{ω ∈ Ω : ai < f1 (ω) < bi }

i=1

=

k

fi−1 (ai , bi ) ∈ F,

i=1

since each fi is F, B(R)-measurable. Hence, by Proposition 2.1.1 (i), f is F, B(Rk )-measurable. To prove (ii), note that the function g1 (x) ≡ x1 + . . . + xk , x = (x1 , . . . , xk ) ∈ Rk is continuous on Rk , and hence, by Proposition 2.1.2, is B(Rk ), B(R)-measurable. Since g = g1 ◦ f , g is F, B(R)-measurable, by Proposition 2.1.1 (ii). The proofs of (iii) and (iv) are similar to that of (ii) and hence, are omitted. 2 Corollary 2.1.4: The collection of F, B(R)-measurable functions from Ω to R is closed under pointwise addition and multiplication as well as under scalar multiplication. The proof of Corollary 2.1.4 is omitted. In view of the above, writing the function T of Example 2.1.2 as T (x) = (sin 2x)I(0,∞) (x) + (1 + cos x)I(−∞,0] (x), x ∈ R, the B(R), B(R)-measurability of T follows. Note that here T is not continuous over R but only piecewise continuous (see also Problem 2.2).

2.1 Measurable transformations

43

Next, measurability of the limit of a sequence of measurable functions is ¯ = R ∪ {+∞, −∞} denote the extended real line and let considered. Let R ¯ ≡ σB(R) ∪ {+∞} ∪ {−∞} denote the extended Borel σ-algebra on B(R) ¯ R. ¯ be a F, B(R)¯ Proposition 2.1.5: For each n ∈ N, let fn : Ω → R measurable function. (i) Then, each of the functions supn∈N fn , inf n∈N fn , lim supn→∞ fn , and ¯ lim inf n→∞ fn is F, B(R)-measurable. (ii) The set A ≡ {ω : limn→∞ fn (ω) exists and is finite} lies in F and ¯ the function h ≡ (limn→∞ fn ) · IA is F, B(R)-measurable. Proof: ¯ (i) Let g = supn≥1 fn . To show that g is F, B(R)-measurable, it is enough to show that {ω : g(ω) ≤ r} ∈ F for all r ∈ R (cf. Problem 2.4). Now, for any r ∈ R, {ω : g(ω) ≤ r} = =

∞ n=1 ∞

{ω : fn (ω) ≤ r} fn−1 ((−∞, r]) ∈ F,

n=1

since fn−1 ((−∞, r]) ∈ F for all n ≥ 1, by the measurability of fn . Next note that inf n≥1 fn = − supn≥1 (−fn ) and hence, inf n≥1 fn is ¯ F, B(R)-measurable. To prove the measurability of lim supn→∞ fn , ¯ define the functions gm = supn≥m fn , m ≥ 1. Then, gm is F, B(R)measurable for each m ≥ 1 and since gm is nonincreasing in m, ¯ inf m≥1 gm ≡ lim supn→∞ fn is also F, B(R)-measurable. A similar argument works for lim inf n→∞ fn . ˜i = (ii) Let h1 = lim supn→∞ fn and h2 = lim inf n→∞ fn , and define h ˜ ˜ hi IR (hi ), i = 1, 2. Note that h1 − h2 is F, B(R)-measurable. Hence, {ω : lim fn (ω) n→∞

=

exists and is finite}

{ω : −∞ < lim sup fn (ω) = lim inf fn (ω) < ∞} n→∞

n→∞

= {ω : −∞ < h2 (ω) = h1 (ω) < ∞} ˜ 1 (ω) = h ˜ 2 (ω)} ∩ {ω : h1 (ω) < ∞, h2 (ω) > −∞} = {ω : h −1 ˜1 − h ˜ 2 ) ({0}) ∩ {ω : h1 (ω) < ∞, h2 (ω) > −∞} ∈ F. = (h Finally, note that h = h1 IA .

2

44

2. Integration

Definition 2.1.4: Let {fλ : λ ∈ Λ} be a family of mappings from Ω1 into Ω2 and let F2 be a σ-algebra on Ω2 . Then, σ{fλ−1 (A) : A ∈ F2 , λ ∈ Λ} is called the σ-algebra generated by {fλ : λ ∈ Λ} (w.r.t. F2 ) and is denoted by σ{fλ : λ ∈ Λ}. Note that σ{fλ : λ ∈ Λ} is the smallest σ-algebra on Ω1 that makes all fλ ’s measurable w.r.t. F2 on Ω2 . Example 2.1.3: Let f = IA for some set A ⊂ Ω1 and Ω2 = R and F2 = B(R). Then, σ{f } = σ{A} = {Ω1 , ∅, A, Ac }. Example 2.1.4: Let Ω1 = Rk , Ω2 = R, F2 = B(R), and for 1 ≤ i ≤ k, let fi : Ω1 → Ω2 be defined as fi (x1 , . . . , xk ) = xi ,

(x1 , . . . , xk ) ∈ Ω1 = Rk .

Then, σ{fi : 1 ≤ i ≤ k} = B(Rk ). To show this, note that any k measurable rectangle A1 × . . . × Ak can be written as A1 × . . . × Ak = i=1 fi−1 (Ai ) and hence A1 × . . . × Ak ∈ σ{fi : 1 ≤ i ≤ k} for all A1 , . . . , Ak ∈ R. Since Rk is generated by the collection of all measurable rectangles, B(Rk ) ⊂ σ{fi : 1 ≤ i ≤ k}. Conversely, for any A ∈ B(R) and for any 1 ≤ i ≤ k, fi−1 (A) = R × . . . × A × . . . × R (with A in the ith position) is in B(Rk ). Therefore, σ{fi : 1 ≤ i ≤ k} = σ{fi−1 (A) : A ∈ R, 1 ≤ i ≤ k} ⊂ B(Rk ). Hence, σ{fi : 1 ≤ i ≤ k} = B(Rk ). Proposition 2.1.6: Let {fλ : λ ∈ Λ} be an uncountable collection of maps from Ω1 to Ω2 . Then for any B ∈ σ{fλ : λ ∈ Λ}, there exists a countable set ΛB ⊂ Λ such that B ∈ σ{fλ : λ ∈ ΛB }. Proof: The proof of this result is left as an exercise (Problem 2.5).

2

2.2 Induced measures, distribution functions Suppose X is a random variable defined on a probability space (Ω, F, P ). Then P governs the probabilities assigned to events like X −1 ([a, b]), −∞ < a < b < ∞. Since X takes values in the real line, one should be able to express such probabilities only as a function of the set [a, b]. Clearly, since X is F, R-measurable, X −1 (A) ∈ F for all A ∈ B(R) and the function PX (A) ≡ P (X −1 (A))

(2.1)

2.2 Induced measures, distribution functions

45

is a set function defined on B(R). Is this a (probability) measure on B(R)? The following proposition answers the question more generally. Proposition 2.2.1: Let (Ωi , Fi ), i = 1, 2 be measurable spaces and let T : Ω1 → Ω2 be a F1 , F2 -measurable mapping from Ω1 to Ω2 . Then, for any measure µ on (Ω1 , F1 ), the set function µT −1 , defined by µT −1 (A) ≡ µ(T −1 (A)), A ∈ F2

(2.2)

is a measure on F2 . Proof: It is easy to check that µT −1 satisfies the three conditions for being a measure. The details are left as an exercise (cf. Problem 2.9). Definition 2.2.1: The measure µT −1 is called the measure induced by T (or the induced measure of T ) on F2 . In particular, if µ(Ω1 ) = 1, then µT −1 (Ω2 ) = 1. Hence, the set function P defined in (2.1) is indeed a probability measure on (R, B(R)). Definition 2.2.2: For a random variable X defined on a probability space (Ω, F, P ), the probability distribution of X (or the law of X), denoted by PX (say), is the induced measure of X under P on R, as defined in (2.2). In introductory courses on probability and statistics, one defines probabilities of events like ‘X ∈ [a, b]’ by using the probability mass function for discrete random variables and the probability density function for ‘continuous’ random variables. The measure-theoretic definition above allows one to treat both these cases as well as the case of ‘mixed’ distributions under a unified framework. Definition 2.2.3: The cumulative distribution function (or cdf in short) of a random variable X is defined as FX (x) ≡ PX ((−∞, x]), x ∈ R.

(2.3)

Proposition 2.2.2: Let F be the cdf of a random variable X. (i) For x1 < x2 , F (x1 ) ≤ F (x2 )

(i.e., F is nondecreasing on R).

(ii) For x in R, F (x) = limy↓x F (y)

(i.e., F is right continuous on R).

(iii) limx→−∞ F (x) = 0 and limx→+∞ F (x) = 1. Proof: For x1 < x2 , (−∞, x1 ] ⊂ (−∞, x2 ]. Since PX is a measure on B(R), F (x1 ) = PX ((−∞, x1 ]) ≤ PX ((−∞, x2 ]) = F (x2 ), proving (i).

46

2. Integration

To prove (ii), let xn ↓ x. Then, the sets (−∞, xn ] ↓ (−∞, x], and PX ((−∞, x1 ]) = P (X ≤ x1 ) ≤ 1. Hence, using the monotone continuity from above of the measure PX (m.c.f.a.) (cf. Proposition 1.2.3), one gets lim F (xn ) = lim PX ((−∞, xn ]) = PX ((−∞, x]) = F (x).

n→∞

n→∞

Next consider part (iii). Note that if xn ↓ −∞ and yn ↑ ∞, then (−∞, xn ] ↓ ∅ and (−∞, yn ] ↑ (−∞, ∞). Hence, part (iii) follows the m.c.f.a. 2 and the m.c f.b. properties of PX (cf. Propositions 1.2.1 and 1.2.3). Definition 2.2.4: A function F : R → R satisfying (i), (ii), and (iii) of Proposition 2.2.2 is called a cumulative distribution function (or cdf for short). Thus, given a random variable X, its cdf FX satisfies properties (i), (ii), (iii) of Proposition 2.2.2. Conversely, given a cdf F , one can construct a probability space (Ω, F, P ) and a random variable X on it such that the cdf of X is F . Indeed, given a cdf F , note that by Theorem 1.3.3 and Definition 1.3.7, there exists a (Lebesgue-Stieltjes) probability measure µF on (R, B(R)) such that µF ((−∞, x]) = F (x). Now define X to be the identity map on R, i.e., let X(x) ≡ x for all x ∈ R. Then, X is a random variable on the probability space (R, B(R), µF ) with probability distribution PX = µF and cdf FX = F . In addition to (i), (ii) and (iii) of Proposition 2.2.2, it is easy to verify that for any x in R, P (X < x) = FX (x−) ≡ lim FX (y) y↑x

and hence P (X = x) = FX (x) − FX (x−).

(2.4)

Thus, the function FX (·) has a jump at x iff P (X = x) > 0. Since a monotone function from R to R can have only jump discontinuities and only a countable number of them (cf. Problem 2.11), for any random variable X, the set {a ∈ R : P (X = a) > 0} is countable. This leads to the following definitions. Definition 2.2.5: (a) A random variable X is called discrete if there exists a countable set A ⊂ R such that P (X ∈ A) = 1. (b) A random variable X is called continuous if P (X = x) = 0 for all x ∈ R. Note that X is continuous iff FX is continuous on all of R and X is discrete iff the sum of all the jumps of its cdf FX is one. It may also be

2.2 Induced measures, distribution functions

47

noted that if FX is a step function, then X is discrete but not conversely. For example, consider the case where the set A in the above definition is the set of all rational numbers. It turns out that a given cdf may be written as a weighted sum of a discrete and a continuous cdfs. Let F be a cdf. Let A ≡ {x : p(x) ≡ F (x) − F (x−) > 0}. As remarked  earlier, A is at most countable. Write α = y∈A p(y) and let F˜d (x) = y∈A p(y)I(−∞,x] (y), and F˜c (x) = F (x) − F˜d (x). It is easy to verify that F˜c (·) is continuous on R. If α = 0, then F (x) = F˜c (x) and F is continuous. If α = 1, then F = F˜d (x) and F is discrete. If 0 < α < 1, F (·) can be written as F (x) = αFd (x) + (1 − α)Fc (x),

(2.5)

where Fd (x) ≡ α−1 F˜d (x) and Fc (x) ≡ (1 − α)−1 F˜c (x) are both cdfs, with Fd being discrete and Fc being continuous. For a further decomposition of Fc (·) into absolutely continuous and singular continuous components, see Chapter 4.

2.2.1

Generalizations to higher dimensions

Induced distributions of random vectors and the associated cdfs are briefly considered in this section. Let X = (X1 , X2 , . . . , Xk ) be a k-dimensional random vector defined on a probability space (Ω, F, P ). The probability distribution PX of X is the induced probability measure on (Rk , B(Rk )), defined by (cf. (2.1)) PX (A) ≡ P (X −1 (A)) A ∈ B(Rk ) .

(2.6)

The cdf FX of X is now defined by FX (x) = P (X ≤ x),

x ∈ Rk ,

(2.7)

where for any x = (x1 , x2 , . . . , xk ) and y = (y1 , y2 , . . . , yk ) in Rk , x ≤ y means that xi ≤ yi for all i = 1, . . . , k. The extension of Proposition 2.2.2 to the k-dimensional case is notationally involved. Here, an analog of Proposition 2.2.2 for the bivariate case, i.e., for k = 2 is stated. Proposition 2.2.3: Let F be the cdf of a bivariate random vector X = (X1 , X2 ). (i) Then, for any x = (x1 , x2 ) ≤ y = (y1 , y2 ), F (y1 , y2 ) − F (x1 , y2 ) − F (y1 , x2 ) + F (x1 , x2 ) ≥ 0. (ii) For any x = (x1 , x2 ) ∈ R2 , lim

y1 ↓x1 ,y2 ↓x2

F (y1 , y2 ) = F (x1 , x2 ),

i.e., F is right continuous on R2 .

(2.8)

48

2. Integration

(iii) limx1 →−∞ F (x1 , a) = limx2 →−∞ F (a, x2 ) = 0 for all a ∈ R; limx1 →∞,x2 →∞ F (x1 , x2 ) = 1. (iv) For any a ∈ R, limy↑∞ F (a, y) = P (X1 ≤ a) and limy↑∞ F (y, a) = P (X2 ≤ a). Proof: Clearly, 0 ≤ P (X ∈ (x1 , y1 ] × (x2 , y2 ]) = P (x1 < X1 ≤ y1 , x2 < X2 ≤ y2 ) = P (X1 ≤ y1 , x2 < X2 ≤ y2 ) − P (X1 ≤ x1 , x2 < X2 ≤ y2 ) =

P (X1 ≤ y1 , X2 ≤ y2 ) − P (X1 ≤ y1 , X2 ≤ x2 ) −[P (X1 ≤ x1 , X2 ≤ y2 ) − P (X1 ≤ x1 , X2 ≤ x2 )]

= F (y1 , y2 ) − F (y1 , x2 ) − F (x1 , y2 ) + F (x1 , x2 ). This proves (i). To prove (ii), note that for any sequence yin ↓ xi , i = 1, 2, the sets An = (−∞, y1n ] × (−∞, y2n ] ↓ A ≡ (−∞, x1 ] × (−∞, x2 ]. Hence, by m.c.f.a property of a probability measure, F (y1n , y2n ) = P (X ∈ An ) ↓ P (A) = F (x1 , x2 ). For (iii), note that (−∞, x1n ] × (−∞, a] ↓ ∅ for any sequence x1n ↓ −∞ and for any a ∈ R. Hence, again by the m.c.f.a. property, F (x1n , a) ↓ 0

as

n → ∞.

By similar arguments, F (a, x2n ) ↓ 0 whenever x2n ↓ −∞. To prove the last relation in (iii), apply the m.c.f.b. property to the sets (−∞, x1n ] × (−∞, x2n ] ↑ R2 for x1n ↑ ∞, x2n ↑ ∞. The proof of part (iv) is similar. 2 Note that any function satisfying properties (i), (ii), (iii) of Proposition 2.2.3 determines a probability measure uniquely. This follows from the discussions in Section 1.3, as (1.3.9) and (1.3.10) follow from (i) and (iii) (Problem 2.12). For a general k ≥ 1, an analog of property (i) above is cumbersome to write down explicitly. Indeed, for any x ≤ y, now a sum involving 2k -terms must be nonnegative. However, properties (ii), (iii), and (iv) can be extended in an obvious way to the k-dimensional case. See Problem 2.13 for a precise statement. Also, for a general k ≥ 1, functions satisfying the properties listed in Problem 2.13 uniquely determine a probability measure on (Rk , B(Rk )).

2.3 Integration Let (Ω, F, µ) be a measure space and f : Ω → R be a measurable function. The goal of this section is to define the integral of f with respect to the

2.3 Integration

49

measure µ and establish some basic convergence results. The integral of a nonnegative function taking only finitely many values is defined first, which is then extended to all nonnegative measurable functions by approximation from below. Finally, the integral of an arbitrary measurable function is defined using the decomposition of the function into its positive and negative parts. ¯ ≡ [−∞, ∞] is called simple Definition 2.3.1: A function f : Ω → R ¯ and sets if there exist a finite set (of distinct elements) {c1 , . . . , ck } ∈ R A1 , . . . , Ak ∈ F, k ∈ N such that f can be written as f=

k

ci IAi .

(3.1)

i=1

Definition 2.3.2: (The integral of a simple nonnegative function). Let ¯ + ≡ [0, ∞] be a simple nonnegative function on (Ω, F, µ) with f :Ω→R  the representation (3.1). The integral of f w.r.t. µ, denoted by f dµ, is defined as  k f dµ ≡ ci µ(Ai ) . (3.2) i=1

Here and in the following, the relation 0·∞=0 is adopted as a convention. It may be verified that the value of the integral in (3.2) does not depend l on the representation of f . That is, if f can be expressed as f = j=1 dj IBj ¯ (not necessarily distinct) and for some sets for some d1 , . . . , dl ∈ R + l k B1 , . . . , Bl ∈ F, then i=1 ci µ(Ai ) = j=1 dj µ(Bj ), so that the value of the integral remains unchanged (Problem 2.17). Also note that for the f in Definition 2.3.2,  0≤

f dµ ≤ ∞.

The following proposition is an easy consequence of the definition and the above remark. Proposition 2.3.1: Let f and g be two simple nonnegative functions on (Ω, F, µ). Then    (i) (Linearity) For α ≥ 0, β ≥ 0, (αf + βg)dµ = α f dµ + β gdµ. (ii) (Monotonicity) If   f ≥ g a.e. (µ), i.e., µ({ω : ω ∈ Ω, f (ω) < g(ω)}) = 0, then f dµ ≥ gdµ.

50

2. Integration

(iii) If f = g a.e. (µ), i.e., if µ({ω : ω ∈ Ω, f (ω) = g(ω)}) = 0, then f dµ = gdµ. Definition 2.3.3: (The integral of a nonnegative measurable function). ¯ + be a nonnegative measurable function on (Ω, F, µ). The Let f : Ω → R  integral of f with respect to µ, also denoted by f dµ, is defined as   f dµ = lim fn dµ, (3.3) n→∞

where {fn }n≥1 is any sequence of nonnegative simple functions such that fn (ω) ↑ f (ω) for all ω.  Note that by Proposition 2.3.1 (ii), the sequence { fn dµ}n≥1 is nondecreasing, and hence the right side of (3.3) is well defined. That the right side of (3.3) is the same for all such approximating sequences of functions needs to be established and is the content of the following proposition. The proof of this proposition exploits in a crucial way the m.c.f.b. and the finite additivity of the set function µ (or, equivalently, the countable additivity of µ). Proposition 2.3.2: Let {fn }n≥1 and {gn }n≥1 be two sequences of simple ¯ + such that as n → ∞, nonnegative measurable functions on (Ω, F, µ) to R fn (ω) ↑ f (ω) and gn (ω) ↑ f (ω) for all ω ∈ Ω. Then   fn dµ = lim gn dµ. (3.4) lim n→∞

n→∞

Proof: Fix N ∈ N and 0 < ρ < 1. It will now be shown that   lim fn dµ ≥ ρ gN dµ. n→∞

(3.5)

k Suppose that gN has the representation gN ≡ i=1 di IBi . Let Dn = {ω ∈ Ω : fn (ω) ≥ ρgN (ω)}, n ≥ 1. Since fn (ω) ↑ f (ω) for all ω, Dn ↑ D ≡ {ω : f (ω) ≥ ρgN (ω)} (Problem 2.18 (b)). Also since gN (ω) ≤ f (ω) and 0 < ρ < 1, D = Ω. Now writing fn = fn IDn + fn IDnc , it follows from Proposition 2.3.1 that    fn dµ ≥ fn IDn dµ ≥ ρ gN IDn dµ = ρ

k

di µ(Bi ∩ Dn ).

(3.6)

i=1

By the m.c.f.b. property, for each  i ∈ N, µ(Bi ∩ Dn ) ↑ µ(Bi ∩ Ω) = µ(Bi ) as n → ∞. Since the sequence { fn dµ}n≥1 is nondecreasing, taking limits

2.3 Integration

in (3.6), yields (3.5). Next, letting ρ ↑ 1 yields limn→∞ for each N ∈ N and hence,   fn dµ ≥ lim gn dµ . lim n→∞



fn dµ ≥



51

gN dµ

n→∞

By symmetry, (3.4) follows and hence, the proof is complete.

2

Remark 2.3.1: It is easy to verify that Proposition 2.3.2 remains valid if {fn }n≥1 and {gn }n≥1 increase to f a.e. (µ). Given a nonnegative measurable function f , one can always construct a nondecreasing sequence {fn }n≥1 of nonnegative simple functions such that fn (ω) ↑ f (ω) for all ω ∈ Ω in the following manner. Let {δn }n≥1 be a sequence of positive real numbers and let {Nn }n≥1 be a sequence of positive integers such that as n → ∞, δn ↓ 0, Nn ↑ ∞ and Nn δn ↑ ∞. Further, suppose that the sequence Pn ≡ {jδn : j = 0, 1, 2, . . . , Nn } is nested, i.e., Pn ⊂ Pn+1 for each n ≥ 1. Now set  if jδn ≤ f (ω) < (j + 1)δn , j = 0, 1, 2, . . . , (Nn − 1) jδn fn (ω) = Nn δn if f (ω) ≥ Nn δn . (3.7) A specific choice of δn and Nn is given by δn = 2−n , Nn = n2n . Thus,with the above choice of {fn }n≥1 in the definition of the Lebesgue integral f dµ in (3.3), the range of f is subdivided into intervals of decreasing lengths. This is in contrast to the definition of the Riemann integral of f over a bounded interval, which is defined via subdividing the domain of f into finer subintervals. Remark 2.3.2: In some cases it may be more appropriate to choose the approximating sequence {fn }n≥1 in a manner different from (3.7). For example, let Ω = {ωi : i ≥ 1} be a countable set, F = P(Ω), the power set of Ω, and let µ be a measure on (Ω, F). Then any function f : Ω → R +∞≡ [0, ∞) is measurable and the integral f dµ coincides with the sum i=1 f (ωi )µ({ωi }) as can be seen by choosing the approximating sequence {fn }n≥1 as  f (ωi ) for i = 1, 2, . . . , n fn (ωi ) = 0 for i > n. Remark 2.3.3: The integral of a nonnegative measurable function can be alternatively defined as    f dµ = sup gdµ : g nonnegative and simple, g ≤ f . The equivalence of this to (3.3) is seen as follows. Clearly the right side above, say, M is greater than or equal to f dµ as in (3.3). Conversely,

52

2. Integration

there exist a sequence {gn }n≥1  of simple nonnegative functions with gn ≤ f for all n ≥ 1 such that limn gn dµ equals the supremum M defined above. Now set hn = max{gj : 1 ≤ j ≤ n}, n ≥ 1. Now it can be verified that for each n ≥ 1, hn is nonnegative, simple, and satisfies hn ↑ f and also that  hn dµ converges to M (Problem 2.19 (b)). Corollary 2.3.3: Let f and g be two nonnegative measurable functions on (Ω, F, µ). Then, the conclusions of Proposition 2.3.1 remain valid for such f and g. Proof: This follows from Proposition 2.3.1 for nonnegative simple functions and Definition 2.3.3. 2  The definition of the integral f dµ of a nonnegative measurable function f in (3.3) makes it possible to interchange limits and integration in a fairly routine manner. In particular, the following key result is a direct consequence of the definition. Theorem 2.3.4: (The monotone convergence theorem or MCT). Let {fn }n≥1 and f be nonnegative measurable functions on (Ω, F, µ) such that fn ↑ f a.e. (µ). Then   fn dµ. (3.8) f dµ = lim n→∞

Remark 2.3.4: The important difference between (3.4) and (3.8) is that in (3.8), the fn ’s need not be simple. Proof: It is similar to the proof of Proposition 2.3.2. Let {gn }n≥1 be a sequence of nonnegative simple functions on (Ω, F, µ) such that gn (ω) ↑ f (ω) for all ω. By hypothesis, there exists a set A ∈ F such that µ(Ac ) = 0 and for ω in A, fn (ω) ↑ f (ω). Fix k ∈ N and 0 < ρ < 1. Let Dn = {ω : ω ∈ A, fn (ω) ≥ ρgk (ω)}, n ≥ 1. Then, Dn ↑ D ≡ {ω : ω ∈ A, f (ω) ≥ ρgk (ω)}. Since gk (ω) ≤ f (ω) for all ω, it follows that D = A. Now, by Corollary 2.3.3,    fn dµ ≥ fn IDn dµ ≥ ρ gk IDn dµ for all n ≥ 1. By m.c.f.b.,



 gk IA dµ = gk dµ as n → ∞, yielding   lim inf fn dµ ≥ ρ gk dµ

gk IDn dµ ↑



n→∞

for all 0 < ρ < 1 and all k ∈ N. Letting ρ ↑ 1 first and then k ↑ ∞, from (3.3) one gets   lim inf fn dµ ≥ f dµ. n→∞

2.3 Integration

By monotonicity (Corollary 2.3.3),   fn dµ ≤ f dµ

53

for all n ≥ 1 2

and so the proof is complete.

Corollary 2.3.5: Let {hn }n≥1 be a sequence of nonnegative measurable functions on a measure space (Ω, F, µ). Then    ∞ ∞  hn dµ . hn dµ = n=1

Proof: Let fn = By the MCT,

n i=1

n=1

hi , n ≥ 1, and let f =   fn dµ ↑ f dµ.

∞ i=1

hi . Then, 0 ≤ fn ↑ f .

But by Corollary 2.3.3,  fn dµ =

n 

hi dµ.

i=1

2

Hence, the result follows.

Corollary 2.3.6: Let f be a nonnegative measurable function on a measurable space (Ω, F, µ). For A ∈ F, let  ν(A) ≡ f IA dµ. Then, ν is a measure on (Ω, F). Proof: Let {An }n≥1 be a sequence of disjoint sets in F. Let hn = f IAn for n ≥ 1. Then by Corollary 2.3.5, ν(



 An )

=

n≥1

=

 f I[


n≥1

  ∞ n=1

An ] dµ

=

f·

 ∞

 IAn dµ

n=1

 ∞  ∞ hn dµ = hn dµ = ν(An ). n=1

n=1

2 Remark 2.3.5: Notice that µ(A) = 0 ⇒ ν(A) = 0. In this case ν is said to be dominated by or absolutely continuous with respect to µ. The Radon-Nikodym theorem (see Chapter 4) provides a converse to this. That is, if ν and µ are two measures on a measurable space (Ω, F) such that

54

2. Integration

ν is dominated by µ and µ is σ-finite, then there exists a nonnegative  measurable function f such that ν(A) = f IA dµ for all A in F. This f is called a Radon-Nikodym derivative (or a density) of ν with respect to µ dν and is denoted by dµ . Theorem 2.3.7: (Fatou’s lemma). Let {fn }n≥1 be a sequence of nonnegative measurable functions on (Ω, F, µ). Then   (3.9) lim inf fn dµ ≥ lim inf fn dµ. n→∞

n→∞

Proof: Let gn (ω) ≡ inf{fj (ω) : j ≥ n}. Then {gn }n≥1 is a sequence of nonnegative, nondecreasing measurable functions on (Ω, F, µ) such that gn (ω) ↑ g(ω) ≡ lim inf fn (ω). By the MCT, n→∞



 gn dµ ↑ But by monotonicity 

gdµ.

 fn dµ ≥

gn dµ for each n ≥ 1,

and hence, (3.9) follows.

2

Remark 2.3.6: In (3.9), the inequality can be strict. For example, take fn = I[n,∞) , n ≥ 1, on the measure space (R, B(R), m) where m is the Lebesgue measure. For another example, consider fn = nI[0, n1 ] , n ≥ 1, on the finite measure space ([0, 1], B([0, 1]), m). Definition 2.3.4: (The integral of a measurable function). Let f be a real valued measurable function on a measure space (Ω, F, µ). Let f + = f I{f ≥0} and f − = −f I{f <0} . The integral of f with respect to µ, denoted by f dµ, is defined as    f dµ = f + dµ − f − dµ, provided that at least one of the integrals on the right side is finite. Remark 2.3.7: Note that both f + and f − are nonnegative measurable + − + − functions and  f = f − f and |f | = f + f . Further, the integrals  + f dµ and f − dµ in Definition 2.3.4 are defined via Definition 2.3.3. Definition 2.3.5: (Integrable functions). A measurable function  f on a measure space (Ω, F, µ) is said to be integrable with respect to µ if |f |dµ < ∞. + − Since |f | = f + + f − , it follows that  −f is integrable iff both f and f are + integrable, i.e., f dµ < ∞ and f dµ < ∞. In the following, whenever

2.3 Integration

55

the integral of f or its integrability is discussed, the measurability of f will be assumed to hold.  Remark on notation: The f dµ is also written as   f (ω)µ(dω) and f (ω)dµ(ω). Ω

Ω

Definition 2.3.6: Let f be a measurable function on a measure space (Ω,F, µ) and A ∈ F. Then integral of f over A with respect to µ, denoted by A f dµ, is defined as   f dµ ≡ f IA dµ, (3.10) A

provided the right side is well defined. Definition 2.3.7: (Lp -spaces). Let (Ω, F, µ) be a measure space and 0 < p ≤ ∞. Then Lp (Ω, F, µ) is defined as Lp (Ω, F, µ) ≡

{f : |f |p is integrable with respect to µ}    = f : |f |p dµ < ∞ for 0 < p < ∞,

and

 L∞ (Ω, F, µ) ≡ f : µ({|f | > K}) = 0

for some

 K ∈ (0, ∞) .

The following is an extension of Proposition 2.3.1 to integrable functions. Proposition 2.3.8: Let f , g ∈ L1 (Ω, F, µ). Then    (i) (αf + βg)dµ = α f dµ + β gdµ for any α, β ∈ R.   (ii) f ≥ g a.e. (µ) ⇒ f dµ ≥ gdµ.   (iii) f = g a.e. (µ) ⇒ f dµ = gdµ. Proof: It is easy to verify (Problem 2.32) that if h = h1 − h2 where h1 and h2 are nonnegative functions in L1 (Ω, F, µ), then h is also in L1 (Ω, F, µ) and    hdµ = h1 dµ − h2 dµ. (3.11) Note that h ≡ αf + βg can be written as =

(a+ − α− )(f + − f − ) + (β + − β − )(g + − g − ) (α+ f + + α− f − + β + g + + β − g − ) − (α+ f − + α− f + + β + g − + β − g + )

= h1 − h 2 ,

say.

56

2. Integration

Since f , g ∈ L1 (Ω, F, µ), it follows that h1 and h2 ∈ L1 (Ω, F, µ). Further, they are nonnegative and by (3.11), h ∈ L1 (Ω, F, µ) and    hdµ = h1 dµ − h2 dµ. Now apply Proposition 2.3.1 to each of the terms on the right side and regroup the terms to get    hdµ = α f dµ + β gdµ. Proofs of (ii) and (iii) are left as an exercise.

2

Remark 2.3.8: By Proposition 2.3.8, if f and g ∈ L1 (Ω, F, µ), then so does αf + βg for all α, β ∈ R. Thus, L1 (Ω, F, µ) is a vector space over R. Further, if one sets  f 1 ≡

|f |dµ,


and identifies a function f with its equivalence class under the relation f ∼ g iff f = g a.e. (µ) , then  · 1 defines a norm on L1 (Ω, F, µ) and makes it a normed linear space (cf. Chapter 3). A similar remark also holds for Lp (Ω, F, µ) for 1 < p ≤ ∞.  Next note that by Proposition 2.3.8, if f = 0 a.e. (µ), then f dµ = 0. However, the converse is not true. But if f is nonnegative a.e. (µ), then the converse is true as shown below. Proposition 2.3.9: Let f be a measurable function on (Ω, F, µ) and let f be nonnegative a.e. (µ). Then  f dµ = 0 iff f = 0 a.e. (µ). Proof: It is enough to prove the “only if” part.  Let D = {ω : f (ω) > 0} and Dn = {ω : f (ω) > n1 }, n ≥ 1. Then D = n≥1 Dn . Since f ≥ f IDn a.e. (µ),   1 0 = f dµ ≥ f IDn dµ ≥ µ(Dn ) ⇒ µ(Dn ) = 0 for each n ≥ 1. n Also Dn ↑ D and so by m.c.f.b., µ(D) = lim µ(Dn ) = 0 . n→∞

Hence, Proposition 2.3.9 follows. A dual to the above proposition is the next one.

2

2.3 Integration

57

Proposition 2.3.10: Let f ∈ L1 (Ω, F, µ). Then, |f | < ∞ a.e. (µ). Proof: Let Cn = {ω : |f (ω)| > n}, n ≥ 1 and let C = {ω : |f (ω)| = ∞}. Then Cn ↓ C and    |f |dµ |f |dµ ≥ |f |ICn dµ ≥ nµ(Cn ) ⇒ µ(Cn ) ≤ . n  Since |f |dµ < ∞, limn→∞ µ(Cn ) = 0. Hence, by m.c.f.a., µ(C) = limn→∞ µ(Cn ) = 0. 2 The next result is a useful convergence theorem for integrals. Theorem 2.3.11: (The extended dominated convergence theorem or EDCT ). Let (Ω, F, µ) be a measure space and let fn , gn : Ω → R be F, R-measurable functions such that |fn | ≤ gn a.e. (µ) for all n ≥ 1. Suppose that (i) gn → g a.e. (µ) and fn → f a.e. (µ);   (ii) gn , g ∈ L1 (Ω, F, µ) and |gn |dµ → |g|dµ as n → ∞. Then, f ∈ L1 (Ω, F, µ),    fn dµ = f dµ and |fn − f |dµ = 0. lim lim (3.12) n→∞

n→∞

Two important special cases of Theorem 2.3.11 will be stated next. When gn = g for all n ≥ 1, one has the standard version of the dominated convergence theorem. Corollary 2.3.12: (Lebesgue’s dominated convergence theorem, or DCT ).  If |fn | ≤ g a.e. (µ) for all n ≥ 1, gdµ < ∞ and fn → f a.e. (µ), then f ∈ L1 (Ω, F, µ),    fn dµ = f dµ and |fn − f |dµ = 0. lim lim (3.13) n→∞

n→∞

Corollary 2.3.13: (The bounded convergence theorem, or BCT ). Let µ(Ω) < ∞. If there exist a 0 < k < ∞ such that |fn | ≤ k a.e. (µ) and fn → f a.e. (µ), then    fn dµ = f dµ and |fn − f |dµ = 0. lim lim (3.14) n→∞

n→∞

Proof: Take g(ω) ≡ k for all ω ∈ Ω in the previous corollary. Proof of Theorem 2.3.11: By Fatou’s lemma,     |f |dµ ≤ lim inf |fn |dµ ≤ lim inf |gn |dµ = |g|dµ < ∞. n→∞

n→∞

2

58

2. Integration

Hence, f is integrable. For proving the second part, let hn = fn + gn and γn = gn − fn , n ≥ 1. Then, {hn }n≥1 and {γn }n≥1 are sequences of nonnegative integrable functions. By Fatou’s lemma and (ii),   (f + g)dµ = lim inf hn dµ n→∞  ≤ lim inf hn dµ n→∞    = lim inf gn dµ + fn dµ n→∞   = gdµ + lim inf fn dµ. n→∞

Similarly,



 (g − f )dµ ≤

 gdµ − lim sup

fn dµ.

n→∞

   By Proposition 2.3.8, (g ± f )dµ = gdµ ± f dµ. Hence,   f dµ ≤ lim inf fn dµ n→∞



and lim sup n→∞

 fn dµ ≤

f dµ

  yielding that limn→∞ fn dµ = f dµ. For the last part, apply the above argument to fn and gn replaced by f˜n ≡ |f − fn | and g˜n ≡ gn + |f |, respectively. 2 Theorem 2.3.14: (An approximation theorem). Let µF be a LebesgueStieltjes measure on (R, B(R)). Let f ∈ Lp (R, B(R), µF ), 0 < p < ∞. Then, for any δ > 0, there exist a step function h and a continuous function g with compact support (i.e., g vanishes outside a bounded interval) such that  |f − h|p dµ < δ, (3.15)  |f − g|p dµ < δ,

(3.16)

k where a step function h is a function of the form h = i=1 ci IAi with k < ∞, c1 , c2 , . . . , ck ∈ R and A1 , A2 , . . . , Ak being bounded disjoint intervals. Proof: Let fn (·) = f (·)IBn (·) where Bn = {x : |x| ≤ n, |f (x)| ≤ n}. By the DCT, for every  > 0, there exists an N such that for all n ≥ N ,  (3.17) |f − fn |p dµF < .

2.4 Riemann and Lebesgue integrals

59

Since |fN
(·)| ≤ N on [−N , N ], for any η > 0, there exists a simple function f˜ such that sup{|fN
(x) − f˜(x)| : x ∈ R} < η. (cf. (3.7))

(3.18)

Next, using Problem 1.32 (b), one can show that for any η > 0, there exists a step function h (depending on η) such that  (3.19) |f˜N
− h|p dµF < η. Since f − h = f − fN
+ fN
− f˜ + f˜ − h,   |f − h|p ≤ Cp |f − fN
|p + |fN
− f˜|p + |f˜ − h|p , where Cp is a constant depending only on p. This in turn yields, from (3.17)–(3.19),  |f − h|p dµF ≤ Cp ( + (µF {x : |x| ≤ N })η p + η).

(3.20)

Given δ > 0, choose  > 0 first and then η > 0 such that the right side of (3.20) above is less than δ. Next, given any step function h and η > 0 there exists a continuous function g with compact support such that µF {x : h(x) = g(x)} < η (cf. Problem 1.32 (g)). Now (3.16) follows from (3.15). 2 Remark 2.3.9: The approximation (3.16) remains valid if g is restricted to the class of all infinitely differentiable functions with compact support. Further it remains valid for 0 < p < ∞ for Lebesgue-Stieltjes measures on any Euclidean space. Remark 2.3.10: The above approximation theorem fails for p = ∞. For example, consider the function f (x) ≡ 1 in L∞ (m).

2.4 Riemann and Lebesgue integrals Let f be a real valued bounded function on a bounded interval [a, b]. Recall the definition of the Riemann integral. Let P = {x0 , x1 , . . . , xn } be a finite partition of [a, b], i.e., x0 = a < x1 < x2 < xn−1 < xn = b and ∆ = ∆(P ) ≡ max{(xi+1 − xi ) : 0 ≤ i ≤ n − 1} be the diameter of P . Let Mi = sup{f (x) : xi ≤ x ≤ xi+1 } and mi = inf{f (x) : xi ≤ x ≤ xi+1 }, i = 0, 1, . . . , n − 1. Definition 2.4.1: The upper- and lower-Riemann sums of f w.r.t. the partition P are, respectively, defined as U (f, P ) ≡

n−1 i=0

Mi · (xi+1 − xi )

(4.1)

60

2. Integration

and L(f, P ) ≡

n−1

mi · (xi+1 − xi ).

(4.2)

i=0

It is easy to verify that if Q = {y0 , y1 , . . . , yk } is another partition satisfying P ⊂ Q, then U (f, P ) ≥ U (f, Q) ≥ L(f, Q) ≥ L(f, P ). Let P denote the collection of all finite partitions of [a, b].  Definition 2.4.2: The upper-Riemann integral f is defined as  f = inf U (f, P ) (4.3) 

P ∈P

and the lower-Riemann integral f , by  f = sup L(f, P ). P ∈P

(4.4)

It can be shown (cf. Problem 2.23) that if {Pn }n≥1 is any sequence of partitions such that ∆(Pn ) → 0 as n → ∞ and Pn ⊂ Pn+1 for each n ≥ 1,   then U (Pn , f ) ↓ f and L(Pn , f ) ↑ f . Definition 2.4.3: f is said to be Riemann integrable if   f = f. The common value is denoted by

 [a,b]

(4.5)

f.

Fix a sequence {Pn }n≥1 of partitions such that Pn ⊂ Pn+1 for all n ≥ 1 and ∆(Pn ) → 0 as n → ∞. Let Pn = {xn0 = a < xn1 < xn2 . . . < xnkn = b}. For i = 0, 1, . . . , kn − 1, let φn (x) ≡ ψn (x) ≡

sup{f (t) : xi ≤ t ≤ xi+1 }, x ∈ [xi , xi+1 ) inf{f (t) : xi ≤ t ≤ xi+1 }, x ∈ [xi , xi+1 )

and let φn (b) = ψn (b) = 0. Then, φn and ψn are step functions on [a, b] and hence, are Borel measurable. Further, since f is bounded, so are φn and ψn and hence are integrable on [a, b] w.r.t. theLebesgue measure m. The Lebesgue integrals of φn and ψn are given by [a,b] φn dm = U (Pn , f )  and [a,b] ψn dm = L(Pn , f ).  It can be shown (Problem 2.24) that for all x ∈ n≥1 Pn , as n → ∞, φn (x) → φ(x) ≡ lim sup{f (y) : |y − x| < δ}

(4.6)

ψn (x) → ψ(x) ≡ lim inf{f (y) : |y − x| < δ}

(4.7)

δ↓0

and δ↓0

2.5 More on convergence

61

Thus, φ and ψ, being limits of Borel measurable functions (except possibly on a countable set), are Borel measurable. By the BCT (Corollary 2.3.13),    f = lim φn dm = φdm n→∞

and



 f = lim

n→∞

 ψn dm =

ψdm. 

  Thus, f is Riemann integrable on [a, b], iff φdm = ψdm, iff (φ − ψ)dm = 0. Since φ(x) ≥ f (x) ≥ ψ(x) for all x, this holds iff φ = f = ψ a.e [m]. It can be shown that f is continuous at x0 iff φ(x0 ) = f (x0 ) = ψ(x0 ) (Problem 2.8). Summarizing the above discussion, one gets the following theorem. Theorem 2.4.1: Let f be a bounded function on a bounded interval [a, b]. Then f is Riemann integrable on [a, b] iff f is continuous a.e. (m) on [a, b]. In  [a, b] and the Lebesgue in this case, f is Lebesgue integrable on tegral [a,b] f dm equals the Riemann integral [a,b] f , i.e., the two integrals coincide. It should be noted that Lebesgue integrability need not imply Riemann integrability. For example, consider f (x) ≡ IQ1 (x) where Q1 is the set of rationals in [0, 1] (Problem 2.25). The functions φ and ψ defined in (4.6) and (4.7) above are called, respectively, the upper and the lower envelopes of the function f . They are semicontinuous in the sense that for each α ∈ R, the sets {x : φ(x) < α} and {x : ψ(x) > α} are open (cf. Problem 2.8). Remark 2.4.1: The key difference in the definitions of Riemann and Lebesgue integrals is that in the former the domain of f is partitioned while in the latter the range of f is partitioned.

2.5 More on convergence Let {fn }n≥1 and f be measurable functions from a measure space (Ω, F, µ) ¯ the set of extended real numbers. There are several notions of conto R, vergence of {fn }n≥1 to f . The following two have been discussed earlier. Definition 2.5.1: {fn }n≥1 converges to f pointwise if lim fn (ω) = f (ω)

n→∞

for all ω in Ω.

Definition 2.5.2: {fn }n≥1 converges to f almost everywhere (µ), denoted by fn → f a.e. (µ), if there exists a set B in F such that µ(B) = 0 and lim fn (ω) = f (ω)

n→∞

for all ω∈B c .

(5.1)

62

2. Integration

Now consider some more notions of convergence. Definition 2.5.3: {fn }n≥1 converges to f in measure (w.r.t. µ), denoted by fn −→m f , if for each  > 0,
 lim µ {|fn − f | > } = 0. (5.2) n→∞

to f in Lp (µ), Definition 2.5.4: Let 0 <p < ∞. Then, {fn }n≥1 converges  p denoted by fn −→L f , if |fn |p dµ < ∞ for all n ≥ 1, |f |p dµ < ∞ and  |fn − f |p dµ = 0. (5.3) lim n→∞

Clearly, (5.3) is equivalent to fn − f p → 0 as n → ∞, where for any F-measurable function g and any 0 < p < ∞,  min{ p1 ,1} gp = . (5.4) |g|p dµ For p = 1, this is also called convergence in absolute deviation and for p = 2, convergence in mean square. Definition 2.5.5: {fn }n≥1 converges to f uniformly (over Ω) if lim

n→∞

sup{|fn (ω) − f (ω)| : ω ∈ Ω} = 0.

(5.5)

Definition 2.5.6: {fn }n≥1 converges to f in L∞ (µ) if lim fn − f ∞ = 0,

n→∞

where for any F-measurable function g on (Ω, F, µ),   g∞ = inf K : K ∈ (0, ∞), µ({|g| > K}) = 0 .

(5.6)

(5.7)

Definition 2.5.7: {fn }n≥1 converges to f nearly uniformly (µ) if for every  > 0, there exists a set A ∈ F such that µ(A) <  and on Ac , fn → f uniformly, i.e., sup{|fn (ω) − f (ω)| : ω ∈ Ac } → 0 as n → ∞. The notion of convergence in Definition 2.5.7 is also called almost uniform convergence in some books (cf. Royden (1988)). The sequence fn ≡ nI[0,1/n] on (Ω = [0, 1], B([0, 1]), m) converges to f ≡ 0 nearly uniformly but not in L∞ (m). When µ is a probability measure, there is another useful notion of convergence, known as convergence in distribution, that is defined in terms of the induced measures {µfn−1 }n≥1 and µf −1 . This notion of convergence will be treated in detail in Chapter 9.

2.5 More on convergence

63

Next, the connections between some of these notions of convergence are explored. Theorem 2.5.1: Suppose that µ(Ω) < ∞. Then, fn → f a.e. (µ) implies fn −→m f . The proof is left as an exercise (Problem 2.26). The hypothesis that ‘µ(Ω) < ∞’ in Theorem 2.5.1 cannot be dispensed with as seen by taking fn = I[n,∞) on R with Lebesgue measure. Also, fn −→m f does not imply fn → f a.e. (µ) (Problem 2.46), but the following holds. Theorem 2.5.2: Let fn −→m f . Then, there exists a subsequence {nk }k≥1 such that fnk → f a.e. (µ). Proof: Since fn −→m f , for each integer k ≥ 1, there exists an nk such that for all n ≥ nk   µ |fn − f | > 2−k (5.8) < 2−k . W.l.o.g., that assume nk+1 > nk for all k ≥ 1. Let Ak = {|fnk − f | > 2−k }. By Corollary 2.3.5,    ∞ ∞  ∞ IAk dµ = IAk dµ = µ(Ak ), k=1

k=1

k=1

∞ which, by (5.8), is finite. by Proposition 2.3.10, k=1 IAk < ∞ a.e. Hence, ∞ (µ). Now observe that k=1 IAk (ω) < ∞ ⇒ |fnk (ω) − f (ω)| ≤ 2−k for all k large ⇒ limk→∞ fnk (ω) = f (ω). Thus, fnk → f a.e. (µ). 2 Remark 2.5.1: From the above result it follows that the extended dominated convergence theorem (Theorem 2.3.11) remains valid if convergence a.e. of {fn }n≥1 and of {gn }n≥1 are replaced by convergence in measure for both (Problem 2.37). Theorem 2.5.3: Let {fn }n≥1 , f be measurable functions on a measure p space (Ω, F, µ). Let fn −→L f for some 0 < p < ∞. Then fn −→m f . Proof: For each  > 0, let An = {|fn − f | ≥ }, n ≥ 1. Then   p |fn − f |p dµ ≥ p µ(An ). |fn − f | dµ ≥ Since fn → f in Lp ,



An

|fn − f |p dµ → 0 and hence, µ(An ) → 0. p

2

It turns out that fn −→m f need not imply fn −→L f , even if {fn : n ≥ 1} ∪ {f } is contained in Lp (Ω, F, µ). For example, let fn = nI[0, n1 ] and f ≡ 0 on the Lebesgue space ([0,  1], B([0, 1]), m), where m is the Lebesgue measure. Then fn −→m f but |fn − f | ≡ 1 for all n ≥ 1. However, under

64

2. Integration

some additional conditions, convergence in measure does imply convergence in Lp . Here are two results in this direction. Theorem 2.5.4: (Scheffe’s theorem). Let {fn }n≥1 , f be a collection of nonnegative  functions  on a measure space (Ω, F, µ). Let fn → f  measurable a.e. (µ), fn dµ → f dµ and f dµ < ∞. Then  lim |fn − f |dµ = 0. n→∞

gn+ and gn− go Proof: Let gn = f − fn , n ≥ 1. Since fn → f a.e. (µ), both  + to zero a.e. (µ). Further, 0 ≤ gn ≤ f and by hypothesis f dµ < ∞. Thus, by the DCT, it follows that  gn+ dµ → 0.    gn dµ → 0. Thus, gn− dµ = gn+ dµ −  Next, note that by hypothesis, 2 gn dµ → 0 and hence, |gn |dµ = gn+ dµ + gn− dµ → 0. Corollary 2.5.5: Let {fn }n≥1 , f be probability  density functions on a measure space (Ω, F, µ). That is, for all n ≥ 1, fn dµ = f dµ = 1 and fn , f ≥ 0 a.e. (µ). If fn → f a.e. (µ), then  |fn − f |dµ = 0. lim n→∞

Remark 2.5.2: The above theorem and the corollary remain valid if the convergence of fn to f a.e. (µ) is replaced by f −→m f . , n = 1, 2, . . . and  {pk }k≥1 be sequences of Corollary 2.5.6: Let {pnk }k≥1 ∞ ∞ nonnegative numbers satisfying k=1 pnk = 1 = k=1 pk . Let pnk → pk ∞ as n → ∞ for each k ≥ 1. Then k=1 |pnk − pk | → 0. Proof: Apply Corollary 2.5.5 with µ = the counting measure on (N, P(N)). 2 A more general result in this direction that does not require fn , f to be nonnegative involves the concept of uniform integrability. Let {fλ : λ ∈ Λ} be a collection of functions in L1 (Ω, F, µ). Then for each λ ∈ Λ, by the DCT and the integrability of fλ ,  aλ (t) ≡ |fλ |dµ → 0 as t → ∞. (5.9) {|fλ |>t}

The notion of uniform integrability requires that the integrals aλ (t) go to zero uniformly in λ ∈ Λ as t → ∞.

2.5 More on convergence

65

Definition 2.5.8: The collection of functions {fλ : λ ∈ Λ} in L1 (Ω, F, µ) is uniformly integrable (or UI, in short) if sup aλ (t) → 0 as t → ∞ .

(5.10)

λ∈Λ

The following proposition summarizes some of the main properties of UI families of functions. Proposition 2.5.7: Let {fλ : λ ∈ Λ} be a collection of µ-integrable functions on (Ω, F, µ). (i) If Λ is finite, then {fλ : λ ∈ Λ} is UI.  (ii) If K ≡ sup{ |fλ |1+ dµ : λ ∈ Λ} < ∞ for some  > 0, then {fλ : λ ∈ Λ} is UI.  (iii) If |fλ | ≤ g a.e. (µ) and gdµ < ∞, then {fλ : λ ∈ Λ} is UI. (iv) If {fλ : λ ∈ Λ} and {gγ : γ ∈ Γ} are UI, then so is {fλ + gγ : λ ∈ Λ, γ ∈ Γ}. (v) If {fλ : λ ∈ Λ} is UI and µ(Ω) < ∞, then  sup |fλ |dµ < ∞.

(5.11)

λ∈Λ

 Proof: By hypothesis, aλ (t) ≡ {|fλ |>t} |fλ |dµ → 0 as t → ∞ for each λ. If Λ is finite this implies that sup{aλ (t) : λ ∈ Λ} → 0 as t → ∞. This proves (i). To prove (ii), note that since 1 < |fλ |/t on the set {|fλ | > t},    |fλ | |fλ |/t dµ ≤ Kt− → 0 as t → ∞. sup aλ (t) ≤ sup λ∈Λ

λ∈Λ

|fλ |>t

For (iii), note that for each t ∈ R, the function ht (x) ≡ xI(t,∞) (x), x ∈ R is nondecreasing. Hence, by the integrability of g,    sup aλ (t) = sup ht (|fλ |)dµ ≤ ht (g)dµ = gdµ → 0 as t → ∞. λ∈Λ

λ∈Λ

{g>t}

 To prove (iv), for t > 0, let a(t) = supλ∈Λ ht (|fλ |)dµ and b(t) =  supγ∈Γ ht (|gγ |)dµ. Then, for any λ ∈ Λ, and γ ∈ Γ,   
|fλ + gγ |dµ = ht |fλ + gγ | dµ {|fλ +gγ |>t}

 ≤


 ht 2 max{|fλ |, |gγ |} dµ

66

2. Integration





 ht 2|fλ | I |fλ | ≥ |gγ | dµ +   ≤ 2 |fλ |dµ + 2



=

{|fλ |>t/2}

≤



 ht 2|gγ | I |fλ | < |gγ | dµ

{|gγ |>t/2}

|gγ |dµ

2[a(t/2) + b(t/2)].

By hypothesis, both a(t) and b(t) → 0 as t → ∞, thus proving (iv). Next consider (v). Since {fλ } : λ ∈ Λ} is UI, there exists a T > 0 such that 
 sup hT |fλ | dµ ≤ 1. λ∈Λ

Hence,  sup

 |fλ |dµ

=

sup

λ∈Λ

λ∈Λ

!



{|fλ |≤T }

|fλ |dµ +

hT (|fλ |)dµ

≤ T µ(Ω) + 1 < ∞. 2

This completes the proof of the proposition.

Remark 2.5.3: In the above proposition, part (ii) can be improved as φ(x) follows: Let  φ : R+ → R+ be nondecreasing and x ↑ ∞ as x ↑ ∞. If supλ∈Λ φ(|fλ |)dµ < ∞, then {fλ : λ ∈ Λ} is UI (Problem 2.27). A converse to this result is true. That is, if {fλ : λ ∈ Λ} is UI then there exists such a function φ. Some examples of such φ’s are φ(x) = xk , k > 1, φ(x) = x(log x)β I(x > 1), β > 0, and φ(x) = exp(βx), β > 0. In part (v) of Proposition 2.5.7, (5.11) does not imply UI. For example, consider the sequence of functions fn = nI[0, n1 ] , n = 1, 2, . . . on [0, 1]. On the other hand, (5.11) with an additional condition becomes necessary and sufficient for UI. Proposition 2.5.8: Let f ∈ L1 (Ω, F, µ). Then for every  > 0, there exists a δ > 0 such that µ(A) < δ ⇒ A |f |dµ < . Fix  > 0. By the DCT, there exists a t > 0 such that  |f |dµ < /2. Hence, for any A ∈ F with µ(A) ≤ δ ≡ 2t , {|f |>t}

Proof: 



 |f |dµ

A

≤



A∩{|f |≤t}

|f |dµ +



≤ tµ(A) +

{|f |>t}

{|f |>t}

|f |dµ

|f |dµ

≤ , proving the claim.

2

2.5 More on convergence

67

The above proposition shows that for every f ∈ L1 (Ω, F, µ), the measure (cf. Corollary 2.3.6)  ν|f | (A) ≡

|f |dµ

(5.12)

A

on (Ω, F) satisfies the condition that ν|f | (A) is small if µ(A) is small, i.e., for every  > 0, there exists a δ > 0 such that µ(A) < δ ⇒ ν|f | (A) < . This property is referred to as the absolute continuity of the measure νf w.r.t. µ. Definition 2.5.9: Given a family {fλ : λ ∈ Λ} ⊂ L1 (Ω, F, µ), the measures {ν|fλ | : λ ∈ Λ} as defined in (5.12) above are uniformly absolutely continuous w.r.t. µ (or u.a.c. (µ), in short) if for every  > 0, there exists a δ > 0 such that   µ(A) < δ ⇒ sup ν|fλ | (A) : λ ∈ Λ < . Theorem 2.5.9: Let {fλ : λ ∈ Λ} ⊂ L1 (Ω, F, µ) and µ(Ω) < ∞. Then, {fλ : λ ∈ Λ} is UI iff supλ∈Λ |fλ |dµ < ∞ and ν|fλ | (·) : λ ∈ Λ is u.a.c. (µ).   Proof: Let fλ : λ ∈ Λ be UI. Then, since µ(Ω) < ∞, L1 boundedness of {fλ : λ ∈ Λ} follows from Proposition 2.5.7 (v). To establish u.a.c. (µ), fix  > 0. By UI, there exists an N such that  |fλ |dµ < /2. sup λ∈Λ

{|fλ |>N }

Let δ = and let A ∈ F be such that µ(A) < δ. Then, as in the proof of Proposition 2.5.8 above,  |fλ |dµ ≤ N µ(A) + /2 < . sup  2N

λ∈Λ

A

 Conversely, suppose fλ : λ ∈ Λ is L1 bounded and u.a.c. (µ). Then, for every  > 0, there exists a δ > 0 such that  |fλ |dµ <  if µ(A) < δ . (5.13) sup 

λ∈Λ

A

Also, for any nonnegative f in L1 (Ω, F, µ) and t > 0,
 tµ {f ≥ t} , which implies that 
 f dµ µ {f ≥ t} ≤ . t



f dµ ≥

 {f >t}

f dµ ≥

(This is known as Markov’s inequality − see Chapter 3). Hence, it follows that    sup µ({|fλ | ≥ t}) ≤ sup |fλ |dµ / t. (5.14) λ∈Λ

λ∈Λ

68

2. Integration

   Now given  > 0, choose T such that supλ∈Λ |fλ |dµ /T < δ where δ is as in (5.13). Then, by (5.14), it follows that  |fλ |dµ < , sup λ∈Λ

{|fλ |≥T
}

i.e., {fλ : λ ∈ Λ} is UI.

2

Theorem 2.5.10: Let (Ω, F, µ) be a measure space with µ(Ω) < ∞, and let {fn : n ≥ 1} ⊂ L1 (Ω, F, µ) be such that fn → f a.e. (µ) and f is F, B(R)-measurable. If {fn : n ≥ 1} is UI, then f is integrable and  |fn − f |dµ = 0. lim n→∞

Remark 2.5.4: In view of Proposition 2.5.7 (iii), Theorem 2.5.10 yields convergence of f dµ to f dµ under weaker conditions than the DCT, provided µ(Ω) < ∞. However, even under the restriction µ(Ω) < ∞, UI of  {fn }n≥1is a sufficient, but not a necessary condition for convergence of In the fn dµ to f dµ (Problem 2.28).  special case where fn ’s are non negative (and µ(Ω) < ∞), fn dµ → f dµ < ∞ if and only if {fn }n≥1 are UI (Problem 2.29). On the other hand, when  µ(Ω) = +∞, UI is no longer sufficient to guarantee the convergence of fn dµ to f dµ (Problem 2.30). Thus, the notion of UI is useful mainly for finite measures and, in particular, probability measures. Proof: By Proposition 2.5.7 and Fatou’s lemma,    |f |dµ ≤ lim inf |fn |dµ ≤ sup |fn |dµ < ∞ n→∞

n≥1

and hence f is integrable. Next for n ∈ N, t ∈ (0, ∞), define the functions gn = |fn − f |, g n,t = gn I(|gn | ≤ t) and g¯n,t = gn I(|gn | > t). Since {fn }n≥1 is UI and f is integrable, by Proposition 2.5.7 (iv), {gn }n≥1 is UI. Hence, for any  > 0, there exists t > 0 such that  sup g¯n,t dµ <  for all t ≥ t . (5.15) n≥1

Next note that for any t > 0, since fn → f a.e. (µ), g n,t → 0 a.e. (µ), and  |g n,t | ≤ t and (t)dµ = tµ(Ω) < ∞. Hence, by the DCT,  lim

n→∞

g n,t dµ = 0

for all t > 0.

(5.16)

2.5 More on convergence

69

By (5.15) and (5.16) with t = t , we get    g n,t dµ ≤  0 ≤ lim sup |fn − f |dµ ≤ sup g¯n,t dµ + lim n→∞

n→∞

n≥1

for all  > 0. Hence,



|fn − f |dµ → 0 as n → ∞.

2

The next result concerns connections between the notions of almost everywhere convergence and almost uniform convergence. Theorem 2.5.11: (Egorov’s theorem). Let fn → f a.e. (µ) and µ(Ω) < ∞. Then fn → f nearly uniformly (µ) as in Definition 2.5.7. Proof: For j, n, r ∈ N, define the sets Ajr Bnr

= {ω : |fj (ω) − f (ω)| ≥ r−1 }

= Ajr , Cr = Bnr j≥n

D

=



n≥1

Cr .

r≥1

It is easy to verify that D is the set of points where fn does not convergence to f . That is, D = {ω : fn (ω) → f (ω)}. By hypothesis µ(D) = 0. This implies µ(Cr ) = 0 for all r ≥ 1. Since Bnr ↓ Cr as n → ∞ and µ(Ω) < ∞, by m.c.f.a., µ(Cr ) = 0 ⇒ limn→∞ µ(Bnr ) = 0. Sofor all r ≥ 1,  > 0, there exists kr ∈N such that µ(Bkr r ) < 2r . Let A = r≥1 Bkr r . Then µ(A) < ε and Ac = r≥1 Bkcr r . Also, for each r ∈ N, Ac ⊂ Bkcr r . So for any n ≥ 1, sup{|fn (ω) − f (ω)| : ω ∈ Ac } ≤ sup{|fn (ω) − f (ω)| : ω ∈ Bkcr r } for all r ∈ N 1 ≤ if n ≥ kr . r That is, sup{|fn (ω) − f (ω)| : ω ∈ Ac } → 0 as n → ∞.

2

Theorem 2.5.12: (Lusin’s theorem). Let F : R → R be a nondecreasing function and let µ = µ∗F be the corresponding Lebesgue-Stieltjes measure
on (R, Mµ∗F ). Let f : R → R be Mµ∗F , B(R)-measurable. Let µ {x :  |f (x)| = ∞} = 0. Then
for every  > 0, there exists a continuous function g : R → R such that µ {x : f (x) = g(x)} < .
Proof: Fix −∞ < a < b < ∞. Since µ([a, b]) < ∞ and µ {x : |f (x)| =  ∞} = 0, for each  > 0, there exists K ∈ (0, ∞) such that 
 µ {x : a ≤ x ≤ b, |f (x)| > K} < . 2

70

2. Integration

Let fK (x) = f (x)I[a,b] (x)I{|f |≤K} (x). Then fK : R → R is bounded, Mµ∗F , B(R)-measurable and zero for |x| > K. Consider now the following claim: For every  > 0, there exists a continuous function g : [a, b] → R such that
  µ {x : fK (x) = g(x), a ≤ x ≤ b} < . (5.17) 2 Clearly this implies that
 µ {x : a ≤ x ≤ b, f (x) = g(x)} < . (5.18) δ , apply Fix δ > 0. Now for each n ∈ Z, take [a, b] = [n, n + 1],  = 2|n|+2 . Let g ˜ be a continuous (5.18) and call the resulting continuous function g n 
function from R → R such that µ {x : n ≤ x ≤ n + 1, g˜(x) = gn (x)} < δ . This can be done by setting g˜(x) = gn (x) for n ≤ x ≤ (n + 1) − δn 2|n|+2 " # δ and linear on (n + 1) − δn , n + 1 for some 0 < δn < 2|n|+3 . Then ∞


 µ {x : f (x) = g˜(x)} ≤

n=−∞ ∞

≤


 µ {x : n ≤ x ≤ n + 1, f (x) = g˜(x)}
 µ {x : n ≤ x ≤ n + 1, f (x) = gn (x)}

n=−∞ ∞

+

<

n=−∞ ∞

2

n=−∞


 µ {x : n ≤ x ≤ n + 1, gn (x) = g˜(x)} δ 2|n|+2

≤ 2δ.

So it suffices to establish (5.17). Since fK : R → R is bounded and Mµ∗F , B(R)-measurable, for each  > 0, there exists a simple function k s(x) ≡ i=1 ci IAi (x), with Ai ⊂ [a, b], Ai ∈ Mµ∗F , {Ai : 1 ≤ i ≤ k} are disjoint, µ(Ai ) < ∞, and ci ∈ R for i = 1, . . . , k, such that |f (x)−s(x)| < 4 for all a ≤ x ≤ b. By Theorem 1.3.4, for each Ai and η > 0, there exist a finite number of open disjoint intervals Iij = (aij , bij ), j = 1, . . . ni such that 

ni

η . Iij < µ Ai  2k j=1 Now as in Problem 1.32 (g), there exists a continuous function gij such that   η −1 µ gij {1}  Iij < , j = 1, 2, . . . , ni , i = 1, 2, . . . , k. kni Let gi ≡

ni j=1

gij , 1 ≤ i ≤ k.

2.6 Problems

71

k Then µ(Ai  gi−1 {1}) < kη . Let g = i=1 ci gi . Then µ({s = g}) < η. Hence for every  > 0, η > 0, there is a continuous function g,η : [a, b] → R such that 
 µ x : a ≤ x ≤ b, |fK (x) − g,η (x)| >  < η. Now for each n ≥ 1, let hn (·) ≡ g 21n , 21n (·) Then, µ(An ) <

1 2n

 1 and An = x : a ≤ x ≤ b, |fK (x) − hn (x)| > n . 2 and hence ∞

µ(An ) < ∞.

n=1

By the MCT, this implies that ∞

 [a,b]


∞

n=1 IAn

IAn < ∞ a.e.

 dµ < ∞ and hence

µ.

n=1

Thus hn → fK a.e. µ on [a, b]. By Egorov’s theorem for any  > 0, there is a set A ∈ B([a, b]) such that µ(Ac ) < /2

and hn → fK

uniformly on A .

By the inner regularity (Corollary 1.3.5) of µ, there is a compact set D ⊂ A such that µ(A \D) < /2. Since hn → fK uniformly on A , fK is continuous on A and hence on D. It can be shown that there exists a continuous function g : [a, b] → R such that g = fK on D (Problem 2.8 (e)). A more general result extending a continuous function defined on a closed set to the whole space is known as Tietze’s extension theorem (see Munkres (1975)). Thus µ({x : a ≤ x ≤ b, fK (x) = g(x)}) < . This completes the proof of (5.17) and hence that of the proposition. 2 Remark 2.5.5: (Littlewood’s principles). As pointed out in Section 1.3, Theorems 1.3.4, 2.5.11, and 2.5.12 constitute J. E. Littlewood’s three principles: every Mµ∗F measurable set is nearly a finite union of intervals; every a.e. convergent sequence is nearly uniformly convergent; and every Mµ∗F measurable function is nearly continuous.

2.6 Problems 2.1 Let Ωi , i = 1, 2 be two nonempty sets and T : Ω1 → Ω2 be a map. Then for any collection {Aα : α ∈ I} of subsets of Ω2 , show that 




 = Aα T −1 Aα T −1 α∈I

α∈I

72

2. Integration

and T −1



 Aα



=

α∈I


 T −1 Aα .

α∈I

c
Further, T −1 (A) = T −1 (Ac ) for all A ⊂ Ω2 . (These are known as de Morgan’s laws.) 2.2 Let {Ai }i≥1 be a collection of disjoint sets in a measurable space (Ω, F). of F, B(R)-measurable functions (a) Let {gi }i≥1 be a collection ∞ from Ω to R. Show that i=1 IAi gi converges on R and is F, B(R)-measurable. (b) Let G ≡ σ{Ai : i ≥ 1}. Show that h : Ω → R is G, B(R)measurable iff g(·) is constant on each Ai . 2.3 Let f, g : Ω → R be F, B(R)-measurable. Set h(ω) =

f (ω) I(g(ω) = 0), g(ω)

ω ∈ Ω.

Verify that h is F, B(R)-measurable. ¯ be such that for every r ∈ R, g −1 ((−∞, r]) ∈ F. Show 2.4 Let g : Ω → R ¯ that g is F, B(R)-measurable. 2.5 Prove Proposition 2.1.6. (Hint: Show that σ{fλ : λ ∈ Λ} =



σ{fλ : λ ∈ L}

L∈C

where C is the collection of all countable subsets of Λ.) 2.6 Let Xi , i = 1, 2, 3 be random variables on a probability space (Ω, F, P ). Consider the random equation (in t ∈ R): X1 (ω)t2 + X2 (ω)t + X3 (ω) = 0.

(6.1)

(a) Show that A ≡ {ω ∈ Ω : Equation (6.1) has two distinct real roots} ∈ F. (b) Let T1 (ω) and T2 (ω) denote the two roots of (6.1) on A. Let  Ti (w) on A , fi (w) = 0 on Ac i = 1, 2. Show that (f1 , f2 ) is F, B(R2 )-measurable.

2.6 Problems

73



2.7 Let M ≡ Xij , 1 ≤ i, j ≤ k, be a (random) matrix of random variables Xij defined on a probability space (Ω, F, P ). (a) Show that Y1 ≡ det(M ) (the determinant of M ) and Y2 ≡ tr(M )(the trace of M ) are both F, B(R)-measurable. (b) Show also that Y3 ≡ the largest eigenvalue of M  M is F, B(R)measurable, where M  is the transpose of M .





M M x) .) (Hint: Use the result that Y3 = sup (x (x
x) x =0

2.8 Let f : R → R. Let f¯(x) = inf δ>0 sup|y−x|<δ f (y) and f (x) = supδ>0 inf |y−x|<δ f (y), x ∈ R. (a) Show that for any t ∈ R, {x : f¯(x) < t} is open and hence, f¯ is B(R), B(R)-measurable. (b) Show that for any t > 0, {x : f¯(x) − f (x) < t} ≡



{x : f¯(x) < t + r, f (x) > r}

r∈Q

and hence is open. (c) Show that f is continuous at some x0 in R iff f¯(x0 ) = f (x0 ). (d) Show that the set Cf ≡ {x : f (·) is continuous at x} is a Gδ set, i.e., an intersection of a countable number of open sets, and hence, Cf is a Borel set. (e) Let D be a closed set in R. Let f : D → R be continuous on D. Show that there exists a g : R → R continuous such that g = f on D. (Hint: Note that Dc is open in R and hence it can be expressed as a countable union of disjoint open intervals {Ij = (aj , bj ) : 1 ≤ j ≤ k ≤ ∞}. Note that aj , bj ∈ D for all j except for possibly the j’s for which aj = −∞ or bj = +∞. Let ⎧ f (x) if x ∈ D ⎪ ⎪ ⎪ (x−aj ) ⎪ f (a ) + (f (b ) − f (a )) if x ∈ (aj , bj ), ⎪ j j j (bj −aj ) ⎪ ⎪ ⎪ ⎨ aj , bj ∈ D g(x) ≡ ) if a f (b j j = −∞, ⎪ ⎪ ⎪ x ∈ (aj , bj ) ⎪ ⎪ ⎪ ⎪ f (a ) if b = ∞, j j ⎪ ⎩ x ∈ (aj , bj ). Now verify that g has the required properties.)

74

2. Integration

2.9 Prove Proposition 2.2.1 using Problem 2.1. 2.10 (a) Show that for any x ∈ R and any random variable X with cdf FX (·), P (X < x) = FX (x−) ≡ limy↑x FX (y). (b) Show that Fc (·) in (2.5) is continuous. 2.11 Let F : R → R be nondecreasing. (a) Show that for x ∈ R, F (x−) ≡ limy↑x F (y) and F (x+) = limy↓x F (y) exist and satisfy F (x−) ≤ F (x) ≤ F (x+). (b) Let D ≡ {x : F (x+) − F (x−) > 0}. Show that D is at most countable. (Hint: Show that D=



Dn,r ,

n≥1 r≥1

where Dn,r = {x : |x| ≤ n, F (x+) − F (x−) > 1r } and that each Dn,r is finite.) 2.12 Suppose that (i) and (iii) of Proposition 2.2.3 hold. Show that for any a1 in R and −∞ < a2 ≤ b2 < ∞, F (a2 , a1 ) ≤ F (b2 , a1 ) and F (a1 , a2 ) ≤ F (a1 , b2 ), (i.e., F is monotone coordinatewise). 2.13 Let F : Rk → R be such that: (a) for x1 = (x11 , x12 , . . . , x1k ) and x2 = (x21 , x22 , . . . , x2k ) with x1i ≤ x2i for i = 1, 2, . . . k, ∆F (x1 , x2 ) ≡ (−1)s(a) F (a) ≥ 0, a∈A

where A ≡ {a = (a1 , a2 , . . . , ak ) : ai ∈ {x1i , x2i }, i = 1, 2, . . . , k} and for a in A, s(a) = |{i : ai = x1i , i = 1, 2, . . . , k}| is the number of indices i for which ai = x1i . (b) For each i = 1, 2, . . . , k,

lim F (xi ) = 0.

xi1 ↓−∞

Let Ck be the semialgebra of sets of the form {A : A = A1 × . . . × Ak , Ai ∈ C for all 1 ≤ i ≤ k} where C is the semialgebra in R defined in (3.7). Set µF (A) ≡ ∆F (x1 , x2 ) if A = (x11 , x21 ] × (x12 , x22 ] × . . . × (x1k , x2k ] is bounded and set µF (A) = limn→∞ µ(A ∩ Jn ), where Jn = (−n, n]k . (i) Show that F is coordinatewise monotone, i.e., if x = (x1 , . . . , xk ), y = (y1 , . . . , yk ) and xi ≤ yi for every i = 1, . . . , k, then F (y) ≥ F (x).

2.6 Problems

75

(ii) Show that C is a semialgebra and µF is a measure on C by using the Heine-Borel theorem as in Problem 1.22 and 1.23. 2.14 Let m(·) denote the Lebesgue measure on (R, B(R)). Let T : R → R be the map T (x) = x2 . Evaluate the induced measure mT −1 (A) of the set A, where (a) A = [0, t], t > 0. (b) A = (−∞, 0). (c) A = {1, 2, 3, . . .}. ∞ (d) A = i=1 (i2 , (i + i12 )2 ). ∞ (e) A = i=1 (i2 , (i + 1i )2 ).
 2.15 Consider the probability space (0, 1), B((0, 1)), m , where m(·) is the Lebesgue measure. (a) Let Y1 be the random variable Y1 (x) ≡ sin 2πx for x ∈ (0, 1). Find the cdf of Y1 . (b) Let Y2 be the random variable Y2 (x) ≡ log x for x ∈ (0, 1). Find the cdf of Y2 . (c) Let F : R → R be a cdf . For 0 < x < 1, let F1−1 (x)

=

inf{y : y ∈ R, F (y) ≥ x}

F2−1 (x)

=

sup{y : y ∈ R, F (y) ≤ x}.

Let Zi be the random variable defined by Zi = Fi−1 (x) 0 < x < 1,

i = 1, 2.

(i) Find the cdf of Zi , i = 1, 2. (Hint: Verify using the right continuity of F that for any 0 < x < 1, t ∈ R, F (t) ≥ x ⇔ F1−1 (x) ≤ t.) (ii) Show also that F1−1 (·) is left continuous and F2−1 (·) is right continuous. 2.16 (a) Let (Ω, F1 , µ) be a σ-finite measure space. Let T : Ω → R be F, B(R)-measurable. Show, by an example, that the induced measure µT −1 need not be σ-finite. (b) Let (Ωi , Fi ) be measurable spaces for i = 1, 2 and let T : Ω1 → Ω2 be F1 , F2 -measurable. Show that any measure µ on (Ω1 , F1 ) is σ-finite if µT −1 is σ-finite on (Ω2 , F2 ).

76

2. Integration

2.17 Let (Ω, F, µ) be a measure space and let f : Ω → [0, ∞] be such that it admits two representations k

f=

ci IAi and f =

i=1



dj IBj ,

j=1

where ci , dj ∈ [0, ∞],and Ai and Bj ∈ F for all i, j. Show that k

ci µ(Ai ) =

i=1



dj µ(Bj ).

j=1

(Hint: Express Ai and Bj as finite unions of a common collection of disjoint sets in F.) 2.18 (a) Prove Proposition 2.3.1. (b) In the proof of Proposition 2.3.2, verify that Dn ↑ D. (c) Verify Remark 2.3.1. (Hint: Let An A

{ω : fn−1 (ω) ≥ fn (ω), gn+1 (ω) ≥ gn (ω)}    ω : lim gn (ω) = g(ω), = An

=

n→∞

n≥1

 lim fn (ω) = f (ω)

n→∞

and f˜n = fn IA , g˜n = gn IA . Verify that µ(Ac ) = 0 and apply gn }n≥1 .) Proposition 2.3.2 to {f˜n }n≥1 and {˜ 2.19 (a) Verify that fn (·) defined in (3.7) satisfies fn (ω) ↑ f (ω) for all ω in Ω. (b) Verify that the sequence {hn }n≥1 of  Remark 2.3.3 satisfies limn→∞ hn = f a.e. (µ), and limn→∞ hn dµ = M . 2.20 Apply Corollary 2.3.5 to show that for any collection {aij : i, j ∈ N} of nonnegative numbers, ⎛ ⎞ ∞  ∞ ∞ ∞ ⎝ aij ⎠ = aij . i=1

j=1

j=1

i=1

2.21 Let g : R → R. (a) Recall that limt→∞ g(t) = L for some L in R if for every  > 0, there exists a T < ∞ such that t ≥ T ⇒ |g(t) − L| < . Show that limt→∞ g(t) = L for some L in R iff limn→∞ g(tn ) = L for every sequence {tn }n≥1 with limn→∞ tn = ∞.

2.6 Problems

77

(b) Formulate and prove a similar result when limt→a g(t) = L for some a, L ∈ R. 2.22 Let {ft : t ∈ R} ⊂ L1 (Ω, F, µ). (a) (The continuous version of the MCT). Suppose that ft ↑ f as t ↑ ∞ a.e. (µ) and for each t, ft ≥ 0 a.e. (µ). Show that   ft dµ ↑ f dµ. (b) (The continuous version of the DCT). Suppose there exists a nonnegative g ∈ L1 (Ω, F, µ) such that for each t, |ft | ≤ g a.e. 1  (µ). Then  f ∈ L (Ω, F, µ) and (µ) and as t → ∞, ft → f a.e. |ft − f |dµ → 0 and hence, ft dµ → f dµ, as t → ∞. 2.23 Let f : [a, b] → R be bounded where −∞ < a < b < ∞. Let {Pn }n≥1 be a sequence of partitions such that ∆(Pn ) → 0. Show that as n → ∞,   U (Pn , f ) → where



f

and L(Pn , f ) →

f,

 f and f are as defined in (4.3) and (4.4), respectively.

(Hint: Given  > 0, fix a partition P = {x0 = a < x1 < . . . < xk = b}  such that f < U (P, f ) < f + . Let δ = min0≤i≤k−1 (xi+1 − xi ). Choose n large such that the diameter ∆(Pn ) < δ. Verify that U (Pn , f ) < U (P, f ) + kB∆(Pn ) where B = sup{|f (x)| : a ≤ x ≤ b} and conclude that limn U (Pn , f ) ≤  f + .) 2.24 Establish (4.6) and (4.7). (Hint: Show that for every  x and any  > 0, φn (x) ≤ φ(x) +  for all n large and that for x ∈ n≥1 Pn , φn (x) ≥ φ(x) for all n.) 2.25 If f (x) = IQ1 (x) where Q1 = Q∩[0, 1], Q being the set of all rationals, then show that for any partition P , U (P, f ) = 1 and L(P, f ) = 0. 2.26 Establish Theorem 2.5.1. (Hint: Verify that D

≡ =

{ω : fj (ω) → f (ω)}



Ajr , r≥1 n≥1 j≥n

where Ajr = {|fj −f | > 1r }. Show that since µ(D) = 0 and µ(Ω) < ∞, µ(Drn ) → 0 as n → ∞ for each r ∈ N, where Drn = j≥n Ajr .)

78

2. Integration

2.27 Let φ : R+ → R+ be nondecreasing and φ(x) ↑ ∞ as x ↑ ∞. x 1 : λ ∈ Λ} be a subset of L (Ω, F, µ). Show that if Also, let {f λ  supλ∈Λ φ(|fλ |)dµ < ∞, then {fλ : λ ∈ Λ} is UI. 2.28 Let µ be the Lebesgue measure on ([−1, 1], B([−1, 1])). For n ≥ 1, define fn (x) = nI(0,n−1 ) (x) − nI(−n−1 ,0)  (x) and f (x) ≡ 0 for x ∈ [−1, 1]. Show that fn → f a.e. (µ) and fn dµ → f dµ but {fn }n≥1 is not UI. 2.29 Let {fn : n ≥ 1} ∪ {f } ⊂ L1 (Ω, F, µ).   (a) Show that |fn − f |(dµ) → 0 iff fn −→m f and |fn |dµ → |f |dµ. (b) Show further that if µ(Ω) < ∞ then the above two are equivalent to fn −→m f and {fn } UI. 2.30 For n ≥ 1, let fn (x) = n−1/2 I(0,n) (x), x ∈ R, and let f (x) = 0, x ∈ R. Let m denote the Lebesgue measure Show that fn → f  on (R, B(R)).  a.e. (m) and {fn }n≥1 is UI, but fn dm → f dm. 2.31 (Computing integrals w.r.t. the Lebesgue measure). Let f ∈ L1 (R, Mm , m) where (R, Mm , m) is the real line with Lebesgue σalgebra, andLebesgue measure, i.e., m = µ∗F where F (x) ≡ x. The + − + definition  − of f dm as f dm− f dm involves computing f dm and f dm which in turn is given in terms of approximating by integrals of simple nonnegative functions. This is not a very practical procedure. For f that happens to be continuous a.e. and bounded on finite intervals, one can compute the Riemann integral of f over finite intervals and pass to the limit. Justify the following steps: (a) Let f be continuous a.e. and bounded on finite intervals and f ∈ L1 (R, Mm , m). Show that for −∞ < a < b < ∞, f ∈ L1 ([a, b], Mm , m) and  ) f dm = f (x)dx, [a,b]

[a,b]

where the right side denotes the Riemann integral of f on [a, b]. (b) If, in addition, f ∈ L1 (R, Mm , m), then   f dm = a→−∞ lim f dm. R

b→+∞

[a,b]

(c) If f is continuous a.e. and ∈ L1 (R, Mm , m), then   f dm = lim φc (f )dm R

a→−∞ b→+∞ c→∞

[a,b]

where φc (f ) = f (x)I(|f (x)| ≤ c)+cI(f (x) > c)−cI(f (x) < −c).

2.6 Problems

(d) Apply the above procedure to compute (i) f (x) =

1 1+x2 , −x2 /2

(ii) f (x) = e

R

f dm for

,

+x −x2 /2

(iii) f (x) = e



79

e

.

2.32 (a) Let a ∈ R. Show that if a1 , a2 are nonnegative such that a = a1 − a2 then a1 ≥ a+ , a2 ≥ a− and a1 − a+ = a2 − a− . (b) Let f = f1 − f2 where f1 , f2 are nonnegative and   are in 1 1 (Ω, F, µ). Show that f ∈ L (Ω, F, µ) and f dµ = f1 dµ − L  f2 dµ. 2.33 Let (Ω, F, µ) be a measure space. Let f : Ω × (a, b) → R be such that for each a < t < b, f (·, t) ∈ L1 (Ω, F, µ). (a) Suppose for each a < t < b, (i) lim f (ω, t + h) = f (ω, t) a.e. (µ). h→0

(ii) sup |f (ω, t + h)| ≤ g1 (ω, t), where g1 (·, t) ∈ L1 (Ω, F, µ). |h|≤1

Show that φ(t) ≡

 Ω

f (ω, t)dµ is continuous on (a, b).

(b) Suppose for each a < t < b. f (ω, t + h) − f (ω, t) = g2 (ω, t) exists a.e. (µ), h * f (ω, t + h) − f (ω, t) * * * (ii) sup * * ≤ G(ω, t) a.e. (µ), h 0≤|h|≤1 (i) lim

h→0

(iii) G(ω, t) ∈ L1 (Ω, F, µ). 

Show that φ(t) ≡

f (ω, t)dµ Ω

is differentiable on (a, b). (Hint: Use the continuous version of DCT (cf. Problem 2.22).) 
2.34 Let A ≡ (aij ) be an infinite matrix of real numbers. Suppose that for each j, limi→∞ aij = aj exists in R and supi |aij | ≤ bj , where  ∞ j=1 bj < ∞. (a) Show by an application of the DCT that lim

i→∞

∞

|aij − aj | = 0.

j=1

(b) Show the same directly, i.e., without using the DCT.

80

2. Integration

2.35 Using the above problem or otherwise, show that for any sequence {xn }n≥1 with limn→∞ xn = x in R, ∞  xn n xj 1+ ≡ exp(x). = n→∞ n j! j=0

lim

2.36 (a) Let {fn }n≥1 ⊂ L1 (Ω, F, µ) such that fn → 0 in L1 (µ). Show that {fn }n≥1 is UI. (b) Let {fn }n≥1 ⊂ Lp (Ω, F, µ), 0 < p < ∞, such that µ(Ω) < ∞, {|fn |p }n≥1 is UI and fn −→m f . Show that f ∈ Lp (µ) and fn → f in Lp (µ). 2.37 (a) Show that for a sequence of real numbers {xn }n≥1 , limn→∞ xn = ¯ holds iff every subsequence {xn }j≥1 of {xn }n≥1 has a x ∈ R j further subsequence {xnjk }k≥1 such that limk→∞ xnjk = x. (b) Use (a) and Theorem 2.5.2 to show that the extended DCT (Theorem 2.3.11) is valid if the a.e. convergence of {fn }n≥1 and {gn }n≥1 is replaced by convergence in measure. 2.38 Let (R, Mµ∗F , µ∗F ) be a Lebesgue-Stieltjes measure space generated ¯ be Mµ∗ -measurable by a F : R → R nondecreasing. Let f : R → R F ∗ such that |f | < ∞ a.e. (µF ). Show that for every k ∈ N and for every , η ∈ (0, ∞), there exists a continuous function g : R → R such that g(x) = 0 for |x| > k and µ∗F ({x : |x| ≤ k, |f (x) − g(x)| > η} < . (Hint: Complete the following: Step I: For all  > 0, there exists Mk, ∈ (0, ∞) such that µ∗F ({x : |x| ≤ k, |f (x)| > Mk, }) < . Step II: For η > 0, there exists a simple function s(·) such that µ∗F ({x : |x| ≤ k, |f (x)| ≤ Mk, , |f (x) − s(x)| > η}) = 0. Step III: For δ > 0, there exists a continuous function g(·) such that g ≡ 0 for |x| > k and µ∗F ({x : |x| ≤ k, s(x) = g(x)}) < δ.) 1 2.39 Recall from  Corollary 2.3.6 that for g ∈ L (Ω, F, µ) and nonnegative function h, νg (A) ≡ A gdµ is a measure. Next for any F-measurable  show that h ∈ L1 (νg ) iff h · g ∈ L1 (µ) and hdνg = hgdµ.

(Hint: Verify first for h simple and nonnegative, next for h nonnegative, and finally for any h.) 2.40 Prove the BCT, Corollary 2.3.13, using Egorov’s theorem (Theorem 2.5.11). 2.41 Deduce the DCT from the BCT with the notation as in Corollary 2.3.12. (Hint: Apply the BCT to the measure space (Ω, F, νg ) and functions hn = fgn I(g > 0), h = fg I(g > 0) where νg is as in Problem 2.39.)

2.6 Problems

81

2.42 (Change of variables formula). Let (Ωi , Fi ), i = 1, 2 be two measurable spaces. Let f : Ω1 → Ω2 be F1 , F2 -measurable, h : Ω2 → R be F2 , B(R)-measurable, and µ1 be a measure on (Ω1 , F1 ). Show that g ≡ h ◦ f , i.e., g(ω) ≡ h(f (ω)) for ω ∈ Ω
1 is in L1 (µ) iff h(·) ∈ L1 (Ω2 , F2 , µ2 ) where µ2 = µ1 f −1 iff I(·) ∈ L1 R, B(R), µ3 ≡ µ2 h−1 where I(·) is the identity function in R, i.e., I(x) ≡ x for all x ∈ R and also that    Ω1

gdµ1 =

Ω2

hdµ2 =

R

xdµ3 .

2

2.43 Let φ(x) ≡ √12π e−x /2 be the standard N (0, 1) pdf on R. Let {(µn , σn )}n≥1 be a sequence in R × R+ µn → µ and 
. Suppose
x−µ 1 n , f (x) = φ σn → σ as n → ∞. Let fn (x) = σ1n φ x−µ σn σ σ   and νn (A) = A fn dm, ν(A) = A f dm for any Borel set A in R, where m(·) is the Lebesgue measure on R. Using Scheffe’s theorem, verify that, as n → ∞, νn (·) → ν(·) uniformly on B(R) and that for any h : R → R, bounded and Borel measurable,   hdνn → hdν.
n 2.44 Let fn (x) = cn 1 − nx I[0,n] (x), x ∈ R, n ≥ 1. (a) Find cn such that



fn dm = 1.

(b) Show that limn→∞ fn (x) ≡ f (x) exists for all x in R and that f is a pdf (c) For A ∈ B(R), let  νn (A) ≡

 fn dm

and ν(A) =

A

f dm. A

Show that νn → ν uniformly on B(R). 2.45 Let (Ω, F, µ) be  a measure space and f : Ω → R be F-measurable. Suppose that Ω |f |dµ < ∞ and D is a closed set in R such that for all B ∈ F with µ(B) > 0, 1 µ(B)

 f dµ ∈ D. B

Show that f (ω) ∈ D a.e. µ. (Hint: Show that for x ∈ D, there exists r > 0 such that µ{ω : |f (ω) − x| < r} = 0.)

82

2. Integration

2.46 Find a sequence of nonnegative continuous functions {fn }n≥1 on [0,1]  such that [0,1] fn dm → 0 but {fn (x)}n≥1 does not converge for any x. (Hint: Let for m ≥ 1, 1 ≤ k ≤ m, gm,k = I" k−1 a reordering of {gn,k : 1 ≤ k ≤ n, n ≥ 1}.)

m

k ,m

# and {fn }n≥1 be

2.47 Let {fn }n≥1 be a sequence of continuous functions from [0,1] to [0,1] such that fn (x) → 0 as n → ∞ for all 0 ≤ x ≤ 1. Show that 1 f (x)dx → 0 as n → ∞ (where the integral is the Riemann in0 n tegral) by two methods: one by using BCT and one without using BCT. Show also that if µ is a finite measure on ([0, 1], B([0, 1])), then f dµ → 0 as n → ∞. [0,1] n 2.48 (Invariance of Lebesgue measure under translation and reflection.) Let m(·) be Lebesgue measure on (R, B(R)). (a) For any E ∈ B(R) and c ∈ R, define −E ≡ {x : −x ∈ E} and E + c ≡ {y : y = x + c, x ∈ E}. Show that m(−E) = m(E) and m(E + c) = m(E) for all E ∈ B(R) and c ∈ R. (b) For any f ∈ L1 (R, B(R), m) and c ∈ R, let f˜(x) ≡ f (−x) and fc (x) ≡ f (x + c). Show that    ˜ f dm = fc dm = f dm.

3 Lp-Spaces

3.1 Inequalities This section contains a number of useful inequalities. Theorem 3.1.1: (Markov’s inequality). Let f be a nonnegative measurable function on a measure space (Ω, F, µ). Then for any 0 < t < ∞,  f dµ . (1.1) µ({f ≥ t}) ≤ t Proof: Since f is nonnegative,



f dµ ≥

 ({f ≥t})

f dµ ≥ tµ({f ≥ t}).

2

Corollary 3.1.2: Let X be a random variable on a probability space (Ω, F, P ). Then, for r > 0, t > 0, P (|X| ≥ t) ≤

E|X|r . tr

Proof: Since {|X| ≥ t} = {|X|r ≥ tr } for all t > 0, r > 0, this follows from (1.1). 2 Corollary 3.1.3: (Chebychev’s inequality). Let X be a random variable with EX 2 < ∞, E(X) = µ and Var(X) = σ 2 . Then for any 0 < k < ∞, P (|X − µ| ≥ kσ) ≤

1 . k2

84

3. Lp -Spaces

Proof: Follows from Corollary 3.1.2 with X replaced by X − µ and with r = 2, t = kσ. 2 Corollary 3.1.4: Let φ : R+ → R+ be nondecreasing. Then for any random variable X and 0 < t < ∞, P (|X| ≥ t) ≤

Eφ(|X|) . φ(t)

Proof: Use (1.1) and the fact that |X| ≥ t ⇒ φ(|X|) ≥ φ(t).

2

Corollary 3.1.5: (Cramer’s inequality). For any random variable X and t > 0, E(eθX ) . P (X ≥ t) ≤ inf θ>0 eθt 
Proof: For t > 0, θ > 0, P (X ≥ t) = P eθX ≥ eθt ≤

E(eθX ) , eθt

by (1.1). 2

Definition 3.1.1: A function φ : (a, b) → R is called convex if for all 0 ≤ λ ≤ 1, a < x ≤ y < b, φ(λx + (1 − λ)y) ≤ λφ(x) + (1 − λ)φ(y).

(1.2)

Geometrically, this means that for the graph of y = φ(x) on (a, b), for each fixed t ∈ (0, ∞), the chord over the interval (x, x + t) turns in the counterclockwise direction as x increases. More precisely, the following result holds. Proposition 3.1.6: Let φ : (a, b) → R. Then φ is convex iff for all a < x1 < x2 < x3 < b, φ(x3 ) − φ(x2 ) φ(x2 ) − φ(x1 ) ≤ , x2 − x1 x3 − x2

(1.3)

which is equivalent to φ(x3 ) − φ(x2 ) φ(x3 ) − φ(x1 ) φ(x2 ) − φ(x1 ) ≤ ≤ . x2 − x1 (x3 − x1 ) x3 − x2

(1.4)

Proof: Let φ be convex and a < x1 < x2 < x3 < b. Then one can write 3 −x2 ) x2 = λx1 + (1 − λ)x3 with λ = (x (x3 −x1 ) . So by (1.2), φ(x2 )

= φ(λx1 + (1 − λ)x3 ) ≤ λφ(x1 ) + (1 − λ)φ(x3 ) (x3 − x2 ) (x2 − x1 ) = φ(x1 ) + φ(x3 ) (x3 − x1 ) (x3 − x1 )

3.1 Inequalities

85

which is equivalent to (1.3). Also, since φ(x3 ) − φ(x2 ) φ(x2 ) − φ(x1 ) φ(x3 ) − φ(x1 ) =λ + (1 − λ) , (x3 − x1 ) (x3 − x2 ) (x2 − x1 ) (1.4) follows from (1.3). Conversely, suppose (1.4) holds for all a < x1 < x2 < x3 < b. Then given a < x < y < b and 0 < λ < 1, set x1 = x, x2 = λx + (1 − λ)y, x3 = y and apply (1.4) to verify (1.2). 2 The following properties of a convex function are direct consequences of (1.3). The proof is left as an exercise (Problem 3.1). Proposition 3.1.7: Let φ : (a, b) → R be convex. Then, (i) For each x ∈ (a, b), φ+ (x) ≡ lim y↓x

φ(y) − φ(x) , (y − x)

φ− (x) ≡ lim y↑x

φ(y) − φ(x) (y − x)

exist and are finite. (ii) Further, φ− (·) ≤ φ+ (·) and both are nondecreasing on (a, b). (iii) φ (·) exists except on the countable set of discontinuity points of φ+ and φ− . (iv) For any a < c < d < b, φ is Lipschitz on [c, d], i.e., there exists a constant K < ∞ such that |φ(x) − φ(y)| ≤ K|x − y|

for all

c ≤ x, y ≤ d.

(v) For any a < c, x < b, φ(x) − φ(c) ≥ φ+ (c)(x − c)

and φ(x) − φ(c) ≥ φ− (c)(x − c). (1.5)

By the mean value theorem, a sufficient condition for (1.3) and hence, for the convexity of φ is that φ be differentiable on (a, b) and φ be nondecreasing. A further sufficient condition for this is that φ be twice differentiable on (a, b) and φ be nonnegative. This is stated as Proposition 3.1.8: Let φ be twice differentiable on (a, b) and φ be nonnegative on (a, b). Then φ is convex on (a, b). Example 3.1.1: The following functions are convex in the given intervals: (a) φ(x) = |x|p , p ≥ 1, (b) φ(x) = ex ,

(−∞, ∞).

(−∞, ∞).

86

3. Lp -Spaces

(c) φ(x) = − log x,

(0, ∞).

(d) φ(x) = x log x,

(0, ∞).

Remark 3.1.1: By Proposition 3.1.7 (iii), the convexity of φ implies that φ exists except on a countable set. For example, the function φ(x) = |x| is convex on R; it is differentiable at all x = 0. Similarly, it is easy to construct a piecewise linear convex function φ with a countable number of points where φ is not differentiable. The following is an important inequality for convex functions. Theorem 3.1.9: (Jensen’s inequality). Let f be a measurable function on a probability space (Ω, F, P ) with P (f ∈ (a, b)) = 1 for some interval (a, b), −∞ ≤ a < b ≤ ∞ and let φ : (a, b) → R be convex. Then    φ f dP ≤ φ(f )dP, (1.6) provided



|f |dP < ∞ and



|φ(f )|dP < ∞.

Remark 3.1.2: In terms of random variables, this says that for any random variable X on a probability space (Ω, F, P ) with P (X ∈ (a, b)) = 1 and for any function φ that is convex on (a, b), φ(EX) ≤ Eφ(X),

(1.7)

provided E|X| < ∞ and  E|φ(X)| < ∞, where for any Borel measurable function h, Eh(X) ≡ h(X)dP .  Proof of Theorem 3.1.9: Let c = f dP . Applying (1.5), one gets


(1.8) Y (ω) ≡ φ(f (ω)) − φ(c) − φ+ (c)(f (ω) − c) ≥ 0 a.e. (P ),   which, when integrated, yields Y (ω)P (dω) ≥ 0. Since (f (ω) − c) P (dω) = 0, (1.6) follows. 2 Remark 3.1.3: Suppose that equality holds in (1.6). Then, it follows that  Y (ω)P (dω) = 0. By (1.8), this implies φ(f (ω)) − φ(c) = φ+ (c)(f (ω) − c)

a.e. (P ).

Thus, if φ is a strictly convex function (i.e., strict inequality holds in (1.2) for all x, y ∈ (a, b) and 0 < λ < 1), then equality holds in (1.6) iff f (ω) = c a.e. (P ). The following are easy consequences of Jensen’s inequality (Problem 3.3). Proposition 3.1.10: Let k ≥ 1 be an integer.

3.1 Inequalities

87

(i) Let a1 , a2 , , . . . , ak be real and p1 , p2 , , . . . , pk be positive numbers such k that i=1 pi = 1. Then k

pi exp(ai ) ≥ exp

k

i=1

 pi ai .

(1.9)

i=1

(ii) Let b1 , b2 , . . . , bk be nonnegative numbers and p1 , p2 , . . . , pk be as in (i). Then k k + pi bi ≥ bpi i , (1.10) i=1

and in particular,

i=1

k

+ k k 1 bi ≥ bi , k i=1 i=1 1

(1.11)

i.e., the arithmetic mean of b1 , . . . , bk is greater than or equal to the geometric mean of the bi ’s. Further, equality holds in (1.10) iff b1 = b2 = . . . = bk . (iii) For any a, b real and 1 ≤ p < ∞, |a + b|p ≤ 2p−1 (|a|p + |b|p ).

(1.12)

Inequality (1.10) is useful in establishing the following: Theorem 3.1.11: (H¨ older’s inequality). Let (Ω, F, µ) be a measure space. p . Let 1 < p < ∞, f ∈ Lp (Ω, F, µ) and g ∈ Lq (Ω, F, µ) where q = (p−1) Then

  p1   q1  p q |f g|dµ ≤ |f | dµ (1.13) |g| dµ , i.e.,

f g1

≤

f p gq .

If f g1 = 0, then equality holds in (1.13) iff |f |p = c|g|q a.e. (µ) for some constant c ∈ (0, ∞).   Proof: W.l.o.g. assume that |f |p dµ > 0 and |g|q dµ > 0. Fixω ∈ Ω. Let p1 = p1 , p2 = 1q , b1 = c1 |f (ω)|p , and b2 = c2 |g(ω)|q , where c1 = ( |f |p dµ)−1  and c2 = ( |g|q dµ)−1 . Then applying (1.10) with k = 2 yields 1 1 c2 c1 |f (ω)|p + |g(ω)|q ≥ c1p c2q |f (ω)g(ω)|. p q

Integrating w.r.t. µ yields 1

1

1 ≥ c1p c2q

 |f (ω)g(ω)|dµ(ω)

(1.14)

3. Lp -Spaces

88

which is equivalent to (1.13). Next, equality in (1.13) implies equality in (1.14) a.e. (µ). Since 1 < p < ∞, by the last part of Proposition 3.1.10 (ii), this implies that b1 = b2 a.e. q 2 (µ), i.e., |f (ω)|p = c2 c−1 1 |g(ω)| a.e. (µ). Remark 3.1.4: (H¨ older’s inequality for p = 1, q = ∞). Let f ∈ L1 (Ω, F, µ) and g ∈ L∞ (Ω, F, µ). Then |f g| ≤ |f | g∞ a.e. (µ) and hence,  f g1 ≡ |f g|dµ ≤ f 1 g∞ . If equality holds in the above inequality, then |f |(g∞ − |g|) = 0 a.e. (µ) and hence, |g| = g∞ on the set {|f | > 0} a.e. (µ). The next two corollaries follow directly from Theorem 3.1.11. The proof is left as an exercise (Problem 3.9). Corollary 3.1.12: (Cauchy-Schwarz inequality). Let f , g ∈ L2 (Ω, F, µ). Then

  12   12  2 2 |f g|dµ ≤ |f | dµ |g| dµ , (1.15) i.e.,

f g1

≤

f 2 g2 .

Corollary 3.1.13: Let k ∈ N. Let a1 , a2 , . . . , ak , b1 , b2 , . . . , bk be real numbers and c1 , c2 , . . . , ck be positive real numbers. (i) Then, for any 1 < p < ∞,  k  p1  k  q1 k |ai bi |ci ≤ |ai |p ci |bi |q ci , i=1

where q =

i=1

(1.16)

i=1

p−1 p .

(ii) k

|ai bi |ci ≤

i=1

 k

2

|ai | ci

 12  k

i=1

 12 2

|bi | ci

.

(1.17)

i=1

Next, as an application of H¨ older’s inequality, one gets Theorem 3.1.14: (Minkowski’s inequality). Let 1 < p < ∞ and f, g ∈ Lp (Ω, F, µ). Then

  p1

  p1   p1 p p p |f + g| dµ ≤ |f | dµ + |g| dµ , i.e.,

f + gp

≤

f p + gq .

(1.18)

3.2 Lp -Spaces

89

Proof: Let h1 = |f + g|, h2 = |f + g|p−1 . Then by (1.12), |f + g|p ≤ 2p−1 (|f |p + |g|p ), implying that h1 ∈ Lp (Ω, F, µ) and h2 ∈ Lq (Ω, F, µ), where q = older’s inequality, Since |f + g|p = h1 h2 ≤ |f |h2 + |g|h2 , by H¨



 |f +g|p dµ ≤ But



hq2 =



|f |p dµ

 p1 

hq2

 q1

 +

|g|p dµ

 p1 

hq2

p (p−1) .

 q1 . (1.19)

|f + g|p and so (1.19) yields (1.18).

2

Remark 3.1.5: Inequality (1.18) holds for both p = 1 and p = ∞.

3.2 Lp -Spaces 3.2.1

Basic properties

Let (Ω, F, µ) be a measure space. Recall the definition of Lp (Ω, F, µ), 0 < p ≤ ∞, from Section 2.5 as the set of all measurable functions f on (Ω, F, µ) such that f p < ∞, where for 0 < p < ∞,  min{ p1 ,1} |f |p dµ f p = and for p = ∞, f ∞ ≡ sup{k : µ({|f | > k}) > 0} (called the essential supremum of f ). In this section and elsewhere, Lp (µ) denotes Lp (Ω, F, µ). The following proposition shows that Lp (µ) is a vector space over R. Proposition 3.2.1: For 0 < p ≤ ∞, f, g ∈ Lp (µ), a, b ∈ R

⇒

af + bg ∈ Lp (µ).

(2.1)

Proof: Case 1: 0 < p ≤ 1. For any two positive numbers x, y,

p p y y x x + = 1. + ≥ x+y x+y x+y x+y Hence, for all x, y ∈ (0, ∞) (x + y)p ≤ xp + y p .

(2.2)

90

3. Lp -Spaces

It is easy to check that (2.2) continues to hold if x, y ∈ [0, ∞). This yields |af + bg|p ≤ |a|p |f |p + |b|p |g|p , which, in turn, yields (2.1) by integration. Case 2: 1 < p < ∞. By (1.12), |af + bg|p ≤ 2p−1 (|af |p + |bg|p ). Integrating both sides of the above inequality yields (2.1). Case 3: p = ∞. By definition, there exist constants K1 < ∞ and K2 < ∞ such that µ({|f | > K1 }) = 0 = µ({|g| > K2 }). This implies that µ({|af + bg| > K}) = 0 for any K > |a|K1 + |b|K2 . Hence, af + bg ∈ L∞ (µ) and af + bg∞ ≤ |a| f ∞ + |b| g∞ . 2 Recall that a set S with a function d : S × S → [0, ∞] is called a metric space if for all x, y, z ∈ S, (i) d(x, y) = d(y, x)

(symmetry)

(ii) d(x, y) ≤ d(x, z) + d(y, z)

(triangle inequality)

(iii) d(x, y) = 0 iff x = y and the function d(·, ·) is called a metric on S. Some examples are (a) S = Rk with d(x, y) =

 k

2 i=1 |xi − yi |

 12

;

(b) S = C[0, 1], the space of all continuous functions on [0, 1] with d(f, g) = sup{|f (x) − g(x)| : 0 ≤ x ≤ 1}; (c) S = a nonempty set, and d(x, y) = 1 if x = y and 0 if x = y. (This d(·, ·) is called the discrete metric on S.) The Lp -norm  · p can be used to introduce a distance notion in Lp (µ) for 0 < p ≤ ∞. Definition 3.2.1: For f , g ∈ Lp (µ), 0 < p ≤ ∞, let dp (f, g) ≡ f − gp . Note that, for any f , g, h ∈ Lp (µ) and 1 ≤ p ≤ ∞, (i) dp (f, g) = dp (g, f ) ≥ 0

(nonnegativity and symmetry), and

(ii) dp (f, h) ≤ dp (f, g) + dp (g, h)

(triangle inequality),

(2.3)

3.2 Lp -Spaces

91

which follows by Minkowski’s inequality (Theorem 3.1.14) for 1 ≤ p < ∞, and by Proposition 3.2.1 for p = ∞. However, dp (f, g) = 0 implies only that f = g a.e. (µ). Thus, dp (·, ·) of (2.3) satisfies conditions (i) and (ii) of being a metric and it satisfies condition (iii) as well, provided any two functions f and g that agree a.e. (µ) are regarded as the same element of Lp (µ). This leads to the following: Definition 3.2.2: For f , g in Lp (µ), f is called equivalent to g and is written as f ∼ g, if f = g a.e. (µ). It is easy to verify that the relation ∼ of Definition 3.2.2 is an equivalence relation, i.e., it satisfies (i) f ∼ f for all f in Lp (µ) (ii) f ∼ g ⇒ g ∼ f

(reflexive)

(symmetry)

(iii) f ∼ g, g ∼ h ⇒ f ∼ h

(transitive).

This equivalence relation ∼ divides Lp (µ) into disjoint equivalence classes such that in each class all elements are equivalent. The notion of distance between these classes may be defined as follows: dp ([f ], [g]) ≡ dp (f, g) where [f ] and [g] denote, respectively, the equivalence classes of functions containing f and g. It can be verified that this is a metric on the set of equivalence classes. In what follows, the equivalence class [f ] is identified with the element f . With this identification, (Lp (µ), dp (·, ·)) becomes a metric space for 1 ≤ p ≤ ∞. Remark 3.2.1: For 0 < p < 1, if one defines  dp (f, g) ≡ |f − g|p dµ,

(2.4)

then (Lp (µ), dp ) becomes a metric space (with the same identification as above of functions with their equivalence classes). The triangle inequality follows from (2.2). Recall that a metric space (S, d) is called complete if every Cauchy sequence in (S, d) converges to an element in S, i.e., if {xn }n≥1 is a sequence in S such that for every  > 0, there exists a N such that n, m ≥ N ⇒ d(xn , xm ) ≤ , then there exists an element x in (S, d) such that limn→∞ d(xn , x) = 0. The next step is to establish the completeness of Lp (µ). Theorem 3.2.2: For 0 < p ≤ ∞, (Lp (µ), dp (·, ·)) is complete, where dp is as in (2.3).

92

3. Lp -Spaces

Proof: Let {fn }n≥1 be a Cauchy sequence in Lp (µ) for 0 < p < ∞. The main steps in the proof are as follows: (I) there exists a subsequence {nk }k≥1 such that {fnk }k≥1 converges a.e. (µ) to a limit function f ; (II) lim dp (fnk , f ) = 0; k→∞

(III) lim dp (fn , f ) = 0. n→∞

Step (I): Let {k }k≥1 and {δk }k≥1 be sequences of positive numbers decreasing to zero. Since {fn }n≥1 is Cauchy, for each k ≥ 1, there exists an integer nk such that  (2.5) |fn − fm |p dµ ≤ k for all n, m ≥ nk . W.l.o.g., let nk+1 > nk for each k ≥ 1. Then, by Markov’s inequality (Theorem 3.1.1),  −p µ({|fnk+1 − fnk | ≥ δk }) ≤ δk |fnk+1 − fnk |p dµ ≤ δk−p k . (2.6) Let A k = {|fnk+1 − fnk | ≥ δk }, k = 1, 2, . . . and A = lim supk→∞ Ak ≡  ∞ k≥j Ak . If {k }k≥1 and {δk }k≥1 satisfy j=1 ∞

δk−p k < ∞,

(2.7)

k=1

∞ then by (2.6), k=1 µ(Ak ) < ∞ and hence, as in the proof of Theorem 2.5.2, µ(A) = 0. that for ω in Ac , |fnk+1 (ω) − fnk (ω)| < δk for all k large. Thus, if Note ∞ c k=1 δk < ∞, then for ω in A , {fnk (ω)}k≥1 is a Cauchy sequence in R and hence, it converges to some f (ω) in R. Setting f (ω) = 0 for ω ∈ A, one gets lim fnk = f a.e. (µ). k→∞

∞ A choice of {k }k≥1 and {δk }k≥1 such that k=1 δk < ∞ and (2.7) holds is given by k = 2−(p+1)k and δk = 2−k . This completes Step (I). Step (II): By Fatou’s lemma, part (I), and (2.5), for any k ≥ 1 fixed,   k ≥ lim inf |fnk − fnk+j |p dµ ≥ |fnk − f |p dµ . j→∞

Since fnk ∈ Lp (µ), this shows that f ∈ Lp (µ). Now, on letting k → ∞, (II) follows.

3.2 Lp -Spaces

93

Step (III): By triangle inequality, for any k ≥ 1 fixed, dp (fn , f ) ≤ dp (fn , fnk ) + dp (fnk , f ). By (2.5) and (II), for n ≥ nk , the right side above is ≤ 2˜ k , where ˜k = k 1/p if 0 < p < 1 and ˜k = k if 1 ≤ p < ∞. Now letting k → ∞, (III) follows. The proof of Theorem 3.2.2 is complete for 0 < p < ∞. The case p = ∞ is left as an exercise (Problem 3.14). 2

3.2.2

Dual spaces

Let 1 ≤ p < ∞. Let g ∈ Lq (µ), where q = if p = 1. Let  Tg (f ) = By H¨olders inequality, Tg is linear, i.e.,



f gdµ,

p (p−1)

if 1 < p < ∞ and q = ∞

f ∈ Lp (µ).

(2.8)

|f g|dµ < ∞ and so Tg (·) is well defined. Clearly

Tg (α1 f1 + α2 f2 ) = α1 Tg (f1 ) + α2 Tg (f2 )

(2.9)

for all α1 , α2 ∈ R and f1 , f2 ∈ Lp (µ). Definition 3.2.3: (a) A function T : Lp (µ) → R that satisfies (2.9) is called a linear functional. (b) A linear functional T on Lp (µ) is called bounded if there is a constant c ∈ (0, ∞) such that |T (f )| ≤ cf p

for all f ∈ Lp (µ).

(c) The norm of a bounded linear functional T on Lp (µ) is defined as   T  = sup |T f | : f ∈ Lp (µ), f p = 1 . By H¨older’s inequality (cf. Theorem 3.1.11 and Remark 3.1.4), |Tg (f )| ≤ gq f p

for all f ∈ Lp (µ),

and hence, Tg is a bounded linear functional on Lp (µ). This implies that if dp (fn , f ) → 0, then |Tg (fn ) − Tg (f )| ≤ gq dp (fn , f ) → 0, i.e., Tg is continuous on the metric space (Lp (µ), dp ).

3. Lp -Spaces

94

Definition 3.2.4: The set of all continuous linear functionals on Lp (µ) is called the dual space of Lp (µ) and is denoted by (Lp (µ))∗ . In the next section, it will be shown that continuity of a linear functional on Lp (µ) implies boundedness. A natural question is whether every continuous linear functional T on Lp (µ) coincides with Tg for some g in Lq (µ). The answer is “yes” for 1 ≤ p < ∞, as shown by the following result. Theorem 3.2.3: (Riesz representation theorem). Let 1 ≤ p < ∞. Let T : Lp (µ) → R be linear and continuous. Then, there exists a g in Lq (µ) such that T = Tg , i.e.,  T (f ) = Tg (f ) ≡ f gdµ for all f ∈ Lp (µ), (2.10) where q =

p p−1

for 1 < p < ∞ and q = ∞ if p = 1.

Remark 3.2.2: Such a representation is not valid for p = ∞. That is, there exists continuous linear functionals T on L∞ (µ) for which there is no g ∈ L1 (µ) such that T (f ) = f gdµ for all f ∈ L∞ (µ), provided µ is not concentrated on a finite set {ω1 , ω2 , . . . , ωk } ⊂ Ω. For a proof of Theorem 3.2.3 and the above remark, see Royden (1988) or Rudin (1987). Next consider the mapping from (Lp (µ))∗ and Lq (µ) defined by φ(Tg ) = g, where Tg is as defined in (2.10). Then, φ is linear, i.e., φ(α1 T1 + α2 T2 ) = α1 φ(T1 ) + α2 φ(T2 ) for all α1 , α2 ∈ R and T1 , T2 ∈ (Lp (µ))∗ . Further, φ(T )q = T 

for all T ∈ (Lp (µ))∗ .

Thus, φ preserves the vector space structure of (Lp (µ))∗ and the norm. For this reason, it is called an isometry between (Lp (µ))∗ and Lq (µ).

3.3 Banach and Hilbert spaces 3.3.1

Banach spaces

If (Ω, F, µ) is a measure space it was seen in the previous section that the  space Lp (Ω, F, µ) of equivalence classes of functions f with Ω |f |p dµ < ∞ is a vector space over R for all 1 ≤ p < ∞ and for p ≥ 1,  · p ≡ ( |f |p dµ)1/p satisfies

3.3 Banach and Hilbert spaces

95

(i) f + g ≤ f  + g, (ii) αf  = |a| f  for every α ∈ R, (iii) f  = 0 iff f = 0 a.e. (µ). The Euclidean spaces Rk for any kk ∈ N is also a vector space. Note that for p ≥ 1, setting xp ≡ ( i=1 |xi |p )1/p if x = (x1 , x2 , . . . , xk ), (Rk , xp ) may be identified with a special case of Lp (Ω, F, µ), where Ω ≡ {1, 2, . . . , k}, F = P(Ω) and µ is the counting measure. Generalizing the above examples leads to the notion of a normed vector space (also called normed linear space). Recall that a vector space V over R is a nonempty set with a binary operation +, a function from V × V to V (called addition), and scalar by the real numbers, i.e., a
multiplication  function from R × V → V , (α, v) → αv satisfying (i) v1 , v2 ∈ V ⇒ v1 + v2 = v2 + v1 ∈ V . (ii) v1 , v2 , v3 ∈ V ⇒ (v1 + v2 ) + v3 = v1 + (v2 + v3 ). (iii) There exists an element θ, called the zero vector, in V such that v + θ = v for all v in V . (iv) α ∈ R, v ∈ V ⇒ αv ∈ V . (v) α ∈ R, v1 , v2 ∈ V ⇒ α(v1 + v2 ) = αv1 + αv2 . (vi) α1 , α2 ∈ R, v ∈ V ⇒ (α1 +α2 )v = α1 v +α2 v and α1 (α2 v) = (α1 α2 )v. (vii) v ∈ V ⇒ 0v = θ and 1v = v. Note that from conditions (vi) and (vii) above, it follows that for any v in V , v + (−1)v = 0 · v = θ. Thus for any v in V , (−1)v is the additive inverse and is denoted by −v. Conditions (i), (ii), and (iii) are called respectively commutativity, associativity, and the existence of an additive identity. Thus V under the operation + is an Abelian (i.e., commutative) group. Definition 3.3.1: A function f from V to R+ denoted by f (v) ≡ v is called a norm if (a) v1 , v2 ∈ V ⇒ v1 + v2  ≤ v1  + v2  (b) α ∈ R, v ∈ V ⇒ αv = |α| v

(triangle inequality)

(scalar homogeneity)

(c) v = 0 iff v = θ. A vector space V with a norm  ·  defined on it is called a normed vector space or normed linear space and is denoted as (V,  · ). Let d(v1 , v2 ) ≡ v1 − v2 , v1 , v2 ∈ V . Then from the definition of  · , it follows that d is a metric on V , i.e., (V, d) is a metric space. Recall that a metric space (S, d)

96

3. Lp -Spaces

is called complete if every Cauchy sequence {xn }n≥1 in S converges to an element x in S. Definition 3.3.2: A Banach space is a complete normed linear space (V,  · ). It was shown by S. Banach of Poland that all Lp (Ω, B, µ) spaces are Banach spaces, provided p ≥ 1 and in particular, all Euclidean spaces are Banach spaces. An example of a different kind is the space C[0, 1] of all real valued continuous functions on [0, 1] with the usual operation of pointwise addition and scalar multiplication, i.e., (f + g)(x) = f (x) + g(x) and (αf )(x) = α · f (x) for all α ∈ R, 0 ≤ x ≤ 1, f , g ∈ C[0, 1] where the norm (called the supnorm) is defined by f  = sup{|f (x)| = 0 ≤ x ≤ 1}. The verification of the fact that C[0, 1] with the supnorm is a Banach space is left as an exercise (Problem 3.22). The space P of all polynomials on [0, 1] is also a normed linear space under the above norm but (P,  · ) is not complete (Problem 3.23). However for each n ∈ N, the space Pn of all polynomials on [0, 1] of degree ≤ n is a Banach space under the supnorm (Problem 3.26). Definition 3.3.3: Let V be a vector space. A subset W ⊂ V is called a subspace of V if v1 , v2 ∈ W, α1 , α2 ∈ R ⇒ α1 v1 + α2 v2 ∈ W . If (V,  · ) is a normed vector space and W is a subspace of V , then (W,  · ) is also a normed vector space. If W is closed in (V,  · ), then W is called a closed subspace of V . Remark 3.3.1: If (V,  · ) is a Banach space and W is a closed subspace of V , then (W,  · ) is also a Banach space.

3.3.2

Linear transformations

Let (Vi ,  · i ), i = 1, 2 be two normed linear spaces over R. Definition 3.3.4: A function T from V1 to V2 is called a linear transformation or linear operator if α1 , α2 ∈ R, x, y ∈ V1 ⇒ T (α1 x + α2 y) = α1 T (x) + α2 T (y). Definition 3.3.5: A linear operator T from (V1 ,  · 1 ) to (V2 ,  · 2 ) is called bounded if T  ≡ sup{T x2 : x1 < 1} < ∞, i.e., the image of the unit ball in (V1 ,  · 1 ) is contained in a ball of finite radius centered at the zero in V2 . Here is a summary of some important results on this topic. By linearity x 1 ) = x T (x) for any x = 0. It follows that T is bounded iff of T , T ( x there exists k < ∞ such that for any x ∈ V1 , T x2 ≤ kx1 . Clearly, then k can be taken to be T . Also by linearity, if T is bounded, then T x1 −T x2  = T (x1 −x2 ) ≤ T  x1 −x2  and so the map T is continuous

3.3 Banach and Hilbert spaces

97

(indeed, uniformly so). It turns out that if a linear operator T is continuous at some x0 in V1 , then T is continuous on all of V1 and is bounded (Problem 3.28 (a)). Now let B(V1 , V2 ) be the space of all bounded linear operators from (V1 , ·1 ) to (V2 , ·2 ). For T1 , T2 ∈ B(V1 , V2 ), α1 , α2 in R, let (α1 T1 +α2 T2 ) be defined by (α1 T1 + α2 T2 )(x) ≡ α1 T1 (x) + α2 T1 (x) for all x in V1 . Then it can be verified that (α1 T1 + α2 T2 ) also belongs to B(V1 , V2 ) and T  ≡ sup{T x2 : x1 ≤ 1}

(3.1)

is a norm on B(V1 , V2 ). Thus (B(V1 , V2 ),  · ) is also a normed linear space. If (V2 ,  · 2 ) is complete, then it can be shown that (B(V1 , V2 ),  · ) is also a Banach space (Problem 3.28 (b)). In particular, if (V2 ,  · 2 ) is the real line, the space (B(V1 , R),  · ) is a Banach space.

3.3.3

Dual spaces

Definition 3.3.6: The space of all bounded linear functions from (V1 , ·) to R (also called bounded linear functionals), denoted by V1∗ , is called the dual space of V1 . Thus, for any normed linear space (V1 , ·1 ) (that need not be complete), the dual space (V1∗ ,  · ) is always a Banach space, where T  ≡ sup{|T x| : x1 < 1} for T ∈ V1∗ . If (V1 ,  · 1 ) = Lp (Ω, F, µ) for some measure space (Ω, F, µ) and 1 ≤ p < ∞, by the Riesz representation theorem (see Theorem 3.2.3), the dual space may be identified with Lq (Ω, F, µ) where q is the conjugate of p, i.e., p1 + 1q = 1. However, as pointed out earlier in Section 3.2, this is not true for p = ∞. That is, the dual of L∞ (Ω, F, µ) is not L1 (Ω, F, µ) unless (Ω, F, µ) is a measure space where Ω is a finite set {w1 , w2 , . . . , wk } and F = P(Ω). An example for the p = ∞ case can be constructed for the space ∞ of all bounded sequences of real numbers (cf. Royden (1988)). The representation of the dual space of the Banach space C[0, 1] with supnorm is in terms of finite signed measures (cf. Section 4.2). Theorem 3.3.1: (Riesz ). Let T : C[0, 1] → R be linear and bounded. Then there exists two finite measures µ1 and µ2 on [0, 1] such that for any f ∈ C[0, 1]   T (f ) = f dµ1 − f dµ2 .

For a proof see Royden (1988) or Rudin (1987) (see also Problem 3.27).

3. Lp -Spaces

98

3.3.4

Hilbert space

A vector space V over R is called a real innerproduct space if there exists a function f : V × V → R, denoted by f (x, y) ≡ x, y (and called the innerproduct) that satisfies (i) x, y = y, x for all x, y ∈ V , (ii) (linearity) α1 x1 + α2 x2 , y = α1 x, y + α2 x2 , y for all α1 , α2 ∈ R, x1 , x2 , y ∈ V , (iii) x, x ≥ 0 for all x ∈ V and x, x = 0 iff x = θ, the zero vector of V . Using the fact that the quadratic function ϕ(t) = x + ty, x + ty = x, x + 2tx, y + t2 y, y is nonnegative for all t ∈ R, one gets the CauchySchwarz inequality , |x, y| ≤ x, xy, y for all x, y ∈ V. , Now setting x = x, x and using the Cauchy-Schwarz inequality, one verifies that x is a norm on V and thus (V,  · ) is a normed linear space. Further, the function x, y from V × V to R is continuous (Problem 3.29) under the norm (x1 , x2 ) = x1  + x2 , (x1 , x2 ) ∈ V × V . Definition 3.3.7: Let (V, ·, ·) be a real innerproduct space. It is called a Hilbert space if (V,  · ) is a Banach space, i.e., if it is complete. It was seen in Section 3.2 that for any measure space (Ω, F, µ), the space 2 F, µ) of all equivalence classes of functions f : Ω → R satisfying L  (Ω, |f |2 dµ < ∞ is a complete innerproduct space with the innerproduct f, g = f gdµ and hence a Hilbert space. It turns out that every Hilbert space H is an L2 (Ω, F, µ) for some (Ω, F, µ). (The axiom of choice or its equivalent, the Hausdorff’s maximality principle, is required for a proof of this. See Rudin (1987).) This is in contrast to the Banach space case where every Lp (Ω, F, µ) with p ≥ 1 is a Banach space but not conversely, i.e., every Banach space need not be an Lp (Ω, F, µ). Next for each x in a Hilbert space H, let Tx : H → R be defined by Tx (y) = x, y. By the defining properties of x, y and the Cauchy-Schwarz inequality, it is easy to verify that Tx is a bounded linear function on H, i.e., Tx (α1 y1 + α2 y2 ) = α1 Tx (y1 ) + α2 Tx (y2 )

for all α1 , α2 ∈ R, y1 , y2 ∈ H (3.2)

and |Tx (y)| ≤ x y ∗

for all y ∈ H.

(3.3)

Thus Tx ∈ H , the dual space. It is an important result (see Theorem 3.3.3 below) that every T ∈ H ∗ is equal to Tx for some x in H and T  = x. Thus H ∗ can be identified with H.

3.3 Banach and Hilbert spaces

99

Definition 3.3.8: Let (V, ·, ·) be an inner product space. Two vectors x, y in V are said to be orthogonal and written as x ⊥ y if x, y = 0. A collection B ⊂ V is called orthogonal if x, y ∈ B, x = y ⇒ x, y = 0. The collection B is called orthonormal if it is orthogonal and in addition for all x in B, x = 1. Note that if x ⊥ y, then x − y2 = x − y, x − y = x, x + y, y = x2 + y2 and√so if B is an orthonormal set, then x, y ∈ B ⇒ either x = y or x − y = 2. Thus, if V is separable under the metric d(x, y) = x − y (i.e., there exists a countable set D ⊂ V such that for every x in V and  > 0, there exists a d ∈ D such that x − d < ) and 1 if B ⊂ V is an orthonormal system, then the open ball Sb of radius 2√ 2 around each b ∈ B satisfies {Sb ∩ D : b ∈ B} are disjoint and nonempty. Thus B is countable. Now let (V, ·, ·) be a separable innerproduct space and B ⊂ V be an orthonormal system. Definition 3.3.9: The Fourier coefficients of a vector x in V with respect on orthonormal set B is the set {x, b : b ∈ B}. Since V is separable, B is countable. Let nB = {bi : i ∈ N}. For a given x ∈ V , let ci = x, bi , i ≥ 1. Let xn ≡ i=1 ci bi , n ∈ N. The sequence {xn }n≥1 is called the partial sum sequence of the Fourier expansion of the vector x w.r.t. the orthonormal set B. A natural question is: when does {xn }n≥1 converge to x? By the linearity property in the definition of the innerproduct ·, ·, it follows that 0 ≤ x − xn 2 = x − xn , x − xn  = x, x − 2x, xn  + xn , xn  and x, xn  =

n

ci x, bi  =

i=1

n

c2i .

i=1

Since {bi }i≥1 are orthonormal, xn 2 = xn , xn  =

n

c2i = x, xn .

i=1

Thus, 0 ≤ x − xn 2 = x2 − xn 2 = x2 −

n

c2i ,

i=1

leading to Proposition 3.3.2: (Bessel’s inequality). Let {bi }i≥1 be orthonormal in an innerproduct space (V, ·, ·). Then, for any x in V , ∞ i=1

x, bi 2 ≤ x2 .

(3.4)

100

3. Lp -Spaces

Now let (V, ·, ·) be a Hilbert space. Since for m > n, xn − xm 2 =

m

x, bi 2 ,

i=n+1

it follows from Bessel’s inequality that {xn }n≥1 is a Cauchy sequence, and since V is complete, there is a y in V such that xn → y. This implies (by the continuity of x, y) that x, bi  = limn→∞ xn , bi  = y, bi  ⇒ x−y, bi  = 0 for all i ≥ 1. Thus, it follows that x, bi  = y, bi  for all i ≥ 1. The last relation implies y = x iff the set {bi }i≥1 satisfies the property that z, bi  = 0

for all i ≥ 1 ⇒ z = 0.

(3.5)

This property is called the completeness of B. Thus B ≡ {bi }i≥1 is a complete orthonormal set for a Hilbert space H ≡ (V, ·, ·), iff for every vector x, ∞ c2i = x2 , (3.6) i=1

where ci = x, bi , i ≥ 1, which in turn holds iff n -x − ci bi - → 0 as n → ∞.

(3.7)

i=1

∞ of real numbers such that i=1 c2i < ∞, Conversely, if {ci }i≥1 is a sequence n then the sequence {xn ≡ i=1 ci bi }n≥1 is Cauchy and hence converges to an x in V . Thus the Hilbert spaceH can be identified with  the space 2 ∞ of all square summable sequences {ci }i≥1 : i=1 c2i < ∞ , in the sense that the map ϕ : x → {ci }i≥1 , where ci = x, bi , i ≥ 1, preserves the algebraic structure as well as the innerproduct, i.e., ϕ is a linear operator from H to 2 and ϕ(x), ϕ(y) = x, y for all x, y ∈ H. Such a ϕ is called an isometric isomorphism between H to 2 . Note also that 2 is simply L2 (Ω, F, µ) where Ω ≡ N, F = P(N), and µ, the counting measure. It can be shown (using the axiom of choice) that every separable Hilbert space does possess a complete orthonormal system, i.e., an orthonormal basis. Next some examples are given. Here, unless otherwise indicated, H denotes the Hilbert space and B denotes an orthonormal basis of H. Example 3.3.1:

∞ (a) H ≡ 2 = {(x1 , x2 , . . .) : xi ∈ R, i=1 x2i < ∞}. B ≡ {ei : i ≥ 1} where ei ≡ (0, 0, . . . , 1, 0, . . .) with 1 in the ith position and 0 elsewhere.
 1 m(A), m(·) being (b) H ≡ L2 ([0, 2π], B [0, 2π] , µ) where µ(A) = 2π Lebesgue measure. B ≡ {cos nx : n = 0, 1, 2, . . . , } ∪ {sin nx : n = 1, 2, . . .}. (For a proof, see Chapter 5.)

3.3 Banach and Hilbert spaces

101

2 (c) Let  kH ≡ L (R, B(R), µ) where µ is a finite measure such that |x| dµ < ∞ for all k = 1, 2, . . .. Let B1 ≡ {1, x, x2 , . . .} and B be the orthonormal set generated by applying the Gram-Schmidt procedure to B1 (see Problem 3.31). It can be shown that B is a basis for H (Problem 3.39). When µ is the standard normal distribution, the elements of B are called Hermite polynomials.

For one more example, i.e., Haar functions, see Problem 3.40. Theorem 3.3.3: (Riesz representation). Let H be a separable Hilbert space. Then every bounded linear functional T on H → R can be represented as T ≡ Tx0 for some x0 ∈ V , where Tx0 (y) ≡ y, x0 . Proof: Let B = {bi }i≥1 be an orthonormal basis for H. Let ci ≡ T (bi ), i ≥ 1. Then, for n ≥ 1, n

c2i

=

i=1

n

=

T

n

≤

c2i

≤

c2i

< ∞.

⇒

 ci bi

i=1 - n

* * * n 2* * ci ** ⇒* i=1 n

ci T (bi )

i=1

T -

i=1

(by the linearity of T )

1/2

n 2 ci bi - = T  ci i=1

T 2

i=1

⇒

∞ i=1

n Thus {xn ≡ i=1 ci bi }n≥1 is Cauchy in H and hence converges to an H. By the continuity of T , for any y, T y = limn→∞ T yn , where x0 in  n yn ≡ i=1 y, bi bi , n ≥ 1. But T yn

=

n i=1

=

y, bi ci =

. / n y, bi ci i=1

y, xn , by the linearity of T

Again by continuity of y, x, it follows that T y = y, x0 .

2

A sufficient condition for an L2 (Ω, F, µ) to be separable is that there exists an at most countable family A ≡ {Aj }j≥1 of sets in F such that F =
σA andµ(Aj ) > 0 for each j. This holds for any σ-finite measure µ on Rk , B(Rk ) (Problem 3.38).

102

3. Lp -Spaces

Remark 3.3.2: Assuming the axiom of choice, the Riesz representation theorem remains valid for any Hilbert space, separable or not (Problem 3.43).

3.4 Problems 3.1 Prove Proposition 3.1.7. (Hint: Use (1.4) repeatedly.) 3.2 Let (Ω, F, µ) be a measure space with µ(Ω) ≤ 1 and f : Ω → (a, b) ⊂ R be in L1 (Ω, F, µ). Let φ : (a, b) → R be convex. Show that if c ≡ f dµ ∈ (a, b) and φ(f ) ∈ L1 (Ω, F, µ) and cφ+ (c) ≥ 0, then

   µ(Ω) φ f dµ ≤ φ(f )dµ. 3.3 Prove Proposition 3.1.10. (Hint: Apply Jensen’s inequality with Ω ≡ {1, 2, . . . , k}, F = P(Ω), P ({i}) = pi , f (i) = ai , i = 1, 2, . . . , k, and φ(x) = ex to get (i). Deduce (ii) from (i) and Remark 3.1.3. For (iii), consider φ(x) = |x|p .) 3.4 Give an example of a convex function φ on (0, 1) with a finite number of points where it is not differentiable. Can this be extended to the countable case? Uncountable case? (Hint: Note that φ+ (·) and φ− (·) are both monotone and hence have at most a countable number of discontinuity points.) 3.5 Let φ : (a, b) → R be convex. (a) Using the definition and induction, show that

 n n φ pi xi ≤ pi φ(xi ) i=1

i=1

for any n ≥ 2, x1 , x2 . . . , xn in (a, b) and {p1 , p2 , . . . , pn }, a probability distribution. (b) Use (a) to prove Jensen’s inequality for any bounded φ. 3.6 Show that a function φ : R → R is convex iff

   φ f dm ≤ φ(f )dm [0,1]

[0,1]

for every bounded Borel measurable function f : [0, 1] → R, where m(·) is the Lebesgue measure.

3.4 Problems

103

3.7 Let φ be convex on (a, b) and ψ : R → R be convex and nondecreasing. Show that ψ ◦ φ is convex on (a, b). 3.8 Let X be a nonnegative random variable on some probability space. 1 (a) Show that (EX)(E X ) ≥ 1. What does this say about the cor1 relation between X and X ?

(b) Let f, g : R+ → R+ be Borel measurable and such that f (x)g(x) ≥ 1 for all x in R+ . Show that Ef (X)Eg(X) ≥ 1. 3.9 Prove Corollary 3.1.13 using H¨ older’s inequality applied to an appropriate measure space. 3.10 Extend H¨ older’s inequality as follows. Let k1 < pi < ∞, and fi ∈ Lpi (Ω, F, µ), i = 1, 2, . . . , k. Suppose i=1 p1i = 1. Show that  
k k i=1 fi dµ ≤ i=1 fi pi . (Hint: Use Proposition 3.1.10 (ii).) 3.11 Verify Minkowski’s inequality for p = 1 and p = ∞. 3.12 (a) Find (Ω, F, µ), 0 < p < 1, f , g ∈ Lp (Ω, F, µ) such that  1/p   1/p   1/p |f + g|p dµ > |f |p dµ + |g|p dµ . (b) Prove (1.18) for 0 < p < 1 with f p =



|f |p dµ.

3.13 Let (Ω, F, µ) be a measure space. Let   {Ak }k≥1 ⊂ F and ∞ lim Ak = 0, where lim Ak = k=1 µ(Ak ) < ∞. Show that µ k→∞ k→∞ ∞  j≥n Aj = {ω : ω ∈ Aj for infinitely many j ≥ 1}. n=1 3.14 Establish Theorem 3.2.2 for p = ∞. (Hint: For each k ≥ 1, choose nk ↑ such that fnk+1 − fnk ∞ < 2−k . Show that there exists a set A with µ(Ac ) = 0 and for ω in A, |fnk+1 (ω) − fnk (ω)| < 2−k for all k ≥ 1 and now proceed as in the proof for the case 0 < p < ∞.)  3.15 Let f , g ∈ Lp (Ω, F, µ), 0 < p < 1. Show that d(f, g) = |f − g|p dµ is a metric and (Lp (Ω, F, µ), d) is complete. 3.16 Let (Ω, F, µ) be a measure  space and f : Ω → R be F-measurable. Let Af = {p : 0 < p < ∞, |f |p dµ < ∞}. (a) Show that p1 , p2 ∈ Af , p1 < p2 implies [p1 , p2 ] ⊂ Af .    (Hint: Use |f |≥1 |f |p dµ ≤ |f |≥1 |f |p2 dµ and |f |≤1 |f |p dµ ≤  |f |p1 dµ for any p1 < p < p2 .) |f |≤1

104

3. Lp -Spaces

 (b) Let ψ(p) = log |f |p dµ for p ∈ Af . By (a), it is known that Af is connected, i.e., it is an interval. Show that ψ is convex in the open interior of Af . (Hint: Use H¨older’s inequality.) (c) Give examples to show that Af could be a closed interval, an open interval, and semi-open intervals. (d) If 0 < p1 < p < p2 , show that f p ≤ max{f p1 , f p2 } (Hint: Use (b).)  (e) Show that if |f |r dµ < ∞ for some 0 < r < ∞, then f p → f ∞ as p → ∞. (Hint: Show first that for any K > 0, µ(|f | > K) > 0 ⇒ lim f p ≥ K. If f ∞ < ∞ and µ(Ω) < ∞, use the fact that

p→∞

1/p f p ≤ f ∞ (µ(Ω)) and reduce the general case under the  hypothesis that |f |p dµ < ∞ for some p to this case.)

3.17 Let X be a random variable on a probability space (Ω, F, µ). Recall  that Eh(X) = h(X)dµ if h(X) ∈ L1 (Ω, F, µ). p1

(a) Show that (E|X|p1 ) ≤ (E|X|p2 ) p2 for any 0 < p1 < p2 < ∞. (b) Show that ‘=’ holds in (a) iff |X| is a constant a.e. (µ). (c) Show that if E|X| < ∞, then E| log |X|| < ∞ and E|X|r < ∞ for all 0 < r < 1, and 1r log(E|X|r ) → E log |X| as r → 0. 3.18 Let X be a nonnegative random variable. (a) Show that EX log X ≥ (EX)(E log X). , √ (b) Show that 1 + (EX)2 ≤ E( 1 + X 2 ) ≤ 1 + EX. 3.19 Let Ω = N, F = P(N), and let µ be the counting measure. Denote Lp (Ω, F, µ) for this case by p . (a)  Show that p is the set of all sequences {xn }n≥1 such that ∞ p n=1 |xn | < ∞. (b) For the following sequences, find all p > 0 such that they belong to p : (i) xn ≡ (ii) xn =

1 n,

n ≥ 1.

1 n(log(n+1))2 ,

n ≥ 1.

3.4 Problems

105

3.20 For 1 ≤ p < ∞, prove the Riesz representation theorem for p . That is, show that if T is a bounded linear functional from p → R, then there exists ∞ a y = {yi }i≥1 ∈ q such that for any x = {xi }i≥1 in p , T (x) = i=1 xi yi . (Hint: Let yi = T (ei ) where ei = {ei (j)}j≥1 , ei (j) = 1 if i = j, 0 if i = j. Use the fact |T (x)| ≤ T  xp to show that for each n ∈ N, n ( i=1 |yi |q ) ≤ T q < ∞.) 3.21 Let Ω = R, F = B(R), µ = µF where F is a cdf on R. If f (x) ≡ x2 , find Af = {p : 0 < p < ∞, f ∈ Lp (R, B(R), µF )} for the following cases: x 2 (a) F (x) = Φ(x), the N (0, 1) cdf, i.e., Φ(x) ≡ √12π −∞ e−u /2 du, x ∈ R. x 1 (b) F (x) = π1 −∞ 1+u 2 du, x ∈ R. 3.22 Show that C[0, 1] with the supnorm (i.e., with f  = sup{|f (x)| : 0 ≤ x ≤ 1}) is a Banach space. (Hint: To verify completeness, let {fn }n≥1 be a Cauchy sequence in C[0, 1]. Show that for each 0 ≤ x ≤ 1, {fn (x)}n≥1 is a Cauchy sequence in R. Let f (x) = lim fn (x). Now show that sup{|fn (x) − n→∞

f (x)| : 0 ≤ s ≤ 1} ≤ lim fn − fm . Conclude that fn converges to m→∞

f uniformly on [0, 1] and that f ∈ C[0, 1].) 3.23 Show that the space (P,  · ) of all polynomials on [0, 1] with the supnorm is a normed linear space that is not complete. 1 (Hint: Let g(t) = 1−t/2 for 0 ≤ t ≤ 1. Find a sequence of polynomials {fn }n≥1 in P that converge to g in supnorm.)

3.24 Show that the function f (v) ≡ v from a normed linear space (V, ·) to R+ is continuous. 3.25 Let (V,  · ) be a normed linear space. Let S = {v : v < 1}. Show that S is an open set in V . 3.26 Show that the space Pk of all polynomials on [0, 1] of degree ≤ k is a Banach space under the supnorm, i.e., under f  = sup{|f (x)|, 0 ≤ s ≤ 1}. k (Hint: Let pn (x) = j=0 anj xj be a sequence of elements in Pk that converge in supnorm to some f (·). Show that {an1 }n≥1 converges and recursively, {ani }n≥1 converges for each i.)  3.27 Let µ be a finite measure on [0,1]. Verify that Tµ (f ) ≡ f dµ is a bounded linear functional on C[0, 1] and that Tµ  = µ([0, 1]).

106

3. Lp -Spaces

3.28 Let (Vi ,  · i ), i = 1, 2, be two normed linear spaces over R. (a) Let T : V1 → V2 be a linear operator. Show that if for some x0 , T x − T x0  → 0 as x → x0 , then T is continuous on V1 and hence bounded. (b) Show that if (V2 ,  · 2 ) is complete, then B(V1 , V2 ) ≡ {T | T : V1 → V2 , T linear and bounded} is complete under the operator norm defined in (3.1). In the following, H will denote a real Hilbert space. 3.29 (a) Use the Cauchy-Schwarz inequality to show that the function f (x, y) = x, y is continuous from H × H → R. (b) (Parallelogram law). Show that in an innerproduct space (V, ·, ·), for any x, y ∈ V x + y2 + x − y2 = 2(x2 + y2 ) where x2 = x, x. 3.30 (a) Let {Qn (x)}n≥0 be defined on [0, 2π] by Qn (x) = cn

 1 + cos x n 2

where cn is such that 

1 2π

2π

0

Qn (x)dx = 1.

Clearly, Qn (·) ≥ 0. (i) Verify that for each δ > 0, sup{Qn (x) : δ ≤ x ≤ 2π − δ} → 0

as

n → ∞.

(ii) Use this to show that if f ∈ C[0, 2π] and if 1 Pn (t) ≡ 2π

 0

2π

Qn (t − s)f (s)ds, n ≥ 0,

(4.1)

then Pn → f uniformly on [0, 2π]. (b) Use this to give a proof of the completeness of the class C of trigonometric functions. (c) Show that if f ∈ L1 ([0, 2π]), then Pn (·) converges to f in L1 ([0, 2π]).

3.4 Problems

107

(d) Let {µn (·)}n≥1 be a sequence of probability measures on (R, B(R)) such that for each δ > 0, µn ({x : |x| ≥ δ}) → 0 as  n → ∞. Let f : R → R be Borel measurable. Let fn (x) ≡ f (x− y)µn (dy), n ≥ 1. Assuming that fn (·) is well defined and Borel measurable, show that (i) f (·) continuous at x0 and bounded ⇒ fn (x0 ) → f (x0 ). (ii) f (·) uniformly continuous and bounded on R ⇒ fn → f uniformly on R. (iii) f ∈ Lp (R, B(R), m), 0 < p < ∞, m(·) = Lebesgue measure ⇒ |fn − f |p dm → 0. (iv) Show that (iii) ⇒ (c). 3.31 (Gram-Schmidt procedure). Let B ≡ {bn : n ∈ N} be a collection of nonzero vector in H. Set b1 b1  = b2 − b2 , e1 e1 , e˜2 (provided ˜ e2  = 0), and so on. = ˜ e2  =

e1 e˜2 e2

If ˜ en  = 0 for some n ∈ N, then delete bn . Let E ≡ {ej : 1 ≤ j < k}, k ≤ ∞, be the collection of vectors reached this way. (a) Show that E is an orthonormal system. (b) Let HB denote the closed linear subspace generated by B, i.e.,  HB

≡

x : x ∈ H, there exists xn of the form  aj ∈ R, such that xn − x → 0 .

n

aj bj ,

j=1

Show that HB is a Hilbert space and E is a basis for HB . 3.32 Let H = L2 (R, B(R), µ), where µ is a probability measure. Let B ≡ {1, x, x2 , . . .}. Assume that |x|k dµ < ∞ for all k ∈ N. Apply the Gram-Schmidt procedure in Problem 3.31 to the set B for the following cases and evaluate e1 , e2 , e3 . (a) µ = Uniform [0, 1] distribution. (b) µ = standard normal distribution. (c) µ = Exponential (1) distribution. The orthonormal basis E obtained this way is called Orthogonal Polynomials w.r.t. the given measure. (See Szego (1939).)

108

3. Lp -Spaces

3.33 Let B ⊂ H be an orthonormal system. Show that for any x in H, {b : x, b = 0} is at most countable. a collection of nonnegative (Hint: Show first that if {yα : α ∈ I} is real numbers such that for some C < ∞, α∈F yα ≤ C for all F ⊂ I, F finite, then {α : yα > 0} is at most countable and apply this to the Bessel inequality.) 3.34 Let B ⊂ H. Define B ⊥ ≡ {x : x ∈ H, x, b = 0, for all b ∈ B}. Show that B ⊥ is a closed subspace of H. 3.35 Let B ⊂ H be a closed subspace of H. (a) Using the fact that every Hilbert space admits an orthonormal basis, show that every x in H can be uniquely decomposed as x=y+z

(4.2)

where y ∈ B and z ∈ B ⊥ and x2 = y2 + z2 . (b) Let PB : H → B be defined by PB x = y where x admits the decomposition in (4.2) above. Verify that PB is a bounded linear operator from H to B and is of norm 1 if B has at least one nonzero vector. (The operator PB is called the projection onto B.) (c) Verify that PB (PB x) = PB x for all x in H. 3.36 Let H be separable and {xn }n≥1 ⊂ H be such that {xn n≥1 } is bounded by some C < ∞. Show that there exist a subsequence {xnj }j≥1 ⊂ {xn }n≥1 and an x0 in H, such that for every y in H, xnj , y → x0 , y. (Hint: Fix an orthonormalbasis B ≡ {bn }n≥1 ⊂ H. Let ani = ∞ xn , bi , n ≥ 1, i ≥ 1. Using i=1 a2ni ≤ C for all n and the BolzanoWierstrass property, show that (a) there exists {nj }j≥1 such that lim anj i = ai exists for all i ≥ 1, j→∞ ∞ 2 a < ∞, i=1 i n (b) lim i=1 ai bi ≡ x0 exists in H, n→∞

(c) xnj , y → x0 , y for all y in H.) 3.37 Let (V, ·, ·) be an innerproduct space. Verify that ·, · is bilinear, i.e., for α1 , α2 , β1 , β2 ∈ R, x1 , x2 , y1 , y2 in V , α1 x1 + α2 x2 , β1 y1 + β2 y2  =

α1 β1 x1 , y1  + α1 β2 x1 , y2  + α2 β1 x2 , y1  + α2 β2 x2 , y2 .

State and prove an extension to more than two vectors.

3.4 Problems

109

3.38 Let (Ω, F, µ) be a measure space. Suppose that there exists an at most countable family A ≡ {Aj }j≥1 ⊂ F such that F = σA and µ(Aj ) > 0 for each j ≥ 1. Then show that for 0 < p < ∞, Lp (Ω, F, µ) is separable. (Hint: Show first that for any A ∈ F with µ(A) < ∞, and  > 0, there exists a countable subcollection A1 of A such that µ(AB) <  where n B = {∪Aj : Aj ∈ A1 }. Now consider the class of functions { i=1 ci IAi , n ≥ 1, Ai ∈ A, ci ∈ Q}.) 3.39 Show that B in Example 3.3.1 (c) is a basis for H. (Hint: Using Theorem 2.3.14 prove that the set of all polynomials are dense in H.) 3.40 (Haar functions). For x in R let h(x) = I[0,1/2) (x) − I[1/2,1) (x). Let h00 (x) ≡ I[0,1) (x) and for k ≥ 1, 0 ≤ j < 2k−1 , let hkj (x) ≡ k−1 2 2 h(2k−1 x − j), 0 ≤ x < 1. (a) Verify that the family {hkj (·), k ≥ 0, 0 ≤ j < 2k−1 } is an orthonormal set in L2 ([0, 1], B([0, 1]), m), where m(·) is Lebesgue measure. (b) Verify that this family is complete by completing the following two proofs: (i) Show "that for  indicator function f of dyadic interval of the form 2kn , 2 n , k < , the following identity holds:   2  − k  f 2 dm = n = f hkj dm . 2 k,j

Now use the fact the linear combinations of such f ’s is dense in L2 [0, 1]. (ii) For each f ∈ L2 ([0, 1], B([0, 1]), m) such  that f is orthogonal to the Haar functions, F (t) ≡ [0,t] f dm, 0 ≤ t ≤ 1 is continuous and satisfies F ( 2jn ) = 0 for all 0 ≤ j ≤ 2n , n = 1, 2, . . . and hence F ≡ 0 implying f = 0 a.e. 3.41 Let H be a Hilbert space over R and M be a closed subspace of H. Let v0 ∈ H. Show that min{v − v0  : v ∈ M } = max{v0 , u, u ∈ M ⊥ , u = 1}, where M ⊥ is the orthogonal complement of M , i.e., M ⊥ ≡ {u : v, u = 0 for all v ∈ M }. (Hint: Use Problem 3.35 (a).)

110

3. Lp -Spaces

3.42 Let B be an orthonormal set in a Hilbert space H. (a) (i) Show that for any x in H and any finite set {bi : 1 ≤ i ≤ k} ⊂ B, k < ∞, k

x, bi 2 ≤ x2 .

i=1

(ii) Conclude that for all x in H, Bx ≡ {b : x, b = 0, b ∈ B} is at most countable. (b) Show that the following are equivalent: (i) B is complete, i.e., x ∈ H, x, b = 0 for all b ∈ B ⇒ x = 0. (ii) For all x ∈ B, there exists anat most countable set Bx ≡ ∞ {bi : i ≥ 1} such that x2 = i=1 x, bi 2 . (iii) For all x ∈ B,  > 0, there exists a finite set {b1 , b2 , . . . , bk } ⊂ B such that k x, bi bi - < . -x − i=1

(iv) If B ⊂ B 1 , B 1 an orthonormal set in H ⇒ B = B 1 . 3.43 Extend Theorem 3.3.3 to any Hilbert space assuming that the axiom of choice holds. (Hint: Using the axiom of choice or its equivalent, the Hausdorff maximality principle, it can be shown that every Hilbert space H admits an orthonormal basis B (see Rudin (1987)). Now let T be a bounded linear functional from H to R. Let f (b) ≡ T (b) for b in B. k Verify that i=1 |f (bi )|2 ≤ T 2 for all finite collection {bi : 1 ≤ i ≤ k} ⊂ B. Conclude that D ≡ {b : f (b) = 0} is countable. Let x0 ≡ b∈D f (b)b. Now use the proof of Theorem 3.3.3 to show that T (x) ≡ x, x0  for all x in H.) 3.44 Let (V, ·) be a normed linear space. Let {Tn }n≥1 and T be bounded linear operators from V to V . The sequence {Tn }n≥1 is said to converge (a) weakly to T if for each w in V ∗ , the dual of V , and each v in V , w(Tn (v)) → w(T (v)), (b) strongly if for each v in V , Tn v − T v → 0, (c) uniformly if sup{Tn v − T v : v ≤ 1} → 0.

3.4 Problems

111

Let Vp = Lp (R, B(R), µL ), 1 ≤ p ≤ ∞. Let {hn }n≥1 ⊂ R be such that hn = 0, hn → 0, as n → ∞. Let (Tn f )(·) ≡ f (· + hn ), T f (·) ≡ f (·). Verify that (i) {Tn }n≥1 and T are bounded linear operators on Vp , 1 ≤ p ≤ ∞, (ii) for 1 ≤ p < ∞, {Tn }n≥1 converges to T weakly, (iii) for 1 ≤ p < ∞, {Tn } converges to T strongly, (iv) for 1 ≤ p < ∞, {Tn } does not converge to T uniformly by showing that for all n, Tn − T  = 1, (v) for p = ∞, show that Tn does not converge weakly to T .

4 Differentiation

4.1 The Lebesgue-Radon-Nikodym theorem Definition 4.1.1: Let (Ω, F) be a measurable space and let µ and ν be two measures on (Ω, F). The measure µ is said to be dominated by ν or absolutely continuous w.r.t. ν and written as µ  ν if ν(A) = 0 ⇒ µ(A) = 0

for all A ∈ F.

(1.1)

Example 4.1.1: Let m be the Legesgue measure on (R, B(R)) and let µ be the standard normal distribution, i.e.,  2 1 √ e−x /2 m(dx), A ∈ B(R). µ(A) ≡ 2π A Then m(A) = 0 ⇒ µ(A) = 0 and hence µ  m. Example 4.1.2: Let Z+ ≡ {0, 1, 2, . . .} denote the set of all nonnegative integers. Let ν be the counting measure on Ω = Z+ and µ be the Poisson (λ) distribution for 0 < λ < ∞, i.e., ν(A) = number of elements in A and µ(A) =

e−λ λj j!

j∈A

for all A ∈ P(Ω), the power set of Ω. Since ν(A) = 0 ⇔ A = ∅ ⇔ µ(A) = 0, it follows that µ  ν and ν  µ.

114

4. Differentiation

Example 4.1.3: Let f be a nonnegative measurable function on a measure space (Ω, F, ν). Let  f dν for all A ∈ F. (1.2) µ(A) ≡ A

Then, µ is a measure on (Ω, F) and ν(A) = 0 ⇒ µ(A) = 0 for all A ∈ F and hence µ  ν. The Radon-Nikodym theorem is a sort of converse to Example 4.1.3. It says that if µ and ν are σ-finite measures (see Section 1.2) on a measurable space (Ω, F) and if µ  ν, then there is a nonnegative measurable function f on (Ω, F) such that (1.2) holds. Definition 4.1.2: Let (Ω, F) be a measurable space and let µ and ν be two measures on (Ω, F). Then, µ is called singular w.r.t. ν and written as µ ⊥ ν if there exists a set B ∈ F such that µ(B) = 0

and ν(B c ) = 0.

(1.3)

Note that µ is singular w.r.t. ν implies that ν is singular w.r.t. µ. Thus, the notion of singularity between two measures µ and ν is symmetric but that of absolutely continuity is not. Note also that if µ and ν are mutually singular and B satisfies (1.3), then for all A ∈ F, µ(A) = µ(A ∩ B c )

and ν(A) = ν(A ∩ B).

(1.4)

Example 4.1.4: Let µ be the Lebesgue measure on (R, B(R)) and ν be defined as ν(A) = # elements in A ∩ Z where Z is the set of integers. Then ν(Zc ) = 0

and µ(Z) = 0

and hence (1.3) holds with B = Z. Thus µ and ν are mutually singular. Another example is the pair m and µc on [0,1] where µc is the LebesgueStieltjes measure generated by the Cantor function (cf. Section 4.5) and m is the Lebesgue measure. Example 4.1.5: Let µ be the Lebesgue measure restricted to (−∞, 0] and ν be the Exponential(1) distribution. That is, for any A ∈ B(R), µ(A) ν(A)

=

the Lebesgue measure of  = e−x dx.

A ∩ (−∞, 0];

A∩(0,∞)

Then, µ((0, ∞)) = 0 and ν((−∞, 0]) = 0, and (1.3) holds with B = (−∞, 0].

4.1 The Lebesgue-Radon-Nikodym theorem

115

Suppose that µ and ν are two finite measures on a measurable space (Ω, F). H. Lebesgue showed that µ1 can be decomposed as a sum of two measures, i.e., µ = µa + µs where µa  ν and µs ⊥ ν. The next theorem is the main result of this section and it combines the above decomposition result of Lebesgue with the Radon-Nikodym theorem mentioned earlier. Theorem 4.1.1: Let (Ω, F) be a measurable space and let µ1 and µ2 be two σ-finite measures on (Ω, F). (i) (The Lebesgue decomposition theorem). uniquely decomposed as

The measure µ1 can be

µ1 = µ1a + µ1s

(1.5)

where µ1a and µ1s are σ-finite measures on (Ω, F) such that µ1a  µ2 and µ1s ⊥ µ2 . (ii) (The Radon-Nikodym theorem). There exists a nonnegative measurable function h on (Ω, F) such that  µ1a (A) = hdµ2 for all A ∈ F. (1.6) A

Proof: Case 1: Suppose that µ1 and µ2 are finite measures. Let µ be the measure µ = µ1 + µ2 and let H = L2 (µ). Define a linear function T on H by  T (f ) =

f dµ1 .

(1.7)

Then, by the Cauchy-Schwarz inequality applied to the functions f and g ≡ 1, |T (f )|



 12   12 µ1 (Ω) f dµ1



 12   12 f dµ µ1 (Ω) .

≤ ≤

2

2

This shows that T is a bounded linear functional on H with T  ≤ M ≡ 1 (µ1 (Ω)) 2 . By the Riesz representation theorem (cf. Theorem 3.3.3 and Remark 3.3.2), there exists a g ∈ L2 (µ) such that  T (f ) = f gdµ (1.8)

116

4. Differentiation

for all f ∈ L2 (µ). Let f = IA for A in F. Then, (1.7) and (1.8) yield  gdµ. µ1 (A) = T (IA ) = A

But 0 ≤ µ1 (A) ≤ µ(A) for all A ∈ F. Hence the function g in L2 (µ) satisfies  gdµ ≤ µ(A) for all A ∈ F. (1.9) 0≤ A

Let A1 = {0 ≤ g < 1}, A2 = {g = 1}, A3 = {g ∈ [0, 1]}. Then (1.9) implies that µ(A3 ) = 0 (see Problem 4.1). Now define the measures µ1a (·) and µ1s (·) by µ1a (A) ≡ µ1 (A ∩ A1 ),

µ1s (A) ≡ µ1 (A ∩ A2 ), A ∈ F.

(1.10)

Next it will be shown that µ1a  µ2 and µ1s ⊥ µ2 , thus establishing (1.5). By (1.7) and (1.8), for all f ∈ H,     f dµ1 = f gdµ = f gdµ1 + f gdµ2  ⇒

 f (1 − g)dµ1 =

f gdµ2 .

(1.11)

Setting f = IA2 yields 0 = µ2 (A2 ). From (1.10), since µ1s (Ac2 ) = 0, it follows that µ1s ⊥ µ2 . Now fix n ≥ 1 and A ∈ F. Let f = IA∩A1 (1 + g + . . . + g n−1 ). Then (1.11) implies that   n (1 − g )dµ1 = g(1 + g + . . . + g n−1 )dµ2 . A∩A1

A∩A1

Now letting n → ∞, and using the MCT on both sides, yields  g µ1a (A) = dµ2 . IA1 (1 − g) A Setting h ≡

g 1−g IA1

(1.12)

completes the proof of (1.5) and (1.6).

Case 2: Now suppose that µ1 and µ2 are σ-finite. Then there exists a countable partition {Dn }≥1 ⊂ F of Ω such that µ1 (Dn ) and µ2 (Dn ) are (n) (n) both finite for all n ≥ 1. Let µ1 (·) ≡ µ1 (· ∩ Dn ) and µ2 ≡ µ2 (· ∩ Dn ). (n) (n) Then applying ‘Case 1’ to µ1 and µ2 for each n ≥ 1, one gets measures (n) (n) µ1a , µ1s and a function hn such that (n)

(n)

(n)

µ1 (·) ≡ µ1a (·) + µ1s (·)

(1.13)

4.1 The Lebesgue-Radon-Nikodym theorem

117

  (n) (n) (n) (n) where, for A in F, µ1a (A) = A hn dµ2 = A hn IDn dµ2 and µ1s (·) ⊥ µ2 . ∞ (n) Since µ1 (·) = n=1 µ1 (·), it follows from (1.13) that where µ1a (A) ≡

(1.14) µ1 (·) = µ1a (·) + µ1s (·),  (n) (n) ∞ n=1 µ1a (A) and µ1s (·) = n=1 µ1s (·). By the MCT,  hdµ2 , A ∈ F, µ1a (A) =

∞

A

∞

where h ≡ n=1 hn IDn . Clearly, µ1a  µ2 . The verification of the singularity of µ1s and µ2 is left as an exercise (Problem 4.2). It remains to prove the uniqueness of the decomposition. Let µ1 = µa + µs and µ1 = µa + µs be two decompositions of µ1 where µa and µa are absolutely continuous w.r.t. µ2 and µs and µs are singular w.r.t. µ2 . By definition, there exist sets B and B  in F such that µ2 (B) = 0, µ2 (B  ) = 0,

and µs (B c ) = 0, µs (B c ) = 0.

Let D = B ∪ B  . Then µ2 (D) = 0 and µs (Dc ) ≤ µs (B c ) = 0. Similarly, µs (Dc ) ≤ µs (B c ) = 0. Also µ2 (D) = 0 implies µa (D) = 0 = µa (D). Thus for any A ∈ F, µa (A) = µa (A ∩ Dc )

and µa (A) = µa (A ∩ Dc ).

Also µs (A ∩ Dc ) ≤ µs (A ∩ B c ) = 0 µs (A ∩ Dc ) ≤ µs (A ∩ B c ) = 0. Thus, µ(A ∩ Dc ) = µa (A ∩ Dc ) + µs (A ∩ Dc ) = µa (A ∩ Dc ) = µa (A) and µ(A ∩ Dc ) = µa (A ∩ Dc ) + µs (A ∩ Dc ) = µa (A ∩ Dc ) = µa (A). Hence, µa (A) = µ(A ∩ Dc ) = µa (A) for every A ∈ F. That is, µa = µa and hence, 2 µs = µs . Remark 4.1.1: In Theorem 4.1.1, the hypothesis of σ finiteness cannot be dropped. For example, let µ be the Lebesgue measure and ν be the counting measure on [0, 1]. Then µ  ν but there does not exist a nonnegative F measurable function h such that µ(A) =  A hdν. To see this, if possible, suppose that for some h ∈ L1 (ν), µ(A) = A hdν for all A ∈ F. Note that µ([0, 1]) = 1 implies that [0,1] hdν < ∞ and hence, that B ≡ {x : h(x) > 0} is countable (Problem 4.3). But µ being the Lebesgue measure, µ(B) = 0 c c c and µ(B ) = 1. Since by definition, h ≡ 0 on B , this implies 1 = µ(B ) = hdν = 0, leading to a contradiction. However, if ν is σ-finite and µ  ν Bc (µ not necessarily σ-finite), then the Radon-Nikodym theorem holds, i.e., there exists a nonnegative F-measurable function h such that  µ(A) = hdν for all A ∈ F. A

118

4. Differentiation

For a proof, see Royden (1988), Chapter 11. Definition 4.1.3: Let µ and ν be measures on a measurable space (Ω, F) and let h be a nonnegative measurable function such that  hdν for all A ∈ F. µ(A) = A

Then h is called the Radon-Nikodym derivative of µ w.r.t. ν and is written as dµ = h. dν If µ(Ω) < ∞ and there exist two nonnegative F-measurable functions h1 and h2 such that   µ(A) =

h1 dν = A

h2 dν A

for all A ∈ F, then h1 = h2 a.e. (ν) and thus the Radon-Nikodym derivative dµ dν is unique up to equivalence a.e. (ν). This also extends to the case when µ is σ-finite. The following proposition is easy to verify (cf. Problem 4.4). Proposition 4.1.2: Let ν, µ, µ1 , µ2 , . . . be σ-finite measures on a measurable space (Ω, F). (i) If µ1  µ2 and µ2  µ3 , then µ1  µ3 and dµ1 dµ2 dµ1 = dµ3 dµ2 dµ3

a.e. (µ3 ).

(ii) Suppose that µ1 and µ2 are dominated by µ3 . Then for any α, β ≥ 0, αµ1 + βµ2 is dominated by µ3 and dµ1 dµ2 d(αµ1 + βµ2 ) =α +β dµ3 dµ3 dµ3 (iii) If µ  ν and

dµ dν

a.e. (µ3 ).

> 0 a.e. (ν), then ν  µ and dν = dµ

dµ dν

−1 a.e. (µ).

(iv) Let {µn }n≥1 be a sequence of measures and {αn }n≥1 be a sequence of positive real numbers, i.e., αn > 0 for all n ≥ 1. Define µ = ∞ n=1 αn µn .

4.2 Signed measures

119

(a) Then, µ  ν iff µn  ν for each n ≥ 1 and in this case, ∞ dµ dµn = αn dν dν n=1

a.e. (ν).

(b) µ ⊥ ν iff µn ⊥ ν for all n ≥ 1.

4.2 Signed measures Let µ1 and µ2 be two finite measures on a measurable space (Ω, F). Let ν(A) ≡ µ1 (A) − µ2 (A),

A ∈ F.

for all

(2.1)

Then ν : F → R satisfies the following: (i) ν(∅) = 0. (ii)  For any {An }n≥1 ⊂ F with Ai ∩ Aj = ∅ for i = j, and with ∞ i=1 |ν(Ai )| < ∞, ∞ ν(A) = ν(Ai ). (2.2) i=1

(iii) Let ν

≡ sup

 ∞

|ν(Ai )| : {An }n≥1 ⊂ F, Ai ∩ Aj = ∅ for

i=1

i = j,



 An = Ω .

(2.3)

n≥1

Then, ν is finite. Note that (iii) holds because ν ≤ µ1 (Ω) + µ2 (Ω) < ∞. Definition 4.2.1: A set function ν : F → R satisfying (i), (ii), and (iii) above is called a finite signed measure. The above example shows that the difference of two finite measures is a finite signed measure. It will be shown below that every finite signed measure can be expressed as the difference of two finite measures. Proposition 4.2.1: Let ν be a finite signed measure on (Ω, F). Let  ∞ |ν(An )| : {An }n≥1 ⊂ F, Ai ∩ Aj = ∅ for i = j, |ν|(A) ≡ sup n=1

n≥1

 An = A .

(2.4)

120

4. Differentiation

Then |ν|(·) is a finite measure on (Ω, F). Proof: That |ν|(Ω) < ∞ follows from part (iii) of the definition. Thus it is enough to verify that |ν| is countably  additive. Let {An }n≥1 be a countable family of disjoint sets in F. Let A = n≥1 An . By the definition of |ν|, for all  > 0 and n ∈ N, there exists a countable  ∞family {Anj }j≥1 of disjoint sets in F with An = j≥1 Anj such that j=1 |ν(Anj )| > |ν|(An ) − 2n . Hence, ∞ ∞ ∞ |ν(Anj )| > |ν|(An ) − . n=1 j=1

n=1

Note that {Anj }n≥1,j≥1 is a countable family of disjoint sets in F such that  A = n≥1 An = n≥1 j≥1 Anj . It follows from the definition of |ν| that |ν|(A) ≥

∞ ∞

|ν(Anj )| >

n=1 j=1

∞

|ν|(An ) − .

n=1

Since this is true for for all  > 0, it follows that |ν|(A) ≥

∞

|ν|(An ).

(2.5)

n=1

To get the opposite inequality, let {Bj }j≥1  be a countable family of disjoint  B = A = sets in F such that j j≥1 n≥1 An . Since Bj = Bj ∩ A =  (B ∩ A ) and ν satisfies (2.2), j n n≥1 ν(Bj ) =

∞

ν(Bj ∩ An )

for all j ≥ 1.

n=1

Thus ∞

|ν(Bj )|

≤

j=1

=

∞ ∞ j=1 n=1 ∞ ∞

|ν(Bj ∩ An )| |ν(Bj ∩ An )|.

(2.6)

n=1 j=1

Note that for each An , {Bj ∩An }j≥1 is a countable family of disjoint sets in F such that An = j≥1 (Bj ∩ An ). Hence from (2.4), it fol∞ ∞ lows that |ν|(An ) ≥ |ν(Bj ∩ An )| and hence, |ν|(An ) ≥ j=1 ∞ ∞ n=1 ∞ j=1 |ν(Bj ∩ An )|. From (2.6), it follows that n=1 |ν|(An ) ≥ n=1 ∞ |ν(B )|. This being true for every such family {B } j j j≥1 , it follows j=1 from (2.4) that ∞ |ν|(A) ≤ |ν|(Ai ) (2.7) i=1

4.2 Signed measures

121

2

and with (2.5), this completes the proof.

Definition 4.2.2: The measure |ν| defined by (2.4) is called the total variation measure of the signed measure ν. Next, define the set functions ν+ ≡

|ν| + ν , 2

ν− ≡

|ν| − ν . 2

(2.8)

It can be verified that ν + and ν − are both finite measures on (Ω, F). Definition 4.2.3: The measures ν + and ν − are called the positive and negative variation measures of the signed measure ν, respectively. It follows from (2.8) that ν = ν+ − ν−.

(2.9)

Thus every finite signed measure ν on (Ω, F) is the difference of two finite measures, as claimed earlier. Note that both ν + and ν − are dominated by |ν| and all three measures are finite. By the Radon-Nikodym theorem (Theorem 4.1.1), there exist functions h1 and h2 in L1 (Ω, F, |ν|) such that dν + = h1 d|ν|

and

dν − = h2 . d|ν|

(2.10)

This and (2.9) imply that for any A in F, 

 h1 d|ν| −

ν(A) = A

 h2 d|ν| =

A

hd|ν|,

(2.11)

A

where h = h1 − h2 . Thus every finite signed measure ν on (Ω, F) can be expressed as  f dµ, A ∈ F (2.12) ν(A) = A

for some finite measure µ on (Ω, F) and some f ∈ L1 (Ω, F, µ). Conversely, it is easy to verify that a set function ν defined by (2.12) for some finite measure µ on (Ω, F) and some f ∈ L1 (Ω, F, µ) is a finite signed measure (cf. Problem 4.6). This leads to the following: Theorem 4.2.2: (i) A set function ν on a measurable space (Ω, F) is a finite signed measure iff there exist two finite measures µ1 and µ2 on (Ω, F) such that ν = µ1 − µ2 .

122

4. Differentiation

(ii) A set function ν on a measurable space (Ω, F) is a finite signed measure iff there exist a finite measure µ on (Ω, F) and an f ∈ L1 (Ω, F, µ) such that for all A in F,  f dµ. ν(A) = A

Definition 4.2.4: Let ν be a finite signed measure on a measurable space on (Ω, F). A set A ∈ F is called a positive set for ν if for any B ⊂ A, B ∈ F, ν(B) ≥ 0. A set A ∈ F is called a negative set for ν if for any B ⊂ A with B ∈ F, ν(B) ≤ 0. Let h be as in (2.11). Let Ω+ = {ω : h(ω) ≥ 0} and

Ω− = {ω : h(ω) < 0}.

(2.13)

From (2.11), it follows that for all A in F, ν(A∩Ω+ ) ≥ 0 and ν(A∩Ω− ) ≤ 0. Thus Ω+ is a positive set and Ω− is a negative set for ν. Furthermore, Ω+ ∪ Ω− = Ω and Ω+ ∩ Ω− = ∅. Summarizing this discussion, one gets the following theorem. Theorem 4.2.3: (Hahn decomposition theorem). Let ν be a finite signed measure on a measurable space (Ω, F). Then there exist a positive set Ω+ and a negative set Ω− for ν such that Ω = Ω+ ∪ Ω− and Ω+ ∩ Ω− = ∅. Let Ω+ and Ω− be as in (2.13). It can be verified (Problem 4.8) that for any B ∈ F, if B ⊂ Ω+ , then ν(B) = |ν|(B). By (2.11), this implies that for all A in F,  A∩Ω+

hd|ν| = |ν|(A ∩ Ω+ ).

It follows that h = 1 a.e. (|ν|) on Ω+ . Similarly, h = −1 a.e. (|ν|) on Ω− . Thus, the measures ν + and ν − , defined in (2.8), reduce to  (1 + h) ν + (A) = d|ν| 2  A (1 + h) (1 + h) d|ν| + |ν| = 2 d A∩Ω+ A∩Ω− = |ν|(A ∩ Ω+ ), and similarly

ν − (A) = |ν|(A ∩ Ω− ).

Note that ν + and ν − are both finite measures that are mutually singular. This particular decomposition of ν as ν = ν+ − ν−

4.2 Signed measures

123

is known as the Jordan decomposition of ν. It will now be shown that this decomposition is minimal and that it is unique in the class of signed measures with mutually singular components. Suppose there exist two finite measures µ1 and µ2 on (Ω, F) such that ν = µ1 − µ2 . For any A ∈ F, ν + (A) = ν(A ∩ Ω+ ) = µ1 (A ∩ Ω+ ) − µ2 (A ∩ Ω+ ) ≤ µ1 (A ∩ Ω+ ) ≤ µ1 (A) and ν − (A) = −ν(A ∩ Ω− ) = µ2 (A ∩ Ω− ) − µ1 (A ∩ Ω− ) ≤ µ2 (A ∩ Ω− ) ≤ µ2 (A). Thus, ν + ≤ µ1 and ν − ≤ µ2 . Clearly, since both µ1 and ν + are finite measures on (Ω, F), µ1 − ν + is also a finite measure. Similarly, µ2 − ν − is also a finite measure. Also, since µ1 − µ2 = ν = ν + − ν − , it follows that µ1 −ν + = µ2 −ν − = λ, say. Thus, for any decomposition of ν as µ1 −µ2 with µ1 , µ2 finite measures, it holds that µ1 = ν + + λ and µ2 = ν − + λ, where λ is a measure on (Ω, F). Thus, ν = ν + − ν − is a minimal decomposition in the sense that in this case λ = 0. Now suppose µ1 and µ2 are mutually singular, i.e., there exist Ω1 , Ω2 ∈ F such that Ω1 ∩ Ω2 = ∅, Ω1 ∪ Ω2 = Ω, and µ1 (Ω2 ) = 0 = µ2 (Ω1 ). Since µ1 ≥ λ and µ2 ≥ λ, it follows that λ(Ω2 ) = 0 = λ(Ω1 ). Thus λ = 0 and µ1 = ν + and µ2 = ν − . Summarizing the above discussion yields: Theorem 4.2.4: Let ν be a finite signed measure on a measurable space (Ω, F) and let µ1 and µ2 be two finite measures on (Ω, F) such that ν = µ1 − µ2 . Then there exists a finite measure λ such that µ1 = ν + + λ and µ2 = ν − + λ with λ = 0 iff µ1 and µ2 are mutually singular. Let S ≡ {ν : ν

is a finite signed measure on

(Ω, F)}.

Also, for any α ∈ R, let α+ = max(α, 0) and α− = max(−α, 0). Note that for ν1 , ν2 in S and α1 , α2 ∈ R, α1 ν1 + α2 ν2

= =

(α1+ − α1− )(ν1+ − ν1− ) + (α2+ − α2− )(ν2+ − ν2− ) (α1+ ν1+ + α1− ν1− + α2+ ν2+ + α2− ν2− ) − (α1+ ν1− + α1− ν1+ + α2+ ν2− + α2− ν2+ )

= λ1 − λ2 ,

say,

where λ1 and λ2 are both finite measures. It now follows from Theorem 4.2.2 that α1 ν1 + α2 ν2 ∈ S. Thus, S is a vector space over R. Now it will be shown that ν ≡ |ν|(Ω) is a norm on S and that (S,  · ) is a Banach space. Definition 4.2.5: For a finite signed measure ν on a measurable space (Ω, F), the total variation norm ν is defined by ν ≡ |ν|(Ω). Proposition 4.2.5: Let S ≡ {ν : ν a finite signed measure on (Ω, F)}. Then, ν ≡ |ν|(Ω), ν ∈ S is a norm on S.

124

4. Differentiation

Proof: Let ν1 , ν2 ∈ S, α1 , α2 ∈R and λ = α1 ν1 + α2 ν2 . For any A ∈ F and any {Ai }i≥1 ⊂ F with A = i≥1 Ai , |λ(Ai )| ≤ |α1 ||ν1 (Ai )| + |α2 ||ν2 (Ai )| for all i ≥ 1 |λ(Ai )| ≤ |α1 | |ν1 (Ai )| + |α2 | |ν2 (Ai )| ⇒ i≥1

i≥1

i≥1

≤ |α1 ||ν1 |(A) + |α2 ||ν2 |(A). Taking supremum over all {Ai }i≥1 yields, |λ|(A) ≤ |α1 ||ν1 |(A) + |α2 ||ν2 |(·) |λ|(·) ≤ |α1 | |ν1 |(·) + |α2 ||ν2 |(·),

i.e., ⇒

λ ≡ |λ|(Ω) ≤ |α1 | |ν1 |(Ω) + |α2 | |ν2 |(Ω) = |α1 |ν1  + |α2 |ν2 .

Taking α1 = α2 = 1 yields ν1 + ν2  ≤ ν1  + ν2 , i.e., the triangle inequality holds. Next taking α2 = 0 yields α1 ν1  ≤ |α1 |ν1 . To get the opposite inequality, note that for α1 = 0, ν1 = α11 α1 ν1 and so ν1  ≤ | α11 |α1 ν1 . Hence, |α1 | ν1  ≤ α1 ν1 . Thus, for any α1 = 0, α1 ν = |α1 |ν. Finally, ν = 0 ⇒ |ν|(Ω) = 0 ⇒ |ν|(A) = 0 for all A ∈ F ⇒ ν(A) = 0 for all A ∈ F, i.e., ν is the zero measure. Thus  ·  is a norm on S. 2 Proposition 4.2.6: (S,  · ) is complete. Proof: Let {νn }n≥1 be a Cauchy sequence in (S,  · ). Note that for each A ∈ F, |νn (A) − νm (A)| ≤ |νn − νm |(A) ≤ νn − νm . Hence, for each A ∈ F, {νn (A)}n≥1 is a Cauchy sequence in R and hence ν(A) ≡ lim νn (A) n→∞

exists.

It will be shown that ν(·) is a finite signed measure and νn  − ν → 0 as n → ∞. Let {Ai }i≥1 ⊂ F, Ai ∩ Aj = ∅ for i = j, and A = i≥1 Ai . Let xn ≡ {νn (Ai )}i≥1 , n ≥ 1, and let x 0 ≡ {ν(Ai )}i≥1 . Note that each xn ∈ 1 where 1 ≡ {x : x = {xi }i≥1 ∈ R, i≥1 |xi | < ∞}. For x ∈ 1 , let x1 =  ∞ |x |. Then x − x  = |ν i n m i≥1 n (Ai ) − νm (Ai )| ≤ |νn − νm |(A) ≤ i=1 |νn − νm |(Ω) → 0 as n, m → ∞. But 1 is complete under  · 1 . So there ) → ν(Ai ) for all exists x∗ ∈ 1 such that xn −x∗ 1 → 0. Since xni ≡ νn (A i ∞ ) for all i ≥ 1 and that i ≥ 1, it follows that x∗i = ν(A i i=1 |ν(Ai )| < ∞. ∞  Also, for  all n ≥ 1, νn (A)  = i=1 νn (Ai ). Since i≥1 |νn (Ai )−ν(Ai )| → 0, νn (A) ≡ i≥1 νn (Ai ) → i≥1 ν(Ai ) as n → ∞. But νn (A) → ν(A). Thus,

4.3 Functions of bounded variation

ν(A) =

∞ i=1

125

ν(Ai ). Also for any countable partition {Ai }i≥1 ⊂ F of Ω,

∞

|ν(Ai )| = lim

∞

n→∞

i=1

|νn (Ai )| ≤ lim νn  < ∞. n→∞

i=1

Thus |ν|(Ω) < ∞ and hence, ν ∈ S. Finally, νn − ν

=

sup

 ∞

|νn (Ai ) − ν(Ai )| : {Ai }i≥1 ⊂ F

i=1

is a disjoint partition of

 Ω .

But for every countable partition {Ai }i≥1 ⊂ F, ∞ i=1

|νn (Ai ) − ν(Ai )| = lim

m→∞

∞

|νn (Ai ) − νm (Ai )| ≤ lim νn − νm .

i=1

m→∞

Thus, νn − ν ≤ limm→∞ νn − νm  and hence, limn→∞ νn − ν ≤ 2 limn→∞ limm→∞ νn − νm  = 0. Hence, νn → ν in S. Definition 4.2.6: (Integration w.r.t. signed measures). Let µ be a finite signed measure on a measurable space (Ω, F) and |µ| be its total variation  measure as in Definition 4.2.2. Then, for any f ∈ L1 (Ω, F, |µ|), f dµ is defined as    + f dµ = f dµ − f dµ− , where µ+ and µ− are the positive and negative variations of µ as defined in (2.8). Proposition 4.2.7: Let µ be a signed measure on a measurable space (Ω, F, P ). Let µ = λ1 − λ2 where λ1 and λ2 are finite measures. Let f ∈ L1 (Ω, F, λ1 + λ2 ). Then f ∈ L1 (Ω, F, |µ|) and    f dµ = f dλ1 − f dλ2 . (2.14) Proof: Left as an exercise (Problem 4.13).

4.3 Functions of bounded variation From the construction of the Lebesgue-Stieltjes measures on (R, B(R)) discussed in Chapter 1, it is seen that to every nondecreasing right continuous function F : R → R, there is a (Radon) measure µF on (R, B(R)) such that

126

4. Differentiation

µF ((a, b]) = F (b) − F (a) for all a < b and conversely. If µ1 and µ2 are two Radon measures and µ = µ1 − µ2 , let ⎧ for x > 0, ⎨ µ((0, x]) −µ((x, 0]) for x < 0, G(x) ≡ ⎩ 0 for x = 0. ⎧ for x > 0, ⎨ F1 (x) − F2 (x) − (F1 (0) − F2 (0)) for x < 0, (F1 (0) − F2 (0)) − (F1 (x) − F2 (x)) = ⎩ 0 for x = 0. Thus to every finite signed measure µ on (R, B(R)), there corresponds a function G(·) that is the difference of two right continuous nondecreasing and bounded functions. The converse is also easy to establish. A characterization of such a function G(·) without any reference to measures is given below. Definition 4.3.1: Let f : [a, b] → R, where −∞ < a < b < ∞. Then for any partition Q = {a = x0 < x1 < x2 < . . . < xn = b}, n ∈ N, the positive, negative and total variations of f with respect to Q are respectively defined as P (f, Q) ≡

n

(f (xi ) − f (xi−1 ))+

i=1

N (f, Q) ≡

n

(f (xi ) − f (xi−1 ))−

i=1

T (f, Q) ≡

n

|f (xi ) − f (xi−1 )|.

i=1

It is easy to verify that (i) if f is nondecreasing, then P (f, Q) = T (f, Q) = f (b) − f (a)

and N (f, Q) = 0

and that (ii) for any f , P (f, Q) + N (f, Q) = T (f, Q). Definition 4.3.2: Let f = [a, b] → R, where −∞ < a < b < ∞. The positive, negative and total variations of f over [a, b] are respectively defined as P (f, [a, b]) ≡

sup P (f, Q)

N (f, [a, b]) ≡

sup N (f, Q)

Q Q

T (f, [a, b]) ≡

sup T (f, Q), Q

4.3 Functions of bounded variation

127

where the supremum in each case is taken over all finite partitions Q of [a, b]. Definition 4.3.3: Let f : [a, b] → R, where −∞ < a < b < ∞. Then, f is said to be of bounded variation on [a, b] if T (f, [a, b]) < ∞. The set of all such functions is denoted by BV [a, b]. As remarked earlier, if f is nondecreasing, then T (f, Q) = f (b) − f (a) for each Q and hence T (f, [a, b]) = f (b) − f (a). It follows that if f = f1 − f2 , where both f1 and f2 are nondecreasing, then f ∈ BV [a, b]. A natural question is whether the converse is true. The answer is yes, as shown by the following result. Theorem 4.3.1: Let f ∈ BV [a, b]. Let f1 (x) ≡ P (f, [a, x]) and f2 (x) ≡ N (f, [a, x]). Then f1 and f2 are nondecreasing in [a, b] and for all a ≤ x ≤ b, f (x) = f1 (x) − f2 (x) Proof: That f1 and f2 are nondecreasing follows from the definition. It is enough to verify that if f ∈ BV [a, b], then f (b) − f (a) = P (f, [a, b]) − N (f, [a, b]), as this can be applied to [a, x] for a ≤ x < b. For each finite partition Q of [a, b], f (b) − f (a)

=

n

(f (xi ) − f (xi−1 ))

i=1

= P (f, Q) − N (f, Q). Thus P (f, Q) = f (b) − f (a) + N (f, Q). By taking supremum over all finite partitions Q, it follows that P (f, [a, b]) = f (b) − f (a) + N (f, [a, b]). If f ∈ BV [a, b], this yields f (b) − f (a) = P (f, [a, b]) − N (f, [a, b]).

2

Remark 4.3.1: Since T (f, Q) = P (f, Q) + N (f, Q) = 2P (f, Q) − (f (b) − f (a)), it follows that if f ∈ BV [a, b], then T (f, [a, b])

= 2P (f, [a, b]) − (f (b) − f (a)) = P (f, [a, b]) + N (f, [a, b]).

Corollary 4.3.2: A function f ∈ BV [a, b] iff there exists a finite signed measure µ on (R, B(R)) such that f (x) = µ([a, x]), a ≤ x ≤ b. The proof of this corollary is left as an exercise.

128

4. Differentiation

Remark 4.3.2: Some observations on functions of bounded variations are listed below. (a) Let f = IQ where Q is the set of rationals. Then for any [a, b], a < b, P (f, [a, b]) = N (f, [a, b]) = ∞ and so f ∈ / BV [a, b]. This holds for f = ID for any set D such that both D and Dc are dense in R. (b) Let f be Lipschitz on [a, b]. That is, |f (x) − f (y)| ≤ K|x − y| for all x, y in [a, b] where K ∈ (0, ∞) is a constant. Then, f ∈ BV [a, b]. (c) Let f be differentiable in (a, b) and continuous on [a, b] and f  (·) be bounded in (a, b). Then by the mean value theorem, f is Lipschitz and hence, f is in BV [a, b]. (d) Let f (x) = x2 sin x1 , 0 < x ≤ 1, and let f (0) = 0. Then f is continuous on [0, 1], differentiable on (0, 1) with f  bounded on (0, 1), and hence f ∈ BV [0, 1]. (e) Let g(x) = x2 sin x12 , 0 < x ≤ 1, g(0) = 0. Then g is continuous on [0, 1], differentiable on (0, 1) but g  is not bounded on (0, 1). This by itself does not imply that g ∈ / BV [0, 1], since being Lipschitz is only a sufficient condition. But it turns out that g ∈ / BV [0, 1]. To see 0 n  1 this, let xn = (2n+1) |g(xi ) − g(xi−1 )| ≥ π , n = 0, 1, 2 . . .. Then 2

n  i=1

1 (2i+1) π 2

i=1

and hence T (g, [0, 1]) = ∞.

(f) It is known (see Royden (1988), Chapter 4) that if f : [a, b] → R then f is differentiable a.e. (m) on (a, b) and is nondecreasing, f  dm ≤ f (b) − f (c), where (m) denotes the Lebesgue measure. [a,b] This implies that  if f ∈ BV [a, b], then f is differentiable a.e. (m) on (a, b) and so, [a,b] |f  |dm ≤ T (f, [a, b]).

4.4 Absolutely continuous functions on R Definition 4.4.1: A function F : R → R is absolutely continuous (a.c.) if for all  > 0, there exists δ > 0 such that if Ij = [aj , bj ], j = 1, 2, . . . , k k k (k ∈ N) are disjoint and j=1 (bj − aj ) < δ, then j=1 |F (bj ) − F (aj )| < . By the mean value theorem, it follows that if F is differentiable and F  (·) is bounded, then F is a.c. Also note that F is a.c. implies F is uniformly continuous.

4.4 Absolutely continuous functions on R

129

Definition 4.4.2: A function F : [a, b] → R is absolutely continuous if the function F˜ , defined by ⎧ if a ≤ x ≤ b, ⎨ F (x) F (a) if x < a, F˜ (x) = ⎩ F (b) if x > b, is absolutely continuous. Thus F (x) = x is a.c. on R. Any polynomial is a.c. on any bounded interval but not necessarily on all of R. For example, F (x) = x2 is a.c. on any bounded interval but not a.c. on R, since it is not uniformly continuous on R. The main result of this section is the following result due to H. Lebesgue, known as the fundamental theorem of Lebesgue integral calculus. Theorem 4.4.1: A function F : [a, b] → R is absolutely continuous iff there is a function f : [a, b] → R such that f is Lebesgue measurable and integrable w.r.t. m and such that  f dm, for all a ≤ x ≤ b (4.1) F (x) = F (a) + [a,x]

where m is the Lebesgue measure. Proof: First consider the “if part.” Suppose that (4.1) holds. Since  |f |dm < ∞, for any  > 0, there exists a δ > 0 such that (cf. Proposi[a,b] tion 2.5.8).  |f |dm < .

m(A) < δ ⇒

(4.2)

A

Thus, if Ij = (aj , bj ), ⊂ [a, b], j = 1, 2, . . . , k are such that δ, then k

k

j=1 (bj − aj )

<

 |F (bj ) − F (aj )| ≤

j=1


k

j=1

|f |dm < , Ij

k k since m( j=1 Ij ) ≤ j=1 (bj − aj ) < δ and (4.2) holds. Thus, F is a.c. Next consider the “only if part.” It is not difficult to verify (Problem 4.18) that F a.c. implies F is of bounded variation on any finite interval [a, b] and both the positive and the negative variations of F on [a, b] are a.c. as well. Hence, it suffices to establish (4.1) assuming that F is a.c. and nondecreasing. Let µF be the Lebesgue-Stieltjes measure generated by F˜ as in Definition 4.4.2. It will now be shown that µF is absolutely continuous w.r.t. the Lebesgue measure. Fix  > 0. Let δ > 0 be chosen so that (aj , bj ) ⊂ [a, b], j = 1, 2, . . . , k,

k j=1

(bj − aj ) < δ ⇒

k j=1

|F (bj ) − F (aj )| < .

130

4. Differentiation

Let A ∈ Mm , A ⊂ (a, b), and m(A) = 0. Then, there exist a countable collection of disjoint open intervals {Ij = (aj , bj ) : Ij ⊂ [a, b]}j≥1 such that

Ij and (bj − aj ) < δ. A⊂ j≥1

j≥1

Thus

 k

Ij ≤ µF A ∩

µF

k

j=1

 Ij

i=1

≤

k

µF (Ij ) =

j=1

k

(F (bj ) − F (aj )) < 

j=1

for all k ∈ N.  Ij , by the m.c.f.b. property of µF (·), µF (A) = Since A ⊂ j≥1 k limk→∞ µF (A ∩ j=1 Ij ) ≤ . This being true for any  > 0, it follows
that µF (A) = 0. Since F is continuous, µF ({a, b}) = 0 and hence µF (a, b)c = 0. Thus, µF  m, i.e., µF is dominated by m. Now, by the Radon-Nikodym theorem (cf. Theorem 4.1.1 (ii)), there exists a nonnegative measurable function f such that A ∈ Mm implies that  µF (A) = A∩[a,b] f dm and, in particular, for a ≤ x ≤ b,  µF ([a, x]) = F (x) − F (a) = f dm, [a,x]

2

i.e., (4.1) holds.

The representation (4.1) of an absolutely continuous F can be strengthened as follows: Theorem 4.4.2: Let F : R → R satisfy (4.1). Then F is differentiable a.e. (m) and F  (·) = f (·) a.e. (m). For a proof of this result, see Royden (1988), Chapter 4. The relation between the notion of absolute continuity of a distribution function F : R → R and that of the associated Lebesgue-Stieltjes measure µF w.r.t. Lebesgue measure m will be discussed now. Let F : R → R be a distribution function, i.e., F is nondecreasing and right continuous. Let µF be the associated Lebesgue-Stieltjes measure such that µF ((a, b]) = F (b) − F (a) for all −∞ < a < b < ∞. Recall that F is said to be absolutely continuous on an interval [a, b] if for each  > 0, there exists a δ > 0 such that for any finite collection of intervals Ij = (aj , bj ), j = 1, 2, . . . , n, contained in [a, b], n j=1

(bj − aj ) < δ

⇒

n j=1

(F (bj ) − F (aj )) < .

4.4 Absolutely continuous functions on R

131

Recall also that µF is absolutely continuous w.r.t. the Lebesgue measure m if for A ∈ B(R), m(A) = 0 ⇒ µF (A) = 0. A natural question is that if F is absolutely continuous on every interval [a, b] ⊂ R, is µF absolutely continuous w.r.t. m and conversely? The answer is yes. Theorem 4.4.3: Let F : R → R be a nondecreasing function and let µF be the associated Lebesgue-Stieltjes measure. Then F is absolutely continuous on [a, b] for all −∞ < a < b < ∞ iff µF  m where m is the Lebesgue measure on (R, B(R)). Proof: Suppose that µF  m. Then by Theorem 4.1.1, there exists a nonnegative measurable function h such that  hdm for all A in B(R). µF (A) = A

Hence, for any a < b in R and a < x < b, F (x) − F (a) ≡ µF ((a, x]) =

 hdm. (a,x]

This implies the absolute continuity of F on [a, b] as shown in Theorem 4.4.1. Conversely, if F is absolutely continuous on [a, b] for all −∞ < a < b < ∞, then as shown in the proof of the “only if” part of Theorem 4.4.1, for all −∞ < a < b < ∞, then µF (A ∩ [a, b]) = 0 if m(A ∩ [a, b]) = 0. Thus, if m(A) = 0, then for all −∞ < a < b < ∞, m(A ∩ [a, b]) = 0 and hence 2 µF (A ∩ [a, b]) = 0 and hence µF (A) = 0, i.e., µF  m. Recall that a measure µ on (Rk , B(Rk )) is a Radon measure if µ(A) < ∞ for every bounded Borel set A. In the following, let m(·) denote the Lebesgue measure on Rk . Definition 4.4.3: A Radon measure µ on (Rk , B(Rk )) is differentiable at x ∈ Rk with derivative (Dµ)(x) if for any  > 0, there is a δ > 0 such that * * µ(A) * * − (Dµ)(x)* <  * m(A) for every open ball A such that x ∈ A and diam. (A) ≡ sup{x − y : x, y ∈ A}, the diameter of A, is less than δ. Theorem 4.4.4: Let µ be a Radon measure on (Rk , B(Rk )). Then (i) µ is differentiable a.e. (m), Dµ(·) is Lebesgue measurable, and ≥ 0 a.e. (m) and for all bounded Borel sets A ∈ B(Rk ),  Dµ(·)dm ≤ µ(A). A

132

4. Differentiation

 (ii) Let µa (A) ≡ A Dµ(·)dm, A ∈ B(Rk ). Let µs (·) be the unique measure on B(Rk ) such that for all bounded Borel sets A µs (A) = µ(A) − µa (A). Then µs ⊥ m and Dµs (·) = 0

a.e. (m).

For a proof, see Rudin (1987). Remark 4.4.1: By the uniqueness of the Lebesgue decomposition, it follows that a Radon measure µ on B(Rk ) is ⊥ m iff Dµ(·) = 0 a.e. (m)  and is  m iff µ(A) = A Dµ(·)dm for all A ∈ B(Rk ). k integrable w.r.t. m on bounded sets. Let µ(A) ≡  Let f : R → R+ be k f dm for A ∈ B(R ). Then µ(·) is a Radon measure and that is  m A and hence by Theorem 4.4.4

Dµ(x) = f (x) for almost all

x(m).

That is, for almost all x(m), for each  > 0, there is a δ > 0 such that * 1  * * * f dm − f (x)* <  * m(A) A for all open balls A such that x ∈ A and diam. (A) < δ. It turns out that a stronger result holds. Theorem 4.4.5: For almost all x(m), for each  > 0, there is a δ > 0 such that  1 |f − f (x)|dm <  m(A) A for all open balls A such that x ∈ A and diam. (A) < δ (see Problems 4.23, 4.24). Theorem 4.4.6: (Change of variables formula in Rk , k > 1). Let V be an open set in Rk . Let T ≡ (T1 , T2 , . . . , Tk ) be a map from Rk → Rk such i (·) that for each i, Ti : Rk → R and ∂T ∂xj exists on V for all 1 ≤ i, j ≤ k.

i (·)  Suppose that the Jacobian JT (·) ≡ det ∂T is continuous and positive ∂xj on V . Suppose further that T (V ) is a bounded open set W in Rk and that T is (1 − 1) and T −1 is continuous. Then (i) For all Borel set E ⊂ V , T (E) is a Borel set ⊂ W . (ii) ν(·) ≡ m(T (·)) is a measure on B(W ) and ν  m with dν(·) = JT (·). dm

4.5 Singular distributions

(iii) For any h ∈ L1 (W, m) 

 hdm =

W

133


 h T (·) JT (·)dm.

V

(iv) λ(·) ≡ mT −1 (·) is a measure on B(W ) and λ  m with
 dλ = |J T −1 (·) |−1 . dm (v) For any µ  m on B(V ) the measure ψ(·) ≡ µT −1 (·) is dominated by m with  dµ 

−1 dψ (·) = T −1 (·) JT T −1 (·) dm dm

on W.

For a proof see Rudin (1987), Chapter 7.

4.5 Singular distributions 4.5.1

Decomposition of a cdf

Recall that a cumulative distribution function (cdf) on R is a function F : R → [0, 1] such that it is nondecreasing, right continuous, F (−∞) = 0, F (∞) = 1. In Section 2.2, it was shown that any cdf F on R can be written as (5.1) F = αFd + (1 − α)Fc , where Fd and Fc are discrete and continuous cdfs respectively. In this section, the cdf Fc will be further decomposed into a singular continuous and absolutely continuous cdfs. Definition 4.5.1: A cdf F is singular if F  ≡ 0 almost everywhere w.r.t. the Lebesgue measure on R. Example 4.5.1: The cdfs of Binomial, Poisson, or any integer valued random variables are singular. It is known (cf. Royden (1988), Chapter 5) that a monotone function F : R → R is differentiable almost everywhere w.r.t. the Lebesgue measure and its derivative F  satisfies  b F  (x)dx ≤ F (b) − F (a), (5.2) a

for any −∞ < a < b < ∞.

134

4. Differentiation

x For x ∈ R, let F˜ac (x) ≡ −∞ Fc (t)dt and F˜sc (x) ≡ Fc (x) − F˜ac (x). If ∞ β˜ ≡ −∞ Fc (t)dt = F˜ac (∞) = 0, then Fc (t) = 0 a.e. and so Fc is singular continuous. If β˜ = 1, then Fc = F˜ac and so, Fc is absolutely continuous. If 0 < α < 1 and 0 < β˜ < 1, then F can be written as F = αFd + βFac + γFsc ,

(5.3)

˜ −1 F˜sc , ˜ γ = (1 − α)(1 − β), ˜ Fac = β˜−1 F˜ac , Fsc = (1 − β) where β = (1 − α)β, and Fd is as in (5.1). Note that Fd , Fac , Fsc are all cdfs and α, β, γ are nonnegative numbers adding up to 1. Summarizing the above discussions, one has the following: Proposition 4.5.1: Given any cdf F , there exist nonnegative constants α, β, γ and cdfs Fd , Fac , Fsc satisfying (a) α + β + γ = 1, and (b) Fd is discrete, Fac is absolutely continuous, Fsc is singular continuous, such that the decomposition (5.3) holds. It can be shown that the constants α, β, and γ are uniquely determined, and that when 0 < α < 1, the decomposition (5.1) is unique, and that when 0 < α, β, γ < 1, the decomposition (5.3) is unique. The decomposition (5.3) also has a probabilistic interpretation. Any random variable X can be realized as a randomized choice over three random variables Xd , Xac , and Xsc having cdfs Fd , Fac , and Fsc , respectively, and with corresponding randomization probabilities α, β, and γ. For more details see Problem 6.15 in Chapter 6.

4.5.2

Cantor ternary set

Recall the construction of the Cantor set from Section 1.3. Let I0 = [0, 1] denote the unit interval. If one deletes the open " middle # third of" I0 ,# then one gets two disjoint closed intervals I11 = 0, 13 and I12 = 23 , 1 . Proceeding similarly with" the# closed intervals " 2 1 # I11 and " 2 I712#, 1 one gets " four # disjoint intervals I21 = 0, 9 , I22 = 9 , 3 , I23 = 3 , 9 , I24 = 89 , 1 , and so on. Thus, at each step, deleting the open middle third of the closed intervals constructed in the previous step, one is left with 2n 2n disjoint closed intervals each of length 3−n after n steps. Let Cn = j=1 Inj ∞ and C = n=1 Cn . By construction Cn+1 ⊂ Cn for each n and Cn ’s are closed sets. With m(·)
ndenoting Lebesgue measure, one has m(C0 ) = 1 and m(Cn ) = 2n 3−n = 23 . ∞ Definition 4.5.2: The set C ≡ n=1 Cn is called the Cantor ternary set or simply the Cantor set.
n Since m(C0 ) = 1, by m.c.f.a. m(C) = limn→∞ m(Cn ) = limn→∞ 23 = 0. Thus,
the Cantor set C has zero Lebesgue measure. Next, let U1 = U11 = 13 , 23 be the deleted interval at the first stage, U2 = U21 ∪ U22 =

4.5 Singular distributions

135


1


7 8 ∪ 9 , 9 be the union of the deleted intervals at the second stage, and  2n−1 ∞
2n−1 similarly, Un = j=1 Unj at stage n. Thus C c = U = n=1 j=1 Unj is open and m(C c ) = 1. Since C ∪ C c = [0, 1] and C c is open, it follows that C is nonempty. In fact, C is uncountably infinite as will be shown now. To do this, one needs the concept of p-nary expansion of numbers in [0,1]. Fix a positive integer p > 1. For each x in [0,1), let a1 (x) = px where t = n if n ≤ t < n + 1. Thus a1 (x) ≤ px < a1 (x) + 1 and a1 (x) ∈ {0, 1, . . . , p − 1}, i.e., a1p(x) ≤ x < a1p(x) + p1 . Thus, if kp ≤ x < k+1 p 2 9, 9

for some k = 0, 1, 2, . . . , p − 1, then a1 (x) = k. Next, let x1 ≡ x − a1p(x) and  " a2 (x) = p2 x1 . Then, x1 ∈ 0, p1 and a2 (x) a1 (x) 1 a2 (x) < ≤ x1 = x − + 2 2 2 p p p p and a2 (x) ∈ {0, 1, 2, p − 1}. Next, let 0 ≤ x2 ≡ x − a1p(x) − a3 (x) = p3 x2  and so on. After k such iterations one gets 0≤x−

k ai (x) i=1

pi

<

∞ ai (x) i=1

pi

<

1 p2

and

1 pk

where ai (x) ∈ {0, 1, 2, . . . , p − 1} for all i. Since the p-nary expansion of x in [0,1) as x=

a2 (x) p2

1 pk

→ 0 as k → ∞, one gets

, ai (x) ∈ {0, 1, 2, . . . , p − 1}.

(5.4)

Notice that if x = kp for some k ∈ {0, 1, 2, . . . , p − 1}, then in the above expansion a1 (x) = k and ai (x) = 0 for i ≥ 2, and the expansion terminates. ∞ ∞ (p−1) But since i=1 (p−1) = 1, one may also write x = kp = k−1 i=1 pi , pi p + this being an expansion which is nonterminating and recurring. It can be shown that for all x in [0,1) of the form x = p m for some positive integers  and m, there are exactly two expansions such that one terminates and the other is nonterminating and recurring. For all other x in [0,1), the p-nary expansion is nonterminating and nonrecurring and is unique. The decimal expansion corresponds with the case p = 10, the binary expansion with the case p = 2, and the ternary expansion with the case p = 3. Here the convention of choosing only a nonterminating expansion for  each x in [0,1) is used. for p = 3, 13 will be replaced
1  Thus, for
1 example,  ∞ 2 by i=2 3i so that a1 3 = 0, ai 3 = 2 for i ≥ 2. Similarly, for
7 p = 3, ∞ 2 7 2 1 2 0 x= = + will be replaced by + + so that a 2 i 1 i=3 3 3 3 3 9 = 2,
9
9  a2 79 = 0, ai 79 = 2 for i ≥ 3. By taking p = 2, i.e., the binary expansion, ∞ it is seen that every x in [0,1) can be uniquely represented as x = i=1 δi2(x) i

136

4. Differentiation

where δi (x) ∈ {0, 1} for all i. Thus, the interval [0,1) is in one-to-one correspondence with the set of all sequences of 0’s and 1’s. It is not difficult to prove the following (Problem 4.31). Theorem 4.5.2: A number x belongs to the Cantor set C iff in (5.4), for all i ≥ 1, ai (x) is either 0 or 2. Corollary 4.5.3: The Cantor set C is in one-to-one correspondence with the set of all sequences of 0’s and 1’s and hence is in one-to-one correspondence with the unit interval [0,1]. Remark 4.5.1: Thus the Cantor ternary set C is a closed subset of [0,1], its Lebesgue measure m(C) = 0, and its cardinality is the same as that of [0,1]. Further, it is nowhere dense, i.e., its complement U is dense in the sense that for every open interval (a, b) ⊂ [0, 1], U ∩ (a, b) is nonempty. It is also possible to get a Cantor like set Cα with (Lebesgue) measure α, 0 < α < 1, by following the above iterative procedure of deleting at each stage intervals of length that is a fraction (1−α) of the full interval (Problem 3 4.32).

4.5.3

Cantor ternary function

The Cantor ternary function F : [0, 1] → [0, 1] is defined as follows: For n ≥ 1, let {Unj : j = 1, . . . , 2n−1 } denote the set of “deleted” intervals at step n in the definition of the Cantor set C. Define F on C c = U by F (x)

= = =

1 2 1 4 3 4

1 2 , and 3 3 1 2 , = 9 9 7 8 , = 9 9

on

U11 =

on

U21

on

U22

and so on. It can be checked that F is uniformly continuous on U and has a continuous extension to I0 = [0, 1]. The extension of the function F (also denoted by F ) maps [0,1] onto [0,1] and is continuous and nondecreasing. Further, on U , it is differentiable with derivative F  ≡ 0. Since m(U c ) = 0, F is a singular cdf (cf. see Definition 4.5.1). It can be shown that if ∞ ai (x) is the ternary expansion of x ∈ (0, 1), then i=1 3i N (x)−1

F (x) =

ai (x) 1 + N (x) i+1 2 2 i=1

(5.5)

∞ = 13 = i=2 32i ⇒ Where N (x) = inf{i : i ≥ 1, ai (x) = 1}. For example, x ∞ N (x) = ∞, a1 (x)  = 0, ai (x) = 2 for i ≥ 2 ⇒ F (x) = i=2 21i = 12 while ∞ 2 4 1 0 1 x = 9 = 3 + 32 + i=3 3i ⇒ N (x) = 1 ⇒ F (x) = 2 .

4.6 Problems

137

It can be shown that if {δn }n≥1 is a sequence of independent {0, 1} valued random variables with P (δ1 = 0) = 12 = P (δ1 = 1), then the above F is ∞ i the cdf of the random variable X = i=1 2δ 3i which lies in the Cantor set w.p. 1.

4.6 Problems 4.1 Show that in the proof of Theorem 4.1.1, µ(A3 ) = 0 where A3 = {g ∈ [0, 1]} and g satisfies (1.9). (Hint: Apply (1.9) separately to A31 = {g > 1} and A32 = {g < 0}.) 4.2 Verify that µ1s , defined in (1.14) and µ2 are singular. (Hint: For each n, by Case 1, there exists a gn in L2 (Dn , Fn , µ(n) ) (n) (n) where µ(n) = µ1 + µ2 and Fn ≡ {A ∩ Dn : A ∈ F}, such that 0 ≤  g dµ(n) ≤ µ(n) (A) for all A in Fn . Let A1n = {w : w  ∈ Dn , 0 ≤ A n gn (w) < 1}, A2n = {w : w ∈ Dn , gn (w) = 1}, and A2 = n≥1 A2n . ∞ Show that µ2 (A2 ) = 0 and µ1s (A) = n=1 µ1n (A ∩ A2n ) and hence µ1s (Ac2 ) = 0.)  4.3 Let ν be the counting measure on [0, 1] and [0,1] hdν < ∞ for some nonnegative function h. Show that B = {x : h(x) > 0} is countable. (Hint: Let Bn = {x : h(x) > each n ∈ N.)

1 n }.

Show that Bn is a finite set for

4.4 Prove Proposition 4.1.2. 4.5 Find the Lebesgue decomposition of µ w.r.t. ν and the Radona Nikodym derivative dµ dν in the following cases where µa is the absolutely continuous component of µ w.r.t. ν. (a) µ = N (0, 1), ν = Exponential(1) (b) µ = Exponential(1), ν = N (0, 1) (c) µ = µ1 + µ2 , where µ1 = N (0, 1), µ2 = Poisson(1) and ν = Cauchy(0, 1). (d) µ = µ1 + µ2 , ν = Geometric(p), 0 < p < 1, where µ1 = N (0, 1) and µ2 = Poisson(1). (e) µ = µ1 + µ2 , ν = ν1 + ν2 where µ1 = N (0, 1), µ2 = Poisson(1), ν1 = Cauchy(0, 1) and ν2 = Geometric(p), 0 < p < 1. (f) µ = Binomial (10, 1/2), ν = Poisson (1). The measures referred to above are defined in Tables 4.6.1 and 4.6.2, given at the end of this section.

138

4. Differentiation

4.6 Let (Ω, F, µ) be a measure space and f ∈ L1 (Ω, F, µ). Let  νf (A) ≡ f dµ for all A ∈ F. A

(a) Show that νf is a finite signed measure.   (b) Show that ν = Ω |f |dµ and for A ∈ F, νf+ (A) = A f + dµ,   νf− (A) = A f dµ, and |νf |(A) = A |f |(dµ). 4.7 (a) Let µ1 and µ2 be two finite measures such that both are dominated by a σ-finite measure ν. Show that the total variation measure of the signed measure µ ≡ µ1 − µ2 is given by  i |µ|(A) = A |h1 − h2 |dν where for i = 1, 2, hi = dµ dν . (b) Conclude that if µ1 and µ2 are two measures on a countable set Ω ≡ {ωi }i≥1 with F ≡ P(Ω), then |µ|(A) = i∈A |µ1 (ωi ) − µ2 (ωi )|. (c) Show that if µn is the Binomial (n, pn ) measure and µ is the Poisson (λ) measure, 0 < λ < ∞, then as n → ∞, |µn −µ|(·) → 0 uniformly on P(Z+ ) iff npn → λ. (Hint: Show that for each i ∈ Z+ ≡ {0, 1, 2, . . .}, µn ({i}) → µ({i}) and use Scheffe’s theorem.) 4.8 Let ν be a finite signed measure on a measurable space (Ω, F) and let |ν| be the total variation measure corresponding to ν. Show that for any B ∈ Ω+ , B ⊂ F, |ν|(B) = ν(B), where Ω+ is as defined in (2.13). (Hint: For any set A ⊂ Ω+ ,   ν(A) = hd|ν| = A

A∩Ω+

hd|ν| ≥ 0.)

4.9 Show that the Banach space S of finite signed measures on (N, P(N)) is isomorphic to 1 , the Banach space of absolutely convergent sequences {xn }n≥1 in R. 4.10 Let µ1 and µ2 be two probability measures on (Ω, F). (a) Show that µ1 − µ2  = 2 sup{|µ1 (A) − µ2 (A)| : A ∈ F}. (Hint: For any A ∈ F, {A, Ac } is a partition of Ω and so µ1 − µ2  ≥ |µ1 (A) − µ2 (A)| + |µ1 (Ac ) − µ2 (Ac )| = 2|µ1 (A) − µ2 (A)|,

4.6 Problems

139

since µ1 and µ2 are probability measures. For the opposite inequality, use the Hahn decomposition of Ω w.r.t. µ1 − µ2 and the fact µ1 − µ2  = |(µ1 − µ2 )(Ω+ )| + |(µ1 − µ2 )(Ω− )|.) (b) Show that µ1 − µ2  is also equal to  * *   * * sup * f dµ1 − f dµ2 * : f ∈ B(Ω, R) where B(Ω, R) is the collection of all F-measurable functions from Ω to R such that sup{|f (ω)| : ω ∈ Ω} ≤ 1. 4.11 Let (Ω, F) be a measurable space. (a) Let {µn }n≥1 be a sequence of finite measures on (Ω, F). Show that there exists a probability measure λ such that µn  λ. ∞ (·) (Hint: Consider λ(·) = n=1 21n µµnn(Ω) .) (b) Extend (a) to the case where {µn }n≥1 are σ-finite. (c) Conclude that for any sequence {νn }n≥1 of finite signed measures on (Ω, F), there exists a probability measure λ such that |νn |  λ for all n ≥ 1. 4.12 Let {µn }n≥1 be a sequence of finite measures on a measurable space (Ω, F). Show that there exists a finite measure µ on (Ω, F) such that µn − µ → 0 iff there is a finite measure λ dominating µ and µn , dµ n n ≥ 1 such that the Radon-Nikodym derivatives fn ≡ dµ dλ → f ≡ dλ in measure on (Ω, F, λ) and µn (Ω) → µ(Ω). 4.13 (a) Let µ1 and µ2 be two finite measures on (Ω, F). Let µ1 = µ1a + µ1s be the Lebesgue-Radon-Nikodym decomposition of µ1 w.r.t. µ2 as in Theorem 4.1.1. Show that if µ = µ1 − µ2 , then for all A ∈ F,  dµ1a |h − 1|dµ2 + µ1s (A) where h = |µ|(A) = dµ2 A is the Radon-Nikodym derivative of µ1a w.r.t. µ2 . Conclude that if µ1 ⊥ µ2 , *then |µ|(·) * = µ1 (·) + µ2 (·) and if µ1  µ2 , then 1 *dµ2 . − 1 |µ|(A) = A * dµ dµ2 (b) Compute |µ|(·), µ if µ = µ1 − µ2 for the following cases (i) µ1 = N (0, 1), µ2 = N (1, 1) (ii) µ1 = Cauchy (0,1), µ2 = N (0, 1) (iii) µ1 = N (0, 1), µ2 = Poisson (λ). (c) Establish Proposition 4.2.7.

140

4. Differentiation

4.14 Give another proof of the completeness of (S,  · )) by verifying the following steps. (a) For any sequence {νn }n≥1 in S, there is a finite measure λ and {fn }n≥1 ⊂ L1 (Ω, F, λ) such that  νn (A) = fn dλ for all A ∈ F, for all n ≥ 1. A

(b) {νn }n≥1 Cauchy in S is the same as {fn }n≥1 Cauchy in L1 (Ω, F, λ) and hence, the completeness of (S, ·)) follows from the completeness of L1 (Ω, F, λ). 4.15 Let f, g ∈ BV [a, b]. (a) Show that P (f + g; [a, b]) ≤ P (f ; [a, b]) + P (g; [a, b]) and that the same is true for N (·; ·) and T (·; ·). (b) Show that for any c ∈ R, P (cf ; [a, b]) = |c|P (f ; [a, b]) and do the same for N (·; ·) and T (·; ·). (c) For any a < c < b, P (f ; [a, b]) = P (f ; [a, c]) + P (f ; [c, b]). 4.16 Let {fn }n≥1 ⊂ BV [a, b] and let limn fn (x) = f (x) for all x in [a, b]. Show that P (f ; [a, b]) ≤ limn→∞ P (fn ; [a, b]) and do the same for N (·; ·) and T (·; ·). 4.17 Let f ∈ BV [a, b]. Show that f is continuous except on an at most countable set. 4.18 Let F : [a, b] → R be a.c. Show that it is of bounded variation. (Hint: By the definition of a.c., for  = 1, there is a δ1 > 0 such k k that j=1 |aj − bj | < δ1 ⇒ j=1 |F (aj ) − F (bj )| < 1. Let M be an integer > b−a δ + 1. Show that T (F, [a, b]) ≤ M .) 4.19 Let F be an absolutely continuous nondecreasing function on R. Let µF be the Lebesgue-Stieltjes measure corresponding to F . Show that for any h ∈ L1 (R, MµF , µF ),   hdµF = hf dm R

where f is a nonnegative measurable function such that F (b)−F (a) =  f dm for any a < b. [a,b]

4.6 Problems

141

4.20 Let F : [a, b] → R be absolutely continuous with F  (·) > 0 a.e. on [a, b], where −∞ < a < b < ∞. Let F (a) = c and F (b) = d. Let m(·) denote the Lebesgue measure on R. Show the following: (a) (Change of variables formula). For any g : [c, d] → R and Lebesgue measurable and integrable w.r.t. m   gdm = g(F )F  dm. [c,d]

[a,b]

(b) For any Borel set E ⊂ [a, b], F (E) is also a Borel set.
 (c) ν(·) ≡ m F (·) is a measure on B([a, b]) and ν  m with dν (·) = F  (·). dm (d) λ(·) ≡ mF −1 (·) is a measure on B([c, d]) and λ  m with 
−1 dλ (·) = F  F −1 (·) . dm (e) For any measure µ  m on B([a, b]) the measure ψ(·) ≡ µF −1 (·) is dominated by m with dµ
−1  
−1 −1 dψ = F (·) F F (·) . dm dm (f) Establish (a) assuming that g and F  are both continuous noting that both integrals reduce to Riemann integrals. (Hint: (i) Verify (a) for g = I[a,b] , c < α < β < d and approximate by step functions. (ii) Show that F is (1 − 1) and F −1 (·) is continuous and hence Borel measurable. (iii) Show that ν(·) = µF , the Lebesgue-Stieltjes measure corresponding to F . (iv) Use the fact that for any c ≤ α ≤ β ≤ d, ψ([α, β])

= µ([γ, δ]), where γ = F −1 (α), δ = F −1 (β),  dµ gdm, where g = = dm [γ,δ]  
  g F −1 F (·)   =
 F (·)dm [γ,δ] F  F −1 F (·)
  g F −1 (·)
 dm by (a). ) =  −1 (·) [α,β] F F

142

4. Differentiation

4.21 Let F : R → R be absolutely continuous on every finite interval. (a) Show that the f in (4.1) can be chosen independently of the interval [a, b]. (b) Further, if f is integrable over R, then limx→−∞ F (x) ≡ F (−∞) and limx→∞ F (x) ≡ F (∞) exist and F (x) = F (−∞) + f dµL for all x in R. (−∞,x) (c) Give an example where F : R → R is a.c., but f is not integrable over R. 4.22 Let F : R → R be absolutely continuous on bounded intervals. Let {I = 1 ≤ j ≤ k ≤ ∞} be a collection of disjoint intervals such that kj   j=1 Ij ≡ R and on each Ij , F (·) > 0 a.e. or F (·) < 0 a.e. w.r.t. m. (a) Show that for any h ∈ L1 (R, m),  
 hdm = h F (·) |F  (·)|dm. R

R


 (b) Show that if µ is a measure on R, B(R) dominated by m then the measure µF −1 (·) is also dominated by m and dµF −1 (y) = dm

xj ∈D(y)

f (xj ) |F  (xj )|

and D(y) = {xj : xj ∈ Ij , F (xj ) = y}.
 (c) Let µ be the N (0, 1) measure on R, B(R) , i.e., where f (·) =

dµ dm

x2 1 dµ (x) = √ e− 2 , −∞ < x < ∞. dm 2n

Let F (x) = x2 . Find

dµF −1 dm .

4.23 Let f : R → R be integrable w.r.t. m on bounded intervals. Show that for almost all x0 in R (w.r.t. m), lim

a↑x0 b↓x0

1 (b − a)



b

|f (x) − f (x0 )|dx = 0. a

(Hint: For each rational r, by Theorem 4.4.4 1 lim a↑x0 (b − a) b↓x 0



b

|f (x) − r|dx = |f (x0 ) − r|. a

4.6 Problems

143

a.e. (m).  Let Ar denote the set of x0 for which this fails to hold. Let A = r∈Q Ar . Then m(A) = 0. For any x0 ∈ A and any  > 0, choose a rational r such that |f (x0 ) − r| <  and now show that  b 1 lim |f (x) − f (x0 )|dx < . ) a↑x0 (b − a) a b↓x 0

4.24 Use the hint to the above problem to establish Theorem 4.4.5. 4.25 Let (Ω, B) be a measurable space. Let {µn }n≥1 and µ be σ-finite measures on (Ω, B). Let for each n ≥ 1, µn = µna + µns be the Lebesgue  of µn w.r.t. µ with  µna  µ and µns ⊥ µ. Let  decomposition λ = n≥1 µn , λa = n≥1 µna , λs = n≥1 µns . Show that λa  µ and λs ⊥ µ and that λ = λa + λs is the Lebesgue decomposition of λ w.r.t. µ.
 4.26 Let {µn }n≥1 be Radon measures on Rk , B(Rk ) and m be the ∞ Lebesgue measure on Rk . Show that if λ = n=1 µn is also a Radon measure, then ∞ Dµn a.e. (m). Dλ = n=1

(Hint: Use Theorem 4.4.4 and the uniqueness of the Lebesgue decomposition.) 4.27 Let Fn , n ≥ 1 be a sequence of nondecreasing functions from R → R. Let F (x) = n≥1 Fn (x) < ∞ for all x ∈ R. Show that F (·) is nondecreasing and Fn (·) a.e. (m). F  (·) = n≥1

4.28 Let E be a Lebesgue measurable set in R. The metric density of E at x is defined as
 m E ∩ (x − δ, x + δ) DE (x) ≡ lim δ↓0 2δ if it exists. Show that DE (·) = IE (·) a.e. m. (Hint: Consider the measure λE (·) ≡ m(E ∩ ·) on the Lebesgue σ-algebra. Show that λE  m and find λE (·) (cf. Definition 4.4.2).) 4.29 Let F , G : [a, b] → R be both absolutely continuous. Show that H = F G is also absolutely continuous on [a, b] and that   F dG + GdF = F (b)G(b) − F (a)G(a). [a,b]

[a,b]

144

4. Differentiation

4.30 Let (Ω, F, µ) be a finite measure space. Fix 1 ≤ p < ∞. Let T : Lp (µ) → R be a bounded linear functional as defined in (3.2.10) (cf. Section 3.2). Complete the following outline of a proof of Theorem 3.2.3 (Riesz representation theorem). (a) Let ν(A) ≡ T (IA ), A ∈ F. Verify that ν(·) is a signed measure on (Ω, F). (b) Verify that |ν|  µ. dν (c) Let g ≡ dµ . Show that g ∈ Lq (µ) where q = and q = ∞ if p = 1.

p p−1 ,

1
(d) Show that T = Tg . 4.31 Prove Theorem 4.5.2 and Corollary 4.5.3. 4.32 For 0 < α < 1, construct a Cantor like set Cα as described in Remark 4.5.1. 4.33 Show that the Cantor ternary function can be expressed as in (5.5). 4.34 Let (Ω, F, µ) be a σ-finite measure space. (a) Let G be a σ-algebra ⊂ F. Let f : Ω → R+ be F, B(R)measurable. Show that there exists a g : Ω → R+ that is G, B(R)-measurable and ν(A) ≡ A f dµ = A gdµ for all A in G. (Hint: Apply Theorem 4.1.1 (b) to the measures ν and µ restricted to G. When µ is a probability measure, g is called the conditional expectation of f given G (cf. Chapter 12).) (b) Now suppose G = σ{Ai }i≥1  where {Ai }i≥1 is a partition of Ω ⊂ F. Determine g(·) explicitly on each Ai such that 0 < µ(Ai ) < ∞. TABLE 4.6.1. Some discrete univariate distributions.

Mean µ Bernoulli (p), 0
1

µ(A), A ∈ B(R)

i=0

n i=0

pi (1 − p)1−i IA (i)


n i n−i IA (i) i p (1 − p)

∞ i=0

p(1 − p)i−1 IA (i)

∞ i=0

i

e−λ λi! · IA (i)

4.6 Problems

145

TABLE 4.6.2. Some standard absolutely continuous distributions. Here m denotes the Lebesgue measure on (R, B(R)).

µ(A), A ∈ B(R)

Measure µ Uniform (a, b), −∞ < a < b < ∞ Exponential (β), β ∈ (0, ∞) Gamma (α, β), α, β ∈ (0, ∞) Beta (α, β), α, β ∈ (0, ∞) Cauchy (γ, σ) γ ∈ R, σ ∈ (0, ∞) Normal (γ, σ 2 ), γ ∈ R, σ ∈ (0, ∞) Lognormal (γ, σ 2 ), γ ∈ R, σ ∈ (0, ∞)

m(A ∩ [a, b])/(b − a) 

1 A β

exp(−x/β)I(0,∞) (x)m(dx)



1 xα−1 exp(−x/β)I(0,∞) (x)m(dx) A Γ(α)β α ∞ where Γ(a) = 0 xa−1 e−x dx, a ∈ (0, ∞) 1 B(α,β)



xα−1 (1 − x)β−1 I(0,1) (x)m(dx), where B(a, b) = Γ(a)Γ(b)/Γ(a + b), a, b ∈ (0, ∞)  1 σ2 m(dx) A πσ σ 2 +(x−γ)2  A



A

√1 2πσ

exp(−(x − γ)2 /σ 2 )m(dx) 2 /2σ 2

e−(log x−γ) √1 x A 2πσ

I(0,∞) (x)m(dx)

5 Product Measures, Convolutions, and Transforms

5.1 Product spaces and product measures Given two measure spaces (Ωi , Fi , µi ), i = 1, 2, is it possible to construct a measure µ ≡ µ1 ×µ2 on a σ-algebra on the product space Ω1 ×Ω2 such that µ(A × B) = µ1 (A)µ2 (B) for A ∈ F1 and B ∈ F2 ? This section is devoted to studying this question. Definition 5.1.1: Let (Ωi , Fi ), i = 1, 2 be measurable spaces. (a) Ω1 × Ω2 ≡ {(ω1 , ω2 ) : ω1 ∈ Ω1 , ω2 ∈ Ω2 }, the set of all ordered pairs, is called the (Cartesian) product of Ω1 and Ω2 . (b) The set A1 × A2 with A1 ∈ F1 , A2 ∈ F2 is called a measurable rectangle. The collection of measurable rectangles will be denoted by C. (c) The product σ-algebra of F1 and F2 on Ω1 × Ω2 , denoted by F1 , ×F2 , is the smallest σ-algebra generated by C, i.e., F1 × F2 ≡ σ{A1 × A2 : A1 ∈ F1 , A2 ∈ F2 }. (d) (Ω1 × Ω2 , F1 × F2 ) is called the product measurable space. Starting with the definition of µ on the class C of measurable rectangles by µ(A1 × A2 ) = µ1 (A1 )µ2 (A2 ) (1.1)

148

5. Product Measures, Convolutions, and Transforms

for all A1 ∈ F1 , A2 ∈ F2 , one can extend it to a measure on the algebra A of all finite unions of disjoint measurable rectangles in a natural way. Indeed, the extension to A is obtained simply by assigning the µ-measure of a finite union of disjoint measurable rectangles as the sum of the µmeasures of the corresponding individual measurable rectangles. Then, by the extension theorem (cf. Theorem 1.3.3), it can be further extended to a complete measure µ on a σ-algebra containing F1 × F2 , defined in (c) above. However, to evaluate µ(A) for an arbitrary A in F1 × F2 that is  not a measurable rectangle and to evaluate hdµ for arbitrary measurable functions h, some further work is needed. For the case of the product of two measure spaces, here an alternate approach that yields a direct way of computing these quantities is presented. The extension theorem approach will be used for the more general case of products of finitely many measure spaces in Section 5.3. The case of products of infinitely many probability spaces will be discussed in Chapter 6. Definition 5.1.2: (a) Let A ∈ F1 × F2 . Then, for any ω1 ∈ Ω1 , the set A1ω1 ≡ {ω2 ∈ Ω2 : (ω1 , ω2 ) ∈ A}

(1.2)

is called the ω1 -section of A and for any ω2 ∈ Ω2 , the set A2ω2 ≡ {ω1 ∈ Ω1 : (ω1 , ω2 ) ∈ A} is called the ω2 -section of A. (b) If f : (Ω1 × Ω2 ) → Ω3 is a F1 × F2 , F3  measurable mapping from Ω1 × Ω2 into some measurable space (Ω3 , F3 ), then the ω1 -section of f is the function f1ω1 : Ω2 → Ω3 , given by f1ω1 (ω2 ) = f (ω1 , ω2 ), ω2 ∈ Ω2 .

(1.3)

Similarly, one may define the ω2 -sections of f by f2ω2 (ω1 ) = f (ω1 , ω2 ), ω1 ∈ Ω1 . The following result shows that the ω1 -sections of a F1 × F2 measurable function is F2 -measurable when considered as a function of ω2 ∈ Ω 2 . Proposition 5.1.1: Let (Ω1 ×Ω2 , F1 ×F2 ) be a product space, A ∈ F1 ×F2 and let f : Ω1 × Ω2 → Ω3 be a F1 × F2 , F3 -measurable function. (i) For every ω1 ∈ Ω1 , A1ω1 ∈ F2 and for every ω2 ∈ Ω2 , A2ω2 ∈ F1 . (ii) For every ω1 ∈ Ω1 , f1ω1 is F2 , F3 -measurable and for every ω2 ∈ Ω2 , f2ω2 is F1 , F3 -measurable. Proof: Fix ω1 ∈ Ω1 . Define the function g : Ω2 → Ω1 × Ω2 by g(ω2 ) = (ω1 , ω2 ), ω2 ∈ Ω2 .

5.1 Product spaces and product measures

149

Note that for any measurable rectangle A ≡ A1 × A2 ∈ F1 × F2 ,  A2 if ω1 ∈ A1 A1ω1 = ∅ if ω1 ∈ A1 and hence g −1 (A1 × A2 ) ∈ F2 . Since the class of all measurable rectangles generates F1 × F2 , for fixed ω1 in Ω1 , g is F2 , F1 × F2 -measurable. Therefore, for A in F1 × F2 , A1ω1 = g −1 (A) ∈ F2 and for f as given, f1ω1 = f ◦ g is F2 , F3 -measurable. This proves (i) and (ii) for the ω1 -sections. The 2 proof for the ω2 -sections are similar. Now suppose µ1 and µ2 are measures on (Ω1 , F1 ) and (Ω2 , F2 ), respectively. Then, for any set A ∈ F1 × F2 , for all ω1 ∈ Ω1 , A1ω1 ∈ F2 and hence µ2 (A1ω1 ) is well defined. If this were an F1 -measurable function, then one might define a set function on F1 × F2 by  µ2 (A1ω1 )µ1 (dω1 ). (1.4) µ12 (A) = Ω1

And similarly, reversing the order of µ1 and µ2 , one might define a second set function  µ21 (A) = µ1 (A2ω2 )µ2 (dω2 ), (1.5) Ω2

provided that µ1 (A2ω2 ) is F2 -measurable. Note that for the measurable rectangles A = A1 × A2 , µ12 (A) = µ1 (A1 )µ2 (A2 ) = µ21 (A) and thus both µ12 and µ21 coincide (with the product measure µ) on the class C of all measurable rectangles. This implies that if the product measure µ is unique on F1 × F2 , and µ12 and µ21 are measures on F1 × F2 , then µ12 , µ21 and µ must coincide on the σ-algebra F1 × F2 . Then, one can evaluate µ(A) for any set A ∈ F1 × F2 using either of the relations (1.4) or (1.5). The following result makes this heuristic discussion rigorous. Theorem 5.1.2: Let (Ωi , Fi , µi ), i = 1, 2 be σ-finite measure spaces. Then, (i) for all A ∈ F1 × F2 , the functions µ2 (A1ω1 ) and µ1 (A2ω2 ) are F1 and F2 -measurable, respectively. (ii) The set functions µ12 and µ21 , given by (1.4) and (1.5) respectively, are measures on F1 × F2 , satisfying µ12 (A) = µ21 (A) for all A ∈ F1 × F2 . (iii) Further, µ12 = µ21 ≡ µ is σ-finite and it is the only measure satisfying µ(A1 × A2 ) = µ1 (A1 )µ2 (A2 ) for all A1 × A2 ∈ C. Proof: First suppose that µ1 and µ2 are finite measures. Define   L = A ∈ F1 × F2 : µ2 (A1ω1 ) is a F1 , B(R)-measurable function .

150

5. Product Measures, Convolutions, and Transforms

For A = Ω1 ×Ω2 , µ2 (A1ω1 ) ≡ µ2 (Ω2 ) for all ω1 ∈ Ω1 and hence Ω1 ×Ω2 ∈ L. Next, let A, B ∈ L with A ⊂ B. Then, it is easy to check that (A \ B)1ω1 = A1ω1 \ B1ω1 . Since µ2 is a finite measure and A, B ∈ L, it follows that µ2 ((A \ B)1ω1 ) = µ2 (A1ω1 \ B1ω1 ) = µ2 (A1ω1 ) − µ2 (B1ω1 ) is F1 , B(R)measurable. Thus, A \ B ∈ L. Finally, let {Bn }n≥1 ⊂ L be such that Bn ⊂ Bn+1 for all n ≥ 1. Then for any ω1 ∈ Ω1 , (Bn )1ω1 ⊂ (Bn+1 )1ω1 for all n ≥ 1. Hence by the MCT, 

  


 ∞ > µ2 = µ2 Bn (Bn )1ω1 = lim µ2 (Bn )1ω1 1ω1

n≥1

n→∞

n≥1


  for all ω1 ∈ Ω1 . By Proposition 2.1.5,this implies that µ2 ( n≥1 Bn )1ω1 is F1 , B(R)-measurable, and hence n≥1 Bn ∈ L. Thus, L is a λ-system. For A = A1 × A2 ∈ C, µ2 (A1ω1 ) = µ2 (A2 )IA1 (ω1 ) and hence C ⊂ L. Since C is a π-system, by Corollary 1.1.3, it follows that L = F1 × F2 . Thus, µ2 (A1ω1 ), considered as a function of ω1 , is F1 , B(R)-measurable for all A ∈ F1 × F2 , proving (i). Next, consider part (ii). By part (i), µ12 is a well-defined set function on F1 × F2 . It is easy to check that µ12 is a measure on F1 × F2 (Problem 5.1). Similarly, µ21 is a well-defined measure on F1 × F2 . Since µ12 (A) = µ21 (A) = µ1 (A1 )µ2 (A2 ) for all A = A1 × A2 ∈ C and C is a π-system generating F1 × F2 , it follows from Theorem 1.2.4 that µ12 (A) = µ21 (A) for all A ∈ F1 ×F2 . This proves (ii) for the case where µi (Ωi ) < ∞, i = 1, 2. Next, suppose that µi ’sare σ-finite. Then, there exist disjoint sets {Bin }n≥1 ⊂ Fi , such that n≥1 Bin = Ωi and µi (Bin ) < ∞ for all n ≥ 1, i = 1, 2. Define the finite measures µin (D) = µi (D ∩ Bin ),

D ∈ Fi ,

for n ≥ 1, i = 1, 2. The arguments above with µi replaced by µin imply that for any A ∈ F1 ×F2 , µ2n (A1ω1 ) is F1 , B(R)-measurable for all n ≥ 1. Since ∞ µ2 is a measure on F2 , µ2 (A1ω1 ) = n=1 µ2n (Aω1 ) and hence, considered ¯ for all A ∈ F1 × F2 . Thus, as a function of ω1 , it is F1 , B(R)-measurable the set function µ12 of (1.4) is well defined in the σ-finite case as well. (m,n) Similarly, µ21 of (1.5) is a well-defined set function on F1 × F2 . Let µ12 (m,n) and µ21 , respectively, denote the set functions defined by (1.4) and (1.5) with µ1 replaced by µ1m and µ2 replaced by µ2n , m ≥ 1, n ≥ 1. Then, by repeated use of the MCT,  µ12 (A) = µ2 (A1ω1 )µ1 (dω1 ) Ω1

= =

∞ 

∞

B1m n=1 m=1 ∞ ∞ 

µ2 (A1ω1

 ∩ B2n ) µ1 (dω1 )

µ2 (A1ω1 ∩ B2n )µ1 (dω1 )

m=1 n=1

B1m

5.1 Product spaces and product measures

=

∞ ∞

(m,n)

µ12

A ∈ F1 × F2

(A),

151

(1.6)

m=1 n=1

and similarly, µ21 (A) =

∞ ∞

(m,n)

µ21

(A),

A ∈ F1 × F2 .

(1.7)

n=1 m=1 (m,n)

(m,n)

Since µ12 and µ21 are (finite) measures, it is easy to check that µ12 (m,n) and µ21 are measures on F1 ×F2 . Also, by the finite case, µ12 (A1 ×A2 ) = (m,n) µ21 (A1 × A2 ) for all n ≥ 1, m ≥ 1 and hence µ12 (A1 × A2 ) = µ21 (A1 × A2 )

for all A1 × A2 ∈ C.

Next note that {B1m × B2n : m ≥ 1, n ≥ 1} is a partition of Ω1 × Ω2 by F1 × F2 sets and by (1.6) and (1.7), for all m ≥ 1, n ≥ 1, µ12 (B1m × B2n ) = µ1 (B1m )µ2 (B2n ) = µ21 (B1m × B2n ) < ∞. Hence, µ12 and µ21 are σ-finite on F1 × F2 . Since µ12 and µ21 agree on C and C is a π-system generating the product σ-algebra, it follows that µ12 (A) = µ21 (A) for all A ∈ F1 ×F2 and it is the unique measure satisfying µ(A1 × A2 ) = µ1 (A1 )µ2 (A2 ) for all A1 × A2 ∈ C. This completes the proof of the theorem. 2 Remark 5.1.1: In the above theorem, the σ-finiteness condition on the measures µ1 and µ2 cannot be dropped. For example, let Ω1 = Ω2 = [0, 1], F1 = F2 = B([0, 1]), µ1 = the Lebesgue measure and µ2 = the counting measure. Clearly µ2 is not σ-finite since [0, 1] is uncountable. Let A be the diagonal set in the product space [0, 1]×[0, 1], i.e., A = {(ω1 , ω2 ) ∈ Ω1 ×Ω2 : ω1 = ω2 }. Then A ∈ F1 × F2 , A1ω1 = {ω1 } and A2ω2 = {ω2 }. Further, µ  2 (A1ω1 ) = 1 for all ω1 and µ1 (A2ω2 ) = 0 for all ω2 . Thus µ12 (A) = µ (A1ω1 )µ1 (dω1 ) = 1, but µ21 (A) = Ω2 µ1 (A2ω2 )µ2 (dω2 ) = 0, and Ω1 2 hence, µ12 (A) = µ21 (A). Remark 5.1.2: Although this approach allows one to compute the product measure µ1 × µ2 of a set A ∈ F1 × F2 , the measure space (Ω1 × Ω2 , F1 × F2 , µ1 × µ2 ) may not be complete even if both (Ωi , Fi , µi ), i = 1, 2 are complete (Problem 5.2). However, the approach based on the extension theorem yields a product measure space that is complete. See Remark 5.2.2 for further discussion on the topic. Definition 5.1.3: (a) The unique measure µ on F1 × F2 defined in Theorem 5.1.2 (iii) is called the product measure and is denoted by µ1 × µ2 .

152

5. Product Measures, Convolutions, and Transforms

(b) The measure space (Ω1 × Ω2 , F1 × F2 , µ1 × µ2 ) is called the product measure space. Formulas (1.4) and (1.5) give two different ways of evaluating µ1 × µ2 (A) for an A ∈ F1 ×F2 . In the next section, integration of a measurable function f : Ω1 × Ω2 → R w.r.t. the product measure µ1 × µ2 is considered and the above approach is extended to justify evaluation of the integral iteratively.

5.2 Fubini-Tonelli theorems Let f : Ω1 × Ω2 → R be a F1 × F2 , B(R)-measurable function. Relations (1.4) and (1.5) suggest that the integral of f w.r.t. µ1 ×µ2 may be evaluated as iterated integrals, using the formulas     f (ω1 , ω2 )µ1 × µ2 (d(ω1 , ω2 )) = f (ω1 , ω2 )µ1 (dω1 ) µ2 (dω2 ) Ω1 ×Ω2

Ω2

(2.1)

and  Ω1 ×Ω2

Ω1

 f (ω1 , ω2 )µ1 × µ2 (d(ω1 , ω2 )) =



Ω1

 f (ω1 , ω2 )µ2 (dω2 ) µ1 (dω1 ).

Ω2

(2.2)

Here, the left sides of both (2.1) and (2.2) are simply the integral of f on the space Ω = Ω1 × Ω2 w.r.t. the measure µ = µ1 × µ2 . The expressions on the right sides of (2.1) and (2.2) are, however, iterated integrals, where integrals of sections of f are evaluated first and then the resulting sectional integrals are integrated again to get the final expression. Conditions for the validity of (2.1) and (2.2) are provided by the Fubini-Tonelli theorems stated below. Theorem 5.2.1: (Tonelli’s theorem). Let (Ωi , Fi , µi ), i = 1, 2 be σ-finite measure spaces and let f : Ω1 × Ω2 → R+ be a nonnegative F1 × F2 measurable function. Then  ¯ is F1 , B(R)-measurable ¯ g1 (ω1 ) ≡ f (ω1 , ω2 )µ2 (dω2 ) : Ω1 → R Ω2

(2.3) and g2 (ω2 ) ≡

 Ω1

¯ f (ω1 , ω2 )µ1 (dω1 ) : Ω2 → R

¯ F2 , B(R)-measurable.

is

(2.4) Further



 f dµ =

Ω1 ×Ω2

Ω1

 g1 dµ1 =

Ω2

g2 dµ2 ,

(2.5)

5.2 Fubini-Tonelli theorems

153

where µ = µ1 × µ2 . Proof: If f = IA for some A in F1 × F2 , the result follows from Theorem 5.1.2. By the linearity of integrals, the result now holds for all simple nonnegative functions f . For a general nonnegative function f , there exist a sequence {fn }n≥1 of nonnegative simple functions such that fn (ω1 , ω2 ) ↑ f (ω1 , ω2 ) for all (ω1 , ω2 ) ∈ Ω1 ×Ω2 . Write g1n (ω1 ) = Ω1 fn (ω1 , ω2 )µ2 (dω2 ). Then, g1n is F1 -measurable for all n ≥ 1, g1n ’s are nondecreasing, and by the MCT,  f (ω1 , ω2 )µ2 (dω2 ) g1 (ω1 ) ≡ Ω1  = lim fn (ω1 , ω2 )µ2 (dω2 ) n→∞

=

lim g1n (ω1 )

(2.6)

n→∞

¯ for all ω1 ∈ Ω1 . Thus, by Proposition 2.1.5, g1 is F1 , B(R)-measurable.  Since (2.5) holds for simple functions, fn dµ = g1n dµ1 for all n ≥ 1. Hence, by repeated applications of the MCT, it follows that    f dµ = lim fn dµ = lim g1n dµ1 n→∞ n→∞   = ( lim g1n )dµ1 = g1 dµ1 . n→∞

The proofs of (2.4) and the second equality in (2.5) are similar.

2

Theorem 5.2.2: (Fubini’s theorem). Let (Ωi , Fi , µi ), i = 1, 2 be σ-finite measure spaces and let f ∈ L1 (Ω1 × Ω2 , F1 × F2 , µ1 × µ2 ). Then there exist sets Bi ∈ Fi , i = 1, 2 such that (i) µi (Ωi \ Bi ) = 0 for i = 1, 2, (ii) for ω1 ∈ B1 , f (ω1 , ·) ∈ L1 (Ω2 , F2 , µ2 ), (iii) the function   Ω2

g1 (ω1 ) ≡

0

f (ω1 , ω2 )µ2 (dω2 ) for for

is F1 -measurable and  Ω1

ω1 in B1 ω1 in B1c

 g1 dµ1 =

Ω1 ×Ω2

f d(µ1 × µ2 ),

(iv) for ω2 ∈ B2 , f (·, ω2 ) ∈ L1 (Ω1 , F1 , µ1 ),

(2.7)

154

5. Product Measures, Convolutions, and Transforms

(v) the function

  Ω1

g2 (ω2 ) ≡

0

f (ω1 , ω2 )µ1 (dω1 ) for for

is F2 -measurable and  Ω2

ω2 in B2 ω2 in B2c

 g2 dµ2 =

Ω1 ×Ω2

f d(µ1 × µ2 ).

(2.8)

Remark 5.2.1: An informal statement of the above theorem is as follows. If f is integrable   on the product space, then the sectional integrals f (ω , ·)dµ and f (·, ω2 )dµ1 are well defined a.e., and their integrals 1 2 Ω2 Ω1 w.r.t. µ1 and µ2 , respectively, are equal to the integral of f w.r.t. the product measure µ1 × µ2 . Proof: By Tonelli’s theorem   |f |d(µ1 × µ2 ) = 

Ω1 ×Ω2



Ω1

Ω2

 |f (ω1 , ω2 )|µ2 (dω2 ) µ1 (dω1 ).

So Ω1 ×Ω2 |f |d(µ1 × µ2 ) < ∞ implies that µ1 (B1c ) = 0 where B1 = {ω1 :  |f (ω1 , ·)|dµ2 < ∞}. Also, by Tonelli’s theorem   g11 (ω1 ) ≡ f + (ω1 , ·)dµ2 and g12 (ω1 ) ≡ f − (ω1 , ·)dµ2 Ω2

Ω2

are both F1 -measurable and   g11 dµ1 = f + d(µ1 × µ2 ), Ω1

Ω1 ×Ω2

 Ω1

 g12 dµ1 =

Ω1 ×Ω2

f − d(µ1 × µ2 ).

(2.9) Since g1 defined in (iii) can be written as g1 = (g11 − g12 )IB1 , g1 is F1 measurable. Also,    |g1 |dµ1 ≤ g11 dµ1 + g12 dµ1 Ω1 Ω Ω1  1  + = f d(µ1 × µ2 ) + f − d(µ1 × µ2 ) Ω1 ×Ω2

<

Ω1 ×Ω2

∞.

 Further, as Ω1 ×Ω2 |f |dµ1 × µ2 < ∞, by (2.9), g11 and g12 ∈ L1 (Ω1 , F1 , µ1 ). Noting that µ1 (B1c ) = 0, one gets   g1 dµ1 = (g11 − g12 )IB1 dµ1 Ω1 Ω  1  = g11 IB1 dµ1 − g12 IB1 dµ1 Ω Ω1  1  = g11 dµ1 − g12 dµ1 Ω1

Ω1

5.2 Fubini-Tonelli theorems

155

  which, by (2.9), equals Ω1 ×Ω2 f + d(µ1 × µ2 ) − Ω1 ×Ω2 f − d(µ1 × µ2 ) =  f d(µ1 × µ2 ). Thus, (ii) and (iii) of the theorem have been estabΩ1 ×Ω2 lished as well as (i) for i = 1. The proofs of (iv) and (v) and that of (i) for i = 2 are similar. 2 An application of the Fubini-Tonelli theorems gives an integration by parts formula. Let F1 and F2 be two nondecreasing right continuous functions on an interval [a, b]. Let µi be the Lebesgue-Stieltjes measure on B([a, b]) corresponding to F by parts’ for  i , i = 1, 2. The ‘integration mula allows one to write (a,b] F1 (x)dF2 (x) ≡ (a,b] F1 dµ2 in terms of   F (x)dF1 (x) ≡ (a,b] F2 dµ1 . (a,b] 2 Theorem 5.2.3: Let F1 , F2 be two nondecreasing right continuous functions on [a, b] with no common points of discontinuity in (a, b]. Then   F1 (x)dF2 (x) = F1 (b)F2 (b) − F1 (a)F2 (a) − F2 (x)dF1 (x). (2.10) (a,b]

(a,b]


, i = 1, 2 are finite measure spaces. Proof: Note that (a, b], B(a, b], µ i
Consider the product space (a, b] × (a, b], B((a, b] × (a, b]), µ1 × µ2 . Define the sets A = {(x, y) : a < x ≤ y ≤ b} B = {(x, y) : a < y ≤ x ≤ b} and C

= {(x, y) : a < x = y ≤ b}.

For notational simplicity, write Ax and Ay for the x-section and the ysection of A, respectively, and similarly, for the sets B and C. Since F1 and F2 have no common points of discontinuity, by Theorem 5.1.2  µ1 × µ2 (C) = µ2 (Cx )µ1 (dx) (a,b]  = µ2 ({x})µ1 (dx) (a,b]

=

0.

(see Problem 5.3). And by Theorem 5.1.2,  µ1 (Ay )µ2 (dy) µ1 × µ2 (A) = (a,b]  = µ1 ((a, y])µ2 (dy) (a,b]  = [F1 (y) − F1 (a)]dF2 (y) (a,b]

(2.11)

(2.12)

156

5. Product Measures, Convolutions, and Transforms

and similarly,

 µ1 × µ2 (B)

= (a,b]

µ2 (Bx )µ1 (dx)

 =

(a,b]

[F2 (x) − F2 (a)]dF1 (x).

(2.13)

Next note that (µ1 × µ2 )((a, b] × (a, b]) = µ1 ((a, b]) · µ2 ((a, b]) = [F1 (b) − F1 (a)][F2 (b) − F2 (a)]. Hence, by (2.11)–(2.13), [F1 (b) − F1 (a)][F2 (b) − F2 (a)] = (µ1 × µ2 )((a, b] × (a, b]) = µ1 × µ2 (A) + µ1 × µ2 (B) − µ1 × µ2 (C)   = [F1 (y) − F1 (a)]dF2 (y) + [F2 (x) − F2 (a)]dF1 (x), (a,b]

(a,b]

which yields (2.10), thereby completing the proof of Theorem 5.2.3.

2

If F1 and F2 are absolutely continuous with nonnegative densities f1 and f2 w.r.t. the Lebesgue measure on (a, b], then (2.10) yields  b  b F1 (x)f2 (x)dx = F1 (b)F2 (b) − F1 (a)F2 (a) − F2 (x)f1 (x)dx. (2.14) a

a

If f1 and f2 are any two Lebesgue integrable function on (a, b] that are not necessarily nonnegative, one can decompose fi as fi = fi+ − fi− and apply (2.14) to fi+ ’s and fi− ’s separately. Then, by linearity, it follows that the relation (2.14) also holds for the given f1 and f2 . Thus, the standard ‘integration by parts’ formula is a special case of Theorem 5.2.3. Relations (2.10) and (2.14) can be extended to unbounded intervals under suitable conditions on F1 and F2 (Problem 5.5). Remark 5.2.2: The measure space (Ω1 ×Ω2 , F1 ×F2 , µ1 ×µ2 ) constructed using the integrals in (1.4) and (1.5) is not necessarily complete. As mentioned at the beginning of this section, the approach based on the extension theorem (applied to the set function defined in (1.1) on the algebra A of finite disjoint unions of measurable rectangles) does yield a complete measure space (Ω1 ×Ω2 , M, λ) such that F1 ×F2 ⊂ M  and λ(A) = (µ1 ×µ2 )(A) for all A in F1 × F2 . To compute the integral f dλ of an M-measurable function f w.r.t. λ, formula (2.5) in Tonelli’s theorem and formulas (2.7) and (2.8) in Fubini’s theorem continue to be valid with some modifications. In the following, a modified statement of Fubini’s theorem is presented. Similar modification also holds for Tonelli’s theorem (cf. Royden (1988), Chapter 12, Section 4). Theorem 5.2.4: Let (Ωi , Fi , µi ), i = 1, 2 be σ-finite measure spaces and let f ∈ L1 (Ω1 × Ω2 , M, µ1 × µ2 ). Let (Ωi , F¯i , µ ¯i ) be the completion of (Ωi , Fi , µi ), i = 1, 2. Then there exist sets Bi in F¯i , i = 1, 2 such that

5.3 Extensions to products of higher orders

157

(i) µ ¯i (Bic ) = 0, i = 1, 2, (ii) for ω1 ∈ B1 , f (ω1 , ·) is F¯2 -measurable,  (iii) Ω2 |f (ω1 , ·)|d¯ µ2 < ∞ for all ω1 ∈ B1 ,   f (ω1 , ·)d¯ µ2 for ω1 in B, Ω2 (iv) g1 (ω1 ) ≡ 0 for ω1 in B1c is F¯1 -measurable,   (v) Ω1 g1 d¯ µ1 = Ω1 ×Ω2 f dλ. Further, a similar statement holds for i = 2.

5.3 Extensions to products of higher orders Definition 5.3.1: Let (Ωi , Fi ), i = 1, . . . , k be measurable spaces, 2 ≤ k < ∞. Then (a) ×ki=1 Ωi = Ω1 × . . . × Ωk = {(ω1 , . . . , ωi ) : ω :∈ Ωi for i = 1, . . . , k} is called the product set of Ω1 , . . . , Ωk . (b) A set of the form A1 × . . . × Ak with Ai ∈ Fi , 1 ≤ i ≤ k is called a measurable rectangle. The product σ-algebra on ×ki=1 Ωi , denoted by ×ki=1 Fi , is the σ-algebra generated by the collection of all measurable rectangles, i.e., ×ki=1 Fi = σ{A1 × . . . × Ak : Ai ∈ Fi , 1 ≤ i ≤ k}. (c) (×ki=1 Ωi , ×ki=1 Fi ) is called the product space or the product measurable space. When (Ωi , Fi ) = (Ω, F) for all 1 ≤ i ≤ k, the product space will be denoted by (Ωk , F k ). To define the product measure, one starts with a natural set function on C, the class of all measurable rectangles, extends it to the algebra A of finite disjoint unions of measurable rectangles by linearity, and then uses the extension theorem (Theorem 1.3.3) to obtain a measure on the product space. The details of this construction are now described below. Define a set function µ on C by µ(A) =

k +

µ(Ai )

(3.1)

i=1

where A = A1 × A2 × ×Ak with Ai ∈ Fi , i = 1, 2, . . . , k.  Next extend it m to the algebra A by linearity. If B ∈ A is of the form B = j=1 Bj where

158

5. Product Measures, Convolutions, and Transforms

{Bj : 1, . . . , m} ⊂ C are disjoint, then define µ(B) by µ(B) =

m

µ(Bj ).

(3.2)

j=1

If the set B ∈ A admits two different representations as finite unions of measurable rectangles then it is not difficult to verify that the value of µ(B) remains unchanged. Next it is shown that µ is a measure on A. Proposition 5.3.1:. Let µ be as defined by (3.1) and (3.2) above on the algebra A. Then, it is countably additive on A.  Proof: Let {Bn }∞ n=1 ⊂ A be disjoint such that B = n≥1 Bn also belongs to A. It is enough to show that µ(B) =

∞

µ(Bn ).

(3.3)

n=1

 Let B and {Bn }∞ n=1 admit the representations B = i=1 Ai and Bj =  j r=1 Ajr where  and j are (finite) integers, Ai , Ajr ∈ C and each of the j , j ≥ 1 is disjoint. Then each Ai can be collections {Ai } i=1 and {Ajr }r=1 written as n  



Bn = (Ai ∩ Anr ). Ai = Ai ∩ n≥1 r=1

n≥1

Suppose it is shown that for each i ≥ 1, µ(Ai ) =

j ∞

µ(Ai ∩ Ajr ).

j=1 r=1

Then, by (3.2), µ(B)

=



µ(Ai )

i=1

=

j ∞

µ(Ai ∩ Ajr )

i=1 j=1 r=1

=

j ∞

µ(Ai ∩ Ajr )

j=1 r=1 i=1

=

j ∞ j=1 r=1

µ(B ∩ Ajr ),

(3.4)

5.3 Extensions to products of higher orders

159

where the last equality follows from the representation of B ∩ Ajr as  i=1 (Ai ∩ Ajr ) and the fact that Ai ∩ Ajr ∈ C for all i, j, r. Since Ajr ⊂ Bj ⊂ B, the above yields j ∞

µ(Ajr ) =

j=1 r=1

∞

µ(Bj )

(by (3.2))

j=1

which establishes (3.3). Thus, itremains to show (3.4). This is implied by the following: Let C = n≥1 Cn where {Cn }∞ n=1 is a collection of disjoint measurable rectangles and C is also a measurable rectangle. Then µ(C) =

∞

µ(Ci ).

(3.5)

i=1

Let C =A1 × A2 × ×Ak and Ci = Ai1 × Ai2 × ×Aik , i = 1, 2, . . .. Since C = n≥1 Cn and {Cn }∞ n=1 are disjoint, this implies IC (ω1 , . . . , ωk ) = ∞ i=1 ICi (ω1 , . . . , ωk ), for all (ω1 , . . . , ωk ) ∈ Ω1 × . . . × Ωk . That is, k +

IAj (ωj ) =

j=1

∞ + k

IAij (ωj )

(3.6)

i=1 j=1

for all (ω1 , . . . , ωk ) ∈ Ω1 × . . . × Ωk . Integration of both sides of (3.6) w.r.t. µ1 over Ω1 yields

+  k ∞ k + IAj (ωj ) = µ1 (Ai1 ) IAij (ωj ), µ1 (A1 ) j=2

i=1

j=2

and by iteration k +

µj (Aj ) =

j=1

∞ + k

µj (Aij ).

i=1 j=1

2

Hence, (3.5) follows.

By the extension theorem (Theorem 1.3.3), there exists a σ-algebra M and a measure µ ¯ such that ¯) is complete, (i) (×ki=1 Ωi , M, µ (ii) ×ki=1 Fi ⊂ M, (iii) µ ¯(A) = µ(A)

and for all A ∈ A.

Thus, the above procedure yields an extension µ ¯ of µ on A, and this extension is unique when all the µi ’s are σ-finite. Further, under the hypothesis that µ1 , . . . , µk are σ-finite, the analogs of formulas (1.4) and (1.5) for

160

5. Product Measures, Convolutions, and Transforms

computing the product measure of a set via the iterated integrals are valid. More generally, the Tonelli-Fubini theorems extend to the k-fold product spaces in an obvious manner. For example, if (Ωi , Fi , µi ) are σ-finite measure spaces for i = 1, . . . , k and f is a nonnegative measurable function on ¯), then (×ki=1 Ωi , ×ki=1 Fi , µ     f d¯ µ= ··· f (ω1 , ω2 , ωk )µi1 (dωi1 ) . . . µik (dωik ) Ωi1

Ωi2

Ωik

for any permutation (i1 , i2 , . . . , ik ) of (1, 2, . . . , k). ¯) is called the comDefinition 5.3.2: The measure space (×ki=1 Ωi , M, µ plete product measure space and µ ¯ the complete product measure. Remark 5.3.1: If (Ωi , Fi , µi ) ≡ (R, L, m) where L is the Lebesgue σalgebra and m is the Lebesgue measure, then the above extension coincides with the (Rk , Lk , mk ), where Lk is the Lebesgue σ-algebra in Rk and mk is the k dimensional Lebesgue measure.

5.4 Convolutions 5.4.1

Convolution of measures on (R, B(R))


 In this section, convolution of measures on R, B(R) is discussed. From this, one can easily obtain convolution of sequences, convolution of functions in L1 (R) and convolution of functions with measures.
 Proposition 5.4.1: Let µ and λ be two σ-finite measures on R, B(R) . For any Borel set A in B(R), let   (4.1) (µ ∗ λ)(A) ≡ IA (x + y)µ(dx)λ(dy).
 Then (µ ∗ λ)(·) is a measure on R, B(R) . Proof: Let h(x, y) ≡ x + y for (x, y) ∈ R × R. Then h : R × R → R is continuous and hence is B(R) × B(R), B(R)-measurable. Consider the measure (µ × λ)h−1 (·) on R, B(R) induced by the map h and the measure µ × λ on R × R, B(R) × B(R). Clearly (µ ∗ λ)(·) = (µ × λ)h−1 (·) 2
 Definition 5.4.1: For any two σ-finite measures µ and λ on R, B(R) , the measure (µ ∗ λ)(A) defined in (4.1) is called the convolution of µ and λ. and hence the proposition is proved.

5.4 Convolutions

161

The following proposition is easy to verify.


 Proposition 5.4.2: Let µ1 , µ2 , µ3 be σ-finite measures on R, B(R) . Then (i) (Commutativity) µ1 ∗ µ2 = µ2 ∗ µ1 , (ii) (Associativity) µ1 ∗ (µ2 ∗ µ3 ) = (µ1 ∗ µ2 ) ∗ µ3 , (iii) (Distributive) µ1 ∗ (µ2 + µ3 ) = µ1 ∗ µ2 + µ1 ∗ µ3 , (iv) (Identity element) µ1 ∗ δ{0} = µ1 where δ{0} (·) is the delta measure at 0, i.e.,  1 if 0 ∈ A, δ{0} (A) = 0 if 0 ∈ A for all A ∈ B(R). k Remark  (Extension to R ). Definition 5.4.1 extends to measures
k 5.4.1: k on R , B(R ) for any integer k ≥ 1 as well as to any space that is a commutative group under addition such as the sequence space R∞ , the function spaces C[0, 1] and C[0, ∞). These are relevant in the study of stochastic processes.

Remark 5.4.2: (Sums of independent random variables). It will be seen in Chapter 7 that if X and Y are two independent random variables on a probability space (Ω, F, P ), then PX ∗ PY = PX+Y where for any random variable Z on
(Ω, F,P ), PZ is the distribution of Z, i.e., the probability measure on R, B(R) induced by P and Z, i.e., PZ (·) = P Z −1 (·) on B(R). Remark 5.4.3: (Extension to
 signed measures). Let µ and λ be two finite signed measures on R, B(R) as defined in Section 4.2. Let µ = µ+ − µ− , λ = λ+ − λ− be the Jordan decomposition of µ and λ, respectively. Then the product measure µ × λ can be defined as the signed measure # " # " µ × λ ≡ (µ+ × λ+ ) + (µ− × λ− ) − (µ+ × λ− ) + (µ− + λ+ ) ≡ γ1 − γ2 , say. (4.2) This is well defined since the measures (µ+ × λ+ ) + (µ− × λ− ) and (µ+ × λ− )+(µ− ×λ+ ) are both finite measures. Now the definition of convolution of measures in (4.1) carries over to signed measures using the definition of integration w.r.t. signed measures discussed in Section 4.2. Definition 5.4.2: Let µ and ν be (finite) signed measures on a measurable
 space R, B(R) . The convolution of µ and ν, denoted by µ ∗ ν is the signed measure defined by   (µ ∗ ν)(A) = IA (x + y)d(µ × ν)

162

5. Product Measures, Convolutions, and Transforms

 

  IA (x + y)dγ1 −

=

IA (x + y)dγ2

(4.3)

where γ1 and γ2 are as in (4.2).

5.4.2

Convolution of sequences

Definition 5.4.3: Let a ˜ ≡ {an }n≥0 and ˜b ≡ {bn }n≥0 be two sequences of real numbers. Then the convolution of a ˜ and ˜b denoted by a ˜ ∗ ˜b is the sequence n aj bn−j , n ≥ 0. (4.4) (˜ a ∗ ˜b)(n) = ∞

j=0

∞

If j=0 |aj | < ∞ and j=0 |bj | < ∞, then a ˜ ∗ ˜b corresponds to the convo˜(i) = ai , ˜b(i) = bi , lution of the signed measures a ˜ and ˜b on Z+ , defined by a i ∈ Z+ . Example 5.4.1: n a ∗ ˜b)(n) = j=0 aj . (a) Let bn ≡ 1 for n ≥ 0. Then for any {an }n≥0 , (˜ (b) Fix 0 < p < 1, k1 , k2 positive integers. For n ∈ Z+ , let

 k1 n an ≡ p (1 − p)k1 −n I[0,k1 ] (n) n

 k2 n bn ≡ p (1 − p)k2 −n I[0,k2 ] (n). n  n
2 p (1 − p)k2 −n I[0,k1 +k2 ] (n). Then (˜ a ∗ ˜b)(n) = k1 +k n (c) Fix 0 < λ1 , λ2 < ∞. Let an

≡

bn

≡

λn1 , n = 0, 1, 2, . . . n! λn e−λ2 2 , n = 0, 1, 2, . . . . n! e−λ1

Then (˜ a ∗ ˜b)(n) = e−(λ1 +λ2 )

(λ1 +λ2 )n , n!

n = 0, 1, 2, . . . .

The verification of these claims is left as an exercise (Problem 5.20). A useful technique to determine a ˜ ∗ ˜b is the use of generating functions. This will be discussed in Section 5.5.

5.4.3

Convolution of functions in L1 (R)

Let L1 (R) ≡ L1 (R, B(R),m) where m(·) is the Lebesgue  measure. In the following, for f ∈ L1 (R), f dm will also be written as f (x)dx.

5.4 Convolutions

163

Proposition 5.4.3: Let f , g ∈ L1 (R). Then for almost all x (w.r.t. m),  |f (x − u)| |g(u)|du < ∞. (4.5) Proof: Let k(x, u) = f (x − u)g(u). Since π(x, u) ≡ x − u is a continuous map from R × R → R, it is B(R) × B(R), B(R)-measurable. Also, since f and g are Borel measurable, it follows that k : R × R → R is B(R) × Also note that by the translation invariance of m, B(R), B(R)-measurable.  |f (x − u)|dx = |f (x)|dx for any Borel measurable f : R → R. Hence, by Tonelli’s theorem,      |k(x, u)|dxdu = |k(x, u)|dx du    = |g(u)| |f (x)|dx du = g1 f 1 < ∞.

(4.6)

Also by Tonelli’s theorem      |k(x, u)|dxdu = |k(x, u)|du dx. By (4.6), this yields  |k(x, u)|du < ∞

a.e. (m)

which is the same as (4.5).

2

Definition 5.4.4: Let f , g ∈ L1 (R). Then the convolution of f and g, denoted by f ∗ g is the function defined a.e. (m) by  (f ∗ g)(x) ≡ f (x − u)g(u)du. (4.7) Note that by (4.5) this is well defined. Proposition 5.4.4: Let f , g ∈ L1 (R). Then (i) f ∗ g = g ∗ f (ii) f ∗ g ∈ L1 (R) and f ∗ g1 ≤ f 1 g1      gdm . f dm (iii) f ∗ g dm = Proof: For (i), use the change of variable u → x − u and the translation invariance of m. For (ii) use (4.6). For (iii), use Fubini’s theorem. 2

164

5. Product Measures, Convolutions, and Transforms

It is easy to see that if µ and λ denote the signed measures with RadonNikodym derivatives f and g, respectively, w.r.t. m, then µ ∗ λ is the signed measure with Radon-Nikodym derivative f ∗ g w.r.t. m. Example 5.4.2: Here are two examples from probability theory. (a) (Convolutions of Uniform [0, 1] densities). Let f = g = I[0,1] . Then  x 0≤x≤1 (f ∗ g)(x) = 2 − x 1 ≤ x ≤ 2. (b) (Convolutions of N (0, 1) densities). Let f (x) ≡ g(x) ≡ −∞ < x < ∞. Then (f ∗ g)(x) =

5.4.4

√1 √1 2π 2

2 − x4

e

√1 2π

e−

x2 2

,

, −∞ < x < ∞.

Convolution of functions and measures

 Let f ∈ L1 (R) and λ be a signed measure. Let µ(A) ≡ A f dm, A ∈ B(R). Then it can be shown that µ∗λ is dominated by m and its Radon-Nikodym derivative is given by  f ∗ λ(x) ≡ f (x − u)λ(du), x ∈ R. (4.8) Note that f ∗λ in (4.8) is well
defined  for any nonnegative Borel measurable f and any measure λ on R, B(R) .

5.5 Generating functions and Laplace transforms Definition 5.5.1: Let {an }n≥0 be a sequence in R and let ρ = −1
lim |an |1/n . The power series n→∞

A(s) ≡

∞

an sn ,

(5.1)

n=0

defined for all s in (−ρ, ρ), is called the generating function of {an }n≥0 . Note that the power series in (5.1) converges absolutely for |s| < ρ and ρ is called the radius of convergence of A(s) (cf. Appendix A.2). The usefulness of generating functions is given by the following: Proposition 5.5.1: Let {an }n≥0 and {bn }n≥0 be two sequences in R. Let {cn }n≥0 be the convolution of {an } and {bn }. That is, cn =

n j=0

aj bn−j , n ≥ 0.

5.5 Generating functions and Laplace transforms

165

Then C(s) = A(s)B(s)

(5.2)

for all s in (−ρ, ρ), where A(·), B(·), and C(·) are, respectively, the generating function of the sequences {an }n≥0 , {bn }n≥0 and {cn }n≥0 and −1 −1

and ρb = lim |bn |1/n . ρ = min(ρa , ρb ) with ρa = lim |an |1/n n→∞

n→∞

Proof: By Tonelli’s theorem applied to the counting measure on the product space (Z+ × Z+ ), 

∞ n n |s| |aj | |bn−j | =

n=0 ∞

j=0

|aj | |s|j

∞

j=0

=

 |s|n−j |bn−j |

n=j

A(|s|)B(|s|) < ∞ if |s| < ρ.

Now by Fubini’s theorem, (5.2) follows.

2

It is easy to verify the claims in Example 5.4.1 by using the above proposition and the uniqueness of the power series coefficients (Problem 5.20). Proposition 5.5.2: (Renewal sequences). Let {an }n≥0 , {bn }n≥0 , {pn }n≥0 be sequences in R such that an = bn +

n

an−j pj , n ≥ 0

(5.3)

j=0

and p0 = 0. Then A(s) =

B(s) 1 − P (s)

for all s such that |s| < ρ and P (s) = 1, where ρ = min(ρb , ρp ), ρb = −1 −1

lim |bn |1/n , ρp = lim |pn |1/n . n→∞

n→∞

For applications of this to renewal theory, see Chapter 8. Definition 5.5.2: (Laplace transform). Let f : [0, ∞) → R be Borel measurable. The function  esx f (x)dx, (5.4) (Lf )(s) ≡ [0,∞)

defined for all s in R such that  [0,∞)

esx |f (x)|dx < ∞,

is called the Laplace transform of f .

(5.5)

166

5. Product Measures, Convolutions, and Transforms

It is easily seen that if (5.5) holds for some s = s0 , then it does for all s < s0 . The analog of Proposition 5.5.1 is the following. Proposition 5.5.3: Let f , g ∈ L1 (R). Then L(f ∗ g)(s) = Lf (s)Lg(s)

(5.6)

for all s such that (5.5) holds for both f and g. transform). Let µ be a measure on
Definition  5.5.3: (Laplace-Stieltjes
 R, B(R) such that µ (−∞, 0) = 0. The function  µ∗ (s) ≡ Lµ(s) ≡ esx µ(dx) [0,∞)

is called the Laplace-Stieltjes transform of µ. Clearly, µ∗ (s) is well defined for all s in R. However, µ∗ (s0 ) = ∞ implies µ∗ (s) = ∞ for s ≥ s0 .
 Proposition
 5.5.4:
Let µ and λ be measures on R, B(R) such that µ (−∞, 0) = 0 = λ (−∞, 0) . Then L(µ ∗ λ)(s) = Lµ(s)Lλ(s) for all s in R. For an inversion formula to obtain a probability measure µ from Lµ(·), see Feller (1966), Section 13.4.

5.6 Fourier series
 In this section, Lp [0, 2π] stands for Lp [0, 2π], B([0, 2π]), m where m(·) is Lebesgue measure and 0 < p < ∞. Definition 5.6.1: For f ∈ L1 [0, 2π], n ≥ 0, let  2π 1 an ≡ f (x) cos nx dx 2π 0  2π 1 bn ≡ f (x) sin nx dx, 2π 0 sn (f, x) ≡ a0 +

n

(aj cos jx + bj sin jx).

(6.1)

(6.2)

j=1

Then {an , bn , n = 0, 1, 2, . . .} are called the Fourier coefficients of f and the sequence {sn (f, x) : n = 0, 1, 2, . . .} is called the partial sum sequence of the Fourier series expansion of f . Since C[0, 2π] ⊂ L2 [0, 2π] ⊂ L1 [0, 2π], the Fourier coefficients and the partial sum sequence of the Fourier series are well defined for f in C[0, 2π]

5.6 Fourier series

167

and f in L2 [0, 2π]. J. Fourier introduced these series in the early 19th century to approximate certain functions exhibiting a periodic behavior that arose in the study of physics and mechanics and in particular in the theory of heat conduction (see K¨ orner (1989) and Bhatia (2003)). A natural question is: In what sense does sn (f, x) approximate f (x)? It turns out that one can prove the strongest results for f in C[0, 2π], less stronger results for f in L2 [0, 2π], and finally, for f in L1 [0, 2π]. In the early 19th century it was believed that for any f in C[0, 2π], sn (f, x) converged to f (x) for all x in [0, 2π]. But in 1876, D. Raymond constructed a continuous function f for which limn→∞ |sn (f ; x0 )| = ∞ for some x0 . However, Fejer showed in 1903 that for f in C[0, 2π], sn (f, ·) does converge to f uniformly in the Cesaro sense. Theorem 5.6.1: (Fejer). Let f ∈ C[0, 2π] and sn (f, ·), n ≥ 0 be as in Definition 3.4.1. Let Dn (f, ·) =

n−1 1 sj (f, ·), n ≥ 1. n j=0

(6.3)

Then Dn (f, ·) → f (·) uniformly on [0, 2π]

as

n → ∞.

(6.4)

For the proof of this result, the following result of some independent interest is needed. Lemma 5.6.2: (Fejer ). Let for m ≥ 1 m−1 2 (m − j) cos jx, x ∈ R. m j=1

Km (x) ≡ 1 +

(6.5)

Then (i)

⎧ ⎨ Km (x) =

⎩

1 m

sin mx 2 sin x 2

2

m

if x = 0

(6.6)

if x = 0 .

(ii) For δ > 0, sup{Km (x) : δ ≤ |x| ≤ 2π − δ} → 0 (iii) 1 2π

 0

as

m → ∞.

(6.7)

2π

Km (x)dx = 1.

(6.8)

168

5. Product Measures, Convolutions, and Transforms

Proof: Clearly, (iii) follows from (6.5) on integration. Also, (ii) follows from (i) since for δ ≤ x ≤ 2π − δ, Km (x) ≤

1 1 →0 m (sin 2δ )2

as

m → ∞.

To establish (i) note that using Euler’s formula (cf. Section A.3, Appendix) eιx = cos x + ι sin x, Km (x) can be written as Km (x)

=

(m−1)



1 m

(m − |j|)eιjx

j=−(m−1)

=

2(m−1) 1 (m − |j − (m − 1)|)eι(j−(m−1))x m j=0

=

1 m

m−1

ι(k− m−1 2 )x

e

2

.

k=0

For x = 0, Km (x)

1  − ι(m−1)x 1 − eιmx 2 2 , e m 1 − eιx

− ιmx ιmx 2 1 e 2 −e 2 , ιx ιx m e− 2 − e 2 2 1  sin mx 2 . x m sin 2

= = =

For x = 0, (6.5) yields Km (0)

=

1+

m−1 2 (m − j) m j=1

=

1+

2 (m − 1)m = m. m 2 2

Proof of Theorem 5.6.1: Let f ∈ C[0, 2π]. Let {an , bn }n≥0 , sn (f, ·), Dm (f, ·) be as in (6.1), (6.2), and (6.3). Then from the definition of an , bn , n ≥ 0, it follows that Dm (f, ·)

= =

1 2π 1 2π



2π

Km (x − u)f (u)du

0



0

2π

f (x − u)Km (u)du

(6.9)

5.6 Fourier series

169

where Km (·) is defined in (6.5) and f (·) is extended to all of R periodically with period 2π. Since f ∈ C[0, 2π], given  > 0, there exists a δ > 0 such that x, y ∈ [0, 2π], |x − y| < δ ⇒ |f (x) − f (y)| < . Also from (6.7), for this δ > 0, there exist an m0 such that m ≥ m0 ⇒ sup{Km (x) : δ ≤ x ≤ 2π −δ} < . Now (6.9) yields, for x ∈ [0, 2π], m ≥ m0  2π 1 |Dm (f, x) − f (x)| ≤ |f (x − u) − f (x)|Km (u)du 2π 0  1 = |f (x − u) − f (x)|Km (u)du 2π (δ≤u≤2π−δ)c

+

1 2π



|f (x − u) − f (x)|Km (u)du (δ≤u≤2π−δ)

≤

1 ( + 2f 2π) 2π

where f  = sup{|f (x)| : x ∈ [0, 2π]}. Thus, m ≥ m0 ⇒ sup{|Dm (f, x) − ) . Since  > 0 is arbitrary, the proof of f (x)| : x ∈ [0, 2π]} ≤  (1+4πf 2π Theorem 5.6.1 is complete. 2 An immediate consequence of Theorem 5.6.1 is the completeness of the trigonometric functions. Theorem 5.6.3: The collection T0 ≡ {cos nx : n = 0, 1, 2, . . .} ∪ {sin nx : n = 1, 2, . . .} is a complete orthogonal system for L2 [0, 2π]. Proof: By Theorem 5.6.1 for each f ∈ C[0, 2π] and  > 0, there exists a finite linear combination Dm (f, ·) of {cos nx : n = 0, 1, 2, . . .} and {sin nx : n = 1, 2, . . .} such that  2π |f − Dm (f, ·)|2 dx 0

≤ 2π(sup{|f (x) − Dm (f, x)| : x ∈ [0, 2π]})2 < 2 . Also, from Theorem 2.3.14 it is known that given any g ∈ L2 [0, 2π], and  2π any  > 0, there is a f ∈ C[0, 2π] such that 0 |f − g|2 dx < 2 . Thus, for any g ∈ L2 [0, 2π],  > 0, there is a f ∈ C[0, 2π] and an m ≥ 1 such that g − Dm (f, ·)2 < 2. That is, the set T of all finite linear combinations of the functions in the class T0 is dense in L2 [0, 2π]. Further, it is easy to verify that h1 , h2 ∈ T0 , h1 = h2 implies  2π

0

h1 (x)h2 (x)dx = 0,

170

5. Product Measures, Convolutions, and Transforms

i.e., T0 is an orthogonal family. Since T0 is orthogonal and T is dense in 2 L2 [0, 2π], T0 is complete. Definition 5.6.2: A function in T is called a trigonometric polynomial. Thus, the above theorem says that trigonometric polynomials are dense in L2 [0, 2π]. Completeness of T in L2 [0, 2π] and the results of Section 3.3 lead to Theorem 5.6.4: Let f ∈ L2 [0, 2π]. Let {(an , bn ), n = 0, 1, 2, . . .} and {sn (f, ·)}n≥0 be the associated Fourier coefficient sequences and partial sum sequence of the Fourier series for f as in Definition 5.6.1. Then (i) sn (f, ·) → f in L2 [0, 2π], (ii)

 2π 2 1 a2n + ˘b2n ) = 2π |f | dx where n=0 (˘ 0  2π 1 2 2 cn = 2π 0 (cos nx) dx = 12 , and for  2π 1 1 2 2π 0 (sin nx) dx = 2 .

∞

for n ≥ 0, a ˘n = an /cn , with ˘ n ≥ 1, bn = dbnn with d2n =

 2π 1 f g dx = a0 α0 + (iii) Further, if f , g ∈ L2 [0, 2π], then 2π 0  ∞ bn βn an αn + d2 , where {(an , bn ) : n = 0, 1, 2, . . .} and n=1 c2n n {(αn , βn ) : n = 0, 1, 2, . . .} are, respectively, the Fourier coefficients of f and g. Clearly (ii) above is a restatement of Bessel’s equality. Assertion (iii) is known as the Parseval identity. As for convergence pointwise or almost everywhere, A. N. Kolmogorov showed in 1926 (see K¨orner (1989)) that there exists an f ∈ L1 [0, 2π] such that limn→∞ |sn (f, x)| = ∞ everywhere on [0, 2π]. This led to the belief that for f ∈ C[0, 2π], the mean square convergence of (i) in Theorem 5.6.4 cannot be improved upon. But L. Carleson showed in 1964 (see K¨orner (1989)) that for f in L2 [0, 2π], sn (f, ·) → f (·) almost everywhere. Finally, turning to L1 [0, 2π], one has the following: Theorem 5.6.5: Let f ∈ L1 [0, 2π]. Let {(an , bn ) : n ≥ 0} be as in (6.1) ∞ and satisfy n=0 (|an | + |bn |) < ∞. Let sn (f, ·) be as in (6.2). Then sn (f, ·) converges uniformly on [0, 2π] and the limit coincides with f almost everywhere. ∞ Proof: Note that n=0 (|an | + |bn |) < ∞ implies that the sequence {sn (f, ·)}n≥0 is a Cauchy sequence in the Banach space C[0, 2π] with the sup-norm. Thus, there exists a g in C[0, 2π] such that sn (f, ·) → g uniformly on [0, 2π]. It is easy to check that this implies that g and f have the same Fourier coefficients. Set h = g − f . Then h ∈ L1 [0, 2π] and the Fourier coefficients of h are all zero. This implies that h is orthogonal to the members of the class T , which in turn yields that h is orthogonal to all

5.6 Fourier series

171

continuous functions in C[0, 2π], i.e.,  2π h(x)k(x)dx = 0 0

for all k ∈ C[0, 2π]. Since h ∈ L1 [0, 2π] and for any interval A ⊂ [0, 2π], there exists a sequence {kn }n≥1 of uniformly bounded continuous functions, such that kn → IA a.e. (m), by the DCT,   h(x)IA (x)dx = lim h(x)kn (x)dx = 0. n→∞

This in turn implies that h = 0 a.e., i.e., g = f a.e.

2

Remark 5.6.1: If f ∈ L2 [0, 2π], the Fourier coefficients {an , bn } are square summable and hence go to zero as n → ∞. What if f ∈ L1 [0, 2π]? If f ∈ L1 [0, 2π], one can assert the following: Theorem 5.6.6: (Riemann-Lebesgue lemma). Let f ∈ L1 [0, 2π]. Then  2π  2π lim f (x) cos nx dx = 0 = lim f (x) sin nx dx. n→∞

n→∞

0

0

Proof: The lemma holds if f = IA for any interval A ⊂ [0, 2π] and since step functions (i.e., linear combinations of indicator functions of intervals) 2 are dense in L1 [0, 2π], the lemma is proved. It can be shown that the mapping f → {(an , bn )}n≥0 from L1 [0, 2π] to bivariate sequences that go to zero as n → ∞ is one-to-one but not onto (Rudin (1987), Chapter 5). Remark 5.6.2: (The complex case). Let T ≡ {z : z = eιθ , 0 ≤ θ ≤ 2π} be the unit circle in the complex plane C. Every function g : T → C can be identified with a function f on R by f (t) = g(eιt ). Clearly, f (·) is periodic on R with period 2π. In the rest of this section, for 0 < p < ∞, Lp (T ) will stand for the  collection of all Borel measurable functions f : [0, 2π] to C such that [0,2π] |f |p dm < ∞ where m(·) is the Lebesgue measure. A trigonometric polynomial is a function of form f (·) ≡

k n=−k

αn eιnx ≡ a0 +

k

(an cos nx + bn sin nx),

n=1

k < ∞, where {αn }, {an }n≥0 , and {bn }n≥0 are sequences of complex numbers. The completeness of the trigonometric polynomials proved in Theorem 5.6.3 implies that the family {eιnx : n = 0, ±1, ±2, . . .} is a complete orthonormal basis for L2 (T ), which is a complex Hilbert space.

172

5. Product Measures, Convolutions, and Transforms

Thus Theorem 5.6.4 carries over to this case. Theorem 5.6.7: (i) Let f ∈ L2 (T ). Then, k

αn eιnx → f

in

L2 (T )

n=−k

where αn ≡ Further,

1 2π

∞



2π

0

f (x)e−ιns dx, n ∈ Z. 

1 2π

|αn |2 =

n=−∞

2π

0

|f |2 dm.

(ii) For any sequence {αn }n∈Z of complex numbers such that   k ∞ 2 ιnx conn=−∞ |αn | < ∞, the sequence fk (x) ≡ n=−k αn e k≥1 2 verges in L (T ) to a unique f such that αn =

1 2π



2π

f (x)e−ιnx dx.

0

(iii) For any f , g ∈ L2 (T ), ∞

αn β¯n =

n=−∞

1 2π



2π

f (x)g(x)dx 0

where

Further,

1 fˆ(n) = 2π

αn

=

βn

= gˆ(n) =

1 2π

∞



2π

0  2π 0

f (x)e−ιnx dx, g(x)e−ιnx dx, n ∈ Z.

|αn βn | < ∞.

n=−∞

(iv) L2 (T ) is isomorphic to 2 (Z), the Hilbert space of all square summable sequences of complex numbers on Z. Similarly, Theorem 5.6.5 carries over to the complex case.

5.7 Fourier transforms on R

173

Theorem 5.6.8: Let f ∈ L1 (T ). Suppose ∞

|fˆ(n)| < ∞

n=−∞

where 1 fˆ(n) = 2π



2π

0

Then sn (f, x) ≡

f (x)e−ιnx dx, n ∈ Z. n

fˆ(j)e−ιjx

j=−n

converges uniformly on [0, 2π] and the limit coincides with f a.e. and hence f is continuous a.e.

5.7 Fourier transforms on R In this section and in Section 5.8, let Lp (R) stand for  |f |p dm < ∞} Lp (R) ≡ {f : f : R → C, Borel measurable, R

where m(·)is Lebesgue measure. Also, for f ∈ L1 (R), written as f (x)dx. Let C0 ≡ {f : f : R → C, continuous and

 R

(7.1)

f dm will often be

lim f (x) = 0}.

|x|→∞

Definition 5.7.1: For f ∈ L1 (R), t ∈ R,  fˆ(t) ≡ f (x)e−ιtx dx

(7.2)

(7.3)

is called the Fourier transform of f . Proposition 5.7.1: Let f ∈ L1 (R) and fˆ(·) be as in (7.3). Then (i) fˆ(·) ∈ C0 . (ii) If fa (x) ≡ f (x − a), a ∈ R, then fˆa (t) = eιta fˆ(t), t ∈ R. Proof: (i) For any t ∈ R, tn → t ⇒ eιtn x f (x) → eιtx f (x) for all x ∈ R and since |eιtn x f (x)| ≤ |f (x)| for all n and x, by the DCT fˆ(tn ) → fˆ(t). To show that fˆ(t) → 0 as |t| → ∞, the same proof as that of Theorem 5.6.6 works. Thus, it holds if f = I[a,b] , for a, b ∈ R, a < b and since the step functions are dense in L1 (R), it holds for all f ∈ L1 (R).

174

5. Product Measures, Convolutions, and Transforms

(ii) This is a consequence of the translation invariance of m(·), i.e., m(A+ a) = m(A) for all A ∈ B(R), a ∈ R. 2 The continuity of fˆ(·) can be strengthened to differentiability if, in addition to f ∈ L1 (R), xf (x) ∈ L1 (R). More generally, if f ∈ L1 (R) and xk f (x) ∈ L1 (R) for some k ≥ 1, then fˆ(·) is differentiable k-times with all derivatives fˆ(r) (t) → 0 as |t| → ∞ for r ≤ k (Problem 5.22). Proposition 5.7.2: Let f , g ∈ L1 (R) and f ∗ g be their convolution as defined in (4.7). Then f
∗ g = fˆgˆ. (7.4) Proof: f
∗ g(t)



−ιtx



e

= R

R

  = R

R

 =

R

 f (x − u)g(u)du dx

 e−ιt(x−u) f (x − u)e−ιtu g(u)du dx

(ft ∗ gt )(x)dx

(7.5)

where ft (x) = e−ιtx f (x), gt (x) = e−ιtx g(x). Thus, by Proposition 5.4.4     ft (x)dx gt (x)dx f
∗ g(t) = R

=

R

fˆ(t)ˆ g (t).

2

The process of recovering f from fˆ (i.e., that of finding an inversion formula) can be developed along the lines of Fejer’s theorem (Theorem 5.6.1). Theorem 5.7.3: (Fejer’s theorem). Let f ∈ L1 (R), fˆ(·) be as in (7.3) and  T 1 ST (f, x) ≡ (7.6) fˆ(t)eιtx dt, T ≥ 0, 2π −T  1 R ST (f, x)dT, R ≥ 0. (7.7) DR (f, x) ≡ R 0 (i) If f is continuous at x0 and f is bounded on R, then lim DR (f, x0 ) = f (x0 ).

R→∞

(7.8)

(ii) If f is uniformly continuous and bounded on R, then lim DR (f, ·) = f (·) uniformly on R.

R→∞

(7.9)

5.7 Fourier transforms on R

(iii) As R → ∞,

DR (f, ·) → f (·)

L1 (R).

in

175

(7.10)

(iv) If f ∈ Lp (R), 1 ≤ p < ∞, then as R → ∞, DR (f, ·) → f (·)

Lp (R).

in

(7.11)

Corollary 5.7.4: (Uniqueness theorem). If f and g ∈ L1 (R) and fˆ(·) = gˆ(·), then f = g a.e. (m). ˆ ≡ 0. Thus, ST (h, ·) ≡ 0 Proof: Let h = f − g. Then h ∈ L1 (R) and h(·) and DR (h, ·) ≡ 0 where ST (h, ·) and DR (h, ·) are as in (7.6) and (7.7). Hence by Theorem 5.7.3 (iii), h = 0 a.e. (m), i.e., f = g a.e. (m). 2 Corollary 5.7.5: (Inversion formula). Let f ∈ L1 (R) and fˆ ∈ L1 (R). Then  1 (7.12) fˆ(t)eιtx dx a.e. (m). f (x) = 2π R Proof: Since fˆ ∈ L1 (R), by the DCT,  1 fˆ(t)eιtx dt ST (f, x) → 2π R

as

T →∞

for all x in R and hence DR (f, x) has the same limit as R → ∞. Now (7.12) follows from (7.10). 2 The following results, i.e., Lemma 5.7.6 and Lemma 5.7.7, are needed for the proof of Theorem 5.7.3. The first one is an analog of Lemma 5.6.2. Lemma 5.7.6: (Fejer ). For R > 0, let KR (x) ≡

1 1 2π R



R

0



T

 eιtx dt dT.

(7.13)

−T

Then (i) KR (x) =

1 (1−cos Rx) , π x2

x = 0

R 2π

x = 0,

(7.14)

and hence KR (·) ≥ 0. (ii) For δ > 0,

 |x|≥δ

KR (x)dx → 0

as

R → ∞.

(7.15)

176

5. Product Measures, Convolutions, and Transforms

(iii)



∞

−∞

KR (x)dx = 1.

(7.16)

Proof: (i) KR (0) =

1 2πR

R 0

(2T )dT = KR (x)

1 2π R.

= =

For x = 0 and R > 0,

1 2πR



R

0

2 sin T x dT x

1 2(1 − cos Rx) . 2πR x2

(ii) For δ > 0,  0≤

|x|≥δ

KR (x)dx



≤

2 2πR

=

2 1 →0 πR δ

|x|≥δ

1 dx x2 as

R → ∞.

(iii) 

∞

−∞

KR (x)dx

= =

 2 1 ∞  1 − cos Rx  dx π R 0 x2  2 ∞  1 − cos u  du. π 0 u2

Now 

∞

1 − cos u du u2 0  L 1 − cos u  = lim du (by the MCT) L→∞ 0 u2  L u 1 = lim sin x dx 2 du L→∞ 0 u 0  L  L 1  = lim sin x du dx (by Fubini’s theorem) 2 L→∞ 0 x u    L sin x  1 L = lim dx − sin x dx L→∞ x L 0 0  L * L * sin x * * = lim dx since * sin x dx* ≤ 1. L→∞ 0 x 0

5.7 Fourier transforms on R

Thus,



∞

1 − cos u du = u2 0 (cf. Problem 5.9). Hence (iii) follows.

 0

∞

π sin x dx = x 2

Lemma 5.7.7: Let f ∈ Lp (R), 0 < p < ∞. Then  ∞ |f (x − u) − f (x)|p dx → 0 as |u| → 0.

177

(7.17) 2

(7.18)

−∞

Proof: The lemma holds if f ∈ CK , i.e., if f is continuous on R (with values in C) and vanishes outside a bounded interval. By Theorem 2.3.14, such functions are dense in Lp (R). So given f ∈ Lp (R), 0 < p < ∞ and  > 0, let g ∈ CK be such that  |f − g|p dm < . For any 0 < p < ∞, there is a 0 < cp < ∞ such that for all x, y, z ∈ (0, ∞), |x + y + z|p ≤ cp (|x|p + |y|p + |z|p ). Then,  |f (x − u) − f (x)|p dx 
≤ cp |f (x − u) − g(x − u)|p + |g(x − u) − g(x)|p   + |g(x) − f (x)|p dx    = cp 2 + |g(x − u) − g(x)|p du. 

So lim

|u|→0

|f (x − u) − f (x)|p dx ≤ cp 2.

Since  > 0 is arbitrary, the lemma is proved.

2

Proof of Theorem 5.7.3: From (7.7)  R  T  ∞   1 ιtx DR (f, x) ≡ e e−ιty f (y)dy dt dT 2πR 0 −T −∞  R  T  ∞   1 1 eιtu f (x − u)du dt dT. = 2π R 0 −T −∞ Now Fubini’s theorem yields  ∞  1 1  R  T   DR (f, x) = f (x − u) eιtu dt dR du 2π R 0 −∞ −T  ∞ f (x − u)KR (u)du (7.19) = −∞

178

5. Product Measures, Convolutions, and Transforms

where KR (·) is as in (7.13). Now let f be continuous at x0 and bounded on R by Mf . Fix  > 0 and choose δ > 0 such that |x − x0 | < δ ⇒ |f (x) − f (x0 )| < . From (7.16) and (7.19), 
 DR (f, x0 ) − f (x0 ) = f (x0 − u) − f (x0 ) KR (u)du implying



|DR (f, x0 ) − f (x0 )|

≤

|u|<δ

|f (x0 − u) − f (x0 )|KR (u)du



|f (x0 − u) − f (x0 )|KR (u)du  <  + 2Mf KR (u)du.

+

|u|≥δ

|u|≥δ

Now from (7.15), it follows that lim |DR (f, x0 ) − f (x0 )| ≤ ,

R→∞

proving (i). The proof of (ii) is similar to this and is omitted. Clearly (iv) implies (iii). To establish (iv), note that for  1 ≤ p < ∞, by Jensen’s inequality (which applies since KR (u) ≥ 0 and KR (u)du = 1), for x in R,  *
*p |DR (f, x) − f (x)|p ≤ * f (x − u) − f (x) * KR (u)du and hence     |f (x − u) − f (x)|p dx KR (u)du. |DR (f, x) − f (x)|p dx ≤ Now (7.18) and the arguments in the proof of (i) yield (iv).

2

5.8 Plancherel transform If f ∈ L2 (R), it need not be in L1 (R) and so the Fourier transform is not defined. However, it is possible to extend the definition using an approximation procedure due to Plancherel.  Proposition 5.8.1: Let f ∈ L1 (R) ∩ L2 (R) and let fˆ(t) ≡ f (x)e−ιtx dx, t ∈ R. Then, fˆ ∈ L2 (R) and   1 |f |2 dm = |fˆ|2 dm. (8.1) 2π

5.8 Plancherel transform

179

Proof: Let f˜(x) = f (−x) and g = f ∗ f˜. Since f and f˜ are in L1 (R), g is well defined and is in L1 (R). Further by Cauchy-Schwarz inequality,  |f (x1 − u) − f (x2 − u)| |f˜(u)|du |g(x1 ) − g(x2 )| ≤ 1/2     2 |f˜(u)|2 du . ≤ |f (x1 − u) − f (x2 − u)| du By Lemma 5.7.7, the right side goes to zero uniformly as |x1 − x2 | → 0. Thus g(·) is uniformly continuous. Also, by Cauchy-Schwarz inequality, * * * * |g(x)| = * f (x − u)f˜(u)du* 1/2   1/2  2 ≤ |f (u)| du |f˜(u)|2 du  = |f (u)|2 du, and hence g(·) is bounded. Thus, g is continuous, bounded, and integrable on R. By Theorem 5.7.3 (Fejer’s theorem) DR (g, 0) → g(0)

as R → ∞.



But g(0) =

 f (u)f (−u)du =

|f |2 dm.

ˆ By Proposition 5.7.2, gˆ(t) = fˆ(t)f˜(t). Also, note that  ˆ ˜ f (t) = f˜(x)e−ιtx dx  = f (−x)e−ιtx dx fˆ(t)

= and hence gˆ(t) = |fˆ(t)|2 ≥ 0. But DR (g, 0)

= =

   1 1 R T gˆ(t)dt dT 2π R 0 −T  R  |t|  1 gˆ(t) 1 − dt. π 0 R

Since gˆ(·) ≥ 0, by the MCT,  0

R

 ∞  |t|  dt ↑ gˆ(t) 1 − gˆ(t)dt R 0

as R ↑ ∞.

(8.2)

(8.3)

180

5. Product Measures, Convolutions, and Transforms

Thus, lim DR (g, 0) = R→∞

1 π

∞ 0

gˆ(t)dt. Since gˆ(−t) = gˆ(t) for all t in R,

1 lim DR (g, 0) = R→∞ 2π



∞

1 gˆ(t)dt = 2π −∞



|fˆ(t)|2 dt.

(8.4) 2

Clearly, (8.2)–(8.4) imply (8.1).

Proposition 5.8.2: Let f ∈ L2 (R) and fn (·) ≡ f I[−n,n] (·), n ≥ 1. Then, {fn }n≥1 , {fˆn }n≥1 are both Cauchy in L2 (R) and hence, convergent in L2 (R). Proof: Since for each n ≥ 1, fn ∈ L2 (R) ∩ L1 (R), by Proposition 5.8.1,  1 fn1 − fn2 22 = |fˆn1 − fˆn2 |2 dm. (8.5) 2π Since f ∈ L2 (R), fn → f in L2 (R), {fn }n≥1 is Cauchy in L2 (R). By (8.5) {fˆn }n≥1 is also Cauchy in L2 (R). 2 Definition 5.8.1: Let f ∈ L2 (R). The Plancherel transform of f , denoted by fˆ, is defined as limn→∞ fˆn , where  n ˆ e−ιtx f (x)dx (8.6) fn (t) = −n

2

and the limit is taken in L (R). Theorem 5.8.3: Let f ∈ L2 (R) and fˆ be its Plancherel transform. Then (i)



|f |2 dm =

1 2π



|fˆ|2 dm.

(8.7)

(ii) For f ∈ L1 (R) ∩ L2 (R), the Plancherel transform coincides with the Fourier transform. Proof: From Propositions 5.8.1 and 5.8.2 and the definition of fˆ,     1 1 |fn |2 dm = |f |2 dm |fˆ|2 dm = lim |fˆn |2 dm = lim n→∞ 2π n→∞ 2π proving (i). If f ∈ L1 (R) ∩ L2 (R), fˆn defined in (8.6) converges as n → ∞ pointwise to the Fourier transform of f (by the DCT). But by Definition 5.7.1, fˆn converges in L2 (R) to the Plancherel transform of f . So (ii) follows. 2 It can also be shown that for f , fˆ as in the above theorem, the following inversion formula holds:  n 1 (8.8) fˆ(t)eιtx dt → f (x) in L2 (R) 2π −n

5.9 Problems

181

and that the map f → fˆ is a Hilbert space isomorphism of L2 (R) onto L2 (R). For a proof, see Rudin (1987), Chapter 9. This in turn implies that if f , g ∈ L2 (R) with Plancherel transforms fˆ and gˆ, respectively, then   1 fˆ g¯ˆ dm. (8.9) f g¯ dm = 2π This is known as the Parseval identity.

5.9 Problems 5.1 Verify that if µ1 and µ2 are finite measures on (Ω1 , F1 ) and (Ω2 , F2 ), respectively, then µ12 (·) and µ21 (·), defined by (1.4) and (1.5), respectively, are measures on (Ω1 × Ω2 , F1 × F2 ). (Hint: Use the MCT.) 5.2 Let (Ω, F, µ) be a complete measure space such that F = P(Ω) and for some A0 ∈ F with A0 = ∅, µ(A0 ) = 0. Let B ∈ P(Ω) \ F. Then show that (µ × µ)(Ω × A0 ) = 0 but B × A0 ∈ F × F. Conclude that (Ω×Ω, F ×F, µ×µ) is not complete. (An example of such a measure space (Ω, F, µ) is the space ([0, 1], ML , m) where ML is the Lebesgue σ-algebra on [0, 1] and m is the Lebesgue measure.) 5.3 Let µi , i = 1, 2 be two σ-finite measures on (R, B(R)). Let Di = {x : µi ({x}) > 0}, i = 1, 2. (a) Show that D1 ∪ D2 is countable. (b) Let φi (x) = µi ({x}) x ∈ R, i = 1, 2. Show that φi is Borel measurable for i = 1, 2.   (c) Show that φ1 dµ2 = z∈D1 ∩D2 φ1 (z)φ2 (z). (d) Deduce (2.11) from (c). 5.4 Extend Theorem 5.2.3 as follows. Let F1 , F2 be two nondecreasing right continuous functions on [a, b]. Then    F1 dF2 + F2 dF1 = F1 (b)F2 (b) − F1 (a)F2 (a) + φ1 dµ2 , (a,b]

(a,b]

(a,b]

where φ1 is as in Problem 5.3. 5.5 Let Fi : R → R be nondecreasing and right continuous, i = 1, 2. Show that if limb↑∞ F1 (b)F2 (b) = λ1 and lima↓−∞ F1 (a)F2 (a) = λ2 exist and are finite, then    F1 dF2 + F2 dF1 = λ1 − λ2 + φ1 dµ2 R

R

where φ1 is as in Problem 5.3.

R

182

5. Product Measures, Convolutions, and Transforms

5.6 Let (Ω, F, µ) be a σ-finite measure space and f be a nonnegative   measurable function. Then, Ω f dµ = [0,∞) µ({f ≥ t})dt. (Hint: Consider the product space of (Ω, F, µ) with (R+ , B(R+ ), m) and apply Tonelli’s theorem to the function g(ω, t) = I(f (ω) ≥ t), after showing that g is F × B(R+ ))-measurable.) 5.7 Let (Ω, F, P ) be a probability space and X : Ω → R+ be a random variable. (a) Show that for any h : R+ → R+ that is absolutely continuous,   h(X)dP = h(0) + h (t)P (X ≥ t)dt Ω [0,∞)  = h(0) + h (t)P (X > t)dt. (0,∞)

(b) Show that for any 0 < p < ∞,   p X dP = ptp−1 P (X ≥ t)dt. Ω

[0,∞)

(c) Show that for any 0 < p < ∞,   1 −p X dP = ψX (t)tp−1 dt, Γ(p) [0,∞) Ω 

where Γ(p) = t ∈ R+ .

[0,∞)

e−t tp−1 dt, p > 0, and ψX (t) =

 Ω

e−tX dP ,

(Hint: (a) Apply Tonelli’s theorem to the function f (t, ω) ≡ h (t)I(X(w) ≥ t) on the product measure space ([0, ∞) × Ω, B([0, ∞)) × F, m × P ), where m is Lebesgue measure on (R+ , B(R+ )).) 5.8 Let g : R+ → R+ and f : R2 → R+ be Borel measurable. Let A = {(x, y) : x ≥ 0, 0 ≤ y ≤ g(x)}. (a) Show that A ∈ B(R2 ). (b) Show that   R+

[0,g(x)]

  f (x, y)m(dy) m(dx) = f IA dm(2) ,

where m(2) is Lebesgue measure on (R2 , B(R2 )) and m(·) is Lebesgue measure on R.

5.9 Problems

183

(c) If g is continuous and strictly increasing show that the two integrals in (b) equal    f (x, y)m(dx) m(dy). [g −1 (y),∞)

R+

5.9 (a) For 1 < A < ∞, let  hA (t) =

A

0

e−xt sin x dx, t ≥ 0.

Use integration by parts to show that |hA (t)| ≤ and hA (t) →

1 + e−t 1 + t2

1 1 + t2

as A → ∞.

(b) Show using Fubini’s theorem that for 0 < A < ∞,  A  ∞ sin x dx. hA (t)dt = x 0 0 (c) Conclude using the DCT that  A  ∞ sin x 1 lim dx = dt. A→∞ 0 x 1 + t2 0 (d) Using Theorem 4.4.1 and the fact that φ(x) ≡ tan x is a (1–1) strictly monotone map from (0, π2 ) to (0, ∞) having the inverse 1 map ψ(·) with derivative ψ  (t) ≡ 1+t 2 , 0 < t < ∞, conclude that  ∞ π 1 dt = . 2 1+t 2 0  ∞ −x2 /2 ,π 5.10 Show that I ≡ 0 e dx = 2 .  ∞  ∞ (x2 +y2 ) (Hint: By Tonelli’s theorem, I 2 = 0 0 e− 2 dxdy. Now use the change of variables x = r cos θ, y = r sin θ, 0 < r < ∞, 0 < θ < π 2 .)
 5.11 Let µ be a finite measure on R, B(R) . Let f , g : R → R+ be nondecreasing. Show that      µ(R) f g dµ ≥ f dµ gdµ .

 (Hint: Consider h(x1 , x2 ) = f (x1 ) − f (x2 ) g(x1 ) − g(x2 ) on R2 and integrate w.r.t. µ × µ.)

184

5. Product Measures, Convolutions, and Transforms


 5.12 Let µ and λ be σ-finite measures on R, B(R) . Recall that   ν(A) ≡ (µ ∗ λ)(A) ≡ IA (x + y)dµdλ, A ∈ B(R). (a) Show that for any Borel measurable f : R → R+ , f (x + y) is B(R) × B(R), B(R)-measurable from R × R → R and    f dν = f (x + y)dµdλ. (b) Show that  ν(A) = R

µ(A − t)λ(dt), A ∈ B(R).

(c) Suppose there exist countable sets Bλ , Bµ such that µ(Bµc ) = 0 = λ(Bλc ). Show that there exists a countable set Bν such that ν(Bνc ) = 0. (d) Suppose that µ({x}) = 0 for all x in R. Show that ν({x}) = 0 for all x in R. dµ (e) Suppose that µ  m with dm = h. Show that ν  m and find dν dm in terms of h, µ and λ.

(f) Suppose that µ  m and λ  m. Show that dµ dλ dν = ∗ . dm dm dm 5.13 (Convolution of cdfs). Let Fi , i = 1, 2 be cdfs on R. Recall that a cdf F on R is a function from R → R+ such that it is nondecreasing, right continuous with F (x) → 0 as x → −∞ and F (x) → 1 as x → ∞.  (a) Show that (F1 ∗ F2 )(x) ≡ R F1 (x − u)dF2 (u) is well defined and is a cdf on R. (b) Show also that (F1 ∗ F2 )(·) = (F2 ∗ F1 )(·).  (c) Suppose t ∈ R is such that etx dFi (x) < ∞ for i = 1,2. Show  tx  tx  tx e dF1 (x) e dF2 (x) . that e d(F1 ∗ F2 )(x) = 5.14 Let f , g ∈ L1 (R, B(R), m). (a) Show that if f is continuous and bounded on R, then so is f ∗ g. (b) Show that if f is differentiable with a bounded derivative on R, then so is f ∗ g. (Hint: Use the DCT.)

5.9 Problems

185

5.15 Let f ∈ L1 (R), g ∈ Lp (R), 1 ≤ p ≤ ∞. (a) Show that if 1 ≤ p < ∞, then for all x in R *p   *  p−1   * * |g(x−u)|p |f (u)|du |f |dm * |f (u)g(x−u)|du* ≤ and hence that

 (f ∗ g)(x) ≡

f (u)g(x − u)du

is well defined a.e. (m) and f ∗ gp ≤ f 1 gp with “=” holding iff either f = 0 a.e. or g = 0 a.e. (b) Show that if p = ∞ then f ∗ g∞ ≤ f 1 g∞ and “=” can hold for some nonzero f and g. (Hint for (a): Use Jensen’s inequality with probability measure dµ = |f |dm f 1 if f ||1 > 0.) 5.16 Let 1 ≤ p ≤ ∞ and q = 1 − p1 . Let f ∈ Lp (R), g ∈ Lq (R). (a) Show that f ∗ g is well defined and uniformly continuous. (b) Show that if 1 < p < ∞, lim (f ∗ g)(x) = 0.

|x|→∞

(Hint: For (a) use H¨ older’s inequality and Lemma 5.7.7. For (b) approximate g by simple functions.) 5.17 Let g : R → R be infinitely differentiable and be zero outside a bounded interval.  (a) Let f : R → R be Borel measurable and A |f |dm < ∞ for all bounded intervals A in R. Show that f ∗ g is well defined and infinitely differentiable. (b) Show that for any f ∈ L1 (R), there exist a sequence {gn }n≥1 of such functions such that f ∗ gn → f in L1 (R). 5.18 For f ∈ L1 (R), let fσ = f ∗φσ where φσ (x) = x ∈ R.

2

x √ 1 e− 2σ2 2πσ

, 0 < σ < ∞,

186

5. Product Measures, Convolutions, and Transforms

(a) Show that fσ is infinitely differentiable. (b) Show that if f is continuous and zero outside a bounded interval, then fσ converges to g uniformly on R as σ → 0. (c) Show that if f ∈ Lp (R), 1 ≤ p < ∞, then fσ → f in Lp (R) as σ → 0. 5.19 Let f ∈ Lp (R), 1 ≤ p < ∞ and h(x) = bounded Borel set and

 A+x

f (u)du where A is a

A + x ≡ {y : y = a + x, a ∈ A}. (a) Show that h = f ∗ g for some g bounded and with bounded support. (b) Show that h(·) is continuous and that lim h(x) = 0. |x|→∞


(Hint: For 1 <p < ∞, use H¨older’s inequality and show that m (A+ x1 )  (A + x2 ) → 0 as |x1 − x2 | → 0.) 5.20 (a) Verify the claims in Examples 5.4.1 directly. (b) Verify the same using generating functions. 5.21 Let f be a probability density on R, i.e., f is nonnegative, Borel  measurable and f dm = 1. Show that |fˆ(t)| < 1 for all t = 0.  
1 − cos t0 (x − θ) (Hint: If |fˆ(t0 )| = 1 for some t0 = 0, show that f (x)dx = 0 for some θ.) 5.22 (a) Let f ∈ L1 (R) and xk f (x) ∈ L1 (R) for some k ≥ 1. Show that fˆ(·) is k-times differentiable on R with all derivatives fˆ(r) (t) → 0 as |t| → ∞ for r ≤ k.  (b) Let f ∈ L1 (R). Suppose |tfˆ(t)|dt < ∞. Show that there exists a function g : R → R such that it is differentiable, lim|x|→∞ (|g(x)| + |g  (x)|) = 0 and g = f a.e. (m). Extend this  to the case where |tk fˆ(t)|dt < ∞ for some k > 1.  −ιtx 1 (Hint: Consider g(x) = 2π fˆ(t)dt.) e 2

5.23 Let f (x) =

x √1 e− 2 2π

, x ∈ R.

(a) Show that fˆ(·) is real valued, differentiable and satisfies the ordinary differential equation fˆ (t) + tfˆ(t) = 0, t ∈ R and fˆ(0) = 1. Find fˆ(t).

5.9 Problems

187

(b) For µ in R, σ > 0, let fµ,σ (x) = √

1 x−u 2 1 e− 2 ( σ ) . 2πσ

Find fˆµ,σ and verify that for any (µi , σi ), i = 1, 2, fµ1 ,σ1 ∗ fµ2 ,σ2 = fµ1 +µ2 ,σ12 +σ22 . (Hint for (b): Use Fourier transforms and uniqueness.) 5.24 (Rate of convergence of Fourier series). Consider the function h(x) = π 2 − |x| in −π ≤ x ≤ π.  π −ιnx 1 ˆ e h(x)dx, n = 0, ±1, ±2. (a) Find h(n) ≡ 2π −π ∞ ˆ < ∞. (b) Show that n=−∞ |h(n)| +n ˆ ιjx (c) Show that Sn (h, x) ≡ converges to h(x) unij=−n h(j)e formly on [−π, π]. (d) Verify that sup{|Sn (h, x) − h(x)| : −π ≤ x ≤ π} ≤ and |Sn (h, 0) − h(0)| ≥

2 1 , n≥2 π (n − 1)

2 1 . π (n + 2)

(Remark: This example shows that the Fourier series of a function can converge very slowly such as in this example where the rate of decay is n1 .) 5.25 Using Fejer’s theorem (Theorem 5.6.1) prove Wierstrass’ theorem on uniform approximation of a continuous function on a bounded closed interval by a polynomial. (Hint: Show that on bounded intervals a trigonometric polynomial can be approximated uniformly by a polynomial using the power series representation of sine and cosine functions (see Section A.3). 5.26 Evaluate 

n

lim

n→∞

−n

sin λx ιtx e dx, 0 < λ < ∞, 0 < x < ∞. x

(Hint: For 0 < λ < ∞, sinxλx ∈ L2 (R) and it is the Fourier transform of f (t) = I[−λ,λ] (·). Now apply Plancherel theorem. Alternatively use n Fubini’s theorem and the fact lim −n siny y dy exists in R.) n→∞

188

5. Product Measures, Convolutions, and Transforms


c 5.27 Find an example of a function f ∈ L2 (R) ∩ L1 (R) such that its Plancherel transform fˆ ∈ L1 (R). (Hint: Examine Problem 5.26.)

6 Probability Spaces

6.1 Kolmogorov’s probability model Probability theory provides a mathematical model for random phenomena, i.e., those involving uncertainty. First one identifies the set Ω of possible outcomes of (random) experiment associated with the phenomenon. This set Ω is called the sample space, and an individual element ω of Ω is called a sample point. Even though the outcome is not predictable ahead of time, one is interested in the “chances” of some particular statement to be valid for the resulting outcome. The set of ω’s for which a given statement is valid is called an event. Thus, an event is a subset of Ω. One then identifies a class F of events, i.e., a class F of subsets of Ω (not necessarily all of P(Ω), the power set of Ω), and then a set function P on F such that for A in F, P (A) represents the “chance” of the event A happening. Thus, it is reasonable to impose the following conditions on F and P : (i) A ∈ F ⇒ Ac ∈ F (i.e., if one can define the probability of an event A, then the probability of A not happening is also well defined). (ii) A1 , A2 ∈ F ⇒ A1 ∪ A2 ∈ F (i.e., if one can define the probabilities of A1 and A2 , then the probability of at least one of A1 or A2 happening is also well defined). (iii) for all A in F, 0 ≤ P (A) ≤ 1, P (∅) = 0, and P (Ω) = 1.

190

6. Probability Spaces

(iv) A1 , A2 ∈ F, A1 ∩ A2 = ∅ ⇒ P (A1 ∪ A2 ) = P (A1 ) + P (A2 ) (i.e., if A1 and A2 are mutually exclusive events, then the probability of at least one of the two happening is simply the sum of the probabilities). The above conditions imply that F is an algebra and P is a finitely additive set function. Next, as explained in Section 1.2, it is natural to require that F be closed under monotone increasing unions and P be monotone continuous from below. That is, if {An }n≥1 is a sequence of events in F such that An implies An+1 (i.e., An ⊂ An+1 ) for all n ≥ 1, then the probability of at least one of the An ’s happening is well defined and is the limit of the corresponding probabilities. In other words, the following conditions on F and P must hold in addition to (i)–(iv) above: (v) An ∈ F, A n ⊂ An+1 for all n = 1, 2, . . . P (An ) ↑ P ( n≥1 An ).

⇒

 n≥1

An , ∈ F and

As noted in Section 1.2, conditions (i)–(v) imply that (Ω, F, P ) is a measure space, i.e., F is a σ-algebra and P is a measure on F with P (Ω) = 1. That is, (Ω, F, P ) is a probability space. This is known as Kolmogorov’s probability model for random phenomena (see Kolmogorov (1956), Parthasarathy (2005)). Here are some examples. Example 6.1.1: (Finite sample spaces). Let Ω ≡ {ω1 , ω2 , . . . , ωk }, 1 ≤ k k < ∞, F ≡ P(Ω), the power set of Ω and P (A) = i=1 pi IA (ωi ) where  k {pi }ki=1 are such that pi ≥ 0 and i=1 pi = 1. This is a probability model for random experiments with finitely many possible outcomes. An important application of this probability model is finite population sampling. Let {U1 , U2 , . . . , UN } be a finite population of N units or objects. These could be individuals in a city, counties in a state, etc. In a typical sample survey procedure, one chooses a subset of size n (1 ≤ n ≤ N ) from this population.
 Let Ω denote the collection of all possible subsets of size n. Here k = N n , each ωi is a sample of size n and pi is the selection probability of ωi . The assignment of {pi }ki=1 is determined by a given sampling scheme. For example, in simple random sampling without replacement, pi = k1 for i = 1, 2, . . . , k. Other examples include coin tossing, rolling of dice, bridge hands, and acceptance sampling in statistical quality control (Feller (1968)). Example 6.1.2: (Countably infinite sample spaces). Let Ω ≡ {ω1 , ω2 , . . .} ∞ ∞ p I be a countable set, F = P(Ω), and P (A) ≡ i A (ωi ) where {pi }i=1 i=1 ∞ satisfy pi ≥ 0 and i=1 pi = 1. It is easy to verify that (Ω, F, P ) is a probability space. This is a probability model for random experiments with countably infinite number of outcomes. For example, the experiment of tossing a coin until a “head” is produced leads to such a probability space.

6.2 Random variables and random vectors

191

Example 6.1.3: (Uncountable sample spaces). (a) (Random variables). Let Ω = R, F = B(R), P = µF , the LebesgueStieltjes measure corresponding to a cdf F , i.e., corresponding to a function F : R → R that is nondecreasing, right continuous, and satisfies F (−∞) = 0, F (+∞) = 1. See Section 1.3. This serves as a model for a single random variable X. (b) (Random vectors). Let Ω = Rk , F = B(Rk ), P = µF , the LebesgueStieltjes measure corresponding to a (multidimensional) cdf F on Rk where k ∈ N. See Section 1.3. This is a model for a random vector (X1 , X2 , . . . , Xk ). (c) (Random sequences). Let Ω = R∞ ≡ R × R × . . . be the set of all sequences {xn }n≥1 of real numbers. Let C be the class of all finite dimensional sets of the form A × R × R × . . ., where A ∈ B(Rk ) for some 1 ≤ k < ∞. Let F be the σ-algebra generated by C. For each 1 ≤ k < ∞, let µk be a probability measure on B(Rk ) such that µk+1 (A × R) = µk (A) for all A ∈ B(Rk ). Then there exists a probability measure µ on F such that µ(A × R × R × . . .) = µk (A) if A ∈ B(Rk ). (This will be shown later as a special case of the Kolmogorov’s consistency theorem in Section 6.3.) This will be a model for a sequence {Xn }n≥1 of random variables such that for each k, 1 ≤ k < ∞, the distribution of (X1 , X2 , . . . , Xk ) is µk .

6.2 Random variables and random vectors Recall the following definitions introduced earlier in Sections 2.1 and 2.2. Definition 6.2.1: Let (Ω, F, P ) be a probability space and X : Ω → R be F, B(R)-measurable, that is, X −1 (A) ∈ F for all A ∈ B(R). Then, X is called a random variable on (Ω, F, P ). Recall that X : Ω → R is F, B(R)-measurable iff for all x ∈ R, {ω : X(ω) ≤ x} ∈ F. Definition 6.2.2: Let X be a random variable on (Ω, F, P ). Let FX (x) ≡ P ({ω : X(ω) ≤ x}), x ∈ R.

(2.1)

Then FX (·) is called the cumulative distribution function (cdf) of X. Definition 6.2.3: Let X be a random variable on (Ω, F, P ). Let PX (A) ≡ P (X −1 (A))

for all A ∈ B(R).

(2.2)

Then the probability measure PX is called the probability distribution of X.

192

6. Probability Spaces

Note that PX is the measure induced by X on (R, B(R)) under P and that the Lebesgue-Stieltjes measure µFX on B(R) corresponding with the cdf FX of X is the same as PX . Definition 6.2.4: Let (Ω, F, P ) be a probability space, k ∈ N and X : Ω → Rk be F, B(Rk )-measurable, i.e., X −1 (A) ∈ F for all A ∈ B(Rk ). Then X is called a (k-dimensional) random vector on (Ω, F, P ). Let X = (X1 , X2 , . . . , Xk ) be a random vector with components Xi , i = 1, 2, . . . , k. Then each Xi is a random variable on (Ω, F, P ). This follows from the fact that the coordinate projection maps from Rk to R, given by πi (x1 , x2 , . . . , xk ) ≡ xi , 1 ≤ i ≤ k are continuous and hence, are Borel measurable. Conversely, if for 1 ≤ i ≤ k, Xi is a random variable on (Ω, F, P ), then X = (X1 , X2 , . . . , Xk ) is a random vector (cf. Proposition 2.1.3). Definition 6.2.5: Let X be a k-dimensional random vector on (Ω, F, P ) for some k ∈ N. Let FX (x) ≡ P ({ω : X1 (ω) ≤ x1 , X2 (ω) ≤ x2 , . . . , Xk (ω) ≤ xk })

(2.3)

for x = (x1 , x2 , . . . , xk ) ∈ Rk . Then FX (·) is called the joint cumulative distribution function (joint cdf) of the random vector X. Definition 6.2.6: Let X be a k-dimensional random vector on (Ω, F, P ) for some k ∈ N. Let PX (A) = P (X −1 (A))

for all A ∈ B(Rk ).

(2.4)

The probability measure PX is called the (joint) probability distribution of X. As in the case k = 1, the Lebesgue-Stieltjes measure µFX on B(Rk ) corresponding to the joint cdf FX is the same as PX . Next, let X = (X1 , X2 , . . . , Xk ) be a random vector. Let Y = (Xi1 , Xi2 , . . . , Xir ) for some 1 ≤ i1 < i2 < . . . < ir ≤ k and some 1 ≤ r ≤ k. Then, Y is also a random vector. Further, the joint cdf of Y can be obtained from FX by setting the components xj , j ∈ {i1 , i2 , . . . , ir } equal to ∞. Similarly, the probability distribution PY can be obtained from PX as an induced measure from the projection map π(x) = (xi1 , xi2 , . . . , xir ), x ∈ Rk . For example, if (i1 , i2 , . . . , ir ) = (1, 2, . . . , r), r ≤ k, then FY (y1 , . . . , yr ) = FX (y1 , . . . , yr , ∞, . . . , ∞), (y1 , . . . , yr ) ∈ Rr and PY (A) = PX (A × R(k−r) ), A ∈ B(Rr ).

6.2 Random variables and random vectors

193

Definition 6.2.7: Let X = (X1 , X2 , . . . , Xk ) be a random vector on (Ω, F, P ). Then, for each i = 1, . . . , k, the cdf FXi and the probability distribution PXi of the random variable Xi are called the marginal cdf and the marginal probability distribution of Xi , respectively. It is clear that the distribution of X determines the marginal distribution PXi of Xi for all i = 1, 2, . . . , k. However, the marginal distributions {PXi : i = 1, 2, . . . , k} do not uniquely determine the joint distribution PX , without additional conditions, such as independence (see Problem 6.1). Definition 6.2.8: Let X be a random variable on (Ω, F, P ). The expected value of X, denoted by EX or E(X), is defined as  XdP, (2.5) EX = Ω

provided  the integral  is well defined. That is, at least one of the two quantities X + dP and X − dP is finite. If X is a random variable on (Ω, F, P ) and h : R → R is Borel measurable, then Y = h(X) is also a random variable on (Ω, F, P ). The expected value of Y may be computed as follows. Proposition 6.2.1: (The change of variable formula). Let X be a random variable on (Ω, F, P ) and h : R → R be Borel measurable. Let Y = h(X). Then    |Y |dP = |h(x)|PX (dx) = |y|PY (dy). (i) Ω

(ii) If

 Ω

R

R

|Y |dP < ∞, then    Y dP = h(x)PX (dx) = yPY (dy). Ω

R

(2.6)

R

Proof: If h = IA for A in B(R), the proposition follows from the definition of PX . By linearity, this extends to a nonnegative and simple function h and by the MCT, to any nonnegative measurable h, and hence to any measurable h. 2 Remark 6.2.1: Proposition 6.2.1 shows that the expectation of Y can be computed in three different ways, i.e., by integrating Y with respect to P on Ω or by integrating h(x) on R with respect to the probability distribution PX of the random variable X or by integrating y on R with respect to the probability distribution PY of the random variable Y . Remark 6.2.2: If the function h is nonnegative, then the relation EY =  h(x)P (dx) is valid even if EY = ∞. X R

194

6. Probability Spaces

Definition 6.2.9: For any positive integer n, the nth moment µn of a random variable X is defined by µn ≡ EX n ,

(2.7)

provided the expectation is well defined. Definition 6.2.10: The variance of a random variable X is defined as Var(X) = E(X − EX)2 , provided EX 2 < ∞. Definition 6.2.11: The moment generating function (mgf ) of a random variable X is defined by MX (t) ≡ E(etX )

for all t ∈ R.

(2.8)

Since etX is always nonnegative, E(etX ) is well defined but could be infinity. Proposition 6.2.1 gives a way of computing the moments and the mgf of X without explicitly computing the distribution of X k or etX . As an illustration, consider the case of a random variable X defined on the probability space (Ω, F, P ) with Ω = {H, T }n , n ∈ N, F = the power set of Ω and P = the probability distribution defined by P ({ω}) = pX(ω) q n−X(ω) where 0 < p < 1, q = 1 − p, and X(ω) = the number of H’s in ω. By the change of variable formula, the mgf of X is given by  etx PX (dx) MX (t) ≡ =

n r=0

etr

 n r n−r p q = (pet + q)n , r

distribution of X, is supported on {0, 1, 2, . . . , n} since PX , the probability
 with PX ({r}) = nr pr q n−r . Note that PX is the Binomial (n, p) distribution. Here, MX (t) is computed using the distribution of X, i.e., using the middle term in (2.6) only. The connection between the mgf MX (·) and the moments of a random variable X is given in the following propositions. Proposition 6.2.2: Let X be a nonnegative random variable and t ≥ 0. Then ∞ n t µn MX (t) ≡ E(etX ) = (2.9) n! n=0 where µn is as in (2.7). ∞ n Proof: Since etX = n=0 tn Xn! and X is nonnegative, (2.9) follows from the MCT. 2

6.2 Random variables and random vectors

195

Proposition 6.2.3: Let X be a random variable and let MX (t) be finite for all |t| < , for some  > 0. Then (i) E|X|n < ∞ (ii) MX (t) =

∞

for all tn

n=0

µn n!

n ≥ 1, for all

|t| < ,

(iii) MX (·) is infinitely differentiable on (−, +) and for r ∈ N, the rth derivative of MX (·) is (r)

MX (t) = In particular,

∞ n t µn+r = E(etX X r ) n! n=0

for

|t| < .

(r)

MX (0) = µr = EX r .

(2.10)

(2.11)

Proof: Since MX (t) < ∞ for all |t| < , E(e|tX| ) ≤ E(etX ) + E(e−tX ) < ∞ Also, e|tX| ≥

for |t| < .

(2.12)

|t|n |X|n n!

for *all n ∈ N and * hence, (i) follows by choosing a t *n (tx)j * in (0, ). Next note that * j=0 j! * ≤ e|tx| for all x in R and all n ∈ N. Hence, by (2.12) and the DCT, (ii) follows. Turning to (iii), since MX (·) admits a power series expansion convergent in |t| < , it is infinitely differentiable in |t| <  and the derivatives of MX (·) can be found by term-by-term differentiation of the power series (see Rudin (1976), Chapter 9). Hence,  ∞ dr tn µn (r) MX (t) = dtr n=0 n! = = =

∞ µn dr (tn ) n! dtr n=0 ∞

µn

n=r ∞ n

tn−r (n − r)!

t µn+r . n! n=0

The verification of the second equality in (2.10) is left in an exercise (see Problem 6.4). 2 Remark 6.2.3: If the mgf MX (·) is finite for |t| <  for some  > 0, then by part (ii) of the above proposition, MX (t) has a power series expansion

196

6. Probability Spaces

in t around 0 and µn!n is simply the coefficient of tn . For example, if X has a N (0, 1) distribution, then for all t ∈ R,  ∞ 2 2 1 MX (t) = etx √ e−x /2 dx = et /2 2π −∞ ∞ 2 k (t ) 1 = . (2.13) k! 2k k=0

 Thus, µn =

0 (2k)! k!2k

if n is odd if n = 2k, k = 1, 2, . . .

Remark 6.2.4: If MX (t) is finite for |t| <  for some  > 0, then all the moments {µn }n≥1 of X are determined and also its probability distribution. However, in general, the sequence {µn }n≥1 of moments of X need not determine the distribution of X uniquely. Table 6.2.1 gives the mean, variance, and the mgf of a number of standard probability distributions on the real line. For future reference, some of the inequalities established in Section 3.1 are specialized for random variables and collected below without proofs. Proposition 6.2.4: (Markov’s inequality). Let X be a random variable on (Ω, F, P ). Then for any φ : R+ → R+ nondecreasing and any t > 0 with φ(t) > 0,
 E φ(|X|) . (2.14) P (|X| ≥ t) ≤ φ(t) In particular, (i) for r > 0, t > 0, P (X ≥ t) ≤ P (|X| ≥ t) ≤

E|X|r , tr

(2.15)

(ii) for any t ≥ 0, P (|X| ≥ t) ≤

E(eθ|X| ) , eθt

for any θ > 0 and hence E(eθ|X| ) . θ>0 eθt

P (|X| ≥ t) ≤ inf

(2.16)

Proposition 6.2.5: (Chebychev’s inequality). Let X be a random variable with EX 2 < ∞, EX = µ, Var(X) = σ 2 . Then for any k > 0, P (|X − µ| ≥ kσ) ≤

1 . k2

(2.17)

6.2 Random variables and random vectors

197

TABLE 6.2.1. Mean, variance, mgf of the distributions listed in Tables 4.6.1 and 4.6.2.

Distribution

Mean

Variance

mgf M(t)

Bernoulli (p), 0
p

p(1 − p)

np

np(1 − p)

(1 − p) + pet , t∈R n
(1 − p) + pet , t∈R

Geometric (p), p ∈ (0, 1)

1 p

1−p p

Poisson (λ) λ ∈ (0, ∞)

λ

λ

a+b 2

(b−a)2 12

β

β2

Binomial (n, p), p ∈ (0, 1), n ∈ N

Uniform (a, b), −∞ < a < b < ∞ Exponential (β), β ∈ (0, ∞)




pet 1−(1−p)et ,

t∈  − ∞, − log(1 − p)
 exp λ(et − 1) , t∈R

ebt −eat (b−a)t ,

t ∈ R \ {0}; M (0) = 1 1 1−βt ,

t ∈ (−∞, β1 )

Gamma (α, β), α, β ∈ (0, ∞)

αβ

αβ 2

Beta (α, β),

α α+β

αβ (α+β)2 (α+β+1)

(1 − βt)−α , t ∈ (−∞, β1 )  ∞ 1 + k=1   k k−1 α+r r=0 α+β+r

α, β ∈ (0, ∞)

t k!

,

t∈R Cauchy (γ, σ), γ ∈ R, σ ∈ (0, ∞) Normal (γ, σ 2 ), γ ∈ R, σ ∈ (0, ∞) Lognormal (γ, σ 2 ), γ ∈ R, σ ∈ (0, ∞)

not defined

not defined

since E|X| = ∞

since E|X|2 = ∞

γ

σ2

eγ+

σ2 2

∞ for all t = 0

2

2

exp(γt + t 2σ ), t∈R 2

[e2(γ+σ ) 2 −e2γ+σ ]

∞ for all t > 0

Proposition 6.2.6: (Jensen’s inequality). Let X be a random variable with P (a < X < b) = 1 for −∞ ≤ a < b ≤ ∞. Let φ : (a, b) → R be convex on (a, b). Then Eφ(X) ≥ φ(EX), provided E|X| < ∞ and E|φ(X)| < ∞.

(2.18)

198

6. Probability Spaces

Proposition 6.2.7: (H¨ older’s inequality). Let X and Y be random variables on (Ω, F, P ) with E|X|p < ∞, E|Y |q < ∞, 1 < p < ∞, 1 < q < ∞, 1 1 p + q = 1. Then E|XY | ≤ (E|X|p )1/p (E|Y |q )1/q ,

(2.19)

with equality holding iff P (c1 |X|p = c2 |Y |q ) = 1 for some 0 ≤ c1 , c2 < ∞. Proposition 6.2.8: (Cauchy-Schwarz inequality). Let X and Y be random variables on (Ω, F, P ) with E|X|2 < ∞, E|Y |2 < ∞. Then , , |Cov(X, Y )| ≤ Var(X) Var(Y ) , (2.20) where Cov(X, Y ) = EXY − EXEY . If Var(X) > 0, then equality holds in (2.20) iff P (Y = aX + b) = 1 for some constants a, b in R (Problem 6.6). Proposition 6.2.9: (Minkowski’s inequality). Let X and Y be random variables on (Ω, F, P ) such that E|X|p < ∞, E|Y |p < ∞ for some 1 ≤ p < ∞. Then (E|X + Y |p )1/p ≤ (E|X|p )1/p + (E|Y |p )1/p . (2.21) Definition 6.2.12: (Product moments of random vectors). Let X = (X1 , X2 , . . . , Xk ) be a random vector. The product moment of order r = (r1 , r2 , . . . , rk ), with ri being a nonnegative integer for each i, is defined as µr ≡ µr1 ,r2 ,...,rk ≡ E(X1r1 X2r2 · · · Xkrk ),

(2.22)

provided E|X1r1 · · · Xkrk | < ∞. The joint moment generating function (joint mgf ) of a random vector X = (X1 , X2 , . . . , Xk ) is defined by MX1 ,...,Xk (t1 , t2 , . . . , tk ) ≡ E(et1 X1 +t2 X2 +···+tk Xk ),

(2.23)

for all t1 , t2 , . . . , tk in R. As in the case of a random variable, if the joint mgf MX1 ,X2 ,...,Xk (t1 , . . . , tk ) is finite for all (t1 , t2 , . . . , tk ) with |ti | <  for all i = 1, 2, . . . , k for some  > 0, then an analog of Proposition 6.2.3 holds. For example, the following assertions are valid (cf. Problem 6.4): (i) E|Xi |n < ∞ for all i = 1, 2, . . . , k

and n ≥ 1.

(ii) For t = (t1 , . . . , tk ) ∈ Rk and r = (r1 , r2 , . . . , rk ) ∈ Zk+ , let tr = tr11 tr22 · · · trkk , r! = r1 !r2 ! · · · rk !, and µr = EX1r1 X2r2 · · · Xkrk .

(2.24)

6.3 Kolmogorov’s consistency theorem

Then, MX (t1 , . . . , tk ) =

tr µr r! k

199

(2.25)

r∈Z+

for all t = (t1 , t2 , . . . , tk ) ∈ (−, +)k . (iii) For any r = (r1 , . . . , rk ) ∈ Zk+ ,

* dr * M (t) = µr , * X dtr t=0

where

dr dtr

=

∂ r1 ∂ r2 r r ∂t11 ∂t22

(2.26)

rk

∂ . . . ∂t rk . k

6.3 Kolmogorov’s consistency theorem In the previous section, the case of a single random variable and that of a finite dimensional random vector were discussed. The goal of this section is to discuss infinite families of random variables such as a random sequence {Xn }n≥1 or a random function {X(t) : 0 ≤ t < T }, 0 ≤ T ≤ ∞. For example, Xn could be the population size of the nth generation of a randomly evolving biological population, and X(t) could be the temperature at time t in a chemical reaction over a period [0, T ]. An example from modeling of spatial random phenomenon is a collection {X(s) : s ∈ S} of random variables X(s) where S is a specified region such as the U.S., and X(s) is the amount of rainfall at location s ∈ S during a specified month. Let (Ω, F, P ) be a probability space and {Xα : α ∈ A} be a collection of random variables defined on (Ω, F, P ), where A is a nonempty set. Then for any (α1 , α2 , . . . , αk ) ∈ Ak , 1 ≤ k < ∞, the random vector (Xα1 , Xα2 , . . . , Xαk ) has a joint probability distribution µ(α1 ,α2 ,...,αk ) over (Rk , B(Rk )). Definition 6.3.1: A (real valued) stochastic process with index set A is a family {Xα : α ∈ A} of random variables defined on a probability space (Ω, F, P ). Example 6.3.1: (Examples of stochastic processes). Let Ω = [0, 1], F = B([0, 1]), P = the Lebesgue measure on [0, 1]. Let A1 = {1, 2, 3, . . .}, A2 = [0, T ], 0 < T < ∞. For ω ∈ Ω, n ∈ A1 , t ∈ A2 , let Xn (ω)

=

sin 2πnω

Yt (ω)

=

sin 2πtω

Zn (ω) Vn,t (ω)

=

nth digit in the decimal expansion of ω

=

Xn2 (ω) + Yt2 (ω).

Then {Xn : n ∈ A1 }, {Zn : n ∈ A1 }, {Vn,t : (n, t) ∈ A1 × A2 }, {Yt : t ∈ A2 } are all stochastic processes.

200

6. Probability Spaces

Note that a real valued stochastic process {Xα : α ∈ A} may also be viewed as a random real valued function on the set A by the identification ω → f (ω, ·), where f (ω, α) = Xα (ω) for α in A. Definition 6.3.2: The family {µ(α1 ,α2 ,...,αk ) (·) ≡ P ((Xα1 , . . . , Xαk ) ∈ ·): (α1 , α2 , . . . , αk ) ∈ Ak , 1 ≤ k < ∞} of probability distributions is called the family of finite dimensional distributions (fdds) associated with the stochastic process {Xα : α ∈ A}. This family of finite dimensional distributions satisfies the following consistency conditions: For any (α1 , α2 , . . . , αk ) ∈ Ak , 2 ≤ k < ∞, and any B1 , B2 , . . . , Bk in B(R), C1: µ(α1 ,α2 ,...,αk ) (B1 ×· · ·×Bk−1 ×R) = µ(α1 ,α2 ,...,αk−1 ) (B1 ×· · ·×Bk−1 ); C2: For any permutation (i1 , i2 , . . . , ik ) of (1, 2, . . . , k), µ(αi1 ,αi2 ,...,αik ) (Bi1 ×Bi2 ×· · ·×Bik ) = µ(α1 ,...,αk ) (B1 ×B2 ×· · ·×Bk ) . To verify C1, note that µ(α1 ,α2 ,...,,αk ) (B1 × B2 × · · · × Bk−1 × R) =

P (Xα1 ∈ B1 , Xα2 ∈ B2 , . . . , Xαk−1 ∈ Bk−1 , Xαk ∈ R)

=

P (Xα1 ∈ B1 , Xα2 ∈ B2 , . . . , Xαk−1 ∈ Bk−1 )

= µ(α1 ,α2 ,...,αk−1 ) (B1 × B2 × · · · × Bk−1 ). Similarly, to verify C2, note that µ(αi1 ,αi2 ,...,αik ) (Bi1 × Bi2 · · · × Bik ) = P (Xαi1 ∈ Bi1 , Xαi2 ∈ Bi2 , . . . , Xαik ∈ Bik ) = P (Xα1 ∈ B1 , Xα2 ∈ B2 , . . . , Xαk ∈ Bk ) = µ(α1 ,α2 ,...,αk ) (B1 × B2 × · · · × Bk ). A natural question is that given a family of probability distributions QA ≡ {ν(α1 ,α2 ,...,αk ) : (α1 , α2 , . . . , αk ) ∈ Ak , 1 ≤ k < ∞} on finite dimensional Euclidean spaces, does there exist a real valued stochastic process {Xα : α ∈ A} such that its family of finite dimensional distributions coincides with QA ? Kolmogorov (1956) showed that if QA satisfies C1 and C2, then such a stochastic process does exist. This is known as Kolmogorov’s consistency theorem (also known as Kolmogorov’s existence theorem). Theorem 6.3.1: (Kolmogorov’s consistency theorem). Let A be a nonempty set. Let QA ≡ {ν(α1 ,α2 ,...,αk ) : (α1 , α2 , . . . , αk ) ∈ Ak , 1 ≤ k < ∞} be a family of probability distributions such that for each (α1 , α2 , . . . , αk ) ∈ Ak , 1 ≤ k < ∞,

6.3 Kolmogorov’s consistency theorem

201

(i) ν(α1 ,α2 ,...,αk ) is a probability distribution on (Rk , B(Rk )), (ii) C1 and C2 hold, i.e., for all B1 , B2 , . . . , Bk ∈ B(R), 2 ≤ k < ∞, ν(α1 ,α2 ,...,αk ) (B1 × B2 × · · · × Bk−1 × R) = ν(α1 ,α2 ,...,αk−1 ) (B1 × B2 × · · · × Bk−1 )

(3.1)

and for any permutation (i1 , i2 , . . . , ik ) of (1, 2, . . . , k), µ(αi1 ,αi2 ,...,αik ) (Bi1 × Bi2 × · · · × Bik ) = µ(α1 ,α2 ,...,αk ) (B1 × B2 × · · · × Bk ).

(3.2)

Then, there exists a probability space (Ω, F, P ) and a stochastic process XA ≡ {Xα : α ∈ A} on (Ω, F, P ) such that QA is the family of finite dimensional distributions associated with XA . Remark 6.3.1: Thus the above theorem says that given the family QA satisfying conditions (i) and (ii), there exists a real valued function on A × Ω such that for each ω, f (·, ω)
is a function on A and for each (α1 , α2 , . . . , αk ) ∈ Ak , the vector f (α1 , ω), f (α2 , ω), . . . , f (αk , ω) is a random vector with probability distribution ν(α1 ,α2 ,...,αk ) . This random function point of view is useful in dealing with functionals of the form . .}, then one M (ω) ≡ {sup f (α, ω) : α ∈ A}. For example, if A1 = {1, 2, .  n might consider functionals such as limn→∞ f (n, ω), limn→∞ n1 j=1 f (j, ω), ∞ j=1 f (j, ω), etc. Since the random functionals are not fully determined by f (α, ω) for finitely many α’s, it is not possible to compute probabilities of events defined in terms of these functionals from the knowledge of the finite dimensional distribution of (f (α1 , ω), . . . , f (αk , ω)) for a given (α1 , . . . , αk ), no matter how large k is. Kolmogorov’s consistency theorem allows one to compute these probabilities given all finite dimensional distributions (provided that the functionals satisfy appropriate measurability conditions). Given a probability measure µ on (R, B(R)), now consider the problem of constructing a probability space (Ω, F, P ) and a random variable X on it with distribution µ. A natural solution is to set the sample space Ω to be R, the σ-algebra F to be B(R), and the probability measure P to be µ and the random variable X to be the identity map X(ω) ≡ ω. Similarly, given a probability measure µ on (Rk , B(R)k ), one can set the sample space Ω to be Rk and the σ-algebra F to be B(Rk ) and the probability measure P to be µ and the random vector X to be the identity map. Arguing in the same fashion, given a family QA of finite dimensional distributions with index set A, to construct a stochastic process {Xα : α ∈ A} with index set A on some probability space (Ω, F, P ), it is natural to set the sample space Ω to be RA , the collection of all real valued functions on A, F to be a suitable σ-algebra that includes all finite dimensional events,

202

6. Probability Spaces

P to be an appropriate probability measure that yields QA , and X to be the identity map. These considerations lead to the following definitions. Definition 6.3.3: Let A be a nonempty set. Then RA ≡ {f | f : A → R}, the collection of all real valued functions on A. If A is a finite set {a1 , a2 , . . . , ak }, then RA can be identified with Rk by associating each f ∈ RA with the vector (f (a1 ), f (a2 ), . . . , f (ak )) in Rk . If A is a countably infinite set {a1 , a2 , a3 , . . .}, then RA can be similarly identified with R∞ , the set of all sequences {x1 , x2 , x3 , . . .} of real numbers. If A is the interval [0, 1], then RA is the collection of all real valued functions on [0, 1]. Definition 6.3.4: Let A be a nonempty set. A subset C ⊂ RA is called a finite dimensional cylinder set (fdcs) if there exists a finite subset A1 ⊂ A, say, A1 ≡ {α1 , α2 , . . . , αk }, 1 ≤ k < ∞ and a Borel set B in B(Rk ) such that C = {f : f ∈ RA and (f (α1 ), f (α2 ), . . . , f (αk )) ∈ B}. The set B is called a base for C. The collection of all finite dimensional cylinder sets will be denoted by C. The name cylinder is motivated by the following example: Example 6.3.2: Let A = {1, 2, 3} and C = {(x1 , x2 , x3 ) : x21 + x22 ≤ 1}. Then C is a cylinder (in the usual sense of the English word), but with infinite height and depth. According to Definition 6.3.4, C is also a cylinder in R3 with the unit circle in R2 as its base. Examples 6.3.3 and 6.3.4 below are examples of fdcs, whereas Example 6.3.5 is an example of a set that is not a fdcs. Example 6.3.3: Let A = {1, 2} and C = {(x1 , x2 ) : | sin 2πx1 | ≤

√1 }. 2

Example 6.3.4: Let A = {1, 2, 3, . . .} and C = {(x1 , x2 , x3 , . . .) : x230 10

−

x242 5

x217 4

+

≤ 10}.

Example 6.3.5: Let A = {1,2, 3, . . .} and D = {(x1 , x2 , x3 , . . .) : xj ∈ R n for all j ≥ 1 and limn→∞ n1 j=1 xj exists} is not a finite dimensional cylinder set (Problem 6.8). Proposition 6.3.2: Let A be a nonempty set and C be the collection of all finite dimensional cylinder sets in RA . Then C is an algebra. Proof: Let C1 , C2 ∈ C and let C1 C2


 = {f : f ∈ RA and f (α1 ), f (α2 ), . . . , f (αk ) ∈ B1 }
 = {f : f ∈ RA and f (β1 ), f (β2 ), . . . , f (βj ) ∈ B2 }

6.3 Kolmogorov’s consistency theorem

203

for some A1 = {α1 , α2 , . . . , αk } ⊂ A, A2 = {β1 , β2 , . . . , βj } ⊂ A, B1 ∈ B(Rk ), B2 ∈ B(Rj ), 1 ≤ k < ∞, 1 ≤ j < ∞. Let A3 = A1 ∪ A2 = {γ1 , γ2 , . . . , γ }, where without loss of generality (γ1 , γ2 , . . . , γk ) = (α1 , α2 , . . . , αk ) and (γ −j+1 , . . . , γ −1 , γ ) = (β1 , β2 , . . . , βj ). Then C1 and C2 may be expressed as
 ˜1 } C1 = {f : f ∈ RA and f (γ1 ), f (γ2 ), . . . , f (γ ) ∈ B
 A ˜2 } C2 = {f : f ∈ R and f (γ1 ), . . . , f (γ ) ∈ B ˜2 = R −j × B2 . Thus, C1 ∪ C2 = {f : f ∈ RA ˜1 = B1 × R −k and B where B ˜ ˜2 }. Since both B ˜1 and B ˜2 lie in B(R ), and (f (γ1 ), . . . , f (γ )) ∈ B1 ∪ B C1 ∪ C2 ∈ C.
 Next note that, C1c = {f : f ∈ RA and f (α1 ), . . . , f (αk ) ∈ B1c }. Since B1c ∈ B(Rk ), it follows that C1c ∈ C. Thus, C is an algebra. 2 Remark 6.3.2: If A is a finite nonempty set, the collection C is also a σ-algebra. Definition 6.3.5: Let A be a nonempty set. Let RA be the σ-algebra generated by the collection C. Then RA is called the product σ-algebra on RA . Remark 6.3.3: If A = {1, 2, 3, . . .} ≡ N and RN is identified with the set R∞ of all sequences of real numbers, then the product σ-algebra RN coincides with the Borel σ-algebra B(R∞ ) on R∞ under the metric 

∞ |xj − yj | 1 d(x, y) = (3.3) 2j 1 + |xj − yj | j=1 for x = (x1 , x2 , . . .), y = (y1 , y2 , . . .) in R∞ (Problem 6.9). Definition 6.3.6: Let A be a nonempty set. For any (α1 , α2 , . . . , αk ) ∈ Ak , 1 ≤ k < ∞, the projection map π(α1 ,...,αk ) from RA to Rk is defined by π(α1 ,α2 ,...,αk ) (f ) = (f (α1 ), f (α2 ), . . . , f (αk )).

(3.4)

In particular, for α ∈ A, πα (f ) = f (α)

(3.5)

is called a co-ordinate map. The projection map πA1 for any arbitrary subset A1 ⊂ A may be similarly defined. The next proposition follows from the definition of RA . Proposition 6.3.3: (i) For each α ∈ A, the map πα from RA to R is RA , B(R)-measurable. (ii) For any (α1 , α2 , . . . , αk ) ∈ Ak , 1 ≤ k < ∞, the map π(α1 ,α2 ,...,αk ) from RA to Rk is RA , B(Rk )-measurable.

204

6. Probability Spaces

Proof of Theorem 6.3.1: Let Ω = RA and F ≡ RA . Define a set function P on C by (3.6) P (C) = µ(α1 ,α2 ,...,αk ) (B) for a C in C with representation
 C = {ω : ω ∈ RA , ω(α1 ), ω(α2 ), . . . , ω(αk ) ∈ B}.

(3.7)

The main steps in the proof are (i) To show that P (C) as defined in (3.6) is independent of the representation (3.7) of C, and (ii) P (·) is countably additive on C. Next, by the Caratheodory extension theorem (Theorem 1.3.3), there exists a unique extension of P (also denoted by P ) to F such that (Ω, F, P ) is a probability space. Defining Xα (ω) ≡ πα (ω) = ω(α) for α in A yields a stochastic process {Xα : α ∈ A} on the probability space (RA , RA , P ) ≡ (Ω, F, P ) with the family QA as its set of finite dimensional distributions. Hence, it remains to establish (i) and (ii). Let C ∈ C admit two representations:
 C ≡ {ω : ω(α1 ), ω(α2 ), . . . , ω(αk ) ∈ B1 } ≡

π(α1 ,α2 ,...,αk ) (B1 )

≡


 {ω : ω(β1 ), ω(β2 ), . . . , ω(βj ) ∈ B2 }

≡

−1 (B2 ) π(β 1 ,β2 ,...,βj )

and C

for some A1 = {α1 , α2 , . . . , αk } ⊂ A, 1 ≤ k < ∞, and some A2 = {β1 , β2 , . . . , βj } ⊂ A, 1 ≤ j < ∞, B1 ∈ B(Rk ) and B2 ∈ B(Rj ). Let A3 = A1 ∪A2 = {γ1 , γ2 , . . . , γ } and w.l.o.g., let (γ1 , γ2 , . . . , γk ) = (α1 , α2 , . . . , αk ) and (γ −j+1 , γ −j+2 , γ −1 , γ ) = (β1 , β2 , . . . , βj ). Then C may be represented as C

˜1 ) = πγ−1 (B 1 ,γ2 ,...,γ ˜2 ) (B = π −1 γ1 ,γ2 ,...,γ

˜1 = B1 ×R −k and B ˜2 = R −j ×B2 . Note that (ω(γ1 ), . . . , ω(γ )) ∈ where B ˜1 iff ω ∈ C iff (ω(γ1 ), . . . , ω(γ )) ∈ B ˜2 and thus B ˜1 = B ˜2 . B

(3.8)

6.3 Kolmogorov’s consistency theorem

205

Next by the first consistency condition (3.1) and induction, ˜1 ) = ν(α ,α ,...,α ) (B1 ). ν(γ1 ,γ2 ,...,γ ) (B 1 2 k

(3.9)

Also by (3.2), for B2 of the form B21 × B22 × · · · × B2j with B2i ∈ B(R) for all 1 ≤ i ≤ j, ν(γ1 ,γ2 ,...,γ ) (R −j × B2 ) = ν(γ −j+1 ,...,γ ,γ1 ,γ2 ,...,γ −j ) (B2 × R −j ). Now note that (a) ν(γ1 ,γ2 ,...,γ ) (R −j × B) and ν(γ −j+1 ,...,γ ,γ1 ,γ2 ,...,γ −j ) (B × R −j ), considered as set functions defined for B ∈ B(Rj ), are probability measures on B(Rj ), (b) they coincide on the class Γ of sets of the form B = B21 × B22 × · · · × B2j with B2i ∈ B(R) for all i, and (c) the class Γ is a π-class and it generates B(Rj ). Hence, by the uniqueness theorem (Theorem 1.3.6), ν(γ1 ,γ2 ,...,γ ) (R −j × B) = ν(γ −j+1 ,...,γ ,γ1 ,γ2 ,...,γ −j ) (B × R −j )

(3.10)

for all B ∈ B(Rj ). Again by (3.1) and induction ν(γ −j+1 ,...,γ ,γ1 ,γ2 ,...,γ −j ) (B2 × R −j ) = ν(γ −j+1 ,...,γ ) (B2 ) = ν(β1 ,β2 ,...,βj ) (B2 ). ˜2 = R −j × B2 , by (3.10) and (3.11) Since B ˜2 ) = ν(β ,β ,...,β ) (B2 ). ν(γ1 ,γ2 ,...,γ ) (B 1 2 j Now from (3.8) and (3.9) it follows that ν(α1 ,...,αk ) (B1 )

˜1 ) ν(γ1 ,γ2 ,...,γ ) (B ˜2 ) = ν(γ1 ,γ2 ,...,γ ) (B =

= ν(β1 ,β2 ,...,βj ) (B2 ), thus establishing (i). To establish (ii), it needs to be shown that a

(ii) P (C1 ∪ C2 ) = P (C1 ) + P (C2 ) if C1 , C2 ∈ C and C1 ∩ C2 = ∅. b

(ii) Cn ∈ C, Cn ⊃ Cn+1 for all n,



n≥1 Cn

= ∅ ⇒ P (Cn ) ↓ 0.

(3.11)

206

6. Probability Spaces

−1 −1 Let C1 = π(α (B1 ) and C2 = π(β (B2 ) for B1 ∈ B(Rk ), B2 ∈ 1 ,...,αk ) 1 ,...,βj ) j B(R ), {α1 , . . . , αk } ⊂ A and {β1 , . . . , βj } ⊂ A, 1 ≤ j, k < ∞. As in the proof of Proposition 6.3.2, C1 and C2 may be represented as −1 ˜i ), i = 1, 2, (B Ci = π(γ 1 ,γ2 ,...,γ )

˜i ∈ B(R ). Since C1 and C2 are disjoint by hypothesis, it follows where B ˜1 and B ˜2 are disjoint. Also, since that B ˜i ), i = 1, 2, P (Ci ) = ν(γ1 ,γ2 ,...,γ ) (B and ν(γ1 ,...,γ ) (·) is a measure on B(R ), it follows that P (C1 ∪ C2 )

˜1 ∪ B ˜2 ) ν(γ1 ,...,γ ) (B ˜1 ) + ν(γ ,...,γ ) (B ˜2 ) = ν(γ1 ,...,γ ) (B 1 =

=

P (C1 ) + P (C2 ),

thus proving (ii)a . To prove (ii)b , note that for any sequence {Cn }n≥1 ⊂ C, there exists a countable set A1 = {α1 , α2 , . . . , αn , . . .}, an increasing sequence {kn }n≥1 of positive integers and a sequence of Borel sets {Bn }n≥1 such that Bn ∈ −1 B(Rkn ) and Cn = π(α (Bn ) for all n ∈ N. Now suppose that 1 ,α2 ,...,αkn ) {Cn }n≥1  is decreasing. It will be shown that if limn→∞ P (Cn ) = δ > 0, then n≥1 Cn = ∅. For each n, by the regularity of measures (Corollary 1.3.5), there exists a compact set Gn ⊂ Bn such that ν(α1 ,...,αkn ) (Bn \ Gn ) <

δ . 2n+1

−1 (Gn ). Then P (Cn \Dn ) < Let Dn = π(α 1 ,α2 ,...,αkn ) Then {Hn }n≥1 is decreasing and

P (Cn \ Hn )

= ≤


P Cn ∩ Hnc = P n

n

P (Cn \ Dj ) ≤

j=1

δ 2n+1 .

Let Hn =

n j=1

Dj .

 (Cn ∩

Djc )

j=1 n

P (Cj \ Dj )

j=1

(since {Cn }n≥1 is decreasing) ≤

n j=1

δ δ < . 2j+1 2

Since P (Cn ) ↓ δ > 0, Hn ⊂ Cn , and P (Cn \ Hn ) < 2δ , it follows that all n ≥ 1. This implies Hn = ∅ for each n. It will now P (Hn ) > 2δ for  be shown that n≥1 Hn = ∅. Let {ωn }n≥1 be a sequence of elements

6.3 Kolmogorov’s consistency theorem

207

from Ω = RA such that for each n, ωn ∈ Hn . Then, since {Hn }n≥1 is a decreasing sequence, for each 1 ≤ j < ∞, ωn ∈ Hj for n ≥ j. This implies that the vector (ωn (α1 ), ωn (α2 ), . . . , ωn (αkj )) ∈ Gj for all n ≥ j. Since G1 is compact, there exists a subsequence {n1i }i≥1 such that limi→∞ ωn1i (α1 ) = ω(α1 ) exists. Next, since G2 is compact, there exists a further sequence {n2i }i≥1 of {n1i }i≥1 such that limi→∞ ωn2i (α2 ) = ω(α2 ) exists. Proceeding this way and applying the usual ‘diagonal method,’ a subsequence {ni }i≥1 is obtained such that limi→∞ ωni (αj ) = ω(αj ) for all 1 ≤ j < ∞. Let ω(α) = 0 for α ∈ {α1 , α2 , . . .}. Since for each j, Gj iscompact, (ω(α  1 ), ω(α2 ), . . . , ω(α  kj )) ∈ Gj and hence ω ∈ Hj . Thus, ω ∈ j≥1 Hj ⊂ j≥1 Cj implying j≥1 Cj = ∅. The proof of the theorem is now complete. 2 When the index set A is countable and identified with the set N ≡ {1, 2, 3, . . .}, it is possible to give a simpler formulation of the consistency conditions. Theorem 6.3.4: Let {µn }n≥1 be a sequence of probability measures such that (i) for each n ∈ N, µn is a probability measure on (Rn , B(Rn )), (ii) for each n ∈ N, µn+1 (B × R) = µn (B) for all B ∈ B(Rn ). Then there exists a stochastic process {Xn : n ≥ 1} on a probability space (Ω, F, P ) with Ω = R∞ , F = B(R∞ ) such that for each n ≥ 1, the probability distribution P(X1 ,X2 ,...,Xn ) of the random vector (X1 , X2 , . . . , Xn ) is µn . Proof: For any {i1 , i2 , . . . , ik } ⊂ N, let j1 < j2 < · · · < jk be the increasing rearrangement of i1 , i2 , . . . , ik . Then there exists a permutation (r1 , r2 , . . . , rk ) of (1, 2, . . . , k) such that j1 = ir1 , j2 = ir2 , . . . , jk = irk . Now define (·) ν(j1 ,j2 ,...,jk ) (·) ≡ µjk πj−1 1 ,j2 ,...,jk where πj1 ,j2 ,...,jk (x1 , . . . , xjk ) = (xj1 , xj2 , . . . , xjk ) for all (x1 , x2 , . . . , xjk ) ∈ Rjk . Next define ν(i1 ,i2 ,...,ik ) (B1 × B2 × . . . × Bk ) ≡ ν(j1 ,j2 ,...,jk ) (Br1 × Br2 × . . . × Brk ) where Bi ∈ B(R) for all i, 1 ≤ i ≤ k. It can be verified that this family of finite dimensional distributions QN ≡ {ν(i1 ,i2 ,...,ik ) (·) : {i1 , i2 , . . . , ik } ⊂ N, 1 ≤ k < ∞}

(3.12)

satisfies the consistency conditions (3.1) and (3.2) of Theorem 6.3.1 and hence the assertion follows. 2

208

6. Probability Spaces

Example 6.3.6: (Sequence of independent random variables). Let {Fn }n≥1 be a sequence of cdfs on R. Consider the problem of constructing a sequence {Xn }n≥1 of random variables on a probability space (Ω, F, P ) such that (i) for each n ∈ N, Xn has cdf Fn and (ii) for any n ∈ N and any {i1 , i2 , . . . , in } ⊂ N, the random variables {Xi1 , Xi2 , . . . , Xin } are independent, i.e., P (Xi1 ≤ x1 , Xi2 ≤ x2 , . . . , Xin ≤ xn ) =

n +

Fij (xj )

(3.13)

j=1

for all x1 , x2 , . . . , xn in R. This problem can be solved by using Theorem 6.3.4. Let µn be the Lebesgue-Stieltjes probability measure on (Rn , B(Rn )) corresponding to the distribution function F1,2,...,n (x1 , x2 , . . . , xn ) ≡

n +

F (xj ),

x1 , . . . , xn ∈ R.

j=1

It is easy to verify that the family {µn : n ≥ 1} satisfies (i) and (ii) of Theorem 6.3.4. Hence, there exist a probability measure P on the sequence space Ω ≡ R∞ equipped with σ-algebra F ≡ B(R∞ ) and random variables Xn (ω) ≡ πn (ω) ≡ ω(n), for ω = (ω(1), ω(2), . . .) in R∞ , n ≥ 1, such that (3.13) holds. Example 6.3.7: (Family of independent random variables). Given a family {Fα : α ∈ A} of cdfs on R for some index set A, a construction similar to Example 6.3.6, but using Theorem 6.3.1 yields the existence of a real valued stochastic process {Xα : α ∈ A} such that for any {α1 , α2 , . . . , αn } ⊂ A, 1 ≤ n < ∞, the random variables {Xα1 , Xα2 , . . . , Xαn } are independent, i.e., (3.13) holds. Example 6.3.8: (Markov chains). Let Q = ((qij )) be a k