MARKOV PROCESSES AND
POTENTIAL THEORY
Pure and Applied Mathematics A Series of Monographs and Textbooks Edited by Paul A. Smith and Samuel Eilenberg
Columbia University, New York
1 : ARNOLD SOMMERFELD. Partial Differential Equations in Physics. 1949 (Lectures on Theoretical Physics, Volume V I ) 2 : REINHOLDBAER.Linear Algebra and Projective Geometry. 1952 BUSEMANN A N D PAUL KELLY.Projective Geometry and Projective 3 : HERBERT Metrics. 1953 BERGMAN A N D M. SCHIFFER.Kernel Functions and Elliptic Differential 4 : STEFAN Equations in Mathematical Physics. 1953 5: RALPHPHILIPBOAS,JR. Entire Functions. 1954 BUSEMANN. The Geometry of Geodesics. 1955 6 : HERBERT Fundamental Concepts of Algebra. 1956 7 : CLAUDECHEVALLEY. HIJ. Homotopy Theory. 1959 8 : SZE-TSEN Solution of Equations and Systems of Equations. Second 9 : A. M. OSTROWSKI. Edition. 1966 ~. of Modern Analysis. 1960 10: J. D I E U D O N NFoundations 11 : S. I. GOLDBERG. Curvature and Homology. 1962 HELGASON. Differential Geometry and Symmetric Spaces. 1962 12 : SIGURDUR Introduction to the Theory of Integration. 1963 13: T. H. HILDEBRANDT. ABHYANKAR. Local Analytic Geometry. 1964 14 : SHREERAM 15 : RICHARDL. BISHOPA N D RICHARD J. CRITTENDEN. Geometry of Manifolds. 1964 A. GAAL.Point Set Topology. 1964 16: STEVEN Theory of Categories. 1965 17: BARRYMITCHELL. 18: ANTHONYP. MORSE.A Theory of Sets. 1965
Pure and Applied Mathematics A Series of Monographs and Textbooks
19 : GUSTAVE CHOQUET.Topology. 1966 20: Z. I. BOREVICH A N D I. R. SHAFAREVICH. Number Theory. 1966 21 : J o s i LUIS MASSERA A N D J U A N JORGE SCHAFFER. Linear Differential Equations and Function Spaces. 1966 22 : RICHARD D. SCHAFER. An Introduction to Nonassociative Algebras. 1966 23: MARTINEICHLER.Introduction to the Theory of Algebraic Numbers and Functions. 1966 24 : SHREERAM ABHYANKAR. Resolution of Singularities of Embedded Algebraic Surfaces. 1966 25 : FRANCOIS TREVES. Topological Vector Spaces, Distributions, and Kernels. 1967 D. LAXand RALPHS. PHILLIPS. Scattering Theory. 1967 26: PETER 27: OYSTEINORE.The Four Color Problem. 1967 28 : MAURICEHEINS.Complex Function Theory. 1968 29 : R. M. BLUMENTHAL A N D R. K. GETOOR. Markov Processes and Potential Theory. 1968
In preparation
HANSFREUDENTHAL A N D H. DE VRIES.Linear Lie Groups.
This Page Intentionally Left Blank
MARKOV PROCESSES AND
POTENTIAL THEORY R. M . BLUMENTHAL DEPARTMENT OF MATHEMATICS UNIVERSITY OF WASHINGTON SEATTLE, WASHINGTON
and R. K . GETOOR DEPARTMENT OF MATHEMATICS UNIVERSITY OF CALIFORNIA AT S A N DIEGO LA JOLLA, CALIFORNIA
1968
ACADEMIC PRESS New York and London
COPYRIGHT 0 1968, BY ACADEMIC PRESS INC. ALL RIGHTS RESERVED. NO PART OF THIS BOOK MAY BE REPRODUCED IN ANY FORM, BY PHOTOSTAT, MICROFILM, OR ANY OTHER MEANS, WITHOUT WRITTEN PERMISSION FROM THE PUBLISHERS.
ACADEMIC PRESS INC. I 1 1 Fifth Avenue, New York 10003
United Kingdom Edition published by ACADEMIC PRESS INC. (LONDON) LTD. Berkley Square House, London W.l
LIBRARY OF CONGRESS CATALOG CARDNUMBER: 68-18659
PRINTED IN T H E UNIlED STATES O F AMERICA
PREFACE The study of the relationship between Markov processes and potential theory began in the early 1930's. Initially attention was directed toward analytic aspects such as differential equations satisfied by the transition probabilities and questions concerning existence and uniqueness of solutions of these equations. Then, through the work of Doob, Feller, Kac, and Kakutani, among others, it emerged that this relationship has a deep probabilistic basis. An extensive study of the relationship between Brownian motion and classical potential theory was given by Doob [4] in 1954. This paper marks the beginning of the modern era in the subject. Then in 1956 and 1957 Hunt (2-41 laid bare a large part of the entire subject: he gave a rather general definition of " potential theory " and associated with each such theory a Markov process in terms of which potential-theoretic objects and operations (superharmonic functions, balayage, etc.) have probabilistic interpretations. He used this relationship to generalize and reinterpret (often in a more illuminating form) many facts from classical potential theory. The purpose of this book is to collect within one cover most of the contents of Hunt's fundamental papers, portions of the theory of additive functionals, and some closely related matters. We hope that we have presented the material in such a way that it is accessible to a diligent advanced graduate student. The reader is assumed to be familiar with general measure theory at the level of a basic graduate course. Aside from this, the book is very nearly self-contained. Still it is unrealistic to recommend it to a reader who does not have some background in probability theory or at least a bit of intuition concerning random variables, independence, conditioning, and the like. Unfortunately the material is somewhat top-heavy; that is, one must start with some extensive (and perhaps dull) measure-theoretic preliminaries. Frequently the need for a particular level of generality (e.g., taking as sample space a general measure space rather than some specific function space) is not immediately apparent. However, we have tried to select the lowest vii
viii
PREFACE
level of abstraction that allows one firstly to operate freely and rigorously in subsequent developments, and secondly to avoid unnecessary restrictions on the transformations of processes one is willing to consider. Most of the measure-theoretic preliminaries are presented in Chapter I, while the more interesting developments begin in Chapter 11; we ask the reader to go at least this far before passing judgement on the entire presentation. The reader especially interested in potential theory can go directly from the end of Chapter I1 to Section 6 of Chapter 111 and from there to Section 1 of Chapter V and Sections 1, 2, and 4 of Chapter VI. These sections are more or less independent of the rest of the book, and they contain the basic potential theoretic facts. Most sections are followed by exercises designed to further the theory as well as give practice. We do not hesitate to use the results of an exercise in subsequent sections. Historical references and credits are collected in the “Notes and Comments.” We hope that we always have told the truth, but realize that it is seldom the whole truth. It is not our intention to give anyone less than his full measure of credit and we apologize in advance to anyone who may feel slighted. This book does not cover all of the theory of Markov processes or even all of probabilistic potential theory. We have not said anything about infinitesimal generators (which, however, receive excellent treatment in Dynkin [2] and in Ito-McKean [l]). Neither have we said anything about the problem of constructing a Markov semigroup starting from a potential operator (or a class of excessive functions) nor about the boundary theory for the representation of harmonic functions-to mention only a few omissions. Finally we have the pleasure of acknowledging a few of our debts. We benefited greatly from the interest which P. A. Meyer took in this project; he read most of the manuscript and set us straight on a number of important matters. Harry Dym and Frank Knight read portions of the manuscript and made useful comments. We received financial support from the National Science Foundation and from the Air Force Office of Scientific Research during part of the writing. We are particularly indebted to our typists Donna Thompson and Olive Lee for the capable manner in which they handled the job. They lightened considerably our work in preparing the manuscript. February, 1968
R. M. BLUMENTHAL R. K. GETOOR
CONTENTS vii
Preface
Chapter 0.
PRELIMINARIES 1. Notation 2. The Monotone Class Theorem 3. Topological Spaces
Chapter I.
1 5 8
MARKOV PROCESSES 1. 2. 3. 4. 5.
6. 7. 8. 9. 10. 11.
12.
General Definitions Transition Functions General Definitions Continued Equivalent Processes The Measures PN Stopping Times Stopping Times for Markov Processes The Strong Markov Property Standard Processes Measurability of Hitting Times Further Properties of Hitting Times Regular Step Processes
11 14 20 24 25 30 36 37 44 52 61 63
Chapter 11. EXCESSIVE F U N C T I O N S 1. Introduction
2. Excessive Functions 3. Exceptional Sets 4. The Fine Topology 5. Alternative Characterization of Excessive Functions
69 72 79 84 89
ix
CONTENTS
X
Chapter 111. MULTIPLICATIVE FUNCTIONALS A N D SUBPROCESSES 1. Multiplicative Functionals
91
2. Subordinate Semigroups 3. Subprocesses
101 105
4. Resolvents and Strong Multiplicative Functionals
113 125 136
5. Excessive Functions 6. A Theorem of Hunt
Chapter IV. ADDITIVE FUNCTIONALS A N D THEIR POTENTIALS 2. Potentials of Additive Functionals 3. Potentials of Continuous Additive Functionals
148 154 160
4. Potentials of Natural Additive Functionals 5. Classification of Excessive Functions
170 186
1. Additive Functionals
Chapter V. FURTHER PROPERTIES OF CONTINUOUS ADDITIVE FUNCTIONALS 1. Reference Measures
2. Continuous Additive Functionals 3. Fine Supports and Local Times 4. Balayage of Additive Functionals 5. Processes with Identical Hitting Distributions
196 205 212 228 233
Chapter VI. D U A L PROCESSES A N D POTENTIAL THEORY 1. Dual Processes 2. Potentials of Measures 3. Potentials of Additive Functionals 4. Capacity and Related Topics
252 265 279 283
Notes and Comments
295
Bibliography
305
Index of Notation
31 1
Subject Index
312
0
PRELIMINARIES
This chapter contains preliminary material that will be used in the main text. Section 1 establishes the basic notation that will be used without further explanation in the sequel. As a result this section should be read before the main text is attempted. The remaining sections of this chapter need not be read by everyone. It will suffice to refer to them whenever necessary. Section 2 contains the so-called “monotone class ” theorems, which are perhaps the most useful results from set theory for us. They will be used extensively in the sequel. In Section 2 we also give a number of typical applications of these theorems in order to illustrate their use to readers who are unfamiliar with this type of argument. Often we will omit such arguments in the body of the text. Finally in Section 3 we collect some results connecting topology and measure theory. 1. Notation
Let R denote the real numbers, Z the integers, and B the extended real numbers { - co} u R u { 0 0 ) with the usual topology. If a and b are in R, then a v b = max(a, b), a A b = min(a, b), and a+ = a v 0. If E is a set, a numerical (real-valued) function on E is a function from E to R(R). For any numerical function f on E, llfll = supxEEIf(x)l. A a-algebra d on E is a collection of subsets of E such that E E d and d is closed under complements and countable unions. We will use the usual notation “ n ” and “ u ” to denote set intersection and union. If A c E, then A‘ will denote the complement of A. Finally A - B = A n B‘ and A A B = ( A - B) u (B - A) will denote set difference and symmetric difference, respectively. If A is any collection of subsets of E, a(&) denotes the a-algebra generated by A, that is, the smallest a-algebra on E containing A. Clearly a(A) is the intersection of all a-algebras
+
1
2
0. PRELIMINARIES
on E containing A. A pair (E, 8)consisting of a set E and a a-algebra on E is called a measurable space. Let (E, 8 ) and (F, 9) be measurable spaces. A function f : E + F will be called measurable relative to 6 and 9 provided that for every B E 9, f E 8. Following Chung and Doob [ 11 we will write simply f E 8/9if f is measurable relative to 8 and 9. In the case where F = R and 9 is the class of all Bore1 subsets of R we write simplyf E 6 for a numerical function f E 8/9.If A is a subset of E we denote the indicator function of A by IA, so that for example, the statements 1, E b and A E d have the same meaning. We denote the collection of all real-valued C measurable functions by r 6 . If X is any collection of numerical functions, then X + denotes the set of nonnegative functions in &’ and b X denotes the set of bounded functions in 2.I f f is a numerical function on E, then f = 0, f 2 0, and f > 0 mean f ( x ) = 0 for all x E E, f ( x ) 2 0 for all x E E, and f ( x ) > 0 for all x E E, respectively. If E, F, and G are three sets and iff : E + F and g : F + G, then we write g of for the composition map from E to G, g o f ( x ) = g1f(x)]. In particular if (E, 8),(F, 9), and (G, ’3) are measurable spaces and iff E 6 / 9 and g E S/’3,then g of E 8/’3. Let Q be a set and let (Ei, 8i)iE, be a collection of measurable spaces indexed by some set I. For each i letfi be a map from R to E,; then a(fi, g i ; i E I) denotes the a-algebra on Q generated by the sets {&‘(Ai): A i E Ci} as i ranges over I. Clearly this is the smallest cr-algebra on R relative to which all the are measurable. We will write merely a(fi ; i E I ) when the Ei and tZi are understood clearly from the context. By a measure on (E, 8) we always will mean a nonnegative measure. A measure space is a triple (E, 8,p) where (E, 8)is a measurable space and p is a measure on (E, 8).We recall that p is finite if p ( E ) < 00 and p is a-finite if there exists a sequence (En} of elements of 6 such that E = U E , and p(En)< 00 for all n. We assume the reader is familiar with the theory of integration on a measure space, Often we write p ( f ) instead of 1f dp for those f E 8 for which the integral exists. A measure space (E, 8, p ) is called complete if B E C and p ( B ) = 0 implies that every subset of B is in 8. Given any measure space (E, 6, p ) we define 8’ to consist of all sets B c E for which there exist B1,B, E 8 such that B, c B c B2 and p(B2 - Bl).= 0. It is evident that b’ is a a-algebra and that the measure p has a unique extension to 8’. We denote the extension again by p. b’ is called the completion of d with respect to p. One verifies easily that d = 8’’if and only if (E, I ,p ) is where the intersection is over all finite p on complete. Define &* = (E, 8). It is easy to see that b* is a a-algebra and every finite measure on C has a unique extension to 8*.8* is called the a-algebra of universally measurable sets over ( E , 8).If p is a measure on (E, 8)we say that p does not charge a set A provided A E 8’’and p ( A ) = 0.
n,&”
3
1. NOTATION
Let (E, 8,p) be a measure space and let (F,9) be a measurable space. E d / 9 , then the formula v(A) = pLf-'(A)] for A E 9defines a measure v on (F, 9).We will write v = pf-'. If h E 9,then h f E d and
Iff
0
in the sense that if either integral exists then so does the other and they are equal. If (Ei, B i ) i E I is a family of measurable spaces, then the product I-Iis,(Ei,Bi)= (E, 4)is a measurable space where E is the ordinary Cartesian product of the Ei's and d is the a-algebra on E generated by sets of the form A, with A , E 8 , for all i and only finitely many A, different from E , . We will write d = d i and call d the product of the a-algebras B i . We assume the reader to be familiar with the theory of product measures and the various forms of the Fubini theorem. A probability space (Q, 9, P) is a measure space with P(Q) = 1. If (E, 8 ) is a measurable space then a map X : Q + E is called an (E, 8)random variable provided X E 918.The distribution of X is the probability measure px = PX-' on (E, 8 ) and we often write P ( X e B) for pcx(B).A numerical random variable is by definition an element of 9.If Xis a numerical random variable, then the expectation of X , E ( X ) , is defined by E ( X ) = X d P provided the integral in question exists. If Xis an (E, 4)random variable and f E b8, then (1.1) becomes
ni
ni
5
and this extends to any f E B for which either side exists, in particular to E 8 , . Often we will establish formulas such as (1.2) forfE bb and then use these formulas for nonnegative f E B without special mention. Obviously this is permissible in view of the monotone convergence theorem. Frequently when X is a function in 9 and A is a set in 9we write E ( X ; A) in place of I, X,dP. We assume that the reader is familiar with the concepts of independence and conditional expectation as set forth, for example, in Lokve [l] or Neveu [I]. In particular if 9 is a a-algebra contained in 9and X E 9is P-integrable we use E(X I Y) to denote any function f such that: (1) f E 8 and (2) E(fi A) = E ( X ; A) for every A E 8. E ( X (9)is called the conditional expectation of X given 8.When X = I", A E 9,we write P(A I 8)in place of E(Z, I 8).Of course conditions (1) and (2) determine f up to a set in Y of P measure 0. On the other hand, when we write f = E(XI Y) we mean simply that f is a function satisfying ( I ) and (2); consequently, relationships involving conditional expectations are not enhanced by the qualifying phrase " almost everywhere relative
f
4
0. PRELIMINARIES
to P." When Y is of the form a(fi, r f i ; i E I ) we sometimes write E(X I fi ;i E I) instead of E(X I Y). Finally we summarize the basic results on supermartingales that will be needed in later chapters. Let (Q, 9, P) be a probability space and let T be a subset of the extended real numbers. For each t E T, let X , be a numerical P) and let 9, be a sub-a-algebra of 9 such that random variable over (Q, 9, 9, c FSwhenever t I s, t and s in T. Then { X , , 9, ; t E T} is a supermartingale (over (Q, 9, P))provided
(i) X,E F , , ~ E T , (1.3)
(ii) E(lX,l) < co, (iii) E { X , I 9,} IX , ;
t E T, s, t E T, s > t .
If the a-algebras 9, are omitted from the definition, then it is understood that 9, = a(X, ; s E T, s I t). We say that { X , , 9,} is a submurtingale provided { - X , , F,} is a supermartingale, and that { X , , 9,} is a martingale provided that it is both a supermartingale and a submartingale. If Z is an arbitrary index set, then a family { Yi; i E Z} of numerical random variables over (Q, 9, P) is said to be uniformly integrable provided that limn E{ I YiI ; I Yil > n } = 0 uniformly on I. We refer the reader to Meyer [l] or Neveu [ l ] for the properties of uniformly integrable families. The next two theorems contain the basic results which will be needed in later chapters. Proofs may be found in Doob [l], Lokve [l], Meyer [l], or Neveu [l]. (1.4) THEOREM.LetN = (1, 2,. . .}andN = N u {GO}.Let {A',,, 9 , , ; n ~ N } be a supermartingale such that sup,, E(lX,,l) < GO. Then X , = limn X,,exists and is finite almost surely. Moreover if {A',,; n E N } is uniformly integrable then {X,,,9,,; n E N} is a supermartingale where Srn = o(US,,). Note that if X,, 2 0 for all n, then (1.3) implies that E(lX,,l) = E(X,,) 5 E(X,), and so a nonnegative supermartingale {X,, , 9,,;n E N} always satisfies the hypothesis of Theorem 1.4.
(1.5) THEOREM: (a) If {X,, 9,; t E (0,GO)}is a supermartingale such that t + X,(w) is right continuous on (0,00) for almost all w , then t + X,(w) has left-hand limits on (0, GO)for almost all w , that is, for almost all w , limsT,XJw) exists (and is finite) for all t E (0, GO). (b) Let { X , , 9,; t E [0, GO)}be a supermartingale, and let D be a countX,(w) and able dense subset of [0, 001. Then for almost all w , limsf,,sED lims~r,soD X,(w) exist (and are finite) for all t E [0, a).
2.
5
THE MONOTONE CLASS THEOREM
The following fact will also be needed in Chapter I. We leave its proof t o the reader as an exercise. (1.6) PROPOSITION. Let { X , , 9,;f E [0, co)} be a nonnegative supermartingale and let D be a countable dense subset of [0, 00). Then for each t E [0, co), P { X , > 0, inf,,,,,,, X , = 0) = 0.
2. The Monotone Class Theorem
In this section we will discuss the monotone class theorem in the form we find most useful for application in probability theory. (2.1) DEFINITION. Let R be a set and Y a collection of subsets of R; then (i) Y is a n-system (on R) if Y is closed under finite intersections; (ii) Y is a d-system (on R) if (a) RE^, (b) if A, B E Y , A c B, then B - A €9, (c) if {A,} is an increasing sequence of elements of 9, then U A , E Y .
Obviously any 0-algebra is both a d-system and n-system. Conversely, one easily verifies that if Y is both a d-system and a n-system, then Y is a a-algebra. If A is any collection of subsets of R, then we define d ( M )to be the smallest d-system on R containing A. The existence of d ( M )is clear since the intersection of an arbitrary number of d-systems is again a d-system. We will say that d ( A ) is the d-system generated by A. We come now to the main result on d-systems. (2.2)
THEOREM.Let Y be a n-system on R; then d ( 9 ) = o(Y).
Proof. Since d ( Y ) c ~(9’) it suffices to show that d(Y) is a 0-algebra, and for this it suffices to show that d ( Y ) is a n-system. To this end define g1= { B E d ( 9 ) : B n A E d ( Y ) for all A
E
9).
The reader will easily verify that 9, is a d-system and that g13 Y since Y is a n-system. Hence 3 d ( 9 ) . But by definition c d ( 9 ) and so g1= d(Y). Next define g2= { B E d ( Y ) : B n A E d ( Y ) for all A
E d(Y)}.
Again one shows without difficulty that Q2 is a d-system. If A E 9, then B n A E d(Y) for all B E Q1 = d(Y), and consequently Y c g2. Hence
6
0. PRELIMINARIES
d ( Y ) = 9,,and this is just the statement that d ( Y ) is closed under finite intersections. Thus Theorem 2.2 is established. We next give a version of Theorem 2.2 that deals with functions rather than sets.
(2.3) THEOREM.Let R be a set and Y a n-system on R. Let &’ be a vector space of real-valued functions on R satisfying: (i) l ~ I a n d I , ~ & ‘ f o r a l l A € Y ; (ii) if {f,}is an increasing sequence of nonnegative functions in I such that f = sup, f.is finite (bounded), then f~ 2. Under these assumptions &‘ contains all real-valued (bounded) functions on R that are a(Y) measurable. Proof. Let 93 = {A: I , E S}. According to Assumption (i) R E 9 and Y c 9.If A, c A, are in 9,then =Z , - Z,, E X since I is a vector space, and so A, - A, E 9. Finally if {A,} is an increasing sequence of sets in 9,then ZuA, = sup I,, which is in Iby (ii). Thus 93 is a d-system on R containing Y and hence 9 = a(Y) by Theorem 2.2. I f f € ~ ( is 9real-valued ) then f =f -f - with f and f - being nonnegative, real-valued, and a(Y) measurable. On the other hand, i f f € o ( Y ) is nonnegative thenfis an increasing limit of simple functions f, = 4ZA; with each A; E ~(9’). Hence eachf, E &’ and using (ii) the result is now immediate. +
+
Theorems 2.2 and 2.3 will be used in the following form: Let R be a set and ( E , , b,),,, be a family of measurable spaces indexed by an arbitrary set I. For each i E Z, let 9, be a n-system generating b iand letf, be a map from R to E , . Using this setup we state the following two propositions. (2.4) PROPOSITION. Let Y consist of all sets of the form n l E J f i - l ( A i ) where A, E 9, for i E J and J ranges over all finite subsets of I . Then 9 is a n-system on R and a(9) = a(fi; i E I ) .
(2.5) PROPOSITION. Let &’ be a vector space of real-valued functions on R such that (i) 1 ~ 8 ; (ii) if {A,} is an increasing sequence of elements of 2 , such that h = sup h, is finite (bounded), then h ES; (iii) &’ contains all products of the form Hie,Z A , o f i where J is a finite subset of Z and A, E 9, for i E J.
2.
THE MONOTONE CLASS THEOREM
7
Under these assumptions 2 contains all real-valued (bounded) functions in cr(fi; i E I ) . Proposition 2.4 is immediate and (2.5) follows from (2.4) and (2.3). In the sequel any of the results, (2.2)-(2.5) will be referred to as the Monotone Class Theorem (MCT). We now give some applications of these results. We begin with the following uniqueness theorem for finite measures. (See Exercise 3.2 for an extension to a-finite measures.) PROPOSITION.Let p and v be finite measures on a measurable space (E. 8).Let 9’c 6 be a n-system containing E and generating 8. If p and v agree on 9, then they agree on 6 . (2.6)
This is an immediate consequence of (2.2) since {A E 6 :p ( A ) = v(A)} is clearly a d-system containing 9. We close this section with two useful facts. PROPOSITION. Let R be a set and (E, 6 ) a measurable space. Let cp : R -,R is a(f) measurable if and only if there exists h E S such that cp = h of. (2.7)
f :R -+ E; then
Proof. If cp = h 0 f with h E 6 then obviously cp E cr(f). To prove the converse let .$ be all real functions on i2 of the form h 0 f with h E r 8 . Clearly I is a vector space containing the constants. Suppose {h, of} is an increasing sequencein.$+ such that JI = sup(h, 0f)isfinite. LetA = { X E E: sup h,(x) < a}; then A E B and f ( i 2 ) c A. Define h = sup h, on A and h = 0 on A‘. Then h E r 6 and = h of. If C ~ a ( f then ) C = f - ’ ( A ) for some A € 8 and so I, = IA f E.$. Thus 2 contains all the real-valued functions in a(f) according to (2.5). If cp E a(f) is numerical valued then cp’ = arctan cp E a(f) and is real valued. So cp’ = h’offor some h’ E 8.Clearly we may assume that h’(E) c [ - zr/2,7r/2] because $takes values only in this interval. Then cp = h 0 f with h = tan h’ and of course h E 8. 0
We should point out that if cp is real valued (bounded) we may assume that h is real valued (bounded). PROPOSITION.Let R be a set, {Yi; i E I} a family of a-algebras on R, and Y = a(%,; i E I}.Then for every A E Y there is a countable subset J c I such that A E a(%i; i E J ) . (2.8)
Proof. The class of sets A having this property is clearly a cr-algebra and it contains each Yi .
8
0. PRELIMINARIES
The following corollary is an easy consequence of (2.7) and (2.8). We leave the details to the reader as an exercise,
(2.9) COROLLARY.Let R be a set and let (Ei, bi)i,rbe a family of measurable spaces. For each i let fi : iZ + Ei and set 9 = o ( f i ;i E I ) . If rp : R 3 R is in 9,then 9 depends on at most countably many coordinates in the sense that there exist a countable subset J of Z and a function h : Die,Ei 3 R which is &i measurable and such that cp = h f, where f, is the map 0 3 ( f i ( O ) ) i e , from to n i E J Ei *
nie,
0
3. Topological Spaces
A topological space E will always mean a Huusdorflspace unless explicitly stated otherwise. If E is a topological space the Borel sets of E is the smallest a-algebra containing the open sets of E. We write W ( E )(or just W) for the Borel sets of E. If E is locally compact, then C(E), C,(E), C,(E) denote, respectively, the bounded real-valued continuous functions on E, the realvalued continuous functions vanishing at m, the real-valued continuous functions with compact support. Clearly C , c Co c C and C , and C are Banach spaces under the supremum norm 11 .I[. Moreover C, is dense in C , . The Buire sets of E is the a-algebra a(C,) = W,(E). Obviously W,c W. In most of our work we will be concerned with locally compact spaces with a countable base (LCCB). We summarize some of the relevant properties of such a space. (3.1) Let E be an LCCB; then (i) W = g o ; (ii) E is metrizable, and one can choose a metric d compatible with the topology, such that (E, d) is a separable complete metric space in which every closed and d bounded set is compact; (iii) if I is a countable base for the topology of E, then a(%) = W ; (iv) Co is separable (as a Banach space).
Let E be an LCCB. A measure p on (E, W(E))is called a Radon measure on E if p ( K ) < co for all compact K . In particular a Radon measure on an LCCB is a-finite. If L is a nonnegative linear functional on C,, then there exists a unique Radon measure p on E such that L ( f ) = p ( f ) for all f~ C , . Every Radon measure is regular; that is, if B E W then p ( B ) = sup{p(K): K c ByK compact} = inf{p(G): G 3 B, G open}.
9
EXERCISES
Let M+ be the set of all Radon measures p on E such that p ( E ) c co and let M = M + - M, be the vector space of all real-valued set functions I on E which have the form I = p - v with p, v E M+ . For 1 E M we define 11111 = sup I1(Ai)l where the supremum is taken over all finite partitions { A l , . . . , A,,} of E into Bore1 sets. Under this norm M is a Banach space and, in fact, M = C z . We will call the topology induced on M by C, (the weak*topology on M) the weak (or vague) topology on M.
1
Exercises
(3.2) Let p and v be two u-finite measures on the measurable space (E, 8). Let Y c d be a n-system generating d and suppose Y contains a sequence {B,,} with UB,, = E. If p and v agree on 9’ and p(B,,) = v(B,,) < co for all n, then p and v agree on 8. (3.3) Let ( E i , &‘J, i = 1, 2, be measurable spaces. If f e
then
f E s:/s:. (3.4) Let (E, 8,p) be a measure space. Show that f E 8’ if and only if there existsf, and fie d such thatf, If I f2 and p({fi f2}) = 0.
-=
(3.5) Prove that W ( E ) = Wo(E)when E is LCCB. (3.6) Let R be a set and E an LCCB. Let {Xi; i E I } be a family of functions from R to E. Let 2 be a vector space of real-valued functions on R satisfying: (i) 1 E &‘; (ii) If {F,,}is an increasing sequence in &’+ such that F = sup F,, is bounded, then F E 2 ; (iii) % . ‘ contains all functions of the form X i where J is a finite subset of I and f i E C, for i E J. Show that &’ contains all bounded functions in u(Xi;i E I).
ni,f,
0
(3.7) Let (R, 9)be a measurable space and E be a metric space. Let {A} be a sequence in .F/B(E).Prove the following statements: (i) iff,(w) +f ( w ) for all o,thenfe .F/W(E). (ii) If E is complete and x, a point in E, then f(o)= limfn(w)
when the limit exists
n
= xo
when the limit does not exist
is in F/&?(E). The completeness hypothesis cannot be entirely eliminated.
10
0. PRELIMINARIES
What happens if W is replaced by W*(the universally measurable sets over (E, W)) throughout ?
(3.8) Let (E, 8 ) be a measurable space and let T be a positive linear map from b b to bb such that whenever {f,}is a decreasing sequence in bb with inf f.= 0 then inf Tf, = 0. Show that there exists a function P(x, A) defined for x E E and A E 6 such that (i) A -+ P(x, A) is a finite measure on b for each x E E; (ii) x -+ P(x, A ) is in bb for each A E 8 ; (iii) T f ( x ) = J P(x, d y ) f ( y )for eachfe b b and x E E. [Hint: let P(x, A) = TZ,(x).]
I MARKOV PROCESSES
1. General Definitions
In this section we will give the general definition of a Markov process following more or less the definition in Doob [I]. However this definition is too general for the purposes of this book, and so very shortly we will restrict ourselves to a more tractable class of processes. Let (Q, 9, P) be a probability space, (E, 8) a measurable space, and T an arbitrary index set. Then a stochastic process with values in ( E , 8 ) andparameter set T (over (Q, 9,P))is a collection X = { X , : t E T} of maps X , from Q to E each of which belongs to 9/8. (See Section 1 of Chapter 0 for notation.) For each w E Q the map t -,X,(w) from T to E is called the path or trajectory corresponding to w. It will sometimes be convenient to write X ( t , w ) for X , ( o ) and also X ( t ) for X , . The measure P clearly plays no role in this definition and so one can define in exactly the same way a stochastic This will sometimes be useful. process over a measurable space (Q, 9). From now on we assume that T is a subset of R (the extended real numt E T} is an increasing family of sub-a-algebras of 9, that is, bers). If {9,; 9,c PSwhenever t and s are in T and t c s, we say that the process X is adapted to {9, provided } X , E F,/& for each t in T. In particular Xis always adapted to the family of sub-a-algebras 9, = a(X,: s I t). Of course in expressions such as this the letters t and s refer to elements of T. (1.1) DEFINITION. Let X = { X , : t E T} be a stochastic process with values in (E, a). One says that X is a Markov process with respect to an increasing t E T} of sub-a-algebras of 9provided (1) X is adapted to family {9*; {9, and } (2) for each t E T the a-algebras 9,and a(Xs:s 2 t) are conditionally independent given X , , that is, 11
12 (1.2)
1. MARKOV
PROCESSES
P ( A n B I XI) = P ( A I Xr)P(B I X t )
whenever A is in F, and B is in o(Xs; s 2 t ) . One says that X is a Markov process (without specifying the a-algebras g1) if it is a Markov process with respect to the family of a-algebras Y, = a(Xs;s It ) . Notice that if X is a Markov process with respect to some family {9,} of a-algebras, then it is also a Markov process with respect to the family {Yr = a(X,; s It ) } . If X is a Markov process with respect to Y, = a(Xs;s It), then the a-algebras a(X,; s It) and o(Xs;s 2 t ) enter symmetrically in the definition. Consequently the Markov property is preserved if one reverses the order in T. The intuitive meaning of this condition should be clear: namely, given the present, X,,the past, o(X,; s It ) , and the future, a(X,; s 2 t) are independent. In discussing conditional expectations the following situation will often arise. Let (a,9,P) be a probability space, and Yl and Yz a-algebras contained in f with Y1 c Y2. In this situation i f f € F, then the equality
should be interpreted as meaning that there exists a function g E Y1 such that E ( f ; A) = E(g; A) for all A E Yz . We now give some equivalent formulations of Definition (1.1). (1.3) THEOREM: (a) Let X = { X , ; t E T} be a stochastic process adapted to the family {Ft}. Then the following statements are equivalent: (i) X is a Markov process with respect t o { P f;} (ii) for each t E T and Y in bo(X,: s 2 t) one has (1.4)
E(Y I 5,) = E(Y1 xr);
(iii) if t and s are in T and t Is then (1.5)
E(f0
xs 1
9 1 )
= E(f0
xs
IX , )
for allfe bb. (b) X is a Markov process (with respect to a(Xs; s It ) ) if for each finite collection tl I. . . 5 t, It from T andfE bb one has (1.6)
E(f0 Xr I XI, . . . > XrJ = E(f0 Xc I Xtn)3
Proof. We will first establish the equivalence of (i) and (ii). Suppose (i) holds. It is immediate from MCT that it suffices to prove (1.4) when Y = I, with B E a(X,: s 2 t ) . If A E F 1 then ,
13
1. GENERAL DEFINITIONS
P ( A n B) = E { P ( A n BI X,)} = E { P ( A I Xr) P(B I Xr)> = ECE{I, P(B I Xr) I Xrll
P(B I Xr)l*
=
Hence (i) implies (ii). Conversely suppose (ii) holds. If A €9, and B E o(X,;s 2 t ) , then P ( A n B I X,) = E { P ( A n B I 9,) I X,}
P(B I 9,) I Xr> = E { I , P(B I Xr) I XI> = P ( A I XI) P(B I XI), =E{I,
and so (ii) implies (i). We next establish the equivalence of (ii) and (iii). Clearly (iii) is a special case of (ii) and so we need only show that (iii) implies (ii). Therefore assume (iii) holds. First observe that if H denotes those elements Y Ebo(X,; s 2 t ) for which (1.4) holds, then H is a vector space containing the constants and closed under bounded monotone limits. Thus by MCT in order to show that H = bo(Xs; s 2 t ) it suffices to show that H contains all Y of the form Y = nl=l ,fi(Xsi)where t Is1 < . .. < s,, and f i E bB for 1 I i In. We will establish this last statement by induction on n. When n = 1 this reduces to (iii). If n > 1, we may write for such a Y E(YI 9,) = E(E(Y I 9 s n - l ) 19,)
From(iii)and(2.7)ofChapterOwehaveE(f,(Xsn)I 9sn-l) = E(fn(Xs,)I XSn-,) = g(XSn_ ,) where g E B may be assumed bounded since f, is. Consequently applying the induction hypothesis to the functions evaluated at X,, , .. . , Xsn-, we obtain E(Y I 9,) = ~ ( i“= f1 i l ~ ( xgs(~x s . -
A, .. ., f n - 2 ,
f,-lg
I xr)
= E( Y I X,).
Thus Part (a) of Theorem 1.3 is established. If X is a Markov process then clearly (1.6) follows from ( l S ) , and so in order to establish (b) it will suffice to show that (1.6) implies E(f X , I 3,) = 0
14
I. MARKOV PROCESSES
I s and f E b6 where 9,= a(X,: s I t). This amounts to showing that for all A E 9,one has
E(f0 X,l A',) for t (1.7)
Let 9 be the collection of all sets A E 9,for which (1.7) holds. Plainly 9 is a d-system. On the other hand it follows from (1.6) that if A = n;=l{Xf, E A j } where 0 I tl < . .. < t,, = t and A j E 6 for 1 5 j 5 n, then A E 9. But A's of this form are a n-system generating 9,and hence 9 =9,completing the proof of Theorem 1.3. 2. Transition Functions In this section we will assume that the parameter set T of any stochastic process under discussion is R, = [0, co) unless explicitly stated otherwise. This will be the case of most interest to us and the reader should have no difficulty in adapting the definitions to other cases of interest: for example, T = Z, = (0, 1,2, ...}. (2.1) DEFINITION.Let (E, 8 ) be a measurable space; then a function Pr,,(x, A) defined for 0 I t < s < co,x in E, and A in &, is called a Markou transition function on ( E , &) provided (i) A -,Pf,,(x,A) is a probability measure on 8 for each t , s, and x; (ii) x -,Pf,,(x,A) is in & for each t, s, and A ; (iii) if 0 I t s < u, then
-=
(2.2)
j
P,,"(X, A ) = Pf,,(X, d Y ) P,,"(Y,A )
for all x and A. The relationship (2.2) is called the Chapman-Kolmogorov equation.
(2.3) DEFINITION.A Markov transition function Pf,,(x,A) on (E, 9)is said to be temporally homogeneous provided there exists a function P,(x, A) defined for t > 0, x E E, and A E I such that Pf,,(x,A) = P s - f ( x ,A) for all t, s, x , and A. In this case P,(x, A) is called a temporally homogeneous Markov transition function over ( E , 8)and the Chapman-Kolmogorov equation becomes
2. TRANSITION FUNCTIONS
15
for all t, s, x, and A. In actual fact the transition functions that we will deal with in this book are the temporally homogeneous ones and we will very shortly restrict ourselves to consideration of these only. DEFINITION.Let X be a stochastic process with values in (E, 6) that is adapted to {F,} and let P,,,(x, A) be a Markov transition function on (E, 8).One says that X is a Markov process with respect to {6,} having P,,,(x, A) as transition function provided (2.5)
(2.6)
E(S
O
x,I 9,)= P,.,(X,
9Sh
for all 0 I t < s and f i n b 6 . ( P , , s ( x , f )= P,,,(x, d y ) f ( y ) . ) Taking conditional expectations with respect to o(X,) in (2.6) one obtains (2.7)
W Xs I Xt) = Pr,s(xt S) 0
9
and upon combining (2.6) and (2.7) with Theorem 1.3 we see that the above definition is consistent with Definition 1.1. We remark in passing that there exist processes satisfying (2.7) but not (2.6). Moreover there exist Markov processes (in the sense of Definition 1.1) which do not possess transition functions. The fact that a Markov process has a transition function means that it is possible to define " nice " conditional probability distributions for the conditional probabilities P ( X , E A I Fr), 0 5 t < s, A E 8.As usual if we say that X is a Markov process with transition function Pf,s(x,A) without it is understood that 6, = o(X,; s 5 t ) . specifying the o-algebras F,, The intuitive meaning of the transition function should be clear: Pf,s(x,A) is " the conditional probability that X , E A given that X , = x when 0 5 t < s." Although this statement is very attractive, one must beware of such statements since conditional probabilities are determined only almost surely. Perhaps we should remark that without further restrictions a Markov process may have more than one transition function. But of course (2.6) implies that if PI and P 2 are transition functions for X then for each fixed t , s, and A, P&(x, A) = P,!,(x, A) for almost all x relative to the distribution in E of X , . A Markov process X with respect to {Pr} is called temporally homogeneous provided it possesses a temporally homogeneous transition function. In particular we will apply the adjective temporally homogeneous only to those Markov processes which have transition functions. Let X be a Markov process with values in (E, 6) and transition function P r , s (A). ~ , Let p be the distribution of X , ; that is, p is the probability measure on d defined by p ( A ) = P ( X , E A). The measure p is called the initial distribution of X . Using the Markov property repeatedly one easily obtains the following formula for the finite-dimensional distributions of X : if 0 It , < t2 c . . . < t, and f E bS" where 8"= 6 x .. . x 6 (n factors), then
16
I. MARKOV PROCESSES
where one integrates first on x,, , then on x,,-~,. .., and finally on xo, and if t , = 0, P0,,(x, * ) is taken to be unit mass at x. Thus all of the finite-dimensional distributions of X are expressible in terms of its initial measure and transition function. It is now natural to ask the following question: Suppose we are given a probability measure p and a transition function P J x , A) on a measurable space (E, 8).Then does there exist a Markov process X with values in (E, 8) which has p for its initial measure and P J x , A) as transition function? If T = Z, , then there is always such a process, but if T = R, the most general conditions on (E, 8)for which the answer is affirmative are unknown. However, if E is a a-compact Hausdorff space and 8 the a-algebra of topological Bore1 sets, then the celebrated theorem of Kolmogorov guarantees the existence of a Markov process with the desired properties. The proof will be outlined in the following paragraphs. Let (E, 8) be a measurable space and let T be an arbitrary index set. Define R = E' and 9 = I' so that (R,9)is the usual product measurable space, For each t E T, let X I : R + E be the coordinate map X,(w) = a(?).In particular 9 = a(X,:t E T). If J is a finite subset of T we will write (E', aJ) for the appropriate product space, and we will denote the natural projection of R on E' by n,; that is, if J = (tl, . . ., t,) nJw = (w(tl),
.. ., w(t,,))E E'.
If J = {t} then nJ = X I . Finally if I and J are finite subsets of T with I c J we will denote the natural projection of E' on E' by ni . Note that n; E 8J/b' and that if K c Jc Ithen ni = nin;.
(2.9) DEFINITION. Let q ( T )be the class of all finite subsets of T and suppose that for each J E q(T) we are given a probability measure P J on (E', 8'). Then the system { P J : J E q ( T ) } is called a projective system over (E, 8) provided (2.10)
PJ(ni)-' = PI
for I c J
E
rp(T).
Let 8: be the algebra of alljinite-dimensional cylinder sets in R, that is, all subsets A of R for which J E q(T)and B E I' exist such that A = n;' B. It is immediate that ~ ( 8 :=) IT.Moreover using condition (2.10) it is easy to construct ajinitely additive probability measure P on 8: such that Pn;' = P I for all J E q ( T ) . If P is countably additive on & ,: then it can be extended uniquely to a probability measure on 8' = 9,which we again denote by P,
2.
TRANSITION FUNCTIONS
17
so that (R, 9, P) is a probability space. Finally X = { X I ;t E T} is a stochastic P) with values in (E, 8)and the finite-dimensional distriprocess over (R, 9, butions of X are the P,'s we started with; that is, if J = ( t l , ..., t,,) and B E 8, then PC(X,, * . Xrn)E B ] = PJ(B). 9
-
9
The following theorem of Kolmogorov gives sufficient conditions under which P is countably additive on 8:. We omit the proof, which may be found in any standard text on probability theory. THEOREM(Kolmogorov). Let E be a a-compact Hausdorff space and 8 be the topological Borel sets of E. If {P,:J E rp(T)} is a projective system over (E, &'), then the finitely additive measure P on 8: constructed above is actually countably additive and hence can be extended to bT. (2.11)
Let us now apply this result to the Markov process situation. Let T = R, and suppose that a probability measure p and a transition function PI,s(x,A) are given on a measurable space (E, 8).If J = {tl f 2 < . . . < t,,} is a finite subset of T, then it is not difficult to see that (2.8) defines a probability measure P, on ( E J ,B J ) , and that {P,:J E rp(T)} is a projective system over (E, 8). Thus if (E, 8)satisfies the conditions of Kolmogorov's theorem we obtain a probability measure P on (R, 9)where R = ET and 9 = BT. The reader should now verify that the coordinate mappings { X I }form a Markov process over (Q, 9, P) with values in (E, 8) which has p as initial measure and Pr,s(x,A) as transition function. We will illustrate this result with an important class of examples. Let E = R" be Euclidean n-space and 8 be the Borel sets of E. Let {pI;t > 0 ) be a semigroup (under convolution) of probability measures on (E, 8) such that pI + E weakly as t + 0 where go is unit mass at the origin. I f f € b 8 and t > 0 define
-=
(2.12)
It is then immediate that P,(x, A) is a temporally homogeneous transition function on (E, 8).Consequently if p is any probability measure on (E, 8) it follows from the above discussion that there exists a (temporally homogeneous) Markov process X with values in (E, &') which has p as initial measure and PI(x,A) as transition function. Note that PI(x,A) is translution invariant; that is, P,(x y , A y ) = P,(x, A) for all y E E. The reader should check for himself that the translation invariance of the transition function implies that the process X has independent increments; that is, if to < r , < . . . < r,,, then the (E, 8) random variables XI,, X,, - XI,, . . ., Xrn- A',"-, are independent.
+
+
18
1. MARKOV PROCESSES
It is well known (for example, Levy [l] or Bochner [I]) that such a semigroup { p , } may be characterized as follows: Let cp,(x) = ei(x*y) p,(dy) be the Fourier transform of p, . Then cp,(x) = exp[ - t $(x)] where
s
with a E R",S a nonnegative definite symmetric operator on R",( S x , x ) the usual quadratic form associated with S, and v a measure on R" satisfying Ix12(1 + Ixl')-' v(dx) < co. Here ( x , y ) is the usual inner product in R" and 1x1 = ( x , x)ll' is the distance between x and 0. Conversely if $ is defined by (2.13) with a, S,and v as described, then there exists a semigroup {p,} of probability measures on R" such that
for all t > 0 and p, -,E,, weakly as z + 0. Some important special cases of such transition functions will be described in the exercises at the end of this section. Other important examples of Markov processes will be introduced later.
Exercises (2.14) Let X be a Markov process over (!2,9, P) with respect to {f,}. Suppose A is a a-algebra contained in f and that for every t, A and f ,are Prove that independent, ( P ( A n 8)= P ( A ) P ( B ) for all A E A and B E 9,). X is a Markov process relative to the family { c ( A u 9,)). (2.15) Let X be a Markov process with values in ( E , 8)and having a transition function Pt,s(x,A), which for each A is jointly measurable in (t, s, x). Prove that the process Y given by Y,(w)= (Xt(w),t) with values in (E x R, , 8 x S?(R+)) is a temporally homogeneous Markov process. (2.16) Let ( 9 , ; t > 0)be a convolution semigroup of probability measures on (0, a).Let P,(x, A) be a transition function on (E, 8) such that for all A the mapping (t, x ) + Pt(x, A ) is jointly measurable. Prove that Q,(x, A) = P,(x, A ) q,(du) defines a transition function on (E, 8). Prove that if P,(x, A) is of the form (2.12) then Q,(x, A) is also of this form. (2.17) Let E = R",g,(x) = (4nt)-"/' exp( - 1xI2/4t) for r > 0, x E R".Verify that P,(x, A) = j A g,(y - x ) dy is a transition function (the Brownian motion transition function in R") of the form (2.12) and that in the representation (2.13), a = 0, v = 0, and +(Sx, x ) = 1x1'. The 'd denotes the element of Lebesgue measure in R".The function g,(x)is called the Gauss kernel for R".
2.
TRANSITION FUNCTIONS
19
(2.18) Given positive numbers d and A let X be a (Poisson) random variable with the distribution P(X = nd) = exp(-A)[A"/n!], n = 0, 1, . .. . Prove that the Laplace transform E(exp( - u X ) ) = exp[ -A( 1 - exp( - du))]. Prove that if v is a measure on W(0, GO)such that 52 [x/(l + x)] v(dx) < 00, then 8(u) = jg { 1 - exp( - x u ) } v(dx), u 2 0, defines a function 8 such that exp( - 8(u)) is the Laplace transform of a probability measure q on ( 0 , ~ ) . [Hint: approximate the integral by a sum and use the result of the first part together with the convolution and continuity theorems for Laplace transforms.] (2.19) Prove that the measure q in (2.18) is q, in a convolution semigroup { q , } of probability measures on (0, GO),and that the measure v is the same one appearing in the representation (2.13). Prove that for E (0, 1) the function 8(u) = uBis a special case of (2.18). (If we denote the corresponding semigroup of measures by {qfl} then the transition function defined by (2.12) with 111 in place of p , is called the one-sided stable transition function of index B.) Prove that the measures qfl have continuous density functions. (2.20) Let E = R". Given a E (0, 2) define a transition function Pa by P,"(x, A) = Pu(x,A)tfZ(du), as in (2.16), where Pu(x, A) is the transition function in (2.17) and the semigroup qP/* is the one in (2.19). (Pa is called the symmetric stable transition function of index u in R".) By (2.16) Pa is of the form (2.12). Verify that in this example $(x) = 1x1' and that in the representation (2.13) of $(x) the first two terms are absent while v(dx) = c I x I - " - ~ dx with
5
(2.21) In the first part of (2.18) take d = 1. Prove that the distribution of X there is p , in a convolution semigroup on R. (The transition function (2.12) in this case is called the Poisson transition function with parameter A.) Find the quantities a, S, and v in the representation (2.13). (2.22) Let (Q, F, P) be a probability space. Let A be a positive number and let { U,,} be a sequence of independent identically distributed random variables on 51 with P( U,, > x) = exp( -Ax), x > 0. Define a stochastic process { X , ; t 2 O} by
X,(w)=O
if
t<
=n
if
V , ( w )+ ... + V,(w) It < U , ( w ) + ... + Un+l(o).
U,(w),
20
I. MARKOV PROCESSES
Prove directly that X is a Markov process and that the transition function arising in (2.21) is a transition function for this process.
(2.23) Let Pr,s(x,A ) be a transition function on a measurable space (E, 8). Let {XI;t E R,} be a stochastic process with values in (E, 8)and suppose that for all f E b8" and 0 Itl < . . . < t, Eq. (2.8) holds. Prove that { X , } is a Markov process with transition function Pr,s(x,A ) and initial distribution p. 3. General Definitions Continued
In this section we will introduce the type of Markov process that we will deal with in this book. Roughly speaking it will be a temporarily homogeneous Markov process defined on a " suitable " R. In this section the parameter set T will be [O,001 and as in Section 2 R + will denote the interval [O,oo). As will become clear, our point of view in this section is somewhat different from that in the previous sections. Consider the following objects : (i) A measurable space (E, 8 ) and point A not in E. We write E A = be the a-algebra in EA generated by 8. Note that E u {A} and let {A} E &A* (ii) A measurable space (Q, A!) and an increasing family {A,:t E T} of sub-a-algebras of .d.Also a distinguished point w,, of R. (iii) For each t E T a map XI : R + EA such that if X,(w) = A then X,(w) = A for all s 2 t, X,(w) = A for all w E 0,and X0(wA)= A. We will sometimes write X(r, w ) for X,(w) and X ( t ) for X,. (iv) For each t E T a map 8, : R + R such that 8, w = wAfor all w. (v) For each x in EA a probability measure P" on (Q, A'). (3.1) DEFINITION. The collection X = (Q, .H, A',, X , , 8,,P") is called a (temporally homogeneous) Markov process (with translation operators) and with state space (E, S) (augmented by A) provided the following axioms hold :
Axiom R (Regularity Conditions) (a) For each t E R, , X,E (b) The map x + P"(X, E B ) from E to [0, 11 is in d for each t E R, and BE&. (c) PA(Xo= A) = 1. Axiom H (Homogeneity) For all t, h E T, X , o Oh = X,+h.Note that this is consistent if either t or h is co.
3.
GENERAL DEFINITIONS CONTINUED
21
Axiom M (Markov Property) P"(X, +,
(3.2) for all x
E E,,
EB
I A,)= PX'"(X, E B),
B E I , ,and t , S E T.
Comments on the axioms. One may check immediately that if Axiom R holds then (b) remains valid when we replace E, I , and R, by E , , d', and T. Define 9:= a(X,: s I t) and 9 '= o(X,: s E T). These are o-algebras in R and it follows from Axiom R(a) that 9:c A, and hence that 9 'c A. Clearly {9: is} an increasing family. It will sometimes be convenient to write 9:for 9'. It follows from Axiom H that if B is in 8 , then X;'(B) = Xs-.h(B)E 9:+,, , and consequently
(3.3) eh F:+h19? for all t, h E T. In particular 8, E 9'/9'. Iff (x) = P"(X, E B), then f E I , and so P x c r (X, ) E B) =f 0 X , is in 9:. In addition the reader should check that (3.2) is consistent with Axiom R(c) if x = A and with (iii) and (iv) if either t or s is equal to 00. Equation (3.2) displays the conditional probability on the left as a measurable function of X I , and so Axiom M does indeed imply that for each x E 8 , the family {X,; t E T} is a Markov process (in the sense of Definition (1.1)) over (Q, A, P") with respect to { A , ;t E T} taking values in ( E , , 8,J.In particular Definition 3.1 requires not one Markov process but a family of them, one for each of the measures P". We will shortly show (Proposition 3.5) that each of them has the same (temporally homogeneous) transition function. Usually we will omit the phrases in parentheses in Definition 3.1 ;thus the collection X = (Q, A, A , ,X , , Or, P") will be called simply a Markov process with state space (E, 8).If the a-algebras A or A, are omitted, then it is understood that we are taking A to be 9 'or A, to be 9:. Here is an intuitive way of thinking about Definition 3.1. If we regard t -,X,(w)as the path (or trajectory) of a particle moving in the space E, then P" should be thought of as the probability law of the particle assuming it starts from x at time t = 0. The particle moves in E until it "dies" at which time it is transported to A where it remains forever. (The point A may be thought of as a "cemetery" or '' heaven " depending on one's point of view.) Roughly speaking Axiom M states that if we know the history of the particle up to time t , then probabilistically its future behavior is exactly the same as that of a particle starting at X,(o), Finally the existence of the translation operators, O f , says that the underlying space R is "sufficiently rich." With this interpretation in mind we define [(o) = inf{t : X,(w) = A}
22
I. MARKOV PROCESSES
provided the set in braces is not empty and c(o)= 00 if it is empty. Since (Q denotes the rationals) (3.4) [ is a numerical random variable. It is called the lifetime of the process X.
(3.5) PROPOSITION. Define Nt(x, A) = P"(X, E A) for ? E T, x E E d , and A E gA.Then N,(x, A), 0 < t < 00, is a (temporally homogeneous) transition function on (Ed, bA). Moreover for each x E E,, N is a transition function for the Markov process {A',; I E T} over (Q, A, P") with values in ( E A ,&A). Proof. The fact that P x is a probability measure on A implies that Nt(x, A) is a probability measure in A. Axiom R(b) and the comments following the axioms imply that N,(x, A) is in as a function of x. As to the ChapmanKolmogorov equation we have N , + Ax, 4 = P " ( K + s E 4 = EX{PX'(X,E A ) } = JWNAX, 9
4)
Finally Eq. (3.2) with the right side replaced by N , ( X , , B) is just the statement that N is the required transition function. It is immediate from Axiom R(c) that "(A, {A}) = 1 for all t and that N,(x, A) is completely determined by its restriction to (E, 8). We will denote the restriction of N,(x, A) to (E, b) by P,(x, A) and call P,(x, A) the (subMarkov) transition function of X. It satisfies all the conditions of a temporally homogeneous Markov transition function on ( E , 8) except that A -+ P,(x, A) need not be a probability measure on 8 ;one can assert only that P,(x, E) I 1. From now on the term transition function with no qualifying adjectives will mean such an object. I f f € b b we write
Clearly P,is a positive linear operator from b& to bb. The following result is analagous to Theorem 1.3 and may be proved in a similar manner. (3.6) THEOREM. (a) x + Ex( Y)is bA measurable for all Y E b 9 ' .
3.
23
GENERAL DEFINITIONS CONTINUED
(b) Under Axioms R and H, Axiom M is equivalent to each of the following : (MA
E " ( f 0 X I + ,1 AI}= EX"'f
0
X,
for all x, t , s, andfE bb,; (M2)
E"{Y
e, I A,}= E ~ ( ~ ) Y
for all x, t , and Y E bFo. (c) If A, = F:,then under Axioms R and H, Axiom M is equivalent to: (M3) Given 0 It , < . . . < t, andf,,
. . . ,f n
E
MAthen
In the remainder of this book a Markov process is always understood in the sense of Definition 3.1 unless explicitly stated otherwise.
Exercises
(3.7) Let Q = R u {A} where A is the usual point at
03.
Let A = A r=
g(Q), the usual a-algebra of Bore1 sets on R u {A}. Let P" denote unit mass at x and for t < 03 and o E R define X,(w) = o t , 0,o = o t. Complete
+
+
the definitions of X , and 8, as required for (3.1) and prove that the resulting process is a Markov process with state space (R,9(R)) (called uniform motion to the right). Show that the resulting transition function, when restricted to R, is of the form (2.12). (3.8) Let X = (Q, A, A , ,X , , e,, P") be a Markov process, with state r ) , o E R, r 2 0). space (E, 8) and fi = Q x [O,03) (with points (I, = (o, E d such that Let = A x B[O,GO) and dr consist of all sets n {Q x ( t , a)} is of the form A x (t, m) for a suitable A E A,. Let P = P" x 1 e-Ardr ( A a positive constant), and finally if (I, = (a,r )
f1(&) = x,(o),
t c r,
= A,
t 2 r.
Define translation operators 0, and a point (I,, as required for Definition 3.1 and prove that the resulting 3 is a Markov process with state space (E, 8).
24
I. MARKOV PROCESSES
Prove that its transition function is e-" P,(x, A) if P,(x, A) is the transition function for X defined following the proof of (3.5). (A more general construction along these lines will be given in Chapter 111.) 4. Equivalent Processes
Again in this section all process have T = [0,00] as parameter set. (4.1) DEFINITION. Two Markov processes with the same state space (E, 8 ) are equuiualent if they have the same transition function.
In this section we will single out from a given equivalence class of Markov processes a particularly " nice " representative defined over a function space where the meaning of the translation operators 8, will be transparent. Let (E, 8) be a measurable space which we extend to ( E d , 8,) as in Section 3. Consider the following objects: (i) W: the space of all maps w : T + EA such that w(co) = A and if w(t) = A then w(s) = A for all s 2 t . Let wA be the constant map wA(t)= A for all t 2 0. (ii) Let Y,, t E T, be the coordinate maps Yt(w)= w(t), and define in W the a-algebras So = a( Y,: t E T), 3; = a( Ys:s 5 t). (iii) Let p, : W - r W be defined by p, w(s) = w(t + s). Note that (paw = wA and that Y , cps = Y,,, for all t, s E T. 0
DEFINITION. A Markov process X = (Q, -U, A,,X , ,8, ,P")with state space (E, 8) is said to be of function space type provided R = W, J? 3 Yo, A, 2 Sp, X , = Y , , and 8, = p,.
(4.2)
THEOREM. Any Markov process X = (Q, A, A,, X , , 8, ,P")with state space (E, 8)is equivalent to a Markov process Y of function space type with (E, 8) as state space. (4.3)
Proof. Using the notation developed above (4.2) we define a map A : R 4 W as follows ( R 4 ( t ) = X,(W).
Hence Y,
0
A
= X,, and consequently
7r-yyl € A ) = 7T-l Y;'(A) = x ; ' ( A )
if A E & A . Therefore n"YY c 9: c A,; that is, A E 9p/Sp for all t E T where as usual 9: = 9' and 9% = Yo. We now define measures P" on (W, Yo) by P" = P"A-' and we claim that Y = (W, So,Yp, Y,, p,, p)is a
25
5. THEMEASURESPp
Markov process equivalent to X. Once this is established Theorem 4.3 will be proved since Y is clearly of function space type. Now
Byr, E A ) = P x - '
Y;'(A) = p" x ; ' ( A )
= P"(X, E A )
and so it only remains to show that Y is a Markov process. It is clear that Y satisfies Axioms R and H, and so only Axiom M needs to be checked. First note that (no, WNS) = x,(et0 ) = x t + , w= ( x ~ +t S) = (cpt ~ w ) ( s ) ; that is, net = cpt x . We next claim that if H : W + R is in bB', then H o x E b9' and $"(If) = E"(H R). The first statement is obvious, while if H = Z,, for A E B', then 0
P(H)= F ( A )
= P"x-'A = Ex(Zz-~A) = E"(IA0 n),
and so the result follows for general H E b8'. To check Axiom M for the process Y we must show that (4.4)
p[{Y,+, E A } n A]
= p{pyct)( Y, E A); A}
for all A E 8;and t , s, x, and A E JA.Using the facts that z-'A that X is a Markov process we see that the left side of (4.4) equals P"[x-' Y,',(A) n x-'A] = P[X,',(A)
E
9;and
n n-'A]
= EX{PX'"(X,E A ) ; x-'A},
while according to the above remark the right side of (4.4) becomes Ex{Prto"(Ys E A ) ; x - ' A } = E"{Px("(YSE A ) ; x-'A} = E"{PX'"(X, E A ) ; n-'A}.
Thus (4.4) is established and so the proof of Theorem 4.3 is completed.
5. The Measures ' P
In this section we will extend the basic o-algebras 9 'and 9;by completing them in an appropriate manner. We assume throughout this section that X = (0, 4, A , ,X,, O r , P")is a given Markov process with state space (E, 6')and that T = [0, 001. We have already remarked that x -,P"(A) is &A measurable whenever A is in 9 '(Theorem 3.6). Therefore, given a finite measure p on &A, we can ' define for any A in 9 (5.1)
P"(A) = J P"(A) p(dx).
26
I. MARKOV PROCESSES
It is immediate that P , is a finite measure on f o(P" is a probability on 9 ' if and only if p is a probability on 8J. Moreover if E, denotes unit mass at x then Pex= P" on 5'. In general the measures PI can not be defined on A unless we make some measurability assumption on the functions x + P"(A) for A in A. We now pause from our main development in order to introduce some terminology that is essential in the sequel. Let (E, 8) be a measurable space. If p is a measure on B and 9is a sub-a-algebra of 8,then gP, as in Chapter 0, denotes the completion o f f with respect to p. Of course f p c8 ''and the measure p extends uniquely to 8".This extension will again be denoted by p. Suppose now that U is a family of finite measures on 8;then the completion, F', of f with respect to U is defined by (5.2)
The reader should verify that (F") = "f " , and that f E 9"if and only if for each p E U there exist fl and f2in f such that fiIf Ifiand p ( f i c f2)= 0. If U is the family of all finite measures on B, then I" = I * ,the a-algebra of universally measurable sets over (E, 8).See Chapter 0.
(5.3) DEFINITION. Let (E, 8)and U be as above. If Y is any a-algebra contained in '8 we define the completion of 9 in I"with respect to U,which we denote for the moment by @, as follows: A E 9 if and only if for each p E U there exists A, E Y such that A - A , and A, - A are in I"and p ( A - A,) = p ( A , - A ) = 0. The following characterization of 9 is useful in some proofs. The notation is that of (5.3).
(5.4) PROPOSITION. A is in g if and only if for each p E U there exist D, in 9,A , and B, in d such that D, - A , c A c D, u B, and p ( A , ) = AB,) = 0. Proof. Let A have the property of Proposition 5.4. Since D, - A , and D, u B, are in E" and p [ ( D , u B,) - (0, - A,)] I p ( A , u B,) = 0, it follows that A is in 8 ' . Hence A - D,and D, - A are in 8".But p ( A - 0,) I p(B,) = 0 and p ( D , - A) I &4,) = 0. Consequently A is in g. Conversely if A is in @, then given p there exists D, in 9 such that A - D, and D, - A are in 8" and are p null. Hence there exist A , and B, in d such that D, - A c A, and A - D, c B, with p(A,) = p(B,) = 0. But
D, - (0,- A ) c A
c
D, u ( A - 0,)
5. THE MEASURES Pp
27
and so D, - A , c A c D , u B , . Thus A meets the requirements of Definition 5.3. The reader should now have no difficulty in checking the following elementary properties : (5.5)
(i)
8 is a a-algebra, 3 c 8 c bu;
(ii)
QU c
(iii)
9;
(g)-= 8.
The following measurability lemma will be of frequent use in the sequel. (This generalizes Exercise 3.3 of Chapter 0.) (5.6) LEMMA.Let (Ei, bi) be measurable spaces and Ui be families of finite measures on b i , i = 1, 2. Let g i be a-algebras contained in &", i = 1,2. Supposefis in 9J3,and in bl/b,. If pf-' E U, for all p E U,, then
fE 91/@2. Proof. If A E 8, we must show that f - ' ( A ) is in 8,.Given p E U, then v = pf E U, and so there exist D, E 9,,A , and B, in 8, such that D, - A , c A c D , u B, and v(A,) = v(B,) = 0. Defining D, =f -'(D,), A , = f - ' ( A , ) , and B , = f - ' ( B , ) itis clear that D , - A , c f -' ( A ) c D, u B, and that p ( A , ) = p(B,) = 0. Consequentlyf-'(A) E g1. In the special case that 3,= b, and 9,= E, the lemma states that if b,/b, and U, f-' c U, thenfe &yl/€yz. In particular b1/8, c Sr/S;. If we assume only 9,= b,, then the above argument actually shows that
-'
f E $!?y1/€yZ. Let us return now to the consideration of a Markov process X = (Q, 4, A f ,X , , Of,P") with state space ( E , b).We definedto be the completion of A with respect to the family of measures {P",x E E,}, and 9 to be the completion of 9'with respect to the family {P";p a finite measure on b,}. We next define d, to be the completion of 4, in d with respect to {P"; x E E,} and 9, to be the completion of 9;in 9 with respect to {P";p a finite measure on S,}. Since 4 I> 9'and A, 3 9; it is immediate that J? I> 9and df2 9,. These completions turn out to be the appropriate ones for our theory. See, for example, Proposition 8.12. The next definition is of central importance. DEFINITION. Let q(w) be a property of w ; then q is said to hold almost surely (as.) on A E 9 if the set A, of w in A for which q(w) fails to (5.7)
28
I. MARKOV PROCESSES
hold is in 9 and P"(Ao) = 0 for all x . If A = R, we simply say "almost surely." The following propositions, (5.10)-(5.12), are designed to show that, roughly speaking, one can replace the a-algebras A, and 9:by A,and Pr provided one replaces 8, by 8:. (Clearly 62, the universally measurable sets over (Ed, &'A), is just the a-algebra in EA generated by b*.) In particular it will follow from our discussion that (R, A,A,,X,,6 , , P") is a Markov process with state space (E, 8*).
(5.8) PROPOSITION.If Y Eb 9 , then x + Ex( Y) is S,* measurable. Proof. Given p there exist Y, and Y2 in b9' such that Y, IY IY , and
EM(Y2 - Y,) = 0. Clearly Ex( Y,) < Ex( Y ) 5 EX(Y,) for all x and x + EX(Y j ) is 8, measurable, j = I, 2. But
J
[Ex( Y2) - Ex( Yl)] p ( d x ) = E"( Y, - Y,) = 0,
and hence x --+ Ex( Y) is in
Sincep is arbitrary this yields Proposition 5.8.
(5.9) REMARK.It follows from the proof of Proposition 5.8 that if Y Eb 9 then EM( Y) = SEX(Y) p(dx) for each p. This relation then holds for any nonnegative Y in 9. (5.10)
PROPOSITION. For each t, X,E 9",82.
Proof. This is an immediate consequence of Lemma 5.6. Actually X,E (9:)p'/82.See the remark following (5.6).
0,
Combining (5.9) and (5.10) we see that x + E xf ( X J is 8; measurable for all t and f E b 82.In particular x + N,(x, A ) = P"(X, E A ) is 82 measurable ifAE82. (5.11)
PROPOSITION.For all t , h E T, 6 , '9, c F,+h,and so 6; '9 c 9.
Since is in 9 : / 9 : + h and in 9'/9' this will follow from Lemma 5.6 once we show that for each finite p on 8 , there exists a finite v on 6, such that PV;' = P". We define a finite measure v on 8, by Proof.
V(A) = P'(XI, E A ) = J P x ( x h E A ) p(dX),
and check that for A E 9 '
5.
THE MEASURES P"
=
1P"[Xh
E
29
d y ] P'(A)
= J ~ ( d yPy(A) ) = P"(A).
(5.12)
PROPOSITION. If Y E b 9 then for each x and t F(Y
8, I 2,) =E~(~)(Y).
Proof. From (5.8) and (5.10) it follows that the real-valued function w + EX'("')( Y) is in 9, , which is contained in A t ,and so we need only check that for each A E 2, E"(Y o 8,; A)
(5.13)
= E"[EX"'(Y);A ] .
Clearly, from the definition of .,#, it suffices to prove this for A E 4,. Given x and t define a measure p on 82 by p ( A ) = P"(X, E A). The computation made in the proof of (5.11) shows that for B E 9 'we have Pp((B)= P"(0; B), and of course for any Y E 6 9 we have E"Y
(5.14)
=
s
EY( Y ) p ( d y ) = E"[EX(')(Y ) ] .
If Y E b 9 let Z E b9' be such that { Y # Z } c r where r E 9' and Pp(r) = 0. Then { Y 8, # Z 0 Of} c 0; 'r and P"(8; 'r) = Pp(r) = 0. Also by (5.14) 0
E"(EX'"IY-ZI)=E'IY
-Z(=O
and so Ex(')Y = Ex(')Z almost surely relative to P". Thus we may replace Y by Z on each side of (5.13) without altering its validity. But with this replacement the validity of (5.13) is a consequence of Theorem 3.6(b), so the proof is complete. It is evident now that (Q, 2, df,X,,O r , P") is a Markov process with state space (E, S*),and hence, in this case, with state space (E, 8). Thus nothing is lost by enlarging the a-algebras A and At to J? and .dt. Consequently we will assume unless explicitly stated otherwise that (5.15)
.&=AT;
A,=J?,.
In particular then 9 c 4 = 2 and 9, c 4, = A,. So far we have made no assumption to insure that P" is actually the probability measure of the process starting from x , although we have stated that P" should be thought of in this manner. We now make this precise.
30
I. MARKOV PROCESSES
(5.16)
DEFINITION.
The Markov process X is called normal provided that
{x} E d and that P"[Xo = x] = 1 for all x in E. The assumption of normality will always be explicitly stated in the hypotheses of those results for which it is needed. We close this section with following simple, but very useful, result. then (5.17) PROPOSITION. (Zero-One Law). Let X be normal. If A E 9,, P"(A) is either zero or one. Proof. We first observe that if B E 9 'then 8;'B = B, and consequently if A E 9then P p ( A - &'A) = Pp(B;'A - A) = 0 for all p. Therefore Px(A) = F ( A n 0; A) = Ex[PX'o'(A);A] = [P"(A)]'.
Exercises (5.18)
each p
In the notation of Definition 5.3 show that SE9 if and only if for U there exist g E Y and A E d such that {f# g} c A and Pp(A)= 0.
E
Consider the process of uniform motion to the right discussed in Exercise 3.7. Find A? and 9 for this example.
(5.19)
Let X be a Markov process with state space (E, 8)where E is a metric space and 8 is the Bore1 sets of E. Suppose X satisfies the conclusion of Proposition 5.17. Let B = {x E EA: P"(Xo = X) = l}. Show that B E & , but that B need not equal E A (that is, X need not be normal). On the other hand show that, for each x and t , P"(X, E B) = 1.
(5.20)
Let E = R, 8 = g(R), Y = (0,R},and U = {p} where p is a fixed nonzero finite measure on (E, 8).Using the notation of (5.2) and (5.3) show that Y" # 9.
(5.21)
6. Stopping Times
In this section we will introduce a class of random variables that will play a fundamental role in the remainder of this book. Let (Q, 9 )be a measurable space and let { S tt:E T} be a fixed increasing family of sub-a-algebras of 9.
6.
STOPPING TLMES
31
Again in this section T = [0, co] and it will be convenient to assume that Frn = 9. We will leave to the reader the task of developing the analogous (but simpler) situation in which T = (0, 1, . . . , co}. A map T : R + T is called a stopping time (with respect (6.1) DEFINITION. provided { T I t} E 2FGrfor all t in [0, a). to {st,}) Note that {T = co} = {T < co}' E 9 = 9,, and ,so {T < t } E 9, for all t E T. Hence T E 9. Of course { T I f } E 9, if and only if {T > f } E S t . Clearly any nonnegative constant is a stopping time. Given the family {F,} we define for each t E [0, 00) a new a-algebra 9,+ = 9,. We have used the notation 9,+ for this a-algebra because it is the standard notation. Unfortunately in our notation this same symbol measurable functions. We will could be used to denote the nonnegative 9, neuer use 9,+ in this latter sense. For convenience we set + = 9. Similarly nw e d e f i n e f o r O < t < c o , F , - = a ( U s < r 9 s ) a n d s e t 9 0 -= 9 0 , 9 r=9. It is immediate that { F r + }and {9,-} are increasing families of suband 9,c Ftt c F r +for all t. a-algebras of 9,
ns,,
The family {PI} is right continuous if .Ft = 9,+ for (6.2) DEFINITION. each t < 00. Note that {PI+} is always right continuous. (6.3) PROPOSITION. Tis a stopping time with respect to {St,+} if and only if {T < t} E 9, for all t < co. Proof.
This follows from the identities
Note that this actually shows that T is a stopping with respect to {9,+} if and only if { T < t} E 9,for all t. If T is a stopping time with respect t o {T,}, then from the above identity {T < t} E St,-and so T is a stopping time with respect to { P r +The } . converse is not true in general. Example. If X is a Markov process then according to (3.4) ( is a stopping time with respect to {9:+}.
If T and S are stopping times (with respect to {P,}), (6.4) PROPOSITION. then so are sup(T, S), inf(T, S),and T S.
+
32
I. MARKOV PROCESSES
Proof. The following identities establish the first two statements : {suP(T,S ) I t} = {T I t } n { S I t } , {inf(T, S) I t } = { T 5 t } u { S I t } . For the third statement we write
{ T + s > t } = (0 < T < t , T + S > t } u { T = 0,T + S > t }
u{T > t , S = 0 } u { T r t , S > 0} = A1 v
A2
u A3 u A4.
We observe that (Q= rationals) A1 =
u
({r< T < t } n { S > t
- r } )E F1,
re(O,t)nQ
A2 = { T = 0 } n { S > t } E F ~ .
Similarly A3 E f,, and finally A4 = {T 2 t } n {S = 0)' T + S is a stopping time.
E P I .Consequently
(6.5) PROPOSITION. If {T,,} is a sequence of stopping times with respect to {F,} then , sup T,, is such a stopping time. If {f,} is right continuous then inf T,, lim sup T,,, and lirn inf T,, are stopping times with respect to {F",.
Proof. The proposition results from the following identities: {SUPT, s t } =
n{T,I t i ,
{inf T, < t } =
u {T,<
n
t},
n
lim sup T,, = inf sup T, , k
nzk
lim inf T,, = sup inf T,. k
n2k
(6.6) DEFINITION. If T is a stopping time with respect to (9,) we define F T to consist of all sets A in f such that A n {TI t } E 9, for all t < 00. The reader should have no difficulty in verifying that F Tis a a-algebra and that if T(w)= a 2 0 for all w then F T= F, ., Intuitively one should think of F, as containing all the information in some physical process up to time t ; then F Tcontains all the information up to the random time T. The defining property of a stopping time means that we can tell whether or not T is greater than t knowing only the information up to time t.
6.
33
STOPPING TIMES
If T is a stopping time with respect to { R t + }we write F T +for the corresponding object, that is, the collection of all A in F such that A n {TI t} E F,+ for all t < 00. It is easy to see that A E F T +if and only if A E F and A n { T < t } €9, for all f
(6.7) PROPOSITION. Let {Fr} be given. (i) If T is a stopping time, then T is FTmeasurable. (ii) If T and S are stopping times and T IS, then RFT c Fs . (iii) If {F, is} right continuous and {T,,}is a sequence of stopping times, then if T = inf Tnone has f T = OnFTn. Proof. (i) We must show that { T I a } E FTfor all a 2 0. But
{ T 5 a } n { T It } = { T Ia
A t}E
FaAr c Ft,
and so T E F T . (ii) If A E FTthen
A n { S Ir } = (A n { T It } ) n { S It } E Fr, and so A E fs. (iii) According to (6.5) T is a stopping time and from (ii) we have F Tc FTn. On the other hand, if A E FT,,then
n,,
0.
and so A E F T . PROPOSITION.Let {R,}be given and suppose that T and S are stopping times; then each of the sets {T c S}, { S < T } ,{ T IS}, {S IT}, and {S= T} is in both P Tand Fs.
(6.8)
Proof.
We have the equality {T < S } n { S It} =
U< ( { T < r } n { r < S It } ) E Rt,
rEQ.r
I
and so { T < S}e R S .If D, = (Q n [0, t ] ) u {r}, then { T < S } n { T It } =
U { T Ir } n { r < S } E 9, , rsDt
and so {T < S}E .FT.By symmetry {S< T} is in both RTand Fs, and Proposition 6.8 now follows by taking complements and differences.
34
I. MARKOV PROCESSES
The following approximation procedure will be useful in some proofs. Given H : R + [0, m] we define (n = 1,2, ., .;k = 0,1, ...)
k+l H'"'(w) = 2" =co
on { H = c o } .
It is obvious that H ( " ) JH for all w and that H(")> H on {H < 00). Morethen so is each H(").Finally it is easy to see that if H is a stopover if H E 9, ping time so is each H(").Thus any stopping time T can be approached pointwise from above by stopping times taking on at most countably many values. Let (R, 9)and {9,; t E T} be given and suppose that for each t we are given a map X , from R into a measurable space (E, 8).Let us call X = (R, f,9, , X,) a randomfunction taking values in (E, 8)if X , E 9,/& for each t E T. Given H : R + [0,00] we denote the map w + XH(o)(w) by X , . Finally we write 9,for the ordinary Borel sets of the interval [0, t] and set W = 9,.
(6.10) DEFINITION. Let X = (a,9,9,,X , ) be a random function taking values in (E, 8).For each t E T let 0,be the map (u, w ) + X J w ) from [O, t ] x R to E and let (D = Om.Then (i) Xis measurable if 0 E (Se x F)/& and (ii) X is progressively measurable (with respect to {F,}) if @, E (9,x 9 , ) / S for each t .
If E is a metric space and d the a-algebra of Borel sets of E and if t + X,(w) is right continuous for all w, then X is progressively measurable. The proof of this useful fact is left to the reader as Exercise 6.13. (6.11) THEOREM. Let X be progressively measurable and let T be an {9,} stopping time; then X , is in pT/8. Proof. If B E d we must show that {X,E B} E F,,or equivalently { X , E B } ~ { T t} < € 9 ,for all t. Let Y t : {TI t } + [0,t] x R be defined by w + (T(w),w), and let 0,be the map defined in (6.10). If Y is the restriction of X , to {T It } , then Y = 0,0 Y, Now by hypothesis 0,E (9,x F,)/S. Moreover Y, E 9 , / ( S e , x 9,) since if 0 Iu, < u2 I t and A E 9, then Y; ' ( ( u l ,u2] x A) = {ul < T 5 u 2 } n A E 9,.Thus Y E 9,/8. But { X , E B } n {TI t } = Y-'(B) E 9,,and this establishes (6.1 1).
.
(6.12) COROLLARY. If X is measurable and H : R + [0, 001 is in f,then
x, E 9"l.
6 . STOPPING TIMES
35
Proof. Let gr= 9 for all t and apply (6.1 I) to the random function (Q, X,). 9
3
3
, 9
Exercises (6.13)
Prove the assertion following Definition 6.10.
Let ( E , b) be a measurable space and U a family of finite measures on 8.Let { b r :t E B + }be an increasing family of sub-a-algebras of 8" and let a superscript tilde denote completion in 8'' with respect to U (Definition 5.3). Prove that (b,,)" = (&J+ for all t . (6.14)
Let (E, b) be a measurable space and suppose that d # (0,E } . Let R = ET and let X,(w) = w(t) be the coordinate mappings. Here, as usual, T = [0, 001. Let 9:= a(X,; s I t). Show that for each t < co,F: # 9:+ . (6.15)
Let ( Q 9 be ) a measurable space, E a metric space, and d the a-algebra of Bore1 sets of E. Let X , : R + E be in 9 / S for each t E R + and suppose that t + Xr(w)is continuous for each o.If A is a subset of E define D A ( o )= inf{t 2 0: X,(w) E A } where the infimum of the empty set is + co by convention. Finally let 9:= o(X,: s 5 t ) . (a) If A is open show that DA is an {S:,} stopping time. (b) Give an example to show that DA need not be an (F:}stopping time when A is open. (c) If A is closed show that D A is an {S:} stopping time. Note that (a) remains valid if one only assumes that t + X,(w) is right continuous for each w. (6.16)
(6.17) Let R be a space and ( E , 8)be a measurable space. Suppose that for each t E R, we are given a map X , : R + E. As usual define 9:= a(X,: s E R,) and 9:= .(A',: s I t ) . Suppose further that for each t and w there exists w' such that Xs(w')= X,, ,(w) for all s E R+ . (a) Show that A E 9: if and only if (i) A E 9: and (ii) wo E A and X,(w) = X,(wo) for all s I t together imply that w E A. (b) T 2 0 is an {9:stopping } time if and only if T E9:and T(wo)I t, X,(w) = X,(wo) for all s I t imply T ( o )I t. (c) Let T be an {S:} stopping time. Show that A E if and only if (i) A E 9: and (ii) w,, E A and X,(wo) = Xs(w) for all s I T(wo) imply that w E A. [Hint : use Corollary 2.9 of Chapter 0.1
Let (R, 9) and (9,; t E T} be as in the first paragraph of this section. stopping time is the infimum of a countable family of Show that any {S,} stopping times T each of which has the following special form: T = a on AE.F',and T = 00 on A'. (6.18)
36
I. MARKOV PROCESSES
7. Stopping Times for Markov Processes
In this section X = ( O , A ,Af, X I ,8,,P") will be a fixed Markov process with state space (E, 8).We assume without loss of generality that A = A and J?, = Af, We will consider stopping times relative to {A,}and (9,) for the most part. In particular we will say that T is a stopping time for X provided it is an (9,) stopping time. If T is an { A , } ( { F stopping ,}) time then of course AT(9,)consists of those sets A e A ( 9 ) such that A n {T I r} E Af(9,) for all t. The reader should have no difficulty in verifying that (7.1)
2,= A
T
(9,= 9,)
where A?,($,) is the completion of A#,) in A(9)with respect to the family {P"; x E EA}({Pp;p a finite measure on 8,)).We say that X is progressively measurable provided that the random function (a,A, A,,X,) taking values in (E,, 8,) is progressively measurable. It then follows from (6.11) that X,E&,/&, whenever T is an {A,}stopping time. Moreover making use of (7.1) and (5.6) it is easy to see that for such a stopping time (7.2)
XT E AT/&:.
The following situation will often arise. Suppose (E, 8) is a given measurA,A,,X , ,e,, P") is a Markov process with state space able space and X = (0, (E, 8*).Clearly A,/&:c &,/&A and so we may consider (a,&, A,, X , ) as a random function taking values in (E4, 8,) as well as in (E,, 8:). If this random function (with values in (EA, 86))is progressively measurable, then for any { A f }stopping time T, X , E & T / 8 A and consequently (7.2) is again valid. For example, let EA be a metric space and 8, = W(E,) (the Bore1 sets). If X is a Markov process with state space (E, b*) such that t + X,(o) is right continuous for each w , then (0, A, A,,X,) is progressively measurable if we regard it as taking values in ( E , , 8,) and so (7.2) holds. However if we regard (a, A, A , , XI)as taking values in (E,, 82)it will not, in general, be progressively measurable. When considering {9,} stopping times the following theorem allows us to stopping times in many situations. restrict our attention to {9:+}
(7.3) THEOREM.Let T be an { S t +stopping } time. Then for each p there such }that P'(T # T,,) = 0. exists a stopping time T,, relative to {9:+ Proof. For each n define T'") as in (6.9) Then each T'") is an {F,} stopping time taking on the discrete set of values {k/2";k = 1,2, , .. , 00). Suppose the measure p is given. Fix n for the moment, let A, = {T'")= k/2"}, and let A , E St12,, be such that P'(AkA Ak) = 0; this is possible because Ak E Sk12,, .
8. THE STRONG MARKOV PROPERTY Let B, = A , , Bk = Ak - u
j < k
R'"'(W)
37
A j for k = 1, ..., 00, and define R'"' by
k 2
Bk
= 5,
W E
=00,
WER-UBk. k
One checks immediately that R'") is an (9:stopping ) time and that P'(R(") # T'"))= 0. If we set S,, = inf,,,, R'k', then according to Propositions (6.4) and (6.5) {S,,}is a sequence of {Pf}stopping times decreasing to a limit, T,, , which is an IF:+}stopping time. Since T'") decreases to Tit is clear that P"(S,,# T'")) = 0 for all n, and consequently Pp(T # T,,) = 0. If H : R + [0, 001 we define the translation operator 0, as follows:
e,
(7.4)
W
=
eH(o)w .
Obviously X , o 8, = X,+,for each fixed t. (7.5) PROPOSITION. If X is a Markov process and for all {A,}stopping times T, X , E A T l b b ,then for any such T, O i ' 9 f c Af+Tand hence e;lso A.
nl=,
Proof. If A = { XfJE B j } with Bj E &A and t, < t , < . . . c t,, 5 t, then 8,'A = n ; = l { X ( J + TE B j } and so O i ' A E A T + The ( . desired conclusion is now obvious.
8. The Strong Markov Property In many arguments it is necessary to use the Markov property (3.2) for certain stopping times T as well as for fixed times 1. In early work it was more or less tacitly assumed that this was possible. However, this requiresproofand in fact is not true for all Markov processes (see Exercise 8.20). In this section we will develop a sufficient condition (Theorem 8.1 1) for this "strong" or " extended " Markov property. In this section all stopping times will be mentioned otherwise.
{&(I
stopping rimes unless explicitly
DEFINITION. Let X = (R,A, .A'(, X , , O r , P") be a Markov process with state space (6 8).Then X is said to have the strong Markov property or simply to be strong Markov, provided that for each stopping time T and f E b b Aone has (8.1)
38
I. MARKOV PROCESSES
(S.R.)
XT
and (S.M.)
EX{f
x~+ T I A T }
= EX'T'[f(X,)l
for all t and x. REMARK.x + E xf (XI) is 8, measurable and so the function EX'T'f(X,)on ZZ is A, measurable. Hence under (S.R.) the right side of (S.M.) has at least the appropriate measurability property. Note that (S.R.) is satisfied whenever X is progressively measurable. We now introduce a convention which will prove extremely useful in the remainder of this book: any numerical function f on E will automatically be extended to E A by setting f(A) = 0 unless explicitly mentioned otherwise. From this point of view bb is identified with the subspace of bb, that consists of all elements (in bd',) vanishing at A. Note also that we may now write P I f ( X ) = E"{f(X,); x,E El = E"f(X,) iffis in bb or in &+ . Clearly PIf(A) = 0 and so our convention is consistent.
(8.2) PROPOSITION. If X satisfies (S.R.), then X is strong Markov if and only if for each stopping time T and f i n bb one has (S.M.)'
E"f(X~+ T ) = E"{EX'T'[f(Xl)I}
for all t and x .
Proof. First of all let us suppose that (S.M.)' holds for all f E bb,. Let T be a stopping time and let A E d TDefine . T A as follows: TA = T o n A, TA = 00 on A'. Since {TA 5 t} = {T It} n A E A t ,TA is a stopping time. Iff E bb, then E"f(Xl
+ T A ) = E X { f ( X l + T ) ; A} -k f(A) px(Ac)
and EX{EX(TA)
Cf(X1)I)= E"{EX'T'rf(Xl)I ; A> +f(N P ( A C ) .
By assumption the left sides of the above equations are equal and so (S.M.) holds. Thus it remains to show that (S.M.)' holds for all f = bb,. Now { fbb,: ~ (S.M.)' holds} is a vector space which contains IEa and, by hypothesis, any f E bb, which vanishes at A. But any f E bb, is a linear combination of two such functions, and so (S.M.)' holds for ally€ bb, . Thus Proposition 8.2 is established since the converse is obvious.
8.
39
THE STRONG MARKOV PROPERTY
(8.3) REMARK.If Af = S t ,then because of Theorem 7.3 it suffices that (S.R.) and (S.M.)' hold for all {S:,} stopping times, and when this is the case then (Q, 9,F t + ,X I , O f , P") is strong Markov. The reader should recall that & t + = Sf+(Exercise 6.14). (8.4) PROPOSITION. Let X be strong Markov and Y E b9'. E x [Y 0 flT I &T] = EX(T)(Y) for all x and all stopping times T.
Then
Proof. By Proposition 7.5, O ; ' F 0 c A , and so Y o 8 , ~ b A . Also o -,EXT(0)( Y ) is in AT and so it suffices to prove that E"[ Y 0 8,; A] = E"[EX(")(Y); A] for all A E AT. As usual it is enough to consider only Y of the form Y = fj0 X t , withfjE bb, and 0 I tl < . . . < f n . Weargue by induction on n. When n = 1 the required equality is just (S.M.). Consider the stopping time R = T tn-l. Since A E ATc ARand n ~ ~ ~ f j [ X f E, +b A T ]R we have
n'&l
+
."[ fi
j =1 f j ( X f j + T ) ;
n
n- 1
A] = E x [
fj(x,,+T)f(xR+fn-fn-~); A]
j=1
If we set gj =fi, 1 < j I n - 2, and gn-l(x)=fn-l(x) EXLf(Xfn-fn-,)], then each gj is in bb, and the last displayed expression may be written E x [ n ; : : gj(Xf,+r);A]. Thus using the induction hypothesis we obtain
completing the proof of (8.4). (8.5) COROLLARY. If X is strong Markov then for each stopping time T and t E T one has O; IFf c A T + and consequently 0; '9 c 4. Proof. Making use of (7.1) this will follow from Lemma 5.6 once we show that for each x E EAthere exists a finite measure v on 8, such that PxO; = P'. But using the strong Markov property one checks exactly as in the proof of (5.1 1) that v(A) = Px[XTE A ] is such a measure.
'
40
I. MARKOVPROCESSES
(8.6) COROLLARY. Let X be strong Markov. Then for each Y E b 9 one has E X [Y o 8 T I A T ] = EX(T)(Y) for all stopping times T and all x.
Proof. Corollary 8.5 implies that Y o E b A and as before EX(T’(Y) is ATmeasurable as a function of w. The proof of (8.6) then goes exactly as the proof of Proposition 5.12. Suppose R and T are stopping times; then one can form R + T O8,. Intuitively if we think of R and T as the times at which certain physical events, o! and B, occur, then R T 0 8 R would be the first time p occurs after o! has occurred and hence should be a stopping time itself. The following theorem makes this precise.
+
(8.7) THEOREM. Suppose X is strong Markov and let R be an { A , + } stopping time and T be an {St+} stopping time. Then R + To 8, is an { A f +stopping } time.
Proof. If S = R
+T
0
8R
then
{S
u
{R
- r; T
0
8R < r}.
rEQ
But {T O R < r } = 8,’{T < r } E d R +byr (8.5), and if A E A R +then r A n {R < t - r } = A n { R r < t } € A fThus . { S < t } E At and so S i s an { A , +stopping } time. 0
+
Of course, one can replace {A,+} and {9,+} by {Af}and {Sf}, respectively, in the hypothesis of (8.7) without altering the conclusion. However it is not clear (and perhaps not true) that with this replacement one can also replace { A , +by } {Af}in the conclusion. We are now going to give some sufficient conditions for the strong Markov property. We begin by introducing the “potential operators ” which will play a central role in the remainder of this book. In Section 3 we defined the transition operators PI f o r f e bd by
(8.8) = E ” [ f ( X , ) ;X,E El
for t 2 0.
It follows from the Chapman-Kolmogorov equations that {P,;t 2 0) is a semigroup of (linear) operators on either b& or b8* to itself. Both 6 8 and bb* are Banach spaces under the supremum norm, )I -11, and llPfII I1. If X is normal, then Po = I. Recall that any numerical function f on E is extended to EA by settingf(A) = 0.
8.
THE STRONG MARKOV PROPERTY
41
Let us now assume that X is measurable relative to 9'; that is, the map 0 : ( t , o)+ X,(w) is in (W x Fo)/S,, W denoting as usual the Borel sets of [O,oo). It now follows from Lemma 5.6 that if I is a finite measure on W and p is a finite measure on b,, then 0 E (9 x Fo)A*p/&X where (Wx Fo)A*p is the completion of W x 9 'with respect to the product measure I x P". In particular one can take , Ito be equivalent to Lebesgue measure. Therefore iffis in &, (&':), then (t, w ) +f[X,(w)] is in W x 9 '((9x for all I , p), respectively. It is now evident that i f f E b b then ( t , x ) -+ P , f ( x ) = E " f ( X , ) is in b(W x b), while i f f € bB* then the above map is (W x measurable for all I and p where the a-algebra in question is the completion of W x &' with respect to the product measure 2 x p. Therefore if u > 0 and f E bb* we can define 00
(8.9)
1
00
U " f ( x )= Jo e-" P , f ( x ) dt = E x
e-"'f(X,) d t .
0
One verifies easily that U" mapsbb(bb*) into bb(M*) and that 11 U"II I u - ' . The operator U" is called the a-potential operator of X and the family { U"; u > 0) is called the resolvent of the semi-group {P,;t 2 O } . It is easy to t 2 0 } ,that the resolvent equation check, using the semigroup property of {P,; (8.10)
U" - up = (/3
- u)U"up,
u, /3
> 0,
holds, and consequently U"Up= U p U " . We now assume that E, is a metric space and that 8, is the a-algebra of Borel sets in E, . A Markov process X is said to be right continuous if almost surely the mapping t + X , ( o ) is a right continuous function from [0,co] to E,. If X is a right continuous Markov process with state space (E, &*), then the random function (Q, A, A f ,X,) with values in ( E , , 8,) is progressively measurable and so the above discussion applies. We come now to the main theorem of this section. (8.11) THEOREM.Let (E, , 8,)be as above and let X = (Q, A,A,,X , ,8, ,P") be a right continuous Markov process with state space (E, &'*). Suppose there exists a linear space L of bounded continuous functions on E such that (i) for every f E L and u > 0 the map f + U " f ( X , ) is right continuous on [0, 4') almost surely and (ii) whenever G is an open subset of E there exists an increasing sequence {f,} in L with f,f Z, . Then (Q, A, At+, X , , O f ,P") is strong Markov. REMARKS. In particular X is then Markov with respect to { A r +Note } . that if CJ" : C , ( E ) + C ( E ) for each a > 0, then L = C,(E) certainly satisfies the conditions in (8.1 I).
42
1.
Proof,
MARKOV PROCESSES
The discussion following (7.2) implies that (S.R.) holds. Let T be an
{ A r +stopping } time and define T(")as in (6.9) for each n. I f f € L and a > 0,
then 00
Ex
joe - " f ( X T + , ) dt = lim Ex n
I
m
e-"'f[X(T("'
0
+ t)] dt
m
1
= lim n k = l EX(jome-.f[X(t
+ k2-")] d t ; T'")= k 2 - " ) .
But (7"")= k2-"} = {(k - 1)2-" IT < k2-"} E A,,-",and so using the ) last displayed expression becomes Markov property (relative to { A , }the lim n
2 Ex(Ex(k'-")[jome-.'f(Xr) d t 1;T'"'
= k2-"
k=l
= lim E x U a f [ X ( T ' " ) ) ] . n
But t -+ U e f ( X r )is right continuous on [0, () and equal to zero on [(, and hence is right continuous on [0,00](as.). Since T'"' J. T we obtain
oc)]
jOme-",E x [ f ( x t + T ) l dt = E " [ ~ " f ( x T ) I = /ome-ulE x { E X ( T ) [ f ( X r )d] t}
for all a > 0. The functions t --t E"lf(X,+T)] and t + E x { E X ( T ) [ f ( X , ) are ]} both right continuous and the above calculation shows that they have the same Laplace transform. Therefore by the uniqueness theorem for Laplace transforms, they are equal. Now using the second property of L we obtain P X [ X , + ,E C ] = Ex{PX'T'(XrE C ) } for all open sets G in E and hence by MCT for all G in 8.An application of Proposition 8.2 now completes the proof of Theorem 8.1 1. Since F,+ c M I + we see that X is Markovian with respect to {.F,+} under the hypotheses of Theorem 8.1 1. In this situation things are very nice indeed as the following proposition shows.
(8.12) each t.
PROPOSITION.
If X i s Markov relative to {9:+}, then F,= F,+for
Proof. By assumption for any Y E bFo,t, and p we have
E"[ Y 0 8,l F:+]= Ex(')(Y ) = E"[Y 0, I F:]. 0
8.
43
THE STRONG MARKOV PROPERTY
Suppose we have established the fact that for all p and Y E b9' Efl( Y I 9;+> = E'( Y I 9;).
(8.13)
If we set Y = I , with A E 9;+, then (8.13) implies that, for each p, I,, differs from an 9;measurable function on a P' null set. (See the discussion above c 9, this implies that A E $7 = 9,.ConseTheorem 1.3.) Since A E 9:+ quently 9;+ c 9,. But (S;+)= 9,+ by (6.14), and so 9,+ c 9,. Thus Proposition 8.12 follows from (8.13). It evidently suffices to prove (8.13) ... for Y = fl'j=lfj X,,, with f j E bbA and 0 Itl < ... < t i It c < 1,. Such a Y can be written as (f15=l,fj X,,)(G 0,) where G = = + f j o X,, - ,. Consequently 0
0
fl;
0
j= 1
= Ep( Y
I 9;).
This completes the proof of (8.12). It is clear that whenever the hypotheses of Theorem 8.11 hold we can assume without loss of generality that A,+= d l
(8.14)
and
2,= A,.
Whenever (8.14) is in force, (8.12) also obtains and in this situation Theorem 8.7 appears more natural. However we will explicitly mention it when we assume (8.14). (8.15) REMARK. If X = (R, d,&,,,'A , 6 , , P") is strong Markov and if X , E 9,./b* whenever Tisan {F,} stoppingtime, then (R, 9,F I ,X , , 0,, P") is also strong Markov. In particular this will be the case whenever E A is a metric space (8,= W(E,) or B(EA)*)and X is right continuous.
Exercises
Let X = (Q, A, A , ,X , , O r , P") be a strong Markov process with state space ( E , 8).Let T be a stopping time and let G : R x R + R be in b ( 4 , x 9). Let H(m) = G(o,8,m). Show that (8.16)
E"{H 1 A , } ( W )
=
1G(o,
m')
PXT@)(dW')
for each x. (8.17)
Let X be as in (8.16) and assume further that X is 9 measurable.
44
1. MARKOV PROCESSES
Let T be a stopping time and S 2 0 be ATmeasurable. I f f € bbz, then E"{f(XT+,) I A T l ( 4 = ~ S ( m ) ( X T ( 4 * . f )
for all x where N , ( x , f ) = E xf ( X , ) . [Hint: suppose first that f E b b , and use (8.16) with G(o,a')=f [Xscm,(w')].] (8.18) Let E be a metric space and I, = a(EA). Let X = (Q, A, A , , X,,Or, P") be a right continuous Markov process with state space (E, a). For each x E E let T, = inf{t: XI # x}. Show that for each x there exists A(x), 0 5 A(x) Im, such that P"(T, > I ) = e-A(x)r.[Hint: consider instead h(t) = Px(Tx2 t) and use the fact that h(t) = Px(Tx> t ) except for countably
many values of
I.]
Let E = R" and I = W(R"). Let X = (Q, A, A,,XI, O,, P") be a Markov process with state space (E, I ) whose transition function is given by (2.12) and which has right continuous paths. (The existence of such a process for given {p,;t > 0} will be established in Section 9.) (a) Show that X is strong Markov with respect to {A,+}. (b) Show that P"(X, E B) = P o ( X , + x E B) for all x, t, and B E I. (c) Let T be a finite stopping time and let Y(t) = X(t + T) - X(T) for t 2 0. Let Y = a( Y , ; t E R+). Show that Y and f T are independent with respect to each PP.[Hint: begin by showing that for each p (8.19)
(8.20) Consider the process defined in (2.22). For each t 2 0 define Z,(w) = limsTIX,(w). Show that P ( 2 , # XI) = 0 for each t and hence that { Z , } is a Markov process (in the sense of Definition 1.1) having as a transition function the one defined in (2.21) (which we denote by N,(x, A)). Let 9:= ~ ( 2s, ;I t). Find an { f : , } stopping time Tand a bounded continuous f such that E f ( Z ~ + T#) E { N ~ ( Z T , f 9. Standard Processes
In this section we introduce the class of Markov processes with which we will be mainly concerned in the remainder of this book-the " standard " processes. In particular we will give conditions for the existence of a standard process with a prescribed transition function. We begin with some definitions of fundamental importance.
9.
STANDARD PROCESSES
45
(9.1) DEFINITION. Let X = (Q, 4, A,,X , , O f , P*) be a Markov process with state space ( E , 8).We assume that EA is a metric space and that &,= a(EA), the class of Borel sets in EA. Suppose that X , E A,/&,*for all {&,} stopping times T-this is just condition (S.R.) of (8.1). Under these conditions we say that X is quasi-left-continuous provided that whenever {T,,}is an increasing sequence of {A,} stopping times with limit T, then almost surely X(T,,) + X(T)on {T < [}.
It is useful to introduce a property somewhat stronger than (9.1) : namely, we will say that Xis quasi-left-continuous on [0, co) provided the convergence X(T,,)+ X ( T ) holds almost surely on { T < co} rather than just on { T < [}. In general quasi-left-continuity will refer to Definition 9.1 ; when confusion is possible we will refer to the property of (9.1) as quasi-left-continuity on [0, [) as opposed to quasi-left-continuity on [0, a).We are, of course, using the topology of EA in discussing the convergence of {X(T,,)}. We come now to the definition of a standard process. (9.2) DEFINITION. A normal Markov process X = (Q, A, MI,X , , O f , P") with state space (E, 8)is called a standard process provided : (i) E is a locally compact space with a countable base and A is adjoined to E as the point at infinity if E is noncompact and as an isolated point if E is compact. Furthermore &, is the a-algebra of Borel sets of EA (or, equivalently, 8 is the a-algebra of Borel sets of E). (ii) A,+= A!, = J Ifor all t . (iii) The paths functions t -+ X l ( o ) are right continuous on [0, 00) and have left-hand limits on [0, 5) almost surely. (iv) X is strong Markov. (v) Xis quasi-left-continuous.
REMARKS.Proposition 8.12 implies that, for a standard process, 9, + = 9, for all t. Also the assumption in (iii) that t + X,(w) has left-hand limits on [0, 4') almost surely is in fact a consequence of the other hypotheses (see Exercise 9.15). A standard process which is quasi-left-continuous on [0, CO) is called a Hunt process. Recall that a subset A of E is bounded provided that there exists a compact subset of K of E such that A c K . The next proposition implies that almost surely the left-hand limits of t --t X , on [0, [) must lie in E. (9.3) PROPOSITION.If X is a standard process, then for each t the set A ( w ) = {X,(w):0 I s I I, t < [ ( w ) } is almost surely bounded.
46
I. MARKOV PROCESSES
Proof. Let { K,,} be an increasing sequence of compacts in E such that K,, c K:, for all n and U K , , = E. (Here KO denotes the interior of K . ) Let T,, = inf{t: XI $ K,,}. Then T,, is an (9,) stopping time because EA - K,, is open. The sequence {T,,}is increasing and if T = lim T,,,then, because of the quasileft-continuity, XTn+ X , almost surely on {T c [}. However, by the right continuity of the paths X,,,+, is not in K,, , and so X , = A almost surely on {T < [} and, hence, also on R. Thus T = [ almost surely. The conclusion of (9.3) is now immediate because { X , ( o ) : 0 I s < T,,(w)}is contained in K,,.
We come now to the basic existence theorem for standard processes. This theorem will be used in the sequel only in the discussion of examples and so a reader might skip the details of the proof on a first reading. However, the techniques of the proof are of general interest. (9.4)
THEOREM. Let E be locally compact with a countable base and let
d be the Bore1 sets of E. Let P,(x, A ) be a sub-Markov transition function on (E, 8)with P o ( x , ) = 8,. Let Co denote the space of continuous functions on E which vanish at co and suppose that (1) P ICo c C, for each t 2 0 ,and (2) P,f -f uniformly on E as t 0 for eachfc Co . Then there exists a standard process (in fact, a Hunt process) with state space ( E , 8) and transition function the given P,(x, A). --f
REMARK. It follows from standard semigroup considerations (see Exercise 9.13) that, in the presence of Condition (l), Condition (2) is equivalent to the apparently weaker requirement that P , f +f pointwise as t + 0 for each fEC0.
Proof. Define E A as in (9.2i) and extend P,(x, A ) to a Markov transition function, N,(x, A ) on ( E A ,gA),by setting for A E d A N t ( x , A ) = Pt(x, A n E ) = lA(A),
+ I,(A)
(I
- P,(x, E ) ) ,
x E E,
x = A.
One checks immediately that N is indeed a transition function on (EA, 8,) with N,(x, EA) = I for all t 2 0, x E E,. Let C = C(E,) and observe that if f E C and if g is the restriction off -f(A) to E, then N , f ( x ) = P , g ( x ) + f ( A ) for x E E and N , f(A) =f(A). It follows from this that, for every f E C , N , f E C and that N , f +f uniformly on EA as t + 0. Also if f E C then N t + ,f - N , f = N,(N, f - f) 0 as s J. 0 and so t + Nt f ( x ) is right continuous. Therefore given a > 0 we may form U"f ( x ) = j; e-"' N , f ( x ) dt and U"C c C. Since aU"f(x>= j; e-' N,,, f ( x ) dt, it follows that aU"f +f uniformly as a co for each f E C. We may regard Co as the subspace of C
-
--f
9.
47
STANDARD PROCESSES
consisting of allfe C satisfyingf(A) = 0. With this interpretation UaCo c Co for each a > 0. In particular i f f € C, is strictly positive on E and if a > 0, then U'f is strictly positive on E. These remarks will be used in the course of the proof. Next let T = [0, a),W = EZ, and 4 = 8': so that ( W , 4)is the usual product measurable space. By Kolmogorov's theorem, (2.1 l), and the ensuing discussion there exists for each x E EA a probability measure P" on ( W , Y) such that over ( W , 9,P") the coordinate mappings X,(w) = w ( t ) form a temporally homogeneous Markov process in the sense of Section 2 with N as transition function and initial distribution E, . Let 4, = a(X,; s I t ) , and let Q denote the rationals in [0, 00). Let A consist of all those w in W which have the following two properties: (a) limsrt,SEQ w(s) exists (in EA) for each t > 0, limslt,sEQw ( s ) exists (in EA) for each t 2 0; (b) w(Q n [0,t ] )is bounded in E for every t E Q such that w ( t ) E E. Denote by A, and j \ b the set of functions defined by conditions (a) and (b), respectively, so that A = A, n A b . We now assert that A E Y and that P"(A) = 1 for all x E EA. To get started on this letfE C, and g = U a f , 01 > 0. Then
1
03
(9.5)
e-a' N,g(x) =
e-au
~ , j ( x du ) Ig ( x ) .
-1
Thus given 0 It < s,
rE
EX{e-asg(X,);
and x
E EA
we have, using (2.6),
r}= e-a-l Ex{e-'(s-r) N I E"{e-"'g(X,); r}.
s--1
g(X-1);l-1
Therefore for each x E EA the family {e-a-lg ( X , ) , Yt; t 2 0 } is a nonnegative supermartingale over ( W , 4, P"). Choosefc C, so thatfis strictly positive on E, and let g = CJaA01 > 0. Then, as noted above, g E Co and g is strictly positive on E. Observe that (Ab)e is precisely the union over all t E Q of the sets
But each Tr E and so Ab E Y. Moreover according to (1.6) of Chapter 0, P"(T,) = 0 for each x E EA and t 2 0. Therefore P"(hb) = 1 for all x E EA. Next let d be a metric for E, and define for E > 0, h,(x, y ) = 1 if d(x, y ) 2 E
48
I. MARKOV PROCESSES
and h,(x, y) = 0 if d(x, y) c E. If U is a finite subset of [0, 00) with an even number of elements, say u1 < .. . < u Z n ,define
Clearly He(U)E 93. For each D c [0, 00) define He(D) = sup He(U)where the supremum is taken over all finite subsets U of D with an even number of elements. If D is countable, then H,(D) is again in 93. Next observe that Aa =
fi {Hl/n(Q n LO, ml) < 001,
n=l m=l
and so Aa E 93. According to Theorem 1.5 of Chapter 0, for each f E C+ the restriction to Q of r + e-"'g(X,) has right- and left-hand limits at all points of [0, co) almost surely P" for all x . This assertion is then also valid for any f E C. (Of course a > 0 and g = U'j) Let A(a, f)denote the set of w's such that the restriction of t + g [ X , ( w ) ] to Q has right- and left-hand limits at each point of [0, co). By the above argument A(a, f)E Y and P"[A(a,f)]= 1 for all x E EA , a > 0, and f E C. Let {a"} be a sequence approaching co and let {f k } be a countable dense subset of C. Then {a, U""fk; n 2 1, k 2 1) is uniformly dense in C since aU'f +f uniformly as a --t 00 whenever f~ C. Now let A' = { w : for everyf E C the restriction o f t + f [ X , ( w ) ] to Q has rightA ( a n , h ) and and left-hand limits at all points of [0, 00)). Clearly A' = so P"(A') = 1 for all x. But it is also evident that A' = Aa and so the assertion that P"(A) = 1 for all x E EA is established. We now simply delete the set A' from W; that is, we define W = W n A , 93; = { A n A : A E ?I 93',} =, { A n A: A E Y}, and we take (P")' to be the restriction of P" to Y' and A',' to be the restriction of X,to W'. Clearly for each x , X,'is a Markov process with respect to (93;) over ( W', Y', (P")')having the original N as transition function. At this point we will drop the primes from our notation; that is, we will assume that A' is deleted from W so that W = A. Now for each t 2 0 and w in W set Z,(w) = lim X,(w). slt.seQ
Of course this limit exists since W = A. Furthermore it is easy to see that for each w the mapping t + Z,(w) is everywhere right continuous and has lefthand limits, and is such that Z,(w) = A whenever Z,(w) = Aand s 2 r. Clearly each 2, is in (Y,+)/cF'~.We next assert that for each x , { Z , ; t 2 0} is a Markov process with respect to {Y,,} over (W, Y, P") having N as transition function, and that P"{X, = Z , } = 1 for all r. Indeed suppose we are given x, r, E Y, , s > t with s - t rational, and f E C. Let {r,} be a sequence of rationals
9.
STANDARD PROCESSES
49
+
decreasing to t; then s, = s - t t, is a sequence of rationals decreasing to s - t. Now r E 9," for each n, and so using the Markov property for X
(9.7)
E"{f(X,); r}= EX{Ns-,f(Xrn); r>
for each n. But Xsn+ Z , and Xtn+ Z, as n -+ co while x + N,-,f(x) is continuous, and so letting n -, 00 in (9.7) we obtain (9.8)
EX{f(Zs);r>= E"{Ns-,f(Z,); l-1.
The restriction that s - t be rational is removed by observing that each side of (9.8) is right continuous in s. Finally, the validity of (9.8) for all f E C implies its validity for all f E b6,. Thus we have established that {Z,, Y, ; t 2 0} is a Markov process over (W, Y, P") with transition function N for each x . Now suppose that g E bd',, f E C, t E [0, m), and {s,} is a sequence of rationals decreasing to t . Then
~X{f(Xs.) g(X,))= E"{N,"-,f(X,) g(X,)), and letting n -, 00 we obtain EX{f(Z,dX,)) ) = E"{f(X,) g(X,)I.
It now follows from MCT that E"{h(Z,, X,)}= E"{h(X,, X,)} for all h E b(6, x 6J, and this implies that P"{X, = Z,} = 1 (take, for example, h ( x , y ) to be a bounded metric for E J . Of course we need this conclusion only for t = 0 since we already know that X and Z have the same transition A) for x in E and function N. We remark at this point that P"(2, E A) = Pt(x, A in 8. For the next step in the construction let R denote the set of all functions w from [0, co] to EA such that t -,w ( t ) is right continuous and has left-hand limits throughout [0, 00), w(00) = A, and w(s) = A if s 2 t and w ( t ) = A. By construction for each w the functions t + Z , ( w ) (with Z,(w) set equal to A) is in R. Let Y,(w) = o(t)and 9; = a( Y,; s 5 t) for all t E [0, a]. Define a mapping II : W - , R by II w ( t ) = Z , ( w ) and note that ~ - ' 9c:Y and c ;Y,, for each t E [0, m). Let px= P 3 r - l . Define translation that I I - ' ~ operators 8, on R by 8, w(s) = w(t + s). It is then easy to see that (0, S", 9;, Y,, O,, p) is a Markov process with state space (E, 8 ) in the sense of Definition 3.1 and having the given P,(x, A) as transition function. as in Section 5, then Y = Moreover if we form the "completions" S and 9, ( R , 9 , 9,+ , Y,, t?,, P")is a strong Markov process because the hypotheses = S,+ by (8.12). We of Theorem 8.11 are certainly satisfied. In particular 9, will drop the """ from bx at this point. We now assert that Y is quasi-leftcontinuous on [0, 00). Once this is established the proof of Theorem 9.4 will be complete.
50
I. MARKOV PROCESSES
To this end let {Tn}be an increasing sequence of stopping times with limit T. In trying to prove that YTn+ Y , almost surely on { T < co} there is no loss of generality in assuming that T is bounded, as we now do. Let L = lirn Y(TJ and for t > 0, L, = lirn Y(Tn+ t), these limits existing by the definition of Y. Since Tn+ t is in [T, T + t ] for all large n and t -+ Y, is right continuous, it follows that lim,io L, = Y,. Letfand g be elements of C.Then for each x
E"{f(L) S(YT)} = lim lim ~ " { f ( Y , , 9(YTn+,)> ) n-tm
1-0
= lim lim EX{f( Y,.) N , g( Y,,)} 1-0 n - t m
= lim 1-0
E"{f(L) N , g(L)l = EX{S(L)g(L)}.
An application of MCT then shows that E"{hW, Yd} = EX{h(L,L))
(9.9)
for all h E b(8, x 8,J,and consequently L = Y , almost surely. The proof of Theorem 9.4 is now complete. We will conclude this section by giving a condition under which the paths of the process constructed above may be assumed to be continuous on [0, c). In order to obtain a simple statement we will retain the assumptions of Theorem 9.4 and work with the process Y constructed in the proof of (9.4). (9.10) PROPOSITION. Assume the hypothesis of (9.4), and in addition suppose that for each compact subset K of E and neighborhood G of K, t-' P,( x , E - C) -+ 0 as t 0 uniformly on K. Then almost surely the sample functions t -, Y , are continuous on [0, 5). --f
Proof. Let d be a metric for E. By (9.3) the set {Y,(o):0 Is It , t c ((o)} is almost surely bounded. Consequently it will suf-lice to show that, for each x , each t > 0, each compact K c E, and each E > 0, (9.11)
P'[U'{d[Y(tk/n), Y(t(k k=O
+ l)/n)] > E } , Y([O,
t]) c
K]
approaches zero as n --f co. Define (9.12)
d t ,
K ) = SUP PLx, E XEK
- B,(x))
-=
where B,(x) = { y E E : d ( x , y ) E } . The reader will verify easily that the additional hypothesis in (9.10) (in addition to the assumptions of (9.4)) is equivalent to the assertion that t-' ctL(f, K) 0 as t -,0 for each E > 0 and --f
9. STANDARD PROCESSES
51
compact K c E. Now the expression in (9.11) is dominated by
as n -+ co, and so (9.10) is established.
Exercises (9.13) Verify the Remark following the statement of Theorem 9.4. [Hint: let P,(x, A ) satisfy the hypotheses of Theorem 9.4. First show that U"Co c Co and that L = LITois independent of LY > 0. (See (8.9) and the remarks 0 as t -+ 0 for all f~ L and hence following it.) Next check that llP, f -fll for allfE L', the uniform closure of L. Finally use the Hahn-Banach theorem to show that L' = Co.] -+
Show that any transition function of the form (2.12) satisfies the hypotheses of Theorem 9.4, and that if X is the resulting Hunt process one may assume that c(w) = 00 whenever X o ( o ) E RN.
(9.14)
Show that the condition that t + X , has left-hand limits on [0, 5) almost surely in the definition of a standard process is a consequence of the other hypotheses. [Hint: let d be a metric for E A . Show that for each E > 0, T = inf{t: d ( X o , X , ) > E } is an (9,) stopping time, and then consider the sequence of stopping times To = 0, T,,, = T, + T 0 &-".I Note that the same argument shows that paths of a Hunt process have left-hand limits (in Ed)on [0, co) almost surely.
(9.15)
(9.16)
Let E = R, d = g ( R ) , and define P,(x,
a)
= ex+, =+E"+,
if x 2 0 or x + t < 0, if x < O and x + t > O .
Verify that P,(x, A ) is a transition function, but note that UuCois not contained in Co. Prove that any standard process with state space ( E , 8) and this transition function will fail to be quasi-left-continuous on [0, 00). (The existence of a standard process with this transition function will follow from the results of Chapter 111.) Show that for the Brownian motion transition function in RN (see (2.17)) the hypotheses of (9.4) and of (9.10) are satisfied. The resulting Hunt process is called the Brownian motion process in RN.In view of (9.10) we may assume that all paths of this process are continuous on [0, co).
(9.17)
52
I. MARKOV PROCESSES
(9.18) Let X be a Hunt process with state space (RN,W(RN)) and a transition function of the form (2.12). Prove that if almost surely t + X,(w) is continuous, then the transition function must satisfy the condition of (9.10). [Hint: in this situation we must show that t-’ p,(Gc) + 0 as t + 0 where {p,} is the semigroup appearing in (2.12) and G is a neighborhood of the origin. Let {t,} be a sequence of numbers decreasing to 0 and let r, be the greatest integer in t,’. Use the equality
Po{IX(kt,) - X ( ( k
+ I)t,,)l s E for all k I r,} = [ p , , ( { x : 1x1 < E})]‘”
to deduce the desired conclusion.] (9.19) Let Xbe a standard process and define j ( t ) = sups t,xsE Ps(x, E A - {x}). Prove that if j ( t ) + O as t + 0 , then almost surely the sample functions t + X, are step functions with only finitely many jumps in any bounded t interval. [Hint: recall from (8.18) the equality Px(T, > t) = ,-Icx)‘ where T, = inf{t: X,# x}. Obtain the estimate Px{T, I t} I 2 j ( t ) by considering separately P”{T, I t ; X,# x} and P”{T, I t; X,= x}. Conclude from this estimate that A(x) is bounded. Define TI = inf{t: A’, # X,} and T,,, = T, + TI o ern. Prove that T, is a stopping time and that almost surely T, + 00.1
10. Measurability of Hitting Times In this section we will introduce certain stopping times which will be of central importance in the following chapters. Suppose X = (0,A, A , ,X,, Or, P”) is a Markov process with state space (E, 8).If A is a subset of E A we define two functions (10.1)
DA(o)
= inf{t 2 0: X , ( w ) E A }
TA(w)= inf{r > 0: X , ( w ) E A }
where in both cases the infimum of the empty set is understood to be + co. We call D A the first entry time of A and TA the first hitting time of A, It is obvious that these definitions are valid for any stochastic process A’. We often omit the adjective “first.” The following properties are immediate consequences of the definitions and we leave their verification to the reader. PROPOSITION (a) s + DA(Osw)= inf{r 2 s: X,(w) E A}, (b) s + TA(esO)= inf{t > s: X,(w) E A}, (c) t + D A 0, = DA on the set { D A2 t},
(10.2)
0
10. MEASURABILITY OF HITTING
(d) t + T, 0 O1 = TA on the set {T, > t } , (e) D, 5 TA and DA(o)= TA(w) if X , ( o )
TIMES
53
4 A.
It is sometimes convenient to refer to s + D, O,(s + T, o 0,) as the first entry (hitting) time of A after s. It is immediate from (10.2) that these are increasing functions of s and that 0
lim(s + D, 0 0,) = lim(s + T A 0 0,) = T, .
(10.3)
s10
s10
The next proposition contains additional elementary properties of DA and TA . PROPOSITION. Let A and B be subsets of EA. Then (a) A c B implies D, 2 D,; TA 2 T,; (b) D A U B = inf(D,, DJ; T A ~=Binf(T,, TB); DB); TAn B 2 TB); (c) D A n B 2 (d) if { A , } is a sequence of subsets of EAand A = A , , then DA = inf D,,; T, = inf TA,.
(10.4)
9
9
u
Proof. Since the first three statements are obvious we restrict our attention to the fourth, and in particular to TA. Clearly inf TAn2 T,. On the other hand if TA(o) < cx) then for each E > 0 there exists a t such that TA(w) I t < TA(o) E and X , ( o ) E A . But then X , ( o )is in some A , and so inf TA, ITA + E which completes the proof. (Choose t > 0 if TA(w)= 0.)
+
We are interested in formulating conditions under which D, and T A are stopping times for a large class of sets A . To this end we introduce the theory of capacity. See Brelot [I], Bourbaki [I], Dynkin [I], or Meyer [l] for a more detailed discussion. (10.5) DEFINITION. Let E be a locally compact separable metric space and let X be the class of all compact subsets of E. A function rp : X + R is called a Choquet capacity provided : (i) if A , B E X and A c B, then cp(A) Icp(B); (ii) given A E X and E > 0 there exists an open set G 3 A such that for every B E X with A c B c G one has cp(B) - rp(A) c E ; (iii) cp(A u B ) + cp(A n B ) I cp(A) cp(B) for all A , B E X .
+
Given such a capacity cp one defines the inner capacity, cp,(A), of an arbitrary set A c E by cp,(A) = supKcAcp(K) where the supremum is taken over all compact subsets of A . One next defines the outer capacity, cp*(A), of an arbitrary set A c E by cp*(A) = infG,, cp,(G) where the infimum is
54
1. MARKOV PROCESSES
taken over all open sets containing A. An arbitrary set A is said to be capacitable provided cp*(A) = cp,(A) and we define the capacity of A to be the common value, which we denote by cp(A). In view of (10.5) the notation is consistent; that is, if K E .fthenKis capacitable and cp,(K) = qn*(K) = qn(K). We are now ready to state Choquet’s fundamental theorem. We refer the reader to Brelot [l], Bourbaki [l], or Dynkin [l] for a proof. Also a very general discussion in an abstract setting will be found in Meyer [l].
(10.6) THEOREM(Choquet). Every Borel (more generally, analytic) subset of E is capacitable. We are now going to apply (10.6) to the study of hitting and entry times. We assume in the remainder of this section that E is a locally compact separable metric space, that A is adjoined to E as the point at infinity if E is noncompact or as an isolated point if E is compact, and that b(bA)is the cr-algebra of Borel subsets of E(EA). We further assume that the given Markov process X has state space ( E , b*) and is right continuous, quasi-leftcontinuous, and that M I 3 Str + . Consequently 9, = Ft+ .For convenience of exposition we will actually assume that t + XI(w) is right continuous for all o. This causes no loss of generality for we can always accomplish this by removing from R a fixed set in F having Pp measure zero for all p. The next theorem and Theorem 10.19 are the main results of this section.
(10.7) THEOREM. Under the above assumptions, DA and T A are {S,} stopping times for all Borel (more generally, analytic) subsets A of EA. We will break up the proof into several steps. First note that if D A is an {F,} stopping time then so is s + DA 0 Os for all s E R + (see the proof of (8.7)), and, consequently, in light of (10.3), we may restrict our attention to D A . (10.8) LEMMA. If G is open (in EA) then D G = T, and D G is an {9bp+} stopping time. Proof. that
This results from the right continuity of the paths, which implies {DG
U
{X,E G} E 9:.
reQn[O,t)
Of course, if t -+ X , ( o ) is only almost surely right continuous one can only assert that D G = T , almost surely and that D G is an {F,} stopping time. Until the proof of Theorem 10.7 is completed A, B, K , G, etc., will always
10.
MEASURABILITY OF HITTING TIMES
55
denote subsets of E. If A c E we define
(10.9)
LEMMA.D A is an {g,} stopping time if and only if dA is.
Proof. Since 5 is an {9:+ stopping } time the “only i f ” statement is immediate. On the other hand D A = dA on { d A < 5) and DA = 00 on {dA= 5) and so {DA
which is in 9, if dA is an
< I} = { d A < t }
{ d< ~ (}
{sf} stopping time.
For any A c E and t 2 0 let R,(A) = {a: X,(w) E A u {A}, for some s, 0 Is It } , R:(A) = { w : X,(w) E A for some s,
o Is I t } .
Obviously R,(G) and R:(G) are in 9:if G is open. Moreover for any set A , {dA< t } = U nRf-lI,,(A),and so to show that dA is an {Pt} stopping time it for all t . Finally note that one always has suffices to show that &(A) E 9, &(A) c { d A I t } . LEMMA.Let K c E be compact; then for all 1 ; (i) R , ( K ) E 9, (ii) if {G,,} is a decreasing sequence of open subsets of E with G,, 3 G, + 2 K for all n and such that K = nG,, = nc,,, then dG,fdK a s . and P’(R,(G,))JP”(R,(K))for all p and t .
(10.10)
Proof. Let {G,} be as in (ii); then {dGn}is an increasing sequence of {9:+} stopping times, and so its limit, which we denote by T, is an {9:+ stopping } time. Clearly T 5 d, I 5. If T = 5 then T = d K , while on { T < C}, dGn= DGn for all n and X(Dcn)+ X ( T ) a s . on { T < 5) by the quasi-left-continuity. But X(DGn)E G, on { DGn< a},and consequently X ( T )E nG, = K a s . on { T < 5). Hence d , IT a s . on { T < 5). Therefore dGnfdKa s . and so dK is stopping time. an {F,} If o E {dK I t } , then either X,(w) E K u {A} for some s Ir or there exists a sequence {s,,} decreasing to t with X J o ) E K u {A} for all n. But K u {A} is closed and so the right continuity of the paths implies that in this case X,(w) E K u {A). In other words {dK5 t } c R,(K) and hence these two sets are equal. Therefore R , ( K )E 9,. Finally {dGnIt } decreases t o {dKIt } a.s. as n + 00, and since
56
1. MARKOV PROCESSES {dG,
I t}
2
R,(G,,) 3 R , ( K ) = {dK I t )
it follows that P’[R,(G,,)] 1P’[R,(K)]. Thus Lemma 10.10 is established. (10.11) REMARK.If X is assumed quasi-left-continuous on [0, 00) rather than on [0, c), then one sees by exactly the same argument that (10.10) remains valid if we replace R by R* and d by D throughout. (10.12) LEMMA.Let p and t be fixed; then cp(K) = P’[R,(K)] is a Choquet capacity on the compact subsets of E.
Proof. We must check that the three properties of Definition 10.5 are valid. Property (i) is obvious. Property (ii) is an immediate consequence of Lemma (10.1Oii). As to Property (iii), if A and B are compact, then
R,(A u B) - R,(W ={o:X,(cu)~A uB
u {A} for some s I t butX,(o)#Bu{A} f o r a n y s s t}
c { w : X , ( o ) E A for some s I t but X,(w) # (A n B) u {A}
for any s I t )
= R,(A) - R,(A n B), and therefore cp(A u B) - cp(B) I cp(A) - cp(A n B). Hence cp is a Choquet capacity. (10.13) REMARK.Under the assumption of (10.11) the set function K + P’[R:(K)] is a Choquet capacity for all p and t. (10.14)
LEMMA.For any Borel set B, R,(B) E 9, .
Proof. Let cp be the capacity cp(K) = P’[R,(K)]. If G is open and K is any compact contained in G, then q ( K ) = P’[R,(K)] I Pp[R,(G)]and hence cp,(G) = supKc cp(K) 5 P’[[R,(G)]. Let {K,,} be an increasing sequence of compact subsets of G whose union is G. Clearly R,(K,,)fR,(G) and so cp(K,,)tP”[R,(G)]. Thus cp,(G) = P’[R,(G)] for any open set G.If B is a Borel set, then by Choquet’s theorem (10.6), cp,(B) = q*(B). Hence for each n there exist a compact set K,,c B and an open set G,, 2 B such that P*(Gn) - d K n ) = P’CRr(Gn)I - P’CRt(Kn)I < I/n*
Let A1 =
u,,R,(K,,) and A 2 n,, =
R,(G,,); then A1 and A2 are in 9, and
10.
MEASURABILITY OF HITTING TIMES
57
A1 c R,(B) c A 2 . But P’(A2 - A1) I P”[R,(G,,)]- Pp[R,(K,,)]c I/n for each n, and hence Pp(A2- A1) = 0. Since p is arbitrary it is now immediate that R,(B) E 9, *
REMARK. The proof of (10.14) actually shows that q ( B ) = P’[R,(B)] for any Borel set B where q ( B )is the capacity of B. Finally note that under the assumptions of (10.1 1) if $ ( K ) is defined to be P’[R:(K)] then again qi is a Choquet capacity and $(B) = P’[R:(B)] for any Borel set B. (10.15)
Theorem 10.7 is now an immediate corollary of the above lemmas. Thus stopping times for any Borel set we now know that T, , D A ,and dAare {9,} A. However useful this information may be, the following approximation theorems are even more useful. THEOREM.If B is a Borel subset of E, then for each p there exist an increasing sequence {K,,}of compact subsets of B and a decreasing sequence {G,,} of open sets containing B such that d K n J. d , and dGnt d , almost surely P p .
(10.16)
Proof. Let p be given. Since q ( B ) = P’[R,(B)], there exists for each t 2 0 an increasing sequence { K i } of compact subsets of B such that Pp[R,(Ki)]f P[R,(B)].Let {q,} be an enumeration of the nonnegative rationals. Define K,, = K,4’ v . . . v K P . Clearly {K,,} is an increasing sequence of compact subsets of B, and so {dKn}decreases to a limit T 2 d,. Now {d, < T} = {d, < q,, < T} and for each m
u,,
{ d , < q n < TI c { d <~ q n < d ~ , } c
Rq,(B)
- Rq,(Km)*
But if m > n then K,,, 2 K Z and so
Id, < 4. < TI c Rq,(B) - Rq,(K%)* Consequently Pp[dB< q,, < T] = 0, and this implies that T = lim dKm = d, almost surely P’. The required sequence {G,,} of open sets is constructed in a similar manner. See Exercise 10.22. COROLLARY. If B is a Borel subset of E, then for each p there exist an increasing sequence {K,,} of compact subsets of B and a decreasing sequence {G,,} of open sets containing B such that DKnJD , almost surely P’ on R and DG,f D , almost surely P” on { D , < m}. (10.17)
Proof.
By (10.16) there exists such a sequence { K,,} with dKn1d, almost surely
58
I. MARKOV PROCESSES
P’. Now on {D, < co} = {D, < [} one has d, = DBand hence dKn< [ for n sufficiently large. Rut then dKn= DKnandconsequently DKn1DBon {D, < co} a.s. P”.But on {D, = co} each DKn = co since B 3 K,, . Again from (10.16) there exists an appropriate sequence {G,,} with dGn7 dB= D , A [ as. P”, and on {D, < oo} = { D , c c} one has dCn= DGn < [. Therefore DG,t D, a.s. P p on {D, < 00). (10.18) REMARK.If X is quasi-left-continuous on [0, 00) then making use of (lO.ll), (10.13), and (10.15) one can repeat the proof of (10.16) to obtain for a given p a decreasing sequence {G,,} of open sets containing B such that DGnt DBalmost surely P”. (10.19) THEOREM.If B is a Bore1 subset of E then for each p there exists an increasing sequence {K,,} of compact subsets of B such that TK, 1TBa.s. P”. Proof. Let { f k } be a strictly decreasing sequence of positive numbers with limit 0. We know that t k D, 0 Or, 1TA as k + co for any set A. Suppose that we could find an increasing sequence {K.} of compact subsets of B such that DKn 0 Of, 1D, elk a s . P” as n + 00 for each k. Then
+
0
+ DKn O,k) = lim lirn ( t k + DKn elk) n k
TB= lirn lim ( f k k
0
n
o
= lirn
TKn a.s. P”,
n
where the interchange of limits is justified since the double sequence in question is decreasing in both n and k. Thus the proof of (10.19) is reduced to constructing a sequence {K,,} with the above property. For the given p define for A E d vk(A) = l p ( d x ) p”[X(tk) E A ]
= P”[X(fk)E A ] .
According to Corollary 10.17 there exists for each k an increasing sequence {K;} of compact subsets of B such that DKk 5. DBa.s. Pvk.Define K,, = K,’ u K: u . .. K; . Clearly {K,,} is an increasing sequence of compact subsets of B. Moreover K,k c K,, c B if n 2 k and so D, I DKn I DKk for n 2 k. Thus (which exists since {K,,}is increasing), DKn1D, a.s. Pvkfor all k. If T = lim,,DKn then we have shown so far that Pvk[[T# D,] = 0 for all k. Clearly DKn 0 O,, 1 TOO,, asn+coand P ~ T elk z D,
elk]= E” P ~ ( ‘ ~ )z( D,) T = Pvk(T# DB) = 0.
10. MEASURABILITY OF HITTING TIMES
59
Therefore DKn0 Or, 1DB 0 Or, as n -,co a s . Pfl for all k, and thus the construction is complete. Let us emphasize that the approximating sequences constructed in (10.16)(10.19) depend in general on the measure p. See Exercise 10.24 for a sharpening of (10.19) under an additional hypothesis. The only place that we have used the Markov property in the present section is in the proof of (10.19). We should also point out that the analogous approximation theorem by open sets Gn 3 B is not valid in general. A simple example is given by uniform motion to therightalong thereal axis(see(3.7)). IfB = {O}thenPo(T, m) = 0, while if G is any open set containing 0 then Po(TG= 0) = 1. In the remainder of this section we will assume that Xis a standard process although somewhat weaker hypotheses would suffice. The above discussion of course applies to any standard process.
-=
PROPOSITION.Let X be a standard process and suppose that B is a Borel set. Define
(10.20)
R;(B) = R , ( B ) LJ {w : X , - ( w ) exists and is in B u {A} for some s, 0 I s It } .
Then Pp[[R;(B)- R,(B)]= 0 for all p. In particular R;(B)E 9,. Proof. Clearly R,(A) c Ri(A) for any set A . For the given B and p there exists a decreasing sequence {G,,} of open sets containing B such that P’[R,(C,,)- R,(B)]I l/n for each n. Also for any open set G, R,(G) = R;(G). Therefore c RKB) c MGn) n
and Proposition 10.26 now follows since the extreme members in this string of inclusions have the same P p measure. Let c’(o) = inf{r: X,(w)= A or X,-(w) exists and equals A}. Clearly
5’ I iand
{c’ < r < (}
c {s
-+
X , is unbounded in E on [0, r ] , r < i}.
Consequently according to (9.3), = I’ almost surely. Combining this with (10.20) one easily sees that almost surely D , = inf{t: X , E B or X , - exists and is in B } for any Borel set B. In many situations it is necessary to deal with a class of sets that lies between the Borel sets and the universally measurable sets. (10.21) DEFINITION.A set A c EA is a nearly Borel set (relative to the given process X ) if for each p there exist Borel subsets B and B’ of EA such that
60
I. MARKOV PROCESSES
B c A c B and Pj"X, E B' - B for some t 2 01 = 0. This is equivalent to P"[DB,- < 001 = 0. Roughly speaking a set is nearly Borel if the process can not distinguish it from a Borel set. Clearly the above definition makes sense for any process and not just for standard processes. The reader should verify for himself that the class of nearly Borel sets is a o-algebra in E A . We denote it by 8;.The o-algebra of nearly Borel subsets of E will be denoted by 8". Obviously 8 c 8"c 8* and a similar relationship holds for EA. A functionfE 8"is called nearly Borel measurable. Such functions are characterized by the property that for each p there exist fi,f2 € 8 such thatf, 5 f If2 and Pp[cfi(X,)# j 2 ( X , ) for some t 2 01 = 0.
It is easy to check that all of the results of this section which were proved for Borel sets extend to nearly Borel sets, and we will use them for nearly Borel sets without special mention. We leave the details to the reader.
Exercises Give a detailed construction of the sequence of open sets {G,} in Theorem 10.16.
(10.22)
Let X be a standard process having as transition function the one defined in (9.16). Let B = (0).Show that if p is a measure with p[( - 00, O)] > 0, then there is no sequence {G,} of open sets containing B and such that DGn + DB almost surely P '. Consequently (10.17) can not in general be strengthened. (10.23)
(10.24) Let X be a standard process with transition function P,(x, A) and suppose that there exists a o-finite measure 1 on 8 such that for each x in E and t > 0 the measure P,(x, .) is absolutely continuous with respect to 1. Show that if B is a Borel set then there is an increasing sequence {K,,} of compact subsets of B such that TK. decreases to T,, almost surely. [Hint: let p be a finite measure equivalent to A and choose the sequence {K,} such that TKn4 TBalmost surely P".] (10.25) Let X be a standard process and let A be a nearly Borel subset of E. Let T = T,, and let a(t) = supxEEP"[T > t ] . Show that for each x E E and t > 0 one has E X ( T )s r[l - a(t)]-'. [Hint: if a(r) = 1, there is nothing to
11. FURTHER PROPERTIES OF HITTING TIMES
61
prove. If a(t) < 1, show that P"[T > n t ] I [a(t)l", and then write E"(T) = PX(T2 s) ds.] Prove that the class of nearly Borel sets in contained in 82.
(10.26)
EA
forms a a-algebra
11. Further Properties of Hitting Times Throughout this section X will denote a given standard process with state space ( E , I ) . If T is any { M I }stopping time and a 2 0 we define the kernel P; as follows: Iff E b b z then
PTf(x) = E"{e-"Tf(XT);T c a}. In particular if f E bb*, then P+ f ( x ) = E"{e-aTf(XT); T < 5). If T is an {AI} stopping time this need not be a measurable function of x. However if T is an {SI}stopping time then P; is a bounded linear operator from bb,* to bbz (or from bb* to bb*). As usual the definition extends to any nonnegative function in 82.If a = 0 we write PT in place of PF . If T = TA where A E 8; we write P i in place of P;", and P A when a = 0. The measure P i ( x , * ) is called the a-order hitting distribution of A starting from x or the a-order harmonic measure of A relative to x. When a = 0 we drop it from our terminology as usual. If A is a nearly Borel subset of E, then the right continuity of the paths implies that all of the measures P i ( x , * ) are concentrated on A (the closure of A in E ) . In actual fact one can make a good deal more precise statement than this, and we next develop the necessary machinery for this more precise statement. If T is an {PI}stopping time then { T = 0) E So,and so, for each x E E A , P"(T = 0) is either zero or one according to the zero-one law (5.17). The point x is said to be irregular for T if this probability is zero and regular for T if this probability is one. (11.1) DEFINITION. If A
E 8; then a point x is regular for A provided it is regular for T A ,and x is irregular for A provided it is irregular for TA .
Thus x is regular for A if P"(TA = 0) = 1 and irregular for A if P"(T, > 0) = 1. Intuitively x is regular for A if the process starting from x is in A at arbitrarily small strictly positive times with probability one. If A' is the set of all points which are regular for the nearly Borel set A, then A' = {x: P"(TA= 0 ) = 1 ) and consequently A' is universally measurable. In
62
I. MARKOV PROCESSES
fact A‘ is itself nearly Bore1 but we cannot prove this until later. If A’ denotes the interior of A, then plainly A’ c A‘ c A. We can now complete the approximation theorems given in Section 10. THEOREM.Let A€&’” and let p be a finite measure such that p(A - A‘) = 0. Then there exists a decreasing sequence {G,,} of open sets containing A such that TGn1TA almost surely P p on {TA < co} and TGnA l t TA A ( almost surely P p (on Q).
(11.2)
Proof. According to (10.16) and (10.17) there exists such a sequence of open sets for which a s . P” one has DG,,fDA on {DA < 03) and D,,, A 5 1DA A l. But for any open set G, DG = T, , and so to complete the proof we need only show that P’((DA TA) = 0. But
+
P”(DA < T A ) = J p ( d x ) P”(DA < TA). If x E A‘ then P”(DA = TA)= 1, while if x E A‘ then 0 I DA I TA = 0 almost surely P x . Since p attributes all its mass to A‘ u A‘ it follows that P’(DA < TA)= 0 and so the proof of (1 1.2) is complete. REMARK.If X is quasi-left-continuous on [0, co), then using (10.18) and the above argument one can strengthen the conclusion of (1 1.2) to TGn T A almost surely Pr.*
(11.3)
The next result gives us the precise information alluded to earlier regarding p x x , .). THEOREM.Let A E 8;; then (i) X(TJ E A u A‘ a s . on { T A < a}; (ii) for each x and u, P X x , * ) is concentrated on A u A’.
(11.4)
Proof. We will prove only (i), for (ii) is an obvious con ,equence of (i). Since X , 4 A for 0 < t < TA it is the case that if X,, 4 A then ‘’‘A + TA 8,, = TA . Consequently for any p 0
P”(TA < 00, X T ,
4 A ) = P”(TA
< 00, X , , 4 A ) = E’{pX‘TA’(TA = 0 ) ; T A < 00, X T , 4 A } . OT,
= 0,
TA
Therefore almost surely P’ on {TA < 00, X,, 4 A} we must have
* Theorem 6.1 of Chapter 111 contains another important extension of ( I 1.2).
12. REGULAR STEP PROCESSES
63
p x y q = 0)= 1 ; that is, X,,
E A‘.
This clearly yields Statement (i) of Theorem 11.4.
Exercises (11.5)
If Tand R are stopping times show that X ,
(11.6) Let A E 8: and a > 0. Prove that x (11.7) Let A
E 8:. Prove that if x
E A‘
o
8, = XR+T.BR
if and only if P;(x, E,J = 1.
4 A then x E A‘ if and only if P,(x,
-)=
8,.
(11.8) Show by an example that the condition “ x $ A ” in (11.7) cannot be eliminated. [Hint: consider the Hunt process “ uniform motion around a circle.” The definition of this process should be clear by analogy with Exercise 3.7.1 (11.9) Let A and B be in 8: with A c B. Show that if X,, $ A - A‘ almost surely on { T , > 0} then P i p : = P: for all a 2 0, and conversely that if P i P: = P i for some a > 0 then X,, $ A - A‘ almost surely on { T B> O } . Show that P:Pi = P i for all a 2 0 provided that X,, E B‘ almost surely on { T , a}, and that if P i P i = P: for some a > 0, then X,, E B‘ almost surely on { T A< a}.
-=
12. Regular Step Processes
In this section we will introduce an important class of Markov processes. However, the material in this section will not be needed in the sequel except as a source of examples, and so the reader may omit a detailed reading of the proofs. The processes to be constructed have a very simple intuitive description; however, we will go into some detail regarding the construction. By a Markou kernel Q on a measurable space ( E , 8) we mean a function Q(x, A ) defined for x E E and A E d such that (a) for each A E 8,x + Q(x, A ) is in I , and (b) for each x E E, A + Q(x, A ) is a probability measure on 8. If in (b) above we only assume that, for each x E E, A + Q(x, A ) is a measure on & with Q(x, E ) I 1, then the resulting object Q is called a sub-Murkov kernel on ( E , &). We now fix a measurable space ( E , d), and in this section we will assume that { x } E d for each x E E. The basic data from which we will construct the process are (i) a function , IE d satisfying 0 < A(x) < co for all x E E, and (ii) a Markov kernel Q(x, A ) on (E, 8)satisfying Q(x, { x } ) = 0
64
I. MARKOV PROCESSES
for all x. We caD now give an intuitive description of the process to be constructed. A particle starting from a point xo E E remains there for an exponentially distributed holding time T, with parameter A(xo) at which time it "jumps " to a new position x1 according to the probability distribution Q(xo, .). It then remains at xi for a length of time T, which is exponentially distributed with parameter A(xl), but which is conditionally independent, given x,, of T,. Then it jumps to X, according to Q(x,, .), and so on. Most of this section is devoted to a rigorous construction of such a process. Let F = E x R, and 9 = d x g ( R + ) , where R, = [0, 00) as usual. Let N = (0,1,2, . ..}, and set R = F N and Y = fN. Thus (F,9) is a measurable space and (R, Y) is the usual infinite product measurable space over (F, 9). The points o E R are sequences {(x,,, f,,); n 2 0} with x,, E E and t,, E R+ for all n. If o = {(x,,, t,,); n 2 0) let Y,,(o)= (x,,, t,,), Z,,(w) = x,,, and T,,(o) = r,, . Thus Y,, is the nth coordinate map and Y,, = (Z,,,z,,). Obviously Y,,E 3/9, Z,, E Y/ 8, and z,, E Y for each n. We next define a Markov kernel n on the measurable space (F, 9)as follows :
Note that n is translation invariant in the R, variables. We now invoke a theorem of Ionescu Tulcea (see Doob [l, pp. 613-6151 or Neveu [I]) which states the following in the present situation: if y is a probability measure on (F,F),then there exists a probability measure P y on (R, 9)such that the coordinate mappings { Y,,; n E N} form a (temporally homogeneous) Markov process over (R, Y, Py)with y as initial measure and II as transition function; that is,
for all A E 9 and n 2 0. If y is unit mass at (x, 0) E F we will write P" for Py. One easily calculates that for each y EY{exPC-a(zn+1 - zn)l} = E'{A(zn)/Ca + 4Zn)l}, and hence letting a + 00 we see that Py(z,,+, 5 r,,) = 0. Moreover from the assumption that Q(x, {x}) = 0 for all x it easily followsthat Py(Z,,+l= 2,) = 0 for all n and y. Finally Px(zo = 0) = 1 for all x. Consequently if R' = {Z,,,, # Z,, and ,z, > T,, for all n and r0 = 0}, then R' E Y and P"(f2') = 1 for all x. If Y' denotes the trace of Y on R', then, for each x, { Y,,; n E N} is a Markov process over (R', Y', P") with E , , ~ as initial measure and IT as transition function. We now drop the prime from our notation; that is, we now let R denote the set of all sequences {(x,, , r,,); n 2 0) with 0 = t o < z, < . . . and
12. REGULAR STEP PROCESSES
65
x , + ~# x, for all n 2 0, and let 9 be the o-algebra in R generated by the coordinate mappings { Y,;n E N}. The measures P" are now regarded as probability measures on this (R, 9). Finally we are ready to define the process which interests us. First let [ ( w ) = limnT,,(w)and then define for t 2 0 X,(w) = Z,(w) (12.3)
=A
if q,(o)I t < z,+ l(w), if [(w) I t.
where A is a point adjoined to E in the usual manner. Of course we set X,(o) = A for all w. Clearly t + X,(w) is a step function on [0, [(w)) for each o.Consequently if E is given the discrete topology then t + X , ( o ) is right continuous on [0, 00) and has left-hand limits on [0, [(w)) for each o. Moreover X , E for each t. We adjoin a point oAto R and set X,(W,) = A for all t. We put {wA}E 9 and set P X ( { w A }= ) 0 for all x E E. Also we set Z,(W,) = A and ?,(@A) = 00 for all n. Next we define translation operators as follows: e,oA= oAfor all t ; if o = {(x,, t,,);n 2 0) and t 2 [(o) = limn t, then 8, o = w A , while if tk I t < t k + l , k 2 0, then 0,w = {(x,+~,[ t n + k- t ] v0); n 2 O}. One checks immediately that X , o 8, = X , + , for all s and t. Finally we let P A be unit mass at oA. We now come to the main result of this section. THEOREM.X = (R, 9,X,,01,P") is a Markov process with state space (E, 8).
(12.4)
Proof. We will carry out the proof in a series of observations and lemmas, leaving detailed verification to the reader. First of all let 9,= a( Y,; 0< n) and let 9:= o(X,; s I t). We assert that if A E 9:, then for each n there is a set An E 9,such that A n {t, I t < z , , + ~ }= A,, n {t < T , + ~ } .
Indeed, the class of sets A with this property is a a-algebra and obviously it contains the sets { X , E A} with A E 8 and s I t. We pause to introduce some additional notation. If u 2 0, we define for xEEandAE8 (12.5)
Q.(x, A) = -Q(x, A ) .
+ A(x)
If K(r, B), r E C,B E g,is any sub-Markov kernel on a measurable space (C, %) we define the iterates, K", of K by Ko(r, B ) = E,(B)and K"+'(r, B) = K(s, B ) K"(r,ds) for n 2 0. In the present case, and with this notation, a simple induction argument shows that
66
I. MARKOV PROCESSES
jOae-_ nn(x, 0 ; A , ds) = Q X ~A,)
(12.6)
for all u 2 0, n 2 0, x E E, and A E 8 . It also follows by induction that nn is translation invariant in its R+ variables. The following lemma contains the basic calculations which we will need. The proof is a straightforward calculation which we will leave to the reader. It makes use of (12.6) and the Markov property for { Yn}(that is, the fact that the finite-dimensional distributions of { Y,} can be written down in terms of the iterates of n).
(12.7) LEMMA.Let g E 8, and a 2 0. Then for each x (i) EX{@,) [e-"'* - e-"rk+l]I Sn}
(ii) EX{e-Uh+I
d z n + l ) ; rn+1 > 11Sn)
= QAzn9 9 ) expC4zn)
rn
- (a + l(zn))(tv
~n)],
for each n 2 0 and t 2 0. Next given u 2 0 and f~ I +(with f extended to A by f(A)= 0) we define U"f(x) = Ex
j
m
e-"'f(X,) dr.
0
Since by definition m
joe-"'f(X,) dt =
c
e-"'f(X,) dt 0
an application of (12.79 yields (12.8)
(12.9) LEMMA. I f f € 8, , t 2 0, a 2 0, and x E E, then
Ex(J)me-a"f(XJ19:) = e-"' U " f ( X , ) .
12. REGULAR STEP PROCESSES
67
Proof. We must show that for A E F G I )we have
I,
m
(12.10)
Ex(
e-auf(Xu) d u ; A
e-af Uaf(X,); A).
c}
c}
We need not concern ourselveswith {t 2 and so we may write A n {t < = u,,(A n {t,,S t < 7,+1}). But A n ( 7 , I t < T , , + ~ } = A,, n {t < T , , + ~ } with A,, E Y,, and An c (7,s t}. Thus t o establish (12.9) we need show only that (12.10) is valid when A is replaced by A,, n {t < T ~ + ~ with } , A,, E Y,, and A,, c (7, I t}. With this replacement we may calculate the right-hand side of (12.10) using (12.7ii) with c1 = 0 and g = 1. The result is e-af Ex(e-Uzn)(f-rn) U af (z 1. I\ ). (12.11) n , n We calculate the left side of (12.10) by writing the integral as + cp=n+lJ:;+I. Using (12.7ii) twice, the first term becomes
e-"'E"{f(Z,,) [c1
+ A(Z,,)]-'
Jp+'
e-A(zn)(f-'n); A,,}.
For the second term we use (12.7i) and (12.7ii) in that order to obtain
Combining these calculations with (12.8) we obtain (12.1 1) for the left side of (12.10) also. Hence Lemma 12.9 is established. The conclusion of Theorem 12.4 is now immediate. Indeed if f E b l + , then Lemma 12.9 implies that
whenever rp is a linear combination of exponentials and hence, by uniform approximation, whenever rp is continuous and vanishes at a.Since f(X,,) is right continuous and bounded, given s 2 0 one can choose a sequence {rp,} such that upon replacing rp by rp,, the left side of (12.12) approaches E"{f(X,+,) I G.' I)} while the right side approaches Ex('){f(X,)}. Thus (3.2) holds and so the proof of Theorem 12.4 is complete. Observe that 71 = inf{t: X , # X,,} and hence 71 is an {F:+} stopping time. Moreover T n + l = T,, + 71 o 8," and each 7, is an {F:+} stopping time. If T is an {PI}stopping time, then X , = Z,, on (7, I T < T . + ~ } and so X , E.A&/'. Iff is any bounded function on E,, then t - f ( X f ) is right continuous since t + XI is right continuous when E, is given the discrete topology. Therefore the proof of Theorem 8.1 1 can be repeated essentially word for word to show that X is a strong Markov process relative to {Fr+}.
68
I. MARKOV PROCESSES
Finally let us show that X is quasi-left-continuous when EA is given the discrete topology. Let {T,,} be an increasing sequence of {Fr} stopping times with limit T. We may assume that T I [. In view of the form of the path functions it is clear that X,,,+ X , for all o in { T < c} {T = z,}. Consequently for a given path f -+ Xr(o)with T ( o )< [ the convergence of X,,, to X , can fail only if for some k we have T = z,+~ and T , I T,,< T , + ~ for all large n. Let k and an initial measure p be given. On the set { T , I T,, < z , + ~ } we have 7 k + l = T,, z1 0 0,. and X(T,,)= 2,. Thus if A = { T k IT,, < z , + ~ for all large n}, then
u,
+
P"(T = z k + l ;A) = lim Efi(e-"('k+I-T) ;A) a+ m
lim lim inf Efi(e-"(zk+l-Tn). a+m
9
zk
n
= lim lim inf EQ(e-"'IoeTn; 7, I a+m
5
Tn
< 7k+l)
T , < zk+1)
n
Ilim E"{(EZk(e-"")} (1'
m
This establishes the quasi-left-continuity of X. If E is a locally compact space with a countable base and B is the Bore1 sets of E then it is an immediate consequence of the above results that X is a standard process.
Exercises (12.13) Given a > 0 and x E E show that a necessary and sufficient condition that P"(C = 00) = 1 is that Qx x , E) -P 0 as n + a.Conclude from this that if 1 is bounded then, for all x, P"([ = 00) = 1. [Hint: use (12.6).] (12.14) Show that relative to each Pyif h E bB, then z,,+~ - z, Z,,+l, and h are conditionally independent given Z,, . (12.15) Let E be locally compact with a countable base and let d = %?(E). Show that if 1 is a bounded continuous function and Q maps C, into C, , then, for each a > 0, U"maps C, into C o wOne may replace C, by C in the hypothesis and conclusion. [Hint: show that if Q maps C, into C, , then so does each Q:. Then use (12.8).]
II EXCESSIVE FUNCTIONS
1. Introduction
Most of the paragraphs which follow have as fixed data a given standard process X = (Q, A, A , ,X,, O f ,P") with state space (E, 8). Often the definitions and theorems involve only the action of the corresponding semigroup {P,}of transition operators and nothing more about X. However our viewpoint is that the process Xas well as the semigroup { P I }is of fundamental interest, and so we will not attempt to separate statements involving only the semigroup from those which also involve the process. When starting with a standard process X it is to be understood that objects such as nearly Bore1 sets, potential operators U", hitting operators P ; , and so forth are defined relative to X. Frequently we will make calculations of the following sort: Let T be an { A , }stopping time, f~ S*, , and a a nonnegative number. Recall that m ~ a j ( x= )
e - a ' j ( x , ) dt.
EX
0
We will refer to U y a s the a-potential o f f . As usual when u = 0 we will drop it from our notation and terminology. Observe that U ( x , B) = U o ( x ,B) is the expected amount of time that the process starting from x spends in B. The integral on r in the above expression for U a f ( x )can be written + j?. If T(o) 00, then using the change of variable t - T = u the second integral becomes
-=
69
70
11. EXCESSIVE FUNCTIONS
and this is valid also if T = co since f(A) = 0. The function in braces is an element of .F and so by the strong Markov property (1.1)
E x /Tae-arj(X,) dt = Ex(e-aT E X ( T /oae-atj(X,) ) dt) = EX{e-aTvaj(XT)>
= PTU"f(x).
Consequently
I
T
(1.2)
~ a j ( x=) E X
e-a'j(X,) dt
+
~ + ~ a j ( x ) .
0
This shows, for example, that P ; V " f l U"fand that P;V"fdecreases as T increases. Generally we will simply write down the results of such calculations, leaving the intermediate steps to the reader.
(1.3) THEOREM. (a) I f f € 8: and A is nearly Borel, then P i U " f l U l f . (b) If in additionfvanishes off A, then PiU"f= Vlf. (c) Iff and g are in S*, with f'vanishing off the nearly Borel set A and U " f i Vag on A, then U " f l V'g everywhere. Proof. The first two statements are immediate consequences of (I .2) applied to the stopping time TA, We will prove (c) as an application of some of the results from Chapter I. If K is a compact subset of A then Pt;U"fl; Pt;U"g, for the measures Pg are carried by K. Fix x and let {K.} be an increasing sequence of compact subsets of A such that TK,1TA a s . P". If B is nearly Borel, then
Ex /T:e-a'f(X,) dt = Pf,U"j(x), and so Lebesgue's monotone convergence theorem shows that Pt;,U"f(x) PplU"f(x). Then from (a) and (b) we have
t
U a j ( x )= P i U " j ( x ) = lim P ; m V a j ( x )s lim PknU' g(x) n
n
5 U"g(x),
which establishes (c). For a > 0 we write Pp = e-"' P , . Note that this agrees with our definition of P; if T = t. The reader should note that, for each a 2 0, { P : ; t 2 0} is a semigroup of bounded operators on bS (or bS*) whose resolvent { Va; /? > 0)
1. INTRODUCTION
71
is given by V B= U a + B In . particular V o = U" is a bounded operator if U a = J0"Pp d f . This notation will be used in the sequel without special mention. u > 0. Note that we now have
Exercises (1.4)
Prove that iff E bB* and T is an {A,}stopping time then
/+
00
~ P ~ , U Q ~ (=X E)X
f
e-a"f(x,,) du T.&
for all x , f , and u 2 0. Let X be the Brownian motion process in R" (see (2.17) and (9.17) of Chapter I). (a) Prove that if n 2 3 the potential operator for this process is given by
(1.5)
for f 2 0, where the integral is over R" and 'd denotes integration with respect to n-dimensional Lebesgue measure. (The precise value of the constant appearing before the integral sign is, of course, relatively unimportant.) Specifically, Uf is the ordinary Newtonian potential off in R".(b) Prove that if n = 1 or 2 then, for f 2 0, Uf = 0 iff = 0 a.e. (Lebesgue measure) and otherwise Vf = 03.
(1.6) Let X be the one-sided stable process of index /I, 0 < /? < 1 , in R (see (2.19) of Chapter I). Show that
[Hint: let hB(r,x) be the continuous density function for the measure qf (Chapter I, (2.19)). Then evaluate the integral k&) = Jgh,(f, x) dt by taking its Laplace transform.] (1.7) Let X be the symmetric stable process of index u, 0 < u < 2, in Rn (see (2.20) of Chapter I). If u < n prove that for f 2 0
Specifically Uf is the Riesz potential of order u. (b) Prove that if u 2 n then Uf has the same description as in Part (b) of Exercise 1.5. [Hint: because of
72
11. EXCESSIVE FUNCTIONS
Exercise 2.20 of Chapter I it suffices to calculate u ) g u w du, Jomdtjrnh.,z(t, 0
where h,,, is defined in (1.6) above and gu is defined in (2.17) of Chapter I. Make this calculation by interchanging the order of integration and then using the results of (1.6).]
2. Excessive Functions
In this section X = (Q, 4, M I ,X , , O f , P") will be a fixed standard process with state space (E, 8). (2.1) DEFINITION.Let a 2 0. A function f E S*, is called a-excessive (relative to X ) if (a) Ppf sffor every t 2 0 and also (b) P,9f-+fpointwise as t -P 0.
Of course Condition (a) implies that Prf increases as t decreases. If a = 0 we simply say " excessive" rather than " 0-excessive." Let 9" denote the class of functions which are a-excessive. Since the process X is fixed in each discussion it need not be referred to in the notation. The next four propositions collect some elementary but important properties of a-excessive functions. Statements involving a are understood to hold for all a 2 0 unless explicitly stated otherwise. (2.2)
PROPOSKION. (a) Constant nonnegative functions are in 9". (b) If
f,g E 9"then f + g E 9"and cf E 9"(c a nonnegative constant). (c) If {f.} E 9" andf, If, I .. . , then limf, E 9". (d) 9" = flyp, the intersection being over all fl > a. (e) I f f € 8: then U " f e 9". Proof. Properties (a)-(d) are immediate consequences of the definition. For Property (e) we write
The term on the right is less than U y = f$P: f du and approaches it as t decreases to 0.
(2.3) PROPOSITION. I f f € ti?*, thenfE 9'" if and only if (a) pU@'ysf for all p 2 0 and (b) pUa'"f-+ f as j3 -+ 00.
2. Proof.
EXCESSIVE FUNCTIONS
73
Iff E 9" then clearly D U a ' s f s f , while
as j-+ co by the monotone convergence theorem. For the sufficiency we first note that iff satisfies (a) and (b) for a particular value of tl, then it follows readily from the resolvent equation (8.10) of Chapter I that f satisfies (a) and (b) for every larger value of a. Thus in view of Property (d) of (2.2) we may (and do) assume that a > 0 without loss of generality. Let us first suppose that f E Bf satisfies (a) only (with a > 0). Then f, = min(f, n) satisfies (a) also, and the resolvent equation implies that for q > p we have
pufl+y,,= puq+y,+ (q - p)u~+=(pus+y,) I puq+y, + (q - p)uq+y,= quq+y,. Letting n -+ 00 we obtain pUa+ysqU"yand consequently p + pUs+Yis an increasing function of p whenever f e 8; satisfies (a). Again from the (2.4)
resolvent equation we obtain (2.5)
us+y,,= u=(f,,- pus'y,).
Consequently from Property (e) of (2.2) we see that PUP'% E 9"because f, - pUp+y, Ebb*,. As p -+ co, pUs+ynincreases to g, which is in 9'"by (c) of (2.2), and, as n -+ co, g,,increases to g E 9'". Clearly g, sf,and so g ~ f But . for each-P> 0, g 2 limn pus'% = / ? U p + yThus iff also satisfies (b) we obtain, on letting p + 00, g 2f.Therefore f = g E 9". (2.6) PROPOSITION. Iff E 9'"and a > 0 there is a sequence {h,} E bbf such that U"h, increases to f as n -+ 00. Proof.
Taking /3 = n in (2.5) we have
nU"+y,, = Ua(n[f, - nU"f%]). Since flUp+y, is increasing in both and n it follows from the proof of (2.3) that n U n + y , increases to f as n -+ 03. Hence we obtain (2.6) by setting h, = n(f,,- nU"+y,,). (2.7) REMARK. The condition in (2.6) that a be strictly positive cannot generally be removed. For example, if X is Brownian motion in R the results of Exercise 1.5 state that, forf 2 0, Uf is either identically infinite or identically 0. For conditions under which (2.6) is valid when a = 0 see Exercise 2.19.
stopping time, then P ; f If. (2.8) PROPOSITION. Let f E Y".If T is an { A , } If A is a nearly Bore1 subset of E,, then P i f E 9".
14
11. EXCESSIVE FUNCTIONS
Proof. Since P ; f increases to P$f as a decreases to /? it follows from (2.2) that we may assume a > 0. Then by Proposition 2.6 it will suffice to consider only f of the form f = Uag with g E b l * ,. The first statement in (2.8) is now an immediate consequence of Eq. (1.2). Furthermore we have (Exercise 1.4) (2.9)
CfiU"g ( x ) = Ex
e-'S g(X,) d s .
1
+ To&
Recall that if A is in 8: then T A is an {F,} stopping time for which t + TA o O r > TA and t + TAo 8, 1TA as t 10. Combining this with (2.9) we see that PiU'g E Y",completing the proof of Proposition 2.8. It follows from Proposition 2.8 taking f = 1 on E that if A is a nearly Bore1 subset of E,, then @ i ( x ) = EX(e-OITA;TA < is a-excessive. The main theorem in this section, Theorem 2.12, concerns the behavior of an excessive function composed with the process X . Before stating it we need some preliminary remarks. Recall from Section 11 of Chapter I, that if A E b", then A' denotes the set of points regular for A and that A' E 6*.
c)
(2.10)
PROPOSITION. Suppose f E Y", A
E P,
and x E A'. Then
inf{f(y): y E A} ~ f ( x I) sup{f(y): y E A}. Proof. To prove the left inequality let I be the infimum in question and K be a compact subset of A. By (2.8)
f ( x ) 2 f i f ( x ) 2 I EX(e-aTK;T~ < c), because X ( T K )E A if TK < 00. By (10.19) of Chapter I and the fact that x E A' we may choose K so that E x ( e - a T K ;TKc [) is close to 1. Hence f ( x ) 2 I. In proving the second inequality we may assume that a > 0 and then by (2.6) that f = U'g, g E b l f . By a now familiar calculation, if K is a compact subset of A
Since there is a sequence {K,} of compacts in A such that Px(TK,+ 0) = 1 and since E XJ;e-"'g(XJ dt < 00 it follows that the first term on the right may be made arbitrarily close to 0. Hence the desired inequality follows. LetfE 8"take on only finite values. For E > 0 define T(w) = W t : If(XO(4) -f(X,(w))l > E l .
If we set Ak,#= { f c k/n - E } u {f> (k
+ I)/n +
E},
for n = 1,2, . . . , and
2. k
= 0, + I , .
75
EXCESSIVE FUNCTIONS
. ., then for any 1 > 0 we have k+l n
n=l k=-m
3
TAk,.
<
1).
The sets Ak," are nearly Borel and so T is an {Ft} stopping time. (2.11) PRoPoSrTroN. Let f~ 9" be nearly Borel measurable and with only finite values, and let T be defined as above. Then no point is regular for T. Also I f ( X o ) -f(X,)l 2 E on { T < 0 0 ) almost surely.
+
Proof. Given x let Bl = {y : f ( y )>f(x) E } , B, = {y : f ( y ) TA. Proof. The regularity assertions in (b) are of course relative to the topology of [0, 001. To get started assumefe 9"and thatfis nearly Borel measurable. Let us first show that as far as (b) is concerned there is no loss of generality in assuming that f is bounded. Indeed define q : [0, co] + [0, 11 by q(x) = 1
- e-x,
x < co,
q(o0) = 1.
Then q is continuous, concave, strictly increasing, and satisfies q(xy) 2 x q(y) for y 2 0 and 0 5 x s 1. From this and Jensen's inequality we have m e - "'qCf(X,)I1 5 m q Ce-"'f(xI)l> qC~"{e-"'f(X,)Il
qCf (x)I;
that is, Pp(q of) 5 q o f for all t. Given x and E > 0 let A = {y: q V ( y ) ]> qLf(x)] + E } . Then A is empty iff(x) = co while iff(x) 00
=-
inf{f(y): Y E A } 2 4-"qCf(x)l
+ E l >fW,
76
11. EXCESSIVE FUNCTIONS
and so x is not regular for A according to Proposition 2.10. Similarly x is c q l f ( x ) ] - E } . Therefore q l f ( X , ) ] + q l f ( x ) ] as not regular for {y: t + 0 almost surely P". Hence q 0 f E 9" whenever f E 9'".We may analyze the truth of Statement (b) using q 0 f as readily asf. So we will assume f is bounded. Now given E > 0 define T as in (2.11) and then define
quo)]
To = 0,
T,+, T,, + T
0
ern,
n = 0, 1 , . . . .
Given x E E and A E F T ,we have EX{e-aTn+*f(XTnt,); A} = EX{e-"Tn P"~f(x,,,); A} E"{e-"Tnf(XTn);A}. That is, relative to P" the family of random variables {e-'Tnf ( X T n ) ;FTn} is a supermartingale. It is bounded since f is bounded and hence has a limit almost surely P" as n + co ; see Theorem 1.4 of Chapter 0.On the other hand If(XT.) -f(XTntl)l 2 E a.s. on {T,,,, < 00) according to (2,11), and so P"(lim T,, = co) = 1. Since x is arbitrary and {lim T,, = co} E 9 it follows that lim T,, = 00 almost surely. This argument can be repeated for E = l/k, k = 1, 2, .. ., with the corresponding iterates being denoted by T : . Almost surely limn T: = 00 for all k. On the other hand it is clear that if w is such that Ti(w)+ co as n + co for all k, then t +f(X,(w)) is right continuous and has left-hand limits on [0, co). This proves Asseition (b) iffis known to be nearly Borel measurable. In proving (a) we may assume that tl > 0. Then in view of Proposition 2.6 it is enough to prove (a) f o r f = U'g, g E b b f . Given a measure p on I define a measure v on b by v(B)= p(dx) U"(x,B). Then there exist gl,g 2 in bI+ such that g1 I g I g2 and v(g2 - gl)= 0. Now U'gl I U'g I Uag2 and the extreme members of this inequality are in bb,. For each fixed c 2 0
= eQ'v(g2
- g l ) = 0.
It now follows that Uagl(Xr) = Uag2(Xr)for all rational 12 0 almost surely P'. But we have already shown that t + U" g i ( X , ) is right continuous in t almost surely since U'gi is Borel measurable. Consequently U" gl(X,) = U" g2(Xr) for all t 2 0 almost surely P', and since ,u was arbitrary this implies
77
2. EXCESSIVE FUNCTIONS
that U"g is nearly Borel measurable. Thus the proofs of (a) and (b) are complete. Let A = {f< co} and B = { f = co}. We now know that A and B are nearly Borel sets. If x E B', then x E B by (2. lo). From (2.8), f ( x ) 2 Pi f ( x ) and this implies that P"(TB < 00) = 0 if x is in A. Consequently if K is a compact subset of A, then P"[TB0 OTK < co] = Ex{PX(TK'(TB < 00)) = 0. For a given x we can find an increasing sequence {K,,} of compact subsets of A such that TKn1T, almost surely P x . But then TB O, + TB 0 OTA almost surely P", and hence Px[TB0 O,, = co] = 1. Since x is arbitrary this yields Statement (c) of Theorem 2.12. 0
The proof of (2.12) yields a little more when a = 0. Specifically, suppose f E 9. Then the exponential factor in the supermartingale e-"=, f (X,,) is absent. It follows that almost surely T,, = co for some n and so lim f ( X J exists also as t -+co. It is easy to see that this limit is finite almost surely on {T, < co} where A = { f < co}. (2.13)
COROLLARY.If A E 6",then A' E 6".
Proof. Let T = T, . If a > 0, then @i(x) = E"(e-"=) is a-excessive according to the remark following Proposition 2.8, and hence is nearly Borel measurable. Since A' = {@> = 1) we obtain Corollary 2.13. (2.14)
COROLLARY.Iff, g E 9" then min(f, g) E 9".
Proof. Let h = min(f, 9). Obviously h 2 Pph. The fact that Pph + h as t + 0 is an immediate consequence of Theorem 2.12(b).
Exercises (2.15) Let f be in 9". Let R and T be { M r }stopping times with R I T. If A E M R show that
E"{e-""f(XR); A} 2 E x { e - " T f ( X , ) ; A} for all x. (2.16) A standard process X is called a strong Feller process if U"f is continuous for all f E 6 6 , having compact support and all a > 0. Show that in this case any a-excessive function (a 2 0) is lower semicontinuous. (2.17) A function f E S*, satisfying (a) of Definition 2.1 is called a-supermean-aalued. If f is a-super-mean-valued show that = lim,+o Ppf exists
78
11. EXCESSIVE FUNCTIONS
(pointwise), that fsf, and that f is a-excessive. Also show that, for all b 2 0 , U s f = U@f.[Hint: for each x the mapping t + P , " f ( x ) has only countably many discontinuities; if t is a continuity point then P , f ( x ) = P If ( x ) . ] Furthermore characterize f as the largest a-excessive function dominated byf. The function f is called the (a-excessiue)regularization off. (2.18) An {Fl}stopping time T is called a terminal time provided t T 0 8,= T almost surely on {T > t} for all r 2 0. Note that if A is a and T nearly Borel set, then both TA and DA are terminal times. I f f € 9"
+
is a terminal time show that P%f is a-super-mean-valued. If in addition T has the property that whenever {t,} is a sequence decreasing to zero one has t. T 0 8," + T almost surely, then PF f E 9". Note that if A is a nearly Borel set, then TA always has this additional property while DA may fail to have it.
+
Let X be a standard process such that U ( x , K) = j;P,(x, K ) dt is bounded in x for each compact subset K of E. Show that in this situation if f i s excessive, then there exists a sequence {g,,}of bounded functions in S*, such that {Vg,} increases tofand each Ug,,is bounded. [Hint: (i) let f be a bounded excessive function such that limf-m P,f = 0. If g 1 = I -'(f- Pf) show that Ug, increases to f a s t + 0. (ii) Let {K,} be an increasing sequence of compact subsets of E whose union is E. If h, = nZKn show that Uh, t 00 and each Uh, is bounded. (iii) Iff is excessive show that f,, = min(f, Uh,,) satisfies the hypotheses of (i). Use this to construct the desired sequence {g,,}.] (2.19)
(i) Let (E, 8 ) and (F, 9) be measurable spaces, p a measure on 9 which is a countable sum of finite measures, and u : E + F be such that (a) u E S/F, (b) there exists a mapping T : F+ E such that z E F/S and u[r(x)]= x for all x in F. Let v = p~-'. Then show that J E ( f o u ) d v = j F f d p for eachfE F+.In particular derive from this the fact that if u is a continuous nondecreasing function on the interval [a, b] and if F(s) is a right continuous nondecreasing numerical function defined on [u(a), u(b)],then (2.20)
fCu(01 dFCu(01 = J
Iu(4),u(b)l
f(0 d F ( 0
for all nonnegative Borel functionsf on [u(a), u(b)]. (ii) Let a > 0 and let h and cp be nonnegative functions in bb*. Define
Show that
JI = U"[h(cp- $)I. [Hint: use (i) to show that
3.
EXCEPTIONAL SETS
79
1 - exp[ - j)(Xs) ds] = j;h(X,) exp[ - /,'h(X.) d u ] ds, and substitute this into the expression defining 49.1 (iii) If cp = U"f with f a nonnegative function in bd*, then show that (notation as in (ii))
Consequently II/ Icp and t+b increases as h increases in this case. Finally show that this last statement continues to hold if cp is only assumed to be a bounded a-excessive function.
3. Exceptional Sets In this section we are going to introduce various classes of sets that will play the role of exceptional sets in the theory to be developed. The importance and the appropriateness of these notions will only become apparent later. In the present section we will merely give the definitions and develop some elementary consequences of them. Again in this section X = (Q, A, Ar, X , , O f ,P") will be a fixed standard process with state space (E, 8).Our definitions will involve the process X, or at least its semigroup, but since we are regarding X as fixed the qualifying phrase "relative to X " will generally be omitted. (3.1) DEFINITION. (a) A set A E b* is of potential zero (or simply "null") if U ( x , A ) = 0 for all x E E. (b) A set A c E is polar if there exists a set D E 8" such that A c D and P"(TD < co) = P"(TD < c) = 0 for all x. (c) A set A c E is thin if there exists a set D E I"such that A c D and D' is empty, that is P"(T, = 0) = 0 for all x. (d) A set A c E is semipolar if A is contained in a countable union of thin sets. (e) A set A c E is thin at x if there exists a set D E 8" such that D 2 A and x is irregular for D; that is, P"(TD = 0) = 0. Since U"(x,A) = e-"' P,(x, A ) dt, it is clear that A is of potential zero if and only if U a ( x ,A) = 0 for all x for some, and hence for all, a 2 0. Intuitively a polar set is one which the process never enters at a strictly positive time alniost surely. If A E I", then A is polar if and only if @:(x) = E x ( e - a T A TA ; < [) = E*(e-uTA;TA < 00) = 0 for all x for some, and hence for all, a 2 0. On the other hand a thin set is one which the process avoids during some initial open time interval almost surely. If A E b", then A is thin if and only if 05 < 1 for some, and hence for all, a > 0. Clearly a polar set
11.
80
EXCESSIVE FUNCTIONS
is thin and a thin set is semipolar. In addition, it will follow from Proposition 3.4 that a semipolar set A in B* is of potential zero. It is evident that a countable union of polar (semipolar) sets is again polar (semipolar). A set may be thin at every point without being a thin set. (See Exercise 3.14.) Often instead of saying A is thin at x we will say that x is irregular for A.
(3.2) PROPOSITION.Iff and g are in 9'"and f 2 g except on a null set, then f 2 g everywhere. In particular iff = g except on a null set then f = g everywhere. Proof. It is immediate from the definition of a null set that pU"'sfr flLJ"'@g for all /?> 0. Letting /3 + 00 and using Proposition 2.3 yields the desired conclusion. The next proposition is a very useful result.
(3.3) PROPOSITION.If A E B", then A - A' is semipolar. Proof. Let a > 0, and let A, = A n {@: I 1 - l/n}. Clearly A - A' is the union of the A,. If @:(x) < 1 , then x is not regular for A and hence is not regular for A,. If @i(x) = 1 , then by (2.10) x is not regular for {a: I 1 - l/n}. Thus no point is regular for A,; that is, A, is thin. This yields Proposition 3.3. An immediate consequence of our next result is the fact, mentioned earlier, that a semipolar set in B* is null. (3.4) PROPOSITION. Let A be semipolar. Then almost surely X , E A for only countably many values of 1.
Proof. We may suppose that A is thin and in 8". Pick a > 0, fl < I , and let B = A n {WAI fl}. The set A is a countable union of sets such as B and so it will suffice to prove X , E B only countably often. Let T , = T , and T,,, = T, + TB &.,. Since B c A and A is thin, B' is empty. Therefore almost surely XTnE B on {T, < a}.Now 0
E*(e-uTn+l) = E ~ [ Q ~ ( Xe-IIT,] ~,) I p Ex(e-aTn). Consequently EX(e-aTn) + 0 as n + 00 and hence T, --t co almost surely. Since X , $ B if T, < t < T,,, the proof of Proposition 3.4 is complete. PROPOSITION.Let f~ 9'"and A = { f = m}. Then A is polar if and only if A is null.
(3.5)
3.
EXCEPTIONAL SETS
81
Proof: Assume A is null and let x E E. Since U ( x , A) = 0 there is a sequence { t,,} approaching 0 such that P "( XrnE A) = 0. By Theorem 2.12(c), P"(X, E A for some r > t,,) = 0. Thus P"(X, E A for some t > 0) = 0. Since x is arbitrary and A E b", this implies that A is polar.
Recall (see Exercise 2.17) that a function f in 8: is a-super-mean-valued provided Ppf 5 f for all t, and that for such a n f , f = limfJ0Ppf is the largest a-excessive function which is dominated by f. Moreover for each /3 2 0, W a f = U a f . Consequently { f # f }= { f < f } is of potential zero. However the following theorem of Doob which generalizes an important theorem of Cartan in classical potential theory allows us to say much more in an important special case.
(3.6) THEOREM.Let {f,} be a decreasing sequence of a-excessive functions and let f = lim,,f,. Thenf is a-super-mean-valued and {f a, and /3 - a < E. According to (2.10), x is not regular for {f 5 a} and by what was said above x is not regular for (12b}. Consequently A: c A,. To complete the proof of Theorem 3.6 it clearly suffices to show that each A, is thin. This is accomplished by the following lemma.
a,
(3.7) LEMMA. Suppose that x is regular for A,, that /3 2 0, and that there stopping times such that (a) P"(T,,-,0) = 1 and is a sequence {T,,}of {9,} (b) f ( x ) 2 P T n f ( x + ) /3 for all n. Then there is a sequence {Q,,} of (9,) stopping times such that (a') P"(Q, 0) = 1 and (b') f ( x ) 2 PQnf(x) + + 4 2 for all n.
82
11. EXCESSIVE FUNCTIONS
Indeed repeated application of (3.7) with B first equal to 0, then equal to ~ / 2 ,and so on, shows that if x E A: then f ( x ) = 00 which contradicts the fact thatfis bounded. Thus (3.7) implies that A, is thin and hence the proof of (3.6) will be complete as soon as (3.7) is established. Proof of' (3.7). Let 6 > 0 and 0 < q < 1, and let R = TA,. Since almost surely P", T,,--t 0 and X, E A, at arbitrarily small strictly positive values of t (xEA:), we may choose n so large that S = T, + R of&-,, satisfies P"(S 2 6/2) < q/2. Moreover X ( S ) E A, almost surely on {S< 03) because A: c A,. Let p(dy) = P"[X(S)E dy; S < 001. Since A' = E, P'(TA > 0) = p(dy) Py(TA> 0) = 0. Consequently we can choose a compact subset K of A such that
If Q = S
+ TK
0
OS, then
But f(XT,) = f ( X T , ) almost surely on {TK < co} because X(T,) E K c A. Therefore the following inequalities obtain: Exf(XQ) = E"{E"'S'[f(XT,)l)
5 EXf(Xs) I E X f ( X s )- & P"(S < 00) 5 E"f(XT,,) - & P"(S < a)
If(4- B - 41 - 1/2) I f ( x ) - B - 42. In view of the fact that 6 and q are arbitrary, these estimates yield Lcmma 3.7.
Exercises
(3.8) (a) Give an example of a thin set which is not polar. (b) Give an example of a semipolar set which is not thin. (c) Give an example of a null set which is not semipolar. [Hint: consider the process of uniform motion to the right.] (3.9)
Let A c E be null. Show that each point of E is regular for A'.
3. EXCEPTIONAL SETS
83
(3.10) Let X be a standard process and assume that there exist a p > 0 and a Radon measure 1 on 6 such that f~ Y pand f = 0 almost everywhere 1 implies t h a t f = 0. Show that there exists a finite measure t on 6 such that A E 6* is null if and only if &A) = 0. Show that under this assumption any a-excessive function is Bore1 measurable. (See Section 1 of Chapter V.) (3.11) Let A E I n and IetfE 9". Show that if every point of A is regular for A, then P; f is the smallest u-excessive function dominatingf on A. Give an example to show that P i fneed not dominate f on A if A - A' is not empty. (3.12) Let A E 6"and let x E E be fixed. Let {T,,} be an increasing sequence of { F l }stopping times with lirn T, = T A almost surely P". If 0 = @> show that @(X,J + 1 almost surely P" on {T,, < TA for all n, TA< a}and hence that lirn,,,, @(X,) = I almost surely P" on this set. [Hint: since 0;increases as a decreases it suffices to consider a > 0. Use (2.15) to show that {e-"Tn@(XT,);9,") is a supermartingale relative to P" and hence that limn e-",. @(XT,) = L exists and L 5 e P T A almost surely P". Next use the strong Markov property to show that if A = {T,, < T A for all n}, then E"(L; A) = Ex(e-uTA; A). Conclude from this that limn@(A',") = 1 almost surely P" on A n {T, < co).]
(3.13) Let A
E 6" and
suppose that lim suplloP,(x, A) > 0. Then show that
x is regular for A.
(3.14) Construct an example of a set A which is thin at every point without being a thin set. [Hint: let X be uniform motion to the right in RZ;that is, starting from any point (xo,yo) E R2 the process moves to the right along the line y = yo with speed one. The reader should have no difficulty writing down a rigorous definition of this process. Let A be a nonmeasurable set (with respect to two-dimensional Lebesgue measure) which intersects each horizontal line in at most one point. See Hewitt and Stromberg [l, 21-27] for the construction of such a set.] (3.15) Let X be the symmetric stable process of index u in R. If a 5 1 show that for each x the singleton {x} is polar. [Hint: using the notation of (1.7) let f , ( t , x ) = @ l ~ " , ~u) ( t g,(x) , du. If a < 1 show y -,( x - yIu-' = c J ; f u ( t , y - x) dt is excessive for each x, and then use Proposition 3.5. If u = 1 and p > 0 let Up(x)= e - p ' f i ( t ,x) dr; then show that U,(O)= co, U,,(x) < co for x # 0, and that y + Up(x- y ) is p-excessive for each x . ]
jr
(3.16) Let X be the Brownian motion process in R. Show that for all points x, x is regular for {x}. Consequently the only thin set is the empty set. [Hint:
84
11. EXCESSIVE FUNCTIONS
use (3.13) and the continuity of the paths.] This result is also valid for stable processes in R of index a with 1 < a < 2 as will be proved later.
(3.17) Show that iffis a-super-mean-valued then {fcf}need not be semipolar. [Hint: use the result of (3.16).] (3.18) Let X be Brownian motion in R. I f f = ILO,m), T = T { o , ,and x < 0, then compute
to obtain PX(Ts t ) = 2P"(X, 2 0). Now use the explicit form of the Gauss kernel g,(x) (see (2.17) of Chapter I) to show that E"{exp(-aTy)} = e-J'lx-YI for all x, y E R and a > 0, where Ty = T{yr, In particular show that Px(Ty< CO) = 1.
(3.19) The notation is that of (3.18). Let a .c x c b and let a > 0. Let T = min(T,, Tb) = inf{t : X,4 (a, b)} almost surely P" and let J, = E"(e-'=; X ( T ) = a) and Jb = E"(e-""; X ( T ) = b). Express EX(e-"To)and EX(e-(ITb) in terms of J, and Jb and solve the resulting equations to obtain J, =
sinhJa(b - x). sinh&(b - a)'
Jb
=
sinh&(x - a ) sinh&(b - a)'
Conclude from this that P"(T, < Tb)= (b - x)(b - a)-' and that E X ( T )= +(x - u)(b - x).
(3.20) Letfe 9"and let {B,,} be a decreasing sequence of nearly Bore1 subsets of E, such that BL c B,,, whenever n > m. Then { P f , f } decreases to an a-super-mean-valued function g. Show that g = S except possibly at those points x in E such that either g(x) = co or x E EL for all n. [Hint: first show that for each n, g = P i n g on { g < 00). Next show that limtloP f g ( x ) 2 lim inftloP:Pin g(x) 2 P i , g(x) if x is not regular for B,, .]
4. The Fine Topology
Once again, in this section the basic datum is a given standard process X = (Q, A, A , ,X , , Or,P " ) with state space (E, 8), and all definitions are relative to this given process.
4.
THE FINE TOPOLOGY
85
(4.1) DEFINITION. A set A c EA is calledjnely open if A' = EA - A is thin at each x in A. In other words for each x E A there is a set D E 8: such that A ' c D and P"(T, > 0) = 1.
Intuitively a set A is finely open provided the process remains in A for an initial interval of time almost surely P" for each ~ E ABy. the right continuity of the paths any open set is finely open. Let O(9,) be the collection of all finely open subsets of E(EA). One checks without difficulty that O(0,) is a topology on E(EA). It is called thejine topology on E(EA). We already observed that the fine topology on E(EJ is finer than the original topology on E(EA). Note that 9 is just the topology that E inherits when we regard E as a subspace of (EA, OA), and that A is isolated in (EA, OA). Consequently we will restrict our attention to the fine topology on E. In this section we will give an alternate description of 0 which will show that in reality 0 depends only on the transition function of X. We will also discuss the behavior o f t + f ( X , ) whenfis nearly Bore1 measurable and finely continuous, that is, continuous relative to 9. As usual the continuity of numerical functions is defined relative to the topology of R, the extended real line. Let us also recall that an elementfof 9" is regarded as a function on E, and hence sets of the formf-'(D) where D c R are subsets of E. Of course in certain formulas we must set f(A) = 0; but {f< I}, for example, is a subset of E. (4.2) PROPOSITION. If YE Y",then f is finely continuous. Proof. Let Z be an open interval in R and let B = f - ' ( I ) . By Proposition 2.10 no point in B is regular for B'. Consequently B is finely open.
(4.3) PROPOSITION.Let A c E and suppose that x is in A and A' is thin at x. Then there exists a compact set K such that x E K c A and K' is thin at x. Proof. By definition there exists a B E 8"such that B c A andP"(T,, > 0) = 1. Since x E A we may assume that x E B, and also that B has compact closure B in E. Of course B' = EA - B. By Theorem 11.2 of Chapter I there exists a decreasing sequence {G,} of open subsets of E containing B'- {A} = E - B 1 E - B such that min(TGn,[) 7 min(T,-,, [) almost surely P". But min(T,-,, C) = TEE, min(TGn,C) = TGnv(A),and G,, u {A} is open in Ed since G,, 1 E - B. Consequently, by the zero-one law, there exists an open set G in EA such that A E G, G' c B, and Px(TG> 0) = 1. If we let K = G ' the proof is complete.
86
11. EXCESSIVE FUNCTIONS
(4.4) PROPOSITION. Let 9 = {B : B E 0 n b" and Then % is a base for the fine topology.
B is compact in E}.
Proof. Let A be a fine neighborhood of the point x. By (4.3) there is a compact subset K of E such that x E K c A and P"(T,, > 0) = 1. Let u > 0 and @'"(x)= and let B = {a" < l}. Since ma E Y",the set B is in 0 n b", and of course x E B. If y E K' then @"(y)= 1 because K' is open, so B c K. Finally B is compact in E because K is compact in E. This completes the proof. (4.5) THEOREM.Let u > 0. Then the fine topology is the coarsest topology relative to which the elements of 9" are continuous.
Proof. In the proof of (4.4) we produced a base for the fine topology consisting of sets of the form {@: < 1) where G ranges over the open sets of E , containing the point A. Hence we obtain Theorem 4.5.
REMARK.We have actually shown that the fine topology is the coarsest topology rendering the functions @,: G as above, upper semicontinuous. (4.6) THEOREM.Suppose that U ( x ,K) is finite for all x whenever K is a compact subset of E. Then the fine topology is the coarsest topology relative to which the elements of Y = Y oare continuous.
Proof. Let Y be the coarsest topology relative to which the elements of Y are continuous. By (4.2), Y c 0, and, by the preceding remark, it will suffice to show that < l} E 9-for all open sets G in E,. Let {G,,} be a sequence of open sets with compact closures such that G,,c G,,, for n 2 1 and UG,, = E. If
{@a
(4.7)
cpn(x) =
u IG,(x) - PGU IG,(x)
then cp, is 9-continuous since both UIGnand P, UIGnare excessive and finite. Clearly cp, increases with n. If cp = limn cp,, then cp is Y-lower-semicontinuous. Thus to complete the proof it suffices to show that {@A < l} = {cp > 0). If rp,(x) > O then Px(TG>O) = 1, and so Od(x) < 1. Hence {cp > 0} c < 1). On the other hand if @a(x)c 1 and x E G,, then cp,,(x) > 0. Consequently {@Q < l} c {cp > 0}, completing the proof of Theorem 4.6.
{@a
(4.8) THEOREM.Suppose f~ 8".Then f is finely continuous if and only if the mapping t + f ( X , ) is right continuous almost surely.
87
4. THE FINE TOPOLOGY
Proof. The implication from the right continuity of the composition to the fine continuity off is immediate; in fact only the right continuity at t = 0 is needed. Coming to the converse implication, suppose that t + f [ X , ( o ) ] is not right continuous and that t 4 X , ( o ) is right continuous. Then there is a point to < c(o),a sequence {t,,} decreasing to to with t, < C(o)for all n, and a pair of intervals Z,, Z2 of the form [ - CQ, r), (s, 001 (perhaps not in that order) with r and s rational and r < s, such thatf[Xt0(w)]E Zland f [ X J w ) ] E Z, for all n 2 1. Let A =f -'(Z1) and B =f -'(Z2) so that A and B are disjoint finely open sets in 8".If cp = @,, then cp[XrO(o)] < 1 and (p[X,,(w)]= 1 for all n. But cp E 9" and hence almost surely t + cp(Xr) is right continuous. Since there are only countably many pairs (I,,I,) of the above form and t + X , ( o ) is right continuous almost surely, the proof of Theorem 4.8 is complete.
Exercises Let A E 8".Show that A is finely closed if and only if A' c A , and that the fine closure of A is A u A'. [Hint: first show that almost surely TAI TAP .] Consequently if x # A , then A is thin at x if and only if x is not in the fine closure of A . Show that in this last statement the condition " x # A '* may not be eliminated. See (1.8) of Chapter V for an extension of this result to arbitrary subsets A of E. (4.9)
(4.10) Assume that {x} is polar for each x E E. Under this assumption show that a sequence {x,,} approaches x in the fine topology if and only if x,, = x for all large n. (4.11)
Let B be a null set. Show that E - B is finely dense in E.
(4.12)
Show that ( E , 0)is a completely regular Hausdorff space.
(4.13) Let A E b", x E A (the closure of A in E ) , and consider the following statements: (i) x is not in A and A is thin at x ; (ii) there exists an f E 9" such that f ( x ) < lim inf,,,,, f ( y ) . Show that (ii) implies (i) and that (i) implies (ii) if c1 > 0. [Hint: to show that (i) implies (ii) for c1 A 0, use Proposition 4.3. Make use of Proposition 2.10 for the other assertion.] (4.14) Let Y E b 4 . Suppose that for each p, Y O,, + Y almost surely P" whenever {T,,} is a sequence of (9,) stopping times decreasing to zero almost surely P'. Show that f ( x ) = E x ( Y) is nearly Bore1 measurable and finely continuous. [Hint: one may assume Y 2 0. Show that 0rUY-t f and so 0
88
11. EXCESSIVE FUNCTIONS
f is nearly Bore1 measurable. If E > 0 show that x is not regular for A = {y : f ( y )2 f ( x ) + E } and B = {y : f ( y )I f ( x ) - E } , and conclude from this that f is finely continuous.] (4.15) Let a > 0 and cp E 9". If A E 0 n &", show that there is a sequence {g,,} in 8; such that each gnvanishes off A and Uagnf P>q.Each g,,may be
taken to have compact support in A, if A is a countable union of closed sets. [Hint: assume first that cp is bounded. Let {h,,} be an increasing sequence of nonnegative elements of b F such that limh,, is infinite on A and each h, vanishes off A-or off some variable compact subset of A if A is a countable union of closed sets. Let $,, be defined as in Exercise 2.20 using rp and h,,. Show that
and use this to show that $,, P>cp. Use the results of (2.20) to complete the proof when cp is bounded. If cp is unbounded apply the above argument to min(cp, a) and let a -,co.] --f
Let X be Brownian motion in R. Prove that CPE,(x) 1 as y + x. [Hint: see (3.18).] Prove that the fine topology coincides with the usual topology of R. [Hint: by the remark following (4.5) it suffices to show that CP; is upper semicontinuous when a > 0 and G is open. For this use the result of the first part of (4.16).] (4.16)
--f
Let X be a standard process such that for all B E 8"either UI, = 0 or UI, = co;that is, VIE= co whenever B is not of potential zero. Show that if B E c f n and UI, # 0, then CP, = 1. Of course CP&) = P"(T, < () = P"(T, < 00) is regarded as a function on E and CP, E 9. [Hint: let YE= limf-m P,@, and O,=CPB-YB. Then P I Y E = Y E .Use (2.19i) and the hypothesis to show that 0,= 0, and hence that P,@, = 0,.Show that there exists an xo such that CPB(xo)= 1. Finally show that if D = {CP, < l}, then U ( x o , D) = 0 and conclude from this that D must be empty.]
(4.17)
Let X be as in (4.17). Show that if B E b" and 0,# 0, then CP, = 1. [Hint: assume B compact and let q = sup 0,> 0. If 0 c 6 c 11 and A = {a, > 6 ) use (4.17) to show CPA = 1. Conclude from this that CP, = q. Finally show that 0,= 1.1
(4.18)
Let X be a standard process such that CP, = 1 whenever B E d'n is nonvoid and finely open. Show that the only excessive functions are constants. Combine this with (4.17) (or (4.18)), (1.5b), and (1.7b) to conclude that the
(4.19)
5. ALTERNATIVE CHARACTERIZATION OF EXCESSIVE FUNCTIONS
89
only excessive functions for Brownian motion in R or R2,or the symmetric stable process in R with u 2 I , are constants. (4.20) Let X be as in (4.19) and assume that E contains at least two distinct points. Show that P x ( [ < 00) = 0 for all x E E. [Hint: show that $(x) = E"(1 - e - [ ) is excessive and hence constant. Use this to show that E X ( e - r )= 0 for all x in E.]
Let X be as in (4.20). If B E &" show that UI, = 0 or UI, = 00. [Hint: suppose that q = sup U I , > 0. If 0 < 6 < q and A = { U I , > S}, then @ A = 1. Use this and (4.20) to show that U I , = 00.1 Note that if excessive functions are lower semicontinuous, then one need only assume @, = 1 whenever B is a nonvoid open subset of E. (4.21)
(4.22) Combine the results of (4.17)-(4.21) to show the equivalence of the following statements. (Assume that E contains at least two distinct points.) (i) UZ, = 0 or U I , = co for B E 8".(ii) 0,= 1 whenever B E &" is nonvoid and finely open (or whenever B is a nonvoid open subset of E if excessive functions are lower semicontinuous). (iii) @, = 1 whenever B is in b" and is not polar. (iv) The only excessive functions are constants. A process satisfying any, and hence all. of these conditions is called recurrent. (4.23) Let X be a standard process such that 111, = 0 or a, > 0 for all B E b". Then if B in b" is not polar, 0,> 0. [Hint: suppose @,(x0) = q > 0 for some xo . Show that if D = {@, > q/2}, then QD > 0. Conclude from this that 0,> 0.1 Note that if there exists a measure ( on & such that P,(x, dy) = f ( t , x , y ) ((dy) with f strictly positive, then X satisfies the above condition.
In particular it is satisfied for Brownian motion or the symmetric stable process in R". Let X be a standard process and assume (i) for some u , U a : C, + C and (ii) for each compact subset K of E, U ( x , K ) is everywhere finite. Show that almost surely X , + A as f + co. [Hint: let K be a compact subset of E, let G be an open set containing K and having compact closure in E, and let u be such that U " : C, + C. Then the assertion is a consequence of the following three observations: (i) U ( x , C ) 2 U " ( x ,C ) 2 6 > 0 for x in K , (ii) P,U I J x ) 2 6 P"(T, 0 0, < 00). and (iii) P , UIG-+ 0 as t + 00.3 (4.24)
5. Alternative Characterization of Excessive Functions
According to (2.8) iff'€ Y " ,then P i f I f for every compact K . In this section we will derive various converses to this assertion. These are useful
90
11. EXCESSIVE FUNCTIONS
for finding the functions excessive relative to specific semigroups and for proving that in various situations excessiveness is a local property. The fundamental theorem, which we will now prove, is due to Dynkin [2] and [4]. As usual X = (0, A, A l ,X,,Or, P " ) is a fixed standard process with state space (E, a). (5.1) THEOREM. Suppose f E 8; and f 2 P i f for every compact subset K of E. Then q U q + " f I f f o r all q > 0.
Proof. We will take c( = 0 to simplify notation. Clearly it is enough to prove the theorem for boundedf. Let p > 0, q > 0, and define h = U q + p fF, = qh -f. The resolvent equation implies that for y > 0
h = U'[(y - P)h - F ]
=
Uyg,
where we have put g = ( y - j ) h - F. Since h is the y potential of the bounded function g we have for any stopping time T
1
T
h(x) = EX
e-7'
g(x,)dt + P ;
h(x),
0
from which we obtain
1
T
(5.2)
Pk F(x) - F ( x ) = -qE"
0
e-" g ( X , ) dt + f ( x ) - P k f ( x ) .
Let A = { F < 0). Choose y < fl and let T = TK, K any compact subset of A. By hypothesis, f - P , f is positive and so f - Pf f is also positive. Of course ( p - y)h is positive and P$ F is negative because XT E K c A almost surely on {T < a}.Consequently, using the definition of g,
/ e - y t F ( X , ) dt. T
F ( x ) I -qE"
0
But E x e - y t F ( X , ) dr 2 0 and we can find compacts K , c A such that P"(TK,+ TA)= 1. Hence F ( x ) I 0; that is, qU"+Bfl f. Letting p decrease to 0 completes the proof of Theorem 5.1. COROLLARY.I f f satisfies the hypotheses of Theorem 5.1 and also lim inf,,, Ppf 2 f,then f E Y".
(5.3)
Proof. By (5.1) and the argument in (2.3), qU,+"fincreases as r] 4 00. Let =f.It follows from the condition in (5.3) and Fatou's lemma that lim inf,,, q U v + " f f 2 f ,and since g s f t h i s implies that g =j:
g = lim, qU"+"f. By (2.3), f~ Y" if and only if g
5.
ALTERNATIVE CHARACTERIZATION OF EXCESSIVE FUNCTIONS
91
Before coming to the applications of (5.3) we need some definitions. DEFINITION. Let T = inf{t: X , # X o } . A point x E EA is called a holding point (for X ) provided P " ( T > 0) = 1. In the alternative case, P X ( T= 0) = 1, the point x is called an instantaneous point. (5.4)
Clearly A is a holding point. Define a sequence of stopping times {S,,} by So = 0 and S,,, = S,, + T OBs,. (5.5) as n
-
DEFINITION.The process X is called a regular step process if S,, --f co almost surely P" for all x in E.
The definition implies of course that for a regular step process every point is a holding point, and that the path functions r + X , are step functions taking the value X(S,,) on [S,,, S,,,) for all n almost surely. These processes are just the ones constructed in Section 12 of Chapter I. Most of the familiar processes which one encounters are either regular step processes or else all points are instantaneous. The next two theorems treat these two cases. THEOREM.Let X be a regular step process. Then f E 8: is in 9"if and only iff 2 Ptf (where T is defined in (5.4)). (5.6)
Proof. First note that for each x E E the set {x} is finely open so that f is finely continuous. Next we observe that f E 8"; indeed given an initial measure p, let p,, be the measure pc,(n) = P"(X(Sn) E A ) ,
and let u,, u, E d be such that u,, sf I u,, and p,,({v,, < u,}) = 0. Of course, we set u,,(A) = u,(A) = 0. If v = sup u,, and u = inf u,, then u, u E 8,v sf Iu, and since X , = X(S,,) on [S,,, S,,,,) we have P"[u(X,) < u ( X , ) for some t ]
Cn P"Cu,(X(S,)) < un(X(Sn))I
= 0.
Consequentlyf'e b". Let K be a compact subset of E and let Q = T , . We next claim that for each n20 (5.7)
f ( x ) 2 E"(e-"Qf(XQ);Q IS,)
+ E"(e-"Snf(XS,);Q > S,,), We will prove this by an induction argument. Clearly (5.7) is valid when n = 0.
92
11. EXCESSIVE
FUNCTIONS
Now assume that (5.7) holds for some fixed value of n. Denoting the second summand on the right side of (5.7) by J and using the inequalityf(Xsn) 2 P;f(X,,,) we have J 2 E"{e-"'" EX'Sn'[e-"Tf(X,)];Q > S,} = E X { e - u S n + l f ( X S ~Q+ ,>) ;S,}.
But { Q > S,} differs from
{ Q > Sn; Q 0s.
=T
0
0
es,,) u { Q > S,;Q
0
0s. > T 0 esn}
by a set having P " measure 0, and Q = S, + Q 0 Osn on { Q > S,}. Consequently J 2 E"{e-"Qf(X,); Q > S , , Q = S,,
,>
+
EX{e-&+
I
f ( X s n t l ) ;Q > % + I } ,
and since { S , < Q I S,,,} = {S, < Q ; Q = S,,,} almost surely, we obtain (5.7) with n replaced by n 1. Thus (5.7) is valid for all n. Now letting n + 00 in (5.7) we o b t a i n f r Pkffor all compact subsets K of E. Finally the fine continuity off and Fatou's lemma imply that lim inffl, PBf r J and so Theorem 5.6 follows from Corollary 5.3.
+
Before stating the next theorem we let d be a fixed metric on E compatible with the topology and define (5.8)
T, = inf{t: d ( X , , X , ) > l/n}.
(5.9) THEOREM. Suppose all points of E are instantaneous. Let f E 8; be finely continuous and suppose that for every compact K c E there is an integer N such thatf(x) 2 P;, f ( x ) for all x E K and n 2 N . ThenfE 9".
Proof. Once again we assume a = 0 to simplify the writing. Let K be a compact subset of E and pick a positive integer N such thatf(x) 2 P T n f ( x ) for all x E K and n 2 N. Fix n 2 N and define R = T,, =0,
if X , E K , if X , # K ,
+
and Ro = 0, Rk+l = Rk R 0 e R , , . Obviouslyfr PRJ and because d(X(Rk), X ( R k + , ) )2 l / n if R k + l < T K E 4 5 it follows from the quasi-left-continuity of X that limk,, Rk 2 T K E almost surely. Given an integer k and E FRkwe have from the strong Markov property E"{f[X(Rk)l;
r}2 E " { P R f ( X ( R k ) ) ;
l-1
= E X { f ( X ( R k +1)); l-1.
5.
ALTERNATIVE CHARACTERIZATION OF EXCESSIVE FUNCTIONS
93
From this it follows easily that for any given t m
Recalling the dependence of Rk on the integer n, define S,,=Rk+l,
= a3,
if R k s t < R k + l , if Rk t for all k.
Then for all n 2 N (5.10)
f(x) 2 EX{S(X(Sn));t < 7 " C l .
It is clear from the definitions that if t < TKc then S,,, is in the interval + T,, o O,].As there are no holding points 7,, 8, approaches 0 as n -, a3 almost surely on {( > t } . By hypothesis and Theorem 1.8 the mapping s + f ( X , ) is continuous on the right almost surely, and so letting n approach co in (5.10) we obtainf(x) 2 E " ( f ( X , ) ; t < TKC). By choosing Klarge enough we may make TKC arbitrarily close to ( and so finallyf2 P, f for all t. This inequality, along with the fact thatfis finely continuous, implies thatfE 9'. Thus the proof of Theorem 5.9 is complete. (t,t
0
We have not attempted to find the best way of combining (5.6) and (5.9). Rather we will content ourselves with the following result in the general situation. THEOREM.Let f be defined, nonnegative, and lower semicontinuous on E, and suppose that, for each x in E, f ( x ) 2 P;" f(x) for a sequence of values of n approaching infinity (the sequence may depend on x; T,, is defined in (5.8)). Then f~ Y a .
(5.11)
Once again we assume c( = 0 to simplify the notation. Clearly f is Bore1 measurable in the present case. Let K be a compact subset of E. We are going to show t h a t f 2 P K J To this end define
Proof.
A,, = {Y E E : 4 y , K ) > l/n andf(y) 2 PTnf(y)).
It then follows from our hypothesis that E - K = R1=T,,,
U,, A , .
Let
if X , , E A , - U A k , k
=0,
if X o # E - K .
Clearly f 2 P R ,.f. Fix x E E - K . If Q is an {F,} stopping time, A E 2FQ, and R = Q + R, o 0,, then
94 (5.12)
11. EXCESSIVE FUNCTIONS
E " { f ( X Q ) ; A> 2 E " { P R , f ( X p ) ; A} = E"{f(x~); A}-
Let /? be any countable ordinal and suppose that R, has been defined for all y < /?.If /? has a predecessor, /? - 1, define R, = R,-l R , 0 O R # - , . If /? is a limit ordinal define R, = sup,,,, R , . In either case R, is an {S,} stopping time and R, 2 R , whenever y /?.Since R, > 0 almost surely on { X , E E - K } , we have for y < /?
+
P X [ R ,= R,; R , < min((, TK)] = 0.
+
For each countable ordinal /?,let q(p) = E"[R,/(I R&]. Since x E E - K , 0 < q(p) I 1 for all /? and q ( y ) I q(p) whenever y I /?.Moreover if y < /? and P X [ R ,< min((, TK)] > 0, then q(y) < q(/?). Consequently there exists a countable ordinal Po such that P"[R,, = min((, TK)]= 1. We know f ( x ) 2 P R I f ( x ) Suppose . that f ( x ) 2 PRYf ( x ) for all y < b. If j has a predecessor, /? - 1, then (5.12) implies thatf(x) 2 P R g f ( x ) If. /? is a limit ordinal, then we can find an increasing sequence {p,} such that R," t R, almost surely P", and consequently X(RB,)+ X ( R B )on { R , < 5) almost surely P". Therefore using the lower semicontinuity o f f and Fatou's lemma we obtain f ( x ) 2 P R g f ( x ) In . particularf(x) 2 P R p , f ( x )= P K f ( x ) . So far we have shown that f ( x ) 2 P K f ( x )for any compact subset K of E and x E E - K . If x is regular for K , then P K f ( x )= f ( x ) . Finally suppose that x E K - K'. Then x must be regular for E - K and so there exists an increasing sequence {K,} of compact subsets of E - K such that Px(TK,J.O) = 1. Since X(TK,)E E - K almost surely on {TK,< 0 0 } and x E E - K, we obtain with two applications of what has already been established E"{f[X(TKn
+
TK
e,Kn)l}
=
EX{P,f[X(TK,)l} PK,f(X)
On the other hand X ( T K ,+ TKo Or,) + X ( T K )almost surely P";in fact there is equality as soon as TKn < TK. Thus Fatou's lemma implies that f ( x ) 2 P K f ( x ) for x E K - K' as well. One final appeal to the lower semicontinuity off and Fatou's lemma yields lim inf,,, P If 2S,and so Theorem 5.11 follows from Corollary 5.3. We are now going to identify some of the objects defined in this chapter with objects from classical potential theory when the process X in question is Brownian motion. Therefore let X = (R, A, A , ,XI, O,,P") be the Brownian motion process in R".We assume that Xis given in its natural function space representation; that is, R consists all maps w : [0, co3 + (R"u {A}) which are continuous on [0, a),w ( w ) = A, and o(t)E R" for all t c 0 0 ;
5. ALTERNATIVE CHARACTERIZATION OF EXCESSIVE FUNCTIONS
95
+
X,(w) = w ( t ) , 0, w(s) = w(s t ) ; and and A, are equal to 9 and 9,, respectively, for all t. If r > 0 let Tr = inf { t : IX, - X,l > r } where Ix - yl is the usual Euclidean metric on R".The key fact which we need is that PTr(x,.) is the uniform distribution of unit mass on S,.(x) = { y : Ix - yl = r } . Clearly the continuity of the paths implies that P,,(x, . ) is carried by Sr(x). First of all let us show that Px(Tr< 03) = 1, which implies that PTr(x,Sr(x))= 1. Fix x and r > 0 and let T be the first hitting time of { y : ly - XI > r } . Let g,(x) be the Gauss kernel defined in (2.17) of Chapter I. Then P"(T, It ) = P"(T It ) 2 P"[lX, - XI > r ] =
j,y,&(Y) d Y , >r
and using the form of the Gauss kernel it is easy to see that this last expression approaches one as t approaches infinity. Hence Px(Tr< co) = 1. Next let 0 be a distance-preserving transformation of R".Then 0 induces a transformation 0 on R by (Oo)(t)= O w ( t ) if t < co and (Ow)(co) = A. We now claim that P x 0 - ' = P o x for any such 0 and all x. Indeed first observe that if P , ( x , A ) is the transition function of X , then P,(x, O - ' A ) = P,(Ox, A ) for any such 0. Consequently if 0 It, < . . . < f k and A l , . . . , A , are in g(R"), then
Thus P " 0 - I = P o x on F0, and hence on 9. Finally note that T,(Ow) = inf{t: IX,(Ow) - X,(Ow)I > r } = inf { I : =
IOX,(o) - OX,(w)/ > r }
TAW),
and so XTr 0 = O X T, .Let x and r > 0 be fixed and suppose Ox = x. Then if r is a Bore1 set 0
P , ~ ( ~0, lr)= P"[x(T,) E 0- '1-1
rl = P O ~ [ X ( T , )E rl
= P X [ O X ( T , )E
= P,,(x,
r);
that is, the measure PTr(x, .) is invariant under all distance-preserving transformations of R" which leave x fixed, Combining this with our previous
96
11. EXCESSIVE FUNCTIONS
observations it follows that PT,(x, .) is indeed the uniform distribution of unit mass on S,(x). In this paragraph we assume that the reader is familiar with classical potential theory as expounded, for example, in Brelot’s monograph [l]. Clearly the Brownian motion process X is a strong Feller process, (2.16), and hence excessive functions are lower semicontinuous. It now follows from Theorem 5.11 and the above evaluation of Ps,(.Jx, .) that f E Y if and only i f f = 00 or f is a nonnegative superharmonic function. In particular, since the only nonnegative superharmonic functions in R or RZ are the constants, we see that the only excessive functions for Brownian motion in one or two dimensions are the nonnegative constants. On the other hand if n 2 3, then X satisfies the condition of (4.6) and so the fine topology is the coarsest topology relative to which the nonnegative superharmonic functions are continuous; that is, it coincides with the Cartan fine topology of classical potential theory; see Brelot [ l , p. 901. The reader familiar with classical potential theory should now have no difficulty in checking the following facts (n 2 3): (i) A is thin at x as defined in this chapter if and only if A is thin at x in the potential theoretic sense; and (ii) the concept of polar set in this theory and in potential theory agree. Moreover it follows from results of potential theory that a thin set, and hence a semipolar set, is polar. Finally we remark that a set is polar if and only if it has capacity zero. These last two statements will be proved in much greater generality by probabilistic methods in Chapter VI.
MULT
AND SUBPROCESSES
In this chapter we develop the fundamental properties of multiplicative functionals of a Markov process. Multiplicative functionals arise naturally when one “kills” a given process according to some rule (see Sections 2 and 3). The reader could skip Section 3 and the second half of Section 5 on a first reading. Also Section 6 stands somewhat apart from the remainder of the chapter. This section contains Hunt’s characterization of the hitting operators Pi. This result is one of the most important in probabilistic potential theory and could have been included in Chapter 11. However we deferred it to the present chapter in order to discuss it in somewhat greater generality than would have been possible in Chapter 11. The reader particularly interested in this result might well go immediately from Section 6 of this chapter to Section 1 of Chapter V where related questions are discussed.
1. Multiplicative Functionals
Let X = (Q, A, A , ,X , , O r , P“) be a Markov process with state space (E, 8).We will shortly impose more conditions on X but the following definition makes sense for any Markov process. DEFINITION. A family M = { M , ; 0 5 t < a}of real-valued random is called a rnultiplicatitie functional of X provided : variables on (Q, 9) (i) M I E Frfor each t 2 0; (ii) M I + ,= M , ( M , 0,) a s . for each t , s 2 0; (iii) 0 I M,(w) I 1 for all t and w.
(1.1)
0
97
98
111. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
Most authors do not assume that multiplicative functionals satisfy Condition (iii). Thus we should perhaps call a multiplicative functional satisfying (iii) a multiplicative functional with values in [0, 11; however, this is the only type of multiplicative functional we will consider in this book. From now on we will write “ M F ” in place of “ multiplicative functional.” We emphasize that the exceptional set in (ii) depends on t and s in general. Also it follows from (ii) and (iii) that, for each t and s, M , , , I M , almost surely. Sometimes it will be convenient to write M ( t , w ) for M,(w) and M ( t ) for M I . We say that a multiplicative functional M is measurable provided the family ( M I }is progressively measurable relative to {F,}. We say that M is right continuous (or continuous) provided t --f M,(w) is right continuous (or continuous) almost surely. In particular a right continuous multiplicative functional is measurable, It will be convenient to let M , = 0 for any MF. The relationship Mo = Mo(Mo 0 8,) = M i a s . implies that almost surely M , is either zero or one. We call a point x in Epermanent for M if P X ( M ,= 1) = 1, and we let E M denote the set of permanent points. Clearly E M E d *. In case X is normal, the zero-one law implies that x E E - E M if and only if P”(Mo = 0) = 1. We next give a few examples. The reader should verify for himself that these are indeed MF’s. (1.2)
For each a 2 0, MI = e-“’ is a M F and
EM
= E.
Recall from (2.18) of Chapter 11 that a terminal time T is an {F, stopping } time satisfying, for each t 2 0,
(1.3) T = t
+T
0
8, almost surely on {T > t ) ,
Let T be a terminal time and define M , ( o ) = 1 if t < T(w)and M,(w) = 0 if r 2 T ( w ) ;that is, M, = (t). In this case EMconsists of those points x in E which are irregular for T, that is, such that P”(T > 0) = 1. (1.4)
Terminal times will play an important role in this and later chapters. Intuitively one should think of a terminal time as thejirst time some physical event occurs. Since t + T 0 8, is then the first time the event occurs after t , the relationship (1.3) becomes intuitively clear. (1.5) Let X be progressively measurable with respect to {F,} and let f E 8 , . Define
M ,= exp( - J ‘ ~ ( xds). ~) 0
99
1. MULTIPLICATIVE FUNCTIONALS
Obviously Example (1.2) is continuous and (1.4) is right continuous. Example (1.5) is continuous i f f is bounded, but need not be even right is right continuous and we continuous i f f i s unbounded. However if {F,} define
then T is a terminal time and Nt
= I [ O , T , ( f ) exp(-S'f(X,) 0
ds)
is a right continuous multiplicative functional. In addition for each o the functions t + N,(w) and t + M,(w) agree for all values of t except possibly t = T(o).
In general we will only be interested in a M F during the time the trajectory t + X, is in E. With this in mind we make the following definition.
(1.6) DEFINITION. Two MF's of X , say M and N, are equivalent provided P"[M, # N , ; X , E El = 0 for all t and x.
If EA is a metric space and Xis right continuous, and if, in addition, both
M and N are right continuous, then this is equivalent to the statement that t + M,(w) and t + N,(w) are identical functions on [0, c(w)) almost surely. Observe that if M is a MF, then N , = M , I E ( X , )is a M F and that M and N are equivalent. We will ordinarily be interested in MF's only up to equivalence and consequently when this is the case we may assume without loss of generality that M,(w) = 0 whenever XI(@)= A . Finally note that if EA is a metric ) equivalent right space and X is right continuous then I E ( X , )and l L 0 , [ ) ( tare continuous MF's. The following proposition is of some interest. We leave its proof to the reader as Exercise 1.10.
PROPOSITION.Let M be a M F of X and let (9,) be right continuous. Then a necessary and sufficient condition that M be equivalent to a right continuous M F of X i s that t + E"(M,) be right continuous at t = 0 for all x.
(1.7)
If M is a M F of X we define for each t 2 0 an operator Q, on bb* by (1.8)
Qrf(x) = E"{f(X,) Mrl
= E " { f ( X t ) Mi;
Xi E El
(recall that numerical functions on E are extended to EA by setting f(A) = 0). Clearly Q, is a positive linear map from bS* to bb* such that Q, IPI where
100
111. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
PI is the transition operator of X . Moreover Qt
+ s f ( x ) = E"Cf(xt + s) = E"C(f0
Xt
0
Mi
+s1
0s) Ms(Mr 0 031
= E"{M,EX("[f(X,) = Qs
M,]}
Qrf(x),
and so { Q , ; t 2 0) is a semigroup-the semigroup generated by M . Obviously equivalent multiplicative functionals generate the same semigroup. The following proposition contains the converse of this statement.
(1.9) PROPOSITION. Two multiplicative functionals are equivalent if and only if they generate the same semigroup { Q , ; t 2 0). Proof. Let M and N be MF's. We must show that if E " { f ( X , ) M I }= E x {f ( X t )N,}for allfE bb*, then M and N are equivalent. For this it suffices to show that if t and x are fixed, then for each H E b F , one has E X { H M , ; XI E E } = E"{HN,; X I E E ) . But the set of H E bFt for which this last relation holds is a vector space containing the constants and closed under monotone convergence. Thus by MCT we need only consider H of the form n ; = , f i ( X , , ) where 0 It , < . . . < t, = t and eachfi E bb*. If n = 1 the desired equality is just our hypothesis. Suppose 0 It, < .. . c t, c t , + , = t ; then
If we let g,(x) =fn(x) Q t - , n f n + l ( ~ )the , induction step is clear. Thus (1.9) is proved. Let us point out that Q, need not be the identity operator; in fact Q o f ( x )= E"{J(X,) M , } . In particular if X is normal Q , f = I E M $ Finally, if Ed is a metric space and Xis right continuous and if M is right continuous, then Q,l(x) = E"{M,; X I E E } approaches Qol(x) = E X { M , ;X , E E } as t 40. In Section 3 we will give conditions under which there exists a Markov process whose transition semigroup is { Q,}. However, before that, we are going to show in the next section that under mild restrictions any semigroup that is dominated by { P , } is generated by a multiplicative functional.
2.
SUBORDINATE SEMIGROUPS
101
Exercises
(1.10) Prove Proposition 1.7. [Hint: show that, under the hypotheses of ~ , ~a ~right ~ continuous M ~ M F equivalent to M.] (1.7), N , = S U ~ ~ , defines (1.11) Let M be a M F such that almost surely the mapping t -,M , is nonincreasing. Let T = inf{t: M , = O}. Show that T is an {F",} stopping time and that T = t T 0 8, almost surely on { T > 1 ) for all t. In particular if {%,} is right continuous, then T is a terminal time.
+
2. Subordinate Semigroups Let X = (Q, A,A , ,X , , 8,, P") be a Markov process with state space ( E ,8).Let {f,}denote the semigroup of transition operators of X and B = bb*. We have seen in Chapter I that P,B c B for each t 2 0.
DEFINITION.A semigroup { Q,: t 2 0 ) of nonnegative linear operators on B is subordinate to {P,}if Q , f r P,ffor each tr 0 andfE B +.
(2.1)
It is an immediate consequence of the definition that IIQ,ll I llPrII I 1. Since P , f ( x ) = P,(x, dy)f(y) f o r f e B, it follows that for each t and x there exists a measure Q,(x, on 6* such that Q,(x, *) I P,(x, *) and Q , f ( x ) = Q,(x, dy),f(y) for allfe B. In particular Q,(x, A ) is a transition function on ( E , b*).We showed in Section 1 that if M is a MF of X , then the semigroup generated by M is subordinate to {P,}.The next theorem, which is due to Meyer [ 2 ] ,states that under mild regularity assumptions on X any semigroup subordinate to {P,}is generated by a MF. Before stating this theorem we introduce the following condition on X : 0
)
(a) There exists a countable family &? c 8 such that a(&?)* = b* and {x} E & for each x E E. (b) Given t > 0 and J, a countable dense subset of [0, t ] containing t , then a(X,: s E J ) " = %, . Recall that if Q c 9 then 9 is
(2.2)
(See (5.3) of Chapter I.) the completion of 9 in 9relative to the family {P}. Observe that if EA is a separable metric space with &?(E,) c bAc 9?(EA)* and if X is right continuous, then (2.2) is certainly satisfied. In particular (2.2) is satisfied if X is a standard process.
(2.3) THEOREM. Let X be normal'and satisfy (2.2). If { Q,} is a semigroup subordinate to {P,},then there exists a multiplicative functional M of X which generates { Q,}. If {F,} is right continuous and t + Q,l(x) is right continuous at t
=0
for all x, then one may take M to be right continuous.
102
111. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
Since X i s normal P,(x, .) = E , . But Qo(x, .) I P,(x, .) and therefore = c(x) E,. Plainly the fact that Q , = Q: implies that c(x) is either zero or one. Let E, = { x : Q o l ( x ) = l } ; then E, E b* and Q,(x, .) = ZEo(x)E , . The points in E, will be called permanent for { Q , } . Observe that for any t and x
Proof.
Qo(x,
a)
so that all of the measures Q,(x, -) are concentrated on E, . Also Q,(x, E) = Qo(x,dy) Q,(y, E) = 0 if x E E - E,; that is, Q,(x, = 0 for all t if x # Eo . Since Q,(x, .) I P r ( x , for each fixed x and t , it follows from the RadonNikodym theorem that there exists a function 9,(x, y) defined on R, x E x E such that Q,(x, dy) = 9?(x,y ) P , ( x , dy). We may assume that 0 I 9,(x, y ) I 1 for all ( x , y ) . Since x + P , ( x , A) and x--, Q,(x, A) are b* measurable for each t > 0 and A E I *it follows from (2.2a) and a theorem of Doob [I, p. 3441 (see also Dynkin [2, p. 2181) that we may assume that 9 , E I * x I*for each t. By the remarks in the preceding paragraph we may suppose that 9 , vanishes off E, x E , . It will be convenient to extend 9 1 to EA x EA by letting 91 be zero if either of its arguments is A. The extended function is again denoted by 9 , . Evidently q1 E I,*x I,* and vanishes off E, x Eo . Given t > 0 let V = {0 = t o < t , < .. . < t, = t} be a finite subset of [0, t] containing 0 and t. Let 9(U)= .(X,: s E V ) " ,and define 9)
a)
Clearly M , ( V ) is in 9 ( V ) c 9,, and 0 I M , ( U ) I 1. Also note that M , ( U ) (0) = 0 if X,(w) = A. As in Eq. (2.8) of Chapter I the formula
In particular, for any f E B and x E E
As we observed just following the statement of Theorem 2.1 I of Chapter I, the family { Q"(x, as U ranges over the finite subsets of [0, t ] containing 0 and r , is a projective system. In particular, if V = (0 = to < t , < .. . < t, = t } , V is a finite subset of [0, t ] containing V, and (D is an w function of the form a)},
2.
103
SUBORDINATE SEMIGROUPS
@ = h(X,, , . . . , X,") with h E bb"", then
EX{@ M , ( U ) } = EX{@M , ( V ) } . Consequently we have (2.6)
E"{M,(V) I m u ) } =
whenever U c V . Let { U,,} be an increasing sequence of finite subsets of [0, r], each containing 0 and t , and such that D = U,, is dense in [0, t ] . Let M: = M,( U,,) and 9" = 9 ( U n ) . It is immediate from (2.6) that { M : ; 9"} is a martingale relative to the measure P" for each x. Since each M: is in 9, and 0 I M: I 1, it follows from the martingale convergence theorem (( 1.4) of Chapter 0) that measurable random variable M , such that 0 I M , I 1 and there exists an 9, M: --+ M , as n + 03 almost surely. Moreover we may assume that M , = 0 if X , = A since each M: has this property. Suppose {On}is another increasing sequence of finite subsets of [0, t ] , each containing 0 and t, and having D = UO,, dense in [O, t ] . If R, is defined relative to the sequence {On}in the same manner as M , is defined relative to { U,,}, then we claim that M , = M t almost surely. To prove this it clearly suffices to suppose that U,, c o,, for each n. But then (2.6) implies that for p > n we have
u
EX(Rp I F n )= M:
= EX(Mp
I 9")
for all x. Letting p + 03 we obtain E"(R,- M , ; A) = 0 for all A E U,,.W' and hence, from (2.2b), for any A E 9,. Therefore M , = a, almost surely since both M , and R, are in 9,. For each t > 0 we define M , as above using an increasing sequence {U,,}. If t = 0 we define Mo = lEo(Xo).Observe that IEo(x)= Qo(x,{x}) = qo(x, x) P,(x, {x}) = qo(x,x). We will now show that { M , } is a M F and that (2.7)
Qtf(x) = E"Cf(x,>Mil ;
f e B.
Obviously 0 5 M , I1 and M , E F, for all t . It is also clear from (2.5) and the definition of M , that (2.7) holds. Moreover M , ( w ) = 0 if X,(w) = A . Thus it remains to verify that (2.8)
M I + ,= M , ( M , 0 0,) almost surely.
Assume first that t > 0 and s > 0. Let D consist of the rationals in [0, t + s], t + s, and all numbers of the form s + r with r a rational in [0, t ] . Let {U,,} be an increasing sequence of finite subsets of D, each of which contains 0, s, and t + s and such that D = U,,. Let V,, = U,, n [0, s] and let W , = { r 2 0 : r + s E U,,}.Clearly { W,,} is an increasing sequence of finite subsets of [0,1], each of which contains 0 and t , whose union is dense in
u
104
111. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
[0, t]. If U,, = (0 = to < tl < . . . < ti < s < is easy to verify that
< . . . < tp = t
Mt+s(Vn) = Ms(vn) CMt(Wn) 0
+ s},
then it
Osl,
and so letting n + 03 weobtain (2.8) in this case. If t = s = 0 then(2.8) isclear. Finally if t > 0 and s = 0 or t = 0 and s > 0, then using the fact that qu vanishes off Eo x Eo one can easily verify that (2.8) holds. We have now proved the first assertion of Theorem 2.3. But E X { M , }= Q , l ( x ) since M , = 0 if X , = A, and so if t + Q , l ( x ) is continuous at t = 0 for all x so is t --t E " { M , } . Therefore we can apply Proposition 1.7 to obtain a right continuous MF, N, equivalent to M provided {F, is} right continuous. This establishes the last sentence of Theorem 2.3. REMARK. Note that Eo is the set of permanent points of M , that is Eo = E M .
Exercises (2.9) Let X be normal and satisfy (2.2). Let Q,(x, A) be a transition function on (E, 8*)such that for each x E E and t 2 0 the measure Q,(x, -)is absolutely continuous with respect to P,(x, .). Let qt(x,y) be a density for Q,(x, .) with respect to P,(x, that is jointly measurable in x and y, and assume that q,(x, y ) = 0 if either x or y equals A. If U is a finite subset of [0, t ] containing 0 and t , then define M I (17) as in (2.4). Show that for each t 2 0 there exists a nonnegative M , E 9, such that M,( V,,)+ M , almost surely whenever { V,,} is an increasing sequence of finite subsets of [0, t], each containing 0 and t , such that UU,, is dense in [0, t]. Show that { M , ; t 2 0 ) satisfies (i) and (ii) of Definition 1.1. Let us call such a family a generalized nonnegatiuemultiplicatiue a)
functional (GMF for short). Show that Q , f ( x ) 2 E " { f ( X , ) M , } if f E B,. Finally show that Q , f ( x ) = E " ( f ( X , )M I }for allfe B, ifand only if { M , ( U ) ; U a finite subset of [0, t] containing 0 and t} is uniformly integrable relative to P".
Let X = (a,Po,F:,A', O f , P") be a Markov process of function space type having (E, 8) as state space (see (4.2) of Chapter 1). Assume X is F:,XI, O,, p) be another normal and satisfies (2.2). Let 9 = (a,Fo, Markov process of function space type with the same state space (E, 8). Show that the following two conditions are necessary and sufficient that the restriction of p to 9 :be absolutely continuous with respect to the restriction of P" to 9:for all x and t: (i) fi,(x, -) 4 P l ( x , .) for all t and x , and (ii) using the notation of (2.9) with Q,(x,A) = fi,(x, A), for each t and x, { M , ( U ) ; U a finite subset of [0, t ] containing 0 and t } is uniformly integrable relative to (2.10)
3. SUBPROCESSES
105
P". Moreover there exists a GMF, { M I } ,of X such that, for each x, M I is a version of the Radon-Nikodym derivative of b"I 9:with respect to P" 1 9:.
3. Subprocesses Let X = (R, A, A , ,X , , O r , P") be a Markov process with state space ( E , a).Let Eo E I* and let 8: be the trace of I *on E, . A Markov process Y with state space ( E , , 8:) is called a subprocess of X provided its transition semigroup { Q , } is dominated by {P,} in the following sense: Q , f ( x ) I P , f ( x ) for all x E E, a n d f 2 0 in bb:, wheref is taken to vanish on E - E, in forming P, f. If we define 0, on B = bI* by setting Q, f ( x ) = Q r f E o ( xfor ) x E E, wheref,, is the restriction o f f t o E, , and Q r f ( x )= 0 for x E E - E , , then { Q,} is a semigroup of nonnegative linear operators on B which agrees with Q , on bb:. The above definition is then equivalent to the statement that the semigroup { Q,} is subordinate to { P I }as defined in Section 2. We will now drop the notation 0, and write Q , for the extended operators defined on B. In the remainder of this section we will assume that X i s normal although this is not necessary for the validity of all that follows. If X satisfies the conditions of Theorem 2.3 and Y is a subprocess of X , then there exists a multiplicative functional M of Xsuch that for each x E E , , t 2 0, and f~ bd* vanishing off Eo one has ,!?"{f(Y,)} = E " { f ( X , ) M , } where fi is the expectation operator corresponding to Y. In this section we are going to establish a partial converse; namely, given a right continuous MF, M , of X we are going to construct a subprocess Y satisfying the above condition. Roughly speaking Y , is obtained by " killing" X I at a rate -dM,/M,; that is, - d M , / M , is the "conditional probability" that XI is killed in the interval ( t , t + dt) given that it is " still alive " at time t. We will now make this discussion precise. A) We begin with some definitions. Let fi = R x [0, co] and write & = (0, for the generic point in fi. Let g be the a-algebra of Bore1 subsets of [0, co] and set .k = A x 9. Let R : fi 4 i2 and y : fi 4 [0, 001 be the natural projections; that is, R ( W , A) = o and y ( o , A) = A. Let &A = (oA, 0) and A) = [(u)A A. Define 8,(&) = X,(w) if set = ([ o n) A y ; that is, ((o, t < A and XI(&) = A if t 2 A ; here, of course, 03 = ( 0 ,A). Note that p = inf{t: 8,= A}. We will often write simply o and A for n(03) and y(&). Define 8,03 = (O,w, (A - t ) v 0) where co - 03 is taken to be zero. Note that 8, 03 = dAand 8,o a,, = 8,+,for all t and h. Let fir= R x ( t , co] E A@, and define A@, to consist of all sets A E .k for which there exists a A E A, such that n fir= A x (1, 031.
e
(3.1) PROPOSITION. {A@,}is an increasing sequence of sub-a-algebras of
106
111. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
4 and 2,E &,/S, for all t. Moreover if
{ A , }is right continuous so is
{-/%,I. Proof. The reader will easily verify that {A,} is an increasing sequence of sub-a-algebras of A. If A E 8, then (8,E A} = { X , E A } x ( t , m] E J?,. Thus the first sentence of (3.1) is established. Regarding the second, fix t < 00 and suppose A E A,+ . Then for each n 2 1, A E and so there is a set A, E such that A n = An x ( t l/n, 001. It is easy to check that all the sets A,, are the same. If we set A = A1 then A E .MI+llnfor all n, and since { A , }is right continuous A E A,. Clearly then A n 6, = A x ( t , co] and so E A,. Thus the proof of (3.1) is complete.
+
We now suppose that we are given a right continuous multiplicative functional M of X . Let R, be the set of w's such that t -,M,(w) is right continuous, nonincreasing, and M,(w) is either 1 or 0. By definition Px(R,) = 1 for all x. For w E R, we define a measure a, on [0, m] by a,((& 001) = M,(w) for 1 E [0, co] and a,({O}) = 0. This defines a unique measure, a,, on the Bore1 sets of [0, m]. Since M , = 0 by convention, a, is a probability measure if M,(w) = I and a, = 0 if M,(w) = 0. For convenience we define aU = E, if o # R, . Since M AE 9, and R, E 9it is immediate that w + a,( I-) is 9 measurable for any r E 9. We next define measures p on as follows: if E d? = M x 9 and if = {A: (0, A) E A}, then A'" E 9 and a,(A'") exists for each w E Q. Moreover w --t a,(A") is in 9 since this is true for rectangles in A x W. (Use MCT.) Recall that E M = { X E E : P"(M, = 1) = I } is the set of permanent points of M. We now define
P(A)= E"[a,(A")] if X E E ~ , p = unit mass at dA= (u,, 0) if x E E , - EM. Clearly each p is a probability measure on (0,A).In order to avoid trivial
(3.2)
notational difficulties we will now assume, as we may without loss of generality, that R, = R. We come now to the basic result of this section. The notation is that established above.
(3.3) THEOREM.2 = (0,A@, A,, 8 , ,8,, p) is a Markov process with , )E"{J(X,) } M,} for all J E bb*; state space (E, b*) such that ~ " { f ( ~ = that is, 8 is a subprocess of X whose transition semigroup is generated by M. = A) = I . If A Proof. Since A 4 EM it is clear that fiA(8, {g,E A } = {XIE A } x (t, m] and so
(3.4)
P12 E A ) = E"{M,; x,E A } .
E b*,
then
107
3. SUBPROCESSES
Therefore x + p ( 8 , ~ A is ) b* measurable for each A E&*. Since (3.4) remains true if one replaces IA by any f E bb*, the only thing that remains to be checked is the Markov property of 8 ;that is, for all B E B*, t , s, and x we must show that (3.5)
F{lB(8,+s); A} = p{fii(f)[1B(8s)]; A}
for all A E A,.If x 4 EM both sides of (3.5) are zero, and so we may suppose that X E E M . Observe first of all that MCT implies that if Y Eb.& and x E EM then
F(Y ) =
E X {
1
Y ( 0 , I ) cc,(dl)).
By definition of d t ,A n f i , = A x ( t , co] where A E A , . Since B c E, {$,+, E B } c and so {8,+, E B } n A = ( ( X , + , E B }n A) x ( t s, co]. Therefore the left side of (3.5) may be written
+
Ex{lB(xt+s)
Mt+s;
A> = E x { E X " ' [ ~ B ( x s )
Ms]Mt;
A}*
On the other hand if F ( y ) = E y [ I B ( 8 J = ] Ey[IB(Xs)M,],then since F ( A ) = 0 the right side of (3.5) becomes
E x [ F ( 8 , ) ;A
n fit] = E X { F ( X , )M,;A},
and hence (3.5) is verified. Thus Theorem 3.3 is established. In the sequel we will refer to the process 8 constructed above as the canonical subprocess corresponding to M . Note that we do not complete the a-algebras A and A?, in the appropriate manner relative to {P.'; x E E A } . Thus for canonical subprocesses we do not assume condition (5.15) of Chapter I. Of course, p is the lifetime of 8. It is immediate that if M and N are equivalent right continuous multiplicative functionals, then the corresponding canonical subprocesses are equivalent processes. Also note that if x E EM, then p[fi,= x] = P x [ X , = x] = 1 while p [ 8 ,E E l = 0 if x E E - EM. Consequently it is natural to ask if X can be considered as a Markov process with state space ( E M ,8;) where 6'; is the trace of b* on EM. Unfortunately this is not always possible. See Exercise 3.19.
(3.6) REMARK.Let A* = A x {S, [0, a]}.One should think of A* as the a-algebra in fi containing the information in the original process X. From the definition of the measures p it follows immediately that
P*{y > t I A*}= M ,
0
n
for all x E E,. In particular the conditional probability depends only on the behavior of the original process in the time interval [0, t ] . Also it is this
108
111. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
equality that is taken as justification for the intuitive statement that obtained by " killing " X at a rate - d M , / M , .
8 is
(3.7) REMARK.Suppose that T is an {St,} stopping time such that t + T 8, = T on {T > t} for each t 2 0. This is slightly stronger than assum0
ing T is a terminal time for X , which requires that the above identity should hold only almost surely for each t 2 0. For simplicity assume that T I (. This can always be accomplished by replacing T by min(T, (). If M , = ILO,T)(t), then M , is a right continuous MF. In this case EM is the set of points in E which are not regular for T, and u, is easily seen to be unit mass at T ( w ) if T(w) > 0. If @ : Q + 6 is the map w + ( w , T ( w ) ) ,then @ E .A?/&. Finally define f , ( w ) = X,(w) if t < T ( w ) and f , ( w ) = A if t 2 T ( w ); 8, w = 8,w if t < T ( w ) and 0,w = w,, if t 2 T(w). It follows from the definitions that ft, O @ = 8 , and 8,+s o as. Also if A E .A? and A = A x (A, 031, then @-'A = A n {T > A} and consequently P W - ' A = P"(A n {T > A}) = E"(M,; A) = p ( A ) . Therefore P" @-' = on & and this makes it clear that 8 = (Q, 9, f , ,8,, P") is a Markov process equivalent to 8. It is usually simpler to work with 8 than with 8 when M has this special form.
=xi
(3.8) REMARK.If T is merely assumed to be a terminal time for X and M ,= I L O , T ) (one f ) can still carry out the above construction o f 8 , and it is evident that, for each x, 8 is a Markov process over (Q, A, P") in the sense of Definition 1.1 of Chapter I and that it is equivalent to 8.The only drawback is that f , , ,= 8, g,, only almost surely in this case. This is an unimportant difficulty, but as a result 8 does not satisfy the assumptions of Definition 3.1 of Chapter I as it stands. 0
We are now going to investigate the regularity properties of 8. For example if EA is a metric space and X is right continuous (or has left-hand limits), then it is obvious from the construction that 8 has the same properties. In order to discuss the strong Markov property for 2 or the quasi-leftcontinuity of 8 we prepare the following lemma. We use the notation already developed in this section. (3.9) LEMMA.Let ?be a stopping time relative to {J?,}.Then: (a) There is a unique { A , }stopping time T (defined on Q) such that T A y = (T 0 n) A y. (b) Let p a n d T be as above. Then given A E J?? there is a set A E &IT such that A n { < y } = (A x [0, 001) n ((0,A): T ( w )< A}. REMARKS.y is an {d,} stopping time since {y It} n 6 , is empty for each t. Observe that y 0 8 , = y - t on { y > t } . The relationship whose
3.
109
SUBPROCESSES
existence is asserted in (a) may, of course, be written
p(0,A) A A
(3.10)
= T(w)A
A
for all w and 1. Proof of (a). First of all suppose that p i s some numerical function on fi and that T is a numerical function on R which is related to $by (3.10); then it is evident that T is uniquely determined. Suppose in addition that p i s an {J?,} stopping time. If A = { w : T ( w ) I a}, then A x (a, co] = {PI a} n fia. But by the definition of {A,} and the fact that pis an {d,} stopping time this last intersection is of the form A, x (a, co] with A, E Aa.Consequently T is an { A , }stopping time. Let f denote the class of all {d,} stopping times p f o r which there exists a numerical function T on R related to by (3.10). Write T = dD(T). We have already seen that T is uniquely determined by T and that T is an { M I }stopping time. Next let { p,} be any family of elements o f f and T, = @(pa). If 5'= inf pa is known to be an {A,} stopping time, then obviously FEf and @(p)= inf T,. Finally suppose that f' is an {A,} stopping time of the form
T(w, A) = a = 03
if
(0,
A) E fa,
if ( w , , ~ ) # f , ,
where faE 2,. Now f,nfi, = rax (a, 001 with r,E M a . If we define T(w) = a for w E r, and T ( w ) = 00 for w 4 r,, then T is an { A , }stopping time and one checks easily that p and Tare related by (3.10). But any { A t } stopping time pis the infimum of a countable family of stopping times of this special form and so from our previous remarks it follows that TE'?. Thus Assertion (a) is established. Proof of (b). Recall that saying A E d7, means that { p I a } E dafor all a E [O, 00). Given A let
ii, = A n { ( w , A): ~
A Ed
and that
An
( wI ) u , A > a } = ii n { T I a } n fia
ua
for each a 2 0, so that A n { p< y } = A,, the union being over the noneach A, is of the form negative rationals, for example. By the definition of d, A, x (a, 001 with A, E A,. Plainly A, c { T I a} for all a, h b c A, if b I a, and A, n { T I b} = n { T I b} if b 5 a. Consequently if b 2 a then A a n { T I b} = A , E .Mac& b , while if b I a, A a n { T I b} = A b n { T I b} E M b. Therefore each A, is in AT. If A = Aa , the union being over the nonnegative rationals, then A E AT and A n {T I a} = A, for any nonnegative rational a. Therefore
ua
110
111. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
(A x [0, m]) n {(a,A): T(a)c A} = u ( A n ( T Ia } ) x ( a , a] a
= U
=Uha=hn{f'
completing the proof of (b). Let us assume now that X is strong Markov (as well as normal) and that M is a (right continuous) M F of X , and let 8 be the canonical subprocess corresponding to M. We are now interested in conditions that insure that 8 is strong Markov. Simple examples show that this is not always the case. (see Exercise 3.19). First note that since M is right continuous { M I } is proand hence with respect to { A , } gressively measurable with respect to {F,}, since we are, of course, assuming that A, 2 F t . See (5.1 5 ) of Chapter I. If T is an { A , }stopping time then Theorem 6.11 of Chapter I implies that M T E A T .
(3.11) DEFINITION.A MF, M, of X is said to be a strong multiplicative functional (SMF) provided that M is right continuous and satisfies E"[f(xt+
7') M I +
TI = E"{EX'T'[f(Xt)
for all x, 1, { A , } stopping times T, and f
E
MT)
bd.
PROPOSITION. Let X be a strong Markov process and let M be a SMF of X. Then the canonical subprocess 8 corresponding to M is strong Markov.
(3.12)
Proof. Let f' be an {A@,}stopping time. By (3.9) there exists an {.A',} stopping time T such that '?(a, A) = T(a)on {f'(a,A) < A}. Therefore if A E b*, then ( 8 ,E A} n { It } n fi, = ({A', E A } n { T I t } ) x ( t , a] and so 8, E dT/&:. Hence according to (8.2) of Chapter I it suffices to show that W ( % + T H
for allfe bS*. Sincef(A)
= 0 we
= Ex(~i"'f(ft,)}
may assume that '?
Bx{f(8~+?)} = B x { f ( x ~ + T ) ; + < 71 = E"(f(Xt+T)
MI+T}
= E"{Ex"'[f(X,) h f , ] M T } =E"{EX(T)[f(8,)]hfT} =
and Theorem 3.12 is established.
&{E"9f(rZI)]},
3.
111
SUBPROCESSES
(3.13) PROPOSITION. Suppose EA is a metric space, 8, =I 9(EA), and X is quasi-left-continuous; then so is 8.
Proof. Let { p,,} be an increasing sequence of {A@,} stopping times with limit p; then we must show that 8(p,,)+ 8 ( p ) almost surely on { T < p } . Since ( = A y we may assume PI y. Thus there exist { A , }stopping times T and T,, such that ?= T A y and p,, = T,, A y. Consequently T,,t T, and therefore if XEE,
P[8(in)i+r7(f); T < [] = li"[X(T,,)t,X(T);T =E"[MT;
<5
A
y]
X(T,,)*X(T); T <
If x 4 E M , then 8(p,,)= A = 8(f) almost surely Proposition 3.13 is complete. In order to study the quasi-left-continuity of the following condition.
c] =o.
p", and
so the proof of
8 on [0, 00) we introduce
Let { R,,} be an increasing sequence of { A r }stopping times with limit R ; then M,,, .+ M, almost surely on { R <
(3.14)
c}.
PROPOSITION. Suppose EA is a metric space, b,= 9(E,), and M satisfies condition (3.14). Then if X is quasi-left-continuous on [0, a),so is 8.
(3.15)
Proof. Let {p,,} be an increasing sequence of {A?,} stopping times with limit F, Let T,, and T be { A r stopping } times such that p,, A y = Tn A y and P Ay = T A y. A simple inspection of the situation, using the definition of shows that it suffices and the fact that Xis quasi-left-continuous on [0, a), to verify that
r = {(a,A): ((a)> A, T,,(a)< A for all n, T ( a )= A} has P " measure zero for all x
E E M .But
for each n
r = w , ~ ~) :~ (<0A I ) ~ ( w ) , and so for each x
E
<
c(m,
EM
p(r)5 E x { M T n- M T ; T < [}
+O
as n + 03 by (3.14). Thus Proposition 3.15 is established. COROLLARY. Suppose X i s a standard process and M is a SMF of X satisfying:
(3.16)
112
111. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
(i) EM = E, i.e., all points are permanent for M; (ii) M, E 9 7 for each t . Then fz = 9, @, , 2,, 0, , "), where @ and 3@, have their usual meanings relative to the process 8,is a standard process (with the same state space as X). If in addition Xis a Hunt process and M satisfies (3.14), then ft is a Hunt process.
(a,
REMARK.Theorem 4.12 of the next section implies that if M is a right continuous MF with EM = E, then M is a SMF. Consequently in the hypothesis of Corollary 3.16 one need not assume that M is a strong MF; this actually follows from Condition (i) of (3.16). Proof. Since X is a standard process, {.,+'if} is right continuous and hence so is {dt}. But @: c d, and so fz is Markov relative to {&:+}. Consequently it follows from (8.12) of Chapter I that {@,} is right continuous. Clearly 2 is right continuous and has left-hand limits on [0, [) almost surely since X enjoys these properties. Next, in view of Theorem 7.3 of Chapter I, in checking the strong Markov property and the quasi-left-continuity of ft it suffices to consider {@:+} stopping times; since @+: c &, these properties follow from Propositions 3.12 and 3.13. So far we have not made any use of the two assumptions on M in our theorem. However (i) implies that ft is normal. (See the discussion following the proof of Theorem 3.3.) Finally if A E 8 thenP(fz, E A) = EX(M,;X , E A ) and so (ii) implies that x + p(8,EA) is 8 measurable. Therefore we may regard (E, 8) as the state space offt, and the first sentence of (3.16) is proved. The second sentence follows immediately from (3.15) of this chapter and (7.3) of Chapter I. (3.17) EXAMPLE.Let X be a standard process and let M, = e - p r , P > 0. Let X B denote the process in (3.16) corresponding to this MF. Then X p is called the b-subprocess of X. It follows from Corollary 3.16 that X a is a standard process, and if Xis a Hunt process then so is X p . If Pf denotes the transition semigroup of X p , then obviously Pf = e-prP,. In particular a function is a-excessive for X Bif and only if it is a + P excessive for X . The use of the P-subprocess will turn out to be an important tool in studying the original process X . Let us close this section by giving conditions under which (EM, b:), where 8; is the trace of b* on EM, may be taken as the state space of 2.An obvious necessary condition is that P x ( f t ,E E - EM for some t < [) = 0 for all x in EM. If we are willing to delete from fi a set which is almost surely be the hitting null, this is also sufficient. With this in mind let R = T E - E M time (for X) of E - EM and let us assume that R is an (9,) stopping time. Since 8,= X , if t < p the probability in question is just
4. RESOLVENTS AND STRONG MULTIPLICATIVE FUNCTIONALS
P[R < c]
= P[R < y ; R = E X { M R ;R
113
< []
< c).
Thus if R = TE-Ew is a stopping time, then a necessary and sufficient condition that we may regard ( E M ,)&; as the state space for 8 (after a trivial modification of is that M R = 0 almost surely on { R < [}, See Proposition 4.21 in this connection.
a)
Exercises
Show that there exists a standard process having the transition function defined in (9.16) of Chapter I. [Hint: let X be uniform motion to the right in R and obtain the desired standard process as a subprocess of X.] Show directly that condition (3.14) is not satisfied by the M F involved in the construction. (3.18)
(3.19) Let X be the Brownian motion process in R. Let M , ( o ) = 1 for all t if Xo(w) # 0 and let M,(w) = 0 for all t if Xo(w) = 0. Show that M = { M I }
is a continuous M F of X . Show that M i s not a SMF and that the corresponding canonical subprocess 8 is not strong Markov. [Hint: consider T = inf{t: X , = 0) and use (3.18) of Chapter 11.1 Describe 8 in this case. Note that E - E M = (0) and that dX[8,E E M for all t < p] = 0 if x # 0. Consequently E M can not be taken as the state space of 2. The notation is that of Proposition 3.1. Show that if f(o,I ) is A, measurable, then f ( w , A) =f(w, p ) whenever I , p > t. [Hint: show that if 3. b d , , then there exists an f E b A I such that f(o, A ) Zfit(o, I ) = (3.20)
f(44l,rn](~).l (3.21) Let X be a standard process and let G be an open subset of E such that each x in EA - G is regular for Eb - G. Show that TEA-fi= TEA-G almost surely. Let T = TEA-aand let r? be constructed as in (3.7) using this T. Show that we may regard (G, %) as the state space of r? where 9 = W(G) is the c-algebra of Bore1 subsets of G.If A is redefined as the point at infinity of G, then show that f is a standard process. f is called the restriction of X to G.
4. Resolvents and Strong Multiplicative Functionals In this section we are going to develop some criteria to insure that a right continuous M F is a SMF. Our main tool is a result about resolvents (Theorem
114
111. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
4.9) which is due to Meyer. Therefore we will first develop some general facts about resolvents and prove Theorem 4.9, after which we will return to the main problem of this section-the characterization of strong multiplicative functionals. We will assume throughout this section that X = (a,A, A , , X , , B , , P") is a given standard process with state space (E, b), although slightly less stringent assumptions would suffice. We will let B denote the Banach space b&*. Let M = {M,} be a right continuous M F of X and let Q , f ( x ) = E X { f ( X , ) M,} denote the corresponding semigroup on B. If YE C = C ( E ) , then t -+ Q, f ( x ) is right continuous for each x and hence ( 8 , x) -P Q, f ( x ) from [0, t) x E -+ R is in b(W, x &*) for each t when f~ C, where, as usual, W, denotes the Bore1 sets of [O, t ) . This last statement then extends to any f~ bb by use of MCT. Thus for any a > 0 we can define V " f ( x )= e-"'Qt f ( x ) dt, and V" maps b b into B. If Y Ebb and f 2 0, then V " ~ I U"fand so V" is given by a measure on (E, b) which we denote by V"(x,dy). Evidently V " f ( x )= j V"(x,d y ) f ( y ) and V" I(x) Ia - l . Each of the measures V"(x, -) extends uniquely to b* and hence we can define V u . f ( x )= V"(x,d y ) f ( y )f o r f e B. We next show that V% c B. To this end let p be a finite measure on b*. Then p V a ( B ) = j p(dx) V"(x,B) defines a finite measure on 8.Thus givenfE B there exist g , h E bB with g sf I h such that pV"g = pV"h. But this implies that V"g IV " ~ I V"h and V'g = V"h, a.e. p. Consequently V ~ B. E Let us next show that V " f ( x )= e-"' Q , f ( x ) dt forfE B. We know that (1, x) -P Q , j ( x ) is in b(W x b*) and that this formula holds for f~ bC. Given a finite measure v on b* and a finite measure 1 on 9 we can define for B E 8
5;
PL(B) =
J[ Qr(x,B ) 4 d t ) v(dx),
and obviously p is a finite measure on 8. Now i f f € B there exist g, h E bb such that g sfs h and p(g) = p(h). Thus Q , g ( x ) IQ r f ( x )IQ , h(x) for all ( t , x) and
Therefore ( 1 , x) -+ Q , f ( x ) is in (W x b*)*," (this is the completion of W x d* with respect to A x v) for all 1 and v. Thus for each x a n d f e B we can form e-"' Q , f ( x ) dt, and since this is a measure in f and agrees with V " f ( x ) forfE b b we must have V " f ( x )= e- Q , f ( x ) dr forfE B. In particular the above argument shows that t Q , f ( x ) is Lebesgue measurable for each x. Finally note that we have enough joint measurability in Q , f ( x ) to use Fubini's theorem.
4.
RESOLVENTS AND STRONG MULTIPLICATIVE FUNCTIONALS
115
There remains one more measurability detail to discuss: namely, to show that V " f ( x )= E x
Jo
e - " ' f ( X , )MI dt
for all f~ B. I f f € C it follows from the right continuity of the paths that W x %, and this continues to hold for f E bB. One now shows by an argument similar to that used above that (t, w)+ f ( X , ( w ) ) is in (9 x %)A,r for all 1and p whenfis in B, where the a-algebra in question is the completion of W x % with respect to 1 x P p , ,Ia finite measure on 9 and p a finite measure on Thus we can form the expression on the right side of (4.1) for anyfE B and since it is a measure which agrees with the left side of (4.1) on C the equality in (4.1) must obtain f o r f e B. Again we have enough joint measurability inf[X,(w)] to use the Fubini theorem. The family { V " ;a > 0 ) has the following properties: (t, w ) + f ( X , ( w ) ) is in
(4.2)
(i)
IIv"~~
I
a-l,
(ii) V " ~ I U"fiffEB+; (iii) 1/" -
V B = (fi - a ) 1 / " V , a, fi
> 0.
This last relation is the resolvent equation and is an easy consequence of the fact that {QI; t 2 0} is a semigroup (or (4.1) and the Markov property). We now make the following definition. DEFINITION. A family { V " ;a > 0 ) of positive linear operators on B is called a resolvent subordinate to { U"}provided it satisfies (4.2).
(4.3)
In particular it follows from the above discussion that the resolvent of a semigroup generated by a right continuous M F is subordinate t o { V " } . It follows from (4.2iii) that V " V B= V B V "for a, fi > 0. Sincef+ U " f ( x ) is a finite measure, Condition (4.2ii) implies thatf- V " f ( x )is a finite measure also. As usual, we denote it by V"(x, Hence iff 2 0 is in B*, V " f = V"(.,f) exists and is in 8: . Also i f f € B and a > 0, j 2 0, then (4.2) implies a).
(4.4)
I]v+Bv"f= V"f- V"+Bf+ ( a - fi)V+BYolf +
V"f(in norm) as I] -,00.
(4.5) DEFINITION. A function f E 8: is called a - V supermedian if fiV"+pS 5.f for all fi > 0, and is called u - V excessive if, in addition, fiVa+pf+f pointwise as j+ co.
Obviously the minimum of two a - V supermedian functions is again
116
111. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
- V supermedian. Also pV"+BI 5 /?U"+Bl5 /I/( +.p), implies that nonnegative constants are a - V supermedian for any a.
a
(4.6) PROPOSITION : (i) Iff E S*,then V y i s a - V excessive. (ii) Iff is a - V supermedian, then p + pV"+pf is increasing and f = lima,, pVu'pf is the largest a- V excessive function dominated by f . We callfthe a - V excessive regularization off, and we have VBf = Vpffor any p > 0. Proof. For bounded f the first statement is an immediate consequence of (4.4). This statement for unbounded f then follows from (4.6ii) if we approximatefbyf, =f A n. Thus we need only prove (4.6ii). If p > q, then p ~ l + S- ?y'+q = pv"+q + p(q - p ) I / " + B j / " + s - qV"+w = ( p - q)V"+V[I
- PV"+@].
It is immediate from this identity that iff is a bounded a - V supermedian function, then p -, V U f af ( x ) is increasing. For general a - V supermedian fletf, = f A n ; thenf, t f and eachf, is a - V supermedian. Hence for each n if p > q, then jVfl+y, 2 q V q + y , and letting n -+ 00 we obtain the first conclusion of (4.6ii). Thus f ( x ) = limfl+,, p V " ' B f ( x ) exists when f is a - V supermedian. Clearly f~ a*, and f If.Suppose g is a - V excessive and g s f; then pV"+flg jV"+pfand letting p-+ 00 we find that g sf.Thus to complete the proof of Proposition 4.6 we must show that f is a - V excessive. Suppose first that f is bounded; then from the monotone convergence theorem and (4.4) we have
Pf=~
f Iim l [q+m
1VY,
qV+"j
= lim q ~ f l V " + ' J f = rl+m
for any > 0. Consequently BV"+Pf= pV"+@ft j as j?-,00, and so f is a - V excessive. I f f is unbounded let f, =f A n again; then pV"+pf,, is increasing in both p and n. Therefore f = limb limnpVa+pf, = lim, 1,. But it is easy to see from what has already been proved that the limit of an increasing sequence of a - V excessive functions is a - V excessive, and so f is a - V excessive. We already know that Vpf,, = Vsf,for each n and p > 0, and hence letting n --f 00 we obtain the last conclusion of (4.6). (4.7) PROPOSITION. I f f E B+, then U . f - Vo'f is a - U supermedian for any a > 0.
4. Proof.
Iff
RESOLVENTS AND STRONG MULTIPLICATIVE FUNCTIONALS E
117
B+, then (U'f - V " f )E B+ for any a > 0. Moreover
uy- vy- pup+yuy- vy>2 uy- vy- pu@+"u"f+ flP'"vy = U"++
va+Sf20,
and so Proposition 4.7 is proved. Recall from (2.3) of Chapter 11 that f is a-excessive (for X) if and only iff is u - U excessive in the sense of Definition 4.5. (4.8) DEFINITION. A resolvent { V"} is exactly subordinate to { U"} provided it is subordinate and, in addition, U y - V y i s a-excessive (relative to X) for allfe B+ and u > 0. Example. Let B be a nearly Bore1 subset of E and T = T B . If V " f ( x )= Exj: e - " ' f ( X t )dt for f~ B, then { V " } is a resolvent subordinate to { U"}; in fact it is just the resolvent corresponding to the right continuous MF, MI = IIO,T)(t).I f f € B+, then one easily computes that U y - V " f = Pi Uy, and so { V"} is exactly subordinate to { U " } according to Proposition 2.8 of Chapter 11.
(4.9) THEOREM. Let { V"} be aresolvent subordinate to { U"}.Then W " f ( x )= PUPV"f ( x ) exists for all f ' B,~ u > 0, and x E E and the family { W"}is a resolvent exactly subordinate to { U " } . Moreover for each u > 0, V"(X, I W a ( x , for all x, with equality precisely for those x for which /3UpVyl(x) + V y I(x) as /3 + 00 for some y > 0. In particular equality holds at any x for which /3Va I(x) + 1 as /3 + co. a)
a)
Proof. We first prove the existence of the limit in question. From the resolvent equation u a = U t + a + fluB+aum UP = UP+" + UUP+"Ufl,
and so iff E B+ we have Uo'f - pusv"f=V+"f+ fluB+"U"f - flUS'"V"f- / 3 u ~ P + + ' ~ P v " f =
/3uP+yuy- vy] + O(l/P)
since IIUyll I y - ' . Because of this equality, Propositions 4.6 and 4.7 imply that W y = JimP,, pUPV"fexistsfor f E B+ and that U y - W y i s the u - U excessive regularization of U y - V y Consequently W y = FUPVy
118
111. MULTIPLICATIVE FUNCTlONALS AND SUBPROCESSES
exists for allfE B. Clearly V p t f ~Wptfr Uptffor f E B,, and hence, for each a > 0, W" is a bounded positive linear operator on B which is given by measures Wa(x,.) satisfying Va(x,.) I W a ( x , I Ua(x,.) for all x in E. Let us next show that the family { W " } is a resolvent. If ~ E B ,it follows from the above remarks and Proposition 4.6 that, for any p > 0, U f l ( U . f - Vptf) = U f l ( U . f - Waf), and hence that U p V " f = U pW y for any f E B,. Therefore U p V " = U BW". Furthermore for f E B, one has a)
0 = Ufl[W" -
V"lf'2
V q W " - V " l f 2 0,
and consequently V pW" = V pV". Now V" - Vp = (B - a)VpV" = (p - a)VpW", and operating on this relation by q U q and letting q + co one obtains W" - W @= ( p - a)WflWa. Thus { W " } is a resolvent and since U Y - W y i s a-excessive (it is the a - U excessive regularization of Uaf - V"f and hence is a-excessive) for any f~ B,, it follows that { W " } is a resolvent exactly subordinate to { U'}. We have already seen that V a ( x ,.) IW a ( x ,.) for all x and a. Suppose /?UflVyl(y) + V yl(y) as p + co for some fixed y > 0 and y E E. By definition /?UflVYl + Wyl pointwise as p + co and hence V y ( y ,.) = W y ( y , Therefore as /?+ co, / ? U f l V Y f ( y ) +W y f ( y )= V y f ( y )for all ~ E B Now . given any a > 0, Val - Vyl = (y - a)VyV"l and operating on this by flupand letting /? -+ 00 we obtain W" l(y) = V y l(y) (y - cr)VyVal(y) = V" l(y). In other words if /?UflVyl(y) + V y I(y) for some y > 0, then this convergence holds for all y > 0. We have now proved all except the last sentence of Theorem 4.9. If /?VBl(y) + 1 as p-+ 03, then PUP l(y) - p V B l(y) -0. Let a > 0. Then aV"1 I1 and so ( P U P- b V p ) V al(y) I (l/a)(pUp - P V p ) l(y) -0. But /?VflV"l+ Val and hence p U p V " l(y) + V" l(y). Therefore W a ( y ,.) = V a ( y ,.) and the proof of Theorem 4.9 is now complete. 9).
+
(4.10) COROLLARY.Let M be a right continuous M F of X and let {Q,} and {V'} denote the corresponding semigroup and resolvent. Let { W " } be the exactly subordinate resolvent corresponding to { V " }; then V " ( X , .) = Wa(x,.) for all x E E M .
Proof. If X E EM then Q, l(x) = E X { M , ;t BVfl l(x) + 1 as p + 00, proving (4.10).
-= [} + 1
as t + 0 , and so
We turn now to the main topic of this section-the characterization of strong multiplicative functionals, which were defined in (3.11). We remind the reader that for any MF, M, we have M , = 0 by convention.
4.
RESOLVENTS AND STRONG MULTIPLICATIVE FUNCTIONALS
119
(4.11) DEFINITION. A right continuous MF, M = { M , ] , is called regular (RMF) provided P X [ X T € E- E M ; M T > O ]= O for all x and all {A,} stopping times T. Perhaps we should point out explicitly that the properties of being a regular MF or a strong MF depend only on the equivalence class (in the set of all right continuous MF's) to which a MF belongs. In other words if M is a RMF (SMF) and N is a right continuous M F equivalent to M , then N is a RMF (SMF). We will eventually show that these two notions are equivalent; that is, M is a RMF if and only if it is a SMF. We begin with the implication in one direction. (4.12) THEOREM. Any RMF is a SMF. Proof. Let M be a RMF and let { Q , } and { Va} be the corresponding semigroup and resolvent. Let { Wa} be the exactly subordinate resolvent associated with { Va} in Theorem 4.9. It follows from (4.10) that for each f E C , and a > 0 the function ga = Was= U y - ( U ' f - Waf) is bounded, nearly Bore1 measurable, finely continuous, and agrees with V"f on E M . Given an {.MI} stopping time T recall that T'")= (k 1)2-" if k2-" 5 T < ( k + 1)2-" and T'")= co if T = co, k = 0, 1 , . , is an { A , }stopping time and T(")1T. Consider for a > 0 andfE C,
..
/ e-"' f ( x +f
+
m
qa(x) = Ex
T)
Mf+ T dt
0
m
= lim
EX
Jo e-a'f[X(t
n
+T
~~ I(
+t
~ ( n ) dt )
cEx(/owe-a~~(xf+,,-n) dt; lim c jOme-a'/(X,) M , dt]hfk,-n;
= lim n
= n
= Iim
Mt+kz-l
k
k ~ x(~ x(, z-n)[
~ ( n= ) k2-"
1
~ ( n= ) k2-")
E " { v " ~ [ X ( T ' " ' ) ] M(T'"')}.
n
Since M is regular this last expression becomes lim EX(ya[X(T'"')] M(T'"')}= Ex{ga(X(T)) MT} 1,
and so
= E x { V a f ( X T )M T } ,
120
111. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
q"(x) = E"[I/"f(XT)
hfT]
= E'P("[
/ome-a'f(XI) hf, d t ] M T ) .
Consequently the functions q(t) = E " { f ( X , + , ) M I + T } and @ ( t ) = E x { E X ( T ) L f (M,]kfT} ~,) have the same Laplace transforms, namely qa(x), and since cp and @ are both right continuous the uniqueness theorem for Laplace transforms implies that they must be identical. But this is just the statement that (3.1 1) holds for any f E C+ and consequently for any f E B. Thus Theorem 4.12 is proved.
(4.13) DEFINITION. A right continuous MF is said to be exact (EMF) provided the corresponding resolvent { V'} is exactly subordinate to { U"}. The following proposition is a consequence of the proof of Theorem 4.12.
(4.14) PROPOSITION. Any EMF is a SMF. Let M be a SMF. Then for all {A,}stopping times (4.15) PROPOSITION, T and Y E b 9 one has E"{(Y 0 e T ) h f f + T ; A} = Ex{EX(T)(YM,)M T ; A} for all A E A T ,t, and x. Proof. As usual it suffices to consider the case A = 0 and Y of the form f i ( X , , ) with t, . .. t, and fjE 68: ,j = 1, .. ., n. Of course, Proposition 4.15 must essentially express the fact that the canonical subprocess 9 corresponding to M is a strong Markov process, and we will deduce (4.15) from this fact. Suppose first of all that we have proved (4.15) for Y's of the above form with 1, It. If t, < ... < t j I t < z ~ <+ ~. < t , , then the strong Markov property for X yields
-= -=
..
which, according to the supposition we have just made, becomes
Thus the general case is reduced to the case t , I2.
4. RESOLVENTS AND STRONG MULTIPLICATIVE
FUNCTIONALS
121
To handle the case t , 5 t we use the fact that 2,the canonical subprocess corresponding to M, is a strong Markov process. The notation is that of stopping time, and so Section 3. If $= T A y, then p i s an {A,}
and hence Proposition 4.15 is proved. (4.16) COROLLARY.Let M be a SMF, T an { A , }stopping time, Y E b 9 , and R E 9, R 2 0. Then
Ex{( Y BT)M(T + R 0 O T ) ; A} = E x { E X ( T )Y( M R ) M T A} ; 0
for all A E ATand x. Proof. We leave it to the reader to check that MR E 9 and M T + R o BETA under the stated conditions. Let R(") have its usual meaning; then {R(') = k2-"} E 9 a n d {R'")0 e T = k2-"} = 0; '{R'")= k 2 - " } . Recalling that M , = 0 by assumption, the left side of the desired equality equals
lim E"{(Y 0 BT)M(T + R(")0 OT); A} n
= lim n
=lim n
C Ex{( Y
0
eT)kfT+kZ-n;
R(")0 e T = k2-"; A}
k
C E x { E X ( T ) [ Y M k yR'"' n ; = k 2 - " ] h f ~A} ; k
= Ex{EX'T'[YhfR]MT; A},
completing the proof of Corollary (4.16). (4.17) THEOREM.Let M be a SMF, T a n (9,) stopping time, and R E A, R 2 0. Then (4.18)
almost surely.
M T ( ~ ) + R ( ~ )=( oM) T ( ~ ) ( UM)R ( ~ ) ( ~ T W )
122
Ill. MULTIPLICATIVE FUNCTIONALS AND SUBPROCESSES
Proof. In general we will omit the o ' s when writing expressions such as (4.18). For example (4.18) will be written M T + R = M , MR(&). The reader should distinguish carefully between MR(OT)= MR(u)(OTo)and M R o 0 , = MR(~To)(eT~ First ). note that (4.18) holds on { R = a} since M , = 0 by convention. Therefore it suffices to consider the case R(o)= t since we can approximate R from above in the usual manner by countably valued random = M,(M, o 0,) almost surely for variables. Thus we must show that each t . The fact that T is an {F,} stopping time implies that M , + T , M,, and M , 0 OT are all in 9. Let X T = a{X,+,; s 2 O } - and Y = a ( X Tu FT)-. We will show that (4.19)
E"{M'r+i;A> = E"{MT(MI 0 ~ )A; }
for all A E Y, 1, and x. It suffices to consider A = D n { X T + , ,E A , } n . ., n {X,+,"EA,,} where D € F T and A i e 8 * , 1 I j I n. But for such a A Proposition 4.15 implies that the left side of (4.19) reduces to while the strong Markov property for X implies that the right side of (4.19) reduces to the same thing. Thus in order to complete the proof it suffices to show that Y = 9. Since this is of some independent interest we formulate it as a proposition. (4.20)
PROPOSITION.Let Y be as above. Then Y
= 9.
Proof. Iff E C and a > 0, then
and so ;j e - " f ( X , ) dt E bY. It now follows from the Stone-Weierstrass approximation theorem that ;j g ( t ) e - ' f ( X , ) dr E bY for any bounded continuous function g on [0, 00). Since t - t f ( X f ) is right continuous almost surely, for each u 2 0 we can find a sequence {g,,}in C([O, 00)) such that gn(t)e - ' f ( X , ) dt + f ( X U ) almost surely. Consequently 9 't Y c 9, and since go= 9 we obtain Proposition 4.20. We next complete the characterization of strong multiplicative functionals. (4.21)
PROPOSITION. M is a SMF if and only if M is a RMF.
Proof. In view of Theorem 4.12 we need only show that a SMF is regular. stopLet M be a SMF and define R = inf{t: M I= 0). Clearly R is an (9,)
4.
RESOLVENTS AND STRONG MULTIPLICATIVE FUNCTIONALS
123
ping time and the right continuity of M implies that M R = 0 almost surely (M, = 0). Let T be any { A , }stopping time; then H = T R 0, i s also a n { A , }stopping time according t o (8.7) of Chapter I. Using Corollary 4.16 we have for each x
+
0
E"{M,; T < (} = E"{EX'T'[hfR] M , ; T < (} = 0.
Consequently T + R 0 OT = H 2 R almost surely on {T < (}, and so P"[X,
EE
- E M ; M, > 01 = P x [ X , E E - E M ; T < R] I P"[X,
EE
- EM ;R 0, > 01 0
= E"{PX'T'(R> 0); X T E E
- EM)
= 0,
since PY(R> 0) = 0 for all y E E - E M . (4.22)
PROPOSITION. Suppose that M is a right continuous M F and that
EM is nearly Bore1 measurable. If T = T E V E M then , M is regular if and only if M ,
=0
almost surely.
Proof. Let x be fixed. Then there exists a n increasing sequence {K,,} of compact subsets of E - EM such that T,, = T K , l T almost surely P". Since X(T,,) E K,, c E - EM almost surely on {T,,< co} one has M(T,,) = 0 almost surely on IT,, < co} if M is regular. But M(T,,) + M ( T ) almost surely P" and hence M ( T ) = 0 almost surely P" on {T < co}. Since x is arbitrary and M, = 0 we obtain M , = 0 almost surely. Conversely suppose M , = 0 almost surely. If R is any { A , }stopping time and X , E E - E M ,then either R 2 Tor R = 0. If R 2 T, then M , = 0 almost surely. On the other hand
P"(X, E E - E M ,
MR
> 0; R = O ) I P"(X0 E E
-E M ,
M o > 0)
= 0,
from the definition of EM and the fact that X is normal. Consequently M is regular. REMARK.It is sometimes convenient t o normalize a right continuous MF, M = {MI},as follows: Define N , ( o ) = 1 for all t if Xo(w) = A, N , ( o ) = M , ( u