MARKOV CHAINS
North-Holland Mathematical Library Board of Advisory Editors:
M. Artin, H. Bass, J. Eells, W. Feit, P. J. Freyd, F. W. Gehring, H. Halberstam, L. V. Hormander, M. Kac, J. H. B. Kemperman, H. A. Lauwerier, W. A. J. Luxemburg, F. P. Peterson, I. M. Singer and A. C. Zaanen
VOLUME 11
NORTH-HOLLAND AMSTERDAM. NEW YORK . OXFORD
Markov Chains D. REVUZ Universitt! de Paris VII Paris, France
Revised Edition
1984
NORTH-HOLLAND AMSTERDAM. NEW YORK . OXFORD
0 Elsevier Science Publishers B.V., 1984
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, machanical, photocopying, recording or otherwise, without the prior permission of the copyright owner.
ISBN: 0444 86400 8
First edition: 1975 Second (revised) edition: 1984 Published by: Elsevier Science Publishers B.V P.O. Box 1991 10oO B Z Amsterdam The Netherlands Sole distributors for the U.S.A. and Canada:
Elsevier Science Publishing Company Inc. 52, Vanderbilt Avenue New York, NY 10017 U.S.A.
Library of Congress Cataloging in Publication Data Revuz, D. Markov chains. (North-Holland mathematical library; v. 11) Includes bibliographical references and index 1 . Markov processes. 1. Title. 11. Series. QA274.7.P.45 1983 519.2'33 83-11586 ISBN 0-444-864M)-8
Transferred to digital printing 2005
PREFACE TO THE FIRST EDITION
This book grew out of a lecture course given at the “3”e cycle” levk. o the universities of Paris during recent years. The first part is thus intended for students at that level; the later chapters aredesignedformoreadvancedreaders. Our purpose is to provide an introduction to homogeneous Markov chains with general (measurable) state space, or perhaps more accurately to the study of “sub-Markovian kernels” or “transition probabilities”. This contains as special cases the study of pointwise transformations and ergodic theory, potential theory, random walks on groups and homogeneous spaces, etc. It would be preposterous to attempt a comprehensive treatment of all these topics within this book, in as much as some of them are far from being complete and offer still challenging problems. Roughly speaking, the first four chapters - at least parts of them - are foundations that,must be learnt by every beginner in the field. It will be no surprise that they include a short chapter on potential theory, since its fruitful relationship with Markov phenomena has been long and widely known. They include also the basic classification of chains according to their asymptotic behaviour and the celebrated Chacon-Ornstein Theorem. After these first chapters we proceed to some portions of the theory such as renewal theory for abelian groups, an introduction to Martin Boundary Theory and the study - the most thorough in this book - of chains recurrent in the Harris sense. The last chapter deals with the construction of chains starting from a kernel satisfying some maximum principle. These topics do not depend very strictly on one another and the reader may skip some according to his own interest. An interdependence table is given below, but this is merely a guide, and some idea of all the previous results will always come in useful. Other important subjects such as invariant measures, pointwise transformations or random walks on non-abelian groups are barely alluded to. Relevant references will be given in the “Notes and Comments” at the end of the book. V
vi
PREFACE
We tried to present at the end of each section a large selection of exercises. We feel that the reader will not benefit greatly from reading the book if he does not tackle a large number of them. They are in particular designed to provide examples and counter-examples; some of them are results which might have been included in the text. With a few exceptions they can be worked out with a knowledge of the previous sections and of classical calculus. References and credits do not appear in the text but are collected under the “Notes and Comments”. We have in no way tried to draw a historical picture of the field, and apologize in advance to anyone who may feel slighted. We merely indicate the papers we have actually used; as a result insufficient tribute is paid to those who have founded the theory of chains, such as A. A. Markov, N. Kolmogoroff, W. Doeblin, J. L. Doob and K. L. Chung. The book is divided into chapters; each chapter is divided into sections and starts with a fresh section 1. In every section, items such as theorems, examples, exercises, paragraphs, remarks, are numbered. The references are given in the following way: the nthchapter is quoted as ch. n ; the Pthsection of ch. n is quoted as ch. n, g f i . Finally, the qtti numbered item in ch. n, $fi is quoted as ch. n, Theorem P.q, etc.; however, for quotations within the same chapter we shall drop “ch. n” and write simply Theorem fi.q, etc. I t is a pleasure to acknowledge a few of my debts. Like many French probabilists, I owe everything to J. Neveu and P. A. Meyer; but for their teaching and advice I would never have been really introduced to Markov theory. I must especially thank M. Duflo, who first aroused my interest in discrete time, and A. Brunel for the work we have done together, part of which finds its way into the present book. Many students and colleagues were kind enough to comment on preliminary drafts of the first chapters, in particular M. Brancovan, P. Jaffard and P. Priouret, and I wish to thank them all. My warm thanks go to M. Sharpe, who read through the entire manuscript and removed the most important inaccuracies in the use of English language (a very difficult language indeed) ; he is not responsible for any awkward points remaining. We hope they will not hamper understanding and that the native speaker will make allowances for the “exotic flavour”. Finally I must thank Martine Mirey, who typed the entire manuscript, and E. Fredriksson of the staff of North-Holland Publishing company for his understanding help. D. REVUZ
PREFACE TO THE SECOND EDITION
This second edition differs from the first by the rearrangement of some of the material already covered and by the addition of a few new topics. Among the latter are the “zero-two’’ laws which were one of the most glaring omissions of the first edition and the introduction of a new section on subadditive ergodic theory which may seem out of place in a book on Markov chains but which we feel warranted by its manifold applications to Markov chains themselves. We also tried to work more consistently with general state spaces and avoid to resort to countable spaces. Finally the number of exercises has been increased and some of them have been rewritten 50 as to provide more hints. Since the first edition, some of the subjects touched upon in this book have known an important development such as ergodic theory or random walks on groups and homogeneous spaces. Attempting to include some of these advances would have increased forbiddingly the size of the book and in any case, these subjects do or will deserve books in their own right. Another omission is the theory of large deviations for Markov chains. We have increased the number of references so as to include some possible readings on these topics; but the remarks of the preface to the first edition as to references and credits are still in force. I finally must say how grateful I am to all the colleagues, students and reviewers who have offered constructive comments on the first edition, especially to R. Durrett and R. T. Smythe. If this second edition is any better than the first, it will be, for a great part, due to them all.
D. REVUZ
vii
INTERDEPENDENCE GUIDE
1.1-+ 1.2 -+ 1.3 -+ 1.4
1 2.1
1 2.2
1 2.3
/l
2.5
c 2.4
l7’1.J/
-+ 3.1 + 3.2 4 3.3 -+
2.6
6.1 -+ 6.2 1
1
5-
4.2
6.3
3.4
1
10.1
5.1
1 5.2
1 8.3 4 9.1
1
7.3
...
Vlll
1
5.3
1
CONTENTS
PREFACE TO FIRST EDITION . . . . . . . . . . . . . . . . v PREFACE TO SECOND EDITION . . . . . . . . . . . . . . . vii INTERDEPENDENCE GUIDE . . . . . . . . . . . . . . . . . viii CONTENTS
. . . . . . . . . . . . . . . . . . . . . . . . . .
CHAPTER 0. PRELIMINARIES 1.Notation. . . . . . . . . . . . . . . 2. Martingales . . . . . . . . . . . . . 3. The monotone class theorem . . . . . 4. Topological spaces and groups . . . .
ix 1
. . . .
. . . .
. . . .
. . . .
. . . . . . . . 1 . . . . . . . . 3 . . . . . . . . . 4 . . . . . . . . . 6
CHAPTER 1. TRANSITION PROBABILITIES. MARKOV CHAINS 1. Kernels. Transition probabilities . . . . . . . . . . . . . . . . 2 . Homogeneous Markov chains. . . . . . . . . . . . . . . . . . 3. Stopping times. Strong Markov property. . . . . . . . . . . . . 4. Random walks on groups and homogeneous spaces . . . . . . . . 5. Analytical properties of integral kernels . . . . . . . . . . . . .
8 8 13 23 28 33
CHAPTER 2. POTENTIAL THEORY 40 1. Superharmonic functions and the maximum principle . . . . . . . 40 2. Reduced functions and balayage . . . . . . . . . . . . . . . . 48 3. Equilibrium, invariant events and transient sets . . . . . . . . . 55 4. Invariant and excessive measures . . . . . . . . . . . . . . . . 60 5. Randomized stopping times and filling scheme . . . . . . . . . . 67 6. Resolvents . . . . . . . . . . . . . . . . . . . . . . . . . . 74 CHAPTER 3. TRANSIENCE AND RECURRENCE 1. Discrete Markov chains . . . . . . . . . . . . . . 2. Irreducible chains and Harris chains . . . . . . . ix
. . . . . .
80
80
. . . . . . . 87
X
CONTENTS
3. Topological recurrence of random walks . . . . . . . . . . . . 4.Recurrence criteria for random walks and applications. . . . . . CHAPTER 4. POINTWISE ERGODIC THEORY 1. Preliminaries . . . . . . . . . . . . . . . . . . . . . 2. Maximal ergodic lemma. Hopf’s decomposition . . . . . . 3. The Chacon-Ornstein theorem for conservative contractions 4. Applications to Harris chains. . . . , . . . . . . . . . 5. Brunel’s lemma and the general Chacon-Ornstein theorem . 6. Subadditive ergodic theory . . , . . . . . . . . . . .
. . . . .
.
. .
98 106
117 . . . 117 . . . 122 . . . 131 . . . 138 . . . 145 . . . 152
CHAPTER5. TRANSIENT RANDOM WALKS. RENEWALTHEORY 1. The theorem of Choquet and Deny . . . . . . . . . . . . . . . 2. General lemmas . . , . . . . . . . . . . . . . . . . . . . . 3. The renewal theorem for the groups R and X . . . . . . . . . . . 4.The renewal theorem . . , . . . . . . . . . . , . . . . . . . 5. Refinements and applications . . . . . . . . . . . . . . . . .
159 159 164 169 177 180
CHAPTER 6. ERGODIC THEORY O F HARRIS CHAINS 186 1 . The zero-two laws . . . . . . . . . . . . . . . . . . . . . . 186 2. Cyclic classes and limit theorems for Harris chains . . . . . . . . 194 3. Quasi-compact transition probabilities and strong ergodic theorem . 20 1 4. Special functions . . . . . . . . . . . . . . . . . . . . . . . 21 1 5. Potential kernels . . . . . . . . . . . . . . . . . . . . . . * 216 6. The ratio-limit theorem . . . . . . . . . . . . . . . . . . . * 226 CHAPTER 7. MARTIN BOUNDARY 1. Regular functions. . . . . . . . . . . . . . . . . . . . . . . 2. Convergence to the boundary. . . . . . . . . . . . . . . . . . 3. Integral representation of harmonic functions . . . . . . . . . .
233 233 241 251
CHAPTER 8. POTENTIAL THEORY FOR HARRIS CHAINS 261 1. Harris chains and duality . . . . . . . . . . . . . . . . . . . 261 2. Equilibrium, balayage and maximum principles. . . . . . . . . . 266 3. Normal chains . . . . . . . . . . . . . . , . . . . . . . . . 271 4. Feller chains and recurrent boundary theory . . . . . . . . . . . 277 CHAPTER 9. RECURRENT RANDOM WALKS 1. Preliminaries . . . . . . . . . . . . . . . .
286
. . . . . . . . . 286
xi
CONTENTS
2. Normality and potential kernels 3. Martin boundary . . . . . . . 4. Renewal theory. . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
298 308 3 15
CHAPTER 10. CONSTRUCTION O F MARKOV CHAINS AND RESOLVENTS 1. Preliminaries and bounded kernels . . . . . . . . . . . . . . . 2. The reinforced principle. Construction of transient Markov chains. . 3. The semi-complete maximum principle . . . . . . . . . . . . .
324 324 330 338
NOTES AND COMMENTS . . . . . . . . . . . . . . . . . .
346
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . INDEX OF NOTATION . . . . . . . . . . . . . . . . . . INDEX O F TERMS . . . . . . . . . . . . . . . . . . . . .
*
*
.
* *
356 371 372
This Page Intentionally Left Blank
CHAPTER 0
PRELIMINARIES
This chapter is devoted t o a list of some basic notations and facts which will be used throughout the sequel without further explanation. We assume that the reader is familiar with measure theory and probability theory as may be found for instance in the first four chapters of the book by Neveu [ Z ] , with point set topology, and that he has some knowledge of topological groups and Fourier transforms, for which we refer to the books by Rudin [I] and by Hewitt and Ross [l]. 1. Notation
1.1. Let N denote the set of positive integers, Z the set of integers and R the set of real numbers ;R will denote the extended real numbers { - a}U R U{
-
+ a}
with the usual topology. For a and b in R, we write a v b = max(a, b),
a
A
b = min(a, b ) .
If E is a set and f a numerical function on E , we shall write
llfll
= suplf(x)I, xsE
ff
= f v 0,
f-
=
- (f
A
0).
1.2. If ( E , B) and ( F , 9) are measurable spaces, we shall write f E B / F t o indicate that the function f : E -+ F is measurable with respect to B and 9. In the case where F is R and 9 the a-algebra of Bore1 subsets we write simply f E 8.The symbol B thus denotes the a-algebra as well as the set of real-valued measurable functions on E . If A is a subset of E , we denote the indicator function of A b y I , , so that the statements I , E B and A E 8 have the same meaning. When A = E we shall often write 7 for 7,.
1.3. If 2 is any set of numerical functions, then &+' denotes the set of nonnegative functions in 2 and b 2 the set of bounded functions. For instance 1
PRELIMINARIES
2
CH. 0, $1
if ( E , 8)is a measurable space, b b is the set of bounded measurable functions and bb+ the set of non-negative bounded measurable functions. We recall that bB is a Banach space for the supremum norm 11 11 described above. We shall denote by 42 the unit ball of bb. The set 42+ is thus the set of measurable numerical functions on ( E , 8) such that for every x E E , 0 f ( 4 1.
<
<
1.4. Let Sa be a set and {(Ei,bi)}cla collection of measurable sets. For each
i E I let f , be a map from L? to E,; the a-algebra on L? generated by the sets {/;'(Ai) : A i E Ei}is the smallest a-algebra on Sa relative to which all the f i are measdrable, and it will be denoted by a(fi,i E I ) . If 9 is any collection of subsets of E, we denote by a ( 9 ) the a-algebra generated by 9. A measurable space ( E ,8)will be said to be separable if d is generated by a countable collection of sets.
1.6. We denote byL/kl(b) the space of a-finite measures on ( E ,8)and by bL/kl(B) the space of finite measures. We shall call ,u+ and p- the positive and negative parts of the measure ,u and set 1,uI = ,u+ ,u-. The space bL/kl(S) is a Banach space when endowed with the norm llpll = l,ul ( E ) . If f E d and the integral of f with respect to the measure p exists, we shall write it in any of the forms
+
The space of integrable functions will be denoted by Y1(p),and as usual L1(p)will be the Banach space of equivalence classes of integrable functions endowed with the norm I l f l ) l = j E I f 1 dp. There is a canonical isometry of L1(p) into b d ( b ) which maps. f onto the measure 1 : A .--r fA f dp; 1 is absolutely continuous with respect to p ( A << p).
1.6. If $2, F,P ) is a probability space we shall, as is customary, use the words random variable 'and expectation in place of measurable function and integral, and we shall write
i n X d P = E[X]. If 9 is a sub-a-algebra of 8,we denote E [ X 131 the conditional expectation of X (assumed to be quasi-integrable) given 9. When X = l,, A E 8, we write P ( A 19)instead of E [ l A 3'1, and if 9 is the smallest a-algebra
1
CH. 0, 92
MARTINGALES
3
I
relative to which the random variable Y is measurable, we may write E [ X Y ] and P ( A Y ) .When we apply conditional expectation successively, we shall often abbreviate E [ E [ X S,] g2]by E [ X PI S,].We recall that conditional expectations are defined up to P-equivalence; in the sequel we shall often omit the qualifying phrase “almost everywhere relative to P” in the relations involving conditional expectations.
1
I I
I
I
2. Martingales Let (JZ, 9, P ) be a probability space and {9,} an ,increasing >,, sequence of sub-@-algebrasof 9. A sequence {X,},,, of random variables over (Q, 9, P) is said to be a supermartingale with’respect to (9, if ) (i) X , E 9, for every n 3 0, (ii) E[X;] < m for every n 3 0, (iii) E [ X , 9,] X , whenever m n. It is a submartingale if {- X,} is a supermartingale and a martingale provided it is both a supermartingale and a submartingale. The proof of the following result, which we shall use freely in the sequel, may be found in the books by Doob [Z], Loeve [l], Meyer [Z] or Neveu 121.
+ <
1
<
Theorem 2.1. Let {X,},,,, be a supermartingale such that then limn X , exists almost surely.
SUP,
E[XJ
< m;
As a result the positive supermartingales converge almost surely. We shall also need the following.
Proposition 2.2. Let 9% be an increasing sequence of sub-a-algebras of S such that 9, generates 9. Let X , be a sequence of random variables converging almost surely to a random variable X , and Y an integrable random variable such that IX,I Y for every n . Then the sequence E [ X , F,] converges almost surely to E [ X 91.
u,
1
< I
Proof. Pick
E
> 0 and set U
=
inf X,, n>m
V = sup X,, n>m
where vn is chosen such that E[V - U ] < E. Then for n
3 m we have
4
PRELIMINARIES
CH. 0, $3
and applying Theorem 2.1 yields
E[U
1 8 1 < lirn E[X,1 Fn] ,< IlZ E[X,I F,] ,< E[V 1 9 1 n
as well as
?I
1
<
<
E[U F] E[X IF] E[V
It follows that
I
I
IF].
<
E [ K E[X, F,] - lirn E[X, Fn]] E ,
which implies that E [ X , equals E [ X IF].
I F,]converges almost surely and that the limit
In the sequel we write P - a s . for “almost surely relative to P” and more simply a s . when there is no risk of mistake.
3. The monotone class theorem
Definition 3.1. Let 52 be a set; a collection 9 . o f subsets of SZ will be called a monotone class if: (i) 52 E Y ; (ii) if A , B E 9, A t B, then B - A E 9; (iii) if { A , } is an increasing sequence of elements of 9, then u A , E 9. Plainly a a-algebra is a monotone class. I t is also clear that the intersection of an arbitrary family of monotone classes is a monotone class; hence for any collection F of subsets of 52 there is a smallest monotone class containing 8,which we call Y ( 8 ) Clearly . Y ( F )c a(F).
Theorem 3.2. If F i s closed under finite intersections, then 9(F) = ~(9). Proof. Clearly, it suffices to show that Y(F) is a a-algebra, and for this it suffices to show that it is closed under finite intersections. To this end we define Y1 = {BE 9’(9 B) n: AE
Y(F)for all A E F } .
I t is easily checked that 9, is a monotone class, and since F is closed under finite intersections, Y13 9, and hence 9, 3 Y ( 9 ) .Since by definition 9, c Y ( F ) ,we get 9, = 9’(F). Now define
9,= { B E 9(F) B: nA
E
9(F) for all A
E
Y(F)}.
CH. 0, 93
THE MONOTONE CLASS T H E O R E M
G
Again 9,is a monotone class, and by the preceding argument Y , > S ; hence Y 22 9'(9 and ), as before we get 9, = Y ( 9 ) But . this is precisely the statement that Y(9) is closed under finite intersections. The proof is thus complete. One generally has to deal with functions rather than sets and then may use the following version of Theorem 3.2.
Theorem 3.3. Let 9 be a collection of subsets of rR closed under finite intersections and 2' a vector space of real valued functions on SZ such that (i) 1 , ~ S a n d l A ~ S f o r a l l A ~ 9 , (ii) if h, i s a n increasing sequence of elements of 2'+such that h = sup h, i s finite (bounded), then h E 2'. Then 2 contains all real-valued (bounded) functions on SZ that are a(*)measurable. Proof. Left to the reader as an exercise. [Hint: Set 9 = { A: 7, prove that 9 = Y ( 9 )= a ( P ) . ]
E2f}
and
The above theorems will be used in the following setup: let 52 be a set and ( E f ,&'J a family of measurable spaces indexed by a set I . For each i E I , let Si be a class of subsets of Ei closed under finite intersections and such that gi = a ( 9 J . Finally let f i be a map from SZ to E i .
ni,
Proposition 3.4. The collection 9 of all sets of the f o r m f r l ( A i ) ,where A , E 9, for i E J and J ranges over all finite subsets of I , i s closed under finite intersections and Y(S) = o(fi,i E I ) . Proposition 3.6. Let 2 be a vector space of real-valued functions on LR such that: (i) 1 E 2'; (ii) if h, i s a n increasing sequence of elements of &+ ' such that h = sup h,, i s finite (bounded),then h E 2 ; (iii) S contains all products of the f o r m l A i o f r where J i s a finite subset of I and A i E Si. T h e n 2 f contains all real-valued (bounded) functions in a(fi,i E I ) .
nfd
We shall also use the following corollary of the monotone class theorem.
Proposition 3.6. Let p and Y be two finite measures on ( E , 8)and let S c B be they agree on a ( S ) . closed under finite intersections. T h e n if p and Y agree on 9,
PRELIMINARIES
6
CH. 0. $4
Proof. One shows that 9 = { A E 8:p ( A ) = v ( A ) }is a monotone class. 4. Topological spaces and groups
Except for Banach spaces of functions, the topological spaces we shall have to consider will be locally compact. If E is such a space, then C ( E )c C,(E) c C,(E) denote respectively the bounded real-valued continuous functions on E , the real-valued continuous functions vanishing at infinity, and the real-valued continuous functions with compact support. The first two of these spaces are Banach spaces under the supremum norm 1) * 11. We shall also denote by d the a-algebra of Borel sets of E. Almost always, the spaces E we shall deal with have a countable basis and we shall write for short: E is an LCCB. In that case we recall that: (i) The a-algebra d of Borel sets is generated by any open countable basis for the topology (hence ( E , B) is separable) as well as by the space C,(E). (ii) The space E is metrizable, and one can choose a metric d compatible with the topology such that ( E ,d ) is a separable complete metric space in which every closed and d-bounded set is compact. (iii) The space E possesses a countable dense subset. If E is an LCCB, a positive measure p on ( E , 8)will be called a Radon measure if p(K) < 03 for every compact K . In particular, a Radon measure is a-finite. If L is a non-negative linear functional on C,, there is a unique Radon measure on ( E ,8)such that L ( f )= p ( f )for all f E C,. Every Radon measure is regular, that is for B E d,
p ( B ) = sup{p(K): K c B , K compact} =
inf{p(G): G 3 B , G open}.
The space of Radon measures on E will be endowed with the vague topology, which is the coarsest topology for which all the functions p +p(f),f E CIc, are continuous. We shall need the following compactness criterion. Proposition 4.1. A set {pi,i E I} of measures i s vaguely relativelv compact if and only if for every compact subset K of E the sets of numbers {pu,(K), i E I} is bounded.
A locally compact metrizable group G is a group which is also an LCCB such that the transformation from G x G onto G which sends ( x , y ) into x-ly is continuous. (We denote by x-l the inverse of the element x and by V-l the set of inverses of elements x in V.)
CH. 0. $4
TOPOLOGICAL SPACES AND GROUPS
I
One can choose a metric compatible with the topology of G and invariant by left translations (or right translations) in G . If e is the identity of G , the filter Y of neighbourhoods of e has the following properties: (i) For every U E Y there is a set V E Y such that VV-l C U ; (ii) If U E Y and x is any element of G then there exists a set V E V such that x-'Vx C U . By (i) the class of symmetric (V = V-I) neighbourhoods is a base for V . Finally the filter of neighbourhoods of x E G is equal to x Y . On every LCCB group G there is a left (right) invariant Radon measure, unique up t o a multiplicative constant, called the left (right) Haar measure of G . Let m be the left Haar measure on G ; then for every integrable function f ,
5
f ( 4 d+)
=
1
f ( Y 4 dm(4
for every y in G . The group G is unimodular if the left Haar measure is also a right Haar measure. Abelian and compact groups are unimodular. Finally, the convolution p * v of two measures p and v is defined by (P
* v, f )
=
55
f(xy) p ( W V(dY)P
provided the integral on the right is meaningful for all f E C,. tion of two functions f and g is defined by
The convolu-
5
(f * g) (4 = f ( Y ) d Y - W 4 d Y ) if m is a left Haar measure. We shall use the fact that the convolution of a function in gP"and O a function in 9is, a continuous function.
CHAPTER 1
TRANSITION PROBABILITIES. MARKOV CHAINS
In this chapter we introduce the basic data of our study. Throughout the sequel a measurable space ( E ,8)is assumed given.
1. liernels. Transition probabilities
+
Definition 1.1. A kernel on E is a mapping N from E x 8 into - a, I.. such that: (i) for every x in E , the mapping A ---t N ( x , A ) is a measure on 8 which will often be denoted by N ( x , * ) ; (ii) for every A in &, the mapping x N ( x , A ) is a measura.ble function with respect to 8 which will often be denoted by N ( - ,A ) . --f
+
The kernel N is said to be positive if its range is in [0, a].It is said to be a-fiizite if all the measures N ( x , ) are a-finite; it is said to be proper if E is the union of an increasing sequence of subsets of E such that the functions N ( , E n ) are bounded. The kernel N is said to be b,ounded if its range is bounded, or in other words, if there is a finite number M such that IN(x, A )I M < co for every x in E and A in 8.A bounded kern'el is a proper kernel and a proper kernel is a-finite, the converse statements being obviously wrong. If N is positive, then N is bounded if and only if the function N(* , E ) is bounded.
<
Definition 1.2. In the sequel we shall deal mainly with positive kernels. If f is in b,, it is then easily seen b y approximating f with simple functions that one defines a function in 8, denoted N / or N ( / )by setting
By defining Nf = Nf+ - Nf-, we may extend this t o every function in 8 such that N f + and N f - are not both infinite. We sometimes write N ( x , f ) for N f ( x ) ;in particular N ( x , A ) = Nl,(x). 8
CH. 1, $1
KERNELS. TRANSITION PROBABILITIES
9
In the same way, let m be a positive measure on 6, and define for AEB,
mN(A) =
5.
m(dx) N ( x , A ) = ( m , N ( * , A ) ) ;
it is easily seen that mN is a positive measure on 6,and as above this may be extended to signed measures. The two mappings thus defined are linear and, whenever the two members are meaningful, we have
Notice further that if we call E, the Dirac measure of the point x , we have E,N( * ) = N ( x , * ). The mappings of b, into itself defined by positive kernels are characterized by the following property.
Proposition 1.3. A n additive and homogeneous ma#$ing V of 8, into itself i s associated with a positive kernel if and only if for every increasing sequence {f,} of functions in 6, one has
Proof. Easy and left to the reader as Exercise 1.9. We proceed to a few examples of kernels.
Examples 1.4. (i) Let A be a positive a-finite measure on 8 and n a positive real-valued function defined on E x E and measurable with respect t o the product a-algebra 6 @ 6. One may then define a kernel N on E by setting
Such a kernel is called an integral kernel with basis A. When E = Rd, d 2 3, = ( x - yI- d+2, we thus get the kernel of newtonian potential theory. The kernel N is positive and for f E b,,
A is the Lehesgue measure, and n ( x , y )
10
TRANSITION PROBABILITIES. MARKOV CHAINS
CH. 1, $1
If n(x, y ) = u ( x ) b ( y ) , where u, b are two measurable functions on E , then
i
N f ( 4 = a ( x ) QY) !(Y)
WY)
= (fb,A)
44.
In that case we write N = a @ b l , and simply N = a @ 1if b = 1. A case which has been extensively studied is the case where E is countable and d the discrete a-algebra on E. The measure may be taken equal to the counting measure of E (A({%}) = 1 for every x in E ) and the integrals then reduce to sums, that is Nf(4= n(x. Y) f(Y).
2
YEE
But since in that case N ( x , {y}) = n(x, y ) , we shall mix the two notational devices and write more simply
It thus suffices to give the numbers N ( x ,y) to define on E a kernel, which may then be viewed as an infinite square matrix indexed by the elements of E . The measures (functions) will be “row” (“column”) vectors and the operators of Definition 1.2 are just the usual operators on vectors defined by square matrices. (ii) Let G be a multiplicative locally compact semi-group and p a positive Radon measure on the Borel sets of G ; one defines a positive kernel N on G by setting N ( x , A ) = (,u * 4 (4, where
* denotes the convolution in G . For a Borel function f
and for a positive measure m on G , mN = p * m. Such a kernel is called a convolution kernel. The newtonian kernel of (i) is a convolution kernel. The case where G is a group will of course be of special interest in the sequel. We remark that it is not true that Nf = ,u * f since (whenever it makes sense) (P * f )
(4 =
1 f(P4
P(dY).
G
If we call i; the image of p by the mapping x
+ x-l,
then Nf
=
,& * f .
CH. 1. $1
KERNELS. TRANSITION PROBABILITIES
11
This may be put in the following more general setting. We say that G (the elements of which are henceforth written g, g', h,. . .) operates on the topological space M , or that M is a G-space, if there is a continuous mapping g x x -+gx from G x M to M such that (glgz)x = g,(g,x). With a positive measure u , on G we associate a kernel N on M , by setting, for a Bore1 function f on M ,
if m is a positive measure on M then mN = ,u * m, where the convolution is defined by the formula (P
* m, f )
=
5
GxM
*
f k x ) ,u(dd m(dx).
By letting G operate to the left on itself, we see that the former example is a special case of the latter. (iii) A measurable mapping 6 of E into itself ( 6 ~ B / bis) called a point transformation of E . With such a mapping we may associate a kernel by setting ~ ( xA ,) = 1,(e(x)) = ~ o - I ( A ) ( ~ ) *
Definition 1.5. The com9osition or prodzlct of two positive kernels M and N is defined by
MN(x,A ) =
5.
M ( x , dy) N ( Y ,A ) .
It is easily seen that M N is a kernel and we have
Proposition 1.6. The comfiosition of positive kernels i s a n associative operation. Proof. By approximating f E 8, by simple functions, it is easily checked that ( ( L M ) N f) = ( L M )( , N f ) = L( * , ( M N f ) )= ( L ( M N ) f) , which is the desired conclusion. From now on, unless the contrary i s stated, we deal only with positive kernels. By virtue of the preceding proposition we may therefore define the powers N n of a positive kernel N ; they are the kernels defined inductively by the formula N"(x, f ) = N ( x , N"-'f) = N"-'(x, N f ) .
12
TRANSITION PROBABILITIES. MARKOV CHAINS
CH. I, $1
For I J = 0, we set N o = I, where I(%, ) = E,( * ). The convolution powers of a positive probability measure p on a group will be denoted ,u* or more simply p”. Let us give yet another definition that we shall need in the sequel.
Definition1.7. Let M and N be two kernels; we say that kl is smaller than N and write M N if, for every f E b,, we have Mf N f . We write M < N if in addition there exists an f E 8, such that Mf < N f .
<
<
The following definitions are basic.
<
Definition 1.8. A kernel N such that N ( x , E ) 1 for all x in E is called a transition Probability or a submarkovian kernel. I t is said to be markovian if N ( x , E ) = 1 for all x in E. In the sequel we shall often write T.P. instead of writing in full the words “transition probability.” Throughout almost all the sequel our basic datum will be a transition probability, denoted by the letter P , the properties of which we shall study from the probabilistic point of view as well as from the potential or ergodic theoretical points of view. The powers of P will be denoted P , rather than Pn;this integer n will appear as a “time” in the sequel. We close this section with a few useful remarks. If P is a T.P., the operators associated with P according to Definition 1.2 will also be denoted by P. They are $ositive operators, that is, they map positive functions (measures) into positive functions (measures). Moreover, they are linear and continuous operators on the Banach spaces bb and bA(b), and their norm is less than or equal to 1. This may be stated: P is a positive contractiolz of these Banach spaces. Finally notice that the product of two T.P’s is itself a T.P.
Exercise 1.9. Prove Proposition 1.3. [Hint. Define the kernel N by setting N ( x , A ) = V l , ( x ) . ] Exercise 1.10. By using the convolution kernels associated with the three following measures on R, prove that Proposition 1.6 may fail to be true if the kernels are not positive: (i) the Lebesgue measure, (ii) - E ~ (iii) , the restriction of the Lebesgue measure to R,. Exercise 1.11. Compute the powers of the kernels defined in Example 1.4 (ii).
CH. 1, 52
HOMOGENEOUS MARKOV CHAINS
13
Exercise 1.12. If N = a @ 3, compute M N and N M for an arbitrary positive kernel M .
Exercise 1.13. If N is the kernel of Example 1.4(iii),prove that Nf WZN= ~ ( w z ) .
=f
0
8 and
Exercise 1.14. If E is countable and Pis a T.P. on E written as a matrix, then Pn can be written as the nthpower of this matrix. Exercise 1.15. For any Bore1 function on a group G , define T,f by T,f(x) = f ( x g ) . Prove that a kernel N on G is a convolution kernel as defined in 1.4(ii) if and only if T E N = NT,. Exercise 1.16. Let M and N be two kernels and suppose that N is an integral kernel; prove then that M N is an integral kernel. Exercise 1.17. Let P be a T.P. on ( E , 8). A sub-a-algebra B of B is said t o be admissible for P i f : (i) it is countably generated; (ii) for any A E 99 the function P ( * , A ) is @measurable (in other words P is a T.P. on ( E ,B ) ) . Prove that any countable collection of sets in d is contained in an admissible a-algebra. [Hint: The smallest algebra go containing the given collection and all the sets { x : P(x, A ) I } for rational I and A in go is countable. The a-algebra is generated by BO.]
<
Exercise 1.18. Let N and A be two positive kernels on ( E , 8).Prove that the kernel S = ( N A ) "N satisfies the relation
znaO S
=
N
+ NAS
=
N
+ SAN.
If B is another positive kernel, then
C (SB)"S = C ( N ( A + B))"N.
n>O
n>O
2. Homogeneous Markov chains Let (Q, 9, Po) be a probability space and X = {X,},,, a sequence of random variables defined on Q with their range in E. Let 9, = a(X,, m n) and 9, be an increasing sequence of a-algebras such that g , , D . F , , f o r every n.
<
14
T R A N S I T I O N PROBABILITIES. MARKOV C H A I N S
CH. 1, $2
Definition 2.1. The sequence X = {X,},,, is said to be a Markov chain with respect to the a-algebras 9, if, for every n, the a-algebras 9, and a(Xm,m n) are conditionally independent with respect to X,; in other words, if for every A E 9,, and B E a(X,, m n)
>
I
I
I
P,[A n B X,] = P,[A X,] P,[B X,]
as.
<
If 9, = 9,, we say simply that X , is a Markov chain; the “past”a(X,, m n) and the “future” a(X,, m 2 n) then play totally symmetric roles in this definition, the intuitive meaning of which is clear: given the present X,, the past and the future are independent. Notice that if the above property is true with the a-algebras 9,, it is a fortiori true with the a-algebras 9,Finally . we emphasize the importance of Po in this definition; if we change the probability measure Po there is no reason why X should remain a Markov chain. Pro osition 2.2. T h e sequence X i s a Markov chain with respect to the a-algebras 9$f and only if, for every random variable Y Eba(X,, m 3 n), Eo[Y I 9,,] = Eo[Y I X,]
Po-as.,
where E , dsnotes the mathematical expectation operator with respect to Po. Proof. Pick A in gnand B in a(X,, m 2 n ) ;then since gn3 a(Xn)
I
E O [ ~7 A B Xnl = E O [ l A
I 1 y n
xnl
= EO[’A E [ 7 B
I ’n] I xnl#
and if the property in the statement is true, this is equal to
which proves that {X,},,, is a Markov chain. Conversely it suffices to show the above property when Y = 1, with B in a(X,, m 2 n), and this amounts to showing that for every A in Yn, IBI
=
EOVA E O V ~
I X~II.
But since {X,} is a Markov chain, the right member is equal to EO[EO[~AE [ I B
1 xnl I X ~ I =I
EO[EO[~A
= E,[E,[I,
which completes the proof.
I Xnl 7B
I Xn11
IX~II =
EOP,
7 B l ~
CH. 1, $2
HOMOGENEOUS MARKOV CHAINS
15
The following definition is basic.
Definition 2.3. The sequence X = {X,},,, of random variables is called a homogeneous Markov chain with respect to the a-algebras 9, with transition Probability P if, for any integers m, n with m < n and any function f E bb, we have EoV(Xm) 3 n 1 = P n - m f ( X m ) P0-a.s.
1
The probability measure v defined by v ( A ) = P o [ X oA~] is called the starting measure. If 9, = F,, we say more simply that X is a homogeneous Markov chain with transition probability P. We leave t o the reader as an exercise the task of showing that a Markov chain in the sense of Definition 2.3 is a Markov chain in the sense of Definition 2.1. To this end the following proposition comes in useful.
Proposition 2.4. The sequence {X,],,, i s a Markov chain with transition probability P if and only if for every finite collection of integers to = 0 < tl * * * < t , and functions f o , . . ., f , in b b one has
Proof. If n = 1, the above formula follows a t once from Definition 2.3. Now, applying Definition 2.3, we get
Since f n P 1 Pt,-t,-l f , is still a function of bb, the “only if” part follows by induction. To obtain the “if” part, it suffices by the monotone class theorem, t o prove that for every integer to = 0 < ti < < tk m < n and functions f o , f l , . . ., f k , f in b b one has 1 . -
<
16
TRANSITION PROBABILITIES. MARKOV CHAINS
CH. 1, $2
but this is an immediate consequence of the above formula.
Remark 2.5. I t is easily seen that the condition in Definition 2.3 is satisfied’ provided it is satisfied for every pair of consecutive integers. The main goal of this section is to show that with every transition probability we may associate a homogeneous Markov chain, which will be one of the main objects of our study. We begin with a few preliminaries. Let P be a T.P. on E . Let A be a point not in E and write E,j = E U { A } and 8, = a(&, { A } ) .We extend P to (Ed,8,)by setting
P(x,{ A } ) = 1 - P(x, E ) if x # A ,
P(A,,{ A } ) = 1.
If E is locally compact we choose as A the point at infinity in the Alexandrov’s compactification of E . We introduce the following convention : any numerical function f on E will automatically be extended to E , by setting / ( A ) = 0. Heuristically speaking, P is the tool which permits us t o describe the random path of a “particle” in E. Starting a t x a t (ime 0, the particle hits a random point x1 at time 1 according to the probability measure P(x, * ), then a random point x g a t time 2 according to P ( x l ; ) and so forth. If P(x, E ) < 1, that means that the particle may disappear or “die” with positive probability; by convention it then arrives at the “fictitious” point A , where it stays for ever after. We ‘assume however that P ( x , E ) > 0 for every x in E ; that is, the particle does not die at once with probability one. Definition 2.6. The point A is called the cemetery. We shall rather say “point a t infinity” when E is locally compact. The space ( E , 6‘) is called the state space. We are now going to give a mathematical formulation of the above description. For every integer n 3 0, let (I?:, 8;)be a copy of (Ed,gd);we call (Q, 9) their product space, namely 0 = ES, and F is the a-algebra generated by the semi-algebra Y of measurable rectangles of Q. We recall that a measurable rectangle is a set of the form
n:=o
fiAn,
An€&:,
n=O
where it is assumed that A , is different from EZ for only finitely many n.
Definition 2.7. The space 52 is called the canonical #robability space. We call X,, n 3 0, the coordinate mappings of 0.Let w = {x,, n 0) be a point
CII. 1, $2
HOMOGENEOUS MARKOV CHAINS
17
in Q, then X , ( o ) = x , ; we also set X,(o) = d for every o in Q. These mappings X , are random variables defined on 9 with range in E , and are clearly measurable with respect to the o-algebras 9, = cr(Xm,m n). A point o in 9 is referred to as a trajectory or a path.
<
We come to our main result.
Theorem 2.8. For every x in E there exists a unique probability measure P, on (9,9) such that for any finite collection no = 0 < nl < n2 < < n, of integers and every rectangle k
Furthermore for every set A
E
3,the map x
+
Px[A]as &,-measurable.
Proof. Equation (2.1) defines clearly an additive set function on 9'which has a unique extension to an additive function on the Boolean algebra d= and the restriction of this set function to each of then-algebras 9,is probability measure. We still call P, the extended set function. The map x -+ PJA] is then in & for every A ~ dindeed ; this is true for measurable rectangles and the class of sets B E 9, for which it is true is, for every n, a monotone class. To prove the first half of the theorem, we set@ = E)L;,and for A ~d and (xo, x l , . . . , X,) a finite collection of points in E , we write A ( x o ,x,, . . ., x,) for the set of points on+lin Qn+lsuch that ( x o , x,,.. . , x,, on+l)is in A . The space 9" being isomorphic to 9, we may define a set function on the algebra generated by the rectangles of Qn in the same way as we have defined Y,.This set function will be denoted by Pz.Again we observe that the map x -+ P=[A]is &-measurable for A ~ dMoreover . using the same argument as to prove Fubini's theorem, we can see that for A E & we have
u:==,9,,
n;=,
c
To prove that P , can be extended to a probability measure on 9we have
TRANSITION PROBABILITIES, MARKOV CHAINS
18
CH. 1, 92
to show that it is a-additive on &’. Let { A j } be a sequence in d decreasing to 0 and let us suppose that limj P,[Af] > 0 . By the last displayed formula applied to A’ we see that this implies, since P(x,* ) is a probability measure, that there exists a point f l E E such that lim, P:,[Aj(x)]> 0. As a result A j ( x ) is non-empty for every j ; moreover reasoning with Pi, and { A j ( x ) }as we did with P , and A f we find that there exists a point Z2 E E such that lim ~ i , [ ~ f a,)] ( x , > 0. i
Proceeding inductively, we find that for every n there is a point I n such that A+, Zl,X2,. . ., 2,) is non-empty for every 1. But since each A j depends on only finitely many coordinates it follows that the point w = ( x , Zl,Z,, . . . , Z n , . . .) belongs to every A f which is a contradiction. To prove the second half of the statement, we observe that the class of sets R such that x + P,[B] is measurable is a monotone class which includesd. Definition 2.9. For every probability measure v on ( E ,b),we define a new probability measure P , on (Q, 9) by setting
ttrdwit, tho tnr*nurt+bllltjy Lit ttw p~irr~bltt~ the~trclcrrHlVra (IPIONII $13 Cilr above formula, which clearly defines a probability measure. For Y sx E, we have P , = P,, and if A is the rectangle in Theorem 2.8, then since A = we have
PJAI =
By comparing this relation with that of Proposition 2.4,we can state:
Proposition 2.10. For every Probability measure P, the sequence X
=
{X,),,,
CH. 1, 52
HOMOGENEOUS MARKOV CHAINS
19
i s a homogeneous Markov chain with transition $robability P and starting measure v. It is called the canonical Markov chain with transition Probability P.
Proof. By the usual argument we may replace the sets Ani in the above relation by functions fni, so that X satisfies the condition of Proposition 2.4. Definition 2.11. Let q(o)be a property of w ; then q is said to hold almost surely (a.s.) on A E 3,if the set of o ' s in A for which q(o)fails to hold is contained in a set A , E 9 such that for every x E E , P,(Ao) = 0. If A = Q we simply say "almost surely". We proceed with some more definitions and notation. Let Z be a positive numerical random variable on (52,s). Its mathematical expectation taken with respect to P , will be denoted E,[Z]; if Y = E,, then we write simply E,[Z]. I t is easily seen that the map x E,[Z] is in Q and that --+
E,[Z] = Furthermore if f have
E b,,
5.
v(dX) E,[Z].
then f ( X , ) is a random variable on (in,9) and we
Indeed, for f = 7, this'formula is a special case of eq. (2.1), and it may be extended to Q, by the usual argument.
Definit,ion 2.12. The shift operator 8 is the point transformation on n defined bY e({x,, X I , . . ., xn,. . .>) = (xi, ~ 2 , .. ., %,+I,. . .>. It is obvious that 8 E F / F and that X , ( ~ ( W )=) Xn+l(o).We write 8, for the fithpower of 8: 8, = B o 8 * o 8 p times. Clearly X,(e,(w)) = Xn+P(w), which will also be written X , o 8, = X,,,. Since
-
Or'{X,
EA
}
= {Xn+, E A ) ,
it is easily seen that 8, E cr(Xn,n 2 $)IS. The following proposition gives the handy form of the Markov property and will frequently be used in the sequel.
20
TRANSITION PKOBABILITlES. MARKOV CHAINS
CH. I , $2
Proposition 2.13 (Markov property). For every fiositive random variable 2 on (Q, F), every starting measure v and integer n,
E,[Z
I
0 , F,] = E,,[Z]
o
P,-u.s.
on the set { X , # A } . The last phrase is necessary to be consistent with the convention that functions on E , in particular the function E.[Z],are extended to E by making them vanish at d ; and there is no reason why the first member should vanish on { X , = A } . In the most frequent case however, where P is markovian, the event {X,, = A } has zero P,-probability for every Y and the above qualification may, and often will, be forgotten in the sequel. Before we proceed to the proof let us also remark that the right-hand side of the above relation is indeed a random variable because it is the composite mapping of the two measurable mappings w + X , ( o ) and x --+ E,[Z]. We also notice that if Z = l(xmsA), A E 8, the above formula becomes
P,CX,+,
EA
1 9,1= P,,,[x,~
E
AI P,-a.s.,
which is the formula of Definition 2.3,
Proof of 2.13. We must prove that for any B E F,, JB
z
0
e n l{XneE)dpv =
5.
E,,LZI u v ;
and by the usual extension argument it suffices to prove this relation for the case in which B is a rectangle, namely B = {X,,, E B,, . . . , X , E Bk} with B i e 8,and in which 2 = 1 A , where A is the rectangle used in Theorem 2.8. The result then follows immediately from eq. (2.2). As much as P , our basic datum will henceforth be the canonical chain X associated with P. We shall be concerned with all probability measures P,, and shall use freely the Markov property of Proposition 2.13. Indeed, although the canonical chain is not the only chain with transition probability P , it has the following universal property which allows one to translate any problem on a chain Y to the analogous problem on the canonical chain X .
Proposition 2.14. Let Y be a homogeneous Markov chain defined on the 9robabil-
CH. 1, $ 2
HOMOGENEOUS MARKOV CHAINS
21
-
ity s*ace ( W ,a, Q ) with transition probability P and starting measure v ( ) = Q[Y,E. 1. T h e n the canonical Markov chain associated with P i s the image of Y by the rna$$ing q which sends w E W to the 9oint 0
of SZ
=
=
~ ( w=) (yo(w), Y l ( w ) , **
9
,
Yn(w),. . .)
E:. T h e image of Q by q i s equal to P,.
Proof. The mapping is measurable because the composite mappings of with the coordinate mappings X , are equal to the random variables Y,. I t is then easily verified that p(Q) = P,. Exercise 2.15. Prove that a real random variable 2 is u(X,, m urable if and only if Z = 2' o 0, where 2' E 9.
3 n)-meas-
Exerciso 2.16. A sequence {X,},,, of random variables defined on (Q, 9, P) is a Markov chain of order r if for every B E d and integer n, PIX,+I
E
I
B u(Xm,m
< n)] = P[X,+,E B 1 o(Xm,n - r + 1 < m < n ) ] .
Prove that the sequence of random variables Y, = ( X , + l , . . . , Xn+r--l)with range in Eris a Markov chain in the ordinary sense.
<
Exercise 2.17. (1) Suppose that for every pair m n of integers there is a T.P. called P m , , such that Pl,,P,,., = Pl,,. Prove that one can associate with these T.P.'s a non-homogeneous Markov chain, that is, a sequence of random variables X , defined on a space (SZ,9,P) and such that for f E b b
EV(Xn)I ~(x,, k <m ) = ~ Pm.n/(Xm) a.s. (2) In the situation of ( 1 ) the space-time chain (X,x n) with range in E x N is a homogeneous Markov chain. ((2) may be solved without previous solution of (I).)
Exercise 2.18. Let X be a homogeneous canonical Markov chain and ( W ,d ,Q) a probability space. Let U be a random variable defined on W and endow the space 0 = 9 x W- with the a-algebras 3, = u(X,, X I , . . ., x,,U ) , and the probability measures
P
=
9=u
u 59, (n10
)
P @ Q. Prove that the sequence
{x,}
22
TRANSITION PROBABILITIES. MARKOV CHAINS
CH. I, $2
x,(w,
defined by to) = X,(w), is still a homogeneous Markov chain with the same T.P. with respect to the a-algebras 9,.
Exercise 2.19. If {X,,] is a homogeneous Markov chain, then E[f(X,,) 1 XJ = P,,-,,, f(X,) as., but the converse is false as the following example shows. Set E = {1,2,. . ., N } and define B = 52, U Bz where sZ1 = E and Q2 is the set of permutations of E . Define a probability measure P on 52 by setting with (obvious notation) P(W1) =
N-2,
P(o~= ) (1 - N-’) (A’!)-’.
< <
Define further random variables X,, 1 n N , with values in E by X,,(wl) = w,, X,(w2) = i if i is the tzth number in the permutation w2. If we define a T.P. on E by P(i,j ) = N - l , prove that
P[X,
=
i I Xn-, = i] = P(i,j ) ,
n
2,
although for N 2 3, the sequence {X,} is not a Markov chain. By using the space B x B x B * build an example where {X,} is indexed by K.
--
Exercise 2.20. The following chain may be seen as describing the evolution of the size of a population where, at each generation, the random number of offsprings of the individuals are independent and equally distributed. I t is called the Galton-Watson chain. The state space is the set N and the conditional probability P[X,+, = x 1 S,] is equal to the law of the sum of X, independent equidistributed random variables of law fi = (#(x), x E N).The T.P. is thus given by P(x,y ) = #*“(y). (1) Prove that for every n there exists a probability measure 9, on E such that P n ( x , Y) = p , * z ( ~ ) . [Hint: Begin by proving that the mapping from Ox into B defined by Xn(p(w1*** *
t
0s)) =
Xn(wJ
+ . + Xn(wz) *
*
sends the product measure P , @ P , @ ... @ P , to P,. This is the mathematical formulation of the following intuitive fact: the chain starting at x behaves as the sum of x independent chains starting at I.] (2) Prove that the generating functions g,,(l) = tX#,(x) are given by the recurrence formula gn+l(t) = g n ( g ( t ) ) = g(gn(t))p
zxsE
where g is the generating function of the measure #. Compute for P1 the
CH. 1, $3
STRONG MARKOV PROPERTY
mean value and variance of to exist.
23
X, as functions of those of X , which are assumed
3. Stopping times. Strong Markov property
Definition 3.1. A stopping time of the canonical Markov chain X is a random variable defined on (52,s) with range in N U ( 0 0 ) and such that for every integer n the event {T = n } is in 9,The . family STof events A E 9 such that for every n, {T = n} n A E 9, is called the a-algebra associated with T . I t is easily verified that F Tis indeed a sub-a-algebra of 9. The constant random variables are stopping times, and if T ( w ) = 72 for every w E 52, then F T = Fn. The stopping times thus appear as generalizations of the ordinary times. The following examples of stopping times are basic.
Definition 3.2. For A E 8, we call first hitting time of A and first return time of A the random variables defined by T A ( w )= inf{n S,(w)
=
inf{n
0 : X,(w) E A } ,
> 0 : X,(o)
EA},
where in both cases the infimum of the empty set is understood to be
+
00.
I t is readily checked that both variables are stopping times. For example n-1
{T,
=
n} =
n {x,E A"} n {x,E A } E 9,.
m=O
In the same way the random variable c ( w ) = inf{n
2 0: X,(o)
= d}
is a stopping time called the death-time of X . If P is markovian, to m.
+
5 is a.s. equal
Definition 3.3. With each stopping time T we associate the following objects: (i) The random variable X, is defined by setting
X , ( o ) = X,(w)
if T ( w ) = n,
X T ( w )= d if T ( w ) =
+
00.
I t gives the position of the chain at time T , and the reader will easily check that X T E % T / 8 d .
CH. 1. 53
TRANSITION PROBABILITIES. MARKOV CHAINS
24
(ii) The point transformation O,(w) = O,(w)
8T
on 9 is defined by setting
if T ( w ) = n,
8T(o) =
if T ( w ) =
w,,
where w,, is the trajectory { A , A , . . . , A , . . .} of eT E9 19 and that
+
00,
SZ. It is easily seen that
Proposition 3.4. Let S and T be two stopping times; then the mapping S T o Os : w -+ S ( w ) T(Os(w))i s a stopping time.
+
+
Proof. We have {S
+ Toes
=
n}
=
u {S = p } n { T
0Os
= n -$}.
P
The event {S = p } is in 9, c 9,and on {S = p } we have Os
{ T o e s = n - p ) = { T o o , = rt
-p}
=
=
8,; hence
q l ( { T = 12 - ~ } ) E F ,
since {T = n - $} E .F,,-,. Intuitively one should think of stopping times as the first time some physical event occurs and of P Tas containing the information,on the chain up t o time T when this event occurs. The time S T o Os is the first time where the “event T” occurs after the “event S” has occurred. For instance if A E b, n + T , o 8, is the first hitting time of A after time n ; in particular S,’ = 1 T , o el. If A , B E 6, then T , T , 0 8, is the first time the chain hits B after having hit A . Despite its simplicity the following result is basic. It implies in particular that if one starts to observe the chain at a stopping time, the ensuing process is still a homogeneous Markov chain with the same T.P. (cf. Exercise 3.14).
+
+
+
Theorem 3.6 (Strong Markov property). For every real positive random variable Z on (SZ, 9), starting measure v and stopping time T ,
By convention the two members vanish on { X ,
= A}.
The right member is the composite mapping of w
X , ( w ) and x
-+
--+
E,[Z].
26
STRONGMARKOVPROPERTY
CH. 1, 53
Proof. We have to show that for A
E
F,,
or equivalently
I t suffices therefore to prove that for every n
0 and B
E Fn,
but this is precisely the Markov property of Proposition 2.13.
Definition 3.6. With each stopping time T we associate a new T.P., denoted
PT,by setting for B E 8, PT(%,B ) It is easily seen that if f
E b,,
= J?,[XTEB].
then
PTf(x)
= Ez[f(XT)l.
We recall that by our conventions f ( X T )= 0 on { X , = A } . We further notice that if T = n as., then P, = P,. Finally if A E 8, we write P A instead of P,; this T.P. is called the balayage operator associated with A . Before we state our next result, we introduce the following notation. Let E 8 ;define I , to be the operator of multiplication by l , , I , f ( x ) = f ( x ) if x E A , I , f ( x ) = 0 if x $ A . We may now give for PAthe following analytical expression.
A
Proposition 3.7. For A
E
8, PA
=
2 (IACP)"I,. n2.O
Proof. Let B
E 8; using
eq. (2.2), we have
26
TRANSITION PROBABILITIES. MARKOV CHAINS
P ( x n - l ? dxn)
CH. 1, 83
IBnA(%n),
which is the desired result. Clearly the measures PA(%,) vanish outside A . Furthermore if x E A , then PA(x, ) = E,; indeed x E A implies P z [ X o= x ] = 1, hence TA = 0 P,-as. and consequently X T A= x P,-a.s. In the same way, if x $ A , then TA = S A P,-as. and P,(x, * ) = PSA(x, * ). We close this section by applying the strong Markov property to compute the products of operators P T . We first notice that if S and T are two stopping times then XT(Os(0))= X s + T o e s ( W ) , which we write simply X T 0 Bs 1
-
=
xS+ToOs-
Proposition 3.8. Let S , T be two stopping times; then PsPT = PS+ToBS. Proof. Let f E bb,. We have, applying Definition 3.6,
We set ITA
=
I, PsA = I, P
The interpretation of the kernel ITA is given in Exercise 3.13.
Exercise 3.9. Let A , B E 8 ;can TAue and TAnB be expressed as functions of TA and TB ? Given A c B , compare TA and T , and prove that PBPA= PA. Exercise 3.10. Let S and T be two stopping times. (1) Prove that inf(S, T ) and sup(S, T ) are stopping times. In particular S A ~tis a stopping time for every n. Moreover Finf(S,T) = 9 s nF T .
STRONG MARKOV PROPERTY
CH. 1, 53
27
(2) I f r E 2 F t s , t h e n r n { S ~ T } € 2 F T . I f S ~ T , t h e n . F t s c ~ 6 , . (3) Prove that the event { S T } is in .Fs n .FT; (4) Prove that the conditional expectations E[ .Fs] and E [ I .FT]commute and that their product is E [ )2Finr(s,T)].
<
I
Exercise 3.11. Let p be a function mapping N into itself. The random variable p ( T ) is a stopping time for every stopping time T , if and only if p enjoys the following property: there is an integer k, which may be co, such that
+
p ( n ) 3 n for n
< k,
~ ( n= )
k for n > k .
If p is now a function mapping Nd into N,then p(T1,T,, . . ., T d )is a stopping time for any d-uple of stopping times T i if and only if the functions obtained from p by fixing d - 1 variables enjoy the above property.
+
Exercise 3.12. (1) Let A E B; prove that T , = n TA 0 8 , on { T , 2 n } . Let A , B E 8 ;has TA T Bo 8 T A the same property as TA? (2) Let T be a stopping time such that T = n T o 8 , on { T > n } . Prove that
+
+
ICT>nl
=
l I T > n - l l * IIT>11
en-l.
(3) Let Y be the sequence of random variables defined by
Y,(w) = X,(o)
if T ( w ) > n ,
<
Y,(w) = d if T(o) n.
Prove that Y is a homogeneous Markov chain with respect to the a-algebras P, on (Q, F). When T = T , we can take A" as its state space; prove then that its T.P. is equal to IAcPIAc.The chain Y is said to be obtained by killing X at time TA.
S T Afor n every probability measure
Exercise 3.13. Let A
T I = SA, T ,
=
E
d and define inductively the times
T1
+ SA
0
OF,,.
. ,, Tn = Tn-1 + S A
0
8Tn-lp..
.
(1) Prove that the times T , are the successive times at which the chain returns in A . (2) Prove that the sequence {Y,= XT,} is a homogeneous Markov chain with respect to the a-algebras 2FT, with T.P. This chain is called the chain induced on A or the trace chain on A .
n,.
Exercise 3.14. Let T be a finite stopping time. Prove that the sequence Y = {XT+n}n>Ois a homogeneous Markov chain with transition probability
28
TRANSITION PROBABILITIES. MARKOV CHAINS
CH. I, 54
P with respect to the a-algebras .FT+n.What can be said if T is not finite ? Exercise 3.16. Stopped chains. If T is a stopping time of X prove that the sequence Y = {XTAn}n>Ois a homogeneous Markov chain if and only if T = T A for a set A E 8.In that case write down the T.P. of the chain Y . Exercise 3.16. Let T be a stopping time and G a real function on SZ x 52 measurable with respect to Sr @ 9. Prove that for any v, f
for w, w' E 52. [Hint: Begin with G ( w , 0 ' ) = ~ ( w $(w'), ) where T E S,.]
Exorcise 3.17. Let T be a stopping time and S a random variable *",-measurable and 2 T a.s. Prove that, for f E b+, Ev[f
xS
I FTl
(w)= P S ( ~ I ) - T ( ~ ) f( )~ T ,
[Hint: Use the preceding exercise with G(w, w') = f(Xs(aIl(w')).]
(a, F), with range in N u {a},
Exercise 3.18. A random variable L defined on is called a death time if L o 0 = ( L - l)+,that is, L08=L-l (1) Let A
E 6'; prove
ifL21,
L 0 8 = 0 ifL=O.
that the last hitting time of A , namely
>
L A = SUP{% 0, X ,
E
A},
where the supremum of the empjy set is taken equal to zero, is a death time. The death time of the chain is a death time. (2) Prove that X L o 8 = X , on (0 < L} if L is a death time. (3) If L and L' are death times, then L v L' and L A L' are death times; ( L - n)+ is a death time for every n.
4. Random walks on groups and homogeneous spaces In this section we define an important family of Markov chains, which will occur frequently in the sequel both for their intrinsic interest and to provide examples about general results. In the sequel G is an LCCB group and we call 9 the a-algebra of its Bore1 sets. The elements of G will be denoted g, g', h,. . . and the inverse of g by g-l. The unit element is denoted e.
RANDOM WALKS
CH. 1, $4
29
Definition 4.1. A right (left) random walk on G is a Markov chain with state space (G, 9)and transition probability E~ * p ( p * EJ, where p is a probability measure on (G, 9 ) which is called the law of the random walk. For the right random walk we have therefore P(g, A ) = .sK * p ( A ) . For h E G we set hA = fhg: g E A } ; we then have P(hg, hA) = P(g, A ) . The right random walk is invariant under left translations (see Exercise 1.15). Of course on an abelian group there is only one random walk of law p.
Proposition 4.2. Let X be the right random walk of law p ; then for every P , the random variables 2, = X;JIX,, n 2 1, are indefiendent and equidistributed with law p. Proof. Let f i , i We have
=
1, 2 , . . ., # be a finite collection of bounded Borel functions.
For every g in G we have E,[f(XG'X,)] = p ( f ) , so that we get inductively P = nEv[fi(zi)I, i=l
which is the desired result.
Remarks 4.3. The random variable X , is thus a s . equal to the product X , Z1 * * Z,, where the Ziare independent with law p. In particular X , is P,-as. equal to the product of n independent equidistributed random var-
-
iables. Of course there is a left-handed version of these results and the left random walk may be written Z , Z,-l * * Z, X,. The invariance by translation is also obvious from this relation. We assume now that G operates on the left on an LCCB space M ; let A be the u-algebra of Borel subsets of M . We may associate with p a T.P. on ( M , .A) by setting (cf. Example 1.4 (ii)), for x E M , A E A,
-
We are going to show an interesting way of constructing a Markov chain with transition probability P.
TRANSITION PROBABILITIES. MARKOV CHAINS
CH. 1, $4
Let X be the left canonical random walk of law p on G . We set
a = M x Q,
30
g,,
= 4 @ 3, = 4 @ S,,, and if v is a probability - measure on ( M , 4 ) we call P , the probability measure v @ P, on (0, F). Next, for 6 = ( x , w ) we set Yo(&)= x and
Y,(&) = X,(O) Yo(&)= X , ( w ) x . Proposition 4.4. The sequence {Yn}n>ois a homogeneous Markov chain with respect to the a-algebras .Fnfor any probability measure P,. Its transition probability is equal to P . Proof. The a-algebra %, is generated by the rectangles and I' E 3,,. For A in 4 we have
I
7A(ym+n)
dPv
=
AXr
where
A
=
y(dx)
5
r
7A(Xm+n(w)
(1 x
I' where (1E
dpe(w)
{g: gx E A } ; but (pm * ex,,) (A)= (pm* E ~ , , X ) ( A ) , so that finally
which is the desired conclusion.
Remark 4.5. The chain Y thus constructed is not the canonical chain associated with P, but it is sometimes useful to know that one can construct a chain associated with P in the above way, for instance when one must deal simultaneously with the random walks on G and on M . In the preceding discussion one could not use the right random walk instead of the left random walk unless G operates on M on the right. We shall, however, show that under an additional hypothesis one may associate with right random walks some interesting random walks on (left) homogeneous spaces of G. We begin with a result of more general scope. Let X be a Markov chain on ( E , 6 ) and a a measurable mapping from ( E , 6 )onto a space (E', 8')such that, for every A' E b',
CH. 1, $4
RANDOM WALKS
31
P ( x,cs-l(A’)) = P(x‘,a-l(A’) if a ) . ( and moreover such that for A
E
x:,= a(X,),
= u(x’),
(4.1)
8,a(A) E 8‘. We set
P‘(x‘, A )
=
P(u-l(x‘),u - l ( A ) ) ,
where a-l(x’) is any point in u--l({z’}).Thanks to the second property of u, it may be checked that P‘ is a T.P. on E‘, and we have Proposition 4.6. T h e sequence {X:},,, (E’, 8’) with transition Probability P’.
i s a homogeneous Markov chain on
Proof. Let A’ E 8’and P v be a probability measure on the space 0 of the chain X ; we have
pv[x:+, E A‘
19n1
= Pv[c(Xrn+n)
13n1 =
= P,[X,,
=
u-l(A‘)]
Pv[Xrn+nEO-YA’)
P:[x:, A’],
since it is easily seen from eq. (4.1) that
Pi(%’,A’)
=
P,(cs-l(x’), a-l(A’)).
Remark and Examples 4.7. Here also the chain X : is not the canonical chain associated with P’, since the random variables X : are defined on the space D of the chain X . This proposition allows us to construct new chains from already known ones. Let us call symmetric a random walk such that p = 1; where fi is the image of ,u by the mapping x x-l. Let X be a symmetric random walk on R or Z. Then we may apply the above result with the map u :x 1x1. We then say that X‘ is obtained from X by reflection at the point 0 . Now let G be a group and K a compact sub-group of G ; let cs denote the canonical, continuous and open mapping G -+ G/K. If x E G / K , d ( x ) is of the form g K and the T.P. of the right random walk of law p on G satisfies the conditions of Proposition 4.6 if ,u is K-invariant, that is to say that sk * ,u = ,u for every k E K or equivalently that ,u = mK * p‘, where m K is the normalized Haar measure for K and p’ any probability measure on G. The image of the right random walk on G by u is then a homogeneous Markov chain on G / K .When G is a semi-simple Lie group with finite centre and K a maximal compact sub-group, we thus obtain random walks on the Riemannian symmetric space C / K . --+
--+
32
CH.1, $4
TRANSITION PROBABILITIES. MARKOV CHAINS
We could have taken the image of the left random walk by well-known equivariance properties of u would then imply that U(2,
z,-1* - 2, X,) *
= 2, 2,-1
-
U,
but the
21 U ( X 0 )
and we would be in the general situation of Proposition 4.4.
Exercise 4.8. A sub-random walk is a Markov chain on a group G with T.P. * p (or p * E ~ but ) with p(G) < 1. Prove, using the same notation as in Proposition 4.2, that the random variables of any finite collection Z,,,. . ., Z,, are independent on the set {t> nJ. E,
Exercise 4.9. (1) Let X be a right random walk of law p and T a stopping time. Prove, after restricting the probability space to Q, = {T < a},that the variables X,'X,+, are independent of 9,and that Y = X,,, is still a right random walk of law p. [Hint: See Exercise 3.14.1 (2) We assume that the underlying group is R or Z and we define inductively the following sequence of random variables T,(w) = 0, ~ ~ ( =winf{n ) 2 1: X,(o)
> Xo(o)),
T,(w) = inf(n > T,-1: X,(w)
> XTfi-,(w)},
+
where the infinum of the empty set is taken equal to 03. Prove that the variables T , are stopping times and that the sequences {T,} and {X,,,} are sub-random walks. Under which condition are they true random walks?
Exercise 4.10. Let X be a random walk of law p on G and T , the nthreturn time (cf. Exercise 3.13) to a closed subgroup H . Prove that Y = { X T n } is a sub-random walk on H . Give its law as a function of p and .,I Under which condition is it a true random walk? Exercise 4.11. Let G be a LCCB group which is the semi-direct product of two groups H and K.Every element g in G may be written uniquely as a pair (h,k ) , and using the additive notation in K we have (h,k) (h',k') = (hh',hk' k ) . (Example: G is the group of rigid motions of the euclidean plane, H is the group of rotations and K the group of translations.) Let X , = (H,, K,) be a
+
PROPERTIES O F INTEGRAL KERNELS
CH. 1, 55
33
left random walk on G ; then H , is a random walk on H and K , the Markov chain induced (cf. Proposition 4.4) by X , on K . (We recall that K is an invariant subgroup of G and therefore that G operates on K in a natural way.)
Exercise 4.12. One can study the random walks on the spaces ( W , d ,P ) equal to the infinite product (G, 99, P ) ~We . let {Z,},,, be the coordinate mappings, which are clearly independent and equidistributed. (1) The probability measure Pg on (9,9) is the image of P by the mapping from W to 9
(2) Prove that P is invariant under finite permutations of the coordinates in W . (3) An event A E&’ is said to be symmetric if it is invariant under finite permutations of coordinates. Prove that the family of symmetric events is a a-algebra and that if A is symmetric then either P ( A ) = 0 or P ( A ) = 1. This is the so-called zero-or-one law for symmetric events. Finally prove that the events in u(Zm,m 2 n) are symmetric. [Hint: To prove the zero-or-one law, approximate sets in d by rectangles C depending on the first n coordinates and use the permutation a exchanging 1 a n d n 1,2 andn 2 , . . ., n and 2n.
n;=,
+
+
6. Analytical properties of integral kernels This section should be omitted at first reading. Its purpose is to collect some results which will be useful later and which will then be referred to. Since these results deal mainly with compactness, we recall:
Dcfinition 6.1. A linear and continuous operator from a Banach space into another Banach space is said to be compact if it maps bounded sets onto relatively compact ones. A kernel on ( E , 8 ) is said to be compact if it maps the unit ball 92 of b b into a relatively compact set in bb. A compact kernel is thus a bounded kernel. Let us also recall that an operator with finite-dimensional range is compact and that every compact operator is the uniform limit (cf. ch. 6, Definition 3.1) of operators with finite dimensional range. Finally the adjoint of a compact operator is itself compact. We characterize below the compact kernels.
34
TRANSITION PROBABILITIES. MARKOV CHAINS
CH. 1. $5
Proposition 6.2. I f ( E , 8)i s separable, a compact kernel i s a n integral kernel. Proof. Let N be a compact kernel; its adjoint N* is a compact endomorphism of bB*. But since b d ( B ) is closed in bC"* and invariant by N , the operator N * , which is equal to N operating on the left, is a compact endomorphism of b.A(B).AS a result the set { N ( x , * ) : x E E } is relatively compact in b d ( B ) , and consequently there is a probability measure A on 8 such that for every x in E one has N ( x , ) << A. The proposition then follows from the following lemma.
-
Lemma 5.3. If ( E , 8)is separable, for every probability measure A on d and every proper kernel N on ( E , 8)there exists a function (x, y ) -+ n(x,y ) which i s B @ &-measurable and such that N i s the s u m of the integral kernel with basis A associated with n and of a kernel N 1 such that all measures "(x, ) are singular with res$ect to A.
-
The function n is called a density of N with respect to A. I t is unique in the sense that if n' is another function with the same properties, then for every x , we have n ( x ; ) = n ' ( x ; ) A-as. Actually, what the lemma says is that one can choose n ( x ; ) within the equivalence class of the RadonNikodym derivative dN(x, * ) / d l in such a way that the resulting bivariate function n is d @ &-measurable.
Proof. By hypothesis there exists a sequence of finite partitions Yn of E such that Pn+lis a refinement of Bnand that 8 is generated by 8". A point y in E belongs to one and only one set in Pn,which we call EJ. Our notation will not distinguish between 8" and a ( P ) . Let v be a probability measure on B ; the functions f, defined by setting
u:
fn(Y) fnb)
v(EJ)/A(EJ) if A(EJ)> 0, =0
if A(E3) = 0,
form a positive martingale relative to the a-algebras 8" and the probability measure 1,hence converge A-as. to a limit f , as n tends to 00. For every AE Bnwe have by Fatou's lemma
+
u:
j"
f, dA
< lim
1 ! *
fn
dA
Pn,this inequality
36
PROPERTIES O F INTEGRAL KERNELS
CH. 1, $5
holds for all sets in 8.Let f be a function such that
<
then, as is easily seen, E[f I Yn] f , I-a.s. and therefore by passing to the f , I-a.s. The function f , being thus the largest (up to limit we get f equivalence) function with this property, is equal to the Radon-Nikodym derivative of v with respect to 1. Let now N be a bounded kernel; we apply the above discussion to the probability measure P ( x , ) = N ( x , ) / N ( x ,E ) . The functions f , defined by setting
<
-
f n ( x ,y )
=
-
P ( x, E;)/I(E;) if R(EY) > 0,
f,(x, y ) = 0
if I(E!)
=
0,
are Q @ &-measurable; their superior limit as n goes to infinity is & @ 6measurable and provides the desired function n. I f N is merely proper, we begin by multiplying it on the right by a strictly positive function h such that N h is bounded (see ch. 2, Proposition 1.14). The property may still be extended to some non-positive kernels.
Corollary 6.4. If ( E , 8)i s separable, then every proper kernel N such that N ( x , ) << I for every x in E is an integral kernel.
-
The following results will go in the converse direction. We start with a preliminary result which is a generalization of Egoroff's theorem.
Proposition 6.6. Let H be a set of finite functions in 8, compact and metrizable for the topology of pointwise convergence; for every E > 0 there exists a set F E 8 such that R(E)< E and I,(H) is com#act for the to$ology of uniform convergence. Proof. By hypothesis there exists a sequence {x,},,o of points in E which separates H and a countable subset H , of H which is dense in H for the topology of pointwise convergence. We set h, = sup{(/
- g ) : f , g E H and If(xk) - g(xk)l < 2-" for k
< n}.
Since H I is dense in H , the supremum may be obtained by considering only the functions of H I ; as a result h, is in 8.Each function h, is finite and moreover h,+, h, and inf, h, = 0. For every sequence {g,} C H which converges pointwise to g E H , we have
<
TRANSITION PROBABILITIES. MARKOV CHAINS
36
CH. 1, $5
the following property: for every n there exists a Po such that $ 2 f i 0 implies (g - g,l 12,. I t follows that for a set F E I,such that {IF h,} converges uniformly to zero, the set 1 , H is compact for the topology of uniform convergence. The desired result then follows from Egoroff's theorem.
<
Theorem 5.6. Let N be a n integral kernel with basis A on ( E , 8)and such that NI is finite; then there exists a n increasing sequence of sets A , in Q szcch that: (i) for every E > 0 there is a n integer n such that A(Ai) < E ; (ii) the kernels I,,, N are comfiact.
Proof. The image N ( 4 ) is equal to the image by N of the unit ball in Loo@), which is compact and metrizable for the weak* topology a(Lm(A), L1(A)). Since N is a continuous operator with respect to this topology and the topology of pointwise convergence, the set N ( 4 ) is compact and metrizable for the topology of pointwise convergence. By Proposition 5.5 there exists a sequence {A,,} of sets in d with the required properties and such that the sets I,, N(%) are compact for the topology of uniform convergence hence such that the operators I,, N are compact. The preceding results are useful in many situations. In this book we shall use them to prove quasi-compactness properties. We are now going to state in a topological setting some properties which may be used to the same end. We assume below that E is an LCCB space and I the a-algebra of Bore1 sets.
Definition 5.7. A sub-markovian kernel N on ( E , 8)is said to be (i) Feller if the map x E,N from E to b d ( Q ) is continuous for the strict topology on b d ( B ) , in other words if N f E C ( E )whenever f E C ( E ); (ii) strong Feller if the same map is continuous for the weak-star topology a ( b A ( I ) , b6), in other words if NfE C ( E )whenever f E b b ; (iii) strong Feller in the strict sense if the same map is continuous for the norm topology on bA(b). --+
Plainly each of these conditions is more stringent than the previous one. The convolution kernels provide examples of Feller kernels, and even of strong Feller kernels, whenever the relevant measure is absolutely continuous. Examples of kernels strongly Feller in the strict sense will be obtained as by-products of the following results.
CH. 1, 55
PROPERTIES OF INTEGRAL KERNELS
37
Proposition 6.8. The following two conditions are equivalent : (i) the kernel N i s strong Feller in the strict sense; (ii) the image by N of bounded sets of C ( E ) are compact for the topology of uniform convergence on compact sets. Proof. If f is in the unit ball of C(E) then for every pair (x, y ) in E x E ,
thus functions N f , f in the unit ball of C ( E ) ,are equi-uniformly continuous on compact sets and (ii) follows from (i). The converse follows a t once from the relation
-
11N(x9 ) - N ( Y , * )I1 =
SUP(lNf(X)
- N f M l : f E C ( E ) , llfll d 11
and the equi-continuity on compact sets of the functions Nf(f E C ( E ) ,llfll
< 1).
R'emark. The image by N of the bounded sets of b b are also compact for the topology of uniform convergence on compact sets. Theorem 6.9. T h e product of two strong Feller kernels i s strong Feller in the strict sense.
Proof. The theorem follows immediately from the preceding proposition and the next two lemmas, in which N is a strong Feller kernel. Lemma 6.10. From every sequence {g,} of functions in the unit ball 4 of bb, one can extract a sub-sequence {gi} such that {Ng:) i s pointwise convergent. Proof. Let (x,} be a countable dense subset of E , and set I = 2,2-"N(x,, * ). The measures N ( x , * ) are all absolutely continuous with respect t o I. Indeed if f is I-negligible, the continuous function Nf vanishes a t all points x,, hence everywhere. Let now {g:} be a subsequence of {g,} convergent in the sense of a(Lm(A).L1(I)). (We indulge in the usual confusion between functions and their equivalence classes.) Then the sequence ( N g l } converges pointwise. Lemma 6.1 1. Let {g,) be a sequence of functions in % converging pointwise to a function g ; then the sequence {Ng,} converges to N g uniformly on every compact set.
TRANSITION PROBABILITIES. MARKOV CHAINS
38
CH. 1, $5
Proof. It suffices to prove the lemma for the case g = 0. Set h, = supm>,lgnl ; as lNg,l Nh, it suffices to show that {Nh,} converges to zero uniformly on compact sets. Since the functions Nh, are continuous and decrease to zero we conclude the proof by applying Dini's lemma.
<
As an application, let us give the following result. The reader may refer to ch. 2 9 6 for the definition of a resolvent. Proposition 6.12. If the kernels {Va}a,oof a submarkovian resolvent are strong Feller, then they are strong Feller i n the strict sense.
Proof. Let p > u ; the map x -+ E,V,V, is continuous for the norm topology by Theorem 5.9, and the maps x &,Vbconverge uniformly to zero whenever p -+ m , since IIc,VBI(< p-l. The result thus follows from the relation
-.
Va = (P - a)Va
J',+
J'b.
Exercise 6.13. If E is an LCCB space, NI is continuous, and N(C,(E)) c C(E), then N is Feller. Exercise 6.14. If G is a group, prove that the convolution kernels E , * ,u on G are Feller. If ,u is absolutely continuous with respect to the Haar measure, then E, * ,u is strong Feller. Prove by an example that it is not always strong Feller in the strict sense. Exercise 6.16. Let M and N be two kernels on the separable space ( E , 8); prove that there exists an & @I &-measurable function f and a kernel "such that for every x in E , the measure N1(x, * ) is singular with respect t o M ( x , * ) and
Exercise 6.16. Let N be a bounded kernel taking both positive and negative values. Prove that the map N+ defined by N+(x, ) = ( N ( x , ))+, is also a kernel. As a result, if we set a(.) = IIN(x, the function cc is &-measurable. [Hint: See ch. 6 32.1
)\I
-
Exercise 6.17. Prove the following extension to Lemma 5.3. If there is a probability measure Y and a family of sets E n increasing to E , such that
CH. 1. 56
PROPERTIES O F INTEGRAL KERNELS
-
39
N ( , En)< a, v-as. for every n, then there exist a function n and a kernel N 1 such that
for v-almost every x .
CHAPTER 2
POTENTIAL THEORY
Except in a short section devoted to resolvents, our basic data will be a transition probability P on ( E , 8)and the associated canonical Markov chain.
1. Superharmonic functions and tho maximum principle Definition 1.1. The Poisson equation with second member g is the equation (I - P)f
= g.
A finite function of d will be called harmonic if it is a solution of the Poisson equation without second member (g = 0 ) . A function f in 6, is called su$erharmonic if f 2 P f everywhere on E . The positive harmonic functions are superharmonic. The function 1 is always superharmonic; it is harmonic if P is markovian. If X is a right random walk of law p on a group G , the function f is harmonic if and only if
f k )=
1 f(@)
.G
P(dh).
Other examples will be found in the exercises. Let us mention without proof that the harmonic functions for random walks on symmetric spaces are harmonic in the classical sense, that is they are solutions of Laplace equations on these spaces. In particular, the classical harmonic functions in the unit disc are the harmonic functions of suitable Markov chains.
Proposition 1.2. The set of superharmonic functions is a left lattice and a convex cone. If { f n } n S O is a sequence of superharmonic functions, then limn f , i s superharmonic. The set of harmonic functions is a vector space. The set of bounded harmonic functions is a Banach subspace of b l . 40
CH. 2, $1
SUPERHARMONIC FUNCTIONS
41
Proof. Given that f , g are superharmonic and a,, b are in R,, the reader can easily check that af bg and f A g are superharmonic. The other properties in the statement are also straightforward.
+
The subject of potential theory, which we are about to develop in this and the next section, is the study of the cone of superharmonic functions, which will be henceforth denoted by 9'.
Definition 1.3. The Potential kernel of P, or of X , is the kernel
c P, m
'c
=
n-0
=
I
+ P + P, +
* *
* .
The reader can easily check that G is indeed a kernel, but will observe that the measures G ( x , * ) are not always a-finite. (Take P = I for instance, but less trivial examples will be seen later on.) We recall (ch. 1, Definition 1.1) that the kernel G is said to be proper if E is the increasing limit of a sequence En of sets in 8 such that the functions G( * , En) are bounded. The probabilistic significance of this condition will be seen in due course. The kernel G satisfies the identity
G =I
+ PG = I + GP.
(1.1)
On the other hand, using ch. 1, eq. (2.3),we may write, for A E 8,
Let f be in B ; if the function Gf is defined it will be called the $otential of the function f . If f = l A , then Gf(x) = G ( x , A ) is called the potential of the set A , and by eq. (1.2) appears as the mean number of times the chain started at x hits the set A . One sees easily that the potentials of positive functions are superharmonic and we have
Thoorom 1.4 (Riesz decomposition theorem). A finite superharmonic function f has a unique re$resentation as a sum f = Gg h, where Gg i s a potential and h a harmonic function.
+
Proof. By hypothesis the sequence of function P,f decreases, and thus admits a positive limit h = limn Pnf. We have
POTENTIAL THEORY
42
CH. 2, $1
hence h is harmonic. Next, g = f - P f is a positive function, and N
2 Png = f
-PN+lfj
0
passing to the limit as N tends to infinity yields Gg = f - h, which is the desired representation. Suppose now that f has another decomposition of the same kind for another pair, say (g’, h’). Applying P to both sides of the equality Gg h = Gg’ h‘, we get Gg - g h = Gg‘ - g’ h’; it follows that g = g’, hence that h = h‘ and the proof is complete.
+
+
+
+
Remark 1.b. If g E B+ and Gg is finite, then it is a solution of the Poisson equation ( I - P ) f = g.
If f is another finite solution, it is a superharmonic function which can be written f = Gg + Iz, where h = limn P,f is harmonic and positive. Thus Gg appears as the smallest positive solution of the Poisson equation with second member g. Corollary 1.6. A superharmonic fa4nction dominated by a finite potential i s a potential. A positive harmonic function dominated by a finite potential vanishes identically.
Proof. By the Riesz decomposition theorem it suffices to show the second sentence in the statement. Let h be harmonic and h ,< Gg < co; for every It we have h = P,h P,Gg and since this sequence converges to zero as n tends to infinity, the proof is complete.
<
Theorem 1.7. If the kernel G is proper every superharmonic function is the increasing limit of a sequence of finite potentials.
Proof. Let us resume the usual notation and set g, = G(n 7En); the sequence g, is a sequence of finite potentials increasing everywhere to 03. If f is
+
superharmonic, the functions f
A
gn are finite potentials and increase to f .
CH. 2, 51
SUPERHARMONIC FUNCTIONS
43
The following two results will hint a t the probabilistic significance of superharmonic functions.
Proposition 1.8. A function f i s superharmonic (bounded harmonic) if and only if the sequence { f ( X n ) }of random variables i s a supermartingale (bounded martingale) with respect to the a-algebras 9, for any Probability measure Pv. If f = Cg i s a finite potential, the supermartingale {Cg(X,)} converges to zero almost-surely.
Proof. For m 2 n and f superharmonic, the Markov property implies that EvV(Xm)
1
9n1
=
EvV(Xm-n
=
E x J f (Xm-n)I
0
en) 1 5 n 1
= Pm-n
f(xn)< f ( X n )*
Pv-a.s.
I
Conversely, integrating the inequality f(X,) E,[f(X,) 9,with ] respect to P , yields f ( x ) 3 P f ( x ) . Let now 2 = limnCg(Xn),then limn Cg(Xn) = 2 a s . (ch. 0, Theorem 2.1). By Fatou’s lemma, for every x in E , we have
EJZ]
< lim E,[Cg(Xn)] = lim P,Gg(x) n
=
0.
n
The positive random variable 2 is thus equal to zero almost-surely, which completes the proof. One of the golden rules in the study of Markov processes is to replace, as much as possible, constant times by stopping times and other kinds of random times. Although this is not as important in the case of Markov chains we will stress the idea by proving the following characterization of superharmonic functions which is little else than the stopping theorem for supermartingales.
Proposition 1.9. A function f is superharmonic if and only if P,f any pair ( S , T ) of stopfiing times such that S T.
<
PTf for
Proof. The sufficiency is obvious by making S = 0 and T = 1. To prove the necessity, let us first consider two finite stopping times S and T such that S T and T - S is a t most equal to 1. Set A , = {S = p } n { T = p + 1); this event is clearly in Fp and by the supermartingale property of Proposition 1.8, for any x E E we have
<
CH.2, $1
POTENTIAL THEORY
44
For S and T as in the statement we consequently have psf(x) 2 P ( s + I ) AT f ( X ) 2 ' ' * 2 p(S+nl
A T&)
2*
tends t o infinity, f(X(S+n) T) converges to f(X,) on {T {T = a}we have f ( X , ) = 0, it follows that
As
IZ
*
.
< a};since on
< !.h and by Fatou's lemma PTf(x) <~ !J IIP(s+fl)T f ( x )< P,f(x). f(xT)
f(X(S+n)A
T)
A
In ch. 1, $3 we saw how one can associate with X other Markov chains, in particular by killing X at time TA for A E 6'. The T.P. of this new chain is IAcPIAc; we shall denote by G A its potential kernel, so that we have m
G A = 2 I , ~ ( P I A c )=~ n=O
m
2 ( I A c P ) IAc. ~ fl=O
By reasoning in the same way as in ch. 1, Proposition 3.7 the reader can prove without difficulty that
for f = I,, this is precisely the mean number of times that X hits B before hitting A . The following result generalizes the formula G = I PG. It could be proved by computations using the analytical expressions of G, GA, P, as functions of P and I,; a simple way of doing this will be introduced in $2.
+
Proposition 1.10. For m y A
E 8,
G
=
GA
+ PAG.
CH. 2, $1
46
SUPERHARMONIC FUNCTIONS
I n particular, if g vanishes on A", then PAGg = Gg. Proof. Let g be in 8,;by applying the strong Markov property, we get m
PAG g(x)
Ex[G g(X,,)I
=
= Ex
1
Since
we have the desired result. The second sentence comes from the fact that GAIA = 0. We are now going to use the foregoing result to prove the very important
Theorem 1.11 (complete maximum principle). Let f be such that Gf is well defined, g be i n I+and a be a constant 0, suck that Gf(4 for x E { f
< Gg + a everywhere on E . = { f > 0 ) . If G f + < Gg + G f - + a on A , since the measures
> 0). Then Gf
Proof. Set A PA(%, * ) vanish outside A , we have
everywhere on E , and by Proposition 1.10,
G f + - GAf+
< Gg + G f - + a.
Since GA/+ = 0 by definition of G A , the proof is complete. The reader may even prove as an exercise that one gets Gf Gg a - f - everywhere.
< +
Corollary 1.12. The kernel G satisfies the following reinforced maximum principle: for every a 2 0 and every pair ( f , g) of functions i n &+ such that
POTENTrAL THEORY
46
we have Gf
CH. 2, $1
< a + Gg - g everywhere.
Proof. Since Gg - g = GPg, the corollary follows easily from the complete maximum principle. For our later needs we shall lay down the following Definition 1.13. A kernel G (not necessarily the potential kernel of a Markov chain) is said to satisfy the complete maximum principle (reinforced maximunz +rinciple) if it satisfies the condition of Theorem 1.11 (1.12). If, in Theorem 1.12, we replace Gg a by Gg (by a) we say that G satisfies the domination principle (the m a x i m u m principle).
+
Clearly a kernel which satisfies the complete principle also satisfies the two last principles. We shall see later that a kernel which satisfies the reinforced principle satisfies the complete principle (see Exercise 2.15). We now give a first application of the maximum principles by showing that for the potential kernel of a Markov chain one can replace “bounded” by “finite” in the definition of “proper”. Proposition 1.14. A positive kernel N i s proper if and only if there i s a function f strictly positive everywhere such that N f i s bounded. The potential kernel G of a Markov chain is $roper if and only if there i s a function f strictly positive such that Gf is finite or equivalently if and only if E i s the union of a sequence Enof sets in d such that the potentials G( , En)are jinite.
-
-
<
> 0,the sets En = { f > l / n }increase to E and N ( ,En) nNf( -). Conversely, if N ( * , E n ) M,, the function f = M;’2-nlEn is strictly positive and Nf is bounded. In the same way, the existence of f > 0 with Gf < to implies the existence of sets En with finite potentials G( - , En). Set
Proof. For f
<
2;
<
H , , = { x E E n : G(x, En) m}; the space E is the union of the sets H,,, and since H , , c En,we have
CH. 2, 81
SUPERHARMONIC FUNCTIONS
47
on H,, and therefore everywhere, by the maximum principle. The kernel G is thus proper, and by the first half of the proof there exists a function f > 0 such that Gf is bounded; this completes the proof.
Exercise 1.16. Prove that there is equality in Proposition 1.9, if f is bounded harmonic and S and T a r e finite. Exercise 1.16. Let X be the chain of translation on integers (P(x,* )
= E,+~(
*
),
x E 2). Compute the kernel G and characterize the harmonic functions and
potentials.
Exercise 1.17. Extend the Riesz decomposition theorem to the case where the superharmonic function takes infinite values. Exercise 1.18. A function h is said to be harmonic outside A i f f ( x ) = P f ( x ) for all x E A". Prove that Gg is harmonic outside {g > O } .
-
Exercise 1.19. If G( , E ) is finite everywhere on E , then every bounded superharmonic function is a potential. Exercise 1.20. A set A c E is said to be absorbing if P(x, A ) = 1 for all x in A . If Gg is a bounded potential, the set {Gg = 0 } is absorbing and the set {Gg > 0) is the union of a sequence of sets with bounded potentials. Exercise 1.21. Principle of lower envelope. The lower envelope of a sequence of finite potentials is a potential. Exercise 1.22. (1) Principle of uniqucness of charges. If ( f , g ) are in 8, and Gf = Gg < co, then f = g. (2) If f and g are two positive functions vanishing outside A and if Gf = Gg < CO on A , then f = g. Exercise 1.23. Let p , be the Poisson distribution of parameter m . We define inductively a sequence of random variables: X 1 is a random variable of law pl, X z is a random variable of law P x , , : . . , X , is a random variable of law $ x " - l , . . . . Prove that the point 0 is reached with probability 1. [Ezvt:This is an example of a Galton-Watson chain (ch. 1 52.20) for which the Iunction f ( n ) = n is harmonic.]
POTENTIAL THEORY
48
CH. 2, 42
Exercise 1.24. Assume that E is the set of non-negative integers and that P is given by P ( x ,x
+ 1)
P(x, x - 1) = q,,
= $,
P(x, x ) = Y,,
+ +
with qo = 0, p , > 0, 9, > 0 and q, > 0 for x =. 0, and p . q, Y , = 1 for every x in E . This chain is called a Birth and Death chain. Find all the harmonic functions for this chain.
Exercise 1.26. If G satisfies the complete maximum principle, then for every positive real a the kernel I aG satisfies the reinforced maximum principle.
+
Exercise 1.26. Let f
E
Y and set E , = (0
< f < a}.Prove that,
by setting
one defines a transition probability on (E,, E , n 8).Prove that a function g is Pf-superharmonic if and only if there is a P-superharmonic function u such that u = f g.
Exercise 1.27. Let f, g be two superharmonic functions; we say that f < g for the proper order of the cone Y if g -‘f E 9’Prove . that 9’ is a lattice for its proper order. [Hint: Prove first that the least upper bound of two harmonic functions h and h‘ is l.u.b.(h, h’) = lim P,[h v 12’3; n
then if f = Gg
+ h, f‘= Gg‘ + h’, prove that l.u.b.(f, 1’) = G[g v g’] + l.u.b.(h, h’).]
2. lteduced functions and balayage This section is devoted to the notion of reduced functions and to some applications. We give two different proofs of the balayage theorem, the first one to hint at the probabilistic aspect of the theorem, the second one of more analytical and general character. We introduce thereafter a family of kernels which will be very important in the sequel.
REDUCED FUNCTIONS
CH. 2, 92
49
Theorem 2.1 (balayage theorem). Let f be superharmonic and A a set in b. The function PAf is the smallest superharmonic function which dominates f on A . It i s called the reduced fzlnction o f f on A .
Proof. By Proposition' 1.9 we have PA/
Moreover the function PAf is
superharmonic ; indeed
<
and since TA S,, we conclude again from Proposition 1.9. Finally, let h E Y and h 2 f on A ; since PA(%,) vanishes outside A , we have
h 2 PAh 2 PAf. The proof is thus complete.
Remark. If f is a potential, PAf is also a potential, by Corollary 1.6. I t is the potential of a function vanishing outside A which is given in Exercise 2.11. Let g E 8, and define inductively a sequence {gn}fl>o of functions in b,, by setting go
z=
g,
gn = gn-1 v Pgn-1,
f i
> 1;
this sequence is clearly increasing, hence converges pointwise to a limit Rg 2 0.
Theorem 2.2. The function Rg i s the smallest superharmonic function which dominates g. Moreover, PRg = Rg on {g < Rg) and Rg = g if and only if g i s superharmonic. Proof. Passing to the limit in the definition of g, as a function of gflp1yields that Rg = Rg v PRg, which proves that Rg is superharmonic. Obviously, Rg > g , and for h e Spwith h > g , we have h Ph 3 Pg; hence h > g v Pg = g,, and, inductively, h 2 g, for every n. I t follows that h 3 Rg, which gives the first conclusion in the statement. We may also consider the sequence {g:} defined inductively by go = g,
g, = g v Pgn-1,
n b 1;
<
this sequence also increases and thus converges to a limit R'g Rg. Passing to the limit we get also R'g = g v PR'g; thus R'g is superharmonic, and since
60
CH. 2, $2
POTENTIAL THEORY
it dominates g we get Rg = R’g. Consequently Rg = g v PRg, which implies that PRg = Rg on {g < Rg}. Remark. Theorem 2.1 is, in a sense, but a particular case of Theorem 2.2. Indeed, if f E Y and A E 8, let us apply the second of the above schemes to g = 1 A f . We have
and since PIAf tively that
< Pf < f, we have g v Pg = IAcP I , f + 1
A
gl = IA f
+ 2 (IAc O
I A
f .It follows induc-
f,
and consequently that Rg = P A / .But in the particular setting of Theorem 2.1, we have a means of computing the reduced function, namely the kernel PA, which has no counterpart in the general case. Moreover Theorem 2.1 stresses the probabilistic aspect of the reduced function. The above results allow 11s to return to the Maximum Principle and give a characterization of superharmonic functions which will be useful in ch. 10. Another proof of the sufficiency is to be found in Exercise 6.12.
Theorem 2.3. Jf G i s proper, a function’ f E 8, is superharmonic if and only if it satisfies the following pro$erty: for a n y function g in d such that Gg i s well defined, the inequality Gg f holds everywhere whenever it holds on { g > O } .
<
<
Proof. Since P Af f for f E Y and A E 8, the necessity is proved in exactly the same way as was proved Theorem 1.11. Conversely, begin with a function f majorized by a finite potential; then the reduced function Rf is also majorized by this potential; hence it is a potential, namely, Rf = Gk, where k = R f - P(Rfl. But since Rf = P(Rf) on { f < Rf}, we have f = Gk on { k > O } ; hence, by the maximum property, f 3 Gk everywhere. I t follows that f = R f ; that is, f is superharmonic. Next suppose f arbitrary and let Gg be a finite potential. The function f A Gg is majorized by a finite potential and satisfies the property of the statement because f and Gg do so. Thus f A Gg is superharmonic. The kernel G being proper, we may apply this to the functions f A Gg,, where Gg, is a sequence of finite potentials converging everywhere to 03.
+
In the course of the proof of the balayage theorem, we saw that for f E .Y the function PAf is superharmonic and also that PP,f = P,,f f , in other
<
CH. 2, 92
REDUCED FUNCTIONS
61
words, that superharmonic functions are also superharmonic for the chain induced on A (ch. 1, Exercise 3.13). We are going to generalize this result, but we must first introduce a family of kernels which will be very useful in several parts of this book. We recall that the unit ball of b b is denoted by 9: thus 9,is the set of measurable functions h such that 0 h 1.
< <
Definition 2.4. With each function h of 9,. we associate the kernel u h
=
2
(PIl-h)n
P
2 P(I,-,P)n
=
n>O
n>O
where the kernel I , is defined by I f g = fg or I , ( % , * ) = ~ ( x ) E , . When h = I , we write U A instead of u h , and if h = c E [0, 11 we write U , instead of
uh.
By reasoning in a way similar to ch. 1 , Proposition 3.7, it is easily seen that, for f E b,,
These kernels will be very important in the sequel, for the following reasons. On the one hand, all useful kernels may be easily expressed as functions of the U h ' S ; for instance IIA
= IAUAIA,
=IA
P A
+
P
IAcUAIA,
=
G =I
U1,
+ Uo.
On the other hand, these kernels are very easy to handle, thanks to the following
Proposition 2.6 (resolvent equation). For h, k E Q, and h =
uh
2
(UkJk-h)n
uk
=
n>O
which im&es U h
= u k
2
Uk(Ik-hUk)n,
u k
+
n>O
+
UhIk-hUk
=
UkIk-hUh.
Proof. This is an immediate consequence of ch. 1, Exercise 1.18, which we solve rapidly below. For every integer n,
( N ( A + B))"N
=
C
((NA)n*N)B ((NA)"'N)B *
k,n,,...,nk k no ..+nk
+ +
I
=n
.B ((NA)nkN)
POTENTIAL THEORY
52
CH. 2, $2
by summing over n and interchanging the order of summations, which is possible since all the kernels are positive, we obtain
2 ( N ( A + B))" N
=
n>O
=
c
((NA)"@N) B
2
k>O no,...,nksNk+1
2
(SB)k
k>O
---
B((NA)"kN)
s.
Remark. This result says that one may compute U h as a function of u k in the same way as as a function of P = U1. We notice also that U h >, u k . Proposition 2.6. If f E 9 'and h, k are fzcnctions in Uhlhf
I n particular U,(h)
\<
Ukrkf
a+such that h < k , then
< 1 for all h in a+and U,(h) increases with h.
Proof. Let us first assume that UkIkf Uk lhf
+
< f , which implies that \<
(UkIk-h) UkIkf
UkIkf
This is, for p = 1, what the inequality
reduces to. Assume this inequality to be true up to p , and multiply through to the left by U k 1 k - h ; then adding through UkI,&fyields
<
uklhf
f
Uklk-hf
= Ukzkf
Thus eq. (2.1) is true for all p . If we let p tend to infinity in eq. (2.1), i t follows by the resolvent equation of Proposition 2.6, that
< = Pf <
Uhlhf
Ukzkf
df.
Now if k = 1, then U k I k f f , and by the preceding discussion UhIhf f for all h E 4,. It follows that the hypothesis with which we started the proof always holds, and consequently the proof is complete.
<
If we had started with f harmonic and positive, since the term lim(U,Ik-h)p UkIkf is not bound to vanish (cf. below), we would still only get
CH. 2, 52
REDUCED FUNCTIONS
63
an inequality in the above result. In the same way PAf = f is not always true, as is seen from the example of the function I , when PI = I and PAI is not equal to I . We have, however,
Lemma 2.7. If h i s harmonic and positive, and if U,I,,h = P,h = h.
Proof. The resolvent equation, U A = P UAh = h
< 00, then PAh
+ UAIAeP, yields
+ UAIAch,
and by substracting UAIAehfrom both members we get
P,h
= UAIAh= h,
We shall now use the reduced functions to give a characterization of potentials. We assume G to be proper.
Proposition 2.8. A finite superharmonic function f i s a potential if and only if lim,P,,f = 0 for every decreasing sequence of sets in d such that n A n =O.
<
Proof. Let f be a potential; since TAn TAn+l,Proposition 1.9 shows that the sequence g, = PAnf decreases; let g denote its limit. Since by Theorem 2.1, Pg, = g, on A : , an application of Lebesgue’s theorem yields Pg = g. By Corollary 1.6, it follows that g vanishes identically. Conversely, we suppose that f is not a potential and we call h the harmonic part of its Riesz decomposition. Let En be an increasing sequence of sets En and G ( x , En)< co for every integer 1z and each x in E . such that E = Then set A: = En n {h n}; we have UA,,IA:h UoIAC;h n G < 03. Therefore PAnh = h for every n, and it follows that limn PAnf 2 limnPA,h = h # 0 although A,, = 0.
u
<
<
<
n
Exercise 2.8. Let f be a finite superharmonic function. For every x in E , the set function L defined on d by L ( A ) = P Af ( x ) is strongly sub-additive, that is L ( A U B ) L(A n B ) L(A) L(B).
+
<
[Hint: Use the results in ch. 1, Exercise 3.9.1
+
CH. 2, $2
POTENTIAL THEORY
54
Exerciso 2.9. Let f be superharmonic and finite and {S,} a sequence of stopping times decreasing to the stopping time S; prove that
P,f
=
lim P,,,f. n
In particular, if { A , } is an increasing sequence of sets with union A , then limn PA,1 = PA1. What can be said if { A , } is decreasing?
<
Exereisc 2.10. (1) Let f be bounded from below and such that P f f on the set A". Prove that the sequence { f ( X T A A , )of} random variables is a supermartingale for every P, and derive therefrom that f 2 PAf. [Hint:Use the stopped chain of ch. 1, Exercise 3.15. The inequality f 3 PAf may actually be shown directly.] (2) Assume further that PAI = 1 and prove the following Minimum principle :
f 2 inf{f(x), x
E A}.
Excrcisc 2.11. For A E 8, we define a kernel H A by setting for f s* -1 H A
f b )=
Ez
(1) Prove that H A
]
2 f(xJ
[n=o
= I A C (PIAc)" = I ,
E
8,
*
+ IAUAIAC.
n>O
In particular H A f vanishes outside A . ( 2 ) Prove that GNA = G - G A = PAG (cf. Proposition 1-10),and derive that if Gg is a potential, then PAGg is the potential of the function HAg which vanishes outside A (the reader will see in the next section other interpretations of H A ) .
Exercise 2.12 (Dirichlet's problem). Let A E 8 and f E &+; prove that PAf is the smallest function equal to f on A and harmonic outside A . Exercise 2.13. For h E @+, show that
Derive therefrom that U,(h)
< 1 and that
U,(h) increases with h. If P is
EQUILIBRIUM
CH. 2, 93
66
markovian and h is bounded away from zero, then U,(h) = 1. This last result may fail to be true if one of the two conditions is not verified; give examples.
Exercise 2.14. For g in b,, prove that the sequence {Rg(X,)} is for every P,, the smallest supermartingale which majorizes the sequence {g(X,)}. Exercise 2.16. Let N be a kernel on ( E , 8)satisfying the reinforced maximum principle and let .?8 be the space of bounded functions f such that Nlfl is bounded. (1) If h E 9 is such that N h 3 0, then N h 3 h. Derive that Nf = Nf'with implies f = f'. f, f' E (2) Prove that N satisfies the complete maximum principle for the functions in 8+.(This exercise is solved in ch. 10.) Exercise 2.16. Prove that the kernel U o satisfies the complete maximum principle if it is proper, and derive (cf. Exercise 1.25) that G satisfies the reinforced maximum principle. Exercise 2.17. ( 1 ) Let L be a death time (ch. 1, Exercise 3.15). Prove that p(x) = P,[L > 01 is superharmonic. Express its Riesz decomposition. Under what conditions is y~ a potential? In particular if L = L A , what condition must A satisfy in order that q~ be a potential ? (2) For every probability measure P , on Q, the sequence { Y,}n20defined by
Y , ( w ) = X , ( w ) if n
< L(w),
if n
3L(o),
Y,(o)
=A
is a homogeneous Markov chain with transition probability PO (notation of Exercise 1.26) and initial measure p Y . (Notice that this measure may have a total mass less than one.)
Exercise 2.18. Show that 9 'is a lattice for the ordinary order. [Hint: For the 1.u.b. use reduced functions.] 3. Equilibrium, invariant events and transiont sets
The following definition is basic.
CH. 2, $3
POTENTIAL THEORY
56
r.
Definition 3.1. An event r e 9 is called invariant if O-l(r) = The class of invariant events is a a-algebra denoted f.A random variable 2 is 9measurable if and only if 2 0 0 = 2 ; such a random variable is called invariant. Two invariant random variables 2 and 2' are said to be equivalent if P,[Z # 2'1 = 0 for every probability measure Y. Proposition 3.2. If P i s markovian, the formula h(x) = EAZl
sets up a one-to-one and onto correspondence between equivalence classes of bounded invariant random variables and bounded harmonic functions. Moreover lim h(X,) = 2 a s . n
Proof. If 2 is invariant, the function E.[Z]is harmonic, as is seen by a straightforward application of the Markov property. Conversely, if h is bounded and harmonic, then { h ( X n ) }converges almost-surely to the random variable 2 = h(X,) which is obviously invariant. By Lebesgue's theorem we have
4%) = EJZI for every x in E. The correspondence is clearly one-to-one. If P i s not markovian, the function E . [ q is superharmonic but not harmonic at points x such that P%[(= 13 > 0. The reader will look at Exercise 3.10 for what can be said in this case.
Definition 3.3. For A
E
8, set W
{n
+ s,
en < a}.
The event R ( A ) is the set of trajectories w which hit A infinitely often; it is clearly an invariant event. We further set h, = P . [ R ( A ) ] ;this function is bounded and harmonic and hA(x) is the probability that the chain started at x hits A infinitely many times. This function has the following potential theoretic interpretation.
Definition 3.4. Let A be in 8 ;the superharmonic function PAI is called the equilibrium potential of A .
EQUILIBRIUM
CH. 2, 93
57
This superharmonic function is actually a potential only in some “classical” cases, as is seen from the following result, which specifies its Riesz decomposition.
Proposition 3.5. For any set A i n 8,we have
where g A is the Probability that the chain never returns to A for x 6 A and gA(x) = P,[S, = CO] for x E A ) . Moreover lim P,,[S,
< CO]
= lim P,l(X,)
n
=
n
lim hA(Xn)= 1,(,,
(gA(X) =
0 for
U.S.
n
Proof. By definition, P A l ( x ) = P,[T, < CO]. By the Riesz decomposition theorem, 1.6, we have P A l ( x ) = G g A ( x )+ hA(x),where h, is harmonic and equal to limn P , P A l ( x ) . But by ch. 1, Proposition 3.8, P , PA I ( x )= P , [ n
+ T Ao 8, <
CO]
=
P,
U {X, E A} . [mIn
]
Consequently,
which is the probability that X hits A infinitely many times. By Theorem 1.4, we have also
By ch. 1, Proposition 3.8 it follows that g, vanishes on A”, and is equal to I - P,[SA < 031 = P,[S, = CO] on A . The second sentence follows a t once from Proposition 1.8 and Proposition 3.2 and the fact that P,,[S, < 031 = Px,[TA < CO] a.s. for n 2 1.
Definition 3.6. A set A in 8 will be called transient if P A l is a potential, hence if h, E 0. A set A is called recurrent if hA I. A subset of a transient set is a transient set and a finite union of transient sets is transient. Moreover we have the following characterization of transient sets in terms of potentials.
68
CH. 2, 93
POTENTIAL THEORY
Proposition 3.7. A set A i s transient if and only if there exists a function g vanishing outside A and such that Gg 3 1 on A and Gg i s finite everywhere on E. I n $articular, if G( * , A ) i s everywhere finite then A i s transient. Proof. The “only if” part is obvious by the foregoing result. For the “if” part, we observe that since Gg is finite we have lim Gg(X,) = 0 a s and therefore lim 7,(Xn) = 0 which is the desired result. This result permits us to understand the probabilistic significance of the condition “G is proper”; it means that E is the union of an increasing sequence of transient sets. It is essential t o point out that a set may be neither recurrent nor transient. If P is not markovian no set is recurrent, not even the set E.If Pis markovian we have the following result which describes an interesting situation occurring for large classes of random walks as well as for other chains studied later on.
Proposition 3.8. If P i s markovian, the following three statements are equivalent : (i) The bounded harmonic functions are constant; (ii) The a-algebra 9i s trivial (u$ to equivalence) ; (iii) Every set i s either recurrent or transient. Proof. The equivalcnce of (i) and (ii) is an immediate consequence of Proposition 3.2. Since, in addition, R ( A ) is in 9, these conditions imply (iii). It remains to show that (iii) implies (ii). Let r g 9 and put A = { x : Pz[r] > a}, where a €10, l[. We know that 7, = limn P,,[r] a s . ; if A is recurrent, then = !2 a s . , and if A is transient, then = 0 a s .
r
r
Exercise 3.9. Prove that a set A is recurrent if and only if U A ( 7 , ) = 1 everywhere on E. Prove that if A is not transient, there are points x for which Iz,(x) is arbitrarily close to 1. Prove the even better result: if h, a < 1 on A , then h, G 0.
<
Exercise 3.10. (1) If I’= {C = co}, prove that r a O-l(r)= 0 a.s. (2) If P is not markovian there is a one-to-one correspondence between bounded harmonic functions It and equivalence claases of bounded invariant random variables 2 which vanish on r”.This correspondence is given by 2 = lim h(X,), n
h ( x ) = E,[Z].
EQUILIBRIUM
CH. 2, 53
In particular if function.
69
< < w as., the only bounded harmonic function is the zero
Exercise 3.11. (1) For A
Ed
and every integer n prove that
- pnpA
=
2
pm(lA
-U A ) ,
m
and derive that if A is transient then P A = G ( I A - U A ) . ( 2 ) If f is superharmonic and A is transient, the superharmonic function PAf is a potential. If G is proper and ( A n }is an increasing sequence of transient sets whose union is E , then f = limn PAnf.
r
Exercise 3.12. (1) Pick a €10, 1[ and set A = { x : P J r ] > a}, where is an invariant event. Prove that = lim{X, E A } a s ( 2 ) A set A E 8 is called regular if lim{X, E A } exists almost surely. Prove that the collection C of regular sets is a sub-algebra of 8.Transient sets are regular, and the class of transient sets is an ideal in Z. (3) Two regular sets A and A’ are said to be equivalent if A A A’ is transient. If A and A’ are equivalent, then P,[R(A)] = P,[R(A’)]for all x in E. [Hint: Prove that R ( A ) A R ( A ’ )C R ( A A A ’ ) . ] (4) Prove that the Boolean algebras of equivalence classes of regular sets and invariant events are isomorphic. [Hint:Associate with A the event lim{X, E A } . ] (5) Prove that if f E b b is such that lim f(X,) exists a s . if and only if f is the uniform limit of simple functions on Z. This exercise is solved in ch. 7 $1.
r
Exercise 3.13. (1) Let m be a positive measure on ( E , 8).The following two statements are equivalent : (i) If r E 9 , then either P v [ r ]= 1 for every v << m or P J r ] = 0 for every v << m ; (ii) If A E 8,then either P.[R(A)]= 1 m-a.e. or P.[R(A)]= 0 m-a.e. ( 2 ) These conditions imply: (iii) the bounded harmonic functions are constant m-a.e. Prove by an example that the converse may fail to be true. Prove that the converse is true if m satisfies the following condition: if m(A”)= 0, then P , [ R ( A ) ]= 1 for all Y << m.
POTENTIAL THEORY
60
CH. 2, $4
4. Invariant and excessive meastires
In the last three sections, P was viewed as operating on the right on functions. We are now going to let it operate on the left on measures and sketch a potential theory relative to measures. We thereafter introduce the important notion of Markov chains in duality. Definition 4.1. A a-finite measure nz is said to be invariant by P , or P-invariant, if m P = m. We shall also say that nz is invariant for the chain X , or simply invariant if there is no risk of confusion. Examples. The left (right) Haar measure on a group G is invariant for the left (right) random walks on G . If P is the T.P. defined in ch. 1, Example 1.4(iii), the measure m is P-invariant if and only if 8 ( m ) = m, namely if m is invariant with respect to 8, in which case 8 is said to be a measure-preserving point transformation of ( E , 8,m). The probabilistic significance of an invariant measure is the following : even if the positive measure m is unbounded, one can define a measure P, on (Q, F) by setting
P m [ r ]=
I.
P z [ r ] dm(x).
If the measure m is invariant, the chain X is then stationary for the measure P,: for all integers n,, n2,. , . , n, and n and sets A , , . . . , A , E 8,
which may also be expressed by saying that P, is invariant with respect to the shift operator 0. Definition 4.2. A positive a-finite measure m is said to be P-excessive if mP in. We also say that m is excessive for X , or simply excessive when there is no fear of mistake.
<
Let Y be a positive measure such that the measure VG is a-finite; we then say that v is a charge and VG is an excessive measure called the potential measure of the charge v. If G is proper, all bounded measures are charges. If m is excessive (invariant) then awL, where a € R,, is still excessive (invariant). In the sequel, when we state that an excessive (invariant) meas-
CH. 2, $4
INVARIANT AND EXCESSIVE MEASURES
61
ure is unique, we mean unique up to multiplication. Moreover the sum of two excessive (invariant) measures is an excessive (invariant) measure; thus the set of excessive (invariant) measures is a convex cone. As for superharmonic functions, one can state a Riesz decomposition theorem.
Proposition 4.3. Every excessive measure may be uniquely written as the sum of an invariant measure and of a potential measure.
Proof. If f E bB,, the sequence of numbers mP,(f) decreases to a limit ~ ( f ) which defines an invariant measure, as is easily seen. The proof then follows the same pattern as for functions. As in the case of functions, we shall see that the restriction to a set A of an excessive measure is excessive for the chain induced on A , namely mP m implies ( I A m )ITA lAm, or equivalently mIAUAIA mIA. This is shown by the following.
<
<
<
Proposition 4.4. If m i s excessive and h, k are in 9,and such that h mIhUh
< k , then
< mIkuk < m.
The proof is the same as for Proposition 2.6 with right multiplicatipn instead of left multiplications. As in the case of functions, if m is invariant, l A mis not bound to be UA-invariant (cf. Exercise 4.11). I n the next chapter, we shall find sufficient conditions implying this result. Let us now prove the result analogous t o Theorem 2.1.
Theorem 4.5 (balayage theorem). Let m be an excessive measure. The measure mHA,where HA = I , IAUAIAC, zs the smallest excessive measure which dominates m on A .
+
Proof. Clearly v ~ H A= m on A , and by Proposition 4.4, mHA by the resolvent equation of Proposition 2.5, we have
But Proposition 4.4 implies that
< m ;moreover,
POTENTIAL THEORY
62
CH. 2, $4
<
so that m H A P mH,, which proves that mH, is excessive. Finally if m’ is excessive and m’IA M I A , then m‘ 2 m’HA the proof is completed.
mH, and
The kernel H A was introduced and studied in Exercise 2.11. It is called the co-balayage operator relative to A , because it acts with respect t o measures as P A acts with respect to functions, as one can see by comparing on the one hand Theorems 2.1 and 4.5 and on the other hand Exercises 2.11 and 4.12. This duality between the operators P A and H A appears in an even more striking form under the “duality hypothesis” that we are going to introduce now.
Definition 4.6. Two kernels N and N on ( E , 8)are said to be in duality relative to the positive a-finite measure m, if for every pair (f, g) of functions in &+,
which will also be written (Nf, g)m = ( f , fig),, or even (Nf, g) = (f, fig) when the relevant measure m cannot be mistaken. Two Markov chains X and fz are said to be in duality relative to m if their T.P.’s are in duality relative to m. One of the two chains is said to be the dual or reversed chain of the other one. The objects defined in terms of will be designated by the symbol and the prefix “co-”. For example, a function co-superharmonic is a function which is superharmonic for P , the T.P. of fz, and the set of co-superharmonic functions i s denoted by 9’. A
Some easy remarks and examples are collected in Exercise 4.14. I n particular, by making g = 1 ( f = 1 ) in the definition it is seen that m is excessive (co-excessive). More generally, we must emphasize the duality relationship between superharmonic functions and co-excessive measures. The reader will notice that, without any special hypothesis, there may be several T.P.’s in duality with a given T.P. Their relationship is settled in the following. Lemma 4.7. If ( E , 8)i s seParable and if each of the two transition Probabilities and p’ i s in duality with P relative to m, then there i s a n m-negligible set N such that p(x,. ) = p’(x,* ) /or all x in No.
P
CII. 2, $4
INVARIANT AND EXCESSIVE MEASURES
63
Proof. The duality hypothesis implies at once that for every f E bb,, we have Pf = P'f m-a.e. We can therefore find an m-negligible set N such t h a t if x is in No, then P(x, A,) = P'(x, A,) for every set A , in a couptable algebra generating 8,hence p(x, ) = fj'(~, ).
.
Definition 4.8. Any T.P. which is in duality with P relative to m is called a modification of P. Proposition 4.9. Let X and be in duality relative to m; then for every h E @+ the kernels U h and o h aye in duality relative to m. Proof. One must prove that, for f , g E bQ,,
but this follows from repeated applications of the formula
which in turn is an obvious consequence of the duality formula of Definition 4.6.
Corollary 4.10. For A E &', the kernels H A and 17, and ITA aye in duality.
P , and the transition probabilities
Proof. Since the operator I , is its own dual, the corollary follows a t once from Proposition 4.9 and the expression of the kernels as functions of U A . For example, (HA/,
g>m = =
(IAfr
g>m
( f . zAg)m
+ (IAUAZACf,g ) m + (1, = (f, zAcoAzAg)m
PAg>ms
Exercise 4.11. Prove by an example that in Proposition 4.4, one may have mI?, < 7 A m,even if m is invariant. [ H i n t : One can use the chain of translation on integers.] Exerciso 4.12 (potential theoretic properties). (1) If such that LG = vG,then il = v.
A
and v are two charges
POTENTIAL THEORY
64
CH. 2, 54
(2) If Y is a charge, the measure v’ = vP, is the unique measure carried by A and such that: (i) v’G = VG on A , (ii) v’G ,< VG everywhere. (3) If v is carried by A and VG AG on A , then VG ilG everywhere. [Hint: Use the operator HA.]
<
<
Exercise 4.13 (capacities). (1) For A E 8,prove that
( I - PsA)GIA = 1,. (2) Let m be an excessive measure for P and f , g two functions in 8, vanishing outside A and such that Gf Gg on A . Prove that
<
5 5 fdm<
gdm.
(3) Prove that for A , BE&’, we have
rHiizt: See Exercise 2.8.1 (4) We call capacity (relative to m) of the transient set A the number C(A)=
I
P,[SA
= CO]
m(dx).
A
Prove that C ( A ) sets we have
< C(B)if A c B, and that for any pair ( A , 13) of transient C ( A u B ) + C ( A n B ) < C ( A ) + C(B).
(5) Let dLp be the subset of 8, of functions g vanishing outside A and such that Gg ,< I on A . Prove that
C ( A ) = SUP g&
1
gdm.
A
( 6 ) Assume furthermore that X is in duality with relative to m, which is then also co-excessive, and prove that if A is transient for both chains then C ( A ) = c ( A ) .
Exercise 4.14. Let P and P be two T.P.’s in duality relative to the measure m. (1) Prove that m is both excessive and co-excessive. If E is countable and Q discrete, then the empty set is the only set of m-measure zero; the duality hypothesis is equivalent to the requirement that
CH. 2, 94
INVARIANT AND EXCESSIVE MEASURES
66
4%) P(x, Y) = m(r) %, 4 for every pair ( x , y ) of points in E. (2) A right (left) random walk of law ,u on a group G in duality relative to the right (left) Haar measure on G with the right (left) random walk of law b. where 1; is the image of ,u by the map x -* x - l . (3) If f E 9, the measure f m is co-excessive. If f is harmonic, the measure f m is co-invariant.
Exercise 4.16. (1) If m is P-excessive then P maps the functions m-a.e. bounded into functions m-a.e. bounded, and thus induces a positive contraction on Lm(E,8,m). [Hint:Use the weaker condition mP << m. Solution in ch. 4, Proposition 1.3.1 (2) Solve the same problem with L1(m)instead of Lm(m).By using Holder's inequality for the measures P(x, ), prove that for every9 E [l, co] the transition probability P induces on Lp(m)a positive contraction (one may also use the Riesz convexity theorem). If P and P are in duality relative to m, the operators induced by P on L1(m)and by P on Lm(m)are adjoint operators. Exorcise 4.16. Let P and P be in duality relative to m and let f be finite and P-superharmonic. Then, with the notation of Exercise 1.26, P f is in duality with P relative to f m. Exercise 4.17. (1) If P is in duality with itself with respect t o m, that is if
S
I
Pfgdm = fPgdm
for every pair (f,g) of functions in d,, then P induces on L2(m)a self-adjoint operator of norm less than or equal to one (see Exercise 4.15 above). There exists therefore a resolution of the identity E(d1) in L2(m)such that
P,
=
5
+1
-1
In particular, if E is countable,
1" E(d1).
I
+1
m(x)Pn(x, Yj =
l n
Pz,w(d~+),
-1
where ,uz,wis the measure defined by ,us,,([- 1, 1[) = ( E [ - 1, I[y,x ) . (In the last formula x and y stand for the elements I{.,) and of L2(m).)
POTENTIAL THEORY
66
CH. 2, 54
(2) The space E still being countable, it is assumed that G(x, y ) <= every pair ( x , y ) of points in E. Prove that the infinite matrix
03
for
is positive definite. Derive that the space Y of functions with finite supports may be endowed with a structure of pre-Hilbert space, by setting (fl
=
2 r(% Y ) f ( 4 g(Y) = (1, Gg),. X#Y
Let VAbe the subspace of V of functions with support in A ; prove that the orthogonal projector projA on V Asatisfies the relation PO ' jA
f
= PAf =
(f 4 H A / %
where the last term is the Radon-Nikodym derivative of ( f m )H A with respect to m. P,[SA = a1 set C ( A ) = m(gA).(See Exercise 4.13(4).) (3) If g A ( x ) = Prove that on the set of functions g of Y Asuch that m(g) = 1, the quantity (g, g) reaches its infimum C(A)-I for the unique function g A / C ( A ) .
Exercise 4.18. (1) Let M be a homogeneous space of a group G admitting a measure v invariant by G, that is E, * v = v for all g E G. Prove that v is invariant for all random walks induced on M.If M is compact, every random walk on M induced by a random walk of law ,u on G has at least one invariant probability measure [Hint: A cluster point of the sequence ~ ~ * v'. ~ where v' is any probability measure on MI and the set of invariant probability measures is a vaguely compact convex set. (2) If v is an invariant'probability measure for the induced random walk, then for every function ~1 E Loo(v),the function
is p-harmonic.
Exercise 4.19 (Continuation of Exercise 1.24). (1) Prove that there exists one and only one excessive measure m such that P is in duality with itself with respect to m. Compute this measure and find under which condition it is an invariant measure. (2) Using the notation and results of Exercise 4.17, prove that there are
~
CH. 2, $5
RANDOMIZED STOPPING TIMES
67
polynomial functions Qk(,?)and a unique measure p such that for every pair ( k , 1) of integers, PCCX.dd4 =
Qd4Qd4 P ( W
and therefore
m(k)Pn(k, 1) =
j:
J~QAI) ~ ~ (AW. 2 )
Exercise 4.20. (1) If E is an LCCB and P maps the functions of C , into continuous functions, every limit in the vague topology of excessive Radon measures is itself an excessive measure. ( 2 ) If P is strong Feller, every excessive measure is a Radon measure. 6. Randomized stopping times and filling scheme This section may be skipped without hampering the understanding of the sequel. It aims at giving another insight into the connections between Potential theory and the probabilistic and ergodic properties of Markov chains. In particular, we are going to describe a procedure of construction of reduced measures thus furthering the results in 9$2 and 4. This procedure, known as the filling scheme has often been used in the theory of Markov chains and, although we shall not do so, we could use it to derive some of the results in the following chapters; some exercises in this and some further sections will present the alternative methods based on the filling scheme. We begin with a generalization of the notion of stopping time.
Definition 6.1. A randomized stopping time T (abbreviated R.S.time T ) of the canonical Markov chain X is a map from 0 = 10, I] x Q to N U {+ a) such that (i) for each u E 10, 13, the map T(zt,* ) is a stopping time; (ii) for each o E Q, the map T ( , o)is decreasing and left continuous. Notice that an ordinary stopping time T may be turned into a R.S.time by setting T(u, ) = T for each u. Let U be the random variable defined on by U(u,w) = u and set Sn = a(X,, . . ., X n ,U ) , 3 ' = a(UrSn).We leave to the reader the task of showing thataR.S.timeisastoppingtimerelativetothefami1ySn,that is {T = ~}ES,, for every n . For every v on ( E , 6) we now define a probability measure p , on (Q, 9)by setting P , = m @I P , where m is the Lebesgue measure on
-
CH. 2, $5
POTENTIAL THEORY
68
30, 13. We will still use X , for the map (21, o)--f X, (o) and X for the collection of all X,’S. For P,, the random variable U is independent of X and X is still a homogeneous Markov chain with transition probability P with respect to the a-algebras 9, (see ch. 1, Exercise 2.18). We have thus enlarged the probability space on which X is defined and a R.S.time is a stopping time in the enlarged setting. As for ordinary stopping times we may define PT by setting, for f E Q,,
where T,, = T(u,* ). A R.S.time thus appears as a “mixture” of ordinary stopping times. From Proposition 1.9 it is now clear that if f is superharmonic, then PTf f , a fact which will be important below.
<
Definition 6.2. A balayage sequence is a sequence (r,, s,), n 2 0, of pairs of bounded measures such that for every n > 0, r,
+
sn =
Sn-lP.
+
The measure ro so is called the initial measure of the sequence and its terminal measure.
c: r ,
With a R.S.time T and a starting measure ilwe may associate a balayage sequence by setting for A E 8 and n 2 0,
r,(A) = P,[(X, E A ) n (T = n ) ] ,
sn(A) = P , [ ( X , E A ) n (T > n ) ] .
If we think of il as a mass moved by the chain and stopped at time T , rn is the mass stopped at time n, whereas s , is the mass still moving at time n. Conversely we have
Proposition 6.3. For any balayage sequence (r,, that r, = ( y o so)PT..
s,)
there i s a R.S.time T such
+ Proof. Set il = ro + so and for every n define a, to be a function in Q, 2:
such that s , = a,(r, + s,); such a function exists by the Radon-Nikodym Theorem. We now define a sequence {M,} of random variables on by M n =
~ X Oal(X1) ) *
* *
an(xn).
This sequence is decreasing, bounded by 1 and M ,
E
9, for every n. We
CH.2, $5
RANDOMIZED STOPPING TIMES
69
clearly define a R.S.time by
-
T(u, )
=
inf{n: M ,
< zt}
+
We thus have proved that sk = a,(ri s:) for every n 0. Since YO = rb sh = 1, it follows that s g = s i hence y o = rh and inductively that r , = ri s: hence s, = s i and in turn r , = 7:. The proof is complete.
+ +
+ + s,
Sg
The sequence {a,} exhibited in the previous proof allows us to give an analytical expression for P , which generalizes that of PA.
Proposition 6.4. For any f aP,f
=
E
wl
2
f dr,
0
8,, W
=
2 ~I,,PI,,PI,~P..
PI,,,-~~I~-,,~.
0
Proof. The first equality is clear. To prove the second we shall compute r , as a function of P and the an's.By the definition of a, we have s, = a,(s,-,P) = S,,-~PI,,; it follows that
r , = S,-lP - s,
=
S,-lP - sn-lPIan = sn-lPI1-,,.
From these formulas we easily derive that Y,
=
s,Pl,,P-
- - PIan-lPIl-an
and so = l I a owhence the result follows.
CH. 2, $ 5
POTENTIAL THEORY
70
Remark. The above reasonings may be performed with any sequence {a,} of functions of @+ and any initial measure A. With a constant sequence we get some of the operators previously described; more specifically if a” = 1, M , = 1 - h for n 3 1 , we get P , = U,I, and for 01, = I A c , n 0, we get Pk
>
It is clear that for a given starting measure A, different R.S.times may have the same balayage sequence. In particular, if in Proposition 5.3 we start with the sequence associated with a time 1 , the time constructed is not necessarily equal to T. But all these times have the same measure APT,. We shall now turn to the converse problem: two probability measures A and v being given, under what condition does there exist a R.S.tinie T such that v = APT or equivalently a balayage sequence with A and v as initial and terminal measures ? If T is a R.S.time we have already pointed out that P T f f for any superharmonic function f . If v = APT,we therefore get A(/) v ( f ) for every f E Y . This necessary condition turns out to be also sufficient. Two bounded positive measures (A,v) being given we shall now describe a particular fashion of constructing a balayage sequence which will enable us to solve the above problem. We define inductively (A,, v,) by
>
A”
=
(A - v)+,
A n = (An-lp
x:
- vn-I)+,
<
Yo =
(A - v)-,
V,
(An-lP - v,-,)-.
Cf
We also put a, = I , and a = I,. The following interpretation explains the title of this section: v is viewed as a “hole” and a heap of dirt is dumped with distribution A ; A. is the part of the dirt which does not fall in the hole whereas vo is the remaining hole. The dirt A. is then moved with the random “shovel” P and A1 is again this part of the dirt which does not fall in the remaining hole, and so on and so forth. We shall find the conditions under which tlie hole is entirely filled.
Lcmma 6.6. (i) For every n, we have - v,+~ = AnP - v,; (ii) for every n, the measures A, and p, are mutually singular; (iii) the sequence {v,} decreases to a limit v, and if we define ro = v - yo, rn = v,-~ - v, for n > 0, the sequence (r,, A,) i s a balayage sequence with initial measure 1. (iv) for every n, A,, AnP, so that for a n y sufierhurnzonic function f , the sequence of numbers { A n ( / ) } i s decreasing.
<
RANDOMIZED STOPPING TIMES
CH. 2, $5
71
Proof. Straightforward. To say that the hole is filled is to say that v, = 0; in any case there is a R.S.time 1 associated with (r,, A,) which is such that APT = Y, = v - v,. As already observed we thus have A ( f ) 3 (v - v, f)for every superharmonic function f . Let us denote b y b Y the set of bounded superharmonic functions and write v < L if v(f) I ( f )for every f E b Y ; with this notation v - v, < A. We moreover have the following
2:
<
Proposition 6.6. Let A E &' be such that v,(AC) = 0 and & ( A ) then v, = (v - A) P A ,and for f E b,Y, v,(f)
=
sup{(v - 1,g ) ; g E b y , g
Proof. Since & ( A ) = 0, we have L,PPA ( A n + l - vn+1)
PA
=
=
=
0 for every n,
< 1).
&PA for every n, and therefore
(1, - vn) PA
=
.
* *
Z r
(A - v) PA.
(5.1)
On the other hand, it follows from Lemma 5.5(iv), that forg E bb+,
<
and inductively An(PAg) A((PIAC), P,g) which goes t o zero as n tends to infinity by the analytical expression of P,g (ch. 1 , Proposition 3.7). Passing to the limit in eq. (5.1) we thus get the first equality of the statement. To prove the second, observe that if g f then
<
+ (v -
< (vw,g ) by the remark preceding the statement, hence (v - A , g ) < (v,, (v - L,g )
=
(% g )
y,
- A,g )
1). On the other hand, the equality obtains for g = PAf, which completes the proof.
Theorem 6.7. The following three statements are equivalent: (i) v, = 0; (ii) there exists a R.S.time T such that v = APT; (iii) v < A.
2;
Proof. If v, = 0, then v = Y , = APT where T is the time associated with (r,, A,) by Proposition 5.3. Thus (i) implies (ii). The fact that (ii) implies (iii) was noted earlier. Finally, to prove that (iii) implies (i), we use the first formula in Proposition 5.6. Since PAY E b Y , (iii) implies that v,(l) 0 which entails that v, = 0.
<
CH. 2, $5
POTENTIAL T H E O R Y
72
Let us observe that there is no uniqueness property for the statement (ii) above; in other words there may exist several R.S.times T such that v = APT. If G is proper these stopping times have a common mathematical expectation; indeed if 9?+is the set of functions f E a+such that Gf is bounded
(A -
Gf
=
EA
r$f(xn)]
for any ~ E Band + therefore E,[T] = sup{(A - v) G f ;f ~ 9 ? + } When . this expectation is finite it is natural to ask if among these R S t i m e s there is one with minimal variance or more generally such that ~!?,[cp(T)] is minimum for a convex function q~ on N. This will be tackled in Exercise 5.15. With G still proper the filling scheme also provides a method for constructing reduced measures.
Proposition 5.8. lf G i s $roper, the measure v,G is the smallest excessive measure larger than u = (v - A) G. Moreover a = v,G - a and in particular if v .< 1, the measure a is a positive solution of the Poisson equation a = UP ( A - v ) .
+
+
Proof. We may write u = (vm ( Y - Y,) - A) G and we have already observed before Proposition 5.6 that (v - vm) G LG when G is proper, so that u v,G. Let now 5 be an excessive measure such that u 5 Y,G. By the Riesz decomposition theorem, there is a bounded measure 8 such that 5 = OG and clearly ( v - A, f ) ( 0 , f ) for f E b y . If now A is a set as in Proposition 5.6, we have for f E b Y ,
.<
<
< <
<
( v - 1,P A f ) d (0, P A f )
<
( 0 9
f ) ?
<
and therefore Y , < 8; consequently v,G OG,hence v, = fl which completes the proof of the first sentence. To prove the second, notice that the equation An - ifn= An-lP - Y , - ~ entails upon summation that an+] = U n P
+ (1- + vn+1; Y)
passing to the limit yields a=
UP + ( A - v) + v,,
and applying G through t o the right of this equality, we get u=
(A - V) G
+ v,G.
The second part of the statement is now obvious.
CH.2, $5
RANDOMIZED STOPPING TIMES
73
When G is not proper the above reasonings break down. These topics will be tak& up in ch. 6, Exercise 6.10 in the case when G is no longer proper. The reader may already solve Exercise 5.11 in this section.
Exercise 6.9. In the setting of Proposition 5.4 prove that the following three properties are equivalent : (i) T is p,-a.s.finite; (ii) lim, s,(E) = 0 ; (iii) limn M , = 0 P A - a s . Excrcisc 6.10. With the notation of Proposition 5.4 prove that the measure u= s, is equal to AIaoPIa,* * PI^,.
2:
C:
-
Exercise 5.11. If v < A and if there is a positive a-finite solution 6 to the Poisson equation 6 = [P A - 11, then the measure a provided by the filling scheme is the minimal positive a-finite solution. This is true even if G is not proper. [Hint:Prove inductively that a, ,< 6 for every 12.3
+
Exercise 6.12. If G is proper, the filling scheme (A,, vn) for (A, v) may be defined inductively by the conditions (i) A, A v, = 0 ; (ii) ( y o - Ao) G = (v - A) G ; (iii) (v,+~ - A,,,) G = (v, - A,) GP v (v, - A,) G. [Hint:Use a set A , such that A,(Ai) = v ( A , ) = 0 and prove that the right-hand side of (iii) is equal to (vn - A,P) G.] Exercise 6.13. A function g E 8, is called subharmonic if Pg 3 g. If v(1) = A(1) and v = APT where T is associated with (A, v) by the filling scheme, then for all v-integrable subharmonic functions g the following three statements are equivalent : (i) v,(g) 2 A&) for every n ; (ii) limn A,(g) = 0 ; (iii) for P,, the submartingale { g ( X T A,)} is equi-integrable.
Exercise 6.14. In the general situation of the filling scheme prove that the following two conditions are equivalent : (i) lim, &(I) = 0, in other words the “dirt” is entirely used u p ; (ii) A(g) v(g) for every bounded subharmonic function g.
<
74
CH. 2, $6
POTENTIAL THEORY
Exercise 5.15. Let (1,v) be such that 1G 2 vG where G is proper. (1) Prove that one defines unambiguously a balayage sequence fin, t i ) by setting
+ th, t i = (t, + t i ) (tYG - vG) where t:' = t, + t , + - + t, + t i . In particular Q G 3 VG for each n. 1 = 1,
A
* *
[Hint: For the last fact use the equality (,+:,,G = t y G - ti.] (2) Prove that t, A ta = 0 for m > n. The sequence (r,, A,) of the filling scheme has the opposite property: r , A A,,, = 0 for n > m. ,, [Hint: Prove that t , A (t,,+,G - vG) = 0 andwrite t, A t; = t, A (t, - t,,,) G.] (3) Prove that rk tk [Hint: Proceed by induction, using the formula rk = (1, rk) - v)+.] and derive that r?G 2 C G with r, = Y, rl * * r , 1,. Then show that (tn, 1:) has terminal measure v. (4) Let y be a positive function on N such that p(n + 1) - y ( n ) 2 p(n) - p(n - 1) and let S (resp. 7') be the R.S.time associated with (t,, t i ) (resp. (r,, 1,)).Suppose that p(0) = 0 and set p(- 1) = 0; prove that I I
t l
2; < 2: (za + + +- + + 2:
I ,
c(dfi+ W
~ , [ d S )=l
0
1) - p ( n - 1))
(L' - 4 G ( E ) .
Deduce therefrom that
6. llosolvents In this section we digress from our main subject to introduce another situation which also gives rise to a potential theory and which is related to Markov processes with continuous time parameters. Some results relevant to this situation may be obtained by mere translation of results about transition probabilities. Numerous exercises in the following chapters will be designed to this end.
Dofinition 6.1. A sub-markovian resolvent on ( E , 8)is a family {V,, u > 0 } of positive kernels such that (i) aV,(x, E ) 1 for each x in E and u > 0 ; (ii) V , - V , = (b - u) V,V, = (/I - a) V,Va for a,/?> 0. Equation (ii) is called the resolvent equation. The reader will notice its resemblance to the equation of Proposition 2.5 and should also refer to Exercise.6.11.
<
CH. 2 , 3 6
RESOLVENTS
75
For f E b,, it follows from the resolvent equation that the map a is decreasing and that we may thus define
Vof = sup V,f
=
+
V,f
lim V a f . a40
a
If (f,) is an increasing sequence of functions with limit f, by interchange of increasing limits we get immediately
Vof
=
lim Vof,; n
this shows that V o is a positive kernel on ( E , 8)which satisfies the identities
Vo = V,
+ aVoVa = V , + aVaVo.
(8.2)
In the remainder of this section we consider only projher resolvents, that is, resolvents such that V , is a proper kernel. For each a > 0, the kernel Pa = aVa is a T.P. on E . We shall call X" the canonical Illarkov chain associated with Pa.
Definition 6.2. A function f in 8, is called supermedian (invariant) for the resolvent {V,} if it is superharmonic (harmonic) for every chain Xu, in other words if aV,f f (.Val = 1) for each a > 0.
<
Proposition 6.3. If f E B,, then Vof i s supermedian. I f Vof i s finite and if h is positive, invariant and smaller than V,]f,then h vanishes identically.
Proof. The first sentence is an obvious consequence of eq. (6.2). If h then eq. (6.2) again yields h = aVah
< Vof,
< aVaVof= Vof - V a f
for every a > 0. As a tends t o zero, Vof - V,f converges to zero and consequently h = 0.
Proposition 6.4. T h e fiotential kernel Ga of X u is equal to I Proof. Since I
+ avo.
+ avois a proper kernel, it suffices t o show that n=m
=
2 (aVa)"f
rt=l
76
CH. 2, $6
POTENTIAL THEORY
for every f E 8, such that V,f is finite. By applying k times the identity of eq. (6.2), we get n=k-1
avof
=
C (aVa)"f + (aJ'a)kV o f ; 1
= ,
if k tends to infinity, the term (aVa)kVof decreases to an invariant function which is smaller than Vof,and hence vanishes identically. The proof is then easily completed.
Remark. The kernels Ga are thus also proper kernels. We now state the complete maximum principle for V o which, as in $2, is a characterization of supermedian functions.
Theorem 6.6. The function g E 8, is supermedian i f and only if whenever Vof is well defined and Vof g on {f> 0 } , then V,f g everywhere.
<
<
Proof. We suppose that for x
E
For every integer n we set f ,
=
Vof,(x)
> 0 } we have
f + A n ; we, a fortiori, have
dg(4
and consequently, for every a G"f,(x)
{f
+ V,f-(x),
xE
If > 01,
> 0,
d4 4
+ G"f-(x) +
fi,
xE
{f> 0).
By Theorem 2.3, applied t o the chain X a , we get G"f,
< ag + GUf- + n
everywhere on E , and upon dividing by a and letting a tend to infinity, it follows that V,f, g V,f-. The proof of the necessity is completed by letting n tend to infinity. Conversely, let f > 0 be such that V o f be bounded, and set g, = g A nf, h, = a(g, - a V ~ , ) .The potential Voh, is well defined and therefore
< +
g 2 avagn
=
Voh, on {g - avagn
> 0))
hence a fortiori on {g, - aVag, > 0) = {h, >.O>. The hypothesis then implies that for every n, g 3 aY,g, everywhere, and by passing to the limit, we obtain g 3 ccVag.Since this is true for all a,the function g is supermedian.
CH.2, $6
RESOLVENTS
77
Exercise 6.6. Let {V,, a > O } be a resolvent and p a real number that the family of kernels VL = Va+bis a proper resolvent.
> 0. Prove
Exercise 6.7 (Resolvent of linear brownian motion). Prove that the family of convolution kernels V , defined on R by the densities v,(x, y ) = (2a)-1/2 exp{- (20r)-l/~Iy - X I }
is a resolvent. Exercise 6.8. The integral kernels defined on R by the densities v,(x, y ) =
v-1
exp{- a ( y - x ) / v ) if y
> x,
if y
< x,
=o form a proper resolvent.
Exercise 6.9. If h is a positive finite function such that V B his finite for all p > 0 and if there is an a such that aV,h = h, then It is invariant. Exercise 6.10. If f E b,, then for every x in E the function a -,V , f ( x ) is decreasing, right continuous and continuous on every open interval on which it is finite. Exercise 6.11. (1) Let {V,, u > 0) be a resolvent and h a function in b8,. Pick a > llhll and define Vh = (VaIa-h)nVa.
C
n>O
These equations contain the resolvent equation, since for h = P we have
V, =
v,3.
[Hint:Use the chains X u for which V , = u-~U,,~-,;in particular for h we have V , = U,, where this kernel is computed for X l . ] (2) The family of kernels Pa = V,J, is a resolvent.
Exercise 6.12. (1) Let P be a T.P. on ( E , 8).Prove that one defines a resolvent by setting
CH. 2, $6
POTENTIAL THEORY
78
V,
= (1
+ a)-'C ((1 + a)-1 P)". n>O
Prove that V o = G. (2) Assume that G is a proper kernel, and prove that a function is superharmonic for P if and only if it is supermedian for Ira. Then derive Theorem 2.3 from Proposition 6.5.
Exercise 6.13. Consider a proper resolvent {V,, a > O } . (1) Prove that, if a < b, the superliarmonic functions for X4 are superharmonic for X u . (2) Let f be supermedian, and for A E d let P:f be the reduced function of f relative to X u . Prove that the following limit RAf
= lim P:f U-WX
exists, and defines the smallest supermedian function which dominates f on A . (3) Prove that RAf f , RAf = f on A and if f , g are supermedian and f g on A , then RAf RAg.If {f,} is an increasing sequence of supermedian functions with limit f , then RAf = lim R,f,. (4) If h is a positive function vanishing outside A , then RAVoh = Voh. The operator RAVois given by a kernel. Is the same thing true for the operator R A ? (5) If f, g are supermedian, then RA(f. g) = RAf R,g. (6) If { A , } is an increasing sequence of sets with union A , then R A f = limn RA,,f.
<
<
<
+
+
Exercise 6.14 (excessive functions). (1) If f is supermedian, the function a -aV,f(x) is increasing for each x in E. We call f excessive for the given resolvent if in addition lima-,maV,f = f . Prove that the potentials V o g , g E t",, are excessive functions. (2) Let f be supermedian and set f* = lim,,,aV,f. Show that f * is an excessive function and that f * f. Show that the set K = { f * < f} on which f * differs from f is of potential zero, that is V,( * , K ) = 0 for one (hence for all) a > 0. Show that f* is the greatest excessive function bounded above by f . (3) Let f be a supermedian function and define, with the notation of Exercise 6.13, (RAf)*= ~ i m u ~ m a ~ uProve R z ~ that f . (RAf)*is excessive and that it is the smallest supermedian function which majorizes f on A up to a set of potential zero.
<
CH. 2, $6
RESOLVENTS
79
Exercise 6.15. A a-finite positive measure m is invaviant (supermedian) for a resolvent {V,) if amV, = m (amTf, m) for all u > 0. It is excessive if it is supermedian and if in addition lima+oumV, = m .
<
(1) Work out for supermedian measures results similar to those in Exercise 6.13. For A in 6 and m supermedian, set
mL, = lim mH:. ,430
=
(2) Prove that the operator V o L Ais given by a kernel and that VoCA RAVo.(Compare with Exercise 2.11.)
Exercise 6.16. If E is locally compact and if lima+maVa(x,* ) = E , in the vague topology, show that the supermedian Radon measures are excessive measures.
CHAPTER 3
TRANSIENCE AND RECURRENCE
I n this chapter we describe the basic classification of Markov chains according t o their asymptotic behaviour. We start with countable state space where the situation is particularly nice and then proceed t o more general state spaces.
1. Discrete Markov chains Definition 1.1. A Markov chain will be called discrete if the state space E is countable and I is the discrete a-algebra on E.
Remark. If E is an uncountable set and d is the discrete a-algebra, then for every T.P. on ( E , b),every point x E E is contained in the countable absorbing set { y : G(x, y ) > O } . It suffices therefore to study countable state spaces, justifying the above terminology. In the same way, in the case of random walks on a discrete group, the identity is contained in a countable absorbing subgroup. In the sequel, the term discrete group will therefore be understood to mean countable and discrete. In this section, we shall deal only with discrete chains, and shall use the notational device of ch. 1, $1.4, namely, we shall not distinguish between the point x and the set { x } ; when p is a measure we write p ( y ) in place of p({y}), and in particular P(x,y) for P ( x ,{ y } ) .We shall also abbreviate S(") (resp. R ( { x } ) , l ~ to ( ~ S~, ) (resp. R(x),h,) and speak of the state x instead of the point x to follow a time-honoured custom.
Definition 1.2. A state x is said to be transient if the set {x} is transient; otherwise it is said to be recurrent. The attention of the reader is drawn to the fact that x may be recurrent without the set {x} being recurrent in the sense of $3 in ch. 2. 80
DISCRETE MARKOV CHAINS
CH. 3, 51
81
Proposition 1.3. The following four properties are equivalent : (i) x is recurrent; (ii) G(x, x ) = + co; (iii) P,[S, < m] = 1 ; (iv) h,(x) = 1 .
+
Proof. If G(x, x) < a) then x is transient by ch. 2 $3.7, so (i) implies (ii). From Proposition 3.5 in ch. 2, we may write 7 = P { , 1 7 ( ~=) G ( x , X ) P,[S, = m]
+ h,(~).
If G(x, x) = + m we thus must have P,[S, = 001 = 0, hence (ii) implies (iii). The same formula shows that (iii) implies (iv) and finally that (iv) implies (i) is obvious. We observe that for any x in E , either h,(x) = 0 and x is transient or h,(x) = 1 and x is recurrent. We also have
Corollary 1.4. The following three properties are equivalent : (i) x i s transient; (ii) P J S , = 003 > 0; (iii) G(x, x ) < + 00 and in that case G ( x , x) = (P,[S, = a])-'.
Proposition 1.5. If the state x is recurrent, then for every Y h,(x) = Ps(S, < 001 = 0 OY h,(x) = P,[S, < 031 = 1; if C,
=
E
E , y # x, either
{ y : h,(x) = l},
every state y in C, is recurrent and C, = C,.
Proof. By ch. 2 53.5, we have lim P,,[S,
< 031
=
lim h , ( X f l ) = 7 R ( v ) P,-as. n
f l
Since x is recurrent we also have
and
6h,(X,) n
=
h,(x)
P,-as.
CH. 3, $1
TRANSIENCE AND RECURRENCE
82
It follows that l R ( , ) = P,[S,
< co] = h,(x)
P,-as.
As a result either P,[S, < co] = h,(x) = 1 or P J S , < m] = h,(x) = 0. If y E C, then h,(x) = 1 and y cannot be transient; it is thus recurrent. Let z be in C,; since the chain goes through x and y infinitely often P,-as., we have lim h,(X,) = h,(x) = h,(y) P,-as. n
Consequently h,(y) = 1, so z E C, and C, c C,. Therefore x E C, and The proof is complete.
C, c C,.
We may sum up the preceding results in the following
Theorem 1.6. There is a unique partitioiz (Ci, i E I ) of the set of recurrent states w c h that each C iis a n absorbing set and /or every @air ( x , y ) of states within the same class Ci,we have h,(y) = 1 . Moreover C ( x ,y ) = 03 or 0 according as x and y are within the same class or not. The sets C iare called the recurrence classes of X .
+
Proof. The sets Ci are obtained as the sets C, where x runs through the set of recurrent states. The reader is invited to solve Exercise 1.11 which gives another, more elementary, approach to the above results.
Definition 1.7. The chain X will be called transient if all the states are transient. In that case G(x, y) < m for every pair of states in E . If C is a class of recurrent states, then for the study of X with respect to the probability measures P,, x E C, we can restrict the state space to C. The chain X will be called irreducible recurrent if all the states are recurrent and if there is only one equivalence class. We then have C ( x , y) = co identically. We shall end this section by studying irreducible recurrent chains. The following proposition, which will be generalized later, shows that the main concern of the potential theory of ch. 2 is with transient chains. In the recurrent case, we will have to give another definition of potential theory.
CH. 3, $1
DISCRETE MARKOV CHAINS
83
Proposition 1.8. I f X i s irreducible recurrent the superharmonic und bounded harmonic functions are constant.
Proof. If f is superharmonic, f(X,) is a positive supermartingale which converges a.s. But for x and y in E , f ( x ) and f ( y ) are limit points of f ( X , ) and therefore f ( x ) = f ( y ) . The same argument works for the bounded harmonic functions.
Theorem 1.9. If X i s irreducible recurrent, there exists a unique (up to multzplication) excessive measure m and this measure i s invariant. If x i s any point in E , we may define m by 4Y)
=
H(&
Y)
Y).
=
U(Z,(%
Proof. Pick a point x in E and set
For every y in E , 0 < m(y) < a ;indeed, there is a strictly positive probability of entering y before returning to x , and on the other hand the chain killed a t S , is clearly transient. Now using the resolvent equation, we have
mp(Y) = H,z,P(x, Y ) = =
~(Z+%
Y)
+ I(z)U(r)I(z]P(x? Y)
Ilr)U{x)(X,Y ) = U,,,(% Y ) .
But for n = 0, we have l v ( X n ) = 0 P,-a.s. if y , # x , and l v ( X n ) = 1 P,-as. if y = x , and on {S, = n } we have also l V ( X , , )= 0 if y # x and l V ( X , J = 1 if y = x ; consequently
Ez
l,,Xn,]
=
E,
[.Z;m] 9
that is exactly H , , , ( x , y ) = U,,,(x, y ) , and it follows that m P = m. To prove that m is the only excessive measure for P we set i ) ( x , y ) = m ( y ) P(y, x)/m(x).The reader will easily check that P is a T.P. in duality with P with respect to m, and that the corresponding chain is irreducible recurrent. If ,u is an excessive measure for P , then f ( ) = ,u( * ) / m ( * ) is superharmonic for p , so that the result follows from Proposition 1.8.
-
84
TRANSIENCE AND RECURRENCE
CH. 3, 51.
Remarks. (1) The empty set is the only set of m-measure zero. ( 2 ) For two different points x in E we get, by the above process, two proportional values for m. (3) The T.P. P defined in the proof above is the only one t o be in duality with P. For an irreducible recurrent chain we may therefore speak freely of its dual chain, which exists and is unique. The two chains play totally symmetric roles, in particular m is invariant for Ij as well as for P. Corollary 1.10. If X i s irreducible recurrent, either E,[S,] < co for every x in E and the invariant measure m i s bounded, or E,[Sx] = 00 for every x in E and the invariant measure m i s unbounded. I n the former case there i s an invariant Probability measure n which i s given b y
n ( x ) = l/E,[SZ], x E E Proof. By the previous result, for every x in E , we have
and since x was arbitrary, this proves the first sentence. To prove the second, observe that n = m/m(E), therefore
4%) = m(x)/m(E) =
U,Z,(?,
x)/Er[Sxl = 1/Ex[SxI.
This last result says that n ( x )measures the frequency with which the chain goes back to x. In the former case the chain is said to be positive, in the latter it is said to be null. The theory of recurrence we have been sketching cannot be carried over in an obvious way to general state spaces. The sequel of this chapter aims a t giving some of the possible extensions. Other viewpoints will be developed in the following chapter.
+
Exercise 1.11. Let S, = S,,. . ., S, = S,-l S, 0 OSn-, the successive random times for which X , = x. (1)Let Z be an integer-valued and 9,,-measurable random variable. Prove that the random variables Z o B, are independent and equally distributed with respect to the probability measure P,. ( 2 ) By applying the foregoing to 2 = S, prove that P,[S, < co] = ( P x [ S ,< a]),and derive Proposition 1.3 and Corollary 1.4. (3) If x is recurrent and Px[S, < 031 > 0,prove that y is recurrent and
CH. 3, $ 1
DISCRETE MARKOV CHAINS
86
P J S , < a] > 0 by the use of Markov property. Derive Proposition 1.5 without using Martingale theory.
Exercise 1.12. Let E be finite and P markovian; prove that there is at least one recurrent state. If there are N states in E and if P,[S, < m] > 0,prove that P,[S, < N ] > 0. Exercise 1.13 (another proof of Proposition 1.8). (1) Assume the chain to be irreducible recurrent and derive from the equality (g/m)
(1 - PI) = f - Pnf,
that every superharmonic function is harmonic. Then prove that if a harmonic function vanishes at one point in E , it vanishes identically, and apply this result to f - ( f A k ) for every choice of the constant k . (2) Prove the following converse to Proposition 1.8: if every bounded superharmonic function is constant, then X is irreducible recurrent.
Exercise 1.14 (continuation of ch. 2, Exercise 4.19, birth and death chain). Prove that the chain is either transient or irreducible recurrent and that the latter case occurs if and only if (1 - A)-ld,u(A) =
+ a.
Exercise 1.15 (Galton-Watson chain; continuation of ch. 1, Exercise 2.20). (1) Call u the smallest solution of the equation g(s) = s in the interval 10, 11. Prove that u = 1 if E,[X,] 1 and u < 1 if E l [ X l ]> 1. (2) Assume that 0 < p(0) < 1, the other cases offering neither interest nor difficulty. Show that 0 is an absorbing (hence recurrent) state and that
<
P, hJ{X', i n
=
1
0) = uz.
Interpret this probability with respect to the relevant population. Prove that all the other states are transient and describe the asymptotic behaviour of the chain according to the value of u. See ch. 2, Exercise 1.23 for a particular case. (3) Assume further that E,[X,] is finite and prove that {X,/(El[X,])">
TRANSIENCE AND RECURRENCE
86
CH. 3, $1
is a Martingale which converges a s . to a random variable Z. Describe completely the asymptotic behavior of X.
Exercise 1.16 (renewal chain). Define a T.P. on the set of positive integers 1) = p,, P(n, 0) = 1 - p , with 0 < p , < 1. Find the condition by P(n, n under which the chain is irreducible recurrent, then compute the invariant measure. Prove that if the chain is transient, there is no invariant measure.
+
Exercise 1.17 (cyclic classes). Let X be irreducible recurrent and for x in E , call d ( x ) the greatest common divisor of the set of integers {n > 0: Pn(x,x )
> 0).
(1) Prove that there is a number d such that d ( x ) = d for every x in E. If d = 1 we say that X is aperiodic, otherwise X is periodic with period d. (2) If d > 1, there exists a partition of E into sets Ci, i = 1 , . . ., d, such that
P ( . , Ci+J
=
lOi, i
=
1 , 2 , . . ., d - 1,
P ( . , C,)
=
ICd.
(3) (The Ehrenfest model of diffu'sion). N particles are distributed in two containers U 1and UB.At every time n a particle is chosen a t random and is moved from its container into the other. Define X , to be the random number of particles in U1 a t time n. Show that the sequence X , is a Markov chain; write down its T.P. and compute its invariant measure. Prove that the chain is periodic of period 2. (4) Find the conditions under which the renewal chain of Exercise 1.16 is irreducible recurrent and periodic.
Exercise 1.18. Let X be a chain on N such that P,[T, < 001 > 0 for any pair ( x , y) of points in N. ( 1 ) Prove that X is recurrent if and only if there is an integer N > 0 and a positive function f such that Pf f on [ N , co[ and Km, f(n) = 03. [Hint: See ch. 2 $2.10.1 (2) Prove that X is transient if and only if there is a bounded positive functionfsuchthatPf
Oandf(x) <sup{f(y):y < N } for some x > N .
<
CH. 3, 92
a7
IRREDUCIBLE AND HARRIS CHAINS
2. Irreducible chains and Harris chains We now return to a general state space ( E ,B), but we assume henceforth that it is separable.
Definition 2.1. The chain X is said to be v-irreducible if there exists a probability measure v on B absolutely continuous with respect to U,(x, ) for all x E E and c E 30, l[.
-
Clearly if the above condition is satisfied for a measure v‘, it is satisfied for a probability measure v equivalent to v’; it is satisfied for all numbers c E 30, 1[ as soon as it is satisfied for one of them. This property can also be spelled out as: if A E B and v(A) > 0, then for all x in E there is an integer n such that Pn(x,A ) > 0. Furthermore if X is v-irreducible and if we set V’ = CVU,, the reader will easily check that X is still v‘-irreducible, but that v‘ has the additional features: (i) v’P<< v’, (ii) P,,[S, < 001 = 0 if v‘(A) = 0. The significance of condition (i)will be seen in the following chapter. The purpose of this section is to show that for a v-irreducible chain either the potential kernel is proper or the chain is recurrent (in a strong sense to be defined later). Following ch. 1, $5.3we shall call p c ( x ,y ) an &‘ @I &‘-measurableversion of the density of U,(x, ) with respect to v. The irreducibility of X implies that for all x in E , $,(x, ) is > 0 v-as.; then p c ( x ,* ) I ~ p c = o ) (*x), is still a density for U c ( x , ) and is strictly positive everywhere. We shall therefore take $, to be strictly positive in the sequel.
+
-,
Proposition 2.2. There is a function h, E Q, strictly eositive and strictly less than 1, and a positive measure mo equivalent to v such that Uh0
2 Un,(ho) 8 mo.
Proof. Let c < c’ < 1 ; by the resolvent equation of ch. 2, Proposition 2.5,
u, > (c’ - c) u,u,r where P(X!
Y) =
s.
$c(%
2)
> (c’ - c) qv, P c k Y ) V(W.
Since $, is B @J &-measurable and > 0, the functions lk(. ) = v({z: p,( * , z ) 3 l / k } ) are measurable and increase to I as k tends to infinity.
CH. 3, 92
TRANSIENCE AND RECURRENCE
88
We can therefore define the integer k(x)
=
inf{k 3 1: f&)
Since { x : k(x) = P ) = { x : iD-l(x)
the map x
--+
2 8).
< a} n { x : ill(%) 2 a>
k(x) is measurable, and setting a(x) = l/k(x), we have v({z: P
c b
4 3 a@)))2 a
for every x in E . In the same way we can find a function b such that, for every y in E , 4 { z : P&P y ) 2 b M 1 ) 2 We then have
a.
hence U , +(c' - c) a @ bv. Set now h, = inf(+c,a ) ; we have 0 < ho < 1. Also, since ho a we still have U , 2 +(cr- c) ho @ bv and since ho c, the resolvent equation implies
<
<
uho
3 Uholc-ho 2 C(Cr
uc
3 gcuho
uc
- C ) Uho(ho @ bv)/4 = uho(ho)@
&(C'
- C ) by.
Now set m, = fc(c' - c) bv; since b is strictly positive mo is equivalent to v and the proof is complete. We recall from ch. 2 that a set F in B is said to be absorbing for X if P(x, F ) = 1 for every x in F , or equivalently if l F Cis superharmonic. The recurrent classes of discrete chains provide examples of absorbing sets. Theorem 2.3. For a v-irreducible chain X , there are only two possibilities: (i) the potential kernel i s a proper kernel; (ii) there is an absorbing s,et F such that Y ( F = ) 0 and a strictly positive function h, E @+, such that for every x in F Uho(ho)( x ) = 1 and Uho(x,' ) 2 mo( ' ).
Proof. Let h, be the function of Proposition 2.2. According as Uh,(ho)= I v-as. or not, we will get the second or the first possibility.
CH. 3, 92
IRREDUCIBLE AND HARRIS CHAINS
89
(1) Suppose that 1 - Uho(h,) is not v-negligible, which implies that (homo, 1 - Uho(ho))= c > 0. Since u,, U,,(h,) €3 m,, we have
Now by ch. 2, Proposition 2.5,
so that Gho is a bounded function, and it follows from ch. 2, Proposition 1.14 that (i) holds. ( 2 ) If, on the other hand, U,,(ho) = 1 ?-as., the set F = {Uho(ho)= I } is such that v(F") = 0, and we claim that it is an absorbing set. The resolvent equation Ub, = P PI~-houho implies
+
- Ubo(hO)
(%)
> p ( ( 7-
It0)
- ubo(hO)))
(x);
-
for x in F the first member of this inequality vanishes, hence P(x, ) vanishes outside the set ((7 - h,) ( I - ubo(ho)) = 0},which is equal to F , since ho < 1 on E. On the other hand, by ch. 2 $2.6, P(x, E ) = 1, because Ubo(ho)( x ) = 1 and consequently P(x, F ) = 1. The set F is thus an absorbing set, and by Proposition 2.3 the proof is complete.
To proceed we will need the following lemma in which B denotes the Banach space of bounded measures v such that v ( E ) = 0.
Lemma 2.4. The Banach space B i s left invariant by any markovian transition probability Q. If the norm A of Q as an endomorphism of B i s strictly less than one, there exists a unique Q-invariant probability measure on ( E , B ) , and for every x in E
n
I Q"(%
*
) - 14 *
)\I < 24".
Proof. The first sentence of the lemma is straightforward. Let v be any probability measure on d and k any integer. The measure v - v Q k is in B and its norm is a t most equal to 2; thus
which shows that the sequence {vQn} is a Cauchy sequence in the space of
90
TRANSIENCE AND RECURRENCE
CH. 3, $2
bounded measures. It has therefore a limit 17 which is plainly Q-invariant and such that [IvQ" - 1711 24".
<
If v' is another probability measure on 8,we have
IIvp- 1711 = II(~'- n)
ZA~;
it follows that if v' is invariant, then v' = 17,hence 17 is the only Q-invariant probability measure. Moreover, letting v' = E,, we obtain the last conclusion in the statement. In case (i) of Theorem 2.3 the space E is the union of an increasing sequence of transient sets. We now want to show that the case (ii) corresponds to a strong recurrence property on F . Since F is absorbing, the chain started at a point x in F stays in F P,-a.s.; it is therefore natural to restrict P to F and thus obtain a T.P. on the space ( F , F n b),the more so in the present instance since v is carried by F . For notational convenience we will rather assume below that F = E or, in other words, that U,,(h,) = I everywhere on E and U,, 3 7 @ mo. We then get
Theorem 2.6. There exists a positive, a-finite, P-invariant measure m such that m >> v and such that m ( A ) > 0 implies
P,
[
m
2 n=l
IA(X")
= co
1
= I
for every x in E . More generally, we have U,(h) = I for every h E %+ such that m(h) > 0.
Proof. Let 1 be a measure in the space B of Lemma 2.4; we have izu,,Iho = A(UhoIh0- 1 @ homo), and the norm of the operator uh,I,, on B is therefore less than or equal to 1 - mo(h,) < 1. By Lemma 2.4, there exists therefore a probability measure 17 that is invariant by U,,I,,, hence such that I? = 17U,,,I,, 2 homo. Next set m = (7(h0)17; the measure m is positive and a-finite, since ho is strictly positive and m 2 m,, which implies m >> v. Finally for g E bQ,, using the resolvent equation uh, = P Uh,Il-hoP# we get
+
IRREDUCIBLE AND HARRIS CHAINS
CH. 3, 52
91
which proves that m is P-invariant. We proceed to prove the last conclusion in the statement. If h is not mnegligible, then h’ = h A ho is also non-negligible, and since V,(h) increases with h (ch. 2, Proposition 2.6), we have u h # ( h ’ ) U h ( h ) 1. I t suffices, therefore, to prove the desired result for h hoe We then have
<
<- <
<
< <
(UhoIho)ng Consequently g ,< ( u h , I h , ) g * we get g ,< n ( g ) everywhere on E. I t follows that g integrating with respect t o 17 yields
n(g)
< n(g) (n,
Uho(hO
<
, and b y Lemma 2.4 U h o ( h O - h) n(g), and
* * *
- h ) ) = ( l - m(h))17(g).
If m(h) > 0, then n ( g ) = 0 and g = 0 everywhere. If A is a set such that m ( A ) > 0, we thus have U A ( I A )= P . [ S A < co] = 1 everywhere on E , so that by ch. 2 $3.5 we have R(A) = 1, a.s., which completes the proof.
Definition 2.6. The chain X will be said t o be recurrent i n the sense of Harris if there exists a positive, a-finite, invariant measure m such that m ( A ) > 0 implies
for all x in E . We shall also say more simply that X is a Harris chain or that X is Harris. The T.P. of a Harris chain will also be said to be Harris. This definition makes perfect sense even if d is not separable. However in the non-separable case a v-irreducible chain may be non-Harris even though its potential kernel is not proper (see Exercise 2.21). In the sequel we keep on working with separable spaces, leaving as exercises to the reader the task of determining which results extend to the general case.
TRANSIENCE AND RECURRENCE
92
CH. 3, $2
A Harris chain is clearly irreducible, and conversely we have just shown that an irreducible chain such that the potential kernel is not proper is recurrent in the sense of Harris up to a negligible set. The irreducible recurrent discrete chains are Harris chains, and we shall study other important examples later on. One of the main subjects of this book will be the study of Harris chains. We start here with a few preliminary results. The condition in Definition 2.6 is equivalent to U A ( l A )= I everywhere on E provided m ( A ) > 0. Actually, it is equivalent to a less stringent condition. Proposition 2.7. The following three statements are equivalent : (i) Xis recurrent in the sense of Harris with respect to the invariant measurem; (ii) there is a strictly positive function h, E @+ and a non-zero measure mo, such that uho(ho)= I 0n.E and u h , >, 1 @ m,; (iii) there i s a non-zero positive measure ml such that
everywhere on E 9rovided m,(A) > 0. The measures mo and ml are then absolutely continuous with respect to m.
Proof. Clearly (i) implies (iii), and we saw that (ii) implies (i). I t remains to show that (iii) implies (ii). If (iii) is satisfied, X is clearly ml-irreducible. Thus, by Proposition 2.2, there is a strictly positive function ko E @+ and a measure mo equivalent to ml such that u h o 2 U,,(h,) @ m,. We must show that Uho(ho)= 1. Since h, is strictly positive there is a number a E 10, 1[ such that the set A = {h, 2 x } is of positive ml-measure. Since ho 2 al,, we have Uh,(ho)2 UaTA(cdA); but the resolvent equation (ch. 2 , Proposition 2.5), applied to the kernels U Aand U a I Ayields , Uix," = (1 - 4" ( U A T A ) " u*,
c
n>O
and it follows that Ua,A(uIA)=
U C(1 - a ) n (U,Z,)n+l n>O
I
=
a 2 (1 - a)" = I, n>O
which completes the proof. We now generalize the results obtained in $1 for irreducible recurrent discrete chains.
CH. 3, 32
IRREDUCIBLE AND HARRIS CHAINS
93
Proposition 2.8. If X i s Harris, the measure m is the only a-finite P-excessive measure (up to mzcltiplication by a positive scalar). Proof. Let A be a a-finite excessive measure and ho the function of Proposition 2.2. The measure hoA is a-finite, and ch. 2, Proposition 4.4 implies that hOA
2 (hd)
uholho
>, ' * *
(hOh) (Uhorho)n
2'
*
J
and passing to the limit yields, by Fatou's lemma, hoA 2 W
O ) (horn).
This implies A(ho)< 00; otherwise hoA would not be a-finite. Now, since m(ho)= 1, the measures on both sides of this inequality have the same total variation, hence they are equal, and the proof is complete. We shall now improve ch. 2, Proposition 4.4, and prove that, for Harris chains, l Am is actually invariant by IIAwhen A is not negligible. Proposition 2.9. If f E 9,and m ( f )> 0, the T.P. U,I, i s Harris with ilzvariant measure fm; moreover m = (fm)U,. I n particzclur for A E 8 and m ( A )> 0, the chain induced on A is Harris, the trace IAm of m on A is nA-invariant and mHA = m. Proof. Set Q = u,I,; for 12 E &+, the kernel denoted by Up and is equal to
Consequently, if m(f1,)
Uh
associated with Q will be
> 0, Theorem 2.5 implies that u!(lA)
= ul~f(lA/)
1.
The T.P. Q is thus Harris; since by ch. 2 94.4, the measure fm is Q-excessive, it is Q-invariant and it is the only Q-excessive measure. Now, by the resolvent equation, we have ( f m ) U f P = (fm)
<
u,.-
( f m )p
+ ( f m )U,I,P;
since (fm)P mP = m, the measure (fm)P is a-finite, and by the above discussion the two last summands cancel. The measure (fm) U , is therefore P-invariant, and consequently there is a number k such that km = (fm) U,.
94
TRANSIENCE AND RECURRENCE
CH. 3, $2
If f is integrable, multiplying to the right by f immediately gives k = 1. If f is not integrable, let g be an integrable function smaller than f . By the above discussion (gm) U , = m ; applying (gm) to the left of both sides of the equality U , = U , U,(f - g) U , gives
+
+
(gm) u,= (gm) u, ( f m )u, - (gm) u,. Since U , >, U,, the measure (gm) U , is a-finite, and by cancellation we obtain
( f m )U ,
=
(gm) U ,
=
m.
Finally, if A E d and m(A) > 0 , the preceding equality yields mIAU,IAC = lAcm; hence m H A = m.
Remark. That m H A = m may be derived from ch. 2 $4.5 and the fact that m is the only excessive measure. This result contains Theorem 1.9. We now turn to the study of superharmonic functions. The next result is dual to Proposition 2.8 (see ch. 2 $4.14 (3)).
Proposition 2.10. The bounded harmonic functions of a Harris chain are constant and the superharmonic functions are constant m-a.e. More firecisely if f is superharmonic there i s a constant C such that f = C m-a.e. and f C everywhere.
>
Proof. If the superharmonic function f is not constant m-a.e. we can find two numbers a < b such that m({f< a>) > 0 and m({f > b}) > 0. The sequence {f(X,)} is a positive supermartingale which converges a.s. to a random variable Z. Since by hypothesis, X hits the sets { f < a } and { f > b}'infinitely often, we have Z < a a s . and Z > b as., which is a contradiction. There is thus a constant C such that f = C m-a.e. and { f ( X n ) }converges to C a s . Since for every x and n we have f ( x ) 2 E,[f(X,)] it follows by Fatou's lemma that f 3 C everywhere. If h is bounded and harmonic, by applying the preceding to h llhll - h , we see that h is equal t o a constant C everywhere.
+ llh11 and
This proposition shows that for a Harris chain the a-algebra f is a s . trivial. A much stronger result along this line will be proved in ch. 6. By what was seen in ch. 2 $3, we also see that every set is either recurrent or transient. The following proposition gives a criterion to decide which one a particular set is.
CH. 3, $2
IRREDUCIBLE AND HARRIS CHAINS
95
Proposition 2.11. The following three properties are equivalent : (i) A is recurrent; (ii) m ( A ) > 0 ; (iii) P,[S, < co] > 0. Proof. If m ( A ) = 0, for every k we have
and A cannot be recurrent; thus (i) implies (ii). That (ii) implies (iii) is obvious. Finally, if (iii) holds, there exists a number a > 0 such that m ( { x : P,[SA
< a] > a } ) > 0.
Since X is Harris we get therefore, using ch. 2, Proposition 3.4,that lim Px,[SA < co] = I,,,, n
2 a > 0 as.
It follows that P,[R(A)J= 1 for each x in E , that is property (i).
Corollary 2.12. Every transient set A i s contained in a transient set B such that Bc i s absorbing. Proof. Pick c €10, 1[ and set B = A U {x: cUc(x,A ) > O}. Then BCis absorbing by the Markov property and m(A) = 0 implies cmU,(A) = 0 and in turn m { x : cU,(X, A ) > 0} = 0.
Exercise 2.13. Prove that there is no measure for which the chain of translation on Z (P(x,) = )) is irreducible.
-
-
Exercise 2.14. If X is Harris a finite function f is superharmonic if and only if it can be written f = C Gg, where C is a constant and g E & + with m(g) = 0.
+
Exorcise 2.14. (1) Using the notation of ch. 1, Lemma 5.3,prove that for any T.P. on a separable state space and any probability measure v such that VP << v,
96
TRANSIENCE AND RECURRENCE
CH. 3, 92
and derive that g ( x ) = limn P:(x, E ) exists for all x in E. (2) If X is Harris and if m and v are not mutually singular, then g(x) = 0 for all x in E. [Hint: Show that I - g is superharmonic.]
Excrcise 2.16. (1) If X is irreducible and if there is a bounded invariant measure, then we have the situation (ii) of Theorem 2.3. Give an example to show that we can have (i) with a non-bounded invariant measure. [ H i d : See $3.1 (2) Prove that in the situation (i) a measurejs invariant if and only if it can be written 1 A I h o u h o , where 1 is a positive measure such that (1,ko Uo(ho))< co and Arl-hoP = 1.
+
Exercise 2.16. Give an example in which the set { U h o ( h O ) < I } of Theorem 2.3 is not empty although P l = 7 . [Hint: Use a discrete irreducible recurrent chain on the space E and take as new space the sum of E and N. Define P such that P(tt;)z 1) = qn and P(n, E ) = 1 - qn, and choose the qn’s suitably.] Exorcise 2.17 (converse to Proposition 2.10). Let X be a Markov chain for which there exists a measure v such that every bounded superharmonic function is v-a.s. equal to a constant and larger than this constant everywhere; prove that X is Harris.
Exercise 2.18. If X is Harris and L is a death time (ch. 1, Exercise 3.18 and ch. 2, Exercise 2.17) prove that either L = 03 a s . or L = 0 P,-as. Exercise 2.19. Resume the notation of ch. 2, Exercise 6.11 and for k
=
I,
write V h = i f A . (1) The resolvent { V,} is said to be (recurrent i n the sense of) Harris if there is a positive measure ml such that V,(t,) = 1 everywhere on E provided m l ( A ) > 0. Prove that the resolvent is Harris if and only if X1 is Harris and that X u is then Harris for each 0: > 0. If the resolvent is Harris, there exists a unique invariant measure m which has the same property as m,. Furthermore the supermedian functions are constant m-a.e. ; if in addition the kernels V , are absolutely continuous with respect to m, the excessive functions are constant. (2) The resolvent {V,} is said to be v-irreducible if there is a probability measure v on ( E , 8)such that v << V,(x, * ) for all x in E. Prove that Theorems 2.3 and 2.5 extend to the present situation.
IRREDUCIBLE AND HARRIS CHAINS
CH. 3, 52
91
Exercise 2.20. Proposition 2.8 cannot be extended to signed measures. For instance, the measure of density x with respect to the Lebesgue measure dx is invariant for the chain X 1 of the resolvent of Brownian motion; see ch. 2, Exercise 6.7. (However, the reader should be cautious, because this measure is a Radon measure, which is not defined for all Bore1 subsets of R.) The same example shows that there may exist non-constant unbounded harmonic functions. Exercise 2.21. The following example shows that the separability assumption in Theorems 2.3 and 2.5 cannot be dropped. The state space is the positive half-line endowed with the u-algebra generated by countable subsets. Let v be the probability measure defined by v ( A ) = 0 if A is countable and v(A) = 1 otherwise; pick a sequence of numbers a, such that 0 < a,, < 1 and a, > 0 and define the T.P. by
nt
where n
< x < n + 1.
Exercise 2.22. If X is Harris on a non-separable space, prove that the only excessive measure. [Hint: Use the admissible a-algebras of ch. 1, Exercise 1.17.1
7,.
la
still
Exercise 2.23 (theory of R-recurrence). We consider a v-irreducible chain X on a space ( E , 8)which is not assumed to be separable. (1) Prove that there exists a real number R 2 1, a v-null set N and a sequence {En} of sets increasing to E such that R is the common radius of convergence of the series 2; rnP,(x, A ) for every x $ N and every A such that A c En for n sufficiently large and v(A) > 0. Moreover this series is convergent for every x and A as above or it is divergent for every x in E and every A as above. If R = I , then N = 0; give an example where N # 0 for R > 1. In the first case (second case) the chain is said to be R-transient (R-recurrent). [Hint: One can use the kernels V r = rn-lP, which satisfy a resolventtype equation.] (2) A finite function / € 8, is called r-superharmonic (r-harmonic) if rPf f (rP! = f ) v-a.e., and a measure p is r-excessive (r-invariant) if rpP p (rpP = p). Prove that for r > R the r-superharmonic functions vanish v-a.e. and the only r-excessive measure is the zero measure. Prove that in general an r-superharmonic function which does not vanish v-a.e. is strictly positive.
cF
< <
98
TRANSIENCE AND RECURRENCE
CH. 3, $3
(3) Assume from now on that X is R-recurrent and prove that the R-superharmonic functions are R-harmonic and the R-excessive measures are Rinvariant. (4) Let U; = (RPlBc)"R P and prove that for v(B) > 0 and f Rharmonic and strictly positive we have
2:
f
=
U i I B f v-a.e.
Deduce therefrom that two R-harmonic functions are proportional v-a.e.
3. Topological recurrence of random walks In this section we deal with a right random walk X of law p on a locally compact separable group G . The results will obviously carry over to the case of left random walks.
Definitions 3.1. We denote by T , (resp. G,) the smallest closed semi-group (resp. closed subgroup) containing the support of p. We shall say that X (or its law p) i s adapted if G , = G , irreducible if T , = G , and aperiodic if it is adapted and the support of p is not contained in a coset of a normal closed subgroup. Unless otherwise stated, we will always assume that the random walks we are dealing with are adapted. The reason for this is the following. I t is necessary and sufficient for a point g to be in T,, that for every neighbourhood V of g, there exists an integer n such that pn(V) = P,[X,E V ]> 0. The group G being separable, we therefore have
and by translation in G , for all g E G ,
The sub-group G , is therefore an absorbing set for X , to which we can restrict ourselves for the probabilistic study of X .
Definition 3.2. The random walk X is said to be recarrent if every open set 0 is recurrent, or in other words if P.[R(O)]= 1 everywhere on G. Otherwise it is said to be transient.
CH. 3, $3
RECURRENCE OF RANDOM WALKS
99
Theorem 3.3. A random walk is recurrent if and only i f for every neighbourhood of e,
v
Proof. Only the sufficiency needs to be shown. Let R, be the set of points g in G such that P,[R(V)]= 1 for every neighbourhood V of g. We take g E R, and h E T , and we claim that h-lg E R,. Let U be a neighbourhood of h-1g; hug-' is then a neighbourhood of e and one can find a neighbourhood V of e such that V-'VC hug-'. Since V h is a neighbourhood of h, there exists an integer k > 0 such that the event A = { X , E V h } is of positive Pe-probability. For o E A , the fact that X n + k ( o E) Vg implies X,'(w) X,,
k ( ~ E)
c U;
h-'V-'Vg
and g being in R,, for P,-almost all o in A , there exist infinitely many integers n such that Xn+k(u)) E Vg, so that
Lm
P, 6 {x;'Xn+k E
u}n A
I
=
P,[A].
{x;'
According to ch. 1 $4, the sequence of random variables same law as the sequence {X,} and is independent of A E F
[u.,
1
P, Tii;;{X,EU }
=
x,+k}
k ,
has the
hence
1.
We have thus shown that T i ' R , c R,, and the hypothesis implies that R, contains at least the point e. On the other hand it is easily seen that R, is closed. The set R, is therefore a closed sub-group containing T,, and consequently R, = G , which completes the proof. We shall prove below that if X is transient, then for every compact subset K in G and every g E G,
P,[R(K)] = P,
fEiii [.+..
{X,EK}
I
= 0.
We begin by generalizing some of the results obtained for discrete chains.
Proposition 3.4. The random walk is recurrent if and only i f for every neighbourhood V of e, P,[S, < w ] = 1.
T R A N S I E N C E AND R E C U R R E N C E
100
CH. 3, $3
Proof. Only the sufficiency needs to be shown. Let d be a left-invariant metric on G and Vr the corresponding open ball of centre e and radius r. Let 0 < r' < r be two reals; from the relation {Xm+n4
vr}
n {Xm E
vr,)c {XilXm+n4 v r - r , > ,
we deduce, since the sequence {X;lXm+n}n>O has the same law as the sequence {Xnln>O, that
n>l
= Pe[S,,-,,
= a]= 0.
Let {rk} increase to r ; as { X , E V,} = lim{X,E V,}, k
passing to the limit in the above inequality yields
i
pc {xm E v r ) n
If L
=
1
n{Xm+n4 V r > n>l
=
0.
L,, (see ch. 1 §3.18),we then have
and the random walk is recurrent.
-
The potential kernel G(g, ) of X is a convolution kernel. We shall call G the measure, possibly non-o-finite, such that G(g, ) = (cg * G) ( ), that is
e = 20"p.
Theorem 3.6. The random walk X i s reczcrrent if and only if c(0)= every open neighbourhood 0 of e.
+ 00 for
Proof. The necessity is obvious. To show the sufficiency let us use the notation of the preceding proof. We have
RECURRENCE O F RANDOM WALKS
CH. 3, $3
>, because x ent,
E
c pe 1{ x m m
V r ,x-ly
E~
r
n> n { x i l ~ m + 4n ~
101
z r ~ ]
n>O
4 V z vimplies y r$ V r ;as X , 1 3 c ( V r )PEESVZ,
By hypothesis G(V,) = co; thus P,[SV,, the random walk X is recurrent.
=
and,X;'X,+,
are independ-
~ 1 . W]
=
0, and since r is arbitrary,
Corollary 3.6. I f the random walk X is transient, then the potential kernel is a proper kernel, the measures G(g, * ) are Radon measures, and the set {G(g, * ), gE G } is vaguely relatively compact.
Proof. By the preceding result there exists an open neighbourhood 0 of e such that G ( 0 ) < 03. Let V be an open neighbourhood of e such that v-'V c 0. If g € V ,
< G(V-lV) < G(O), and by the maximum principle G( -,V ) < c(0)everywhere. For every cornG(g, V ) = G(g-lV)
pact K of G , there exists a finite sequence gl, . . ., g, of points in I< such that the sets g,TIV form an open covering of I<. For all g E G we then have n
n
which completes the proof.
Remark. If a random walk is recurrent, the dual random walk is also recurrent. On a non-abelian random walk, the right random walk is recurrent if and only if the left random walk is recurrent. We end this section by studying the relationship between the topological recurrence of random walks and the recurrence in the sense of Harris. This leads us to consider the important class of laws defined below.
Lemma 3.7. For a Probability measure ,u on a group G , the following two fro$erties are equivalent : (i) there exists an integer p such that , u p is not singular with resfect to a right Haar measure m on G .
TRANSIENCE AND RECURRENCE
102
CH. 3, $3
(ii) there exists an integer q such that pq dominates a multiple of m on a nonempty open subset of G .
Proof. That (ii) implies (i) is clear. Conversely if (i) holds, there exists a nonzero function g in Lm(m)n L1(m)such that , u p 2 gm. Then p 2 p >, (g * g) m and, as g * g is continuous and non-zero, (ii) holds for q = 2$. Definition 3.8. A probability measure p will be called spread ozlt if it satisfies the equivalent conditions of Lemma 3.7. A random walk whose law p is spread out will also be called sfiread out. This class is much larger than the class of absolutely continuous probability measures. A probability measure can be singular with respect t o Haar measure and yet be spread out (see Exercise 3.16). On a discrete group, all probability measures are spread out. Let us observe that if p is spread out, for any c in 10, 1[ there is a non-zero g in 2?y(m) such that U,(x,* ) 3 E , * ( g m ) , and therefore for any bounded positive Bore1 function h such that m ( h ) > 0, the function U,h will dominate a positive non-zero continuous function. This will be used to prove
Theorem 3.9. A recurrent random walk is recurrent in the sense of Harris if and only if its law p i s spread out.
Proof. Since the right Haar measure m is a-finite, positive and invariant, if X is Harris, its invariant measure must be m. Then m ( A ) > 0 implies m
m
n=O
#=O
2 2-"pn(A) = 2 2-" P,[X, E A ] > 0; 2;
the measures m and 2-"pn cannot be mutually singular, and ,u is therefore spread out. Conversely, if X is recurrent, U,p, > 0 for any non-zero y E C i . Let A be such that m(A) > 0; by the above remark, Uc31Adominates a non-zero continuous function and by the resolvent equation with c > c', we get U , I A (c - c') U,U,.IA > 0. A s a result, X is v-irreducible for any v - m . Suppose now that h is a bounded, strictly positive function such that Uoh is bounded. We then have
cUoU,h
< Uoh <
00
CH. 3, $3
RECURRENCE O F RANDOM WALKS
103
which is impossible since U,h dominates a non-zero positive continuous function. As a result, X enjoys property (ii) of Theorem 2.3. There exists an absorbing set F such that nz(FC)= 0, and for all g E F and A E &' with m ( A ) > 0, P , [ R ( A ) ] = 1. The random walk X being translation invariant, all the sets obtained from F by translations have the same property and every point in G belongs to one of these sets; so for m ( A ) > 0, P . [ R ( A ) ]= 1 everywhere on G .
Corollary 3.10. If X is recurrent and sfiread out, a set is transient if and only i f at is of Haar measure zero; otherwise it is recurren.t.
Proof. This corollary follows immediately from Proposition 2.1 1. Exercise 3.11. Every random walk on a compact group is recurrent. Exercise 3.12. Let G be R or Z and assume that
Show that the random walk X is transient. If , I> 0, show that X , converges a s . towards 03. [Hint: Use the strong law of large numbers.]
+
Exercise 3.13 (Right continuous random walks on Z). (1) Assume that G = Z and p ( x ) = 0 for x > 1. Show that, if, with the notation of the preceding exercise, 1> 0, then G(0, x ) = 1-l for x 2 0. [Hint: Show first that G(0, x) = G(0, 0) for all x 3 0. Then write m
1 = P(0, 1) -
0
2 2 P(% y )
x=l y=-m
and use the identity 0
2 G(0,
x=-m
0
X)
-1 =
2 G(0, Y)2 P ( Y ,n).] Y
n=-m
(2) Particularize to the Bernoulli case: p ( 1 ) = 9, p(- 1) = 1 - 9 = q with 9 > 4, and show that for x < 0, G(0, x) = ( 9 - q)-l ($/q)".
Exercise 3.14. (1) Let X be a transient random walk on R or Z.Show that t-%([- t, t ] ) is bounded by 2c([-- 1, 13) for any integer t > 0. Extend the result to RP @ ZQ.
104
T R A N S I E N C E A N D RECURRENCE
CH. 4, $4
[Hint: Split up [- t , t ] into disjoint intervals of length 1 and apply the maximum principle.] (2) Assume that j 1x1 dp(x) < Q) and j x d,u(x) = 0 and show that the random walk is recurrent. Combine this and Exercise 3.13 to prove that the only recurrent Bernoulli random walk is obtained for $ = q = 4. [Hint: From the law of large numbers, there exists an integer n, such that n > n, implies P,[IX,I < ~ n > ] 4;then show that t-1
G([-
t, t])>
&(&-I
-n p )
and finish by using the fact that E is arbitrary.] (3) Show that the resolvent of brownian motion (ch. 2 $6.7) is recurrent in the sense of Harris.
Exercise 3.16. Let H be the smallest closed normal sub-group containing the support of ,u * ,& (where as usual 1; is the law of the reversed random walk). Show that H is the smallest closed normal sub-group such that the support of ,u is contained in a coset of H in G . Give an example in which H is a proper sub-group of G , or in other words in which Xis not aperiodic. The following exercises are designed to give a collection of easy results about spread-out probability measures.
Exercise 3.16. Give examples of recurrent random walks non-recurrent in the sense of Harris. Give examples of transient spread-out random walks which are not irreducible. Give examples of singular and spread-out probability measures. [Hint: For the last one, it i s possible to use the nilpotent group of upper triangular matrices, that is R3 with the multiplication law ( x , y, z) (x‘, Y’,2’) = ( x
+ x ’ , y + Y’, + z’ + x y ’ ) .
Check that Lebesgue measure is still the Haar measure and that a probability measure carried by the plane xOy and thus singular, can be spread out.]
Exorcisc 3.17. Call S , the set of points g such that a power , u p of ,u dominates a multiple of m on a neighbourhood of g. (1) The set S, is non-empty if and only if u , is spread-out; it is an open sub-semigroup of G contained in T,. Finally, prove that S,T, c S,. (2) For a closed semigroup T the following three conditions are equivalent:
CH. 3, 53
RECURRENCE OF RANDOM WALKS
106
(i) there exists a spread-out probability measure p such that T,, = T ; (ii) the interior of T is not empty; (iii) the Haar measure of T is strictly positive. (3) If p is spread-out, aperiodic and irreducible, for any f E C i , there is a real a and a power pp such that , u p 2 a(fm). Exercise 3.18. If G is connected, any spread-out probability measure on G is adapted. Exorcise 3.19. A random walk for which the a-algebra f of invariant events is a.s.trivia1 is spread out. Exercise 3.20. Let p be spread out and D , a set carrying the absolutely continuous component of pn and such that D: carries the singular component of pun.Show that pn(Dn)tends to one as n tends to infinity and derive the existence of an a s . finite stopping time T such that P,(x, ) is absolutely continuous with respect to Haar measure, for all x E G . [Hint:Take T to be the first time for X n to be in D,.]
-
Exercise 3.21. This exercise is designed to give a proof of Theorem 3.9 which does not rely on the theory of irreducible chains. (1) Using a countable basis for the topology of G , show that there exists a P,-negligible set N such that, for w 4 N and for every open set 0, W
2 7 d X n ( w ) )= n=1
CL)
(2) Let A be a set such that m(A) > 0. Set m
and show that m(B(w))= 0. Then, using Fubini's theorem, derive that for almost all g, the set
1
m
w :2
7&-1
X,(w)) <
n=l
is P,-negligible. (3) Prove the difficult part of Theorem 3.9. For that purpose one may use the stopping time of Exercise 3.20.
106
TRANSIENCE AND RECURRENCE
CH. 3, $4
Exercise 3.22. Show that if p is spread out, bounded harmonic functions are continuous (solution in ch. 5 $1). The result can be extended t o random walks on homogeneous spaces. Exercise 3.23. If X is irreducible with respect to Haar measure, the random walk induced on a homogeneous space is irreducible with respect t o quasiinvariant measures. Exercise 3.24. Suppose that p is an adapted probability measure on a discrete group G . If H i s a subgroup, either the random walk of law ,u on G / H (ch. 1 $4.4) is irreducible recurrent or all its states are transient. Using the group of positive affine motions of the real line prove that this result does not carry over to non-discrete groups. (See however ch. 4 w.12.) 4. Recurrence criteria for random walks and applications The computation of the potential kernels of the preceding section is not, as a rule, an easy matter. We need therefore a more tractable criterion t o decide whether a particular random walk is recurrent or not. A related problem is to find the groups for which there exist recurrent random walks. For abelian groups, the harmonic analysis of transition functions provides satisfactory answers to both questions. In this section, G will be a locally compact abelian and metrizable group. We denote by I' the dual group of G and 0 the identity in I'. If y E I',we denote by y(g) the value of the character y on the point g E G. The Haar measure of I ' will be denoted by m, or dy, and will be adjusted so that the inversion formula holds. We proceed to study a random walk of law p on G , whose Fourier transform will be written
Proposition 4.1. The law p is adapted if and only if 0 is the only point y E I' such that p(y) = 1. Proof. As 1 is an extremal point in the unit disc, if p(y) = 1 then y(g) must be equal to one p a s . , and therefore if G , = G , y(g) equals 1 identically. The converse is clear. In the following, 1 will stand for a positive real number less than one.
RECURRENCE CRITERIA
CH. 3, 54
107
It is convenient to remark once for all that if N is a symmetric compact neighbourhood of 0 in then
r,
Theorem 4.2. The random walk X is transient if and only if there exists a neighbourhood N of 0 in I' such that
Proof. Let N, be a symmetric compact neighbourhood of 0 in of the convolution in we get
r,
=
Ir
y(g) mr
(
~
I'. Making use
n yi ~ i d)y ;
taking the integrals of both members with respect to ,IL yields
The analogous formula being true for all powers of p, we get, by summing UP
9
Let N be a symmetric compact neighbourhood of 0 in r ' a n d choose N, such that N;c N . The function y -+m,(N, n yN,) is zero outside N, so the left-hand side of the preceding equality is a real number less than
(SNI
In the right-hand side, the function g -+ y(g) dy)2 is continuous and takes the value (mr(N1))' a t e ; it is therefore greater than 2-l(rn,(N1))2 on a neighbourhood V of e in G. We thus get
TRANSIENCE AND RECURRENCE
108
and if
CH. 3, 94
X is recurrent, then
Conversely, assume X to be transient, and let V , be a symmetric compact neighbourhood of e in G. The function (7,:
*
Iv1
*
(h) =
JG
mc(hV1 ngV1) P W
is continuous, positive definite, and in L1(G),so that it is the inverse Fourier transform of ~ ( y(J,, ) y(g) dg)2. We thus get
=
J G m G ( v lngv,) p(dg)
< m G ( ~ 1A)V ~ ) .
The similar inequality is also true for pn, and we get, by summing up,
Let us choose a symmetric compact neighbourhood N of 0 in 'l and the neighbourhood V1 such that C(e, V?) < cu and Rey(g) 2 9 for all (g x y ) E V l x N.We then have
and letting t converge to one, we get
Corollary 4.3. If X is transient, for every compact neighbourhood N of 0 in Re(1 - cp(y))-l is integrable on N .
r,
CH. 4, $4
RECURRENCE CRITERIA
109
Proof. For all t and y , Re(1- t p(y))-I is 2 0 and converges t o Re(1 - p(y))-l. One can therefore apply Fatou’s lemma to the effect that, with the same N as in the above proof,
Since Re((1 - p(y))-I) is continuous and non-zero outside 0, the result is true for all N .
Remarks 4.4. (1) This criterion is in fact necessary and sufficient for X to be transient, but the proof of the sufficiency relies on a profound study of potential theory for recurrent random walks, which will be dealt with in ch. 9. (2) If G is compact, ‘I is discrete, so that m,({O})> 0, and it follows from Corollary 4.3 that all random walks on G are recurrent. This is actually straightforward even if G is non-abelian. (See Exercise 3.11.) The remaining of the section is devoted to applications of the preceding criterion. We first consider the groups Rd*@ Zdm.We recall that their dual group is isomorphic to Rdl @ Td*.We shall denote by y the generic element of this group, y ( x ) its scalar product with x E G , and p(y) = JeiY(z) dp(x), the Fourier transform of p.
Proposition 4.5. T h e random walk of law p i s yecurrent in the following two cases : (i) G = R OY Z, J 1x1 dp(x) < 03 and J x dp(x) = 0. (ii) C = Rdl @ Zds with
Proof. (i) It is true that
Re
1-t (Re(1 - tp))2
(&)’
+ t2(Imp)2’
and
(Re(1 - tv))2 = [(l - t )
+ Ret(1 - p)j2< 2(1 - t ) 2 + 2t2(Re(l - p))2.
The hypothesis implies that p is continuously differentiable and that its derivative is zero in 0. Take E > 0; by Taylor’s formula there exists OL > 0
110
TRANSIENCE AND RECURRENCE
such that for Iy1
CH. 3, $4
< a we have
and thus
If X were transient, there would exist a. > 0 such that Re((1 - t v(y))-l) was positive on [- ao,ao] and
For a
< ao, we still would
have
but this contradicts the fact that E is arbitrary. The proof of (ii) is almost identical using polar coordinates and the open ball of radius a and is left to the reader as an exercise.
+
d2 > 2 , all the random walks are We now show that, conversely, if dl transient. For that purpose we need the following
Proposition 4.6. If G
=
Rdl @ Zdr,there exists a constant c suck that
on a neighbourhood of 0 in
r.
+
Proof. Since G is generated by the support of p, one can find d = dl d2 elements xl,. . . , xd linearly independent and belonging to the support of p. We set L = maxlxil; then the quadratic form
CH. 3, $4
RECURRENCE CRITERIA
111
is positive-definite. Indeed, if y # 0, we have y(x,) # 0 for a t least one xi and therefore y(xJ2 > 0 on a neighbourhood of xi;as a result Q(y) is strictly positive. The smallest, proper value A of Q is then strictly positive and Q(Y) 2 AlYl2. On the other hand, r
sin2 .by(x)dp(x). llXI
<
<
< L, then r, we have
< <
where S = { x : 1x1 L , l y ( x ) ( T ) . If (yI v L - l and 1x1 Iy(x)l IT; thus on the neighbourhood {y: IyI nL-l} of 0 in
<
r
which yields the result with c = 2 h r 2 .
Theorem 4.7. If d,
+ d, > 2 , all random walks on Rdl @ Zdmare transient.
Proof. Let P be a neighbourhood of 0 in
r
on which Re inequality of Proposition 4.6 holds. Then, for t < 1,
and therefore if d l
+ d,
is
3 0 and
the
> 2,
which implies that all random walks are transient. We now deal with the characterization of the groups for which there exist recurrent random walks.
112
TRANSIENCE AND RECURRENCE
CH. 3, $4
Definition 4.8. A group G (not necessarily abelian) will be called recurrent if there exists a probability measure p on the Bore1 sets of G such that the random walk of law ,u is recurrent. Compact groups, groups isomorphic to Itd1@ Zd* with d, d2 2 are recurrent, but the most general recurrent group is not yet known. We deal below with abelian groups, but the first lemma is true for all groups.
+ <
Lemma 4.9. A n open sub-group of a recurrent group is recurrent. I n @articular every sub-group of a discrete recurrent grou@ is recurrent. Proof. Left to the reader as an exercise.
Lemma 4.10. Let H be a compact sub-group of an abelian group G and u the canonical continuous and open mapping from G to GIH. Let p be a probability on G , and ,iiits image by u; then, if the random walk of law ,iii s recurrent, the random walk of law p is recurrent. Proof. Let I/ be a compact neighbourhood of the identity in G. The sets V H and a(VH) are then also compact neighbourhoods of the identity in G and G / H . Since, as is easily seen, m
m
0
0
2 p V H ) = 2 ,ii"(U(l/'H)), it follows that the random walk of law p is recurrent if the random walk of law ,ii is recurrent.
Theorem 4.11. A denumerable abelian group is recurrent if and only if its rank (maximal number of linearly independent elements) is at most 2 . Proof. If the rank of G is greater than 2, there exists a sub-group isomorphic to Z3which is not a recurrent group; by Lemma 4.9 the group G cannot be recurrent. Conversely, let us assume that, the rank of G is less than or equal to 2. If G is of rank 1, let a, be any element of infinite order; if G is of rank 2, let a,, a2 be two linearly independent elements and in all cases (rank 0, 1 or 2) let (an}nsNbe a sequence generating G and such that a,, is not in the subgroup G , generated by a,, . . ., a,. The fundamental theorem on finitely generated abelian groups asserts that, for every m,G, is of the form Z2 @ K or Z @ K or K , where K is a finite group. I t then follows from Proposi-
CH. 3, 54
RECURRENCE CRITERIA
113
tion 4.5 and Lemma 4.10 that all the random walks we are about to define on G , are recurrent. We define inductively a sequence p(,) of probability measures carried by G, by setting p ( l ) ( u l )= p(l)(- al) = 4, and for m 3 2,
,u'"'(g)
= (1 - Ym) p'"-l'(g)
if g E G,-,, and p(m)(a,) = p y -
a,)
=
ig,.
Every point g e G belongs to at least one G,; we can thus define
n m
=
(1 - Y i ) ptrn)(g)?
i=mi 1
and it is easily checked that this number doesmot depend on m. Now if the infinite product (1 - qi) is convergent, we have
n:=2
2 p(g) = lim 2 p(g) = lim n (1 - qi) = 1, m
gG G
m
m i=mtl
g€Cm
and ,u defines a probability measure on G. Plainly G is generated b y the support of ,u and we shall show that the gi may be choosen in such a way that p will be recurrent. Call Xm and X the random walks of law ,utrn)and p, Py and P, the corresponding probability measures. We choose once for all a sequence of numbers r , E 10, 1[ such that (1 - r,) converges. We choose a number N 1 and a sequence of numbers A f E 10, 1[ such that
n a,
j= 1
and we set g2 = min(r, A : ) . The random walk X 2 is then known and we choose N 2 and A ; such that
n W
N,
C
(1 - A ; ) ~ * P f [ X i = e]
3=1
> 1,
k=N,+-1
and we set g3 = min(r2, A ; , A:), which allows us to define X 3 . We follow this process inductively. Given N,-l and gnw1, we choose N , and A3 such that
114
TRANSIENCE AND RECURRENCE
CH. 3, $4
Such choices are always possible since the random walks X n are recurrent. Plainly, the infinite product (1 - qi) converges, and we have, G being the potential measure of X ,
nr
since m
Thus X is recurrent and the proof is completed. 4.12. The above result leads to a characterization of abelian locally compact metrizable recurrent groups, By the structure theorems, such a group i 2 is irottrarplilc t o 11" x Go, wliero Go ir a lorally coinpsct abellrm group containing a compact open sub-group K . From Lemma 4.10 the group G is recurrent if and only if G/(e} x K is recurrent. But this latter group is isomorphic to Itn x G , , where G , is countable and of rank Y . The group G is recurrent if and only if n I 2. Indeed, if n r 2, it contains the recurrent and dense sub-group Q* x G I . For instance, the groups Q2 x K , where K is compact, and the groups of @-adicnumbers, are recurrent.
+ <
+ <
Exercise 4.13. Show that ~ ( X , ) / r p ( yis) ~a complex martingale for P,. Exercise 4.14. If G is abelian, X is aperiodic if and only if J y ( y ) l< 1 for every y # 0.
CH. 3, $4
RECURRENCE CRITERIA
116
Exercise 4.16. Prove Exercise 3.12 by means of Theorem 5.2. Exercise 4.16. If G is discrete, T is compact. Show that one can then replace N by I' in Theorems 4.2 and 4.3. [Ziint : Express pn(e) by mean of inverse Fourier transform.] Show that, if G = Z d , the inequality 4.6 is true on the whole group
r.
Exercise 4.17. Show that, if the law p is symmetric (p = p), the criterion 4.3 is necessary and sufficient for X to be transient. [Hint: Notice that q is real.]
Exercise 4.18. If G is non-discrete, thus
r non-compact, and if p is spread
out, then
iiiii Idr)l < 1. Y-+m
This allows one to show that some non-discrete singular laws are not spread out. For example, the probability measure on R whose characteristic function equals cos(t/n!) is singular and is not spread out.
n:=l
The following exercises deal with recurrent, not necessarily abelian groups.
Exercise 4.19. With the notation of Lemma 4.10 but H not necessarily abelian, show that if p is recurrent, then ,6 is recurrent. Exercise 4.20. Every discrete group such that every finite subset generates a finite sub-group is recurrent.
Exercise 4.21. Let G be the free group with two generators a and b. (1) Define a law p on G by p(u) = p(b) = p(u-l) = p(b-') = 4. Show that the random walk of law p is transient. [Hint: If Igl is the minimal number of symbols a, b, u-l, b-*, necessary to write g E G, the function g + 3-lC1is superharmonic.] (2) More generally show that G is not recurrent. [Hint: Every denumerable group is isomorphic to a sub-group of a quotient group of the free group with two generators.]
Exercise 4.22. A group G is said to have the fixed point pro$erty if, whenever G acts continuously on a compact convex subset Q of a locally convex linear topological space b y affine transformations, then G has a fixed point in Q.
116
T R A N S I E N C E A N D RECURRENCE
CH. 3. $4
(1) Show that a group G has the fixed point property if and only if for every compact G-space M there exists a probability measure on M invariant by G. [Hint:For the “if” part, use the barycenter of a probability measure on Q invariant by G.] (2) Show that a recurrent group has the fixed point property and is therefore amenable. [Hint:See ch. 2, Exercise 4.18 and ch. 5, Exercise 1.10.1 Deduce from this fact that free groups with more than one generator and semi-simple Lie groups are not recurrent.
CHAPTER 4
POINTWISE ERGODIC THEORY
This chapter may be seen as the most important in the whole book. I t culminates with the celebrated theorems of Birkhoff and Chacon-Ornstein for which we will give or sketch three different proofs; they have many applications in Markov chain theory as well as in other fields which are thus related to one another. We will also touch on subadditive ergodic theory which has proved extremely useful in many situations.
1. Preliminaries We have already noticed that with a transition probability P we may associate two positive contractions on the Banach spaces b b and b A ( 6 ); these two contractions are dual in the duality between bounded functions and bounded measures, as is clear from the formula
In this chapter we shall deal mainly with the contraction on b A ( 8 ) . Among the closed subspaces of b A ( 8 ) , we find the space of bounded measures absolutely continuous with respect to a a-finite positive measure p ; this space is isomorphic to L1(E,6, p). We shall be interested in the subspaces of this kind which are invariant by P.
Proposition 1.1. T h e space of bounded measures absolutely continuous with respect to the a-finite positive measure p is invariant b y P if and only if pP << p. Proof. If pP << p and if p ( A ) = 0, then P(* , A ) = 0 p-a.e., and consequently for every f in L1(p)we have ( f p, P( * , A ) ) = 0. Conversely, since p is a-finite, there is a sequence {/,,I of elements of L1(p) increasing to I ; if p ( A ) = 0, the hypothesis implies that, for every n, we have (f, p, P( * , A ) ) = 0, and passing to the limit yields $'(A) 2= 0. 117
POINTWISE ERGODIC THEORY
118
CH. 4, $ 1
Remark. The condition p P << p is satisfied if p is excessive. It is also satisfied if p = 2-nvP,,, where v is any bounded measure. As a result, for any sequence of bounded measures, there is a measure p such that p P << p and stronger than any measure in the given sequence.
Definition 1.2. If p P << p the linear operator which maps f E L1(p)onto the Radon-Nykodim derivative of ( f p) P with respect to p is called the operator induced by P on L1(p). Proposition 1.3. I f ,UP << p, the operator induced by P i s a positive contraction. Proof. This follows at once from the fact that the map f --* (f p) from L1(p) into b A ( 6 ) is an isometry, and that P is a contraction on the latter space. The positivity is obvious. The adjoint operator of an operator induced by a T.P. on L1(p) is an operator on Loo(p)which is described in the following result.
Proposition 1.4. If p P << p, then P maps functions equal p-a.e. to functions in bB into functions of the same kind, and thus defines on L*)(p)a eositive contraction which i s the adjoint operator of the operator induced by P on L1(p). B is bounded above p-a.e. by a number M , the set A = (I > M ) has pP-measure zero; in other words, P( A ) = 0 p-a.e. and consequently Pf M p-a.e. In the same way it is seen that if f = g p-a.e. then Pf = P g p-a.e. The proof is now easily completed.
Proof. If f
<
E
a ,
Rather than study the operators just described, we shall study an arbitrary positive contraction T on a space L1(E,8,p). Its adjoint operator which is a positive contraction on L*(E, 8,p) will be denoted T*. We shall need the following lemma, which is obvious when T is induced by a T.P. For f in L1(p)and g in Lm(p),we shall set
and often drop the subscript p in the left-hand side when there is no risk of ambiguity.
PRELIMINARIES
CH. 4, 51
119
Lemma 1.6. Let {h,} be a sequence of elements of Lm(p) increasing p-a.e. to an elenient h of L*(p); then {T*h,} increases p-a.e. to T*h.
Proof.The sequence {T*h,) increases to afunctiong there is a non-zero element f in L: such that (1, g ) f in L: we have ( f ,g )
=
< ( f , T*h); but for every
lim(f, T*hn) = lim(T1, h,) n
n
=
< T*h. If p{g < T*h} > 0,
( T f ,h ) = ( f , T*h),
so that we get a contradiction. In what follows, the equalities between sets and functions must always be understood to hold up to p-equivalence, and we shall indulge in the usual confusion between functions and their equivalence classes. The powers of T and T* will be denoted T n and T*n. We shall use with respect to T* the notations and definitions introduced for transition probabilities. For instance we say that c p L",p) ~ is superharmonic if T*cp q~ p-a.e., in particular cp = 1 is superharmonic, and we set
<
02
P:
=z
2 (IAcT*)"I*. n=O
In ch. 2, the proof of the properties of P Aused only the linearity and positivity of the relevant operators. We thus have in the same way that P x l is the limit of the sequence defined inductively by go = l,, g, = 1 , v T*gn-l, hence that P,*l 1 and that limn (IACT*)" P,*l = 0. Moreover, if B 3 A , then Pip,* = PA*.
<
Most relationships between the notions for T* and the same notions for P , when T is induced by P,are of the same kind as that described in the following
Proposition 1.6. If T is induced by P, there i s a one-to-one correspondence between the space of T*-harmonic functions and the space of equivalence classes of bounded P-harmonic functions. More precisely, if T*h = h, there is a bounded function h' belonging to the equivalence class h such that Ph' = h'. Proof. We prove only the second sentence. Pick a bounded function g in the class h ; by Proposition 1.4, we have Pg = g p-a.e. Define go = g and gn+l
120
POINTWISE ERGODIC THEORY
CH. 4, $1
Pg,; this sequence decreases to a function g, equal t o g p-a.e. and g,. The function h’ = lim,, P,g, answers our question. such that Pg, = g, A
We will finally need the following Proposition 1.7. The operator T m a y be uniquely extended to a n operator, still denoted by T , on b,, in such a w a y that for every increasing sequence {f,} of functions in b,, T(lim, f,) = limn Tf,. Proof. Let h be a function in 8, and define Th as the limit of the sequence {Th,} where {h,} is a sequence in L\ increasing to h. If {g,} is another sequence increasing to h, we have lim Th, 2 lim T(h, A g,) = Tg,, n
n
since {h, A g,} converges to g, in L1. It is now clear that Th does not depend on the particular sequence {hn} and the proof is easily completed. The following proposition is, t o some extent, a converse to the preceding discussion. It shows that positive contractions of a space L1 are not much more general than those induced by transition probabilities. We recall that a compact class x‘ is a family of subsets of E such that, for every sequence {C, n 2 l } of sets in X with intersection C, empty, there exists an integer N such that C, is empty. I t is always possible to assume that such a class is closed under finite unions. The family of compact subsets of a topological space provides an example of compact class.
nnGN
n
Proposition 1.8. Su@fiose ( E , 8)is separable, and there is a compact class x’ such that u , has the approxilrcation property with respect to X , that i s , for every F in 8, p ( F ) = sup{p(K),K E X ,K c F ) . Then every positive contraction T of L1(E,8, p ) is induced by at least one transition 9robability.
Proof. Let us call B,, n 2 1, the elements of a countable boolean algebra Li? generating b. Since x’ may be taken closed under finite unions, there exists for every n an increasing sequence { C t } k > l of sets in x’ such that p(B,) = lim, ,u(CL). The boolean algebra 9 generated by GY and by the sets C: is countable and generates 8.
PRELIMINARIES
CH. 4, $1
121
For F E Q, we pick a function P ( F ) within the equivalence class of T*l,. These functions possess the following properties : (i) for every D E 9, 0 P ( * , D) 1 p-a.e. ; (ii) for every pair D, D' of disjoint sets in 9 a ,
<
P ( - ,D
<
+ D')= P ( - , D)+ P(
a
,
D') p-a.e.;
(iii) for every 12, the sequence {P(* , Ck)}k>l increases p-a.e. to P ( * , B,); this follows by applying Lemma 1.5 to the sequence 1.; which increases p-a,e. to I,,. Since 9 is countable, there is a negligible set N such that, for x E N c , the set function P(x,* ) satisfies the above properties, hence is a positive and additive set function on 37. We claim that it is a countably additive set function. Let { B j }be a subsequence of {B,}, decreasing to 0. For every j , every E > 0 and x E N C ,we may find a set C, E X n 9 such that C, c B , and
P(x, B,) < P(x, C,)
n
Since C, c the formula
+
2-jE.
n B , = 0, there is an integer J such that njsJCj = 0. From B.,
=
nB~ c u (B, - c,) jSJ
jSJ
we get
n
Since e is arbitrary it follows that P(x, B,) = 0 and therefore P(x, ) has a unique extension to a countably additive set function P(x, * ) on B. For x in N , we set P(x, ) = v( ), where v is any probability measure equivalent to p. By construction, the function P( - ,B ) is measurable for B E B, and by the monotone class theorem it is measurable for B E & . It is now plain that P is a transition probability on ( E ,8)such that ,UP << ,u and that T is induced by P.
-
-
Remarks. (1) If E is an LCCB space, or more generally a Polish space, and d is the a-algebra of Bore1 sets, and if p has the approximation property with respect to compact sets, the conditions of the preceding proposition are satisfied. (2) If p is a probability measure on Q and $9 is a sub-a-algebra of 8,the map f -+ E[f 1 91 is a positive contraction of L1(E,8,p) to which we may
CH. 4, $2
POINTWISE ERGODIC THEORY
122
apply the above result. In fact we may even choose the functions P ( . , F ) in 9 and the set N will be in 9. I t is then easily seen that the T.P. thus constructed has the additional property that P ( . , F ) is 9-measurable for every F E 6. Such a T.P. is called a conditional Probability distribution because for every f E b,,
Exercise 1.9. If ,u is excessive for P and T is the contraction induced by P on 1 . (For ,u unbounded, T1 is defined in Proposition 1.7.)
L1(,u), then TI
<
Exercise 1.10. Let T be a bounded linear operator on the space Lhof equivalence classes of complex integrable functions. Prove that there exists a positive (that is, mapping positive real functions into positive real functions) operator t on Lh such that (i) )Tfl tlfl for every f E Lb, (4 lkll IITII. For f E L;, t is given by tf = SUP JTgl.
< <
!RIG/
The operator t is called the linear modulus of T . Show that tnlfl2 ITnfl for every f in Lk. Exercise 1.11. Prove for T* the result similar to Proposition 1.7. Exercise 1.12. Retain the situation of Remark (2) below Proposition 1.8, but assume in addition that 9 is generated by a countable algebra d . Prove that for any x in E the atom A , containing x , that is the set { B : B E 9, x E B } is equal to { A : A E d ,x E A } and is therefore a set of 9. Prove then that the conditional probability distribution may be chosen with the additional feature that P(x, A,) = 1 ; it is then said to be regular.
n
2. Maximal ergodic lemma. Hopf’s decomposition
Theorem 2.1 (Maximal ergodic lemma). If f E L*(p) and
n
CH.4, 92
123
MAXIMAL ERGODIC LEMMA
then
r
for every su9erharmonic function 47 in L y ( p ) .
Proof. Let h ~ = s u p ( O , f , f + T,..., f f + T f +.*.+TNf); the sequence {hN}is increasing and the sets E N Since T is a positive operator, we have ThN 3 sup(0, Tf, Tf
=
{hN > 0) increase to E,.
+ T2f,.. ., Tf + T2f +
*
*
+ TN+lf)
and consequently
f +ThN>sup(f,f + T f , . * . , f + T f
+.'*
+ TNflf)*
The right term of this inequality being equal to hN+l on EN+1, we have
>
5
vhN+lap-
5
vhNdp>O*
By letting N tend to infinity we thus get the desired result. In the sequel, for every pair (f, g) of elements of L1(p) we set on(/, g) =
2
m=O
T m t / k Tmg m=O
on the set where the denominator does not vanish. Our goal is to describe the asymptotic behaviour of these ratios when n tends to infinity. Lemma 2.2. If f and g are in L:, the function g vanishes almost everywhere on the set F = {sup, Dn(f,g) = m}.
124
POINTWISE ERGODIC THEORY
CH. 4, $2
Proof. By applying the maximal ergodic lemma with rp = 1 to the function f - cg, where c is a strictly positive real, we find
1
Et
-c t
g dP
< c-lllfllP
The result follows by letting c tend to infinity, since F c Ef-cgfor every c.
Theorem 2.3 (Hopf's decomposition theorem). There exists a set C in 8,unique up to equivalence, such that for every f in L:, m
2 Tnf = 0
or
n=O m
n=O
T"f <
+
a3
+*
on C, on D
= Cc.
Proof. Let g be a strictly positive element in L1. For every f E L:, Lemma 2.2 implies that Tnf is finite on Tng < a}.If f is also strictly positive, it follows that
{c:=o
c:=o
{go
Tnk!
< a} =
{go
T"f
< *}
P-a.e.
{c:=o
The set C = Tng = a}thus does not depend on the choice of g, provided that g is chosen strictly positive. Let now h E L: be such that Tnh < 03 on a subset B of C ; then for every k , Tn(Tkh)< 03 on B, hence sup,, Dn(g,T k h ) = 03 on B, and by Lemma 2.2, Tkh = 0 y-a.e. on B. It follows that z2=o Tkh = 0 y-a.e. on B and the proof is thus complete.
cr=o
c:=o
+
+
+
Definitions 2.4. The set C is called the conservative part and the set D the dissipative part of E with respect to T . The decomposition of E into its conservative and dissipative part is called Hopf's decomposition of E. Finally, if D is empty, T is said to be a conservative contraction. We shall, for a while, restrict ourselves to the study of conservative contractions.
Proposition 2.6. Assume T to be conservative. T h e class %' the equivalence classes
of
sets which are in
MAXIMAL ERGODIC LEMMA
CH. 4, §2
C, =
125
I"C Tflf = a}, \fl=O
where f runs through L:, is a sub-a-algebra of 8.For a function h E L,", the followingproperties are equivalent (i) T*h h p-a.e. ; (ii) T*h = h p-a.e.; (iii) h i s %-measurable. I n particular, a set B is in V if T*lB = 1, on B and only if T*lB = I B everywhere.
<
Proof. Let h E L+" be such that T*h strictly positive; for every integer N ,
< h and pick a function f
in L1 and
Since T is conservative, this is impossible unless h = T*h p-a.e., which shows the equivaIence between (i) and (ii). I n particular T*l = 1. Call Z the sub-space of Lm of functions h such that T*h = h. It contains the function 1 and is closed under increasing limits. Moreover, if h and h' are in &, then h A h' is in &; indeed if a is a constant such that a h >, 0 and a h' 2 0 then
+ a + T*(h
+
A
h')
=
T*(a + h A h')
< (a + h)
A
(a
+ h') = a + h A h',
which gives our claim by the equivalence of (i) and (ii). Consequently, there a sub-a-algebra % of d such that X = bV. If B E V we have therefore T*lB = l B . On the other hand, if T*lB = 1, on B, then T * l p = 0 on b;, hence T * l p l B cso that, by definition of V , Bc, and hence B, is in V. Next, we get from Theorem 2.3 that pxiqtq
<
which proves that T*Ic; = 0 on C, and consequently that C, is in V . Finally, let B be any set in 5f and choose f in L: such that B = { f > 0); we have c2=o Tflf = + 03 on B,since the first summand is strictly positive on B. On the other hand, for every n 0, we have
( I Bcl TVf)
=
(T*'lBc, f ) =
(IBc,
f)
=0
POINTWISE ERGODIC THEORY
126
which implies that
cF=o Tnf
=
0 on B“, hence B
=
CH. 4, $2
C,.
Definition 2.6. The sets of %? are called the invariant sets. The %‘-measurable functions are called ilzvariant. If V is trivial p-a.e., the conservative contraction T is said to be ergodic. When T is induced by a transition probability P the superharmonic function for P are invariant. Finally we observe that all &-measurable p-negligible sets are in V. In the sequel we shall need the following property of invariant functions.
Lemma 2.7. If h is invariant and bounded, T(hf) = h(Tf) for every f ilz L1. Proof. We shall first show that T*(hg) = h(T*g) for every g in Lm.It suffices to prove this property when h = I , for B E %‘ and g = lA.We have T*(l,lE)
< inf(T*IA, T*l,)
= inf(T*IA, 1,) = IB(T*lA),
<
and in the same way T*(lAcIB) I,(T*lAc); adding these two inequalities yields that T*IB I,. Since by hypothesis T*lB = I,, this shows that the above inequalities are in fact equalities; thus T*(IAIB) = lB(T*lA). The lemma is then a consequence of the following equalities, where g runs through Lm
<
( V f ) , g)
=
(Tf, hg) =
(f* T*(hg))= ( f , h(T*g)) = ( W f ) , g).
The dissipative part may be viewed as a “transient” part, while the conservative part may be viewed as a “recurrent” part and the invariant sets as “recurrent classes”. The following result as well as Exercise 2.10 support this somewhat heuristic assertion.
Proposition 2.8. W e have T*l, where
=
0 p-a.e. on C . Moreover D = ess sup
Proof. Pick a strictly positive function f in L1.If Tf = 0 on a set A , it is easily seen that T * l , = 0 p-a.e. and it is enough to prove that T*lD,* = 0 p-a.e. on C. Equivalently we may suppose Tf > 0 on D. Choose then a sequence {Dk}of sets of finite measure increasing to D and for c > 0 set
CH. 4, 52
MAXIMAL ERGODIC LEMMA
I
1
Tnf
h, = inf 1, c In:1
127
.
Plainly, h, increases to 1, as c tends to infinity. On the other hand
2:=o
Tnf is infinite, it follows that T*(h,lDk)= 0 on C . since on C , the sum By letting c , then k, tend to infinity, we get the first part of the statement. Set now F , = { h , > a}; since h, > 2 I,,,, it follows from the above inequalities that T*l(l D k l F n )is a.e.finite. By using the complete maximum principle for T* as in ch. 2, Proposition 1.14, it follows readily that D is the union of a sequence of sets of a. On the other hand, if B E and f is strictly positive and in L1 then
czl
which is impossible unless B c D a.e. The proof of the second part of the statement is thus complete. If we refer to the case in which T is induced by a T.P., the above results say that the chain does not pass from C to D . We also have that if f is a positive function vanishing on D , then Tf vanishes on D ; indeed
We can therefore study the restriction of T to C , that is, to the functions vanishing on D , and this restriction is clearly a conservative contraction. In the sequel, by invariant functions we shall mean functions vanishing on D and invariant for the restriction of T to C , and the a algebra 'if will be a a-algebra of subsets of C. In this setting we prove
Lemma 2.9. For every measurable subset A of C there exists a smallest equivalence) invariant set A containing A , and
(UP to
where the conditional expectation is taken with respect to any Probability measure equivalent to p. Moreover, 1,- = 1 , P z l .
POINTWISE ERGODIC THEORY
128
CH. 4, $2
Proof. Since is complete with respect to p, it is a classical exercise to show that the set A = { E [ l AI U ] > 0 } does not depend on the particular probability measure used in the conditional expectation and that it is the smallest set in V containing A . Moreover even if A has infinite measure it follows easily from Propositions 1.7 and 2.5 that
and that this set is in %' and contains A , hence every n,
A . On the other hand, for
( I Z CT"1A) , = (T*"lIc, 1,) = (lac, 1,) = 0, which implies that =
A.
2; T n l , = 0 on A" and therefore that { c:=o Tnl, =
03)
In the same way as in ch. 2, it is seen that P z l is the smallest superharmonic function greater than 1 , ; it is therefore equal to 1,- on C.
Exercise 2.10. Use the results of this section to give another proof of some results in ch. 3 31, namely that there is a partition D, C,, C2,. . . of the countable space E such that: (i) Pn(x,y ) < co for any x in E if y E D ; (ii) Pn(x,y ) = co or 0 if x E Ci,according as y is in Ci or not.
c:=o
c:=o
Exorcise 2.11. (1) Let P be a T.P. possessing a bounded invariant measure p. Show that the contraction induced by P on L1(p) is conservative and moreover that it is ergodic if and only if p cannot be written as fhe sum of two different (non-proportional) invariant measures. If in addition E is countable all points with positive p-measure are recurrent. (2) (PoincarC's recurrence theorem), Let 8 be a measure-preserving point transformation of the probability measure space ( E , 8,m). Prove that for every set A with m ( A ) > 0 and for almost every x E A there exists an iniinite sequence of numbers ni for which On+) E A . (3) (Kac's recurrence theorem). The situation being that of (Z), set Y,(x)
=
inf{rt
> 0: On(%)
EA}
if this set is non-empty, vA(x)= 03 otherwise. Prove that
CH.4, 52
THE CHACON-ORNSTEIN THEOREM
[Hint: Show that {vA < a)= A , then use the formula mH, $4, which in the present case is an equality.]
129
< m of ch. 2
Exercise 2.12. Let T be a conservative contraction of L1(E,8, p). (1)Prove that a function f in L1 belongs to the closure of the image of I - T if and only if it is orthogonal to Lm(E,V, p ) . (2) Given that f and g are two functions in L1(E,&',/A), prove that n - l z ; : : Tk((f- g) converges to zero in the L1 sense as n ---* Q) if and only if, for every C EW, r
r
Exercise 2.13. Continuation of Exercise 1.10. Assume that t is conservative and let V be the relevant a-algebra of invariant sets. Prove that there is a function s in L," and a set I'+z% such that (i) Is1 = 1 p-a.e. on 'l and T f = S t ( s f ) for every f E L1(I') (we denote by L1(A)the set of functions of L1 vanishing outside A ) ; (ii) if d = E\r, ( I - T ) L 1 ( d )is dense in L l ( A ) ; (iii) this partition of E is unique up to equivalence; (iv) a function r satisfies the properties stated for s if and only if there is a function 1 E L,"(p) such that 111 = 1 p-a.e. on I', t*l = 1 p-a.e. on and Y = sl.
r,
Exercise 2.14. Prove the following converse to Proposition 2.5: if T*f (f, f E L $ , implies T f = f , then T is conservative.
<
Exercise 2.15. (1) Let P be a T.P. and p be such that pP << p. Let 8 be the shift operator of the associated canonical Markov chain. Prove that Z Zo8 is a positive contraction on L1(Pu)and call S its adjoint. (2) If C u V is a Hopf decomposition for the contraction T induced by P on Lf(,u),prove that (o: X,(o) E C} u ( 0 :X,(o) E D } is a Hopf decomposition for S . Show that S is ergodic if and only if T is ergodic. Exercise 2.16. If P satisfies the equivalent conditions of ch. 2, Exercise 3.13 for every x in E , and if the contraction it induces on Ll(E, 8, v) is conservative, the a-algebra V is atomic. Exercise 2.17. If T is conservative and induced by P , the a-algebra %? is equal
POINTWISE ERGODIC THEORY
130
CH. 4, $2
to the class of sets which may be written A U N where A is absorbing for P and p ( N ) = 0. [Hint: Use Proposition 1.6.1
Exercise 2.18 (Another proof of Hopf's decomposition theorem). We assume that,p(E) < 03. (1) For c > 1 and f E I,: with llfllm 1, set
<
m
zfEo <
Show that on {h, > 0 } , T*h, c2 for every integer N . Prove that the same inequality holds on {h, = 01, hence everywhere. (2) When c tends to infinity, the sets B , = {h, 2 c-l} increase to
prove that B belongs to the class a of Proposition 2.8. (3) Show that % is ? closed I under finite unions. Set D = ess sup 9 and C = D" and prove that for h E L y one has T*"h = 0 or 03 on C. Prove that T * l , = 0 on C. (4)Prove that the partition of E into C and D is a Hopf decomposition of E for T .
+
2;
Exercise 2.19. For f E L1 define inductively the sequence {f,} by fo
=
f,.
* *,
fn+l =
TI,+ - f,.
Notice that if T is induced by P, this is nothing else that the filling scheme applied to the pair (f+p,f-p), which makes some of the results below intuitively obvious. (1) Prove by induction that
2 Tkf+ k
z
f:
kin
=
2 Tkf- - 2 Tn-l-"f, k
2 0.
k
(2) Set f, = lim fi and notice that J f d p 2 - f , dp. Furthermore deduce from (1) that f, = 0 on the set E, and prove that this has Theorem 2.1 with = 1 as an immediate consequence.
Exercise 2.20. This exercise is designed to show that a property stronger
CH. 4, 93
THE CHACON-ORNSTEIN THEOREM
131
than the maximal ergodic lemma may be derived from the properties of the filling scheme. Take T induced by P and for f E L1(p)set A = f + p , v = f - p . (1) Prove that for every n, n
?I
2(fp) < 2 pk
0
Ik.
0
[Hint: Suppose the result to be true up to the order n - 1 and apply this induction hypothesis to f' = Tf+ - f-.] (2) Using the fact that P - v, < A prove that for any A c E,, one has vIA < A and that this implies Theorem 2.1.
3. The Chacon-Omstein theorem for conservative contractions In this section we study a conservative contraction and prove the ChaconOrnstein theorem; we will get Birkhoff's theorem as a corollary. The study of general contractions is postponed to $5. We begin with the technical
Lemma 3.1. Let g be a strictly $ositive element in L 1 ( p ) ;then for every f in L1(p)and integer k,
Proof. We can obviously make k measure gp, and for E > 0, set f o = f - Eg,. . ., f , =
We have f ,
=
=
0 in the proof. Denote by v the bounded
T"f - E
Tfn-l - Eg, and consequently
and summing up on n, we obtain OD
,
A , = {f,
> 0).
132
POINTWISE ERGODIC THEORY
By the Borel-Cantelli lemma we conclude that Y(= the proof.
CH. 4. 53
A,) = 0, which completes
Theorem 3.2 (Chacon-Ornstein theorem). If T i s conservative and g i s a strictly+ositive element in L1(,u),then for every element f in L1(p)the ratios Dn(f,g) converge almost everywhere to E [ f U]/E[g U] as n tends to infinity.
I
I
The meaning of the ratios of conditional expectations when p is not bounded will be explained in the proof.
Proof. We first assume that p is bounded. The principle of the proof is t o show the result for larger and larger classes of functions. (a) If f = hg with h invariant, then by Lemma 2.7 we have Tnf = k(Tng); hence on(/, g) = b. The result follows at once since
I %I = k E k I Wl. = ( I - T ) k with k E L', then E [ f I U ] = 0; indeed E[hg
(b) If f
(h, f ) = ( k , ( I - T ) k ) = 0
Tmf = k - Tn+lk,it suffices to show that for every invariant It. Since T n + l k / z ;Tmg converges to zero as n tends to infinity, but this follows from Lemma 3.1. (c) The subspace L, of L1 of functions of the form hg ( I - 7') k is dense in L1. Indeed, let I be an element of Lm orthogonal to L,. Then ( I , ( I - T ) k ) = 0 for every k E L1; which proves that T*l = 1, that is that 1 is invariant. Thus lg is in L,, and therefore we have also ( I , Zg) = Sl2g dp = 0, which shows that 1 = 0, since g is strictly positive. I t remains thus t o show the result for the functions in the closure C,, of L,. Let {f,) be a sequence of elements in L1 such that l l f a l l l < a,; we claim that the sequence supn JD,(f,,g)I converges to zero a s . Pick E > 0; by the maximal ergodic lemma applied to I/,] - Eg it follows that
+
2,
By the Borel-Cantelli lemma,
+
CH. 4, $3
THE CHACON-ORNSTEIN THEOREM
133
which proves that supn Dn(fv,g) converges to zero p-a.e. as p tends to infinity. Let now f be in L,; we may find a sequence of functions f v in L, converging to f in the L1-sense and such that - fPlll < a, (this may be achieved by taking a sub-sequence) ; we have therefore
c,, /If
lim sup D,(f - f , , g) = 0 a.s. f
i
n
But
Ion(/> g) - E [ f I V I / E [ ~1 %I I < lDn(fzug) - E [ f v 1 % I / E [I~%I 1
+ SUPIDnV -
fvs g)
n
I + IECf - f v I V / E [ gI W 1.
If we let n and then p tend to infinity, we see that on the right-hand side of the inequality the first term converges to zero because f ,is in L,, the second one on account of the above remark and the third one because the conditional expectation being a contraction for the L1-norm, we have IlE[f-f, I V]lll < a, hence the series 2, IE[f - f , I %is?as. I1 finite and the conditional expectations themselves converge to zero a.s. The proof is.thus complete for the case in which p is a bounded measure. When p is unbounded, the conditional expectations are no longer meaningful. For instance, if V contains an atom B such that p ( B ) = co, the conditional expectation of a function f E L: should be constant on B, which is impossible, since we should then have E[f U ] dp = 0 or a,,whereas 0 < Se f dp < co. This is due to the fact that the restriction of p to V is not bound to be a-finite, and therefore that the Radon-Nykodim theorem does not apply. This difficulty will be dealt with in the following way. Suppose now that p is a-finite, and let h be a strictly positive function in L1(p); call p’ the bounded measure hp which is equivalent to p. A function f is in L1(p’)if and only if hf is in L 1 ( p ) For . f E L 1 ( p ’ )we , define T‘f = h-lT(fh) ; T’ is a positive and conservative contraction of L1(p’), as is easily seen. Moreover, the spaces Lm(p)and Lm(p’) are identical, and the adjoint contraction of T‘ is still T*, since
2,
SB
I
+
The a-algebra of invariant sets is thus the same for T and T’, and for f,gELl(p’) such that (g > 0 } = E ,
1
I
lim D:(f, g) = E,,[f KI/E,,[~VI p-a.e. n
CH. 4, 53
POINTWISE ERGODIC THEORY
134
Let f , g be now in L1(p)and such that {g > 0) result to f / h and g/h, we find that
=
I
E ; by applying the preceding
I
lim on(/, g) = E , . [ f / h %j/E,,[g/h UI n
p-a.e.
The ratio on the right-hand side thus does not depend on the particular function h - this can be seen directly - and we say that it is equal to E,[f %]/E,[g V ] , thus defining the ratio of two conditional expectations even if conditional expectations themselves are meaningless. The proof of the Chacon-Ornstein theorem is now complete.
I
I
The hypothesis of the Chacon-Ornstein theorem may be weakened in several ways that we shall indicate in the sequel and in the exercises. We prove below that it is not necessary to take g strictly positive. Indeed if we call A = {g > 0 } , the set A being in 9,by the same argument as below Proposition 2.8, we may restrict T to A and then apply the Chacon-Ornstein theorem to the functions g and E[g U ] ,since the latter is strictly positive on A. We thus find that D,(g, E[g 1591) tends to I on A. If f is now any function in L1, by considering the product
1
we obtain, on account of Lemma 2.9, the following.
Thcorcm 3.3. If T is conservative and g positive, for every function f in L1, the ratio on(/, g) converges a.e. to E [ f U]/E[g %] on the set
I
I
If we think of the case in which T is induced by a T.P., we see that we have obtained a theorem about T , but that it is T* which has probabilistic significance. It is therefore advisable to prove an ergodic theorem for T*.To this end we shall need an additional hypothesis. Proposition 3.4. If h is a finite and strictly positive function (not necessarily in L1) such that T h h p-a,e., the space 2 = { f E L1:f/h E La} i s dense i n L1 and the map f 4 IzT*(f/h)from 3E" to L1 extends to a unique fiositive contraction S of L1. The HoPf decomfiositions are the same for S and T and the a-algebras of invariant sets are equal.
<
CH. 4, 93
T H E CHACON-ORNSTEIN THEOREM
135
Proof. Notice that f / h is in Lm if and only if there is a constant M such that If( M h a.e.; if g is in I.:, the functions g, = g A nh are in 3? and increase to g, hence converge to g in the L1-sense. I t follows that 2 = L1. 0 we have Next, for f E 2,set Sf = hT*(g/h);for f
<
and consequently the operator S may be uniquely extended into a positive contraction of L1(,u).Using the result in Exercise 1.11, it is easily seen that for the extended operator S we still have Sf = hT*(f/h). We now observe that S and T play completely symmetric roles. If f E LT and g E 2,we have (S*f, g) = =
(1, Sg)
=
(ft hT*(g/h))
(fk T*(g/h))= (T(fh),glh),
<
which proves that for f in 2 we have Tf = hS*(f/h). (Note that Sh h.) Let now C, (resp. C,) be the conservative part for T (resp. S) and f be a strictly positive element in L1. Set D, = { Snf < co} and let g be a bounded positive function such that D, = { g > 0) and (g, Snf) < co. By duality, this implies that
2:
2;
the series in the left-hand side diverges a.e. on the set D, n CT which perforce must be empty. It follows that CT c Cs and by symmetry we get C, = CT. Finally the invariant sets are the same for S and T . Indeed, assume S and T to be conservative and let g be a bounded invariant function for T ; on account of Lemma 2.7 it is easily checked that T(gh) = g(Th), hence that S*g = g(Th/h)< g ; the function g is thus invariant for S and we conclude once again from the symmetric roles played by S and T .
Remark. If T is induced by a T.P. P and if p is P-excessive, the hypothesis of the preceding proposition is satisfied with h = 1 (see Exercise 1.9); in that case, S = T* = P. If moreover there exists a T.P. P in duality with P relative to ,u the contraction S is the contraction induced by p . We now turn to the ergodic theorem for T*.
136
POINTWISE ERGODIC THEORY
CH. 4, $3
Theorem 3.5. If T is conservative, and if there exists a finite and strictly $ositive function h (notnecessarily in L1),and such that T h h p-a.e., then for every pair f , g of functions in L!+(h,u), the ratios
<
1
I
1
converge p-a.e. to E [ f h V]/E[gh U] on the set {E[gh U]
> 0).
Proof. By the above discussion, all we have to do is to apply Theorem 3.3 to the operator S and to the functions fh and gh. We will end this section by proving as a corollary to the Chacon-Ornstein theorem the fundamental theorem of Birkhoff which will be generalized in 56.
Theorern 3.6. Let 8 be a measure preserving point transformation of the probability measure space ( E , 8,m) and 9 the a-algebra of sets A such that A = O-l(A) m-as. Then for every function f in L1(m), n
lim (n n
+ 1)-1 C f
o
Ok = E [ f 1 9 1
0
m-as. and in the L1-sense.
Proof. Because of the invariance of m under 8, the map T : f -+f o 8 is a positive contraction of L1(m); it is conservative since f o Ok is obviously infinite for the function f = I which is in L1(m).Furthermore 9 is the +algebra of invariant sets for T ; indeed the sets { f o Ok = w}, f E L:(m), are plainly invariant by 0 and on the other hand if A = W ( A )m-a.e., then A = { 2 , " 1 A o Ok = w} m-a.e. The almost-sure convergence in the statement follows now by applying the Chacon-Ornstein theorem to T and to the functions f and 1. Iff is bounded the convergence holds also in the L1-sense by Lebesgue's theorem. The usual density argument then gives the L1-convergence for any f E L1(m).
2;
2;
Exercise 3.7. (1) Prove that in Lemma 2.7 one may drop the assumption that h is bounded, and take h merely %-measurable. (2) Prove that the Chacon-Ornstein theorem is still true if f , g are not supposed integrable but merely such that E [ f lU] < to and E[g IU] < 03. [Hint:f/E[f 1'31 is integrable.]
CH. 4, 93
T H E CHACON-ORNSTEIN THEOREM
137
Exercise 3.8. (1) Let f, g be in L: and set h = supn D,(f, g). Show that, for $ > I , one gets llhllp G
( P / ( P - 1)) l l / / ~ l l ~
where the norm is that of LP(E, 8,gp). \Hint: Prove that
1
.l h r a ) f d p > a l
( h > a )g
dp
and integrate this inequality with respect to the measure Pup-z da on [0, a[.] (2) Given that T is conservative and g strictly positive, prove that lim
o n ( / > g) =
n
I
I
EV % I / E [ ~VI
in the sense of LP(E, 8,gp), provided that f/g E LP(E, 8,gp). (3) By using the restriction of the Lebesgue measure to [l, m[ and the inequality a log b a log+ a be-', solve the same problem for p = 1 and prove that the convergence holds in the sense of L1(gp)provided that
<
+
I
fg-' log+(fg-') g dp
< m.
Exercise 3.9 (continuation of Exercise 2.13). Let g be a real, strictly positive function in L' and any element in L;. Prove that the ratio T k f / 2 frkg f converges p-a.s. to a V-measurable limit I(/, g). Moreover, for every C E%, show that
2;
Exercise 3.10. Prove that if Th and h is invariant.
< h and if T is conservative, then Th = h, 2:
Exercise 3.11. For A €8,define H , = I, (TIAc),. In the setting of Theorem 3.2 call Li the space generated by the functions H,g where A runs through 8. Prove that LL is dense in L1 and could be used in lieu of L, in the proof of Theorem 3.2. Exercise 3.12. If ,u is excessive for P and C U D is the Hopf decomposition of E for the contraction f -+ Pf of L1(,u)then PI, I , p-a.e. and PI, I, ,u-a.e.
<
<
POINTWISE ERGODIC THEORY
138
CH. 4, 54
Exercise 3.13. Retain the situation and notation of Theorem 3.6. (1) (Ergodicity). In accordance with $2 we say that 8 is ergodic if 9 is m-a.s.trivia1. Prove that the following three conditions are equivalent : (i) 8 is ergodic; (ii) if W ( A )c A m-as. then m(A) = 0 or m ( A ) = 1; (iii) for every pair ( A , B ) of sets in #-I
lim n-1 n
2 m(A n 8--k(B))= m ( A )m(B). 0
If 8 is invertible and 8-l is a measurable mapping, m is also invariant with respect to 8-l and 8-1 is ergodic if and only if 8 is. ( 2 ) If m, and m2 are two probability measures invariant by the same point transformation 8 and if 8 is ergodic with respect to both, then either ml = m2 or the two measures are mutually singular. (3) If 8 is the shift operator of a canonical Markov chain, under what condition on v does the space ( 9 , 9 P,,) , satisfy the conditions of (1)? (4) Use (1) and the space ( W ,d, P ) of ch. 1, Exercise 4.12 to give a proof of the law of large numbers. Exercise 3.14. Let P be a Feller T.P. on a compact metrizable space E. (1) Prove that there is at least one invariant measure for P. [Hint: Use the same device as in ch. 2 $4.18.1 (2) The following three conditions are equivalent : (i) P has only one invariant probability measure; (ii) for every continuous function f on E , the sequence n-l 2g-l P k f converges to a constant uniformly on E ; (iii) for every continuous function f on E , there is a sequence n, of integers such that the sequence ni' 2 : j - I Pkf converges to a constant pointwise. (3) The set of invariant probability measures for P is convex and compact for the topology of vague convergence. Characterize the extremal points of this set. [Hint: See (2) in the preceding exercise.]
4. Applications to Harris chains In this section we will apply the results of the previous three to the case of Harris chains. We will moreover study the relationship between conservativity and, the recurrence in the sense of Harris. Throughout the section
APPLICATIONS TO HARRIS CHAINS
CH. 4, 54
139
the a-algebra d is assumed separable although some of the results extend to the general case. We begin with
Proposition 4.1. If X is Harris with invariant measure m, the map f a conservative and ergodic contraction of Ll(m).
--f
Pf is
Proof. By the remark following Proposition 3.4 this map is the operator S of the preceding section. It is obvious that for f E L:(m), the sets C, ={ P,f = a }are m-a.e. equal to E or to the empty set according as m ( f ) > 0 or m(f) = 0, and this proves the proposition.
2;
As an immediate corollary we get
Theorem 4.2. If X is a Harris chain with invariant measure m and if f, g are ilz L:(m) with m(g) > 0, then lim E . n
2 KO
f(xk)
I/ Lo 1 E.
2 g(xk)
= m(f)/m(g) m-a.e.
Proof. Since P,f(x) = E , [ f ( X , ) ] the result follows a t once from Theorem 3.3 and from the preceding result. If X is an irreducible recurrent discrete chain, the empty set is the only set of measure zero, so that the above convergence holds everywhere. If f andg are characteristic functions of sets in 8, the preceding theorem expresses that the ratio of the mean times spent by the chain in these two sets tends eventually to the ratio of their measures. This result will be sharpened in ch. 6. We proceed with another application to Harris chains. Let X be a canonical chain recurrent in the sense of Harris, with invariant measure m. The map T : 2 -+T 0 8 is a positive contraction of L 1 ( 9 ,9, P,), since by the Markov property, we have
I
2 o 8 dP, =
5.
m(dx) E,[Z
o
131 =
I
m P(dx) E,[Z] =
I
Z dP,.
We claim that this contraction is conservative and ergodic. If m ( A ) > 0 and if 2 = 1 A ( X O ) , then by ch. 3 $2, m
On the other hand the sets
rn
CH. 4, $4
POINTWISE ERGODIC THEORY
140
are invariant by 8, hence equal P,-a.e. to $2 or to 0.
Theorem 4.3. Let j , g be in L:(E, 8,m) with m(g) > 0, then the ratios
2;f(xk)/x:g(xk) converge P,-as. to m(f)/m(g)for any Probability measzcrev.
Proof. Let A be the set of 0's for which the above ratios converge to the assigned limit, that is
A
=
i
n
EQ:
lim
f(xk(w)) 0
I:2
g(xk(0))
1
= m(f)/m(g) *
By the above discussion and the Chacon-Ornstein theorem we have P,[AcJ = 0. On the other hand it is easily seen that the equivalence class of the set A is invariant by 8, so that, by the results in ch. 3 $2, the function P.[A]is identically zero or one. The proof is now easily completed.
Itemark. If X is positive and m ( E ) = 1, we may take g = 7 and it turns out that n-l 2;t-l f ( x k ) converges a s . to m(f). This is the so-called "Law of large numbers for functionals of a Markov chain". We now want to investigate the converse to Proposition 4.1, namely the conditions under which a chain which induces a conservative and ergodic contraction on a L1-space is a Harris chain. We will need the following Definition 4.4. Let v be a probability measure on 8.The chain X is said to be v-essentially irreducible if for every f E 8, such that v ( f )> 0 we have U J f ) > 0 v-as. for one (hence for all) c E 10, l[. This condition is obviously weaker than irreducibility. I t is easily checked that a chain which induces a conservative and ergodic contraction on the space L ~ ( vof) a probability measure v is v-essentially irreducible; this is in particular the case for Harris chains if v is taken equivalent to m. We will now work in the converse direction and see how v-essential irreducibility relates to the v-irreducibility of ch. 3 $2.
Theorem 4.6. For a v-essentially irreducible chain X such that VP<< v, there are only two possibilities: (i) there is an absorbing set El such that v(EY) = 0 and the restriction of X to El is v-irreducible;
CH. 4, 94
APPLICATIONS TO HARRIS CHAINS
141
(ii) there i s an absorbing set E 2 such that v(Ei) = 0 and for all x in E2,the measures v and U , ( x , . ) are mutually singzclar for one, hence /or all, c E 10, I[.
Proof. Pick c E 10, 1[ and set
we claim that A is in 8.Let 9 be a denumerable algebra of sets generating d and put
9,,= {GE 9 :v(G) 2 l/n}; then v is not absolutely continuous with respect to Uc(x,* ) if and only if there is an integer n > 0 such that, for all B > 0, there is a G E 3 ' such that U,(x, G ) ,< 6 and v(G) 2 l/n. Consequently,
A" =
u H
{
I
x : inf Uc(x,G) = 0 , CE9r"
and since each Y nis countable the set A is in 8. Plainly A does not depend on the choice of c within 10, I[. We proceed to show that A" = { x : v I U c ( x , )}. Let x be a point in E such that UC(x, ) is not singular with respect to v ; if v ( F ) > 0 the resolvent equation and the v-essential irreducibility imply that, for d < c,
-
U d ( x , F ) >, (C - d)
1
Uc(x,d y ) U a ( y , F ) > 0.
Thus v <( U , ( x , . ), and x IS in A . Let x be a point in E such that Ud(x,A ) > 0. For every F in d such that v ( F ) > 0, we have U c ( F ) > 0 on A , and consequently a ,
U,(X,F ) 2 (C - d )
Uo(x,dy) Uc(y,F ) > 0 ; !A
it follows that v << U,(x,. ), hence that x is in A . Thus U,(. , A ) = 0 on A" and A" is an absorbing set. Moreover, we have either v ( A ) = 0 or v ( A C )= 0; indeed, if v ( A ) > 0, by hypothesis U,(. , A ) > 0 v-a.s., and since U c ( ., A ) = 0 on A", we get v(A")= 0. Thus either v ( A ) = 0 and (ii) is verified for E , = A", or v ( A C )= 0 and we shall show that (i) is verified for E l = { x : U c ( x ,A") = 0). Since vP <( v we have v U c << v ; hence v U c ( A C = ) 0, which proves that v(Ei) = 0. Moreover one easily sees that El is absorbing. Finally, since A" is absorbing, we have U,(. , A") > 0 on A";hence El c A and therefore v << U , ( x , - ) for any x in E l .
142
POINTWISE ERGODIC THEORY
CH. 4, $4
The second case in the above theorem is usually what happens for ergodic point transformations, the first one leads to the characterization of Harris chains which we are looking for. Roughly speaking a conservative and ergodic chain is Harris provided there is some absolute continuity of the T.P. with respect to the given measure.
Theorem 4.6. If P induces a conservative and ergodic contraction on L1(v), set A, = { x : U,(X,.) 1v, c €10, l[}. T h e n P i s Harris, after possible deletion of a v-null set, if and only if v(A:) > 0 for some, hence for a n y , c in 10, l[. Proof. The “only if” part is given by Proposition 4.1 and the results in ch. 3 52. To prove the “if” part we observe that we are in the setting of the preceding result, but that the hypothesis v(Az) > 0 rules out the possibility (ii). There is therefore an absorbing set El which carries v and the restriction of X to El is v-irreducible. By Theorems 2.3 and 2.5 in ch. 3 the chain is consequently either Harris (after possibly deleting a v-null set) or the potential kernel is proper. In the latter case we would be able to find a set F such that v(F) > 0 and C( , F) be bounded and this would contradict the conservativity of P. The proof is complete. We will finally use the foregoing results to study duality for Harris chains. In ch. 3, we saw that an irreducible recurrent discrete chain always has a dual chain which is also irreducible recurrent. Under suitable regularity assumptions we will prove the analogous property for Harris chains. Another proof yielding more, but relying on more elaborate notions, will be given in ch. 8. We begin with a preliminary lemma. Let m be a positive a-finite measure and N a positive kernel on ( E , a). We define a a-finite measure U Non c“ @I d by setting
Lemma 4.7. The density ii, of 17, with respect to m to the density n of N with respect to m.
@I
In i s equal m @I m-a.e.
Proof. Clearly IIN 3 n(m @ m ) , hence b 3 n m @ m-a.e. If this inequality were not an equality there would exist a positive non-negligible function ip on E x E such that
CH. 4, 54
APPLICATIONS TO HARRIS CHAINS U N 1 3
d m
143
c3 m ) ,
where N 1 is the singular part of N (ch. 1, $5); hence for all A
Letting A run through a denumerable algebra of sets generating b, we obtain that for almost all x , w x 9
dY) 2 qJ(%
Y) 4 d Y )
which contradicts the definition of N'.
Theorcm 4.8. If P and are two T.P.'s in duality with respect to the measure m, and i f P is Harris, then there is hodification of P which is also Harris.
Proof. Since P is Harris, the measure m is the invariant measure for P . Let
p , be a strictly positive density for U , with respect to m and 6,a version of the density of 6,. The duality formula of ch. 2 $4 between U , and 0, implies immediately that 17, = 17cc,hence that fi, = p c m @I m-a.e. On the other hand, by the discussion in the preceding section, the contraction induced by P is conservative and ergodic since that induced by P is. By Theorem 4.6, P is Harris up to a m-null set N , which can be stated by saying that there is a modification of P which is Harris. Indeed we get the desired modification by setting p'(x,-) = p(x,. ) for x 4 N and P ' ( x , . ) = 1 @ f m f o r x E N , w h e r e f E & ' + a n d m ( f )= 1. Finally, the existence of a dual T.P. is settled by the following theorem which applies in particular to Harris chains.
Theorem 4.9. .If the measure m is excessive for P and has the approximation property with respect to a compact class, there is at least one T . P . which is an duality with P with respect to m. Proof. The operator S : f -+ Pf is a positive contraction of L1(m).By Proposition 1.8, there exists therefore a T.P. denoted by such that mP <( m and Sf = d(fm)pldm. If f and g are in b b n L: (m) we have, confusing functions and equivalence classes of functions,
and therefore P is in duality with P with respect to m.
144
POINTWISE ERGODIC THEORY
CH. 4, $4
As a result of the foregoing discussions, every Harris chain on a Polish space such that m has the approximation property with respect to compact sets has at least one dual chain which is also Harris. It may have several which are modifications of one another; in most usual cases however, for instance random walks, one of them is naturally given and has a canonical character; it will often be referred to as the dual chain.
Exercisc 4.10 (Doeblin's ratio limit theorem). Let X be discrete and irreducible recurrent; then for any four states x , y, z, t show that
[Hint: Use the dual chain.] Exercise 4.11. Let X be a Harris chain and, with the notations of ch. 3 $2.14, raJIS, ttie positive contraction (Jf L'(irrj Indui,cxll hy P!,. Prove that for every
/ E L'(in), 7.
F'
Y
SJ
<
r
wr-a.e.
0
Exercise 4.12. Let G be a LCCB group; H a closed subgroup and p a probability measure on G adapted and spread-out. Set M = G / H and call Y the Markov chain on M with transition probability P ( x , . ) = p * E,. (1) Prove that if A is a Bore1 subset of M such that P1A = 1 , m-a.e., where m is a quasi-invariant measure on M , then either m(A) = 0 or m(A") = 0. [Hint: One may translate the hypothesis on G and use Proposition 1.6 together with the fact (ch. 5 $1) that bounded harmonic functions are continuous.] (2) If M = UF'=OMk, where h M k= 0 for every k , every compact set in M has a bounded potential. (3) When T p = G , prove the following dichotomy property for Y : either Y is recurrent in the sense of Harris with respect to m or the potential of any compact set is bounded. (4) Prove the same result with the following set of hypothesis: the measure m is relatively invariant that is there exists a continuous function x on G such that E# * m = x(g) m for every g E G , and C = s x ( g ) dp(g) 1. If C < 1 we are in the latter case and if C = 1 and we are in the former case then m is the invariant measure for Y . [Hint: Notice that m is excessive and use Exercise 3.12.1
<
CH. 4, $5
BRUNEL’S LEMMA
146
Exercise 4.13. Let p be a probability measure on a group G ; the random walk of law p (b) is supposed to be recurrent but not necessarily Harris. Recall that the convolution by p (F) is a positive contraction of Ll(m,). (1) Prove that these contractions are conservative. Derive that G( * , A ) = m,-a.e. if and only if m,(A) > 0; otherwise it is zero m,-a.e. (2) The corresponding a-algebras V are trivial, in other words, if f E bB and f = f * p m,-a.e., then f is constant mG-a.e. (3) Prove in this setting a result similar to Theorem 4.1. (4) Derive from (1) and (2) that if Y << m, and A E 6,then P , [ R ( A ) ]= 0 or 1 according as m,(A) is positive or zero. (See ch. 2, Exercise 3.13.) Exercise 4.14. Let X be a positive Harris chain and f be a bounded measurable function on (Ek,Bat). Prove that n-1
converges a s . and compute the limit.
6. Brunel’s lemma and the general Chacon-Ornstoin theorem In this section, we give another method of proof, which will enable us to state the Chacon-Ornstein theorem in a slightly more general fashion. This method also sheds more light on the relationship with the potential theory of ch. 2. Finally, it is easily extended to resolvents, as will be seen in the exercises at the end of the section. We could write this section independently of the results in $9 2 and 3, but this would involve repeating some definitions and proofs already given, and therefore we shall not do so. As before, T is a positive contraction of L 1 ( E , B , p ) . Theorem 6.1 (Brunel’s maximal ergodic lemma). For f
we have
fP2l dp >, 0.
E L1(p)and
a n y set
CH. 4, $5
POINTWISE ERGODIC THEORY
148
Proof. Let h be a strictly positive element of L', and assume that for E we have
"
> 0,
"
2 T k f3 E ZTkh 0
on A for infinitely many n. If g, =
f
=
Agm
xy
0
(TIAc)*f, we have
+ (TIAc)m+lf -k lACgm-
gms
and consequently "
+ (TIAc),+lf;this function is in L1(p). Dividing the 2F,oTkh,we easily see that, on the set A ,
Set i,b = I A g , equality by
'
Tk+
k=O
/k:O
above
2 Tkh2 + Tn+l(lAcg,)/k:O 2 Tkh E
for infinitely many n. It follows from Lemma 3.1 that A c E , ; also since PA*1 is superharmonic, the maximal ergodic Lemma 2.1 yields
I t follows that
2-
J
I ~ ~ ( I , C T * ) ~ + 'dp. PA*~ E
When m tends to infinity, the last term tends to zero, and the first one to Se fP,*l dp, which gives, in that case, the desired conklusion. Now the function f Eh and any set A c E; always satisfy the requirements of the beginning of the proof; thus
+
5
(f
+ ~4Px1 d p Z 0,
and since E is arbitrary the proof is complete.
CH. 4, $5
BRUNEL'S LEMMA
147
Our next step is to use the above result to show the existence of a limit for the ratios D,.
Proposition 5.2. If f E L1(p)and g E L i (p),the sequence {Dn(f,g ) } has p-a.e. a finite limit on the set Trig > 0).
{C:
Proof. Set A
=
{Trig > o} n {lim D,(f, g)
For every real number c
+ m}.
> 0, one has A c EiPcg;hence by
I
Since T*P;l
=
fP;ldp>c
5
Theorem 5.1,
gP,*ldp.
< P,*l, one has furthermore f P,*l dp 3 c
5
g T*" P,*l dp = c
5
Tng P,*l dp.
Since Tng is strictly positive on A and P z l 3 l,, by taking c arbitrarily on(/, g) is finite ,u-a.e. on large we see that p ( A ) = 0. As a result, W
0
Let a, b be two rational numbers such that a
< b. The set
is contained in Eig-, n E;-bg so that, by Theorem 5.1,
I
(ag - f ) P z l dp
I
2 0 and (f
-
bg) P,*l dp
summing up these two inequalities gives (a-b)
I
gP,*ldp>O,
which is impossible unless the integral is zero. But
> 0;
CH. 4,$5
POINTWISE ERGODIC THEORY
148
S which is
g T*nP,*l dp =
> 0 unless A is negligible. The proof
/Trig
P,*l dp
is thus complete.
We then derive Hopf's decomposition theorem.
Thoorom 6.3. There exists a set C , unique up to equivalence, such that for every f i n L:, m
2 Tnf = 0
or
+ co
on C,
0
cTnf < + m
03
on D
Cc
0
2;
Proof. Let It be a strictly positive element of L1, and set C = { Tnh = a}. If f E L:, the preceding proposition applied to D,(It, f ) implies that 2 ; T"f = +coonCn{C:Tnf >O};appliedtoD,(f,h),itimpliesthat Tnf < +a, on D . The proof is complete, since it is now plain that C is independent of the choice of h.
2:
As in $9 2 and 3, one may restrict T to C and define the a-algebra V of invariant sets. We are now going to identify the limit, the existence of which was shown in Proposition 5.2.
Proposition 6.4. If T i s conservative, the limit in Proposition 5.2 is equal to Elf V1l-m @I.
1
I
Proof. We assume that p is bounded; the extension to a-finite p, as well as the definition for that case of the ratios of conditional expectations, is dealt with as in Theorem 3.2. We may also assume f positive. Given two real numbers a , b such that 0 a < b < co, we will prove that
<
on the set A = ( a < lim D,(f, g) < b } ; this will imply the desired conclusion, because simple functions which approximate lim on(/, g) will also approximate p a s . the ratio of conditional expectations.
CH. 4, $5
BRUNEL'S LEMMA
149
Since this ratio is %-measurable we must prove that it lies between a and b on the set A, hence that for B E%, B c A we have
which is equivalent to
I.
(f - ag) d p
0 and
(bg - 1) d p 2 0.
Clearly, B D B n A ; on the other hand, the set B\B n A is invariant, contained in A and disjoint from A ; by the definition of it is thus empty and B = B n A . By Lemma2.9 we can then replace 1 , by P:n,l, and the above integrals may be written
5
Ip:nB1 (f - ag) dp and p,*nBI (bg - f ) dp; it follows from Theorem 5.1 and the definition of A that they are positive, thus completing the proof. Now let T be such that the conservative and dissipative parts are nonempty. For f , g E L:, it is plain that Dn(f,g) converges on D n { Tng > 0 ) to T n f l c : Thg. If f , g vanish on D , then the preceding result gives the limit on C. In order to find this limit when these functions do not vanish on D , we introduce the operator H , = I , (TI,)", ~ which ~ is adjoint to P: and similar to the operator 8, of ch. 2. For g E L:, the functions H,g vanish on D , and if g vanishes on D,then H,g = g. Furthermore
ocm
20"
Proposition 6.6. The ratio D,(H,g, g) converges to 1 on C n whenever n tends to infinity.
{c;
Tng > 0)
Proof. On C , the ratios D,(H,g, g) and Dn(TkHcg,Tkg)have the same limit. For a > 1, the set {lim D,(H,g, g) > a ) is contained in EL,_,. If A c {Tkg > 0) n {lim D,(H,g, g)
> u} nC,
Theorem 5.1 implies that
5
P,*l (TkHCg- aTkg)d p 2 0.
160
POINTWISE ERGODIC THEORY
CH. 4, $5
But on the one hand P,*l is harmonic for T*, since P i 1 = 1,- on C and T*P;I = P,*l on D,as is easily checked; on the other hand, since C 3 A , we have P,*P,*I = P:l. Thus we may write
(P,*I, T k H c g ) = ( T * k P i l , H,g) = ( P Z I , H,g)
=
( T * k P ; l , g) = (P;I, Tkg),
and consequently the above inequality involving integrals reduces to (1 - a ) i P , * l Tkgdp 2 0 ,
which is impossible unless P:l, and hence A , is negligible. That the limit cannot be less than one is shown in the same way. We are now ready to summarize all these results and state the general form of the Chacon-Ornstein theorem.
Theorem 6.6. If f , g are in Ll(,u),the sequence of ratios Dn(f,g) converges p-a.e. as n tends to infinity to
$ I
Tnf/$
Tng on D n
1
E [ H $ U]/E[H,g U] on C n
Proof. By the preceding proposition,
and since H,f 5.4.
and H,g vanish outside C, the result follows from Proposition
Exercise 6.7 (Ergodic theory for Abel sums). (1)Let T be a positive contraction of L1and set, for ;IE [0, 11, 03
G, =
2 A"Tn. 0
CH.4, $5
BRUNEL’S LEMMA
161
For f E L1, set EY = {%iA,l G,f > 0). Prove that E; is measurable, that E; c E;, and derive that if A c E;’, then
1
P,*l f dp 2 0.
(2) If T is conservative, then for f , g E L: the ratio DA(f, g) = GAf/G,g converges p-a.e., when ;Itends to 1, to E[f q / E [ g U ] on {G,g > 0).For the case in which T is not conservative, prove a result similar to Theorem 4.6. (3) Let {Va}a,o be a Harris resolvent with invariant measure m ; prove that if f , g are in L:(m), with m(g) > 0, then
I
I
lim (V,f/V,g) = m(f)/m(g) m-a.e. a40
Exercise 6.8. (1) In the situation of Exercise 2.20, Brunel’s lemma may be stated :
< (f’m) for any A c E;. (2) Prove that limD,(f,g) 1 on A , if and only if ( f m )P A < (gm). In particular this limit is 1 p-a.e., if and only if (fm)< (gm);this result is obvious for a dissipative T .
<
<
Exercise 6.9 (continuation of Exercise 2.19). This exercise describes another proof of the Chacon-Ornstein theorem for conservative contractions. (1) Suppose that T is conservative and that A = {k, > 0) is not empty. Prove that h: < co on A. [ H i d : Prove that hz (I,cT*)’7, d p h+(IACT*)p+n7A dp.] (2) For f , g in L: and b > 0, set B = {bE[g I > E [ f 1 U]}.By applying (1) and Exercise 2.19(1) to the function h = (f - bg), prove that fi on(/, g) b on the set B. Derive therefrom the Chacon-Ornstein theorem for conservative T .
2;
I
<
<
Exorcise 5.10. If T is conservative, Brunel’s lemma follows directly from Theorem 2.1 and the fact that the bounded superharmonic function P i ? vanishes outside E,.
CH. 4, $6
POINTWISE ERGODIC THEORY
152
6. Subadditive ergodic theory Throughout this section we will use the setting and notation of $3.6; we will often write Tf instead off o 8. We are going to prove that the convergence in Theorem 3.6 still holds under a weaker hypothesis. The result thus obtained has proved very useful in widespread situations and in particular in the study of Markov chains.
Definition 6.1. A sequence {s,, n > 0) of functions in respect to 8 ) if, for every pair (n, k) of integers > 0, sn+k
d sn
+
sk
L1
is szlbadditive (with
en.
It is additive if the inequality is replaced by an equality. Of course an additive sequence is nothing else than the sequence of partial sums Tksl. In all cases, it will be useful to set so = 0. If E has only one point, subadditive sequences boil down to subadditive sequences {a,,} of real numbers. For such sequences we have the following well-known
c;::
Lemma 6.2. If y
=
inf,{n-la,}
> - a,then lim n-la, = y. n
< +
Proof. Pick E > 0; one can find no such that nt'a,, y E . Any integer n may be uniquely written n = kn, + r with r < no. Thanks to the subadditivity of {aa}, we have n-la,
< n;lane + n-la,;
passing to the limit in this inequality yields y
< lim n-la, < Eii n-la, < ngla,,, < y + n n
E,
and since E is arbitrary the proof is complete. The theorem of Kingman which is the main goal of this section subsumes both Birkhoff's theorem and the above lemma.
Definition 6.3. A subadditive sequence {s,} is integrable if y = infn{n-'m(sn)} > - co. The number y is then called the time-constant of the sequence s,.
CH. 4, $6
SUBADDITIVEERGODIC THEORY
163
An additive sequence is always integrable and its time-constant is m(sl). In what follows we study a given subadditive and integrable sequence {s,} ;we will set k
P)k
= k-'
2
- TSi-l), k > 0.
(Si
i=l
Lemma 6.4. (i) limn n-lm(s,) = y ; (ii) m(vk)= k-lm(sk)2 y for every k
> 0.
Proof. Using the invariance of m with respect to 8, it is easily seen that the sequence {m(s,)} is subadditive, so that (i) follows from Lemma 6.2. The proof of (ii) is straightforward. We now turn to a very important result.
Definition 6.6. An exact minorant of {s,} is an additive sequence {t,} = { 2S-l T k f )such that {s, - t,} is positive and has time-constant zero. The function f = tl itself will also be referred to as an exact minorant. Proposition 6.6. For a subadditive and integrable sequence, any limit point of {pl,} in the o(L1,Lm)-topologyis an exact minorant. Proof. For an integer n n-1
k-1
n-1
i=O
i=t
j=O
2 Tiplk = (I - T") 2 si + 2 T ~ s ,
J =k
k-n-1
n
=
~i i= 1
since s , + ~ - Tnsi
J
<
< k , we have
i=l
< s,
n-1
i=k-n
j=O
for every i, we get n-1
n Si i=1
k-1
+ 2 (s,+~ - Tnsi)- 2 Tnsi + 2 Tjs,;
+ (k - n - 1) + 2 T'(Sk - T"-'Sk-,+j). S,
j=O
Using once more the subadditivity of {s,} we obtain
k
n-1
n
n-1
i=O
i=l
j=O
2 Ti($),< 2 si + (k - n - 1) s, + 2 Tfs,-,.
(64
CH. 4, 56
POINTWISE ERGODIC THEORY
164
Suppose that there exists a limit point f in a(L1, La) for the sequence {yk}. Then by Lemma 6.4 we have m(f) = y and passing to the limit in the inequality (6.1) yields, since T is a(L1, La)-continuous,
c
fl-1
Tjf
j=O
< s?a*
The existence of exact minorants could thus be proved by showing that there does exist limit points as in the preceding result; unfortunately this cannot be yet done in a completely elementary way and we are going to perform the proof in a slightly different fashion. However, in practical situations the above proposition allows one to find explicit exact minorants.
Theorem 6.7 (decomposition theorem). Every subadditive and integrable seqwnce has at least one exact minorant.
2;t-l
Proof. By subtracting if necessary the additive sequence { Tisl} we may suppose that {s,} is negative. By the criterion for relative a(L1, La)compactness of sets in L1(m),we may extract from {pm} a subsequence {ym,} such that for every integers i and p , the sequence {Tiymkv (- p ) } converges in o(L1,La) to a function ,lip. The sequence {Aj,} is obviously decreasing in p for every z and we may therefore set Ai = lim, lip.It follows from eq. (6.1) that n-1
2 < sn Ai
1-0
for every n 2 1. Now, because T is a(L1,Lm)-continuousand positive,
TIi,
=
T (lim (Tiymkv (- p ) ) ) = lim (T(Tiymkv (k
k
3 lim (Ti+lymkv k
(-
p ) ) = l(i+llp.
Consequently TIi 2 ,li+l, and we can write
where all the summands are negative. Let us put 00
f
=
+C i=O
(At+,
- T&);
p)))
SUBADDITIVE ERGODIC THEORY
CH. 4, $6
y by Lemma 6.4. The function f
the function f is negative and m(f) = m(l,) is thus in L1(m) and moreover
cTif < cli <
n-1
n-1
i=O
i=O
166
,s,
which proves that f is an exact minorant.
Remark. The above decomposition is not unique as may be surmised from the fact that any limit point of {vk}in the a(L1, L")-topology will do. Actually even when {Vk} is convergent, there still may exist several exact minorants, as will be seen in Exercise 6.11. The preceding theorem allows us to prove the convergence theorem which is the main result of this section.
Theorem 6.8 (Kingman's theorem). The sequence {n-'s,} has a limit [ m-a.s. and in L1. The limit [ is §-measurable and for any A E .%
Proof. By Birkhoff's theorem and the decomposition theorem above it is enough to prove that for a positive subadditive sequence {s,} with time constant zero, {s,/n} converges to zero m-a.s. and in L1 and for this it is enough, since s,/n sl, to prove that En (s,/n) = 0 m-a.s. Pick an integer k ; any integer n may be uniquely written n = p k r with r < k. Using the subadditivity we have
<
+
n-Is,
<
P-1
2 Tk'sk +- n-lT%,..
n-1
i=O
As n goes to infinity, the last term on the right of this inequality goes to zero; indeed, for any e > 0, because of the invariance of m by 8 we have
c W
m(n-lTPksr 2 e)
n= 1
W
sup s, 2 E n < 2 m (o
<
E-'
(
sup s,) dnt
O
< co,
POINTWISE ERGODIC THEORY
166
CH. 4, 46
and the result follows from the Borel-Cantelli lemma. Passing to the limit in the above inequality thus yields n
(Sn/n)
<
E[(Tksk/k)
14
and consequently
5
(s,/n) dm
< k-lm(sk).
Since k is arbitrary the integral on the left is less than or equal to the time constant which is zero; this completes the proof. Remark. If 6' is ergodic the limit in the preceding theorem is equal to y. It is more generally equal to E[f I 4 where f is an exact minorant of s,. I t is important to observe that in some applications a limit point of in the a(L1, Lm)-topology may be computed and thus provides the value of the limit; this is in particular the case when { v k } or, even better, {s, - s,-~ o e} has a limit.
{vk}
We must finally record that some applications of the foregoing results may be obtained in the following setting.
<
Corollary 6.9. Let {Vk,l, 1 k < 1} be a f a m i l y of doubly integer indexed random variables such that (i) V k S v Vk.2 V 2 . Dwhenever k < 1 < p ; (ii) the joint distribulions of thg sequence {vk+l,L+l] are the same as those of the sequence { vk,L}; (iii) the expectations of the variables V l , nexist and inf, ta-lE[V1,,] > - 00. T h e n limn n-lVl,n exists a.s. and in L1.
<
+
Proof. Let ( W ,d ,Q) be the probability space on which the variables V k P l are defined. Let E be the space of all real valued functions x defined on the set of pairsof integers (k,Z) withO
+
CH. 4, $6
SUBADDITIVE ERGODIC THEORY
167
This corollary is actually equivalent to Theorem 6.6 as is seen in
Exercise 6.10. Derive Theorem 6.8 from Corollary 6.9. Exercise 6.11. In the setting of ch. 1 $4.12 suppose that G = R and that Set Y , = 2 7 and X , = Y,'. p = +(tl (1) The sequence {X,} is subadditive relative to the shift 8 on W and its time constant is zero. Prove that for any a E [0, 13 the sequence {aY,} is an exact minorant of X,. ( 2 ) Prove that the sequence { X , - X,-l o e}, hence the sequence { V k } associated with {X,}, converge in a(L1, Lm)to Y 1 / 2but do not converge in the L1-sense. This shows that the convergence in a(L1, L") of the sequence {qk}is not a sufficient condition to ensure uniqueness in the decomposition theorem.
+
Exercise 6.12. If E ,
=
Unal{s, > 0 } , then JBs
s1 dm
3 0.
Exercise 6.13. Let { Y,} be a sequence of independent, equally distributed strictly positive, random matrices and call x y the (i,i)" entry of the matrix X , = Y I Y z . Y,. State an integrability condition under which limnn-l log xf exists as. and is independent of (i,i).
--
Exercise 6.14 (law of large numbers for random walks on groups). Let G be a group generated by a compact neighbourhood V of e. Set 6(g) = inf{n
> 0: g E V"}.
If p is a probability measure on G such that
* I
lim n-16(X,) = lim f l
s 6 dp <
03,
prove that
6 dpn P,-a.s.,
where X is the right random walk of law p.
Exercise 6.16. Let A E 8 and m be excessive for the Markov chain X . Prove that the function g on N defined by g(.) = P,[S, a] is subadditive. Suppose further that m ( A ) < 03 and prove that n-lg(.) has a finite limit L as n tends to infinity.
<
158
CH.4, $6
POINTWISE ERGODIC THEORY
(2) If X i s in duality with 2 with respect to m and if PI = 1 then L = C ( A ) where C ( A ) is the capacity of A defined in ch. 2, Exercise 4.13. [Hiat: Compute g(n) - g(it - 1) - e ( A ) and prove that this quantity goes to zero as n tends to infinity.] (3) The chain X is henceforth a right random walk on a group G and m a right Haar measure. Put
v,(w) = m
e:
U {x:X x k ( 0 ) E A}
and prove that E,[V,] = g(n) for any y E; G . The reader will visualize the above set in relation to the set “swept out” by the random walk when started at e. If G is discrete and A = ( e } , V,(w) is the cardinality of the range of X, that is the set {X,(w), X,(w), . . ., X , ( W ) } . (4) Prove that n-lV, converges a s . to C ( A ) . If X is Harris this limit is zero; explain this fact. ( 5 ) Prove that the sequence { V,, - V,-l o 0) has a limit pointwise and find an exact minorant of {V,}. Use this to give another proof of the above results.
Exercise 6.16. (1) For an additive sequence (s,} put M , = S U ~ ~ < ~ ~ , S ~ I , = infO 0 (see ch. 3 $3.12) and set I = inf,(X,). Prove that
+
1=
In
I+dP,.
+
CHAPTER 5
TRANSIENT RANDOM WALKS. RENEWAL THEORY
This chapter is devoted to the study of renewal theory for transient random walks on abelian groups. For non-abelian groups the theory is now almost as complete as in the abelian case but it lies beyond the scope of this book. In the first section we show the fundamental theorem of Choquet and Deny. The renewal theorem will be shown in $53 and 4, and will be applied to the study of hitting probabilities and recurrent sets in the last section.
1. The theorem of Choquet and Deny For a random walk on an abelian group, the theorem of Choquet and Deny asserts that all bounded continuous harmonic functions are constant. We shall need the following
Lemma 1.1. Let TI and T , be two commuting contractions of a Banach space B. Zf a1 and a2 are two positive real numbers such that al a2 = 1 and (alT1 a,T,) x = x for some x E B , then T,x = T,x = x .
+
+
Proof. If T is a contraction of B, then limll(Z - T ) exp[- n ( 1 - T)ll = 0. n
Indeed, exp[- n(Z - T ) ] =
2aj,Jf, i>O
where aj,* = nj/enjl. For fixed n, the sequence aiSnincreases up to j = n and then decreases. Therefore
< 2i
))(I - T ) ~ x P [ - n(I - T)I1)
- aj-1.n) < 2an.m
and the Stirling formula shows that an,,,tends to zero as n tends to infinity. 169
160
CH. 5, $1
T R A N S I E N T RANDOM W A L K S
Let us now set S = alTl
+ azT,; as TIand T, commute, we have
exp[- a l l ] exp[ai'S] = exp[- ( I - T,)] exp[I - a;']
exp[(a;'
- I ) T,],
and taking the nth power, exp[- na;l] exp[na;'~] = exp[- n ( ~ TI)]U n , where U is a contraction. From Sx = x we get x = exp[-- n ( l - T1)] Unx,
(I- Tl)x = (I - T,) exp[- n ( I - T,)] Unx, and letting n tend to infinity yields ( I - T,)x
=
0.
We shall now prove the theorem of Choquet and Deny in a slightly more general setting. Let S be a locally compact separable semi-group, and p a probability measure on S. As usual, a Bore1 function h on S is called pharmonic if
Theorem 1.2. If S is abelian and if g belongs to the su##ort of p, then h(xg) = h(x) for all x E S and all bounded continuous p-harmonic functions.
Proof. Let g be a point in the support of p and V n ,n E N,a basis of neighbourhoods of g. For each n, the restriction pn of p to V , is non-zero; hence if h is a bounded, continuous harmonic function, we can write
1
I 1Ipnll-l pn(dy) h(x) = I I ~ ~ I h(xy) S
+ I I -~ pnll j
S
h(xY) I
I -~ Pnll-1 (p - pn) (dy).
For every probability measure v on S, the mapping h Th(x)
=
5
4
Th, where
W Y ) 4dY)
is a positive contraction on the Banach space of bounded continuous functions. The semi-group S being abelian, we may apply the preceding lemma and find
that
CH. 6, $1
161
CHOQUET AND DENY’S THEOREM
But as n tends to infinity, the sequence II,unll-l ,u, converges weakly to eg, so that h(x) = h(xg) for all x E S,and the proof is complete. From now on we study a random walk of law ,u on an abelian separable locally compact group G. We shall denote by e or 0 the identity of G,and b y m a Haar measure of G . We suppose that, as usual, p is adapted. In this context, the preceding result reads as follows.
Theorem 1.3 (Choquet-Deny theorem). The bounded continuous harmonic functions are constant.
+
Proof. I t is easily checked that the set of points y in G such that h ( x y ) = h ( x ) for all x in G and every bounded continuous harmonic function h is a closed sub-group of G . The result thus follows immediately from Theorem 1.2. This theorem implies the following corollaries.
Corollary 1.4. If v is a n invariant measure for X (i.e. ,u * v = v), and if the set { E *~ v ; x E G } i s vaguely relatively compact, then v is a Haar measure. Proof. For any pl E C i , the function f defined by
is bounded and continuous. Furthermore, f is p-harmonic, since
The function f is therefore constant and it follows that v is a Haar measure.
Corollary 1.6. The bounded harmonic functions aye constant m-a.e.
Proof. If h is bounded harmonic, the measure v = hm satisfies the conditions of Corollary 1.4 for the dual random walk. Indeed j2
* (hm) = (@ * h) m = hm,
162
TRANSIENT RANDOM WALKS
CH. 5. $1
and it is easily checked that {E, * v, x E G } is vaguely relatively compact. Therefore v is a Haar measure and h is constant m-a.e.
Remark. These results may be carried over to some classes of non-abelian groups but are not true for every group, as is shown in Exercise 1.10. When the law p is spread out, matters become even nicer: all harmonic bounded functions are constant, so that the a-algebra of invariant events is trivial (ch. 2 $3). This follows from the following important proposition, which does not require G to be abelian.
Proposition1.6. I f p is s+read out, every bounded harmonic function is continuous.
+
Proof. Choose n such that pn = a B, where a is absolutely continuous with respect to m and p singular with 11PIl < 1. Call a p and p, the absolutely continuous and singular parts of p n V ; it is easily seen that llppll < l,4?ll”. If h is bounded and harmonic, for every we have
+
The function h * 8 , is continuous, since it is the convolution of a bounded function and an integrable one. The function h is thus the uniform limit of continuous functions, and is therefore continuous. In the subsequent section the Choquet-Deny theorem will be used as a tool in renewal theory. Exercise 1.11 and ch. 6, Exercise 1.14 give an insight into the relationship between the Choquet-Deny theorem and the limit theorems for powers of transition functions. This relationship is necessary to enlarge the scope of validity of the Choquet-Deny theorem to certain classes of non-abelian groups, but it will not be used in the remainder of the book.
Exercise 1.7 (another proof of the Choquet-Deny theorem). (1) Let (B be a p-harmonic bounded function. Prove that for every x e G the sequence of random variables pl(x + 2, + + 2,) defined on the probability space (W, d ,P ) of ch. 1, Exercise 4.12 is a martingale which converges a.s. and in the mean to a random variable H , which is symmetric and thus constant a.s. The reader will notice that the fact that G is abelian is necessary.
CH. 6, 51
CHOQUET AND DENY’S THEOREM
+
(2) Derive from (1) that ~ ( x 2,) = cp(x) a s . ; hence v ( x for palmost every x’. (3) Prove the Choquet-Deny theorem.
163
+ x’) = ~ ( x )
Exercise 1.8. (1) Let X be a random walk on an abelian group G . Prove that F is P,-trivial whenever v is absolutely continuous with respect to m. (2) Derive that for a recurrent random walk (not necessarily Harris) the function P.[R(A)] = 1 m-a.e. or 0 m-a.e. according as m(A) > 0 or m ( A ) = 0. (3) If 9 is a s . trivial, then p is spread out. (See also ch. 7, Theorem 1.13.) Exercise 1.9. A continuous homomorphism of an abelian group G into the multiplicative group of positive real numbers is called an ex#mnentiaZ. Show that extremal (cf. ch. 7 $3)positive harmonic functions are exponentials. Exercise 1.10. The theorem of Choquet-Deny is not true for all groups. Every element in the group G = SL(2, R) of real, unimodular matrices of order 2 can be written uniquely as the product of an element of the group K of rotations by an element of the group S of upper-triangular unimodular matrices (a special case of the Iwasawa decomposition of semi-simple Lie groups with finite centre). The circle S1 may be identified with CIS, and is thus a compact homogeneous space of G . (1) Prove that there does not exist any measure on 6 invariant by G . (2) Let p be a spread-out probability measure on G . Prove that there exists at least one probability measure v on S1such that y * v = v, and derive the existence of non-constant y-harmonic functions. [Hint: See ch. 2, Exercise 4.18.1 Exercise 1.11. If p is a probability measure on a group G such that lim,lJpn * vII = 0 for all v << m and such that v ( E ) = 0, then the Choquet-Deny theorem is true for the left random walk of law p on G . What can be said if the assumption v << m is dropped ? Exercise 1.12. If y is spread-out deduce Theorem 3.3 of ch. 3 from the ChoquetDeny theorem.
+
Exercise 1.13. For any g E S1 the map x -+ g x from S1into itself preserves the Lebesgue measure on S1. Prove that this map is ergodic (ch. 4 $3.13) if and only if g is irrational. Generalize to the n-dimensional torus.
CH. 6, $2
T R A N S I E N T RANDOM WALKS
164
2. General lemmas
From now on, the random walk X is supposed to be transient. According to ch. 3 5 4, the potential measure G is then’a Radon measure, the asymptotic behaviour of which we are going to study. Throughout the sequel, f stands for an arbitrary function in C i .
-
Proposition 2.1. (i) The set of measures {G(x, ), x com+act. (ii) The function Gf is uniformly continuous.
E
G } is vaguely relatively
Proof. The statement (i) was proved in ch. 3, Corollary 3.6. For every compact K , there exists therefore a constant MK such that G(x, K ) MK for all X E G. Let H be the compact support of f . Since f is uniformly continuous, for every E > 0, we can pick a compact symmetric neighbourhood J of e, such that If(x y ) - f ( x )I < E for y E J and all x E G. For y E J , the function g( * ) = f ( * + y ) - f( * ) vanishes outside the compact K = H + J , and it is easily checked that Gg(x) = Gf(x y) - Gf(x). We thus get, from (i),
<
+
+
for all x
E
G and y E J , which yields (ii).
-
We now aim at finding the closure of the set {G(x, ), x E G } in the vague topology. Since the mapping x + G ( x ; ) is continuous, as follows from Proposition 2.1, this i s equivalent to the problem of finding the limit points of the set {G(x, * ), x E G } when x tends to A , the point at infinity in the Alexandroff compactification of G .
Theorem 2.2. The limit 9oints of the set {G(x, * ), x E G } when x tends to A are Haar measures. Moreover they always include the zero measure.
Proof. Let {x,} be a sequence of points in G , converging to A and such that the measures G(xn, ) converge to the measure v in the vague topology. Now the measures E,, * G * p converge vaguely to v * p ; indeed (&a+,
*G*
f>
=
Gf(xn
+ Y )P ( ~ Y )
CH. 5, $2
GENERAL LEMMAS
<
165
+
and since IGf(xn+ y)l llGfll and Gf(x, y ) converges to (v * E,, f ) for every y , our claim follows from Lebesgue's theorem. Passing to the limit in the equality Exn
* G = Exn + E"" * G * p
yields v = v * p. I t is now clear from Proposition 2.1 that v satisfies the hypothesis of Corollary 1.4 and is therefore a Haar measure. To prove the second sentence take f non-zero and observe that by ch. 2 $1.8, we have limn Gf(Xn) = 0 a s . Since every compact set is transient and P is markovian, we may consequently find a path o such that limn Gf(Xn(w))= 0, limn Xn(o)= d and Xn(w) # d for every n. By the first sentence, the measures G(Xn(w),* ) converge to a Haar measure which gives the value zero to f and is therefore the zero measure. The proof is complete.
Remark. Using a countable dense subset of C i , it is easy to see that the second sentence is still true for non-abelian groups even though the first sentence is not. The second sentence is actually true for any markovian T.P., in a suitable analytical set-up. I t is no longer true if the T.P. is not markovian as is shown by important examples in ch. 9. We will now work to prove that there is a t most one limit point beside the zero measure.
Lemma 2.3. For every com9act K , lim(Gf(x
+ y) - Gf(x)) = 0
x+A
zcniformly in y E K .
Proof. After Propositions 2.1-2.2 every sequence converging to A contains a subsequence (xn} such that the measures G(x,, * ) converge in the vague sense towards a Haar measure um. Then lim(Gf(x,
xn+A
+ Y) - Gf(xn))
=
a((&,* m,f ) - (m,f ) ) = 0,
which shows that the limit in the statement exists. For every E > 0, there exists a compact K , such that IGf(x y) - Gf(x)l < &E for x $ K,. Moreover, Gf being uniformly continuous, for every E > 0, every y E G possesses a neighbourhood V , such that z E V , implies
+
TRANSIENT RANDOM WALKS
166
IGf(x uniformly in x
E
+
2)
- Gf(x
+ Y)I
CH. 5 , $2
< he
G . Then, for z E V v and x $ K,,
IGf(x
+
2)
- Cf(x)l< E ,
and the lemma follows by applying the Heine-Bore1 property. The following proposition is a key for further results. It can be extended to large classes of non-abelian groups. (See Exercise 2.9.)
Proposition 2.4. For every f E C i , iim C f ( x )Gf(- x ) = 0. x+A
Proof. Take E > 0; by Lemma 2.3, there is a compact set K , such that for 4 K , and every y in the compact support K of f we have
x
+
C f ( 4 < G f ( x Y) + E , and a fortiori,
<
G f ( 4 ~f(Y)/llGfll G f ( x
+ Y) +
(2.1)
E.
Applying the complete maximum principle to G f ( y )yields that the inequality (2.1) holds for every x 4 K , and every y E G . Now by Theorem 2.2, we may choose a point y s such that Gf(y,) E , and putting - x y, in place of y in eq. (2.1) yields
<
+
G f ( x )G f ( -
+ < 211Gfll YE)
"
The proposition then follows from the fact that Gf(- x ) - Gf(- x tends to zero when x tends to A .
+ y,)
-
Theorem 2.5. The set {G(x, ), x E G } has at most one non-zero limit point when x tends to A.
Proof. Let us suppose that there exist two non-zero limit points. Then there is a function f E C i , of support K , such that the set { G f ( x ) , x E G } has two strictly positive limit points when x tends t o A. Let us call I the smaller one. We take E > 0 and define inductively a sequence {x,,} of points in G in the following way: we set xo = e ; then by Lemma 2.3 and Proposition 2.4 we may choose a point x1 E G such that
167
GENERAL LEMMAS
CH. 6, $2
+ 2 - 2 ~ , G I ( - xi) < 2 - 2 ~ , IGf(x1) - G f ( x l + x ) I < 2% for x E K ,
Gf(x1) < 1
\ G f ( - xl) - G f i - x1
+
the sets K U { x l K } and K u {choose x2 such that
+ x ) I < 2% for x E K ; x1 + K } being also compact, we may
+ 2-3&, G / ( - x 2 ) < 2-3&, IGf(x2) - Gf(x2 + x ) I < F3&for x K u { - x1 + K } , IGf(- G f (- x2 + x ) I < 2 - 3 ~ for x E K u {xl + K } ; G/(x,) < I
E
x2)
in the same way, for every
GI(%,) < I
72,
we choose x, such that
+ 2-"-'&,
IGf(x,) - G f ( x ,
G I ( - x,)
+ x ) I < 2-n-45
< 2-"--l&, n-I
for x
u { - xi + K } ,
E
i=O
IGf(- x,) - Gf(- x , -tx ) I < 2-n-1& for x
9%-1
E
u {xi+ K } .
i=O
The support of the function
where z = x ,
+ x + x , - ~E { x , - ~+ K } , and this can be written
Summing u p all these inequalities yields
CH. 5, 92
TRANSIENT RANDOM WALKS
168
+
u:=,
for every x E { - xi I<}. The complete maximum principle implies that this inequality holds everywhere. By Lemma 2.3, we have
Ii;fi G [ f ( * )
%-+A
+ f ( + xi) + + f ( *
*
*
't x,)] (x) = ( n
+ 1) KG G f ( x ) ; x+A
hence, for every n,
-
Ilm G f ( 4
.?+A
< (nl/(n+ 1)) + (IlCfll + 4 / n .
Since 1 is the smallest of the positive limits, we have a contradiction. This leads us to lay down the following
Definition 2.6. A transient random walk X will be said to be of type I if the measures G(x, ) converge vaguely to zero as x tends to A . It will be said to be of tyfie I1 if there exist a non-zero limit point.
-
The preceding theorem says that for type I1 random walks there exists only one non-zero limit point and we know that this limit point is then a Haar measure. In the following sections we shall classify all random walks according to these two types and compute the actual value of the non-zero limit in the case of type I1 walks.
Exercise 2.7. Any symmetric random walk, that is such that ,u = G, on an abelian group is type I. Exercise 2.8. A random walk X on a general non-abelian group G has the property (P) if the set of measures { E , * G * E,,} is vaguely relatively compact. (1) Prove that (P) holds if and only if the set ( 8 , * e * is vaguely relatively compact. (2) Prove that (P) holds whenever G is unimodular and ,u is spread out. [Hint : Use Theorem 5.1.] (3) Assume that (P) holds and that the Choquet-Deny theorem is true for X,and show that Theorem 2.2 and Lemma 2.3 are still valid. Prove that Proposition 2.4 is true for x converging to A in the center of G . (4) Using the setting of Exercise 4.11 in ch. 1 with H the multiplicative group R, and K = R (G is then the group of positive affine motions of the real line) prove that (P) no longer holds for a non-unimodular group and that Theorem 2.2 cannot be extended to this case.
RENEWAL THEOREM FOR R AND Z
CH. 6, 53
169
Exercise 2.9. Show that Proposition 2.4 is true for all random walks on a (not necessarily abelian) discrete group ; more precisely lim G(x, e ) G(xy-',
e)
=
0
x+o
for y arbitrary but fixed in G . [Hint: Setting N ( x , e) = PslC,(x, e ) , show that G ( x , e ) = Z(x,
8)
+ N ( x , e) G(e, e) ;
then for x # e and x # y-l
1
X [ gq
lxg(Xn) = G ( x , e) G(e, x ~ ) / G ( e6 ), .
EX
For every 9, the first member of this equality is less than h
C Px[S,e) < %I
n=O
m
+ 2 Px[Xn = ~ Y I , n=pt1
and therefore tends to zero as x tends to A . ] 3. The renewal theorem for the groups R and Z In this section we study the cases of the group R of real numbers and of the group Z of integers. In the latter case the proofs will be written down only if they are really different from those in the former case. The letter m will stand for the Lebesgue measure (counting measure for Z) and we shall often abbreviate m ( d x ) to dx. The notations limx-.+mand limz+-m have their usual meaning.
+
-
Theorem 3.1. When x tends to co (re$. - a),the measures G(x, ) converge vaguely towards a multiple c+m (resp. c m ) of the Lebesgue measure. A t least one of the two constants c+ and c- i s zero, the other one being positive or zero.
+
Proof. First consider the case of Z.Suppose that when x tends to a,there exist two limit points, which by Theorem 2.5 are 0 and cm with c > 0. Then, for any function f E C i and any E > 0, and for x sufficiently large, Gf(x) is either greater than c m ( f )- E or less than E . One can therefore find a sequence {xn} converging to co and such that for every n
+
CH. 6, 53
TRANSIENT RANDOM WALKS
170
By Lemma 2.3, this is impossible, and we have a contradiction. Let us now consider the group R. As the function Cf is continuous, if there exist two limit points 0 and cm(f), every point in the interval [0, cm(f)] would also1 be a limit point, and by Theorem 2.5 this is impossible. In the following, we want to identify the constants c+ and c-. For this purpose we need some prerequisites. Let f be a measurable function on R. For h > 0 we call U , and 0, the minimum and maximum of f in the interval [(n - 1) It, nh], n E Z, and we set
-
Definition 3.2. The function f will be said to be directly Riemann integrable (abbreviated R-integrable) if the series a and 5 converge, and if for every E > 0, 5 - a < E , for h sufficiently small. -
-
The functions in C R are R-integrable. Let f be a function vanishing on m, 0 [ ,decreasing on [0, m[ and such that f(co) = 0; we have (0, - U,) f ( O ) , so that the two series a and 5 either both diverge or both converge. The function f is then R-integrable if and only iff is integrable in the ordinary Lebesgue sense. This example will be useful below in connection with the following
2:
1-
<
Proposition 3.3. If f i s R-integrable, the function Gf is bounded and converges to c+m(f) (resp. c-m(f)) as x tends to (resp. - m).
+
Proof. From the vague convergence it is easy to derive the convergence of functions Gf, where f is the characteristic function of an interval. Fix now h > 0 and set f,= I~,n--l)h,nhI. By the maximum principle, there is a constant M such that Gf, M for every n. Let {a,}wz be a sequence of positive real numbers such that 2 2 ; a, < co, and set f = anfn. For every n
<
+n
+n
akGfk(x) -n
z?:
+
d G f ( x )< 2 a k G f k ( x ) -n
+
2
uk#
lkl>*
which shows that Gf is bounded. Then letting x tend to f a,shows that the proposition is true for a function f of this kind.
RENEWAL, THEOREM FOR R AND Z
CH. 5, 53
171
Let now f be a positive R-integrable function; with the previous notation, we set
-< <
Gf Gf, and as f and f are of the kind just studied, letting Obviously Gf x tend to infinity yields
-
lim Gf(x)
< lim Gf(x)< liT;i Gf(x) < lim Gf(x).
x+*m-
%++
m
%--+*
m
x+*
OD
Thus
- < in^ Gf(x) < & Gf(x)< c J ,
C ~ U
%++
%-+fm
m
which completes the proof. We now proceed to compute c+ and c-. The features are different according to whether there exists or does not exist a first moment for p, that is whether 11x1 dp(x) < m or not; if it exists we shall call the mean of X the number
s
1 = x dp(x). We recall that 1 cannot be zero, since the random walk is transient (ch. 3 $4).
Proposition 3.4. If the first moment exists, the random walk i s of ty+e I I . If 2 > 0, then c- = 1-1 and c+ = 0. For 1 < 0, the symmetric conclusion holds.
<
Proof. Set g ( x ) = p([- x , m[) for x 0, and g ( x ) = - p(]- 00, - x[) for x > 0. Let k be the function defined on R2by k(x, y ) = 1 for x < 0, y - X, k ( x , y ) = - 1 for x 2 0 , y < - x and k ( x , y) = 0 elsewhere. Since
I
R'
Ik(%Y)l d m ( 4 dP(Y) =
5
+m
--m
IYI dP(Y) <
+
03,
we mayapplytheFubinitheoremto J-k(x,y) dm(x)dp(y),andget J-+zg(x) dm(x) = A. The function g is thus R-integrable, hence the potential Gg is bounded. Set h = 7,-m*0,; we have Ph(x) = p(]- 00, - x [ ) and thus ( I - P ) h. = g. The function 1 = h - Gg is therefore a bounded harmonic function which is m-a.e. equal t o a constant a. We may find two sequences x, and y, of points in R converging respectively t o - m and + 00 and such that for every
n, I@,)
CH. 6, $3
TRANSIENT RANDOM WALKS
172
= Z(yn) = a . Passing to the limit in the equalities
yields, by Proposition 3.3, that 1 = c-1
+ a,
0 = c+l
+ a.
We cannot have c+ > 0, since it would imply c- = 0, a = 1 and therefore 1 < 0. We thus have c+ = 0; hence a = 0 and c- = I-l, which completes the proof.
Remark. The reader will find, in ch. 3, Exercise 3.13, examples of random walks for which the above result is obvious.'The reader will also find in Exercise 3.10 why it is natural to consider the function g in the above proof. The same argument works when p is carried by [0,a].The expression 1 = JR x dp(x) is then always meaningful, but I may be 03 and we will then put 1-l = 0.
+
Corollary 3.5. If p i s carried by [0, co], then c-
=
1-l (of course we have
c+ = 0).
+
Proof. If 1 < m, this is Proposition 3.4. To deal with 1 = 03, we use the notation of Proposition 3.4. The function g vanishes on 10, a[,the potential Gg is the smallest positive solution of the equation ( I - P)f = g, and consequently Gg h (actually Gg = h m-a.e.). The function g, = g 71--n,0, is R-integrable and Ggn Gg = h. Letting x tend to - co in this inequality yields
<
<
I
+m
c-
-m
g,(4 dx
this cannot be true for every n unless C-
=
< 1; 0.
The foregoing results are now applied to prepare the final proof of this section. However, the following lemmas are of intrinsic interest. We call T+ the hitting time of 30, .o[.
CH. 5. 53
RENEWAL THEOREM FOR R AND Z
sofa
Lemma 3.6. If I+ = x d&) vaguely to zero as x tends to - co.
=
173
+ Q),the measures P,, (x, - ) converge
Proof. Let f be a positive continuous function with compact support contained in 30,
+ m[. By the strong Markov property, Gf(4=
1;
PT+
(% dr) Gf(Y)*
If p is carried by 10, co[, by Corollary 3.5 the function G f ( x )tends to zero as x tends to - co, and as Gf 2 f this implies the desired conclusion. If p is not carried by 10, co[, we define inductively a sequence {S,} of stopping times by
S, = 0,. . ., S ,
=
inf{n:
X, > Xs,-,}, ...
+
where the infimum of the empty set is understood to be m. By ch. 1, Exercise 4.9, the random variables X,, - Xs,-l are independent and equidistributed with law pf = PS1(O,* ) = PT+(O, ); indeed S , and T+ are Po-a.s.equa1. The sequence {X,,} is thus a random walk whose law p' is carried by 10, coy, and since p' is clearly larger than the restriction of p to 10, a)[ we have &' x dp'(x) = m. Now starting from x < 0, the hitting distributions of 10, m[ are the same for X and X' because the first time that X s k is > 0 is then almost surely equal to the first time X , is > 0 ; we then conclude by applying the first part of the proof to p'.
-
+
Lemma 3.7. If 1- = for all x E R.
em(- x ) dp(x) <
03
and if 1 > 0, then P,[T+ < co] = 1
Proof. Let us first assume that 1+ < 00; the strong law of large numbers then implies that X n / n converges Po-as. towards 1. Thus X , converges a s . to co and P,[T+ < co] = 1 for every x E R. Now if A+ = co, we choose a number c such that & x dp(x) > 1-,and set 2; = 2, if 2, c and Zf, = c if 2, > c. We may apply the above reasoning Zi, and as X , 2 X i as., this completes the proof. to Xf, =
+
2;
<
We are now ready to prove the renewal. theorem for the case in which the first moment does not exist.
Theorem 3.8. If the first moment does not exist, the random walk as of type I .
CH. 6, 53
TRANSIENT RANDOM WALKS
174
Proof. Assume c- > 0, so that also c+ = 0, and let us suppose that I + = By the strong Markov property, if f E C i vanishes on 3- co,0[,
Gf(x) =
1;
pT+
+
do.
dy) Gf(y)-
Since c+ = 0 for every E > 0, there is a number a implies Gf ( y ) < E , and hence
>0
such that y
>a
< [p,, ( x , dY) G f M + if we let x tend to P,+(x, - ) converges vaguely to zero, and therefore for x sufficiently small we get Gf(x) < 2.5, which is impossible since c- is strictly positive. We thus have I + < co and therefore 1- = + Cf(4 00,
03.
If we consider I; in place of ,u, this is equivalent to supposing that c- = 0, c+ > 0, 1- < 03, I+ = 03. We then have P,[T+< co] = 1 for every x E R; on the other hand P,+(x, * ) converges vaguely to zero as x tends to - 03. We thus get, with f as above, 0 = c-m(f) = lim Gf(x) = lim X-+-CC
I
8
x--*--m
PT+ ( x , dy) Gf(y) = lim Gf(y) = c+m(f); Y-+W
we have a contradiction and the proof is completed. These results extend to the groups which are isomorphic to R @ K and Z @ K , where K is a compact group. As before, we write the proof only in
the former case. Let G = R @ K ; every point x E G can be written uniquely as a pair ( y , k ) , with y E R and k E K. We call th, the canonical mapping from G to R: (y, k ) -+ y and we shall say that x tends to + 03 (resp. - 03) in G if $ ( x ) tends to + 03 (resp. - co) in R. Finally, the Haar measure m , on G will be taken equal to the product of the Lebesgue measure m on R and of the normalized Haar measure on K.
Theorem 3.9. I f G i s isomor9hic to R @ K or Z @ K , and i f the random walk of Caw ,u i s of type 11, then
RENEWAL THEOREM FOR R AND 2
CH. 6, $3
I n that case, if 1 =
176
sG$(x) d,u(x) > 0, then in the vague topology, -
lim G(x, ) = 0 and lim G(x, ) = I-lrn. x-tfm
x+-m
For 1 < 0 the symmetric covclusion holds.
Proof. By 9 2, from every sequence converging to + 00 we can extract a sequence {z,} = {(y,, K,)} such that {G(x,, * )} converges vaguely to a multiple amG of the Haar measure of G. Let f E Cj$ be constant on the co-sets of K . By Lemma 2.3, we have lim GI(%,) = lirn Gf(y*, 0). r,,-++m
Y*'+m
Put f ( y ) = f(x) and = $(,u), and call G the potential of the random walk of law fi on R.Then Gf(yn,0) = ef(y,), and moreover, applying Propositions 3.4-3.8 to this random walk, we get lim M Y n ,
=
c+m(fL
y,-r+m
where c+ is non-zero if and only if
and
Since furthermore m ( f )= m,(f), the proof is complete.
In the following section we shall show that, up to an isomorphism, these groups are the only abelian ones for which there exist type I1 random walks.
Exercise 3.10. (1) Under the hypothesis of Corollary 3.5, show that 3- co, O [ is a transient set and that g is the probability of never returning to 3 - m,O[. (2) Similarly, show thatif a > 0,1,-,,,, = Gg(x - a). Show that P T + ( x ,[O, a [ ) is the potential of a function vanishing outside [0,a [ . Compute this function and derive that if A is a relatively compact subset of 10, 03 [ whose boundary is of Haar measure zero, then
+
lim PT+ (x, A ) x-t-m
=
1-1
176
CH. 5, $3
T R A N S I E N T RANDOM WALKS
[ H i n t :Use the duality between the kernels H , and P , described in ch. 2 $ 4. This result will be generalized in 3 5.1 Exercise 3.11. (1) Set R+ = [0, a[,and show that for a transient random walk the set of measures { H , + ( x , ), x E R} is vaguely relatively compact. ( 2 ) Under the hypothesis of Corollary 3.5, prove that limz+.-a H R + ( x , * ) exists for the vague topology, and compute this limit. [Hint: Consider sequences for which the limit exists and then apply the duality between P A and HA.] Exercise 3.12. Assume the hypothesis of Corollary 3.5, with A
< 00,
and set
(1) Show that N , is an a s . finite random variable. (2) Find the starting measure v such that the function t -+ E,[N,] is linear. [Hint:Show that this amounts to finding v such that v * = urn+, where a E R+ and m+ is the restriction of the Lebesgue measure to [O, a[.It turns out that Y has density A-l ,u([t,0 0 [ ) . ] If A = 03 there is no such measure. (3) Deduce from (2)the renewal theorem for this particular set of hypothesis.
+
Exercise 3.13. Under the hypothesis of Corollary 3.5, with A J x 2 d,u(x) < 03, show that
<w
and
c2 =
lim G([O,x ] ) - I-% = a2/212. x-+m
[Hint: Use the Poisson equation with second member
Exercise 3.14. This exercise is designed to show that in the discrete case the renewal theorem is equivalent t o a limit theorem for recurrent chains which will be studied in ch. 6 $2 in greater generality. (1) Let X be a recurrent irreducible discrete chain and using the notation of ch. 3 $1.11, set Y o = S , and Y, = Sn+l - S, for n > 0. Prove that for every probability measure P, the random variables {Y,,n 3 l} are independent and equidistributed of law ,u = P J S , = * ] . Prove that ,u is
CH. 6, 94
THE RENEWAL THEOREM
177
adapted if and only if X is aperiodic (ch. 3 $1.17). (2) Suppose henceforth that X is aperiodic and prove that for every pair ( x , y ) of points in E , lim Pn(x,y ) = (Ev[Sv,)-l. n-
00
[Hint:Use the renewal theorem for the random walk S, on Z.] (3) Show that the measure m such that m ( y ) is equal to the limit in (2) is invariant for X and derive that E,[S,] is either finite for every x in E or is infinite for every x in E. This is another proof of Theorem 1.9 and Corollary 1.10 in ch. 3. (4)We now go in the converse direction. Let (2,) be an adapted random walk on Z with law ,u carried by the integers n 2 1. Define xn(w) =
fi
- z k ( w ) if
Zk(0)
n?k+l(w).
Prove that for Po, the sequence {X,, n 0) is a Markov chain with initial distribution e0 of the type studied in Exercise 1.16 of ch. 3. Deduce then the renewal theorem from the convergence result in (2).
4. The renewal theorem We now return to the case of a general abelian LCCB group G . As before, f is an arbitrary non-zero function in C t .
Proposition 4.1. If G has a compactly generated, open, non-compact sub-group GI such that GIGI is infinite, all transient random walks on G are type I . Proof. Suppose that a random walk X on G is of type I1 and let {x,,} be a sequence such that the measures G(x,, ) converge vaguely to cm ( c > 0 ) . We shall work towards a contradiction. We may assume that the cosets x, G1 are pairwise disjoint. Indeed if no sub-sequence of (x,} has this property, then there is a coset of G , which contains infinitely many x,. This amounts to assuming that {x,} is contained in one and the same co-set. Pick now a sequence { z k } , such that the cosets zk GI are pairwise disjoint. By Lemma 2.3, for every k we have
+
+
lim Gf(x,
+
Zk)
n
For every k , pick an integer
%k
such that
=
cm(f).
178
CH. 5, $4
TRANSIENT RANDOM WALKS IGf(%n,
+
zk)
- cm(f)I
< 2-k,
+
and set y r = xnk 2,; then the sequence { y k }has the desired property. Now, since GI is compactly generated but non-compact, there exists a point x E Gl such that nx -+d as n + co. Using Proposition 2.4 and Lemma 2.3 in the same way as in the proof of Theorem 3.1, it is easily seen that limn Gf(nx) = 0 or limn Gf(- nx) = 0. Without loss of generality we assume the former contingency. Then, for every n, there is a smallest integer h, such that for n' >, h, Gf(xn n'x) ,< t ~ m ( f ) .
+
For otherwise there would be an integer no and a subsequence {n'}such that Gf(Xn,
+ n'x) > & 4 f ) .
and this is impossible since, by Lemma 2.3, Gf(x,, + n'x) and Gf(n'x) have the same limit, which is zero. For fixed I , G f ( x , i x ) tends to c m ( f ) ;hence k, > 1 for n sufficiently large. Thus, for n sufficiently large,
+
Gf(xn
+ hnx - x ) > ac m(f),
Gf(xn
+
+
The sequence { x , h,x - x ) converges to d because GI being open, G/Gl is discrete and every compact subset of G is contained in the union of a finite number of cosets of GI. As a result, Gf(x,t h,x - x ) converges to cm(f) as n tends to infinity and Gf(x, hnx) to zero which by Lemma 2.3 is a contradiction. The proof is complete.
+
+
Corollary 4.2. I f G is isomorphic to Rdl @ Z d l @ K ( K a compact group) and if dl + d2 > 1, every transient random walk on G is of ty#e I . Proof. By the argument of Theorem 3.9, it suffices to consider the case G = Rdl @ Zd*. If d2 3 1, we are done thanks to Proposition 4.1.If G = Rd*. with dl > 1, then G is the union of an increasing sequence of compact sets whose complements are connected. If the continuous function Gf has a nonzero limit a, every point in [0, a] is also a limit point, which by Theorem 2.6 is a contradiction. Before we proceed let us recall that a compact sub-semigroup of a group is a compact subgroup.
CH. 5, $4
THE RENEWAL THEOREM
179
Proposition 4.3. If every element of the group G is compact, then every transient random walk on G i s of type I . Proof. Let X be a type I1 random walk, and let {z,} be a sequence such that {G(z,, )} converges to cm (c > 0 ) . For every n, the closure of the set {kz,, k > 0 } is a compact sub-semigroup of C , hence a subgroup of G, and there is a sequence k , of positive integers such that {kg,}converges to - z, as p tends to infinity. Thus for every E > 0, one can find an integer k , such that
and since, by Proposition 2.6, G f ( - )2, tends to zero as n tends to infinity, the sequence Gf(k,z,) tends to zero. Choose M > max(IIGf)I,t c m(f)).Since c2 ~ z ( f ) ~ / 4<Mc m ( f ) ,for all sufficiently large values of n 2 1, we may pick the largest integer h, < K, such that
By the proof of Proposition 2.4, for n sufficiently large,
Moreover, for n sufficiently large, Gf(z,) 2 +C m ( f ) ,which implies
and
Choosing E in an appropriate way, we see that there exist four constants a,p, y , B such that for n sufficiently large,
0
< a < Gf(hnzn)< P < c m ( f ) ,
180
CH. 5, $5
TRANSIENT RANDOM WALKS
Since {z,,} tends to A when n tends to infinity, at least one of the sequences {h,z,} and {(hn 1) z,} contains a subsequence {y,} converging to A . The sequence G f ( y n ) has at most 0 and cm(f) as limit points; we thus have a contradiction.
+
Theorem 4.4 (renewal theorem). If there exists on G a random walk of ty$e II, the group G i s isomorphic to R @ K or Z @ K , where K i s a compact grot@. T h e asymptotic behaviour of the measures G ( x , ) i s then described by Theorem 3.9.
Proof. By Proposition 4.3, there exists a non-compact element y . As every group, G has a compactly generated open sub-group G'; the subgroup GI generated by G' and y is non-compact, compactly generated and open. Thus by Proposition 4.1, the group G/GI is finite; hence G also is compactly generated. The theorems on the structure of abelian groups then assert that G is isomorphic to Rd*@ Zdl @ K , where K is a compact group. By Corollary 4.2, the proof is complete. 6. Refinements and applications Throughout this section, p will be taken spread out, and we shall give applications of the renewal theorem to hitting probabilities and to recurrent sets. The first result is true if G is merely unimodular.
e
e",
Theorem 5.1. The measure can be written G = G' + where G' i s a n absolutely continuous measure having a bounded continuous density f~ and G" a finite measure. Proof. By hypothesis, there is an integer k and a bounded integrable function i,!i such that pk = #m + u, where u is a finite measure of norm less than 1. Setting no = 2k it is easily seen that one can write pne= pm v, where p E C i and I/YI( < 1. The measure G can be written
+
Setting G =
c:',,
pnn8,we have
CH. 6, 55
REFINEMENTS AND APPLICATIONS
181
and inductively, for every n 2 1,
C
=
E,
+ v + . - -+ + + - . *+ * 'pm*G + vn
(E,
vn)
v*+1*
C.
(5.1)
s
If f E C,, the function x -+ f(xx') C(dx') is bounded and continuous, since C is the potential measure corresponding to pun*.From this remark, we derive easily that limn-rmvn+l * C = 0 in the vague topology. On the other hand, v' = vn is a finite measure, and letting n tend to infinity in eq. (5.1) yields G=v'+d*qm*G.
2;
which shows that Theorem 5.1 is true for C. It is then verified for G"= ( C ~ - ' p n ) * v ' a n d p = € ' ' * p * G .
c with
As is apparent from the proof, this decomposition is not unique, but the renewal theorem settles the asymptotic behaviour of p .
Theorem 6.2. If the random walk is of type I , then limp(x) = 0. %-+A
If the random walk is of type 11 with A
> 0, then
lim p ( x ) = A-l,
lim p ( x ) = 0.
%++m
x--+-m
For 1 < 0, the symmetric conclusion holds.
,"
Proof. If X is of type I, the random walk of law pn'is also type I and therefore lim(G * 'p) ( x ) = 0, that is C * 'p E C,; since is finite, 9 = c" * p * C is also in C,, and the proof is completed in that case. If X is of type I1 with 1 > 0, then using the notation of Theorem 3.9,
and therefore, by the reneyal theorem for C, Iim (G * 'p) ( x ) = 0, x+-m
lim
x++m
(G* p) ( x ) = nt('p)/noil.
Since G"(G)= no/m(p),it follows that limp&) x+-
OD
=
0 and limp(,) x++m
= A-1.
TRANSIENT RANDOM WALKS
182
CH. 5, $6
Corollary 6.3. For every bounded Borel function f with compact support, lim Gf(x) = 0 +-+A
if the random walk i s of type I , and lim Gf(x) = A-lm(f), r+-m
lim G f ( x ) = 0 r++m
if the random walk i s of type I I with 1 > 0. Proof. We have
The first term tends to zero as x tends to A since C" is finite, and the result follows by a simple application of the Lebesgue theorem. We now turn to the study of the asymptotic behaviour of hitting probabilities. As usual, the symbol refers to the dual random walk.
Theorem 6.4. Let B be a n y relatively compact Borel set, and f a n y bounded Borel function with compact support. T h e n if X i s type I lim U , f ( x ) = 0 X+A
and if X i s type I I with A
> 0,
lim U B f ( x ) = 0, x++m
where f B
=
lim U B f ( x )= 1-l x-.-
m
P.[SB = co].
-
Proof. Consider first the type I1 case. Clearly the kernels Uo(x, ) and G ( x , * ) have the same limits on functions with compact supports as x tends to infinity. Passing to the limit in the identity
CH. 6. 56
REFINEMENTS AND APPLICATIONS
183
OB (ch.
2, Proposition 4.9)
Using the duality between the kernels U B and then yields lim U B f ( x ) r-+-co
The proof is then easily completed.
Corollary 6.6. W i t h the same hyeothesis, the hitting distributions PB(x,* ) converge vaguely to zero or to I-llBm.
Proof. Straightforward since the function is equal to lBfB.
gB which
was defined in ch. 2 $3
By ch. 2 $3 and the Choquet-Deny theorem, when ,u is spread out every set is either recurrent or transient. This raises the problem of characterizing the recurrent sets. The following result gives a partial but useful answer. If the group G is isomorphic to R @ K or Z @ K , with K a compact group, we denote by G , the set of points ( x , k ) such that x 2 0.
Proposition 6.6. Zf the random walk i s of type ZI with 1 > 0, a set A is recurrent if and only if m ( A n G,) = + Q). Proof. We take G = R, the other groups requiring only obvious changes. The set G- = G : is transient as is easily seen by applying the law of large numbers (ch. 3, Exercise 3.12). We therefore take A C G,. If m ( A ) < 00, Theorem 5.2 implies that G(x, A ) < co for every x E G , and thus A is transient. Suppose conversely that m ( A ) = 03. We can write A = A k , where the sets A , can be chosen disjoint, of finite measure and such that the sequence (m(A,)} increases t o oc,. Each A , being transient by the preceding argument, to hit A infinitely many times, the random walk must hit infinitely many sets A , and therefore
+
21“
+
+
By the strong Markov property, we get easily (see ch. 2, Theorem 1.11) pC[T,k
< Q)l b G(Ak)/sup
‘ ( Y 9
Y E Ak
Next, by Theorem 5.1, SUP G(Y, A k ) YEAR
< Ilc’’ll + 11$1(
m(Ak)B
184
TRANSIENT RANDOM WALKS
CH. 6, $5
As k tends to infinity, G ( A k ) is equivalent to l - l m ( A k ) ,by Theorem 5.1, hence
k iP,[T,k
k-+w
< 031 > (A/lbl/)-l>
The set A is recurrent and the proof is complete.
Corollary 6.7. A set A c G , i s recurrent if and only if G ( A ) =
+
03.
Proof. Obvious. Remark. For the case in which the first moment does not exist, but J!?.a 1x1 dp(x) < 03, one could hope for similar results, but in fact they are not true (cf. Exercise 5.12). Exercise 6.8. Investigate the asymptotic behaviour as x tends to d of PJl" < T B ]and of G A ( x ,B ) , where A and B are relatively compact Borel sets. Exercise 6.9. Try to carry over Theorem 5.4 to non-spread-out laws. [ H i n t : Use relatively compact Borel sets whose boundaries have zero Haar measure.] Exercise 6.10. Prove Theorem 5.4 without relying on Theorem 5.1. Exercise 6.11. Let A be a transient set of infinite Haar measure. Prove that its capacity relative to m (ch. 2, Exercise 4.13) is equal to 03 if the random walk is of type I and to 111 if the random walk is of type I1 with mean 1.
+
Exercise 6.12. If G = R and 1x1 dp(x) < 00, prove that the following two statements are equivalent : (i) for every set A such that m(G+ n A ) = 03, we have Po[S, < co] = 1 ; (ii) the random walk is spread-out, type I1 and 1 is > 0. [Hint: Observe that a set with property (i) is recurrent; then use the fact that for a non-spread-out random walk the a-algebra 9 is not a.s.trivia1 and that there exists therefore a set which is neither recurrent nor transient.]
CH. 5 , 95
REFINEMENTS AND APPLICATIONS
186
Exercise 6.13. Let X be a transient random walk on a discrete group. Prove that PJT,,, < a)] < 1 for all x # e with one and only one exception: the group is isomorphic to Z and either X is right continuous (ch. 3, Exercise 3.13) and the mean A exists and is > 0, or X is left continuous with the symmetric conditions. Exercise 6.14. Let H be a Bore1 sub-group of G.Show that H is a recurrent set if and only if G(e, H ) = Q).
+
CHAPTER 6
ERGODIC THEORY OF HARRIS CHAINS
In this chapter, we aim at proving limit theorems for Harris chains. For this purpose we shall need a certain amount of potential theory, which will be introduced in $9 4 and 5, but potential theory for Harris chains will really be dealt with in ch. 8. Throughout the chapter, the state space ( E , 8)is separable. 1. The zero-two laws
In this section we do not deal with Harris chains but with general chains for which we introduce several notions which will prove useful for Harris chains as well as in other situations. Definition 1.1. The a-algebra
is called the asymptotic or tail a-algebra of the canonical Markov chain X . A random variable measurable with respect to d is said to be an asynzptotic or tail random variable and in particular, a set belonging to d is an asymptotic or tail event. Clearly, an event A is asymptotic if and only if for every n there exists an event A , E .F such that A = 6L1(An).The a-algebra .f of invariant events is thus contained in d . Moreover, the sets A , may be chosen in a canonical fashion. The shift operator 0 is not one-to-one, so that 8-' is not a mapping and e-l(w) is the set of trajectories w' such that 6(w') = w . However, if Z E a(Xm,m 2 n), this random variable does not depend on the first n coordinates and one can define unambiguously Z o 6-1 and Z o 6-* for p n. For w = (xo, xl,. . .), z0 O-'(w) = Z(y, xo, X I , . . .),
<
186
THE ZERQTWO LAWS
CH. 6, $1
187
where y is arbitrary in E . If Z is asymptotic, one can therefore define unambiguously and for every n E N another asymptotic random variable Z,, by setting 2, = Z o 8 - n . We then have Z = 2, o 8, and Z, = Z,, o 8, for every pair of integers n and m. We shall write 8-, rather than 8-,, in order to make 8, (nE Z) appear as a transformation group of d .
Definition 1.2. Two asymptotic random variables Z and 2' are said to be equivalent if, for every n E Z, the asymptotic random variables Z o 8, and 2' o 0 , are almost surely equal.
<
Notice that if suffices to have the equality for n 0, the equality for n > 0 following from the Markov property. This notion of equivalence is stronger than the usual almost sure equality ; two invariant random variables equivalent in this sense are equivalent in the sense of ch. 2 $3. Finally, the operators 0 , operate on the equivalence classes of asymptotic random variables, and the classes of invariant random variables are those which are invariant under 8,. We shall state a result similar t o ch. 2, Proposition 3.2, for which we need the following notational device: we define a T.P. denoted by Q on ( E x N,d @ B ( N ) ) by setting if m = n 1, (R(x, A ) Q(@, n ) ,A x (4) = if m # n 1.
+ +
For every probability measure P, on (SZ,F) and every integer m , the sequence { Y , } = { ( X n ,n m ) } is a homogeneous Markov-chain with T.P. Q, called a s$ace-time chain (ch. 1, Exercise 2.17(2)).
+
Proposition 1.3. If P is markovian, the formula
sets u$ a one-to-one and onto correspondence between the equivalence classes of asym$totic and bounded random variables and the bounded Q-harmonic functions. Moreover, lim h(X,, n) n-+m
Proof. If Z E b d , we have
=2
ax
188
ERGODIC THEORY O F HARRIS CHAINS
CII. 6, $1
and h is therefore Q-harmonic. Conversely if h is Q-harmonic and bounded, h(X,, n) is a bounded martingale which converges P,-as. and in L1(P,) for every x E E. The limit Z is almost surely asymptotic, since for every m E N the sequence h ( X n ,n m ) converges also to a limit Z ,, and obviously 2 = 2, o Om a s . The convergence in the L1 sense implies clearly that h(x, n) = EJZn1.
+
Remark. The space X of bounded Q-harmonic functions is a Banach subspace of b ( b @ B(N)) and it is easily seen that for h E X , llhll
=
lim n
114 , .)[I. *
Moreover, if we make 2 = &, h(X,, n), the random variable 2 is in b d and it follows from the above formulas that llhll = llZll where the norms are the uniform norms on the respective spaces. The correspondence described above thus becomes a Banach isomorphism. If h is a bounded Q-harmonic function, by setting h, = h ( - , n ) , EN, we get a sequence of bounded functions on E such that Pnh,+k = hk for n, k 3 0. Conversely, if ho is a bounded function such that for every n there is a bounded function h, such that Phl = ho and Ph,+l = h, for every n > 0, the function defined on E x N by h( * ,n) = h, is Q-harmonic. We will denote by 9 the set of such functions ho; we clearly have a one-to-one and onto correspondence between 9 and the equivalence classes of bounded asymptotic random variables expressed, when P is markovian, by the formula
We henceforth assume that P is markovian and we are going to express asymptotic properties of {P,}in terms of d .In the following if y is a bounded measure on (Q, 9) we shall write llylld for the norm of y when it is restricted to d .
Theorem 1.4. For any bounded measure v on ( E , 8)we have lim I l Y P n l l = IIPvlld, n
Proof. The sequence { IlvPnII} is clearly decreasing hence convergent. We are going to prove that its limit is equal to sup h dv; h E 9,llhll 1) which is equal to IIPvIId thanks to the remark following Proposition 1.3
{s
<
THE ZERO-TWO LAWS
CH. 6, $1
189
together with the fact that if h E X , then h( * ,n) is in 9 for every n. By ch. 4 $1 we may choose a probability measure p such that pP << p and v << p and we set f = dv/dp. Using the operator T o n L1(E, 8 ,p) associated with P we have
where B is the unit ball in Lm(p).I t follows that
n
On the other hand, for any B such that for every n,
E
> 0,we can find a sequence {g,}
of points in
I t *T*ngndp 2 l l v ~ f i l l - & . Since ( E , 8) is separable, B is @OD, L1)-sequentially compact so that we can find a subsequence {nk}such that {T*nkg,,} converges to a limit goin B. The operator T* being weakly continuous, for each n the function go is in the compact set T*nB and therefore go is in T*,B. By passing to the limit in the above inequality we get
n:
J /go d p 2 lim llvpnll -
E,
hence
n;
T*"B onto itself. Indeed we clearly have Now I' maps the set A = T*A c A and if k E A , then for every n there is a function k , such that k = T*nk,. Each of the sets T*"B being a(Lw,L1)-compact the sequence {T*(n-l)k,} has a limit point I which lies in A and as T is u(Lw,L1)-continuous we have T*l = k , and T* is onto. As a result, for anyg E A we can inductively find a sequence {g,} of points in B such that gn = T*gn+l and go = g. By applying now Proposition 1.6 of ch. 4 to the transition probability Q and to the measure p @ m where m is the counting measure on the integers, we see that there is a one-to-one correspondence between classes of equivalent functions in A and functions in 9. The proof is then complete.
ERGODIC THEORY O F HARRIS CHAINS
190
We now set, for k
CH. 6, $1
E %,
The limit exists since the sequence involved is clearly decreasing.
Lemma 1.6. For every k , the function ak is measurable and Pak >, ak. Proof. To prove that ak is measurable we shall prove the stronger property: the positive part N + ( x , * ) of the bounded kernel N is itself a kernel. We shall use the set-up of Lemma 5.3 in ch. 1. Let x be fixed and choose for probability measure 1 the suitable multiple of IN@, * the functions fn defined by
)I;
fn(y) = N ( x , E;)/A(E;) if
A(E;) > 0 ,
f , ( y ) = 0 otherwise,
are bounded uniformly and converge almost surely to the density of N ( x , * ) with respect to 1. Consequently {I,'} converges as. to the density of N+(x, ) and because of the uniform boundedness, for every A E 8,
-
N + ( x ,A )
=
lim n
sA
5
A
I,'
d1.
But if A E Bkand n > k , then 1', d1 = N , f ( x , A ) where N: is the positive part of the restriction of N to Bn and the map x + N , f ( x ,A ) is easily seen to be measurable. Consequently N + ( , A ) is measurable for every A in Bk and by the monotone class theorem N+ is a kernel. To prove the second assertion put yn(x, ) = Pn+k(x, ) - Pn(x, ) and & ( x ) = IIY,(x, )I I. For A E 8,we have
-
uk
0
Sn(x)
-
2 Yn(x, A ) - Y ~ ( xA"), ,
hence
P 6 n ( x ) 2 ~ n + l ( xA, ) - Y n + l ( X , A").
-
By taking for ( A , A") a Jordan decomposition for Y , , + ~ ( X , ) we see that whence Pak >, ak follows readily. P 6, >,
< <
Clearly, 0 ak 2, but one can actually say much more as will be seen in the next result. Let us first observe that to say that d = $ a s is t o say that for any Z E b d , there is a Z' E b y such that Z = 2' a s . or equivalently that Z = Z o 0 a s . By Proposition 1.3 and Proposition 3.2 in ch. 2 ,
CH. 6, $1
THE ZERGTWO LAWS
191
this is also equivalent to saying that the bounded Q-harmonic functions h do not depend on the variable n, or in other words that h(x, n) = h(x) with h a harmonic function for P . An example of an asymptotic event which is not a.s.equa1 to an invariant event is provided by K { X n = n} where X is the chain of translation on integers. Another example is to be found in Exercise 1.9.
Theorem 1.7 (first zero-two law). For every k, the number s ~ p ~ , ~ a is~ ( x ) either zero or two. For k = 1, the former case occurs if and only if d = 9 a s . and then for every Probability measure v on 6,
Proof. It is enough to prove the result fork = 1 because it can then be applied to the T.P. PI,{ to get the result for all k’s. If d = 9 a.s. then by the Markov property P, and P,, agree on d and the “if” part follows from Theorem 1.4. Suppose conversely that .dis not equal to Ia.s. There is then a probability measure v and a set F E .d such that P , [ F A O-l(F)] > 0, and by replacing if necessary v by Z-n-lvP, we may assume that vP << v . If P,[F\O-l(F)] > 0 we set G = F\O-l(F); the set G is in a’, P,[G] is > 0 and G n O-l(G) = (F\O-l(F)) n (O-1(F)\8-2(F)) = 0 . If P,[F\O-l(F)] = 0, then P,[O-l(F)\F] > 0; but this is equal to P,,[F\Fl] and therefore we have P,[F\Fl] > 0.Then again the set G = F\F, is in d , has positive P,-measure and G n F ( G ) = (F\Fl) n (V(F)\F) = 0. In each case we nQw set 2 = 1, - 7,o 8 and h,(x) = E,[Z,]. By Proposition 1.3 we have
2:
lim h n ( X n )= 2, n
lim J Z ~ + ~ ( X= , , )Z1 P,-as. n
But 12 - ZII = 2 on G and therefore, for any E > 0, there is an x E E and an integer 1 such that Ih,+l(x)- h,(x)l > 2 - E . Since Ilh,,ll 1 and Pkhn+k = h, it follows that for every n,
<
We consequently get al(x) > 2 - E , which completes the proof.
ERGODIC THEORY OF HARRIS CHAINS
192
CH. 6, $1
Before we proceed, let us recall that a sub-a-algebra '3 of F is said to be a.s. trivial if for every A E '3 the function P . [ A ] is identically zero or one; this is a more stringent condition than to be P,-trivial for every x E E .
Theorem 1.8 (second zero-two law). For any chain, the number SUP x,y&
lim I IPn(x, ) - Pn(ys * n
)II
is either zero or two. The former case holds if and only if the following equivulent conditions hold : (i) 2 is a.s.trivia1; (ii) the bounded Q-harmonic functions are constant; (iii) for every pair ( v l , y 2 ) of probability measures on ( E , 8) lim I((vl - v2) Pn[l= 0; ?I
(iv) u1 = 0 and the bounded harmonic functions are constant.
Proof. The equivalence of (i) and (ii) follows readily from Proposition 1.3. If sd is astrivial, then [lPv,-y211d= 0 and (iii) follows from (i) by Theorem 1.4. If (iii) holds, by taking v1 = E,, v2 = P(x, ) we get a1 = 0 . Furthermore if h is bounded and harmonic,
-
Ih(x) - ~ ( J JI = ) IPnh(x) - P A Y )I
< IIhl I IIPn(x8
*
)
- Pn(yp )I I *
for every n, so that h is constant. If (iv) holds, then since u1 = 0 , we see from the last theorem that d = 3 a.s. and since the bounded harmonic functions are constant, 9 and hence d,is a.s.trivia1. It remains to prove that if d is not a.s.trivia1 we are in the second case of the first sentence in the statement. But we can then find a set F E and~ a probability measure v such that 0 < P J F ] < 1 ; set Z = l F - 1, and g,(x) = E,[Z,]. Since lim,g,(X,) = 2 P,-a.s., for any E > 0 there exist an integer n and points x and y in E such that g,(x) > 1 - E andgn(y) < - 1 E . As in the proof of Theorem 1.7we then get
+
lim k
IIPk(x9 *
- P ~ Y. , 2 Ign(x) - gn(y)l > 2 - E
and the proof is now easily completed. Exercise 1.9. Prove that for the chain of Exercise 1.24 of ch. 2 the a-algebra
THE ZEROTWO LAWS
CH. 6, $1
193
n:
Jal is strictly larger than 9 when p , > 0.Give explicitly an event which is in d and is not a.s.equa1 to an invariant event. The renewal chain of ch. 3 $1.16 also provides examples of the same kind.
Exercise 1.10. For any T.P. R and 0 < a < 1 show that for the chain with T.P. P = a1 (1 a) R the algebras Jal and 3 are a.s.equa1.
+ -
Exercise 1.11. (1) With the notation of Theorem 1.4 prove that
<
where X is the set of bounded harmonic functions h such that llhll 1. (2) The number sup,,,(lim, I ~ - ~ I I ( E , - E ~ ) Pkll) is equal to either zero or two. The first case occurs if and only if J is a.s.trivial and then
2;
for any pair
(yl, v2)
of probability measures on ( E , 8 ) .
Exercise 1.12. Suppose that P induces a conservative contraction on L1(p). Prove that al takes on only the values 0 and 2 p-almost-surely. [Hint: The set A = {a < a1 < b}, where 0 < a < b < 2 is in % hence almost-surely equal to an absorbing set B c A ; apply Theorem 1.7 to the trace chain.] Exercise 1.13. Prove that the equivalent conditions in Theorem 1.8 are also equivalent to the following statement: for any B EFand any initial measure Y, lim sup IP,(A n B ) - P , ( A )P,(B)I = 0, n-+ m A€&',,
where Jaln = O;'(T).
Exercise 1.14. Let p be a probability measure on a group G, and H the smallest closed normal subgroup containing the support of p * (cf. ch. 3, Exercise 3.15). (1) Prove that the set of bounded measures on G such that limn (Ipn* vII = 0
is a right ideal (in the convolution algebra of G ) contained in the kernel of the canonical mapping from b.M(G) into b.M(G/H).
194
ERGODIC THEORY O F HARRIS CHAINS
CH. 6, 92
(2) Show that one can extend uniquely Q-harmonic functions on E x N to Q-harmonic functions on E x Z, and restate Proposition 1.3 in that case. (3) Compute the sub-group H’ of periods of continuous Q-harmonic functions in terms of H and prove that G x ZIH’ is isomorphic to G / H . (4) Assume that G is abelian and that p is spread-out and prove that the inclusion in (1) is then an equality. If in addition G is connected, for every bounded measure v such that v ( G ) = 0, 1imJIp* vI/ = 0. n
2. Cyclic classes and limit theorems for Harris chains In this section we take up the study of Harris chains for which we want to prove that the limit behaviour of Theorem 1.8 occurs. Actually simple examples show that this cannot hold when the chain has a periodic behaviour and this leads to the notion of cyclic classes (see ch. 3, Exercise 1.17). Our study will rest on the following Proposition 2.1. For each integer k either a k ( x ) = 0 for every x in E , or for m-almost every x in E.
=2
Proof. By Lemma 1.5, the functions 2 - ak are superharmonic hence constant m-a.e. and larger than this constant everywhere. The result follows therefore immediately from Theorem 1.7. We shall need the following notation. According to Lemma 5.3 in ch. 1, there exists for every k an d @I &-measurable function p k such that the Lebesgue decomposition of Pk with respect to m may be written pk(x,
dY)
=
fik(x,
Y ) m(dy)
+
dY).
-
We shall write P:(x, ) for the first part of this decomposition. Because of the irreducibility, for every x there is an n such that P:(x, E ) > 0 and actually, it is easily proved (ch. 3,Exercise 2.14) that Pi( , E ) decreases to zero as n increases to infinity.
-
Theorem 2.2. For a Harris chain there are only two possibilities: (i) either for a n y pair ( v l , vz) of probability measwes on d lim llvlPn - vzPnII ?t--bW
=
0,
CYCLIC CLASSES
CH. 6, 92
195
(ii) or there exist a n integer d and a measurable partition c1, cz, . . ., c d , F m(F) = 0 and P( * , Ci)= 1 on Ci-l, P( , Cl)= 1 on C d .
of E szcch that
<
Proof. I t is easily shown that ak+[ ak + at and it follows that the set G = { k : ak = 0 } is a subgroup of the group Z.We consequently have G = { 0 } , G = Z or G = d Z for an integer d > 1. We are going to prove that the first case is impossible and that the latter two correspond to the two situations described above. Let us first suppose that G = (0); by discarding if necessary a set of measure zero we may assume that 1IP,+k(x, * ) - Pn(x, )I[ = 2 for every x in E and every integers n , k > 0. Pick a point x and an integer n such that P,"(x,E ) > 0 and set A = ( y : fin(%,y) > 0). Let B be a set of measure zero such that P:(x, Bc) = 0. Because the chain is Harris, we can find k > n such that P,(x, A\B) > 0, hence such that @k(x, ) is strictly positive on a subset of AB \ of positive m-measure. I t follows that
-
2:
I IPn(x, ) - P k ( x , )I I < 2, '
*
which is a contradiction. The case G = (0) is impossible. If G = Z we have in particular a1 = 0 and since the bounded harmonic functions are constant it follows at once from Theorem 1.8 that (i) holds. We now turn to the case G = dZ with d > 1. By discarding an m-null set which will eventually be lumped with the set F of the statement we may assume that for every x in E , lim
IIPn+d(X.
'
) - Pn(x>
n
*
)/I
=
O,
IIPn+j(xt
*
) - Pn(x,
*
)/I =
for every n > 0 and every j which is not a multiple of d. Pick an arbitrary point xo in E , and for i = 1, 2,. . ., d, set a,
B, =
n=O
{ y :Pnd+j(XO, y )
> O>*
For every pair (2, j ) , we have m(Bi n B,) = 0; otherwise one could find integers n, and np such that
m ( y :pnld+i(XO>y ) >
and p n 2 d + i ( x 0 * y )
-
> O} >
-
and it would follow that IIPn,d+i(~O, ) - P,@+j(%O,)]I < 2 which is impossible. On the other hand, because of the recurrence property and the fact that P;(xo, ) is singular with respect to m, the union of the sets
2;
CH. 6, §2
ERGODIC THEORY O F HARRIS CHAINS
196
B , is equal to E up to a set of measure zero. We can therefore assume that the sets B , are painvise disjoint and that their union has full measure. For i # i 1 (modulo d ) , set D = { y E B i : P (y, B,) > 0 } and suppose m(D) > 0. There exists an integer n such that Pl$+Jx0,D ) > 0, hence such that
+
But this is impossible since B , n Bi+l is empty. As a result m(D) = 0 and P( , Bi+l) = 1 m-a.e. on B,. By defining
-
Ci = Bi\{y
and F =
(uiCJ
E Bi: P ( y ,Bi+I) < 1)
we get the situation (ii).
Dcfinition 2.3. In the first case of Theorem 2.2 the chain is said to be aperiodic. In the second case it is said to be periodic and the integer d is called the period of X . The sets C, are called the cyclic classes and the partition of E into C1,C 2 , .. . , C,, F is a cyclic decomposition of E . This cyclic decomposition is unique up to equivalence. I t is worth recording that the limit property in part (i) of Theorem 2.2 characterizes aperiodic Harris chains among conservative and ergodic chains (see Exercise 2.12). In that case the asymptotic a-algebra is a.s.trivia1 as we have just seen. In the periodic case these properties have a weaker version.
Proposition 2.4. If X is periodic with period d and if v1 and v 2 are two probabilities which agree on F and satisfy vl(Ci) = vz(Ci) for every i d , then
<
lim II(vl - vZ)PnlI= 0. n
Furthermore the a-algebra d is atomic i n the sense that every bounded asymptotic random variable can be written, u# to equivalence, d
=
C aillirn(X&ECi)
i=l
H-m
where the ui's are arbitrary real numbers. Proof. The first sentence is an easy consequence from the fact that each Ci
CYCLIC CLASSES
CH. 6, $2
197
is absorbing for P , and that P, restricted to C, is Harris and aperiodic. To prove the second sentence we first restrict ourselves to P, for x in F". The events = lim {XndE Ci}are in J&' and PZ[ri] = 0 or 1 according as x belongs to C, or not. In other words P, is carried by Tiif x E Ci. Let now be an arbitrary event of d and h the associated bounded Qharmonic function; from the inequality
r,
r
IW n) - W ' ,41 d llhll I(P*(X, - 1 - PdX', * )I( and the convergence property of the first sentence it follows that, for every n , h( , n ) is constant on each Ci.If x E C,, h(Xn,n) is thus constant P,-a.s. for every n and consequently, its almost-sure limit 1, is constant P,-as. This constant can only be zero or one, hence either = 0 P,-a.s. or = Ti P,-a .s , For every bounded asymptotic random variable Z there are therefore real numbers a, such that Z = ailri P,-as. for every x in F". Call h the space-time harmonic function associated with 2. For x in F",we have
r
r
zf=,
if x is in F , since F is transient, we eventually have P,-as.
zf=l ailri P,-as. As h(Xn,n) converges also to Z P,-a.s.
which converges to the proof is complete.
Remark. The reader will find in Exercise 2.20 another description of d in the periodic case. As in ch. 3 $1 we lay down the following Definition 2.4. The chain X is said to be positive or ergodic if m ( E ) < co (and we then always make m(E) = 1 ) and null if m(E) = 03. Our next result says that aperiodic positive chains evolve towards a stationary state.
19H
ERGODIC THEORY OF HARRIS CHAINS
CH. 6, $2
Proposition 2.5. If X is positive and aperiodic, lim llvPn - m ( (= 0 n
for any probability measure v on 8.
Proof. Make v1
=Y
and v2 = m in Theorem 2.2(i). I
Null chains and positive chains have quite different asymptotic behaviours. We have just seen that, for a positive and aperiodic chain, limn P, f(x) = m(f) for every x E E and f E bd. The following result, which does not require X to be aperiodic, shows that a null chain is “less recurrent” than a positive chain. Other evidence to support this statement is given in ch. 3, Corollary 1.10 and in Exercises 2.10-2.11.
Theorem 2.6. lf X is null, for every function f E L1(m)n bB and every X E E lim P, f(x) = 0. n-+W
Proof. We shall show the more stringent property: for every x E E and E > 0, lim{P,f(x)/(m(f) n
+ 41 = 0,
the convergence being uniform in f E a+n Ll(m). We first assume that X is aperiodic. If our statement-were false, there would exist asequence{n,} tendingtoinfinity, asequenceof functions f,in @+nL1(m), a point xo, and two numbers E , 6 > 0 such that p n , fk(xO)/(m(fk)
+
2
for every k. Since m(E) = co, Egoroff’s theorem and Theorem 2.2 allow us to pick a set B in I such that m(B) > 26-’ and such that IIP?Ik(XO, *
1 - Pn,(Y, * ,I1
tends to zero uniformly in y on B. Hence
if k is sufficiently large to have
CYCLIC CLASSES
CH. 6, 82
199
on B. These two relations together imply 1
> m ( f r ) / ( m ( f+ k)E) 2
I.
m(dx) (6 - ( t 4 / ( m ( f d
+ 4) 2 fSm(B),
which is a contradiction. Suppose that X is periodic and let C1,.. ., C,, F be a cyclic decomposition of E . If x E Ci, then P(fl,) ( x ) = 0. Considering the chain on C, of T.P. Pd, we conclude from above that P,f(x)/(m(f) E ) tends to zero uniformly on Q, n L1(m).If x E F , then for Y < n
+
Since F is transient, Pn(x,F ) tends to zero as n tends to infinity, and one can choose Y large enough to make the second term on the right as small as we pleaseuniformly inn andf. After r hasbeenchosen,sincePn-jf(x5)/(m(f) E ) tends to zero on E\F, and is dominated by the integrable function lie, we can make the first term as small as we choose by taking n sufficiently large.
+
Exercise 2.7. A point transformation 8 on ( E , 8) (ch. 1, Examples 1.4) possessing an invariant probability measure v, is said to be strongly mixing if, for every pair ( A , B ) of sets in 8, lim v(O-n(A) n B )
=
v(A)v(B).
n
(1) Prove that if the limit property holds for all A's and B's in a semialgebra generating 8,then 0 is strongly mixing. Observe also that a strongly mixing 0 is ergodic (ch. 4, Exercise 3.13). (2) Show that the shift operator 8 of an aperiodic positive Harris chain is strongly mixing on (Q, 9, P,). If X is a Markov chain with invariant probability measure m, 0 is ergodic if and only if P,f converges to m(f) in the a(L1, LW) sense for every f E L1(m).
Exercise 2.8. Prove by an example that an aperiodic Harris chain may induce on a subset a periodic trace chain. Exercise 2.9. If X is periodic and in duality with a chain X,then is periodic with the same period and cyclic classes (up to equivalence) but these are visited in reverse order.
ERGODIC THEORY O F HARRIS CHAINS
200
CH. 6. $2
Exercise 2.10. If X is positive prove that m ( A ) > 0 implies that E,[S,] is finite for m-almost every x in E. [Hint: Start with the equality mH, = m.] Exercise 2.11. Suppose that E is an LCCB space and that m is Radon. Prove that there exists a sequence {nK}of integers such that { X n h }converges almostsurely to d if and only if X is null. Exercise 2.12. Let v be a a-finite measure such that vP << v and suppose that (i) the induced operator on L1(v)is conservative; (ii) for every pair ( x , y) of points in E , lim IIPn(x,* ) - Pn(y, U
)I[ = 0;
prove then that P is an aperiodic Harris chain. [Hint: Use the results of ch. 4 54.1
Exercise 2.13. Show,that,if X is Harris and positive, then for every f'k Lp(m), the sequence P,f converges to m(f)in the L* sense (1 p < a).
<
Exercise 2.14. Prove that if X is Harris, for every pair ( x , y) of points in E the sequence of functions $,,(x, ) - p,(y, ) converges to zero in L1(m).If X is positive p,(x, * ) converges to 7 in Ll(m). +
-
Exercise 2.15. If X is null then P,f converges to zero for all functions f which belong to the closure of b d n Ll(m) in the topology of uniform convergence. If E is LCCB and P maps C, into Co, then P,f converges to zero for all functions f in C,. Exercise 2.16. With the notation of Exercise 1.14prove that if the left random walk of law p is Harris then His open, G / H is finite, the cosets of H a r e the cyclic classes and the inclusion of Exercise 1.14 is an equality. What can be said along the same line if X is not Harris but merely recurrent ? Excrcise 2.17. Let m be an invariant probability measure for P. Show that the following two conditions are equivalent : (i) for m @ m-almost every ( x , y) there exists an integer n such that Pn(x, ) and P,(y, - ) are not singular with respect to one another;
-
CH. 6, 53
QUASI-COMPACT T R A N S I T I O N PROBABILITIES
201
(ii) for m-almost every x
Exercise 2.18. (1) Prove that there exist Harris random walks on the group of rigid motions of the Euclidean plane. .(2) Let y be the law of such a random walk. For every pair (vl, vz) of probability measures on the plane,
(Of course the plane is to be considered as a homogeneous space of the group of rigid motions.)
Exercise 2.19. For resolvents recurrent in the sense of Harris (ch. 3, Exercise 2.19), state and prove results similar to Theorems 2.2 and 2.6.
[Hint: One should look for the asymptotic behaviour of aV, as a tends to zero. For a < 1 one can write m
aV,
=
2 (1 - a)" (V1)n+l.] n=O
Exercise 2.20. If X is periodic and the set F is empty, d is axequal to the a-algebra generated by the events { X , E Ci>. Exercise 2.21. Extend the results of this section to Harris chains with a non-separable state space. [Hint : Use admissible a-algebras.]
3. Quasi-compact transition probabilities and the strong ergodic theorem Following the usual terminology for convergence of operators on a Banach space, one defines, for the contractions P, of bb, several types of convergence.
Definition 3.1. The operators P, are said to converge to a bounded operator T (i) uniformly if
ERGODIC THEORY O F HARRIS CHAINS
202
(ii) strongly if, for every f
CH. 6, $3
E b&,
lim I(Pnf- T f / /= lim suplP, f(x) - T f(x)I = 0 ; n+m
n+m
(iii) weakly if, for every f
Ebb
XEE
and every
Y E b.M(b),
lim (v, P,f - T f ) = 0. w r n
The weak convergence is equivalent to pointwise convergence on E of functions P,f. Let us also observe that if P is a T.P. and {P,} converges uniformly to T , then T is itself a T.P. Theorem 2.4 asserts that if X is positive recurrent, the operators Pnd converge weakly to the operator with finite range 1, €9 mi. The present section is devoted to the study of the necessary and sufficient conditions for this convergence to be uniform. For this purpose we need the following
cf='=l
Definition 3.2. The transition probability P is said to be quasi-compact if there exists a sequence {U,} of compact operators on b b such that limllP, - U,II = 0. n
In the following section, we shall show that with each Harris chain we can associate plenty of quasi-compact operators. We begin with another definition of quasi-compactness.
Proposition 3.3. The T.P. P is quasi-compact if and only if there exist a compact operator U and an integer no such that
Proof. The necessity is obvious. To prove the sufficiency, let us first recall that the set of compact operators is a closed two-sided ideal in the algebra of bounded operators. Assume that Pno= U Q, where U is compact and IlQll < 1. An integer n > no may be written n = mnO 1 in only one way with m 3 1 and 0 < I < no;so P n = ( u Q)" P I = u n Q" PI,
+
+
+
+
where U , is compact. Since
lie" pLitG (..~III',II) IIQTI 0, +
I
the T.P. P is quasi-compact.
n+m
CH. 6, 53
QUASI-COMPACT TRANSITION PROBABILITIES
203
Remark. In Definition 3.2 and Proposition 3.3, one can put “operator with finite range” instead of “compact operator” without altering their validity.
Theorem 3.4. The following two statements are equivalent: ( i ) {P,} converges uniformly to an operator of rank one; (ii) P is Harris, aperiodic and quasi-compact. If these equivalent conditions hold, then the chain i s positive and the rank one operator is 7 @ m where m is the invariant probability measure. Proof. If (i) holds, then P is obviously quasi-compact. Moreover the rank one operator is a T.P., hence is of the form g @ m . But g = limn P,7 = 7 and m is clearly invariant. The chain is now obviously m-irreducible, and by 2.4 and the results in ch. 4 3 4, it is easily seen that P is Harris and aperiodic. We leave to the reader as an exercise the task of filling in the details. Suppose conversely that (ii) holds. Since P is quasi-compact, for any f E bB, the sequence { P n f }is relatively compact in bB. Indeed, if we fix E > 0, we may pick a k such that / / P IC UII < 4 2 , where U is a compact operator. A finite number of balls of radius 4 2 cover the set U ( 4 ) ;the balls with the same centers and radius E cover the set Pk(%),which, since E is arbitrary, proves our claim. Now using a diagonal process, from any sequence of integers we may extract a subsequence { n j }such that for every r 2 0, {Pnj+f}converges in bB to a function g,. Obviously P,g,+, = g, for any pair ( r , s), but since X is aperiodic all these functions are equal to the same constant a. For n > n3we consequently have
IlPnf - all
=
IIPn-nj(Pnjf -
< IJPnjf- all.
It follows that {Pnf}converges in bB to a constant S f . The mapping S is a contraction and PS = SP = S . The sequence {P,} converges strongly to S and we now show that this convergence is actually uniform. Let E and k be as above. For f E 4, we have
IlPntkf - sfll = II(Pn - S ) Pkfll
< E + Il(Pn - S ) ufll;
since U ( @ )is relatively compact, the convergence lim Il(P, - S ) Ufll = 0 n
is uniform on %. I t follows upon letting n tend to infinity and then
E
to zero
EAGODIC THEORY O F HARRIS CHAINS
204
CH. 6, 53
that the convergence of {P,}to S is uniform. As a result S is a Transition Probability and the proof is now easily completed. We will now remove the assumption of aperiodicity in the above result.
Theorem 3.6. The following three conditions are equivalent: (i) P i s Harris and quasi-compact; (ii) there is a bounded invariant probability measure m, the bounded harmonic functions are constant and
( I - P ) b t = bob
z:-'
where bob = { f E bb: m ( f )= 0 } ; (iii) the sequence n-I Pk converges uniforndy to a n operator of rank one. If these equivalent conditions hold the chain i s Harris and positive and the rank one operator i s equal to I 8 m.
Proof. (i) 3 (ii). By the preceding result, if X is aperiodic and P quasicompact the chain is positive and for n sufficiently large the norm of the restriction of P , to bobis strictly less than one. As a result I - P is invertible on bob,hence ( I - P) bob = bob and the proof is complete in that case. If X is periodic the T.P., c U , is easily seen to be Harris (ch. 3, Proposition 2.9), aperiodic and quasi-compact and m is still the invariant measure which is thus bounded. Consequently ( I - cU,) b b = bob. But on the other hand the resolvent equation shows that I -cu,
=
(I - P) (I
+ ( 1 - c) U C ) ,
and therefore ( I - P) bb ZI( I - cU,) b b which completes the proof. (ii) * (iii). The hypothesis entails that (Z - P ) is one-to-one and onto from bob into bob. By Banach's theorem it has therefore a continuous inverse Q. For h E %, we have therefore
cPkh
n- 1
n-1
w1
I1
--
- ( I 8 m) h
n-1
I1
which completes the proof. (iii) (i). The proof follows exactly the same pattern as for the proof of (i) (ii) in the preceding result.
CH. 6, 93
QUASI-COMPACT TRANSITION PROBABILITIES
205
The remainder of this section may be skipped without hampering the understanding of the sequel. It will be devoted to a more thorough discussion of quasi-compactness so as t o see how it relates to Harris recurrence in general but the results thus obtained will not be used in the following sections. We first recall a few definitions and facts pertaining to the general theory of Riesz spaces. Definition 3.6. A continuous linear functional on b b is called a mean. Every mean has a minimal decomposition as a sum of a finite measure and of a pure mean; a pure mean is a mean that is singular with respect to all measures. A mean pl is said to be invariant if for every f E bb, we have (pl,
f)
= (pl,
Pf).
Theorem 3.7. If X i s Harris the conditions of Theorem 3.5 are equivalent to each of the following conditions : (iv) there exists no invariant pure mean; (iv)’there exists no positive invariant pure mean; ____ (v) the measure m i s bounded and ( I - P ) bb = bog; (vi) the measure m i s bounded and i s the only invariant mean.
Proof. We first observe that (iv)’ is equivalent to (iv). Indeed, if pl is an invariant pure mean, call y + and pl- the positive and negative part of its minimal decomposition. From the equality y + - y - = pl+P - pl-P, it follows that pl+ y+P and plpl-P; but since PI = 1 these inequalities are in fact equalities. Thus yf and pl- are positive invariant pure means, and the result follows. (iii) (iv). If pl is a pure mean, there exists a set A € 8 such that m ( A )> 0 and y ( A ) = 0. If y were invariant, for every n we should have
<
<
-
and this is impossible since the sequence on the right converges in b b to the constant m ( A ) . (iv) => (i). We begin with the following remark. If P has no invariant pure mean, then Pk, for any integer k , has the same property. Indeed if pl is a plPn is invariant by P , hence pure mean and PPk = pl, then the mean is not pure. Consequently, there is an integer 0 < n < k such that pPn is not
2t-l
206
ERGODIC THEORY OF HARRIS CHAINS
CH. 6, $3
pure, and then pPk = @,Pk., is not pure either, which yields a contradiction. By 5 1. we know that, X being Harris, there exists an integer n such that P , is the sum of an integral kernel K and of another kernel. By ch. 1, Theorem 5.7 there exist sets A , E 8 of arbitrarily large measure such that I,K is a R, where Q is compact operator on bQ. We may therefore write P, = Q compact and m{Rl < 7) > 0. Moreover, we may pick n outside the set of multiples of d in order that m be the only measure invariant by P,. We will be finished if we show that there exists a k such that JIRkll < 1. Let pl be an invariant mean for R and 9 = q ~ +- pl- its minimal decomposition into positive andnegative parts. From plR = pl, we derive plf p+R pl+P,, and since P,1 = 7, p+ = pl+P,. Let v+ = p1 p12, where y1 is a measure and plz a pure mean. The equality pll plz = p1Pn plzP, implies that p1 plrP,, and since X is Harris, vl = qlP,, and thus p12 = p12Pn.Since P , has no invariant pure mean, p2 = 0 and pl+ is zero or a multiple of m. I n the latter case, since m(R1 < 1 ) > 0, we have
+
+
(v+, 1 ) > (pl+,
+ +
<
<
Rl),
which is a contradiction. In the same way we prove that pl- = 0, so that the kernel R has no invariant mean. Consequently, by the Hahn-Banach theorem, we have ( I - R) b b = bb, since a mean which is zero on ( I - R ) b d is R-invariant. For every E > 0, there exists therefore a function g E b b such that Ilg - Rg - 711 < E . Hence
<
E
+ 2/lglln-l;
c;=l
it follows that n-l Rkl converges uniformly to zero, and since R k l is a decreasing function of k , this proves that IfRklll < 1 for k sufficiently large. (ii) 3 (v). Obvious. (v) 3 (vi). The invariant means take on the value zero on ( I - P ) bb. If this subspace is equal to bog, the measure m is the only invariant mean. (vi) * (iv). Obvious. We now turn to a direct study of quasi-compactness. Plainly, if some subsequence of {Pn} converges uniformly to an operator with finite range, the T.P. P is quasi-compact. We are going t o prove that this condition is
CH. 6, 93
QUASI-COMPACT TRANSITION PROBABILITIES
201
also sufficient ; this will be done independently of the results we have already obtained. The space X used below was introduced in the remark following Proposition 1.3.
Proposition 3.8. T h e following two statements are equivalent : (i) the a-algebra of equivalence classes of asymptotic events i s a s . finite; (ii) the dimension of T i s finite. If these equivalent conditions are satisfied, there exists a finite m m b e r of integers d , ( p = 1 , 2 , . ., 7 ) and functions Uo,a in b,, where 6 i s a n integer taken modulo do, such that every element f in .% has a unique representation
.
where C p . 6 E R. I f we set U p = Ekl Up,a,every bounded harmonic fwzction f has a unique representation
cbOU0. r
f
=
p=l
Moreover one m a y choose the functions Up,din one and only one w a y in order that
Proof. The equivalence of (i) and (ii) is an obvious consequence of Proposition 1.3. Assume now that these conditions are satisfied, and consider a finite partition B of D consisting of atoms of d ,For every A E B the event 8;'(A) is also an atom of d ; otherwise we would have l , o O n = l,, l,, with B1 n BR = 0, hence I , = I,, 0 8;' I,, o e;',
+
+
and since these two sets are disjoint we would have a contradiction. Consequently the group 8, (nE Z ) induces a one-to-one group of transformations of 8. One can therefore partition B into 7 classes invariant under 8 and within each, label the atoms as A0,*in'suchaway that 8;'(A,J = Ao,d-.n. It remains to set = P.[A,,,]to get the desired result.
Proposition 3.9. If P i s quasi-compact, then X i s finite dimensional.
208
ERGODIC THEORY O F HARRIS CHAINS
CH. 6, $3
Proof. A Banach space is finite dimensional if and only if it is locally compact. It suffices therefore to prove that the unit ball .X1of $ can be covered, for any E > 0, by a finite number of balls of radius less than E. This amounts to showing that there exist at most a finite number of points in .XI such that their pairwise distance is greater than E . Fix E > 0 and choose n such that IIPn - UII < for a compact operator U. There exist a finite number N of balls of radius $E, with centres f k ( k = 1, 2 , . . ., N ) which cover U(.X,). Therefore the N balls with centre f k and radius && cover Pn(Xl) and one cannot find more than N points in P n ( X l ) whose pairwise distance is greater than E . Then the same property is true for XI,since otherwise there would exist N + 1 points g k in such that, for k # k' and m sufficiently large,
If P is quasi-compact, we are therefore in the situation described in Proposition 3.8,the notation of which we .use below.
Theorem 3.10. If P is quasi-compact, there exist d , 1 ( p = 1 , . . ., r ) bounded Up.a = 1 , and measurable functions Up,a(6 = 1 , 2 , . . ., d ) suck that probability measures mp,acarried by the pairwise disjoint sets ED,a= {Up,d = 1) such that, if d denotes the least common multiple of the d,, then for every k ,
2, cd
lim n+m
I1
Pnd+k
7
I/
dP
- 2 2 U p . d - k 8 llZp,d = O* p=lr=l
The sets E , = U, Ep;6are absorbing sets, and the chain induced on E Dis a Harris mDSa. T h e sets Epeaand the number d , chain with invariant measure m, = are the corresponding cyclic class'es and period.
cd
Proof. We have already shown in the proof of Theorem 3.4 that for f E b d the sequence {Pnf}is relatively compact in bd. Therefore using the diagonal procedure there exists a sequence n j such that g( , m) = limj Pnjd-,,fexists for every m in the sense of convergence in the norm of bg. Clearly the function g is in .XI and by virtue of Proposition 3.8, g( , nd + m) = g( * , m) for every n and therefore Pn&&( , 0) = g( * , m ) as long as nd m 0.
-
-
>
CH. 6, $3
Consequently, if nd 2 njd IIPnd-mf
209
QUASI-COMPACT T R A N S I T I O N P R O B A B I L I T I E S
- g( * * m)ll
=
+ m,
IIp(n-y)d-m
(Pnjf
- g( *, O))ll
<
lIPnjdf
- g( ' O).ll. P
Letting n, and then n j , tend to infinity, we get that Pnd-,f converges to g( , m) in bb, and thus the operators Pnd-, converge strongly. This allows us to define a contraction s of b b by setting sf = limn Pndf. For every integer m, SPmd = P,dS = S. Exactly as in the proof of Theorem 3.4 we can then prove that the convergence limn P,, = S is uniform. This implies that P n d + m also converges uniformly to P,S = SP,. By virtue of Proposition 3.8, for every f E b b there exist uniquely determined numbers C,,,(f) such that 9
Since S is the uniform limit of tfdnsition probabilities, it is easily seen that the mappings f c p , d ( f ) are defined by probability measures mp,d, and thus S can be written +
r
S
=
dp
CC
p-1 d = 1
up,a
8 mp.6.
imply that ? n p , d P , = mp,d+n. Now denote Ep,a= { u p , d = I}. Clearly mp,d(Ep,d)= 1, and by Proposition 3.8 the sets are pairwise disjoint. From the equality
- uPnd(x)
=
I
p ( x , dy) (7 -
uo.d+I(Y))
it follows that EOed = { x : P(x, Ep,d+l) = 1). The sets Epare therefore absorbing sets, the measure mp is clearly invariant and by the uniform convergence the restriction of X to E , is m,-irreducible and therefore recurrent in the sense of Harris. The last statement is obvious. We finally record that quasi-compactness has been used in another guise.
210
ERGODIC THEORY O F HARRIS CHAINS
CH. 6, $3
If we denote by v any convex combination of the probability measures mpr6, it is easily seen that P satisfies the following condition, known as Doeblin's condition: there exist an integer n, two real numbers 8 < 1 and 17 > 0, and a $robability measure v on ( E , 8)such that, for A E 8,
v(A) > 8
implies
P,( -,A ) >, 7 on E.
This condition is in fact also sufficient for P to be quasi-compact (see Exercise 3.17 and the Notes and comments).
Exercise 3.11. If E is finite, all T.P.'s on E are quasi-compact. Show by an example that the union of the sets E p 9 6may be a proper subset of E. Exercise 3.12. Set U ,
=
2 k l Up,,, and prove that r
P, - 2 Up @ m p p=l
State and prove a similar result for positive Harris chains.
Exercise 3.13. With the notations of Proposition 3.8 and Theorem 3.10 show that A p , a= lim{X, E.E,,~+,} a s . n
Prove a similar result for invariant events. Prove that
Can one give a simiiar result for U p , a?
Exercise 3.14. In the situation of Proposition 3.8, we set 1o.k
=
exp(2i.rrk/do),
f0.k
=
2
(jl0.k)'
up.6,
8
where k is an integer modulo d,. Prove that P f p , k = A p . k f p , k . Show that for every complex number 1 of modulus 1, the space of bounded measurable functions f such that Pf = A/ is finite dimensional and admits as a basis equal to 1. the f o , k corresponding to those of the [Hint: Notice that g( *, n) = 1-"f is Q-harmonic.]
Exercise 3.16. In Theorem 3.5(iii), one can use strong convergence instead of uniform convergence.
CH. 6, 94
SPECIAL FUNCTIONS
21 1
Exercise 3.16. If a T.P. satisfies Theorem 3.7(iv), then there exists a bounded invariant measure. [Hint: Make P operate on the convex compact set (for a(bQ*, bQ)) of positive invariant means, and apply a fixed point theorem.]
Exercisc 3.17. For a Harris chain prove that quasi-compactness is equivalent to Doeblin's condition. Exercise 3.18. Let X be Harris and positive and aperiodic and B the set of Lemma 2.4 in ch. 3. Prove that P is quasi-compact if and only if there is a bounded operator G on B such that for every v E B lim C v P , - v G nII,
I/
=0
the norm being that of B. Then ( I - P ) G = I - I @ m ; G is thus a potential operator for measures (see $5).
Exercise 3.19. Let M be a compact homogeneous space of agroup G . We assume that there exists a continuous cross-section s: M -+ G.Prove that, if p is a spread-out probability measure on G , the T.P. p * E , ( x E M ) is quasi-compact, and derive a convergence theorem. (See example in ch. 5 $ 1.10.) In particular, if G is a compact connected group with normalized Haar measure m, and p a spread-out probability measure on G , then lim supJ(pn* E, - ml/= lim suplle, * pa - mJI = 0. n--+w %€G
'
n+m
xeG
Exercise 3.20. Let ( V J ,a > 0, be a resolvent such that V1 is quasi-compact. Prove that, as a tends to zero, orb', converges uniformly to an operator of finite rank. (See also Exercises 2.19 and 4.14.)
4. Special functions We proceed with the study of Harris chains. Our goal is twofold: we want to build a potential theory suitable for the recurrent case - in particular we want to solve the Poisson equations - this will be initiated in the next section and achieved in chs. 8 and 9 ; on the other hand we want to show a general quotient limit theorem generalizing ch. 4, Exercise 4.10, and this will be done in S 6. Throughout this study the class of functions to be defined below will play a prominent role.
CH. 6, $4
ERGODIC THEORY O F HARRIS CHAINS
212
Definition 4.1. A function f E 8, is said to be special if for every non-negligible function g E %+, the function U s ( f )is bounded. A set A E d is called special if its characteristic function 1, is special. Proposition 4.2. The set 9 'of special functions i s a convex hereditary sub-cone of L:(m).
Proof. Let g E %+ and 0 < m(g) < 0;); by ch. 3, Proposition 2.9,
4 1 ) = m(sU,(f))< m(g)I l q f ) I l < a. Moreover, every function dominated by a special function is obviously special. To give a first characterization of special functions as well as for other important purposes, we need the following:
Proposition 4.3. There exists a strictly fositive function h E 4, such that m(h) < 1 and uh > 1 @ m. Proof. By ch. 3, Proposition 2.7, there is a strictly positive function ho E %+ and a measure mo equivalent to m such that uho2 1 @ m,. Let f be a Strictly positive function such that m, = fm. The function h = hof/(l f ) is a strictly positive function of %+, and
+
Uh
> uholho-huh 2 1 @ ((ho- h) mo) Uh =
1 @ (hm) Uh = 1 @ m
by virtue of ch. 3, Proposition 2.9. Since 1 = U,(h) > m(h), the proof is complete.
Proposition 4.4. Let h, be a strictly positive functionin %+ such that Uh, 2 1 @ mo for a positive measure mo.Then a function f E &+ is special if and only if uho(f) is bounded on E .
<
Proof. The necessity is obvious. Conversely suppose uho(f) a ; for0 < h we have Uh,(ho - h) = 1 - Uho(h) 1 - mo(h)< 7, and therefore
< h,,
<
I f h is the function of Proposition 4.3,the function h' = h A ho has the same
CH. 6, 94
SPECIAL FUNCTIONS
<
213
properties as h, since u h * 2 u h and 0 < h‘ ho, so that u h * ( f ) 0,
<
U,(f) d a’/m(g) < a* Finally, if g is any non-negligible function in 4+,the function g A h’ satisfies the preceding conditions, and since U,(f) UgA,,(f),we get the desired result.
<
Remark. Special functions are integrable by the measure mo. Corollary 4.5. T h e cone V of special functions i s invariant under the operators where h E 4 , and m(h) > 0. I n $articular, it i s invariant under P.
Ihuh,
and this is finite since f is special. The following corollary shows that there exist special functions, and that E is the union of ari increasing sequence of special sets.
Corollary 4.6. Every strictly positive function ho E 9,such that Uh, 2 I 8 m, for a non-zero measure m, i s sfiecial. Proof. Since Uho(ho)= I this follows immediately from Proposition 4.4. I t follows from ch. 3, Lemma 2.4 and ch. 6 $3 that the T.P. U h o I h o is quasi-compact. We shall now show a converse to this result, which will lead to another characterization of special functions.
Lemma 4.7. Let h E @+, c be a real number in [O, I[ and A E 8. For every T.P. on ( E , 8)the following inequality holds on { u A ( I A ) = I } :
CH. 6, 54
ERGODIC THEORY O F HARRIS CHAINS
214
Proof. By ch. 2, Definition 2.4, r
1
If
U A I A ( X ) = 1, the stopping time S A is P,-a.s. finite. The inequality 1 - ch(x) 2 (1 - c ) ~ ' , ) , which holds since c and h(x) are in [0, 11, and Jenssen's inequality then imply
the proof is complete.
Proposition 4.8. For every non-negligible bounded special function f and every positive real number c strictly less than llfll-l, there is a measure ,uoequivalent to m such that U,, 2 I 8 pC.
[If11
Proof. We may suppose < 1. By the proof of Proposition 2.2 in ch. 3, for 0 < c < 1, there are two strictly positive functions a and b such that U c a @I bm, and consequently
>
The function c(1 - f ) a being strictly positive on E , there is a real E > 0 and a set A of positive m-measure such that & I A c(1 - f ) a. By Lemma 4.7, we have therefore
<
Since f is special, U,(f) is bounded; hence
and it remains to call this measure ,uc to get the desired result. If we argue as in ch. 3, Theorem 2.5, the preceding proposition implies
CH. 6, 54
S P E C I A L FUNCTIONS
215
that c U e f I fis quasi-compact. This will be used in the following important results.
Proposition 4.9. If X is Harris, the following two statements are equivalent: (i) P is quasi-compact; (ii) every positive bounded function is special.
Proof. If f is special for every integer n, the function f, = n-1 Ckn,lPkf is special by virtue of Corollary 4.5. If P is quasi-compact the sequence (1,) converges uniformly to m ( f ) . If m (f) > 0, there is therefore an integer n such that f n majorizes a positive constant, and consequently all bounded functions are special. Conversely if the function 7 is special, Proposition 4.8 implies that for c E 30, 1[ the T.P. c U , is quasi-compact ; hence, by Theorem 3.5, and its proof P is quasi-compact.
Remark. The last proof could be written using Theorem 3.7 instead of Theorem 3.5. In that way one would avoid the use of the results in ch. 4 $4. Theorem 4.10. A non-negligible function f transition function U f I fi s quasi-compact.
E
9,is special if and only if the
Proof. I t is known (ch. 3, Proposition 2.9) that the T.P. Q = U f I fis Harris, and that for h E @+, the kernel Uf is equal to U h f I f .Let g ebb,; f g is special for P and therefore @(g) = Uaf(/g)is bounded on E . The bounded functions are therefore special for Q; hence Q is quasi-compact. Conversely, if Q is quasi-compact and if h, E @+ is such that uho2 1 @ mo, then uho(f)
<
uhof(f)
=
u!0(7)~
which is a bounded function. Thus f is special and the proof is complete.
Exercise 4.11. (1) Prove that, if E is countable, all functions with finite support are special. More generally if E is LCCB and if P is strong Feller, the bounded functions with compact support are special. The result can even be shown for Feller transition probabilities. [This is solved in ch. 8 (2) For a random walk recurrent in the sense of Harris, all bounded functions with compact support are special.
w.3
216
ERGODIC THEORY O F HARRIS CHAINS
CH. 6, $5
Exercise 4.12. If f E @+ let Q = U,I,, as in Theorem 4.10. Show that if g is special for Q then fg is special for P. Is the converse true? Exercise 4.13. Prove that, for every bounded special function f , there is constant c such that U,,I,, is 1-quasi-compact, i.e. there is a compact operator K such that ~ ~ U c fI cKI1 f < 1. Exercise 4.14. Let the resolvent { V a } be, Harris. (1) Prove that the set of special functions is the same for all the chains Xa and is equal to the set of functions f such that v h ( f ) is bounded whenever v h > 1 @ m. We shall call these functions the seecia1 functions of the resolvent { V a } . ( 2 ) If aVa is quasi-compact for an a > 0, it is quasi-compact for every a > 0. (See also Exercise 3.20.) (3) Prove that f is special if and only if V,I, is quasi-compact. Exercise 4.16. Let P be a T.P. enjoying the two following properties: (i) There is a measure v such that vP <( v and P induces a conservative and ergodic contraction of L1(v). (ii) There is a function f E Q, such that v(f) > 0 and U,Z, is quasi-compact. Then prove that there exists an absorbing set F such that v(F)= 0 and that the restriction of P to F is Harris. Exercise 4.16. For A E 8,m(A) > 0, a function f is said to be A-special if for every h E 4 , with m(h) > 0 we have sup A
uh(f)
<
(1) Prove that if f is A-special, bounded and non-negligible there exists a measure m’ equivalent to wt and a number c < 11-l such that U,, I , @ m’. (2) Prove that f is A-special if and only if U,f is bounded on A and is special for the T.P. Q = UAZA.If A is special, f is A-special if and only if U,f is bounded on A. ( 3 ) If f is A-special, bounded, and vanishes outside A, then f is special.
/If
>
6. Potential kernels In ch. 2 it was shown that for transient chains, the kernel G gives rise to a satisfactory potential theory, and in particular permits us to solve the Poisson
CH. 6, 95
217
POTENTIAL KERNELS
equation. In the present situation of Harris chains, the kernel G is no longer proper, and we shall look for another kernel which could be used for similar purposes.
Definition 6.1. A charge is a function f E b l such that I/l is special andm(f) = 0. The set of charges will be denoted by H.A proper kernel is said to be a potential kernel if it maps special functions into finite functions and if, for every f E N , (I - P) = 1.
r
rf
Theorem 6.2. W i t h each function h in %+ such that Uh > 1 @ m, one can associate a positive proper kernel W such that, for every non-negligible function gE
@+I
U,
+ WI,U,
The positive proper kernel
1
=W
+ -2.-8 (hm) U,.
r = I + W i s a potential kernel.
Proof. For clarity we divide the proof into four parts. (1) We may write U h = V + 7 €3 m with V a strictly positive kernel. Since U h ( h ) = 7 and (hm) Uh = m, we have V h = 7 - m(h),
(hm) V
=
( 1 - m(h))m.
Recalling the formula uO
(Uhrh)n
=F
uhr
n>O
we define
w = n>O 2 ('VIh)n T.' = 2 V(IhV)n. n>O
The kernel W is clearly a positive kernel, and setting q = (1 - m(h))/m(h), we have Wh= (1 - m(h))"+l = q,
2
n>O
and ( h m ) W = qm. (2) I t is easily seen that
V
=
P
+ PI,_,V
- Ph €3 m.
ERGODIC THEORY O F HARRIS CHAINS
218
Thus the obvious identity W = V
w = P + PIl-hV But since (hm)W
w
=P
- Ph
@m
+ VI,W
CH. 0, $5
may be written
+ PIhw f PIl-),vIhw
w.
- Ph @ (hm)
qm, i t follows that
=
+ PIhw + PIl-h(v + V I n w ) - Ph 8 (1 f 9) m,
and consequently
W =P
+ PW - lPh/m(h))@I m,
(5.3)
which is eq. (5.1) in the special case g = 7. From this equation, it is easily derived by induction that, for every integer n,
Let c be a real number with 0
< c < 1;from the last displayed equality we get
Now it is easily calculated that
2 (1 - c)"-'(-$
P.) = c-luo,
*,I
and consequently
+
U, cU,W
= W
+ (U,h/m(h))@ m,
(5.4)
which is eq. (5.1) in the special case g = c. In analogous fashion, we get the following special cases of eq. (5.2) :
+ W P = W + m(h)-' u, + cwu, = w + m(h)-' P
@
(hm)P ,
@ (hm)
u,.
(5.5) (5.6)
(3) Now let g be a non-negligible function in @+, with llgll < 1. We may write eq. (5.3)in the form
P(I
+ I,w) + PI1-,W
=W
+ (Ph/m(h))@ m.
We repeat 9 times the following computation: multiply through the above equality to the left by PIl-, and add P ( I + I,W) to both sides. We get
2 (PI,-,)"
n e
P(I
+ I,W + (PI,-,)P+' w
CH. G. $6
POTENTIAL KERNELS
= W
219
+ ( 2(J'I1-Jn Ph/m(h))8 m. n
Letting
p
tend to infinity in the resulting inequality
yields
U,(I
+ I,W) < w + ( q h / m ( h ) )8 m.
(5.7)
Pick a number c such that llgll < c < 1; the function h' = cUch is strictly positive, and if we apply h' to the right of both sides of eq. (5.7) we get, on account of eq. (5.6),
+ U,I,(Wh - Uch + m(h)-l (hm)U,h) < Wh - Uch + m(h)-l (hm) Uch + U f i .
cU,U,h
Using the fact that W h is constant, this inequality reduces, after cancellation, to CU,Uch - UgIgUch U f i - Uch;
<
but the left-hand side is equal to UgIC-,Uch,and by the resolvent equation of ch. 2 5 2, this inequality is in fact an equality. As a result, the hequality (5.7)is an equality, and eq. (5.1) holds in that case. Finally, if llgll = 1, we deduce from the preceding discussion and the resolvent equation ug= ucg (1 - 4 U/,UC,#
+
where 0 < c < 1, that eq. (5.1) is true in full generality. We now turn to the proof of eq. (5.2). Following the same pattern as above, with right multiplications instead of left multiplications, we get, for any non-negligible function g in 4+,
U,
+ WIgUg< W + m(h)-l @ (hm) Ug.
We claim that this inequality is in fact an equality. From eq. (5.6), it follows that Wh' = cWUch W h (h, cU,h)/m(h) is a bounded function. Consequently, if 0 / h', the function
< < < k
=
U,t
+
+ W I g U g f- W f - (h, U,t)/m(h)
is bounded. Moreover, k is harmonic; this follows from writing the equalities
ERGODIC THEORY O F HARRIS CHAINS
220
P
+ P(W + m(h)-1 @ (hm)U,)
=W
CH. 6. 56
+ m(h)-l €3 (hm)U , + (Ph/m(h))@
WI
and taking their difference. By ch. 3 5 2, the function k is thus constant. On the other hand, combining the two equalities
(hm) (U, (hm) (W
+ w',q
= (h4
u, + qm
+ m(h)-' €3 (hm) U,) = qm + W )u,
proves, since the measure hm is bounded, that (hm, k ) = 0 and therefore that k is identically zero. The proof of eq. (5.2) is thus complete. (4) We have already noticed that Wh' is bounded; since h' is strictly positive this implies that W is a proper kernel. Furthermore, if f is special, eq. (5.2) written for g = k yields
wf <
<
+ W(hUAf)
vA(f)
IlvA(f)ll
(l
+ w(h))=
IIUhfll/m(h)*
The kernel W thus maps special functions into bounded functions and =I W maps special functions into finite functions. Moreover, if f E M , then by eq. (5.1),
r
+
f which reads
+ W f = f + P f + PWf, (I - P)
We observe that
rf = f .
rf is then bounded.
Corollary 6.3. Let f be a charge; then the Poisson equation (I--P)g=f has a bounded solution, unique u$ to addition of a constant function.
Proof. By the preceding result, I'f is a bounded solution of the Poisson equation. The difference between two such solutions is a bounded harmonic function, hence a constant function.
CH. 6, 95
POTENTIAL KERNELS
221
r
The potential kernel is not unique and there are always many potential kernels. Indeed, if q~is a finite function and v a measure taking finite values on special functions, then the kernel I' q~ 69 m I @ v is also a potential kernel. In the following result, we prove that in this way we obtain all possible potential kernels. We notice that some of these kernels are not positive.
+
+
r
Proposition 6.4. Let and r' be two potential kernels; then t h r e i s a finite function q~ and a measure v taking finite values on special functions such that
r'= r + p @ m + I @ v .
rf
Proof. I f f E M , then and I"f are two bounded solutions of the Poisson equation and thus differ by a constant. Pick xo in E and set =
the measure
Y
ryX0, . - r(xo, - 1;
takes finite values on special functions and
r'f - rf + v ( f ) . Now pick a special function f o such that m(fo)= 1. For every special function f , the function f - m(f)f o is a charge, and consequently
I"(f - m ( f )f o ) hence
r'f = rf
-
r(f4- m ( f 0 ) )=
y(f)
+ m ( f )( I f f 0- rfo -
- m ( f )V ( f O ) ,
Y(f0))
+ 4f).
L v ( f O )to get the desired result, since it I t remains to put = r ' f o - rfO is easily seen that two measures equQl on special functions are equal.
r
The case in which and r' are hssociated with positive special functions by Theorem 5.2 is of special intekst, since in that case one can compute p and v. When necessary to avoid possible misinterpretation, we shall write wh and r h for the kernels associated in Theorem 5.2 with the function h. Corollary 6.5. If hl and h2 are two positive special functions such that U , (i = 1 , 2), then the kernels whi (i = 1,2) are related b y the formula
> 1 @I m
222
ERGODIC THEORY OF HARRIS CHAINS
CH. 6, §5
Proof. Within the course of this proof let us abbreviate W,, to W i . B y Proposition 5.4, there is a function q~ and a measure v such that
Since Wl(hl)= q1 and (hzm)W z = qzm, by multiplying this equality to the left by (hzm)and to the right by h, we get
v = m(hz)-l ((hzm)W 1
+ bm),
bER,
and we may write
The existence of potential kernels gives yet another characterization of special functions. Proposition 5.6. A positive function f is special if and only if any potential kernel =I LV p @ m 1 8 v
r
+ +
rfis bounded for
+
such that q~ i s bounded. Proof. The necessity follows from Theorem 5.2 and Proposition 5.4. To prove the sufficiency, we start by noticing that if I'f is bounded, then Wf is bounded, and since (hm)is a bounded measure we have m(f) = q-l(hm) Wf The result then follows from the inequality
< 00.
CH. 6, 55
POTENTIAL KERNELS
223
The next result, another version of which is given in Exercise 6.16, may be seen as a refinement of the ratio-limit theorem of ch. 4, Theorem 4.2 and will be used t o give a sharper form of this theorem in the following section.
Proposition 6.7. If f E N then ,
Proof. From ( I - P ) r f = f , it is easily derived that n
C Pmf = rf- p , , , r f ;
m=O
therefore
We now turn to the problem of solving the Poisson equation for measures. For this purpose, we lay down the following
Definition 6.8. A positive measure ,u on ( E , 8)is said to be sfecial if there is a bounded measure a and a non-negligible function g E 4,, such that ,u aU,.
<
Proposition 6.9. Let h be a strictly positive function in @+ such that u h > 1 @ m. Then a measure ,u is special if and only if there is a bounded measure p such that /A pub.
<
Proof. The sufficiency is obvious. Conversely assume that ,u U , U,,,,, we may assume g h. The measure
<
<
< aU,;
since
is bounded because
By the resolvent equation, we get p u h
=
aU,, which proves the necessity.
Remark. The measure m is special since m = (km) U,.
ERGODIC THEORY O F HARRIS CHAINS
224
CII. 6, $5
Corollary 6.10,Special measures take finite values on special functions. They are thus a-finite.
Proof. Let p and f be special; then
The convex cone of special measures is obviously hereditary to the left and we have the following
Proposition 6.11. The cone of special measures is invariant by the ofierators Ih uh* Proof. Left to the reader as an exercise.
Proposition 5.12. For every bounded measure v such that v ( E ) = 0, there is a measure p such that Ip - vI is sfiecial and v = p(1- P ) . The measure p is unique t@ to addition of a multiple of m .
Proof. If p is a bounded measure, then pW is special since
and
2 dV1h)"
"20
The measure =
= p(E)/m(h)
vr = + v+w- v-w
is such that Ip - v1 is special, and by Theorem 5.2 is the desired measure. We turn to show the latter half of the statement. Let p' be another measure with the same properties. Then I = p - p' is such that \ I ) is special and 1 = 1P,which implies If I*P. It remains to show that the multiples of m are the only special measures I such that 1 IP. If 1 AP, we may obtain inductively the inequality
<
<
<
CH. 6, 96
Then if 1
POTENTIAL KERNELS
225
< pUh, we have +
as pU,(h) = p ( E ) < 03, it follows that limn (Z,-hP)n h = 0, and since h is strictly positive, passing to the limit in the next to the last displayed inequality, we get (hil) (h.1)IhU,,. These two measures are bounded since
<
<
pUh(h)
=
ptE)
< O0
and have the same total variation, hence they are equal. The measure h l is thus U,I,-invariant, and therefore by ch. 3 9 2.9, there is a constant c such that h1 = chm. Since h is strictly positive, 3, = cm.
Remark. The condition that lp - vI is special is essential to have the uniqueness property as will be seen from examples in ch. 9 (see also ch. 3, Exercise 2.20). The reader will also find in Exercise 6.10 the condition under which the solution v may be chosen positive. Exercise 6.13. Prove that the measure (hm) Up,which appears in eq. (6.2), is majorized by a multiple of m. Exercise 6.14. With the notation used at the beginning of $2 prove that the singular part W1 of W with respect to m is equal to Pi.
2;
Exercise 6.16. Let 6 be the set of bounded special functions such that m(f) = 1. (1) Prove that for any f E 6 there is at least one proper kernel I', such that (i) r , g is bounded for any special and bounded g ; (ii) ( I - P) T,= I - f @ m. ( 2 ) If 1and v are two probability measures show that v -< 1if and only if A , = ( ( A - Y ) T,)-is absolutely continuous with respect to m for one (hence for all) f E 6. Exercise 6.16. For every special set K there is a constant C , such that, for every bounded charge f vanishing outside K ,
Exercise 6.17. Assume X aperiodic, and let f be a charge. Prove that for any
226
ERGODIC THEORY O F HARRIS CHAINS
CH. 6, 96
pair (vl, v2) of probability measures on ( E ,&'),
fl+W
r
where is any potential kernel. Derive that 'the sequence WP,f converges pointwise to zero. What can be said for positive periodic X ?
+
Exercise 6.18. The kernel I W I , satisfies the reinforced complete maximum principle. The kernel W satisfies the positive maximum principle, that is, sup W f(x) = sup W f ( x ) . XEE
W > O )
+
Exercise 6.19. In the situation of Theorem 4.10, prove that I W I , is a potential kernel for Q. I f f is special, it is then a bounded kernel. More precisely, I W is bounded if and only if P is quasi-compact,
+
Exercise 6.20 (Potential kernels for Harris resolvents). As for chains, a charge is a function f such that If1 is special and m ( f ) = 0. A potential kernel is a proper kernel W khich maps special functions into finite ones, and which is such that, for every charge f and every a > 0, (1
+ aw)V,f
=
W f = Va(I
+ aW)f .
(1) Prove that there is a whole family of potential kernels and describe
their pairwise relationship. (2) Define special measures and prove a result similar to Theorem 5.12.
6. The ratio-limit theorem We now want t o sharpen ch. 4, Theorem 4.2, which was obtained by a straightforward application of the Chacon-Ornstein theorem along the line of ch. 4, Exercise 4.10. The most general result that one could hope for is the following: for every pair (1,g) of integrable functions and every pair (vl,v2) of probability measures on ( E , &),
Unfortunately, it turns out that this result is not true, but below we shall give several theorems which come very close t o it.
CH. 6, $6
THE RATIO-LIMIT THEOREM
227
Proposition 6.1. Iffand g are special functions and m(g) > 0, for everyprobability measure v on 6, lim
MW
(k m=O
vpmf/>ovpm~)
=
m(t)/m(g).
Proof. The function m(g)f - m(f)g is a charge; thus, by Proposition 5.7, there is a number 111 such that, for every integer n,
Since
C:=ovPmg > O for n sufficiently large,
and this converges to zero as n tends to infinity. In the sequel, we shall use the kernel W associated, as in Theorem 5.2, with a strictly positive function h. We shall denote by A the bounded measure hm, and put a,, = 11111-'(m $= l
Lemma 6.2. For every E
Pmh).
> 0, there is a constant c,
such that, for every integer n,
Proof. Set
< (1 + E ) an Integrating with respect to A the two terms of the inequality n
C Pmh b (1 + 1 yields and therefore
E)
an74
(IJ(1 a n > (1 + E ) anA(A3.
228
ERGODIC THEORY O F HARRIS CHAINS
4An) For f
E
3 11J11~/(1
+
CH. 6 . 56
E).
9L+, it is easy to derive inductively the formula
By applying eq. (6.1) to the function f n = hl,,, we get
But
and, since U,,(fn)
Now, because of the resolvent equation,
Lemma 6.3. FOYevery 6 E 30, 1[ and every probability measuye v, there are two numbers a, ,!l> 0 such that (i) UahIah 2 (l - 6, 1 1 1 1 1 '@ (ii) VUah+B(ah)2 1 - 6. Proof. If a < 1, we have
that is, with the notation of $ 5,
CH. 6, §6
THE RATIO-LIMIT THEOREM
229
easy computations prove that
It is then easily seen that (i) is satisfied for sufficiently small a. The number a being chosen, when p decreases to zero the operator increases to Uahand therefore lim VU,h+,(ah)
uah+,
VUah(Orh)= 7 ;
=
P-0
The condition (ii) is thus satisfied for sufficiently small p.
Lemma 6.4. For every $robability m e a s u r e v on ( E , B),
Proof. Denote by R, the ratio which occurs in the statement. Because of Lemma 6.2, we have E, R , llAl/-l, and we are going to prove the reverse inequality for the inferior limit. Let 6 E 10, 1[ and set
<
B, = The inequality
c
{:
2 P,h
2
(1
- 6 ) a,
I
.
n
Pmh
2 (l - 8) an7Bn,
1
together with eq. (6.1), implies n
2 P,h
\
/ n
2 U f I f 2 P h - U,P,h ( 1
1
2 (l - dl With the notation of Lemma 6.3, put f
anUf(f7B,,)
=
ah
- UfPnh*
+ p ; it follows that
vUf(f7B,,)3 ~Uan+8(ahlB,)3 vUan(ah74,) - 6, since by Lemma 6.3(ii), v(Uahlah
-
Uah+B1ah)
By Lemma 6.3(i),it then follows that
(E)
< 6.
(6.2)
CH. 6, $6
ERGODIC THEORY O F HARRIS CHAINS
230
VUf(flB,)
2 (1 - 6) llA1l-l A(Bn) - 6.
Let us study the asymptotic behaviour of A(Bn).By virtue of Lemma 6.2, we have n
2 Pmh < (1 - 4 a,
on Bi,
1
n
2 Pmh < c, + (1 + E ) an
on Bn;
1
taking the integral with respect to A yields anllYI
G
+ + (1 +
(1 - 6) an(llAll - A(Bn))
hence
6anllJ-ll
< + (6 + (ca
( ~ 8
E)
E)
a,)
A(Bn)t
an) 4Bn).
Since (a,} converges to infinity, we find
lim A(Bn)2 s\lAll/(6 + E ) , n and since E is arbitrary, follows that
l h , A(Bn) 11A11; hence limn A(B,)
=
11A11. It then
lim v U f ( f l B n )2 1 - 26. n
Finally as f
p and Pnh < I , for every n we have
< U b 1 = p-l.
UfPnh Returning to eq. (6.2),we see that
lirn Rn n
llAll-l (1 - 6) (1 - 2 4 ,
and since 6 is arbitrary, the proof is complete.
Theorem 6.5. For every pair ( f ,g) of special functions with m(g)> 0 and every pair ( v l , vz) of Probability measures on ( E ,8)
Proof. Beginning the summations at 0 instead of 1 causes no trouble, since the denominator diverges. By applying Lemma 6.4 with v l , and then with
CH. 6, $6
T H E RATIO-LIMIT THEOREM
231
v2, and taking the ratio, we get
We then apply Proposition 6.1 twice, first to v l , f , h, then to v2,g, h, and writing
we get the desired result. In tlie same way we may obtain
Theorem 6.6. Let f , g be lwo fuizctions i n L1(nz)with m(g) > 0. There exist two negligible sets N , and N , depending only on f and g and such that for x # N , and y 4 N, I n
/
n
\
Proof. I t can be performed by following the pattern of Theorem 6.5, except that instead of Proposition 6.1 we use ch. 4, Theorem 4.2. The set N , is the negligible set which appears in ch. 4, Theorem 4.2, as applied to f and h. Remark. One could think of enlarging the scope of validity of the preceding theorems to functions f such that P,f is special. Exercise 6.7 shows that nothing can be gained in that way. Exercise 6.7. If f is such that U,(f) is special for a function is special.
g E
a+,then f
Exercise 6.8. Prove that if there is a Harris random walk on the group G , then G is unimodular. [Hint: Apply Theorem 6.5 to the left and right random walks with 1 1 = v2 = &,.I Exercise 6.9. Carry over the result of this section to the case of Harris resolvents. Exercise 6.10 (continuation of Exercise 5.15). Retain the notation of ch. 2 95 and suppose that X is Harris.
232
as
CH. 6, $6
ERGODIC THEORY O F HARRIS CHAINS
(1) Prove that either the measure a is a-finite or that @(A) = co as soon m ( A ) > 0. Prove that in the former case v < 1 and 0
=
a(/) m
+ (A - Y) T,
for any f E 6. (2) Prove that the Poisson equation 11 = qP L - v has a a-finite and positive solution if and only if v .< A and = dA,/dm is bounded m-a.e. in which case c is the minimal positive solution and a(/) = for any f E 6. In particular if P is quasi-compact and v < 1and if T is the R.S.time of the filling scheme then E,[T] 3: Il+rllm.
+
$f
Il$fl o
CHAPTER 7
MARTIN BOUNDARY
The purpose of Martin boundary theory is twofold. From the potential theoretic point of view it is to give an integral representation of harmonic functions which turns them into “potentials” of charges carried by the boundary of the space. We thus wish to generalize the classical Poisson integral representation of harmonic functions in the unit &sc, namely the one-to-one correspondence between the non-negative harmonic functions h and the Bore1 measures ph on the circle S1 given by (with classical notation) h(rei@)=
1 - r2 1 - 2r cos (0 - t )
+ r2 dPh(&
The function 1 corresponds to the Lebesgue measure m on S, and if f is the density of ,uh with respect to m, then Fatou’s theorem states that h(rei@) converges to f(t) whenever rei@converges to t non-tangentially. From the probabilistic point of view we wish to study the asymptotic behaviour of transient Markov chains. If the potential kernel G is proper, the space E is the union of an increasing sequence of transient sets, and one can thus say, loosely speaking, that the chain eventually leaves the space. For example, a transient random walk on R with positive first moment converges towards co ; we can say that it leaves the space on the right. We wish to describe in general how a transient chain leaves its state space.
+
1. Regular functions In this section we shall investigate more thoroughly some of the notions introduced in ch. 2 $3. For f E Q we set
R ( f )= Iii-5f ( X , ) , n
and put R(1,) = R ( A ) ,t o be consistent with the notation of ch. 2 $3. The random variable R(f) is invariant. 233
MARTIN BOUNDARY
234
CH. 7, $1
Definition 1.1. A function f in 8 is said to be regular if limn f(X,) exists a s . A set A E d is said to be regular if I , is a regular function. Two regular functions f and g are said to be equivalent if R ( f ) = R(g) a s . Finite superharmonic functions and bounded harmonic functions are regular. Absorbing sets are regular ; transient sets are regular and equivalent to the empty set.
Proposition 1.2. T h e set W of regular functions i s a n algebra (for ordinary $ointwise multiplication) and a lattice. T h e collection of regular sets i s a Boolean algebra of sets which will also be denoted b y W .
Proof. Obvious. We call bW the sub-algebra of bounded regular functions, and b& the collections of equivalence classes in W and bW. We observe that the family of transient sets is an ideal in the Boolean algebra W and set of W by this ideal.
is the quotient
Proposition 1.3. T h e ma$ A -+ R ( A ) is an isomor$hism of the Boolean algebra 9%'onto the Boolean algebra of equivalence classes of invariant events.
Proof. Plainly R ( A C )= R(A)" and R ( A U B ) = R ( A ) u R ( B ) if A , B are regular; moreover R ( A ) = 0 a s . if and only if A is transient. Finally, if r E 9 , the set A = { x : P S [ r ]> 3) is regular and = R ( A ) , because limn P,,[r] = I, according to the reasoning in ch. 2 $3.
r
Proposition 1.4. A function f in bb i s regular if and only if it i s the uniform limit of simple functions over the Boolean algebra 9. Proof. The sufficiency is obvious. Conversely, let f E b 9 and pick an invariant, simple random variable Y = ailri such that IIY - W(f)ll < E . Let A i be regular sets such that R ( A J = Ti a s . , and set g = w i l A , ; the set A , = - gl > E } is easily seen to be transient. Let h be any bounded simple function such that Ih - < E on A , and set f, = g IA: h l A e ;then f , is a simple and regular function, and - f,l/ < E , which yields the desired result.
2
(If
fl
[If
2
+
I n ch. 2 $3, we have seen how t o establish a one-to-one correspondence between bounded harmonic functions and equivalence classes of a s . bounded
CH. 7, $ 1
REGULAR FUNCTIONS
236
invariant random variables. We may summarize all these results in the following statement.
Theorem 1.6. If (i) the space (ii) the space (iii) the space
P i s markovian, the jollowing three vector spaces are isomorphic: of bounded harmonic functions; of equivalence classes of a.s. bounded invariant functions ; bG.
This theorem has the following corollary. We recall that a positive harmonic function h is said to be extremal if, for every harmonic function h’ such that 0 h’ h, there is a real number c such that h’ = ch.
< <
Corollary 1.6. The extremal bounded harmoiaic functions are in one-to-one correspondence with the atoms of the Boolean algebra &.! We are now going t o exhibit a particular class of regular functions which will be of paramount importance in the following sections. We assume henceforth that P is markovian and that G is proper. There is therefore an increasing sequence of sets E k in 8 with union E , such that the potentials G ( , E k ) are bounded, and if we write L k for the last hitting time of E k (see ch. 1, Exercise 3.18) then L k is as.-finite, L k L k + 1 and lim, L k = a, a s . We define Y k = X L k and %k = a ( Y 1 ,1 2 k ) .
-
<
+
Proposition 1.7. The a-algebra 9 of invariant events i s a.s. equal to the a-algebra nk
*k.
Proof. Let r E 9 and set A from ch. 2 $3 that
=
{ x : P J r ] > S} ; since P is markovian we know
r = lim{X, E A }
as.,
n
and by the afore-mentioned properties of times
r = lim{YkE A }
L k
we also have
as.
k
Conversely, it is easily seen that L k o 8 = L k - 1, hence , the set (0 < L k < a};consequently, if r E S k then
Y k o
8=
Y k
on
MARTIN BOUNDARY
238
CH. 7, $1
If r e f! Zt, it follows from the properties of times L , that which is the desired conclusion.
F ( r )= r a.s.,
Let v be a probability measure on 8;the measure 6 is a-finite and for any probability measure y on 8,we write yG = y K . vG vG1 where y K is a function in 8, and vG1 is singular with respect to vG. We denote by A , a set such that vG(A;) = 0 and vG1(Ar) = 0 ; clearly P , [ u , { X , E A",] = 0. We are going to prove that y K is regular and we begin with some lemmas.
+
> 1, set A,, XLB-n+lE A l , . . ., X L , E A,, n < L , < a}.
Lemma 1.8. Let A , c A,., A l , . . ., A , , B be in d and for n (1 =
{XL,-,
E
Then 'YLAl
=
Ev[yK(XLB-fl)I A ] .
Proof. We have m
P,[Al
=
=
2 PJX,-,E P=n
5j
> 151
yP,-n(dxo) pzO[rl, A0
p=fl
where
A o , .. ., X,E A n n B , X , $ B , q
r = { X 1 eA 1 , . . ., X,E AnnB , X , $ B , q > n}.
Since yG1(Ao) = 0, it follows that P y [ 4
=
Lo
=
y W x 0 ) p z O [ r l vG(&o)
Y G W O ) P z O L ~=l !Ao
51
p=n
vP,-n(dxo)
Ao
y ~ ( x 0 p) z o [ r 1
= Ev[YK(X,,-n)
Let p and v be two probability measures. The Radon-Nilkodym derivative of the restriction of p to the sub-a-algebra Li? with respect to the restriction of v t o g will be denoted (dp/dv)l.
Lemma 1.9. For every R we have
CH. 7, $1
REGULAR FUNCTIONS
237
Proo!. Let A,, A l , . . ., A , be in 8,and let us compute
+
For p > 0, one has Lk+, = p L k + n o O,, hence Y k + , = Yk+,0 0,. on the event {Lk = $}. By an application of Markov property, we get
where
r = {L, = 0, x,E Ao, Y,+l E A l , . . ., Y,+, E A,}.
As in the above lemma, it follows that
since the event
u { X , E A;} has zero Pv-measure, we get the desired result.
Corollary 1.10. One has lim yK( Y,) n
=
(dPy/dPv), P,-as.
Proo!. This is a well-known result in the theory of martingales. We turn to the main result of this section. Theorem 1.11. For every probability measure y on 8,we have lim y K ( X n ) = (dPy/dPv),,P , - a s . n
Proof. Set L
=
L,, where B is any set in 8,and define the random variables
CH. 7, $1
MARTIN BOUNDARY
238
zn by 2,
= yK(XL-,)
on {n
< L < a},
on {I. < n} u { L = a}.
2, = 0
We claim that the sequence (2,) is a supermartingale with respect to the a-algebrasg, = a ( X L - k , k n). Indeed3?n-1 is generated by the sets
<
A
= { X L - n + l E Alp.
.
* p
XL-n+j
E A j},
where A , E 8,and we prove that
Since P , [ u { X , E A;}] = 0, we may take A , c A , for every i, and by Lemma 1.8 we then have E,[Zn I n ] = EdZn 1, = EJZn-1
I ( n < ~ < a r ) I=
1,
<
E,[IA 7 ( n ~ ~ < m ) l
I{n-1<~
E ~ [ ' A 7{n-l,cLim)]
= EJZn-1 Inl.
Now let a, b be two real numbers with a < b, and let (:(L) denote the number of upcrossings of the interval ]A, b[ by {Z,}. I t is known from martingale theory that
which implies that
Ev[&41
< ( b - 4 - l (a + E,[Z,I) ,< ( b - a)-'(a + 1).
= (b
- 4-l ( a
+ P,CL < a3)
The number 6; of upcrossings of ]a, 01 by the sequence { y K ( X , ) } is the increasing limit of the sequence {[;(L,)}.Consequently E,[& < co for every pair (a, b). By the classical argument in martingale theory it follows that lim,yK(X,) exists P,-as., and the limit variable is identified by means of Corollary 1.10.
Remark. Exercise 1.16 hints at the idea which lies beneath the above proof. We may now state
CH. 7, $1
REGULAR FUNCTIONS
Corollary 1.12. If P,
<< P, on 9 for every x E E , then y K
for every probability measure
239
is a regular function
y.
Proof. The event yK(X,)
=
(dP,/dP,),
is plainly an invariant event, and by the previous results Py[Ac]= 0. If P , (< P , on .T, we have P,[Ac] = 0 for every x in E , which is the desired result. The following result gives a useful criterion to decide whether the condition in the above corollary is satisfied or not. We write, according to ch. 1 5 5,
for the Lebesgue decomposition of P , with respect to v. If VP<< v, it is easily checked that PA,,,, Pip:, and therefore that the sequence {Pi( E ) } decreases to a measurable function g. We then have
<
a ,
Theorem 1.13. I f vP << v, then for any x i n E the following two conditions are equivalent: (i) P , << P , on 9 ;(ii) g ( x ) = 0.
Proof. We first show that (ii) implies (i). Let I' be an invariant event such that P y j r ]= 0. The bounded harmonic function P . [ r ] vanishes v-a.e., and therefore for every n ,
It follows that if g ( x ) = 0, then P , << P , on 9. To prove the converse let us denote by An a set in d such that v(An) = 0 and PA(%, (A")")= 0, and set A = An. We have
un
g ( x ) = lim P,[X, E An] n
< P,[lim{X, E A } ] .
The event A = k i( X , E A } is invariant, and since v ( A ) = 0 we have vG(A) = 0; hence PJA] = 0. If P , << P,, it follows that P,[A] = 0, hence that g ( x ) = 0 and thus (i) implies (ii).
Remark 1.14. It is now readily seen that the condition of Corollary 1.12 is
MARTIN BOUNDARY
240
CH. 7, $1
-
satisfied if P(x, ) << Y for every x E E , hence for discrete chains; it is also satisfied for spread-out random walks if Y is equivalent to a Haar measure. We now apply the above results in a way which will be useful in the next with section. We assume henceforth that X is in duality with a chain respect to a measure m.
x
Proposition 1.16. For every pair f, g of functions in dpl+(m),we have lim Gt(xn)/cg(xn)=
(dP,rn/@grn),
n
Prn-a.s.9
and hence P,-a.s. for every v << m. If, in addition, P, << P, on Ifor every x in E , the convergence holds almost surely.
Proof. Let g’ be a strictly positive function in F ( m ) with m(g’) = 1, and let f be any function in B 1 ( m )with m(f) = 1. In Theorem 1.11, we can take v = g’m and y = f m , and then y K = d(fm) G/d(g’m) G. But (g’m)G << m, and cg‘ is a version of the Radon-Nykodim derivative d(g’m) Gldm. By the well-known properties of these derivatives we thus have y K = cf,k?g’ and it follows that
and hence Pm-as., since g’ is strictly positive. Now if g is any function in B:(m), by considering the ratios ( c f / c g ’ ) / ( e g / c g ‘we ) get the general result. The last statement is obtained in the same way as Corollary 1.12.
Exercise 1.16. Assume X to be discrete, and retain the notation of Theorem 1.11. Prove that the sequence of random variables
2, = xL-,
on{n
<m>,
2, = A
(8,)defined by on{L < n > u { L = a )
is a homogeneous Markov chain for the probability measure P, with starting measure P v [ x o= x ] = v G ( x ) gA(x) (cf. ch. 2, Proposition 3.5) and transition probability P defined by the identity
YG(4
P(% Y ) = G Y ) JYY, 4 .
The reader will observe that P and
Exercise 1.17. Let every x in E .
Y
are in duality with respect to YG.
be a probability measure such that P, << P, on ,T for
CH. 7, $2
CONVERGENCE TO THE BOUNDARY
24 1
(1) Prove that the space Y of a s . bounded invariant random variables is a Banach algebra when endowed with the norm
(2) We endow the space b X of bounded harmonic functions with the following multiplication
(h * h’) ( x ) = E,[R(h) R(h‘)]. Prove that bS‘ is a Banach algebra for the supremum norm. Prove that the map h -R(h) is a norm-preserving isomorphism between the Banach algebras b X and 2’.
Exercise 1.18. Let X be a right random walk of law p on a group G . (1) Prove that the results in the preceding exercise are still true even if ,i,iis not spread-out. (2) Prove that the space d of left uniformly continuous, harmonic functions is a closed subalgebra of bA“, hence a C*-algebra. (3) The spectrum of d is denoted by IT, and called the Poisson space of p. I t is a compact and metrizable G-space. There is an isometry i from C(I7,) onto d . If we set ( v , f ) = i(f)(e), then Y is a probability measure on I7, called the Poisson kernel of p and
i(f)(4 =
I
f ( g 4 v(d4,
f E C(fl”)*
“P
The measure v is invariant by p ( p * v = v ) . (4) If p is spread out, then v is absolutely continuous with respect to some quasi-invariant measure 7,Prove that 7 may be chosen such that the integral formula of (3) defines an isomorphism between the Banach spaces b X and Lm(R;, 7). If G = SL(2, R) and T , = G then IT,,is equal to G / K = S1, where K is the compact sub-group of-rotations.
Exercise 1.19. In the setting of Theorem 1.13 prove that g(x) is equal to the norm of the singular part of the restriction of P, t o 9 with respect to the restriction of P,to 9. 2. Convergence to the boundary In this section the following set of hypotheses will be in force: (i) the space E is an LCCB;
242
MARTIN BOUNDARY
CH. 7, $2
(ii) there exists a transition probability P in duality with P with respect to a Radon measure m ; (iii) for any f E C b , the function Pf is continuous; (iv) ef is bounded and continuous whenever f is in CI(; (v) PI = I , in other words I is a harmonic function. Let us observe that if PI = 1, it is enough that Pf E Cb for f E C i to entail that P,g E c, for any g E c b and every n, and in particular property (iii). These hypotheses may always be achieved if E is countable, since in that case we can take m = vG for any probability measure v such that v G ( x ) > 0 for every x in E , and set
R x , Y ) = V G ( Y ) P(Y, X ) / V G ( X ) . They are also in force for transient random walks on groups.
Definition 2.1. A reference function is a positive m-integrable function r such that e r is bounded, continuous and strictly positive.
By condition (iv) above, there are plenty of reference functions, but we shall prove a more precise result. We recall that a Bore1 function f is said to be locally integrable if, for every compact K , we have ( m , I f / IK) < CQ. If E is countable, every finite function is locally integrable. For a spread-out random walk all m-a.e. finite superharmonic functions are locally integrable. (See Exercise 2.13.) Lemma 2.2. For every locally integrable superharmonic function f , there is a reference function Y such that m(fr) < to. Proof. Let { y n } be an increasing sequence in C i with limit I at every point in E. Then let C, be the maximum among the numbers m(p,), m(p, f ) , IlC~nIl; it is easily checked that m
r =
2 c;'
v,,
2-n
c= 1
is the required reference function.
Remark. The particular reference function just constructed is continuous '
and strictly positive. All rm-integrable functions are then locally integrable.
Proposition 2.3. I f r is a reference function, then for every f E C i , the function l?f = d f / d r is bounded and continuous. Fuythermore if {fn}n>lis dense in C K
CH. 7, $2
CONVERGENCETOTHEBOUNDARY
243
for the topology of uniform convergence on compacta then (l?fn}n>,l i s dense in (21,f E C,) for the topology of uniform convergence.
Proof. That Rf is continuous is clear. Since we have Gr > 0, there exists a constant a > 0 such that e r 2 a on { f > 0). Consequently
Gf
< a-ll@ll
Gr on {f
> 01,
and hence everywhere, by the complete maximum principle. Let now f be in C, and { f r } a subsequence of {f,} converging uniformly to f and such that the sets { f k > 0) are contained in a fixed compact set K . Since
IlW -
fk)ll
d Ilf - f k l l Gc * ) K ) ,
for any E > 0 we may find k such that /1G(f - f k ) l [ < E . The latter half of the statement is then easily proved by the same argument as the former half. We shall denote by R(x,* ) the kernel e ( x , * )/Gr(x).The measures &(x, * ) are co-excessive Radon measures such that (I?(%, ), v ) = 1, and it follows from the above proposition that the set {R(x, ), x E E } is vaguely relatively compact. We are now ready to construct the Martin boundary of E. We first choose a metric 6 on E , compatible with the topology of E,and such that the Cauchy completion of (E,6) is obtained by adjoining the Alexandrov's point A . We next choose a reference function Y such that m(r) = I, and a sequence {fn}n>l dense in CK.For x , y in E , we set
-
m
d ( x , Y) =
2 wn(R n= 1
- R fn(y)I,
where the positive real numbers W , are chosen such that Then 6 d is a metric on E.
+
21"W,ll&f,ll
< co.
Definition 2.4. We define E* to be the Cauchy completion of the metric space (E,6 d ) and set M = E*\E. The set E* is called the Martin space for X started with distribution rm, the set M the Martin exit boundary for X started 'i, with distribution rm. The pseudo-metric d generates the uniformity relative to which all the functions Rf, f E C,, are uniformly continuous; consequently the space E* depends neither on the choice of { f , } nor on the choice of {W,,}, and each function Rf may be extended continuously to E*;the extended function will
+
MARTIN BOUNDARY
244
CH. 7, 52
still be denoted by ftf. Let x be a point in M ; a sequence {x,) of points in E converges to x if and only if x, -.A in the original topology of E and ft f(x,) -+ R f ( x ) for every f E CK.
Proposition 2.6. The space E* is compact; E is a dense open subset of E* and its relative topology coincides with the original topology.
Proof. By the choice of 8, and since the functions I?f, are bounded, from every sequence of points in E* one may, by use of Cantor's diagonal process, extract a subsequence { x k } which is Cauchy for 6 and such that {I? f n ( x k ) ) is Cauchy in R for every 72. The subsequence { x k ) is thus Cauchy in (E*, 6 + d ) , hence it is convergent, and E* is a compact metric space. That E is dense in E* follows from the definition of E * ; furthermore, since d 6 >/ 6, the original topology is coarser than the relative topology, and E is open in E*. On the other hand, the functions Rf being continuous, a sequence of points converging to a point in E in the original topology converges in the relative topology, which proves that the latter is finer than the former. The proof is thus complete.
+
Remark. The space E* may depend on the choice of the reference function r ; in other words, it may happen that the spaces E* obtained 'for two different functions are not homeomorphic. It is now plain that Borel sets in E are Borel sets in E*. Furthermore, for every point x in M the map f -+ ft f ( x ) , f E CK,is clearly a Radon measure on E which will be denoted by k(x, ). The extended function k is a kernel on E*, the measures R(x,* ) vanishing outside E. Finally, if {x,} is a sequence of points in E converging to a point x in M ,the measure k(x,* ) is the limit of the measures &(xn, ) in the vague topology on E. This entails that li'r(x) 1 for every x in E*, and also
-
<
Proposition 2.6. The measures k(x,* ), x
E
M , are co-excessive.
Proof. For f E C i , pf is continuous; hence if (p,,} is a sequence of Radon measures converging vaguely to a measure p, then (
~
% G lim(pnt PI). n t
The proposition then follows from the previous discussions.
CH. 7, 52
246
CONVERGENCE TO THE BOUNDARY
As a result, if u is a probability measure on the Bore1 sets of M , the measure
aft is a co-excessive Radon measure and (al?, r ) < 1. This will be of significance in the forthcoming results, which are the main results of this section. . We shall call A the set of trajectories o such that the sequence { X , ( w ) ) has a limit in E* as n co.Since E* is compact and metrizable, it is easily seen that A is measurable, and moreover, it is an invariant event. Let us pick an arbitrary point x in M and set -+
X,(o) = lim X,(w) if o E A ,
X,(w)
=x
if w E A‘.
?I
We thus define an invariant random variable X,, and in the sequel the statement {X,) converges to X, P,-as. will mean that P J A ] = 1. We are now ready to state
Theorem 2.7. The sequence {X,} converges P,-a.s. to X , and the image u of Pr, by X , is carried by M . Furthermore m = u K ;in other words, for every f E C K
m ( f )=
1
.M
K f ( s )dub).
Finally if P , << P, on 9 for every x in E , the above convergence holds almost sure1y .
Proof. Since the potentials of compact sets are bounded it follows that {X,} converges almost surely to d in the topology of E , hence for the metric 6. The convergence in E* P,-a.s. is then a consequence of Proposition 1.16 and of the definition of E*. The last sentence of the statement is shown in exactly the same way. The equality m = u K is shown by the following set of equations, where f ranges through C$ :
Finally a vanishes on E because X, E M P,,-a.s. The preceding theorem may be generalized as follows. In the remainder of this chapter, we shall use “harmonic function” to mean “positive harmonic function”, but at variance with the convention of ch. 2, we will allow these functions to take infinite values on a negligible set. We denote by .iPTthe set of harmonic functions h such that m(rh) 1. For such a function we set Eh = {0 < h < a}.We define a new T.P. on (Eh,Eh n 8) by setting for X E Eh and A E Eh n 8,
<
246
MARTIN BOUNDARY
Ph(x,A ) = h(x)-'
CH. 7, $2
P ( x ,dy) h(y). {A
Plainly, (Ph))" = Ih-lPnI),, and a function f is Ph-(super) harmonic if and only if hf is &(super) harmonic. In particular 7 is Ph-harmonic. It is also easily seen that Ph is in duality with $' with respect to the measure Itm. The functions Cf/& are therefore the same for P and Ph, and the preceding theorem will be applied to the chains with transition probability Ph. Such a chain may be constructed bn the space (L?,F)of the chain X . For any positive measure Y on Eh, we call P: the measure on (52,s for ) which {X,} is a homogeneous Markov chain with transition probability Ph and starting measure hv. For any finite collection A,, 0 i n, of sets in 6, one has P ~ [ X " € A , , X , € A ,, . . . , X ' , € A , ]
< <
In this formula, E,, is the usual expectation associated with the probability measure Pzo;in other words, if Iz = 1, then Ph = P and P: = P,. I t is clear that P![X, 4 Eh] = 0 for every integer n. Finally, if h and k are two harmonic functions, then P!+' = P! Pk
+
V.
Theoreni 2.8. Let h be a locally integrable function in #,.. The sequence {X,} converges Pi-a.s. to X,; the image ahof P:mb y X, is carried by M and hm = ahK. Finally if Pt << PL on § for every x i n E , the convergence holds Pk,-a.s. for every x in Eh.
Proof. We cannot immediately apply the preceding theorem, because Eh is no longer LCCB. By Proposition 1.15 we still have the convergence PFm-a.s.of the sequences I? f(X,). I t remains therefore to prove that X, converges P:,,,-a.s. to d in E. Let I< be any compact subset of E ; calling Gh the potential kernel of Ph, it is easily seen that h Gh( * , K ) = G(h IK), and therefore, since h is locally integrable, (hy, G h l K ) , = (r, G(h IK)),,, = (CY,h IK),,, < CO. As a result, P,h,[R(K)] = 0 and ( X , } converges Pfm-a.s. to A . The first part of the statement then follows from the definitions of E* and M ; the equality hm = a h K is derived as was the equality m = OK of Theorem 2.7.
CONVERGENCE TO THE BOUNDARY
CH. 7, 52
247
Remarks. (1) The measure a of Theorem 2.7 thus appears as a special case of ah obtained when h = I , . (2) The map h + a h is an affine map from the cone of locally integrable functions in X , into the space of bounded positive measures on M , and if h k rm-a.e., then ah ak.
<
<
We conclude this section by giving the probabilistic version of the classical Fatou’s theorem in the unit disc and some of its applications.
Theorem 2.9. If Gg is a finite potential, then limn Gg(X,) = 0 a s . I f h i s a locally integrable harmonic function in Zr,and if ah = ua a,“ is the Lebesgue decomposition of ah with respect to a, then
+
P,-as.
lim h(X,) = u(X,) n
This convergence holds a s . if P,
<< P ,
on 9 for every x in E .
Proof. Since Gg(X,) is a positive supermartingale, the limit exists a s . by martingale theory. Now by Fatou’s lemma, for every x in E , E,[lim Gg(X,)]
< lim E,[Gg(X,)]
=
lim P,Gg(x)
=
0;
the proof of the first part of the statement is thus complete. For the second part, let us observe that, by the definition of ah, we have for any f in Ck,
.-
Seth’ = E.[u(X,)] ;the function h‘ is harmonic andsince h is locally integrable, u(X,) is P,,-integrable and consequently h’(X,) converges Prm-as. and in L1(P,,) to u(X,). Since this is true for every f E C i , we get lini h’(X,)
=
u(X,)
P,-a.s.
Furthermore
-
m(fh’) = m ( f P,h’)
=
Etm[h’(X,)],
and because of the Lkonvergence above, we may pass to the limit to get
CH. 7, 92
MARTIN BOUNDARY
248
1
=
l?f.uda;
M
since the functions K f , f E C$, separate M we thus obtain that ah' = W . As a result h' \< h m-a.e. Now the function h' A his superharqonic ;let Gg 4-h" be its Riesz decomposition. Since m{h' > h} = 0, it is easily derived that lim(h'
A
h) ( X , )
=
u(X,)
P,-a.s.,
n
and by the first part of the theorem lim, Gg(X,) lim h"(X,)
=
u(X,)
=
0; consequently
P,-as.
n
<
+
Since h" h everywhere, we may write h = 12" k , where k is harmonic, ,, and from the relation P:, = P:,,, P:,,,, we derive that ak = at. To complete the proof of the theorem it suffices to show that if ah and c are mutually singular, then {h(X,)} converges to zero P,-a.s. Let h be such that a and ah are mutually singular, and call h' the harmonic part in the Riesz decomposition of h A 1 . Since h' \< h and h' I , we have a h ' \< ahand ah' a, which entails that oh' = 0, hence h' = 0 m-a.e. It follows by a simple application of Fatou's lemma that
+
<
<
lim h'(X,)
=
0 P,-a.e.,
n
and by the first part of the theorem, lim(h A I ) (X,) = 0 P,-a.e. n
As a result, lim, h(X,)
=
0, and the proof is complete.
Corollary 2.10. I f r is continuous and strictly positive, the map u positive isometry from L'(M, a) onto L1(9,9, Prm).
-.,u(X,)
is a
CH. I. 32
CONVERGENCE TO THE BOUNDARY
Proof. Clearly, if u is in L l ( M , u), then .(X,)
249
is in L'(Q, 9, PTm) and
Now let Z be in L:(IR, 9, PTm); the harmonic function h = E.[Z] is in L1(rvt), hence is locally integrable. and therefore there exists an element u of Ll(M, u) such that ah = ZM ut. By the previous results, we know that h(X,) converges to u(X,) P,,-a.s., and on the other hand
+
which by the martingale theorem converges to 2 P,,-as. 2 = u(X,) P,,-as., and the proof is complete.
It follows that
The following result is important. I t asserts that the Banach space of bounded harmonic functions is isomorphic to the space L" of a probability measure defined on a Borel subset of a compact metrizable space. This provides a generalization of what is known for spread-out random walks on groups. (See Exercise 1.18.)
Theorem 2.11. If P, << P , on $ for every x in E and, if r is contiwous and strictly positive, the Baizach space b X of bounded harmonic functions i s isomorphic to th.e Banach space Lm(Af,a).
Proof. I t was seen in ch. 2 53 that the vector space b X is isomorphic to the vector space of a s . bounded invariant random variables. Since P, << P,, on $ for every x in E , the equivalence a s for invariant random variables is the same as equivalence P,,-as As a result, b X is isomorphic to L m ( Q , Y , PTm) m d e r the mapping h+R(h), whose inverse mapping is Z + E . [ Z ] . If 12 .1' < c P,,-as., it follows that 121 < c P,-as. for every x in E , and clearly these two mappings are norm-decreasing ; they are therefore Banach isomorphisms. I t remains to prove that the Banach space Lm(SL, 9, Prm)is isomorphic to L m ( M ,a). This is done by using the map u + u(S,) in the same way as in the above corollary.
Rcmark. If we compare this result with Theorem 1.5, we see that there is a one-to-one correspondence between the Borel subsets of M , up to uequivalence, and the equivalence classes of regular sets.
MARTIN BOUNDARY
250
CH. 7, $2
We close this section with a few comments on what can be said if 1 is no longer a harmonic function. In that case, the chain may “disappear” a t a finite time and is then placed in the cemetery A , which we shall consider as a n isolated point. If the other hypotheses of this section are in force we may construct E* and M as before and prove Theorem 2.8; the measure ahwill be carried by M . The boundary may be viewed as the union of ill and the isolatedpointd,andtheimageofP,,byX,willbeequaltoP,,[~ < c o ] +ah, ~ ~ where h is the harmonic part of the Riesz decomposition of I.
Exercise 2.12. With the notation of Theorem 2.9, prove that the following two conditions are equivalent : (i) h(X,) converges in L1(P,,) ; (ii) the measure at vanishes. These two conditions are satisfied if h is bounded. Exercise 2.13. Prove that, if X is a spread-out random walk, every a.e. finite superharmonic function is locally integrable. [Hint: Choose n such that pn > cm on an open set V . For any x in G , there is a point y such that f ( y ) < co and x E yV. Prove that f dm < GO.]
syv
Exercise 2.14. Let X be a right random walk of law ,u on a group G . (1) Prove that the following two qonditions are’equivalent: (i) there exist reference functions in C ; ; (ii) there exists a compact set K such that G = K T,. (2) Assume henceforth that the conditions of (1) are satisfied. Prove that the space E* obtained for different reference functions in C; are homeomorphic G-spaces. Prove that in Theorem 2.7 the convergence holds a.s. even if X is not spread out. (3) Let Y be a probability measure on G equivalent to nz. Call a”the image of P , by X , and N the support of a,. Prove that N does not depend on v and is a compact G-space. (4) From now onwards X is assumed to be spread out. Prove that there is an isomorphism h -+ h between the Banach spaces b 2 of bounded harmonic functions and L*(N, a,) such that
h(X,)
=
lirn h(X,)
a.s.
and h(g) = (gy, h),
where y is the image of P, by X,. Compare with Exercise 1.18. (5) Assume further that ,u << m, and prove that there exists a function k on G x N such that k ( - ,x ) is invariant for a,-almost every x in N , and if
CH. 7, $3
251
HARMONIC FUNCTIONS
Exercise 2.16. (1) Prove that for any pair of pc)ints x , y in E , the equality I?@, * ) = K ( y , * ) holds if and only if x = y. (2) Prove that there exists a unique (up t a homeomorphism) compact metrizable space F such that: (i) E is a dense subset of F and the relative topology of E is coarser than its original topology; (ii) the functions Kf, f E C,, extend to continuous functions on F which separate F . (3) State and prove a theorem of convergence to the boundary in F . Exercise 2.16. Denote by (a, b), a > 0, b real, the generic point in the group of positive affine motions of the real line. Let X be a right random walk of law p and suppose that
(1) Prove that {X,}converges P,-a.s. to (0.2)where Z is a real random variable. The group being identified with the part {u > 0) of the plane, the line { a = 0) may obviously be seen as a boundary and the foregoing is thus a theorem of convergence to the boundary. [Hint :One may use the fact that if { U,} is a sequence of positive, independent and equally distributed random variables such that E[log(l U,)] < 00, then lim U;'" = 1 P,-as.] (2) Prove further that if v is the law of Z, then ,LA * v = v and deduce therefrom that there are non-constant bounded continuous p-harmonic functions. [Hint: For the last point, see ch. 2, Exercise 4.18.1
+
Exercise 2.17. Suppose that Gg is a strictly positive potential and that m(rGg) 1 . Define PGR as Ph was defined in the text and prove a theorem of convergence similar to Theorem 2.3. What will be the distribution of X,?
<
3. Integral representation of harmonic fiinetions
In the preceding section, we have associated with each positive locally integrable harmonic function h of Z r a measure oh on the boundary. As a
CH. 7, 93
MARTLN BOTJNDdRY
252
result, the co-invariant Radon measurn Am had an integral representation as the image of On by the kernel Conversely, if 17 is a bounded measure on the boundary, q K is a co-excessive Radon measure such that (qK,Y ) < co. We wish to go deeper into this to get an integral representation of all coexcessive Radon measures-subject t o some integrability condition as above -and study the question of the uniqueness of such a representation; then we want t o turn this into an integral representation of harmonic functions recalling the classical Poisson representation in the unit disk. In what follows I is a fid reference function. We call 9, the set of coexcessive measures lj such that E[Y) 1. We will have t o add another hypothesis to the set of conditions in $2, namely
a.
<
-
-
Condition 3.1. For every x in E, the measures P(x. ) and P(x, ) are absolutely continuous with respect t o m. This condition is obviously in force for. discrete chains and for random walks with an absolutely continuous law. It entails the following proposition for which we need some further notation: we denote by M , the set of points s in M such that K ( s , ) is co-invariant. If M , is a proper subset of M we will see below that it is a Bore1 subset of M .
-
Proposition 3.2. I f 6 is co-invariant then f << m and there is a unique harmonic function h sigh that 6 = hm. Moreover there is a unique function k on E x M , such that (i) k is b @ b*-measurable; (ii) for every s EM,, the function k( * , s) is in Srand R(s, ) = k( ,s ) m ; (iii) for any h E 3Ear,
-
1 r
h=
-
k ( * , S) ah(ds).
M
Proof. That [ << m is obvious from Condition 3.1. Furthermore if g is any function within the class of dEldm, it is also clear by duality that Pg = g m-a.e. and that Pg does not depend on g. If we set h = Pg,then h is harmonic and 5 = hm; the uniqueness of h follows also from Condition 3.1.Finally if [ is in Y,, then h is in S ras is seen by duality. To prove the second part of the statement, we observe that by Proposition 5.3 in ch. 1, we can choose g( , s) in the equivalence class of dR(s, )/dm in such a way that the resulting bivariate function is b @ 8* measurable.
-
-
-
Then using a standard argament, R( ,s) = Pg( ,s) is also s&n tw be in C @I &'* and is the desired fun&-on. The third p& follows tisen easily by Thiesrarm 2.8. w e may now turn to the study of the set YTand state
Proof. The set YTis obviously convex; it is proved to be compact by the same reasoning as in Proposition 2.6. Let now 6 be in 9,; by the Riesz decomposition the ore^ in ch. 2 fj4, = & + R where 2 is ceinvariant. By Proposition 3.2, there is therefore a harmonic function h. in XTsnch that E =q~+hmThemeasure&-~+&isthedeskdmasmea. Counterexamples (see Exercise 3.17) show that the representation just obtained is not unique. We shall now look for a B o d subset M eof M such that we get uniqueness provided that we restrict the measures u to those carried by E u M,. Clearly we must exclude the set M\M,; indeed if R(s, ) = hm with r) # 0, then x ( s , ) mrresponds to both E, and C r - q +oh.
-
+
-
Proposition 3.4. The set M,, is a Bore1 slrbset of M and ah(M\M,) every k E Zr.
= 0 for
Proof. Let {f n ) be a countable dense subset of C i (E);the set M\MI is the union of the sets (s: Kf,,(s) > KPf,(s))which are clearIy B o d subsets of M . The second half of the statement is obvious. Before we proceed, we must introduce the following termjnobgy. Definition 3.6. We call extremal1Izeaswe.s the extremal points of the compact convex set Y,..
The zero measure is extremal and a non-zero measure 6 is extremal if, there is a real t E LO, 13 such that whenever = El + E2 with El and E2 in YT,
MARTIN BOUNDARY
264
CH. 7, $3
& = t[ and f z = (1 - t ) 6.This is equivalent to saying that if 6‘ E 9, and 5 - 5‘ E 9,, then there is a real t E [0, 11 such that 6‘= t[. I n particular if 6 is co-invariant and extremal and if 5‘ is co-invariant and E’ < 6,then 5‘ = t [ with t E [0, 11. Finally we observe that if [ is non-zero and extremal then [ ( r ) = 1. In the sequel we shall use the term “extremal” to mean
“extrenaal and non-zero”. From the Riesz decomposition theorem we deduce that an extremal measure is either a co-invariant or a co-potential measure. Moreover we have
Lemma 3.6. A co-poteratiul masure is extremal if and only if it i s equal to R ( y , ) with y in E.
-
Proof. The sufficiency is easy to prove. Let conversely be in 9, and such that K ( y , - 5 be in gPr. The co-invariant part of [ being bounded above by the co-potential I?(y, ), vanishes identically and 6 = r$ for a positive measure q. I t follows that R ( y , ) - f is a co-potential, hence E, - 9 a positive measure which implies that q is a multiple of E,.
-
-
-
If there is a points in Mh such that k(s, ) is notextremal, then the uniqueness property that we are looking for cannot hold. Indeed, we would have K(s, ) = Ez with # &(s, ) and R(s,* ) would correspond to E, as well as to crl c2 # E,, where ui is associated with tiby Theorem 3.3. We shall therefore try to restrict the boundary still further, namely to the points s for which R(s,* ) is co-invariant extremal. For s E M h t we can, following $2, associate with the harmonic function k ( , s ) the probability measures and the measure uk(.*s)which will be denoted for short by P: and wq.We will also write E8 for Ek(.ss).
+
-
+
-
P;(.ls)
Lemma 3.7. For any A E S , the map s + Ps[A] i s &*-measurable and for h E sr,
P f [ A ]=
L
P”,A] oA(ds).
For any Bore1 subset A of M , the map s --+ as(A)i s &*-measurable and &(A)
=
1
M
Proof. By $2, if A
E .Fn, we
have,
# ( A ) crh(ds).
CH. 7, $3
266
HARMONIC FUNCTIONS
PJA] = Ev[7,4 k(Xm s)]
;
the result is thus clear for A E S,,, and the proof is compieted by means of the monotone clasq theorem. If now x is in Eh and A still in F,, we, have for oh almost every s,
EJ1, k(Xn, s)]
=
J'",A] ;
this is true if x E E3 by definition: of P i ; if k ( x , s) = 0, then since k ( X , , s) is a supermartingale it vanishes P,-a.s. and finally uh(s:k(x, s) = m} = 0 since x is in Eh. We can thus write
For v carried by Eh we get the announced result by integrating with respect to v, then by using the monotone class theorem. The second half of the statement is an easy consequence of the first and of the definitions of d and oh as the images of Pi,,, and P,,,by , X,.
Proposition 3.8. For
s E Mh,
d = E, and then P",,[X,
the measwre K ( s ,* ) is extremal if and only if
= s] = 1.
Proof. Let us write for short h instead of k ( . , s) and suppose that g is a non-constant, bounded, Ph-harmonic function. We may suppose that llgll < 1 and by Condition 3.1 that m({g < 1)) > 0. The measure (gh) m is coinvariant and strictly less than hm. Indeed, remembering that P h and p are in duality with respect to hm we have, for any f E b,,
-
Consequently, if k ( s ; ) is extremal there are no non-constant bounded harmonic functions, and the a-algebra Y is P,,,,-a.s.trivial. The random variable X , is therefore Pf,,,-a.s.constant, hence ah is the unit mass at some point in M . As a result of the definition of the topology of E* and of Theorem 2.8 this unit mass must be E, and P:,[X, = s] = 1. Conversely, if k ( s , * ) = t1 t2where & is co-invariant and not propor-
+
MARTIN BOUNDARY
266
CH. 7, 93
-
tional t o K(s, ), then ti = h,m where hi is not proportional to k( , s) and therefore us = d1 6.cannot be equal to E ~ .
+
As a consequence we get
-
Proposition 3.9. The set M eof points in M , such that K ( s , ) i s extremal is a Borel subset 01 M , and for any It E Z rwe have u h ( w )= 0.
Proof. By Proposition 3.8, the set M e is the set of points s in M such that us = eS. Let (B,),ao be a countable algebra of sets generating the a-algebra of Borel subsets of M. Then by the monotone class theorem,
and by Lemma 3.7, M , is a Bore1 set. Next, from the formula uh(A)= SuS(A)duh(s) of Lemma 3.7, it follows that
us(A) = IA(s) uh-a.s. for every Borel subset A of M. Consequentiy
ad(B,) = 7,4s)
uh-a.s.
for every n 2 0 ; hence by the monotone class theorem, the set of points s for which crR # E~ has uh-measure zero. We may now state the main result of this section, which is the uniqueness result that we were looking for.
+ -
Theorem 3.10. Let 6 = qe h m be a co-excessive measure in 9,; the measure u = & r ] uh i s the unique measure carried by the set E u M , such that
- +
5=
-
K ( x . ) dfJ(x).
Proof. By the previous discussions, it suffices to show that if 0 is a probability measure on M,and h m = Jar, B(s, ) q(&) then uh = 0. Let A be a Borel subset of M, by the same argument as in Lemma3.7 weget
-
-
CH. 7, 93
HARMONIC FUNCTIONS
but since for a-almost all s, P:,[X, and therefore ah = 8.
EA] =
l A ( s ) ,we have P;,[X,
257
EA] =
8(A)
From this uniqueness property and the fact that X , is in M , P:-a.s. for every h E Z n and x in E , we see that the set M e is the useful part of the boundary of E . Furthermore, it is often easier to describe the set M e , which is isomorphic to a set of extremal points of a compact convex set, than the whole set M . In applications, the set M will therefore be forgotten, and the enlarged state space will be E u M e . I t may happen that it is no longer a compact space, but that causes no inconvenience in general. Using Proposition 3.2 the above representation theorem may be turned into a representation theorem for harmonic functions.
Corollary 3.11. For every harmonic fuitction h in Xrthere i s one and only one measure uh carried b y M e such that h=
I
k( * , s) ah(ds), c
and uh i s the image of Pk b y X,. Conversely, for a n y probability measure u on M e the integral above defines a function of &‘r and this function i s extremal if and only if it is equal to k ( , s) for some s in M e .
-
This corollary allows us to return to the exit distributions of the chain, namely the distributions of X, under the different probability measures P,.
Proposition 3.12. For a n y harmonic function h in Xr,a n y x in Eh, and a n y Bore1 subset A of M e ,
Pt[X, E A ] =
I
k ( x , S) uh(ds).
A
I n particular, if u i s the measure on M e of the reflresentation of the harmonic function 1 , then for every x E E ,
MARTIN BOUNDAKY
268
CH. 7, $3
Proof. Let us start with the case where h. = k ( * , s) for s E M e . We know from Proposition 3.8 that y = P”X, E A ] = In(s) rm-a.e. Moreover y / k ( , s) is P*(.#s)-harmonic and since this T.P. is easily seen to be absolutely continuous with respect to m, we have the equality everywhere. Consequently the proposition is true when h = k( * , s) and the general result follows from Lemma 3.7 and its proof.
-
It is finally probably worth recording that in the case of discrete chains the function k which appears in the above results may be computed without any reference to a dual chain. From now on we assume that X is discrete. Let p be a probability measure such that the potential pG is bounded and strictly positive on E. By setting wz = pG
and P(x,y )
=
P ( y , x ) nz(y)/m(x)
we get a chain X in duality with X with respect to m and we have the hypothesis of this and the last sections. If we set ~ ( x = ) p(x)/pG(x) it is easily checked that & = 1 and thus r may be used as a reference function. Furthermore a function is integrable with respect to rm if and only it it is integrable with respect to p and all finite functions are locally integrable. We may define the boundary M and thc space E* by using this particular reference function r and for s E M e , x E E , we get
k ( x , 4 = k(s,x)lpG(x).
We can extend k t o B x b’* by putting k(x, S ) = G ( x ,s)/pG(x), x E E , s E E*. I t is easily seen that the uniformity relative to which the functions Kf,f E C, are uniformly continuous is the same as the uniformity relative to which the functions k ( x , * ) are uniformly continuous. As a result, the space E* could have been defined by using the functions k ( x , * ) instead of the functions l?f. A sequence {s,} of points in E converges to a point s of E if and only if either s is in E and s, = s for n sufficiently large, or s is in M and k ( x , s,) k(x, i s) for every x in E. I t thus turns out that the ancillary dual chain used above disappears completely and may therefore be forgotten.
Exercise 3.13. Let X be discrete and retain the notation a t the end of the section. (1) Prove that the set 9 p of superharmonic functions f such that p ( f ) 1
<
CH. 7, 93
HARMONIC FUNCTIONS
269
is a compact convex set for the topology of pointwise convergence. (2) Prove that the extremal elements of 9 ' p are either harmonic functions or potentials and that a potential is extremal if and only if it has the form G( * , y ) for some y in E. (3)Prove that the extremal harmonic functions are the functions k( , s) for s E M e .
-
Exercise 3.14. With the hypothesis of this section show that
Exercise 3.16 (fine topology). (1) If s E M e and A is a Borel subset of E* the probability P L [ b ( X nE A } ] is either 0 or 1. (2) One can define a topology on E in the following way: if s 4 M , then {s} is open; if s E M e , the collection of sets A which are closed for the 6 d metric on E* and such that the probability in (1) is equal to 1, is a basis of neighbourhoods of s. Prove that this topology is finer than the 6 d-topology.
+
+
Exerciso 3.17. Let X be the random walk of ch. 3, Exercise 3.13(2). (1) Notice that, with the notation of the end of this section, we may take p = e0 and then all finite harmonic functions are p-integrable. Compute k and prove that E* is obtained by adjoining two points to the space E = Z. Prove that both points correspond to a non-zero extremal function; compute these functions. Give all the harmonic functions and compare with ch. 5, Exercise 1.9. (2) Take now p ( x ) = 0 for x < 0 and p ( x ) = (P/q)"( ( p - q)/$) for x 2 0. Do the same work as in (1). One of the points of the boundary does not correspond to an extremal function, thus M e is different from M in this case. (3) More generally, if X is any random walk on Z and p any probability measure on Z, there is at most one p-integrable extremal function beside the function 7. Exercise 3.18. Let E be the set of lattice points (n,i ) in the plane with i n, and define P by
0
< <
260
MARTIN BOUNDARY
CH. 7, $3
A point in E is thus identified with i heads in n tosses of a fair coin. (1) Observe that the unit mass at (0,0) is a reference measure, and compute the corresponding kernel K . (2) Prove that a sequence (n,, jk) is Cauchy in the corresponding space E* if and only if there is a number s E [O, 11 such that lim, i k / n k = s. Prove that M is homeomorphic to [0, 11 endowed with its usual topology and that for s E M , K((m,i ) ,s) = 2msy1 - s)m--i. Prove that M , = M . (3) What is the measure u corresponding to the harmonic function l ? Describe h and P h for the case in which u h is the Lebesgue measure on M, and that in which uh = E , for s E M .
Exercise 3.19. Let E be the set of all finite sequences of letters a and b. The empty sequence is taken as the starting state. A transition probability on E is defined in the following way: if x is a finite sequence, and we denote by xa and xb the sequences obtained by adjoining a or b on the right of x, then
P ( x ,X U ) = P ( x ,xb) = 4. (1) Prove that the boundary M is the set of all infinite sequences of letters a and b. Find the topology of M . (2) Compute the measure u corresponding to the harmonic function 1.
CHAPTER 8
POTENTIAL THEORY FOR HARRIS CHAINS
In this chapter, we proceed with the study of a potential theory suitable for Harris chains, which was initiated in ch. 6. Special functions will still play an important role, but we shall consider only the bounded ones. Thus, in all statements of this chapter, “special” must be understood as “special and bounded”.
1. Harris chains and duality As was seen in ch. 4 94, most Harris chains, and actually all the usual ones, have a dual chain which was shown to be equally Harris u p to modification. We shall give below a proof of the latter result that does not rely on the notion of essential irreducibility. Afterwards we shall study the relationship between special and co-special (special for the dual chain) functions.
Theorem 1.1. If P i s Harris and in duality with P , there i s a modification of P which i s Harris. Furthermore if h E a,, h > 0 and uh > I @ m, this modification m a y be chosen so that o h ( h )= 1 and oh > I @ m.
Prool. As m is the only invariant measure for P , the duality between P and P holds with respect to m.For f , g E B,, (ohfrg) = ( f , uhg)
> m ( f )m(g),
which implies that, for all f E bB,, ohf > m(f)m-a.e. By letting f run through a denumerable algebra of sets generating 8 we get that there is an m-null set N , such that * ) > m for x outside N , . Moreover, for every g E bb,,
(uh(h),g) = (h, u h k ) )
=
(hm)crh(g) = m(g)
according t o ch. 3, Proposition 2.9. Therefore o,(h) = 7 outside an m-null set N,. 261
POTENTIAL THEORY
262
CH. 8, §l
Since m is P-excessive, one can find, as in ch. 3, Corollary 2.12, an m-null set N containing N 1 U N , and such that N" is an absorbing set. We then pick a point xo in N c and set
Pyx, * )
=
P(x, * )
P'(x,* )
=
P ( x o , * ) for x E N ;
for x E NC,
the T.P. P' is a modification of P for which the conditions o,(h) = 1 and 0, > 1 m hold. It is then clear from ch. 3, Proposition 2.7 that P I is Harris, and the proof is complete. In the following, we do assume that there is at least one T.P. P in duality with P. On account of ch. 6, Corollary 4.6, the function h in the preceding result is both special and co-special. I t is natural to ask whether every special function is, as a rule, also co-special (up to a modification of P ) .On account of ch. 6, Proposition 4.9,this would imply that, for every quasi-compact T.P., there is at least one quasi-compact modification of P . This turns out to be false, as is shown in Exercise 1.9, and we shall below describe the set of functions both special and co-special. For this purpose, we need the following classical
Lemma 1.2. Let {anlnEN be a bounded sequence of real numbers converging to zero in the sense of Cesaro and set f ( s ) = 2; ansn (0 s < 1 ) ; then
<
lim (1 - s) f ( s )
=
0.
S-Pl
Proof. The sequence U, = n-lC;Iak converges to zero, and an easy computation yields W
(1 - s) f ( s ) = (1
For
E
> 0, there is an integer N
- s)2
S" 2o un--. n
such that lU,1
<
E
€or n 2 N , hence
Now, the first term on the right of this inequality can be made arbitrarily small by choosing s near one, and the second term is less than E , which completes the proof.
CH. 8, 51
HARRIS CHAINS AND DUALITY
203
Before we proceed, let us remark that if f is special for X and g special for 8, then f A g is special and co-special.
Proposition 1.3. A function h in bb+ i s special and co-special if and only if there is a real number c such that u,h
>I
@m
and
och
>I
@ m.
Proof. The “if” part is obvious. To prove the converse, we start with P and P quasi-compact. Since the constant functions are special there is, by virtue of ch. 6, Lemma 4.7, a function f > 0 and a constant a < 1 such that U , > I @ fm. For a’ < a, we have U,. > (a - a’) U,U,,, and therefore, for all g E b&‘+ and x E E ,
P is quasi-compact, there is, by < a. implies a’o,.f k > 0; thus
Since a’
Lemma 1.2, a number a, such that
This proves that, for a’ sufficiently small, U,, > I @ m. Symmetrically, one can find a’’ such that l?,, > 1 @ m. Then, taking c = a‘ A a‘’, we get the desired result for constant functions. Let us now shift to the general case and suppose h > 0. The T.P.’s Q = U h I , and Q = O h I h are quasi-compact and in duality with respect to the measure hm. Since U: = U c h I h , the first part of the proof implies that for c sufficiently small, u c h I h > 7 @ hm and o c h l h > 1 @ hm; hence Ueh > I m and Och> I m. Now, any special and co-special f u n d o n h is majorized by a strictly positive special function k and by a strictly positive co-special function &. The function k A fi is strictly positive, special and co-special, and majorizes 12. Since U h increases when h decreases, the proof is complete. The importance of functions h such that u h > I @ m was shown in ch. 6 $5. The above results lead one to think that it is exactly those func-
CH. 8, $1
POTENTIAL THEORY
264
tions which are both special and co-special. The following results lead in that direction.
Proposition 1.4. Let h be sfiecial; there is a real c such that u , h and only if there i s a modification of P such that h is co-sfiecial.
> 1 63 m if
Proof. The sufficiency follows from Proposition 1.3. The necessity may be shown in a way very similar to Theorem 1.1, and is left to the reader as an exercise.
Corollary 1.6. Let P be quasi-compact ; then there is a quasi-compact modification of P if and only if there is real c such that U , > 1 @I m. Proof. Obvious. In Proposition 1.4, the modification of P may depend on the function h. However, in important cases, one can find a modification of P which works for all functions h. To state the corresponding result we shall use the notational device of ch. 1 95.3 and recall (ch. 3 32.14) that if Ptf. is the singular part of P , with respect to m, then limn PA( , E ) = 0 pointwise. The following statement covers the case where P << m, hence in particular the case of discrete chains, the case of Harris random walks on groups and the case of quasi-compact P .
-
Proposition 1.6. If limnl1~,(* , E)I( = 0 and if h is sfiecial, then there i s a real number c such that u , h > I @ m af and only if h is co-s@ecial. Proof. Only the necessity requires a proof. Without losing any generality we may assume that JlhlJ= a < 1 and u h > I @ m. As in Theorem 1.1, we may prove that for every g in bb,, we have
cz
Pick c such that a < c < 1. The kernel t ) , = (1 - c)" P,,, may be written 0, = 0: + t):, where 0:is the singular part. The hypothesis implies that there is a number a > 0 such that t)!( E ) 2 a. By the resolvent equation, we thus have a ,
o& 2
- h) ( o h g ) 3
~ c ( C
oq(C- a ) o&,
CH. 8, $1
and since
266
HARRIS CHAINS AND DUALITY ohg
> m(g)m-a.e., for every g E bb,,
Od > a(c - a ) m(g). I t follows that h is co-special.
Exercise 1.7. (1) Let X and be two Harris chains in duality and h a strictly positive, special and co-special function. If h is sufficiently small, prove that the associated kernels w h and @h are in duality relative to m. If rp and $ are bounded functions, the kernels
r=I+
wh
+ 1 8 ym + $ 8 m,
P
=
I
+ W,, + q~ 8 m + 1 8 $m-
are potential kernels in duality. (2) If f is in 9 ( m ) , prove that for any potential kernel the functions and Pl'lfl are m-almost everywhere finite. If moreover m(f) = 0, then
r
rlfl
( I - P ) rf= f m-a.e. Exercise 1.8. Prove that in the results of this section one can avoid the use of modifications of p whenever E is an LCCB space, p a Feller kernel and k a continuous function. [Hint: Use the results given in 34.1 Exercise 1.9. Let E be the unit interval [O. 11 of the real line, Q the a-algebra of Bore1 subsets of E and m the Lebesgue measure on 8.Set i ( x ) = x for x E E , and define a pointwise transformation 0 of E by
(1) Compute the Lebesgue derivative of @(m)with respect to m. (2) Calling p the derivative dO(m)/dm, set, for f E 8,
sf = f and
P
=
8-1,
4s + (1 8 4y) m,
sf = (f e) qJ P
=
4s + (1 - ap) 8 m.
Prove that P and P are two T.P.'s in duality relative to m, that P is quasicompact and that no modification of p is quasi-compact. (3) The same example provides a counter-example to the following sharpening of ch. 6, Exercise 2.13: if P is quasi-compact, P , converges to 1 @ m in the norm of linear operators on L1(m).
CH. 8, §2
POTENTIAL THEORY
266
2. Equilibrium, balayage and maximum principles In this section we show that for each one of the potential operators I' defined in ch. 6, Proposition 5.4 one can develop a potential theory which parallels the theory in ch. 2. The function q in is supposed to be bounded.
r
Theorem 2.1 (equilibrium principle). For every set A in d such that m(A) > 0 and every potential kernel there is one and only one triple (uA,C,, v A ) where (i)uA(vA)is a function (measure)vanishing outside A and C A a constant; (ii) (m, z t A ) = ( v A ,I ) = 1 ; (iii) r z t A = C, on A and v A r = CAmon A . Moreover, cA= V A r U A , alzd the following identities are satisfied: (i) PA = r ( Z A - nA) 7 @ v A ; (2.1) (ii) H A = ( Z A - I T A ) UA @ m; (2.2) (iii) r- G A = PAT (ruA - CA) @ m = T H A 7 @ ( V A r - CAm). (2.3)
r,
+
r+ +
+
Proof. We start with the kernel formulae therein, we have
r = I + W of ch. 6, Theorem 5.2. By the
+ m(h)-l (hm) UAh.
UAh + WIAUAh= Wh Restricting this equality to A yields
rIAUAh= Wh
+ m(h)-l (hm) UAh
on A . Since Wh is a constant function and (m, zAUAh)= m(h)by virtue of ch. 3, Proposition 2.9, the function uA = IAUAh/m(h) has the desired property, with C A = m(h)-' (1 - m(h) (hm) UAh).
+
The process is similar for
'uA.
Integrating the equality
with respect to the measure (hm) yields
+ tn(h)-l (hm) UAhmI,,
(hm) UAIArlA = (hm) wzA
and since (hm)W = ((1 - m(h))/m(h)) m, one gets
(hm) UAIArIA = m(h)-l (1 - m(h)
+ (hm)u ~ h~ )Z A .
CH. 8, $2
EQUILIBRIUM PRINCIPLE ETC.
267
As U A I A = 7 , we have ((hm) U A I A , 7) = m(h); the measure m(h)-l (hm) U A I A has thus the desired property, with the same value for C ., Thus we have shown the existence of a triple (uA,vA, C A ) . We turn t o showing that it is unique. From ch. 6, Theorem 5.2 it is easily seen that
Moreover, ztA and v A are positive, and C A = V A r U A . Let u be a function with the same properties as uA. Multiplying to the right by u on both side,s of eq. (2.5) yields
uA Since r u = k on A and UA
+
Z A W u
+ IAWU =
UAIA
(IAUAIA)
ru.
= 1 , this may also be written
= IArt.4 = I A u
+
IAWU
= 21
+
IAWU,
and since W u is finite on A it follows that u = uA. The uniqueness of vA may be shown in the same way. The identities of eqs. (2.1) and (2.2) are straightforward consequences from eqs. (2.4) and (2.5) and from the formulae giving PA,UA,H A as functions of U,. To prove eq. (2.3), let us recall that P A G A = Z IAcUA; using this and ch. 6, Theorem 5.2, we find
+
PA=
r - GA+ VZ(~)-'
IAc
+
UA(h)@ m ;
multiplying on the right by uA the two members of this equality yields, thanks to the properties of z t A , C A
=
ruA
+ m(h)-'
IAc
UA(h),
which completes the proof, the other equality being shown in a very similar way. I t remains to show that all is still true for a kernel I" equal to r I @Y q~ 8 m. For instance, it is easy to show that if the measure v> occurs in eq. ( 1 . 1 ) with I" instead of then v i = v A - v ( I A - IIA). By means of eq. (2.5)and of the equality v H A I A = vIA,one may compute ( v A- v(ZAI" and find that it is a multiple of m on A ; moreover, it is clear that (v;, 1 ) = 1. In the same way one can show the existence of u i .
+
+
r,
nA))
CH. 8, 52
POTENTIAL THEORY
268
r+
Remark. The result may be extended to kernels 1 @ v + q @ m, where y~ is merely finite, if we restrict ourselves to sets A such that 17,~is finite. This will be of significance for Harris random walks. Definition 2.2. We call v A ( u A ) the equilibrium measure (function) of A with respect to We call C A the Robin's constant of A with respect to
r.
r.
A probabilistic interpretation of v A will be seen in the next section under an additional hypothesis. We may already note that v A is absolutely continuous with respect to m.
r
Proposition 2.3. Let and P be potential kernels for X and ft in duality with respect to m. Then for every A in 6 such that m(A) > 0, we have C A = and vA = i A m .
cA
Proof. For every g E b,, vanishing outside A , Thus, (uAm)f C A
=
=
C i m on A , and since m(uA) = 1 we have uAm = i Aand
CA.
In Exercise 1.7, the reader will find how to get potential kernels in duality. We are now going to use the identities in Theorem 2.1 to prove Potential Theory principles, in a way parallel to ch. 2, Corollary 1.12 and ch. 2, Theorem 4.5 for the transient case.
Theorem 2.4 (balayage principle). For every A in d such that m ( A ) > 0 , and for every probability measure v, the probability measure v A = vP, i s the only probability measure such that VAT = V r km on A for a constant k.
+
-
Proof. In view of eq. (2.3), together with the fact that the measures P ( x ,) vanish on A , it is clear that vA satisfies the required property with k = ( v , ruA - C A ) . Now let v' be another probability measure such that v ' r = vT k'm on A . The measures HA(%, ) vanish for x $ A , hence
+
V'rHA
=
VrHA
+ k'mHA,
CH. 8. $2
269
EQUILIBRIUM PRINCIPLE ETC.
and since by ch. 3, Proposition 2.9, mH,
= m,
we get
Now eq. (2.3). along with the fact that GA(x,* ) vanishes for x that v i r = V A T (k' - k ) m.
E
A , implies
+
Multiplying to the right the two sides of the latter equation by ( I - P ) yields Y'
- m ( h ) - l . (hm)P
provided
I'
=
I+W
+ p(1 - P ) = Y,
+I
@p
+
+~
- m(h)-'. (hm)P
(-1
@ m, and consequently v' = vA.
qj
Theorem 2.6 (maximum principle). Let f be in A'- and such that m(lf1)> 0. If there is a number k such that rf k on (f > 0 } , then I'f k - f- every-, where.
<
Proof. Set A = { f
> O};
<
we have clearly m(A) > 0, and eq. (2.3) implies
rf < GAf + PArf; and since GAf
< - f-
and P A r f
< k everywhere, the theorem is established.
Definition 2.6. The kernel T is said to satisfy the reinforced semi-complete maximum principle. The maximum principle may be stated in another fashion.
Corollary 2.7. Let f l and f 2 be s$ecial and such that m(fl) = m ( f 2 )> 0. If I'f2 k on { f l > 01, then rf, I'f2 k everywhere.
I'fl
<
+
<
+
Proof. We apply Theorem 2.5 to the function f {f > 0) { f l > 01.
=
r
+ +
=
f l - f 2 after noticing that
+ +
Exercise 2.8. If =I W I @I v q~ @I m, with v a positive measure, show that in Theorem 2.5 the assumption m(lf1) > 0 may be dropped. [Hint: If m(lf1)= 0, then I'f = Gf v ( f )and G satisfies the reinforced maximum principle.] Exercise 2.9. Show that the kernel F
=
I
+ W satisfies the following max-
POTENTIAL THEORY
210
CH. 8. 52
imum principle: if f and g are special and such that m(f)>, m(g), then the inequality I'f I'g k holds everywhere if it holds on { f > 0). [ H i n t : This may be proved either by using the methods of this section or by calling upon ch. 2 32.10.1
< +
Exercise 2.10. If f is in A'" and vanishes outside A , then, as in the transient case, PArf = I'f for every potential kernel
r.
Exercise 2.11. (1) Assume that there is a point x in E such that the set { x } is recurrent (e.g., X is discrete and irreducible recurrent), then the kernel G c z )is a potential kernel. A function f is then special if and only if G'")f is bounded. (2) Let X be the chain with state space E = N and P ( 0 ; ) = p ( * ) , P(%,* ) = E ~ - ~ ),( where p is a probability measure on N. Prove that X is irreducible recurrent ; find a necessary and sufficient condition on p for X to be null, and characterize the special functions.
r
+
Exercisc 2.12 (capacities). Let =I W ; then for every A such that m ( A ) > 0 we have C A > 0. Define a set function y by y ( A ) = - C A if m(A) > 0, y ( A ) = - co if m ( A ) = 0, and prove that y is increasing and strongly subadditive. Exereisc 2.13. Prove that for every set A in 8 such that m(A) > 0, and every special function /,
[ H i d : Use ch. 6, Exercise 5.17.1 Excrcise 2.14. Prove the following domination principle which is dual t o the maximum principle of Theorem 2 . 5 : If p and Y are two mutually singular probability measures and A is a set such that p ( A ) = 0, v(AC)= 0 and m ( A ) > 0, if ( p - Y) cm is positive on A €or some constant c > - 00, then ( p - v) 'I - p cm is positive everywhere. [ H i n t : Use the operator H A . ]
r+ +
Exercise 2.15. Prove that the potential kernels of Harris resolvents satisfy the following semi-complete maxinzum principle: if f E JV and k E R, {Wf k}
<
CH. 8, 53
27 1
NORMAL CHAINS
<
on ( f > 0 ) implies W f k everywhere. Conversely, for every a satisfies the reinforced semi-complete maximum principle.
> 0, I
+ aW
3. Normal cha.ins In ch. 6 $6, we saw that for any pair ( f , g) of special functions and any probability measure v, the ratio of the numbers n
m(f)-lCv p m f 0
n
VPmg
and m(g)-' 0
converges to 1 as n tends to infinity. One could think of sharpening this result by showing that their difference tends to a finite limit. In view of ch. 6, Proposition 5.7, this amounts to showing that for f E N , the sums P,f converge pointwise. This is also of interest from the potential theoretic point of view, since the above convergence would imply that f has a potential in the sense of ch. 2. As we will see below, such a result permits one to single out from all the kernels exhibited in ch. 6, Proposition 5.4 a subclass of canonical character. Notably, we have as yet not provided recurrent random walks with a potential kernel which is a convolution kernel. The following study is a first step in the search for such a kernel. Unfortunately the above limit may fail to exist, as is shown in Exercise 3.11, and we shall proceed to what may be said if it exists.
27
Theorem 3.1. For a Harris chain the three following statements are equivalent: (i) for every charge f , limn Pmf exists; (ii) for every non-negligible special function h of a+there i s a probability measure 1, such that, for every g in bb, and x in E ,
ct
lim PnUhI, g(x) = &(g) ; n
(iii) for every non-negligible special function h of Q, every x in E and B in 8, the sequence ( P n u h z,(x, B ) }has a finite limit as n tends to infinity.
Whenever h = I,, we write i l A in place of 1,; we then have A A = limn PnPA, and A , vanishes outside A . Let us also remark that by ch. 6, Proposition 5.7, the convergence in (i) is bounded.
Proof. Obviously (ii) implies (iii). Conversely, for every B E 8,the function
POTENTIAL THEORY
272 Ah(
*,
CH. 8, $3
B ) = lim P,uh I h ( B ) a ,
n
is bounded and harmonic, hence constant, and we can set &&(B) = lim PnUI, I h (
a ,
B).
n
The Vitali-Halin-Saks theorem then implies that &, is a probability measure for which (ii) is satisfied. We proceed to show that (ii) implies (i). Let f E Nand h a function in a+,special, strictly positive and such that lfl/h is bounded (for instance a multiple of I f [ g where g is special and strictly positive). For any kernel W , we have wf = u h f uhIhwf = U h l h ( f / h wf),
+
+
+
and since Wf =
2;Pmf + P,+lWf, this may be written n
1
Pmf
=
wf -
Pfn+lUhlh(f/h
f
wf)*
+
The function f/h W f is bounded and consequently (ii) implies (i). Conversely, assume h special and non-negligible, and let g be in bb+. The function f = hg - huh(&)is a charge because If1 2hllglI and
<
m(f) = (hm, g)
- (hm, uhI&) = 0
by ch. 3, Proposition 2.9. By the resolvent equation it is easily seen that Pf = ( I - P ) U,I,g, which leads to n
2 1
Pmf
=
uh(hg)
- pn+luhlI&s
and it follows that (i) implies (iii) and therefore (ii).
Definition 3.2. A Harris chain satisfying the equivalent conditions of Theorem 3.1 will be called normal. Proposition 3.3. A chain X i s normal if a i d only if the pointwise limit limn P,Wf exists for every function f such that I f / is special and any kernel W . It suffices that the limit exists fov f in N. Proof. The identity n
W f = C Pmf 1
+ Pfn+IWf,
NORMAL CHAINS
CH. 8, 93
273
which holds for f EX,shows that if the limit in the statement exists for f E X , then X is normal. Conversely if X is normal, then P,+,Wf converges for f in X . Let f be special and h be the function used to construct W ;the function m(h)f - m ( f )h is in M , hence
has a limit when n tends to infinity, which completes the proof.
Theorem 3.4. If X is normal, there exists a potential kernel
f
r such that for evevy
EM, n
lim n
2 P,f
=
O
rf
and lim PJf = 0. n
Moreover, the probability measure A A of Theorem 3.l(ii) is the equilibrium measure of A with respect to r.
Proof. By the preceding proof, it suffices to show that there is a measure v such that, for every special function f , v ( f ) = limn P,Wf. The kerneL =I W - 1 @ v will then satisfy the first part of the statement. The function limn P,Wf is bounded and harmonic and therefore constant. Let hl be special and > 0 ; the sequence of bounded measures P,WI,, converges on every bounded function by Proposition 3.3, so that, by the VitaliHahn-Saks theorem, there is a bounded measure v1 such that for f with If1 < hl,
r
+
lim P,Wf
=
vl(f)
n
Let h2 be another function with the same properties and v2 the corresponding measure. The measures v1 and v2 agree on the positive functions less than h, A h,, hence are equal. Since every special function is majorized by a strictly positive special function, there is indeed a measure v such that lim PnWf
= v(f)
n
for every special function f . Now let v A be the equilibrium measure of A with respect to we have, for A special and non-negligible,
r.By eq. (2.1)
CH. 8, 93
POTENTIAL THEORY
274
For f in b d the function ( I , -
nA)f is in N ,and passing to the limit in
PnP*f = pflw.4
-U A ) f
yields a n ( / ) = A A ( f ) .
+
VA(f)
By ch. 6, Proposition 5.4 if I' and I" are two kernels with the properties stated in Theorem 3.4, then = q~ @ m , where q~ is a finite function. The equilibrium measures are the same for the two kernels but the Robin's constants are different, If we assume that 8 is also normal we may narrow still further the class of potential kernels with a canonical character.
r r'
Proposition 3.6. I f X and 2 are normal, there is a pair of kernels and f in duality with res+ect to m and such that r(P)i s a potential kernel for X ( 2 ) satisfying the conditions of Theorem 3.4.
r
Proof. We first show that the measure Y in the proof of Theorem 3.4 is absolutely continuous with respect to m. Indeed if f is special and m ( f ) = 0 we have for every g E 9: v ( f ) = lim m(g)-1 ( P n W f ,g) = lini m(g)-' ( f , FV'P,~)= 0. n
n
We can thus choose a finite measurable function 9 such that Y = fm. By the first section h may be chosen such that also 0,> 1 @ m. Then W is in duality with W . If X is normal, then p,,m converges to 1 @ Y, and by the above &scussion we can choose a finite function y such that = ym. The kernels
T
=
I + W - 1 @ f m- 7 @m,
f=I +W-I
@ ywt -
9@m
thus have the desired properties. We notice that they map special functions into finite ones but not always into bounded ones. The functions y and 9 are defined up to equivalence. In the next chapter we shall see that for random walks there is a canonical choice for these functions. They are also uniquely determined for discrete chains, where for instance
for any y
E
E. We recall (ch. 6, Exercise 4.11) that all functions with finite
CH. 8, $ 3
NORMAL CHAINS
276
(r,
support are special. In these cases, the pair P ) is unique up to addition of the same multiple of 1 @I m, so that the Robin’s constant, which is the same for and f: is determined up to addition of a constant which is the same for all sets. For a further study of Robin’s constant see Exercises 3.12 and 4.18. Normality is a rather stringent condition, and one could think of demanding the convergence of {PnWf>only for functions f in subclasses of 9’. Actually if these subclasses are rich enough, one can develop the same theory as above with the obvious changes. One can even solve the Poisson equation with second member the opposite of a special function. This will be sketched for discrete chains in the next section, and studied for random walks in ch. 9. Unfortunately, Exercise 3.11 shows that there are chains for which the convergence property fails to be true for any reasonable subclass. It is therefore a natural task to find out whether all “classical” chains are normal in a more or less stringent sense. Here is a first easy result.
r
Proposition 3.6. All aperiodic positive chains are normal.
Proof. The proposition is an obvious consequence of ch. 6 $2. In the following chapter, we shall prove results of this kind for Harris random walks on abelian groups, and the reader will also find examples of normal chains in Exercise 4.17.
zy
Exercise 3.7. If f E 9 l ( m ) and P,f has a finite limit when n tends to infinity, then m ( f ) = 0. If g E bb fl P ( m ) and f = (I - P ) g, then P,f tends to a finite limit when n tends to infinity. Prove that the functions of this form are not always charges. Exercise 3.8. If X is normal, prove that for any f limit limn P,U,g exists.
27
E
a+and g special, the
Exercise 3.9. A set A E 8 is said to be small if there is a probability measure 1, such that for all g in bb, lim P,P,g = AA(g). n
(1) Prove that a subset B of a small set A is sinall and that ,IB = AAPB. Then show that for a positive chain all sets are small and that A, = mP,. (2) If X is normal, special sets are small sets, but the converse is not true.
POTENTIAL THEORY
276
CH. 8, $3
Show that the union of a small set and a special set is small.
Exorcise 3.10. Assume that X is normal and show that in ch. 6, Theorem 6.5 one may assume that f and g in Lfl(nt)provided that v 1 and v 2 are absolutely continuous with respect to M , the Radon-Nikodym derivatives being cospecial. Exercise 3.11. Let X be discrete and irreducible recurrent. (1) Show that if 2 is normal, then for every pair ( x , y ) of points in E the limit N
lim N n=O
(Pn(yt Y ) - Pn(x, Y))
exists and is finite. (2) Pick two points a and b in E and set F = {a, b} and for t~ [0, 1[ m
A ( t ) = C tnP,[SF = n ] , fi=l
Show that if
m
B(t) = 2 t n P ~ [ S = p n]. *=l
9 is normal, the following limit exists: lim X ( t ) = lim (1 - A ( t ) ) / ( l- B(t)). 1-1
1+1
[Hint: Express the sums 2 : Pn(x,y ) t n as functions of
2 tnP.[{S,, = n } n {X,, = - }], m
1
and then use (I).] (3) Assume now that the states are labelled a, =
a, a l , u 2 , . . ., a,,,.. .
bo = b, b1, b z , . . ., b,,,.
..
and that the transition probability is given by
P(a, ai)
= pi =
P(b, bi) = pi
Ci Ci
=
P(ai,b), P(ai,ui) = qi = 1 - pi, P(bi,a ) ,P(b,, bi) = q:
= 1
- pi,
with pi = = 1, all other entries being 0. The chain X is irreducible recurrent, as is easily checked. Let {nk} and {n;} be two increasing sequences of integers such that nk < n; < nk+l.The numbers pi are chosen in the following way: the first
CH. 8, 54
FELLER CHAINS
277
nl of them are equal to = (2n1)-l, the n2 following ones are equal to (2%,)-l, and so forth; there are therefore nk consecutive pi's equal to &k = (2kn,)-1. The same pattern is followed for the numbers f j i , but with the use of {n;}in place of { n k } . Show that one can choose the sequences {nk)and {n;}in such a way that t 2=
lim ~ ( -1 E,,) # lim ~ ( 1 E:), -a
n-*m
and deduce that the corresponding chain 2 is not normal. (4) By a slight modification of the above example, prove that the limit in (1) may exist for 2 and fail to exist for X . ICxereiso 3.12 (capacities). (1) Let X and 2 be two normal chains in duality, and P two potential kernels as in Proposition 3.5. For every non-negligible special set A , show that there is a measure V A absolutely continuous with respect to m and such that
r
lim P,GA = 1 @ y A ,
I A r
=
+ vA.
CAm
n
(2) Let A , B be special and L be any special set containing A U B , then
C , - C A = (vA- vB$ML).
<
(3) Show that C A C, if A c B. For a negligible set A , we shall set C A = - m. Then show that for any special sets A , , . . ., A,,,
Excrciso 3.13. Extend the definitions and results in this section to Harris resolvents. 4. Feller chains and recurrent boundary theory
Throughout this section, E is an LCCB and the transition probabilities P and P are Feller kernels. Since PI = 7, in order that P be Feller it suffices that P maps C K into C,. Since the topology of E has a countable basis, it is readily checked that there is a largest open set 0 such that m ( 0 ) = 0'. We leave to the reader as an exercise the task of showing that 0" is an absorbing set. The set 0
POTENTIAL THEORY
278
CH. 8, $4
is thus of no significance in the study of X , and in the sequel we shall always assume that the following condition is satisfied.
Condition 4.1. For every open set U we have m(U) > 0. This condition is naturally satisfied for all the usual chains such as discrete chains and Harris random walks. Proposition 4.2. There exists a special function which i s strictly positive and continuous. Proof. Let h be a strictly positive special function. There exists a number u such that co > m({h a } ) > 0. The set {h a } contains a compact set K with m ( K ) > 0, and K is special. For 0 < c < I, the function U,( K ) is upper-semi-continuous, strictly positive and special. The space E is the union of the closed sets { U,( , K ) 2 7r1}. By Baire's theorem one of these sets has a non-empty interior; there exists a non-zero function p in C i that has support in this set and so is special. B y Condition 4.1 we have m(p) > 0, and therefore U,p is special, continuous a ,
-
and strictly positive. This proposition has the following obvious
Corollary 4.3. Every function in bb, with compact support is special. The measure m i s a Radon measure. Let /(I)be continuous, special (co-special) and strictly positive. Then by Proposition 1.3, there exists a multiple, say h, of f A f such that U h > 1 8 m and > 1 @ m. The function h is continuous and strictly positive. For any non-zero function in C i there is a multiple, say g, such that U , > 1 m and U g > 1 @ m. In the sequel h and g will always stand for functions enjoying the above properties.
oh
Proposition 4.4. If f E C, and I f ) < ch for a positive ,number c (in particular if E C,) then U,(f) and If',(/) are bounded and continuous. Furthermore W,(f) is bounded and continuous for every g.
f
Proof. If suffices to consider a function f such that 0
< f < h. Then V,(l)
is lower semi-continuous because it is defined by a series of positive continuous functions. Since h - f h, Uh(h- f ) = 1 - U,(f) is also lower semi-
<
CH. 8, $4
FELLER CHAINS
279
continuous; hence U,(f) is continuous. The same argument works for W,(f), because W,(h) is constant and therefore continuous. Finally, by ch. 6 $5, we have
and the latter part of the statement follows from the former. In $3, we saw that aperiodic positive chains are normal. This was due to the fact that the powers P, of the T.P. converge to the bounded invariant measure m. We are now going to give for null chains a result relating normality t o some kind of convergence of the powers P,. Let I’ = I W be a potential kernel, where W is associated with one of the functions h or g above. Choose a denumerable set { f n } n > l of functions dense in C K for the topology of uniform convergence on compact sets. By the above proposition the functions are bounded and continuous. As in ch. 7 $2, we choose a metric 6 on E, compatible with the topology of E and such that the Cauchy completion of ( E , 6 ) is obtained by adjoining the Alexandrof’s point A . Furthermore, for x, y in E we set
+-
rf,
m
d ( x , Y) =
Then d
C I r f n ( ~) ~fn(Y)l/2nllrfnll. n= 1
+ 6 is a metric on E,and we lay down
Definition 4.5. We define E* to be the Cauchy completion of the metric space ( E , d 6 ) and set M = E*\E. The set M is called the recurrent boundary of E for X.
+
For every compact subset K of E, there is a number MK such that if f , g E bB and vanish outside K , then
{rfn}
is dense in the set {rf, f E C,} for the topology As a result, the sequence of uniform convergence, and I‘f is uniformly continuous on ( E ,d + 6) for every f E C K . These functions thus extend continuously to E*;the extended functions will still be denoted by rf.If x is a point in M , the map f -+ r f ( x ) defines clearly a Radon measure on E which will be denoted by T ( x ,* ). Let us observe that a sequence {x,} of points in E converges to a point x in M , if and only if {x,} converges to A in the original topology of E and if
POTENTIAL THEORY
280
-
CH. 8, $4
+
the measures r(x,, ) converge vaguely to r ( x , ). The metric d 6 is also extended to E* by using the extended kernel The space E* may depend on the particular function h or g used in defining I'. In the following proposition, we accordingly write E: and E t . 0
r.
Proposition 4.6. U p to homeomorphism, the space Eg* does not depend on the particular function g, and there is a cpntinuous map from E,* onto E:. Proof. From the identity
it follows that if {x,} is a sequence of points in E converging to d and such that W,(X,, ) converges vaguely, then Wg(x,, ) converges vaguely. We define the map j from E: to E J by setting Wg(j(x),) = lim W,(x,, ) if W&, * ) = lim Wh(x,, ). If {x,} converges to x in E J , one may extract a subsequence {x,} from {x,} such that W,(X,, ) converges; consequently the map j is onto. It is also clearly continuous, and that Eg* does not depend on g is now evident.
-
-
-
-
Remark. If x and y are two points in E: such that j ( x ) = i ( y ) , the kernels W,(x, ) and W,(y, ) are equal up to addition of a multiple of 7 @ m. This will be useful in the following chapter, where E* will be computed in the
-
case of random walks. From now on, E* will denote the Cauchy completion of E obtained for a function g E C, n 4Y+ and such that U, > 7 8 m. We observe that if x and y are two points in E such that r ( x , ) = r ( y , ), then, by ch. 6, Theorem 6.2, we have W x ,* m(g)-' @ (gm)p = W Y , m(g1-l B ( g 4 p,
-
+
- +
r
and since W = - I, it follows that x = y. If x and y are in M and r ( x ; ) = r ( y ; ) , the sequence whose terms are alternately x and y is a Cauchy sequence and thus x = y. But it may happen that for a point x in E and a point s in M we have r ( x , ) = r ( s , ). Also r ( s , ) may be a barycenter of measures r ( s ' , ) for other points s' in M . This recalls the situation of ch. 7 which explains the additional condition under which we are going to work. If s is in M , the measure r ( s , * ) is the limit of the measures r ( x n , * ) for a
-
CH. 8, 54
FELLER CHAINS
281
suitable sequence {x,}. By passing to the limit, for the vague topology on E , in the identity
r p f = rf where f
E
-f
+ ( g 4 Pflm(g)
Cg, we get
- + ( g m )Pim(g).
-
r ( s , ) P G r(s,
Let us call 6 the set of positive Radon measures Asuch that A(g)= ( 1 -m(g))/m(g) and
RP
< A + ( g m )Plm(g).
The significance of the following condition will be made clearer later on in the case of discrete chains.
Condition 4.7. For any 1 E 6, there is a unique measure
PA on E* such that
r
We may now turn to the result we were aiming at.
Definition 4.8. The chain X is said to be weakly normal if the sequences
2; Pkf have a finite limit whenever n converges to + co, for every f E N n C K .
Proposition 4.9. The chain X i s weakly normal if and only if {P,I'f} converges flointwise for a n y f E C,. T h e limit i s then equal to v ( f ) where v E 6. Proof. The reasonings are the same as in Proposition 3.3 and Theorem 3.4. The fact that Y belongs to 6 follows upon passing to the limit in the equation
pnrpf = P n r f - Pnf
+ (gm)P f If f i k).
Theorem 4.10. If X i s null and {P,(x, * )} converges vaguely on E* for every x in E , then X is weakly normal. If Condition 4.7 i s in force and X i s weakly
-
normal, then (P,(x, )} converges vaguely on E* for a n y x
E
E . The limit P
i s independent of x and carried by M .
Proof. Since the functions I'f, f first sentence is obvious.
E C K , have
a continuous extension to E*,the
282
POTENTIAL THEORY
CH. 8 , $4
Conversely, since E* is compact, for any x, we may find a subsequence { n k}such that {Pn& ) } converges vaguely on E* to a limit ,& By Theorem 2.6 in ch. 6, the measure p is carried by M . Since for any f E C,:, the function rf extends continuously to E* we have
prf
=
lim Pn,r/(x) = v(/), k
in other words
v=
j r(s,.; PW).
By the uniqueness in Condition 4.7 we see that p does not depend on x nor on the particular subsequence. Thus (P,} converges vaguely on E* which completes the proof.
Remarks. (1) If the equivalent conditions in the above theorem hold, the sequences { P J f } converge for every function f such that rf extends to a function in C(E*). We will see in the next chapter that, even for random walks, the class of such functions does not always include all special and continuous functions. (2) The measure 1 has the following intuitive interpretation. Assume E to be discrete and let K be a finite set; then P,P,(x, y ) may be viewed as the probability that the chain started at x at time - n enters K at point y after time zero. If A, is the probability measure defined in Theorem 3.1, then ,I& appears ) as the probability that the chain started at time - 03 enters K at point y after time zero. If we let K swell to E , then ,IK must converge to an entrance distribution for the chain. I t will be seen in Exercise 4.16 that the limit of jlK is precisely the measure p. That p is an entrance distribution is also well seen from the discussion below. In the remainder of this section we shall assume that X is a recurrent irreducible discrete chain and we shall try to illustrate the results of this and the previous sections. In the following chapter a similar study will be performed for Harris random walks on abelian groups. In the sequel, e will stand for an arbitrary but fixed point in E . By the above discussions, we may define = I W by taking as function g a suitable multiple of I,,,. To lightcn the notation we shall drop the brackets and write I , for the characteristic function of the set { e } and Ge(x, ) instead of GIe)(x,. ).
r
+
CH. 8, $4
283
FELLER CHAINS
From ch. 3, Theorem 1.9 it is known that H,(x, y ) = 0 if x # e and H,(e, y ) = m(y)/m(e). I t follows then from Theorem 2.1 that
but
r(
*,
e) =
uliere v = km
I,
+ W ( I , ) ,and W(1,)is a constant function; hence
+ v e r - C,m
for a constant k. This will be used to prove
Proposition 4.11. The space E* i s isoinorphic to the Martin spacc of the tvansient chain. P whose T.P. i s defined by Q(x,y ) = P ( x , y ) if y # e and $(x, e ) = 0, started with distribution 17 = E,. Proof. The chain P is clearly transient, and is in duality relative to nz with the chain Y whose T.P. is given by Q(x, y ) = P(x, y ) if x # e and Q(e, y) = 0. The potential of Y is easily computed and is equal to G”(x; ) = C ’ ( x ; )
+1 @
E,.
Furthermore G Y l , = 7 so that l , / m ( e ) may be used as a reference function for P. Now the space E* is lionieomorpliic t o the closure of the set {Qx, ), x E E } for the vague topology, but this is also liomeomorpliic to the closure of the set { G e ( x ,* ), x E E ) , which in turn is homeomorphic to the closure of the set { C y ( x ,* ), x E E}, and the proof is completed.
-
We can now see the significance of Condition 4.7; in the present case it means that, with the notation of ch. 7 $3, the boundary M is equal to its extremal or useful part M e . We may also use the representation theorem for &-harmonic functions whose significance for the chain fi is given by the following
Proposition 4.12. A finite positive function f with f ( e ) = 1 i s Q-harmonic if and only if it i s a solution of the Poisson equation
(I-P)f Proof. By definition of
=
- P ( * ,4 .
0, the equation Qf
2 Qx, yze
=
Y )f(Y) =
f reads
f(4;
CH. 8. $4
POTENTIAL THEORY
284
in other words
PI(%)- P ( X , 8)
=
f(4,
and the result follows. We thus see that @harmonic functions are co-subharmonic functions and provide solutions of the Poisson equation with second member the opposite of a multiple of f‘( - ,e). The function p ( * , e ) is co-special and if k is another co-special function, then q~ = m(K)p ( * , e) - k m(e) is a co-charge and the Poisson equation ( I - P) f = cp has the solution pq~.If f is &-harmonic, such that f ( e ) = m(k), then FIJI f is a solution of the Poisson equation ( I - P ) f = - k. As a result, if there exists a &-harmonic function, one can solve the above Poisson equation whatever the co-special function k is. I t is not obvious that there are always &-harmonic functions such that f ( e ) = 1. The above discussion shows that weak normality entails the existence of such functions for which we have an integral representation. Indeed if s is in Me, the density = d r ( s , )/dm is such that p p = p” P( , e) and p(e)= m(e)(1- (m(g))/m(g) = a. In the present setting theintegralrepresentation theorem becomes
+
-
+
Theorem 4.13. I f X is null, for every positive solution f of the equation ( I - p ) f = - fi( , e) such that f ( e ) = a, there is a unique probability measure /If, carried by M e and such that
-
This raises the problem of describing M e for classical recurrent chains; that will be done in the next chapter for Harris random walks on abelian groups.
Exercise 4.16. Call F the Caucliy completion of E with respect to the metric d in the text. Prove that X is weakly normal if and only if the sequences Pn(x,* ) converge vaguely on F. Is the corresponding measure p still carried by the boundary ? Exercise 4.16. Let X be weakly normal and { f k ) a sequence of functions in C i increasing to 7,; let Ak = limflPflUfkZfk (cf. Theorem 3.1), and prove that, under Condition 4.7, the sequence {Ak} converges vaguely on E* to the measure p.
CH. 8, $4
FELLER CHAINS
286
Exercise 4.17. Assume that X is an irreducible null recurrent discrete chain on E = N such that P(k, I) = 0 if k 3 1 2. (See, for instance, ch. 2, Exercise 1.24.) (1) Prove that m is the only invariant measure for P even if we allow the measures to take negative values. (2) Prove that from every sequence of integers converging to infinity one may extract a sub-sequence such that, for all pairs (x,y ) of points in E ,
+
lim P,W,(x, Y) i
-
v(y)
exists. Notice that Pf has finite support if f has, and prove that
Then derive that X is weakly normal.
Exercise 4.18 (degenerate discrete chains). We use this term for discrete, weakly normal chains such that the value C, of the Robin's constant is the same for all finite non-empty sets A . (1) Assume that is also normal in the same sense and show that the kernels and I'in Proposition 3.5 may be chosen in such a way that C A = 0 for every finite non-empty set A , and that in that case r ( x , x ) = P(x,x ) = 0 for all x in E . (2) Let ( x , y ) be any pair of points in E and set A = { x , y}. Prove that either T(Y,x ) = 0, T(x,y ) > 0, &(y) = 1, A,(X) = 0 or 4 4 ( Y ) = 0, U x ) = 1. r(Y, x ) > 0. T(x,Y ) = 0,
r
(3) Let y < x stand for r ( y , x ) = 0, and show that < is an ordering on E . Show that if z < y < x, then P J T , < T,] = 1, and conclude that in moving to the right, the chain can move at most at one step at a time. (4) Prove that the ordering of states must be that of the integers, the positive integers, or the negative integers. (5) Let X be the renewal chain of Exercise 1.16 in ch. 3.Find the condition under which it is null recurrent, and prove that it is then weakly normal and degenerate. Other important examples of degenerate chains will be given in the next chapter.
CHAPTER 9
RECURRENT RANDOM WALKS
This chapter is devoted to the study of Harris random walks. Although everything may be carried over to non-abelian groups, we shall deal here only with abelian groups, in order to avoid some technicalities and keep within reasonable bounds. Throughout this chapter, we thus study a Harris random walk with law ,u on an abelian localiy compact metrizable group G . To save space, we shall use the multiplicative notation in G . When G is compact, all the results are either trivial or meaningless, and we shall therefore always assume G to be non-compact. Finally, a Harris random walk being a Feller chain, we shall use freely the results of ch. 8, items 4.1-4.4.
1. Preliminaries In the sequel, t will stand for any translation operator in G , and we shall write t, when we want to emphasize the relevant element a in G , namely t, f ( x ) = f(xa). We write z-l for the inverse of t.
Lemma 1.1. For any function h E %+ and such that t h has the same properties and w,h
u h
>1
m, the functioit
= twht-’.
Proof. Left to the reader as an exercise. We shall first construct a function h with particularly nice properties,
Lemma 1.2. Let f E Co and I< be a compact set in G ; the function g :x -+infbsKf(xb) is in C,. Proof. Since the function f is uniformly continuous, for every E > 0 there is a neighbourhood V of e such that if xy-l E V ; then for all b in G ,
If(W - f ( Y b ) l < 6 . 286
PRELIMINARIES
CH. 9, $ 1
287
and taking the infimum over b e K , first in the left member then in the right one, yields g(x) < d Y )
The inequality g ( y ) < g(x) the proof is complete.
+
E
+
6.
may be shown in the same way, and thus
Proposition 1.3. There is a strictly positive symmetric function h in %+ n Co and such that (i) U h > I @manu' I @m; (ii) limb+el\Wh(T,,h- h)ll = limb..+ell@h(~bh - h)JI = 0.
oh>
Proof. Let u be a symmetric non-negligible function in C i . Since X is Harris, the function W
h, = u c 2-^P,u = u(u
+ 2-1uz-lu),
0
where a E R,, is strictly positive, in Co, and special. In the same way put m
h,
=
u c 2-nPnu; 0
by ch. 8, Proposition 1.3, if we t a k e a small enough, then hz = hl symmetric, in c0n %.+ and uh2 > I @ m , o h 2 > I @ m. Let I/' be a symmetric compact neighbourhood of e and set
A
h1 is
h(x) = inf hi(xb). be V
We will show that k satisfies (ii), and since it clearly satisfies all the other requirements it is the desired function. Pick E > 0 and call K the compact set {h, 3 E } . Since V is symmetric, for every x in G and b in V we have h(xb) < h?&); hence Ih(x) - h(xb)l < hi(x), and it follows that, for x in K" and b in V ,
Ih(x) - h(xb)l < Ehz(X). On the other hand, since h is uhiformly continuous, there is a neighbourhood V' C V such that, for b E V' and x E K ,
288
CH. 9, I1
RECURRENT RANDOM WALKS
Ih(x) - h(xb)l
< < &2
&hZ(X).
Combining the two inequalities yields, for every b E V’,
lrbh - hl Consequently IIWhz(tbh- h)II Corollary 5.5,
Wh(tbh - h)
< &hZ*
< &m(h,)/((l- m(h2)),and
= W h z ( t b h - h)
it follows that limb,,~lWh(t,h- h)ll same way.
=
-
since, by ch. 6 ,
(h, W h 2 ( t b h- h ) ) ,
0. The dual result is obtained in the
Throughout the sequel we shall use the kernels W hand @h (and we often write simply W and W ) associated with this particular function h. For every z, the function t h has similar properties, and we shall also use the kernels W,h and They satisfy the following
wZh.
The desired result is then a consequence of Lemma 1.1. We now proceed to the key technical results.
Lemma 1.6. For any x in G and f is bounded.
E
C i , the set of n.unzbers { W h ( t a f( )x ) , a E G }
Proof. Let V be a compact neighbourhood of e, and choose f’ E C i such t b f ’ 2 f for every b E v.Then
that
PRELIMINARIES
CH. 9, $1
289
Suppose there exists a sequence {a,} converging to A , and such that wA(ta,f) (x) converges to 00 ; the formula
+
lim n
wh(ra,f’)
(y) = f
the convergence being uniform in y E x V . It follows if g E C i a n d g 2 1,“ that
Proposition 1.6. For f
E
C ,,
the sets of functions
{ W ( t , f ) ,a E G ) and
{ W P , f , n E N)
are equi-uniformly continuous on every comeact set in G .
Let K be the compact support of f, H a symmetric compact neighbourhood of e and f’ a function in C i equal to 1 on H K . For E > 0, one can find a neighbourhood V of e such that, for b E V ,
I d - fl < E f ‘ .
290
RECURRENT RANDOM WALKS
CH. 9, $1
By the choice of h, we may now assert that there is a neighbourhood V’ of e such that, for b E V’,
+
IA(b)I <
EWh(Taf’)
(x).
By the preceding lemma, the functions W,(t,,f) are thus equi-uniformly continuous on a neighbourhood of x, and this implies the result by the HeineBore1 property. For the set {WPnf,n E N}, the conclusion follows readily from the first one and the fact that J
Symmetrically, we have
Proposition 1.7. For f E C K ,the set of functions (PnWf,n E N} is equi-uniformly continuous on every compact set in G .
Proof. This is much simpler owing to the fact that P,Wf
< llWfll for every n ,
and it is left to the reader as an exercise. The next result is a first step towards normality theorems.
Proposition 1.8. Let f be special; for every pair ( x , y ) of floints in G , lim (PnW f ( x ) - P,W f ( y ) )= 0. n--+W
Proof. Let U be an ultrafilter finer than Frechet’s filter. Since Wf is bounded for every x in G , L ( x ) = limu P,W f ( x ) exists. Moreover, for k fixed,
L(x) = lim P,+,W f ( x ) . ll
Indeed k
PnW f ( x ) - Pn+,W
f (XI
=
C Pn+i[f- ( ~ ( f h/nt(h))I ) t=1
and since f - ( m ( f )h/nz(h))is bounded and nz-integrable, by ch. 6, Theorem 2.6 the right member converges to zero as n tends to infinity. To simplify the notation, we write qn below in place of PnWf. The measure p being spread out, for k sufficiently large we have ,uk= u + p, where a = gnz. But q n + k ( x ) = (a *
E,,
qn>
+ (P *
E ~ pn>, ,
PRELlMINARIES
CH. 9, $1
29 1
and therefore
The sequence {yn}is bounded; hence it admits a limit ht, along U in a(Lm,L1). (As usual we shall not distinguish between functions and equivalence classes.) Passing to the limit in the last inequality, we thus get
1
IL(4 - $(Y) g(yx-') dY/ ,< IIWflI llPll> which implies
IL(%)
< 211wf(l llfill.
- pk
AS k tends t o infinity, Pk$ converges uniformly to L , which is therefore measurable and invariant, hence constant. The proof is then easily completed. The following result is the key t o the whole of the sequel.
Proposition 1.9. Every sequence
of integers converging to infinity contains a subsequence s = { n j } with the following property: for every 1 in CK,the four sequences { W P y f } , {PniWf),{WPnjf),{PnjWf}converge uniformly on every compact subset as tends to infinity. Moreover, the sequence of measures E, WP, converges vaguely to a multiple ys((x)m of the Haar measure. The function yyis positive, uniformly continuous, and such that Py? = ys m(h)-l Ph,
+
and for every
t, the
function ( I - z)yqis bounded.
Of course, the same result holds for e,WPnj.
Proof. The group G being metrizable, the space C, contains a dense countable subset. I t suffices to prove the first statement for the functions of this subset; but this follows from Propositions 1.6-1.7 and from repeated applications of Cantor's diagonal process. We turn to the second part of the proposition. From the first part, it is plain that the sequence {e,WPniJ.converges vaguely to a Radon measure vz and that the map x + v x ( f ) is continuous. For gE C K , we have v x ( f ) g ( x ) m(dx) = lim (WP,), I
g ) = lim ( f , pniWg), J
292
CH. 9, $1
RECURRENT RANDOM WALKS
so that if m ( f ) = 0, then v"(f) = 0 which shows that v" = y"x) m. We thus have y"(x) = limj W p n jf ( x ) if f is any function in C i with m ( f ) = 1. Let f be such a function; then t-lf is also such a function, and we have tys =
lim tWPnJt-lf = lim W,,,Pnjf. i
j
By Lemma 1.4, we thus have
by the choice of h the function y" is therefore uniformly continuous, and we also see that ( I - t)y' is the bounded limit of ( I - t )WP,,f, hence is bounded for every T . We proceed to show that y* satisfies the equality in the statement. First passing to the limit in the relation
Pn+jf
+ PWP,f
=
+ m(h)-l Ph
WP,f
yields, by mean of Fatou's lemma, that
Pys
< y* + m(h)-'
Ph.
Next, by the bounded convergence above, P(Z - t)WP,f P ( I - t )y". I t follows that, for every t,
+ m(h)-'
P ( I - t )yq = ( Z - t)yq
converges t o
( Z - t )Ph,
and since all the functions herein are finite, we may rewrite this as y" - Py*
+ m(h)-' Ph = t(ys - Pys + m(h)-' Ph).
There is consequently a number k
Pya = y8
3 0 such that
+ m(h)-l Ph - k ,
which implies U
Pnyq= ys
+ m ( h ) - l C Pkh - nk. 1
The function h being integrable, the term on the right converges to - 03 if k > 0 , and this contradicts the fact yqis positive. Hence k = 0 and
PyS
= ys
+ m(h)-1 Ph.
The following result will be important in the sequel.
PRELIMINARIES
CH. 9, $1
293
Proposition 1.10. Every co-sfiecial function is integrable by the measure yqm. I n particular,
I
hys dm
= m(h)-l (1 - m(h))=
q.
Proof. Let g be co-special and f in C: with m(f) = 1. By Fatou's lemma
I
< limi (g, W p n j f ) = lim (pn,@g> f) < i
gys dm
ll@h(g)Ilt
which proves the first conclusion. Since w h ( h )= q we also have
s
hy* dm
< q.
(1.1)
If g is in C i , since the convergence of W P , f is uniform on compact sets, we actually have
I
gys a m = lim (Pnj@,g,f ) = lim i
bnjWhg,
i
since this last limit is a constant function. The same equality is true for all functions P k g . Indeed
5-
Pgys dm
=
I
=
lim pnj@g m(h)-1
gPy* dm
=
I
gys dm
+ m(h).-l
I
gPh dm
+
i
which by ch. G , Theorem 5.2 may be written
I
&ys dm = lim
pnj@(&) + lim P,,+lg, i
i
and the last term is zero, by ch. 6 , Theorem 2.6. Inductively we get the result for all k's. Now set q) = z p k P k g ;we have
2:
but for every j , ch. 6 , Theorem 5.2 implies that Pnj@I(Pkg)
< II@gII + k)jgIL
294
RECURRENT RANDOM WALKS
CH. 9, 51
so that we can interchange summation and passage to the limit to get
I P
pys dm
=
lim f',,j@p i
By the way h was chosen, we can manage to have p tion of Fatou's lemma then gives
2 h. Yet another applica-
Combining this with eq. (1.1) yields the desired result.
+
m(h)-l Ph, Clearly if ys were the only solution of the equation Pf = f the measures E,WP, would converge vaguely to a limit, and using the duality it would not be difficult to derive a theorem of normality for random walks. That is why we shall now study the uniqueness of the solutions of the above equation. In what follows a real character of G will be a homomorphism of G into It.
Proposition 1.11. Let g be a special function a n d , f l , f 2 two functions in 6, s w h that P f , - f l = P f z - f 2 = g.
+
If for every t E G , the function ( I - t) ( f l - f z ) i s bounded, then f l - f z = k x where k is a constant and x a continuous real character of G such that PIXI - 1x1 i s Ha-integrable.
Proof. Set f = f l - f z ; plainly P ( I - t )f - ( I - t )f = 0, and since ( I - t )f is bounded it follows that for every a E G , there is a constant C(a) such that for all x in G ,
f ( 4- f ( x 4 = C ( 4 . Making x = e in this equation yields C(a) = f ( a ) - f ( e ) ; hence
f(x4
=
f ( 4 + f ( a )- f ( 4
I t remains to set k = f(e) and ~ ( x = ) f(x)!- f ( e ) to get the desired constant and character. Since the character x is measurable, it is also continuous.
PRELIMINARIES
CH. 9, $1
295
It is easily checked that Px = x, which implies Plxl 3 1x1. If the functions v = Plxl - 1x1 were not m-integrable, by the Chacon-Ornstein theorem (ch. 4, Theorems 3.2 and 3.5) the sequence 2; P k v I c ; )Pkg would converge to infinity with n. But since
1x1 < k + f l + f z for every n , n
< + + + 2
Pnlxl
fZ
fl
pk&!?
1
and we would have a contradiction. The existence of a non-trivial character demands that the group G as well as the law p satisfy some stringent conditions that we describe below.
Propositmion1.12. If there is a non-trivial real continuous character x such that PIXI - 1x1 is m-integrable, the groufi G is isomorfihic to R @ K or Z @ K , where K is a comn$act abelian grou#. Moreover i s unique u# to multifilication.
x
x
Proof. The kernel Ker x of the character is a closed subgroup of G . Let us call m , and m2 Haar measures on Ker x and G/Ker 2. For any f E &+, the integral
Ik) =
1
KerX
f k x d dml(x1)
is constant on the co-sets of Ker x, and therefore we may write
IG(PlYl-
1x1) dm =
5
d%!(%?) GjKerX
5
(PIXI KerX
1x1, ( x 2 4 dml(xl),
if ml and m 2 are suitably chosen. Now the function PIx1 on the co-sets of Ker x, because if g E Ker x,
PIXI
i
(4= 1x1 (Y%)
PWY) =
1x1 is constant
Plxl ( 4 P
and of course Ix((xg)= ( ~ [ ( x Consequently, ). we have c
(PIXI JG
1x1) dm
=
m ( K e r x)
J
G/KerX
(W - 1x1) ( x 2 ) dmz(xz).
1x1
If the last integral vanishes, then PIXI = 1x1. But this implies that is harmonic and hence constant, which is impossible since x is non-trivial. I t follows then from the hypothesis that m,(Ker x ) < co, and therefore that Ker x is a compact sub-group of G .
RECURRENT RANDOM WALKS
296
CH. 9, 51
Now the group G/Ker x is isomorphic to a sub-group of R, and is locally compact for a topology finer than the topology of R.By the structure theorem, G/Ker x has an open sub-group isomorphic to Rn @ K , where K is compact ; since x is one-to-one on G/Ker x, it follows that G/Ker x is either isomorphic to R or discrete. In the latter case, a moment’s thought proves that PIXI - 1x1 is the restriction to G/Ker x of a continuous function on R ; hence is greater than a number k > 0 on a neighbourhood of e in R. Consequently the integral of this function with respect to the Haar measure of the sub-group is not finite unless there is only a finite number of elements of the sub-group in this neighbourhood of e. But this implies that the sub-group is isomorphic to Z. The group G is therefore compactly generated, hence isomorphic to Rn @ Z @ K , with K a compact group. By the above argument it is either R @ K or Z @ K . The uniqueness property in the statement is now clear. From now on, when we deal with such a group, it will always be assumed that its Haar measure m is the product of the Lebesgue measure by the normalized Haar measure on the compact factor. Moreover, a point in G will be a pair (x, k ) , where x is in R(Z) and k is in K , and x will denote the character such that ~ ( xk ), = x. It is then known, by ch. 3 53.12, that fc xdp = 0 if the integral is meaningful, and we define o2 =
5
c
x2dp.
We have
Theorem 1.13. There i s a non-trivial real continuous character such that Plxl - 1x1 i s m-integrable if and only if the group G i s of the above type and a2<
fa.
Proof. Call the image of p by the mapping x and P the T.P. on G/Ker x associated with @.Since x2 and PIXI - 1x1 are constant on the co-sets of Ker x, it is easily seen that g2 =
jcx2 r
dp =
1
x2 d@(x),
G/Ker r
CH. 9, $1
297
PRELIMINARIES
I t suffices therefore to prove that I = crz when G = R or Z. We do it for R, the proof for Z being identical. Let and be the positive and negative parts of x. From the equality Px = we deduce that Px+ = Px- - x-, whereupon
x+ x,
x-
x+
PIXI
-
1x1 = 2(PX+ - x+) = 2(Px- - x-).
Since X+ vanishes on R- and
I
=
2
5
x-
on R, it follows that
0
P X + ( Xdx )
and using the dual random walk,
1
=
2
-OD
=
2
=
2
x+(4 %m,od4 x h,-m,ol(x) dx
1-;
Px-(x)dx,
t w
+W
I
:1 + 5
+2
--m
dx
2
--m
x-k)
m",+m,(x) dx
-2
x p ( [ x , co[) d~ - 2
1;
x p(1- co,x [ ) dx. m
We may apply Fubini's theorem in each of the two integrals, since the functions therein have constant signs, and we get the desired result.
Definition 1.14. The random walk is of tyfie I I if the conditions of the preceding theorem hold. Otherwise it is of type I . In the sequel, when we deal simultaneously with type I and type I1 random walks, the character occurring in the formulae will be understood as identically zero in the former case. If the random walk is of type I1 we shall set
x
Gf = {x E G : x(x) 3 0 } and G - = (G+)-l,
We shall need the following
Lemma 1.16. If the random walk i s of tyfie 11, for every a E G the sequence ( I - T ~ ,P,/xI ) converges boundedly to zero. Proof. Pick x and a in G ; we may write
RECURRENT RANDOM WALKS
298
CH. 9, $2
1x1
where a is equal t o ( a ) and # is a function with compact support which depends on x but is majorized by a constant which depends only on a. (The reader is invited to draw the graph of ( I - z,) when G = R.)I t follows that (1- t o ) PnlxI ( x ) = a pn(G+)- pn(G-) P n $J(x)-
1x1
+
Since cr2 < co, we may use the central limit theorem to the effect that lim pn(G+)= lim pn(G-).=
4,
n
n
and by ch. 6, Theorem 2.11 we have limn P, $J(x) = 0, which completes the proof.
Exercise 1.16. Extend all the results of this section t o non-abelian groups. [Hint:In Proposition 1.6 use left translations instead of right translations.] 2. Normality and potential kernels In this section we prove a first result of normality, from which we derive the construction of convolution kernels that are potential kernels in the sense of ch. 8. We start by showing that the function y s in Proposition 1.9 does not depend on s.
Thcorcm 2.1. There i s a uniformly continuous positive function y such that Py - y = m(h)-l Ph, and such that, for every f in C,,
uniformly on compact sets.
If we want to emphasize the dependence of y on h we shall write y h . Proof. Let s = { n j }and s' = {n;}be two sequences such that the sequences {WP,} and {WP,,j} converge vaguely t o ys @ m and y8' @ m with ys # y"'. By Propositions 1.9-1.11, there are two constants CI and /3 such that ys - ys' = a + Px, where vanishes if the random walk is of type I. By Proposition 1.10, we have
x
I
hya dm
=
I
hy8' dm
= (1 - m(h)/m(h));
C H . 9, 92
NORMALITY AND POTENTIAL KERNELS
299
hence
since h is symmetric, Xh dm = 0 and therefore u = 0, and the proof is completed for type I random walks. Now let X be type I1 and g a function in C$ such that m(g)= 1 ; then
Px
=
lim(WPnJg- WP,,,g). J
By taking sub-sequences, we may assume that n j Ker from the equality
x;
< n;. Let a be in G outside
we derive that n';
By ch. 6, Proposition 5.7, the sequence on the right is bounded, and since is nz-integrable, we have
$(XI
1x1
and this limit is zero, by Lemma 1.15. I t follows that p = 0, which completes the proof.
Remark. Of course, the symmetric result holds for the dual chain, and we call 9 the corresponding function. I t is easily seen that + ( x ) = y(x-1). The following result of normality will be sharpened a t the end of the section.
Theorem 2.2. For every positive function f majorized by a multiple of h, we have
300
RECURRENT RANDOM WALKS
f f dm,
lim
n
I
pn@f= y f dm.
n
Proof. I t suffices to prove the result for the dual chain and for f By Proposition 1.8, we may find a sequence n j such that lim P,@f
=
CH. 9, $2
< 12.
Iim Pn@f, n
nj
and this limit is a constant function. Let g be in C z , with m(g) = 1 ; passing to the limit in the equality
(pnjJ@f, g>
=
( f , WPnjg),
we get, using Fatou’s lemma, that
hP,@f 3 9%
But h
I I
yfdm.
- f is also positive and less than h, and therefore in^ p n @ ( h- f ) 3 y ( h - f ) dm. n
Since PnWh
=
$ y h dm, it follows that n
which completes the proof. We turn to the construction of a potential kernel.
Theorem 2.3. The kernel A
=
I
+ W - I @ f m-y @ m
is a convolution kernel which has the following $ro$erties : (i) it maps special functions into functions finite and bounded above, and maps charges into bounded functions; (ii) iff i s a charge such that I f 1 h, then
<
n
lim n
2 PKf = Af
and lim P,Af
O
(iii) for every special function f , we have
n
= 0;
30 1
NORMALITY AND POTENTIAL KERNELS
CH. 9, 52
( I - P ) Af = f . Proof. If f is a charge, it is easily seen that n
21 If, in addition,
Pkf
=
wf - p n + l w f -
I f 1 < It, the preceding result yields U
limxPkf= f n
o
+ Wf -
I
f f d m = Af,
and it is obvious that lim, PnAf = 0. We proceed to show that A is a convolution kernel. Let f be a positive function majorized by a multiple of k and such that m ( f )= 1. The function v = f - m(h)-1 h satisfies the above condition, and thus, using Proposition 1.10, we have n
lim n
2 Pkp, = Wp, -
f f dm
+ m(h)-2 (1 - m(h)).
1
On the other hand,
Letting n tend to infinity, it follows that
-1
Since
f f dm
(k W v ) = (W v)
+ ~ ( l t ) (1- ~- m(h)).
= 0,
by setting
we get Af
=
(I
+ W - y @ m - 1 @ fm)f
r n
= lim n
I2 o
1
Pkf - Cn m(f)
RECURRENT RANDOM WALKS
302
CH. 9, $2
which proves that A is a convolution kernel. Finally, A satisfies (iii) because, for special f , since y and Py are finite, Af and P A f are also finite. Thus
(I - P) A f
=
f - Pf
+ ( I - P ) W f - y m ( f )+ Pr m ( f ) ,
and sinc.e
( I - P ) Wf we get
=
Pf - m(h)-' Ph m(f) and Py = y
+ m(h)-I Ph,
( I - P ) Af = f . Of course the same results are true for the dual chain, and we have the following
Corollary 2.4. The kernel
i s a convolution kernel in duality with A , and has the same property with res@ect to 8 as A has with respect to X .
Proof. Straightforward.
Definition 2.6. If f is special we say that Af is the potential of f .
-
We shall denote by a the Radon measure such that A ( x , ) = a * E,. If B is the image of a by the mapping x -+ x-l, we also have A(%, ) = B * E, and
-
We have shown that the kernel A provides us with a solution to the Poisson equation with special second member. We now turn to the problem of knowing to what extent it is the only kernel with such a property.
Theorem 2.6. Let A' be a convolution kernel satisfying properties (i) through (iii) in Theorem 2.3; then there i s a constant c such that A' = A c @ m. Let A' be a convolution kernel satisfying only (i) and (iii) in Theorem 2.3; then if X i s ty$e I , there i s a constant c such that A' = A c @ m, and if X i s type 11, there are two constants c and d such that
+
+
NORMALITY AND POTENTIAL KERNELS
CH. 9, 52
A’=A+c@m+d(l
303
@xm--@m).
Remarks. The kernel 1 €3 xm - x €3 m is a convolution kernel, since it may be written (xm) * E,. In order that (i) be satisfied, the constant d must lie in the interval [- c2, a-2], as will be shown in the next section. Proof. If A’ is a convolution kernel satisfying (i), then for every t in G and f special, (I - t) A ’ / = A ‘ ( I - t) f is bounded. If A’ also satisfies (iii), it follows from Proposition 1.11 that Af - A’f
= a(/)
-t x b(f),
where a ( / ) and b ( f ) are constants. If we set x = e in this equality we get a(/) =
I
I
A ( i dY) f(Y) - A ’ k , dY)
/(YIP
thus a ( / ) = J / d v l for a measure vl. If, in the same way, we set x = x,,, where x,, is not in Ker x, we get that b ( f ) = J / dvz for a measure vz. Thus A‘ = A 1 @ v 1 x @ vz. But since A and A‘ are convolution kernels, for every a we must have
+
+
1 @ v1ra
+ x €3
1 €3
vzra
~1
+
tax
€3 vZ,
that is
7 @ vita
+ x €3
VZta
= 1@
3’1
+ x(a) 8 vz + x €3
VZ;
this implies that for every a in G,vZta= vz, so that vz = dm for some constant d . I t follows that for every a, vlta =
v1
+ dx(a) m .
+
Setting v 1 = v 1 dxm we get at once that v1 = cnz - dxm. We thus have
A’
=
A
vita = vi, and therefore that
+ c ( 1 €3 m) + d ( x €3 m - 1 €3 xm),
and the proof is now easily completed.
As a result, we see that if we take another function It, the corresponding kernel will differ from A only by a multiple of 1 @ m. We end this section by strengthening the normality theorem (2.2). One might hope that random walks were normal in the sense of ch. 8 $3; actually
RECURRENT RANDOM WALKS
304
CH. 9, 92
this is not always true, and we shall prove a slightly less general result. We need some preliminary results and notation. We define r
?'(a) =
m(h)-'
1
ht,ydm;
the function y' is positive, uniformly continuous and unbounded. Furthermore, from f ( x ) = y(x-') it is easily derived that f ' ( x ) = y'(x-1).
Lemma 2.7. For every x and a i n G ,
Proof. Choose f in C$ such that m(f) = 1; then lim t,WP,f = lim WsanPn(taf),
= n
U
so that by ch. 6, Corollary 5.5,
ray = y
- m(h)-l W(tah) + m(h)-l lim(h,
W,a,P,(t,f)).
9l
But (It, WIQhP,(t,f)) -= (
R a w 3 4 8 f),
and by the argument in Proposition 1.10 we could, by choosing 2 ty'h, find a subsequence {n,} such that {Pnji?(ti'h)} converges boundedly to J y t z l h dm. Consequently lim(h, WlahPn(t.f)) = n
which completes the proof.
Proposition 2.8. For every x and a in G ,
Proof. The first part follows immediately from the preceding lemma, and implies in turn that
which i s the second conclusion in the statement.
CH. 9. 52
NORMALITY AND POTENTIAL KERNELS
305
Lemma 2.9. The function y - y' is bozcnded.
Proof. Choose a function f in Cg such that m(f) = 1 and integrate the equality in Lemma 2.7 with respect to the measure f m . We get
5
( f , m(h)-l W(tah))= f y dm -
5
tar/
dm
+~ ' ( 4 ;
since the left-hand side is a bounded function of a, it remains to prove that - fTayf dm is bounded; but this is equal to
y(a)
1
f ( x ) (1- Tz) r ( a ) dm(x),
so that the result follows from the fact that ( I - tz)y(a) is bounded uniformly for x in a compact set.
The following result is the key technical one.
Proposition 2.10. There exists a constant C such that, for every a in G,
< y + 9 + C.
m(h)-l W(tah)
Proof. In view of the two preceding results, there is a constant C' such that
in view of Proposition 2.8; hence
and another application of the preceding lemma yields the desired conclusion. We turn to our main result on normality.
Theorem 2.11. For every sfiecial and co-special function f ,
n
306
CH. 9, $2
RECURRENT RANDOM WALKS
Proof. We first observe that by integrating the inequality of the preceding with respect to ,iin(da), we get proposition, written for
x,
< + +C
m(hl-1 WPJZ y
for every integer n. The left-hand side tends to f as n tends to infinity and a special and co-special function f is integrable by the measure ( y 9 C ) m ; we may thus apply the Lebesgue theorem to obtain
+ +
lim(P,Wf, ~n(h)-lh )
=
lim(f, nt(h)-l
WP,h)
=
We now conclude the proof by using Proposition 1.8.
If the special functions are also co-special (in particular if ;l = h), then X is normal in the sense of ch. 8 $3. It will also be shown in $4that all type I1 random walks are normal in the sense of ch. 8 $3. With the obvious changes we may now apply the results of ch. 8 $92 and 3. For instance, if K is .a special and co-special set, then P,P, has a limit AK which is the equilibrium measure of K with respect to A . In the same way, we may define the Robin's constant. The reader will have no difficulty in translating the results of ch. 8 to the present situation. Exercise 2.12. Prove that lim WP,f(x) tion f. [Hint: Use ch. 6, Exercise 5.17.1
=
y ( x ) m(f) for every special func-
Exercise 2.13. Prove that y(x-l) = f(x). Exercise 2.14. If f is in C$, then Af is continuous. Exercise 2.15. (1) Let G be discrete, and prove that the kernel N
A(%,Y) = lim N-+m
2 ( P n ( x , Y) - P n ( e t e ) ) 0
<
exists and is finite and that A ( x , y) 0 for every pair (x,y) of points in G . Furthermore, if f is a charge with finite support, then N
lim N
hence A is a potential kernel.
2 P, f(x) = A f ( x ); O
CH. 9, $2
307
NORMALITY AND POTENTIAL KERNELS
(3) Set, according to Definitions 2.5, a(.) = A ( e , x ) . Prove that a(x) < 0 for every x # e and that a(x y) 3 a(%) a(y). ( 4 ) Prove that f E bb, is special and co-special if and only if
+
+
Prove that if f is special and symmetric it is also co-special. (5) For any pair (x, y) of points of G, the function a satisfies the following two inequalities:
+ 4%)W, d a(yx) 4x-l) + a(%)4 y ) .
4 - l ) d a(xy) at-') a(%)a(+)
[Hint:Use the maximum principle of ch. 8 $2.9.1 Exercise 2.16. (1) Prove the following generalization of the non-positivity property in Exercise 2.15. There is a positive bounded measure a1 such that a - a1 is absolutely continuous with respect to nz, and with a density $J bounded above. [Hint:See ch. 5 $5.1.1 (2) Denote by P, the singular part of , u n ; then prove that the singular parts of the kernels A and W are the convolution kernels E, * Pn and Ex
2:
*2 1" Pn.
Exercise 2.17. For every special and co-special function h', there is a function y h f such that limn W,.P,f = m(f)y' for every f in C,. Check the properties of Y h , . Exercise 2.18. If g E CL n %, there exists a function L , such that limn U,P,f = L , m(f) for every f in C K and
n
Exercise 2.19. Prove that the kernel A determines p uniquely. [ H i n t : Prove that A determines II,[ for any compact set K and prove
RECURRENT RANDOM WALKS
308
CH. 9, 33
that P = limnlZK,,, where K , is an increasing sequence of compact sets whose interiors cover E. The reader should also refer to ch. 10 $3.1 Exercise 2.20 (Fourier criterion for recurrence). Let G be countable and retain the notation of ch. 3 $4. The random walk X is assumed to be recurrent. (1) Prove that for every x in G , the function
Y ..+2 0 - Re r!x))/Re(l - v(Y))-l is integrable on I'. We will call its integral I(%). [Hint:For every x in G , there is an integer n = % ( x ) such that pn(x) = c > 0; then 1 - Re I&)" 2 c ( l - Re p(y)).] (2) With the notation of Exercise 2.15 we have - I ( % )= a(x)
(3) Prove that
B
=
5
r
+ a(-
x ) = G(6'(x,x ) .
Re(1 - ~ ( y ) ) - ' d y =
+
a).
[Hint: If the integral were finite then we would have - I ( x ) and this is impossible since it may be shown that lim G(e'(x, x )
=
< B < a,
+ co.
%+A
For the latter point see, for instance, $4 below.] (4) Try to extend the above result to arbitrary abelian groups. Exercise 2.21. (1) Compute the potential kernel of the Bernoulli random walk on Z (p(1) = p ( - 1) = 2-l) by means of Exercise 2.20. Exercise 2.22. Extend the results of this section to non-abelian groups. 3. Martin boundary As has been shown in ch. 8 $4, the boundary points of G for X are in one-to-one correspondence with the cluster points for the vague topology of the sets {W(x,* ), x E G } .We are therefore going to study these cluster points, and we shall find that there are either one or two such points, according to whether the random walk is of type I or 11. Our first result parallels Proposition 1.8.
MARTIN BOUNDARY
CH. 9, 53
309
Proposition 3.1. For every special function f s w h that lim,,,Pf(x) particular f E C o ) , and every x in G , lim (W /(xu)
= 0 (in
- W / ( a ) )= 0.
4 4 A
va = taw/ and B = IlWfll. Let U be an ultrafilter finer than the filter of neighbourhoods of d, and define
Proof. Set
L(x) = lim, va(x), Since fl
pnwf =
wf - 2 P k + l f 0
fl
+ m(h)-l 2
pk+lh
m(/)i
0
for fixed n we have
L
lim (Pnp, - v0) = Iim z,
-
,-+A
a+A
owing to the assumed property of f . If n is sufficiently large, pn = g,m in L\(m), and knva(x)
I
n
1
2 Pkflf+ m(h)-' 2 Pk+$ d f ) = 0, 0
+ ,On, where g , is a non-zero function
- v a ( Y x ) gn(Y) m(dY)l< BIIP~II.
Pick a Bore1 function $ within the equivalence class of the limit of va along
u in a ( L m , L1) ; clearly 11.(x)
j
- $ ( Y 4 &(Y) m(dY)l< ~IIPnll*
and consequently I L ( x ) - Pn $(x)l
< 2BJIPnll.
By letting n tend to infinity, it follows that L = limn P,+, hence that L is a constant function, and that limu (pa(%) - pa(e))= 0 ; the desired conclusion is then easily obtained. Remark. The reader will notice without difficulty that the condition on f is also necessary.
Proposition 3.2. Every segzcence converging to A contains u sub-sequence (a,} such that, for every f in CK,the sequence { W ( t , , f ) } converges uniformly on
RECURRENT RANDOM WALKS
310
CH. 9, 93
r is a continuous function.
compact sets to r(x)m ( f ) where ,
Proof. By Lemma 1.6, for any x in G the mappings
form a vaguely relatively compact set of positive Radon measures. Next, by Proposition 1.6, and by the Cantor diagonal process, every sequence converging to d contains a sub-sequence {a,,} such that the sequence { W ( t a n f i ) ) converges uniformly on compact sets for every function f i in a denumerable dense subset of CK. As a result, the measures ,u& convqge vaguely to a measure vx, and for every f E C, the function x v x ( f ) is continuous. Let f , g be in C K and b in G . We have --+
y2(f
Wan(/ - sf))g dm
- t b f ) g ( 4 d 4 ~=)
= lim n
1
dm
- Wg(b%,'x))
f ( x ) (Wg(a;'x)
= lim n
-
to,,(/ t b fJ@g )
dm(x) = 0,
l
in view of Proposition 3.1. I t follows that vx is translation invariant, hence that v x = F(x)m, where is continuous.
r
As in the preceding sections, we observe that if I' did not depend on the sub-sequence we would obtain a limit theorem; we therefore study the function
r.
r = y + a + Px. Proof. It is easily seen from the definition of F that Pr < I' + m(h)-l Ph. Proposition 3.3. There are two constants a and ,8 such that
Define C =
I' - y ; if
/E C i
and m(f) = 1, then
C ( 4 = lim W(r,,, - P,) f ( x ) , n
so that for b E G , C ( x ) - C ( b x ) = lim(1 - t b ) W(t,,
- Pn)f(x)
n
=
lim(1 - tb)IVr,h(~an - P,) t,f(e). n
CH. 9, 93
MARTIN BOUNDARY
311
But (tan- P,) z,f is a charge, and if g is a charge, then (1- tb)(W@ - w h ) g = 0 by ch. 6, Corollary 5.5; consequently, lim(1 - tb)Wr,h(to, - P,) t,f(e)
=
lim(1 - t b ) W(t,, - Pn) t,f(e), n
U
and since m(t,f) is still equal to 1, this limit is equal to C(e) - C ( b ) .It follows that C = a f , where f is a real character. AS I f ) .l' y a, it follows that PIf1 < 00 and
< + +
+
+
Pnlfl
< r + y + 2m(h)-l (Ph + - . - + J',h),
hence that lim n-lP,(f( = 0. Next, if
Pf-f= then
Pnf = f
+ np
I
fdp=P,
and n-IP,,f
by letting n tend to infinity it follows that
Pr
=
p
= n-lf f =
p;
0, hence Pf
=f
and
+
.l' m(h)-' P h ;
as in Proposition 1.11 we may now prove that Plfl The proof is then complete.
121 is m-integrable.
Proposition 3.4. From every sequence converging to A , we may select a subsequence {a,,} such that the measuyes W(a,, ) converge vaguely to (+ Px) m, where ,8 i s a constant.
-
+
-
Proof. Since the set {W(x, ), x E: G } is vaguely relatively compact it is clear that we may find a sub-sequence {a,,} such that the sequence W(a,, ) converges. In view of Proposition 3.1, for every f in C K , Wf(a,x) converges then to a constant C. For g in C , we have therefore C m(g)
-
1, g)
=
lim(wf(a,
=
lim(f, W G ' g ) )
*
s + +
= m(g)
s
(P + a + Px) f dm,
and it follows that C = (f cc fix) f dm. Now if f is Co and if Wf(a,) converges to (f a property is true for Pf, because by ch. 6, Theorem 5.2,
s + + Px) f dm, the same
312
CH. 9, $3
RECURRENT RANDOM WALKS
w P f ( a n ) = Wf(an) - Pf(an)
+ m(h)-’
J
hPf dm,
and thus the right-hand side has a limit which is equal to
2
If v E C i , this property is still true for y = 2-nP,v, because the series converge uniformly, and we can manage to choose v so that q 2 h. By an argument of lower semi-continuity it follows that
2 2-nWP,v
5 J 5
lim J+‘(q- h) (an) 3 (1; and
+ a + Px)
(p1
- h) dm
lim Wh(a,) 3 (9 + 0: + Px) h dm, so that lim Wh(a,) =
s
(1; + a + Px) h dm.
Since Wh = s f h dm and Xh dm = 0, it follows that a = 0, and the proof is thus complete. Actually, using lower-semi-continuity we can show a bit more, namely that W f ( a n converges ) to (1; PX)f dm for every continuous function dominated by a multiple of h.
+
Definition 3.6. If the random walk is of type 11, we say that x tends to f co if ~ ( x tends ) to f co. The following theorem is the main result of this section. It states that for type I random walks, the Martin compactification is obtained by adjoining one point to the group G , whereas for type I1 random walks one must adjoin two points, one a t each “end” (in the obvious sense) of the group G . In view of the remark below ch. 8, Proposition 4.6, the compactification thus obtained is the same as if W had been associated with a funktion h E C;.
MARTIN BOUNDARY
CH. 9, 53
313
Theorem 3.6. If the random walk i s of tyee I , the measures W ( x ,* ) converge vaguely to f m when x tends to A . If the random walk i s of tyee 11, the measures W(x , ) converge to f m f c 2 X m when x tends to f co.
-
Proof. For type I random walks, there is nothing to prove, in view of the preceding result. Consider a type I1 random walk, and let {a,} and {a:} be two sequences such that {W(a,, ) } and {W(a:, ) } converge vaguely. By the above proposition, there is a constant P such that
-
-
lim(W(a,, Pick f in
*
-
- ~ ( a : ,1) = PXm.
Cx and b 4 Ker x ; on account of Proposition 3.1, W ( I - .b)
- w(l - t b ) f(xa:)
converges boundedly to
Now, ( I - t b ) f is a charge with compact support, so that by $2,
W ( I - zb)f
-
5
N
f ( Z - t b ) f dm
=
lim N
2 P,(I 1
- tb)f ,
and the convergence is bounded ; consequently (21,
W(I - t b ) f ( *
N
a,)
- W(I- Td f(- 4)
P,(I - tb)/(
-
N
a,)
-
1
P,(I - tb)f ( * a,)
and, because of Proposition 1.15, this limit is equal t o
314
RECURRENT RANDOM WALKS
=
- (Pf, (I - tb')
Ixl(
*
CH. 9, $3
- (I - t:')
a;')
*
a:-*)).
(3.2)
From the form of the function ( I - t;') 1x1 which was described in $1, it follows that if {a,} and {a:} either both converge to 03 or both converge t o - 03, the above limit is zero. As a result p = 0, and therefore the measures W ( x ,* ) have a limit when x converges either to co or to - co. The functions W f thus have a limit a t each end of the group G, and since P, W converges to f m and
+
+
lim P,1,+
= lim P,,lG- =
-
+
4,
+
it follows that if W ( x , ) converges to Pm 1xm as x 03, then W ( x ,* ) converges to Pm - 1xm as x --r - 03. It remains to compute 1. Let {a,} be a sequence converging to 03 and {a;) a sequence converging to - co. If we resume the above calculations it is seen that ,8 = 21 and that the limit in eq. (3.2) is equal to 2 m(f)~ ( b )Comparing . with eq, (3.1), it follows that 1 m(v) = 1 ; that is, 1 = a-2, and the proof is complete. --+
+
As remarked a t the end of the proof of Proposition 3.4, the above convergences hold not only for functions in C, but also for continuous functions majorized by a multiple of h. Actually, the scope of validity of the preceding theorem may be still further enlarged by the use of the same device as in Theorem 2.11.
Theorem 3.7. If f is both special and Go-special, and if 1ima+*Pf(a) = 0, then
I
lim Wf(x)= f f dm x+A
if X is type I , and x+fm
if X is type II.
Proof. Let us write down the proof for type I walks, leaving to the reader the obvious changes needed for type 11. Let g be a function in C i with m(g) = 1 ; from Proposition 3.2 and Theorem 3.6, it is easily derived that 1ima.+*W(tag) = y , and by Proposition 2.1s: there is a constant D such that W(tag) D(Y P C).
<
+ +
CH. 9, 54
RENEWAL THEORY
316
Now on account of Proposition 3.1, from every sequence converging to A we may select a sub-sequence {a,} such that Wf(a,x) converges boundedly to a constant p. Then r
p = lirn(t,,Wf, g) = lim(f, @(ra;lg)) = n
n
] gf dm,
by applying the preceding remarks to the dual chain. The above results may also be stated for the potential kernel A . Corollary 3.8. I f f is u charge such that I f ) i s co-special and lima+, Plfl ( a ) = 0, then lim A f ( x ) = 0 %-+A
tor type I random walks, and lim A f ( x ) x-+*
m
for type 11 random walks.
Proof. Straightforward. Exercise 3.9. If g E C$
n 4,then for type I random walks,
J
lim U,f(x) = L,f dm x+A
(Lg was defined in Exercise 2.18). For type I1 random walks there exist L b and two functions Lf such that 2L, = L:
+
I
lim U,f(x) = Lff dm. X+fW
4. Renewal theory
In the preceding section, we have studied the asymptotic behaviour of potentials Af when f is a charge. We want now to fulfil the same task when f 3 0. Clearly this amounts to studying the asymptotic behaviour of functions y . Since these functions are positive and sub-harmonic, €hey are unbounded; therefore when x tends to A , y ( x ) has always a, & a cluster point. The problem is to find out whether it is the only one, and in that case to compute the speed of convergence of y to co. Wi? &all find that
+
+
CH. 9, $4
RECURRENT RANDOM WALKS
316
type I and type I1 random walks have quite different features with regard to this problem, and require different proofs. We start, however, with some applications of the preceding section which concern both types.
Proposition 4.1. Zf f i s in C i and m(f) = 1, then for any a in G , lim(Z - ta)Af(x) = lim(Z - ta)y ( x ) x+A
=
0
x
for type I random walks, and lim(I - 7,) Af(x) = - lim(I - ta)y ( x ) x+&m
=
f c2 x(a)
X--+*CC
for type I T random walks. Moreover, the convergence i s uniform for a in a compact set.
Proof. Since ( I - t a ) A f ( x ) = (1- t a ) f ( x )
+ (1-
ta)
w f ( x ) - (1- ta) Y(x),
it follows from Theorem 3.6 that the two limits are opposite. Indeed for type I1 random walks x and ax converge to the same end of the group whatever a is. On the other hand, since A is a convolution kernel, we have (I - ta)A f = A ( I - ta)f , and therefore by Corollary 3.8, the limit is zero in the former case, and
I
f a-2 x ( Z - ta)f dm = f a-2 x(a) in the latter. The last conclusion follows from the uniform continuity of functions y .
Corollary 4.2. Let a be a point in G such that an converges to A ; then lim n-ly(a")
=
0
)I
for type Z random walks, and lim n-'y(a")
= a-21~1(a)
U
for type 11 random walks. Proof. By the preceding result, the differences y(a") - y(a"-l) converge to
RENEWAL THEORY
CH. 9, 54
317
the assigned limits; since Cesaro convergence is weaker than ordinary convergence, the proof is .complete. For a while we shall deal only with type I1 random walks. The asymptotic behaviour of y is then completely settled by the following
Theorem 4.3. For type 11 random walks,
Proof. When a is in Ker x, the functions ( I - z,) y are uniformly bounded; it suffices thus to prove the result when G is either R or Z. Since the function y’ is subadditive (Proposition 2.8), it is a classical result (see ch. 4 56.2) that y ’ ( x ) / ( x I( x ) has a limit whenever 1x1 ( x ) tends to infinity, or equivalently x tends t o A . As the difference y’ - y is bounded (Lemma 2.9), it follows that y ( x ) / l x I ( x ) also has a limit. This limit is u - ~ since , by Corollary4.2, y(na)/lxl (nu) tends t o c r 2 as n -+ 03. l h i s result may be restated in terms of potentials. Corollary 4.4. Let f be special, then lim A f ( x ) / l ~( Ix ) = - a-2 m ( f ) . %-+A
Proof. Obvious. This result enables us t o complete Theorem 2.6 by stating
Proposition 4.5. The constant d which appears in Theorem 2.6 must lie in the interval [a-21.
+
+
Proof. Consider the kernel A‘ = A c ( l @ m) d(7 @ xm - x @ m) and let f be a special function; in the right-hand side of the equality A’f = f
+ FVf + c m ( f ) -
5
f f dm
+d
5
xf
dm - (y
+ dx) m ( f ) ,
all the summands are bounded but the last one. It is now plain that A’f cannot be bounded from above unless d lies in the interval [- u - ~ a-7. ,
RECURRENT RANDOM WALKS
318
CH. 9, 54
We now turn to type I random walks, for which the situation as well as the proofs are much more intricate.
Proposition 4.6. lim(y(x)
+ y(x-1))
= lim(y‘(x)
x+A
+
y’(x-1))
=
+
00.
x-+A
Proof. Since y - y’ is bounded, it suffices to prove the result for y’. If the result were false, there would exist a sequence {a,} and a constant C such that y‘(a,) y’(u;’) C < 03. But as y‘ is subadditive for every x in G . we have
+
<
y‘(x) = y’(xana,’)
< y’(xan) + y’(a,’)t
and since y’(xa,) - y’(an) converges to zero as n -+ CO (cf. Proposition 4.1) the function y‘ would be bounded, which is a contradiction.
Remark. The above result may also be stated as lim(y(x)
+ $+))
= lim(y’(x)
+ y’@-1)) = +
03,
and also if f is special as lim(Af(x)
+ Af(x-l)) = lim(df(x) + Af(x)) = -
03.
We now turn to the main result for type I random walks. The proof is somewhat similar to a proof given for transient random walks.
+
Theorem 4.7. If the fzcnction y does not converge to 00, there is one and only one finite number L for Zprhich there exist seqzcences {x,} such that limn y(x,) = L, and then limn y(x,’) = co.
+
Proof. The last conclusion follows at once from Proposition 4.6. To prove the first one, consider a function f in C i with support V , and set g~ = - Af. For any a in G , we have (t,,- I ) g~ =
(I
+ W ) ( I - ta)f + constant.
Since ( I - ta)fis a charge, we may apply the semi-complete maximum principle (ch. 8 $2) to infer that there exists E E V such that, for all x in G ,
P W - d x ) < dE4 - dt),
CH. 9, $4
RENEWAL THEORY
319
Repeated replacements of x by xa yield
and by summing up and dividing by n,
<
+
Finally, ~ ( l )n-' ~ ( x ) q ( [ a ) , and since n is arbitrary, for every a E G there exists 6in V such that ~(6) ~ ( [ a )hence , such that inf,, ~ ( x ) ~ ( [ a ) . By Proposition 4.1, we know that y ( a ) - a(&) converges uniformly in 6 E V t o zero whenever a -+ A, and we may therefore conclude that
<
<
lim ~
( a ) inf ~ ( x ) ,
a+d
X€V
Now let L be a finite cluster point for g~and {a:} a sequence converging to A and such that L = limn I & , ) . We may select from { u i } a sub-sequence {a,} such that lim v(&z~'a,) = L
uniformly if 5' E V and j
< n,
n
lim y([a;'a,)
=
+ co.
n
This may be done recursively in the following way. The points a',. . . , being already chosen, we choose a j + l in such a way that
dtaL'ai+J - da,+l)
< 2-'
<
for every 5' E V and k j ; this is possible on account of Proposition 4.1. But we may also choose a,+' in order that fp(&zTllak)2 2 L for every k i and 6E V ; indeed, by Proposition 4.6, fp(ai-'@k) converges to co, and by Proposition 4.1, (I - t Cq(ui-'uk) ) converges to zero uniformly for 6 in a compact set. Let us next set "
+
<
This function is also of the form - A f , and as such satisfies the inequality (4.1) with the appropriate set V . On the other hand it is plain from Proposition 4.1 that
CH. 9, 94
RECURRENT RANDOM WALKS
320
lim @(x) = !iIJ +). x+A
x+A
I t follows that
and a fortiori n
inf{q(ta;'aj) : t E V , 1
n-l
*=l
< k f n} < lirn q,(x). x-+A
As soon as i is sufficiently large, the above infimum is assumed for a number k i and thus is near L ; as a result, we obtain L l h z + Ap(x), and consequently L = l h 3 + A~ ( x ) I.t follows that L is the only finite cluster point.
<
<
Definition 4.8. A type I random walk is said to be of type Ib if y has a finite cluster point; it is otherwise said to be of type l a . There cannot be type I b random walks on any recurrent abelian group, and our next goal is to give the relevant classification of groups. The process is somewhat parallel to the proof of the renewal theorem for transient random walks. In the following proofs we use the additive notation for the operation in G .
Lemma 4.9. I f G has an open, non-compact, compactly generated sub-group G I such that GIG, is infinde, then every random walk is of type Ia. Proof. We use the notation of the proof of Theorem 4.7, and assume that X is type Ib. By Theorem 4.7, and the same device as in the proof of ch. 5, Proposition 4.1,we may find a sequence (2,) of points in G such that the sets z,G1 are pairwise disjoint and such that limn p(zn) = L < 03. Since G I is compactly generated but not compact, we can choose x E G1 such that nx + A as n +to, and by Proposition 4.1 and Theorem 4.7, we can suppose that limn ~ ( n x=) co. Arguing as in ch. 5, Proposition 4.1, it is easily seen that for n sufficiently large, there is an integer k , > 0 such that
+
+ k n x ) < L + 1,
~ ( z n
Now z , get
+ kx
d uniformly in k as n
-+
+ + 1)
~ ( z n
(kn
+ co,
X)
ZL
+ 1.
so that by Proposition 4.1 we
CH. 9, 54
RENEWAL THEORY
+
lim ~ ( z , k,x) = L U
321
+ 1,
which contradicts Theorem 4.7.
Corollary 4.10. I f G has a closed sub-group isomorfJhicto R d * @ Zdm@ K,where K is a compact group and dl dz 2 2, then all random walks are type l a .
+
+
+
Proof. We choose x and y in G such that nx ky .--c co as n k -+ 03, and use the proof of Lemma 4.9. One could also argue as in ch. 6, Corollary 4.2.
Lemma 4.11. If every element on G i s of type l a .
of G
is compact, then every Harris random walk
Proof. We suppose that X is of type Ib, and we denote by H an open compact sub-group containing the support of f (we recall that q~ = - A f ) . Then G I H is infinite, and arguing as in Lemma 4.9, we may pick a sequence {z,} such that zo = e and the co-sets z,H are disjoint, and lim, p(zn) = L < co. By Theorem 4.7, lim, y ( - z,) = cx). By the hypothesis, for every element z in G , the sequence {nz},>, intersects H , so that we can set'for every n
+
+
h, For n =
+
=
inf{m
1: mz,
E H}.
>, 1, h, > 1, and since (h, - 1) z, E z;'H, we have limn q ~ ( ( h, 1) z,) Thus h, > 2 for n sufficiently large, and we can choose k , to be
00.
the largest positive integer less than h, - 1 such that ip(k,z,) Then for n sufficiently large, qJ(knZn)
d 2L + 2 < d ( k , + 1) 2,).
Now, by using the identities of ch. 8, Theorem 2.1, we can write
< 2 L + 2.
RECURRENT RANDOM WALKS
322
CH. 9, $4
Since the support of f is contained in H , we have GHf(x) = 0 ; on the other hand, TuH= C H in e, since e E H . Finally, owing to the definitions of P , and GH, we get, after cancellation,
q(’
+Y)- v(’) - p(Y) = -
cH(TWf)
1(’ - P H A
+
(Tvf)(x) P H A f ( x )
+ A ( t v f ) (e)*
We may suppose that A f is negative, since this amounts to adding to A a multiple of 7 @J m. On the other hand, as a consequence of Proposition 4.1, we have lim[A(Tvf)
- PHA(tvf)
=
Y-+A
uniformly in x. We may thus conclude that, uniformly in x E G , P(X
where O(y) -+ 0 as y
--).
+ Y) - $44- d Y ) < O ( Y )
d . Letting y
P((k?l -t 1) 2),
where 0,(1)
L
-+
0 as n
.+ co.
= z,
- Ql(k,zn)
and x
=
k,,z,, we get that
< dzw) + 0,(1),
For n sufficiently large, we thus have
+ 1 < y(k,z,) < 2L + 2 < pl((k, + 1) < 3L + 3. 2 ),
Since either (knzn>or {(k,, + 1) 2,) has a sub-sequence which converges to infinity, we have a contradiction with Theorem 4.7.
Theorem 4.12. If the random walk i s of type Zb, then G i s isomorphic either to R @ K or Z @ K , where K i s a conzfiact grou+. Furthermore, there i s a finite number L such that (with the obvious meaning for f oc)) lim A f ( x ) = L m ( f ) , x++w
lim A&)
= - oc)
x-+--m
or
for every function f
EC
i.
Proof. The first part is shown in the same way as ch. 5, Theorem 4.4. The second part then follows easily from the above results and the arguments in ch. 5 , Theorem 3.1.
CH. 9, 54
RENEWAL THEORY
323
Unfortunately we do not know of a characterization of the laws of type I b walks. Exercise 4.16, however, gives some information on this subject.
Exercise 4.13. (1) Prove that type I1 random walks are normal in the sense of ch. 8 $3. [Hint: Observe that in Proposition 2.10, f may be replaced by a multiple of Y.1 (2) Derive that the resolvent of linear Brownian motion is normal in the sense of ch. 8 $3. Exercise 4.14. Prove that with the notation of Exercise 2.15, if X is a recurrent random walk on Z,then a(.) = Ax for all x 0 if and only if X is left continuous (see ch. 3, Exercise 3.13)and type I1 with A = - ra. Compute the potential kernel of the Bernoulli random walk (p(1) = p ( - 1) = 2-l).
<
Exercise 4.16. For a type I random walk on R or Z, prove that limz+dx - l y ( x ) = 0. Exercise 4.16. (1) All recurrent random walks on Z have the property that a(.) < 0 for x # 0 with one single exception: left- or right-continuous random walks of type I. In the left-continuous case, a ( x) = 0 for x 0 and a(x) < 0 for x > 0. The reader is invited to return to ch. 8, Exercise 4.18. (2) Prove that if a random walk on Z is type I, and p(x) = 0 for all but a finite number of positive entries x , then there is a finite cluster point for y. (3) Use the random walks described in (1) to prove that there exist random walks which are not normal in the sense of ch. 8 53.
<
CHAPTER 10
CONSTRUCTION OF MARKOV CHAINS AND RESOLVENTS
We have seen that the potential kernels of Markov chains and resolvents satisfy maximum principles in the recurrent as well as in the transient case. We will deal here with the converse problem: is any kernel satisfying an appropriate maximum principle the potential kernel of a Markov chain or of a resolvent ? The answer is negative (cf. Exercise 1.8) in general, but can be made positive b y means of additional conditions.
1. Preliminaries and bounded kernels In this and the following section, we study positive and proper kernels denoted by G or V . We denote by B the set of functions h in b b such that Glkl (or Vlhl according to the context) is bounded, and by 97+ the set of positive functions in B. Once for all, we remark that two measures which are equal on B+ are equal. These kernels are supposed to satisfy some of the maximum principles defined in ch. 2. From now on we abbreviate “complete maximum principle” into C.M.P. and “reinforced complete maximum principle” into R.C.M.P., and shall write G E (R.C.M.P.) or V E (C.M.P.) to mean that G ( V ) satisfies the corresponding principle. I t may happen that the kernels satisfy the maximum principles only for subclasses of 8+,for instance 3?+;when this is the case it will be explicitly stated.
Proposition 1.1. T h e kernel V satisfies the C.M.P. if and only if for every f in d such that Vf i s meaningful and assumes positive values, sup V f ( x ) = sup V f ( x ) . XEE
XW>O)
be the right-hand side of the above equation. We have Vf+ on {/+ > 0 } , hence everywhere, and Vf a v 0. Since Vf assumes positive values, we have a v 0 = a > 0, which proves the necessity.
Proof. Let
a
< a v 0 + Vf-
<
324
CH. 10, $1
326
BOUNDED K E R N E L S
< +
Assume conversely that f , g E 8, and V f a Vg (a > 0) on { f > O } ; then V ( f - g) ,< a on { f - g > 0 } c { f > 0 } , and if V satisfies the above equation we get V ( f - g ) a everywhere, which completes the proof.
<
Proposition 1.2. I f V E (C.M.P.) and f
E
b,, the kernel VT, E (C.M.P.).
Proof. Left to the reader as an exercise.
+
Proposition 1.3. I f V E (C.M.P.), then for every a > 0, I aV E (R.C.M.P.). I f G E (R.C.M.P.), then G E (C.M.P.) and G is a one-to-one operator and may be written G = I V , where V is a proper kernel.
+
Proof. Let f , g E 6, and assume that
f
+ EVf d a + g + aVg - g
on {f > O } . A fortiori V f we consequently have
=
a
+ aVg
< a/a + Vg on {f > O } , hence everywhere. On {f = 0 } ,
f
+ a V f < a + g + aVg -g,
+
so that I aV E (R.C.M.P.). Now let G E (R.C.M.P.) and assume that f , g E 8,are such that Gf a Gg on { f > 0). It would be easy to show that this inequality holds everywhere if g vanished on. {f > O } ; we reduce the problem to this case by setting f‘ = f - f A g andg’ = g - f A g. We then have
< +
Gf’
Since g’
=
0 on {f’
< a + Gg’
on
{i > 0 } 3 { f ’ > 0).
> 0 } , we get
< a + Gg‘ - g’ on {f’ > 0 } , hence everywhere, and a fortiori Gf’ < a + Gg’ everywhere, which implies Gf = Gf’ + G(f g) < a + Gg‘ + G(f g) = a + Gg Gf‘
A
everywhere. Let now f E 99 be such that Gf Gf+ - f+
A
3 0. We have 3 Gf-
on {f-
> 0);
hence Gf 3 f + f . If Gf = 0, it follows that f + = 0, and symmetrically that = 0, which proves that G is one-to-one.
f-
CONSTRUCTION O F MARKOV C H A I N S
326
CH. 10, $1
On the other hand if we set V = G - I , it is clear that the mapping V from 39 to b b is positive and majorized by G, hence is a positive proper kernel.
Corollary 1.4 (Uniqueness theorem). I f V E (C.M.P.), there i s at most one resolvent such that V o = V . More precisely, for every a 2 0, there is at most one kernel V , such that
aVV,
=
aV,V = V - V,.
I f G E (R.C.M.P.), there is at most one T . P . such that G = Proof. Assume that there are two such kernels, V , and
cr P,.
P,:
then for f ~ 3 9 + ,
+
and since I ctV is one-to-one, V,f = a,/. In the same way, if P and P are two T.P.'s with potential kernel G, we have
GP
=
GP = G - I.
anci ine result follows once more from the fact that G is one-to-one.
As we have already pointed out, there does not always exist such a resolvent (transition probability). We begin with the special case where V is a bounded kernel; the space 39 is then equal to bb, and V is an endomorphism of b b with norm IIVII. The following result is basic for the sequel.
Theorcm 1.6. A bounded kernel V satisfies the complete m a x i m u m principle if and only if there exists a submarkovian resolvent {Vm}a,osuch that V o , = V . Proof. The sufficiency was shown in ch. 2 $6. We proceed to the necessity. Svppose that there exists an operator V,, a > 0, on bd such that
aVV,
=
aV,V
=
V - V,.
(1.1)
This operator is positive; indeed, let g be a negative function in bd such that V,g = V ( g - aV,g) assumes a positive maximum. By Proposition 1.1, this maximum is assumed on {g > aV,g}, that is on the set where V,g is strictly negative, which is a contradiction.
CH. 10, $1
BOUNDED KERNELS
<
327
Furthermore, l\aVa\l 1; indeed, V a being positive, i t suffices t o check that aVaI I . First, we observe that it is not true that 1 aV,I everywhere, since we would then have
<
<
> V [ a ( l- aV,I)] = aV,I 2 1 V[a(I - aV,I)] < I on the non-empty set {a(7 - a V J ) > 0}, and 0
Then hence everywhere, by the complete maximum principle. For a > 0, we now set
V,= V ( I - a V + a 2 V 2 - . . - ) ; this series converges for a < llV/l-l, and thus defines an operator V , which satisfies eq. (1.1); it follows that V , is positive and that IIVall < a-l. Now choose a < llVll-l and ,4? < a, and set
Va+8= V a ( I - p p l r a + p 2 V ~ - . * - ) ;
(1.2)
the operator thus defined satisfies the relation
The operator V commutes with V,, hence with Va+#,and multiplying the two members of eq. (1.3) by V yields
(a
+ P ) VVa+,
=
(a
+ P ) Va+BV = V -
Vat8,
+
which is eq. (1.1) for a P. We have thus defined operators Va satisfying eq. (1.1) for a € [0, 211V11-1[. The same pattern permits us then to define V , for a E [O, 411Vll-1[, and by iteration for all a > 0. We notice that the map a --+ V , is continuous for the uniform topology of operators. We now show that the operators V , satisfy the resolvent equation on bb. Since I aV is one-to-one, it suffices to prove that for f E b 8
+
+
(1 X V )( V a - V,) f
=
(I
+ aV) (P - a) VaVBf,
but this is an obvious consequence of eq. (1.1). Finally if { f n } n > O is a sequence of functions in bb+ decreasing to zero, for every a > 0, lim Vafn lim Vfn = 0, n
<
n
which proves that the operators V , are the restrictions to b l of kernels Va(x, ). It is now easy to see that these kernels form the desired resolvent.
CONSTRUCTION OF MARKOV CHAINS
328
CH. 10, $1
Corollary 1.6. Under the same hypotheses, if moreover E i s locally compact and V maps C, into C,, the kernels V , have the same property.
Proof. The above proof may be performed by using the Banach space C, instead of b l .
To proceed with the general problem, we shall have t o use resolvents of the kind just described. We shall deed the notion of reduced functions in the cone of supermedian functions relative to this resolvent. Proposition 1.7. Let {V,},,, be a proper resolvent and A E 8.With each supermedian function f we may associate a supermedian function RAf called the reduced function of f on A , which satisfies the following properties : (i) RAf is the smallest sufermedian function which dominates f on A ;RAf = f on A and if f , g are two supermedian functions such that f < g on A , then RAf R*g; (ii) if {f,} is a sequence of supermedian functions increasing to f , then RAf = limn RAf,; (iii) if h E I , vanishes outside A , then RAVoh = Voh; (iv) R A ( f g) = R A f RAg; (v) if A C B ,then R A f RBf.
<
+
+
<
Proof. We use the notation of ch. 2 $6. We first prove that if a < P, then the functions which are superharmonic for X R are superharmonic for Xu.It suffices to consider the bounded potentials f = ( I PV,) h, and then
+
uV,f
=
aVa(I
+ PV,) h = aV,h + P(V,h - Vah) = (a -
Consequently, if f is supermedian P:f
RAf
=
P) Vah + PVOh d PVOh d f .
< P,$f,so that the limit lim Pif a+oo
is well defined. We proceed to show that the operator RA thus defined has the desired properties (i)-(v). We first prove that RAf is a supermedian function. Pick a ,8 > 0;for every u > ,8, P5f is PB-superharmonic, so RAf is PR-superharmonic. Since this is true for all 8, the function RAf is supermedian. Next we note that if f g on A , then Pif P i g , and consequently RAf RAg;since clearly RAg g, it follows that RAf g, and the proof of (i) is complete.
<
<
<
< <
CH. 10, $1
320
BOUNDED KERNELS
The property (ii) follows at once from interchanging increasing limits. Now let h be finite and vanish outside A ; by the known properties of balayage operators, P i ( I avo) h = ( I avo) h,
+
+
whence, dividing by a and letting u tend to infinity, we get R,Voh = Voh. The proof of (iii) is then completed by using approximation by potentials. Properties (iv) and (v) are obvious. We will also need the notion of excessive function which was introduced in Exercise 6.14 of ch. 2, the results of which will be used freely in the sequel.
Exercise 1.8. (1) Let V be the potential kernel of a proper Markovian resolvent {Va}.Define a new space E by adjoining to E a point A , and let d? be the a-algebra a(&', ( A ) ) . Define a kernel P on ( 8 , i )be setting
- +
P(x, * ) = V ( x , )
Ed(
),
x E E,
V(d, * ) = &A( * ). Prove that P E (C.M.P.). (2) Assume that there is a resolvent { P a } on E such that P = Yo. Then uPaI, = I, for every u > 0, and this is'not consistent with the definition of 8.
Exercise 1.9. Prove that a positive kernel V satisfying the C.M.P. for the functions in 9?+satisfies the C.M.P. for the functions in 8,. Exercise 1.10. One could think of generalizing the problem dealt with in this section in the following manner. Let V E (C.M.P.) and h~ bb,. Does there exist a kernel V , such that (VhIJ* V,? (i) V = V , VIAV,, (ii) V = V , VhIhV, (iii) V =
+
+
2 n
Clearly, if V , satisfies (iii) it satisfies (i) and (ii). (1) Prove that there exists at most one kernel V,, satisfying (i) and hence (iii). If h, k are in b8, and h k , and if there exist two operators V , and V k satisfying (i), then V , = V , VJk-hVp
<
+
(2) Let ( E ,b) be {- a>u Z u {+ m} with the discrete a-algebra and define the kernel V by V ( x ,y) = 0 if x < y and V ( x ,y ) = 1 otherwise. Prove that V E C.M.P. and that there exists a kernel V , satisfying (i) (resp. (ii)) if and
CONSTRUCTION O F MARKOV C H A I N S
330
CH. 10, 92
c:"
zIrn
only if h(z) < 00 (resp. h(z) < 0 0 ) . As a result a kernel V h may satisfy (i) without satisfying (ii). (3) If h E B+,there does exist a kernel V , satisfying (iii) and such that Vh(h) = 1. If f is V-supermedian (see next section) then Vh(hf) f .
<
2. The reinforced principle. Construction of transient Markov chains We now study a proper kernel G satisfying the R.C.M.P. for the functions of B,., and we shall give a necessary and sufficient condition under which it is the potential kernel of a transient Markov chain. A slight variation t o Exercise 1.8 provides an example of a kernel which satisfies the R.C.M.P. and is nevertheless not the potential kernel of a transient Markov chain. Recalling the results of ch. 2, we lay down the following
Definition 2.1. A function f E 8, is called G-superharmonic if the inequality f, g E B, holds everywhere provided it holds on {g > 0).
Gg
<
The following result provides important examples of G-superharmonic functions.
Proposition 2.2. Let a E R+ and f
E
g+; the functions a
+ Gf and a + Gf - f
are G-superharmonic.
Proof. For the first one it follows a t once from the fact that G satisfies the C.M.P. (cf. Proposition 1.1). Assume next that, for g E .@,we have
+ Gf - f 3 Gg on {g > 0); a + G(f + g-) - (f + g-) 2 Gg+ on {g+ > O } , a
this implies
hence everywhere, since G E (R.C.M.P.). It follows that a
+ Gf - f 3 Gg
everywhere, which completes the proof. By ch. 2 31.14 and Proposition 1.2 there exists a bounded strictly positive function f such that the kernel V = GI, is a bounded kernel satisfying the C.M.P. for the functions in b b + and such that V(bb+) C G ( g + ) . By the
CH. 10, $2
THE REINFORCED PRINCIPLE
33 1
preceding section there exists one and only one resolvent {V,}, a that V , = V and we have the following
> 0, such
Proposition 2.3. T h e following three f a n d i e s of functions are equal (i) the supermedian functions with respect to {V,}; (ii) the excessive functions with respect to {V,}; (iii) the G-superharmonic functions.
Proof. To prove the equality of (i) and (ii) it suffices by ch. 2, Exercise 6.14 to prove that the empty set is the only set of potential zero. But = G(f7,) = 0 implies, from Proposition 1.3, that f 7 , = 0, and since f is strictly positive, A is empty. As f is strictly positive, the equality of (iii) and (i) follows from ch. 2, Proposition 6.5. As a result, neither the cone of supermedian functions relative to {V,} nor the operator RA in this cone depend on the choice of f . Making use of Proposition 1.7, we may therefore speak of the reduced function R,h of the G-superharmonic function h on the set A , and the operator RA enjoys the properties of Proposition 1.7. The following definition is basic.
Definition 2.4. A G-superharmonic function h is said to vanish at the boundary if there exists a decreasing sequence {A,,},,, of sets in d such that: (i) A,, = 0 and G( -,A:) is bounded for all n ; (ii) inf, RA,h = 0.
n,
+
Let us further recall that if h = f g , where f and g are G-superharmonir, then h is said to be s$ecifically greater than f or g. This is the intrinsic or proper order in the cone of superharmonic functions.
Lemma 2.5. Let { h , } be a specifically increasing sequence of G-superharmonic furtctions vanishing at the boundary relative to one and the same sequence {A,,}. If h = sup h , is finite, it vanishes at the boundary relative to {An}.
+ +
Proof. For every p we may write h = h, g,, where g, is G-superharmonic. Then by Proposition 1.7, RA,h = R,,h, RA,g, RAnh, g,. Given X E E and E > 0, we may find an integer f l such that g,(x) < E , and then an integer n such that R,,h,(x) < E , which yields the desired result.
<
+
Proposition 2.6. The following three conditions are equivalent:
CONSTRUCTION O F MARKOV CHAINS
332
CH. 10, 92
(i) there exists a strictly positive function g E B+ such that Gg vanishes at the boundary ; (ii) there exists a sequence {K,} of sets in B with K , = E and such that GIKm i s bounded and vanishes at the boundary for all m, relative to one and the same sequence {A,}; (iii) for all f E B+,G f vanishes at the boundary relative to one and the sawie sequence {A,}.
u
Proof. (ii) implies (i). Indeed, there exists a sequence of numbers b, E 10, 11 such that the series b,lK, and b,GIKm converge uniformly. Setting g= b,l Km, the result follows from Lemma 2.5. Next, (i) implies (iii). Let f E a+,and set f n = ng A f ; then Gf, increases specifically to G f and each G f , vanishes at the boundary relative to any sequence {A,} for which Gg vanishes a t the boundary. We conclude the result once more from Lemma 2.5. Finally, that (iii) implies (ii) is obvious.
Z,,,
2,
2,
As has been proved in ch. 2, if G is the potential kernel of a Markov chain, then all potentials Gg vanish at the boundary. We are going to show that, conversely, this condition is sufficient to assert that G is the potential kernel of a Markov chain. We must first make some preparation.
Lemma 2.7. Let h be a bounded superharmonic function, and set D,h
=
nf(h - nV,h);
then GD,h increases to h as n tends to infinity.
Proof. The lemma follows from the fact that h is excessive for {V,}, and the equalities GD,h = V(D,h/f) = V(n (h- nV,h) = nVnh.
Lemma 2.8. Let h be a bounded superharmonic function, and {u,} a sequence of functions in B+ such that Gu, increases to h. The sequence {u,} converges then u = g and Gu = h. boundedly to a function u and if h = Gg (g E 9+), Proof. From the inequalities h 3 Gun 3 u,, we get that the sequence {u,,} is uniformly bounded. Furthermore, since for n m we have Gun 3 Gum, it u, - u,, which can be follows from Proposition 1.3 that G(u, - u,) written as un urn G(un - urn).
> >
<
+
CH. 10, $2
333
THE REINFORCED PRINCIPLE
Pick a point x in E , and two sequences {nk} and { m k } such that nR2 m k , lim unk(x)= E zc,(x) k
and
lim zcmk(x)= lirn u,(x) ; k
<
letting k tend to infinity in the above inequality yields k iu,(x) lirn ~ , ( x ) , which proves the existence of the limit u. The inequality Gu h follows from Fatou's lemma. If h = Gg, the inequality G(g - u,) 2 0 implies, as above, that g U, G ( g - u,) and, passing to the limit, that g u. As a result, h = Gg Gu; hence Gg = Gu. The operator G being one-to-one, it follows that u = g.
<
<
<
<
+
Lemma 2.9. Let {u,} be a sequence i n 9?+converging to a function u and such that the potentials Gun are smaller than a G-superharmonic bounded function h vanishing at the boundary. Then Gu = limn Gun.
Proof. Let { A , } be a sequence relative to which k vanishes at the boundary, and choose a decreasing sequence of numbers a , such that the series a,G( * , A:) converges uniformly. The function
2,
is in g+and bounded from below on each set A:. We assume that for every x in E , lim a-+m
5
G(x, dy) u , ( y )
(un>acu)
=
(2.1)
0
uniformly in n. By a classical argument we have
<
( G N x ) - Gun(%)[
1
<
1%'
~ ( xdy) , aw)
(Un(Y)
+ NY))
the hypothesis entails that G u h, so we may choose a in order that the first integral be small, then letting n tend to infinity, we get the desired result. I t thus remains to prove the uniform integrability property of eq. (2.1). The integral therein is majorized by
334
CONSTRUCTION OF MARKOV CHAINS
where C, = { h > aw} contains {Gun > am}, hence also {u, C, we have RcaGup = Gu, 2 G(up7ca),
CH. 10, 92
> aw}. But
on
and since R,Gu, is G-superharmonic, we get R,Gu, 2 G(upIc,)everywhere. A fortiori G(uPlcG) Rcah, and since h vanishes at the boundary, it suffices to show that C, C A , for sufficiently large a. Since w is bounded from below by a,, on A: we have C, c A , as soon as llhll < aa,, and the proof is complete.
<
Theorem 2.10. Let G be a profier kernel satisfying the R.C.M.P.; then the two following conditions are equivalent (i) G i s the potential kernel of a transition probability P ; (ii) every bounded potential vanishes at the boundary.
Proof. We know from ch. 2 that (i) implies (ii). We proceed to prove the converse. If (i) is true and if gE9#+, then GPg = Gg - g. We therefore set Pg = limn D,(Gg - g), which is meaningful by Lemmas 2.7 and 2.8. Let us study the properties of the operator P thus defined on 8,. The G-superharmonic function Gg - g vanishes a t the boundary, since it is majorized by the bounded potential Gg. By Lemma 2.9, we have therefore GPg = Gg - g, so that P g is also in 99+, and we can define the iterates P,g. From the equality GP,g = GP,-g - P,_,g, we derive Gg
=g
+ Pg + + Prig + GPn+,g. * * *
This implies that P,g -+0 as n m, and since Gg vanishes a t the boundary it follows from Lemma 2.9 that GP,+,g + 0, and hence that -+
Gg=g+Pg+.*.+
P,,g+***.
Now let {g,} be a sequence of functions in B'+ decreasing to zero. By Lebesgue's theorem the sequence (Gg,} also decreases to zero; since Pg, >, 0, it follows from Proposition 1.3 that Pg, GPg, = Gg, -g, and the sequence {Pg,} also decreases to zero. Finally, let g be a function in 9#+ with g I ; from the inequality 1 g = G(g - Pg), we get
<
<
G(g - Pg)'
< 7 + G(g - Pg)-
- (g - PS)-
CH. 10, 92
336
T H E REINFORCED PRINCIPLE
on {(g - Pg)+i 0}, hence everywhere. This implies that
Pg
=
G(g - Pg) - (g - Pg)
< 1.
The space g being clearly a lattice, the last two properties prove, b y means of Daniell’s theorem, that P is given by a T.P. still denoted P defined on the a-algebra generated by g.Since G is proper, the a-algebra generated by 9? is equal to 8,and the proof is easily completed. The above condition (ii) is not easy to check, but it yields some useful corollaries if some additional assumptions are made. One of these is given in Exercise 2.19; another is obtained in a topological setting.
Corollary 2.11. If E i s locally compact and countable at infinity, and if G i s a kernel on E mapping C, into C,, then G i s the potential of a Markov chain. I f , moreover, G(C,) is dense in C,, then the Markov chain is a Feller chain. Proof. The condition of Proposition 2.6(ii) is satisfied by taking for {K,} as well as for { A n } any increasing sequence of compact sets whose interiors cover E ; consequently the preceding theorem applies, and G is the potential kernel of a transition probability P. If g E C,, from PGg = g Gg it is easily deduced that PGg is in C,, and the second part of the corollary follows.
+
We now apply the same methods to the problem of kernels satisfying the C.M.P.
Definition 2.12. Let V be a kernel satisfying the C.M.P. A function h E 8, is called supermedian if for every g E 97,the inequality Vg h holds everywhere whenever it holds on {g > 0).
<
The potentials Vf(f E B+)are supermedian functions. Since V is proper, there is a strictly positive function f in g+. The kernel 8 = V I , is bounded and satisfies the C.M.P., so that there is a resolvent (8,) such that Po = 8. The supermedian functions are the same for V and P, and therefore by following the pattern of $1 we can define the reduced functions in the cone of supermedian functions. We define the functions vanishing at the boundary in the same way as above and clearly Proposition 2.6 is still available.
336
CONSTRUCTION O F MARKOV CHAINS
CH. 10, $2
+
For every a > 0, the kernel Ga = I aV satisfies the R.C.M.P., and the space g is the same for V and Ga. Furthermore, the supermedian functions are Ga-superharmonic, as is easily checked. As a result, if h is supermedian, its reduced function on A in the cone of Ga-superharmonic functions is smaller than its reduced function on A in the cone of supermedian functions. We may thus state
Lemma 2.13. Let {u,} be a sequence of functions converging to u and szcch that {Gau,} is majorized b y a potential Vg vanishing at the boundary; then Gau = limn Gau,.
Proof. The same as the proof of Lemma 2.9. This enables us to state
Theorem 2.14. For a proper kernel V satisfying the C.M.P., the following two statemenfs are equivalent : (i) there exists a submarkovian resolvent {V,} such that V,, = V ; (ii) every bounded fiotential vanishes at the boulzdary.
Proof. We first prove that (ii) implies (i). Let g E 9+.Since Vg is Ga-superharmonic, by Lemma 2.7 there is a sequence (I,,} such that Gal, increases to Vg. Since the sequence {Gal,, - I,} increases and is majorized by Vg, it converges, and consequently the sequence
{In}
=
{Gal, - (Gal, - I,)}
converges to a limit which we call Vag.By Lemma 2.13, we have GaVag= Vg, and therefore Vag does not depend on the particular zequence {I,}, since Ga is one-to-one. Finally Vag lies in =@+. As in the proof of Theorem 2.10, it is verified that if g 7, then aV,g 7. That the operators V a satisfy the resolvent equation is seen as in the proof of Theorem 1.5. Finally Daniell's theorem enables us to show that V a is given by a kernel. It remains to show that V o = V . We already know that Vag Q Vy fpr every a, hence that Vog Q Vg. Now the identity ( I + aV) Vag = Vg implies that, for every n,
<
(I
+ aV) (aV,)"g = aV(aVa)n-lg,
and summing up these equations yields
<
CH. 10, $2
337
THE REINFORCED PRINCJPLE n
C1 (aVa)&g + aV(aV,)" g = J'g.
<
This proves that V(aV,)"g Vg and limn (aVa)"g= 0, which implies limn aV(aV,)" g = 0. Finally, it follows that m
which proves, on account of ch. 2, Proposition 6.4, that V , = V . We now turn t o proving that (i) implies (ii). Let {a,} be a sequence of real numbers decreasing to zero. Let f be strictly positive and in g+.The sets A : = { Vanf< a,} decrease to N = { Vf = O } . We have V ( x , ) = 0 for every x in N because f is strictly positive, hence V ( , N ) = 0 on N a n d V ( , N ) = 0 everywhere by the C.M.P. If we set A , = A:\,,N, we have therefore
-
-
-
and the sequence {A,,} decreases to the empty set. We will complete the proof by showing that RAnVf goes to 0 as n tends to infinity. Since A n c A: we have RA,Vf R,;Vf; moreover, on A:, we have, by the resolvent equation,
<
and since the function on the right side is supermedian,
Letting n tend to infinity yields the desired result.
As before we have the corollary
Corollary 2.16. If E as locally compact and countable at infinity, and if V m a p s C K into C,, there i s a sub-markovian resolvent { V m }such that V , = V . I f , ilz addition, V(C,) is dense in Co, then V am a p s C, into C, for each a > 0. Proof, The same as for Corollary 2.11. Remark. This corollary is not the best available, as is pointed out in the exercises and in the notes and comments.
Exercise 2.16. Prove that h - G u in Lemma 2.8 is superharmonic.
338
CONSTRUCTION OF MARKOV CHAINS
CH. 10, $3
Exercisc 2.17. Show that every bounded G-superharmonic function vanishing at the boundary is a potential.
+
Excrcisc 2.18. The kernel I 1 @ v, where v is a non-zero a-finite measure, satisfies the R.C.M.P. and is not the potential kernel of a Maikov chain. Exercise 2.19. Prove that if V E (C.M.P.) and V7 is finite, then there is a submarkovian resolvent {V,} such that V , = V . Solve the same problem for a kernel G E (R.C.M.P.). Exercise 2.20. Resume the hypotheses of Theorem 2.14. Let {K,} be an increasing sequence of sets such that K , = E and VI,, is bounded for alln.Let {b,} beasequenceofnumbersin]O,l[suchthatifweseta = ~ , b , l r m , then a and Va are bounded. Set a, = n a A I and V n = V I 0 %define ; {Vz} to be the resolvent with V i = V . Iinally define 8, to be the set of functions g such that {g # 0) c I<, for some m. (1) Prove that if g E c?; then , for fixed a, V:(g/an) is a decreasing function of n. ( 2 )Giveaproof of thefirst partofTheorem2.14bysetting V,g=lim, V:(g/a,). (3) Prove that Corollary 2.15 holds without the density hypothesis.
u
Exercise 2.21. With the hypothesis of Corollary 2.11 (resp. 2.15), if in addition E is a group and G (resp. V ) is translation-invariant, the corresponding kernel P (resp. xV,) is the T.P. of a random walk.
3. The semi-completc maxirnum principle The goal of this section is essentially to raise the following problem. As we have seen in ch. 8, the recurrent counterpart of the C.M.P. is the so-called semi-complete maximum principle ; does there exist a characterization of recurrent potential kernels by means of this principle similar to the characterization found in the foregoing sections for the transient kernels ? To give a full answer to this question would take too much space to be attempted here, and we shall content ourselves with a few remarks and a very incomplete answer. Once for all, a measure nz is given on ( E ,8). and any word such as “negligible” will rcfer to this measure.
CH. 10, $3
339
THE SEMI-COMPLETE MAXIMUM P R I N C I P L E
Definition 3.1. A kernel W(r) is said to satisfy the (reinforced) semi-complete maximum $rinci$le if, for every bounded integrable function f such that m(If1)> 0, m ( f ) = 0, and W f ( r f ) is bounded, the inequality V Y f ( r f ) a , a everywhere (I'f a - f- everywhere, or a G It, on { f > 0) implies Wf equivalently I'f - f a everywhere).
<
<
<
<
As in the preceding sections, we, shall use the simplifying notations R.S.C.M. and S.C.M. We call N the space of functions f having the properties in the definition; of course N depends on m and on the kernel studied. I t may happen that some kernels satisfy the principles only for subclasses of&-. We may now state our problem in a proper way: is the kernel in Definition 3.1 the potential kernel of a Harris Markov chain? We start with some easy remarks. In what follows we suppose TE (R.S.C.M.).
r
Proposition 3.2. If W E (S.C.M.), then I
+ UW E (K.S.C.M.) for every u > 0.
Proof. Easy and left to the reader. Proposition 3.3. If f E &- then i s one-to-one on &-.
r
rf
rf cannot be constant on { f
rf
# O}. In particular,
rf< a - f-
Proof. If = a on {f # 0 } ,in particular = a on { f > O}; hence everywhere, and in particular on {f-> 0). I t follows that f consequently that f f is negligible.
=
0, and
A clue to the difficulty of the problem is given by the following theorem, which states that Harris chains are not the only chains which give rise to potential kernels satisfying R.S.C.M. W e do not strive for the greatest generality .
Theorem 3.4. Let E be L.C.C.B. and G be a proper kernel on E satisfying the R.C.M.P. and mapping C, into C,. If the set of measures {G(x, . ), x E E } has a non-zero cluster point m as x -+ A , then G satisfies the R.S.C.M. with respect to m for functions in N n C., Proof. There exists a function u E C, which is strictly positive everywhere and such that the kernel V = G I , is bounded and satisfies the C.M.P. There exists therefore a resolvent { V , } such that V , = V , and we denote by P: the balayage operator relative to A and the chain X u of T.P. aV,. If h vanishes
CONSTRUCTION OF MARKOV CHAINS
340
CH. 10, 53
<
Let f E C, be non-zero and such that m(f) = 0, m(f+) > 0, and Gf a on { f > 0). If we let f = fu-' then (urn, f ) = 0 and V f a on A = {f > 0). We have
<
+ a V ) h4 < (Ilfll + a 4 W ( 4 . Let {x,*} be a sequence converging to d such that V(x,, - ) converges to urn. (1
Passing to the limit along {x,} in the above inequality yields
(llfll + 4 fifi; GI(&) n
2 0.
Now, passing to the limit in the same way in the relation
(I
+ ccV) f +
= Pi(I
+ aV)f+,
we get
0 < a(um, I+)
d (Ilf'll
+ allVf+ll)fi
It follows that E n P,I(x,) > 0 ;hence sequently a 0. As a result we have.
>
Gf+
< a + Gj-
- f-
C](%I).
n
Ilfll + cra
0 for all
Q,
and con-
on {f+ > 0 ) with a E R+
and we get the desired result by applying the R.C.M.P. to G. This theorem holds as well with the C.M.P. and S.C.M. in place of the reinforced principles, and if G is the potential of a transient chain or resolvent, then nz is an excessive measure. We remark that there are indeed many kernels satisfying the hypotheses of this theorem : for instance, the potential kernels of transient random walks of type 11. Furthermore, if G satisfies the R.S.C.M. with respect to m, then the kernel G 7 @I v p @ m, where p is a bounded function and v a measure such that v ( IfI) < co for every f E N, also satisfies the R.S.C.M. This shows that there are plenty of kernels satisfying the R.S.C.M. that are nevertheless not the potential kernels of Harris chains. This will be made plain by the following result. Finally, it is worth recording that the kernels dealt with in the above result are precisely those for which Corollaries 2.11 and 2.15 do not apply. We now turn to a result parallel to Corollary 1.4. The notion of modification refers to m.
+
+
CH. 10, 93
THE SEMI-COMPLETE MAXIMUM P R I N C I P L E
34 1
Proposition 3.6 (uniqueness theorem). If ( E , 8)is separable, and i f G is a proper kernel satisfying the R . S . C . M . , there exists at most one chain up to modification such that G is its potential kernel either in the sense of ch. 2 or in the sense of Harris chains.
2;
Proof. If P is a T.P. such that G = P,,then for every f E bb, and such that GfE bB,, G(I - P)f = 0. If P is a Harris chain such that G is one of its potential kernels in the sense of ch. G $5, then there is a measure v such that G(I - P)f = v ( f ) . Now let P and P be two T.P.'s for which G is a potential kernel in any of the two meanings which we deal with. I t follows that if g E bb, n 9 ( m ) is such that Gf is bounded, then G(Pf- Pf) is constant. Proposition 3.3 implies therefore that Pf = Pf m-a.e. The usual arguments prove that P is a modification of P . Remark. It suffices of course that G satisfies the R.S.C.M. for a sufficiently rich subclass of N. We will now give a partial answer to our problem. We will assume that E is L.C.C.B. and that m is a Radon measure such that the only open set with zero measure is the empty set. (See ch. 8 $4.) For any compact set K , we denote by W ( K ) the space of Bore1 functions vanishing outside K and by V ( K )the space of functions continuous on K and vanishing outside K . We study a kernel G satisfying the R.S.C.M. for the functions with compact support; but we assume further that G may be written G = I U , where u has the following property: for every compact K , the map f (uf)7, is compact from b b into V ( K ) , the spaces being endowed with the usual supremum norm. These conditions are satisfied by the potential kernels of Harris chains whose T.P. is strong Feller in the strict sense, hence in particular if E is countable. The following proposition should be compared with ch. 8, Theorem 2.1.
-.+
Proposition 3.6. For every compact K such that m ( K ) > 0, there exists a unique Radon measure v K on K such that: (i) ( v K , 7 ) = 1 ; (ii) ( v K , Gf) = 0 for every f E N n W ( K ) .
-.
Proof. In view of our hypotheses, the map f ( U f )7, is compact from % ( K ) into V(K), hence from N n '&(K)into % ( K ) . The subspace X n V ( K ) has
342
CONSTRUCTION O F MARKOV C H A I N S
CH. 10, $3
co-dimension 1 in V ( K ) .The operator I is thus of index 1 from into V ( K ); hence by the index theorem, the operator
+ (uf) I K = (Gf)
f -f
N nV ( K )
IIf
is also of index 1 . By Proposition 3.3, this operator is one-to-one on .N; its range is thus of co-dimension 1, and therefore is closed by an application of Banach's theorem. On the other hand, Proposition 3.3 asserts also that 1, is not in this range. There exists, therefore, a unique probability measure vK on K such that (v,, Cf)= 0 for every f E N n V ( K ) . Now let f be any function in %(I(); the function m ( K )f - m ( f ) 1, is in N n V ( K ) ,and therefore 0 = V K G(m(K)f - m ( f ) 7 ~ = ) m ( K )VKGf - ~ ( f ( V) K , G1K).
The measure v,G is thus a multiple of m on K , and the proof is complete. For g E bd define hK = (g - Z ) K ( g ) ) 1,; the function hK is in g ( K ) and vK(hK)= 0. By the above result there is a function f K in N n g ( K )such that hK = (CfK) l K . Keeping in mind the formulae of ch. 8, Theorem 2.1 we define pKg
flKg
+ = GfK + v K ( g ) - f" =
GfK
vK(g)i
=
PKg - f"
= Uf"
+ v&).
We notice that if g E g ( K ) , then PKg = g, and that f l K g is a continuous function. On the other hand, P,g = f l K g on K". Finally, we have: Proposition 3.7. The o#erators P K and flKare given by transition probabilities. The measures P,&, * ) and fl&, * ) vanish outside K and furthermore (i) m I K 1 I K = (ii) ( I - nK) Gf = f for every f E N n B ( K ) . Proof. If g vanishes on K , then hK = 0 ;hence f K = 0 and consequently PKg = IIKg = 0. If g E @(K)+,since P K g = g on K , we have GIK vK(g) >, 0 on K ; hence G(- f") d y K ( g ) On {f" # O},
+
and by R.S.C.M., G(- f K ) and
< v K ( g )- (-
n,,g
= Gf"
fK)-everywhere.As a result, P K g 2 0,
+ v,'(g) - I" > 0.
CH. 10, 53
T H E SEMI-COMPLETE MAXIMUM P R I N C I P L E
343
nK
That P , and IT,, are kernels is clear from the form of as a function of Finally ifg = 1, then hK = f K = 0, and therefore P,I = nKl= vK(1) = 1. Next, for any g E bb,, we have
u.
since f K is in JV n B ( K ) and P,g = g on K . This proves (i). To prove (ii), pick f in .A-' n 99(K);then Gf is in bb and vli(Gf) = 0. If we put g = Gf, we have hK = gl,; hence f K = f, and consequently
The T.P.'s Z7, and P , must be the balayage kernels of the chain we are looking for. The following result goes further in this direction.
Lcmma 3.8. If H and K are two compact sets with H (i) PKPH = P H ; (ii) I7KPH = nH; (iii) for any g E B(H)+, Z7K(g) d nH(g).
+
C
K , then
Proof. Set h = P,g = GfH v&), where f H E Nn B ( H ) .There is a function f K E JV n B ( K ) such that P,h = GfK v K ( h ) .Since P K k = h on K , we have
+
hence, by proposition 3.3, f H = f" and VK(h) = v&), Next, from f H = f K we derive n K p &
=
SO
that PKPHg = PHg.
PKPHg - f" = PHg - f H = ITHg.
Finally, if g E @ ( H ) + , then PHg ready to state
g, and therefore IIKg
< HHg.We are now
Theorcm 3.9. There exists a strong Feller transition $robability P with excessive measure m and such that, for any f E JV with compact sufiport,
(I - P ) Gf
=
f.
Proof. Let {K,} be a sequence of compact sets whose interiors cover E and are increasing to E . If g E b b + has compact support, the precedinglemmaimplies
CONSTRUCTION OF MARKOV CHAINS
344
CH. 10, $3
that the sequence 17,,g decreases everywhere (at least for n sufficiently large) to a limit that we call Pg. We have
+ vK,,(fKn)).
P g ( x ) = lim nKng(x)= lim(UfKn(x) n
n
As soon as K , contains the support of g, we have nKng= g - f K n ; hence IlfKnII 211gll. It follows that the family { U f K n is } equi-continuous on each compact set ; one may therefore find a sub-sequence converging uniformly on every compact set, and as a result the function P g is continuous. Clearly P is a transition probability. Furthermore 7 Kn g converges to Pg, and since
<
nKn
(m, 7 K n 1 7 K & > = (m, 7 K n g > # by Fatou's lemma, passing to the limit yields (m, Pg) excessive for P. Now let f E JV have compact support, and set g = Gf
+ IlGfll
=
f
< m(g),and thus m is
+ Uf + IlGfll.
By Proposition 3.7, as soon as f vanishes outside K,, we have DKng= Gf - f llGfl 1; as a result, the lower semi-continuity of g and the fact that IIKn(z,) converges to P ( x , ) vaguely entail
+
-
-
pg hence PGf
d hnK,g
= Gf
< Gf - f . Applying this to (PGf
=
-f
+ \lGfll;
f ) yields
Gf - f ,
and the proof is complete. Unfortunately we do not know of a satisfactory criterion to decide whether the T.P. of the preceding theorem is recurrent or transient. (See, however, Exercise 3.12 and the notes and comments.) But we have the following partial answer.
Theorem 3.10. If m i s bounded, the T.P. of Theorem 3.9 i s Harris, and m i s its invariant measure. Proof. If m is bounded we may, in the preceding proof, apply Lebesgue's theorem instead of Fatou's lemma, and we get the equality m(Pg) = m ( g );
CH. 10, 93
THE SEMI-COMPLETE MAXIMUM P R I N C I P L E
345
thus m is invariant by P. Since m is bounded and invariant, it is easily seen that for f E Ck the series P,f is either co or 0, or equivalently that P is a conservative contraction of L1(m).We prove that it is ergodic. If P were not ergodic, we could find two functions f, g in L1(m)such that h = f - g is in M , and a set A with m ( A ) > 0 such that for x E A , P, f ( x ) = + 03 and P , g ( x ) = 0. From the equality (I - P ) Gh = h we derive
+
2
2
Since Gh is bounded, we get a contradiction on letting k tend to infinity. Thus, for any A E B with m ( A ) > 0, we have U A ( l A = ) 1 m-as.; but since P is strong Feller, it follows by the usual continuity argument that U A ( l A = ) 1, and hence that P is Harris. This proof could also be completed by calling upon ch. 4 $4. If E is compact, then m is bounded and the preceding theorem applies.
Exercise 3.11. Assume that the operator U used in the proof of Proposition 3.6 satisfies the S.C.M., and then prove that it is the potential operator of a strong Feller resolvent. Assume further that m is bounded, and prove that the resolvent is Harris. Exercise 3.12. Let E be countable and m a measure on E ; G a kernel on E satisfying the R.S.C.M. for the functions of .N (relative to m) with compact supports. (1) Fix an arbitrary state e in E and define Ge(x,y )
=
G(x, Y ) - G(e, Y) - (G(x, e) - G(e, e)) m ( ~ ) / m ( e ) .
Prove that the kernel G e satisfies the R.C.M.P. on E\{e}. The reader is invited t o look a t ch. 8, Exercise 2.11 to see the significance of G". (2) Prove that if every bounded potential Gef vanishes at the boundary in the sense of 5 2, then there is a Harris chain on E for which G is a potential operator. Prove that if m is bounded, then the above condition on Ge-potentials is satisfied, which thus gives another proof of Theorem 3.10 for countable state space.
NOTES AND COMMENTS
Before we proceed to comment on the text we feel that the title of the book deserves an explanation. There has been over the years some confusion on what is meant by “chain”. According t o the author, this word refers sometimes t o the discreteness of the set of time parameters, and sometimes to the discreteness of the state space. In our opinion the most important feature is the time: a denumerable state space is a special case of a general state space, whereas discrete time is not a particular case of continuous time. In particular, the set of integers has only onc accumulation point,’ namely co, whereas the set of positive reals has plenty. As a result, most of the problems in discretetime processes concern the asymptotic behaviour a t co. For continuoustime processes, the analogous problems are often solved by straightforward applications of the results known for discrete time. But a host of new problems, with no discrete counterpart, arises in connection with the other accumulation points: for instance, the study of local properties of sample paths. A good terminology must thus stress the nature of the set of time parameters. We therefore stick to using “chain” when the time is discrete and “process” when it is continuous. If one wants to specify the kind of state space or analytical properties, one can add an adjective or a noun. For instance a chain can be Harris, discrete, . . ., a process can be a diffusion process, a Feller process, a Hunt process. Although this book is devoted to chains, (continuous-time) processes will be alluded to by means of resolvents.
+
+
Chapter 1 The first three sections are of standard character. Some proofs as well as some exercises are taken from Blumenthal and Getoor [l]. The proof of Theorem 2.8 is from Ionescu-Tulcea [l]. The notion of admissible a-algebra is taken from Doob [Z]. When the space is discrete, transition probabilities may be written and handled as matrices as is done extensively in the book by Kemeny et al. [l]. 346
NOTES AND COMMENTS
341
Section 5. The important Lemma 5.3 is taken from Doob [2] and Theorem 5.7 from Mokobodzki [l]. Some results of Mokobodzki on what may be said for a-finite kernels in the line of Lemma 5.3 are described in Meyer and Yor [l]. An ergodic theory without measure has been developed for Feller kernels. We refer the reader t o the expository paper by Foguel 121.
Chapter 2 Most of the results of the first two sections are originally due to Deny [l] and Doob [3]. We have also used Meyer [2] and Neveu [l]. The introduction and systematic use of the kernels U , is due to Neveu [GI. I t is interesting to describe their continuous-time counterpart. If X is a standard process with resolvent {V,}, and we study the chain with T.P. P = V l , then for h E a+ and f E b+,
I
W
Uht(4 = E,
.o
M t f F , ) dt
where M , = exp[-- Jb h(X,) ds]. The proof of Theorem 2.3 has been borrowed from Meyer [6]. The proof of Theorem 2.2 is related t o the work of Rost [l, 21 and Mokobodzki 121. For a more thorough study of reduced functions see the third part of Dellacherie and Meyer [l] where one will also find a general method of conducting computations about kernels. Let us mention that the identity between harmonic functions for random walks on symmetric spaces and harmonic functions in the classical sense is shown by Furstenberg [l]. We also refer t o Baxter [2]. Exercise 1.24 and especially the subsequent ones dealing with Birth and Death chains are taken from Karlin and McGregor [l]. This was generalized by Kemeny [3]. Section 3. The fruitful idea of relating harmonic functions and invariant events is due t o Blackwell [3]. Section 5. The filling scheme procedure was introduced as a tool in ergodic theory (see Chacon and Ornstein [l]) and was used in many contexts, for instance random walks (Ornstein [l]).The material in this section as well as the exercises are mainly from Rost. Our presentation borrows t o Meyer [7] and t o Baxter and Chacon [l].Exercise 5.15 is from Baxter [l]. Section 6. The goal of this section is to prepare for the discussion in Chapter 10 as well as t o show in numerous exercises of the following chapters how some of the results obtained for chains may be translated to he continuous time situation.
348
NOTES AND COMMENTS
Exercise 6.13 is taken from Meyer [3] and Exercises 6.14-6.15 from Watanabe [2].
Chapter 3 Section 1 is standard, at least for the results, and is intended as an introduction to the sequel as well as t o provide examples for the more general situations. The existence of the invariant measure for irreducible recurrent discrete chains was first proved in Derman [l]. Section 2. The main results on irreducible chains, namely Theorems 2.3 and 2.5 are due to Jain and Jamison [l], but the proofs in the text, as well as most of the subsequent ones, are from Neveu [5]. The whole subject was started by Harris who proved the implication (iii) 3 (ii) in Proposition 2.7. The proof of Proposition 2.2 uses a pattern similar to Harris’ original proof. The useful remark in Exercise 2.14 is from Doob [l] and Exercise 2.21 owes to Blackwell [l]. The theory of R-recurrence roughly sketched in Exercise 2.23 has been developed in the papers of Tweedie [I]. I t is important in as much as the number R thus isolated is the inverse of the largest possible value for an eigenvalue of the operator P acting on functions or on measures. In some contexts, for instance random walks (see Guivarc’h [3]) it is better to assume a topological irreducibility, but the proofs go through the same way. Sectiorc 3. The first results are from Chung and Fuchs [l] in the case of the real line. The extension to arbitrary groups is straightforward. Theorem 3.9 seems to have been widely known, a t least for abelian groups; the present proof is probably new. The proof in Exercise 3.21 was given by Bretagnolle and Dacunha-Castelle [l]. Exercise 3.13 was taken from the basic book by Spitzer [l] and Exercise 3.14 from Chung and Ornstein [l]. Spread-out (in French LttaZLees) random walks are generally referred to in the literature as “non-singular”. Section 4. For the few results of harmonic analysis which are used in this section we refer to Rudin [I] and to Hewitt and Ross [l]. The main result of this section, namely Criterion 5.2, is due to Chung and Fuchs [l] in the case of the real line. I t was extended to arbitrary abelian groups by Loynes [l]. Theorem 5.11 was proved by Dudley [l]. The structure theorem used in its proof may be found in Kurosh [I] and in Hewitt and Ross [l]. For the characterization of non-abelian recurrent groups the state of the subject may be found in Guivarc’h et al. [l] and in Baldi [l].
NOTES AND COMMENTS
349
In this book, we aimed at keeping the use of Fourier transform to a minimum, as is seen from chs. 5 and 9 on random walks. The reader who wishes to study random walks by means of Fourier transforms is referred t o the book by Spitzer [l] and to the papers by Port and Stone, especially their final paper [l]. Chapter 4
Section 7. This owes much to the fifth chapter of the book by Neveu [2]. Exercise 1.10 is from Chacon and Krengel [l]. For extensions of Proposition 1.8 see Getoor [l]. Section 2. The results of this section are due to Hopf 111. The simple proof of the basic Maximal ergodic lemma is from Garsia [l]. Exercise 2.12 is from Krengel [l], Exercise 2.13 from Ackoglu and Brunel [l] and Exercise 2.18 from Brunel \unpublished]. The proofs of the ergodic Lemma in Exercises 2.19 and 2.20 are taken respectively from Neveu [8] and Meyer [7]. Section 3. The proof given here of the Chacon-Omstein Theorem is due to Neveu [3J,Lemma 3.1 being an original lemma of Chacon and Ornstein. Exercise 3.14 is a trivial generalization of a result of Oxtoby [I]. Section 4. The characterization of Hams chains given in Theorem 4.6 is taken as their definition when one wants t o perform their study by purely analytical means. We refer the reader to Foguel [l, 41. This characterization is useful to implement the following general “principle” : if a conservative chain has some strong property of discrete irreducible recurrent chains then it is Harris. Theorem 4.5 is from Neveu [6] (see also Jain [ZJ). The use of the measures I7, to prove the duality is suggested in Foguel [l]. The result in Exercise 4.10 was proved by Doeblin (at least the existence of the limit) and was the first result of this kind upon which the following ones were patterned. The dichotomy property of Exercise 4.12 was first proved in Hennion and Roynette [l]. Let us finally mention that there exist many central-limit theorems for functionals of positive Harris chains which are too numerous to be listed here. One can see for example Orey [5] and Maigret [l]. Srclion 5. The form of Brunel’s lemma given here is due to Ackoglu [ l ] and its proof is from Garsia [2]. The other proofs of this section are borrowed from Meyer [I]. Exercise 5.8 was taken in Meyer [7] and the proof of the Chacon-Ornstein theorem hinted at in Exercise 5.9 is from Neveu [8].
350
NOTES AND COMMENTS
Serfion 6. The bulk of this section is due to Kingman [l] whom we follow rather closely in our exposition. The proof of the decomposition theorem however is borrowed from Ackoglu and Sucheston [l] who deal with the more general case of markovian operators and prove a ratio limit theorem for superadditive processes. The subadditive convergence theorem turned out to be very useful in widespread situations as is illustrated in the expository paper of Kingman [2] from which we have borrowed some of our exercises (see also Derrienic [5]). The result i n Exercise 6.15 was originally due to Spitzer (see Spitzes [l]). Another proof of the convergence theorem which avoids the decomposition theorem may be found in Derrienic [3]. Finally, we must record that the hypothesis of invariance of the probability measure may be weakened by the use of ideas of Krengel [Z] (see Abid [l]). This chapter might have included a section on invariant measures. We refer to the books by Foguel [I] and Friedman [l] and also to the paper by Brunel [Z].
Chapter 6 Section I . The Choquet-Deny theorem was announced in Choquet and Deny [2] and its proof given in Deny [2]. The proof presented here is from Guivarc’h, Lemma 1.1 being from Brunel [2]. The proof in Exercise 1.7 is from Doob et al. [I]. Example 1.10 is taken from the fundamental paper by Furstenberg [l]. Further work in this direction was accomplished by Azencott [l], Guivarc’h [l], and especially Raugi [l]. Section 2. For the history of the renewal theorem, we refer the reader to the book by Feller [a]. To write this section, we used the book by Feller already quoted and the paper by Herz [l], from which the proof of Theorem 2.5 is taken. Exercise 2.8 is from Bougerol and Elie [I] and Exercise 2.9 from Spitzer [l]. Section 3. The most general form of the Renewal Theorem for the real line was given by Feller and Orey [l]. We follow here the presentation of Feller [4]. Sectioiz 4 . The Renewal Theorem 4.4 was shown by Port and Stone 113 after Kesten and Spitzer [2] had opened the way. For non-abelian groups the results are now almost complete thanks to the work of Elie [l] and Sunyach [2]. Section 5 . Theorem 5.1 was first shown in Stone [l] for the real line and was extended to general abelian groups in Port and Stone [l]. The present proof is from Bellaiche-Fremont and Sueur-Pontier [l]. Proposition 5.6 is
NOTES AND COMMENTS
35 1
taken from Spitzer [l] in the case of integers and from Bretagnolle and Dacunha-Castelle [l] for the real line. Further results on recurrent sets of transient random walks may be found in Spitzer [l] and Jain and Orey [l]; Exercise 5.13 is borrowed from Spitzer [l]. Let us also mention that there is an enormous literature on the Renewal theory for Semi-Markov chains for the definition of which we refer to Cinlar [l] and Jacod [l].
Chapter 6 Section I. “Zero-two” laws have been discovered by Ornstein and Sucheston [l] and were extended to continuous time by Winkler. In our presentation we follow closely Derrienic [4] which stresses the probabilistic point of view. We also refer to Foguel [3]. Section 2. The important advances for the results of this section were made by Orey. He proved the existence of Cyclic classes in [l], then proved the limit theorem for aperiodic chains (Property2.2(i))in the case of irreducible recurrent discrete chains. Another method of proof of this result was given by Blackwell and Freedman [l] and finally carried over to the general Harris case by Jamison and Orey [l]. The presentation adopted here, which rests on the zero-two laws is that of Revuz [5]. The original method of Orey described in Orey [5] and in the first edition of this book, is by means of a differentiation lemma, t o prove the existence of a set C such that infx,yecp k ( x ,y ) > 0 for some K. This set can then be used as points were used in the discrete case (see ch. 3, Exercise 1.17) to exhibit the cyclic classes. Athreya and Ney [l] and also Nummelin [I] have taken advantage of the existence of such so-called “C-sets” to give a proof of the existence of the invariant measure and of the limit theorem relying on Renewal theory ideas along the line described in Exercise 3.14 of ch. 5. This is the “splitting” technique which can be used to give proofs of many results on Harris chains among which we would like to mention the result on the speed of convergence in the limit theorem given in Nummelin and Tweedie [l]. In connection with these results let us also draw attention to the papers of Griffeath [ l , 21. Finally, another approach to cyclic classes will be found in Foguel [ l , 41. Theorem 2.6 and its proof are from Jain [l]. An analogous limit theorem for L+ functions is proved by Horowitz [2]. Exercise 2.10 is from Cogburn [l] and Exercise 2.19 from Duflo [I]. The question of what can be said in place of Exercise 2.7(2) when X is null
352
NOTES AND COMMENTS
has been investigated by Krickeberg [ l , 21. This is related to the study of the asymptotic behaviour of the ratios Pn+,,,(x,A)/P,(y, B), which has been tackled by several authors. We refer the reader interested in this topic t o Orey [ 5 ] . For the study of point transformations and the notions of mixing, see the book by Friedman [l]. Section 3. This section is intended mainly as a first step in the study of special functions and it is Theorem 3.5 which is important in this respect. Condition (ii) therein appears in Bmnel [3] together with the fact that I - P is a n isomorphism of bob. The proof of (if 3 (ii) uses ideas of Foguel [4] to which we refer the reader for another more general approach to many results on Markov chains. Many results, especially the conditions in Theorem 3.7 are from Horowitz [l]. Finally the general study of quasi-compact operators is due to Yosida and Kakutani [l] and our presentation borrows much t o Neveu [ 2 ] . For the equivalence between quasi-compactness and Doeblin’s condition see Doob [Z] and Fortet [l]. Finally some further results and an exposition of the spectral theory for quasi-compact operators may be found in Brunel and Revuz [2]. Section 4. Special functions which generalize the former “bounded sets” (Orey [5], Metivier [l], Brunel [3]) appear in Neveu [5]. Together with the potential kernels of the next section, they constitute a major progress in the theory of Harris chains, permitting one to simplify and sharpen all the previous results, Neveu proved all the results of this section up to Proposition 4.8. Proposition 4.9 and 4.10 are from Brunel and Revuz [l]. The idea of relating “boundedness” of sets t o quasi-compactness appears in Brunel [3]. Horowitz [l] has shown that if a chain is conservative and ergodic in the sense of ch. 4 and possesses a “bounded” set, then it is Harris, a result which was extended to special functions by Lin [Z]. (See Exercise 4.15.) The results on Harris resolvents of this and the following section are mostly from Brancovan [l]. For Exercise 4.16 see Nummelin [ 1]. Section 5. Almost all this section is taken from Neveu [ 5 ] where, however, Theorem 5.2 is proved only for strictly positive functions; Proposition 5.4 was first stated by Orey [4]in the case of discrete chains, and enables one t o simplify the proof of Corollary 5.5. Proposition 5.7 and its sharpening in Exercise 5.16 had been obtained before for “bounded” sets by Ornstein [l], Metivier [l] and Duflo [l]. Section 6. The first ratio limit theorem for recurrent discrete chains was given by Doeblin. The proofs of this section are due to Neveu [7] ; they are patterned after Levitan [l]. The reader will find another interesting proof in Metivier [ Z ] . For earlier work, see Jain [l], Isaac [Z] and Metivier [l]. A counterexample
NOTES AND COMMENTS
353
which shows that the most general ratio limit theorem fails to be true was designed by Krengel [I]. The important Exercise 6.8 is from Brunel e t al. [l]; the result is still true for general recurrent random walks.
Chapter 7 Section I. The results on Martingales needed in this section may be found in Neveu [2] or Dellacherie and Meyer (in particular on page 21 in Volume 11). The ideas in the first part of the section are from Blackwell [Z]. The ideas involved in the proof of Theorem 1.11 are originally due to Hunt [2] in the case of discrete chains and were used b y Abrahamsee [l] for general state space. The observation contained in Theorem 1.13 is from Roudier [l]. For the important Exercise 1.18 see Furstenberg [l] and Azencott [l]. Section 2. The Martin boundary for Markov chains was introduced independently by Doob [3] and Watanabe [l]. The metric we use here to complete the space is the metric of Hunt [2]. To extend the theory to a case more general than discrete chains we have used the paper by Derriennic [l]. Section 3. The boundary M of $2 is not easily computed. The third section gives a way of computing its interesting part and at the same time gives the desired integral representation of harmonic functions. What lies beneath the present discussion is the theory of convex cones of Choquet. We refer the reader to Neveu [l] for an exposition of the Martin boundary relying on the theory of Choquet. In this line, the results of Mokobodzky (see Meyer [5]) would perhaps lead to some generalizations in more general state spaces. The Martin boundary has now be worked out for several important classes of discrete chains. The reader should look in Doob et al. [l], Dynkin and Malyutov [l], Lamperti and Snell 113, Blackwell and Kendall [l] and Ney and Spitzer [l]. For random walks on groups and symmetric spaces see also Furstenberg [2] and Azencott and Cartier [l]. Their results use the theory of Poisson spaces, sketched in Exercise 1.18. The most general non-bounded harmonic functions are not known for all groups. See, however, Furstenberg [3] and Deny [!I. Chapter 8
Section 1. This is taken from Brunel and Revuz [l]. Section 2. This is taken mostly from Neveu [5],but these kinds of results were already known for discrete chains (see, for instance, Neveu [l] or Kemeny et al. [l]). The observation in Exercise 2.11 is due to Orey [4].
354
NOTES AND COMMENTS
Exercise 2.13 is due to Chung in the case of discrete chains; the proof hinted at is from Lin [2]. Section 3. The theory of normal chains is due to Kemeny and Snell [l] in the case of discrete chains. Their results are also contained in the book by Kemeny et al. [l]. I t was shown by Orey [2j that there exist non-normal chains (see Exercise 3.11). Exercise 3.9 for the case in which X is discrete was taken from Kemeny et al. [l]. Section 4. The theory of boundaries is only sketched in this section. More complete results for discrete chains will be found in the last chapter of the book of Kemeny et al. [l] where in particular Theorem 4.10 is proved by another method. We also refer to the paper of Orey [4] and the exposition in Neveu [l] where integral representations are given for potential kernels. Theorem 4.13 is in some sense a version of these results which have been carried over to a more general setting by Bronner [3]. A general theory for Feller-Harris chains generalizing all the results known either for discrete chains or for random walks has yet to be written. To this end the methods of Brunel and Revuz [3, V] may possibly prove useful. In the case of random walks it is shown that the conditions of Theorem 4.10 hold, but furthermore that the convergence of the powers P , hold for all special and co-special functions, whether continuous or not. This is due in particular to the fact that for Harris random walks, P, is “asymptotically” strong Feller. Are similar results true for all chains? More precisely, if a chain satisfies the equivalent conditions of Theorem 4.9 what can be said of the class of functions on which P, converges ?
Chapter 9 The study of potential theory for recurrent random walks was initiated by Spitzer, who treated the case of Z p and discovered the main features such as the classification into types I and 11. His results are collected in his basic book [l]. The theory was enlarged to abelian discrete groups by Kesten and Spitzer [2], to discrete groups by Kesten [4], to the real line by Ornstein [l], and to general abelian locally compact groups by Port and Stone [l]. They deal also with non-Harris recurrent random walks. Finally, in Brunel and Revuz [2] the case of general non-abelian groups is dealt with in the more general setting of special functions. Chapter 9 follows mostly the paper by Brunel and Revuz [2] with the exception of the end of $4, which comes from Port and Stone [l], but the
NOTES AND COMMENTS
355
original ideas are due to Kesten and Spitzer [2]. Many exercises are borrowed from Spitzer’s book. The Fourier criterion (2.20) is taken from Kesten and Spitzer [2]. It is also true for general groups and non-Harris recurrent random walks, as y a s shown by Ornstein [l] and Stone [3] for the real line and by Port and Stone [l] in full generality. Many results which lie beyond the scope of this book,’especiallyasymptotic theorems of probabilistic nature, have been shown for random walks. We refer the reader to the papers by Kesten, Spitzer, Ornstein, Port and Stone and Bougerol.
Chapter 10 The subject was started by Hunt in his fundamental paper [l]. He proves the basic Theorem 1.5 and Corollary 2.15 by means of probabilistic methods. In fact he proves not only the existence of a resolvent but also of a semi-group. Section 1. Aside from 1.5, the results in this section are taken mostly from Meyer [3]. Exercise 1.13 is from Bronner [l] to which we refer for further results in this line. Section 2. The main result, namely Theorem 2.10, as well as its proof, are from Meyer [3]. The proof of 2.14 follows Kondo [11 for the sufficiency and Hirsch [2] and Taylor [2] for the necessity. Another proof, outlined in Exercise 2.20, is due to Taylor [l]. It follows in part the proof given by Lion [l], who showed that the density hypothesis in 2.15 may be dropped, and gave a purely analytical proof. Actually it is known from the work of Hirsch [l] that the countability assumption on E may also be dropped. The reader is also referred to Hansen [l] and Mokobodzki and Sibony [l]. The notion of “vanishing at the boundary” was emphasized in Meyer [3]. Proposition 2.6, which clarifies the matter, is due to Taylor [l]. In view of Proposition 1.3, one could think of deriving Theorem 2.10 from Theorem 2.14 or Theorem 2.14 from Theorem 2.10 without having to repeat the proof. Unfortunately this does not seem to work. Theorem 2.14 may be generalized to the problem raised in Exercise 1.13. We refer the reader t o Hirsch [ Z ] and Taylor [2]. Section 3. This section is only intended as an introduction t o a subject where the best results are those of Sunyach [l]. Theorems 3.9 and 3.10 were given by Kondo [ Z ] for countable E and the proofs in the text are merely an extension of Kondo’s proofs to strong Feller kernels. We also refer to Oshima [ l ] and Kondo and Oshima [l].
REFERENCES
A. I . F . = A n n . Inst. Fourier; A . I. H . P . = A n n . Inst. Henri Poincare'; A . M . S . = A n n . Math. Statist.; A . P . = Ann. Probability; C.R.A.S. = Comptes Rendus Acad. Sci. Paris; I. J . M . = Illinois J . Math.; T.A.M.S. = Trans. Amer. Math. SOC.; T . P . = Theory Probability Appl. (Translation of Teoriki Primeneniya) ; Z. W . = Z . Wahrscheinlichkeitstheorie Verw. Geb.
M. ABID [13 Un thCorbme ergodique pour des processus sous-additifs e t sur stationnaires, C.R.A.S. 287 (1978) 149-152. A. F. ABRAHAMSEE [l] The Martin Potential kernel for improperly essential chains, Z.W. 18 (1971) 2.18-2.26. M. A. ACKOGLU [l] An ergodic lemma, Proc. A m . Math. SOC.16 (1965) 388-392. [23 Pointwise ergodic theorems, T.A.M.S. 136 (1966) 296-309. M. A. ACKOGLU a n d A. BRUNEL [l] Contractions on L1-spaces, T.A.M.S. 166 (1971) 315-325. M. A. ACKOGLU and R. W. SHARPE [ l ] Ergodic theories and boundaries, T.A .M.S. 132 (1968) 447-460.
M. A. ACKOGLU and L. SUCHESTON [l] A ratio ergodic theorem for superadditive processes, Z . W . 44 (1978) 269-278. K. B. ATHREYAand P. NEY [l] A new approach to t h e limit theory of recurrent Markov chains, T.A.M.S. 245 (1978) 493-501.
R. AZENCOTT [l] Espaces de Poisson des Groupes Localenaent Compacts, Lecture Notes in Math. 148 (Springer, Berlin, 1970). R. AZENCOTT and P. CARTIER [l] Martin boundaries of random walks on locally compact groups, in: Proc. 6th Berkeley Symp. on Mathematical Statistics and Probability, Vol. 111 (Univ. of California Press, Berkeley, Calif., 1970) 87-129. 356
REFERENCES
35 7
P. BALDI [l] Caractkrisation des groupes de Lie connexes rCcurrents, A.Z.H.P. 17 (1981) 281-308.
J . R. BAXTER [ l ] Ealayage in least time, unpublished. [2] Restricted mean values and harmonic functions, T.A.M.S. 167 (1972) 451-463.
J . R. RAXTERand R. V. CHACON [l] Stopping times for recurrent Markov processes, Z.J.M. 20 (1976) 467-475. c . BELLA'I'CHE-FREMONT and M. SUEUR-PONTIER [ 11 Thkorkme de renouvellement pour groupes abkliens localemeht compacts, in : Astdrisque 1 (SOC.Math. de France, Paris, 1973).
D. BLACKWELL (11 The existence of anormal chains, Bull. Amev. Math. Soc. 61 (1945) 465-468. [el A renewal theorem, Duke Math. J . 15 (1948) 145-150. [3] On transient Markov processes with a countable number of states and stationary transition probabilities, A .M.S. ?G (1955) 654-658. D. BLACKWELL and D. FREEDMAN [l] The tail a-field of a Markov chain and a theorem of Orey, A.M.S. 35 (1964) 1291-1295.
D. RLACKWELL and D. KENDALL [l] The Martin boundary for Polya's urn scheme and an application to stochastic population growth, J . A p p l . Probab. 1 (1964) 284-296. A. BLANC-LAPIERRE and R. FORTET [ 13 Thdovzes des Fonctions Aldatoires (Masson, Paris, 1953). R. M. BLUMENTHAL and R. K. GETOOR [l] Markov Processes and Potential Theory (Academic Press, New York, 1968). P. BOIJGEROL [ 11 Fonctions de concentration sur certains groupes localement compacts, 2 .W . 46 (1978) 135-157. 121 Comportement asymptotique des puissances de convolution d'une probabilitk sur un a p a c e symiitrique, in: Astdrzsque 74 (SOC.Math. France, Paris, 1980) 29-46. [3] T h k o r h e central liniite local sur certains groupes de Lie, Ann. Scz. Ecole Novm. Sup. 14 (1981) 403-432.
P. BOUGEROT. and L . ELIE [l] Sur une propriktC de compacitk du noyau potentiel associk L une probabilitk sur un groupe, Z.W. 62 (1980) 59-68. M. BRANCOVAN [l] Quelques propriktks des resolvantes rkcurrentes au sens de Harris, A .I.H.P. 9 (1973) 1-18.
358
REFERENCES
J. BRETAGNOLLE and D. DACUNHA-CASTELLE [l] Sur une classe de marches albatoires, A.Z.H.P. 3 (1967) 403-431. F. BRONNER [l] Principe d u maximum e t rbsolvantes sous-markoviennes, C 3 . A .S. 277 (1973) 221-223. [2] Mbthodes probabilistes pour la dbtermination des rdsolvantes sous-markoviennes, A.Z.H.P. 11 (3) (1975) 253-264. [3] Frontihre de Martin d’un processus rbcurrent au sens de Harris, Z.W. 44 (1978) 227-251.
A. BRUNEL (see Ackoglu) [l] Sur un lemme voisin d u lemme de E. Hopf e t sur une de ses applications, C . R . A . S . 256 (1963) 6481-5484. [2] New conditions for existence of invariant measures, in: Contributions to Ergodic and Probability, Lecture Notes in Math. 160 (Springer, Berlin, 1970). [3] Chaines abstraites de Markov vCrifiant une condition de Orey. Extention B ce cas d‘un thCor6me ergodique de M. MCtivier, Z . W . 19 (1971) 323-329. [4] Sur les sommes d’itbrbs d’un opdrateur positif, in: Thtorie Ergodique, Lecture Notes in Math. 532 (Springer, Berlin, 1976) 19-34. A. BRUNEL, P. CROPEL,Y. GUIVARC’H and M. KEANE [l] Marches albatoires rbcurrentes sur les groupes localement compacts, C.R.A .S. 275 (1972). A. BRUNEL and D. REVUZ [l] Un crithre probabiliste de compacitb des groupes, A . P . 2 (4) (1974) 475-476. [2] Quelques applications probabilistes de la quasi-compacitd, A .I.H.P. 10 (1974) 301-337. [3] Marches de Harris sur les groupes localement compacts I. A n n . Sci. Ecole Norm. Sup. 7 (1974) 273-310; 11, Hull. SOC.Math. France 104 (1976) 3-31; 111, Symfiosia Math. XXI (1977) 55-63; IV, Ann. Math. 105 (1977) 361-396; V, A.Z.H.P. 15 (1979) 205-234. [4] Sur le thCoreme du renouvellement pour les groupes non abkliens, Israel J. Math. 20 (1975) 46-56.
R . V. CHACON(see Raxter) [l] Identification of the limit of operator averages, J . Math. Mech. 11 (1962) 961-968. [2] Convergence of operator averages, in : Ergodic Theory (Academic Press, New York, 1963) 82-120.
R. V. CHACONand U. KRENGEL [l] Linear modulus of a linear operator, Proc. Amer. Math. SOC.15 (1964) 553-559.
R . V. CHACONand D. S. ORNSTEIN [l] A general ergodic theorem, I . J . M . 4 (1960) 153-160. E . CINLAR [I] Introduction to Stochastic Processes (Prentice-Hall, Englewood Cliffs, N J. 1975).
359
REFERENCES G. CHOQUET [l] Lectures in Analysis (Benjamin, New York, 1969).
G. CHOQUET and J . DXNY [l] Modbles finis en thCorie du potentiel, J . Anal. Math. 6 (1956/1957) 77-135. [2] Sur 1’Cquation de convolution p = p IF u, C.R.A.S. 260 (1960) 799-801.
K. L. CHUNG [l] The general theory of Markov processes according to Doeblin, Z.W. 2 (1964) 230-250. [2] Markov Chains with Stationavy Transition Probabilities, 2nd ed. (Springer, Berlin, 1967).
K. L.CHUNG and W. H. FUCHS [l] On the distribution of values of sums of random variables, Mem. A m . Math. SOC.6 (1951). K. L. CHUNGand D. S. ORNSTEIN [l] On the recurrence of sums of random variables, Bull. A m . Math. SOG.68 (1962) 30-32.
R . COGBURN [l] A uniform theory for sums of Markov chains transition probabilities, A . P . 3 (1975) 191-214.
r’. CREPEL [1] Fonctions spkciales pour des contractions de France, Paris, 1973).
L’,in: Astdrisgue 4 (SOC.Math.
C . DELLACHERIE and P. A. MEYER [l] ProbabzZitCs et Potentiels (Hermann, Paris, I, 1975, 11, 1980, 111, to appear).
J . DENY(see Choquet) [ l ] Familles fondamentales, noyaux associCs, A . I . F . 8 (1951) 73-101. [2] Sur 1’Cquation de convolution p = p * u, in: Sem. de Thkorie du Potentiel (Inst. Henri PoincarC, Paris, 1959/60) 41ne annCe.
c. I I E R M A N 111 A solution to a set of fundamental equations in Markov chains, PYOC.Avnev. Math. SOC.5 (1954) 332-334. Y. DERRIIENNIC [l] Sur la frontibre de Martin des processus a temps discret, A .I.H.P. 9 (3) (1973) 233-258. [2] Marche aleatoire sur le groupe libre et frontiere de Martin, Z . W . 82 (1975) 261-276. [3] Sur le thkorbrne ergodique sous-additif, C.R.A.S. 281 (1975) 985-988. [4] Lois “zero ou cleux” pour les processus de Markov. Applications aux marches alCatoires, A . I . H . P . 12 (2) (1976) 111-129. [5] Quelques applications d u thCorbme ergodique sous-additif, in : Astkrisque 74 (SOC.Math. France, Paris, 1980) 183-201.
360
REFERENCES
Y . DERRIENNIC and Y.GUIVARC'H [ 11 ThCorhme de renouvellement pour les groupes non moyennables, C.R.A .S. 277 (1973)613-615.
Y.DEKRIENNIC and M. LIN [l]Sur la tribu asymptotique des marches alkatoires sur les groupes, C.N.A.S.
M. D. DONSKER and S. R. S. VARADHAN [ 11 Asymptotic evaluation of certain Markov process expectations for large time, Comm. Pure Appl. Math. I , 27 (1975) 1-47; 111, 29 (1976)389-461.
J. L. DOOB [l] Asymptotic properties of Markoff transition probabilities, T.A .M.S. 63 (1948) 393-42 1. [2] Stochastic Processes (Wiley, New York, 1953). [3] Discrete potential theory and boundaries, J . Math. Mech. 8 (1959)433-458.
J. I>. DOOB,J. L. SNELLand R. E. WILLIAMSON [l] Application of boundary theory to sums of independent random variables, in : Contributions to Probability and Statistics (Stanford Univ. Press, Stanford, Calif., 1960). R. M. DUDLEY [l] Random walk on abelian groups, Proc. A m . Math. Soc. 13 (1962)447-450.
M. DUFLO [I] OpCrateurs potentiels des chaines et des processus de Markov irrkductibles, Bull. Soc. Math. France 98 (1970)127-163. and J. T. SCHWARZ N. DUNFORD [l]Linear Operators, Part I : General Theory (Interscience, New York, 1953). E. B. DYNKIN and M. B. MALJUTOV [l] Random walks on groups with a finite number of generators, Soviet Math. Dokl. 2 (1961)399-402 (English Transl.).
D. A. EDWARDS [l] On potentials and general ergodic theorems for resolvents, Z.W. 20 (1971)1-8. L. ELIE(see Bougerol) [l] ThCorie du renouvellement sur les groupes, Ann. Sci. Ecole Norm. Sup. 16 (1982)257-364.
J . FELDMAN [l] Subinvariant measures for Markov operators, Duke Math. J . 29 (1962)71-98. [2] Integral kernels and invariant measures for Markov transition functions, Ann. Math. Statist. 36 (1965)617-523.
W.FELLER [l] Boundaries induced by non-negative matrices, T . A . M . S . 83 (1956)19-54. [2] Non-Markovian processes with the semi-group property, A .M.S. 30 (1959) 1252-1253.
REFERENCES
361
131 A simple proof for renewal theorems, Commun. Pure A+$. Math. 14 (1961) 285-293. [4] A n Introduction to Probability Theory and its Applications, Vol. 2 (Wilpy, New York, 1966). W. FELLERand S. OREY [l] A renewal theorem, J . Math. Mech. 10 (1961) 619-624.
S . R . FOCUEL [l] The Ergodic Theory of Markov Processes (Van Nostrand, New York, 1969). [2] The ergodic theory of positive operators on continuous functions, A n n . Scuola Norm. S u p . Pisa 27 (1973) 19-51. [3] More on t h e zero-two law, Proc. Amer. Math. SOC.61 (1976) 262-264. 141 Harris operators, Zsrael J . Math. 33 (1979) 281-309. S. R. I ~ O G U E Land N. GHOUSSOIJR [I] Ornstein-MCtivier-Hrunel Theorem revisited, A .Z.H.P. 16 (1979) 293-301.
S. R. FOCUEL and M. LIN [ l ] Some ratio limit theorems for Markov operators, Z.W. 23 (1972) 55-66. R. FORTET (see Rlanc-Lapierre) [ l ] Condition d e Doeblin e t quasi-compacitk, A .Z.H.P. 14 (1978) 379-390. D. FREEDMAN (see Blackwell) [ l ] Markov Chains (Holden-Day, San Francisco, 1971). N. A. FRIEDMAN [ l ] Introduction to Ergodic Theory (Van Nostrand, New York, 1970).
H. FURSTENBERG [11 A Poisson formula for semi-simple Lie groups, A n n . Math (2) 77 (1963) 335-386. [2] Non-commuting random products, T.A .M.S. 108 (1963) 377-428. [3] Translation-invariant cones of functions on semi-simple Lie groups, Bull. A m . Math. SOC.71 (2) 271-326 (1965). L. GALLARDO a n d SCHOTT [11 Marches a l k t o i r e s sur les espaces h o m o g h e s d e certains groupes de Lie d e type rigide, i n : AstCrZsque 74 (SOC. Math. France, Paris, 1980) 149-170. A. GARSIA [l] A simple proof of E. Hopf's maximal ergodic theorem, J . Math. Mech. 14 (1965) 381-382. [2] More about the maximal ergodic lemma of Brunel, Proc. Nut. Acad. Sci. 67 (1967) 21-24. [3] Topics in Almost Everywhere Convergence, Lectures in Adv. Math. (Markham, Chicago, Ill., 1970). R . K . GETOOR (see Rlumenthal) [I] On t h e construction of kernels, in : Skmimzzre de Probabilzte's Z X , Lecture Notes in Math. 466 (Springer, Berlin, 1975).
362
REFERENCES
I. V. GIRSANOV [l] Strong Feller processes I. General properties, T.P. 5 (1960) 7-28. N. GHOUSSOUR (see Foguel) [l] Processus dc Harris abstraits, A . I. H . P . 11 (1975) 381-395
D. GRIFFEATH [l] A maximal coupling for Markov chains, Z.W. 31 (1976) 95-106. [2] Partial coupling and loss of memory for Markov chains, A .P. 4 (1976) 850-858. Y . GUIVARC’H (see Brunel, Derriennic) [ 11 Croissance polynomiale e t pCriodes des fonctions harmoniques, Bull. SOC.Math. France 101 (1973) 333-379. [2] Sur la loi des grands nombres e t le rayon spectral d’une rnarche alCatoire, in : Aste‘risque 74 (SOC.Math. France, Paris, 1980) 47-98. j3] ThCorhnes quotient pour les marches alCatoires, in: Aste’rzsque 74 (SOC.Math. France, Paris, 1980) 15-28. Y . GUIVARC’H, M. KEANI.:and B. ROYNETTE [ 11 Marches Ale‘atoires sur les Groupes de Lze, Lecture Notes in Math. 624 (Springer, Berlin, 1977).
W. HANSEN [ 13 Konstruktion von Halbgruppen und Markoffschen Prozessen, Invent. Math. 3 (1967) 179-214.
T. E. HARRIS [ l ] Recurrent Markov processes IT, A .M.S. 26 (1955) 152-153 (abstract). [2] The existence of stationary measures for certain Markov processes, in: Proc. 3rdBerkeley Symp. on Mathematical Statistics and Probability Vol. 2 (Univ. of California Press, Berkeley, Calif., 1956) 113-124. P. I-. HENNEQUIN [l] Les proccssus de Markov en cascades, A.I.H.P. (12) 18 (1963) 109-196.
H. I ~ E N N I Oand N B. ROYNETTI~ [l] Un thCor6me de dichotomie pour une marche alkatoire sur un espace homogbne, in: Aste‘risque 74 (Soc. Math. France, Paris, 1980) 99-122. C. S. HERZ [l] Lcs thCor&rnesde renouvellement, A . I . F 15 (1) (1965) 169-188. K. HEWITTand K. A. Ross [l] Abstract Harmonzc Analysis I (Springer, Berlin, 1963).
F. HIRSCH [l] Familles rCsolvantes, gCnCrateurs, cogCnCrateurs, potenticls, A . I . F . 22 (1972) 89-210. 121 Conditions necessaires e t suffisantes d’existence dc rCsolvantes, Z. W . 29 (1974) 73-85.
REFERENCES
363
F. HIRSCH and J . C. TAYLOR [ 13 Kenouvellement e t existence d e rCsolvantes, in : SCininaire de ThCovie du Potentiel, Lecture Notes in Math. 624 (Springer, Berlin, 1977).
E. HOPF [ 11 The general temporally discrete Markov process, J . Rational. Math. Mech. Anal. 3 (1054) 13-45. 121 On the ergodic theorem for positive linear operators, J . Reine Angew. Math. 106 (1960/6l) 101-106. S. HOROWITZ [ l ] Transition probabilities and contractions of Loo,Z . W . 24 (1972) 263-274. [2] Pointwise convergence of t h e iterates of a Harris-recurrent Markov operator, Zsrael J . Math. 33 (1979) 177-180. G. A. H U N T [l] Markov processes and potentials I, 11. 111, I . J . M . 1 (1957) 44-93, 316-389, 2 (1958) 151-213. [2] hlarkov chains and Martin boundaries, Z.J.M. 4 (1960) 119--132. [3] L a thCorie d u potentiel e t les processus rCcurrents, A.Z.F. 15 (1) (1965) 3-12. C. IONESCU-TULCEA [l] RICasures dans les espaces produits, Atti Accad. Naz. Lincei. Rend. 7 (1949) 208--211. R. ISAAC [ l ] Non-singular recurrent Markov processes have stationary measures, A . M . S . 35 (1964) 869-871. 121 On t h e ratio-limit theorem for Markov processes recurrent in t h e sense of Harris, Z . J . M . 11 (1967) 608-615.
J.
JACOII
[ 11 ThCorBme d e renouvellenient e t classification pour les chaines semi-rnarkoviennes, A . I . H . P . 7 (1971) 83-129.
N. C. J X I N [l] Some limit theorems for a general Markov process, Z. W . 6 (1966) 206-223. 1-21 A note o n invariant measures, A . M . S . 37 (1966) 729-732. N. C. J A I N and B. JAMISON [l] Contributions t o Doeblin’s Theory of Markov processes, Z.W. 8 (1967) 19-40. N. C. J A I N and S. OREY [l] Some properties of random walk paths, J . Math. Anal. AppZ. 43 (1973) 795-816.
B. JAMISON and S. OREY [I] Tail a-field of Markov processes recurrent in t h e sense of Harris, Z. W . 8 (1967) 41-48.
364
REFERENCES
A. DELJ U N C O [ l ] On the decomposition of a subadditive stochastic process, A . P . 6 (1977) 298-302. S. KARLIN and J. MCGREGOR [l] Random walks, I .J . M . 3 (1959) 66-81.
J . G. KEMENY [l] Representation theory for denumerable Markov chains, T . A .M.S. 126 (1966) 47-62. J . G. KEMENY and J. L. SNELL [l] Potentials for denumerable Markov chains, J . Math. Anal. Appl. 3 (1961) 196--260. [2] Notes on discrete potential theory, J . Math. Anal. A p p l . 3 (1961) 117-121. [3] Boundary theory for recurrent Markov chains, T . A . M . S . 106 (1963) 495-520. 141 A new potential operator for recurrent Markov chains, J . London Math. SOC. 38 (1963) 359-371. J . G. KEMENY,J . L. SNELLand A. W. KNAPP [I] Denumerable Markov Chains (Van Nostrand, New York, 1966).
J. H. B. KEMPERMAN [l] The Passage Problem for a Stationary Markov Chain (Chicago Univ. Press, Chicago, Ill., 1961). H. KESTEN [l] Symmetric random walks on groups, T . A . M . S . 92 (1959) 336-354. [2] Full Banach mean values on countable groups, Math. Scand. 7 (1959) 146-156. [3] Ratio theorems for random walks 11, J . Anal. Math. 11 (1963) 323-379. [4] The Martin boundary of recurrent random walks on countable groups, in: Proc. gfh Berkeley Symp. on Mathematical Statistics and Probability, Vol. I1 (Univ. of California Press, Berkeley, Calif., 1967) 51-75. H. KESTENand F. SPITZER [l] Ratio theorems for random walks I, J . Anal. Math. 11 (1963) 285-322. [2] Random walk on countably infinite Abelian groups, Acta. Math. 114 (1965) 237-265. J . F. C. KINGMAN [l] The ergodic theory of subadditivc stochastic processes, J . Zioy. Statist. SOC. Ser. B 30 (1968) 499-510. [2] Subadditive ergodic theory, A . P . 1 (1973) 883-909.
R. KONDO [l] On potential kernels satisfying the complete maximum principle, Proc. Japan. Acad. 44 (1968) 193-197. [23 A construction of recurrent Markov chains, Osaka. J . Math. G (1969) 13-28. [3] A construction of recurrent Markov chains 11, T . P . 16 (1970) 499-507.
REFERENCES
365
R. KONDO and Y. OSHIMA [ 11 A characterization of weak potential kernels for strong Feller recurrent Markov chains, Proc. SOC.Japan-USSR S y m p . Prob. Theory, Lecture Notes in Math. 330 (Springer, Berlin, 1973).
U. KRENCEL (see Chacon) [l] On the global limit behaviour of Markov chains and of general nonsingular Markov processes, Z.W. 6 (4) (1966) 302-316. [2] Un thCorkme ergodique pour les processus surstationnaires, C . R . A.S. 282 (1976) 1019-1021.
K. KRICKEBERC [ 13 Strong mixing properties of Markov chains with infinite invariant measures, in : Proc. 5th Berkeley Symp. on Mathematical Statistics and Probability (California Univ. Press, Berkeley, Calif.. 1966). [2] Mischende Transformationen auf Mannigfaltigkeiten unendlichen MaBes, Z.W. 7 (1967) 235-247.
A. G. KUROSH [l] The Theory of Groups (Chelsea, New York, 1955)
J . LAMPERTI and J. L. SNELL [13 Martin boundaries for certain Markov chains, J . Math. SOC.Japan 16 (1963) 113-128.
M. I,. LEVITAN [ 11 Some ratio limit theorems for a general state space Markov process, Z . W . 1G (1970) 29-50. [2] A generalized Doeblin ratio limit theorem, A.M.S. 42 (1971) 9 0 6 9 1 1 .
M. LIN (see Derriennic, I’oguel) [ l ] Mixed ratio limit theorems for Markov processes, Israel J . Math. 8 (1970) 357-366. 121 Mixing for Markov operators, 2.W . 18 (1971) 231-242. j3] On quasi-compact operators, A .P. 2 (1!)74) 464-475. 141 On the “zero-two” law for conservative Markov processes, Z. W . 61 (1982) 5 13-525.
G. LION [l] Familles d’opkrateurs et frontikre en thkorie du potentiel, A.Z.F. 16 (1966) 389-453.
M. LOEVE 113 Probability Theory, 2“* edition (Van Nostrand, New York, 1960).
R. M. LOYNES [I] Products of independent random elements in a topological group, Z.W. 1 (1963) 446-455.
366
REFERENCES
N. MAICRET [l] ThCorkme de limite centrale fonctionnel pour une chaine de Markov rCcurrente au sens de Harris et positive, A . Z . H . P . 14 (1978) 425-440. J . F. MERTENS,E. SAMUEL-CAHN and S. ZAMIR [11 Necessary and sufficient conditions for recurrence and transience of Markov chains, J . Apfil. Probability 15 (1978) 848-851.
M. METIVIER [l] Existence of an invariant measure and an Ornstein's ergodic theorem, A .M.S. 40 (1969) 74-96. [2] Thkorkme limite quotient pour les chaines de Markov rCcurrentes au sens de Harris, A . I . H . P . 8 (2) (1972) 93-105. P. A. MEYER [l] ThCorie ergodique et potentiels, A . I . F . 16 (1) (1965) 89-102. [2] Probability and Potevitials (Blaisdell, Waltham, Mass., 1966). [3] CaractCrisation des noyaux potentiels des semi-groupes discrets, A .Z.F. 16 (2) (1966) 225-240. [4] Les resolvantes fortement Felleriennes d'aprks Mokobodzki, in: S t m . de Probabilitts 11, Lecture Notes in Math. 61 (Springer, Berlin, 1968). [5] Representation intCgrale des fonctions excessives, rksultats de Mokobodzki, in: Stm. de Probabilitts V , Lecture Notes in Math. 191 (Springer, Berlin, 1971). [6] Deux petits rksultats d u thkorie d u potentiel, in: S t m . de Probabilitts V , Lecture Notes in Math. 191 (Springer, Berlin, 1971). [7] Travaux de H. Rost et thCorie du balayage, in: S t m . de Probabilitts V , Lecture Notes in Math. 191 (Springer, Berlin, 1971). [S] Solutions de 1'Cquation de Poisson dans le cas ¤t, in: Skm. de Probabilitds V , Lecture Notes in Math. 191 (Springer, Berlin, 1971).
P. A. MEYER and M. YOR [l] Sur l'extension d'un t h C o r h e de Doob 5, un noyau a-fini, in: Stminaire de Probabilitts X I I , Lecture Notes in Math. 649 (Springer, Berlin, 1978). G. MOKOBODZKI [l] Noyaux absolument mesurables et opkrateurs nuclkaires, C.R.A.S. 270 (1970) 1673-1675. [2] DensitC relative de deux potentiels comparables, in : Stminaire de Probabilitds I V , Lecture Notes in Math. 124 (Springer, Berlin, 1970).
S. T. C. MOY [l] Period of an irreducible positive operator, I . J . M . 11 (1967) 24-39. [2] I-continous Markov chains, T . A . M . S . 117 (1965) 68-99. J. NEVEU [l] Potentiels markoviens'discrets, A n n . Univ. Clermont 24 (1964) 37-89. [2] Mathematical Foun.dations of the Calculus of Probability (Holden Day, San Francisco, Calif., 1966).
REFERENCES
367
[3] Relations entre la thCorie des martingales et la thCorie ergodique, A . I . F . 16 (1965) 31-42. [4] Existence of bounded invariant measures in ergodic theory, in : Symp. 5ffi Berkeley Symp. on Mathematical Statistics and Probability (Univ. of California Press, Berkeley, Calif., 1966). [5] Potentiel markovien recurrent des chaines de Harris, A . I . F . 22 (2) (1972) 85-130. [6] Sur I'irr6ductibilitC des chaines de Markov, A . I . H . P . 8 (3) (1972) 249-254. [7] GCnCralisation d'un thCor&melimite-quotient, in: Trans. 6'" Prague Conf. on Information Theory, Statistical Decision Functions, Random Processes (Czech. Acad. Sci., Prague, 1973). [8]The filling scheme and the Chacon-Ornstein theorem, Israel J . Math. 3 3 (1979) 368-377. 1'. NEY and F. SPITZER [ 11 The Martin boundary for random walk, T . A.M.S. 121 (1966) 116-132.
E. NUMMELIN [ I ] A splitting technique for Harris recurrent Markov chains, Z . W . 43 (1978) 309-3 18. [2] Strong ratio limit theorems for precurrent Markov chains, A . P . 7 (1979) 639-650. [3] On the Poisson equation for precurrent Markov chains, to appear. E. NUMMELIN and R . L. TWEEDIE [ l ] Geometric ergodicity and R-positivity for general Markov chains, A . P . B (1978) 404-420. S . OREY(see Feller, Jain and Jamison) [l] Recurrent Markov chains, Pacific J . Math. 9 (1959) 805-827. [2] Sums arising in the theory of Markov chains, Proc. A m . Math. SOC.12 (1961) 847-856. [3] An ergodic theorem for recurrent Markov chains, Z . W . 1 (1962) 174-176. [4] Potential kernels for recurrent Markov chains, J . Math. Anal. A p p l . 8 (1964) 104-132. [5] Limit Theorems for Markov Chain Transition Probabilities (Van Nostrand, New York, 1971). D. S. ORNSTEIN (see Chacon and Chung) [l] Random walks I, 11, T . A . M . S . 138 (1969) 1-43, 45-60. 121 The sums of the iterates of a positive operator, Advances in Probability and Related lopics 2 (1970) 85-115. D. S. ORNSTEIN and L. SUCHESTON [l] An operator theorem on L1-convergence to zero with applications to Markov kernels, A . M . S . 41 (1970) 1631-1639.
368
REFERENCES
Y . OSHIMA(see Kondo) [l] A necessary and sufficient condition for a kernel to be a weak potential kernel of a recurrent Markov chain, Osaka J . Math. 6 (1969) 29-37. J . C. OXTOBY [l] Ergodic sets, Bull. Amev. Math. Soc. 68 (1952) 116-136. F. PAPANCELOU [l] A martingale approach to the convergence of iterates of a transition function, Z.W. 37 (1977) 211-226. J . PITMAN [ 11 Uniform rates of convergence for Markov chain transition probabilities, Z . W . 29 (1974) 199-227.
s. c. P O R T [l] Limit theorems involving capacities, J . Math. Mech. 16 (1966) 805-832. S. C. PORTand C. J . STONE [l] Potential theory of random walks on abelian groups, Acta Math. 122 (1969) 19-1 14.
A . RAUGI [ 11 Fonctions harmoniqucs sur les groupes localement compacts k base dCnombrable, Adem. SOC.Math. France 64 (1977) 5-118.
D. REVUZ(see Brunel) [l] ThCorhmes liniites pour les rksolvantes rkcurrentes, Rend. Civc. Mat. PaZevmo
19 (1970) 294-300. [2] Sur la theorie du potentiel pour les processus de Markov rkcurrents, A .Z.F. 21 (1971) 245-262. [3] Le principe semi-complet du maximum, in: Stm. de ProbabiZitts V I , Lecture Notes in Math. 258 (Springer, Berlin, 1972). 143 On the filling scheme for recurrent Markov chains, Duke Math. J . 45 (1978) 681-689. [ 5 ] Sur la dCfinition des classes cycliques dcs chaines de Harris, Israel J . Math. 33 (1979) 378-383. 161 Sur le thCorbme de dichotomie d’Hennion-Roynette, to appear. J . ROUDIER [ l j Chaine de Markov p-continue I? l’infini, A . I . H . 8 (3) (1972) 241-248.
H . ROST [ 13 Darstellung einer Ordnung von Massen durch Stoppzeiten, 2.W . 16 (1970) 19-28. [2] Markoff-Ketten bei sich fullenden Liichern im Zustandsraum, A . I . F . 21 (1) (1971) 253-270.
REFERENCES
369
B. ROYNETTE (see Guivarc’h) [ l ] Marches alkatoires sur le groupe des dkplacements de Rd, Z.W. 31 (1974) 25-34, [2] Thkorkme central-limite pour le groupe des dkplacements de Rd, A.Z.H.P. 10 (1974) 391-398. W. RUDI N [l] Fourier Analysis on Groups (Interscience, New York, 1967).
F. SPITZER (see Kesten, Ney) 11) Principle of Random Walks (Van Nostrand, New York, 1964).
A. J. STAM [I] On shifting iterated convolutions I, Conzposztio Math. 17 (1966) 268-280
C. J. STONE(see Port) [l] On absolutely continuous components and renewal theory, A.M.S. 37 (1966) 271-275. 121 Ratio limit theorem for random walks on groups, T . A . M . S . 1% (1966) 86-100. [3] On the potential operator for one-dimensional recurrent random walks, T . A . M . S . 136 (1969) 413-426.
c. SUNYACH [ 13 Principes du maximum rkcurrent e t construction de rCsolvantes rkcurrentes, C . R . A . S . 282 (1976) 747-750. [2] CapacitCs e t thkorie du renouvellement I, Bull. Soc. Math. France 109 (1981) 283-296.
J . C. TAYLOR (see Hirsch) [l] On the existence of sub-Markovian resolvents, Invent. Math. 17 (1972) 8 , 5 9 3 . [2) A characterization of the kernels lirnAloV , for submarkovian resolvents ( V A ) , A . P . 3 (1975) 355-357.
P. TUOMINEN and R. I,. TWEEDIE [I] Markov chains with continuous components. Proc. London Math. Soc. 38 (1979) 89-1 14.
R . I,. TWEEDIE (see Nurnmelin, Tuominen) [ l ] R-theory for Markov chains on a general state space I and 11, A . P . 2 (1974) 840-864 and 865-878. [Z] Criteria for classifying general Markov chains, Adv. Appl. Probability 8 (1976) 737-77 1 . 131 Topological aspects of Doeblin decompositions for Markov chains, 2.W . 4G (1979) 299-305.
T. UENO [l] Some limit theorems for temporally discrete Markov processes, J . Fuc. Sci. Unzv. I’okyo (I) 5 (1957).
370
REFERENCES
T. WATANABE [13 On the theory of Martin boundaries induced by countable Markov processes, Ale?%. Coll. S c i . Unzv. Kyoto (A)33 (1960)39-108. [2]On balayees of excessive measures and functions with respect to resolvents, in: Skm. de Probubzlatis V , Lecture Notes in Math. 191 (Springer, Berlin, 1971). W. WINKLER [l]A note on a continuous parameter zero-two law, A . P . 1 (1973)341-344.
K. YOSIDA and S. KAKUTANI [ 13 Operator-theoreticaltreatment of Markov’s process and mean ergodic theorem, Ann. Math. 42 (1941)188-228.
INDEX OF NOTATION
LA nA
G GA
Gff ‘h
2 7 8 10 12 18 17 19 23 23 23,24 25 25 28 26 41 44 51 51 54 62 63 74 75 77
%*Gfl
G T, T*
Ef cf
c,D
T+ h (yk
4 .r8
w,w,,,r
R
4 p; 7
X RA
C.M.P. R.C.M.P. R.S.C.M. S.C.M. T.P. U
371
98 100 118 122 125 124 172 171 190 56 186 217 56,233 256 246 298 294 328 324 324 339 339 12 71
INDEX OF TERMS
Directly Riemann integrable 170 Discrete, d. chain 80; d. group 80 Dissipative part 124 Doeblin, D. condition 210;D. theorem 144 Duality, Markov chainsin d. 62; kernels in d. 6 2
Absorbing sets 47 Adapted random walks 98 Admissible o-algebras 13 Almost-sure 19 Aperiodic, a. discrete chains 86; a. Harris chains 198; a. random walks 98 Asymptotic, a. events 186; a. random variables 186; equivalent a. random variables 187
Equilibrium, e. potential 56; e. function 268; e. measure 268 Ergodic, maximal e. lemma 122; e. conservative contraction 126; e. Harris chain 197; e. point transformation 138 Exact minorant 153 Excessive, e. measure 60; e. function 78 Extremal measures 253
Balayage, b. operator 25; b. sequence 68; b. theorem 49 Birth and death chain 48 Birkhoff theorem 136 Brownian motion, resolvent of B. m. 77 Brunel's maximal ergodic lemma 145
Fatou theorem 247 Feller, F. kernels 36; strong F. kernels 36 Filling scheme 70
Canonical, c. probability space 16; c. Markov chain 19 Capacity, transient c. 64; recurrent c. 277 Cemetary 16 Chacon-Ornstein theorem 132 Charge 217 Choquet-Deny theorem 161 Co (prefix co) 62 Conditional probability distribution 122 Conservative, c. contraction 124; c. part 124 Continuous, left and right c. random walks 103 Contraction, positive c. 12 Convergence, strong c. 202; uniform c. 201 Convolution kernel 10 Cyclic, c. class 196; c. decomposition of a Harris chain 196
Galton-Watson chain 22 Haar measure 7 Harmonic functions 40 Harris chains, chains recurrent in the sense of H. 91 Homogeneous Markov chains 15 Hopf decomposition 124 Induced, i. chain 27; i. operator 118 Invariant, i. events 56; i. functions 126; i. measures 60; i. random variables 56; i. sets 126 Integral kernel 9 Irreducible, i. recurrent discrete chain 82; v-i. chain 87; u-essentially i. chain 140; i. random walk 98 3 72
INDEX OF TERMS
373
Kac recurrence theorem 128 Kernel, bounded k. 8; o-finite k. 8; proper k. 8; positive k. 8; potential k. 41; recurrent potential k. 217; integral k. 9; Feller k. 36; strong Feller k. 36 Killed chain 27
Poisson, P. equation 40; P. integral representation 233; P. space 241 Positive, p. contraction 12; p. Harris chain 197 Potential, p. kernel 41;p. kernel of a Harris chain 217; p. kernel of a Harris random walk 301
Law of large numbers 138 LCCB space 6 Linear modulus 122
Quasi-compact 202
Markov, M. chain 14; homogeneous M. chain 15; M. property 20; strong M. property 24 Markovian 12 Martingale 3 Martin, M. transient boundary 243; M. recurrent boundary 279 Maximal ergodic lemma 122 Maximum, m. principle 46; complete m. principle 46; reinforced m. principle 46; reinforced semi-complete m. principle 339; semi-complete m. principle 339 Mean 205; pure m. 205; m. of a random walk 171 Measure preserving point transformation 6 0 Mixing, strongly m. point transformation 199 Modulus, linear m. 122 Moment, first m. of a random walk 171 Monotone class theorem 4 Normal chain 272; weakly n. c. 281 Null Harris chain 197 Operate, group operating on 11 Period of a Harris chain 1 9 8 Periodic Harris chain 198 Poincark recurrence theorem 128 Point transformation 11
Random walk 29; symmetric I. w. 31 Range of a random walk 158 Ratio-limit theorem 230 Recurrent, r. group 112; I. in the sense of Harris 91; r. random walk 98; r. sets 57; r. states of a discrete chain 80 R-recurrence 97 Reduced function 49 Renewal, I. chain 86; I. theorem 180 Regular, I. function 234; I. set 234; I. conditional probability distribution 122 Resolvent, r. equation 74; sub-markovian I. 74; proper I. 75 Riesz decomposition 41 Robin’s constant 268 Shift operator 19 Space time, s. t. chain 21; s. t. harmonic functions 187 Special, s. function 212; s. set 212 Spread-ou t 102 Superharmonic functions 40 Supermedian, s. function 75; s. measures 79 Subadditive sequence 152; integrable s. s. 152 Starting measure 1 5 Stopped chain 28 Stopping time 23; randomized s. t. 67 Symmetric, s. events 33; s. random walk 168
374
INDEX OF TERMS
Time, death t. 23; first hitting t. 23; first return t. 23; stopping t. 23; t. constant 152 Trajectory 17 Transient, t. random walk 98; t. set 57; t. state 80 Transition probability 12; (sub-Imarkovian t.p. 12 Translation on the integers 47
Transient invariant 29 Type, t. of a transient random walk 168; t. of a recurrent random walk 297 Vague topology 6 Walk, left and right random w. 29 Zero-two laws 191
This Page Intentionally Left Blank
