One Thousand Exercises in Probability

GEOFFREY GRIMMETI and DAVID STIRZAKER

One Thousand Exercises in Probability GEOFFREY R. GRIMMETT Statistical Laboratory, University of Cambridge and

DAVID R. STIRZAKER Mathematical Institute, University of Oxford

OXFORD UNIVERSITY PRESS

OXFORD UNIVERSITY PRESS

Great Clarendon Street, Oxford

OX2 6DP

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide in Oxford New York Athens Auckland Bangkok Bogota Buenos Aires Cape Town Chennai Dar es Salaam Delhi Florence Hong Kong Istanbul Karachi Kolkata Kuala Lurnpur Madrid Melbourne Mexico City Murnbai Nairobi Paris Slio Paulo Shanghai Singapore Taipei Tokyo Toronto Warsaw with associated companies in Berlin Ibadan Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries Published in the United States by Oxford University Press Inc., New York

© Geoffrey R. Grimmett and David R. Stirzaker 2001 The moral rights of the author have been asserted Database nght Oxford University Press (maker) First published 2001 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly pemiitted by law, or under terms agreed with the appropriate reprographics rights organization. Enquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above You must not circulate this book in any other binding or cover and you must impose this same condition on any acquirer A catalogue record for this title is available from the British Library Library of Congress Cataloging in Publication Data Data available ISBN 0 19 857221 2 10 9 8 7 6 5 4 3 2 1 Typeset by the authors Printed in Great Britain on acid-free paper by BiddIes Ltd, Guildford & King's Lynn

Preface This book contains more than 1000 exercises in probability and random processes, together with their solutions. Apart from being a volume of worked exercises in its own right, it is also a solutions manual for exercises and problems appearing in our textbook Probability and Random Processes (3rd edn), Oxford University Press, 2001, henceforth referred to as PRP. These exercises are not merely for drill, but complement and illustrate the text of PRP, or are entertaining, or both. The current volume extends our earlier book Probability and Random Processes: Problems and Solutions, and includes in addition around 400 new problems. Since many exercises have multiple parts, the total number of interrogatives exceeds 3000. Despite being intended in part as a companion to PRP, the present volume is as selfcontained as reasonably possible. Where knowledge of a substantial chunk of bookwork is unavoidable, the reader is provided with a reference to the relevant passage in PRP. Expressions such as 'clearly' appear frequently in the solutions. Although we do not use such terms in their Laplacian sense to mean 'with difficulty', to call something 'clear' is not to imply that explicit verification is necessarily free of tedium. The table of contents reproduces that of PRP; the section and exercise numbers correspond to those of PRP; there are occasional references to examples and equations in PRP. The covered range of topics is broad, beginning with the elementary theory of probability and random variables, and continuing, via chapters on Markov chains and convergence, to extensive sections devoted to stationarity and ergodic theory, renewals, queues, martingales, and diffusions, including an introduction to the pricing of options. Generally speaking, exercises are questions which test knowledge of particular pieces of theory, while problems are less specific in their requirements. There are questions of all standards, the great majority being elementary or of intermediate difficulty. We ourselves have found some of the later ones to be rather tricky, but have refrained from magnifying any difficulty by adding asterisks or equivalent devices. If you are using this book for self-study, our advice would be not to attempt more than a respectable fraction of these at a first read. We pay tribute to all those anonymous pedagogues whose examination papers, work assignments, and textbooks have been so influential in the shaping of this collection. To them and to their successors we wish, in tum, much happy plundering. If you find errors, try to keep them secret, except from us. If you know a better solution to any exercise, we will be happy to substitute it in a later edition. We acknowledge the expertise of Sarah Shea-Simonds in preparing the TEXscript of this volume, and of Andy Burbanks in advising on the front cover design, which depicts a favourite confluence of the authors. G.R.G. D. R. S.

Cambridge and Oxford April2001

v

Life is good for only two things, discovering mathematics and teaching it. Simeon Poisson In mathematics you don't understand things, you just get used to them. John von Neumann Probability is the bane of the age. Anthony Powell Casanova's Chinese Restaurant The traditional professor writes a, says b, and means c; but it should be d. George P6lya

Contents

1

Solutions

1 2 3

135 135 137 139

4 4

140 141

10 10

11 11 12

151 152 152 152 153

12

154

16 16 17 18 19 19 20 21 22 23 23

158 158 161 162 165 165 167 169 170 171 172

Events and their probabilities 1.1

1.2 1.3

1.4 1.5 1.6 1.7 1.8 2

Questions

Introduction Events as sets Probability Conditional probability Independence Completeness and product spaces Worked examples Problems

Random variables and their distributions

2.1 2.2 2.3 2.4 2.5 2.6 2.7

Random variables The law of averages Discrete and continuous variables Worked examples Random vectors Monte Carlo simulation Problems

3 Discrete random variables 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11

Probability mass functions Independence Expectation Indicators and matching Examples of discrete variables Dependence Conditional distributions and conditional expectation Sums of random variables Simple random walk Random walk: counting sample paths Problems vii

Contents

4

Continuous random variables

4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12 4.13 4.14

5

29 29 30 30 31 32 33 34 35 36 36 37 38 39

187 188 189 190 191 193 195 199 201 202 204 205 206 209

48 49 50 51 52 52 53 54 55 56 57 57

230 232 234 238 239 241 241 244 247 249 253 254

64

65 66 67 68 69 70 71 72

272 275 276 281 286 287 289 290 293

73 74 74 75 76

297 299 301 303 304

Generating functions and their applications

5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12 6

Probability density functions Independence Expectation Examples of continuous variables Dependence Conditional distributions and conditional expectation Functions of random variables Sums of random variables Multivariate normal distribution Distributions arising from the normal distribution Sampling from a distribution Coupling and Poisson approximation Geometrical probability Problems

Generating functions Some applications Random walk Branching processes Age-dependent branching processes Expectation revisited Characteristic functions Examples of characteristic functions Inversion and continuity theorems Two limit theorems Large deviations Problems

Markov chains

6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13 6.14 6.15

Markov processes Classification of states Classification of chains Stationary distributions and the limit theorem Reversibility Chains with finitely many states Branching processes revisited Birth processes and the Poisson process Continuous-time Markov chains Uniform semigroups Birth-death processes and imbedding Special processes Spatial Poisson processes Markov chain Monte Carlo Problems viii

Contents

7

Convergence of random variables 7.1 Introduction Modes of convergence 7.2 Some ancillary results 7.3 7.4 Laws of large numbers 7.5 The strong law The law of the iterated logarithm 7.6 7.7 Martingales Martingale convergence theorem 7.8 Prediction and conditional expectation 7.9 7.10 Uniform integrability 7.11 Problems

85 85 86 88 88 89 89 90 90 91 91

323 323 326 330 331 331 331 332 333 334 336

97 97 98 99

349 350 351 352

99

353

101 101 102 102 103 103 104

355 356 357 359 359 360 361

10 Renewals The renewal equation 10.1 Limit theorems 10.2 10.3 Excess life 10.4 Applications Renewal-reward processes 10.5 10.6 Problems

107 107 108 108 109 109

370 371 373 375 375 376

11 Queues 11.1 Single-server queues MIMI 1 11.2 MIGl1 11.3 11.4 GIMl1 GIGl1 11.5 11.6 Heavy traffic 11.7 Networks of queues 11.8 Problems

112 113 113 113 114 114 115

382 384 384 385 386 386 387

8

9

Random processes 8.1 Introduction Stationary processes 8.2 Renewal processes 8.3 Queues 8.4 8.5 The Wiener process 8.6 Existence of processes 8.7 Problems Stationary processes 9.1 Introduction Linear prediction 9.2 Autocovariances and spectra 9.3 Stochastic integration and the spectral representation 9.4 The ergodic theorem 9.5 9.6 Gaussian processes Problems 9.7

IX

Contents

12 Martingales 12.1 Introduction 12.2 Martingale differences and Hoeffding's inequality 12.3 Crossings and convergence 12.4 Stopping times 12.5 Optional stopping 12.6 The maximal inequality 12.7 Backward martingales and continuous-time martingales 12.8 Some examples 12.9 Problems

118 119 119 120 120

396 398 398 399 400

121

403

121

403

13 Diffusion processes Introduction 13.1 13.2 Brownian motion 13.3 Diffusion processes First passage times 13.4 13.5 Barriers 13.6 Excursions and the Brownian bridge 13.7 Stochastic calculus The Ito integral 13.8 Ito's formula 13.9 13.10 Option pricing 13.11 Passage probabilities and potentials 13.12 Problems

126 127 127 127 127 128 129 129 130 130

411 413 413 413 415 416 417 418 420 420

Bibliography

429

Index

430

x

1

Events and their probabilities

1.2 Exercises. Events as sets 1.

Let {Ai: i

E

I} be a collection of sets. Prove 'De Morgan's Laws't:

(l)Aiy

=

I

2.

(8

nAr, I

I

Ai

Y=

l)Ar. I

Let A and B belong to some a-field :F. Show that Tcontains the sets A n B, A \ B, and A ~ B.

A conventional knock-out tournament (such as that at Wimbledon) begins with 2n competitors and has n rounds. There are no play-offs for the positions 2, 3, ... , 2n - 1, and the initial table of draws is specified. Give a concise description of the sample space of all possible outcomes.

3.

4. Let Tbe a a-field of subsets of Q and suppose that BET. Show that fJ, = {A n B : A a -field of subsets of B.

E

J1 is a

Which of the following are identically true? For those that are not, say when they are true. n C) = (A U B) n (A U C); (b) An (B n C) = (A n B) n C; (c) (A U B) n C = AU (B n C);

5.

(a) AU (B

(d) A \ (B

n C)

= (A \

B) U (A \ C).

1.3 Exercises. Probability

i

-b : :

1. LetA and B be events with probabilities IP'(A) = andlP'(B) = j. Show that IP'(AnB) :::: j, and give examples to show that both extremes are possible. Find corresponding bounds for IP'(A U B). 2. A fair coin is tossed repeatedly. Show that, with probability one, a head turns up sooner or later. Show similarly that any given finite sequence of heads and tails occurs eventually with probability one. Explain the connection with Murphy's Law. 3. Six cups and saucers come in pairs: there are two cups and saucers which are red, two white, and two with stars on. If the cups are placed randElmly onto the saucers (one each), find the probability that no cup is upon a saucer of the same pattern.

t Augustus De Morgan is well known for having given the first clear statement of the principle of mathematical induction. He applauded probability theory with the words: "The tendency of our study is to substitute the satisfaction of mental exercise for the pernicious enjoyment of an immoral stimulus". 1

[1.3.4]-[1.4.5] 4.

Events and their probabilities

Exercises

Let A 1, A 2, ... , An be events where n

lP'(

U

Ai)

i=l

= Z)I"(Ai) i

2:)"(Ai i<j

~

2, and prove that

L

n Aj) +

lP'(Ai

n Aj n Ak)

i<j
- ... + (_l)n+llP'(Al n A2 n··· nAn). In each packet of Com Flakes may be found a plastic bust of one of the last five Vice-Chancellors of Cambridge University, the probability that any given packet contains any specific Vice-Chancellor being ~, independently of all other packets. Show that the probability that each of the last three Vice-Chancellors is obtained in a bulk purchase of six packets is 1 - 3(~)6 5.

Let A r , r ~ 1, be events such that lP'(Ar)

=

+ 3(~)6 _

1 for all r. Show that lP' (n~l Ar)

=

(~)6. 1.

6.

You are given that at least one of the events A r , 1 :::: r :::: n, is certain to occur, but certainly no more than two occur. IflP'(A r ) = p, and lP'(Ar n As) = q, r =I- s, show that p ~ lin and q :::: 21n. 7. You are given that at least one, but no more than three, of the events A r , 1 :::: r :::: n, occur, where n ~ 3. The probability of at least two occurring is If lP'(Ar) = p, lP'(Ar n As) = q, r =I- s, and lP'(Ar n As nAt) = x, r < s < t, show that p ~ 3/(2n), and q :::: 41n.

5;.

1.4 Exercises. Conditional probability 1. Prove that lP'(A I B) = lP'(B lP'(A), then lP'(B I A) > lP'(B). 2.

I A)lP'(A)/lP'(B) whenever lP'(A)lP'(B) =I- O. Show that, if lP'(A I B) >

For events A 1, A2, ... , An satisfying lP'(A 1 n A2

n ... n An- d > 0, prove that

3. A man possesses five coins, two of which are double-headed, one is double-tailed, and two are normal. He shuts his eyes, picks a coin at random, and tosses it. What is the probability that the lower face of the coin is a head? He opens his eyes and sees that the coin is showing heads; what is the probability that the lower face is a head? He shuts his eyes again, and tosses the coin again. What is the probability that the lower face is a head? He opens his eyes and sees that the coin is showing heads; what is the probability that the lower face is a head? He discards this coin, picks another at random, and tosses it. What is the probability that it shows heads? 4. What do you think of the following 'proof' by Lewis Carroll that an urn cannot contain two balls of the same colour? Suppose that the urn contains two balls, each of which is either black or white; thus, in the obvious notation, lP'(BB) = lP'(BW) = lP'(WB) = lP'(WW) = We add a black ball, so that lP'(BBB)

= lP'(BBW) = lP'(BWB) = lP'(BWW) = i.

that the ball is black is (using conditional probabilities) 1·

i.

Next we pick a ball at random; the chance

i + ~ . i + ~ . ~ + j- . i = ~. However, if

there is probability ~ that a ball, chosen randomly from three, is black, then there must be two black and one white, which is to say that originally there was one black and one white ball in the urn. 5. The Monty Hall problem: goats and cars. (a) Cruel fate has made you a contestant in a game show; you have to choose one of three doors. One conceals a new car, two conceal old goats. You

2

Exercises

Independence

[1.4.6]-[1.5.7]

choose, but your chosen door is not opened immediately. Instead, the presenter opens another door to reveal a goat, and he offers you the opportunity to change your choice to the third door (unopened and so far unchosen). Let p be the (conditional) probability that the third door conceals the car. The value of p depends on the presenter's protocol. Devise protocols to yield the values p = p = ~.

i,

[i,

Show that, for cx E ~], there exists a protocol such that p = cx. Are you well advised to change your choice to the third door? (b) In a variant of this question, the presenter is permitted to open the first door chosen, and to reward you with whatever lies behind. If he chooses to open another door, then this door invariably conceals a goat. Let p be the probability that the unopened door conceals the car, conditional on the presenter having chosen to open a second door. Devise protocols to yield the values p = 0, p = 1, and deduce that, for any cx E [0,1], there exists a protocol with p = cx. 6. The prosecutor's fallacyt. Let G be the event that an accused is guilty, and T the event that some testimony is true. Some lawyers have argued on the assumption that JP'(G I T) = JP'(T I G). Show that this holds if and only if JP'(G) = JP'(T). 7. Urns. There are n urns of which the rth contains r - 1 red balls and n - r magenta balls. You pick an urn at random and remove two balls at random without replacement. Find the probability that: (a) the second ball is magenta; (b) the second ball is magenta, given that the first is magenta.

1.5 Exercises. Independence 1. Let A and B be independent events; show that A c, B are independent, and deduce that A c, B C are independent. 2. We roll a die n times. Let Ai} be the event that the ith and jth rolls produce the same number. Show that the events {Ai} : 1 .5 i < j .5 n} are pairwise independent but not independent. 3. A fair coin is tossed repeatedly. Show that the following two statements are equivalent: (a) the outcomes of different tosses are independent, (b) for any given finite sequence of heads and tails, the chance of this sequence occurring in the first m tosses is 2- m , where m is the length of the sequence. 4. Let Q = {1, 2, ... , p} where p is prime, Fbe the set of all subsets of Q, and JP'(A) = IAll p for all A E F. Show that, if A and B are independent events, then at least one of A and B is either 0 or Q.

5. Show that the conditional independence of A and B given C neither implies, nor is implied by, the independence of A and B. For which events C is it the case that, for all A and B, the events A and B are independent if and only if they are conditionally independent given C? 6. Safe or sorry? Some form of prophylaxis is said to be 90 per cent effective at prevention during one year's treatment. If the degrees of effectiveness in different years are independent, show that the treatment is more likely than not to fail within 7 years. 7. Families. Jane has three children, each of which is equally likely to be a boy or a girl independently of the others. Define the events: A

= {all the children are of the same sex},

B = {there is at most one boy}, C = {the family includes a boy and a girl}. tThe prosecution made this error in the famous Dreyfus case of 1894.

3

[1.5.8]-[1.8.3] (a) (b) (c) (d)

Exercises

Events and their probabilities

Show that A is independent of B, and that B is independent of C. Is A independent of C? Do these results hold if boys and girls are not equally likely? Do these results hold if Jane has four children?

8. Galton's paradox. You flip three fair coins. At least two are alike, and it is an evens chance that the third is a head or a tail. Therefore JP'(all alike) = Do you agree?

1.

9. Two fair dice are rolled. Show that the event that their sum is 7 is independent of the score shown by the first die.

1.7 Exercises. Worked examples 1. There are two roads from A to B and two roads from B to C. Each of the four roads is blocked by snow with probability p, independently of the others. Find the probability that there is an open road from A to B given that there is no open route from A to C. If, in addition, there is a direct road from A to C, this road being blocked with probability p independently of the others, find the required conditional probability. 2. Calculate the probability that a hand of 13 cards dealt from a normal shuffled pack of 52 contains exactly two kings and one ace. What is the probability that it contains exactly one ace given that it contains exactly two kings? 3. A symmetric random walk takes place on the integers 0, 1, 2, ... , N with absorbing barriers at 0 and N, starting at k. Show that the probability that the walk is never absorbed is zero. 4.

The so-called 'sure thing principle' asserts that if you prefer x to y given C, and also prefer x to

y given CC, then you surely prefer x to y. Agreed?

5. A pack contains m cards, labelled 1,2, ... , m. The cards are dealt out in a random order, one by one. Given that the label of the kth card dealt is the largest of the first k cards dealt, what is the probability that it is also the largest in the pack?

1.8 Problems 1. (a) (b) (c) (d)

A traditional fair die is thrown twice. What is the probability that: a six turns up exactly once? both numbers are odd? the sum of the scores is 4? the sum of the scores is divisible by 3?

2. (a) (b) (c) (d)

A fair coin is thrown repeatedly. What is the probability that on the nth throw: a head appears for the first time? the numbers of heads and tails to date are equal? exactly two heads have appeared altogether to date? at least two heads have appeared to date?

3. Let Fand fJ, be a-fields of subsets of Q. (a) Use elementary set operations to show that Fis closed under countable intersections; that is, if Ai, A2, ... are in:F, then so is Ai. (b) Let Je = Fn fJ, be the collection of subsets of Q lying in both Fand fJ,. Show that Jeis a a-field. (c) Show that FU fJ" the collection of subsets of Q lying in either For fJ" is not necessarily a a-field.

ni

4

Exercises

Problems

[1.8.4]-[1.8.14]

4. Describe the underlying probability spaces for the following experiments: (a) a biased coin is tossed three times; (b) two balls are drawn without replacement from an urn which originally contained two ultramarine and two vermilion balls; (c) a biased coin is tossed repeatedly until a head turns up. 5.

Show that the probability that exactly one of the events A and B occurs is lP'(A)

6.

+ lP'(B) -

2lP'(A

n B).

Prove that lP'(A U B U C) = I - lP'(A C I B C n CC)lP'(B C I CC)lP'(CC).

7. (a) If A is independent of itself, show that lP'(A) is 0 or 1. (b) If lP'( A) is 0 or I, show that A is independent of all events B. 8. Let Fbe a cr-field of subsets of Q, and suppose lP' : F -+ [0, 1] satisfies: (i) lP'(Q) = 1, and (ii) lP' is additive, in that lP'(A U B) = lP'(A) + lP'(B) whenever A n B = 0. Show that lP'(0) = O. 9. Suppose (Q, :F, lP') is a probability space and B E Fsatisfies lP'(B) > O. Let IQl : F -+ [0, 1] be defined by IQl(A) = lP'(A I B). Show that (Q, F, 1Ql) is a probability space. If C E F and IQl(C) > 0, show that IQl(A I C) = lP'(A IBn C); discuss.

10. Let Bl, B2, .. , be a partition of the sample space Q, each Bi having positive probability, and show that 00

lP'(A)

= LlP'(A I Bj)lP'(Bj). j=l

11. Prove Boole's inequalities:

lP'(n Ai) i=l

~ 1- tlP'(Aj). i=l

12. Prove that lP'(nAi)=LlP'(Ai)-LlP'(AiUAj)+ L lP'(AiUAjUAk) 1 i i<j i<j
13. Let AI, A2, ... , An be events, and let Nk be the event that exactly k of the Ai occur. Prove the result sometimes referred to as Waring's theorem:

lP'(Nk) =

~ (_I)i (k ; i) Sk+i, where Sj =. 1=0

l[

, L ,lP'(Ai1
n Ai2 n ... n Ai).

Use this result to find an expression for the probability that a purchase of six packets of Com Flakes yields exactly three distinct busts (see Exercise (1.3.4)). 14. Prove Bayes's formula: if AI, A2, ... , An is a partition of Q, each Ai having positive probability, then

5

[1.8.15]-[1.8.22]

Exercises

Events and their probabilities

15. A random number N of dice is thrown. Let Ai be the event that N = 2- i , i ::: 1. The sum of the scores is S. Find the probability that: (a) N = 2 given S = 4; (b) S = 4 given N is even; (c) N = 2, given that S = 4 and the first die showed 1; (d) the largest number shown by any die is r, where S is unknown.

=

i, and assume that

IP'(Ai)

16. Let AI, A2, ... be a sequence of events. Define 00

Bn =

n 00

U Am,

Cn

=

Am.

m=n

m=n

Clearly Cn S; An S; Bn. The sequences {Bn} and {Cn} are decreasing and increasing respectively with limits n

n

n m?:,n

n

m~n

The events Band C are denoted lim sUPn---;.oo An and lim infn---;.oo An respectively. Show that (a) B = {w E Q : wEAn for infinitely many values of n}, (b) C = {w E Q : wEAn for all but finitely many values of n}. We say that the sequence {An} converges to a limit A = lim An if Band C are the same set A. Suppose that An ---+ A and show that (c) A is an event, in that A E :F, (d) IP'(An) ---+ IP'(A).

17. In Problem (1.8.16) above, show that Band C are independent whenever Bn and Cn are independent for all n. Deduce that if this holds and furthermore An ---+ A, then IP'(A) equals either zero or one. 18. Show that the assumption that IP' is countably additive is equivalent to the assumption that IP' is continuous. That is to say, show that if a function IP' : :F ---+ [0,1] satisfies 1P'(0) = 0, IP'(Q) = 1, and IP'(A U B) = IP'(A) + IP'(B) whenever A, B E :Fand A n B = 0, then IP' is countably additive (in the sense of satisfying Definition (1.3.Ib)) if and only iflP' is continuous (in the sense of Lemma (1.3.5)).

19. Anne, Betty, Chloe, and Daisy were all friends at school. Subsequently each of the (i) = 6 subpairs meet up; at each of the six meetings the pair involved quarrel with some fixed probability p, or become firm friends with probability 1 - p. Quarrels take place independently of each other. In future, if any of the four hears a rumour, then she tells it to her firm friends only. If Anne hears a rumour, what is the probability that: (a) Daisy hears it? (b) Daisy hears it if Anne and Betty have quarrelled? (c) Daisy hears it if Betty and Chloe have quarrelled? (d) Daisy hears it if she has quarrelled with Anne? 20. A biased coin is tossed repeatedly. Each time there is a probability p of a head turning up. Let Pn be the probability that an even number of heads has occurred after n tosses (zero is an even number). Show that PO = 1 and that Pn = p(l- Pn-l) + (1- p) Pn-l if n ::: 1. Solve this difference equation. 21. A biased coin is tossed repeatedly. Find the probability that there is a run of r heads in a row before there is a run of s tails, where rand s are positive integers.

22. A bowl contains twenty cherries, exactly fifteen of which have had their stones removed. A greedy pig eats five whole cherries, picked at random, without remarking on the presence or absence of stones. Subsequently, a cherry is picked randomly from the remaining fifteen.

6

Exercises

Problems

[1.8.23]-[1.8.31]

(a) What is the probability that this cherry contains a stone? (b) Given that this cherry contains a stone, what is the probability that the pig consumed at least one stone? 23. The 'menages' problem poses the following question. Some consider it to be desirable that men and women alternate when seated at a circular table. If n couples are seated randomly according to this rule, show that the probability that nobody sits next to his or her partner is -1 ~ L..(-1) k -2n- (2n - k) (n - k)! n!k=O 2n-k k

You may find it useful to show first that the number of ways of selecting k non-overlapping pairs of adjacent seats is (2nkk)2n(2n - k)-l. 24. An urn contains b blue balls and r red balls. They are removed at random and not replaced. Show that the probability that the first red ball drawn is the (k Find the probability that the last ball drawn is red.

+ 1)th ball drawn equals (r+~=7-1) /

et

b

).

25. An urn contains a azure balls and c carmine balls, where ac =1= O. Balls are removed at random and discarded until the first time that a ball (B, say) is removed having a different colour from its predecessor. The ball B is now replaced and the procedure restarted. This process continues until the last ball is drawn from the urn. Show that this last ball is equally likely to be azure or carmine. 26. Protocols. A pack of four cards contains one spade, one club, and the two red aces. You deal two cards faces downwards at random in front of a truthful friend. She inspects them and tells you that one of them is the ace of hearts. What is the chance that the other card is the ace of diamonds? Perhaps 1? Suppose that your friend's protocol was: (a) with no red ace, say "no red ace", (b) with the ace of hearts, say "ace of hearts", (c) with the ace of diamonds but not the ace of hearts, say "ace of diamonds". Show that the probability in question is 1. Devise a possible protocol for your friend such that the probability in question is zero. 27. Eddington's controversy. Four witnesses, A, B, C, and D, at a trial each speak the truth with probability ~ independently of each other. In their testimonies, A claimed that B denied that C declared that D lied. What is the (conditional) probability that D told the truth? [This problem seems to have appeared first as a parody in a university magazine of the 'typical' Cambridge Philosophy Tripos question.] 28. The probabilistic method. 10 per cent of the surface of a sphere is coloured blue, the rest is red. Show that, irrespective of the manner in which the colours are distributed, it is possible to inscribe a cube in S with all its vertices red. 29. Repulsion. The event A is said to be repelled by the event B if lP'(A I B) < lP'(A), and to be attracted by B iflP'(A I B) > lP'(A). Show that if B attracts A, then A attracts B, and Be repels A. If A attracts B, and B attracts C, does A attract C? 30. Birthdays. If m students born on independent days in 1991 are attending a lecture, show that the probability that at least two of them share a birthday is p = 1 - (365)!1 {(365 - m)! 365 m ). Show that p > when m = 23.

1

31. Lottery. You choose r of the first n positive integers, and a lottery chooses a random subset L of the same size. What is the probability that: (a) L includes no consecutive integers?

7

[1.8.32]-[1.8.36] (b) (c) (d) (e)

Exercises

Events and their probabilities

L includes exactly one pair of consecutive integers? the numbers in L are drawn in increasing order? your choice of numbers is the same as L? there are exactly k of your numbers matching members of L?

32. Bridge. During a game of bridge, you are dealt at random a hand of thirteen cards. With an obvious notation, show that lP'(4S, 3H, 3D, 3C) ~ 0.026 and lP'(4S, 4H, 3D, 2C) ~ 0.018. However if suits are not specified, so numbers denote the shape of your hand, show that lP'(4, 3, 3, 3) ~ 0.11 and lP'(4, 4, 3, 2) ~ 0.22. 33. Poker. During a game of poker, you are dealt a five-card hand at random. With the convention that aces may count high or low, show that: lP'(1 pair)

~

lP'(straight)

~

0.0039,

lP'(4 of a kind)

~

0.00024,

0.423,

~

0.0475,

lP'(3 of a kind)

~

0.021,

lP'(flush)

~

0.0020,

lP'(full house)

~

0.0014,

lP'(straight flush)

~

0.000015.

lP'(2 pairs)

34. Poker dice. There are five dice each displaying 9, 10, J, Q, K, A. Show that, when rolled: lP'(1 pair)

~

0.46,

lP'(no 2 alike)

~

0.093,

lP'(5 of a kind)

~

0.0008.

lP'(2 pairs)

~

0.23,

lP'(3 of a kin d)

~

0.15,

lP'(full house)

~

0.039,

lP'(4 of a kind)

~

0.019,

35. You are lost in the National Park of Bandrikat. Tourists comprise two-thirds of the visitors to the park, and give a correct answer to requests for directions with probability ~. (Answers to repeated questions are independent, even if the question and the person are the same.) If you ask a Bandrikan for directions, the answer is always false. (a) You ask a passer-by whether the exit from the Park is East or West. The answer is East. What is the probability this is correct? (b) You ask the same person again, and receive the same reply. Show the probability that it is correct . 1 IS 2. (c) You ask the same person again, and receive the same reply. What is the probability that it is correct? (d) You ask for the fourth time, and receive the answer East. Show that the probability it is correct .

IS

27

70.

(e) Show that, had the fourth answer been West instead, the probability that East is nevertheless . 9 correct IS ill.

36. Mr Bayes goes to Bandrika. Tom is in the same position as you were in the previous problem, but he has reason to believe that, with probability E, East is the correct answer. Show that: (a) whatever answer first received, Tom continues to believe that East is correct with probability E, (b) if the first two replies are the same (that is, either WW or EE), Tom continues to believe that East is correct with probability E, (c) after three like answers, Tom will calculate as follows, in the obvious notation: 9E lP'(East correct IEEE) = - - - , 11 - 2E Evaluate these when E =

lP'(East correct I WWW)

to.

t A fictional country made famous in the Hitchcock film 'The Lady Vanishes'. 8

lIE

= --. 9+2E

Exercises

Problems

[1.8.37]-[1.8.39]

37. Bonferroni's inequality. Show that

JP' (

U

Ar) :::

r=!

t

JP'(Ar) -

r=!

L JP'(Ar n Ak)· r
38. Kounias's inequality. Show that

39. The n passengers for a Bell-Air flight in an airplane with n seats have been told their seat numbers. They get on the plane one by one. The first person sits in the wrong seat. Subsequent passengers sit in their assigned seats whenever they find them available, or otherwise in a randomly chosen empty seat. What is the probability that the last passenger finds his seat free?

9

2

Random variables and their distributions

2.1 Exercises. Random variables 1.

Let X be a random variable on a given probability space, and let a (i) aX is a random variable, (ii) X - X = 0, the random variable taking the value 0 always, and X

E

JR. Show that

+ X = 2X.

2. A random variable X has distribution function F. What is the distribution function of Y where a and b are real constants?

= aX +b,

3.

A fair coin is tossed n times. Show that, under reasonable assumptions, the probability of exactly k heads is (k)C~)n. What is the corresponding quantity when heads appears with probability p on

each toss? 4. Show that if F and G are distribution functions and 0 :::: 'A :::: 1 then 'AF +(l-'A)G is a distribution function. Is the product FG a distribution function?

5. Let F be a distribution function and r a positive integer. Show that the following are distribution functions: (a) F(x)', (b) 1- {l- F(x)}', (c) F(x) + {I - F(x)}log{1 - F(x)}, (d) {F(x) - l}e + exp{l - F(x)}.

2.2 Exercises. The law of averages 1. You wish to ask each of a large number of people a question to which the answer "yes" is embarrassing. The following procedure is proposed in order to determine the embarrassed fraction of the population. As the question is asked, a coin is tossed out of sight of the questioner. If the answer would have been "no" and the coin shows heads, then the answer "yes" is given. Otherwise people respond truthfully. What do you think of this procedure? 2. A coin is tossed repeatedly and heads turns up on each toss with probability p. Let Hn and Tn be the numbers of heads and tails in n tosses. Show that, for E > 0, IP' (2P - 1 -

E: : ~(Hn -

Tn) :::: 2p - 1 +

E)

-+ 1

as n -+

00.

3. Let {X r : r :::: I} be observations which are independent and identically distributed with unknown distribution function F. Describe and justify a method for estimating F(x).

10

Exercises

Worked examples

[2.3.1]-[2.4.2]

2.3 Exercises. Discrete and continuous variables Let X be a random variable with distribution function F, and let a = (am: -00 < m < (0) be a strictly increasing sequence of real numbers satisfying a- m ---+ -00 and am ---+ 00 as m ---+ 00. Define G(x) = lP'(X :s am) when am-I :s x < am, so that G is the distribution function of a discrete random variable. How does the function G behave as the sequence a is chosen in such a way that sUPm lam - am-II becomes smaller and smaller?

1.

2. Y

3.

Let X be a random variable and let g : JR. ---+ JR. be continuous and strictly increasing. Show that

= g(X) is a random variable. Let X be a random variable with distribution function if x :s 0, ifO<x:s1, if x> 1.

Let F be a distribution function which is continuous and strictly increasing. Show that Y = F- I (X) is a random variable having distribution function F. Is it necessary that F be continuous and/or strictly increasing? Show that, if f and g are density functions, and 0 :s A :s 1, then Af the product f g a density function?

4.

5.

+ (1

- A)g is a density. Is

Which of the following are density functions? Find c and the corresponding distribution function

F for those that are. d (a) f(x) = {Cox-

(b) f(x)

=

x > 1, otherwise. x x ce (1 + e )-2, x E JR..

2.4 Exercises. Worked examples 1. Let X be a random variable with a continuous distribution function F. Find expressions for the distribution functions of the following random variables: (a) X2, (b) .JX, (c) sin X, (d) G-I(X), (e) F(X), (f) G-I(F(X», where G is a continuous and strictly increasing function. 2. Truncation. Let X be a random variable with distribution function F, and let a < b. Sketch the distribution functions of the 'truncated' random variables Y and Z given by

Y

={

a

if X < a,

X

if a :s X :s b,

b

if X> b,

Z

Indicate how these distribution functions behave as a ---+

11

={ -00,

X 0

iflXI:s b, if IXI > b.

b ---+

00.

[2.5.1]-[2.7.4]

Exercises

Random variables and their distributions

2.5 Exercises. Random vectors 1. A fair coin is tossed twice. Let X be the number of heads, and let W be the indicator function of the event {X = 2}. Find IP'(X = x, W = w) for all appropriate values of x and w. 2. Let X be a Bernoulli random variable, so that IP'(X = 0) = 1 - p, IP'(X and Z = XY. Find IP'(X = x, Y = y) andlP'(X = x, Z = z) for x, y,z E 3.

= 1) = p. Let Y = 1 to, l}.

X

The random variables X and Y have joint distribution function

FX,Y(x,

y) ~ {;1 - ,-X) G+ ~ ,,,-I y)

if x < 0, if x::::

o.

Show that X and Y are (jointly) continuously distributed.

4.

Let X and Y have joint distribution function F. Show that lP'(a < X ::: b, c < Y ::: d)

= F(b, d) -

F(a, d) - F(b, c)

+ F(a, c)

whenever a < band c < d.

5.

Let X, Y be discrete random variables taking values in the integers, with joint mass function Show that, for integers x, y, I(x, y)

= IP'(X:::: x,

Y ::: y) -IP'(X:::: x

+ 1,

I.

Y ::: y)

-1P'(X::::x, Y:::y-1)+IP'(X::::x+l, Y:::y-1).

Hence find the joint mass function of the smallest and largest numbers shown in r rolls of a fair die. 6. Is the function F(x, y) = 1 - e- xy , 0 ::: x, y < 00, the joint distribution function of some pair of random variables?

2.7 Problems 1. Each toss of a coin results in a head with probability p. The coin is tossed until the first head appears. Let X be the total number of tosses. What is IP'(X > m)? Find the distribution function of the random variable X. (a) Show that any discrete random variable may be written as a linear combination of indicator variables. (b) Show that any random variable may be expressed as the limit of an increasing sequence of discrete random variables. (c) Show that the limit of any increasing convergent sequence of random variables is a random variable.

2.

3.

(a) Show that, if X and Y are random variables on a probability space (Q, :F, IP'), then so are X + Y, XY, and min{X, Y}. (b) Show that the set of all random variables on a given probability space (Q, :F, IP') constitutes a vector space over the reals. If Q is finite, write down a basis for this space.

4.

Let X have distribution function if x < 0, if 0 ::: x ::: 2, if x> 2,

12

Exercises

Problems

and let Y = X2. Find (a) lP'(~ :s X :s ~), (c) lP'(Y :s X), (e) lP'(X + Y :s ~), 5.

(b) lP'(1 :s X < 2), (d) lP'(X :s 2Y), (f) the distribution function of Z

[2.7.5]-[2.7.13]

= .JX.

Let X have distribution function if x < -1,

0 F(x)

=

I-p 1

{ ~ - P + 2xP

Sketch this function, and find: (a) lP'(X

=

if - 1 :s x < 0, if 0 :s x :s 2, if x> 2.

(b) lP'(X

-1),

= 0),

(c) lP'(X :::: 1).

6. Buses arrive at ten minute intervals starting at noon. A man arrives at the bus stop a random number X minutes after noon, where X has distribution function

lP'(X

:s x) =

{

0 x/60

if x < 0,

1

if x> 60.

if 0

:s x :s 60,

What is the probability that he waits less than five minutes for a bus?

lo

inde7. Airlines find that each passenger who reserves a seat fails to tum up with probability pendently of the other passengers. So Teeny Weeny Airlines always sell 10 tickets for their 9 seat aeroplane while Blockbuster Airways always sell 20 tickets for their 18 seat aeroplane. Which is more often over-booked? 8. A fairground performer claims the power of telekinesis. The crowd throws coins and he wills them to fall heads up. He succeeds five times out of six. What chance would he have of doing at least as well if he had no supernatural powers? 9.

Express the distribution functions of

X+

= max{O, X},

X-

=-

min{O, X},

IXI

= X+ + X-,

-X,

in terms of the distribution function F of the random variable X. 10. Show that Fx(x) is continuous at x

= xo if and only if lP'(X = xo) = O.

11. The real number m is called a median of the distribution function F whenever limytm F(y) :s ~ :s F(m). Show that every distribution function F has at least one median, and that the set of medians of F is a closed interval of R 12. Show that it is not possible to weight two dice in such a way that the sum of the two numbers shown by these loaded dice is equally likely to take any value between 2 and 12 (inclusive). 13. A function d : S x S -+ lR is called a metric on S if: (i) d(s, t) = d(t, s) :::: 0 for all s, t E S, (ii) d(s, t) = 0 if and only if s = t, and (iii) d(s, t) :s d(s, u) + d(u, t) for all s, t, U E S. (a) Levy metric. Let F and G be distribution functions and define the Levy metric ddF, G)

= inf {E >

0 : G(x - E) - E :s F (x)

:s G(x + E) + E for all x}.

Show that dL is indeed a metric on the space of distribution functions.

13

[2.7.14]-[2.7.18]

Exercises

Random variables and their distributions

(b) Total variation distance. Let X and Y be integer-valued random variables, and let dTV(X, Y) = ~]JP'(X = k) -JP'(Y = k)l. k

Show that dTV satisfies (i) and (iii) with S the space of integer-valued random variables, and that dTV(X, Y) = 0 if and only if JP'(X = Y) = 1. Thus dTV is a metric on the space of equivalence classes of S with equivalence relation given by X ~ Y if JP'(X = Y) = 1. We call dTV the total variation distance. Show that dTV(X, Y)

= 2 sup IJP'(X E

A) -JP'(Y E A)I.

A<;Z

14. Ascertain in the following cases whether or not F is the joint distribution function of some pair (X, Y) of random variables. If your conclusion is affirmative, find the distribution functions of X and Y separately. (a)

F(x, y)

={

I - e- x - y 0

if x, y ::: 0,

I - e- x - xe- Y (b)

F(x, y)

={

~

-

e- Y

-

ye- Y

otherwise. ifO::sx::sy, ifO::sy::sx,

otherwise.

15. It is required to place in order n books BI, B2, ... , Bn on a library shelf in such a way that readers searching from left to right waste as little time as possible on average. Assuming that each reader requires book Bj with probability Pi, find the ordering of the books which minimizes JP'(T ::: k) for all k, where T is the (random) number of titles examined by a reader before discovery of the required book. 16. Transitive coins. Three coins each show heads with probability ~ and tails otherwise. The first counts 10 points for a head and 2 for a tail, the second counts 4 points for both head and tail, and the third counts 3 points for a head and 20 for a tail. You and your opponent each choose a coin; you cannot choose the same coin. Each of you tosses your coin and the person with the larger score wins £1010. Would you prefer to be the first to pick a coin or the second?

17. Before the development of radar and inertial navigation, flying to isolated islands (for example, from Los Angeles to Hawaii) was somewhat 'hit or miss'. In heavy cloud or at night it was necessary to fly by dead reckoning, and then to search the surface. With the aid of a radio, the pilot had a good idea of the correct great circle along which to search, but could not be sure which of the two directions along this great circle was correct (since a strong tailwind could have carried the plane over its target). When you are the pilot, you calculate that you can make n searches before your plane will run out of fuel. On each search you will discover the island with probability P (if it is indeed in the direction of the search) independently of the results of other searches; you estimate initially that there is probability a that the island is ahead of you. What policy should you adopt in deciding the directions of your various searches in order to maximize the probability of locating the island? 18. Eight pawns are placed randomly on a chessboard, no more than one to a square. What is the probability that: (a) they are in a straight line (do not forget the diagonals)? (b) no two are in the same row or column?

14

Exercises

Problems

[2.7.19]-[2.7.20]

19. Which of the following are distribution functions? For those that are, give the corresponding density function f. (a) F(x)

={

(b) F(x) = {

1 - e-

o e-l/x

o

~ 0, otherwise. x > 0

x2

x

'

otherwise. (c) F(x) = eX j(e X + e- X), x E R 2 (d) F(x) = e- x + eX j(e X + e- X), x

E

IR.

20. (a) If U and V are jointly continuous, show that IP'(U = V) = O. (b) Let X be unifonnly distributed on (0,1), and let Y = X. Then X and Y are continuous, and IP'(X = Y) = 1. Is there a contradiction here?

15

3

Discrete random variables

3.1 Exercises. Probability mass functions 1. For what values of the constant C do the following define mass functions on the positive integers 1,2, ... ? (a) Geometric: f(x) = C2- x . (b) Logarithmic: f(x) = CT x Ix. (c) Inverse square: f(x)

= Cx- 2 .

(d) 'Modified' Poisson: f(x) 2. (i) (ii) (iii)

= C2 x Ix!.

For a random variable X having (in tum) each of the four mass functions of Exercise (1), find: JP(X > 1), the most probable value of X, the probability that X is even.

3. We toss n coins, and each one shows heads with probability p, independently of each of the others. Each coin which shows heads is tossed again. What is the mass function of the number of heads resulting from the second round of tosses? 4. Let Sk be the set of positive integers whose base-lO expansion contains exactly k elements (so that, for example, 1024 E S4). A fair coin is tossed until the first head appears, and we write T for the number of tosses required. We pick a random element, N say, from Sr, each such element having equal probability. What is the mass function of N? 5. Log-convexity. (a) Show that, if X is a binomial or Poisson random variable, then the mass function f(k) = JP(X = k) has the property that f(k - l)f(k + 1) ::s f(k)2. (b) Show that, if f(k)

= 90/(nk)4,

k::: 1, then f(k - l)f(k

(c) Find a mass function f such that f(k)2 = f(k - l)f(k

+ 1) :::

+ 1),

f(k)2.

k::: 1.

3.2 Exercises. Independence 1. Let X and Y be independent random variables, each taking the values -1 or 1 with probability ~, and let Z = XY. Show that X, Y, and Z are pairwise independent. Are they independent? 2. Let X and Y be independent random variables taking values in the positive integers and having the same mass function f(x) = 2- x for x = 1,2, .... Find: (a) JP(min{X, Y} ::s x), (b) JP(Y > X), (c) JP(X = Y), (d) JP(X ::: kY), for a given positive integer k, (e) JP(X divides Y), (f) JP(X = rY), for a given positive rational r. 16

Exercises

Expectation

[3.2.3]-[3.3.5]

3.

Let X I, X 2, X 3 be independent random variables taking values in the positive integers and having 1 mass functions given by IP'(Xi = x) = (l - Pi)Pt- for x = 1,2, ... , and i = 1,2,3. (a) Show that (l - PI)(1 - P2)P2P~ IP'(X 1< X 2< X) 3= (l - P2P3)(1 - PI P2P3) (b) FindlP'(Xl ::: X2::: X3)·

4.

Three players, A, B, and C, take turns to roll a die; they do this in the order ABCABCA ....

(a) Show that the probability that, of the three players, A is the first to throw a 6, B the second, and C the third, is 216/1001. (b) Show that the probability that the first 6 to appear is thrown by A, the second 6 to appear is thrown by B, and the third 6 to appear is thrown by C, is 46656/753571.

5.

Let X r , 1 ::: r ::: n, be independent random variables which are symmetric about 0; that is, Xr and -Xr have the same distributions. Show that, for all x, IP'(Sn ::: x) = IP'(Sn ::: -x) where Sn = Z=~=l X r . Is the conclusion necessarily true without the assumption of independence?

3.3 Exercises. Expectation 1.

Is it generally true that lE(l/ X)

= l/lE(X)? Is it ever true that lE(l/ X) = l/lE(X)?

2. Coupons. Every package of some intrinsically dull commodity includes a small and exciting plastic object. There are c different types of object, and each package is equally likely to contain any given type. You buy one package each day. (a) Find the mean number of days which elapse between the acquisitions of the jth new type of object and the (j + I)th new type. (b) Find the mean number of days which elapse before you have a full set of objects.

3.

Each member of a group of n players rolls a die. (a) For any pair of players who throw the same number, the group scores 1 point. Find the mean and variance of the total score of the group. (b) Find the mean and variance of the total score if any pair of players who throw the same number scores that number.

4.

8t Petersburg paradoxt. A fair coin is tossed repeatedly. Let T be the number of tosses until the first head. You are offered the following prospect, which you may accept on payment of a fee. If T = k, say, then you will receive £2k. What would be a 'fair' fee to ask of you?

5.

Let X have mass function f(x)

= { ~X(X + l)}-l

if x

= 1,2, ... ,

otherwise, and let ex

E

JR. For what values of ex is it the caset that lE(XO!) < oo?

tThis problem was mentioned by Nicholas Bernoulli in 1713, and Daniel Bernoulli wrote about the question for the Academy of St Petersburg. :j:If ex is not integral, than JE(X"') is called the fractional moment of order ex of X. A point concerning notation: for real ex and complex x = re W , x'" should be interpreted as r'" eilJ ", , so that Ix'" I = r"'. In particular, JE(lX'" I) = JE(lXI"')· .

17

[3.3.6]-[3.4.8] 6.

Exercises

Show that var(a

+ X) =

Discrete random variables

var(X) for any random variable X and constant a.

7. Arbitrage. Suppose you find a warm-hearted bookmaker offering payoff odds of n(k) against the kth horse in an n-horse race where L:k=1 (n(k) + 1}-1 < 1. Show that you can distribute your bets in such a way as to ensure you win. 8. You roll a conventional fair die repeatedly. If it shows 1, you must stop, but you may choose to stop at any prior time. Your score is the number shown by the die on the final roll. What stopping strategy yields the greatest expected score? What strategy would you use if your score were the square of the final roll? 9. Continuing with Exercise (8), suppose now that you lose c points from your score each time you roll the die. What strategy maximizes the expected final score if c = ~? What is the best strategy if c = I?

3.4 Exercises. Indicators and matching 1. A biased coin is tossed n times, and heads shows with probability p on each toss. A run is a sequence of throws which result in the same outcome, so that, for example, the sequence HHTHTTH contains five runs. Show that the expected number of runs is 1 + 2(n - I)p(l- p). Find the variance of the number of runs. 2. An urn contains n balls numbered 1,2, ... , n. Weremovek balls at random (without replacement) and add up their numbers. Find the mean and variance of the total. 3. Of the 2n people in a given collection of n couples, exactly m die. Assuming that the m have been picked at random, find the mean number of surviving couples. This problem was formulated by Daniel Bernoulli in 1768. 4. Urn R contains n red balls and urn B contains n blue balls. At each stage, a ball is selected at random from each urn, and they are swapped. Show that the mean number of red balls in urn Rafter stage k is ~ n {1 + (l - 2/ n)k). This 'diffusion model' was described by Daniel Bernoulli in 1769. 5. Consider a square with diagonals, with distinct source and sink. Each edge represents a component which is working correctly with probability p, independently of all other components. Write down an expression for the Boolean function which equals 1 if and only if there is a working path from source to sink, in terms of the indicator functions Xi of the events {edge i is working} as i runs over the set of edges. Hence calculate the reliability of the network. 6. A system is called a 'k out of n' system if it contains n components and it works whenever k or more of these components are working. Suppose that each component is working with probability p, independently of the other components, and let Xc be the indicator function of the event that component c is working. Find, in terms of the Xc, the indicatorfunction of the event that the system works, and deduce the reliability of the system. 7. The probabilistic method. Let G = (V, E) be a finite graph. For any set W of vertices and any edge e E E, define the indicator function IW(e) = {

Set Nw

=

~

if e connects Wand We, otherwise.

L:eEE Iw(e). Show that there exists W ~ V such that Nw 2: ~IEI.

8. A total of n bar magnets are placed end to end in a line with random independent orientations. Adjacent like poles repel, ends with opposite polarities join to form blocks. Let X be the number of blocks of joined magnets. Find JE(X) and var(X).

18

Exercises

Dependence

[3.4.9]-[3.6.5]

9. Matching. (a) Use the inclusion-exclusion formula (3.4.2) to derive the result of Example (3.4.3), namely: in a random permutation of the first n integers, the probability that exactly r retain their original positions is 1( 1 1 ( _l)n-r ) ;-j" 2! -3T+ ... + (n-r)! . (b) Let d n be the number of derangements of the first n integers (that is, rearrangements with no integers in their original positions). Show that d n+ 1 = ndn + ndn-l for n 2: 2. Deduce the result of part (a).

3.5 Exercises. Examples of discrete variables 1. De Moivre trials. Each trial may result in any of t given outcomes, the ith outcome having probability Pi. Let Ni be the number of occurrences of the ith outcome in n independent trials. Show that n! n] n2 nt JI"(Nj = nj for 1 .:s i .:s t) = Pl P2 ... Pt nl!n2! ···nt! for any collection nl, n2, ... , nt of non-negative integers with sum n. The vector N is said to have the multinomial distribution.

2. In your pocket is a random number N of coins, where N has the Poisson distribution with parameter A. You toss each coin once, with heads showing with probability P each time. Show that the total number of heads has the Poisson distribution with parameter Ap. 3. Let X be Poisson distributed where JI"(X = n) = Pn(A) = Ane-A/n! for n 2: O. Show that JI"(X .:s n) = 1Pn(x)dx.

It

4. Capture-recapture. A population of b animals has had a number a of its members captured, marked, and released. Let X be the number of animals it is necessary to recapture (without re-release) in order to obtain m marked animals. Show that JI"(X

= n) = ~ b

(a - 1) (b -a) / (b - 1), m-l

n-m

n-l

and find lEX. This distribution has been called negative hypergeometric.

3.6 Exercises. Dependence 1. Show that the collection of random variables on a given probability space and having finite variance forms a vector space over the reals.

2.

Find the marginal mass functions of the multinomial distribution of Exercise (3.5.1).

3.

Let X and Y be discrete random variables with joint mass function

I(x, y)

=

c (x

+Y -

l)(x

+ y)(x + y + 1)

,

x,y

= 1,2,3, ....

Find the marginal mass functions of X and Y, calculate C, and also the covariance of X and Y. 4. Let X and Y be discrete random variables with mean 0, variance 1, and covariance p. Show that lE(max{X2, y2}) .:s 1 +

0-7.

5.

Mutual information. Let X and Y be discrete random variables with joint mass function

19

I.

[3.6.6]-[3.7.2]

Exercises

Discrete random variables

(a) Show that E(log fx(X)) ::: E(log fy(X)). (b) Show that the mutual information 1- E (10 { f(X, Y) }) g fx(X)fy(Y)

satisfies I ::: 0, with equality if and only if X and Y are independent. 6. Voter paradox. Let X, Y, Z be discrete random variables with the property that their values are distinct with probability 1. Let a = JP>(X > Y), b = JP>(Y > Z), c = JP>(Z > X). (a) Show that min{a, b, c} ::::

j, and give an example where this bound is attained.

i.

(b) Show that, if X, Y, Z are independent and identically distributed, then a = b = c = (c) Find min{a, b, c} and sUPp min{a, b, c} when JP>(X = 0) = 1, and Y, Z are independent with JP>(Z = 1) = JP>(Y = -1) = p, JP>(Z = -2) = JP>(Y = 2) = 1 - p. Here, sUPp denotes the supremum as p varies over [0, 1]. [Part (a) is related to the observation that, in an election, it is possible for more than half of the voters to prefer candidate A to candidate B, more than half B to C, and more than half C to A.] 7. Benford's distribution, or the law of anomalous numbers. If one picks a numerical entry at random from an almanac, or the annual accounts of a corporation, the first two significant digits, X, Y, are found to have approximately the joint mass function f(x, y)

= 10glO (1 +

lOx

1

+y

),

1:::: x :::: 9,

0:::: y

:::: 9.

Find the mass function of X and an approximation to its mean. [A heuristic explanation for this phenomenon may be found in the second of Feller's volumes (1971).] 8.

Let X and Y have joint mass function . k _ c(j f() )

,

+ k)a Hk

-

where a is a constant. Find c, JP>(X = j), JP>(X

)"k' ..

+Y

'

)',k _> 0,

= r), and E(X).

3.7 Exercises. Conditional distributions and conditional expectation 1. Show the following: (a) E(aY + bZ I X) = aE(Y I X) + bE(Z I X) for a, b E R, (b) E(Y I X) ::: if Y ::: 0, (c) E(1 I X) = 1, (d) if X and Y are independent then E(Y I X) = E(y), (e) (,pull-through property') E(Y g(X) I X) = g(X)E(Y I X) for any suitable function g, (f) ('tower property') E{E(Y I X, Z) I X} = E(Y I X) = E{E(Y I X) I X, Z}.

°

2. Uniqueness of conditional expectation. Suppose that X and Y are discrete random variables, and that 4> (X) and 1fr(X) are two functions of X satisfying E(4)(X)g(X))

= E(1fr(X)g(X)) = E(Yg(X))

for any function g for which all the expectations exist. Show that 4>(X) and 1fr(X) are almost surely equal, in that JP>(4)(X) = 1fr(X)) = 1.

20

Exercises

Sums of random variables

[3.7.3]-[3.8.3]

3. Suppose that the conditional expectation of Y given X is defined as the (almost surely) unique function 1fr(X) such that E(1fr(X)g(X)) = E(Yg(X)) for all functions g for which the expectations exist. Show (a)-(f) of Exercise (1) above (with the occasional addition of the expression 'with probability 1'). 4. How should we define var(Y E(var(Y I X)) + var(E(Y I X)).

I X), the conditional variance of Y given X? Show that var(Y)

=

5. The lifetime of a machine (in days) is a random variable T with mass function f. Given that the machine is working after t days, what is the mean subsequent lifetime of the machine when: (a) f(x) = (N + 1)-1 for x E {O, 1, ... , N}, (b) f(x)=T x forx=I,2, ....

(The first part of Problem (3.11.13) may be useful.)

6. Let X I, X2, ... be identically distributed random variables with mean fl" and let N be a random variable taking values in the non-negative integers and independent of the Xi. Let S = X I + X 2 + ... + XN. Show that E(S I N) = fl,N, and deduce that E(S) = fl,E(N). 7. A factory has produced n robots, each of which is faulty with probability ¢. To each robot a test is applied which detects the fault (if present) with probability 8. Let X be the number of faulty robots, and Y the number detected as faulty. Assuming the usual independence, show that E(X

I Y)

= {n¢(1 - 8)

+ (1- ¢)Y}/(1- ¢8).

8. Families. Each child is equally likely to be male or female, independently of all other children. (a) Show that, in a family of predetermined size, the expected number of boys equals the expected number of girls. Was the assumption of independence necessary? (b) A randomly selected child is male; does the expected number of his brothers equal the expected number of his sisters? What happens if you do not require independence? 9.

Let X and Y be independent with mean fl,. Explain the error in the following equation: 'E(X I X

+ Y = z) = E(X I X = z -

= E(z -

Y)

Y)

=z

- fl,'.

10. A coin shows heads with probability p. Let Xn be the number of flips required to obtain a run of n consecutive heads. Show that E(Xn) = EZ=I p-k.

3.8 Exercises. Sums of random variables 1. Let X and Y be independent variables, X being equally likely to take any value in {O, 1, ... , m}, and Y similarly in {O, 1, ... , n}. Find the mass function of Z = X + Y. The random variable Z is said to have the trapezoidal distribution. 2.

Let X and Y have the joint mass function f(x,y)=

c (x

+y -

Find the mass functions of U = X

1)(x

+ y)(x + y + 1)

+ Y and V =

,

x,y=I,2,3, ....

X - Y.

3. Let X and Y be independent geometric random variables with respective parameters Show that JP'(X + Y = z) = ~{(1 - (3)Z-1 - (1 - a)Z-I}.

a-f3

21

a and f3.

[3.8.4]-[3.9.6]

Discrete random variables

Exercises

4. Let {Xr : 1 ::0 r ::0 n} be independent geometric random variables with parameter p. Show that Z :::: L:~=l Xr has a negative binomial distribution. [Hint: No calculations are necessary.]

5. Pepys's problemt. Sam rolls 6n dice once; he needs at least n sixes. Isaac rolls 6(n he needs at least n + 1 sixes. Who is more likely to obtain the number of sixes he needs?

+ 1) dice;

6. Let N be Poisson distributed with parameter A. Show that, for any function g such that the expectations exist, E(N g(N» :::: AEg(N + 1). More generally, if S :::: L:~=l X r , where {X r : r ~ O} are independent identically distributed non-negative integer-valued random variables, show that

E(Sg(S») :::: AE(g(S + Xa)Xa).

3.9 Exercises. Simple random walk 1. Let T be the time which elapses before a simple random walk is absorbed at either of the absorbing barriers at 0 and N, having started at k where 0 ::0 k ::0 N. Show that lP'(T < 00) :::: 1 and E(Tk) < 00 for all k ~ 1. 2. For simple random walk S with absorbing barriers at 0 and N, let W be the event that the particle is absorbed at 0 rather than at N, and let Pk :::: lP'(W I Sa :::: k). Show that, if the particle starts at k where 0 < k < N, the conditional probability that the first step is rightwards, given W, equals PPk+lI Pk· Deduce that the mean duration Jk of the walk, conditional on W, satisfies the equation PPk+l Jk+l - PkJk

+ (Pk -

PPk+l)Jk-l :::: -Pb

for 0 < k < N.

Show that we may take as boundary condition Ja :::: O. Find Jk in the symmetric case, when P ::::

!.

3. With the notation of Exercise (2), suppose further that at any step the particle may remain where it is with probability r where P + q + r :::: 1. Show that Jk satisfies PPk+l Jk+l - (1 - r)PkJk

+ qPk-l Jk-l

:::: -Pk

and that, when p :::: q / P =1= 1,

+ pN) _

Jk :::: _1_ . 1 {k(pk P _ q pk _ pN

2NpN (1 - pk) } . 1 _ pN

4. Problem of the points. A coin is tossed repeatedly, heads turning up with probability P on each toss. Player A wins the game if m heads appear before n tails have appeared, and player B wins otherwise. Let Pmn be the probability that A wins the game. Set up a difference equation for the Pmn. What are the boundary conditions? 5. Consider a simple random walk on the set {O, 1,2, ... , N} in which each step is to the right with probability P or to the left with probability q :::: 1 - p. Absorbing barriers are placed at 0 and N. Show that the number X of positive steps of the walk before absorption satisfies E(X) ::::

HDk -

k

+ N(1

- Pk)}

where Dk is the mean number of steps until absorption and Pk is the probability of absorption at O. 6. (a) "Millionaires should always gamble, poor men never" [J. M. Keynes]. (b) "If I wanted to gamble, I would buy a casino" [Po Getty]. (c) "That the chance of gain is naturally overvalued, we may learn from the universal success of lotteries" [Adam Smith, 1776]. Discuss. tPepys put a simple version of this problem to Newton in 1693, but was reluctant to accept the correct reply he received.

22

Problems

Exercises

[3.10.1]-[3.11.6]

3.10 Exercises. Random walk: counting sample paths 1. Consider a symmetric simple random walk S with So = O. Let T time of the first return of the walk to its starting point. Show that

= min{n

:::: 1 : Sn

= O} be the

lP'(T = 2n) = _1_ (2n) 2-2n, 2n -1 n and deduce that JE(T"') < nn+!e- n5.

00

if and only if a <

!.

You may need Stirling's formula: n! ~

2. For a symmetric simple random walk starting at 0, show that the mass function of the maximum satisfies lP'(Mn = r) = lP'(Sn = r) + lP'(Sn = r + 1) for r :::: O.

3. For a symmetric simple random walk starting at 0, show that the probability that the first visit to S2n takes place at time 2k equals the product lP'(S2k = 0)lP'(S2n-2k = 0), for 0 .:s k .:s n.

3.11 Problems (a) Let X and Y be independent discrete random variables, and let g, h : JR ~ lR. Show that g (X) and heY) are independent. (b) Show that two discrete random variables X and Y are independent if and only if fx,Y(x, y) = fx (x)fy (y) for all x, y E JR. (c) More generally, show that X and Y are independent if and only if fx,Y(x, y) can be factorized as the product g(x)h(y) of a function of x alone and a function of y alone.

1.

2. lP'(X

3.

Show that if var(X) = 0 then X is almost surely constant; that is, there exists a = a) = 1. (First show that if JE(X2) = 0 then lP'(X = 0) = 1.)

E

JR such that

(a) Let X be a discrete random variable and let g : JR ~ JR. Show that, when the sum is absolutely convergent, JE(g(X» = I'>(x)lP'(X = x). x

(b) If X and Y are independent and g, h : JR whenever these expectations exist.

4.

Let Q

~

JR, show that JE(g(X)h(Y» = JE(g(X»JE(h(Y»

= {WI, W2, W3}, with lP'(WI) = lP'(W2) = lP'(W3) = j-. = 1, = 2, Z(WI) = 2,

X(WI)

X(W2)

Y(WI)

Y(W2) Z(W2)

= 2, = 3, = 2,

Define X, Y, Z :

X(W3) Y(W3) Z(W3)

Q

~ JR by

= 3, = 1, = 1.

Show that X and Y have the same mass functions. Find the mass functions of X Find the conditional mass functions fYlz and fZIY'

+ Y, XY, and X/Yo

For what values of k and a is f a mass function, where: + I)}, n = 1,2, ... , (b) fen) = kn"', n = 1,2, ... (zeta or Zipf distribution)?

5.

(a) fen) = k/{n(n

6. Let X and Y be independent Poisson variables with respective parameters). and JL. Show that: (a) X + Y is Poisson, parameter). + JL, (b) the conditional distribution of X, given X + Y = n, is binomial, and find its parameters.

23

[3.11.7]-[3.11.14]

Exercises

Discrete random variables

7. If X is geometric, show that lP'(X = n + k I X > n) = lP'(X = k) for k, n ~ 1. Why do you think that this is called the 'lack of memory' property? Does any other distribution on the positive integers have this property?

8. Show that the sum of two independent binomial variables, bin(m, p) and bin(n, p) respectively, is bin(m + n, p). 9. Let N be the number of heads occurring in n tosses of a biased coin. Write down the mass function of N in terms of the probability p of heads turning up on each toss. Prove and utilize the identity

in order to calculate the probability Pn that N is even. Compare with Problem (1.8.20). 10. An urn contains N balls, b of which are blue and r (= N - b) of which are red. A random sample of n balls is withdrawn without replacement from the urn. Show that the number B of blue balls in this sample has the mass function

This is called the hypergeometric distribution with parameters N, b, and n. Show further that if N, b, and r approach 00 in such a way that b / N ~ P and r / N ~ 1 - p, then

You have shown that, for small n and large N, the distribution of B barely depends on whether or not the balls are replaced in the urn immediately after their withdrawal.

11. Let X and Y be independent bin(n, p) variables, and let Z = X + Y. Show that the conditional distribution of X given Z = N is the hypergeometric distribution of Problem (3.11.10). 12. Suppose X and Y take values in {O, I}, with joint mass function f(x, y). Write f(O, 0) = a, f(O, 1) = b, f(l, 0) = c, f(l, 1) = d, and find necessary and sufficient conditions for X and Y to be: (a) un correlated, (b) independent. 13. (a) If X takes non-negative integer values show that 00

lE(X) =

L lP'(X > n). n=O

(b) An urn contains b blue and r red balls. Balls are removed at random until the first blue ball is drawn. Show that the expected number drawn is (b + r + l)/(b + 1). (c) The balls are replaced and then removed at random until all the remaining balls are of the same colour. Find the expected number remaining in the urn.

14. Let Xl, X2, ... , Xn be independent random variables, and suppose that Xk is Bernoulli with parameter Pk. Show that Y = X I + X2 + ... + Xn has mean and variance given by n

n

lE(Y) =

LPko

var(y)

I

=

L Pk(l I

24

Pk)·

Exercises

Problems

[3.11.15]-[3.11.21]

Show that, for JE(Y) fixed, var(y) is a maximum when PI = P2 = ... = Pn. That is to say, the variation in the sum is greatest when individuals are most alike. Is this contrary to intuition? 15. Let X = (Xl, X2, ... , Xn) be a vector of random variables. The covariance matrix V(X) of X is defined to be the symmetric n by n matrix with entries (vij : 1 ~ i, j ~ n) given by Vij = COV(Xi, Xj). Show that IV(X)I = 0 if and only if the Xi are linearly dependent with probability one, in that lP'(aIXI + a2X2 + ... + anXn = b) = I for some a and b. (IVI denotes the determinant of V.) 16. Let X and Y be independent Bernoulli random variables with parameter ~. Show that X + Y and IX - Y I are dependent though uncorrelated. 17. A secretary drops n matching pairs of letters and envelopes down the stairs, and then places the letters into the envelopes in a random order. Use indicators to show that the number X of correctly matched pairs has mean and variance I for all n 2: 2. Show that the mass function of X converges to a Poisson mass function as n --+ 00. 18. Let X = (Xl, X2, ... , Xn) bea vector of independent random variables each having the Bernoulli distribution with parameter p. Let f : {O, l}n --+ JR be increasing, which is to say that f(x) ~ f(y) whenever Xi ~ Yi for each i. (a) Let e(p) = JE(f(X». Show that e(PI) ~ e(P2) if PI ~ P2. (b) FKG inequalityt. Let f and g be increasing functions from {O, l}n into JR. Show by induction on n that cov(f(X), g(X» 2: O. 19. Let R(p) be the reliability function of a network G with a given source and sink, each edge of which is working with probability p, and let A be the event that there exists a working connection from source to sink. Show that R(p)

= L1A(W)pN(w)(l- p)m-N(w) w

where w is a typical realization (i.e., outcome) of the network, N(w) is the number of working edges of w, and m is the total number of edges of G. Deduce that R' (p) = cov (I A, N) / {p (l - p)}, and hence that R(p)(l - R(p» p(l - p)

< R'( ) < p-

mR(p)(l - R(p» p(l- p)

20. Let R (p ) be the reliability function of a network G, each edge of which is working with probability

p. (a) Show that R(PIP2) ~ R(p})R(P2) if 0 ~ PI, P2 ~ 1. (b) Show that R(pY) ~ R(p)Y for all 0 ~ p ~ I and Y 2: 1. 21. DNA fingerprinting. In a certain style of detective fiction, the sleuth is required to declare "the criminal has the unusual characteristics ... ; find this person and you have your man". Assume that any given individual has these unusual characteristics with probability 10- 7 independently of all other individuals, and that the city in question contains 107 inhabitants. Calculate the expected number of such people in the city. (a) Given that the police inspector finds such a person, what is the probability that there is at least one other? (b) If the inspector finds two such people, what is the probability that there is at least one more? (c) How many such people need be found before the inspector can be reasonably confident that he has found them all? tNamed after C. Fortuin, P. Kasteleyn, and 1. Ginibre (1971), but due in this form to T. E. Harris (1960).

25

[3.11.22]-[3.11.30]

Discrete random variables

Exercises

(d) For the given population, how improbable should the characteristics of the criminal be, in order that he (or she) be specified uniquely? 22. In 1710, J. Arbuthnot observed that male births had exceeded female births in London for 82 successive years. Arguing that the two sexes are equally likely, and 2- 82 is very small, he attributed this run of masculinity to Divine Providence. Let us assume that each birth results in a girl with probability p = 0.485, and that the outcomes of different confinements are independent of each other. Ignoring the possibility of twins (and so on), show that the probability that girls outnumber boys in 2n live births is no greater than enn ) pn qn {q / (q - p)}, where q = 1 - p. Suppose that 20,000 children are born in each of 82 successive years. Show that the probability that boys outnumber girls every year is at least 0.99. You may need Stirling's formula. 23. Consider a symmetric random walk with an absorbing barrier at N and a reflecting barrier at 0 (so that, when the particle is at 0, it moves to 1 at the next step). Let adj) be the probability that the particle, having started at k, visits 0 exactly j times before being absorbed at N. We make the convention that, if k = 0, then the starting point counts as one visit. Show that

adj) =

N-k(

---r:i2

1)1-1 '

1- N

j?:.I,O~k~N.

24. Problem of the points (3.9.4). A coin is tossed repeatedly, heads turning up with probability p on each toss. Player A wins the game if heads appears at least m times before tails has appeared n times; otherwise player B wins the game. Find the probability that A wins the game. 25. A coin is tossed repeatedly, heads appearing on each toss with probability p. A gambler starts with initial fortune k (where 0 < k < N); he wins one point for each head and loses one point for each tail. If his fortune is ever 0 he is bankrupted, whilst if it ever reaches N he stops gambling to buy a Jaguar. Suppose that p < ~. Show that the gambler can increase his chance of winning by doubling the stakes. You may assume that k and N are even. What is the corresponding strategy if p ?:. ~?

26. A compulsive gambler is never satisfied. At each stage he wins £ 1 with probability p and loses £ 1 otherwise. Find the probability that he is ultimately bankrupted, having started with an initial fortune of £k. 27. Range of random walk. Let {Xn : n ?:. I} be independent, identically distributed random variables taking integer values. Let So = 0, Sn = 2:.:1=1 Xi. The range Rn of SO, SI,"" Sn is the number of distinct values taken by the sequence. Show thatll"(Rn = Rn-I + 1) = 1I"(SIS2'" Sn ¥ 0), and deduce that, as n --+ 00, 1

-E(Rn) --+ lI"(Sk

n

¥

0 for all k ?:. 1).

Hence show that, for the simple random walk, n-1E(R n ) --+

Ip - ql as n --+

00.

28. Arc sine law for maxima. Consider a symmetric random walk S starting from the origin, and let Mn = max{Si : 0 ~ i ~ n}. Show that, for i = 2k, 2k + 1, the probability that the walk reaches M2n for the first time at time i equals ill"(S2k = 0)1I"(S2n-2k = 0). 29. Let S be a symmetric random walk with So = 0, and let N n be the number of points that have been visited by S exactly once up to time n. Show that E(Nn ) = 2. 30. Family planning. Consider the following fragment of verse entitled 'Note for the scientist'. People who have three daughters try for more, And then its fifty-fifty they'll have four, Those with a son or sons will let things be,

26

Exercises

Problems

[3.11.31]-[3.11.37]

Hence all these surplus women, QED. (a) What do you think of the argument? (b) Show that the mean number of children of either sex in a family whose fertile parents have followed this policy equals 1. (You should assume that each delivery yields exactly one child whose sex is equally likely to be male or female.) Discuss.

31. Let

f3

> 1, let P1, P2, ... denote the prime numbers, and let N(1), N(2), ... be independent f3

random variables, N(i) having mass function JP'(N(i) = k) = (l - Yi)d for k :::: 0, where Yi =

pi

for all i. Show that M = rr~l pf(i) is a random integer with mass function JP'(M = m) = Cm- f3 for m :::: 1 (this may be called the Dirichlet distribution), where C is a constant satisfying

C

=}]00 (1 - pf1) =(00 E 1)-1 m f3

32. N + 1 plates are laid out around a circular dining table, and a hot cake is passed between them in the manner of a symmetric random walk: each time it arrives on a plate, it is tossed to one of the two neighbouring plates, each possibility having probability ~. The game stops at the moment when the cake has visited every plate at least once. Show that, with the exception of the plate where the cake began, each plate has probability 1I N of being the last plate visited by the cake.

33. Simplex algorithm. There are (~) points ranked in order of merit with no matches. You seek to reach the best, B. If you are at the j th best, you step to anyone of the j - 1 better points, with equal probability of stepping to each. Let rj be the expected number of steps to reach B from the jth best vertex. Show that rj = 'L{;;;t k- 1 . Give an asymptotic expression for the expected time to reach B from the worst vertex, for large m, n.

34. Dimer problem. There are n unstable molecules in a row, m 1, m2, ... ,m n . One of the n - 1 pairs of neighbours, chosen at random, combines to form a stable dimer; this process continues until there remain Un isolated molecules no two of which are adjacent. Show that the probability that m 1 remains isolated is 'L~:6(-1Y Ir! ---+ e- 1 as n ---+ 00. Deduce that limn--+oo n- 1lEUn = e- 2 . 35. Poisson approximation. Let {lr : 1 ::s r

::s n} be independent Bernoulli random variables with ::s n} satisfying Pr ::s c < 1 for all r and some c. Let A = 'L~=1 Pr

respective parameters {Pr : 1 ::s r and X = 'L~=1 X r . Show that

JP'(X =k) =

A~~{ k ! 1 +0

(

~

Am;x Pr + TmaxPr

)}

.

36. Sampling. The length of the tail of the rth member of a troop of N chimeras is Xr. A random sample of n chimeras is taken (without replacement) and their tails measured. Let lr be the indicator of the event that the rth chimera is in the sample. Set

Xr

1 N LXr,

= xr/r, Y = -n

Show thatlE(Y) =

jL,

a

r=l

2

= N1

N -2 ~(Xr - x) . r=l

~

and var(Y) = (N -n)a 2 /{n(N -l)}.

37. Berkson's fallacy. Any individual in a group G contracts a certain disease C with probability y; such individuals are hospitalized with probability c. Independently of this, anyone in G may be in hospital with probability a, for some other reason. Let X be the number in hospital, and Y the

27

[3.11.38]-[3.11.40]

Exercises

Discrete random variables

number in hospital who have C (including those with C admitted for any other reason). Show that the correlation between X and Y is p(X, Y)

=

(1 - a)(1 - yc)

yp

where p = a + c - ac. It has been stated erroneously that, when p(X, Y) is near unity, this is evidence for a causal relation between being in G and contracting c. 38. A telephone sales company attempts repeatedly to sell new kitchens to each of the N families in a village. Family i agrees to buy a new kitchen after it has been solicited Ki times, where the Ki are independent identically distributed random variables with mass function fen) = JP'(Ki = n). The value 00 is allowed, so that f(oo) :::: O. Let Xn be the number of kitchens sold at the nth round of solicitations, so that Xn = 2:~1 I{K;=n}. Suppose that N is a random variable with the Poisson distribution with parameter v. (a) Show that the Xn are independent random variables, Xr having the Poisson distribution with parameter vf(r). (b) The company loses heart after the Tth round of calls, where T = inf{n : Xn = OJ. Let S = Xl + X2 + ... + X T be the number of solicitations made up to time T. Show further that JB:(S) = vJB:(F(T» where F(k) = f(1) + f(2) + ... + f(k). 39. A particle performs a random walk on the non-negative integers as follows. When at the point n (> 0) its next position is uniformly distributed on the set to, 1,2, ... , n + I}. When it hits 0 for the first time, it is absorbed. Suppose it starts at the point a. (a) Find the probability that its position never exceeds a, and prove that, with probability 1, it is absorbed ultimately. (b) Find the probability that the final step of the walk is from 1 to 0 when a = 1. (c) Find the expected number of steps taken before absorption when a = 1. 40. Let G be a finite graph with neither loops nor multiple edges, and write d v for the degree of the vertex v. An independent set is a set of vertices no pair of which is joined by an edge. Let a(G) be the size of the largest independent set of G. Use the probabilistic method to show that a(G) :::: 2:v 1/(dv + 1). [This conclusion is sometimes referred to as Turtin's theorem.]

28

4 Continuous random variables

4.1 Exercises. Probability density functions 1.

For what values of the parameters are the following functions probability density functions? 1

(a) f(x) (b) f(x)

= C{x(l - x) }-'Z, 0 = C exp( - x - e- X ),

(c) f(x)

= C(1 + x 2 )-m,

< x < 1, the density function of the 'arc sine law'.

x E JR, the density function of the 'extreme-value distribution'.

x E lE..

2. Find the density function of Y = aX, where a > 0, in terms of the density function of X. Show that the continuous random variables X and - X have the same distribution function if and only if fx(x) = fx(-x) for all x E lE.. 3. If f and g are density functions of random variables X and Y, show that af + (1 - a)g is a density function for 0 ::: a ::: 1, and describe a random variable of which it is the density function. 4. Survival. Let X be a positive random variable with density function f and distribution function F. Define the hazard Junction H(x) = -log[1 - F(x)] and the hazard rate r(x)

= lim ..!..JP'(X ::: x + h I X > h+O h

x),

x::: o.

Show that: (a) r(x) = H'(x) = f(x)/{1 - F(x»), (b) If r (x) increases with x then H (x) / x increases with x, (c) H(x)/x increases with x if and only if [1 - F(x)]a ::: 1 - F(ax) for all 0::: a::: 1, (d) If H(x)/x increases with x, then H(x + y) ::: H(x) + H(y) for all x, y ::: o.

4.2 Exercises. Independence 1. I am selling my house, and have decided to accept the first offer exceeding £ K . Assuming that offers are independent random variables with common distribution function F, find the expected number of offers received before I sell the house. 2. Let X and Y be independent random variables with common distribution function F and density function f. Show that V = max{X, Y} has distribution function JP'(V ::: x) = F(x)2 and density function fv(x) = 2f(x)F(x), x E R Find the density function of U = minIX, Y}. 3. The annual rainfall figures in Bandrika are independent identically distributed continuous random variables {Xr : r ::: I}. Find the probability that:

29

[4.2.4]-[4.4.5]

Continuous random variables

Exercises

(a) Xl < X2 < X3 < X4, (b) Xl > X2 < X3 < X4·

4. Let {Xr : r ~ I} be independent and identically distributed with distribution function F satisfying F(y) < 1 for all y, and let Y(y) = rnin{k : Xk > y}. Show that lim lP'(Y(y):s: lEY(y»)

y---+oo

= 1-

e- l .

4.3 Exercises. Expectation 1.

For what values of ex is lE(IXIO!) finite, if the density function of X is: (a) f(x) = e- x for x ~ 0,

(b) f(x) = C(l

+ x 2 )-m for x

E IR? If ex is not integral, then lE(IXIO!) is called the fractional moment of order ex of X, whenever the expectation is well defined; see Exercise (3.3.5).

2. Let Xl, X2, ... , Xn be independent identically distributed random variables for which lE(Xil) exists. Show that, if m :s: n, then lE(Sml Sn) = min, where Sm = Xl + X2 + ... + X m . 3.

Let X be a non-negative random variable with density function

for any r 4.

Show that

1 for which the expectation is finite.

Show that the mean fL, median m, and variance a 2 of the continuous random variable X satisfy

(fL - m)2

5.

~

f.

:s: a 2 .

Let X be a random variable with mean fL and continuous distribution function F. Show that

f

a

F(x)dx

1

00

[1- F(x)]dx,

a

-00

if and only if a

=

= fL. 4.4 Exercises. Examples of continuous variables

Prove that the gamma function satisfies ret) = (t - l)r(t - 1) for t > 1, and deduce that r(n) = (n - I)! for n = 1,2, .... Show that r(!) = and deduce a closed form for r(n + !) for n = 0, 1,2, ....

1.

..;n

2.

Show, as claimed in (4.4.8), that the beta function satisfies B(a, b)

= r(a)r(b)1 rea + b).

3. Let X have the uniform distribution on [0, 1]. For what function g does Y = g(X) have the exponential distribution with parameter 1? 4. Find the distribution function of a random variable X with the Cauchy distribution. For what values of ex does IXI have a finite (possibly fractional) moment of order ex? 5. Log-normal distribution. Let Y function of Y.

= eX

where X has the N(O, 1) distribution. Find the density

30

Exercises

Dependence

[4.4.6]-[4.5.5]

6.

Let X be N(Il, a 2). Show that E{(X - ll)g(X)} = a 2E(g'(X)) when both sides exist.

7.

With the terminology of Exercise (4.1.4), find the hazard rate when:

(a) X has the Weibull distribution, JP'(X > x)

= exp( -CiX{3-1),

x 2: 0,

(b) X has the exponential distribution with parameter A,

°

(c) X has density function af + (1 - a)g, where < a < 1 and f and g are the densities of exponential variables with respective parameters A and Il. What happens to this last hazard rate rex) in the limit as x -+ oo? 8. Mills's ratio. For the standard normal density ifJ (x), show that ifJ' (x) + xifJ (x) = 0. Hence show that 1 1 - (x) 1 1 3 ---<---'--'-<---+x> 0. x

x3

ifJ(x)

x

x3

x5'

4.5 Exercises. Dependence 1.

Let f(x, y) =

Ix I exp { -Ixl- 21 X 2 y 2} , ro= '\I8n

X,YEIR.

Show that f is a continuous joint density function, but that the (first) marginal density function g(x) = J~oo f(x, y) dy is not continuous. Let Q = {qn : n 2: I} be a set of real numbers, and define 00

fQ(x, y)

=

L(-~)n f(x - qn, y). n=l

Show that f Q is a continuous joint density function whose first marginal density function is discontinuous at the points in Q. Can you construct a continuous joint density function whose first marginal density function is continuous nowhere?

2. Buffon's needle revisited. Two grids of parallel lines are superimposed: the first grid contains lines distance a apart, and the second contains lines distance b apart which are perpendicular to those of the first set. A needle of length r ( < min {a, b}) is dropped at random. Show that the probability it intersects a line equals r(2a + 2b - r)/(nab). ButTon's cross. The plane is ruled by the lines y = n, for n = 0, ±1, ... , and on to this plane we drop a cross formed by welding together two unit needles perpendicularly at their midpoints. Let Z be the number of intersections of the cross with the grid of parallel lines. Show that E( Z 12) = 21 n and that

3.

var(Z/2)

3-,)2

= -n- -

4 2'

n

If you had the choice of using either a needle of unit length, or the cross, in estimating 21n, which would you use?

4. U

5.

Let X and Y be independent random variables each having the uniform distribution on [0, 1]. Let and V = max {X, Y}. Find E(U), and hence calculate cov(U, V).

= min{X, Y}

Let X and Y be independent continuous random variables. Show that E(g(X)h(Y))

= E(g(X))E(h(y)),

whenever these expectations exist. If X and Y have the exponential distribution with parameter 1, find E{exp(i(X + Y))}.

31

[4.5.6]-[4.6.9] 6.

Continuous random variables

Exercises

Three points A, B, C are chosen independently at random on the circumference of a circle. Let X7f. Show that

b(x) be the probability that at least one of the angles of the triangle ABC exceeds

b(x)

={

1::: x ::: ~,

I - (3x - 1)2

if

3(l-x)2

if1:::x:::1.

Hence find the density and expectation of the largest angle in the triangle.

7. X

Let (X r : 1 ::: r ::: n} be independent and identically distributed with finite variance, and define L~=l X r . Show that cov(X, Xr - X) =

8. V

o.

= n- l =

Let X and Y be independent random variables with finite variances, and let U = X XY. Under what condition are U and V uncorrelated?

+ Y and

9. Let X and Y be independent continuous random variables, and let U be independent of X and Y taking the values ± 1 with probability Define S = U X and T = U Y. Show that S and T are in general dependent, but S2 and T2 are independent.

1.

4.6 Exercises. Conditional distributions and conditional expectation 1. A point is picked uniformly at random on the surface of a unit sphere. Writing E> and for its longitude and latitude, find the conditional density functions of E> given <1>, and of given E>. 2. Show that the conditional expectation 1/!(X) = JE(Y for any function g for which both expectations exist.

3. JE(Y

I X)

satisfies JE(1/!(X)g(X»

Construct an example of two random variables X and Y for which JE(Y) I X) < 00 almost surely.

=

00

=

JE(Y g(X»,

but such that

4. Find the conditional density function and expectation of Y given X when they hav~ joint density function: (a) f(x, y) = ),.2 e -)"y for 0 ::: x::: y < 00, (b) f(x, y)

= xe-x(y+l) for x, y

2: O.

5.

Let Y be distributed as bin(n, X), where X is a random variable having a beta distribution on [0, 1] with parameters a and b. Describe the distribution of Y, and find its mean and variance. What is the distribution of Y in the special case when X is uniform?

6.

Let (X, : r 2: I} be independent and uniformly distributed on [0, 1]. Let 0 < x < 1 and define

N = rnin(n 2: 1 : Xl

+ X2 + ... + Xn

> x}.

Show that W(N > n) = xn In!, and hence find the mean and variance of N. 7.

Let X and Y be random variables with correlation p. Show that JE(var(Y

I X»

::: (l - p2) var Y.

8. Let X, Y, Z be independent and exponential random variables with respective parameters),., /1-, v. Find W(X < Y < Z). 9. Let X and Y have the joint density f(x, y) = cx(y - x)e- y , 0::: x ::: y < 00. (a) Find c. (b) Show that: fXIY(x fYlx(y

I y) = I x) =

6x(y - x)y-3, (y - x)e x - y ,

32

x::: 0::: x:::

0:::

y,

y <

00.

Exercises

Functions of random variables

=

(c) Deduce thatlE(X I Y)

~Y and lE(Y

I X) =

X

[4.6.10]-[4.7.12]

+ 2.

10. Let {Xr : r ::: O} be independent and identically distributed random variables with density function f and distribution function F. Let N = min(n ::: 1 : Xn > Xo} and M = min(n ::: 1 : Xo ::: Xl ::: ... ::: X n-1 < X n }. Show that XN has distribution function F + (1 - F) log(1 - F), and find lP'(M = m).

4.7 Exercises. Functions of random variables 1. Let X, Y, and Z be independent and uniformly distributed on [0, 1]. Find the joint density function of XY and Z2, and show thatlP'(XY < Z2) = ~. 2. Let X and Y be independent exponential random variables with parameter 1. Find the joint density function of U = X + Y and V = X/eX + Y), and deduce that V is uniformly distributed on [0,1]. 3.

Let X be uniformly distributed on [0, !;rr]. Find the density function of Y

4.

Find the density function of Y

=

=

sin X.

sin -1 X when:

(a) X is uniformly distributed on [0, 1], (b) Xis uniformly distributed on [-1, 1]. 5.

Let X and Y have the bivariate normal density function f(x, y)

=

2;rr

n

1 - p2

exp {-

2 1 2 (x - 2pxy 2(1 - p )

+ y2)} .

Show that X and Z = (Y - pX)/~ are independent N(O, 1) variables, and deduce that lP'(X > 0, Y > 0)

6. Z 7. Z

1

1

4

2;rr

= - +-

sin- 1 p.

Let X and Y have the standard bivariate normal density function of Exercise (5), and define Show that lE(Z) = .J(1 - p )/;rr, and lE(Z2) = 1.

= max(X, Y}.

Let X and Y be independent exponential random variables with parameters A and /1,. Show that < Y}. Find:

= min{X, Y} is independent of the event (X (a) lP'(X = Z),

= max{X -

(b) the distributions of U

(c) lP'(X

.:s t

< X

+ Y) where t >

Y, O}, denoted (X - Y)+, and V

= max{X, Y} -

minIX, Y},

O.

8. A point (X, Y) is picked at random uniformly in the unit circle. Find the joint density of Rand X, where R2 = X2 + y2. 9. A point (X, Y, Z) is picked uniformly at random inside the unit ball of ffi.3. Find the joint density of Z and R, where R2 = X2 + y2 + Z2.

10. Let X and Y be independent and exponentially distributed with parameters A and /1-. Find the joint distribution of S = X + Y and R = X/eX + Y). What is the density of R? 11. Find the density of Y

= a/(1 + X2),

where X has the Cauchy distribution.

12. Let (X, Y) have the bivariate normal density of Exercise (5) with 0 [l - (a)][1 - (c)]

.:s lP'(X >

a, Y > b)

.:s [l 33

.:s p

(a)][1 - (c)]

+

< 1. Show that

p¢l(b)[l - (d)] ¢lea) ,

[4.7.13]-[4.8.8] Exercises

Continuous random variables

where c = (b - pa)/~, d = (a - pb)/~, and ifJ and are the density and distribution function of the N (0, 1) distribution.

13. Let X have the Cauchy distribution. Show that Y = X-l has the Cauchy distribution also. Find another non-trivial distribution with this property of invariance. 14. Let X and Y be independent and garruna distributed as rCA, a), rCA, fl) respectively. Show that W = X + Y and Z = X/ (X + Y) are independent, and that Z has the beta distribution with parameters a, fl.

4.8 Exercises. Sums of random variables 1.

Let X and Y be independent variables having the exponential distribution with parameters A and + Y.

/L respectively. Find the density function of X

2. Let X and Y be independent variables with the Cauchy distribution. Find the density function of aX + flY where afl (Do you know about contour integration?)

to.

3. Find the density function of Z = X i(x + y)e-(X+Y), x, y :::: o.

+ Y when X and

Y have joint density function f(x, y)

=

4. Hypoexponential distribution. Let {Xr : r :::: I} be independent exponential random variables with respective parameters {Ar : r :::: I} no two of which are equal. Find the density function of Sn = 2:~=l X r · [Hint: Use induction.] 5.

(a) Let X, Y, Z be independent and uniformly distributed on [0, 1]. Find the density function of X+Y+Z.

(b) If {Xr : r :::: I} are independent and uniformly distributed on [0, 1], show that the density of 2:~=l Xr

at any point x

E

(0, n) is a polynomial in x of degree n - l.

6. For independent identically distributed random variables X and Y, show that U = K + Y and V = X - Y are un correlated but not necessarily independent. Show that U and V are independent if X and Y are N(O, 1). 7.

Let X and Y have a bivariate normal density with zero means, variances 0'2, r2, and correlation

p. Show that:

(a) E(X

I Y)

(b) var(X (c) E(X

= pO' Y, r

I Y) = 0'2(1

- p2),

I X + Y = z) =

(d) var(X I X

(0'2 + par)z 0'2 + 2par + r 2 '

+ Y = z) =

a2r2(1- p2) r2 + 2par + 0'2·

8. Let X and Y be independent N(O, 1) random variables, and let Z and density of Z given that X > 0 and Y > O. Show that E(Z

IX

> 0, Y > 0)

34

=

2J2/n.

=

X + Y. Find the distribution

Exercises [4.9.1]-[4.9.9]

Multivariate normal distribution

4.9 Exercises. Multivariate normal distribution 1. A symmetric matrix is called non-negative (respectively positive) definite if its eigenvalues are non-negative (respectively strictly positive). Show that a non-negative definite symmetric matrix V has a square root, in that there exists a symmetric matrix W satisfying W2 = V. Show further that W is non-singular if and only if V is positive definite. 2.

If X is a random vector with the N(IL, V) distribution where V is non-singular, show that Y =

(X -IL)W- l has the N(O, I) distribution, where I is the identity matrix and W is a symmetric matrix satisfying W2 = V. The random vector Y is said to have the standard multivariate normal distribution. Let X = (Xl, X2, ... , Xn) have the N(IL, V) distribution, and show that Y = alXI + a2X2 + ... + anXn has the (univariate) N(/-t, a 2 ) distribution where

3.

n

n

/-t = I>ilE(Xi),

a

2

=

i=l

4.

Lat var(Xi) i=l

+ 2 LaiajCov(Xi, Xj). i<j

Let X and Y have the bivariate normal distribution with zero means, unit variances, and correlation + Y and X - Y, and their marginal density functions.

p. Find the joint density function of X

S.

Let X have the N(O, 1) distribution and let a > 0. Show that the random variable Y given by

Y

={

X -X

if IXI < a if IXI ~ a

has the N(O, 1) distribution, and find an expression for pea) = cov(X, Y) in terms of the density function c/J of X. Does the pair (X, Y) have a bivariate normal distribution?

6.

Let {Yr : 1 :::: r :::: n} be independent N(O, 1) random variables, and define Xj 1 :::: r :::: n, for constants Cjr. Show that

= 2::~=1 CjrYr,

What is var(Xj I Xk)?

7.

Let the vector (Xr : 1 :::: r :::: n) have a multivariate normal distribution with covariance matrix V = (vij). Show that, conditional on the event 2::'1 Xr = x, Xl has the N(a, b) distribution where a = (ps/t)x, b = s2(l - p2), and s2 = V11, t 2 = 2::ij Vij, P = 2:: i ViI/Cst).

8. Let X, Y, and Z have a standard trivariate normal distribution centred at the origin, with zero means, unit variances, and correlation coefficients PI> P2, and P3. Show that 1 1 JP'(X > 0, Y > 0, Z > 0) = - + -{sin- l PI 8 411

+ sin- l P2 + sin- l P3}.

9. Let X, Y, Z have the standard trivariate normal density of Exercise (8), with PI = p(X, Y). Show that IE(Z

I X,

+ (P2 - PIP3)Y}/(1 - Pf), pi - p~ + 2PIP2P3 }/(l - pf)·

Y) = {(P3 - PIP2)X

var(Z I X, Y)

= {I -

Pf -

35

[4.10.1]-[4.11.7]

Continuous random variables

Exercises

4.10 Exercises. Distributions arising from the normal distribution 1. Let X I and X2 be independent variables with the Show that X I + X2 has the x2 (m + n) distribution.

x2 (m)

and

x2 (n)

distributions respectively.

2. Show that the mean of the t (r) distribution is 0, and that the mean of the F (r, s) distribution is s / (s - 2) if s > 2. What happens if s :s 2? 3.

Show that the t(l) distribution and the Cauchy distribution are the same.

4. Let X and Y be independent variables having the exponential distribution with parameter 1. Show that X/ Y has an F distribution. Which? 5. Use the result of Exercise (4.5.7) to show the independence of the sample mean and sample variance of an independent sample from the N(p" 0"2) distribution. 6. Let {Xr : 1 :s r :s n} be independent N(O, 1) variables. Let IJ1 E [0, rr] be the angle between the vector (X I, X2, ... , Xn) and some fixed vector in ]R.n. Show that IJ1 has density 1(1/1) = (sin 1/I)n-2 / B(i, in 0 :s 1/1 < rr, where B is the beta function.

-1),

4.11 Exercises. Sampling from a distribution 1.

Uniform distribution. If U is uniformly distributed on [0, 1], what is the distribution of X = LnUJ+l?

2. Random permutation. Given the first n integers in any sequence So, proceed thus: (a) pick any position Po from (1, 2, ... ,n} at random, and swap the integer in that place of So with the integer in the nth place of So, yielding SI. (b) pick any position PI from (1,2, ... , n - 1} at random, and swap the integer in that place of SI with the integer in the (n - l)th place of S}, yielding S2, (c) at the (r - l)th stage the integer in position Pr-I, chosen randomly from (1,2, ... , n - r + I}, is swapped with the integer at the (n - r + 1)th place of the sequence Sr-I. Show that Sn-I is equally likely to be any of the n! permutations of (1, 2, ... , n}. 3. Gamma distribution. Use the rejection method to sample from the gamma density r (J..., t) where t (::: 1) may not be assumed integral. [Hint: You might want to start with an exponential random variable with parameter l/t.] 4. Beta distribution. Show how to sample from the beta density f3(a, f3) where a, f3 ::: 1. [Hint: Use Exercise (3).] 5.

Describe three distinct methods of sampling from the density I(x)

= 6x(l -

x), 0

:s x :s 1.

6. Aliasing method. A finite real vector is called a probability vector if it has non-negative entries with sum 1. Show that a probability vector p of length n may be written in the form 1 p= --1

n-

n

LVr, r=1

where each Vr is a probability vector with at most two non-zero entries. Describe a method, based on this observation, for sampling from p viewed as a probability mass function. 7.

Box-Muller normals. Let UI and U2 be independent and uniformly distributed on [0, 1], and

let Ti

= 2Ui

- 1. Show that, conditional on the event that R X=

~J-210gR2,

Y=

36

=

JTl

+ Ti :s

~J-210gR2,

1,

Exercises

Coupling and Poisson approximation

[4.11.8]-[ 4.12.3]

are independent standard normal random variables. 8. Let U be uniform on [0, 1] and 0 < q < 1. Show that X distribution.

= 1 + Llog U Ilog qJ

9. A point (X, Y) is picked uniformly at random in the semicircle x 2 + y2 distribution of Z = Y I X?

:::

has a geometric

1, x 2: O. What is the

10. Hazard-rate technique. Let X be a non-negative integer-valued random variable with her) = JP>(X = r I X 2: r). If {Ui : i 2: O} are independent and uniform on [0, 1], show that Z = min{n : Un ::: hen)} has the same distribution as X. 11. Antithetic variables. Let g(XI, X2, ... , x n ) be an increasing function in all its variables, and let fUr : r 2: I} be independent and identically distributed random variables having the uniform distribution on [0, 1]. Show that

cOV{g(UI, U2, ... , Un), g(l - UI, 1 - U2, ... , 1 - Un)} ::: O.

[Hint: Use the FKG inequality of Problem (3.10.18).] Explain how this can help in the efficient estimation of / = g(x) dx.

JJ

12. Importance sampling. We wish to estimate / = J g(x)fx(x) dx = E(g(X», where either it • is difficult to sample from the density fx, or g(X) has a very large variance. Let fy be equivalent to fx, which is to say that, for all x, fx(x) = 0 if and only if fy(x) = O. Let {Yi : 0 ::: i ::: n} be independent random variables with density function fy, and define J

=~

t

g(Yr)fx(Yr ).

n r=1

fY(Yr)

Show that: (a) E(J)

= / =E

(b) var(J)

=~ n

[g(y)fx(Y)] fy(y) ,

[E (g(Y)2fy(Y)2 fx(y)2) _ /2],

(c) J ~ / as n -+ 00. (See Chapter 7 for an account of convergence.) The idea here is that fy should be easy to sample from, and chosen if possible so that var J is much smaller than n- I [E(g(X)2) - /2]. The function fy is called the importance density. 13. Construct two distinct methods of sampling from the arc sin density 2

f(x) =

~' 1CV 1 - x-

0::: x:::

1.

4.12 Exercises. Coupling and Poisson approximation 1. Show that X is stochastically larger than Y if and only if E(u(X» decreasing function u for which the expectations exist.

2: E(u(y» for any non-

2. Let X and Y be Poisson distributed with respective parameters A and /-t. Show that X is stochastically larger than Y if A 2: /-t.

3.

Show that the total variation distance between two discrete variables X, Y satisfies dTV(X, Y)

= 2 sup IJP>(X E A) A~lR

37

JP>(Y

E

A)I.

[4.12.4]-[4.13.6]

Exercises

Continuous random variables

4. Maximal coupling. Show for discrete random variables X, Y that lP'(X = y) :::: 1- idTV(X, y), where dTV denotes total variation distance. 5.

Maximal coupling continued. Show that equality is possible in the inequality of Exercise (4.12.4) in the following sense. For any pair X, Y of discrete random variables, there exists a pair X', Y' having the same marginal distributions as X, Y such that lP'(X' = Y') = 1 - idTV(X, y).

6. Let X and Y be indicator variables with EX = p, EY = q. What is the maximum possible value of lP'(X = y), as a function of p, q? Explain how X, Y need to be distributed in order that lP'(X = y) be: (a) maximized, (b) minimized.

4.13 Exercises. Geometrical probability With apologies to those who prefer their exercises better posed ... 1.

Pick two points A and B independently at random on the circumference of a circle C with centre Let n be the length of the perpendicular from 0 to the line AB, and let e be the angle AB makes with the horizontal. Show that (n, e) has joint density

o and unit radius.

f(p,e)=

2~'

0:::: p :::: 1, 0::::

rr y 1 - p-

e<

2rr.

2. Let SI and S2 be disjoint convex shapes with boundaries of length b(SI), b(S2), as illustrated in the figure beneath. Let b(H) be the length of the boundary of the convex hull of SI and S2, incorporating their exterior tangents, and b(X) the length of the crossing curve using the interior tangents to loop round SI and S2. Show that the probability that a random line crossing SI also crosses S2 is {b(X) - b(H)}/b(SI). (See Example (4.13.2) for an explanation of the term 'random line'.) How is this altered if SI and S2 are not disjoint?

The circles are the shapes SI and S2. The shaded regions are denoted A and B, and b(X) is the sum of the perimeter lengths of A and B.

3. Let S 1 and S2 be convex figures such that S2 ~ S I. Show that the probability that two independent random lines)'1 and ),.2, crossing Sj, meet within S2 is 2rrIS21/b(SI)2, where IS21 is the area of S2 and b(SI) is the length of the boundary of SI. (See Example (4.13.2) for an explanation of the term 'random line'.) 4. Let Z be the distance between two points picked independently at random in a disk of radius a. Show thatE(Z) = 128a/(45rr), andE(Z2) = a 2 . 5. Pick two points A and B independently at random in a ball with centre O. Show that the probability that the angle is obtuse is ~. Compare this with the corresponding result for two points picked at random in a circle.

WE

6. A triangle is formed by A, B, and a point P picked at random in a set S with centre of gravity G. Show that EIABPI = IABGI.

38

Problems

Exercises

[4.13.7]-[4.14.1]

7. A point D is fixed on the side BC of the triangle ABC. Two points P and Q are picked independently at random in ABD and ADC respectively. Show that E IAPQI = IAGl G21 = ~ IABC! , where Gl and G2 are the centres of gravity of ABD and ADC. 8. From the set of all triangles that are similar to the triangle ABC, similarly oriented, and inside ABC, one is selected uniformly at random. Show that its mean area is IABC!.

lo

9. Two points X and Y are picked independently at random in the interval (0, a). By varying a, show that F(z, a) = lP'(IX - YI ~ z) satisfies 8F 2 2z - + - F = -2, 8a a a

and hence find F(z, a). Let r :::: 1, and show that mr(a)

o ~ z ~ a, = E(iX -

yn satisfies

Hence find mr(a).

10. Lines are laid down independently at random on the plane, dividing it into polygons. Show that the average number of sides of this set of polygons is 4. [Hint: Consider n random great circles of a sphere of radius R; then let R and n increase.] 11. A point P is picked at random in the triangle ABC. The lines AP, BP, CP, produced, meet BC, AC, AB respectively at L, M, N. Show that EILMNI = (10 - ;rr2)IABC!.

12. Sylvester's problem. If four points are picked independently at random inside the triangle ABC, show that the probability that no one of them lies inside the triangle formed by the other three is ~. 13. If three points P, Q, R are picked independently at random in a disk of radius a, show that E IPQR I = 35a 2 / (48;rr). [You may find it useful that Ief Ief sin 3 x sin 3 y sin Ix - y Idx dy = 35;rr /128.] 14. Two points A and B are picked independently at random inside a disk C. Show that the probability that the circle having centre A and radius IABllies inside Cis

!.

15. Two points A and B are picked independently at random inside a ball S. Show that the probability that the sphere having centre A and radius IABllies inside S is

do.

4.14 Problems 1.

2

(a) Show that I~ e- x dx

= v'Ji, and deduce that 1

f(x)

= a $ exp

{(X - JL)2} 2a 2

'

-00

< x <

00,

is a density function if a > O. (b) Calculate the mean and variance of a standard normal variable. (c) Show that the N(O, 1) distribution function satisfies x> O.

These bounds are of interest because has no closed form. (d) Let X be N(O, 1), and a> O. Show that lP'(X > x + a/x I X > x)

39

~

e- a as x

~

O.

[4.14.2]-[4.14.11]

Exercises

Continuous random variables

Let X be continuous with density function f(x) = C(x - x 2 ), where a < x < (a) What are the possible values of a and f3? (b) What is C?

2.

3.

f3 and C

> O.

Let X be a random variable which takes non-negative values only. Show that 00

00

L(i -1)IAi ::: X < LilAi' i=1

i=1

where Ai = {i - 1 ::: X < i}. Deduce that 00

00

LlP'(X :::: i) ::: E(X) < 1 + LlP'(X:::: i). i=1

i=1

4.

(a) Let X have a continuous distribution function F. Show that (i) F(X) is uniformly distributed on [0,1], (ii) -log F(X) is exponentially distributed. (b) A straight line I touches a circle with unit diameter at the point P which is diametrically opposed on the circle to another point Q. A straight line QR joins Q to some point R on I. If the angle PQR between the lines PQ and QR is a random variable with the uniform distribution on [- ~ rr, rr], show that the length of PR has the Cauchy distribution (this length is measured positive or negative depending upon which side of P the point R lies).

i

5. Let X have an exponential distribution. Show that lP'(X > s + x I X > s) = lP'(X > x), for x, s :::: O. This is the 'lack of memory' property again. Show that the exponential distribution is the only continuous distribution with this property. You may need to use the fact that the only non-negative monotonic solutions of the functional equation g(s + t) = g(s)g(t) for s, t :::: 0, with g(O) = 1, are oftheformg(s) = elLS. Can you prove this? 6.

Show that X and Y are independent continuous variables if and only if their joint density function = g (x)h (y) of functions of the single variables x and y alone.

f factorizes as the product f (x, y)

7. Let X and Y have joint density function f(x, y) = 2e- x - y , 0 < x < y < independent? Find their marginal density functions and their covariance.

00.

Are they

8. Bertrand's paradox extended. A chord of the unit circle is picked at random. What is the probability that an equilateral triangle with the chord as base can fit inside the circle if: (a) the chord passes through a point P picked uniformly in the disk, and the angle it makes with a fixed direction is uniformly distributed on [0, 2rr), (b) the chord passes through a point P picked uniformly at random on a randomly chosen radius, and the angle it makes with the radius is uniformly distributed on [0, 2rr).

Jd

9.

Monte Carlo. It is required to estimate J = g(x)dx where 0 ::: g(x) ::: 1 for all x, as in Example (2.6.3). Let X and Y be independent random variables with common density function f(x) = 1 if 0 < x < 1, f(x) = 0 otherwise. Let U = I(Y:;:g(x)), the indicator function of the event that Y ::: g(X), and let V = g(X), W = i{g(X)+g(1- X)}. Show thatE(U) = E(V) = E(W) = J, and that var(W) ::: var(V) ::: var(U), so that, of the three, W is the most 'efficient' estimator of J. 10. Let Xl, X 2, ... , Xn be independent exponential variables, parameter A. Show by induction that S = Xl + X2 + '" + Xn has the r(A, n) distribution.

11. Let X and Y be independent variables, r(A, m) and r(A, n) respectively. (a) Use the result of Problem (4.14.10) to show that X + Y is r(A, m +n) when m and n are integral (the same conclusion is actually valid for non-integral m and n). 40

Exercises

Problems

(b) Find the joint density function of X

[4.14.12]-[4.14.19]

+ Y and X/(X + Y), and deduce that they are independent.

(c) If Z is Poisson with parameter At, and m is integral, show that JP'(Z < m) = JP'(X > t). (d) If 0 < m < n and B is independent of Y with the beta distribution with parameters m and n - m, show that Y B has the same distribution as X.

12. Let Xl, X2, ... , Xn be independent N(O, 1) variables. (a) Show that Xf is

x2 (1).

(b) Show that Xf + X~ is X2(2) by expressing its distribution function as an integral and changing to polar coordinates. (c) More generally, show that Xf

+ X~ + ... + X~ is x2 (n).

13. Let X and Y have the bivariate normal distribution with means fJ-l, fJ-2, variances correlation p. Show that (a) lE(X I Y)

= fJ-l + pal (Y -

fJ-2)/a2,

(b) the variance of the conditional density function fXIY is var(X I Y) =

af, ai, and

af(1 _ p2).

14. Let X and Y have joint density function f. Find the density function of Y / X. 15. Let X and Y be independent variables with common density function has the uniform distribution on (- ~n, ~n) if and only if

1

f.

Show that tan -1 (Y / X)

1

00

f(x)f(xy)lxldx =

-00

n(1

+y

Y

2 '

)

E

lR.

Verify that this is valid if either f is the N(O, 1) density function or f(x) constant a.

= a(1 + x 4 )-1

for some

16. Let X and Y be independent N (0, 1) variables, and think of (X, Y) as a random point in the plane. Change to polar coordinates (R, e) given by R2 = X2 + y2, tan e = Y / X; show that R2 is X 2 (2), tan has the Cauchy distribution, and Rand are independent. Find the density of R. Find lE(X 2 / R2) and

e

e

rninUXI, IYi} } lE { max{IXI,IYi} .

17. If X and Y are independent random variables, show that U have distribution functions FU(u)

=

1 - {I - Fx(u)}{l - Fy(u)},

= rnin{X, Y} and V = max{X, Y}

Fv(v)

= Fx(v)Fy(v).

Let X and Y be independent exponential variables, parameter 1. Show that (a) U is exponential, parameter 2, (b) V has the same distribution as X

+ ~ Y.

Hence find the mean and variance of V.

18. Let X and Y be independent variables having the exponential distribution with parameters A and fJ- respectively. Let U = rnin{X, Y}, V = max{X, Y}, and W = V - U. (a) Find JP'(U

= X) = JP'(X :s Y).

(b) Show that U and W are independent.

19. Let X and Y be independent non-negative random variables with continuous density functions on (0, 00). (a) If, given X + Y = u, X is uniformly distributed on [0, u] whatever the value of u, show that X and Y have the exponential distribution.

41

[4.14.20]-[4.14.27]

Exercises

Continuous random variables

(b) If, given that X + Y = u, X/u has a given beta distribution (parameters a and (3, say) whatever the value of u, show that X and Y have gamma distributions. You may need the fact that the only non-negative continuous solutions of the functional equation g(s + t) = g(s)g(t) for s, t 2:: 0, with g(O) = 1, are of the form g(s) = eJ-Ls. Remember Problem (4.14.5).

20. Show that it cannot be the case that U = X + Y where U is uniformly distributed on [0, 1] and X and Y are independent and identically distributed. You should not assume that X and Y are continuous variables. 21. Order statistics. Let Xl, X 2, ... , Xn be independent identically distributed variables with a common density function f. Such a collection is called a random sample. Foreachw E n, arrange the sampIe values Xl (w), ... , Xn(w) in non-decreasing order X(l)(w) ::::: X(2)(W) ::::: ... ::::: X(n)(w), where (1), (2), ... , (n) is a (random) permutation of 1,2, ... ,n. The new variables X(l)' X(2), ... , X(n) are called the order statistics. Show, by a symmetry argument, that the joint distribution function of the order statistics satisfies lP'(X(l) ::::: YI,"" X(n) ::::: Yn)

= n!lP'(XI =

::::: YI,"" Xn ::::: Yn, Xl < X 2 < ... < Xn)

J... J~2:5.Y2

(I:5.YI L(XI' ... , Xn)n! f(XI) ... f(xn) dXI ... dX n

where L is given by L(x)

and x = (XI,X2,'" ,xn ).

={

~

if Xl < X2 < ... < Xn ,

otherwise, Deduce that the joint density function of X{l), ... , X(n) is g(y)

n! L(y)f(YI)'" f(Yn).

22. Find the marginal density function of the kth order statistic X (k) of a sample with size n: (a) by integrating the result of Problem (4.14.21), (b) directly.

23. Find the joint density function of the order statistics of n independent uniform variables on [0, T]. 24. Let Xl, X 2, ... , Xn be independent and uniformly distributed on [0, 1], with order statistics X{l), X(2),"" X(n)' (a) Show that, for fixed k, the density function of nX(k) converges as n --+ 00, and find and identify the limit function. (b) Show that log X (k) has the same distribution as - 2:.f=k i-I Yi, where the Yi are independent random variables having the exponential distribution with parameter 1. (c) Show that ZI, Z2,"" Zn, defined by Zk = (X(k)/ X(k+l)k for k < nand Zn independent random variables with the uniform distribution on [0, 1].

=

(X(n»n, are

25. Let Xl, X2, X3 be independent variables with the uniform distribution on [0, 1]. What is the probability that rods of lengths Xl, X2, and X3 may be used to make a triangle? Generalize your answer to n rods used to form a polygon.

26. Let Xl and X 2 be independent variables with the uniform distribution on [0, 1]. A stick of unit length is broken at points distance Xl and X 2 from one of the ends. What is the probability that the three pieces may be used to make a triangle? Generalize your answer to a stick broken in n places. 27. Let X, Y be a pair of jointly continuous variables. (a) HOlder's inequality. Show that if p, q > 1 and p-I

+ q-I = 1 then

lE.IXYI::::: {lE.IXPI}I/P{lE.lyql}I/q.

42

Exercises

Problems

Set p = q = 2 to deduce the Cauchy-Schwarz inequality lE(XY)2 (b) Minkowski's inequality. Show that, if p 2: 1, then

[4.14.28]-[4.14.34]

:s lE(X2)lE(y2).

Note that in both cases your proof need not depend on the continuity of X and Y; deduce that the same inequalities hold for discrete variables. 28. Let Z be a random variable. Choose X and Y appropriately in the Cauchy-Schwarz (or Holder) inequality to show that g(p) = log lElZPI is a convex function of p on the interval of values of p such that lElZPI < 00. Deduce Lyapunov's inequality: {lElZr nllr 2: (lElZ S nIls

whenever r 2: s > O.

You have shown in particular that, if Z has finite rth moment, then Z has finite sth moment for all positive s :s r.

I

29. Show that, using the obvious notation, lE{lE(X I Y, Z) Y}

= lE(X

I Y).

30. Motor cars of unit length park randomly in a street in such a way that the centre of each car, in tum, is positioned uniformly at random in the space available to it. Let m (x) be the expected number of cars which are able to park in a street of length x. Show that m(x

+ 1) = ~ (X {m(y) +m(x x Jo

y)

+ l}dy.

It is possible to deduce that m(x) is about as big as ~x when x is large.

31. ButTon's needle revisited: ButTon's noodle. (a) A plane is ruled by the lines y = nd (n = 0, ±1, ... ). A needle with length L « d) is cast randomly onto the plane. Show that the probability that the needle intersects a line is 2Lj(rcd). (b) Now fix the needle and let C be a circle diameter d centred at the midpoint of the needle. Let A be a line whose direction and distance from the centre of C are independent and uniformly distributed on [0, 2rc] and [0, 1d] respectively. This is equivalent to 'casting the ruled plane at random'. Show that the probability of an intersection between the needle and A is 2Lj(rcd). (c) Let S be a curve within C having finite length L(S). Use indicators to show that the expected number of intersections between S and A is 2L(S)j(rcd). This type of result is used in stereology, which seeks knowledge of the contents of a cell by studying its cross sections. 32. ButTon's needle ingested. In the excitement of calculating rc, Mr Buffon (no relation) inadvertently swallows the needle and is X-rayed. If the needle exhibits no preference for direction in the gut, what is the distribution of the length of its image on the X-ray plate? If he swallowed Buffon's cross (see Exercise (4.5.3» also, what would be the joint distribution of the lengths of the images of the two arms of the cross? 33. Let Xl, X(2)

Xn be independent exponential variables with parameter A, and let be their order statistics. Show that

X2,""

:s ... :s X(n)

X(1) <

l
43

[4.14.35]-[4.14.42]

Exercises

Continuous random variables

are independent and uniformly distributed on [0, 1]. This is equivalent to Problem (4.14.33). Why?

35. Secretary/marriage problem. You are permitted to inspect the n prizes at a fete in a given order, at each stage either rejecting or accepting the prize under consideration. There is no recall, in the sense that no rejected prize may be accepted later. It may be assumed that, given complete information, the prizes may be ranked in a strict order of preference, and that the order of presentation is independent of this ranking. Find the strategy which maximizes the probability of accepting the best prize, and describe its behaviour when n is large. 36. Fisher's spherical distribution. Let R2 = X2 + y2 + Z2 where X, Y, Z are independent normal random variables with means A, /L, v, and common variance a 2 , where (A, /L, v) #- (0,0,0). Show that the conditional density of the point (X, Y, Z) given R = r, when expressed in spherical polar coordinates relative to an axis in the direction e = (A, /L, v), is of the form

fee, 1jJ) = where a

~

4n smha

eacosli sine,

0:::

e<

n, 0::: IjJ < 2n,

= riel.

37. Let IjJ be the N(O, 1) density function, and define the functions Hn, n 2: 0, by Ho (_1)n HnljJ = ljJ(n), the nth derivative of 1jJ. Show that: (a) Hn (x) is a polynomial of degree n having leading term xn, and

1

00

Hm(x)Hn(x)ljJ(x)dx=

{O

-00

ifm Ifm

,.

n.

=

1, and

#-n,

= n.

38. Lancaster's theorem. Let X and Y have a standard bivariate normal distribution with zero means, unit variances, and correlation coefficient P, and suppose U = u (X) and V = v (Y) have finite variances. Show that Ip(U, V)I ::: Ipl. [Hint: Use Problem (4.14.37) to expand the functions u and v. You may assume that u and v lie in the linear span of the Hn.] 39. Let X(1), X(2), [0, 1]. Show that:

... , X(n)

r

(a) E(X(r» = n + l'

be the order statistics of n independent random variables, uniform on r(n-s+1)

(b) cov(X(r)' X(s» = (n + 1)2(n + 2) for r ::: s.

40. (a) Let X, Y, Z be independent N(O, 1) variables, and set R = y'X2 + y2 + Z2. Show that X2 / R2 has a beta distribution with parameters ~ and 1, and is independent of R2. (b) Let X, Y, Z be independent and uniform on [-1, 1] and set R density of X2 / R2 given that R2 ::: l.

=

y'X2 + y2 + Z2. Find the

41. Let IjJ and be the standard normal density and distribution functions. Show that: (a) (x) = 1 - (-x), (b) f(x) = 21jJ (x) (AX), -00 < x < 00, is the density function of some random variable (denoted by Y), and that IYI has density function 21jJ. (c) LetX be a standard norrnal random variable independentofY, and define Z = (X+AIYI)/y'l +A2. Write down the joint density of Z and IY I, and deduce that Z has density function f.

42. The six coordinates (Xi, Yi), 1 ::: i ::: 3, of three points A, B, C in the plane are independent N(O, 1). Show that the the probability that C lies inside the circle with diameter AB is

i.

44

Exercises

Problems

[4.14.43]-[4.14.49]

43. The coordinates (Xi, Yi, Zi), 1 ::: i ::: 3, of three points A, B, C are independent N (0, 1). Show that the probability that C lies inside the sphere with diameter AB is 44. Skewness. Let X have variance a 2 and write mk skw(X) = lE[(X - mj)3]/a 3 . Show that: (a) skw(X) = (m3 - 3mjm2

=

~3 _ .J3. 4n

lE(X k). Define the skewness of X by

+ 2mI)/a 3,

(b) skw(Sn) = skw(Xj)/.Jr!, where Sn = 2:~=j Xr is a sum of independent identically distributed random variables, (c) skw(X) = (1 - 2p)/ .jnpq, when X is bin(n, p) where p + q = 1, (d) skw(X) (e) skw(X)

= 1/...A, when X is Poisson with parameter A, = 2/.,fi, when X is gamma r(A, t), and t is integral.

45. Kurtosis. Let X have variance a 2 and lE(Xk) lE[(X - mj)4]/a 4 . Show that: (a) (b) (c) (d)

=

mk. Define the kurtosis of X by kur(X)

=

kur(X) = 3, when X is N(p" a 2 ), kur(X) = 9, when X is exponential with parameter A, kur(X) = 3 + A-j, when X is Poisson with parameter A, kur(Sn) = 3 + {kur(Xj) - 3}/n, where Sn = 2:~=j Xr is a sum of independent identically distributed random variables.

46. Extreme value. Fisher-Gumbel-Tippett distribution. Let X r , 1 ::: r ::: n, be independent and exponentially distributed with parameter 1. Show that X(n) = max{Xr : 1 ::: r ::: n} satisfies lim JP'(X(n) -logn ::: x) = exp(-e- X ).

n-+oo

Hence show that Jooo {I - exp( _e- X )} dx

= y where y is Euler's constant.

47. Squeezing. Let S and X have density functions satisfying b(x) ::: fs(x) ::: a(x) and fs(x) ::: fx(x). Let U be uniformly distributed on [0,1] and independent of X. Given the value X, we implement the following algorithm: if Ufx(X) > a(X),

reject X;

otherwise: if Ufx(X) < b(X),

accept X;

otherwise: if Ufx(X) ::: fs(X),

accept X;

otherwise: reject X. Show that, conditional on ultimate acceptance, X is distributed as S. Explain when you might use this method of sampling. 48. Let X, Y, and {Ur : r 2: I} be independent random variables, where: JP'(X

= x) = (e -

l)e

-x

, JP'(Y

and the Ur are uniform on [0, 1]. Let M exponentially distributed.

= y) =

1 for x, y (e-l)y!

= 1,2, ... ,

= max{Uj, U2, ... , Uy}, and show that Z = X -

49. Let U and V be independent and uniform on [0, 1]. Set X where a> O.

= -a-jlog U

and Y

(a) Show that, conditional on the event Y 2: ~ (X _a)2, X has density function f(x) for x> O.

45

=

M is

-log V 1 2

= .j2/ne -'1 x

[4.14.50]-[4.14.56]

Exercises

Continuous random variables

(b) In sampling from the density function f, it is decided to use a rejection method: for given a > 0, we sample U and V repeatedly, and we accept X the first time that Y 2: (X - a)2. What is the optimal value of a? (c) Describe how to use these facts in sampling from the N(O, 1) distribution.

i

50. Let S be a semicircle of unit radius on a diameter D. (a) A point P is picked at random on D. If X is the distance from P to S along the perpendicular to D, show JE(X) = n14. (b) A point Q is picked at random on S. If Y is the perpendicular distance from Q to D, show JE(Y)

= 21n.

51. (Set for the Fellowship examination of St John's College, Cambridge in 1858.) 'A large quantity of pebbles lies scattered uniformly over a circular field; compare the labour of collecting them one by one: (i) at the centre 0 of the field, (ii) at a point A on the circumference.' To be precise, if LO and LA are the respective labours per stone, show that JE(LO) = ~a and = 32al (9n) for some constant a. (iii) Suppose you take each pebble to the nearer of two points A or B at the ends of a diameter. Show in this case that the labour per stone satisfies JE(LA)

JE(LAB)

=

4a {16 3n 3

17 - (;h + 2:1 10g(1 + h) } : : : 1.13 x 32 a .

(iv) Finally suppose you take each pebble to the nearest vertex of an equilateral triangle ABC inscribed in the circle. Why is it obvious that the labour per stone now satisfies JE(LABc) < JE(LO)? Enthusiasts are invited to calculate JE(LABc)'

52. The lines L, M, and N are parallel, and P lies on L. A line picked at random through P meets M at Q. A line picked at random through Q meets N at R. What is the density function of the angle e that RP makes with L? [Hint: Recall Exercise (4.8.2) and Problem (4.14.4).] 53. Let l:!. denote the event that you can form a triangle with three given parts of a rod R.

i.

(a) R is broken at two points chosen independently and uniformly. Show that lP'(l:!.) = (b) R is broken in two uniformly at random, the longer part is broken in two uniformly at random. ShowthatlP'(l:!.) =log(4Ie). (c) R is broken in two uniformly at random, a randomly chosen part is broken into two equal parts. Show that lP'(l:!.) =

i.

(d) In case (c) show that, given l:!., the triangle is obtuse with probability 3 -

2h.

54. You break a rod at random into two pieces. Let R be the ratio of the lengths of the shorter to the longer piece. Find the density function fR, together with the mean and variance of R. 55. Let R be the distance between two points picked at random inside a square of side a. Show that JE(R2) = ta2, and that R2 I a 2 has density function

fer)

={

r-

4,Jr +n

4.;;:-=-1- 2 - r

if 0 ::: r ::: 1,

+ 2 sin- 1 ~

- 2 sin- 1

\11 -

r- 1 if 1 ::: r ::: 2.

56. Show that a sheet of paper of area A cm2 can be placed on the square lattice with period 1 cm in such a way that at least I A l points are covered.

46

Problems

Exercises

[4.14.57]-[4.14.63]

57. Show that it is possible to position a convex rock of surface area S in sunlight in such a way that its shadow has area at least S.

!-

58. Dirichlet distribution. Let {Xr : 1 :::0 r :::0 k

+ 1} be independent rCA, f3r) random variables

(respectively). (a) Show that Yr = Xr/(Xl + ... + X r ), 2:::0 r :::0 k + 1, are independent random variables. (b) Show that Zr = Xr / (X 1 + ... + Xk+d, 1 :::0 r :::0 k, have the joint Dirichlet density

= (X lr, X2r, ... , X mr ), 1 :::0 r :::0 n, be independent multivariate normal random vectors having zero means and the same covariance matrix V = (vij). Show that the two random variables

59. Hotelling's theorem. Let Xr

n-l

Tij

=L

XirXjr,

r=l

are identically distributed.

60. Choose P, Q, and R independently at random in the square Sea) of side a. Show that lElPQRI = lla 2 /144. Deduce that four points picked at random in a parallelogram form a convex quadrilateral with probability (~)2. 61. Choose P, Q, and R uniformly at random within the convex region C illustrated beneath. By considering the event that four randomly chosen points form a triangle, or otherwise, show that the mean area of the shaded region is three times the mean area of the triangle PQR.

62. Multivariate normal sampling. Let V be a positive-definite symmetric n x n matrix, and L a lower-triangular matrix such that V = L'L; this is called the Cholesky decomposition of V. Let X = (Xl, X2, ... , Xn) be a vector of independent random variables distributed as N(O, 1). Show that the vector Z = JL + XL has the multivariate normal distribution with mean vector JL and covariance matrix V. 63. Verifying matrix multiplications. We need to decide whether or not AB = C where A, B, C are given n x n matrices, and we adopt the following random algorithm. Let x be a random to, l}n-valued vector, each of the 2n possibilities being equally likely. If (AB - C)x = 0, we decide that AB = C, and otherwise we decide that AB =1= C. Show that

= 1 lP' (the decision is correct) { >.! -

2

if AB

= C,

if AB =1= C.

Describe a similar procedure which results in an error probability which may be made as small as desired.

47

5 Generating functions and their applications

5.1 Exercises. Generating functions 1. Find the generating functions of the following mass functions, and state where they converge. Hence calculate their means and variances. (a) fern) = (n+;::-l)pn(1_ p)m,forrn 2: O. (b) fern) (c) fern)

= {rn(rn + l)}-l, for rn 2: l. = (1- p)plmlj(1 + p), for rn = ... , -1, 0,1, ....

The constant p satisfies 0 < p < l. 2. Let X (2: 0) have probability generating function G and write ten) = JP'(X > n) for the 'tail' probabilities of X. Show that the generating function of the sequence {ten) : n 2: O} is T(s) = (1- G(s»j(1 - s). Show that E(X) = T(1) and var(X) = 2T'(1) + T(1) - T(1)2.

3.

Let Gx,y(s, t) be the joint probability generating function of X and Y. Show that Gx(s) Gx,Y(s, 1) and Gy(t) = Gx,Y(1, t). Show that

;]2 E(XY) = --Gx yes, t)

as at'

I

=

.

s=f=l

4. Find the joint generating functions of the following joint mass functions, and state for what values of the variables the series converge. (a) f(j,k) = (1-a)(f3 - a)a j f3 k- j - 1, forO::: k::: j, where 0 < a < 1, a < f3. (b) f(j, k) = (e - 1)e-(2k+l)k j jj!, for j, k 2: O. (c) f(j,k) =

(7) p H

k(1- p)k-j /[klog{lj(l- p)}J, forO::: j::: k, k 2: 1, where 0 < p < l.

Deduce the marginal probability generating functions and the covariances. 5. A coin is tossed n times, and heads turns up with probability p on each toss. Assuming the usual independence, show that the joint probability generating function of the numbers H and T of heads and tails is G H,T(X, y) = {px + (1 - p)y}n. Generalize this conclusion to find the joint probability generating function of the multinomial distribution of Exercise (3.5.1). 6. Let X have the binomial distribution bin(n, U), where U is uniform on (0, 1). Show that X is uniformly distributed on {O, 1, 2, ... , n}. 7.

Show that G(X, y, z, w) =

k(xyzw + xy + yz + zw + zx + yw + xz + 1) 48

Exercises

Some applications

[5.1.8]-[5.2.5]

is the joint generating function of four variables that are pairwise and triplewise independent, but are nevertheless not independent. 8. Let Pr > 0 and ar E lR for 1 :'S r :'S n. Which of the following is a moment generating function, and for what random variable? n

n

(a) M(t)

= 1+

L Prtr,

(b) M(t)

=

LPreart. r=l

r=l

Let GI and G2 be probability generating functions, and suppose that 0 :'S a :'S 1. Show that GIG2, and aGI + (1 - a)G2 are probability generating functions. Is G(as)/G(a) necessarily a probability generating function? 9.

5.2 Exercises. Some applications 1.

Let X be the number of events in the sequence AI, A2, ... , An which occur. Let Sm

the mean value of the random binomial coefficient

= lE(!),

(!), and show that for 1 :'S i :'S n,

where

L n

Sm =

(. J-

j=m

2.

1)

m -1

lP'(X ::: j),

for 1 :'S m :'S n.

Each person in a group of n people chooses another at random. Find the probability:

(a) that exactly k people are chosen by nobody, (b) that at least k people are chosen by nobody.

3.

Compounding.

(a) Let X have the Poisson distribution with parameter Y, where Y has the Poisson distribution with parameter J-t. Show that Gx+y(x) = exp{J-t(xe x - 1 - I)}. (b) Let X I, X 2, . .. be independent identically distributed random variables with the logarithmic mass function k _ (1- p)k k::: 1, f( ) - klog(1/p)'

where 0 < P < 1. If N is independent of the Xi and has the Poisson distribution with parameter J-t, show that Y = 2:~1 Xi has a negative binomial distribution. 4.

Let X have the binomial distribution with parameters n and P, and show that

lE

(_1_) = 11+ X

Find the limit of this expression as n --+ np --+ A where 0 < A < 00. Comment.

00

(1- p)n+l + l)p

(n

and p --+ 0, the limit being taken in such a way that

5. A coin is tossed repeatedly, and heads turns up with probability p on each toss. Let h n be the probability of an even number of heads in the first n tosses, with the convention that 0 is an even number. Find a difference equation for the h n and deduce that they have generating function (1 + 2ps - s)-l + (1 - s)-l }.

H

49

[5.2.6]-[5.3.6]

Exercises

Generating functions and their applications

6. An unfair coin is flipped repeatedly, where lP'(H) = P = 1 - q. Let X be the number of flips until HTH first appears, and Y the number of flips until either HTH or THT appears. Show that E(sX) = (p2 qs 3)/(l - s + pqs2 - pq2 s 3) and find E(sY). 7. Matching again. The pile of (by now dog-eared) letters is dropped again and enveloped at random, yielding Xn matches. Show that lP'(Xn = j) = (j + 1)lP'(Xn+l = j + 1). Deduce that the derivatives of the G n (s) = E(sXn) satisfy G~+l = G n , and hence derive the conclusion of Example (3.4.3), namely: 1( 1 1 (_1)n-r ) lP'(Xn = r) = - - - - + ... + . r! 2! 3! (n-r)! 8. Let X have a Poisson distribution with parameter A, where A is exponential with parameter /1-. Show that X has a geometric distribution. 9. Coupons. Recall from Exercise (3.3.2) that each packet of an overpriced commodity contains a worthless plastic object. There are four types of object, and each packet is equally likely to contain any of the four. Let T be the number of packets you open until you first have the complete set. Find E(sT) and lP'(T = k).

5.3 Exercises. Random walk 1. For a simple random walk S with So = 0 and p = 1 - q < M = max{Sn : n ~ O} satisfies lP'(M ~ r) = (p/ql for r ~ O.

i, show that the maximum

2. Use generating functions to show that, for a symmetric random walk, (a) 2kfo(2k) = lP'(S2k-2 = 0) for k ~ 1, and (b) lP'(SlS2'" S2n i= 0) = lP'(S2n = 0) for n ~ 1. 3. A particle performs a random walk on the corners of the square ABeD. At each step, the probability of moving from corner c to corner d equals Pcd, where PAD = PDA = PBC = PCB = {3, = PBA = PCD = Poc = a, 0, a + {3 = 1. Let GA(s) be the generating function of the sequence (PAA(n) PAB

and a, {3 > : n ~ 0), where p AA (n) is the probability that the particle is at A after n steps, having started at A. Show that GA(S)

=~

L~

+ 1 -1{3 ~ al 2s2 }.

s2

Hence find the probability generating function of the time of the first return to A. 4. A particle performs a symmetric random walk in two dimensions starting at the origin: each step is of unit length and has equal probability of being northwards, southwards, eastwards, or westwards. The particle first reaches the line x + y = m at the point (X, Y) and at the time T. Find the probability generating functions of T and X - Y, and state where they converge.

i

5. Derive the arc sine law for sojourn times, Theorem (3.10.21), using generating functions. That is to say, let L2n be the length of time spent (up to time 2n) by a simple symmetric random walk to the right of its starting point. Show that lP'(L2n

6. Sn

= 2k) = lP'(S2k = 0)lP'(S2n-2k = 0)

for O::oS k ::oS n.

Let {Sn : n ~ O} be a simple symmetric random walk with So = 0, and let T = min{n > 0 : Show that

= OJ.

E(min{T, 2m})

= 2EIS2ml = 4mlP'(S2m = 0) 50

for m

~

O.

Exercises

Branching processes

[5.3.7]-[5.4.6]

7. Let Sn = ~~=o Xr be a left-continuous random walk on the integers with a retaining barrier at zero. More specifically, we assume that the Xr are identically distributed integer-valued random variables with XI ::: -1, JP>(XI = 0) =F 0, and if Sn > 0, if Sn

= o.

Show that the distribution of So may be chosen in such a way that lE(zSn) = lE(zSO) for all n, if and only if lE(X I) < 0, and in this case

lE(z~n) 8.

=

(l - z)lE(X I )lE(zx 1) 1 -lE(zXj)

Consider a simple random walk starting at 0 in which each step is to the right with probability be the number of steps until the walk first reaches b where b > O. Show that

p (= 1 - q). Let lE(n I < 00) =

n

n

blip - ql· 5.4 Exercises. Branching processes

1. Let Zn be the size of the nth generation in an ordinary branching process with Zo = 1, lE(ZI) = fJ-, and var(ZI) > O. Show that lE(ZnZm) = fJ-n-mlE(Z~) for m :::: n. Hence find the correlation coefficient p(Zm, Zn) in terms of fJ-. 2. Consider a branching process with generation sizes Zn satisfying Zo = 1 and JP>(ZI = 0) = o. Pick two individuals at random (with replacement) from the nth generation and let L be the index of the generation which contains their most recent common ancestor. Show that JP>(L = r) = lEe Z;:-I) lE(Z;:-~ I) for 0 :::: r < n. What can be said if JP>(Z I = 0) > O?

3.

Consider a branching process whose family sizes have the geometric mass function I(k) = qpk, + q = 1, and let Zn be the size of the nth generation. Let T = min{n : Zn = O} be the extinction time, and suppose that Zo = 1. Find JP>(T = n). For what values of p is it the case that lE(T) < oo?

k ::: 0, where p

4. Let Zn be the size of the nth generation of a branching process, and assume Zo = 1. Find an expression for the generating function G n of Zn, in the cases when ZI has generating function given by: (a) G(s)

= 1-

a(l - s)fJ, 0 < a, fJ < 1.

(b) G(s) = I-I {P(f(s))}, where P is a probability generating function, and I is a suitablefunction satisfying I (l) = 1. (c) Suppose in the latter case that I (x) the answer explicitly.

= xm and P (s) = s {y -

(y - l)s }-1 where y > 1. Calculate

5. Branching with immigration. Each generation of a branching process (with a single progenitor) is augmented by a random number of immigrants who are indistinguishable from the other members of the population. Suppose that the numbers of immigrants in different generations are independent of each other and of the past history of the branching process, each such number having probability generating function H(s). Show that the probability generating function G n of the size of the nth generation satisfies Gn+1 (s) = Gn(G(s))H(s), where G is the probability generating function of a typical family of offspring. 6. Let Zn be the size of the nth generation in a branching process with lE(sZj) = (2 - s)-I and Zo = 1. Let Vr be the total number of generations of size r. Show that lE(VI) = 7f 2 , and

i

lE (2 V2 - V3 )

I 2 = (;7f -

I

4

907f .

51

[5.5.1]-[5.6.5]

Exercises

Generating functions and their applications

5.5 Exercises. Age-dependent branching processes 1. Let Zn be the size of the nth generation in an age-dependent branching process Z(t), the lifetime distribution of which is exponential with parameter J.... If Z (0) = 1, show that the probability generating function G t (s) of Z (t) satisfies

a

-Gt(s) = J...{ G(Gt(s» - Gt(s)}.

at

Show in the case of 'exponential binary fission', when G(s) = s2, that

G t (s)

=

se- M -1---s-(-1-_-e---;-Mc-)

and hence derive the probability mass function of the population size Z(t) at time t. 2.

Solve the differential equation of Exercise (1) when J... = 1 and G(s) =

!(l + s2), to obtain

2s+t(l-s) Gt(s) = 2+t(l-s)'

Hence find lP'(Z(t) ::: k), and deduce that

I

lP'(Z(t)/t ::: x Z(t) >

0)

---+ e- 2x

as t ---+

00.

5.6 Exercises. Expectation revisited 1. Jensen's inequality. A function u : R ---+ R is called convex if for all real a there exists J..., depending ona, such that u(x) ::: u(a)+J...(x-a) forallx. (Draw a diagram to illustrate this definition.) Show that, if u is convex and X is a random variable with finite mean, then lE(u(X» ::: u(lE(X». 2.

Let Xl, X2, ... be random variables satisfying lE(L~l IXi I) <

00.

Show that

3. Let (X n ) be a sequence of random variables satisfying Xn ::: Y a.s. for some Y with lElYI < Show that

00.

lE (lim sup Xn) ::: limsuplE(Xn). n--+oo

n--+oo

4. Suppose that lElX r I < 00 where r > O. Deduce that xrlP'(IXI ::: x) ---+ 0 as x ---+ 00. Conversely, suppose thatxrlP'(IXI::: x) ---+ Oasx ---+ 00 where r::: 0, and showthatlElXsl < 00 forO::: s < r.

5. Show that lElXI < 00 if and only if the following holds: for all that lE(IXIIA) < E for all A such that lP'(A) < o.

52

E

> 0, there exists 0 > 0, such

Exercises

Characteristic functions

[5.7.1]-[5.7.9]

5.7 Exercises. Characteristic functions 1.

Find two dependent random variables X and Y such that cf>x+y(t)

= cf>x(t)cf>y(t) for all t.

2.

If cf> is a characteristic function, show that Re{l - cf>(t)} :::: iRe{l - cf>(2t)}, and deduce that 1-1cf>(2t)1 ~ 8{1-Icf>(t)I}·

3.

The cumulant generating function Kx(e) of the random variable X is defined by Kx(e) = log JE(e Ox), the logarithm of the moment generating function of X. If the latter is finite in a neighbourhood of the origin, then K x has a convergent Taylor expansion:

and kn (X) is called the nth cumulant (or semi-invariant) of X. (a) Express kl (X), k2 (X), and k3 (X) in terms of the moments of X. (b) If X and Y are independent random variables, show that kn (X + Y)

4.

Let X be N(O, 1), and show that the cumulants of X are k2(X)

=

= kn (X) +

1, km(X)

kn (Y).

= 0 for m "# 2.

5. The random variable X is said to have a lattice distribution if there exist a and b such that X takes values in the set L (a, b) = {a + bm : m = 0, ± 1, ... }. The span of such a variable X is the maximal value of b for which there exists a such that X takes values in L(a, b). (a) Suppose that X has a lattice distribution with span b. Show that lcf>x(2nlb) I = 1, and that Icf>x(t) I < 1 forO < t < 2nlb. (b) Suppose that Icf>x (e) I = 1 for some e "# O. Show that X has a lattice distribution with span 2nkle for some integer k.

6.

Let X be a random variable with density function

f.

Show that

Icf>x (t) I ~ 0 as t

7. LetXb X2, ... , Xn be independent variables, Xi being N(J.Li, 1), and let Y Show that the characteristic function of Y is cf>y(t)

=

=

~

±oo.

Xr+Xi+ . +X~.

1 ( ite ) (l _ 2it)n/2 exp 1 - 2it

where e = J.Lr + J.L~ + ... + J.L~. The random variables Y is said to have the non-central chi-squared distribution with n degrees of freedom and non-centrality parameter e, written x2(n; e). Let X be N(J.L, 1) and let Y be x2(n), and suppose that X and Y are independent. The random variable T = XI,JY In is said to have the non-central t-distribution with n degrees of freedom and non-centrality parameter J.L. If U and V are independent, U being x2(m; e) and V being x2 (n), then F = (U Im)/(V In) is said to have the non-central F -distribution with m and n degrees of freedom and non-centrality parameter e, written F(m, n; e).

8.

(a) Show that T2 is F(l, n; J.L2). (b) Show that

JE(F)

=

n(m + e) men - 2)

if n > 2.

9. Let X be a random variable with density function f and characteristic function cf>. Show, subject to an appropriate condition on f, that

1

00

-00

f(x)2 dx = - 1 2n

53

1

00

-00

Icf>(t) 12 dt.

[5.7.10]-[5.8.11]

Generating junctions and their applications

Exercises

10. If X and Y are continuous random variables, show that

i:

if>x(y)fy(y)e-

ity

dy =

i:

if>y(x - t)fx(x)dx.

11. Tilted distributions. (a) Let X have distribution function F and let T be such that M(T) E(e rX ) < 00. Show that Fr(x) = M(T)-l J~oo e ry dF(y) is a distribution function, called a 'tilted distribution' of X, and find its moment generating function. (b) Suppose X and Y are independent and E(e rX ), E(e rY ) < 00. Find the moment generating function

of the tilted distribution of X

+ Y in tenns of those of X and Y.

5.8 Exercises. Examples of characteristic functions 1. If if> is a characteristic function, show that (j), if>2, 1if>12, Re(if» are characteristic functions. Show that 1if>1 is not necessarily a characteristic function. 2.

Show that lP'(X 2: x)::: inf{e-txMx(t)}, t=::O

where Mx is the moment generating function of X. 3. Let X have the rCA, m) distribution and let Y be independent of X with the beta distribution with parameters nand m - n, where m and n are non-negative integers satisfying n ::: m. Show that Z = XY has the rCA, n) distribution.

4.

Find the characteristic function of X2 when X has the N(p"

0- 2 )

distribution.

5. Let Xl, X2, ... be independent N(O, 1) variables. Use characteristic functions to find the distribution of: (a) X?, (b) I:?=l Xl, (c) XliX2, (d) XIX2, (e) XIX2 + X3X4. 6. Let Xl, X2, ... , Xn be such that, for all aI, a2, ... , an E ~, the linear combination al Xl + a2X2 + ... + anXn has a nonnal distribution. Show that the joint characteristic function of the Xm is exp(itJL' - ~tVt'),foranappropriate vector JL and matrix V. Deduce that the vector (X 1, X2, ... , Xn) has a multivariate nonnal density junction so long as V is invertible. 7. Let X and Y be independent N (0, 1) variables, and let U and V be independent of X and Y. Show that Z = (U X + V Y) I U2 + V2 has the N (0, 1) distribution. Fonnulate an extension of this result to cover the case when X and Y have a bivariate nonnal distribution with zero means, unit variances, and correlation p.

J

8. Let X be exponentially distributed with parameter A. Show by elementary integration that E(e itx ) = A/(A - it). 9.

Find the characteristic functions of the following density functions:

(a) f(x)

=

~e-ixi for x

E

~,

(b) f(x) = ~Ixie-ixi for x E~. 10. Is it possible for X, Y, and Z to have the same distribution and satisfy X = U (Y + Z), where U is unifonn on [0, 1], and Y, Z are independent of U and of one another? (This question arises in modelling energy redistribution among physical particles.) 11. Find the joint characteristic function of two random variables having a bivariate nonnal distribution with zero means. (No integration is needed.)

54

Inversion and continuity theorems

Exercises

[5.9.1]-[5.9.8]

5.9 Exercises. Inversion and continuity theorems 1. Let Xn be a discrete random variable taking values in {l, 2, ... , n}, each possible value having probability n-l. Show that, as n --+ 00, lP'(n- 1 Xn ~ y) --+ y, for 0 ~ y ~ 1. 2.

Let Xn have distribution function sin(2mrx) , 2mr

Fn(x)=x-

O~x~1.

(a) Show that Fn is indeed a distribution function, and that Xn has a density function. (b) Show that, as n --+ 00, Fn converges to the uniform distribution function, but that the density function of Fn does not converge to the uniform density function.

3.

A coin is tossed repeatedly, with heads turning up with probability p on each toss. Let N be the minimum number of tosses required to obtain k heads. Show that, as p 0, the distribution function of 2Np converges to that of a gamma distribution. 4.

+

If X is an integer-valued random variable with characteristic function lP'(X

= k) = ~ 27r

1 Jr

4>, show that

e- itk 4>(t)dt.

-Jr

What is the corresponding result for a random variable whose distribution is arithmetic with span A (that is, there is probability one that X is a multiple of A, and A is the largest positive number with this property)? 5.

Use the inversion theorem to show that

1

sin(at) sin(bt) d

00

t

-00

2

t

. {

= 7r mm a, b}.

Stirling's formula. Let fn (X) be a differentiable function on ~ with a a global maximum at exp{fn (x)} dx < 00. Laplace's method of steepest descent (related to Watson's lemma and saddlepoint methods) asserts under mild conditions that 6.

a > 0, and such that

10

fef

00

exp{fn (x)} dx

~ 10

00

exp{Jn (a)

+ ! (x

- a)2

f~/ (a)} dx

as n --+

00.

By setting fn(x) = nlogx - x, prove Stirling's formula: n! ~ nne -n.J27rn. 7. Let X = (Xl, X2,"" Xn) have the multivariate normal distribution with zero means, and covariance matrix V = (Vij) satisfying IVI > 0 and Vij > 0 for all i, j. Show that

af

=

aVij

a2 f aXiaXj

if i -=j:. j,

1 a2 f { -2 ax2

ifi=j,

1

and deduce that lP'(maxk::::n Xk ~ u) 2: I1Z=IlP'(Xk ~ u). 8.

Let Xl, X 2 have a bivariate normal distribution with zero means, unit variances, and correlation

p. Use the inversion theorem to show that

a ap

-lP'(XI > 0, X2 > 0)

Hence find lP'(XI > 0, X2 > 0).

55

=

1

~.

27ryl-p2

[5.10.1]-[5.10.9]

Exercises

Generating functions and their applications

5.10 Exercises. Two limit theorems 1.

Prove that, for x

~

0, as n ---+

00,

(a)

k

L -nk!

(b)

~e

n

JX -x

k:

2

I -e _lu 2 du

./2ir

.

Ik-nl:::x JI!

2. It is well known that infants born to mothers who smoke tend to be small and prone to a range of ailments. It is conjectured that also they look abnormal. Nurses were shown selections of photographs of babies, one half of whom had smokers as mothers; the nurses were asked to judge from a baby's appearance whether or not the mother smoked. In 1500 trials the correct answer was given 910 times. Is the conjecture plausible? If so, why?

3. Let X have the r (I, s) distribution; given that X = x, let Y have the Poisson distribution with parameter x. Find the characteristic function of Y, and show that Y - E(y) .p.. N(O, I) -vlvar(Y) Explain the connection with the central limit theorem.

as s ---+

00.

4. Let X I, X2, ... be independent random variables taking values in the positive integers, whose common distribution is non-arithmetic, in that gcd{n : lP'(X I = n) > O} = 1. Prove that, for all integers x, there exist non-negative integers r = r(x), s = s(x), such that lP'(XI

+ ... + Xr

- Xr+1 - ... - X r+s

= x)

> 0.

5. Prove the local central limit theorem for sums of random variables taking integer values. You may assume for simplicity that the summands have span I, in that gcd{lxl : lP'(X = x) > O} = 1. 6. Let XI, X2,'" be independent random variables having common density function f(x) = 1/{2Ixl(log IxJ)2} for Ixl < e- I . Show that the Xi have zero mean and finite variance, and that the density function fn of X I + X2 + ... + Xn satisfies fn (x) ---+ 00 as x ---+ 0. Deduce that the Xi do not satisfy the local limit theorem. 7.

First-passage density. Let X have the density function f(x) = -vl2]!' x- 3 exp( -{2x }-I), x > 0.

Show that ¢(is) = E(e- Sx ) = e- 5s , s > 0, and deduce that X has characteristic function exp{ -(1 - i),Jt} ¢ (t) = { exp{ -(1 + i).JltT}

if t ~ 0, if t ::: 0.

[Hint: Use the result of Problem (5.12.18).] 8. Let {X r : r ~ I} be independent with the distribution of the preceding Exercise (7). Let Un = n- I ~~=I X r , and Tn = n-IUn . Show that: (a) lP'(Un < c) ---+ for any c < 00, (b) Tn has the same distribution as X I.

°

9. A sequence of biased coins is flipped; the chance that the rth coin shows a head is E>r, where E>r is a random variable taking values in (0, I). Let Xn be the number of heads after n flips. Does Xn obey the central limit theorem when: (a) the E>r are independent and identically distributed? (b) E>r = E> for all r, where E> is a random variable taking values in (0, I)?

56

Exercises

Problems

[5.11.1]-[5.12.4]

5.11 Exercises. Large deviations 1. A fair coin is tossed n times, showing heads Hn times and tails Tn times. Let Sn = Hn - Tn. Show that JP'(Sn > an)l/n ~

"

V(l + a)1+a(l -

if 0 < a < 1. a)l-a

What happens if a ::: I? 2.

Show that

as n

~ 00,

where 0 < a < I and

Tn

~ (~).

=

Ik-~nl>!an

Find the asymptotic behaviour of

Tn =

T'; /n where

L k:

nk

0'

where a> O.

.

k>n(1+a)

3. Show that the moment generating function of X is finite in a neighbourhood of the origin if and only if X has exponentially decaying tails, in the sense that there exist positive constants).. and jJ, such that JP'([X[ ::: a) :::: jJ,e-J..a for a > O. [Seen in the light of this observation, the condition of the large deviation theorem (5.11.4) is very natural].

4. Let X I, X2, ... be independent random variables having the Cauchy distribution, and let Sn = Xl + X2 + ... + X n . Find JP'(Sn > an).

5.12 Problems 1.

A die is thrown ten times. What is the probability that the sum of the scores is 277

2. A coin is tossed repeatedly, heads appearing with probability p on each toss. (a) Let X be the number of tosses until the first occasion by which three heads have appeared successively. Write down a difference equation for f(k) = JP'(X = k) and solve it. Now write down an equation for JE(X) using conditional expectation. (Try the same thing for the first occurrence of HTH). (b) Let N be the number of heads in n tosses of the coin. Write down GN(S), Hence find the probability that: (i) N is divisible by 2, (ii) N is divisible by 3.

A coin is tossed repeatedly, heads occurring on each toss with probability p. Find the probability generating function of the number T of tosses before a run of n heads has appeared for the first time.

3.

4.

Find the generating function of the negative binomial mass function f(k)

=

(k -l)pr(l_ p)k-r, r - I

57

k

= r, r + 1, ... ,

[5.12.5]-[5.12.13]

Exercises

Generating functions and their applications

where 0 < p < I and r is a positive integer. Deduce the mean and variance. 5. For the simple random walk, show that the probability Po (2n) that the particle returns to the origin at the (2n )th step satisfies Po (2n) ~ (4 pq)n / .,firij, and use this to prove that the walk is persistent if 1 and only if p = You will need Stirling's formula: n! ~ nn+Ze- n £.

1.

6. A symmetric random walk in two dimensions is defined to be a sequence of points {(X n , Yn ) : n ::: O} which evolves in the following way: if (X n , Yn ) = (x, y) then (Xn+l, Yn+l) is one of the four points (x ± I, y), (x, y ± I), each being picked with equal probability ~. If (Xo, Yo) = (0,0): (a) show that E(X~ + Y;) = n, (b) find the probability po(2n) that the particle is at the origin after the (2n)th step, and deduce that the probability of ever returning to the origin is I. 7.

Consider the one-dimensional random walk {Sn} given by

Sn+l

={

Sn + 2 Sn - I

with probability p, with probability q = I - p,

where 0 < p < I. What is the probability of ever reaching the origin starting from So = a where a> O? 8.

Let X and Y be independent variables taking values in the positive integers such that

for some p and all 0 ::::: k ::::: n. Show that X and Y have Poisson distributions. 9. In a branching process whose family sizes have mean fL and variance a 2 , find the variance of Zn, the size of the nth generation, given that Zo = 1. 10. Waldegrave's problem. A group {Al, A2, ... , Ar} of r (> 2) people play the following game. A land A2 wager on the toss of a fair coin. The loser puts £ I in the pool, the winner goes on to play A3. In the next wager, the loser puts £ I in the pool, the winner goes on to play A4, and so on. The winner of the (r - I )th wager goes on to play A l, and the cycle recommences. The first person to beat all the others in sequence takes the pool. (a) Find the probability generating function of the duration of the game. (b) Find an expression for the probability that Ak wins. (c) Find an expression for the expected size of the pool at the end of the game, given that Ak wins. (d) Find an expression for the probability that the pool is intact after the nth spin of the coin. This problem was discussed by Montmort, Bernoulli, de Moivre, Laplace, and others. 11. Show that the generating function Hn of the total number of individuals in the first n generations of a branching process satisfies Hn(s) = sG(Hn_l (s)). 12. Show that the number Zn of individuals in the nth generation of a branching process satisfies JP(Zn > N [ Zm = 0) ::::: G m (O)N for n < m. 13. (a) A hen lays N eggs where N is Poisson with parameter A. The weight of the nth egg is Wn , where Wl, W2, ... are independent identically distributed variables with common probability generating function G(s). Show that the generating function Gw of the total weight W = L~l Wi is given by Gw(s) = exp{ -A + AG(S)}. W is said to have a compound Poisson distribution. Show further that, for any positive integral value of n, Gw(s)l/n is the probability generating function of some random variable; W (or its distribution) is said to be infinitely divisible in this regard.

58

Exercises

Problems

[5.12.14]-[5.12.19]

(b) Show that if H (s) is the probability generating function of some infinitely divisible distribution on the non-negative integers then H(s) = exp(-J... + J...G(s») for some J... (> 0) and some probability generating function G(s). 14. The distribution of a random variable X is called infinitely divisible if, for all positive integers n, n there exists a sequence yl(n), yin), ... , YA ) of independent identically distributed random variables n such that X and yl(n) + yin) + ... + YA ) have the same distribution.

(a) Show that the normal, Poisson, and gamma distributions are infinitely divisible. (b) Show that the characteristic function c/J of an infinitely divisible distribution has no real zeros, in that c/J (t) of 0 for all real t.

15. Let X I, X2, ... be independent variables each taking the values 0 or 1 with probabilities 1 - P and P, where 0 < P < 1. Let N be a random variable taking values in the positive integers, independent of the Xi, and write S = XI + X2 + ... + XN. Write down the conditional generating function of N given that S = N, in terms of the probability generating function G of N. Show that N has a Poisson distribution if and only ifE(xN)p = E(x N I S = N) for all P and x. 16. If X and Y have joint probability generating function

where PI

+ P2

:::: 1,

find the marginal mass functions of X and Y, and the mass function of X + Y. Find also the conditional probability generating function G XIY(s I y) = E(sX I Y = y) of X given that Y = y. The pair X, Y is said to have the bivariate negative binomial distribution. 17. If X and Y have joint probability generating function GX,Y(s, t)

= exp{ a(s -

1)

+ fJ(t

- 1)

+ y(st -

I)}

find the marginal distributions of X, Y, and the distribution of X + Y, showing that X and Y have the Poisson distribution, but that X + Y does not unless y = O. 18. Define

for a, b > O. Show that (a) l(a,b)=a-II(l,ab), (c) l(a, b) = fie- 2ab /(2a).

(b)81/8b=-21(l,ab),

(d) If X has density function (d/.jX)e- c / x - gx for x> 0, then E(e-tX)=dV

7f

g

+t

exp(-2vc (g+t)),

t>-g.

1

(e) If X has density function (27fx 3 )-Z e- I /(2x) for x > 0, then X has moment generating function given by E(e- tX ) = exp{ -.J2t}, t ::: O. [Note that E(X n ) = 00 for n ::: 1.] 19. Let X, Y, Z be independent N(O, 1) variables. Use characteristic functions and moment generating functions (Laplace transforms) to find the distributions of (a) U = X/Y, (b) V = X- 2 , (c) W

= XYZ/VX2y2 + Y2Z2 + Z2X2. 59

[5.12.20]-[5.12.30]

Exercises

20. Let X have density function Deduce that

Generating functions and their applications

f

andcharacteristicfunction ¢, and suppose that I~ 1¢(t)1 dt < 00. f(x)

= -1

2n

1

00 . e-ztx¢(t)dt.

-00

21. Conditioned branching process. Consider a branching process whose family sizes have the geometric mass function f (k) = qpk, k :::: 0, where /L = pi q > 1. Let Zn be the size of the nth generation, and assume Zo = 1. Show that the conditional distribution of Znl /L n , given that Zn > 0, converges as n --+ 00 to the exponential distribution with parameter 1 - /L -I. 22. A random variable X is called symmetric if X and - X are identically distributed. Show that X is symmetric if and only if the imaginary part of its characteristic function is identically zero. 23. Let X and Y be independent identically distributed variables with means 0 and variances 1. Let ¢ (t) be their common characteristic function, and suppose that X + Y and X - Y are independent. Show that ¢(2t) = ¢(t)3¢( -t), and deduce that X and Y are N(O, 1) variables. More generally, suppose that X and Y are independent and identically distributed with means 0 and variances 1, and furthermore that E(X - Y I X + Y) = 0 and var(X - Y I X + Y) = 2. Deduce that ¢(s)2 = ¢'(s)2 - ¢(s)¢"(s), and hence that X and Y are independent N(O, 1) variables.

24. Show that the average Z = n- I ~7=1 Xi of n independent Cauchy variables has the Cauchy distribution too. Why does this not violate the law of large numbers? 25. Let X and Y be independent random variables each having the Cauchy density function f (x) = (n(1 + x2)}-I, and let Z = i(X + Y). (a) Show by using characteristic functions that Z has the Cauchy distribution also. (b) Show by the convolution formula that Z has the Cauchy density function. You may find it helpful to check first that f(x)f(y - x)

where g(y)

=

f(x) + f(y - x) n(4 + y2)

+ g(y){xf(x) + (y -

x)f(y - x)}

= 2/{ny(4 + y2)}.

26. Let X I, X2, ... , Xn be independent variables with characteristic functions ¢I, ¢2, ... , ¢n. Describe random variables which have the following characteristic functions: (b) I¢I (t)12, (a) ¢I (t)¢2(t)··· ¢n(t), (c) ~'i Pi¢i(t) where Pi :::: 0 and ~'i Pi = 1, (d) (2 - ¢I (t»-I, 00 (e) ¢I (ut)e- U duo

10

27. Find the characteristic functions corresponding to the following density functions on (-00,00): (a) II cosh(nx), (b) (1- cosx)/(nx 2 ), (c) exp(-x - e- X ), (d) ie- ixi . Show that the mean of the 'extreme-value distribution' in part (c) is Euler's constant y.

28. Which of the following are characteristic functions: (a) ¢(t) = 1 - It I if It I ~ 1, ¢(t) = 0 otherwise, (b) ¢(t) = (1 + t 4 )-I, (c) ¢(t) = exp(-t 4 ), (d) ¢(t) = cost, (e) ¢(t) = 2(1- cost)lt 2 .

29. Show that the characteristic function ¢ of a random variable X satisfies 11 - ¢(t)1

~

EltXI.

30. Suppose X and Y have joint characteristic function ¢ (s, t). Show that, subject to the appropriate conditions of differentiability,

60

Problems

Exercises

[5.12.31]-[5.12.39]

for any positive integers m and n. 31. If X has distribution function F and characteristic function 0

(a) (b)

32. Let Xl, X2, ... be independent variables which are uniformly distributed on [0, 1]. Let Mn

=

max{X 1, X2, ... , Xn} and show that n(l - Mn) ~ X where X is exponentially distributed with parameter 1. You need not use characteristic functions. 33. If X is either (a) Poisson with parameter A, or (b) f'(l, A), show that the distribution of Y). (X -lEX)/v'var X approaches the N(O, 1) distribution as A -+ 00. (c) Show that

e -n

(

1

+ n + -n2 + ... + -nn) n!

2!

1 -+2

as n -+

00.

34. Coupon collecting. Recall that you regularly buy quantities of some ineffably dull commodity. To attract your attention, the manufacturers add to each packet a small object which is also dull, and in addition useless, but there are n different types. Assume that each packet is equally likely to contain anyone of the different types, as usual. Let Tn be the number of packets bought before you acquire a complete set of n objects. Show that n- 1(Tn - n log n) ~ T, where T is a random variable with distribution function JP'(T :'S x)

= exp( _e- X ),

-00

< x <

00.

35. Find a sequence (
=

36. Use generating functions to show that it is not possible to load two dice in such a way that the sum of the values which they show is equally likely to take any value between 2 and 12. Compare with your method for Problem (2.7.12). 37. A biased coin is tossed N times, where N is a random variable which is Poisson distributed with parameter A. Prove that the total number of heads shown is independent of the total number of tails. Show conversely that if the numbers of heads and tails are independent, then N has the Poisson distribution. 38. A binary tree is a tree (as in the section on branching processes) in which each node has exactly two descendants. Suppose that each node of the tree is coloured black with probability p, and white otherwise, independently of all other nodes. For any path rr containing n nodes beginning at the root of the tree, let B (rr) be the number of black nodes in rr, and let X n (k) be the number of such paths rr for which B(rr) ::: k. Show that there exists fJe such that lE{X n (fJn)} -+ {

0 00

if fJ > fJe, if fJ < fJe,

and show how to determine the value fJe. Prove that

0 JP' ( Xn(fJn) ::: 1) -+ { 1

if fJ > fJe, if fJ < fJe.

39. Use the continuity theorem (5.9.5) to show that, as n -+

00,

(a) if Xn is bin(n, A/n) then the distribution of Xn converges to a Poisson distribution,

61

[5.12.40]-[5.12.46]

Exercises

Generating functions and their applications

(b) if Yn is geometric with parameter p nential distribution.

=

).../n then the distribution of Yn/n converges to an expo-

40. Let X I, X2, ... be independent random variables with zero means and such that E IX] I < all j. Show that Sn = X I

00

for

+ X2 + ... + Xn satisfies Sn / y'var(Sn) -S N (0, 1) as n --+ 00 if

The following steps may be useful. Let a] = var(Xj), a(n)2 = var(Sn), Pj = EIX]I, and
+ z) -

zl ::::: Izl2 if Izl :::::

i, where the logarithm has its principal value.

(iii) Show that aJ ::::: Pj, and deduce from the hypothesis that maxl:oj:on aj /a(n) --+ 0 as n --+ implying that maxl:oj:on I
00,

(iv) Deduce an upper bound for Ilog
41. Let X I, X2, ... be independent variables each taking values Show that

i.

+1 or -1 with probabilities

i and

n

{!;3"n L

kXk

-S N(O, 1)

as n --+

00.

k=1

42. Normal sample. Let X I, X2, ... , Xn be independent N(p." ( 2) random variables. Define X = n- I 'L,1 Xi and Zi = Xi - X. Find the joint characteristic function of X, ZI, Z2, ... , Zn, and hence prove that X and S2 = (n - 1)-1 'L,1(Xi - X)2 are independent. 43. Log-normal distribution. Let X be N (0, 1), and let Y distribution. Show that the density function of Y is f(x)

=

= eX;

1 J;C exp{ -i(logx)2}, xv2rr

Y is said to have the log-normal

x> O.

For la I ::::: 1, define fa (x) = {I + a sin (2rr log x) } f (x). Show that fa is a density function with finite moments of all (positive) orders, none of which depends on the value of a. The family {fa : lal ::::: I} contains density functions which are not specified by their moments. 44. Consider a random walk whose steps are independent and identically distributed integer-valued random variables with non-zero mean. Prove that the walk is transient. 45. Recurrent events. Let (Xr : r ?:. I} be the integer-valued identically distributed intervals between the times of a recurrent event process. Let L be the earliest time by which there has been an interval of length a containing no occurrence time. Show that, for integral a,

46. A biased coin shows heads with probability p (= 1 - q). It is flipped repeatedly until the first time Wn by which it has shown n consecutive heads. Let E(sWn) = Gn(s). Show that G n =

62

Exercises

Problems

[5.12.47]-[5.12.52]

psGn-J!(l - qsGn-I), and deduce that

47. In n flips of a biased coin which shows heads with probability p (= 1 - q), let Ln be the length of the longest run of heads. Show that, for r :::: 1,

48. The random process (X n : n :::: 1) decays geometrically fast in that, in the absence of external input, Xn+1 = iXn. However, at any time n the process is also increased by Yn with probability where (Yn : n :::: 1) is a sequence of independent exponential random variables with parameter A. Find the limiting distribution of Xn as n -+ 00.

i,

Jd

49. Let G(s) = E(sx) where X :::: O. Show that E{(X + 1)-1) = G(s)ds, and evaluate this when X is (a) Poisson with parameter A, (b) geometric with parameter p, (c) binomial bin(n, p), (d) logarithmic with parameter p (see Exercise (5.2.3». Is there a non-trivial choice for the distribution of X such that E{(X + 1)-1) = {E(X + 1))-I? 50. Find the density function of r;~=1 X r , where (Xr : r :::: 1) are independent and exponentially distributed with parameter A, and tv. is geometric with parameter p and independent of the Xr. 51. Let X have finite non-zero variance and characteristic function ¢J(t). Show that

is a characteristic function, and find the corresponding distribution. 52. Let X and Y have joint density function

Ixl Show that ¢Jx(t)¢Jy (t)

= ¢Jx+Y (t), and that X and Y

63

< 1,

are dependent.

Iyl

< 1.

6 Markov chains

6.1 Exercises. Markov processes 1. Show that any sequence of independent random variables taking values in the countable set S is a Markov chain. Under what condition is this chain homogeneous? 2. A die is rolled repeatedly. Which of the following are Markov chains? For those that are, supply the transition matrix. (a) The largest number Xn shown up to the nth roll. (b) The number N n of sixes in n rolls. (c) At time r, the time C r since the most recent six. (d) At time r, the time Br until the next six. 3. Let {Sn : n :::: 0) be a simple random walk with So = 0, and show that Xn = ISnl defines a Markov chain; find the transition probabilities of this chain. Let Mn = max{Sk : 0 :s k :s n], and show that Yn = Mn - Sn defines a Markov chain. What happens if So =f. O? 4. Let X be a Markov chain and let (nr : r :::: 0) be an unbounded increasing sequence of positive integers. Show that Yr = Xnr constitutes a (possibly inhomogeneous) Markov chain. Find the transition matrix of Y when nr = 2r and X is: (a) simple random walk, and (b) a branching process. 5. Let X be a Markov chain on S, and let I : Sn --+ {O, 1). Show that the distribution of X n , X n+!, ... , conditional on {l (X!, ... , Xn) = 1) n {Xn = i], is identical to the distribution of X n , X n+!, ... conditional on {X n = i). 6. Strong Markov property. Let X be a Markov chain on S, and let T be a random variable taking values in (O, 1,2, ... ) with the property that the indicator function I{T=nj, of the event that T = n, is a function of the variables X!, X2, ... , X n . Such a random variable T is called a stopping time, and the above definition requires that it is decidable whether or not T = n with a knowledge only of the past and present, Xo, X!, ... , X n , and with no further information about the future. Show that JP'(XT+m

for m :::: 0, i, j

E

=

j

IXk = Xk for 0 :s k <

T, XT

= i) = JP'(XT+m =

j I XT

= i)

S, and all sequences (Xk) of states.

7. Let X be a Markov chain with state space S, and suppose that h : S --+ Tis one-one. Show that Yn = h(Xn) defines a Markov chain on T. Must this be so if h is not one-one? 8. Let X and Y be Markov chains on the set;Z of integers. Is the sequence Zn a Markov chain? 9.

Let X be a Markov chain. Which of the following are Markov chains?

64

=

Xn

+ Yn necessarily

Exercises

Classification of states

[6.1.10]-[6.2.5]

(a) X m +r for r ::: O. (b) X2m for m ::: O. (c) The sequence of pairs (X n , Xn+l) forn::: O. 10. Let X be a Markov chain. Show that, for 1 < r < n, lP'(X r = k [ Xi = xi for i = 1,2, ... , r - 1, r

+ 1 ... , n) = lP'(Xr =

k [ Xr-l = Xr-l, Xr+l = Xr+l)'

11. Let {X n : n ::: I} be independent identically distributed integer-valued random variables. Let Sn = 2:~=1 X r , with So = 0, Yn = Xn + Xn-l with Xo = 0, and Zn = 2:~=o Sr. Which of the following constitute Markov chains: (a) Sn, (b) Yn , (c) Zn, (d) the sequence of pairs (Sn, Zn)? 12. A stochastic matrix P is called doubly stochastic if2:i Pij = 1 for all j. !tis called sub-stochastic if 2:i Pij ::s 1 for all j. Show that, ifP is stochastic (respectively, doubly stochastic, sub-stochastic), then pn is stochastic (respectively, doubly stochastic, sub-stochastic) for all n.

6.2 Exercises. Classification of states 1. Last exits. Let lij (n) = lP'(X n = j, Xk =f i for 1 ::s k < n [ Xo = i), the probability that the chain passes from i to j in n steps without revisiting i. Writing 00

Lij(S) = Lsnlij(n), n=l

show that Pij(s) = Pu (s)Lij (s) if i =f j. Deduce that the first passage times and last exit times have the same distribution for any Markov chain for which Pu (s) = Pjj(s) for all i and j. Give an example of such a chain. 2. Let X be a Markov chain containing an absorbing state s with which all other states i communicate, in the sense that Pis (n) > 0 for some n = n(i). Show that all states other than s are transient. 3. Show that a state i is persistent if and only if the mean number of visits of the chain to i, having started at i, is infinite. 4. Visits. Let Vj = [{n ::: 1 : Xn = j)[ be the number of visits of the Markov chain X to j, and define TJij = lP'(Vj = 00 [ Xo = i). Show that: .. _

{I

TJII -

0

(b) TJij = {

o

(a)

5.

if i is persistent, if i is transient,

lP'(1) <

00 [

Xo

= i)

if j is persistent, . . .. , . where 1) = mm{n ::: 1 : Xn = J}. If } IS transIent,

Symmetry. The distinct pair i, j of states of a Markov chain is called symmetric if lP'(1) < Tj [ Xo = i) = lP'(Tj < 1) [ Xo = j),

where Tj = min{n ::: 1 : Xn = i}. Show that, if Xo = i and i, j is symmetric, the expected number of visits to j before the chain revisits i is 1.

65

[6.3.1]-[6.3.8]

Exercises

Markov chains

6.3 Exercises. Classification of chains Let X be a Markov chain on {O, 1,2, ... } with transition matrix given by POj = aj for j ::: 0, 1 - r for i ::: 1. Classify the states of the chain, and find their mean recurrence times.

1.

Pii

= rand Pi,i-l =

2.

Determine whether or notthe random walk on the integers having transition probabilities Pi,i+2 = = 1 - p, for all i, is persistent.

p, Pi,i-l

3.

Classify the states of the Markov chains with transition matrices

1-~2P (

(a)

P

0

(b) (

0)

2p 1 - 2p 2p

P 1 - 2p

pOo

I~P I~P

1

,

-~ P) .

I-p

In each case, calculate Pij (n) and the mean recurrence times of the states. 4. A particle performs a random walk on the vertices of a cube. At each step it remains where it is with probability ,\:, or moves to one of its neighbouring vertices each having probability ,\:. Let v and w be two diametrically opposite vertices. If the walk starts at v, find: (a) the mean number of steps until its first return to v, (b) the mean number of steps until its first visit to w, (c) the mean number of visits to w before its first return to v.

5.

Visits. With the notation of Exercise (6.2.4), show that

= IJji = 1,

(a) if i ---+ j and i is persistent, then IJij

= 1 if and only ifJP'(1j < 00 [ Xo = i) = JP'(Tj < 00 [ Xo = j) = 1. 6. First passages. Let TA = min{n ::: 0 : Xn E A}, where X is a Markov chain and A is a subset of the state space S, and let IJj = JP'(TA < 00 [ Xo = j). Show that (b) IJij

IJj

=

{

I

if j E A,

L PjkIJk

if j

f-

A.

kES

Show further that if x all j.

7.

= (Xj

: j E S) is any non-negative solution of these equations then Xj ::: IJj for

Mean first passage. In the notation of Exercise (6), let Pj

Pj

={

~+L

= E(TA

[ Xo

= j).

Show that

if j E A, PjkPk

if j

f-

A,

kES

and that if x = (Xj : j E S) is any non-negative solution of these equations then Xj ::: Pj for all j.

8. Let X be an irreducible Markov chain and let A be a subset of the state space. Let Sr and Tr be the successive times at which the chain enters A and visits A respectively. Are the sequences {XSr : r ::: l}, {XTr : r 2: I} Markov chains? What can be said about the times at which the chain exits A?

66

Exercises

Stationary distributions and the limit theorem

[6.3.9]-[6.4.6]

9. (a) Show that for each pair i, j of states of an irreducible aperiodic chain, there exists N = N (i, j) such that Pij (r) > 0 for all r 2: N. (b) Show that there exists a function f such that, ifP is the transition matrix of an irreducible aperiodic Markov chain with n states, then Pij (r) > 0 for all states i, j, and all r 2: f (n). (c) Show further that f(4) 2: 6 and fen) 2: (n - l)(n - 2). [Hint: The postage stamp lemma asserts that, for a, b coprime, the smallest n such that all integers strictly exceeding n have the form cta + f3b for some integers ct, f3 2: 0 is (a - l)(b - 1).]

10. An urn initially contains n green balls and n + 2 red balls. A ball is picked at random: if it is green then a red ball is also removed and both are discarded; if it is red then it is replaced together with an extra red and an extra green ball. This is repeated until there are no green balls in the urn. Show that the probability the process terminates is l/(n + 1). Now reverse the rules: if the ball is green, it is replaced together with an extra green and an extra red ball; if it is red it is discarded along with a green ball. Show that the expected number of iterations until no green balls remain is "L}=l (2j + 1) = n(n + 2). [Thus, a minor perturbation of a simple symmetric random walk can be non-null persistent, whereas the original is null persistent.]

6.4 Exercises. Stationary distributions and the limit theorem 1. The proof copy of a book is read by an infinite sequence of editors checking for mistakes. Each mistake is detected with probability P at each reading; between readings the printer corrects the detected mistakes but introduces a random number of new errors (errors may be introduced even if no mistakes were detected). Assuming as much independence as usual, and thatthe numbers of new errors after different readings are identically distributed, find an expression for the probability generating function of the stationary distribution of the number Xn of errors after the nth editor-printer cycle, whenever this exists. Find it explicitly when the printer introduces a Poisson-distributed number of errors at each stage. 2. Do the appropriate parts of Exercises (6.3.1)-(6.3.4) again, making use of the new techniques at your disposal. 3. Dams. Let Xn be the amount of water in a reservoir at noon on day n. During the 24 hour period beginning at this time, a quantity Yn of water flows into the reservoir, and just before noon on each day exactly one unit of water is removed (if this amount can be found). The maximum capacity of the reservoir is K, and excessive inflows are spilled and lost. Assume that the Yn are independent and identically distributed random variables and that, by rounding off to some laughably small unit of volume, all numbers in this exercise are non-negative integers. Show that (X n ) is a Markov chain, and find its transition matrix and an expression for its stationary distribution in terms of the probability generating function G of the Yn . Find the stationary distribution when Y has probability generating function G(s)

=

p(1- qs)-l.

4. Show by example that chains which are not irreducible may have many different stationary distributions. 5. Diagonal selection. Let (Xi (n) : i, n 2: 1) be a bounded collection of real numbers. Show that there exists an increasing sequence n 1, n2, ... of positive integers such that limr--+ 00 Xi (n r ) exists for all i. Use this result to prove that, for an irreducible Markov chain, if it is not the case that Pij (n) --+ 0 as n --+ 00 for all i and j, then there exists a sequence (nr : r 2: 1) and a vector ex. (~ 0) such that Pij (nr) --+ ctj as r --+ 00 for all i and j. 6. Random walk on a graph. A particle performs a random walk on the vertex set of a connected graph G, which for simplicity we assume to have neither loops nor multiple edges. At each stage it moves to a neighbour of its current position, each such neighbour being chosen with equal probability.

67

[6.4.7]-[6.5.3]

Exercises

Markov chains

If G has 17 « 00) edges, show that the stationary distribution is given by Jr v = d v /(217), where d v is the degree of vertex v.

7.

Show that a random walk on the infinite binary tree is transient.

8. At each time n = 0,1,2, ... a number Yn of particles enters a chamber, where (Yn : n 2: OJ are independent and Poisson distributed with parameter A. Lifetimes of particles are independent and geometrically distributed with parameter p. Let Xn be the number of particles in the chamber at time n. Show that X is a Markov chain, and find its stationary distribution. 9. A random sequence of convex polygons is generated by picking two edges of the current polygon at random, joining their midpoints, and picking one of the two resulting smaller polygons at random to be the next in the sequence. Let Xn + 3 be the number of edges of the nth polygon thus constructed. Find E(X n ) in terms of Xo, and find the stationary distribution of the Markov chain X.

10. Let s be a state of an irreducible Markov chain on the non-negative integers. Show that the chain is persistent if there exists a solution y to the equations Yi 2: Lj:jf-s Pij Yj, i of s, satisfying Yi ~ 00. 11. Bow ties. A particle performs a random walk on a bow tie ABCDE drawn beneath on the left, where C is the knot. From any vertex its next step is equally likely to be to any neighbouring vertex. Initially it is at A. Find the expected value of: (a) the time of first return to A, (b) the number of visits to D before returning to A, (c) the number of visits to C before returning to A, (d) the time of first return to A, given no prior visit by the particle to E, (e) the number of visits to D before returning to A, given no prior visit by the particle to E. A

D

[?<J E

B

A

B

12. A particle starts at A and executes a symmetric random walk on the graph drawn above on the right. Find the expected number of visits to B before it returns to A.

6.5 Exercises. Reversibility 1. A random walk on the set to, 1,2, ... , bJ has transition matrix given by Poo = 1 - AO, Pbb = 1 - /tb, Pi,i+! = Ai and Pi+!,i = /ti+! for 0 ::::: i < b, where 0 < Ai, /ti < 1 for all i, and Ai + /ti = 1 for 1 ::::: i < b. Show that this process is reversible in equilibrium. 2. Kolmogorov's criterion for reversibility. Let X be an irreducible non-null persistent aperiodic Markov chain. Show that X is reversible in equilibrium if and only if PhhPhh ... Pjn-ljnPjnh = PhjnPjnjn-l ... Phh

for all n and all finite sequences

h, jz, ... , jn of states.

3. Let X be a reversible Markov chain, and let C be a non-empty subset of the state space S. Define the Markov chain Y on S by the transition matrix Q = (qij) where qij = {

C and j

{3Pij

if i

Pij

otherwise,

68

E

1: C,

Exercises [6.5.4]-[6.6.3]

Chains with finitely I1Ulny states

for i =1= j, and where .B is a constant satisfying 0 < .B < 1. The diagonalterrns % are arranged so that Q is a stochastic matrix. Show that Y is reversible in equilibrium, and find its stationary distribution. Describe the situation in the limit as .B O.

+

4.

Can a reversible chain be periodic?

5. Ehrenfest dog-flea model. The dog-flea model of Example (6.5.5) is a Markov chain X on the state space {O, 1, ... , m l with transition probabilities i

i

P, i i + l = l - m -,

Show that, if Xo

for

O:s i :s m.

= i,

E(Xn 6.

Pi,i-l = ;;;'

~)

=

(i - ~) (1 - ~) n -+ 0 as n -+

00.

Which of the following (when stationary) are reversible Markov chains?

(1 ~ 1~ .B )

(a) The chain X

= {Xnl having transition matrix P =

(b) Tb"l>tin Y

~ {Y"} h"ing "~,ition =trix P ~ ( 1 ;

(c) Zn

a

p

1

where a

+.B

> O.

~ P 1 ~ P) where 0 < p < 1.

= (X n , Yn ), where Xn and Yn are independent and satisfy (a) and (b).

7.

Let X n , Yn be independent simple random walks. Let Zn be (X n , Yn ) truncated to lie in the region Xn 2: 0, Yn 2: 0, Xn + Yn :s a where a is integral. Find the stationary distribution of Zn.

8. Show that an irreducible Markov chain with a finite state space and transition matrix P is reversible in equilibrium if and only if P = DS for some symmetric matrix S and diagonal matrix D with strictly positive diagonal entries. Show further that for reversibility in equilibrium to hold, it is necessary but not sufficient that P has real eigenvalues. 9. Random walk on a graph. Let G be a finite connected graph with neither loops nor multiple edges, and let X be a random walk on G as in Exercise (6.4.6). Show that X is reversible in equilibrium.

6.6 Exercises. Chains with finitely many states The first two exercises provide proofs that a Markov chain with finitely many states has a stationary distribution.

1. The Markov-Kakutani theorem asserts that, for any convex compact subset C of]Rn and any linear continuous mapping T of C into C, T has a fixed point (in the sense that T (x) = x for some x E C). Use this to prove that a finite stochastic matrix has a non-negative non-zero left eigenvector corresponding to the eigenvalue 1. 2. Let T be a m x n matrix and let v E ]Rn. Farkas's theorem asserts that exactly one ofthe following holds: (i) there exists x E ]Rm such that x 2: 0 and xT = v, (ii) there exists Y E ]Rn such that yv' < 0 and Ty' 2: O. Use this to prove that a finite stochastic matrix has a non-negative non-zero left eigenvector corresponding to the eigenvalue 1.

3. Arbitrage. Suppose you are betting on a race with m possible outcomes. There are n bookmakers, and a unit stake with the ith bookmaker yields tij if the jth outcome of the race occurs. A vector

69

[6.6.4]-[6.7.3]

Exercises

Markov chains

x = (Xl, x2, ... , xn), where Xr E (-00, 00) is your stake with the rth bookmaker, is called a betting scheme. Show that exactly one of (a) and (b) holds: (a) there exists a probability mass function p values of i,

= (PI, P2, ...

, Pm) such that

(b) there exists a betting scheme x for which you surely win, that is, 4.

=

Let X be a Markov chain with state space S

C-

p=

0

P

2:j=I tij Pj = 0 for all

2:7=1 Xi tij

> 0 for all j.

(1, 2, 3) and transition matrix

P l-p

0

P

IU

where 0 < P < 1. Prove that pn =

where aln

+ wa2n + w2a3n = (l -

P

a2n aln a3n

( ai,

a3n a2n

a3, ) a2n aln

+ pw)n, w being a complex cube root of 1.

5. Let P be the transition matrix of a Markov chain with finite state space. Let 1 be the identity matrix, U the lSI x lSI matrix with all entries unity, and 1 the row lSI-vector with all entries unity. Leur be a non-negative vector with 2:i lri = 1. Show thaur P = 1l" if and only if 1l" (I - P + U) = 1. Deduce that if P is irreducible then 1l" = 1 (I - P + U) -1.

6. Chess. A chess piece performs a random walk on a chessboard; at each step it is equally likely to make anyone of the available moves. What is the mean recurrence time of a corner square if the piece is a: (a) king? (b) queen? (c) bishop? (d) knight? (e) rook? 7. Chess continued. A rook and a bishop perform independent symmetric random walks with synchronous steps on a 4 x 4 chessboard (16 squares). If they start together at a corner, show that the expected number of steps until they meet again at the same corner is 448/3. 8.

Find the n-step transition probabilities

Pij (n)

for the chain X having transition matrix 1

2:

1

4

1

4

f,) . 12

6.7 Exercises. Branching processes revisited 1. Let Zn be the size of the nth generation of a branching process with Zo = 1 and lP'(ZI = k) = 2- k for k :::. O. Show directly that, as n -+ 00, lP'(Zn ::::: 2yn I Zn > 0) -+ 1 - e- 2y , y > 0, in agreement with Theorem (6.7.8). 2. Let Z be a supercritical branching process with Zo = 1 and family-size generating function G. Assume that the probability 17 of extinction satisfies 0 < 17 < 1. Find a way of describing the process Z, conditioned on its ultimate extinction. 3. Let Zn be the size of the nth generation of a branching process with Zo = 1 and lP'(ZI = k) = qpk for k :::. 0, where P + q = 1 and P > Use your answer to Exercise (2) to show that, if we condition on the ultimate extinction of Z, then the process grows in the manner of a branching process with generation sizes 2n satisfying 20 = 1 and lP'(ZI = k) = pqk for k :::. O.

1.

70

Exercises [6.7.4]-[6.8.7]

Birth processes and the Poisson process

(a) Show that JE(X [ X > 0) ::::: JE(X 2 )/JE(X) for any random variable X taking non-negative values. (b) Let Zn be the size of the nth generation of a branching process with Zo = I and lP'(ZI = k) = qpk for k ::: 0, where p > Use part (a) to show that JE(Zn/ pP [ Zn > 0) ::::: 2p/(p - q), where fJ.,= p/q.

4.

i.

(c) Show that, in the notation of part (b), JE(Zn/ fJ.,n [ Zn > 0) -;. p/(p - q) as n -;.

00.

6.8 Exercises. Birth processes and the Poisson process 1. Superposition. Flies and wasps land on your dinner plate in the manner of independent Poisson processes with respective intensities A and fJ.,. Show that the arrivals of flying objects form a Poisson process with intensity A + fJ.,. 2. Thinning. Insects land in the soup in the manner of a Poisson process with intensity A, and each such insect is green with probability p, independently of the colours of all other insects. Show that the arrivals of green insects form a Poisson process with intensity Ap. 3. Let Tn be the time of the nth arrival in a Poisson process N with intensity A, and define the excess lifetime process E(t) = TN(t)+1 - t, being the time one must wait subsequent to t before the next arrival. Show by conditioning on TI that

lP'( E(t) >

x) = e-A(t+x) + fot lP'( E(t

- u) >

x )Ae- AU duo

Solve this integral equation in order to find the distribution function of E (t). Explain your conclusion. 4. Let B be a simple birth process (6.S.llb) with B(O) = I; the birth rates are An = nA. Write down the forward system of equations for the process and deduce that

k ::: I. Show also that JE(B(t))

= I eAt and var(B(t)) = I e 2At (1 -

e- At ).

5. Let B be a process of simple birth with immigration (6.S.llc) with parameters A and v, and with B(O) = 0; the birth rates are An = nA + V. Write down the sequence of differential-difference equations for Pn (t) = lP'(B(t) = n). Without solving these equations, use them to show that m (t) = JE(B(t)) satisfies m'(t) = Am(t) + v, and solve for m(t). 6. Let N be a birth process with intensities AO, AI, ... , and let N (0) lP'(N(t) = n) is given by

provided that Ai

=I- Aj whenever i =I-

=

O. Show that Pn (t) =

j.

7. Suppose that the general birth process of the previous exercise is such that L: nA;;-I < 00. Show that AnPn(t) -;. f(t) as n -;. 00 where f is the density function of the random variable T = sup{t : N(t) < oo}. Deduce that JE(N(t) [ N(t) < (0) is finite or infinite depending on the convergence or divergence of L:n nA;;-1 . Find the Laplace transform of f in closed form for the case when An expression for f.

71

=

(n

+ i)2, and deduce an

[6.9.1]-[6.9.8]

Exercises

Markov chains

6.9 Exercises. Continuous-time Markov chains 1.

Let A{t > 0 and let X be a Markov chain on {I, 2} with generator G

=(

-{t A

{t) -A .

(a) Write down the forward equations and solve them for the transition probabilities Pij (t), i, j 1,2. (b) Calculate G n and hence find L~O(tn /n!)G n . Compare your answer with that to part (a). (c) Solve the equation JrG = 0 in order to find the stationary distribution. Verify that Pij (t) ---+ as t ---+ 00. 2.

=

lrj

As a continuation of the previous exercise, find:

(a) lP'(X(t)

(b) lP'(X(t)

= 21 = 21

X(O) X(O)

= 1, X(3t) = 1), = 1, X(3t) = 1, X (4t) = 1).

3. Jobs arrive in a computer queue in the manner of a Poisson process with intensity A. The central processor handles them one by one in the order of their arrival, and each has an exponentially distributed runtime with parameter {t, the runtimes of different jobs being independent of each other and of the arrival process. Let X (t) be the number of jobs in the system (either running or waiting) at time t, where X (0) = O. Explain why X is a Markov chain, and write down its generator. Show that a stationary distribution exists if and only if A < {t, and find it in this case. 4. Pasta property. Let X = {X (t) : t :::: O} be a Markov chain having stationary distribution Jr. We may sample X at the times of a Poisson process: let N be a Poisson process with intensity A, independent of X, and define Yn = X(Tn +), the value taken by X immediately after the epoch Tn of the nth arrival of N. Show that Y = (Yn : n :::: O} is a discrete-time Markov chain with the same stationary distribution as X. (This exemplifies the 'Pasta' property: Poisson arrivals see time averages.) [The full assumption of the independence of N and X is not necessary for the conclusion. It suffices that (N (s) : s :::: t} be independent of (X (s) : s ::: t}, a property known as 'lack of anticipation'. It is not even necessary that X be Markov; the Pasta property holds for many suitable ergodic processes.] 5. Let X be a continuous-time Markov chain with generator G satisfying gi = - gii > 0 for all i. Let HA = inf(t :::: 0: X(t) E A} be the hitling time of the set A of states, and let TJj = lP'(HA < 00 X (0) = j) be the chance of ever reaching A from j. By using properties of the jump chain, which you may assume to be well behaved, show that Lk gjkTJk = 0 for j 1. A. 1

6. In continuation of the preceding exercise, let {tj is the minimal non-negative solution of the equations {tj

=0

if j

E

A,

1+

L

= lE(HA gjk{tk

1

X (0)

=0

if j

= 1.

j). Show that the vector JL

A.

kES

7. Let X be a continuous-time Markov chain with transition probabilities Pij (t) and define Fi inf(t > TJ : X(t) = i} where TJ is the time of the first jump of X. Show that, if gii =f. 0, then lP'(Fi < 00 X(O) = i) = 1 if and only if i is persistent. 1

8.

Let X be the simple symmetric random walk on the integers in continuous time, so that Pi,i+J (h)

= Pi,i-J (h) = iAh + o(h).

Show that the walk is persistent. Let T be the time spent visiting m during an excursion from O. Find the distribution of T.

72

Exercises

Birth-death processes and imbedding

[6.9.9]-[6.11.4]

9. Let i be a transient state of a continuous-time Markov chain X with X (0) = i. Show that the total time spent in state i has an exponential distribution.

10. Let X be an asymmetric simple random walk in continuous time on the non-negative integers with retention at 0, so that Ah + o(h) Pij(h)= { J-th+o(h)

if j = i

+ 1,

i 2: 0,

if j = i - 1, i 2: 1.

Suppose that X (0) = 0 and A > J-t. Show that the total time Vr spent in state r is exponentially distributed with parameter A - J-t. Assume now that X (0) has some general distribution with probability generating function G. Find the expected amount of time spent at 0 in terms of G.

11. Let X = (X (t) : t 2: OJ be a non-explosive irreducible Markov chain with generator G and unique stationary distribution 1r. The mean recurrence time J-tk is defined as follows. Suppose X (0) = k, and let U = inf(s : X(s) =f kj. Then J-tk = IE(inf(t > U : X(t) = k)). Let 2 = (2n : n 2: OJ be the imbedded 'jump chain' given by 20 = X(O) and 2n is the value of X just after its nth jump. (a) Show that 2 has stationary distribution ii satisfying

where gi = -gU, provided ~i Jrigi < 00. When is it the case that ii = 1r? (b) Show that Jri = 1/(J-ti gi) if J-ti < 00, and that the mean recurrence time 11k of the state k in the jump chain 2 satisfies 11k = J-tk ~i Jrigi if the last sum is finite.

12. Let 2 be an irreducible discrete-time Markov chain on a countably infinite state space S, having transition matrix H = (hij) satisfying hu = 0 for all states i, and with stationary distribution v. Construct a continuous-time process X on S for which Z is the imbedded chain, such that X has no stationary distribution.

6.11 Exercises. Birth-death processes and imbedding 1.

Describe the jump chain for a birth-death process with rates An and J-tn.

2. Consider an immigration-death process X, being a birth-death process with birth rates An = A and death rates J-tn = nJ-t. Find the transition matrix of the jump chain Z, and show that it has as stationary distribution

Jrn=2(~!) (l+~)pne-p where p

= AI J-t.

Explain why this differs from the stationary distribution of X.

3. Consider the birth-death process X with An = nA and J-tn = nJ-t for all n 2: O. Suppose X (0) and let 1](t) = lP'(X(t) = 0). Show that 1] satisfies the differential equation 1]' (t)

Hence find 1](t), and calculate lP'(X(t)

=

1

+ (A + J-t)1](t) = J-t + A1](t)2. = 0 I X(u) = 0) for 0

< t < u.

4. For the birth-death process of the previous exercise with A < J-t, show that the distribution of X(t), conditional on the event (X(t) > OJ, converges as t --+ 00 to a geometric distribution.

73

[6.11.5]-[6.13.2]

Exercises

Markov chains

5. Let X be a birth-death process with An = nA and J1.n the time T at which X (t) first takes the value 0 satisfies

JE(T [ T < 00)

=

=

nJ1., and suppose X(O)

~ log ( ~)

itA < J1.,

(J1. A ) { - log - J1. A-J1.

if A > J1..

~

~

=

1. Show that

What happens when A = J1.?

6. Let X be the birth-death process of Exercise (5) with A -=I J1., and let Vr (t) be the total amount of time the process has spent in state r :::: 0, up to time t. Find the distribution of VI (00) and the generating function LrsrJE(Vr(t». Hence show in two ways that JE(Vl(OO» = [max{A,J1.}]-I. Show further that JE(Vr (00» = Ar - 1r- 1 [max{A, J1.}rr. 7.

Repeat the calculations of Exercise (6) in the case A = J1..

6.12 Exercises. Special processes 1. Customers entering a shop are served in the order of their arrival by the single server. They arrive in the manner of a Poisson process with intensity A, and their service times are independent exponentially distributed random variables with parameter J1.. By considering the jump chain, show that the expected duration of a busy period B of the server is (J1. - A)-1 when A < J1.. (The busy period runs from the moment a customer arrives to find the server free until the earliest subsequent time when the server is again free.) 2. Disasters. Immigrants arrive at the instants of a Poisson process of rate v, and each independently founds a simple birth process ofrate A. At the instants of an independent Poisson process ofrate 8, the population is annihilated. Find the probability generating function of the population X (t), given that X (0) = o. 3. More disasters. In the framework of Exercise (2), suppose that each immigrant gives rise to a simple birth-death process of rates A and J1.. Show that the mean population size stays bounded if and only if 8 > A - J1.. 4. The queue MlG/oo. (See Section 11.1.) An ftp server receives clients at the times of a Poisson process with parameter A, beginning at time O. The ith client remains connected for a length Sj of time, where the Sj are independent identically distributed random variables, independent of the process of arrivals. Assuming that the server has an infinite capacity, show that the number of clients being serviced at time t has the Poisson distribution with parameter A i6[1 - G(x)] dx, where G is the common distribution function of the Sj.

6.13 Exercises. Spatial Poisson processes 1. In a certain town at time t = 0 there are no bears. Brown bears and grizzly bears arrive as independent Poisson processes Band G with respective intensities fJ and y. (a) Show that the first bear is brown with probability fJ / (fJ + y). (b) Find the probability that between two consecutive brown bears, there arrive exactly r grizzly bears. (c) Given that B(I) = 1, find the expected value of the time at which the first bear arrived. 2. Campbell-Hardy theorem. Let IT be the points of a non-homogeneous Poisson process on lRd with intensity function A. Let S = LXEI1 g(x) where g is a smooth function which we assume for

74

Exercises

Markov chain Monte CarZo

convenience to be non-negative. ShowthatE(S) provided these integrals converge.

=

In~.d g(U)A(U) du and var(S)

[6.13.3]-[6.14.4]

= J]Rd

g(U)2A(U)du,

3. Let IT be a Poisson process with constant intensity A on the surface of the sphere of lR 3 with radius 1. Let P be the process given by the (X, Y) coordinates of the points projected on a plane passing through the centre of the sphere. Show that P is a Poisson process, and find its intensity function. 4.

Repeat Exercise (3), when IT is a homogeneous Poisson process on the ball {(Xl, X2, X3) :

xi +X5:::: 1}.

xf +

5. You stick pins in a Mercator projection of the Earth in the manner of a Poisson process with constant intensity A. What is the intensity function of the corresponding process on the globe? What would be the intensity function on the map if you formed a Poisson process of constant intensity A of meteorite strikes on the surface of the Earth? 6. Shocks. The rth point Tr of a Poisson process N of constant intensity A on lR+ gives rise to an effect Xre-a(t-Tr ) at time t 2: Tr , where the Xr are independent and identically distributed with finite variance. Find the mean and variance of the total effect Set) = L~~~) Xre-a(t-Tr ) in terms of the first two moments of the X r , and calculate cov(S(s), S(t». What is the behaviour of the correlation p (S(s), S(t» as s -+ 00 with t - s fixed? 7. Let N be a non-homogeneous Poisson process on lR+ with intensity function A. Find the joint density of the first two inter-event times, and deduce that they are not in general independent. S. Competition lemma. Let {Nr (t) : r 2: 1} be a collection of independent Poisson processes onlR+ with respective constant intensities {Ar : r 2: 1}, such that Lr Ar = A < 00. Set N(t) = Lr N r (t), and let I denote the index of the process supplying the first point in N, occurring at time T. Show that JP'(l

= i,

T 2: t)

= JP'(l = i)JP'(T

2: t)

A'

= ; e- At ,

6.14 Exercises. Markov chain Monte Carlo 1. Let P be a stochastic matrix on the finite set e with stationary distribution Jr. Define the inner product (x, y) = LkEE> XkYkJrk, and let Z2(Jr) = {x E lRE> : (x, x) < oo}. Show, in the obvious notation, that P is reversible with respect to Jr if and only if (x, Py) = (Px, y) for all x, y E Z2(Jr). 2. Barker's algorithm. Show that a possible choice for the acceptance probabilities in Hastings's general algorithm is Jrjgji .. _ bIJ , Jrigij

where G

=

+ Jrjgji

(gij) is the proposal matrix.

3. Let S be a countable set. For each j E S, the sets Ajko k E S, form a partition of the interval [0,1]. Let g : S x [0,1] -+ S be given by gU, u) = k if U E Ajk. The sequence {Xn : n 2: O} of random variables is generated recursively by Xn+l = g(X n , Un+l), n 2: 0, where {Un: n 2: 1} are independent random variables with the uniform distribution on [0, 1]. Show that X is a Markov chain, and find its transition matrix. 4.

Dobrushin's bound. Let U

=

(ust) be a finite lSI x

coefficient is defined to be d(U)

=

ITI stochastic matrix.

1 sup L lUi( -

Dobrushin's ergodic

ujtl.

i,jES tET

(a) Show that, if V is a finite

ITI x lUI stochastic matrix, then d(UV) :::: d(U)d(V). 75

[6.15.1]-[6.15.7]

Exercises

Markov chains

(b) Let X and Y be discrete-time Markov chains with the same transition matrix P, and show that

~]lE'(Xn

= k)

-lE'(Yn

= k)1 :s d(p)n LIlE'(Xo = k)

k

-lE'(Yo

= k)l.

k

6.15 Problems 1. Classify the states of the discrete-time Markov chains with state space S transition matrices 2

(a)

(1

J

0

2 0 0

0

1

1

4 0

1

D

1

2 0 0 0

(1

(b)

{l,2,3,4} and

2 0 0

i)

In case (a), calculate h4(n), and deduce that the probability of ultimate absorption in state 4, starting from 3, equals ~. Find the mean recurrence times of the states in case (b). 2. A transition matrix is called doubly stochastic if all its column sums equal I , that is, if Li Pij for all j E S.

=

I

(a) Show that if a finite chain has a doubly stochastic transition matrix, then all its states are non-null persistent, and that if it is, in addition, irreducible and aperiodic then Pij (n) -+ N- 1 as n -+ 00, where N is the number of states. (b) Show that, if an infinite irreducible chain has a doubly stochastic transition matrix, then its states are either all null persistent or all transient.

3.

Prove that intercommunicating states of a Markov chain have the same period.

(a) Show that for each pair i, j of states of an irreducible aperiodic chain, there exists N = N (i, j) such that Pij (n) > 0 for all n ::: N. (b) Let X and Y be independent irreducible aperiodic chains with the same state space S and transition matrix P. Show that the bivariate chain Zn = (X n , Yn ), n ::: 0, is irreducible and aperiodic. (c) Show that the bivariate chain Z may be reducible if X and Y are periodic.

4.

Suppose {X n : n ::: O} is a discrete-time Markov chain with Xo of visits made subsequently by the chain to the state j. Show that

5.

ifn

= i.

Let N be the total number

= 0,

if n ::: 1, and deduce that lE'(N

= (0) =

1 if and only if Ii}

=

Ijj

=

1.

6. Let i and j be two states of a discrete-time Markov chain. Show that if i communicates with j, then there is positive probability of reaching j from i without revisiting i in the meantime. Deduce that, if the chain is irreducible and persistent, then the probability Ii} of ever reaching j from i equals 1 for all i and j. 7. Let {Xn : n ::: O} be a persistent irreducible discrete-time Markov chain on the state space S with transition matrix P, and let x be a positive solution of the equation x = xP. (a) Show that i, j E S,

76

n:::

1,

Exercises

Problems

[6.15.8]-[6.15.13]

defines the n-step transition probabilities of a persistent irreducible Markov chain on S whose first -passage probabilities are given by

i=/-j,n:::l, where lji(n) = lP'(Xn = i, T > n I Xo = j) and T = min{m > 0: Xm (b) Show that x is unique up to a multiplicative constant. (c) Let Tj = min{n ::: 1 : Xn for all i, j E S. 8.

= j) and define hij = lP'(Tj

Renewal sequences. The sequence

U

= {un:

.::: Ti

= j).

I Xo = i). Show thatxihij = Xjhji

n ::: 0) is called a 'renewal sequence' if

n

uo

= 1,

Un

=L

fiUn-i

for n ::: 1,

i=l

for some collection f = Un : n ::: 1) of non-negative numbers summing to 1. (a) Show that U is a renewal sequence if and only if there exists a Markov chain X on a countable state space S such that Un = lP'(Xn = s I Xo = s), for some persistent s E S and all n ::: 1. (b) Show that if U and v are renewal sequences then so is {unvn : n ::: 0). 9. Consider the symmetric random walk in three dimensions on the set of points {(x, y, z) : x, y, z = 0, ±l, ±2, ... ); this process is a sequence {Xn : n ::: 0) of points such that lP'(Xn+l = Xn + €) = for € = (±l, 0, 0), (0, ±l, 0), (0,0, ±l). Suppose that Xo = (0,0,0). Show that

!

lP'(X2n=(0,0,0))= (

1)2n

6

L

L

(2n)! (1)2n (2n) (n!)2 C"'k,)2= 2 3n.,"k' i+j+k=n I.J. . n i+j+k=n I.J . .

and deduce by Stirling's formula that the origin is a transient state. 10. Consider the three-dimensional version of the cancer model (6.12.16). If K of Theorem (6.12.18) inevitable in this case? 11. Let X be a discrete-time Markov chain with state space S p= (

=

1, are the empires

= {I, 2], and transition matrix

l-a

fJ

Classify the states of the chain. Suppose that afJ > 0 and afJ =/- 1. Find the n-step transition probabilities and show directly that they converge to the unique stationary distribution as n ---+ 00. For what values of a and fJ is the chain reversible in equilibrium? 12. Another diffusion model. N black balls and N white balls are placed in two urns so that each contains N balls. After each unit of time one ball is selected at random from each urn, and the two balls thus selected are interchanged. Let the number of black balls in the first urn denote the state of the system. Write down the transition matrix of this Markov chain and find the unique stationary distribution. Is the chain reversible in equilibrium? 13. Consider a Markov chain on the set S = {O, 1,2, ... ) with transition probabilities Pi,i+ 1 = ai, Pi,O = 1 - ai, i ::: 0, where (ai : i ::: 0) is a sequence of constants which satisfy 0 < ai < 1 for all i. Let bo = 1, bi = aOal ... ai-l for i ::: 1. Show that the chain is (a) persistent if and only if bi ---+ 0 as i ---+ 00, (b) non-null persistent if and only if Li bi < 00, and write down the stationary distribution if the latter condition holds.

77

[6.15.14]-[6.15.19]

Exercises

Markov chains

Let A and f3 be positive constants and suppose that ai = 1 - Ai -fJ for all large i. Show that the chain is (c) transient if f3 > 1, (d) non-null persistent if f3 < 1. Finally, if f3 = 1 show that the chain is (e) non-null persistent if A > 1, (f) null persistent if A :s 1.

14. Let X be a continuous-time Markov chain with countable state space S and standard semigroup {Pd. Show that Pij(t) is a continuous function of t. Let get) = -logPii(t); show that g is a continuous function, g(O) = 0, and g(s + t) known theorem gives the result that lim get) = A 1-1-0 t

:s g(s) +

exists and

get). We say that g is 'subadditive', and a well

get) A = sup - 1>0 t

Deduce that gii = liml-l-O t- I {Pii (t) - l} exists, but may be

:s 00.

-00.

15. Let X be a continuous-time Markov chain with generator G = (gij) and suppose that the transition semi group PI satisfies PI = exp(tG). Show that X is irreducible if and only if for any pair i, j of states there exists a sequence kl , k2' ... , k n of states such that gi,k1 gk1 ,k2 ... gkn,j "# O. 16. (a) Let X = {X(t) :

-00 < t < oo} be a Markov chain with stationary distribution 1C, and suppose that X (0) has distribution 1C. We call X reversible if X and Y have the same joint distributions, where Y (t) = X (-t). (i) If X (t) has distribution 1C for all t, show that Y is a Markov chain with transition probabilities Ptj (t) = (Jrj /Jri) Pji (t), where the Pji (t) are the transition probabilities of the chain X.

(ii) If the transition semigroup {PI} of X is standard with generator G, show that Jri gij = Jrj gji (for all i and j) is a necessary condition for X to be reversible. (iii) If PI = exp(tG), show that X (t) has distribution 1C for all t and that the condition in (ii) is sufficient for the chain to be reversible. (b) Show that every irreducible chain X with exactly two states is reversible in equilibrium. (c) Show that every birth-death process X having a stationary distribution is reversible in equilibrium.

17. Show that not every discrete-time Markov chain can be imbedded in a continuous-time chain. More precisely, let

p-( -

a I-a

l-aa);:lor some 0 < a < 1

be a transition matrix. Show that there exists a uniform semigroup {Pd of transition probabilities in continuous time such that PI = P, if and only if < a < 1. In this case show that {PI} is unique and calculate it in terms of a.

i

18. Consider an imrnigration-death process X(t), being a birth-death process with rates An = A, JLn = nJL. Show that its generating function G(s, t) = E(sx(t) is given by

where p = A/ JL and X (0) = I. Deduce the limiting distribution of X (t) as t --+

00.

19. Let N be a non-homogeneous Poisson process on lR+ = [0, 00) with intensity function A. Write down the forward and backward equations for N, and solve them. Let N (0) = 0, and find the density function of the time T until the first arrival in the process. If A(t) =c/(l +t),showthatE(T) < ooifandonlyifc > 1.

78

Exercises

Problems

[6.15.20]-[6.15.27]

20. Successive offers for my house are independent identically distributed random variables X I , X 2, ... , having density function f and distribution function F. Let YI = X I, let Y2 be the first offer exceedingYI,andgenerallyletYn+1 bethefirstofferexceedingYn . ShowthatYI, Y2, ... are the times of arrivals in a non-homogeneous Poisson process with intensity function A(t) = f(t)/(l - F(t)). The Yi are called 'record values' . Now let 21 be the first offer received which is the second largest to date, and let 22 be the second such offer, and so on. Show that the 2i are the arrival times of a non-homogeneous Poisson process with intensity function A. 21. Let N be a Poisson process with constant intensity A, and let YI, Y2, ... be independent random variables with common characteristic function 4J and density function f. The process N* (t) = YI + Y2 + ... + YN(t) is called a compound Poisson process. Yn is the change in the value of N* at the nth arrival of the Poisson process N. Think of it like this. A 'random alarm clock' rings at the arrival times of a Poisson process. At the nth ring the process N* accumulates an extra quantity Yn . Write down a forward equation for N* and hence find the characteristic function of N*(t). Can you

see directly why it has the form which you have found? 22. If the intensity function A of a non-homogeneous Poisson process N is itself a random process, then N is called a doubly stochastic Poisson process (or Cox process). Consider the case when A(t) = A for all t, and A is a random variable taking either of two values AI or ,1,.2, each being picked Find the probability generating function of N(t), and deduce its mean and with equal probability variance.

i.

23. Show that a simple birth process X with parameter A is a doubly stochastic Poisson process with intensity function A(t) = AX(t). 24. The Markov chain X = (X (t) : t the time t and are given by lP'(X(t

as h

to.

~

0) is a birth process whose intensities Ak(t) depend also on

1 + ILk + h) = k + 11 X(t) = k) = - h + o(h) 1 + ILt

Show that the probability generating function G(s, t)

aG

aG}

s- 1 {

a-t=I+ILt

O<s
G+ILs-a;'

Hence find the mean and variance of X(t) when X(O)

= E(sX(t)) satisfies

= I.

25. (a) Let X be a birth--death process with strictly positive birth rates ,1,.0, AI, ... and death rates ILl, IL2, .... Let TJi be the probability that X (t) ever takes the value 0 starting from X (0) = i. Show that j ~ 1,

2::1

and deduce that TJi = 1 for all i so long as ej = 00 where ej (b) For the discrete-time chain on the non-negative integers with

= ILIIL2 ... ILj 1(,1,. 1,1,.2' .. Aj). j2

and

Pj,j-I

=

j2

+ (j + 1)2'

find the probability that the chain ever visits 0, starting from 1. 26. Find a good necessary condition and a good sufficient condition for the birth--death process X of Problem (6. 15.25a) to be honest. 27. Let X be a simple symmetric birth--death process with An extinction. Show that lP'(T :::: x I X (0)

= l) = 79

= ILn = nA, and let T

Ax ( 1 + Ax

)1

'

be the time until

[6.15.28]-[6.15.33]

Exercises

Markov chains

and deduce that extinction is certain if lP'(X (0) < 00) = 1. Show that lP'(AT / I ::s x I X(O) = I) -+ e- 1/ x as I -+ 00.

28. Immigration-death with disasters. Let X be an immigration-death-disaster process, that is, a birth-death process with parameters Ai = A, /-Li = i /-L, and with the additional possibility of 'disasters' which reduce the population to O. Disasters occur at the times of a Poisson process with intensity 8, independently of all previous births and deaths. (a) Show that X has a stationary distribution, and find an expression for the generating function of this distribution. (b) Show that, in equilibrium, the mean of X(t) is A/(8

+ /-L).

29. With any sufficiently nice (Lebesgue measurable, say) subset B of the real line IE. is associated a random variable X(B) such that (i) X(B) takes values in (0, 1,2, ... ), (ii) if B1, B2, ... , Bn are disjoint then X (Bd, X(B2), ... , X(Bn) are independent, and furthermore X(B1 U B2) = X(B1) + X(B2), (iii) the distribution of X (B) depends only on B through its Lebesgue measure ('length') IB I, and lP'(X (B) ::: 1) lP'(X(B) = 1) -+ 1

as

IBI

-+

O.

Show that X is a Poisson process.

30. Poisson forest. Let N be a Poisson process in IE. 2 with constant intensity A, and let R(1) <

R(2)

<

... be the ordered distances from the origin of the points of the process. (a) Show that R~l)' R~2)' ... are the points of a Poisson process on IE.+ (b) Show that

R(k)

=

[0, 00) with intensity An.

has density function 2

f(r)

=

2n Ar(Anr 2)k-1 e -)..rrr (k-1)! '

r > O.

31. Let X be a n-dimensional Poisson process with constant intensity A. Show that the volume of the largest (n-dimensional) sphere centred at the origin which contains no point of X is exponentially distributed. Deduce the density function of the distance R from the origin to the nearest point of X. Show thatJE(R) = r(l/n)/{n(Ac)l/n} where c is the volume of the unit ball ofIE.n and r is the gamma function.

+ 1 people suffers an epidemic. Let X (t) be the number of ill people at time t, and suppose that X (0) = 1 and X is a birth process with rates Ai = Ai (N + 1 - i). Let T be the length of time required until every member of the popUlation has succumbed to the illness. Show that

32. A village of N

1 N JE(T)

=i

1

L keN + 1 k=l

k)

and deduce that 2(log N + y) + O(N-2) A(N + 1) where y is Euler's constant. It is striking that JE(T) decreases with N, for large N. JE(T)

=

33. A particle has velocity Vet) at time t, where Vet) is assumed to take values in (n Transitions during (t, t

+ h) are possible as follows:

(v lP'(V(t

+ h) = w I Vet) = v) =

{

+ !)h + o(h)

1- 2vh +o(h) (v - !)h

80

+ o(h)

if w

ifw if w

= v + 1, = v, = v-I.

+!

:n ::: OJ.

Exercises

Problems

Initially V (0)

= ~.

[6.15.34]-[6.15.39]

Let 00

G(s, t)

L snlP'(V(t) = n + ~).

=

n=O

(a) Show that

aG =

at

and deduce that G(s, t)

= {I + (I

(I_d

aG

as

_ (I-s)G

- s)t)-I.

(b) Show that the expected length mn(T) oftime for which V is given by mn(T)

= n + i during the time interval [0, T]

= loT lP'(V(t) = n + ~) dt

and that, for fixed k, mk(T) -log T -+ - L~=I i-I as T -+

00.

(c) What is the expected velocity of the particle at time t? 34. A random sequence of non-negative integers {Xn : n ::: 0) begins Xo produced by with probability ~,

=

0, X I

I, and is

with probability ~. Show that Yn = (Xn-I, Xn) is a transient Markov chain, and find the probability of ever reaching (I, I) from (I, 2). 35. Take a regular hexagon and join opposite comers by straight lines meeting at the point C. A particle performs a symmetric random walk on these 7 vertices, starting at A. Find: (a) the probability of return to A without hitting C, (b) the expected time to return to A,

(c) the expected nmber of visits to C before returning to A, (d) the expected time to return to A, given that there is no prior visit to C. 36. Diffusion, osmosis. Markov chains are defined by the following procedures at any time n: (a) Bernoulli model. Two adjacent containers A and B each contain m particles; m are of type I and m are of type II. A particle is selected at random in each container. If they are of opposite types they are exchanged with probability a if the type I is in A, or with probability f3 if the type I is in B. Let Xn be the number of type I particles in A at time n. (b) Ehrenfest dog-flea model. Two adjacent containers contain m particles in all. A particle is selected at random. If it is in A it is moved to B with probability a, if it is in B it is moved to A with probability f3. Let Yn be the number of particles in A at time n. In each case find the transition matrix and stationary distribution of the chain. 37. Let X be an irreducible continuous-time Markov chain on the state space S with transition probabilities Pjk(t) and unique stationary distribution 1C, and write lP'(X(t) = j) = aj(t). If c(x) is a concave function, show that d(t) = LjES 7rjc(aj (t)/7rj) increases to c(l) as t -+ 00. 38. With the notation of the preceding problem, let Uk(t) = lP'(X(t) = k I X(O) = 0), and suppose the chain is reversible in equilibrium (see Problem (6.15.16». Show that uO(2t) = Lj (7rO/7rj )Uj (t)2, and deduce that uo(t) decreases to 7ro as t -+ 00. 39. Perturbing a Poisson process. Let IT be the set of points in a Poisson process on lR,d with constant intensity A. Each point is displaced, where the displacements are independent and identically distributed. Show that the resulting point process is a Poisson process with intensity A.

81

[6.15.40]-[6.15.46]

Exercises

Markov chains

40. Perturbations continued. Suppose for convenience in Problem (6.15.39) that the displacements have a continuous distribution function and finite mean, and that d = 1. Suppose also that you are at the origin originally, and you move to a in the perturbed process. Let LR be the number of points formerly on your left that are now on your right, and RL the number of points formerly on your right that are now on your left. Show that lE(LR) = lE(Rd if and only if a = ~ where ~ is the mean displacement of a particle. Deduce that if cars enter the start of a long road at the instants of a Poisson process, having independent identically distributed velocities, then, if you travel at the average speed, in the long run the rate at which you are overtaken by other cars equals the rate at which you overtake other cars. 41. Ants enter a kitchen at the instants of a Poisson process N of rate A; they each visit the pantry and then the sink, and leave. The rth ant spends time Xr in the pantry and Yr in the sink (and Xr + Yr in the kitchen altogether), where the vectors Vr = (Xr, Yr ) and Vs are independent for r =j: s. At time t = the kitchen is free of ants. Find the joint distribution of the numbers A(t) of ants in the pantry and B(t) of ants in the sink at time t. Now suppose the ants arrive in pairs at the times of the Poisson process, but then separate to behave independently as above. Find the joint distribution of the numbers of ants in the two locations.

°

42. Let {Xr : r 2: I} be independent exponential random variables with parameter A, and set Sn = ~~=I X r . Show that: (a) Yk = Ski Sn, 1 :s k :s n - 1, have the same distribution as the order statistics of independent variables {Uk : 1 :s k :s n - I} which are uniformly distributed on (0, 1), (b) Zk = Xkl Sn, 1 :s k :s n, have the same joint distribution as the coordinates of a point (UI, ... , Un) chosen uniformly at random on the simplex ~~=I Ur = 1, U r 2: for all r.

°

43. Let X be a discrete-time Markov chain with a finite number of states and transition matrix P = (Pij) where Pij > for all i, j. Show that there exists A E (0, 1) such that [Pij (n) - 1lj [ < An, where 1C is the stationary distribution.

°

44. Under the conditions of Problem (6.15.43), let Vi (n) = ~~':6 I{Xr=iJ be the number of visits of the chain to i before time n. Show that

Show further that, if

f

is any bounded function on the state space, then

lE

(/

1 ;;

n-I ~ f(X,) -

~ f(i)1li

/2)

-'?

0.

45. Conditional entropy. Let A and B = (Bo, BI, ... , Bn) be a discrete random variable and vector, respectively. The conditional entropy of A with respect to B is defined as R(A [ B) = lE(lE{-log f(A [ B) B}) where f(a [ b) = JP'(A = a [ B = b). Let X be an aperiodic Markov chain on a finite state space. Show that

I

and that R(Xn+I [ Xn)

-'? -

L L Pij log Pij 1li

as n

-'? 00,

j

if X is aperiodic with a unique stationary distribution

1C.

46. Coupling. Let X and Y be independent persistent birth-death processes with the same parameters (and no explosions). It is not assumed that Xo = Yo. Show that: 82

Exercises

Problems

[6.15.47]-[6.15.49]

(a) for any A c:; JR, 1lP'(Xt E A) -lP'(Yt E A)I -+ 0 as t -+ 00, (b) iflP'(XO ::: YO) = 1, then E[g(X t )] ::: E[g(Yt )] for any increasing function g. 47. Resources. The number of birds in a wood at time t is a continuous-time Markov process X. Food resources impose the constraint 0 ::: X(t) ::: n. Competition entails that the transition probabilities obey Pk,k+1 (h)

= A(n -

k)h

+ o(h),

= /J.,kh + o(h).

Pk,k-I (h)

Find E(sX(t», together with the mean and variance of X(t), when X(O) t -+ oo?

=

r. What happens as

48. Parrando's paradox. A counter performs an irreducible random walk on the vertices 0, 1, 2 of the triangle in the figure beneath, with transition matrix

P=

(~I

Po

o

where Pi

+ qi = 1 for all i.

qo)

PI

q2

P2

o

Show that the stationary distribution lTo

=

1 - q2PI 3 - qlPO - q2PI - qoP2

1C

has

,

with corresponding formulae for lTI, lT2.

o

2

Suppose that you gain one peseta for each clockwise step of the walk, and you lose one peseta for each anticlockwise step. Show that, in equilibrium, the mean yield per step is Y

= 2)2Pi

- l)lTi

= 3(2poPIP2 -

. I

+

POPI - PIP2 - P2PO Po 3 - qlPo - q2PI - qOP2

+ PI + P2 -1).

Consider now three cases of this process:

= ~ - a for each i, where a > O. Show that the mean yield per step satisfies YA < O. We have that Po = fa - a, PI = P2 = ~ - a, where a > O. Show that YB < 0 for sufficiently

A. We have Pi B.

small a. C. At each step the counter is equally likely to move according to the transition probabilities of case A or case B, the choice being made independently at every step. Show that, in this case, Po = a, PI = P2 = ~ - a. Show that YC > 0 for sufficiently small a. The fact that two systematically unfavourable games may be combined to make a favourable game is called Parrando's paradox. Such bets are not available in casinos.

to -

49. Cars arrive at the beginning of a long road in a Poisson stream of rate A from time t = 0 onwards. A car has a fixed velocity V > 0 which is a random variable. The velocities of cars are independent and identically distributed, and independent of the arrival process. Cars can overtake each other freely. Show that the number of cars on the first x miles of the road at time t has the Poisson distribution with parameter AE[V- I min{x, VtJ].

83

[6.15.50]-[6.15.51]

Exercises

Markov chains

50. Events occur at the times of a Poisson process with intensity A, and you are offered a bet based on the process. Let t > O. You are required to say the word 'now' immediately after the event which you think will be the last to occur prior to time t. You win if you succeed, otherwise you lose. If no events occur before t you lose. If you have not selected an event before time t you lose. Consider the strategy in which you choose the first event to occur after a specified time s, where 0< s < t. (a) Calculate an expression for the probability that you win using this strategy. (b) Which value of s maximizes this probability? (c) If At 2: 1, show that the probability that you win using this value of s is e -1.

51. A new Oxbridge professor wishes to buy a house, and can afford to spend up to one million pounds. Declining the services of conventional estate agents, she consults her favourite internet property page on which houses are announced at the times of a Poisson process with intensity A per day. House prices may be assumed to be independent random variables which are uniformly distributed over the interval (800,000, 2,000,000). She decides to view every affordable property announced during the next 30 days. The time spent viewing any given property is uniformly distributed over the range (1,2) hours. What is the moment generating function of the total time spent viewing houses?

84

7 Convergence of random variables

7.1 Exercises. Introduction 1.

Letr 2: 1, and define IIXllr

(a) IlcXllr

= lei . IIXllr

for c

= {EIXrl}l/r.

Show that:

E ~,

(b) IIX + Yllr :::: IIXllr + IIYllr, (c) IIXllr = 0 if and only iflP'(X = 0) = 1. This amounts to saying that II . Ilr is a norm on the set of equivalence classes of random variables on a given probability space with finite rth moment, the equivalence relation being given by X ~ Y if and only if lP'(X = Y) = 1. Define (X, Y) = E(XY) for random variables X and Y having finite variance, and define IIXII .J(X, X). Show that:

2.

(a) (aX

+ bY, Z) = a(X, Z) + b(Y, Z), = 2(IIX11 2 + IIYI1 2), the parallelogram property,

(b) IIX + n2 + IIX - n2 (c) if (Xi, Xj) = 0 for all i

f:

j then

IItXil12 1=1

= tllxif. 1=1

IE

3. Let E > O. Let g, h : [0, 1] ~ ~, and define dE(g, h) = dx where E Ig(u) - h(u)1 > E}. Show that dE does not satisfy the triangle inequality. 4.

=

=

{u E [0,1] :

Levy metric. For two distribution functions F and G, let d(F, G)

= int{ 8 >

0 : F(x - 8) - 8 :::: G(x) :::: F(x

+ 8) + Hor all x

E ~}.

Show that d is a metric on the space of distribution functions. 5. Find random variables X, XI, X 2 , ... such that E(IXn - X12) ~ 0 as n ~ for all n.

7.2 Exercises. Modes of convergence 1.

(a) Suppose Xn ~ X where r 2: 1. Show that EIX~I ~ EIXrl.

(b) Suppose Xn (c) Suppose Xn

I

~

2

~

X. Show that E(Xn)

~

X. Show that var(Xn )

E(X). Is the converse true?

~

var(X).

85

00,

but EIXnl

= 00

[7.2.2]-[7.3.3]

Exercises

Convergence of random variables

Dominated convergence. Suppose IXn I ::s Z for all n, where E(Z) <

2.

00.

Prove that if Xn

~

X

I

then Xn ---+ X. 3. Give a rigorous proof that E(XY) = E(X)E(y) for any pair X, Y of independent non-negative random variables on (r2, :F, JlD) with finite means. [Hint: For k 2: 0, n 2: 1, define Xn = kin if kin ::s X < (k + l)ln, and similarly for Yn . Show that Xn and Yn are independent, and Xn ::s X, and Yn ::s Y. Deduce that EX n ~ EX and EYn ~ EY, and also E(XnYn ) ~ E(XY).] 4. Show that convergence in distribution is equivalent to convergence with respect to the Levy metric of Exercise (7.1.4). (a) Suppose that Xn

S.

XnlYn

~

~ X and Yn ~ c, where c is a constant. Show that XnYn ~ cX, and that

Xlc if c"# O.

(b) Suppose that Xn

~ 0 and Yn ~ Y, and let g : ][~.z ~ JR. be such that g(x, y) is a continuous p

function of y for all x, and g(x, y) is continuous at x = 0 for all y. Show that g(X n , Yn ) ---+ g(O, Y). [These results are sometimes referred to as 'Slutsky's theorem(s)'.]

6. A

Let Xl, X2, ... be random variables on the probability space (r2,:F, JlD). Show that the set (w E r2 : the sequence Xn (w) converges} is an event (that is, lies in :F), and that there exists a random variable X (that is, an T-measurable function X : r2 ~ JR.) such that Xn(w) ~ X(w) for WE A.

=

7.

Let {X n } be a sequence of random variables, and let {en} be a sequence of reals converging to the limit c. For convergence almost surely, in rth mean, in probability, and in distribution, show that the convergence of Xn to X entails the convergence of cnXn to cX.

8. Let {Xn} be a sequence of independent random variables which converges in probability to the limit X. Show that X is almost surely constant.

9. Convergence in total variation. The sequence of discrete random variables X n , with mass functions fn, is said to converge in total variation to X with mass function f if

L

Ifn(x) - f(x)1 ~ 0

as

n ~

00.

x

Suppose Xn

~

X in total variation, and u : JR.

~

JR. is bounded. Show that E(u(Xn))

~

E(u(X)).

10. Let {X r : r 2: I} be independent Poisson variables with respective parameters {A r : r 2: I}. Show that L:~l Xr converges or diverges almost surely according as L:~l Ar converges or diverges.

7.3 Exercises. Some ancillary results (a) Suppose that Xn ~ X. Show that {X n } is Cauchy convergent in probability in that, for all > 0, JlD(IX n - Xm I > E) ~ 0 as n, m ~ 00. In what sense is the converse true? (b) Let {X n } and {Yn } be sequences of random variables such that the pairs (Xi, Xj) and (Yi, Yj)

1.

E

have the same distributions for all i, j. If Xn ~ X, show that Yn converges in probability to some limit Y having the same distribution as X. 2. Show that the probability that infinitely many of the events {An : n 2: I} occur satisfies JlD(An i.o.) 2: lim sUPn--+oo JlD(An). 3. Let {Sn : n 2: O} be a simple random walk which moves to the right with probability p at each step, and suppose that So = O. Write Xn = Sn - Sn-l.

86

Exercises

Some ancillary results

[7.3.4]-[7.3.10]

= 0 i.o.} is not a tail event of the sequence {X n }. Show that lP'(Sn = 0 i.o.) = 0 if p =1= -!. Let Tn = Sn/.Jli, and show that

(a) Show that {Sn (b) (c)

inf Tn ::5 -x} { lim n--->oo

n {lim sup Tn ~ x} n--->oo

is a tail event of the sequence {Xn}, for all x > 0, and deduce directly that lP'(Sn

= 0 i.o.) =

1 if

1

p= ,,;.

4.

Hewitt-Savage zero-one law. Let Xl, X2, ... be independent identically distributed random variables. The event A, defined in terms of the X n , is called exchangeable if A is invariant under finite permutations of the coordinates, which is to say that its indicator function IA satisfies IA(XI, X2, ... , X n , ... ) = IA (Xii' Xi2' ... , Xi n , Xn+I, ... ) for all n ~ 1 and all permutations (iI, i2,···, in) of 0, 2, ... , n). Show that all exchangeable events A are such that either lP'(A) = 0 orlP'(A) = 1. 5. Returning to the simple random walk S of Exercise (3), show that {Sn = 0 i.o.} is an exchangeable event with respect to the steps of the walk, and deduce from the Hewitt-Savage zero-Dne law that it has probability either 0 or 1.

6. Weierstrass's approximation theorem. Let f : [0, 1] ---+ IE. be a continuous function, and let Sn be a random variable having the binomial distribution with parameters n and x. Using the formula JE(Z) = JE(ZIA) + JE(ZIAc) with Z = f(x) - f(n- I Sn) and A = {[n- I Sn - x[ > 8}, show that

t

lim sup If(X) f(k/n) (n)xkO_ X)n-kl n-+oo O<x
= O.

You have proved Weierstrass's approximation theorem, which states that every continuous function on [0, 1] may be approximated by a polynomial uniformly over the interval. 7. Complete convergence. A sequence Xl, X 2, . .. of random variables is said to be completely convergent to X if for all € > O. LlP'([X n - X[ > €) < 00 n

Show that, for sequences of independent variables, complete convergence is equivalent to a.s. convergence. Find a sequence of (dependent) random variables which converges a.s. but not completely. 8. Let Xl, X 2, . .. be independent identically distributed random variables with common mean and finite variance. Show that

(;)-1 9.

Let {Xn : n

~

as n ---+

I} be independent and exponentially distributed with parameter 1. Show that

. sup XnlP' ( hm n-+oo logn 10. Let {Xn : n

~

00.

I~i<j~n

=

1)

=

1.

I} be independent N(O, 1) random variables. Show that:

~ =.J2) = 1,

(a) lP' (lim sup n-+oo vlogn

87

f.L

[7.3.11]-[7.5.1]

Exercises

. { 01 (b) IP'(Xn > an 1.0.) =

Convergence of random variables

if ~n IP'(X I > an) < if ~n IP'(XI > an)

00,

= 00.

11. Construct an example to show that the convergence in distribution of Xn to X does not imply the convergence of the unique medians of the sequence Xn. 12. (i) Let {Xr : r 2: I} be independent, non-negative and identically distributed with infinite mean. Show that lim sUPr-+oo Xr / r = 00 almost surely. (ii) Let {X r } be a stationary Markov chain on the positive integers with transition probabilities j Pjk

=

{

j+2 _2_ j+2

if k

= j + 1,

if k

=

1.

(a) Find the stationary distribution of the chain, and show that it has infinite mean. (b) Show that limsuPr-+oo Xr/r

::s 1 almost surely.

13. Let {Xr : 1 ::s r ::s n} be independent and identically distributed with mean = n- I ~~=I X r . Show that

~

and finite variance

(f2. Let X

converges in distribution to the N(O, 1) distribution as n ____

00.

7.4 Exercise. Laws of large numbers 1.

Let X2, X3, ... be independent random variables such that IP'(Xn

= n) = IP'(Xn =

-n)

1 2n logn

= ---,

lP'(X n

= 0) =

1 1- --. nlogn

Show that this sequence obeys the weak law but not the strong law, in the sense that n- I ~1 Xi converges to 0 in probability but not almost surely. 2. Construct a sequence {Xr : r 2: I} of independent random variables with zero mean such that n- I ~~=I Xr ---- -00 almost surely, as n ---- 00. 3. Let N be a spatial Poisson process with constant intensity A in lRd , where d 2: 2. Let S be the ball ofradius r centred at zero. Show that N(S)/ISI ---- A almost surely as r ____ 00, where lSI is the volume of the ball.

7.5 Exercises. The strong law 1. Entropy. The interval [0, 1] is partitioned into n disjoint sub-intervals with lengths PI, P2, ... , Pn, and the entropy of this partition is defined to be n

h = -

L Pi log Pi· i=1

88

Martingales

Exercises

[7.5.2]-[7.7.4]

Let X I, X2, . " be independent random variables having the uniform distribution on [0, 1], and let Zm(i) be the number ofthe XI, X2, ... , Xm which lie in the ith interval of the partition above. Show that

R m --

IIn Pizm(i)

i=1

satisfies m-Ilog Rm """"* -h almost surely as m """"*

00.

2. Recurrentevents. Catastrophes occur at the times TI, T2, ... where Ti = XI +X2+" ,+Xi and the Xi are independent identically distributed positive random variables. Let N(t) = max{n : Tn :::: t} be the number of catastrophes which have occurred by time t. Prove that if E(X I) < 00 then N(t) """"* 00 and N(t)/t """"* l/E(X I) as t """"* 00, almost surely. 3. Random walk. Let X I, X 2, ... be independent identically distributed random variables taking values in the integers Z and having a finitemean. Show that the Markov chain S = {Sn} given by Sn = L:} Xi is transient if E(X I) =f. O.

7.6 Exercise. Law of the iterated logarithm 1.

A function rJ>(x) is said to belong to the 'upper class' if, in the notation of this section, J1D(Sn > = O. A consequence of the law of the iterated logarithm is that .ja log log x is in the upper class for all a > 2. Use the first Borel-Cantelli lemma to prove the much weaker fact that rJ>(x) = .ja logx is in the upper class for all a > 2, in the special case when the Xi are independent N(O, 1) variables.

rJ>(n)...jn i.o.)

7.7 Exercises. Martingales 1. Let X I, X2, . .. be random variables such that the partial sums Sn determine a martingale. Show that E(Xi Xj) = 0 if i =f. j.

=

XI

+ X 2 + ... + X n

2. Let Zn be the size of the nth generation of a branching process with immigration, in which the family sizes have mean JL (=f. 1) and the mean number of immigrants in each generation is m. Suppose that E(Zo) < 00, and show that

is a martingale with respect to a suitable sequence of random variables. 3. Let Xo, XI, X2, ... be a sequence of random variables with finite means and satisfyingE(Xn+1 I Xo, Xj, ... , Xn) = aXn + bXn-1 for n 2: 1, where 0 < a, b < 1 and a + b = 1. Find a value of a for which Sn = aXn + Xn-I, n 2: 1, defines a martingale with respect to the sequence X. 4. Let Xn be the net profit to the gambler of betting a unit stake on the nth play in a casino; the Xn may be dependent, but the game is fair in the sense that E(Xn+1 I XI, X2, ... , Xn) = 0 for all n. The gambler stakes Yon the first play, and thereafter stakes fn (X I, X2, ... , Xn) on the (n + l)th play, where fI, h, ... are given functions. Show that her profit after n plays is n

Sn

= LXdi-I(XI,X2"",Xi-I), i=1

where fo = Y. Show further that the sequence S = {Sn} satisfies the martingale condition E(Sn+1 X I, X2, ... , Xn) = Sn, n 2: 1, if Y is assumed to be known throughout.

89

I

[7.8.1]-[7.9.5]

Exercises

Convergence of random variables

7.8 Exercises. Martingale convergence theorem 1. Ko)mogorov's inequality. Let Xl, X2, ... be independent random variables with zero means and finite variances, and let Sn = Xl + X2 + ... + X n . Use the Doob-Kolmogorov inequality to show that for

E

> O.

2. Let Xl, X 2, ... be independent random variables such that 2::n n -2 var(Xn ) < mogorov's inequality to prove that

t i=l

Xi

-~(Xi) ~

Y

as n ____

00,

as n ____

00.

00.

Use Kol-

I

for some finite random variable Y, and deduce that 1 n - L(Xi -lEXi) n i=l

~0

(You may find Kronecker's lemma to be useful: if (an) and (bn ) are real sequences with b n 1 2::i ai/bi < 00, then b; 2::7=1 ai ---- 0 as n ---- 00.)

t

00

and

3. Let S be a martingale with respect to X, such that lE(S;) < K < 00 for some K E 1Ft. Suppose that var(Sn) ____ 0 as n ____ 00, and prove that S = limn--+oo Sn exists and is constant almost surely.

7.9 Exercises. Prediction and conditional expectation 1.

Let Y be uniformly distributed on [-1, 1] and let X = y2. (a) Find the best predictor of X given Y, and of Y given X. (b) Find the best linear predictor of X given Y, and of Y given X.

2.

Let the pair (X, Y) have a general bivariate normal distribution. Find lE(Y I X).

3. Let Xl, X 2, ... , Xn be random variables with zero means and covariance matrix V = (Vij), and let Y have finite second moment. Find the linear function h of the Xi which minimizes the mean squared error lE{(Y - h(X1, ... , Xn))2}. 4. Verify the following properties of conditional expectation. You may assume that the relevant expectations exist. (i) lE{lE(Y I g,)} = lE(Y). (ii) lE(aY + {3Z I g,) = alE(Y I g,) + {3lE(Z I g,) for a, {3 E 1Ft. (iii) lE(Y I g,) ::: 0 if Y ::: O. (iv) lE(Y I g,) = lE{lE(Y I Je) I g,} if g, ~ Je. (v) lE(Y I g,) = lE(Y) if Y is independent of IG for every G E g,. (vi) Jensen's inequality. g{lE(Y I g,)} ~ lE{g(Y) I g,} for all convex functions g. (vii) If Yn ~ Y and IYn I ~ Z a.s. where lE(Z) < 00, then lE(Yn I g,) ~ lE(Y (Statements (ii)-{vi) are of course to be interpreted 'almost surely'.) 5. lE(Y

Let X and Y have joint mass function f(x, y) = {x(x I X) < 00 while lE(Y) = 00.

90

+ 1)}-1 for x

I g,).

= y = 1,2, .... Show that

Exercises

Problems

[7.9.6]-[7.11.5]

6. Let (n, :F, IP') be a probability space and let fJ, be a sub-a-field of:F. Let H be the space of fJ,-measurable random variables with finite second moment. (a) Show that H is closed with respect to the norm II . 112. (b) Let Y be a random variable satisfying lE(y2) < two statements for any M E H: (i) lE{(Y - M)Z} = 0 for all Z E H, (ii) lE{(Y - M)IG}

= 0 for all G

00,

and show the equivalence of the following

fJ,.

E

7.10 Exercises. Uniform integrability 1. Show that the sum {Xn + Yn } of two uniformly integrable sequences {X n } and {Yn } gives a uniformly integrable sequence. (a) Suppose that Xn ~ X where r :::: 1. Show that {IXnI T : n :::: I} is uniformly integrable, and deduce that lE(X~) ~ lE(XT) if r is an integer. (b) Conversely, suppose that {IXnI T : n :::: I} is uniformly integrable where r :::: 1, and show that T .f P Xn -+ Xl Xn -+ X.

2.

3. Let g : [0, (0) ~ [0, (0) be an increasing function satisfying g(x)/x ~ 00 as x that the sequence {X n : n:::: I} is uniformly integrable ifsuPnlE{g(IXnl)} < 00.

~ 00.

Show

4. Let {Zn : n :::: O} be the generation sizes of a branching process with Zo var(Zl) =J= O. Show that {Zn : n :::: O} is not uniformly integrable.

5.

Pratt's lemma. Suppose that Xn ::'S Yn ::'S Zn where Xn ~ X, Yn ~ Y, and Zn ~ Z. If

lE(Xn)

~

lE(X) and lE(Zn)

~

lE(Z), show that lE(Yn)

~

lE(Y).

6. Let {Xn : n :::: I} be a sequence of variables satisfying lE(suPn IXnl) < uniformly integrable.

00.

Show that {Xn} is

7.11 Problems 1.

Let Xn have density function

n::::

1.

With respect to which modes of convergence does Xn converge as n 2.

~

oo?

(i) Suppose that Xn ~ X and Yn ~ Y, and show that Xn + Yn ~ X + Y. Show that the corresponding result holds for convergence in rth mean and in probability, but not in distribution.

(ii) Show that if Xn ~ X and Yn ~ Y then XnYn ~ XY. Does the corresponding result hold for the other modes of convergence? 3.

Let g : JR

~ JR be continuous. Show that g(Xn) ~ g(X) if Xn ~ X.

4. Let Y1, Y2, ... be independent identically distributed variables, each of which can take any value in {O, 1, ... , 9} with equal probability Let Xn = 2:: =1 Yi lO-i. Show by the use of characteristic

fa.

1

functions that Xn converges in distribution to the uniform distribution on [0, 1]. Deduce that Xn ~ Y for some Y which is uniformly distributed on [0, 1]. 5.

Let N(t) be a Poisson process with constant intensity on JR.

91

[7.11.6]-[7.11.13]

Exercises

Convergence of random variables

(a) Find the covariance of N(s) and N(t). (b) Show that N is continuous in mean square, which is to say that E({N(t + h) - N(t)}2) -+ 0 as h -+ o. (c) Prove that N is continuous in probability, which is to say that lP'(IN(t + h) - N(t)1 > E) -+ 0 as h -+ 0, for all E > O. (d) Show that N is differentiable in probability but not in mean square.

6. Prove that n -I 2:: =1 Xi ~ 0 whenever the Xi are independent identically distributed variables with zero means and such that E(Xt) < 00.

1

7.

Show that Xn ~ X whenever

8.

Show that if Xn

9.

2:: n E(IXn -

~ X then aXn + b ~

If X has zero mean and variance

a 2,

aX

Xn <

00

for some r > O.

+ b for any real a and b.

show that

lP'(X :::: t)

.:s

a2

-2--2

+t

a

for t > O.

10. Show that Xn ~ 0 if and only if

11. The sequence {Xn} is said to be mean-square Cauchy convergent if E{(Xn - Xm)2} -+ 0 as 00. Show that {Xn} converges in mean square to some limit X if and only if it is mean-square Cauchy convergent. Does the corresponding result hold for the other modes of convergence? m, n -+

12. Suppose that {Xn} is a sequence of uncorrelated variables with zero means and uniformly bounded variances. Show that n- I 2:: =1 Xi ~ O.

1

13. Let XI, X2, ... be independent identically distributed random variables with the common distribution function F, and suppose that F(x) < 1 for all x. Let Mn = max{XI, X2, ... , Xn} and suppose that there exists a strictly increasing unbounded positive sequence ai, a2, ... such that lP' (Mn/an .:s x) -+ H(x) for some distribution function H. Let us assume that H is continuous with 0 < H(1) < 1; substantially weaker conditions suffice but introduce extra difficulties. (a) Show that n[l - F(anx)] -+ -log H(x) as n -+ 00 and deduce that 1 - F(anx)

I - F(a n )

-+

log H(x) log H(1)

--=--- ifx > O.

(b) Deduce that if x > 0 1 - F(tx)

I - F(t)

(c) Set x

-+

log H(x) log H(1)

as t -+

00.

= XlX2 and make the substitution log H(e X ) g(x) = log H(1)

to find that g(x

+ y) = g(x)g(y), and deduce that H(x)

= { ~xp(-ax-,B) 92

ifx :::: 0, ifx < 0,

Problems

Exercises

[7.11.14]-[7.11.19]

for some non-negative constants ex and {3. You have shown that H is the distribution function of y-l, where Y has a Weibull distribution.

14. Let Xl, X2, ... , Xn be independent and identically distributed random variables with the Cauchy distribution. Show that Mn = max{Xl, X2, ... , Xnl is such that rrMn/n converges in distribution, the limiting distribution function being given by H(x) = e- 1jx if x ::::: o.

15. Let Xl, X 2, ... be independent and identically distributed random variables whose common characteristic function t/J satisfies t/J'(O)

= iJ1,.

Show that n- 12:.1=1 Xj

~

J1,.

16. Total variation distance. The total variation distance dTV (X, Y) between two random variables X and Y is defined by dTV(X, Y) = sup [E(u(X» - E(u(y»[ u:llulloo=l where the supremum is over all (measurable) functions u : IE. -+ IE. such that lIulioo = supx lu(x)1 satisfies lIu 1100 = 1. (a) If X and Y are discrete with respective masses fn and gn at the points x n , show that

dTV(X, Y)

=L

Ifn - gnl

=2

n

sup [IP'(X

(b) If X and Y are continuous with respective density functions

dTV(X, Y)

=

1

E

A) -IP'(Y

E

A)[.

AS;:1R

00

If(x) - g(x)1 dx

f

and g, show that

= 2 sup [IP'(X E A) -IP'(Y E

A)[.

AS;:1R

-00

(c) Show that dTV(X n , X) -+ 0 implies that Xn -+ X in distribution, but that the converse is false. (d) Maximal coupling. Show that IP'(X i= Y) ::::: !dTV(X, Y), and that there exists a pair X', Y' having the same marginals for which equality holds. (e) If Xi, Yj are independent random variables, show that

17. Let g : IE. -+ IE. be bounded and continuous. Show that 00

(nA)k

Lg(k/n)--e-nA -+ g(A) k=O k!

asn -+

00.

18. Let Xn and Ym be independent random variables having the Poisson distribution with parameters nand m, respectively. Show that (X n - n) - (Ym - m)

-JXn

+ Ym

-S. N(O, 1)

asm,n -+

00.

19. (a) Suppose that Xl, X2, ... is a sequence of random variables, each having a normal distribution,

-S.

and such that Xn X. Show that X has a normal distribution, possibly degenerate. (b) For each n ::::: I, let (X n , Yn ) be a pair of random variables having a bivariate normal distribution. Suppose that Xn distribution.

~ X and Yn ~ Y, and show that the pair (X, 93

Y) has a bivariate normal

[7.11.20]-[7.11.26]

Exercises

Convergence of random variables

20. Let X I, X2, ... be random variables satisfying var(Xn) < c for all n and some constant c. Show that the sequence obeys the weak law, in the sense that n- l L;'i(Xj -lEXj) converges in probability to 0, if the correlation coefficients satisfy either of the following: (i) p(Xj, Xj) ::s 0 for all i =1= j, (ii) p(Xj, Xj) ~ 0 as Ii - jl ~ 00.

21. Let X I, X2, ... be independent random variables with common density function f(x)

if Ixl

{O

=

c

x 2 loglxl

::s 2,

if Ixl > 2,

where c is a constant. Show that the Xj have no mean, but n- l convergence does not take place almost surely.

L;1=1 Xj ~ 0 as n ~

00.

Show that

22. Let Xn be the Euclidean distance between two points chosen independently and uniformly from the n-dimensional unit cube. Show thatlE(Xn)/Fn ~ 1/,)6 asn ~ 00.

23. Let Xl, X2, ... be independent random variables having the uniform distribution on [-1,1]. Show that

24. Let X I, X 2, ... be independent random variables, each Xk having mass function given by 1 IP'(Xk = k) = IP'(Xk = -k) = 2k2 '

IP'(Xk = 1) = IP'(Xk = -1) = Show that Un

= L;'i Xj

satisfies Un / Fn

~ ( 1-

k; )

if k > 1.

.E.. N(O, 1) but var(Un / Fn) ~ 2 as n ~ 00.

25. Let X I, X 2, . .. be random variables, and let N I, N2, . .. be random variables taking values in the positive integers such that Nk

~

00

as k

~

00.

Show that:

(i) if Xn

.E.. X and the Xn are independent of the Nk, then XNk .E.. X as k ~ 00,

(ii) if Xn

~

X then XNk

~

X ask

~

26. Stirling's formula. (a) Let a(k, n) = n k /(k - I)! for 1 ::s k

00.

::s n + 1. Use the fact that 1 -

a(n - k, n) < e-k2/(2n) a(n + 1, n) -

x

::s e-x if x

if k :::: O.

(b) Let X I, X2, ... be independent Poisson variables with parameter 1, and let Sn Define the function g : lR ~ lR by g(x)

=

-x { 0

if 0 ::::

x::::

-M,

otherwise,

where M is large and positive. Show that, for large n, lE ( g { Sn

Fn

n}) e-

n

= Fn{a(n+l,n)-a(n-k,n)}

94

:::: 0 to show that

= X I + ... + X n .

Exercises

Problems

where k fonnula:

[7.11.27]-[7.11.33]

LMn 1/2 J. Now use the central limit theorem and (a) above, to deduce Stirling's

as n --+

00.

27. A bag contains red and green balls. A ball is drawn from the bag, its colour noted, and then it is returned to the bag together with a new ball of the same colour. Initially the bag contained one ball of each colour. If Rn denotes the number of red balls in the bag after n additions, show that Sn = Rnl (n + 2) is a martingale. Deduce that the ratio of red to green balls converges almost surely to some limit as n --+ 00. 28. Anscombe's theorem. Let {Xi: i ::: l} be independent identically distributed random variables with zero mean and finite positive variance a 2 , and let Sn = L1 Xi. Suppose that the integer-valued random process M(t) satisfies t- I M(t) SM(t)

a./Bi

~ N(O,

1)

~ and

e as t

e is a positive constant. Show that

--+ 00, where

SM(t)

aJM(t)

~ N(O, 1) as t

--+ 00.

You should not assume that the process M is independent of the Xi.

29. Kolmogorov's inequality. Let X I, X2, . .. be independent random variables with zero means, andSn = XI +X2+" ·+Xn . LetMn = maxI::;:k::;:n ISkl and show thatE(S;IAk) > c 2lP'(Ak) where Ak = (Mk-I ::: c < Md and c > O. Deduce Kolmogorov's inequality: c > O.

30. Let XI, X2, ... be independent random variables with zero means, and let Sn = XI + X2 ... + X n . Using Kolmogorov's inequality or the martingale convergence theorem, show that: (i) L~I Xi converges almost surely ifL~1 E(X~) < 00, (ii) if there exists an increasing real sequence (bn ) such that bn --+

L~I E(X~)lb~ <

00,

then b;;1 L~I Xk ~ 0 as n --+

00,

+

and satisfying the inequality

00.

31. Estimating the transition matrix. The Markov chain Xo, X I, ... , Xn has initial distribution

Ii

= lP'(Xo = i) and transition matrix P. The log-likelihood function A(P) is defined as A(P)

log(fxo PXO,X\PX\,X2'" PXn_\,x n ). Show that:

(a) A(P)

= log fxo + Li,j Nij log Pij

where Nij is the number of transitions from i to j,

(b) viewed as a function of the Pij' A(P) is maximal when Pij (c) if X is irreducible and ergodic then Pij ~ Pij as n --+

= Pij

where Pij

= Nij ILk Nib

00.

32. Ergodic theorem in discrete time. Let X be an irreducible discrete-time Markov chain, and let J-ti be the mean recurrence time of state i. Let Vi(n) = L~:J I{Xr=iJ be the number of visits to i up to n - 1, and let! be any bounded function on S. Show that: -I V; ( ) a.s. () an i n ---+

J-ti-I as n --+ (b) if J-ti < 00 for all i, then

00,

1 n-I - L!(Xr ) --+ L!(i)lJ-ti n r=O iES

asn --+

00.

33. Ergodic theorem in continuous time. Let X be an irreducible persistent continuous-time Markov chain with generator G and finite mean recurrence times J-t j.

95

[7.11.34]-[7.11.37]

Exercises

Convergence of random variables

lot

1 a.s. 1 I{X(s)=il ds ----+ - - as t --+ 00; (a) Show that t 0 fLjgj (b) deduce that the stationary distribution 1C satisfies 11:j = 1I (fLj gj ); (c) show that, if f is a bounded function on S,

-1 iot t

f(X(s» ds ~ L11:i!(i)

0

as t --+

00.

i

34. Tail equivalence. Suppose that the sequences {Xn : n ::: 1) and {Yn : n ::: I} are tail equivalent, which is to say that L:~lJP'(Xn i= Yn ) < 00. Show that: (a) L:~1 Xn and L:~1 Yn converge or diverge together, (b) L:~1 (X n - Yn ) converges almost surely, (c) ifthereexist a random variable X and a sequence an such that an too and a;l L:~=1 Xr ~ X, then 1 n

-LYr~X. an r=l

35. Three series theorem. Let {Xn : n ::: 1) be independent random variables. Show that L:~1 Xn converges a.s. if, for some a > 0, the following three series all converge: (a) L:n JP'(IXnl > a),

(b) L:n var(XnIllxnl::::a}), (c) L:n E(XnI{IXnl::::a})' [The converse holds also, but is harder to prove.]

36. Let {Xn : n ::: 1) be independent random variables with continuous common distribution function F. We call Xk a record value for the sequence if Xk > Xr for 1 :::; r < k, and we write h for the indicator function of the event that Xk is a record value. (a) Show that the random variables h are independent. (b) Show that Rm

= L:k=l

Ir satisfies Rm/log m ~ 1 as m --+

00.

37. Random harmonic series. Let {Xn : n ::: 1) be a sequence of independent random variables with JP'(Xn = 1) = JP'(Xn = -1) = ~. Does the series L:~=1 Xr Ir converge a.s. as n --+ oo?

96

8 Random processes

8.2 Exercises. Stationary processes 1.

Flip-flop. Let {Xn} be a Markov chain on the state space S = {O, I} with transition matrix p= (

l-a

f3

where a + f3 > O. Find: (a) the correlation p(X m , X m +n ), and its limit as m --+ (b) limn ..... oo n- 1 2::~=llP'(Xr = 1).

00

with n remaining fixed,

Under what condition is the process strongly stationary?

2. Random telegraph. Let {N(t) : t 2: O} be a Poisson process of intensity A, and let To be Define T(t) = To(-l)N(t). Show an independent random variable such that IP'(To = ±1) = that {T(t) : t 2: O} is stationary and find: (a) p(T(s), T(s + t)), (b) the mean and variance of X(t) = T(s)ds.

i.

fJ

3.

Korolyuk-Khinchin theorem. An integer-valued counting process {N(t) : t 2: O} with N(O)

ois called crudely stationary if Pk(S, t) =

IP'(N(s

+ t) -

=

N(s) = k) depends only on the length t and

not on the location s. It is called simple if, almost surely, it has jump discontinuities of size 1 only. Show that, for a simple crudely stationary process N, limt-!-o t-11P'(N(t) > 0) = JE(N(1)).

8.3 Exercises. Renewal processes 1.

Let (fn : n 2: 1) be a probability distribution on the positive integers, and define a sequence (un: n 2: 0) by Uo = 1 and Un = 2::~=1 frun-r, n 2: 1. Explain why such a sequence is called a renewal sequence, and show that u is a renewal sequence if and only if there exists a Markov chain U and a state s such that Un = IP'(Un = s I Uo = s). 2. Let {Xi: i 2: I} be the inter-event times of a discrete renewal process on the integers. Show that the excess lifetime Bn constitutes a Markov chain. Write down the transition probabilities of the sequence {Bn} when reversed in equilibrium. Compare these with the transition probabilities of the chain U of your solution to Exercise (1). 3. Let (un: n 2: 1) satisfy Uo = 1 and Un = 2::~=1 frun-r for n 2: 1, where (fr : r 2: 1) is a non-negative sequence. Show that: (a) Vn = pnun is a renewal sequence if p > 0 and 2::~1 pn fn = 1, (b) as n --+ 00, pnu n converges to some constant c.

97

[8.3.4]-[8.4.5]

Random processes

Exercises

4. Events occur at the times of a discrete-time renewal process N (see Example (5.2.15)). Let Un be the probability of an event at time n, with generating function U(s), and let F(s) be the probability generating function of a typical inter-event time. Show that, if Is I < 1:

5.

Prove Theorem (8.3.5): Poisson processes are the only renewal processes that are Markov chains.

8.4 Exercises. Queues 1. The two tellers in a bank each take an exponentially distributed time to deal with any customer; their parameters are ).. and fl, respectively. You arrive to find exactly two customers present, each occupying a teller. (a) You take a fancy to a randomly chosen teller, and queue for that teller to be free; no later switching is permitted. Assuming any necessary independence, what is the probability p that you are the last of the three customers to leave the bank? (b) If you choose to be served by the quicker teller, find p. (c) Suppose you go to the teller who becomes free first. Find p. 2. Customers arrive at a desk according to a Poisson process of intensity)... There is one clerk, and the service times are independent and exponentially distributed with parameter fl,. At time there is exactly one customer, currently in service. Show that the probability that the next customer arrives before time t and finds the clerk busy is

°

3. Vehicles pass a crossing at the instants of a Poisson process of intensity)..; you need a gap of length at least a in orde,r to cross. Let T be the first time at which you could succeed in crossing to the other side. Show that E(T) = (e aA - 1)/)", and find E(e IlT ). Suppose there are two lanes to cross, carrying independent Poissonian traffic with respective rates ).. and fl,. Find the expected time to cross in the two cases when: (a) there is an island or refuge between the two lanes, (b) you must cross both in one go. Which is the greater?

4. Customers arrive at the instants of a Poisson process of intensity).., and the single server has exponential service times with parameter fl,. An arriving customer who sees n customers present (including anyone in service) will join the queue with probability (n + 1)/(n + 2), otherwise leaving for ever. Under what condition is there a stationary distribution? Find the mean of the time spent in the queue (not including service time) by a customer who joins it when the queue is in equilibrium. What is the probability that an arrival joins the queue when in equilibrium? 5. Customers enter a shop at the instants of a Poisson process of rate 2. At the door, two representatives separately demonstrate a new corkscrew. This typically occupies the time of a customer and the representative for a period which is exponentially distributed with parameter 1, independently of arrivals and other demonstrators. If both representatives are busy, customers pass directly into the shop. No customer passes a free representative without being stopped, and all customers leave by another door. If both representatives are free at time 0, show the probability that both are busy at time t is ~ - ~e-2t + --±-e-5t 5

3

15

98

Exercises

Problems

[8.5.1]-[8.7.4]

8.5 Exercises. The Wiener process 1.

For a Wiener process W with W (0)

= 0, show that

~+~ sin- l ~ 2rr Vt

lP(W(s) > 0, W(t) > 0) =

4

for s < t.

Calculate lP(W(s) > 0, W(t) > 0, W(u) > 0) when s < t < u.

2.

Let W be a Wiener process. Show that, for s < t < u, the conditional distribution of W(t) given W(s) and W(u) is normal

N(

U-t)W(S)+(t-S)W(U) (U-t)(t-S») , . u-s u-s

Deduce that the conditional correlation between W(t) and W(u), given W(s) and W(v), where s < t < u < v, is (v - u)(t - s) (v - t)(u - s)

+ bW2

3.

For what values of a and b is aWl independent standard Wiener processes?

a standard Wiener process, where WI and W2 are

4. Show that a Wiener process W with variance parameter a 2 has finite quadratic variation, which is to say that n-l

I)W((j

+ l)t/n)

- W(jt/n)}2

~ a 2t

as n ---+

00.

j=o 5.

Let W be a Wiener process. Which of the following define Wiener processes? (a) -W(t),

(b) -JiW(l),

(c) W(2t) - W(t).

8.7 Problems 1. Let {Znl be a sequence of uncorrelated real-valued variables with zero means and unit variances, and define the 'moving average'

,

Yn = LaiZn-i, i=O

for constants ao, aI, ... ,a,. Show that Y is stationary and find its autocovariance function. 2. Let {Znl be a sequence of uncorrelated real-valued variables with zero means and unit variances. Suppose that {Ynl is an 'autoregressive' stationary sequence in that it satisfies Yn = aYn-1 + Zn, -00 < n < 00, for some real a satisfying lal < 1. Show that Y has autocovariance function c(m) a1ml/(l - ( 2 ).

=

3. Let {Xnl be a sequence of independent identically distributed Bernoulli variables, each taking values 0 and 1 with probabilities 1 - P and p respectively. Find the mass function of the renewal process N(t) with interarrival times {Xnl. 4. Customers arrive in a shop in the manner of a Poisson process with parameter A. There are infinitely many servers, and each service time is exponentially distributed with parameter /L. Show that the number Q(t) of waiting customers at time t constitutes a birth-death process. Find its stationary distribution.

99

[8.7.5]-[8.7.7]

Exercises

Random processes

5. Let X (t) = Y cos(et) + Z sin(et) where Y and Z are independent N(O, 1) random variables, and let X(t) = R cos(et IJI) where Rand IJI are independent. Find distributions for R and IJI such that the processes X and X have the same fdds.

±

6. Bartlett's theorem. Customers arrive at the entrance to a queueing system at the instants of an inhomogeneous Poisson process with rate function A(t). Their subsequent service histories are independent of each other, and a customer arriving at time s is in state A at time s + t with probability pes, t). Show that the number of customers in state A at time t is Poisson with parameter J~oo )..(u)p(u, t - u) duo 7. In a Prague teashop (U Mysaka), long since bankrupt, customers queue at the entrance for a blank bill. In the shop there are separate counters for coffee, sweetcakes, pretzels, milk, drinks, and ice cream, and queues form at each of these. At each service point the customers' bills are marked appropriately. There is a restricted number N of seats, and departing customers have to queue in order to pay their bills. If interarrival times and service times are exponentially distributed and the process is in equilibrium, find how much longer a greedy customer must wait if he insists on sitting down. Answers on a postcard to the authors, please.

100

9

Stationary processes

9.1 Exercises. Introduction 1. Let ... , Z_I, Zo, ZI, Z2, ... be independent real random variables with means 0 and variances 1, and let a, f3 E 1ft Show that there exists a (weakly) stationary sequence {Wn } satisfying Wn = aWn-I + f3Wn-2 + Zn, n = ... , -1,0,1, ... , if the (possibly complex) zeros of the quadratic equation z2 - az - f3 = 0 are smaller than 1 in absolute value.

2. Let U be uniformly distributed on [0, 1] with binary expansion U the sequence

=

2::~1 XiZ-i. Show that

00

Vn

=

L Xi+n2-i,

n:::: 0,

i=1

is strongly stationary, and calculate its autocovariance function. 3. Let {X n : n = ... , -1,0, 1, ... } be a stationary real sequence with mean 0 and autocovariance function c(m). (i) Show that the infinite series 2::~0 anXn converges almost surely, and in mean square, whenever 2::~0 lanl < 00. (ii) Let 00

Yn

=

L akXn-k.

n= ... ,-I,O,I, ...

k=O

where 2::~0 lak I < that

00. Find an expression for the autocovariance function cy of Y, and show 00

L

Icy(m)1 <

00.

m=-oo

4. Let X = {Xn : n :::: O} be a discrete-time Markov chain with countable state space S and stationary distribution 1C, and suppose that Xo has distribution 1C. Show that the sequence {f(X n ) : n :::: O} is strongly stationary for any function f : S -* 1ft

9.2 Exercises. Linear prediction 1.

Let X be a (weakly) stationary sequence with zero mean and autocovariance function c(m).

(i) Find the best linear predictor Xn+1 of Xn+1 given X n . (ii) Find the best linear predictor Xn+1 of Xn+1 given Xn and Xn-I.

101

[9.2.2]-[9.4.2]

Stationary processes

Exercises 2

~

-

2

.

(iii) Find an expression for D = E{(Xn+l - Xn+l) } - E{(Xn+l - Xn+l) }, and evaluate this expression when: (a) Xn = cos(nU) where U is uniform on [-7r, 7r],

(b) X is an autoregressive scheme with c(k)

= a 1kl

where

lal

< 1.

Suppose lal < 1. Does there exist a (weakly) stationary sequence {Xn : zero means and autocovariance function

2.

=

c(k)

{~ +a 2

1

o

-00

< n < oo} with

= 0, iflkl = 1, if k

if Ikl > 1.

Assuming that such a sequence exists, find the best linear predictor Xn of Xn given X n- 1 , X n -2, ... , and show that the mean squared error of prediction is (l + a 2 )-1. Verify that {Xn} is (weakly) stationary.

9.3 Exercises. Autocovariances and spectra 1. Let Xn = A cos (nJ..) + B sin(nJ..) where A and B are uncorrelated random variables with zero means and unit variances. Show that X is stationary with a spectrum containing exactly one point. 2.

Let U be uniformly distributed on (-7r, 7r), and let V be independent of U with distribution function F. Show that Xn = ei(U- Vn) defines a stationary (complex) sequence with spectral distribution function F.

3.

Find the autocorrelation function of the stationary process {X (t) : density function is: (i) N(O, 1),

(ii) f(x)

= 1e-1xl,

-00

< x <

-00

< t < oo} whose spectral

00.

4. Let Xl, X2, ... be a real-valued stationary sequence with zero means and autocovariance function c(m). Show that .

t

var(~n j=l Xj)

= c(O)

1

(-n,n]

(Sin.
where F is the spectral distribution function. Deduce that n- 1 'L.'j=l Xj ~ 0 if and only if F (0) - F (0-) = 0, and show that

c(O){ F (0) - F (0-)}

1 n-l ' " c(j).

= n-+oo lim n

L...J

j=O

9.4 Exercises. Stochastic integration and the spectral representation 1. Let S be the spectral process of a stationary process X with zero mean and unit variance. Show that the increments of S have zero means.

2. Moving average representation. Let X be a discrete-time stationary process having zero means, continuous strictly positive spectral density function f, and with spectral process S. Let Yn

=

1

(-n,n]

einA J27rf(J..)

102

dS(J..).

Exercises

Gaussian processes

[9.4.3]-[9.6.2]

Show that ... , Y-I, Yo, YI, ... is a sequence of uncorrelated random variables with zero means and unit variances. Show that Xn may be represented as a moving average Xn = L,'J=-oo aj Yn- j where the aj are constants satisfying

V2]fj(A) =

00

aje- ijA

L

for A E (-]f, ]fl.

j=-oo 3.

Gaussian process. Let X be a discrete-time stationary sequence with zero mean and unit variance, and whose fdds are of the multivariate-normal type. Show that the spectral process of X has independent increments having normal distributions.

9.S Exercises. The ergodic theorem 1.

Let T = {1, 2, ... } and let 1 be the set of invariant events of (~T , :I3 T ). Show that 1 is a (T -field.

2.

Assume that XI, X2, ... is a stationary sequence with autocovariance function c(m). Show that In

)

var ( - LXi n i=1 Assuming that j-l L,{~J

c(i)

=

2 n j -1 c(O) 2" LLc(i) - - . n j=1 i=O n

-+ (T2 as j -+ 00, show that

var(~

t Xi)

-+ (T2

as n -+

00.

n i=1

3. Let XI, X2, ... be independent identically distributed random variables with zero mean and unit variance. Let 00

Yn = LaiXn+i

forn ~ 1

i=O

where the ai are constants satisfying L,i at < 00. Use the martingale convergence theorem to show that the above summation converges almost surely and in mean square. Prove that n -I L,t=1 Yi -+ 0 a.s. and in mean, as n -+ 00.

9.6 Exercises. Gaussian processes 1.

Show that the function c(s, t)

= min {s, t} is positive definite.

That is, show that

n

L

c(tt. tj )Z/Zk > 0

j,k=1 for all 0 ::::: non-zero.

tl

< t2 < ... < tn and all complex numbers ZI, Z2, ... , Zn at least one of which is

2. Let XI, X 2, . .. be a stationary Gaussian sequence with zero means and unit variances which satisfies the Markov property. Find the spectral density function of the sequence in terms of the constant p = COV(Xl, X2).

103

[9.6.3]-[9.7.7]

3.

Exercises

Stationary processes

Show that a Gaussian process is strongly stationary if and only if it is weakly stationary.

4. Let X be a stationary Gaussian process with zero mean, unit variance, and autocovariance function c(t). Find the autocovariance functions of the processes X2 = {X(t)2 : -00 < t < oo} and X 3 = {X(t)3 : -00 < t < oo}.

9.7 Problems 1. Let ... , X-I, Xo, XI, ... be uncorrelated random variables with zero means and unit variances, and define 00

Yn

=

Xn

+ 0: L

i

l

.8 - Xn-i

for -

00

< n <

00,

i=1 where 0: and.8 are constants satisfying 1.81 < 1, 1.8 - 0:1 < 1. Find the best linear predictor of Yn+1 given the entire past Yn , Yn - I , ....

2.

Let {Yk :

-00

< k < oo} be a stationary sequence with variance af, and let r

Xn

=

L

akYn-k.

-00

< n <

00,

k=O

where ao, al, ... , a r are constants. Show that X has spectral density function

ai

= var(X I), and Ga(z) = 2:k=o akz k . where fy is the spectral density function of Y, Calculate this spectral density explicitly in the case of 'exponential smoothing', when r ak = J.l(l - J.L), and 0 < J.L < 1.

= 00,

3. Suppose that Yn+l = O:Yn + .8Yn - 1 is the best linear predictor of Yn+l given the entire past Yn , Yn-I, ... of the stationary sequence {Yk : -00 < k < oo}. Find the spectral density function of the sequence. 4. Recurrent events (5.2.15). Meteorites fall from the sky at integer times Tl, T2, ... where Tn = Xl + X2 + ... + X n · We assume that the Xi are independent, X2, X3, '" are identically distributed, and the distribution of X 1 is such that the probability that a meteorite falls at time n is constant for all n. Let Yn be the indicator function of the event that a meteorite falls at time n. Show that {Yn } is stationary and find its spectral density function in terms of the characteristic function of X2. 5. Let X = {Xn : n ::: I} be given by Xn = cos(nU) where U is uniformly distributed on [-]f, ]fl. Show that X is stationary but not strongly stationary. Find the autocorrelation function of X and its spectral density function. 6.

(a) Let N be a Poisson process with intensity A, and let 0: > O. Define X(t) = N(t + 0:) - N(t) for t ::: O. Show that X is strongly stationary, and find its spectral density function.

(b) Let W be a Wiener process and define X = (X(t) : t ::: I} by X(t) = Wet) - Wet - 1). Show that X is strongly stationary and find its autocovariance function. Find the spectral density function of X. 7. Let 21, 22, ... be uncorrelated variables, each with zero mean and unit variance. (a) Define the moving average process X by Xn = 2n + 0:2n -l where 0: is a constant. Find the spectral density function of X.

104

Exercises

Problems (b) More generally, let Yn = 2:1=0 ai Zn-i, where ao spectral density function of Y.

=

[9.7.8]-[9.7.16]

1 and a1, ... , a r are constants. Find the

8.

Show that the complex-valued stationary process X = {X(t) : -00 < t < oo} has a spectral density function which is bounded and uniformly continuous whenever its autocorrelation function p oo is continuous and satisfies Ip (t) Idt < 00.

Jo

9.

Let X

=

cov(Xo, Xn)

{Xn : n ::: I} be stationary with constant mean JJ., --,>

0 as n

--,> 00.

Show that n- 12:J=l X)

~

=

E(Xn) for all n, and such that

JJ.,.

10. Deduce the strong law of large numbers from an appropriate ergodic theorem. 11. Let IQl be a stationary measure on (RT , /B T ) where T = {I, 2, ... }. Show that IQl is ergodic if and only if 1 n a.s. and in mean Yi --,> E(y) n i=l

-L

for all Y : ll~.T --,> llHor which E(y) exists, where Yi : ]RT usual, r is the natural shift operator on ]RT •

--,> ]R

is given by Yi (x) = Y (r i - 1 (x)). As

12. The stationary measure IQl on (]RT, /B T ) is called strongly mixing if IQl(A n r- n B) --,> IQl(A)IQl(B) as n --,> 00, for all A, B E /B T ; as usual, T = {I, 2, ... } and r is the shift operator on ]RT. Show that every strongly mixing measure is ergodic. 13. Ergodic theorem. Let (Q, :F, lP') be a probability space, and let T : Q measure preserving (Le., lP'(T- 1 A) = lP'(A) for all A E :F). Let X : Q --,> and let Xi be given by Xi (w) = X(T i - 1 (w)). Show that

--,>

]R

Q be measurable and be a random variable,

1 n

- LXi

--,>

E(X 11)

a.s. and in mean

n i=l

where 1 is the a -field of invariant events of T.

If T is ergodic (in that lP'(A) equals 0 or 1 whenever A is invariant), prove that E(X almost surely.

I 1) = E(X)

14. Consider the probability space (Q,:F, lP') where Q = [0, 1), :Fis the set of Borel subsets, and lP' is Lebesgue measure. Show that the shift T : Q --,> Q defined by T(x) = 2x (mod 1) is measurable, measure preserving, and ergodic (in that lP'(A) equals 0 or 1 if A = T- 1 A). Let X : Q --,> ]R be the random variable given by the identity mapping X (w) = w. Show that the proportion of 1's, in the expansion of X to base 2, equals almost surely. This is sometimes called 'Borel's normal number theorem'.

i

15. Let g : ]R --,> ]R be periodic with period 1, and uniformly continuous and integrable over [0, 1]. Define Zn = g (X + (n - l)a), n ::: 1, where X is uniform on [0, 1] and a is irrational. Show that, as n --,> 00,

-1 Ln

Z)

--,>

n )=1

16. Let X (a) X (0)

= {X (t)

la1 g(u) du

a.s.

0

: t ::: O} be a non-decreasing random process such that:

= 0, X takes values in the non-negative integers,

(b) X has stationary independent increments,

(c) the sample paths {X (t, w) : t ::: O} have only jump discontinuities of unit magnitude. Show that X is a Poisson process.

105

[9.7.17]-[9.7.22]

Exercises

Stationary processes

17. Let X be a continuous-time pr6cess. Show that: (a) if X has stationary increments and m (t) = E(X (t)) is a continuous function of t, then there exist a and (J such that m(t) = a + (Jt, (b) if X has stationary independent increments and v(t) = var(X(t) - X (0)) is a continuous function oft then there exists a 2 such that var(X(s + t) - X(s)) = a 2t for all s. 18. A Wiener process W is called standard if W(O) = 0 and W(1) has unit variance. Let W be a standard Wiener process, and let a be a positive constant. Show that: (a) aW(t/a 2) is a standard Wiener process, (b) W(t + a) - W(a) is a standard Wiener process, (c) the process V, given by V(t) = tW(1/t) for t > 0, V(O) = 0, is a standard Wiener process, (d) the process W(I) - W(1- t) is a standard Wiener process on [0,1].

19. Let W be a standard Wiener process. Show that the stochastic integrals X(t)

=

l

dW(u),

are well defined, and prove that X (t) 1(e- ls - tl - e- s - t ), s < t.

Y(t)

=

fot e-(t-u) dW(u),

t 2: 0,

= W (t), and that Y hasautocovariancefunction cov(Y(s), Y(t)) =

20. Let W be a standard Wiener process. Find the means of the following processes, and the autocovariance functions in cases (b) and (c): (a) X(t) = IW(t)l, (b) Y(t) = ew(t),

(c) Z(t) = J~ W(u)du. Which of these are Gaussian processes? Which of these are Markov processes?

21. Let W be a standard Wiener process. Find the conditional joint density function of W(t2) and W(t3) given that W(tI) = W(t4) = 0, where t1 < t2 < t3 < t4· Show that the conditional correlation of W (t2) and W (t3) is

p=

(t4 - t3)(t2 - tI) (t4 - t2)(t3 - t1)

22. Empirical distribution function. Let UI, U2, ... be independent random variables with the uniform distribution on [0, 1]. Let Ij (x) be the indicatorfunction of the event {Uj

~ x},

and define

O~x~1.

The function Fn is called the 'empirical distribution function' of the Uj. (a) Find the mean and variance of Fn(x), and prove that .Ji1(Fn(x) -x) ~ Y(x) as n ~ 00, where Y (x) is normally distributed. (b) What is the (multivariate) limit distribution of a collection of random variables of the form {.Ji1(Fn (Xi) - Xi) : 1 ~ i ~ k}, where 0 ~ Xl < x2 < ... < xk ~ I? (c) Show that the auto covariance function of the asymptotic finite-dimensional distributions of .Ji1(Fn (x) - x), in the limit as n ~ 00, is the same as that of the process Z(t) = W(t) - t W(1), o ~ t ~ 1, where W is a standard Wiener process. The process Z is called a 'Brownian bridge' or 'tied-down Brownian motion'.

106

10 Renewals

In the absence of indications to the contrary, {X n : n 2: I} denotes the sequence of interarrival times of either a renewal process N or a delayed renewal process N d . In either case, Fd and F are the distribution functions of X 1 and X2 respectively, though Fd #- F only if the renewal process is delayed. We write J1- = E(X2)' and shall usually assume that 0 < J1- < 00. The functions m and m d denote the renewal functions of Nand N d • We write Tn = 2:7=1 Xj, the time of the nth arrival.

10.1 Exercises. The renewal equation 1. Prove that E(e8N (t) < 00 for some strictly positive () whenever E(X 1) > O. [Hint: Consider the renewal process with interarrival times X~ = EI{xe:El for some suitable E.] 2. Let N be a renewal process and let W be the waiting time until the length of some interarrival time has exceeded s. That is, W = inf{t : C(t) > s}, where C(t) is the time which has elapsed (at time t) since the last arrival. Show that Fw(x)

={

o

if x < s,

1 - F(s)

s

+ fo

Fw(x - u) dF(u)

ifx 2: s,

where F is the distribution function of an interarrival time. If N is a Poisson process with intensity A, show that E(e8W ) _ A - () -

-A-_-()e-'(;-;-;"--""'8)'---s

and E(W) = (e;"s -1)/A. You may find it useful to rewrite the above integral equation in the form of a renewal-type equation. 3. Find an expression for the mass function of N(t) in a renewal process whose interarrival times are: (a) Poisson distributed with parameter A, (b) gamma distributed, r(A, b).

4. Let the times between the events of a renewal process N be uniformly distributed on (0, 1). Find the mean and variance of N(t) for 0::: t ::: 1.

10.2 Exercises. Limit theorems 1. Planes land at Heathrow airport at the times of a renewal process with interarrival time distribution function F. Each plane contains a random number of people with a given common distribution and finite mean. Assuming as much independence as usual, find an expression for the rate of arrival of passengers over a long time period. 2. Let 21,22, ... be independent identically distributed random variables with mean 0 and finite variance a 2 , and let Tn = 2:7=1 2j. Let M be a finite stopping time with respect to the 2j such that E(M) < 00. Show that var(TM) = E(M)a 2 .

107

[10.2.3]-[10.4.1] 3.

Exercises

Renewals

Show that E(TN(t)+k) = p,(m(t)+k) for all k 2: 1, but that it is not generally true that E(TN(t)

=

p,m(t).

4. Show that, using the usual notation, the family {N (t) 1 t : 0 How might one make use of this observation?

::s

t < oo} is uniformly integrable.

5. Consider a renewal process N having interarrival times with moment generating function M, and let T be a positive random variable which is independent of N. Find E(sN(T) when: (a) T is exponentially distributed with parameter v, (b) N is a Poisson process with intensity A, in terms of the moment generating function of T. What b)? is the distribution of N(T) in this case, if T has the gamma distribution

rev,

10.3 Exercises. Excess life 1. Suppose that the distribution of the excess lifetime E(t) does not depend on t. Show that the renewal process is a Poisson process. 2.

Show that the current and excess lifetime processes, C(t) and E(t), are Markov processes.

3. Suppose that X 1 is non-arithmetic with finite mean p,. (a) Show that E(t) converges in distribution as t --+ 00, the limit distribution function being H(x)

= (X

io

2.[1 - F(y)Jdy. p,

(b) Show that the rth moment of this limit distribution is given by oo

loo

xr dH(x) =

E(Xr+l) 1

p,(r

+ 1)

,

assuming that this is finite. (c) Show that E(ECtn

= E({CXl

- t)+r)

+

lot

h(t - x)dm(x)

for some suitable function h to be found, and deduce by the key renewal theorem that E( E (t E(X~+l)/{p,(r + I)} as t --+ 00, assuming this limit is finite.

r) --+

4. Find an expression for the mean value of the excess lifetime E(t) conditional on the event that the current lifetime C(t) equals x. 5.

Let M (t)

= N (t) + 1, and suppose that X 1 has finite non-zero variance 0'2. 2 O' (m(t) + 1).

(a) Show that var(TM(t) - p,M(t» =

(b) In the non-arithmetic case, show that var(M(t»lt --+

O'

2

1p,3 as t --+ 00.

10.4 Exercise. Applications 1. Find the distribution of the excess lifetime for a renewal process each of whose interarrival times is the sum of two independent exponentially distributed random variables having respective parameters A and p,. Show that the excess lifetime has mean 1

Ae-(H/t)t

P,

A(A

+ p,

-+-----

+ p,)

108

Exercises

Problems

[10.5.1]-[10.5.6]

10.5 Exercises. Renewal-reward processes 1.

If X (t) is an irreducible persistent non-null Markov chain, and u (-) is a bounded function on the integers, show that

~ t

t u(X(s)) ds ~ L 1l'iu(i),

10

iES

where 1C is the stationary distribution of X (t).

2.

Let M(t) be an alternating renewal process, with interarrival pairs {X r , Yr : r ::: l}. Show that Id ~ tI 10t{M(s) is even} s

as t

EXl EX 1 + EYl

~

00.

3. Let C (s) be the current lifetime (or age) of a renewal process N (t) with a typical interarrival time X. Show that

~

t C(s) ds ~ E(X2) 2E(X)

10

t

as t

~

00.

Find the corresponding limit for the excess lifetime. 4. Let j and k be distinct states of an irreducible discrete-time Markov chain X with stationary distribution 1C. Show that lP'(T- < Tk J

I X 0 = k) =

1/1l'k

=-:-:::-:--=--:-:--'---=--:=--:--::c::------:-:-

E(1j

I Xo = k) + E(Tk I Xo =

j)

where Ti = min {n ::: 1 : Xn = i} is the first passage time to the state i. [Hint: Consider the times of return to j having made an intermediate visit to k.]

10.5 Problems 1.

(a) Show that lP'(N (t) ~ 00 as t ~ (0) = l. (b) Show that met) < 00 if JL =I O. (c) More generally show that, for all k > 0, E(N(t)k) <

2.

Let vet)

= E(N(t)2).

00

if JL

=I O.

Show that vet) = met)

+2

lol

met - s)dm(s).

Find vet) when N is a Poisson process.

3.

Suppose that a 2

= var(X l)

> O. Show that the renewal process N satisfies

N(t) - (t/JL)

v

~ N(O, 1),

t a 2 /JL3

4.

as t

~

Find the asymptotic distribution of the currentlife C(t) of N as t

00.

~ 00

when Xl is not arithmetic.

Let N be a Poisson process with intensity A. Show that the total life D (t) at time t has distribution functionlP'(D(t)::: x) = 1- (1 +Amin{t,x})e- AX for x::: O. Deduce that E(D(t)) = (2-e- At )/A.

5.

6. A Type I counter records the arrivals of radioactive particles. Suppose that the arrival process is Poisson with intensity A, and that the counter is locked for a dead period of fixed length T after

109

[10.5.7]-[10.5.12]

Exercises

Renewals

each detected arrival. Show that the detection process N is a renewal process with interarrival time distribution F(x) = I - e-J...(x-T) if x 2: T. Find an expression for IP'(N(t) 2: k).

7. Particles arrive at a Type I counter in the manner of a renewal process N; each detected arrival locks the counter for a dead period of random positive length. Show that

where F L is the distribution function of a typical dead period.

S.

(a) Show that met) =

1M - ~(l -

e- 2At ) if the interarrival times have the gamma distribution

r(A,2).

(b) Radioactive particles arrive like a Poisson process, intensity A, at a counter. The counter fails to register the nth arrival whenever n is odd but suffers no dead periods. Find the renewal function in of the detection process N.

9. Show that Poisson processes are the only renewal processes with non-arithmetic interarrival times having the property that the excess lifetime E(t) and the current lifetime C(t) are independent for each choice of t. 10. Let Nl be a Poisson process, and let N2 be a renewal process which is independent of Nl with non-arithmetic interarrival times having finite mean. Show that N(t) = Nl (t) + N2 (t) is a renewal process if and only if N2 is a Poisson process. 11. Let N be a renewal process, and suppose that F is non-arithmetic and that a 2 = var(X 1) < 00. Use the properties of the moment generating function F *(-0) of X 1 to deduce the formal expansion m*(O)

=-

I

0,.,.,

a2 +

_,.,.,2

2,.,.,

2

+0(1)

asO ~ O.

Invert this Laplace-Stieltjes transform formally to obtain t a2 met) = - +

,.,.,

_,.,.,2

2,.,.,

2

+ 0(1)

as t

~

00.

Prove this rigorously by showing that met)

= -t ,.,.,

FE(t)

+ lot [1- FE(t -x)]dm(x), 0

where FE is the asymptotic distribution function of the excess lifetime (see Exercise (10.3.3)), and applying the key renewal theorem. Compare the result with the renewal theorems.

12. Show that the renewal function m d of a delayed renewal process satisfies

where m is the renewal function of the renewal process with interarrival times X2, X3, .... 110

Problems

Exercises

[10.5.13]-[10.5.19]

13. Let m (t) be the mean number of living individuals at time t in an age-dependent branching process with exponential lifetimes, parameter A, and mean family size v (> 1). Prove that m(t) = I e(v-1)At where I is the number of initial members.

14. Alternating renewal process. The interarrival times of this process are Zo, Y1, Z1, Y2, ... , where the Yi and Zj are independent with respective common moment generating functions My and M z. Let p(t) be the probability that the epoch t of time lies in an interval of type Z. Show that the Laplace-Stieltjes transform p* of p satisfies

p* (8) = __1_-_M---.:Z,-,(_-_8_)_ 1 - My(-8)Mz(-8) 15. Type 2 counters. Particles are detected by a Type 2 counter of the following sort. The incoming particles constitute a Poisson process with intensity A. The jth particle locks the counter for a length Yj of time, and annuls any after-effect of its predecessors. Suppose that Y1, Y2, ... are independent of each other and of the Poisson process, each having distribution function G. The counter is unlocked at time O. Let L be the (maximal) length of the first interval of time during which the counter is locked. Show that H(t) = lP'(L > t) satisfies H(t)

= e- At [1

- G(t)]

+ fot H(t -

x)[1 - G(x)]Ae- AX dx.

Solve for H in terms of G, and evaluate the ensuing expression in the case G(x) J-t> O.

= 1-

e-/-LX where

16. Thinning. Consider a renewal process N, and suppose that each arrival is 'overlooked' with probability q, independently of all other arrivals. Let M(t) be the number of arrivals which are detected up to time t / p where p = 1 - q. (a) Show that M is a renewal process whose interarrival time distribution function Fp is given by Fp(x) = Z=~1 pqr-1 Fr(x/p), where Fn is the distribution function of the time of the nth arrival in the original process N. (b) Find the characteristic function of Fp in terms of that of F, and use the continuity theorem to show that, as p .} 0, Fp (s) -+ 1 - e- s/ /-L for s > 0, so long as the interarrival times in the original process have finite mean J-t. Interpret! (c) Suppose that p < 1, and M and N are processes with the same fdds. Show that N is a Poisson process. 17. (a) A PC keyboard has 100 different keys and a monkey is tapping them (uniformly) at random. Assuming no power failure, use the elementary renewal theorem to find the expected number of keys tapped until the first appearance of the sequence of fourteen characters oW. Shakespeare'. Answer the same question for the sequence 'omo'. (b) A coin comes up heads with probability p on each toss. Find the mean number of tosses until the first appearances of the sequences (i) HHH, and (ii) HTH.

18. Let N be a stationary renewal process. Let s be a fixed positive real number, and define X(t) = + t) - N(t) for t ::: O. Show that X is a strongly stationary process.

N(s

19. Bears arrive in a village at the instants of a renewal process; they are captured and confined at a cost of $c per unit time per bear. When a given number B bears have been captured, an expedition (costing $d) is organized to remove and release them a long way away. What is the long-run average cost of this policy?

111

11

Queues

11.2 Exercises. M1M11 1. Consider a random walk on the non-negative integers with a reflecting barrier at 0, and which moves rightwards orleftwards with respective probabilities p/(1 + p) and 1/(1 + p); when at 0, the particle moves to 1 at the next step. Show that the walk has a stationary distribution if and only if p < 1, and in this case the unique such distribution 1f is given by 7T0 = (1- p), 7Tn = (1- p2) pn-l for n 2: 1.

i

i

2. Suppose now that the random walker of Exercise (1) delays its steps in the following way. When at the point n, it waits a random length of time having the exponential distribution with parameter ()n before moving to its next position; different 'holding times' are independent of each other and of further information concerning the steps of the walk. Show that, subject to reasonable assumptions on the ()n, the ensuing continuous-time process settles into an equilibrium distribution v given by Vn = C7Tn /()n for some appropriate constant C. By applying this result to the case when ()O = A, ()n = A+ tL for n 2: 1, deduce that the equilibrium distribution of the M(A)JM(tL)11 queue is Vn = (1 - p )pn, n 2: 0, where p = A/ tL < 1. 3. Waiting time. Consider a M(A)JM(tL)11 queue with p = A/ tL satisfying p < 1, and suppose that the number Q(O) of people in the queue at time 0 has the stationary distribution 7Tn = (1- p)pn, n 2: O. Let W be the time spent by a typical new arrival before he begins his service. Show that the distribution of W is given by lP'(W ::: x) = 1- pe-X(/t-A) for x 2: 0, and note that lP'(W = 0) = 1- p. 4. A box contains i red balls and j lemon balls, and they are drawn at random without replacement. Each time a red (respectively lemon) ball is drawn, a particle doing a walk on {O, 1,2, ... } moves one step to the right (respectively left); the origin is a retaining barrier, so that leftwards steps from the origin are suppressed. Let 7T(n; i, j) be the probability that the particle ends at position n, having started at the origin. Write down a set of difference equations for the 7T(n; i, j), and deduce that 7T(n; i, j) = A(n; i, j) - A(n

where A(n; i, j) 5.

+ 1; i, j)

for i ::: j

+n

= (~)/e!n).

Let Q be a M(A)JM(tL)11 queue with Q(O) = O. Show that Pn (t) = lP'(Q(t) = n) satisfies

where the 7T(n; i, j) are given in the previous exercise. 6. Let Q(t) be the length of an M(A)JM(tL)11 queue at time t, and let Z = {Zn} be the jump chain of Q. Explain how the stationary distribution of Q may be derived from that of Z, and vice versa.

112

GIGI]

Exercises

[11.2.7]-[11.5.2]

7. Tandem queues. Two queues have one server each, and all service times are independent and exponentially distributed, with parameter J1-i for queue i. Customers arrive at the first queue at the instants of a Poisson process of rate A « min{J1-I, J1-2}), and on completing service immediately enter the second queue. The queues are in equilibrium. Show that: (a) the output of the first queue is a Poisson process with intensity A, and that the departures before time t are independent of the length of the queue at time t, (b) the waiting times of a given customer in the two queues are not independent.

11.3 Exercises. M/G/1 1. Consider M(A)ID(d)/l where p = Ad < 1. Show that the mean queue length at moments of departure in equilibrium is ~p(2 - p)/(l - p). 2. Consider M(A)IM(J1-)/l, and show that the moment generating function of a typical busy period is given by (A + J1- - s) (A + J1- - s)2 - 4AJ1-

vi

MB~)=

n

for all sufficiently small but positive values of s. 3. Show that, for a MlG/l queue, the sequence of times at which the server passes from being busy to being free constitutes a renewal process.

11.4 Exercises. GlMIl 1. Consider GIM(J1-)/l, and let CXj = E«J1-X)je-/LX/j!) where X is a typical interarrival time. Suppose the traffic intensity p is less than 1. Show that the equilibrium distribution 1C of the imbedded chain at moments of arrivals satisfies 00

1fn

= LCXi1fn+i-1

for n :::: 1.

i=O

Look for a solution of the form 1fn = en for some e, and deduce that the unique stationary distribution is given by 1fj = (l - TJ)TJj for j :::: 0, where TJ is the smallest positive root of the equation s = MX(J1-(s - 1». 2. Consider a GIM(J1-)/l queue in equilibrium. Let TJ be the smallest positive root of the equation x = MX(J1-(x - 1» where Mx is the moment generating function of an interarrival time. Show that the mean number of customers ahead of a new arrival is TJ(l - TJ)-I, and the mean waiting time is TJ{J1-(l - TJ)}-I. 3. Consider D(l)IM(J1-)/1 where J1- > 1. Show that the continuous-time queue length Q(t) does not converge in distribution as t --+ 00, even though the imbedded chain at the times of arrivals is ergodic.

11.5 Exercises. G/G/l 1. Show that, for a G/GIl queue, the starting times of the busy periods of the server constitute a renewal process. 2. Consider a GIM(J1-)/l queue in equilibrium, together with the dual (unstable) M(J1-)/GIl queue. Show that the idle periods of the latter queue are exponentially distributed. Use the theory of duality

113

[11.5.3]-[11.7.5]

Exercises

Queues

of queues to deduce for the former queue that: (a) the waiting-time distribution is a mixture of an exponential distribution and an atom at zero, and (b) the equilibrium queue length is geometric. 3. Consider G/M(fL)/1, and let G be the distribution function of S - X where S and X are typical (independent) service and interarrival times. Show that the Wiener-Hop! equation F(x) =

1:00

F(x - y)dG(y),

x:::: 0,

for the limiting waiting-time distribution F is satisfied by F (x) = 1 - TJe-/1,(I-T/)x, x :::: O. Here, TJ is the smallest positive root of the equation x = MX(fL(x - 1», where Mx is the moment generating function of X.

11.6 Exercise. Heavy traffic 1. Consider the M(A)/M(fL)/1 queue with p = AlfL < 1. Let Qp be a random variable with the equilibrium queue distribution, and show that (1 - p)Qp converges in distribution as p t 1, the limit distribution being exponential with parameter 1.

11.7 Exercises. Networks of queues 1. Consider an open migration process with c stations, in which individuals arrive at station j at rate Vj, individuals move from i to j at rate Aij
4. Difficult customers. Consider an M(A )/M(fL)/1 queue modified so that on completion of service the customerleaves with probability 8, or rejoins the queue with probability 1- 8. Find the distribution of the total time a customer spends being served. Hence show that equilibrium is possible if A < 8fL, and find the stationary distribution. Show that, in equilibrium, the departure process is Poisson, but if the rejoining customer goes to the end of the queue, the composite arrival process is not Poisson. 5. Consider an open migration process in equilibrium. If there is no path by which an individual at station k can reach station j, show that the stream of individuals moving directly from station j to station k forms a Poisson process.

114

Exercises

Problems

[11.8.1]-[11.8.7]

11.8 Problems 1. Finite waiting room. Consider M(,1,.)IM(fL)/k with the constraint that arriving customers who see N customers in the line ahead of them leave and never return. Find the stationary distribution of queue length for the cases k = 1 and k = 2. 2. Baulking. Consider M(,1,.)IM(fL)/1 with the constraint that if an arriving customer sees n customers in the line ahead of him, he joins the queue with probability p(n) and otherwise leaves in disgust. (a) Find the stationary distribution of queue length if p(n)

= (n + 1)-1.

(b) Find the stationary distribution 1C of queue length if p(n) = 2- n , and show that the probability that an arriving customer joins the queue (in equilibrium) is fL(l - :rro)/,1,..

3.

Series. In a Moscow supermarket customers queue at the cash desk to pay for the goods they want; then they proceed to a second line where they wait for the goods in question. If customers arrive in the shop like a Poisson process with parameter A and all service times are independent and exponentially distributed, parameter fLl at the first desk and fL2 at the second, find the stationary distributions of queue lengths, when they exist, and show that, at any given time, the two queue lengths are independent in equilibrium. 4. Batch (or bulk) service. Consider M/G/1, with the modification that the server may serve up to m customers simultaneously. If the queue length is less than m at the beginning of a service period then she serves everybody waiting at that time. Find a formula which is satisfied by the probability generating function of the stationary distribution of queue length at the times of departures, and evaluate this generating function explicitly in the case when m = 2 and service times are exponentially distributed. 5. Consider M(,1,.)IM(fL)/1 where A < fl. Find the moment generating function of the length B of a typical busy period, and show that 18:(B) = (fL - ,1.)-1 and var(B) = (A + fL)/(fL - ,1.)3. Show that the density function of B is

where

II

is a modified Bessel function.

6. Consider M(,1,.)/G/1 in equilibrium. Obtain an expression for the mean queue length at departure times. Show that the mean waiting time in equilibrium of an arriving customer is ~,1,.18:(S2)/(l - p) where S is a typical service time and p = ,1,.18:(S). Amongst all possible service-time distributions with given mean, find the one for which the mean waiting time is a minimum. 7. Let Wt be the time which a customer would have to wait in a M(,1,.)/G/1 queue if he were to arrive at time t. Show that the distribution function F(x; t) = IP'(Wt :s x) satisfies

aF

-

at

aF

= -

ax

- ,1,.F

+ ,1,.1P'(Wt + S :s x)

where S is a typical service time, independent of W t . Suppose that F (x, t) -+ H (x) for all x as t -+ 00, where H is a distribution function satisfying h - ,1,.H + ,1,.1P'(U + S :s x) for x > 0, where U is independent of S with distribution function H, and h is the density function of H on (0, 00). Show that the moment generating function Mu of U satisfies (l-p)() Mu«() = ,1.+ () - ,1,.Ms«()

o=

where p is the traffic intensity. You may assume that IP'(S

115

= 0) = o.

[11.8.8]-[11.8.14]

Exercises

Queues

8. Consider a G/G/l queue in which the service times are constantly equal to 2, whilst the inter arrival times take either of the values I and 4 with equal probability ~. Find the limiting waiting time distribution. 9. Consider an extremely idealized model of a telephone exchange having infinitely many channels available. Calls arrive in the manner of a Poisson process with intensity A, and each requires one channel for a length of time having the exponential distribution with parameter p" independently of the arrival process and of the duration of other calls. Let Q(t) be the number of calls being handled at time t, and suppose that Q(0) = I. Determine the probability generating function of Q(t), and deduce lE(Q(t)), lP'(Q(t) = 0), and the limiting distribution of Q(t) as t -+ 00. Assuming the queue is in equilibrium, find the proportion of time that no channels are occupied, and the mean length of an idle period. Deduce that the mean length of a busy period is (eA/iJ- - 1)/,1,.

10. Customers arrive in a shop in the manner of a Poisson process with intensity A, where 0 < A < 1. They are served one by one in the order of their arrival, and each requires a service time of unit length. Let Q(t) be the number in the queue at time t. By comparing Q(t) with Q(t + 1), determine the limiting distribution of Q(t) as t -+ 00 (you may assume that the quantities in question converge). Hence show that the mean queue length in equilibrium is ,1,(1 - ~A)/(l - A). Let W be the waiting time of a newly arrived customer when the queue is in equilibrium. Deduce from the results above that lE(W) = ~A/(1 - A).

11. Consider M(A)/D(1)/l, and suppose that the queue is empty at time O. Let T be the earliest time at which a customer departs leaving the queue empty. Show that the moment generating function MT of T satisfies + log MT(S) = (s - ,1,)(1 - MT(S)), log (1 -

D

and deduce the mean value of T, distinguishing between the cases A < 1 and A 2': 1. Suppose A < p" and consider a M(A)IM(p,)/l queue Q in equilibrium. Show that Q is a reversible Markov chain. Deduce the equilibrium distributions of queue length and waiting time. Show that the times of departures of customers form a Poisson process, and that Q(t) is independent of the times of departures prior to t. (d) Consider a sequence of K single-server queues such that customers arrive at the first in the manner of a Poisson process, and (for each j) on completing service in the jth queue each customer moves to the (j + l)th. Service times in the jth queue are exponentially distributed with parameter p,j, with as much independence as usual. Determine the (joint) equilibrium distribution of the queue lengths, when A < p,j for all j.

12. (a) (b) (c)

13. Consider the queue M(A)IM(p,)/k, where k 2': 1. Show that a stationary distribution and only if A < kp" and calculate it in this case. Suppose that the cost of operating this system in equilibrium is

1f

exists if

00

Ak

+ B 2)n - k + l)Jl'n, n=k

the positive constants A and B representing respectively the costs of employing a server and of the dissatisfaction of delayed customers. Show that, for fixed p" there is a unique value ,1,* in the interval (0, p,) such that it is cheaper to have k = 1 than k = 2 if and only if A < ,1,*.

14. Customers arrive in a shop in the manner of a Poisson process with intensity A. They form a single queue. There are two servers, labelled 1 and 2, server i requiring an exponentially distributed 116

Exercises

Problems

[11.8.15]-[11.8.19]

time with parameter P,i to serve any given customer. The customer at the head of the queue is served by the first idle server; when both are idle, an arriving customer is equally likely to choose either. (a) Show that the queue length settles into equilibrium if and only if A < J-Ll + P,2. (b) Show that, when in equilibrium, the queue length is a time-reversible Markov chain. (c) Deduce the equilibrium distribution of queue length. (d) Generalize your conclusions to queues with many servers. 15. Consider the D(1)IM(p,)11 queue where p, > 1, and let Qn be the number of people in the queue just before the nth arrival. Let QJL be a random variable having as distribution the stationary distribution of the Markov chain {Qn}. Show that (1 - p,-l)QJL converges in distribution as p, t 1, the limit distribution being exponential with parameter 2. 16. Taxis arrive at a stand in the manner of a Poisson process with intensity r, and passengers arrive in the manner of an (independent) Poisson process with intensity 7f. If there are no waiting passengers, the taxis wait until passengers arrive, and then move off with the passengers, one to each taxi. If there is no taxi, passengers wait until they arrive. Suppose that initially there are neither taxis nor passengers at the stand. Show that the probability that n passengers are waiting at time t is 1 (7f/r) 2n e-(1f+r)t In (2t.,jirT) , where In (x) is the modified Bessel function, i.e., the coefficient of zn in the power series expansion of exp{ 1x(z

+ z-l)}.

17. Machines arrive for repair as a Poisson process with intensity A. Each repair involves two stages, the ith machine to arrive being under repair for a time Xi + Yi, where the pairs (Xi, Yi), i = 1,2, ... , are independent with a common joint distribution. Let U (t) and V (t) be the numbers of machines in the X-stage and Y-stage of repair at time t. Show that U(t) and V(t) are independent Poisson random variables. 18. Ruin. An insurance company pays independent and identically distributed claims {Kn : n :::: I} at the instants of a Poisson process with intensity A, where AE(Kl) < 1. Premiums are received at constant rate 1. Show that the maximum deficit M the company will ever accumulate has moment generating function OM (1- p)e E(e ) = A + e _ AE(eOK) . 19. (a) Erlang's loss fonnula. Consider M(A)IM(p,)/s with baulking, in which a customer departs immediately if, on arrival, he sees all the servers occupied ahead of him. Show that, in equilibrium, the probability that all servers are occupied is where p = Alp,.

(b) Consider an M(A)IM(p,)/oo queue with channels (servers) numbered 1,2,.... On arrival, a customer will choose the lowest numbered channel that is free, and be served by that channel. Show in the notation of part (a) that the fraction pc of time that channel c is busy is Pc = P(7fc-l - 7fc ) for c::::2,andpl =7fl.

117

12 Martingales

12.1 Exercises. Introduction 1.

(i) If (Y, s:) is a martingale, show that lE(Yn)

= lE(Yo) for all n.

(ii) If (Y, s:) is a submartingale (respectively supermartingale) with finite means, show that lE(Yn) :::: lE(Yo) (respectively lE(Yn ) ~ lE(Yo)). 2.

Let (Y, s:) be a martingale, and show that lE(Yn+ m 1 Tn) = Yn for all n, m ::::

o.

3. Let Zn be the size of the nth generation of a branching process with Zo = 1, having mean family size fJ., and extinction probability 1'/. Show that ZnfJ.,-n and 1'/zn define martingales. 4. Let {Sn : n :::: O} be a simple symmetric random walk on the integers with So = k. Show that Sn and S; - n are martingales. Making assumptions similar to those of de Moivre (see Example (12.1.4)), find the probability of ruin and the expected duration of the game for the gambler's ruin problem. S. Let (Y, s:) be a martingale with the property that lE(Y;) < 00 for all n. Show that, for i ~ j ~ k, lE{ (Yk - Yj) Yj} = 0, and lE{ (Yk - Yj)2 1 .'Fj) = lE(Yk' 1 .'Fj) - lE(Y/1 .'Fj). Suppose there exists K such that lE(Y;) ~ K for all n. Show that the sequence {Yn } converges in mean square as n -+

00.

6. Let Y be a martingale and let u be a convex function mapping JR to JR. Show that {u(Yn) : n :::: O} is a submartingale provided that lE(u(Yn)+) < 00 for all n. Show that 1Yn I, tions are satisfied.

y;, and Y;i constitute submartingales whenever the appropriate moment condi-

7. Let Y be a submartingale and let u be a convex non-decreasing function mapping JR to JR. Show that {u(Yn) : n :::: O} is a submartingale provided that lE(u(Yn)+) < 00 for all n. Show that (subject to a moment condition) Y;i constitutes a submartingale, but that 1Yn 1 and Y; need not constitute submartingales. 8. Let X be a discrete-time Markov chain with countable state space S and transition matrix P. Suppose that 1{! : S -+ JR is bounded and satisfies "L-jEs Pij1{!(j) ~ ),,1{!(i) for some)" > 0 and all i E S. Show that)" -n1{!(X n ) constitutes a supermartingale. 9. Let G n (s) be the probability generating function of the size Zn of the nth generation of a branching process, where Zo = 1 and var(Zd > O. Let Hn be the inverse function of the function G n , viewed as a function on the interval [0, 1], and show that Mn = {Hn (s) }Zn defines a martingale with respect to the sequence Z.

118

Exercises

Crossings and convergence

[12.2.1]-[12.3.4]

12.2 Exercises. Martingale differences and Hoeffding's inequality 1.

Knapsack problem. It is required to pack a knapsack to maximum benefit. S1,lppose you have n objects, the ith object having volume Vi and worth Wi, where VI, V2, ... , Vn , WI, W2, ... , Wn are independent non-negative random variables with finite means, and Wi :s M for all i and some fixed M. Your knapsack has volume c, and you wish to maximize the total worth of the objects packed in it. That is, you wish to find the vector ZI, Z2, ... ,Zn of o's and 1's such that"L:'i Zi Vi :s c and which maximizes "L:'i Zi Wi. Let 2 be the maximal possible worth of the knapsack's contents, and show that lP'(12 - E21 2: x) :s 2exp{-x 2/(2nM2)} for x> o. 2. Graph colouring. Given n vertices VI, V2, •.. , V n , for each 1 :s i < j :s n we place an edge between Vi and Vj with probability p; different pairs are joined independently of each other. We call Vi and Vj neighbours if they are joined by an edge. The chromatic number X of the ensuing graph is the minimal number of pencils of different colours which are required in order that each vertex may be coloured differently from each of its neighbours. Show that lP'(lx - Ex I 2: x) :s 2 exp{ - ix2/n} for x> o.

12.3 Exercises. Crossings and convergence 1.

Give a reasonable definition of a downcros sing of the interval [a, b] by the raodom sequence

Yo, YI,····

(a) Show that the number of downcrossings differs from the number of upcrossings by at most 1. (b) If (Y, :F) is a submartingale, show that the number Dn (a, b; Y) of downcrossings of [a, b] by Y up to time n satisfies E{(Y - b)+} EDn(a,b;Y):s ;-a . 2. Let (Y, :F) be a supermartingale with finite means, and let Un (a, b; Y) be the number of upcrossings of the interval [a, b] up to time n. Show that

EUn(a,b;y):s Deduce that EUn (a, b; Y)

:s a/ (b -

E{(Y -a)-} ;-a .

a) if Y is non-negative and a 2: o.

3. Let X be a Markov chain with countable state space S and transition matrix P. Suppose that X is irreducible and persistent, and that 1/1 : S -+ S is a bounded function satisfying "L: j ES Pij 1/1 (j) :s 1/1 (i) for i E S. Show that 1/1 is a constant function. 4.

Let 2 I, 22, . .. be independent random variables such that: with probability 1n-2, with probability 1 - n- 2 , with probability 1n-2,

where al = 2 and an = 4 "L:j:t aj. Show that Yn = "L:}=12j defines a martingale. Show that Y = lim Yn exists almost surely, but that there exists no M such that EI Yn I :s M for all n.

119

[12.4.1]-[12.5.5]

Exercises

Martingales

12.4 Exercises. Stopping times 1. If TI and T2 are stopping times with respect to a filtration !F, show that TI and min{TI, T2} are stopping times also.

+ T2, max{TI, T2},

2. Let XI, X2,'" be a sequence of non-negative independent random variables and let N(t) = max{n : X I + X2 + ... + Xn ::: t}. Show that N(t) + 1 is a stopping time with respect to a suitable filtration to be specified. 3.

Let (Y, :J:) be a submartingale and x > O. Show that

m x) ::: .!..JB:(Y n+). x

lP' ( max Y 2: O::;:m::;:n

4.

Let (Y, :J:) be a non-negative supermartingale and x > O. Show that

m x) ::: .!..JB:(Yo). x

lP' ( max Y 2: O::;:m::;:n

5. Let (Y, :J:) be a submartingale and let Sand T be stopping times satisfying 0 ::: S ::: T ::: N for some deterministic N. Show that JB:Yo ::: JB:Ys ::: JB:YT ::: JB:YN. 6. Let {Sn} be a simple random walk with So = 0 such that 0 < P = lP'(SI = 1) < ~. Use de Moivre's martingale to show that JB:(suPm Sm) ::: p/(l - 2p). Show further that this inequality may be replaced by an equality. 7. Let 3='be a filtration. For any stopping time T with respect to !F, denote by TT the collection of all events A such that, for all n, A n {T ::: n} E Tn. Let Sand T be stopping times. (a) Show that TT is a a-field, and that T is measurable with respect to this a-field. (b) If A E Ts, show that A n {S ::: T} E TT. (c) Let Sand T satisfy S ::: T. Show that Ts ~ TT.

12.5 Exercises. Optional stopping 1. Let (Y, :J:) be a martingale and T a stopping time such that lP'(T < 00) = 1. Show that JB:(YT) JB:(Yo) if either of the following holds: (a) JB:(suPn IYT An J) < 00,

=

(b) JB:(IYT AnIIH) ::: c for some c, .5 > 0 and all n.

2. Let (Y, :J:) be a martingale. Show that (YT An, Tn) is a uniformly integrable martingale for any finite stopping time T such that either: (a) JB:IYT I < 00 and JB:(IYn II(T>n)) -+ 0 as n -+ (b) {Yn } is uniformly integrable.

00,

or

3. Let (Y, :J:) be a uniformly integrable martingale, and let S and T be finite stopping times satisfying S ::: T. Prove that YT = JB:(Y00 I TT) and that Ys = JB:(YTITs), where Y00 is the almost sure limit asn -+ ooofYn . 4. Let {Sn : n 2: O} be a simple symmetric random walk with 0 < So < N and with absorbing barriers at 0 and N. Use the optional stopping theorem to show that the mean time until absorption is JB:{SO(N - So)}· 5.

Let {Sn : n 2: O} be a simple symmetric random walk with So = O. Show that

Yn =

cos{),,[Sn - ~(b - a)]} (cos )..)n

----~--~------

120

Exercises

Problems

[12.5.6]-[12.9.2]

constitutes a martingale if cos A #- 0. Let a and b be positive integers. Show that the time T until absorption at one of two absorbing barriers at -a and b satisfies lE ({COSA}

-T)

=

cos{1A(b-a)} cos{1A(b + a)}

,

0< A <

_Jr_.

b+a

6. Let {Sn : n 2: o} be a simple symmetric random walk on the positive and negative integers, with So = 0. For each of the three following random variables, determine whether or not it is a stopping time and find its mean: U

= min{n 2: 5: Sn = Sn-5 + 5},

V

=U

- 5,

W

= min{n : Sn =

I}.

LetSn = a+ ~~=1 Xr be a simple symmetric random walk. The walk stops at the earliesttime T when it reaches either of the two positions Oor K where < a < K. Show thatMn = ~~=o Sr - ~S~

7.

is a martingale and deduce that lE(~;=o Sr)

= ~(K2

°

- a 2)a + a.

8.

Gambler's ruin. Let Xi be independent random variables each equally likely to take the values ±1, and let T = min{n : Sn E (-a,b)). Verify the conditions of the optional stopping theorem (12.5.1) for the martingale S~ - n and the stopping time T.

12.7 Exercises. Backward martingales and continuous-time martingales Let X be a continuous-time Markov chain with finite state space S and generator G. Let 1/ = (1'}(i) : i E S) be a root of the equation G1/' = O. Show that 1'}(X (t)) constitutes a martingale with respect to:Fi = a({X(u) : u ::: tl).

1.

Let N be a Poisson process with intensity A and N(O) = 0, and let Ta = min{t : N(t) = a}, where a is a positive integer. Assuming that lE{exp(1/ITa)} < 00 for sufficiently small positive 1/1, use the optional stopping theorem to show that var(Ta) = aA -2.

2.

3. Let Sm = ~~1 X r , m ::: n, where the Xr are independent and identically distributed with finite mean. Denote by U 1, U2, ... , Un the order statistics of n independent variables which are uniformly distributed on (0, t), and set Un +l = t. Show that Rm = Sm/Um+l, 0::: m ::: n, is a backward martingale with respect to a suitable sequence of a-fields, and deduce that

I

lP'(Rm 2: 1 for some m::: n Sn =

y) ::: min{y/t, 1}.

12.9 Problems 1. Let Zn be the size of the nth generation of a branching process with immigration in which the mean family size is J-( (#- 1) and the mean number of immigrants per generation is m. Show that

Yn

= J-(-n { Zn

1- m - -J-(n} 1-J-(

defines a martingale. 2. In an age-dependent branching process, each individual gives birth to a random number of offspring at random times. At time 0, there exists a single progenitor who has N children at the subsequent

121

[12.9.3]-[12.9.8]

Exercises

Martingales

times B1 ::: B2 ::: ... ::: BN; his family may be described by the vector (N, B1, B2,"" BN)' Each subsequent member x of the population has a family described similarly by a vector (N(x), B1 (x), ... , BN(x) (x» having the same distribution as (N, B1, ... , BN) and independent of all other individuals' families. The number N (x) is the number of his offspring, and Bi (x) is the time between the births of the parent and the ith offspring. Let {Bn,r : r 2: I} be the times of births of individuals in the nth 2:r e-OBn,r, and show that Yn Mn (8)jlE(M1 (8»n defines a martingale generation. Let Mn (8) with respect to :Fn = a({Bm,r : m ::: n, r 2: In, for any value of 8 such that lEM1 (8) < 00.

=

3.

=

Let (Y, :F) be a martingale with lEYn

= 0 and lE(Y;)

<

00

for all n. Show that

x> O.

4. Let (Y, :F) be a non-negative submartingale with Yo = 0, and let {en} be a non-increasing sequence of positive numbers. Show that

x> O. Such an inequality is sometimes named after subsets of Hajek, Renyi, and Chow. Deduce Kolmogorov's inequality for the sum of independent random variables. [Hint: Work with the martingale Zn = cnYn - 2: =l qlE(Xk I :Fk-1) + 2: =l (Ck-1 - q)Yk-1 where Xk = Yk - Yk-1·]

k

k

Supposethatthesequence{Xn: n 2: I} of random variables satisfies lE(Xn I X1,X2,· .. ,Xn -1) = 0 for all n, and also 2:~llE(IXk < 00 for some r E [1,2]. Let Sn = =1 Zi where Zi = Xii i, and show that

5.

nj e

2:1

x> O.

2:7

Deduce that Sn converges a.s. as n -+ 00, and hence that n- 1 Xk ~ O. [Hint: In the case 1 < r::: 2, prove and use thefactthath(u) = lulr satisfiesh(v)-h(u)::: (v-u)h'(u)+2h«v-u)j2). Kronecker's lemma is useful for the last part.]

6.

Let Xl, X2, ... be independent random variables with

Xn =

Let Y1

{

I

with probability (2n)-1,

0

with probability 1 - n- 1 ,

-1

with probability (2n)-1.

= Xl and for n 2: 2 Yn

={

Xn

if Yn-1

= 0,

nYn -1lX n l

ifYn -1

#0.

Show that Yn is a martingale with respect to :Fn = a(Y1, Y2, ... , Yn). Show that Yn does not converge almost surely. Does Yn converge in any way? Why does the martingale convergence theorem not apply? 7. Let Xl, X2, ... be independent identically distributed random variables and suppose that M(t) = lE(e tXj ) satisfies M(t) = 1 for some t > O. Show that lP'(Sk 2: x for some k) ::: e- tx for x > 0 and such a value of t, where Sk = Xl + X2 + ... + Xk. 8. Let Zn be the size of the nth generation of a branching process with family-size probability generating function G(s), and assume Zo = 1. Let ~ be the smallest positive root of G(s) = s.

122

Problems

Exercises

Use the martingale convergence theorem to show that, if 0 < lP'(Zn -+ (0) = 1-~. 9.

Let (Y, :F) be a non-negative martingale, and let Y;

~

= max{Yk

[Hint: alog+ b.:s alog+ a + ble if a, b 2:: 0, where log+ x

[12.9.9]-[12.9.14]

< 1, then lP'(Zn -+ 0) =

:0

.:s k .:s n}.

~

and

Show that

= max{O, 10gx}.J

10. Let X = {X(t) : t 2:: O} be a birth-death process with parameters Ai, J1,i, where Ai only if i = O. Define h(O) = 0, h(1) = 1, and

=

0 if and

Show that h(X(t)) constitutes a martingale with respect to the filtration :Ft = a({X (u) : 0 .:s u .:s tl), whenever lEh(X(t)) < 00 for all t. (You may assume that the forward equations are satisfied.) Fix n, and let m < n; let n(m) be the probability thatthe process is absorbed at 0 before it reaches size n, having started at size m. Show that n(m) = 1 - {h(m)1 h(n)}. 11. Let (Y, !F) be a submartingale such that lE(Y:)

.:s

M for some M and all n.

(a) Show that Mn = limm~oo lE(Y:+ m I :Fn ) exists (almost surely) and defines a martingale with respect to !F. (b) Show that Yn may be expressed in the form Yn = Xn - Zn where (X, !F) is a non-negative martingale, and (Z, :F) is a non-negative supermartingale. This representation of Y is sometimes termed the 'Krickeberg decomposition'. (c) Let (Y,:F) be a martingale such that lElYnl .:s M for some M and all n. Show that Y may be expressed as the difference of two non-negative martingales. 12. Let £Yn be the assets of an insurance company after n years of trading. During each year it receives a total (fixed) income of £ P in premiums. During the nth year it pays out a total of £C n in claims. Thus Yn+1 = Yn + P - Cn+1. Suppose that C1, C2, ... are independent N(J1" a 2) variables and show that the probability of ultimate bankruptcy satisfies

lP'(Yn

.:s 0 for some n) .:s exp { -

2(P - J1,)Yo} a2 .

13. Polya's urn. A bag contains red and blue balls, with initially r red and b blue where rb > O. A ball is drawn from the bag, its colour noted, and then it is returned to the bag together with a new ball of the same colour. Let Rn be the number of red balls after n such operations. (a) Show that Yn = Rnl(n + r + b) is a martingale which converges almost surely and in mean. (b) Let T be the number of balls drawn until the first blue ball appears, and suppose that r = b = 1. Show that lE{(T + 2)-1} =

t.

(c) Suppose r = b = 1, and show that lP'(Yn 2:: ~ for some n)

.:s

~.

14. Here is a modification of the last problem. Let {An: n 2:: I} be a sequence of random variables, each being a non-negative integer. We are provided with the bag of Problem (12.9.13), and we add balls according to the following rules. At each stage a ball is drawn from the bag, and its colour noted; we assume that the distribution of this colour depends only on the current contents of the bag and not on any further information concerning the An. We return this ball together with An new balls of the same colour. Write Rn and Bn for the numbers of red and blue balls in the urn after n operations, and

123

[12.9.15]-[12.9.19] let:Fn Ro

=

Exercises

Martingales

= Rn/(Rn + Bn) defines a martingale. Suppose be the number of balls drawn until the first blue ball appears, and show that

a ({Rk, Bk : 0 ::: k ::: n}). Show that Yn

= Bo = 1, let T

E

so long as 2:n(2

(

) 1 + AT T 2+ 2:i=l Ai

1 =-, 2

+ 2:;=1 Ai)-l = 00 a.s.

15. Labouchere system. Here is a gambling system for playing a fair game. Choose a sequence Xl, x2, ... ,Xn of positive numbers.

Wager the sum of the first and last numbers on an evens bet. If you win, delete those two numbers; if you lose, append their sum as an extra term xn+l (= Xl + xn) at the right-hand end of the sequence. You play iteratively according to the above rule. If the sequence ever contains one term only, you wager that amount on an evens bet. If you win, you delete the term, and if you lose you append it to the sequence to obtain two terms. Show that, with probability 1, the game terminates with a profit of 2:7 Xi, and that the time until termination has finite mean. This looks like another clever strategy. Show that the mean size of your largest stake before winning is infinite. (When Henry Labouchere was sent down from Trinity College, Cambridge, in 1852, his gambling debts exceeded £6000.)

16. Here is a martingale approach to the question of determining the mean number of tosses of a coin before the first appearance of the sequence HHH. A large casino contains infinitely many gamblers Gl, G2, ... , each with an initial fortune of $1. A croupier tosses a coin repeatedly. For each n, gambler G n bets as follows. Just before the nth toss he stakes his $1 on the event that the nth toss shows heads. The game is assumed fair, so that he receives a total of $ p -1 if he wins, where p is the probability of heads. If he wins this gamble, then he repeatedly stakes his entire current fortune on heads, at the same odds as his first gamble. At the first subsequent tail he loses his fortune and leaves the casino, penniless. Let Sn be the casino's profit (losses count negative) after the nth toss. Show that Sn is a martingale. Let N be the number of tosses before the first appearance of HHH; show that N is a stopping time and hence find E(N). Now adapt this scheme to calculate the mean time to the first appearance of the sequence HTH.

17. Let {(Xk, Yk) : k 2:: I} be a sequence of independent identically distributed random vectors such that each Xk and Yk takes values in the set {-I, 0,1,2, ... }. Suppose that E(Xl) = E(Yl) = 0 and E(XlYl) = e, and furthermore Xl and Yl have finite non-zero variances. Let Uo and Vo be positive integers, and define (Un+l' Vn+l) = (Un, Vn ) + (Xn+l' Yn+l) for each n 2:: O. Let T = min{n : Un Vn = O} be the first hitting time by the random walk (Un, Vn ) of the axes of ]E.2. Show that E(T) < 00 if and only if e < 0, and that E(T) = -E(Uo Vo)/e in this case. [Hint: You might show that Un Vn - en is a martingale.] 18. The game 'Red Now' may be played by a single player with a well shuffled conventional pack of 52 playing cards. At times n = 1, 2, ... , 52 the player turns over a new card and observes its colour. Just once in the game he must say, just before exposing a card, "Red Now". He wins the game if the next exposed card is red. Let Rn be the number of red cards remaining face down after the nth card has been turned over. Show that Xn = R n /(52 - n), 0 ::: n < 52, defines a martingale. Show that there is no strategy for the player which results in a probability of winning different from ~.

19. A businessman has a redundant piece of equipment which he advertises for sale, inviting "offers over £ 1000". He anticipates that, each week for the foreseeable future, he will be approached by one prospective purchaser, the offers made in week 0,1, ... being £1000Xo, £ 1OOOX 1, ... , where Xo, Xl, ... are independent random variables with a common density function f and finite mean. Storage of the equipment costs £ lOOOe per week and the prevailing rate of interest is a (> 0) per

124

Exercises

Problems

[12.9.20]-[12.9.24]

week. Explain why a sensible strategy for the businessman is to sell in the week T, where T is a stopping time chosen so as to maximize

{L(T)

=

E{

T

(l

+ a)-T XT

-

L (l + a)-nc }. n=1

Show that this problem is equivalent to maximizing E{(1 + a)-T ZT} where Zn = Xn Show that there exists a unique positive real number y with the property that ay =

+ cia.

lOO lP'(Zn > y)dy,

and that, for this value of y, the sequence Vn = (l + a) -n max {Zn, y} constitutes a supermartingale. Deduce that the optimal strategy for the businessman is to set a target price r (which you should specify in terms of y) and sell the first time he is offered at least this price. In the case when f(x) = 2x- 3 for x 2: 1, and c = a = 910' find his target price and the expected number of weeks he will have to wait before selling. 20. Let Z be a branching process satisfying Zo = 1, E(ZI) < 1, and lP'(ZI 2: 2) > O. Show that E(suPn Zn) ~ 1)/(1) - 1), where 1) is the largest root of the equation x = G(x) and G is the probability generating function of Z 1. 21. Matching. In a cloakroom there are K coats belonging to K people who make an attempt to leave by picking a coat at random. Those who pick their own coat leave, the rest return the coats and try again at random. Let N be the number of rounds of attempts until everyone has left. Show that EN = K and var(N) ~ K. 22. Let W be a standard Wiener process, and define M(t) =

lW(U)dU-~W(t)3.

Show that M(t) is a martingale, and deduce that the expected area under the path of W until it first reaches one of the levels a (> 0) or b « 0) is - ~ab(a + b). 23. Let W = (WI, W2,"" Wd) be a d-dimensional Wiener process, the Wi being independent one-dimensional Wiener processes with Wi (0) = 0 and variance parameter a 2 = d- 1 . Let R(t)2 = WI (t)2 + W2(t)2 + ... + Wd(t)2, and show that R(t)2 - t is a martingale. Deduce that the mean time to hit the sphere of ~d with radius a is a 2 . 24. Let W be a standard one-dimensional Wiener process, and let a, b > O. Let T be the earliest time at which W visits either of the two points -a, b. Show that lP'(W(T) = b) = a/(a + b) and E(T) = abo In the case a = b, find E(e- sT ) for s > O.

125

13 Diffusion processes

13.3 Exercises. Diffusion processes 1. Let X = {X (t) : t 2: O} be a simple birth-death process with parameters An Suggest a diffusion approximation to X.

= nA and Jtn = nJt.

2. Bartlett's equation. Let D be a diffusion with instantaneous mean and variance aCt, x) and bet, x), and let M(t, e) = E(e 8D (t)), the moment generating function of D(t). Use the forward diffusion equation to derive Bartlett's equation:

-aM = ea at

(a ) t ' ae

M

a)

I 2b ( t + -e 2 ' ae

M

where we interpret

if get, x) = 2:~o Yn(t)x n . 3. Write down Bartlett's equation in the case of the Wiener process D having drift m and instantaneous variance 1, and solve it subject to the boundary condition D(O) = o. 4. Write down Bartlett's equation in the case of an Omstein-Uhlenbeck process D having instantaneous mean aCt, x) = -x and variance bet, x) = 1, and solve it subject to the boundary condition D(O) = O. 5. Bessel process. If WI (t), W2(t), W3(t) are independent Wiener processes, then R(t) defined as R2 = Wt + Wi + Wl is the three-dimensional Bessel process. Show that R is a Markov process. Is this result true in a general number n of dimensions? 6.

Show that the transition density for the Bessel process defined in Exercise (5) is J(t, y I s, x)

=

a

a/'(R(t) ::'S y I R(s)

= ~2:/rx_ s)

{exp ( -

= x)

~t-_x;~) -

exp

(--~-(t+-_x;-~) _ }.

7. If W is a Wiener process and the function g : lR ---+ lR is continuous and strictly monotone, show that g(W) is a continuous Markov process. 8.

Let W be a Wiener process. Which of the following define martingales? (b) cW(t/c 2 ), (c) tW(t) W(s)ds.

(a) errw(t),

JJ

126

Exercises

Stoclwstic calculus

9.

[13.3.9]-[13.7.1]

Exponential martingale, geometric Brownian motion. Let W be a standard Wiener process

and define S(t)

= eat+bW(t).

Show that:

(a) S is a Markov process, (b) S is a martingale (with respect to the filtration generated by W) if and only if a in this case JE(S(t» = 1.

+ 1b2 = 0, and

10. Find the transition density for the Markov process of Exercise (9a).

13.4 Exercises. First passage times 1. Let W be a standard Wiener process and let X(t) = exp{iBW(t) + 1B 2 t} where i = that X is a martingale with respect to the filtration given by :Fr = a ({ W (u) : u ::: t D.

.J=T. Show

2. Let T be the (random) time at which a standard Wiener process W hits the 'barrier' in space-time given by y = at+b where a < 0, b ::: 0; that is, T = inf{t : W(t) = at+b}. Use the result of Exercise (1) to show that the moment generating function ofT is given byJE(e VrT ) = exp{ -b( a 2 - 21/1+a)}

v'

for

1/1 < ~a2.

You may assume that the conditions of the optional stopping theorem are satisfied.

3. Let W be a standard Wiener process, and let T be the time of the last zero of W prior to time t. Show that IP'(T ::: u) = (2/n) sin- 1 JU1t, 0::: u ::: t.

13.5 Exercise. Barriers 1. Let D be a standard Wiener process with drift m starting from D(O) = d > 0, and suppose that there is a reflecting barrier at the origin. Show that the density function F(t, y) of D(t) satisfies F(t, y) -+ 0 as t -+ 00 ifm ::: 0, whereas F(t, y) -+ 2lmle-2lmly for y > 0, as t -+ 00 ifm < O.

13.6 Exercises. Excursions and the Brownian bridge 1.

Let W be a standard Wiener process. Show that the conditional density function of W (t), given

that W(u) > 0 for 0 < u < t, is g(x)

= (x It)e- X2 /(2t), x

> O.

2. Show that the autocovariance function of the Brownian bridge is c(s, t) = rnin{s, t} - st, 0 ::: s,t:::1. 3. Let WbeastandardWienerprocess, andletW(t) = W(t)-tW(1). Show that {W(t) : 0::: t::: I} is a Brownian bridge. 4. If W is a Wiener process with W(O) = 0, show that W(t) = (1 - t)W(t 1(1 - t» for 0 ::: t < I, W(1) = 0, defines a Brownian bridge. 5. Let 0 < s < t < I. Show that the probability that the Brownian bridge has no zeros in the interval (s, t) is (2/n) cos- 1 -J(t - s)/[t(1 - s)].

13.7 Exercises. Stochastic calculus 1.

Doob's L2 inequality. Let W be a standard Wiener process, and show that

127

[13.7.2]-[13.8.6]

Exercises

Diffusion processes

2. Let W be a standard Wiener process. Fix t > 0, n 2: 1, and let 8 = tin. Show that Zn = I:j:-J (W(j+I)c5 - Wjc5)2 satisfies Zn --+ t in mean square as n --+ 00. 3. Let W be a standard Wiener process. Fix t > 0, n 2: 1, and let 8 11j = VJ+ 1 - Vj. Evaluate the limits of the following as n --+ 00:

= tin.

Let"J

= Wjc5

and

(a) h(n) =I:j Vj I1j ,

(b) h(n) (c) h(n)

= I:j VJ+ll1j, = I:j ~(VJ+I + Vj)l1j,

(d) 14(n) =

I:j

W(J+!)c5l1j.

4. Let W be a standard Wiener process. Show that U(t) Omstein-Uhlenbeck process.

5.

Let W be a standard Wiener process. Show that Ut Omstein-Uhlenbeck process.

=

=

e-lltW(e2Ilt) defines a stationary

W t - fJ

16 e-Il(t-s)ws ds defines an

13.8 Exercises. The Ito integral In the absence of any contrary indication, W denotes a standard Wiener process, and :Ft is the smallest a-field containing all null events with respect to which every member of {Wu : 0 ::: u ::: t} is measurable.

1.

(a) Verify directly that fot s dWs

(b) Verify directly that fot W; dWs (c) Show that lE ([fot Ws dWs

f)

= tWt = ~ W?

=

fot Ws ds.

- fot Ws ds.

fot lE(wh ds.

16

2. Let X t = Ws ds. Show that X is a Gaussian process, and find its autocovariance and autocorrelation function. 3.

Let (Q, :F, JP') be a probability space, and suppose that Xn ~ X as n --+

that lE(Xn

4.

I f}.) ~

lE(X

00.

If f}. £ :F, show

I f}.).

Let 1/11 and 1/12 be predictable step functions, and show that

whenever both sides exist. Assuming that Gaussian white noise G t = dWt/dt exists in sufficiently many senses to appear as an integrand, show by integrating the stochastic differential equation dX t = -fJXt dt + dWt that

5.

if Xo 6.

= o.

Let 1/1 be an adapted process with 111/111 <

00.

Show that II I (1/1) 112 = 111/111·

128

Exercises

Option pricing

[13.9.1]-[13.10.1]

13.9 Exercises. Ito's formula In the absence of any contrary indication, W denotes a standard Wiener process, and Fi is the smallest a-field containing all null events with respect to which every member of (Wu : 0 :::: u :::: t} is measurable.

1.

Let X and Y be independent standard Wiener processes. Show that, with Rl = xl Zt

=

+ Y?,

t Xs + lot -dYs Y Rs Rs loo -dXs s

0

is a Wiener process. [Hint: Use Theorem (13.8.13).] Hence show that R2 satisfies Rl = 2

lot Rs d Ws + 2t.

Generalize this conclusion to n dimensions. 2. Write down the SDE obtained via Ito's formula for the process Yt = Wt4 , and deduce that E(Wf) = 3t 2 .

3.

Show that Yt = t Wt is an Ito process, and write down the corresponding SDE.

4. Wiener process on a circle. Let Yt = e iW1 • Show that Y = Xl circle satisfying

Find the SDEs satisfied by the processes: (a) X t = Wt/(1 + t), (b) X t = sin Wt, (c) [Wiener process on an ellipse] X t = a cos Wf, Yt

+ i X2 is a process on the unit

5.

= b sin Wt, where ab =1= O.

13.10 Exercises. Option pricing In the absence of any contrary indication, W denotes a standard Wiener process, and Fi is the smallest a-field containing all null events with respect to which every member of (Wu : 0 :::: u :::: t} is measurable. The process St = exp((JL -1a2) + a Wt) is a geometric Brownian motion, and r :::: 0 is the interest rate.

1.

(a) Let Z have the N (y, r2) distribution. Show that E ( (ae

Z

- K)

y)

+) = ae r+lr2 (lOg(a/ K) + + r "' r

- K

(IOg(a/ K) +

y)

r

where is the N(O, 1) distribution function. (b) Let IQi be a probability measure under which a W is a Wiener process with drift r - JL and instantaneous variance a 2 . Show for 0 :::: t :::: T that

where dl(t,X)

=

log(x/ K)

+ (r + 1a 2 )(T a

~ T - t

t)

129

,

d2(t,x)=dl(t,x)-a~.

[13.10.2]-[13.12.2]

Exercises

Diffusion processes

2. Consider a portfolio which, at time t, holds ~ (t, S) units of stock and 1/1 (t, S) units of bond, and assume these quantities depend only on the values of Su for 0 ~ u ~ t. Find the function 1/1 such that the portfolio is self-financing in the three cases: (a)

~(t,

S) = I for all t, S,

(b) ~(t, S) = St, (c)

~(t, S) = fot Sv dv.

3. Suppose the stock price St is itself a Wiener process and the interest rate r equals 0, so that a unit of bond has unit value for all time. In the notation of Exercise (2), which of the following define self-financing portfolios? (a)

~(t,

S)

= 1/I(t, S) =

I for all t, S,

-sl- t, -t, 1/I(t, S) = f6 Ss ds, f6 Ss ds, 1/I(t, S) = - f6 S}ds.

(b) ~(t, S) = 2St , 1/I(t, S) = (c) ~(t, S) =

(d) ~(t, S) =

4. An 'American call option' differs from a European call option in that it may be exercised by the buyer at any time up to the expiry date. Show that the value of the American call option is the same as that of the corresponding European call option, and that there is no advantage to the holder of such an option to exercise it strictly before its expiry date. 5. Show that the Black-Scholes value at time 0 of the European call option is an increasing function of the initial stock price, the exercise date, the interest rate, and the volatility, and is a decreasing function of the strike price.

13.11 Exercises. Passage probabilities and potentials Let G be the closed sphere with radius E and centre at the origin of lE.d where d 2': 3. Let W be a d-dimensional Wiener process starting from W (0) = w ¢:. G. Show that the probability that W visits Gis (E/r)d-2, where r = Iwl.

1.

2. Let G be an infinite connected graph with finite vertex degrees. Let L!!.n be the set of vertices x which are distance n from 0 (that is, the shortest path from x to 0 contains n edges), and let Nn be the total number of edges joining pairs x, y of vertices with x E L!!.n, y E L!!.n+l' Show that a random walk on G is persistent if 2:i N i- 1 = 00. 3. Let G be a connected graph with finite vertex degrees, and let H be a connected subgraph of G. Show that a random walk on H is persistent if a random walk on G is persistent, but that the converse is not generally true.

13.12 Problems 1. Let W be a standard Wiener process, that is, a process with independent increments and continuous sample paths such that W(s + t) - W(s) is N(O, t) for t > O. Let ex be a positive constant. Show that: (a) exW(t/ex 2 ) is a standard Wiener process, (b) Wet + ex) - W(ex) is a standard Wiener process, (c) the process V, given by Vet) = tW(1/t) for t > 0, V(O) = 0, is a standard Wiener process. 2. Let X = {X(t) : t 2': O} be a Gaussian process with continuous sample paths, zero means, and autocovariance function c(s, t) = u(s)v(t) for s ~ t where u and v are continuous functions. Suppose

130

Exercises

Problems

[13.12.3]-[13.12.9]

that the ratio r(t) = u(t)/v(t) is continuous and strictly increasing with inverse function r-I. Show that W(t) = X (r- I (t))/v(r- I (t)) is a standard Wiener process on a suitable interval of time. If c(s, t) = s(I - t) for s :s t < 1, express X in terms of W.

3. Let fJ > 0, and show that U(t) standard Wiener process.

= e-fttW(e2ftt -

1) is an Omstein-Uhlenbeck process if W is a

Let V = (V(t) : t ::: O} be an Omstein-Uhlenbeck process with instantaneous mean a(t, x) = -fJx where fJ > 0, with instantaneous variance b(t, x) = (J"2, and with U(O) = u. Show that V(t) is N(ue- ftt ,(J"2(I - e- 2ftt )/(2fJ)). Deduce that V(t) is asymptotically N(O, ~(J"2/fJ) as t -+ 00, and 4.

show that V is strongly stationary if V (0) is N (0, ~(J"2 / fJ). Show that such a process is the only stationary Gaussian Markov process with continuous autocovariance function, and find its spectral density function.

5. Let D = (D(t) : t ::: O} be a diffusion process with instantaneous mean a(t, x) = ax and instantaneous variance b(t, x) = fJx where a and fJ are positive constants. Let D(O) = d. Show that the moment generating function of D(t) is 2adBeat } M(t, B) = exp { fJB(I _ eat) + 2a .

Find the mean and variance of D(t), and show that lP'(D(t) = 0) -+ e- 2da / ft as t -+ 00.

6. Let D be an Omstein-Uhlenbeck process with D(O) = 0, and place reflecting barriers at -c and d where c, d > O. Find the limiting distribution of D as t -+ 00. 7.

Let Xo, X j, ... be independent N (0, 1) variables, and show that

W(t)

=

t

-Xo

.jJi

+ ff~ - L.. -sin(kt) -Xk rr k=1

k

defines a standard Wiener process on [0, rr].

8.

Let W be a standard Wiener process with W (0) = O. Place absorbing barriers at -b and b, where b > 0, and let W a be W absorbed at these barriers. Show that Wa(t) has density function

fa(y,t)=

1

00

{;c:

{

L(-llexp

(y _ 2kb)2 } 2t '

-b < y < b,

y2rrt k=-oo

which may also be expressed as . (nrr(y + b)) fa(y, t) = ~ L.. ane- A nt sm 2b '

-b < y < b,

n=1

where an = b- I sin(~nrr) and An = n 2rr2/(8b 2 ). Hence calculate lP'(suPo:::;s::;rIW(s)1 > b) for the unrestricted process W. 9. Let D be a Wiener process with drift m, and suppose that D(O) = O. Place absorbing barriers at the points x = -a and x = b where a and b are positive real numbers. Show that the probability Pa that the process is absorbed at -a is given by e2mb - 1 Pa = e2m(a+b) _ 1 .

131

[13.12.10]-[13.12.18]

Diffusion processes

Exercises

10. Let W be a standard Wiener process and let F (u, v) be the event that W has no zero in the interval (u, v).

I

(a) If ab > 0, show that JP>(F(O, t) W(O) = a, W(t) = b) = I - e- 2ab / t . (b) If W(O) = 0 and 0 < to :::: tl :::: t2, show that

(c) Deduce that, if W(O) = 0 and 0 < tl :::: t2, then JP>(F(O, t2)

I F(O, tl))

= ./t1lt2.

11. Let W be a standard Wiener process. Show that

sup IW(s)1 2':

JP> (

O::;s::;t

Set t = 2n and w t ~ 00.

=

w) : : 2JP>(jW(t)1 2': w):::: 2~

for w > O.

w

22n/3 and use the Borel-Cantelli lemma to show that t- 1 W(t) ~ 0 a.s. as

12. Let W be a two-dimensional Wiener process with W(O) = w, and let F be the unit circle. What is the probability that W visits the upper semicircle G of F before it visits the lower semicircle H? 13. Let WI and W2 be independent standard Wiener processes; the pair W(t) = (WI (t), W2(t)) represents the position of a particle which is experiencing Brownian motion in the plane. Let 1 be some straight line in JR2, and let P be the point on 1 which is closest to the origin O. Draw a diagram. Show that (a) the particle visits I, with probability one, (b) if the particle hits 1 for the first time at the point R, then the distance PR (measured as positive or negative as appropriate) has the Cauchy density function f(x) = df(JT(d 2 +x2)}, -00 < x < 00, where d is the distance OP, (c) the angle PaR is uniformly distributed on [-1JT, 1JT]. 14. Let cfJ(x + iy) = u(x, y) + iv(x, y) be an analytic function on the complex plane with real part u(x, y) and imaginary part v(x, y), and assume that

Let (WI, W2) be the planar Wiener process of Problem (l3) above. Show that the pair U(Wl, W2), V(Wl, W2) is also a planar Wiener process. 15. Let M(t) = maxO<s
132

where (xi : I :::: j :::: n) is a given j :::: n), let Ar be the number of strictly positive, and let Br be the :::: n) first occurs at the rth place.

Exercises

Problems

[13.12.19]-[13.12.24]

19. Arc sine laws. For the standard Wiener process W, let A be the amount of time u during the time interval [0, t] for which W(u) > 0; let L be the time of the last visit to the origin before t; and let R be the time when W attains its maximum in [0, t]. Show that A, L, and R have the same distribution function F(x) = (2jJT) sin-1,JXTi for 0 :s x :s t. [Hint: Use the results of Problems (13.12.15)-(13.12.18).] 20. Let W be a standard Wiener process, and let Ux be the amount of time spent below the level x (::: 0) during the time interval (0, 1), that is, U x = I{W(t)<xj dt. Show that U x has density function

IJ

1

lux (u) = JT ,vu(1 Show also that sup(t

Vx = { 1

(

_ u) exp

:s 1 : Wt = x}

x2)

- 2u

'

O
if this set is non-empty, otherwise,

has the same distribution as Ux. 21. Let sign(x) = 1 if x > 0 and sign(x) = -1 otherwise. Show that Vt a standard Wiener process if W is itself such a process.

= IJ sign(Ws) dWs defines

22. After the level of an industrial process has been set at its desired value, it wanders in a random fashion. To counteract this the process is periodically reset to this desired value, at times 0, T, 2T, .... If Wt is the deviation from the desired level, t units of time after a reset, then (Wt : 0 :s t < T} can be modelled by a standard Wiener process. The behaviour of the process after a reset is independent of its behaviour before the reset. While Wt is outside the range (-a, a) the output from the process is unsatisfactory and a cost is incurred at rate C per unit time. The cost of each reset is R. Show that the period T which minimises the long-run average cost per unit time is T*, where

R

=C

2 T* ---exp a ( -a- ) dt. o ,v(2JTt) 2t

lo

23. An economy is governed by the Black-Scholes model in which the stock price behaves as a geometric Brownian motion with volatility (1, and there is a constant interest rate r. An investor likes to have a constant proportion y (E (0, 1)) of the current value of her self-financing portfolio in stock and the remainder in the bond. Show that the value function of her portfolio has the form Vt = I (t) Sr where I(t) = c exp((l - Y)(! y(12 + r)t} for some constant c depending on herinitial wealth. 24. Let u(t, x) be twice continuously differentiable in x and once in t, for x E lR and t E [0, T]. Let W be the standard Wiener process. Show that u is a solution of the heat equation

au

au

1 2

iii = "2 ax 2 if and only if the process U t

= u (T

- t, Wt ), 0

:s t :s T, has zero drift.

133

1

Events and their probabilities

1.2 Solutions. Events as sets 1. (a) Let a E (U Ai)c. Then a ¢ U Ai, so that a E Ai for all i. Hence (U Ai)C ~ n Ai. Conversely, if a E Ai, then a ¢ Ai for every i. Hence a ¢ U Ai, and so Ai ~ (U Ai)c. The first De Morgan law follows.

n

n

(Ui Ait = ni (Ai)C = ni Ai·

(b) Applying part (a) to the family {Ai : i E l}, we obtain that Taking the complement of each side yields the second law.

2. Clearly (i) AnB=(ACUBC)C, (ii) A \ B = An B C = (A CU B)C, (iii) A L:. B = (A \ B) U (B \ A) = (A CU B)C U (A U BC)c. Now :F is closed under the operations of countable unions and complements, and therefore each of these sets lies in :F.

3. Let us number the players 1,2, ... , 2n in the order in which they appear in the initial table of draws. The set of victors in the first round is a point in the space Vn = {1, 2} x {3, 4} x ... x {2n - 1, 2n}. Renumbering these victors in the same way as done for the initial draw, the set of second-round victors can be thought of as a point in the space Vn-I, and so on. The sample space of all possible outcomes of the tournament may therefore be taken to be Vn x Vn-l x ... x Vb a set containing 2n 2 . 22n-' 2 - ... 21 = 22n-1 pomts. Should we be interested in the ultimate winner only, we may take as sample space the set {1, 2, ... , 2n} of all possible winners. 4. We must check that 9. satisfies the definition of a a-field: (a) 0 E :F, and therefore 0 = 0 n B E 9.,

(b) if AI, A2, ... E :F, then Ui(Ai n B) = (Ui Ai) n BE 9., (c) if A E :F, then AC E :Fso that B \ (A n B) = A Cn BE fl.. Note that 9. is a a-field of subsets of B but not a a-field of subsets of Q, since C E that CC = Q \ C E 9.. 5.

(a), (b), and (d) are identically true; (c) is true if and only if A

~

9. does not imply

C.

1.3 Solutions. Probability 1.

(i) We have (using the fact that JP> is a non-decreasing set function) that JP>(A

n B) = JP>(A) + JP>(B) - JP>(A U B) 2: JP>(A) + JP>(B) - 1 = 135

fz.

[1.3.2]-[1.3.4]

Solutions

Events and their probabilities

Also, since A nBS; A and A nBS; B, IP'(A n B) :s min{IP'(A), IP'(B)} = j. Theseboundsareattainedinthefollowingexample. Pick a number at random from {I, 2, ... , 12}. Taking A = {I, 2, ... , 9} and B = {9, 10, 11, 12}, we find that A n B = {9}, and so IP'(A) = IP'(B) = j, IP'(A B = {l, 2, 3, 4}.

=

n B)

i,

/2.

To attain the upper bound for IP'(A n B), take A

(ii) Likewise we have in this case IP'(A U B) :s min{IP'(A) + IP'(B), I} max{IP'(A), IP'(B)} = These bounds are attained in the examples above.

i.

2.

(i) We have (using the continuity property of lP'(no head ever)

=

= {l, 2, ... , 9} and

1, and IP'(A U B) ::::

IP') that

= n-+oo lim lP'(no head in first n tosses) = lim 2- n = 0, n-+oo

so that lP'(some head turns up) = 1 -1P'(no head ever) = 1. (ii) Given a fixed sequence s of heads and tails of length k, we consider the sequence of tosses arranged in disjoint groups of consecutive outcomes, each group being of length k. There is probability 2- k that any given one of these is s, independently of the others. The event {one of the first n such groups is s} is a subset of the event {s occurs in the first nk tosses}. Hence (using the general properties of probability measures) we have that lP'(s turns up eventually)

= n-+oo lim lP'(s occurs in the first nk tosses) :::: lim lP'(s occurs as one of the first n groups) n-+oo

=

1-

=

1-

lim lP'(none of the first n groups is s)

n-+oo

lim (l - Z-k)n

n-+oo

=

1.

3. Layout the saucers in order, say as RRWWSS. The cups may be arranged in 6! ways, but since each pair of a given colour may be switched without changing the appearance, there are 6! -:- (2!)3 = 90 distinct arrangements. By assumption these are equally likely. In how many such arrangements is no cup on a saucer of the same colour? The only acceptable arrangements in which cups of the same colour are paired off are WWSSRR and SSRRWW; by inspection, there are a further eight arrangements in which the first pair of cups is either SW or WS, the second pair is either RS or SR, and the third either RW or WR. Hence the required probability is 10/90 =

b.

4. We prove this by induction on n, considering firstthe case n = 2. Certainly B = (An B) U (B \ A) is a union of disjoint sets, so that IP'(B) = IP'(A n B) + IP'(B \ A). Similarly AU B = AU (B \ A), and so IP'(A U B) = IP'(A) + IP'(B \ A) = + {IP'(B) -IP'(A n B)}.

IP'(A)

Hence the result is true for n = 2. Let m :::: 2 and suppose that the result is true for n true for pairs of events, so that

l lP'eG 1

:s m. Then it is

Ai) = IP'(UAi) + IP'(Am+l) -IP'{ (UAi) n Am+l} 1

= IP'( U

1

Ai) +

IP'(Am+I>

1

-IP'{ U(Ai n Am+l)}. 1

Using the induction hypothesis, we may expand the two relevant terms on the right-hand side to obtain the result.

136

Solutions [1.3.5]-[1.4.1]

Conditional probability

Let A 1, A2, and A3 be the respective events that you fail to obtain the ultimate, penultimate, and ante-penultimate Vice-Chancellors. Then the required probability is, by symmetry, 3

I -lP'(

UAi) =

1 - 3lP'(A1)

+ 3lP'(A1 n A2) -lP'(A1 n A2 n A3)

1

5.

By the continuity oflP', Exercise (1.2.1), and Problem (1.8.11),

lP'(

n

Ar)

r=l

= n~moo lP'(

n

Ar)

r=l

= 1- n--+oo lim lP'(U

= n~moo [1 -lP'( ( A;) :::: I -

r=l

n r)] Ar

r=l

lim tlP'(A;) n--+oo r=l

= 1.

We have that 1 = lP'(UAr) = LlP'(Ar) - LlP'(Ar n As) = np - in(n - l)q. Hence 1 r r<s p :::: n- 1, and ~n(n - l)q = np - 1 ~ n - 1.

6.

7.

Since at least one of the Ar occurs,

n As) + L

1= lP'(UAr) = LlP'(Ar) - LlP'(Ar 1 r r<s

Since at least two of the events occur with probability

i=lP'(U(ArnAs)) =LlP'(ArnAs)-i r<s

r<s

lP'(Ar

n As nAt)

r<s
i, L lP'(Ar r<s t
n As nAt n Au) + ....

By a careful consideration of the first three terms in the latter series, we find that

Hence

i = np -

G)x, so that p :::: 3/(2n). Also, (~)q

= 2np - ~, whence q ~ 4/n.

1.4 Solutions. Conditional probability 1.

By the definition of conditional probability, lP'(A

I B)

- lP'(A n B) _ lP'(B n A) lP'(A) _ lP'(B lP'(B) lP'(A) lP'(B) -

137

I A) lP'(A) lP'(B)

[1.4.2]-[1.4.5] if lP'(A)lP'(B)

Solutions

i- O.

Events and their probabilities

Hence lP'(A

lP'(B I A) lP'(B) ,

I B)

lP'(A)

whence the last part is immediate. 2.

Set Ao =

n for notational convenience.

Expand each tenn on the right-hand side to obtain

3.

Let M be the event that the first coin is double-headed, R the event that it is double-tailed, and Let H{ be the event that the lower face is a head on the ith toss, TJ the event that the upper face is a tail on the ith toss, and so on. Then, using conditional probability ad nauseam, we find:

N the event that it is nonnal.

lP' ( HI1)

(i)

3 2 (HI I I M ) + slP' 1 (HI 1 1 R) + slP' 2 ( HI 1 1 N) = s 2+0 + s2. 1 = slP' '2 = s·

n HJ)

(ii)

lP'(HI

I Hu) =

lP'(Hl

(iii)

lP'(H?

I HJ) =

I ·lP'(M

1

1

1

lP'(Hu)

lP'(M)

2/3

= -lP'(H -1- = S )

S

2

= j.

I

I HJ) + ilP'(N I HJ)

1) = j2 + 2:1.1 5 = lP' ( HI11 Hu)1 + '21( I -lP' (11 HI Hu) j = 6·

(iv)

~ 4 = -2--1- =-. 4 . lP'(N) 3' + 10 5

lP'(M)

I . lP'(M)

+

1

(v) From (iv), the probability that he discards a double-headed coin is ~, the probability that he discards a nonnal coin is ~. (There is of course no chance of it being double-tailed.) Hence, by conditioning on the discard,

4. The final calculation of ~ refers not to a single draw of one ball from an urn containing three, but rather to a composite experiment comprising more than one stage (in this case, two stages). While it is true that {two black, one white} is the only fixed collection of balls for which a random choice is black with probability ~, the composition of the urn is not determined prior to the final draw. After all, if Carroll's argument were correct then it would apply also in the situation when the urn implying that originally contains just one ball, either black or white. The final probability is now the original ball was one half black and one half white! Carroll was himself aware of the fallacy in this argument.

i,

5. (a) One cannot compute probabilities without knowing the rules governing the conditional probabilities. If the first door chosen conceals a goat, then the presenter has no choice in the door to be opened, since exactly one of the remaining doors conceals a goat. If the first door conceals the car, then a choice is necessary, and this is governed by the protocol of the presenter. Consider two 'extremal' protocols for this latter situation. (i) The presenter opens a door chosen at random from the two available. (ii) There is some ordering of the doors (left to right, perhaps) and the presenter opens the earlier door in this ordering which conceals a goat. Analysis of the two situations yields p

=

~ under (i), and p

138

=

i under (ii).

Solutions

Independence

[1.4.6]-[1.5.3]

Let a E [~, ~], and suppose the presenter possesses a coin which falls with heads upwards with probability f3 = 6a - 3. He flips the coin before the show, and adopts strategy (i) if and only if the coin shows heads. The probability in question is now ~ f3 + ~ (l - f3) = a. You never lose by swapping, but whether you gain depends on the presenter's protocol. (b) Let D denote the first door chosen, and consider the following protocols: (iii) If D conceals a goat, open it. Otherwise open one of the other two doors at random. In this case p = O. (iv) If D conceals the car, open it. Otherwise open the unique remaining door which conceals a goat. In this case p = 1. As in part (a), a randomized algorithm provides the protocol necessary for the last part.

6.

This is immediate by the definition of conditional probability.

7. Let Ci be the colour of the ith ball picked, and use the obvious notation. (a) Since each urn contains the same number n - 1 of balls, the second ball picked is equally likely to be any of the n(n -1) available. One half of these balls are magenta, whence JP>(C2 = M) = (b) By conditioning on the choice of urn,

!.

JP>( C 2 = M I C 1 = M ) =

JP>(C1,C2=M) JP>(C1 = M)

~(n-r)(n-r-l)/l 2 - = - . 2 3

= L...J r=1 n(n - l)(n - 2)

1.5 Solutions. Independence 1.

Clearly JP>(A C

n B) = JP>(B \

= JP>(B) -

{A

n B)) = JP>(B)

-JP>(A

n B)

JP>(A)JP>(B) = JP>(Ac)JP>(B).

For the final part, apply the first part to the pair B, AC •

2. Suppose i < j and m < n. If j < m, then Aij and Amn are determined by distinct independent rolls, and are therefore independent. For the case j = m we have that JP>(Aij

n Ajn) =

JP>(ith, jth, and nth rolls show s~me number) 6

=

L ~JP>(jth and nth rolls both show r \ ith shows r) = i6 = JP>(Aij )lP'(Ajn) , r=1

as required. However, if i

i- j i- k,

3. That (a) implies (b) is trivial. Suppose then that (b) holds. Consider the outcomes numbered i 1, i2, ... , i m, and let Uj E {H, T} for 1 ::: j ::: m. Let Sj be the set of all sequences oflength M = max{ij : 1 ::: j ::: m} showing Uj in the ijth position. Clearly ISjl = 2 M - 1 and \nj Sj\ = 2M-m. Therefore,

139

[1.5.4]-[1.7.2] so that JP> (nj

Events and their probabilities

Solutions

Sj) = ITj JP>(Sj).

4. Suppose IAI = a, IBI = b, IA n BI = c, and A and B are independent. Then JP>(A n B) = JP>(A)lP'(B), which is to say that clp = (alp) . (blp), and hence ab = pc. If ab =j:. 0 then p I ab (i.e., p divides ab). However, p is prime, and hence either p I a or p I b. Therefore, either A = Q or B = Q (or both). 5. (a) Flip two coins; let A be the event that the first shows H, let B be the event that the second shows H, and let C be the event that they show the same. Then A and B are independent, but not conditionally independent given C. (b) Roll two dice; let A be the event that the smaller is 3, let B be the event that the larger is 6, and let C be the event that the smaller score is no more than 3, and the larger is 4 or more. Then A and B are conditionally independent given C, but not independent. (c) The definitions are equivalent if JP>(C) = 1.

1·

6.

(fO)7 <

7.

(a) JP>(A n B) = = = JP>(A)JP>(B), and JP>(B n C) = n C) = 0 =j:.JP>(A)JP>(C).

k i ·1

i = ~ . i = JP>(B)JP>(C).

(b) JP>(A

(c) Only in the trivial cases when children are either almost surely boys or almost surely girls. (d) No.

i.

8.

No. JP>(all alike) =

9.

JP>(lst shows r and sum is 7)

= to = ~ . ~ = JP>(lst shows r )JP>(sum is 7).

1.7 Solutions. Worked examples 1. Write EF for the event that there is an open road from E to F, and EF" for the complement of this event; write E ~ F if there is an open route from E to F, and E F if there is none. Now {A ~ C} = AB nBC, so that

-+

JP>(AB I A

-+ C) =

-+

JP>(AB, A C) JP>(A C)

-+

=

-+

JP>(AB, B C) 1 _ JP>(A ~ C)

(1 - p2)p2

= -:--1---(----=1-_----=p"2)"2 .

By a similar calculation (or otherwise) in the second case, one obtains the same answer: JP>(AB I A

-+ C) = 1 -

2) 3 (1 - p p (1- p2)2p - (1 - p)

=

(1 2) 2 - p p . 1 - (1 _ p2)2

2. Let A be the event of exactly one ace, and KK be the event of exactly two kings. Then JP>(A KK) = JP>(A n KK)/JP>(KK). Now, by counting acceptable combinations,

so the required probability is

7.11 .4737 ~ 0.44. 10 / (4)2 (48) 11 = 3.46. (4)1 (4)2 (44) 140

I

Solutions

Problems

[1.7.3]-[1.8.2]

3. First method: Suppose that the coin is being tossed by a special machine which is not switched off when the walker is absorbed. If the machine ever produces N heads in succession, then either the game finishes at this point or it is already over. From Exercise (1.3.2), such a sequence of N heads must (with probability one) occur sooner or later. Alternative method: Write down the difference equations for Pb the probability the game finishes at

ohaving started at k, and for ih, the corresponding probability that the game finishes at N; actually these two difference equations are the same, but the respective boundary conditions are different. Solve these equations and add their solutions to obtain the total 1.

4.

It is a tricky question. One of the present authors is in agreement, since if lI"(A I C) > lI"(B I C) and lI"(A ICC) > lI"(B I CC) then

+ lI"(A I C)lI"(C) + lI"(B

lI"(A) = lI"(A I C)lI"(C) > lI"(B

I CC)lI"(Cc) I CC)lI"(Cc)

= lI"(B).

The other author is more suspicious of the question, and points out that there is a difficulty arising from the use of the word 'you'. In Example (1.7.10), Simpson's paradox, whilst drug I is preferable to drug II for both males and females, it is drug II that wins overall.

5.

Let Lk be the label of the kth card. Then, using symmetry, lI"(Lk

= miLk>

Lr for 1 < r < k) -

= lI"(Lk

lI"(Lk =m)

> Lr for 1 :::: r < k)

= -m1/1-k = kim.

1.8 Solutions to problems 1. (a) Method I: There are 36 equally likely outcomes, and just 10 of these contain exactly one six. The answer is therefore = Method II: Since the throws have independent outcomes,

¥tl fs·

lI"(first is 6, second is not 6)

= lI"(first is 6)1I"(second is not 6) = ~ . i = ft;.

There is an equal probability of the event {first is not 6, second is 6}. (b) A die shows an odd number with probability

1; by independence, lI"(both odd) = 1.1= i·

(c) Write S for the sum, and {i, j} for the event that the first is i and the second j. Then lI"(S 11"(1,3) + 11"(2, 2) + 11"(3,1) = (d) Similarly

16.

lI"(S divisible by 3)

= 4) =

= lI"(S = 3) + lI"(S = 6) + lI"(S = 9) + lI"(S = 12) = {1I"(l, 2) + 11"(2, 1)} + {1I"(l, 5) + 11"(2, 4) + 11"(3, 3) + 11"(4, 2) + 11"(5, I)} + {11"(3, 6) + 11"(4, 5) + 11"(5, 4) + 11"(6, 3)} + 11"(6, 6) = j~ =~.

2. (a) By independence, lI"(n - 1 tails, followed by a head) = Tn. (b) If n is odd, 11"(# heads = # tails) = 0; #A denotes the cardinality of the set A. If n is even, there are (nl2) sequences of outcomes with heads and tails. Any given sequence of heads and tails

1n

has probability

1n

Tn; therefore 11"(# heads = # tails) = 2-n (nl2). 141

[1.8.3]-[1.8.9]

Solutions

Events and their probabilities

(c) There are G) sequences containing 2 heads and n - 2 tails. Each sequence has probability 2- n , and therefore lP'(exactly two heads) = G)2- n . (d) Clearly

(~) Tn.

lP'(at least 2 heads) = 1 - lP'(no heads) -lP'(exactly one head) = 1 - Tn -

3. (a) Recall De Morgan's Law (Exercise 0.2.1»: ni Ai = (Ui Aft, which lies in Fsince it is the complement of a countable union of complements of sets in :F. (b) Jf is a CT -field because: (i) 0 E Fand 0 E fJ,; therefore 0 E Jf. (ii) If A I, A2, ... is a sequence of sets belonging to both F and fJ" then their union lies in both F and fJ" which is to say that Jf is closed under the operation of taking countable unions. (iii) Likewise A C is in Jf if A is in both F and fJ,. (c) We display an example. Let Q={a,b,c},

F={{a},{b,c},0,Q},

fJ,={{a,b},{c},0,Q}.

Then Jf = FU fJ, is given by Jf = {{a}, {c}, {a, b}, {b, c}, 0, Q}. Note that {a} but the union {a, c} is not in Jf, which is therefore not aCT-field.

E

Jf and {c}

E

Jf,

4. In each case F may be taken to be the set of all subsets of Q, and the probability of any member of F is the sum of the probabilities of the elements therein. (a) Q = {H, T}3, the set of all triples of heads (H) and tails (T). With the usual assumption of independence, the probability of any given triple containing h heads and t = 3 - h tails is ph (l _ p)t, where p is the probability of heads on each throw. (b) In the obvious notation, Q

t.

= {U, V}2 =

= lP'(VV) =

{UU, VV, UV, VU}. Also lP'(UU)

~ . ~ and

lP'(UV) = lP'(VU) = ~ . (c) Q is the set of finite sequences of tails followed by a head, {Tn H : n :::: O}, together with the infinite sequence TOO of tails. Now, lP'(TnH) = 0 - p)n p, and lP'(TOO ) = limn---+oo(l - p)n = 0 if p =1= O.

= lP'( (A U B) \ lP'(A n B»)

S.

As usual, lP'(A L.. B)

6.

Clearly, by Exercise 0.4.2), lP'(A U B U C)

= lP'(A U B) -lP'(A n B).

= lP'(AC n B Cn CC)C) = 1 -lP'(AC n B Cn CC) = 1 -lP'(A c I B Cn CC)lP'(B c I CC)lP'(Cc).

7. (a) If A is independent of itself, then lP'(A) = lP'(A n A) = lP'(A)2, so that lP'(A) = 0 or l. (b) If lP'(A) = 0 then 0 = lP'(A n B) = lP'(A)lP'(B) for all B. If lP'(A) = 1 then lP'(A n B) = lP'(B), so that lP'(A n B) = lP'(A)lP'(B).

8. Q U 0 = Q and Q n 0 = 0, and therefore 1 = lP'(Q U 0) = lP'(Q) + lP'(0) = 1 + lP'(0) , implying that lP'(0) = O. 9. (i) (Q(0) = lP'(0 I B) = o. Also (Q(Q) = lP'(Q I B) = lP'(B)/lP'(B) = l. (ii) Let AI, A2, '" be disjoint members of F. Then {Ai n B : i :::: I} are disjoint members of :F, implying that

00) =lP' (00 UAi (I I

(Q UAi

I) B

=

00

lP'(UOO(A n B») I i = L lP'(B)

142

I

lP'(A

i

n B)

lP'(B)

00

= L(Q(Ai). I

Solutions [1.8.10]-[1.8.13]

Problems

Finally, since Q is a probability measure,

n C) IP'(A n C I B) IP'(A n B n C) Q(C) = IP'(C I B) = IP'(B n C) = IP'(A IBn C).

Q(A Q(A

I C) =

The order of the conditioning (C before B, or vice versa) is thus irrelevant.

10. As usual,

11. The first inequality is trivially true if n = 1. Let m 2: 1 and assume that the inequality holds for n::: m. Then

:::1P'(UAi) +1P'(A m +l):::

~1P'(Ai)'

1

1

by the hypothesis. The result follows by induction. Secondly, by the first part,

12. We have that

1P'(~Ai) =IP'( (YAir) = 1-IP'(Y Ai) = 1-

~1P'(Aj) + ~1P'(Ai n Ai) I

= 1-

...

+ (-l)nlP'(

n+

(n)2-.

+

LIP'(Ai)

.

LIP'(Ai U Aj) _

I

= (l - l)n

.

l<j

+ (_l)n (:)

n

Ai)

by Exercise (1.3.4)

1

l<j

- (_l)nlP'

+ ~1P'(Ai) -

... -

(y

Ai)

(-WIP' (

(n)3 + ...

using De Morgan's laws again

Y

Ai)

by the binomial theorem.

13. Clearly, IP'(Nk)

=

L

lP'(nAinAi)'

S£;{1,2, ... ,n}

iES

H-S

ISI=k

For any such given S, we write As

IP'(

n n Ai

iES

H-S

Ai)

= niES Ai.

Then

= IP'(As) - LIP'(Asu[j) H-S

143

+

L j
j,k'l-S

IP'(ASU[j,k) - ...

[1.8.14]-[1.8.16]

Solutions

Events and their probabilities

by Exercise (1.3.4). Hence

where a typical summation is over all subsets S of {I, 2, ... , n} having the required cardinality. Let Ai be the event that a copy of the ith bust is obtained. Then, by symmetry,

where ctj is the probability that the j most recent Vice-Chancellors are obtained. Now ct3 is given in Exercise (1.3.4), and ct4 and ct5 may be calculated similarly.

14. Assuming the conditional probabilities are defined, JP'(A-

1

B)

=

JP'(B

JP'(Aj n B) JP'(B)

1

1

Aj)JP'(Aj)

JP'(B n

(U1 Ai))

15. (a) We have that JP'(N

= 21

S

= 4) =

= 2} n {S = 4}) = JP'(S = 4)

JP'({N

1

JP'(S = 41 N = 2)JP'(N = 2) Li JP'(S = 4 1 N = i)JP'(N = i)

1

12 . 4 1 3 1 1 1· (; . 2 + 12 . 4 + 216 . "8 + 64 . 16 1

1

1

(b) Secondly, JP'(S

=4

N even)

1

=

JP'(S

= 41

N

= 2)i + JP'(S = 41

N

= 4)ft,

-----~:-:---:__----=

JP'(N even)

11

1

1

23

43 +1 44 3 3 .

12·4+64·16

i + ~ +...

(c) Writing D for the number shown by the first die,

JP'(N

=2

1

S

= 4, D =

1)

= 2, S = 4, D = JP'(S = 4, D = 1)

JP'(N

=

1)

=

1

1

1

= -r

(

6 . (; . 4

g. g.i 1

2

1

1

1·

+ 6 . 36 . "8 + 64 . 16

(d) Writing M for the maximum number shown, if 1 ::: r ::: 6,

JP'(M.::s r)

00 = LJP'(M.::s r

1

N

= j)Tl. = L00

j=l

Finally, JP'(M = r) = JP'(M

j=l

(r)j 1 J 6 2

12

r

1- 12

)-1 = - r . 12 - r

.::s r) -JP'(M .::s r - 1).

16. (a) WEB if and only if, for all n, W of the An. (b) WEe if and only if, for some n, W number of the An.

E U~n

Ai, which is to say that W belongs to infinitely many

E n~n

Ai, which is to say that W belongs to all but a finite

144

Solutions [1.8.17]-[1.8.20]

Problems

(c) It suffices to note that B is a countable intersection of countable unions of events, and is therefore an event. (d) We have that

n 00

Cn =

00

Ai S; An S;

i=n

U Ai =

Bn ,

i=n

and therefore IP'(Cn ) :::: IP'(An) :::: IP'(Bn). By the continuity of probability measures (1.3.5), if C n -+ C then IP'(Cn ) -+ IP'(C), and if Bn -+ B then IP'(Bn) -+ IP'(B). If B = C = A then IP'(A) = IP'(C):::: lim IP'(An) :::: IP'(C) = IP'(A).

n..... oo

17. If Bn and C n are independent for all n then, using the fact that C n S; B n , as n -+

00,

and also IP'(Bn)IP'(Cn ) -+ IP'(B)IP'(C) as n -+ 00, so that IP'(C) = IP'(B)IP'(C), whence either IP'(C) = 0 or IP'(B) = 1 or both. In any case IP'(B n C) = IP'(B)IP'(C). If An -+ A then A = B = C so that IP'(A) equals 0 or 1.

18. It is standard (Lemma (1.3.5» that IP' is continuous if it is countably additive. Suppose then that IP' is Ai = limn ..... oo Ai, finitely additive and continuous. Let AI, A2, ... be disjoint events. Then so that, by continuity and finite-additivity,

Ui'"

Ui

19. The network of friendship is best represented as a square with diagonals, with the comers labelled A, B, C, and D. Draw a diagram. Each link of the network is absent with probability p. We write EF for the event that a typical link EF is present, and EF" for its complement. We write A ++ D for the event that A is connected to D by present links. (d)

IP'(A ++ D I ADc ) = IP'(A ++ D I ADc n BCc)p + IP'(A ++ D I ADc n BC)(1 - p)

{I- (1- (1- p)2)2}p + (1- p 2l(1- p). =IP'(A ++ D I AD c nBCc)p +IP'(A ++ D I BCc nAD)(1= {I - (1 - (1 - p)2)2} P + (1 - p). = IP'(A ++ D I ABC n ADc)p +IP'(A ++ D I ABC n AD) (1 =

(c)

IP'(A ++ D I BCc)

(b)

IP'(A ++ D I ABc)

p)

p)

= (1 - p){ 1 - p(1 - (1 - ph} p + (I - p). (a)

IP'(A ++ D) = IP'(A ++ D I ADc)p +IP'(A ++ D I AD)(1- p) =

{I- (1-

(1- p)2l}p2 + (1- p2)2p(1_ p) + (1- p).

20. We condition on the result of the first toss. If this is a head, then we require an odd number of heads in the next n - 1 tosses. Similarly, if the first toss is a tail, we require an even number of heads in the next n - 1 tosses. Hence Pn

= p(1

- Pn-I)

+ (I -

P)Pn-1

=

(1 - 2P)Pn-1

+P

with Po = 1. As an alternative to induction, we may seek a solution of the form Pn = A + BAn. Substitute this into the above equation to obtain A + BAn = (1- 2p)A + (1- 2p)BA n - 1 + p

145

[1.8.21]-[1.8.23] and A

+B

Solutions

= 1. Hence A =

Events and their probabilities

1, B = 1, A. =

1 - 2p.

21. Let A = {run of r heads precedes run of s tails}, B = {first toss is a head}, and C = {first s tosses are tails}. Then

where p = 1 - q is the probability of heads on any single toss. Similarly IP'(A I B) = pr-l + IP'(A BC)(l- pr-l). We solvefor IP'(A I B) and IP'(A I BC), and use the fact that IP'(A) = IP'(A I B) p+ IP'(A BC)q, to obtain

I I

22. (a) Since every cherry has the same chance to be this cherry, notwithstanding the fact that five are now in the pig, the probability that the cherry in question contains a stone is =

to !-.

(b) Think about it the other way round. First a random stone is removed, and then the pig chooses his fruit. This does not change the relevant probabilities. Let C be the event that the removed cherry contains a stone, and let P be the event that the pig gets at least one stone. Then IP'(P I C) is the probability that out of 19 cherries, 15 of which are stoned, the pig gets a stone. Therefore IP'(P

I C) = 1 -1P'(pig chooses only stoned cherries I C) = 1 -

M. M. H. M.H·

23. Label the seats 1, 2, ... , 2n clockwise. For the sake of definiteness, we dictate that seat 1 be occupied by a woman; this determines the sex of the occupant of every other seat. For 1 :::: k :::: 2n, let Ak be the event that seats k, k + 1 are occupied by one of the couples (we identify seat 2n + 1 with seat 1). The required probability is

Now, IP'(Ai) = n(n - 1)!2 /n!2, since there are n couples who may occupy seats i and i + 1, (n -I)! ways of distributing the remaining n - 1 women, and (n - I)! ways of distributing the remaining n - 1 men. Similarly, if 1 :::: i < j :::: 2n, then (n - 2)!2 IP'(Ai nA j)=

subject to IP'(AI

on(n-l)

ifli-jl#1

n!2

{

ifli-jl=l,

n A2n) = O. In general, IP'(Ail

n Ai2 n ... n Aik) =

n!

(n - k)!2

(n-k)!

n!

if il < i2 < ... < ik and i}+l - ij :::: 2 for 1 :::: j < k, and 2n probability is O. Hence 2n

IP' (

)

n

nAf = L ( - l l 1

k=O

(n - k)!

2

n!

+ il

- ik :::: 2; otherwise this

(n _ k)' " Sk,n

n.

where Sk,n is the number of ways of choosing k non-overlapping pairs of adjacent seats. Finally, we calculate Sk,n' Consider first the number Nk,m of ways of picking k non-overlapping pairs of adjacent seats from a line (rather than a circle) of m seats labelled 1, 2, ... , m. There is a oneone correspondence between the set of such arrangements and the set of (m - k)-vectors containing

146

Solutions [1.8.24]-[1.8.27]

Problems

k 1's and (m - 2k) O's. To see this, take such an arrangement of seats, and count 0 for an unchosen seat and 1 for a chosen pair of seats; the result is such a vector. Conversely take such a vector, read its elements in order, and construct the arrangement of seats in which each 0 corresponds to an unchosen seat and each 1 corresponds to a chosen pair. It follows that Nk,m = (m-;;k). Turning to Sk,n, either the pair 2n, 1 is chosen or it is not. If it is chosen, we require another k - 1 pairs out of a line of 2n - 2 seats. If it is not chosen, we require k pairs out of a line of 2n seats. Therefore

1) +

2n - k Sk,n=Nk-l,2n-2+ N k,2n= ( k-I

(2n - k) k

=

(2n - k) 2n k 2n-k'

24. Think about the experiment as laying down the b + r balls from left to right in a random order. The number of possible orderings equals the number of ways of placing the blue balls, namely (bt'). The number of ways of placing the balls so that the first k are blue, and the next red, is the number of ways of placing the red balls so that the first is in position k + 1 and the remainder are amongst the r + b - k - 1 places to the right, namely C+~=7-1). The required result follows. The probability that the last ball is red is r/(r ball in any other given position in the ordering.

+ b), the same as the chance of being red for the

25. We argue by induction on the total number of balls in the urn. Let Pac be the probability that the last ball is azure, and suppose that Pac = whenever a, C :::: 1, a + c ::::; k. Let a and a be such that a, a :::: 1, a + a = k + 1. Let Ai be the event that i azure balls are drawn before the first carmine ball, and let Cj be the event that j carmine balls are drawn before the first azure ball. We have, by taking conditional probabilities and using the induction hypothesis, that

i

a

a

Paa = LPa-i,alP'(Ai)

+ LPa,a-jlP'(Cj) j=1

i=1

a-I

= PO,alP'(Aa ) + Pa,OlP'(Ca ) + i L

a-I

lP'(Ai)

+iL

Now PO,a

lP'(Cj).

j=1

i=1

= 0 and Pa,O = 1. Also, by an easy calculation, a a+a

lP'(Aa) = - - .

a-I 1 ala! . .. - - = = lP'(Ca ). a+a-I a+I (a+a)!

It follows from the above two equations that Paa

=

1(i=lP'(Ai) + tlP'(Cj») + ~(lP'(Ca) -lP'(Aa») = 1· i=1

j=1

26. (a) If she says the ace of hearts is present, then this imparts no information about the other card, which is equally likely to be any of the three other possibilities. (b) In the given protocol, interchange hearts and diamonds.

27. Writing A if A tells the truth, and AC otherwise, etc., the only outcomes consistent with D telling Likewise, the the truth are ABCD, ABcCcD, NBCcD, and NBcCD, with a total probability of only outcomes consistent with D lying are NBcCcDc , AcBCDc , ABcCDc , and ABCcDc , with a total probability of Writing S for the given statement, we have that

H.

M.

lP'(D n S) lP'(D I S) - lP'(D n S) + lP'(DC

147

n S) -

~i

11

13

+~

41

81

[1.8.28J-[1.8.33J

Solutions

Events and their probabilities

Eddington himself thought the answer to be ~I; hence the 'controversy'. He argued that a truthful denial leaves things unresolved, so that if, for example, B truthfully denies that C contradicts D, then we cannot deduce that C supports D. He deduced that the only sequences which are inconsistent with the given statement are ABcCD and ABcCcDc , and therefore 25 25 IP'(D I S) = 25 "8T 46 = 71 "8T+iIT

Which side are you on? 28. Let Br be the event that the rth vertex of a randomly selected cube is blue, and note that IP'(Br) By Boole's inequality,

fa.

8

8

r=1

r=1

IP'(

U Br) ~ LIP'(Br ) = ib <

=

1,

so at least 20 per cent of such cubes have only red vertices.

= IP'(A n B)/IP'(A) = IP'(A I B)IP'(B)/IP'(A) > IP'(B). = IP'(A n BC)/IP'(B C) = (IP'(A) -IP'(A n B)}/IP'(B C) < IP'(A).

29. (a) IP'(B I A) (b) IP'(A I B C)

(c) No. Consider the case A n C = 0. 30. The number of possible combinations of birthdays of m people is 365 m ; the number of combinations of different birthdays is 365!/(365 - m)!. Use your calculator for the final part.

e;) / m). Now use your calcula-

32. In the obvious notation, lP'(wS, xH, yD, zC) = (!;) (~) (~) tor. Turning to the 'shape vector' (w, x, y, z) with w 2': x 2': y 2': z, lP'(w, x, y, z)

={

41P'(wS, xH, yD, zC)

if w =f. x

= y = z,

121P'(wS, xH, yD, zC)

if w

=x

=f. y =f. z,

on counting the disjoint ways of obtaining the shapes in question. 33. Use your calculator, and divide each of the following by

148

ef).

Solutions

Problems

[1.8.34]-[1.8.36]

34. Divide each of the following by 6 5 .

6!5! 3! (2!)2 '

6! 5! 3! (2!)3 ' 6!5! 4! 3! 2!'

6!,

6!5! 2!(3!)2' 6!5! (4!)2 '

6!5! (5!)2 .

35. Let Sr denote the event that you receive r similar answers, and T the event that they are correct. Denote the event that your interlocutor is a tourist by V. Then T lP'(T IS) r

Hence: (a) lP'(T lSI) (b) lP'(T I S2)

=

n V n Sr) lP'(Sr)

lP'(T

=

n Vc =

0, and

_lP'(-,---T_n_Sr-,I_V--,-)_lP'--,--(V---,-) lP'(Sr)'

= ~ x VI = i· = (~)2. V[{(~)2 + (!)2H +~] =

i.

+ (!)3H + j] = in. I S4) = (~)4. V[{(~)4 + (!)4H + j] =~.

(c) lP'(T I S3)

= (~)3.

V[{(~)3

(d) lP'(T (e) If the last answer differs, then the speaker is surely a tourist, so the required probability is

36. Let E (respectively W) denote the event that the answer East (respectively West) is given. (a) Using conditional probability,

lP'(Eastcorrect I E)

=

lP'(East correct I W)

=

ElP'(E I East correct)

(

lP' E) 1

E((;

=

E(~.! +~)

+

1

3)

2 3

+ 3'4(1- E)

(b) Likewise, one obtains for the answer EE,

and for the answer WW,

(c) Similarly for EEE,

149

1

ZE

E . ~. ~

2 1

1

+ (-3'4 + 3)(1 + E)

= E.

= E,

[1.8.37]-[1.8.39]

Solutions

Events and their probabilities

and for WWW,

€{ (~)(})3

€[(~)(~)3 + ~l Then for €

=

+

+

n

11€

(1-€)~(£)3 =

9+2€'

-la, the first is Paz; the second is ~, as you would expect if you look at Problem (1.8.35).

37. Use induction. The inductive step employs Boole's inequality and the fact that

38. We propose to prove by induction that

lP'( UAr) :s I)'(A r ) r=1

r=1

L

lP'(Ar

n AI)·

2~r~n

There is nothing special about the choice of A I in this inequality, which will therefore hold with any suffix k playing the role of the suffix 1. Kounias's inequality is then implied. The above inequality holds trivially when n = I. Assume that it holds for some value of n (:::: 1). We have that

:s tlP'(Ar ) -

lP'(Ar

L

n AI) + lP'(An+l)

2~r~n

r=1

-lP'( An+1 n UAr) r=1

n+1

:SLlP'(Ar )-

since lP'(An+1 n AI)

L

lP'(ArnAI)

2~r~n+1

r=1

:s lP'(An+1 n U~=I A r ).

39. We take n :::: 2. We may assume without loss of generality that the seats are labelled I, 2, ... , n, and that the passengers are labelled by their seat assignments. Write F for the event that the last passenger finds his assigned seat to be free. Let K (:::: 2) be the seat taken by passenger I, so that lP'(F) = (n-1)-1 I: k=2 akwhereak = lP'(F I K = k). Notethatan = O. Passengers 2, 3, ... , K-I occupy their correct seats. Passenger K either occupies seat I, in which case all subsequent passengers take their correct seats, or he occupies some seat L satisfying L > K. In the latter case, passengers K + I, K + 2, ... , L - I are correctly seated. We obtain thus that ak =

Therefore ak

=

I

n-k+1

(1 +ak+1 +ak+2

+ ... +an ),

~ for 2:s k < n, by induction, and so lP'(F)

150

=

2:s k < n.

~(n - 2)/(n -1).

2

Random variables and their distributions

2.1 Solutions. Random variables 1. (i) If a > 0, x E JR, then (w : aX(w) ::: x} = (w : X(w) ::: x/a} E Tsince X is a random variable. If a < 0,

(w: aX(w) ::: x}

= (w:

X(w) 2: x/a}

= { U {w:

X(w) :::

~ - ~}

n?: 1

r

which lies in T since it is the complement of a countable union of members of :F. If a = 0,

(w: aX(w) ::: x}

={

~

if x < 0, if x 2: 0;

in either case, the event lies in :F. (ii) For WEn, X (w) - X (w) = 0, so that X - X is the zero random variable (that this is a random variable follows from part (i) with a = 0). Similarly X(w) + X(w) = 2X (w). 2.

Set Y

= aX + b.

We have that

lP'(Y < y) Finally, if a

={

lP'(X ::: (y - b)/a)

=

F(y - b)/a)

lP'(X 2: (y - b)/a) = l-lim x t(y-b)/a F(x)

= 0, then Y = b, so that lP'(Y ::: y) equals 0 if b >

if a > 0, if a < O.

yand 1 if b ::: y.

3. Assume that any specified sequence of heads and tails with length n has probability 2- n . There are exactly (Z) such sequences with k heads. If heads occurs with probability p then, assuming the independence of outcomes, the probability of any given sequence ofk heads and n-k tails is pk(1- p)n-k. The answeris therefore (Z) pk(1- p)n-k. 4. Write H = )"'F + (1 - )",)0. Then limx-+-oo H(x) = 0, lim x-+ oo H(x) = 1, and clearly H is non-decreasing and right-continuous. Therefore H is a distribution function. S. The function g(F (x» is a distribution function whenever g is continuous and non-decreasing on [0, 1], with g(O) = 0, g(1) = 1. This is easy to check in each special case.

151

[2.2.1]-[2.4.1]

Solutions

Random variables and their distributions

2.2 Solutions. The law of averages 1.

Let p be the potentially embarrassed fraction of the population, and suppose that each sampled individual would truthfully answer "yes" with probability p independently of all other individuals. In the modified procedure, the chance that someone says yes is p + ~(1 - p) = ~(1 + p). If the proportion of yes's is now cp, then 2cp - 1 is a decent estimate of p. The advantage of the given procedure is that it allows individuals to answer "yes" without their being identified with certainty as having the embarrassing property. 2.

Clearly Hn

+ Tn = n, so that (Hn

lP' (2P - 1 as n -+

00,

E:s ~(Hn -

- Tn)/n

= (2Hn /n)

Tn) :s 2p - 1 +

- 1. Therefore

(I

E) = lP' ~Hn -

pi :s

n

-+ 1

by the law of large numbers (2.2.1).

LetIn(x) be the indicator function of the event{Xn :s x}. By the law of averages, n- I L~=I [rex) converges in the sense of (2.2.1) and (2.2.6) to lP'(Xn :s x) = F(x).

3.

2.3 Solutions. Discrete and continuous variables 1.

With 8

= sUPm lam

- am-II, we have that

IF(x) - G(x)1 :s IF(am) - F(am-dl :s IF(x

for x

E

+ 8) -

F(x - 8)1

[am-I, am). Hence G(x) approaches F(x) for any x at which F is continuous.

2.

For y lying in the range of g, {Y :s y} = {X :s g-I(y)} E :F.

3.

Certainly Y is a random variable, using the result of the previous exercise (2). Also lP'(Y:s y)

= lP'(F-I(X)

:s y)

= lP'(X

:s F(y»)

= F(y)

as required. If F is discontinuous then F- I (x) is not defined for all x, so that Y is not well defined. If F is non-decreasing and continuous, but not strictly increasing, then F- I (x) is not always defined uniquely. Such difficulties may be circumvented by defining F- I (x) = inf {y : F (y) 2: x}. 4. The function AI + (1- A)g is non-negative and integrable over lIHo 1. Finally, I g is not necessarily a density, though it may be: e.g., if 1= g = 1, 0 :s x :s 1 then I(x)g(x) = 1, 0 :s x :s 1. (a) If d > 1, then It) cx- d dx = c/(d -1). Therefore I is a density function if c = d - 1, and F(x) = 1 - x-(d-l) when this holds. If d :s 1, then I has infinite integral and cannottherefore be a density function. (b) By differentiating F(x) = eX /(1 + eX), we see that F is the distribution function, and c = 1.

5.

2.4 Solutions. Worked examples 1.

(a) If y 2: 0,

lP'(X 2 :s y)

= lP'(X :s .JY) -lP'(X

< -..fY)

(b) We must assume that X 2: O. If y 2: 0,

152

= F(..fY) -

F(-..fY).

Solutions

Random vectors ~

(c) If -I

y

~

[2.4.2]-[2.5.6]

I, 00

L

lP'(sinX~y)=

1P'«2n+l)rr-sin-1y~X~(2n+2)rr+sin-1y)

n=-oo 00

L

=

{F«2n

+ 2)rr + sin- 1 y)

- F«2n

+ l)rr -

sin- 1 y)}.

n=-oo

(d) IP'(G- 1(X) ~ y) = IP'(X ~ G(y» = F(G(y».

(e) If 0 ~ y ~ 1, then IP'(F(X) ~ y) = IP'(X ~ F- 1(y» = F(F- 1(y» = y. There is a small difficulty if F is not strictly increasing, but this is overcome by defining F- 1(y) = sup{x : F(x) = y}. (f) IP'(G- 1(F(X» ~ y) = IP'(F(X) ~ G(y» = G(y).

2.

It is the case that, for x E JR, Fy(x) and Fz(x) approach F(x) as a --+ -00, b --+ 00.

2.5 Solutions. Random vectors 1. Write fxw x,w. 2.

= IP'(X = x, W = w).

Then faa

=

121

=

i, flO = i, and fxw = 0 for other pairs

(a) We have that if (x, y)

= (1,0),

if (x, y) = (0, 1), otherwise. (b) Secondly, if (x, z)

l-P fx,z(x, z) =

3.

Differentiating gives fx,Y(x, y)

4.

Let A IP'(A)

=

= {X ~ b,

c < Y ~ d}, B

F(b, d) - F(b, c),

{

~

if (x, z)

otherwise.

= e- x /(rr(1 + y2)}, x

= {a

IP'(B)

=

= (0,0), = (1,0),

< X

:::: 0, y E R

~ b, Y ~ d}.

F(b, d) - F(a, d),

Clearly IP'(A U B)

=

F(b, d) - F(a, c);

now IP'(A n B) = IP'(A) + IP'(B) -IP'(A U B), which gives the answer. Draw a map ofJR 2 and plot the regions of values of (X, Y) involved.

5.

The given expression equals IP'(X

Secondly, for 1 ~ x

= x,

~

y

~

Y

~

y) -IP'(X

= x,

Y

~

y - 1)

= IP'(X = x,

Y

= y).

6,

ifx < y, if x = y.

6.

No, because F is twice differentiable with a2 F /axay < O.

153

[2.7.1]-[2.7.6]

Random variables and their distributions

Solutions

2.7 Solutions to problems 1.

By the independence of the tosses, lP'(X > m) = lP'(first m tosses are tails) = (l _ p)m.

Hence

I - (l - p) LxJ

={ 0

lP'(X < x)

-

Remember that 2.

if x :::: 0, if x < O.

Lx J denotes the integer part of x.

(a) If X takes values {Xi: i :::: I} then X = L~I Xi/A; where Ai = {X =

xd.

(b) Partition the real line into intervals of the form (k + l)2-m), -00 < k < 00, and define Xm = L~-oo k2- m h,m where h,m is the indicator function of the event {k2- m ~ X < (k + l)2-m }. Clearly Xm is a random variable, and Xm(w) t X(w) as m --+ 00 for all w. [k2- m ,

(c) Suppose {Xm} is a sequence of random variables such that Xm(w) t X(w) for all w. Then {X ~ x} = {Xm ~ x}, which is a countable intersection of events and therefore lies in J::

nm

3.

(a) We have that 00

{X+Y~x}=n

U

({X~r}n{Y~x-r+n-ll)

n=1 rEIQI+

where Q+ is the set of positive rationals. In the second case, if X Y is a positive function, then X Y = exp{log X + log Y}; now use Exercise (2.3.2) and the above. For the general case, note first that IZI is a random variable whenever Z is a random variable, since {IZI ~ a} = {Z ~ a} \ {Z < -a} for a :::: o. Now, if a :::: 0, then {XY ~ a} = {XY < O} U {IXYI ~ a} and {XY < O} = ({X < O}

n {Y

ol)

>

U ({X> O}

n {Y

<

ol).

Similar relations are valid if a < O. Finally {min{X, Y} > x} = {X > x}

n {Y

> y}, the intersection of events.

(b) It is enough to check that aX + fJY is a random variable whenever a, fJ variables. This follows from the argument above. If Q is finite, we may take as basis the set {/A

4.

:

E

lR and X, Y are random

A E J1 of all indicator functions of events.

(a) F(~) - F(~) = ~.

(b) F(2) - F(l)

=

1.

(c) lP'(X ~ X) = lP'(X ~ 1) = ~. 2

(d) lP'(X ~ 2X2) = lP'(X :::: (e)lP'(X +X2 ~

1) =

~.

i) =lP'(X ~~) = 1.

(f) lP'(-vIx ~ z) = lP'(X ~ z2) = 1z2 if 0 ~ Z ~

s.

.J2.

lP'(X = -1) = 1 - p, lP'(X = 0) = 0, lP'(X :::: 1) =

6.

1P'

There are 6 intervals of 5 minutes, preceding the arrival times of buses. Each such interval has . 6 . 12 5=1 1 = 2:' 1 b pro ab1'l'Ity 60 12' so the answer IS

154

Problems

Solutions

[2.7.7]-[2.7.12]

7. Let T and B be the numbers of people on given typical flights of TWA and BA. From Exercise (2.1.3),

_ _(10) (~)k (~) IO-k

IP'(T - k) -

k

10

)k ( )20-k

20) ( 9 1 ( k 10 10

IP'(B =k) =

,

10

Now

= IP'(T =

IP'(TWA overbooke d)

10)

9!0 , (10)

=

IP'(BA overbooked) = IP'(B ~ 19) = 20( -;&)!9 (to)

+ (-&)20,

of which the latter is the larger.

8.

Assuming the coins are fair, the chance of getting at least five heads is (~)6

9.

(a) We have that

+ 6( ~)6 =

l4'

if x < 0,

IP'(X+ < x) = { 0 F(x)

if x

~

O.

(b) Secondly,

~ x) = {O

IP'(X(c) IP'(JXI

~

= IP'(-x

x)

~

X

~

~

x)

=

if x < 0,

if x

~

O.

x) if x ~ O. Therefore

IP'(JXI < x) (d) IP'(-X

.

1 - hmyt-x F(y)

if x < 0,

={0

if x

F(x) -limyt-x F(y)

~

O.

1 -limyt_x F(y).

10. By the continuity of probability measures (1.3.5), IP'(X

= xo) =

lim lP'(y < X ytxo

~

xO)

= F(xo) -

lim F(y) ytxo

= F(xO) -

F(xo-),

using general properties of F. The result follows. 11. Define m

= sup{x : F(x)

< ~}. Then F(y) < ~ for y < m, and F(m) ~ ~ (if F(m) < ~ then

F(m') < ~ for some m' > m, by the right-continuity of F, a contradiction). Hence m is a median, and is smallest with this property.

A similar argument may be used to show that M = sup{x : F(x) ~ ~} is a median, and is largest with this property. The set of medians is then the closed interval [m, M]. 12. Let the dice show X and Y. Write S IP'(S = 2) = IP'(S = 7) = IP'(S = 12) =

= X + Y and

A. Now

fi

= IP'(X = i), gi = IP'(Y = i). Assume that

= 2) = IP'(X = 1)IP'(Y = 1) = fIg!, = 12) = IP'(X = 6)IP'(Y = 6) = f6g6, ~ IP'(X = 1)IP'(Y = 6) + IP'(X = 6)IP'(Y = 1) = IP'(S

IP'(S

IP'(S

= 7)

-1 = IP'(S = 7) 11

~

fIg! (g6 g!

155

+ -f6) = -1 fI

11

(x

fIg6

+ -1 ) x

+ f6g!.

[2.7.13]-[2.7.14]

Random variables and their distributions

Solutions

where x = g6/ g1. However x

+ x -1

> I for all x > 0, a contradiction.

13. (a) Clearly dL satisfies (i). As for (ii), suppose that ddF, G)

+ E) + E} =

F(x) .::: lim{G(x E.j,O

= O.

Then

G(x)

and F(y) :::: lim{G(y - E) - E} E.j,O

= G(y-).

Now G(y-) :::: G(x) if y > x; taking the limit as y ..j. x we obtain F(x) :::: lim G(y-) :::: G(x), y.j,x

implying that F(x) = G(x) for all x. Finally, if F(x) .::: G(x + E) + E and G(x) .::: H(x + 8) + 8 for all x and some E, 8 > 0, then F(x) .::: H(x + 8 + E) + E + 8 for all x. A similar lower bound for F(x) is valid, implying that ddF, H).::: ddF, G) +ddG, H).

= 0 if and only if IP'(X = Y) =

(b) Clearly dTV satisfies (i), and dTV(X, Y) inequality, ilP'(X

= k)

-IP'(Z

= k)i

= k)

.::: ilP'(X

-IP'(Y

1. By the usual triangle

= k)i + ilP'(Y = k) -IP'(Z = k)i,

and (iii) follows by summing over k. We have that 2ilP'(X E A) -IP'(Y E A)i = i(IP'(X E A) -IP'(Y E A») - (IP'(X E AC) -IP'(Y E AC»)i

=

/2:(IP'(X

= k)

-IP'(Y

= k»)JA(k)/

k

where J A (k) equals I if k

E

A and equals -I if k

2ilP'(X E A) -IP'(Y E A)i .::: 2:ilP'(X

E

A C. Therefore,

= k)

-IP'(Y

= k)i·IJA(k)1

.::: dTV(X, Y).

k

Equality holds if A

=

= k)

{k : IP'(X

> IP'(Y

= k)}.

14. (a) Note that

a2F

- - = -e axay

-x-

Y

<0,

x,y > 0,

so that F is not a joint distribution function. (b) In this case ifO.:::x.:::y, ifO.:::y.:::x,

and in addition

roo roo

io io

2 a F dx dy = 1. axay

Hence F is a joint distribution function, and easy substitutions reveal the marginals: FX(x) = Fy(y) =

lim F(x, y) = I - e- x ,

y-+oo

x:::: 0,

lim F(x, y) = I - e- Y - ye- Y ,

x-+oo

156

y:::: O.

Solutions [2.7.15]-[2.7.20]

Problems

15. Suppose that, for some i =I j, we have Pi < Pj and Bi is to the left of Bj. Write m for the position of Bi and r for the position of Bj, and consider the effect of interchanging Bi and Bj. For k ::; m and k > r, lP'(T ::: k) is unchanged by the move. For m < k ::; r, lP'(T ::: k) is decreased by an amount Pj - Pi, since this is the increased probability that the search is successful at the mth position. Therefore the interchange of Bi and Bj is desirable. It follows that the only ordering in which lP'(T ::: k) can be reduced for no k is that ordering in which the books appear in decreasing order of probability. In the event of ties, it is of no importance how the tied books are placed. 16. Intuitively, it may seem better to go first since the first person has greater choice. This conclusion is in fact false. Denote the coins by Cl, C2, C3 in order, and suppose you go second. If your opponent chooses Cl then you choose C3, because lP'(C3 beats Cl) = ~ + ~ . ~ = i~ > !. Likewise lP'(CI beats C2) = lP'(C2 beats C3) = ~ > !. Whichever coin your opponent picks, you can arrange to have a better than evens chance of winning.

17. Various difficulties arise in sequential decision theory, even in simple problems such as this one. The following simple argument yields the optimal policy. Suppose that you have made a unsuccessful searches "ahead" and b unsuccessful searches "behind" (if any of these searches were successful, then there is no further problem). Let A be the event that the correct direction is ahead. Then lP'(A

lP'(current knowledge I A)lP'(A) I current knowledge) = - - - - - - - = - - - - - lP'(current knowledge) (1 - p)aa

which exceeds! if and only if (l - p)aa > (l - p)b(1 - a). The optimal policy is to compare (l - p)a a with (l - p)b (l - a). You search ahead if the former is larger and behind otherwise; in the event of a tie, do either. 18. (a) There are (
x2

X (c) The density function is f(x) (d) This is not a distribution function because F' (l) < O.

20. We have that lP'(U

=

,

x :::

= F' (x) = x 2e- l / x , x > o. = F'(x) = 2(e + e- x )-2, x

V)

=

r

o. E~.

fu,v(u, v) du dv

}{(u,v):u=vj

= O.

The random variables X, Y are continuous but not jointly continuous: there exists no integrable function f : [0, 1]2 --+ ~ such that lP'(X::;x,Y::;y)=

rx

[Y

f(u,v)dudv,

}u=O}v=O

157

O::;x,y::;l.

3

Discrete random variables

3.1 Solutions. Probability mass functions 1.

(a) C- 1 = L:f 2- k = l.

(b) C- 1 = L:f 2- k /k = log 2. (c) C- 1 = L:f k- 2 (d) C- 1

=

= rr 2 /6.

L:f2k/k!

= e2 -l.

2. (i) ~; 1 - (210g 2)-1; 1 - 6rr- 2 ; (e 2 - 3)/(e 2 - 1). (ii) 1; 1; 1; 1 and 2. (iii) It is the case that lP'(X even) = L:~IlP'(X = 2k), and the answers are therefore (a) ~, (b) 1 - (log 3)/(log 4), (c) (d) We have that

!.

00

22k

00

L (2k) ., = i=O L k=1

2i

+ (_2)i 2("')

1

- 1 = z(e

2

+e

-2

) - 1,

I.

so the answer is ~(l - e- 2 ). 3. The number X of heads on the second round is the same as if we toss all the coins twice and count the number which show heads on both occasions. Each coin shows heads twice with probability p2, so lP'(X = k) = G)p2k(1 _ p2)n-k.

4.

Let Dk be the number of digits (to base 10) in the integer k. Then

5. (a) The assertion follows for the binomial distribution because k(n - k) The Poisson case is trivial. (b) This follows from the fact that kg 2: (k 2 - 1)4. (c) The geometric mass function f(k) = qpk, k 2: o.

:s (n - k + l)(k + 1).

3.2 Solutions. Independence 1.

We have that lP'(X

= 1, Z = 1) =

lP'(X

= 1, Y = 1) = 158

! = lP'(X =

l)lP'(Z = 1).

Solutions

Independence

[3.2.2]-[3.2.3]

This, together with three similar equations, shows that X and Z are independent. Likewise, Y and Z are independent. However IP'(X

= 1, Y = 1, Z = -1) = 0 =I ~ = IP'(X = 1)1P'(Y = 1)IP'(Z = -1),

so that X, Y, and Z are not independent. 2.

(a) If x 2: 1,

IP'( min{X, Y} ~ x)

= 1 -IP'(X > x, Y > x) = 1 -IP'(X = 1 - 2- x ·2-x = 1 _ 4- x .

> x)IP'(Y > x)

(b) IP'(Y > X) = IP'(Y < X) by symmetry. Also IP'(Y > X)

+ IP'(Y

+ IP'(Y = X) = 1.

< X)

Since x

we have that IP'(Y > X) (c)

1by part (b).

=

x

1. 00

(d)

IP'(X 2: kY)

=L

IP'(X 2: kY, Y

= y)

y=l 00

=

00

L IP'(X 2: ky, Y = y) = L IP'(X 2: ky)IP'(Y = y) y=l 00

= '"'" '"'" 2- ky - x 2- y = L.J L.J y=lx=O 00

(e)

IP'(X divides Y)

y=l 2

00

2k+l - 1 00

= L IP'(Y = kX) = L k=l

=

.

00

L IP'(Y = kx, X = x)

k=lx=l

ffrkxr

X

k=lx=l

=

f

k:

k=l

.

2 + -1

(0 Let r = min where m and n are coprime. Then IP'(X

3.

1

00

00

k=l

k=l

= rY) = '"'" IP'(X = km, Y = kn) = '"'" rkmrkn = m n . L.J L.J 2 + - 1

(a) We have that IP'(XI < X2 < X3)

=

L

(1- Pl)(1- P2)(1-

P3)P~-1 p~-l p~-l

i<j
= L(1 - Pl)(1 -

. 1

. 1

.

P2)P~- P~- P~

i<j

=

L (1 -

Pl)(1 - P2)P~-I(P2P3)i P3 1 - P2P3

i

(1- Pl)(1 - P2)P2P5 (1 - P2P3)(1 - PIP2P3)

159

[3.2.4]-[3.2.5]

Solutions

Discrete random variables

4. (a) Either substitute P1 = P2 = P3 = ~ in the result of Exercise (3b), or argue as follows, with the obvious notation. The event {A < B < C} occurs only if one of the following occurs on the first round: (i) A and B both rolled 6, (ii) A rolled 6, Band C did not, (iii) none rolled 6. Hence, using conditional probabilities,

In calculating JP>(B < C) we may ignore A's rolls, and an argument similar to the above tells us that

fob\.

Hence JP>(B < C) = 161' yielding JP>(A < B < C) = (b) One may argue as above. Alternatively, let N be the total number of rolls before the first 6 appears. The probability that A rolls the first 6 is JP>(N

E

L

(l, 4, 7, ... }) =

a)k-1

i=

~.

k=1,4,7, ...

Once A has thrown the first 6, the game restarts with the players rolling in order BCABCA. ... Hence the probability that B rolls the next 6 is ~~ also, and similarly for the probability that C throws the third 6. The answer is therefore (~)3. 5. The vector ( - X r : I follows. Let X

:s r :s n) has the same joint distribution as (X r

+ 2 and Y + 2 have joint mass function f, where !i,j

: I

:s r :s n), and the claim

is the (i, j)th entry in the matrix

1

12

U

1

6

1

12

*) 6 1 12

'

l:si,j:s3.

Then

JP>(X

= -1) = JP>(X = 1) = JP>(Y = -1) = JP>(Y = 1) = ~, JP>(X = 0) = JP>(Y = 0) = ~, 1 JP>(X + Y = -2) = 1 6 of- 12 = JP>(X +2 Y = ).

160

Expectation

Solutions

[3.3.1]-[3.3.3]

3.3 Solutions. Expectation 1.

(a) No!

=

(b) Let X have mass function: f(-l)

E (X) -- -91

~, f(~)

=

~, f(2)

1 -_ -91

+ 92 + 98 -_

=

~. Then

+ "98 + "92

) -- E( 1I X.

2. (a) If you have already j distinct types of object, the probability that the next packet contains a different type is (c - j) I c, and the probability that it does not is j I c. Hence the number of days required has the geometric distribution with parameter (c - j)/c; this distribution has mean cl(c - j). (b) The time required to collect all the types is the sum of the successive times to collect each new type. The mean is therefore c-l

c

~_c_. =c~~.

j=o

3.

c- }

k=l

k

(a) Let Iij be the indicator function of the event that players i and j throw the same number. Then 6

= JJ!'(lij =

E(li})

1)

=~

OY = i,

i

=I

j.

i=l

The total score of the group is S

= Li<j

Iij, so

E(S) =

~ E(lij) = ~ (n) . . . 6 2

1<]

We claitn that the family {Iij : i < j} is pairwise independent. The crucial calculation for this is as follows: if i < j < k then 6

E(li}Ijk)

= JJ!'(i, j, andk throw same number) = ~ (i)3 =

l6 = E(lij)E(ljk)·

r=l

Hence var(S)

= var (

~ Iij ) = ~ var(lij) = (;) var(l12) i)·

1<]

i

1<]

by symmetry. But var(l12) = (1 (b) Let Xi} be the common score of players i and j, so that Xij = 0 if their scores are different. This time the total score is S = Li <j Xij, and E(S)

= (;) E(X 12) = (;) ~ . ~ =

172 (; ).

The Xij are not pairwise independent, and you have to slog it out thus: var(S)

= E {(

~

Xi)

1<]

r} -

E(S)2

~ (;).(X1 + (;). (X12 X 23) + { (;)' 2)

=~~(;)+::2(;). 161

(;) - (;) }. (xi2l' -

C7,)' (;)'

[3.3.4]-[3.4.1]

Solutions

Discrete random variables

4. The expected reward is 2::~1 2- k ·2k = fee is 2::~1 2- k u(2k ). For example, if u(k) fee is c

~ Z-k(1 _ L

00. If your utility function is u, then your 'fair' entrance = c(1

Z-ak)

- k- a ) for k ::: I, where c, ct > 0, then the fair

=c

k=1

(I _2a+I-1 I ).

This fee is certainly not 'fair' for the person offering the wager, unless possibly he is a noted philanthropist. 5.

We have that E(X a )

6.

Clearly

= 2::~1 x a /{x(x +

I)}, which is finite if and only if ct < 1.

var(a + X) = E( {(a + X) - E(a + X) }2) = E({X - E(X)}2) = var(X). 7.

For each r, bet {l + n(r)}-I on horse r. If the rth horse wins, your payoff is {nCr) + I}{I + = I, which is in excess of your total stake 2::k{n(k) + I}-I.

n(r)}-I

8. We may assume that: (a) after any given roll of the die, your decision whether or not to stop depends only on the value V of the current roll; (b) if it is optimal to stop for V = r, then it is also optimal to stop when V > r. Consider the strategy: stop the first time that the die shows r or greater. Let S (r) be the expected score achieved by following this stategy. By elementary calculations, S(6) = 6 ·lP'(6 appears before I) + I .lP'(l appears before 6) =

i,

i.

and similarly S(5) = 4, S(4) = 4, S(3) = Ii, S(2) = The optimal strategy is therefore to stop at the first throw showing 4, 5, or 6. Similar arguments may be used to show that 'stop at 5 or 6' is the rule to maximize the expected squared score. 9.

Proceeding as in Exercise (8), we find the expected returns for the same strategies to be: S(6) =

i - 3c,

S(5) = 4 - 2c,

S(4) = 4 - ~c,

If c = ~,it is best to stop when the score is at least 4; if c least 3. The respective expected scores are and

i

¥.

S(3) =

=

1f - ~c,

S(2) =

i-c.

I, you should stop when the score is at

3.4 Solutions. Indicators and matching 1. Let Ij be the indicator function of the event that the outcome of the (j + I)th toss is different from the outcome of the jth toss. The number R of distinct runs is given by R = I + 2::;~t Ij. Hence E(R) = 1+ (n - I)E(lJ) = 1 + (n - 1)2pq,

where q = 1- p. Now remark that Ij and h are independent if jj - kl > I, so that

E{(R - 1)2}

= E{ (

n-I 2} ~ Ij) = (n -

I)E(lJ) + 2(n - 2)E(llh)

J=I

+ {(n - 1)2 - (n - I) - 2(n - 2) }E(lI)2.

162

Solutions [3.4.2]-[3.4.5]

Indicators and matching

var(R)

= var(R -

I)

=

(n - I)E(l1)

+ 2(n -

2)E(l1h) - {(n - I)

+ 2(n -

2) }E(ld 2

= 2pq(2n - 3 - 2pq(3n - 5)).

2. The required total is T = L:}=1 Xi, where Xi is the number shown on the ith ball. Hence E(T) = kE(X1) = ~k(n + I). Now calculate, boringly,

E{

(t 1=1

Xi

r}

= kE(Xy) + k(k k

= -n

{--..2 L.J J 1

I)E(X1 X2)

k(k - I) 2'" .. L.J IJ

+ n(n -

1)

.

.

I>J

= ~ {j-n(n + I)(n + 2) -1n(n + I)} +

k(k - I) n Lj{n(n+I)-j(j+I)} n(n - 1) j=1

= i,k(n + 1)(2n + I) + bk(k -

1)(3n

+ 2)(n + 1).

+ I)}

= -(z(n

Hence var(T) = ken 3.

+ I){ i,(2n + I) + b(k -

1)(3n

+ 2) -

,tk(n

+ I)k(n -

k).

Each couple survives with probability

so the required mean is n

(I _ ~) (I __2nm2n

I

).

4. Any given red ball is in urn R after stage k if and only if it has been selected an even number of times. The probability of this is

and the mean number of such red balls is n times this probability. 5.

Label the edges and vertices as in Figure 3.1. The structure function is t(X)

= X5 + (l

- X5){ (l - X1)X4 [X3

+ (1 -

X3)X2 X 6]

+X1 [X2

163

+ (1- X2)(X3(X6 + X4(1- X6)))]}.

[3.4.6]-[3.4.9]

Solutions

Discrete random variables

2

3

s

Figure 3.1. The network with source s and sink t. For the reliability, see Problem (1.8.19a).

6. The structure function is I{S~k}' the indicator function of {S ~ k} where S reliability is therefore LI=k (7) pi (1 _ P )n-i . 7.

11 E I. There must exist one or more possible values of N L which

Let Ir be the indicator function that the rth pair have opposite polarity, so that X

We have that lP'(Ir !(n-l).

9.

The

Independently colour each vertex livid or bronze with probability ~ each, and let L be the random

set of livid vertices. Then ENL = are at least as large as its mean. S.

= L~=I XC.

=

1)

= ~

and lP'(Ir

=

Ir+1

=

1)

=

= 1 + L~:t I r .

!, whence EX = ~(n + 1) and var X =

(a) Let Ai be the event that the integer i remains in the ith position. Then

lP'( UAr) = LlP'(Ar) r=1

r

= n ·1- -

n

(n)

LlP'(Ar r<s

n As) + ... + (-I)n-IlP'(nAr) r

1 +···+(-1)n-I -1. 2 n(n - 1) n!

Therefore the number M of matches satisfies l i n 1 lP'(M = 0) = - - - + ... + (-1) - . 2! 3! n!

Now lP'(M

= r) = =

(~)lP'(r given numbers match, and the remaining n -

r are deranged)

n! (n-r)!(1 1 n-r 1 ) - ---+···+(-1) --. r!(n - r)! n! 2! 3! (n - r)!

(b) n+1

dn+1

=L

#{derangements with 1 in the rth place}

r=2

= n{ #{derangements which swap 1 with 2}

+ #{derangements in which 1 is in the 2nd place and 2 is not in the 1st place}} = ndn-I + ndn , 164

Solutions

Dependence

[3.5.1]-[3.6.2]

where # A denotes the cardinality of the set A. By rearrangement, d n+1 - (n + 1)dn = - (dn - ndn-l). Set Un = d n - ndn-l and note that U2 = 1, to obtain Un = (_1)n, n 2: 2, and hence dn

n!

n!

2!

3!

= - - - + ... + (-1)

nn!

-. n!

Now divide by n! to obtain the results above.

3.5 Solutions. Examples of discrete variables 1. There are n! / (n 1 ! n2! ... nt!) sequences of outcomes in which the ith possible outcome occurs nj times for each i. The probability of any such sequence is P~ 1 p;2 ... p~1 , and the result follows. 2.

The total number H of heads satisfies

= x) = ~ 'Jl'(H = x I N = n)'Jl'(N = n) = ~ : 00

'Jl'(H

00

('Ap)Xe-)..P

=

x!

00

L

n=x

(

)

pX (1 _ p)n-x

+, 'An -)..

{'A(1 _ p)}n-x e-)..(1-p) (n-x)! .

The last summation equals 1, since it is the sum of the values of the Poisson mass function with parameter 'A (1 - p).

3.

dpn/d'A

= Pn-l

- Pn where P-I

= O.

Hence (d/d'A)'Jl'(X ::0 n)

= Pn('A).

4. The probability of a marked animal in the nth place is a/b. Conditional on this event, the chance of n - 1 preceding places containing m - 1 marked and n - m unmarked animals is

a-I) (b-a) /(b-I), ( m-I n-m n-I as required. Now let Xj be the number of unmarked animals between the j - Ith and jth marked animals, if all were caught. By symmetry, EXj = (b - a)/(a + 1), whence EX = m(EXI + 1) = m(b + 1)/(a + 1).

3.6 Solutions. Dependence 1. Remembering Problem (2.7.3b), it suffices to show that var(aX var(X), var(Y) < 00. Now,

var(aX + bY)

+ bY)

<

00

if a, b

E

lR and

= E ({ax + bY - E(aX + bY) }2) = a 2 var(X) + 2ab cov(X, Y) + b2 var(y) ::0 a2 var(X) + 2abJvar(X) var(Y) + b2 var(y) = (aJvar(X) + bJvar(Y))2

where we have used the Cauchy-Schwarz inequality (3.6.9) applied to X - E(X), Y - E(Y). 2. Let Ni be the number of times the ith outcome occurs. Then Ni has the binomial distribution with parameters n and Pi.

165

[3.6.3]-[3.6.8] 3.

For x

Solutions

Discrete random variables

= 1, 2, ... , 00

JP(X

= x) = L

= x, Y = y)

JP(X

y=1

=

ooC{

L"2 y=l

(x

+y -

I I } l)(x + + + + 1) y) - (x

C+ ="2C(l - +1)

= 2x(x

~

1)

x

1

y)(x

'

and hence C = 2. Clearly Y has the same mass function. Finally lE(X) the covariance does not exist. 4.

i (u + v) + i lu -

Max{u, v) =

lE(max{X 2 , y2))

y

= 2::~1 (x + 1)-1 = 00, so '

vi, and therefore

i lE(X 2 + y 2) + ilEl(X - y)(X + Y)I ::: 1 + 1VlE( (X - Y)2)lE( (X + Y)2) =

= 1 + h/(2 - 2p)(2

+ 2p) = 1 +

p,

where we have used the Cauchy-Schwarz inequality. 5.

(a) log y ::: y - 1 with equality if and only if y = 1. Therefore, lE(lO

fY(X»)
g fx(X)

with equality if and only if fy = fx. (b) This holds likewise, with equality if and only if f (x, y) say that X and Y are independent.

-1]

=0

,

= fx (x) fy (y) for all x, y, which is to

6. (a) a + b + c = lE{l{x>y} + I{y>Z} + I{z>X}) = 2, whence min{a, b, c) ::: ~. Equality is attained, for example, if the vector (X, Y, Z) takes only three values with probabilities f(2, 1,3) = f(3, 2,1) = f(l, 3,2) = (b) JP(X < Y) = JP(Y < X), etc.

1.

(c) We have that c 7.

= a = p and b = 1 -

p2. Also supmin{p, (1- p2»)

= 1(v's -

1).

We have for 1 ::: x ::: 9 that

fx(x)

=

t (1 + 1+ ) = IT (1 + _1_) = (1 + ~) . +

y=o

log

log

lOx

y

y=o

j

T T

(ii) fx+y(r)

T craT cra 2 ="" . . = --, L...J J' (r - J)'. r'. j=o'

r::: 1.

166

log

lOx

y

x

Solutions

Conditional distributions and conditional expectation

(iii) E(X + Y - 1)

[3.7.1]-[3.7.4]

~ cr(r - 1)(2aY = L...J f = 2a. Now E(X) = E(Y), and therefore E(X) = a + '1'1 r. r= 1

3.7 Solutions. Conditional distributions and conditional expectation 1.

(a) We have that E(aY

+ bZ I X = x) = L(ay + bz)lP'(Y = y, Z = z I X = x) Y,z

= a LYlP'(Y

= y, Z = z I X = x) + b LZlP'(Y = y, Z = z I X = x)

y,z

Y,z

=aLYlP'(Y=y I X=x)+bLzlP'(Z=z

I X=x).

y

Parts (b)-(e) are verified by similar trivial calculations. Turning to (f),

<{ECY I X,Z) IX

~x} ~ ~ {~Y'CY ~ Y I X ~x,Z ~z)'CX ~x,Z ~z I X ~X)} = LLylP'(Y = y,X =x,Z = z) . lP'(X =X, Z =z) z y lP'(X =x,Z =z) lP'(X =x) = LYlP'(Y

= y I X = x) = E(Y I X = x)

y

= E{ E(Y I X) IX =

x, Z = z},

by part (e).

2. If ¢ and 1/1 are two such functions then E ((¢ (X) - 1/1 (X))g (X)) = 0 for any suitable g. Setting g(X) = l{x=xJ for any x E lR such that lP'(X = x) > 0, we obtain ¢(x) = 1/1 (x). Therefore lP'(¢(X) = 1/I(X)) = 1. 3. We do not seriously expect you to want to do this one. However, if you insist, the method is to check in each case that both sides satisfy the appropriate definition, and then to appeal to uniqueness, deducing that the sides are almost surely equal (see Williams 1991, p. 88). 4.

The natural definition is given by

Now, var(Y)

= E({Y -

EY}2) = E [E( {Y - E(Y I X)

= E(var(Y

+ E(Y I X) -

Ey}21 X)]

I X)) + var (E(Y I X))

since the mean of E(Y I X) is EY, and the cross product is, by Exercise (Ie),

I

2E[E( {Y - E(Y I X) }{E(Y I X) - EY} X)] =2E[{E(Y

167

I X)-EY}E{Y-E(Y I X)IX}] =0

[3.7.5]-[3.7.10]

Solutions

since E{Y - E(Y

5.

I X) IX}

Discrete random variables

= E(Y

I X) -

I X)

E(Y

o.

=

We have that lP'(T > t + r) L lP'(T > t + r IT> t) = L . r=O r=O lP'(T > t) 00

E(T - tiT> t) =

00

N-t N - t - r

(a)

E(T - tiT> t) = ~ L r=O

(b)

E(T - tiT> t) =

= i(N - t

N-t

+ 1).

2-(t+r)

00

L 2=t = 2. r=O

6.

Clearly E(S I N =n)

= E(tXi) =

p.,n,

1=1

and hence E(S I N) = p.,N. It follows that E(S) = E{E(S I N)} = E(p.,N).

7. A robot passed is in fact faulty with probability rr = {¢(1 - 8)}/(1 - ¢8). Thus the number of faulty passed robots, given Y, is bin(n - Y, rr), with mean (n - Y){¢(l - 8)}/(l - ¢8). Hence E(X

I y) =

Y

+

(n - y)A.(l - 8) 'I'

.

1-¢8

8. (a) Let m be the family size, ¢r the indicator that the rth child is female, and p.,r the indicator of a male. The numbers G, B of girls and boys satisfy m

B =

L p.,r,

E(G) = im = E(B).

r=1

(It will be shown later that the result remains true for random m under reasonable conditions.) We have not used the property of independence. (b) With M the event that the selected child is male,

E(G

I M)

L ¢r

m-l

= E(

)

= i(m - 1) = E(B).

r=1

The independence is necessary for this argument.

9. Conditional expectation is defined in terms of the conditional distribution, so the first step is not justified. Even if this step were accepted, the second equality is generally false. 10. By conditioning on X n -l, EXn

= E[E(Xn I Xn-d] = E[p(Xn- l + 1) + (1 -

where Xn has the same distribution as X n . Hence EX n = (1 EXI = p-l.

168

P)(Xn - l

+ 1 + Xn)]

+ EXn-d/p.

Solve this subject to

Solutions

Sums of random variables

[3.8.1]-[3.8.5]

3.8 Solutions. Sums of random variables 1.

By the convolution theorem, lP'(X

+ Y = z) = LlP'(X = k)lP'(Y = z -

k)

k

k+1

if

(m + I)(n + I) (m 1\ n) + I (m

+ I)(n + I)

m+n+l-k (m

+ I)(n + I)

°: :

k :::: m

1\

n,

if m

1\

n < k < m

V

n,

if m

V

n :::: k :::: m

+ n,

where m 1\ n = minIm, n} and m v n = maxIm, n}. 2.

If z 2: 2,

lP'(X

c

00

+ Y = z) =

L lP'(X = k, Y = z -

k)

= z(z

k=l

+ I)

.

Also, if z 2: 0, 00

L lP'(X = k + z, Y = k)

= z) =

lP'(X - Y

k=l

I

00

- C -

L (2k + z -

-c::-:---:-:-:::-:----:-:-::-::----::-:-

OO{ = iC { ; 1

= :1

By symmetry, if z :::: 0, lP'(X 3.

L a(1 - a/- f3(l _

z-l

1

1)(2k

k=l

?;

I

1)(2k

+ z)

- (2k

I}

+ z)(2k + z + I)

(_I)r+l

00

C

+z -

(2k

+ z)(2k + z + I)

+ z)(r + z + I)' Y = z) = lP'(X - Y = -z) = lP'(X (r

P.{(I

(3)z-r-l = ap

-

(I

p.)Z-l p

~l

-

-

a

Y

= Izl).

)Z-l}

.

a-f3

4. Repeatedly flip a coin that shows heads with probability p. Let Xr be the number of flips after the r - lth head up to, and including, the rth. Then Xr is geometric with parameter p. The number of flips Z to obtain n heads is negative binomial, and Z = 2:~=1 Xr by construction. 5. Sam. Let Xn be the number of sixes shown by 6n dice, so that X n+ 1 = Xn + Y where Y has the same distribution as X 1 and is independent of Xn. Then, 6

lP'(Xn+l 2: n

+ I) = L

lP'(Xn 2: n

+I-

r)lP'(Y

= r)

r=O 6

= lP'(Xn

2: n)

+L

[lP'(Xn 2: n

+I

- r) -lP'(Xn 2: n)]lP'(Y = r).

r=O

We set g(k) = lP'(Xn = k) and use the fact, easily proved, that g(n) 2: g(n - I) 2: ... 2: g(n - 5) to find that the last sum is no bigger than 6 g(n) L(r - 1)lP'(Y

= r) = g(n) (lE(Y) -

r=O

169

I).

[3.8.6]-[3.9.2]

Solutions

Discrete random variables

The claim follows since E(Y)

L

=

1.

00

6.

(i) LHS =

ng(n)e-A)..n In! =)..

n=O

L

00

()-A g n e

n=1

(n - I)!

)..n-I = RHS.

(ii) Conditioning on Nand XN. LHS

= E(E(Sg(S) I N)) = E{ NE(XN g(S) IN)}

-A)..n

=L(:-I)!

J

((n-I )) dF(x) LXr+x

xE g

r=1

n =)..

J

xE(g(S

+ x)) dF(x) =

RHS.

3.9 Solutions. Simple random walk 1. (a) Consider an infinite sequence of tosses of a coin, anyone of which turns up heads with probability p. With probability one there will appear a run of N heads sooner or later. If the coin tosses are 'driving' the random walk, then absorption occurs no later than this run, so that ultimate absorption is (almost surely) certain. Let S be the number of tosses before the first run of N heads. Certainly lP'(S > N r) .::: (l - pN)', since N r tosses may be divided into r blocks of N tosses, each of which is such a run with probability pN. Hence lP'(S = s) .::: (l - pN) Ls / N J, and in particular E(Sk) < 00 for all k ::: 1. By the above argument, E(Tk) < 00 also.

2.

If So = k then the first step X I satisfies

lP'(XI = 1)lP'(W I XI = 1)

lP'(XI = 11 W) =

lP'(W)

PPk+1 = --. Pk

Let T be the duration of the walk. Then

h = E(T I So = k, = E(T

=

(1

W)

I So = k, W, XI = 1)lP'(XI = 1 I So = k, W) + E(T I So = k, W, XI = -1)lP'(XI = -1 I So =

+ h+d

Pk+IP Pk

= 1 + PPk+lh+1 Pk

+

+ (1 + 1k-I)

k, W)

(1 _ Pk+IP) Pk

(Pk - PPk+I) h-I. Pk

as required. Certainly 10 = O. If P = ~ then Pk = 1 - (kiN), so the difference equation becomes (N - k - l)h+1 - 2(N - k)h

for 1 .::: k .::: N - 1. Setting Uk

= (N -

+ (N -

k

+ l)h-1 = 2(k -

N)

k)h. we obtain

Uk+1 - 2Uk

+ Uk-I = 2(k -

N),

with general solution Uk = A + Bk - ~(N - k)3 for constants A and B. Now Uo = UN = 0, and therefore A = ~N3, B = _~N2. implying that h = ~{N2 - (N - k)2}, 0.::: k < N.

170

Solutions [3.9.3]-[3.10.3]

Random walk: counting sample paths

3.

The recurrence relation may be established as in Exercise (2). Set Uk the fact that Pk = (pk - pN)j(1 - pN) where p = qjp, to obtain pUk+1 - (1 - r)uk

+ qUk-1 = pN _

=

(pk - pN)h and use

pk.

The solution is Uk

=

A

+ Bpk + k(pk + pN), p-q

for constants A and B. The boundary conditions, 4.

uo

= UN = 0, yield the answer.

Conditioning in the obvious way on the result of the first toss, we obtain Pmn

= PPm-l,n + (1- P)Pm,n-l,

ifm,n:::: 1.

The boundary conditions are PmO = 0, POn = 1, if m, n :::: 1. 5.

Let Y be the number of negative steps of the walk up to absorption. Then lE(X

+ y)

= Dk and

if the walk is absorbed at N,

N - k

X-Y= { -k

if the walk is absorbed at O.

Hence lE(X - Y) = (N - k)(1 - Pk) - kPb and solving for lEX gives the result.

3.10 Solutions. Random walk: counting sample paths 1.

Conditioning on the first step X I ,

lP'(T

= 2n) = =

1lP'(T

= 2n I XI

1f -I (2n -

1)

= 1)

+ 1lP'(T = 2n I XI

+ 1fI (2n -

=-1)

1)

where fb(m) is the probability that the first passage to b of a symmetric walk, starting from 0, takes place at time m. From the hitting time theorem (3.10.14),

fI(2n -1)

=

f_I(2n -1)

=

_1-lP'(S2n_1 2n-l

=

1) = _1_ (2n -1)T(2n- l l, 2n-l n

which therefore is the value of lP'(T = 2n). For the last part, note first that lP'(T = 2n) = 1, which is to say that lP'(T < 00) = 1; either appeal to your favourite method in order to see this, or observe that lP'(T = 2n) is the coefficient of s2n in the expansion of F (s) = 1 - ~. The required result is easily obtained by expanding the binomial coefficient using Stirling's formula.

Ef

2.

By equation (3.10.13) of PRP, for r :::: 0,

lP'(Mn

= r) = lP'(Mn

2: r) -lP'(Mn 2: r

+ 1)

+ 1) + lP'(Sn = r) - 2lP'(Sn :::: r + 2) -lP'(Sn = r + 1) + lP'(Sn = r + 1) max{lP'(Sn = r), lP'(Sn = r + I)}

= 2lP'(Sn 2: r

= lP'(Sn =

= r)

since only one of these two terms is non-zero. 3. By considering the random walk reversed, we see that the probability of a first visit to S2n at time 2k is the same as the probability of a last visit to So at time 2n - 2k. The result is then immediate from the arc sine law (3.10.19) for the last visit to the origin.

171

[3.11.1]-[3.11.4]

Solutions

Discrete random variables

3.11 Solutions to problems (a) Clearly, for all a, b

1.

JP>(g(X)

E

JR,

= a, h(Y) = b) =

JP>(X=x,Y=y) x,y: g(x )=a,h(y )=b

L

JP>(X

= x)JP>(Y = y)

x,y: g(x)=a,h(y)=b

L

= x)

JP>(X

x:g(x)=a

L

JP>(Y

= y)

y:h(y)=b

= JP>(g(X) = a)JP>(h(Y) = b).

(b) See the definition (3.2.1) of independence. (c) The only remaining part which requires proof is that X and Y are independent if fx,Y(x, y) = g(x )h(y) for all x, y E JR. Suppose then that this holds. Then fx(x) = L

fx,Y(x, y) = g(x) Lh(y),

y

fy(y)

=L

fx,Y(x, y) = h(y) Lg(x).

x

y

x

Now 1 = Lfx(x)

=

Lg(x) Lh(y),

x

x

y

so that fx(x)fy(y)

= g(x)h(y) L

g(x) L

x

h(y)

2. If lE(X2) = Lx x 2JP>(X = x) = 0 then JP>(X Therefore, if var(X) = 0, it follows that JP>(X - lEX

3.

= g(x)h(y) =

fx,Y(x, y).

y

= x) = 0 for = 0) = 1.

x =1= O. Hence JP>(X = 0) = 1.

(a)

lE(g(X»)=LyJP>(g(X)=Y)=L y

y

L

yJP>(X=x)

x:g(x)=y

= Lg(x)JP>(X=x) x

as required. (b)

lE(g(X)h(Y») = L

g(x)h(y)fx,Y(x, y)

by Lemma(3.6.6)

x,y

= Lg(x)h(Y)fx(x)fy(Y)

by independence

x,y

= Lg(x)fx(x) Lh(y)fy(y) x

4.

(a) Clearly fx(i) = fy(i)

=

(b) (X+y)(Wl) = 3, (X+y)(W2)

~ for i

= lE(g(X»lE(h(Y».

y

=

1,2,3.

= 5, (X+Y)(W3) = 4,andtherefore fX+y(i)

= ~ fori = 3, 4, 5.

(c) (Xy)(Wl) = 2, (Xy)(W2) = 6, (XY)(w3) = 3, and therefore fxy(i) = ~ for i = 2,3,6.

172

Solutions

Problems

(d) Similarly !x/y(i) = (e)

j

[3.11.5]-[3.11.8]

for i = ~> ~, 3.

=

!YIZ(2 12)

lP'(Y=2,Z=2) lP'(Z = 2)

lP'(Wl)

= lP'(WI U wz)

1

2'

and similarly !Ylz(3 12) = ~, !Ylz(l 11) = 1, and other values are O. (f) Likewise !Zly(2 I 2) = !Zly(2 I 3) = !Zly(l 11) = 1. 5.

(a)

L00

k

n=l n(n

+ 1)

I}

00{1- - -- = k, and therefore k = 1. =k L n=l n n+1

(b) 2::~1 kn Cl = k~( -a) where ~ is the Riemann zeta function, and we require a < -1 for the sum to converge. In this case k = ~(_a)-l. 6.

(a) We have that lP'(X + Y

n

= n) = L lP'(X = n -

n

k)lP'(Y

k=O

e-A-J.L ~ =-~

(n) n-k k A J-i k=O k

n!

(b)

lP'(X=k

I X+Y=n)=

-AAn-k e-J.LJ-ik _e---,----- . - k=O (n - k)! k!

= k) = L

e-A-J.L(A + J-i)n = -------'-n!

lP'(X = k, X + Y = n) ------lP'(X + Y = n)

= lP'(X = k)lP'(Y = n lP'(X + Y = n) Hence the conditional distribution is bin(n, A/(A + J-i». 7.

k)

=

(n) AkJ-in-k . k (A + J-i)n

(i) We have that lP'(X=n+k

IX

lP'(X = n + k, X> n) >n)= - - - - - - - - ' - lP'(X > n) )n+k-l (1 p - p . 2::~n+l p(l - p)J-l

= p(l -

pi- 1

= lP'(X = k).

(ii) Many random variables of interest are 'waiting times', i.e., the time one must wait before the occurrence of some event A of interest. If such a time is geometric, the lack-of-memory property states that, given that A has not occurred by time n, the time to wait for A starting from n has the same distribution as it did to start with. With sufficient imagination this can be interpreted as a failure of memory by the process giving rise to A. (iii) No. This is because, by the above, any such process satisfies G(k + n) = G(k)G(n) where G(n) = lP'(X > n). Hence G(k + 1) = G(1)k+l and X is geometric.

8.

Clearly, k

lP'(X + Y

= k) = LlP'(X = k -

j, Y

= j)

j=o

=

t (: j=o

k

.)pk-jqm-k+j J

= pq k m+n-k

(~)pjqn-j J

~ ( m. ) (n) ~ k . j=o

- J

173

J

= pq k m+n-k

(m k+ n)

[3.11.9]-[3.11.13]

Discrete random variables

+ n, p).

which is bin(m 9.

Solutions

Turning immediately to the second request, by the binomial theorem,

as required. Now, JP'(N even)

L (:) pk (1 _ p )n-k

=

k even

=

H(p + 1 -

p)n

+ (1

- p - p)n}

=

!{l + (1 -

2p)n}

in agreement with Problem (1.8.20).

10. There are (i) ways of choosing k blue balls, and (~~k) ways of choosing n - k red balls. The total number of ways of choosing n balls is (~), and the claim follows. Finally, n) b! (N - b)! (N - n)! JP'(B=k)= ( k (b-k)!' (N-b-n+k)!' N!

=(:){!.b~I ... b-~+I} x { N -; b ... N - b - ; ---+ (:) pk(1 - p)n-k

+ k + 1} {

as N ---+

Z...

N -;

+ 1} -I

00.

11. Using the result of Problem (3.11.8),

12. (a) E(X) = c + d, E(Y) = b + d, and E(Xy) = d, so cov(X, Y) = d - (c + d)(b + d), and X and Y are uncorrelated if and only if this equals O. (b) For independence, we require f(i, j) = JP'(X = i)JP'(Y = j) for all i, j, which is to say that a

= (a + b)(a + c),

b

=

(a

+ b)(b + d),

c

= (c + d)(a + c),

d

= (b + d)(c + d).

Now a + b + c + d = I, and with a little work one sees that anyone of these relations implies the other three. Therefore X and Y are independent if and only if d = (b + d)(c + d), the same condition as for uncorrelatedness.

13. (a) We have that 00

E(X)

=

00

m-I

00

00

L mJP'(X = m) = L L JP'(X = m) = L L m=O

m=O n=O

n=Om=n+1

174

00

JP'(X

= m) =

L JP'(X > n). n=O

Solutions [3.11.14]-[3.11.14]

Problems (b) First method. Let N be the number of balls drawn. Then, by (a), r

E(N)

=L

r

JP>(N > n)

=L

n=O r

JP>(first n balls are red)

n=O r

r- I

= ~ b +r b +r -

=~~ (b+r)!

f;:o

r- n 1 ... b

+r -

+1 r r! (b + r - n)! n + 1 = ~ (b + r)! (r - n)!

(n +b b) = b+b+r+1 1 '

where we have used the combinatorial identity E~=o the simple identity (r'::'l) summation, we find that

+ G) = ctl)

ct

b

)

= rt!tl).

To see this, either use

repeatedly, or argue as follows. Changing the order of

~xr~ (n ;b) = 1 ~x ~xn(n ;b) = (1 -

x) -(b+2)

=

'f (b + + 1) xr

r=O

r b+1

by the (negative) binomial theorem. Equating coefficients of xr, we obtain the required identity. Second method. Writing m(b, r) for the mean in question, and conditioning on the colour of the first ball, we find that b r m(b,r) = -b- + {I +m(b,r -l)}-b-· +r +r With appropriate boundary conditions and a little effort, one may obtain the result. Third method. Withdraw all the balls, and let Ni be the number of red balls drawn between the ith and (i + l)th blue ball (No = N, and Nb is defined analogously). Think of a possible 'colour sequence' as comprising r reds, split by b blues into b + 1 red sequences. There is a one-one correspondence between the set of such sequences with No = i, N m = j (for given i, j, m) and the set of such sequences with No = j, Nm = i; just interchange the 'Oth' red run with the mth red run. In particular E(No) = E(Nm ) for all m. Now No + Nl + ... + Nb = r, so that E(Nm ) = r/(b + 1), whence the claim is immediate. (c) We use the notation just introduced. In addition, let Br be the number of blue balls remaining after the removal of the last red ball. The length of the last 'colour run' is Nb + B r , only one of which is non-zero. The answer is therefore r/(b + 1) + b/(r + 1), by the argument of the third solution to part (b).

14. (a) We have that E(Xk) = Pk and var(Xk) = Pk(1- Pk), and the claims follow in the usual way, the first by the linearity of E and the second by the independence of the Xi; see Theorems (3.3.8) and (3.3.11). (b) Let s = Ek Ph and let Z be a random variable taking each of the values Pl, P2, ... , Pn with equal probability n-l. Now E(Z2) - E(Z)2 = var(Z) 2: 0, so that

Lk

1 (1) L

;;Pf 2:

;;Pk

k

2

s2

= n2

with equality if and only if Z is (almost surely) constant, which is to say that Pl Hence

175

= P2 = ... = pn.

[3.11.15]-[3.11.18]

Solutions

Discrete random variables

with equality if and only if PI = P2 = ... = Pn. Essentially the same route may be followed using a Lagrange multiplier. (c) This conclusion is not contrary to informed intuition, but experience shows it to be contrary to much uninformed intuition.

15. A matrix V has zero determinant if and only if it is singular, that is to say if and only if there is a non-zero vector x such that xVx' = O. However,

Hence, by the result of Problem (3.11.2), L:k Xk(Xk - EXk) is constant with probability one, and the result follows.

16. The random variables X cov(X

+ Y,

+ Y and IX -

IX - YI)

YI are uncorrelated since

= E{ (X + y)IX - YI} = i + i-I . ~ = o.

E(X

+ y)E(IX -

YI)

However,

i = Jl»(X + Y = 0, IX so that X

+ Y and IX -

YI

= 0) # Jl»(X + Y = O)Jl»(IX -

YI

= 0) = i

'1 = k,

Y I are dependent.

17. Let h be the indicator function of the event that there is a match in the kth place. Then Jl»(h 1) = n -I, and for k # j, Jl»(h Now X

=

1

= 1, Ij = 1) = Jl»(lj = 1 I h = 1)Jl»(h = 1) = -n(n - 1)

= L:Z=I h, so that E(X) = L:Z=1 n- I = 1 and var(X) = E(X2) - (EX)2

=E

(t

h) 2 - 1

I

= tE(h)2

+ LE(ljh) j#

I

1= 1+

2(n) 2

1

n(n - 1)

- 1 = 1.

We have by the usual (mis)matching argument of Example (3.4.3) that 1 n-r (_I)i

Jl»(X

= r) = - ,L.. "'" r.i=o

which tends to e-I/r! as n ---+

.,

'

0.::: r .::: n -

2,

l.

00.

18. (a) Let YI, Y2, ... , Yn be Bernoulli with parameter P2, and Z I , Z2, ... , Zn Bernoulli with parameter PI/ P2, and suppose the usual independence. Define Ai = Yj Zj, a Bernoulli random variable that has parameter Jl»(Aj = 1) = Jl»(Yj = 1)Jl»(Zj = 1) = Pl' Now (AI, A2, ... , An) .::: (YI, Y2, ... , Yn ) so that f(A) .::: f(Y). Hence e(PI) = E(f(A)) .::: E(f(Y)) = e(P2). (b) Suppose first that n = 1, and let X and X' be independent Bernoulli variables with parameter p. We claim that {J(X) - f(X') }{g(X) - g(X')} :::: 0;

176

Solutions [3.11.19]-[3.11.19]

Problems

to see this consider the three cases X = X', X < X', X > X' separately, using the fact that J and g are increasing. Taking expectations, we obtain lE({f(X) - J(X'»){g(X) - g(X')}) ::: 0, which may be expanded to find that

o ::s lE(J(X)g(X») -lE(J(X')g(X») -lE(J(X)g(X'») + lE(J(X')g(X'») = 2{ lE(J(X)g(X»)

-lE(f(X»lE(g(X»}

by the properties of X and X'. Suppose that the result is valid for all n satisfying n < k where k ::: 2. Now lE(J(X)g(X»)

= lE {lE(J(X)g(X) IXI, X2,···, Xk-I)};

here, the conditional expectation given X I, X2, ... , Xk-I is defined in very much the same way as in Definition (3.7.3), with broadly similar properties, in particular Theorem (3.7.4); see also Exercises (3.7.1, 3). If X I, X2, ... , Xk-I are given, then J(X) and g(X) may be thought of as increasing functions of the single remaining variable Xb and therefore

by the induction hypothesis. Furthermore

are increasing functions ofthek-l variables X I, X2, ... , Xk-I, implying by the induction hypothesis that lE(f' (X)g' (X» ::: lE(f' (X»lE(g' (X». We substitute this into (*) to obtain lE(J(X)g(X») ::: lE(f' (X»lE(g' (X»

= lE(f(X»lE(g(X»

by the definition of J' and g'.

19. Certainly R(p) = lE(lA) Differentiating, we obtain

=

2:wIA(W)lP'(W) andlP'(w)

=

pN(w)qm-N(w) wherep+q

=

1.

Applying the Cauchy-Schwarz inequality (3.6.9) to the latter covariance, we find that R' (p) ::s (pq)-I y'var(lA) var(N). However IA is Bernoulli with parameter R(p), so thatvar(lA) = R(p)(lR(p», and finally N is bin(m, p) so that var(N) = mp(1 - p), whence the upper bound for R'(p) follows. As for the lower bound, use the general fact thatcov(X + Y, Z) = cov(X, Z)+cov(Y, Z) to deduce that cov(lA, N) = cov(lA, IA) + cov(lA, N - IA). Now IA and N - IA are increasing functions of w, in the sense of Problem (3.11.18); you should check this. Hence cov(lA, N) ::: var(lA) + 0 by the result of that problem. The lower bound for R' (p) follows.

177

[3.11.20]-[3.11.21]

Solutions

Discrete random variables

20. (a) Let each edge be blue with probability PI and yellow with probability P2; assume these two events are independent of each other and of the colourings of all other edges. Call an edge green if it is both blue and yellow, so that each edge is green with probability PI P2' If there is a working green connection from source to sink, then there is also a blue connection and a yellow connection. Thus lP'(green connection)

.:s lP'(blue connection, and yellow connection) = lP'(blue connection)lP'(yellow connection)

so that R(PI P2) .:s R(PI )R(P2)' (b) This is somewhat harder, and may be proved by induction on the number n of edges of G. If n = I then a consideration of the two possible cases yields that either R(p) = I for all p, or R(p) = P for all p. In either case the required inequality holds. Suppose then that the inequality is valid whenever n < k where k ~ 2, and consider the case when G has k edges. Let e be an edge of G and write w (e) for the state of e; w (e) = I if e is working, and w(e) = 0 otherwise. Writing A for the event that there is a working connection from source to sink, we have that R(pY) = lP'pY (A

I w(e)

+ lP'pY (A I w(e) = 0)(1 l)Y pY + lP'p(A I w(e) = O)Y (1 -

= 1)pY

.:s lP'p(A I w(e) =

pY) pY)

where lP'a is the appropriate probability measure when each edge is working with probability a. The inequality here is valid since, if w(e) is given, then the network G is effectively reduced in size by one edge; the induction hypothesis is then utilized for the case n = k - 1. It is a minor chore to check that x YpY

+ yY (1 -

p)Y

.:s

{xp

+ y(1 -

if x

p)}Y

~

y

~

0;

to see this, check that eq uality holds when x = y ~ 0 and that the derivative of the left-hand side with respect to x is at most the corresponding derivative of the right-hand side when x, y ~ O. Apply the latter inequality with x = lP'p(A I w(e) = 1) and y = lP'p(A I w(e) = 0) to obtain R(pY)

.:s

{lP'p(A

I w(e)

= l)p

+ lP'p(A I w(e) =

0)(1 - p)

y

= R(p)Y.

21. (a) The number X of such extraordinary individuals has the bin(107 , 10- 7 ) distribution. Hence EX = 1 and lP'(X > 1 I X > 1) = lP'(X > 1) = 1 - lP'(X = 0) -lP'(X = 1) lP'(X > 0) --I---lP'-(X-=--:-O-)-7

7

I - (1_10- 7 )10 _107 .10- 7 (1_10- 7 )10 -1

1 - (1- 10-7 )107 11

2e- 1

-e

-1::::: 0.4.

(Shades of (3.5.4) here: X is approximately Poisson distributed with parameter 1.) (b) Likewise lP'(X > 2 I X ~ 2) ::::: (c) Provided m

«

1 - 2e- 1 1- 2e-

1 e- 1

~::::: 0.3.

N = 107 ,

lP'(X = m) =

1 N! (N m!(N-m)!

178

)m (1- N1 )N-m : : : em-!I,

Solutions [3.11.22]-[3.11.23]

Problems

the Poisson distribution. Assume that "reasonably confident that n is all" means that lP'(X > n I X ::: n):,,= rforsomesuitablesmallnumberr. AssumingthePoissonapproxirnation,lP'(X > n):,,= rlP'(X::: n) if and only if 00 1 00 1 e- 1 ' " _ < re- 1 ' " - . L...J k! L...J k! k=n+l k=n For any given r, the smallest acceptable value of n may be determined numerically. If r is small, then very roughly n ~ l/r will do (e.g., if r = 0.05 then n ~ 20). (d) No level p of improbability is sufficiently small for one to be sure that the person is specified uniquely. If p = 10- 7a, then X is bin (l 0 7 ,10- 7a), which is approximately Poisson with parameter a. Therefore, in this case, lP'(X > 1 I X ::: 1)

~

1 - e- a - ae- a

1- e- a

= p,

say.

An acceptable value of p for a very petty offence might be p ~ 0.05, in which case a ~ 0.1 and so p = 10- 8 might be an acceptable level of improbability. For a capital offence, one would normally require a much smaller value of p. We note that the rules of evidence do not allow an overt discussion along these lines in a court of law in the United Kingdom.

22. The number G of girls has the binomial distribution bin(2n, p). Hence

L

lP'(G ::: 2n - G) = lP'(G ::: n) = 2n (2n) k pkq 2n-k k=n

:"=

where we have used the fact that With p

en

~

2n) L...Jpq k 2n-k ( n k=n

:"=

=

(2n) pq n n - q- , n q-p

enn) for all k.

= 0.485 and n = 104 , we have using Stirling's formula (Exercise (3.10.1)

that

~{(l-0.03)(l + O.03)r 00.5.0135

2n)pnqn_q_:,,= ( n q- p y(mr)

4

=

0.515 - ( 1 - - 9)10 < 1.23 x 10-5 .

3,j7r

104

-

It follows that the probability that boys outnumber girls for 82 successive years is at least (1 - 1.23 x 10- 5 )82 ::: 0.99899.

23. Let M be the number of such visits. If k =f. 0, then M ::: 1 if and only if the particle hits 0 before it hits N, an event with probability 1 - kN- 1 by equation (1.7.7). Having hit 0, the chance of another visit to 0 before hitting N is 1 - N- 1, since the particle at 0 moves immediately to 1 whence there is probability 1 - N- 1 of another visit to 0 before visiting N. Hence lP'(M::: r

I So =

k) =

(1- Nk) (1 - N1 )r-l '

r ::: 1,

so that lP'(M = j

I So =

k) = lP'(M ::: j

=

I So =

k) -lP'(M ::: j

(1 _~) (1 _~ y-l ~, 179

+ 1 I So =

0)

[3.11.24]-[3.11.27]

Solutions

Discrete random variables

24. Either read the solution to Exercise (3.9.4), or the following two related solutions neither of which uses difference equations. First method. Let Tk be the event that A wins and exactly k tails appear. Then k < n so that lP'(A wins) = I:;::JlP'(Tk). However lP'(Tk) is the probability that m + k tosses yield m heads, k tails, and the last toss is heads. Hence

whence the result follows. Second method. Suppose the coin is tossed m + n - 1 times. If the number of heads is m or more, then A must have won; conversely if the number of heads is m - 1 or less, then the number of tails is n or more, so that B has won. The number of heads is bin(m + n - 1, p) so that lP'(A wins)

L

= m+n-l

(+ 1) m

; -

pkqm+n-I-k.

k=m

25. The chance of winning, having started from k, is 1 _ (qjp)k 1 - (qjp)N

I

which may be written as

I

1- (qjp)'1k

1 + (qjp)'1k .

IN 1 - (qjp) 2.

1 + (qjp)'1N

I

'

see Example (3.9.6). If k and N are even, doubling the stake is equivalent to playing the original game with initial fortune k and the price of the Jaguar set at iN. The probability of winning is now

i

I k

1 - (qjp)'1 I

1 - (qjp)'1N

'

which is larger than before, since the final term in the above display is greater than 1 (when p <

i,

If p = doubling the stake makes no difference to the chance of winning. If p > to decrease the stake.

i).

i, it is better

26. This is equivalent to taking the limit as N ---+ 00 in the previous Problem (3.11.25). In the limit when p #the probability of ultimate bankruptcy is

i,

={

lim (qjpi - (qjp)N N--+oo 1 - (qjp)N where p

+q =

1. If p

=

(qjp)k 1

·f P > 2' I

1

·f

1

I

P < 2'

!, the corresponding limit is limN--+oo (1 -

kj N)

=

1.

27. Using the technique of reversal, we have that lP'(Rn = Rn-l

+ 1) =

lP'(Sn-1 #- Sn, Sn-2 #- Sn, ... , So #- Sn)

=lP'(Xn #-O,Xn-1 +Xn #-O, ... ,XI +···+Xn #-0)

= lP'(XI

#- 0, X2

+ Xl

#- 0, ... , Xn

+ ... + Xl

#- 0)

= lP'(SI #- 0, S2 #- 0, ... , Sn #- 0) = lP'(SIS2··· Sn #- 0).

It follows that lE(Rn) = lE(Rn-I>

+ lP'(SI S2· .. Sn

#- 0) for n 2: 1, whence

L

1 -lE(Rn) = -1 { 1 + n lP'(SI S2· .. Sm #- 0) } ---+ lP'(Sk #- 0 for all k 2: 1) n n m=l

180

Solutions

Problems

[3.11.28]-[3.11.29]

since IP'(SIS2· .. Sm '" 0) .t- IP'(Sk '" 0 for all k ~ 1) as m --+ 00. There are various ways of showing that the last probability equals Ip - q I, and here is one. Suppose p > q. If X I = 1, the probability of never subsequently hitting the origin equals I - (q I p), by the calculation in the solution to Problem (3.11.26) above. If X I = -1, the probability of staying away from the origin subsequently is O. Hence the answer is p (l - (q I p» + q . 0 = p - q. If q > p, the same argument yields q - p, and if p

= q = i the answer is o.

28. Consider first the event that M2n is first attained at time 2k. This event occurs if and only if: (i) the walk makes a first passage to S2k (> 0) at time 2k, and (ii) the walk thereafter does not exceed S2k. These two events are independent. The chance of (i) is, by reversal and symmetry, IP'(S2k-1 < S2k,S2k-2 < S2k, ... ,So < S2k)

= IP'(X2k > 0, X2k-1 + X2k > 0, ... , Xl + ... + X2k = IP'(XI > 0, Xl + X2 > 0, ... , Xl + ... + X2k > 0)

= IP'(Si

> 0 for I ~ i ~ 2k)

= ~1P'(S2k = 0)

= ilP'(Si '" 0 for 1 ~ i

> 0)

~ 2k)

by equation (3.10.23).

As for the second event, we may translate S2k to the origin to obtain the probability of (ii):

where we have used the result of Exercise (3.10.2). The answer is therefore as given. The probabilities of (i) and (ii) are unchanged in the case i = 2k + I; the basic reason for this is that S2r is even, and S2r+ I odd, for all r. 29. Let Uk = IP'(Sk = 0), fk = IP'(Sk = 0, Si '" 0 for 1 ~ i < k), and use conditional probability (or recall from equation (3.10.25» to obtain n

=

U2n

L U2n-2khk· k=l

Now NI = 2, and therefore it suffices to prove that E(Nn ) = E(Nn-d for n ~ 2. Let N~_l be the number of points visited by the walk Sl, S2, ... , Sn exactly once (we have removed So). Then

N~_l Nn

Hence, writing an

=

{

+I

= So for exactly one k in {I, 2, ... , n},

N~_l - I

if Sk

N~_l

otherwise.

= IP'(Sk '" 0 for

E(Nn )

if Sk '" So for 1 ~ k ~ n,

I ~ k ~ n),

= E(N~_I) + an = E(Nn-d

-1P'(Sk

+ an - {h a n-2 + f4 a n-4 + ... + hLn/2J}

where LxJ is the integer part of x. Nowa2m even, then

If n = 2k

= So exactly once)

=

a2m+1

+ 1 is odd, then

181

=

U2m by equation (3.10.23). If n

=

2k is

[3.11.30]-[3.11.34]

Solutions

Discrete random variables

In either case the claim is proved. 30. (a) Not much. (b) The rhyme may be interpreted in any of several ways. Interpreting it as meaning that families stop at their first son, we may represent the sample space of a typical family as {B, GB, a2B, ... }, with lP'(GnB) = 2-(n+l). The mean number of girls is E~l nlP'(anB) = E~l nZ-(n+l) = 1; there is exactly one boy. The empirical sex ratio for large populations will be near to 1: 1, by the law of large numbers. However the variance of the number of girls in a typical family is var(#girls) = 2, whilst var(#boys) = 0; # A denotes the cardinality of A. Considerable variation from 1: 1 is therefore possible in smaller populations, but in either direction. In a large number of small populations, the number of large predominantly female families would be balanced by a large number of male singletons.

31. Any positive integer m has a unique factorization in the form m = TIi integers m(l), m(2), .... Hence,

lP'(M=m)=

.) IT (1- 1) ~m(i) 1 =c (IT.

IT lP' ( .

N(I)=m(l) = .

.

I

where C = TIi(1-

7PI

I

i:

fJ

). Now

Em lP'(M =

PI

m) = 1, so that c- 1 =

°

pr(i)

for non-negative

C

_m(i))fJ Pi = mfJ

I

Em m-fJ .

°

32. Number the plates 0, 1,2, ... , N where is the starting plate, fix k satisfying < k :s N, and let Ak be the event that plate number k is the last to be visited. In order to calculate lP'(Ak), we cut the table open at k, and bend its outside edge into a line segment, along which the plate numbers read k, k + 1, ... , N, 0, 1, ... , k in order. It is convenient to relabel the plates as -(N + 1 - k), -(N k), ... , -1,0, 1, ... , k. Now Ak occurs if and only if a symmetric random walk, starting from 0, visits both -(N - k) and k - 1 before it visits either -(N + 1 - k) or k. Suppose it visits -(N - k) before it visits k - 1. The (conditional) probability that it subsequently visits k - 1 before visiting -(N + 1 - k) is the same as the probability that a symmetric random walk, starting from 1, hits N before it hits 0, a probability of N- 1 by (1.7.7). The same argument applies if the cake visits k - 1 before it visits -(N - k). Therefore lP'(Ak) = N- 1. 33. With j denoting the jth best vertex, the walk has transition probabilities Pjk = (j - 1)-1 for 1 :s k < j. By conditional expectation,

1 rj =

1+

-.-

j-l

L rb

'1 =0.

J - 1 k=l

Induction now supplies the result. Since rj log (~).

~

log j for large j, the worst-case expectation is about

34. Let Pn denote the required probability. If (mr, mr+l) is first pair to make a dimer, then ml is ultimately uncombined with probability Pr-l. By conditioning on the first pair, we find that Pn = (PI + P2 + ... + Pn-2)/(n - 1), giving n(Pn+l - Pn) = -(Pn - Pn-l). Therefore, n!(Pn+l - Pn) = (_1)n-l (P2 - PI) = (_1)n, and the claim follows by summing. Finally, n

lEUn = LlP'(mr is uncombined) = Pn

+ PIPn-l + ... + Pn-lPl + Pn,

r=l

since the rth molecule may be thought of as an end molecule of two sequences oflength rand n - r + 1. Now Pn ~ e- 1 as n ~ 00, and it is an easy exercise of analysis to obtain that n-llEUn ~ e- 2 .

182

Solutions

Problems

[3.11.35]-[3.11.36]

35. First,

)...k

= (LPir = i

L Pr,Pr2'" Prk :::: k! L Pr,Pr2'" Prk' r, ,r2, ... ,rk (r" ... ,rk)

where the last summation is over all subsets {r1, ... , rkl of k distinct elements of {I, 2, ... , n}. Secondly,

)...k:::Ok!

L (q, ... ,rkl

:::0 k!

pr,pr2",prk+(~)Lpr i

L Pr, Pr2'" Prk r" ... ,rk

L Pr,Pr2"'Prk-2 r" ... ,rk_2

+ (~) mtXPi (~pj )k-1 J

Hence

By Taylor's theorem applied to the function log (1-x), there exist Or satisfying 0 < Or < {2(1-c)2)}-1 such that n

II (1- Pr) = II exp{-Pr r=1

OrP;}

= exp{ -)... -

r

Finally,

The claim follows from (*) and (**). 36. It is elementary that

We write l' - lE(l') as the mixture of indicator variables thus: N

-Y - lE(Y) -

= """' L -Xr r=1 n

( Ir - -n ) . N

It follows from the fact n

lE(Ii1j)

= N'

n-I

N-I'

183

i =f. j,

)",0 (mtx Pi) }.

[3.11.37]-[3.11.38]

Solutions

Discrete random variables

Hnc

c y

1- c

Hnc

H

nC

................................. (1- y)a

Hnc Figure 3.2. The tree of possibility and probability in Problem (3.11.37). The presence of the disease is denoted by C, and hospitalization by H; their negations are denoted by C and B.

that

37. The tree in Figure 3.2 illustrates the possibilities and probabilities. If G contains n individuals, X is bin(n, yp + (1- y)a) and Y is bin(n, yp). It is not difficult to see that cov(X, Y) = nyp(1- v) where v = yp + (1- y)a. Also, var(y) = nyp(1- yp) and var(X) = nk(l- v). The result follows from the definition of correlation. 38. (a) This is an extension of Exercise (3.5.2). With IP'n denoting the probability measure conditional on N = n, we have that

184

Problems

Solutions

[3.11.39]-[3.11.39]

where s = n - 2:7=1 ri. Therefore, 00

IP'(Xi

= ri for 1 :::: i :::: k) = L

IP'n(Xi

= ri for 1 :::: i

n=O

=

IT

{vri !(iYi,e-V!(i)} rl

;=1

:::: k)IP'(N

f

~(kns e-v(1-F(k».

S

v (1 -

s=O

•

= n)

s.

The final sum is a Poisson sum, and equals 1. (b) We use an argument relevantto Wald's equation. The event {T :::: n -I} depends only on the random variables Xl, X2, ... , Xn-b and these are independent of X n . It follows that Xn is independent of the event {T 2: n} = {T :::: n -l}c. Hence, 00

E(S)

00

00

= LE(X;Im:iJ) = LE(Xi)E(I{T~i}) = LE(Xi)IP'(T 2: i) ;=1

i=l 00

;=1

00

t

00

= L v!(i) LIP'(T t=i i=l

= t) = v LIP'(T = t) L

!(i)

;=1

t=l

00

= v LIP'(T = t)F(t) = E(F(T)). t=l

39. (a) Place an absorbing barrier at a + 1, and let Pa be the probability that the particle is absorbed at O. By conditioning on the first step, we obtain that

Pn

1

= --2 (Po + PI + P2 + ... + Pn+1),

1:::: n :::: a.

n+

The boundary conditions are Po = 1, Pa+1 = O. It follows that Pn+1 - Pn for 2:::: n :::: a. We have also that P2 - PI = PI - 1, and

Pn+1 - Pn Setting n

=

i(n

= a we obtain that -Pa =

+ I)! (P2 -

i(a

PI)

+ I)! (PI

=

i(n

+ 1)! (PI

= (n + I)(Pn

- Pn-d

- po)·

- 1). By summing over 2:::: n < a, a

Pa - PI

= (PI

- PO)

+ i(Pl

- PO) Lj!, j=3

and we eliminate PI to conclude that

Pa

(a + I)! = 4 + 3! + 4! + ... + (a + I)! .

It is now easy to see that, for given r, Pr = Pr(a) --+ 1 as a --+ 00, so that ultimate absorption at 0 is (almost) certain, irrespective of the starting point. (b) Let Ar be the probability that the last step is from 1 to 0, having started at r. Then Al = ~(1

(r

+ 2)A r =

Al

+ Al + A2), + A2 + ... + Ar+ 1, 185

r 2: 2.

[3.11.40]-[3.11.40]

Solutions

It follows that

Discrete random variables

1 Ar - Ar-l = --(Ar+l - Ar), r+1

whence

1 A3 - A2 = 4.5 ... (r

Letting r -+

00,

+ 1) (Ar+l

- Ar),

r 2: 3.

we deduce that A3 = A2 so that Ar = A2 for r 2: 2. From (**) with r = 2, A2 = ~Al>

and from (*) Al = ~. (c) Let /-Lr be the mean duration of the walk starting from r. As above, /-La /-Lr

1

= 1 + --2(/-Ll + /-L2 + ... + /-Lr+l), r+

= 0, and

r 2: 1,

whence /-Lr+l - /-Lr = (r + 1)(/-Lr - /-Lr-l) - 1 for r 2: 2. Therefore, V r+l = (/-Lr+l - /-Lr)/(r satisfies v r + 1 - vr = -1/ (r + I)! for r 2: 2, and some further algebra yields the value of /-L 1.

+ I)!

40. We label the vertices 1, 2, ... , n, and we let 7T be a random permutation of this set. Let K be the set of vertices v with the property that 7T(W) > 7T(V) for all neighbours W of v. It is not difficult to see that K is an independent set, whence a(G) 2: IKI. Therefore, a(G) 2: JEIKI = 2:v lP'(v E K). For any vertex v, a random permutation 7T is equally likely to assign any given ordering to the set comprising v and its neighbours. Also, v E K if and only if v is the earliest element in this ordering, whence lP'(v E K) = 1/(dv + 1). The result follows.

186

4 Continuous random variables

4.1 Solutions. Probability density functions I

1. (a) {x(l - x)}-2: is the derivative of sin- l (2x - 1), and therefore C (b) C = 1, since

1

00 -00

(c) Substitute v = (1

exp(-x-e-X)dx= lim K-+oo

= JT-I.

[exp(-e-X)l~K=l.

+ x 2 )-1 to obtain

where B(·, -) is a beta function; see paragraph (4.4.8) and Exercise (4.4.2). Hence, if m >

C- I

2.

= B( I

m _ I) 2' 2

r( I )r(m

=

2

r(m)

i,

I)

- "2

(i) The distribution function Fy of Y is Fy(y)

= JP>(Y

~

y)

= JP>(aX

So, differentiating, fy(y) = a-I fx(y/a). (ii) Certainly LX(x) = JP>(-X ~ x)

y)

= JP>(X

= JP>(X

::: -x)

~

~

=

y/a)

=

Fx(y/a).

I-JP>(X

~

-x)

since JP>(X = -x) = O. Hence f-x(x) = fx(-x). If X and -X have the same distribution function then f-x(x) = fx(x), whence the claim follows. Conversely, if fx( -x) = fx(x) for all x, then, by substituting u = -x, JP>(-X

~ y) = JP>(X::: -y) =

whence X and 3.

1

00

-Y

fx(x)dx

=

1 Y

fx(-u)du

-00

=

1 Y

fx(u)du

-00

-x have the same distribution function.

Since a ::: 0, f ::: 0, and g ::: 0, it follows that af

187

+ (1 -

a)g ::: O. Also

= JP>(X ~

y),

[4.1.4]-[4.2.4]

Solutions

Continuous random variables

If X is a random variable with density f, and Y a random variable with density g, then a f +(l-a) g is the density of a random variable Z which takes the value X with probability a and Y otherwise. Some minor technicalities are necessary in order to find an appropriate probability space for such a Z. If X and Yare defined on the probability space (Q, :F, JP», it is necessary to define the product space (Q, .1", JP» x (1;, g" Q) where 1; = {O, I}, g, is the set of all subsets of 1;, and Q(O) = a, Q(l) = 1 - a. For w x a E Q x 1;, we define Z(w x a) =

X(w) { Y(w)

= 0,

if a

ifa=l.

1 F(x + h) - F(x) f(x) (a) By definition, r(x) = lim = ---'--h-l-O h 1 - F(x) 1 - F(x) (b) We have that

4.

H(x) =..!!...x dx

{2. Jo{X x

r(Y)d } = r(x) _ ~ Y x x2

which is non-negative if r is non-increasing. (c) H(x)/x is non-decreasing if and only if, for 0

1

-H(ax) ax

(X r(y)dy = ~

Jo

x2

.:s a .:s 1,

1

.:s -H(x)

which is to say that -a-1log[I - F(ax)]

(x[r(x) -r(y)]dy,

Jo

for all x 2: 0,

x

.:s -log[I -

F(x)]. We exponentiate to obtain the claim.

.:s aH(t) for 0 .:s a .:s

(d) Likewise, if H(x)/x is non-decreasing then H(at) H(at) + H(t - at) .:s H(t) as required.

1 and t 2: 0, whence

4.2 Solutions. Independence 1. Let N be the required number. Then JP>(N distribution with mean [1 - F(K)r 1.

2.

= n) =

F(K)n-l [1- F(K)] for n 2: 1, the geometric

.:s v if and only if X .:s v and Y .:s v. Hence, by independence, JP>(max{X, Y} .:s v) = JP>(X .:s v, Y .:s v) = JP>(X .:s v)JP>(Y .:s v) = F(v)2.

(i) Max{X, Y}

Differentiate to obtain the density function of V = max{X, Y}. (ii) Similarly minIX, Y} > u if and only if X >

U

and Y > u. Hence

JP>(U.:s u) = I-JP>(U > u) = I-JP>(X > u)JP>(Y > u) = 1- [1- F(u)]2,

giving fu(u) = 2f(u)[1 - F(u)]. 3. The 24 permutations of the order statistics are equally likely by symmetry, and thus have equal probability. Hence JP>(XI < X2 < X3 < X4) = and JP>(XI > X2 < X3 < X4) = ~, by enumerating the possibilities.

-i4,

4. JP>(Y(y) > k) Therefore,

=

F(yi for k 2: 1. Hence EY(y)

=

F(y)/[l - F(y)] --+

00

as y --+

JP>(Y(y) > EY(y») = {I - [1 - F(y)l}LF(y)/[l-F(Y)lJ

~ exp {I- [1- F(y)]

188

l

F(y)

1 - F(y)

J}

--+

e- 1 as y --+

00.

00.

Solutions

Expectation

[4.3.1]-[4.3.5]

4.3 Solutions. Expectation 1.

(a) E(Xa) =

Jooo x a e-x dx < 00 if and only if a

> -1.

(b) In this case

if and only if -1 < a < 2m - 1. 2.

We have that 1 =E

(~1 Xi)

= tE(Xi/Sn).

Sn

i=l

By symmetry, E(Xi/Sn) = E(XliSn) for all i, and hence 1 = nE(XliSn). Therefore m

E(SmISn) = LE(Xi/Sn) = mE(X1/Sn ) = min. i=l

3.

Either integrate by parts or use Fubini's theorem: r

roo xr-1lP'(X > x) dx = r roo x r - 1 { roo

k

=

f(y) d } dx

h= Y roo f(y) {[Y rx r - 1 dX} dy = roo yr f(y)dy. k

Jy=O

Jx=o

Jo

An alternative proof is as follows. Let Ix be the indicator of the event that X > x, so that Jooo Ix dx = X. Taking expectations, and taking a minor liberty with the integral which may be made oo rigorous, we obtain EX = Jo E(lx) dx. A similar argument may be used for the more general case. 4. We may suppose without loss of generality that f-L = 0 and a = 1. Assume further that m > 1. In this case, at least half the probability mass lies to the right of 1, whence E(XI[x:c: m }) 2: ~. Now 0= E(X) = E{X[I[X:c:m}

+ I[X<m}]}' implying that E(XI[X<m}) :s -i. Likewise, 1 E(X 2 I[X:c:m} ) 2: 2'

By the definition of the median, and the fact that X is continuous, E(X I X < m)

:s

-1,

E(X 2

IX

< m)

:s

1.

It follows that var( X I X < m) :s 0, which implies in turn that, conditional on (X < m j, X is almost surely concentrated at a single value. This contradicts the continuity of X, and we deduce that m :s 1. The possibility m < -1 may be ruled out similarly, or by considering the random variable - X.

5. It is a standard to write X = X+ - X- where X+ = max{X, OJ and X- = - min{X, OJ. Now X+ and X- are non-negative, and so, by Lemma (4.3.4), f-L = E(X) = E(X+) - E(X-) =

fooo lP'(X > x) dx - fooo lP'(X < -x) dx

= roo[l_F(x)]dx_ rooF(_x)dx= rOO[l_F(X)]dx_jO

Jo

Jo

Jo

F(x)dx.

-00

It is a triviality that f-L

=

foIL F(x)dx

+ foIL[l-

F(x)]dx

and the equation follows with a = f-L. It is easy to see that it cannot hold with any other value of a, since both sides are monotonic functions of a.

189

[4.4.1]-[4.4.6]

Solutions

Continuous random variables

4.4 Solutions. Examples of continuous variables 1.

(i) Integrating by parts, r(t)

= fooo xt-1e- x dx = (t -

1)

fooo x t - 2 e- x dx = (t -

1)r(t - 1).

= (n-1)r(n-I) = ... = (n-I)! r(1) wherer(1) = 1. = x, that

Ifn is an integer, then it follows that r(n) (ii) We have, using the substitution u 2 r(1)2 =

{foOO x-~ e- x dx

r

{foOO 2e- u2 du

=

rr Jr=O Je=o j

= 4 (00 e-u2 du (00 e-v2 dv = 4 (00

Jo

r

Jo

2

e- r2 rdrd8

=n

as required. For integral n, r(n

2.

1 + I) = (n -

1

1

I)r(n - I)

= ... = (n -

1

3

1

1

z)(n - I)'" Ir(I)

=

(2n) ! '4 n n! 'lin.

By the definition of the gamma function, r(a)r(b)

Now set u = x

= fooo xa-1e- x dx fooo yb-Ie-y dy = fooo fooo e-(x+y\a-Iyb-I dxdy.

+ y, v =

x/ex

+ y), obtaining

(00 (I e-uua+b-Iva-I(1_ v)b-I dvdu

Ju=o Jv=o

= fooo ua+b-1e- u du fol

va - I (1- v)b-I dv

=

rea

+ b)B(a, b).

If g is strictly decreasing then JP'(g(X) ~ y) = JP'(X :::: g-I (y)) = 1 - g-I (y) so long as 1. Therefore JP'(g(X) ~ y) = 1 - e- Y , y :::: 0, if and only if g-I(y) = e- Y , which is to say that g(x) = -logx for 0 < x < 1.

3.

o ~ g-I(y) ~ 4.

We have that JP'(X < x) =

-

x

1

1

1

j-00 n(l + u ) du = -2 + -n tan-I x. 2

Also,

is finite if and only if la I < 1. 5.

Writing

fy(y)

y)

= ~fx(logy) = _I_e-~(logy)2, y

6.

~

y.J2ir

= JP'(X ~ logy) =
00.

Integrating by parts, LHS = i : g(x)

{(X - JL)~1J (x: JL)} dx

=-[g(x)a1JC:JL)]~oo + i:gl(x)a1JC:JL) 190

dx=RHS.

Dependence

7.

= afJx f3- 1.

(a) rex)

(b) rex)

= A.

(c) rex) = 8.

Solutions [4.4.7]-[4.5.3]

Aae- Ax A

ae- x

+ IL(I - a)e-/LX + (I - a)e-/LX ' which approaches mintA, IL} as x

-+ 00.

= -xcp. Using this identity and integrating by parts repeatedly,

Clearlyel/ 1- (x) =

00 1x

100 x -100

cp(u)du = -

= cp(x) x

_ cp(x) x3

cp'(u) cp(x) --du = - -

u

x

3cp'(u) du uS

x

=

+

100 x

cp(x) _ cp(x) x x3

cp'(u) -3-du

U

+ 3cp(x) xS

-1 00 x

15cp(u) duo u6

4.5 Solutions. Dependence 1.

(i) As the product of non-negative continuous functions, g(x)

!e- ixi

=

00

I

-00

J2nx- 2

1

f is non- negative and continuous. Also

1 2 2 Y dy

e--':x

=

!e- ixi

if x =f::. 0, since the integrand is the N(O, x- 2 ) density function. It is easily seen that g(O) g is discontinuous, while

1-00 00

g(x) dx =

1-0000 ~e-ixi

= 0, so that

dx = 1.

(ii) Clearly f Q ?: 0 and

00 100 1-00 -00

fQ(x,y)dxdy =

E00 (~r·1

= 1.

Also f Q is the unifonn limit of continuous functions on any subset of ~2 of the fonn [- M, M] x ~; hence f Q is continuous. Hence f Q is a continuous density function. On the other hand

00 1

-00

00

fQ(x, y)dy

=L

(ir g(x -

qn),

n=l

where g is discontinuous at O. (iii) Take Q to be the set of the rationals, in some order. 2. We may assume that the centre of the rod is uniformly positioned in a square of size a x b, while the acute angle between the rod and a line of the first grid is unifonn on [0, ~n]. If the latter angle is () then, with the aid of a diagram, one finds that there is no intersection if and only if the centre of the rod lies within a certain inner rectangle of size (a - r cos ()) x (b - r sin ()). Hence the probability of an intersection is -

2

n~

Iolr/2 {ab 0

(a - r cos (})(b - rSin(})} d(}

= -2r( a + b n~

~r).

3. (i) Let I be the indicator of the event that the first needle intersects a line, and let J be the indicator that the second needle intersects a line. By the result of Exercise (4.5.2), E(l) = E(J) = 2/n; hence Z = I + J satisfies E( ~ Z) = 2/n.

191

[4.5.4]-[4.5.6]

Solutions

Continuous random variables

(ii) We have that

var(~ Z) = HlE( 2) = HlEU)

+ lE(J2) + 2lEU 1)} -lE(~ Z)2

+ lE(J) + 2lEU J)}

4 n

1 4 nn

- 2" = - - 2"

1

+ -lEU J). 2

!-n,

z

In the notation of (4.5.8), if 0 < () < then two intersections occur if < ~ min {sin (), cos ()} or 1 - z < ~ min {sin (), cos ()}. With a similar component when ~ n .::: () < n, we find that lEU 1) = lP'(two intersections) =

~

II

(z,Ii):

dz d()

o
410

=-

1T 2 / ~min{sin(),cos()}d()=-

4101T / 4 sin()d()=-4 ( 1-

n o n

n

0

1r ;); '

",2

and hence

1 r;; n1 - n42 + n(2 - ",2) =

1

var(2 Z ) =

3-,)2 4 - n - - n2'

(iii) For Buffon's needle, the variance of the number of intersections is (2In) - (2In)2 which exceeds var(~ Z). You should therefore use Buffon's cross.

J6

4. (i) Fu(u) = 1-(1-u)(1-u) if 0 < u < 1, and so lE(U) = 2u(1-u) du = j-. (Alternatively, place three points independently at random on the circumference of a circle of circumference 1. Measure the distances X and Y from the first point to the other two, along the circumference clockwise. Clearly X and Y are independent and uniform on [0, 1]. Hence by circular symmetry, lE(U) lE(V - U) = lE(1 - V) = ~.) (ii) Clearly UV = XY, so that lE(UV) = lE(X)lE(Y) = ~. Hence cov(U, V) = lE(UV) -lE(U)lE(V) = ~ -

j-(1 -

~) =

:k;,

since lE(V) = 1 -lE(U) by 'symmetry'.

5.

(i) If X and Y are independent then, by (4.5.6) and independence, lE(g(X)h(Y)) = =

II I

g(x)h(y)fx,y(x,y)dxdy

g(x)fx(x)dx

I

h(y)fy(y)dy=lE(g(X))lE(h(Y)).

(ii) By independence

6. If 0 is the centre of the circle, take the radius OA as origin of coordinates. That is, A = (1,0), B = (1, e), C = (1, <1», in polar coordinates, where we choose the labels in such a way that o .::: e .::: <1>. The pair e, <1> has joint density function f«(), 4» = (2n2)-1 for 0 < () < 4> < 2n. The three angles of ABC are !-e, ~ (<1> - e), n - ~<1>. You should plot in the ()/4> plane the set of pairs «(), 4» such that 0 < () < 4> < 2n and such that at least one of the three angles exceeds xn.

192

Solutions [4.5.7]-[4.6.3]

Conditional distributions and conditional expectation

Then integrate f over this region to obtain the result. The shape of the region depends on whether or not x < ~. The density function g of the largest angle is given by differentiation:

g(x) =

The expectation is found to be

6(3x - 1) { 6(1 - x)

ifl<x
if~.:sx.:sl.

t§-n.

7.

We have that JE(X) = J1" and therefore JE(Xr - K) = O. Furthermore,

8.

The condition is that JE(Y) var(X)

+ JE(X) var(Y) =

O.

9. If X and Yare positive, then S positive entails T positive, which displays the dependence. Finally, S2 = X and T2 = Y.

4.6 Solutions. Conditional distributions and conditional expectation 1.

The point is picked according to the uniform distribution on the surface of the unit sphere, which is to say that, for any suitable subset C of the surface, the probability the point lies in C is the surface integral Ie(4n)-1 dS. Changing to polar coordinates, x = cos () cos cp, y = sin () cos cp, z = sincp, subjectto x 2 + y2 + z2 = 1, this surface integral becomes (4n )-1 Ie I coscpl d() dcp, whence the joint density function of e and is

f«(), cp) =

1 4n

Icpl .:s ~n, O.:s () < 2n.

- I coscpl,

The marginals are then fe«() = (2n)-I, fcf>(cp) = ~I coscpl, and the conditional density functions are

for appropriate () and cpo Thus e and are independent. The fact that the conditional density functions are different from each other is sometimes referred to as 'Borel's paradox'.

2.

We have that

1/I(x)

=

100

-00

y fx,Y(x, y) dy fx(x)

and therefore

JE(1/I(X)g(X») = =

00 100 fx y(x y) 1-00 -09 y "fx (x) g(x)fx(x) dx dy

L: L:

{y g(x)}fx,Y(x, y) dx dy = JE(Y g(X».

3. Take Y to be a random variable with mean 00, say fy(y) = y-2 for 1 .:s y < Then JE(Y I X) = X which is (almost surely) finite.

193

00,

and let X = Y.

[4.6.4]-[4.6.6] 4.

Solutions

Continuous random variables

(a) We have that

so that !Ylx(Y I x) = AeA(x-y), for 0:::: x :::: Y < 00. (b) Similarly, !x(x) = xe- x (y+1) dy = e-X,

fooo

so that !Ylx(Y I x) = xe- xy , for 0:::: Y < 5.

0:::: x <

00,

0:::: x <

00,

00.

We have that IP'(Y=k) = =

1IP'(Y=k I X =x)!x(x)dx = 101 (n) x k (1_x)n-k x a- 1(1- x)b-1 dx

10o

0

(n) B(a +B(a, k, n- k + k

k

B(a,b)

b) .

b)

In the special case a = b = 1, this yields II" Y

(

= k) =

+ 1)r(n - k + 1) = _1_ r(n + 2) n + 1'

(n) r(k k

0:::: k:::: n,

whence Y is uniformly distributed. We have in general that E(Y)

=

1

10o

E(Y I X

na

= x)!x(x)dx = - - , a +b

and, by a similar computation ofE(y2), var(Y) = (a 6.

nab(a + b +n) + b)2(a + b + 1) .

By conditioning on Xl, Gn(x)

= IP'(N

Now Go(v) = 1 for all v

E

> n)

= foX G n-1 (x

- u) du

= foX G n-1 (v) dv.

(0,1], and the result follows by induction. Now, 00

EN= LIP'(N >n)=ex . n=O

More generally, GN(S)

=

00

LsnlP'(N n=l

= n) =

Lsn 00

n=l

(n-1 n) -::..x (n - I)! n!

194

= (s

_1)e SX

+ 1,

Solutions

Functions of random variables

[4.6.7]-[4.7.1]

7. We may assume without loss of generality that EX = EY = O. By the Cauchy-Schwarz inequality, Hence,

8.

One way is to evaluate

Another way is to observe that minty, Z} is exponentially distributed with parameter /-L + v, whence IP'(X < minty, Z}) = A/(A + /-L + v). Similarly, IP'(Y < Z) = /-L/(/-L + v), and the product of these two terms is the required answer.

9.

By integration, for x, y > 0,

whence c = 1. It is simple to check the values of fXIY(x I y) = f(x, y)/ fy(y) and iYlx(Y and then deduce by integration that E(X I Y = y) = Y and E(Y I X = x) = x + 2.

1

I x),

10. We have that N > n if and only if Xo is largest of {Xo, Xl, ... , X n }, an event having probability 1/(n + 1). Therefore, IP'(N = n) = 1/{n(n + 1)} for n ~ 1. Next, on the event {N = n}, Xn is the largest, whence 00

IP'(XN .::: x) =

F(x)n+1

00

L n(n + 1) = n=l L n=l

F(x)n+1 n -

00

L n=l

F(x)n+1 n 1

+

+ F(x),

as required. Finally,

4.7 Solutions. Functions of random variables 1.

We observe that, if 0 .::: u .::: 1, IP'(XY .::: u) = IP'(XY .::: u, Y .::: u) 1

=u+

1 u

+ IP'(XY .::: u, Y

> u) = IP'(Y .::: u)

+ IP'(X .::: u/Y,

Y > u)

u -dy=u(I-logu).

y

By the independence of XY and Z, IP'(XY .::: u, Z2 .::: v) = IP'(XY .::: u)IP'(Z .::: J'V) = uJ'V(1-logu),

195

0 < u, v < 1.

[4.7.2]-[4.7.5]

Solutions

Continuous random variables

Differentiate to obtain the joint density function 10g(1/u)

2Jv '

g(u, v) =

Hence

rr

lP'(XY .::: Z2) =

JJ

0.::: u, v .::: 1.

log(l/u) du dv = 5.

2Jv

O:::;u:::;v:::;l

Arguing more directly, lP'(XY .::: Z2) =

fff

lJ

dxdydz =

~.

O:::;x,y,z:::; 1 xy:::;z2

2.

The transformation x

= uv, y = u -

uv has Jacobian

=II-v v

I

u -u

1= -u.

Hence III = lui, and therefore fu,v(u, v) = ue- u , for 0.::: u < 00,0.::: v .::: 1. Hence U and V are independent, and fv(v) = Ion [0,1] as required. 3.

Arguing directly, 2 lP'(sin X.::: y) = lP'(X .::: sin- 1 y) = - sin- 1 y,

rr

so that fy (y) variables. 4.

=

2/ (rr~), for 0 .::: y

(a) lP'(sin- 1 X .::: y)

O.:::y.:::~rr.

0.::: y

.::: 1,

.::: 1. Alternatively, make a one-dimensional change of

=lP'(X .::: siny) =siny, for 0 .::: y .::: irr. Hence fy(y) =cosy, for

(b) Similarly, lP'(sin- 1 X .::: y) = i(1 -lrr 2 -< y -< lrr. 2 5. Consider the mapping w and Jacobian

+ siny), for -irr .:::

y .::: irr, so that fy(y) = icosy, for

=X,z =(y-px)/~ with inverse x =w,y =pw+zJ1"=P2 1=

I!

VI ~p21 = p.

The mapping is one-one, and therefore W (= X) and Z satisfy

implying that Wand Z are independent N(O, 1) variables. Now {X> 0, Y > O}

=

{W > 0, Z > -Wp / P},

and therefore, moving to polar coordinates, lP'(X>O, Y>O)=

1 in -dO 1 rin 1 -e-'1' rdrdO= r ,=0 2rr Ja 2rr 00

JO=a

196

I 2

Solutions [4.7.6]-[4.7.9]

Functions of random variables where a = - tan -1 (p /

6.

v'1=P2) =

- sin -1 p.

We confine ourselves to the more interesting case when p

#-

1. Writing X = U, Y = pU

+

VI - p2V, we have that U and V are independent N(O, 1) variables. It is easy to check that Y > X if and only if (1 - p) U < v'1=P2 V. Turning to polar coordinates, 00 E(max{X, Y)) = 10 ~ [£"'+1T {prcose +nfl~sine} de + £~1T rcosede] dr

where tan 1/1 =

v'(1 -

p)/(1 + p). Some algebra yields the result. For the second part,

by the symmetry of the marginals of X and Y. Adding, we obtain 2E(max{X, Y)2) = E(X2) + = 2.

E(y2)

7.

We have that )..

lP'(X < Y, Z > z) = lP'(z < X < y) = _ _ e-(A+/L)z = lP'(X < y)lP'(Z > z). )..+fL )..

(a)lP'(X = Z) =lP'(X < y) = - - . )..+fL

(b) By conditioning on Y, lP'( (X - y)+ =

0)

)..

lP'(X - y)+ >

= lP'(X ::: y) = ).. + fL'

w)

= _fL_e->.w )..+fL

for

w>

O.

By conditioning on X, lP'(V > v) = lP'(IX - YI > v) =

10

00

1

00

lP'(Y > v + x)fx(x) dx +

v>

lP'(Y < x - v)fx(x) dx

o.

(c) By conditioning on X, the required probability is found to be

8. that

Either make a change of variables and find the Jacobian, or argue directly. With the convention r2 - u 2 = 0 when r2 - u 2 < 0, we have that

V

F(r,x) f(r,x)

9.

= lP'(R::: r, X::: x) = -2 n

a2 F

=-

arax

=

2r

~'

n V r2 - x 2

jX Vr2 -

u 2 du,

-r

Ixl < r < 1.

As in the previous exercise, lP'(R::: r,

Z::: z) =

3 4n

-

197

jZ -r

n(r 2 - w 2 )dw.

[4.7.10]-[4.7.14]

Solutions

Continuous random variables

= ~r for Izl < r < 1. This question may be solved in spherical polars also. The transformation s = x + y, r = x/(x + y), has inverse x = rs, y = (l - r)s and Jacobian

Hence f(r, z)

10.

J = s. Therefore, fR(r) = _

-

roo

roo

fRs(r,s)ds = fxy(rs,(l-r)s)sds Jo' Jo' 00

10o

,-J...rs

!I.e

Me

AM

-JL(1-r)s d _ s

s -

(Ar+M(l-r)}

O~r~1.

2'

11. We have that

1) =

~ y) = lP' ( X2 ~ ~ -

lP'(Y

2lP' ( X

whence

1

fy(y)=2fx(-V(a/y)-I)=

n

-1) ,

~ -J~

o ~ y ~ a.

..Jya-y ( )'

12. Using the result of Example (4.6.7), and integrating by parts, we obtain

1

00

lP'(X > a, Y > b) =

= [1 -

¢(x) { 1 - (

~) }

(a)][1 - (c)]

+ Jaroo [1 -

dx

(x)]¢

h

(~) 1 - p2

1 _ p2

dx.

Since [1 - (x)]!¢(x) is decreasing, the last term on the right is no greater than

1

1 - (a) ¢(a) a

00

¢(x)¢

b - px)

(

p

Jl=P2 Jl=P2

dx,

which yields the upper bound after an integration.

13. The random variable Y is symmetric and, for a > 0, lP'(Y > a)

by the transformation v

= lP'(0 < = l/u.

X < a

)

Jo

lx-2 2

r

a

= Joo

-v- 2 dv n(l + v-2)'

if x> 1 '

{ ~

ifO~x~1.

= x + y, z = x/(x + y) has inverse x = wz, y = (1 A(AwZ)a-1e-J...wz

f(w, z) =

du n(l + u2)

For another example, consider the density function f(x) =

14. The transformation w J = w, whence

r =

a- 1

-1

r(a)'

W·

A(AW)a+tJ-1e-J...w

r(a

+ f3)

z)w, and Jacobian

A(A(l- z)w)tJ- 1e -J...(1-z)w reB)

za-1(l _ z)tJ- 1 B(a, f3)

w > 0, 0 < Z < 1.

Hence Wand Z are independent, and Z is beta distributed with parameters a and f3.

198

Solutions [4.8.1]-[4.8.4]

Sums of random variables

4.8 Solutions. Sums of random variables 1.

+ Y has density function

By the convolution formula (4.8.2), Z = X

z ~ 0, if A i= 2.

iJ,.

What happens if A =

(Z has a gamma distribution in this case.)

iJ,?

Using the convolution formula (4.8.2), W h~-

1

= aX + f3Y

1

00

-00

na(1

has density function

1

+ (x/a)2)

. nf3(1 + {(w -

x)/f3}2)

b

,

which equals the limit of a complex integral:

1,

lim af3 . _ _ 1_ . 1 R-*oo D n 2 z2 + a 2 (z - w)2

+ f32

dz

where D is the semicircle in the upper complex plane with diameter [ - R, R] on the real axis. Evaluating the residues at z = ia and z = w + if3 yields fw(w) = af32ni {_1_. 1 n2 2ia (ja - w)2

1 - n(a

+ f32

+ _1_ .

1

2if3

(w

1

+ f3)

. 1 + {w/(a

+ f3)}2'

+ if3)2 + a 2

< w <

-CXJ

}

CXJ

after some manipulation. Hence W has a Cauchy distribution also. 3.

Using the convolution formula (4.8.2), fz(z)

4.

=

foZ ~ze-z dy = ~z2e-z,

z ~ O.

Let fn be the density function of Sn. By convolution, -/qe -A1 x f 2 (x ) -,

A 2. -A2- A1

+'A2e -A2 x . - A-1 - -A1- A2

II

2 n A '""' , -Ar X L...JAre - -s- .

s=I As - Ar sir

r=1

This leads to the guess that n

fn(x)

n

A

= LAre-ArX II __s_, r=1

n

~

2,

s=1 As - Ar sir

which may be proved by induction as follows. Assume that (*) holds for n

199

:s N.

Then

[4.8.5]-[4.8.8]

Solutions

Continuous random variables

for some constant A. We integrate over x to find that

1=

N N+l A A L II _ s _ + - , r=l s=l As - Ar AN+l sofr

= N + 1 on solving for A. The density function of X + Y is, by convolution,

and (*) follows with n

5.

h(x) =

{

X

ifO:::x:::1,

2- x

if 1 ::: x ::: 2.

Therefore, for 1 ::: x ::: 2, h(x)

=

1 10 o

h(x-y)dy=

11

(x-y)dy+

Io

1

x -

(2-x+y)dY=i-(x_~)2.

0

x-l

Likewise,

A simple induction yields the last part. 6. The covariance satisfies cov(U, V) = lE(X 2 - y2) = 0, as required. If X and Y are symmetric random variables taking values ±1, then lP'(U = 2, V = 2) = 0

but

lP'(U = 2)lP'(V = 2) > O.

If X and Y are independent N(O, 1) variables, iu, V(u, v) a function of u multiplied by a function of v.

7.

From the representation X

=

1 (2

(4Jr)-l e -4 u

= a pU + a Vf=P2 V, Y = • U, where U

N (0, 1), we learn that

apy

lE(X I Y = y) = lE(apU I U = y/.) = Similarly,

•

2)

+V , which factorizes as and V are independent

.

whence var(X I Y) = 0'2(1 - p2). For parts (c) and (d), simply calculate that cov(X, X + pa., var(X + Y) = 0'2 + 2pa. + .2, and

+ Y)

=

0'2

8. First recall that lP'(lXI ::: y) = 2(y) - 1. We shall use the fact that U = (X + Y)/../2, V = (X - Y) /../2 are independent and N (0, 1) distributed. Let Ll be the triangle ofll~2 with vertices (0,0), (0, Z), (Z, 0). Then lP'(Z :::

z IX

> 0, Y > 0) = 4lP'(X, Y) E Ll) = lP'(IUI :::

= 2{2 (z/..J2) _1}2, 200

z/..J2, IVI

:::

z/..J2)

by symmetry

Solutions [4.9.1]-[4.9.5]

Multivariate normal distribution whence the conditional density function is

fez) =

2J2{2(z/J2) -l}q,(z/J2).

Finally,

E(Z I X > 0, Y > 0) = 2E(X I X > 0, Y > 0)

= 2E(X I X

= 4E(XI{x>O}) = 4

> 0)

00

10o

x 1 2 r;:ce- Zx dx.

'\I2rr

4.9 Solutions. Multivariate normal distribution Since V is symmetric, there exists a non-singular matrix M such that M' = M- l and V = MAM- l , where A is the diagonal matrix with diagonal entries the eigenvalues )'-1, A2, ... , An of V. 1.

1

Let A:Z be the diagonal matrix with diagonal entries

1

v'Il, ../Ai, ... , ..;J:;;; A:Z

is well defined since

1

V is non-negative definite. Writing W = MA 2M', we have that W = W' and also

1

as required. Clearly W is non-singular if and only if A:Z is non-singular. This happens if and only if Ai > for all i, which is to say that V is positive definite.

°

2. By Theorem (4.9.6), Y has the multivariate normal distribution with mean vector 0 and covariance matrix

3. Clearly Y = (X - JL)a' + JLa' where a = (al' a2, ... , an). Using Theorem (4.9.6) as in the previous solution, (X - JL)a' is univariate normal with mean 0 and variance aVa'. Hence Y is normal with mean JLa' and variance aVa'.

4.

Make the transformation u

= x + y, v = x

- y, with inverse x

=

~(u

+ v), y =

~(u - v), so

that IJ I = ~. The exponent of the bivariate normal density function is

1

-

2

2(1 - p )

and therefore U

(x 2 -2pxy+i)=-

= X + Y,

V

=

1

2

4(1 - p )

{u 2(1_p)+v 2(1+p)},

X - Y have joint density

2

feu, v)

1 {u = 4rr~ exp -4(1 + p)

v2}

- 4(1- p)

whence U and V are independent with respective distributions N(O, 2(1 5.

That Y is N(O, 1) follows by showing that lP'(Y

Iyl < a, y

~

~

y) = lP'(X

~

,

+ p)) and N(O, 2(1

y) for each of the cases y

- p )). ~

-a,

a.

Secondly,

p(a)=E(Xy)

=

j-aa

x2q,(x)dx-

j-a -00

x2q,(x)dx-

201

100 x 2q,(x)dx=1-4 100 x 2q,(x)dx.

a

a

[4.9.6]-[4.10.1]

Solutions

Continuous random variables

The answer to the final part is no; X and Y are N(O, 1) variables, but the pair (X, Y) is not bivariate normal. One way of seeing this is as follows. There exists a root a of the equation p(a) = O. With this value of a, if the pair X, Y is bivariate normal, then X and Y are independent. This conclusion is manifestly false: in particular, we have that lP'(X > a, Y > a) =I- lP'(X > a)lP'(Y > a). 6.

Recall from Exercise (4.8.7) that for any pair of centred normal random variables E(X

I y)

= cov(X, Y) Y, varY

var(X

I Y)

= {l - p(X, Y)2} var X.

The first claim follows immediately. Likewise,

7. As in the above exercise, we calculate a = E( X 1 I L;'i X r ) and b = var( X 1 I L;'i Xr ) using the facts that var Xl = Vll, var(L;'i Xi) = L;ij Vij, and COV(XI' L;'i Xr) = L;r VIr· 8.

Let p = lP'(X > 0, Y > 0, Z > 0) = lP'(X < 0, Y < 0, Z < 0). Then

1 - P = lP'({X > O} U {Y > O} U {Z > OJ) = lP'(X > 0) + lP'(Y > 0) + lP'(Z > 0) + P - lP'(X > 0, Y > 0) - lP'(Y > 0, Z > 0) - lP'(X > 0, Z > 0) =

9.

. -1 PI + Sill. -1 P2 + Sill. -1 P3 }] . 23+ p - [34 + 2nI {Sill

Let U, V, W be independent N (0, l) variables, and represent X, Y, Z as X

VI - P[V,

Z = P3 U

+ P2 -

PIP3 V

VI-P[

We have that U = X, V = (Y - PI X)/ conditional variance.

= U,

Y

= PI U +

+

V

I - P[ and E(Z

I X,

Y) follows immediately, as does the

4.10 Solutions. Distributions arising from the normal distribution 1.

First method. We have from (4.4.6) that the

x2 (m) density function is x 2: O.

The density function of Z

= Xl + X2 is, by the convolution formula,

202

Distributions arising from the normal distribution

Solutions [4.10.2J-[4.10.6J

by the substitution u = xlz, where c is a constant. Hence g(z) = an appropriate constant c', as required.

c'z~(m+n)-le-~z

for z 2: 0, for

Second method. If m and n are integral, the following argument is neat. Let Zl, Z2, ... , Zm+n be independent N (0, 1) variables. Then X 1 has the same distribution as Z? + Z~ + ... + Z~, and X2 the same distribution as Z~+l the same distribution as Z?

2.

+ Z~+2 + ... + Z~+n (see Problem (4.14.12». + ... + Z~+n' i.e., the x 2(m + n) distribution.

Hence Xl

+ X2 has

(i) The t(r) distribution is symmetric with finite mean, and hence this mean is O.

(ii) Here is one way. Let U and V be independent x2(r) and X2(s) variables (respectively). Then

by independence. Now U is

-1

E(V)=

10

00

o

r(-!, ~r) and V is n-!, ~s), so that E(U) = rand

1 2-s/ 2 1 s-l _1 v n~s - 1) ---v:Z e:Z dv= v n~s) 2r(1-s)

10 0

00

1

2-:z(s-2)

r(~s - 1)

1 s-2 _1 v

v:Z

1

e:Z dv=--

s- 2

if s > 2, since the integrand is a density function. Hence

Ulr) s E ( Vis =s-2 (iii) If s 3.

.:s 2 then E(V- 1)

Substitute r

=

if s > 2.

00.

= 1 into the t (r) density function.

4. First method. Find the density function of X I Y, using a change of variables. The answer is F(2,2). Second method. X and Y are independent X2(2) variables Gust check the density functions), and hence XIY is F(2, 2). 5. The vector (X, Xl - X, X2 - X, ... , Xn - X) has, by Theorem (4.9.6), a multivariate normal distribution. We have as in Exercise (4.5.7) that cov(X, Xr - X) = 0 for all r, which implies that X is independent of each Xr. Using the form of the multivariate normal density function, it follows that X is independent of the family {X r - X : 1 .:s r .:s n}, and hence of any function of these variables. Now S2 = (n - 1)-1 2:r(X r - X)2 is such a function. 6. The choice of fixed vector is immaterial, since the joint distribution of the Xj is spherically symmetric, and we therefore take this vector to be (0, 0, ... , 0, 1). We make the change of variables U 2 = Q2 + X~, tan III = QI X n , where Q2 = 2:~:t and Q 2: O. Since Q has the x2(n - 1) distribution, and is independent of Xn, the pair Q, Xn has joint density function

X;

x

E

JR, q > O.

The theory is now slightly easier than the practice. We solve for U, Ill, find the Jacobian, and deduce the joint density function /u, \jJ (u, 1jf) of U, Ill. We now integrate over u, and choose the constant so that the total integral is 1.

203

[4.11.1]-[4.11.7]

Solutions

Continuous random variables

4.11 Solutions. Sampling from a distribution 1.

Uniform on the set {I, 2, ... , n}.

2.

The result holds trivially when n = 2, and more generally by induction on n.

3. We may assume without loss of generality than = I (since Z/A is rCA, t) if Z is ro, t)). Let U, V be independent random variables which are uniformly distributed on [0, I]. We set X = -t log V and note that X has the exponential distribution with parameter 1/ t. It is easy to check that

I xt-Ie- x < cf (x) __

ret)

where c

= tt e- t +1/

-

for x> 0,

X

ret). Also, conditional on the event A that

X has the required gamma distribution. This observation may be used as a basis for sampling using the rejection method. We note that A = {log U :5 (n - 1) (log(X/n) - (X/n) + I)}. We have that peA) = 1/c, and therefore there is a mean number c of attempts before a sample of size I is obtained.

4.

Use your answer to Exercise (4.11.3) to sample X from ro, a) and Y from ro, f3). ByExercise (4.7.14), Z = X/eX + Y) has the required distribution.

S. (a) This is the beta distribution with parameters 2, 2. Use the result of Exercise (4). (b) The required r 0, 2) variables may be more easily obtained and used by forming X = - loge U I U2) and Y -log(U3U4) where {Ui : I :5 i :5 4} are independent and uniform on [0, I]. (c) Let UI, U2, U3 be as in (b) above, and let Z be the second order statistic U(2). That is, Z is the middle of the three values taken by the Ui; see Problem (4.14.21). The random variable Z has the required distribution. (d) As a slight variant, take Z = max{UI, U2} conditional on the event {Z :5 U3}. (e) Finally, let X = .JUl/(.JUl + ..JUi.), Y = the event {Y :5 I}, is as required.

.JUl +..JUi..

The distribution of X, conditional on

6. We use induction. The result is obvious when n = 2. Let n ~ 3 and let P = (PI, P2, ... , Pn) be a probability vector. Since P sums to I, its minimum entry P(I) and maximum entry P(n) must satisfy

I

I

P(I) :5 - < - - , n n-I

P{l)

+ P(n)

~ P(I)

We relabel the entries of the vectorp such that PI (n - I)PI, 0, ... ,0). Then I P = - - I VI

n-

n- 2

+ --IPn-1 n-

where

Pn-I

+

I-P(I)

n-I

=

1+(n-2)p{l)

n-I

~

= P(I) and P2 = P(n), and set vI = n-I =- ( 0, PI + P2 n-2

I n-I

I --.

n-I

((n -1)PI, 1-

- - - , P3,'"

, Pn

) ,

is a probability vector with at most n - I non-zero entries. The induction step is complete.

It is a consequence that sampling from a discrete distribution may be achieved by sampling from a collection of Bernoulli random variables.

!

7. It is an elementary exercise to show that P(R 2 :5 I) = 77: , and that, conditional on this event, the vector (TI, T2) is uniformly distributed on the unit disk. Assume henceforth that R2 :5 I, and write (R, 8) for the point (TI, T2) expressed in polar coordinates. We have that R and 8 are independent with joint density function fR,e(r, e) = r/77:, 0 :5 r :5 1,0 :5 e < 277:. Let (Q, \11) be the polar

204

Solutions

Coupling and Poisson approximation

coordinates of (X, Y), and note that \11

= e and e-~Q2 = R2.

[4.11.8]-[4.12.1]

The random variables Q and \11 are 1 2

independent, and, by a change of variables, Q has density function fQ(q) = qe-'1 Q , q > O. We recognize the distribution of (Q, \11) as that of the polar coordinates of (X, Y) where X and Y are independent N(O, 1) variables. [Alternatively, the last step may be achieved by a two-dimensional change of variables.]

8.

We have that

9.

The polar coordinates (R, e) of (X, Y) have joint density function

fR ,e(r, (})

2r

= -, n

Make a change of variables to find that Y / X

= tan e has the Cauchy distribution.

10. By the definition of Z, m-1

lP'(Z

= m) = hem)

II (1- her»)

r=O

= lP'(X >

0) 11" (X > 1 I X > 0) .. ·lP'(X

=m IX >

m - I)

= lP'(X = m).

11. Suppose g is increasing, so that hex) = -g(l- x) is increasing also. By the FKG inequality of Problem (3.1U8b), K = cov(g(U), - g(l - U» 2: 0, yielding the result. Estimating [ by the average (2n) -1 L:;~ 1 g (Ur) of 2n random vectors U r requires a sample of size 2n and yields an estimate having some variance 2na 2 . If we estimate [ by the average (2n)-1 {L:~=1 g(Ur ) + g(l - Ur )}, we require a sample of size only n, and we obtain an estimate with the smaller variance 2n(a 2 - K). 12. (a) By the law of the unconscious statistician, JE [g(Y)fx(Y)] fy(y)

=

J

g(Y)fx(Y)f ( )d fy(y) y Y Y

= [.

(b) This is immediate from the fact that the variance of a sum of independent variables is the sum of their variances; see Theorem (3.3.11 b). (c) This is an application of the strong law of large numbers, Theorem (7.5. I).

= sin(~n U) has the required distribution. This is an example of the inverse transform method. (b) If U is uniform on [0, I], then I - U 2 has density function g(x) = {2.Jf=X} -1, 0 ::: x ::: 1. Now g(x) 2: (n/4)f(x), which fact may be used as a basis for the rejection method. 13. (a) If U is uniform on [0, I], then X

4.12 Solutions. Coupling and Poisson approximation 1.

Suppose that JE(u(X» 2: JE(u(Y» for all increasing functions u. Let c Ie(x) = {

I

o 205

ifx > c, . if x ::: c,

E

IR. and set u

= Ie where

[4.12.2]-[4.13.1]

Solutions

Continuous random variables

to find that IP'(X > c) = lE(Ic(X» 2: lE(Ic(y» = IP'(Y > c). Conversely, suppose that X 2:st Y. We may assume by Theorem (4.12.3) that X and Y are defined on the same sample space, and that IP'(X 2: Y) = 1. Let u be an increasing function. Then lP'(u(X) 2: u(Y» 2: IP'(X 2: Y) = I, whence lE(u(X) - u(Y» 2: 0 whenever this expectation exists.

2.

Let O! Then Z =

= /-il).., and let {Ir

: r 2: I} be independent Bernoulli random variables with parameter O!. /-i, and Z .:s X.

2::;=1 Ir has the Poisson distribution with parameter)..O! =

3.

Use the argument in the solution to Problem (2.7.13).

4.

For any A S; JR,

IP'(X

=I-

Y) 2: IP'(X E A, Y E A C ) = IP'(X E A) -IP'(X E A, YEA)

2: IP'(X E A) - IP'(Y E A), and similarly with X and Y interchanged. Hence,

IP'(X

=I-

Y) 2: sup jlP'(X E A) -IP'(Y E A)j = ~dTV(X, Y). A9R

5. For any positive x and y, we have that (y - x)+ follows that

+x

1\

Y

=

2)fx(k) - fy(k)}+ = ~)fy(k) - fx(k)}+ = I k

y, where x 1\ y

L

k

fx(k)

1\

=

min{x, y}. It

fy(k),

k

and by the definition of dTV(X, Y) that the common value in this display equals ~dTV(X, Y) = 8. Let U be a Bernoulli variable with parameter I - 8, and let V, W, Z be independent integer-valued variables with

IP'(V = k) = (fx(k) - fy(k)}+ 18, IP'(W = k) = (fy(k) - fx(k)}+ 18, IP'(Z = k) = fx(k)

1\

fy(k)/(l - 8).

Then X' = U Z + (l - U) V and yl = U Z + (l - U) W have the required marginals, and IP'(X ' yl) = IP'(U = I) = I - 8. See also Problem (7.11.16d).

=

6. Evidently dTV(X, Y) = Ip - ql, and we may assume without loss of generality that p 2: q. We have from Exercise (4.12.4) that IP'(X = Y) .:s I - (p - q). Let U and Z be independent Bernoulli variables with respective parameters I-p+q andql(l-p+q). The pair X, = U(Z-I)+I, yl = UZ has the same marginal distributions as the pair X, Y, and IP'(X ' = yl) = IP'(U = I) = I - p + q as required. To achieve the minimum, we set X" = I - X' and yll = yl, so that IP'(X" = yll) = I -IP'(X ' = yl) = P -q.

4.13 Solutions. Geometrical probability 1. The angular coordinates \II and}:; of A and B have joint density f(1jf, a) = (2n)-2. We make the change of variables from (p, e) 1-+ (1jf, a) by p = cos{ 1(0' - 1jf)}, e = ~ (n + a + 1jf), with inverse l 1 1jf = In - i cos- e p, 0'= - In + cos -1 p,

e-

and Jacobian

IJI = 2/~.

206

Solutions

Geometrical probability

[4.13.2]-[ 4.13.6]

2. Let A be the left shaded region and B the right shaded region in the figure. Writing A for the random line, by Example (4.13.2), lP'(A meets both Sl and S2) = lP'(A meets both A and B) = lP'(A meets A) ()( b(A)

+ lP'(A meets B) -

+ b(B) -

lP'(A meets either A or B)

b(H) = b(X) - b(R),

whence lP'(A meets S2 I A meets Sl) = [b(X) - b(H)]lb(Sl)' The case when S2 ~ Sl is treated in Example (4.13.2). When Sl n S2 =j:. 0 and Sl /:::,. S2 =j:. 0, the argument above shows the answer to be [b(Sl) + b(S2) - b(H)]lb(Sd. 3. With III the length of the intercept I of Al with S2, we have that lP'(A2 meets I) = 2IIllb(Sl), by the Buffon needle calculation (4.13.2). The required probability is

4.

If the two points are denoted P

= (Xl, Y1), and Q = (X2, Y2), then

We use Crofton's method in order to calculate E(Z). Consider a disc D of radius x surrounded by an annulus A of width h. We set A(X) = E(Z I P, QED), and find that A(X

+ h) = A(X)

(1 - ~ -

Now 2 E(Z I P ED, Q E A) = - 2

!o!tr

nx

whence

+ 2E(Z I P ED,

O(h))

Q E A)

!o2xCOSt:/ 2

0

r drd8

0

c:

+ O(h))

.

32x 9n

+0(1) = - ,

4A 128 dx x 9n which is easily integrated subject to A(O) = 0 to give the result. dA

-=--+-,

(i) We may assume without loss of generality that the sphere has radius 1. The length X = IAOI has density function f (x) = 3x 2 for 0 :::: x :::: 1. The triangle includes an obtuse angle if B lies either in the hemisphere opposite to A, or in the sphere with centre ~ X and radius ~ X, or in the segment cut off by the plane through A perpendicular to AO. Hence,

5.

lP'(obtuse)

= i +E((iX)3) + (1n)-IE

= i + to + (1)-l E (j -

X

(!xl

n(1- y2)dY )

+ 1X3) =

i·

(ii) In the case of the circle, X has density function 2x for 0 :::: x :::: 1, and similar calculations yield lP'(obtuse) 6.

1 2

1 8

1 n

= - + - + -E(cos- l

Choose the x-axis along AB. With P EIABPI

= (X, Y)

~

X - Xv 1 - X2)

and G

= (Y1, Y2),

= ~IABIE(Y) = ~IABIY2 = IABGI· 207

3

= -. 4

[4.13.7]-[4.13.11] 7.

Solutions

Continuous random variables

We use Exercise (4.13.6). First fix P, and then Q, to find that

b

With = IABI and h the height of the triangle ABC on the base AB, we have that 101021 the height of the triangle A0102 is ~ h. Hence,

EIAPQI

=

1b and

= ~ ·1b. ~h = ~IABq.

8. Let the scale factor for the random triangle be X, where X E (0, I). For a triangle with scale factor x, any given vertex can lie anywhere in a certain triangle having area (I - x)2IABC/. Picking one at random from all possible such triangles amounts to supposing that X has density function f(x) = 3(1 - x)2, 0::: x ::: I. Hence the mean area is

9.

We have by conditioning that, for F(z, a

+ da) = F(z, a)

°: : z ::: a, )2 +

a (a +da

2da) = F(z, a) ( 1- a

IP'(X ::: a - z) .

z -2da + -. a

a

2ada 2 (a +da)

+ o(da)

+o(da),

and the equation follows by taking the limit as da ,(, 0. The boundary condition may be taken to be F(a, a) = I, and we deduce that 2z

F(z, a) = -; -

(Z)2 -;; ,

0:::

z ::: a.

Likewise, by use of conditional expectation,

m, (a

+ da) =

2da)

m, (a) ( I - -;;-

2da + E( (a - X)') . -;;+ o(da).

Now, E«a - X)') = a' /(r + I), yielding the required equation. The boundary condition is m, (0) = 0, and therefore

2a'

= (r + I)(r + 2)·

m,(a)

10. If n great circles meet each other, not more than two at any given point, then there are 2(~) intersections. It follows that there are 4(~) segments between vertices, and Euler's formula gives the number of regions as n(n - 1) + 2. We may think of the plane as obtained by taking the limit as R ~ 00 and 'stretching out' the sphere. Each segment is a side of two polygons, so the average number of sides satisfies 4n(n - 1)

2

+ n(n -

1)

~4

asn

~

00.

11. By making an affine transformation, we may without loss of generality assume the triangle has vertices A = (0, 1), B = (0,0), C = (1,0). With P = (X, y), we have that

N=(O,-Y ). 1- X 208

Solutions

Problems

[4.13.12]-[4.14.1]

Hence, lE/BLN/ = 2

lol (

1

xy dxdy = ABC 2(l-x)(1 - y)

2 x ) dx = -n - 3 -x - --logx I-x 6 2'

0

and likewise lE/CLM/ = lE/ANM/ = ~n2 - ~. It follows that lE/LMN/ = ~(10 - n 2 ) = (10-

n 2 )/ABC/. 12. Let the points be P, Q, R, S. By Example (4.13.6), lP( one lies inside the triangle formed by the other three) = 4lP(S E PQR) = 4 .

-b.

13. We use Crofton's method. Let mea) be the mean area, and condition on whether points do or do not fall in the annulus with internal and external radii a, a + h. Then mea

+h)

= mea)

C:hr + [~h +O(h)]

mea),

where mea) is the mean area of a triangle having one vertex P on the boundary of the circle. Using polar coordinates with P as origin,

Letting h

~

0 above, we obtain dm

6m

6

35a 2

da

a

a

36n

-=--+-.--, whence mea)

=

(35a 2 )/(48n).

14. Let a be the radius of C, and let R be the distance of A from the centre. Conditional on R, the required probability is (a - R)2/a 2 , whence the answer is lE«a - R)2/a 2 ) = r)22r dr = ~.

JJ(1 -

15. Let a be the radius of C, and let R be the distance of A from the centre. As in Exercise (4.13.14), (1 - r)33r 2 dr = the answer is lE«a - R)3/a 3) =

JJ

to.

4.14 Solutions to problems 1.

(a) We have that

Secondly,

f

~ 0, and it is easily seen that J~oo

f

(x) dx = I using the substitution y = (x -

J-L)/(a./2). 1

1 2

1 2

(b) The mean is J~oo x(2n)-'Z e-'Zx dx, which equals 0 sincexe-'Zx is an odd integrable function. 1

1 2

The variance is J~oo x 2 (2n) - 'Z e - 'Z x dx, easily integrated by parts to obtain 1.

209

[4.14.2]-[4.14.5]

Solutions

Continuous random variables

(c) Note that

I 2

and also 1 - 3y-4 < 1 < 1 + y-2. Multiply throughout these inequalities by e-'zY l.,fiif, and integrate over [x, (0), to obtain the required inequalities. More extensive inequalities may be found in Exercise (4.4.8). (d) The required probability is a(x) = [1 - (x + alx)]f[l - (x)]. By (c),

as x --+

2.

Clearly

f ::: 0 if and only if 0 :s a

00.

f3 :s 1. Also

<

3. The Ai partition the sample space, and i-I :s X (w) < i if w E Ai. Taking expectations and using the fact that E(li) = lP'(Ai), we find that S :s E(X) :s 1 + S where 00

S

00

= LCi -l)lP'(Ai) = L i=2

4.

i-I

00

00

L 1 ·lP'(Ai) = L L

i=2j=1

00

lP'(Ai)

j=1 i=j+1

= LlP'(X:::

j).

j=1

(a) (i) Let F-1(y) = sup{x : F(x) = y}, so that lP'(F(X) :s y)

= lP'(X :s F- I (y)) = F(F- I (y)) = y,

O:sy:s1.

(ii) lP'(-logF(X):S y) = lP'(F(X)::: e- Y) = 1- e-Y if y::: O. (b) Draw a picture. With D = PR,

lP'(D :s d)

= lP'(tanPQR:s d) = lP'(PQR:s tan- 1 d) = ~

G· +

tan- 1 d).

Differentiate to obtain the result. 5.

Clearly lP'(X > s

+x IX >

s)

=

lP'(X > s + x) lP'(X > s)

=

e-A(s+x)

e-

A S

=e

-Ax

if x, s ::: 0, where A is the parameter of the distribution. Suppose that the non-negative random variable X has the lack-of-memory property. Then G (x) = lP'(X > x) is monotone and satisfies G(O) = 1 and G(s + x) = G(s)G(x) for s, x ::: O. Hence G(s) = e- AS for some A; certainly A > 0 since G(s) :s 1 for all s. Let us prove the hint. Suppose that g is monotone with g(O) = 1 and g(s + t) = g(s)g(t) for s, t ::: O. For an integer m, gem) = g(l)g(m - 1) = ... = g(l)m. For rational x = min, g(x)n = gem) = g(l)m so that g(x) = g(l)X; all such powers are interpreted as exp{x logg(l)}. Finally, if x is irrational, and g is monotone non-increasing (say), then g(u) :s g(x) :s g(v) for all

210

Solutions [4.14.6]-[4.14.9]

Problems

rationals u and v satisfying v ~ x ~ u. Hence g(l)U ~ g(x) ~ g(l)v. Take the limits as u .j, x and v t x through the rationals to obtain g (x) = eJ.LX where M = log g (I). 6.

=

If X and Y are independent, we may take g = g(x)h(y). Then

f(x, y)

fx(x)

= g(x)

and 1=

Hence fx(x)fy(y)

i:

i:

h(y)dy,

fy(y)dy =

= g(x)h(y) =

fy(y)

1

00

= h(y)

i: i: g(x)dx

fy.

i:

Suppose conversely that

g(x)dx

h(y)dy.

f(x, y), so that X and Y are independent.

= OwhereaslP'(Y

7. TheyarenotindependentsincelP'(Y < 1, X> 1) As for the marginals, fx(x) =

=

fx and h

2e- X - Y dy = 2e- 2x ,

< 1) > OandlP'(X > 1) > O.

fy(y) = loy 2e- X - Y dx = 2e- Y (1- e- Y ),

for x, y 2: O. Finally,

roo roo xy2e-x-y dx dy = 1 ~, implying that cov(X, Y) = !. JE(XY)

=

ix=o iy=x

and JE(X) = ~, JE(Y) =

8. As in Example (4.13.1), the desired property holds if and only if the length X of the chord satisfies X ~ J3. Writing R for the distance from P to the centre 0, and e for the acute angle between the chord and the line OP, we have that X = 2--/1 - R2 sin2 e, and therefore lP'(X ~ J3) = lP'(R sin e 2: ~). The answer is therefore lP'

(R >- _1_) =~ 2sine n

which equals ~ - J3/(2n) in case (a) and ~ 9.

r~1f lP' (R ->

io

_1_) 2sine

de '

+ n- 1 10g tan(n /12) in case (b).

Evidently, JE(U) = lP'(Y

~ g(X»

=

11

dx dy =

O:::x,y:::1

lo1 g(x) dx, 0

y:::g(x)

JE(V) = JE(g(X» = JE(W) =

~ 10

10

1 g(x)dx,

1 {g(x) +g(1-x)}dx =

10

1 g(x)dx.

Secondly, JE(U 2) = JE(U) = J,

JE(W2)

=

! { 2 10

10

JE(V2) =

1 g(x)2 dx

+ 2 10

1 g(x)2 dx

~J

since g

1 g(x)g(1 - x) dX}

1

= JE(V2) -

~ 10

= JE(V2) -

! ior {g(x) -

g(x){g(x) - g(1 - x)} dx 1

2 g(1 - x)} dx

211

~ JE(V 2).

~ 1,

[4.14.10J-[4.14.11J

Solutions

Continuous random variables

Hence var(W) .::: var(V) .::: var(U).

10. Clearly the claim is true for n = 1, since the f().., 1) distribution is the exponential distribution. Suppose it is true for n .::: k where k ::: 1, and consider the case n = k + 1. Writing in for the density function of Sn, we have by the convolution formula (4.8.2) that

which is easily seen to be the f().., k

+ 1) density function.

11. (a) Let ZI, Z2, ... , Zm+n be independent exponential variables with parameter)... Then, by Problem (4.14.10), X' = ZI + ... + Zm is f().., m), y' = Zm+l + ... + Zm+n is f().., n), and X' + y' is f().., m + n). The pair (X, Y) has the same joint distribution as the pair (X', Y'), and therefore X + Y has the same distribution as X' + Y', i.e., f()", m + n). (b) Using the transformation u = x + y, v = x/(x + y), with inverse x = uv, y = u(l - v), and Jacobian J we find that U

= I1 -vv

u I=-u '

-u

= X + Y, V = X/(X + Y) have joint density function

for u ::: 0,0.::: v .::: 1. Hence U and V are independent, U being f().., m distribution with parameters m and n. (c) Integrating by parts,

1

00

lP'(X> t) =

)..m

(m - I)!

t

=

[

)..m-l

-

(m-l)!

= e- At

x m- 1e- h dx

] x m- 1e- h

00

+

(m - I)!

+ lP'(X'

1

00

t

t

(At)m-l

+ n), and V having the beta

)..m-l

(m-2)!

x m- 2 e- h dx

> t)

where X' is f()", m - 1). Hence, by induction, m-l lP'(X > t) =

L e-

At

()..d

---y:r

k=O

= lP'(Z < m).

.

(d) This may be achieved by the usual change of variables technique. Alternatively, reflect that, using x + y, v x/(x + y) maps a pair the notation and result of part (b), the invertible mapping u X, Y of independent (f()", m) and f().., n)) variables to a pair U, V of independent (f().., m +n) and B(m, n)) variables. Now UV = X, so that (figuratively)

=

"f().., m

+ n)

x B(m, n) = f().., m)".

Replace n by n - m to obtain the required conclusion.

212

=

Solutions

Problems

[4.14.12]-[4.14.15]

12. (a) Z = X? satisfies

z 2: 0, the

rei, i) or x2 (1) density function.

(b) If z 2: 0, Z = X?

+ X~ satisfies

the X 2 (2) distribution function. (c) One way is to work in n-dimensional polar coordinates! Alternatively use induction. It suffices to show that if X and Y are independent, X being x 2 (n) and Y being X 2 (2) where n 2: 1 , then Z = X + Y is x2 (n + 2). However, by the convolution formula (4.8.2),

Z

for some constant c. This is the

x2 (n + 2) density function as required.

13. Concentrate on where x occurs in fx[y(x

f X[Y ( x I y)

=

fx,Y(x, y) fy(y)

2: 0,

I y); any multiplicative constant can be sorted out later: 1

= Cl () Y exp { -

(x2

2XILI

2(1 - p2) -af - -af- -

2px(y - IL2))} aW2

by Example (4.5.9), where Cl (y) depends on y only. Hence

x for some C2(y). This is the normal density function with mean ILl

af(1 - p2). See also Exercise (4.8.7).

+ pal (y -

E

JR,

IL2)/a2 and variance

14. Set u = y/x, v = x, with inverse x = v, y = uv, and IJI = Ivl. Hence the pair U = Y / X, V = X has joint density fu, v(u, v) = fx,Y(v, uv)lvl for -00 < u, v < 00. Therefore fu(u) = J~oo f(v, uv)lvl dv. 15. By the result of Problem (4.14.14), U = Y / X has density function fu(u)

=

L:

f(y)f(uy)lyldy,

and therefore it suffices to show that U has the Cauchy distribution if and only if Z uniform on (-in, in). Clearly JP'(Z

:s (}) = JP'(U :s tan (}), 213

=

tan- l U is

[4.14.16]-[4.14.17] whence fz«()

Solutions

Continuous random variables

sec 2 (). Therefore fz«()

= fu(tan ()

= lr- 1 (for I() I < < u <

-00

When

ilr) if and only if

00.

f is the N(O, 1) density,

1

00

f(x)f(xy)lxldx =2

laoo _e-:Z 1 x (1+y )xdx, I

0

-00

2

2

2lr

which is easily integrated directly to obtain the Cauchy density. In the second case, we have the following integral:

1

00

-00

In this case, make the substitution

16. The transformation x

a21xl (1 +x 4 )(1 +x4y4) dx.

z = x 2 and expand as partial fractions.

= r cos (), y = rsin () has Jacobian J = r, so that r > 0,

Therefore R and e are independent,

0

:s ()

< 2lr.

e being uniform on [0, 2lr), and R2 having distribution function

this is the exponential distribution with parameter

i (otherwise known as r(i, 1) or X2(2».

The

I 2

density function of R is f R (r) = re -:z r for r > O. Now, by symmetry,

In the first octant, i.e., in {(x, y) : 0 :s y :s x}, we have min{x, y} = y, max{x, y} = x. The joint density fx,Y is invariant under rotations, and hence the expectation in question is

8

y

lao.:sy.:sx

-fx,Y(x,y)dxdy=8

17r/41°O tan()

x

2 lr

I 2

_ _ re-:z r drd()=-10g2.

11=0

2lr

r=O

17. (i) Using independence, lP'(U

:s u) =

1 -lP'(X > u, Y > u) = 1 -

(1- Fx(u»)(l

- Fy(u»).

Similarly lP'(V

(ii) (a) By (i), lP'(U (b) Also, Z = X

:s

:s v) =

lP'(X

:s

v, Y

:s v) =

Fx(v)Fy(v).

u) = 1 - e- 2u for u 2: O.

+ i Y satisfies

1

00

lP'(Z>v)= laOOlP'(Y > 2(v-x»)fx(x)dx= lave-2(V-X)e-Xdx+ = e- 2V (e V - 1)

+ e- V =

1 - (1- e- v )2 = lP'(V > v).

214

e- X dx

Solutions [4.14.18]-[4.14.19]

Problems

Therefore lE(V)

= lE(X) + !lE(Y) = ~, and var(V) = var(X) + ! var(y) = ~ by independence.

18. (a) We have that

(b) Clearly, for w > 0, lI"(U ::: u, W > w)

= lI"(U

::: u, W > w, X ::: Y)

+ lI"(U ::: u, W

> w, X > Y).

Now lI"(U ::: u, W > w, X ::: Y)

= lI"(X ::: u, Y

> X

+ w) = iou Ae-Axe-JL(x+w) dx

= _A_e-JLw(l _ e-(A+JL)U) A+JL

and similarly lI"(U < u W > w X> Y)

-

,

Hence, for 0 ::: u ::: u + w <

,

_ e-(A+JL)U).

00,

lI"(U < u W > w)

-

= ~e-AW(1 A+JL

,

=

(1 _ e-(A+JL)U) (_A_ e - JLW

A+JL

+ ~e-AW) A+JL

'

an expression which factorizes into the product of a function of u with a function of w. Hence U and W are independent.

19. U

= X + Y,

V

= X have joint density function fv u(v

I u) =

I (a) We are given that fvlU(v

fu,v(u, v) = fu(u)

I u) = u- l

fy(u - v)fx(v), 0::: v ::: u. Hence U

10

fy(u - v)fx(v) . fy(u - y)fx(y)dy

for 0::: v ::: u; then

fy(u - v)fx(v)

= -1

lou fy(u -

y)fx(y)dy

u 0

is a function of u alone, implying that fy(u - v)fx(v)

=

fy(u)fx(O)

= fy(O)fx(u)

by setting v = 0 by setting v = u.

In particular fy(u) and fx(u) differ only by a multiplicative constant; they are both density functions, implying that this constant is I, and fx = fy. Substituting this throughout the above display, we find that g(x) = fx(x)lfx(O) satisfies g(O) = 1, g is continuous, and g(u - v)g(v) = g(u) for 0::: v ::: u. From the hint, g(x) = e- Ax for x ::: 0 for some A> 0 (remember that g is integrable). (b) Arguing similarly, we find that fy(u - v)fx(v)

C = ua+.BI va-leu -

215

v).B-

I

10f"

fy(u - y)fx(y)dy

[4.14.20]-[4.14.22]

Solutions

Continuous random variables

for O:s v :s u and some constant c. Setting fx(v) = X(v)v a - 1 , fy(y) = l'/(y)ytJ- 1, we obtain I'/(u - v)X (v) = h(u) for 0 :s v :s u, and for some function h. Arguing as before, we find that 1'/ and X are proportional to negative exponential functions, so that X and Y have gamma distributions.

20. We are given that U is uniform on [0, 1], so thatO :s X, Y :s

i almost surely. For 0 <

E

<

i,

E = lP'(X + Y < E) :s lP'(X < E, Y < E) = lP'(X < E)2,

and similarly E = lP'(X + Y > 1 - E) :s lP'(X >

implying that lP'(X < E) ::::

..ft and lP'(X

i - E, Y >

> i-E) ::::

..ft.

i-E)

= lP'(X >

i - E)2,

Now

2E=lP'(i- E <X+Y i-E,Y<E)+lP'(X<E,Y> i-E)

= 2lP'(X

i - E)lP'(X < E) :::: 2(..ft)2.

>

Therefore all the above inequalities are in fact equalities, implying that lP'(X < E) ..ft if 0 < E < Hence a contradiction:

i.

~

= lP'(X +

Y <

g) = lP'(X, Y <

~) -lP'(X, Y <

= lP'(X

g, X + Y :::: g) < lP'(X < g, Y <

> i-E) =

~) =

g.

21. Evidently lP'(X(l) :s Yl,···, X(n) :s Yn) = LlP' (X1I"1 :s Yl,···, X1I"n :s Yn, X1I"1 < ... < X1I"n) 11"

where the sum is over all permutations 1( = (1(1,1(2, ... , 1(n) of (1, 2, ... , n). By symmetry, each term in the sum is equal, whence the sum equals

The integral form is then immediate. The joint density function is, by its definition, the integrand.

22. (a) In the notation of Problem (4.14.21), the joint density function of X (2),

... ,

X (n) is

where F is the common distribution function of the Xj. Similarly X (3), ... , X (n) have joint density

and by iteration, X (k),

" .,

X (n) have joint density

We now integrate over Yn, Yn-b ... , Yk+l in tum, arriving at

216

Solutions [4.14.23]-[4.14.25]

Problems

(b) It is neater to argue directly. Fix x, and let lr be the indicator function of the event {Xr :::: x}, and let S = lj + lz + ... + In. Then S is distributed as bin(n, F(x)), and IP'(X(k) :::: x) = IP'(S 2: k) =

t (~)

F(x)l(1 - F(x))n-l.

l=k

Differentiate to obtain, with F = F(x), fX(k/X) =

t (~)

f(x) {IF I - l (1 - F)n-l - (n -1)FI (1 - F)n-l-l }

l=k

= k

(~) f(x)F k - l (1 _

F)n-k

by successive cancellation of the tenns in the series.

23. Using the result of Problem (4.14.21), the joint density function is g(y) = n! L(y)T- n for 0:::: Yi :::: T, 1 :::: i :::: n, where y = (Yl, Y2, ... , Yn). 24. (a) We make use of Problems (4.14.22)-(4.14.23). The density function of X(k) is fdx) k(Z)x k - l (1 - x)n-k for 0:::: x :::: 1, so that the density function of nX(k) is 1 - Jk(xln) n

= -k

k!

n(n - 1) ... (n - k nk

+ 1) x k-l

(

x )n-k 1 k-l-x ~ --- x e n (k-l)!

1- -

as n ~ exp. The limit is the r(1, k) density function. (b) For an increasing sequence x(1), X(2), ... , X(n) in [0, 1], we define the sequence Un = -n log x(n), Uk = -klog(X(k)lx(k+l)) for 1 :::: k < n. This mapping has inverse

X(k)

= X(k+l)e -u k/ k = exp {

-?=

n ·-1 I

ui

}

,

l=k

with Jacobian J = (_l)n e - U l-u2-"'- Un In!. Applying the mapping to the sequence X(1), X(2), ... , X(n) we obtain a family Ul, U2,.·., Un of random variables with joint density g(Ul, U2,.·., un) = e- u1 - u2 -"'-Un for Ui 2: 0, 1 :::: i :::: n, yielding that the Ui are independent and exponentially distributed, with parameter 1. Finally log X (k) = - ,,£7=k i-I Ui. (c) In the notation of part (b), Zk = exp( -Uk) for 1 :::: k :::: n, a collection of independent variables. Finally, Uk is exponential with parameter 1, and therefore IP'(Zk :::: z) = IP'(Uk 2: -logz) = e 10gz = Z,

O
25. (i) (X 1, X2, X3) is uniformly distributed over the unit cube of]R3 , and the answer is therefore the volume of that set of points (Xl, x2, X3) of the cube which allow a triangle to be fonned. A triangle is impossible if Xl 2: X2 + X3, or X2 2: xl + X3, or X3 2: xl + X2. This defines three regions of the cube which overlap only at the origin. Each of these regions is a tetrahedron; for example, the region X3 2: Xl + X2 is an isosceles tetrahedron with vertices (0,0,0), (1,0, 1), (0, 1, 1), (0,0, 1), Hence the required probability is 1 - 3 . = ~. with volume (ii) The rods oflength xl, x2, ... , Xn fail to fonn a polygon if either Xl 2: X2 + X3 + ... + Xn or any of the other n - 1 corresponding inequalities hold. We therefore require the volume of the n-dimensional hypercube with n corners removed. The inequality Xl 2: X2 + X3 + ... + Xn corresponds to the convex hull of the points (0, 0, ... ,0), (1,0, ... ,0), (1, 1,0, ... ,0), (1,0,1,0, ... ,0), ... , (1,0, ... ,0,1).

i.

i

217

[4.14.26]-[4.14.27]

Solutions

Continuous random variables

°

Mapping XI r-+ 1 - XI, we see that this has the same volume Vn as has the convex hull of the origin together with the n unit vectors el, e2, ... , en. Clearly V2 = ~, and we claim that Vn = 1/ n!. Suppose this holds for n < k, and consider the case n = k. Then

where Vk-I (0, XI e2, ... ,XI ek) is the (k - I)-dimensional volume of the convex hull ofO, XI e2, ... , xlek. Now

so that xk-I 1 _1_-dxI = -

1

Vk =

10o

(k-I)!

k!'

The required probability is therefore 1 - n/(n!) = 1 - {en _l)!}-I. 26. (i) The lengths of the pieces are U = min{X I, X2J, V = IXI - X21, W = 1 - U - V, and we require that U < V + W, etc, as in the solution to Problem (4.14.25). In terms of the Xi we require either: or:

1, X21 < 1,

IXI - X21 <

1- X2 < ~,

IXI -

1- XI <

1.

Plot the corresponding region of ~2. One then sees that the area of the region is ~, which is therefore the probability in question. (ii) The pieces may form a polygon if no piece is as long as the sum of the lengths of the others. Since the total length is 1, this requires that each piece has length less than Neglecting certain null events, this fails to occur if and only if the disjoint union of events Ao U A I U ... U An occurs, where

1.

AO = {no break in (0, ~l),

Ak = {no break in (Xt, Xk

+ ~l) for 1 ~ k

~ n;

remember that there is a permanent break at 1. Now JP'(Ao) = (~)n, and for k 2: 1,

+ I)2- n whence the required probability is 1 - (n + I)2- n . (a) The function get) = (t P / p) + (t-q /q), for t > 0, has a unique minimum att = 1, and hence

Hence JP'(Ao U Al U··· U An) = (n

27. get) 2: g(I) = 1 for t > O. Substitute t = X

X I/q y-I/ P

where

IXI = -----;-;--

{EIXPIJI/p'

(we may as well suppose that JP'(XY = 0) IXIP pEIXPI

=I

1) to find that

IYlq

+ qElyql

IXYI 2: {EIXPIJI/P{ElyqIJI/q'

Holder's inequality follows by taking expectations.

218

Solutions

Problems

[4.14.28]-[4.14.31]

(b) We have, with Z = IX + YJ, E(ZP) = E(Z. Zp-l) ::: E(IXIZP-l)

+ {EIYPI}I/p{E(ZP)}I/q

::: {EIXPI}I/p{E(ZP)}I/q

by Holder's inequality, where p-l

+ E(IYIZP-l)

+ q-l = 1. Divide by {E(ZP)}I/q

to get the result.

28. Apply the Cauchy-Schwarz inequality to IZI!(b-a) and IZI~(b+a), where 0::: a::: b, to obtain {EIZbI}2::: EIZb-aIElzb+al. Now take logarithms: 2g(b) ::: g(b - a) + g(b +a) forO::: a::: b. Also g(p) -+ g(O) = 1 as p -I- 0 (by dominated convergence). These two properties of g imply that g is convex on intervals of the form [0, M) if it is finite on this interval. The reference to dominated convergence may be avoided by using Holder instead of Cauchy-Schwarz. By convexity, g(x)/x is non-decreasing in x, and therefore g(r)/r ~ g(s)/s if 0 < s ::: r.

29. Assume that X, Y, Z are jointly continuously distributed with joint density function f. Then E(X

I Y = Y, Z = z) =

J

xfx[Y z(x ,

I Y, z) dx =

J

x

f(x, y,z) iY,z(y, z)

dx.

Hence E{E(X

I Y, Z) IY = y} =

J

E(X

I Y = y, Z = z)fz[y(z

I y) dz

=

({x f(x,y,z) iY,z(y,z) dxdz }} iY,z(y, z) fy(y)

=

({xf(x,y,z)dxdz=E(XIY=y). }} fy(y)

Alternatively, use the general properties of conditional expectation as laid down in Section 4.6. 30. The first car to arrive in a car park of length x + 1 effectively divides it into two disjoint parks of length Y and x - Y, where Y is the position of the car's leftmost point. Now Y is uniform on [0, x], and the formula follows by conditioning on Y. Laplace transforms are the key to exploring the asymptotic behaviour of m(x)/x as x -+ 00.

31. (a) If the needle were oflength d, the answer would be 2/lr as before. Think about the new needle as being obtained from a needle of length d by dividing it into two parts, an 'active' part of length L, and a 'passive' part of length d - L, and then counting only intersections involving the active part. The chance of an 'active intersection' is now (2/lr)(L/d) = 2L/(lrd). (b) As in part (a), the angle between the line and the needle is independent of the distance between the line and the needle's centre, each having the same distribution as before. The answer is therefore unchanged. (c) The following argument lacks a little rigour, which may be supplied as a consequence of the statement that S has finite length. For E > 0, let Xl, x2, ... , Xn be points on S, taken in order along S, such that Xo and Xn are the endpoints of S, and IXi+1 - xii < E for 0 ::: i < n; Ix - yl denotes the Euclidean distance from x to y. Let Ji be the straight line segment joining Xi to xi+l, and let I; be the indicator function of {Ji n A '1= 0}. If E is sufficiently small, the total number of intersections between Jo U h U··· U In-l and S has mean n-l

LE(li) . 0

1=

2

=-

n-l

L

lrd 1= . 0

219

IXi+1

-xii

[4.14.32]-[4.14.34]

Solutions

Continuous random variables

by part (b). In the limit as E .J.. 0, we have that 2:i E(li) approaches the required mean, while

-nd2 L I n-1

X i+1-

x i!

2L(S)

~ --. nd

i=O

32. (i) Fix Cartesian axes within the gut in question. Taking one end of the needle as the origin, the other end is uniformly distributed on the unit sphere of JR3. With the X-ray plate parallel to the (x, y)-plane, the projected length V of the needle satisfies V 2: v if and only if IZ I ::::; ~,where Z is the (random) z-coordinate of the 'loose' end of the needle. Hence, for v ::::; I,

°: :;

since 4n ~ is the surface area of that part of the unit sphere satisfying Izl ::::; ~ (use Archimedes's theorem of the circumscribing cylinder, or calculus). Therefore V has density function

fv(v)

= v/~ forO::::;

v::::; 1.

e

(ii) Draw a picture, if you can. The lengths of the projections are determined by the angle between the plane of the cross and the X-ray plate, together with the angle \II of rotation of the cross about an axis normal to its arms. Assume that and \II are independent and uniform on [0, ~ n]. If the axis system has been chosen with enough care, we find that the lengths A, B of the projections of the arms are given by

e

with inverse \II

= tan

-lMA I-B

2. ---2

Some slog is now required to calculate the Jacobian J of this mapping, and the answer will be fA,B(a, b) = 41Jln- 2 for 0< a, b < 1, a 2 + b 2 > 1.

33. The order statistics of the Xi have joint density function

on the set I of increasing sequences of positive reals. Define the one-one mapping from I onto (0, oo)n by Yr = (n + 1 - r)(Xr - Xr -1) for 1 < r ::::; n, with inverse Xr = 2:;;=1 Yk/(n - k function of Y1, Y2, ... , Yn is

+ 1) for r

2: 1. The Jacobian is (n!)-l, whence the joint density

34. Recall Problem (4.14.4). First, Zi = F(Xi), I ::::; i ::::; n, is a sequence of independent variables with the uniform distribution on [0, 1]. Secondly, a variable U has the latter distribution if and only if - log U has the exponential distribution with parameter I.

220

Solutions [4.14.35]-[4.14.37]

Problems

It follows thatLj = -log F(Xj), 1 .:s i .:s n, is a sequence of independent variables with the exponentialdistribution. The order statistics L(I), ... , L(n) are in order -log F(X (n)), ... , -log F(X (I)), since the function -log FO is non-increasing. Applying the result of Problem (4.14.33), EI -nlogF(X(n)) and

Er

=

-(n

+1-

r){log F(X(n+l-r) -log F(X(n+2-r))} ,

1< r

are independent with the exponential distribution. Therefore exp( - E r ), 1 .:s r with the uniform distribution.

.:s n,

.:s n, are independent

35. One may be said to be in state j if the first j - 1 prizes have been rejected and the jth prize has just been viewed. There are two possible decisions at this stage: accept the jth prize if it is the best so far (there is no point in accepting it if it is not), or reject it and continue. The mean retum of the first decision equals the probability j I n that the jth prize is the best so far, and the mean retum of the second is the maximal probability f (j) that one may obtain the best prize having rejected the first j. Thus the maximal mean return V (j) in state j satisfies V(j) = max{jln, f(j)}.

Now j In increases with j, and f(j) decreases with j (since a possible stategy is to reject the (j + l)th prize also). Therefore there exists J such that j In:::: f (j) if and only if j .:s J. This confirms the optimal stategy as having the following form: reject the first J prizes out of hand, and accept the subsequent prize which is the best of those viewed so far. If there is no such prize, we pick the last prize presented. Let fl} be the probability of achieving the best prize by following the above strategy. Let Ak be the event that you pick the kth prize, and B the event that the prize picked is the best. Then, IT}

=

L

n lP'(B k=}+1

I Ak)lP'(Ak) =

L

- -

n (k)( J k=}+1 n k-l

. -1) = -J Ln -1, k

nk=}+lk-1

and we choose the integer J which maximizes this expresion. When n is large, we have the asymptotic relation IT} ~ (J In) log(nl J). The maximum of the function hn(x) = (xln) log(nlx) occurs at x = nle, and we deduce that J ~ nle. [A version of this problem was posed by Cayley in 1875. Our solution is due to Lindley (1961).] 36. The joint density function of (X, Y, Z) is f(x, y,z)

=

1

23/2 exp{ _~(r2 - 2AX - 2ILY - 2vz

(27rlY )

+ A2 + J-t2 + v 2)}

where r2 = x 2 + y2 + z2. The conditional density of X, Y, Z given R = r is therefore proportional to exp{Ax + J-ty + vz}. Now choosing spherical polar coordinates with axis in the direction (A, J-t, v), we obtain a density function proportional to exp(a cose) sine, where a = rVA 2 + J-t2 + v 2 • The constant is chosen in such a way that the total probability is unity.

37. (a) ¢'(x) = -x¢(x), so HI (x) = x. Differentiate the equation for Hn to obtain HII+I(x) = x Hn (x) - H~ (x), and use induction to deduce that Hn is a polynomial of degree n as required. Integrating by parts gives, when m .:s n, [ : Hm(x)Hn(x)¢(x)dx

= (_1)n [ : Hm(x)¢(n)(x)dx

= (_1)n-1 [ : H:n(x)¢(n-I\x)dx

1 H~m)(x)¢(n-m)(x)dx, 00

= ... = (_1)n-m

-00

221

[4.14.38]-[4.14.39]

Solutions

Continuous random variables

and the claim follows by the fact that H~m) (x) = m!. (b) By Taylor's theorem and the first part, tn t)n L ,Hn(x) = L ~¢(n)(x) = ¢(x 00

¢(x)

00

n=O n.

(

n=O

t),

n.

whence

38. The polynomials of Problem (4.14.37) are orthogonal, and there are unique expansions (subject to mild conditions) of the form u(x) = ~~O arHr(x) and v(x) = ~~O brHr(x). Without loss of generality, we may assume that lE(U) = lE(V) = 0, whence, by Problem (4.14.37a), ao = bo = O. By (4.14.37a) again, 00

00

var(U) = lE(u(X)2) =

L a;r!,

L b;r!.

var(V) =

r=l

r=l

By (4.14.37b),

By considering the coefficient of sm tn,

and so cov(U, V)

Ip(u, V)I

= lE ( =

L amHm(X) L bnHn(Y) 00

00

)

=

L anbnpnn!, 00

m=l

n=l n=l a b pn-ln'l ~oo la bin' L..n=l n n ..:::; Ipl L..n=l n n · .:::; Ipl, v~oo 2, ~oo b2 , v~oo 2, ~oo b 2 , L..n=l nn. L..n=l ann. L..n=l nn. L..n=l ann.

I pll ~oo

where we have used the Cauchy-Schwarz inequality at the last stage.

39. (a) Let Yr = X(r) - X(r-l) with the convention that X(O) = 0 and X(n+l) = 1. By Problem (4.14.21) and a change of variables, we may see that Yl, Y2, ... , Yn+l have the distribution of a point chosen uniformly at random in the simplex of non-negative vectors y = (Yl, Y2, ... , Yn+ l) with sum 1. [This may also be derived using a Poisson process representation and Theorem (6.12.7).] Consequently, the Yj are identically distributed, and their joint distribution is invariant under permutations of the indices of the Yj. Now ~~~f Yr = 1 and, by taking expectations, (n lE(X(r)) = rlE(Yl) = r/(n + 1). (b) We have that l

lE(Yf) =

1

!oo

x 2n(1 - x)n-l dx =

2 (n+l)(n+2)

+ l)lE(Yl) =

,

= lE [ (~ Yr) 2] = (n + l)lE(Yf) + n(n + l)lE(Yl Y2), r=l

222

1, whence

Solutions [4.14.40]-[4.14.42]

Problems implying that 1

E(YI Y2)

=

(n

+ I)(n + 2) ,

and also 2

E(X(r)X(s» = rE(Y I )

+ res -

r(s+l) I)E(YI Y2) = -(n-+-I)-(-n-+-2-)

The required covariance follows.

40. (a) By paragraph (4.4.6), X2 is r(~, ~) and y2 + Z2 is rei, I). Now use the results of Exercise (4.7.14). (b) Since the distribution of X2 / R2 is independent of the value of R2 = X2 + y2 + Z2, it is valid also if the three points are picked independently and uniformly within the sphere. 41. (a) Immediate, because the N(O, 1) distribution is symmetric. (b) We have that

i:

2¢(x)<'P(h)dx = =

i: i: i:

{¢ (x)<'P (h) +¢(x)[1- <'P(-h)1}dx ¢(x)dx

+

¢ (x)[<'P (h) - <'P(-h)]dx

=

I,

because the second integrand is odd. Using the symmetry of ¢, the density function of jYj is ¢(x)

+ ¢(x){ <'P(h) -

<'P(-h)}

+ ¢(-x) + ¢(-x){ <'P(-h) -

<'P(h)} = 2¢(x).

(c) Finally, make the change of variables W = jYj, Z = (X + AjYJ)/~, with inverse jyj = w, x

= z~ -

AW, and Jacobian ~. Then

iw,z(w, z)

=

v'i+)J ix,[Y[ (zv'i+)J - AW, w)

= v'i+)J. ¢(zv'i+)J - AW) .2¢(w),

W

> 0,

X E

JR.

The result follows by integrating over wand using the fact that

(00 ¢(z~-Aw)¢(w)dw= ¢~.

Jo

1 + A2

42. The required probability equals lP'({X3 -

~(XI + X2)}2 + {Y3

-

~(YI + Y2)}2

:s

i(XI - X2)2

+ i(YI

- Y2)2)

= lP'(Uf + ui :s where UI, U2 are N(O, therefore

p = lP'(i

= lP' ( KI

+ Vi>

i), VI, V2 are N(O, ~), and UI, U2, VI, V2 are independent. The answer is

(N? + Ni) :s 1(NS + Nl))

= lP'(KI :s

V?

1K2)

+ K2 :s 4I)

KI

where the Nj are independent N(O, I) where the Kj are independent chi-squared X 2(2)

= lP'(B

I :s 4) =

223

I 4

[4.14.43]-[4.14.48]

Solutions

Continuous random variables

where we have used the result of Exercise (4.7.14), and B is a beta-distributed random variable with parameters I, I.

43. The argument of Problem (4.14.42) leads to the expression JP>(V[ +

vi + V~ ::::: Vf + vi + vi) = JP>(KI ::::: ~ K2) 1

= JP>(B ::::: 4)

I

2 where the Kj are X (3)

= 3" -

.f3

4n'

where B is beta-distributed with parameters ~, ~.

44. (a) Simply expand thus: E[(X - Jl)3] = E[X 3 - 3X 2Jl + 3XJl2 - Jl3] where Jl = E(X). (b) var(Sn) = nO' 2 and E[(Sn - nJl)3] = nE[(X 1 - Jl)3] plus terms which equal zero because E(XI - Jl)

= 0.

(c) If Y is Bernoulli with parameter p, then skw(Y) = (1 - 2p)/...(jiij, and the claim follows by (b). (d) ml =).." m2 =).., + )..,2, m3 =)..,3 + 3)..,2 +).." and the claim follows by (a). (e) SinceAX is r(l, t), we may as well assume that).., = 1. It is immediate that E(X n ) = r(t+n)/ r(t), whence t(t+I)(t+2)-3t·t(t+1)+2t 3 2 skw(X) = t3/2 = ../i. 45. We find as above that kur(X) = (m4 - 4m3ml + 6m2my - 3mt)/O'4 where mk = E(X k ). (a) m4 = 30'4 for the N(O, 0'2) distribution, whence kur(X) = 30'4/0'4. (b) m T = r! /)..,T , and the result follows. (c) In this case, m4 =)..,4 + 6)..,3 + 7)..2 +).." m3 =)..,3 + 3)..,2 +).." m2 =)..,2 +).." and ml =)..,. (d) (var Sn)2 = n 2O' 4 and E[(Sn - nml)4] = nE[(Xl - ml)4] + 3n(n - 1)0'4. 46. We have as n

~ 00

that

By the lack-of-memory property, E(X(l)

I n

= -,

I I E(X(2) = - + - - , n n-I

whence, by Lemma (4.3.4), lim E(X(n)-logn)= lim (.!.+_I_+ ... +I-IOgn)=y. 10rOO{l_e-e-x}dx= n-+oo n-+oo n n- I 47. By the argument presented in Section 4.11, conditional on acceptance, X has density function is. You might use this method when is is itself particularly tiresome or expensive to calculate. If a(x) and b(x) are easy to calculate and are close to is, much computational effort may be saved.

48. M = max{Vl, V2, ... , Vy} satisfies JP>(M ::::: t)

et - I

= E(t Y ) = - - . e -I

224

Solutions

Problems

[4.14.49]-[4.14.51]

Thus,

LzJ + 2) + lP'(x = LzJ + 1,

lP'(Z ?: z) = lP'(X ?:

(e - l)e- LzJ - 2 -'----I--'-----e--1;---

:s

LzJ + I - Z) LZ L J 1 e J+l-z - I + (e - I)e- Z - . e _ 1 = e- z Y

49. (a) Y has density function e- Y for y > 0, and X has density function fx(x) = exe- Ctx for x > O. Now Y ?: ~ (X - ex)2 if and only if

I 2

which is to say that aVfx(X) :s f(X), where a = ex- 1e 1Ct ../2/n. Recalling the argument of Example (4.11.5), we conclude that, conditional on this event occurring, X has density function f. (b) The number of rejections is geometrically distributed with mean a-I, so the optimal value of ex is I 2

that which minimizes exe -1 Ct ../if1'1, that is, ex = I. (c) Setting Z = {

+X - X

with probability ~} with probability ~

~ (X _ ex)2 ,

conditional on Y >

we obtain a random variable Z with the N (0, I) distribution.

10 ~du = n/4. 1

50. (a) E(X) = 2 (b) E(y) = -

n

In

{1 sine de

Jo

= 2/n.

51. You are asked to calculate the mean distance of a randomly chosen pebble from the nearest collection point. Running through the cases, where we suppose the circle has radius a and we write P for the position of the pebble,

(i)

I EIOPI = na 2

(ii)

EIAPI = na 2

(iii)

E(JAPI/\ IBPI) = --;

2a

2

r dr de =

(~n {2aCOS(}

2

na

{2n {a

Jo Jo

Jo Jo

3' 32a

2

r dr de =

%.

[{in {a sec(} r2 dr de + {!n {2a cos(} r2 dr de] Jo Jo Jin Jo

4a {16

= 3n "3 -

17 ~ I 6",2 + 2 log (1

~} + ",2)

:::::

2a 3

X

1.13.

(iv) E(JAPI/\ IBPI/\ ICPI)

= -;.{ na

{tn {X r2 dr de + {~n {2acos(} r2 dr de} Jo Jo Jtn Jo where x

=2a -

n

- e)

3 n {lo!n -3.J3cosec 1 (-+e)de+ ~!n 8cos 3 ede } 0 8 3 tn

= 2a {16 -.!...!..J3 +

n

= a sin(1n) cosec(~n

3

4

225

3.J310g~} ::::: 2a 16

2

3

x 0.67.

[4.14.52]-[4.14.55]

Solutions

Continuous random variables

52. By Problem (4.14.4), the displacement of R relative to P is the sum of two independent Cauchy random variables. By Exercise (4.8.2), this sum has also a Cauchy distribution, and inverting the transformation shows that e is uniformly distributed. 53. We may assume without loss of generality that R has length 1. Note that b,. occurs if and only if the sum of any two parts exceeds the length of the third part. (a) If the breaks are at X, Y, where 0 < X < Y < 1, then b,. occurs if and only if 2Y > 1, and 2(Y - X) < 1 and 2X < 1. These inequalities are satisfied with probability ~.

1.

(b) The length X of the shorter piece has density function Ix (x) = 2 for 0 ::: x ::: The other pieces are oflength (1- X)Y and (1- X)(1 - y), where Y is uniform on (0,1). The event b,. occurs if and only if 2Y < 2X Y + 1 and X + Y - X Y > and this has probability

i,

2

(c) The three lengths are X,

1.

Ioo~ {l

2(1-x)

i (1 -

X),

1-

2x } dx = 10g(4/e). 2(1-x)

-

i (1 -

X), where X is uniform on (0, 1). The event

b,.

occurs

if and only if X < (d) This triangle is obtuse if and only if

IX 2

1

1 >

2(1-X)

r,:;' '\12

which is to say that X > ./2 - 1. Hence,

lP'(obtuse

I b,.) =

lP'(./2 - 1 < X <

54. The shorter piece has density function

1 lP'(X < 2)

JE(R)

=

JE(R2)

=

= 2/ (1

2

Ix (x) = 2 for 0::: x

lP'(R < r) = lP' -

with density function I R (r)

I)

=3-

:::

2./2.

i. Hence,

X) 2r- < r =(l-Xl+r'

- r)2 for 0 ::: r ::: 1. Therefore,

1 101 1 r lP'(R>r)dr= --dr=2log2-l, 0 1 +r 10o 1 10o

2rlP'(R > r) dr

= 101 0

2r(1 - r)

1+r

dr

=3 -

4 log 2,

and var(R) = 2 - (2 log 2)2.

55. With an obvious notation,

By a natural re-scaling, we may assume that a = 1. Now, Xl - X2 and Y1 - Y2 have the same triangular density symmetric on (-1, 1), whence (Xl - X2)2 and (Y1 - Y2)2 have distribution

226

Solutions [4.14.56]-[4.14.59]

Problems

function F (z) = 2.JZ - Z and density function the density f given by

r fer) = { r

fz (z)

1

= Z-:! - 1, for 0 .::; Z .::; 1. Therefore R2 has

if 0 .::; r .::; 1,

(_1 _ 1) ( _ 1 _ 1) dz

~

io.JZ l

(_1 _ 1) (_1_ _ 1) dz

if 1 .::; r .::; 2.

~

ir-I.JZ

The claim follows since

l

b

I - l-d - z=

a .JZ~

2 ( sm . -I

~- -

. -I sm

r

l) -

forO'::;a.::;b.::;1.

r

56. We use an argument similar to that used for Buffon's needle. Dropping the paper at random amounts to dropping the lattice at random on the paper. The mean number of points of the lattice in a small element of area dA is dA. By the additivity of expectations, the mean number of points on the paper is A. There must therefore exist a position for the paper in which it covers at least A l points.

r

57. Consider a small element of surface dS. Positioning the rock at random amounts to shining light at this element from a randomly chosen direction. On averaging over all possible directions, we see that the mean area of the shadow cast by d S is proportional to the area of d S. We now integrate over the surface of the rock, and use the additivity of expectation, to deduce that the area A of the random shadow satisfies JE(A) = cS for some constant C which is independent of the shape of the rock. By considering the special case of the sphere, we find c = It follows that at least one orientation of the rock gives a shadow of area at least

i.

is.

58. (a) We have from Problem (4.14.l1b) that Y r = Xrl(XI + ... + X r ) is independent of XI + ... + X r , and therefore of the variables Xr+l, X r +2, ... ,Xk+l, X I + ... + Xk+l. Therefore Yr is independent of {Yr +s : s ::: I}, and the claim follows. (b) Let S = XI + ... + Xk+l. The inverse transfonnation XI = ZIS, X2 = Z2s, ... , Xk = Zks, Xk+1 = S - ZIS - Z2S - ... - ZkS has Jacobian S

0

0

S

0 0

0

0

0

Zk

-s

-s

-s

1 - ZI - ... - Zk

ZI

Z2 =sk.

1=

The joint density function of X I, X2, ... , Xb S is therefore (with a

IT {

= ~~!t f3r),

).J3r (zr s )f3r- I e-).zr s } . ).J3k+1 {s(1 - ZI - ... - Zk)}f3k+ 1-l e -).s(1-zl-···-Zk)

r (f3r )

r= I

r (f3k+ j) k

= f()..,

p, s)

(II lr-

I

) (1 - ZI - ... - Zk)f3k+I- I ,

r=1

where

f

is a function of the given variables. The result follows by integrating over s.

In

= (crs ) be an orthogonal n x n matrix with Cni = II for 1 .::; i .::; n. Let Yir = ~~=I XisCrs, and note that the vectors Y r = (YIn Y2n ... , Ynr ), 1 .::; r .::; n, are multivariate

59. Let C

nonnal. Clearly JEYir = 0, and

I,U

I,U

227

[4.14.60]-[4.14.60]

Solutions

Continuous random variables

where Btu is the Kronecker delta, since C is orthogonal. It follows that the set of vectors Yr has the same joint distribution as the set of X r . Since C is orthogonal, Xir = 2:~=1 Csr Yis, and therefore

n-l

= LYisYjs

- YinYjn = L

s

YisYjs'

s=l

This has the same distribution as Tij because the Y r and the Xr are identically distributed.

60. We sketch this. Let lEJPQRJ = mea), and use Crofton's method. A point randomly dropped in Sea + da) lies in Sea) with probability a)2 (a+da

= 1 - -2da + o(da). a

Hence dm da

6m a

6mb a

-=--+--, where mb(a) is the conditional mean of JPQRJ given that P is constrained to lie on the boundary of Sea). Let b(x) be the conditional mean of JPQRJ given that P lies a distance x down one vertical edge.

x

Tl

Rl

T2

P R2

By conditioning on whether Q and R lie above or beneath P we find, in an obvious notation, that

By Exercise (4.13.6) (see also Exercise (4.13.7», mR"R2 = ~(~a)(~a) = ~a2. In order to find mR" we condition on whether Q and R lie in the triangles Tl or T2, and use an obvious notation. Recalling Example (4.13.6), we have that mT, that example,

=

mT2

= ~ . ~ax.

Next, arguing as we did in

1 II }. mT"T2 = 2"1 . l}4{ ax - 'lax - 'lax - gax

Hence, by conditional expectation, mR,

=

141141143 4' TI . 2'ax 4" • TI . 'lax 2" • l} . gax

+

+

=

13

TOR ax .

We replace x by a - x to find m R2 , whence in total b x ( )

= (~)2 a

13ax

108

+

(a a

X) 2 13a(a 108

x)

+

228

2 2x(a - x) . a a2 8

= ~a2 _ 108

12ax

108

+

12x2. 108

Solutions [4.14.61]-[4.14.63]

Problems Since the height of P is unifonnly distributed on [0, a], we have that 2

1 loa 11a b(x) dx = - - . a 0 108

mb(a) = -

We substitute this into the differential equation to obtain the solution m (a) = ma2. Turning to the last part, by making an affine transformation, we may without loss of generality take the parallelogram to be a square. The points form a convex quadrilateral when no point lies inside the triangle formed by the other three, and the required probability is therefore 1 - 4m (a) I a 2 = 44 25 1 - 144 = 16·

61. Choose four points P, Q, R, S unifonnly at random inside C, and let T be the event that their convex hull is a triangle. By considering which of the four points lies in the interior of the convex hull of the other three, we see that lP'(T) = 4lP'(S E PQR) = 4EIPQRI/ICI. Having chosen P, Q, R, the four points form a triangle if and only if S lies in either the triangle PQR or the shaded region A. Thus, lP'(T) = (IAI + EIPQRIl/ICl, and the claim follows on solving for lP'(T). 62. Since X has zero means and covariance matrix I, we have that E(Z) = fL covariance matrix of Z is E(L'X'XL) = L'IL = V.

+ E(X)L =

fL, and the

63. Let D = (dij) = AD - C. The claim is trivial if D = 0, and so we assume the converse. Choose i, k such that dik =1= 0, and write Yi = '£J=l dijXj = S + dikXk. Now lP'(Yi = 0) = E(lP'(Xk = -Sldik I S»). For any given S, there is probability at least ~ that Xk =1= -S Idik, and the second claim follows. Let Xl, X2, ... ,Xm be independent random vectors with the given distribution. If D =1= 0, the probability that Dxs = for 1 :::s s :::s m is at most (~)m, which may be made as small as required by choosing m sufficiently large.

°

229

5 Generating functions and their applications

5.1 Solutions. Generating functions 1.

(a) If lsi < (1 _ p)-l,

= n(1 -

Therefore the mean is G'(1) (b) If lsi < 1,

=

G(s)

f

sm

p)1 p. The variance is G"(1)+G'(1) -G'(1)2

(~m

m=l

Therefore G' (1)

_1_) = 1+ + 1

m

= n(1 _

p)1 p2.

(I-S) log(1-s). s

= 00, and there exist no moments of order 1 or greater.

(c) If p < lsi < p-l,

G(s)=

f

m=-oo The mean is G'(1) 2.

sm(I-P)plml=I-P{I+~+ 1+ p

1+ p

1 - sp

= 0, and the variance is G"(1) = 2p(1

pis }. 1 - (pis)

_ p)-2.

(i) Either hack it out, or use indicator functions fA thus:

T(s)

00

=~s

n

IP'(X > n)

(00

= IE ~ s

n

f{n<XJ

)

(X-l

= IE ~ s

n)

(I-SX)

= IE 1=7 =

(ii) It follows that T(1)

= lim stl

{I -1 -

G(S)} s

= lim G'(s) = G'(1) = IE(X) stl

1

by L'Hopital's rule. Also,

, . { -(1 - s)G'(s) + 1 - G(S)} T (1) = hm 2 stl (1-s) = ~G"(1) = i{var(X) - G'(1) + G'(1)2}

230

I-G(s) 1- s .

Solutions [5.1.3]-[5.1.5]

Generating functions

whence the claim is immediate. 3. (i) We have that G X,y(s, t) (ii) If JEIXYI < 00 then

JE(XY)

= JE(sX t Y ), whence G X,y(s,

= JE (XYS X - 1t Y -

1)

1)

= G xes) and G X,y(1, t) = Gy(t).

2

Is=I=1 = --Gx,Y(s, t) I asaat

.

s=I=1

4.

We write G(s, t) for the joint generating function. 00

(a)

G(s, t) =

j

L L sj t k (1 -

a)(f3 - a)a j f3k- j-1

j=Ok=O

=

f

(as) j (1 - a) (13

j=O

13

-

a) . I - (f3t)1+1

13

if f3ltl < I

1 - f3t

{I

(1 - a) (13 - a) f3t } (1- f3t)f3 1- (as/f3) - 1- ast (1 - a)(f3 - a) (1 - ast)(f3 - as)

=

. a If -lsi < 1

13

(the condition alstl < 1 is implied by the other two conditions on s and t). The marginal generating functions are I-a G(s, 1) = (1 - a)(f3 - a) , G(1,t) = -, (1 - as)(f3 - as) I-at and the covariance is easily calculated by the conclusion of Exercise (5.1.3) as a(1 - a)-2. (b) Arguing similarly, we obtain G(s, t) = (e - 1)/{e(1 - te s - 2 )} if Itles - 2 < I, with marginals G(s, 1)

=

1 - e- 1

1- e

S-

1- e- 1

2'

G(1,t)=

1 - te-

l'

and covariance e(e _1)-2. (c) Once again,

G(s, t)

=

log{l- tp(1- P +sp)} log(1 _ p)

if Itp(1 - p

+ sp)1

< 1.

The marginal generating functions are

G(s, 1)

=

log {I

P + p2 (1 - s) } log(1 _ p) ,

-

and the covariance is p2{p

+ log(1 -

G(1, t) = log(1 - tp) , log(1 - p)

p)}

(1 - p)2{log(1 - p)}2' 5.

(i) We have that

231

[5.1.6]-[5.2.1]

Solutions

Generating functions and their applications

where p + q = l. (ii) More generally, if each toss results in one of t possible outcomes, the ith of which has probability Pi, then the corresponding quantity is a function of t variables, Xl, x2, ... , Xt, and is found to be (PIXI + P2X2 + ... + ptXt)n. 6.

We have that

I U)}

JE(sX) = JE{JE(sX

1

=

10o

I I sn+l {I + u(s - I)}n du = - - . , n+1 I-s

the probability generating function of the uniform distribution. See also Exercise (4.6.5).

7.

We have that GX.y.z(X, y,z) = G(x, y,z, I) = ~(xYZ +xy + yz +ZX +X + y +z + I)

= ~(x

+ I)~(y + 1)1(z + I) = Gx(x)Gy(y)Gz(z),

whence X, Y, Z are independent. The same conclusion holds for any other set of exactly three random variables. However, G(x, y, z, w) 1= Gx(x)Gy(y)Gz(z)Gw(w).

8.

(a) We have by differentiating that JE(x2n) = 0, whence JP(X = 0) = l. This is not a moment generating function. (b) This is a moment generating function if and only if Er Pr = I, in which case it is that of a random variable X with JP(X = ar ) = Pro

9. The coefficients of sn in both combinations of G I, G2 are non-negative and sum to I. They are therefore probability generating functions, as is G (as) / G (a) for the same reasons.

5.2 Solutions. Some applications 1.

Let G(s)

= JE(sX) and G s(s) = EJ=o sj Sj.

By the result of Exercise (5.l.2),

~ ~ k s(l - G(s» 1- sG(s) T(s) = L...J smJP(X 2: m) = I +s L...Js JP(X > k) = 1+ = . m=O

I - s

k=O

I - s

Now,

so that

T(s) - T(O)

Gs(s - I) - Gs(O) s s - I where we have used the fact that T (0) = G s (0) = I. Therefore n

n

L(s - l)j-ISj'

Lsi-IJP(X 2: i) = i=1 j=l

Equating coefficients of si-l, we obtain as required that JP(X 2: i)

=

t

..

J=l

Sj

I)

(~- I I -

(_I)j-i,

232

I

~

i

~

n.

Solutions [5.2.2]-[5.2.4]

Some applications

Similarly,

+ s) -

T(l

Gs(s) - Gs(O)

T(O)

1+ s

s whence the second fonnula follows. 2.

Let Ai be the event that the ith person is chosen by nobody, and let X be the number of events

AI, A2, ... , An which occur. Clearly

J!"(Aij nAi2 n···nAi·) J

j -I) (n- j) (n -n-I j

= n-I

n- j

if i 1 i= i2 i= ... i= ij' since this event requires each of iI, ... , ij to choose from a set of n - j people, and each of the others to choose from a set of size n - j - 1. Using Waring's Theorem (Problem (1.8.13) or equation (5.2.14»,

= k) =

J!"(X

t(-I)j-k

(i)

Sj

J=k

where

(n -n-I I)n-

(~) (~)j

Sj =

)

j -

n-I

j .

Using the result of Exercise (5.2.1),

I)

n ( }. J!"(X ~ k) = L(-I)j-k . k k- I

Sj'

I

:s k :s n,

J=

while J!"(X

3.

~

0) = 1.

(a)

I Y)}

lE(xX+Y) = lE{lE(xX+Y

= lE{ x YeY(x-I)} = lE{ (xex-I)Y} = exp{tl(xex - I -

(b) The probability generating function of X 1 is ~ {s(l- p)}k

G(s) = L...J

k=1 klog(1/p)

=

10g{1- s(l- p)} logp

.

Using the 'compounding theorem' (5.1.25), Gy(s) = GN(G(S» = eJL(G(s)-I) =

4.

P )-P)IOgp . ( I - s(l - p)

Clearly,

( I) (10

lE - I +X where q

= 1- p.

1

= lE

t X dt ) =

0

100 lE(tX)dt = 100 (q + pt)n dt = Ip(n- q+n+1I) 1

1

In the limit, I

lE ( I

)

+X =

I - (I - )../n)n+1 )..(n + I)/n

233

+ 0(1) --+

I -e -J..

1)}.

[5.2.5]-[5.3.1]

Solutions

Generating functions and their applications

the corresponding moment of the Poisson distribution with parameter A. Conditioning on the outcome of the first toss, we obtain h n = qhn-l + p(l - hn-l) for n ~ 1, where q = 1 - p and hO = 1. Multiply throughout by sn and sum to find that H(s) = L:~o snhn satisfies H(s) - 1 = (q - p)sH(s) + ps/(l - s), and so

5.

H (s)

=

1 - qs (l-s){l-(q-p)s}

I }. = -21 { -+I l-s l-(q-p)s

6. By considering the event that HTH does not appear in n tosses and then appears in the next three, we find that JP'(X > n) p2q = JP'(X = n + 1) pq + JP'(X = n + 3). We mUltiply by sn+3 and sum over n to obtain 1 - JE(sx) ---,_ _ p2 qs 3 = pqs2JE(sX) + JE(sX), l-s which may be solved as required. Let Z be the time at which THT first appears, so Y By a similar argument,

+ 1) pq + JP'(X = n + l)pq + JP'(Z =

+ 3) + JP'(Z = n + 3) + JP'(X =

= min{X, Z}.

+ 2) p, n + 2)q.

JP'(Y > n) p2 q = JP'(X = Y = n

Y = n

Y = n

JP'(Y > n)q2 p = JP'(Z = Y =

Y =

Y =

We multipy by sn+l, sum over n, and use the fact that JP'(Y = n) = JP'(X = Y = n) + JP'(Z = Y = n). 7. Suppose there are n + 1 matching letter/envelope pairs, numbered accordingly. Imagine the envelopes lined up in order, and the letters dropped at random onto these envelopes. Assume that exactly j + 1 letters land on their correct envelopes. The removal of anyone of these j + 1 letters, together with the corresponding envelope, results after re-ordering in a sequence of length n in which exactly j letters are correctly placed. It is not difficult to see that, for each resulting sequence of length n, there are exactly j + 1 originating sequences of length n + 1. The first result follows. We multiply by sj and sum over j to obtain the second. It is evident that Gl(S) = s. Either use induction, or integrate repeatedly, to find that Gn(s) = L:~=o(s /r!.

nr

8.

We have for

lsi

< J1,

+ 1 that

JE(SX) = JE{JE(sx

9.

I A)} = JE(eA(s-I»

=

J1, J1, - (s -

= 1)

f

_J1,_ (_S_)k J1, 1 k=O J1, 1

+

+

Since the waiting times for new objects are geometric and independent, JE(s T ) -s (-) ( -s- ) ( -s- ) 3S 4- s 2- s 4 - 3s .

Using partial fractions, the coefficient of sk is 12 {1(i)k-4 - 4(1)k-4

+ ~(i)k-4}, for k

~ 4.

5.3 Solutions. Random walk 1.

Let Ak be the event that the walk ever reaches the point k. Then Ak ;2 Ak+ 1 if k r-l

JP'(M ~ r)

=

JP'(Ar)

= JP'(AO)

II JP'(Ak+l I Ak) = (p/q)r,

k=O

234

r ~ 0,

~

0, so that

Solutions

Random walk

since lP'(AkH

2.

I Ak)

= lP'(Al

lAo)

[5.3.2]-[5.3.4]

= p/q for k 2: 0 by Corollary (5.3.6).

(a) We have by Theorem (5.3.1c) that 00

,,2k

~s

2klo(2k)

I = sFo(s) =

k=l

s

2

00

~

V 1-s~

,,2k = s 2 lP'o(s) = ~s po(2k -

2),

k=l

and the claim follows by equating the coefficients of s2k. (b) It is the case that an = lP'( S1S2 ... S2n =f. 0) satisfies 00

L

an =

lo(k),

k=2n+2 keven

with the convention that ao = 1. We have used the fact that ultimate return to 0 occurs with probability 1. This sequence has generating function given by 00

00

L s2n L n=O

!k-l

00

lo(k) =

k=2n+2

L

lo(k)

k=2

k even

k even

=

L

s2n

n=O

1 - Fo(s)

1

~

by Theorem (5.3.1c)

00

= lP'o(s) =

L s 2n lP'(S2n = 0). n=O

Now equate the coefficients of s2n. (Alternatively, use Exercise (5.1.2) to obtain the generating function of the an directly.) 3. Draw a diagram of the square with the letters ABCD in clockwise order. Clearly p AA (m) = 0 if m is odd. The walk is at A after 2n steps if and only if the numbers of leftward and rightward steps are equal and the numbers of upward and downward steps are equal. The number of ways of choosing 2k horizontal steps out of 2n is @). Hence

with generating function

Writing FA (s) for the probability generating function of the time T of first return, we use the argument which leads to Theorem (5.3.1a) to find that GA(S) = 1 + FA(S)GA(S), and therefore FA(S)

= 1- GA(S)-l.

4.

Write (X n , Yn ) for the position of the particle at time n. It is an elementary calculation to show that the relations Un = Xn + Yn , Vn = Xn - Yn define independent simple symmetric random walks U and V. Now T = min{n : Un = m}, and therefore GT(S) = {s-l (1- ~)}m for lsi::; 1 by Theorem (5.3.5). 235

[5.3.5]-[5.3.6]

Solutions

Now X - Y

Generating functions and their applications

= VT, so that

where we have used the independence of U and V. This converges if and only if I ~(s + s-I)I ::::: 1, which is to say that s = ± 1. Note that GT (s) converges in a non-trivial region of the complex plane.

5.

Let T be the time of the first return of the walk S to its starting point O. During the time-interval (0, T), the walk is equally likely to be to the left or to the right of 0, and therefore L2n

={

+ L'

TR 2nR

if T ::::: 2n, if T > 2n,

1,

where R is Bernoulli with parameter L' has the distribution of L2n-T, and Rand L' are independent. = E(sL2n) satisfies

It follows that G2n (s)

n

G2n(S)

~(1

=L

+ s2k)G2n_2ds)f(2k) + L

~(1

+ s2n)f(2k)

k>n

k=1

where f(2k) = lP'(T = 2k). (Remember that L2n and T are even numbers.) Let H(s, t) L~o t2nG2n (s). Multiply through the above equation by t 2n and sum over n, to find that H(s, t) = 1-H(s, t) (F(t)

+ F(st)} + 1- (J(t) + J(st)}

where F(x) = L~O x 2k f(2k) and 00

J(x) = L x n=O

2n

1

L f(2k) = k>n

~' 1

Ixl < 1,

x

by the calculation in the solution to Exercise (5.3.2). Using the fact that F(x) = 1 - ~, we deduce that H(s, t) = I/J(1 - t 2 )(1- s2t 2 ). The coefficient of s2kt 2n is

6.

We show that all three terms have the same generating function, using various results established for simple symmetric random walk. First, in the usual notation, 00

L 4mlP'(S2m m=O

= O)s

2m' = 2sPO(s)

=

2s2 (1- S

23/2· )

Secondly, by Exercise (5.3.2a, b), m

E(T /\ 2m)

= 2mlP'(T

> 2m)

+L

m

2kfo(2k)

k=1

= 2mlP'(S2m = 0) + L k=1

236

lP'(S2k-2

= 0).

Solutions

Random walk

[5.3.7]-[5.3.8]

Hence,

I> 00

m=O

2m

s2 PO(s)

lE(T A 2m)

,

= - - 2 +sPO(s) = 1- s

2s2 23/2· (1 - s )

Finally, using the hitting time theorem (3.10.14), (5.3.7), and some algebra at the last stage, 00

L s m=O

m

00

2m

m

00

2lEI S2ml = 4 L s2m L 2klP'(S2m = 2k) = 4 L s2m L 2mhk(2m) m=O k=l m=O k=l d ~ 2m ~

= 4s- L.S

L hk(2m)

dsm=O

k=l

d

= 4s-

F1(S)2

dsl-F1(S)

2

=

2s2 23/2·

(l-S)

7. Let In be the indicator of the event {Sn = OJ, so that Sn+l = Sn + Xn+l + In. In equilibrium, lE(So) = lE(So) + lE(X 1) + lE(Io), which implies that lP'(SO = 0) = lE(IO) = -lE(X1) and entails lE(X 1) .:::: O. Furthermore, it is impossible that lP'(SO = 0) = 0 since this entails lP'(SO = a) = 0 for all a < 00. Hence lE(X 1) < 0 if S is in equilibrium. Next, in equilibrium,

Now, lE{ zSn+Xn+! +In (1 - In)} = lE(zSn

I Sn

> O)lE(zX! )lP'(Sn > 0)

lE(zSn+ Xn +! +In In) = zlE(zX! )lP'(Sn = 0).

Hence lE(ZSO) = lE(zX! ) [(lE(zSO) - lP'(So = O)}

+ zlP'(So = 0)1

which yields the appropriate choice for lE(zSO). 8.

The hitting time theorem (3.10.14), (5.3.7), states that lP'(TOb = n) = (lbl/n)lP'(Sn = b), whence lE(TOb

I TOb

< 00)

=

b

LlP'(Sn lP'(TOb < 00) n

The walk is transient if and only if p =j:. ~, and therefore lE(TOb

I

= b).

TOb < 00) < 00 if and only if

p =j:. ~. Suppose henceforth that p =j:. ~.

The required conditional mean may be found by conditioning on the first step, or alternatively as follows. Assume first that p < q, so that lP'(TOb < 00) = (p/q)b by Corollary (5.3.6). Then L:n lP'(Sn = b) is the mean of the number N of visits of the walk to b. Now lP'(N

= r) =

(~r pr-1(1 -

where p = lP'(Sn = 0 for some n 2: 1) = 1 - Ip lE(TOb

I TOb

< 00)

We have when p > q that lP'(TOb < 00) the fact that lE(TOb) = blE(TOl).

p),

r 2: 1,

ql. Therefore lE(N)

=

ql and

b (p/q)b (p/q)b . Ip _ ql·

= 1, and lE(TOl) = (p -

237

= (p/q)b /Ip -

q)-l. The result follows from

[5.4.1]-[5.4.4]

Solutions

Generating junctions and their applications

5.4 Solutions. Branching processes 1. Clearly lE(Zn I Zm) = ZmJ1,n-m since, given Zm, Zn is the sum of the numbers of (n - m)th generation descendants of Zm progenitors. Hence lE(ZmZn I Zm) = Z~J1,n-m and E(ZmZn) = E{E(ZmZn I Zm)} = lE(Z~)J1,n-m. Hence cov(Zm, Zn) = J1,n- m E(Z;') - E(Zm)E(Zn) = J1,n-m var(Zm), and, by Lemma (5.4.2),

z

( Pm,

Z) = n-m Jvar Zm = { VJ1,n-m(l- J1,m)/(l- J1,n) n J1, Z r.::;-r;; var n v mf n

if J1, i= 1, if J1, = 1.

2. Suppose 0 ::::: r ::::: n, and that everything is known about the process up to time r. Conditional on this information, and using a symmetry argument, a randomly chosen individual in the nth generation has probability 1/Zr of having as rth generation ancestor any given member of the rth generation. The chance that two individuals from the nth generation, chosen randomly and independently of each other, have the same rth generation ancestor is therefore 1/Zr . Therefore

I Zr)}

JP'(L < r) = E{JP'(L < r

= E(l- Z;I)

and so JP'(L

= r) = JP'(L

< r

+ 1) -JP'(L

= E(Z;I) - E(Z;~I)'

< r)

O:::::r < n.

If 0 < JP'(ZI = 0) < 1, then almost the same argument proves that JP'(L I'/r - I'/r+l for 0::::: r < n, where I'/r = E(Z;1 I Zn > 0).

3.

=

r

I Zn > 0) =

The number Zn of nth generation decendants satisfies n JP'(Zn

= 0) = Gn(O) = {

~

if p

nq+(p -q n) pn+l _ qn+l

= q,

ifpi=q,

whence, for n ::: 1,

1 JP'(T

n(n

= n) = JP'(Zn = 0) -JP'(Zn-l = 0) = {

It follows that E(T) <

4.

00

if p

+ 1) p

n-l

qn(p _ q)

= q,

Z

ifpi=q.

if and only if p < q.

(a) As usual,

This suggests that Gn(s) = 1 - a 1+,8+.+,8n-1 (l - s),8n for n ::: 1; this formula may be proved easily by induction, using the fact that G n (s) = G(Gn-l (s». (b) As in the above part (a), Gz(s)

=

j-l(p(f(f-l(p(f(s»»»

= 238

j-l(p(p(f(s»»

=

j-l(pz(f(s»)

Solutions

Age-dependent branching processes

where P2(S)

=

P(P(s)). Similarly Gn(s)

=

j-I(Pn(f(s))) forn::: 1, where Pn(s)

[5.4.5]-[5.5.1]

=

P(Pn-1 (s)).

(c) With P (s) = as / {I - (l - a)s} where a = y -I, it is an easy exercise to prove, by induction, that Pn (s) = an s/ {I - (l - an)s } for n ::: 1, implying that

5. Let Zn be the number of members of the nth generation. The (n + l)th generation has size Cn+ 1 + I n+ 1 where Cn+ 1 is the number of natural offspring of the previous generation, and I n+ 1 is the number of immigrants. Therefore by the independence,

whence

6.

By Example (5.4.3), Z E(s n)

Differentiate and set s

=

n - (n - l)s n

+ 1-

ns

n- 1

= -- + n

1 n 2 (l

+ n- I

s)

-

n::: O.

,

= 0 to find that

Similarly, n

00

lE(V2)

=L

n=O

(n

00

E(V3)

=L

(n

n=O

1

00

(n

+ 1)2

n2

(n

+ 1)2 -

n=O 00

+ 1)4 = L

n=O

The conclusion is obtained by eliminating

1

00

+ 1)3 = L

(n

1 2

1

00

L (n + 1)3 = oJr - n=O L (n + 1)3' n=O

-

2(n + 1) + 1 + 1)4 =

2

1

(;Jr

1

4

+ 90Jr

00

- 2

2:n (n + 1)-3.

5.5 Solutions. Age-dependent branching processes 1.

(i) The usual renewal argument shows as in Theorem (5.5.1) that

1

00

Gt(s)

=

fot G(Gt-u(s))fT(u) du

+

sfT(u) duo

Differentiate with respect to t, to obtain

a -Gt(s) = G(Go(s))fT(t) at Now Go(s) = s, and

+ fot -a 0

at

(G(Gt-u(s))} fT(u) du - sfT(t).

a

a

- (G(Gt-u(s))} = - - (G(Gt-u(S))}, at au

239

1

L (n + 1)3· n=O

[5.5.2]-[5.5.2]

Solutions

Generating functions and their applications

so that, using the fact that fT(U) = ).e-J...u if U 2: 0, (G(Gt-u(s))} h(u) du = -[G(Gt_u(S))h(u)]t - A r G(Gt-u(s))h(u)du, Jor ~ at 0 Jo

having integrated by parts. Hence

:t Gt(s)

= G(s)).e-J...t + { -G(s)).e-At + AG(Gt(S)) }

1

00

- A { Gt(s) -

sh(u) dU} - SAe- At

= A (G(Gt(s)) - Gt(s)}.

(ii) Substitute G(s) = s2 into the last equation to obtain

aG t

-

at

2

=A(G -G t ) t

with boundary condition Go(s) = s. Integrate to obtain At + c(s) = 10g{I - GIl} for some function c(s). Using the boundary condition at t = 0, we find that c(s) = 10g(I - G 01} = 10g(I - s-l}, and hence Gt(s) = se-J...t I{I - s(I - e- At )}. Expand in powers of s to find that Z(t) has the geometric distribution IP'(Z(t) = k) = (1 - e-At)k-le-At for k 2: 1. 2.

The equation becomes

aG t 1 2 at = 2(1 +G t ) -Gt with boundary condition Go(s) = s. This differential equation is easily solved with the result Gt(s) = 2s + t(I - s) = -:--_4-'c-It_-,2+t(l-s) 2+t(1-s)

2-t

We pick out the coefficient of sn to obtain

IP'(Z(t)

t) = n) = -4- (-n , n 2: t(2 + t)

2 +t

1,

and therefore

( () - )-?;

IP'Zt >k -

It follows that, for x >

IP'(Z(t) 2: xt

00

4 -t (2 + t)

°and in the limit as t

I Z(t)

> 0) =

(t)n -- -2 ( -t- )k 2 +t - t 2 +t ' --+ 00,

IP'(Z(t) > xt) ( t ) LxtJ-l ( 2) l-LxtJ 2x = -= 1+--+ e- . IP'(Z(t) 2: I) 2+t t

240

Solutions

Characteristic junctions

[5.6.1]-[5.7.2]

5.6 Solutions. Expectation revisited 1. Set a = E(X) to find that u(X) 2: u(EX) obtain the result. 2. Certainly Zn = 2:7=1 Xj and Z dominated convergence.

3.

+ A(X -

EX) for some fixed A. Take expectations to

= 2:~1 IXj I are such that IZn I ::s Z, and the result follows by

Apply Fatou's lemma to the sequence {- Xn : n 2: l} to find that E(limsup Xn) = -E(lim inf -X n ) 2: -lim inf E( -Xn) = lim supE(Xn). n-+oo

4.

n-+oo

Suppose that EI Xr I <

00

n-+oo

n-+oo

where r > O. We have that, if x > 0,

xrlP'(IXI 2: x)

::s

1

r u dF(u)

~

0

as x

~

00,

[x ,00)

where F is the distribution function of IXI. Conversely suppose that xrlP'(IXI 2: x) ~ 0 where r 2: 0, and let 0 M limM -+00 Jo US d F (u) and, by integration by parts,

::s s

< r. Now EIXsl

=

The first term on the right-hand side is negative. The integrand in the second term satisfies sus -1lP'( IX I > ::s sus-I. u- r for all large u. Therefore the integral is bounded uniformly in M, as required.

u)

5. Suppose first that, for all E > 0, there exists 8 = 8(E) > 0, such that E(JXIIA ) < E for all A satisfying lP'(A) < 8. Fix E > 0, and find x (> 0) such that lP'(JXI > x) < 8(E). Then, for y > x,

Hence

J!.y lui dFx(u) converges as y ~

ConverselysupposethatEIXI <

00.

00,

whence EIXI <

00.

It follows thatE (IXII(!XI>Y}) ~ Oasy ~

and find y such that E (IXII(!XI>Y}) < ~E. For any event A, IA Hence

00.

LetE > 0,

::s IAnBc+IB whereB =

{IXI > y}.

E(JXIIA)::s E(IXIIAnBc) +E(IXIIB)::S ylP'(A)

+ 1E.

Writing 8 = E/(2y), we have that E(IXIIA) < E iflP'(A) < 8.

5.7 Solutions. Characteristic functions Let X have the Cauchy distribution, with characteristic function ¢(s) = e- isi . Setting Y = X, we have that ¢x+y(t) = ¢(2t) = e- 2ltl = ¢x(t)¢y(t). However, X and Yare certainly dependent.

1. 2.

(i) It is the case that Re{¢(t)} = E(costX), so that, in the obvious notation, Re{l - ¢(2t)} =

L: L:

::s 4

(l - cos(2tx)} dF(x) = 2 (l - cos(tx)} dF(x)

L:

= 4Re{l

241

(l - cos(tx»){l - ¢(t)}.

+ cos(tx)} dF(x)

[5.7.3]-[5.7.6]

Solutions

Generating functions and their applications

(ii) Note first that, if X and Y are independent with common characteristic function ¢, then X - Y has characteristic function 1/I'(t)

= E(eitX)E(e-itY) = ¢(t)¢(-t) = ¢(t)¢(t) = 1¢(t)12.

Apply the result of part (i) to the function 1/1' to obtain that 1 - 1¢(2t) 12 ~ 4(1 - 1¢(t)1 2). However 1¢(t)1 ~ 1, so that 1 -1¢(2t)1 ::::: 1 -1¢(2t)1 2 ~ 4(1-I¢(t)1 2) ~ 8(1-I¢(t)l).

3.

(a) With mk

= E(X k ), we have that

say, and therefore, for sufficiently small values of 0,

L - or+ 00

Kx(O) =

(

r=l

1

S(O/.

r

Expand S(Ol in powers of 0, and equate the coefficients of 0,0 2,0 3, in tum, to find that k1 (X) k2(X)

= m2 -

mI, k3(X)

= m1, + 2miKx+y(O) = log{E(e 8X )E(e 8Y )} = Kx(O) + Ky(O), whence the

= m3 -

(b) If X and Y are independent, claim is immediate.

3m1m2

4.

The N(O, 1) variable X has moment generating function E(e 8X )

5.

(a) Suppose X takes values in L(a, b). Then

I 2

= e'18

,so that Kx(O)

= J-02.

since only numbers of the form x = a + bm make non-zero contributions to the sum. Suppose in addition that X has span b, and that l¢x(T)1 = 1 for some T E (0,2rr/b). Then ¢x(T) = eic for some c E JR. Now E(cos(TX - c))

= !E(eiTX-ic + e-iTX+ic) = 1,

using the fact that E(e- iTX ) = ¢x(T) = e- ic . However cos x ~ 1 for all x, with equality if and only if x is a multiple of 2rr. It follows that T X - c is a multiple of 2rr, with probability 1, and hence that X takes values in the set L (c / T, 2rr / T). However 2rr / T > b, which contradicts the maximality of the span b. We deduce that no such T exists. (b) This follows by the argument above. 6. This is a form of the 'Riemann-Lebesgue lemma'. It is a standard result of analysis that, for E > 0, there exists a step function g€ such thatJ~oo If(x) - g€(x)1 dx < E. Let ¢€ (t) = J~oo eitxg€(x)dx. Then

If we can prove that, for each E, I¢€ (t) 1 -+ large t, and the claim then follows.

°as t

-+ ±oo, then it will follow that I¢ X (t) 1 < 2E for all

242

Solutions

Characteristic functions

[5.7.7]-[5.7.9]

Now g" (x) is a finite linear combination of functions of the form CIA (x) for reals c and intervals A, that is gE (x) = I:,f=l CkIAk (x); elementary integration yields

where ak and bk are the endpoints of Ak. Therefore

I
2 K -t '"' L.J Ck -+

0,

as t -+ ±oo.

k=l

7.

If X is N(p" 1), then the moment generating function of X2 is M 2(S) = lE(e Sx 2 ) = X

1

00

1 I( ) 2 dx = eSx 2 --e-Z X-f.J,

v'2ii

-00

2

1

p, s ) , exp ( .JI=2S 1 - 2s

if s < ~, by completing the square in the exponent. It follows that

My(s) =

{1 ( p,~s }].JI=2S n

exp

_J_) }

1 - 2s

(1- 1

=

2s)n/2

exp

(

-s()- ) . 1 - 2s

It is tempting to substitute s = it to obtain the answer. This procedure may be justified in this case using the theory of analytic continuation.

8.

(a) T2

= X 2 /(Y/n), where X2 is X2(1; p,2) by Exercise (5.7.7), and Y is x2(n).

Hence T2 is

F(1, n; p,2).

(b) F has the same distribution function as (A 2

+ B)/m

Z= ----'---

V/n

where A, B, V are independent, A being N(JB, 1), B being x 2 (m-l), and V being x 2 (n). Therefore

lE(Z) = =

~ {lE(A 2 )lE (~) + (m _ ~ m

{(1

l)lE

(B/~/~

1)) }

+(})_n_ + (m _l)_n_} = n(m +(}), n- 2 n- 2 m(n - 2)

where we have used the fact (see Exercise (4.10.2)) that the F(r, s) distribution has mean s/(s - 2) if s > 2.

~t X be independent of X with the same distribution. Then X - X and, by the inversion theorem,

9.

1

00

-1

27r

Now set x

= o.

lIP (t)1 2 e-1tX . dt =

1

lIP 12 is the characteristic function of

00

fx_x(x) =

-00

f(y)f(x

+ y)dy.

-00

We require that the density function of X -

243

X be differentiable at o.

[5.7.10]-[5.8.2]

Solutions

Generating functions and their applications

10. By definition, e-ity¢x(y) =

1:

eiy(x-t) fx(x)dx.

Now multiply by fy (y), integrate over y E JR, and change the order of integration with an appeal to

Fubini's theorem.

J:

11. (a) We adopt the usual convention that integrals of the form g (y) d F (y) include any atom of the distribution function F at the upper endpoint v but not at the lower endpoint u. It is a consequence that Fr: is right-continuous, and it is immediate that F, increases from to 1. Therefore F, is a distribution function. The corresponding moment generating function is

°

MrCt) =

1

1 e tx dFrCx) = - -00 M(t) 00

1

00

e tx +rx dF(x) =

-00

M(t + r) . M(t)

(b) The required moment generating function is MX+y(t

+ r)

MX(t

+ r)My(t + r)

Mx (t)My (t)

Mx+y(t)

the product of the moment generating functions of the individual tilted distributions.

5.8 Solutions. Examples of characteristic functions (i) We have that ¢jet) = lE(e itx ) = lE(e- itX ) = ¢-x(t). (ii) If X I and X 2 are independent random variables with common characteristic function ¢, then ¢Xl +X2 (t) = ¢Xl (t)¢X2 (t) = ¢ (t)2.

1.

(iii) Similarly, ¢X,-X2 (t) = ¢Xl (t)¢-X2 (t) = ¢ (t)¢ (t) = I¢ (t)12.

(iv) Let X have characteristic function ¢, and let Z be equal to X with probability otherwise. The characteristic function of Z is given by

i and to -X

where we have used the argument of part (i) above. (v) If X is Bernoulli with parameter ~, then its characteristic function is ¢ (t) = ~ + ~eit. Suppose Y is a random variable with characteristic function 1/I(t) = 1¢(t)l. Then 1/1 (t)2 = ¢(t)¢(-t). Written in terms of random variables this asserts that YI + Y2 has the same distribution as X I - X2, where the Yi are independent with characteristic function 1/1, and the Xi are independent with characteristic function ¢. Now Xj E (O, l), so that XI - X2 E {-1,0, I}, and therefore Yj E Write

(-i, i).

I

ex = JP>(Yj = 2"). Then JP>(YI

+ Y2

2 = 1) = ex = JP>(X I - X2 = 1) = ~,

JP>(YI +Y2 = -1) = (l-ex)2 =JP>(XI - X2 = -1) =~,

implying that ex 2 = (1 - ex)2 so that ex = such variable Y exists.

2.

i, contradicting the fact that ex 2 =

For t ::: 0,

Now minimize over t ::: 0.

244

~. We deduce that no

Solutions [5.8.3]-[5.8.5]

Examples of characteristic junctions

3.

The moment generating function of Z is MZ(t)

= lE {lE(e tXY I Y)} = lE{Mx(tY)} = lE { =

1(

lao

A

--

)m yn-1(1_ y)m-n-1

A-ty

C. ~

tY ) m}

dy.

B(n,m-n)

Substitute v = 1/ y and integrate by parts to obtain that 00 (v - l)m-n-1

Imn =

1

(AV - t)m

1

dv

satisfies 1 (V_l)m-n-1]00 I mn = - A(m _ 1) (AV - t)m-1 1 [

+

m-n-l A(m _ 1) I m-1,n =cmnAI ( , , ) m-1,n

for some c(m, n, A). We iterate this to obtain Imn

,

= c In+1 ' n = C

'100 1

dv +1

(AV-t)n

c' nA

(A - t)n

for some c' dependingonm, n, A. Therefore Mz(t) = C"(A - t)-n forsomec" dependingonm, n, A. However Mz(O) = 1, and hence c" = An, giving that Z is f'(A, n). Throughout these calculations we have assumed that t is sufficiently small and positive. Alternatively, we could have set t = is and used characteristic functions. See also Problem (4.14.12).

4.

We have that

The integral is evaluated by using Cauchy's theorem when integrating around a sector in the complex plane. It is highly suggestive to observe that the integrand differs only by a multiplicative constant from a hypothetical normal density function with (complex) mean /L(1 - 2O' 2it)-1 and (complex) variance 0'2(1 - 2O' 2it)-1.

5.

(a) Use the result of Exercise (5.8.4) with /L

characteristic function of the

=

0 and 0'2

1: cf>x 2 (t) 1

x2 (1) distribution.

(b) From (a), the sum S has characteristic function cf>s(t) of the X2 (n) distribution. (c) We have that

245

1

=

1

(1 - 2it)-'1, the

= (1- 2it)-'1 n , the characteristic function

[5.8.6]-[5.8.6]

Solutions

Generating functions and their applications

Now lE(exp{ - i t2

/X~}) =

1$ 00

_1_ exp

-00

(_~ _ 2x

x2) dx. 2

There are various ways of evaluating this integral. Using the result of Problem (5.12.18c), we find that the answer is e- Itl , whence X1/ X2 has the Cauchy distribution. (d) We have that lE(eitXIX2)

= lE{lE(eitXIX2 I X2)} = lE(I/>Xl (tX2») = lE(e-! t2xi)

1

00

=

-00

~exp{-ix2(1 +t 2 )}dx = ~, 2 1+t

'\I2l!

1

on observing that the integrand differs from the N(O, (1 + t2)-~) density function only by a multiplicative constant. Now, examination of a standard work of reference, such as Abramowitz and Stegun (1965, Section 9.6.21), reveals that 00

10o

cos(xt) d ~

y 1+

t2

( ) t=Kox,

where Ko(x) is the second kind of modified Bessel function. Hence the required density, by the inversion theorem, is f(x) = Ko(lxi)/l!. Note that, for small x, KO(x) ~ -logx, and for large positive x, KO(x) ~ e- x Jl!x/2. As a matter of interest, note that we may also invert the more general characteristic function I/>(t) = (1 - it)-a (1 + it)-f3. Setting 1 - it = -z/x in the integral gives f(x)

= -1

2l!

1

00

-00

1-

x +ixOO e- itx e- Xx a - 1 e- Z dz dt = ------,;-:--f3 (1- it)a(1 + it)f3 2 2l!i -x-ixoo (_z)a (1 +z/(2x»f3

1

eX (2x p (f3-a) -----Wl 1 (2x) r(a) z(a-f3),z(1-a-f3)

where W is a confluent hypergeometric function. When a

= f3 this becomes

1

(x/2)a-~

f(x) =

r(a),J7r Ka_! (x)

where K is a Bessel function of the second kind. (e) Using (d), we find that the required characteristic function is I/>Xl X2 (t)I/> X3 X4 (t) = (1 + t 2 )-1. In order to invert this, either use the inversion theorem for the Cauchy distribution to find the required density to be f(x) = ie-'x, for -00 < x < 00, or alternatively express (1 + t 2 )-1 as partial fractions, (1 + t 2 )-1 = ~{(1 - it)-l + (1 + it)-l}, and recall that (1- it)-l is the characteristic function of an exponential distribution.

6. The joint characteristic function of X = (X 1, X2, ... , Xn) satisfies I/>x(t) = lE(e itX') = lE(e iY ) where t = (t1, t2, ... , tn ) E ~n and Y = tX' = t1X1 + ... + tnXn. Now Y is normal with mean and variance n

lE(Y)

= '2)jlE(Xj) = tIL',

n

var(Y)

j=l

=

L

tjtkcOV(Xj, Xk)

= tVt',

j,k=l

where IL is the mean vector of X, and V is the covariance matrix of X. Therefore I/>x(t) = I/>y(l) exp(itlL' - itVt') by (5.8.5).

246

=

Solutions

Inversion and continuity theorems Let Z

e-

=X-

[5.8.7]-[5.9.2]

/L. It is easy to check that the vector Z has joint characteristic function ¢z(t)

=

itvt', which we recognize by (5.8.6) as being that of the N (0, V) distribution.

1 2 1 2 7. We have that E(Z) = 0, E(Z2) = 1, and E(e tZ ) = E{E(e tZ I U, V)} = E(e2t ) = e2t . If X and Y have the bivariate normal distribution with correlation p, then the random variable Z = (UX + Vy)/VU2 + 2pUV + V2 is N(O, 1). 8. By definition, E(e itX ) = E(cos(tX» + m(sin(tX». By integrating by parts, 00

10o

A2

00

cOs(tx)Ae- Ax dx = -2--2' A +t

10o

•

sm(tx)Ae

-Ax

At dx = -2--2' A +t

and

A2+iAt A2 + t 2

9.

(a) We have that e-[x[ = e- x I{x2:O}

¢(t) =

(b) By a similar argument applied to the 1

¢(t)

= 2:

A A - it'

+ eX I{x
I(

2:

=

1 1 - it

1)

+ 1 + it

1

= 1 + t2 .

rei, 2) distribution, we have in this case that

(1

(1 - it)2

1)

+ (1 + it)2 =

2

1- t (1 + t 2 )2'

10. Suppose X has moment generating function M(t). The proposed equation gives M(t) =

1

10o M(ut)2 du =

lot

-1 M(v)2 dv. t 0

Differentiate to obtain tM' + M = M2, with solution M(t) = A/(A distribution has the stated property.

+ t).

Thus the exponential

11. We have that ¢X,Y(s, t) = E(eisX+itY) = ¢sx+ty(1). Now s X + t Y is N (0, s2a 2 + 2sta rp + r2) where a 2 = var(X), r2 = var(y), p therefore ¢X,Y(s, t) = exp{ - i(s2a 2 + 2starp + t 2 r2)}.

= corr(X, y), and

The fact that ¢x, Y may be expressed in terms of the characteristic function of a single normal variable is sometimes referred to as the Cramer-Wold device.

5.9 Solutions. Inversion and continuity theorems 1.

Clearly, for 0 ~ y ~ 1, JP'(Xn ~ ny)

= n- 1 Lny J -+

y as n -+

00.

2. (a) The derivative of Fn is In(x) = 1 - cos(2nnx), for 0 ~ x ~ 1. It is easy to see that In is non-negative and fol In (x) dx = 1. Therefore Fn is a distribution function with density function In. (b) As n -+

00,

I

sin(2nnx) 1 < --+0 2nn - 2nn '

I

247

[5.9.3]-[5.9.6]

Generating functions and their applications

Solutions

and so Fn (x) ---+ x for 0 ::: x ::: 1. On the other hand, cos (2mr x) does not converge unless x E {O, I}, and therefore fn(x) does not converge on (0,1).

3. We may express N as the sum N = Tl + T2 + ... + Tk of independent variables each having the geometric distribution 1P'(1j = r) = pqr-l for r ~ 1, where p + q = 1. Therefore k

¢N(t)

= ¢Tl (t) = {I

.}k

pelt _ qe it

'

implying that Z = 2Np has characteristic function

¢z(t) = ¢N(2pt)

=

{P(1 + 2pit + o(p» p(1 _ 2it + 0(1»

pe2pit }k = { I _ (1 _ p)e2pit

}k

. -k ---+ (1 - 21t)

+

as p 0, the characteristic function of the r( ~, k) distribution. The result follows by the continuity theorem (5.9.5).

4.

All you need to know is the fact, easily proved, that "l/fm(t)

l

rr

-rr

d "l/fj (t)"l/fdt) t

=

{2rc

0

if j if j

= e itm

satisfies

+ k = 0, + k # 0,

for integers j and k. Now, ¢(t)

-1 2rc

l

rr

-rr

= I:~-oo eitjlP'(X = j), so that

L

. 1 00 IP'(X=j) e-ttk¢(t)dt=2rc . }=-oo

l

rr

-rr

I "l/fj(t)"I/f-k(t)dt=-.IP'(X=k)·2rc. 2rc

If X is arithmetic with span A, then X/A is integer valued, whence IP'(X = kA) = - A 2rc

l

rr/A

e-itkA¢x(t) dt.

-rr/A

5. Let X be uniformly distributed on [-a, a], Y be uniformly distributed on [-b, b], and let X and Y be independent. Then X has characteristic function sin(at)/(at), and Y has characteristic function sin(bt)/(bt). We apply the inversion theorem (5.9.1) to the characteristic function of X + Y to find that

I -2 rc 6.

1

00

I ¢x+y(t) dt = -2 -00 rc

1

00

-00

sin(at) sin(bt) a /\ b b 2 dt = fx+Y(O) = - 2 b . a t a

It is elementary that

In addition, a

= n, f~'(a) = _n- 1, and

and Stirling's formula follows.

248

Solutions [5.9.7]-[5.10.1]

Two limit theorems

7. The vector X has joint characteristic function ifJ(t) = exp( - ~tVt'). By the multidimensional version of the inversion theorem (5.9.1), the joint density function of X is

f exp( -itx' JRn

f(x) = _1_

(2n)n

Therefore, if i

=1=

j,

and similarly when i = j. When i 8 -lP(maxXk::=:U) = 8 Vi)

itVt') dt.

k

=1=

j,

1 Q

-8f dx 8vij

whereQ={X:Xk::=:ufork=I,2, ... ,n}

l' .

where dx' is an integral over the variables Xk for k =1= i, j. Therefore, lP(maxk Xk ::=: u) increases in every parameter Vi), and is therefore greater than its value when Vi) = 0 for i =1= j, namely Ilk lP(Xk ::=: u). 8.

By a two-dimensional version of the inversion theorem (5.9.1) applied to JE(e itX'), t = (tl, t2),

~lP(Xl>0,X2>0)=~ (Xl fOO{~ ff exp(-itx'-~tVt')dt}dX 8p

8p

8

Jo Jo I

ff

= 8p 4n 2 JJR2

4n

JJR2

exp(-~tVt') (itJ}(it2)

dt

l ')dt = 2n = - I2 i~ exp(-:ztVt

~

~

JiV=II 2 = ~

I ~.

~yl-~

We integrate with respect to p to find that, in agreement with Exercise (4.7.5),

1 lP(X I > 0, X2 > 0) = 4

I 1 + -sinp.

2n

5.10 Solutions. Two limit theorems 1. (a) Let {Xi: i ~ I) be a collection of independent Bernoulli random variables with parameter ~. Then Sn = 2:1 Xi is binomially distributed as bin(n, ~). Hence, by the central limit theorem,

where cI> is the N (0, 1) distribution function.

249

[5.10.2]-[5.10.4]

Solutions

Generating junctions and their applications

(b) Let {Xj : i ::: l} be a collection of independent Poisson random variables, each with parameter 1. Then Sn = Xi is Poisson with parameter n, and by the central limit theorem

2:1

e- n

k nk! = JP> (ISn-n In l

L

::: x )

---+ (x) - (-x),

as above.

k: Ik-nl~xJn

2. A superficially plausible argument asserts that, if all babies look the same, then the number X of correct answers in n trials is a random variable with the bin(n, ~) distribution. Then, for large n, JP>

X - In

(

-1_2- > 3

) ::::: 1 - (3) :::::

'tJn

-rdoo

by the central limit theorem. For the given values of n and X, X - ~n

iJn

= 910 -

750 5./15::::: 8.

in

Now we might say that the event {X > ~Jn} is sufficiently unlikely that its occurrence casts doubt on the original supposition that babies look the same. A statistician would level a good many objections at drawing such a clear cut decision from such murky data, but this is beyond our scope to elaborate.

3.

Clearly ¢y(t) =lE{lE(e itY

=lE{exp(X(e it

I X)}

-l))} = (

~t

1 - (e l

-

1)

)S = (_I_. )S 2- e t l

It follows that

1 I lE(Y) = -:-¢y(O) = s, I

whence var(Y) = 2s. Therefore the characteristic function of the normalized variable Z = (Y lEY)/v'var(Y) is Now, log{ ¢y (t /J2s)} = -s log (2

= it# -

-

e it / v

'2s)

= s (eit/vTs -

1) + ~s (eit/vTs - 1)2 + 0(1)

!t 2 - !t 2 + 0(1),

where the 0(1) terms are as s ---+ 00. Hence log{¢z(t)} ---+ _~t2 as s ---+ 00, and the result follows by the continuity theorem (5.9.5). Let PI, P2, ... be an infinite sequence of independent Poisson variables with parameter 1. Then Sn = PI + P2 + .. , + Pn is Poisson with parameter n. Now Y has the Poisson distribution with parameter X, and so Y is distributed as Sx. Also, X has the same distribution as the sum of s independent exponential variables, implying that X ---+ 00 as s ---+ 00, with probability 1. This suggests by the central limit theorem that Sx (and hence Y also) is approximately normal in the limit as s ---+ 00. We have neglected the facts that s and X are not generally integer valued.

4. Since X 1 is non-arithmetic, there exist integers n 1, n2, ... , nk with greatest common divisor 1 and such that JP>(X 1 = ni) > 0 for 1 ::: i ::: k. There exists N such that, for all n ::: N, there exist nonnegative integers tll, tl2, ... , tlk such that n = tll n 1 + ... + tlknk. If x is a non-negative integer, write

250

Solutions

Two limit theorems

[S.10.S]-[S.10.S]

N = f3r n 1 + ... + f3knk, N + x = YI n 1 + ... + Yknk for non-negative integers f3I, ... , f3b YI ... , Yk· Now Sn = Xl + ... + Xn is such that k

lP'(SB = N) ::: lP'(Xj = ni for Bi-I < j

:s Bi, 1 :s i :s k)

=

II lP'(XI = ni){Ji > 0 i=I

where Bo = 0, Bi = f31 + f32 + G = YI + Y2 + ... + Yk. Therefore

... + f3i,

B = Bk· Similarly lP'(SG = N

+ x)

> 0 where

lP'(SG - SG.B+G = x) ::: lP'(SG = N +X)lP'(SB = N) > 0

where SG,B+G S.

= 2:f=+GG+ I Xi.

Also, lP'(SB - SB,B+G

= -x)

> 0 as required.

Let Xl, X2, ... be independent integer-valued random variables with mean 0, variance 1, span 1,

and common characteristic function ¢. We are required to prove that JnlP'(Un as n ---? 00 where Xl + X2 + ... + Xn 1 Un = - Sn = ----=.---=---=0-------'-

In and x is any number of the form kj In for integral k.

I 2

= x) ---? e -:1 x

j.Jiif

In

The case of general J.t and (52 is easily derived

from this. By the result of Exercise (5.9.4), for any such x,

since Un is arithmetic. Arguing as in the proof of the local limit theorem (6),

2rr IJnlP'(Un = x) - f(x)1 where

f is the N (0,

:s In + ln

1) density function, and

Now ln = 2.Jiif (1 - cI> (rr In)) for In, pick /) E (0, rr). Then

---?

0 as n

---? 00,

where cI> is the N (0, 1) distribution function. As

I 2

The final term involving e-'1 t is dealt with as was In. By Exercise (5.7.5a), there exists such that 1¢(t)1 < J... if /) :s It I :s rr. This implies that

and it remains only to show that as n

251

---? 00.

J... E

(0,1)

[5.10.6]-[5.10.7]

Solutions

Generating functions and their applications

The proof of this is considerably simpler if we make the extra (though unnecessary) assumption that m3 = E[Xy [ < 00, and we assume this henceforth. It is a consequence of Taylor's theorem (see Theorem (5.7.4» that ¢(t) = 1- ~t2 - t,it3m3 +0(t 3 ) as t -+ O. It follows that¢(t) = e _~t2+t3e(t) [x[e 1xl , and therefore

.:s

for some finite 8(t). Now [eX - I[

Let K8 = sup{[8(u)[ : [u[ .:s 15}, noting that K8 < and 15K8 < For [t[ < 15.,Jii,

!.

00,

and pick 8 sufficiently small that 0 < 8 < n

en _lt 2 [ [t[3 2 12 [t[3 _I t 2 [¢(t/vn) -e 2 .:sK8.,Jiiexp(t15K8-zt).:sK8.,Jiie 4 ,

and therefore as n -+

00

as required. 6.

The second moment of the Xi is 2

e-

loo

I

x2 dx 2x (log x)2

=

1-

1

-00

e2u -d u u2

(substitute x = e U ), a finite integral. Therefore the X's have finite mean and variance. The density function is symmetric about 0, and so the mean is O. By the convolution formula, if 0 < x < e- 1,

1

e- I

jz(x) =

-e- I

X

f(y)f(x - y) dy::::

r Jo

X

f(y)f(x - y) dy :::: f(x)

r Jo

f(y) dy,

since f(x - y), viewed as a function of y, is increasing on [0, x]. Hence jz(x) >

~=

- 210g [xl

1 4[x[(log [x[)3

for 0 < x < e- 1• Continuing this procedure, we obtain kn fn(x) :::: [x [(log [x [)n+l '

for some positive constant kn . Therefore fn (x) -+ 00 as x -+ 0, and in particular the density function of (X 1 + ... + Xn) / .,Jii does not converge to the appropriate normal density at the origin.

7.

We have for s > 0 that ¢(is)

= ~ v 2n

roo exp( -(2x)-1 - xs )x-3j2 dx 10 exp (-Zy 1 2 - sy - 2) 2dy by substituting x = y- 2 Jo

00

1 =~

v2n

0

= exp(-5s), 252

Solutions

Large deviations

[5.10.8]-[5.11.3]

by the result of Problem (5.I2.I8c), or by consulting a table of integrals. The required conclusion follows by analytic continuation in the upper half-plane. See Moran 1968, p. 271. (a) The sum Sn = 2:~=1 Xr has characteristic function E(e itSn ) = ¢(t)n = ¢(tn 2 ), whence Un = Snln has characteristic function ¢(tn) = E(eitnXl). Therefore,

8.

lP(Sn
<~) ~

as n

0

~

00.

(b) E(e itTn ) = ¢(t) = E(e itX 1).

9. (a) Yes, because Xn is the sum of independent identically distributed random variables with non-zero variance. (b) It cannot in general obey what we have called the central limit theorem, because var(Xn ) = (n 2 - n) var(6) + nE(6)(1 - E(6» and n var(Xl) = nE(6)(1 - E(6» are different whenever var(6) =I O. Indeed the right 'normalization' involves dividing by n rather than In. It may be shown when var(6) =I 0 that the distribution of Xn In converges to that of the random variable 6.

5.11 Solutions. Large deviations 1. We may write Sn = 2:7 Xi where the Xi have moment generating function M(t) = i(e t +e- t ). Applying the large deviation theorem (5.11.4), we obtain that, for 0 < a < 1, lP(Sn > an)l/n ~ inft>o{g(t)} where g(t) = e- at M(t). Now g has a minimum when e t = J(1 + a)/(1 - a), where it takes the value II y' (1 + a)1+a (1 - a)l-a as required. If a :::: 1, then lP(Sn > an) = 0 for all n.

i.

2. (i) Let Yn have the binomial distribution with parameters nand Then 2Yn - n has the same distribution as the random variable Sn in Exercise (5.11.1). Therefore, if 0 < a < 1,

and similarly for lP(Yn - in < -ian), by symmetry. Hence

(ii) This time let Sn = Xl + ... + X n , the sum of independent Poisson variables with parameter 1. Then Tn = enlP(Sn > n(1 + a». The moment generating function of Xl - 1 is M(t) = exp(e t - 1 - t), 1 and the large deviation theorem gives that Tn /n ~ einft>o{g(t)} where g(t) = e- at M(t). Now g' (t) = (e t - a-I) exp(e t - at - t - 1) whence g has a minimum at t = log(a + 1). Therefore T;/n ~ eg(log(I

+ a» = {el(a + l)}a+l.

Suppose that M(t) = E(e tX ) is finite on the interval [-8,8]. Now, for a > 0, M(8) :::: e 8a lP(X > a), so that lP(X > a),:::: M(8)e- 8a . Similarly, lP(X < -a),:::: M(-8)e- 8a . Suppose conversely that such A, JL exist. Then 3.

M(t) .:::: E(e 1tX1 ) =

r

e 1t1x dF(x)

i[o,oo)

where F is the distribution function of [XI. Integrate by parts to obtain M(t).:::: 1 + [-e1t1x[l-

F(x)]l~ + 10 253

00

[t[e1t1x[l- F(x)]dx

[5.11.4]-[5.12.2]

Solutions

Generating functions and their applications

(the term '1' takes care of possible atoms at 0). However 1 - F(x) ~ J1-e-J...x, so that M(t) < It I is sufficiently small.

4.

00

if

The characteristic function of Snln is {e-1t/nl)n = e- 1tl , and hence Snln is Cauchy. Hence

JP'(Sn > an) =

1

dx

00

a

n(1

2

+ x)

1 = n

(n- 2

1 )

tan- a .

5.12 Solutions to problems 1.

The probability generating function of the sum is

1_ {(iI~ i} = (1) ~ 6

S

10

(is

10 {

s6 } 10

=

(1) (is

10

(1 - lOs

6

+ ... )(1 + lOs + ... ).

The coefficient of s27 is

2.

(a) The initial sequences T, HT, HHT, HHH induce a partition of the sample space. By conditioning on this initial sequence, we obtain f(k) = qf(k - 1) + pqf(k - 2) + p2qf(k - 3) for k > 3, where p + q = 1. Also f(1) = f(2) = 0, f(3) = p3. In principle, this difference equation may be solved in the usual way (see Appendix I). An alternative is to use generating functions. Set G(s) = I:~1 sk f(k), multiply throughout the difference equation by sk and sum, to find that G(s) = p3 s 3/{1 - qs - pqs2 - p2qs 3). To find the coefficient of sk, factorize the denominator, expand in partial fractions, and use the binomial series. Another equation for f(k) is obtained by observing that X = k if and only if X> k - 4 and the last four tosses were THHH. Hence

f(k)

= qp3 ( 1 -

k-4

)

~ f(i) ,

k> 3.

1=1

Applying the first argument to the mean, we find that J1-

J1-)

+ p2q(3 + J1-) + 3p 3 and hence J1- = (1 + p + p2)1 p3.

= E(X) satisfies J1- = q(1 + J1-) + pq(2+

As for HTH, consider the event that HTH does not occur in n tosses, and in addition the next three tosses give HTH. The number Y until the first occurrence of HTH satisfies

= JP'(Y = n + l)pq + JP'(Y = n + 3), (pq + l)/(p2 q ).

JP'(Y > n)p2q

n ~ 2.

Sum over n to obtain E(y) = = (q + ps)n, in the obvious notation. (i) JP'(2divides N) = ~{GN(1) + GN(-I)), since only the coefficients of the even powers of s

(b) G N (s)

contribute to this probability. (ii) Let w be a complex cube root of unity. Then the coefficient of JP'(X G N (w 2 )) is

~{1 +w3 +(6 )

=

1,

~{1+w+(2)=0,

~{1

2

+ w +(

4

)

= 0, 254

ifk

= 3r,

ifk=3r+l, if k

= 3r + 2,

= k) in ~ {G N (1) + G N (w) +

Solutions

Problems

~{GN(1)

for integers r. Hence + GN(W) + GN(W 2 )} is a mUltiple of 3. Generalize this conclusion.

=

[5.12.3]-[5.12.6]

2:;!~J lP'(N = 3r), the probability that N

3. We have that T = k if no run of n heads appears in the first k - n - 1 throws, then there is a tail, and then a run of n heads. Therefore lP'(T = k) = lP'(T > k - n - l)qpn for k :::: n + 1 where p + q = 1. Finally lP'(T = n) = pn. Multiply by sk and sum to obtain a formula for the probability generating function G of T: 00 n+j L lP'(T = j) = qpn L lP'(T = j) L sk k=n+1 j>k-n-1 j=l k=n+1 qpnsn+1 00 . qp n sn+1 l-s LlP'(T=j)(1-sJ)= l-s (1-G(s». j=l 00

G(s) - pnsn = qpn

=

L

sk

Therefore

4.

The required generating function is G(s)

where p 5.

+q =

=

L00 S k k=r

(k - 1)

pr(1- p) k -r

pS = (_ _) r

r - 1

1 - qs

1. The mean is G' (1) = r / p and the variance is Gil (1) + G'(1) - {G' (1)}2 = rq / p2.

It is standard (5.3.3) that Po (2n)

=

n en ) (pq)n. Using Stirling's formula,

The generating function Fo(s) for the first return time is given by Fo(s) = 1 - PO(s)-l where Po(s) = 2:n s2n po(2n). Therefore the probability of ultimate return is Fo(1) = 1 - A-I where, by Abel's theorem, 'f 1 1 P=q=2> A = L po(2n){ = 00 n

<00

ifpi=q.

= 1 if and only if p = !. = X~ + Y; satisfies

Hence Fo(l) 6.

(a) Rn

Hence Rn = n + Ro = n. (b) The quick way is to argue as in the solution to Exercise (5.3.4). Let Un = Xn + Yn , Vn = Xn - Yn . Then U and V are simple symmetric random walks, and furthermore they are independent. Therefore

PO(2n) = lP'(U2n = 0, V2n = 0) = lP'(U2n = 0)lP'(V2n = 0) = {

255

(~) 2n (~) }

2 ,

[5.12.7]-[5.12.8]

Solutions

Generating functions and their applications

by (5.3.3). Using Stirling's formula, po(2n) ~ (mr)-l, and therefore ~n po(2n) = 00, implying that the chance of eventual return is I. A longer method is as follows. The walk is at the origin at time 0 if and only if it has taken equal numbers of leftward and rightward steps, and also equal numbers of upward and downward steps. Therefore Po(2n)

(I) J;

= 4

2n

n

(2n)! (m!)2{(n - m)!}2

( I ) 4n (2n) 2 n

= 2:

7. Let eij be the probability the walk ever reaches j having started from i. Clearly eao = ea,a-lea-l,a-2···elO, since a passage to 0 from a requires a passage to a - I, then a passage to a - 2, and so on. By homogeneity, eao = (elO)a. By conditioning on the value of the first step, we find that elO cubic equation x = px 3 + q has roots x = I, c, d, where

c=

-p - Vp2 +4pq

Icl > I, and Idl ~ I if and only if p2 + 4pq = I if p ::: 1, so that eao = I if p ::: 1·

Now elO

d=

2p

When p >

-p

= pe30 + qeoo = peIo + q.

+ Vp2 +4pq 2p

The

.

~ 9p2 which is to say that p :::

1. It follows that

1, we have that d < I, and it is actually the case that elO = d, and hence eao

=(

_p

+ Vp2 +4 pq )a

'f P > 3"' 1

1

2p

In order to prove this, it suffices to prove that eao < I for all large a; this is a minor but necessary chore. Write Tn = Sn - So = ~7=1 Xi, where Xi is the value of the ith step. Then eaO = JP'(Tn ::: -a for some n ~ I) = JP'(nj.t - Tn ~ nj.t

+ a for some n

~

I)

00

::: L

JP'(nj.t - Tn ~ nj.t

+ a)

n=l

where j.t

= E(X 1) = 2p -

q > O. As in the theory of large deviations, for t > 0,

where X is a typical step. Now E(et(/L-X)) = I + o(t) as t -!- 0, and therefore we may pick t > 0 such that B(t) = e-t/LE(et(/L-X)) < 1. It follows that eao ::: ~~l e-taB(t)n which is less than I for all large a, as required. 8.

We have that

where p + q = 1. Hence Gx,y(s, t) = G(ps + qt) where G is the probability generating function of X + Y. Now X and Y are independent, so that G(ps

+ qt) =

G x(s)Gy(t) = G x,Y(s, I)G X,y(l, t) = G(ps

256

+ q)G(p + qt).

Solutions [5.12.9]-[5.12.11]

Problems

Write f(u) = G(l + u), x = s - I, y = t - I, to obtain f(px + qy) = f(px)f(qy), a functional equation valid at least when -2 < x, y ::: O. Now f is continuous within its disc of convergence, and also f(O) = 1; the usual argument (see Problem (4.14.5» implies that f(x) = eAx for some A, and therefore G(s) = f(s - 1) = eA(s-l). Therefore X + Y has the Poisson distribution with parameter A. Furthermore, Gx(s) = G(ps + q) = eAp(s-l), whence X has the Poisson distribution with parameter Ap. Similarly Y has the Poisson distribution with parameter Aq.

9. In the usual notation, G n+l(S) = Gn(G(s». It follows that G~+I(l) = G~(1)G'(l)2 G~ (1)G" (l) so that, after some work, var(Zn+l) = fJ,2 var(Zn) + fJ,n(J2. Iterate to obtain var(Zn+l) = (J2(fJ,n for the case fJ,

1=

+ fJ,n+l + ... + fJ,2n)

1. If fJ, = 1, then var(Zn+l) = (J2(n

(J2fJ,n(l- fJ,n+l)

=

1

'

-fJ,

+

n:::: 0,

+ 1).

10. (a) Since the coin is unbiased, we may assume that each player, having won a round, continues to back the same face (heads or tails) until losing. The duration D of the game equals k if and only if k is the first time at which there has been either a run of r - 1 heads or a run of r - 1 tails; the probability of this may be evaluated in a routine way. Alternatively, argue as follows. We record S (for 'same') each time a coin shows the same face as its predecessor, and we record C (for 'change') otherwise; start with a C. It is easy to see that each symbol in the resulting sequence is independent of earlier symbols and is equally likely to be S or C. Now D = k if and only if the first run of r - 2 S's is completed at time k. It is immediate from the result of Problem (5.12.3) that (ls)r-2(l_ls) GD(S)=

2

2

+ (1sr-1

1- s

.

(b) The probability that Ak wins is 00

Jrk

=

L lP'(

D

= n(r -

1)

+k -

1).

n=1

Let W be a complex (r - 1)th root of unity, and set Wk(S)

1 { 1 1 2 =- GD(S) + /(TGD(WS) + --Z-(k 1) GD(W s) r-l wW 1

+"'+w(r-2)(k-l)GD(W

r-2

} s).

It may be seen (as for Problem (5.12.2» that the coefficient of si in Wk(S) is lP'(D = i) if i is of the form n(r - 1) + (k - 1) for some n, and is 0 otherwise. Therefore lP'(Ak wins) = Wk(l). (c) The pool contains £ D when it is won. The required mean is therefore E(D

I Ak wins)

=

E(DI

.)

{Ak WIns)

lP'(Ak wins)

W'(I) = _k_. Wk(l)

(d) Using the result of Exercise (5.1.2), the generating function of the sequence lP'(D > k), k :::: 0, is T (s) = (l - G D (s» / (l - s). The required probability is the coefficient of sn in T (s).

11. Let Tn be the total number of people in the first n generations. By considering the size ZI of the first generation, we see that

z\ Tn = 1 + LTn- l (i) i=1

257

[5.12.12]-[5.12.14]

Solutions

Generating functions and their applications

where Tn-l (1), Tn-l (2), ... are independent random variables, each being distributed as Tn-I' Using the compounding formula (5.1.25), Hn (s) = sG(Hn_l (s».

12. We have that lP'(Z > N n

I Z = 0) = m

lP'(Zn > N, Zm = 0) lP'(Zm = 0)

f =f =

r=l

= 0 I Zn = N + r)lP'(Zn = N + r) lP'(Zm = 0) lP'(Zm-n = o)N+rlP'(Zn = N + r) lP'(Zm

lP'(Zm = 0)

r=l

::::

lP'(Zm = O)N+l 00 lP'(Zm = 0) LlP'(Zn

=N

N

+r):::: lP'(Zm

= 0) = Gm(O)

N .

r=l

13. (a) We have that Gw(s) = GN(G(S» = e)..(G(s)-l). Also, Gw(s)l/n = e)..((G(s)-l)/n, the same probability generating function as G w but with A replaced by AIn. (b) We can suppose that H (0) < 1, since if H (0) = 1 then H (s) = I for all s, and we may take A = 0 and G (s) = I. We may suppose also that H (0) > o. To see this, suppose instead that H (0) = 0 so that H(s) = sr L-f=O sj hj+r for some sequence (hk) and some r ~ 1 such that hr > O. Find a positive integer n such that r I n is non-integral; then H(s)l/n is not a power series, which contradicts the assumption that H is infinitely divisible. Thus we take 0 < H(O) < 1, and so 0 < 1 - H(s) < 1 for 0:::: s < 1. Therefore log H(s)

= 10g(1- {I

- H(s)})

= A( -I + A(s»)

where A = -log H(O) and A(s) is a power series with A(O) = 0, A(1) 00 . L-j=l ajsJ , we have that

1. Writing A(s)

as n ~ 00. Now H(s)l/n is a probability generating function, so that each such expression is nonnegative. Therefore aj ~ 0 for all j, implying that A (s) is a probability generating function, as required.

14. It is clear from the definition of infinite divisibility that a distribution has this property if and only if, for each n, there exists a characteristic function 1/In such that ifJ(t) = 1/In(t)n for all t. (a) The characteristic functions in question are

Poisson (A) :

.

1 2 2

ifJ(t)

= e ll /L-ZC1 = e)..(e -1)

ifJ(t)

=

ifJ(t)

t

l'(

(_A. )/L A -zt

In these respective cases, the 'nth root' 1/In of ifJ is the characteristic function of the N(!Lln, (52 In), Poisson (Aln), and r(A, !LIn) distributions.

258

Solutions [5.12.15]-[5.12.16]

Problems

(b) Suppose that ¢ is the characteristic function of an infinitely divisible distribution, and let 1/In be a characteristic function such that ¢(t) = 1/In (t)n. Now 1¢(t)1 :::: 1 for all t, so that 11/In(t)1

=

{Io

1¢(t)II/n --+

~f 1¢(t)1 =1= 0, If 1¢(t)1 = O.

For any value of t such that ¢(t) =1= 0, it is the case that 1/In(t) --+ 1 as n --+ 00. To see this, suppose instead that there exists f) satisfying 0 < f) < 2rr such that 1/In(t) --+ e ie along some subsequence. Then 1/In (t)n does not converge along this subsequence, a contradiction. It follows that 1/I(t)

= n---+oo lim 1/In (t) = {

I

if ¢(t) =1= 0,

0

if ¢(t)

.

= O.

Now ¢ is a characteristic function, so that ¢(t) =1= 0 on some neighbourhood of the origin. Hence 1/I(t) = 1 on some neighbourhood of the origin, so that 1/1 is continuous at the origin. Applying the continuity theorem (5.9.5), we deduce that 1/1 is itself a characteristic function. In particular, 1/1 is continuous, and hence 1/I(t) = 1 for all t, by (*). We deduce that ¢(t) =1= 0 for all t.

15. We have that JP'(N

= n I S = N) =

JP'(S = n IN = n)JP'(N = n) 2:k JP'(S = kiN = k)JP'(N = k)

pnJP'(N = n) = . 2:bI pkJP'(N = k)

Hence E(x N I S = N) = G(px)/G(p). If N is Poisson with parameter A, then E(x N I S

= N) =

e)..(px-I) e)..(p-I)

= e)..p(x-I) = G(x)P.

Conversely, suppose that E(x N I S = N) = G(x)P. Then G(px) = G(p)G(x)P, valid for Ixl :::: 1, 0< P < 1. Therefore f(x) = 10gG(x) satisfies f(px) = f(p) + pf(x), and in addition f has a power series expansion which is convergent at least for 0 < x :::: 1. Substituting this expansion into the above functional equation for f, and equating coefficients of pi x j , we obtain that f (x) = - A(l - x) for some A ::: O. It follows that N has a Poisson distribution.

16. Certainly Gx(s)

= Gx.y(s.

1) = (1 -

(PI + P2»)n. 1 - P2 - PIS

Gx+Y(s)

giving that X, Y, and X specifically,

JP'(X

= k) =

+Y

(n

+~ -

= PI/(1

= Gx.y(s. s) =

= Gx.y(l. t) =

(1 -

(PI + P2»)n • 1 - PI - P2 t

1 - (PI + P2) ) n ' ( 1 _ (PI + P2)S

have distributions similar to the negative binomial distribution. More

JP'(X

for k ::: 0, where a

Gy(t)

1) a k (l _ a)n,

+ Y = k) =

- P2),,B

JP'(Y

= k) =

(n

+~ -

(n +~-I)yk(l_ y)n,

= P2/(l

- PI), Y = PI

259

+ P2·

1),Bk (1 _ ,B)n,

[5.12.17]-[5.12.19]

Solutions

Generating functions and their applications

Now E(sx

I Y = y) =

E(sX I

{y=y} JP'(Y = y)

)

A B

where A is the coefficient of t Y in Gx.y(s, t) and B is the coefficient of t Y in Gy(t). Therefore E(sX I Y

(1-

= y) =

PI - p2)n I-PIS

=(

(~)Y/ I-PIS

{(I-

PI - p2)n

(~)Y}

I-PI

I - PI ) n+y . 1 - PIS

17. As in the previous solution,

= y I a to obtain

18. (a) Substitute u

(b) Differentiating through the integral sign,

-al = 10 ab

00

0

= -

10

{2b 2 u 2 -b 2 u -2 ) } du --exp(-a u2 00

2exp( _a 2b 2y-2 - y2) dy = -21 (1, ab),

by the substitution u = b I y . (c) Hence al lab

-2aI, whence 1= c(a)e- 2ab where

=

c(a) = I(a, 0) =

00

10o

2 2

e- a

U

,.fir

du = - . 2a

(d) We have that E(e- tX ) = by the substitution x (e) Similarly

=

10o

d

e-tx_e-c/x-gx dx

..jX

= 2dl(.,fi+t,,JC)

y2.

E(e-tx) by substituting x

00

=

roo e-tx _1_e-I/(2x) dx =

io

J2n x 3

= y-2.

19. (a) We have that

260

!II

V-;

(_1_,0) .ji

I-PI

Solutions [5.12.20]-[5.12.22]

Problems

in the notation of Problem (5.12.18). Hence U has the Cauchy distribution. (b) Similarly

for t > O. Using the result of Problem (5.12.18e), V has density function f(x)

(c) We have that W- 2 = X- 2

=

_1_ e - 1/(2X), -J2:rrx3

x> O.

+ y-2 + Z-2. Therefore, using (b),

for t > O. It follows that W- 2 has the same distribution as 9V = 9X- 2 , and so W 2 has the same distribution as ~ X2. Therefore, using the fact that both X and W are symmetric random variables, W has the same distribution as

1X, that is N (0, ~).

20. It follows from the inversion theorem that F(x + h) - F(x) . ----'------::---'--'-= - 1 11m

h

2:rr N-+oo

jN 1 - e- ith . e-ztx¢(t) dt. -N it

Since I¢I is integrable, we may use the dominated convergence theorem to take the limit as h within the integral: f(x)

=~

-.I-

0

lim jN e-itx¢(t)dt.

2:rr N-+oo -N

The condition that ¢ be absolutely integrable is stronger than necessary; note that the characteristic function of the exponential distribution fails this condition, in reflection of the fact that its density function has a discontinuity at the origin. 21. Let G n denote the probability generating function of Zn. The (conditional) characteristic function

ofZn/J-lnis

It is a standard exercise (or see Example (5.4.3)) that

whence by an elementary calculation as n

--+ 00,

the characteristic function of the exponential distribution with parameter 1 - J-l- 1 . 22. The imaginary part of ¢x(t) satisfies

261

[5.12.23]-[5.12.24]

Solutions

Generating functions and their applications

for all t, if and only if X and -X have the same characteristic function, or equivalently the same distribution. 23. (a) U

=

X

+ Y and V = X -

Y are independent, so that ¢Ju+v

= ¢Ju¢Jv, which is to say that

¢J2X = ¢Jx+Y¢Jx-y, or

Write 1jJ(t)

= ¢J(t)/¢J(-t).

Then 1jJ(2t)

=

¢J(2t)

¢J( -2t)

=

¢J(t)3¢J(-t)

= 1jJ(t)2.

¢J( -t)3¢J(t)

Therefore for n ::: 0. However, as h -+ 0,

so that 1jJ(t) = {I + 0(t 2 /2 2n )}2 ¢J( -t) = ¢J(t). It follows that ¢J(t)

n -+

1 as n -+

00,

= ¢J(~t)3¢J(_~t) = ¢J(it)4 = ¢J(t/2n)2zn 2

=

1 - -I . -t2n { 2 2

Zn 2n} 2

+ o(t 2 /2)

1 for all t, giving that

whence 1jJ(t)

for n ::: 1

lIZ

as n -+

-+ e-Z

so that X and Y are N(O, 1). (b) With U = X + Y and V = X - Y, we have that 1jJ(s, t)

= E(eisU+iIV) satisfies

1jJ(s, t) = E(ei(s+I)X+i(s-I)Y) = ¢J(s

+ t)¢J(s -

t).

Using what is given,

However, by (*),

a2~ I at

= 2{¢J"(S)¢J(S) - ¢J'(s)2},

1=0

yielding the required differential equation, which may be written as d , -(¢J /¢J) =-1. ds

Hence 10g¢J(s) = a

+ bs -

1 2

1sZ

ZS for constants a, b, whence ¢J(s) = e-Z

24. (a) Using characteristic functions, ¢JZ(t) (b) EIXii = 00.

= ¢Jx(t/n)n = e- i1i • 262

00,

.

Solutions

Problems

[5.12.25]-[5.12.27]

25. (a) See the solution to Problem (5.12.24). (b) This is much longer. Having established the hint, the rest follows thus: fx+Y(Y)

=

L:

f(x)f(y -x)dx

1

00

=

I

n(4

where J =

=

2

+ y)

L:

{xf(x)

lim

+ f(y

{j(X)

+ J g(y) =

- X)} dx

-00

+ (y -

2

n(4

+y

2

)

+ J g(y)

x)f(y - X)} dx

[~{IOg(l+x2)_IOg(I+(Y_X)2)}]N~ =0.

~~oo~

Finally, fz(z) = 2fx+y(2z) =

I

n(l

2 .

+z )

26. (a) Xl +X2+",+Xn. (b) X I - Xl' where X I and Xl are independent and identically distributed. (c) XN, where N is a random variable with lP'(N = j) = Pj for I :s j :s n, independent of Xl, X2, ... , X n . (d) '£~I Zj where ZJ, Z2, ... are independent and distributed as X J, and M is independent of the Zj with lP'(M = m) = (1)m+I for m ~ O.

(e) YXJ, where Y is independent of Xl with the exponential distribution parameter 1.

27. (a) We require 2eitx

00

ifJ(t)

=

1

-00

e

rex

+ e -rex dx.

First method. Consider the contour integral

where C is a rectangular contour with vertices at ±K, ±K

z

= 1i, with residue e - 2 t / (i n ). 1

+ i.

The integrand has a simple pole at

Hence, by Cauchy's theorem,

as K -+

00.

Second method. Expand the denominator to obtain

I

00

cosh(nx)

k=O

- - - = L(-I)k exp{ -(2k + I)nlxl}. Multiply by e itx and integrate term by term.

263

[5.12.28]-[5.12.30]

Solutions

Generating functions and their applications

(b) Define ¢(t) = 1 - It I for It I .:s 1, and ¢(t) = 0 otherwise. Then

~ 2;rr

~ (1

(00 e-itx¢(t)dt = Loo

2;rr

L1

= -1 10 ;rr

0

e- itx (1-ltJ)dt

1

(1-t)cos(tx)dt

1 = -2(1-cosx). ;rrx

Using the inversion theorem, ¢ is the required characteristic function. (c) In this case, (00 eitxe-x-e-x dx = y-ite-Y dy = ['(1- it)

(00

10

Loo

where [' is the gamma function. (d) Similarly,

L:

1eitxe-lxl dx

= 1{foOO eitxe-x dx + 10 =

00

e-itxe-x dX}

~ { 1 ~ it + 1 ~ it} = 1 ~ t 2 .

(e) We have that lE(X) = -i¢'(O) = -["(1). Now, Euler's product for the gamma function states that . n!n Z ['(z) = hm n-+oo z(z

+ 1) ... (z + n)

where the convergence is uniform on a neighbourhood of the point

z = 1. By differentiation,

I I )+ }

, . { -n- ( l o g n - l - - - · · · - - ['(1)= hm n-+oo n 1 2 n 1

+

=-y.

28. (a) See Problem (S.12.27b). (b) Suppose ¢ is the characteristic function of X. Since ¢' (0) = ¢" (0) = ¢"' (0) = 0, we have that lE(X) = var(X) = 0, so that JP'(X = 0) = 1, and hence ¢(t) = 1, a contradiction. Hence ¢ is not a characteristic function. (c) As for (b). (d) We have that cos t = (e it + e- it ), whence ¢ is the characteristic function of a random variable taking values ± 1 each with probability (e) By the same working as in the solution to Problem (S.12.27b), ¢ is the characteristic function of the density function f(x) = { 1 - Ixl if Ixl ~ 1, o otherwise.

i

1.

29. We have that 11 - ¢(t)1

.:s lEI 1 -

e itX I = lEV(1 - e itX )(1 - e- itX )

= lEV2{1 - cos(tX)}

since 2(1 - cos x)

.:s lEltXI

.:s x 2 for all x.

30. This is a consequence of Taylor's theorem for functions of two variables:

264

Solutions [5.12.31]-[5.12.33]

Problems

where
whence the claim follows.

31. (a) We have that x2

-

x2

x4

< -

- -

3 - 2!

< 1 - cos x

4!-

if Ixl ::::: I, and hence

1

(tx)2 dF (x):::::

[-t-',t-']

1

i:

3{I-cos(tx»)dF(x)

[-t-',t-']

::::: 3

{I - cos(tx») dF(x)

= 3{1

- Re
(b) Using Fubini's theorem,

~

t {1 t Jo

1x=-oo ~ itv=o 1 (100

Re
=

{I - cos(vx)} dv dF(x)

t

00

=

Sin(tX») dF(x)

-00

::: r

x:

Jtxl?:.1

(1-

1

tx

Sin(tX») dF(x) tx

since 1 - (tx)-1 sin(tx) ::: 0 if Itxl < 1. Also, sin(tx) ::::: (tx) sin 1 for Itxl ::: I, whence the last integral is at least

r

x:

Jtxl?:.1

(1- sin l)dF(x) :::

11P,(IXI ::: t- 1).

1

32. It is easily seen that, if y > 0 and n is large,

33. (a) The characteristic function of Y). is 1/J).(t)

= lE{ exp(it(X - A)/..fi)} = exp{A(eit/.)I - I) -

as A -+ 00. Now use the continuity theorem. (b) In this case, 1/J).(t) so that, as A -+

= e-it.)I

it..fi}

= exp{ _~t2 + 0(1)}

(1- ~)-). ..fi '

00,

2

1 2. 10g1/J).(t)=-it..fi-Alog ( 1 -it-) =-it..fi+A ( -it- - +tO ( A -1) ) -+--t

..fi

..fi

265

2A

2

[5.12.34]-[5.12.36]

Solutions

Generating functions and their applications

(c) Let Zn be Poisson with parameter n. By part (a), IP'

(ZJn n ::: 0)

<1>(0) =

-?

i

where is the N(O, I) distribution function. The left hand side equals IP'(Zn ::: n) = '5:."=0 e-nn k /kL

34. If you are in possession of r - 1 different types, the waiting time for the acquisition of the next new type is geometric with probability generating function (n - r

G r (s) =

Therefore the characteristic function of Un 1/fn(t)

+ I)s

n - (r - I)s

= (Tn

.

- n log n) / n is

= e-itlogn II Gr(e it / n ) = n- it II

n {( I) it /n } n - r + e. n - (r - I)e lt / n r=1

n

r=1

=

-it ,

rr r=O n- I

n ~. . (ne- lt / n - r)

The denominator satisfies n-I

n-I

II (ne- it / n -

r)

=

(1

+ 0(1)) II (n -

r=O

as n

-? 00,

it - r)

r=O

by expanding the exponential function, and hence n-itn! 1 = r(1 - it), n-+oo rr~;:;;o(n - it - r)

lim 1/fn(t) = lim n-+oo

where we have used Euler's product for the gamma function: n

n!n Z

rrr=O(z

+ r)

-?

r(z)

asn

-? 00

the convergence being uniform on allY region of the complex plane containing no singularity of The claim now follows by the result of Problem (5.12.27c).

r.

35. Let Xn be uniform on [-n, n], with characteristic function
It follows that, as n

t

-? 00,


1

n I. { sin(nt) _e ltX dx = nt -n 2n 1

-?

ift

#

0,

if t = O.

DOt, the Kronecker delta. The limit function is discontinuous at

= 0 and is therefore not itself a characteristic function.

36. Let G i (s) be the probability generating function of the number shown by the i th die, and suppose that 12 2(1 11) ""' 1 k s -s GI (S)G2(S) = L..J ITs = 11(1 _ ) , k=2 s

so that 1 - sl1 = 11(1 - s)HI (S)H2 (s) where Hi (s) = s-I Gi(S) is a real polynomial of degree 5. However 5

1 - sl1 = (1 - s)

II k=1

266

(Wk -

s)«(i'ik - s)

Solutions

Problems

[5.12.37]-[5.12.38]

where wI, wI, ... , w5, W5 'are the ten complex eleventh roots of unity. The Wk come in conjugate pairs, and therefore no five of the ten terms in ITk=I (Wk - S)(Wk - s) have a product which is a real polynomial. This is a contradiction. 37. (a) Let H and T be the numbers of heads and tails. The joint probability generating function of HandTis

where p

=I

- q is the probability of heads on each throw. Hence G H,T(S, t) = GN(qt

+ ps) =

exp {}.(qt

+ ps

- I)).

It follows that

so that G H,T(S, t) = G H(S)GT(t), whence Hand T are independent. (b) Suppose conversely that Hand T are independent, and write G for the probability generating function of N. From the above calculation, G H,T(S, t) = G(qt + ps), whence G H(S) = G(q + ps) and GT(t) = G(qt + p), so that G(qt + ps) = G(q + ps)G(qt + p) for all appropriate s, t. Write f(x) = G(l - x) to obtain f(x + y) = f(x)f(y), valid at least for all 0 ::::; x, y ::::; min{p, q). The only continuous solutions to this functional equation which satisfy f(O) = I are of the form f(x) = e lLx for some JL, whence it is immediate that G(x) = e),,(x-I) where}. = -JL. 38. The number of such paths T( containing exactly n nodes is 2n- I , and each such T( satisfies lP'(B(T() ::: k) = lP'(Sn ::: k) where Sn = YI + Y2 + ... + Yn is the sum of n independent Bernoulli variables having parameter p (= 1- q). Therefore E{Xn(k)) = 2n- I lP'(Sn ::: k). We set k = nf3, and need to estimate lP'(Sn ::: nf3). It is a consequence of the large deviation theorem (5.11.4) that, if p ::::; f3 < I, lP'(Sn ::: nf3)1/n -+ inf M(t)} t>O

where M(t)

{e- tf3

= E(e tY\) = (q + pet). With the aid of a little calculus, we find that lP'(Sn ::: nf3)1/n -+

(~ )

f3 (II =~ )I-f3

Hence E{Xn(f3n)) -+

{O

~fY(f3) <

00

where y(f3)

= 2 (~r

If

p::::;f3<1.

I,

y(f3) > I,

C=~r-f3

is a decreasing function of f3. If p < ~, there is a unique f3c E [p, 1) such that y (f3c) then y (f3) > I for all f3 E [p, I) so that we may take f3c = 1. Turning to the final part, if f3 > f3c. As for the other case, we shall make use of the inequality E(N)2 lP'(N#O) > - - E(N2)

267

= I; if p

:::

i

[5.12.38]-[5.12.38]

Solutions

Generating functions and their applications

for any N taking values in the non-negative integers. This is easily proved: certainly

IN

var(N

=1= 0)

whence

= E(N 2 I N

=1= 0) -

E(N 2 )

E(N

IN

=1= 0)2 :::: 0,

E(N)2

---> . W'(N =1= 0) - W'(N =1= 0)2

We have that E{XnCBn)2} = I:]!",p EU]!"Ip) where the sum is over all such paths n, p, and I]!" is the indicator function of the event {B(n) :::: ,Bn}. Hence E{X n (,Bn)2} = LEU]!") ]!"

+L

EU]!"Ip) = E{Xn(,Bn)}

+2n- 1 L E(h I p )

]!"#p

p#L

where L is the path which always takes the left fork (there are 2n - 1 choices for n, and by symmetry each provides the same contribution to the sum). We divide up the last sum according to the number of nodes in common to p and L, obtaining 2 n- m- 1E(hIM) where M is a path having exactly m nodes in common with L. Now

I::!t-==\

where Tn - m has the bin(n - m, p) distribution (the 'most value' to 1M of the event {h = I} is obtained when all m nodes in L n M are black). However

whence, by (*), W'(N =1= 0) > - E(N)-l If ,B < ,Be then E(N) -+

Suppose finally that p >

I

+ 1I::!t-==\ (2p)-m

.

as n -+ 00. It is immediately evident that W'(N =1= 0) -+ I if p ::: and,B < ,Be. By the above inequality,

00

1

W'(Xn(,Bn) >

0) :::: c(,B)

i.

for all n

where c(,B) is some positive constant. Find E > 0 such that ,B + E < ,Be. Fix a positive integer m, and let /Pm be a collection of 2m disjoint paths each of length n - m starting from depth m in the tree. Now W'(Xn(,Bn) = 0) ::: W'(B(v) <,Bn for all v E /Pm) = W'(B(v) < ,Bn)2m where v

E

/Pm. However W'(B(v) < ,Bn) ::: W'(B(v) < (,B

if ,Bn < (,B

+ E)(n -

m), which is to say that n :::: (,B

268

+ E)(n - m»)

+ E)m/E.

Hence, for all large n,

Solutions

Problems

by (**); we let n -+

and m -+

00

00

[5.12.39]-[5.12.42]

in that order, to obtain Y!(Xn(f3n) = 0) -+ 0 as n -+

00.

39. (a) The characteristic function of Xn satisfies

.x

E(e lt

n)

(

=

)..

1- ~

)...)n = + ~elf

( 1 + ~[elt )...)n lt. - 1] -+ exp ()..[e -

1]),

the characteristic function of the Poisson distribution. (b) Similarly, eit In ).. E(eitYnln) = p . -+ ~~ lfln 1 - (1 - p)e ).. - it as n -+

00,

the limit being the characteristic function of the exponential distribution.

40. If you cannot follow the hints, take a look at one or more of the following: Moran 1968 (p. 389), Breiman 1968 (p. 186), Loeve 1977 (p. 287), Laha and Rohatgi 1979 (p. 288). 41. With Yk = kXk, we have that E(Yk) = 0, var(Yk) = k 2 , Yl + Y2 + ... + Yn is such that

= k 3 . Note that Sn =

Elyll

as n -+ 00, where c is a positive constant. Applying the central limit theorem «5.10.5) or Problem (5.12.40)), we find that Sn D ~ ----* N(O, 1), as n -+ 00, vvarSn where var Sn = 2.:k=1 k 2

'V

~n3 as n -+

00.

42. We may suppose that JL = 0 and a = 1; if this is not so, then replace Xi by Yi = (Xi - JL) / a. Let t = (to, tl, t2, ... , tn ) E IRn+1 , and set 7 = n -1 2.:J=1 tj. The joint characteristic function of the n + 1 variables X, ZI, Z2, ... , Zn is ¢(t) =

=

E{

fJ

exp(itox

exp

+

titjZj) } = J=1

E{ IT (i [~+ -7] exp

tj

Xj) }

J=1

(-~ [~+tj -7f)

by independence. Hence ¢(t) = exp ( - -In L [t-.2. 2 j=1 n

+ (tj

-7) ]

2)

n = exp {t2 -~ - -1 L(tj - 7)2 } 2n 2 j=1

where we have used the fact that 2.:J=1 (tj - 7) = O. Therefore

whence X is independent of the collection Z 1, Z2, ... , Zn. It follows that S2 = (n - 1)-1 2.:J=1 ZJ. Compare with Exercise (4.10.5).

269

X is

independent of

[5.12.43]-[5.12.47]

Solutions

Generating functions and their applications

43. (i) Clearly, IP'(Y :::: y) = IP'(X :::: log y) =

0, where

10o

°

if lal :::: I, and

I 1 I 2 asin(2nlogx)--e- 2 (ogx) dx=

x.../2ir

1

00

-00

I 1 2 --asin(2ny)e- 2Y dy=O

.../2ir

since sine is an odd function. Therefore J~oo fa (x) dx = I, so that each such fa is a density function. For any positive integer k, the kth moment of fa is J~oo xk f(x) dx laCk)

=

1

I

00

-00

k

1 2

~a sin(2ny)e Y-2 Y dy

v2n

=

+ laCk) where

°

since the integrand is an odd function of y - k. It follows that each fa has the same moments as f.

44. Here is one way of proving this. Let X I, X 2, ... be the steps of the walk, and let Sn be the position of the walk after the nth step. Suppose f-t = JE(X I) satisfies f-t < 0, and let em = IP'(Sn = for some n ::: I I So = -m) where m > 0. Then em :::: L:~IIP'(Tn > m) where Tn = XI + X2 +

°... +

Xn

=

Sn - SO. Now, for t > 0,

IP'(Tn > m)

= IP'(Tn

- nf-t > m - nf-t) :::: e-t(m-nJL)JE(et(Tn-nJL)

= e- tm {e tJL M(t)

r

where M (t) = JE(et(XI-JL). Now M(t) = I +0(t 2 ) as t ~ 0, and therefore there exists t (> 0) such that get) = e tJL M (t) < I (remember that f-t < 0). With this choice of t, em :::: L:~I e-tmg(t)n ~ as m ~ 00, whence there exists K such that em < ~ for m ::: K. Finally, there exist 8, E > such that IP'(X 1 < -8) > E, implying that IP'(SN < -K I So = 0) > EN where N = fK/81, and therefore

°

IP'(Sn

° =f:. °

for all n ::: I

I So = 0) :::

(1 - e K )E

N

::: ~EN;

therefore the walk is transient. This proof may be shortened by using the Borel-Cantelli lemma.

45. Obviously, if XI >a, L={;I+L

where

L has the same distribution as L.

if XI ::::a,

Therefore, a

JE(sL)

= salP'(XI

> a)

+ LsrJE(sL)IP'(XI = r). r=1

46. We have that with probability p, with probability q, where Wn is independent of Wn-I and has the same distribution as Wn . Hence G n (s) = psGn_1 (s)+ qsGn-1 (s)Gn(s). Now Go(s) = I, and the recurrence relation may be solved by induction. (Alternatively use Problem (5.12.45) with appropriate Xi.)

47. Let Wr be the number of flips until you first see r consecutive heads, so that IP'(Ln < r) = IP'(Wr > n). Hence,

270

Solutions

Problems where lE(s Wr)

[5.12.48]-[5.12.52]

= Gr(s) is given in Problem (5.12.46).

48. We have that Xn+1

!, with probability !.

iXn

with probability

={1

zXn + Yn

Hence the characteristic functions satisfy .

¢n+1 (t)

= lE(e

itX

n+!)

1

=

1

z¢n(zt)

1 1 A + 'l¢n(zt) A _ it

iit = ¢n-I(4t) 1 A - !it n A - it2- n A A _ it = ... = ¢1(t2-) A - it --+ A - it

1 AA _ it

= ¢n(zt) as n --+

00.

The limiting distribution is exponential with parameter A.

49. We have that

(a)(1-e- A)/A, (b) _(p/q2)(q+log p), (c)(1_qn+1 )/[(n+l)p], (d) -[1+(p/q) log p]/ log p. (e) Not if lP'(X + 1 > 0) = 1, by Jensen's inequality (see Exercise (5.6.1)) and the strict concavity of the function f(x) = l/x. If X + 1 is permitted to be negative, consider the case when lP'(X + 1 = -1) = lP'(X + 1 = 1) =!. 50. By compounding, as in Theorem (5.1.25), the sum has characteristic function GN(¢X(t))

=

p¢x(t)

Ap

1 - q'f'X A. ( ) t

= -'-------:-t' ",P-I

whence the sum is exponentially distributed with parameter Ap. 51. Consider the function G(x) = (lE(X 2)}-1 J~oo y2 dF(y). This function is right-continuous and increases from 0 to 1, and is therefore a distribution function. Its characteristic function is

52. By integration, fx(x) are not independent. Now,

fx+Y(z)

= fy(y) = !, Ixl =

1 1

-I

< 1,

f(x, z - x)dx

Iyl

=

< 1. Since f(x, y) =1= fx(x)fy(y), X and Y

{!(Z+2)

if - 2 <

1

4(2 - z)

if 0 <

z < 0,

z < 2,

the 'triangular' density function on (-2,2). This is the density function of the sum oftwo independent random variables uniform on (-1,1).

271

6 Markov chains

6.1 Solutions. Markov Processes 1.

The sequence Xl, X2, ... of independent random variables satisfies

whence the sequence is a Markov chain. The chain is homogeneous if the Xi are identically distributed. 2.

(a) With Yn the outcome ofthe nth throw, Xn+l

Pij

=

{

= max{Xn, Yn+ll, so that

o

ifj
1· 61

If}

"

t

= I.

if j > i,

for I :::: i, j :::: 6. Similarly, if j < i

= i.

if j

If j > i, then Pij(n) = IP'(Zn = j), where Zn = max{Yl, Y2, ... , Yn }, and an elementary calculation yields Pij(n)

= (~.)n _ (.J~ l)n ,

i < j :::: 6.

(b) Nn+l - N n is independent of Nl, N2,"" N n , so that N is Markovian with

~

Pij = {

if j = i

+ 1,

if j = i,

o

otherwise.

(c) The evolution of C is given by Cr+l

={0

Cr

if the die shows 6,

+1

whence C is Markovian with

1

t 6

Pij =

{

otherwise, j =0, j = i + 1, otherwise.

272

Solutions

Markov processes

[6.1.3]-[6.1.4]

(d) This time, if Br > 0,

Br-l Br+l = { Yr

if Br

= 0,

where Yr is a geometrically distributed random variable with parameter ~, independent of the sequence Bo, B2, ... , B r . Hence B is Markovian with

Pij

(i) If Xn = i, then Xn+l

3.

(*) lP'(Xn+l = i

+ 1 I Xn

E

=

if j

I { (t)j-l ~

{i - 1, i

+ I}.

ifi

= i - I :::: 0, = 0, j :::: 1.

Now, for i :::: 1,

+ 1 I Sn = i, B)lP'(Sn = i I Xn = i + 1 I Sn = -i, B)lP'(Sn =

= i, B) = lP'(X n+l = i

+ lP'(X n+l

= i, B) -i

I Xn =

i, B)

where B = {X r = ir forO::: r < n} and io, il,"" i n -l are integers. Clearly lP'(Xn +l = i

+ 1 I Sn =

i, B) = p,

lP'(Xn +l = i

+ 1 I Sn

= -i, B) = q,

where P (= 1 - q) is the chance of a rightward step. Let I be the time of the last visit to 0 prior to the time n, I = max{r : ir = O}. During the time-interval (1, n], the path lies entirely in either the positive integers or the negative integers. If the former, it is required to follow the route prescribed by the event B n {Sn = i}, and if the latter by the event B n {Sn = -i}. The absolute probabilities of these two routes are Jr2 = P

1 -2

(n-I-i)

q

i (n-I+i) ,

whence lP'(Sn = i

I Xn =

i, B) = _Jr_l_ = Jrl

+ Jr2

L+ pi

ql

= 1 - lP'(Sn = -i

I Xn =

i, B).

Substitute into (*) to obtain .

lP'(Xn+l = I

+ 1 I Xn

.

pi+l

= I, B) =

.

+ qi+l

= 1 -lP'(Xn+l = i - I

.

pi +ql

I Xn =

i, B).

Finally lP'(X n+l = 1 I Xn = 0, B) = 1. (ii) If Yn > 0, then Yn - Yn+l equals the (n + l)th step, a random variable which is independent of the past history of the process. If Yn = 0 then Sn = Mn, so that Yn+l takes the values 0 and 1 with respective probabilities p and q, independently of the past history. Therefore Y is a Markov chain with transition probabilities for i > 0,

Pij

=

p { q

ifj=i-l

'f"J =

1

I

+ 1,

POj

=

P { q

ifj=O ifj=1.

The sequence Y is a random walk with a retaining barrier at O.

4.

For any sequence iO, ii, ... of states, lP'(Yk+l

= ik+l I Yr = ir for 0::: r ::: k) =

lP'(Xns

= is for 0 < S < k + 1) . -= Is for 0 ::: s ::: k)

lP'(Xns

_ rr~=o Pis ,is+l (ns+l - rr;:6 Pis ,is+l (ns+l -

ns) ns)

= Pibik+l (nk+l - nk) = lP'(Yk+l = ik+l

273

I Yk =

ik),

[6.1.5]-[6.1.9]

Solutions

Marlwv chains

where Pij (n) denotes the appropriate n-step transition probability of X. (a) With the usual notation, the transition matrix of Y is

lrij

=

{

= i + 2,

p2

if j

2pq

if j

= i,

q2

if j

=i -

2.

(b) With the usual notation, the transition probability lrij is the coefficient of sj in G(G(s))i. 5.

Writing X = (X I, X2, ... , Xn), we have that

.)

( I

= 1, Xn = I =

lP' F I(X)

lP'(F,/(X)=l,Xn=i) ( .) lP' I(X) = 1, Xn = I

where F is any event defined in terms of X n , Xn+l, .... Let A be the set of all sequences x (Xl, x2, ... , xn-l, i) of states such that I (x) = 1. Then

XEA

XEA

I

by the Markov property. Divide through by the final summation to obtain lP'(F I(X) i) = lP'(F I Xn = i).

6.

Let Hn

= (Xk = Xk for 0:::: k

< n, Xn

lP'(XT+m = j, HT) lP'(HT)

=

=

1, Xn =

= i). The required probability may be written as 2:n lP'(XT+m = j, HT, T lP'(HT)

= n)

Now lP'(X T +m = j I HT, T = n) = lP'(Xn+m = j I Hn, T = n). Let I be the indicator function of the event Hn n (T = n), an event which depends only upon the values of X I, X2, ... , X n . Using the result of Exercise (6.1.5), lP'(Xn+m

= j I Hn, T = n) = lP'(Xn+m = j I Xn = i) = Pij(m).

Hence lP'(X

7.

-' T +m - }

'" lP'(R T I HT ) -- PI].. (m) L..Jn n, lP'(HT)

n)

_

.. ( )

- P,] m .

Clearly lP'(Yn+1

= j I Yr = ir for 0:::: r

:::: n)

= lP'(Xn+1 = b I Xr = ar for 0:::: r

:::: n)

where b = h-l(j), a r = h-l(ir); the claim follows by the Markov property of X. It is easy to find an example in which h is not one-one, for which X is a Markov chain but Y is not. The first part of Exercise (6.1.3) describes such a case if So #- o. 8. Not necessarily! Take as example the chains Sand Y of Exercise (6.1.3). The sum is Sn Mn, which is not a Markov chain. 9.

+ Yn =

All of them. (a) Using the Markov property of X, lP'(Xm+r

= k I Xm = im , ... , Xm+r-l = im+r-l) = lP'(Xm+r = k I Xm+r-l = im+r-l)· 274

Solutions

Classification of states

(b) Let {even} Then,

=

{X2r

=

i2r for 0 :::: r :::: m} and {odd}

",IP'(X2m+2 IP'(X2m+2 = k I even) = 0

= k,

X2m+1

=

{X2r+1

= i2m+1,

=

[6.1.10]-[6.2.1]

i2r+1 for 0 :::: r :::: m - I}.

even, odd)

lP'(even)

= L,IP'(X2m+2 =

k, X2m+1 = i2m+1

I X2m

= i2m)lP'(even, odd)

lP'(even) = IP'(X2m+2 = k

I X2m

= i2m)'

where the sum is taken over all possible values of is for odd s. (c) With Yn = (X n , X n+1),

I

I

IP'(Yn+1 = (k, I) YO = (io, i1),···, Yn = (in, k)) = IP'(Yn+1 = (k, I) Xn+1

= IP'(Yn+1 = (k, I) I Yn =

= k) (in, k)),

by the Markov property of X.

10. We have by Lemma (6.1.8) that, with fly) = IP'(Xi

11. (a) Since Sn+1 (b) We have that

=

j),

= Sn + Xn+1, a sum of independent random variables, S is a Markov chain.

IP'(Yn+1 = k

I Yi

= Xi +Xi-1 for

1:::: i

:::: n) = IP'(Yn+1 = k

I Xn

= xn)

by the Markov property of X. However, conditioning on X n is not generally equivalent to conditioning on Yn = Xn + X n -1, so Y does not generally constitute a Markov chain. (c) Zn = nX 1 + (n - I)X2 + ... + X n , so Zn+1 is the sum of Xn+1 and a certain linear combination of Zl, Z2, ... ,Zn, and so cannot be Markovian. (d) Since Sn+1 = Sn + Xn+1' Zn+1 this is a Markov chain.

=

Zn

+ Sn + Xn+1' and Xn+1

is independent of Xl, ... ,Xn ,

12. With 1 a row vector of 1's, a matrix P is stochastic (respectively, doubly stochastic, sub-stochastic) if PI' = 1 (respectively, IP = 1, PI' :::: 1, with inequalities interpreted coordinatewise). By recursion, P satisfies any of these equations if and only if pn satisfies the same equation.

6.2 Solutions. Classification of states 1. Let Ak be the event that the last visit to i, prior to n, took place at time k. Suppose that Xo = i, so that Ao, A 1, ... , An-1 form a partition of the sample space. It follows, by conditioning on the Ai, that n-1

Pij (n) = L

Pii (k)lij (n - k)

k=O for i =1= j. Multiply by sn and sum over n (:=: 1) to obtain Pij (s) = Pii (s)Lij (s) for i =1= j. Now Pij (s) = Fij (s)Pjj (s) iii =1= j, so that Fij (s) = Lij (s) whenever Pii (s) = Pjj (s). As examples of chains for which Pii (s) does not depend on i, consider a simple random walk on the integers, or a symmetric random walk on a complete graph.

275

[6.2.2]-[6.3.1]

Solutions

Markov chains

2. Let i (# s) be a state of the chain, and define nj = min{n : pjs(n) > O}. If Xo = i and X nj = s then, with probability one, X makes no visit to i during the intervening period [1, nj -1]; this follows from the minimality of nj. Now s is absorbing, and hence lP'(no return to i

I Xo = i) 2:

lP'(X nj

3. Let h be the indicator function of the event {Xk visits to i. Then E(N)

= s I Xo = i)

= i},

00

00

k=O

k=O

= L E(h) = L

so that N

> O.

= I:~o h

is the number of

Pii (k)

which diverges if and only if i is persistent. There is another argument which we shall encounter in some detail when solving Problem (6.15.5).

4. We write lP'j(.) = lP'(. I Xo (6.15.5). Note that lP'j(Vj 2: 1) (a) We have that

= i).

One way is as follows, another is via the calculation of Problem < (0).

= lP'j(Tj

by the strong Markov property (Exercise (6.1.6» applied at the stopping time Tj. By iteration, lP'j (Vj 2: n) = lP' j (Vj 2: l)n, and allowing n -+ 00 gives the result. (b) Suppose i

#

j. For m 2: 1,

by the strong Markov property. Now let m --+

5. Let e = lP'(Tj < Tj before visiting i. Then

I Xo = i) = lP'(Tj lP'(N 2: 1 I Xo

Likewise, lP'(N 2: k

00,

< Tj

and use the result of (a).

I Xo =

= i) = lP'(Tj

I Xo = i) = e(1 - ei- 1 for k

< Tj

j), and let N be the number of visits to j

I Xo = i) = e.

2: 1, whence

00

E(N

I Xo = i) = Le(1- e)k-l =

1.

k=l

6.3 Solutions. Classification of chains If r = 1, then state i is absorbing for i 2: 1; also, 0 is transient unless ao = 1. Assume r < 1 and let J = sup{j : aj > O}. The states 0, 1, ... , J form an irreducible persistent class; they are aperiodic if r > O. All other states are transient. For 0 .:'S i .:'S J, the recurrence time Tj of i satisfies lP'(Tj = 1) = r. If Tj > 1 then Tj may be expressed as the sum of

1.

TP) := time to reach 0, given that the first step is leftwards, T?) := time spent in excursions from 0 not reaching i, r I (3 )

:= time taken to reach i in final excursion.

276

Solutions

Classification of chains

[6.3.2]-[6.3.3]

It is easy to see that E(T?)) = 1 + (i - 1)/(1 - r) if i :::: 1, since the waiting time at each intennediate point has mean (1 - r ) -1. The number N of such 'small' excursions has mass function IP'(N = n) = (1i(1- (1i)n, n :::: 0, where (1i = L;~i aj; hence E(N) = (1- (1i)l(1i. Each such small excursion has mean duration

.r; (1 i-1

.)

+1

_J_ r

~ _ 1+ (1i -

1-

.r; i-1

.

Jaj (1 - (1i)(1 - r)

and therefore

By a similar argument, 00

E(Ti (3) ) = -1 (1' I

2:

(

j=i

j -i ) aj. 1+ 1- r

Combining this infonnation, we obtain that

and a similar argument yields E(To) = 1 + L;j j aj 1(1- r). The apparent simplicity of these formulae suggests the possibility of an easier derivation; see Exercise (6.4.2). Clearly E(Ti) < 00 for i :s J whenever L;j jaj < 00, a condition which certainly holds if J < 00. 2. Assume that 0 < P < 1. The mean jump-size is 3 P - 1, whence the chain is persistent if and only if P = see Theorem (5.10.17).

1;

3.

(a) All states are absorbing if P P = BAB- 1 where

B~

(l

= O.

1 0

-1 A=

~I) ,

0

Therefore

p"

Assume henceforth that P

~ BA"rl ~B

B- 1

0

2

o

(1 - 2p)n

o

o

(1- 4p)n

In particular,

= Pl1 (n)

_II) ,

1

by symmetry.

277

1

4

1-4p

0

G

(l

o ) .

1-2p

whence Pij (n) is easily found.

and P33(n)

=

1

2: 0

=I- O. Diagonalize P to obtain

)

B- 1

[6.3.4]-[6.3.5]

Solutions

Markov chains

Now Fjj(s) = 1 - Pjj(s)-I, where

Pll(S)

=

P33(S)

=

P22(S) =

1 4(1-s) 1

1

1

+ 2{I-s(l-2p)} + 4{I-s(l-4p)}' 1

2(l-s)

+ 2{I-s(l-4p)} .

After a little work one obtains the mean recurrence times JIi = F[i (1): JII = JI3 = 4, JI2 = 2. (b) The chain has period 2 (if p =I 0), and all states are non-null and persistent. By symmetry, the mean recurrence times JIi are equal. One way of calculating their common value (we shall encounter an easier way in Section 6.4) is to observe that the sequence of visits to any given state j is a renewal process (see Example (5.2.15)). Suppose for simplicity that p =I O. The times between successive visits to j must be even, and therefore we work on a new time-scale in which one new unit equals two old units. Using the renewal theorem (5.2.24), we obtain

2

Pij (2n) -'>- if JIj .

Jj - i I is even,

Pij (2n

+ 1) -'>-

2 -

if Jj - i I is odd;

JIj

note that the mean recurrence time of j in the new time-scale is iJIj. Now L,j Pij (m) = 1 for all m, and so, letting m = 2n -'>- 00, we find that 4/ JI = 1 where JI is a typical mean recurrence time. There is insufficient space here to calculate Pij (n). One way is to diagonalize the transition matrix. Another is to write down a family of difference equations of the form PI2 (n) = P . P22 (n 1) + (1 - p) . P42(n - 1), and solve them.

4. (a) By symmetry, all states have the same mean-recurrence time. Using the renewal-process argument of the last solution, the common value equals 8, being the number of vertices of the cube. Hence JIv = 8. Alternatively, let s be a neighbour of v, and let t be a neighbour of s other than v. In the obvious notation, by symmetry, JIv JItv

= 1 + ~JIsv, I I I = 1 + zJIsv + 4JItv + 4JIwv,

JIsv

I I = 1 + 4JIsv + zJItv,

JIwv

I 3 = 1 + 4JIwv + 4JItv,

a system of equations which may be solved to obtain JIv (b) Using the above equations, JIwv

= 8.

= :?, whence JIvw = jO by symmetry.

(c) The required number X satisfieslP'(X = n) = en-I(l- e)2 forn 2: 1, wheree is the probability that the first return of the walk to its starting point precedes its first visit to the diametrically opposed vertex. Therefore 00

JE(X)

=

L nen-I(l -

e)2

=

1.

n=1

5.

(a) Let lP'i(·) = lP'(.

I Xo

= i). Since i is persistent,

1 = lP'i(Vi = 00) = lP'i(Vj = 0, Vi = 00)

+ lP'i(Vj

> 0, Vi = 00)

:s lP'i(Vj = 0) + lP'i(Tj < 00, Vi = 00) :s 1 -lP'i(Tj < 00) + lP'i(Tj < oo)lP'j(Vi = 00), by the strong Markov property. Since i -'>- j, we have that lP'j (Vi = 00) 2: 1, which implies 'fJji Also, lP'i(Tj < 00) = 1, and hence j -'>- i and j is persistent. This implies 'fJij = 1. (b) This is an immediate consequence of Exercise (6.2.4b).

278

= 1.

Solutions

Classification of chains

6. Let lP'i (-) = IP'(. I Xo = i). It is trivial that TJj step and use the Markov property to obtain TJj

If x = (Xj : j j if A, Xj

E

=L

=

L PjklP'(TA < 00 kES

=

1 for j

I Xl = k) =

L

PjkTJk· k

S) is any non-negative solution of these equations, then Xj = 1 :::: TJj for j

PjkXk

=

kES

= IP'j(TA =

1)

L Pjk kEA

+L

+L

PjkXk

pjk{ L

We obtain by iteration that, for j

= IP'j(TA =

1)

Pki

+L

A. For

PjkXk

PkiXi}

= IP'j(TA

:::: 2)

i~A

+L k~A

Pjk L

PkiXi·

i~A

rt A,

rt A. We let n

-+ 00 to find that Xj :::: IP'j (TA < 00)

7. The first part follows as in Exercise (6.3.6). Suppose x to the equations. As above, for j if A,

= 1 + LPjkXk = IP'j(TA

:::: 1)

+L

p j k(1

k~A

k :::: 1)

E

k~A

iEA

where the sum is over all kl, k2, ... , k n

= IP'j (TA

+L

k~A

k~A

Xj

if A, condition on the first

A. For j

E

[6.3.6]-[6.3.9]

+ IP'j (TA

:::: 2)

+ ... + IP'j (TA

= (Xj

= TJj.

: j E S) is a non-negative solution

+ LPkiXi) i~A

:::: n)

+L

PjkJ PkJk2 ... Pkk_Jkn Xkn

n

:::: L

IP'(TA:::: m),

m=1 where the penultimate sum is over all paths of length n that do not visit A. We let n -+ that Xj :::: lEj(TA) = Pj.

00

to obtain

8. Yes, because the Sr and Tr are stopping times whenever they are finite. Whether or not the exit times are stopping times depends on their exact definition. The times Ur = min{k > Ur -l : XUr E A, XUr +l rt A} are not stopping times, but the times U r + 1 are stopping times. 9. (a) Using the aperiodicity of j, there exist integers rl, r2, ... , rs having highest common factor 1 and such that Pj j (rk) > 0 for 1 :::: k :::: s. There exists a positive integer M such that, if r :::: M, then r = Lk=1 akrk for some sequence ai, a2, ... , as of non-negative integers. Now, by the ChapmanKolmogorov equations, s

Pjj (r) ::::

II Pjj (rk)ak > 0, k=1

sothatPjj(r) > ofor all r :::: M. Finally, find m such that Pij (m) > O. Then

ifr:::: M. (b) Since there are only finitely many pairs i, j, the maximum R(P) = max{N(i, j) : i, j E S} is finite. Now R(P) depends only on the positions of the non-negative entries in the transition matrix P.

279

[6.3.10]-[6.3.10]

Markov chains

Solutions

There are only finitely many subsets of entries of P, and so there exists f(n) such that R(P) :::: f(n) for all relevant n x n transition matrices P. (c) Consider the two chains with diagrams in the figure beneath. In the case on the left, we have that PII (5) = 0, and in the case on the right, we may apply the postage stamp lemma with a = n and b=n-1. 3

4 2

2

4

n 10. Let Xn be the number of green balls after n steps. Let ej be the probability that Xn is ever zero when Xo = j. By conditioning on the first removal, ej

with eo

=

=

j

j+2

2(j

+ 1) eJ+I + 2(j + 1) ej_l,

j:::: 1,

1. Solving recursively gives

(*)

ej

= 1-

(l - eJ}

ql

{

1+ -

PI

+ ... +

Qlq2 ... qj-l} PIP2'" Pj-I

,

where j+2 Pj

= 2(j + 1)'

It is easy to see that j-I

'"

QIQ2" 'Qj-I

L PIP2'" PJ'-I r=O

=2-

2

- - --+ 2 j +1

as J' --+

00.

By the result of Exercise (6.3.6), we seek the minimal non-negative solution (ej) to (*), which is attained when 2(1 - q) = 1, that is, el = ~. Hence

1 j-I QIQ2'" Q'_I

ej

= 1 - 2 r=O L PIP2'"

J

Pj-I

J'

+1

For the second part, let dj be the expected time until j - 1 green balls remain, having started with j green balls and j + 2 red. We condition as above to obtain j

dj

= 1 + 2(j + 1) {dJ+I + dj}.

We set ej = dj - (2j + 1) to find that (j time to remove all the green balls is n

+ 2)ej = jeJ+I, whence ej = !j (j + 1)el. The expected

n

L dj = L {ej + 2(j j=1

n

1)}

= n(n + 2) + el

j=1

L !j (j + 1). j=1

The minimal non-negative solution is found by setting q (6.3.7).

280

= 0, and the conclusion follows by Exercise

Solutions [6.4.1]-[6.4.2]

Stationary distributions and the limit theorem

6.4 Solutions. Stationary distributions and the limit theorem 1. Let Yn be the number of new errors introduced at the nth stage, and let G be the common probability generating function of the Yn . Now Xn+l = Sn + Yn+l where Sn has the binomial distribution with parameters Xn and q (= I - p). Thus the probability generating function G n of Xn satisfies Gn+l(S)

= G(s)E(sSn) = G(s)E{E(sSn I Xn)} = G(s)E{(p +qs)xnj = G(s)Gn(p + qs) = G(s)G n (1- q(1 - s)).

Therefore, for s < I, Gn(s)

= G(s)G(1

- q(1- S))G n-2(1- q2(1- s))

= ...

n-l

= GO (1 -

qn (1 - s))

II G (I -

00

q' (1 - s)) --+

,=0

II G (I -

q' (1 - s) )

,=0

as n --+ 00, assuming q < 1. This infinite product is therefore the probability generating function of the stationary distribution whenever this exists. If G(s) = eA(s-l), then

IT

G(I- q'(1- s))

= exp { A(S

r=O

-I) "f,qr} ,=0

= eMs-l)lp,

so that the stationary distribution is Poisson with parameter Aj p. 2. (6.3.1): Assume for simplicity that sup{j : aj > OJ = 00. The chain is irreducible if r < 1. Look for a stationary distribution n with probability generating function G. We have that

Hence where A(s)

=

sG(s) = nosA(s) + rs(G(s) -no) '£f=,oajs j , and therefore G(s)

Taking the limit as s

t

= no

(

+ (1

- r)(G(s) -nO)

SA(S) - (1- r + sr)) . (1 - r)(s - I)

I, we obtain by L'Hopital's rule that G(1)

= nO

+I 1- r

AI (1) (

r) .

There exists a stationary distribution if and only if r < I and A' (1) < G (s)

=

00,

in which case

00.

The mean recurrence time

sA(s)-(1-r+sr) -(s-----:-I)-(A-cI:-(1:-)-+-I:---r) .

Hence the chain is non-null persistent if and only if r < I and A' (1) < iti is found by expanding G and setting /-ti = I j rri . (6.3.2): Assume that 0 < p < I, and suppose first that p the equations

=1=

i

281

=1=

t. Look for a solution {Yj : j 0,

=1=

OJ of

[6.4.3]-[6.4.3]

Solutions

Markov chains

as in (6.4.10). Away from the origin, this equation is Yi auxiliary equation p8 3 - 8 + q = O. Now p8 3 - 8 + q 0: Note that 0 <

fJ

=

-p - Vp2 +4pq

<-I ,

2p

< I if p > }, while

fJ

=

+ PYi+2 where P + q = I, with 1)(8 - 0:)(8 - fJ) where

qYi-1

= p(8 -

= - p + Vp2 + 4pq

fJ

> O.

2p

> I if p < }.

For P > }, set

. = { A + BfJi C + Do: i

ifi:::: I, if i .:"0 -I,

Yl

the constants A, B, C, D being chosen in such a manner as to 'patch over' the omission of 0 in the equations (*): Y-2 = qY-3,

Y-I = qY-2

+ PYI,

YI = PY3·

The result is a bounded non-zero solution {Yj} to (*), and it follows that the chain is transient. For p < }, follow the same route with if i :::: I, ifi.:"O-I, the constants being chosen such that Y-2

= qY-3, Y-I = QY-2.

Finally suppose that p = }, so that 0: = -2 and fJ = 1. The general solution to (*) is Yi

={

A

i

+ Bi + Co: .

D+ Ei +Fo:l

ifi :::: I, ifi .:"0 -I,

subject to (**). Any bounded solution has B = E = C = 0, and (**) implies that A = D = F = O. Therefore the only bounded solution to (*) is the zero solution, whence the chain is persistent. The equation x = xP is satisfied by the vector x of I 's; by an appeal to (6.4.6), the walk is null. (6.3.3): (a) Solve the equation 7r

= 7rP to find a stationary distribution 7r =

(!, ~, !) when PI- O.

Hence the chain is non-null and persistent, with III = nIl = 4, and similarly 112 = 2, 113 = 4. · '1 arIy, 7r = (I4' 4' I 4' I 4I).IS a statIOnary . dIstn "b ' (b) SlITU utIOn, an d Ili = n i-I =4. (6.3.4): (a) The stationary distribution may be found to be ni

=

~ for all i, so that Ilv

3.

= 8.

The quantities X I, X 2, ... , Xn depend only on the initial contents of the reservoir and the rainfalls Yo, YI, ... , Yn-I. The contents on day n + I depend only on the value Xn of the previous contents and the rainfall Yn . Since Yn is independent of all earlier rainfalls, the process X is a Markov chain. Its state space is S = {O, 1,2, ... , K - I} and it has transition matrix

[80 HI 0

g2 gl gO

g3 g2 gl

gK-I gK-2 gK-3

GK-I GK GK-2

0

0

0

gO

GI

gO

lP'=

1

where gi = lP'(YI = i) and Gi = 2:.J:=i gj. The equation 7r = 7rP is as follows: no nr nK-I

= no(gO + gl) + nlgO, = nOgr+1 + nlgr + ... + nr+lgO, = nOGK +nIGK-I + ... +nK-IGI· 282

0 < r < K - I,

Solutions

Stationary distributions and the limit theorem

The final equation is a consequence of the previous ones, since ~~(/ lri v = (VI, V2, ... ) is an infinite vector satisfying

=

[6.4.4]-[6.4.5]

1. Suppose then that

Multiply through the equation for Vr by sr+l, and sum over r to find (after a little work) that 00

N(s)

satisfy sN(s)

=

= N(s)G(s) + vOgO(s

00

G(s)

2:>isi, i=O

=

Lgisi i=O

- I), and hence I -N(s) Vo

gO(s - I)

=

s - G(s)

.

The probability generating function of the lri is therefore a constant mUltiplied by the coefficients of sO, s I, ... , s K -I in go (s - I) / (s - G (s», the constant being chosen in such a way that ~~(/ lri = 1. When G(s)

= p(l

- qs)-I, then gO

= P and

go(s - I) s-G(s)

p(l - qs) p-qs

--::--:-:- =

=

p

q

+ -,---,-'----:---:I-(qs/p)

The coefficient of sO is I, and of si is qi +1/ pi if i ::: I. The stationary distribution is therefore given bYlri =qlrO(q/p)i fori::: I, where

if p

=f. q, and lrO = 2/(K + I) if p = q =

4.

The transition matrices

i.

P,

have respective stationary distributions lr I for any 0 ~ p ~ 1.

= (p,

~ (1

2:

I

0

I

2:

0

0

2:

0

2:

I - p) and lr2

I I

=

(~p, ~ p, ~ (l - p), ~(l - p»)

5. (a) Set i = I, and find an increasing sequence nl (I), nl (2), ... along which XI (n) converges. Now set i = 2, and find a subsequence of (nl(j) : j ::: I) along which x2(n) converges; denote this subsequence by n2(l), n2(2), .... Continue inductively to obtain, for each i, a sequence ni = (ni(j) : j ::: I), noting that: (i) ni+ I is a subsequence of ni, and (ii) limr--+oo Xi (ni (r» exists for all i. Finally, define mk = nk(k). For each i ::: I, the sequence mi, mi+I, ... is a subsequence of ni, and therefore limr--+oo Xi (mr) exists. (b) Let S be the state space of the irreducible Markov chain X. There are countably many pairs i, j of states, and part (a) may be applied to show that there exists a sequence (nr : r ::: I) and a family (Cii) : i, j E S), not all zero, such that Pi} (nr) ---+ Cii) as r ---+ 00.

283

[6.4.6]-[6.4.10]

Solutions

Markov chains

Now X is persistent, since otherwise Pij (n) ~ 0 for all i, j. The coupling argument in the proof of the ergodic theorem (6.4.17) is valid, so that Paj(n) - Pbj(n) ~ 0 as n ~ 00, implying that Ciaj = Cibj for all a, b, j.

6.

Just check that 1C satisfies 1C

= 1CP and ~v Jl'v =

1.

Let Xn be the Markov chain which takes the value r if the walk is at any of the 2 r nodes at level r. Then Xn executes a simple random walk with retaining barrier having P = 1 - q = ~, and it is thus transient by Example (6.4.15). 7.

8.

Assume that Xn includes particles present just after the entry of the fresh batch Yn . We may write Xn

= 2..: Bi,n + Yn

Xn+l

i=l

where the Bi,n are independent Bernoulli variables with parameter 1 - p. Therefore X is a Markov chain. It follows also that Gn+l (s)

= lE(sxn+!) =

Gn(p + qs)eA(s-l).

In equilibrium, Gn+l = G n = G, where G(s) = G(p + qs)eA(s-l). There is a unique stationary distribution, and it is easy to see that G (s) = eA(s-l)/ P must therefore be the solution. The answer is the Poisson distribution with parameter A/ p.

9. The Markov chain X has a uniform transition distribution Pjk Therefore, lE(Xn ) =lE(lE(Xn [Xn-l))

(~)n

= 1-

= ~(1

=

1/(j

+ 2),

0 :::: k :::: j

+ 1.

+lE(Xn-l)) = ...

+ (~)nXo.

The equilibrium probability generating function satisfies

whence

d

ds

{(1 -

subject to G(I) = 1. The solution is G(s) the Poisson distribution with parameter 1.

s)G(s)}

=

= -sG(s),

e S - 1, which is the probability generating function of

10. This is the claim of Theorem (6.4.13). Without loss of generality we may take s = 0 and the Yj to be non-negative (since if the Yj solve the equations, then so do Yj + c for any constant c). Let T be the matrix obtained from P by deleting the row and column labelled 0, and write Tn = (tij (n) : i, j i= 0). Then Tn includes all the n-step probabilities of paths that never visit zero. We claim first that, for all i, j it is the case that tij (n) ~ 0 as n ~ 00. The quantity tij (n) may be thought of as the n-step transition probability from i to j in an altered chain in which s has been made absorbing. Since the original chain is assumed irreducible, all states communicate with s, and therefore all states other than s are transient in the altered chain, implying by the summability of tij (n) (Corollary (6.2.4)) that tij (n) ~ 0 as required. Iterating the inequality y :::: Ty yields y :::: Tny, which is to say that 00

Yi::::

00

2..: tij(n)Yj:::: min{Yr+s} 2..: s>l

j=l

-

j=r+l

284

tij(n),

Solutions

Stationary distributions and the limit theorem

Let An

= {Xk ¥= 0 fork

[6.4.11]-[6.4.12]

.::: n). Fori:::: I, 00

IP'(Aoo [ Xo

= i) = n-+oo lim IP'(An

[ Xo

= i) = ~ tij(n) L j=!

L tij (n) + mills:::! . Yi {Yr+s) r

.::: lim

} .

n-+oo { j=!

Let E > 0, and pick R such that Yi

----,--..:....':..-----:-<E.

mins::: J{y R+s) Take r = R and let n --+ 00, implying that IP'(Aoo [ Xo by irreducibility that all states are persistent.

= i) = O.

11. By Exercise (6.4.6), the stationary distribution is lrA (a) By Theorem (6.4.3), the answer is /LA

=

l/lrA

It follows that 0 is persistent, and

= lrB = lrD = lrE = r"

lrC

=

i·

= 6.

= lrD/LA = lrD/lrA = 1. = lrC/lrA = 2. = i), let 1) be the time of the first passage to state j, and let Vi = lP'i(TA

(b) By the argument around Lemma (6.4.5), the answer is PD(A) (c) Using the same argument, the answer is pc(A) (d) LetlP'iO = IP'(. [ Xo By conditioning on the first step,

·th

WI

53! ! I . so utlOn V A = "8' vB = 'I> Vc = 2"' vD = 4· A typical conditional transition probability rij

= lP'i(X! =

< TE).

j [ TA < TE) is calculated as follows:

and similarly, rAC

= ~,

rBA

= ~,

rBC

=

i, rCA = ~;. rCB = ~, rCD = ~, rDC = 1.

We now compute the conditional expectations Iii = lEi(TA [ TA < TE) by conditioning on the first step of the conditioned process. This yields equations of the form liA = I + ~ liB + ~ lic, whose solution gives

liA =

1#.

(e) Either use the stationary distribution of the conditional transition matrix T, or condition on the first step as follows. With N the number of visits to D, and 17i = lEi(N [ TA < TE), we obtain 17A

= ~17B + ~17C'

whence in particular 17A

17B

= 0 + i17C,

17C

= 0 + ~17B + ~(l + 17D),

17D

= 17C,

= !~.

12. By Exercise (6.4.6), the stationary distribution has lrA ~, lrB around Lemma (6.4.5), the answer is PB(A) = lrB/LA = lrB/lrA = 2.

285

+. Using the argument

[6.5.1]-[6.5.6]

Solutions

Markov chains

6.S Solutions. Reversibility Look for a solution to the equations 7ri Pij = 7rj Pji. The only non-trivial cases of interest are those with j = i + 1, and therefore Ai7ri = lli+l7ri+1 forO.:s i < b, with solution

1.

O.:s i .:s b,

III 112 ... Ili

an empty product being interpreted as 1. The constant 7rO is chosen in order that the 7ri sum to 1, and the chain is therefore time-reversible.

2.

Leur be the stationary distribution of X, and suppose X is reversible. We have that7ri Pij for all i, j, and furthermore 7r i > 0 for all i. Hence 7riPijPjkPki

= Pji 7rjPjkPki =

PjiPkj 7r kPki

=

Pji7rj

= PjiPkjPik 7r i

as required when n = 3. A similar calculation is valid when n > 3. Suppose conversely that the given display holds for all finite sequences of states. Sum over all values of the subsequence h, ... , jn-l to deduce that Pij (n - I)Pji = Pij Pji (n - 1), where i = jJ, j = jn. Take the limit as n -+ <Xl to obtain 7rj Pji = Pij7ri as required for time-reversibility. 3.

With:rr the stationary distribution of X, look for a stationary distribution v of Y of the form ifi

¢ C,

ifi E C. There are four cases.

= c7rif3Pij = cfj7rjPji = VNji, i, j E C: Viqjj = C7riPij = C7rjPji = VNji, i ¢ C, j E C: Viqjj = cfj7riPij = cfj7rjPji = Vjqji, i, j ¢ C: viqij = cfj7riPij = cfj7rjPji = Vjqji.

(a) i E C, j ¢ C: Viqjj (b) (c)

(d) Hence the modified chain is reversible in equilibrium with stationary distribution v, when

In the limit as fj ..} 0, the chain Y never leaves the set C once it has arrived in it. 4.

Only if the period is 2, because of the detailed balance equations.

5.

With Yn

= Xn

- ~m,

Now iterate. 6. (a) The distribution 7r1 so this chain is reversible.

= {3/(a + fj),

7r2

= a/(a + fj)

(b) By symmetry, the stationary distribution is 7r equations if and only if P

=

= ~.

(c) This chain is reversible if and only if P

=

~.

286

satisfies the detailed balance equations,

(~, ~, ~), which satisfies the detailed balance

Solutions

Chains with finitely many states

[6.5.7]-[6.6.2]

A simple random walk which moves rightwards with probability P has a stationary measure A(pjq)n, in the sense that 1r is a vector satisfying 1r = 1rP. It is not necessarily the case that this 1r has finite sum. It may then be checked that the recipe given in the solution to Exercise (6.5.3)

7.

lTn

=

pip4/

PI P2

L(r,s)EC as stationary distribution for the given process, where C is the yields lTO, j) = relevant region of the plane, and Pi = pi/ qi and Pi (= 1 - qi) is the chance that the i th walk moves rightwards on any given step.

S. Since the chain is irreducible with a finite state space, we have that lTi > 0 for all i. Assume the chain is reversible. The balance equations lTi Pij = lTj Pji give Pij = lTj Pji/lTi. Let D be the matrix with entries IjlTi on the diagonal, and S the matrix (lTj Pji), and check that P = DS. Conversely, ifP balance equations. Note that

= DS, thendi-

1

Pij

= dT I Pji, whenCelTi = d i-

Pi'

=

J

l

/ Lk d

k

1

satisfies the detailed

_1_ §..Pi.-ftj.

.jlfi

VlTj

J

J

If the chain is reversible in equilibrium, the matrix M = (JlTi/lTj Pij) is symmetric, and therefore M, and, by the above, P, has real eigenvalues. An example of the failure of the converse is the transition matrix 1

o o

i (twice), and stationary distribution 1r = (~, ~, b). However, ! = lT3P3l, so that such a chain is not reversible.

which has real eigenvalues 1, and lTIP13

9.

= 0 =f.

Simply check the detailed balance equations lTi Pij

= lTj Pji.

6.6 Solutions. Chains with finitely many states 1. Let P = (Pij : 1 ::::: i, j ::::: n) be a stochastic matrix and let C be the subset of]Rn containing all vectors x = (Xl, x2, ... , xn) satisfying Xi :::: 0 for all i and LI=l Xi = 1; for x E C, let IIxll = maxj{xj}. Define the linear mapping T : C --+ ]Rn by T(x) = xP. Let us check that T is a continuous function from C into C. First,

IIT(x)11 =

mr{ ~XiPij

} :::::

allxll

I

where

hence

IIT(x) - T(y)11 ::::: allx - YII. Secondly, LT(x)j

T(x)j :::: 0 for all j, and

= LLXiPij = LXi LPij =

j

1.

j

j

Applying the given theorem, there exists a point 1C in C such that T (1C) 1C = 1CP.

2.

Let P be a stochastic m x m matrix and let T be the m x (m t .. IJ -

Pi'J -8i'J { 1

287

+ 1) matrix with (i, j)th entry

if j ::::: m, if j

= 1C, which is to say that

= m + 1,

[6.6.3]-[6.6.4]

Solutions

Markov chains

where Dij is the Kronecker delta. Let v = (0,0, ... ,0, 1) is valid, there exists y = (Y1, Y2, ... , Ym+1) such that

~m+1. If statement (ii) of the question

E

m

Ym+1 < 0,

+ Ym+1

'2)Pij - Dij )Yj j=l

:::

°

for 1 :::0 i :::0 m;

this implies that m

L Pij Yj ::: Yi - Ym+1 > Yi j=l

for all i,

and hence the impossibility that LJ=l Pij Yj > maxifYil. It follows that statement (i) holds, which is to say that there exists a non-negative vector x = (Xl, x2, ... ,xm ) such that x(P - I) = 0 and L~l Xi = 1; such an x is the required eigenvector. 3. Thinking of x n+ 1 as the amount you may be sure of winning, you seek a betting scheme x such that X n + 1 is maximized subject to the inequalities n

for 1:::0 j :::0 m.

Xn+1 :::0 LXitij i=l

Writing aij

= -tij

for 1 :::0 i :::0 nand a n+1,j

maximize

x n+1

=

1, we obtain the linear program: n+1 L Xiaij :::0 i=l

subject to

°

for 1 :::0 j :::0 m.

The dual linear program is: minimize

°

m

subject to

L aij Yj j=l

=

°

for 1 :::0 i :::0 n,

m

Lan+1,jYj j=l

= 1,

Yj:::

°

for 1:::0 j :::0 m.

Re-expressing the aij in terms of the tij as above, the dual program takes the form: minimize

°

m

subject to

L tij Pj j=l

=

°

for 1 :::0 i :::0 n,

m

LPj=l, j=l

Pj:::O

for1:::Oj:::om.

The vector x = 0 is a feasible solution of the primal program. The dual program has a feasible solution if and only if statement (a) holds. Therefore, if (a) holds, the dual program has minimal value 0, whence by the duality theorem of linear programming, the maximal value of the primal program is 0, in contradiction of statement (b). On the other hand, if (a) does not hold, the dual has no feasible solution, and therefore the primal program has no optimal solution. That is, the objective function of the primal is unbounded, and therefore (b) holds. [This was proved by De Finetti in 1937.] 4. Use induction, the claim being evidently true when n = 1. Suppose it is true for n = m. Certainly pm+1 is of the correct form, and the equation p m+ 1x' = p(pm x') with x = (1, (1), (1)2) yields in its first row

288

Solutions

Branching processes revisited

[6.6.5]-[6.7.2]

as required.

5. The first part follows from the fact that JlI' = 1 if and only if JlU = 1. The second part follows from the fact that lri > 0 for all i if P is finite and irreducible, since this implies the invertibility of

I-P+ U. 6. The chessboard corresponds to a graph with 8 x 8 = 64 vertices, pairs of which are connected by edges when the corresponding move is legitimate for the piece in question. By Exercises (6.4.6), (6.5.9), we need only check that the graph is connected, and to calculate the degree of a comer vertex. (a) For the king there are 4 vertices of degree 3, 24 of degree 5, 36 of degree 8. Hence, the number of edges is 210 and the degree of a comer is 3. Therefore p,(king) = 420/3 = 140. (b) p,(queen) = (28 x 21 + 20 x 23 + 12 x 25 + 4 x 27)/21 = 208/3. (c) We restrict ourselves to the set of 32 vertices accessible from a given comer. Then p,(bishop) = (14 x 7 + 10 x 9 + 6 x 11 + 2 x 13)/7 = 40. (d) p,(knight) = (4 x 2 + 8 x 3 + 20 x 4 + 16 x 6 + 16 x 8)/2 = 168. (e) p,(rook) = 64 x 14/14 = 64. 7. They are walking on a product space of 8 x 16 vertices. Of these, 6 x 16 have degree 6 x 3 and 16 x 2 have degree 6 x 5. Hence p,(C)

8.

IP - All

=

(6 x 16 x 6 x 3 + 16 x 2 x 6 x 5)/18

= ().. -1)().. + ~)().. + i).

= 448/3.

Tedious computation yields the eigenvectors, and thus

-4 -1

5

6.7 Solutions. Branching processes revisited

=

1. We have (using Theorem (5.4.3), or the fact that Gn+l(S) generating function of Zn is Gn(s)

=

n - (n - 1)s n

+1-

ns

G(Gn(s))) that the probability

,

so that JP'(Zn

= k) =

n

( n

)k+l

+1

for k 2: 1. Therefore, for y > 0, as n --+

JP'(Z <2ynlZ >0)= n -

2.

n

1

-

(n-l) ( n n +1 n +

1

)k-l

-

k 1 n (n + 1)k+l

00,

nk- 1

L2ynJ

'" 1 _ Gn(O) ~ (n

+ 1)k+l

=1-

(1) 1+n

-L2ynJ

---+ l-e- 2y .

Using the independence of different lines of descent, .. ~ skJP'(Zn = k, extinction) ~ skJP'(Zn = k)1)k 1 z = ~ = -G n (S1), E(s n I extmctlOn) = ~ k=O JP'(extinction) k=O 1) 1)

where G n is the probability generating function of Zn.

289

[6.7.3]-[6.8.1]

3.

Solutions

We have that rJ

Markov chains

= G(rJ).

In this case G(s)

= q(1

- ps)-l, and therefore rJ

= q/ p.

Hence

which is G n (s) with p and q interchanged.

4.

(a) Using the fact that var(X E(X2)

= E(X 2 I X

IX

> 0) :::: 0,

> O)lP'(X > 0) :::: E(X

IX

> 0)2lP'(X > 0)

= E(X)E(X I X

> 0).

(b) Hence E(Zn/ f-ln

where Wn

=

I Zn

E(Z2)

> 0) < n - f-lnE(Zn)

= E(W';)

Zn/E(Zn). By an easy calculation (see Lemma (5.4.2)),

where a 2 = var(ZI) = p/q2. (c) Doing the calculation exactly,

E Z

n

(n/f-l

where rJ

n

I

Z > 0 _ E(Zn/f-l ) _

1

)-lP'(Zn>O) -l-Gn(O) ---+ l-rJ

n

= lP'(ultimate extinction) = q / p. 6.S Solutions. Birth processes and the Poisson process

1. Let F and W be the incoming Poisson processes, and let N(t) = F(t)+ Wet). Certainly N(O) = 0 and N is non-decreasing. Arrivals of flies during [0, s] are independent of arrivals during (s, t], if s < t; similarly for wasps. Therefore the aggregated arrival process during [0, s] is independent of the aggregated process during (s, t]. Now lP'(N(t +h)

= n + 1 [N(t) = n) = lP'(A L" B)

where A

=

{one fly arrives during (t, t

+ h J),

B

= {one wasp arrives during (t, t + h J).

We have that lP'(A L" B)

= lP'(A) + lP'(B) - lP'(A n B) = )"h + f-lh - ()..h) (f-lh) + o(h) = ().. + f-l)h + o(h).

Finally lP'(N(t

+ h)

>

n + 1 [ N(t) = n) :::: lP'(A n B) + lP'(C U D), 290

Solutions, [6.8.2]-[6.8.4]

Birth processes and the Poisson process

where C = {two or more flies arrive in (t, t + h]) and D = {two or more wasps arrive in (t, t This probability is no greater than (Ah)(f1-h) + o(h) = o(h).

+ h]).

2. Let I be the incoming Poisson process, and let G be the process of arrivals of green insects. Matters of independence are dealt with as above. Finally,

lP'( G(t + h) = n + 11 G(t) = n) = plP'(! (t + h) = n + 11 I (t) = n) + o(h) = = n):'S lP'(I(t +h) > n+ 11 let) = n) =o(h).

pAh

+ o(h),

lP'(G(t +h) > n+ 1IG(t)

3.

Conditioning on Tl and using the time-homogeneity of the process,

lP'(E(t»xITl=u)=

{

lP'(E(t-U»X)

ifu:'St,

~

iftt+x,

(draw a diagram to help you see this). Therefore 00

lP'(E(t) > x) =

=

10

lP'(E(t) > x [ Tl = u )Ae- AU du

r lP'(E(t Jo

u) > x )Ae- AU du

+

1

00

Ae- AU duo

t+x

You may solve the integral equation using Laplace transforms. Alternately you may guess the answer and then check that it works. The answer is lP'(E(t) :'S x) = 1 - e- h , the exponential distribution. Actually this answer is obvious since E(t) > x if and only if there is no arrival in [t, t + x], an event having probability e- h .

4.

The forward equation is i :'S j,

with boundary conditions Pij (0) = oij, the Kronecker delta. We write Gi (s, t) = L,j sj Pij (t), the probability generating function of B(t) conditional on B(O) = i. Multiply through the differential equation by sj and sum over j: aGi 2aGi aGi - - =AS - - -AS--, at as as

a partial differential equation with boundary condition G i (s, 0) = si. This may be solved in the usual way to obtain Gi (s, t) = g(e At (l - s-l» for some function g. Using the boundary condition, we find that g(l - s-l) = si and so g(u) = (1 - u)-i, yielding

The coefficient of sj is, by the binomial series,

as required.

291

[6.8.5]-[6.8.6]

Solutions

Markov clwins

Alternatively use induction. Set j = i to obtain P;i(t) = -AiPii(t) (remember Pi,i-I (t) and therefore Pii (t) = e- Ail . Rewrite the differential equation as

Set j = i

+ 1and solve to obtain Pi,HI (t)

The mean is E(B(t))

= ie-Ail

(1 - e- At ). Hence (*) holds, by induction.

= ~GI(S, t)1

as

by an easy calculation. Similarly var(B(t))

=

A

= 0),

s=1

=

+ E(B(t)) -

Ie

At

,

E(B(t))2 where

Alternatively, note that B(t) has the negative binomial distribution with parameters e- At and I.

5.

The forward equations are P~ (t)

where Ai = iA

= An-I Pn-I (t)

n::: 0,

- AnPn (t),

+ v. The process is honest, and therefore met) = 00

m'(t)

=

L:n npn(t) satisfies

00

L n [en - I)A n=1

+ V]Pn-1 (t) -

L

n(nA

+ v)Pn(t)

00

=L

{A[(n

+ l)n -

2 n ] + v[(n

+ 1) -

nl} Pn(t)

n=O 00

= L(An n=O

+ v)Pn(t) =

Am(t)

+ v.

Solve subject to m(O) = 0 to obtain met) = v(e At - 1)/A.

6. Using the fact that the time to the nth arrival is the sum of exponential interarrival times (or using equation (6.8.15)), we have that

is given by

which may be expressed, using partial fractions, as

where a'-

IIn

A' _J_

I - A ' -A' j=O J I ji.i

292

Solutions

Continuous-time Markov chains

so long as Ai

=1=

A} whenever i

=1= j.

[6.8.7]-[6.9.1]

The Laplace transform Pn may now be inverted as

See also Exercise (4.8.4).

7. Let Tn be the time of the nth arrival, and let T Exercise (6.8.6), AnPn(8)

=

= lim n-+ oo Tn =

A' IIn __

l -

i=O Ai

+8

sup(t : N(t) < 00). Now, as in

= lE(e- 61Tn )

since Tn = Xo + X 1+ ... + Xn where Xk is the (k + 1)th interarrival time, a random variable which is exponentially distributed with parameter Ak. Using the continuity theorem, lE(e- 6ITn ) -+ lE(e- 6IT ) as n -+ 00, whence AnPn(8) -+ lE(e- 61T ) as n -+ 00, which may be inverted to obtain AnPn(t) -+ f(t) as n -+ 00 where f is the density function of T. Now

I

lE(N(t) N(t) < 00)

= L:~o npn(t) L:n=O Pn(t)

which converges or diverges according to whether or not L:n npn (t) converges. However Pn (t) ~ A;;-I f(t) as n -+ 00, so that L:n npn(t) < 00 if and only if L:n nA;;-1 < 00.

When An

=

(n

+ ~)2, we have that lE(e- 61T )

=

II 00

{

n=O

1+

8I 2

(n

+ 2)

}-I

= sech (7l'Je).

Inverting the Laplace transform (or consulting a table of such transforms) we find that

where 81 is the first Jacobi theta function.

6.9 Solutions. Continuous-time Markov chains 1.

(a) We have that

where PI2 = 1 - PII, P21 = 1 - P22· Solve these subject to Pi} (t) obtain that the matrix PI = (Pi) (t)) is given by

= 8ij, the Kronecker delta, to

(b) There are many ways of calculating Gn ; let us use generating functions. Note first that GO the identity matrix. Write

n:::: 0, 293

=

I,

[6.9.2]-[6.9.4]

Solutions

and use the equation G n +1

Markov clwins

G· G n to find that

=

Hencean+1 = -(J.II).)cn+1 forn::: 0, and the first difference equation becomes an+1 = -().+J.I)an, n ::: I, which, subject to al = -J.I, has solution an = (_l)n J.I()' + J.I)n-l, n ::: 1. Therefore Cn = (_l)n+I).(). + J.I)n-1 for n ::: I, and one may see similarly that b n = -an, dn = -Cn for n ::: 1. Using the facts that ao = do = I and bo = Co = 0, we deduce that L~o(tn In!)Gn = Pt where Pt is given in part (a).

(c) With 11" = (7TI, 7T2), we have that -J.I7TI + ).7T2 = 0 and J.I7TI - ).7T2 = 0, whence 7TI = ().I J.I)7T2. + 7T2 = I if7TI = ).I(). + J.I) = I -7T2·

In addition, 7TI 2.

(a) The required probability is lP'(X(t) = 2, X(3t) = I [X(O) = I) lP'(X(3t)

=

I [X(O)

=

Pl2(t)P21(2t)

I)

Pll(3t)

using the Markov property and the homogeneity of the process. (b) Likewise, the required probability is Pl2 (t) P2I (2t) Pll (t)

Pll (3t) Pll (t)

the same as in part (a). 3. The interarrival times and runtimes are independent and exponentially distributed. It is the lackof-memory property which guarantees that X has the Markov property. The state space is S = {O, 1,2, ... } and the generator is

G~ Solutions of the equation 11" G

c ~

).

-(). + J.I)

).

0

0 0

J.I

-().+J.I)

).

...

)

= 0 satisfy

with solution7Ti = 7To().IJ.I)i. We have in addition that Li 7Ti = I if). < J.Iand7To = {I- ().IJ.I)}-I.

4.

One may use the strong Markov property. Alternatively, by the Markov property, lP'(Yn+1 = j [ Yn = i, Tn = t, B) = lP'(Yn+ 1 = j [ Yn = i, Tn = t)

for any event B defined in terms of {X (s) : s :::: Tn}. Hence lP'(Yn+ 1 = j [ Yn = i, B) =

10

00

lP'(Yn+ 1 = j [ Yn = i, Tn = t)!Tn(t)dt

= lP'(Yn+1 =

j [ Yn

= i),

so that Y is a Markov chain. Now % = lP'(Yn+ 1 = j [ Yn = i) is given by

294

Solutions

Continuous-time Markov chains

[6.9.5]-[6.9.8]

by conditioning on the (n + 1)th interarrival time of N; here, as usual, Pij (t) is a transition probability of X. Now

5. The jump chain Z = {Zn : n 2: 0) has transition probabilities hij = gij / gi, i #- j. The chance that Z ever reaches A from j is also T/j' and T/j = L:k h jkT/k for j rf. A, by Exercise (6.3.6). Hence -gjT/j = L:k gjkT/b as required. 6. Let Tl of X. For j

= inf{t rf. A,

: X (t)

#-

X (0»), and more generally let Tm be the time ofthe mth change in value JLj

= lEj(T1) + 2.= hjkJLb k#j

where lEj denotes expectation conditional on Xo = j. Now lEj (Tj) = gj-l , and the given equations follow. Suppose next that (ak : k E S) is another non-negative solution of these equations. With Ui = Ti+l - Ti and R = min{n 2: 1 : Zn E A), we have for j rf. A that

where 1.: is a sum of non-negative terms. It follows that aj 2: lEj(UO)

= lEj

+ lEj(UII{R>I}) + ... + lEj(UnI{R>n})

(t

UrI{R>r})

= lEj (min {Tn , HA))

-+ lEj(HA)

r=O

as n -+

00,

by monotone convergence. Therefore, fL is the minimal non-negative solution.

7. First note that i is persistent if and only if it is also a persistent state in the jump chain Z. The integrand being positive, we can write

where {Tn : n 2: 1) are the times of the jumps of X. The right side equals 00

1

00

n=O

I

n=O

2.= lE(Tl I X(O) = i)hii(n) = g. 2.= hii(n) where H = (hij) is the transition matrix of Z. The sum diverges if and only if i is persistent for Z. 8. Since the imbedded jump walk is persistent, so is X. The probability of visiting m during an excursion is a = (2m)-I, since such a visit requires an initial step to the right, followed by a visit to m before 0, cf. Example (3.9.6). Having arrived at m, the chance of returning to m before visiting 0 is 1 - a, by the same argument with 0 and m interchanged. In this way one sees that the number N of visits to m during an excursion from 0 has distribution given by IP'(N 2: k) = a(l - a)k-l, k 2: 1. The 'total jump rate' from any state is A, whence T may be expressed as L:~o Vi where the Vi are exponential with parameter A. Therefore, lE(e er )=GN

(A)e -A-

= ( l - a ) + aaA --. aA -

295

e

[6.9.9]-[6.9.11]

Solutions

Markov chains

The distribution of T is a mixture of an atom at 0 and the exponential distribution with parameter aA. 9. The number N of sojourns in i has a geometric distribution IP'(N = k) = jk-l (1 - f), k 2': 1, for some j < 1. The length of each of these sojourns has the exponential distribution with some parameter gi. By the independence of these lengths, the total time T in state i has moment generating function gi(l-f) JE(eeT) = jk-l(l _ f) ( )k gi(l - f) k=l gl

f

g~

e

e

The distribution of T is exponential with parameter gi (1 - f). 10. The jump chain is the simple random walk with probabilities 1../ (A + p,) and p,/ (A + p,), and with

POl = 1. By Corollary (5.3.6), the chance of ever hitting 0 having started at 1 is p,/A, whence the probability of returning to 0 having started there is j = p,/ A. By the result of Exercise (6.9.9),

as required. Having started at 0, the walk visits the state r 2': 1 with probability 1. The probability of returning to r having started there is p, jr= A+p,

A

p,

2p,

+ A+p, ·i=

A+p,'

and each sojourn is exponentially distributed with parameter gr whence, as above, JE(eeVr)

=

= A + p,.

Now gr (l - jr)

= A-

p"

1..- p, A -p,-e

The probability of ever reaching 0 from X (0) is (p,/ A)X (0), and the time spent there subsequently is exponential with parameter A - p,. Therefore, the mean total time spent at 0 is JE (P,/A)X(O))

=

G(p,/A). A-p,

A-p,

11. (a) The imbedded chain has transition probabilities

hkj

where gk

= - gkk.

= { ogkj /gk

if k if k

=I- j,

=

j,

Therefore, for any state j,

where we have used the fact that1rG distribution of Y.

= O.

Also nk 2': 0 and Lk nk

=

1, and therefore ii is a stationary

Clearly nk = Jrk for all k if and only if gk = Li Jrigi for all k, which is to say that gi all pairs i, k. This requires that the 'holding times' have the same distribution.

= gk for

(b) Let Tn be the time of the nth change of value of X, with To = 0, and let Un = Tn+l - Tn. Fix a state k, and let H = min{n 2': 1 : Zn = k}. Let Yi(k) be the mean time spent in state i between two consecutive visits to k, and let }Ii (k) be the mean number of visits to i by the jump chain in between two

296

Solutions

Birth-death processes and imbedding

[6.9.12]-[6.11.2]

visits to k (so that, in particular, Yk(k) = gj;l and n(k) = 1). With lEj and IP'j denoting expectation and probability conditional on X (0) = j, we have that Yi (k) = lEk

(f

f

UnI{Zn=i, H>n}) =

n=O 00

=L

lEk(Un

I I{Zn=i})IP'k(Zn =

i, H > n)

n=O

1

-lP'k(Zn n=O gi

=

i, H > n)

=

I -Yi(k). gi

The vector y(k) = (Yi(k) : i E S) satisfies y(k)H = y(k), by Lemma (6.4.5), where H is the transition matrix of the jump chain Z. That is to say, for j

E

S,

whence L:i Yi (k)gij = 0 for all j E S. If iJ,k = L:i Yi (k) < 00, the vector (Yi (k) / iJ,k) is a stationary distribution for X, whence Tri = Yi (k) / iJ,k for all i. Setting i = k we deduce that Trk = 1/ (gk iJ,k). Finally, if L:i Trigi < 00, then

~

1

iJ,k = -;:;- = Trk

L:i Tri gi '" = iJ,k LTrigi· Trkgk i

12. Define the generator G by gii = -Vi, gij = Vihij, so that the imbedded chain has transition matrix H. A root of the equation JrG = 0 satisfies

0= LTrigij = -TrjVj

+

L (TriVi)hij i:i,ij

s

whence the vector S = (Trj Vj : j E S) satisfies S = SH. Therefore = av, which is to say that Trj Vj = aVj ,for some constant a. Now Vj > 0 for all j, so thatTrj = a, which implies that L:j Trj #- 1. Therefore the continuous-time chain X with generator G has no stationary distribution.

6.11 Solutions. Birth-death processes and imbedding 1.

The jump chain is a walk {Zn} on the set S = {O, 1,2, ... } satisfying, for i ::: I, IP'(Zn+l = j

ifj=i+l,

I Zn = i) = { Pi

1- Pi

if j

=

i-I,

where Pi = Ai/(Ai + iJ,i)· Also IP'(Zn+l = 1 I Zn = 0) = 1. 2.

The transition matrix H = (hij) of Z is given by

~iiJ,

if j

= i-I,

h lJ.. -_ { A A

A+iiJ,

ifj=i+l.

To find the stationary distribution of Y, either solve the equation 11' = JrQ, or look for a solution of the detailed balance equations Trihi,i+l = Tri+l hi+l,i' Following the latter route, we have that i ::: 1,

297

[6.11.3]-[6.11.4] whence 7ri

Solutions

Markov chains

= 7rOpi (1 + i I p)1 i! for i

2: 1. Choosing

7ro

accordingly, we obtain the result.

It is a standard calculation that X has stationary distribution v given by Vi = pi e - PI i! for i 2: O. The difference between 7r and v arises from the fact that the holding-times of X have distributions which depend on the current state.

3.

We have, by conditioning on X(h), that rJ(t

+ h) = E{IP'(X(t + h) = 0 IX(h»)} = J-Lh . 1 + (1 - Ah - J-Lh)rJ(t) + Ah~(t) + o(h)

where ~(t) = IP'(X (t) = 0 I X(O) = 2). The process X may be thought of as a collection of particles each of which dies at rate J-L and divides at rate A, different particles enjoying a certain independence; this is a consequence of the linearity of An and J-Ln. Hence ~(t) = rJ(t)2, since each of the initial pair is required to have no descendants at time t. Therefore rJ' (t) = J-L - (A

+ J-L)rJ(t) + ArJ(t)2

subject to rJ(O) = O. Rewrite the equation as rJ' ------=1 (1 - rJ)(J-L - ArJ)

and solve using partial fractions to obtain if)..

= J-L,

if).. '" J-L.

Finally, if 0 < t < u, IP'(X(t) = 0 [X(u) = 0) = IP'(X(u) = 0 [X(t) = 0)

4.

IP'(X(t) = 0) IP'(X(u)

= 0)

The random variable X (t) has generating function G (s, t)

J-L(1 - s) - (J-L - As)e-t()..-p,)

= --------,-;---,A(1 - s) - (J-L - As)e-t()..-p,)

as usual. The generating function of X(t), conditional on {X(t) > O}, is therefore

~ Sn IP'(X(t) = n) = G(S, t) - G(O, t). ~ IP'(X(t) > 0) 1 - G(O, t) Substitute for G and take the limit as t -+

H(s)

where, with p

=

00

to obtain as limit

(J-L - A)S J-L -AS

= AI J-L, we have that Pn = pn-l(1

=

~ n

LS Pn n=l

- p) for n 2: 1.

298

rJ(t)

= -.

rJ(u)

Solutions

Special processes

5.

[6.11.5]-[6.12.1]

Extinction is certain if A < J-i, and in this case, by Theorem (6.11.10), JE(T)

l)O

=

00

=

IT

< (0)

=

00

10o

l)O {1 -

JE(sX (t)) Is=o} dt

(J-i - A)eO"'-/L)t 1 (J-i) dt = - log - - . J-i - Ae().·-/L)t A J-i - A

= J-i/A, so

If A > J-i then lP'(T < (0) JE(T

10o

=

lP'(T > t) dt

{A } 1 - -JE(sX(t))1 =0 dt J-i

In the case A = J-i, lP'(T < (0)

s

=

1 and JE(T)

= 10

00

0

(A J-i)e(/L-)..)t - (-)..)t dt A - J-ie /L

=

1 (A) -log - - . J-i A - J-i

= 00.

6. By considering the imbedded random walk, we find that the probability of ever returning to 1 is max{A, J-i}/(A + J-i), so that the number of visits is geometric with parameter min{A, J-i}/(A + J-i). Each visit has an exponentially distributed duration with parameter A + J-i, and a short calculation using moment generating functions shows that VI (00) is exponential with parameter min{A, J-i}. Next, by a change of variables, Theorem (6.11.10), and some calculation,

l

~SrJE(Vr(t)) = JE(~ =

t

loo

JE(s

sr I{X(u)=r} dU)

X(u)

J-it

=- -

) du

A

1 { As(e - 1) } -log 1 t J-ie P -A

A

= J-i - A. We take the limit as t ~ If A = J-i then, by Theorem (6.11.1 0),

where P 7.

JE(sX(t))

=

00

(l

sX(u) dU)

1 {A(1 - s) - (J-i - AS)e-()..-/L)t} -log A A - J-i

pt

=-

= JE

+ terms not involving s,

and we pick out the coefficient of sr.

)..t(1-s)+s )..t(1 - s) + 1

= 1-

1-s

)..t(l - s)

+1

and t

loo JE(sX(u)) du = t -

1 -log{)..t(1 - s)

A

= - -A1 log { 1 -

+ I}

)..ts} 1 +)..t

. s. + terms not . mvolvmg

Letting t ~ 00 and picking out the coefficient of sr gives JE(Vr(oo)) = (rA)-I. An alternative method utilizes the imbedded simple random walk and the exponentiality of the sojourn times.

6.12 Solutions. Special processes 1. The jump chain is simple random walk with step probabilities A/ (A expected time J-i1O to pass from 1 to 0 satisfies

299

+ J-i) and J-i/ (A + J-i).

The

[6.12.2]-[6.12.4]

Solutions

Markov chains

whence fLlO = (fL + 'A)/(fL - 'A). Since each sojourn is exponentially distributed with parameter fL + 'A, the result follows by an easy calculation. See also Theorem (11.3.17).

2.

We apply the method of Theorem (6.12.11) with

the probability generating function of the population size at time u in a simple birth process. In the absence of disasters, the probability generating function of the ensuing population size at time v is

H(s, v)

= exp (v lov [GN(S, u)

- 1]

dU)

=

{s

+ (l

- s)e

AV }-viA.

The individuals alive at time t arose subsequent to the most recent disaster at time t - D, where D has density function 8e- 8x , x > O. Therefore,

3. The mean number of descendants after time t of a single progenitor at time 0 is e(A-/.L)t. The expected number due to the arrival of a single individual at a uniformly distributed time in the interval on [0, x] is therefore 1 loX

-

x

0

e

(A- ) /.L U du

e(A-/.L)X -

1

= ----('A - fL)X

The aggregate effect at time x of N earlier arrivals is the same, by Theorem (6.12.7), as that of N arrivals at independent times which are uniformly distributed on [0, x]. Since E(N) = vx, the mean population size at time x is v[e(A-/.L)X - 1]1 ('A - fL). The most recent disaster occurred at time t - D, where D has density function 8e -8x, x > 0, and it follows that

This is bounded as t

--+ 00

if and only if 8 > 'A - fL.

4.

Let N be the number of clients who arrive during the interval [0, t]. Conditional on the event {N = n}, the arrival times have, by Theorem (6.12.7), the same joint distribution as n independent variables chosen uniformly from [0, t]. The probability that an arrival at a uniform time in [0, t] is still in service at time tis f3 = J~[l - G(t - x)]t- 1 dx, whence, conditional on {N = n}, the total number M still in service is bin(n, f3). Therefore,

whence M has the Poisson distribution with parameter 'Af3t parameter approaches 'AE(S) as t --+ 00.

300

=

'A J~[l

-

G(x)] dx. Note that this

Solutions

Spatial Poisson processes

[6.13.1]-[6.13.3]

6.13 Solutions. Spatial Poisson processes 1. It is easy to check from the axioms that the combined process N(t) = B(t) + G(t) is a Poisson process with intensity f3 + y. (a) The time S (respectively, T) until the arrival of the first brown (respectively, grizzly) bear is exponentially distributed with parameter f3 (respectively, y), and these times are independent. Now,

roo f3e-fJse-Ys ds =

JP>(S < T) =

Jo

_f3_.

f3 + y

(b) Using (a), and the lack-of-memory of the process, the required probability is

(c) Using Theorem (6.12.7),

I

E(min(S, T} B(1)

= 1) = E .

{ I} = 1+ ey -

G(1)+2

y

2. Let Br be the ball with centre 0 and radius r, and let N r (6.13.11) that Sr = Z=XEnnBr g(x) satisfies (

E(Sr

1

Nr ) = Nr

E(S; 1 N r )

=E

(l L

EnnB r

E

Br

H

2 ).(u) g(u) - - du A(Br)

We have by Theorem

).(u)

JBr g(u)).(u) du, implying by monotone conver-

(

~

g(x)g(y))

x,YEnnBr

+ N r (Nr

- 1)

t1u u,vEB r

g(u)g(v)

whence E(S;)

.

g(X)f)

~ C~B' g(X)2») = 1 Nr

= 1TI n Br I.

Y

JBr g(u) A(Br) du,

= JYEB ).(y) dy. Therefore, E(Sr) = gence that E(S) = JlRd g(u)).(u) duo Similarly, where A(B)

2

= 1sr g(u)2).(u) du +

(1sr g(u)).(u) du

). (u)). (v) du dv, A(Br)

2

r

By monotone convergence,

and the formula for the variance follows.

3. If Bl, B2, ... , Bn are disjoint regions of the disc, then the numbers of projected points therein are Poisson-distributed and independent, since they originate from disjoint regions of the sphere. By

301

[6.13.4]-[6.13.8]

Solutions

Markov chains

elementary coordinate geometry, the intensity function in plane polar coordinates is 2).../~, 0::::r::::I,0::::tI<2n.

4.

The same argument is valid with resulting intensity function

2)... ~.

5. The Mercator projection represents the spherical coordinates (tI, 1) as Cartesian coordinates in the range 0 :::: 1 < 2n, 0 :::: tI :::: n. (Recall that tI is the angle made with the axis through the north pole.) Therefore a uniform intensity on the globe corresponds to an intensity.function Asin tI on the map. Likewise, a uniform intensity on the map corresponds to an intensity A/ sin tI on the globe. 6. Let the X, have characteristic function 1. Conditional on the value of N(t), the corresponding arrival times have the same distribution as N(t) independent variables with the uniform distribution, whence lE(e i61s (t»)

= lE{lE(e i61S (t)

I N(t))}

= lE{lE(ei61Xe-au )N(t)}

= exp{At(lE(ei61Xe-au) - I)} = exp { Alot {1(tle- au ) -I}dU}, where U is uniformly distributed on [0, tl. By differentiation,

Now, for s < t, S(t) = S(s)e-a(t-s) + S(t distribution as S(t - s). Hence, for s < t,

as s ---+ 7.

00

with v

=t -

-

s) where S(t

s fixed. Therefore, p(S(s), S(s

-

s) is independent of S(s) with the same

+ v)) ---+

e- av as s ---+

00.

The first two arrival times TI, T2 satisfy

Differentiate with respect to x and y to obtain the joint density function A(X)A(X + y)e-A(x+ y ), x, y ::: o. Since this does not generally factorize as the product of a function of x and a function of y, TI and T2 are dependent in general. 8.

Let Xi be the time of the first arrival in the process Ni. Then lP(l

= 1,

T ::: t)

= lP(t :::: Xl =

1

< inf[X2, X3, ... })

00

lP(inf[X2, X3, ... } > X)Ale-)'\X dx

t

302

= ~e-At. A

Solutions

Markov chain Monte Carlo

[6.14.1H6.14.4]

6.14 Solutions. Markov chain Monte Carlo 1.

If P is reversible then

RHS = L(LPijXj )Yini = LniPijXjYi = LnjpjiYiXj i j i,j i,j

=

Lnjxj (LPjiYi) = LHS. j i

Suppose conversely that (x, Py) = (Px, y) for all x, y E 12(n). Choose x, y to be unit vectors with 1 in the ith and jth place respectively, to obtain the detailed balance equations ni Pij = nj Pji. 2. Just check that 0 (6.14.3).

:s

bij

:s

1 and that the Pij = gij bij satisfy the detailed balance equations

3. It is immediate that Pjk = IAjkl, the Lebesgue measure of Ajk. This is a method for simulating a Markov chain with a given transition matrix.

4. (a) Note first from equation (4.12.7) that d(U) = ~ SUPi#j dTV(Ui., Uj.), where Ui. is the mass function Uit, t E T. The required inequality may be hacked out, but instead we will use the maximal coupling of Exercises (4.12.4, 5); see also Problem (7.11.16). Thus requires a little notation. For i, j E S, i =f. j, we find a pair (Xi, Xj) of random variables taking values in T according to the marginal mass functions Ui., Uj., and such that JP'(Xi =f. Xj) = ~dTV(Ui" uj-). The existence of such a pair was proved in Exercise (4.12.5). Note that the value of Xi depends on j, but this fact has been suppressed from the notation for ease of reading. Having found (Xi, Xj), we find a pair (Y(Xi)' Y(Xj)) taking values in U according to the marginal mass functions vXi" VXj" and such that JP'(Y (Xi) =f. Y (Xj)

I Xi,

Xj)

= ~dTV(VXi"

VXr)' Now, taking a further liberty with the notation,

JP'(Y(Xi) =f. Y(Xj))\=L

JP'(Xi

= r,

Xj

= s)JP'(Y(r) =f.

yes))

JP'(Xi

= r,

Xj

= S)idTV(Vr"

Vs·)

,SES

=Is

= L r,sES

r=ls

:s

g SUpdTV(Vr., Vs.)}JP'(Xi =f. Xj), r=ls

whence d(UV) = suPJP'(Y(Xi) =f. Y(Xj))

g sup dTV (Vr., Vs.)}{SUPJP'(Xi =f. Xj)}

:s

r=ls

i#j

i,j

and the claim follows. (b) Write S = {I, 2, ... , m}, and take U

=

(JP'(Xo JP'(YO

= 1)

=

1)

JP'(XO JP'(YO

= 2)

= 2)

m))

JP'(Xo = JP'(YO = m)

.

The claim follows by repeated application of the result of part (a). It may be shown with the aid of a little matrix theory that the second largest eigenvalue of a finite stochastic matrix P is no larger in modulus that d(P); cf. the equation prior to Theorem (6.14.9).

303

[6.15.1]-[6.15.6]

Solutions

Marlwv chains

6.15 Solutions to problems 1. (a) The state 4 is absorbing. The state 3 communicates with 4, and is therefore transient. The set (1, 2} is finite, closed, and aperiodic, and hence ergodic. We have that 134(n) = (~)n-1~, so that 134 = 2::n 134(n) = ~. (b) The chain is irreducible with period 2. All states are non-null persistent. Solve the equation 5 1f = 1f P to find the stationary distribution 1f = (~, ?6' 16' ~) whence the mean recurrence times are 8 16 16 8· d or' "3' J' , ill or er.

2.

(a) Let P be the transition matrix, assumed to be doubly stochastic. Then

~Pij(n) = ~~Pik(n i

i

I)Pkj

= ~(~Pik(n

k

k

-1»)Pkj

i

whence, by induction, the n-step transition matrix pn is doubly stochastic for all n :::: 1. If j is not non-null persistent, then Pij (n) -+ 0 as n -+ oo,for all i, implying that 2::i Pij (n) -+ 0, a contradiction. Therefore all states are non-null persistent. If in addition the chain is irreducible and aperiodic then Pij (n) -+ nj, where 1f is the unique stationary distribution. However, it is easy to check that 1f = (N- 1 , N- 1, ... , N- 1) is a stationary distribution if P is doubly stochastic. (b) Suppose the chain is persistent. In this case there exists a positive root of the equation x = xP, this root being unique up to a multiplicative constant (see Theorem (6.4.6) and the forthcoming Problem vector of 1's. By the (7». Since the transition matrix is doubly stochastic, we may take x = 1, ~e above uniqueness of x, there can exist no stationary distribution, and theret e the chain is null. We deduce that the chain cannot be non-null persistent. 3.

By the Chapman-Kolmogorov equations, m,r,n:::: O.

Choose two states i and j, and pick m and n such that a Pii(m

+ r + n)

=

Pij (m) Pj i (n) > O. Then

:::: apjj(r).

Set r = 0 to find that Pii (m + n) > 0, and so d(i) I (m + n). If d(i) t r then Pii (m + r that Pjj(r) = 0; therefore d(i) I d(j). Similarly d(j) I d(i), giving that d(i) = d(j).

+ n) = 0, so

4. (a) See the solution to Exercise (6.3.9a). (b) Let i, j, r, s E S, and choose N(i, r) and N(j, s) according to part (a). Then p(Zn

= (r,s) IZo = (i, j») = Pir(n)Pjs(n)

> 0

if n :::: max(N(i, r), N (j, s)}, so that the chain is irreducible and aperiodic. (c) Suppose S = (1, 2} and

P=

Cb)·

In this case {(I, I}, (2, 2}} and {(1, 2}, (2, I}} are closed sets of states for the bivariate chain. 5. Clearly P(N = 0) = 1 - iij, while, by conditioning on the time of the nth visit to j, we have that P(N :::: n + 1 IN:::: n) = ijj for n :::: 1, whence the answer is immediate. Now P(N = (0) = 1 - 2::~0 P(N = n) which equals 1 if and only if iij = ijj = 1. 6. Fix i =1= j and let m = min(n : Pij (n) > O}. If Xo = i and Xm = j then there can be no intermediate visit to i (with probability one), since such a visit would contradict the minimality of m.

304

Solutions

Problems

[6.15.71-[6.15.81

Suppose Xo = i, and note that (1 - /ji)Pij(m) :5 1 - iii, since if the chain visits j at time m and subsequently does not return to i, then no return to i takes place at all. However iii = 1 if i is persistent, so that /j i = 1. 7.

(a) We may take S

= {O, 1,2, ... }.

Note that % (n) ::: 0, and 00

L%(n) j

=

qij(n

1,

+ 1) = LQil(l)qlj(n), 1=0

whence Q = (% (1)) is the transition matrix of a Markov chain, and Qn persistent since for all i,

=

(qij (n)). This chain is

n

n

and irreducible since i communicates with j in the new chain whenever j communicates with i in the original chain. That

=I-

i

is evident when n

= 1 since both sides are qij (1). lji(m

L

+ 1) =

j, n ::: 1,

Suppose it is true for n

= m where m :::

1. Now

i =I- j,

Ijk(m)Pki,

k:klj

so that

X'

-.l..lji(m Xi

+ 1) =

L

gkj (m)qik (1),

i =I- j,

k:klj

which equals gij (m + 1) as required. (b) Sum (*) over n to obtain that i

=I-

j,

where Pi(j) is the mean number of visits to i between two visits to j; we have used the fact that 2:n gij (n) = 1, since the chain is persistent (see Problem (6.15.6)). It follows that Xi = XOPi (0) for all i, and therefore x is unique up to a multiplicative constant. (c) The claim is trivial when i = j, and we assume therefore that i =I- j. Let Ni (j) be the number of visits to i before reaching j for the first time, and write lP'k and lEk for probability and expectation conditional on Xo = k. Clearly, lP'j(Ni(j)::: r) = hji(1- hijt- 1 for r::: 1, whence

The claim follows by (**). 8.

(a) If such a Markov chain exists, then n

Un

=L

iiUn-i,

i=l

305

n::: 1,

[6.15.9]-[6.15.9]

Solutions

Marlwv chains

where Ii is the probability that the first return of X to its persistent starting point s takes place at time i. Certainly Uo = 1. Conversely, suppose u is a renewal sequence with respect to the collection (fm : m :::: 1). Let X be a Markov chain on the state space S = {O, 1, 2, ... } with transition matrix .. _ { lP'(T :::: i + 2 IT:::: i + 1) 1 - lP'(T :::: i + 2 IT:::: i + 1)

if j = i + 1, if j = 0,

PI] -

where T is a random variable having mass function the first return to 0 takes place at time n is

lP'( Xn

= 0,

IT

Xi

1m = lP'(T = m).

I- 0 I Xo = 0) = POIP12'"

(1 _

G(n + G(n)

= G(n) where G(m)

= lP'(T

:::: m)

= L:~m In.

Now Vn

Vo = 1,

Vn

= 0, the chance that

Pn-2,n-lPn-l,O

1

=

With Xo

G(n

1)) IT G(i + 1) i=l

+ 1) =

G(i)

In

= lP'(Xn = 0 I Xo = 0) satisfies

n

=

L liVn-i

forn::::I,

i=l

whence Vn = Un for all n. (b) Let X and Y be the two Markov chains which are associated (respectively) with u and v in the above sense. We shall assume that X and Y are independent. The product (unvn : n :::: 1) is now the renewal sequence associated with the bivariate Markov chain (Xn, Yn). 9. Of the first 2n steps, let there be i rightwards, j upwards, and k inwards. Now X2n = 0 if and only if there are also i Ieftwards, j downwards, and k outwards. The number of such possible combinations is (2n)!/ {(i! j! k!)2}, and each such combination has probability (t)2(i+ j+k) = (t)2n. The first equality follows, and the second is immediate. Now

lP'(X2n where

= O):s ("21) 2n

(2n) n

M=max{3n'~!"k': J ..

M

n! L 3n" "k' i+ +k=n J. . j

1•

i,j,k::::O, i+j+k=n}.

I.

It is not difficult to see that the maximum M is attained when i, j, and k are all closest to

1n, so that

Furthermore the summation in (*) equals 1, since the summand is the probability that, in allocating n balls randomly to three urns, the urns contain respectively i, j, and k balls. It follows that

lP'(X

2n

= 0)

<

-

(2n)' 12nn!

306

(l1.n J!)3

Solutions [6.15.10]-[6.15.13]

Problems

3

which, by an application of Stirling's fonnula, is no bigger than Cn-'2 for some constant C. Hence Ln 1P'(X2n = 0) < 00, so that the origin is transient.

10. No. The line of ancestors of any cell-state is a random walk in three dimensions. The difference between two such lines of ancestors is also a type of random walk, which in three dimensions is transient.

11. There are one or two absorbing states according as whether one or both of a and fJ equal zero. If afJ =f. 0, the chain is irreducible and persistent. It is periodic if and only if a = fJ = 1, in which case it has period 2. IfO < afJ < 1 then

is the stationary distribution. There are various ways of calculating pn; see Exercise (6.3.3) for example. In this case the answer is given by

proof by induction. Hence as n ---+

00.

The chain is reversible in equilibrium if and only if nl Pl2 = n2P2l, which is to say that afJ = fJa !

12. The transition matrix is given by

Pij

=

(NN- i)2 i (NN- i ) 1 _ (N ) i (N )2 2 _

ifj=i+l, 2

if j

= i,

if j

= i-I,

for 0 :s i :s N. This process is a birth-death process in discrete time, and by Exercise (6.5.1) is reversible in equilibrium. Its stationary distribution satisfies the detailed balance equation ni Pi,i+l = ni+lPi+l,i for O:s i < N, whence ni =

no(~)2 for O:s i :s N, where

~ = t (~)2 = (2N).

no

i=O

I

N

13. (a) The chain X is irreducible; all states are therefore of the same type. The state 0 is aperiodic, and so therefore is every other state. Suppose that Xo = 0, and let T be the time of the first return to O. Then IP'(T > n) = aOal ... an-l = b n for n 2: 1, so that 0 is persistent if and only if b n ---+ 0 as n ---+ 00. (b) The mean of T is 00

00

E(T) = LIP'(T > n) = L bn . n=O

n=O

307

[6.15.14]-[6.15.14]

Solutions

Marlwv chains

The stationary distribution 1C satisfies 00

1CO

=

L 1Ck(l- ak), k=O

1Cn

= 1Cn-lan -l

for n ::: 1.

Hence 1Cn = 1Cobn and 1CO1 = L:~o b n if this sum is finite. (c) Suppose ai has the stated form for i ::: I. Then n-l bn

= bl

II (l -

Ai- f3 ),

n ::: I.

i=1

Hence b n --+ 0 if and only if L:i Ai- f3 = 00, which is to say that persistent if and only if f3 ::::: 1. (d) We have that 1 - x ::::: e- x for x ::: 0, and therefore

f3 ::::: 1. The chain is therefore

00 00 { n-l} 00 L b n ::::: bI L exp -A L i- f3 ::::: bl L exp {-An. n- f3 } < n=I n=1 i=1 n=1

(e) If f3

=

00

iff3<1.

1 and A > 1, there is a constant c I such that

::

00 00 { n-l 1 L bn ::::: bl L exp -A n=I n=1 1=1

~·d

00

00

CJ Lexp{-Alogn} = CJ L

n=I

n-

A

<

00,

n=I

giving that the chain is non-null. (f) If f3

= 1 and A

::::: 1,

Therefore L:n b n =

00,

and the chain is null.

14. Using the Chapman-Kolmogorov equations, IPij(t +h) - Pij(t)1

=

IL(Pik(h) - 8ik)Pkj(t)1 ::::: k

::::: (1 - Pii(h))

+ (1- Pii(h))

(1- Pii (h))pij (t) + LPik(h) k# --+

0

as h ,j,. 0, if the semigroup is standard. Now log x is continuous for 0 < x ::::: 1, and therefore g is continuous. Certainly g(O) addition Pii (s + t) ::: Pii (S)Pii (t) for s, t ::: 0, whence g(s + t) ::::: g(s) + get), s, t ::: O. For the last part

1 get) Pii (t) - 1 -(Pii(t)-I)=-. --+-A t t -log{l - (l - Pii (t))) as t ,j,. 0, since x /log(l - x) --+ -1 as x ,j,. O.

308

= O.

In

Solutions

Problems

[6.15.15]-[6.15.16]

15. Let i and j be distinct states, and suppose that Pij (f) > 0 for some f. Now 00

" 1 n (C n )ij, Pij(f)=L,f n=O

n.

implying that (Cn)ij > 0 for some n, which is to say that

for some sequence k l , k2, ... , kn of states. Suppose conversely that (*) holds for some sequence of states, and choose the minimal such value of n. Then i, k l , k2, ... , kn , j are distinct states, since otherwise n is not minimal. It follows that (Cn)ij > 0, while (Cm)ij = 0 for 0:::: m < n. Therefore 1 k k = f n" L k! f (C )ij 00

Pij(f)

k=n

is strictly positive for all sufficiently small positive values of t. Therefore i communicates with j.

16. (a) Suppose X is reversible, and let i and j be distinct states. Now

= i, X(f) = j) = lP'(X(t) = i, X(O) = j), = 7rj Pji (f). Divide by f and take the limit as

lP'(X(O)

which is to say that 7ri Pij (t) 7rigij

t -.l- 0 to obtain that

= 7rjgji·

Suppose now that the chain is uniform, and X (0) has distribution 1r. If t > 0, then

so that X(f) has distribution 1r also. Now let f < 0, and suppose that X(f) has distribution J,L. The distribution of X (s) for s :::: 0 is J,LP s- t = 1r, a polynomial identity in the variable s - f, valid for all s :::: O. Such an identity must be valid for all s, and particularly for s = f, implying that J,L = 1r . Suppose in addition that 7ri gij = 7rj gji for all i, j. For any sequence k l , k2, ... , k n of states, 7rigi,kJgk/,k2' ··gkn,j

= gkl,i7rklgkl,k2"

·gkn,j

= ... = gkl,igk2,kl

.. ·gj,kn7rj·

Sum this expression over all sequences k l , k2, ... , k n oflength n, to obtain 7ri(Cn+l)ij

It follows, by the fact that Pt

= 7rj(C n+ l )ji,

n:::: O.

= e tG , that

foralli,j,f. Forfl < f2 < ... < tn, lP'(X(tI)

= ii, X(f2) = i2,""

X(tn)

= in)

= 7ril Pil,iz (t2 - tl) ... Pin-l ,in (tn - f n- J}

= Pi2, il (t2 -

tJ}7ri2 Pi2,i3 (t3 - (2) ... Pin-l ,in (tn - t n- J}

= Pi2,i1 (f2 - tJ}··· Pin,in-l (tn - tn-I) 7r i n = lP'(Y(tJ}

= ii, Y(f2) = i2,"" 309

Y(tn)

= in),

= ...

[6.15.17]-[6.15.19]

Solutions

Markov chains

giving that the chain is reversible. (b) Let S = {I, 2} and

G

(-a a)

=

f3

-f3

where af3 > O. The chain is uniform with stationary distribution

and therefore Jqg12 = Jr2g21. (c) Let X be a birth-death process with birth rates Ai and death rates /-ii. The stationary distribution Jr satisfies

Therefore JrHI/-iHI

= JrkAk for k

:::: 0, the conditions for reversibility.

17. Consider the continuous-time chain with generator G

=

(-f3 y

f3). -y

It is a standard calculation (Exercise (6.9.1)) that the associated semigroup satisfies (f3

+ y)Pt

_ ( y +f3h(t) y(l - h(t))

-

f3(l - h(t))) f3 + yh(t)

where h(t) = e-t(fHy). Now PI = P if and only if y + f3h(l) = f3 + yh(l) to say that f3 = y = -1Iog(2a - 1), a solution which requires that a > 1.

18. The forward equations for Pn(t) poet)

p~(t)

= /-iPI = APn-l

= a(f3 + y), which is

= lP'(X(t) = n) are

APO, - (A

+ n/-i)Pn + /-i(n + I)Pn+l,

n::::

1.

In the usual way, aG at

= (s _

1) (AG _ /-i aG) as

with boundary condition G(s, 0) = sf. The characteristics are given by dt

ds = --,---,/-i(s - 1)

dG A(S - I)G'

and therefore G = eP(s-l) f ((s-l)e-/U ), for some function f, determined by the boundary condition to satisfy eP(s-l) f(s - 1) = sf. The claim follows. As t -+

00,

G(s, t) -+ eP(s-l), the generating function of the Poisson distribution, parameter p.

19. (a) The forward equations are

a

-pu(s,t) at

= -A(t)PU(s,t),

at Pij (s, t)

= -A(t)Pij (s, t) + A(t)Pi,j-l (t),

a

310

i < j.

Solutions

Problems

Assume N (s)

=i

[6.15.20]-[6.15.20]

and s < t. In the usual way, 00

G(s, t; x) = I>jlP'(N(t) = j

I N(s) = i)

j=i

satisfies

aG at =

).,(t) (x - I)G.

We integrate subject to the boundary condition to obtain G(s, t; x)

= xi exp { (x

whence Pij (t) is found to be the probability that A ).,(u) duo parameter The backward equations are

J:

a

as Pij(s, t)

- 1)

=j

it

).,(u) dU} ,

- i where A has the Poisson distribution with

= ).,(S)Pi+1.j(S, t) - ).,(S)Pij(S, t);

using the fact that Pi+1.j (t) = Pi,j -1 (t), we are led to

aG -- = as

).,(s)(x - I)G.

The solution is the same as above. (b) We have that lP'(T > t)

=

paa(t)

= exp {

-

fot ).,(u) du },

so that fret)

In the case ).,(t)

=

fot ).,(U)dU},

).,(t)exp { -

t 2: O.

= c/(l + t), E(T)

=

OO

foa

lP'(T> t)dt

= loOO a

du

(l

+ u)C

which is finite if and only if c > 1.

20. Let s > O. Each offer has probability 1 - F (s) of exceeding s, and therefore the first offer exceeding s is the Mth offer overall, where lP'(M = m) = F(s)m-1[1 - F(s)], m 2: 1. Conditional on {M = m}, the value of XM is independent of the values of Xl, X2, ... , XM-1, with lP'(XM > u

I M = m) =

1 - F(u)

1 - F(s)

,

0< s::S u,

and Xl, X2, ... , XM-1 have shared (conditional) distribution function F(u) G(u Is) = F(s)'

311

O::s u ::s S.

[6.15.21]-[6.15.21]

Solutions

Markov chains

For any event B defined in terms of Xl, X2, ... , X M _ [, we have that 00

JP'(XM > u, B)

=

L

JP'(XM > u, B [ M

= m)JP'(M = m)

m=l 00

=

L

JP'(XM > u [ M

= m)JP'(B

[M

= m)JP'(M = m)

m=l 00

= JP'(XM > u)

L

= m)JP'(M = m)

JP'(B [ M m=l = JP'(XM > u)JP'(B), 0<

s.:::: u,

where we have used the fact that JP'(XM > u [ M = m) is independent of m. It follows that the first record value exceeding s is independent of all record values not exceeding s. By a similar argument (or an iteration of the above) all record values exceeding s are independent of all record values not exceeding s. The chance of a record value in (s, s + h] is JP'(s < XM < s +h) -

=

F(s

+ h) -

F(s)

I-F(s)

=

f(s)h I-F(s)

+o(h).

A very similar argument works for the runners-up. Let XMl ' XM2' ... be the values, in order, of offers exceeding s. It may be seen that this sequence is independent of the sequence of offers not exceeding s, whence it follows that the sequence of runners-up is a non-homogeneous Poisson process. There is a runner-up in (s, s + h] if (neglecting terms of order o(h)) the first offer exceeding s is larger than s + h, and the second is in (s, s + h]. The probability of this is

(

21. Let Ft(x)

l-F(S+h)) (F(S+h)-F(S)) I-F(s) I-F(s)

= JP'(N*(t)

f(s)h I-F(s)

(h) +0.

.:::: x), and let A be the event that N has a arrival during (t, t

Ft+h(X)

= J...hJP'(N*(t +h).:::: x [A) + (l-J...h)Ft(x) +o(h)

where JP'(N*(t +h).:::: x

Hence

+ o(h ) =

~Ft(x) = at

[A) =

-J...Ft(x)

L:

Ft(x - y)f(y)dy.

+J...1°O Ft(x -

y)f(y)dy.

-00

Take Fourier transforms to find that 4>t(8) = E(e i8N * (t») satisfies

an equation which may be solved subject to 4>0(8)

= 1 to obtain 4>t(8) = e)..((¢(8)-1).

Alternatively, using conditional expectation,

where N (t) is Poisson with parameter At.

312

+ h). Then

Solutions [6.15.22]-[6.15.25]

Problems

22. We have that E(SN(t))

whence E(N(t)) = 1(Al

= E{E(SN(t)

=

[ A)}

+ A2)t and var(N(t))

+ eA2t (s-I)},

1{e Ajt (s-l)

= 1(Al

+ A2)t + i(AI

- A2)2t 2 .

23. Conditional on {X (t) = i}, the next arrival in the birth process takes place at rate Ai.

= IP'(X(t) = n) are

24. The forward equations for Pn(t) , pn(t)

=

1 + f-L(n - 1) 1 + f-Ln 1 + f-Lt Pn-l (t) - 1 + f-Lt Pn(t),

n 2: 0,

with the convention that P-l (t) = O. Multiply by sn and sum to deduce that

aG

aG

2aG

- G - f-LSas as

(1 +f-Lt)- =sG+ f-LS at

as required. Differentiate with respect to s and take the limit as s met)

t

1. If E(X (t)2) < 00, then

= E(X(t)) = -aGI as s=1

satisfies (1 + f-Lt)m'(t) = 1 + f-Lm(t) subject to m(O) = I. Solving this in the usual way, we obtain m(t) = 1 + (1 + f-LI)t. Differentiate again to find that 2

net)

GI

a2 = E(X(t)(X(t) -1)) = as

satisfies (1

+ f-Lt)n'(t) = 2(m(t) + f-Lm(t) + f-Ln(t)) net) = l ( l - 1)

The variance of X(t) is net)

s=1

subjectton(O)

= l(l-

1). The solution is

+ 2/(1 + f-LI)t + (1 + f-LI)(1 + f-L + f-L/)t 2 .

+ met) -

m(t)2.

25. (a) Condition on the value of the first step:

as required. Set Xi = 1)i+l - 1)i to obtain AjXj = f-LjXj -1 for j 2: 1, so that j

Xj

i=1

It follows that

W

=xoTI A~' I

j

1)j+l = 1)0

+L

j

Xk = 1

k=O

The 1)j are probabilities, and lie in [0, 1]. If that 1)j = 1 for all j.

+ (1)1

-

1)

L ek· k=O

Lf ek = 00 then we must have 1)1 = 1, which implies 313

[6.15.26]-[6.15.28]

Solutions

Markov chains

(b) By conditioning on the first step, the probability TJj, of visiting 0 having started from j, satisfies (j TJj

=

+ 1)2TJj+1 + PTJj-1 j2 + (j + 1)2

j

1 - TJ'+1 J

= (1

- TJ1)

~ (k + 1)2 -+ (1 k=O 1

1

TJ1)-n 6

2

as j -+

00.

By Exercise (6.3.6), we seek the minimal non-negative solution, which is achieved when TJ1 (6/n 2 ).

= 1-

26. We may suppose that X(O) = O. Let Tn = inf{t : X(t) = n}. Suppose Tn = T, and let Y = Tn+1 - T. Condition on all possible occurrences during the interval (T, T + h) to find that E(Y)

= (Anh)h + Mnh(h + E(Y'» + (1 -

Anh - Mnh)(h

where y' is the mean time which elapses before reaching n to obtain that mn

= Mnh(mn-1 + m n ) + mn + h{1 -

+ 1 from n -

1. Set mn

= E(Tn+1

- Tn)

+ Mn)m n } + o(h). 1 + Mnmn-1, n ~ 1. Therefore

(An

Divide by h and take the limit as h ..} 0 to find that Anmn = mn

+ E(Y» + o(h),

... M1 = -1 + -Mnm n -1 = ... = -1 + - Mn - - + ... + MnMn-1 , A An

An

An

AnAn-1

An n-1 .. 'AO

since mo = A01 The process is dishonest if L:~o mn < 00, since in this case Too = lim Tn has finite mean, so that !P(Too < 00) = 1. On the other hand, the process grows no faster than a birth process with birth rates Ai, which is honest if L:~o I/A n = 00. Can you find a better condition?

27. We know that, conditional on X(O)

= /,

G(s, t)

=

X(t) has generating function

)..t(1-S)+S)1 ' ( )..t(1 _ s) + 1

so that !P(T :::: x

I X(O) = /) = !P(X(x) = 0 I X(O) = I) = G(O, x) =

It follows that, in the limit as x -+ !P(T :::: x)

Cx~ 1

r

00,

~ ( -AX- )1 !P(X(O) = I) = GX(O) =~ 1=0 Ax + 1

For the final part, the required probability is {x / / (x / e- 1/ x as / -+ 00.

( -AX- ) -+ 1. Ax + 1

+ 1)}1 = {I + (x I) -1 }-I , which tends to

28. Let Y be an imrnigration-death process without disasters, with Y (0) = O. We have from Problem (6.15.18) that yet) has generating function G(s, t) = exp{p(s - 1)(1- e- JLt )} where p = AIM. As seen earlier, and as easily verified by taking the limit as t -+ 00, Y has a stationary distribution.

314

Solutions

Problems

[6.15.29]-[6.15.31]

From the process Y we may generate the process X in the following way. At the epoch of each disaster, we paint every member of the population grey. At any given time, the unpainted individuals constitute X, and the aggregate population constitutes Y. When constructed in this way, it is the case that yet) :s X(t), so that Y is a Markov chain which is dominated by a chain having a stationary distribution. It follows that X has a stationary distribution n (the state 0 is persistent for X, and therefore persistent for Yalso). Suppose X is in equilibrium. The times of disasters form a Poisson process with intensity 8. At any given time t, the elapsed time T since the last disaster is exponentially distributed with parameter 8. At the time of this disaster, the value of X (t) is reduced to 0 whatever its previous value. It follows by averaging over the value of T that the generating function H(s) = L:~o snnn of X(t) is given by H(s) =

by the substitution x

oo 8e- 8u G(s, u)du = _e 8 P (s-l)

loo

1o1 x(8/1L)-l e-p(s-l)x dx

/.L

0

= e- lLu . The mean of X (t) is

29. Let G(IBI, s) be the generating function of X(B). If Bnc = 0, then X(BUC) = X(B) +X(C), so that G(a+{3, s) = G(a, s)G({3, s)for lsi :s I, a, {3 2: O. The only solutions to this equation which are monotone in a are of the form G(a, s) = eo!A(S) for lsi :s I, and for some function A(s). Now any interval may be divided into n equal sub-intervals, and therefore G(a, s) is the generating function of an infinitely divisible distribution. Using the result of Problem (S.12.13b), A(s) may be written in the form A(s) = (A(s) -1).1. for some A and some probability generating function A(s) = L:(f ais i . We now use (iii); if IBI = a, lP'(X(B) 2: I) lP'(X(B) = 1) as a .} O. Therefore ao + al = I, and hence A(s) distribution with parameter proportional to IB I.

=

ao

+ (1

- ao)s, and X(B) has a Poisson

30. (a) Let M (r, s) be the number of points of the resulting process on lR+ lying in the interval (r, s]. Since disjoint intervals correspond to disjoint annuli of the plane, the process M has independent increments in the sense that M(rl, SI), M(r2, S2), ... , M(rn , sn) are independent whenever rl < SI < r2 < ... < rn < Sn. Furthermore, for r < s and k 2: 0, {An(s - r)}k e-AJl"(s-r)

lP' (M (r, s) = k) = lP' (N has k points in the corresponding annulus) =

k!

.

(b) We have similarly that

:s x) = lP'(N has least k points in circle of radius x) = L 00

lP'(R(k)

(An x2y e- AJl"X

r= k

I

r.

2 '

and the claim follows by differentiating, and utilizing the successive cancellation. 31. The number XeS) of points within the sphere with volume S and centre at the origin has the Poisson distribution with parameter AS. Hence lP'(X (S) = 0) = e- AS , implying that the volume Vof the largest such empty ball has the exponential distribution with parameter A.

315

[6.15.32]-[6.15.33]

Solutions

Markov chains

It follows that JP'(R > r) = JP'(V > crn) ball in n dimensions. Therefore

=

e- Acrn for r :::: 0, where c is the volume of the unit

r :::: O. Finally, E(R)

= Jooo e- Acrn dr, and we set v = Acr n .

32. The time between the kth and (k

+ l)th infection has mean Akl, whence N

1

E(T)=~-. k=l Ak

Now N i l {N 1 N 1 } = - ~-+~-keN + 1 - k) N + 1 k=l k k=l N + 1 - k

~

k=l

2 N 1 2 = - - ~ - = --{logN +y +O(N- 1 )}. N + 1 k=l k N +1 It may be shown with more work (as in the solution to Problem (5.12.34» that the moment generating function of A(N + I)T - 2 log N converges as N --+ 00, the limit being {r(l - 8)}2.

33. (a) The forward equations for Pn(t) = JP'(V(t) = n

+ ~) are n:::: 0,

with the convention that P-l (t) = O. It follows as usual that

-aG = -aG at

(aG 2s as

as

+ G) +

(2 aG + s as

sG )

as required. The general solution is

G(s, for some function (b) Clearly

t) = _1_f (t + _1_) l-s l-s

f. The boundary condition is G(s, 0)

= 1, and the solution is as given.

T mn(T)=E(lo Intdt) = loT E(lnt)dt by Fubini's theorem, where Int is the indicator function of the event that V (t) = n + ~. As for the second part,

~

Lsnmn(T)

= loT G(s,t)dt=

n=O

so that, in the limit as T

10g[1

o

1- s

s)T]

,

--+ 00,

~ 1 (1+(l-S)T) Lsn(mn(T) -logT) = --log n=O

+ (l -

l-s

T

316

--+

10g(l-s) l-s

=- ~ Lsnan n=l

Solutions

Problems

if lsi < 1, where an

[6.15.34]-[6.15.35]

= L:7=1 i-I, as required.

(c) The mean velocity at time t is

1 -+ -aG I 2

as

s=1

1 =t+-.

2

~'-----1---""

- - - - .

~---I~--.I.<::.--------'-<

- - - - .

(2, 1) (0, 1)

(1,3)

(1, 1)

(3,1)

""""'--o4_--,.=--.-----,.. - - - - .

(1,0)

(1,2)

"""'=----------..,.. - - - - . (2,3)

34. It is clear that Y is a Markov chain, and its possible transitions are illustrated in the above diagram. Let x and y be the probabilities of ever reaching (1, 1) from (1,2) and (2, 1), respectively. By conditioning on the first step and using the translational symmetry, we see that x = y + x 2

1

1(3

1 1

and y = + 1xy, Hence x 3 - 4x 2 + 4x - 1 = 0, an equation with roots x = 1, ± -v's). Since x is a probability, it must be that either x = 1 or x = (3 - 0), with the corresponding values of y = 1 and y = 1(0 - 1). Starting from any state to the right of (1,1) in the above diagram, we see by recursion that the chance of ever visiting (1, 1) is of the form x Cl yf3 for some non-negative integers ex, {3. The minimal non-negative solution is therefore achieved when x = (3 - 0) and y

1

= 1(0 -

1

1). Since x < 1, the chain is transient.

35. We write A, 1, 2, 3, 4, 5 for the vertices of the hexagon in clockwise order. Let Ti 1: Xn = i} and lP'iO = IP'(. I Xo = i). (a) By symmetry, the probabilities Pi

whence PA

= lP'i(h

=

min {n :::

< Tc) satisfy

= 17' ~ for i i=- C, whence

(b) By Exercise (6.4.6), the stationary distribution is ne = n -1 = 8.

/-LA

=

A

(c) By the argument leading to Lemma (6.4.5), this equals /-LAne (d) We condition on the event E bi = lP'i (E) satisfy

= {h

= 2.

< Tc} as in the solution to Exercise (6.2.7). The probabilities

317

[6.15.36]-[6.15.37]

Solutions

ft'

yielding bl = b2 equations of the fonn

' 'I arIY '21 an d SlIm

= j;, b3 = ~,

The transition probabilities conditional on E are now found by

H ence,

7 '23 = 9' 2 '32 = 2' I 'IA = 6j ' = 9'

f.L2A

giving f.LIA

Markov chains

= 1 + ~f.LIA + ~f.L3A'

f.L3A

WI'th

= I + f.L2A,

theob' , VIOUS notatIOn, f.LIA

= I + jf.L2A,

= .!p, and the required answer is 1 + f.LIA = I +.!p = ¥,

36. (a) We have that Pi,i+1

=

f3(m - i)2

Look for a solution to the detailed balance equations

to find the stationary distribution

(b) In this case, Pi,i+1

=

f3(m - i) Pi+I,i

m Look for a solution to the detailed balance equations f3(m - i) 7ri

m

= 7ri+1

/X(i

=

/X(i

+ I) m

+ 1) m

yielding the stationary distribution

37. We have that

by the Chapman-Kolmogorov equations

by the concavity of c

318

Solutions [6.15.38]-[6.15.41]

Problems

where we have used the fact that Lj JrjPjk(t) = Jrk. Now aj(s) ---+ Jrj as

S

---+ 00, and therefore

d(t) ---+ c(l).

38. By the Chapman-Kolmogorov equations and the reversibility, uO(2t) = LlP'(X(2t) = 0 I X(t) = j)lP'(X(t) = j

I X(O) = 0)

j

= "JrO L ----:-lP'(X(2t) = j

j I X(t) = O)Uj(t) = JrO "LJrj

JrJ

(u

.(t))2 JrJ

_J_.

j

The function c(x) = _x 2 is concave, and the claim follows by the result of the previous problem. 39. This may be done in a variety of ways, by breaking up the distribution of a typical displacement and using the superposition theorem (6.13.5), by the colouring theorem (6.13.14), or by Renyi's theorem (6.13.17) as follows. Let B be a closed bounded region of ]Rd. We colour a point of IT at x E ]Rd black with probability lP'(x + X E B), where X is a typical displacement. By the colouring theorem, the number of black points has a Poisson distribution with parameter

r

l'ft1.d

AlP'(x

+ X E B) dx =

Ai

dy

YEB

=A1 YEB

r

1XE'ftI.d

dy1

VE'ftI.d

lP'(X E dy - x) lP'(XEdv)=AIBI,

by the change of variables v = y - x. Therefore the probability that no displaced point lies in B is e- A1B1 , and the claim follows by Renyi's theorem. 40. Conditional on the number N (s) of points originally in the interval (0, s), the positions of these points are jointly distributed as uniform random variables, so the mean number of these points which lie in (-00, a) after the perturbation satisfies AS

r ~lP'(X + U :s a) du ---+ A roo FX(a s 10

10

u)du = E(RL)

where X is a typical displacement. Likewise, E(LR) = A if and only if

1

00

[I - Fx(v)] dv

= [a

Jooo [1 -

oo

Fx(a

as s ---+ 00,

+ u)] duo

Equality is valid

Fx(v) dv,

which is equivalent to a = E(X), by Exercise (4.3.5). The last part follows immediately on setting Xr = Vrt, where Vr is the velocity of the rth car.

41. Conditional on the number N (t) of arrivals by time t, the arrival times of these ants are distributed as independent random variables with the uniform distribution. Let U be a typical arrival time, so that U is uniformly distributed on (0, t). The arriving ant is in the pantry at time t with probability Jr = lP'(U + X > t), or in the sink with probability p = lP'(U + X < t < U + X + Y), or departed with probability I - p - Jr. Thus, E(xA(t)yB(t))

= E{E(xA(t)yB(t) I N(t))} = E{ (Jrx + py + I - Jr - p)N(t)} = eArr(x-l) eAp(y-l).

Thus A(t) and B(t) are independent Poisson-distributed random variables. If the ants arrive in pairs and then separate,

319

[6.15.42]-[6.15.45]

Solutions

=

p. Hence,

where y

I-

lr -

Markov chains

whence A(t) and B(t) are not independent in this case.

42. The sequence {X r } generates a Poisson process N(t) = max{n : Sn .::: t}. The statement that Sn = t is equivalent to saying that there are n - I arrivals in (0, t), and in addition an arrival at t. By Theorem (6.12.7) or Theorem (6.13.11), the first n - I arrival times have the required distribution. Part (b) follows similarly, on noting that fu(u) depends on u = (UI, U2, ... , un) only through the constraints on the Ur. 43. Let Y be a Markov chain independent of X, having the same transition matrix and such that Yo has the stationary distribution Jr. Let T = min{n :::: I : Xn = Yn } and suppose Xo = i. As in the proof of Theorem (6.4.17), IPij (n) - lrj I =

IL

lrk(Pij (n) - Pkj (n))

I.:::

k

L

lrklP'(T > n)

= lP'(T

> n).

k

Now, lP'(T > r

where €

= minij {Pij } > O.

+I

IT> r) .::: I - €2

for r :::: 0,

The claim follows with A = I - €2.

44. Let hen) be the indicator function ofa visit to k at time n, so thatE(h(n)) say. By Problem (6.15.43), lak(n) - lrkl .::: An. Now,

Let s

= min{m, r} and t = 1m -

= nl2

= lP'(Xn = k) = ak(n),

rl. The last summation equals

L ,

L { (ai (s) - lri)(Pii (t) - lri) m

+ lri (pu (t)

- lri)

+ lri(ai(s) - lri) - lri(ai(r) - lri) - lri(ai(m) - lri )}

I

.::: 2" LL(A s + t +A t +A s +A r +Am) n

r

m

as n -+

00,

where 0 < A < 00. For the last part, use the fact that I:~,;:;-J f(X,) = I:iES fCi)Vi(n). The result is obtained by Minkowski's inequality (Problem (4.14.27b)) and the first part.

45. We have by the Markov property that f(Xn+l I X n , Xn-l, ... , Xo) lE(log f(X n+ 1 I X n , Xn-l, ... ,

= f(Xn+l

I Xn), whence

Xo) I X n , ... , Xo) = lE(log f(Xn+l I Xn) I Xn). 320

Solutions [6.15.46]-[6.15.48]

Problems

Taking the expectation of each side gives the result. Furthermore, H(X n+l I Xn)

=-

Now X has a unique stationary distribution is finite, and the claim follows.

1C,

'2)Pij log Pij )lP'(Xn i,j

so that lP'(Xn

= i)

= i).

-+ lri as n -+ 00. The state space

46. Let T = inf{t : X t = Yd. Since X and Y are persistent, and since each process moves by distance 1 at continuously distributed times, it is the case that lP'(T < 00) = 1. We define if t < T,

Xt Zt= { Yt

ift::: T,

noting that the processes X and Z have the same distributions. (a) By the above remarks, lP'(Yt = k)1 = 1lP'(Zt = k) - lP'(Yt = k)1 = k, T :::: t) + lP'(Zt = k, T > t) -lP'(Yt = k, :::: lP'(X t = k, T > t) + lP'(Yt = k, T > t).

1lP'(Xt

= k) -

:::: 1lP'(Zt

T :::: t) -lP'(Yt

= k,

T >

t)1

We sum over k E A, and let t -+ 00. (b) We have in this case that Zt :::: Yt for all t. The claim follows from the fact that X and Z are processes with the same distributions. 47. We reformulate the problem in the following way. Suppose there are two containers, Wand N, containing n particles in all. During the time interval (t, t + dt), any particle in W moves to N with probability Mdt +o(dt), and any particle in N moves to W with probability Adt +o(dt). The particles move independently of one another. The number Z (t) of particles in W has the same rules of evolution as the process X in the original problem. Now, Z (t) may be expressed as the sum of two independent random variables U and V, where U is bin(r, et ), V is bin(n - r, 1/It), and et is the probability that a particle starting in W is in W at time t, 1/It is the probability that a particle starting in N at 0 is in W at t. By considering the two-state Markov chain of Exercise (6.9.1), )" _ Ae-(A+JL)t 1/It =

A+ M

and therefore

= E(sU)E(s V) = (set + 1 - s)r (s1/lt + 1 _ s)n-r. Also, E(X(t)) = ret + (n - r)1/It and var(X(t)) = ret (1- et ) + (n - r)1/It(1-1/It). n -+ 00, the distribution of X(t) approaches the bin(n, Aj()" + M)) distribution. E(sX(t))

In the limit as

48. Solving the equations

gives the first claim. We have that y = 2::i (Pi - qi )lri, and the formula for y follows. Considering the three walks in order, we have that: A. lri =

1for each i, and YA =

-2a < O.

B. Substitution in the fdrmula for JIB gives the numerator as 3 { - !6a + o(a) }, which is negative for small a whereas the denominator is positive.

321

[6.15.49]-[6.15.51]

Solutions

Markov chains

C. The transition probabilities are the averages of those for A and B, namely, Po

i(i -

rb -

a) + a) = a, and so on. The numerator in the formula for YC equals which is positive for small a.

=

i (fa -

-rfu +0(1),

49. Call a car green if it satisfies the given condition. The chance that a green car arrives on the scene during the time interval (u, u + h) is )"hlP'(V < x/(t - u)) for u < t. Therefore, the arrival process of green cars is an inhomogeneous Poisson process with rate function )..(u)

={

AlP'(V < x/(t - u))

ifu < t,

o

ifu2:t.

Hence the required number has the Poisson distribution with mean

).. fot lP' (V < t ~

J

du

=)..

=)..

fot lP' (V < ~) du fot E(l/vu<xj) du

=

)"E(V- 1 min{x, VtJ).

50. The answer is the probability of exactly one arrival in the interval (s, t), which equals g(s) = s)e-).(t-s). By differentiation, g has its maximum atS = max{O, t - )..-1), and g(s) = e- 1 when t 2: ).. -1 . )..(t -

51. We measure money in millions and time in hours. The number of available houses has the Poisson distribution with parameter 30)", whence the number A of affordable houses has the Poisson distribution with parameter ·30)" = 5).. (cf. Exercise (3.5.2)). Since each viewing time T has moment

t

generating function E(efJT)

= (e 2fJ -

e fJ )/(), the answer is

322

7 Convergence of random variables

7.1 Solutions. Introduction 1. (a) EI(eX),1 = lei' . {lIXllrl'. (b) This is Minkowski's inequality. (c) Let E > O. Certainly IXI ~ IE where IE is the indicator function of the event {IXI > d. Hence EIX' I ~ Ell; I = lP'(IXI > E), implying that lP'(IXI > E) = 0 for all E > O. The converse is trivial.

+ bY}Z) = aE(X Z) + bE(Y Z). (b) E({X + Y}2) + E({X - Y}2) = 2E(X2) + 2E(y 2 ). 2.

(a) E({aX

(c) Clearly

3. Let feu) dE(f, h) = 1.

=

~E, g(u)

=

0, h(u)

=

-~E, for all u. Then dE(f, g)

+ ddg, h) =

0 whereas

~

Either argue directly, or as follows. With any distribution function F, we may associate a graph F obtained by adding to the graph of F vertical line segments connecting the two endpoints at each discontinuity of F. By drawing a picture, you may see that .J2d(F, G) equals the maximum distance between F and G measured along lines of slope -1. It is now clear that d(F, G) = 0 if and only if F = G, and that d(F, G) = d(G, F). Finally, by the triangle inequality for real numbers, we have that d(F, H) .::: d(F, G) + d(G, H).

5.

Take X to be any random variable satisfying E(X2)

= 00, and define Xn = X for all n.

7.2 Solutions. Modes of convergence 1.

(a) By Minkowski's inequality,

let n --+

00

to obtain lim infn-+oo EIX~ I ~ EIX' I. By another application of Minkowski's inequality,

323

[7.2.2]-[7.2.4]

Solutions

Convergence of random variables

(b) We have that IE(X n ) - E(X)I

=

IE(Xn - X)I ::: EIX n - XI --+ 0

as n --+ 00. The converse is clearly false. If each Xn takes the values ±1, each with probability then E(Xn) = 0, but EIX n - 01 = 1.

i,

(c) By part (a), E(X~) --+ E(X 2 ). Now Xn X by Theorem (7.2.3), and therefore E(Xn ) --+ E(X) by part (b). Therefore var(Xn ) = E(X~) - E(Xn)2 --+ var(X).

--l

2. Zn

Assume that Xn ~ X. Since IXnl ::: Z for all n, it is the case that IXI ::: Z a.s. Therefore IX n - XI satisfies Zn ::: 2Z a.s. In addition, if E > 0,

=

As n --+ 00, lP' (IZn I > E) --+ 0, and therefore the last term tends to 0; to see this, use the fact that E(Z) < 00, together with the result of Exercise (5.6.5). Now let E tOto obtain that EIZn I --+ 0 as n --+ 00.

3. We have that X - n- 1 ::: Xn ::: X, so that E(Xn ) --+ E(X), and similarly E(Yn ) --+ E(Y). By the independence of Xn and Yn ,

E as n --+

00,

{( 1) ( 1) }= X - ;;-

Y - ;;-

E(XY) -

E(X)

I

+ E(Y) n + n2

--+ E(Xy)

so that E(Xn Yn ) --+ E(Xy).

4. Let Fl, F2, ... be distribution functions. As in Section 5.9, we write Fn --+ F if Fn (x) --+ F(x) for all x at which F is continuous. We are required to prove that Fn --+ F if and only if d (Fn, F) --+ O. Suppose that d(Fn, F) --+ O. Then, for E > 0, there exists N such that F(x - E) - E ::: Fn(x) ::: F(x

+ E) + E

for all x.

Take the limits as n --+ 00 and E --+ 0 in that order, to find that Fn (x) --+ F (x) whenever F is continuous at x. Suppose that Fn --+ F. Let E > 0, and find real numbers a = Xl < X2 < ... < Xn = b, each being points of continuity of F, such that (i) Fi(a) < E for all i, F(b) > 1 - E, (ii) IXi+1 - xi I <

E

for I ::: i < n.

In order to pick a such that Fi (a) <

Efor all i, first choose a ' such that F(a ' ) iE

<

iE and F is

continuous at ai, then find M such that !Pm (a ' ) - F(a ' ) I < for m ::: M, and lastly find a continuity point a of F such that a ::: a ' and Fm(a) < E for 1::: m < M. There are finitely many points Xi, and therefore there exists N such that !Pm (Xi) - F (Xi) I < E for all i and m ::: N. Now, ifm ::: N and Xi ::: x < Xi+l,

and similarly Fm(x) ::: Fm(Xi) > F(Xi) - E ::: F(x - E) - E.

Similar inequalities hold if x ::: a or x ::: b, and it follows that d(Fm, F) < d(Fm, F) --+ 0 as m --+ 00.

324

E

if m ::: N. Therefore

Modes of convergence

Solutions

[7.2.5]-[7.2.7]

5. (a) Suppose c > 0 and pick 8 such that 0 < 8 < c. Find N such that lP'(IYn - cl > 8) < 8 for n ::: N. Now, for x ::: 0, lP'(XnYn

~ x) ~ lP'(XnYn ~ x, IYn -

cl

~ 8) + lP'(IYn -

cl > 8)

~ lP' ( Xn ~

c : 8)

+ 8,

and similarly lP'(XnYn > x)

~ lP'(XnYn

> x, IYn - cl

~ 8) + 8 ~ lP' ( Xn >

c : 8)

+ 8.

Taking the limits as n --+ 00 and 8 -!- 0, we find that lP'(XnYn ~ x) --+ lP'(X ~ x/c) if x/c is a point of continuity of the distribution function of X. A similar argument holds if x < 0, and we conclude that XnYn

~

cX if c > O. No extra difficulty arises if c < 0, and the case c

& c- 1 if Yn & c For the second part, it suffices to prove that from the fact that Iy,;-l - c-11 < E/{Icl(lcl - E») if IYn - cl < E « Icl). (b) Let E > O. There exists N such that yn- 1

lP'(IXnl > E) < E,

lP'(IYn - YI > E) < E,

= 0 is similar.

(#

0). This is immediate

ifn::: N,

and in addition lP'(IYI > N) < E. By an elementary argument, g is uniformly continuous at points of the form (0, y) for Iyl ~ N. Therefore there exists 8 (> 0) such that Ig(x', y') - g(O, y)1 <

If IXnl

~

8, IYn - YI

~

~

8, and IYI

if Ix'i ~ 8, Iy' - yl ~ 8.

E

N, then Ig(Xn , Yn ) - g(O, y)1 < E, so that

lP'(lg(Xn , Yn ) - g(O, y)1 ::: E) ~ lP'(IXnl > 8)

+ lP'(IYn -

YI > 8)

+ lP'(IYI

> N) ~ 3E,

p for n ::: N. Therefore g(Xn , Yn ) --* g(O, Y) as n --+ 00.

6.

The subset A of the sample space Q may be expressed thus:

nun 00

A

=

00

00

{IXn+m

1

Xnl < k- },

-

k=l n=l m=l a countable sequence of intersections and unions of events. For the last part, define X(w)

={

The function X is .'F-measurable since A

limn->oo Xn(w)

ifw E A

o

ifw

E

1- A.

.'F.

7. (a) If Xn(w) --+ X(w) then cnXn(w) --+ cX(w). (b) We have by Minkowski's inequality that, as n --+ 00, lE(lcnX n -cXn ~ IcnlTlE(IXn - Xn

+ ICn

_cITlEIXTI--+ O.

(c) If c = 0, the claim is nearly obvious. Otherwise c # 0, and we may assume that c > O. For E < c, there exists N such that ICn - cl < E whenever n ::: N. By the triangle inequality, IcnXn - cXI ~ Icn(Xn - X)I + I(cn - c)XI, so that, for n ::: N,

o<

lP'(lcnXn - cXI > E) ~ lP'(cnIXn - XI> iE)

~ lP' (IXn --+ 0

XI >

as n --+

2(c

00.

325

+ lP'(lc n -

cl·IXI > iE)

+ E) ) + lP' (IXI

E

>

E ) 21cn - cl

[7.2.8]-[7.3.1]

Solutions

Convergence of random variables

(d) A neat way is to use the Skorokhod representation (7.2.14). If Xn

~

X. find random variables

Yn • Y with the same distributions such that Yn ~ Y. Then cnYn ~ cY. so that cnYn ~ cY. implying the same conclusion for the X's.

8.

If X is not a.s. constant, there exist real numbers c and I' such that 0 <

lP'(X > c

+ E)

> 21'. Since Xn

~

I'

< ~ and lP'(X < c) > 21',

X, there exists N such that

lP'(Xn > c + E) > 1',

lP'(Xn < c) > 1',

if n 2: N.

Also, by the triangle inequality, IX r - Xsi :::: IX r - XI + IXs - XI; therefore there exists M such that lP'(IX r - Xs I > E) < 1'3 for r, s 2: M. Assume now that the Xn are independent. Then, for r, s 2: max{M, N), r =I- s, 1'3 > lP'(IXr - Xs I >

E)

2: lP'(Xr < c, Xs > c

+ E) = lP'(X r

< c)lP'(Xs > c

+ E)

> 1'2,

a contradiction.

9. Either use the fact (Exercise (4.12.3)) that convergence in total variation implies convergence in distribution, together with Theorem (7.2.19), or argue directly thus. Since luOI :::: K < 00,

[E(u(X n )) - E(u(X))[ =

II: u(k){fn(k) -

f(k))1 :::: K

I: Ifn(k) -

k

f(k)1 -*

o.

k

10. The partial sum Sn = ~~=l Xr is Poisson-distributed with parameter an = ~~=l Ar. For fixed x, the event {Sn :::: x) is decreasing in n, whence by Lemma (1.3.5), if an -* a < 00 and x is a non-negative integer,

00)

lP' ( "Xr::::x ~ r=1

Hence if a <

00, ~~l

then e- CYn ~J=o a1 /j!

-CY

x

j

= n-+oo lim lP'(Sn::::X)="~. ~ i' j=O

.

Xr converges to a Poisson random variable. On the other hand, if an -*

-* 0, giving that diverges with probability 1, as required.

lP'(~~1 Xr

>

x) =

1 for all

x,

00,

and therefore the sum

7.3 Solutions. Some ancillary results 1.

(a) If IX n - Xml >

I'

then either IX n - XI > ~E or IX m - XI >

iE, so that

as n, m -* 00, for I' > o. Conversely, suppose that {X n ) is Cauchy convergent in probability. For each positive integer k, there exists nk such that for n, m 2: nk. The sequence (nk) may not be increasing, and we work instead with the sequence defined by Nl = nl, Nk+1 = max{Nk + 1, nk+ll. We have that

I:lP'(IXNk+l - XNk 12: Z-k) < k

326

00,

Solutions

Some ancillary results

[7.3.2]-[7.3.3]

whence, by the first Borel-Cantelli lemma, a.s. only finitely many of the events {[ X Nk+ 1 2- k } occur. Therefore, the expression

-

X Nk [ 2:.

00

X

= XNl + I)XNk+l

- XNk)

k=l

converges absolutely on an event C having probability one. Define X(w) accordingly for WE C, and X(w) = 0 for w rf. C. We have, by the definition of X, that XNk ~ X as k -+ 00. Finally, we 'fill in the gaps'. As before, for E > 0,

as n, k -+

00,

(b) Since Xn

where we are using the assumption that {Xn} is Cauchy convergent in probability.

!.. X, the sequence {X n } is Cauchy convergent in probability. Hence lP'([Yn - Ym [ >

for

E

= lP'([Xn

- Xm[ >

E)

-+ 0

as n, m -+

00,

> O. Therefore {Yn } is Cauchy convergent also, and the sequence converges in probability to

some limit Y. Finally, Xn

2.

E)

.s X and .s X, so that X and Yn

Y have the same distribution.

Since An <; U~=n Am, we have that

limsuplP'(An)::::: lim lP'( n---+oo

n---+oo

U

m=n

Am) = lP'( lim

U

n---+oo m=n

Am) = lP'(An Lo.),

where we have used the continuity of lP'. Alternatively, apply Fatou's lemma to the sequence indicator functions. 3. (a) Suppose X2n = 1, X2n+l = -1, for n 2:. 1. Then {Sn = 0 Lo.} occurs if Xl if Xl = 1. The event is therefore not in the tail a-field of the X's. (b) Here is a way. As usual, lP'(S2n = 0) = enn)!p(1 - pW, so that LlP'(S2n

= 0)

< 00

if p =I-

=

I.4~

of

-1, and not

i,

n

implying by the first Borel-Cantelli lemma that lP'(Sn = 0 i.o.) = O. (c) Changing the values of any finite collection of the steps has no effect on I = lim inf Tn and J = lim sup Tn, since such changes are extinguished in the limit by the denominator '-/fi'. Hence I and J are tail functions, and are measurable with respect to the tail a -field. In particular, {I ::::: -x} n {J 2:. x} lies in the a -field. Take x = 1, say. Then, lP'(I ::::: -1) = lP'(J 2:. 1) by symmetry; using Exercise (7.3.2) and the central limit theorem, lP'(J 2:. 1) 2:.lP'(Sn 2:. -/fi Lo.) 2:.limsuplP'(Sn 2:. -/fi) n---+oo

= 1-

<.p(1) > 0,

where <.P is the N (0, 1) distribution function. Since {J 2:. I} is a tail event of an independent sequence, it has probability either 0 or 1, and therefore lP'(I ::::: -1) = lP'(J 2:. 1) = 1, and also lP'(I ::::: -1, J 2:. 1) = 1. That is, on an event having probability one, each visit of the walk to the left of --/fi is followed by a visit of the walk to the right of -/fi, and vice versa. It follows that the walk visits 0 infinitely often, with probability one.

327

[7.3.4]-[7.3.8]

Solutions

Convergence of random variables

4. Let A be exchangeable. Since A is defined in terms of the Xi, it follows by a standard result of measure theory that, foreachn, there exists an event An E a(XI, X2, ... , Xn), such that lP'(A,0, An) -+ o as n -+ 00. We may express An and A in the form An

= {Xn

E Bn},

A

= {X E

B},

where Xn = (Xl, X2, ... , Xn), and Bn and B are appropriate subsets oflR. n and lR. oo . Let A~

=

{X~ E Bn},

A'

=

{X' E B},

where X~

= (Xn+l, X n+2, ... , X2n) and X' = (Xn+l, X n+2, ... , X2n, Xl, X2,···, X n , X2n+l, X2n+2, ... ). Now lP'(An n A~) = lP'(An)lP'(A~), by independence. Also, lP'(An) = lP'(A~), and therefore

By the exchangeability of A, we have that lP'(A ,0, A~) = lP'(A' ,0, A~), which in turn equals lP'(A,0, An), using the fact that the Xi are independent and identically distributed. Therefore, [lP'(An

n A~)

-lP'(A)[ ::::: lP'(A,0, An)

Combining this with (*), we obtain that lP'(A)

+ lP'(A,0, A~)

-+ 0

as n -+

00.

= lP'(A)2, and hence lP'(A) equals 0 or 1.

5. The value of Sn does not depend on the order of the first n steps, but only on their sum. If Sn = 0 Lo., then S~ = 0 i.o. for all walks {S~} obtained from {Sn} by permutations of finitely many steps. 6. Since f is continuous on a closed interval, it is bounded: [f (y) [ ::::: c for all y E [0, 1] for some c. Furthermore f is uniformly continuous on [0, 1], which is to say that, if E > 0, there exists 8 (> 0), such that [fey) - f(z)[ < E if [y - z[ ::::: 8. With this choice of E, 8, we have that [E(ZIAc)[ < E, and [E(ZIA)[ ::::: 2clP'(A) ::::: 2c·

x(l - x) n8 2

by Chebyshov's inequality. Therefore [E(Z)[ <

2c

E

+ -2' n8

which is less than 2E for values of n exceeding 2C/(E8 2 ). 7. If {Xn} converges completely to X then, by the first Borel-Cantelli lemma, [Xn - X[ > E only finitely often with probability one, for all E > O. This implies that Xn ~ X; see Theorem (7.2.4c). Suppose conversely that {Xn} is a sequence of independent variables which converges almost surely to X. By Exercise (7.2.8), X is almost surely constant, and we may therefore suppose that Xn ~ c where c E lR.. It follows that, for E > 0, only finitely many of the (independent) events ([Xn - c[ > Ej occur, with probability one. Using the second Borel-Cantelli lemma, LlP'([Xn -c[ >

E)

<

00.

n

8.

Of the various ways of doing this, here is one. We have that

(n)-l ""L.. 2

1~I<J~n

x.x. __n_(~~x.)2_ n(n 1_ 1) ~x? I ) - n - 1 n L. I L. I' i=l

328

i=l

Solutions

Some ancillary results

Now n

-1

L;'i Xi

D

--+ p" by the law oflarge numbers (5.10.2); hence n

-1

L;'i Xi

[7.3.9]-[7.3.12]

P

--+ P, (use Theorem

(7.2.4a». It follows that (n- 1 L;'i Xi)2 ~ p,2; to see this, either argue directly or use Problem (7.11.3). Now use Exercise (7.2.7) to find that n

(1

n

- - - LXi n - 1 n i=l

)2

p

--+ p,2.

Arguing similarly, 1

n

-----,----:-:-"'X? ~ 0 n(n -1) 6

1

'

1=1

and the result follows by the fact (Theorem (7.3.9» that the sum of these two expressions converges in probability to the sum of their limits. 9.

Evidently, IP'

Xn ) = -1-->1+1" ( logn n1+E'

for 11"1 < 1.

By the Borel--Cantelli lemmas, the events An = (Xn/logn :::: 1 + E} occur a.s. infinitely often for -1 < I" ::: 0, and a.s. only finitely often for I" > O. 10. (a) Mills's ratio (Exercise (4.4.8) or Problem (4.14.1c» informs us that 1 - (x) ~ x- 1cp(x) as x -+ 00. Therefore,

The sum over n of these terms converges if and only if I" > 0, and the Borel-Cantelli lemmas imply the claim. (b) This is an easy implication of the Borel--Cantelli lemmas. 11. Let X be uniformly distributed on the interval [-1, 1], and define Xn = I{X:::(-l)n In). The distribution of Xn approaches the Bernoulli distribution which takes the values ±1 with equal probability ~. The median of Xn is 1 if n is even and - 1 if n is odd. 12. (i) We have that

for x > O. The result follows by the second Borel--Cantelli lemma. (ii) (a) The stationary distribution 7r is found in the usual way to satisfy k:::: 2.

Hence Jrk = (k(k + 1)}-1 for k:::: 1, a distribution with mean L;~1 (k (b) By construction, IP'(Xn ::: Xo + n) = 1 for all n, whence . sup -Xn ::: 1) IP' ( hm n---+oo n

It may in fact be shown that 1P'(lim sUPn---+oo Xnln

=

= 0) = 1.

329

1.

+ 1)-1 = 00.

[7.3.13]-[7.4.3]

Solutions

Convergence of random variables

13. We divide the numerator and denominator by y'na. By the central limit theorem, the former converges in distribution to the N(O, I) distribution. We expand the new denominator, squared, as

-

ILn (X

na 2

r=1

22-

r - 11,) - - (2X - /1) na

n L

(X r - /1)

1- + -(X 2

r=1

a

2

/1) .

By the weak law oflarge numbers (Theorem (5.10.2), combined with Theorem (7.2.3», the first term converges in probability to I, and the other terms to O. Their sum converges to I, by Theorem (7.3.9), and the result follows by Slutsky's theorem, Exercise (7.2.5).

7.4 Solutions. Laws of large numbers 1.

LetSn=X1+X2+",+Xn.Then n.

2

2 ",I n lEeS ) = L - < n i=2 logi - logn

and therefore Sn/n ~ O. On the other hand, I:i lP'(IXii :::: i) = I, so that IXii :::: i i.o., with probability one, by the second Borel-Cantelli lemma. For such a value ofi, we have that lSi - Si-11 :::: i, implying that Sn/ n does not converge, with probability one. 2.

Let the Xn satisfy

lP'(Xn

= -n) = I

I

I n

3

lP'(Xn = n - n) = 2'

- 2' n

whence they have zero mean. However,

implying by the first Borel-Cantelli lemma that lP'(Xn/n -+ -I) = 1. It is an elementary result of real analysis that n -1 I:~= 1 xn -+ -I if Xn -+ -I, and the claim follows. 3. The random variable N(S) has mean and variance AISI = cr d , where c is a constant depending only on d. By Chebyshov's inequality,

lP'

(

I

N(S) 1 ) A (A)2 lSi-A:::: E :::: E21S1 = ;-

crd'

By the first Borel-Cantelli lemma, IISkl-1N(Sk) - AI :::: E for only finitely many integers k, a.s., where Sk is the sphere of radius k. It follows that N(Sk)/ISkl ~ A as k -+ 00. The same conclusion holds as k -+ 00 through the reals, since N(S) is non-decreasing in the radius of S.

330

Martingales

Solutions

[7.5.1]-[7.7.1]

7.5 Solutions. The strong law 1.

Let Ii} be the indicator function of the event that Xj lies in the ith interval. Then n

=

log Rm

n

m

m

L Zm (i) log Pi = L L Iij log Pi = L Yj i=l

i=l j=l

j=l

~7=1 Iij log Pi is the sum of independent identically distributed

where, for 1 ::: j ::: m, Yj variables with mean

n

lE(Yj)

=

L Pi log Pi = -h. i=l

By the strong law, m-1log Rm ~ -h. 2. The following two observations are clear: (a) N(t) < n if and only if Tn > t, (b) TN(t) ::: t < TN(t)+I' IflE(XI) < 00, then lE(Tn) <

00,

so that lP'(Tn > t) -+ 0 as t -+

lP'(N(t) < n) implying that N(t) ~ Secondly, by (b),

00

as t -+

= lP'(Tn

> t) -+ 0

_t_ < TN(t)+1 . (1 N(t) - N(t) N(t) + 1

00,

Therefore, by (a),

00,

00.

_TN_(_t) <

Take the limit as t -+ tIN(t) ~ lE(XI).

as t -+

00.

+ N(t)-l).

using the fact that Tnln ~ lE(XI) by the strong law, to deduce that

3. By the strong law, Snln ~ lE(XI) =I- O. In particular, with probability 1, Sn = 0 only finitely often.

7.6 Solution. The law of the iterated logarithm 1.

The sum Sn is approximately N(O, n), so that _1",

lP'(Sn> Janlogn)

= 1- (Jalogn)

<

n 2 alogn

,JaIOgn

for all large n, by the tail estimate of Exercise (4.4.8) or Problem (4.14.1c) for the normal distribution. This is summable if a > 2, and the claim follows by an application of the first Borel-Cantelli lemma.

7.7 Solutions. Martingales 1.

Suppose i < j. Then lE(XjX i )

= lE{ lE[(Sj

- Sj-I)Xi

I So, Sl,"" Sj-d}

= lE{ Xi [lE(Sj I So, Sl,···, Sj-l) 331

Sj-d}

=0

[7.7.2]-[7.8.2]

Convergence of random variables

Solutions

by the martingale property. 2.

Clearly EISnl < E(Sn+1

00

for all n. Also, for n :::: 0,

I Zo, Zl, ... , Zn) =

= 3.

Certainly EISnl <

00

4. Si

= (1

1{ + m

J1-n+1

I Zo, ... , Zn)

J1-Zn - m

(1

n 1 - m (1-J11 _ J1-+ )} n 1

1 _J1- J1-+ )} = Sn·

for all n. Secondly, for n :::: 1,

E(Sn+1

which equals Sn if a

1 { E(Zn+1 J1-n+1

I Xo, Xl,···, Xn)

= aE(Xn+1 I Xo,···, Xn) = (aa + I)Xn +abXn -1,

+ Xn

- a)-l.

The gambler stakes Zi = fi-1 (Xl, ... , X i -1) on theithplay, ataretumof Xi per unit. Therefore Si-1 + XiZi for i :::: 2, with Sl = X1Y. Secondly,

=

where we have used the fact that Zn+1 depends only on Xl, X2, ... , X n .

7.8 Solutions. Martingale convergence theorem 1. It is easily checked that Sn defines a martingale with respect to itself, and the claim follows from the Dooh-Kolmogorov inequality, using the fact that n

E(S;) = Lvar(Xj). j=l

2. It would be easy but somewhat perverse to use the martingale convergence theorem, and so we give a direct proof based on Kolmogorov's inequality of Exercise (7.8.1). Applying this inequality to the sequence Zm, Zm+1, ... where Zi = (Xi - EXi)/i, we obtain that Sn = Zl + Z2 + ... + Zn satisfies, for E > 0,

We take the limit as r

~ 00,

n~m

~ 00

E

var(Zn).

n=m+1

using the continuity of IP', to obtain

IP' ( sup ISn - Sml > Now let m

t

E) :s ~

IP' ( max ISn - Sml > m
E

:s

)

1

2" E

00

1 L 2 var(Xn )· n=m+1n

to obtain (after a small step) IP' (lim sup ISn - Sml m-+oo n?:.m

:s E) = 1

for all

E

Any real sequence (xn) satisfying lim sup

m-+ 00 n?:.m

IXn - xml

:s E

332

for all

E

> 0,

> O.

Solutions

Prediction and conditional expectation

[7.8.3]-[7.9.4]

is Cauchy convergent, and hence convergent. It follows that Sn converges a.s. to some limit Y. The last part is an immediate consequence, using Kronecker's lemma. 3. By the martingale convergence theorem, S = limn ..... oo Sn exists a.s., and Sn ~ S. Using Exercise (7.2.1c), var(Sn) -+ var(S), and therefore var(S) = O.

7.9 Solutions. Prediction and conditional expectation 1.

(a) Clearly the best predictors are E(X I Y)

=

y2, E(Y

I X) = o.

(b) We have, after expansion, that

since E(Y)

=

E(y3)

=

O. This is a minimum when b

1.

=

E(y2)

=

~, and a

=

O. The best linear

predictor of X given Y is therefore Note that E(Y I X) = 0 is a linear function of X; it is therefore the best linear predictor of Y given X.

2.

By the result of Problem (4.14.13), E(Y

3.

Write

I X) = f-t2 + pa2(X

- f-tl)/al, in the natural notation.

n

g(a)

= LaiXi = aX', i=l

and v(a)

= E{ (Y

Let a be a vector satisfying va'

- g(a))2}

= E(YX').

= E(y2) -

+ aVa'.

Then

v(a) - v(a) = aVa' - 2aE(YX')

= aVa' -

2aE(YX')

+ 2iE(YX') - ava' = (a - a)V(a - a)'::: 0,

2aVa' + ava'

since V is non-negative definite. Hence v(a) is a minimum when a is non-singular, a = E(YX)V- 1.

= a, and the answer is g(a). If V

4. Recall that Z = E(Y I g,) is the ('almost') unique g,-measurable random variable with finite mean and satisfying E(Y - Z)Ie} = 0 for all G E g,. (i) Q E g" and hence E(E(Y I g,)In} = E(ZIn) = E(Y In). (ii) U = aE(Y I g,) + ,BE(Z I g,) satisfies E(U Ie)

= aE{E(Y I g,)Ie} + ,BE{E(Z I g,)Ie} = aE(Y Ie) + ,BE(ZIe) = E{ (aY + ,BZ)Ie},

Also, U is g,-measurable. (iii) Suppose there exists m (> 0) such that G = (E(Y I g,) < -m} has strictly positive probability. Then G E g" and so E(Y Ie) = E(E(Y I g,)Ie}. However Y Ie ::: 0, whereas E(Y I g,)Ie < -m. We obtain a contradiction on taking expectations. (iv) Just check the definition of conditional expectation. (v) If Y is independent of g" then E(Y Ie) = E(Y)IP'(G) for G E g,. Hence E(Y - E(Y))Ie} = 0 for G E g" as required.

333

[7.9.5]-[7.10.1]

Solutions

Convergence of random variables

(vi) If g is convex then, for all a

E

JR, there exists )..(a) such that

g(y) ::: g(a)

+ (y -

a))..(a);

furthermore).. may be chosen to be a measurable function of a. Set a = E(Y I 9.) and y obtain g(Y) ::: g{E(Y I 9.)} + {Y - E(Y I 9.) })"{E(Y I 9.)}.

=

Y, to

Take expectations conditional on 9., and use the fact that E(Y I 9.) is 9.-measurable. (vii) We have that where Vn = E { sUPm~n IYm - Y I I 9. }. Now {Vn : n ::: I} is non-increasing and bounded below. Hence V = limn-HXJ Vn exists and satisfies V ::: O. Also E(V)

:s E(Vn ) = E {sup

m?.n

IYm - YI},

which tends to 0 as m ->- 00, by the dominated convergence theorem. Therefore E(V) = 0, and hence JP'(V = 0) = 1. The claim follows.

5.

E(Y I X)

=

X.

6. (a) Let {Xn : n ::: I} be a sequence of members of H which is Cauchy convergent in mean square, that is, E{IXn - Xml 2 } ->- 0 as m,n ->- 00. By Chebyshov's inequality, {Xn : n ::: I} is Cauchy convergent in probability, and therefore converges in probability to some limit X (see Exercise (7.3.1)). It follows that there exists a subsequence {Xnk : k ::: I} which converges to X almost surely. Since each X nk is 9.-measurable, we may assume that X is 9.-measurable. Now, as n ->- 00,

where we have used Fatou's lemma and Cauchy convergence in mean square. Therefore Xn ~ X. That E(X2) < 00 is a consequence of Exercise (7.2.la). (b) That (i) implies (ii) is obvious, since I G E H. Suppose that (ii) holds. Any Z (E H) may be written as the limit, as n ->- 00, of random variables of the form men)

Zn

=

2: ai(n)IGi(n)

i=l

for reals ai(n) and events Gi(n) in 9.; furthermore we may assume that IZnl :s IZI. It is easy to see that E{ (Y - M)Zn} = 0 for all n. By dominated convergence, E{(Y - M)Zn} ->- E{(Y - M)Z}, and the claim follows.

7.10 Solutions. Uniform integrability 1.

It is easily checked by considering whether Ixl

Now substitute x

= Xn

and y

=

:s a or Iyl :s a that, for a >

Yn , and take expectations.

334

0,

Solutions

Uniform integrability

[7.10.2]-[7.10.6]

2. (a) Let E > O. There exists N such that E(IXn - XI') < E if n > N. Now Elxrl < Exercise (7.2.1a), and therefore there exists 0 (> 0) such that E(lXl r fA) < E, E(IXnl r fA) < E for 1 :::: n :::: N, for all events A such that lP'(A) <

o.

o.

Therefore {IX n

by

By Minkowski's inequality,

+ {E(IX( fA)}l/r

{E(IXn ( fA)}l/r :::: {E(IXn - X( fA)}l/r if lP'(A) <

00,

:::: 2E1/r

ifn > N

r :n :::: I} is uniformly integrable.

If r is an integer then {X~ : n :::: I} is uniformly integrable also. Also X~ ~ Xr since Xn ~ X (use the result of Problem (7.11.3». Therefore E(X~) ~ E(X r ) as required. (b) Suppose now that the collection (lXn Ir : n :::: l} is uniformly integrable and Xn ~ X. We show first that EIX r I < 00, as follows. There exists a subsequence {X nk : k :::: l} which converges to X almost surely. By Fatou's lemma, EIXrl

= E (liminf IXnklr)

::::

k-HXJ

liminfElX~kl :::: supEIX~1 n

<

00.

k-HXJ

If E > 0, there exists 0 (> 0) such that E(IXrlfA) <

E,

E(IX~lfA) < E

for all n,

whenever A is such that lP'(A) < O. There exists N such that Bn(E) = {IX n - XI > E} satisfies lP'(Bn(E» < 0 for n > N. Consequently n > N, E(IXn - XI') :::: Er + E (IX n - Xl r fBn(E)) , of which the final term satisfies {E (IX n - Xl r fBn(E))} l/r :::: {E (IX~lfBn(E))} l/r

+ {E (IX r IfBn(E))} l/r

:::: 2E1/r.

Therefore, Xn ~ X. 3.

Fix E > 0, and find a real number a such that g(x) > x / E if x > a. If b :::: a, E (IXnlf[lXnl>b}) < EE{g(IXnl)}:::: EsupE(g(IXnl)}, n

whence the left side approaches 0, uniformly in n, as b 4.

~ 00.

Here is a quick way. Extinction is (almost) certain for such a branching process, so that Zn ~ 0,

and hence Zn ~ O. If {Zn : n :::: O} were uniformly integrable, it would follow that E(Zn) n ~ 00; however E(Zn) = 1 for all n. 5.

~ 0 as

We may suppose that X n, Yn , and Zn have finite means, for all n. We have that 0:::: Yn - Xn ::::

Zn - Xn where, by Theorem (7.3.9c), Zn - Xn

EIZn - Xnl

= E(Zn

p ~

- Xn)

Z - X. Also

~

E(Z - X)

= EIZ -

XI,

so that (Zn - Xn : n :::: I} is uniformly integrable, by Theorem (7.10.3). It follows that {Yn - Xn : n :::: l} is uniformly integrable. Also Yn - Xn ~ Y - X, and therefore by Theorem (7.10.3), EIYn -Xnl ~ ElY -XI, which is to saythatE(Yn ) -E(Xn) ~ E(Y) -E(X); henceE(Yn ) ~ E(Y). It is not necessary to use uniform integrability; try doing it using the 'more primitive' Fatou's lemma.

6. For any event A, E(IXnlfA) :::: E(ZfA) where Z by the assumption that E(Z) < 00.

335

= sUPn IXnl.

The uniform integrability follows

[7.11.1]-[7.11.2]

Solutions

Convergence of random variables

7.11 Solutions to problems 1.

E IX~ I = 00 for r 2: 1, so there is no convergence in any mean. However, if E > 0, as n --+

00,

p

so that Xn -+ O. You have insufficient information to decide whether or not Xn converges almost surely: (a) Let X be Cauchy, and let Xn = X/no Then Xn has the required density function, and Xn ~ O. (b) Let the Xn be independent with the specified density functions. For E > 0,

so that 2:n lP'(IXnl > E) = 00. By the second Borel--Cantelli lemma, one, implying that Xn does not converge a.s. to O.

IXnl > E i.o. with probability

2. (i) Assume all the random variables are defined on the same probability space; otherwise it is meaningless to add them together. (a) Clearly Xn(w) + Yn(w) --+ X(w) + Yew) whenever Xn(w) --+ X(w) and Yn(w) --+ Yew). Therefore {Xn + Yn -,<> X + Y} <; {Xn -,<> Xl U {Yn -,<> Y}, a union of events having zero probability. (b) Use Minkowski's inequality to obtain that

(c) If E > 0, we have that

and the probability of the right side tends to 0 as n --+

00.

(d) If Xn ~ X and the Xn are symmetric, then -Xn ~ X. However Xn + (-Xn) ~ 0, which generally differs from 2X in distribution. (ii) (e) Almost-sure convergence follows as in (a) above. (f) The corresponding statement for convergence in rth mean is false in general. Find a random variable Z such that EI Zr I < 00 but EI Z2r I = 00, and define Xn = Yn = Z for all n. p

p

(g) Suppose Xn -+ X and Yn -+ Y. Let E > O. Then lP'(IXnYn -

XYI > E)

=

lP'(I(Xn -

X)(Yn -

::::: lP'(IXn - XI

·IYn -

+ lP'(IXI

Y) + (Xn

- X)Y

+ X(Yn

-

YI > tE) + lP'(IXn - XI ·IYI > tE) ·IYn -

YI > tE).

Now, for 8 > 0,

lP'(IXn - XI·IYI > jE) : : : lP'(IXn - XI> E/(38») +lP'(IYI > 8), 336

Y)I > E)

Solutions

Problems which tends to 0 in the limit as n -;.

00

and 8 -;.

00

[7.11.3]-[7.11.5]

in that order. Together with two similar facts, we

obtain that XnYn ~ XY. (h) The example of (d) above indicates that the corresponding statement is false for convergence in distribution.

3. Let E > O. We may pick M such that IP'(IXI 2: M) :::: E. The continuous function g is uniformly continuous on the bounded interval [- M, M]. There exists 8 > 0 such that Ig(x) - g(y)1 ::::

If Ig(Xn ) - g(X)1 >

E,

then either IX n -

if Ix - yl :::: 8 and Ixl :::: M.

E

XI > 8 or IXI 2:

XI > 8) + IP'(IXI 2:

1P'(lg(Xn ) - g(X)1 > E) :::: 1P'(IXn -

in the limit as n -;.

4.

00.

It follows that g(Xn )

Clearly lE(e itXn )

=

IP'(IXI 2:

M) :::: E,

n . n {I 1 itjlOj-l } II lE(eitYjjlOJ) = II -. - e. . . . 10 1 _ eltjlOJ

-

-

00.

M) -;.

~ g(X).

J=l

as n -;.

M. Therefore

J=l

1 - e it

1 _ e it

IOn (l - eitjlOn)

-;.----

it

The limit is the characteristic function of the uniform distribution on [0, 1].

Now Xn :::: Xn+l :::: 1 for all n, so that Yew) = limn---+oo Xn(w) exists for all w. Therefore a.s. Y; hence Xn ~ D Y , wh ence Y has the um'form d "b ' Xn ---+ 1stn utIon.

5.

(a) Suppose s < t. Then lE(N(s)N(t») = lE(N(s)2) +lE{N(s)(N(t) - N(s»} = lE(N(s)2)

+ lE(N(s»lE(N(t) -

since N has independent increments. Therefore cov(N(s), N(t»

= lE(N(s)N(t») -lE(N(s»lE(N(t» = (As)2 + AS + AS{A(t - s)} - (AS)(At) = AS.

In general, cov(N(s), N(t» = Amin{s, t}. (b) N(t + h) - N(t) has the same distribution as N(h), if h > O. Hence

which tends to 0 as h -;. O. (c) By Markov's inequality, 1P'(IN(t

+ h) -

N(t)1 >

E) :::: El2lE ({ N(t + h) -

N(t) }2)

,

which tends to 0 as h -;. 0, if E > O. (d) Let E > O. For 0 < h < C 1,

IP' (I N(t + h~ -

N(t)

1> E) = IP'(N(t + h) 337

N(t) 2:

I)

= Ah

+ o(h),

N(s»),

[7.11.6]-[7.11.10]

which tends to 0 as h --+ On the other hand,

o.

which tends to

o.

00

as h . ),.

By Markov's inequality, Sn

6.

Convergence of random variables

Solutions

= 2::1=1 Xi

satisfies

lP'(ISnl > nE)

for

E

:s

JE(S~)

--4

(nE)

> O. Using the properties of the X's,

since JE(X i)

= 0 for all i.

Therefore there exists a constant C such that

implying (via the first Borel-Cantelli lemma) that n-1Sn ~ O.

7.

We have by Markov's inequality that < 00

for

E

> 0, so that Xn ~ X (via the first Borel-Cantelli lemma).

8. Either use the Skorokhod representation or characteristic functions. Following the latter route, the characteristic function ofaXn + b is

where ¢n is the characteristic function of X n . The result follows by the continuity theorem.

9.

For any positive reals c, t,

lP'(X;::t)=lP'(X+c;::t+c):s Set c

= (Y2/ t

JE(X + c)2) ( )2 t+c

to obtain the required inequality.

10. Note that g(u)

= u/(l + u) is an increasing function on [0, 00).

lP'(IXnl>E)=lP'(

Therefore, for E > 0,

IXnl > E ) < I+E JE( IXnl ) 1+IXnl I+E - -E-· 1+IXnl

338

Solutions [7.11.11]-[7.11.14]

Problems

by Markov's inequality. If this expectation tends to 0 then Xn Suppose conversely that Xn [Xn[)

E ( 1+[Xn [

as n

~ 00,

p

~

~ o.

O. Then E

S - - ·lP'([Xn [ S E)

I+E

+ 1 ·lP'([Xn [ >

E)

~

E

--

I+E

for E > O. However E is arbitrary, and hence the expectation has limit O.

11. (i) The argument of the solution to Exercise (7.9.6a) shows that (Xn} converges in mean square if it is mean-square Cauchy convergent. Conversely, suppose that Xn ~ X. By Minkowski's inequality,

as m, n ~ 00, so that (Xn} is mean-square Cauchy convergent. (ii) The corresponding result is valid for convergence almost surely, in rth mean, and in probability. For a.s. convergence, it is self-evident by the properties of Cauchy-convergent sequences of real numbers. For convergence in probability, see Exercise (7.3.1). For convergence in rth mean (r ::: 1), just adapt the argument of (i) above.

12. Ifvar(Xi) S M for all i, the variance of n- 1 2:1=1 Xi is 1 n M 2: Lvar(Xi) S - ~ 0 n i=1 n

asn ~

00.

13. (a) We have that

If x S 0 then F (anx)n -logH(x)

~

0, so that H (x)

= O.

Suppose that x >

o.

Then

= - n---+-oo lim {nlog[l- (1- F(anx»l} = lim {n(1n---+-oo

F(anx»}

since -y- 1 Iog (1 - y) ~ 1 as y {, O. Setting x = 1, we obtain n(1 - F(a n » ~ -log H(1), and the second limit follows. (b) This is immediate from the fact that it is valid for all sequences (an}. (c) We have that 1 - F(tex+Y) 1 - F(t)

1 - F(te x +y ) 1 - F(te

X)

1 - F(te X ) 1 - F(t)

~

10gH(e Y ) --=--10gH(1)

10gH(eX ) 10gH(1)

as t ~ 00. Therefore g (x + y) = g (x)g (y). Now g is non-increasing with g (0) = 1. Therefore g(x) = e- f3x for some fJ, and hence H(u) = exp( -au- f3 ) for u > 0, where a = -log H(1).

14. Either use the result of Problem (7.11.13) or do the calculations directly thus. We have that

if x> 0, by elementary trigonometry. Now tan-I y lP'(Mn sxn/TC)

=

(1-

= y + o(y) as y

1 1 x n +0(n-

339

)r ~e-l/x

~ 0, and therefore

asn

~

00.

[7.11.15]-[7.11.16]

Solutions

Convergence

01 random variables

15. The characteristic function of the average satisfies as n --+

00.

By the continuity theorem, the average converges in distribution to the constant /-i, and hence in probability also.

16. (a) With Un

= u(xn), we have that IE(u(X» - E(u(Y»1 :::: ~ /un/·/ln - gn/ :::: ~ /In - gn/ n

n

if I/ul/oo :::: 1. There is equality if Un equals the sign of In - gn. The second equality holds as in Problem (2.7.13) and Exercise (4.12.3). (b) Similarly, if I/ul/oo :::: 1, IE(u(X» - E(u(Y»1 ::::

i:

/u(x)/ . /I(x) - g(x)/ dx ::::

i:

/I(x) - g(x)/ dx

with equality if u(x) is the sign of I(x) - g(x). Secondly, we have that IJP'(X E A) - JP'(Y E A) I = ~ IE(u(X» - E(u(y» I :::: ~dTV(X, Y),

where

I

u(x) - { -1

if x

E

A,

if x ¢: A.

Equality holds when A = (x E JR : I(x) ::: g(x)}. (c) Suppose dTV(Xn , X) --+ O. Fix a E JR, and let u be the indicator function of the interval Then /E(u(Xn» - E(u(X»/ = /JP'(Xn :::: a) -JP'(X :::: a)/, and the claim follows.

(-00, a].

.s

o. However, by part (a), On the other hand, if Xn = n- 1 with probability one, then Xn = 2 for all n. (d) This is tricky without a knowledge of Radon-Nikodym derivatives, and we therefore restrict ourselves to the case when X and Y are discrete. (The continuous case is analogous.) As in the solution to Exercise (4.12.4), JP'(X =1= Y) ::: ~dTV(X, Y). That equality is possible was proved for Exercise (4.12.5), and we rephrase that solution here. Let /-in = minUn, gnj and /-i = 2:n /-in, and note that

dTV(Xn,O)

n

n

It is easy to see that 1

'1 dTV (X, Y)

= JP'(X =1= Y) =

{I0

if /-i if /-i

=

0,

=

1,

and therefore we may assume that 0 < /-i < 1. Let U, V, W be random variables with mass functions JP' (U --

II

Xn ) -!::!!.. ,

/-i

JP'(V --

Xn ) --

max{/'n - g n, OJ JP'(W _ , - Xn ) -_

1- /-i

-

mint/,n - gn, OJ ,

1 - /-i

and let Z be a Bernoulli variable with parameter /-i, independent of (U, V, W). We now choose the pair X', Y' by (X' Y') ,

={

(U, U) (V, W)

340

if Z if Z

=

1,

= O.

Solutions [7.11.17]-[7.11.19]

Problems

It may be checked that X' and y' have the same distributions as X and Y, and furthermore, IP'(X' # y') = IP'(Z = 0) = I - /-i = ~dTV(X, Y). (e) By part (d), we may find independent pairs (X;, Y[), I :::: i :::: n, having the same marginals as (Xi, Yi), respectively, and such thatlP'(X; # Y[) = ~dTV(Xi' Yi). Now,

n :::: 21P' ( LX; i=l

#

n) n L Y[ :::: 2 LIP'(X; i=l i=l

n

# y[) = 2 LdTV(Xi, Yi)· i=l

17. If Xl, X2, ... are independent variables having the Poisson distribution with parameter A, then Sn = Xl +X2 + ... +Xn has the Poisson distribution with parameter An. Now n-1Sn E(g(n-1Sn» ~ g(A) for all bounded continuous g. The result follows.

~

A, so that

18. The characteristic function 'ifrmn of U mn =

(Xn - n) - (Ym - m) ~

ym+n

satisfies

as m, n ~ 00, implying by the continuity theorem that Umn distributed with parameter m + n, and therefore

v:mn --

J+

Xn Ym -+ P 1 m+n

~ N(O, 1). Now Xn + Ym is Poisson-

asm,n

~

00

o

by the law oflarge numbers and Problem (3). It follows by Slutsky's theorem (7 .2.5a) that Umn / Vmn -+ N(O, 1) as required.

19. (a) The characteristic function of Xn is !/In (t)

=

exp(i /-int - ~a;t2} where /-in and a; are

1 2 the mean and variance of X n . Now, limn-+ oo !/In(l) exists. However !/In(l) has modulus e-'I an , and therefore a 2 = limn-+oo a; exists. The remaining component ei/Lnt of !/In (t) converges as n ~ 00, say ei/Lnt ~ 8 (t) as n ~ 00 where 8 (t) lies on the unit circle of the complex plane. Now 1 22 !/In(t) ~ 8(t)e-'I a t , which is required to be a characteristic function; therefore 8 is a continuous function of t. Of the various ways of showing that 8 (t) = ei/L t for some /-i, here is one. The sequence

'ifrn(t) = ei/Lnt is a sequence of characteristic functions whose limit 8(t) is continuous at t = O. Therefore 8 is a characteristic function. However'ifrn is the characteristic function of the constant /-in, which must converge in distribution as n ~ 00; it follows that the real sequence {/-in} converges to some limit /-i, and 8(t) = ei/L t as required. This proves that !/In(t) ~ exp(i/-it - ~a2t2}, and therefore the limit X is N(/-i, a 2). (b) Each linear combination S Xn + t Yn converges in probability, and hence in distribution, to SX +t Y. Now s Xn + t Yn has a normal distribution, implying by part (a) that s X + t Y is normal. Therefore the joint characteristic function of X and Y satisfies

!/Jx,Y(s, t)

= !/Jsx+ty(l) = exp{iE(s X + t Y) - ~ var(sX + t Y)}

= eXP{i(S/-iX + t/-iy) - ~(s2a1 + 2stPxyaxay + t 2af;)} 341

[7.11.20]-[7.11.21J

Solutions

Convergence of random variables

in the natural notation. Viewed as a function of sand t, this is the joint characteristic function of a bivariate normal distribution. When working in such a context, the technique of using linear combinations of Xn and Yn is sometimes called the 'Cramer-Wold device'.

2:1=1 Yi. It suffices to show that n-ITn ~ O. Now, as

20. (i) Write Yi = Xi -lE(Xi) and Tn = n --+ 00,

(ii) Let E > O. There exists I such that Ip(Xi, Xj) I :::; n

L

cov(Xi , X j ):::;

L

L

COV(Xi, Xj) +

li- jl:::J I:s,i,j::::n

i,j=1

E,

COV(Xi,Xj):::;2nlc+n 2 Ec,

li-jl>! I:s,i,j:s,n

2 2 2Ic lE(Tn / n ) :::; n

This is valid for all positive

if Ii - jl :::: I. Now

E

+ EC --+

as n --+

EC

00.

and the result follows.

21. The integral

(XJ_C_

12

dx

x log Ixl

diverges, and therefore lE(X I) does not exist. The characteristic function cjJ of X I may be expressed as cjJ(t)

= 2c

whence cjJ(t) -cjJ(O)

1

00

2

=_

2c

cos(tx) - 2 - - dx

x logx

roo l-cos(tx) dx.

12

x 2 logx

Now 0:::; 1 - cose :::; rnin{2, e 2 }, and therefore cjJ(t) - cjJ(O)

I

2c

11/t

I< -

2

l1

Now

-

U

2

t --dx+ log x

1

00

2 ---dx I/t x 2 logx '

dx ----+0

as u --+

u 2 logx

and

1

00

u

2 x 2 log x

1

- - - dx < - -

1

00

- log u u

-

2

x2

dx

ift > O.

00,

= -2-

u log u '

u> 1.

Therefore

IcjJ(t) ;c cjJ(O) I = o(t)

as t

+O.

Now cjJ is an even function, and hence cjJ'(O) exists and equals O. Use the result of Problem (7.11.15) to deduce that n- I Xi converges in distribution to 0, and therefore in probability also, since 0 is constant. The Xi do not obey the strong law since they have no mean.

2:1

342

Problems

Solutions

[7.11.22]-[7.11.24]

22. If the two points are U and V then

and therefore 12 1~ 2p1 -Xn = - L...J(Ui - Vi) ~n n i=1 6

as n --+

by the independence of the components. It follows that Xnl In Problem (7.11.3) or by the fact that

00,

~ 1/.J6 either by the result of

23. The characteristic function of Yj = Xi 1 is

rjJ(t)

=1

lo

l .ll

(e

/

x +e- II. / X )dx=

2 0

by the substitution x

10 1 cos(tlx)dx=ltl

1

00

It I

0

= It II y.

cosy -dy y2

Therefore 1 - cosy

1

00

rjJ(t) = 1 - It I

2

It I

y

dy = 1 - Iitl

+ o(ltl)

as t --+ 0,

where, integrating by parts,

_ 10

1It follows that Tn

as t --+

00,

00

o

= n- 1 L.}=1 XiI

-looo

1 - cos y sin u _ 1T 2 dy - - du - - . YOu 2 has characteristic function

whence 2Tnl1T is asymptotically Cauchy-distributed. In particular,

lP'(12Tnl1T l>

1)

21

--+ -

1T

00

1

du

1 2

as t --+

--2 = -

l+u

00.

24. Let mn be a non-decreasing sequence of integers satisfying 1 S mn < n, mn --+

00,

and define

noting that Ynk takes the value ±1 each with probability! whenever mn < k < n. Let Zn

L.k=1 Ynk· Then n

lP'(Un

=I

Zn) S

n

L lP'(IXk I 2: mn) S L k=1

k=mn

343

1

k2

--+

0

as n --+

00,

[7.11.25]-[7.11.26] from

Solutions

Convergence of random variables

~hich it follows that Un / In

-R N(O, I) if and only if Zn/ In -R N(O, I). Now mn Zn = L Ynk k=l

+ Bn- mn

where Bn - mn is the sum of n - mn independent summands each of which takes the values ±I, each possibility having probability Furthermore

i.

1

1 mn 1 2 -LYnk :::: mn In k=l In

which tends to 0 if mn is chosen to be mn

n- 1B

n- mn Finally,

= Ln 1/5 J;

with this choice for mn, we have that

-R N(O, 1), and the result follows. var(Un )

=

t (2 - ;) k

k=l

so that var(Un / Ji1)

1 n

L

=2-

1

2" -* 2. n k=l k

25. (i) Let rpn and rp be the characteristic functions of Xn and X. The characteristic function 1/fk of XNk is 00

1/fk(t) = Lrpj(t)lP'(Nk = j) j=l

whence 00

l1/fk(t) - rp(t)1 :::: L Irpj(t) - rp(t)IlP'(Nk = j). j=l

LetE > O. Wehavethatrpj(t) -* rp(t)asj -* 00, and hencefor any T > O,thereexistsJ(T)suchthat Irpj (t) - rp(t)1 < E if j ::: J (T) and It I :::: T. Finally, there exists K(T) such that lP'(Nk :::: J (T)) :::: E if k ::: K(T). It follows that

if It I :::: T and k ::: K(T); therefore 1/fk(t) -* rp(t) as k -* (ii) Let Yn = supm:::n IX m - XI. For E > 0, n ::: I,

00.

+ lP'(IXNk - XI> E, Nk > n) :::: n) + lP'(Yn > E) -* lP'(Yn > E)

lP'(IXNk - XI> E) :::: lP'(Nk :::: n) :::: lP'(Nk

Now take the limit as n -*

00

and use the fact that Yn ~ O.

26. (a) We have that (

k)

an-,n --'----'= a(n

+ I, n)

IIk (I - -.) < exp (k.) - L..J - . 1

i=O

n

344

",I

-

i=O n

as k -*

00.

Solutions

Problems

[7.11.27]-[7.11.28]

(b) The expectation is En =

Lg (j ~n) nj~~n .

where the sum is over all j satisfying n - M In g ( j - n ) njeIn j!

J.

yn

J

n

=

::::

e-

j :::: n. For such a value of j,

n

In

j (nJ+l _ n ) j! (j - 1)! '

whence En has the form given. (c) Now g is continuous on the interval [-M, 0], and it follows by the central limit theorem that

Also,

where k

=

LM JnJ. Take the limits as n -+ 1 - - < lim

.JZn -

and M -+

00

n-'>OO

00

{ I} nn+ 2e -n n!

in that order to obtain 1 < --. - .JZn

27. Clearly

since a red ball is added with probability Rn/(n

+ 2).

Hence

and also 0 :::: Sn :::: 1. Using the martingale convergence theorem, S surely and in mean square.

28. Let 0 <

E

<

=

limn-,>oo Sn exists almost

1, and let

and let Imn(t) be the indicatorfunction of the event {met) :::: M(t) < net)}. Since M(t)/t ~ we may find T such that E(lmn (t» > 1 - E for t ::: T. We may approximate SM(t) by the random variable Sk(t) as follows. With Aj = {ISj - Sk(t) I > E.Jk(t) },

e,

lP'(AM(t») :::: lP'(AM(t), Imn(t) k(t)-l

::::lP'( U

Aj)

= 1) + lP'(AM(t), Imn(t) = 0) n(t)-l

+lP'( U

j=m(t)

<

-

Aj) +lP'(Imn(t)

j=k(t)

{k(t) - met) }cr 2 E 2 k(t)

+

{n(t) - k(t) }cr 2 E 2 k(t)

ift::: T,

345

+E

=0)

[7.11.29]-[7.11.32]

Solutions

Convergence of random variables

by Kolmogorov's inequality (Exercise (7.8.1) and Problem (7.11.29». Send t -+ as t -+

Now Sk(t)/ Jk(t)

00

to find that

00.

E.. N(O, 0'2) as t -+ 00, by the usual central limit theorem. Therefore

which implies the first claim, since k(t)/(Bt) -+ 1 (see Exercise (7.2.7». The second part follows by Slutsky's theorem (7.2.5a).

29. We have that Sn

=

Sk

+ (Sn

- Sk), and so, for n

~

k,

Now Sf I Ak ~ c2 I Ak; the second term on the right side is 0, by the independence of the X's, and the third term is non-negative. The first inequality of the question follows. Summing over k, we obtain lE(S~) ~ c 2 11"(Mn > c) as required. 30. (i) With Sn

= ~i=l

Xi, we have by Kolmogorov's inequality that

11"

max ISm+k - Sml > E ) ( l~k~n

.:s

1 m+n

L lE(X'f)

2"

E k=m

for E > O. Take the limit as m, n -+ 00 to obtain in the usual way that (Sr : r ~ 0) is a.s. Cauchy convergent, and therefore a.s. convergent, if ~llE(Xf) < 00. It is shorter to use the martingale convergence theorem, noting that Sn is a martingale with uniformly bounded second moments. (ii) Apply part (i) to the sequence Yk = Xk/bk to deduce that ~~1 Xk/bk converges a.s. The claim now follows by Kronecker's lemma (see Exercise (7.8.2». 31. (a) This is immediate by the observation that eA(P)

= fxo II p~ij . i,j

/

(b) Clearly ~j Pij = 1 for each i, and we introduce Lagrange multipliers (ILi : i E S) and write V = )..(P) + ~i ILi ~j Pij' Differentiating V with respect to each Pij yields a stationary (maximum) value when (Nij / Pij) + ILi = O. Hence ~k Nik = - ILk> and

(c) We have thatNij = ~E\ Nik Ir where Ir is the indicator function of the event that the rth transition out of i is to j. By the Markov property, the Ir are independent with constant mean Pij' Using the strong law of large numbers and the fact that ~k Nik ~

00

as n -+

00,

Pij ~ lEU 1) = Pij'

32. (a) If X is transient then Vi(n) < 00 a.s., and ILi = 00, whence Vi (n)/n ~ 0 = ILi 1. If X is persistent, then without loss of generality we may assume X0 = i. Let T (r) be the duration of the rth

346

Solutions

Problems

[7.11.33]-[7.11.34]

excursion from i. By the strong Markov property, the T (r) are independent and identically distributed with mean ILi. Furthermore, Vi(n)-I n 1 Vi(n) " T(r) < - - < - - " T(r) Vi(n) Vi(n) - Vi(n)

f=i

-

f=i

By the strong law of large numbers and the fact that Vi (n) ~ sandwich the central term, and the result follows. (b) Note that I:~:J f(X r ) = unique stationary distribution,

I:iES f(i)Vi(n).

:s where Ilflloo = sup{lf(i)1 : i The other sum satisfies

E

" {L

iEQ

00

.

as n --+

00,

the two outer terms

With Q a finite subset of S, and lfi = ILil, the

IVi(n) 1 I+ L ,,(Vi(n) - - --:-n

IL,

S}. The sum over i

i¢Q

n

1 )} Ilflloo, + --:IL,

Q converges a.s. to 0 as n --+

E

00,

by part (a).

"L (Vi(n) 1) _ 2 - "L (Vi(n) --+--+lfi ) i¢Q n ILi iEQ n

which approaches 0 a.s., in the limits as n --+

00

and Q

t

S.

33. (a) Since the chain is persistent, we may assume without loss of generality that Xo = j. Define the times RI, R2, ... of return to j, the sojourn lengths SI, S2, ... in j, and the times VI, V2, ... between visits to j. By the Markov property and the strong law oflarge numbers, 1

n 1 " Sr~-, a.s.

1 -Rn

r=1

n

-~

n

Also, Rn/ Rn+1 ~ 1, since ILj

gj

= JE(RJ}

<

00.

f(X(s))ds

=

n

r=1

If Rn < t < Rn+ I, then

Let n --+ 00 to obtain the result. (b) Note by Theorem (6.9.21) that Pij (t) --+ lfj as t --+ part (a), and the claim follows as in Corollary (6.4.22). (c) Use the fact that

lo

1~ a.s. L Vr ----* ILj·

=-

Ll

00.

We take expectations of the integral in

I{X(s)=j}ds

jES 0

together with the method of solution of Problem (7 .11.32b).

34. (a) By the first Borel-Cantelli lemma, Xn = Yn for all but finitely many values of n, almost surely. Off an event of probability zero, the sequences are identical for all large n. (b) This follows immediately from part (a), since Xn - Yn = 0 for all large n, almost surely. (c) By the above,

a;;1 I:~I (X r -

Yr ) ~ 0, which implies the claim.

347

[7.11.35]-[7.11.37]

Solutions

Convergence of random variables

35. Let Yn = Xnl{[Xnl.sa}. Then,

n

n

by assumption (a), whence {Xn} and {Yn } are tail-equivalent (see Problem (7.11.34)). By assumption (b) and the martingale convergence theorem (7.8.1) applied to the partial sums 2::::'=1 (Yn - IK(Yn )), the infinite sum 2::~1 (Yn - IK(Yn )) converges almost surely. Finally, 2::~1 IK(Yn ) converges by assumption (c), and therefore 2::~1 Yn , and hence 2::~1 X n , converges a.s. 36. (a) Let nl < n2 < ... < nr

= n.

Since the h take only two values, it suffices to show that r

IF'(lns

= 1 for 1 .:5 s

.:5 r)

= II IF'(lns = 1). s=1

Since F is continuous, the Xi take distinct values with probability 1, and furthermore the ranking of Xl, X2, ... , Xn is equally likely to be any of the n! available. Let xl, x2, ... , Xn be distinct reals, and write A = {Xi = xi for 1 .:5 i .:5 n}. Now, IF'(lns = 1 for 1 .:5 s .:5 r [ A)

1 = ., n.

1

{(n -

1) (n - 1 - n s -l)! . }

ns-l

1

{(n

s- 1 - 1) (n s -l - 1 - n s -2)! } ... (nl - I)! ns-2

1

and the claim follows on averaging over the Xi. (b)WehavethatIK(h) = IF'(h = 1) = k- 1 and var(h) = k- 1(l-k- 1), whence 2::k var(hflog k) < 00. By the independence of the h and the martingale convergence theorem (7.8.1), 2::~1 (h k- 1 ) flog k converges a.s. Therefore, by Kronecker's lemma (see Exercise (7.8.2)),

_1 t(l.-~) ~O

logn j=1

The result follows on recalling that

}

asn

~

J

2::']=1 j-l

~ log n as n ~

00.

00.

37. By an application of the three series theorem of Problem (7.11.35), the series converges almost surely.

348

8 Random processes

8.2 Solutions. Stationary processes 1.

With ai (n) = lP'(X n = i), we have that

cov(X m , X m+n )

= lP'(Xm+n = 1 I Xm = 1)lP'(Xm = 1) -lP'(Xm+n = 1)lP'(Xm = 1) = al (m)Pll (n) - al (m)al (m + n),

and therefore,

+

al (m)pll (n) - al (m)al (m n) p(Xm,Xm+n)=-r~~~==~~~~~~==7===~ y'al (m)(1 - al (m»al (m n)(1 - al (m n»

+

Now, al (m) ---+ a/(a

+ fJ) as m ---+

+

00, and

Pll (n)

= -a- + -fJ- (1 a+fJ

a+fJ

n

- a - fJ) ,

whence p(Xm , X m+n ) ---+ (1 - a - fJ)n as m ---+ 00. Finally, . 1 n a hm - LlP'(X r = 1) = - - . a + fJ

n~oo n r=l

The process is strictly stationary if and only if Xo has the stationary distribution. We have that E(T(t» = 0 and var(T(t» = var(To) = 1. Hence: (a) p(T(s), T(s + t» = E(T(s)T(s + t» = E[(_1)N(t+s)-N(s)] = e- 2M .

2.

(b) Evidently, E(X(t» = 0, and

E[X(t)2] = E ( l l T(U)T(V)dUdV) = 2 (

~
= 'i1 3.

(

E(T(u)T(v») du dv = 2

I I _2M ) t t - 2)" + 2A e ~ 'i

We show first the existence of the limit).,

g(x

+ y) =

t

(V

e- 2).,(v-u) du dv

}V=O}U=O as t ---+ 00.

= limt to g (t) / t, where g (t) = lP'(N (t)

+ y) > 0) = lP'(N(x) > 0) + lP'({N(x) = O} n (N(x + y) for x, y ~ O. .::: g(x) + g(y) lP'(N(x

349

N(x) >

o})

> 0). Clearly,

[8.3.1]-[8.3.4]

Solutions

Raruiom processes

Such a function g is called subadditive, and the existence of A follows by the subadditive limit theorem discussed in Problem (6.15.14). Note that A = 00 is a possibility. Next, we partition the interval (0, 1] into n equal sub-intervals, and let In (r) be the indicator function of the event that at least one arrival lies in ((r - 1) / n, r / n 1 .::: r .::: n. Then ~~=l In (r) t N(1) as n --+ 00, with probability 1. By stationarity and monotone convergence,

l,

lE(N(1))

1 = lE(n~oo lim ~ In(r)) = lim lE(~ In (r)) = lim ng(n- ) = A. ~ n~oo ~ n~oo

r=l

r=l

8.3 Solutions. Renewal processes 1.

See Problem (6.15.8).

2.

With X a certain inter-event time, independent of the chain so far, if Bn = 0,

X- 1 Bn+l = { Bn - 1

if Bn > O.

Therefore, B is a Markov chain with transition probabilities Pi,i-l = 1 for i > 0, and PO} = fJ+l for j 2: 0, where fn = lP'(X = n). The stationary distribution satisfies If} = lfJ+l + lfofJ+l, j 2: 0, with solution If} = lP'(X > j)/lE(X), provided lE(X) is finite. The transition probabilities of B when reversed in equilibrium are _ Pi,i+l

lfi+l

= ----;;- =

lP'(X>i+l) lP'(X > i) ,

fi+l PiO = lP'(X > i) ,

for i 2: O.

These are the transition probabilities of the chain U of Exercise (8.3.1) with the fJ as given. 3. We have that pnun = ~~=l pn-kun_kpk fk, whence Vn = pnun defines a renewal sequence provided p > 0 and ~n pn fn = 1. By Exercise (8.3.1), there exists a Markov chain U and a state s such that Vn = lP'(Un = s) --+ lfs, as n --+ 00, as required. 4.

Noting that N(O)

= 0, 00

00

LlE(N(r))sr r=O

=

r

00

L L u k Sr r=l k=l

=~

Uksk ~1-s k=l

Let Sm

= ~k'=l Xk 00 ~/lE

= O.

and So

k+

[(N(t)

=

k)] = ~/ ~ =

00 ~/

LUk L s k=l r=k

U(s) - 1 l-s

= n) = lP'(Sn

Then lP'(N(r) 00

=

00

[

=

n k

E

Now,

350

00

(

r

F(s)U(s). l-s

:5 r) -lP'(Sn+l :5 r), and

+ k) (lP'(Sn

(

1+

00

n+ k _k

~

:5 t) -lP'(Sn+l :5

1)

]

lP'(Sn'::: t) .

t))

Solutions

Queues

[8.3.5]-[8.4.4]

whence, by the negative binomial theorem,

u(d -

(1 - s)(l - F(s))k

l-s

5. This is an immediate consequence of the fact that the interarrival times of a Poisson process are exponentially distributed, since this specifies the distribution ofthe process.

8.4 Solutions. Queues 1. We use the lack-of-memory property repeatedly, together with the fact that, if X and Y are independent exponential variables with respective parameters A and J-i, then JP'(X < Y) = A/(A + J-i). (a) In this case, p

1 {A J-i A + J-i . A + J-i

= 2:

1 {J-i A A + J-i . A + J-i

J-i}

+ A + J-i + 2:

A}

1

2AJ-i

+ A + J-i = 2: + (A + J-i)2 .

(b) If A < J-i, and you pick the quicker server, p = 1 _ (_J-i_) 2 A+J-i 2AJ-i (c) And finally, p = (A + J-i)2 . 2.

The given event occurs if the time X to the next arrival is less than t, and also less than the time

Y of service of the customer present. Now,

3.

By conditioning on the time of passage of the first vehicle, JE(T)

= loa (x + JE(T) )Ae- AX dx + ae- Aa ,

and the result follows. If it takes a time b to cross the other lane, and so a + b to cross both, then, with an obvious notation, (a)

JE(Ta) +JE(Tb)

(b)

JE(Ta+b)

e aA - 1

eb/L - 1

= - - + --, A J-i e(a+b)(A+/L) - 1

= ----A+J-i

The latter must be the greater, by a consideration of the problem, or by turgid calculation. 4.

Look for a solution of the detailed balance equations J-i n n+l

=

A(n + 1) n+2 nn,

n 2:

o.

to find thatnn = pnno/(n+ 1) is a stationary distribution if p < 1, in which case no = -pi log(l- p). Hence ~n nnn = Ano/(J-i - A), and by the lack-of-memory property the mean time spent waiting for service is pno/(J-i - A). An arriving customer joins the queue with probability

L n + 1 nn = p + log(l- p). 00

n=O n

+2

p log(l - p)

351

[8.4.5]-[8.5.5] 5.

Solutions

Random processes

By considering possible transitions during the interval (t, t + h), the probability Pi (t) that exactly

i demonstrators are busy at time t satisfies:

+ h) = PI (t)2h + P2(t)(1 - 2h) + o(h), PI (t + h) = po(t)2h + PI (t)(1 - h)(1 - 2h) + P2(t)2h + o(h), Po(t + h) = po(t)(1 - 2h) + PI (t)h + o(h). P2 (t

Hence, p~ (t) = 2Pl (t) - 2P2 (t),

PI (t) = 2po(t) - 3Pl (t)

+ 2P2 (t),

poet) = -2po(t)

+ PI (t),

and therefore P2(t) = a + be- 2t + ce- 5t for some constants a, b, c. By considering the values of P2 and its derivatives at t = 0, the boundary conditions are found to be a + b + c = 0, -2b - 5c = 0,

4b + 25c

= 4, and the claim follows. 8.5 Solutions. The Wiener process

1. We might as well assume that W is standard, in that a 2 = I. Because the joint distribution is multivariate normal, we may use Exercise (4.7.5) for the first part, and Exercise (4.9.8) for the second, giving the answer I 1 { sm . -1 -+-

8

4n

ft-+sm. V;-+sm. V;}- . -1

-1

u

t

u

Writing W(s) = "fix, Wet) = y'tZ, and W(u) = .JUY, we obtain random variables X, Z with the standard trivariate normal distribution, with correlations PI = ..jSfU, P2 = y'tfU, P3 = .fiTi. By the solution to Exercise (4.9.9),

2.

Y,

var(Z I X, Y) =

(u-t)(t-s) t(u - s)

,

yielding var(W(t) I W(s), W(u)) as required. Also, lE(W(t)W(u) I W(s), W(v))

= lE {

[

(u - t)W(s) + (t - S)W(U)] I } u_ s W(u) W(s), W(v) ,

which yields the conditional correlation after some algebra.

3.

Whenever a 2 + b 2

4.

Let

t:.}

(n)

=

1.

= W(U + l)t In)

- WUt In). By the independence of these increments,

5. They all have mean zero and variance t, but only (a) has independent normally distributed increments.

352

Solutions

Problems

[8.7.1]-[8.7.3]

8.7 Solutions to problems 1. E(Yn ) = 0, and cov(Ym , Ym+n ) = 2:::-=0 aian+i for m, n ::: 0, with the convention that ak for k > r. The covariance does not depend on m, and therefore the sequence is stationary.

=0

2. We have, by iteration, that Yn = Sn (m) + a m+ 1Yn-m-l where Sn (m) = 2::'}=0 a} Zn-}. There are various ways of showing that the sequence {Sn (m) : m ::: I} converges in mean square and almost surely, and the shortest is as follows. We have that a m+ 1Yn-m-l ---+ 0 in m.s. and a.s. as m ---+ 00; to see this, use the facts that var(a m+ 1 Yn-m-d = a 2 (m+l) var(Yo), and

E

It follows that Sn(m) = Yn - a m+ 1Yn-m-l converges in m.s. and a.s. as m ---+ the same conclusion is as follows. For r < s,

> O.

00.

A longer route to

whence {Sn(m) : m ::: 1} is Cauchy convergent in mean square, and therefore converges in mean square. In order to show the almost sure convergence of Sn(m), one may argue as follows. Certainly

whence 2::~0 a} Zn-} is a.s. absolutely convergent, and therefore a.s. convergent also. We may express limm~ooSn(m) as 2::~oa}Zn-J' Also, am+1Yn_m_l ---+ 0 in mean square and a.s. as m ---+ 00, and we may therefore express Yn as 00

Yn

= LaJZn-J

a.s.

J=O It follows that E(Yn ) = lirnm~oo E(Sn(m)) = O. Finally, for r > 0, the autocovariance function is given by c(r) = cov(Yn , Yn- r ) = E{ (aYn- 1 + Zn)Yn- r } = ac(r - 1),

whence

a lrl c(r) = alrlc(O) = - - 2 ' I-a

r = ... , -1,0, 1, ... ,

since c(O) = var(Yn ).

3. Ift is a non-negative integer, N(t) is the number ofO's and l's preceding the (t+ l)th 1. Therefore N(t) + 1 has the negative binomial distribution with mass function

If t is not an integer, then N(t)

=

NUt J).

353

[8.7.4]-[8.7.6] 4.

Solutions

Random processes

We have that

lP'(Q(t

+ h) = j I Q(t) = i) = {

+ o(h) + o(h) 1 - (A + p,i)h + o(h)

Ah

if j

p,ih

if j if j

= i + 1, = i-I, = i,

an immigration--death process with constant birth rate A and death rates P,i = i p,. Either calculate the stationary distribution in the usual way, or use the fact that birth--death processes are reversible in equilibrium. Hence Ani = p,(i + l)ni+l for i ::: 0, whence i :::

s.

We have that X(t)

=

o.

R cos(lJ!) cos(et) - R sin(lJ!) sin(et). Consider the transformation u

=

r cos 1/f, v = -r sin 1/f, which maps [0, (0) x [0, 2n) to ]R2. The Jacobian is

whence U

=

R cos IJ!, V

=-

au

au

ar av

a1/f

ar

a1/f

av = -r,

R sin IJ! have joint density function satisfying

rfu,v(rcos1/f, -rsin 1/f) = fR,W(r, 1/f). Substitute fu, v(u, v)

1 (2

= e-1

2)

u +V j(2n), to obtain r > 0, O:s

1/f

< 2n.

Thus Rand IJ! are independent, the latter being uniform on [0, 2n). 6. A customer arriving at time u is designated green if he is in state A at time t, an event having probability p(u, t - u). By the colouring theorem (6.13.14), the arrival times of green customers form a non-homogeneous Poisson process with intensity function A(U)p(U, t - u), and the claim follows.

354

9 Stationary processes

9.1 Solutions. Introduction 1.

We examine sequences Wn of the fonn 00

Wn = LakZn-k k=O

for the real sequence {ak : k::: O}. Substitute, toobtainao = 1, al = a,a r with solution if a 2 + 4f3 = 0,

= aar-l +f3ar -2, r

::: 2,

otherwise, where)'! and A2 are the (possibly complex) roots of the quadratic x2 - ax - f3 = 0 (these roots are distinct if and only if a 2 + 4f3 =J:. 0). Using the method in the solution to Problem (8.7.2), the sum in (*) converges in mean square and almost surely if IAll < 1 and IA21 < 1. Assuming this holds, we have from (*) that lE(Wn ) = 0 and the autocovariance function is c(m)

= lE(Wn W n - m ) = ac(m

- 1)

+ f3c(m

m:::

- 2),

1,

by the independence of the Zn. Therefore W is weakly stationary, and the autocovariance function may be expressed in tenns of a and f3.

2. We adopt the convention that, if the binary expansion of U is non-unique, then we take the (unique) non-terminating such expansion. It is clear that Xi takes values in {O, I}, and lP'(Xn+l

=

11 Xi

= Xi

for 1 :s i

:s n) =

~

for all Xl, X2, ... , Xn; therefore the X's are independent Bernoulli random variables. For any sequence kl < k2 < ... < kr, the joint distribution of Vkl' Vk2' ... , Vkr depends only on that of Xkj +1, Xkj +2, .... Since this distribution is the same as the distribution of Xl, X2, ... , we have that (Vkj' Vk2' ... , Vkr ) has the same distribution as (Vo, Vk2-kl' ... , Vkr -k j ). Therefore V is strongly stationary. Clearly lE(Vn )

= lE(Vo) = ~, and, by the independence of the Xi, 00

cov(Vo, Vn )

=

2i n L T - var(Xi) i=l

355

= l2(~)n.

[9.1.3]-[9.2.1]

Solutions

Stationary processes

3. (i) For mean-square convergence, we show that Sk vergent as k -+ 00. We have that, for r < s,

= ~~=O anXn is mean-square Cauchy con-

E{(Ss-Sr)2}= ..t aiajC(i-j):SC(O){.t z,]=r+l z=r+l

,ai,r

since Ic(m) I :s c(O) for all m, by the Cauchy-Schwarz inequality. The last sum tends to 0 as r, s -+ if ~i lai I < 00. Hence Sk converges in mean square as k -+ 00. Secondly, E(

t

lakXkl)

k=l

:s

t

:s

lakl . EIXkl

k=l

V

E (X5)

t

00

lakl

k=l

which converges as n -+ 00 if the lak I are summable. It follows that ~k=l lakXk I converges absolutely (almost surely), and hence ~k=l akXk converges a.s. (ii) Each sum converges a.s. and in mean square, by part (i). Now 00

cy(m) =

L

ajakC(m

+k -

j)

j,k=O

whence

4. Clearly Xn has distribution 1r for all n, so that U(X n ) : n :::: m} has fdds which do not depend on the value of m. Therefore the sequence is strongly stationary.

9.2 Solutions. Linear prediction 1.

(i) We have that

which is minimized by setting a (ii) Similarly

= c(l)/c(O).

Hence Xn+l

= c(l)Xn/c(O).

an expression which is minimized by the choice

(3 =

c(l)(c(O) - c(2)) c(O)2 _ c(J)2 '

y

=

c(O)c(2) - c(l)2 c(O)2 - c(l)2 ;

Xn+l is given accordingly. (iii) Substitute a, {3, y into (*) and (**), and subtract to obtain, after some manipulation,

{c(l)2 - c(O)c(2)}2 D - -'-:c:--:-=-.-'-_--;;.,. - c(O){c(O)2 - c(l)2}'

356

Solutions

Autocovariances and spectra 1

(a) In this case c(O) = '2> and c(l) (b) In this case D = 0 also.

= c(2) = O.

-

Therefore X n+1

=

-

Xn+1

[9.2.2]-[9.3.2]

= 0, and D = O.

In both (a) and (b), little of substance is gained by using Xn+1 in place of X n+1.

2. Let {Zn : n = ... , -1,0, I, ... } be independent random variables with zero means and unit variances, and define the moving-average process Xn=

Zn + aZn -1 ~.

It is easily checked that X has the required autocovariance function. By the projection theorem, Xn - Xn is orthogonal to the collection {X n - r : r > I}, so that 00 . E{(Xn - Xn)X n - r } = 0, r ::: 1. Set Xn = 2:s=1 bsXn - s to obtam that for s ::: 2, where a = a/(I + a 2 ). The unique bounded solution to the above difference equation is bs (-1)s+ 1as, and therefore 00

Xn = L(-I)s+1 a s X n - s . s=1

The mean squared error of prediction is E{ (Xn - Xn)2}

= E {(I:(-a)S Xn_s)2} = _1-2 E(Z~) = _1_2 . I+a

=0

I+a

Clearly E(Xn) = 0 and 00

cov(Xn , X n- m ) = L

brbsc(m

+r -

s),

m::: 0,

r,s=1

so that X is weakly stationary.

9.3 Solutions. Autocovariances and spectra 1.

It is clear that E(Xn) cov(Xm , X m+n )

= 0 and var(Xn ) =

1. Also

= cos(mA) cos{(m +

n)A} + sin(mA) sin{(m + n)A} = cos(nA),

so that X is stationary, and the spectrum of X is the singleton {A}.

2.

Certainly ¢u(t)

=

(e itrr - e- itrr )/(2rrit), so that E(Xn )

cov(Xm , X m+n )

= ¢u(I)¢v(n) = O. Also

= E(XmXm+n ) = E(ei{U-Vm-U+V(m+n))) = ¢v(n),

whence X is stationary. Finally, the autocovariance function is c(n)

= ¢v(n) =

J

whence F is the spectral distribution function.

357

in e ).. dF(A),

[9.3.3]-[9.3.4]

Solutions

Stationary processes

3.

The characteristic functions of these distributions are

(i)

p(t)

= e- Z1 t2 ,

p(t)

= 2" 1 -

(ii)

4.

1(

1

it

1)

1

+ 1 + it = 1 + t 2 '

(i) We have that 1 n c(O) L cov(Xj, Xk) = - 2 ( nInj=l Xj ) = 2" n j,k=l n

var - L

1

(n.. ) L el(j-k)A dF()"). j,k=l

(-rr,rr]

The integrand is 1 - cos(n)..) 1 - cos).. ,

whence 1

~)

var ( -;:; ~ Xj

r

= c(O) J(-rr,rr]

(Sin(n)..j2) ) 2 n sin()..j2) dF()").

It is easily seen that I sin 8 I .:'S 18 I, and therefore the integrand is no larger than

)../2)2 1 2 ( sin()..j2) .:'S (ZJT) . As n -+ 00, the integrand converges to the function which is zero everywhere except at the origin, where (by continuity) we may assign it the value 1. It may be seen, using the dominated convergence theorem, that the integral converges to F(O) - F(O-), the size of the discontinuity of F at the origin, and therefore the variance tends to 0 if and only if F(O) - F(O-) = O. Using a similar argument,

n-l

-1 L cU) n j=O

= _c(0) n

1 (n-l) (-rr,rr]

LeijA j=O

dF()..)

= c(O)

1

gn()..) dF()..)

(-rr,rr]

where if)"

= 0,

if)..

i= 0,

is a bounded sequence of functions which converges as before to the Kronecker delta function DAO. Therefore

n-l

1 - LCU) -+ c(O) (F(O) - F(O-)) n j=O

358

as n -+

00.

Solutions

The ergodic theorem

[9.4.1]-[9.5.2]

9.4 Solutions. Stochastic integration and the spectral representation 1. Let H X be the space of all linear combinations of the Xi, and let H X be the closure of this space, that is, Hx together with the limits of all mean-square Cauchy-convergent sequences in Hx. All members of Hx have zero mean, and therefore all members of H X also. Now S(A) E H X for all A, whence JE(S(A) - S(JL)) = 0 for all A and JL. 2. First, each Ym lies in the space H X containing all linear combinations of the Xn and all limits of mean-square Cauchy-convergent sequences of the same form. As in the solution to Exercise (904.1), all members of H X have zero mean, and therefore JE(Ym) = 0 for all m. Secondly,

As for the last part,

This proves that such a sequence Xn may be expressed as a moving average of an orthonormal sequence. 3. Let H X be the space of all linear combinations of the X n , together with all limits of (meansquare) Cauchy-convergent sequences of such combinations. Using the result of Problem (7.11.19), all elements in H X are normally distributed. In particular, all increments of the spectral process are normal. Similarly, all pairs in H X are jointly normally distributed, and therefore two members of H X are independent if and only if they are uncorrelated. Increments of the spectral process have zero means (by Exercise (9.4.1)) and are orthogonal. Therefore they are uncorrelated, and hence independent.

9.5 Solutions. The ergodic theorem 1. With the usual shift operator" it is obvious that ,-1 0 = 0, so that 0 E 1. Secondly, if A E 1, then ,-I(AC) = (,-I A)C = AC, whence A C E 1. Thirdly, suppose AI, A2, ... E 1. Then

so that

ur Ai

E

1.

2. The left-hand side is the sum of covariances, c(O) appearing n times, and c(i) appearing 2(n - i) times for 0 < i < n, in agreement with the right-hand side. Let E > O. Ife(j) when j ~ J. Now

= r l L:{~~ c(i) -+ 0'2 as j

2 Ln

2" jC(j) n j=1 as n -+

00.

2{JL ::s 2" n

j=1

jC(j)

-+ 00, there exists J such that le(j) - 0'21 < E

L n

+

j (0'2

+ E) }

-+ 0'2

+E

j=J+I

A related lower bound is proved similarly, and the claim follows since E (> 0) is arbitrary.

359

[9.5.3]-[9.6.3] 3.

Solutions

It is easily seen that Sm

Stationary processes

= 2:1=0 ai Xn+i constitutes a martingale with respect to the X's, and

whence Sm converges a.s. and in mean square as m --+ 00. Since the Xn are independent and identically distributed, the sequence Yn is strongly stationary; also E(Yn ) = 0, and so n- I 2:1=1 Yi --+ Z a.s. and in mean, for some random variable Z with mean zero. For any fixed m (::: 1), the contribution of XI, X2, ... , Xm towards 2:1=1 Yi is, for large n, no larger than Cm

=

If(ao +

al

+ ... + aj_l)Xj I·

J=1

Now n- 1C m --+ 0 as n --+ 00, so that Z is defined in terms of the subsequence Xm+l, X m+2, ... for all m, which is to say that Z is a tail function of a sequence of independent random variables. Therefore Z is a.s. constant, and so Z = 0 a.s.

9.6 Solutions. Gaussian processes 1. The quick way is to observe that c is the autocovariance function of a Poisson process with intensity 1. Alternatively, argue as follows. The sum is unchanged by taking complex conjugates, and hence is real. Therefore it equals

where

to = O.

2. For s, t ::: 0, X (s) and X (s + t) have a bivariate normal distribution with zero means, unit variances, and covariance c(t). It is standard (see Problem (4.14.13)) that E(X(s + t) I X(s)) = c(t)X(s). Now c(s

+ t) = E(X(O)X(s + t)) = E{ E(X(O)X(s + t) I X(O),

X(s))}

= E(X(O)c(t)X(s)) = c(s)c(t)

by the Markov property. Therefore c satisfies c(s + t) = c(s)c(t), c(O) plsl. Using the inversion formula, the spectral density function is 1

f( "A)

00

= -2n '""' L

s=-oo

c(s)e-

. ISA

=

= 1, whence c(s) = c(1)lsl =

1 _ p2 ---~-= iA 2n 11

_

pe

12 '

Note that X has the same autocovariance function as a certain autoregressive process. Indeed, stationary Gaussian Markov processes have such a representation. 3. If X is Gaussian and strongly stationary, then it is weakly stationary since it has a finite variance. Conversely suppose X is Gaussian and weakly stationary. Then c(s, t) = cov(X(s), X(t)) depends

360

Solutions

Problems

[9.6.4]-[9.7.2]

on t - s only. The joint distribution of X (tl), X (t2), ... , X (tn ) depends only on the common mean and the covariances C(ti, tj). Now C(ti, tj) depends on tj - ti only, whence X (tl), X (t2), ... , X (tn ) have the same joint distribution as X(s + tl), X(s + t2)"'" X(s + tn ). Therefore X is strongly stationary. 4.

(a) If s, t > 0, we have from Problem (4.14.13) that

whence COV(X(S)2, X(s

+ t)2)

+ t)2) - 1 = E{ X(s)2 E (X(s + t)2 1 X(s))} - 1 = c(t)2E(X(s)4) + (1- c(t)2)E(X(s)2) = E(X(s)2 X (s

by an elementary calculation. (b) Likewise cov(X (s)3, X (s + t)3)

1 = 2c(t)2

= 3(3 + 2c(t)2)c(t).

9.7 Solutions to problems 1. It is easily seen that Yn of Y is given by

= Xn + (a -

f3)X n -1

+ f3Yn-l, whence the autocovariance function c

_1_+_a_2_-"2_f3_2

if k

= 0,

if k

1= o.

1-13 c(k) =

{ f3 lkl - 1

{

a(l: c:!f3~

2 }

13 )

Set Yn+1 = L:~o ai Yn-i and find the ai for which itis the case thatE{(Yn+1 - Yn+I)Yn-kl for k ::: O. These equations yield 00

+ 1) = L aic(k -

c(k

i),

k ::: 0,

i=O

which have solution ai = a(f3 - a)i for i ::: O. 2.

The autocorrelation functions of X and Y satisfy r

aipx(n)

= al

L

ajakPy(n

+k -

j).

j,k=O

Therefore 2

aifx()..)

= ~~

00

L

in e- )..

n=-oo 2

r

L

ajakPy(n +k - j)

j,k=O

r

00

= a y ' " a' aF i (k- j)).. ' " e -i (n+k- j)).. py (n 2rr L...J J L...J j,k=O n=-oo 2 ')" 2 = aylGa(e )1 fy()..)· l

361

+k _

j)

=0

[9.7.3]-[9.7.5]

Solutions

Stationary processes

In the case of exponential smoothing, Ga(e iA ) fX(A) =

where c

- p,)j(1 - p,e iA ), so that

= (1

c(1 - p,)2 fY(A) ,

IAI <Jr,

1 - 2p,COSA + p,2

= a~jai is a constant chosen to make this a density function.

Consider the sequence (X n } defined by

3.

Xn

= Yn - Yn = Yn -

aYn-1 - fJYn -2·

Now Xn is orthogonal to {Yn-k : k ::: I}, so that the Xn are uncorrelated random variables with spectral density function fX(A) = (2Jr)-I, A E (-Jr, Jr). By the result of Problem (9.7.2), aifX(A)

whence f( A)Y

-

- ae iA - fJe 2iA I2 fY(A),

= a~II

a 2 ja 2 Y

X

-Jr
2Jr 11 - ae iA - fJ e 2iA 12 '

4. Let {X~ : n ::: I} be the interarrival times of such a process counted from a time at which a meteorite falls. Then Xl' Xl' . .. are independent and distributed as X2. Let Y~ be the indicator function of the event {X:n = n for some m}. Then lE(YmYm + n )

= lP'(Ym = 1, Ym+ n = 1) = lP'(Ym+n = 1 I Ym = 1)lP'(Ym =

1)

= lP'(Y~ =

where a = lP'(Ym = 1). The autocovariance function of Y is therefore c(n) n :::: 0, and Y is stationary. The spectral density function of Y satisfies

=~

fY(A)

fe-inA

= Re {

c(n)

a(I - a)

2Jr n=-oo

Now

f

1

I)a

=

a(lP'(Y~

einAc(n) -

00

00

n=O

n=O

~}. 2Jr

Jra(1 - a) n=O

L e inA Y~ = L eiAT~ where T~

= Xl + Xl + ... + X~; just check the non-zero terms. 00

"einAc(n) = alE

L

n=O

when e iA

#

00

a2 - --.- =

I }

" e iATn

L

1-

n=O

ell.

Therefore a

1 - 4>(A)

a2

- --.-

1_

ell.

1, where 4> is the characteristic function of X2. It follows that fY(A)

5.

{

=

1 {I

Jr(1 - a)

Re

1 - 4>(A)

We have that lE(cos(nU))

=

j

Jr

-Jr

-

1

2Jr

cos(nu) du

a} 1

- -1 - eiA

= 0, 362

-2Jr '

=

IAI <Jr.

1) - a},

Solutions

Problems

[9.7.6]-[9.7.7]

for n ::: 1. Also lE(cos(mU) cos(nU))

= lE{

i (cos[(m + n)U] + cos[(m -

if m =f- n. Hence X is stationary with autocorrelation function p(k) function f(A) = (2n)-i for IAI < n. Finally

n)U1) }

=

DkO,

=0

and spectral density

+ n)U] + cos[(m - n)U1) cos(rU)} = Hp(m + n - r) + p(m - n - r)}

lE{ cos(mU) cos(nU) cos(rU)} = ilE{ (cos[(m

which takes different values in the two cases (m, n, r)

= (1,2,3),

(2,3,4).

6. (a) The increments of N during any collection of intervals {(Ui, Vi) : 1 ::: i ::: n} have the same fdds if all the intervals are shifted by the same constant. Therefore X is strongly stationary. Certainly lE(X (t)) = Aa for all t, and the autocovariance function is c(t)

= cov(X(O),

X(t))

={

ift > a,

0

A(a - t)

if 0 ::: t ::: a.

Therefore the autocorrelation function is 0 pet) = { 1 _ It/al

if It I > a, if It I ::: a,

which we recognize as the characteristic function ofthe spectral density f (A) = (1-cos(aA) }/(an A2); see Problems (S.l2.27b, 28a). (b) We have that lE(X (t)) = 0; furthermore, for s ::: t, the correlation of X (s) and X(t) is

1 "2cov(X(s), X(t)) (J"

1

= "2cov(W(s) - W(s -1), Wet) (J" = s - minis, t - I) - (s - 1) + (s =

Wet

-1))

1)

ifs
I { s-t+l

This depends on t - s only, and therefore X is stationary; X is Gaussian and therefore strongly stationary also. The autocorrelation function is 0 p(h) = { 1 _ Ihl

if Ihl ::: I, if Ihl < 1,

which we recognize as the characteristic function of the density function I(A) = (I - cos A)/(n A2). 7. We have from Problem (8.7.1) that the general moving-average process of part (b) is stationary with autocovariance function c(k) = 2::j=o ajak+ j' k ::: 0, with the convention that as = 0 if s < 0 ors > r. (a) In this case, the autocorrelation function is

p(k)

=

{~ +a 1

2

o

ifk

= 0,

iflkl

= I,

iflkl>l,

363

[9.7.8]-[9.7.13]

Solutions

Stationary processes

whence the spectral density function is

I(A)

=

1 (

2n

p(O)

+ eZ'A p(I) + e- z'A) pC-I) =

1 (

2n

2aCOSA) 1 + 1 + a2

'

IAI < n.

(b) We have that

1(') = - 1 F\,

where c(O)

8.

2n

Z'A

00

'L...J " e -ikA p (k)

k=-oo

=

1 '" i}'A", ---L...Ja·e L...Jak+· e -i(k+}')A = IA(e )1 2n c(O) j } k} 2n c(O)

2

= 'L,j aJ and A(z) = 'L,j ajZ j . See Problem (9.7.2) also.

The spectral density function

I

is given by the inversion theorem (5.9.1) as

1

00

I(x) = - 1

2n

under the condition

Iooo Ip(t)1 dt

<

00;

. e- ztx pet) dt

-00

see Problem (5.12.20). Now

1 ~1

00

I/(x)1 :::: - 1

2n

and I/(x

+ h) -

Ip(t)1 dt

-00

00

l(x)1 ::::

2n

le

ith

- 11· Ip(t)1 dt.

-00

The integrand is dominated by the integrable function 2Ip(t)I. Using the dominated convergence theorem, we deduce that II (x + h) - I (x) I --+ 0 as h --+ 0, uniformly in x.

9. By Exercise (9.5.2), var(n- I 'L,}=I Xj) --+ a 2 if C n cov(Xo, Xn) --+ 0 then C n --+ 0, and the result follows.

= n- I

'L,j,;::;J cov(Xo, Xj) --+ a 2 . If

10. Let X I, X2, ... be independent identically distributed random variables with mean /-t. The sequence X is stationary, and it is a consequence of the ergodic theorem that n- I 'L,}=I Xj --+ Z a.s. and in mean, where Z is a tail function of X I, X2, ... with mean /-t. Using the zero-one law, Z is a.s. constant, and therefore lP'(Z = /-t) = 1. 11. We have from the ergodic theorem that n- I 'L,f=1 Yi --+ JE(Y I 1) a.s. and in mean, where 1 is the a-field of invariant events. The condition of the question is therefore JE(Y 11)

= JE(Y)

a.s., for all appropriate Y.

Suppose (*) holds. Pick A E 1, and set Y = IA to obtain IA = «:)l(A) a.s. Now IA takes the values 0 and 1, so that «:)l(A) equals 0 or 1, implying that «:)l is ergodic. Conversely, suppose «:)l is ergodic. Then JE(Y I 1) is measurable on a trivial a-field, and therefore equals JE(Y) a.s.

12. Suppose «:)l is strongly mixing. If A is an invariant event then A = r- n A. Therefore «:)l(A) = «:)l(A n r- n A) --+ «:)l(A)2 as n --+ 00, implying that «:)l(A) equals 0 or 1, and therefore «:)l is ergodic.

13. The vector X = (X I, X2, ... ) induces a probability measure «:)l on (l~T, JBT). Since T is measurepreserving, «:)l is stationary. Let Y : jRT --+ jR be given by Y(x) = XI for x = (XI, X2, ... ), and define Yi(X) = Y(ri-I(x)) where r is the usual shift operator on jRT. The vectorY = (YI, Y2, ... ) has the same distributions as the vector X. By the ergodic theorem for Y, n- I 'L,f=1 Yi --+ JE(Y I fJ) a.s. and in mean, where fJ is the invariant a -field of r. It follows that the limit 1 n

Z

= n-+oo lim - ' " Xi n L...J i=1

364

Solutions

Problems

exists a.s. and in mean. Now U

= lim sUPn-+oo (n- I

L;, Xi)

[9.7.14J-[9.7.15J

is invariant, since a.s.,

implying that U(w) = U(Tw) a.s. It follows that U is i-measurable, and it is the case that Z = U a.s. Take conditional expectations of (*), given i, to obtain U = E(X Ii) a.s. If T is ergodic, then i is trivial, so that E(X Ii) is a.s. constant; therefore E(X I i) = E(X) a.s.

14. If (a, b) ~ [0, 1), then T- I (a, b) Secondly, I IP'(T- (a, b»)

= (~a, ~b) u (~ + ~a, ~ + ~b), and therefore T = 2(~b - ~a) = b -

a

is measurable.

= 1P'(a, b»),

T- I

so that preserves the measure of intervals. The intervals generate :B, and it is then standard that T- I preserves the measures of all events. Let A be invariant, in that A = T- I A. Let 0 Therefore WE A if and only if w IP'(A

+~

E A,

n E) = ~IP'(A) =

::s w

< ~; it is easily seen that T(w) = T(w

implying that A

n [~, 1) = ~ + {A n [0, ~)}; hence

for E

IP'(A)IP'(E)

+ ~).

= [0, ~),

[~, 1).

This proves that A is independent of both [0, ~) and [~, 1). A similar proof gives that A is independent of any set E which is,for some n, the union of intervals ofthe form [k2- n , (k+ 1)2- n ) for 0 ::s k < 2n. It is a fundamental result of measure theory that there exists a sequence E I, E2, ... of events such that (a) En is of the above form, for each n, (b) IP'(A /':;. En) -+ 0 as n -+ 00. Choosing the En accordingly, it follows that IP'(A IIP'(A

n En) -

n En) = IP'(A)IP'(En) -+ IP'(A)2 IP'(A) I ::s IP'(A /':;. En) -+ O.

by independence,

Therefore IP'(A) = IP'(A)2 so that IP'(A) equals 0 or 1. For wEn, expand w in base 2, w = O.WI W2 ... , and define Y (w) = WI. It is easily seen that y(Tn-Iw) = Wn, whence the ergodic theorem (Problem (9.7.13» yields that n- I L;t=1 Wi -+ ~ as n -+ 00 for all w in some event of probability 1.

15. We may as well assume that 0 < a < 1. Let T : [0,1) -+ [0,1) be given by T(x) = x + a (mod 1). It is easily seen that T is invertible and measure-preserving. Furthermore T(X) is uniform on [0, 1], and it follows that the sequence ZI, Z2, ... has the same fdds as Z2, Z3, ... , which is to say that Z is stationary. It therefore suffices to prove that T is an ergodic shift, since this will imply by the ergodic theorem that 1 n Zj -+ E(ZI) = g(u) duo n j=1 0

rl

-L

io

We use Fourier analysis. Let A be an invariant subset of [0, 1). The indicator function of A has a Fourier series: 00

IA(X) ~

L n=-oo

where en (x)

= e2rrinx

and

365

anen(x)

[9.7.16]-[9.7.18]

Solutions

Stationary processes

Similarly the indicator function of T- 1 A has a Fourier series, IT-1A(X) ~ Lbnen(X) n

where, using the substitution y = T(x),

-2rrina anen (x ) . I T-IA (x ) ~ '""' ~e n

Now IA = IT-l A since A is invariant. We compare the previous formula with that of (*), and deduce that an = e-2rrina an for all n. Since ex is irrational, it follows that an = 0 if n 1= 0, and therefore fA has Fourier series ao, a constant. Therefore IA is a.s. constant, which is to say that either JP'(A) = 0 or JP'(A) = 1.

16. Let Gt(z) = E(zX(t)), the probability generating function of X(t). Since X has stationary independent increments, for any n (2: 1), X(t) may be expressed as the sum n

X(t)

=

L{X(it/n) - X«i -1)t/n)} i=l

of independent identically distributed variables. Hence X(t) is infinitely divisible. By Problem (5.12.13), we may write Gt(z) = e-).(t)(1-A(z))

for some probability generating function A, and some A(t). Similarly, X(s + t) = X (s) + (X (s + t) - X(s )}, whence Gs+t(z) = Gs(z)Gt(z), implying that = eJ.L(z)t for some /1(z); we have used a little monotonicity here. Combining this with (*), we obtain that Gt(z) = e- M (1-A(z)) for some A. Gt(z)

Finally, X (t) has jumps of unit magnitude only, whence the probability generating function A is given by A(z) = z.

17. (a) We have that X(t) - X(O)

=

{X(s) - X(O)}

+ {X(t) -

X(s)},

0:::: s :::: t,

whence, by stationarity, {m(t) -m(O)}

=

{m(s) -m(O)}

Now m is continuous, so that m(t) - m(O) (b) Take variances of (*) to obtain v(t) a 2.

= (Jt,

+ {m(t -s) -m(O)}.

t 2: 0, for some (J; see Problem (4.14.5). s), 0:::: s :::: t, whence v(t) = a 2t for some

= v(s) + v(t -

18. In the context of this chapter, a process Z is a standard Wiener process if it is Gaussian with Z(O)

= 0, with zero means, and autocovariance function c(s, t) = minIs, t}. 366

Solutions

Problems

(a) Z(t)

[9.7.19]-[9.7.20]

= aW(tla 2 ) satisfies Z(O) = 0, E(Z(t» = 0, and cov(Z(s), Z(t))

= a 2 minis la 2 , t la 2 } = min{s, t}.

(b) The only calculation of any interest here is cov(W(s

(c) YeO)

+ a)- W(a), Wet + a) - W(a») = c(s + a, t + a) - c(a, t + a) - c(s + a, a) + c(a, a) = (s + a) - a - a + a = s, s ::: t.

= 0, and E(V(t» = O. Finally, if s, t cov(V(s), V(t»)

(d) Z(t)

= W(l)

= stcov(W (lIs),

- W(l - t) satisfies Z(O)

> 0,

W(l/t))

= st min{l/s, lit} = min{t, s}.

= 0, E(Z(t» = O.

cov(Z(s), Z(t)) = I - (l - s) - (l - t) = minis, t},

Also Z is Gaussian, and

+ min{l

- s, I - t}

O'::;s,t'::;l.

19. The process W has stationary independent increments, and G(t) = E(IW(t)1 2 ) satisfies G(t) t --* 0 as t --* 0; hence ¢ (u) dW (u) is well defined for any ¢ satisfying

fer

=

It is obvious that ¢(u) = I[O,t](u) and ¢(u) = e-(t-u) I[O,t](u) are such functions. Now X (t) is the limit (in mean-square) of the sequence n-l

Sn(t) = 2:)W«(j

+ l)tln) -

W(jtln)},

n::;:l.

j=o

However Sn(t) = Wet) for all n, and therefore Sn(t) ~ Wet) as n --* 00. Finally, yes) is the limit (in mean-square) of a sequence of normal random variables with mean 0, and therefore is Gaussian with mean O. If s < t, cov(Y(s), Y(t))

=

lX!

(e-(s-u) I[O,s](u») (e-(t-u) I[O,t](u») dG(u)

-los _ 1 ( s-t -e -s-t) . e 2u-s-t d u-ze

o

Y is an Ornstein-Uhlenbeck process.

20. (a) Wet) is N(O, t), so that EIW(t)1

=

lui 1 2 __ e-Z(u It) du =

1 .J2iri 00

v2t l n ,

-00

var(IW(t)l)

= E(W(t)2) - ~ = t 367

(I - ~) .

[9.7.21]-[9.7.21]

Solutions

Stationary processes

The process X is never negative, and therefore it is not Gaussian. It is Markov since, if s < t and B is an event defined in terms of (X(u) : u :::: s}, then the conditional distribution function of X(t) satisfies

I

lP'(X(t) :::: y Xes)

= x, B) = lP'(X(t)

I

+ lP'(X(t) =

= x, B)lP'(W(s) = x IXes) = x, B) y I W(s) = -x, B)lP'(W(s) = -x IXes) = x, B)

:::: y W(s) ::::

H

I

= x) + lP'(X(t)

lP'(X(t) :::: y W(s)

I

:::: y W(s)

= -x)},

which does not depend on B. (b) Certainly, E(Y(t» =

Secondly, W(s)

+ Wet) =

1

00

1

eU

2

1

_ _ e-2(u It) du = e2t. -00 ,J2nt 2W(s) + (Wet) - W(s)} is N(O, 3s + t) if s < t, implying that E(Y(s)Y(t))

and therefore cov(y(s), Y(t))

= E (eW(s)+W(t)) = e~(3s+t), 1 = ez(3s+t) -

1

eZ(s+t),

s < t.

W(I) is N(O, 1), and therefore Y(I) has the log-normal distribution. Therefore Y is not Gaussian. It is Markov since W is Markov, and yet) is a one-one function of Wet).

(c) We shall assume that the random function W is a.s. continuous, a point to which we return in Chapter 13. Certainly, E(Z(t» E(Z(s)Z(t»)

= fot E(W(u» =

11

du

= 0,

E(W(u)W(v») dudv

O
= is

{[U

u=o }v=O

vdv+

t

}v=u

UdV} du

= ~s2(3t -s),

s < t,

since E(W(u)W(v» = min{u, v}. Z is Gaussian, as the following argument indicates. The single random variable Z(t) may be expressed as a limit of the form lim

~ (!.-) W(it/n), n

n-->ooL i=1

each such summation being normal. The limit of normal variables is normal (see Problem (7.11.19», and therefore Z(t) is normal. The limit in (*) exists a.s., and hence in probability. By an appeal to (7.l1.19b), pairs (Z(s), Z(t» are bivariate normal, and a similar argument is valid for all n-tuples of the Z(u). The process Z is not Markov. An increment Z(t) - Z(s) depends very much on W(s) = Z'(s), and the collection (Z(u) : u :::: s} contains much information about Z' (s) in excess of the information contained in the single value Z(s).

21. Let Ui = X(ti). The random variables A = UI, B = U2 - UI, C = U3 - U2, D = U4 - U3 are independent and normal with zero means and respective variances tl, t2 - tl, t3 - t2, t4 - t3. The Jacobian of the transformation is 1, and it follows that UI, U2, U3, U4 have joint density function

e-iQ fu(u)

=

(2n)2 ,Jtl (t2 - tJ}(t3 - t2)(t4 - t3)

368

Solutions

Problems

[9.7.22]-[9.7.22]

where UI

(U2 - U1)2

t1

t2 - t1

Q=-+

+

(U3 - U2)2 t3 - t2

(U4 - U3)2

+~-~

t4 - t3

Likewise U1 and U4 have joint density function

where Hence the joint density function of U2 and U3, given U1

= U4 = 0, is

where 2 ( )2 2 S=~+ u3- u 2 +~.

t2 - t1

t3 - t2

t4 - t3

Now g is the density function of a bivariate normal distribution with zero means, marginal variances (t2 - t1)(t3 - t2) t3 - t1

and correlation

See also Exercise (8.5.2).

22. (a) The random variables (Ij (x) : I ::::: j ::::: n} are independent, so that lE(Fn(x))

= x,

var(Fn(x))

=

I

-var(ll(x))

n

=

x(l - x)

n

.

By the central limit theorem, In{Fn(x) - x} ~ y(x), where Y(x) is N(O, x(1 - x)). (b) The limit distribution is multivariate normal. There are general methods for showing this, and here is a sketch. If Xl < x2 ::::: 1, then the number M2 (= nF(x2)) of the Ij not greater than X2 is approximately N(nx2, nX2(1 - X2)). Conditional on (M2 = mJ, the number M1 = nF(x1) is approximately N(mu, mu(1 - u)) where u = xllx2. It is now a small exercise to see that the pair (M 1 , M2) is approximately bivariate normal with means nX1, nX2, with variances nXl (1 - Xl), nX2(l - X2), and such that

°: :

whence cov(M1 , M2) ~ nXl (I - X2). It follows similarly that the limit of the general collection is multivariate normal with mean 0, variances Xi (1 - Xi), and covariances Cij = Xi (I - Xj). (c) The autocovariance function of the limit distribution is c(s, t) = minis, t} - st, whereas, for 0::::: s ::::: t ::::: 1, we have thatcov(Z(s), Z(t)) = s - ts - st + st = minis, t} - st. It may be shown that the limit of the process { In (Fn (x) - x) : n ~ I} exists as n --+ 00, in a certain sense, the limit being a Brownian bridge; such a limit theorem for processes is called a 'functional limit theorem'.

369

10 Renewals

10.1 Solutions. The renewal equation 1.

Since lE(X d > 0, there exists E (> 0) such that lP'(X 1 2:

E)

>

E.

Let X k = El{Xe~:E}, and denote

by N' the related renewal process. Now N(t) S N' (t), so that lE(eeN(t» S lE(eeN' (t», for & > o. Let Zm be the number of renewals (in N') between the times at which N' reaches the values (m - I)E and mE. The Z's are independent with lE(eezm )

whence lE(eeN' (t» S

(a e {I -

Ee e

-

if(1- E)ee < 1,

-----,;-

- 1 - (1 - E)ee '

(1 - E)ee} -1) tiE for sufficiently small positive &.

2. Let X 1 be the time of the first arrival. If Xl> s, then W = s. On the other hand if Xl < s, then the process starts off afresh at the new starting time Xl. Therefore, by conditioning on the value of Xl, FW(x) =

foOO lP'(W s x

= fos lP'(W

I Xl

= u)dF(u) =

S x - u) dF(u)

if x 2: s. It is clear that F W (x) standard form

=

u)dF(u)

+

100

1· dF(u)

F(s)}

0 if x < s. This integral equation for F W may be written in the

FW(x)

where H and

+ (l -

foS lP'(W s x -

= H(x) + foX Fw(x -

u)dF(u)

F are given by H(x)

= { ~ _ F(s)

if x < s, ifx2:s,

~

F(x)

= {F(X) F(s)

if x < s, ifx2:s.

This renewal-type equation may be solved in the usual way by the method of Laplace-Stieltjes transforms. We have that Fw = H* + FwF*, whence Fw = H* /(1 - F*). If N is a Poisson process then F (x) = 1 - e- Ax . In this case H*(&)

= foOO e-ex dH(x) = e-(A+e)S,

since H is constant apart from a jump at x

= s.

Similarly

370

Solutions [10.1.3]-[10.2.2]

Limit theorems

so that

* FW(8) =

(J... 8

+ 8)e-(HO)s + J...e-(HO)s .

Finally, replace 8 with -8, and differentiate to find the mean. 3.

= n) = IP'(Sn

We have as usual that IP'(N(t)

::::: t) -1P'(Sn+1 ::::: t). In the respective cases,

Lt J 1

IP'(N(t)

(a)

= n) = L

r=O

_ _t

(b)

4.

IP'(N(t) - n) -

By conditioning on Xl, m(t)

Jo

,{e-An(J...n)r - e- A(n+1) [J...(n r.

+ 1)1'},

{J...nbxnb-1 J...(n+1)b x (n+1)b-1}_Ax r(nb) r«n + 1)b) e dx.

= E(N(t))

satisfies

0::::: t

::::: 1.

Hence m' = 1 + m, with solution m(t) = e t - 1, for 0 ::::: t ::::: 1. (For larger values of t, m(t) 1 + m(t - x) dx, and a tiresome iteration is in principle possible.)

JJ

=

With v(t) = E(N(t)2), v(t)

Hence v'

=

l

[v(t -x) +2m(t -x)

= v + 2e t -

+ 1] dx = t +2(e t - t -1) + fot v(x)dx, 0::::: t:::::

1, with solution v(t)

= 1-

et

+ 2tet for 0::::: t

1.

::::: 1.

10.2 Solutions. Limit theorems 1. Let Zi be the number of passengers in the ith plane, and assume that the Zi are independent of each other and of the arrival process. The number of passengers who have arrived by time t is S(t) = l:~f) Zi· Now ~S(t) = N(t) . S(t) -+ E(Zl) a.s. t

t

J1-

N (t)

by the law ofthe large numbers, since N(t)/t -+ 1/ J1- a.s., and N(t) -+ 2.

00

a.s.

We have that

{(t

E(T~) = E

ZiI{M":iJ)

1=1

since I{M":iJ I{M":jJ

2} = fE(zlI{M":iJ) + 2 .~ 1=1

E(ZiZjI{M,,:jJ)

1::51 <) <00

= I{M":iV jJ, where i v j = max {i, j}.

Now

since {M ::::: i - 1} is defined in terms of Zl, Z2,"" Zi-1, and is therefore independent of Zi. Similarly E(Zi Zj I{M,,:jJ) = E(Zj )E(Zi I{M":jJ) = 0 ifi < j. It follows that 00

E(T~)

00

= LE(Zl)IP'(M:::: i) = a 2 LIP'(M :::: i) = a 2 E(M). i=l

i=l

371

[10.2.3]-[10.2.5]

Solutions

Renewals

3. (i) The shortest way is to observe that N(t) + k is a stopping time if k :::: 1. Alternatively, we have by Wald's equation that lE(TN (t)+1) = Il(m(t) + 1). Also

= lE{lE(XN(t)+k IN(t))} = Il,

lE(XN(t)+k)

k:::: 2,

and therefore, for k :::: 1, k

lE(TN(t)+k)

= lE(TN (t)+1) + LlE(XN(t)+j) = Il(m(t) + k). j=2

(ii) Suppose p

1=

1 and p lP'(X 1 =a)= { 1- p

Then Il

=2-

p

1=

ifa=l,

if a

= 2.

1. Also

lE(TN (1))

=

(l - p)lE(To

whereas m(l) = p. Therefore lE(TN(1))

I N(l) = 0) + plE(T1 I N(l) =

1)

=

p,

1= Ilm(l).

4. Let Vet) = N(t) + 1, and let WI. W2, ... be defined inductively as follows. W1 = V(l), W2 is obtained similarly to W1 but relative to the renewal process starting at the V(l)th renewal, i.e., at time TN(1)+1, and Wn is obtained similarly:

+ 1) -

Wn = N(TXn_l

N(TXn_l)

+ 1,

n :::: 2,

whereXm = W1 + W2+···+ W m . For each n, Wn is independent of the sequence W1, W2,···, Wn-I. and therefore the Wn are independent copies of V (1). It is easily seen, by measuring the time-intervals

:s 2:J~1 Wi, and hence

covered, that V (t)

1 - Vet)

t

1 rtl

:s -t L

Wi --+ lE(V(l))

a.s. and in mean, as t --+

00.

i=1

It follows that the family {m- 1 2: =1 Wi : m :::: I} is uniformly integrable (see Theorem (7.10.3)). Now N(t) :s Vet), and so {N(t)/t : t :::: OJ is uniformly integrable also.

1

Since N (t) / t ~ 1l- 1, it follows by uniform integrability that there is also convergence in mean.

5.

(a) Using the fact that lP'(N(t) lE(sN(T))

By integration by parts,

= k) = lP'(Sk :s t)

-lP'(Sk+1

:s t), we find that

(~SklP'(N(t) = k)) ve- vt dt

=

10

=

~ sk {1oOO [lP'(Sk :s t) -lP'(Sk+1 :s t)] ve- vt dt } .

00

JoOO lP'(Sk :s t)ve- vt dt =

lE(sN(T))

=

M( _v)k for k :::: O. Therefore,

I=sk{M(-V)k _ M(_v)k+1} k=O

(b) In this case, lE(sN(T)) = lE(e AT (s-1)) Mr(8) = {v/(v - 8)jb, and lE(SN(T))

= MT ()..(s -1)).

=

(_V_)b v+)..

= 1- M(-v)

When T has the given gamma distribution,

(1- ~)b v+)..

The coefficient of sk may be found by use of the binomial theorem.

372

.

1 - sM(-v)

Solutions

Excess life

[10.3.1]-[10.3.3]

10.3 Solutions. Excess life 1. Let g(y) of E(t),

= lP'(E(t)

> y), assumed not to depend on t. By the integral equation for the distribution

+ y) + g(y) l dF(x). Write h(x) = 1 - F(x) to obtain g(y)h(t) = h(t + y), for y, t :::: O. With t = 0, we have that g(y)h(O) = h(y), whence g(y) = h(y)/ h(O) satisfies g(t + y) = g(t)g(y), for y, t :::: O. Now g is g(y) = 1 - F(t

left-continuous, and we deduce as usual that g(t) = e- At for some A. Hence F(t) = 1 - e- At , and the renewal process is a Poisson process.

(a) Examine a sample path of E. If E(t) = x, then the sample path decreases (with slope -1) until it reaches the value 0, at which point it jumps to a height X, where X is the next interarrival time. Since X is independent of all previous interarrival times, the process is Markovian. (b) In contrast, C has sample paths which increase (with slope 1) until a renewal occurs, at which they drop to O. If C (s) = x and, in addition, we know the entire history of the process up to time s, the time of the next renewal depends only on the length ofthe spent period (Le., x) of the interarrival time in process. Hence C is Markovian.

2.

3.

(a) We have that lP'(E(t) .:sy) =F(t+y)-l G(t+y-x)dm(x)

where G(u) = 1- F(u). Check the conditions of the key renewal theorem (10.2.7): g(t) satisfies: (i) g(t) :::: 0,

10

00

= G(t + y)

Igo[l -

(ii) g(t) dt .:s F(u)] du = E(X 1) < 00, (iii) g is non-increasing. We conclude, by that theorem, that lim lP'(E(t).:s y) t .....

oo

1 = 1- -1 looo g(x)dx = loy -[1F(x)]dx. J-i

0 J-i

0

(b) Integrating by parts,

oo -[1 xr -

loo

J-i

looo -x r +-1 dF(x) =

1 F(x)] dx = J-i

0

r

+1

E(X r + 1) 1

J-i(r

+ 1)

.

See Exercise (4.3.3). (c) As in Exercise (4.3.3), we have that E(E(t)'} E(E(t)r) = E({(X1 - t)+}')

+

= 10

00

l:o 1~0

ryr-1lP'(E(t) > y) dy, implying by (*) that

ryr-1lP'(Xl > t

+y -

x) dm(x) dy,

whence the given integral equation is valid with

1

00

h(u)

=

ryr-1lP'(Xl > u

+ y)dy = E({(X1

- u)+}').

Now h satisfies the conditions of the key renewal theorem, whence lim E(E(t)'} t ..... oo

=.!.. roo h(u) du =.!.. rr yr dF(u + y) du J-i 10 J-i llo
=-

J-i

looo y lP'(Xl >y)dy= E(Xi+ 1) . r

J-i(r

0

373

+ 1)

[10.3.4]-[10.3.5] 4.

Solutions

Renewals

We have that p(E(t»yIC(t)=X)=P(XI>y+xIXI>X)=

1 - F(y +x) , 1 - F(x)

whence

I

lE(E(t) C(t)

5.

= x) =

roo 1- F(y +x) dy = lE{(XI -x)+}.

Jo

1 - F(x)

1 - F(x)

(a) Apply Exercise (10.2.2) to the sequence Xi - fJ.-, 1 ::: i < 00, to obtain var(TM(t) - fJ.-M(t)) =

0-2lE(M(t)).

(b) Clearly TM(t) = t + E(t), where E is excess lifetime, and hence fJ.-M(t) = (t fJ.-M(t)), implying in tum that fJ.-2 var(M(t)) = var(E(t))

where SM(t)

= TM(t)

+ var(SM(t)) -

+ E(t)) -

(TM(t) -

2cov(E(t), SM(I)),

- fJ.-M(t). Now

as t

--+ 00

if lE(Xr) < 00 (see Exercise (lO.3.3c)), implying that 1

- var(E(t)) --+ 0 t

as t

--+ 00.

This is valid under the weaker assumption that lE(Xf) < 00, as the following argument shows. By Exercise (lO.3.3c), lE(E(t)2) = a(t)

+

lol

a(t - u)dm(u),

where a(u) = lE({(XI - u)+}2). Now use the key renewal theorem together with the fact that a(t)::: lE(XfI{x»I}) --+ 0 as t --+ 00. Using the Cauchy-Schwarz inequality,

as t

--+ 00,

by part (a) and (**). Returning to (*), we have that 2 -(m(t) -fJ.-2 var(M(t)) --+ lim {0t 1-+00 t

374

+ 1) }

2

=0-- . fJ.-

Solutions

Renewal-reward processes

[10.4.1]-[10.5.3]

10.4 Solution. Applications 1. Visualize a renewal as arriving after two stages, type 1 stages being exponential parameter A and type 2 stages being exponential parameter IL. The 'stage' process is the flip-flop two-state Markov process of Exercise (6.9.1). With an obvious notation,

Hence the excess lifetime distribution is a mixture of the exponential distribution with parameter IL, and the distribution of the sum of two exponential random variables, thus, fE(t)(X) = Pll (t)g(x)

+ (1 -

Pll (t»lLe-j1.X,

where g(x) is the density function of a typical interarrival time. By Wald's equation, E(t

+ E(t»

= E(SN(t)+l) = E(Xl)E(N(t)

+ 1) = (~+~) (m(t) + 1).

We substitute E(E(t»

=

11) Pll(t) ( - + AIL

+ (1

1

- Pll(t»IL

= -1 +Pll(t) -A

IL

to obtain the required expression.

10.5 Solutions. Renewal-reward processes 1. Suppose, at time s, you are paid a reward at rate u(X(s». By Theorem (10.5.10), equation (10.5.7), and Exercise (6.9.11b), 1

t

Suppose lu(i)1 ::::: K <

I~ l

u(X(s» ds -

00

lot I\X(s)=jJds --+ a.s. 1 - - =Trj. ILjgj

0

for all i

E

S, and let F be a finite subset of the state space. Then

~ TriU(i)1 = I~ u(i) (~ I

I

:::::K~ -1

l

Tt(F)

•

lot I\X(s)=ijds-Tri

iEF 1 t

where is the total time spent in F t S, to obtain the required result.

I\X(s)=ij ds - Tri) I

I

+K

(t - Tt(F») +K~Tri'

o t

Fup to time t. Take the limit as t -+

00

i¢F

using (*), and then as

2. Suppose you are paid a reward at unit rate during every interarrival time of type X, Le., at all times t at which M(t) is even. By the renewal-reward theorem (10.5.1),

lot

1 a.s. E(reward during interarrival time) I M s is even ds --+ t 0 \ () j E(length of interarrival time)

EX 1

= . EX 1 + EYI

3. Suppose, at time t, you are paid a reward at rate C (t). The expected reward during an interval (cycle) of length X is s ds = X2, since the age C is the same at the time s into the interval. The

Il

1

375

[10.5.4]-[10.6.3]

Solutions

Renewals

result follows by the renewal-reward theorem (10.5.1) and equation (10.5.7). The same conclusion is valid for the excess lifetime E(s), the integral in this case being (X - s) ds = ~ X2.

fl

4. Suppose Xo = j. Let VI = min{n :::: 1 : Xn = j, Xm = k for some 1 .::: m < n}, the first visit to j subsequent to a visit to k, and let Vr+l = min{n :::: Vr : Xn = j, Xm = k for some Vr + 1 .::: m < n}. The Vr are the times of a renewal process. Suppose a reward of one ecu is paid at every visit to k. By the renewal-reward theorem and equation (10.5.7),

By considering the time of the first visit to k, lE(V1

I Xo =

j)

= lE(Tk I Xo = j) + lE(1j I Xo

= k).

The latter expectation in (*) is the mean of a random variable N having the geometric distribution JP>(N = n) = p(1 - p)n-l for n :::: 1, where p = JP>(1j < Tk I Xo = k). Since lE(N) = p-l, we deduce as required that 1/JP>(1j < I Xo = k) lfk

=

n

---~'-------'~---

lE(Tk

I Xo = j) + lE(1j I Xo = k)

10.6 Solutions to problems 1. (a) For any n, JP>(N(t) < n) .::: JP>(Tn > t) ---+ 0 as t ---+ 00. (b) Either use Exercise (10.1.1), or argue as follows. Since J-i > 0, there exists JP>(XI > E) > O. For all n,

= 1-

JP>(Tn .::: nE)

E

(> 0) such that

JP>(Tn > nE) .::: 1 - JP>(X I > E)n < 1,

so that, if t > 0, there exists n = net) such that JP>(Tn .::: t) < 1. Fix t and let n be chosen accordingly. Any positive integer k may be expressed in the form k = an + fJ where 0 .::: fJ < n. Now JP>(Tk .::: t) .::: JP>(Tn .::: t)a for an .::: k < (a + l)n, and hence 00

00

k=l

a=O

L JP>(Tk .::: t) .::: L nJP>(Tn .::: t)a <

met) =

00.

(c) It is easiest to use Exercise (10.1.1), which implies the stronger conclusion that the moment generating function of N (t) is finite in a neighbourhood of the origin. 2.

(i) Condition on X I to obtain vet)

= iot

lE{ (N(t - u)

+ 1) 2 } dF(u) = iot

{v(t - u)

+ 2m(t -

u)

+ I} dF(u).

Take Laplace-Stieltjes transforms to find that v* = (v* + 2m* + I)F*, where m* = F* + m* F* as usual. Therefore v* = m*(1 + 2m*), which may be inverted to obtain the required integral equation. (ii) If N is a Poisson process with intensity A, then m (t)

3.

Fix x

E ~.

Then

376

= At, and therefore vet) = (At)2 + At.

Solutions

Problems

[10.6.4]-[10.6.7]

l(t / fL) + x Vta2 / fL3 J. Now,

where aCt) =

lP'(T. ( ) < t) a t -

However a(t)

~ t / fL

as t

--+ 00,

= lP' (TaU)

- fLa(t) < t - fLa(t») .

a J(i(J)

-

a J(i(J)

and therefore t - fLa(t)

---'-==0a J(i(J)

--+

-x

as t

--+ 00,

implying by the usual central limit theorem that lP'(N(t)-(t/ fL ) vta 2 / fL3

~x)

--+ eIl(-x)

as t

--+ 00

where ell is the N(O, 1) distribution function. An alternative proof makes use of Anscombe's theorem (7.11.28). 4.

We have that, for y ::::: t, lP'(C(t) ~

y) = lP'(E(t - y)

= [00 ~[1

Jy

>

y)

lim lP'(E(u) >

--+

u~oo

y)

as t

--+ 00

_ F(x)] dx

fL

by Exercise (10.3.3a). The current and excess lifetimes have the same asymptotic distributions.

5.

Using the lack-of-memory property ofthe Poisson process, the current lifetime C (t) is independent of the excess lifetime E(t), the latter being exponentially distributed with parameter A. To derive the density function of C(t) either solve (without difficulty in this case) the relevant integral equation, or argue as follows. Looking backwards in time from t, the arrival process looks like a Poisson process up to distance t (at the origin) where it stops. Therefore C(t) may be expressed as min{Z, t} where Z is exponential with parameter A; hence JCCt)(s)

and lP'(C(t) = t) = e- At . Now D(t) convolution formula) to be as given.

=

A-AS

={

0e

C(t)

+ E(t),

if s ::::: t, if s > t, whose distribution is easily found (by the

6. The ith interarrival time may be expressed in the form T + Zj where Zj is exponential with parameter A. In addition, Zl, Z2, ... are independent, by the lack-of-memory property. Now 1 - F(x)

= lP'(T + Zl

> x)

= lP'(Zl

> x - T)

= e-A(x-T),

x

~

T.

Taking into account the (conventional) dead period beginning at time 0, we have that k

lP'(N(t)

~ k) =lP'(kT + ~Zj::::: t)

=lP'(N(t -kT)

~ k),

t

~ kT,

1=1

where N is a Poisson process. 7. We have that Xl = L + E(L) where L is the length of the dead period beginning at 0, and E(L) is the excess lifetime at L. Therefore, conditioning on L,

377

[10.6.8]-[10.6.10]

Solutions

Renewals

We have that lP'(E(t)SY)=F(t+ y

)-fo

t

{I-F(t+ y -X)}dm(X).

By the renewal equation, m(t+y)=F(t+y)+

Jt+

y

F(t+y-x)dm(x),

o

whence, by subtraction,

I

t+ y

lP'(E(t)SY)= t

{1-F(t+y-x)}dm(x).

It follows that X

lP'(Xlsx)=l

(X {1-F(x-y)}dm(y)dFL(I)

1=0 Jy=1

+

= [FL(l) IX {I - F(x - y)} dm(y)]X 1

1=0

using integration by parts. The term in square brackets equals

(X FL (I) { 1 - F(x -I)} dm(l)

~

o.

8. (a) Each interarrival time has the same distribution as the sum of two independent random variables with the exponential distribution. Therefore N(t) has the same distribution as L~ M(t)J where M is a Poisson process with intensity A. Therefore met) = ~lE(M(t)) - ilP'(M(t) is odd). Now lE(M(t)) = At, and 00

lP'(M(t) is odd)

=~ ~

(At)2n+l e-At (2n+l)!

= ~e-At(eAt _e- At ).

n=O With more work, one may establish the probability generating function of N(t). (b) Doing part (a) as above, one may see that met) = met).

9. Clearly C(t) and E(t) are independent if the process N is a Poisson process. Conversely, suppose that C(t) and E(t) are independent, for each fixed choice of t. The event (C(t) ::: yJ n (E(t) ::: xJ occurs if and only if E(t - y) ::: x + y. Therefore

lP'(C(t) ::: Take the limit as t ---+ G(y)G(x)

00,

y)lP'(E(t) ::: x)

= lP'(E(t -

y) ::: x

+ y).

remembering Exercise (10.3.3) and Problem (10.6.4), to obtain that ::: 0, where

= G(x + y) if x, y

G(u)

=

1

00

U

1 -[1 - F(v)] dv. /-i

Now 1 - G is a distribution function, and hence has the lack-of-memory property (Problem (4.14.5)), implying that G(u) = e- AU for some A. This implies in turn that [1 - F(u)]j /-i = -G' (u) = Ae- AU , whence /-i = I/A and F(u) = 1 - e- AU •

10. Clearly N is a renewal process if N2 is Poisson. Suppose that N is a renewal process, and write A for the intensity of Nl, and F2 for the interarrival time distribution of N2. By considering the time X 1 to the first arrival of N,

378

Solutions

Problems

[10.6.11]-[10.6.12]

Writing E, Ei for the excess lifetimes of N, Ni, we have that

Take the limit as t

~ 00,

using Exercise (10.3.3), to find that

1

00

x

1 - F(u)] du -[1 J.L

1

00

= e-J...x

x

1 -[1 - F2(U)] du, J.L2

where J.L2 is the mean of F2. Differentiate, and use (*), to obtain

I

which simplifies to give 1 - F2(X) = C xoo[1 - F2(U)] du where c equation has solution F2 (x) = 1 - e-ex.

= AJ.L/(J.L2

- J.L); this integral

11. (i) Taking transfonns of the renewal equation in the usual way, we find that m* 8 _ F*(8) ( ) - 1 - F*(8)

where F*(8)

as 8

~

= JE(e-ex,) = 1 -

8J.L

-----1 1 - F*(8)

+ !8 2(J.L2 + a 2) + 0(8 2)

O. Substitute this into the above expression to obtain

and expand to obtain the given expression. A fonnal inversion yields the expression for m. (ii) The transfonn of the right-hand side of the integral equation is

~J.L8

F1:(8)

+ m*(8) -

F1:(8)m*(8).

By Exercise (10.3.3), F1:(8) = [1 - F*(8)]/(J.L8), and (*) simplifies to m*(8) - (m* - m* F* F*)/(J.L8), which equals m*(8) since the quotient is 0 (by the renewal equation). Using the key renewal theorem, as t ~ 00,

1

1 [1 - FE(t - x)] dm(x) ~ o J.L t

1

2

00

[1 - FE(U)] du

0

by Exercise (1O.3.3b). Therefore,

12. (i) Conditioning on Xl, we obtain

379

=

JE(X2) a + J.L2 --i= 2J.L 2J.L 2

[10.6.13]-[10.6.16]

Solutions

Renewals

Therefore m d* = F d* +m* Fd*. Also m*

=

F* +m* F*, so that

= F d* + m d* F*, the transform of the given integral equation. (ii)ArguingasinProblem(10.6.2), v d* = Fd*+2m* Fd*+v* F d* wherev* = F*(1+2m*)/(l-F*)

whence m d*

is the corresponding object in the ordinary renewal process. We eliminate v* to find that

by (*). Now invert.

13. Taking into account the structure of the process, it suffices to deal with the case I = 1. Refer to Example (10.4.22) for the basic notation and analysis. It is easily seen that f3 = (v - 1»).,. Now F(t) = 1 - e- vAt . Solve the renewal equation (10.4.24) to obtain get)

= h(t) + fot h(t -

x) diii(x)

where iii(x) = vAx is the renewal function associated with the interarrival time distribution Therefore get) = 1, and met) = ef3 t .

F.

z+ p* F*, where F* = FYFz.Solve to obtain * 1- F z p = 1- F*F*

14. We have from Lemma (10.4.5) that p*

=1-

F

y

z

15. The first locked period begins at the time of arrival of the first particle. Since all future events may be timed relative to this arrival time, we may take this time to be O. We shall therefore assume that a particle arrives at 0; call this the Oth particle, with locking time Yo. We shall condition on the time X I of the arrival of the next particle. Now lP'(L > t

I Xl = u) = {

lP'(Yo>t)

ifu>t,

lP'(Yo > u)lP'(L' > t - u)

ifu::::: t,

where L' has the same distribution as L; the second part is a consequence of the fact that the process 'restarts' at each arrival. Therefore lP'(L > t)

= (1- G(t»)lP'(XI

> t)

+ llP'(L

> t - u)(1 - G(u»)fXI (u)du,

the required integral equation. If G(x) = 1 - e-fl- X , the solution is lP'(L > t) = e-fl- t , so that L has the same distribution as the locking times of individual particles. This striking fact may be attributed to the lack-of-memory property of the exponential distribution.

16. (a) It is clear that M(tp) is a renewal process whose interarrival times are distributed as Xl + X2 + ... + XR where lP'(R = r) = pqr-l for r :::: 1. It follows that M(t) is a renewal process whose first interarrival time X(p)

= inf{t

: M(t)

=

I}

=

380

p inf{t : M(tp)

=

I}

Solutions

Problems

[10.6.17]-[10.6.19]

has distribution function 00

p(X(p) ::::: x)

=

L P(R = r)Fr(x/ p). r=1

(b) The characteristic function p of Fp is given by p(t)

=

I=

r=1

I=

=

pq r-1jOO e ixt dFr(t/p)

pq r-1(pt)r

=

r=1

-00

where is the characteristic function of F. Now (pt)

= 1 + i fJpt + o(p)

(t)= l+ifJpt+o(p) p 1-ifJ,t+o(1)

p(pt) 1 - q(pt)

as p ,j, 0, so that

1+0(1) 1-ifJ,t

as p ,j, O. The limit is the characteristic function of the exponential distribution with mean fJ" and the continuity theorem tells us that the process M converges in distribution as p ,j, 0 to a Poisson process with intensity 1/fJ, (in the sense that the interarrival time distribution converges to the appropriate limit). (c) If M and N have the same fdds, then p (t) = (t), which implies that (pt) = (t)/(p + q(t)). Hence 1fr(t) = (t)-1 satisfies 1fr(pt) = q + p1fr(t) for t E R Now 1fr is continuous, and it follows as in the solution to Problem (5.12.15) that 1fr has the form 1fr(t) = 1+,Bt, implying that(t) = (1 +,Bt)-1 for some,B E C. The only characteristic function of this form is that of an exponential distribution, and the claim follows.

17. (a) Let N(t) be the number oftimes the sequence has been typed up to the tth keystroke. Then N is a renewal process whose interarrival times have the required mean fJ,; we have that lE(N(t))/ t -+ fJ,-1 as t -+ 00. Now each epoch of time marks the completion of such a sequence with probability (160)14, so that l i t -lE(N(t)) = - " t t L...J n=14

(1) 100

14

(1)

-+-

14

100

as t -+

00,

implying that fJ, = 1028 . The problem with 'omo' is 'omomo' (i.e., appearances may overlap). Let us call an epoch of time a 'renewal point' if it marks the completion of the word 'omo', disjoint from the words completed at previous renewal points. In each appearance of 'omo', either the first '0' or the second '0' (but not both) is a renewal point. Therefore the probability Un, that n is a renewal point, satisfies (160)3 = Un + Un-2( 160)2. Average this over n to obtain

C~oy =n~moo ~ ~ {un +U n-2 C~or} =; +; c~Or, and therefore fJ, = 106 + 102 . (b) (i) Arguing as for 'omo', we obtain p3 (ii) Similarly, p2q = Un

= Un + PU n-1 + p2 un _2, whence p3 =

+ pqu n-2, so that fJ, =

(1

(1

+ p+ p2)/fJ,.

+ pq)/(p2q).

18. The fdds of (N(u) - N(t) : U ::: t} depend on the distributions of E(t) and of the interarrival times. In a stationary renewal process, the distribution of E(t) does not depend on the value of t, whence (N(u) - N(t) : u ::: t} has the same fdds as (N(u) : u ::: O}, implying that X is strongly stationary.

19. We use the renewal-reward theorem. The mean time between expeditions is B fJ" and this is the mean length of a cycle of the process. The mean cost of keeping the bears during a cycle is iB(B - l)cfJ" whence the long-run average cost is (d + B(B - 1)cfJ,/2}/(BfJ,).

381

11

Queues

11.2 Solutions. M1M11 1. The stationary distribution satisfies 1C = equations

1C P

when it exists, where P is the transition matrix. The

:n: n

=

p:n:n-l

I+p

+ :n:n+l

I+p

for n > 2 - ,

with 2:~O:n:i = I, have the given solution. If p ::: I, no such solution exists. It is slightly shorter to use the fact that such a walk is reversible in equilibrium, from which it follows that 1C satisfies :n:l

for n ::: 1.

:n:o=--,

I+p

(i) This continuous-time walk is a Markov chain with generator given by gO! = eo, gn,n+l = + p) and gn,n-l = en/(l + p) for n ::: I, other off-diagonal terms being O. Such a process is reversible in equilibrium (see Problem (6.15.16», and its stationary distribution v must satisfy vngn,n+l = vn+lgn+l,n' These equations may be written as 2.

enP/(l

for n ::: 1. These are identical to the equations labelled (*) in the previous solution, with:n:n replaced by vnen . It follows that Vn = c:n:n/en for some positive constant C. (ii) If eo = A, en = A + j), for n ::: I, we have that

whence C

= 2A and the result follows.

3. Let Q be the number of people ahead of the arriving customer at the time of his arrival. Using the lack-of-memory property of the exponential distribution, the customer in service has residual servicetime with the exponential distribution, parameter j)" whence W may be expressed as S1 + S2 + ... + SQ, the sum of independent exponential variables, parameter j),. The characteristic function of W is tPw(t)

= IE{IE(e itW I Q)} = IE {

(j),

~

iJ Q}

I-p =(l-P)+P( j),-A ). 1 - pj),/(j), - it) j), - A - it

382

MIMI]

Solutions

[11.2.4]-[11.2.7]

This is the characteristic function of the given distribution. The atom at 0 corresponds to the possibility that Q = O.

4. We prove this by induction on the value of i + j. If i + j = 0 then i = j = 0, and it is easy to check that 1T(0; 0, 0) = I and A(O; 0,0) = I, A(n; 0, 0) = 0 for n ::: 1. Suppose then that K ::: I, and that the claim is valid for all pairs (i, j) satisfying i + j = K. Leti and j satisfy i + j = K + 1. The last ball picked has probability i / (i + j) of being red; conditioning on the colour of the last ball, we have that

i~j1T(n-l;i-I,j)+ i~j1T(n+l;i,j-I).

1T(n;i,j) =

Now (i - I)

+j

+ (j

= K = i

1T(n; i, j) = i

~j

- I). Applying the induction hypothesis, we find that

{A(n - I; i-I, j) - A(n; i-I, j)}

+i~j

{A(n

+ I; i, j

- I) - A(n

+ 2; i, j

-

1)}.

Substitute to obtain the required answer, after a little cancellation and collection of terms. Can you see a more natural way?

5. Let A and B be independent Poisson process with intensities Aand j.J., respectively. These processes generate a queue-process as follows. At each arrival time of A, a customer arrives in the shop. At each arrival-time of B, the customer being served completes his service and leaves; if the queue is empty at this moment, then nothing happens. It is not difficult to see that this queue-process is M(A)/M(j.J.,)/1. Suppose that A(t) = i and B(t) = j. During the time-interval [0, t], the order of arrivals and departures follows the schedule of Exercise (11.2.4), arrivals being marked as red balls and departures as lemon balls. The imbedded chain has the same distributions as the random walk of that exercise, and it follows that JlD(Q(t) = n [ A(t) = i, B(t) = j) = 1T(n; i, j). Therefore Pn(t) = L.i,j 1T(n; i, j)JlD(A(t) = i)JlD(B(t) = j). 6.

With p = A/ j.J." the stationary distribution of the imbedded chain is, as in Exercise (11.2.1),

~ 1Tn=

{~(I-P) ~(I_p2)pn-l

ifn=O, ifn:::1.

In the usual notation of continuous-time Markov chains, go = A and gn = A + j.J., for n ::: I, whence, by Exercise (6.10.11), there exists a constant c such that 1To =

;A

(1- p),

1Tn =

2(A: j.J.,)

(1- p2)pn-l

for n ::: 1.

Now L.i 1Ti = 1, and therefore c = 2A and 1Tn = (1 - p )pn as required. The working is reversible.

7. (a) Let Qi (t) be the number of people in the ith queue attime t, including any currently in service. The process Q 1 is reversible in equilibrium, and departures in the original process correspond to arrivals in the reversed process. It follows that the departure process of the first queue is a Poisson process with intensity A, and that the departure process of Q 1 is independent of the current value of Q 1. (b) We have from part (a) that, for any given t, the random variables Ql (t), Q2(t) are independent. Consider an arriving customer when the queues are in equilibrium, and let Wi be his waiting time (before service) in the ith queue. With T the time of arrival, and recalling Exercise (11.2.3), JlD(W I = 0, W2 = 0) > JlD(Qi(T) = 0 for i = 1,2) = JlD(QI (T) = 0)JlD(Q2(T) = 0) = (1 - Pl)(l - P2) = JlD(WI = 0)JlD(W2 = 0). Therefore WI and W2 are not independent. There is a slight complication arising from the fact that T is a random variable. However, T is independent of everybody who has gone before, and in particular of the earlier values of the queue processes Qi.

383

[11.3.1]-[11.4.1]

Solutions

Queues

11.3 Solutions. MlG!l 1.

In equilibrium, the queue-length Qn just after the nth departure satisfies

where An is the number of arrivals during the (n + l)th service period, and h(m) Qn and Qn+l have the same distribution. Take expectations to obtain

o = E(An) where E(An) expectations:

Use the facts that An is independent of Qn, and that Qnh(Qn)

+ J....d} + lP'(Qn

> 0)

8m o. 'Now

lP'(Qn > 0),

= J....d, the mean number of arrivals in an interval oflength d.

0= {(J....d)2

=I-

+ 2{ (J....d -

Next, square (*) and take

= Qn, to find that

I)E(Qn) - J....dlP'(Qn > O)}

and therefore, by (**),

2. From the standard theory, MB satisfies MB(S) = Ms(s - J.... + J....MB(S», where Ms(8) /L/(/L -8). Substitute to find that x = MB(S) is a root of the quadratic J....x 2 -x(J.... + /L-s) + /L = O. For some small positive s, M B (s) is smooth and non-decreasing. Therefore M B (s) is the root given. 3. Let Tn be the instant of time at which the server is freed for the nth time. By the lack-of-memory property of the exponential distribution, the time of the first arrival after Tn is independent of all arrivals prior to Tn, whence Tn is a 'regeneration point' of the queue (so to say). It follows that the times which elapse between such regeneration points are independent, and it is easily seen that they have the same distribution.

11.4 Solutions. GIM/l 1. The transition matrix of the imbedded chain obtained by observing queue-lengths just before arrivals is

... ...

The equation 1r

= 1r P A

)

may be written as 00

lfn

= LQ(ilfn+i-l

forn 2: 1.

i=O

It is easily seen, by adding, that the first equation is a consequence of the remaining equations, taken in conjunction with ~O" lfi = 1. Therefore 1r is specified by the equation for lfn , n 2: 1.

384

Solutions [11.4.2]-[11.5.2]

GIGI] The indicated substitution gives 00

en

= e n-] Lai ei i=O

which is satisfied whenever

e satisfies

It is easily seen that A(e) = Mx(JL(e - 1)) is convex and non-decreasing on [0,1], and satisfies A(O) > 0, A(1) = 1. Now A'(1) = JLE(X) = p-] > 1, implying that there is a unique 71 E (0,1) such that A(71) = 71. With this value of 71, the vector 1r given by Trj = (1 - 71)71j, j ::: 0, is a stationary distribution of the imbedded chain. This 1r is the unique such distribution because the chain is irreducible.

(i) The equilibrium distribution is Trn = (1- 71)71n for n ::: 0, with mean 2:~o nTrn = 71/(1- 71). (ii) Using the lack-of-memory property of the service time in progress at the time of the arrival, we see that the waiting time may be expressed as W = S] + S2 + ... + S Q where Q has distribution lr, given above, and the Sn are service times independent of Q. Therefore

2.

E(W)

= E(S])E(Q) =~. (1 - 71)

3. We have that Q(n+) = 1 + Q(n-) a.s. foreachintegern, whence limt--+oolP'(Q(t) = m) cannot exist. Since the traffic intensity is less than 1, the imbedded chain is ergodic with stationary distribution as in Exercise (11.4.1).

11.5 Solutions. G/GIl 1. Let Tn be the starting time of the nth busy period. Then Tn is an arrival time, and also the beginning of a service period. Conditional on the value of Tn, the future evolution of the queue is independent of the past, whence the random variables {Tn+] - Tn : n ::: l} are independent. It is easily seen that they are identically distributed. 2. If the server is freed at time T, the time] until the next arrival has the exponential distribution with parameter JL (since arrivals form a Poisson process). By the duality theory of queues, the waiting time in question has moment generating function Mw(s) = (1 - 0/(1 - ~ M/(s)) where M/(s) = JL/(JL - s) and ~ = IP'(W > 0). Therefore, Mw(s)

= ~JL(1- 0 + (1- 0JL(1-

0 - s

the moment generating function of a mixture of an atom at 0 and an exponential distribution with parameter JL(1 - O· If G is the probability generating function of the equilibrium queue-length, then, using the lackof-memory property of the exponential distribution, we have that Mw(s) = G(JL/(JL - s)), since W is the sum of the (residual) service times of the customers already present. Set u = JL 1(JL - s) to find that G(u) = (1 - 0/(1 - ~u), the generating function of the mass function f(k) = (1 - O~k for k ::: O.

385

[11.5.3]-[11.7.2]

Solutions

Queues

It may of course be shown that ~ is the smallest positive root of the equation x = Mx (/L(x - 1)), where X is a typical interarrival time. 3.

We have that 1 - G(y)

= lP'(S -

X> y)

=

10

00

lP'(S > u

+ y)dFx(u),

Y

E

JR,

where S and X are typical (independent) service and interarrival times. Hence, formally, dG(y) = -

since fs(u

roo dlP'(S > u

Jo

+ y) = e-JL(u+y) if u >

+ y)dFx(u) =

dy

Joo-y

/Le-JL(u+y) dFx(u),

-y, and is 0 otherwise.

With F as given,

J x

F(x - y)dG(y)

=

-00

rr

JJ -oo
{I -

7Je-JL(1-'l)(x-Y)}/Le-JL(u+y) dFx(u)dy.

-Y
First integrate over y, then over u (noting that FX (u) to F(x), when x ~ O.

= 0 for u

< 0), and the double integral collapses

11.6 Solution. Heavy traffic 1.

Qp has characteristic function 00

¢ (t) = ~ e itn pn(1 _ p) = L.J n=O

p

1- p . 1 - pelt

Therefore the characteristic function of (1 - p)Qp satisfies ¢p((1- p)t)

=

1- p 1 1 _ pei(l-p)t -+ 1 - it

aspt1.

The limit characteristic function is that of the exponential distribution, and the result follows by the continuity theorem.

11.7 Solutions. Networks of queues 1. The first observation follows as in Example (11.7.4). The equilibrium distribution is given as in Theorem (11.7.14) by c

rr (n)

=

n

i=]

ni -a-

a· e ' ---','----wI

,.

the product of Poisson distributions. This is related to Bartlett's theorem (see Problem (8.7.6)) by defining the state A as 'being in station i at some given time'. 2. The number of customers in the queue is a birth-death process, and is therefore reversible in equilibrium. The claims follow in the same manner as was argued in the solution to Exercise (11.2.7).

386

Solutions

Problems

[11.7.3]-[11.8.1]

3. (a) We may take as state space the set {O, 1', I",2,3, ... }, where i E {O, 2,3, ... } is the state of having i people in the system including any currently in service, and l' (respectively I") is the state of having exactly one person in the system, this person being served by the first (respectively second) server. It is straightforward to check that this process is reversible in equilibrium, whence the departure process is as stated, by the argument used in Exercise (11.2.7). (b) This time, we take as state space the set {O', 0",1', 1",2,3, ... } having the same states as in part (a) with the difference that 0' (respectively 0") is the state in which there are no customers present and the first (respectively second) server has been free for the shorter time. It is easily seen that transition from 0' to I" has strictly positive probability whereas transition from I" to 0' has zero probability, implying that the process is not reversible. By drawing a diagram of the state space, or otherwise, it may be seen that the time-reversal of the process has the same structure as the original, with the unique change that states 0' are 0" are interchanged. Since departures in the original process correspond to arrivals in the time-reversal, the required properties follow in the same manner as in Exercise (11.2.7). 4. The total time spent by a given customer in service may be expressed as the sum of geometrically distributed number of exponential random variables, and this is easily shown to be exponential with parameter 8 J1-. The queue is therefore in effect a M (A)/M (8 J1-)/1 system, and the stationary distribution is the geometric distribution with parameter p = A/(8J1-), provided p < 1. As in Exercise (11.2.7), the process of departures is Poisson. Assume that rejoining customers go to the end of the queue, and note that the number of customers present constitutes a Markov chain. However, the composite process of arrivals is not Poisson, since increments are no longer independent. This may be seen as follows. In eqUilibrium, the probability of an arrival of either kind during the time interval (t, t + h) is Ah + PJ1-(l- 8)h +o(h) = (A/8)h +o(h). If there were an arrival of either kind during (t - h, t), then (with conditional probability 1 - O(h)) the queue is non-empty at time t, whence the conditional probability of an arrival of either kind during (t, t + h) is Ah + J1-(I - 8)h + o(h); this is of a larger order of magnitude than the earlier probability (A/8)h + o(h). 5. For stations r, s, we write r --* s if an individual at r visits s at a later time with a strictly positive probability. Let C comprise the station j together with all stations i such that i --* j. The process restricted to C is an open migration process in equilibrium. By Theorem (11.7.19), the restricted process is reversible, whence the process of departures from C via j is a Poisson process with some intensity 1;. Individuals departing C via j proceed directly to k with probability Ajkl/Yj (nj) J1-jl/Yj(nj)

Ajk

+ 2:rEtC Ajrl/Yj(nj)

= J1-j

+ 2:rEtC Ajr'

independently of the number n j of individuals currently at j. Such a thinned Poisson process is a Poisson process also (cf. Exercise (6.8.2)), and the claim follows.

11.8 Solutions to problems 1. Although the two cases may be done together, we choose to do them separately. When k = 1, the equilibrium distribution 'lC satisfies: J1-'lC1 - A'lCO = 0, 1::: n < N, + J1-)'lCn + A'lCn-l = 0, -J1-'lCN + A'lCN-l = 0, a system of equations with solution 'lCn = 'lCo(A./ J1-)n for 0::: n ::: N, where (if A =I J1-) J1-'lC n+l - (A

N 'lC- 1 = ~(A/ )n o J1-

;;0

1

(A/ )N+l J11-(A./J1-)

= -

387

[11.8.2]-[11.8.3]

Solutions

Queues

Now let k = 2. The queue is a birth-death process with rates A A' - { I 0

ifi < N, ifi 2: N,

if i = 1, ifi 2: 2.

It is reversible in equilibrium, and its stationary distribution satisfies AilTi = fLi+llTi+l. We deduce that lTi = 2pilTO for 1 ~ i ~ N, where P = A/(2fL) and N

lTal

= 1 + L2pi. i=1

2. The answer is obtainable in either case by following the usual method. It is shorter to use the fact that such processes are reversible in equilibrium. (a) The stationary distribution:7r satisfies lTnAp(n) = lTn+l fL for n 2: 0, whence lTn = lTopn In! where p = A/fL. Therefore lT n = pn e - p In!. (b) Similarly, n-l lT n

= lTopn

II p(m) = lTOpn2-~n(n-1),

n 2: 0,

m=O

where 1

00

-1 _ ~

lTo

- L.J P n=O

n(l) 2 n (n-l) 2 .

At the instant of arrival of a potential customer, the probability q that she joins the queue is obtained by conditioning on its length: 00

q

=L

00

p(n)lTn

n=O

= lTO L

1

p n 2- n -Z- n (n-l)

00

= lTO L

n=O

1

p n 2-Z- n (n+l)

n=O

1

= lTO-(lTal -

I}.

p

3.

First method. Let (Ql, Q2) be the queue-lengths, and suppose they are in equilibrium. Since Ql is a birth-death process, it is reversible, and we write Ql (t) = Ql (-t). The sample paths of

Q1 have increasing jumps of size 1 at times of a Poisson process with intensity A; these jumps mark arrivals at the cash desk. By reversibility, Ql has the same property; such increasing jumps for Ql are decreasing jumps for Q1, and therefore the times of departures from the cash desk form a Poisson process with intensity A. Using the same argument, the quantity Ql (t) together with the departures prior to t have the same joint distribution as the quantity Q1 (-t) together with all arrivals after -t. However Ql (-t) is independent of its subsequent arrivals, and therefore Ql (t) is independent of its earlier departures. It follows that arrivals at the second desk are in the manner of a Poisson process with intensity A, and that Q2 (t) is independent of Q I (t). Departures from the second desk form a Poisson process also. Hence, in equilibrium, QI is M(A)IM(fLI)/l and Q2 is M(A)IM(fL2)/1, and they are independent at any given time. Therefore their joint stationary distribution is

where Pi = A/fLi· Second method. The pair (QI (t), Q2(t» (lTmn : m, n 2: 0) satisfies (A

is a bivariate Markov chain. A stationary distribution

+ fLl + fL2) lTmn = AlTm-l,n + fLIlTm+l,n-1 + fL2 lTm,n+l, 388

m,n2:1,

Solutions

Problems

[11.8.4]-[11.8.5]

together with other equations when m = 0 or n = O. It is easily checked that these equations have the solution given above, when Pi < 1 for i = 1, 2. 4. Let Dn be the time of the nth departure, and let Qn = Q(Dn +) be the number of waiting customers immediately after Dn. We have in the usual way that Qn+] = An + Qn - h(Qn), where An is the number of arrivals during the (n + l)th service time, and hex) = minIx, m}. Let G(s) = ~~o rrisi be the equilibrium probability generating function of the Qn. Then, since Qn is independent of An,

where E(SAn) =

10

00

= MS(A(S

eAU(s-]) fs(u)du

-1)),

Ms being the moment generating function of a service time, and

Combining these relations, we obtain that G satisfies

smG(s)

=

MS(A(S

-1)) {G(S) + f:,(Sm

- si)rri}'

1=0

whenever it exists. Finally suppose that m

= 2 and Ms(()) = /.1)(/1 G(s)

=

()). In this case,

+ 1) + rrIS} + 1) - As2

/1{rro(s /1(s

Now G(1) = 1, whence /1(2rro + rrj} = 2/1 - A; this implies in particular that 2/1 - A > O. Also G(s) converges for [s [ :'S 1. Therefore any zero of the denominator in the interval [-1, 1] is also a zero of the numerator. There exists exactly one such zero, since the denominator is a quadratic which takes the value -A at s = -1 and the value 2/1 - A at s = 1. The zero in question is at

and it follows that rro

+ (rro + rr] )so = O.

Solving for rro and rr], we obtain I-a G(s) = - - , I-as

where a 5.

= n/ {/1 + y' /1 2 + 4A/1}.

Recalling standard M/G/l theory, the moment generating function M B satisfies MB(S)

= Ms (s - A + AMB(S)) = ____/1_ _ __ /1 - (s - A + AMB(S)}

whence MB(S) is one of (A

+ /1 -

s)

± y' (A

+ /1 -

n 389

s)2 - 4A/1

[11.8.6]-[11.8.7]

Queues

Solutions

Now M B (s) is non-decreasing in s, and therefore it is the value with the minus sign. The density function of B may be found by inverting the moment generating function; see Feller (1971, p. 482), who has also an alternative derivation of M B. As for the mean and variance, either differentiate MB, or differentiate (*). Following the latter route, we obtain the following relations involving M (= MB):

+ M + (s -A - fL)M' = 2AMM" + 2A(M')2 + 2M' + (s - A - fL)M" = 2AMM'

Sets

= 0 to obtain M'(O) =

(fL-A)-l and M"(O)

0,

O.

= 2fL(fL-A)-3, whence the claims are immediate.

6. (i) This question is closely related to Exercise (11.3.1). With the same notation as in that solution, we have that

where hex) = min{l, x}. Taking expectations, we obtain JID(Qn > 0) = JB:(An) where

= 10

JB:(An)

00

= s) dFs(s) = AJB:(S) = p,

JB:(An [ S

and S is a typical service time. Square (*) and take expectations to obtain

where JB:(A~) is found (as above) to equal p

+ A2JB:(S2).

(ii) If a customer waits for time Wand is served for time S, he leaves behind him a queue-length which is Poisson with parameter A(W + S). In equilibrium, its mean satisfies AJB:(W + S) = JB:(Qn),

whence JB:(W) is given as claimed. (iii) JB:(W) is a minimum when JB:(S2) is minimized, which occurs when S is concentrated at its mean. Deterministic service times minimize mean waiting time. 7. Condition on arrivals in (t, t+h). If there are no arrivals, then Wt+h :s x if and only ifWt :s x+h. If there is an arrival, and his service time is S, then Wt+h :s x if and only if Wt :s x + h - S. Therefore F(x; t

+ h) = (1 -

(x+h

Ah)F(x

+ h; t) + Ah Jo

F(x

+h -

s; t) dFs(s)

+ o(h).

Subtract F(x; t), divide by h, and take the limit as h ,!- 0, to obtain the differential equation. We take Laplace-Stieltjes transforms. Integrating by parts, for e :s 0,

(

/)x dh(x) = -h(O) -

e{Mu(e) - H(O)),

J(O,oo)

(

eex dH(x)

= Mu(e) -

H(O),

J(O,oo)

(

eex dJID(U

+ S :s x) = Mu(e)Ms(e),

J(O,oo)

and therefore 0= -h(O) - e{Mu(e) - H(O)}

+ AH(O) + AMu(e){Ms(e) -

390

I}.

Solutions [11.8.8]-[11.8.10]

Problems

Set 8

= 0 to obtain that h(O) = AH(O), and therefore H(O)

I

= -eMu(8){A(Ms(8) - I) - 8}.

Take the limit as 8 --+ 0, using L'Hopital's rule, to obtain H(O) = 1- AE(S) = I - p. The moment generating function of U is given accordingly. Note that Mu is the same as the moment generating function of the equilibrium distribution of actual waiting time. That is to say, virtual and actual waiting times have the same equilibrium distributions in this case.

In this case U takes the values I and -2 each with probability ~ (as usual, U = S - X where S and X are typical (independent) service and interarrival times). The integral equation for the limiting waiting time distribution function F becomes

8.

F(O)

= ~ F(2),

F(x)

The auxiliary equation is 8 3 - 28 can contribute, whence

= HF(x -

I)

+ F(x + 2)}

+ I = 0, with roots I and F(x)

=A+B

~ (1 ±

for x

JS).

= 1,2, ....

Only roots lying in [ -I, I]

(-I: J"Sr

for some constants A and B. Now F (x) --+ I as x --+ 00, since the queue is stable, and therefore A = 1. Using the equation for F(O), we find that B = ~(I - J"S).

Q is a M(A)IM(fL)/oo queue, otherwise known as an immigration-death process (see Exercise (6.11.3) and Problem (6.15.18)). As found in (6.15.18), Q(t) has probability generating function

9.

G(s, t)

where p

= AI fL.

= {I + (s -

I)e-I-U} I exp{p(s - 1)(1 - e- Ilt )}

Hence E(Q(t))

= Ie- Ilt + p(1 -

lP'(Q(t)

= 0) =

lP'(Q(t)

= n)

--+

e- Ilt ),

(1 - e- Ilt / exp{ -p(1 - e- Ilt )}, I _pn e -

p

n!

as t --+

00.

IfE(l) and E(B) denote the mean lengths of an idle period and a busy period in equilibrium, we have that the proportion of time spent idle is E(l)/{E(l) + E(B)). This equals limt-+oolP'(Q(t) = 0) = e- p . Now E(l) = A-I, by the lack-of-memory property of the arrival process, so that E(B) = (e P - 1)1 A.

10. We have in the usual way that Q(t

+ I) = At + Q(t) -

min{l, Q(t))

where At has the Poisson distribution with parameter A. When the queue is in equilibrium, E(Q(t)) + I)), and hence

=

E(Q(t

lP'(Q(t) > 0) = E(min{l, Q(t))) = E(A t ) = A.

We have from (*) that the probability generating function G(s) of the equilibrium distribution of Q(t) (= Q) is

391

[11.8.11]-[11.8.13]

Solutions

Queues

Also, G(s)

and hence G(s)

= E(sQ I{Q2:l)) + lP'(Q = 0),

= eA(S-!) {~G(S) + (1 - ~)

whence G(s)

=

(1 -

A) }

(1-S)(1-A) 1 _ se-A(s-!) .

The mean queue length is G' (1) = iA(2 - A)/(1 - A). Since service times are of unit length, and arrivals form a Poisson process, the mean residual service time of the customer in service at an arrival time is so long as the queue is non-empty. Hence

i,

A

E(W) = E(Q) - ilP'(Q > 0) = - - 2(1 - A)

11. The length B of a typical busy period has moment generating function satisfying M B (s) = exp{s - A + AMB (s)}; this fact may be deduced from the standard theory of M/G/1, or alternatively by a random-walk approach. Now T may be expressed as T = I + B where I is the length of the first idle period, a random variable with the exponential distribution, parameter A. It follows that MT(S) = AMB(S)/(A - s). Therefore, as required, (A - S)MT(S)

= A exp{s -

A + (A - s)MT(S)}.

If A :::: 1, the queue.llength at moments of departure is either null persistent or transient, and it follows that E(T) = 00. If A < 1, we differentiate (*) and set s = 0 to obtain AE(T) - 1 = A2E(T), whence E(T) = {A(1 - A)}-!.

12. (a) Q is a birth-death process with parameters Ai equilibrium; see Problems (6.15.16) and (11.8.3).

= A, P,i = p"

and is therefore reversible in

(b) The equilibrium distribution satisfies AJTi = P,JTi+! for i :::: 0, whence JTi = (1 - p)pi where p = A/ p,. A typical waiting time W is the sum of Q independent service times, so that MW(s)

= GQ(Ms(s)) =

I-p I-pp,/(p,-s)

=

(1-p)(p,-s) p,(I-p)-s

.

(c) See the solution to Problem (11.8.3). (d) Follow the solution to Problem (11.8.3) (either method) to find that, at any time t in equilibrium, the queue lengths are independent, the jth having the equilibrium distribution ofM(A)/M(p,j )/1. The joint mass function is therefore K

!(X!, X2, .. ·, XK)

=

II (1- Pj)p? j=!

where Pj

= A/p,j.

13. The size of the queue is a birth-death process with rates Ai = A, P,i = P, min{i, k). Either solve the equilibrium equations in order to find a stationary distribution JT, or argue as follows. The process is reversible in equilibrium (see Problem (6.15.16)), and therefore AiJTi = P,i+!JTi+! for all i. These 'balance equations' become if 0 ::: i < k, if i :::: k.

392

Solutions

Problems

[11.8.14]-[11.8.14]

These are easily solved iteratively to obtain

Jri

={

Jroai Ii!

if 0 ::: i ::: k,

Jro(al k)i kk I k!

ifi :::: k

where a = AI IL. Therefore there exists a stationary distribution if and only if A < kIL, and it is given accordingly, with Jr

a= ]

k-] i ",a L -:-;i=O I.

kk

00

" . + kI 'L(alk)l. .

i=k

The cost of having k servers is

where Jro

= Jro(k).

One finds, after a little computation, that Ba C] = A + - - ,

I-a

Therefore C2 - C]

2Ba 2

C2 =2A+ - - 2 ' 4-a

a 3 (A - B) + a 2 (2B - A) - 4a(A + B) + 4A = --'---------'-------,---'----,------:-'--,.,-----'---------'----2

(1 - a)(4 - a )

Viewed as a function of a, the numerator is a cubic taking the value 4A at a = 0 and the value -3B at a = 1. This cubic has a unique zero at some a* E (0,1), and C] < C2 if and only if 0 < a < a*.

14. The state of the system is the number Q (t) of customers within it at time t. The state 1 may be divided into two sub-states, being 0'] and 0'2, where O'i is the state in which server i is occupied but the other server is not. The state space is therefore S = {O, 0'],0'2,2,3, ... ). The usual way of finding the stationary distribution, when it exists, is to solve the equilibrium equations. An alternative is to argue as follows. If there exists a stationary distribution, then the process is reversible in equilibrium if and only if

for all sequences i], i2, ... , ik of states, where G = (guV)U,VES is the generator of the process (this may be shown in very much the same way as was the corresponding claim for discrete-time chains in Exercise (6.5.3); see also Problem (6.15.16)). It is clear that (*) is satisfied by this process for all sequences of states which do not include both 0'] and 0'2; this holds since the terms guv are exactly those of a birth-death process in such a case. In order to see that (*) holds for a sequence containing both 0'] and 0'2, it suffices to perform the following calculation: gO,al gal ,2g2,a2ga2,0

= (i A)AIL2IL] = gO,a2 ga2,2g2,al gal ,0·

Since the process is reversible in equilibrium, the stationary distribution =I v. Therefore

Jr

satisfies Jruguv

Jrvgvu for all u, v E S, u

and hence A

Jral

= -2 Jro, IL]

A

Jra2

= -2 JrO, IL2

Jru

A2 = --2IL]IL2

393

(

A IL]

+ IL2

)U-2

JrO

for u :::: 2.

[11.8.15]-[11.8.17]

Solutions

Queues

This gives a stationary distribution if and only if A < JLI + JL2, under which assumption no is easily calculated. A similar analysis is valid if there are s servers and an arriving customer is equally likely to go to any free server, otherwise waiting in tum. This process also is reversible in equilibrium, and the stationary distribution is similar to that given above.

15. We have from the standard theory that Q/J- has as mass function nj

=

T/ is the smallest positive root of the equation x (1- JL-I)Q/J- is

e/J-(x -I).

M/J-(B)=E(exp{B(l-JL-I)Q/J-}) =

= (l

- T/)T/j, j :::: 0, where The moment generating function of 1-T/

1 - T/e 0 (1-/J-

-I.

Writing JL = 1 + E, we have by expanding e/J-(TJ-I) as a Taylor series that T/ as E -!- O. This gives

M/J-(B)

as

E

=

2E + O(E) 1-(l-2E)(1+BE)+0(E)

=

)

= T/(E) =

1 - 2E + O(E)

2E + O(E) 2 (2-B)E+O(E) -+ 2-B

-!- 0, implying the result, by the continuity theorem.

16. The numbers P (of passengers) and T (of taxis) up to time t have the Poisson distribution with respective parameters nt and if. The required probabilities Pn = IP'(P = T + n) have generating function 00

00

L L n=-oo

IP'(P

= m + n)lP'(T = m)zn

n=-OOm=O 00

=

L

IP'(T

= m)z-mG p(z)

m=O

= GT(z-I)Gp(z) = e-(1T+r)t e (Jrz+rz- l )t, in which the coefficient of zn is easily found to be that given.

17. Let N(t) be the number of machines which have arrived by time t. Given that N(t) = n, the times TI, T2, ... , Tn of their arrivals may be thought of as the order statistics of a family of independent uniform variables on [0, t], say UI, U2, ... , Un; see Theorem (6.12.7). The machine which arrived at time Ui is, at time t, in the x-stage} { aCt) in the Y -stage with probability f3 (t) repaired 1 - aCt) - f3(t)

=

=

IP'(U + X > t) and f3(t) IP'(U + X :S t < U + X + Y), where U is uniform on [0, t], where aCt) and (X, Y) is a typical repair pair, independent of U. Therefore IP'(U(t)

= .

Vet)

= k I N(t) = n) =

n!a(t)j f3(ti(l-a(t) - f3(t))n-k-j ' J·'k'( . . n - ]·-k)'.

J, implying that 00

IP'(U(t)

= j, Vet) = k) =

L

n=O

e-At ()..t)n

,

n.

]

.,. 394

IP'(U(t)

= j, Vet) = k I N(t) = n) k!

Solutions [11.8.18]-[11.8.19]

Problems

18. The maximum deficit Mn seen up to and including the time of the nth claim satisfies Mn = max { Mn-I, t(Kj - Xj)} = max{O, UI, UI

+ U 2 ,···, UI + U2 + ... + Un},

J=I

where the Xj are the inter-claim times, and Uj = Kj - Xj. We have as in the analysis of G/G/l that Mn has the same distribution as Vn = max{O, Un, Un + Un-I, ... , Un + Un-I + ... + UI}, whence Mn has the same distribution as the (n + l)th waiting time in a M(A)/G/l queue with service times Kj and interarrival times Xj. The result follows by Theorem (11.3.l6).

19. (a) Look for a solution to the detailed balance equations AJri = (i + l)/LJri+l, 0 ::: i < s, to find that the stationary distribution is given by Jri = (pi / i !)JrO' (b) Let pc be the required fraction. We have by Little's theorem (10.5.18) that

and PI = Jrl, where Jrs is the probability that channels 1,2, ... , s are busy in a queue MIM/s having the property that further calls are lost when all s servers are occupied.

395

12 Martingales

12.1 Solutions. Introduction 1. (i) We have that JK(Ym ) = JK{JK(Ym+l [ Tm)} = JK(Ym+l), and the result follows by induction. (ii) For a submartingale, JK(Ym) ::: JK{JK(Ym+l [ Tm)} = JK(Ym+l), and the result for supermartingales follows similarly.

2.

We have that

if m 2: I, since Tn S; T n+m -l. Iterate to obtain JK(Yn+m [ Tn)

3.

= JK(Yn

[ Tn)

= Yn ·

(i) 2n jJ., -n has mean I, and

where Tn

= a(2 1 , 22, ... , 2n).

(ii) Certainly T}Zn ::: 1, and therefore it has finite mean. Also,

where the Xi are independent family sizes with probability generating function G. Now G(T}) and the claim follows.

4.

=

T},

(i) With Xn denoting the size of the nth jump,

where Tn

= a(X 1 , X2, ... , Xn). Also JK[Sn[ = var(Sn) = n, and

::: n, so that {Sn} is a martingale.

(ii) Similarly JK(S~)

(iii) Suppose the walk starts at k, and there are absorbing barriers at 0 and N (2: k). Let T be the time at which the walk is absorbed, and make the assumptions that JK(ST) = So, JK(Sf - T) = S5. Then the probability Pk of ultimate ruin satisfies

o . Pk + N

. (1 - Pk)

= k,

O· Pk

396

+ N 2 . (1 -

Pk) - JK(T)

= k2,

Solutions

Introduction

and therefore Pk 5.

=

I - (kj N) and E(T)

= keN -

[12.1.5]-[12.1.9]

k).

(i) By Exercise (12.1.2), for r ::: i, E(YrYi)

= E{E(YrYi I Tj}} = E{YiE(Yr I Tj}} = E(Y?),

an answer which is independent of r. Therefore if i ::: j ::: k. (ii) We have that

E{ (Yk - Yj)21:Fj} = E(Yfl :Fj) - 2E(YkYj I :Fj) + E(Y] I :Fj). Now E(Yk Yj I Ti) = E{ E(Yk Yj I Jj) ~ Ti} = lE(¥; (iii) Taking expectations of the last conclusion,

I Ti), and the claim follows.

Now {E(Y;) : n ::: I} is non-decreasing and bounded, and therefore converges. Therefore, by (*), {Yn : n ::: I} is Cauchy convergent in mean square, and therefore convergent in mean square, by Problem (7.11.11). 6.

(i) Using Jensen's inequality (Exercise (7.9.4)),

I

E(u(Yn+d Tn) ::: u(E(Yn+l

I Tn))

= u(Yn).

(ii) It suffices to note that Ix I, x 2 , and x+ are convex functions of x; draw pictures if you are in doubt

about these functions. 7. (i) This follows just as in Exercise (12.1.6), using the fact that u{E(Yn+ 1 I Tn)} ::: u(Yn ) in this case. (ii) The function x+ is convex and non-decreasing. Finally, let {Sn : n ::: O} be a simple random walk whose steps are + I with probability P (= I - q > ~) and -I otherwise. If Sn < 0, then E

(ISn+lll Tn) = p(ISnl- I) +q(ISnl + I) = ISnl- (p -

q) <

ISnl;

note that lP'(Sn < 0) > 0 if n ::: I. The same example suffices in the remaining case. S. Clearly EIA- n1/r(Xn)1 ::: A- n sup{1 1/r(j) I : j E S}. Also,

I

E(1/r(X n+d Tn)

= LPXn,j1/r(j)::: A1/r(Xn) jES

where Tn gale.

= a(X 1, X2, ... , Xn).

Divide by An+l to obtain that the given sequence is a supermartin-

9. Since var(ZI) > 0, the function G, and hence also G n , is a strictly increasing function on [0, I]. Since I = Gn+l (Hn+l (s)) = Gn(G(Hn+l (s))) and Gn(Hn(s)) = I, we have that G(Hn+l (s)) = Hn(s). With Tm = a(Zk : 0::: k ::: m),

I

E(Hn+l (s)zn+! Tn)

= G(Hn+l (s))Zn =

397

Hn(s)Zn.

[12.2.1]-[12.3.2]

Solutions

Martingales

12.2 Solutions. Martingale differences and Hoeffding's inequality 1. Let:Fj = a((Vj, Wj : 1 ::: j ::: i)) and Yi without using the jth object, we have that

= E(Z I :Fj). With Z(j) the maximal worth attainable Z(j) ::: Z ::: Z(j)

+ M.

Take conditional expectations of the second inequality, given :Fj and given :Fj -I, and deduce that IYj - Yj_11 ::: M. Therefore Y is a martingale with bounded differences, and Hoeffding's inequality yields the result. 2. Let:Fj be the a-field generated by the (random) edges joining pairs (va, Vb) with 1 ::: a, b ::: i, and let Xi = E(X I :Fj). We write X (j) for the minimal number of colours required in order to colour each vertex in the graph obtained by deleting Vj. The argument now follows that of the last exercise, using the fact that X (j) ::: X ::: X (j) + 1.

12.3 Solutions. Crossings and convergence 1.

Let TI

= min(n T2k-1

: Yn :::: b), T2

= min(n

= min(n

> TI : Yn ::: a}, and define Tk inductively by

> T2k-2 : Yn :::: b),

T2k

= min(n

> T2k-1 : Yn ::: a}.

The number of downcrossings by time n is Dn(a, b; Y) = max(k : T2k ::: n}. (a) Between each pair of upcrossings of [a, b 1, there must be a downcrossing, and vice versa. Hence IDn(a, b; Y) - Un (a, b; y)1 ::: 1. (b) Let Ii be the indicator function of the event that i E (T2k-l, T2kl for some k, and let n

Zn

= I)i(Yi

- Yi-I>,

n:::: O.

i=1

It is easily seen that Zn ::: -(b - a)Dn(a, b; Y)

+ (Yn -

b)+,

whence

Now Ii is :Fj_I-measurable, since (Ii

= 1) = U ((T2k-1

::: i-I} \ (T2k ::: i

-1}).

k

Therefore,

since In :::: 0 and Y is a submartingale. It follows that E(Zn) :::: E(Zn_l) :::: ... :::: E(Zo) the final inequality follows from (*). 2. If Y is a supermartingale, then -Y is a submartingale. Upcrossings of [a, downcrossings of [-b, -a] by -Y, so that

398

= 0, and

bl by Y correspond to

Solutions

Stopping times

by Exercise (12.3.1). If a, Yn

~

[12.3.3]-[12.4.5]

0 then (Yn - a)- ::: a.

3. The random sequence {1/I(X n ) : n ~ I} is a bounded supermartingale, which converges a.s. to some limit Y. The chain is irreducible and persistent, so that each state is visited infinitely often a.s.; it follows that limn---+oo 1/1 (X n ) cannot exist (a.s.) unless 1/1 is a constant function.

2:f

4. Yisamartingalesince Yn is the sum of independent variables with zero means. Also P(Zn i0) = n- 2 < 00, implying by the Borel-Cantelli lemma that Zn = 0 except for finitely many values of n (a.s.); therefore the partial sum Yn converges a.s. as n ---+ 00 to some finite limit.

2:f

It is easily seen that an = San-l and therefore an if and only if IZn I = an. Therefore

which tends to infinity as n ---+

= 8· sn-2, if n

~ 3. It follows that IYn I ~ ~an

00.

12.4 Solutions. Stopping times 1.

We have that n

+ T2 = n} =

{Tl

U ({ Tl = k) n {T2 = n -

k}) ,

k=O

{max{Tl, T2} ::: n}

=

{min{Tl, T2} ::: n}

= {TI

{TI ::: n}

n {T2

::: n},

::: n} U {T2 ::: n}.

Each event on the right-hand side lies in :Fn .

2.

Let:Fn

= a(XI, X2, ... , Xn) and Sn = Xl + X2 + ... + X n . Now {N(t)

+ 1 = n} = {Sn-l

::: t}

n {Sn

> t} E :Fn .

3. (y+ , :F) is a submartingale, and T = min {k : Yk ~ x} is a stopping time. Now 0 ::: T /\ n ::: n, so that E(Yd) ::: E(yt"n) ::: E(Y,;), whence

4.

We may suppose that E(Yo) <

00.

With the notation of the previous solution, we have that

5. It suffices to prove that EYs ::: EYT, since the other inequalities are of the same form but with different choices of pairs of stopping times. LetIm be the indicator function of the event {S < m ::: T}, and define n

Zn

=

L

Im(Ym - Ym-l),

m=l Note that 1m is :Fm_I-measurable, so that

399

0::: n:::

N.

[12.4.6]-[12.5.2]

Solutions

Martingales

since Y is a submartingale. Therefore JB:(ZN) ::: JB:(ZN-I) ::: ... ::: JB:(Zo) = O. On the other hand, ZN = YT - Ys, and therefore JB:(YT)::: JB:(Ys).

6. De Moivre's martingale is Yn = (q/ p)Sn, where q = 1 - p. Now Yn ::: 0, and JB:(Yo) = 1, and the maximal inequality gives that IP' (max Sm::: x) =IP' (max Ym ::: (q/P)x) Osmsn OsmSn

Take the limit as n

-c> 00

:s (p/q)x.

to find that Soo = sUPm Sm satisfies 00

L

JB:(Soo) =

IP'(Soo ::: x)

:s -p-. q- P

x=1

We can calculate JB:(Soo) exactly as follows. It is the case that Soo ::: x if and only if the walk ever visits the point x, an event with probability for x ::: 0, where f = p / q (see Exercise (5.3.1)). The inequality of (*) may be replaced by equality.

r

7.

(a) First,

0

n {T :s

n} = 0 E :Tn. Secondly, if An {T :s n} E :Tn then A

C

n {T :s n}

Thirdly, if AI, A2, ... satisfy Ai n {T

= {T

:s n} \ (A n {T :s n}) E :Tn.

:s nj E :Tn for each i, then

(U Ai) n {T :s n} = U(Ai n {T :s nj) E :Tn· 1

1

Therefore :TT is a () -field. For each integer m, it is the case that {T < n} {T<mjn{T
an event lying in :Tn. Therefore {T :s m} (b) Let A E :Ts. Then, for any n,

E

ifm > n,

ifm:S n,

:TT for all m.

n

(A

n {S :s

Tj)

n {T :s

n}

=

U (A n {S:s m}) n {T = m}, m=O

the union of events in :Tn, which therefore lies in :Tn. Hence A n {S :s T} E :TT. (c) We have {S :s T} = Q, and (b) implies that A E :TT whenever A E :Ts.

12.5 Solutions. Optional stopping 1. Under the conditions of (a) or (b), the family {YT/\n : n ::: OJ is uniformly integrable. Now T 1\ n -c> T as n -c> 00, so that YT/\n -c> YT a.s. Using uniform integrability, JB:(YT/\n) -c> JB:(YT), and the claim follows by the fact that JB:(YT /\n) = JB:(Yo). 2. It suffices to prove that {YT/\n : n ::: O} is uniformly integrable. Recall that {X n : n ::: OJ is uniformly integrable if

400

Solutions

Optional stopping

[12.5.3]-[12.5.5]

(a) Now, E (IYT An II{IYT !\nl~a})

= E (IYT II{T::;n,IYT I~a}) + E (IYn II{T>n,IYn I~a}) ::: E (IYTII{IYTI~a})

+ E (IYnII{T>n})

= g(a)

+ hen),

say. We have that g(a) ---+ 0 as a ---+ 00, since EIYTI < 00. Also hen) ---+ 0 as n ---+ 00, so that sUPn>N hen) may be made arbitrarily small by suitable choice of N. On the other hand, E (IYnII{IYnl~a}) ---+ 0 as a ---+ 00 uniformly in n E {O, 1, ... , N}, and the claim follows. (b) Since Y;i defines a submartingale, we have that sUPn E(ytAn) ::: sUPn E(Y;i) < 00, the second inequality following by the uniform integrability of {Yn }. Using the martingale convergence theorem, YT An ---+ YT a.s. where EIYT I < 00. Now

Also P(T > n) ---+ 0 as n ---+

00,

so that the final two terms tend to 0 (by the uniform integrability of

the Yi and the finiteness of EI YT I respectively). Therefore YT An standard theorem (7.10.3).

~

YT, and the claim follows by the

3. By uniform integrability, Y00 = limn-+oo Yn exists a.s. and in mean, and Yn = E(Y00 I Tn). (a) On the event {T = n} it is the case that YT = Yn and E(Y00 I TT) = E(Y00 I Tn); for the latter statement, use the definition of conditional expectation. It follows that YT = E(Y00 I TT)' irrespective of the value of T. (b) We have from Exercise (12.4.7) that Ts S; TT. Now Ys = E(Yoo ITs) = E{E(Yoo I TT) I Ts) = E(YT ITs)·

4. Let T be the time until absorption, and note that {Sn} is a bounded, and therefore uniformly integrable, martingale. Also JP'(T < (0) = 1 since T is no larger than the waiting time for N consecutive steps in the same direction. It follows that E(SO) = E(ST) = NJP'(ST = N), so that JP'(ST = N) = E(SO)/ N. Secondly, {S; - n : n ::: O} is a martingale (see Exercise (12.1.4», and the optional stopping theorem (if it may be applied) gives that

and hence E(T) = NE(So) - E(S5) as required. It remains to check the conditions of the optional stopping theorem. Certainly JP'(T < (0) = 1, and in addition E(T2) < 00 by the argument above. We have that EISf - TI ::: N 2 + E(T) < 00. Finally, E{ (S; - n)I{T>n)} ::: (N 2 + n)JP'(T > n) ---+ 0 as n ---+

00,

since E(T2) <

00.

5. Let Tn = a(Sl, S2, ... , Sn). It is immediate from the identity cos(A 2 cos A cos A that COS[A(Sn E(Yn+l I Tn) =

+1-

+ A) + cos(A

- A)

=

~(b - a»] + COS[A(Sn - 1 - ~(b - a»] 2(cosA)n+l = Yn ,

and therefore Y is a martingale (it is easy to see that ElYn I <

00

for all n).

Suppose that 0 < A < rr/(a +b), and note that 0 ::: IA{Sn - ~(b -a)}1 < ~A(a +b) < ~rr for n ::: T. Now YT An constitutes a martingale which satisfies

cos{~A(a+b)} 1 < YTAn < T . (COS A)T An - (cos A)

-----'=-----,-;;;-:--

401

[12.5.6]-[12.5.8]

Solutions

Martingales

If we can prove that E{ (cos)..) - T} < 00, it will follow that {YTAn} is uniformly integrable. This will imply in tum that E(YT) = limn-+oo E(YT An) = E(Yo), and therefore

as required. We have from (*) that

Now T

1\

n --+ T as n --+

00,

implying by Fatou's lemma that cos{i)..(a - b)} cos{i)..(a

+ b)}'

6. (a) The occurrence of the event {U = n} depends on Sl, S2, ... , Sn only, and therefore U is a stopping time. Think of U as the time until the first sequence of five consecutive heads in a sequence of coins tosses. Using the renewal-theory argument of Problem (10.5.17), we find that E( U) = 62. (b) Knowledge of Sl, S2,"" Sn is insufficient to determine whether or not V is not a stopping time. Now E(V) = E(U) - 5 = 57. (c) W is a stopping time, since it is a first-passage time. Also E(W) persistent.

7.

=

00

=

n, and therefore V

since the walk is null

With the usual notation,

E(Mm+n I :Fm)

=

E(t

r=O

Sr

~

+

~(Sm+n -

Sr -

Sm

+ Sm)3/:Fm )

r=m+l

= Mm + nSm = Mm + nSm -

SmE{(Sm+n - Sm)2} nSmE(XI)

=

Mm·

Thus {Mn : n ::: O} is a martingale, and evidently T is a stopping time. The conditions of the optional stopping theorem (12.5.1) hold, and therefore, by a result of Example (3.9.6),

8. We partition the sequence into consecutive batches of a + b flips. If any such batch contains only 1's, then the game is over. Hence JP'(T > n (a + b)) :s {1 - (~)a+b}n --+ 0 as n --+ 00. Therefore, EISf -

TJ :s E(Sf) + E(T) :s

(a

+ b)2 + E(T)

<

00,

and as n --+

402

00.

Solutions

Problems

[12.7.1]-[12.9.2]

12.7 Solutions. Backward martingales and continuous-time martingales 1.

Lets:::: t. We have that lE(1)(X(t))

I F s , Xs = i) = "£} Pi} (t

~lE(1)(X(t)) IF s , Xs = i) = dt so that lE(1) (X (t)) I F s , Xs

- s)1)(j). Hence

(Pt-sG71')z'

= 0,

= i) = 1)(i), which is to say that lE(1)(X(t))

I Fs)

= 1)(X(s)).

2. Let Wet) = exp{-8N(t) + M(1 - e-l:i)} where 8 :::: 0. It may be seen that Wet /\ Ta ), t :::: 0, constitutes a martingale. Furthermore

where, by assumption, the limit has finite expectation for sufficiently small positive 8 (this fact may be checked easily). In this case, {W (t /\ Ta) : t :::: O} is uniformly integrable. Now W (t /\ Ta) ~ W (Ta) a.s. as t ~ 00, and it follows by the optional stopping theorem that

Write s = e-I:i to obtain s-a = lE{eATaCl-s)}. Differentiate at s a(a + 1) = A2 lE (Tl), whence the claim is immediate.

=

I to find that a

=

AlE(Ta) and

3. Let g,m be the a-field generated by the two sequences of random variables Sm, Sm+l ... , Sn and Um+l, Um+2, ... , Un. It is a straightforward exercise in conditional density functions to see that 1

lE ( Um+l

--

I g,m+l)

lU

m +2

o

(m + 1)x m - 1 m +1 (U )m+l dx -- -U- - ' m+2 m m+2

whence lE(Rm I g,m+l) = Rm+l as required. [The integrability condition is elementary.] Let T = max {m : Rm :::: I} with the convention that T = I if Rm < 1 for all m. As in the closely related Example (12.7.6), T is a stopping time. We apply the optional stopping theorem (12.7.5) to the backward martingale R to obtain that lE(RT I g,n) = Rn = Snit. Now, RT :::: I on the event {Rm:::: 1 for some m :::: n}, whence

2:'. t

= lE(RT I Sn = y):::: lP'(Rm::::

I

1 forsomem:::: n Sn

=

y).

[Equality may be shown to hold. See Karlin and Taylor 1981, pages 110-113, and Example (12.7.6).]

12.9 Solutions to problems 1.

Clearly lE(Zn) :::: (jh

+ m)n,

and hence lEI Yn I <

00.

Secondly, Zn+ 1 may be expressed as

,,£f':;l Xi + A, where Xl, X2, ... are the family sizes of the members of the nth generation, and A is the number of immigrants to the (n lE(Yn+l I Zn)

=

+ l)th generation.

1{

jhn+l

jhZn

+m

Therefore lE(Zn+l I Zn)

(1 1-

jhn+l ) } 1- jh

=

=

jhZn

+ m, whence

Yn .

2. Each birth in the (n + 1)th generation is to an individual, say the sth, in the nth generation. Hence, for each r, BCn+l),r may be expressed in the form BCn+l),r = Bn,s + Bj(s), where Bj(s) is the age of the parent when its jth child is born. Therefore lE{

I> -l:iB(n+lJ,r \ Fn} = lE{ ~ e-l:iCBn,s+BjCs)) \ Fn} = I>-l:iBn,s Ml (8), r

S,}

S

403

[12.9.3]-[12.9.5]

which gives that E(Yn+ 1 I Tn) 3.

Martingales

Solutions

=

=

Yn . Finally, E(YI (e))

1, and hence E(Yn (e))

=

1.

If x, C > 0, then

Now (Yk + c)2 is a convex function of Yk, and therefore defines a submartingale (Exercise (12.1.7)). Applying the maximal inequality to this submartingale, we obtain an upper bound ofE{(Yn +c)2}/(x+ c)2 for the right-hand side of (*). We set c = E(Y;)/x to obtain the result. 4. (a) Note that Zn = Zn-l + cn{Xn - E(Xn I Tn-d), so that (Z,!F) is a martingale. Let T be the stopping time T = min{k : CkYk 0': x}. Then E(ZT An) = E(ZO) = 0, SO that

since the final term in the definition of Zn is non-negative. Therefore n

xlP'(T::: n)::: E{cTAnYTAn}::: I>kE{E(Xk I Tk-d}, k=l

where we have used the facts that Yn 0': 0 and E(Xk I Tk-d 0': O. The claim follows. (b) Let Xl, X 2, ... be independent random variables, with zero means and finite variances, and let

Yj

= "L{=l Xi.

Then Y} defines a non-negative submartingale, whence

5. The function h(u) = lul r is convex, and therefore Yi(m) = lSi - Sml r , i 0': m, defines a submartingale with respect to the filtration :Fi = (J ({ Xj : 1 ::: j ::: i}). Apply the HRC inequality of Problem (12.9.4), with Ck = 1, to obtain the required inequality. If r = 1, we have that m+n

E(ISm+n - Sml):::

L

EIZkl

k=m+l

by the triangle inequality. Let m, n -+ 00 to find, in the usual way, that the sequence {Sn} converges a.s.; Kronecker's lemma (see Exercise (7.8.2)) then yields the final claim. Suppose 1 < r ::: 2, in which case a little more work is required. The function h is differentiable, and therefore h(v) - h(u)

=

(v - u)h'(u)

+ fov-u {h'(u +x) -

h'(u)} dx.

Now h'(y) = rlylr-lsign(y) has a derivative decreasing in Iyl. It follows (draw a picture) that h' (u + x) - h' (u) ::: 2h' (1x) if x 0': 0, and therefore the above integral is no larger than 2h(1(v - u)). Apply this with v = Sm+k+l - Sm and u = Sm+k - Sm, to obtain

404

Solutions [12.9.6]-[12.9.10]

Problems

Sum over k and use the fact that

to deduce that

m+n

E(ISm+n - Smn

::s 22 - r

L

E(IZkn·

k=m+l

The argument is completed as after (*). 6.

With

h=

IIYk=O}, we have that

E(Yn

I .Tn-d = E(Xn l n-l + nYn-llXn 1(1 - In-d l.Tn-l) = I n-l E (X n ) + nYn-l (1 - In-dEIXn I = Yn-l

since E(Xn) = 0, EIXnl = n- 1 . Also EIYnl ::s E{IXn l(1 EIYn I < 00. Therefore (Y,!F) is a martingale. Now Yn = 0 if and only if Xn = O. Therefore JP>(Yn

+ nIYn -ll)}

and EIYll <

00,

whence

= 0) = JP>(Xn = 0) = 1 - n- 1 --+ 1 hand, ~n JP>(Xn =j:. 0) = 00, and therefore

as n --+ 00, implying that Yn !.. O. On the other JP>(Yn =j:. 0 i.o.) = 1 by the second Borel-Cantelli lemma. However, Yn takes only integer values, and therefore Yn does not converge to 0 a.s. The martingale convergence theorem is inapplicable since sUPn EIYnl = 00. 7. Assume that t > 0 and M(t) = 1. Then Yn = e tSn defines a positive martingale (with mean 1) with respect to.Tn = a(X 1, X2, ... , Xn). By the maximal inequality,

and the result follows by taking the limit as n --+

00.

8. The sequence Yn = ~zn defines a martingale; this may be seen easily, as in the solution to Exercise (12.1.15). Now {Yn } is uniformly bounded, and therefore Yoo = limn--+oo Yn exists a.s. and satisfies E(Yoo) = E(Yo) =~. Suppose 0 < ~ < 1. In this case Z 1 is not a.s. zero, so that Zn cannot converge a.s. to a constant c unless C E {O, oo}. Therefore the a.s. convergence of Yn entails the a.s. convergence of Zn to a limit random variable taking values 0 and 00. In this case, E(Y00) = 1 . JP>(Zn --+ 0) + 0 . JP>(Zn --+ (0), implying that JP>(Zn --+ 0) = ~, and therefore JP>( Zn --+ (0) = 1 - ~.

9. It is a consequence of the maximal inequality that JP>(Y; :::: x) Therefore E(Y;)

=

laoo JP>(Y; :::: x) dx ::s

1

00

::s 1 +E{Yn

1+

1

::s x-I E(Yn I(y,;':o:x}) for x

00

JP>(Y; :::: x) dx

1

x- I(1,y,;'](X)dX}

= 1 + E(Yn log+ Y;) ::s 1 + E(Yn log+ Yn ) + E(Y;)/e. 10. (a) We have, as in Exercise (12.7.1), that

I

E(h(X(t)) B, Xes)

= i) =

LPij(t)h(j) j

405

fors < t,

> O.

[12.9.11]-[12.9.13]

Solutions

Martingales

for any event B defined in tenns of (X (u) : u ::0 s j. The derivative of this expression, with respect to t, is (PtGh');, where P t is the transition semigroup, G is the generator, and h = (h(j) : j ::: 0). In this case,

(Gh')j = ~ gjkh(k) = Aj {h(j

+ 1) -

h(j)} - {Lj {h(j) - h(j -

l)}

= 0

k

for all j. Therefore the left side of (*) is constant for t ::: s, and is equal to its value at time s, i.e. X(s). Henceh(X(t)) defines a martingale.

= min{t : X(t) E (O, n)) to obtain lE(h(X(T))) = h(m) as required. It is necessary but not difficult to

(b) We apply the optional stopping theorem with T lE(h(X(O))), and therefore (l - n(m))h(n)

=

check the conditions of the optional stopping theorem.

11. (a) Since Y is a submartingale, so is y+ (see Exercise (12.1.6)). Now

Therefore (lE(Y:+ m I Tn) : m ::: OJ is (a.s.) non-decreasing, and therefore converges (a.s.) to a limit Mn. Also, by monotone convergence of conditional expectation,

and furthennore lE(Mn) = limm-> 00 lE(Y';;+n) ::0 M. It is the case that Mn is Tn-measurable, and therefore it is a martingale. (b) We have that Zn = Mn - Yn is the difference of a martingale and a submartingale, and is therefore a supennaljingale. Also Mn ::: Y,t ::: 0, and the decomposition for Yn follows.

this

case Zn is a martingale, being the difference of two martingales. Also Mn ::: lE(Y,t Y,t ::: Yn a.s., and the claim follows.

(c) In

I Tn) =

12. We may as well assume that {L < P since the inequality is trivial otherwise. The moment generating function of P - Cl is M(t) = et(p-!IHiu2t2, and we choose t such that M(t) = 1, i.e., t = - 2(P - {L) / cr 2 . Now define Zn = min {e tYn , 1j and Tn = cr (C 1, C2, ... , Cn). Certainly lElZnl < 00; also lE(Zn+l I Tn) ::0 lE(e tYn +1 I Tn) = /Yn M(t) = /Yn and lE(Zn+l I Tn) ::0 1, implying that lE(Zn+l I Tn) ::0 Zn. Therefore (Zn, Tn) is a positive supennartingale. Let T = inf{n : Yn ::0 OJ = inf{n : Zn = Ij. Then T !\ m is a bounded stopping time, whence lE(Zo) ::: lE(ZT Am) ::: JP'(T ::0 m). Let m -+ 00 to obtain the result.

13. Let Tn = cr(Rl, R2,· .. , Rn). (a) 0::0 Yn ::0 1, and Yn is Tn-measurable. Also lE(Rn+l

I Rn) =

Rn

Rn

+ n+r+b ,

whence Yn satisfies lE(Yn+ 1 I Tn) = Yn . Therefore (Yn : n ::: OJ is a uniformly integrable martingale, and therefore converges a.s. and in mean. (b) In order to apply the optional stopping theorem, it suffices that JP'(T < (0) = 1 (since Y is uniformly integrable). However JP'(T > n) = ~ . ~ ... n~l = (n + 1)-1 -+ O. Using that theorem,

= lE(Yo), which is to say that lE{T / (T (c) Apply the maximal inequality.

lE(YT)

+ 2) j = 406

~, and the result follows.

Solutions [12.9.14]-[12.9.17]

Problems

14. As in the previous solution, with f},n the a-field generated by AI, A2,'" and Tn, lE(Y If},) _ ( Rn + An ) ( Rn ) n+l n Rn + Bn + An Rn + Bn Rn

-=---=- = Rn +Bn

+(

Rn

+

Rn Bn

+ An

) (

Bn Rn

+ Bn

)

Yn ,

= lE{lE(Yn+l I f},n) I Tn} = Yn · Also IYnl ::: l,andthereforeYn is a martingale. We need to show that lP'(T < 00) = I. LetIn be the indicator function of the event{T > n J. We have by conditioning on the An that sothatlE(Yn +l I Tn)

lE(In

II

I)

I A) = n-l ( 1 - - j=o

2 + Sj

-+

II 00

j=O

(

1)

1- -2 + Sj

as n -+ 00, where Sj = 2:,{=1 Ai. The infinite product equals 0 a.s. if and only if2:,j(2+Sj )-1 = 00 a.s. By monotone convergence, lP'(T < 00) = I under this condition. If this holds, we may apply the optional stopping theorem to obtain that lE(YT) = lE(Yo), which is to say that lE

(1 _ + I

AT) 2+ ST

= ~. 2

15. At each stage k, let Lk be the length of the sequence 'in play', and let Yk be the sum of its entries, so that Lo = n, Yo = 2:,1=1 xi. If you lose the (k + l)th gamble, then Lk+l = Lk + I and Yk+l = Yk + Zk where Zk is the stake on that play, whereas if you win, then Lk+l = Lk - 2 and Yk+l = Yk - Zk; we have assumed that Lk :::: 2, similar relations being valid if Lk = I. Note that Lk is a random walk with mean step size -1, implying that the first-passage time T to 0 is a.s. finite, and has all moments finite. Your profits at time k amount to Yo - Yko whence your profit at time T is Yo, since YT = O. Since the games are fair, Yk constitutes a martingale. Therefore lE(YT Am) = lE(YO) =j:. 0 for all m. However T /\ m -+ T a.s. as m -+ 00, so that YT Am -+ YT a.s. Now lE(YT) = 0 =j:. limm-> 00 lE(YT Am), and it follows that (YT Am : m :::: I) is not uniformly integrable. Therefore lE(suPm YTAm) = 00; see Exercise (7.10.6). 16. Since the game is fair, lE(Sn+l I Sn) = Sn. Also ISnl ::: 1+ 2 + ... + n < 00. Therefore Sn is a martingale. The occurrence of the event {N = n} depends only on the outcomes of the coin-tosses up to and including the nth; therefore N is a stopping time. A tail appeared at time N - 3, followed by three heads. Therefore the gamblers Gl, G2,"" G N -3 have forfeited their initial capital by time N, while G N -i has had i + I successful rounds for 0::: i ::: 2. Therefore SN = N - (p-l + p-2 + p-3), after a little calculation. It is easy to check that N satisfies the conditions of the optional stopping theorem, and it follows that lEeSN) = lE(So) = 0, which is to say thatlE(N) = p-l + p-2 + p-3. In order to deal with HTH, the gamblers are re-programmed to act as follows. If they win on their first bet, they bet their current fortune on tails, returning to heads thereafter. In this case, SN = N - (p-l + p-2q-l) where q = I - P (remember that the game is fair), and therefore

= p-l + p-2q-l. Let Tn = a ({ Xi, Yi

lE(N)

17. : 1 ::: i ::: n J), and note that T is a stopping time with respect to this filtration. Furthermore lP'(T < 00) = 1 since T is no larger than the first-passage time to 0 of either of the two single-coordinate random walks, each of which has mean 0 and is therefore persistent.

ai

Let ar = var(X 1) and = var(Yd. We have that Un - Uo and Vn - Vo are sums of independent summands with means 0 and variances ar and respectively. It follows by considering

ai

407

[12.9.18]-[12.9.19]

Solutions

Martingales

the martingales (Un - UO)2 -na} and (Vn - VO)2 -nar (see equation (12.5.14) and Exercise (10.2.2» that E{(UT - UO)2} = arE(T), E{(VT - VO)2} = arE(T). Applying the same argument to (Un

+ Vn ) -

(Uo

+ Vo), we obtain

Subtract the two earlier equations to obtain

E{ (UT - UO)(VT - Vo)} = cE(T) if E(T) < 00. Now UT VT = 0, and in addition E(UT) = Uo, E(VT) = Vo, by Wald's equation and the fact that E(X 1) = E(Y1) = O. It follows that -E(Uo Vo) = cE(T) if E(T) < 00, in which case c < O. Suppose conversely that c < O. Then (*) is valid with T replaced throughout by the bounded stopping time T /\ m, and hence

Therefore E(T /\ m) ::: E(Uo Vo)/(2Icl) for all m, implying that E(T) and so E(T) = -E(Uo Vo)lc as before.

= limm-+oo E(T /\ m)

< 00,

18. Certainly 0 ::: Xn ::: 1, and in addition Xn is measurable with respect to the a-field Tn = a(R1, R2,"" Rn). Also E(Rn+1 I Rn) = Rn - Rnl(52 - n), whence E(Xn+1 I Tn) = X n . Therefore Xn is a martingale. A strategy corresponds to a stopping time. If the player decides to call at the stopping time T, he wins with (conditional) probability XT, and therefore lP'(wins) = E(XT), which equals E(Xo) (= ~) by the optional stopping theorem. Here is a trivial solution to the problem. It may be seen that the chance of winning is the same for a player who, after calling "Red Now", picks the card placed at the bottom of the pack rather than that at the top. The bottom card is red with probability ~, irrespective of the strategy of the player. 19. (a) A sums of money in week t is equivalent to a sum s 1(1 +a)t in week 0, since the latter sum may be invested now to yield s in week t. If he sells in week t, his discounted costs are I:~= 1 c I (I + a)n and his discounted profit is Xt!(1 + a)t. He wishes to find a stopping time for which his mean discounted gain is a maximum. Now T

-L(1+a)-n c =-=-{(1+a)-T n=1

-I},

a

= E{ (1 + a)-T ZT } - (cia). (b) The function h(y) = ay - fyoo lP'(Zn > y) dy is continuous and strictly increasing on [0, '00), with h(O) = -E(Zn) < 0 and hey) --+ 00 as y --+ 00. Therefore there exists a unique y (> 0) such that hey) = 0, and we choose y accordingly. (c) Let Tn = a(Z1' Z2,"" Zn). We have that so that /-L(T)

E(max{Zn, y})

= y + loo [1

- G(y)] dy

where G(y) = lP'(Zn ::: y). Therefore E(Vn+1 I Tn) non-negative supermartingale.

408

=

(1

=

(1 +a)y

+ a)-ny

::: Vn , so that (Vn, Tn) is a

Solutions [12.9.20]-[12.9.21]

Problems

Let J-t(r) be the mean gain of following the strategy 'accept the first offer exceeding r - (cia),. The corresponding stopping time T satisfies JP>(T = n) = G(r)n (l - G(r)), and therefore 00

J-t(r)

+ (cia) =

L E{ (1 n=O

+ a)-T ZT I{T=n)}

00

= L(1 + a)-nG(r)n(1

- G(r))E(ZI I ZI > r)

n=O

=

1 +a { r(l - G(r)) 1 + a - G(r)

+

1

00

(1 - G(y)) dy } .

T

Differentiate with care to find that the only value of r lying in the support of Z 1 such that J-t' (r) = 0 is the value r = y. Furthermore this value gives a maximum for J-t(r). Therefore, amongst strategies of the above sort, the best is that with r = y. Note that J-t(y) = y (l + a) - (cia). Consider now a general strategy with corresponding stopping time T, where JP>(T < (0) = 1. For anypositiveintegerm, T I\m is abounded stopping time, whenceE(VTAm) ::: E(Vo) = y(1+a). Now IVT Am I ::: L:~o lVii, and L:~o ElVi I < 00. Therefore {VT Am : m 2: O} is uniformly integrable. Also VT Am --+ VT a.s. as m --+ 00, and it follows that E(VT Am) --+ E(VT). We conclude that J-t(T) = E(VT) - (cia) ::: y(1 +a) - (cia) = J-t(y). Therefore the strategy given above is optimal. (d) In the special case, JP>(ZI > y) = (y - 1)-2 for y 2: 2, whence y = 10. The target price is therefore 9, and the mean number of weeks before selling is G(y)/(1 - G(y)) = 80.

20. Since G is convex on [0, (0) wherever it is finite, and since G(l) = 1 and G' (1) < 1, there exists a unique value of T/ (> 1) such that G(T/) = T/. Furthermore, Y n = T/ 2n defines a martingale with mean E(Yo) = T/. Using the maximal inequality (12.6.6),

for positive integers k. Therefore 00

E(suPZn)::: n

1

L-· T/ - 1

k=1

= K, and Mn+1 = Mn -Xn+l, n 2: 1, where Xn is the number of matches in the nth round. By the result of Problem (3.11.17), EX n = 1 for all n, whence E(Mn+1 + n + 1 I Tn) = Mn + n, 21. LetMn be the number present afterthe nth round, soMo

where Tn is the a-field generated by Mo, MI, ... , Mn. Thus the sequence {Mn + n} is a martingale. Now, N is clearly a stopping time, and therefore K = Mo + 0 = E(MN + N) = EN. We have that

+ n + 1)2 + Mn+1 I Tn} = (Mn + n)2 - 2(Mn + n)E(Xn+1

E{ (Mn+1

- 1)

+ Mn + E{<Xn+1

:::(Mn+n)2+Mn,

where we have used the fact that var(Xn+1 I Tn)

=

I { 0

409

if Mn > 1, if Mn

=

1.

I

- 1)2 - Xn+1 Tn}

[12.9.22]-[12.9.24]

Solutions

Hence the sequence (Mn supermartingales, K2

Martingales

+ n)2 + Mn)

is a supermartingale. By an optional stopping theorem for

+ K = Mo2 + Mo

::: E { (MN

} = E(N 2 ), + N) 2 + MN

and therefore var(N) ::: K. 22. In the usual notation, E(M(s

+ t) I:Fs )

= E

(loS

W(u) du

= M(s) + tW(s) -

+

ls+

t

W(u) du -

W(s)E([W(s

HW(s + t) -

+ t) -

W(s)]21 :Fs )

W(s)

+ W(s)} 31 :Fs )

= M(s)

as required. We apply the optional stopping theorem (12.7.12) with the stopping time T = inf{u : W(u) E (a, b)}. The hypotheses of the theorem follow easily from the boundedness of the process for t E [0, T], and it follows that

, Hence the required area A has mean

[We have used the optional stopping theorem twice actually, in that E(W(T))

= a) = -b/(a - b).] With:Fs = a(W(u) : 0 ::: u

= 0 and therefore

lP'(W(T)

23.

E(R(t)2

::: s), we have for s < t that

I :Fs ) = E(IW(s)1 2 + IW(t)

- W(s)1 2 + 2W(s)· (W(t) - W(s)) I :Fs )

= R(s)2 + (t

and the first claim follows. We apply the optional stopping theorem (12.7.12) with T IW(u)1 = a}, as in Problem (12.9.22), to find that 0 = E(R(T)2 - T) = a 2 - E(T).

=

- s),

inf(u :

24. We apply the optional stopping theorem to the martingale W(t) with the stopping time T to find that E(W(T)) = -a(1 - Pb) + bPb = 0, where Pb = lP'(W(T) = b). By Example (12.7.10), W(t)2 - t is a martingale, and therefore, by the optional stopping theorem again,

whence E(T) = abo For the final part, we take a martingale exp[eW(t) - ~e2t] to obtain

= b and apply the optional stopping theorem to the

= b is the same as that given W (T) = -b. = 1/ cosh(be), and the answer follows by substituting s = ~e2.

on noting that the conditional distribution of T given W (T) I 2

Therefore, E(e- Ze T)

410

13 Diffusion processes

13.3 Solutions. Diffusion processes 1.

It is easily seen that E{X(t

+ h) -

X(t) 1 X(t)}

E( {X (t + h) - X(t)} 21 X(t))

+ o(h), = (A. + jj,)X (t)h + o(h), =

(A. - jj,)X(t)h

which suggest a diffusion approximation with instantaneous mean aCt, x) neous variance bet, x) = (A. + jj,)x.

=

(A. - jj,)x and instanta-

2. The following method is not entirely rigorous (it is an argument of the following well-known type: it is valid when it works, and not otherwise). We have that

-aM = at

1

00

e()Y -af dy -00 at

=

1

00

{ea(t, y)

+ ~e2b(t, y) }e()Y f

by using the forward equation and integrating by parts. Assume that a(t, y) 2:n !3n(t)yn. The required expression follows from the 'fact' that

1

00

e()Yyn f dy

3.

1

00

e()Y f dy

=

2:n an (t)yn, b(t, y)

=

an M = --. n ae

-00

Using Exercise (13.3.2) or otherwise, we obtain the equation aM at

with boundary condition M(O, e)

4.

= -ann ae

-00

dy,

-00

=

= emM + 1e2M 2

1. The solution is M(t)

= exp(~e(2m + e)t}.

Using Exercise (13.3.2) or otherwise, we obtain the equation aM aM 1 2 -=-e-+-e M at ae 2

with boundary condition M(O, e)

=

1. The characteristics of the equation are given by dt 1

de

e

2dM e2 M'

with solution M(t, e) = e!()2 g(ee- t ) where g is a function satisfying 1 M = exp(!e 2 (1 - e- 2t )}.

411

=

e!()2 gee). Therefore

[13.3.5]-[13.3.10]

Solutions

Diffusion processes

5. Fixt > 0. Suppose we are given Wj(s), W2(S), W3(s),forO::::: s::::: t. By Pythagoras's theorem, R(t + u)2 = Xi + X~ + X~ where the Xi are independent N(Wi (t), u) variables. Using the result of

Exercise (5.7.7), the conditional distribution of R(t + u)2 (and hence of R(t + u) also) depends only on the value of the non-centrality parameter e = R(t)2 of the relevant non-central X2 distribution. It follows that R satisfies the Markov property. This argument is valid for the n-dimensional Bessel process. 6. By the spherical symmetry of the process, the conditional distribution of R(s +a) given R(s) = x is the same as that given W (s) = (x, 0, 0). Therefore, recalling the solution to Exercise (13.3.5), JlD(R(s

+ a) =

::::: y [ R(s)

i {(U-X)2+ v 2+ w 2} (u,v,w): 2na 3/2 exp 2a dudvdw u2+v 2+w 2 :sy2 ( )

J l

y

=

= x)

p=O

= [Y

Jo

12n in I {p2 - 2pxcose +x2} 2 . 3/2 exp p smeded¢dp ¢>=O 11=0 (2na) 2a

~ {exp (_ (p -

X)2) _ exp (_ (p + X)2)} dp, 2a 2a

~2na

and the result follows by differentiating with respect to y. 7. Continuous functions of continuous functions are continuous. The Markov property is preserved because g(.) is single-valued with a unique inverse. 8.

(a) Since lE(eO"W(t»)

1 2

= ezO"

t, this is not a martingale.

(b) This is a Wiener process (see Problem (13.12.1)), and is certainly a martingale. (c) With :Fi lE{ (t

= a(W(s)

:

°: :

s ::::: t) and t, u > 0,

+ u)W(t + u) _lot+u W(s)ds l:Ft } = (t + u)W(t) -lot W(s)ds _It+u W(t)ds = tW(t) -lot W(s)ds,

whence this is a martingale. [The integrability condition is easily verified.] 9. (a) With s < t, Set) = S(s) exp{a(t - s) + b(W (t) - W (s))}. Now Wet) - W(s) is independent of {W (u) : u ::::: s}, and the claim follows. (b) S (t) is clearly integrable and adapted to the filtration !F = (:Fi) so that, for s < t,

°: :

lE(S(t)

[:Fs ) = S(s)lE(exp{a(t -

which equals S(s) if and only if a

s)

+ b(W(t) -

W(s))}

[:Fs) = S(s)exp{a(t -s) + ib2(t -

s)),

+ ib2 = 0. In this case, lE(S(t)) = lE(S(O)) = I.

10. Either find the instantaneous mean and variance, and solve the forward equation, or argue directly as follows. With s < t, JlD(S(t)::::: y [S(s)

= x) = JlD(bW(t):::::

-at +Iogy [bW(s)

=

-as +Iogx).

Now b(W(t) - W(s)) is independent of W(s) and is distributed as N(O, b 2 (t - s)), and we obtain on differentiating with respect to y that _ I ( (IOg(Y/X)-a(t-s))2) jet, y [s, x) - yV2nb 2 (t _ s) exp 2b2(t _ s) ,

412

x,y > 0.

Solutions

Excursions and the Brownian bridge

[13.4.1]-[13.6.1]

13.4 Solutions. First passage times 1.

Certainly X has continuous sample paths, and in addition IEIX(t)1 < IE(X(t) I :Fs )

=

1 2(.

X(s)e 2 {;/ t-s)IE(e,{;/{W(t)-W(s)} I :Fs )

=

00.

1 2(

Also, if s < t,

)

1 2(

X(s)e2:{;/ t-s e-2{;/ t-s

)

=

Xes)

as required, where we have used the fact that Wet) - W(s) is N(O, t - s) and is independent of :Fs .

2. Apply the optional stopping theorem to the martingale X of Exercise (13.4.1), with the stopping time T, to obtain IE(X(T)) = 1. Now WeT) = aT + b, and therefore IE(e VrT + ieb ) = 1 where 1{1 = iaB + ~B2. Solve to find that IE(e VrT )

= e- ieb =

exp {-b( Ja 2 - 21{1

+ a) }

is the solution which gives a moment generating function.

3.

We have that T :::: u if and only if there is no zero in (u, t], an event with probability 1 (2/n) cos- 1 {..jU7t}, and the claim follows on drawing a triangle.

13.5 Solution. Barriers 1.

Solving the forward equation subject to the appropriate boundary conditions, we obtain as usual

that fr(t, y)

=

get, y I d)

+ e- 2md g(t, y

I -d) -

L:

2me 2mx g(t, y I x)dx

1

where get, y I x) = (2nt)-2 exp{ -(y - x - mt)2 1(2t)}. The first two terms tend to 0 as t -+ 00, regardless of the sign of m. As for the integral, make the substitution u = (x - y - mt) I vir to obtain, as t -+ 00,

-

l

-(d+ y +mt)/.fi

1 2

2me

2m e-2: u { 2lmle-2lmlY Y - - du-+

-00

v'2n

0

ifm < 0, ifm

~

O.

13.6 Solutions. Excursions and the Brownian bridge 1.

Let f(t, X)

=

1

2

(2nt) - 2: e- x /(2t). It may be seen that

lP'(W(t) > x I z, W(O)

= 0) =

lim lP'(W(t) > x I z, W(O)

w,(,D

= w)

where Z = (no zeros in (0, t]); the small missing step here may be filled by conditioning instead on the event (W(E) = w, no zeros in (E, t]), and taking the limit as E O. Now, if w > 0,

+

1

{jet, y - w) - f(t, Y + w)} dy

i

f(t, y)dy

00

lP'(W(t) > x, Z I W(O)

= w) =

by the reflection principle, and

lP'(Z I W(O) =

w) = 1 - 2

OO

413

=

L:

f(t, y)dy

[13.6.2]-[13.6.5]

Solutions

Diffusion processes

by a consideration of the minimum value of W on (0, t]. It follows that the density function of W (t), conditional on Z n {W(O) = w}, where w > 0, is x _ f(t,x-w)-f(t,x+w) f (t, Y) d Y ' -w

h

x> O.

JW

w( ) -

Divide top and bottom by 2w, and take the limit as w {. 0: lim hw(x)

= __1_

w,j.o

2.

af f(t,O) ax

= ~e-x2/(2t) t

It is a standard exercise that, for a Wiener process W, lE{W(t) [W(s) =a, W(l)

= o} =a

lE{W(s)2 [W(O) = W(1) = if 0

:s s :s t :s

x>

'

O}

o.

C=:),

= s(1- s),

1. Therefore the Brownian bridge B satisfies, for 0

:s s :s t :s

1,

lE(B(s)B(t)) = lE{ B(s)lE(B(t) [B(s))} = 1 - t lE(B(s)2) = s(1- t) l-s as required. Certainly lE(B(s))

= 0 for all s, by symmetry.

W is a zero-mean Gaussian process on [0, 1] with continuous sample paths, and also W(O) = = O. Therefore W is a Brownian bridge if it has the same autocovariance function as the Brownian bridge, that is, c(s, t) = min{s, t} - st. For s < t,

3.

W(I)

cov(W(s), W(t)) = cov(W(s) - sW(1), Wet) - tW(1)) = s - ts - st since cov(W(u), W(v))

= min{u, v}.

+ st =

s - st

The claim follows.

4. Either calculate the instantaneous mean and variance of W, or repeat the argument in the solution to Exercise (13.6.3). The only complication in this case is the necessity to show that Wet) is a.s. continuous aU = 1, i.e., that u- 1 W(u - 1) ~ 0 a.s. as u ~ 00. There are various ways to show this. Certainly it is true in the limit as u ~ 00 through the integers, since, for integral u, W(u - 1) may be expressed as the sumofu -1 independent N(O, 1) variables (use the strong law). It remains to fill in the gaps. Let n be a positive integer, let x > 0, and write Mn = max {[ W (u) - W (n) [ : n :s u :s n + I}. We have by the stationarity of the increments that 00

LlP'(Mn ::: nx) n=O

=

00

lE(M1)

n=O

X

LlP'(M1 ::: nx):s 1 + - - <

00,

implying by the Borel-Cantelli lemma that n- 1 Mn :s x for all but finitely many values of n, a.s. Therefore n- 1 Mn ~ 0 a.s. as n ~ 00, implying that lim _1_[W(u)[:s lim ~{[W(n)[ +1 n-+oo n

u-+oo U

+ Mn} ~ 0

a.s.

5. In the notation of Exercise (13.6.4), we are asked to calculate the probability that W has no zeros in the time interval between s /(1 - s) and t /(1 - t). By Theorem (13.4.8), this equals

1 - -2 cos- 1 7r

s(1-y) 2 -1 t(1-s) = ; cos

414

~

V~.

Solutions

Stochastic calculus

[13.7.1]-[13.7.5]

13.7 Solutions. Stochastic calculus 1. Let Ts = o-(Wu : 0 ::: u ::: s). Fix n :::: 1 and define Xn(k) = IWktj2nl for 0 ::: k ::: 2n. By Jensen's inequality, the sequence {Xn (k) : 0 ::: k ::: 2 n l is a non-negative submartingale with respect to the filtration Tkt j2n, with finite variance. Hence, by Exercise (4.3.3) and equation (12.6.2), X~ = max{Xn (k) : 0 ::: k ::: 2n l satisfies

lE(X~2) = 210 xlP'(X~ > x) dx ::: 210 00

= 2lE(W/ X~)

:::

00

lE(W/

2VlE(whlE(X~2)

I{X~~x)) dx = 2lE{ W/ loX~ dX}

by the Cauchy-Schwarz inequality.

Hence lE(X~2) ::: 4lE(W?). Now X~2 is monotone increasing in n, and W has continuous sample paths. By monotone convergence,

2.

See the solution to Exercise (8.5.4).

3.

(a) We have that

The first summation equals W?, by successive concellation, and the mean-square limit of the second summation is t, by Exercise (8.5.4). Hence limn--+oo II (n) = i W? - it in mean square. Likewise, we obtain the mean-square limits: lim /zen)

n--+oo

4.

Clearly lE(U (t))

= o.

=

i W?

+ it,

The process U is Gaussian with autocovariance function

Thus U is a stationary Gaussian Markov process, namely the Omstein-Uhlenbeck process. [See Example (9.6.10).] 5.

Clearly lE(Ut )

= O.

lE(Us Us+t )

For s < t,

= lE(Ws Wt) + ,82lE (l~o l~o e-,8(s-u) Wue-,8(t-v) Wv du dV) -lE(Wt ,8 loS e-,8(S-U)WudU) -lE(Ws lot e-,8(t-V)Wv dV) s =s+,82 e -,8(s+t)l

it

u=o v=o

e,8(U+V)min{u,vldudv

t -,8 loS e-,8(S-U)min{u,t l du-lo e-,8(t-V)min{s,vldv e

2,8s

-

2,8

1

-,8(s+t)

e 415

[13.8.1]-[13.8.1]

Solutions

Diffusion processes

after prolonged integration. By the linearity of the definition of U, it is a Gaussian process. From the calculation above, it has autocovariance function c(s, s +t) = (e- f3 (t-s) - e- f3 (t+s»)/(2{3). From this we may calculate the instantaneous mean and variance, and thus we recognize an Omstein-Uhlenbeck process. See also Exercise (13.3.4) and Problem (13.12.4).

13.8 Solutions. The Ito integral 1. (a) Fix t > 0 and let n 2: 1 and 8 = t / n. We write tj = j t / nand Vj correlation of Wiener increments, and the Cauchy-Schwarz inequality,

=

W tj . By the absence of

Therefore,

= n-+OO lim ( tWt (b) As n --+

n-l ) - I:Vj+1(tj+l-tj) =tWt j=o

t

Jro Wsds.

00,

n-l

I: V;(Vj+l -

n-l

Vj)

=

j=o

i I: {V/+l -

V/ - 3Vj(Vj+l - Vj)2 - (Vj+l - Yj)3}

j=o

= i W? -

n-l

I: [Vj (tj+l -

n-l

tj)

+ Vj {(Vj+l

- Vj)2 - (tj+l - tj)}] -

j~

--+

i I: (Vj+l -

Vj)3

j~

i W? -lot W(s)ds + 0 + O.

The fact that the last two terms tend to 0 in mean square may be verified in the usual way. For example,

IE(II:(Vj+l }=O

Vj)3n

=

I: IE

[(Vj+l - Vj)6] }=O n-l n-l 3 =6I:(tj+l-tj)3=6I:(!.-) --+0 j=O j=O n

416

as n --+

00.

Solutions

Ito's formula (c) It was shown in Exercise (13.7.3a) that

JJ Ws dWs = i W? -

[13.8.2]-[13.9.1]

it. Hence,

= t. let 8 =

and the result follows because lE(Wh

2. Fix t > 0 and n :::: 1, and tin. We set Vj = Wjt/n. It is the case that X t = limn--+oo Lj Vj(tj+l - tj). Each term in the sum is normally distributed, and all partial sums are multivariate normal for all 8 > 0, and hence also in the limit as 8 ---+ O. Obviously lE(X t ) = O. For s .::: t, lE(XsX t )

Hence var(X t)

=

=

fat faslE(WuWv)dUdV= fat fas min{u,v}dudv

=

s 2 loo -u

1 2 du

+ los u(t 0

By the Cauchy-Schwarz inequality, as n ---+

4. We square the equation III( Vrl i = 1, 2, to deduce the result. S.

= s2

(t2 6s) - - -

.

~ t 3 , and the autocovariance function is

p(Xs, Xt) = 3

3.

u) du

. V~t (~2 - ~) 6t 00,

+ Vr2) 112 = [ Vrl + Vr211 and use thefact that [[ I (Vri) 112 = [ Vri II for

The question permits us to use the integrating factor efJ t to give, formally, efJ t X t

=

t s dWs ds = efJ t Wt - f3 lot efJ s Ws ds loo efJ __ ds 0

on integrating by parts. This is the required result, and substitution verifies that it satisfies the given equation.

6. Find a sequence cp = «(jJ(n)) of predictable step functions such that [[(jJ(n) -Vr II ---+ 0 as n ---+ 00. By the argument before equation (13.8.9), I«(jJ(n)) ~ I(Vr) as n ---+ 00. By Lemma (13.8.4), [[I «(jJ(n)) [Iz = [[(jJ(n) II, and the claim follows.

13.9 Solutions. Ito's formula 1. The process Z is continuous and adapted with Zo lE(Zt - Zs [ Ts) = 0, and by Exercise (13.8.6) that

=

O. We have by Theorem (13.8.11) that

The first claim follows by the Levy characterization of a Wiener process (12.7.10).

417

[13.9.2]-[13.10.2]

Diffusion processes

Solutions

We have in n dimensions that R2 = Xr + X~ + ... + X~, and the same argument yields that Zt = l:i (Xi / R) dXi is a Wiener process. By Example (13.9.7) and the above,

Jd

X. --':..dXi +ndt = 2RdW +ndt. i=l R

n

n

d(R2) = 2 LXi dXi +ndt = 2R L i=l

= wt4 we obtain dYt = 4W? dWt + 6W? dt.

2.

Applying Ito's formula (13.9.4) to Yt

3.

Apply Ito's formula (13.9.4) to obtain dYt

4.

Note that Xl

= Wt dt + t dWt . Cf. Exercise (13.8.1).

= cos W and X2 = sin W. By Ito's formula (13.9.4),

dY = d(XI

+ iX2) =

dXI

=5.

Hence,

+ i dX 2 =

sin W dW -

+ i d(sin W) ~ cos W dt + i cos W dW d(cos W)

~ sin W dt.

We apply Ito's formula to obtain:

(a) (l+t)dX=-Xdt+dW,

(b) dX

= -~X dt + )1- X 2 dW,

(C)d(:)

=-~(:)dt+(b~a -~b) (:)dW. 13.10 Solutions. Option pricing

1.

(a) We have that

IE«ae Z

_

K)+)

= ()Q

(ae Z

_

K)

~ exp (_ (z -

v2nr: 2

Jlog(K/a)

2r:

1 2

=

1

00

e-2Y (aeY+
5

ex

2100 e-!(y-,)2 dy = ae +2' Y

r;:;-=

1 2

= ae Y + 2 ' = ae z = (r -

dz

z- y

log(K/a) - y

r:

r:

= - - , a = --'-----

1

Kct>(-a)

v2n

ex

(b) We have that ST normal with mean y the result of part (a).

where y

;)2)

ct>(r: - a) - Kct>(-a).

where a

=

St and, under the relevant conditional Q-distribution, Z is = a 2(T - t). The claim now follows by

ia2)(T - t) and variance r: 2

2. (a) Set ~(t, S) = ~(t, St) and 1/I(t, S) = 1/I(t, St), in the natural notation. By Theorem (13.10.15), we have 1/Ix = 1/It = 0, whence 1/I(t, x) = c for all t, x, and some constant c. (b) Recall that dS = jJ,S dt + as dW. Now,

418

Solutions

Option pricing

[13.10.3]-[13.10.5]

by Example (13.9.7). By equation (13.10.4), the portfolio is self-financing if this equals SdS + ,prert dt, and thus we arrive at the SDE ert d,p = -S dS - a 2S 2 dt, whence ru S dS - a2lot e- ru S2 du ,I,(t ' P , S) -- -lot eu u u .

o

(c) Note first that Zt

sl dt, whence

=

0

JJ Su du satisfies dZ t = St dt. By Example (13.9.8), d(StZt) = Zt dSt + d (~S + ,pert) = Zt d St

+

sl dt + ert d,p + re rt dt.

Using equation (13.10.4), the portfolio is self-financing if this equals Zt dSt dt, which is to say that require that ert d,p =

-sl

,p(t, S)

=

-l

e-

ru

+ ,prert dt, and thus we

S~ duo

3. We need to check equation (13.10.4) remembering that dM t = O. Each of these portfolios is self-financing. (a) This case is obvious. (b) d(~S +,p) = d(2S 2 - S2 - t) = 2SdS +dt -dt = ~ dS. (c)d(~S +,p) = -S - tdS + S = ~dS. (d) Recalling Example (13.9.8), we have that

4. The time of exercise of an American call option must be a stopping time for the filtration (:Ft). The value of the option, if exercised at the stopping time T, is V, = (S, - K)+, and it follows by the usual argument that the value at time 0 of the option exercised at T is lE((JI(e- rr V,). Thus the value at time 0 of the American option is sup, (lE((JI (e- rT V,)}, where the supremum is taken over all stopping times T satisfying lP'( T S T) = 1. Under the probability measure 1Qi, the process e- rt Vt is a martingale, whence, by the optional stopping theorem, lE((JI(e- rr V,) = Vo for all stopping times T. The claim follows. 5. We rewrite the value at time 0 of the European call option, possibly with the aid of Exercise (13.10.1), as

where N is an N(O, 1) random variable. It is immediate that this is increasing in So and r and is decreasing in K. To show monotonicity in T, we argue as follows. Let Tl < T2 and consider the European option with exercise date T2. In the corresponding American option we are allowed to exercise the option at the earlier time Tl. By Exercise (13.10.4), it is never better to stop earlier than T2, and the claim follows. Monotonicity in a may be shown by differentiation.

419

[13.11.1]-[13.12.2]

Solutions

Diffusion processes

13.11 Solutions. Passage probabilities and potentials 1. Let H be a closed sphere with radius R (> Iw I), and define P R (r) Then PR satisfies Laplace's equation in ]R.d, and hence

= lP' (G before H II W (0) I = r).

!!"'(rd-1dPR) =0 dr dr

since PR is spherically symmetric. Solve subject to the boundary equations PR(E) to obtain r 2-d - R2-d d-2 as R ~ 00. PR(r) = 2-d R2-d ~ (E/r) E

=

1, PR(R)

= 0,

-

2. The electrical resistance Rn between 0 and the set ""n is no smaller than the resistance obtained by, for every i = 1, 2, ... , 'shorting out' all vertices in the set This new network amounts to a linear chain of resistances in series, points labelled "" i and "" i + 1 being joined by a resistance if size N i- 1 . It follows that 00 1 R(G) = lim Rn::::

""i.

2:-.

i=O Ni

n--+oo

By Theorem (13.11.18), the walk is persistent if I:i N i-

1

= 00.

3. Thinking of G as an electrical network, one may obtain the network H by replacing the resistance of every edge e lying in G but not in H by 00. Let 0 be a vertex of H. By a well known fact in the theory of electrical networks, R(H) :::: R(G), and the result follows by Theorem (13.11.19).

13.12 Solutions to problems (a) T(t) = a Wet /( 2) has continuous sample paths with stationary independent increments, since W has these properties. Also T(t)/a is N(O, t/( 2), whence T(t) is N(O, t). (b) As for part (a). (c) Certainly V has continuous sample paths on (0,00). For continuity at 0 it suffices to prove that t W (t- 1) ~ 0 a.s. as t ..j., 0; this was done in the solution to Exercise (13.6.4). If (u, v), (s, t) are disjoint time-intervals, then so are (v- 1, u- 1), (t-1, s-1); since W has independent increments, so has V. Finally,

1.

V(s

+ t) -

V(s) = tW((s

+ t)-1) -

s{W(s-1) - W((s

+ t)-1)}

is N(O, f3) if s, t > 0, where f3

= ~ + s2 s+t

(! __1_) = s

t.

s+t

2. Certainly W is Gaussian with continuous sample paths and zero means, and it is therefore sufficient to prove that cov(W(s), Wet)) = rnin{s, t}. Now, if s ::s t, cov(W(s), Wet))

= cov(X(r- 1(s)), X(r- 1(t))) = u(r- 1(s))v(r- 1(t)) = r(r- 1(s)) = s v(r- 1(s))v(r- 1(t))

as required.

420

v(r- 1(s))v(r- 1(t))

Solutions

Problems Ifu(s)

= s, vet) =

thiscaseX(t) 3.

1 - t, then ret)

= (1-t)W(tl(1- t».

= tl(1- t), and r- 1(w) =

Certainly U is Gaussian with zero means, and U(O) lE{U(t

= O.

wl(1

[13.12.3]-[13.12.4]

+ w) for O::s w

Now, with St

= e2fJt

<

00.

In

- 1,

+ h) I U(t) = u} = e-fJ(t+h)lE{W(St+h) I West> = uefJ t } = ue-fJ(t+h)e fJt = u - f3uh + o(h),

whence the instantaneous mean of U is aCt, u) therefore

=

-f3u. Secondly, St+h

lE{ U (t + h)21 U (t) = u} = e- 2fJ (t+h)lE{ W (st+h)21

= St + 2f3e 2fJt h + o(h), and

West>

= uefJ t }

= e- 2fJ (t+h) (u 2e2fJt + 2f3e 2fJt h + o(h») = u 2 - 2f3h(u 2 - 1) + o(h). It follows that lE{IU(t

+ h) -

U(t)1

2

1U(t)

= u} = u 2 -

2f3h(u

= 2f3h

+ o(h),

and the instantaneous variance is bet, u) 4.

2

-1) - 2u(u - f3uh)

+ u 2 + o(h)

= 2f3.

Bartlett's equation (see Exercise (13.3.4» for M(t, 8)

aM at

= lE(eIlV(t»

aM

1 2 2

a8

2

is

-=-f38-+-a 8 M

with boundary condition M (8,0)

= e llu .

Solve this equation (as in the exercise given) to obtain

18

the moment generating function of the given normal distribution. Now M (t, 8) --+ exp{ 2a 21(2f3)} 2 1(3) as t --+ 00, whence by the continuity theorem V (t) converges in distribution to the N (0, distribution. If V (0) has this limit distribution, then so does V (t) for all t. Therefore the sequence (V (t1), ... , V (tn» has the same joint distribution as (V (t1 + h), ... , V (tn + h» for all h, t1 ( ... , tn, whenever V (0) has this normal distribution. In the stationary case, lE(V(t)) = 0 and, for s ::s t, cov(V(s), Vet»)

1a

= lE{V(s)lE(V(t) I V(s»)} = lE{V(s)2 e- fJ (t-s)} = c(O)e-fJlt-sl

where c(O) = var(V(s»; we have used the first part here. This is the autocovariance function of a stationary Gaussian Markov process (see Example (9.6.10». Since all such processes have autocovariance functions of this form (i.e., for some choice of (3), all such processes are stationary Omstein-Uhlenbeck processes. The autocorrelation function is p(s) density function

= e- fJ lsi, which is the characteristic function of the Cauchy 1

f(x)

=

f3n{1

+ (xlf3)2}' 421

x ER

[13.12.5]-[13.12.7] 5.

Diffusion processes

Solutions

Bartlett's equation (see Exercise (13.3.2)) for M is

aM

-aM = ae -aM + -21 {3e 2 -ae at ae subject to M (0, e)

= eed.

The characteristics satisfy

dM

The solution is M = g(ee at I(a The solution follows as given. By elementary calculations,

+ i ,Be)) where g is a function satisfying gee I(a + i,Be)) = eed.

E(D(t))

=

E(D(t)2)

=

I

= 0) =

which converges to e- 2ad / f3 as t --+

6.

aM 1 = deat, ae e=o 2 ,Bd a M --2 = _eat (eat - 1) ae e=o a

= (,Bdla)eat(e at lP'(D(t)

2de

dt 1

o

whence var(D(t))

'

+ d 2e 2at,

l). Finally lim

e--+-oo

M(t, e)

= exp {

2adeat

,BO - e a

t

} )

00.

The equilibrium density function g(y) satisfies the (reduced) forward equation

d --(ag) dy where a (y) are

1 d2

+ --(bg) =0 2 dy2

= -,By and b(y) = a 2 are the instantaneous mean and variance. y

The boundary conditions

= -c,d.

Integrate (*) from -c to y, using the boundary conditions, to obtain

-c S Y S d. Integrate again to obtain g(y)

=

2

Ae- f3y2 /a . The constant A is given by the fact that J~c g(y) dy

=

1.

7. First we show that the series converges uniformly (along a subsequence), implying that the limit exists and is a continuous function of t. Set Zmn(t)

=

n-1 sin(kt) -k-Xk. k=m

L

Mmn

= suP{IZmn(t)1

;0

s

t S

rr}.

We have that

M2 < mn -

n- 1 eikt 12 n-1 X2 n-m-1 n-Z-1 X·X· 1 su " - Xk < " ---.k + 2 " " } }+Z P ~ k - ~ k2 ~ ~ . ( . + I) . O:st:srr k=m k=m Z=l I j=m ] ]

I

422

Solutions

Problems

[13.12.8]-[13.12.8]

The mean value of the final tenn is, by the Cauchy-Schwarz inequality, no larger than n-m-l

2

1

n-l-l

L

'" L..J j2(j j=m

[=1

+ 1)2

< 2(n - m) -

H-m -m4

.

Combine this with (*) to obtain

It follows that

L M2n -I,2n n=1 00

E(

)

:s L

6

00

2n/2 < 00,

n=1

implying that I:~1 M2n -1 2n < 00 a.s. Therefore the series which defines W converges uniformly with probability 1 (along a ~ubsequence), and hence W has (a.s.) continuous sample paths. Certainly W is a Gaussian process since Wet) is the sum of nonnal variables (see Problem (7.11.19)). Furthennore E(W(t)) = 0, and cov ( W(s), Wet) )

st

2 ~ sin(ks) sin(kt)

:rr

:rr k=1

= - + - L..J

2

k

since the Xi are independent with zero means and unit variances. It is an exercise in Fourier analysis to deduce that cov(W(s), Wet)) = minis, t}.

8.

We wish to find a solution get, y) to the equation Iyl < b,

satisfying the boundary conditions g(O, y)

= 8y o

if Iyl

:s b,

g(t,y) =0

iflyl =b.

Let get, y I d) be the N(d, t) density function, and note that g(.,. I d) satisfies (*) for any 'source' d. Let 00

get, y)

L

=

(-l)kg(t, y 12kb),

k=-oo

a series which converges absolutely and is differentiable tenn by tenn. Since each summand satisfies (*), so does the sum. Now g(O, y) is a combination of Dirac delta functions, one at each multiple of2b. Only one such multiple lies in [-b, b], and hence g(y, 0) = 8dO. Also, setting y = b, the contributions from the sources at -2(k - l)b and 2kb cancel, so that get, b) = O. Similarly get, -b) = 0, and therefore g is the required solution. Hereis an alternative method. Look for the solution to (*) of thefonn e- Anl sin{ in:rr(y + b)/b}; such a sine function vanishes when Iyl = b. Substitute into (*) to obtain A.n combination of such functions has the fonn

~

get, y) = L..J ane n=1

-AnI'

sm

423

(n:rr(y + b)) 2b .

= n 2:rr 2 /(8b 2 ). A linear

[13.12.9]-[13.12.11]

Solutions

Diffusion processes

We choose the constants an such that g(O, y) = 8y o for Iyl < b. With the aid of a little Fourier analysis, one finds that an = b- 1 sin(~mr). Finally, the required probability equals the probability that W a has been absorbed by time t, a probability expressible as 1 - J~b fa (t, y) dy. Using the second expression for fa, this yields

9. Recall that U(t) = e- 2mD (t) is a martingale. Let T be the time of absorption, and assume that the conditions of the optional stopping theorem are satisfied. Then JE(U(O» = JE(U(T», which is to say that 1 = e 2ma Pa + e- 2mb (l - Pa).

10. (a) We may assume that a, b > O. With

= lP'(W(t)

pt(b)

> b, F(O, t) I W(O)

= a),

we have by the reflection principle that pt(b)

= lP'(W(t) = lP'(b -

> b I W(O)

a < Wet) <

= a) -lP'(W(t) < -b I W(O) = a) b + a I W(O) = 0),

giving that apt (b)

----ab = f(t, b + a) -

f(t, b - a)

where f(t, x) is the N(O, t) density function. Now, using conditional probabilities, lP'(F(O, t) I W(O)

= a, Wet) = b) = _

1 apt(b) f(t, b - a) ab

= 1_

e- 2ab / t .

(b) We know that lP'(F(s, t»

=

2

1 - - cos- 1 { Js7t} 7f

2

=-

7f

sin- 1 { Js7t}

if 0 < s < t. The claim follows since F (to, t2) ~ F (tl , t2). (c) Remember that sin x = x + o(x) as x ..j, O. Take the limit in part (b) as to ..j, 0 to obtain ./t1lt2.

11. Let M(t) symmetry,

= sup{W(s)

: O:s s :s t} and recall that M(t) has the same distribution as IW(t)I. By

lP'( sup IW(s)l::::

w)

:s 2lP'(M(t) ::::

w) = 2lP'(IW(t)1

::::

w).

o:ss:::t

By Chebyshov's inequality,

Fix E > 0, and let An(E)

=

{IW(s)l/s >

E for

somes satisfying2n -

Note that

424

1

< s:s 2n}.

Solutions [13.12.12]-[13.12.13]

Problems for all large n, and also

Therefore 2:n IP'(An(E» < 00, implying by the Borel-Cantelli lemma that (a.s.) only finitely many of the An(E) occur. Therefore t- I W(t) --'? 0 a.s. as t --'? 00. Compare with the solution to the relevant part of Exercise (13.6.4).

12. We require the solution to Laplace's equation '\7 2 p = 0, subject to the boundary condition p(w)

=

0 ifw E H, G { I·f 1 WE.

Look for a solution in polar coordinates of the form 00

p(r, 8)

=

L rn{ an sin(n8) + bn cos(n8)}. n=O

Certainly combinations having this form satisfy Laplace's equation, and the boundary condition gives that 00

H(8)

= bo +

L {an sin(n8) + bn cos(n8)},

181

< n,

n=1

where H(8) = {

~

if - n < 8 < 0, ifO<8
The collection {sin(m8), cos(m8) : m 2: O} are orthogonal over (-n, n). Multiply through (*) by sin(m8) and integrate over (-n, n) to obtain nam = {I - cos(nm)}/m, and similarly bo = and bm = 0 for m 2: 1. 13. The joint density function of two independent N(O, t) random variables is (2nt)-1 exp{ _(x 2 +

i

y2)/(2t)}. Since this function is unchanged by rotations of the plane, it follows that the two coordinates of the particle's position are independent Wiener processes, regardless of the orientation of the coordinate system. We may thus assume that I is the line x = d for some fixed positive d. The particle is bound to visit the line I sooner or later, since IP'(WI (t) < d for all t) = O. The first-passage time T has density function

t > O. Conditional on {T fD(U)

=

00

10o

= t}, D = fDIT(u

W2(T) is N(O, t). Therefore the density function of Dis

I t)fT(t) dt =

1000 __ d e-(u 2 +d 2 )/(21) dt = 0

2nt 2

d , n(u 2 + d 2)

giving that D / d has the Cauchy distribution. The angle G =

PaR satisfies 8 =

tan -I (D / d), whence

IP'(G ::::: 8) = IP'(D ::::: dtan8) =

425

1

8

2 +;,

181

<

in.

U E

JR,

[13.12.14]-[13.12.18]

Solutions

Diffusion processes

14. By an extension of Ito's formula to functions of two Wiener processes, U V = V(WI, Wz) satisfy

=

U(WI' Wz) and

+ Uy dWz + ~(uxx + Uyy ) dt, + vydWz + ~(vxx + vyy)dt,

dU = Ux dWI dV = VxdWI

where ux, Vyy , etc, denote partial derivatives of U and v. Since 4> is analytic, u and v satisfy the Cauchy-Riemann equations Ux = v y , Uy = -Vx, whence u and v are hannonic in that Uxx + Uyy = Vxx + Vyy = O. Therefore,

u y ) is an orthogonal rotation oflR z whenu;+u~ = 1. Sincethejointdistribution Ux of the pair (WI, Wz) is invariant under such rotations, the claim follows.

Thematrix

(~xUy

15. One method of solution uses the fact that the reversed Wiener process {W (t - s) - W (t) : 0 :::: s :::: t} has the same distribution as {W(s) : 0 :::: s :::: t}. Thus M(t)- W (t) = maxo:os:odW (s) - W(t)} has the same distribution as maxO:OUg{ W (u) - W (O)} = M (t). Alternatively, by the reflection principle, lP'(M(t) 2: x, W(t) :::: y)

= lP'(W(t)

2: 2x - y)

for x 2: max{O, y}.

By differentiation, the pair M(t), W (t) has joint density function -24>' (2x - y) for y :::: x, x 2: 0, where 4> is the density function of the N(O, t) distribution. Hence M(t) and M(t) - W(t) have the joint density function -24>'(x + y). Since this function is symmetric in its arguments, M(t) and M(t) - W(t) have the same marginal distribution.

16. The Lebesgue measure A(Z) is given by A(Z)

= 10

00

IrW(t)=u}

du,

whence by Fubini's theorem (ct. equation (5.6.13)), lE(A(Z))

=

10

17. Let 0 < a < b < c < d, and let M(x, y) M(c, d) - M(a, b)

=

00

lP'(W(t)

= u) dt = O.

= maxx:::s:oy W(s).

max {W(s) - W(c)} c:os:od

+ {W(c) -

Then

W(b)} -

max {W(s) - W(b)}. a:os:ob

Since the three terms on the right are independent and continuous random variables, it follows that countably many rationals, we deduce that = 1, and the result follows.

lP'( (M(c, d) = M(a, b)) = O. Since there are only lP'( (M(c, d) = M(a, b) for all rationals a < b < c < d)

18. The result is easily seen by exhaustion to be true when n = 1. Suppose it is true for all m :::: n - 1 where n 2: 2. (i) If Sn :::: 0, then (whatever the final term of the permutation) the number of positive partial sums and the position of the first maximum depend only on the remaining n - 1 terms. Equality follows by the induction hypothesis. (ii) If Sn > 0, then n

Ar

=L

Ar-I (k),

k=1

426

Solutions [13.12.19]-[13.12.20]

Problems

where Ar-l (k) is the number of permutations with Xk in the final place, for which exactly r - 1 of the first n - 1 terms are strictly positive. Consider a permutation n = (XiI' Xi2' ... , Xin_1 ' Xk) with Xk in the final place, and move the position OfXk to obtain the new permutation n' = (Xk, XiI' Xi2' ... , Xin_I). The first appearance of the maximum in n' is at its rth place if and only if the first maximum of the reduced permutation (XiI' Xi2' ... , Xin_l) is at its (r - l)th place. [Note that r = 0 is impossible since Sn > 0.] It follows that n

Br

=

L Br-l (k),

k=l where Br-l (k) is the number of permutations with Xk in the final place, for which the first appearance of the maximum is at the (r - l)th place.

By the induction hypothesis, Ar-l(k) terms excluding Xk. The result follows.

=

Br-l(k), since these quantities depend on the n - 1

19. Suppose that Sm = "L,J=l Xj, 0 ::s m ::s n, are the partial sums of n independent identically distributed random variables Xj. Let An be the number of strictly positive partial sums, and Rn the index of the first appearance of the value of the maximal partial sum. Each of the n! permutations of (Xl, X2, ... , Xn) has the same joint distribution. Consider the kth permutation, and let h be the indicator function of the event that exactly r partial sums are positive, and let h be the indicator function that the first appearance of the maximum is at the rth place. Then, using Problem (13.12.18), 1 n!

1 n!

IP'(An

L lE(h) = -n! k=l L lE(h) = IP'(Rn = r). n! k=l

= r) = -

=

L,j

We apply this with Xj W(jtln) - W((j - 1)tln), so that Sm has the same distribution as

= W(mtln).

Thus An

I{W(jt/n»O}

Rn

= rnin{k 2': 0 : W(kt In) =

By Problem (13.12.17), Rn ~ R as n --+

m~x W(jt In)}.

O:::o}:::on

By Problem (13.12.16), the time spent by W at zero

00.

is a.s. a null set, whence An ~ A. Hence A and R have the same distribution. We argue as follows to obtain that that Land R have the same distribution. Making repeated use of Theorem (13.4.6) and the symmetry of W, IP'(L < x)

= 1P'(~~~t W(s)

= 21P'(

0) + IP'C1~~t W(s) > 0)

sup {W(s) - W(x») < -W(x») x:::os:::ot

= 1P'(IW(t) = IP'(

<

= 21P'(IW(t) -

W(x)1 < W(x»)

W(x)1 < IW(x)l)

sup {W(s) - W(x)j < x:::os:::ot

sup {W(s) - W(x»))

= IP'(R ::s x).

O:::os:::ox

Finally, by Problem (13.12.15) and the circular symmetry of the joint density distribution of two independent N(O, 1) variables U, V, 1P'(IW(t) - W(x)1 < IW(x)l)

= 1P'((t -

20. Let

Tx

={

inf (t

x)V

2

::s xU 2 ) = IP' (U2 ~ V2 ::s

::s 1 : W (t) = x)

1

if this set is non-empty, otherwise,

427

7) = ~

sin-

l

~.

[13.12.21]-[13.12.24]

Solutions

Diffusion processes

and similarly Vx = sup{t ::::: 1 : W (t) = x}, with Vx = 1 if W (t) I' x for all t E [0, 1]. Recall that Uo and Vo have an arc sine distribution as in Problem (13.12.19). On the event lUx < I}, we may write (using the re-scaling property of W) Ux = Tx

+ (1

Vx = Tx

- Tx)Uo,

+ (1

- Tx)Vo,

where Vo and Vo are independent of Ux and Vx , and have the above arc sine distribution. Hence Ux and Vx have the same distribution. Now Tx has the first passage distribution of Theorem (13.4.5), whence

Therefore,

and

lux (u) = loU iTx,ux(t, u) du =

T[

v'U(~ _ u) exp ( - ~:),

0 < x < 1.

21. Note that V is a martingale, by Theorem (13.8.11). Fix t and let 1jfs = sign(Ws ), 0 ::: s ::: t. We have that [[1jf1l = .;t, implying by Exercise (13.8.6) that lE(V?> = [[I(1jf)[[~ = t. By a similar calculation, lE( V? [ :Fs) = V} + t - s for 0 ::: s ::::: t. That is to say, V? - t defines a martingale, and the result follows by the Levy characterization theorem of Example (12.7.10). 22. The mean cost per unit time is

Differentiate to obtain that f.L' (T) R

= 0 if

= 2C {loT
T
= aC loT t-1rjJ(aj.;t) dt,

where we have integrated by parts.

23. Consider the portfolio with w(t, St)

~(t,

= x~(t, x) + e rt 1jf(t, St).

St) units of stock and 1jf(t, St) units of bond, having total value By assumption,

(1 - y)xW, x)

= yert 1jf(t, x).

Differentiate this equation with respect to x and substitute from equation (13.10.16) to obtain the differential equation (1 - y)~ + x~x = 0, with solution ~(t, x) = h(t)x y - l , for some function h(t). We substitute this, together with (*), into equation (13.10.17) to obtain that h' - h(1- y)(!y0'2

+ r) = O.

!

+ r) t }, where A is an absolute constant to be determined according to the size of the initial investment. Finally, w(t, x) = y-l x~(t, x) = y-l h(t)x Y .

It follows that h (t)

= A exp {(1 -

y) ( yO' 2

24. Using Ito's formula (13.9.4), the drift term in the SDE for Ut is (-UI(T - t, W)

+ !U22(T -

t, W») dt,

where UI and U22 denote partial derivatives of u. The drift function is identically zero if and only if 1 ul = Z U 22·

428

Bibliography

A man will turn over half a library to make one book.

Samuel Johnson

Abramowitz, M. and Stegun, I. A. (1965). Handbook ofmathematical functions with fonnulas, graphs and mathematical tables. Dover, New York. Billingsley, P. (1995). Probability and measure (3rd edn). Wiley, New York. Breiman, L. (1968). Probability. Addison-Wesley, Reading, MA, reprinted by SIAM, 1992. Chung, K. L. (1974). A course in probability theory (2nd edn). Academic Press, New York. Cox, D. R. and Miller, H. D. (1965). The theory of stochastic processes. Chapman and Hall, London. Doob, J. L. (1953). Stochastic processes. Wiley, New York. Feller, W. (1968). An introduction to probability theory and its applications, Vol. 1 (3rd edn). Wiley, New York. Feller, W. (1971). An introduction to probability theory and its applications, Vol. 2 (2nd edn). Wiley, New York. Grimmett, G. R. and Stirzaker, D. R. (2001). Probability and random processes, (3rd edn). Oxford University Press, Oxford. Grimmett, G. R. and Welsh, D. J. A. (1986). Probability, an introduction. Clarendon Press, Oxford. Hall, M. (1983). Combinatorial theory (2nd edn). Wiley, New York. Harris, T. E. (1963). The theory of branching processes. Springer, Berlin. Karlin, S. and Taylor, H. M. (1975). Afirst course in stochastic processes (2nd edn). Academic Press, New York. Karlin, S. and Taylor, H. M. (1981). A second course in stochastic processes. Academic Press, New York. Laha, R. G. and Rohatgi, V. K. (1979). Probability theory. Wiley, New York. Loeve, M. (1977). Probability theory, Vol. 1 (4th edn). Springer, Berlin. Loeve, M. (1978). Probability theory, Vol. 2 (4th edn). Springer, Berlin. Moran, P. A. P. (1968). An introduction to probability theory. Clarendon Press, Oxford. Stirzaker, D. R. (1994). Elementary probability. Cambridge University Press, Cambridge. Stirzaker, D. R. (1999). Probability and random variables. Cambridge University Press, Cambridge. Williams, D. (1991). Probability with martingales. Cambridge University Press, Cambridge.

429

Index

Abbreviations used in this index: c.f. characteristic function; distn distribution; eqn equation; fn function; m.g.f. moment generating function; p.g.f. probability generating function; pro process; r.v. random variable; r.w. random walk; s.r.w. simple random walk; thm theorem. A absolute value of s.r. W. 6.1.3 absorbing barriers: s.r.w. 1.7.3, 3.9.1, 5, 3.11.23, 25-26, 12.5.4--5, 7; Wiener pro 12.9.22-3, 13.12.8-9 absorbing state 6.2.1 adapted process 13.8.6 affine transformation 4.13.11; 4.14.60 age-dependent branching pro 5.5.1-2; conditional 5.1.2; honest martingale 12.9.2; mean 10.6.13 age, see current life airlines 1.8.39, 2.7.7 alarm clock 6.15.21 algorithm 3.11.33, 4.14.63,6.14.2 aliasing method 4.11.6 alternating renewal pro 10.5.2, 10.6.14 American call option 13.10.4 analytic fn 13.12.14 ancestors in common 5.4.2 anomalous numbers 3.6.7 Anscombe's theorem 7.11.28 antithetic variable 4.11.11 ants 6.15.41 arbitrage 3.3.7, 6.6.3 Arbuthnot, J. 3.11.22 arc sine distn 4.11.13; sample from 4.11.13 arc sine law density 4.1.1 arc sine laws for r.w.: maxima 3.11.28; sojourns 5.3.5; visits 3.10.3

arc sine laws for Wiener pro 13.4.3, 13.12.10, 13.12.19 Archimedes's theorem 4.11.32 arithmetic r.v. 5.9.4 attraction 1.8.29 autocorrelation function 9.3.3, 9.7.5,8 autocovariance function 9.3.3, 9.5.2, 9.7.6, 19-20, 22 autoregressive sequence 8.7.2, 9.1.1,9.2.1,9.7.3 average, see moving average

B babies 5.10.2 backward martingale 12.7.3 bagged balls 7.11.27,12.9.13-14 balance equations 11.7.13 Bandrika 1.8.35-36, 4.2.3 bankruptcy, see gambler's ruin Barker's algorithm 6.14.2 barriers: absorbing/retaining in r.w. 1.7.3,3.9.1-2,3.11.23, 25-26; hitting by Wiener pr. 13.4.2 Bartlett: equation 13.3.2-4; theorem 8.7.6, 11.7.1 batch service 11.8.4 baulking 8.4.4, 11.8.2, 19 Bayes's formula 1.8.14, 1.8.36 bears 6.13.1, 10.6.19 Benford's distn 3.6.7 Berkson's fallacy 3.11.37 Bernoulli: Daniel 3.3.4, 3.4.3-4; Nicholas 3.3.4

430

Bernoulli: model 6.15.36; renewal 8.7.3; shift 9.17.14; sum of r.v.s 3.11.14, 35 Bertrand's paradox 4.14.8 Bessel: function 5.8.5, 11.8.5, 11.8.16; B. pro 12.9.23, 13.3.5-6, 13.9.1 best predictor 7.9.1; linear 7.9.3, 9.2.1-2, 9.7.1, 3 beta fn 4.4.2, 4.10.6 beta distn: b.-binomial 4.6.5; sample from 4.11.4-5 betting scheme 6.6.3 binary: expansion 9.1.2; fission 5.5.1 binary tree 5.12.38; r.w. on 6.4.7 binomial r.v. 2.1.3; sum of 3.11.8, 11 birth process 6.8.6; dishonest 6.8.7; forward eqns 6.8.4; divergent 6.8.7; with immigration 6.8.5; non-homogeneous 6.15.24; see also simple birth birth-death process: coupled 6.15.46; extinction 6.11.3, 5,6.15.25, 12.9.10; honest 6.15.26; immigration-death 6.11.3, 6.13.18, 28; jump chain 6.11.1; martingale 12.9.10; queue 8.7.4; reversible 6.5.1, 6.15.16; symmetric 6.15.27; see also simple birth-death birthdays 1.8.30 bivariate: Markov chain 6.15.4; negative binomial distn 5.12.16; p.g.f. 5.1.3 bivariate normal distn 4.7.5-6, 12,4.9.4--5,4.14.13, 16, 7.9.2,

Index sample from 4.11.9; sum 4.8.2, 5.11.4,5.12.24-25 Black-Scholes: model 13.12.23; Cauchy-Schwarz inequality value 13.10.5 4.14.27 Bonferroni's inequality 1.8.37 central limit theorem 5.10.1,3,9, books 2.7.15 5.12.33,40, 7.11.26, 10.6.3 Boole's inequalities 1.8.11 characteristic function Borel: normal number theorem 5.12.26--31; bivariate normal 9.7.14; paradox 4.6.1 5.8.11; continuity theorem 5.12.39; exponential distn Borel-Cantelli lemmas 7.6.1, 13.12.11 5.8.8; extreme-value distn 5.12.27; first passage distn bounded convergence 12.1.5 5.10.7-8; joint 5.12.30; law bow tie 6.4.11 of large numbers 7.11.15; Box-Muller normals 4.11.7 multinormal distn 5.8.6; tails branching process: age-dependent 5.7.6; weak law 7.11.15 5.5.1-2, 10.6.13; ancestors Chebyshov's inequality, one-sided 5.4.2; conditioned 5.12.21, 7.11.9 6.7.1-4; convergence, 12.9.8; cherries 1.8.22 correlation 5.4.1; critical 7.10.1; extinction 5.4.3; chess 6.6.6--7 geometric 5.4.3, 5.4.6; chimeras 3.11.36 imbedded in queue 11.3.2, chi-squared distn: non-central 11.7 .5, 11; with immigration 5.7.7; sum 4.lO.1, 4.14.12 5.4.5,7.7.2; inequality 5.12.12; martingale 12.1.3, 9, 12.9.1-2, Cholesky decomposition 4.14.62 8; maximum of 12.9.20; chromatic number 12.2.2 moments 5.4.1; p.g.f. 5.4.4; coins: double 1.4.3; fair 1.3.2; supercritical 6.7.2; total first head 1.3.2, 1.8.2,2.7.1; population 5.12.11; variance patterns 1.3.2, 5.2.6, 5.12.2, 5.12.9; visits 5.4.6 10.6.17, 12.9.16; transitive bridge 1.8.32 2.7.16; see Poisson flips Brownian bridge 9.7.22, 13.6.2-5; colouring: graph 12.2.2; sphere autocovariance 9.7.22, 13.6.2; 1.8.28; theorem 6.15.39 zeros of 13.6.5 competition lemma 6.l3.8 Brownian motion; geometric complete convergence 7.3.7 13.3.9; tied-down, see Brownian bridge complex-valued process 9.7.8 Buffon: cross 4.5.3; needle 4.5.2, compound: Poisson pr. 6.15.21; 4.l4.31-32; noodle 4.14.31 Poisson distn 3.8.6, 5.12.13 busy period 6.12.1; in GIG/l compounding 5.2.3, 5.2.8 11.5.1; in MIG/! 11.3.3; in computer queue 6.9.3 MIMI1 11.3.2,11.8.5; in concave fn 6.15.37 MlM/oo 11.8.9 conditional: birth-death pr. 6.11.4-5; branching pro C 5.12.21, 6.7.1-4; convergence cake, hot 3.l1.32 13.8.3; correlation 9.7.21; call option: American 13.lO.4; entropy 6.l5.45; expectation European 13.10.4-5 3.7.2-3,4.6.2,4.14.13,7.9.4; Campbell-Hardy theorem 6.l3.2 independence 1.5.5; probability capture-recapture 3.5.4 1.8.9; s.r.w. 3.9.2-3; variance car, parking 4.14.30 3.7.4,4.6.7; Wiener pr. cards 1.7.2,5, 1.8.33 8.5.2,9.7.21, 13.6.1; see also regression Carroll, Lewis 1.4.4 casino 3.9.6, 7.7.4, 12.9.16 continuity of: distn fns 2.7.lO; marginals 4.5.1; probability Cauchy convergence 7.3.1; in measures 1.8.16, 1.8.18; m.s. 7.11.11 transition probabilities 6.15.14 Cauchy distn 4.4.4; maximum 7.l1.14; moments 4.4.4; continuity theorem 5.12.35, 39 7.11.19; c.f. 5.8.11; positive

part 4.7.5,4.8.8,5.9.8

431

continuous r.V.: independence 4.5.5, 4.14.6; limits of discrete r.v.s 2.3.1 convergence: bounded 12.1.5; Cauchy 7.3.1, 7.1l.l1; complete 7.3.7; conditional 13.8.3; in distn 7.2.4, 7.11.8, 16,24; dominated 5.6.3, 7.2.2; event of 7.2.6; martingale 7.8.3, 12.1.5, 12.9.6; Poisson pro 7.11.5; in probability 7.2.8, 7.11.15; subsequence 7.11.25; in total variation 7.2.9 convex: fn 5.6.1, 12.1.6--7; rock 4.14.47; shape 4.13.2-3, 4.14.61 corkscrew 8.4.5 Com Flakes 1.3.4, 1.8.13 countable additivity 1.8.18 counters 10.6.6-8, 15 coupling: birth-death pro 6.15.46; maximal 4.12.4-6, 7.11.16 coupons 3.3.2, 5.2.9, 5.12.34 covariance: matrix 3.11.15, 7.9.3; of Poisson pro 7.11.5 Cox process 6.15.22 Cp inequality Cramer-Wold device 7.11.19, 5.8.11 criterion: irreducibility 6.15.15; Ko!rnogorov's 6.5.2; for persistence 6.4.lO Crofton's method 4.13.9 crudely stationary 8.2.3 cube: point in 7.11.22; r.w. on 6.3.4 cumulants 5.7.3-4 cups and saucers 1.3.3 current life lO.5.4; and excess lO.6.9; limit lO.6.4; Markov 10.3.2; Poisson 10.5.4

D dam 6.4.3 dead period lO.6.6--7 death-immigration pro 6.11.3 decay 5.12.48, 6.4.8 decimal expansion 3.1.4, 7.11.4 decomposition: Cholesky 4.14.62; Krickeberg 12.9.11 degrees of freedom 5.7.7-8 delayed renewal pro lO.6.12 de Moivre: martingale 12.1.4, 12.4.6; trial 3.5.1 De Morgan laws 1.2.1 density: arc sine 4.11.13; arc sine law 4.1.1; betsa 4.11.4,

Index

4.14.11,19,5.8.3; bivariate normal 4.7.5-6, 12,4.9.4-5, 4.14.13, 16,7.9.2,7.11.19; Cauchy 4.4.4,4.8.2,4.10.3, 4.14.4,16,5.7.1,5.11.4, 5.12.19,24-25,7.11.14; chi-squared 4.10.1, 4.14.12, 5.7.7; Dirichlet 4.14.58; exponential 4.4.3, 4.5.5, 4.7.2, 4.8.1,4.10.4,4.14.4-5, 17-19, 24,33,5.12.32,39,6.7.1; extreme-value 4.1.1, 4.14.46, 7.11.13; F(r, s) 4.10.2,4, 5.7.8; first passage 5.10.7-8, 5.12.18-19; Fisher's spherical 4.14.36; gamma 4.14.10-12, 5.8.3,5.9.3,5.10.3,5.12.14, 33; hypoexponential 4.8.4; log-normal 4.4.5, 5.12.43; multinormal 4.9.2, 5.8.6; normal 4.9.3, 5, 4.14.1, 5.8.4-6, 5.12.23, 42, 7.11.19; spectral 9.3.3; standard normal 4.7.5; Student's t 4.10.2-3, 5.7.8; uniform 4.4.3, 4.5.4, 4.6.6,4.7.1,3,4,4.8.5,4.11.1, 8, 4.14.4, 15, 19, 20, 23-26, 5.12.32, 7.11.4, 9.1.2, 9.7.5; Weibull 4.4.7, 7.11.13 departure pro 11.2.7, 11.7 .2-4, 11.8.12 derangement 3.4.9 diagonal selection 6.4.5 dice 1.5.2, 3.2.4, 3.3.3, 6.1.2; weightedlloaded 2.7.12, 5.12.36 difference eqns 1.8.20, 3.4.9, 5.2.5 difficult customers 11.7.4 diffusion: absorbing barrier 13.12.8-9; Bessel pro 12.9.23, 13.3.5-6, 13.9.1; Ehrenfest model 6.5.5, 36; first passage 13.4.2; Ito pro 13.9.3; models 3.4.4, 6.5.5, 6.15.12, 36; Ornstein-Uhlenbeck pro 13.3.4, 13.7.4-5, 13.12.3-4, 6; osmosis 6.15.36; reflecting barrier 13.5.1, 13.12.6; Wiener pro 12.7.22-23; zeros 13.4.1, 13.12.10; Chapter 13 passim diffusion approximation to birth-death pro 13.3.1 dimer problem 3.11.34 Dirichlet: density 4.14.58; distn 3.11.31 disasters 6.12.2-3, 6.15.28 discontinuous marginal 4.5.1 dishonest birth pro 6.8.7

distribution: see also density; arc sine 4.11.13; arithmetic 5.9.4; Benford 3.6.7; Bernoulli 3.11.14,35; beta 4.11.4; beta-binomial 4.6.5; binomial 2.1.3,3.11.8, 11,5.12.39; bivariate normal 4.7.5-6, 12; Cauchy 4.4.4; chi-squared 4.10.1; compound 5.2.3; compound Poisson 5.12.13; convergence 7.2.4; Dirichlet 3.11.31,4.14.58; empirical 9.7.22; exponential 4.4.3, 5.12.39; F(r, s) 4.10.2-4; extreme-value 4.1.1, 4.14.46, 7.11.13; first passage 5.10.7-8. 5.12.18-19; gamma 4.14.10-12; Gaussian, see normal; geometric 3.1.1, 3.2.2, 3.7.5, 3.11.7, 5.12.34, 39, 6.11.4; hypergeometric 3.11.1 0-11; hypoexponential 4.8.4; infinitely divisible 5.12.13-14; inverse square 3.1.1; joint 2.5.4; lattice 5.7.5; logarithmic 3.1.1, 5.2.3; log-normal 4.4.5, 5.12.43; maximum 4.2.2, 4.14.17; median 2.7.11, 4.3.4, 7.3.11; mixed 2.1.4, 2.3.4, 4.1.3; modified Poisson 3.1.1; moments 5.11.3; multinomial 3.5.1, 3.6.2; multinormal 4.9.2; negative binomial 3.8.4,5.2.3,5.12.4, 16; negative hypergeometric 3.5.4; non-central 5.7.7-8; normal 4.4.6, 8, 4.9.3-5; Poisson 3.1.1,3.5.2-3,3.11.6,4.14.11, 5.2.3, 5.10.3, 5.12.8, 14, 17, 33, 37, 39, 7.11.18; spectral 9.3.2, 4; standard normal 4.7.5; stationary 6.9.11; Student's t 4.10.2-3, 5.7.8; symmetric 3.2.5; tails 5.1.2, 5.6.4, 5.11.3; tilted 5.7.11; trapezoidal 3.8.1; trinormal 4.9.8-9; uniform 2.7.20,3.7.5,3.8.1,5.1.6, 9.7.5; Weibu1l4.4.7, 7.11.13; zeta or Zipf 3.11.5 divergent birth pro 6.8.7 divine providence 3.11.22 Dobrushin's bound and ergodic coefficient 6.14.4 dog-flea model 6.5.5, 6.15.36 dominated convergence 5.6.3, 7.2.2 Doob's L2 inequality 13.7.1 Doob-Kolmogorov inequality 7.8.1-2

432

doubly stochastic: matrix 6.1.12, 6.15.2; Poisson pro 6.15.22-23 downcrossings inequality 12.3.1 drift 13.3.3, 13.5.1, 13.8.9, 13.12.9 dual queue 11.5.2 duration of play 12.1.4

E Eddington's controversy 1.8.27 editors 6.4.1 eggs 5.12.13 Ehrenfest model 6.5.5, 6.15.36 eigenvector 6.6.1-2, 6.15.7 embarrassment 2.2.1 empires 6.15.10 empirical distn 9.7.22 entrance fee 3.3.4 entropy 7.5.1; conditional 6.15.45; mutual 3.6.5 epidemic 6.15.32 equilibrium, see stationary equivalence class 7.1.1 ergodic: coefficient 6.14.4; measure 9.7.11; stationary measure 9.7.11 ergodic theorem: Markov chain 6.15.44, 7.11.32; Markov pr. 7.11.33, 10.5.1; stationary pro 9.7.10-11, 13 Erlang's loss formula 11.8.19 error 3.7.9; of prediction 9.2.2 estimation 2.2.3, 4.5.3, 4.14.9, 7.11.31 Euler: constant 5.12.27,6.15.32; product 5.12.34 European call option 13.10.4-5 event: of convergence 7.2.6; exchangeable 7.3.4-5; invariant 9.5.1; sequence 1.8.16; tail 7.3.3 excess life 10.5.4; conditional 10.3.4; and current 10.6.9; limit 10.3.3; Markov 8.3.2, 10.3.2; moments 10.3.3; Poisson 6.8.3, 10.3.1, 10.6.9; reversed 8.3.2; stationary 10.3.3 exchangeability 7.3.4-5 expectation: conditional 3.7.2-3, 4.6.2, 4.14.12, 7.9.4; independent r.v.s 7.2.3; linearity 5.6.2; tail integral 4.3.3, 5; tail sum 3.11.13, 4.14.3 exponential distn: c.f. 5.8.8; holding time 11.2.2; in Poisson pro 6.8.3; lack-of-memory

Index

property 4.14.5; limit in branching pro 5.6.2, 5.12.21, 6.7.1; limit of geometric distn 5.12.39; heavy traffic limit 11.6.1; distn of maximum 4.14.18; in Markov pro 6.8.3, 6.9.9; order statistics 4.14.33; sample from 4.14.48; sum 4.8.1,4,4.14.10, 5.12.50, 6.15.42 exponential martingale 13.3.9 exponential smoothing 9.7.2 extinction: of birth-
F F(r, s) distn 4.10.2, 4;

non-central 5.7.8 fair fee 3.3.4 families 1.5.7, 3.7.8 family, planning 3.11.30 Farkas's theorem 6.6.2 filter 9.7.2 filtration 12.4.1-2, 7 fingerprinting 3.11.21 finite: Markov chain 6.5.8, 6.6.5, 6.1 5.4 3-44; stopping time 12.4.5; waiting room 11.8.1 first passage: c.f. 5.10.7-8; diffusion pro 13.4.2; distn 5.10.7-8,5.12.18-19; Markov chain 6.2.1, 6.3.6; Markov pro 6.9.5-6; mean 6.3.7; m.g.f. 5.12.18; s.r.w. 5.3.8; Wiener pro 13.4.2 first visit by s.r.w. 3.10.1, 3 Fisher: spherical distn 4.14.36; E-Tippett-Gumbel distn 4.14.46 FKG inequality 3.11.18, 4.11.11 flip-flop 8.2.1 forest 6.15.30 Fourier: inversion thm 5.9.5; series 9.7.15, 13.12.7 fourth moment strong law 7.11.6 fractional moments 3.3.5, 4.3.1 function, upper-class 7.6.1 functional eqn 4.14.5, 19

gambling: advice 3.9.4; systems 7.7.4 gamma distn 4.14.10-12, 5.8.3, 5.9.3,5.10.3,5.12.14,33; g. and Poisson 4.14.11; sample from 4.11.3; sum 4.14.11 gamma fn 4.4.1,5.12.34 gaps: Poisson 8.4.3, 10.1.2; recurrent events 5.12.45; renewal 10.1.2 Gaussian dis tn, see normal distn Gaussian pro 9.6.2-4, 13.12.2; Markov 9.6.2; stationary 9.4.3; white noise 13.8.5 generator 6.9.1 geometric Brownian motion, Wiener pro 13.3.9 geometric distn 3.1.1, 3.2.2, 3.7.5, 3.11.7, 5.12.34, 39; lack-of-memory property 3.11.7; as limit 6.11.4; sample from 4.11.8; sum 3.8.3-4 goat 1.4.5 graph: colouring 12.2.2; r. w. 6.4.6, 9, 13.11.2-3

H Hajek-Renyi-Chow inequality 12.9.4-5 Hall, Monty 1.4.5 Hastings algorithm 6.14.2 Hawaii 2.7.17 hazard rate 4.1.4, 4.4.7; technique 4.11.10 heat eqn 13.12.24 Heathrow 10.2.1 heavy traffic 11.6.1, 11.7.16 hen, see eggs Hewitt-Savage zero-one law 7.3.4-5 hitting time 6.9.5-6; theorem 3.10.1,5.3.8 Hoeffding's inequality 12.2.1-2 Holder's inequality 4.14.27 holding time 11.2.2 homogeneous Markov chain 6.1.1 honest birth-death pro 6.15.26 Hotelling's theorem 4.14.59 house 4.2.1,6.15.20,51 hypergeometric distn 3.11.1 0-11 hypoexponential distn 4.8.4

I G

idle period 11.5.2, 11.8.9 Galton's paradox 1.5.8 imbedding: jump chain 6.9.11; Markov chain 6.9.12, 6.15.17; gambler's ruin 3.11.25-26,12.1.4, queues: DIM/I 11.7.16; GIM/l 12.5.8

433

11.4.1,3; M/GIl 11.3.1, 11.7.4; unsuccessful 6.15.17 immigration: birth-i. 6.8.5; branching 5.4.5, 7.7.2; i.-death 6.11.2, 6.15.18; with disasters 6.12.2-3, 6.15.28 immoral stimulus 1.2.1 importance sampling 4.11.12 inclusion-exclusion principle 1.3.4, 1.8.12 increasing sequence: of events 1.8.16; of r.v.s 2.7.2 increments: independent 9.7.6, 16-17; orthogonal 7.7.1; spectral 9.4.1, 3; stationary 9.7.17; of Wiener pro 9.7.6 independence and symmetry 1.5.3 independent: conditionally 1.5.5; continuous r.v.s 4.5.5, 4.14.6; current and excess life 10.6.9; customers 11.7.1; discrete r.v.s 3.11.1, 3; events 1.5.1; increments 9.7.17; mean, variance of normal sample 4.10.5, 5.12.42; normal distn 4.7.5; pairwise 1.5.2,3.2.1, 5.1.7; set 3.11.40; triplewise 5.1.7 indicators and matching 3.11.17 inequality: bivariate normal 4.7.12; Bonferroni 1.8.37; Boole 1.8.11; Cauchy-Schwarz 4.14.27; Chebyshov 7.11.9; Dobrushin 6.14.4; Doob-Kolmogorov 7.8.1; Doob L2 13.7.1; downcrossings 12.3.1; FKG 3.11.18; Hajek-Renyi-Chow 12.9.4-5; Hoeffding 12.2.1-2; Holder 4.14.27; Jensen 5.6.1, 7.9.4; Kolmogorov 7.8.1-2, 7.11.29-30; Kounias 1.8.38; Lyapunov 4.14.28; maximal 12.4.3-4, 12.9.3, 5, 9; m.g.f. 5.8.2, 12.9.7; Minkowski 4.14.27; triangle 7.1.1, 3; upcrossings 12.3.2 infinite divisibility 5.12.13-14 inner product 6.14.1, 7.1.2 inspection paradox 10.6.5 insurance 11.8.18, 12.9.12 integral: Monte Carlo 4.14.9; normal 4.14.1; stochastic 9.7.19, 13.8.1-2 invariant event 9.5.1 inverse square distn 3.1.1 inverse transform technique 2.3.3 inversion theorem 5.9.5; c.f. 5.12.20

Index Markov process: Gaussian 9.6.2; reversible 6.15.16 Markov property 6.1.5, 10; strong 6.1.6 Markov renewal, see Poisson pro Markov time, see stopping time J Jaguar 3.11.25 Markovian queue, see MIMIl Jensen's inequality 5.6.1, 7.9.4 marriage problem 3.4.3, 4.14.35 joint: c.f. 5.12.30; density 2.7.20; martingale: backward 12.7.3; distn 2.5.4; mass fn 2.5.5; birth-death 12.9.10; moments 5.12.30; p.g.f. 5.1.3-5 branching pro 12.1.3,9, jump chain: of MIMII 11.2.6 12.9.1-2, 8,20; casino 7.7.4; continuous parameter 12.7.1-2; convergence 7.8.3, K 12.1.5, 12.9.6; de Moivre key renewal theorem 10.3.3, 5, 12.1.4, 12.4.6; exponential 10.6.11 13.3.9; finite stopping time Keynes, J. M. 3.9.6 12.4.5; gambling 12.1.4, knapsack problem 12.2.1 12.5.8; Markov chain 12.1.8, Kolmogorov: criterion 6.5.2; 12.3.3, 12.7.1; optional inequality 7.8.1-2, 7.11.29-30 stopping 12.5.1-8; orthogonal Korolyuk-Khinchin theorem 8.2.3 increments 7.7.1; partial sum Kounias's inequality 1.8.38 12.7.3; patterns 12.9.16; Poisson pr. 12.7.2; reversed Krickeberg decomposition 12.9.11 M 12.7.3; simple r.w. 12.1.4, Kronecker's lemma 7.8.2, machine 11.8.17 12.4.6, 12.5.4--7; stopping 7.11.30,12.9.5 magnets 3.4.8 time 12.4.1,5,7; urn 7.11.27, kurtosis 4.14.45 marginal: discontinuous 4.5.1; 12.9.13-14; Wiener pro 12.9.22-23 multinomial 3.6.2; ord«r L statistics 4.14.22 mass function, joint 2.5.5 L2 inequality 13.7.1 Markov chain in continuous matching 3.4.9, 3.11.17, 5.2.7, Labouchere system 12.9.15 time: ergodic theorem 7.11.33, 12.9.21 lack of anticipation 6.9.4 10.5.1; first passage 6.9.5-6; matrix: covariance 3.11.15; lack-of-memory property: irreducible 6.15.15; jump chain definite 4.9.1; doubly exponential distn 4.14.5; 6.9.11; martingale 12.7.1; stochastic 6.1.12, 6.15.2; geometric distn 3.11.7 mean first passage 6.9.6; multiplication 4.14.63; square mean recurrence time 6.9.11; ladders, see records root 4.9.1; stochastic 6.1.12, renewal pro 8.3.5; reversible Lancaster's theorem 4.14.38 6.14.1; sub-stochastic 6.1.12; 6.15.16, 38; stationary distn large deviations 5.11.1-3, 12.9.7 transition 7.11.31; tridiagonal 6.9.11; two-state 6.9.1-2, 6.5.1, 6.15.16 last exits 6.2.1, 6.15.7 6.15.17; visits 6.9.9 lattice distn 5.7.5 maximal: coupling 4.12.4--6, Markov chain in discrete time: 7.11.16; inequality 12.4.3-4, law: anomalous numbers 3.6.7; absorbing state 6.2.2; bivariate 12.9.3, 5, 9 arc sine 3.10.3, 3.11.28, 5.3.5; 6.15.4; convergence 6.15.43; De Morgan 1.2.1; iterated maximum of: branching pro dice 6.1.2; ergodic theorem logarithm 7.6.1; large numbers 12.9.20; multinormal 5.9.7; 7.11.32; finite 6.5.8, 6.15.43; 2.2.2; strong 7.4.1,7.8.2, r.w. 3.10.2, 3.11.28, 5.3.1, first passages 6.2.1, 6.3.6; 7.11.6, 9.7.10; unconscious 6.1.3, 12.4.6; uniforms homogeneous 6.1.1; imbedded statistician 3.11.3; weak 7.4.1, 5.12.32; Wiener pro 13.12.8, 6.9.11,6.15.17,11.4.1; last 7.11.15,20-21; zero-one 11,15,17 exits 6.2.1, 6.15.7; martingale 7.3.4--5 maximum r.v. 4.2.2, 4.5.4, 12.1.8, 12.3.3; mean first Lebesgue measure 6.15.29, 4.14.17-18, 5.12.32, 7.11.14 passage 6.3.7; mean recurrence 13.12.16 mean: extreme-value 5.12.27; time 6.9.11; persistent 6.4.10; left-continuous r.W. 5.3.7, 5.12.7 first passage 6.3.7, 6.9.6; renewal 10.3.2; reversible level sets of Wiener pro 13.12.16 negative binomial 5.12.4; 6.14.1; sampled 6.1.4,6.3.8; normal 4.4.6; recurrence time simulation of 6.14.3; stationary Levy metric 2.7.13, 7.1.4, 7.2.4 6.9.11; waiting time 11.4.2, distn 6.9.11; sum 6.1.8; limit: binomial 3.11.10; 11.8.6, 10 two-state 6.15.11,17,8.2.1; binomial-Poisson 5.12.39; visits 6.2.3-5, 6.3.5, 6.15.5, 44 measure: ergodic 9.7.11-12; branching 12.9.8; central Lebesgue 6.15.29, 13.12.16; Markov-Kakutani theorem 6.6.1 limit theorem 5.10.1,3,9, irreducible Markov pro 6.15.15 iterated logarithm 7.6.1 Ito: formula 13.9.2; process 13.9.3

5.12.33,40,7.11.26, 10.6.3; diffusion 13.3.1; distns 2.3.1; events 1.8.16; gamma 5.9.3; geometric~xponential 5.12.39; lim inf 1.8.16; lim sup 1.8.16, 5.6.3,7.3.2,9-10, 12; local 5.9.2, 5.10.5-6; martingale 7.8.3; normal 5.12.41, 7.11.19; Poisson 3.11.17; probability 1.8.16-18; r.v. 2.7.2; uniform 5.9.1 linear dependence 3.11.15 linear fn of normal r.v. 4.9.3-4 linear prediction 9.7.1, 3 local central limit theorem 5.9.2, 5.10.5-6 logarithmic distn 3.1.1, 5.2.3 log-convex 3.1.5 log-likelihood 7.11.31 log-normal r.v. 4.4.5, 5.12.43 lottery 1.8.31 Lyapunov's inequality 4.14.28

434

Index

norm 7.1.1,7.9.6; equivalence stationary 9.7.11-12; strongly class 7.1.1; mean square 7.9.6; mixing 9.7.12 rth mean 7.1.1 median 2.7.11, 4.3.4, 7.3.11 normal distn 4.4.6; bivariate menages 1.8.23 4.7.5--6, 12; central limit Mercator projection 6.13.5 theory 5.10.1,3,9,5.12.23, meteorites 9.7.4 40; characterization of 5.12.23; metric 2.7.13, 7.1.4; Levy 2.7.13, cumulants 5.7.4; limit 7.11.19; 7.1.4, 7.2.4; total variation linear transfomations 4.9.3-4; 2.7.13 Mills's ratio 4.4.8, 4.14.1; m.g.f. inequality 5.8.2, 12.9.7 moments 4.4.6, 4.14.1; multivariate 4.9.2, 5.8.6; migration pr., open 11.7.1,5 regression 4.8.7, 4.9.6, 4.14.13, millionaires 3.9.4 7.9.2; sample 4.10.5, 5.12.42; Mills's ratio 4.4.8, 4.14.1 simulation of 4.11.7,4.14.49; minimal, solution 6.3.6-7, 6.9.6 square 4.14.12; standard 4.7.5; Minkowski's inequality 4.14.27 sum 4.9.3; sum of squares misprints 6.4.1 4.14.12; trivariate 4.9.8-9; mixing, strong 9.7.12 uncorrelated 4.8.6 mixture 2.1.4, 2.3.4, 4.1.3, 5.1.9 normal integral 4.14.1 modified renewal 10.6.12 normal number theorem 9.7.14 moments: branching pro 5.4.1; now 6.15.50 fractional 3.3.5, 4.3.1; generating fn 5.1.8, 5.8.2, 5.11.3; joint 5.12.30; problem 5.12.43; renewal pro 10.1.1; tail occupation time for Wiener pro 13.12.20 integral 4.3.3, 5.11.3 open migration 11.7.1, 5 Monte Carlo 4.14.9 optimal: packing 12.2.1; price Monty Hall 1.4.5 12.9.19; reset time 13.12.22; Moscow 11.8.3 serving 11.8.13 moving average 8.7.1, 9.1.3, optimal stopping: dice 3.3.8-9; 9.4.2, 9.5.3, 9.7.1-2, 7; marriage 4.14.35 spectral density 9.7.7 optional stopping 12.5.1-8, multinomial distn 3.5.1; marginals 12.9.19; diffusion 13.4.2; 3.6.2; p.g.f. 5.1.5 Poisson 12.7.2 multinormal distn 4.9.2; c.f. order statistics 4.14.21; 5.8.6; conditioned 4.9.6-7; exponential 4.14.33; general covariance matrix 4.9.2; 4.14.21; marginals 4.14.22; maximum 5.9.7; sampling uniform 4.14.23-24, 39, from 4.14.62; standard 4.9.2; 6.15.42, 12.7.3 transformed 4.9.3, 4.14.62 Ornstein-Uhlenbeck pro 9.7.19, Murphy's law 1.3.2 13.3.4, 13.7.4-5, 13.12.3-4,6; mutual information 3.6.5 reflected 13.12.6 orthogonal: increments 7.7.1; polynomials 4.14.37 N osmosis 6.15.36 needle, Buffon's 4.5.2, 4.14.31-32 p negative binomial distn 3.8.4; bivariate 5.12.16; moments pairwise independent: events 5.12.4; p.g.f. 5.1.1, 5.12.4 1.5.2; r.v.s 3.2.1, 3.3.3, 5.1.7 negative hypergeometric distn paradox: Bertrand 4.14.8; 3.5.4 Borel 4.6.1; Carroll 1.4.4; Newton, I. 3.8.5 Galton 1.5.8; inspection non-central distn 5.7.7-8 10.6.5; Parrando 6.15.48; St Petersburg 3.3.4; voter 3.6.6 non-homogeneous: birth pro 6.15.24; Poisson pro 6.13.7, parallel lines 4.14.52 6.15.19-20 parallelogram 4.14.60; property 7.1.2; noodle, Buffon's 4.14.31

o

435

parking 4.14.30 Parrando's paradox 6.15.48 particle 6.15.33 partition of sample space 1.8.1 0 Pasta property 6.9.4 patterns 1.3.2, 5.2.6, 5.12.2, 10.6.17, 12.9.16 pawns 2.7.18 Pepys's problem 3.8.5 periodic state 6.5.4, 6.15.3 persistent: chain 6.4.10, 6.15.6; r.w. 5.12.5-6, 6.3.2, 6.9.8, 7.3.3, 13.11.2; state 6.2.3-4, 6.9.7 Petersburg, see St Petersburg pig 1.8.22 points, problem of 3.9.4, 3.11.24 Poisson: approximation 3.11.35; coupling 4.12.2; flips 3.5.2, 5.12.37; sampling 6.9.4 Poisson distn 3.5.3; characterization of 5.12.8, 15; compound 3.8.6, 5.12.13; and gamma distn 4.14.11; limit of binomial 5.12.39; modified 3.1.1; sum 3.11.6, 7.2.10 Poisson pro 6.15.29; age 10.5.4; arrivals 8.7.4, 10.6.8; autocovariance 7.11.5, 9.6.1; characterization 6.15.29, 9.7.16; colouring theorem 6.15.39; compound 6.15.21; conditional property 6.13.6; continuity in m.s. 7.11.5; covariance 7.11.5; differentiability 7.11.5; doubly stochastic 6.15.22-23; excess life 6.8.3, 10.3.1; forest 6.15.30; gaps 10.1.2; Markov renewal pro 8.3.5, 10.6.9; martingales 12.7.2; non-homogeneous 6.13.7, 6.15.19-20; optional stopping 12.7.2; perturbed 6.15.39-40; renewal 8.3.5, 10.6.9-10; Renyi's theorem 6.15.39; repairs 11.7.18; sampling 6.9.4; spatial 6.15.30-31, 7.4.3; spectral density 9.7.6; sphere 6.13.3-4; stationary increments 9.7.6, 16; superposed 6.8.1; thinned 6.8.2; total life 10.6.5; traffic 6.15.40,49, 8.4.3 poker 1.8.33; dice 1.8.34 Polya's urn 12.9.13-14 portfolio 13.12.23; self-financing 13.10.2-3 positive definite 9.6.1, 4.9.1

Index positive state, see non-null postage stamp lemma 6.3.9 potential theory 13.11.1-3 power series approximation 7.11.17 Pratt's lemma 7.10.5 predictable step fn 13.8.4 predictor: best 7.9.1; linear 7.9.3, 9.2.1-2,9.7.1, 3 probabilistic method 1.8.28, 3.4.7 probability: continuity 1.8.16, 1.8.18; p.g.f. 5.12.4, 13; vector 4.11.6 problem: matching 3.4.9, 3.11.17, 5.2.7, 12.9.21; menages 1.8.23; Pepys 3.8.5; of points 3.9.4, 3.11.24; Waldegrave 5.12.10 program, dual, linear, and primal 6.6.3 projected r.w. 5.12.6 projection theorem 9.2.10 proof-reading 6.4.1 proportion, see empirical ratio proportional investor 13 .12.23 prosecutor's fallacy 1.4.6 protocol 1.4.5, 1.8.26 pull-through property 3.7.1

Q quadratic variation 8.5.4, 13.7.2 queue: batch 11.8.4; baulking 8.4.4, 11.8.2, 19; busy period 6.12.1, 11.3.2-3, 11.5.1, 11.8.5, 9; costs 11.8.13; departure pro 11.2.7, 11.7.2-4, 11.8.12; difficult customer 11.7.4; D/M/1 11.4.3, 11.8.15; dual 11.5.2; Erlang's loss fn 11.8.19; finite waiting room 11.8.1; G/G/1 11.5.1,11.8.8; G/M/1 11.4.1-2, 11.5.2-3; heavy traffic 11.6.1, 11.8.15; idle period 11.5.2, 11.8.9; imbedded branching 11.3.2, 11.8.5, 11; imbedded Markov pro 11.2.6, 11.4.1,3,11.4.1; imbedded renewal 11.3.3, 11.5.1; imbedded r.w. 11.2.2, 5; Markov, see MIMI1; M/D/1 11.3.1, 11.8.10-11; MlG/1 11.3.3, 11.8.6-7; MlG/oo 6.12.4, 11.8.9; migration system 11.7.1,5; MIMII 6.9.3, 6.12.1, 11.2.2-3, 5-6, 11.3.2, 11.6.1, 11.8.5, 12; M/Mlk 11.7.2,11.8.13; M/M/oo 8.7.4; series 11.8.3, 12; supermarket 11.8.3; tandem 11.2.7, 11.8.3; taxicabs 11.8.16; telephone

orthogonal 7.7.1; p.g.f. 5.12.4, 13; Poisson 3.5.3; standard normal 4.7.5; Student's t 4.10.2-3, 5.7.8; symmetric 3.2.5,4.1.2, 5.12.22; tails 3.11.13,4.3.3, 5,4.14.3,5.1.2, 5.6.4, 5.11.3; tilted 5.1.9, 5.7.11; trivial 3.11.2; truncated 2.4.2; uncorrelated 3.11.12, 16, 4.5.7-8,4.8.6; uniform 3.8.1, R 4.8.4,4.11.1,9.7.5; waiting radioactivity 10.6.6-8 time, see queue; Weibull 4.4.7; zeta or Zipf 3.11.5 random: bias 5.10.9; binomial coefficient 5.2.1; chord random walk: absorbed 3.11.39, 4.13.1; dead period 10.6.7; 12.5.4-5; arc sine laws harmonic series 7.11.37; 3.10.3, 3.11.28, 5.3.5; on integers 6.15.34; line 4.13.1-3, binary tree 6.4.7; conditional 4.14.52; paper 4.14.56; 3.9.2-3; on cube 6.3.4; parameter 4.6.5, 5.1.6, first passage 5.3.8; first 5.2.3, 5.2.8; particles 6.4.8; visit 3.10.3; on graph 6.4.6, pebbles 4.14.51; permutation 9,13.11.2-3; on hexagon 4.11.2; perpendicular 4.14.50; 6.15.35; imbedded in queue polygons 4.13.10, 6.4.9; rock 11.2.2, 5; left-continuous 5.3.7, 4.14.57; rods 4.14.25-26, 5.12.7; martingale 12.1.4, 53-54; sample 4.14.21; 12.4.6, 12.5.4-5; maximum subsequence 7.11.25; sum 3.10.2, 3.11.28, 5.3.1; 3.7.6, 3.8.6, 5.2.3, 5.12.50, persistent 5.12.5-6, 6.3.2; 10.2.2; telegraph 8.2.2; triangle potentials 13.11.2-3; projected 4.5.6,4.13.6-8, 11, 13; 5.12.6; range of 3.11.27; velocity 6.15.33, 40 reflected 11.2.1-2; retaining barrier 11.2.4; returns to origin random sample: normal 4.10.5; 3.10.1, 5.3.2; reversible 6.5.1; ordered 4.12.21 simple 3.9.1-3, 5, 3.10.1-3; on random variable: see also density square 5.3.3; symmetric 1.7.3, and distribution; arc sine 3.11.23; three dimensional 4.11.13; arithmetic 5.9.4; 6.15.9-10; transient 5.12.44, Bernoulli 3.11.14, 35; beta 6.15.9,7.5.3; truncated 6.5.7; 4.11.4; beta-binomial 4.6.5; two dimensional 5.3.4, 5.12.6, binomial 2.1.3, 3.11.8, 11, 12.9.17; visits 3.11.23, 6.9.8, 5.12.39; bivariate normal 10; zero mean 7.5.3; zeros of 4.7.5-6, 12; Cauchy 4.4.4; 3.10.1,5.3.2,5.12.5-6 c.f. 5.12.26-31; chi-squared range of r.w. 3.11.27 4.10.1; compounding rate of convergence 6.15.43 5.2.3; continuous 2.3.1; ratios 4.3.2; Mills's 4.4.8; sex Dirichlet 3.11.31, 4.14.58; expectation 5.6.2, 7.2.3; 3.11.22 exponential 4.4.3, 5.12.39; record: times 4.2.1, 4, 4.6.6, 10; extreme-value 4.1.1, 4.14.46; values 6.15.20, 7.11.36 F(r, s) 4.10.2,4, 5.7.8; recurrence, see difference gamma 4.11.3, 4.14.10--12; recurrence time 6.9.11 geometric 3.1.1, 3.11.7; recurrent: event 5.12.45, 7.5.2, hypergeometric 3.11.10--11; 9.7.4; see persistent independent 3.11.1, 3,4.5.5, red now 12.9.18 7.2.3; indicator 3.11.17; infinitely divisible 5.12.13-14; reflecting barrier: s.r.w. 11.2.1-2; drifting Wiener pr. 13.5.1; logarithmic 3.1.1, 5.2.3; Ornstein-Uhlenbeck pro log-normal 4.4.5; median 13.12.6 2.7.11,4.3.4; m.g.f. 5.1.8, regeneration 11.3.3 5.8.2, 5.11.3; multinomial regression 4.8.7, 4.9.6, 4.14.13, 3.5.1; multinormal 4.9.2; 7.9.2 negative binomial 3.8.4; rejection method 4.11.3-4, 13 normal 4.4.6, 4.7.5-6, 12; exchange 11.8.9; two servers 8.4.1, 5, 11.7.3, 11.8.14; virtual waiting 11.8.7; waiting time 11.2.3, 11.5.2-3, 11.8.6, 8,10 quotient 3.3.1,4.7.2, 10, 13-14, 4.10.4,4.11.10,4.14.11,14, 16,40,5.2.4,5.12.49,6.15.42

436

Index reliability 3.4.5-6, 3.11.18-20, renewal: age, see current life; alternating 10.5.2, 10.6.14; asymptotics 10.6.11; Bernoulli 8.7.3; central limit theorem 10.6.3; counters 10.6.6-8, 15; current life 10.3.2, 10.5.4, 10.6.4; delayed 10.6.12; excess life 8.3.2, 10.3.1-4, 10.5.4; r. function 10.6.11; gaps 10.1.2; key r. theorem 10.3.3, 5, 10.6.11; Markov 8.3.5; m.gJ. 10.1.1; moments 10.1.1; Poisson 8.3.5, 10.6.9-10; r. process 8.3.4; r.-reward 10.5.1-4; r. sequence 6.15.8,8.3.1,3; stationary 10.6.18; stopping time 12.4.2; sum/superposed 10.6.10; thinning 10.6.16 Renyi's theorem 6.15.39 repairman 11. 7 .18 repulsion 1.8.29 reservoir 6.4.3 resources 6.15.47 retaining barrier 11.2.4 reversible: birth-
S a-field 1.2.2,4, 1.8.3, 9.5.1, 9.7.13; increasing sequence of 12.4.7 St John's College 4.14.51 St Petersburg paradox 3.3.4 sample: normal 4.10.5, 5.12.42; ordered 4.12.21 sampling 3.11.36; Poisson 6.9.4; with and without replacement 3.11.10 sampling from distn: arc sine 4.11.13; beta 4.11.4-5; Cauchy 4.11.9; exponential 4.14.48; gamma 4.11.3; geometric 4.11.8; Markov chain 6.14.3; multinormal 4.14.62; normal 4.11.7,4.14.49; s.r.w. 4.11.6; uniform 4.11.1

secretary problem 3.11.17, 4.14.35 self-financing portfolio 13.10.2-3 semi-invariant 5.7.3-4 sequence: of c.f.s 5.12.35; of distns 2.3.1; of events 1.8.16; of heads and tails, see pattern; renewal 6.15.8, 8.3.1, 3; of r.v.s 2.7.2 series of queues 11.8.3, 12 shift operator 9.7.11-12 shocks 6.13.6 shorting 13.11.2 simple birth pro 6.8.4-5, 6.15.23 simple birth-
437

5; queue length 8.4.4, 8.7.4, 11.2.1-2,6,11.4.1,11.5.2, and Section 11.8 passim; r.w. 11.2.1-2; waiting time 11.2.3, 11.5.2-3, 11.8.8 stationary excess life 10.3.3 stationary increments 9.7.17 stationary measure 9.7.11-12 stationary renewal pro 10.6.18, Stirling's formula 3.10.1,3.11.22, 5.9.6, 5.12.5, 6.15.9, 7.11.26 stochastic: integral 9.7.19, 13.8.1-2; matrix 6.1.12, 6.14.1,6.15.2; ordering 4.12.1-2 stopping time 6.1.6, 10.2.2, 12.4.1-2,5,7; for renewal pro 12.4.2 strategy 3.3.8-9, 3.11.25,4.14.35, 6.15.50, 12.9.19, 13.12.22 strong law of large numbers 7.4.1,7.5.1-3,7.8.2,7.11.6, 9.7.10 strong Markov property 6.1.6 strong mixing 9.7.12 Student's t distn 4.10.2-3; non-central 5.7.8 subadditive fn 6.15.14, 8.3.3 subgraph 13.11.3 sum of independent r.v.s: Bernoulli 3.11.14, 35; binomial 3.11.8, 11; Cauchy 4.8.2, 5.11.4,5.12.24-25; chi-squared 4.10.1,4.14.12; exponential 4.8.1,4,4.14.10,5.12.50, 6.15.42; gamma 4.14.11; geometric 3.8.3-4; normal 4.9.3; p.gJ. 5.12.1; Poisson 3.11.6,7.2.10; random 3.7.6, 3.8.6,5.2.3,5.12.50, 10.2.2; renewals 10.6.10; uniform 3.8.1,4.8.5 sum of Markov chains 6.1.8 supercritical branching 6.7.2 supermartingale 12.1.8 superposed: Poisson pro 6.8.1; renewal pro 10.6.10 sure thing principle 1.7.4 survival 3.4.3, 4.1.4 Sylvester's problem 4.13.12, 4.14.60 symmetric: r.v. 3.2.5,4.1.2, 5.12.22; r.w. 1.7.3; state 6.2.5 symmetry and independence 1.5.3 system 7.7.4; Labouchere 12.9.15

lrulex

T

two-dimensional: r.w. 5.3.4, 11.4.2; in MID!1 11.8.10; 5.12.6,12.9.17; Wiener pro in MlG!1 11.8.6; in MIMII 13.12.12-14 11.2.3; stationary distn 11.2.3, 5.7.8 11.5.2-3; virtual 11.8.7 tail: c.f. 5.7.6; equivalent 7.11.34; two server queue 8.4.1, 5, 11.7.3, 11.8.14 Wald's eqn 10.2.2-3 event 7.3.3,5; function 9.5.3; two-state Markov chain 6.15.11, integral 4.3.3, 5; sum 3.1l.!3, Waldegrave's problem 5.12.10 17,8.2.1; Markov pro 6.9.1-2, 4.14.3 Waring's theorem 1.8.13, 5.2.1 6.15.16-17 tail of distn: and moments 5.6.4, Type: T. one counter 10.6.6-7; T. weak law of large numbers 7.4.1, 5.11.3; p.g.f. 5.1.2 7.1l.!5, 20-21 two counter 10.6.15 tandem queue 11.2.7, 11.8.3 Weibull distn 4.4.7, 7.11.13 taxis 11.8.16 Weierstrass's theorem 7.3.6 telekinesis 2.7.8 U Mysaka 8.7.7 white noise, Gaussian 13.8.5 telephone: exchange 11.8.9; sales unconscious statistician 3.11.3 3.11.38 Wiener process: absorbing uncorrelated r.v. 3.1l.!2, 16, barriers 13.13.8,9; arc testimony 1.8.27 4.5.7-8,4.8.6 sine laws 13.4.3, 13.12.10, thinning 6.8.2; renewal 10.6.16 uniform integrability: Section 13.12.19; area 12.9.22; three-dimensional r.w. 6.15.10; 7.10 passim, 10.2.4, 12.5.1-2 Bartlett eqn 13.3.3; Bessel transience of 6.15.9, uniform distn 3.8.1, 4.8.4, 4.11.1, pro 13.3.5; on circle 13.9.4; three-dimensional Wiener pro 9.7.5; maximum 5.12.32; conditional 8.5.2, 9.7.21, 13.1l.! order statistics 4.14.23-24, 39, 13.6.1; constructed 13.12.7; 6.15.42; sample from 4.1l.!; three series theorem 7.11.35 d-dimensional 13.7.1; drift sum 3.8.1, 4.8.4 13.3.3, 13.5.1; on ellipse tied-down Wiener pro 9.7.21-22, 13.9.5; expansion 13.12.7; uniqueness of conditional 13.6.2-5 first passage 13.4.2; geometric expectation 3.7.2 tilted distn 5.1.9, 5.7.11 13.3.9, 13.4.1; hits sphere time-reversibility 6.5.1-3, 6.15 ..16 upcrossings inequality 12.3.2 13.11.1; hitting barrier upper class fn 7.6.1 total life 10.6.5 13.4.2; integrated 9.7.20, urns 1.4.4, 1.8.24-25, 3.4.2, total variation distance 4.12.3-4, 12.9.22, 13.3.8, 13.8.1-2; level 4,6.3.10,6.15.12; Palya's 7.2.9,7.1l.!6 sets 13.12.16; martingales 12.9.13-14 12.9.22-23, 13.3.8-9; tower property 3.7.1, 4.14.29 maximum 13.12.8, 11, 15, traffic: gaps 5.12.45, 8.4.3; V 17; occupation time 13.12.20; heavy 11.6.1, 11.8.15; Poisson value function 13.10.5 quadratic variation 8.5.4, 6.15.40, 49, 8.4.3 variance: branching pro 5.12.9; 13.7.2; reflected 13.5.1; sign transform, inverse 2.3.3 conditional 3.7.4, 4.6.7; normal 13.12.21; standard 9.7.18-21; transient: r.w. 5.12.44,7.5.3; 4.4.6 three-dimensional 13 .1l.! ; Wiener pro 13.11.1 tied-down, see Brownian vector space 2.7.3,3.6.1 transition matrix 7.11.31 bridge; transformed 9.7.18, Vice-Chancellor 1.3.4, 1.8.13 13.12.1,3; two-dimensional transitive coins 2.7.16 virtual waiting 11.8.7 13.12.12-14; zeros of 13.4.3, trapezoidal distn 3.8.1 visits: birth-death pro 6.11.6-7; 13.12.10 Markov chain 6.2.3-5, 6.3.5, trial, de Moivre 3.5.1 Wiener-Hopf eqn 11.5.3, 11.8.8 r.w. 3.11.23, 6.9.9,6.15.5,44; triangle inequality 7.l.!, 3 29, 6.9.8, 10 Trinity College 12.9.15 voter paradox 3.6.6 x triplewise independent 5.1.7 X-ray 4.14.32 trivariate normal distn 4.9.8-9 W trivial r. v. 3.11.2 waiting room 11.8.1, 15 truncated: r.v. 2.4.2; r.w. 6.5.7 waiting time: dependent 11.2.7; Tunin's theorem 3.11.40 zero-one law, Hewitt-Savage for a gap 10.1.2; in G!GIl turning time 4.6.10 7.3.4-5 11.5.2, 11.8.8; in G/M/l t, Student's 4.10.2-3; non-central

u

z

438

FROM A REVIEW OF PROBABILITY ANO RANDOM PROCESSES: PROBLEMS AND SOLUTIONS