Lecture Notes in Mathematics Edited by A. Dold and B. Eckmann
675 Janos Galambos Samuel Kotz
Characterizations of Probability Distributions A Unified Approach with an Emphasis on Exponential and Related Models
Springer-Verlag Berlin Heidelberg New York 1978
Authors Janos Galambos Samuel Kotz Temple University College of Liberal Arts Department of Mathematics Philadelphia, PA 19122/USA
AMS Subject Classifications (1970): 62 El0, 62 H 05
ISBN 3-540-08933-0 Springer-Verlag Berlin Heidelberg NewYork ISBN 0-387-08933-0 Springer-Verlag New York Heidelberg Berlin This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically those of translation, reprinting, re-use of illustrations, broadcasting, reproduction by photocopying machine or similar means, and storage in data banks, Under § 54 of the German Copyright Law where copies are made for other than private use, a fee is payable to the publisher, the amount of the fee to be determined by agreement with the publisher. © by Springer-Verlag Berlin Heidelberg 1978 Printed in Germany Printing and binding: Beltz Offsetdruck, Hemsbach/Bergstr. 2t41/3140-543210
PREFACE The stimulus for this research project was supplied during the 1974 Calgary Conference on Characterizations
of Distributions organized by G.P. Patil; both of
us were active participants in this conference and could not held but notice the multitude of results on characterizations widely scattered in the literature. Our aim in writing this monograph is twofold. One is to bring together the results in this useful and rapidly growing field in order to encourage further fruitful research in this area.
We also have in mind a somewhat different objec-
tive, which in our opinion, is not less important and urgent at this time of rapid and unprecedented growth in periodical literature.
The field of characterizations
was independently developed in different branches of applied probability and pure mathematics which inevitably resulted in a certain amount of duplication of efforts. Moreover the terminology used in different sub-disciplines by various authors is not the same. Our second aim is therefore to unify the existing theory.
We found
that seemingly unrelated matheamtical theorems turned out to be part of the same general theory.
We therefore tried to discover the unifying thread which passes
through the majority of the papers written in the past one and a half decades in the field of characterizations of statistical distributions (in particular those dealing with the exponential distribution, both univariate and multivariate, and its monotonic transformations, which is the subject of this monograph). However, we should point out that there are still a number of as yet unsolved problems, for example, those stemming from Rossberg's results which require additional investigation. This monograph contains no overlap with existing books on characterizations, since earlier books concentrated mainly on normality, which we do not discuss in our book. Moreover, as a rule, results dealing with exponential distributions which have appeared previously in other monographs or books are not reproduced here.
This
is true in particular for the material on this subject contained in Johnson and Kotz' (1972) Multivariate Continuous Distributions and the recent book by Galambos (1978) The Asymptotic Theory of Extreme ~der Statistics, both of which were publish ed by John Wiley and Sons. To make the work self-contained, we have included the derivation of a number of results dealing with basic distribution theory, in particular those related to the distribution of order statistics.
On the other hand, not all the theorems
stated are supplemented with detailed proofs.
In many instances, it was sufficient
to present a comprehensive proof of the basic "unifying" theorem from which the subsequent results follow almost directly or require only minor modifications of the arg~aent.
IV
The level of mathematics was kept to a minimum, but the rigor of mathematical analysis was scrupulously maintained.
Horeover, we have included extensive comments
on the "practicality' of various mathematically equivalent characterizations.
We
thus feel that the monograph, which contains an up-to-date bibliography of over 250 items, will he useful and of interest to the general mathematical community, to probabilists and statisticians as well as to researchers and practitioners of industrial engineering and operations research and various scientists specializing in the natural and behavioral sciences, in particular those who are interested in the foundations and applications of probabilistic model building.
Philadelphia, Pa. April, 1978.
Janos Galambos;
Samuel Kotz
ACKNOWLEDGEMENTS The authors acknowledge with thanks the financial support of the National Science Foundation during the period May 1976 - April 1977.
It was during that
period that the plans for this monograph were conceived and the broad outline of the project was formulated. The authors are also indebted to their colleagues throughout the world too numerous to mention individually - who generously supplied us with reprints and on occasion preprints of their work on characterizations of distributions and related topics which facilitated our task of compiling this monograph. Thanl~s are also due to ~s. June ?laxwell for her careful and accurate typing of the text.
TABLE OF CONTENTS
Chapter I . I.i
Preliminaries and Basic Results Introduction
1.2
Preliminaries
1.3
The classical case of the lack of memory property
1.4 Motivations (Applications of characterization theorems in I0
model building) 1.5
Basic properties of the exponential distribution
1.6
Monotonic transformations of the exponential
Chapter 2. 2.1
12 distribution
Characterizations Based on Truncated Distributions Extended forms of the lack of memory property
18
2] 22
2.2 A stability theorem
28
2.3
Characterizations based on conditional expected values
30
Characterizations by Properties of Order Statistics
36
Chapter 3.
36
3.1
Introduction
3.2
Characterizations by distributional properties
38
3.3
Independence of functions of order statistics
46
3.4
Characterizations through moment assumptions
53
Characterization of the Poisson Process
64
Chapter 4. 4.1
Introduction
64
4.2
Basic properties of Poisson processes
67
4.3
Characterizations based on age and residual life
80
4.4
Rarefactions, geometric compounding and damage models
95
VIII
Characterizations of H u l t i v a r i a t e Exponential D i s t r i b u t i o n s
Chapter 5.
103
5.I
Introduction
103
5.2
Bivariate extensions based on LMP
104
5.3
The Multivariate Marshall-Olkin exponential distribution and its extensions
5.4
112
Characterization of MVE distributions based on hazard rate properties
5 ,~5 S~mnary
Chapter 6.
Miscellaneous Results
120 132
133
Characterizations by s t a t i s t i c a l properties
133
6.2
Records
136
6.3
Further results in terms of order statistics
138
6.4
Some additional characterizations
141
6.1
Bibliography
143
Subject Index
162
CHAPTER 1 PRELIMINARIES AND BASIC RESULTS l.i.
Introduction The emergence of modern probability theory dates from the end of the nineteenth
century and the early twentieth century, culminating with Kolmogoro~'s axioms in 1933. Mathematical statistics were initiated even later: it was only in the early forties of the twentieth century that the basic concepts of this discipline crystalized.
The flood of periodical and monographical literature in this area, which has
persisted up until now at an ever accelerating rate, began after World War II. At the same time, the theories of functions of real and complex variables, which serve as the foundations and basic tools of both probabilistic and mathematical-statistical arguments, reached their maturity at the end of the nineteenth century.
In fact, the works of H. Lebesgue and A. Picard in the early ~art of the
20th century can be considered as the final development of these theories, at least as far as applications to nonabstract probability theory and mathematical statistics are concerned. Probability theory and mathematical statistics are based on a number of specific concepts which are rather simple in meaning when the corresponding translation into pure mathematical terminology is made. distribution function.
Typical of these is the concept of the
It is merely a right continuous, nondecreasing and uniformly
bounded function; moreover, the distribution of a sum of two independent random variables corresponds to the convolution of the two respective distribution functions. Some simple forms of conditional expectation correspond to integrals with predetermined upper or lower limits; the expectation of order statistics can be ex]~ressed as the integral of a polynomial of the distribution function, and so on.
On the other
hand, without this translation, an uncritical manipulation with concepts of probability theory and mathematical statistics may result, and has indeed resulted, in a multitude of seemingly unrelated theorems and corollaries.
This, instead of clari-
fying the structure and properties of the basic notions of these disciplines, cause confusion and often completely obscure the relatively simple underlying natural interrelations. There is perhaps no other branch of mathematical statistics in which this sad state of affairs is manifested so strongly as in the area dealing with characterizations of statistical distributions.
Characterization theorems are located on the
borderline between probability theory and mathematical statistics, and utilize numerous classical tools of mathematical analysis such as (advanced) theory of functions of complex variables, differential equations of different types, theory of series
and, last but not least, the theory of functional equations. It is the purpose of this monograph to show, by analyzing various characterizations of the exponential distribution and its natural transforms, the simplicity and clarity of characterizations of statistical distributions, when these are properly interpreted via elementary concepts of mathematical analysis and simple functional equations.
We intend to demonstrate that almost all characterizations of the expon-
ential distribution can be reduced to the analysis of one of five elementary properties enjoyed by the exponential function
G(x) = e -ax, x _> 0.
By doing so, we hope
to reduce the further proliferation of artificial characterizations of the exponential distribution and its transforms.
By "artificial" we mean characterizations
which are expressed and often couched in terms of complicated concepts of mathematical statistics without the realization of its initial basic and completely equivalent pure and simple mathematical interpretation.
This will perhaps allow the
researchers in this area of statistics to concentrate on more meaningful and original problems which still await solution. A major part of our monograph can be viewed as an imitation of a treatise written by a competent nineteenth century mathematician who is familiar with the basic concepts of modern probability theory and mathematical statistics but whose main emphasis is on the mathematical aspects of the problems. Let us mention here that, to the best of our knowledge, there is no treatise in any language which would overlap with ours.
The basic books on characterizations
(the one by Luk~cs and Laha and the more comprehensive and more recent work by Kagan, Lirmik and Rao) are geared almost entirely towards the normal distribution.
In
addition, their mathematical tools are much more advanced than ours. Why has progress in solving characterization problems in mathematical statistics, in particular for the exponential distribution, been so erratic and inefficient?
In our opinion, at least a partial answer to this question lies in the some-
what peculiar development of the theory of statistical distributions. The development of probability theory has been overwhelmingly dominated by the (continuous) normal distribution since the early eighteenth century.
Indeed, it
started with DeMoivre's Doctrine of Chances (1713), then came Laplace's and Gauss' contributions in the nineteenth century; the beginnings of the twentieth century marked the culmination of the results related to the central limit theorem.
The
dominance of the normal distribution in statistics is perhaps even greater and only in the last two decades have we witnessed a mild retreat from, and rebellion against, its accepted universality.
The development of Pearson's curves and Gram-Charlier
and Edgeworth's series in this century are basically formal mathematical generalizations of the normal density via its characterizing properties. It is therefore not surprising that the exponential distribution on its own appeared in statisticalliterature at a very late stage.
As late as 1931, T. Kondo,
in his paper in Biometrika, "Theory of Sampling Distribution of Standard Deviation," refers to the exponential distribution as Pearson's type X curve.
(It should be
pointed out that the study of the exponential distribution in actuarial, biological and engineering literature preceded its organized study in statistics by over 20 years; see Steffensen (1930), Teissier (1934) and Weibull (1939))~
P.V. Sukhatme
was probably the first, who in 1937 in the Annals of Eugenics suggested that the exponential distribution (which he called the chi-squared distribution with two degrees of freedom) could serve as an alternative to normal distributions "to the type of problems where the form of variation in the population is known and is far from normal."
Sukhatme, in this same paper, determined the joint distribution of differ-
ences of order statistics from an exponential distribution, from which the following basic and important property follows.
Namely, for the exponential distribution, the
differences of order statistics are independent and exponential variates.
This pro-
perty was rediscovered a number of times by several authors including Malmquist (1950), R~nyi (1953) and Epstein and Sobel (1953), whose papers had much influence on the development of the theory of exponential distributions in later years.
We
shall also use this result for characterization purposes in the present monograph. One more look at the early history of probability and statistics can reveal that these theories were not ripe earlier for accepting the exponential distribution. The Poisson distribution (as an approximation to the binomial) appeared in Section 81 of the famous R~cherches sur Za Probabilit~ des Jugements (1837; or 1832 according to certain sources) by S.D. Poisson.
But, as F.A. Haight pointed out in 1967,
"Poisson then turned to other matters and did not refer to his discovery, either in the remainder of the book or in his subsequent publications."
The Poisson distribu-
tion surfaced again in the literature some 60 years later in L. Bortkiewicz' monograph, Das Gesetz der kleinen Zahlen (1898) but did not win acceptance at this time either.
It is noteworthy that Czuber's comprehensive monograph Probability Theory
(1899) did not mention either the Poisson or the exponential distributions, in spite of Whitworth's Choice and Chance (1886) mentioning both and pointing out their strong relation.
This relation also appeared in a physical context in Boltzmarm's Studien
~ber das Gleichgewicht der lebendigen Kraft zwischen bewegten materiellen Punkten (1868).
But statistics had to wait until the 1950's for the general acceptance of
the exponential distribution. The groundbreaking paper was that of W. Weibull in 1951, in which he briefly investigated an extension of the exponential distribution which now bears his name. In 1952, D.J. Davis published a paper entitled "An analysis of some failure data" (J. Amer. Statist. Assoc.), in which he defines "the exponential theory of failures" and compares it with its normal counterpart in 13 applications ranging from bank statement errors to bus motor failures. A serious study of the properties of the exponential distributions in statistical literature commenced with the publication in 1953 in the J. Amer. Statist. Assoc. of B. Epstein and M. Sobel's paper "Life testing" which started the ball rolling.
A comprehensive bibliography of some 82 items on this subject up to 1969
can be found in the book by N.L. Johnson and S. Kotz: Continuous Univariate Distri-
butions I, pp. 228-32 (Wiley, New York, 1970). This now explains why characterizations of the exponential distribution, or of its natural transforms, started so late.
The basic lack of memory characterization
of the exponential distribution is simply the logarithmic variant of the functional equation
f(x+y) = f(x)f(y), which is due to Cauchy (1821) and N. Loba~evskii (1829)
(see, e.g., J. Acz~l (1966)).
(A complete solution of this equation for continuous
and discontinuous cases was given by E.B. Wilson (1899) (the logarithmic variant) and by G. Hamel (1905).) Feller's book in 1950.
This is discussed in detail in the first edition of It is conjectured by us--based on some indirect evidence--
that the first explicit characterization of this distribution appeared sometime in the 19th century in literature devoted to the physical sciences, in particular, in connection with some earlier theories of the atom (since atoms exhibit the lack of memory property).
Unfortunately, we have not as yet found the relevant references.
The actual earliest explicit characterization of the exponential distribution known to us is via extreme value theory.
~st
prominently, the classical papers by Fr@chet
(1927) and by Fisher and Tippet (1928) should be mentioned (the exact statements and more details on this line of history are given in J. Galambos' book (1978)).
These
earlier contributions, however, did not have sufficient impact among probabilists or statisticians.
The modern area of characterization theorems pertaining to the expon-
ential distribution (and its monotone transforms) was originated by European mathematicians such as Fisz (1958) (characterizations via order statistics), basic and elaborate results of H.J. Rossberg (1960) (characterizations based on the range and ratios of order statistics) and R@nyi (1956) (via Poisson processes and, more generally, renewal processes).
Quite independently in the U.S.A., the first characteri-
zations of the exponential distribution were due to Ghurey (1960) and Teicher (1961), who modified the available characterizations of the normal distribution to the exponential case (via the distribution of t-type statistics and by maximum likelihood property, respectively).
This fact serves as an additional indication of the often
undeserved prominence enjoyed by the normal distribution among a large group of statisticians.
It is
only recently that we detect a relaxation and even a drifting
away from this direction.
Here computer technology should receive credit for making
possible this healthy widening of horizons in the field of statistical distribution theory and methodology. As far as the normal distribution is concerned, the first modern characterization result is due perhaps to P61ya (1923) via independence of a variable and a linear combination of two variables. which states that if
X+Y
This was followed by Cram~r's (1936) result
is normal then so are X and Y, where X and Y are inde-
pendent nondegenerate random variables.
In the same year, Geary (1936) obtained
that, within a large class of distributions, normality is characterized by the independence of the arithmetical mean and the variance of a sample.
His result was
refined by Luk~cs (1942), and put into final form by T. Kawata and H. Sakamoto (1949).
The P61ya-Cram~r
type of investigations were continued by M. Kac (1939)
and S.N. Bernstein (1941), whose works can be considered as the foundations of a general theory.
Luk~cs' method proved fruitful for another line of research in the
theory of characterizations, which is well presented in the monograph of Luk~cs and Laha (1964).
The more recent book by Kagan, Linnik and Rao (1973), on the other
hand, gives a good collection of results of various kinds for the normal distribution.
This is the main reason why we do not deal with normality in the present
monograph. In sun~nary, the study of the exponential distribution within the framework of mathematical statistics started at a very late date and after numerous basic concepts of this discipline had been fully established. of the exponential distribution began even later.
The theory of characterizations
It was developed by statisticians
who became used to modern statistical terminology and concepts without sufficient emphasis and attention to the underlying mathematical structure.
As will be seen
later, the difference in terminology usually means that one has to integrate by parts or has to use a substitution, but one arrives at the same integral equation.
This
neglect resulted in: (a)
a duplication or triplication of research efforts and unnecessary restate-
ments of virtually identical results using different terminology.
Laurent (1974)
correctly points out that the fact that several characterization theorems are "rediscovered in applied statistical literature may be a consequence of compartmentalization and specialization." (b)
obscuring the simplicity of the mathematical properties associated with
the exponential function and its monotone transformations. Our aim is therefore to present an up-to-date account of the theory without the above stated misconceptions or oversights.
1.2. Preliminaries For notational purposes, let us introduce the following basic concepts of probability theory. We always denote the underlying probability space by (~,A,P), where a is the sample space, A is the set of events (e-algebra of subsets of a) and P is the probability measure on A.
Random variables will be denoted by capital letters, in most
cases without any explicit reference to
space ~.
Thus X is a random variable, which
is a function on a and which is measurable with respect to A; i.e., for any real number x, the set of elements of a for which X < x is a member of A. suppress a~a
We again
in this notation and we abbreviate
{ ~ :X(~) < x , w c ~ } = {X < x )
.
For a given random variable X, the function F Cx) = P(X < x) is called its distribution funotion. When necessary we emphasize X by adding a subscript such as FX(X ) .
With this definition, a distribution function F(x) is nondecreasing, continuous from the left and F(+~) = I, F(-~) = 0. We call a distribution function F(x) degenerate at the point a if F(x)
= ~ 1
if x > a
I 0
otherwise .
If F(x) has no point of discontinuity, we then refer to it as a continuous distribution function.
To define discrete and absolutely continuous distribution functions,
we introduce a concept for sequences of real numbers. We call a sequence Ul,U2,... of real numbers simple if lujl, j ~ I, can be arranged in a nondecreasing order and if every finite interval (a,b) contains at most a finite m~nber of the uj. Using this concept, F(x) is called absolutely continuous if it is continuous and if it is differentiable for all x, except perhaps on a simple sequence.
Finally,
F(x) is discrete if its set of discontinuities is a simple sequence and if F(x) is parallel to the x-axis between points of discontinuity (in other words, F(x) is a
step function). We shall use the same descriptive adjective for a random variable as for its distribution function.
Thus we speak of degenerate, continuous absolutely continu-
ous and discrete random variables which means that their distribution functions belong to the respective categories. In view of the general properties of distribution functions, the following remark concerning these special classes of distributions is in order.
Since F(x) is
always nondecreasing, its set of points of discontinuity is denumerable and it is differentiable for almost all x in the sense of Lebesgue.
One, however, cannot ex-
ploit these properties in characterization theorems because the points of discontinuity can form a nonsimple sequence.
Furthermore, a set of measure zero can be so
large that no analytical advantages can be gained from knowing a property for all points except on such a set (although in some cases almost sure properties suffice, see, e.g., Chapter 2).
In particular, if f(x) = F'(x), defined for almost all x,
then F(x) can be expressed as an integral of f(x), but very little can be gained from such an integral representation.
On the other hand, if F(x) is absolutely continuous
and if f(x) = F'(x) for all x, except perhaps on a simple sequence, then X
F(x) = f f(t)dt
is a Riemann improper integral and this equation holds for all x.
When we have an
equation determining F(x), then it may be an advantage that F(x) be replaced by an integral for all x.
Hence a solution among absolutely continuous distributions may
easily be found, but it may be a difficult task to show that a given equation does not have any other solution than absolutely continuous ones. cases when this difficulty arises.
We shall see several
The following formula based on integration by parts will be utilized on several occasions in the sequel.
kemma 1 . 2 . 1 . Assume t h a t (1)
] xadF(x) T
In particular,
(2)
L e t X ~ 0 b e a random v a r i a b l e
E(X a) i s f i n i t e = Ta[1-F(T)]
+ a
= f [1-F(x)]dx T
Let T < N < +~
and t h e i n t e g r a l side converges
.
. Then integrating parts yields
N - a / xa-lF(x)dx T N - Na[1-F(N)] + a f xa-l[1-F(x)]dx T
Ta[1-F(T)]
.
+%
<
+oo + f xadF(x) N
+oo = f xadF(x) T
on t h e r i g h t
hand side
t o z e r o as N ÷ +~.
+oo
is finite,
Hence,
t h e s e c o n d t e r m on t h e l e f t
hand
in view of the inequality
+co
f xadF(x)
_> Na f d F ( x ) = N a [ 1 - F ( N ) ]
N
Na[1-F(N)]
xa-l[1-F(x)]dx
be arbitrary.
Since, for any T _< N
f xadF(x) T
F(x).
= NaF(N) - TaF(T)
=
N
function
if E(X) is finite, then
N
f xadF(x) T
with a distribution
T h e n , f o r any 0 ~ T < + ~ ,
T
/ (x-T)dF(x) T
Proof:
f o r some a > 0.
,
N
÷ 0 a s N + +~.
Fomula
(1) t h u s f o l l o w s .
On t h e o t h e r
hand,
(2) i s a
special case of (i) with a = i, hence the proof is completed.
1.3.
J~3 classical case of the lack of memory property. It is convenient to describe the classical case of the lack of memory property
of a random variable or of its distribution in terms of life length of a hypothetical industrial device.
Let X be the random life length of an item.
Evidently, X ~ 0.
We say that X, or the distribution function F(x) of X, has the lack of memory property if, for all x, z ~ 0, for which P(X ~ z) > 0, (3)
P(X~x+zlX~z)
= P(X~x)
.
If P(X=0) ¢ I, then for at least one z 0 > 0, P(X~z0) > 0.
But then a sequential
argument yields from (3) that, for any integer n ~ I, P(X~nz 0) > 0 and thus P(X~z) > 0 for any real number z ~ 0. (3a)
P(X~x+z) = P(X~x)P(Xaz)
or equivalently
,
Hence, we can rewrite (3) in the form
all x,z ~ 0
(3b)
l-F(x+z) = [l-F(x)][l-F(z)] ,
all x,z -> 0 .
Although (3) and (3a,b) are equivalent only under the assumption that F(x) is nondegenerate at zero, we can consider (3b) as the equation for the lack of memory, since it yields all the solutions of (3), including the degenerate one.
The term
"classical case" refers to the domain of (3b), namely, that we require it for all x,z -> 0.
In Chapter 2, we shall discuss extensions of this property in several dir-
ections and in Section 5 of this chapter, we shall analyze a statistical meaning of (3b) in the particular case x = z. The solution of (3b) is now well known. Theorem 1.3. I. bution functions.
We present this solution in
There are only two solutions of the equation (3b) among distri-
Either F(x) is degenerate at zero, or, with some contant b > 0,
F(x) = l-e -bx, x >- 0. Remark.
Obviously, the distribution function which is degenerate at zero is a
solution of (3b).
Hence we have to prove that if F(x) is a solution of (3b) which is
nondegenerate at zero then F(x) is of the exponential form as stated in the theorem. First Proof: l-F(x). (4)
We first present the classical method of solving (3b).
Set G(x) =
Then, by induction, (3b) implies that for any xj _> 0, 1 -< j -< n,
G(Xl+X2+...+Xn ) = G(Xl)G(x2)...g(Xn)
.
If we choose xj = x for all j, we obtain (4a)
G(nx) = Gn(x)
for any integral value of n.
With x = I/n we now have
G
(s)
G(sn-] ~ Gn/s(1) .
Notice that 0 < G(1) < i.
Indeed, if G(1) = 0 or G(1) = ], (4a) would yield G(y) ---0
or G(y) - 1 which is not possible for G(y) = l-F(y) with F(O) = 0 and F(x)'s being nondegenerate at zero.
Hence b = -log G(1) is finite and positive.
We can therefore
rewrite (S) as (Sa)
G(x) = e
-bx
for any rational x.
Now let x be irrational.
Let {x~ n) } and {x~ n) } be two sequences of rational numbers
such that x~n)< x < x~ n) where n ÷ +~.
and lira x~ n ) =
lira x~n) = x ,
Since G(x) is nonincreasing,
(Sa) yields
exp(-bx~n)) < G(x) < exp(-bx~ n)) ,n~l -
-
.
Letting n ÷ +~ results in G(x) = e -bx for all x ~ 0, which was to be proved. Second Proof:
Let us put again G(x) = l-F(x).
B
Since F(x) is not degenerate at zero,
there is an x 0 > 0 such that 0 < G(x 0) < 1 (otherwise, by (4a), G(x) would be identically one or zero).
In this case, however,
let n be such that nx 0 ~ x < (n+l)x 0. G[(n+l)x0]
~ g(x) ~ G(nx0)
0 < G(x) < 1
for any x > 0.
Indeed,
Then, since G(x) is nonincreasing,
.
On the other hand, by (4a) both G[(n+l)x0]
and G(nx 0) are between zero and one.
Hence, on any finite interval both G(x) and I/G(x) are finite and different from one. (6)
Now let b > 0
H(x) = G(x)e bx
satisfies H(x0) = I. (7)
be chosen in such a way that the function
Then H(x) satisfies
H(x+z) = H(x)H(z)
and, for any x ~ 0,
(8)
H(x+x 0) = H(x)
.
Moreover, b o t h H(x) and 1/H(x) a r e f i n i t e now t h a t t h e r e i s
on an a r b i t r a r y
a 0 < x I < x 0 s u c h t h a t H(x l ) ¢ 1.
finite
interval.
Assume
We may assume t h a t H ( x l ) > 1,
b e c a u s e o t h e r w i s e we c o u l d a r g u e w i t h t h e f u n c t i o n 1 / I t ( x ) , w h i c h a l s o s a t i s f i e s b o u n d e d n e s s p r o p e r t y as w e l l as e q u a t i o n s
(7) and ( 8 ) .
the
Now, by ( 8 ) , f o r any i n t e g e r
N~I, H(Nxl+x0) = H(NXl) , and t h u s by (7) H(NxI+x0) = HN(x1 ) . Since H(x I) > i, this implies that, as N ÷ + % limH(NXl+X0) However,
= +~
this limit leads to a contradiction.
NXl+X 0 = Mx0+Y H(NXl+X0)
Indeed, by (8), if we write
with 0 ; y < x0, = H(y),
and thus H(NXl+X 0) is bounded by the bound of H(y) on the finite interval which was shown to be a finite number, x I > 0, H(Xl) ¢ i, and thus G(x)e bx = tt(x) ~ i , which proves the assertion.
[0,x0),
llence it is not possible that, for some
10
The first proof is the best known one and it reappears in the literature.
It
actually goes back to A.C. Cauchy (1821) and G. Darbaux (1875) (see, e.g., J. Acz61 (1966)).
The second proof is essentially due to G.S. Young (1958).
We shall see
further proofs in Section 5 of the present chapter (where several equivalent forms of (3b) are established) and several extensions are given in Chapter 2.
1.4.
Motivations
(Applications of characterization theorems in model building).
In applied statistics, one observes a random quantity X a number of times and, based on these observations, one would like to conclude facts about the distribution F(x) of X.
The usual approach is to start with a family F of distributions and to
select from this family a distribution F(x) which is the most acceptable one in a given sense.
Unfortunately, in many cases, F simply consists of a single function
which is dependent on one or several parameters, and the observations are used merely to approximate its parameters (estimation). F(x).
The function thus obtained is chosen as
Characterization theorems are the only methods which allow us to avoid the
subjective choice of F and lead to the accurate F(x) through simple properties.
To
show how important the accuracy of the population distribution is, we borrow an example from the book of J. Galambos (1978). Let Y be a standard lognormal variate and let X = (yt-l)/t, t > 0.
It is well
known (see, e.g., Johnson and Kotz (1970a)) that if t is small, then the distribution of X is close to the standard normal distribution.
(Indeed, by Taylor's expansion
X = (Yt-l)/t = (et log Y-l)/t = log Y + ut(log y)2 , where lul ~ i.
Hence, as t ÷ 0, the second term on the extreme fight hand side tends
to zero in probability.
Thus, elementary probability theory shows that the distri-
bution of X approaches that of log Y.)
Let us therefore compare the values H(2.6)
of the distribution function If(x) of the maximum of 50 observations on X under two different assumptions:
X is normal and then X =(y0"l-1)/0.1.
Easy calculations
yield that H(2.6) = 0.59
under normality
H(2.6) = 0.78
under lognormality .
The fact that goodness of fit tests would accept either one of the assumptions on the distribution of X, when the latter is the true form of X, can be seen from S. Kotz's (1973) analysis.
It is evident that the decision should be different in the
two cases, but classical statistics is simply unable to solve this controversy.
An
acceptable characterization theorem, however, can lead to a clear-cut distinction between such choices. We describe here three applications of characterization theorems for illustration purposes.
We wish to emphasize their simplicity and the nonmathematical
nature of the underlying assumptions leading to the characterizations.
11
Insurance companies can collect accident records (history) of drivers.
They
label a driver "good" if the driver has the property that his probability of having an accident remains the same small number irrespective of the passage of time. mathematical terms, this can be translated as follows.
In
Let X be the random time
period up to the first accident of the driver in question.
Then
P(X ~ s + uIX ~ u) = P(X ~ s) . This is the lack of memory property of the preceding section, which, as was seen, has a single nondegenerate solution in P(X < x), namely the exponential distribution (we shall return to this point in the next section).
Therefore, the insurance company
is not subject to guessing in determining its risk for such a driver, but they can go ahead with a well defined model for setting the amount of the premium. Let us now turn to another type of problem. of n persons.
Let a store service a con~nunity
These persons visit the store independently of each other and their
actual times of entering the store have the same distribution.
Therefore, the n
individuals can be associated with n independent and identically distributed random variables, namely, with their random times Xj, 1 s j s n, of entering the store. The store owner evidently observes the X. in an increasing order. Let ] Xl:nSX2:nS...SXn:n be the successive arrivals at the store. Assume that the store owner observes that the interarrivals Xl:n,X2:n-Xl:n .... ,Xn_l:n-Xn: n are also independent.
We shall see in Section 3.3 that this fact implies that the con~non distri-
bution F(x) of the Xj is again exponential. arrived at a single possibility for F(x).
The emphasis here is that we have again This means that the advice to all shops,
or big department stores, depends on a single, well defined model, when a decision is to be made on the number of employees, availability of items, etc. We formulate our third model in terms of telephones, but we shall see in Chapter 4,
that this model is equivalent to several others (a fact which was not recog-
nized earlier).
Let calls arrive at a switchboard at independent and identically
distributed random intervals.
Each call reaches the intended person with probability
p and the call does not go through with probability l-p.
If the intervals between
the calls which go through are again independent and, apart from a scale factor, have the same distribution as the intervals between the actual calls, then the calls arrive according to a so-called Poisson process. obtained here greatly facilitates planning.
The fact that a single model is
For example, if a company receives sales
orders through the telephone, the company can plan ahead with the number of telephone lines to be installed. A single model in each of the preceding examples represents a characterization theorem.
We hope that they will illustrate the advantage and importance of having a
characterization result.
On the other hand, our first example has pointed out the
disadvantage of the lack of a characterization theorem for a specific problem. succeeding sections contain a variety of characterization theorems.
The
While we shall
describe some other applied models as well, we do not intend to translate each
42
theorem into a specific model.
We would lose the advantages of mathematical abstrac-
tion by giving the impression that each theorem is presumably associated with a speci fic practical problem.
This practice persists
in the literature and this is one of
the reasons that closely related results were rediscovered so many times without recognizing their apparent similarities.
1.5. Basic properties of the exponential distribution Let X be a random variable with distribution (9)
Fx(X) = l-e -b(x-a)
function
, x z a, b > 0 .
Then X is called a (negative) exponential variate.
Since Y = X + a has the distri-
bution (i0)
F(x) = l-e -bx , x ~ O , b > O ,
many properties of X are equivalent to those of Y.
A further simplification of the
exponential distribution is possiblc by considering Z = bY. (ii)
Its distribution is
Fz(X ) = l-e -x , x a 0 ,
which is called the unit exponential distribution and Z a unit exponential variate. The following properties of F(x) of (I0) are immediate: (pl_)
(1-F(x))'
= -btl-Ftx))
, x > O, F(O+)
= 0 ;
+~
(P2__)
( 1 - F ( x ) ) d x = F1 (I-F(z))
f
for all z > 0 ' F(0+) = 0 '•
Z
(E~)
1-F(x+z) = ( 1 - F ( x ) ) ( 1 - F ( z ) )
for all x,z e 0 .
B e f o r e g o i n g on w i t h t h e d i s c u s s i o n we w i s h t o g i v e t h e r e a s o n f o r i n c l u d i n g F(0+) = 0 i n p r o p e r t i e s
(Pl) and (P2).
of the exponential distribution
Our aim w i t h l i s t i n g
some b a s i c p r o p e r t i e s
is to arrive at characterization
theorems.
we drop F(0+) = 0 i n (Pl) and (P2) t h e n e v i d e n t l y t h e s e p r o p e r t i e s 1-F(x) i s r e p l a c e d by c [ 1 - F ( x ) ] w i t h a r b i t r a r y F(x) = 1-e - b x , b > O, s a t i s f i e s
Fc(X) =
1 - c e bx
if x > 0
0
otherwise ,
where 0 ~ c s l and b>0.
c o n s t a n t c > 0.
these properties,
so do a l l
The above f a m i l y o f d i s t r i b u t i o n s
Now, i f
remain t o h o l d i f
Therefore, since
functions
has a jump a t z e r o .
That i s ,
if X is a random variable with distribution function Fc(X) then P(X=0) = l-c. sequently, F(0+).
Con-
(Pl) and (P2) cannot lead to a characterization without some assunrption on
The assumption F(0+) = 0 actually selects the exponential distribution
from the family Fc(X ) .
By this choice the degenerate distribution,
sented by Fc(X ) with c = 0, is also excluded from (Pl) and (P2), while fied for the degenerate distribution at zero.
(i0)
that is repre(P3) is satis-
Therefore, when we speak of the
13
relation of (P1) and (P3), or (P2) and (P3), we limit our discussion to nondegenerate distributions. Let us observe that (P3) can be written in the following equivalent form (P3*) P(Y ~ x+zIY ~ z) = P(Y ~ x)
for all x,z ~ 0 .
Recall that the property (P3*) is called the lack of memory.
While it is evident
that (PI) and (P2) imply that (i0) holds, and thus each of them is a characterization theorem, the equivalence of (P3) and (i0) among nondegenerate distributions is much less evident.
However, we have settled this equivalence in the preceding section.
Let us draw the attention of the reader to the important fact that the domain for which a property holds is an essential part of that property.
For example, if (P3),
or its equivalent form (P3*), is assumed to be valid for some values of x and z only, then functions other than (i0) may satisfy it.
The most significant of these possi-
bilities is the case when the equation of (P3) is required to hold for positive integers x and z only.
Then its solution is the distribution function of a geometric
variate U, whose distribution is given by the sequence P(U=k) = pk(l-p), k = 0,1,2,...,0 < p < i. (i)
This fact ir~aediately raises a question
on how large a set {(x,z)} must the equation of (P3) be valid to guarantee that its only solution be the function given in (i0)?
We reformulate this question in a more general manner: (ii)
given any property which leads to the exponential distribution, how does this property relate to (P3*)?
We concentrate on (P3*) since it is a nonmathematical property and it is easy to decide in practice whether it can be assumed in a given situation.
Namely, if we
interpret Y related to (I0) as the random life length of an item A, then (P3*) says that '~ is not ageing," but its life is terminated by a sudden event.
This interpre-
tation is evident, since (P3*) can be translated as saying that if A has lived for at least z units of time, then it will live for an additional x units with the same probability as a new item similar to A.
Therefore, if a property (Pj) characterizes
the exponential distribution (I0), then it is necessarily a new interpretation of the lack of memory property.
The properties so far discussed are (Pj), j = 1,2,3, which
are seen by the above indirect argument to be equivalent to (P3*). Let us, however, establish this equivalence by a direct mathematical proof. and (P3*) is of course trivial.
This again follows from a very simple and short argument. z to infinity, we obtain (P2).
The equivalence of (P3)
We next establish the equivalence of (PI) and (P2). If we integrate (PI) from
On the other hand, observing first that the left hand
side of (P2) is continuous and thus so is the right hand side, we get that the two sides are differentiable.
Differentiation now yields (PI), which shows that (PI)
and (P2) are indeed equivalent.
It now suffices to prove the equivalence of (P2) and
(P3), say, in order to establish the equivalence of each of (Pj), j = 1,2,3 and (P3*) (recall that we limit our discussion to nondegenerate distributions).
14
We first show that, for a function satisfying
(12)
(P3),
/ (i F(x))dx = 1/b 0
is finite.
Indeed, if we apply (P3) n times with x = z, we get, for all integers
n ~ 1 and all real numbers z > 0 (t3a)
1-F(nz)
= (1-F(z)) n
or, for all integers n ~ 1 and all real numbers x > 0, (13b) l-F(x) = (l-F(x/n)) n . Thus, applying
(13b) with x = n, we have
f (l-F(x))dx 0 where F(1)
[ (1-F(n)) n=0
> 0, s i n c e
contradicts
=
otherwise
the definition
with respect is
s
(13b) w o u l d i m p l y t h a t
of a distribution
to x from zero to infinity,
i m m e d i a t e s i n c e we a s s u m e d t h a t
seen as follows.
If
the function
= ebX(1-F(x))
stant
[ (l-F(1)) n = l/F(1) n=0
g(x)
B ~ 0, g ( x ) / B g(x+z)/B
=
(P2) h o l d s
F(x)
= 1, f o r a l l
x > 0.
{g(x)/B}{g(z)/B}
F(x)
function.
we g e t
(P2)
(Pl),
g'(x)
= 0 for all
x, which
I f we now i n t e g r a t e (the fact
is nondegenerate).
then so does satisfies
,
that
F(0+)
The c o n v e r s e
a s was s e e n e a r l i e r .
= 0 and thus,
with
(P3) = 0 in
is
(P3)
easily
But then
a suitable
con-
Consequently,
for all
x,z
> 0 ,
or, for all x,z > 0, S(1-F(x+z))
=
(1-F(x))(1-F(z))
.
Letting z tend to zero yields B = 1 and thus (P3) follows. This rather brief discussion, when (13a) is properly reformulated, a new characterization important meaning.
and raises further problems.
Namely,
leads us to
(13a) has the following
If a random variable Z has distribution F(x) and ZI,Z2,...,Z n are
independent observations
on Z, then
ZI: n = min(Zi,Z 2 .... ,Zn) satisfies P(ZI: n ~ x ) Thus,
= (l-F(x)) n .
(13a) implies that, if F(x) satisfies the lack of memory property
(P3*) then
(P4*) ZI: n is distributed as Z/n for all n ~ i. Let us show that (P4*) is in fact a characteristic property of the exponential distribution, provided that Z is nondegenerate. that (P4*) implies (13a,b).
(P3).
Evidently it is sufficient to prove
In other words, we want to deduce the validity of (P3) from
For this aim, we shall show that a function satisfying
(13a,b) is continu-
15
ous for all x ~ 0 and it satisfies
(P3) for all rational x,z > 0.
evidently imply the validity of (P3) for all x,z > 0. continuous at x = 0. F(0) = 0.
These two facts
We first prove that F(x) is
Indeed, since F(x) is nondecreasing,(13a)
implies that
Since we have excluded from our investigation the possibility of F(x) be-
ing degenerate at zero,
(13a) also implies that F(0+) = 0, which yields our claim.
Next, we deduce from (13a,b) that (P3) holds for rational values of x,z > 0.
Putting
G(x) = l-F(x), we have from (13a,b) that, for positive integers m and n, G(m+n) = G(1) m+n = G(1)mG(1) n = G(m)G(n) and, for rational numbers x = u/v and z = p/q , G(x+z) = C ( U q _ ~ ) = G (1/qv) uq+vp = G(1/qv) uqc (1/qv) vp = C(uq/qv)G(vp/qv) = g(x)g(z)
.
We now show t h a t g(x) i s c o n t i n u o u s a t an a r b i t r a r y l e t x and z be r a t i o n a l
x = x 0.
numbers such t h a t x < x 0 < x+z.
I n t h e above e q u a t i o n ,
Let x ÷ x 0 and z + 0.
Be-
c a u s e we have e s t a b l i s h e d t h a t G(x) i s c o n t i n u o u s a t z e r o and t h a t G(0) = 1, we o b t a i n t h a t G(x0+0 ) = G(x0-0 ) = G(x0) , which p r o v e s t h e c o n t i n u i t y o f g(x) a t x 0. Observe t h a t s i n c e (P4*) i s m a t h e m a t i c a l l y e q u i v a l e n t t o (13a,b) t h e e q u i v a l e n c e b e t w e e n (P4*) and (P3) i s an answer t o t h e q u e s t i o n ( i ) p o s e d e a r l i e r section. it
in this
We do n o t d i s c u s s t h i s q u e s t i o n f u r t h e r h e r e , s i n c e a c o m p l e t e answer t o
i s g i v e n i n a s e p a r a t e s e c t i o n d e v o t e d t o t h e l a c k o f memory p r o p e r t y .
Here we
o n l y remark t h a t t h e o t h e r s o l u t i o n s t o t h i s q u e s t i o n i n v o l v e o n l y m a t h e m a t i c a l niceties. Let us r e t u r n t o (Pl) and (P2) and l e t us r e s t a t e language.
them i n an a p p l i e d s c i e n t i s t ' s
For t h i s p u r p o s e we s h a l l u s e t h e f o l l o w i n g n o t a t i o n .
Definition. its h a z a r d r a t e r(x) =
Let F(x) be an a b s o l u t e l y c o n t i n u o u s d i s t r i b u t i o n
Then
r(x) is defined as
F' (x) 1-F(x)
•
The c o n c e p t o f h a z a r d r a t e p l a y s a p i v o t a l r o l e i n r e l i a b i l i t y ance s t a t i s t i c s Evidently,
function.
(where i t
is termed the " f o r c e of m o r t a l i t y " )
theory, in insur-
and i n b i o m e t r i c s .
(Pl) i s e q u i v a l e n t t o
(Pl*) t h e h a z a r d r a t e i s c o n s t a n t . T u r n i n g t o (P2), one can e a s i l y o b s e r v e t h a t i t conditional expectation.
Let Z r e p r e s e n t a l i f e
function F(x) should satisfy F(0) = 0. definition,
is related
to the notion of
l e n g t h and t h u s i t s d i s t r i b u t i o n
Assume that E(Z) is finite.
Then, by
~6
E(Z_zlZez) = f (Z_z)dp(.iZ~z)
=
1
!
p(z--27~_>z{z z}
I
= 1-F(z-----~{ (x-z)dP(Z<x)
(Z-z)dP
.
Thus, by formula (2) o f Len~na 1 . 2 . 1 , +m
f (1-F(x))dx E(Z-zIZez) = z Therefore,
1-F(z)
(P2) can be r e s t a t e d
as
(P2__~*) E(Z-zlZ~z) i s c o n s t a n t in z , z ~ 0 and P(Z ~ 0) = 0 . By t r a n s l a t i n g
(Pl) and (P2) i n t o (Pl *) and (P2*), t h e i r s e e m i n g l y p u r e l y mathemati-
cal content attains
important practical
significance.
While an a p p l i e d argument
would h a r d l y e v e r l e a d to (P2), i t s e q u i v a l e n t form (P2*) has a simple meaning i n tenm~ o f a p p l i c a t i o n s .
I f an i t e m has s u r v i v e d f o r z u n i t s ,
its remaining lifetime
Z-z has e x a c t l y t h e same e x p e c t e d v a l u e as when i t was new. It is instructive
to compare (P2 *) and (P3*).
clear that they are equivalent. lence is quite surprising,
From p r e v i o u s arguments i t i s
However, i f we examine t h e i r meaning, t h i s e q u i v a -
since logically
(P2*) i s a weaker assumption than (P3*),
i f one i s not concerned w i t h t h e f i n e m a t h e m a t i c a l p o i n t s i n t h e assumptions. in (P2), the i n t e g r a b i l i t y
o f 1-F(x) i s an assumption, w h i l e i n (P3), t h i s p r o p e r t y
was deduced as a consequence o f an i n i t i a l cit in their looked.
NamelL
assumption.
These a r e , o f c o u r s e ,
impli-
e q u i v a l e n t forms (P2*) and (P3*), which t h e r e f o r e should n o t be o v e r -
Yet, when s t a t i s t i c a l
methods a r e used to check a g i v e n assumption,
(P2*)
i s indeed s i m p l e r than (P3*). Let us emphasize h e r e t h e f a c t t h a t the f o u r p r o p e r t i e s and (P4 *) r e p r e s e n t f o u r seemingly d i f f e r e n t in the literature established.
f o r model b u i l d i n g .
It is particularly
a characterization
(P2*),
(P3 ~)
a p p l i e d problems and t h e y have been used
However, t h e i r
interrelations
p l e a s a n t to s e e t h a t t h e p r o p e r t y
i n terms o f o r d e r s t a t i s t i c s ,
tary theory of characterizations
(Pl*),
have n e v e r been (P4~), which g i v e s
i s an i n t e g r a l p a r t o f the elemen-
of the exponential distribution.
The r e a d e r i s
a l s o a d v i s e d to keep t h e f o l l o w i n g f a c t i n mind f o r t h e forthcoming c h a p t e r s : the f o u r p r o p e r t i e s
discussed are equivalent,
a u t o m a t i c a l l y g i v e s an e x t e n s i o n o f t h e o t h e r s as w e l l . equivalent properties
A summary o f t h e s e f o u r
i s p r e s e n t e d i n Table ] on t h e n e x t page.
In t h e p r e c e d i n g s e c t i o n we have g i v e n r e f e r e n c e s f o r t h e c l a s s i c a l o f t h e l a c k o f memory, which i s our p r o p e r t y and t h e i r
solutions,
since
an e x t e n s i o n o f one o f t h e s e p r o p e r t i e s
(P3) or (P3*).
Properties
solution (Pl) and (P2)
can be found in any e l e m e n t a r y t e x t b o o k on c a l c u l u s as t h e
s i m p l e s t p o s s i b l e form o f a l i n e a r d i f f e r e n t i a l
or integral
equation.
Strangely
indeed, t h e i r r e f o r m u l a t i o n in t h e forms o f (Pl *) and (P2*) induced much r e s e a r c h and p u b l i c a t i o n .
(P2*) f i r s t
appears as an e x e r c i s e i n t h e monograph o f D.R. Cox
17
g,
g-,
g, I,q
"~=
~-o
4.~ .r4
U
~
4J
0
~2t
q-I q4
{D
•~
©
0
~
©
.~
.~
c~ I= ~
4~
0
bq
0
b3
~o
.r-I
o°
~
o oU~
=
U
,--~oTM
o
,~
U
,.-..,i
© 4-3
4-}
Q
©
© I.-4-O >'-, c'-.
O
O
II
II
~'~
;
O
O
AI
,.-to
N
A[ A " ~ '
X.aq
N
"2 i
,4
r-I
"-4" O 4-
f..q ~
II
i
,, •g,
@ II
-
,--p
'-4'
+
,--p
,q
,q
,q
8
II
,,~-- , " 2I,q II
18
(1962, p. 128) and then in the papers of G. Guerrieri (1965), H.M. Cundy {1966) and H.E. Reinhardt (1968).
Cundy mentions that he heard on a radio program that the
expectation of life for the average sparrow is about 1.2 years, independently of the age of the bird.
This induced his investigation, where the initial assumption is
clearly (P2*) and it thus leads to the exponential distribution as the life distribution of sparrows.
In the solution, Cundy ass~es absolute continuity which was
dropped by Reinhardt.
The result is reobtained in many other publications.
several works, (P2*) is discussed in the following equivalent form:
In
let h(t) be a
strictly increasing and nonnegative function and such that h(A) = 0 and h(B) = +~ for some A < B. (14)
Let T be a random variable and assume that
E{h(r) IT>y} = h(y) + h{b)
where b is a constant.
for all A ~ y < B ,
It is then deduced that h(T) is exponential.
The above pro-
perty clearly coincides with (P2*), since, by assumptions, {T>y} = {h{T) > h(y)} and thus with Z = h{T) and z = h(y), we have the following form of (14): E(ZIZ>z) = z+h(b)
for all 0 ~ z < +~ .
This reformulation of {P2*) is due to M.A. Hamdan (1972) and it was followed by several others.
(The main purpose of G.B. Swartz's (1975) short note is, e.g., to
point out that (14) and {P2*) are equivalent.) The fact that (P4*) characterizes the exponential distribution is a special case of the results by R.A. Fisher and L.H.C. Tippett (1928) and M. Fr6chet (1927). It was also reobtained by M.M. Desu (1971), using a somewhat different approach. We summarize properties {Pj) and (Pj*), 1 < j ~ 4 in Table i.
In this table,
we refer to (13a) as (P4), which is the mathematical equivalent of (P4*). The table is given on the preceding page. Several extensions of the characterization of the exponential distribution based on (Pj) or (Pj*), 1 s j s 4, are given in Chapters 2 and 3.
They will provide
a general answer to both of our questions (i) and (ii) posed in the beginning of the present section. 1.6.
Monotonic t r a n s f o r m a t i o n s o f the e x p o n e n t i a l d i s t r i b u t i o n Before proceeding with the discussion of the exponential distribution, we
should point out that all of its properties can be translated into a property of an arbitrary continuous distribution.
Indeed, if X is a random variable with continu-
ous distribution function F(x) then (15)
Y = - l o g F(X)
or C16)
w = -log[Z-F(X)]
are u n i t e x p o n e n t i a l v a r i a t e s .
Although not a l l p r o p e r t i e s o f Y and W w i l l
properties of X, and vice versa, most of them will.
be usable
Therefore, a discussion of
19
characterizations of the exponential distribution actually yields a family of characterization results.
Let us write (15) and (16) in detail for a number of specific
distributions which are widely used in applications.
At this point, we rewrite only
one property of the exponential distribution for each of the distributions listed. The obtained characterizing property is possibly the most indicative for the transformed distribution. i.
The uniform distribution.
that Y = -log X is unit exponential.
If F(x) = x for 0 ~ x ~ I, then (15) implies The lack of memory property becomes
P(-log X ~ x+zl-log X ~ z) = P(-log X ~ x), all x,z > 0 . With the substitution e -x = u and e -z ~ v, we thus have directly from the lack of memory characterization that the property P(X ~ uvIX ~ v) = P(X ~ u) , 0 ~ u, v ~ 1 characterizes the uniform distribution.
One couldperhaps call it a lack of muZtipli-
cative memory as compared with the lack of additive memory for the exponential distribution. 2.
The Weibull distribution.
Let a > 0.
Then the Weibull distribution is
defined as F(x) = 1 - exp(-x a) , x ~ 0 . By (16), W = X a is a unit exponential variate.
Because a > 0, we have immediately
from (P4~) the following characteristic property of the Weibull distribution.
If
XI: n is the minimum of n independent observations on X then nl/aXl: n is distributed as the population X. 3.
The Pareto distribution.
F(x) = 1 - x -a,
A random variable X with distribution function
a > 0, x ~ I,
is called a Pareto variate.
The transformation (16) becomes here W = a log X and
the lack of memory property states in this case that
P(X ~ uvlX ~ v) = P(X ~ u) , u , v ~ 1 . We t h u s o b t a i n e d t h a t t h i s p r o p e r t y i s a c h a r a c t e r i z a t i o n 4.
The e x t r e m e v a l u e d i s t r i b u t i o n
exp(-e-X).
of the Pareto distribution.
If the distribution
of X is
F(x) = e x p ( - e - X ) , t h e n (15) y i e l d s t h a t Y = exp(-X) i s a u n i t e x p o n e n t i a l v a r i a t e . We now transform (P4*) to obtain a characterization of the above distribution of X. Let Xn: n be the maximum of n independent observations on X. value distribution F(x) if, and only if, X
n:n
Then X has the extreme
- log n is distributed as X for all
nkl. 5.
The logistic distribution.
tion by the formula
We define the one parameter logistic distribu-
20
F(x) = 1 / ( l + e -ax)
, a ~ 0, x r e a l
.
By ( 1 6 ) , Y = l o g ( l + e -ax) i s a u n i t e x p o n e n t i a l v a r i a t e . characteristic
property of the logistic
distribution
L e t us now d e t e r m i n e t h e
via the property
(P2) f o r Y.
Since 1 P(Y ) x) = P[X ( - ~ l o g ( e X - 1 ) ]
, x ) 0 ,
we h a v e f F[- ~1 l o g ( e X _ l ) ] & x = al F[- ~1 l o g ( e Z _ l ) ] Z
as a c h a r a c t e r i s t i c simplified
property of the logistic
by t h e s u b s t i t u t i o n
distribution.
u = (-1/a)log(eX-1).
T h i s can o f c o u r s e be
The e q u a t i o n above becomes
t f F(u) du = a - 2 F ( t ) _~ l + e au
.
Because o r d e r s t a t i s t i c s tion theorems in the sequel, vation.
w i l l p l a y an i m p o r t a n t r o l e i n s e v e r a l c h a r a c t e r i z a l e t us c o n c l u d e t h i s
If a double subscript
section with the following obser-
as i n Zk: n d e n o t e s t h e k - t h order s t a t i s t i c
p l e o f s i z e n on a random v a r i a b l e
Z, t h e n t h e o r d e r s t a t i s t i c s
a b l e s o f (15) and (16) a r e r e l a t e d
by t h e f o r m u l a s
(17)
o f a sam-
on t h e random v a r i -
Yk:n = - l o g F ( X n _ k + l : n )
and
(18)
Wk: n = - l o g [ 1 - F ( X k : n ) ] •
These e q u a t i o n s w i l l b e s t a n d a r d and f r e q u e n t l y u s e d f o r t r a n s l a t i n g the exponential distribution example, d i f f e r e n c e s
of order statistics
transformed into ratios transformations
to those of the six distributions
of order statistics
above.
of the exponential distribution of the uniform distribution.
f o r t h e u n i f o r m as w e l l as f o r o t h e r d i s t r i b u t i o n s ,
succeeding chapters.
properties
listed
of For
w i l l be Other
w i l l be g i v e n in
CHAPTER 2 CHARACTERIZATIONS BASED ON TRUNCATED DISTRIBUTIONS One can view the lack of memory property (l)
P(X ~ x+zlX ~ z) = P(X ~ x) , all x,z ~ 0 ,
where we assume that the distribution of X is nondegenerate, as a characterization of the exponential distribution based on a property on the truncated distribution P(X~yIX~z), y ~ z ~ O, of X.
Another characterization based on a truncation proper-
ty, namely, that
(2)
E(XlXez) = z+c 1, z > 0, P(Xz0) = 0 ,
where E(X) i s f i n i t e e q u i v a l e n t t o (1). characterizations
and c 1 i s c o n s t a n t , was shown i n S e c t i o n 5 o f C h a p t e r 1 t o be The p r e s e n t c h a p t e r i s d e v o t e d t o s e v e r a l e x t e n s i o n s o f t h e s e
of the exponential distribution.
B e f o r e we s t a r t w i t h d e t a i l s , distribution
l e t us r e e m p h a s i z e t h a t a c h a r a c t e r i z a t i o n
of a
o b t a i n e d from (1) o r (2) by a m o n o t o n i c t r a n s f o r m a t i o n s h o u l d n o t be
c o n s i d e r e d as an e x t e n s i o n . ponential distribution
Therefore, every characterization
t h e o r e m f o r t h e ex-
is in fact a family of characterizations.
For example, i f we
assume t h a t (2a)
E(X2[Xez) = z2+c2 , z > 0, P(X~0) = 0 ,
where c 2 i s a c o n s t a n t , t h e n , w i t h h(x) = / x and Xe = h ( X ) , which we know t h a t X~ i s a n e g a t i v e e x p o n e n t i a l v a r i a t e . P(X<x) = P(X* < ¢~) = 1 - e x p ( - b ¢ ~ ) is a Weibull v a r i a t e .
, x > 0 ,
strictly
i n an e q u i v a l e n t form l i k e t h a t o f ( 2 a ) .
increasing h(x),
Similarly,
(2) can be r e w r i t t e n
the equation
p(X2zx2+z21Xez) = P(Xex) , a l l x , z > 0 ,
does n o t d i f f e r
from (1).
and z a r e r e p l a c e d by t h e i r x 2 = x 1 and z 2 = z 1. again.
Consequently,
I n t h i s d i s c u s s i o n t h e s p e c i a l form o f h ( x ) above was, o f
c o u r s e , i m m a t e r i a l ; w i t h an a r b i t r a r y
(la)
(2a) becomes ( 2 ) , from
Because t h e s e t s {X>z} and {Xzx} a r e n o t changed i f X, x squares (since x,z > 0),
Hence, (1) i m p l i e s t h a t
( l a ) becomes (1) w i t h X2 = Xl,
(la) c h a r a c t e r i z e s
Here, o f c o u r s e , X2 can be r e p l a c e d by Xa w i t h a r b i t r a r y
a Weibull v a r i a t e a > 0.
This f a c t
was o v e r l o o k e d i n t h e p a p e r by Y.H. Wang (1976). Let us now t u r n t o some e x t e n s i o n s o f c h a r a c t e r i z i n g t i o n by r e l a x i n g t h e c o n d i t i o n s i n (1) and ( 2 ) .
the exponential distribu-
We s t a t e most t h e o r e m s f o r t h e
22
exponential distribution.
But since they represent a family of oharacterizations,
the reader is advised to restate each theorem in several equivalent forms. cular, it is r e c o ~ e n d e d
In parti-
to restate the results which follow for the distributions
listed in Section 6 of Chapter I.
2.1.
Extended forms o f the lack o f memory p r o p e r t y We have seen in Section 5 of Chapter I, that the condition "all x,z ~ 0" in
(I) can be relaxed without enlarging the set of distributions (i).
As a matter of fact, if we write
(ib)
l-F(x+z) = [l-F(z)][l-F(x)]
satisfying equation
(I) as
, x,z ~ 0 ,
where F(x) is the distribution function of X, we obtain by induction (3)
G(Xl+X2+...+Xn ) = G(Xl)G(x2)...G(Xn)
where xj ~ 0 and G(x) = l-F(x). was that (3) characterizes
,
Now, one of the results in Section 5 of Chapter 1
the exponential distribution if (3) holds for all integers
n ~ 1 and Xl=X2=...=Xn = x ~ 0.
We now give several other extensions of the classi-
cal case. We first prove a result presented in R. Fortet's monograph.
(The English
translation of this monograph appeared in 1977; see also the works of Jurkat
(1965)
and de Bruijn (1966).) Theorem 2.1.1.
Let F(x) be a nondegenerate distribution function.
Assume
that F(x) satisfies the equation of (Ib) for almost all x ~ 0 and z ~ 0 with respect to
Lebesgue
Proof:
measure.
Then F(x) = l-e -bx, x -> 0, where b > 0 is a constant.
Recall from Section 2 of Chapter 1 that any distribution function F(x) is
differentiable
for almost all x in the sense of Lebesgue.
Let F'(x) = f(x) be the
derivative of F(x), which is therefore defined for almost all x. Let us rearrange (4)
(Ib) as
F(x+z) - F(z) = F(x)[l-F(z)]
and let us divide this equation by x > 0 to obtain
(5)
F(x+z)-F(z,) x
Now if z ~ 0
_ F(x) x
[1-F(z)]
is such that f(z) is defined, then the limit of the left hand side
exists as x + 0+.
Hence
lira F(x) = b ~ 0 x x=0 + exists and it is finite. any z ~ 0 as x ÷ 0+. get
But then the limit of the left hand side of (5) exists for
Denoting this limit--the right hand derivative--by f÷(z), we
23
(6)
f+(z)
Furthermore,
= b[1-F(z)]
.
we h a v e shown t h a t
But any d i s t r i b u t i o n (5) i s c o n t i n u o u s
function
F(z)
is continuous
is continuous
at every point.
Hence,
from the right
from the left
for all
z -> 0.
and t h u s F ( z ) s a t i s f y i n g
(4) c a n b e r e w r i t t e n
as
F(y) - F(y-t) = F(t)[l-F(y-t)] , t > 0 fr~n which, just as above, the existence of the left hand derivative f-(y) of F(y) follows and, since F(y) is continuous, (7)
f-(y)
= b[1-F(y)]
By (6) and ( 7 ) ,
f(z)
, all
exists
y -> 0 .
for all
z -> 0 and
f(z) = b[l-F(z)] . The solution of this last equation is the claimed exponential function with b > 0, because F(x) was assumed to be nondegenerate.
The proof is completed.
Let us modify the assumption of Theorem 2.1.I as follows.
Let Q be the
uniquely determined measure on the Borel sets of the nonnegative real line which, for intervals [c,d), 0 -< c < d is defined by Q{[c,d)} = F(d)
- F(c)
, c ~ 0 .
Now, if we assume that B is a Borel set of nonnegative real numbers such that Q(B) = 1 and (Ib) holds for all x,zcB, then we are again dealing with the validity of (ib) for almost all x,z but now with respect to Q rather than with respect to Lebesgue measure.
This change has a substantial effect on the conclusion.
Namely,
now not only the exponential function F(x) = l-e -bx, x -> 0, b > 0, but other functions as well can satisfy (Ib). For example, if X is a nonnegative integer-valued random variable with P(X=k) = pk(l-p) , 0 < p < I, k = 0,1,2,... , or, equivalently, (8)
F(x) = P(X<x) =
~ pk(1-p) 0-
,
(a geometric variate), then F(x) as given by (8) we choose B = (0,i,2,...).
is a possible solution of (ib) if
Indeed, B is a Borel set and Q(B) = 1 under (8).
Furthermore, if zcB, that is, z is a nonnegative integer, then 1-F(z)
=
+~pk (l_p) = p Z ( 1 - p ) +~ [ [ pk = p z , k=z k=0
which evidently satisfies (Ib) for all x,zEB.
The difficulty involved in this
approach is that the set B is defined in terms of the unknown solution F(x) of (ib) We do not discuss this problem in detail, but refer the reader to R.T. Durrett and S.G. Ghurye (1976).
Our further discussion will be limited to those cases when the
24
domain of the validity of the equation in (ib) does not depend on the unknown solution F (x). Theorem 2.1.2. G(x) = l-F(x).
Let F(x) be a nondegenerate distribution function and put
Assume that (3) holds for two integral values of n, n I and n2, say,
such that (log nl)/(log n2) is irrational, arbitrary.
where in (3) ,
Xl=X2=...=x n -- x ~ 0 is
Then F(x) = l-e -bx, x ~ 0, with some b > 0.
This result is due to J. Sethuraman
(1965).
His result is actually more gen-
eral than the statement above; the more general case will be discussed in subsequent chapters.
Theorem 2.1.2 was later reobtained by B. Arnold ~1971)
(assuming n 2 =
nl+l), whose proof is different from the one presented below. Proof:
By assumption, Gn(x) = G(nx)
for all x -~ 0
and for n = n I or n 2 .
Since this equation can also be written as G(x/n) = Gl/n(x), we get by induction s t that, for any integer N = nln 2 with arbitrary integers s and t (both negative and positive), (9)
GN(x) = G(Nx)
.
From the elements of mathematics we know that the set (I0)
u = log N = s log n I + t log n 2
is dense on the real line as s and t go through the integers.
Therefore, we can
find two sequences s = s(k) and t = t(k) such that the corresponding u = u(k) ÷ 0. Let u = u(k) be such a sequence.
Then, if we put g(x) = log I-log G(eX)], z = log x
and u = log N, we get from (9) g(u+z)
- g(z)
= u
or
(11)
g(u+z)-g(z)
=
1
.
u
Hence, if u ~ 0, we have g'(z) = 1 whenever the derivative exists (it does not follow from the last equation that the derivative exists because the sequence u is not arbitrary).
Now, because G(z) is monotonic,
of the form of (i0) is dense in R 1.
Therefore,
so is g(z).
In addition,
the set u
if ~v~ is an arbitrary set such
that v + 0 then we can construct two sequences u I < v ~ u 2 such that u i ~ 0, u i is of the form of (I0) and ui/v + i for i=1,2. g(ul+z)-g(z ) uI
Consequently
uI
g(v+z)-g(z)
g(u2+z)-g(z )
u2
v
v
u2
v
which, by (ii), reduce to
25
u I ~ g(v+z)-g(z) ~ _u _2
_
_
V
V
V
Since the two extreme sides tend to one as v ÷ 0, g'(z) exists and equals one.
This
now gives G(e z) = exp(-e z+c) , i.e.,
G(x) = e -bx
b > 0
which was to be proved.
D
By a similar argument, one can prove the following result of G. Marsaglia and A. Tubilla (1975). Theorem 2.].3.
If a distribution function F(x) satisfies (ib) for two values
z I and z 2 of z such that 0 < z I < z 2 and Zl/Z 2 is irrational and for all values of x ~ O, then F(x) is exponential. It may sound artificial to assume that Zl/Z 2 is irrational.
But without this
assumption, the remaining conditions of Theorem 2.1.3 do not lead to a unique distribution F(x).
For example, if we assume that equation (Ib) is satisfied for z = 1
and 2 and for all x ~ O, then several distributions can be obtained as its solutions.
Indeed, if T(x) is an arbitrary distribution function with T(0+) = 0 then
if we write x > 0 as x = n+y where n ~ 0 is an integer and 0 ~ y < 1 , (12)
l-F(x) = pn[l-T(y)]
satisfies
, p=l-F(1)
,
(ib) for all integers z ~ 0 and all x ~ 0.
We can, of course, understand the mathematical reason for the need of Zl/Z 2 to be assumed to be irrational.
Just as in the case of Theorem 2.1.2, an essential
step of the proof of Theorem 2.1.3 is the fact that the set {u: u = sz I + tz 2 , s,t integers} is dense on the real line if Zl/Z 2 is irrational, while this fails for a rational value of Zl/Z 2. In the discussion following the proof of Theorem 2.1.1 we pointed out that a modification of the assumptions of Theorem 2.1.1 can lead, among others, to a geometric variate. (13)
The geometric distribution
P(X=k) = pk(l-p)
, 0 < p < I, k = 0,1,2 ....
has a natural relation to the exponential in that the survival function,for integer x, can be represented as +~ k x -bx P(X>_x) = (l-p) ~ p = p ~= e , k=x where b = log(i/p) > 0, and is therefore formally the same as the survival function of the exponential distribution.
Therefore, an important fact about the distributic~
26 given by (12) is that it has no similarity to the exponential distribution. The fact that the functional equation (14)
f(x+z) = f(x)f(z)
for some x and z ,
may have solutions which do not resemble the exponential function was first pointed out by M.V. Menon (1966).
J. Acz~l (1975) gives a complete solution of (14) if x
and z are assumed to belong to a finite segment of an arithmetic progression (i.e., to a finite lattice). We now turn to another kind of extension of the classical lack of memory characterization.
(15)
Let us write (i) in the form
P(X~x+z) = P(X~x)P(X~z) .
The constants x and z can be considered as random variables with degenerate distributions.
(16)
In other words, if Y and Z are random variables and if
P(XeY+Z) = P(XeY)P(XeZ)
holds for Y and Z with degenerate distributions then X is necessarily exponential (Theorem 1.3.1).
Following this approach, A. Obretenov (1970) gives a new proof of
Theorem 1.3.1using Laplace transforms.
On the other hand, N. Krishnaji (1971) omits
the assumption of Y's and Z's being degenerate. Theorem 2.1.4.
His main result is as follows:
Let D 1 and D 2 be two families of random variables such that
(i) each member of both D 1 and D 2 is independent of X, (ii) each member of D 1 is independent of each member of D 2 and (iii) for every interval [a,b), 0 ~ a < b < +~, there is a member V i of D i such that P(a~Vi
0, i=1,2.
Assume that the distribu-
tion F(x) of X is continuous for x ~ 0, F(0+) = 0 and satisfies (17)
l-F(y+z)
~ [l-F(y)][l-F(z)]
for all y,z ~ 0 .
If (16) holds for all Y from D 1 and all Z from D2, then F(x) is exponential. Proof:
If equality always applies in (17), then the conclusion follows directly
from Theorem 1.3.1 without anyfurther assumptions.
Therefore, let Yl and z I be such
that strict inequality applies in (17) with y = Yl and z = z I.
Since F(.) is contin-
uous by assumption, there are numbers 0 ~ a < b < +~ and 0 ~ c < d < 4~o such that a ~ Yl < b and c ~ z I < d, and strict inequality applies in (17) for all a ~ y ~ b and c ~ z ~ d.
Let us fix a,b,c and d.
spectively, and such that P(a~Y
Let Y and Z be members of D 1 and D2, re-
> 0 and P(c~Z 0.
For this choice of Y and
Z, (16) yie]ds +¢o4~o
(18)
0 = f
f [G(y+z) - G(y)G(z)]dP(Y
bd f f [G(y+z) - G(y)G(z)]dP(Y
,
ac
w h e r e , as u s u a l ,
G(x) = 1 - F ( x ) .
Now, s i n c e G(x) i s c o n t i n u o u s ,
the intervals
[a,b]
27
and [c,d] are finite and strict inequality applies in (17) as y and z vary over [a,b] and [c,d], respectively, the theorem of Weierstrass in integral calculus implies that G(y+z) - G(y)G(z) ~ A > 0
for all a ~ y ~ b, c ~ z ~ d .
Hence, by the choice of Y and Z, the right hand side of (18) is strictly positive which is a contradiction.
Hence, strict inequality cannot apply in (17). The asser
tion of the theorem thus follows from Theorem 1.3.1.
D
Notice that D 1 and D 2 may each contain a single random variable Y and Z, respectively.
Then the assumptions on Y and Z are that they are independent of each
other as well as of X and that for every 0 ~ a < b < +~, P(a~Y 0 and P(a~Z O.
For this special case it would, at least, be interesting to relax the
very restrictive assumptions on X: the validity of (17) and the continuity of F(x). We conclude this section with an apparently new characterization of the logistic distribution which could be reduced to an extended form of the lack of memory. However, the transformed form would be of no interest, while the original characterization may have some practical value. Theorem 2.1.5.
Let the distribution function F(x) of the random variable X
be continuous and sy~aetric about the origin. F(x) -
1 l+e-~X '
Then
~ > 0
if, and only if, P(-x<XIX<x) = l-e -~x
all x > 0 ,
or if, and only if, l-F(x+y)
[1-F(x)] [l-F(y) 1 Proof:
=
F(x+y)
F(x)F(y)
for all x, y ~ 0
The logistic distribution F(x) = (l+e-XX) -I is evidently continuous, syn~e-
tric about the origin, i.e., F(-x) = l-F(x) for all x ~ 0, and satisfies both assump tiQns of the theorem.
Hence, only the converse needs proof.
Because F(x) is symmetric about the origin, P(-x<XIX<x)
2F(x) -I
Since this conditional distribution was assumed to be exponential, 2F(x)-I _ l_e-XX
F (x) the solution of which is evidently the claimed logistic distribution. Now let us assume that the second property holds. symmetry,
By the assumption about
28
G(x) = P(X<-xlX<x) = ~
,
and thus the second assumption reduces to G(x+y) = G(x)G(y)
, all x,y m 0 .
In view of Theorem 1.3.1, its only continuous solution is G(x) = e x m 0.
-~X
with some ~ >0,
llence, the first part of the theorem is applicable, from which it follows
that F(x) is a logistic distribution function.
The proof is completed.
The reader is advised to restate the theorem in the exponential form.
That
is, transform X by our rule of Chapter 1 to the variable y = log(l+e -X) and reformulate the assumptions in terms of Y.
It then becomes clear that the
theorem in the exponential version is too complicated to be of interest.
2.2. A stability theorem As it was already indicated in Chapter i, the subject matter of the theory of characterizations is to show that only one distribution (which may contain one or several parameters) can satisfy a set of specific properties.
In most applications,
however, statistical or mathematical properties can be verified only approximately. The question thus arises whether several different distributions can be the result of an investigation if a property is slightly modified. called the stability theory of characterizations.
This area of research is
In an abstract form, we can form-
ulate the problem as follows. Let A be a property which members of a family F of distributions may or may not possess.
If there is only one member F of F
that satisfies A then A leads to a
characterization theorem (as an example, let A be the lack of memory property and F be the family of all nondegenerate distributions). ty which can be defined for members of F.
Assume that B is another proper-
Furthermore, let the properties A and B
be such that one can assign a precise meaning to the statement: "the distance d(A,B)" between A and B is smaller than c > 0. any two members F 1 and F 2 of F.
Let D(FI,F2) be a distance between
The characteristic property A of F is called stable
if, to any e > 0, there exists a ~ = 6(e) such that sup{D(F,Fl): F possesses A and F 1 possesses B} < whenever d(A,B) < e. Note that a stability theorem has two arbitrary distance concepts involved. Therefore, one can prove several stability theorems in connection with a single characterization theorem.
While a complete unification is hardly possible, we wish
to draw the attention of the reader to a recent attempt in this direction by V.M. Zolotarev (1976).
29
We now present one stability theorem in connection with the lack of memory property.
This result is due to H.N. Hoang (1968).
We introduce this theorem by going
through our definition of stability step-by-step which can also serve to reduce the abstract nature of this definition. Let F be the family of all continuous distributions F(.) with F(0) = 0. be a random variable whose distribution belongs to F. (19)
h(x,y)
= P(X~x+ylX~y)
Let A be the property h(x,y)
that
equals
- P(X~x) h(x,y)
perty
that
(20)
d(A,B) = sup{hl(X,y):
Let X
Set
.
= 0 for
a prescribed
all
function
x ~ 0 a n d y ~ 0, a n d B d e n o t e hl(X,y ).
the pro-
We i n t r o d u c e
x ~ 0, y ~ 0}
and (21)
D(FZ,F
2)
= sup{IFz(X)-~z(x)l:
~
< x
< +~}
,
where F 1 and F 2 are arbitrary members of F (the latter is known in the literature as the Kolmogorov distance). Theorem 2.2.1.
We now establish Hoang's result.
If the distribution function F(x) of X belongs to F and if
F(x) has property B for which d(A,B) < ~, then there is a constant b > 0 such that D(F(x), l-e -bx) < 2 /-6.
In other words, if F(x) almost satisfies the lack of memory then F(x) is close to an exponential distribution in the sense of the Kolmogorov distance. Proof:
Put G(x) = l-F(x).
Then (19) can be rewritten as
G(x+y) = G(y) [ G ( x ) + h ( x , y ) ]
.
T h u s we h a v e G(2x) = G2(x) + G ( x ) h ( x , x )
,
G(3x) = G(2x+x) = G3(x) + G2(x)h(x,x) + G(x)h(2x,x)
,
which can be continued to obtain G(nx) = Gn(x) + Gn-l(x)h(x,x) + Gn-2(x)h(2x,x)+...+G(x)h[(n-l)x,x]
•
Hence, using the assumption d(A,B) < c, we have for any x >- 0 for which G(x) < i, +oo
(z2)
IG(nx) - Gn(x) l < c X Gk(x)
=
k=l Now l e t
(23)
cG(x) < 1-G(x) - 1-~(x)
"
x 0 > 0 and b > 0 be such that
l-G(x0) = F(x0) = /-E = l-exp(-bx0)
.
Such an x 0 > 0 exists because F(x) is continuous and F(0) = 0.
Then, by (22), for
30
all n ~ i,
(24)
]G(nXO) -e -nbxO] < V'E'.
On the other hand, for an arbitrary x > 0 (n-l)x0 ~ x < nx 0.
there is a unique integer n > 1 such thal
Hence, from the monotonicity of G(x) and by (24)
-nbx 0 (25)
/7 + e
-(n-l)bx 0 < G(nx0) ~ G(x) ~ g[(n-l)x 0] < e
+ /7
But, in view of (23), -bx 0-<e
-bnx0
-b(n-l)x0
-e
-<e
-bnx0 -e
-bx0
=gg
-
and similarly -b(n-l)x0
-bx -e
0-<e
_<¢7.
Consequently, (25) results in IG(x) - e-bXl < 2¢~
as claimed. The estimate in the conclusion can be improved.
See T.A. Azlarov, et aZ.
(1972).
2.3.
Characterizations based on conditional expected values Formula (2) expresses an elementary characterization of the exponential dis-
tribution.
We have also mentioned that (2) can be restated in a number of equiva-
lent forms by using monotonic transformations which lead to characterizations of other distributions without providing any new information.
Our aim in this section
is to obtain new characterization theorems by changing the assumptions in (2). First notice that, by letting z tend to zero in (2), we get c I = E(X). D.N. Shanbhag (1970)points out that if c I is specified in this way then the additional assumption P(X~0) = 0 is superfluous. E(xlx~o)
>
E(x)/p(x~o)
w h e n e v e r P(X O.
Namely,
,
Hence
E(XTX~z) = z + E(X)
fails
as z goes to zero.
On t h e o t h e r h a n d ,
would not be a characteristic pointed
out in Section
1.5,
jump a t z e r o , w o u l d s a t i s f y Conversely,
i f we s p e c i f y
property
i f we d r o p P(X~0) = 0 i n ( 2 ) ,
of a single
distribution
a f a m i l y Fc(X ) o f d i s t r i b u t i o n s , (2).
For these distributions,
anymore.
then it As we
each of which has a c I = E(X)/[1-P(X=O)].
c 1 i n (2) a s
E(XlX~z) = z + kE(X), a l l
z > 0, P(X~0) = 1 ,
where k > ! is a given number, then this
e q u a t i o n becomes a c h a r a c t e r i s t i c
property
31
of Fc(X ) with c = I/k. follows.
That is, what we can conclude from this discussion is as
If we assume that E(XIX_>z) is of a special form (z + constant), then sev-
eral distributions can be obtained for X (Fc(X) with arbitrary c), but if we specify the unknowns as in the above equation (we give a meaning to the constant) then we arrive at a characterization theorem.
With this general formulation we shall soon
see that the linearity of E(XIX>-z) is not important.
The simplest form of this fact,
which is due to A.G. Laurent (initiated by him in the 1950's, but for convenience we refer to his published paper of 1974), is given below. obtained
A similar result was also
by M.N. Vartak (1974).
Theorem 2.3.1. 0 -< y < +~.
Let g(y) > 0 be a decreasing and differentiable function for
Assume further that g'(y) _> -i with g'(0) ~ -i and such that
(26) Af l+g' g(y)(y)
dy
0 is finite or infinite according as A is finite or infinite.
If X is a nonnegative
random variable with continuous distribution F(x) and such that (27)
E(X-ylX_>y) = g(y) , all y _> 0 ,
then the distribution F(x) is uniquely determined.
(28)
F(x) = 1-exp{- fx l+_g'(y) dy) 0
Proof:
g (y)
Specifically,
x -~ 0 '
"
As we have seen i n S e c t i o n 1.5 ( a c t u a l l y a consequence o f Lemma t . 2 . 1 ) , +co
f [1 r(x)]dx E(X-y[X->y) = y
1-~(y)
Hence, i f we p u t G(y) = 1 - F ( y ) ,
(27) becomes
+m
g(y) G(y) = f g(x) dx , Y Since by assumption G(x) is continuous (continuity for x > 0 would follow from this last equation but not at x = 0), the right hand side is differentiable. so is the left hand side. G' (y) follows. _ g'(y)
G(y)
Consequently,
But g' (y) exists by assumption and thus the existence of
We can now differentiate and we obtain = l+g'(y)
g(y)
I n t e g r a t i o n and t h e assumption o f c o n t i n u i t y o f F(x) thus y i e l d t i o n s o n g(y) g u a r a n t e e t h a t F(x) o f (28) i s indeed a d i s t r i b u t i o n
(28).
The assump-
function.
This
completes the proof. When g(y) is a given constant then we get back the negative exponential distribution.
But since in most practical applications, a natural assumption is that
32
g(y) is strictly decreasing,
(28) provides the population distribution for a wide
variety of problems. Another modification of (2) can be that instead of considering the conditional expectation E(XIXzz), we replace it by E[h(X) IXez] or E[h(X-z) IXez], where h(.) is a given function.
(29)
In considering the equation
E[h(X) BX~z] = g ( z )
, z ~ 0, P(X~0) = 1 ,
however, we h a v e t o b e c a r e f u l n o t t o r e p e a t thus getting
trivialities
(2) i n d i s g u i s e d b u t e q u i v a l e n t
(see the discussion
following (2a)).
form,
The o t h e r e q u a t i o n
of interest which we mentioned above is
(30)
E [ h ( X - z ) lX~z] = g(z)
where h ( ' )
and g ( . )
are given functions.
such that the resulting extension is possible rather
, z ~ 0 ,
solution, by r e q u i r i n g
t h a n on t h e whole p o s i t i v e All solutions
of h(-)
the validity real
being strictly
monotonic.
case (equation (27)),
h(').
Therefore,
this
function.
Further
o f (29) and (30) on an i n t e r v a l start
with the assumption
T h i s c a s e , however, can e a s i l y be r e d u c e d t o t h e
s i n c e (X~z} = {h(X) ~ h ( z ) } f o r s t r i c t l y
increasing
c a s e does n o t p r e s e n t any new m a t h e m a t i c a l p r o b l e m .
(29) even w i t h m o n o t o n i c h ( ' )
tion of distributions
of course, has to be
line.
o f ( 2 9 ) , known t o t h e p r e s e n t a u t h o r s ,
linear
other hand,
The f u n c t i o n g ( z ) ,
i f any, s h o u l d b e a d i s t r i b u t i o n
On t h e
may g i v e a new and m e a n i n g f u l c h a r a c t e r i z a -
f o r which E(X) i s i n f i n i t e .
For example, i f t h e d i s t r i b u t i o n
function of X is F(x) = l-l/x, x ~ I, then E(X) = +2 but (31)
E(C~lXez)
= 2~
uniquely characterizes
, z e I, P(X~I) = 1 , the distribution F(x) = 1-I/x, x > I, among all continuous
distribution functions. (32)
Indeed, if we put Y = ¢~ and u = erz, then (31) reduces to
E(YIYeu) = 2u , u e I, P(Y~I) = 1 ,
which is of the form (27). function then so does Y.
Now if we assume that X has a continuous distribution Hence, all conditions of Theorem 2.3.1 are satisfied, with
g(u) = u (we apply Theorem 2.3.1 to Y-I to make (321) applicable for all u e 0). Thus (28) yields
P(Y
UZdy}=l_ 7
1 7' u
u_>l
.
Hence
F(x) = P(X<x) = P(y2<x) = P(Y< ¢r~) = 1 - 1
X > 1
X
as claimed. In connection with (29), for various choices of g(.), see the papers by I.I. Kotlarski
(1972), D.N. Shanbhag and M. Bhaskara Rao (1975), R.C. Gupta (1975) and
A.C. Dallas
(1976a).
33
To show that (30) has a unique solution for certain given g(z) is far more difficult.
One cannot appeal here to inverse functions to reduce it to the linear
case even if h(.) is assumed monotonic.
Hence, a direct solution is required.
Even
the simplest case h(u) = u k for some k ~ 2 and g(z) = E(X k) (a constant) is significantly more difficult than the linear case given in Theorem 2.3.1.
The difficulty
lies in the fact that when we reduce (30) to a differential equation, then its order is k and its general solution is thus a linear combination of k independent solutions.
From this, one has to deduce that only one of these linear combinations
can be a distribution function.
This last step can, of course, be avoided by addi-
tional assumptions, see H.N. Nagaraja (1975).
The case k = 2 was treated by several
authors (A.G. Laurent (1972 and 1974), T.A. Azlarov, et al. (1972) and A.C. Dallas (1975)). (1974).
The case of arbitrary k ~ 2 was settled by O.M. Sahobov and A.A. Geshev A general theorem, which is applicable to a general class of functions
h(.), was recently obtained by L.B. Klebanov (1977). tools.
Klebanov uses deep analytic
Because his paper has not yet appeared at the time of this writing, we do
not give details.
However, we present Sahobov and Geshev's result below.
Theorem 2 . 3 . 2 .
Let X ~ 0 be a random variable with distribution function F(x).
Assume that E(Xk) is finite where k ~ 2 is a given integer. (33) E[(X-z)k]x~z] = E(Xk)
If
for all z ~ 0 ,
then F(x) = l-e -bx, x _> 0, and b > 0. Proof:
Let m = E(xk).
E[(X-z)kIx~z]
Since
= +f~ (y-z) k dF(y)/[1-F(z)] , Z
an argument similar to the one applied in Lemma 1.2.1 and (32) yields
(34)
f (y-z) k-i G (y)dy = [ m G(z) , z _ > 0 , Z
where G(u) = l-F(u).
If we denote the left hand side of (34) by H(z), then repeated
differentiation in (34) gives
(35)
(-1)kin H(k) k!
(z) = H(z)
.
I t i s w e l l known from t h e e l e m e n t a r y t h e o r y o f l i n e a r
differential
equations that the
general solution of (35) is (36)
H(z) =
k b.z [ c.e 3 j=l J
where the cj are arbitrary constants and bj, 1 s j ~ k, are the k (complex) solutions of the equation
34
(-l)km t k : 1 . k! Let its negative solution be denoted by b I = -b, b > 0 (evidently, real root of (k!)/m). H(z) -> 0,
b is the positive
Because, by definition and by (34), H(z) = (m/k)G(z),
decreases and H(z) ÷ 0 as z ÷ +~.
We shall show that these properties
can be satisfied only if (37)
k bz ~ c. e J j=2 j
identically.
= 0
First let k = 3.
Reb 2 = Reb 3 = r > 0 and r < b.
Considering the conjugate complex roots, we have Then lexp(bjz) l ÷ +~ as z + +~ for j = 2,3.
H(z) ~ 0 and H(z) ~ 0 can be satisfied only if c 2 = c 3 = 0. Reb 3 = 0 and b 4 = b.
Hence,
If k = 4, then Reb 2 =
In a similar manner as above, we easily verify that c 4 = 0.
On the other hand, since b2z
b3z
c2e
+ c3e
has to be real, H(z) cannot be decreasing for large z unless c 2 = c 3. above expression is identically zero.
But then the
Now the general case can be handled similarly.
If one combines two roots bj and bj+l, which are complex conjugates of each other, step by step one indeed arrives at the validity of (37). H(z) = cle
-bz
Hence
, b > 0 ,
from which F(z)
kCl = 1-G(z) = 1 - - - e
But s i n c e m = E ( x k ) , F(0+) = 0.
property
Theorem 2.3.3. (X-u) + = max(X-u,0).
, b > 0,
z ~ 0 .
(34) w i t h z = 0 and Len~rna 1 . 2 . 1
Consequently,
Let us r e t u r n interesting
-bz
m
imply that
G(0+) = 1 o r
(kCl)/m~l , which completes the proof.
to the linear
case.
of the exponential
D
K-M. Chong (1977) points out the following distribution.
Let X ~ 0 be a random variable with finite expectation.
Let
If the distribution of X is continuous at zero, then X has a
negative exponential distribution if, and only if, (38)
E[(X-s)+]E[(X-t) +] = mE[(X-s-t) +] , all s,t ~> 0 ,
with some constant m > 0. Proof:
Because X ~ 0, setting t = 0 in (38) yields m = ECX).
m-iE[(X-u)+],
then the assumption leads to the equation
g(s)g(t) = g(s+t)
, all s,t -> 0 .
Since g(u) is evidently monotonic, Theorem 1.3.1 implies
If we put g(u) =
35
E[(X-u) +] = me-bu , u e 0 f o r some c o n s t a n t b > O.
The l a s t
e q u a t i o n becomes
+m
f G ( t ) d t = me -bu , u e O, U
where G(t) = P ( X e t ) .
The r e a d e r can now e a s i l y c o n c l u d e t h e p r o o f by d i f f e r e n t i a -
t i o n and by u t i l i z i n g
the assumption that the distribution
o f X does n o t have a jump
at zero. From the proof, it is clear that the theorem can be modified in the same manner as the lack of memory characterizations were modified Chapter.
in the beginning of this
For example, if "all s,t ~ 0" is changed to "all s ~ 0 and two values of
t, t I and t2, say, for which tl/t 2 is irrational," the conclusion remains to hold. Other variants may be deduced from previous discussions.
The assumption of contin-
uity at zero cannot be dropped, however, because a jump of the distribution of X at zero has no effect on the other conditions. T.A. Azlarov, et al. (1972) obtained stability theorems related to most of the characterizations discussed in the present chapter.
Several of their theorems,
however, were previously obtained by T.A. Azlarov (1972) in a somewhat weaker form. The recent survey by E. Lukacs (1977) gives a good account of the pre-1976 results on stability.
CHAPTER 3 CHARACTERIZATIONS BY PROPERTIES OF ORDER STATISTICS 3.1.
Introduction
Let XI,X2,...,X n be independent and identically distributed random variables with common distribution function F(x).
Let us rearrange the X's in increasing order
XI: n ~ X2: n ~...~ Xn: n We call Xr: n the r-th order statistic.
If two values Xj and X t are equal for j # t,
then we do not stipulate which variable would precede the other one.
If F(x) is
continuous then P(Xj=Xt) = 0 for any j # t; therefore in this case the order statistics are distinct with probability one.
Since our aim is to obtain characterization
theorems for the exponential distribution and for its strictly monotonic transformations, we shall assume in most cases that F(x) is continuous.
Therefore, the present
chapter does not cover the literature of discrete distributions.
The reader can
find a comprehensive bibliography and a detailed exposition (with proofs and with several new results) for discrete distributions in J. Galmnbos (1975b). Let us collect a few basic formulas from the theory of order statistics which will be needed in the present chapter. By the basic assumption of the X.'s being independent and identically distri] buted, (I)
P(Xn: n < x) = Fn(x)
and (2)
P(XI: n ~ x) = [l-F(x)] n .
In general, Hence,
the event
applying
{Xr:n<X}
the binomial
means
distribution n
(3)
that at least
r of the events
with parameters
{Xj<x}
occur.
n and F(x),
n
Fr:n(X) = P(Xr:n<X) = j~r(j)F j = (x)[l-F(x)] n-] = r(n)~ f(X)tr-l(l-t)n-rdt0
B
Formulas (i) and (2) are, of course, special cases of (3). If the density f(x) of Xj exists, then (3) implies that so does the density fr:n(X) of Xr: n for each r and (4)
fr:n(X) = r(~)Fr-l(x)[l-F(x)]n-rf(x)
.
Let us give a more direct proof of formula (4), which has the advantage of being applicable to the case of the joint density of several order statistics.
Let
37 Ax > 0 and let F(x) be absolutely continuous.
Then the probability that at least
two of the X. fall into the interval (x,x+Ax) is 0[(Ax) 2] as Ax ÷ 0. Hence J < x+Ax) can be calculated as follows. The event {x~X < x+Ax} aDproxir:n r:n mately means that r of the Xj are smaller than x+Ax, one of which is between x and P(x~X
x+&x and the remaining r-I are smaller than x.
The rest are larger than x+Ax.
Therefore P(X~Xr: n < x+ Ax) ~ r(7)Fr-l(x)[l-F(x)]n-r[F(x+Ax)-F(x)] the error being 0[(Ax)2].
If we divide by Ax and let Ax ÷ 0 we get an exact formu-
la for fr:n(X), which is indeed the same as (4). This same method yields the following result. tinuous.
Let l~rl
be integers.
Let F(x) be absolutely con-
Then the joint density of the vector (Xrl:n,Xr2:n,...,Xrk:n) exists and
equals
(5)
k k [F(Xj+l) -F(xj) ]rj+l-rj-I fk(x,r) = n![ ~ f(x4)] ~ ~ j=l J j=0 (rj+l-rj-l)!
'
whenever Xl~X2~...~Xk and zero otherwise, where x = (Xl,X2,...,Xk) and % = (rl,r2,...,rk), x 0 = -~, Xk+ 1 = ÷~, r 0 = 0 and rk+ 1 = n. For the exponential distribution F(x) = l-e -bx, x ~ 0, a direct substitution into (5) yields the following result, which was first observed by P.V. Sukhatme (1937). Theorem 3.].]. (6)
Let F(x) = l-e -bx, x e 0.
dr: n = Xr+l: n - Xr: n , r _> 0, X0: n
=
Then the differences
0 ,
are independent exponential variables with P(d . :r n<X) = l_e-b(n-r)x
,X~0.
Another substitution into (5), namely that of F(x) = x, 0 ~ x~ i, leads to a result for the uniform distribution. Theorem 3.1.2.
Let F(x) = x for 0 ~ x ~ i.
Then the diffetences d
r:n
of (6)
are identically distributed with common distribution function Fl(X ) = l-(l-x) n. Notice that the differences dr: n in Theorem 3.1.2 are not independent.
In
fact, we shall show that the independence of the sequence dr: n is a characteristic property of the exponential distribution.
However, formulas (17) and (18) of
Section 1.6 can be applied to transform this property into one for arbitrary continuous distributions.
For example, we i~ediately get from Theorem 3.1.1, that the
ratios Xr+l:n/Xr: n are independent for the uniform distribution F(x) = x, 0 ~ x ~ i. We record yet one more property of order statistics. Let F(x) be continuous. order statistics
Then, as remarked in Section 1.6, the transformed
38
Wk: n : - l o g [ l - F ( X k : n ) ] are order s t a t i s t i c s
from a u n i t exponential d i s t r i b u t i o n .
Hence, by Theorem 3.1.1,
Wk: n can be represented as a sum of independent random variables.
From this repre-
sentation it is i~ediate that Wk:n, k ~ I, and thus Xk: n, k ~ i, as well, form a Markov chain.
The exp~licit form of the representation of Wk: n as the sum mentioned
above also implies the following important result. Theorem 3.].3.
Let F(x) be continuous.
P(Xk:n<XlXr:n=y), k • r, is the
Then the conditional distribution
same as the distribution of the (k-r)-th order
statistic in a sample of size n-r from a distribution F(x)-F(y) F*
(x)
:
l- r
if x ~ y
(~)
0
otherwise .
The three theorems proved here will be basic to the characterization theorems of the succeeding sections.
We have seen in Chapter I, that certain characterization
theorems based on XI: n are strongly related to the lack of memory property and to several of its equivalent forms.
This fact links closely the content of the present
chapter with the preceding ones.
3.2.
Characterizations by distributional properties One of the basic characterizations of the exponential distribution in Section
1.5 was property (P4*) which states that among the nondegenerate distributions only the exponential has the property that nXl:n, for all n ~ l, is distributed as the population.
We have extended this characterization in Theorem 2.1.2 by relaxing the
condition "for all n ~ I" to "for two values n I and n 2 of n such that (log nl)/(log n2) is irrational."
This theorem can further be extended as follows.
(This result is also due to J. Sethuraman (1965).) Theorem 3.2.].
Let F(x) be a nondegenerate distribution function.
Ass~e
that there are two sequences A n and Bn • 0 of numbers such that (XI:n-An)/Bn is distributed as the population for two values n I and n 2 of n such that Bn2 # 1 and (log Bnl)/log Bn2 is irrational.
Then there are constants, a, b • 0 and c • 0 such
that F(a+bx) is either
~l-exp [- (-x)-c]
Ll'c(X) =
if x < 0 ifx_>0
,
or i-exp(-x c) L2'c(X) = <0
if x > 0 if x _ < 0
.
On the other hand, if Bn2 = 1 and A 1/A 2 is irrational then F(a+bx) = l-exp(-ex)
39 with some constants a and b > 0. We do not present here the proof of this theorem because it is practically the same as that of Theorem 2.1.2 after establishing that if B
Anl/(1-Bnl)
=
An2/ (l-Bn2).
> i then so is B n2
and
n1
If we now specify that A n = 0 and Bn = i/n for n = n I and n2, then to obtain Theorem 2.1.2 from Theorem 3.2.1 it is only necessary to check which of the three distributions can satisfy the assumptions.
We thus get that, for A n = 0, Bn = I/n,
the resulting distribution is F(x) = L2,l(X), the exponential distribution. The three distributions appearing in the conclusion of Theorem 3.2.1 are those which are the possible limiting distributions for normalized minima for independent and identically distributed random variables (see Chapter 2 in the book by J. Galambos (1978)).
Theorem 2.1.2 and the characterization based on the property
(P4*) are therefore integral parts of the asymptotic theory of extremes.
Since we
would like to avoid any overlap with any existing book, we refer the reader for those characterization theorems which follow from the asymptotic theory of extremes to the recent book by J. Galambos (1978). Although the following theorem is related to a limit theorem, its proof does not require the general asymptotic theory of extremes.
Both the result and the
actual method of proof are due to B. Arnold (1971) and R.C. Gupta (1973). Theorem 3.2.2.
Assume that for an n ~ 2, nXl: n has the same distribution
F(x) as the population.
If F(x) is such that, as x ÷ 0+, lim F(x)/x = b > 0 finite,
then F(x) = l-e -bx, x ~ 0. Proof:
Let n(k) = n k, k ~ I, where n ~ 2 is fixed with the property of the theorem.
If, in a set XI,X2,...,Xnfk], k ~ 2, of observations, we form blocks ~
J
(Xl,X 2 . . . . . Xn(k_l)), (Xn(k-1)+l,Xn(k-1)+2 . . . . . X2n(k_l)) . . . . . (X(n_l)n(k_l)+ 1 , Xfn_l]n(k_l%+ 2 . . . . ,Xnrk?), and i f the minima of these blocks are denoted by k
X~(k-1) (7)
, x(2)
l:n(k-1)
Xl:n(k)
=
,
/
(n)
.... Xl:n(k_l),
m rx(i) lnl l : n ( k - 1 ) '
thenevidentay
X (2) l:n(k-1)"'"
, X (n) l l:n(k-1) j "
Since the o r i g i n a l Xj are independent and i d e n t i c a l l y d i s t r i b u t e d , so are the (J) x(J) assumpl:n(k-l)" Hence, f o r k=2, each Xi~n(k_l? is d i s t r i b u t e d as Xl: n which by tion has distribution F(nx).
Therefore, by the basic formula (2)
[1-F(x) ] n = 1-F (nx) and thus by (7) P(Xl:n(2) e x) = [1-F(nx)] n = 1-F(n2x)
I
Formulas (2) and (7) and i n d u c t i o n over k now y i e l d P(Xl:nrk'vJ e x) = [1-F(x)] n(k) = 1-F[n(k)x]
.
40
Hence, on one hand P(Xl:n(k) < n - ~ )
= I- [l-F(n~)in(k)
and on the other
P ( x l :n (k)
n - ~ ) : F (x) ,
<
w h e r e n -> 2, a n d n ( k )
= n k w i t h k >- 1.
We t h u s g e t
k (8)
F(x)
= 1-[1-F(@]n n
However, by assumption, elementary
for all
k -> 1 .
a s k ÷ +% F ( x n - k )
= axn -k + o(n-k).
Consequently,
the
relation
l i m (1 + y + o ( % ) s = e y S=+Oo
a n d (8) i m p l y t h a t ,
f o r a n y x > 0, k
F(x)
= 1 - lira(1 - ~ k=+ ~ n
+ o(@) n
n
= 1-e -bx , x > 0 .
The proof is con-~leted.
U
The method of proof of Theorem 3.2.2 is a very useful one.
It is called the
method of Zimit Zaws in Galambos (1975a), because in this method we generated a sequence n(k) from a single n, for which sequence a function had the same property for each k.
Therefore, one can let k tend to infinity and this limit determines the
fixed value expressed in the property valid for all k (in our case, formula (8)). We shall use this same method in the sequel in other proofs as well. It is clear from the proof that (8) can be satisfied for functions other than the exponential if we do not assume that F(x)/x ÷ a > 0 as x + 0 ÷.
Namely, if we
write F(x) = l-exp[g(x)], where g(x) < 0 is nonincreasing then (8) requires that (9)
g(x) = nkg(-~) n
for all x and all k -> 1 ,
where n -> 2 is a fixed integer.
J.S. Huang (1974) observes that such functions
g(x) can easily be constructed even under the additional assumption that g(x) is differentiable. ample.
Out of the general class given by Huang we give here only one ex-
Let g(x) = 0 for all x -< 0.
If x > 0,
negative) j such that 2j < x ~ 2j+l. g(x) = - 2 J I l ° g x - ~Io 21°g 2
define the integer (positive or
With this value of j we now choose
+ i)
Here (9) evidently holds for n = 2; however, F(x) is clearly not exponential. Let us return to (P4*) once again.
We can view this property as a converse of
one part of Theorem 3.1.1, i.e., the condition that the distribution of d0: n is the
41
same as that of the population is a characteristic property of the exponential distribution.
There are evidently several other ways of formulating converse state-
ments to parts of Theorem 3.1.1 and we can ask whether these are characteristic properties of the exponential distribution. (i)
is it a characteristic property of the exponential distribution that d
(ii)
Let us list some such possibilities:
r:n
is exponential?
is it a characteristic property of the exponential distribution that the distribution of (n-r)dr: n does not depend on r?
~iii)
is it a characteristic property of the exponential distribution that d
r:n
and d
s:n
are independent for r ~ s?
We intentionally posed our questions vaguely without specifying requirements on r and n ~whether just one or all values), which formulation allows us to refer to these questions after different specifications.
We shall discuss questions (i) and
(ii) here in this section and we devote a new section to question (iii). In regard to question (i), we should emphasize that the case of r = 0 has been mentioned and the answer is almost trivial.
Indeed, d0: n = XI: n, and we have re-
marked that, in view of (2), any preassigned distribution for XI: n uniquely determines the population distribution. Now let r = 1 and n
= 2.
Therefore, r ~ 1 only is of interest.
Then dl: 2 = X2:2 - XI: 2 = IXI-X21. If the common
distribution of X 1 and X 2 is exponential, then Theorem 3.1.1 implies that so is dl: 2.
This fact, of course, can easily be computed by means of the convolution
formula which yields that the density of XI-X 2 equals ½e-lXlfromwhich we get the exponentiality of dl: 2 = IXI-X21.
But if one starts with the distribution
F(x) = l-e-X[l + 4b-2(l-cos bx)] , x ~ 0, b ~ 2 4
,
as the common distribution of X 1 and X2, then the convolution formula leads to the same density ½e -Ixl for XI-X 2 and thus dl: 2 again is exponential. further assumptions, the answer to question (i) is negative.
Therefore, without
The above example is
due to H.-J. Rossberg ~1972a), who also recognized the reason why distributions other than the exponential may have the property that dr: n is exponential.
Namely, he re-
alized that for all distributions F(x) with F(0+) = 0 for which dr: n is exponential, the Laplace transform (i0)
Ur(S ) = f e-SxdFr(x) 0
has zeros for some s such that Res ~ 0.
Therefore, he posed the question whether
under the assumption of Ur(S) ~ 0 for Res ~ 0, the exponentiality of dr: n implies that the population distribution is exponential.
He proved the answer to this quest-
ion in the affirmative by utilizing the following result.
42
Lemma 3.2.1.
Let R(t) be a r e a l v a l u e d f u n c t i o n o f bounded v a r i a t i o n
normegative real line.
Let G(x) be a d i s t r i b u t i o n
G(x) > 0 f o r a l l x > 0.
on t h e
f u n c t i o n such t h a t G(0+) = 0 and
Assume t h a t
+co
(11)
f G(t-x)dR(t) = 0 , 0
x >- 0 ,
and t h a t t h e L a p l a c e t r a n s f o r m +co
G*(s) = f e-SXdG(x) -co
does
n o t v a n i s h f o r Res ~> 0. O u t l i n e of p r o o f :
Rl(t)
= R(t)
identically
Then R(t)
i s c o n s t a n t f o r t > 0.
Let R(+~) d e n o t e t h e l i m i t o f R ( t ) a t i n f i n i t y .
- R(+~) t h e n (11) means t h a t t h e c o n v o l u t i o n H(x) o f R l ( t ) z e r o f o r x _> 0.
and G ( - t ) i s
T h e r e f o r e , t h e L a p l a c e t r a n s f o r m He(s) o f H(x) i s con-
t i n u o u s and bounded f o r Res _< 0.
On t h e o t h e r hand, i f R~(s) d e n o t e s t h e L a p l a c e
transform of Rl(t), then H~(s) = -G~(-s)R~(s) have singularities
I f we p u t
for Res -< 0.
for Res = 0 and H~(s)/G*(-s)
does not
From here one can easily conclude that R~(s) can be
extended to Res < 0 by the formula {-Ile(s)/G*(-s)}, which is bounded now on the whole complex plane.
The basic theorem of Liouville on entire functions implies
that R~(s) is constant.
Hence, the uniqueness theorem of Laplace transforms implies
that Rl(t ) also is constant.
This completes the proof.
We can now prove Rossberg's result. Theorem 3.2.3.
Let the population distribution F(x) be such that F(0+) = 0
but F(x) > 0 for all x > 0. not vanish for Res _> 0. (121
Assume that the Laplace transform Ur(S ) of (i0) does
If, for some r _> i,
P(dr: n -> x) = e-(n-r)x ,
x _> 0
then F(x) : 1-e -x , x -> 0. Proof:
By the method utilized in Theorem 3.1.3, we can prove +oo
P(dr:n_>X) = _(n) f F r ( u _ x ) d [ l _ F ( u ) ] n - r •
.
0
Hence, by (12), f o r a l l x >_ 0, +co
_(n) ~ F r ( u _ x ) d [ l _ F ( u ) ] n - r = e - ( n - r ) x
which can a l s o be w r i t t e n as f Fr(u-x)dR(u) = 0 , 0
x -> 0 ,
where R(u) = [1-F(u+y)] n - r - e - ( n - r ) Y [ 1 - F ( u ) ] n - r
, u > 0 ,
43
with some fixed y > 0.
Hence, Ler~na 3.2.1 yields that R(u) is constant for u > 0.
This constant value is evidently zero since lim R(u) = 0 as u ÷ +~. I ~ = l-F(u)
e-y
That is,
for all y > 0 and u > 0.
Letting u ÷ 0 we get F(y) = 1-e -y , y > 0, which was to be proved.
Let us turn to question (ii). There are only a few solutions to this problem under rather restrictive assumptions.
Some of these solutions, however, are very
valuable in engineering applications.
Namely, several failure models can be approx-
imated by a model in which the components are independent (but not identically distributed).
For such models, however, the asymptotic extreme value distributions are
of monotonic hazard rate (see J. Galambos (1978), Chapter 3, particularly Sections 3.9 through 3.12).
Therefore, if one can characterize a distribution in the class
of all distributions with monotonic hazard rate, then the general theory just mentioned combined with such a characterization theorem gives as good a practical solution as a mathematically more general characterization which assumes much less on the population distribution.
Of course, for practical problems not related to
the asFmptotic theory of extremes, other kinds of assumptions are needed on the population.
Hence, a solution to question (ii) would be of wider interest than the
following result due to M. Ahsanullah (1976 and 1977a), Iheorem 3.2.4. f(x) = F'(x).
Let F(×) be absolutely continuous with density function
Assume that F(0) = 0, F(x) is strictly increasing for all x > 0 and
the hazard rate H(x) =
1-F(x)
i s monotonic f o r a l l x ~ 0. identically
distributed
I f f o r some n ~ 2, nd0: n = nXl: n
and ( n - 1 ) d l : n a r e
t h e n F(x) = 1-e - b x , x ~ 0, w i t h some b > 0.
N o t i c e t h a t we a r e assuming o n l y t h a t two d ' s a r e i d e n t i c a l l y after normalization.
It is not essential
t h a t we choose nd0: n and ( n - 1 ) d l : n .
t h e p r o o f below no m o d i f i c a t i o n i s r e q u i r e d i f we assume t h a t (n-i*l)di_l: n are identically cally distributed. proof algebraically
distributed
distributed In
( n - i ) d i : n and
or t h a t nd0: n and ( n - i ) d i : n a r e i d e n t i -
These changes i n t h e a s s u m p t i o n would make t h e f o r m u l a s i n t h e b u t n o t c o n c e p t u a l l y more c o m p l i c a t e d , which i s t h e s o l e r e a s o n
f o r our c h o i c e o f a s s u n p t i o n s . Proof: (13)
On a c c o u n t o f ( 2 ) , P(nd0:n<X) = 1 - [ I - ~ ( ~ ) ] n .
When e v a l u a t i n g t h e d i s t r i b u t i o n
o f ( n - 1 ) d l : n, we c o n d i t i o n i t on Xl: n = y and we
use the continuous v e r s i o n of the t o t a l p r o b a b i l i t y
rule.
We g e t
44
P[(n-1)dl:n<X]
= f P[(n-1)dl:n<XlXl:n=Y]fl:n(Y)dY 0
•
Now we substitute the expression (4) for fl:n(y ) and we apply Theorem 3.1.3. We obtain +co
(14)
P[(n-1)dl:n<X]
= f 0
iI- I
= 1 - n_
l-F(y + n-~) n-1 ]-~-F(~)i I n[I-F(y) ]n-lf(y)dy
r_+~ ]l-F(y +l_F(y)n--~) ]n-i r ~ d v rl -~F] y . n - l f . y . ~
o
0
If we write [I-F(~)] n = n f [l-F(~)]n[l-F(y)]n-lf(y)dy , 0 (13) and (14) and the assumption that the distributions of d0: n and dl: n are the same yield (15) f K(x,y,n)[l-F(y)]n-lf(y)dy = 0 for all x > 0 , 0 where l-F(y + n_-~) n-I
.
(16) K(x,y,n)= [I-F(~)]n - ]
TF_F~y~
]
Since the hazard rate H(x) = - d log[l-F(x)] is monotonic, log[l-F(x)] is either concave or convex according as H(x) is increasing or decreasing. For definiteness let H(x) be increasing. Then the concavity of log[l-F(x)] implies + )] ~ ~ log[1-F(y)]
log[l-F(y
+ n-] log[1-F(y + n
)] -
which inequality is equivalent to x n [ 1 - F ( y + n) ] -~ [ 1 - F ( y ) ] [ 1 - F ( y
x . in-1 + n_--y) j
.
Therefore X
(17) K(x,y,n)> [l-F(n x-)]n -
where the last
i l-F(y + n)]n -
inequality
1-r(y)
again follows
_~
0
,
from the fact
that
log[1-F(x)]
Now (17) implies that (15) can hold only if K(x,y,n) = 0 for all x,y _> 0 . That is, (18)
l-F(y + ~ x)
= [I-F(y)][I-F(x) ]n/(n-l) for all x,y _> 0.
is concave.
45
This is exactly the lack of memory property, because if we write z = x/(n-l), (18) says that
z~_~+z) I-F(~ does not depend on y. l-F(z).
But in that case y = 0 gives that this fraction is necessarily
Since the lack of memory equation, when it is assumed to hold for all y ~ 0
and z ~ 0 has only two solutions (Theorem 1.3.1), namely, exponential or degenerate, our claim follows from the assumption that F(x) is continuous. The case when H(x) is decreasing is similar to the above argument; we thus do not repeat it.
This completes the proof.
Our proof is somewhat different from the original proof given by Ahsanullah (1977a). With this new proof, we aimed to show that the assumptions of the theorem lead back to the basic properties described in Chapter i. ing fact.
This is quite a surpris-
We believe that the conclusion of the theorem can be shown in the above
elementary way even without assuming the monotonicity of H(x). variants of the theorem, but we always needed some assumptions.
We can prove several We therefore left
the formulation of the theorem in Ahsanullah's original form because of its practical value as explained earlier. If one repeats the proof of Theorem 3.2.4 under the modified assumption that nd0: n and (n-i)di: n are identically distributed then one immediately realizes that the exponential distribution has the following characteristic property: Xl:n_ r are identically distributed.
dr: n and
Since we conclude this fact from the proof of
Theorem 3.2.4 (in its modified form), the additional assumptions (absolute continuity and monotonic hazard rate) are, of course, used in this characterization. restrictions, however, are not necessary.
These
H.-J. Rossberg (1972a) obtains the above
characterization with the slight assumptions that F(x) is not degenerate and is not a lattice distribution.
M. Ahsanullah (1975) extends this type of characterization
to a comparison of the distributions of Xs: n - Xr: n and Xs_r:n_ r for two values of s, each of which is strictly larger than r. line of the proof of Theorem 3.2.3.
Rossberg's method is analytic on the
The basic idea of Ahsanullah is similar to his
proof of Theorem 3.2.4,where the lack of monotonicity of H(x) can be overcome by utilizing two values of s to derive the desired conclusion. If we restate Rossberg's (1972a) theorem for n = 2, we get a result in which dr: n is compared with the population distribution F(x).
In fact, if n = 2, the
statement that dl: 2 has the same distribution as XI: 1 simply means that with two independent observations X 1 and X 2, dl: 2 = IXI-X21 has the same distribution as the c o ~ o n distribution of X 1 and X 2. exponential distribution and H. Rubin (1970).
The fact that this property characterizes the
(among continuous distributions) was obtained by P.S. Puri
Within a more restrictive family (which is, however, useful
in engineering statistics), M. Ahsanullah (1977b) obtains that it is a characteristic
46
property of the exponential distribution that (n-r)dr: n has the same distribution as the population.
See also M. Ahsanullah and M. Rahman (1972) and J.S. Huang (1974a).
For a characterization of the uniform distribution by distributional properties of dr: n of the type that it does not depend on r and F(x) is either super-additive or subadditive, see J,S. Huang, B.C. Arnold and M. Ghosh (1977).
3.3.
Independence of functions of order s t a t i s t i c s . We present in this section a number of characterization theorems based on the
independence of certain functions of order staitstics.
In addition to the actual
results, we also wish to draw attention to the methods of proof.
The method based
on Theorem 3.1.3 reduces characterizations by independence of functions of order statistics to some forms of the lack of memory property (Section 2.1).
Another meth-
od utilizing characteristic functions has the potential of extending some results of the present section to stability theorems.
We shall return to this possibility at
the end of this section. We start with the following result which was first proved by M. Fisz (1958) under somewhat different assumptions. Theorem 3.3.1.
Let X 1 and X 2 be independent random variables with cormnon con-,
tinuous distribution function F(x). increasing for all x > 0.
Assume that F(0) = 0 and that F(x) is strictly
Then X2:2 - XI: 2 and XI: 2 are independent if, and only if,
F(x) = l-e -bx with some b > 0. Pro____o~: The special case n = 2 of Theorem 3.1.1 implies that X2:2 ~b~l:2 and Xl: 2 are indeed independent for the exponential distribution F(x) = l-e
, x e 0, b > 0.
Hence, only the converse statement needs proof. If XI: 2 and X2:2 - Xl: 2 are independent then
(19) P(X2:2 - Xl: 2 < XlXl: 2 = z) = P(X2:2 - Xl: 2 < x) for almost all z > 0.
Here, "almost all" can refer to Lebesgue measure because of
the assur~tions on FCx) that it is continuous and strictly increasing for all x > 0. On the other hand, by Theorem 3.1.3,
P(X2:2 - XI: 2 < XlXl: 2 = z) = P(X2:2 < x+ZlXl: 2 = z) = P(X[: 1 < x+z)
,
where X~:I is the indicated order statistic from a population with parent distribution
(x)-F(z) F*(x) = f FI-F(z) 0
if x >- z , otherwise .
47
That is, if we denote the right hand side of (19) by H(x), then
(20)
F(x+z) - F(z) 1-F(z)
= H(x)
f o r a l l x ~ 0 and almost a l l z > 0.
Letting z ÷ 0 implies that F(x) = H(x).
Thus,
if we write F(x+z) - F(z) 1-F(z)
1-F(x+z) = 1 -
(2) becomes 1 - F(x+z) = [1-r(x)] [1-F(z)]
for all x ~ 0 and almost all z > 0. property discussed in Section 2.1.
This is an extended form of the lack of memory In particular, we have from Theorem 2.1.1 that
F(x) is exponential as claimed.
D
Although the assumptions of Theorem 3.3.1 are sufficient but not necessary for its conclusion, the following len~nas show that more general cases can easily be reduced to these assumptions.
Lemma 3.3.1.
Let X 1 and X 2 be independent random variables with cormnon non-
degenerate distribution function F(x).
If XI: 2 and X2:2 - XI: 2 are independent then
F(x) < 1 for all x. Proof:
Assume that there is a finite number A such that F(A) = i.
Then X~ = XI-A
and X~ = X2-A are nonpositive with probability one and X~: 2 and X}: 2 - X~1:2 are evidently independent, where a star in the superscript of the order statistics indicates that we are dealing with the order statistics of X~ and X~.
But for non-
positive values
X~: 2 = X~: 2 + X~: 2 - X~: 2 -< X~: 2 - X~: 2 with probability one, or equivalently, with probability one,
2
->
2
-
2
,
which contradicts the independence of X~: 2 and X m2:2 - X~:2 and the assumption that F(x) i s n o n d e g e n e r a t e . Ttiis completes t h e p r o o f . D Lemma 3.3.2.
Under the assumptions of Lemma 3.3.1, there is a finite number B
such that F(B) = 0. Proof:
Assume that the opposite is true.
Then for any A < 0,
That is, for any finite number A, F(A) > 0
48
(21)
P(XI: 2 < A, X2:2 - XI: 2 k ]A]) ~ 2P(XI
On t h e o t h e r hand, by i n d e p e n d e n c e ,
(22)
P(XI: 2 < A, X2:2 - Xl: 2 ~ IAI) = P(XI: 2 < A)P(X2:2 - XI: 2 = [1-
(1-F(A))2]p(X2:2
: F(A)[2 - F ( A ) ] P ( X 2 : 2
IAI)
Xl: 2 e I A I ) : - Xl: 2 e IAI) .
We t h u s g e t from (21) and (22) t h a t
[i - ½F(A)]P(X2:2
- XI: 2 ~ IAI) ~ P(X 2 ~ 0) .
Letting A ÷ -~ we obtain P(X 2 ~ 0) = 0, which contradicts Lemma 3.3.1. established.
D
Lemma 3 . 3 . 3 . tric variables,
The len~aa is
Let X 1 and X 2 be independent and identically distributed geome-
i.e.
P(Xl=k) : P(X2=k) = pk(l-p)
, 0 < p < i, k = 0,I,2 . . . . .
Then XI: 2 and X2:2 - XI: 2 are independent.
Proof:
Since, for any integers n z 0 and t ~ 0, satisfying n+t > 0,
P(X2:2 - Xl: 2 = n,
Xl: 2 = t) = 2P(X 1 = t, X 2 = n+t)
= 2pt(l-p)pn+t(l-p)
= 2pn+2t(l-p) 2 ,
and moreover,
P(X2:2 - X I : 2 = 0,
XI: 2 = O) = P(X 1 = X 2 = 0) = (l-p) 2 ,
we ir~nediately get that XI: 2 and X2:2 - XI: 2 are independent.
lemma 3.3.4.
Let the conditions of Ler~na 3.3.1 be fulfilled;
F(x) has the following property: F(b). Proof:
assume also that
there are two numbers a < b such that 0 < F(a) =
Then F(x) is not continuous. We again assume that the opposite is true, i.e., F(x) is continuous but not
strictly increasing on a semiline. F(a) > 0 and F(b) = F(a). z < a.
D
Furthermore,
In other words,
there are a < b such that
We can, of course, choose a so that F(z) < F(a) for all
because F(.) is continuous, we can choose two values z I < z 2
49
such that 0 < F(Zl) < F(z 2) < F(a) and a - z I < ¢, where ~ > 0 is arbitrary.
We
shall impose one more restriction on the choice of z I and z 2 later. Now if we start with (19), we again get (20), but the condition that "almost all z > 0" refers here to the measure Q(D) on the Borel sets D of the real line for which Q(D) = F(d) - F(c) for intervals D = (c,d). a < z < b, where a < b
are as defined above.
Hence,(20)
may not hold for
But, if D = (a-e,a), then by the
choice of a, Q(D) = F(a) - F(a-E) > 0, and thus we can choose two points z I < z 2 in this interval D for which (20) holds with all x a 0.
Therefore,
if we substitute
first z = z 2 and x = a-z 2 and then z = z 2 and x = b-z 2 into (20) and subtract the two equations,
the left hand sides cancel out and thus (20) implies
0 = H(a-z2) - H(b-z2)
.
S i n c e H(x) d e n o t e s t h e d i s t r i b u t i o n
f u n c t i o n on t h e r i g h t hand s i d e o f ( 1 9 ) , t h i s
means t h a t
P(a-z 2-< X2:2 - XI: 2 < b-z2) = 0 . This last equation, however, contradicts the fact that P(Zl~Xl 0 and P(z2~X2
= F(a) - FCz2) > 0 and that 0 < a-z I < ~, where ~ > 0 is arbitrary.
Namely, since F(x) is continuous,
it has positive probability that X 1 is between z 1
and z 2 and X 2 is arbitrarily close to a.
For such values of X 1 and X2, however,
X2:2 - XI: 2 = X 2 - X 1 is larger than a-z 2 and is smaller than ~ > 0, which was arbitrary
(hence, smaller than b-z2).
have 0 < F(a) = F(b) for some a < b
We have thus obtained that it is impossible to for continuous distribution functions F(x)
satisfying the conditions of the Lemma.
Hence, the proof is complete.
[]
A careful analysis of the preceding proof shows that the following more general result is valid. Lemma 3.3.5. discrete.
If the
conditions of Lemma 3.3.4 are satisfied then F(x) is
Since we do not need any essential modification of the preceding proof, we omit details. We now summarize Len~as 3.3.1 - 3.3.5 and Theorem 3.3.1 as Theorem 3.3.2.
Let X 1 and X 2 be independent random variables with common dis-
tribution function F(x).
If XI: 2 and X2:2 - XI: 2 are independent,
either discrete or F(x) = 1 - exp[-b(x-B)],
then F(x) is
x ~ B, where b > 0 and B are finite
constants. We have seen in Lemma 3.3.3, that there are discrete distributions XI: 2 and X2:2 - XI: 2 are independent.
for which
One additional class of discrete distributions
is the degenerate one for which the above independence property still holds.
It is,
however, true that there are no other discrete distributions with this independence
50
property.
Since we do not analyze in detail the discrete distributions, we do not
prove this claim but we add that the support of the geometric distribution does not necessarily have to be the set of the nonnegative integers as it was specified in Lemma 3.3.3. Several extensions of Theorem 3.3.2 are known in the literature. fore formulating some of these extensions, we give some references.
However, be-
Although the
proofs presented here are new, different versions of Theorem 3.3.2, and thus of the len~nas as well, can be found in several publications.
As usual, we do not make
a distinction between two results if they differ only in a monotonic transformation (see for example formulas (17) and (18) in Section 1.6).
As mentioned earlier, the
first result along this line is due to M. Fisz (1958), who assumes absolute continuity and that F(x) is not positive o~ the whole real line.
HM.
Rossberg (1960)
relaxes the condition to continuity, for which case Z. Govindarajulu (1966) a new proof.
gives
Govindarajulu also investigates several other characterizations based
on the independence of some linear functions of order statistics.
One of them is an
extension of a result by E.A. Tanis (1964), who proved that, among absolutely continuous distributions, only the exponential one possesses the property that n
n
Sn = j~IXj - nXl:n = j~I(Xj.=:n - Xl:n)
and XI: n are independent. n
Notice that Sn can also be written as n
~ c.J = 0, c I ¢ 0 Sn = j ~ICjXj :n ' j=l where the coefficients cj are constants,(in fact, they are specified above).
Guided
by this observation, Rossberg (1972b) established the following very general extension of Theorem 3.3.1, proof of which will be discussed later, Theorem 3 . 3 . 3 . continuous distribution
L e t X l , X 2 , . . . , X n be i n d e p e n d e n t random v a r i a b l e s w i t h cormnon function F(x).
Let Ck, C k + l , . . . , c n be c o n s t a n t s s a t i s f y i n g
c k ~ 0, and c k + C k + l + . . . + c n = O. Then Xk: n and Sk, n = CkXk:n+...+ CnXn: n a r e i n d e p e n d e n t i f , and o n l y i f , F(x) = 1 - e x p [ - b ( x - B ) ] , x ~ B, where b > 0 and B a r e finite
constants.
Some variants of this theorem, under the assumption of absolute continuity, are also stated by A.P. Basu (1965). Theorem 3.3.2 follows from results by T.S. Ferguson (1967) and J.H.B. Kemperman (1971).
They actually determine all distributions F and G under which
min(Xl,X2) and IXI-X2] are independent, where X 1 and X 2 are independent random variables with distribution functions F and G, respectively.
Earlier, under the
stronger assumption of min(Xi,X2) and XI-X 2 being independent, Ferguson (1964) and (1965) and G.B. Crawford (1966) found that necessarily F and G are either both ex-
51
ponential or both geometric (see Chapter 6 for further details of the results by Ferguson, Crawford and Kemperman). J. Galambos (1972 and 1975c) proved two additional extensions of Fisz's theorem.
In the first one, the independence of XI: 2 and g(Xl:2, X2:2) is assumed,
where g(x,y) 2 0 is a Borel measurable function (see Theorem 3.3.4).
In the second,
the independence of XI: 2 and X2:2 - XI: 2, truncated at a given value~ is assumed, and the conclusion is again that the parent distribution is exponential.
This
latter characterization is of significant practical value for the following reason. In many practical situations when it is expensive to wait for random occurrences we can save by comparing XI: 2 and the time elapsed until the second "occurrence" X2:2 or until a given time, whichever occurs first, rather than measuring the actual value of X2: 2.
This is a continuous analog of R.C. Srivastava's (1974) result for
the geometric distribution.
Namely, Srivastava obtained that if the parent distri-
bution F(x) is discrete and if XI: 2 is independent of the event {X2:2-XI: 2 = 0} (which can be considered as a truncated version of X2:2 - XI:2), then F(x) is a geometric distribution function in the extended sense that the set of jumps of F(x) is not specified. Rossberg uses characteristic functions in his proof of Theorem 3.3.3.
The
method of characteristic functions is also the basic tool of Galambos (1972) and, for another class of problems, of Rossberg (1968).
We shall illustrate this method
by means of one of the quoted results of Galambos (1972).
The advantage of the
method of characteristic functions is that it is a promising avenue for extending a characterization theorem to a stability theorem.
Namely, if instead of independence,
we assume only almost independence in a well defined sense, then the deviation of the joint characteristic function from the corresponding product (which would have been obtained under independence) can be estimated.
On the other hand, there are stand-
ard methods in probability theory to estimate the difference of two distribution functions when an estimate is available for the difference of their characteristic functions.
Such a method is well exemplified by some proofs in the monograph by
E. Lukacs and R.G. !Lscha (1964) (see pp. 150-170).
It is an important and challenging
area of research to extend the results of the present section to stability theorems. Theorem 3.3.4.
Let g(x,y) ~ 0
be a Borel measurable function.
Let X 1 and
X 2 be independent random variables with common absolutely continuous distribution function F(x).
Assume that U = g(XI:2,X2: 2) has finite expectation.
are independent, then there is a constant c such that +co
(23) f g(z,x)f(x)dx
=
cp-F(x)]
Z
for all z, where f(x) = F' (x).
If XI: 2 and U
52
Proof:
We use characteristic functions.
For the basic properties of characteristic
functions that we apply here, see for example, E. Lukacs and R.G. Laha (1964, pp. 23-24).
Let the characteristic functions of the variables XI:2, U and the vector
(XI:2,U) be denoted by ~l(t), $2(u) and ~(t,u), respectively.
Then the independence
of XI: 2 and U is equivalent to the validity of the equation
(24)
¢(t,u)
= ~l(t)¢2(u)
for all real numbers t and u.
Now the joint density of (Xl:2,X2:2) is 2f(x)f(y)
for x < y, and zero otherwise. +oo
Hence
+oo
qb(t,u) = 2 f
f exp{itx + iug(x,y)}f(x)f(y)dxdy
-co
,
X
which, after the substitution x+v = y, becomes +co
~(t,u)
+00
= 2 f
f exp{itx + iug(x,x+v))f(x)f(x+v)dvdx
-m
.
0
A similar calculation yields +co
~l(t)
+oo
= 2 f
f
-co
0
and
eltXf(x)f(x+v)dvdx
+co
~2(u) = 2 f -co
f exp{iug(x,x+v)}f(x)f(x+v)dvdx
.
0
If we write these explicit expressions in (24), then differentiate with respect to u and set u = 0, we get +c~
f -~
+oo
.
+00
f eltXg(x,x+v)f(x)f(x+v)dvdx 0
= c f -~
+m
.
f eltXf(x)f(x+v)dvdx 0
.
Since g(x,x+v) ~ 0 by assumption, the inversion formula for characteristic functions implies that +~
f g(x,x+v) f(x)f(x+v)dv 0
+co
= c f f(x+v)dv . 0
This equation is equivalent to (23) (substitute z = x+v), and thus the proof is complete.
O
Notice that if g(x,y) = y-x for y ~ x reduces to the Fisz theorem. +00
f (x-z)f(x)dx = c[1F(z)]
and zero otherwise, then Theorem 3.3.4
Indeed, in this case (23) takes the form
53
or equivalently
f xf(x)dx
- z[l-F(z)]
= c[l-F(z)]
.
Z
Differentiating
with
respect
to z, we have
1 - F(z) = cf(z) , which evidently leads to the exponential distribution. Other applications are i~nediate.
The reader is invited to write down some
special cases by choosing other explicit forum for g(x,y).
In particular, it will
be interesting to work out special cases for which it is not straightforward that the resulting distribution is a simple transform of the exponential distribution. Some of the results of the next section, namely, those characterizations which are based on constant regression, are closely related to the present section. are
They
not necessarily generalizations, however, because the assumptions will differ,
mainly by imposing finiteness of expectations, which was not a standard assumption in this section.
3.4.
Characterizations
through moment assumptions.
Let XI,X 2 .... ,Xn be independent random variables with cor~non distribution function F(x).
We introduce the notation
(25) Er: n = E(Xr:n) , which, when used, is always assumed to be finite. (26)
E
+~ r:n = f XdFr:n(X) -~
= r(~)
f
iF_l(u)ur_ 1
In view of (3),
(l-u)n-rdu
,
0
whenever the right hand side is finite, where F-l(u) = inf(x : F(x) ~ u} . We thus have from (26) and the elementary relation
n (n-r) (n) (l-u) + (r+l) (r+l)U = n(nr I) that, for any integers 0 < r < n, n ~ 2,
(27)
(n-r)Er: n + rEr+l: n = nEr:n_ 1 .
Hence, if r = r(n) is an arbitrary function of n such that r(n) is an integer with 1 _< r(n) <_ n, then the sequence Er(n):n' n _> i, where El: 1 = E(XI), uniquely deter-
54
mines all values Er: n, 1 ~ r ~ n, n ~ i, on account of (27).
We thus obtained the
following basic result, which was first pointed out independently by J.B. Kadane (1974) and by Galambos (1975a). Lemma 3.4.].
If the integers r = r(n) satisfy 1 ~ r(n) ~ n
for all n ~ I,
then the following two statements are equivalent: (i)
property A holds for the sequence Er(n):n, n ~ I,
(ii) property A holds for all values Er:n, 1 ~ r ~ n, n ~ i, where El: 1 = E(X!). As a special case, we get from Len~na 3.4.1, that statements such as "El:n, n ~ i, uniquely determines the population distribution" or "the triangular array Er:n, 1 ~ r ~ n, n ~ I, of numbers uniquely determines the population distribution" are equivalent.
Therefore, the following result, due to W. Hoeffding (1953), is as
general as subsequent "generalizations" (L.K. Chan (1967), A.G. Konheim (1971), M. Pollack (1973), R.C. Gupta (1974) and M.M. Ali (1976)). Theorem 3.4.1.
The triangular array Er: n, 1 ~ r ~ n, n ~ I, of numbers uni-
quely determines the population distribution F(x). Proof: T(x).
Let YI,Y2,...,Yn be i.i.d, random variables with distribution function Assume that, for all 1 ~ r ~ n, n ~ I, E(Yr:n) = Er: n ,
where Er: n is as defined at (2S) for the basic sequence XI,X2,...,Xn (recall that the notation Er: n automatically assumes that this quantity is finite).
Utilizing
now (26), we get 1 1 (28a) f F-l(u)ur-l(l-r)n-rdu = f T-l(u)ur-l(l-u)n-rdu 0 0 for all 1 ~ r ~ n, n ~ 1.
Let r ~ 1 be fixed and consider the last equation for al~
n ~ r. Substituting y = l-u and putting fr(y ) = F-l(l-y)(l-y) r-I and tr(Y ) = T-l(l-y)(l-y) r-I results in 1
(28)
1
~ fr(y)ykdy = f tr(y)ykdy 0
,
k = n - r -~ 0 .
Since both fr(y) and tr(Y ) are nonnegative and their integrals are equal (set k = 0 in the above equation) there exists a constant c > 0 such that both Cfr(y) and Ctr(Y) are densities over the finite interval (0,i).
Hence (28) says that all mom-
ents of two absolutely continuous distributions are equal, which distributions are supported by the finite interval (0,I). The classical moment problem (see, e.g., J. Galambos (1978), Appendix II) thus implies that fr(y) = tr(Y ) which in turn
55
yields F(x) = T(x) as claimed. This theorem admits many interesting corollaries. Corollary 3.4.1. Proof:
If El: n = I/n
For example
for all n ~ I, then F(x) = 1-e -x , x ~ 0.
If F(x) = 1-e -x , x ~ 0, then El: n = I/n
for all n ~
i.
Thus, by Len~na
3.4.1, E is the same for all 1 ~ r ~ n, n ~ I, as for a population distribution r:_nx F(x) = l-e . Theorem 3.4.1 now implies our statement. D Corollary 3.4.2. Proof:
If El: n = I/(n+l), n -> I, then F(x) = x for 0 -< x _< i.
The argument is similar to the preceding
one.
Since, with F(x) = x,
0 ~ x ~ i, El: n = i/(n+l), Lemma 3.4.1 and Theorem 3.4.1 imply that no other population distribution can have this property. It is evident from Corollaries 3.4.1 and 3.4.2 that the limits of nEl:n, as n ÷ +~,
cannot characterize the population distribution.
Galambos (1975a) gives
an heuristic argument to show that asymptotic values Ek: n ~ h(k,n), 1 ~ k ~ n, n ÷ +~, may characterize population distributions within some families.
It would
be interesting to make that argument rigorous. We state another consequence of Corollary 3.4.2. nonnegative random variables. Rj, n = Sj/Sn+ I.
Let YI,Y2,...,Yn+I be i.i.d.
Define Sj = YI+Y2+...+Yj and consider the sequence
We then have
Corollary 3.4.3.
If the random variables Rij,n' 1 ~ j ~ n, are distributed as
the order statistics Xj:n, 1 ~ j ~ n, of a sample from a population with parent distribution F(x), then F(x) = x, 0 ~ x ~ I. Proof:
Since Y~j ~ 0, the random variables R~, nJ
finite.
Furthermore, E(Yi/Sn+ I)~
are bounded.~iHence, E(R~,n)j is
does not depend on j and ~ E ( Y D / S n + I ) _
= I.
Con-
sequently, n+l 1 E(RI,n) = E(YI/Sn+I) = ~ j ! I E ( Y j / S n + I
1 ) = n+l "
Now, if Ri,n,~ 1 <_ j _< n, are the order statistics of a sample from a population with parent distribution F(x), then Corollary 3.4.2 is applicable, whose conclusion is exactly our claim.
D
The statistics Rj,n, 1 -< j -< n, of the sample YI,Y2,...,Yn+I are widely used in goodness of fit tests.
Namely, if the common distribution of the Y's is
F(x) = l-e -bx, then the joint distribution of the Rij,n is indeed the same as that of the order statistics of a sample from a uniform population on (0,i).
The ad-
vantage of the transformation R. is that it does not depend on the unknown paraj,n meter b of F(x). We shall return to this subject matter, and the corresponding
56
characterization theorems, in Chapter 6. The following interesting fact was observed by J.S. Huang (1974b) in connection with Corollary 3.4.1.
If we write El: n = i/n in the form of E(nXI: n) = E(X I)
(which we assumed to be one in Corollary 3.4.1), then the condition of Corollary 3.4.1 becomes strongly related to the property (P4*) discussed in Section 1.5.
The
seemingly additional assumption here, compared with (P4~), is that E(XI) is assumed to be finite.
Now, Huang observed that if (P4*) holds then E(XI) is finite and thus
Corollary 3.4.1 is a much stronger statement than (P4*). Let us return to Theorem 3.4.1.
The question arises of how large should a
subset of the numbers Er:n, 1 ~< r <_ n, n -> i, be in order to characterize the population distribution?
In view of L e n a 3.4.1, a reduction in the set of values of
r alone does not lead to stronger results, hence a meaningful extension of Theorem 3.4.1 can only be obtained by considering subsets of all the consecutive integers n -> i.
Such an extension can indeed be achieved if we reexamine the essential step
in the proof of Theorem 3.4.1.
Namely, we obtained at (28) that Theorem 3.4.1 is
essentially equivalent to the classical moment problem of bounded random variables. This, in turn, can be considered from the point of view of orthogonality or complete hess of a set of functions by writing (28a) as (29)
f 1 [F-l(u) _ T- 1 (u)]ur-l(l-u)n-rdu = 0 0
for all I ~ r ~ n, n ~ I.
Since it is known in functional analysis, that only the
identically zero function can be orthogonal to the set of polynomials ur-l(l-u) n-r, 1 ~ r ~ n, n ~ I,
it follows that F-l(u) = r-l(u) and thus F(u) = r(u) (we can
easily get rid of the restriction that statements are valid only almost surely with respect to Lebesgue measure).
This theorem is known as the completeness of the
above polynomials over the interval (0,I). The question is therefore which subset of the above polynomials is complete over the interval (0,i)?
We quote, without
proof, one solution to this question as proposed by J.S. Huang (1975). Theorem 3.4.2.
Let the sequence r(n) of integers be such that 1 ~ r(n) ~ n
and, with some fixed m ~ I, r(m) ~ r(n) ~ r(m)+n-m for all n ~ m.
Then if
E(Xr(m):m ) is finite, the sequence Er(n):n, n ~ m, uniquely determines the population distribution F(x). As an example, consider the Pareto distribution F(x) = 1-I/x, x ~ i.
If
XI,X2,...,Xn are i.i.d, with this distribution F(x), then Theorem 3.4.1 is not applicable, because E(XI) = El: 1 = +~.
But, since El: 2 is finite, Theorem 3.4.2
implies that Er(n):n , n ~ 2, characterizes the Pareto distribution F(x) = 1-i/x, x ~ i, whenever 1 ~ r(n) ~ n-i for n ~ 2 (we chose m = 2 and r(m) = 1). In this same paper, Huang (1975) points out that a certain population distribution may not be characterized by the sequence Er(n):n simply because its members
57
are not finite for any n, while through an appropriate monotonic transformation this same distribution can be characterized by a moment sequence (recall that any continuous distribution can be transformed into exponential). 1-i/log x, x ~ e.
Such an example is F(x) =
Indeed, by virtue of (2),
P(Xl:n<X) = 1 - (log x) -n , and thus, for any r -> i,
Er:n
El:n
= / [l_P(Xl:n<X)ldx = / e e
dx - +oo ( l o g x) n
However, with the transformed sequence log log Xr: n = X*r:n' C o r o l l a r y
3.4.1
is
applicable and we get that if, for all n ~ I, E(log log Xl:n) = i/n then the population distribution is F(x) = 1-I/log x, x ~ e. Although we stated our ~heorems in terms of the first moments of order statistics, they are directly extendable to higher moments as well.
For example, if the
basic random variables Xj are nonnegative then the monotonic transformation X~ = X~ ] ] transforms the order statistics Xr: n to X t t > 0, and thus Theorem 3.4.1 applied r:n' to X~, implies that the nt~nbers E(X$:n) , 1 ~ r ~ n, n a I, uniquely determine the population distribution F(x), whenever E(X~)_ is finite.
If Xj a 0
is not assumed
then the argument still works if t is an odd integer. We have to be careful with subsequences Er(nk):nk , however.
Huang (1975) gives
the example that if F(x) is symmetric about the origin then E(XI) = 0 and in fact Er(n):n = 0 for all odd n and r(n) = ½(n+l).
Therefore, this particular subsequence
evidently does not characterize any distribution.
On the other hand, if we take
extreme order statistics rather than quantiles, then Er(nk ) :nk does characterize the population distribution for a large class of subsequences.
Namely, the following
result holds (see Huang (1975)). Theorem 3.4.3.
Assume that there is a pair (t,m) of integers such that
1 -< t -< m and Et: m is finite.
Then Er(nk ) :nk, k -> I, uniquely determines the popu-
lation distribution whenever 1 -< r(nk) _< t and n k _> m for all k -> I, and if
k:l nk It is, of course, evident that we could use upper extremes as well, i.e. 0 -< n k - r(nk) < t. As the case of Theorem 3.4.2, this theorem also reduces to a completeness theorem for the polynomials appearing in (29). Such a completeness theorem can be found for example in the book by R.P. Boas (1954, p. 235), which we do not reproduce here.
58
For further results on subsets of the numbers Er:n, 1 ~ r ~ n, n ~ i, which characterize the population distribution, see B.C. Arnold and G. Meeden (1975).
Al-
though Arnold and Meeden consider arbitrary moments (not just first ones), most of their results can be given in an equivalent f o m
in terms of first moments, as was
remarked in the second paragraph preceding Theorem 3.4.3. The investigation by Arnold and Meeden is related to the following area of research developed by J.B. Kadane (1971 and 1974) and by C. Mallows (1973). A = (ar,n: 1 ~ r ~ n, 1 ~ n ~ N) be a finite triangular array of numbers.
Let What con-
ditions on A can guarantee that there exists a distribution function F(x) concentrated on the nonnegative real line such that, if XI,X2,...,X n are i.i.d, random variables with co~aon distribution function F(x),
= a for all entries r:n r,n In their solutions, both Kadane and Mallows reduce this problem to a moment
of A?
then E
problem, combined with a recursive formula similar to (27). The reduction of this problem to a moment problem can, of course, go on the line of our arriving at the equation (28), but the emphasis is different here.
Not so much the uniqueness of
F(x) is emphasized, but rather the existence of at least one F(x) with the above property when A is given.
Kadane (1974) observes that a recursive formula similar
to (27) makes it possible that only the last entries a
in each column are needed n,n to decide the existence of a distribution function F(x) with the previously stated properties. follows.
His result which also contains the earlier result of Mallows, is as
If the elements of A satisfy the quoted recursive relation and if the
numbers m k = dk, k = 0,1,...,N-2, where d n = an, n - an_l,n_l, 2 ~ n ~ N, k-th moments of a positive measure on (0,I), such that Er: n = ar, n for each entry of A.
This, of course, becomes a characteriza-
tion theorem whenever the moments m k characterize a distribution. with finite N.
are the
then there is a distribution fumction
It is very rare
However, if N is infinite then Theorem 3.4.1 implies that, for a
given A, either there is a unique distribution F(x) with the above properties or there is none. Let us now return to problems of characterization by the set Er: n of first moments. Let us introduce the differences Ar: n E0: n = 0. and n ~ I.
Er: n
Er_l:n, 1 ~ r ~ n, n ~ i, where
It is, of course, equivalent to specify Ar: n or Er: n for all 1 ~ r ~ n However, if only some values of r are considered for each n, then it is
a new problem whether the sequence Ar(n):n, n ~ l, characterizes the population distribution.
In this connection, the most general known result is due to Z. Govin-
darajulu et aZ. (1975), which we state below. Theorem 3.4.4.
Let XI,X2,...,X n be independent random variables with common
distribution function F(x).
Assume that there are two integers 1 ~ k < m such that
both Ek: m and Ek+l: m are finite.
Then the sequence Ak+l: n, n ~ m, uniquely deter-
mines the family (F(x+~)~ of distribution functions, where c is an arbitrary real number.
59
Remark:
Since upper extremes can be transformed into lower extremes by turn-
ing to the sequence {-Xj}, it follows that the sequence An_k: n, n e m, also determines the family F(x+c), given that both Em_k: m and Em_k.l: m are finite for some k and m. Proof:
Since the proof is exactly the same as that of Theorem 3.4.1, we therefore
give a short outline only.
If we start with (26) again, then with the notations of
the preof of Theorem 3.4.1, we get
(30)
1 f [F-l(u) - T-l(u)](nu-k)uk-l(l-u)n-k-ldu = 0 , n e m . 0
Because now the multiplier of (l-u)n-k-I depends on the exponent n, one additional trick is needed to reduce (30) to a completeness theorem as compared with the case of (29). The major idea of Govindarajulu et aZ. (1975) is to recognize that with the beta function
~ua-l(l_u)b-i Ba,b(U) = F(a)F(b) and with the constant c = /l[F-l(u) - T-l(u)]Bk,m~k(U)du , 0 (30) is equivalent to 1
f [F-l(u) - r-l(u) - cluk-l(1-u)J-ldu = 0
for all j ~ m-k ,
0 which can also be written as
(31)
1 f fk(u)uSdu = 0 , 0
s = 0,1,2 .....
where fk(u) = [F-l(u) - T-l(u) _ c]uk-l(l_u) m-k-I
Now, just as in the proof of all u.
Theorem 3.4.1, (31) implies that fk(u) = 0 for almost
Since F and T are distribution functions, we obtained that T(x) = F(x+c) as
claimed. For characterizations by sequences of variances and covariances of order statistics, see Z. Govindarajulu (1966 and 1975). The results presented so far in this section reduce distributional properties
60
to expectations.
On the other hand, if we want to simplify characterizations based
on independence, then conditional expectations are the natural tools to be used.
We
devote the rest of this section to such characterizations, i.e., when our assumptions are in terms of conditional expectations (regression) of functions of order statistics.
Such results are not necessarily extensions of those in Section 3.3,
since the finiteness of conditional expectations presumes that the corresponding expectations (without condition) are also finite, which was not assumed in Section 3.3. However, the actual results are easier to apply in many cases. Again, let XI,X 2 .... , X be independent random variables with cow,non distribution function F(x).
Motivated by the theorem of Fisz (Theorem 3.3.1), we first
prove the following result. Theorem 3.4.5.
If E(XI) is finite and if F(x) is continuous, then
E(X2:2-XI:21XI: 2 = y) is constant almost surely with respect to F(x) if, and only if, F(x) is exponential. Proof:
Since, for any y such that 0 < F(y) < I,
E(X2:2-XI:2]XI:
2 = y)
= E(X2:2]XI: 2 = y)
- y ,
Theorem 3.1.3 implies that, for almost all y with respect to F(y), for which 0 < F(y) < I, q-co
(32)
E(X2:2-XI:zIXI:
2 = y)
= f xdF*(x) Y
- y ,
where
(33)
F*(x) =
F (x) - F (y) 1-F(y)
for x -> y .
If the left hand side of (32) is constant almost surely then (32) is equivalent to +oo
f xdF*(x) = c + y Y
a.s. (with respect to F)
or to +oo
(34)
.f
x~(x)
= (c+y)[l-F(y)]
a.s. (with respect to F),
Y
where y is such that 0 < F(y) < I. We first show that (34) in fact is valid for all y, for which 0 < F(y) < I. Indeed, if a < b are two real numbers such that 0 < F(a) = F(b) < I, then
61
f xdF(x) = f xdF(x) a
b
but the right hand side of (34) increases from (c+a)[l-F(a)]
to (c+b)[l-F(a)].
Hence, if a and b have a neighborhood in which F(x) increases then, by the continuity of F(x), (34) fails in some neighborhoods of a and b, although these neighborhoods have positive F-measure.
This contradiction leads to the property that F(x) is
strictly increasing for all x for which 0 < F(x) < I.
But then, if (34) holds for
almost all y then it holds for all y because F(y) is assumed to be continuous (y is again restricted to 0 < F(y) < i). The completion of the proof is now routine.
If we integrate by parts, we get
(see Lenuna I. 2. I) +co
+co
f xdF(x) = y[1-F(y)] Y
+ f [1-F(x)]dx Y
.
Hence (34) b e c o m e s
f [l-F(x)]dx = c[l-F(y)] Y whenever 0 < F(y) < i. (35)
Therefore 1
F(y) : 1 - exp[- ~ (y-B)]
w h e r e B i s an a r b i t r a r y
constant
, and y i s s u c h t h a t
0 < F(y) < 1,
Since F(y)
is
continuous by assumption, (35) is valid for all y ~ B, which was to be proved.
D
The above result, and essentially the proof itself, is due to T. Ferguson (1967).
Ferguson actually obtains the same conclusion when the sample size is not
restricted to n = 2, which modifies F*(x) of (33) slightly, but the argument remains unchanged
(the case
E(Xm+l:n-Xm:nlXm:n
= y) = constant
can easily be reduced to
E(X2:n-Xl:nlXl: n = y) = constant by an appeal to Theorem 3.1.3).
Implicitly,some results of Ferguson are contained,
under sccae additional assumptions, in the works by G.S. Ro~ers (1963) and M.S. Srivastava (1967). An interesting observation by M. I. Beg and S.N.U.A. Kirmani (1974) is that the following result can be reduced to Theorem 3.4.5.
62
Theorem 3.4.6.
Let F(x) be continuous and assume that E(XI) is finite.
If
denotes the arithmetical mean (I/n) (XI+...+Xn) and if E(X-ylXI: n = y) is constant almost surely (with respect to F), then F(x) is exponential. Proof:
We first calculate the conditional distribution of Xj, given XI: n = y.
We
easily get either by direct calculations, or from Theorem 3.1.3, that
1 n-I F(x)-F(y) P(Xj<x]XI:n = Y) = n + n l-F(y)
if x > y
while this conditional distribution is zero for x -< y.
Hence,
+co
(36)
E(Xj I X l : n = y ) = zn +
n-1 / xdF(x) n[1-F(y) ] Y
a.s.
(F) .
Now observe t h a t the r i g h t hand s i d e in (36) does not depend on j .
Thus, i f we de-
note the conditional expectation (37)
E(Xj I X l : n : y) = gn(y) , 1
then E(X[XI:n ~ y) = n X~'=IE(Xj IXl:n = y)
also equals gn(y).
However, E(XIXI:n = y) =
y + cn, hence
(38)
gn(y) : y + cn ,
a.s. (F) .
Furthermore, notice that, in view of (36), 1 n-i E(nXj-ylXI:n= y) = I E ( n X j _ X I : n l X I : n =
does not depend on n.
y)
Thus its value is the same for all n.
In particular, for
n = 2 we get from (37) and (38)
E(2Xj-XI:21XI:2: Y) = 2E(XDIXI:2: Y) - y : y + c 2 • On -the other hand, since E (X1 IX1 :2= y) = E (X 21X 1 :2 = Y) almost surely,
E(2XD-XI:21XI:2 = Y) = E(XI+X2-XI:2[XI:2 = Y)
= E(XI:2+X2:2-XI:21XI:2 = Y) = E(X2:21XI:2 = Y) • Conbining
these two expressions, we obtain
E(X2:2[XI:2 = y) : y + c 2
a.s. (F) ,
63
which is exactly the assumption of Theorem 3.4.5.
We thus get the conclusion of
Theorem 3.4.6 from Theorem 3.4.5. Theorem 3.4.6 was first proved in a dissertation by Y.H. Wang (1971) and independently by A.C. Dallas (1973), whose direct proofs are also quite simple.
In
a recent paper, R.C. Srivastava and Y.H. Wang (1978) considered the problem of characterizing distributions by assumptions on the regression E(ZklXk:n = y) where n
Zk =
~ (Xd - Xk. n) j=k+l j :n
.
Notice that n
1 E ( X - y l X l : n = y) = ~ E ( . X I x j : n j= Consequently, the investigation
1 - n X l : n l X l : n = y) = ~ E ( Z l l X l : n = Y) •
by S r i v a s t a v a
and Wang i s r e l a t e d
t o Theorem 3 . 4 . 6 .
One way o f r e d u c i n g t h e c a s e o f g e n e r a l k t o k = 1 would be t o a p p l y Theorem 3 . 1 . 3 . Instead of doing this,
however, S r i v a s t a v a
w h i c h wi11 p r o b a b l y b e f r u i t f u l
and Wang d e v e l o p a d i r e c t method o f p r o o f ,
in other investigations
related
to functions of or-
der statistics. All results
in terms of regression
than constant regressions. cient of y different ceding arguments. ilies details
can e a s i l y b e e x t e n d e d t o l i n e a r
Such an e x t e n s i o n w i l l
lead, e.g.
rather
i n (34) t o a c o e f f i -
from one, b u t no o t h e r change i s r e q u i r e d i n any o f t h e p r e Depending on t h i s
of distributions
as s o l u t i o n s
of these calculations
coefficient,
o f c o u r s e , we g e t d i f f e r e n t
to the corresponding regression
are left
to the reader.
problem.
famThe
CHAPTER 4 CHARACTERIZATIONS OF THE POISSON PROCESS 4.1.
Introduction. The simplest way to define the Poisson process is as follows.
Let XI,X2,...
be independent and identically distributed nonnegative random variables.
Then the
sequence SI,$2,... of points, where Sn = X 1 + X 2 +...+ Xn , is called a renewal process.
In the case when the common distribution function of
the Xj is F(x) = l-e -bx, b > 0, x -> 0, then the process {Sn, n~l} is called a
Poisson process.
Emphasizing another viewpoint, the renewal process {Sn, n~l} is a
sequence of random variables 0 ~ S 1 ~ $2~...,
for which the intervals Sj-Sj_I, j ~ i,
with S O = 0, are independent and identically distributed random variables.
Through-
out this chapter, we shall assume that the common distribution function F(x) of the intervals Sj-Sj_ 1 is continuous.
Hence, Sj>Sj_ 1 with probability one.
refer to the Sj as points of the process and to F(x) of the process.
We shall
as the interval distribution
The notation {Sn, n~l; F(x)} will always mean a renewal process
whose points are Sn, n~l, and whose interval distribution is F(x).
Furthermore, S O
is always zero. A more general concept than that of a renewal process is a point process. Here we adopt the simplest definition by calling any set {Ts,SCT} of no,negative random variables ~s a point process.
Hence, if T = {0,1,2 .... } and the differences
TS-TS_I, S~I, T O = 0, are independent and identically distributed
random variables
then the point process {Ts,S~T} is a renewal process. In a point process, the variables T s will also be referred to as the points of the process. cess.
We always denote the points by T s if we speak of a general point pro-
However, if the process is known to be renewal then the previous notation
{ S , n~l; F(x)} will be used. For a given point process, N(t) is defined as the (random) number of points in the interval (0,t). For a renewal process, N(t) is finite with probability one. On the other hand, in all cases of our investigation, the conditions on a point process will guarantee the finiteness (with probability one) of N(t).
Hence, we can
speak of N(t2) - N(tl) , which is the number of points of the process under consideration in the interval (tl,t2) , t I < t 2.
Since the distributions of the points T s are
always assumed to be continuous, endpoints of intervals can be ignored or included at will. Let us return to renewal processes.
The reason a Poisson process is named
65
"Poisson" is that
(1)
P(N(t)=k) =
(bt) ke -bt
k!
,
k = 0,1,2
. . . . .
where b > 0 is the parameter of the interval distribution F(x) = l-e -bx, x >_ 0. prove (I) by the following simple argument.
(2)
We
Notice that
P(N(t):0) = P(SI_>t) = l-F(t) = e -bt ,
which is formula (I) with k = 0.
Now, since
P ( N ( t ) = l ) = P ( S l < t , S2_>t) , the continuous version of the total probability rule yields
(3)
P(N(t)=I)
=
f
P ( S I < t , S2~t[Sl=X)be-bXdx
0 t t = f P(S2->tlSl=X)be-bXdx = f P(X2->t-x)be-bXdx 0 0 = bfte-b(t-X)e-bXdx = bt e -bt 0 Similarly, since
P(N(t)=k) = P(Skt), k -> 1, we have +¢o
(4)
P(N(t)=k) = f P(Sk
where fk(x) i s the d e n s i t y f u n c t i o n o f Sk. I t i s , however, w e l l known (and i t can e a s i l y be o b t a i n e d by the c o n v o l u t i o n formula) t h a t i f the Xj are e x p o n e n t i a l v a r i a b l e s then fk(x) i s a gamna (Erlang) d e n s i t y . bkxk-le-bX (5)
fk (x) =
(k-l)!
, x ~ 0, k ~ i,
and thus, from (4), te-b (t-x) bkx k-le-bx
P(N(t)=k) = f 0
(k-l)'
dx ,
which is indeed equal to the value given in (I).
That i s ,
66
In the next section we shall present a basic characterization theorem for Poisson processes, which theorem can serve as an alternate definition as well. Before turning to the mathematical theory, however, let us list some examples of renewal processes. At a busy telephone exchange, the successive calls arriving to a given operator can usually be assumed to form a renewal process.
Similarly, the successive
breakdowns of a new piece of equipment (within a reasonable warranty period); arrivals of buses at a stop; successive time epochs of accidents, and others can serve as typical examples for renewal processes. Now if a practical model can be assumed to be a renewal process then in order to reduce it to a Poisson process it is required to characterize its interval distribution F(x) to be exponential.
This is how the present chapter relates to the
others in the present monograph. There are two ways to apply previous characterization theorems for characterization of Poisson processes within the class of all renewal processes.
Namely, if
~Sn, n~l; F(x)) is a renewal process, then the sequence X 1 = S 1 and Xj = Sj-Sj_I, j ~ 2, consists of independent and identically distributed random variables with common distribution function F(x).
Hence, through the evolution of a single process,
we can form order statistics of the variables XI,X2,...,XN, as well as use other properties of this sequence which lead to a (unique) family of distributions for F(x).
Another method of characterizing F(x) is to observe the process {Sn, n~l;F(x)}
several times and to use these observations to determine F(x).
In this second method
we do not have to follow up the whole process; it suffices to make observations on the first occurrence S 1 only.
Let us illustrate these two alternative methods by
an example for each case. Example I.
Let {Sn, n~l; F(x)} be a renewal process.
Hence, X 1 = S 1 and
Xj = Sj-Sj_I, j ~ i, are independent random variables with co~aon distribution function F(x).
Let us combine the variables X2j+I and X2j+2, j ~ 0, and put
Wj = min(X2j+l,X2j+2 )
and UD = IX2D+I-X2D+21
.
Now, through the evolution of the process (Sn, n~l; F(x)}, we can observe (W0,U0) , (WI,UI) ,..., (WN,UN).
If, by a statistical method, we can conclude that the compon-
ents of (Wj,Uj) are independent, then Theorem 3.3.2 implies that F(x) is exponential and thus that the process in question is Poisson. Example 2.
Again let {Sn, n~l; F(x)} be a renewal process.
Instead of the
evolution of the process, we now make independent observations SI,I,SI,2,... on the time S 1 to the first occurrence of an event in the process {Sn).
Now, if for
one k > 2, k times the smallest WI, k of SI,I,SI,2,...,SI, k is distributed as the SI, j, that is, if the distribution function of kWl, k is F(x), and if F(x)/x ÷ b > 0
67
as x ÷ 0+, then Theorem 3.2.2 implies that the process in question is Poisson. While these two methods provide a wide variety of tools to decide whether a renewal process is Poisson, our aim here is different from listing these possibilities as corollaries to the theory developed in the previous chapters.
We shall
rather investigate properties which involve both the points Sn, n ~ i, and the number N(t) of points in a fixed interval (0,t). Thus new questions arise which cannot be formulated in terms of a single sequence or a single distribution.
On the other
hand, we shall try to relate all new characterizations to be obtained to previous ones whenever it is possible. 4.2.
Basic p r o p e r t i e s of Poisson p r o c e s s e s . Starting with relation
(1), we i m m e d i a t e l y o b t a i n t h e f o l l o w i n g p r o p e r t i e s
of
a Poisson process: (i)
(ii)
lim 1-P(N(t)=0) = b > 0 • t t=0 P (no point is in (tl,t2)) depends on the length t2-t I of the interval only;
(iii)
the events {no point is in (tj,tj+l)}, tj_l, are independent if the intervals (tj ,tj+l), j->l, are disjoint;
and (iv)
lim P ( N ( t ) = l ) t= 0 t
~ b > 0 .
Indeed, (iv) is obtained by a straight substitution into (I), while (i) follows from (I) upon applying a finite Taylor expansion of l-e -bt. (6)
For (ii), notice that
P(no point is in (tl,t2)) = P(N(t2) - N(tl) ~ 0) +co
=
where S O = 0.
+co
~ P(N(t 1) = k, N(t2) = k) = ~ P ( S k < t l , Sk+l_>t2) , k=0 k=0 Applying again the argument used in (4), we get, for k a l, t1
P(Sk
Hence, on account of (6),
68 which shows that this expression is a function of t2-t I only as claimed in (ii). The proof of (iii) is similar to the preceding argument.
We demonstrate it
for two disjoint intervals (tl,t2) and (t3,t4) , where t I < t 2 ~ t 3 < t 4.
Let A
and B denote the events that no points of the process fall into the intervals (tl,t2) and (t3,t4),respectively.
Then AB is the event that,for some integers 0 ~ k ~ m,
{Sk
(7)
If this latter event is denoted by Ck,m, then
P(AB) = k=0 )~ m~k p(Ck'm)=
We now distinguish two cases.
First we evaluate P(Cm,m).
Applying the method utili-
zed for the derivation of (4), we get t1
(8)
P(Cm, m) = f P(Sm+I~t41S m = x ) f ( x ) d x 0 tl
-bt 4
= f P(Xm+l~t4-x)fm(x)dx = e 0 We now turn to P(Ck,m) with k < m.
btT m!
Since, in this case, Sk < tl, Sk+ 1 ~ t 2 but
Sk+ 1 ~ Sm < t 3, and Sm+ 1 ~ t4, we have tI t3 (9)
P(Ck, m) ~ ~ ~2P(Sk+l~t2 , Sm+l~t41S k = x, Sm = y)gk,m(X,y)dydx
~1 t3 = 0
f P(t2-X~Xk+l~Y-X,Xm+l~t4-YlSk = x, Sm = y)gk,m(X,y)dydx , t2
where gk,m(X,y) is the joint density function of the vector (Sk,Sm) , k < m.
Sm = Sk + (×k+l+...+~) = Sk + Sm~- k '
Since
s~,
gk,m(X,y) can easily be evaluated by observing that Sk and S~_ k are independent with density functions fk(x) and f_k(Z), respectively, where fk(x) is the function defined at (5). Furthermore, (9a)
{Sk = x, Sm = y} = {Sk = x, S~_ k = y-x}.
gk,m(X,Y) = fk(x)f_k(Y-X)
Therefore
, 0 -< x ~ y, 1 ~ k < m.
The evaluation of P(AB) in (7) can now be easily completed as follows.
Upon observ-
ing the special relation of the condition {Sk = x, Sm = y} in (9) to Xk+ 1 and Xm+l, we first simplify the integrand in (9) by an additional conditioning on (Xk+ 1 = z}. We then substitute these integrals into (7), where interchanging summation and integration and utilizing (ga) and (5) yield
69
P(AB) = e x p [ - b ( t 2 - t l ) ] e x p [ - b ( t 4 - t 3 )
] .
This shows t h e i n d e p e n d e n c e o f A and B i n view o f t h e b a s i c p r o p e r t y ( i i ) ,
estab-
lished earlier. The p r o o f o f ( i i i )
f o r more t h a n two d i s j o i n t
intervals
(tj,tj+l),
j e 1, i s
Notice that,
i n view
q u i t e s i m i l a r and we s h a l l omit t h e c o m p u t a t i o n a l d e t a i l s . L e t us now r e t u r n t o t h e b a s i c p r o p e r t i e s
(i)
- (iv).
of the obvious equation
(10)
P ( N ( t ) = 0) + P(N(t) = 1) + P(N(t) >- 2) = 1 ,
properties (i) and (iv) imply (v)
l i m P(N(t)>-2) t t=0
- 0
In fact, a direct consequence of (I) is that ~(bt)ke -bt P(N(t) ->2) =
L k=2
b
+~
k-2
-< e- t(bt) 2 ~ ~ k=-2 ' * " - ' "
~ "
= (bt) 2
For the sake of future reference, we include this last estimate in the following Theorem 4.2.1.
The Poisson process {Sn, n~l; F(x)}, where F(x) = 1 - e -bx,
x ~ 0, satisfies the basic properties (i) - (iv). Furthermore, the following inequality (vi)
P(N(t)~2)
~ (bt)
2
,
holds. As we s h a l l s e e l a t e r ,
properties
p r o c e s s among p o i n t p r o c e s s e s .
(i)
- ( i v ) and ( v i ) c h a r a c t e r i z e t h e P o i s s o n
Notice the fact that these properties
do n o t r e f e r
t o t h e f a c t t h a t t h e p o i n t s o f t h e p r o c e s s i n q u e s t i o n form a r e n e w a l p r o c e s s :
each
of the properties (i) - (vi) is formulated in terms of the distribution of N(t) (with varying t).
It is, of course, evident that some of the properties (i) - (vi) follow
from the others.
For example, (v) is a consequence of (vi), or of the combination
of (i) and (iv). Our aim with stating all of these properties is to emphasize the most basic ones pertaining to a Poisson process. with these properties to define a Poisson process.
In fact, several text books start However, we wanted to relate
Poisson processes to the exponential distribution from the very start, which warranted our particular definition. Before proving that properties (i) - (vi) characterize the Poisson process among point processes {%i } (with some minor restrictions which would guarantee that P(~j = %k ) = 0
for all j # ~ , w e deduce a consequence of the basic properties
70
(i)
- (iv).
To simplify expressions, we introduce an additional notation.
Let A be the
union of a finite number of disjoint finite intervals (tj, tj+l), 1 _< j -< M.
We
define N(A) to be the number of points of the process in question which fall into the set A.
Hence
M N(A) = j~l[N(tj+l) - N(tj)]
.
We always assume that the process {~j} is such that P(~j = t) = 0 for any t and P(Tj = Tk) = 0 for all j ~ k, hence the endpoints of intervals may be omitted or included at will without affecting the results.
For renewal processes, we can
achieve this by assuming that the interval function F(x) is continuous, but for our very general definition of a point process, this requires more than the continuity of the distribution function at each point (the reader will find examples in Chapter S where a vector (X,Y) is such that X -~ 0, Y ~ 0, each component has an exponential distribution, but P(X=Y) > 0). We shall also express P(N[A) = 0) for the sets A introduced above (Ii)
P(N[A) = 0) = e -m[A) ,
whenever the left hand side is positive.
Although (II) is just a change in notation,
it helps us to formulate very simple conditions for P(~j = ~k) = 0 (j ~ k) in the case of processes which satisfy property (iii). Namely, under (iii), re(A), is a nonnegative additive set function on the finite unions of finite intervals.
Therefore,
the simple additional requirement that m(A) should be nonatomic is equivalent to our conditions on the sequence {zi }. We now prove the following general result. Theorem 4.2.2.
Let m(A) in (ii) be a nonatomic, nonnegative and additive set
function on the finite unions of finite intervals.
Assume that the point process
{Tj} is such that, for finite intervals I, (12)
P(N(I)
-> 2) _< b l I J 2 ,
where b > 0 is some fixed constant and Ill signifies the length (or Lebesgue measure) of the interval I.
Then, if lj = (tj, tj+l) , tj < tj+l, 1 _~ j -~ M, are disjoint
intervals, the random variables N(Ij), 1 -~ j -~ M, are independent. Proof:
The fact that re(A) in (II) is an additive set function implies that the events
{N(Ij) = 0}, 1 -< j _< M, are independent. A=
M UI. , j=l ]
Indeed, if
71
then
m(A) : m(I1) + m ( I 2 ) + . . . + m(I M) , and, thus, in view of (Ii),
P(N(A) = 0) = P ( N ( I j ) = 0, 1 -< j -< M) = e -m(A) M M -m(Ij ) M = exp[- j =~l m(Ij)] = j =~ l e = j =~ l P ( N ( I j ) = 0) "
Since the above argument i s e v i d e n t l y v a l i d f o r any s u b - c o l l e c t i o n
of the i n t e r v a l s
I j , we have shown the independence of the e v e n t s {N(Ij) = 0}, 1 _< j -< M. We now prove the independence of the random v a r i a b l e s N ( I j ) , o t h e r words, we want to prove t h a t ,
for arbitrary
1 -< j -< M.
In
n o n n e g a t i v e i n t e g e r s k j , 1 <- j -< M,
the d i f f e r e n c e M
(13)
P(N(I~) = k~, 1 -< j -< M) - j =IIl P ( N ( I j ) = k~) J J J
equals zero.
Let us divide the intervals Ij into r equal parts, Ij,k, say, where
now 1 -< j -< M and 1 -< k -< r.
Let ej,k be the indicator variable of the event
{N(lj,k) = 0), i.e., ej, k = 1 or 0 according to whether N(Ij,k) = 0 or N(Ij, k) -> I. Arguing as above with the events (N(Ij) = 0}, independent for all j and k.
Ej = Therefore,
we get that the variables ej ,k are
But then so are the random variables
r ~ ( 1 - e j , k) , 1-< j -< ) 4 . k=l i f we r e p l a c e N ( I j ) by Ej i n (13), then the v a l u e o f the d i f f e r e n c e s i m i -
l a r to the one in (13) is zero, i.e., (14)
M =IIIP(E j = k~)j = 0 .
P(E4j = ks,j 1 -< j _< M) -
J However, s i n c e r
P ( N ( I j ) ~ Ej) -< k=l ~ P ( N ( I j , k ) _> 2) , we have from (12) that for any fixed j,
(is)
IP(N(Ij) = kj) - P(Ej = kj) I < P(N(Ij) # Ej)
-< b ~ I I j , k 12 = b ~ I l j 1 2 - b l - j k=l k=l r 2-
72
Similarly,
(16)
b M IP(N(Ij) = kj, 1 _< j _< M) _ P(Ej = kj, I -< j -< M) I -< r j=~iIIj[2
Since, b, M and the lengths IIjl are fixed, relations (14) - (16) imply that the absolute value of the difference in (13) is smaller than c/r, where c > 0 is a suitable constant and r is an arbitrary positive integer.
Letting r ÷ +~ yields that
the difference (13) is zero, which means the independence of the random variables N(Ij), 1 -< j -< M.
The proof is completed.
D
The meaning of Theorem 4.2.2 is that among the basic properties (i) - (vi), (iii) can freely be replaced by the seemingly stronger condition (iii e)
the random variables N(li)~ are independent, where Ij = (ti, ti+l),
tj < tj+l,
1 _< j _< M, and the i n t e r v a l s I j a r e nonoverlapping.
The usual d e f i n i t i o n o f a Poisson process in the l i t e r a t u r e
i s to assume t h a t i t i s
a p o i n t process s a t i s f y i n g the b a s i c p r o p e r t i e s ( i ) ,
( i i i ~) and ( i v ) , where
(ii~),
( i i *) i s the following modified v e r s i o n o f ( i i ) : (ii ~)
the distribution of N(1), where I is an interval, depends on the length of the interval only.
We shall now prove that such a definition is equivalent to our definition given in the introduction.
Other theorems of the present section will be devoted to the
consequences of other combinations of the basic properties (i) - (vi).
Theorem 4.2.3. Let {~s } be a p o i n t process on the nonnegative r e a l l i n e . Assume t h a t the random v a r i a b l e s N ( I j ) , where the Ij a r e f i n i t e i n t e r v a l s , s a t i s f y conditions (i),
(ii~),
( i i i ~) and ( i v ) , where N(t) denotes N(I) f o r I = ( 0 , t ) .
Then
the p o i n t s {~s } can almost s u r e l y be arranged i n t o an i n c r e a s i n g sequence {Sj } and the process {Sj, j>-l} i s Poisson. Proof:
We s h a l l f i r s t prove t h a t , f o r any f i x e d t > 0, (bt) k e - b t
(17)
P(N(t)= k)=
k!
,
k = 0,1,2 .....
where b is given in (i). (This was established in (I) for Poisson processes.)
Let
us put
Pk(t) = P(N(t) = k ) , k = 0,1 . . . . . We shall prove (17) by induction over k.
First let k = 0.
Then, for any t z 0 and
S ~ 0, {N(t+s) = 0} means that N(t) = 0 and that there are no r's in I l = (t, t+s).
73
Thus, in view of the assumptions (ii*) and (iii*),
P0(t+s)
= P0(t)P0(s)
.
Also, P0(t) is monotonic.
Since, in view of condition (i), P0(t) is not constant in
t, Theorem 1.3.1 implies that
P0(t) = e -at , t -> 0 ,
where a > 0 is a suitable constant.
One more appeal to (i) now yields that a = b
and we have thus verified (17) for k = 0. Turning now to Pl(t), we calculate Pl(t+s) as above by splitting the interval (0,t+s) into the union of (0,t) and I 1 = (t, t+s).
Since, by (iii*), N(t) and
N(II) are independent and since {N(t+s)
= 1) = { N ( t )
= O, N ( I 1) = 1} U ( N ( t )
= 1, N ( I 1 )
= O}
we have Pl(t+s)
= Po(t)P(N(I1)
= 1) + P I ( t ) P ( N ( I 1 )
= O) .
If we now apply (ii*) and (17) with k = 0, we get from the last equation
Pl(t+s)
= Po(t)Pl(S)
+ PI(t)Po(s)
= P l ( S ) e - b t + P l ( t ) e -bs . Denote g l ( u )
gl(t+s)
= Pl(u)ebU"
= gl(s)
+ gl(t)
w h i c h can be r e w r i t t e n
gl (t+s)-gl (s) t
The above e q u a t i o n
t h e n becomes
,
as
--
gl (t) t
-
-
P1 (t) t
bt e
On account of (iv), the right hand side converges to b > 0 as t ÷ 0. the left hand side, which means that gl(s) is differentiable and gi(s)
= b
.
Cons e q u e n t l y , gl(s)
= P l ( S ) e bs = bs + c ,
Hence, so does
74
where c is a constant.
Applying
(iv) again, we get c = 0, and thus (17) is verified
for k = i. Assume now that (17) is valid for some k and consider Pk+l(t).
Arguing as be-
fore, we obtain
(18)
Pk+l(t+s)
= Pk+l(t)P0(s)
where I 1 = (t,t+s),and P(N(I1)
thus,
~ 2) = P ( N ( s ) = o(s)
Consequently,
by ( i ) ,
+ O ( P ( N ( I 1) ~ 2))
,
as
s÷
= 0) - P ( N ( s )
P~+l(t)
= 1)
0.
(18) as
= Pk+l (t)
P0(s)-i - s
-
Pl(S) +
Pk ( t )
we obtain, on account of (i) and (iv),that Pk+l(t)
(19)
,
( i i *) and ( i v ) ,
e 2) = 1 - P ( N ( s )
if we rearrange
Pk+l(t+s) - Pk+l(t) s
+ Pk(t)Pl(S)
s
+ o(s)
,
is differentiable
and moreover,
= -bPk+l(t ) + bPk(t ) .
Applying the assumption of induction to Pk(t) and utilizing the fact that Pk+l(t) + 0 as t ÷ 0, which is an immediate consequence of (i), we obtain from (19) that (17) is valid for k+l.
Hence,
(17) is established.
It now follows from (17) that the points {~ ) can be rearranged into an ins creasing sequence S 1 < S 2 <... with probability one. In fact, we have from (17) that, with probability one, any finite interval can contain only a finite number of z-points.
Therefore,
the z-points can be arranged into a single increasing sequence
by first considering the points in the interval
(0,i), then those in (1,2), (2,3),
and so on. In order to complete the proof, we now show that the sequence 0 < S 1 < S 2 <... forms a Poisson process.
In other words, we have to prove that the random intervals
Sj - Sj_I, j ~ i, S O = 0, are independent with cor~aon distribution function F(x) = l-e -bx, x ~ 0.
We shall prove this again by induction.
P(S 1 _~ x) = P ( N ( x ) = 0)
Since
,
we get from (17) that
P(S 1 < x) = 1 - e
-bx
, x -> 0 .
Next, we calculate the joint distribution of S 1 and $2-SI, by appealing to the con-
75 tinuous version of the total probability rule.
We get
P(SI<X, S2-SI~Y) = ~ P(Sl<X, S2-SI~YIS 1 = z)be-bZdz 0 X
= f P(S2->z+YlS 1 = z)be-bZdz . 0 However, given S 1 = z, the inequality S 2 ~ z+y means that the interval (z, z+y) contains no points of the original process. Hence, the probability P(S2~z+yIS 1
z),
on account of (ii*), is equal to P(N(y) = 0), which, by (17) again, equals e- ~y. Consequently P(SI<X, S2-SI->y) = be-Y fXe-bZdz = e-bY(l_e -bx) 0 Hence, we have shown that S 1 and $2-S 1 are independent and identically distributed. Assume now that S1, S2-S1, .... Sk-Sk_ 1 are independent with common distribution function F(x) = l-e -bx, x ~ 0. Then, putting dj = Sj-Sj_ I, j ~ i, where S O = 0,
P(dl<X I, d2<x2,...,dk<Xk , dk+l~Y) = f --. / P(dj<xj, l~j~k, dk+l~Yld j = zj, l<-j~k)hk(Zl,...,zk)dZl...dz 0
k ,
0
where hk(Zl,Z2,...,zk)
= bkexp[-b(Zl+Z2+...+Zk )] ,
the joint density of (dl,d 2 .... ,dk). However, if zj < xj, 1 ~ j < k, then the conditional probability appearing in the integral above reduces once more to the probability that the interval (Zl+Z2+...+Zk , Zl+Z2+...+zk+y) contains no points of the process. Therefore, we obtain from (ii *) and (17) that this probability equals e -by . On the other hand, if zj ~ xj for some j, then this same conditional probability is zero.
Consequently, the last integral yields
P(dj<xj, l~j~k, dk~y ) = e-bYF(xl)F(x2 )...F(xk) where F(x) = l-e -bx, x ~ 0.
,
This completes the proof.
D
The results obtained so far in this section can be sm~arized into the following Corollary 4.2.1. set of real numbers.
Let {Ts, s~T} be a point process, where T is an arbitrary Let A be the union of a finite number of finite intervals of
the nonnegative real line.
Assume that P(N(A) = 0) > 0 for A's of positive Lebesgue
measure, and define m(A) by the equation
76
P(N(A) = 0) = e -re(A) Let m(A) be a nonnegative, nonatomic, additive set function.
Then the process (Ts)
is a Poisson process if, and only if, properties (ii*) and (vi) hold. Proof:
Let us first observe that all the assumptiomof the corollary are valid for
a Poisson process.
Namely, on account of (ii) and (17), m(A)/b equals the Lebesgue
measure of the set A, where b is the parameter of the interval distribution F(x). Hence, m(A) is indeed nonnegative, nonatomic and additive.
In addition, (ii*) can
be verified by the same simple calculation as the one applied at (6). Finally, it was shown in Theorem 4.2.1 that (vi) is valid for a Poisson process. Conversely, if m(A) is an additive set function, then property (iii) evidently holds.
However, property (ii), which is a special case of our assumption (ii*), and
property (iii) imply that m(A) is proportional to Lebesgue measure.
Indeed, if we
set I1 = (0,t) and 12 = (t, t+s), where t > 0, s > 0, then by (ii)
P(N(I2)
= 0) = P(N(s) = 0) = e -re(s)
,
where, for intervals l(x) = (0,x), we use the notations m(x) = m[I(x)] and N(x) = N[I(x)].
On the other hand, by (iii),
-m(llU 12 ) e
= P ( N ( I 1 U I 2) = 0) = P(N(I 1) = 0, N(I 2) = 0) = P(N(I 1) = 0 ) P ( N ( I 2) = 0) = e
-m(I 1) -m(I 2) e
and thus (20) m(t+s) = m(t) + re(s) ,
t,s > 0.
This is a variant of the lack of memory equation G(t+s) = G(t)G(s), t, s >- 0 , through the relation m(x) = -log G(x).
Since, by definition, m(x) is nondecreasing,
Theorem 1.3.1 implies that m(x) = bx for all x ~ 0, where b > 0
is a constant, be-
cause m(A) is nonatomic and is positive for all A of positive Lebesgue measure (and thus m(x) cannot be constant over a finite interval).
This specific form of m(x)
implies the validity of property (i). Therefore, (iv) also holds in view of our assumption (vi), while Theorem 4.2.2 implies that (iii*) is valid.
Hence, Theorem
4.2.3 is applicable, which completes the proof. In the preceding argument it was crucial to guarantee the validity of (20) for an arbitrary additive set function m(A).
This was achieved by the assumption
that (ii) should hold (we assumed the stronger version (ii*), but, for (20), the weaker form (ii) would have sufficed).
The question therefore arises whether condi-
77
tion (ii) is essential for the conclusion of Corollary 4.2.1 itself (i.e., perhaps wi~h amother proof we could have avoided it).
We shall see from the following
theorem, which is due to A. R6nyi (1967), that the answer to this question is yes; although the distribution of N(A) remains Poisson without assuming (ii) or (ii*), the point process {rs } is not Poisson in the sense of our definition unless (ii) is valid. Theorem 4.2.4.
For the point process {~s' s~T} let the set function m(A) be
defined as in Corollary 4.2.1.
As before, let m(A) be nonatomic and additive.
If,
for finite intervals I, (21)
P(N(I)
e 2) s b m 2 ( I )
where b > 0 is (22)
P(N(I)
a constant,
, then
= k) = m k ( I ) e - m ( I ) k!
Remark I.
,
k
=
0,1,2,...
.
The relation of (21) to (12) or to property (vi) is evident.
If
m(I) is proportional to Lebesgue measure then (21) reduces to (12) and if, in addition the distribution of N(1) depends only on the length of the interval I, then it further reduces to (vi). Remark 2.
The assumption in (21) can be weakened to the following condition
without affecting the conclusion of the theorem:
there is a strictly decreasing
function g(x) > 0 which tends to zero as x ÷ 0 and such that P(N(I)
-> 2) -< m ( I ) g [ m ( I ) ]
Remark 3.
.
The assumption that m(A) is nonatomic implies that m(1)+0 as d ÷ c
where I is the interval [c,d). Namely, otherwise c would be an atom of the function m(A).
Similarly, we obtain that m(I) is a continuous function of d. Remark 4.
Notice that (22) again implies that {~s } can be rearranged into a
single nondecreasing sequence SI~$2~ ....
However, this sequence is not a Poisson
process unless m(A) is proportional to Lebesgue measure (which is equivalent to the validity of (20), which in turn is equivalent to the validity of (ii)).
As a matter
of fact, P(SI<X ) = 1 - P(N(x)
= 0) = 1 - e -re(x)
,
where we again abbreviated m[(Q,x)] as m(X). Hence, even if the sequence SI~$2~... were a renewal process, its interval distribution is, in general, not exponential. We can also see from Theorem 4.2.4 that, in Corollary 4.2.1, (ii~) can be replaced by the weaker assumption (ii).
78 Proof of Theorem 4.2.4.
Let I = (c,d) and consider the characteristic
function (23) ~(u) = ECe iuN(I)) . Let us divide the interval into M equal subintervals, and put lj
=
(c + (j-l) (d-c) M
, c
+
j (%-c))
, j
=
1,2,...,M
.
Hence, (24) N(I) = N(II) + N(I2)+...+N(IM)
.
Now, since m(A) is additive, and since e -m(1) = P(N(I) = 0) = P(N(Ij) = 0, 1 -< j -< M) we have H P(N(Ij) = 0, 1 -< j -< H) = exp[-j~im(lj)] = P(N(I1) In other words,
= 0)P(N(I2)
the events
{N(Ij)
t h e same a r g u m e n t as i n t h e p r o o f of the events also
independent.
~(u) =
Therefore,
= 0)
.
= 0}, 1 -~ j s M, a r e i n d e p e n d e n t .
We c a n now u s e
o f Theorem 4 . 2 . 2
the independence
a b o v e and (21) i m p l y t h a t
M (2s)
= 0)...P(N(IM)
to conclude
t h e random v a r i a b l e s
on a c c o u n t
that
N(Ij),
1 -< j -< M, a r e
o f (23) a n d ( 2 4 ) ,
iuN(Ij )
~(e
)
.
j=l However, ~j (u) = E{exp [iuN(Zj ) ] } = E(e
iuN(Ij) [N(Ij)
= P(N(Ij) where
Rj = E(e
iuN(Ij)
= 0)P(N(Ij) = 0) + E(e iuN(Ij) ]N(Ij) = I)P(N(Ij)= I) +R..]
= 0) + e i U p ( N ( I j )
IN(Ij]-> 2)P(N(Ij)
= 1) + Rj ,
_> 2) = O(m2(Ij)).
Hence (26)
~j(U) = e -m(Ij) + eiU(l-e -m(Ij)] + O(m2(lj)) .
As M ÷ +0% m(li ) ÷ 0
(see Remark 3).
Therefore, by Taylor's expansion, (26)
79 becomes Cj(u) = l-m(Ij)(l-e iu) + O(m2(Ij)) , from which the Taylor expansion of log Cj (u) yields (27)
log Cj(u) = m(Ij)(eiU-l) + O(m2(Ij)) ,
where the error term is uniform in j.
Applying (27) to the logarithm of (25), we
get M
(28)
.
M
log ¢(u) = j~im(lj)(elU-l)= + O(j=~ m2(lj))
m(~)
(eiU-l) + O( max m(I~)) . 3 l-~j-<M
Notice that the left hand side does not depend on M, hence the limit of the right hand side, as M ÷ +~, gives log ¢(u).
As we have noted, however, (Remark 3), m(I)
is continuous in the upper end point of I, and thus, Heine's theorem of elementary calculus implies that it is uniformly continuous.
Consequently,
max m(I~) ÷ 0 as M + +~ J l_~j_<M and thus we get from (28) that ¢(u) = exp[m(I) (eiU-l)] . This is, however, equivalent to (22) by the inversion formula of characteristic functions.
The theorem is established.
D
We conclude this section with an example exhibiting the possible general nature of m(A).
Let g(t) be a nonnegative continuous function for t ~- 0.
Let
(29) re(A) = f g(t)dt , A where A is always the union of a finite number of finite intervals. integral in (29) is a Riemann integral.
Hence, the
For the special case A = (0,x), in which
case we write re(x) = m(A), X
(30) re(x) = f g(t)dt . 0 Evidently, m(x)
uniquely determines m(A).
Poisson process in Theorem 4.2.4.
Notice that g(t) = b > 0 leads to a
The function g(t) has the following important
meaning under the conditions of Theorem 4.2.4.
Let I = (t, t+s) in (22).
Then
80
t+s
P ( N ( I ) = O) = e x p [ - f g ( u ) d u ] t and thus lira 1 - P ( N ( I )
= O)
= g(t)
.
s
S= 0
In addition, again by (22), l i m P ( N ( I ) = 1) _ g ( t ) s= 0 s It is instructive (iv).
t o compare t h e s e l i m i t
T h e i r meanings a r e t h a t ,
e x a c t l y one p o i n t f a l l i n g the interval 4.2.4.
.
into
relations
(t,
t+s),
(t, t+s),
the probability
is approximately linear
of
in the length of
b o t h f o r a P o i s s o n p r o c e s s and f o r p r o c e s s e s d e s c r i b e d i n Theorem
Hence, what a c t u a l l y
distinguishes
a P o i s s o n p r o c e s s from a p r o c e s s r e p r e ~
s e n t e d i n Theorem 4 . 2 . 4 i s t h a t t h i s p r o b a b i l i t y (t,
w i t h t h e b a s i c f o r m u l a s ( i ) and
in short intervals
does n o t depend on t h e l o c a t i o n o f
t+s) for a Poisson process, while the location
Because o f t h i s
strong relation
a point process satisfying
is important in the latter
between t h e s e two t y p e s o f p r o c e s s e s ,
the conditions
we s h a l l
case. call
o f Theorem 4 . 2 . 4 w i t h m(A) d e f i n e d i n (29),
a time dependent Poisson process.
4.3.
Characterizations based on ave and residual l i f e . Let {Sn, n~l; F(x)} be a renewal process.
For fixed t > 0 we call the random
variables (31)
a(t)
= t-SN(t)
and (32)
r(t) = SN(t)+l-t
the age and the residual life at t, respectively,where, as always, N(t) denotes the number of the points of the process in the interval (0,t).
In other words, N(t)
is such that SN(t) < t -< SN(t)+I . Since we always assume F(x) to be continuous, the equality actually has no effect above and thus both a(t) and r(t) are positive random variables (with probability one). The distributions of r(t) and a(t) take simple forms for a Poisson process. As a matter of fact, the distribution of r(t) is F(x) itself and that of a(t) is Ft(x ) which is F(x) for x < t and Ft(x ) = 1 for x > t.
We shall develop several
characterizations of Poisson processes based on these simple (but nonetheless
81
surprising) properties of r(t) and a(t).
The reader will recognize the following
consequence of the mentioned properties, which is known in the literature as the
waiting time paradox.
As an example, consider the sequence 0 < S 1 < $2<... as the
successive arrival time epochs of buses at a given station.
Then the intervals
Sj-Sj~ 1 are identically distributed with common distribution function F(x), except that interval when I want to board a bus:
that special bus I was waiting for comes
later after the preceding one as compared with other consecutive buses.
Namely, if
t represents my arrival time then SN(t) is the time epoch of the bus I missed and SN(t)+I is the one that I will board.
SN(t)+I - SN(t) = a ( t )
Hence, this special time interval
+ r(t)
is longer than a "usual" interval Sj - Sj_I, since r(t) above contributes as much as Sj - Sj_ 1 in the case of "the other buses." In order to prove this paradox and to deduce several characterizations from it, we evaluate the distribution of a(t) and r(t), respectively, for an arbitrary renewal process.
In the calculations we shall need the following function
U(x).
For the interval distribution function F(x), this is defined as +co
(33)
U(x) =
where F~nJ(x )r~
~ F(n)(x) n=O
,
denotes the n-fold convolution of F(x).
function.
Notice the special meaning of U(x).
(34)
= E[N(x)]
U(x)
We call U(x) the renewal
We claim that
+ 1 .
Indeed, s i n c e P(N(x)->n) = P(Sn<X ) = F ( n ) ( x ) +oo
E[N(x)] =
, q-co
X nP(N(x)= n)= n=l
X P(N(x)->n) n=l
+o~
X
F(n)(x)
=U(x)
- 1 ,
n=l as c l a i m e d .
R e l a t i o n (34) can be i n t e r p r e t e d
as s a y i n g U(x) r e p r e s e n t s t h e
e x p e c t e d number o f p o i n t s o f t h e p r o c e s s which f a l l
into the interval
[ 0 , x ) , where
SO = 0 i s a l s o counted. It is instructive
to verify
seen t h a t t h e d i s t r i b u t i o n
(34) f o r a P o i s s o n p r o c e s s d i r e c t l y .
o f N(x) i s P o i s s o n w i t h p a r a m e t e r bx.
hand s i d e o f (34) e q u a l s b~ + 1.
We have
Hence, t h e r i g h t
On t h e o t h e r hand, t h e l e f t hand s i d e can be
computed by t h e e x a c t formula (see (5))
82
fx bntn-le-bt -~T~!-dt . 0
=
F (n) (x)
We have x
+~ bntn-le-bt ~ (n-l)! dt= n=l
II(x) = 1 + f 0
We s h o u l d add t h a t t h e c a l c u l a t i o n
x 1 + f b d t = 1 + bx . 0
o f F (n) ( x ) , and t h u s o f U ( x ) , can b e v e r y d i f f i -
c u l t even f o r v e r y s i m p l e f u n c t i o n s F ( x ) . be s i m p l i f i e d
by
I n most c a s e s , however, c a l c u l a t i o n s
can
turning to Fourier transforms.
It is evident from its definition that U(x) is increasing and U(0 +) = i. Hence, we can speak of integrals with respect to U(x) if we establish that U(x) is finite for all x.
This is the content of the next leTmna.
Lemma 4.3.1. Proof:
The r e n e w a l f u n c t i o n U(x) i s f i n i t e
Recall that F(x) is continuous with F(0) = 0.
f o r a l l x. We first prove that, for an
arbitrary y > 0, there is an integer M ~ 1 such that F (M) (y) < I. F(y) is continuous, 0 < F(x0) < I.
F(0) = 0 and F ( + @
Indeed, since
= i, there is an x 0 > 0 such that
It then follows that, for any integer m ~ I,
F (m) (mxo) < 1. Namely, F(x0) < 1 means that P(Sl~X 0) > 0 and
F (m) (mxo) = P(Sm>_mXo) _> P(Sj-Sj_I_>xo,
L<j_<m)
= Fm(x 0) > 0 , where S O = 0.
Hence, for y > 0, we can choose any M ~ 1 such that ~ 0
~ y' for
which it is assured that F(M)(y) < 1. Now, by the convolution formula,
F (n) (y) =
~ F (m) ( y - z ) d F (n-m) (z) 0
-< F (m) (y) fY dF (n-m) (z) = F (m) (y) F (n-m) (y) . 0 Therefore,
i f M i s s u c h t h a t F (M)(y) < 1, t h e n , w r i t i n g n = ~.~ + R, where 0 -< R < M,
we have
F (n) (y) -~ IF(M) (y)] k . Hence,
8:3
u(y) ~ M X [F(M)(Y)] k M k=0 1-F(~0(y)
+co
which completes the proof. We can now calculate the distribution of r(t). Lemma 4.3.2.
For a renewal process (Sn, n~l; F(x)}, the distribution functiol
of the residual life r(t) at t is given by t Rt(x ) = P(r(t)<x)
= 1 - / [1-F(t+x-y)]dU(y) 0
,
where U(y) is the function defined at (33). Proof:
Let us calculate 1 - Rt(x)
= P(r(t)
-> x)
.
We decompose this last probability as .t-co
(35)
P(r(t)>x)
=
~ P(r(t)->x, n=O
N(t)
= n)
.t-oo
P(r(t)>-x, Sn < t -< Sn+l) n=O ÷oo
P(Sn- t+x) n=O
where S O = 0.
We now apply the method often utilized in the past, which yields
t P(Sn
= Sn+ 1 - Sn.
(35) and interchanging s ~ a t i o n
Substituting this last expression into
and integration, we obtain the required expres-
sion for Rt(x ) .
D
Lemma 4.3.3.
With the notations of Lemma 4.3.2, the following recursive
formula is valid: t P(r(t)>-x)
Proof:
,
= 1-F(t+x)
In the formula
+ [ P(r(t-y)>x)dF(y) 0
.
84
(36)
P(r(t)~x)
= / P(r(t)extS 1 = y)dF(y) 0
we cut the integration at y = t.
,
When evaluating this integral for t ~ y, we make
use of the fact that r(t) ~ x means that S 1 ~ t+x.
= y) = I 1
Therefore
if t+xgy
P(r(t)~xlS 1
L0
if t~y
which implies
(37)
f P ( r ( t ) ~ x I S 1 = y ) d F ( y ) = f dF(y) = 1 - F ( t + x ) t t+x
.
On the other hand, if y < t, then the condition S 1 = y means that the origin can be shifted to y and thus r(t) becomes r(t-y) in this new coordinate system.
Hence, we
have +~ f P(r(t)~xlS 1 = y)dF(y) 0
Substituting
this
last
t f P(r(t-y)~x)dF(y) 0
e q u a t i o n and (37) i n t o
as stated in the lemma. kemma 4.3.4.
=
( 3 6 ) , we o b t a i n t h e r e c u r s i v e
This completes the proof.
formula D
With the notations of Lenvaa 4.3.2, the joint distribution of
r(t) and a(t) is given by
(38)
P(r(t)->x, a(t)
t f [l-F(t+x-y)]dU(y) t-z
,
where x -> 0 and 0 < z < t.
Remark.
When z ~ t ,
t h e n t h e e v e n t { a ( t ) < z } becomes t h e s u r e e v e n t and t h u s
{r(t)->x, a ( t ) < z } = {r(t)~>x) . Notice that, for z = t, the formula of Lemraa 4.3.4 indeed reduces to that of Lemma 4.3.2. As a consequence of Lemma 4.3.4, we obtain the distribution of a(t). substituting x = 0 into (38) we obtain
I
(39)
P(a(t)
=
t / [1-FCt-y)ldU(y) t-z 1
if 0 < z < t
ifz_>t.
Namely,
85 Proof of Lemma 4 . 3 . 4 .
We s h a l l f i r s t
evaluate
P(r(t)>-x, a ( t ) < z , N(t) = n l S n = y) ,
n -> 1.
This conditional probability is evidently zero if y does not satisfy the inequalities t-z
On the other hand, if y belongs to the interval (t-z,t), then, by
shifting the origin to y, the requirements r(t)~x, a(t)
Hence, for t > z, +eo
P(r(t)->x, a ( t ) < z )
P(r(t)->x, a ( t ) < z , N(t) = n)
=
n=l =
+~ t ~ f P(r(t)->x, a(t)
=
+~ t ~ f [1-F(t+x-y)]dF(n)(y) n=l t-z t
=
+00
f [1-F(t+x-y)]d{ ~ F(n)(y)} t-z n=l
.
which is the formula (38). The proof is completed. Let us evaluate the integral on the right hand side of (38) for a Poisson process.
Here F(x) = 1-e -bx and, by (34) or by d i r e c t
calculation,
U(x) = l+bx.
Thus, for t > z, t P(r(t)_>x, a(t)~z)=
f
e-b(t+x-Y)bdy
t-z
= e-bX _ e-b(x+z) = e - b X ( l _ e -bx)
.
This conclusion is restated in the following Corollary 4.3.1.
For a Poisson process, r(t) and a(t) are independent.
distribution of r(t) is the same for all t > 0 distribution F(x) = l-e -bx.
The
and coincides with the interval
The distribution function Ft(z) of a(t) equals F(z)
for z < t and equals one for z >- t. We now analyze the extent to which these properties characterize the Poisson processes among renewal processes.
The following results are combinations of those
obtained by K.L. Chung (1972), K.B. Erickson and H. Guess (1973), E. Cinlar and P. Jagers (1973) and P.T. Holmes (1974).
We start with a result due to Erickson
and Guess. Theorem 4.3.1.
Let {Sn, n>_l, F(x)} be a renewal process.
If r(t) and a(t)
are independent for one value t -- t* > 0, then the distribution function of r(t *)
86
coincides with F(x)
(recall that throughout this chapter, F(x) is assumed to be
continuous).
Proof: (40)
Because o f t h e independence o f r ( t * ) and a ( t * ) , P(r(t*)_>x]a(t*)
: P(r(t*)_>x)
.
On the other hand,
P(r(t*)~xla(t*)
= P(r(t*)~x, a(t*)
We can therefore apply (38) and (39) in (40) and (41) to evaluate the distribution of r(t*).
We get t
(42)
f [1-F(t+x-y) ] dU(y) p(r(te)_>x ) = t - z t f [ 1 - F ( t - y ) ] dU(y) t-z
where 0 < z < t.
Since the left hand side does not depend on z, we can simplify
the expression for the distribution of r(t*) by evaluating the limit of the right hand side as z tends to zero.
For this purpose, we estimate the numerator and the
denominator as follows: t
t
f [1-F(t+x-y)]dU(y) _> f { 1 - F [ t + x - ( t - z ) ] } d U ( y ) t-z t-z = [1-F(x+z)][U(t)
- U(t-z)]
;
t t f [1-F(t+x-y)]dU(y) -< f [ 1 - F ( t + x - t ) ] d U ( y ) t- z
t :~z
= [1-F(x)][U(t) - U(t-z)] t f [1-F(t-y)]dU(y)
;
t -> f ( 1 - F [ t - ( t - z ) ] } d U ( y )
t-z
t-z =
[1-F(z)][O(t)
- U(t-z)]
;
and, finally, t
f [1-F(t-y)]dU(y) t-z
t
_< f
dU(y) = U(t) - U ( t - z )
t-z
With these estimates, we get from (42),
.
87
1-F(x+z) < _ P(r(t*)mx)
~
~1-F(x)
"
Since F(x) is continuous, and P(0) = 0, letting z ÷ 0 results in P(r(t*)~x)
= 1-F(x)
,
which was to be proved.
Q
Theorem 4.3.1 shows that the independence of r(t) and a(t) implies a distributional property.
Therefore, if the fact that the distribution of r(t) does not
depend on t characterizes the Poisson process then so does the mentioned independence property.
Before fo~ulating this possibility into an explicit statement,
however, we state a theorem obtained independently by Cinlar and Jagers (1973) and Holmes (1974). Theorem 4.3.2. tinuous.
Let {Sn, n~l; F(x)} be a renewal process, where F(x) is con-
If E[r(t)] is finite, then (Sn} is a Poisson process if, and only if,
E[r(t)] does not depend on t. Corollary 4.3.2.
Each one of the following properties characterizes the
Poisson processes among renewal processes. (i)
the distribution of r(t) does not depend on t and [1-F(x)]dx
< +~
;
0
(ii)
the random variables a(t) and r(t) are independent and
/ [1-~(x~]ax
<
+~
0 proof of Corollary 4.3.2: Poisson processes.
Corollary 4.3.1 implies properties (i) and (ii) for
Conversely, if (i) holds, then the conm~on distribution of r(t)
is evidently the interval distribution F(x).
Hence, (i) reduces to the conditions
of Theorem 4.3.2, ~nd thus (i) is a characteristic property of a Poisson process. Moreover, since, in view of Theorem 4.3.1, (ii) implies (i), property (ii) also characterizes the Poisson processes. Proof of Theorem 4.3.2.:
D
Since the random variable r(t) is nonnegative, we have
from Lemma 1.2.1 that E[r(t)] = / P(r(t)~x)dx = r, say . 0 Therefore, integrating the recursive formula of Lemma 4.5.3 with respect to x, we get
88
+® r = f [1-F(t+x)]dx 0
+~ t + f { f P(r(t-y)_>x)dF(y)}dx 0 0
+co
= f t
t
[l-F(y)]dy
4~o
+ f [f P ( r ( t - y ) _ > x ) d x ] d F ( y ) 0 0
.
However, recognizing that the inner integral in the last term on the right hand side is the expectation of r(t-y), which also equals r, we obtain the equation +co
t
r = f [1-F(y)]dy t
+~
+ r f dF(y) = f [1-F(y)]dy 0 t
+ rF(t)
,
or equivalently (43) r[l-F(t)] = f [l-F(y)]dy , t > 0 . t Since F(x) is assumed to be continuous with F(0) = 0, this last equation is exactly our basic property (P2) in Section 1.5, from which it follows that F(x) is exponential and thus {Sn} is Poisson.
The theorem is established.
While in Theorem 4.3.2, a property of r(t) is assumed for all t, the following theorem, due to Erickson and Guess (1973), leads to a characterization of the Poissor process, in terms of the independence of r(t) and a(t) for one value of t. Theorem 4.3.3.
Let {Sn, n~l; F(x)) be a renewal process, where F(0) = 0,
F(x) is continuous and F(x) > 0 for all x > 0. if, and only if, there is one t = t~ > 0
Then the process {Sn} is Poisson
such that r(t e) and a(t e) are independent.
Erickson m~d Guess give two proofs for this theorem and both proofs would fit into the general approach of the present monograph.
Since we do not want to
include both proofs, we give the outline of one and details of the other. One proof is based on the idea of what we called earlier the method of limit laws.
That is, first generate a sequence t I < t2<..,
from the single te such
that the independence of r(tj) and a(tj) is valid for all j.
Then develop a limit
theorem based on the independence of r(t) and a(t) on a sequence t ÷ +~ limit theorem involves F(x). is carried out as follows.
Then the actual limit will determine F(x).
which This idea
The authors first observe that if r(t ~) and a(t e) are
independent then so are r(tj) and a(tj) for all tj = jt e > 0, j ~ i.
Therefore,
in view of Theorem 4.3.1, the distribution of r(tj) is F(x) for all tj = jt ~, j ~ i, i~e., (44)
P(r(t*j)<x)
= F(x)
, j
= 1,2 .....
On the other hand, a general theorem (which is an easy consequence of the so-called
89
renewal theorem and which we do not prove here; see Feller (1966) , p. 347) states X
(45)
lim P(r(t)<x) = b f [1-F(y)]dy t=+~ 0
for any renewal process even under milder conditions than those assigned about F(x), where +~
1
= f [1-F(y)]dy 0
with the understanding that if the integral on the right hand side in this last formula is infinite then b = 0 and b = +~ when the integral is zero.
It follows
from (43) and (44) that X
(46)
F(x) = b f [l-F(y)]dy, 0
x > 0 ,
from which we see that 0 < b < +~ under our assumptions on F(x).
However, (46) is
again (P2) of Section 1.5 just as in the case of (43). Namely, X
b f [l-F(y)] 0
+~
+~
= b[f .... 0
f'--] x
+~
= b ( ~ - f [1 F ( y ) ] d y ) ~ x
i
Hence, F(x) i s e x p o n e n t i a l and thus (S n} i s P o i s s o n as c l a i m e d . Before t u r n i n g t o t h e more d e t a i l e d p r o o f o f Theorem 4 . 3 . 3 , we remark t h a t a p r o o f s i m i l a r t o t h e one above ( t h a t i s , u t i l i z i n g
t h e renewal theorem and t h e l i m i t
relation (45)) is also applied by Chtmg (1972) to establish a characterization of Poisson processes through distributional properties of r(t) on some sets of t.
We
should also add that three of the papers we referred to (those by Chung, Erickson and Guess, and Cinlar and Jagers) arrive at equation (43) or (46) and conclude that its only solution is the exponential distribution without assuming continuity. We therefore reempbasize here once again that the continuity of F(x) does follow from (43) for all x > 0, but a solution for (43) may have a jump at x = 0 if the continuity of F(x) is not assumed. We now present the second proof of Theorem 4.3.3, which is also due to Erickson and Guess. Proof of Theorem 4.3.3.:
In view of Corollary 4.3.1, we have only to prove that the
conditions of the theorem imply that the process {Sn} is Poisson. Since r(t*) and a(t *) are independent, P ( r ( t ~) > x , a ( t ~) < z) = P ( r ( t * )
>_ x ) P ( a ( t ~) < z ) ,
which, on account of Le~na 4.3.2 and formulas (38) and (39), is equivalent to the validity of the equation
90
(4T)
t t * x ) f [1-F(t-y)dU(y), f [ 1 - F ( t + x - y ) ] d U ( y ) = Rt( t-z
t-z
where 0 < z < t
and R[(x) = 1-Rt(x ) as given i n Len~na 4 . 3 . 2 .
We now c l a i m t h a t the
above e q u a t i o n can h o l d o n l y i f (48)
1 - F ( t + x - y ) = Rt(x ) [ 1 - F ( t - y ) ]
where x > 0 and t-z ~ y ~ t, except perhaps a set of y's, the U-measffre of which is zero.
Namely,
continuous,
if (48) fails for some x,t and y, then, since F is assumed to be
it fails on a whole interval of y; if the left hand side was originally
larger than the right hand one, then it remains larger on an interval of y and vice versa.
In ~other words, the failure of (48) for one
triplet (t,x,y) implies that
there is an interval
(a,b), a < b, such that (48) fails for all (t,x,y*), where
a ~ y* ~ b.
for example, that the left hand side is larger.
(49)
Assume,
Then
1-F(t+x-y*) > Rt(x ) [ 1 - F ( t - y * ) ]
for all y*~I = [a,b].
However, U(1) is positive for any interval in view of our
assumptions that F(0) = 0 and F(x) > 0 for all x > 0. (47) if we choose a,b and z appropriately. x > 0 and all t-z ~ y ~ t
Therefore,
In other words,
(49) contradicts
(48) holds for all
(the continuity assumption leads to all y rather than to
almost all y with respect to U).
If we change the variable y in (48) to t-y = s,
we get
1-F(x+s)
=
at(x) [1-F(s)]
.
Setting s = 0 yields Rt(x ) = l-F(x).
Hence, we arrived at the lack of memory equa-
tion
1-F(x+s)
=
[l-F(x)]
[l-~(s)]
where x ~ 0 and 0 ~ s ~ z
,
with some z > 0.
The basic assumptions on F(x) now
imply that the only solution of this equation is the exponential distribution, which was to be proved. The conclusions of Theore~m 4.3.3 and 4.3.1 and Corollary 4.3.1 can be combined in several different ways.
For example, if for a renewal process {Sn, n_>l; F(x)}~
r(t) and a(t) are independent for one value of t > 0, then each of the following statements holds:
(i)
r(t) and a(t) are independent for all t -> 0; (ii) the distri-
bution function of r(t) is F(x) for all t >- 0;
(iii) E(r(t)) = E(SI) for all t -> 0
and (iv) the distribution function of a(t) equals F(x) for all x < t (recall our
91
basic assumptions on F(x), which are permanent in this chapter).
However, if we
replace "all t ~ 0" in (ii) - (iv) by "one t > 0," it is not known whether the conclusion that F(x) is exponential is still valid.
It would, for example, be inter-
esting to prove that either there is no other process besides the Poisson process for which E[r(t)] = E(SI) (assumed finite), or to give an example for a renewal process when E[r(t)] = E(SI) for some t > 0 but Elf(t)] # E(SI) for some other values of t. We have not yet discussed characterizations based on a(t) alone.
However,
with completely similar arguments as for r(t), we can conclude that an appropriate specification of the distribution of a(t) or of E[a(t)] characterizes the Poisson process among renewal processes.
Therefore, we state without proof the corres-
ponding theorems on a(t). Theorem 4.3.4.
Let {Sn, nal; F(x)} be a renewal process with continuous in-
terval distribution function F(x).
Then, if r(t) and a(t) are independent for a
fixed t > 0, the distribution of a(t) is F(x) for x < t.
Theorem 4.3.5.
With the notations of the preceding theorem, define
F(x)
if 0 -< x < t
1
ifx>t
F(x;t) = .
Then the process {Sn} is Poisson if, and only if, t E[a(t)] = / xdF(x;t) , 0
for all t > 0 .
For a further extension, when S 1 may have a distribution different from the intervals Sj - Sj_ 1 ~ 0, j ~ i, and S 1
may even be negative, see F.L. Garagorry
and M. Ahsanullah (1977). Cinlar and Jagers (1973) deduce from Theorem 4.3.5 (for which they also give a detailed proof) the following result. Theorem 4.3.6. tinuous.
(50)
Let {Sn, n_>l; F(x)} be a renewal process, where F(x) is con-
Then this process is Poisson if, and only if,
E [ a ( t ) IN(t) = n) = E ( S I [ N ( t )
= n)
for all t > 0 and all n -> I.
We separate the "only if' part of the theorem and we actually prove the following result.
92
Theorem 4.3.7.
For a Poisson process,
the conditional distribution of S 1 and
S n, given N(t) = n, n m I, is the same as the distribution of the minimum and maximum, respectively, interval (0,t).
of n independent random variables uniformly distributed on the
That is, x n
P(Sl<XEN(t)
=n)
=
I
(I-~)
,
and
X n
P(Sn
,
where n ~ 1 and 0 < x,y < t. Remark.
A consequence of Theorem 4.3.7 is that, for a Poisson process, the
conditional distributions
of S 1 and a(t), given N(t) = n m i, coincide.
Hence, one
part of Theorem 4.3.6 follows from Theorem 4.3.7. Proof of Theorem 4.3.7.:
Let 0 <x < t.
By definition
P(SI~X , N(t) = n)
i51)
PiSl~XIN(t)
= n) =
P(N(t) = n)
However, {SlmX, Nit) = n} = {N(x) = 0, N(A) = n} ,
where A is the interval (x,t).
By Theorem 4.2.3, N(x) and N(A) are independent and
the distribution of NIA) is the same as that of Nit-x).
Hence, on account of (i)
and (51) PiSlmX[N(t ) = n) = e-bX[b(t-x)]ne-b(t-x) (bt) ne-bt (~)n =
_ x n , =
il
~9
which proves our claim on S 1 . Arguing similarly, we get P(Sn
= n) = P(N(x) = n)P(N(t-x) P i N ( t ) = n)
w h i c h i n d e e d becomes ( x / t ) n a f t e r N(x), N(t-x) and N(t).
= 02
the substitution
The proof is completed.
o f (1) i n t o t h e d i s t r i b u t i o n
of D
We add, without going into detail, that, given N(t) = n, n ~ i, the joint dis tribution of the points SI,S2,...,S n in a Poisson process coincides with the joint distribution of the n order statistics of n independent random variables which are
93
uniformly distributed on the interval (0,t). Its proof is a matter of easy calculation similar to those presented in the case of Theorem 4.3.7. Proof of 7he0rem 4.3.6.:
In view of Theorem 4.3.7 (see the Remark following the
statement of Theorem 4.3.7), we have to prove that if (50) holds for all t > 0 and all n ~ I, then the process {Sn} is Poisson. Let, for n a 0,
{~
(t)
a n(t)
=
if
N(t)
= n
otherwise
.
Then +co
E[a(t)] = E[a0(t)] +
~ E[an(t)] n=l +eo
= t[1-F(t)]
+
~ E[an(t)] n=[
where in the last term on the right hand side we can utilize (50).
We get
4-00
(51)
!IE(SI,n ) ,
E[a(t)] = t[l-F(t)] + n
where
Sl'n = n = 1,2, ....
if N(t) = n
0
otherwise ,
However,
+~ (52)
S
+~
t
n!iE(Sl,n ) = E(n!ISI, n) = /0 xdF(x) ,
because ~ i S l , n
= S 1 if N(t) ~ 1
and the previous sum is zero if N(t) = 0.
is, the sum of S1, n i s S 1 w h e n e v e r S 1 < t .
That
Substituting (52) into (51), we obtain
that E[a(t)] where F(x;t)
t = f xdF(x;t) 0 was i n t r o d u c e d
, t > 0 , i n Theorem 4 . 3 . 5 .
Theorem 4.3.5 implies that the process
is Poisson. It is instructive to compare the meaning of Theorem 4.3.6 with results in Section 3.4. E
In particular, Lemma 3.4.1 and Theorem 3.4.1 imply that the sequence
uniquely determines the population distribution.
Now, Theorem 4.3.7 indicates n:n that E[a(t) IN(t) = n] is related to En: n in the sense that if the process in ques-
94
tion is Poisson then it is exactly t - En:n, where the population distribution is uniform on the interval (0,t). However, for non-Poissonian processes, the notation E[a(t) IN(t) = n] : t - En: n
is not justified because the distribution of Sn, given
N(t) = n, is not that of the maximum of independent and identically distributed random variables.
But if we redefine En: n as the expected value of the maximum of a
set YI,Y2,...,Y of (not necessari]y independent and/or identically distributed) random variables, then, of course, the expected value of act), given N(t) = n, is t - En: n.
Namely, given N(t) = n, select one of the points SI,S2,...,Sn at random
and call it YI"
Then select a second point and call it Y2' etc.
the maximum of the Y's.
Evidently, Sn is
Therefore, Theorem 4.3.6 actually asserts that En: n char-
acterizes the structure of the sequence Yj as well as the interval distribution F(x) under the assumption (50). This leads to the following general question: under what general dependence structure does the sequence En:n, n ~ I, characterize a given model? areas.
An answer to this question would be interesting in two general
One could be within the theory of renewal processes (when (50) is approp-
riately replaced to lead to a characterization of a process different from the Poisson processes).
Another area is to assume that the multivariate distribution of
the sequence YI,Y2,... belongs to a given family.
As an indication in regard to
the types of results that can be expected as well as to the difficulty of such an attempt, see Section 5.3, in particular Theorems 5.3.2 and 5.3.3, which are stated in the form of minima rather than maxima. We do not discuss further the pre-1973 results on the Poisson process; the reader is referred to the comprehensive discussions by D.J. Daley and D. Vere-Jones (1972) and M. Westcott (1973). The following type of extension of part (i) of CorOllary 4.3.2 was initiated by
S. Kotz and N.L. Johnson (!974).
define r(t) as before.
Let {Sn, n~l; F(x)} be a renewal process and
However, let us now choose the point t > 0 with some speci-
fic probability distribution K(u).
Our interest is again to relate the distribution
of r(t) to F(x) and to seek conditions under which a characterization of the Poisson process is obtained.
Kotz and Johnson show that if t is selected conditionally
uniformly then the distribution of r(t) is still F(x) for the Poisson process and this property does characterize the Poisson process among all renewal processes. However, the uniformity assumption is not necessary here.
V. Isham, D.N. Shanbhag
and M. Westcott (1975) obtained the following general result. Theorem 4.3.8.
Let {Sn, n~l; F(x)} be a renewal process with E[S I) < +~.
We assume that F(0) = 0 and that F(x) is continuous.
Let t be a nonnegative random
variable with distribution function K(u) = P(t
If the distribution function V(y) = PCr(t) < y)
equals F(y) for one fixed K(u) then the process is Poisson.
95
Conversely,
for a Poisson process, V(y) = F(y) for an arbitrary distribution
K(u). Remark.
In the original statement, the continuity of F(x) is not required.
We included it here only for the sake of conformity with our previous assumptions. Outline of Proof:
Since V(y) = F(y), the random variables N(t+x) - N(t) and N(x)
are identically distributed.
Consequently, their expectations are also equal and
thus, on account of (34),
(53)
U(x)
-
1 = E[N(t+x)
- N(t)]
= E[U(t+x)
- U(t)]
where, in the last step, we interchanged taking expectations with respect to {Sn} and t, respectively.
Now, by the above equation and by the method of limit laws,
we can easily deduce that U(x) - 1 = bx with some constant b > 0.
Namely, from
(53), E[U(t+x+y)
- U(t+~)]
= E[U(t+x+y)
- E[U(t+y)
- U(t)]
- U(t)]
= U(x+y)
- U(y)
.
Hence, if t I and t 2 are independent random variables with the same distribution as t then the substitution y = t 2 and t = t I above and (53) yield
E[U(tl+t2+x)
- U(tl+t2)
] = E[U(x+t2)
- U(t2)
] : U(x)
-
1
.
Similarly, we obtain by induction that if tl,t 2 .... ,tn are independent copies of t and if Tn = tl+t2+...+tn , then
(54)
E[U(Tn+X)
- U(Tn)]
= U(x)
- 1
for all n ~ 1 and all x > 0.
It therefore requires a limit theorem for the left
hand side to determine U(x).
Basically the renewal theorem, which we have referred
to earlier in connection with (45) (Feller (1966), p. 347), guarantees that the left hand side of (54) converges to x/E, where E = E(SI). U(x) = l+x/E.
Nence, by (54),
The inverse Fourier transfom of l+x/E, in view of (33), yields that
F(x) is exponential and thus the process {Sn} is Poisson. The converse part of the statement is a matter of easy calculation, hence its details are omitted.
4.4.
Rarefactions,
D
9eometric compoundin9 and damage models.
As the title of the present section suggests, we shall relate three different areas of research which were developed independently of each other in the literature.
96
Although our aim is to emphasize characterizations in connection with these models, we also want to emphasize that the corresponding theories in other aspects as well are strongly related and, in most cases, coincide. We first describe the three models mentioned in the title.
Rarefactions of renewal processes.
Let {Sn, n~l; F(x)} be a renewal process.
Suppose each point Sn is erased with probability factor I/q.
l-q. Expand the time scale by a
We then obtain a new point process {S~, n~l}.
The above procedure
will be called a rarefaction, where the erasure of the points Sn will be assumed to be independent of the process (Sn} and also the decisions on erasing distinct points Sn will be made independently of each other. rarefaction procedure; hence Rq{S n} = {S~}. 0 < q < i.
We shall use the notation Rq for a Evidently,the only case of interest is
Since we shall show that Rq{Sn} also is a renewal process, we can also
speak of the interval distribution
Fe(x) of {S*}; we also use the notation n
RqF(X) = Fe(x) to reflect this fact.
Geometric compounding.
Let TI,T2,... be independent and identically distri-
buted random variables with common distribution function F(x).
Let m(p) be a
geometric variable with parameter 0 < p < i, that is, P(m(p) = k) = pk-l(l-p), k = 1,2,....
We assume that m(p) is independent of the T's.
Set
qS ~ = TI+T2+...+Tm(p) , q = l-p , and F*(x) for the distribution function of S *. F(x) and Fe(x),
We are interested in the relation of
and, in particular, in the theory of limiting distribution of S e
as p + !. The fact that we used identical notations to those in the first model is not accidental:
the meaning of the concepts will become identical.
Damage models.
In the description of the model, we shall use one of the
original examples from zoology, which induced the early investigations by C.R. Rao and H. Rubin ~1964). eggs.
Consider the time epochs 0 ~ S 1 < $2<... when a bird lays
A squirrel runs around and, at each time, with the same probability, steals
an egg from the nest.
Therefore, if the ranger did not see the actual epochs when
the eggs were laid nor when some eggs were stolen, then the ranger observes "a damaged number" of eggs at any given time.
The interest is to compare the distribu-
tion of the number of eggs actually laid with the distribution of the number of eggs in the nest,with the aim tO characterize these distributions under simple assumptions. The strong interrelation amongthe three models described above is evident. In the damage model, we have a point process (Sn}, and the eggs remaining are a rarefied set of the S . The difference is that we did not assume that the S n
a renewal process.
n
form
The relation of the geometric compounding model to the rare-
faction model is easily seen if we look at S~ in the rarefied sequence.
It is,
apart from the change of time scale, the sum of a random number of intervals
97 Sj - Sj_ I, namely, S~ = Skq , where k is the first index which was not erased. It is straightforward to compute the distribution of k which turns out to be geometric. model.
Therefore, a result in one model can be translated into one in another
What is surprising in the development of these three theories is that the
initial questions and their solutions, although posed without reference to each other, were also related.
We shall return to this after some basic results on the
rarefaction model. The rarefaction model was first introduced in the literature by A. R~nyi (1956).
All basic results which follow are due to him.
Theorem 4.4.].
For a renewal process {Sn, n->l; F(x)}, the rarefied process
R { S n] = (St} is also a renewal process.
Its interval distribution function F*(x)
can be computed as follows +co
(55)
F*(x) -- q ~ F (k)(~)pk-lr k=l
where 0 < q < 1, p = 1-q and F (k) i s the k - f o l d c o n v o l u t i o n of F.
Furthermore,
E(Sl) = E(S[). Proof:
Let us first evaluate the distribution of S~.
notation F*(x) for this distribution function.
From the start, we use the
By the total probability rule
+oo
F*(x) =
[ P(S[<xlS [ = qSk)P(S ~ = qSk) . k=l
Now, the c o n d i t i o n S[ = qSk means t h a t S1,$2, . . . . Sk_ 1 were e r a s e d and Sk i s p r e s e r v e d . Since the d e l e t i o n s a r e i n d e p e n d e n t and i n d e p e n d e n t o f the o r i g i n a l proc e s s , the last formula becomes F*(x) =
+co
x, k-I
~ P(S k < ~Jp k=l
4<0
~ F(k) x
q = q
(q) p
k-I
k=l
as claimed. Let us now c a l c u l a t e the j o i n t d i s t r i b u t i o n
o f S~ and S~ - S~.
t o t a l p r o b a b i l i t y r u l e a g a i n , we g e t P(S~<x, S~-S~
~ P(S~<x, S~-S~
where Bk, m = {S~ = qSk, S~ = qSm},
1 -< k < m .
Since k-i P(Bk,m) = p
m-k-I qP
q = p
m-2 2 q
Applying the
98
the independence assumptions reduce the above formula to +ce
s
-q
+oo
y, m-2 2 Z P(Sk < q, Sm-Sk < ~JP q
}~
k=l m=k+l +co
+m
q2 ~ Z F (k) (q) F(m-k) Lq)P ~y" m-2 k=l m=k+l
=
=q
(q)pk-1 k=l
(m-k) i m-k-1 m=k+l
= F* (x) F* (y)
.
Continuing this argument, induction over n yields that SI,S 2 . *-SI,..., e S*-S*n n-I are independent with cor~aon distribution function F *. Rq{S n)
That is, the rarefied process
is a renewal process, whose interval distribution function has the form
(55).
Moreover, if E(SI) < + %
then
+oo
E(s~) = ~ E(s~ls~
= qsk)P(S ~ = qS k)
k=l
+co
+c~
= q ~ E(Sk)pk-lq _- E(SI)q2 Z kpk-1 k=l k=l which indeed equals E(SI). E(SI) = +~.
On the other hand, since E(S~) ~ qE(SI), E(S~) = +~ if
This completes the proof.
Corollary 4.4.].
If {Sn, n~l; F(x)} is a Poisson process then so is q { S n)
with RqF(X) = F(x). Proof:
If F(x) = 1 - e -bx , then x ~k k-i -by F (k)(x) = f o y e dy 0
(k-1)~
x > 0, k > 1 '
-
-
"
Hence, substituting into the formula (55), we have
bkyk- 1e-bY pk-11 dy (k-l) r
F* (x) = q ~ / q I ~ 0 ~k=l
bYli 0
'k=l
(k-l)!
-bx =
1
-
e
=
F(x)
.
J
99 Our claim now follows from Theorem 4.4.1. Let us compute the characteristic function @* (s) of F*(x) from formula (55). If @(s) denotes the characteristic function of F(x), then
(56)
~*(s) = q F c k ( q s ) p k-1 = q~(qs) k=l 1-p¢(qs)
'
which we also write in the form (57)
q~,](s---y- I
= I
1 ~%(--~-
- 1)
.
When we want to emphasize the relation of @*(s) to @(s), we use the notation Rq¢ = @*, just as we did for the interval distributions. We now show that the relation RqF(X) = F(x) characterizes the Poisson pro,-:
cesses. Theorem 4.4.2.
If, for a renewal process (Sn, nel; F(x)} such that E(SI) < + %
RqF(X) = F(x) for one 0 < q < i, then the process is Poisson. Proof:
We use the method of limit laws.
Rq@ = ¢.
First, since RqF(X) = F(x), we have
Next, we show that
(58a) Rq(RqF(X)) = R 2F(x) , q and, in general,
(58)
Rql(Rq2F(x))
= Rqlq2F(~)
.
In other words, successive rarefactions can be replaced by a single rarefaction pro~ cedure, its parameter being obtained as qlq2.
As a matter of fact, a repeated
application of (57) immediately yields (58), from which we get (58a) by choosing ql = q2"
Now, since RqF(X) = F(x), (58a) implies
R 2F(x)
= Rq(RqF(X))
= RqF(X)
= F(x)
,
q and, by induction over n, we get from (58) that Rq(n) F(x) = F(x) , q(n) = qn, n >- 1 . Hence, from (56)
1 O0
(59)
%(s) =
qn%(qns) n ~ 1 . l_(l_qn)@(qns) '
Letting n + +~, qn + 0, and thus (59) yields (we write u = qn)
uO (us) ~(S) = liln 1-~(us) + u~(us) = lira u=0
u=0 ~(0)
_
-~'(0)s + ~(0)
where B = E(SI).
~ (us) l-~(us)
u
+ ~(us)
1
1-isB '
This is the familiar form of the characteristic function of the
exponential distribution function F(x) = 1-e -b×, x >- 0 and b = I/B.
The theorem is
established. Let us turn to the geometric compounding model.
We recognize in (55) that
the distribution of S* is exactly F*(x) of (55) and we thus have the following consequence of Theorem 4.4.2. Coro]lary 4.4.2.
If, in a geometric compounding model, the distribution func
tion of S* coincides with the distribution of T 1 for one value 0 < p < I, then this con~non distribution is exponential. The relation between the rarefaction model and the model of geometric compounding was first discussed by I.N. Kovalenko (1965) and further analyzed by T. Sz&ntai (1971).
Direct proofs for Corollary 4.4.2 were given independently by
B. Arnold (1973) and Azlarov et al. (1972).
We do not intend to make the biblio-
graphy on these topics complete; the reader will find a complete bibliography and further results on these two models in the paper by Daley and Vere-Jones (1972), in the extensive investigations by J. Mogyorodi (1969-1973), and in the works by D. Szasz (1976), D. Szynal (1976) and Serfozo (1977).
Although most of the results
in these papers are formulated in terms of limit theorems, they reduce to characterizations by the method of limit laws. We finally turn to the damage models.
Our aim again is limited here because
the Proceedings of the International Conference on Characterizations, held in Calgary, Canada, in 1974 (it is published as Vol. 3 in the series of Statistical Distributions in Scientific Work, Ed.: G.P. Patil, S. Kotz and J.K. Ord, D. Reidel Publishing Company, Dordrecht-Boston, 1975) devotes several papers to this subject area.
We mainly want to point out the relation of some characterization theorems
based on damage models to rarefaction models. We first introduce notations. random variables, where M ~ N
Let N and M be nonnegative integer valued
(in our earlier example, N is the number of laid
eggs and M is the number of unstolen ones).
Our assumption on the actions of the
101
squirrel is that, given N = n, the distribution of M is binomial.
Now, if we
view N as the number of eggs laid during the period (0,t), say, and SI<S2<...<SN are the actual times when the eggs are laid, then the above distributional assumption on M corresponds to the fact that each S. has the same probability of being J "erased" and the erasures occur independently of each other. The question is now that what additional assumption is required to guarantee that N be Poisson?
Within
the family of distributions for N which can be obtained as the distribution of N(t) in a renewal process, Theorem 4.4.2 provides an answer to this question.
However,
it turns out that a simple condition characterizes the Poisson distribution among all distributions for N.
The following result is due to C.R. Rao and H. Rubin
(1964). Theorem 4 . 4 . 3 . Assume t h a t , 0 < p < 1.
(60)
Let M ~ N be n o n n e g a t i v e i n t e g e r v a l u e d random v a r i a b l e s .
given N = n, the d i s t r i b u t i o n Then N i s P o i s s o n i f ,
o f M i s b i n o m i a l w i t h p a r a m e t e r s n and
and o n l y i f ,
P(M = m) = P(M = mlM = N) .
Remark.
From the description preceding the statement of the theorem it is
clear that a characterization of a Poisson distribution for N i~lies a characterization of the Poisson process by a rarefaction procedure.
Therefore, the most
general model is a damage model out of the three models of the present section.
The
possibility of characterizations of Poisson processes by rarefaction, through utilizing known results for damage models was pointed out by R.C. Srivastava (1971), but this relation was not exploited in any other papers. We do not prove Theorem 4.4.3.
Instead, the reader is referred to the paper
by G.P. Patil and M.V. Ratnaparkhi (1975), who follow the simplest known method of proof, due to D.N. Shanbhag (1974).
Patil and Ratnaparkhi also give several ex-
tensions of Theorem 4.4.3, and they give a good collection of references.
See
also the paper by R.C. Srivastava and J. Singh (1975) in the same Proceedings. V.I. Oliker and J. Singh (1977) make some advance on solving the equation (1.2) of the paper by Srivastava and Singh (1975), but a complete solution is still lacking. Other significant contributions to characterizations by damage models, besides those in the
Calgary Proceedings
mentioned earlier, are those by S. Talwalker
(1970), R.C. Srivastava and A.B.L. Srivastava (1970), D.N. Shanbhag (1977) and D,N. Shanbhag and J. Pararetos (1977).
Talwalker gives a bivariate variant of the
Rao-Rubin theorem (Theorem 4.4.3), while Srivastava and Srivastava obtain a kind of converse to Theorem 4.4.3.
They show that if N is a Poisson variable with parameter
and if the conditional distribution of M, given N = n, does not depend on ~, then this latter (conditional) distribution is binomial if, and only if, the Rao-Rubin condition (60) holds.
Shanbhag unifies the results of Rao and Rubin, Talwalker
102
and Srivastava and Srivastava.
Implicitly, through his method of proof, Shanbhag
also relates renewal theory and damage models.
Shanbhag and Panaretos carry
the
method of Shanbhag (1977) further and show that if the parameter p in Theorem 4.4.3 is allowed to depend on n then its conclusion does not remain valid.
They also
show that the major assumptions of Srivastava and Srivastava cannot be relaxed. Two simple examples of Shanbhag and Pararetos disprove two conjectures raised at the Calgary Conference on Characterizations.
CHAPTER 5 CHARACTERIZATIONS OF MULTIVARIATE EXPONENTIAL DISTRIBUTIONS 5.1.
Introduction The development of multivariate exponential distributions got underway very
late mainly due to the following two factors:
a) the rather tardy development of
univariate exponential models as indicated in Chapter I, and b) the overwhelming predominance of "the" multivariate normal distribution in both statistical practice and theory.
The universal belief in correlation and regression techniques, coupled with
the unjustified popularity of univariate normal distributions were responsible for the unchallenged position of "the" multivariate distribution for over 80 years since this distribution involves univariate normal marginals and utilizes the correlation coefficient as the dependence mechanism in its structure. It is interesting to note that bivariate (and multivariate) extensions related to exponential distributions or their transforms were first developed for extremevalue distributions rather than for the simpler exponential distributions.
These
extensions appeared in J. Geffroy's lengthy papers (1958, 1959) and in J. Tiago de Oliveira's somewhat shorter contribution (1958) as well as in Sibuya's work in early 1960.
These were all motivated by Gumbel's famous treatise (1953, 1954, 1958)
on extreme-value distributions in which the rudiments of bivariate and multivariate extensions of extreme value distributions were presented as well.
Unfortunately,
the above mentioned contributions--which appeard in European and Japanese journals-went unnoticed for some time.
In fact, the characterization theorems for multi-
variate extreme value distributions constitute one of the most complete and developed areas of multivariate characterizations. Chapter 5 of J. Galambos'
These are discussed in detail in
(1978) book and are therefore not presented in this mono-
graph. It was only in 1960 that an authoritative paper, specifically devoted to exponential multivariate distributions was published in a journal with wide circulation.
E. Gumbel (1960) introduced a number of bivariate exponential distributions
mainly as a warning against overreliance on multivariate normal techniques, by indicating that many properties of bivariate exponential distributions are quite different from those of the bivariate normal. Gumbel's contribution was followed by Freund (1961), but it was the paper by Marshall and Olkin (1967a) that initiated the intensive investigations into the realm of multivariate exponential distributions that we are witnessing today.
Since
our subject matter is essentially in its infancy it would seem inappropriate to attempt to unify the results obtained.
We shall therefore confine ourselves to a
104
critical summary of the characterization theorems available thus far. here is somewhat different from our aims in preceding chapters.
Our purpose
We would like to
stimulate additional work by focusing attention on the difficulties and ambiguities involved in extending univariate concepts into the multivariate domain. Numerous devices and algorithms for generating multivariate distributions have been developed over the past few decades.
Some of these are briefly sketched below
without commenting on their relative merits. i)
Chronologically, the earliest method was to formally generalize a system
or equation defining a univariate distribution to the multivariate case.
The numer-
ous approaches to extending Pearson's system of distributions, e.g. van Uven (1947), Sagrista (1962), Steyn (1960), can be cited as examples of this approach. 2)
Another method is to define a multivariate distribution by explicitly
specifying the mathematical relations between the joint distribution and its marginals.
This was initiated by Fr~chet (1951), and more recently extended by Morgen-
stern (1956), Gumbel (1960), Farlie (1960) and Johnson and gotz (1975@ among other~ 3)
Still another approach is to postulate a specified multivariate form for
the density by reproducing in the multivariate setting the functional form of a particular univariate family of densities.
This includes linear-exponential type
distributions originated by Bildikar and Patil (1968), quadratic-exponential type by Day (1968) and multivariate 0-generalized distributions of Goodman and Kotz (1973). The most d i r e c t approach--modelling--is p r e s e n t e d i n the works of Freund
4)
(1961) and Marshall and Olkin (1967a, 1967b), whereby a m u l t i v a r i a t e d i s t r i b u t i o n is obtained by i n d i c a t i n g a s p e c i f i c s t o c h a s t i c model comprising a system of s e v e r a l "components" r e p r e s e n t i n g the random v a r i a b l e s under c o n s i d e r a t i o n .
The multinomial
d i s t r i b u t i o n i s a primary " g e n e r i c " exan~le o f such a c o n s t r u c t i o n . 5)
F i n a l l y an even sounder aqproach i s to extend ( h o p e f u l l y uniquely) a
meaningful c h a r a c t e r i z i n g p r o p e r t y of the u n i v a r i a t e d i s t r i b u t i o n to the m u l t i v a r i a t e case.
I t i s t h e r e f o r e the a u t h o r s ' strong b e l i e f t h a t c h a r a c t e r i z a t i o n theorems
should p l a y a d e c i s i v e r o l e in determining the m u l t i v a r i a t e extensions o f u n i v a r i a t e distributions.
In t h i s connection we should be mindful of the f a c t t h a t ,
in gener-
a l , t h e r e are i n f i n i t e l y many m u l t i v a r i a t e d i s t r i b u t i o n s which have s p e c i f i e d onedimensional marginal d i s t r i b u t i o n s ; t h i s o b s e r v a t i o n i s e s p e c i a l l y v a l i d i n both the normal and (in s p i t e o f some misconceptions to the contrary) ex p o n en t i al cases. 5.2.
B i v a r i a t e extensions based on LMP In previous chapters we have shown the e q u i v al en ce between v ar i o u s c h a r a c t e r -
i z i n g p r o p e r t i e s o f the u n i v a r i a t e e x p o n e n ti a l d i s t r i b u t i o n and i t s monotone t r a n s forms.
These e q u i v a l e n t p r o p e r t i e s w i l l serve as the d ep ar t u r e p o i n t s f o r b u i l d i n g
a p p r o p r i a t e m u l t i v a r i a t e d i s t r i b u t i o n s and at the same time may provide a c h a r a c t e r i z i n g theorem.
As we s h a l l see, the e q u i v a l e n t p r o p e r t i e s are g e n e r a l i z e d and
105
extended in different directions and result in distinct classes of multivariate exponential distributions. The first basic characterizing property distribution,
(P3) of the univariate exvonential
that of the Lack of Memory Property (LMP) was extended in the pio-
neering paper of ~fmrshall and Olkin (1967a) and that of Basu and Block (1974). The univariate statement
(1)
(in the notation of the previous chapters)
G(s+t) = G(s)G(t)
for all s -> 0 and t -> 0 can be extended in various ways. (2)
One obvious extension is
G(sl+tl,S2+t2 ) = G(Sl,S2)G(tl,t2)
for all s I > 0, s 2 > 0, t I > 0 and t 2 > 0. Unfortunately
it is too strong a generalization
Theorem 5.2.].
as the following theorem indicates:
The only solution of equation
(2) is
G(s,t) = exp{-(hlS+h2t )} for some h I > 0 and h 2 > O.
(In other words, we obtain the "trivial" bivariate ex-
vonential distribution which is a product of its marginals,
thus corresponding
to a
device consisting of two independent components .) Proof:
Setting s 2 = t 2 = 0 in (2) we immediately obtain Gl(Sl+t2 ) - G(Sl+tl,0 ) = Gl(Sl)Gl(tl)
for all s I, t I -> 0, which as we know from Chapter 1 implies that Gl(S ) = exp(-hlS) for some h I > 0.
Analogously, we have G2(t ) - G(0,t) = exp(-12t)
for some h 2 > 0.
Now choosing s 2 = t I = 0, we have G(Sl,t2) = Gl(Sl)G2(t2) as claimed. We are therefore guided to relax the requirement for the validity of (2) to a smaller set of values of si,ti, i = 1,2. A physically meaningful relaxation of (2) is the following (2a)
G(Sl+t , s2+t ) = G(Sl,S2)G(t,t )
for all s I >_ 0, s 2 -> 0 and t > 0. Requirement
(2a) states that if two components in a two-component system
reached ages e ~ e e d i n g
s I and s2, then the probability that both components are
still functioning t time units later is the same as if both were new. two random variables X and Y (2a) can be written as: (2b)
P(X>Sl+t,Y>s2+tIX>Sl,Y>s2 ) = P(X>t, Y>t)
In terms of
106
for all s I ~ 0, s 2 ~ 0
and t ~ 0.
The following remarkable result is due to
~ r s h a l l and Olkin (1967a): Theorem 5.2.2.
The unique solution of equation (2b) (or that of (2a)) under
the assumption that the marginal distributions of X and Y are exponential is: (3)
G(s,t) =
P(X>s, Y > t ) =
where 11,12 and 112 Remark:
e~p{-lls-12t-l12m~\(s,t)}
are nonnegative.
This is the bivariate exponential distribution of Marshall and Olkin
(which can also be derived, as noted above, from several distinct plausible modelling considerations, some of which are described below). Proof:
Setting s I = s 2 into (2a) yields G(s+t,s+t) = G(s,s)G(t,t)
which implies that G(s,s) = exp(-~s) for some ~ > 0.
Therefore
G(s+t,t) = G(s,0)e -6t . Now since the marginals of X and Y are assumed to be exponential we have G(s+t,t) = exp(- (OlS+~t))
for some 01 ~ 0
which means that (since t > 0) (4a)
G(x,y) = exp(-6Y-Ol(X-y))
for x ~ y .
Analogously we obtain that (4b)
G(x,y) = exp(-6x-O2(Y-X))
for x ~ y
(for some 02 ~ 0). (4a) and (4b) together define a bivariate distribution for all x~y ~ 0.
Since
G(x,y) is monotonically decreasing in y, we have 6 ~ 01, hence X 2 = 6-01 is nonnegative.
Similarly, since G(x,y) is decreasing in x, 6 ~ 02, thus 11 = 6-02 is
also nonnegative. 01+02 ~ ~.
Now let 112 ~ 01+02-~.
To show that 112 ~ 0 we must show that
However, the bivariate distribution as given by (4a) and (4b) generates
the univariate distribution given by the distribution function F(x)
= F(x,x)
= 1-G(x,x)
+ Cl(X)
-elx where Gl(X) = e
+ C2(x)
-02x
, G2(x) = e
and G(x,x) is given by (4a) and (4b) with y = x.
Thus -OlX f(x) = F'(x) = 01e for all x.
-02x - ~e -~x ~ 0 + 02e
(Here we have essentially utilized the fact that for any two points
(xl,Yl) and (x2,Y2) a bivariate survival function must satisfy
107
G(Xl,Y I) + G(x2,Y 2) - G(xl,Y 2) - G(x2,Yl) ~ 0 .) Letting x ÷ 0 we observe that G 1 + @2 -~ > 0 which shows that XI2 is nonnegative. Substituting now ~ = h I + h 2 + ~12 ' @I = ~I + h12 and @2 = h2 + XI2 into (4a) and (4b) we obtain that the unique solution for equstion (2a) is indeed the distribution given by (3), the ~rshall-Olkin bivariate exponential distribution.
Observe that
the nonnegativity of hl2 is a consequence of the "rectangular positivity" of any bivariate distribution which is not necessarily a consequence of the monotonicity of G(.) in each one of the variables separately. The bivariate exponential distribution of Marshall and Olkin is not absolutely continuous, in fact it involves a singular component Gs(x,y ) . Namely,
Theorem 5.2.3.
The survival G(x,y) given by (3) can be written as the mix-
ture
(S)
G(x,y) -
hl+h2 ¢ Ga(x,y ) +
where G s ( x , y ) = exp[-X m a x ( x , y ) ] continuous part is given by (6) Proof:
Gs(x,y ) and as above 6 = ~l + X2 + h12 and t h e
absolutely
6 ~12 exp[-6 max(x,y)] Ga(X,Y) - hl+h 2 exp[-~ix-~2Y-hl2 max(x,y)] - hl+h 2
.
The decomposition G(x,y) = aGa(x,y ) + (l-a)Gs(x,y) and (F) yields ~2G(x,y ) -gx--~y
lh2(Xl+hl2)G(x,y) = ~fa (x'y) = ~Xl(h2+kl2)G(x,y )
if x > y if x < y ,
where fa(X,y) is the corresponding two dimensional density and the integration of afa(X,y) yields the value of a as given in (S). Observe that the event X = Y has, in this case, a positive probability, while the two-dimensional Lebesgue measure of the line x = y is 0.
The presence of the
singular component is in a sense a "natural phenomenon" as the following theorem shows. Theorem 5.2.4.
The assumption of absolute continuity in addition to assump-
tions of the lack of memory property and exponential marginals yield a unique bivariate distribution with independent exponential marginals (identical to the one obtained as a solution of equation (I) above). Proof: form
(zc)
Observe that the basic equation (2a) can be rewritten in an equivalent
G(x,y)
=
le -6YGl(x,y)
if x ~ Y ~ 0
[e-~XG2(Y-X)
if y > x ~ 0
108
for some 6 > 0, where Gl(X) = G(x,0), G2(Y) = G(0,y) and G(0,0) = I. ting (Sl,S2)
Indeed, set-
= (s,s) into (2a) we have
g(s+t,s+t)
= G(s,s)G(t,t);
hence as above we have G(s,s) = exp(-6s)
for some 6 > 0.
Now setting s 2 = 0 into (2a) we obtain G(Sl+t,t ) = G(Sl,0)G(t,t ) = G(Sl,0)exp(-6t ) • Noting that t >_ 0
we have the first part of the statement
(2b).
The second Dart
follows analogously.
D
Moreover, we also have the following result. Theorem 5.2.5.
If the marginals are assumed to have absolutely continuous
densities then G(x,y) given by (2b) is absolutely continuous if
(7)
6 = fl(0) + f2(0)
where fj (-), j = 1,2, are the corresponding Proof:
densities.
Recall that G(x,y) = ~Ga(X,y)
+ (l-@Gs(x,y)
, 0 -< ~ -< 1
where Ga(.) and Gs(.) are as above, the absolute continuous and singular components of G(-) respectively.
(The singular component is located in the line x = y so it
can be obtained from ( I - @ G (x,x) = G(x,x) - uG (x,x).) From (2b) 2 s a a G(x,y) second derivative -Sx~y = afa(X,y) we have, in partzcular, 1
f afa(X,y)dx x_>y
dy = 1 - ~ f l ( 0 )
/ afa(X,y)dx x_
dy = 1 - ~1 f2 (0)
and s i n c e F a ( % ~ ) co
= f 0
taking the
= 1, we o b t a i n
oo
f ~ f a ( X , y ) d x dy = 2 - 1 [ f l ( 0 ) + f z ( 0 ) 0
] ;
hence if G(.) is absolutely continuous and thus a = i, (7) follows.
E]
Now we are ready to prove the basic assertion due to Block and Basu (1974). Theorem 5.2.6.
If (X,Y) is a positive bivariate random vector which is abso-
lutely continuous and possesses the DIP property,
then the exponentionality
marginals implies that both X and Y are independent. Proof:
Let the marginals be
fl(x)
= a 1 exp(-alx )
and
of the
109
f2 = (y) = a2 exp(-a2Y) where a I and a 2 > 0.
for x,y > 0 ,
From the above we obtain
fl(0) + f2(O) = a I + a 2 = and hence from (2b) /exp (- (al+a2)Y) exp (-a I (x-y)) g (x, y) = [exp (- (al+a2) x) exp (-a 2 (y-x))
ifx~y~O ify~x~0.
In other words G(x,y) = exp(-alx)exp(-a2Y )
for x,y ~ 0
which shows that X and Y are independent. To obtain yet another characterization of a bivariate Narshall-Olkin exponential distribution we shall observe that for this distribution as given by (3), we have P(min(X,~
> s) = G(s,s) = e -~s , s > 0 ,
where, as above, ~ = h I + X2 + XI2 " In other words, a part of the univariate characterizing property (P4~)--for any two independent exponential random variables the minimum is distributed exponentially--continues to hold for this particular bivariate extension. In fact we have Theorem 5.2.7.
(XI,X 2) is a bivariate M-0 exponential distribution if and
only if there exist exponential random variables UI, U 2 and UI2 such that X 1 = min(Ui,U12 ) and X 2 = min(U2,Ul2 ) . (Note, however, that the property that min(Xi,X2) is exponentially distributed does not generally imply that (X1,X2) has a bivariate M-0 exponential distribution as the following example indicates:
_Xl_X2+2(i/x I + i/x2)-I
Let (XI,X2) satisfy P(XI~XI,X2~x2) = e In this case P(min(Xi,X2)~x ) = e Proof:
-X
, x i ~ 0, i=1,2 .
, x ~ 0) .
Although a direct proof of this assertion is straightforward, it is instruc-
tive to couch it in the language of the fatal shock model which serves as a basis for this distribution. Consider a system consisting of two dependent components.
Shocks of three
types are present in the environment and arrive independently and at random (i.e. three independent Poisson processes govern the occurrence of shocks).
A shock of
the first type destroys component I; it has an expected rate of arrival XI' i.e. it arrives at random time U 1 where P(UI>t ) = e -X1t , t _> 0. type two, with expected rate of arrival h 2 P(U2>t) = e -h2t, t _> 0,
Similarly, a shock of
arriving at random time U 2 where
destroys the second component.
Noreover, both components
fail if they are subject to a shock of type three with expected rate hl 2 arriving
110
-Xl2t at random time UI2, where P(Ul2>t) = e
, t ~ 0.
Therefore the (random) life-
length T i of component i = 1,2, satisfies X i = min(Ui,Ul2 )
i = 1,2,
and the joint survival function: G(tl,t2) = P(X i > t i, i=l,2) P[no type 1 arrivals at time tl] × P[no type 2 arrivals at time t2] x
(8)
P[no type 3 arrivals at time max(tl,t2)] -Xltl-~2t2-~12max(tl,t2 ) e
,
(for t I > 0, t 2 ~ 0)
which is identical to (3). From this model it is obvious that if X 1 and X 2 have survival distribution (3) then there exist independent exponential variables UI,U 2 and UI2 satisfying X 1 = min(Ui,Ul2 ) and X 2 = min(U2,Ul2) and conversely. (Note:
D
This representation also yields an alternative proof of the decomposi-
tion G(x,y) as given by (S) into an absolutely continuous and singular component (cf. eqs. (5) and (6)); the singular component is P[Xl>t I, X2~t2[UI2 ~ min(Ui,U2) ] , and the absolute continuous one is the conditional distribution P[Xl>t I, X2>t2JU12 > min(Ui,U2) ] • Finally, we observe that recently Block (1977) succeeded in providing a new characterization of the M-0 bivariate exponential distribution.
This characteriza-
tion is of the type developed by Ferguson (1964) for the univariate exponential distribution (see Chapter 3). Block's characterization of the M-0 bivariate exponential distribution states: Theorem 5.2.8.
A bivariate random variable (X,Y) has the survival function
given by (3) iff:
Proof:
a)
(X,Y) has exponential marginals
b)
U ~ min(X,Y) is exponential
c)
U ~ min(X,Y) is independent of V ~ X-Y . To prove this result it is sufficient to note that an equivalent statement
would be:
for a nonnegative bivariate random variable (X,Y) with absolutely con-
tinuous marginals and U and V as defined above, the D ~ (i) (ii)
U and V are independent U is exponential with mean 5 -1 for some ~.
holds iff:
111
(Indeed, recall that the M-0 bivariate exponential distribution is the only bivariate distribution with exponential m~ginals satisfying
The if part of the
(2a).)
last statement follows from direct computations of P([J= min(X,Y)
~ s, V = X-Y ~ t)
recalling that the iMP implies the existence of ~ > 0 such that P(U = min(X,Y) =
exp(-@s),
The only if part is, however, more illuminating.
s > 0.
> s)
Indeed,
assuming that U and V are independent and that U is exponential we arrive at: (9)
Y G(x,y) = P(X>x, Y>y) = P(U>y) + f P(V~u-y)dP(U
Y = exp(-@y) + f P[V~u-y]
• @ exp(-@u)du
,
for 0 ~ x ~ y .
X
Consequently,
for a fixed y > 0 and 0 ~ x ~ y such that P(V~x-y)
is continuous we
have ~--~ G(x,y) = -P(V~x-y)@ exp(-@x) Equivalently,
(10) Ss 1
.
recalling that G(t,t) = P(U>t)
,
G(Sl+t, s2+t) = G(t,t)-~S~l G(Sl,S2)
is valid for all s I (except for at most a countable number of them) such that 0 ~ s I ~ s2, 0 ~ t, with some s 2 > 0 and t ~ 0 fixed.
Finally, integrating
with respect to s I over 0 ~ s I ~ s 2 we obtain that equation range 0 ~ s I ~ s 2, 0 ~ t.
Similar computations verify the validity of (2a) for
0 ~ s 2 ~ s I and t ~ 0 .
D
Observe that the assumption that U ~ min(X,Y) the above characterization.
is exponential
This is because a nonexponential
bution of U implies that G(t,t) = P(U>t) is nonexponential, dicts the iMP.
(i0)
(2a) is valid on the
is essential for
choice for the distri-
which in general contra-
Compare also with the example presented following the statement of
Theorem 5.2.7. A bivariate Weibull distribution was introduced briefly by Marshall and Olkin (1967) and was studied in detail by ~ e s c h b e r g e r
(1974).
The distribution is of
the form
i/a I (11)
(X~, X½) =(X 1
1/a 2 , X2
)
where a i > O, i = 1,2, and (X1,X2) has the M-O b i v a r i a t e exponential d i s t r i b u t i o n . The only characterization the extension of Theorem 5.2.7 Theorem 5.2.Ya.
theorem for this distribution available so far is which is valid provided a i = a (i = 1,2).
(X~, X~) is a bivariate Weibull distribution given by
112
(12)
G(Xl,X 2) : exp{-)~lX ~ - 12x2 - X12max(xl,x2)}
, x i -> O,
~ > 0 ,
if and only if there exist independent Weibull random variables U~, I;~ and U~2 such that X{ = min(U{, UI2 ) and X~ = min(U½, UI2 ). The proof of the direct part is a straightforward verification that if U{, U½, U{2 are independent Weibull random variables with equal shape oarameters and X{ = min(U{,U{2 ) and X½ = min(U½,U{2), then (XI,X½) is bivariate Weibull.
The reverse re-
lationship follows from the definition of the distribution via (ll) and its validity for the M-O exponential bivariate distribution, or employing a "contrapositive" argument as suggested by Moeschberger (1974).
5.3.
The M u l t i v a r i a t e
Marshall-Olkin exponential distribution
and i t s extensions
The bivariate Marshall-Olkin exponential distribution can be generalized to n dimensions but the notation becomes rather formidable.
First, however, we note that
the extension of property (2) to n dimensional (the strong lack of memory property) will lead us, in this case, to a multivariate exponential distribution which is a product of its (exponential) marginals, thus forming a system consisting of n independent components, with the survival function of the form n G(tl,t 2 ..... tn) = exp{- [ liti } , t i > 0; h i > 0, i=l
i = 1 .....n .
On the other hand, Theorem 5.3.].
The extension of the "no deterioration" property as given by
(2a) coupled with the requirement that the marginals are "exponentials" yields an n-dimensional Marshall-Olkin exponential distribution with the joint survival distribution (13) G(t I .... ,tn) = exp[- ~ Xit i - i!jkij max(ti,tj) ~ijkmax(ti,tj,tk)--..-Xl2...nmaX(tl ..... tn)] , i<j
It is sufficient to show that the n-dimensional distribution given by (13)
is the only distribution with (n-l)-dimensional marginals of the same form for which the joint survival distribution of a set of n components each of age t is the same as that of a set of new components.
Now this property can be written as:
P[Xl>Sl+t .... ,Xn>Sn+tiXl>t .... ,Xn>t] = P[Xl>Sl,...,Xn>Sn ] or G(Sl+t,...,Sn+t) = G(s I .... ,Sn)G(t,...,t);
t a 0, V s i ~ 0 .
As in the bivariate case, setting Sl=S2=...=Sn=S , we have G(s,...,s) = e -6s for some > 0.
Next setting sn = 0 we have
113
G(Sl+t,...,Sn_l+t,t) = G(Sl,...,Sn_l,0)G(t,...,t ) = e-~tGn_l(Sl,...,Sn_l ) , where Gn_l(-) is an (n-l)-dimensional marginal and thus, by assumption, is a Marshal~ Olkin distribution given by (13) with n replaced by n-l.
Using this assumption we
obtain that (13) is valid for all s i ~ 0 such that sn ~ s i, i = I, ....n-I (since t ~ 0).
However, by symmetry (13) holds for all s i ~ 0, i = l,...,n.
D
A number of characterizations of the distribution (13) are known, since most of the bivariate characterizations discussed above are easily extendable to the general case.
For example, Theorem 5.3.2.
A random vector (XI,...,Xn) has the (n-dimensional)
Marshall
and Olkin exponential distribution if and only if there exists a collection Hj, J~J (where J is the class of nonempty subsets of (l,2,...,n}) of independent exponential random variables such that ~re
X i = min(Hj, J~J, i~J), i = l,...,n.
explicitly, denote by V the set of vectors ~ = (Vl,...,v n) where each v i
is 0 or 1 but (vI ..... Vn) ~ (0,...,0).
Equation (13) can then be rewritten as
(13a) G(t I ..... tn) = exp[- ~ ~Vl ..... v nmax(tlVl .... 'tnVn)]' V
ti ~ 0 .
The characterization of (13a) in terms of minima asserts the existence of 2n-I independent exvonential random variables Z , E~V such that X i = m~n{ZvlXi = i}. Proof:
The claim of the theorem is an immediate extension of the corresponding two-
dimensional property discussed in the previous section and has especially intuitive verification if we define the multivariate Marshall and Olkin exponential distribution via the fatal shock model.
In other words we postulate independent Poisson
processes Zi(t) with rates Xi which govern the occurrence of fatal shocks to comvonent i, for i = 1,2,...,n, Poisson processes Zij(t) with rates ~ij governing the occurrence of fatal shocks to components i and j simultaneously for 1 ~ i ~ j ~ n,..., and the Poisson process Zl2...n(t) with the rate Xl2...n
which governs the occur-
rence of fatal shocks to all n components simultaneously.
The joint survival dis-
tribution for this model is given by (13). Using this model the existence of independent exponential random variables Zv, %cV satisfying X i = min{ZvlV i = I} follows V
in exactly the same manner as in the two-dimensional case. Note:
Observe also that
X 1 = min{Zv: Vl=l} = min[min{Zv: Vl=l,v2=0}, min{Zv: Vl=l,v2=l}] . This characterizing property has important imvlications in reliability theory in view of the concept introduced by Esary and Marshall (1974).
These authors de-
fined a set of nonnegative random variables (YI' ....Yn) to have a joint distribution with exponential minima if
114
-@jt P[min Yi>t] = e ieJ
,
t ~ 0
for some @j > 0 and all nonempty sets J~{l,...,n).
Clearly, the Marshall and Olkin
distribution is a joint distribution with exponential minima.
Moreover, a rather
straightforward argument shows that Theorem 5.3.3.
The joint distribution of variables (YI,...,Yn) with exponen-
tial minima can be marginally equated in minima to a }~rshall-Olkin distribution of variables (XI,...,Xn) in the sense that P[min Yi>t] = P[min Xi>t ] , Vt ~ 0 , ieJ icJ for each none~gty set Jc{l,2,...,n}. Details are given in Esary and Marshall (1974). The characterization of the Harshall-Olkin multivariate distribution as a unique n-dimensional distribution satisfying the lack of memory property for k = n with all (n-l)-dimensional marginals being Harshall-Olkin distributions, can be rephrased by saying that the lack of memory property holds for any k-dimensional marginal for k = 1,2 .... ,n. variate marginals.)
(The D4P for k = 1 yields the exDonentionality of the uni-
The assmnption of the lack of m e m o 7 property for k = 2,3,...,n
implies that the minimum of two or more of the random variables XI,X2,...,Xn with survival function G(Xl,...,Xn) is exponential.
Therefore for any Jc{l,2,...,n}
con-
taining more than one element we have a @3. > 0 such that P[min Xi>t ] = exp[-@jt] and ieJ the lack of memory property implies that G(x I ..... x n) = exp([email protected])G(Xl-Xil,...,0,...,Xn-Xil) for 0 ~ Xil s Xl~...s x n
where the 0 appears in the iI position.
Successively applying this argument we obtain: G (xI ..... Xn) = exp (-912...nxil )exp (912.. .in (Xiz-xil)) ...
...exp(-@ i-n-lln . (x.in_ 1-Xin)) Gin(X~ . .n -Xin_ . I) for 0 = x. -< x. < ...< x. , where G. is the survival function of the i -th univari0 iI in in n iate marginal. (Note that so far we don't require the LMP for k = I.) More compactly, n-i (14) G(x) = exp(-k=[I GJk(Xik-Xik- 1))G'in(X.ln-X.in_l) for 0 = x. ~ x. ~ ...~ x. . 10 11 in Moreover, since G(-) is a survival function, it can be easily verified, taking monotonicity into account that
115
@Jn-I s @Jn_2~...~@jl for all permutations {il,i2,...,i n} of {l,2,...n}, where Jl = {l,2,...,n} and Jk = {ik' ik+l'""in}
for k = 2,...,n.
It can also be shown, extending the corres-
ponding argt~aent presented by Marshall and Olkin (1967a) for the bivariate case (see, e.g. Block (1974) for a more detailed statement) that the requirement of absolute continuity of G(.) implies the equality • = @. + @. +...+ @. @in_lin in_ k in-k+ 1 in
for k = 1,2,...,n-i .
Hence assuming the existence of exponential densities for the marginals (which is a consequence of the lack of memory property for k = I) we arrive from (14) at the following expression for an absolute continuous G(-) satisfying the LMP for k = 2,...,n with exponential marginals: n
n-i
G(x) = exp(- k[l@Jk(Xik-Xik_l)) n
= exp(- k[O(@Jk-@Jk+l)Xik) n
= exp(- k[1OikXik ) = e x p ( - k=l [ okxk) ' i.e.
G(') i s a p r o d u c t o f i t s e x p o n e n t i a l m a r g i n a l s . Theorem 5 . 3 . 4 .
In o t h e r words, we have proved
The a b s o l u t e c o n t i n u i t y o f t h e j o i n t d i s t r i b u t i o n
coupled w